From 634b7097313a0450f23ce1f7ad19e5a722da78b7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E9=B9=A4=E7=94=B7?= Date: Thu, 23 Apr 2026 16:52:27 +0800 Subject: [PATCH 001/382] [TE] EFA SRD shared-endpoint refactor: drop per-peer fid_ep model (#1944) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(efa): idempotent warmupSegment for repeated calls Adds EfaContext::peekEndpoint() — a non-creating, normalized-key lookup — and uses it in EfaTransport::warmupSegment() to short-circuit when every (local_ctx, peer_nic) pair is already connected. Repeated warmup calls on the same segment now return immediately instead of firing a 256-thread std::async fan-out each time. Verified on p6-B300 (16 NICs × 16 peer NICs = 256 endpoints): iter 0: 7.95s (full handshake) iter 1: 0.000s (short-circuit hit) iter 2: 0.000s (short-circuit hit) Co-Authored-By: Claude Opus 4.7 * feat(efa): evict-on-ENOMEM + drop-on-failure for endpoint store Recover from QP exhaustion (fi_enable returns -FI_ENOMEM at the device cap, 768/device on p6-B300) and prevent dead endpoints from piling up under callers that drift endpoint keys (e.g. keys carrying a timestamp). Three places: * EfaContext::endpoint (slow path): on construct failure, evict stale endpoints once and retry. Eviction only scans when we actually ran out of room, so normal traffic pays nothing. * EfaTransport::warmupSegment: if setupConnectionsByActive fails, drop the endpoint via deleteEndpoint(normalizedKey) instead of leaving a dead fid_ep holding its QP slot. * EfaContext::submitPostSend: if submitPostSend returns non-zero and the endpoint is no longer connected, drop it for the same reason. Reproduced on B300: without the fix, injecting ":drift" into the peer_nic_path exhausts fi_enable at iter 48 (768/16 peer NICs). With the fix, 325 iterations complete with RSS bounded at ~6 GB. Co-Authored-By: Claude Opus 4.7 * feat(efa): SRD shared-endpoint refactor — drop per-peer fid_ep model Under SRD (FI_EP_RDM), a single fid_ep per local NIC can address any number of peers via fi_av_insert. The old model created 16 per-peer endpoints per connection (one per local NIC), consuming 16 QPs each — hitting the 768 QP cap after only 48 peers and forcing an LRU + evict-on-ENOMEM dance. This change: - EfaContext owns a single shared fid_ep + peer address vector (`peer_map_`) - EfaEndPoint becomes a thin per-peer fi_addr_t holder (no fi_endpoint, no fi_enable) — handshake is RPC + fi_av_insert only - submitPostSend delegates to EfaContext::submitSlicesOnPeer, using the peer's AV index against the shared endpoint - setPeerNicPath detaches stale AV slot on peer reconnect so drift scenarios re-handshake cleanly - Deletes EfaEndpointStore + LRU/ENOMEM eviction machinery — no longer needed - warmupSegment short-circuit updated for the shared-endpoint flow P5EN drift stress (1 target ↔ 1 initiator, customer shape 328 GB, 16 NIC, 180s target lifetime): - iter 0 (fresh peer) first_batch: 33.6 ms (was ~8.95 s → 266× faster) - iter 1–3 (drift) first_batch: 260–340 ms (was ~8.95 s → ~26× faster) - steady: ~215–220 GB/s, 0 failures across 204 M completed ops - QP growth per added peer: 0 (was 16) — no more 48-peer cliff See mooncake-transfer-engine/example/efa_srd_refactor_validation/ for raw CSV and target-teardown timings. Co-Authored-By: Claude Opus 4.7 * feat(efa): expose warmup_efa_segment in Python binding The C API (warmupEfaSegment) and Rust FFI (warmup_efa_segment) already pre-connect every (local_ctx, peer_nic) pair so the first submitTransfer does not stall on handshake RPC + fi_av_insert. The pybind TransferEngine did not expose this, so Python-driven integrations (vLLM / SGLang) could not benefit from the first-call latency improvement from the SRD shared- endpoint refactor (iter 0 first_batch 33.6 ms vs 8.95 s baseline). Mirror the C API shape: thin pass-through that dynamic_casts the installed "efa" transport and calls EfaTransport::warmupSegment. No-op on non-EFA builds or when the EFA transport is not installed. Tested: built with USE_EFA=ON on p5en, imported engine.TransferEngine, and confirmed `warmup_efa_segment(segment_name: str) -> int` is bound. Co-Authored-By: Claude Opus 4.7 * chore(efa): apply code_format.sh (clang-format-20) Co-Authored-By: Claude Opus 4.7 * chore(efa): drop internal drift_stress/validation dirs + stale gflags doc note - Remove mooncake-transfer-engine/example/drift_stress/ — depends on the out-of-tree customer_pattern Rust binary and isn't usable by upstream contributors. - Remove mooncake-transfer-engine/example/efa_srd_refactor_validation/ — validation artifacts for the SRD refactor; the headline numbers live in the PR description instead. - Remove the libgflags-dev note from docs/.../efa_transport.md — it is pulled in transitively by dependencies.sh and no longer needs a manual install step. Co-Authored-By: Claude Opus 4.7 * fix(efa): close shared endpoint before dropping peer map in teardown Calling fi_av_remove() on peer addresses after the shared fid_ep was still open — or calling it at all during context teardown when the AV itself is about to be fi_av_close()d — trips a libfabric EFA-provider assertion, causing the efa_transport_test unit tests to segfault on engine destruction. Reorder EfaContext::deconstruct() to close shared_ep_ first, then detach every peer handle via a new markDetachedForTeardown() that just resets the AV-slot field without calling fi_av_remove(). fi_av_close() below invalidates every slot in one shot, so no per-slot removal is required on the teardown path. Verified: all 5 EFATransportTest cases (InstallTransport, LoopbackWrite, WriteAndRead, MultiWrite, StressMultipleBatches) now pass on P5EN. Co-Authored-By: Claude Opus 4.7 * test(efa): add coverage for warmupSegment, batch register, large xfer, re-open Five additional EFATransportTest cases, filling gaps in the existing suite (which only exercised install + 4 single-buffer write/read paths): - WarmupSegmentLoopback — covers EfaTransport::warmupSegment() and its idempotent short-circuit on the second call, plus the empty / self-name no-op path. - WarmupSegmentNotFound — warmupSegment must fail cleanly (not hang) for an unknown segment name. - RegisterMemoryBatch — registerLocalMemoryBatch + unregisterLocalMemoryBatch round-trip. - LargeTransfer — 128 MB buffer, 64 x 1 MB slices; exercises WR / CQ pacing in EfaContext::submitSlicesOnPeer well past the 16 x 64 KB MultiWrite ceiling. - RepeatedOpenSegment — openSegment() on the same peer repeatedly must keep transferring correctly; guards the setPeerNicPath detach path that target-restart drift depends on. docs/.../efa_transport.md's Unit Tests table updated to match. Co-Authored-By: Claude Opus 4.7 * docs(efa): restrict CTest invocation to the efa tests Running the full ctest suite on an EFA-only host trips etcd / mooncake_master requirements in the TCP / metadata / store tests, which can read as "EFA broke the build". The EFA doc should not recommend that invocation — filter to the two EFA binaries and call out the reason in a note. Co-Authored-By: Claude Opus 4.7 * docs(efa): drop redundant MC_METADATA_SERVER / MC_LOCAL_SERVER_NAME block The defaults (P2PHANDSHAKE and 127.0.0.1:12345) are what the loopback unit tests actually want, and nothing in the current test suite needs an override, so documenting these "env var exports" next to the test invocation just makes readers wonder whether they are mandatory. Co-Authored-By: Claude Opus 4.7 * docs(efa): drop obsolete CUDA_VISIBLE_DEVICES tip for CPU-to-CPU runs The tip existed because libfabric's EFA provider used to dlopen libcudart at fi_getinfo time, creating a CUDA context even for CPU-only benchmarks. EfaContext::construct now sets FI_HMEM=system when the build is not GPU-enabled, which already prevents that context from being created — the CUDA_VISIBLE_DEVICES="" workaround is no longer needed. Co-Authored-By: Claude Opus 4.7 * docs(efa): lead with a bench command that hits line rate The previous "Initiator Node" snippet used block_size=65536 / threads=8 / buf=1GB, which reaches only ~50 GB/s on an 8x400G host and reads as "EFA is broken" to anyone copy-pasting it. Replace with block=1MB / threads=32 / buf=4GB — the same values that produce the GB/s numbers quoted in the Benchmark Results tables — and add a short sentence telling readers exactly why block_size matters. Also clarify that the MC_SLICE_SIZE no-effect note covers both CPU-to-CPU and GPU-to-GPU paths (verified: no EFA code reads globalConfig().slice_size, only rdma_transport and kunpeng_transport do). Co-Authored-By: Claude Opus 4.7 * docs(efa): drop MC_SLICE_SIZE references, sync target --buffer_size EFA transport never reads globalConfig().slice_size — the only readers are rdma_transport and kunpeng_transport — so the env-var table row, the "MC_SLICE_SIZE has no effect" tuning-tip bullet, and the pre-optimization tuning-history details block were all documentation debt rather than user guidance. Drop all three. Also: the target command defaulted to --buffer_size=1GB (the flag's default), but the recommended initiator command now uses 4GB. Since the target allocates and registers exactly FLAGS_buffer_size bytes and the initiator writes into offsets [0, buffer_size), they have to match. Add --buffer_size=4294967296 to the target snippet and a short line calling out the constraint. Co-Authored-By: Claude Opus 4.7 * fix(efa): skip preTouchMemory for GPU buffers, clarify target flags preTouchMemory() does a CPU-side store per page, which was gated only by "chunk >= 4 GiB". That trips a segfault the moment a user registers a >=4 GiB cudaMalloc buffer — exactly what the recommended benchmark config does. Restrict the pre-touch to host (cpu:*) memory so VRAM registration stays a pure libfabric call. Also amend the benchmark docs: - call out that --use_vram / --gpu_id apply to the target too, and that GPU-to-GPU runs need --gpu_id=-1 on both sides - mention --use_vram=false as the alternative to a -DUSE_CUDA=OFF build for CPU-to-CPU runs Co-Authored-By: Claude Opus 4.7 * docs(efa): default benchmark snippet to gpu_id=-1 + threads=16 Align the benchmark quick-start with what actually reaches line rate on a p5en.48xlarge pair (≈350 GB/s write, verified). The previous snippet recommended --threads=32 and no --gpu_id override, which: - ran on one GPU (default gpu_id=0), so buffers sat on one NUMA node and half the NICs were cross-NUMA — settled at ≈56 GB/s - ran into the SRD shared-endpoint WR cap (16 × 256 = 4096 WRs vs 32 × 128 = 4096 outstanding slices, zero headroom) and triggered "timed out waiting for CQ drain" Rework the section: lead with the two knobs that matter (--gpu_id=-1 on both sides, --block_size=1048576), add --gpu_id=-1 to the target snippet for symmetry, switch the initiator to --threads=16, and explain in a note why 16 is the ceiling (so readers know where the limit comes from, not just that we picked it). Co-Authored-By: Claude Opus 4.7 * docs(efa): complete the Key Parameters table The table was missing several flags that the bench snippet in the same section actually uses (--mode, --protocol, --metadata_server, --segment_id, --use_vram), and some of the flags it did list had defaults / descriptions that did not match the binary (e.g. default operation is read, not write). Sync every row to DEFINE_* in transfer_engine_bench.cpp and add explicit "what this is for on EFA" hints next to the two flags (--block_size, --gpu_id) that most determine whether the run hits line rate. Co-Authored-By: Claude Opus 4.7 * docs(efa): reorder benchmark sections by peak throughput Lead with p6-b300 (752 GB/s peak), then p5en (347 GB/s), then p6-b200 (313 GB/s). Readers skim the first table to decide whether EFA transport is fast enough for them; putting the best-performing platform first avoids undersell. Co-Authored-By: Claude Opus 4.7 * docs(efa): reorder benchmarks by hardware generation, not raw GB/s Previous reorder put p5en (H200) above p6-b200 (B200) because the p5en rows happen to show a higher headline number, but B200 is a newer generation than H200 and the p6-b200 numbers there are an older snapshot. Fix the ordering to reflect hardware recency: p6-b300 → p6-b200 → p5en. Same change applied to the Cross-Transport Comparison table. Co-Authored-By: Claude Opus 4.7 * docs(efa): refresh p5en bench results (366 GB/s write / 304 GB/s read) Re-swept on P5EN (H200, 16×200G) with the SRD shared-endpoint build and --buffer_size=4GB per GPU. New peaks: - Write: 365.66 GB/s @ threads=16, batch=128 (up from 347 GB/s) - Read: 303.78 GB/s @ threads=16, batch=32 (up from 308 GB/s, but now with a clear optimal config) The new table preserves enough rows to show the two interesting axes — write wants big batches, read wants small ones — and the Cross-Transport Comparison is updated to 366 GB/s (~91% of the 400 GB/s line rate). B200/B300 rows are unchanged (no hardware available to re-sweep). Co-Authored-By: Claude Opus 4.7 * docs(efa): mark B200/B300 bench rows stale, refresh p5en CPU, drop cross-transport table - p5en CPU-to-CPU re-swept on current SRD shared-endpoint build: peak write 213.57 GB/s @ threads=48/batch=32, peak read 212.18 GB/s @ threads=16/batch=32 (vs old 192 GB/s / 182 GB/s). Table shows a few representative points — throughput is essentially flat across the parameter grid because the run is DRAM-bound. - B200 and B300 sections tagged as predating the SRD refactor and the recent tuning work; a re-sweep is pending hardware availability. - Drop the "Cross-Transport Comparison" subsection and the "EFA vs RoCE RDMA" paragraph — both were derived numbers that quickly rot and were already redundant with the per-platform tables above them. Co-Authored-By: Claude Opus 4.7 * docs(efa): add block_size + buffer_size sweep tables as evidence Two sweeps on p5en GPU-to-GPU (SRD shared-endpoint build, --gpu_id=-1) at the peak config (threads=16, batch=128), backing up the tuning claims elsewhere in this file: - block_size: 64 KB default reaches only 26% of peak. Write throughput climbs steeply through 512 KB, plateaus between 1 MB and 2 MB. 2 MB is MC_EFA_STRIPING_THRESHOLD (a different code path). 1 MB is the recommended value — within 4% of the 2 MB peak and safely below the striping boundary. - buffer_size: only needs to clear block × batch × threads (2 GB in the peak config). 2 GB vs 4 GB differs by ~3% on write, read is flat within noise. The example commands use 4 GB purely as a generous default, not because smaller fails. Co-Authored-By: Claude Opus 4.7 * docs(efa): rewrite Tuning Tips around the SRD shared-endpoint model Two stale claims under Tuning Tips were written for the old per-peer endpoint code: - "Increase threads to 32-48": under the shared endpoint each NIC has a 256 WR cap, and threads × batch past 16×256 = 4096 trips the CQ-drain timeout. The sweep on p5en shows 16 threads is the ceiling, not the floor. - "Write peaks at threads=32, read at threads=16 / buffer=2GB max per GPU": both taken from the pre-refactor B300 snapshot. Current p5en peaks are both at threads=16; the 2GB/GPU limit was the VRAM preTouch segfault that is now fixed. Rewrite the bullets to describe what the SRD shared-endpoint actually constrains: block_size sweet spot, threads × batch WR cap, the write-vs-read batch-size split, and --gpu_id=-1 on both sides. Add an explicit line that buffer_size only needs to clear block × batch × threads (pointing readers at the sweep table). Co-Authored-By: Claude Opus 4.7 * refactor(efa): remove broken per-request striping across NICs submitTransferTask had a branch that, when request.length exceeded MC_EFA_STRIPING_THRESHOLD (default 2 MB), split the request into num_nics slices and dispatched one per NIC. The intent was to speed up large single transfers by paralleling across all NICs. A sweep on p5en (SRD shared-endpoint build, 16 NICs x 200G) shows the branch is a ~20x negative optimization in every bench config: block=1 MB, threads=16, batch=128: ON 16 GB/s vs OFF 366 GB/s block=8 MB, threads=16, batch=16: ON 18 GB/s vs OFF 355 GB/s block=32 MB, threads=16, batch=4: ON 18 GB/s vs OFF 328 GB/s Even the designed-for scenario (threads=1, batch=1, single huge request) only wins by 1.2x (27 vs 23 GB/s) — far below the 16x expected from parallel dispatch, because the per-slice post_lock / peer lookup still serialize the dispatch. And that "win" is only reachable with threads=1 — any realistic caller with multiple threads can parallelize on its own without this mechanism. Since the branch silently degrades every common bench / workload configuration above the 2 MB threshold and delivers no real win even in its best case, delete it: - efa_transport.cpp: drop the LARGE TRANSFER if-branch and the kStripingThreshold read. - config.h / config.cpp: drop the efa_striping_threshold field and the MC_EFA_STRIPING_THRESHOLD env var. - docs/.../efa_transport.md: drop the Environment Variable row, the "Note on EFA slicing" striping explanation, and the "striping off" callouts in the benchmark tables and tuning tips. Verified: all 10 EFATransportTest cases pass, and the peak p5en bench run is unchanged (352.81 GB/s write @ threads=16, batch=128). Co-Authored-By: Claude Opus 4.7 * docs(efa): number the sequential subsections for easier skimming Four groups of level-3 / level-4 headings in this doc read as step-by-step instructions (do A first, then B), but only two of them were numbered while the others relied on order alone. Result: readers had to count positions to tell "how far through" they were. Number every such group consistently: - Performance Benchmark: Target Node (1) / Initiator Node (2) - Usage with vLLM: Prefill Instance (1) / Decode Instance (2) - Benchmark Results: p6-b300 (1) / p6-b200 (2) / p5en (3) Reference-style sections (Key Parameters, Tuning Tips, Warmup, Technical Details, Troubleshooting) stay unnumbered — those aren't ordered and numbering them would imply a reading path that doesn't exist. Co-Authored-By: Claude Opus 4.7 * test(efa): add efa_first_submit_probe for warmup / first-submit timing A standalone cross-node probe that measures, separately: - warmupSegment() latency, and - per-submit latency for the first N single-transfer submits so we can isolate the handshake / av_insert cost (warmup=OFF makes it land on submit #0) and compare pre-refactor vs post-refactor code without having to squint at 10s throughput averages. Not enabled by default ctest — only built when USE_EFA=ON. Meant for manual two-host runs alongside transfer_engine_bench. Co-Authored-By: Claude Opus 4.7 * docs(efa): replace warmup numbers with a cross-branch p5en measurement The old "Eager endpoint warmup" numbers (4 s stall / 13.5 ms warmed) were from a B300 snapshot that predated the SRD shared-endpoint refactor, so they described the OLD code. Replace with a fresh cross-node measurement on p5en that compares both branches head to head using efa_first_submit_probe: - Warmup itself: 1.1 s (SRD shared endpoint) vs 17 s (per-peer fid_ep on upstream main) — ~15x faster and much more stable (SRD runs were all 1.13-1.14 s; old code swung 9-17 s across three reps). - Cold first submit with NO warmup: 26 ms (SRD) vs 99 ms (old) — ~4x faster, because the shared endpoint removes the per-peer fi_endpoint / fi_enable step that used to dominate the first send. The prior text's "first-batch stall" framing also reads as if the whole first submitTransfer always pays the full handshake cost. That's not quite how it works — the initiator round-robins across local NICs, so only a handful of pairs warm on each call, and the stall amortizes across the first several submits. Fix the wording. Add a pointer to the probe source (example/efa_first_submit_probe.cpp) so the measurement is reproducible. Co-Authored-By: Claude Opus 4.7 * docs(efa): reframe "first-request latency" around two audiences The previous version led with a long explanation of warmupSegment and its internals, which made it sound like callers had to opt in to get any first-request speedup from this PR. That isn't the case: the SRD shared-endpoint refactor cuts the cold submit from 99 ms to 26 ms automatically, and vLLM / SGLang (which don't call warmupSegment) see that 4x win without any integration work on their side. The explicit warmupSegment API is an additional 15x improvement on top of that, but it matters mainly to direct Mooncake callers. Restructure the section accordingly: - Lead with a 2x2 table showing both numbers side by side. - Frame the ~4x cold-submit win as the default, no-code-change benefit everyone gets. - Frame warmupSegment as the opt-in path for callers that want sub-10 ms first-request latency. - Explicitly note that vLLM / SGLang currently don't call it. - Keep the peer handshake-daemon bottleneck note so readers understand why the warmup scales linearly. Co-Authored-By: Claude Opus 4.7 * docs(efa): reference #1944 explicitly instead of "this PR" "This PR" dangles once the doc lands on main — it's fine in review context, less fine afterwards. Use the concrete issue number so readers (including future us) can follow the link. Co-Authored-By: Claude Opus 4.7 * docs(efa): unwrap soft-wrapped paragraphs, let the renderer flow them Several blockquote notes, tuning-tip bullets, and the "first-request latency" section used ~70-column soft wraps, which look fine rendered but make the raw markdown look ragged — the line after 'Peer addressing resolves lazily:' used to stop before the line was actually full, which reads as a formatting bug in plain text. Let each logical paragraph / bullet be a single line and defer wrapping to the browser / sphinx renderer. No rendered change. Also update the PR body on #1944 the same way. Co-Authored-By: Claude Opus 4.7 * docs(efa): fix post-refactor stale references (architecture, thread safety, comparison) Four places still described the pre-#1944 code: 1. docs/.../efa_transport.md "EFA Transport Architecture" diagram — showed fid_ep inside EfaEndpoint, which is wrong under the shared-endpoint model. fid_ep lives on EfaContext (shared_ep_); EfaEndPoint only holds the peer's fi_addr_t AV slot. Redraw. 2. docs/.../efa_transport.md "Thread Safety" — said the transport "adds per-endpoint spinlocks" (plural, per-endpoint). In the new code there's exactly one post_lock_ per EfaContext guarding the shared fid_ep. Reword. 3. docs/.../efa_transport.md "EFA vs RoCE RDMA" comparison rows — used the pre-refresh p5en GPU number (347 GB/s) and the stale B200 row (313 GB/s). Replace with the current p5en peak from the benchmark tables above (365 GB/s GPU, 213 GB/s CPU), drop the row pointing at a now-absent B200 re-sweep. 4. efa_context.cpp:589 — "Fast path: peer info pre-resolved by submitTransferTask's striping path". The striping path was removed in an earlier commit in this PR; the fast path now just catches slices whose peer_nic_path the caller already filled in. Reword without referencing the removed mechanism. No behavioral change. Co-Authored-By: Claude Opus 4.7 * refactor(efa): address pr review — drop unused params, sysconf once, drop friend-class, expand probe comment Addresses trivial review feedback on #1944: - EfaContext::construct() no longer takes num_comp_channels / port / gid_index. These three parameters were plumbed through from the original RDMA-era signature but the EFA/libfabric path never read them; the previous commit just silenced warnings with (void) casts. Remove them from the signature and the one caller in EfaTransport::installSglangTransport. - preTouchMemory() was calling sysconf(_SC_PAGESIZE) twice (once for the >0 check and once for the value). Cache the result in a local. - `friend class EfaEndPoint` in efa_context.h was unnecessary — EfaEndPoint only touches public methods on EfaContext (nicPath, localEpAddr, engine, insertPeerAddr, removePeerAddr, submitSlicesOnPeer). Drop the friend decl. - Expand the header comment on efa_first_submit_probe.cpp to say what problem it exists to measure and why transfer_engine_bench alone isn't sufficient — reviewer asked what the example is for. No behavioral change. Co-Authored-By: Claude Opus 4.7 * perf(efa): faster hex<->binary AV address handling Reviewer pointed out two sources of overhead in the handshake path: 1. localEpAddr() built the hex string via std::ostringstream with std::hex / std::setw / std::setfill, which pulls in stream formatting for every byte. Replace with a direct table-driven encoder writing into a pre-sized std::string. 2. insertPeerAddr() decoded the hex using substr() + strtol() for every byte (a temporary std::string and a libc base-16 parse per pair). Replace with a single pass using a hexNibble() helper and a pre-sized buffer. 3. Loopback called insertPeerAddr(localEpAddr()), encoding the bytes to hex and then immediately decoding back. Expose the raw bytes via localEpAddrBytes() and add an insertPeerAddrBytes() overload; loopback now skips the round-trip entirely. Handshake path is not the dominant cost in steady-state serving, but warmupSegment() fires 256 of these on a fresh 16x16 topology and every one of them was doing the extra work. The new path also avoids the per-byte std::string allocation (strtol path). Co-Authored-By: Claude Opus 4.7 * refactor(efa): use std::atomic for wr_depth / cq outstanding, fix O(N^2) erase Replace `volatile int` + __sync_* builtins with std::atomic on the shared-endpoint pacing counters (EfaContext::wr_depth_ and EfaCq::outstanding). `volatile` never implied atomicity under the C++ memory model, and mixing it with __sync builtins was technically UB; the atomic version is both portable and more explicit about the intended memory ordering (acq_rel on the mutating ops, relaxed on the speculative load used for CAS). Also fix an O(N^2) batch consumption pattern in submitSlicesOnPeer: `slice_list.erase(begin, begin + batch_count)` shifted the tail on every iteration. Consume via a moving cursor instead, and splice retry slices in place when -FI_EAGAIN forces a partial post. Behavior is unchanged on the happy path; the retry path now preserves the original slice order at the cursor, matching the previous insert-at- front semantics. Co-Authored-By: Claude Opus 4.7 * docs(efa): drop stale efa_latency_bench.{py,png}, document efa_first_submit_probe The `efa_latency_bench.py` script and its `efa_latency_bench.png` output shipped with the original EFA transport PR (#1509) two years ago. No markdown doc references them, the script hard-codes old private IPs as defaults, and nothing guarantees the chart is still representative. Remove both. Expand the First-request latency section of efa_transport.md to actually document `efa_first_submit_probe`: what it measures (cold-submit + eager-warmup cost, which `transfer_engine_bench`'s 10s average hides), how to run it (target / initiator commands), the expected output layout, all flags, and the two situations where it is worth running (deciding whether your app needs `warmupSegment()`, and comparing PR branches on the same hardware). Marked it explicitly as EFA-specific — RDMA/TCP transports do not have an equivalent warmup concept, so folding this into `transfer_engine_bench` would muddy the generic benchmark for no gain. Co-Authored-By: Claude Opus 4.7 --------- Co-authored-by: Claude Opus 4.7 Co-authored-by: EC2 Default User --- .../design/transfer-engine/efa_transport.md | 370 ++++++---- .../transfer_engine/transfer_engine_py.cpp | 20 + .../transfer_engine/transfer_engine_py.h | 5 + .../example/CMakeLists.txt | 6 + .../example/efa_first_submit_probe.cpp | 199 ++++++ .../example/efa_latency_bench.png | Bin 64738 -> 0 bytes .../example/efa_latency_bench.py | 454 ------------- mooncake-transfer-engine/include/config.h | 1 - .../transport/efa_transport/efa_context.h | 148 ++-- .../transport/efa_transport/efa_endpoint.h | 111 +-- .../transport/efa_transport/efa_transport.h | 15 +- mooncake-transfer-engine/src/config.cpp | 10 - .../transport/efa_transport/efa_context.cpp | 640 ++++++++++++------ .../transport/efa_transport/efa_endpoint.cpp | 393 ++--------- .../transport/efa_transport/efa_transport.cpp | 164 ++--- .../tests/efa_transport_test.cpp | 171 +++++ 16 files changed, 1265 insertions(+), 1442 deletions(-) create mode 100644 mooncake-transfer-engine/example/efa_first_submit_probe.cpp delete mode 100644 mooncake-transfer-engine/example/efa_latency_bench.png delete mode 100755 mooncake-transfer-engine/example/efa_latency_bench.py diff --git a/docs/source/design/transfer-engine/efa_transport.md b/docs/source/design/transfer-engine/efa_transport.md index f1fd0df3..d8c7a23a 100644 --- a/docs/source/design/transfer-engine/efa_transport.md +++ b/docs/source/design/transfer-engine/efa_transport.md @@ -32,13 +32,6 @@ sudo ./dependencies.sh -y This installs all system packages, git submodules (including pybind11 and yalantinglibs), and Go. -**Additional EFA-specific dependencies** (not covered by `dependencies.sh`): - -```bash -# gflags is needed by transfer_engine_bench and EFA unit tests -sudo apt-get install -y libgflags-dev -``` - > **Note:** The EFA driver and libfabric are **not** installed by `dependencies.sh`. They must be pre-installed on the instance (see section 1 above). ## Building Mooncake with EFA Support @@ -119,34 +112,53 @@ The test suite includes: | `WriteAndRead` | Write then read with data integrity check | | `MultiWrite` | Batch write (16 requests) | | `StressMultipleBatches` | Stress test (20 batches x 8 requests) | +| `WarmupSegmentLoopback` | `warmupSegment()` handshake path + idempotent re-call | +| `WarmupSegmentNotFound` | `warmupSegment()` fails cleanly for an unknown segment | +| `RegisterMemoryBatch` | `registerLocalMemoryBatch` / `unregisterLocalMemoryBatch` round-trip | +| `LargeTransfer` | 128 MB buffer, 64 x 1 MB slices — exercises WR / CQ pacing | +| `RepeatedOpenSegment` | `openSegment()` on the same peer repeatedly still transfers correctly | -You can also run all unit tests via CTest: +You can also run the EFA tests via CTest: ```bash -cd build && ctest --output-on-failure +cd build && ctest --output-on-failure -R 'efa' ``` -Environment variables for test configuration: - -```bash -export MC_METADATA_SERVER=P2PHANDSHAKE # default -export MC_LOCAL_SERVER_NAME=127.0.0.1:12345 # default -``` +> **Note:** `ctest --output-on-failure` without a filter runs every test in the build, including TCP / metadata / master-service suites that require an etcd server or a running `mooncake_master`. Those will fail or hang on a machine that is only provisioned for EFA testing — the failures are not EFA-specific. Use `-R 'efa'` to restrict the run to the EFA tests. ## Performance Benchmark Use `transfer_engine_bench` to measure EFA transport throughput between two nodes. -### Target Node (receiver) +The following commands are the GPU-to-GPU configuration that produces +the headline numbers in the [Benchmark Results](#benchmark-results) +tables (≈ 350 GB/s write on a p5en.48xlarge pair, ≈ 302 GB/s on +p6-b200.48xlarge). Two things matter the most: + +- `--gpu_id=-1` on **both** sides — this fans buffers across every GPU, + which in turn lets both NUMA nodes' NICs saturate. Pinning a single + GPU (the default `--gpu_id=0`) halves throughput because half the + NICs end up cross-NUMA. +- `--block_size=1048576` (1MB, not the 64 KB default) — each block + becomes one `fi_write` / `fi_read`, so larger blocks amortize + per-op overhead and are the main knob for hitting line rate. + +### 1. Target Node (receiver) ```bash ./build/mooncake-transfer-engine/example/transfer_engine_bench \ --mode=target \ --protocol=efa \ - --metadata_server=P2PHANDSHAKE + --metadata_server=P2PHANDSHAKE \ + --buffer_size=4294967296 \ + --gpu_id=-1 ``` -### Initiator Node (sender) +`--buffer_size` must be at least as large as the initiator's +`--buffer_size` — the initiator writes into offsets `[0, buffer_size)` +on the target, so keep these in sync. + +### 2. Initiator Node (sender) ```bash ./build/mooncake-transfer-engine/example/transfer_engine_bench \ @@ -156,78 +168,57 @@ Use `transfer_engine_bench` to measure EFA transport throughput between two node --segment_id=: \ --operation=write \ --duration=10 \ - --threads=8 \ - --block_size=65536 \ + --threads=16 \ + --block_size=1048576 \ --batch_size=128 \ - --buffer_size=1073741824 \ + --buffer_size=4294967296 \ + --gpu_id=-1 \ --report_unit=GB ``` -> **Tip:** For CPU-to-CPU benchmarks, prepend `CUDA_VISIBLE_DEVICES=""` to prevent the CUDA runtime from being initialized. Without it, `nvidia-smi` may show GPU memory usage (due to CUDA context initialization) even though the benchmark only uses DRAM. +Replace `:` with the target node's +address shown in the target's startup log (e.g., `ip-172-31-29-226:12345`). -Replace `:` with the target node's address shown in the target's startup log (e.g., `ip-172-31-29-226:12345`). +> **CPU-to-CPU** (no GPUs): build with `-DUSE_CUDA=OFF`, **or** pass `--use_vram=false` to a CUDA-enabled binary. Drop `--gpu_id=-1` in that case — the bench will spread buffers across NUMA nodes instead. + +> **Why `threads=16` and not 32:** the SRD shared endpoint caps outstanding WRs per NIC (default 256 — see `MC_MAX_WR`). With `threads × batch ≤ NICs × max_wr` the CQ never saturates; going higher triggers backoff and times out. 32 threads × 128 batch = 4096 slices chasing 16 × 256 = 4096 WRs has no headroom, so the steady-state config settles at 16 threads. ### Key Parameters | Parameter | Default | Description | |-----------|---------|-------------| -| `--block_size` | 65536 | Bytes per transfer request | +| `--mode` | initiator | `initiator` (sender) or `target` (receiver) | +| `--protocol` | rdma | Transport protocol; use `efa` here | +| `--metadata_server` | `192.168.3.77:2379` | etcd address or `P2PHANDSHAKE` for standalone use | +| `--segment_id` | `192.168.3.76` | Initiator only: `:` from the target's startup log | +| `--operation` | read | `read` or `write` | +| `--block_size` | 65536 | Bytes per transfer request; **1 MB (1048576) is the main knob for EFA throughput** | | `--batch_size` | 128 | Requests per batch | -| `--threads` | 12 | Concurrent submission threads | -| `--buffer_size` | 1 GB | Total buffer size (per GPU when `--gpu_id=-1`) | +| `--threads` | 12 | Concurrent submission threads (initiator) | +| `--buffer_size` | 1 GB | Buffer size (per GPU when `--gpu_id=-1`, otherwise total) | | `--duration` | 10 | Test duration in seconds | -| `--operation` | write | `read` or `write` | -| `--report_unit` | GB | `GB\|GiB\|Gb\|MB\|MiB\|Mb` | -| `--gpu_id` | 0 | GPU device ID; `-1` to use all GPUs (requires `-DUSE_CUDA=ON`) | +| `--report_unit` | GB | `GB\|GiB\|Gb\|MB\|MiB\|Mb\|KB\|KiB\|Kb` | +| `--use_vram` | true | Allocate from GPU VRAM (requires `-DUSE_CUDA=ON`); pass `--use_vram=false` for CPU-to-CPU on a CUDA build | +| `--gpu_id` | 0 | GPU device ID when `--use_vram=true`; `-1` fans buffers across every GPU and is what actually saturates all NICs in a GPU-to-GPU run | +| `--init_mem` | true | Zero-fill the allocated buffer; rarely needs to change | +| `--auto_discovery` | false | Auto-discover topology on init; off for reproducible runs | -| Environment Variable | Default | Description | -|---------------------|---------|-------------| -| `MC_SLICE_SIZE` | 65536 | Slice size for RDMA transport. **Not used by EFA transport** (see note below). | -| `MC_EFA_STRIPING_THRESHOLD` | 2097152 | Transfers larger than this (bytes) are striped across all NICs | - -> **Note on EFA slicing:** Unlike RDMA transport which splits every transfer into fixed `MC_SLICE_SIZE` chunks, EFA transport uses a different strategy: transfers ≤ `MC_EFA_STRIPING_THRESHOLD` (default 2MB) are sent as a **single `fi_write`/`fi_read`** whose size equals `block_size`; transfers larger than the threshold are striped across all NICs (one chunk per NIC). This means **`block_size` directly determines per-operation size** and is the key tuning parameter for EFA, while `MC_SLICE_SIZE` has no effect. +> **Note on EFA slicing:** EFA transport does not split each transfer +> into fixed-size slices the way RDMA transport does — each transfer +> is sent as a single `fi_write` / `fi_read` whose size equals +> `block_size`, round-robin'd across NICs per request. **`block_size` +> is the key tuning parameter** for EFA throughput. > **Note:** `buffer_size` must be >= `block_size * batch_size * threads`. The benchmark auto-adjusts if too small. ### Benchmark Results -#### p6-b200.48xlarge (B200, 8 EFA × 400 Gbps) - -Tested on two p6-b200.48xlarge instances in the same AWS placement group. - -**GPU-to-GPU** (build with `-DUSE_CUDA=ON`, `--gpu_id=-1` for all 8 GPUs): - -| Configuration | Write | Read | -|---------------|-------|------| -| block=1MB, threads=32, batch=64, buf=2GB/GPU | 285-296 GB/s | 312 GB/s | -| **block=1MB, threads=16, batch=128, buf=2GB/GPU** | **302 GB/s** | **313 GB/s** | - -**CPU-to-CPU** (build with `-DUSE_CUDA=OFF`): - -| Configuration | Write | Read | -|---------------|-------|------| -| block=1MB, threads=32, batch=128, buf=4GB | **222 GB/s** (stable over 6 runs) | **226 GB/s** | - -
-CPU Parameter Tuning History (p6-b200) - -Earlier CPU-to-CPU tuning results (before EFA striping optimization, when `MC_SLICE_SIZE` was still used by EFA): - -| block_size | threads | batch_size | MC_SLICE_SIZE | Throughput | -|-----------|---------|------------|---------------|-----------| -| 64KB | 8 | 128 | default (64KB) | 69.47 GB/s | -| 128KB | 32 | 128 | default | 92.33 GB/s | -| 128KB | 32 | 128 | 256KB | 156.18 GB/s | -| 128KB | 48 | 128 | 256KB | 160.34 GB/s | - -> **Note:** These results predate the EFA striping optimization. With the current code, `MC_SLICE_SIZE` no longer affects EFA performance. Use `--block_size=1048576` (1MB) instead, which achieves 222 GB/s. - -
- -#### p6-b300.48xlarge (B300, 16 EFA × 400 Gbps) +#### 1. p6-b300.48xlarge (B300, 16 EFA × 400 Gbps) Tested on two p6-b300.48xlarge instances (Intel Xeon Platinum 8559C, 8× B300, 16 EFA devices) in the same AWS placement group. +> **Note:** numbers below predate the SRD shared-endpoint refactor (#1944) and current EFA tuning work. They are a lower bound for the current code; we will re-sweep and update when the hardware is available again. + **GPU-to-GPU** (build with `-DUSE_CUDA=ON`, `--gpu_id=-1` for all 8 GPUs, `--buffer_size=2147483648`): | Configuration | Write | Read | @@ -244,85 +235,178 @@ Tested on two p6-b300.48xlarge instances (Intel Xeon Platinum 8559C, 8× B300, 1 | Configuration | Write | Read | |---------------|-------|------| | **block=1MB, threads=32, batch=128, buf=4GB** | **230 GB/s** | 180 GB/s | -| block=16MB, threads=32, batch=8, buf=8GB (striping off) | 233 GB/s | - | > CPU-to-CPU is bounded by DRAM bandwidth (~250 GB/s/socket on Xeon 8559C). Per-NIC sampling shows NUMA-0 NICs at 90 Gbps and NUMA-1 NICs at 53 Gbps, confirming DRAM controller saturation rather than NIC limit. -#### p5en.48xlarge (H200, 16 EFA × 200 Gbps) +#### 2. p6-b200.48xlarge (B200, 8 EFA × 400 Gbps) -Tested on two p5en.48xlarge instances (Intel Xeon 8488C, 8× H200 141GB, 16 EFA devices) in the same AWS placement group. +Tested on two p6-b200.48xlarge instances in the same AWS placement group. + +> **Note:** numbers below predate the SRD shared-endpoint refactor (#1944) and +> current EFA tuning work. They are a lower bound for the current +> code; we will re-sweep and update when a B200 pair is available +> again. **GPU-to-GPU** (build with `-DUSE_CUDA=ON`, `--gpu_id=-1` for all 8 GPUs): | Configuration | Write | Read | |---------------|-------|------| -| block=1MB, threads=8, batch=128, buf=1GB/GPU | 236 GB/s | 271 GB/s | -| block=1MB, threads=16, batch=128, buf=2GB/GPU | 271 GB/s | **297-308 GB/s** | -| **block=1MB, threads=32, batch=64, buf=2GB/GPU** | **337-347 GB/s** | 274 GB/s | - -> GPU HBM bandwidth (>3 TB/s) eliminates the memory bottleneck, allowing full EFA utilization. Write and read have different optimal thread counts: write peaks at 32 threads, read peaks at 16 threads. - -> **Note:** EFA memory region registration (fi_mr_reg) for GPU memory segfaults at 4GB+ per GPU. Use `--buffer_size=2147483648` (2GB) as the maximum per-GPU buffer. +| block=1MB, threads=32, batch=64, buf=2GB/GPU | 285-296 GB/s | 312 GB/s | +| **block=1MB, threads=16, batch=128, buf=2GB/GPU** | **302 GB/s** | **313 GB/s** | **CPU-to-CPU** (build with `-DUSE_CUDA=OFF`): | Configuration | Write | Read | |---------------|-------|------| -| Single instance (block=1MB, threads=32, batch=128, buf=4GB) | 179 GB/s | 185 GB/s | -| NUMA-split (block=1MB, 2 instances, 8 NICs each, threads=16, buf=2GB) | **192 GB/s** | **182 GB/s** | +| block=1MB, threads=32, batch=128, buf=4GB | **222 GB/s** (stable over 6 runs) | **226 GB/s** | -> CPU-to-CPU throughput is bottlenecked by DRAM bandwidth (~155 GB/s per NUMA node, measured with STREAM Copy). +#### 3. p5en.48xlarge (H200, 16 EFA × 200 Gbps) -#### Cross-Transport Comparison +Tested on two p5en.48xlarge instances (Intel Xeon 8488C, 8× H200 141GB, 16 EFA devices) in the same AWS placement group. -| Transport | Throughput | Notes | -|-----------|-----------|-------| -| **EFA GPU-to-GPU (B300)** | **752 GB/s** | p6-b300.48xlarge, 16×400G, block=1MB, ~94% line rate | -| **EFA GPU-to-GPU (H200)** | **347 GB/s** | p5en.48xlarge, 16×200G, block=1MB | -| **EFA GPU-to-GPU (B200)** | **313 GB/s** | p6-b200.48xlarge, 8×400G, block=1MB | -| **EFA CPU-to-CPU (B300)** | **230 GB/s** | p6-b300.48xlarge, 16×400G, block=1MB, DRAM-limited | -| **EFA CPU-to-CPU (B200)** | **222 GB/s** | p6-b200.48xlarge, 8×400G, block=1MB, DRAM-limited | -| **EFA CPU-to-CPU (H200)** | **192 GB/s** | p5en.48xlarge, block=1MB, NUMA-split, DRAM-limited | -| EFA (default params) | 69.47 GB/s | Default block=64KB | -| TCP (iperf3 baseline) | 9.5 GB/s | Kernel TCP stack, 8 parallel streams | +**GPU-to-GPU** (build with `-DUSE_CUDA=ON`, `--gpu_id=-1` for all 8 GPUs, `--buffer_size=4294967296`): -**EFA vs RoCE RDMA**: On comparable 8×400 Gbps RoCE networks, Mooncake's RDMA transport achieves ~190 GB/s. Tuned EFA **exceeds** RoCE performance with GPU memory (313-347 GB/s) and on CPU-to-CPU (222 GB/s). +| Configuration | Write | Read | +|---------------|-------|------| +| block=1MB, threads=8, batch=128 | 318.71 GB/s | 277.06 GB/s | +| **block=1MB, threads=16, batch=128** | **365.66 GB/s** | 284.22 GB/s | +| **block=1MB, threads=16, batch=32** | 297.23 GB/s | **303.78 GB/s** | +| block=1MB, threads=32, batch=64 | 357.12 GB/s | 279.61 GB/s | +| block=1MB, threads=32, batch=128 | 364.21 GB/s | 250.90 GB/s | +| block=1MB, threads=48, batch=64 | 363.47 GB/s | 268.43 GB/s | + +> **Peak write: 365 GB/s** at `threads=16, batch=128` — ~91% of the 400 GB/s theoretical line rate (16×200 Gbps). Write saturates on batch size, so `batch=128` outperforms smaller batches as long as `threads × batch ≤ 16 × 256 = 4096` (the shared-endpoint WR cap). **Peak read: 304 GB/s** at `threads=16, batch=32` — reads tolerate smaller in-flight queues, and throughput drops as batch grows. + +**CPU-to-CPU** (build with `-DUSE_CUDA=OFF`, or `--use_vram=false` on a CUDA build, `--buffer_size=4294967296`): + +| Configuration | Write | Read | +|---------------|-------|------| +| block=1MB, threads=8, batch=128 | 210.76 GB/s | 209.93 GB/s | +| block=1MB, threads=16, batch=128 | 212.71 GB/s | 211.21 GB/s | +| **block=1MB, threads=16, batch=32** | 211.67 GB/s | **212.18 GB/s** | +| block=1MB, threads=32, batch=128 | 212.99 GB/s | 210.33 GB/s | +| **block=1MB, threads=48, batch=32** | **213.57 GB/s** | 206.92 GB/s | + +> CPU-to-CPU is DRAM-bound — throughput is essentially flat (~205–214 GB/s) across every thread / batch combination that doesn't hit the WR cap. Peak write 213.57 GB/s, peak read 212.18 GB/s. + +**block_size sweep** (p5en GPU-to-GPU, `threads=16 batch=128 buf=4GB --gpu_id=-1`): + +| block | Write | Read | +|-------|-------|------| +| 64 KB (default) | 97.11 GB/s | 96.87 GB/s | +| 128 KB | 190.82 GB/s | 203.88 GB/s | +| 256 KB | 324.90 GB/s | 296.19 GB/s | +| 512 KB | 357.62 GB/s | 289.47 GB/s | +| **1 MB (recommended)** | **352.59 GB/s** | **301.14 GB/s** | +| 2 MB | 366.87 GB/s | 302.25 GB/s | + +> The 64 KB default only reaches ~26% of peak. Write throughput climbs steeply up to ~512 KB and plateaus between 1 MB and 2 MB; read saturates at ~256 KB. 1 MB is the recommended value — within a few percent of the 2 MB peak with more headroom for `batch_size` under the shared-endpoint WR cap. + +**buffer_size sweep** (p5en GPU-to-GPU, `threads=16 batch=128 block=1MB --gpu_id=-1`): + +| buffer_size | Write | Read | +|-------------|-------|------| +| 2 GB (min: `block × batch × threads`) | 353.51 GB/s | 297.03 GB/s | +| 4 GB | 364.86 GB/s | 293.79 GB/s | + +> `buffer_size` only needs to satisfy `buffer_size ≥ block_size × batch_size × threads` (the bench auto-adjusts if smaller, but silently). Anything larger than that minimum does not change throughput — 2 GB vs 4 GB differs by ~3% on write, read is flat within noise. The example commands use 4 GB because it is safe for any reasonable threads/batch combination without having to recompute the minimum. ### Tuning Tips -- **Use `--block_size=1048576` (1MB)** — this is the most important tuning parameter for EFA. Each `block_size`-sized transfer becomes a single `fi_write`/`fi_read` call, so larger blocks amortize per-operation overhead. 1MB gives ~2× throughput over the 64KB default. -- `MC_SLICE_SIZE` has **no effect** on EFA transport (it only applies to RDMA transport). Use `block_size` instead. -- Increase `--threads` to 32-48 to saturate multiple EFA devices (2-4 threads per device is a good starting point) -- For **CPU-to-CPU**: use `--block_size=1048576` (1MB) with NUMA-split (separate instances per NUMA node) for best results -- For **GPU-to-GPU**: use `--block_size=1048576` (1MB), `--gpu_id=-1` (all GPUs), and `--buffer_size=2147483648` (2GB max per GPU). Write peaks at threads=32, read at threads=16 -- Keep `--batch_size` such that `block_size * batch_size * threads <= buffer_size` -- Allocate buffers on both NUMA nodes for balanced NIC utilization (the bench tool does this by default for CPU mode) -- On 16-NIC instances (p5en), writes are NUMA-sensitive: 8 local-NUMA NICs reach 90 Gbps each, while 8 cross-NUMA NICs only reach ~20 Gbps without NUMA-split +- **Use `--block_size=1048576` (1MB)** — the single most important knob. The 64 KB default reaches only ~26% of peak. 1 MB is within a few percent of the 2 MB plateau while leaving headroom for `batch_size` under the shared-endpoint WR cap. +- **Keep `threads × batch_size ≤ num_nics × max_wr`** — under the SRD shared endpoint each NIC carries one `fid_ep` with a 256 WR cap (`MC_MAX_WR`), giving `16 NICs × 256 = 4096` in-flight slots on a 16-NIC host. Exceeding this trips "timed out waiting for CQ drain". In practice `threads=16, batch=128` is a solid baseline; going higher rarely adds throughput and routinely hits the cap. +- **Write vs read:** write benefits from larger batches (peak at `batch=128`); read prefers smaller in-flight queues (peak at `batch=32` on p5en). +- For **GPU-to-GPU**: pass `--gpu_id=-1` on **both** sides so buffers fan out across every GPU. Pinning a single GPU halves throughput because half the NICs end up cross-NUMA. +- For **CPU-to-CPU**: DRAM bandwidth is the ceiling. NUMA-split (separate initiator/target instances per NUMA node) can help reduce contention when one instance can't saturate both nodes. +- `--buffer_size` only needs `≥ block × batch × threads`; larger + values do not improve throughput. The example commands use 4 GB + because that is safe for any reasonable config. -### Eager endpoint warmup (first-request latency) +### First-request latency -libfabric `FI_EP_RDM` endpoints resolve peer addresses lazily: `fi_av_insert()` and the metadata handshake fire on the first send to each `(local_ctx, peer_nic)` pair. On 16-NIC instances that gives `16 × N_peer_NICs` serial handshakes inside the first `submitTransfer`, which shows up as a single-digit-second first-batch stall (measured ~4 s on p6-B300 for a 100 × 0.5 MB batch; the first batch runs at <0.1 GB/s while the CQ drains, steady-state afterwards is unaffected). +Peer addressing resolves lazily: `fi_av_insert()` and the metadata handshake fire on the first send to each `(local_NIC, peer_NIC)` pair. On 16-NIC hosts, the first few `submitTransfer` calls carry this cost before steady state. -Mooncake exposes an explicit eager-warmup API to eliminate the stall: +**Measured on p5en (16 × 16 NICs, cross-node, 1 MB write, 3 reps, median):** -- C++: `EfaTransport::warmupSegment(const std::string& segment_name)` -- C: `int warmupEfaSegment(transfer_engine_t engine, const char *segment_name)` -- Rust: `TransferEngine::warmup_efa_segment(name: &str)` - -Call it once per peer segment, right after `openSegment` (or after any metadata change that adds a new peer). Every `(local_ctx, peer_nic)` endpoint is connected concurrently via `std::async`; the critical path becomes `max(handshake RTT)` instead of `sum(handshake RTT)`. The call is idempotent — safe to re-run. - -Measured on p6-B300 (16 local NICs × 16 peer NICs, dual-NUMA initiator, 100 × 0.5 MB batch): - -| | first-batch latency | steady-state | +| | cold submit #0 (no warmup) | `warmupSegment()` (all 256 pairs) | |---|---:|---:| -| No warmup | 4,043 ms | 141 GB/s | -| `warmup_efa_segment` (256 endpoints connected in 4.1 s) | **13.5 ms** (~300×) | 230 GB/s | +| SRD shared endpoint (#1944) | **26 ms** | **1.1 s** | +| Per-peer `fid_ep` (upstream main) | 99 ms | 17 s | +| Speedup | **~4×** | **~15×** | -The warmup call itself takes roughly the same wall time as the stall it replaces — the win is that it's a one-time setup cost decoupled from the critical path of the first real transfer, not paid inside your latency budget. +The SRD shared-endpoint refactor (#1944) speeds up first-request latency two different ways: + +- **Without any code change from callers** — the cold `submitTransfer` is ~4× faster (26 ms vs 99 ms), because the shared endpoint removes the per-peer `fi_endpoint` / `fi_enable` that used to dominate. This is what existing Mooncake callers (vLLM, SGLang, etc.) will see. +- **For callers that want sub-10 ms first-request latency**, an explicit eager-warmup API lets you pay the handshake cost up front, outside the critical path: + - C++: `EfaTransport::warmupSegment(const std::string& segment_name)` + - C: `int warmupEfaSegment(transfer_engine_t engine, const char *segment_name)` + - Rust: `TransferEngine::warmup_efa_segment(name: &str)` + - Python: `engine.warmup_efa_segment(segment_name)` + + Call once per peer right after `openSegment`. The call is idempotent. Under this refactor `warmupSegment` itself is ~15× faster than the pre-#1944 code (1.1 s vs 17 s), bounded by the peer's single-threaded handshake RPC daemon (`accept` + JSON parse serialized on one thread), so it scales linearly with the number of fresh NIC pairs. + +vLLM and SGLang do not currently call `warmupSegment` — they go through the generic `TransferEngine` interface and pick up the 4× cold-submit speedup automatically. The API is there for direct Mooncake callers that want the larger win. + +#### Reproducing the numbers — `efa_first_submit_probe` + +`mooncake-transfer-engine/example/efa_first_submit_probe.cpp` is a two-host probe that isolates the two latency costs `transfer_engine_bench` hides inside its 10-second throughput average: + +1. **Cold first-submit** — the handshake + `fi_av_insert()` that fires on the first send to each `(local_NIC, peer_NIC)` pair. +2. **Eager warmup** — how much of that is paid up front by an explicit `warmupSegment()` call. + +It is EFA-specific (there is no `warmupSegment` on the RDMA / TCP transports — they establish connections at `connect` time, so "pre-warming" has no meaning there) and intentionally not wired into `ctest`: it needs two hosts. + +**Run**: + +```bash +# On the target host: +./build/mooncake-transfer-engine/example/efa_first_submit_probe \ + --mode=target \ + --metadata_server=P2PHANDSHAKE \ + --local_server_name=$(hostname):12345 + +# Note the "[target] ready, addr=:" line, then on the initiator: +./build/mooncake-transfer-engine/example/efa_first_submit_probe \ + --mode=initiator \ + --metadata_server=P2PHANDSHAKE \ + --local_server_name=$(hostname):12346 \ + --segment_id=: \ + --warmup=1 \ + --iters=5 +``` + +**What it prints**: + +``` +warmup: 1094.74 ms (rc=0) # present only with --warmup=1 +submit #0: 6.95 ms # first user-visible submit +submit #1: 5.45 ms # steady state +submit #2: 5.92 ms +submit #3: 5.67 ms +submit #4: 0.09 ms +``` + +- `warmup:` is the wall time of `EfaTransport::warmupSegment()` — it opens every `(local_NIC, peer_NIC)` pair (16 × 16 = 256 on p5en) up front. +- `submit #0` through `#N-1` are the timings of single 1 MB `submitTransfer + poll`. With `--warmup=1` they are all in the steady-state regime. With `--warmup=0`, `#0` pays the handshake cost, `#1+` are steady state. + +**Flags**: + +| Flag | Default | What it controls | +|---|---|---| +| `--warmup` | `true` | Call `warmupSegment()` before the first submit | +| `--iters` | `5` | How many timed submits after warmup | +| `--xfer_size` | `1<<20` (1 MB) | Bytes per submit | +| `--buffer_size` | `1<<30` (1 GB) | Registered buffer size | +| `--local_server_name` | `hostname:12345` | Local metadata advertise name | + +**Use cases**: + +- **Deciding whether your application needs `warmupSegment()`**: if `submit #0` in the `--warmup=0` run is acceptable for your use case, you don't need to call `warmupSegment` at all. +- **Comparing PR branches**: run the probe on the same hardware on this PR's branch vs `main` (or whatever upstream you're benchmarking against) to see per-NIC-pair handshake cost directly, without having the number drowned in a 10-second throughput average. ## Usage with vLLM -### Prefill Instance +### 1. Prefill Instance ```bash VLLM_MOONCAKE_BOOTSTRAP_PORT=8998 \ @@ -332,7 +416,7 @@ vllm serve -tp 8 \ --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_connector_extra_config":{"mooncake_protocol":"efa"}}' ``` -### Decode Instance +### 2. Decode Instance ```bash vllm serve -tp 8 \ @@ -398,32 +482,35 @@ AWS EFA exposes RDMA-like devices through the ibverbs interface, but does not su ### EFA Transport Architecture +Under the SRD shared-endpoint model every peer is addressed through one `fid_ep` per local NIC — peers are AV-slot entries, not separate endpoints. + ``` -┌─────────────────────────────────────────────────────┐ -│ EfaTransport │ -├─────────────────────────────────────────────────────┤ -│ EfaContext (per device) │ -│ ├── fi_info (fabric info) │ -│ ├── fid_fabric (fabric handle) │ -│ ├── fid_domain (protection domain) │ -│ ├── fid_av (address vector for peer lookup) │ -│ ├── fid_cq (completion queues) │ -│ └── fid_mr (memory regions) │ -├─────────────────────────────────────────────────────┤ -│ EfaEndpoint (per connection) │ -│ ├── fid_ep (RDM endpoint) │ -│ ├── fi_addr_t (peer address) │ -│ └── local_addr (local endpoint address) │ -└─────────────────────────────────────────────────────┘ +┌───────────────────────────────────────────────────────────┐ +│ EfaTransport │ +├───────────────────────────────────────────────────────────┤ +│ EfaContext (per local NIC) │ +│ ├── fid_fabric (fabric handle) │ +│ ├── fid_domain (protection domain) │ +│ ├── fid_av (address vector — one slot per peer) │ +│ ├── fid_cq (completion queues) │ +│ ├── fid_mr (memory regions) │ +│ ├── shared_ep_ (the single fid_ep that serves every │ +│ │ peer via fi_addr_t lookup in the AV) │ +│ └── peer_map_ (normalized nic_path -> EfaEndPoint) │ +├───────────────────────────────────────────────────────────┤ +│ EfaEndPoint (per peer) │ +│ └── peer_fi_addr_ (AV slot index for this peer; sends │ +│ route through the owning context's │ +│ shared_ep_ with this fi_addr_t) │ +└───────────────────────────────────────────────────────────┘ ``` ### Thread Safety -The EFA transport requests `FI_THREAD_SAFE` from the libfabric provider and adds per-endpoint spinlocks to serialize `fi_write`/`fi_read` calls. This is necessary because: +The EFA transport requests `FI_THREAD_SAFE` at the domain level and guards the shared endpoint with a single `post_lock_` spinlock (one per `EfaContext`, i.e. one per local NIC) to serialize `fi_write`/`fi_read` calls. This is necessary because: -- Multiple submission threads may route slices to the same endpoint concurrently -- libfabric RDM endpoints default to `FI_THREAD_UNSPEC` (no thread safety guarantees) -- Concurrent `fi_write`/`fi_read` without serialization corrupts provider internals, causing completions to silently vanish +- Multiple submission threads may route slices through the same shared endpoint concurrently. +- libfabric's EFA RDM endpoints are not thread-safe for concurrent `fi_write`/`fi_read` even under `FI_THREAD_SAFE` at the domain level — concurrent posts corrupt provider internals and completions silently vanish. CQ completion queues are polled by dedicated worker threads (one per EFA device) that run independently of submission threads. @@ -435,9 +522,8 @@ CQ completion queues are polled by dedicated worker threads (one per EFA device) | Endpoint type | `FI_EP_RDM` (message-based) | Queue Pairs (true RDMA) | | Write operation | Software-emulated via messages + ACKs | Hardware-offloaded one-sided RDMA | | CPU overhead | Moderate (provider processes ACKs) | Minimal (NIC handles everything) | -| Throughput CPU-to-CPU (8×400G) | 222 GB/s (tuned) | ~190 GB/s | -| Throughput GPU-to-GPU (16×200G) | 347 GB/s (tuned) | N/A | -| Throughput GPU-to-GPU (8×400G) | 313 GB/s (tuned) | N/A | +| Throughput GPU-to-GPU (16×200G, p5en) | 365 GB/s (tuned) | N/A | +| Throughput CPU-to-CPU (16×200G, p5en) | 213 GB/s (tuned) | — | | AWS availability | All EFA-enabled instances | Not available on AWS | ### Supported AWS Instance Types diff --git a/mooncake-integration/transfer_engine/transfer_engine_py.cpp b/mooncake-integration/transfer_engine/transfer_engine_py.cpp index 02bcab91..3bede64d 100644 --- a/mooncake-integration/transfer_engine/transfer_engine_py.cpp +++ b/mooncake-integration/transfer_engine/transfer_engine_py.cpp @@ -21,6 +21,10 @@ #include #include "transport/rpc_communicator/rpc_interface.h" +#ifdef USE_EFA +#include "transport/efa_transport/efa_transport.h" +#endif + #ifdef USE_HIP #include "transport/hip_transport/hip_transport.h" #endif @@ -951,6 +955,20 @@ void TransferEnginePy::batchTransferReadOnCuda( } #endif +int TransferEnginePy::warmupEfaSegment(const std::string& segment_name) { +#ifdef USE_EFA + pybind11::gil_scoped_release release; + auto* t = engine_->getTransport("efa"); + if (!t) return 0; // EFA transport not installed; nothing to do. + auto* efa = dynamic_cast(t); + if (!efa) return 0; + return efa->warmupSegment(segment_name); +#else + (void)segment_name; + return 0; +#endif +} + uintptr_t TransferEnginePy::getFirstBufferAddress( const std::string& segment_name) { Transport::SegmentHandle segment_id = @@ -1086,6 +1104,8 @@ PYBIND11_MODULE(engine, m) { py::arg("device_name") = nullptr) .def("get_first_buffer_address", &TransferEnginePy::getFirstBufferAddress) + .def("warmup_efa_segment", &TransferEnginePy::warmupEfaSegment, + py::arg("segment_name")) .def("get_notifies", &TransferEnginePy::getNotifies) .def("get_engine", &TransferEnginePy::getEngine) .def("get_engine_ptr", &TransferEnginePy::getEnginePtr); diff --git a/mooncake-integration/transfer_engine/transfer_engine_py.h b/mooncake-integration/transfer_engine/transfer_engine_py.h index 083ff357..0e9ac138 100644 --- a/mooncake-integration/transfer_engine/transfer_engine_py.h +++ b/mooncake-integration/transfer_engine/transfer_engine_py.h @@ -150,6 +150,11 @@ class TransferEnginePy { uintptr_t getFirstBufferAddress(const std::string &segment_name); + // Pre-connect every (local_ctx, peer_nic) pair for `segment_name` so the + // first submitTransfer does not stall on handshake RPC + fi_av_insert. + // No-op on non-EFA builds or when the EFA transport is not installed. + int warmupEfaSegment(const std::string &segment_name); + int writeBytesToBuffer(uintptr_t dest_address, char *src_ptr, size_t length) { memcpy((void *)dest_address, (void *)src_ptr, length); diff --git a/mooncake-transfer-engine/example/CMakeLists.txt b/mooncake-transfer-engine/example/CMakeLists.txt index 54a6cdfb..15eaba45 100644 --- a/mooncake-transfer-engine/example/CMakeLists.txt +++ b/mooncake-transfer-engine/example/CMakeLists.txt @@ -24,6 +24,12 @@ add_executable(transfer_engine_bench_with_notify ${WORKSPACE}/transfer_engine_bench_with_notify.cpp) target_link_libraries(transfer_engine_bench_with_notify PUBLIC transfer_engine) +if(USE_EFA) + add_executable(efa_first_submit_probe + ${WORKSPACE}/efa_first_submit_probe.cpp) + target_link_libraries(efa_first_submit_probe PUBLIC transfer_engine) +endif() + add_executable(memory_pool ${WORKSPACE}/memory_pool.cpp) target_link_libraries(memory_pool PUBLIC transfer_engine) diff --git a/mooncake-transfer-engine/example/efa_first_submit_probe.cpp b/mooncake-transfer-engine/example/efa_first_submit_probe.cpp new file mode 100644 index 00000000..ffef327e --- /dev/null +++ b/mooncake-transfer-engine/example/efa_first_submit_probe.cpp @@ -0,0 +1,199 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// First-submit latency probe for EFA transport. +// +// transfer_engine_bench reports a 10-second throughput average, which +// hides the two costs that matter for first-request latency: the +// handshake / fi_av_insert that fires on the first send to each +// (local_NIC, peer_NIC) pair, and the optional warmupSegment() that +// pre-pays that cost outside the critical path. This probe times +// those two things separately so the cost of each is directly visible. +// +// Reports: +// - warmup: time for EfaTransport::warmupSegment() (if enabled) +// - submit #0..#N-1: time for a single 1 MB submitTransfer + poll +// +// The first submit carries the handshake cost when warmup is off; the +// rest are steady-state. It's a two-host tool (not hooked into ctest) +// used to: +// (a) decide whether to call warmupSegment() in your application — +// does it actually remove the stall? +// (b) compare this PR's shared-endpoint cost against upstream +// per-peer-endpoint cost (see the numbers in docs/.../efa_transport.md +// under "First-request latency"). +// +// Usage: +// Target: ./efa_first_submit_probe --mode=target ... +// Initiator: ./efa_first_submit_probe --mode=initiator \ +// --segment_id=: \ +// --warmup=1 --iters=5 + +#include +#include +#include + +#include +#include +#include +#include + +#include "transfer_engine.h" +#include "transport/efa_transport/efa_transport.h" +#include "transport/transport.h" + +using namespace mooncake; + +DEFINE_string(mode, "initiator", "initiator or target"); +DEFINE_string(metadata_server, "P2PHANDSHAKE", "metadata backend"); +DEFINE_string(local_server_name, "", + "local server name (host:port); default = hostname:12345"); +DEFINE_string( + segment_id, "", + "target : from target's startup log (initiator only)"); +DEFINE_bool(warmup, true, "call warmupSegment before first submit"); +DEFINE_int32(iters, 5, "number of timed submits after warmup"); +DEFINE_uint64(xfer_size, 1 << 20, "bytes per submit (default 1 MB)"); +DEFINE_uint64(buffer_size, 1ULL << 30, "registered buffer size (default 1 GB)"); + +static std::string defaultLocalServerName() { + char host[256]; + if (gethostname(host, sizeof(host)) != 0) return "127.0.0.1:12345"; + return std::string(host) + ":12345"; +} + +static double toMs(std::chrono::steady_clock::duration d) { + return std::chrono::duration(d).count(); +} + +static int runTarget() { + auto engine = std::make_unique(false); + engine->getLocalTopology()->discover({}); + + std::string name = FLAGS_local_server_name.empty() + ? defaultLocalServerName() + : FLAGS_local_server_name; + auto hp = parseHostNameWithPort(name); + int rc = + engine->init(FLAGS_metadata_server, name, hp.first.c_str(), hp.second); + LOG_ASSERT(!rc) << "engine init failed"; + + Transport *xport = engine->installTransport("efa", nullptr); + LOG_ASSERT(xport) << "installTransport(efa) failed"; + + void *buf = numa_alloc_onnode(FLAGS_buffer_size, 0); + LOG_ASSERT(buf) << "numa_alloc_onnode failed"; + + rc = engine->registerLocalMemory(buf, FLAGS_buffer_size, "cpu:0"); + LOG_ASSERT(!rc) << "registerLocalMemory failed"; + + LOG(INFO) << "[target] ready, addr=" << engine->getLocalIpAndPort(); + LOG(INFO) << "[target] Ctrl-C to stop"; + + pause(); // block until SIGINT + + engine->unregisterLocalMemory(buf); + numa_free(buf, FLAGS_buffer_size); + return 0; +} + +static int runInitiator() { + if (FLAGS_segment_id.empty()) { + LOG(ERROR) << "--segment_id is required for initiator"; + return 1; + } + + auto engine = std::make_unique(false); + engine->getLocalTopology()->discover({}); + + std::string name = FLAGS_local_server_name.empty() + ? defaultLocalServerName() + : FLAGS_local_server_name; + auto hp = parseHostNameWithPort(name); + int rc = + engine->init(FLAGS_metadata_server, name, hp.first.c_str(), hp.second); + LOG_ASSERT(!rc) << "engine init failed"; + + Transport *xport = engine->installTransport("efa", nullptr); + LOG_ASSERT(xport) << "installTransport(efa) failed"; + + void *buf = numa_alloc_onnode(FLAGS_buffer_size, 0); + LOG_ASSERT(buf) << "numa_alloc_onnode failed"; + memset(buf, 0xAB, FLAGS_xfer_size); + + rc = engine->registerLocalMemory(buf, FLAGS_buffer_size, "cpu:0"); + LOG_ASSERT(!rc) << "registerLocalMemory failed"; + + auto segment_id = engine->openSegment(FLAGS_segment_id); + LOG_ASSERT(segment_id != (SegmentID)-1) << "openSegment failed"; + + auto seg_desc = engine->getMetadata()->getSegmentDescByID(segment_id); + LOG_ASSERT(seg_desc) << "getSegmentDescByID failed"; + uint64_t remote_base = (uint64_t)seg_desc->buffers[0].addr; + + LOG(INFO) << "[initiator] peer=" << FLAGS_segment_id + << " warmup=" << (FLAGS_warmup ? "ON" : "OFF") + << " iters=" << FLAGS_iters << " xfer_size=" << FLAGS_xfer_size; + + if (FLAGS_warmup) { + auto *efa = dynamic_cast(xport); + LOG_ASSERT(efa) << "transport is not EfaTransport"; + auto t0 = std::chrono::steady_clock::now(); + int wrc = efa->warmupSegment(FLAGS_segment_id); + auto elapsed = std::chrono::steady_clock::now() - t0; + LOG(INFO) << "warmup: " << toMs(elapsed) << " ms (rc=" << wrc << ")"; + } + + for (int i = 0; i < FLAGS_iters; ++i) { + auto t0 = std::chrono::steady_clock::now(); + + auto batch_id = engine->allocateBatchID(1); + TransferRequest req; + req.opcode = TransferRequest::WRITE; + req.length = FLAGS_xfer_size; + req.source = (uint8_t *)buf; + req.target_id = segment_id; + req.target_offset = remote_base; + + Status s = engine->submitTransfer(batch_id, {req}); + LOG_ASSERT(s.ok()) << "submitTransfer #" << i + << " failed: " << s.ToString(); + + TransferStatus status; + for (int poll = 0; poll < 20'000'000; ++poll) { + s = engine->getTransferStatus(batch_id, 0, status); + LOG_ASSERT(s.ok()); + if (status.s == TransferStatusEnum::COMPLETED || + status.s == TransferStatusEnum::FAILED) + break; + } + LOG_ASSERT(status.s == TransferStatusEnum::COMPLETED) + << "submit #" << i << " did not complete"; + + engine->freeBatchID(batch_id); + + auto elapsed = std::chrono::steady_clock::now() - t0; + LOG(INFO) << "submit #" << i << ": " << toMs(elapsed) << " ms"; + } + + engine->unregisterLocalMemory(buf); + numa_free(buf, FLAGS_buffer_size); + return 0; +} + +int main(int argc, char **argv) { + gflags::ParseCommandLineFlags(&argc, &argv, false); + google::InitGoogleLogging(argv[0]); + FLAGS_logtostderr = 1; + + if (FLAGS_mode == "target") return runTarget(); + if (FLAGS_mode == "initiator") return runInitiator(); + + LOG(ERROR) << "--mode must be 'target' or 'initiator'"; + return 1; +} diff --git a/mooncake-transfer-engine/example/efa_latency_bench.png b/mooncake-transfer-engine/example/efa_latency_bench.png deleted file mode 100644 index 34693c0710e20d6ea46a2fdce1137fe938cf7af6..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 64738 zcmce8bx>7*^z8)%1TG*TT}nzwh?JCpiUNY5gtQVO-Q7rt($b*^P4ws=Kb+z9y5Nw#J%U9Pwl<-T5F#`#e262@oDf;C=}rx8EGXH>J%D+PEe54nCj91GBF!$Xn*(zxpc#$E51XC}h@D`yg1me^T~+SzB^gIoB}SR{YG!ZI97uAu(@(Bcxw zcjoW!(f_}{wG8V>6zeLrF;ewE&vf$TB(3d|{PYsK&eHA-_Wi3@Q7tVk4?aC;>3$n< z@+HGyxiWgKqM{-oD9Fa{0?M)Mil?_+DUB67MG!VB|JTB}jC2C~`STb#m06V)6`FLf znlrmPpIHez%LkRWm@JdL=Ccg5p8Bens|MGzw7bA`kA$2Y`>KpgKoB#DRd8hF*(e?@ ze9z=$#<8(6N9VcT_mdWT>KB)8WgA-w>*?t^abmou!TEEuOyORn|IgnFMN*KESk;&c zDW~6~M|u1Btp7}EMy_9a4G&o{Tsbx}Nl8i6&fZ>`2o3xqY%pDeRQk^s3l=N-e|$UG zW>J0Eh(#=3vsgS53}evN*7o)Fwd~b$SZ!fXKEuekJa9j`-{Z_Prscbq7$(kGNBi-^t|SSgZy%0eqPqnk`G>~ zU+asfmf54>%;2)UH1%3l`z>s{NN#mf3k!>bgM$k?RpN{Ng%gPm!+J9viPRnCdrri3 zA}djkEnhR=4pyviw0c3wYrgGDCsDOCPEc;c_27eLoehjttK7cba;Vg1#?V+(g^*6< zn|YR}C#oY^Dp^l6LnU*eQOm6R(?j(mVF{aQ^m64kL*}DGrpuSlGBPqA93J8k5Txi- zySK#(_(w)Y#=A?NtW@!JLG)lV0dGa+u{r&GPv(<@oz>6k`F=`iccGitrtmrkqqHRjO6A>e7!=_bc+hV!ojZNUV@&)&f zoSdAAtel*Fdz>J0A!-4`9LurmFadew$FI3m9k+fN?H=tf^P}ssctu5HpELzqE%fn6 zUDRkSU#@C}-Tx8BTaJKYt874E;M=F)-xZ5{xVnlbyYF$pbX821W^Mj5 zN?x6gw+Ign)$L7+jI6J6+wUv3V1#XBu<`r5`2Irvz3Qq#wYzukJ__#~8A8OVey4oxw4R;D#6r z9Cn#&slHzEYWaq*hE_E!DjQbD>M#Z_9pvQjaZ?an^%E}6dQ63*AijDkFfi~G4$k=4 z*q7%7l$YqMUAE2Htn{9D|1fGFp>}%seyh^Z%xh4GH-tg~tlL{i|YTO?2NY+z7 zfZd{&ql=O3wxu35-M;+n=Lfy*!_MH(Nx}|Gn&k>J5Y6D=;+S#~C%onh2nm_V@04s+ zc|7kt-H~V#&Aqa*F}dC;WwbK-#nCZ!gw=ku219c1H*>iV65PWEQ>&$ie50eIhgQ3j z8&$G2W0`KgNTtWc!7()MSgt-EUeaDL8Cj@WZKSn7T%T@lRcRY>+eXd^)@U8E6w8GR zzsn}lsZuMk)O$2CAZ>3IP|2{ zaa}FzZE34MK58ua5;&! z=FMlKKWJB$-Oqd^nBKk(>qpDLz<{HRYJ{j>{x73}9 zMqJll4OK`uZ#4I8pX=_n8FUzNyEXOo^90$zGunsCN!^Th(iM`0+6?oD<}Kiu*{{1oY*PC%C;gpPl~7t>9@`E+l5tLyf@6hmbM^AUMv<@ku9~WL}6i0-sc~ zm*)6>{Ej*$d_UzDq2$rd6K(fh19Y7(d?rlTu=&gMboLv5=&{|Iih^3 zB;De;RI^rgCPLQtm#YOVM{5ccGPxdps{j2xC7*n`OF>HB)RY@?tXvG=`5S3Yw_%OC zs)&e)8V3djvX1+&2k&n$r<)BF+0S@oR@HR7y1HI?RB++@_wU9t1`QCT7J~8Q;(g;2~)hB77P}#t?nXs~A zc{hqXcB|sOLUNMcri^{%W{-yL?{63x-CtF8+!|zgE2VQEH<&wDndk_Z^%t~QlT@ar zvPkYuMifjyGK~{5djT`m6v8+&`o%}H)Ozau6EY-Bltz)_m`|ofYrlhEZY+;T-I%h= zA&+obDxJxDSiuD`n$dm7i-`UTswi#PVaSfM++i8Vv~>jn!}YP+=lS9@3%~N!A&Tj` z+_1N^s+}(_t3r~)r45~ZwvF+IAix2`r62Kpey*EO!b|O&U`OVQB?0(=z$@*y{;0^b z<((~+B)W8SsvAn&JB=wdwV|}D7py8)ONn-)pFH7ktdwM zJdb1dPb!(t`3gRemcExeSipqdgQN&Sm>O2&NWRly(d519uRL{0VKz3u)@a^z`s1wu zi$SNEMB^qrJUqyCOd7|$FA!|18S;#(BrICnIS`k@@!UmqMZk0YurG=#2LXi`+sN6%-M%9emg>>|bbI{ns1GrmA`)C;hWclP zN*|Td3Ov#)roX+tVW0fdVQEm*tgxjpgN)9sJVwi+a#Q7zn^$(#9_PheTY)UMcKsQ_ z2k)fuFflQI1dq5ZOTg}HgRNcaxXM&jRRx>Ni{j#=L0i`oJf*5!TO;2FkPOaY8hr>A zzc;;@{mOtvbuP^4-Dn=dK1m6WgVm<|%7+8&$*$pgVikLv^F|j&?U@11Om56}IXXJ> z>DBt88)&j;=H3MaM``N}L|r-RfBZuD%J!`WO6Aww>OYotc2+0so5;CTQg4Sa#F~@+ z4oLA1=q=K6NsJeD7_IR#&JHA?r_@V9A!p)J$YSP|Er9=Z%{>WT>BSeidn0hk29Grjs1VS(VH`hk3-kRhwNubvtb!j~^c5=$L~0*lz&3OlP030-5JL6qONoWoALUV&EE6rJAOC>&l+S2KhA!k@r?M|1?XvR|k2%;sR;2wn~l)imCU#||Q{2_(S zOhy*F$V#3U zS1s)%d`8^&Ujcp{lxzOgQ*5DAKA|9WczN;FQIXkz8UT(*IslJzDABgJtrr24@$Wk{zTmH_j!dm8i1uwV1137Aa<1xWnw&p z%K5ThXrfItuTBR9p}>$3TvNO#IH-x*mCpCyNe33#1T#oDcXhI|54Oc0#|ju3?o-g+ z&eArpcwvbj?M%F+7Zb`fgYnq6ym)sr=tqIE@>1otcDc=Z zf3=3Y!KGLvH{J}QU}a%pk)BYF5Q6 z8>QbDQfWR^)>yecY;yMEqn}xo2CY%*Ne-|#{i35)`%91aM~)G^1NWL*NY447xi8`Tm9IRGcAJ(ekpHf1Z50LWR_!k# zNWxqg%EZGw^7X0K=vPn@8SD(J!fgHS%T4+I_6BYWMAhEIBORBl7X3@esqppprv<=f z?}h*?<&Hx@r|6oM)m46gsxM*PXcDj3U4prP;?NO#{zl%0n~3X9&X5xX8vue5bdo1n z+$Q_Tp?Wxk1pwZ5B^n?95>yZL!ESRO(-)>SVH8nGO$eIm5QFz>c>q2ZaP7=~Lph<&EMA!VoHJV2&&t*dC?R9Ks9-a<9be)LH*5~^bj;jS#5Zr0++__VH z)#?NMkFIAi5Yd!m8X6w41p4l2LAplSPU-lyDPnuuzLGr2z!!a z?b1QU{-a{^#4&l7xfD2DehrUXHU{^X$~!;6Qbcvg%TsACBM5b(IgD9-k3ln@uVgqy zHk!wN#!yf%$9eNvxs}`AMvLtN)SY_5Te(dPtv}XF?$@4UVyazPSt*%>ScaINSD%0uY&((01GG=9W-xp*cLy`G@5Js%W53q=U~7d%D!u%lwWdGoUj=z?cxs z{qMf!y#GP*ytsq}hstNF_ITkLA7aT-01NMdL{N`gK0aDLM#^$sjktY4_34YN+Q4;` z4=}(UY0S~9OE^x=8e3lOgq=$*X3q^^{Va#lDIiTaAAG2R7~helRsMUpvb@d)uvN#> zP`MN2(T|F6F`*uTSNa0>YYe7ug}1M6Z+F_u0ctSpFEBP8D0*d-HRiC z0r}{SZHRRh zHkWSKdwDeVC0~G8I@^N~DprAf_?{+931}&f!a`T~930;#kRBHLa+x)XvnC5ng6Ori zn%5se3VNrO``Eb2X1c9KkzmyDLAEv$Ylhaub@z|wp{PJJN+tl5Pz106G~UYkdJ0Ub z_`ynDTav^;mgg)~c?fG_JXpe?CL7(@)PxI@gCs4e31WpU7yv-n-aUaK;j-amx{K}0 z^{*r3{KBx}TUrUbKitO(YN#EqbkU-JG5JJwL|H<&2E}$S;dh>z5`nR{Mx_pmRM-5U z17N-pM^T*PD)RXd?MXAz_w41(gu;?b?=B|^Ga#JFidv>h`9)PI(WfJ8qmi7imT74z z`uP~2DBIrda;#GARcYyZz}$EB^;uBv?(PsGxb$GVF%`4TURv10JKCdow^e>6j`rl0 zNFZckZ%Fi$pU`V1wq)(^{;nSj^xx}*QH5kH6J2-9F25Ek6%%AZ`rHccAuU9SlwY|N) z95%Oa-BN;tD-%_60NPg)ooC4x8ER^0AlB4s${Mi7S;^kEV}O3B&5Hv&4b?DzY%dwvEX`^|BR`5 z1Ya{=r{q4jYL@00q`;{Wk7GMKkv}&uj*pj{*Pi5eiBOA7hv$B6jhFC< zYXrobJOE*kzATKhU;;%_^Ot8#Ma z_a+1sfs+FN_-alQ{9#;=lyAX|t5@=wpA(`dA<)n#Ojl;r|*cW$-SJ8bK>S+NUjXkfNl zrFC2-UHZ3jk_NpFW3|{QA+vsIAk;2smhe>{?PUx`Tp2vV6K#!PuMJ+h1{W>$YW<$GdAsj2!}vXe1>!7I4SB_x^DA4OhdXN}nDKm6 z0*2@(jW6Fqk@%8M^hfs?(3K2NsBi#9dtP)3Rllwdn`iCK2dtG|;T#(r7M+eho!m4t zjry;PHFeC}6RqrnYcyJc5;lxi4Ty?fT zV{FkuV-aQN4%wgCb!l|Z7ydbCpmAUElgd(4G;yhXEdq8;2TQ(Zwez9Niy$9Q*{l6I zDd7=HqSN=j;Fwj6)sem8cA;<0CsXoET0ZrDkpH!=FN*sZE1Z%0 zt3$p(&_X(cyUrrQwoD=aE4V135{67PVG}l84UO*%7hnF z%69oMZ)f}Ss?+Wb))M0PPtwnM;F_QHe8zj~ zVF5Fg{z3vTzOgQf@!Z(;#-nd#MSG{Z%e~VbcyQUFWd9Shq^*Zr7K_YCn5Vr3r{eCB z`Pvn*;(rxXcJmg$ej+%*by}!Z2o^LRbLl&7biOFNTu#S6l2j{*9*N zH5IYgdq`Q=+s0lVUNbP;#GBjpzkU;Lu7&1Am!jiC`v6447@eOFs;-t|yBGq;v$Tq! zNv}(-?+P_3F9gk@Z;*`W6Tf^ePH{>+dJdze%TeQo|DlS8NP!Jq-T0X;PeQ`s3CVuK zp(s24h*;e1Qt++=0VKpDb(9~HFZ5*3Ty~-PD%7#-*kHU79r9=0^v(CrstL7i6&6cf zcCXzkWX0Y(f^co=`lNB)71OMk?y@`Si>S^@c+bE6K+=R@iVqtTg(+{GDf>1BOz8+H zLkz^zWt;}?`j!#C zGD-DMpJPceQKd{RyP`fih9_9|ixfL;en#6goa^5QhrP_w(y|m((0RHRnJIZqadz;P ztTh1gKZouyFDr?PN2HU-+jrmg?3%>b0tQwr?NEPDWYgyZ6i;Gnh161{8Mb$cU?yL) zKf11l4-K;F;ac2?TVetWl1JT}=$7M_*8{wTkeNIG#DqsDfGlB{7Cciz4=_hNh#f{8m zH_brPNF+_@1)BVh@)hrRR%kuMbqNr2`L2?>ePkWGq6NI(PS zA05P;%i7PQRaOrbE08Ip2fOQ$7w*?~>)nx@7#q9R@PZrzDMJ|;zUBr61kmmqVYIoU zKezl6Lk1g1isPgS*%Adv7lzZ<@I64EXS4l*wxjYCyjfB43<@Lz79O5^cvvk$#Kgp) zR1%#i#3{(xxF%*)dvNTvI`T;_yLM&t!G&HClbQIn4o+n&9#-@kUmc%}T*1XxA z;Kd8fQK&2)WNH3tAOVJFkg3jdMEm)xk+foLRj(t0a~}%k$HT>e^+85LLgHXlSd-mZ zSJX0V#`^5)@vfp;`JKQgStG}ClWZU_(@NqH(LHsz2f9a=SQY5X)L9WRF*4uZ-Q@DB&h7R?wR~xjfSA~!Z#d;PF+L?v!rP(mlJh_cPS!prKmjk30Qx8i z4UMyME*lHWN{@!ccOb^lE&llAoR%BE@0Ta?5B|!?uKHSHHQ~74E*vZFwDw8?F+_yL zM}(iwXMA2+{l56M367gPj*i(7PcNMOzB@8SIZt~zq83GOm|nl+Ri!=XcKgni9-@X` zu4VH5I1#JIMV-4FT?+JBY?k^^x}bdh{E*XN*cS68h_@ol-_RwWwix7p~tD^c9C%6j76mWvthJaPQBc z9o$R2qYW4JTSn4%*K3@j4&374$H@+2)trT7fh;0^)kIJifMOYinzJwWiPy%d+V&k# zv`*vUfdom^fzPi4G&!%IUaFe~a;ED8Nhfsou(n#fKW%@xA@G_zlp8V!N+WNyg+n8t zDsnPMYEOw&;odK$H{m!^hZ|&I(6C+Xe+@hp!YlK%$O9E43^6>?2FgLOO#hXi^)J>)9if=Zts%F2U7GxcE% zESyvc=dA@Ng74`XC6?zwZaIJDN*u_~WHdBiA&MD)e}@fBa0lNJqN@OsLVN-sUjE2^ zavLOVN3bv;tTe)|(0YJy4P;X-j9w0G79a{rsbstc(JC3#deN>DxEsWz0aC=~e6QG4 zDWW@^b&P(e`{QtBd3vL$jQB>#@|T6zyG~3*7xY3qsr|k*aJ3X~#8hsM$f;6ja>3H& z7azIE_O=o~KLnZUSBVufY>reQPf;85y~LgXiH-I}KWmw^E(tyR*_`uq10cjV}V_`?(0uwh4xAzuUZbZof+O*Mp0Y=dF;eN^o)occrtpz=x z3lK|2bF)m6>)IWV9lyeg0aXU_lc>uUw}QmxS)e-%L8CnYvJt!p!C++BD$5l#AEc5w zK7wHeG+zvmz$O!`ogJDrd&S`2@uY@%xtlqnLg6OydTF2U*}01H%c|MC(+AZ~Hb*1E zZbuz@+h-m$i~eeFi<-r10~HRK6~m|5YPs*~RzVm_2g32{9;Y{O1y6(q+)H2&M)Vh& zvPdmTzsP&ZB?oS(S3xCqp4fP&J%#Q@-*lsIIAZW3X75GTV>Hx`(R zPz!^lUDOH>;pC)_mRX|N?EGTtX3ZQVu@-OD*c};ClrpfQS#-*Wi(UX1Xw#SUtLO#f zIN5z?pnk6Iee!;AxVJem6|G}9_w%FrJ*xVqCL3^T+>NwsXDHqtv{CRY0`&;f68>6H z1nZS&GjngLWP%8O88t=+dOI-h&DqsQ2$KQD>IAIbsW%WEjEf&?W;!en_sUbHx_5Sh z7EsB~8ZN6u=9iSTXUD1@wp?cOVQ`;x%280WZch#OOI<0|btlQmoZ)&noEqDl|t>f20Cf$8Z+{hgja2mD~}))Mkvbgd}^ zsy~T1m5+Tp5bbSXF~5Q5BVTR zxy93<=K_Oc^m}`4jY&u-O2>WoIS6^9e(TuS*cfMMY3B#89zQ93ls({)B7FRjn?i%X z75psz8KhYhbad!YmDckay(KEo-ksZYVL^b|FH38GGhtt@@3Us~lFu6zO1N39c%AkW z;|D`q^vkB4Q4+L~dOsQW^B?Tw#8eBzPhWalC4%0=@xElz!(CG(eGF1hY;fWwgS(*1 zB77d?#QOUBL0e5A13eeF{7fv;ML^<^?{q2r`TBI)-Pd2&zP=hT&u0x*Qf1eihtt_B zA+_nE1SBo@Em$rlCKTPsDP@=I;~rds@(e}eewz&TW}4$F1+w0w^9Ius427#DU;OJC z*mg>8n-$Abc$)3kfd~Z@gs&bI5o2y{E(GE=2uNh)EkrSvogf?7E08@J=tc zm$(y_mOHm-26H>44F_9pW?}Z?pFWLjS{8QpkrhI#PvuKx1fKHoLZS-_hpFx(Rc=K4 z8bzjfHNYQtnbV_UuGl?`deZy?_!hFDhU-Am@9ZzXEdEF*?idOx_xZ&VU{4Fo2BKj6 z!;TY#W?=_8CT=WA;oj6E@xV_!Zh=KGsW zkQS$Ekg~MHt>;zUrKMPwY+yW2I4^WQ_NoqV|3lt8{*T@|SsQ!-VMSmO913(WKb>ou zun)P_^G*>HhtLi``v#&k7<^hGol#%XZzh~s0>(Ajbqx<*p9&#~dQZ$@sSPwMW01Fj z2W$l67qBb{or_8W>j|hGV!-RlLjVS+nGl%04uF*whnV0Bf|$B{Qg0VzIs4_|cCgRL zfq?lDA~p!FEN}u477FZO5H#viLL}qtJ3u6q1s!e|Txe=VrqqNqmq&rn1)3l%J<0h6 z2-_WFh2X}3&qA1L;QvHf-ARn{)VP72CPYa|NkKVRsNo$DP|_n+U9)ieOPs0L7WXM) zdH=g+nzYd+7Z0bH?OiL|Y_oOzmNo@cdOCv;PB#+i! z*%L9lIcczng3m(#_~_7h;H=?$_}hcbN3S6`uRtP6AMy^J1L~R;GA3B__&|mTkBH#V zsd9Z(`$oSd97)B|I_|OE8LFTP;?Z1wS_A5VSBt*&qgvD`4Eh~ljF;2`U|JE%ftdM% z;^N)dO9qa+Q_-_S<;6ogoDV;(Kr%ouTNFra;0pZkFoOaL9CkiFzAoP{Wp?v$Kt(#m z*X5$@;}KZN@y-4ay`F?QQQiH17-MCnQ}O6tZzHRDyZ+NxX|mvQ_{(1l&_k^0iu$0M zp+ZPZJajEQ`4xji9LPUOFt%^N^s4$#*tWD3$rVSd_PnqpqPZNfI?ULCyoY?0oKoO9`2O~w1-d%n**v5qV4)-PfnPG6tJiA`qSRZ>%< zg3J~Wa9Y(L^Mjw4OcwR!r)MDBOu#f)+g;#s000Z70692FHr(n6%{@*5tOr71Uv!9t1IK&>CWI| zQQr6%2UQFHS>&DfVHx?+J}%3Zz1?q1ej*?P*wzcGSqT|8PTuK)#)_&W;JTFx1q^No zx7T=Yb|UMzTp6`8gn!`Iv-!{ipS)kGk_QuIp$|R0-po@@Q&GqBy=7>eZY zS#aIQ+UxYMWz{5qeOC8hJyjCNiajlbU2c?CJfuRr*MVLAI4ejN|MI{0YjUuX7)s&0 zc*t9D4V+~-qIUJsq>Mi6_}}~Qu93(5q9o#$wJ5J7-$)`FKH?#eJ#|v!TQ%3!0at6* zkOpRym6dOe*I@?r1FiTb&Wq#;Rc6jrelsC;+%4nb2|GR^jK*ua@qFMcQ{IL;*?i_J zc1#c1(NMy$N+s7u49bvK^(O1%+kY#uutD2dak*LibceL!?pdtFYRvrMcXx*0s-_^& zDL}?NP>e95yr#5r){KzRhbYR;#J zZi$v->mprUIM+uM;?2JX)<3;x6MwwA6zPmNaDpa!Wvq z{y12FOQ+_s+G6;9a0qEeXJ(>j*E{a~2l_vmCia4&1me$4FjRG%VzUO{+gHGvB39$4 zZU@tSc~)3<1?;#jogIw6j7#|n6a}siTQFO?U^}lB0_@iBr(|GQ%Bck02_=BYjq;Rt zhX0qsPdS~1(Yz77GP6+ffURZFrc(k!6-(z?n+%ZH;7SeP8aV{h7=l<_L3snUN<2Cx z!ux`t9_kDk*$p`T1@238t{HP#3ICVhXijhy_O@BLlQF<}G*Ja1QW_=;DG=Z`el&i3p`2ji2cOa-} z+}gjlUxk82yMW%s@p=fJVs^G{9cc>@E4qGoi{p&Lr9+V=eJ?G8(6YEU79&-#>-RQh zpY$ftgXF-40+!#pui9$j78G@j;D{&zH}&kdw^&g0A?fI|PWAAZD3o{OP<2m(2V@2S z2msA4C?&yPf1>#wA8l6`CJvhFD#Tz6#2F0GW@@q};^(J_ zK~R=h511A6TMUcG30ri40+fGPk^c_H4kQA~!^6X~pagj<0O-X=3aF1Efg!0JoO_YX zt4Kr`%xgB_%VcNK$)k_29P{G%gI8MEy7`)y(O9HBSB$bEPpPdNH*N_g*U~2@s$@I(w1zw;kuqUl6J!8Ipf zJt>3e>;+ZD!_GmVMs|)R@KzsOGU?4q0~>}b5HH5h>7c%70&s%R6s5;U`?G*_Kpuyw z0?X!A#Mc8nP272>?@-~}FAd&-_JYNJ;Ef!v00M$SGVtZgwx0El7oZnF1ZsvwN9k3`9+rIr=PbNWXHq-B+a$&=K zYERffO6B%uaH9=y`%eLmMNJGAnovR;2T^t<5G7knL$XkyQD3qBpfv(+f*5h9OW=yi zH<5xWnv6~E4bY`88KoE)lQ;D5fp-!)>~`}#2B6a;HcDB3ATV0>TtEw&?arWt1U&&q ztHV~)9Qr1(bU|U~d36KHqdr>sy2I!AxQm`Fqa1gPE_kE^nk~Hp66#-ZxMa#n@^C|W z*yGp@DqN(@N5~4e08%^Z!-bJ54_G~VKz;1e7mMed2RNg+R6F!s3K& zqv#>F>O8o`kj(%@UAuMdIZ4SRz_y4z!2b}jpT7YHiJJ5rcyBfqeno)c@K<5Bc4ge; zr2dKRNdMX=2sNO8h8iz=wL$H)hS(5|yV$_B1|~03)QLrvi&ry|Pv&H+B*;!6I4n2c zj+EP#XeT>3(&EfUBCY{V$z|>^jj{ieaQ#!hT^YRX+8&2?IK&Kc5NW#lnVFeO!JUFM zk^r({0206}|9Z4WDA@pKmPTPQt)M4u0b@%tfm~W#7R9=BS%>0pN^2o*Df8mOMc zn+~>g%-IoD1Q12<&ui>lJSvrB z#R8;a=4BSNtBbm>FztWDE>RZ%2U7 z=<8E+-<_g_O4y6GgYvAtOBGasMElAP0?qf#P(ckvZJ7|$sh<%If80sEenPX`Y1-@AGN`{*EtHSpvQ1yx4l>~3UPZZl%igWYxhpFcqViaA$ z0M~T8?E}q4{PgDs5{k*G#FGz8I@jzm0y$%$hf${La>e}G9$xxXfvs87lOS*5V2i+0Q0Of1&o4ep4hGDx&vMCy2OYaVwUKt4 zwRpdI$ygOIX;j8~J_xCQ!E|;jjOpg3Oq)@eS2TijMbO$53cfb&ZL(+J-T?vw3Ll6k z;^Nlls+BW@fBt@5X(Qeiny_%7W_}$?5~$8Kiny06*6;ak!Rx~5>lx|k+o4wrd}VJy zvL|Xvo%@x~4%{w~9UN$y%fJ1siY!Kwz~$l#T3T0zYUA=Yxc;94mkri#d@!M=10@21 zHV68r7~D4QyYGzQa#I9p$sx{j@DuAJOemZf*#7ma{~0y2hD5)GM8R*bzc z>+0RRFNQRM_nr#7%`gV64A@4m6qpau0HcRAAVFUfViRARY@vRPK+Tf~b8u_|%O5*`M4P7F z%7=yzxPSk?==yPJi6D9JS;Wdw60hkYuc3UQVu9RsJAJt$O0$A3cUz%eX2sL@bEYJD z%S$l!DQD2U=4@Gm2J5cq* zFzvrJ z+o&KLHv(PT42c^-tMEF+#&#!yO;kJ`unxCYStR1y0n}*&-GWQ7QxHT1DJb97%=7#W zKPVmQ5K0^n0MdQwy1QnN=L3C+5uu?qPeLT3GD{Kd#XOcZ&~qfYY3AFqv3$#0!hppM z3BQnup&aQov52t5yqR67t7@B?{Gq2)BhIKdORK>2S2&=O6`+3C+5|d9SgQay>*lKU zA-xPwpvR|wZ2{J7w(GmAInpEuE>VPs2U{VSe9~nP3n0RCHD!GV2Md7errak@r-_KF z#vBi&Sc*CA$kqO~#S&G%Zy1{FH@UKl-?o5@oAqh_K`yKKWvq%ro2`DOH{hh#6N(3& z4*F?5pHXS{!d^vm8%iF{QK$l~9_3xde~;MDvou8!PF&SLoEQAvz;{HiK{uHJhx$3| z3^0iFmDzE@aWVkhBh1|l$lRQhpW*#xo&v=KV`4XV6^K|#q&Tq%5sdVf0_s9)AvCQ} zTle`pk~0un!K07t4A2hZGwNC#{D>Lv1(LENd=2;k&z^%G;h=_h`d-irXg7r^H;Z$- z=Y8?RWqjg$*x3heCfPD7z$l2tDSFB4V4uA6{dZ+yy`@vnL0wKJ*2hi&79*`H2+8>OA0!NK-+GE78iJzLEtE!+@ipS#x0w-BP*tdedqe#w&_)wUQkB>uDLP`KwWIZ;+3y+-oi=c{Gr{KoFP z!e5ywZTDN45%X>tCuf-JKg!ICG3u-uAC-N(oCQH9!Qr%*4uAA>)^NuE26vtnZ(1D3 z#r+O&eWw_b3>m(IKSmosbe2q(KxVD@=&Ug#LtkKBEUI83&y>47OmrIDwCUT^({ z@Xmi1+U?xp*vUTb+p-%w*LQp(LhGbSi1Xo9;Fq#Vk_i7XdcbV+w&3O1KiFGcbEpYV zaM!FPuuVoK4VN;+1&Dn=Qam}=;nk+em{#mvA=&pRel8J8xGUyet;hT8AqqQ~= z&FEz;b0;vom`Y)V^LH*-j@4F^4(Gfxp7-DoS2mDXQ5=q(XjDv2BeADrwos-!oa_Aa zlKzqPgF$!d+l@1WVfnED7C|~eK_~6YU#9j}(o9U)4>t7Fb}<%`u&H_(Xqy2u2Tw$pIA9~s zATNEm*LoXDV>0Nav9~+T#I)4Jr@PFgw#tw&@ML6WmOvio+WO3wY&GZd=r6N8d!{FqzZZGEbnl26y!B0t6MTLsEQ{d+-&-5Ay9 zy)*V5Ig6H-g==Hqw#+W{)(hssOaAT14iXQd9(Q~Vca&_no6b{ujdyR9H27jQ$&v%~ zgZ^61tWAK(*vrEA#aCHyI}ji+}gm|2CK~$(3VF_j#W#tG;g= zJIYTb`+B0JZqEQw*n!OP%t>FQ%$8cX9vJZs`Rfz&me&dV%C)`yLcF50;YuXw|3;qI z`0RN=Y0jEkfTaG&3_Z$$$m_k&7vXP$tAPQ^CpSLs*`m*M?i|V!I#}g6zp*l5$Ld|3 zR8o_~>{uvT7TCW!!G7-G<_fXByem^U^B>L~9;;!~MDKt)LrOY6{#068^Y?FOwJjdH zHItQspoQ1YdU6g=9mKxd#gH&PY-S~U3SK- zIMgIf3Ye7m2MGi+$hWvX$(QSPW%ea`J)R0tExF==00O&aQvKhHnCm=!TJ?4Cc2fkV9kl2%G7?&G^GhsG)epKB|!otIj*M` zbi#9&gZJNV#w#*$Z%>ehx!(B`f5Q%V_Rf5!!z(w!@N*kk%zU#rpjhU`#Y7h7D);Xd zoJ*HKA5j&_W175h)0xHjW=}OZ`~z9)s=w|{8Fw2|nj<sZyH(?0$x0+!cfvt!0D6?VTa6xM+9ukg~ zzXy&F4U7j09#sJSx0JV8n9SLyFCgLQ@RVP7Rko|hpalA>kiEPq6Vu;-HLeM1Z&hrd zSN=HKDx+>EKWAFGZw*T=VsZca{MCWtw@7HRX4fVOll%7$xqwO~VDW8Js$&ge6j_%M7k5R;@u;O~*Nikdv7CdPT_2zJ+BWJU7ccF8wtvG-TK z4Bm8SfDfNMPozx4svGd)r%CoN+M>Ptl0Hz&17B{GR{z<9S#{UyPmCZwGqBW z62|>ER@762XE$_iPT$J`HMonuOpBJpD5Z=KXqNxG3@0zJ6pUcM?1o>fbLw~$Ouf>R zY=2i0Y93O(Zrs@Ieg8gtsS^99QzT*HZFSukBCSx2?XJ`j&kk3LL6HRX+1R|x`nHc& ziKQ1J0fR?#VqyYm<8^}8{1VGC?{#l@iV#WILFY5CWCp+1@3A3)Qf3-n0Fy#W+VOoz zm~Z9ie!)y{vpYI)0+#s}7<~|K4rCWrCMIv7U&9-MDAz&CCZYSBrjK9^Z0$knn8VX_ zhwF_`dV{^44-39tm0lw(R@eLXSzQ2W1O{QD8T?|A!0xvI& zP!(Ve8Ev4{u~qsB_S5Lyot@E%2^(3i9_jj5^x6bZmcKsAL!I05g|shI_4vrAl8Jp{tzLYxZk-<5p-4_Et`7EOn#T+md zfwJ;H1gdKWd4d7$Nv}9|eQvdG5A&7=JI-sLr6=bL?)A*@$0jny2gnd9){fEugOBw_ zF7^_zssqYRC!N*t0%)~ka&8w~w5+p>8vE}x!Lyv>pPHe+B$5=T7oIPIThrOusaFkO zLnX8y@2B|HZTa95E5QRh3ih+4*UI6c2xV7Jk)&TjGcOw1K7PyumLU+XC~vb`n{4D? z*_XqkNmYjhE z0P>#X-wkrlfGTI|c8p~)RKQy*D?hKW5=f`kaOU9oK-c#f3W`RP>`KF41~7<1^O`iM z6tHw4e6nUP!-KTOpfW-PCFG%0_RYvq95)SI<=eiqv+0z>a@70OTP6F(NU35;30Ddk zi4{k2DXSjjotISnwOJRzUJs8fcu!xAJP~A}j|8!QgO=C?k4L!Fwh1Tk#I6E97-(C5 z+Lv=4#)}w&KywTPeoEXZ`6V)MlNI?cqccZm3Z*LBt3@_qbTzcjF)7vjpZbCfjI+R_ zAY?JzhR9J;9+F@XN&#yP(mo20oQZzi@PJwsqAH8d06)FE8;Md@YVm6eB8e*h8pYlQ zR;F7}nlmSv9rY1|A@mcDrtqrXRH@HUXMP_1K|4siMb^whZ7vb~W{BU6T! z)%IIdUL2^?@$IqjR6kChVMn$Cs94Z|2u=B8bBZWoaEK$bXRUQVY-JrK;?m7H{`|%D z?N!CJ#QAYjl9rUe>0!IrMF^yz@bRvJub2I_rLHSd|GBfjuoIRWG+3-RGGR=Pj%WK` zJ(pf9o;E@Du-f~+^cgy+?^hfRfc^(`QJl!JI>}`h2lbdTFf8XFUIhW8M zJ3LKxN~_vS>`Ey&Hw5ax^@|lWZetQ8`PKJr6hTL2cA#Q`cg8!oIn{qR7Z!f1)eo9% z`ih0!Ntc|U`9+f~IzUG9Z_i_iKS7gO@wtS)@Zo8CRARMn+z=SV17zIOP9QoKblriE z3lrflI8{CeAa&dWrtpu4)pZruELqtqGxgTE_l2Mtd z>`D}K(nFPWIjj#g_{r4T(on1t#T^YLSzz#~HVp6+)Is@z+$Upv*KK&6*+RzYu z{r>0;!xOeCU)GH*v3RV$fb%*q`hWh$b7bP};dU3b-J*YRHG5M8Z)p@ksVDPaQd!|1 zV(t6mj?&Q0$xPl~j~kVDT=;2fcoRV2|9k~Q&QwDNgze;=zlRNDA5}G;d=Z$$%gVgd z?!O!Hq4PLHwfCkek>j=#W4=x$kUF+q$1z>C{O9(CU%!R}pIiN3!9sV9wdG<1MYf%A zo=EDuVRqlN;N_=#;6HJ2b_hb_c`QH}2Bc~>N;=WxVdU#axEKpHBkTiEvRq?>N%>CY4HFV>F$M^WbcuM7_Z=L#v) z$CPXQyYuplRQq>R(mk-dfCTP8_7M$1ZUMUfZ)qR#oXg_dy@fq*ZLXRAcM`7-qAD*L zO8bnr-42o;*_JY+LtAECDU^p*Z71m8zTWlXTQf*U00aNKQ85|D{??`k{`{KEgdnth zdOha=`C2{i|L4y`J=U-nFtj8#Wd#(hef^UneG$E&imJ&oIwXsW5gVXD5I0{0)$1(V z``{a|rn9lcnJXk#Fp<)n#kkc4^7O~Xe_NYwI#9>+SVfzxSl%CD#3pF}g z)iE;R#OH%>3jsEkb4*Sq00Sq8y@KbzlS!H)=6=iZ=?#k~r`bB+GjjDUNcjgF?aCM4 z+&YdoDiRT70Ar(+(BRMrB2;OALp^5?2L}nbfDwDC9?;02qmRz8MLrU5dQl?EW*@Y> zxJqkx_Tu@(ZR196x|c5W?_<7o?#j8dh=oBmvWBD?0p8{;dLwA8ta=0r*1Jnzu7hfM z5~gm*oOZz3fHY5`W9aX{#K@ySoPdZ_AW1954CE-n(I@06@&ly2-9kg1+;WGL$62uS z7ymWq=Y3)Qyy!VLX-}eJ7q?WbI9wbg0IUPN8$)7CNT*<1-u*qp!hf7VfH;n_K_I1o zt*7zS&<`X-e&{I@Ck|o{8Ryy?6}@g5GIuEP>lUA5o^f_`i*`2bpXdKJ)JfJwsRgdA z1D<_o8-gjj7R({3eMf=+-jV;ami$0pLMUmFG}B9Qh2aT)MLL9pl2+;D$-6eutU1k# z&%{TMWwV60Yd)ct7vH!N6PfK_MXPgi%Q*BrXI&FQs3b^~9e>yJC)OYna!7|X6)isC zM0};tk@YYGRGi(CJlC3^=jEL}YCKl2jyo`d7gN`&vGHjWmjej&?Tvo0Jq& z?3bn+Q0Q6Sa^=$2x`)a0vp++lRTv}K;c(VXS7o2eMV5QnCCj@br1HCO5!yG};NJqC z!_j^Q{G(R*q2iaxauD_Z&O@nX9eZK!KuhJS_+nA|6L>UnXIQC)G?P1!zC6W(@qme= zlP6^PT}U{J#H|XfP>OQyyL`_{c|D4d3aIz~OeX#mia3dj~@J6$f)6Baj|jB#*GyqJiOI;P2445L4fRB?8De9 zs4kq%5Kej34Ks9PoQ61&_;!Q+45J0D0KPEykmMVqBk1<9B~fVz2T-pDrq8@{Ys^FO z^9AjiRDI~>rd#qTddcHVLr)ieUx%UM+wVRFKEa6cuD3D?{ zk_Jy;>+Py9;tQ1L6}WFNX?}BYIS`+5;+qT6okmO z3MfDLXi*LNFHiHsZDbb2nAnwfGie+@AwI|)m^Uz*#{270BOi_r@>By=Wv=@9fHlh@ zrmrSswKlVbjiKZbx{Dic9VTWsQ4ct9U7^){f+sJvoTszqqThKlQ?lgRU*nGQxuNhi z-(`y*?K{qnhLoN92~Mc@HiAhx@cs812T;d@fSSq6x7t`C&BXgBAUCUJAQ~-Y?jH(k zmj3c|7c3e#Np5^oTByNPQZH5b(1@$MlXm(8Q(oeM$4}k_BYV@UeVN6FbTV|fC2OL| z_|A9{4G+X5JHfMBs|2a{aGA}|{LSd4!&~IPJEte~A2)De+x#rQo_s_*`}s|7i)(*9 zgxDGPPEcl_;TY^NB`oUs5Mk8dxI_W9Fb$8#UAR~!4TzpkE$pmZ)*0HT4WHRMV)AeN z*^arz%Hc;X=f{|8+sF~6;dr^2q-EgP#Tl0PzlAl^Ew-Hp(FP|izu_{Z)Z^g3T;!w% zHQs4?>G3*%oXF9%`#?+9qgncn&Tn$(4nt=x4Swgz_g5?g)FSQLcB57T>H&s1xlKLn z{ADF6VofQc**aenJKj8Skb;K3Lh9aeQgYi1`cod2Dx6R34BwSH*H>M>IHCG4gcggu z(e57|`Touq!NdU7g9670_#6O599F*-H0)z&Qdswd$!t7^HS-^J@RqtCO(aWx|9b;9 zR;ZI1Hw6?|*Fij&_SDD2A3{fHX1RR6frUb(yda$33tP)PVR%_%<^yz{aeu*u%6Ll3 zbY<@MzkQ3~1FVErDltv*H*7t<9&TgipYCqM`iKIDq&uRQR6F+WIR3*Pk7@6V+(*GU zPwD^n6{W!1H(?n$r?oa>2k3PpbP?qa9y1|efonKHY*($o`lzb70nWoX>IV?MxR73c zLKFNDoYO46zxrO|PdW@;`JzWl%IwG5E{t7}hPwt};yXJ}Z_&wPJd~+a|17!g^*KnK z0qth{{i1|CqZ1-IBQ$p{RQD|`ED!?A)q!u1!5U!`(qw?eFTkZh6#j@VR?Y8jlFSwl z*T=`_!@q%~4`apjxJA-^QNQeADq4t|9o!zR0NyUPQ>QZb=K)dpA51_L!qvz3VfXpUgY>1co|6N&J(^cb4XvT4f<6Q*a zeO;(LGjHz6X`C3S`VPVdPWdl5-=IeINHXjZLTZ@%Ma;@x3S6IL{k%=7{3TqHR)L9$ zm)EF@%6PALQp?(~`}Y1$7qzG(C#MZx{MhVqV`wgnwJ_C0ntx#4mEz#_8+&TaC2l+z zeSJhUENOB&AUpa)gt%R$fx?N0dmkfC5c3YaHR4u7Du}4+aq<-~P=+uX^5#hJm(Jw5 zUDbJa=>VKE(3N&9`vtavgR;>R>zkQQDtGk=!l`{x_k`V~M!E64s!}Y?V;h4V_OS&| zCEUA5P3JDs*e>Y?L7hfNmFRUAuB%GeVg27;hVMz-K$eh(KV`X8a~Ku##MYe(-JDDs zFWgeB_Z_(JbaAS3LHaN-$NFZ{7!GY^cc*h&Fi~ZTQS5M08yDBTaJ`x7U?)osS3KBB zD>-ye$UVNM0;)_6$JBLQ-X5zSYihKJ&^#MFB(U!>cB8|Mm9$mn_vFu1<4r>kX&=e#Emm@+;?bq(fr-wFUxe`%b%Q0acDei zu2dgg44U-ZI;L=w=+jE&Gvn_5P49dq^Si(cI}R6@9=AxWW9^l4>eK5$1uD7cB_SKv z&!u$le#X@lMz!z#p~t+-@As)@1sLSZ4HGR=ENIeX<<=f_xoxoa+iz=CwtLyzox4$+ z+OJy4K+}rFJAT%rdeY7BXA0|*2-zL=__Dih*rUq*|J^p_n9|oaF{&ym7n@QIp=#Lx zxWy)SSdtKi78j(#fB7_ygY0MzA~!P&%XK)8LeTyQG)iIzN5jj(9Qpys;VIZb5#0*> z2?eZewd~+Pbk(4+F}QQiDb?Bm20$wNE=jG@Z(Yq)b1gN5oFXduecv`vqs>(Yh$-rf znKn{j&^1hMe zr6gMggut_mijaWV^QZq%yT9uk1u!0a`@La&>6kG^7}Nl8$&|DtoP)XquTfz z>-uDzN4Qh+rZyp%6CF5`n0NU^I|CPIeYfBI@QaIfT1S>ItNG$vep;s*->Op~>x@o9U+I!_`OCp3Ku5IZy4xMq>6 zhY}GfgrN|*x|QTe+O(?R;DsvqttvYOC^$Fp$BytRBzA;-3bf?&gM~OQPEc_+(&bxI zdwa(2)?R;>U5C-H&#Ac3;BoB@@<;+(aMBUt2og}v67gmKE?6iL)i_droXT<&r=19c z2V_e;jj)pY>yQ^wT|~Ak6;oASE~wfl3}f!8>+s(Q{ZMk*?}m`!l<}Cwx$~1OF&}zu zH>5Lt8=_t4h3nX(zk|Ragq~n$+tVfrFl=!j+AV3-R0z&F@hzpLqvI774FOZAKGpCL z#L8jYlNdqArd7i7*bE>oF(fyRSd6o=m4YpG9+qwwege_@0UEP&yQ?5b?)e{I97g*A z^l*q7iKqhZG@41@aiHxf<@UMHQ`p}u{!D0oKNs3z#x_Ybn-y;?wb1M@2sr+_sO0s4 zGB^w-$3o_(lC;wW$jwJEGOABFMod{b{fn^n98JC#;R{m0+W2OiH_gxik(^ILA`DQXF71tZVR(7LS6Z&9 zUpDpM_H36{FKp<&o|_i4w#4bJffD)P;R`Hpuxs=?zdZ?P{4+lO7?gI1sqa9qK~@d; zatCpRaIYjZ?7f#h6Epqs&xe@WCK>6$tD}gD8yy|Bl^Qt$8Tm2z_7Y?HVa<(%V-FTO zy6-W+nusGn>H=8qFvyO0@j{Jm7kr+O&Rsz#TKztlF8I}KkB^Cqllfe z{QmUuoAVp)B;591%_*>_;+Jli-BFFOUGJ#VvtE}3drJ+PXJm8z57ujzo>Za|ZaFwe z*lyS+Y0`bCM6RG8uUF+=VU?XaH&vbU4^9?d zEp^(Yar)WR?}}B}634~A|$ zd>f+v`>#{`Idhx<>tYU%Mcm`~?<D1hz?0AwvD)^$a4L{)(4%gVNd|-O!J!m{c^DA!G$N4ro@Cw#af1~F zSpHW^a)4E=DMazdJ!RYeVdFKpc$f$sAwd)#tdd^lbMJpI6@C@$=1kfUlI%15^uh=yx zW$!-IrFqjkJX`}yPYO;6lfVliH=sax3y%OuSuLQ})S3G~n10boXbVuNMN0-F3^)H3 zIiE`W6o?^BmQ7pQ2BO82#(6KyW#(=bZS|AGkxet?owc8I4*k%gjB(o5ccNEj*cNSh zCg&-4Wt~eS_JYS^-WPXdw%ByTQ>}yi;NNp!%DibW6PIiEm^gu3Q3m%-m_5@GKW-@4 zOmRX%9yS4Js5$t~ty@N53~toiLqt#D>F*(%E*8wzGe|oiK`I0za|iPJh+bH}5J(Kc_5m_hatn!92zc4B1XxX4g^&f= zGAeT6fLe8NjxA6w%8T2XMIotoCSr9s%~>0gy1knANz6QeGpxcwVEfg5mt`8>{Hjpq8_ zgJJ%!zJXwhIFT?w|IZI(rw>slcch7gIaWb+R2X6=vtPYQE(;KWsAe94#_nok(Bm;m z$X5VHhH0#BUQatwXj0jRicXoz&f!Z5&<4V@PfNShiPjfvY#*W`lwHOv5+O~)j$+~D zL?L@v{P&5Zi3GmlThkJw(?ZlZ`W#N&2Moe&zYS+`gs}E16YDwRZwYfa9Ays6HEq}f z?bqT@rTFwdd8oDSY@s7Ci2Y9dedKA^5ih{?mg3fZYjhZ36;{aAl^1EDtkv-B=(xpo z=Ym{iCJVs?^hl}&%n!jS?Mi$iOyW^n#mG6=Ms365RnfoFDLro|23khO_*=o{O0)DM zw)c+H32Si}!7^#Y-?$qTM?jT$;WExpx|8%~;Q<0M8deMB&0Pa`|D=Ed;=v`IF6Z87 zai16+sbVz|_>rOHBAhs+xaHuGk1i~HA6BkW-Ei3r51E)fk{aO+lN)H9YJG2Rh7IOW zd=e=lc}0K0c+MR$4NR|Ep_C5?$s)gyn^|HHqN@p(82~@XSSZc?3GV9Z^PLVvhWDp_ zTMKApj~;r_A~-k^`7ZVuzDyF}0CC9gVBHJwj0$w`U)XyjwZ@o%EfBgwsDo)2+n~7G z3Y%`6Cu2ZGWR{<&eMYHEq}1A}#fR;hFF56;H|*N$d_fN%7$yR62Kth5#Fx(?Todsh zf(NYs)co!J4Nt;_6&-cdg=Y>Cfs)pHJ8m<16N8jTE}Zg{tZcdCqQ}X*I7?^vQ#NI) zwt_5e`(d0KIUuLM;WRm)fo4g9Km82i#@c*>2xeCTaLUi{OhZvI*2J@%N%kC)`7`bX zV3bI-O9Po6x!@<1ucl@6v9gn+dP&Dz!8clfv* z_=4imAs|p8bQc-D&6Z7QHOwgHDY>|K=C|B7;GhpuI^(9_u>vGjq3rtRdjJhmAZ`Zd z;rayh#u)#JkFJP%KvqCZ7^iven$CgP_4C&a3l9$aV*BCr$Y?73-Xi!0865m3TF1EI zHwrz=o)YqsA(-GipY~5@hOyYI7MlYnS1z*x(P@>lcOsUyoZZ>h)PX!$m?%(^P9kDg zCH@rUg{-42woiqHoLOP9`Pab>pymj-E`$thz^j%4QnX;|qrL)mRZ)Mdb8$9ztgnAk zl1VJvd+#^*rj>^ghucD;QqO7#^rw$CWl87Nh;eR-q5c(}L%K-J6=xjw8UlJExhNt! zwuGpl7^xSvLoMt};vZddU2Z1vx`v#|I;WOl-69Ecj5@64;I`C*#!J@VotrleKR-G^ zgin})WGaCiG*J3Gh$4EHb0!DA&vm?gw!B|esMR$%Nk0U2?r!fz57=MrX4qabLQ8Z% zXyZWc8Al{xKepiwK4mzm1st0$`SoHlv$yWRs6{g^3G3WgTNe?3<;P1|q0gQeO0R?d zi5)cqcgXP10!4pA!VX%Ai(2VA9e1(}7nkat4~~sp7yYS}+qv>C62HD}qDIev7bF8R z;7KfhW)()Rjq~uLyrsN|X!~ElM=#+8l8cMF8~Zv#w>yvjN-mW~^B2cPdoPEP?zxk1 zUXlhp`prdnWO({$Tye@?>0ejgjvWS;Bbkx4{B63}K%GbLjeG0Y?+&gy)u$rIKkxgv z&a&{(p0g>$&oZqP?+`Ai(KR{>ZM|vQ-u#n~55hR`%kqKs>1{zrV!-g=hO>E0Znw4a zNNrIek@zNMa(sLn;J8DfhmO9{%Rx(s1G1i#gJwfCa-lDVQ0am%IC zkAnyriDDm3GEp*HVF512hA1BPS~cB;QiT;0z0PRt|5d#nHzlT6oZ-onC$RJD0+B@f zeOoQRQAqLhbknHsNwl_dPRlK?6%l)&=V%8YW7~|@2NBCaGA!f0C{F?Sl;ky}ip6zc z$(6vZfH(m!rw(h%zpN9_Ul+3=nD<(T#%0KP$LZSTeG{l__wRW<+**t<4~^?`~vdpdshCB&|DuDL-dg>zW(rLjrTLNulO|eMuiobC!<=UiD5K<7eRf6^V0~ zUXGN}TIS^K=VG&8PdN^BG*-GpP|Pxmp{ zz>@w^Wh3D!HlRiTN5{WOGs$4N_01{Ay1 z9lxW3X0L9LA}Eo3SH$8fStP;$jngbuj~}o691#UA*?6toXzR*Lia}O@wFhBWhskm5 ze@|(q!0@0hG3(sm+O~2Zjn3AaB>40wX5ABq#ohh3am>6^MfT?U``H7D{pIjOLFepp zL-H|vM7=^Du7Bs}ObtK3x)u&825-vd53iR`zb+PmlAA4TAk`4>XMo$klXtdhLdi*J zX#Yp@e>W(;^j6PxJ=Y87nysl5S|<~oYcFWg@hv1Tn@s;dR6k_p*f*9Rui+ffG{p<1 zuk2`93IBl|J-!D(u8ft5#ciqrVH-3g3d9SvZy>3e5MA#`h11E=y}o7TPXAf|z&eQ> z$`7=K*+D3y{%?L{ZHq`}zgy&w#{6tUkoY85q6A;vr_f!;k(2gdhDQSI1CaisnAqdl zGoVRU$D2%RI<)~S=ybiAw?sZ6=X7AtPId1r1t<~qJ;o0sp;k{$B%fEHEizW#l4_{t zwi2~ew)6tJCCmVnEhR>8#a+fO!n&M639u5eY$7E#G`VQYva+xgli={y$Zt6+Rk&jX zl^vR@ShduJ5t(6Y@I}JR(U&ZGVBiI_D~FSz|>)! z-hC(U7x z!O-gF%D0dUc@scuJsiHqFt+2%PtpL!N~}YS-(y}2>Zoy6-V@NVK(E+X<}}!_4q*!| z&cWqf_lNrE$91D_$5wX9PrCsTTA1=%xN9cD7z~+HL}{rqoM!5qF-Z?D%iWm%wQ^o% zaI;HI&ZcB{-dOp%e9`#2kKi2v8$DJouIL6$baE*`akG+|Lh6Xq<&@k`TC% zcG{fpD(LcGIjk2+GK8Q*i5Un#%jYeI ze0qi~$Fhi3=#ka-m8`)b98n()7FK^lagayeW#YKq1i%I!Q|dV7F13G<0V&oDdRLS* zP^xw<-vEc7p^AncT+7>+Ig5M|%pu~GwBm#|_LyZ7PGscyf?KyjVq;H_!J&1GH$`XzEtSOSK#2022}w6zq{Ri)O2R76AFKUmI*SI6z({I z$bD|$x>(VbLQA0^vwFa0c3_l8>$QRdYr57L#{-&LqrYEik-G0k8jMjtNSJecc@UO; z?-C{eIK{ZB9(2NE4F!_&6@!Qps~_CGtAsH`#O@ymEtY~G@})n3R8E853oDYy*Rwc> zj^atq06U>&WPFagiyUnTDSMoT&L@@%gAI6c(r_j}7S-3i<&;8YN(7FruueTV`HW91 zpzGwD*W%fl2rY6>Lxm;|wGKf{3NzP09sUY}HKx!W?<~%oJ90F56B+16P8am_lhXG# zf|UgKF)6qvf`7vUf0V?(+OIVcc6#qc0$68KRe$t3bBU$YnmA>5e3Fg)QJ1fO0P#nG z5$>_9Bzx{r{m=I_sdGR@qA$(YXAeK}Lii6+F8ikO(^HT@Lk!#Glj`0YfJu(fi!)3C(WGVWD58M;la;?0Jqv&CnLjRkk8x6Kf zW^0&{94Sk@z0~+pJ{K6K?z~^RO`EK!UN(UO;U1#zzZkdQeHDz4in(etFlegb%>`m~ z@*7RYuf{~;4}5R%{TJ@X$-{4dIVTt7-;EO%pHrG1|2!oEaxF={gj*cTzHe*2tD7v| zg!!ZtxRK6H@y&6vBW~C?j7q?z(Gk7aTO9=@`x;am*ru-Vs~~C3=^jKoLF@QcE7D#o9UHcdbdKOC>?#?+P?7P>UAGYsNUETV73$f1l!pgND`6Vm|?ai!Y%f|bK z-Z1w0xAGcdL@yH#kN6TV&`5Ia8(CRdR}-_<11OH8 zOPBiB^(FluByeq{e>TzqDqeQuU5B*-wsx`Q=RQ9vJ;Haf_A`gUGa=fF{73O^VH&?z z@PB?qM5)*1-I3`N3_{zDRVu2^F{X5~OX+X&ALfet5kmiRK^qg3DOyAsae_RH9mrh05fYlQAFndEyVh>J@(%Br@w~8u;{yP_f)=KBPDW_7+m-eRy_GSv46H( zG!CidjDC|Xw9evqAaH)y88iXOcIT-J_d)=G>ho-}FM5S{N+Cj3{e+Mg~%&8JSL@L)KA$1>_8 zRx0>h+yPG_0kh61_Cd-22Yor>gSO+wbHz3i6}UtGy_O?Xi~ zXuo00;RDb|qF4VRFq|VQbDWbtGrr?=82=&f)B-7{y}POo1y`ODZL+V)%T0P<;(1?E zxZU$s`E!?xoUVP6e!9mwiJo5DzBLR1julNjX!(Btt!j>5JPXK$nt)(ZF-hnN%*@S^ zLAxP5it50PhX0+hJ^aav{wTt(plEDBjz(e8uOEt9E8itq=oq_K+tn}&t~+nLXVwH5 zKWx{(9T0u5FK!e6`1ptMN)H3EJ#B*zI{#9{%cqC&wDK8ttibLrOAFJ4&j%FaM9W4) zK<>mz6ofQ@ZugvOR{%5y~;%B^jB=vRvhZkKaR@P zIE@;qZ$=<;-{(x74zo)HfK>5u?6BhgD!IkDw!!=636+Y^JpF^m>ZnbBe@zg+uS%7@ z^S2>!SbH@~cTNHd`+kZwWy3g>^F-n3%GfJXG+|7sDfsI;R8o zsLc)T;KHC8>pcA&UisDp`OTJor$5P!CNtVmXC32&iEEm}ujAx+MGhDR);?mDDZIxZ zBH%4~y+-PxCC*z9w0qTX4a7YhbH8E68bs%K{BA?o-^Hsn|i8>NgpxeQKa^s(c3hSQEjpz zA1?H9hvaPEdpG|?)t6^B7IbscOB2)oTN0!mKW7H{6jF&$2Eu~&|0s5Yu%J(K^(#=$ z->|*hJTv2hv8t@OnUFuR3!hhljKp@^MI0Fw)`_C}orx#PXDudn*Om&H^-qsab}Y(f zjoXDgd=t@4Q1X;smKbIyCa9+N^Eb(KzdFMD&TXB}JdaE3{CG7s#ZZR)=Zp?irKMaOXR$c86 zA7jOEpMXu*7|*I@hRlfRwAJ|;ZoZF+#0r1EH)i3a$)7;xf|Fp)VnNO7ViTNRX*kW9_at$(v7VDV<87_k9)G z&T{a?Xa!$X)ZCo$Cq84hwNdv*=l)h!Rm=3Xy9(%s<8bwj!MjI9oZq|6#u+b-(L1WVydK`R2qFF!>Ee#qZFDaRu2hdJT#(6 zlS*id-|LCC7O@W;0!uvS2Ndpqw#kyyuy(X5nyLD+1|5r@-ro0{pK6nC!ihtg2X+TJ z$8_Ate#(8UCgroOWYN9+FM8fy9A#61g;ttrOAnUcxL0gqhldkuOzWwJUanPId6b@; zY=m0>q|>8`(#k?j`u>O&*USi;&gGUN=a78FN1EvH^P9ZGc%z>dM>-XG&AXWhdhL6{ zx_g_o{Cd-g#{_%aYdcrqQ#VRssj017y*0a6aec$<1Mvgk>)LyI*X^GMNq{GN3l5+b zIO8Jhz$P*=k3R{_`6+Uo;`9(*kx{C|5q7qMsCKwL^g01$%ITB+8YQp_Mn-iW601W ztBp5sF)?zv8fFs*^jHi=u16});}n{Q=YbKv!mCv_#2+@enJ0^b`PFgdVNn!+T#ucn39|84+4Pp?WM*IY8W_qT$HDj|E|Y(IvQn_{bJ zqVKRgxO?h%nq_1vj9EBlW2@A*qu<@I*zrW}99BaA&h?7P_$^Mr08U*F7(;c^{W zDcNUs2-FB$?IpxKm9NG%K1&N*raVqHn57lo6YMPhDOJfrg*+N{-{+3S? zhtdQ3O(G01;s(!@Qtu|B6SIum%bdd=nG2jeW&HN?>8S?`vObo6?E7f9@S^ti0Q+;_ zynto;!-?_`zts5ysoGV|?=`MPE%_I!+*yVJ|C)1BN8d3SJJ5Pcb4cOD&AhHF*VPTJ zvu#W`-fV0S4OB8xgSr09rl1&kzNTs-#rT3J!Op&3x@4B`wA9@6PrL@<46mGY_ z&eRp+z_+j9V8IWnGyje_F3euwY}vjlT|1rj;j%SAX?i`Q%!927L!y=wOaW07kIPN! zJ`ynUsu?G>+#S>LqF%m;viO8gy)VR{Ur6OC*z)71L2$panhF=bMaD9}(4}^yuCwar zA$uEb-NaM1Zfmw}mgnH;u#;9fant&Mb-#&7HSss*e2sSosuL7WxoA&HWyma_ z5}V>*H|1RDipXkh5z8*mVZHuw;{M&uj09q#K&t@f{BksUw@YZ6Z;!LR{KnajHp2#^ zvNd0;rX_3}l$!l!#ycbU@>^Z6&l|b>Ir`Gf*bN7&&zmWm%LdFzY4+DEEwACS`&Sth za={Q0vYc4wFQk9N>w!CjaDe>M9W*2y*=>@p5psr#WJCeq^rT?0J@n`S}Uk~XS|`gB&IsE;dj$T#kx@&zCJkb;#Q>6 zNpu9sbrwpIuo5}Na67vuhdCYLaxZO81IuEr%uQ!W?rkC|Q&;ILqnpLv`#cSMzAQSl zkQU4=Nt8O^bXr^Yl6jx#+RgrM%TL1oL6kxDbsiQMTVq}s&1l{D2OEXW_8gWxy5Z;p zTKX}=f7c=%H#~P)9#rNqg?y+wV1qaH(VVNd^F)1;+OJ7k?%;L06o9>GZa_c#NiF1wE<502aeQI21{dy*JpYJ_Y|3ky>i4T(Zuln~FzeXo zd)|z3_*%+?MSazp<;~Zus>l#u>)#V!)!gh`R6YCI@z(OTvG@qO#jwv=eO=GM*Egd! z{Bu7{<1k&iMNf7swOG()`eVb(0X8GO&wK1w+sS)p8XD}W^d`3WUV8WhM}o1VGy?)_ zjin{y+d*Y@zr_Qa4`sU>F+HCQMmNEA>p_c=<&Piuam(-*YwGOP>8VrB#orPOtIGG- z#~MpU2g84WvUU?S$X;bts@{@NVoV$#VqBc=KBjNr7+CUhcEi8A(Jf_dtnT8^k5UPQ zv#dGo?bZjC&ABQC#`rdd@~nFqvs6VJQVuQ&?Pu3C-nnDP*cH;w5N%+Qi?>#+)wN`D zBuB>*qF(F%-4TWtisjeGzEfU4(ta_%@#W7tWOzhdchZ&Q3tI+r2w6ZUYMz&ir*yOF z;yjzO*e)C3~E^ckE_+oR+&-op=I4q`?bh<_ z4{>KK!_J+ocIRSs#a7Z@!*AoMZ^lYa_E=Vf^J>#Srwzwn-%ZU-u1Q*_ z_m7t$r`?h0f!^`%%+YNBaktQ0YBPTU*+Si>y^Y&tcbn@u%Vz$aKIvPpv-}3?N2O#3 zH}u5f5FPE*yS~2d=gR9@%JV-zGk<+~7pn-_N$+7u* z9~LDgF}`*4zU-GDV%=g_K6C!f#s+3Z+aQ`JtUF11rebN#nL50ArdQ*ln5gBbE3@8T z_8H4jGm;zO*yUn;Ha_s`P$|F3+5S7h3>|N1_h)<9xqlEbV^ZF6I+R3**T+t65nk9n zKd-GFUBM8(<;It3A8lLGKHC&lVmQ{;ZlZ7C85kd8uGK&KQz6n_VCtgIl2c@j4&9Cu z%U(&Z4_7MmvG~4QFtw&UMWTx}j-<;^!1qADNdcV9@brFIpub!l-BwgTbLK;S?|PcB ziCvR-`kNzi1*a~0q^K+ehYFCHYM~7si5f9`=#SG?How2^ll0T)k<%9Xq@C&Kg(rtw zH~_S+Wg^1>Y7U3i)OHOEx;PFShDp%jy#_Z_?8hESi;kc;{*~CTf8H6UJ{V`_BCeC45@PWZK9e8iuM-A4AfO=!*Qx^ko5hn<^q zjP-lH&S zn=Ng@nUu$CRjVF5E8sC4ySbsO{fd|OVr<8+!TH6pUFSWx*b*KD2M^FR<})$x=TcP1 zLS#4k`I{SyDDv63pnY~9ZQ$BTvJZ1Q=WD5EUEKwgLLF7e!|yfeJ9vK9cfviMg42sW z*?micJk33Kjy*GOk&CNlF0~~fu6PcqRrX|j570*PanN9Hz2XQbPgTm1E5&s!T|WKF zn*zv^t~q>jLqoyYDVH|-)M+7=^STYazxveJ81mkZ6a9%rFM|Hp&?_Y9^DsWcx@Ko4 zfhotqj>h9d^_IYs3HBuH*Xs@Gt<4dv+R_*6ijZ$~Hz`Dv#yHs#Q)p<*+NGxI8D9q6e zl2HuNN~_PWu2zGYBJ{%D#4`i9Ugwh_(aSw+4=d|X=ayWYtxI&l@j4Y-`}(tDg5!>9 zwV@)M0nIy)>hP1W_3(y4t>})FpBwCIyp?QCbc9jJVC345A3tn{wY9YgRTcbwxNera z#s93UyL0z$W%!A689{-2%|A-g7!1i5aal5j-;FSM?5%cZ?Z`vUsx@sT_jlaYJV9@Z zcbC;=PWE~~u-M&FsNd=`uJSZW!Bci=?&s2{s;ccmLWEpylif+3wHxHKdHipMq`ukj zDPAAJ7ZzG*F7RH`j@4C8*!ec1$#pbi5Ava~#c| ze#~5^rG*bd)veFQ&i+0)2a|o;%7|5DFnfHUimAhEeiE*X7xQlk7BrVsSS2OZSzWK9 z$>@t#3^84_4cL?5sLOO@SN*$7$F2veoVm|NlJn%J`|ey=VqwT({@_vPspn98740+7 ztIsy~{rdG0$~_s68M_<%e?kbe5tGKTbbus(#Y7v?_x-#!`Ln%rsw`%;F%_?~+R2o- z)rCNy2~*tlH@|ssFeteb=mX}IO%Hqz@?M;XU&o#lxK`SnPB=*MV;WERC!sXfEkkt# zsgV^gvyLu7m};Nf<})oR5N4c_6I;?^WYq9EWZuR@_`>#8o96ff_tKL9v3K7T%XvAb zh5`!zr%vVfY9%=BTXKR=JoGgXpr>Y##+*pd<}vo<%Kml`#v%9F2MmUrOkx7N-e_x)U)l7gViN!^Gww87$6oJp^6tAELh4~p9h zB3s&j`ak&LJIrz8x<}s3u-}Dj*DJ`s|7M*|kbGE{%Y);qR{Y|i+*m`?KYn_p42DgS z;2xtQY5rb>d%@SK)%j&Vz9>l zsKEH=cM9mpfAcZJvvtwYr}J=S;@0bIS3oqK=bLFTl+CI*J;7-ak-spdi`90_%S-$4 zDCWIgg9xBNzfTwRb-#h1pe^+Py~fPHobQ_nfx_SLEa!433gg+qOX_O?Kdl!0v2z#n zlsA1#n>kZ(-v4CYdPUoj*!u9Fw}*TS?%23150p&6LNM^1*<3`)AV;B>SqG6uP6e2ZR1K zVw{vKrpL_oKfMFp39qp5Z8D0K4DQ0b&4#%T#C06h5Ih4k^~xaR{d(Q=d|@cU}ysWDa8 z!mnOAuC|FPH{C6WyFrHeMsZD41k zF-BA<%yg{ADsa{8Ys>eP!@>0wWZ3&t-q^@CcWz)r4HHv}_z1KYjzSKXB%|%6xxu13 zA=L&#i9FB;^9@OX?-=|I?}q7?cAhoo-MURAe)~&TzJy&z7c;W zA;V%f?G5udSN~uxDTksp9Aoev0QY}wA?fi$v0W&V)mn_uIOx%iGc*A1Az;36!)|TZ zpHWv?hc&k*=Ui8_>&1I5T|3Rht7Di#^H6e}HQ}U=ii)Cu^KXtqr!50xzzuM%qRCVI z-To6m!&nX|I^$!QXl#}@_;C&YVbpiK~P*34<#(51rL3_-j$q5F`E|&7?h>=dd zyU|vWJHFpM{say0i+{vIslfjJD0P?V?zek(CN-3w7*71*#%rE*n2!Pe`!kINi)#wu ziPd|YouU=iar1qOONQv_S^dnF=%1u=`TM|&GospuwWYFI z&5YAg@wPYT2M>9U=)E$~R@nw)Yw8b*Ijs{{-u9((OX&|LD=y zw6wHcx9K54js8VJjQF#Rf{4s@aenrlWn%~?^WK8`o?_Ws9TV>~b^4T)ABrK1AFn>x znC7$nml-`@-#1&$Hy^E@I7TYA@viFM!mKp?N$LGzs`XqOl}zqJrcVo8iauWVTvBm> zA*F&E&-ybd=am!Gf>by$?#L%OS=h+f80}3JzeO_K2Hhmg15Sd4GTB>n^z^RX!BQ9? z=W@s&OWV(|Fl4$jTJgX!?IPKOxrNnPk0)jUkvvSU99YA~#�Fe!D0Zyx)*JZyLNR zw@u$7jkIe?AJ3*mlAVLYvF&PO&c{C~-V0UdddA=dVhjfIn^nO~@?#WadJJZST)lNl zSsvE@^%FrCRB0*&sZn6;c-v^(a8CIVQ``1czqf4cP-9eHZZ0Xsaa1ED%mx*9t zkv#9T^Q^^K!OxP#Un9deQUZeFR~KBYE9HOP4sK!%B#*@Fgt)iBDI_5=5yBTtOkEx< z1f3STfrX@8`7aF*74~OL z8}rNX)jx|LI3MyluB{uLDc)_`@9fvt6=r5w>`QBYZcitkLN|Vr(tesqkS(h7{m{%a z7%Dz48|L<&&i!ez@wy6YmmO!!r(IWs_PX?_g>@)v8s^nnNE`_GELpp8)w7#Efmn0Y z>sf~aet3+~&8Yw8s40sJKfgG!Gh#%IH|3MMORD0Txx;%rm2R7{2kw2u9x9GBloUM$ z^@}j&%d+3+I`p=jB=R7v$qs59`ZqmlU@-^Y8VES778VxfSG_`?^UBG!eBHKp9lsyd z{Ftz{lV?YRUaN($Yxq*gcJWP>kq!o({_W;@=eS@fM63hm1o~#e=tA)`h>Eb*62#i|Kvq2ba!!R1}OECa<== z!Y;#xI_>qR-j3Xb-^p_Kj_HTSDjCpsyS+2LJUUisn>zTSskcMfiTUUrx?|O;VfQS4 zT{7EtY@?`{l<4Cj9mKH_tp74ZH>-wS}k5P zFXeN}^TlG;oxO)8qifEoeMY5RR&*s)xP;hX(fDR_qn=~1u7SnheoO};BaT$#jXZQjGN$GlUrJ>NcC z$l|V+$@We)i4MEV`MSYA*Nxosy4Ftql|7^J$jY*~p3kR1Bgkb?j5qkj;rRwm%%P+} z%lgH;f_I+CP%>nHqDqqS{CDMN1(M{}e;gt@RE4GwE@%$Bz6YzrH17t;?BZa={ZG;) z?h~XkarQ2yK^wh$frBm2jAf(()BL@YRy&E9SWlHzS$WsQcMUV}Ihx@S3#D%a*=s}c)RFrYdy{gg1FI-z&II(&N!gEJuO zsUrlK-A2BHky-|x_sLJvoAy;RTrXulqgRSy?eDXIHN*My?T zYnSv6-nj3)EYuyXboNmASYhxhSbGxSJBDo8e_T(G8{^uLRTNr#=;&<1be2B2mTkFp zUisnuu!A!rv`v2^+}*){_{FupzGx5edb@%8_E_bPt-D2j-se=Qv|F53HZ#l%TDN$5 zI*?+~?U(D7I}BT8KQTG(5yNMTXa#|~GCNCrrU>(x47nxlV5oko;KeyRXs~n~UI6l& zp9JpJO7yz^G3!wCVwz+SlbS2bkv3Yh6Z9LHEkys^vU(G>aNYB8~eA8Z!Fi~$FZ+kA=AUS&$_ zQ%^;x0PTHKejc=Y4cpk zccP$^QXx41_NC(mmn5fRO8>-Ta8jHJ`gv4yu6r^J_I{-)y=3MSxK|DzqcR8tb~OHa zO-fFd^TyR$d9;t44G$|nR76H-I-Yob%9zFVuv77kg{XR|hpIG^)hDcGhu5lRZ4-{& zXP4Q+e1z&>{l3OVezK2k5x{+*T?Y~p$tpXC>INKV4>(ou~7v?m+TUkPm(oUBVV zVN4j>X}QO}q^CUg@QV4mzi8~ zd-Ai8xBj0rck1tR=l^P@K5$>X`3Hwt{<__( z$pTkdt&<~|(|phat>9|y>|oM*yXW;~L*{m2RffH-PQI}vl{IXEb_xw=J<(kyR_7KM z((cG*sjBIXwkr&%NB*I5$r@J!J91t1e}VbIYH5b-?t&KD$&9-fE+tJ*xJI40!YI@d zk~&}S6Jos})D{9r{vjFUoA7tWm038C=T=%(^L3++%EoO@VWgQQLlf~a{?$7jZ_^>h zr(w zujajldSTNh*D4Xb{Bh~XMN1>-9Qc=lPtE=i{-orLZ%qG;i^FFJpbt62*avtaI1t#?RgVNow$}oOf zjN-wxV=unCUOQ!%o{28r8$q&JKXy4CXp2i5S4)d(UE+Jlgv4@qwFj@LO(15*kq@sz z&$6`ai>jTaK>FsW+a)s|;eWgx*Y1A3_QkfL9!sl>_EO<MTFVQyRC?(%QAR9#8Thx@EtFKWKj*IVEcq=gvn#?w`p4FYH+-Z~Y zX;u-L)GB3LUmdF5OBxnQk|=f0RK|-@uM7V(aLF~ZXSEBR>B+>iJToB=edfR0uG{jz zHWt@UlXkgHVM=qFr{FuEUmYF&qS0e(sji{`!Vt!~g|W5SX-|k2Sy`a~KxcR>=H_*o zzwY#_^m-KCBzyCjNs)|PBTk0>FX{e+6P%tKzgjY9T%q#_@W%!ZE;yxTFr_KU2K^MT*gf%yDHskGu?1oq?+|0!{kl+ z6xZhjGiy$^V(hJNi)pixQtZZLD~{bNj-D@mXl%pPeFiv$22PD0yw#NE#DkY5WVfJm zFOurED|}u_JSwj1%`bHB;hDPe(8I!HrwTx+ z7hH*fcw$$({QZT(0n!>iunT3c$=Dxg4JXxC^6s%h!@TjN|F#z|tI{7Vb!hw=*4P+4 zv{g-&ZA2ybG!AGNDNr2MJR#`0xMpG2cxJle>%CTP8kNiBJ|w`c>}1OU6VWyAwbx!D z_D1`OsgwRjU#@5jZmzytc!F-AjWOs9^*Yd|{UN3O*vKTQSx2>2bbqi|Gq&Y2?h>zb)HnQEw@bhDmd>FZ(=mZ9; zxxsUHVJ7V7)OT|wM7eH^Y@^@p7MU5tFYGNC>~*}A`=+WtV3-9@FJN2ZULKG3r1J+S z>#%+<@)9O*I@eKrmCRL>hLZqw*(gs68Yu0DbWW>mTDM5r*2v6jE(9&r<3vJrfk0qh zo|Fxcr_%B`B^7h-<%N`2Q86NNT+wa&VsChH-JrGkdOZvSy5a)Fms|HfE{>WhF4_8f z_>gtk8|z{n{aZIw7rFa&iFtZ4&7IM8oV8)o;Rti{gA38$cHhL|%(?GDbEeZ6AG>^tuKB%rExGKJv!L};tR0Q?yFF-FfKQeZ3D>BT_{q)J%D(q+ZuTsyjoH=Y z!u7N&oo6%MAoC!(AE4isq7P_31J@fWM4F8&&)9`q8 z{C!N#O~h8|pvq3Yf%iJn2HLWOA$(=`U-BH;IMb&2-Pa zVxw6$s{h9Dp(>xn{}THZA+(O*Ws<%2Vi5zXuYKK{rBm?j{wZBHo|zj~*zGE=>&hC{ z)z_oNJxDB^boMVv-I=!&AJd{TvpKu@?U}!P#i_eqke4qyZg1>k7dvXy3v}S#95lT@$m5J`;n|zvv_K(MDMQ%U7L&^ zf&Q6r4yspTr@X#yD7z!=WTxM&8_rTLPtn6k6Xe!Y!OO*w{TB#UcH+HjAYzNQr6oxf zmks920|uh&jwJ8bxt++g`LgV0svZyJI?3YxZOQ_0r*q`6Sv&S0{716J*_)G{2yDr)*t`IAH3nzkLgS%!h(;?qe>~9 ztGOk7UPUZLUH`4XkH8H$(@NwFWt=t(m=cPRGn(1IZ6USW^@hKruX@6&&imydno6;E z5!CVWA9{8ps`=fs-=^gBpx<~gXpPk2FF-@667`!8&CR(RFK9-})u_8d&)#Mou&~}y z`@HSB>UN=^9QMYuJJe$ImDcK^i@&}fd+^q0NY!ug%~i;fB&8EKH#Gy8a?uu9mefn@ z{mfZ~co@c{uqO{35j=lbwCT=mMT3JhHtt%E?z;CVjDuEYI5~(^)3?shnRaWYyNx5f zIaXJ))VJ7B|Dq=+w#jqXy_~CbSv*#^wT?b0&4Zj6xH6o{)6S`Ntm>54b6Fk}0Jfr4 zL{UjEcvHT=;qKQr)IV$o=BRa(Sf#-0`b!2QKeu!Fq04pr^26R>0tM^NvSt}&+Fzt1Go8raRWN}s}-Oq{jS@vzF zT5MS@_S#PBQQkF?*LKY3R(YmHHsy}yYGv|z5@^>XrCuF40lW8-sAZVli25UkZ}84c zwxT6&<_nLkSX6_?nynKFQ3ASOjNXew#yUSWRaL)(Jopi6p$k2n%V)D%g&E%6A857yQYcC(k3Op$34JNyN1)UZUWPf z(z8#vDoe9K@QL37&=_W_o}?6sJwve2 zROA>&$o|>MU$EiWZzsnM160iwwmuwWHF_cLY6VhniUo8bxJ5a)Q%(Kv%?bo0^g zB_Zra5u)Qk&3A6s{19v1_FL|r{|PT|Utpoj@?@fzfb=ad4t+3tDTYm4P{DDX*fm#+ zAD=_lUj{2KgAFrr0d#sB|aAJ5bmR8K7GDj@_=4m#-O{>rAzORh0E`~ z`=i1_l!1LXeiJ9iGBHGuH1$`& z@^vG#*3?)>K+W0hNFFeb&%> z^>=VEu8S(5YTtcn7k}83XAAuPRq;d3An!wHaG))WPI9+YI2B-FcKZcIKQ!jPz3+iD z7Vu@K=*EmqR6xv~T#s#}C#ElA?@8Stw?WeGO#XW&*yl&f6dm3PL7GXlGejtUm#ufxPSJ0QL2V(2U3?SWCw9v^9Oy zuox$G&0bBnS#6$lY4< z`8=EMq}_quD0IvVTioMA=Ii!m$&SuU@zgI%i3P48h*8mr~8lcR7W?ZZ`RqDgvaH2&*zEQ6<>ro8$ z_~E)63JD?Arl;q($s1YGxUF@m=VWf--r~c#s0N0NuqVnR$CUJ9Q_M%F%F=vdJUq@a z_5WTi_Y}c(n3c(Y5DOIQ+>8H~C-QkFi{a(f&1mwES)4lLO@R(iS>79$n(aRs%kLk1 z;2^dYyGtpSi)^p89<=t}s~YyNdug4r_I$nO*PdIMMkszh@7|D-AgD+7&(2>*MPkg5E zL568n_;=19VV{*^gUnf|Q`+J_RTWgeW3POce!NR)>iXK=CtuoYPF%#Tb0q*esOlV^ z$BE*|jiL6|M{}}KoEPOgQGOFYPk*iCRz=3Fx1%WEOj4SH^h`?4({meeG?N}9Pf5z% zv-hftQ1J#Ke-uR9TRC|aXlsf#^XFY**!${!Q+c4j4Y_o(*K;5MwWJ$5D-I&fZ|G0Z zbFd?l2Vs!_!G-a8v)fTfb+`zlil2u9eqI>JPE6<4=uN?@a4o zH#7Y`RM?ASGWQ!fa@^oQnzx`QWt+YZ3vc-_!#-UA8?UFHBegk9RSclQ*r0$aBH%Et zR(8sp-$yLOh_BQj1lBL02!Z{xvh-Zn*nx`I^6~G!FqCqXG3UDj^Rn-sW&J+H8yd?8`|2b3m>ENMXtBEfts3n%K&H&b_+a{1QpRw$w!XcyA9-a4xzCSEF z=HtCNX)fc9q-XG9lGQ$gB#j*5y( z7*MjeZzrt@Z>8Rbf(eGPq{*_hUzAd>TMFbJMmhgc%wF*}3?$ToN^9#! zn72~TBQ@PZhIth2XQhofj-T#r3;bgCMU?O-+IZ=mCg_uKaAl2uIm{V$_pXwzE;B~) zMZjadg0aXMZ$a^1`!|XK1(o({ABulGGkZztWOU7hXXd=1#LrIo z*_4%33bAfx^wlf%wwB|)m89_!9nGAY)4dj;v@>ZYl(+*+gi`?bHC##{YJ7w+B%_%C zyU-w_dH(!47(OF9cEeU`*Xa~yK$PMMthFWm+)CgCz#n$9j7 zgMfW}MkANNX`{6%m$rB>eJhaLLnG)-KWEsL(#BJFu$QS>*GD)66{#}?LIIOUvi9fgB(Vu2Lm%Mxe zpdKQ?U|d$Hh3|d`aKK>{ztmJ!rCvm%b#<%V!5YXaYZne1nr-w=g%sLL)TH8k)QZ`C zwZ?+jEVYMVAB^w_^Gxy#+ZAD9AZIJYfJI;lAb9AUb4f~O9ehpk z9V9tCxM2WN22(cC;P(Id4jP&%`D<@PpLUZF!Yt0iV}6HIv%-T#6HO)VI+K;E=eEMu zaAN9z$;X`P(}n8}YcGwyz@zqA+CV)Y=LO~v-Nila)3%_-jF^rht=?BV4~J5G2CGDd z%&j;7DCeflow@F@D)>TT+8j+;@3Mbaw(l9O#>eijBb)x(vXv_EQ|A<|(Dl<3X5BH5 zX^@?7s#8%mCox-W5UxO?re_y+U#*+2J3loAoa8E}_gi(o%|&DSMyJIa-BnC`_QtAk z3tTn*?%b>YPlH?l_wpP4sLX!4@rnhOo!i%nipp*Io*RDep5erHn=H|B^(LoXK`z(C zn&YEGGGiapupjbbVQP&$pZ-3E4WIX<2#u%^Pt6wcG!_ z=ppS6jwf5Cv#-0!ySvxewrUL)On=4t4%N8< z)*nw_5+A9g=Svp*WvgyI+qaCDLpf+3FO6PP05RMp1bYO<_wv@uWL;&OFNe0CwV_0* z7R?vg4fX++!;G8&I7N!gFmw<1=gzXmqz?RefkuP+K^ndpnIVR4ANQnONGZA%a9(aF zG4MAEcG7N`JoiqO?OL@}S#9kf$Ho=Rz?y2|+l+_)v+4fR=?7+;>VUU*pR?h*wQDI1 z^aUCShqsQAJNNtd?+G`MflIyC)P~b~36I$BDX%}}lisR~9`1np#xG^?JKs*aD$?4~ zCyk;H*1)04-Dvybg1yFdPFPhj7}tKc4HdQdHj9dOXCVs!ikg3Fio{|i2+8NmqX0-; z{N2Xl3jnQ7i*bvQOllUWlFn?I>%sPs3BrOPHK1W2TruR*<0?6L8dMKeGB`lA_0xInhxE1ye-WgDIo zv{|-nTe|t-$BU1LSFNT`nUdCoRtDPCfD?%O%$9Lp>b|kNM*g_X(l+|pu~oCb&+p+k z&iPDL=IHv{_w$pN6{X>k)rck$#=&UwROlT{OWa31$Ut$lt-^4&ES)QlMDdNazeF zHm#T2NW%+Vo&Ecqw<}A^b`Bpkoq!!3)^hY}dTO{4g*cJNj^h3@fo76JGk;Y*?L^$9 zZ^=zF6UJ%y9U@Buoc@K$pC>R80H~%laPAI0U+e0c0M!N};DQ}dg=l{3F2)!{4azWw zf~kfhiJ)ohkkt`Vj=i(kSv-DiyKH4a!APIenDYucLG6X9K)M+4UJ;dkC+SxaknwB6 zjGX1b^^nBI@i`J3Ys`J1?+}xd zRDXuiC%L7Ms-D)Shg20w+?!(*hOYBIr;7q{WnJF^lk;`fe~(v03NbicH!FpMS>?0u zI6v@NUTPY41rdhMgo(|5SGIQrv@a5`+_@0Y^~uPos{ql+=$G9Y5o+q{t4<#+e6aYt zsP&IkpSF|8#pH!^5M0UDxZV?a8mKyYFZ}#XSFyodYQyiKT;HdtGE) zRr`-~6ujw_CK_EZm75W(uN5=xGd7v`IFL)b?_4QD#FFnr2LgopwyA4;w4Pep((p^u zaw6>FO?xdS{*=&2c5<#%4`G&DPpHgyR@3x-;)Ctg1&$v=gs1k7ZkW)puj@Xe9-py> z{2bb=o4&4N?R|Udiq@~-gh$d~U1Q`9J&{*Lp=58p@l*nN3sEX=FxwGWw3 z-*?PrYv_~y{7qDGp-JWf&o477-hahpepyL4@354Rw})>^=^fTu&Xp>q$a)q~KcR18 zc7$2=W0v!d6wlH>1)3Cexs(k>R0<25fA#U!dSqUq-B9+&%|Gha;e!#86(mnRNb_Rm zX_~7K@8P=f7wny~GS_LnxMDu@-8409>F(*{*Ja*Z7x@3GM%LXHS~|e!Z9FU?kbG|?@Q2mEut6&Ig!W#`>z;rVd0_x{a5eHtn$s@vil_x(;RLMK} z6Tu@})d16o2#tdu`TrcLC5u((OBTzn|KJC?Gt)WS%l>Hb=J<=9_}qVTtHj;QB+ZGv z?f%5XbnUfMFA|kM^U2(%EqE0DS@uqsVW~mJ8e7rF#^P)q$ILLIGwYx!D&4TjZ}DwR zi%{&qDl_p6k}N2(#&4pj6J5OEdWB8Qz==s?cJnNaq^)k9D8Nci^;}J8eEM}KTQkhg z>*fec`k^nx(#dxRK!Ksfx85C$7=QhM`Pm^adPMRJVJ+yS=sgi|%&!&P79Kvmee-2y(R9r%W%*g}8`Tq+2qmxT1O_kO>{p93=?mAI zsqpz{Gsj*O!;?0VHY^rW0tFxr(qxSdHtUEJV%Yvt4p#Tv81#oRt+xh z-KLuA&8Rr@fr4l9rkPoys>;Rnr`BI#{D4~Kwrz9=&Ypc{?N1(Z3<)m!>ze+xKC7=4 zzG$DF5-PdPCh_rpglvT2&t&D%vIiFdebNe@nvirhwWmTSC0p*-Hn`7~SNa-xS#@Vv zCU7LZ_fi){Vl6-OMl`Y8D-5=MF&SPuP3EXvJ+rSg=Sw^sb)edX89>_{%nreb!Lbexi|4;|U`yY^iNwkLZfrmQ4_ z4j2H4gb5Pr?VuZ^Tt)M2^Z_|)yf|tT8E$($-S_EcRJD|vbye-V-|p@r8>QyBr0;ik z8=oD$lpRBgLbOV1=HMABBsDc795wGcYR*u#iw7M@Mp-jAx5kpsHJ`ogS%n5z761#v ze7cSNk(AN)p0vJmGt*a!{M3!9I=92|hSXTOrA_EY+C4A%9bR9{^>a%@;ElgqeO=z+ z&*g`2aFJIN77LqPeW*oo_Lr6FQf^AMY;nvU<^?wJ0)+5LWM9u*9k0)ks&MMzj_*P!f0$$M3&Rd+NuU9|VRh06wog(KlrPAc2fx(4i&FnM^ z93S^*44d*QBz{PB^#m|j<5y4H_-H9d*X;byMlzL;=F?|cTbsO_7fhR1ZCt+LG54^vPV!GW8r$<(Uq}7@N~So>P1)roY$VOa3_{q^2*80eTY%YnB; z0d(ip&D|Y?Ik?IXLNm)Zw-t7{#6FE0vFu2>qA|cU?5cqh$Pm3u6ibTi?ITv}GuEIb z>5L?rUAcBoH4#EVR?m+u@--~+cbni?abq9rQFuR8Xml^;N^%o|3jkENX??iMpiLFuxx_Q#|p_<4zZEt@#H#k?iDmJ137B#0I|OG#NX0+v9bLKpH#X zfbHe(f`ZTQqM?l@sus_>Bhqxy8BVhG>nB9xm^(6G+8y@q_@WrAOx*R~21-I-T3KGd z>2PO>)7{LO=(WI%Q8M7Uli%%`e*yyWP!cl8_F&D{30|AqY*2YmsEpEn4IRiFGmvEDZ z%(bc2Uv=HKN1VG6v!#_?f>ZZK8bu;a=!$}@QjG-m*H{u~pmKKcbIy-shG(+h^uzl5 zdj~n9?$}wD#ZZ+59KV1R{tI@a}2Q2^mk*ceOG@3DPPIm{SEA@mhC0t-JwaREM8oWxzbF9 zww-8}^{$@FD#D=7F=Dh9sXBIUaRuq9&lh`?z!1?*T)%iysYbd?_(N^&>S9cm0)W4D zh%hc|E8~)G$Cl|pY%s9;W6Q$9@8dp-MST02boj!$?tQJr4qnqsUk%!x9JF7NUiy}?>x zI$-6_ptdu$dL=9YlT2m>-Or8P8XG5rEzU+A_beFnF(3KwisIRMX>y#yA^~8qky48@ z#liDRNfFq-7BI#1&2mgS>+TvJFD);#Ns$fTC6#bScX;-viE;UKum!4#<&)R9@Bf?~ z#zoW%F?f9h>+XU6>FjE>5mD^0uQ=uhL|*Zp+V>3yn|{%I%Ot$dTUA4an%L`P0m+|3 zmR6@{oXYRTkl^O{`xasYl#-5N3%2Y{KiC#mGg4c$`E8Bo!FwL8K+Dh$Rh7pUnAG(k z+BT8g=ba+(E++PbRp0M|gUCRKDL+{j@0$oD#Zz)5CkgN&P)s_MsAuAY~C zZ@r)|K}}4FdS2pW(#n>uV}&1*;7c!^i>l&y<-;HvT8txFF_pi`>Mh;5gKT1XRv&*0 z1K84<8&^^y?z*k|op!a%Du9Z*3|sRyQ6Z@oiOc-7V)B$~&@N1}l4UyhyYh8K?$fq# zY9o-1It%~e`7HCt_Fk2s1s)F1*mv!ajRoV~)3WlCqQ2(~xL6v+?x1-zI`c*03|V~! zn{9dXw+9jGl?W+M2fXME_~{+ObV}@`uD!MML7Ac2EqwdYd>x*>E!hz~z*3Tvz<}S> z;Ne2H06IoKApc6bToiA3-00~_r#FFUAHj7JF_SN9hFLM)Z?xnj1d~Ebj?x}4`3X9Y^Jv@u1 zWDS*K?Ha-B>vw;s&Bx{m%M*^PCb&Fb6gJEnQiZU}byZknt@%Ou51P;O6#B0^)0AFs zWfgkDLwZp9W=~+cS0b7X<5S&!O-_xG*8neO-FuQjnhhdUAQ7a%m+apbg2z(rL0g5F zckflp4W*i#-}bvMstfn3jE>9AoOcJ2{$IqGs0iS1N-^X>K(($t5!(+uJyt z!z1-^r3WV%x_f6))62yn!>8_-{}>Ul0*plJG%#!T)M8R(e`;)8@Z@4Edp5Ul?yZ2q zMz)$Jv5Yl&hp9~JB8mPC7Saz3dE9`)Sb<7*S-+?!u8H6O-nZOq2i!M(aXR%isgm_YKmv{D;U5GLX7J9 zdjbFAgap8NimtFo>eCEH#aBNM2#EbX^V*<)ZFnZdxxz+f#&mU^&xY_silA(3P!@gk zk=J@UrdVi12m@jJEtDG5Sc7(~yks=5k1)TH-0M-xuh^v1(C|dSU`o3^a=SQ#>oo!{ zSxJ1aoc5N9)Oy;p+I4N$7xnpjao<;Lmr^VeN4<4rPBnXUr?)q){n#QKazRjW&A`vx zrolaOudchSK80i_!e#ZzO8!01XL9`(V@K@Q{@nx4YPjM1duq?T*uG>wGyTQqs@g4y zKiq4I;*~f0?0?ZBuwlr|HuvS$m7rfKDR>i&fZ@0Up-fqYK%=yVUoq!{xoLfpV8)ua zpLhaYY(64Sz(%X79y#|?tE&uPmnR#p+k8m-`vrP*xK-XVU-pr*M&}YM0lN`T%1@V9 zAC$Qyn04fv3nKy~A>?tum+pGxp^_sYKwRK|h7&8Lk15mt{T|+@R~6My-(9CZ;-9GC z?8xb?py9cN|7eeohoKdNE^Z_-Mp+nN;?+W}mD|^a?fcfN$BEQ|E zi||nA##uYCw5`R{)fWKP_h#pp zLMO)S64mjRY`iAvpK+)CDAw`uZC9_7h&5^R7hVd>(DO^Xpn2i(srjP6q4Z1P%pwtM zq;~P@Kp}t#nC`zgXSnKSaeKefcQo+3eLe}Uwl0@yhil!LdzZ<4SI}y!zw=#2Z>=^) zkAC1MQY-jK@54`};-LywbuQ5U>XTm_k!ipWK5IX7uLC9#Ga9yTtQ1bnPMuE`v}9E> z*8IC6Il1X#ciKrVj2JadDCeqd#(-~mE%ih*)1*FXVRh^@wvR3{mR#)!Lt?4i$^378 z&tT(#>m?sa**NoLVt!(TO31i>X1AM6)OO%!L)KsSJ2UZM{%WARZN9&^ec9jt8NnC@$IgzNJEio#0!gj&5I5(%*Zp;hc<}l2(HH{^a{GiRnz9!p z=6Vy7-#7}C?PZy!IaS5IPIMxu*;_pP`HX?Z=0KhSi`z2%QlKH`e@C0?+)uCF~TQ`k2pJqyr4fu)H$J+v5SW%kS{gI@7+5k zkoXpZZMXT2&1OiQX3FSXzAVFJ;6w>}gd|iP4BY!7(5Q>UeFlJvBm;!xP=aJzknmid z5Sbgk6p{RdX;M5?c%bOKft#r&$UN`{dXFD53h9@KyYHxx#-}L!R-JkH;DYUbhyS73 zFY|RffAKYP!^$5S{vH9nc*bzyOFv?)n;Lkg~R2_$Yws$1Lss zJg5Q@9Su+gV6|ds44!zSp9oQm!au8kw_ELiR1S1j*n>b-4lJ$(@*G9C0}*5 zb0!N3HKhwQtqjA~6nX)5uRg7_P|EjhxJSd*6881n}I; z6+9<7F`-O>MY|-mDJ;-5qj?s5-;@w!5xC~9qptqQPERjZ-fIpQZYuD|etL-VzH*zE z!+(w6gH?wjwQ$asf{fm?)=o_|8l$GqywAZP;HoclEzz$ncF3Nce z5uZVqnwjzdn{L0))B$Y;sVgsU9qX!QyLP^{={JI}WD9}}#3w$J@x1k)|9J>rq9NC| zTKnCb;0?BN3xcnNZmgf^j_tbllaax<@^oLq+P?>rYpRaxhc*v4H|6Qv%?E8f26D<} zn53dOPFtZ_2pz#M5Ma@0;QvJZJX0p|6vvs!Gn4M=UF(Eyev4{%SY2#(!f>4p3tid$ z`}ZlJ(iIF*flY7fehs)~z7fxR^7t`Y-Nh9Z6<kd(efHU6_ppd-d*s1mD$$EVjE(Qh^!0Cc#N0AW!P9(_cJy!Ql$~y zS*_~hGj|zJl%b@z#uu#<@C7`*8|jeQq}xvjKJl?3L%9UIN&&Upw{=UTp&a5u)Y$L| z`;mw@I8!ax7tvXk7M~Q2U{`*dj9!J-5c6QbY>^~IbKRNZBmjHDjJN;9% z2>8=W>zB~5ufxk(ntZB|dyfycrE}rOzAd0%fVM8Sr-2uqfm=c%0y@;8z!Ki;SwRKj z3f#r{~yvCfGtz zk7;|tr4Y&g4A65bkbL){6Dx;?uqR~8Ev&6U2{Mxg0Xc4>yg^p?y`6{{=0;aONnLNE zSwKU!@~LwImi*s0wD$ktf6rRLXY5Y6@Ydf!{7e}!OGHKC^3ptjpLZZF1&0$2+ASD(`*%N;Z6m|N6rmA| zMl`I!X<6;YH{vcpjs76ea8O{O?;Hr`cVI8dp;ykw{>n}-_BGUq#q1jRLCA6~J9{69 zCo~fF$DmKEk|+(5`kDL^y_>pElO)^UzPU^IuAPlqXVC|nUPV&#AG{Q z6}<|}e>4vjY}tK?^{A=ogY%%$XZ2;+yxH6U`RoQdt&WUMKE7(1u{VvUvlhsN=xUK678-;N@gg6h!7-@dCMD{z^JGw1TVp0?7#Fq z;`(U^^cn*zEkr3Exs7yag?^O~INDww!WQ!R^YK-}*y`3R*XE8gFoVR+dtgwumJh{7 zKKTW6CGp8BgOV9?Hv7D%UKq20RLUh&4j`hfp@6I$E!nDL&~H^=4##X5!xIPGbU9l&39MucX@83?7S@GQAoY$cm+TK3Q#PU=ZJGjt=GB zh`V>0@k2Bwo)FG)@WQ?j$Nr}auk4*Rjl&P}V!U3uXL)I4IT&D@Dk)G5gwXoW8RWCD zu#gK|3*{SADth ztrqKTSVR?c^QOTtWQz7--er0AaDX`>cv&TE1Zh7^`y@T|^ZN&sTpTJz-iyK5*r(%! zLC0{kJ!D_US&H%^5LxJjG~y4MA)@sw_PuUQou-olbO&_EZ716vVgA4MVa+p%&sD2M4tA0sCR2n9kcLkJQ; z{ZlVxME$Yu^zCcy1b$yKq+#7g|pkt~AvOm6Ur!K#QpW0`1pQ{M!IJGd;9|&g| zWHXZ4U*L@9p}BtL;X|(SU>1LzQK%w!-||`5NqEJU=NgMtj~#PmBWF1er%!_jFV^qQojVk~ z1skifqe3^^&>MOGv7`_zoHc8FN-1GD1TiM;xH~a1Y7Pz#Jyg6WL(!B+7_wVj{I2`N zkAIgZ-))3uyNpaMvFOzjjiUfVP0S6wat0J(6C2w{#L7Qd$*szuj}G|vVO7;NMRWD9 zK=43C_5h25_kI*_O))eL!ov?~Yinm#Nc0vw-b53Uba6=fzrFbl zahH!#LZHFvC;_b#`KHwA*i@I#Q-TQqA}j`h%VQLrPn$V7jCwBNpnS8mvihe+AQ!ri z?)WLyqer=+a*RWmxUk?!c)S{SpTvcwnPpx1AxuN`fUw;|H0A$Xg$NSE#*M|0=sFAT zCc6}D&s12$$f7{Do{p2W{wh$$qyRv)#XIbW$w)^*n^W0(Og>H2DvTssQb-sN^;KLK z&w{lX0|A-{n2#Lt2A=4#BZiBW$#19kt+wg`8l^i@-vJmRKLFLnyMu%-fa&R!fx)lX z6Xf6dlXL?r?O)*Hfd>K8O%IrZ;DL*}9DU~5PCe0h0OmU!uWB8y)bOZ&L$XrD0{oGt=p#KrljWb9Q+RijDb~Y5?(6K7U>W zGR!J3ZxOp_I3Qsw85bkx;To_6`aNo2u!H2RoZNb_7~;<*VSk9~0R*HsL9q<-4el5e zA{b3D9GL;P;)@k4BrC>os_@kl^P0fMGl`0}j}bBWPFTGVeLVcQM=ACp;-x3Y?i~T| z5GcOHgx-!G3=qLC1ev6Tr6mI^>)`Y{%_I?i!@RpN^}pXv9Lt`r<1IyG12)bQ2}xn$ z<2bGEU|*)bkN-qyM+psUs;N;ZCN+h@GA^Tng; z8Cr>U``~$R-`@B-BSYGIK?($7FPsbxb?B3Ga^$J~2((T~d>=7A{ctBZ4|@%qQ6rbr+}5MX=CAKDr27XUXVSSz(#baZup4g$PK)?@BSwq4Dq zPw#|-*A~U8XBP-8CaDo?^LU>u^eG5*(b~e|?@<`Zls)@-J5dZJQQ!)@SW~7Q6YSFD z@Aml=^KW;qs)%CB0n)cpp0mObVNCg41W5}D!TevZ%ju6MsVFNe8=QZ?m27D6%Z&m0 z3y;dn-U>nx$e~f)HEiq~&|}gRrF&HO3h=DrKiBf#2#AYsA@;h{mvR+RpbH2HfD*IT zPa(sb$MY5=@d$&6AXXQnPRu*pndicHT+#zUuei=sfvZr#WF7AI+LFUnz1cSYgY}>JDB!M*) z!H`BG`#p7W1{aNO`J;mb3>4Q-0ZO%bT$Dq)piX49l6Px4x_ z&$U_+zv)Xx=yF?N5l>}-=XI4ZMGvwcx1rWIHsLtkCKTl`uKl*eA#F8*mXLO{F?NKU zT>Bb&dPxd%h{&c+_x4&4hel(PVvzgVwA=A^hU!Z|q*$RACQ5;*wmfIL#m?|nSy@>K z*uO8!D21(ELJ*L*bk-P;RgS4OzD3?;ntEZno((>;{*=rJPsOKD8FQ1VYeVGKI@|QtgnCVp)N`5qG}?vy_}@yNi~*Hkp7tp#d)gRUAd7q}veRYtRLj7B*E z^kAe2EfL#Evd#8u3U!nZYuBy~1{A#QdPs-_#3q9wwP=nYOy%UsJ0yPnwpnw3*POp$ zyAC#BfH@Ptt_hLbLNj(B{va3|j%BFvmE!`_j+6c^yXfc+VUwJ5&2PUnoY@#mA|kAU z%7FvEE72ol?!6;cZr8r@4tKdSlc5E~0ND@)t03tP0qGShs6G|RKVj6=Pcu;r8HvLc zj}M%8DRA9q-(+HdcwYqwlCF^vM{*kkiOM!P6-{?2?#|zuGMXz}<^I-`WI%_f-zO+OSwt7x(mEMYL@pnr>Cis(J0?ZX^nX^K_y{cEh!@t^ucH1bbJJ zy~|ymyq}ZP^tB5q&*mfy%w7M$;+lUC(Z^koon}MvHwY#YuL{x{X2{?c1~h%lfCNvV z&uAvD1ES_ubxCSUhY$Dk^c=zj09b)75o(j5$P@~NfYxbVYcb)t{BsAk5Jm^7temGD zGXNm2gm-vPiXg$k?{H_ZmN+R;$_U5LPxbZo=G-#};+)L5T@47(y3P$baSpl5 z03yFb%1XO_{mVu>CbV|dNJg1dKaXrWTG;fT)%4r8@82I!>WK)Dj~f_hpfZP)6y%N( zhdJD@e*bQQsP~|v;+3am4zDjbI}^rbJR)YY{2*Z)1IA?#Dpr2_v>H2tU`p%w@nS-ohG>5cDM-wIO78~alU zroxzRhOphLVkCu9(B*xx%`T&vVsw1m_HI*61Z+G=CCA3cyD=qzjQlin_?{r%X=_%DDbQv19|N!U%mR!k3IHm)<@v#Ip;C)}LUK<=;e{hJM=zERqMr zhEETtw`M!`x7fYn=Ha0+Ha5<48>vBMQ`5@IDzgIOr$+95Tgz{8Ld4^$-Qqk5!7K5_ zr_4V^5Z*dU9PMPuGZZySik6jr$u~Jr63MK}obM~*(TF}iFu@KCcIWca3>GB{RRfgh z2U@d*d@w4mcH~HM?MLeG_}hJu;)NjNk#K4XRA71l8He<){Nm(S)^O_Y@sy(@6JTJ|2=!xtE#CV%G5Sq6$mV;{MK%Tviy>3s?T zP*NkNC$$2m8v`QYVPPdWGrHLh>;yN=r=8-YZ)cQz|30B>y#YA!r+P*(t)mp8u)H8f ze!*wBwO?PZ1C00sVWm}5qXo7cg}6jC{f)V86H?gzmJRq&8&L1iVMa#I@-s|B#nu^FItIiwKTCjKPi}6-%HIl^dCVJ1& z`gRl`jVy0$j6xhdju$zBj1DY%?^JGJj4@XJ?}4zJ8mvz>3)C@m(JH7VR2*>sCA$+n zOX1l_swyiR*Vbl0RQF)~&p`eRf&baZ6%fv=_5{;0MzJ`&XH9YVe()N&EzF)LB)@_= z-~EofR0QCL63l!r}C zcYulqA@&$y-(zO}z~=b21kAVwPZJN@v|mBg3~0GF%GrY#JzcUK+6S|YiWVCEsMi_1 z--0;rWOfhXt7{a!K@1iwBQZ#|wPeu9z`0tYx3|{`nM|Ym6UAf4LWj!VCdaOWeciEd zACoVk-#ym<>J~ubW8eKZ5cS65T#rL8Dj}@Y#fU9y-R;pc4<>T;FTdeDT2auL#G|Ml zFIS5jUtbIfJz-p}eGZOk;AR)XxmQ!t2nN2(kQz}$XfCQ|>x)sX`^kQK>=M8c!VCOC`6L1~}4 zf!#az?D;?uOelt5zIyiT18hFRIDo~DMfB&uPOdLTF12+0ML+)(c0hjjQFZkf7e9fa z*U%hv&$>@1ba-~`C~+x-Vc#hu(=ydJGBPsAhPk`+OJm)vwKIrOS$wt2><>@Z(v{`b%d|=Lh1uzPoF*8Gd?lF$InN?8_J)ZMGANT>w=nPWSCkl-q=+X zPCq~I$TnoBD|34Pr1LiF!*gRatcv6B9gW{7EDFzFXPl#W6`dMhW-Gf zaO(K}qQzlHO5Job<$DRsu+|j1{E@t`9<>O;2@G=s#7$4nmJNfx{r6y%QQJ@eXv7Q)9|J9j>+ zi|az3l`q$UcG7qL{W_cokx@y0R9A?KNPc1Mr%xB_H#KEg-?ED-h7q~iY%tZo`tkd5 za61mhW*M;|b0u{%;v)O89eZJH2&p_N+ms16<+};ta-4`s_Yb>h8Zj23X@k@%wgIuJSFDgqO;k4 z<*BNvnO5E1yMXFOsw8;6v5QgV^qt`O)a7(LF>%E87rvbGC#qBh39@_kJgX`k7h7F5 zQg?jr#w8nr%o9tehVt;Y>u^~i^6UFsn`IPi2$6a-(n_R{y`i$dowg>XtOJ!kPE{?| z(xx>_01B}$CcnA}81$`ocdu`c*Lm`0&7)VZ93#xDb}J<(4}9v>P_`!yfm+F(C@O6B|3o`#~o*_}O6KsJ(nlDqnv4@3|$6XfOw!=+XJ3 znN~}3gxueGLufKMpW`E6(fC5R`_KltcbIX#LJDb2La7&`g6bMQ&=qJL%xrM^p=Kj` zKTm*verl+Ht}wY>uqFV_Y|K|uOSyf0cH~c@kC(Lv;_Sc5iEs2a?>=_%Xaxe4+98)v z7)QMSe~J+@Rl(Y8U{|)4BYMUO8~qd-)}{q_4BpL9N7^TWGvUu=RpyzEk?fBi_duI&gh3 zSn{UF=Pn}%`nxmuPZ)bSy6b%lUqM~}L+@1jRZv@RXo;m^iZ2x&Cc zcid<4lyl-sWE8hx2q@%Hz*h0-FkTC}a4+Nbi%Z6H$S5IY-~rKde|^V-%$6}^yfok6 z-PoLZnP&a-g^3Xa9;55p{Vis26I5?mf!$?%lhMAn|23 z1skf^iXQA6Tc7#X&NXy&Noi@fW5ld4bicb{TVZWwC5k=->MfK=vRwss&%);N24^?h z@1OhmNGsvhq?H@!gh`2Z%8SLN8{QH`^8XZ*)-S8$!=XF_6KqKd4s&!z;Xf$)L+nw0 zv2#I9)D5nu(Ilfe2g82uzjOO-_gkTqnre29)Vgd_Dvmccw&uGzK790uvfu7i6?!9} z;4cxTBZZ;q-|hNO`=8%Jnac?VuQ*r#uoh}=6zHC|N#8>AD0Ty8d6bg0hU#B1tz!~= zS5a{dR}{N6YO(}QdS}?(MrIswNZS7?TAo)xT=!egkD^AiQeB~hbaKvQ+HR%4w#paT zkqFB1D4=NfJP2%~f?}r4c>}Qv28nCf0!vUm`GZE4Vb9Z_E>p8Tw>Gn}1)*Sw zK=cQSh8OsJqwn3r*7Gbwb%90J)&5@1B32QHpNVhAVxy4b9w``Elvq~6w9a-HjP2f$kB`iwyEbgAUvN3IwRX3ROkUM1kQ67EN)tUCR#Y@nozc!_fy;1E zur8hjVYl|vMmy%fG1w$FUr6vAF6(lFXfeVh(kE+A_zI^72SHjgq&k!Oc>Z^OEz+@z zvWNdHXBRD;QBqN1MJ(;0Fb_MLQ*{YN{E@tlQzuUjF0_DGudgl(B3E2ejkzPO*|!i- z2SZI_6BE;6C>rPH<{~R4DKPShTJ&FZP3IOcqTck{avcGt=wd@cBnYjz^Jol@;w*`h zcj&V)#+%rRIotO^Oc{{e>2U97$3ak4S4R|d9pSDR$g&%EzhD3S-(oo_Qj;PC60z{e z$Ut}i`~?z-XjoKX6!~TD&0ws)3%VK>$k*&!-rPXpAuK*#!^nsOvHiv2mIx%+i3mWy zNXR})y>Q{ehsw(5uZvz=Qj^>woabmac4mK^V__GXETk&T%&dwP#MjdSykc1(ZwbXK z8VurppB?J~j3knFGtX)1EHnyHx*UTmNk`mA61QUs`_)h$z/dev/null; sleep 1; echo done" - ) - run_ssh(target_host, kill_cmd, user, ssh_opts, timeout=15) - run_cmd( - "ps aux | grep '[t]ransfer_engine_bench' | awk '{print $2}' " - "| xargs -r kill 2>/dev/null", - timeout=5, - ) - time.sleep(2) - - -def start_target(target_host, build_dir, buffer_size, user, ssh_opts): - """Start the target process via SSH. Returns target address or None.""" - bench_bin = os.path.join( - build_dir, "mooncake-transfer-engine/example/transfer_engine_bench" - ) - log_file = "/tmp/efa_bench_target.log" - - target_cmd = ( - f"cd {build_dir} && " - f"env MC_METADATA_SERVER=P2PHANDSHAKE " - f"{bench_bin} " - f"--mode=target --protocol=efa --metadata_server=P2PHANDSHAKE " - f"--buffer_size={buffer_size} " - f"> {log_file} 2>&1" - ) - ssh_args = [ - "ssh", "-n", - *ssh_opts.split(), - f"{user}@{target_host}", - target_cmd, - ] - subprocess.Popen( - ssh_args, - stdout=subprocess.DEVNULL, - stderr=subprocess.DEVNULL, - stdin=subprocess.DEVNULL, - ) - - for _ in range(20): - time.sleep(1) - rc, stdout, _ = run_ssh( - target_host, - f"grep 'listening on' {log_file} 2>/dev/null", - user, ssh_opts, timeout=10, - ) - if rc == 0 and "listening on" in stdout: - match = re.search(r"listening on (\S+:\d+)", stdout) - if match: - return match.group(1) - - _, log_out, _ = run_ssh( - target_host, f"tail -20 {log_file}", user, ssh_opts, timeout=10 - ) - print(f" Target log:\n{log_out}", file=sys.stderr) - return None - - -def run_initiator(initiator_host, build_dir, target_addr, config, - buffer_size, duration, operation, user, ssh_opts): - """Run the initiator benchmark. Returns throughput in GB/s or None.""" - bench_bin = os.path.join( - build_dir, "mooncake-transfer-engine/example/transfer_engine_bench" - ) - flags = config["flags"] - - env_parts = ["MC_METADATA_SERVER=P2PHANDSHAKE"] - for key, val in config.get("env", {}).items(): - env_parts.append(f"{key}={val}") - env_str = " ".join(env_parts) - - bench_cmd = ( - f"cd {build_dir} && " - f"{env_str} {bench_bin} " - f"--mode=initiator --protocol=efa --metadata_server=P2PHANDSHAKE " - f"--segment_id={target_addr} " - f"--operation={operation} " - f"--duration={duration} " - f"--threads={flags['threads']} " - f"--block_size={flags['block_size']} " - f"--batch_size={flags['batch_size']} " - f"--buffer_size={buffer_size} " - f"--report_unit=GB " - f"2>&1" - ) - - timeout = duration + 60 - rc, stdout, stderr = run_ssh( - initiator_host, bench_cmd, user, ssh_opts, timeout=timeout - ) - - combined = stdout + "\n" + stderr - match = re.search(r"throughput\s+([\d.]+)\s+GB/s", combined) - if match: - return float(match.group(1)) - - print(f" WARNING: Could not parse throughput", file=sys.stderr) - lines = combined.strip().split("\n") - for line in lines[-3:]: - print(f" {line}", file=sys.stderr) - return None - - -def plot_results(results, cache_sizes_gb, annotation_gb, output_path): - """Generate and save the latency vs cache size chart.""" - import matplotlib - matplotlib.use("Agg") - import matplotlib.pyplot as plt - - fig, ax = plt.subplots(1, 1, figsize=(7, 5)) - - for entry in results: - ax.plot( - cache_sizes_gb, - entry["latencies"], - color=entry["color"], - marker=entry["marker"], - markersize=5, - linewidth=1.5, - label=entry["label"], - ) - - ax.set_xlabel("Cache Size (GB)", fontsize=12) - ax.set_ylabel("Latency (s)", fontsize=12) - ax.set_xlim(0, max(cache_sizes_gb)) - ax.set_ylim(bottom=0) - ax.grid(True, alpha=0.3) - - # Speedup annotation - if len(results) >= 2 and annotation_gb in cache_sizes_gb: - idx = cache_sizes_gb.index(annotation_gb) - tuned_lat = results[0]["latencies"][idx] - default_lat = results[1]["latencies"][idx] - - if tuned_lat > 0: - speedup = default_lat / tuned_lat - ax.axvline(x=annotation_gb, color="gray", linestyle="--", alpha=0.5) - mid_y = (tuned_lat + default_lat) / 2 - ax.annotate( - "", - xy=(annotation_gb + 1, tuned_lat), - xytext=(annotation_gb + 1, default_lat), - arrowprops=dict(arrowstyle="<->", color="black", lw=1.5), - ) - ax.text( - annotation_gb + 3, mid_y, - f"{speedup:.1f}x", - fontsize=11, fontweight="bold", va="center", - ) - - ax.text( - 0.02, 0.98, "8 x 400 Gbps EFA NICs", - transform=ax.transAxes, fontsize=10, - verticalalignment="top", fontweight="bold", - ) - ax.legend(fontsize=10, loc="upper left", bbox_to_anchor=(0.0, 0.90)) - - plt.tight_layout() - plt.savefig(output_path, dpi=150, bbox_inches="tight") - print(f"\nChart saved to: {output_path}") - - -def main(): - args = parse_args() - cache_sizes_gb = [float(x) for x in args.cache_sizes.split(",")] - - print("=" * 60) - print("EFA Latency vs Cache Size Benchmark") - print("=" * 60) - print(f" Target host: {args.target_host}") - print(f" Initiator host: {args.initiator_host}") - print(f" Build dir: {args.build_dir}") - print(f" Duration: {args.duration}s per measurement") - print(f" Operation: {args.operation}") - print(f" Buffer size: {args.buffer_size} bytes") - print(f" Cache sizes: {cache_sizes_gb} GB") - print(f" Output: {args.output}") - print(f" Configs: {len(CONFIGS)}") - n_runs = len(CONFIGS) * len(cache_sizes_gb) - print(f" Total runs: {n_runs} " - f"(each {args.duration}s + overhead)") - print() - - results = [] - - for ci, config in enumerate(CONFIGS): - label = config["label"] - print(f"[{ci+1}/{len(CONFIGS)}] Config: {label}") - print(f" threads={config['flags']['threads']}, " - f"block_size={config['flags']['block_size']}, " - f"batch_size={config['flags']['batch_size']}") - if config.get("env"): - print(f" env: {config['env']}") - print() - - throughputs = [] - latencies = [] - - # Start target once for all cache sizes of this config - kill_bench(args.target_host, args.ssh_user, args.ssh_opts) - print(f" Starting target...", end="", flush=True) - target_addr = start_target( - args.target_host, args.build_dir, args.buffer_size, - args.ssh_user, args.ssh_opts, - ) - if not target_addr: - print(" FAILED") - continue - print(f" ready ({target_addr})") - print() - - for si, cache_gb in enumerate(cache_sizes_gb): - tag = f" [{si+1}/{len(cache_sizes_gb)}] {cache_gb:6.0f} GB" - print(f"{tag} ...", end="", flush=True) - - tp = run_initiator( - args.initiator_host, args.build_dir, - target_addr, config, args.buffer_size, - args.duration, args.operation, - args.ssh_user, args.ssh_opts, - ) - - if tp is None: - print(" FAILED") - throughputs.append(None) - latencies.append(None) - continue - - lat = cache_gb / tp - throughputs.append(tp) - latencies.append(lat) - print(f" {tp:7.2f} GB/s lat={lat:.3f}s") - - # Cleanup - kill_bench(args.target_host, args.ssh_user, args.ssh_opts) - - valid = [t for t in throughputs if t is not None] - if not valid: - print(f" ERROR: No results for {label}\n", file=sys.stderr) - continue - - results.append({ - "label": label, - "throughputs": throughputs, - "latencies": latencies, - "color": config["color"], - "marker": config["marker"], - }) - print() - - if not results: - print("ERROR: No results collected.", file=sys.stderr) - sys.exit(1) - - # Summary table - print("=" * 60) - print("Results Summary") - print("=" * 60) - header = f"{'Cache(GB)':>10}" - for r in results: - header += f" {r['label']:>24}" - print(header) - print("-" * len(header)) - for i, cache_gb in enumerate(cache_sizes_gb): - row = f"{cache_gb:>10.0f}" - for r in results: - tp = r["throughputs"][i] - lat = r["latencies"][i] - if tp is not None: - row += f" {tp:7.2f} GB/s lat={lat:.3f}s" - else: - row += f" {'N/A':>24}" - print(row) - print() - - # Plot only valid points - plot_cache = [] - plot_data = [{**r, "latencies": []} for r in results] - for i, cache_gb in enumerate(cache_sizes_gb): - if all(r["latencies"][i] is not None for r in results): - plot_cache.append(cache_gb) - for j in range(len(results)): - plot_data[j]["latencies"].append(results[j]["latencies"][i]) - - plot_results(plot_data, plot_cache, args.annotation_gb, args.output) - - -if __name__ == "__main__": - main() diff --git a/mooncake-transfer-engine/include/config.h b/mooncake-transfer-engine/include/config.h index bf632ad3..954e42e3 100644 --- a/mooncake-transfer-engine/include/config.h +++ b/mooncake-transfer-engine/include/config.h @@ -65,7 +65,6 @@ struct GlobalConfig { int ib_pci_relaxed_ordering_mode = 0; bool ascend_use_fabric_mem = false; bool ascend_agent_mode = false; - size_t efa_striping_threshold = 2 * 1024 * 1024; // 2MB default // ub config parameters size_t num_jfc_per_ctx = 2; size_t num_jfce_per_ctx = 2; diff --git a/mooncake-transfer-engine/include/transport/efa_transport/efa_context.h b/mooncake-transfer-engine/include/transport/efa_transport/efa_context.h index 77aa09d6..c43c7e62 100644 --- a/mooncake-transfer-engine/include/transport/efa_transport/efa_context.h +++ b/mooncake-transfer-engine/include/transport/efa_transport/efa_context.h @@ -33,6 +33,7 @@ #include #include #include +#include #include "common.h" #include "efa_transport.h" @@ -46,7 +47,7 @@ class EfaTransport; struct EfaCq { EfaCq() : cq(nullptr), outstanding(0) {} struct fid_cq* cq; - volatile int outstanding; + std::atomic outstanding; }; struct EfaMemoryRegionMeta { @@ -56,63 +57,30 @@ struct EfaMemoryRegionMeta { uint64_t key; }; -// Endpoint store for EFA with LRU eviction support. -// When the store approaches the AV capacity (max_endpoints), stale endpoints -// that have been inactive beyond `inactive_timeout_sec` are evicted to free -// AV slots. This prevents AV exhaustion in long-running services that -// communicate with many transient peers. -class EfaEndpointStore { - public: - static constexpr double kDefaultInactiveTimeoutSec = 5.0; // 5 seconds - - explicit EfaEndpointStore( - size_t max_endpoints = 65536, - double inactive_timeout_sec = kDefaultInactiveTimeoutSec); - - std::shared_ptr get(const std::string& peer_nic_path); - // Atomically get-or-insert: returns existing endpoint or inserts new_ep. - // Prevents duplicate endpoint creation from concurrent callers. - // Triggers eviction when the store is at or above max_endpoints. - std::shared_ptr getOrInsert( - const std::string& peer_nic_path, std::shared_ptr new_ep); - void add(const std::string& peer_nic_path, - std::shared_ptr endpoint); - void remove(const std::string& peer_nic_path); - int disconnectAll(); - size_t size() const; - - // Evict endpoints that have been inactive longer than the configured - // timeout. Returns the number of evicted endpoints. - size_t evictStale(); - - // Remove endpoints whose connections are broken (not connected and not - // initializing). Returns the number of removed endpoints. - size_t removeDisconnected(); - - private: - // Must be called with write lock held. - size_t evictStaleLocked(); - - mutable RWSpinlock lock_; - std::unordered_map> endpoints_; - size_t max_endpoints_; - double inactive_timeout_sec_; -}; - // EfaContext represents the set of resources controlled by each local EFA -// device, including Memory Region, CQ, EndPoint, etc. using libfabric +// device: one libfabric domain, one address vector (AV), one shared endpoint +// (fid_ep), one or more CQs, and the MR table. +// +// Key design point (SRD shared-endpoint model): there is exactly ONE fid_ep +// per local NIC for the entire process. Every peer lives as an fi_addr_t +// entry inside the AV. Adding a peer is an O(1) `fi_av_insert` into the +// existing AV — it does NOT consume a QP slot. Consequences: +// * QP usage is a constant 1 per local NIC, regardless of peer count. +// * Cold warmup is ~ms per peer (handshake + fi_av_insert) instead of +// ~35 ms per peer (fi_endpoint + fi_enable). +// * Scale-out is bounded only by AV capacity (65536) instead of 768/NIC. class EfaContext { public: EfaContext(EfaTransport& engine, const std::string& device_name); ~EfaContext(); - int construct(size_t num_cq_list = 1, size_t num_comp_channels = 1, - uint8_t port = 1, int gid_index = -1, size_t max_cqe = 4096, + int construct(size_t num_cq_list = 1, size_t max_cqe = 4096, int max_endpoints = 65536); private: int deconstruct(); + int buildSharedEndpoint(size_t max_wr, size_t max_inline); public: // Memory Region Management @@ -132,10 +100,21 @@ class EfaContext { void set_active(bool flag) { active_ = flag; } public: - // EndPoint Management + // Get or create a per-peer handle. Does NOT open an fid_ep or call + // fi_enable — the shared endpoint was created once at construct() time. + // The returned EfaEndPoint only carries {peer_fi_addr_t, status, mutex} + // and, when connected, routes sends through this context's shared_ep_. std::shared_ptr endpoint(const std::string& peer_nic_path); + + // Non-creating lookup under the normalized key. Returns nullptr if the + // peer handle does not yet exist. Safe for idempotency checks. + std::shared_ptr peekEndpoint(const std::string& peer_nic_path); + int deleteEndpoint(const std::string& peer_nic_path); int disconnectAllEndpoints(); + + // Number of live peer handles. Historically named "QP number"; with the + // shared endpoint model the actual QP count is always 1 per context. size_t getTotalQPNumber() const; public: @@ -146,30 +125,20 @@ class EfaContext { // Submit slices for transfer int submitPostSend(const std::vector& slice_list); + // Hot-path submit: post a batch of slices to `peer_fi_addr` via the + // shared endpoint. Handles WR / CQ reservation, MR descriptor prep, + // and the fi_write / fi_read burst under post_lock_. Called by + // EfaEndPoint::submitPostSend once the peer is connected. + int submitSlicesOnPeer(fi_addr_t peer_fi_addr, + std::vector& slice_list, + std::vector& failed_slice_list); + // Poll completion queue for completed operations int pollCq(int max_entries, int cq_index = 0); - // Evict stale endpoints to free EFA resources - size_t evictStaleEndpoints() { - return endpoint_store_ ? endpoint_store_->evictStale() : 0; - } - // Get CQ count size_t cqCount() const { return cq_list_.size(); } - // Round-robin CQ assignment for new endpoints (mirrors RDMA transport) - std::shared_ptr nextCq() { - if (cq_list_.empty()) return nullptr; - int index = (next_cq_index_++) % cq_list_.size(); - return cq_list_[index]; - } - - // Get CQ outstanding count pointer - volatile int* cqOutstandingCount(int cq_index) { - if (cq_index < 0 || (size_t)cq_index >= cq_list_.size()) return nullptr; - return &cq_list_[cq_index]->outstanding; - } - public: // Device name, such as `rdmap0s2` std::string deviceName() const { return device_name_; } @@ -185,6 +154,29 @@ class EfaContext { struct fi_info* info() const { return fi_info_; } std::string localAddr() const; + // Local (shared-endpoint) address in hex, for inclusion in handshake. + // Populated by construct() -> buildSharedEndpoint(). + std::string localEpAddr() const; + + // Raw bytes of the local endpoint address. Use this for loopback + // (skip the hex encode/decode round-trip) or for any caller that + // already has the bytes. + const std::vector& localEpAddrBytes() const { + return local_ep_addr_; + } + + // Insert a peer's hex-encoded EFA address into this context's AV and + // return the resulting fi_addr_t. Thread-safe (fi_av_insert is safe + // under libfabric's domain-level threading). + int insertPeerAddr(const std::string& peer_hex_addr, fi_addr_t& out); + + // Binary variant — avoids the hex-decode when the caller already + // has the raw address bytes (e.g. loopback). + int insertPeerAddrBytes(const uint8_t* addr, size_t len, fi_addr_t& out); + + // Remove a peer from the AV. No-op if fi_addr is FI_ADDR_UNSPEC. + void removePeerAddr(fi_addr_t fi_addr); + // Compatibility methods (libfabric doesn't use lid/gid like ibverbs) uint16_t lid() const { return 0; } std::string gid() const { return localAddr(); } @@ -202,9 +194,29 @@ class EfaContext { bool active_; - std::shared_ptr endpoint_store_; + // ---- Shared endpoint (one per local NIC, serves ALL peers) ---- + struct fid_ep* shared_ep_; + std::vector local_ep_addr_; // bytes returned by fi_getname() + // Pacing for outstanding work requests on the shared endpoint. Shared + // across all peers routed through this context. std::atomic so the + // submit-path fetch_add and the CQ-poller fetch_sub obey the C++ memory + // model; plain `volatile int` + __sync_* was UB under the current + // standard. + std::atomic wr_depth_; + int max_wr_depth_; + // CQ that shared_ep_ is bound to (FI_TRANSMIT|FI_RECV). Points into + // cq_list_[0]; kept here to avoid re-indexing on the hot path. + std::shared_ptr shared_cq_; + // Serializes fi_write / fi_read calls on shared_ep_. libfabric's RDM + // endpoints are not thread-safe for concurrent post, even with + // FI_THREAD_SAFE at the domain level. + std::atomic_flag post_lock_; + std::vector> cq_list_; - std::atomic next_cq_index_{0}; + + // ---- Peer handles (one entry per peer, each ~constant size) ---- + mutable RWSpinlock peer_map_lock_; + std::unordered_map> peer_map_; RWSpinlock mr_lock_; std::map mr_map_; diff --git a/mooncake-transfer-engine/include/transport/efa_transport/efa_endpoint.h b/mooncake-transfer-engine/include/transport/efa_transport/efa_endpoint.h index 89f04238..610dbc48 100644 --- a/mooncake-transfer-engine/include/transport/efa_transport/efa_endpoint.h +++ b/mooncake-transfer-engine/include/transport/efa_transport/efa_endpoint.h @@ -24,7 +24,6 @@ #include #include -#include #include #include @@ -37,35 +36,37 @@ namespace mooncake { class EfaContext; -// Custom context for libfabric operations - stores slice pointer for completion -// handling This struct MUST have fi_context as its first member +// Custom context for libfabric operations - stores slice pointer for +// completion handling. This struct MUST have fi_context as its first member. struct EfaOpContext { struct fi_context fi_ctx; // Must be first member Transport::Slice* slice; // Slice pointer for completion handling - volatile int* wr_depth; // Pointer to endpoint's wr_depth_ for CQ - // completion decrement + // Pointer to the context's wr_depth_ for CQ completion decrement. + // std::atomic (not volatile int) so the CQ-poller decrement + // and the submit-path fetch_add play by the C++ memory model. + std::atomic* wr_depth; }; -// EfaEndPoint represents a libfabric endpoint for EFA communication. -// Unlike RDMA QPs, EFA uses RDM (Reliable Datagram) endpoints with -// an address vector for peer addressing. +// Per-peer handle in the shared-endpoint SRD model. +// +// This class does NOT own an fid_ep. A single `shared_ep_` on the owning +// EfaContext services every peer; each EfaEndPoint just carries one +// fi_addr_t (AV index) plus the handshake state needed to populate it. +// +// Lifecycle: +// 1. construct() : trivial; no libfabric resources allocated. +// 2. setupConnectionsByActive(peer): handshake -> fi_av_insert -> CONNECTED. +// 3. submitPostSend() : delegates to EfaContext::submitSlicesOnPeer. +// 4. disconnect() / dtor : fi_av_remove(peer_fi_addr_). class EfaEndPoint { public: using HandShakeDesc = TransferMetadata::HandShakeDesc; enum Status { INITIALIZING, UNCONNECTED, CONNECTED }; - EfaEndPoint(EfaContext& context); + explicit EfaEndPoint(EfaContext& context); ~EfaEndPoint(); - // Construct endpoint with specified completion queue - int construct(struct fid_cq* cq, volatile int* cq_outstanding, - size_t num_qp_list = 1, size_t max_sge = 4, - size_t max_wr = 256, size_t max_inline = 64); - - private: - int deconstruct(); - public: void setPeerNicPath(const std::string& peer_nic_path); @@ -79,34 +80,22 @@ class EfaEndPoint { int setupConnectionsByPassive(const HandShakeDesc& peer_desc, HandShakeDesc& local_desc); - bool hasOutstandingSlice() const; + // Always false under the shared-endpoint model: outstanding work is + // tracked at the context (shared) level, not per-peer. Retained for + // API compatibility with callers that still ask. + bool hasOutstandingSlice() const { return false; } - bool active() const { return active_; } - - void set_active(bool flag) { - RWSpinlock::WriteGuard guard(lock_); - active_ = flag; - if (!flag) inactive_time_ = getCurrentTimeInNano(); - } - - double inactiveTime() { - if (active_) return 0.0; - return (getCurrentTimeInNano() - inactive_time_) / 1000000000.0; - } - - void touchLastUsed() { last_used_time_ = getCurrentTimeInNano(); } - - double lastUsedAge() const { - return (getCurrentTimeInNano() - last_used_time_) / 1000000000.0; - } - - public: bool connected() const { return status_.load(std::memory_order_relaxed) == CONNECTED; } void disconnect(); - int destroyQP(); + + // Called during EfaContext teardown: forget the AV slot WITHOUT calling + // fi_av_remove(). The AV itself is about to be closed, which invalidates + // every slot in one shot; calling fi_av_remove after the shared endpoint + // has been closed trips an assertion inside the EFA provider. + void markDetachedForTeardown(); private: void disconnectUnlocked(); @@ -114,57 +103,27 @@ class EfaEndPoint { public: const std::string toString() const; - // Submit RDMA write/read operations via libfabric + // Submit a batch of slices bound for this peer. Internally establishes + // the connection if needed, then delegates to + // EfaContext::submitSlicesOnPeer using this peer's fi_addr_t. int submitPostSend(std::vector& slice_list, std::vector& failed_slice_list); - // Get the number of endpoints (always 1 for EFA RDM) + // Always 1: the shared endpoint backs every peer. Kept so callers that + // sum "QP count" across endpoints don't break. size_t getQPNumber() const { return 1; } - // Get local endpoint address for handshake - std::string getLocalAddr() const; - - // Get peer's fi_addr fi_addr_t getPeerFiAddr() const { return peer_fi_addr_; } EfaContext& context() { return context_; } - private: - // Setup connection using peer's address from handshake - int doSetupConnection(const std::string& peer_addr, - std::string* reply_msg = nullptr); - - // Insert peer address into address vector - int insertPeerAddr(const std::string& peer_addr); - private: EfaContext& context_; std::atomic status_; - RWSpinlock lock_; + RWSpinlock lock_; // protects peer_nic_path_ and status_ std::string peer_nic_path_; - - // Libfabric endpoint - struct fid_ep* ep_; - struct fid_cq* tx_cq_; - struct fid_cq* rx_cq_; - fi_addr_t peer_fi_addr_; // Peer's address in the AV - - // Local endpoint address (for handshake) - std::vector local_addr_; - size_t local_addr_len_; - - volatile int wr_depth_; - int max_wr_depth_; - volatile int* cq_outstanding_; - - // Spinlock to serialize fi_write/fi_read calls on this endpoint. - // libfabric RDM endpoints are not thread-safe by default. - std::atomic_flag post_lock_ = ATOMIC_FLAG_INIT; - - volatile bool active_; - volatile uint64_t inactive_time_; - volatile uint64_t last_used_time_; // Updated on connection and I/O + fi_addr_t peer_fi_addr_; // slot in context_.av() }; } // namespace mooncake diff --git a/mooncake-transfer-engine/include/transport/efa_transport/efa_transport.h b/mooncake-transfer-engine/include/transport/efa_transport/efa_transport.h index 63c6d4ae..a1711986 100644 --- a/mooncake-transfer-engine/include/transport/efa_transport/efa_transport.h +++ b/mooncake-transfer-engine/include/transport/efa_transport/efa_transport.h @@ -70,18 +70,15 @@ class EfaTransport : public Transport { int unregisterLocalMemoryBatch( const std::vector& addr_list) override; - // Eagerly establish EFA endpoints to every NIC of `segment_name`. + // Eagerly populate the address vector with every (local_ctx, peer_nic) + // handshake for `segment_name`. // - // Rationale: libfabric FI_EP_RDM endpoints resolve peer addresses lazily - // via fi_av_insert() on first send. With 16 local NICs × N peer NICs, - // the first submitTransfer() of a batch serializes ~N*16 - // handshake+fi_av_insert round-trips, producing a single-digit-second - // stall (observed ~6 s on B300 for the first 100 × 0.5 MB batch). After - // this call returns, every (local_ctx, peer_nic) endpoint is CONNECTED - // and the first real submitTransfer() goes straight to fi_write/fi_read. + // Under the shared-endpoint model each warmup is handshake RPC + + // fi_av_insert — no fi_endpoint/fi_enable. This removes the first- + // submit stall (handshakes on the data path) without consuming QPs. // // Safe to call multiple times (idempotent: endpoint() + setup are both - // idempotent). Re-run after any openSegment() on a new peer. + // idempotent). Re-run after any openSegment() on a new peer. int warmupSegment(const std::string& segment_name); private: diff --git a/mooncake-transfer-engine/src/config.cpp b/mooncake-transfer-engine/src/config.cpp index 93815b3e..d0575d41 100644 --- a/mooncake-transfer-engine/src/config.cpp +++ b/mooncake-transfer-engine/src/config.cpp @@ -195,16 +195,6 @@ void loadGlobalConfig(GlobalConfig& config) { } } - const char* efa_striping_env = std::getenv("MC_EFA_STRIPING_THRESHOLD"); - if (efa_striping_env) { - size_t val = strtoull(efa_striping_env, nullptr, 10); - if (val > 0) - config.efa_striping_threshold = val; - else - LOG(WARNING) << "Ignore value from environment variable " - "MC_EFA_STRIPING_THRESHOLD"; - } - const char* retry_cnt_env = std::getenv("MC_RETRY_CNT"); if (retry_cnt_env) { size_t val = atoi(retry_cnt_env); diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp index 4c96b7e1..9bc86830 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp @@ -19,12 +19,12 @@ #include #include -#include -#include -#include #include +#include #include +#include #include +#include #include "config.h" #include "cuda_alike.h" @@ -34,117 +34,8 @@ namespace mooncake { -// EfaEndpointStore implementation - -EfaEndpointStore::EfaEndpointStore(size_t max_endpoints, - double inactive_timeout_sec) - : max_endpoints_(max_endpoints), - inactive_timeout_sec_(inactive_timeout_sec) {} - -std::shared_ptr EfaEndpointStore::get( - const std::string& peer_nic_path) { - RWSpinlock::ReadGuard guard(lock_); - auto it = endpoints_.find(peer_nic_path); - if (it != endpoints_.end()) { - it->second->set_active(true); - return it->second; - } - return nullptr; -} - -std::shared_ptr EfaEndpointStore::getOrInsert( - const std::string& peer_nic_path, std::shared_ptr new_ep) { - RWSpinlock::WriteGuard guard(lock_); - auto it = endpoints_.find(peer_nic_path); - if (it != endpoints_.end()) { - it->second->set_active(true); - return it->second; // Another thread already created it - } - // Evict stale endpoints if at capacity - if (endpoints_.size() >= max_endpoints_) { - size_t evicted = evictStaleLocked(); - if (evicted == 0 && endpoints_.size() >= max_endpoints_) { - LOG(WARNING) << "EfaEndpointStore at capacity (" << max_endpoints_ - << ") with no stale endpoints to evict"; - } - } - endpoints_[peer_nic_path] = new_ep; - new_ep->set_active(true); - return new_ep; -} - -void EfaEndpointStore::add(const std::string& peer_nic_path, - std::shared_ptr endpoint) { - RWSpinlock::WriteGuard guard(lock_); - endpoints_[peer_nic_path] = endpoint; -} - -void EfaEndpointStore::remove(const std::string& peer_nic_path) { - RWSpinlock::WriteGuard guard(lock_); - endpoints_.erase(peer_nic_path); -} - -int EfaEndpointStore::disconnectAll() { - RWSpinlock::WriteGuard guard(lock_); - for (auto& entry : endpoints_) { - if (entry.second) { - entry.second->disconnect(); - } - } - return 0; -} - -size_t EfaEndpointStore::size() const { - RWSpinlock::ReadGuard guard(lock_); - return endpoints_.size(); -} - -size_t EfaEndpointStore::evictStale() { - RWSpinlock::WriteGuard guard(lock_); - return evictStaleLocked(); -} - -size_t EfaEndpointStore::evictStaleLocked() { - size_t evicted = 0; - for (auto it = endpoints_.begin(); it != endpoints_.end();) { - auto& ep = it->second; - double age = ep ? ep->lastUsedAge() : 0; - bool outstanding = ep ? ep->hasOutstandingSlice() : false; - if (ep && !outstanding && age > inactive_timeout_sec_) { - LOG(INFO) << "Evicting stale EFA endpoint: " << it->first - << " (idle " << age - << "s, timeout=" << inactive_timeout_sec_ << "s)"; - ep->disconnect(); - it = endpoints_.erase(it); - ++evicted; - } else { - ++it; - } - } - if (endpoints_.size() > 0) { - VLOG(1) << "evictStale: " << endpoints_.size() << " endpoints, " - << evicted << " evicted"; - } - return evicted; -} - -size_t EfaEndpointStore::removeDisconnected() { - RWSpinlock::WriteGuard guard(lock_); - size_t removed = 0; - for (auto it = endpoints_.begin(); it != endpoints_.end();) { - auto& ep = it->second; - if (ep && !ep->connected() && !ep->hasOutstandingSlice()) { - LOG(INFO) << "Removing disconnected EFA endpoint: " << it->first; - it = endpoints_.erase(it); - ++removed; - } else { - ++it; - } - } - return removed; -} - // EfaContext implementation + EfaContext::EfaContext(EfaTransport& engine, const std::string& device_name) : engine_(engine), device_name_(device_name), @@ -153,17 +44,18 @@ EfaContext::EfaContext(EfaTransport& engine, const std::string& device_name) fabric_(nullptr), domain_(nullptr), av_(nullptr), - active_(true) {} + active_(true), + shared_ep_(nullptr), + wr_depth_(0), + max_wr_depth_(0), + post_lock_(ATOMIC_FLAG_INIT) {} EfaContext::~EfaContext() { if (fabric_) deconstruct(); } -int EfaContext::construct(size_t num_cq_list, size_t num_comp_channels, - uint8_t port, int gid_index, size_t max_cqe, +int EfaContext::construct(size_t num_cq_list, size_t max_cqe, int max_endpoints) { - endpoint_store_ = std::make_shared(max_endpoints); - #if !defined(USE_CUDA) && !defined(USE_HIP) // When built without GPU support, prevent libfabric's EFA provider from // dlopen-ing libcudart/libcuda at fi_getinfo/fi_domain time. That @@ -232,7 +124,9 @@ int EfaContext::construct(size_t num_cq_list, size_t num_comp_channels, return ERR_CONTEXT; } - // Create address vector + // Create address vector. Capacity sized for the largest peer count we + // expect to support in a single process. AV entries are cheap (no QP + // cost), so we over-provision. struct fi_av_attr av_attr = {}; av_attr.type = FI_AV_TABLE; av_attr.count = max_endpoints; @@ -269,18 +163,105 @@ int EfaContext::construct(size_t num_cq_list, size_t num_comp_channels, cq_list_[i] = cq; } + // Build the shared endpoint that services every peer through AV lookup. + ret = buildSharedEndpoint(globalConfig().max_wr, 64); + if (ret) { + LOG(ERROR) << "EfaContext::construct: buildSharedEndpoint failed for " + << device_name_; + return ret; + } + LOG(INFO) << "EFA device (libfabric): " << device_name_ << ", domain: " << fi_info_->domain_attr->name - << ", provider: " << fi_info_->fabric_attr->prov_name; + << ", provider: " << fi_info_->fabric_attr->prov_name + << " (shared endpoint, max_wr=" << max_wr_depth_ << ")"; return 0; } +int EfaContext::buildSharedEndpoint(size_t max_wr, size_t max_inline) { + (void)max_inline; + if (shared_ep_) return 0; + + shared_cq_ = cq_list_.empty() ? nullptr : cq_list_[0]; + if (!shared_cq_) { + LOG(ERROR) << "EfaContext::buildSharedEndpoint: no CQ available"; + return ERR_CONTEXT; + } + max_wr_depth_ = static_cast(max_wr); + + int ret = fi_endpoint(domain_, fi_info_, &shared_ep_, nullptr); + if (ret) { + LOG(ERROR) << "fi_endpoint failed: " << fi_strerror(-ret); + shared_ep_ = nullptr; + return ERR_ENDPOINT; + } + + ret = fi_ep_bind(shared_ep_, &av_->fid, 0); + if (ret) { + LOG(ERROR) << "fi_ep_bind(av) failed: " << fi_strerror(-ret); + fi_close(&shared_ep_->fid); + shared_ep_ = nullptr; + return ERR_ENDPOINT; + } + + ret = fi_ep_bind(shared_ep_, &shared_cq_->cq->fid, FI_TRANSMIT); + if (ret) { + LOG(ERROR) << "fi_ep_bind(tx_cq) failed: " << fi_strerror(-ret); + fi_close(&shared_ep_->fid); + shared_ep_ = nullptr; + return ERR_ENDPOINT; + } + + ret = fi_ep_bind(shared_ep_, &shared_cq_->cq->fid, FI_RECV); + if (ret) { + LOG(ERROR) << "fi_ep_bind(rx_cq) failed: " << fi_strerror(-ret); + fi_close(&shared_ep_->fid); + shared_ep_ = nullptr; + return ERR_ENDPOINT; + } + + ret = fi_enable(shared_ep_); + if (ret) { + LOG(ERROR) << "fi_enable failed: " << fi_strerror(-ret); + fi_close(&shared_ep_->fid); + shared_ep_ = nullptr; + return ERR_ENDPOINT; + } + + // Cache our own libfabric address for handshake advertisement. + size_t addr_len = 64; + local_ep_addr_.assign(addr_len, 0); + ret = fi_getname(&shared_ep_->fid, local_ep_addr_.data(), &addr_len); + if (ret) { + LOG(ERROR) << "fi_getname failed: " << fi_strerror(-ret); + fi_close(&shared_ep_->fid); + shared_ep_ = nullptr; + local_ep_addr_.clear(); + return ERR_ENDPOINT; + } + local_ep_addr_.resize(addr_len); + return 0; +} + int EfaContext::deconstruct() { - // Destroy all endpoints before closing domain/fabric/AV. - // Endpoints hold fi_ep handles that reference the domain, so they must - // be closed first. - endpoint_store_.reset(); + // Teardown order matters for the EFA provider: the shared endpoint must + // be closed before the AV it is bound to. We also cannot call + // fi_av_remove() after fi_close(ep) — the provider faults — so we just + // clear the peer map (dropping shared_ptrs) and let fi_av_close() below + // invalidate every AV slot in one shot. + if (shared_ep_) { + fi_close(&shared_ep_->fid); + shared_ep_ = nullptr; + } + + { + RWSpinlock::WriteGuard guard(peer_map_lock_); + for (auto& entry : peer_map_) { + if (entry.second) entry.second->markDetachedForTeardown(); + } + peer_map_.clear(); + } { RWSpinlock::WriteGuard guard(mr_lock_); @@ -299,6 +280,7 @@ int EfaContext::deconstruct() { } } cq_list_.clear(); + shared_cq_.reset(); if (av_) { fi_close(&av_->fid); @@ -331,6 +313,7 @@ int EfaContext::deconstruct() { int EfaContext::registerMemoryRegionInternal(void* addr, size_t length, int access, EfaMemoryRegionMeta& mrMeta) { + (void)access; if (length > (size_t)globalConfig().max_mr_size) { LOG(ERROR) << "Buffer length " << length << " exceeds device max_mr_size " @@ -417,24 +400,22 @@ int EfaContext::registerMemoryRegion(void* addr, size_t length, int access) { int EfaContext::unregisterMemoryRegion(void* addr) { RWSpinlock::WriteGuard guard(mr_lock_); auto it = mr_map_.find((uint64_t)addr); - if (it != mr_map_.end()) { - if (it->second.mr) { - int ret = fi_close(&it->second.mr->fid); - if (ret) { - LOG(ERROR) << "Failed to unregister memory " << addr << ": " - << fi_strerror(-ret); - return ERR_CONTEXT; - } - } - mr_map_.erase(it); + if (it == mr_map_.end()) { + return 0; } + if (it->second.mr) { + fi_close(&it->second.mr->fid); + } + mr_map_.erase(it); return 0; } int EfaContext::preTouchMemory(void* addr, size_t length) { - volatile char* ptr = (volatile char*)addr; - for (size_t i = 0; i < length; i += 4096) { - ptr[i] = ptr[i]; + volatile char* p = static_cast(addr); + const long sc = sysconf(_SC_PAGESIZE); + const size_t page_size = sc > 0 ? static_cast(sc) : 4096; + for (size_t off = 0; off < length; off += page_size) { + p[off] = p[off]; } return 0; } @@ -445,7 +426,7 @@ uint64_t EfaContext::rkey(void* addr) { if (it != mr_map_.begin()) { --it; if ((uint64_t)addr < it->first + it->second.length && it->second.mr) { - return it->second.key; + return fi_mr_key(it->second.mr); } } return 0; @@ -477,58 +458,65 @@ void* EfaContext::mrDesc(void* addr) { std::shared_ptr EfaContext::endpoint( const std::string& peer_nic_path) { - if (!endpoint_store_) return nullptr; - // Use normalized key (strip port) so the same physical peer reuses its - // endpoint across reconnections. Each P2PHANDSHAKE run picks a random + // handle across reconnections. Each P2PHANDSHAKE run picks a random // port, producing a different peer_nic_path for the same peer host+NIC. std::string key = normalizeNicPath(peer_nic_path); - // Fast path: endpoint already exists for this physical peer. - // Update peer_nic_path in case the port changed (new initiator run). - // setPeerNicPath disconnects old connection (fi_av_remove) if needed. - auto ep = endpoint_store_->get(key); - if (ep) { - ep->setPeerNicPath(peer_nic_path); - return ep; - } - - // Slow path: create new endpoint, then atomically insert (or get existing - // if another thread raced us). getOrInsert prevents duplicate endpoints - // and duplicate AV entries for the same peer. - auto new_endpoint = std::make_shared(*this); - auto cq = nextCq(); - if (cq) { - int ret = new_endpoint->construct(cq->cq, &cq->outstanding, 1, 4, - globalConfig().max_wr, 64); - if (ret != 0) { - LOG(ERROR) << "Failed to construct EFA endpoint"; - return nullptr; + { + RWSpinlock::ReadGuard guard(peer_map_lock_); + auto it = peer_map_.find(key); + if (it != peer_map_.end()) { + it->second->setPeerNicPath(peer_nic_path); + return it->second; } } - // Still set the full peer_nic_path (with port) for handshake routing - new_endpoint->setPeerNicPath(peer_nic_path); - ep = endpoint_store_->getOrInsert(key, new_endpoint); - // If another thread won the race, new_endpoint is discarded (RAII cleanup) - return ep; + + auto new_ep = std::make_shared(*this); + new_ep->setPeerNicPath(peer_nic_path); + + RWSpinlock::WriteGuard guard(peer_map_lock_); + auto it = peer_map_.find(key); + if (it != peer_map_.end()) { + it->second->setPeerNicPath(peer_nic_path); + return it->second; + } + peer_map_[key] = new_ep; + return new_ep; +} + +std::shared_ptr EfaContext::peekEndpoint( + const std::string& peer_nic_path) { + RWSpinlock::ReadGuard guard(peer_map_lock_); + auto it = peer_map_.find(normalizeNicPath(peer_nic_path)); + if (it == peer_map_.end()) return nullptr; + return it->second; } int EfaContext::deleteEndpoint(const std::string& peer_nic_path) { - if (endpoint_store_) { - endpoint_store_->remove(peer_nic_path); + std::shared_ptr ep; + { + RWSpinlock::WriteGuard guard(peer_map_lock_); + auto it = peer_map_.find(normalizeNicPath(peer_nic_path)); + if (it == peer_map_.end()) return 0; + ep = it->second; + peer_map_.erase(it); } + if (ep) ep->disconnect(); // runs fi_av_remove return 0; } int EfaContext::disconnectAllEndpoints() { - if (endpoint_store_) { - return endpoint_store_->disconnectAll(); + RWSpinlock::WriteGuard guard(peer_map_lock_); + for (auto& entry : peer_map_) { + if (entry.second) entry.second->disconnect(); } return 0; } size_t EfaContext::getTotalQPNumber() const { - return endpoint_store_ ? endpoint_store_->size() : 0; + RWSpinlock::ReadGuard guard(peer_map_lock_); + return peer_map_.size(); } std::string EfaContext::nicPath() const { @@ -536,11 +524,7 @@ std::string EfaContext::nicPath() const { } std::string EfaContext::localAddr() const { - // Return a hex string representation of the local address info - if (!fi_info_ || !fi_info_->src_addr) { - return ""; - } - + if (!fi_info_ || !fi_info_->src_addr) return ""; std::ostringstream oss; const uint8_t* addr = static_cast(fi_info_->src_addr); for (size_t i = 0; i < fi_info_->src_addrlen; ++i) { @@ -549,27 +533,82 @@ std::string EfaContext::localAddr() const { return oss.str(); } +std::string EfaContext::localEpAddr() const { + static constexpr char kHex[] = "0123456789abcdef"; + std::string out; + out.resize(local_ep_addr_.size() * 2); + for (size_t i = 0; i < local_ep_addr_.size(); ++i) { + out[2 * i] = kHex[(local_ep_addr_[i] >> 4) & 0xF]; + out[2 * i + 1] = kHex[local_ep_addr_[i] & 0xF]; + } + return out; +} + +// Decode one hex nibble, -1 on invalid input. +static inline int hexNibble(char c) { + if (c >= '0' && c <= '9') return c - '0'; + if (c >= 'a' && c <= 'f') return c - 'a' + 10; + if (c >= 'A' && c <= 'F') return c - 'A' + 10; + return -1; +} + +int EfaContext::insertPeerAddr(const std::string& peer_hex_addr, + fi_addr_t& out) { + if (peer_hex_addr.empty() || (peer_hex_addr.size() % 2) != 0) { + LOG(ERROR) << "insertPeerAddr: invalid hex length " + << peer_hex_addr.size(); + return ERR_INVALID_ARGUMENT; + } + const size_t n = peer_hex_addr.size() / 2; + std::vector bin(n); + for (size_t i = 0; i < n; ++i) { + int hi = hexNibble(peer_hex_addr[2 * i]); + int lo = hexNibble(peer_hex_addr[2 * i + 1]); + if (hi < 0 || lo < 0) { + LOG(ERROR) << "insertPeerAddr: non-hex char at offset " << (2 * i); + return ERR_INVALID_ARGUMENT; + } + bin[i] = static_cast((hi << 4) | lo); + } + return insertPeerAddrBytes(bin.data(), bin.size(), out); +} + +int EfaContext::insertPeerAddrBytes(const uint8_t* addr, size_t len, + fi_addr_t& out) { + if (!addr || len == 0) return ERR_INVALID_ARGUMENT; + int ret = fi_av_insert(av_, addr, 1, &out, 0, nullptr); + if (ret != 1) { + LOG(ERROR) << "fi_av_insert failed: " << fi_strerror(-ret); + return ERR_ENDPOINT; + } + return 0; +} + +void EfaContext::removePeerAddr(fi_addr_t fi_addr) { + if (fi_addr == FI_ADDR_UNSPEC) return; + int ret = fi_av_remove(av_, &fi_addr, 1, 0); + if (ret) { + LOG(WARNING) << "fi_av_remove failed: " << fi_strerror(-ret); + } +} + int EfaContext::submitPostSend( const std::vector& slice_list) { - // Route slices to appropriate endpoints for sending - // Group slices by peer NIC path + // Route slices to appropriate peer handles. Group by peer NIC path. std::unordered_map> slices_by_peer; - std::vector failed_slices; for (auto* slice : slice_list) { if (!slice) continue; - // Fast path: peer info already resolved by submitTransferTask's - // striping path (dest_rkey and peer_nic_path pre-set on slice). - // This eliminates per-slice metadata lookup, selectDevice(), and - // string construction — the main bottleneck for multi-NIC striping. + // Fast path: peer info already filled in by the caller + // (dest_rkey and peer_nic_path set on the slice before dispatch). if (!slice->peer_nic_path.empty()) { slices_by_peer[slice->peer_nic_path].push_back(slice); continue; } - // Slow path: resolve peer info per-slice (non-striped transfers) + // Slow path: resolve peer info per-slice. auto peer_segment_desc = engine_.meta()->getSegmentDescByID(slice->target_id); if (!peer_segment_desc) { @@ -579,7 +618,6 @@ int EfaContext::submitPostSend( continue; } - // Find the buffer and device for this destination address int buffer_id = -1, device_id = -1; if (EfaTransport::selectDevice(peer_segment_desc.get(), slice->rdma.dest_addr, slice->length, @@ -590,46 +628,232 @@ int EfaContext::submitPostSend( continue; } - // Set the remote key from the peer's registered memory region slice->rdma.dest_rkey = peer_segment_desc->buffers[buffer_id].rkey[device_id]; - // Construct peer NIC path: "server_name@device_name" std::string peer_nic_path = peer_segment_desc->name + "@" + peer_segment_desc->devices[device_id].name; slice->peer_nic_path = peer_nic_path; - slices_by_peer[peer_nic_path].push_back(slice); } - // Now send to each peer endpoint for (auto& entry : slices_by_peer) { const std::string& peer_nic_path = entry.first; auto& peer_slices = entry.second; - // Get or create endpoint for this peer auto ep = endpoint(peer_nic_path); if (!ep) { - LOG(ERROR) << "Cannot create endpoint for peer " << peer_nic_path; - for (auto* slice : peer_slices) { - slice->markFailed(); - } + LOG(ERROR) << "Cannot create peer handle for " << peer_nic_path; + for (auto* slice : peer_slices) slice->markFailed(); continue; } - // Submit to endpoint std::vector failed_slice_list; - ep->submitPostSend(peer_slices, failed_slice_list); + int rc = ep->submitPostSend(peer_slices, failed_slice_list); + for (auto* slice : failed_slice_list) slice->markFailed(); - // Handle any slices that failed to post - for (auto* slice : failed_slice_list) { - slice->markFailed(); + // Drop peer handle if it is no longer connected after submit, + // freeing its AV entry for reuse. Under the shared-endpoint model + // this is cheap (no fid_ep to destroy). + if (rc != 0 && !ep->connected()) { + deleteEndpoint(normalizeNicPath(peer_nic_path)); } } return 0; } +int EfaContext::submitSlicesOnPeer( + fi_addr_t peer_fi_addr, std::vector& slice_list, + std::vector& failed_slice_list) { + // Batched submission against the shared endpoint. Mirrors the previous + // per-endpoint submit path but uses context-level wr_depth / post_lock. + // + // 1. Reserve N WR+CQ slots in bulk (single CAS each) + // 2. Prepare MR descriptors and op contexts outside the lock + // 3. Hold post_lock_ once for the entire batch of fi_write calls + const int kMaxBackoffYields = 100000; + const int cq_limit = static_cast(globalConfig().max_cqe); + std::atomic* cq_outstanding = + shared_cq_ ? &shared_cq_->outstanding : nullptr; + + struct BatchEntry { + Transport::Slice* slice; + void* local_desc; + EfaOpContext* op_ctx; + }; + + // Consume slice_list via a moving index instead of erase-from-front, + // which was O(N^2) on large batches. retry_slices accumulate across + // passes and are applied by rewinding the cursor. + size_t cursor = 0; + std::vector retry_slices; + while (cursor < slice_list.size() || !retry_slices.empty()) { + if (!retry_slices.empty()) { + // Splice retry slices back in at the current cursor so the next + // pass picks them up. O(retry_slices.size()) per retry wave, + // which is bounded by valid_count of the last batch. + slice_list.insert(slice_list.begin() + cursor, retry_slices.begin(), + retry_slices.end()); + retry_slices.clear(); + std::this_thread::yield(); + } + + const size_t remaining = slice_list.size() - cursor; + int batch_count = 0; + int backoff = 0; + bool timed_out = false; + while (batch_count == 0) { + int cur_wr = wr_depth_.load(std::memory_order_relaxed); + int wr_avail = max_wr_depth_ - cur_wr; + if (wr_avail <= 0) { + if (++backoff > kMaxBackoffYields) { + timed_out = true; + break; + } + std::this_thread::yield(); + continue; + } + int want = std::min(wr_avail, (int)remaining); + if (cq_outstanding) { + int cur_cq = cq_outstanding->load(std::memory_order_relaxed); + int cq_avail = cq_limit - cur_cq; + if (cq_avail <= 0) { + if (++backoff > kMaxBackoffYields) { + timed_out = true; + break; + } + std::this_thread::yield(); + continue; + } + want = std::min(want, cq_avail); + if (!wr_depth_.compare_exchange_weak( + cur_wr, cur_wr + want, std::memory_order_acq_rel, + std::memory_order_relaxed)) { + continue; + } + cur_cq = cq_outstanding->load(std::memory_order_relaxed); + cq_avail = cq_limit - cur_cq; + if (cq_avail < want) { + wr_depth_.fetch_sub(want, std::memory_order_acq_rel); + continue; + } + if (!cq_outstanding->compare_exchange_weak( + cur_cq, cur_cq + want, std::memory_order_acq_rel, + std::memory_order_relaxed)) { + wr_depth_.fetch_sub(want, std::memory_order_acq_rel); + continue; + } + } else { + if (!wr_depth_.compare_exchange_weak( + cur_wr, cur_wr + want, std::memory_order_acq_rel, + std::memory_order_relaxed)) { + continue; + } + } + batch_count = want; + } + + if (timed_out) { + LOG(WARNING) << "EFA submitSlicesOnPeer: timed out waiting for CQ" + << " drain (wr_depth=" + << wr_depth_.load(std::memory_order_relaxed) + << ", max=" << max_wr_depth_ << ", cq_outstanding=" + << (cq_outstanding ? cq_outstanding->load( + std::memory_order_relaxed) + : -1) + << ", max_cqe=" << cq_limit << ")"; + for (size_t i = cursor; i < slice_list.size(); ++i) { + failed_slice_list.push_back(slice_list[i]); + } + slice_list.clear(); + return 0; + } + + std::vector batch(batch_count); + int valid_count = 0; + + for (int i = 0; i < batch_count; i++) { + Transport::Slice* slice = slice_list[cursor + i]; + void* local_desc = mrDesc(slice->source_addr); + if (!local_desc) { + LOG(ERROR) << "No MR descriptor found for address " + << slice->source_addr; + failed_slice_list.push_back(slice); + continue; + } + EfaOpContext* op_ctx = new EfaOpContext(); + memset(op_ctx, 0, sizeof(EfaOpContext)); + op_ctx->slice = slice; + op_ctx->wr_depth = &wr_depth_; + batch[valid_count++] = {slice, local_desc, op_ctx}; + } + + int mr_failures = batch_count - valid_count; + if (mr_failures > 0) { + wr_depth_.fetch_sub(mr_failures, std::memory_order_acq_rel); + if (cq_outstanding) + cq_outstanding->fetch_sub(mr_failures, + std::memory_order_acq_rel); + } + + if (valid_count > 0) { + while (post_lock_.test_and_set(std::memory_order_acquire)) { + } + for (int i = 0; i < valid_count; i++) { + auto& entry = batch[i]; + ssize_t ret; + if (entry.slice->opcode == Transport::TransferRequest::READ) { + ret = fi_read(shared_ep_, (void*)entry.slice->source_addr, + entry.slice->length, entry.local_desc, + peer_fi_addr, entry.slice->rdma.dest_addr, + entry.slice->rdma.dest_rkey, + &entry.op_ctx->fi_ctx); + } else { + ret = fi_write(shared_ep_, (void*)entry.slice->source_addr, + entry.slice->length, entry.local_desc, + peer_fi_addr, entry.slice->rdma.dest_addr, + entry.slice->rdma.dest_rkey, + &entry.op_ctx->fi_ctx); + } + if (ret == 0) { + entry.slice->status = Transport::Slice::PENDING; + } else if (ret == -FI_EAGAIN) { + delete entry.op_ctx; + int not_posted = valid_count - i; + wr_depth_.fetch_sub(not_posted, std::memory_order_acq_rel); + if (cq_outstanding) + cq_outstanding->fetch_sub(not_posted, + std::memory_order_acq_rel); + for (int j = i; j < valid_count; j++) { + if (j > i) delete batch[j].op_ctx; + retry_slices.push_back(batch[j].slice); + } + break; + } else { + LOG(ERROR) + << "fi_read/fi_write failed: " << fi_strerror(-ret) + << " (source=" << entry.slice->source_addr + << ", len=" << entry.slice->length + << ", dest=" << (void*)entry.slice->rdma.dest_addr + << ", rkey=" << entry.slice->rdma.dest_rkey << ")"; + delete entry.op_ctx; + wr_depth_.fetch_sub(1, std::memory_order_acq_rel); + if (cq_outstanding) + cq_outstanding->fetch_sub(1, std::memory_order_acq_rel); + failed_slice_list.push_back(entry.slice); + } + } + post_lock_.clear(std::memory_order_release); + } + + cursor += batch_count; + } + + slice_list.clear(); + return 0; +} + int EfaContext::pollCq(int max_entries, int cq_index) { if (cq_index < 0 || (size_t)cq_index >= cq_list_.size()) { return 0; @@ -638,15 +862,13 @@ int EfaContext::pollCq(int max_entries, int cq_index) { struct fid_cq* cq = cq_list_[cq_index]->cq; if (!cq) return 0; - // Use fi_cq_data format for completions struct fi_cq_data_entry entries[64]; int to_poll = std::min(max_entries, 64); ssize_t ret = fi_cq_read(cq, entries, to_poll); if (ret > 0) { - // Process completions outside the lock (markSuccess / delete are safe) - std::unordered_map wr_depth_set; + std::unordered_map*, int> wr_depth_set; for (ssize_t i = 0; i < ret; i++) { EfaOpContext* op_ctx = reinterpret_cast(entries[i].op_context); @@ -659,18 +881,17 @@ int EfaContext::pollCq(int max_entries, int cq_index) { } } for (auto& entry : wr_depth_set) { - __sync_fetch_and_sub(entry.first, entry.second); + entry.first->fetch_sub(entry.second, std::memory_order_acq_rel); } - __sync_fetch_and_sub(&cq_list_[cq_index]->outstanding, - static_cast(ret)); + cq_list_[cq_index]->outstanding.fetch_sub(static_cast(ret), + std::memory_order_acq_rel); return static_cast(ret); } else if (ret == -FI_EAGAIN) { return 0; } else if (ret < 0) { - // CQ error - drain all queued error entries under the domain lock int err_count = 0; struct fi_cq_err_entry err_entry; - std::unordered_map wr_depth_set; + std::unordered_map*, int> wr_depth_set; while ((ret = fi_cq_readerr(cq, &err_entry, 0)) > 0) { EfaOpContext* op_ctx = @@ -690,10 +911,11 @@ int EfaContext::pollCq(int max_entries, int cq_index) { } for (auto& entry : wr_depth_set) { - __sync_fetch_and_sub(entry.first, entry.second); + entry.first->fetch_sub(entry.second, std::memory_order_acq_rel); } if (err_count > 0) { - __sync_fetch_and_sub(&cq_list_[cq_index]->outstanding, err_count); + cq_list_[cq_index]->outstanding.fetch_sub( + err_count, std::memory_order_acq_rel); } return err_count; } diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_endpoint.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_endpoint.cpp index 67a94cb6..29a1ca2a 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_endpoint.cpp @@ -16,122 +16,22 @@ #include -#include -#include -#include -#include +#include #include -#include #include "config.h" namespace mooncake { EfaEndPoint::EfaEndPoint(EfaContext& context) - : context_(context), - status_(INITIALIZING), - ep_(nullptr), - tx_cq_(nullptr), - rx_cq_(nullptr), - peer_fi_addr_(FI_ADDR_UNSPEC), - local_addr_len_(0), - wr_depth_(0), - max_wr_depth_(0), - cq_outstanding_(nullptr), - active_(true), - inactive_time_(0), - last_used_time_(getCurrentTimeInNano()) {} + : context_(context), status_(INITIALIZING), peer_fi_addr_(FI_ADDR_UNSPEC) {} -EfaEndPoint::~EfaEndPoint() { - if (ep_) deconstruct(); -} - -int EfaEndPoint::construct(struct fid_cq* cq, volatile int* cq_outstanding, - size_t num_qp_list, size_t max_sge, size_t max_wr, - size_t max_inline) { - if (status_.load(std::memory_order_relaxed) != INITIALIZING) { - LOG(ERROR) << "EFA Endpoint has already been constructed"; - return ERR_ENDPOINT; - } - - tx_cq_ = cq; - rx_cq_ = cq; // Use same CQ for TX and RX - max_wr_depth_ = max_wr; - cq_outstanding_ = cq_outstanding; - - // Create endpoint - int ret = fi_endpoint(context_.domain(), context_.info(), &ep_, nullptr); - if (ret) { - LOG(ERROR) << "fi_endpoint failed: " << fi_strerror(-ret); - return ERR_ENDPOINT; - } - - // Bind endpoint to AV - ret = fi_ep_bind(ep_, &context_.av()->fid, 0); - if (ret) { - LOG(ERROR) << "fi_ep_bind (av) failed: " << fi_strerror(-ret); - fi_close(&ep_->fid); - ep_ = nullptr; - return ERR_ENDPOINT; - } - - // Bind endpoint to TX CQ - ret = fi_ep_bind(ep_, &tx_cq_->fid, FI_TRANSMIT); - if (ret) { - LOG(ERROR) << "fi_ep_bind (tx_cq) failed: " << fi_strerror(-ret); - fi_close(&ep_->fid); - ep_ = nullptr; - return ERR_ENDPOINT; - } - - // Bind endpoint to RX CQ - ret = fi_ep_bind(ep_, &rx_cq_->fid, FI_RECV); - if (ret) { - LOG(ERROR) << "fi_ep_bind (rx_cq) failed: " << fi_strerror(-ret); - fi_close(&ep_->fid); - ep_ = nullptr; - return ERR_ENDPOINT; - } - - // Enable endpoint - ret = fi_enable(ep_); - if (ret) { - LOG(ERROR) << "fi_enable failed: " << fi_strerror(-ret); - fi_close(&ep_->fid); - ep_ = nullptr; - return ERR_ENDPOINT; - } - - // Get local endpoint address - local_addr_len_ = 64; // EFA addresses are typically 32 bytes - local_addr_.resize(local_addr_len_); - ret = fi_getname(&ep_->fid, local_addr_.data(), &local_addr_len_); - if (ret) { - LOG(ERROR) << "fi_getname failed: " << fi_strerror(-ret); - fi_close(&ep_->fid); - ep_ = nullptr; - return ERR_ENDPOINT; - } - local_addr_.resize(local_addr_len_); - - status_.store(UNCONNECTED, std::memory_order_relaxed); - return 0; -} - -int EfaEndPoint::deconstruct() { - if (ep_) { - fi_close(&ep_->fid); - ep_ = nullptr; - } - return 0; -} - -int EfaEndPoint::destroyQP() { return deconstruct(); } +EfaEndPoint::~EfaEndPoint() { disconnect(); } void EfaEndPoint::setPeerNicPath(const std::string& peer_nic_path) { RWSpinlock::WriteGuard guard(lock_); if (peer_nic_path_ == peer_nic_path) return; // No change - if (connected()) { + if (status_.load(std::memory_order_relaxed) == CONNECTED) { LOG(INFO) << "Peer reconnected with new address, re-establishing: " << peer_nic_path_ << " -> " << peer_nic_path; disconnectUnlocked(); @@ -139,62 +39,28 @@ void EfaEndPoint::setPeerNicPath(const std::string& peer_nic_path) { peer_nic_path_ = peer_nic_path; } -std::string EfaEndPoint::getLocalAddr() const { - std::ostringstream oss; - for (size_t i = 0; i < local_addr_.size(); ++i) { - oss << std::hex << std::setw(2) << std::setfill('0') - << (int)local_addr_[i]; - } - return oss.str(); -} - -int EfaEndPoint::insertPeerAddr(const std::string& peer_addr) { - // Convert hex string to binary address - std::vector addr_bin; - addr_bin.reserve(peer_addr.size() / 2); - - for (size_t i = 0; i < peer_addr.size(); i += 2) { - std::string byte_str = peer_addr.substr(i, 2); - uint8_t byte = (uint8_t)strtol(byte_str.c_str(), nullptr, 16); - addr_bin.push_back(byte); - } - - // Insert into address vector - int ret = fi_av_insert(context_.av(), addr_bin.data(), 1, &peer_fi_addr_, 0, - nullptr); - if (ret != 1) { - LOG(ERROR) << "fi_av_insert failed: " << fi_strerror(-ret); - return ERR_ENDPOINT; - } - - return 0; -} - int EfaEndPoint::setupConnectionsByActive() { RWSpinlock::WriteGuard guard(lock_); - if (connected()) { - LOG(INFO) << "EFA Connection has been established"; - return 0; - } + if (status_.load(std::memory_order_relaxed) == CONNECTED) return 0; - // Loopback mode + // Loopback: handshake against ourselves. Use the binary overload + // so we avoid hex-encoding the local address just to have + // insertPeerAddr decode it right back. if (context_.nicPath() == peer_nic_path_) { - // For loopback, insert our own address - int ret = insertPeerAddr(getLocalAddr()); - if (ret != 0) { - return ret; - } + const auto& bytes = context_.localEpAddrBytes(); + int ret = context_.insertPeerAddrBytes(bytes.data(), bytes.size(), + peer_fi_addr_); + if (ret != 0) return ret; status_.store(CONNECTED, std::memory_order_release); LOG(INFO) << "EFA loopback connection established: " << toString(); return 0; } - // Exchange addresses via handshake + // Exchange addresses via the transfer-metadata handshake RPC. TransferMetadata::HandShakeDesc local_desc, peer_desc; local_desc.local_nic_path = context_.nicPath(); local_desc.peer_nic_path = peer_nic_path_; - // Store our EFA endpoint address in efa_addr field (hex encoded) - local_desc.efa_addr = getLocalAddr(); + local_desc.efa_addr = context_.localEpAddr(); auto peer_server_name = getServerNameFromNicPath(peer_nic_path_); auto peer_nic_name = getNicNameFromNicPath(peer_nic_path_); @@ -212,14 +78,10 @@ int EfaEndPoint::setupConnectionsByActive() { return ERR_REJECT_HANDSHAKE; } - // Insert peer's address into our AV - rc = insertPeerAddr(peer_desc.efa_addr); - if (rc != 0) { - return rc; - } + rc = context_.insertPeerAddr(peer_desc.efa_addr, peer_fi_addr_); + if (rc != 0) return rc; status_.store(CONNECTED, std::memory_order_release); - touchLastUsed(); VLOG(1) << "EFA connection established: " << toString() << " peer_fi_addr=" << peer_fi_addr_; return 0; @@ -228,7 +90,7 @@ int EfaEndPoint::setupConnectionsByActive() { int EfaEndPoint::setupConnectionsByPassive(const HandShakeDesc& peer_desc, HandShakeDesc& local_desc) { RWSpinlock::WriteGuard guard(lock_); - if (connected()) { + if (status_.load(std::memory_order_relaxed) == CONNECTED) { LOG(WARNING) << "Re-establish EFA connection: " << toString(); disconnectUnlocked(); } @@ -244,27 +106,24 @@ int EfaEndPoint::setupConnectionsByPassive(const HandShakeDesc& peer_desc, return ERR_REJECT_HANDSHAKE; } - // Insert peer's address from handshake if (peer_desc.efa_addr.empty()) { local_desc.reply_msg = "No EFA address provided"; LOG(ERROR) << "Peer did not provide EFA address"; return ERR_REJECT_HANDSHAKE; } - int ret = insertPeerAddr(peer_desc.efa_addr); + int ret = context_.insertPeerAddr(peer_desc.efa_addr, peer_fi_addr_); if (ret != 0) { local_desc.reply_msg = "Failed to insert peer address"; return ret; } - // Provide our address to peer (using efa_addr field, not reply_msg) local_desc.local_nic_path = context_.nicPath(); local_desc.peer_nic_path = peer_nic_path_; - local_desc.efa_addr = getLocalAddr(); - // reply_msg should be empty on success + local_desc.efa_addr = context_.localEpAddr(); + // reply_msg empty on success status_.store(CONNECTED, std::memory_order_release); - touchLastUsed(); VLOG(1) << "EFA connection established (passive): " << toString(); return 0; } @@ -275,16 +134,15 @@ void EfaEndPoint::disconnect() { } void EfaEndPoint::disconnectUnlocked() { - // Remove peer address from AV to free the slot for reuse. - // Without this, AV entries leak on every initiator disconnect, - // eventually exhausting the AV and causing fi_av_insert/fi_endpoint - // failures ("No data available") after 3-5 initiator runs. if (peer_fi_addr_ != FI_ADDR_UNSPEC) { - int ret = fi_av_remove(context_.av(), &peer_fi_addr_, 1, 0); - if (ret) { - LOG(WARNING) << "fi_av_remove failed: " << fi_strerror(-ret); - } + context_.removePeerAddr(peer_fi_addr_); + peer_fi_addr_ = FI_ADDR_UNSPEC; } + status_.store(UNCONNECTED, std::memory_order_release); +} + +void EfaEndPoint::markDetachedForTeardown() { + RWSpinlock::WriteGuard guard(lock_); peer_fi_addr_ = FI_ADDR_UNSPEC; status_.store(UNCONNECTED, std::memory_order_release); } @@ -293,28 +151,12 @@ const std::string EfaEndPoint::toString() const { return "EfaEndPoint[" + context_.nicPath() + " <-> " + peer_nic_path_ + "]"; } -bool EfaEndPoint::hasOutstandingSlice() const { return wr_depth_ > 0; } - -int EfaEndPoint::doSetupConnection(const std::string& peer_addr, - std::string* reply_msg) { - int ret = insertPeerAddr(peer_addr); - if (ret != 0) { - if (reply_msg) *reply_msg = "Failed to insert peer address into AV"; - return ret; - } - - status_.store(CONNECTED, std::memory_order_release); - return 0; -} - int EfaEndPoint::submitPostSend( std::vector& slice_list, std::vector& failed_slice_list) { - if (!connected()) { - // Try to establish connection first + if (status_.load(std::memory_order_relaxed) != CONNECTED) { int ret = setupConnectionsByActive(); if (ret != 0) { - // Move all slices to failed list for (auto* slice : slice_list) { failed_slice_list.push_back(slice); } @@ -323,177 +165,18 @@ int EfaEndPoint::submitPostSend( } } - // Batched submission — mirrors RDMA's ibv_post_send approach: - // 1. Reserve N WR+CQ slots in bulk (single CAS each) - // 2. Prepare MR descriptors and op contexts outside the lock - // 3. Hold post_lock once for the entire batch of fi_write calls - // This reduces per-slice overhead from 2 CAS + 1 spinlock to amortized - // 2/N CAS + 1/N spinlock per slice. - const int kMaxBackoffYields = 100000; - const int cq_limit = static_cast(globalConfig().max_cqe); - - struct BatchEntry { - Transport::Slice* slice; - void* local_desc; - EfaOpContext* op_ctx; - }; - - while (!slice_list.empty()) { - // --- Step 1: Atomically reserve a batch of WR+CQ slots --- - int batch_count = 0; - int backoff = 0; - bool timed_out = false; - while (batch_count == 0) { - int cur_wr = wr_depth_; - int wr_avail = max_wr_depth_ - cur_wr; - if (wr_avail <= 0) { - if (++backoff > kMaxBackoffYields) { - timed_out = true; - break; - } - std::this_thread::yield(); - continue; - } - int want = std::min(wr_avail, (int)slice_list.size()); - if (cq_outstanding_) { - int cur_cq = *cq_outstanding_; - int cq_avail = cq_limit - cur_cq; - if (cq_avail <= 0) { - if (++backoff > kMaxBackoffYields) { - timed_out = true; - break; - } - std::this_thread::yield(); - continue; - } - want = std::min(want, cq_avail); - if (!__sync_bool_compare_and_swap(&wr_depth_, cur_wr, - cur_wr + want)) { - continue; - } - cur_cq = *cq_outstanding_; - cq_avail = cq_limit - cur_cq; - if (cq_avail < want) { - __sync_fetch_and_sub(&wr_depth_, want); - continue; - } - if (!__sync_bool_compare_and_swap(cq_outstanding_, cur_cq, - cur_cq + want)) { - __sync_fetch_and_sub(&wr_depth_, want); - continue; - } - } else { - if (!__sync_bool_compare_and_swap(&wr_depth_, cur_wr, - cur_wr + want)) { - continue; - } - } - batch_count = want; - } - - if (timed_out) { - LOG(WARNING) << "EFA submitPostSend: timed out waiting for CQ drain" - << " (wr_depth=" << wr_depth_ - << ", max=" << max_wr_depth_ << ", cq_outstanding=" - << (cq_outstanding_ ? *cq_outstanding_ : -1) - << ", max_cqe=" << cq_limit << ")"; - for (auto* slice : slice_list) { - failed_slice_list.push_back(slice); - } - slice_list.clear(); - return 0; - } - - // --- Step 2: Prepare MR descriptors outside the lock --- - std::vector batch(batch_count); - int valid_count = 0; - - for (int i = 0; i < batch_count; i++) { - Transport::Slice* slice = slice_list[i]; - void* local_desc = context_.mrDesc(slice->source_addr); - if (!local_desc) { - LOG(ERROR) << "No MR descriptor found for address " - << slice->source_addr; - failed_slice_list.push_back(slice); - continue; - } - EfaOpContext* op_ctx = new EfaOpContext(); - memset(op_ctx, 0, sizeof(EfaOpContext)); - op_ctx->slice = slice; - op_ctx->wr_depth = &wr_depth_; - - batch[valid_count++] = {slice, local_desc, op_ctx}; - } - - int mr_failures = batch_count - valid_count; - if (mr_failures > 0) { - __sync_fetch_and_sub(&wr_depth_, mr_failures); - if (cq_outstanding_) - __sync_fetch_and_sub(cq_outstanding_, mr_failures); - } - - // --- Step 3: Post entire batch under a single lock hold --- - std::vector retry_slices; - if (valid_count > 0) { - while (post_lock_.test_and_set(std::memory_order_acquire)) { - } - for (int i = 0; i < valid_count; i++) { - auto& entry = batch[i]; - ssize_t ret; - if (entry.slice->opcode == Transport::TransferRequest::READ) { - ret = fi_read(ep_, (void*)entry.slice->source_addr, - entry.slice->length, entry.local_desc, - peer_fi_addr_, entry.slice->rdma.dest_addr, - entry.slice->rdma.dest_rkey, - &entry.op_ctx->fi_ctx); - } else { - ret = fi_write(ep_, (void*)entry.slice->source_addr, - entry.slice->length, entry.local_desc, - peer_fi_addr_, entry.slice->rdma.dest_addr, - entry.slice->rdma.dest_rkey, - &entry.op_ctx->fi_ctx); - } - if (ret == 0) { - entry.slice->status = Transport::Slice::PENDING; - } else if (ret == -FI_EAGAIN) { - delete entry.op_ctx; - int not_posted = valid_count - i; - __sync_fetch_and_sub(&wr_depth_, not_posted); - if (cq_outstanding_) - __sync_fetch_and_sub(cq_outstanding_, not_posted); - for (int j = i; j < valid_count; j++) { - if (j > i) delete batch[j].op_ctx; - retry_slices.push_back(batch[j].slice); - } - break; - } else { - LOG(ERROR) - << "fi_read/fi_write failed: " << fi_strerror(-ret) - << " (source=" << entry.slice->source_addr - << ", len=" << entry.slice->length - << ", dest=" << (void*)entry.slice->rdma.dest_addr - << ", rkey=" << entry.slice->rdma.dest_rkey << ")"; - delete entry.op_ctx; - __sync_fetch_and_sub(&wr_depth_, 1); - if (cq_outstanding_) - __sync_fetch_and_sub(cq_outstanding_, 1); - failed_slice_list.push_back(entry.slice); - } - } - post_lock_.clear(std::memory_order_release); - touchLastUsed(); - } - - // Remove consumed batch from slice_list, prepend any retry slices - slice_list.erase(slice_list.begin(), slice_list.begin() + batch_count); - if (!retry_slices.empty()) { - slice_list.insert(slice_list.begin(), retry_slices.begin(), - retry_slices.end()); - std::this_thread::yield(); - } + fi_addr_t peer; + { + RWSpinlock::ReadGuard guard(lock_); + peer = peer_fi_addr_; + } + if (peer == FI_ADDR_UNSPEC) { + for (auto* slice : slice_list) failed_slice_list.push_back(slice); + slice_list.clear(); + return ERR_ENDPOINT; } - return 0; + return context_.submitSlicesOnPeer(peer, slice_list, failed_slice_list); } } // namespace mooncake diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp index d3a305c1..eeebf268 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp @@ -147,12 +147,10 @@ void EfaTransport::workerThreadFunc(int thread_id) { } } - // Stale endpoint eviction happens on-demand in - // EfaEndpointStore::getOrInsert when the store is at capacity, - // not periodically here. Periodic eviction is unsafe because - // target-side endpoints never get touchLastUsed() updates - // (the initiator writes remotely via fi_write, bypassing - // the target's endpoint code). + // Under the shared-endpoint model there is no per-peer QP to evict: + // a new peer costs one AV entry (~bytes), not an fi_endpoint slot. + // Stale peers are reclaimed when submitPostSend() drops a peer whose + // handshake failed. // If no work was done, yield CPU briefly if (!did_work) { @@ -406,7 +404,10 @@ int EfaTransport::registerLocalMemoryInternal(void* addr, size_t length, size_t chunk_len = chunks[ci].second; const auto& assigned_nics = nic_assignments[ci]; - bool do_pre_touch = context_list_.size() > 0 && + // preTouchMemory does a CPU-side store to each page, which segfaults + // on GPU VRAM (cudaMalloc'd pointers). Restrict it to host memory. + bool is_host_mem = resolved_name.rfind("cpu", 0) == 0; + bool do_pre_touch = is_host_mem && context_list_.size() > 0 && std::thread::hardware_concurrency() >= 4 && chunk_len >= (size_t)4 * 1024 * 1024 * 1024; if (do_pre_touch) { @@ -698,14 +699,32 @@ int EfaTransport::warmupSegment(const std::string& segment_name) { peer_paths.emplace_back(segment_name + "@" + dev.name); } - // Warm up every (local_ctx, peer_nic) pair concurrently. Each - // EfaContext::endpoint() + setupConnectionsByActive() is idempotent and - // takes its own lock, so parallel calls across distinct peer_nic_paths - // (and distinct contexts) are safe. We use std::async to get roughly - // per-pair parallelism — the critical path is now max(handshake RTT) not - // sum(handshake RTT). auto t0 = std::chrono::steady_clock::now(); size_t n_pairs = context_list_.size() * peer_paths.size(); + + // Idempotent short-circuit: if every (local_ctx, peer_nic) pair already + // has a connected endpoint, skip the whole async dispatch. Matters for + // callers that invoke warmupSegment per request loop — without this the + // 256-thread fan-out runs every time even though there is no work to do. + size_t already_ready = 0; + for (auto& ctx : context_list_) { + for (const auto& path : peer_paths) { + auto ep = ctx->peekEndpoint(path); + if (ep && ep->connected()) ++already_ready; + } + } + if (already_ready == n_pairs) { + VLOG(1) << "EfaTransport::warmupSegment('" << segment_name << "'): all " + << n_pairs << " endpoints already connected, " + << "skipping"; + return 0; + } + + // Warm up every (local_ctx, peer_nic) pair concurrently. Under the + // shared-endpoint model each warmup is just a handshake RPC + + // fi_av_insert (no fi_endpoint, no fi_enable), so the critical path is + // max(handshake RTT), not sum. We still dispatch with std::async for + // concurrency, but total wall time is typically ms-level. std::vector> futs; futs.reserve(n_pairs); for (auto& ctx : context_list_) { @@ -720,7 +739,15 @@ int EfaTransport::warmupSegment(const std::string& segment_name) { return -1; } if (ep->connected()) return 0; - return ep->setupConnectionsByActive(); + int rc = ep->setupConnectionsByActive(); + if (rc != 0) { + // Handshake failed: drop the peer handle so the AV + // slot is freed and the next warmup retry starts + // clean. Cheap under the shared-endpoint model — + // no fi_endpoint teardown required. + ctx->deleteEndpoint(normalizeNicPath(path)); + } + return rc; })); } } @@ -767,15 +794,6 @@ Status EfaTransport::submitTransferTask( auto local_segment_desc = metadata_->getSegmentDescByID(LOCAL_SEGMENT_ID); assert(local_segment_desc.get()); const int kMaxRetryCount = globalConfig().retry_cnt; - // Striping threshold: transfers larger than this stripe across all - // active local NICs (one chunk per NIC) instead of creating thousands - // of small slices. This dramatically reduces per-slice overhead - // (spinlock acquisition, atomic ops, MR lookup, heap allocation). - // Configurable via GlobalConfig (MC_EFA_STRIPING_THRESHOLD env var). - // Default 2MB: below this, single-NIC is faster (avoids endpoint overhead - // for 16 tiny chunks); above this, multi-NIC striping with pre-resolved - // peer info gives up to +54% throughput. - const size_t kStripingThreshold = globalConfig().efa_striping_threshold; for (size_t index = 0; index < task_list.size(); ++index) { assert(task_list[index]); @@ -794,97 +812,9 @@ Status EfaTransport::submitTransferTask( request_device_id = -1; } - if (request.length > kStripingThreshold && request_buffer_id >= 0) { - // LARGE TRANSFER: stripe across all active local NICs. - // Creates one slice per NIC instead of (length / 64KB) slices, - // e.g. 240MB transfer: 4-32 slices vs 3840 slices. - std::vector>> - active_devs; - for (int d = 0; d < static_cast(context_list_.size()); ++d) { - auto& ctx = context_list_[d]; - if (!ctx || !ctx->active()) continue; - if (static_cast(request_buffer_id) < - local_segment_desc->buffers.size() && - local_segment_desc->buffers[request_buffer_id].lkey.size() > - static_cast(d) && - local_segment_desc->buffers[request_buffer_id].lkey[d] != - 0) { - active_devs.push_back({d, ctx}); - } - } - - if (active_devs.empty()) { - LOG(ERROR) << "No active EFA device for striping transfer of " - << request.length << " bytes"; - for (auto& entry : slices_to_post) - for (auto s : entry.second) s->markFailed(); - return Status::AddressNotRegistered( - "No active EFA device found for striping"); - } - - size_t num_nics = active_devs.size(); - size_t chunk_size = request.length / num_nics; - - // Pre-resolve remote peer info ONCE for all striped slices. - // This eliminates per-slice overhead in submitPostSend: - // - getSegmentDescByID lookup - // - selectDevice() call - // - string construction for peer_nic_path - // Using retry_count=i distributes slices across remote NICs. - auto peer_segment_desc = - metadata_->getSegmentDescByID(request.target_id); - - for (size_t i = 0; i < num_nics; ++i) { - Slice* slice = getSliceCache().allocate(); - assert(slice); - slice->peer_nic_path.clear(); - slice->rdma.dest_rkey = 0; - - size_t offset = i * chunk_size; - size_t len = - (i == num_nics - 1) ? request.length - offset : chunk_size; - - slice->source_addr = (char*)request.source + offset; - slice->length = len; - slice->opcode = request.opcode; - slice->rdma.dest_addr = request.target_offset + offset; - slice->rdma.retry_cnt = request.advise_retry_cnt; - slice->rdma.max_retry_cnt = kMaxRetryCount; - slice->task = &task; - slice->target_id = request.target_id; - slice->status = Slice::PENDING; - slice->ts = 0; - task.slice_list.push_back(slice); - - int dev_id = active_devs[i].first; - auto& context = active_devs[i].second; - slice->rdma.source_lkey = - local_segment_desc->buffers[request_buffer_id].lkey[dev_id]; - - // Pre-resolve remote NIC: set dest_rkey and peer_nic_path - // so submitPostSend() can skip expensive per-slice lookups. - if (peer_segment_desc) { - int remote_buffer_id = -1, remote_device_id = -1; - if (selectDevice(peer_segment_desc.get(), - request.target_offset + offset, len, - remote_buffer_id, remote_device_id, - static_cast(i)) == 0) { - slice->rdma.dest_rkey = - peer_segment_desc->buffers[remote_buffer_id] - .rkey[remote_device_id]; - slice->peer_nic_path = - peer_segment_desc->name + "@" + - peer_segment_desc->devices[remote_device_id].name; - } - } - - slices_to_post[context].push_back(slice); - __sync_fetch_and_add(&task.total_bytes, slice->length); - __sync_fetch_and_add(&task.slice_count, 1); - } - } else if (request_buffer_id >= 0 && request_device_id >= 0) { - // SMALL TRANSFER (or single-NIC): one slice, no sub-slicing. - // Round-robin NIC selection is handled by selectDevice above. + if (request_buffer_id >= 0 && request_device_id >= 0) { + // One slice per request. Round-robin NIC selection is + // handled by selectDevice above. auto& context = context_list_[request_device_id]; if (!context || !context->active()) { LOG(ERROR) << "EFA Device " << request_device_id @@ -1093,10 +1023,8 @@ int EfaTransport::initializeEfaResources() { for (auto& device_name : efa_devices) { auto context = std::make_shared(*this, device_name); auto& config = globalConfig(); - int ret = context->construct(config.num_cq_per_ctx, - config.num_comp_channels_per_ctx, - config.port, config.gid_index, - config.max_cqe, config.max_ep_per_ctx); + int ret = context->construct(config.num_cq_per_ctx, config.max_cqe, + config.max_ep_per_ctx); if (ret) { local_topology_->disableDevice(device_name); LOG(WARNING) << "EfaTransport: Disable device " << device_name; diff --git a/mooncake-transfer-engine/tests/efa_transport_test.cpp b/mooncake-transfer-engine/tests/efa_transport_test.cpp index 066421ac..704a3384 100644 --- a/mooncake-transfer-engine/tests/efa_transport_test.cpp +++ b/mooncake-transfer-engine/tests/efa_transport_test.cpp @@ -21,6 +21,7 @@ #include #include "transfer_engine.h" +#include "transport/efa_transport/efa_transport.h" #include "transport/transport.h" using namespace mooncake; @@ -329,6 +330,176 @@ TEST_F(EFATransportTest, StressMultipleBatches) { destroyEngine(setup); } +// Test 6: warmupSegment on loopback peer +// +// Exercises EfaTransport::warmupSegment() which is the C++ entry point +// behind the warmup_efa_segment() Python binding / warmupEfaSegment() C API. +// Loopback is enough to cover the handshake + fi_av_insert path AND the +// idempotent short-circuit on the second call. +TEST_F(EFATransportTest, WarmupSegmentLoopback) { + auto setup = createEngine(); + + auto *efa = dynamic_cast(setup.xport); + ASSERT_NE(efa, nullptr) + << "installTransport did not return an EfaTransport"; + + // First call: should connect every (local NIC x peer NIC) pair. + int rc = efa->warmupSegment(setup.engine->getLocalIpAndPort()); + EXPECT_EQ(rc, 0) << "warmupSegment should succeed on loopback"; + + // Second call: should short-circuit (all endpoints already connected). + rc = efa->warmupSegment(setup.engine->getLocalIpAndPort()); + EXPECT_EQ(rc, 0) << "warmupSegment should be idempotent"; + + // Empty / self-name: short-circuit path returning 0 without touching AV. + rc = efa->warmupSegment(""); + EXPECT_EQ(rc, 0) << "warmupSegment(\"\") should be a no-op"; + + destroyEngine(setup); +} + +// Test 7: warmupSegment on a non-existent segment name should fail cleanly +// (no crash, no hang) rather than blocking for the poll timeout. +TEST_F(EFATransportTest, WarmupSegmentNotFound) { + auto setup = createEngine(); + + auto *efa = dynamic_cast(setup.xport); + ASSERT_NE(efa, nullptr); + + int rc = efa->warmupSegment("127.0.0.1:1"); // not openSegment'd + EXPECT_NE(rc, 0) << "warmupSegment should fail for unknown segment"; + + destroyEngine(setup); +} + +// Test 8: registerLocalMemoryBatch / unregisterLocalMemoryBatch round-trip. +// Covers the batched MR path which the single-buffer tests above never hit. +TEST_F(EFATransportTest, RegisterMemoryBatch) { + auto engine = std::make_unique(false); + engine->getLocalTopology()->discover({}); + auto hp = parseHostNameWithPort(local_server_name_); + int rc = engine->init(metadata_server_, local_server_name_, + hp.first.c_str(), hp.second); + ASSERT_EQ(rc, 0); + + Transport *xport = engine->installTransport("efa", nullptr); + ASSERT_NE(xport, nullptr); + + const size_t kBufSize = 4ull << 20; // 4 MB each + const int kNumBufs = 4; + std::vector addrs; + std::vector entries; + for (int i = 0; i < kNumBufs; ++i) { + void *a = allocateMemoryPool(kBufSize, 0); + ASSERT_NE(a, nullptr); + addrs.push_back(a); + entries.push_back({a, kBufSize}); + } + + rc = engine->registerLocalMemoryBatch(entries, "cpu:0"); + EXPECT_EQ(rc, 0) << "registerLocalMemoryBatch should succeed"; + + rc = engine->unregisterLocalMemoryBatch(addrs); + EXPECT_EQ(rc, 0) << "unregisterLocalMemoryBatch should succeed"; + + for (void *a : addrs) freeMemoryPool(a, kBufSize); +} + +// Test 9: Larger transfer (64 MB total split into 1 MB slices) to exercise +// the WR / CQ pacing logic in EfaContext::submitSlicesOnPeer beyond what the +// 16 x 64 KB MultiWrite test reaches. +TEST_F(EFATransportTest, LargeTransfer) { + const size_t kBufSize = 128ull << 20; // 128 MB + auto setup = createEngine(kBufSize); + + auto segment_desc = + setup.engine->getMetadata()->getSegmentDescByID(setup.segment_id); + ASSERT_NE(segment_desc, nullptr); + uint64_t remote_base = (uint64_t)segment_desc->buffers[0].addr; + + const size_t kSliceLen = 1ull << 20; // 1 MB per slice + const int kNumSlices = 64; // 64 MB total + ASSERT_LE(static_cast(kNumSlices) * kSliceLen, kBufSize / 2); + + // Fill first half with known data + uint8_t *buf = (uint8_t *)setup.addr; + for (size_t i = 0; i < static_cast(kNumSlices) * kSliceLen; ++i) + buf[i] = (uint8_t)(i & 0xFF); + + auto batch_id = setup.engine->allocateBatchID(kNumSlices); + std::vector requests; + requests.reserve(kNumSlices); + for (int i = 0; i < kNumSlices; ++i) { + TransferRequest entry; + entry.opcode = TransferRequest::WRITE; + entry.length = kSliceLen; + entry.source = buf + i * kSliceLen; + entry.target_id = setup.segment_id; + entry.target_offset = remote_base + (kBufSize / 2) + i * kSliceLen; + requests.push_back(entry); + } + + Status s = setup.engine->submitTransfer(batch_id, requests); + ASSERT_TRUE(s.ok()) << "submitTransfer failed: " << s.ToString(); + + for (int task_id = 0; task_id < kNumSlices; ++task_id) { + TransferStatus status; + const int kMaxPollIterations = 2000000; + int i = 0; + for (; i < kMaxPollIterations; ++i) { + s = setup.engine->getTransferStatus(batch_id, task_id, status); + ASSERT_TRUE(s.ok()); + if (status.s == TransferStatusEnum::COMPLETED) break; + ASSERT_NE(status.s, TransferStatusEnum::FAILED); + } + ASSERT_EQ(status.s, TransferStatusEnum::COMPLETED) + << "task " << task_id << " did not complete"; + } + + s = setup.engine->freeBatchID(batch_id); + ASSERT_TRUE(s.ok()); + + // Verify byte-level integrity of the last slice (spot check). + EXPECT_EQ(0, memcmp(buf + (kNumSlices - 1) * kSliceLen, + buf + (kBufSize / 2) + (kNumSlices - 1) * kSliceLen, + kSliceLen)); + + destroyEngine(setup); +} + +// Test 10: Repeated open/close of the same remote segment must not leak AV +// slots or break loopback transfers — this is the setPeerNicPath-detach path +// that target restarts depend on under the SRD shared-endpoint model. +TEST_F(EFATransportTest, RepeatedOpenSegment) { + auto setup = createEngine(); + + auto actual_addr = setup.engine->getLocalIpAndPort(); + + // First write via setup.segment_id (from createEngine()). + auto segment_desc = + setup.engine->getMetadata()->getSegmentDescByID(setup.segment_id); + ASSERT_NE(segment_desc, nullptr); + uint64_t remote_base = (uint64_t)segment_desc->buffers[0].addr; + memset(setup.addr, 0xCD, 4096); + EXPECT_TRUE(submitAndWait(setup.engine.get(), setup.segment_id, setup.addr, + remote_base, 4096, TransferRequest::WRITE)); + + // Re-open same segment several times; each should return a working handle + // and subsequent writes should still succeed. + for (int i = 0; i < 5; ++i) { + SegmentID sid = setup.engine->openSegment(actual_addr); + ASSERT_NE(sid, (SegmentID)-1); + auto desc = setup.engine->getMetadata()->getSegmentDescByID(sid); + ASSERT_NE(desc, nullptr); + uint64_t base = (uint64_t)desc->buffers[0].addr; + ASSERT_TRUE(submitAndWait(setup.engine.get(), sid, setup.addr, base, + 4096, TransferRequest::WRITE)) + << "write #" << i << " after re-open failed"; + } + + destroyEngine(setup); +} + } // namespace mooncake int main(int argc, char **argv) { -- 2.34.1 From 0a7e38fd190a967d7922742476243a52e37c2257 Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Thu, 23 Apr 2026 22:47:44 +0800 Subject: [PATCH 002/382] [Build] Upgrade yalantinglibs to 6a0e067d9a43492cf8e4e280b531924fbd724dbd (#1961) --- extern/yalantinglibs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/extern/yalantinglibs b/extern/yalantinglibs index 73dea196..6a0e067d 160000 --- a/extern/yalantinglibs +++ b/extern/yalantinglibs @@ -1 +1 @@ -Subproject commit 73dea196d23ad8fcd4914c6ef1238f390b9a1c48 +Subproject commit 6a0e067d9a43492cf8e4e280b531924fbd724dbd -- 2.34.1 From c251eefa4f81cb3f8ee3913dc163e298d761c8ba Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Fri, 24 Apr 2026 09:42:05 +0800 Subject: [PATCH 003/382] [TE] reduce reg overhead (#1964) --- .../include/transfer_engine_impl.h | 16 ++- .../transport/rdma_transport/rdma_context.h | 10 +- .../src/transfer_engine_impl.cpp | 117 ++++++++++++------ .../transport/rdma_transport/rdma_context.cpp | 76 +++++++----- 4 files changed, 151 insertions(+), 68 deletions(-) diff --git a/mooncake-transfer-engine/include/transfer_engine_impl.h b/mooncake-transfer-engine/include/transfer_engine_impl.h index 4016d822..db2357e2 100644 --- a/mooncake-transfer-engine/include/transfer_engine_impl.h +++ b/mooncake-transfer-engine/include/transfer_engine_impl.h @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -379,11 +380,24 @@ class TransferEngineImpl { bool remote_accessible; }; + using MemoryRegionMap = std::map; + + MemoryRegionMap::iterator findMemoryRegionContaining(uintptr_t addr); + + MemoryRegionMap::const_iterator findMemoryRegionContaining( + uintptr_t addr) const; + + bool hasOverlapLocked(uintptr_t addr, uint64_t length) const; + + void insertMemoryRegionLocked(const MemoryRegion& region); + + void eraseMemoryRegionLocked(void* addr); + std::shared_ptr metadata_; std::string local_server_name_; std::shared_ptr multi_transports_; std::shared_mutex mutex_; - std::vector local_memory_regions_; + MemoryRegionMap local_memory_regions_; std::shared_ptr local_topology_; RWSpinlock send_notifies_lock_; diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h index 825de6a1..299cbb77 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -90,6 +91,13 @@ class RdmaContext { int registerMemoryRegionInternal(void *addr, size_t length, int access, MemoryRegionMeta &mrMeta); + using MemoryRegionMap = std::map; + + MemoryRegionMap::iterator findMemoryRegionContaining(uintptr_t addr); + + MemoryRegionMap::const_iterator findMemoryRegionContaining( + uintptr_t addr) const; + public: bool active() const { return active_; } @@ -187,7 +195,7 @@ class RdmaContext { ibv_gid gid_; RWSpinlock memory_regions_lock_; - std::vector memory_region_list_; + MemoryRegionMap memory_region_map_; std::vector cq_list_; std::shared_ptr endpoint_store_; diff --git a/mooncake-transfer-engine/src/transfer_engine_impl.cpp b/mooncake-transfer-engine/src/transfer_engine_impl.cpp index 9caeffda..3d8a5aea 100644 --- a/mooncake-transfer-engine/src/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/src/transfer_engine_impl.cpp @@ -35,6 +35,14 @@ namespace mooncake { +namespace { +bool overlapWithRegion(uintptr_t addr, uint64_t length, void* region_addr, + uint64_t region_length) { + return overlap(reinterpret_cast(addr), length, region_addr, + region_length); +} +} // namespace + static bool setFilesLimit() { struct rlimit filesLimit; if (getrlimit(RLIMIT_NOFILE, &filesLimit) != 0) { @@ -384,7 +392,7 @@ Transport* TransferEngineImpl::installTransport(const std::string& proto, // shared lock here. If future modifications allow installTransport() to be // invoked concurrently, a std::shared_lock should be // added to ensure thread safety. - for (auto& entry : local_memory_regions_) { + for (auto& [_, entry] : local_memory_regions_) { int ret = transport->registerLocalMemory( entry.addr, entry.length, entry.location, entry.remote_accessible); if (ret < 0) return nullptr; @@ -488,13 +496,7 @@ int TransferEngineImpl::removeLocalSegment(const std::string& segment_name) { bool TransferEngineImpl::checkOverlap(void* addr, uint64_t length) { std::shared_lock lock(mutex_); - for (auto& local_memory_region : local_memory_regions_) { - if (overlap(addr, length, local_memory_region.addr, - local_memory_region.length)) { - return true; - } - } - return false; + return hasOverlapLocked(reinterpret_cast(addr), length); } int TransferEngineImpl::registerLocalMemory(void* addr, size_t length, @@ -518,8 +520,7 @@ int TransferEngineImpl::registerLocalMemory(void* addr, size_t length, } std::unique_lock lock(mutex_); - local_memory_regions_.push_back( - {addr, length, location, remote_accessible}); + insertMemoryRegionLocked({addr, length, location, remote_accessible}); return 0; } @@ -531,13 +532,7 @@ int TransferEngineImpl::unregisterLocalMemory(void* addr, } std::unique_lock lock(mutex_); - for (auto it = local_memory_regions_.begin(); - it != local_memory_regions_.end(); ++it) { - if (it->addr == addr) { - local_memory_regions_.erase(it); - break; - } - } + eraseMemoryRegionLocked(addr); return 0; } @@ -611,9 +606,9 @@ int TransferEngineImpl::mp_registerLocalMemory( { std::unique_lock lock(mutex_); for (const auto& record : success_records) { - local_memory_regions_.push_back({record.addr, record.length, - record.location, - record.remote_accessible}); + insertMemoryRegionLocked({record.addr, record.length, + record.location, + record.remote_accessible}); } } @@ -654,13 +649,7 @@ int TransferEngineImpl::mp_unregisterLocalMemory( std::unique_lock lock(mutex_); for (const auto& buffer : buffer_list) { - for (auto it = local_memory_regions_.begin(); - it != local_memory_regions_.end(); ++it) { - if (it->addr == buffer.addr) { - local_memory_regions_.erase(it); - break; - } - } + eraseMemoryRegionLocked(buffer.addr); } } return 0; @@ -683,8 +672,7 @@ int TransferEngineImpl::registerLocalMemoryBatch( std::unique_lock lock(mutex_); for (auto& buffer : buffer_list) { - local_memory_regions_.push_back( - {buffer.addr, buffer.length, location, true}); + insertMemoryRegionLocked({buffer.addr, buffer.length, location, true}); } return 0; } @@ -698,17 +686,74 @@ int TransferEngineImpl::unregisterLocalMemoryBatch( std::unique_lock lock(mutex_); for (auto& addr : addr_list) { - for (auto it = local_memory_regions_.begin(); - it != local_memory_regions_.end(); ++it) { - if (it->addr == addr) { - local_memory_regions_.erase(it); - break; - } - } + eraseMemoryRegionLocked(addr); } return 0; } +TransferEngineImpl::MemoryRegionMap::iterator +TransferEngineImpl::findMemoryRegionContaining(uintptr_t addr) { + auto upper = local_memory_regions_.upper_bound(addr); + if (upper == local_memory_regions_.begin()) { + return local_memory_regions_.end(); + } + auto candidate = std::prev(upper); + return overlapWithRegion(addr, 1, candidate->second.addr, + candidate->second.length) + ? candidate + : local_memory_regions_.end(); +} + +TransferEngineImpl::MemoryRegionMap::const_iterator +TransferEngineImpl::findMemoryRegionContaining(uintptr_t addr) const { + auto upper = local_memory_regions_.upper_bound(addr); + if (upper == local_memory_regions_.begin()) { + return local_memory_regions_.end(); + } + auto candidate = std::prev(upper); + return overlapWithRegion(addr, 1, candidate->second.addr, + candidate->second.length) + ? candidate + : local_memory_regions_.end(); +} + +bool TransferEngineImpl::hasOverlapLocked(uintptr_t addr, + uint64_t length) const { + if (length == 0) { + return false; + } + + auto containing = findMemoryRegionContaining(addr); + if (containing != local_memory_regions_.end()) { + return true; + } + + auto next = local_memory_regions_.lower_bound(addr); + if (next != local_memory_regions_.end() && + overlapWithRegion(addr, length, next->second.addr, + next->second.length)) { + return true; + } + + if (next != local_memory_regions_.begin()) { + auto prev = std::prev(next); + if (overlapWithRegion(addr, length, prev->second.addr, + prev->second.length)) { + return true; + } + } + + return false; +} + +void TransferEngineImpl::insertMemoryRegionLocked(const MemoryRegion& region) { + local_memory_regions_[reinterpret_cast(region.addr)] = region; +} + +void TransferEngineImpl::eraseMemoryRegionLocked(void* addr) { + local_memory_regions_.erase(reinterpret_cast(addr)); +} + #ifdef WITH_METRICS // Helper function to convert string to lowercase for case-insensitive // comparison diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index 282a4ea6..59746324 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -39,6 +39,14 @@ static int isNullGid(union ibv_gid *gid) { return 1; } +namespace { +bool containsAddress(const MemoryRegionMeta ®ion, uintptr_t addr) { + const auto region_start = reinterpret_cast(region.addr); + const auto region_length = static_cast(region.mr->length); + return region_start <= addr && addr - region_start < region_length; +} +} // namespace + RdmaContext::RdmaContext(RdmaTransport &engine, const std::string &device_name) : device_name_(device_name), engine_(engine), @@ -162,13 +170,13 @@ int RdmaContext::deconstruct() { endpoint_store_->destroyQPs(); - for (auto &entry : memory_region_list_) { + for (auto &[_, entry] : memory_region_map_) { int ret = ibv_dereg_mr(entry.mr); if (ret) { PLOG(ERROR) << "Failed to unregister memory region"; } } - memory_region_list_.clear(); + memory_region_map_.clear(); for (size_t i = 0; i < cq_list_.size(); ++i) { if (!cq_list_[i].native) continue; @@ -275,29 +283,21 @@ int RdmaContext::registerMemoryRegion(void *addr, size_t length, int access) { return ret; } RWSpinlock::WriteGuard guard(memory_regions_lock_); - memory_region_list_.push_back(mrMeta); + memory_region_map_[reinterpret_cast(mrMeta.addr)] = mrMeta; return 0; } int RdmaContext::unregisterMemoryRegion(void *addr) { RWSpinlock::WriteGuard guard(memory_regions_lock_); - bool has_removed; - do { - has_removed = false; - for (auto iter = memory_region_list_.begin(); - iter != memory_region_list_.end(); ++iter) { - if (iter->addr <= addr && - addr < (char *)(iter->addr) + iter->mr->length) { - if (ibv_dereg_mr(iter->mr)) { - LOG(ERROR) << "Failed to unregister memory " << addr; - return ERR_CONTEXT; - } - memory_region_list_.erase(iter); - has_removed = true; - break; - } - } - } while (has_removed); + auto iter = findMemoryRegionContaining(reinterpret_cast(addr)); + if (iter == memory_region_map_.end()) { + return 0; + } + if (ibv_dereg_mr(iter->second.mr)) { + LOG(ERROR) << "Failed to unregister memory " << addr; + return ERR_CONTEXT; + } + memory_region_map_.erase(iter); return 0; } @@ -313,11 +313,8 @@ int RdmaContext::preTouchMemory(void *addr, size_t length) { uint32_t RdmaContext::rkey(void *addr) { RWSpinlock::ReadGuard guard(memory_regions_lock_); - for (auto iter = memory_region_list_.begin(); - iter != memory_region_list_.end(); ++iter) - if (iter->addr <= addr && - addr < (char *)(iter->addr) + iter->mr->length) - return iter->mr->rkey; + auto iter = findMemoryRegionContaining(reinterpret_cast(addr)); + if (iter != memory_region_map_.end()) return iter->second.mr->rkey; LOG(ERROR) << "Address " << addr << " rkey not found for " << deviceName(); return 0; @@ -325,16 +322,35 @@ uint32_t RdmaContext::rkey(void *addr) { uint32_t RdmaContext::lkey(void *addr) { RWSpinlock::ReadGuard guard(memory_regions_lock_); - for (auto iter = memory_region_list_.begin(); - iter != memory_region_list_.end(); ++iter) - if (iter->addr <= addr && - addr < (char *)(iter->addr) + iter->mr->length) - return iter->mr->lkey; + auto iter = findMemoryRegionContaining(reinterpret_cast(addr)); + if (iter != memory_region_map_.end()) return iter->second.mr->lkey; LOG(ERROR) << "Address " << addr << " lkey not found for " << deviceName(); return 0; } +RdmaContext::MemoryRegionMap::iterator RdmaContext::findMemoryRegionContaining( + uintptr_t addr) { + auto upper = memory_region_map_.upper_bound(addr); + if (upper == memory_region_map_.begin()) { + return memory_region_map_.end(); + } + auto candidate = std::prev(upper); + return containsAddress(candidate->second, addr) ? candidate + : memory_region_map_.end(); +} + +RdmaContext::MemoryRegionMap::const_iterator +RdmaContext::findMemoryRegionContaining(uintptr_t addr) const { + auto upper = memory_region_map_.upper_bound(addr); + if (upper == memory_region_map_.begin()) { + return memory_region_map_.end(); + } + auto candidate = std::prev(upper); + return containsAddress(candidate->second, addr) ? candidate + : memory_region_map_.end(); +} + std::shared_ptr RdmaContext::endpoint( const std::string &peer_nic_path) { if (!active_) { -- 2.34.1 From 889326a8482e3259e70973722277ae42fa0f8f50 Mon Sep 17 00:00:00 2001 From: yejj Date: Mon, 27 Apr 2026 01:10:13 +0800 Subject: [PATCH 004/382] [Doc] update conductor design & api docs (#1977) --- .../conductor-architecture-design.md | 182 ++++++ .../design/conductor/indexer-api-design.md | 536 +++++++++++------- docs/source/index.md | 2 + 3 files changed, 521 insertions(+), 199 deletions(-) create mode 100644 docs/source/design/conductor/conductor-architecture-design.md diff --git a/docs/source/design/conductor/conductor-architecture-design.md b/docs/source/design/conductor/conductor-architecture-design.md new file mode 100644 index 00000000..e21cb74e --- /dev/null +++ b/docs/source/design/conductor/conductor-architecture-design.md @@ -0,0 +1,182 @@ +# Mooncake Conductor Architecture + +## Overview + +Mooncake Conductor is the kv-cache indexer used by cache-aware routers. It +subscribes to KV cache events from inference engines or storage backends, +normalizes those events, maintains a global prefix cache table, and exposes +HTTP APIs for dynamic service registration and cache-hit queries. + +The design goal is to let routers answer a simple scheduling question: +for this request prefix, which registered instance has the best reusable KV +cache locality, and on which cache tiers is that prefix available? + +## Architecture + +```mermaid +flowchart TD + subgraph Router["Gateway or router"] + HTTPClient["HTTP client"] + end + + subgraph Conductor["Mooncake Conductor"] + EventManager["EventManager"] + ZMQClient["ZMQClient"] + KVEventHandler["KVEventHandler"] + PrefixCacheTable["PrefixCacheTable"] + end + + subgraph Publishers["vLLM, SGLang, Mooncake, or cache daemons"] + InferenceService["Inference or cache service"] + KVEvent["KV event stream"] + end + + EventManager -->|creates and manages| ZMQClient + ZMQClient -->|registers handler| KVEventHandler + InferenceService -->|publishes events| KVEvent + KVEvent -->|ZMQ SUB or DEALER| ZMQClient + ZMQClient -->|decoded batches| KVEventHandler + KVEventHandler -->|normalizes events| PrefixCacheTable + HTTPClient -->|/query, /register, /unregister| EventManager + EventManager -->|cache-hit computation| PrefixCacheTable + PrefixCacheTable -->|hit result by instance| EventManager + EventManager -->|HTTP response| HTTPClient +``` + +## Components + +| Component | Responsibility | +|---|---| +| `EventManager` | Owns the Conductor lifecycle, HTTP server, dynamic registration, active service map, and tenant-to-instance map. | +| `ZMQClient` | Connects to publisher endpoints, consumes event frames, decodes event batches, tracks sequence numbers, and requests replay after reconnects. | +| `KVEventHandler` | Adapts decoded engine events into Conductor store/remove events enriched with registration metadata. | +| `PrefixCacheTable` | Maintains model-context-specific prefix maps, engine-hash to conductor-hash mappings, medium metadata, DP-rank metadata, and query-time hit computation. | + + +## Data model + +The prefix index is scoped by `ModelContext`: + +```text +(tenant_id, model_name, lora_name, block_size, additional_salt, instance_id) +``` + +Within each context, Conductor stores: + +- a mapping from engine-provided block hash to Conductor prefix hash; +- a prefix hash map that records replica count, medium set, DP-rank set, and + per-instance access metadata; +- a DP-rank set used to report rank-level hit information. + +The current implementation computes complete-block prefix hashes from token IDs +and ignores trailing partial blocks during `/query`. + +## Event flow + +1. A service is registered statically from `conductor_config.json` or + dynamically through `POST /register`. +2. `EventManager` creates one `ZMQClient` per `(instance_id, tenant_id, + dp_rank)` service key. +3. `ZMQClient` subscribes to the publisher endpoint and consumes frames in the + form `[topic, sequence, payload]`. +4. The payload is decoded into a batch of engine events. Today, the implemented + parser supports vLLM `BlockStored` and `BlockRemoved` msgpack events. +5. `KVEventHandler` enriches events with registration metadata such as model, + LoRA, tenant, instance, block size, and additional salt. +6. `PrefixCacheTable` updates the prefix map for stored or removed blocks. +7. If a reconnect detects missed sequence numbers, the `replay endpoint` can be + used to request missed events. + +## Query flow + +1. A router obtains prompt token IDs, usually from an engine tokenizer endpoint. +2. The router calls `POST /query` with `model`, `token_ids`, `block_size`, and + optional `tenant_id`, `instance_id`, `lora_name`, and `cache_salt`. +3. Conductor computes complete-block prefix hashes for the request. +4. The prefix table is scanned in order. The first miss terminates the scan so + prefix continuity is preserved. +5. Conductor returns per-instance `longest_matched`, medium hit counts, and + DP-rank hit counts. +6. The router selects the best target instance and forwards the request. + +## Dynamic registration + +Conductor supports runtime registration so routers or control planes can add +and remove KV event publishers without restarting the process. + +```json +{ + "endpoint": "tcp://127.0.0.1:5557", + "replay_endpoint": "tcp://127.0.0.1:5558", + "type": "vLLM", + "modelname": "qwen2.5", + "lora_name": "", + "tenant_id": "default", + "instance_id": "vllm-prefill-node1", + "block_size": 128, + "dp_rank": 0, + "additionalsalt": "" +} +``` + +Static configuration uses the same fields under `kvevent_instance`: + +```json +{ + "http_server_port": 13333, + "kvevent_instance": { + "vllm-prefill-node1": { + "endpoint": "tcp://127.0.0.1:5557", + "replay_endpoint": "tcp://127.0.0.1:5558", + "type": "vLLM", + "modelname": "qwen2.5", + "lora_name": "", + "tenant_id": "default", + "instance_id": "vllm-prefill-node1", + "block_size": 128, + "dp_rank": 0, + "additionalsalt": "" + } + } +} +``` + +## Environment variables + +| Variable | Default | Description | +|---|---|---| +| `CONDUCTOR_LOG_LEVEL` | `INFO` | Log level: `DEBUG`, `INFO`, `WARN`, or `ERROR`. | +| `CONDUCTOR_CONFIG_PATH` | `~/.mooncake/conductor_config.json` | Path to the static configuration file. | +| `CONDUCTOR_SEED` | random | Legacy seed option for hash computation experiments. | + +## Build and run + +```bash +cd mooncake-conductor/conductor-ctrl +go mod tidy +go build -o mooncake_conductor . +``` + +```bash +export CONDUCTOR_CONFIG_PATH=../example/conductor_config.json +export CONDUCTOR_LOG_LEVEL=INFO +./mooncake_conductor +``` + +## Project structure + +```text +mooncake-conductor/ ++-- conductor-ctrl/ +| +-- common/ # shared types, helpers, and sync map +| +-- kvevent/ # EventManager and KVEventHandler +| +-- prefixindex/ # prefix cache table and hit computation +| +-- zmq/ # ZMQ client, event decoding, event types +| +-- main.go # process entry point ++-- example/ # demo config and cache-aware proxy ++-- build.sh ++-- CMakeLists.txt +``` + +See [Indexer API](./indexer-api-design.md) for the HTTP API and KV Events wire +format. diff --git a/docs/source/design/conductor/indexer-api-design.md b/docs/source/design/conductor/indexer-api-design.md index 876cfaf6..e79367e6 100644 --- a/docs/source/design/conductor/indexer-api-design.md +++ b/docs/source/design/conductor/indexer-api-design.md @@ -1,224 +1,362 @@ -# Mooncake Conductor Indexer +# Mooncake Conductor Indexer API -## Introduction -The Mooncake Conductor Indexer is a specialized service designed to efficiently track and report token hit counts across various caching levels for different model instances. It provides a list of APIs that allow users to query token hit statistics based on token ID or chunked token hash, thereby facilitating optimized the performance of LLM inference.The figure below illustrates the architecture of Mooncake KVindexer: ![Mooncake KVindexer](../../image/conductor/architecture.png) +## Overview -## tiered storage & pools -We drew inspiration from the definition of [KVBM components](https://github.com/ai-dynamo/dynamo/blob/main/docs/kvbm/kvbm_components.md) and divided the KV cache into three levels: G1, G2, and G3. The detailed introduction is as follows: +Mooncake Conductor is a KV cache indexer used by routers and gateways to make +cache-aware scheduling decisions. It consumes KV cache events from inference +engines or storage backends, maintains prefix-hit metadata across cache tiers, +and exposes HTTP APIs for service registration and cache-hit queries. -- **Device Pool(G1)**: Device-resident KV block pool. Allocates mutable device blocks, registers completed blocks (immutable), serves lookups by sequence hash, and is the target for onboarding (Host→Device, Disk→Device). -- **Host Pool(G2)**: Mooncake registered memory KV pool. Receives Device offloads (Device→Host), can onboard to Device (Host→Device), and offloads to Disk. For high-performance, zero-copy data transfers, it utilizes the Mooncake Transfer-Engine. -- **Disk Pool(G3)**: SSD NVMe-backed KV pool. Receives Host offloads (Host→Disk), and provides large space for storing KV. +This document incorporates the latest API direction from: +- [RFC #1403: Mooncake KV-Store Indexer API Standardization](https://github.com/kvcache-ai/Mooncake/issues/1403) +- [RFC #1527: KV Events API Standardization](https://github.com/kvcache-ai/Mooncake/issues/1527) -## Indexer API +The Conductor can serve multiple model groups in one process. Each query is +scoped by model identity, block size, LoRA identity, tenant isolation, and the +registered instance that can receive traffic. + +## Concepts + +### Storage tiers + +The indexer tracks KV cache availability across three logical tiers: + +- **G1, Device Pool**: Device-resident KV blocks, such as GPU, NPU, HBM, or + other accelerator memory owned by inference engines. +- **G2, Host Pool**: CPU or host DRAM KV blocks, including Mooncake registered + memory pools. +- **G3, Disk Pool**: SSD, 3FS, DFS, NFS, or other disk-backed KV storage. + +The `medium` field identifies the concrete tier or device type. Common values +are `gpu`, `cpu`, and `disk`. Engines may add other values as new media are +supported. + +### Identity dimensions + +KV cache hits are interpreted under the following dimensions: + +| Dimension | Description | +|---|---| +| `model_name` or `model` | Model identifier. KV blocks from different models are incompatible. | +| `block_size` | Number of tokens per KV block. Different block sizes produce different token-to-block mappings. | +| `additional_salt` | Opaque salt used to separate hash namespaces for quantization, model revision, tenant isolation, or other deployment-specific dimensions. | +|`cache_salt`| Ensure cached data blocks are kept separate for different customers| +| `lora_name` | LoRA adapter name. Empty or `null` means the base model. | +| `tenant_id` | Upstream tenant or customer identity. Used for isolation and to keep query output bounded. | +| `instance_id` | Routable API server or engine instance returned by the Indexer API. Routers use this value as the scheduling target. | +| `backend_id` | KV Events identity for the entity that owns the KV blocks. It may be an inference worker, a Mooncake storage daemon, or another cache backend. | +| `medium` | Cache medium where the blocks are present. | +| `dp_rank` | Data-parallel rank that owns or can serve the blocks. | + +`instance_id` and `backend_id` intentionally have different meanings. +`instance_id` is the router-facing target in the Indexer API. `backend_id` is +the event-facing cache owner in the KV Events API. In deployments where cache +storage is decoupled from inference workers, `backend_id` can identify a cache +daemon while `instance_id` still identifies the engine endpoint that receives +requests. + +## Hashing standard + +The standardized event contract recommends **XXH3-64 with seed `S`**. + +- **Local block hash**: + `XXH3(token_bytes_le, S)`, where tokens are little-endian `u32` values + concatenated for one block. +- **Rolling sequence hash**: + The first block uses `seq_hash[0] = local_block_hash[0]`. Each subsequent + block uses: + + ```text + seq_hash[i] = XXH3(seq_hash[i-1]_le || local_block_hash[i]_le, S) + ``` + + Here `||` means byte concatenation, not a logical OR. + +All hashes used by the standardized KV Events API are rolling sequence hashes. +A `seq_hash` identifies the whole prefix up to that block depth, so equal +prefixes produce equal hashes until the first differing block. + +If an engine does not follow the standardized hashing scheme, it must provide +`token_ids` in `stored` events so the consumer can recompute the indexer's hash +representation. + +## HTTP APIs + +### `POST /register` + +Registers a KV event publisher and starts consuming events from it. + +```json +{ + "endpoint": "tcp://1.1.1.1:5557", + "replay_endpoint": "tcp://1.1.1.1:5558", + "type": "vLLM", + "modelname": "deepseek", + "lora_name": "sql-adapter", + "tenant_id": "default", + "instance_id": "vllm-prefill-node1", + "block_size": 128, + "dp_rank": 0, + "additionalsalt": "w8a8" +} +``` + +| Field | Required | Description | +|---|---|---| +| `endpoint` | Yes | ZMQ KV event publisher endpoint. | +| `replay_endpoint` | No | ZMQ replay endpoint used to recover missed events. | +| `type` | Yes | Publisher type, such as `vLLM`, `SGLang`, or `Mooncake`. | +| `modelname` | Yes | Model name for this publisher. This is the HTTP API wire name for `model_name`. | +| `lora_name` | No | LoRA adapter name. Empty or omitted means base model. | +| `tenant_id` | No | Tenant identity. Defaults to `default`. | +| `instance_id` | Yes | Router-facing engine or API server instance identity. | +| `block_size` | Yes | KV block size in tokens. | +| `dp_rank` | Yes | Data-parallel rank for this publisher. | +| `additionalsalt` | No | HTTP API wire name for `additional_salt`. Defaults to an empty string. | + +Successful response: + +```json +{ + "status": "registered successfully", + "instance_id": "vllm-prefill-node1" +} +``` + +### `POST /unregister` + +Stops consuming events for a registered publisher. + +```json +{ + "type": "vLLM", + "modelname": "deepseek", + "lora_name": "sql-adapter", + "tenant_id": "default", + "instance_id": "vllm-prefill-node1", + "block_size": 128, + "dp_rank": 0 +} +``` + +`tenant_id` defaults to `default`. The current implementation removes the +subscription identified by `(instance_id, tenant_id, dp_rank)`. + +Successful response: + +```json +{ + "status": "unregistered successfully", + "removed_instances": ["vllm-prefill-node1|default|0"] +} +``` ### `POST /query` - query token hit count. -- **Input**: - - **Body** (JSON): - ```json - { - "model": "deepseek", - "lora_name": "xx-adapter", - "lora_id": 12, // defined for backward compatibility and should not be used together with `lora_name` - "token_ids": [1, 15, 100], - "tenant_id": None, - "cache_salt": None, - } - ``` - - **Parameter Description**: - - `model`: (required, string) model name - - `lora_name`: (optional, string) The name of the LoRA adapter, default is `None`(indicating no LoRA adapter is used) - - `lora_id`: (optional, int) The ID of the LoRA adapter. This parameter is defined for backward compatibility and should not be used together with `lora_name`(Only one of them can be specified). Default is `-1`(indicating no LoRA adapter is used) - - `token_ids`: (required, [int]) prompt token id list - - `tenant_id`: (optional, int) In a multi-tenant architecture, tenant_id is the key identifier for distinguishing and isolating data from different tenants (such as different companies or user groups). All data operations are logically isolated based on this ID. If you provide it, the indexer will only return the token hit information for this tenant. Default is None(meaning there is only one tenant) - - `cache_salt`: (optional, int) An optional salt value to ensure cached data blocks are kept separate for different customers. This prevents one customer's kv-index data from being served to another. Default is None, meaning no salt is used. - - **example**: - ```json - { - "model": "deepseek-v3", - "lora_name": "sql_adapter", - "token_ids": [101, 15, 100, 55, 89], - } - ``` -- **Output**: - ```json - { - "data": { - "tenant_id": { - "api_server_unique_name": { - "longest_matched": 100, // the number of longest prefix matched token among multiple DPs(if there are) - "GPU": 20, - "DP": { - 0: 10, - 1: 20 - }, - "CPU": 60, - "DISK": 10 - }, - ... // other engine instance - }, - ... // other tenant - } - } - ``` - - **Parameter Description** - - `tenant_id`: tenant id, only used in multi-tenant scenario. - - `api_server_unique_name`: it is a unique name for a LLM API server endpoint in the engine side. For example, two service instances are currently started separately by running the `vllm server` command, and they are registered in the indexer with different names(such as vllm-1,vllm-2) - - `longest_matched`: the number of longest prefix matched token among G1/G2/G3. Indexer will sequentially query the hit status of each token-block according to the prefix order. If it hits, count the situation of this token-block at each level; If it missed, terminate the query (ensuring prefix continuity). - - `GPU`, `CPU`, `DISK`: token ids hit count for each tiered storage medium. The Indexer will track the storage status of KV-cache across various media. This requires different KV publishers to inform the Indexer of the actual storage medium type via kv-events. The following examples list several common names, such as using GPU or NPU to represent the Device Pool, using CPU to represent the Host Pool, and using DISK to represent the Disk Pool. - - `DP`: token ids hit count for each DP rank. - - **example**: - Assume the input token_ids are [101, 15, 100, 55, 89, 63], the block_size is 2, and the dp2 strategy is enabled. There are three block hashes to match [H1, H2, H3], where H1 hits in GPU (dp0, dp1), CPU, and DISK; H2 hits in GPU (dp0) and CPU; and H3 hits in DISK. - ```json - { - "vllm-1": { - "longest_matched": 6, - "GPU": 4, - "DP": { - 0: 4, - 1: 2 - }, - "CPU": 4, - "DISK": 4 - } - } - ``` +Query cache hits by token IDs. + +```json +{ + "model": "deepseek", + "lora_name": "sql-adapter", + "token_ids": [101, 15, 100, 55, 89], + "tenant_id": "default", + "instance_id": "vllm-prefill-node1", + "block_size": 64, + "cache_salt": "w8a8" +} +``` + +| Field | Required | Description | +|---|---|---| +| `model` | Yes | Model name. | +| `lora_name` | No | LoRA adapter name. Empty or omitted means base model. | +| `lora_id` | No | Deprecated compatibility field. Do not use together with `lora_name`. | +| `token_ids` | Yes | Prompt token IDs. Only complete blocks are considered. | +| `tenant_id` | No | Tenant identity. Defaults to `default`. | +| `instance_id` | No | If set, query one instance. If omitted, query all instances registered under the tenant. | +| `block_size` | Yes | KV block size in tokens. | +| `cache_salt` | No | Query-side hash namespace salt. Corresponds to the event `additional_salt` concept. | + +Response: + +```json +{ + "default": { + "vllm-prefill-node1": { + "longest_matched": 256, + "GPU": 128, + "DP": { + "0": 128, + "1": 256 + }, + "CPU": 256, + "DISK": 0 + } + } +} +``` + +| Field | Description | +|---|---| +| `longest_matched` | Longest continuous prefix hit in tokens across all tracked media and DP ranks for this instance. | +| `GPU`, `CPU`, `DISK` | Matched prefix tokens available on each medium. Names are examples; future media can be added. | +| `DP` | Matched prefix tokens grouped by data-parallel rank. | ### `POST /query_by_hash` - query token hit count by chunked_token hash key. Each model service uses its own independent page_size, `longest_matched = page_size * matched hash_key` -- **Input**: - - **Body** (JSON): - ```json - { - "model": "deepseek", - "lora_name": "xx-adapter", - "lora_id": 12, // defined for backward compatibility and should not be used together with `lora_name` - "block_hash": ["hash_key_by_chunked_tokens"], - "tenant_id": None, - "cache_salt": None, - } - ``` - - **Parameter Description**: - - `model`: (required, string) model name - - `lora_name`: (optional, string) The name of the LoRA adapter, default is `None`(indicating no LoRA adapter is used) - - `lora_id`: (optional, int) The ID of the LoRA adapter. This parameter is defined for backward compatibility and should not be used together with `lora_name`(Only one of them can be specified). Default is `-1`(indicating no LoRA adapter is used) - - `block_hash`: (required, [int]) chunk_token hash list - - `tenant_id`: (optional, int) In a multi-tenant architecture, tenant_id is the key identifier for distinguishing and isolating data from different tenants (such as different companies or user groups). All data operations are logically isolated based on this ID. If you provide it, the indexer will only return the token hit information for this tenant. Default is None(meaning there is only one tenant) - - `cache_salt`: (optional, int) An optional salt value to ensure cached data blocks are kept separate for different customers. This prevents one customer's kv-index data from being served to another. Default is None, meaning no salt is used. -- **Output**: - ```json - { - "data": { - "tenant_id": { - "api_server_unique_name": { - "longest_matched": 100, // the number of longest prefix matched token among multiple DPs(if there are) - "GPU": 20, - "DP": { - 0: 10, - 1: 20 - }, - "CPU": 60, - "DISK": 10 - }, - ... // other engine instance - }, - ... // other tenant - } - } - ``` - - **Parameter Description**: The output result is same as `/query` api. +Queries cache hits by precomputed rolling sequence hashes. This API avoids +sending long token lists over the network. -## Indexer KVEvents Structure -Typically, the device pool is used for loading model weights, with the remaining space registered for KV blocks by the model inference service runtime, the host pool and disk pool are managed uniformly by the Mooncake Store. There is a difference in the management unit for KV data between the two: the device pool uses blocks as the smallest unit for KV data, while the host pool and disk pool use Mooncake Store Objects as the smallest unit for KV storage. In practice, users may split a complete KV block into multiple Mooncake Store Objects for maintenance according to parallel strategies such as tensor parallelism (tp) and context parallelism (cp). - -### G1 KVEvents -[vLLM](https://github.com/vllm-project/vllm/blob/main/vllm/distributed/kv_events.py) and [SGLang](https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/disaggregation/kv_events.py) publishes events using the `EventBatch` structure, with each batch containing three types of events: - -- `BlockStored`:Adds a single KV block. -- `BlockRemoved`:Removes a single KV block. -- `AllBlocksCleared`:Clears all KV blocks. - -```py -EventBatch: +```json { - ts: float, # timestamp - events:list[BlockStored | BlockRemoved | AllBlocksCleared], - data_parallel_rank: int | None = None, # vLLM use this to indicate dp rank - attn_dp_rank: int | None = None, # SGLang use this to indicate dp rank + "model": "deepseek", + "lora_name": "sql-adapter", + "seq_hashes": [1234567890, 9876543210], + "tenant_id": "default", + "instance_id": "vllm-prefill-node1", + "block_size": 64, + "cache_salt": "w8a8" } - - -BlockStored: -{ - block_hashes: list[int] - parent_block_hash: int | None - token_ids: list[int] - block_size: int - - lora_id: int | None - """Deprecated: use `lora_name` for KV block key hash. - Retained for backward compatibility. - """ - - medium: str | None - """KV cache is categorized by tier. Currently, the following types are supported: - set "GPU", "NPU" for device pool(G1), - set "CPU" for host pool(G2), - set "DISK" for disk pool(G3). - In the future, more medium types can be supported for each tier. For example, "TPU" and "AMD" could be added for device pool. - """ - lora_name: str | None -} - -BlockRemoved: -{ - block_hashes: list[int] - lora_name: str | None -} - ``` -### G2/G3 KVEvents -Mooncake Store is a distributed key-value (KV) store. To ensure system consistency, a timestamp is assigned to each KVEvent for maintenance. +For compatibility with earlier drafts, clients may call the hash list +`block_hash`, but new clients should use `seq_hashes` to make it explicit that +the values are rolling sequence hashes rather than local block hashes. -Mooncake publishes events using the `EventBatch` structure, with each batch containing three types of events: -- `BlockStoreEvent`:Adds a single Mooncake Store Object. -- `BlockUpdateEvent`:Updates a single Mooncake Store Object. -- `RemoveAllEvent`:Removes all Mooncake Store Objects. +The response shape is the same as `/query`. For this API, +`longest_matched = block_size * matched_hash_count`. -```cpp -EventBatch { - std::vector> events, -} -BlockStoreEvent { - "BlockStoreEvent", // Event type identifier, string type - float ts, // timestamp - std::string mooncake_key, - std::vector addr_list, // Storage location of each replica +## KV Events API - uint64_t block_hash, - uint64_t parent_block_hash, - std::vector token_id, - uint32_t block_size, +The KV Events API is the wire contract between cache owners and indexers. +Conductor normalizes engine-specific events into this model. - std::string model_name, - std::string lora_name, - uint32_t lora_id, // Retained for backward compatibility -} +### Event envelope -BlockUpdateEvent { - "BlockUpdateEvent", - float ts, // timestamp - mooncake_key, - std::vector addr_list, -} +Every standardized event carries the same envelope: -RemoveAllEvent { - "RemoveAllEvent" +```json +{ + "event_id": 42, + "timestamp": 1739145600000, + "event_type": "stored", + "model_name": "llama-3.1-8b", + "block_size": 64, + "additional_salt": null, + "lora_name": null, + "tenant_id": "default", + "backend_id": "worker-0", + "medium": "gpu", + "dp_rank": 0 } ``` + +| Field | Type | Description | +|---|---|---| +| `event_id` | `u64` | Monotonically increasing sequence number scoped by the event stream dimensions. Authoritative for ordering. | +| `timestamp` | `u64 or null` | Unix epoch milliseconds. Informational only, not used for ordering. | +| `event_type` | `string` | One of `stored`, `removed`, or `cleared`. | +| `model_name` | `string or null` | Model identifier. | +| `block_size` | `u32 or null` | Tokens per block. | +| `additional_salt` | `string or null` | Opaque deployment salt or namespace. | +| `lora_name` | `string or null` | LoRA adapter name, or `null` for the base model. | +| `tenant_id` | `string` | Tenant or customer identity. | +| `backend_id` | `string` | Entity that owns the KV blocks. This can be an engine worker or a decoupled cache daemon. | +| `medium` | `string or null` | Cache medium such as `gpu`, `cpu`, or `disk`. | +| `dp_rank` | `u32 or null` | Data-parallel rank. | + +Events must be processed in consecutive `event_id` order within each stream +identified by `(model_name, block_size, additional_salt, lora_name, tenant_id, +backend_id, medium, dp_rank)`. + +### `stored` + +Published when one or more consecutive blocks are committed to a KV cache. + +```json +{ + "event_id": 42, + "timestamp": 1739145600000, + "event_type": "stored", + "model_name": "llama-3.1-8b", + "block_size": 64, + "additional_salt": null, + "lora_name": null, + "tenant_id": "default", + "backend_id": "worker-0", + "medium": "gpu", + "dp_rank": 0, + "seq_hashes": [1234567890, 9876543210, 1122334455], + "base_block_idx": 5, + "parent_hash": 9999999999, + "token_ids": null +} +``` + +| Field | Type | Description | +|---|---|---| +| `seq_hashes` | `u64[]` | Rolling sequence hashes of consecutive stored blocks. | +| `base_block_idx` | `u32 or null` | Zero-based depth of the first block in this event. | +| `parent_hash` | `u64 or null` | Rolling sequence hash at depth `base_block_idx - 1`; `null` at the root. | +| `token_ids` | `u32[] or null` | Tokens across all blocks in this event. Required when the publisher does not use the standardized hash. | + +At least one of `base_block_idx` or `parent_hash` must be present so the +consumer can locate the blocks in the sequence. + +### `removed` + +Published when one or more blocks are evicted. + +```json +{ + "event_id": 43, + "timestamp": 1739145601000, + "event_type": "removed", + "model_name": "llama-3.1-8b", + "block_size": 64, + "additional_salt": null, + "lora_name": null, + "tenant_id": "default", + "backend_id": "worker-0", + "medium": "gpu", + "dp_rank": 0, + "seq_hashes": [1122334455], + "base_block_idx": 7 +} +``` + +`seq_hashes` is required. `base_block_idx` is optional but recommended for +collision detection and observability. + +### `cleared` + +Published when all blocks for the event stream dimensions are purged. + +```json +{ + "event_id": 44, + "timestamp": 1739145602000, + "event_type": "cleared", + "model_name": "llama-3.1-8b", + "block_size": 64, + "additional_salt": null, + "lora_name": null, + "tenant_id": "default", + "backend_id": "worker-0", + "medium": "gpu", + "dp_rank": 0 +} +``` + +No additional payload fields are required. + +## Compatibility notes + +The current Conductor implementation consumes vLLM ZMQ msgpack batches and +normalizes `BlockStored` and `BlockRemoved` into the internal prefix index. +Registration metadata supplies fields such as `modelname`, `tenant_id`, +`instance_id`, `block_size`, and `additionalsalt` when the engine event does +not carry the full standardized envelope. diff --git a/docs/source/index.md b/docs/source/index.md index a22dab44..1a171d97 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -114,6 +114,8 @@ design/transfer-engine/index design/tent/overview design/tent/tebench design/hicache-design +design/conductor/conductor-architecture-design +design/conductor/indexer-api-design ::: % Q&A for Mooncake -- 2.34.1 From fe33367062d5848fb2059103171ecc137f0cde02 Mon Sep 17 00:00:00 2001 From: zhangzuo21 <99539591+zhangzuo21@users.noreply.github.com> Date: Mon, 27 Apr 2026 10:32:19 +0800 Subject: [PATCH 005/382] [docs] Document TCP port exhaustion and pool limitations (#1954) * [TE][docs] Add TCP port-exhaustion troubleshooting entry * [TE][docs] Document TCP connection pool known limitations --- .../source/troubleshooting/troubleshooting.md | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/docs/source/troubleshooting/troubleshooting.md b/docs/source/troubleshooting/troubleshooting.md index b85f55a7..1593543b 100644 --- a/docs/source/troubleshooting/troubleshooting.md +++ b/docs/source/troubleshooting/troubleshooting.md @@ -123,6 +123,49 @@ Note: In most cases, the errors output, except for the first occurrence, are `wo In addition, if the error `Failed to get description of XXX` is displayed, it indicates that the Segment name input by the user when calling the `openSegment` interface cannot be found in the etcd database. For memory read/write scenarios, the Segment name needs to strictly match the `local_hostname` field filled in by the other node during initialization. +## TCP Transport +### Recommended Troubleshooting Directions + +1. If sustained, high-concurrency TCP traffic (for example, PD-disaggregated KV transfers over the TCP transport) fails with `connect: Cannot assign requested address`, the initiator side has exhausted its ephemeral port range. Each transfer opens a fresh short-lived socket, and ports held in `TIME_WAIT` accumulate faster than the kernel can reclaim them. + + **Diagnostic Commands:** + ```bash + # Confirm large numbers of TIME_WAIT sockets to the peer + ss -tan state time-wait | wc -l + + # Check the local ephemeral port range + sysctl net.ipv4.ip_local_port_range + ``` + + **Solutions:** + - Enable the TCP connection pool so that long-lived sockets are reused across transfers instead of being opened per transfer: + ```bash + export MC_TCP_ENABLE_CONNECTION_POOL=1 + ``` + - Widen the ephemeral port range if the workload genuinely needs many distinct connections: + ```bash + sysctl -w net.ipv4.ip_local_port_range="1024 65535" + ``` + - As a last resort, enable `TIME_WAIT` reuse on the initiator. `tcp_tw_reuse` only affects outbound connections and requires TCP timestamps to be enabled on both sides: + ```bash + sysctl -w net.ipv4.tcp_tw_reuse=1 + ``` + +2. The TCP connection pool (`MC_TCP_ENABLE_CONNECTION_POOL=1`) has two known limitations in the current implementation. The pool is functional for most workloads, but operators running long-lived services should be aware of them: + + * **Idle-expired connections are not always reclaimed.** `cleanupIdleConnections()` only inspects the tail of each endpoint's deque. When a newer `in_use` entry has been pushed behind an older idle-expired entry, the loop terminates at the tail and the idle entry is never removed. The pool's reported state diverges from reality: it still lists the connection as idle while the socket has been sitting past `kConnectionIdleTimeout`, may have been half-closed by the peer, and will fail on the next `getConnection()` that tries to reuse it. In long-running services this also leaks sockets and file descriptors until the process is restarted. + + **Diagnostic Commands:** + ```bash + # Watch for unbounded growth of open sockets from the process + ls /proc/$PID/fd | wc -l + ss -tan | awk '$1=="ESTAB"' | wc -l + ``` + + **Workaround:** restart the process periodically if fd usage climbs without bound. + + * **`asio::socket::close()` runs under `pool_mutex_`.** `close()` cancels outstanding async operations and posts completion handlers to the io_context. Handlers such as `returnConnection()` also acquire `pool_mutex_`, so the current code is one scheduling step away from a circular wait. No deadlock has been observed in practice, but if the transfer engine hangs with all worker threads stuck waiting on `pool_mutex_`, this is the first place to look. + ## SGLang Common Questions ### Do I need RDMA to run SGLang and Mooncake? -- 2.34.1 From 489c020778a36836b69bf7c5b846aa47413ada8f Mon Sep 17 00:00:00 2001 From: ccs668899 <153548002+ccs668899@users.noreply.github.com> Date: Mon, 27 Apr 2026 10:32:34 +0800 Subject: [PATCH 006/382] [transfer_engine] fix: drain endpoint waiting list via periodic reclaim (#1952) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [transfer_engine] fix: add periodic endpoint reclaim from monitorWorker reclaimEndpoint() is currently invoked only from RdmaContext::endpoint() after a new insertion. Under healthy load, insertions and evictions are 1:1 so this works. Under failure load -- many error completions trigger deleteEndpoint(), but new-insertion traffic stalls because the dead peer isn't generating new connection paths -- waiting_list_ grows without bound and QPs never get destroyed. Add a 1Hz reclaimEndpoints() call from monitorWorker on the existing 1-second context heartbeat. This decouples reclaim cadence from insertion traffic. See issue #1845. * [transfer_engine] test: endpoint_store reclaim coverage for #1845 Adds unit + integration coverage for the periodic reclaim fix. endpoint_store_test (5 tests, no RDMA device, runs under ctest): - reclaim drains quiescent entries on its own - reclaim leaves active entries alone (gate preserved) - reclaim is idempotent when empty - leak manifests without reclaim call (1118-eviction mirror of reporter) - reclaim works without active map (guard against insert/reclaim coupling) endpoint_store_integration_test (requires RDMA device, not auto-registered): - Verifies WorkerPool::monitorWorker actually calls reclaimEndpoints at ~1 Hz by constructing a real RdmaContext and waiting for the tick to drain injected entries. Confirms the end-to-end fix wiring. Supporting changes: - EndpointStore::waitingListSize() accessor (diagnostics + tests) - SIEVEEndpointStore::testOnlyInsertWaiting() for test injection - RdmaContext::endpointStore() accessor (diagnostics + tests) * [docs] note periodic reclaim behavior and #1845 symptom - design/transfer-engine: add a sentence to Endpoint Management explaining that waiting_list_ drains both on insertion and on the monitorWorker heartbeat, so accumulated reclaim does not stall under failure load. - troubleshooting: extend the "Failed to create QP: Cannot allocate memory" entry with a bullet pointing at issue #1845 so operators seeing the symptom find the cause and the fix. * [transfer_engine] fix: guard FIFOEndpointStore::waitingListSize with atomic counter Per PR #1952 review: FIFO variant returned waiting_list_.size() on std::unordered_set without holding endpoint_map_lock_, racing concurrent modification. Mirror the SIEVE pattern with an atomic waiting_list_len_ incremented in delete/evict, decremented in reclaim. * [transfer_engine] test: suppress intentional RdmaTransport leak under LSAN CI build (3.10/3.12) runs with -DENABLE_ASAN=ON and LSAN flagged the 5 × 288 byte allocation the test fixture intentionally leaks (~RdmaTransport dereferences a null metadata_ unless install() ran). Gate on __SANITIZE_ADDRESS__ / __has_feature and mark the pointer with __lsan_ignore_object so real leaks are still caught. * [transfer_engine] fix: widen waiting_list_len_ atomic to size_t waitingListSize() returns size_t but the underlying counter was atomic, which quietly narrowed on load. Promote to atomic in both FIFO and SIEVE so the getter is a clean pass-through with no implicit conversion. * [transfer_engine] docs: pin reclaimEndpoint lock contract on base interface monitorWorker now calls reclaimEndpoint() via RdmaContext; it already acquired endpoint_map_lock_ internally, but nothing declared that. Codify the precondition on the base so future callers know not to hold the lock. RWSpinlock is non-reentrant, so recursive acquisition would deadlock. * [transfer_engine] refactor: narrow RdmaContext endpoint store test surface Previously exposed a raw EndpointStore* via RdmaContext::endpointStore() for the integration test. A raw pointer is easy to misuse outside of tests and couples the caller to the concrete store via dynamic_cast. Replace with two narrow methods on RdmaContext: waitingListSize() (value return) and testOnlyInsertWaiting(shared_ptr). The latter is lifted onto the EndpointStore base interface and implemented on both FIFO and SIEVE, so the integration test no longer downcasts. * [transfer_engine] test: register endpoint_store_integration_test with ctest Integration test was previously unregistered and invoked manually. Now self-skips via GTEST_SKIP when no RDMA device is present, so it runs cleanly on CI runners without RDMA (skips) and on rxe/mlx5 hosts (executes). Labeled "rdma" for ctest -L filtering. * [transfer_engine] perf: short-circuit FIFO reclaim when waiting list is empty monitorWorker now drives reclaim at ~1 Hz regardless of activity. On FIFO this grabbed endpoint_map_lock_ as WriteGuard every tick even in the common steady-state case where waiting_list_ is empty. Add the same counter-check short-circuit SIEVE already has. * [transfer_engine] test: skip integration test when RdmaContext::construct fails GHA ubuntu-22.04 runners enumerate a phantom mlx5_0 via ibv_get_device_list without a working port/GID, so pickRdmaDevice() returns a non-empty name and the earlier GTEST_SKIP on empty device list doesn't fire. Then construct() fails with ERR_CONTEXT and the hard ASSERT_EQ fails the test. Convert the assertion to a GTEST_SKIP on construct failure. Matches the "attempt setup, skip on failure" convention used elsewhere in the repo (e.g., client_local_hot_cache_test.cpp:794-799). --- docs/source/design/transfer-engine/index.md | 2 + .../source/troubleshooting/troubleshooting.md | 3 +- .../transport/rdma_transport/endpoint_store.h | 26 ++- .../transport/rdma_transport/rdma_context.h | 15 ++ .../rdma_transport/endpoint_store.cpp | 18 ++ .../transport/rdma_transport/rdma_context.cpp | 10 + .../transport/rdma_transport/worker_pool.cpp | 5 + mooncake-transfer-engine/tests/CMakeLists.txt | 11 ++ .../tests/endpoint_store_integration_test.cpp | 124 ++++++++++++ .../tests/endpoint_store_test.cpp | 178 ++++++++++++++++++ 10 files changed, 389 insertions(+), 3 deletions(-) create mode 100644 mooncake-transfer-engine/tests/endpoint_store_integration_test.cpp create mode 100644 mooncake-transfer-engine/tests/endpoint_store_test.cpp diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index f5fb3875..b2e90c81 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -75,6 +75,8 @@ If a connection fails due to link errors, it is removed from the endpoint pools on both sides and re-established during the next data transfer attempt. +Evicted and deleted endpoints are moved to an internal `waiting_list_` and reclaimed asynchronously once their outstanding slices drain. Reclaim runs on every new endpoint insertion, and additionally on a ~1 Hz heartbeat from the per-context `monitorWorker`, so the waiting list drains even under failure load where new insertions stall while evictions continue. + ### Fault Handling In a multi-NIC environment, one common failure scenario is the temporary unavailability of a specific NIC, while other routes may still connect two nodes. Mooncake Store is designed to adeptly manage such temporary diff --git a/docs/source/troubleshooting/troubleshooting.md b/docs/source/troubleshooting/troubleshooting.md index 1593543b..8b6d7eb9 100644 --- a/docs/source/troubleshooting/troubleshooting.md +++ b/docs/source/troubleshooting/troubleshooting.md @@ -110,9 +110,10 @@ Errors in this part usually indicate that the error occurred within the `mooncak * hard memlock unlimited ``` -7. If the error `Failed to create QP: Cannot allocate memory` is displayed, it is typically caused by too many QP have been created, reaching the driver limit. You can use `rdma resource` to trace how many QP is created. One possible way to resolve this issue: +7. If the error `Failed to create QP: Cannot allocate memory` is displayed, it is typically caused by too many QP have been created, reaching the driver limit. You can use `rdma resource` to trace how many QP is created. Possible ways to resolve this issue: - Update Mooncake to version v0.3.5 or later - Set the environment variable `MC_ENABLE_DEST_DEVICE_AFFINITY=1` before starting the application + - If the leak persists under sustained peer failures (many `endpoint evicted` log lines accompanying the QP growth), update to a version that includes the fix for [issue #1845](https://github.com/kvcache-ai/Mooncake/issues/1845). Prior to that fix, the endpoint store's `waiting_list_` only drained when new endpoints were inserted, so evictions under failure load accumulated QPs until the driver limit was hit. The fix adds a periodic reclaim tick to `monitorWorker`. ## RDMA Transfer Period ### Recommended Troubleshooting Directions diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/endpoint_store.h b/mooncake-transfer-engine/include/transport/rdma_transport/endpoint_store.h index 87798000..0190b02b 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/endpoint_store.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/endpoint_store.h @@ -44,6 +44,8 @@ class EndpointStore { const std::string &peer_nic_path, RdmaContext *context) = 0; virtual int deleteEndpoint(const std::string &peer_nic_path) = 0; virtual void evictEndpoint() = 0; + // Takes endpoint_map_lock_; caller must not hold it (RWSpinlock is + // non-reentrant, so recursive acquisition deadlocks). virtual void reclaimEndpoint() = 0; virtual size_t getSize() = 0; @@ -52,12 +54,21 @@ class EndpointStore { // Get the total number of QPs across all endpoints virtual size_t getTotalQPNumber() = 0; + + // Number of endpoints awaiting reclaim (evicted or explicitly deleted but + // not yet destructed). Exposed for tests and for operator observability. + virtual size_t waitingListSize() const = 0; + + // Test-only: push a pre-constructed endpoint into waiting_list_ so reclaim + // logic can be exercised without standing up an RDMA device. + virtual void testOnlyInsertWaiting(std::shared_ptr ep) = 0; }; // FIFO class FIFOEndpointStore : public EndpointStore { public: - FIFOEndpointStore(size_t max_size) : max_size_(max_size) {} + FIFOEndpointStore(size_t max_size) + : waiting_list_len_(0), max_size_(max_size) {} std::shared_ptr getEndpoint( const std::string &peer_nic_path) override; std::shared_ptr getEndpointByPtr( @@ -73,6 +84,11 @@ class FIFOEndpointStore : public EndpointStore { int disconnectQPs() override; size_t getTotalQPNumber() override; + size_t waitingListSize() const override { + return waiting_list_len_.load(std::memory_order_relaxed); + } + + void testOnlyInsertWaiting(std::shared_ptr ep) override; private: RWSpinlock endpoint_map_lock_; @@ -82,6 +98,7 @@ class FIFOEndpointStore : public EndpointStore { std::list fifo_list_; std::unordered_set> waiting_list_; + std::atomic waiting_list_len_; size_t max_size_; }; @@ -106,6 +123,11 @@ class SIEVEEndpointStore : public EndpointStore { int disconnectQPs() override; size_t getTotalQPNumber() override; + size_t waitingListSize() const override { + return waiting_list_len_.load(std::memory_order_relaxed); + } + + void testOnlyInsertWaiting(std::shared_ptr ep) override; private: RWSpinlock endpoint_map_lock_; @@ -119,7 +141,7 @@ class SIEVEEndpointStore : public EndpointStore { std::optional::iterator> hand_; std::unordered_set> waiting_list_; - std::atomic waiting_list_len_; + std::atomic waiting_list_len_; size_t max_size_; }; diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h index 299cbb77..f0a29c45 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h @@ -112,6 +112,21 @@ class RdmaContext { int deleteEndpoint(const std::string &peer_nic_path); + // Drain the endpoint store's waiting list. Safe to call on any thread; + // intended to be invoked periodically from monitorWorker so reclaim is + // not gated on new endpoint insertions (which can stall under failure + // load while evictions/deletions continue). See issue #1845. + void reclaimEndpoints(); + + // Number of endpoints awaiting reclaim. For tests and operator + // observability. + size_t waitingListSize() const; + + // Test-only: push a pre-constructed endpoint into the store's + // waiting_list_ so the reclaim path can be exercised without standing up + // a real RDMA QP. + void testOnlyInsertWaiting(std::shared_ptr ep); + int disconnectAllEndpoints(); // Get the total number of QPs across all endpoints in this context diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/endpoint_store.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/endpoint_store.cpp index 7dfc550a..ce4dae95 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/endpoint_store.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/endpoint_store.cpp @@ -81,6 +81,7 @@ int FIFOEndpointStore::deleteEndpoint(const std::string &peer_nic_path) { // remove endpoint but leaving it status unchanged // in case it is setting up connection or submitting slice if (iter != endpoint_map_.end()) { + waiting_list_len_++; waiting_list_.insert(iter->second); iter->second->set_active(false); endpoint_map_.erase(iter); @@ -97,17 +98,20 @@ void FIFOEndpointStore::evictEndpoint() { fifo_list_.pop_front(); fifo_map_.erase(victim); LOG(INFO) << victim << " evicted"; + waiting_list_len_++; waiting_list_.insert(endpoint_map_[victim]); endpoint_map_.erase(victim); return; } void FIFOEndpointStore::reclaimEndpoint() { + if (waiting_list_len_.load(std::memory_order_relaxed) == 0) return; RWSpinlock::WriteGuard guard(endpoint_map_lock_); std::vector> to_delete; for (auto &endpoint : waiting_list_) if (!endpoint->hasOutstandingSlice()) to_delete.push_back(endpoint); for (auto &endpoint : to_delete) waiting_list_.erase(endpoint); + waiting_list_len_ -= to_delete.size(); } size_t FIFOEndpointStore::getSize() { return endpoint_map_.size(); } @@ -135,6 +139,13 @@ size_t FIFOEndpointStore::getTotalQPNumber() { return total_qps; } +void FIFOEndpointStore::testOnlyInsertWaiting( + std::shared_ptr ep) { + RWSpinlock::WriteGuard guard(endpoint_map_lock_); + waiting_list_.insert(ep); + waiting_list_len_++; +} + std::shared_ptr SIEVEEndpointStore::getEndpoint( const std::string &peer_nic_path) { RWSpinlock::ReadGuard guard(endpoint_map_lock_); @@ -262,6 +273,13 @@ int SIEVEEndpointStore::disconnectQPs() { size_t SIEVEEndpointStore::getSize() { return endpoint_map_.size(); } +void SIEVEEndpointStore::testOnlyInsertWaiting( + std::shared_ptr ep) { + RWSpinlock::WriteGuard guard(endpoint_map_lock_); + waiting_list_.insert(ep); + waiting_list_len_++; +} + size_t SIEVEEndpointStore::getTotalQPNumber() { RWSpinlock::ReadGuard guard(endpoint_map_lock_); size_t total_qps = 0; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index 59746324..eb6a194b 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -386,6 +386,16 @@ int RdmaContext::deleteEndpoint(const std::string &peer_nic_path) { return endpoint_store_->deleteEndpoint(peer_nic_path); } +void RdmaContext::reclaimEndpoints() { endpoint_store_->reclaimEndpoint(); } + +size_t RdmaContext::waitingListSize() const { + return endpoint_store_->waitingListSize(); +} + +void RdmaContext::testOnlyInsertWaiting(std::shared_ptr ep) { + endpoint_store_->testOnlyInsertWaiting(std::move(ep)); +} + size_t RdmaContext::getTotalQPNumber() const { return endpoint_store_->getTotalQPNumber(); } diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp index c469ab26..99ab93b6 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp @@ -493,6 +493,11 @@ void WorkerPool::monitorWorker() { auto current_ts = getCurrentTimeInNano(); if (current_ts - last_reset_ts > 1000000000ll) { context_.set_active(true); + // Drain endpoint_store_->waiting_list_ even when no new + // insertions are happening. Without this, reclaim only runs + // from RdmaContext::endpoint() and the waiting list grows + // unboundedly under failure load. See issue #1845. + context_.reclaimEndpoints(); last_reset_ts = current_ts; } struct epoll_event event; diff --git a/mooncake-transfer-engine/tests/CMakeLists.txt b/mooncake-transfer-engine/tests/CMakeLists.txt index 1c630b59..0dafe93e 100644 --- a/mooncake-transfer-engine/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tests/CMakeLists.txt @@ -16,6 +16,17 @@ add_executable(transport_uint_test ${WORKSPACE}/transport_uint_test.cpp) target_link_libraries(transport_uint_test PUBLIC transfer_engine gtest gtest_main ) add_test(NAME transport_uint_test COMMAND transport_uint_test) +add_executable(endpoint_store_test ${WORKSPACE}/endpoint_store_test.cpp) +target_link_libraries(endpoint_store_test PUBLIC transfer_engine gtest gtest_main) +add_test(NAME endpoint_store_test COMMAND endpoint_store_test) + +# Integration test for the monitorWorker reclaim tick (issue #1845). +# Self-skips when no RDMA device is present, so safe to register with ctest. +add_executable(endpoint_store_integration_test ${WORKSPACE}/endpoint_store_integration_test.cpp) +target_link_libraries(endpoint_store_integration_test PUBLIC transfer_engine gtest gtest_main) +add_test(NAME endpoint_store_integration_test COMMAND endpoint_store_integration_test) +set_tests_properties(endpoint_store_integration_test PROPERTIES LABELS "rdma") + add_executable(rdma_transport_test2 ${WORKSPACE}/rdma_transport_test2.cpp) target_link_libraries(rdma_transport_test2 PUBLIC transfer_engine gtest gtest_main ) # add_test(NAME rdma_transport_test2 COMMAND rdma_transport_test2) diff --git a/mooncake-transfer-engine/tests/endpoint_store_integration_test.cpp b/mooncake-transfer-engine/tests/endpoint_store_integration_test.cpp new file mode 100644 index 00000000..c724dc11 --- /dev/null +++ b/mooncake-transfer-engine/tests/endpoint_store_integration_test.cpp @@ -0,0 +1,124 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +// Integration test for #1845. Verifies the end-to-end wiring of the fix: +// WorkerPool::monitorWorker actually calls RdmaContext::reclaimEndpoints at +// ~1 Hz, causing quiescent entries in the endpoint store's waiting_list_ to +// drain without any further insertion traffic. The unit tests in +// endpoint_store_test.cpp verify the reclaim method itself; this file +// verifies that the scheduler invokes it. +// +// Requires an RDMA device. Passes on soft-RoCE (`rdma_rxe`) as well as real +// NICs. Self-skips (GTEST_SKIP) when no device is present, so it is safe to +// register with ctest on CI runners without RDMA. +// +// Environment override: set MC_TEST_DEVICE_NAME to force a specific device; +// otherwise the first device returned by ibv_get_device_list is used. + +#include +#include + +#include +#include +#include +#include +#include + +#include "config.h" +#include "transport/rdma_transport/endpoint_store.h" +#include "transport/rdma_transport/rdma_context.h" +#include "transport/rdma_transport/rdma_endpoint.h" +#include "transport/rdma_transport/rdma_transport.h" + +#if defined(__has_feature) +#define MC_HAS_FEATURE(x) __has_feature(x) +#else +#define MC_HAS_FEATURE(x) 0 +#endif +#if defined(__SANITIZE_ADDRESS__) || MC_HAS_FEATURE(address_sanitizer) +#include +#define MC_LSAN_IGNORE_OBJECT(p) __lsan_ignore_object(p) +#else +#define MC_LSAN_IGNORE_OBJECT(p) ((void)(p)) +#endif + +using namespace mooncake; + +namespace { + +std::string pickRdmaDevice() { + const char *override_name = std::getenv("MC_TEST_DEVICE_NAME"); + if (override_name && *override_name) return override_name; + int num_devices = 0; + ibv_device **list = ibv_get_device_list(&num_devices); + if (!list || num_devices == 0) return ""; + std::string name = ibv_get_device_name(list[0]); + ibv_free_device_list(list); + return name; +} + +// Build an RdmaEndPoint with no QPs and active_=false. The store's reclaim +// path only inspects hasOutstandingSlice(), which for an endpoint with empty +// qp_list_ reduces to !active_. Safe to destruct because qp_list_ is empty. +std::shared_ptr makeQuiescentEndpoint(RdmaContext &ctx) { + auto ep = std::make_shared(ctx); + ep->set_active(false); + return ep; +} + +// Verifies the full fix wiring: after construct() spawns monitorWorker, a +// quiescent entry injected into the store's waiting_list_ is drained by the +// scheduler within ~1.5 s with no further insertion traffic. +TEST(EndpointStoreIntegration, MonitorWorkerTickDrainsWaitingList) { + const std::string device = pickRdmaDevice(); + if (device.empty()) { + GTEST_SKIP() << "no RDMA device available — integration test requires " + "rxe0, mlx5, or similar. Set MC_TEST_DEVICE_NAME to " + "override."; + } + + // RdmaTransport's destructor dereferences metadata_ which is null until + // init(); leak the engine to avoid touching that path. Marked ignored so + // LSAN under ASAN builds doesn't flag this intentional leak. + auto *transport = new RdmaTransport(); + MC_LSAN_IGNORE_OBJECT(transport); + auto context = std::make_shared(*transport, device); + auto &config = globalConfig(); + int rc = context->construct(config.num_cq_per_ctx, + config.num_comp_channels_per_ctx, config.port, + config.gid_index, config.max_cqe, + /*max_endpoints=*/4); + if (rc != 0) { + GTEST_SKIP() << "RdmaContext::construct failed on device " << device + << " (rc=" << rc << "); no usable RDMA device on this " + << "host (e.g., CI runners may enumerate a phantom " + << "mlx5_0 without a working port)."; + } + + context->testOnlyInsertWaiting(makeQuiescentEndpoint(*context)); + context->testOnlyInsertWaiting(makeQuiescentEndpoint(*context)); + context->testOnlyInsertWaiting(makeQuiescentEndpoint(*context)); + ASSERT_EQ(context->waitingListSize(), 3u); + + // monitorWorker's reclaim tick fires every ~1 s. Give it enough margin + // for scheduling jitter but keep the test fast. + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + + EXPECT_EQ(context->waitingListSize(), 0u) + << "monitorWorker must call reclaimEndpoints within ~1 s. If this " + "fails, either the periodic tick in worker_pool.cpp was removed or " + "reclaim is failing on quiescent entries."; +} + +} // namespace diff --git a/mooncake-transfer-engine/tests/endpoint_store_test.cpp b/mooncake-transfer-engine/tests/endpoint_store_test.cpp new file mode 100644 index 00000000..5f3b1216 --- /dev/null +++ b/mooncake-transfer-engine/tests/endpoint_store_test.cpp @@ -0,0 +1,178 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Regression coverage for #1845. Asserts that +// SIEVEEndpointStore::reclaimEndpoint drains quiescent entries from +// waiting_list_ without requiring a subsequent insertEndpoint call. This is the +// invariant the periodic-reclaim tick in monitorWorker depends on. + +#include + +#include + +#include "transport/rdma_transport/endpoint_store.h" +#include "transport/rdma_transport/rdma_context.h" +#include "transport/rdma_transport/rdma_endpoint.h" +#include "transport/rdma_transport/rdma_transport.h" + +#if defined(__has_feature) +#define MC_HAS_FEATURE(x) __has_feature(x) +#else +#define MC_HAS_FEATURE(x) 0 +#endif +#if defined(__SANITIZE_ADDRESS__) || MC_HAS_FEATURE(address_sanitizer) +#include +#define MC_LSAN_IGNORE_OBJECT(p) __lsan_ignore_object(p) +#else +#define MC_LSAN_IGNORE_OBJECT(p) ((void)(p)) +#endif + +using namespace mooncake; + +namespace { + +// Build an RdmaEndPoint that owns zero QPs and has active_=false. construct() +// is deliberately not called — the store's reclaim logic only inspects +// hasOutstandingSlice(), which for an endpoint with empty qp_list_ returns +// whatever active_ is. +std::shared_ptr makeQuiescentEndpoint(RdmaContext& ctx) { + auto ep = std::make_shared(ctx); + ep->set_active(false); + return ep; +} + +std::shared_ptr makeActiveEndpoint(RdmaContext& ctx) { + // Default ctor leaves active_=true. + return std::make_shared(ctx); +} + +class EndpointStoreTest : public ::testing::Test { + protected: + // Leaked on purpose: RdmaTransport's destructor dereferences metadata_, + // which is null when the engine was never init()ed. We only need a live + // reference for RdmaContext's constructor; the engine object is otherwise + // unused by the reclaim logic under test. + RdmaTransport* transport_ = nullptr; + std::unique_ptr ctx_; + + void SetUp() override { + transport_ = new RdmaTransport(); + // Intentional leak: ~RdmaTransport dereferences metadata_, which is + // null until install(). Marking it ignored keeps LSAN under ASAN + // builds from flagging this one allocation while still catching + // real leaks elsewhere. + MC_LSAN_IGNORE_OBJECT(transport_); + ctx_ = std::make_unique(*transport_, "unused"); + } +}; + +// The core invariant behind #1845's fix: reclaimEndpoint must drain quiescent +// entries on its own, without needing a subsequent insertEndpoint to trigger +// it. Before the fix, reclaim ran only on insertion, so if insertions stopped +// (e.g., all peers died), waiting_list_ grew unboundedly. The periodic tick +// from monitorWorker calls this method every second; this test asserts its +// contract in isolation. +TEST_F(EndpointStoreTest, ReclaimDrainsQuiescentEntries) { + SIEVEEndpointStore store(/*max_size=*/4); + + constexpr size_t kN = 10; + for (size_t i = 0; i < kN; ++i) { + store.testOnlyInsertWaiting(makeQuiescentEndpoint(*ctx_)); + } + EXPECT_EQ(store.waitingListSize(), kN); + + store.reclaimEndpoint(); + EXPECT_EQ(store.waitingListSize(), 0u) + << "reclaimEndpoint must drain quiescent entries with no insertion " + "prerequisite"; +} + +// Negative control: reclaim must leave entries in place if they still report +// outstanding slices. Ensures we didn't break the hasOutstandingSlice gate. +TEST_F(EndpointStoreTest, ReclaimLeavesActiveEntries) { + SIEVEEndpointStore store(4); + + store.testOnlyInsertWaiting(makeActiveEndpoint(*ctx_)); + store.testOnlyInsertWaiting(makeActiveEndpoint(*ctx_)); + store.testOnlyInsertWaiting(makeQuiescentEndpoint(*ctx_)); + EXPECT_EQ(store.waitingListSize(), 3u); + + store.reclaimEndpoint(); + EXPECT_EQ(store.waitingListSize(), 2u) + << "reclaim should drop only the quiescent endpoint, keep the two " + "active ones"; +} + +TEST_F(EndpointStoreTest, ReclaimIsIdempotentWhenEmpty) { + SIEVEEndpointStore store(4); + + store.reclaimEndpoint(); + EXPECT_EQ(store.waitingListSize(), 0u); + + store.testOnlyInsertWaiting(makeQuiescentEndpoint(*ctx_)); + store.reclaimEndpoint(); + EXPECT_EQ(store.waitingListSize(), 0u); + + store.reclaimEndpoint(); // second call is a no-op + EXPECT_EQ(store.waitingListSize(), 0u); +} + +// Demonstrates the #1845 failure mode: once insertions stop but evictions +// keep landing in the waiting list, nothing drains them without an explicit +// reclaim call. Before this fix, reclaimEndpoint ran only from insertEndpoint, +// so "many evictions, no new peers to connect to" meant waiting_list_ grew +// without bound. This test simulates that workload without any RDMA or +// scheduler; the assertion is a strict "zero reclaim calls leaves the leak +// at its peak." +TEST_F(EndpointStoreTest, LeakManifestsWithoutReclaimCall) { + SIEVEEndpointStore store(/*max_size=*/4); + + constexpr size_t kEvictions = 1118; // match reporter's eviction count + for (size_t i = 0; i < kEvictions; ++i) { + store.testOnlyInsertWaiting(makeQuiescentEndpoint(*ctx_)); + } + + // Without a reclaim call the leak is at its peak. + EXPECT_EQ(store.waitingListSize(), kEvictions) + << "baseline confirmation: waiting_list_ accumulates as expected"; + + // The fix is a 1 Hz invocation of this single method from monitorWorker. + // One call is enough to drain the entire backlog (because the entries are + // quiescent by the time the peer-death path finishes). This is the + // invariant the PR relies on. + store.reclaimEndpoint(); + EXPECT_EQ(store.waitingListSize(), 0u) + << "a single reclaim call drains the full backlog once insertions " + "stop; this is what the periodic tick in monitorWorker provides"; +} + +// Guards against a future regression that re-breaks the reclaim contract — +// e.g., someone changing reclaimEndpoint to no-op when endpoint_map_ is +// empty, on the incorrect assumption that reclaim only runs from +// insertEndpoint. Walking 1000 quiescent entries should still drain them. +TEST_F(EndpointStoreTest, ReclaimDoesNotRequireActiveMap) { + SIEVEEndpointStore store(4); + EXPECT_EQ(store.getSize(), 0u); // endpoint_map_ empty + + for (size_t i = 0; i < 1000; ++i) { + store.testOnlyInsertWaiting(makeQuiescentEndpoint(*ctx_)); + } + EXPECT_EQ(store.getSize(), 0u); // still empty + EXPECT_EQ(store.waitingListSize(), 1000u); + + store.reclaimEndpoint(); + EXPECT_EQ(store.waitingListSize(), 0u); +} + +} // namespace -- 2.34.1 From ae292ee837e344839eecba943363a8cd84ba49b3 Mon Sep 17 00:00:00 2001 From: Yan Date: Sun, 26 Apr 2026 23:01:43 -0700 Subject: [PATCH 007/382] [transfer_engine] feat: make RDMA QP pkey_index configurable via MC_PKEY_INDEX (#1985) * [transfer_engine] feat: make RDMA QP pkey_index configurable via MC_PKEY_INDEX Previously the QP attr.pkey_index was hardcoded to 0 during connection setup, which prevented use of non-default partition keys. Add a pkey_index field to GlobalConfig (default 0) that can be overridden through the MC_PKEY_INDEX environment variable, and apply it when transitioning the QP to INIT state. Co-Authored-By: Claude Opus 4.7 (1M context) * patch * add test and also update doc * Use try catch blocks instead --------- Co-authored-by: Yan Huang Co-authored-by: Claude Opus 4.7 (1M context) --- docs/source/design/transfer-engine/index.md | 1 + docs/source/zh_archive/transfer-engine.md | 1 + mooncake-transfer-engine/include/config.h | 3 +- mooncake-transfer-engine/src/config.cpp | 19 +++++ .../rdma_transport/rdma_endpoint.cpp | 2 +- .../tests/config_test.cpp | 83 +++++++++++++++++++ 6 files changed, 107 insertions(+), 2 deletions(-) create mode 100644 mooncake-transfer-engine/tests/config_test.cpp diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index b2e90c81..85a0c29a 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -276,6 +276,7 @@ For advanced users, TransferEngine provides the following advanced runtime optio - `MC_IB_TC` Adjust RDMA NIC Traffic Class when switch/NIC defaults differ or for traffic planning. Default value -1 - `MC_IB_PCI_RELAXED_ORDERING` Setting the PCIe ordering to relaxed for the network adapter sometimes results in better performance. Can set 1 to enable RO function. Default value 0 - `MC_GID_INDEX` The GID index used per device instance, default value 3 (or the maximum value supported by the platform) +- `MC_PKEY_INDEX` The QP `pkey_index` (partition key table index) used when transitioning the QP to the INIT state. Valid range: 0 to 65535. Default value 0. Set this when the partition key required for your fabric is not at index 0 of the HCA's pkey table - `MC_MAX_CQE_PER_CTX` The CQ buffer size per device instance, default value 4096 - `MC_MAX_EP_PER_CTX` The maximum number of active EndPoint per device instance, default value 65536. **Note:** For versions prior to 0.3.7.post1, the default value is 256, and it cannot be manually set to 65536. The maximum supported value is 65535! - `MC_NUM_QP_PER_EP` The number of QPs per EndPoint, the more the number, the better the fine-grained I/O performance, default value 2 diff --git a/docs/source/zh_archive/transfer-engine.md b/docs/source/zh_archive/transfer-engine.md index 20c0de97..3f02a341 100644 --- a/docs/source/zh_archive/transfer-engine.md +++ b/docs/source/zh_archive/transfer-engine.md @@ -396,6 +396,7 @@ int init(const std::string &metadata_conn_string, - `MC_IB_TC` 当使用`RDMA`通信协议时,在交换机和网卡默认配置不一致场景/需要流量规划场景下,可能需要修改 RDMA 网卡的 Traffic Class 配置,默认值 -1 - `MC_IB_PCI_RELAXED_ORDERING` 将网络适配器的PCIe顺序设置为放宽有时会带来更好的性能。可设置 1 以启用RO功能,默认值 0 - `MC_GID_INDEX` 每个设备实例使用的 GID 序号,默认值 3(或平台支持的最大值) +- `MC_PKEY_INDEX` QP 转换到 INIT 状态时使用的 `pkey_index`(partition key 表索引)。有效范围:0 到 65535,默认值 0。当 fabric 所需的 partition key 不在 HCA pkey 表的 0 号位置时需要设置该值 - `MC_MAX_CQE_PER_CTX` 每个设备实例中 CQ 缓冲区大小,默认值 4096 - `MC_MAX_EP_PER_CTX` 每个设备实例中活跃 EndPoint 数量上限,默认值 65536。**注意:** 小于 0.3.7.post1 的版本,这里默认值是 256,但是无法手动设置为 65536,最大值支持 65535!请注意 - `MC_NUM_QP_PER_EP` 每个 EndPoint 中 QP 数量,数量越多则细粒度 I/O 性能越好,默认值 2 diff --git a/mooncake-transfer-engine/include/config.h b/mooncake-transfer-engine/include/config.h index 954e42e3..5761209a 100644 --- a/mooncake-transfer-engine/include/config.h +++ b/mooncake-transfer-engine/include/config.h @@ -34,7 +34,8 @@ struct GlobalConfig { size_t num_cq_per_ctx = 1; size_t num_comp_channels_per_ctx = 1; uint8_t port = 1; - int gid_index = -1; // -1 for auto-selection, >=0 for user-specified + int gid_index = -1; // -1 for auto-selection, >=0 for user-specified + uint16_t pkey_index = 0; // QP attr.pkey_index; override via MC_PKEY_INDEX uint64_t max_mr_size = 0x10000000000; size_t max_cqe = 4096; int max_ep_per_ctx = 65536; diff --git a/mooncake-transfer-engine/src/config.cpp b/mooncake-transfer-engine/src/config.cpp index d0575d41..6001353a 100644 --- a/mooncake-transfer-engine/src/config.cpp +++ b/mooncake-transfer-engine/src/config.cpp @@ -67,6 +67,24 @@ void loadGlobalConfig(GlobalConfig& config) { << "Ignore value from environment variable MC_GID_INDEX"; } + const char* pkey_index_env = std::getenv("MC_PKEY_INDEX"); + if (pkey_index_env) { + try { + int val = std::stoi(pkey_index_env); + if (val >= 0 && val <= UINT16_MAX) { + config.pkey_index = static_cast(val); + } else { + LOG(WARNING) + << "Ignore value from environment variable MC_PKEY_INDEX, " + << "value " << pkey_index_env + << " out of range (should be 0-65535)"; + } + } catch (const std::exception& e) { + LOG(WARNING) << "Invalid MC_PKEY_INDEX environment value: " + << pkey_index_env << ". Error: " << e.what(); + } + } + const char* max_cqe_per_ctx_env = std::getenv("MC_MAX_CQE_PER_CTX"); if (max_cqe_per_ctx_env) { size_t val = atoi(max_cqe_per_ctx_env); @@ -399,6 +417,7 @@ void dumpGlobalConfig() { << config.num_comp_channels_per_ctx; LOG(INFO) << "port = " << config.port; LOG(INFO) << "gid_index = " << config.gid_index; + LOG(INFO) << "pkey_index = " << config.pkey_index; LOG(INFO) << "max_mr_size = " << config.max_mr_size; LOG(INFO) << "max_cqe = " << config.max_cqe; LOG(INFO) << "max_ep_per_ctx = " << config.max_ep_per_ctx; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp index 104c0322..4c21b678 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp @@ -589,7 +589,7 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const std::string &peer_gid, memset(&attr, 0, sizeof(attr)); attr.qp_state = IBV_QPS_INIT; attr.port_num = context_.portNum(); - attr.pkey_index = 0; + attr.pkey_index = globalConfig().pkey_index; attr.qp_access_flags = IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_READ | IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_ATOMIC; ret = ibv_modify_qp( diff --git a/mooncake-transfer-engine/tests/config_test.cpp b/mooncake-transfer-engine/tests/config_test.cpp new file mode 100644 index 00000000..ba108d9f --- /dev/null +++ b/mooncake-transfer-engine/tests/config_test.cpp @@ -0,0 +1,83 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include + +#include + +#include "config.h" + +namespace mooncake { +namespace { + +class PkeyIndexEnvTest : public ::testing::Test { + protected: + void TearDown() override { ::unsetenv("MC_PKEY_INDEX"); } +}; + +TEST_F(PkeyIndexEnvTest, DefaultIsZeroWhenUnset) { + ::unsetenv("MC_PKEY_INDEX"); + GlobalConfig config; + loadGlobalConfig(config); + EXPECT_EQ(config.pkey_index, 0); +} + +TEST_F(PkeyIndexEnvTest, ValidOverrideIsApplied) { + ASSERT_EQ(::setenv("MC_PKEY_INDEX", "7", 1), 0); + GlobalConfig config; + loadGlobalConfig(config); + EXPECT_EQ(config.pkey_index, 7); +} + +TEST_F(PkeyIndexEnvTest, MaxBoundaryIsApplied) { + ASSERT_EQ(::setenv("MC_PKEY_INDEX", "65535", 1), 0); + GlobalConfig config; + loadGlobalConfig(config); + EXPECT_EQ(config.pkey_index, 65535); +} + +TEST_F(PkeyIndexEnvTest, OutOfRangeIsIgnored) { + ASSERT_EQ(::setenv("MC_PKEY_INDEX", "70000", 1), 0); + GlobalConfig config; + config.pkey_index = 3; // sentinel preserved when env var is rejected + loadGlobalConfig(config); + EXPECT_EQ(config.pkey_index, 3); +} + +TEST_F(PkeyIndexEnvTest, NegativeIsIgnored) { + ASSERT_EQ(::setenv("MC_PKEY_INDEX", "-1", 1), 0); + GlobalConfig config; + config.pkey_index = 5; + loadGlobalConfig(config); + EXPECT_EQ(config.pkey_index, 5); +} + +TEST_F(PkeyIndexEnvTest, NonNumericKeepsDefault) { + ASSERT_EQ(::setenv("MC_PKEY_INDEX", "abc", 1), 0); + GlobalConfig config; + config.pkey_index = 9; + loadGlobalConfig(config); + EXPECT_EQ(config.pkey_index, 9); +} + +TEST_F(PkeyIndexEnvTest, EmptyStringKeepsDefault) { + ASSERT_EQ(::setenv("MC_PKEY_INDEX", "", 1), 0); + GlobalConfig config; + config.pkey_index = 4; + loadGlobalConfig(config); + EXPECT_EQ(config.pkey_index, 4); +} + +} // namespace +} // namespace mooncake -- 2.34.1 From e68696d1e9c3eeb55347c5eeebe082b1404f4dce Mon Sep 17 00:00:00 2001 From: yejj Date: Mon, 27 Apr 2026 16:06:15 +0800 Subject: [PATCH 008/382] [Doc] add mermaid extensions for html doc pages (#1981) --- docs/source/conf.py | 2 ++ requirements_docs.txt | 1 + 2 files changed, 3 insertions(+) diff --git a/docs/source/conf.py b/docs/source/conf.py index 817b116d..6fc3b9a8 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -46,11 +46,13 @@ extensions = [ "sphinxarg.ext", "sphinx_design", "sphinx_togglebutton", + "sphinxcontrib.mermaid", ] myst_enable_extensions = [ "colon_fence", "fieldlist", ] +myst_fence_as_directive = ["mermaid"] autodoc2_packages = [ { "path": "../../Mooncake", diff --git a/requirements_docs.txt b/requirements_docs.txt index 9c267eda..2aacb268 100644 --- a/requirements_docs.txt +++ b/requirements_docs.txt @@ -4,6 +4,7 @@ sphinx-book-theme==1.1.4 sphinx-copybutton==0.5.2 sphinx-design==0.6.1 sphinx-togglebutton==0.3.2 +sphinxcontrib-mermaid==2.0.1 myst-parser==3.0.1 # `myst-parser==4.0.1` breaks inline code in titles msgspec snowballstemmer<3 # https://github.com/snowballstem/snowball/issues/229 -- 2.34.1 From cd42d4d1fd6946c59bb66c4fac67ca50da9d1689 Mon Sep 17 00:00:00 2001 From: QiuRuiqi <875423353@qq.com> Date: Mon, 27 Apr 2026 17:07:10 +0800 Subject: [PATCH 009/382] [Store] Fix DSO missing error when building with USE_3FS=ON (#1983) --- mooncake-store/src/CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mooncake-store/src/CMakeLists.txt b/mooncake-store/src/CMakeLists.txt index 06fcd2bd..d9e0c2ff 100644 --- a/mooncake-store/src/CMakeLists.txt +++ b/mooncake-store/src/CMakeLists.txt @@ -118,7 +118,7 @@ if(USE_3FS) if(NOT HF3FS_API_LIB) message(FATAL_ERROR "hf3fs_api_shared library not found in /usr/lib") endif() - set(EXTRA_LIBS ${HF3FS_API_LIB}) + list(APPEND EXTRA_LIBS ${HF3FS_API_LIB}) endif() # io_uring support (auto-detected) -- 2.34.1 From 16ad1ba92d267ccd25dba370fa80c799fb6fca13 Mon Sep 17 00:00:00 2001 From: Stary Date: Mon, 27 Apr 2026 17:08:21 +0800 Subject: [PATCH 010/382] [CI] route fork PR to pull_request_target for ascend/integration tests (#1989) Fork PRs trigger pull_request events where GitHub Actions withholds vars and secrets. This causes vars.ASCEND_GITHUB_MIRROR_URLS to resolve as empty on the self-hosted Ascend runner, blocking mirror-based checkout. Add pull_request_target as a complementary trigger and route both ascend-test and integration-test by fork origin: non-fork PRs continue on pull_request, fork PRs are handled by pull_request_target where vars and secrets are available. Duplicate runs are avoided by skipping the opposite event for each case. Signed-off-by: staryxchen --- .github/workflows/ci.yml | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e72c7e46..b4c54bd2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -6,6 +6,9 @@ on: pull_request: branches: [ "main" ] types: [opened, synchronize, reopened, labeled] + pull_request_target: + branches: [ "main" ] + types: [opened, synchronize, reopened, labeled] workflow_dispatch: {} permissions: @@ -748,13 +751,29 @@ jobs: ascend-test: needs: [build, check-paths] - if: needs.check-paths.outputs.should-run-downstream == 'true' + if: >- + needs.check-paths.outputs.should-run-downstream == 'true' && + ( + github.event_name == 'push' || + github.event_name == 'workflow_dispatch' || + (github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork) || + (github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.fork) + ) uses: ./.github/workflows/ci_ascend.yml secrets: inherit + with: + checkout_ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.sha || '' }} integration-test: needs: [build, check-paths] - if: needs.check-paths.outputs.should-run-downstream == 'true' + if: >- + needs.check-paths.outputs.should-run-downstream == 'true' && + ( + github.event_name == 'push' || + github.event_name == 'workflow_dispatch' || + (github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork) || + (github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.fork) + ) uses: ./.github/workflows/integration-test.yml secrets: inherit -- 2.34.1 From 93ff81c66aaacd21c0419159fa5305b73b1bd11e Mon Sep 17 00:00:00 2001 From: fatSheep <50252876+00fish0@users.noreply.github.com> Date: Mon, 27 Apr 2026 21:36:44 +0800 Subject: [PATCH 011/382] [CI] optimize build-flags job and add TENT compilation test (#1775) Co-authored-by: fatSheep --- .github/workflows/ci.yml | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b4c54bd2..45d565b6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -584,6 +584,22 @@ jobs: sudo cmake --install . shell: bash + - name: Configure project with TENT + run: | + mkdir build-tent + cd build-tent + cmake -G Ninja .. -DUSE_TENT=ON -DUSE_HTTP=ON -DUSE_CUDA=ON -DENABLE_SCCACHE=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/cuda/lib64/stubs" + shell: bash + + - name: Build project with TENT + run: | + export LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LD_LIBRARY_PATH + export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LIBRARY_PATH + cd build-tent + cmake --build . + sudo cmake --install . + shell: bash + - name: Build nvlink_allocator.so run: | mkdir -p build/mooncake-transfer-engine/nvlink-allocator -- 2.34.1 From 8aa8f25cc3efa9fc43b9a3ebbb0accf7c05e2fb0 Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Mon, 27 Apr 2026 21:38:01 +0800 Subject: [PATCH 012/382] fix(test): avoid in-process master port collisions (#1987) --- mooncake-store/tests/test_server_helpers.h | 39 +++++++++++++++++++--- 1 file changed, 34 insertions(+), 5 deletions(-) diff --git a/mooncake-store/tests/test_server_helpers.h b/mooncake-store/tests/test_server_helpers.h index 40b3fd10..5336e0e1 100644 --- a/mooncake-store/tests/test_server_helpers.h +++ b/mooncake-store/tests/test_server_helpers.h @@ -1,8 +1,10 @@ #pragma once +#include #include #include #include +#include #include #include #include @@ -35,17 +37,44 @@ class InProcMaster { int needed = (!config.rpc_port.has_value()) + (!config.http_metrics_port.has_value()) + (!config.http_metadata_port.has_value()); - auto free_ports = getFreeTcpPorts(needed); - if (static_cast(free_ports.size()) < needed) return false; + std::vector available_ports; + if (needed > 0) { + std::vector reserved_ports; + if (config.rpc_port.has_value()) { + reserved_ports.push_back(config.rpc_port.value()); + } + if (config.http_metrics_port.has_value()) { + reserved_ports.push_back(config.http_metrics_port.value()); + } + if (config.http_metadata_port.has_value()) { + reserved_ports.push_back(config.http_metadata_port.value()); + } + auto free_ports = getFreeTcpPorts( + needed + static_cast(reserved_ports.size())); + available_ports.reserve(needed); + for (int port : free_ports) { + if (std::find(reserved_ports.begin(), reserved_ports.end(), + port) == reserved_ports.end()) { + available_ports.push_back(port); + if (static_cast(available_ports.size()) == + needed) { + break; + } + } + } + if (static_cast(available_ports.size()) < needed) { + return false; + } + } int idx = 0; rpc_port_ = config.rpc_port.has_value() ? config.rpc_port.value() - : free_ports[idx++]; + : available_ports[idx++]; http_metrics_port_ = config.http_metrics_port.has_value() ? config.http_metrics_port.value() - : free_ports[idx++]; + : available_ports[idx++]; http_metadata_port_ = config.http_metadata_port.has_value() ? config.http_metadata_port.value() - : free_ports[idx++]; + : available_ports[idx++]; // Optional HTTP metadata server if (http_metadata_port_ > 0) { -- 2.34.1 From 02d1975ffddb3fd99124ad72affde135911fcb6e Mon Sep 17 00:00:00 2001 From: Cruz Zhao Date: Mon, 27 Apr 2026 23:39:40 +0800 Subject: [PATCH 013/382] [Store] Unified parallel tensor IO (#1389) * [Store] add tensor object metadata and TP upsert APIs Introduce explicit tensor object metadata for tensor read/write paths, add TP-aware tensor upsert wrappers, and update tests/docs for the new serialized layout. --------- Co-authored-by: Claude Opus 4.6 --- .../design/unified-parallel-tensor-io.md | 405 +++++ .../python-api-reference/mooncake-store.md | 230 ++- mooncake-integration/integration_utils.h | 205 ++- mooncake-integration/store/store_py.cpp | 494 +++-- .../store/store_py_internal.h | 1483 ++++++++++++++++ .../store/store_py_parallel_read.h | 1312 ++++++++++++++ .../store/store_py_parallel_write.h | 1443 +++++++++++++++ mooncake-store/include/real_client.h | 9 + mooncake-store/src/real_client.cpp | 22 + scripts/test_tensor_api.py | 1581 ++++++++++++++++- scripts/test_upsert_api.py | 607 ++++++- 11 files changed, 7641 insertions(+), 150 deletions(-) create mode 100644 docs/source/design/unified-parallel-tensor-io.md create mode 100644 mooncake-integration/store/store_py_internal.h create mode 100644 mooncake-integration/store/store_py_parallel_read.h create mode 100644 mooncake-integration/store/store_py_parallel_write.h diff --git a/docs/source/design/unified-parallel-tensor-io.md b/docs/source/design/unified-parallel-tensor-io.md new file mode 100644 index 00000000..ff9f1f3b --- /dev/null +++ b/docs/source/design/unified-parallel-tensor-io.md @@ -0,0 +1,405 @@ +# Unified Parallel Tensor IO + +## Goal + +This document defines the single source of truth for Mooncake's next-generation tensor IO API. The purpose is to keep implementation aligned around one explicit design and avoid drifting into ad hoc TP/EP/DP/PP-specific interfaces. + +The target outcome is: + +- one unified write API family +- one unified read API family +- one unified upsert API family +- TP-specific APIs retained only as compatibility wrappers +- minimal caller-facing inputs, with layout and planning details derived internally +- one stable abstraction that can cover TP / DP / EP / PP combinations without exploding the public API surface + +## Design principles + +1. **One API family, not one family per parallelism type.** + Public APIs should not branch into separate long-term `*_with_tp`, `*_with_ep`, `*_with_pp`, and `*_with_dp` families. +2. **Parallelism is modeled as explicit axes.** + `mixed` must not be a vague enum value. A shard should be described by the exact axis coordinates that identify it. +3. **Reads must encode caller intent explicitly.** + A read request must say whether the caller wants the stored view, a target shard/view, or the reconstructed full tensor. +4. **Only require true caller intent.** + If Mooncake can derive a field from the input tensor, stored metadata, or planning context, it should not be required in the public API. +5. **Planner/runtime complexity stays internal.** + Byte ranges, payload offsets, reconstruction plans, and lowering to `get_into_ranges(...)` are runtime concerns, not public API concerns. +6. **Compatibility wrappers stay thin.** + Existing TP methods should lower to the same unified implementation path. + +## Core model + +### Parallelism is an axis list + +A tensor object may be identified by one axis or by multiple orthogonal axes. + +Examples: + +- pure TP shard: `[TP]` +- DP + TP shard: `[DP, TP]` +- PP + TP shard: `[PP, TP]` +- DP + PP + TP shard: `[DP, PP, TP]` +- DP + PP + EP + TP shard: `[DP, PP, EP, TP]` + +So the design should not use: + +```python +kind = "mixed" +``` + +Instead it should use: + +```python +class ParallelAxis: + kind: Literal["tp", "dp", "ep", "pp"] + rank: int + size: int + split_dim: Optional[int] = None + expert_id: Optional[int] = None + stage_id: Optional[int] = None +``` + +```python +class TensorParallelism: + axes: list[ParallelAxis] +``` + +Single-axis cases are just special cases of this model: + +- TP only: `axes=[TP(...)]` +- EP only: `axes=[EP(...)]` +- PP + TP: `axes=[PP(...), TP(...)]` + +## Public API shape + +### Write / upsert side + +```python +def put_tensor_with_parallelism( + self, + key: str, + tensor: torch.Tensor, + parallelism: TensorParallelism | None = None, + replica: ReplicateConfig | None = None, +) -> int +``` + +```python +def batch_put_tensor_with_parallelism( + self, + keys: list[str], + tensors: list[torch.Tensor], + parallelisms: list[TensorParallelism | None] | None = None, + replica: ReplicateConfig | None = None, +) -> list[int] +``` + +```python +def upsert_tensor_with_parallelism( + self, + key: str, + tensor: torch.Tensor, + parallelism: TensorParallelism | None = None, + replica: ReplicateConfig | None = None, +) -> int +``` + +```python +def batch_upsert_tensor_with_parallelism( + self, + keys: list[str], + tensors: list[torch.Tensor], + parallelisms: list[TensorParallelism | None] | None = None, + replica: ReplicateConfig | None = None, +) -> list[int] +``` + +### Read side + +Read-side requests need one extra structure because the request must encode not only the target parallel coordinates, but also the materialization mode. + +```python +class ReadTarget: + mode: Literal["as_stored", "shard", "full"] + parallelism: TensorParallelism | None = None +``` + +```python +def get_tensor_with_parallelism( + self, + key: str, + target: ReadTarget | None = None, +) -> torch.Tensor +``` + +```python +def batch_get_tensor_with_parallelism( + self, + keys: list[str], + targets: list[ReadTarget | None] | None = None, +) -> list[torch.Tensor] +``` + +Optional zero-copy forms follow the same model: + +```python +def get_tensor_with_parallelism_into( + self, + key: str, + buffer_ptr: int, + size: int, + target: ReadTarget | None = None, +) -> torch.Tensor +``` + +```python +def batch_get_tensor_with_parallelism_into( + self, + keys: list[str], + buffer_ptrs: list[int], + sizes: list[int], + targets: list[ReadTarget | None] | None = None, +) -> list[torch.Tensor] +``` + +## Caller inputs vs internal derived fields + +### Caller must provide + +These are true caller intent and belong in the public API: + +- the tensor itself +- whether the write target is full or shard-like +- the shard identity axes when writing a shard object +- the read target mode: `as_stored`, `shard`, or `full` +- the target axis coordinates when requesting a target shard/view +- `ReplicateConfig` when the caller wants replication / publish-like behavior + +### Mooncake should derive internally + +These should not be mandatory public inputs when they are derivable: + +- logical shape +- local shard shape +- metadata encoding/version details +- payload offsets +- source and destination byte ranges +- reconstruction plans +- lowering to `get_into_ranges(...)` + +## Why `ReadTarget` is required + +`parallelism` alone is not enough to describe a read. + +For example, if a tensor is stored as TP shards, a request that carries TP axis coordinates is ambiguous unless it also says whether the caller wants: + +- the stored local shard +- a target shard/view +- the reconstructed full tensor + +So reads must explicitly encode: + +```python +ReadTarget(mode="as_stored") +ReadTarget(mode="shard", parallelism=...) +ReadTarget(mode="full") +``` + +This distinction must not be guessed from the axis metadata. + +## Mixed-parallel scenarios in training and inference + +`mixed` is not a mode. It means the shard identity needs more than one axis coordinate to be uniquely described. + +### Scenario table + +| Scenario | Typical axis list | Meaning | +|---|---|---| +| TP training | `[TP]` | one TP slice of a logical tensor | +| DP + TP training | `[DP, TP]` | one TP slice within one DP replica/group | +| PP + TP training | `[PP, TP]` | one TP slice owned by one pipeline stage | +| DP + PP + TP training | `[DP, PP, TP]` | one TP slice in one stage in one DP replica | +| DP + PP + EP + TP training | `[DP, PP, EP, TP]` | one expert-local TP slice in one stage and one DP replica | +| TP inference | `[TP]` | one TP slice used by one inference rank | +| PP + TP inference | `[PP, TP]` | one TP slice owned by one inference pipeline stage | +| EP inference | `[EP]` or `[EP, TP]` | one expert shard, optionally further TP-sliced | +| multi-replica serving inference | `[DP, TP]` or `[DP, PP, TP]` | one shard scoped to a serving replica plus model-parallel axes | + +### Important distinctions + +- TP and EP often affect the tensor's physical layout directly. +- PP and serving-replica / DP often act more like ownership or scope tags, even when they do not themselves change the byte layout inside the local shard. + +## Write-side matrix + +The unified write family should be driven by the identity of the object being written, not by method-name proliferation. + +| Caller holds | Wants to store | API | Required `parallelism` | +|---|---|---|---| +| full tensor | full tensor | `put_tensor_with_parallelism(..., parallelism=None)` | none | +| full tensor | TP shard | `put_tensor_with_parallelism(...)` | `axes=[TP(rank,size,split_dim)]` | +| full tensor | DP-scoped shard/replica | `put_tensor_with_parallelism(...)` | `axes=[DP(rank,size)]` plus layout axis if actually sharded | +| full tensor | EP shard | `put_tensor_with_parallelism(...)` | `axes=[EP(rank,size,expert_id)]` plus `split_dim` if needed | +| full tensor | PP stage shard | `put_tensor_with_parallelism(...)` | `axes=[PP(rank,size,stage_id)]` | +| full tensor | combined shard | `put_tensor_with_parallelism(...)` | explicit axis list such as `[PP(...), TP(...)]` | +| shard tensor | shard object | `put_tensor_with_parallelism(...)` | explicit axis list describing that shard identity | + +For TP-containing **multi-axis** layouts, the write semantic is now: the caller may pass the **full source tensor**, and the provided TP rank/layout tells Mooncake which uniform shard to materialize and persist. Callers no longer need to pre-split the tensor themselves for `dp_tp` / `pp_tp` / `ep_tp` style writes. + +Single-axis TP compatibility wrappers and the preserved plain-TP `with_parallelism` behavior still accept shard input rather than auto-materializing from a full tensor. + +Pure DP still does not invent a split axis on its own. If the stored object is actually sharded, the request must still include the layout axis that defines the shard shape. + +The same matrix applies to `upsert_tensor_with_parallelism(...)`. + +## Read-side matrix + +The unified read family should be driven by `ReadTarget`. + +| Stored layout | Caller wants | API | `ReadTarget` | +|---|---|---|---| +| full tensor | stored full tensor | `get_tensor_with_parallelism(...)` | `None` or `mode="as_stored"` | +| TP shard object | stored shard | `get_tensor_with_parallelism(...)` | `mode="as_stored"` | +| TP shard set | target TP shard | `get_tensor_with_parallelism(...)` | `mode="shard", parallelism=TP(...)` | +| TP shard set | full tensor | `get_tensor_with_parallelism(...)` | `mode="full"` | +| EP shard set | target expert shard | `get_tensor_with_parallelism(...)` | `mode="shard", parallelism=EP(...)` | +| EP shard set | full tensor | `get_tensor_with_parallelism(...)` | `mode="full"` | +| PP + TP shard set | target PP+TP shard | `get_tensor_with_parallelism(...)` | `mode="shard", parallelism=[PP(...), TP(...)]` | +| mixed shard set | full tensor | `get_tensor_with_parallelism(...)` | `mode="full"` | + +## Source-layout to target-layout matrix + +The planner must eventually cover these conversions, but the public API should remain the same across all of them. + +| Stored source layout | Requested target layout | Support model | +|---|---|---| +| TP | TP | direct shard fetch or shard-local fast path | +| TP | full | reconstruct full tensor | +| TP | EP / PP / DP / mixed | planner-driven remap | +| EP | EP | direct shard fetch or shard-local fast path | +| EP | full | reconstruct full tensor | +| EP | TP / PP / DP / mixed | planner-driven remap | +| PP | PP | direct shard fetch | +| PP | full | reconstruct full tensor when meaningful | +| PP | TP / EP / DP / mixed | planner-driven remap | +| mixed | mixed | direct fetch if exact match, else planner-driven remap | +| mixed | full | reconstruct full tensor | +| mixed | TP / EP / PP / DP | planner-driven remap | + +The important point is that these combinations must not create public API explosion. + +## Compatibility wrappers + +Existing TP APIs remain compatibility wrappers. + +Conceptually: + +```python +put_tensor_with_tp(key, tensor, tp_rank, tp_size, split_dim) +``` + +lowers to: + +```python +put_tensor_with_parallelism( + key, + tensor, + TensorParallelism(axes=[TP(rank=tp_rank, size=tp_size, split_dim=split_dim)]), +) +``` + +and: + +```python +get_tensor_with_tp(key, tp_rank, tp_size, split_dim) +``` + +lowers to: + +```python +get_tensor_with_parallelism( + key, + ReadTarget( + mode="shard", + parallelism=TensorParallelism( + axes=[TP(rank=tp_rank, size=tp_size, split_dim=split_dim)] + ), + ), +) +``` + +Existing TP behavior stays stable, but implementation should flow through the unified path. + +## Runtime lowering direction + +Internally, reads should be planner-driven. + +When a request can be lowered to explicit: + +- key +- src offset +- dst offset +- size + +ranges, the runtime should reuse `get_into_ranges(...)` to assemble the result directly into the output buffer. + +Otherwise it can fall back to the simpler whole-object path. + +This keeps the public interface stable while allowing future planner work to add DP / TP / EP / PP remapping and optimized reconstruction without another API redesign. + +## Current implementation status + +The current `store_py.cpp` implementation now reflects the main shape of this design: + +- unified write APIs are exposed as `put_tensor_with_parallelism(...)` and `batch_put_tensor_with_parallelism(...)` +- unified read APIs are exposed as `get_tensor_with_parallelism(...)` and `batch_get_tensor_with_parallelism(...)` +- unified upsert APIs are exposed as `upsert_tensor_with_parallelism(...)` and `batch_upsert_tensor_with_parallelism(...)` +- zero-copy `_into` and `_from` variants exist for the unified API family +- TP-specific APIs remain available as compatibility wrappers and should not be treated as the long-term surface area + +### Implemented write-side convenience: `writer_partitions` + +Batch write and batch upsert paths also support `writer_partitions` as a convenience input for full tensors that should be written as stored shards. + +This is intentionally narrower than the full `TensorParallelism` model: + +- it is a write-side convenience, not a replacement for `TensorParallelism` +- it is primarily for batch full-tensor writes where the caller already knows rank / size / split_dim per item +- it should not change the unified read-side abstraction + +`writer_partitions` remains a separate explicit route. The newer TP-containing `parallelism` write semantic now overlaps with it for the common case of “full tensor in, store one requested shard”, but `writer_partitions` is still useful when the caller wants a lighter write-side request shape without constructing `TensorParallelism` objects. + +### Implemented read-side behavior + +Read-side support includes: + +- returning the stored local object (`mode="as_stored"`) +- returning a target shard (`mode="shard"`) +- reconstructing the full tensor (`mode="full"`) +- lowering reconstruction-oriented paths onto existing runtime helpers such as `get_into_ranges(...)` where appropriate + +### Compatibility boundary + +The project should continue to preserve this boundary: + +- keep old TP APIs functional +- keep them thin +- do not expand the old TP-specific family as the primary interface +- document and evolve the unified `*_with_parallelism` family instead + +## Scope still intentionally limited + +This document describes the stable public API direction, but not every theoretical source-layout to target-layout remap is fully implemented. + +In particular, the design should continue to avoid over-promising planner coverage for arbitrary remaps across all DP / TP / EP / PP combinations until those paths are explicitly implemented and tested. + +The safe documented contract today is: + +1. define the public API structures and signatures clearly +2. align `store_py.cpp` with `axes + ReadTarget` +3. keep TP wrappers working by lowering into the unified path +4. do not introduce a new long-term `kind="mixed"` model +5. do not push planner internals such as byte ranges and derived shapes into public arguments +6. do not expand into many parallelism-specific public methods +7. reuse existing runtime helpers like `get_into_ranges(...)` rather than inventing a parallel reconstruction path from scratch diff --git a/docs/source/python-api-reference/mooncake-store.md b/docs/source/python-api-reference/mooncake-store.md index bb3610b1..ce1e7987 100644 --- a/docs/source/python-api-reference/mooncake-store.md +++ b/docs/source/python-api-reference/mooncake-store.md @@ -485,6 +485,231 @@ config.prefer_alloc_in_same_node = "True ``` --- +## Unified Parallel Tensor IO API + +Mooncake Store also provides a unified tensor IO family for tensors that are stored either as full objects or as explicitly identified parallel shards. + +This API family is the long-term interface for TP / DP / EP / PP-aware tensor IO: + +- write and upsert use `TensorParallelism` +- reads use `ReadTarget` +- legacy TP-only APIs remain available as compatibility wrappers + +### ParallelAxis + +`ParallelAxis` describes one axis in a shard identity. + +```python +axis = mooncake.store.ParallelAxis() +axis.kind = "tp" # one of: "tp", "dp", "ep", "pp" +axis.rank = 0 +axis.size = 8 +axis.split_dim = 1 # used for layout-sharding axes such as TP +axis.expert_id = 3 # optional, for EP +axis.stage_id = 1 # optional, for PP +``` + +**Fields:** +- `kind`: Parallelism axis kind. +- `rank`: Current shard rank on that axis. +- `size`: Total number of shards on that axis. +- `split_dim`: Optional tensor split dimension for layout-sharding axes. +- `expert_id`: Optional expert identifier for EP layouts. +- `stage_id`: Optional pipeline stage identifier for PP layouts. + +### TensorParallelism + +`TensorParallelism` is an ordered list of axes that identifies the stored or requested shard. + +```python +parallelism = mooncake.store.TensorParallelism() +parallelism.axes = [ + tp_axis, +] +``` + +Examples: +- TP shard: `axes=[TP(...)]` +- DP + TP shard: `axes=[DP(...), TP(...)]` +- PP + TP shard: `axes=[PP(...), TP(...)]` +- EP shard: `axes=[EP(...)]` + +### ReadTarget + +`ReadTarget` tells Mooncake whether the caller wants the stored form, a specific shard view, or the reconstructed full tensor. + +```python +target = mooncake.store.ReadTarget() +target.mode = "full" # one of: "as_stored", "shard", "full" +target.parallelism = None # required for target shard reads +``` + +**Fields:** +- `mode`: Read materialization mode. +- `parallelism`: Optional `TensorParallelism`. Required when `mode="shard"`. + +### put_tensor_with_parallelism() + +Store a tensor using the unified parallelism model. + +```python +def put_tensor_with_parallelism( + self, + key: str, + tensor, + parallelism: mooncake.store.TensorParallelism | None = None, + config: ReplicateConfig | None = None, + writer_partition = None, +) -> int +``` + +Use `parallelism=None` to store a full tensor object. Provide `TensorParallelism` to store a shard-scoped object. + +`writer_partition` is an optional write-side shorthand for full-tensor inputs that should be stored as one shard. It describes the writer's `(rank, size, split_dim)` and is mutually exclusive with `parallelism`; do not provide both in one call. + +For TP-containing multi-axis layouts, the caller may pass the full source tensor; Mooncake derives and persists the uniform shard selected by the requested TP rank/layout. That applies to layouts such as `dp_tp`, `pp_tp`, and `ep_tp`. + +Plain single-axis TP remains shard-input for compatibility. + +Pure DP still does not imply a split axis by itself. + +### batch_put_tensor_with_parallelism() + +Batch version of unified tensor writes. + +```python +def batch_put_tensor_with_parallelism( + self, + keys: list[str], + tensors: list, + parallelisms: list[mooncake.store.TensorParallelism | None] | None = None, + config: ReplicateConfig | None = None, + writer_partitions = None, +) -> list[int] +``` + +`writer_partitions` is an optional write-side convenience input for batch full-tensor writes that should be partitioned into stored shards. Each entry describes the target shard write as `(rank, size, split_dim)`. + +Use `writer_partitions` when the caller has full tensors and wants Mooncake to derive the stored shard objects from writer-side partition info instead of constructing full `TensorParallelism` objects per element. TP-containing `parallelisms` can now express the same full-tensor-input behavior too; `writer_partitions` remains the lighter explicit write-side shorthand. + +### get_tensor_with_parallelism() + +Read a tensor through the unified read path. + +```python +def get_tensor_with_parallelism( + self, + key: str, + target: mooncake.store.ReadTarget | None = None, +) +``` + +Typical modes: +- `target=None` or `mode="as_stored"`: return the stored local object. +- `mode="shard"`: return the target shard described by `target.parallelism`. +- `mode="full"`: reconstruct and return the full tensor. + +### batch_get_tensor_with_parallelism() + +Batch version of unified tensor reads. + +```python +def batch_get_tensor_with_parallelism( + self, + keys: list[str], + targets: list[mooncake.store.ReadTarget | None] | None = None, +) -> list +``` + +### get_tensor_with_parallelism_into() / batch_get_tensor_with_parallelism_into() + +Zero-copy unified read forms. The destination buffers must be registered with `register_buffer()` before calling them. + +```python +def get_tensor_with_parallelism_into( + self, + key: str, + buffer_ptr: int, + size: int, + target: mooncake.store.ReadTarget | None = None, +) +``` + +```python +def batch_get_tensor_with_parallelism_into( + self, + keys: list[str], + buffer_ptrs: list[int], + sizes: list[int], + targets: list[mooncake.store.ReadTarget | None] | None = None, +) -> list +``` + +### upsert_tensor_with_parallelism() + +Unified upsert form for tensor objects. + +```python +def upsert_tensor_with_parallelism( + self, + key: str, + tensor, + parallelism: mooncake.store.TensorParallelism | None = None, + config: ReplicateConfig | None = None, + writer_partition = None, +) -> int +``` + +The write semantics match `put_tensor_with_parallelism()`, including full-tensor input for TP-containing layouts and the mutually exclusive `writer_partition` shorthand. + +### batch_upsert_tensor_with_parallelism() + +Batch unified upsert form. + +```python +def batch_upsert_tensor_with_parallelism( + self, + keys: list[str], + tensors: list, + parallelisms: list[mooncake.store.TensorParallelism | None] | None = None, + config: ReplicateConfig | None = None, + writer_partitions = None, +) -> list[int] +``` + +The write semantics match `put_tensor_with_parallelism()`, including full-tensor input for TP-containing layouts. + +### *_from zero-copy write variants + +The unified write and upsert family also has `_from` variants for registered-memory inputs, including: + +- `put_tensor_with_parallelism_from(...)` +- `batch_put_tensor_with_parallelism_from(...)` +- `upsert_tensor_with_parallelism_from(...)` +- `batch_upsert_tensor_with_parallelism_from(...)` + +These APIs accept registered buffer pointers that contain serialized tensor objects in the current Mooncake tensor format: + +```text +[TensorObjectHeader + layout metadata][tensor data] +``` + +As with other zero-copy APIs, every source pointer must be registered with `register_buffer()` before use. + +### Compatibility wrappers + +Legacy TP-only methods such as: + +- `put_tensor_with_tp(...)` +- `batch_put_tensor_with_tp(...)` +- `get_tensor_with_tp(...)` +- `batch_get_tensor_with_tp(...)` +- corresponding `_into`, `_from`, and upsert variants + +remain supported for compatibility, but they are wrapper-style APIs around the unified parallel tensor IO model. Prefer the unified `*_with_parallelism` family for new code and new documentation examples. + +--- + ## Non-Zero-Copy API (Simple Usage) For simpler use cases, use the standard API without memory registration: @@ -1824,7 +2049,7 @@ def upsert_tensor(self, key: str, tensor: torch.Tensor) -> int #### upsert_tensor_from() Upsert a tensor directly from a pre-allocated buffer. The buffer layout must be -`[TensorMetadata][tensor data]`, matching the layout used by +`[TensorObjectHeader+layout metadata][tensor data]`, matching the layout used by `get_tensor_into()`. ```python @@ -1844,7 +2069,7 @@ def upsert_tensor_from(self, key: str, buffer_ptr: int, size: int) -> int #### batch_upsert_tensor_from() Upsert multiple tensors directly from pre-allocated buffers. Each buffer must -use layout `[TensorMetadata][tensor data]`. +use layout `[TensorObjectHeader+layout metadata][tensor data]`. ```python def batch_upsert_tensor_from(self, keys: List[str], buffer_ptrs: List[int], sizes: List[int]) -> List[int] @@ -1927,6 +2152,7 @@ def batch_upsert_pub_tensor(self, keys: List[str], tensors_list: List[torch.Tens **Note:** This function requires `torch` to be installed and available in the environment. Not supported for dummy client. + --- ### PyTorch Tensor Operations (Zero Copy) diff --git a/mooncake-integration/integration_utils.h b/mooncake-integration/integration_utils.h index 8179b72c..144b76e0 100644 --- a/mooncake-integration/integration_utils.h +++ b/mooncake-integration/integration_utils.h @@ -4,8 +4,15 @@ #include #include +#include #include +#include +#include +#include #include +#include +#include +#include namespace py = pybind11; @@ -96,10 +103,200 @@ inline TensorDtype get_tensor_dtype(py::object dtype_obj) { return TensorDtype::UNKNOWN; } -struct TensorMetadata { - int32_t dtype; - int32_t ndim; - int64_t shape[4]; +constexpr uint32_t kTensorObjectMagic = 0x4d4f4f4e; +constexpr uint16_t kTensorObjectVersion = 1; +constexpr size_t kMaxTensorDims = 8; +constexpr size_t kMaxLayoutAxes = 4; + +enum class TensorLayoutKind : uint32_t { + FULL = 0, + SHARD = 1, }; +enum class LayoutAxisKind : int32_t { + DP = 0, + TP = 1, + EP = 2, + PP = 3, + RESERVED = 4, +}; + +struct TensorShape { + int64_t dims[kMaxTensorDims]; +}; + +struct LayoutAxis { + int32_t kind; + int32_t axis_index; + int32_t shard_rank; + int32_t shard_count; + int32_t split_dim; + int32_t reserved0; + int64_t reserved1; +}; + +struct TensorLayoutMetadata { + TensorShape global_shape; + TensorShape local_shape; + uint32_t axis_count; + uint32_t reserved0; + LayoutAxis axes[kMaxLayoutAxes]; +}; + +struct TensorObjectHeader { + uint32_t magic; + uint16_t version; + uint16_t header_size; + int32_t dtype; + int32_t ndim; + uint32_t layout_kind; + uint32_t reserved_flags; + uint64_t data_offset; + uint64_t data_bytes; +}; + +struct TensorMetadata { + TensorObjectHeader header; + TensorLayoutMetadata layout; +}; + +struct ParsedTensorMetadata { + TensorMetadata metadata; + size_t data_offset; + size_t data_bytes; +}; + +inline TensorShape MakeTensorShape(const std::vector &dims) { + TensorShape shape{}; + std::fill(std::begin(shape.dims), std::end(shape.dims), -1); + for (size_t i = 0; i < dims.size() && i < kMaxTensorDims; ++i) { + shape.dims[i] = dims[i]; + } + return shape; +} + +inline std::vector TensorShapeToVector(const TensorShape &shape, + int32_t ndim) { + std::vector dims; + dims.reserve(ndim); + for (int32_t i = 0; i < ndim; ++i) { + dims.push_back(shape.dims[i]); + } + return dims; +} + +inline TensorMetadata BuildTensorMetadata( + int32_t dtype, const std::vector &global_shape, + const std::vector &local_shape, + TensorLayoutKind layout_kind = TensorLayoutKind::FULL, + std::span axes = {}) { + TensorMetadata metadata{}; + metadata.header.magic = kTensorObjectMagic; + metadata.header.version = kTensorObjectVersion; + metadata.header.header_size = sizeof(TensorMetadata); + metadata.header.dtype = dtype; + metadata.header.ndim = static_cast(global_shape.size()); + metadata.header.layout_kind = static_cast(layout_kind); + metadata.header.reserved_flags = 0; + metadata.header.data_offset = sizeof(TensorMetadata); + metadata.header.data_bytes = 0; + + metadata.layout.global_shape = MakeTensorShape(global_shape); + metadata.layout.local_shape = MakeTensorShape(local_shape); + metadata.layout.axis_count = + static_cast(std::min(axes.size(), kMaxLayoutAxes)); + metadata.layout.reserved0 = 0; + for (size_t i = 0; i < metadata.layout.axis_count; ++i) { + metadata.layout.axes[i] = axes[i]; + } + return metadata; +} + +inline bool ValidateTensorMetadata(const TensorMetadata &metadata, + size_t total_length) { + if (metadata.header.magic != kTensorObjectMagic || + metadata.header.version != kTensorObjectVersion || + metadata.header.header_size != sizeof(TensorMetadata)) { + return false; + } + + if (metadata.header.dtype < 0 || + metadata.header.dtype >= static_cast(TensorDtype::NR_DTYPES)) { + return false; + } + + if (metadata.header.ndim < 0 || + metadata.header.ndim > static_cast(kMaxTensorDims)) { + return false; + } + + if (metadata.layout.axis_count > kMaxLayoutAxes) { + return false; + } + + if (metadata.header.layout_kind > + static_cast(TensorLayoutKind::SHARD)) { + return false; + } + + if (metadata.header.data_offset < sizeof(TensorMetadata) || + metadata.header.data_offset > total_length) { + return false; + } + + if (metadata.header.data_bytes != + total_length - metadata.header.data_offset) { + return false; + } + + for (int32_t i = 0; i < metadata.header.ndim; ++i) { + if (metadata.layout.global_shape.dims[i] <= 0 || + metadata.layout.local_shape.dims[i] < 0) { + return false; + } + } + + for (size_t i = metadata.header.ndim; i < kMaxTensorDims; ++i) { + if (metadata.layout.global_shape.dims[i] != -1 || + metadata.layout.local_shape.dims[i] != -1) { + return false; + } + } + + for (size_t i = 0; i < metadata.layout.axis_count; ++i) { + const auto &axis = metadata.layout.axes[i]; + if (axis.kind < static_cast(LayoutAxisKind::DP) || + axis.kind > static_cast(LayoutAxisKind::RESERVED)) { + return false; + } + if (axis.shard_count <= 0 || axis.shard_rank < 0 || + axis.shard_rank >= axis.shard_count) { + return false; + } + if (axis.split_dim < -1 || axis.split_dim >= metadata.header.ndim) { + return false; + } + } + + return true; +} + +inline std::optional ParseTensorMetadata( + const char *data, size_t total_length) { + if (!data || total_length < sizeof(TensorMetadata)) { + return std::nullopt; + } + + ParsedTensorMetadata parsed{}; + std::memcpy(&parsed.metadata, data, sizeof(TensorMetadata)); + if (!ValidateTensorMetadata(parsed.metadata, total_length)) { + return std::nullopt; + } + + parsed.data_offset = + static_cast(parsed.metadata.header.data_offset); + parsed.data_bytes = static_cast(parsed.metadata.header.data_bytes); + return parsed; +} + } // namespace mooncake diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index 8872e4ed..324faea2 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -2,6 +2,11 @@ #include #include +#include +#include +#include +#include + #include "pyclient.h" #include "dummy_client.h" #include "real_client.h" @@ -20,54 +25,62 @@ struct PyTensorInfo { uintptr_t data_ptr; size_t tensor_size; TensorMetadata metadata; + py::object owner; - // Check validity bool valid() const { // Basic size check - if (tensor_size == 0 || data_ptr == 0) { + if (data_ptr == 0 && tensor_size != 0) { return false; } // Validate metadata + TensorMetadata validated = metadata; + validated.header.data_bytes = tensor_size; + // Check dtype is within valid range (0 to TensorDtype::NR_DTYPES, // excluding UNKNOWN=-1) - if (metadata.dtype < 0 || - metadata.dtype >= static_cast(TensorDtype::NR_DTYPES)) { + if (validated.header.dtype >= + static_cast(TensorDtype::NR_DTYPES)) { return false; } // Check ndim is within valid range (0 to shape array size) - const int kMaxDims = std::size(metadata.shape); - if (metadata.ndim < 0 || metadata.ndim > kMaxDims) { + const int kMaxDims = kMaxTensorDims; + if (validated.header.ndim < 0 || validated.header.ndim > kMaxDims) { return false; } // Validate shape array // For valid dimensions (0 to ndim-1), shape should be >= 0 - for (int i = 0; i < metadata.ndim; ++i) { - if (metadata.shape[i] <= 0) { + auto shape = TensorShapeToVector(validated.layout.local_shape, + validated.header.ndim); + for (int i = 0; i < validated.header.ndim; ++i) { + if (shape[i] <= 0) { return false; // Invalid dimension size } } - // For padding dimensions (ndim to kMaxDims-1), shape should be -1 - // (placeholder) - for (int i = metadata.ndim; i < kMaxDims; ++i) { - if (metadata.shape[i] != -1) { - return false; // Padding dimensions should be -1 - } - } - - return true; + return ValidateTensorMetadata( + validated, validated.header.data_offset + tensor_size); } }; +TensorMetadata build_full_tensor_metadata(const py::handle &tensor, + TensorDtype dtype_enum, + size_t tensor_size); +std::optional parse_tensor_metadata_from_buffer( + BufferHandle *buffer_handle, char *usr_buffer, int64_t data_length, + bool *take_ownership, char **exported_data, size_t *total_length); +std::pair calculate_shard_range(int64_t dim_size, int rank, + int shard_count); + PyTensorInfo extract_tensor_info(const py::object &tensor, const std::string &key_name = "") { PyTensorInfo info = { 0, 0, {}, + py::none(), }; if (!(tensor.attr("__class__") @@ -80,9 +93,12 @@ PyTensorInfo extract_tensor_info(const py::object &tensor, } try { - info.data_ptr = tensor.attr("data_ptr")().cast(); - size_t numel = tensor.attr("numel")().cast(); - size_t element_size = tensor.attr("element_size")().cast(); + py::object contiguous_tensor = tensor.attr("contiguous")(); + info.owner = contiguous_tensor; + info.data_ptr = contiguous_tensor.attr("data_ptr")().cast(); + size_t numel = contiguous_tensor.attr("numel")().cast(); + size_t element_size = + contiguous_tensor.attr("element_size")().cast(); info.tensor_size = numel * element_size; pybind11::object shape_obj = tensor.attr("shape"); @@ -92,28 +108,24 @@ PyTensorInfo extract_tensor_info(const py::object &tensor, if (dtype_enum == TensorDtype::UNKNOWN) { LOG(ERROR) << "Unsupported tensor dtype" << (key_name.empty() ? "" : " for " + key_name); - return {0, 0, {}}; + return {0, 0, {}, py::none()}; } pybind11::tuple shape_tuple = pybind11::cast(shape_obj); int32_t ndim = static_cast(shape_tuple.size()); - if (ndim > 4) { - LOG(ERROR) << "Tensor has more than 4 dimensions: " << ndim; - return {0, 0, {}}; + if (ndim > static_cast(kMaxTensorDims)) { + LOG(ERROR) << "Tensor has more than " << kMaxTensorDims + << " dimensions: " << ndim; + return {0, 0, {}, py::none()}; } - info.metadata.dtype = static_cast(dtype_enum); - info.metadata.ndim = ndim; - - for (int i = 0; i < 4; i++) { - info.metadata.shape[i] = - (i < ndim) ? shape_tuple[i].cast() : -1; - } + info.metadata = + build_full_tensor_metadata(tensor, dtype_enum, info.tensor_size); } catch (const std::exception &e) { LOG(ERROR) << "Error extracting tensor info: " << e.what(); - return {0, 0, {}}; + return {0, 0, {}, py::none()}; } return info; @@ -156,15 +168,26 @@ pybind11::object buffer_to_tensor(BufferHandle *buffer_handle, char *usr_buffer, TensorMetadata metadata; memcpy(&metadata, exported_data, sizeof(TensorMetadata)); - if (metadata.ndim < 0 || metadata.ndim > 4) { + auto parsed = ParseTensorMetadata(exported_data, total_length); + if (!parsed.has_value()) { if (take_ownership) { delete[] exported_data; } - LOG(ERROR) << "Invalid tensor metadata: ndim=" << metadata.ndim; + LOG(ERROR) << "Invalid tensor metadata"; return pybind11::none(); } - TensorDtype dtype_enum = static_cast(metadata.dtype); + int ndim = metadata.header.ndim; + if (ndim < 0 || ndim > static_cast(kMaxTensorDims)) { + if (take_ownership) { + delete[] exported_data; + } + LOG(ERROR) << "Invalid tensor metadata: ndim=" << ndim + << ", max supported=" << kMaxTensorDims; + return pybind11::none(); + } + + TensorDtype dtype_enum = static_cast(metadata.header.dtype); size_t tensor_size = total_length - sizeof(TensorMetadata); if (tensor_size == 0 || dtype_enum == TensorDtype::UNKNOWN) { @@ -192,10 +215,10 @@ pybind11::object buffer_to_tensor(BufferHandle *buffer_handle, char *usr_buffer, exported_data, sizeof(TensorMetadata), tensor_size, take_ownership); // Reshape - if (metadata.ndim > 0) { + if (ndim > 0) { std::vector shape_vec; - for (int i = 0; i < metadata.ndim; i++) { - shape_vec.push_back(metadata.shape[i]); + for (int i = 0; i < ndim; i++) { + shape_vec.push_back(metadata.layout.local_shape.dims[i]); } py::tuple shape_tuple = py::cast(shape_vec); np_array = np_array.attr("reshape")(shape_tuple); @@ -245,6 +268,9 @@ std::vector> CastAddrs2Ptrs( inline int to_py_ret(ErrorCode error_code) { return static_cast(error_code); } + +#include "store_py_internal.h" + } // namespace // Python-specific wrapper functions that handle GIL and return pybind11 types class MooncakeStorePyWrapper { @@ -276,7 +302,7 @@ class MooncakeStorePyWrapper { return store_->health_check(); } - std::string get_tp_key_name(const std::string &base_key, int rank) { + std::string get_tp_key_name(const std::string &base_key, int rank) const { return base_key + "_tp_" + std::to_string(rank); } @@ -605,77 +631,29 @@ class MooncakeStorePyWrapper { tp_size, split_dim); } + std::vector batch_put_tensor_infos_impl( + const std::vector &keys, + const std::vector &infos, + const ReplicateConfig &config = ReplicateConfig{}) { + return batch_write_tensor_impl( + keys, infos, config, "put", + [this, &config](const std::vector &write_keys, + const std::vector &buffer_ptrs, + const std::vector &buffer_sizes) { + return store_->batch_put_from(write_keys, buffer_ptrs, + buffer_sizes, config); + }); + } + std::vector batch_put_tensor_impl( const std::vector &keys, const pybind11::list &tensors_list, const ReplicateConfig &config = ReplicateConfig{}) { std::vector infos(keys.size()); - std::vector results(keys.size(), 0); - - // 1. Extract Metadata (GIL Held) for (size_t i = 0; i < keys.size(); ++i) { infos[i] = extract_tensor_info(tensors_list[i], keys[i]); - if (!infos[i].valid()) - results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); } - - // 2. Prepare Buffers and Execute (GIL Released) - { - py::gil_scoped_release release_gil; - - // Temporary containers for the batch operation - std::vector valid_keys; - std::vector buffer_ptrs; - std::vector buffer_sizes; - std::vector original_indices; // Map back to results - - // Note: In batch mode, we need contiguous memory for Metadata + - // Data. - std::vector> temp_allocations; - - for (size_t i = 0; i < infos.size(); ++i) { - if (!infos[i].valid()) continue; - - size_t total_size = - sizeof(TensorMetadata) + infos[i].tensor_size; - auto alloc_result = - store_->client_buffer_allocator_->allocate(total_size); - - if (!alloc_result) { - LOG(ERROR) - << "Failed to allocate buffer for key: " << keys[i]; - results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); - continue; - } - - // Copy Metadata & Data - char *dst = static_cast(alloc_result->ptr()); - memcpy(dst, &infos[i].metadata, sizeof(TensorMetadata)); - memcpy(dst + sizeof(TensorMetadata), - reinterpret_cast(infos[i].data_ptr), - infos[i].tensor_size); - - valid_keys.push_back(keys[i]); - buffer_ptrs.push_back(alloc_result->ptr()); - buffer_sizes.push_back(total_size); - original_indices.push_back(i); - - // Transfer ownership to temp_allocations so it survives until - // batch_put_from returns - temp_allocations.push_back( - std::make_unique(std::move(*alloc_result))); - } - - if (!valid_keys.empty()) { - std::vector op_results = store_->batch_put_from( - valid_keys, buffer_ptrs, buffer_sizes, config); - for (size_t i = 0; i < op_results.size(); ++i) { - results[original_indices[i]] = op_results[i]; - } - } - } - - return results; + return batch_put_tensor_infos_impl(keys, infos, config); } std::vector batch_put_tensor(const std::vector &keys, @@ -703,7 +681,6 @@ class MooncakeStorePyWrapper { std::vector processed_indices; std::vector final_results(base_keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); - try { // Chunking phase (GIL Held) for (size_t i = 0; i < base_keys.size(); ++i) { @@ -852,6 +829,44 @@ class MooncakeStorePyWrapper { return store_->batch_put_from(keys, buffers, sizes, ReplicateConfig{}); } + int put_tensor_info_impl(const std::string &key, const PyTensorInfo &info, + const ReplicateConfig &config) { + if (!info.valid()) return to_py_ret(ErrorCode::INVALID_PARAMS); + + std::vector> values; + values.emplace_back(reinterpret_cast(&info.metadata), + info.metadata.header.data_offset); + values.emplace_back(reinterpret_cast(info.data_ptr), + info.tensor_size); + + py::gil_scoped_release release_gil; + int ret = store_->put_parts(key, values, config); + if (ret != 0) + LOG(ERROR) << "put_parts failed for key " << key << " with code " + << ret; + return ret; + } + + int put_manifest_impl(const std::string &key, + const WriterShardManifest &manifest, + const ReplicateConfig &config) { + return write_manifest_impl( + key, manifest, "put", + [this, &key, &config](std::span bytes) { + return store_->put(key, bytes, config); + }); + } + + int upsert_manifest_impl(const std::string &key, + const WriterShardManifest &manifest, + const ReplicateConfig &config) { + return write_manifest_impl( + key, manifest, "upsert", + [this, &key, &config](std::span bytes) { + return store_->upsert(key, bytes, config); + }); + } + int put_tensor_with_tp_from(const std::string &key, uintptr_t buffer_ptr, size_t size, int tp_rank = 0, int tp_size = 1, int split_dim = 0) { @@ -967,8 +982,62 @@ class MooncakeStorePyWrapper { return final_results; } +#include "store_py_parallel_write.h" + + std::shared_ptr get_real_client() const { + if (use_dummy_client_) { + return nullptr; + } + return std::dynamic_pointer_cast(store_); + } + + std::optional + resolve_registered_buffer_region(uintptr_t buffer_ptr, size_t size, + const std::string &context) const { + auto real_client = get_real_client(); + if (!real_client) { + LOG(ERROR) << context << ": real client is not available"; + return std::nullopt; + } + + auto region = real_client->resolve_registered_buffer( + reinterpret_cast(buffer_ptr)); + if (!region.has_value()) { + LOG(ERROR) << context << ": buffer is not registered"; + return std::nullopt; + } + + if (region->offset + size > region->size) { + LOG(ERROR) << context << ": buffer range exceeds registered region"; + return std::nullopt; + } + + return region; + } + +#include "store_py_parallel_read.h" + // --- Upsert tensor methods --- + int upsert_tensor_info_impl(const std::string &key, + const PyTensorInfo &info, + const ReplicateConfig &config) { + if (!info.valid()) return to_py_ret(ErrorCode::INVALID_PARAMS); + + std::vector> values; + values.emplace_back(reinterpret_cast(&info.metadata), + info.metadata.header.data_offset); + values.emplace_back(reinterpret_cast(info.data_ptr), + info.tensor_size); + + py::gil_scoped_release release_gil; + int ret = store_->upsert_parts(key, values, config); + if (ret != 0) + LOG(ERROR) << "upsert_parts failed for key " << key << " with code " + << ret; + return ret; + } + int upsert_tensor_impl(const std::string &key, pybind11::object tensor, const ReplicateConfig &config) { auto info = extract_tensor_info(tensor, key); @@ -997,28 +1066,32 @@ class MooncakeStorePyWrapper { return upsert_tensor_impl(key, tensor, ReplicateConfig{}); } - int upsert_tensor_from(const std::string &key, uintptr_t buffer_ptr, - size_t size) { - if (buffer_ptr == 0) { - LOG(ERROR) << "Buffer pointer cannot be null"; + int upsert_tensor_with_tp_impl( + const std::string &key, pybind11::object tensor, + const ReplicateConfig &config = ReplicateConfig{}, int tp_size = 1, + int split_dim = 0) { + try { + py::tuple chunks = + tensor.attr("chunk")(tp_size, split_dim).cast(); + if (static_cast(chunks.size()) != tp_size) { + LOG(ERROR) << "Chunking failed: got " << chunks.size() + << " chunks, expected " << tp_size; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + for (int rank = 0; rank < tp_size; ++rank) { + pybind11::object chunk = chunks[rank].attr("contiguous")(); + std::string tp_key = get_tp_key_name(key, rank); + + int ret = upsert_tensor_impl(tp_key, chunk, config); + if (ret != 0) return ret; + } + return 0; + + } catch (const std::exception &e) { + LOG(ERROR) << "Failed to upsert tensor with tp: " << e.what(); return to_py_ret(ErrorCode::INVALID_PARAMS); } - void *buffer = reinterpret_cast(buffer_ptr); - if (!is_client_initialized()) { - LOG(ERROR) << "Client is not initialized"; - return to_py_ret(ErrorCode::INVALID_PARAMS); - } - if (use_dummy_client_) { - LOG(ERROR) - << "upsert_tensor_from is not supported for dummy client"; - return to_py_ret(ErrorCode::INVALID_PARAMS); - } - if (size <= sizeof(TensorMetadata)) { - LOG(ERROR) << "Buffer size too small for tensor metadata"; - return to_py_ret(ErrorCode::INVALID_PARAMS); - } - py::gil_scoped_release release_gil; - return store_->upsert_from(key, buffer, size, ReplicateConfig{}); } std::vector batch_upsert_tensor_from( @@ -1152,6 +1225,43 @@ class MooncakeStorePyWrapper { return batch_upsert_tensor_impl(keys, tensors_list, ReplicateConfig{}); } + int upsert_tensor_from(const std::string &key, uintptr_t buffer_ptr, + size_t size) { + if (buffer_ptr == 0) { + LOG(ERROR) << "Buffer pointer cannot be null"; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + void *buffer = reinterpret_cast(buffer_ptr); + if (!is_client_initialized()) { + LOG(ERROR) << "Client is not initialized"; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + if (use_dummy_client_) { + LOG(ERROR) + << "upsert_tensor_from is not supported for dummy client"; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + if (size <= sizeof(TensorMetadata)) { + LOG(ERROR) << "Buffer size too small for tensor metadata"; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + py::gil_scoped_release release_gil; + return store_->upsert_from(key, buffer, size, ReplicateConfig{}); + } + + int validate_replicate_config( + const ReplicateConfig &config = ReplicateConfig{}) { + if (!config.preferred_segments.empty() && + config.preferred_segments.size() != config.replica_num) { + LOG(ERROR) << "Preferred segments size (" + << config.preferred_segments.size() + << ") must match replica_num (" << config.replica_num + << ")"; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return 0; + } + int upsert_pub_tensor(const std::string &key, pybind11::object tensor, const ReplicateConfig &config = ReplicateConfig{}) { if (!is_client_initialized() || use_dummy_client_) { @@ -1190,19 +1300,6 @@ class MooncakeStorePyWrapper { } // --- End Upsert tensor methods --- - - int validate_replicate_config( - const ReplicateConfig &config = ReplicateConfig{}) { - if (!config.preferred_segments.empty() && - config.preferred_segments.size() != config.replica_num) { - LOG(ERROR) << "Preferred segments size (" - << config.preferred_segments.size() - << ") must match replica_num (" << config.replica_num - << ")"; - return to_py_ret(ErrorCode::INVALID_PARAMS); - } - return 0; - } int pub_tensor(const std::string &key, pybind11::object tensor, const ReplicateConfig &config = ReplicateConfig{}) { if (!is_client_initialized() || use_dummy_client_) { @@ -1549,6 +1646,43 @@ PYBIND11_MODULE(store, m) { return self.shm_helper_->free(reinterpret_cast(ptr)); }); + py::class_(m, "ParallelAxis") + .def(py::init<>()) + .def_readwrite("kind", &ParallelAxisSpec::kind) + .def_readwrite("rank", &ParallelAxisSpec::rank) + .def_readwrite("size", &ParallelAxisSpec::size) + .def_readwrite("split_dim", &ParallelAxisSpec::split_dim) + .def_readwrite("expert_id", &ParallelAxisSpec::expert_id) + .def_readwrite("stage_id", &ParallelAxisSpec::stage_id); + + py::class_(m, "TensorParallelism") + .def(py::init<>()) + .def_readwrite("axes", &TensorParallelismSpec::axes); + + py::class_(m, "ReadTarget") + .def(py::init<>()) + .def_property( + "mode", + [](const ReadTargetSpec &self) { + switch (self.mode) { + case ReadTargetMode::AS_STORED: + return std::string("as_stored"); + case ReadTargetMode::SHARD: + return std::string("shard"); + case ReadTargetMode::FULL: + return std::string("full"); + } + return std::string("as_stored"); + }, + [](ReadTargetSpec &self, const std::string &mode) { + auto parsed = parse_read_target_mode(py::str(mode)); + if (!parsed.has_value()) { + throw std::runtime_error("Unsupported ReadTarget mode"); + } + self.mode = *parsed; + }) + .def_readwrite("parallelism", &ReadTargetSpec::parallelism); + // Create a wrapper that exposes DistributedObjectStore with Python-specific // methods py::class_(m, "MooncakeDistributedStore") @@ -1834,7 +1968,8 @@ PYBIND11_MODULE(store, m) { " size: The size of buffer.\n" " tp_rank: The current tensor parallel rank (default 0).\n" " tp_size: The total tensor parallel size (default 1).\n" - " split_dim: The dimension to split the tensor along (default 0).") + " split_dim: The tensor dimension used for TP sharding " + "(default 0).") .def( "batch_get_tensor_with_tp_into", &MooncakeStorePyWrapper::batch_get_tensor_with_tp_into, @@ -1893,6 +2028,91 @@ PYBIND11_MODULE(store, m) { py::arg("tensors_list"), py::arg("config") = ReplicateConfig{}, "Batch upsert PyTorch tensors with configurable replication " "settings") + .def("get_tensor_with_parallelism", + &MooncakeStorePyWrapper::get_tensor_with_parallelism, + py::arg("key"), py::arg("target") = py::none(), + "Get a PyTorch tensor from the store using a ReadTarget request.") + .def("batch_get_tensor_with_parallelism", + &MooncakeStorePyWrapper::batch_get_tensor_with_parallelism, + py::arg("keys"), py::arg("targets") = py::none(), + "Get a batch of PyTorch tensors from the store using ReadTarget " + "requests.") + .def("get_tensor_with_parallelism_into", + &MooncakeStorePyWrapper::get_tensor_with_parallelism_into, + py::arg("key"), py::arg("buffer_ptr"), py::arg("size"), + py::arg("target") = py::none(), + "Get a PyTorch tensor from the store directly into a " + "pre-allocated buffer using a ReadTarget request.") + .def("batch_get_tensor_with_parallelism_into", + &MooncakeStorePyWrapper::batch_get_tensor_with_parallelism_into, + py::arg("keys"), py::arg("buffer_ptrs"), py::arg("sizes"), + py::arg("targets") = py::none(), + "Get a batch of PyTorch tensors into pre-allocated buffers using " + "ReadTarget requests.") + .def("put_tensor_with_parallelism", + &MooncakeStorePyWrapper::put_tensor_with_parallelism, + py::arg("key"), py::arg("tensor"), + py::arg("parallelism") = py::none(), + py::arg("config") = ReplicateConfig{}, + py::arg("writer_partition") = py::none(), + "Put a PyTorch tensor into the store using a TensorParallelism " + "request or a writer_partition request.") + .def("batch_put_tensor_with_parallelism", + &MooncakeStorePyWrapper::batch_put_tensor_with_parallelism, + py::arg("keys"), py::arg("tensors_list"), + py::arg("parallelisms") = py::none(), + py::arg("config") = ReplicateConfig{}, + py::arg("writer_partitions") = py::none(), + "Put a batch of PyTorch tensors into the store using " + "TensorParallelism requests or writer_partition requests.") + .def("put_tensor_with_parallelism_from", + &MooncakeStorePyWrapper::put_tensor_with_parallelism_from, + py::arg("key"), py::arg("buffer_ptr"), py::arg("size"), + py::arg("parallelism") = py::none(), + py::arg("config") = ReplicateConfig{}, + py::arg("writer_partition") = py::none(), + "Put a tensor directly from a pre-allocated buffer using a " + "TensorParallelism request or a writer_partition request.") + .def("batch_put_tensor_with_parallelism_from", + &MooncakeStorePyWrapper::batch_put_tensor_with_parallelism_from, + py::arg("keys"), py::arg("buffer_ptrs"), py::arg("sizes"), + py::arg("parallelisms") = py::none(), + py::arg("config") = ReplicateConfig{}, + py::arg("writer_partitions") = py::none(), + "Put a batch of tensors directly from pre-allocated buffers using " + "TensorParallelism requests or writer_partition requests.") + .def("upsert_tensor_with_parallelism", + &MooncakeStorePyWrapper::upsert_tensor_with_parallelism, + py::arg("key"), py::arg("tensor"), + py::arg("parallelism") = py::none(), + py::arg("config") = ReplicateConfig{}, + py::arg("writer_partition") = py::none(), + "Upsert a PyTorch tensor into the store using a TensorParallelism " + "request or a writer_partition request.") + .def("upsert_tensor_with_parallelism_from", + &MooncakeStorePyWrapper::upsert_tensor_with_parallelism_from, + py::arg("key"), py::arg("buffer_ptr"), py::arg("size"), + py::arg("parallelism") = py::none(), + py::arg("config") = ReplicateConfig{}, + py::arg("writer_partition") = py::none(), + "Upsert a tensor directly from a pre-allocated buffer using a " + "TensorParallelism request or a writer_partition request.") + .def("batch_upsert_tensor_with_parallelism_from", + &MooncakeStorePyWrapper::batch_upsert_tensor_with_parallelism_from, + py::arg("keys"), py::arg("buffer_ptrs"), py::arg("sizes"), + py::arg("parallelisms") = py::none(), + py::arg("config") = ReplicateConfig{}, + py::arg("writer_partitions") = py::none(), + "Upsert a batch of tensors directly from pre-allocated buffers " + "using TensorParallelism requests or writer_partition requests.") + .def("batch_upsert_tensor_with_parallelism", + &MooncakeStorePyWrapper::batch_upsert_tensor_with_parallelism, + py::arg("keys"), py::arg("tensors_list"), + py::arg("parallelisms") = py::none(), + py::arg("config") = ReplicateConfig{}, + py::arg("writer_partitions") = py::none(), + "Upsert a batch of PyTorch tensors into the store using " + "TensorParallelism requests or writer_partition requests.") .def( "upsert_from", [](MooncakeStorePyWrapper &self, const std::string &key, diff --git a/mooncake-integration/store/store_py_internal.h b/mooncake-integration/store/store_py_internal.h new file mode 100644 index 00000000..0734bede --- /dev/null +++ b/mooncake-integration/store/store_py_internal.h @@ -0,0 +1,1483 @@ +std::vector tensor_shape_to_vector(const py::handle &tensor) { + py::tuple shape_tuple = py::cast(tensor.attr("shape")); + std::vector shape; + shape.reserve(shape_tuple.size()); + for (const auto &dim : shape_tuple) { + shape.push_back(dim.cast()); + } + return shape; +} + +enum class ReadTargetMode { + AS_STORED, + SHARD, + FULL, +}; + +struct ParallelAxisSpec { + std::string kind; + int rank{0}; + int size{1}; + std::optional split_dim; + std::optional expert_id; + std::optional stage_id; +}; + +struct TensorParallelismSpec { + std::vector axes; +}; + +struct ReadTargetSpec { + ReadTargetMode mode{ReadTargetMode::AS_STORED}; + std::optional parallelism; +}; + +struct WriterPartitionSpec { + int rank{0}; + int size{1}; + int split_dim{0}; +}; + +struct WriterShardManifestHeader { + uint32_t magic; + uint16_t version; + uint16_t header_size; + int32_t dtype; + int32_t ndim; + int32_t split_dim; + int32_t shard_count; + int32_t reserved0; +}; + +struct WriterShardManifest { + WriterShardManifestHeader header; + TensorShape global_shape; +}; + +struct ParsedWriterShardManifest { + WriterShardManifest manifest; + std::vector global_shape; +}; + +using ParallelismManifest = WriterShardManifest; +using ParsedParallelismManifest = ParsedWriterShardManifest; + +struct WriterShardTensorInfo { + PyTensorInfo info; + std::string shard_key; + WriterShardManifest manifest; +}; + +struct ParallelismShardTensorInfo { + PyTensorInfo info; + ParallelismManifest manifest; +}; + +struct RawTensorShardWritePlan { + TensorMetadata metadata; + std::vector> data_ranges; +}; + +struct TensorIntoFragment { + std::string read_key; + size_t dst_offset{0}; + size_t src_offset{0}; + size_t size{0}; +}; + +struct TensorIntoPlan { + uintptr_t user_buffer_ptr{0}; + uintptr_t registered_buffer_ptr{0}; + size_t registered_buffer_size{0}; + size_t total_length{0}; + std::vector fragments; + std::optional materialized_metadata; +}; + +struct ResolvedTensorRead { + std::string read_key; + ParsedTensorMetadata metadata; + std::shared_ptr buffer_handle; +}; + +enum class ParallelismShardReadStorageRoute { + PARALLELISM_SHARD_KEY, + WRITER_PARTITION_COMPATIBLE_SHARD, +}; + +struct ReconstructedShardSource { + std::string read_key; + ParsedTensorMetadata metadata; +}; + +struct FullTensorReconstructionSources { + std::vector sources; + std::vector global_shape; + int split_dim{0}; + int32_t dtype{0}; + bool allow_empty_fragments{false}; +}; + +std::optional parse_layout_axis_kind(const std::string &kind); +std::optional build_shard_metadata_from_shapes( + int32_t dtype, const std::vector &global_shape, + const std::vector &local_shape, + const std::vector &axes, size_t tensor_size); +std::optional build_shard_metadata_from_parallelism( + const py::handle &full_tensor, const py::handle &shard_tensor, + TensorDtype dtype_enum, const std::vector &axes, + size_t tensor_size); +std::optional find_tp_axis_index( + const std::vector &axes); +bool validate_uniform_shard_request(const std::vector &shape, + int split_dim, int shard_count, + const std::string &error_context); +std::optional materialize_shard_tensor(const py::handle &tensor, + int split_dim, int rank, + int shard_count); +std::string get_parallelism_key_name(const std::string &base_key, + const TensorParallelismSpec ¶llelism); +bool parallelism_matches_metadata(const TensorParallelismSpec ¶llelism, + const TensorMetadata &metadata); +std::optional parallelism_from_metadata( + const TensorMetadata &metadata); +std::optional parse_tensor_parallelism_spec( + const py::object ¶llelism_obj); +std::optional validate_parallelism_spec( + const std::optional ¶llelism, + const std::string &error_context, bool allow_empty); +std::optional canonicalize_parallelism_spec( + const TensorParallelismSpec ¶llelism); +std::optional parse_writer_partition_spec( + const py::object &obj, const std::string &error_context); + +constexpr uint32_t kWriterShardManifestMagic = 0x574d414e; +constexpr uint16_t kWriterShardManifestVersion = 1; + +std::string get_writer_manifest_key_name(const std::string &base_key) { + return base_key + "__writer_manifest"; +} + +std::string get_parallelism_manifest_key_name(const std::string &base_key) { + return base_key + "__parallelism_manifest"; +} + +std::string get_writer_shard_key_name(const std::string &base_key, + const WriterPartitionSpec &writer) { + return base_key + "__writer_" + std::to_string(writer.rank) + "of" + + std::to_string(writer.size) + "_sd" + + std::to_string(writer.split_dim); +} + +std::optional parse_writer_shard_manifest( + BufferHandle *buffer_handle) { + if (!buffer_handle || buffer_handle->size() < sizeof(WriterShardManifest)) { + return std::nullopt; + } + + ParsedWriterShardManifest parsed{}; + std::memcpy(&parsed.manifest, buffer_handle->ptr(), + sizeof(WriterShardManifest)); + if (parsed.manifest.header.magic != kWriterShardManifestMagic || + parsed.manifest.header.version != kWriterShardManifestVersion || + parsed.manifest.header.header_size != sizeof(WriterShardManifest) || + parsed.manifest.header.ndim < 0 || + parsed.manifest.header.ndim > static_cast(kMaxTensorDims) || + parsed.manifest.header.shard_count <= 0 || + parsed.manifest.header.split_dim < 0 || + parsed.manifest.header.split_dim >= parsed.manifest.header.ndim) { + return std::nullopt; + } + + parsed.global_shape = TensorShapeToVector(parsed.manifest.global_shape, + parsed.manifest.header.ndim); + for (auto dim : parsed.global_shape) { + if (dim <= 0) { + return std::nullopt; + } + } + return parsed; +} + +WriterShardManifest build_writer_shard_manifest_from_shape( + const std::vector &shape, int32_t dtype, + const WriterPartitionSpec &writer) { + WriterShardManifest manifest{}; + manifest.header.magic = kWriterShardManifestMagic; + manifest.header.version = kWriterShardManifestVersion; + manifest.header.header_size = sizeof(WriterShardManifest); + manifest.header.dtype = dtype; + manifest.header.ndim = static_cast(shape.size()); + manifest.header.split_dim = writer.split_dim; + manifest.header.shard_count = writer.size; + manifest.header.reserved0 = 0; + manifest.global_shape = MakeTensorShape(shape); + return manifest; +} + +ParallelismManifest build_parallelism_manifest_from_shape( + const std::vector &shape, int32_t dtype, int split_dim, + int shard_count) { + ParallelismManifest manifest{}; + manifest.header.magic = kWriterShardManifestMagic; + manifest.header.version = kWriterShardManifestVersion; + manifest.header.header_size = sizeof(ParallelismManifest); + manifest.header.dtype = dtype; + manifest.header.ndim = static_cast(shape.size()); + manifest.header.split_dim = split_dim; + manifest.header.shard_count = shard_count; + manifest.header.reserved0 = 0; + manifest.global_shape = MakeTensorShape(shape); + return manifest; +} + +WriterShardManifest build_writer_shard_manifest( + const py::handle &tensor, TensorDtype dtype_enum, + const WriterPartitionSpec &writer) { + return build_writer_shard_manifest_from_shape( + tensor_shape_to_vector(tensor), static_cast(dtype_enum), + writer); +} + +TensorMetadata build_writer_shard_tensor_metadata( + const py::handle &full_tensor, const py::handle &shard_tensor, + TensorDtype dtype_enum, const WriterPartitionSpec &writer, + size_t tensor_size) { + ParallelAxisSpec axis_spec{"tp", writer.rank, + writer.size, writer.split_dim, + std::nullopt, std::nullopt}; + auto metadata = build_shard_metadata_from_parallelism( + full_tensor, shard_tensor, dtype_enum, {axis_spec}, tensor_size); + return metadata.value_or(TensorMetadata{}); +} + +TensorMetadata build_full_tensor_metadata(const py::handle &tensor, + TensorDtype dtype_enum, + size_t tensor_size) { + auto shape = tensor_shape_to_vector(tensor); + TensorMetadata metadata = BuildTensorMetadata( + static_cast(dtype_enum), shape, shape, TensorLayoutKind::FULL); + metadata.header.data_bytes = tensor_size; + return metadata; +} + +std::optional build_layout_axis_from_spec( + const ParallelAxisSpec &axis_spec, size_t axis_index, + int64_t local_split_extent = -1) { + auto kind = parse_layout_axis_kind(axis_spec.kind); + if (!kind.has_value()) { + return std::nullopt; + } + + LayoutAxis axis{}; + axis.kind = static_cast(*kind); + axis.axis_index = static_cast(axis_index); + axis.shard_rank = axis_spec.rank; + axis.shard_count = axis_spec.size; + axis.split_dim = axis_spec.split_dim.value_or(-1); + axis.reserved0 = 0; + axis.reserved1 = 0; + + if (*kind == LayoutAxisKind::EP && axis_spec.expert_id.has_value()) { + axis.reserved0 = axis_spec.expert_id.value(); + } + if (*kind == LayoutAxisKind::PP && axis_spec.stage_id.has_value()) { + axis.reserved0 = axis_spec.stage_id.value(); + } + if (local_split_extent >= 0) { + axis.reserved1 = local_split_extent; + } + return axis; +} + +std::optional build_shard_metadata_from_shapes( + int32_t dtype, const std::vector &global_shape, + const std::vector &local_shape, + const std::vector &axes, size_t tensor_size) { + if (axes.empty() || axes.size() > kMaxLayoutAxes || + global_shape.size() != local_shape.size()) { + return std::nullopt; + } + + std::vector layout_axes; + layout_axes.reserve(axes.size()); + for (size_t i = 0; i < axes.size(); ++i) { + int64_t local_split_extent = -1; + if (axes[i].split_dim.has_value()) { + int split_dim = axes[i].split_dim.value(); + if (split_dim < 0 || + split_dim >= static_cast(local_shape.size())) { + return std::nullopt; + } + local_split_extent = local_shape[split_dim]; + } + auto layout_axis = + build_layout_axis_from_spec(axes[i], i, local_split_extent); + if (!layout_axis.has_value()) { + return std::nullopt; + } + layout_axes.push_back(*layout_axis); + } + + TensorMetadata metadata = BuildTensorMetadata( + dtype, global_shape, local_shape, TensorLayoutKind::SHARD, + std::span(layout_axes.data(), layout_axes.size())); + metadata.header.data_bytes = tensor_size; + return metadata; +} + +std::optional build_shard_metadata_from_parallelism( + const py::handle &full_tensor, const py::handle &shard_tensor, + TensorDtype dtype_enum, const std::vector &axes, + size_t tensor_size) { + return build_shard_metadata_from_shapes( + static_cast(dtype_enum), tensor_shape_to_vector(full_tensor), + tensor_shape_to_vector(shard_tensor), axes, tensor_size); +} + +bool validate_uniform_shard_request(const std::vector &shape, + int split_dim, int shard_count, + const std::string &error_context); + +TensorMetadata build_tp_shard_metadata(const py::handle &full_tensor, + const py::handle &shard_tensor, + TensorDtype dtype_enum, int tp_rank, + int tp_size, int split_dim, + size_t tensor_size) { + ParallelAxisSpec axis_spec{"tp", tp_rank, tp_size, + split_dim, std::nullopt, std::nullopt}; + auto metadata = build_shard_metadata_from_parallelism( + full_tensor, shard_tensor, dtype_enum, {axis_spec}, tensor_size); + return metadata.value_or(TensorMetadata{}); +} + +std::optional find_tp_axis_index( + const std::vector &axes) { + for (size_t i = 0; i < axes.size(); ++i) { + auto kind = parse_layout_axis_kind(axes[i].kind); + if (kind == std::optional(LayoutAxisKind::TP)) { + return i; + } + } + return std::nullopt; +} + +std::optional build_direct_parallelism_shard_info( + const py::handle &tensor, const std::vector &axes, + const std::string &key_name) { + TensorDtype dtype_enum = get_tensor_dtype(tensor.attr("dtype")); + if (dtype_enum == TensorDtype::UNKNOWN) { + return std::nullopt; + } + + auto info = extract_tensor_info(py::reinterpret_borrow(tensor), + key_name); + auto metadata = build_shard_metadata_from_parallelism( + tensor, tensor, dtype_enum, axes, info.tensor_size); + if (!metadata.has_value()) { + return std::nullopt; + } + info.metadata = *metadata; + if (!info.valid()) { + return std::nullopt; + } + return info; +} + +std::optional build_direct_parallelism_shard_info( + const py::handle &shard_tensor, const std::vector &axes, + const std::string &key_name, bool infer_global_shape) { + auto tp_axis_index = find_tp_axis_index(axes); + if (!tp_axis_index.has_value()) { + return std::nullopt; + } + + const auto &tp_axis = axes[*tp_axis_index]; + if (!tp_axis.split_dim.has_value()) { + return std::nullopt; + } + + auto local_shape = tensor_shape_to_vector(shard_tensor); + int split_dim = tp_axis.split_dim.value(); + if (split_dim < 0 || split_dim >= static_cast(local_shape.size())) { + return std::nullopt; + } + + auto global_shape = local_shape; + if (infer_global_shape) { + global_shape[split_dim] *= tp_axis.size; + } + if (!validate_uniform_shard_request( + global_shape, split_dim, tp_axis.size, + "build_direct_parallelism_shard_info")) { + return std::nullopt; + } + + TensorDtype dtype_enum = get_tensor_dtype(shard_tensor.attr("dtype")); + if (dtype_enum == TensorDtype::UNKNOWN) { + return std::nullopt; + } + + ParallelismShardTensorInfo shard_info; + shard_info.info = extract_tensor_info( + py::reinterpret_borrow(shard_tensor), key_name); + auto metadata = build_shard_metadata_from_parallelism( + shard_tensor, shard_tensor, dtype_enum, axes, + shard_info.info.tensor_size); + if (!metadata.has_value()) { + return std::nullopt; + } + shard_info.info.metadata = *metadata; + shard_info.info.metadata.layout.global_shape = + MakeTensorShape(global_shape); + if (!shard_info.info.valid()) { + return std::nullopt; + } + shard_info.manifest = build_parallelism_manifest_from_shape( + global_shape, static_cast(dtype_enum), split_dim, + tp_axis.size); + return shard_info; +} + +std::optional +build_requested_parallelism_shard_info(const py::handle &tensor, + const TensorParallelismSpec ¶llelism, + const std::string &key_name, + const std::string &error_context) { + auto tp_axis_index = find_tp_axis_index(parallelism.axes); + if (!tp_axis_index.has_value()) { + return std::nullopt; + } + + const auto &tp_axis = parallelism.axes[*tp_axis_index]; + if (!tp_axis.split_dim.has_value()) { + return std::nullopt; + } + + const int split_dim = tp_axis.split_dim.value(); + auto global_shape = tensor_shape_to_vector(tensor); + if (!validate_uniform_shard_request(global_shape, split_dim, tp_axis.size, + error_context)) { + return std::nullopt; + } + + TensorDtype dtype_enum = get_tensor_dtype(tensor.attr("dtype")); + if (dtype_enum == TensorDtype::UNKNOWN) { + return std::nullopt; + } + + auto shard_tensor = + materialize_shard_tensor(tensor, split_dim, tp_axis.rank, tp_axis.size); + if (!shard_tensor.has_value()) { + return std::nullopt; + } + + ParallelismShardTensorInfo shard_info; + shard_info.info = extract_tensor_info(*shard_tensor, key_name); + auto metadata = build_shard_metadata_from_parallelism( + tensor, *shard_tensor, dtype_enum, parallelism.axes, + shard_info.info.tensor_size); + if (!metadata.has_value()) { + return std::nullopt; + } + shard_info.info.metadata = *metadata; + if (!shard_info.info.valid()) { + return std::nullopt; + } + shard_info.manifest = build_parallelism_manifest_from_shape( + global_shape, static_cast(dtype_enum), split_dim, + tp_axis.size); + return shard_info; +} + +bool is_uniform_shardable_dim(int64_t dim_size, int shard_count) { + return dim_size >= 0 && shard_count > 0 && dim_size % shard_count == 0; +} + +bool validate_uniform_shard_request(const std::vector &shape, + int split_dim, int shard_count, + const std::string &error_context) { + if (split_dim < 0 || split_dim >= static_cast(shape.size()) || + shard_count <= 0) { + LOG(ERROR) << error_context << ": invalid shard parameters"; + return false; + } + if (!is_uniform_shardable_dim(shape[split_dim], shard_count)) { + LOG(ERROR) << error_context << ": only uniform sharding is supported"; + return false; + } + return true; +} + +std::optional materialize_shard_tensor(const py::handle &tensor, + int split_dim, int rank, + int shard_count) { + auto shape = tensor_shape_to_vector(tensor); + if (rank < 0 || rank >= shard_count || + !validate_uniform_shard_request(shape, split_dim, shard_count, + "materialize_shard_tensor")) { + return std::nullopt; + } + + py::object torch_tensor = py::reinterpret_borrow(tensor); + const auto [start, size] = + calculate_shard_range(shape[split_dim], rank, shard_count); + return torch_tensor.attr("narrow")(split_dim, start, size) + .attr("contiguous")(); +} + +std::optional> build_tp_shard_infos( + const py::handle &tensor, int tp_size, int split_dim, + const std::function &key_for_rank, + const std::vector &axes = {}) { + TensorDtype dtype_enum = get_tensor_dtype(tensor.attr("dtype")); + if (dtype_enum == TensorDtype::UNKNOWN) { + return std::nullopt; + } + + auto shape = tensor_shape_to_vector(tensor); + if (!validate_uniform_shard_request(shape, split_dim, tp_size, + "build_tp_shard_infos")) { + return std::nullopt; + } + + auto tp_axis_index = find_tp_axis_index(axes); + std::vector infos; + infos.reserve(tp_size); + + for (int rank = 0; rank < tp_size; ++rank) { + auto chunk = materialize_shard_tensor(tensor, split_dim, rank, tp_size); + if (!chunk.has_value()) { + return std::nullopt; + } + + auto info = extract_tensor_info(*chunk, key_for_rank(rank)); + if (axes.empty()) { + info.metadata = + build_tp_shard_metadata(tensor, *chunk, dtype_enum, rank, + tp_size, split_dim, info.tensor_size); + } else { + auto shard_axes = axes; + if (!tp_axis_index.has_value()) { + return std::nullopt; + } + shard_axes[*tp_axis_index].rank = rank; + auto metadata = build_shard_metadata_from_parallelism( + tensor, *chunk, dtype_enum, shard_axes, info.tensor_size); + if (!metadata.has_value()) { + return std::nullopt; + } + info.metadata = *metadata; + } + if (!info.valid()) { + return std::nullopt; + } + infos.push_back(info); + } + return infos; +} + +bool is_shard_tensor_metadata(const TensorMetadata &metadata) { + return metadata.header.layout_kind == + static_cast(TensorLayoutKind::SHARD); +} + +const LayoutAxis *find_layout_axis(const TensorMetadata &metadata, + LayoutAxisKind kind) { + for (size_t i = 0; i < metadata.layout.axis_count; ++i) { + if (metadata.layout.axes[i].kind == static_cast(kind)) { + return &metadata.layout.axes[i]; + } + } + return nullptr; +} + +std::optional parse_layout_axis_kind(const std::string &kind) { + std::string normalized = kind; + std::transform(normalized.begin(), normalized.end(), normalized.begin(), + [](unsigned char c) { return std::toupper(c); }); + if (normalized == "TP") return LayoutAxisKind::TP; + if (normalized == "DP") return LayoutAxisKind::DP; + if (normalized == "EP") return LayoutAxisKind::EP; + if (normalized == "PP") return LayoutAxisKind::PP; + LOG(ERROR) << "Unsupported parallel axis kind: " << kind; + return std::nullopt; +} + +bool is_single_axis_parallelism_kind(const TensorParallelismSpec ¶llelism, + LayoutAxisKind kind) { + return parallelism.axes.size() == 1 && + parse_layout_axis_kind(parallelism.axes[0].kind) == + std::optional(kind); +} + +bool uses_legacy_tp_storage_key(const TensorParallelismSpec ¶llelism) { + return is_single_axis_parallelism_kind(parallelism, LayoutAxisKind::TP); +} + +bool should_use_legacy_single_tp_write_route( + const TensorParallelismSpec ¶llelism) { + return false; +} + +bool can_read_single_tp_request_from_writer_partition_storage( + const TensorParallelismSpec ¶llelism) { + return uses_legacy_tp_storage_key(parallelism); +} + +std::vector +resolve_parallelism_shard_read_routes( + const TensorParallelismSpec ¶llelism) { + std::vector routes{ + ParallelismShardReadStorageRoute::PARALLELISM_SHARD_KEY, + }; + if (can_read_single_tp_request_from_writer_partition_storage(parallelism)) { + routes.push_back(ParallelismShardReadStorageRoute:: + WRITER_PARTITION_COMPATIBLE_SHARD); + } + return routes; +} + +std::optional writer_partition_parallelism_from_metadata( + const TensorMetadata &metadata) { + auto parallelism = parallelism_from_metadata(metadata); + if (!parallelism.has_value() || !uses_legacy_tp_storage_key(*parallelism)) { + return std::nullopt; + } + return parallelism; +} + +enum class ParallelismWriteStorageRoute { + DIRECT_FULL_OBJECT, + WRITER_PARTITION_SHARD, + LEGACY_SINGLE_TP, + TP_SHARDED_PARALLELISM, + GENERIC_PARALLELISM_SHARD, +}; + +struct ResolvedParallelismWriteRequest { + ParallelismWriteStorageRoute route; + std::optional parallelism; + std::optional writer_partition; +}; + +struct TensorWriteStoreOps { + const char *parts_operation_name; +}; + +std::optional +resolve_parallelism_write_request(const py::object ¶llelism_obj, + const py::object &writer_partition_obj, + const std::string &error_context) { + if (!writer_partition_obj.is_none()) { + if (!parallelism_obj.is_none()) { + LOG(ERROR) + << error_context + << ": writer_partition cannot be combined with parallelism"; + return std::nullopt; + } + auto writer_partition = + parse_writer_partition_spec(writer_partition_obj, error_context); + if (!writer_partition.has_value()) { + return std::nullopt; + } + return ResolvedParallelismWriteRequest{ + ParallelismWriteStorageRoute::WRITER_PARTITION_SHARD, + std::nullopt, + *writer_partition, + }; + } + if (parallelism_obj.is_none()) { + return ResolvedParallelismWriteRequest{ + ParallelismWriteStorageRoute::DIRECT_FULL_OBJECT, + std::nullopt, + std::nullopt, + }; + } + + auto parallelism = validate_parallelism_spec( + parse_tensor_parallelism_spec(parallelism_obj), error_context, false); + if (!parallelism.has_value()) { + return std::nullopt; + } + if (should_use_legacy_single_tp_write_route(*parallelism)) { + return ResolvedParallelismWriteRequest{ + ParallelismWriteStorageRoute::LEGACY_SINGLE_TP, + *parallelism, + std::nullopt, + }; + } + for (const auto &axis : parallelism->axes) { + if (parse_layout_axis_kind(axis.kind) == + std::optional(LayoutAxisKind::TP)) { + return ResolvedParallelismWriteRequest{ + ParallelismWriteStorageRoute::TP_SHARDED_PARALLELISM, + *parallelism, + std::nullopt, + }; + } + } + return ResolvedParallelismWriteRequest{ + ParallelismWriteStorageRoute::GENERIC_PARALLELISM_SHARD, + *parallelism, + std::nullopt, + }; +} + +std::optional validate_batch_request_list( + const py::object &request_list_obj, size_t expected_size, + const std::string &error_context, const char *request_name) { + if (!py::isinstance(request_list_obj)) { + LOG(ERROR) << error_context << ": " << request_name + << " must be a list or None"; + return std::nullopt; + } + + py::list request_list = py::cast(request_list_obj); + if (request_list.size() != expected_size) { + LOG(ERROR) << error_context << ": keys and " << request_name + << " must have the same length"; + return std::nullopt; + } + return request_list; +} + +std::optional validate_batch_parallelism_list( + const py::object ¶llelisms, size_t expected_size, + const std::string &error_context) { + return validate_batch_request_list(parallelisms, expected_size, + error_context, "parallelisms"); +} + +std::optional validate_batch_writer_partition_list( + const py::object &writer_partitions, size_t expected_size, + const std::string &error_context) { + return validate_batch_request_list(writer_partitions, expected_size, + error_context, "writer_partitions"); +} + +bool axis_specs_equal(const ParallelAxisSpec &lhs, + const ParallelAxisSpec &rhs) { + auto lhs_kind = parse_layout_axis_kind(lhs.kind); + auto rhs_kind = parse_layout_axis_kind(rhs.kind); + return lhs_kind.has_value() && rhs_kind.has_value() && + lhs_kind == rhs_kind && lhs.rank == rhs.rank && + lhs.size == rhs.size && lhs.split_dim == rhs.split_dim && + lhs.expert_id == rhs.expert_id && lhs.stage_id == rhs.stage_id; +} + +int canonical_axis_kind_order(LayoutAxisKind kind) { + switch (kind) { + case LayoutAxisKind::DP: + return 0; + case LayoutAxisKind::TP: + return 1; + case LayoutAxisKind::EP: + return 2; + case LayoutAxisKind::PP: + return 3; + default: + return 4; + } +} + +std::optional canonicalize_parallelism_spec( + const TensorParallelismSpec ¶llelism) { + TensorParallelismSpec canonical = parallelism; + std::sort(canonical.axes.begin(), canonical.axes.end(), + [](const ParallelAxisSpec &lhs, const ParallelAxisSpec &rhs) { + auto lhs_kind = parse_layout_axis_kind(lhs.kind); + auto rhs_kind = parse_layout_axis_kind(rhs.kind); + if (!lhs_kind.has_value() || !rhs_kind.has_value()) { + return lhs.kind < rhs.kind; + } + int lhs_order = canonical_axis_kind_order(*lhs_kind); + int rhs_order = canonical_axis_kind_order(*rhs_kind); + if (lhs_order != rhs_order) { + return lhs_order < rhs_order; + } + return lhs.kind < rhs.kind; + }); + return canonical; +} + +const ParallelAxisSpec *find_axis_spec_by_kind( + const TensorParallelismSpec ¶llelism, LayoutAxisKind kind) { + for (const auto &axis : parallelism.axes) { + auto axis_kind = parse_layout_axis_kind(axis.kind); + if (axis_kind == std::optional(kind)) { + return &axis; + } + } + return nullptr; +} + +bool parallelism_specs_equal_by_kind(const TensorParallelismSpec &lhs, + const TensorParallelismSpec &rhs, + bool allow_tp_rank_mismatch = false) { + if (lhs.axes.size() != rhs.axes.size()) { + return false; + } + + for (const auto &lhs_axis : lhs.axes) { + auto lhs_kind = parse_layout_axis_kind(lhs_axis.kind); + if (!lhs_kind.has_value()) { + return false; + } + const auto *rhs_axis = find_axis_spec_by_kind(rhs, *lhs_kind); + if (!rhs_axis) { + return false; + } + if (*lhs_kind == LayoutAxisKind::TP && allow_tp_rank_mismatch) { + if (lhs_axis.size != rhs_axis->size || + lhs_axis.split_dim != rhs_axis->split_dim || + lhs_axis.expert_id != rhs_axis->expert_id || + lhs_axis.stage_id != rhs_axis->stage_id) { + return false; + } + continue; + } + if (!axis_specs_equal(lhs_axis, *rhs_axis)) { + return false; + } + } + return true; +} + +bool is_default_replicate_config(const ReplicateConfig &config) { + return config.replica_num == 1 && !config.with_soft_pin && + !config.with_hard_pin && config.preferred_segments.empty() && + config.preferred_segment.empty() && + !config.prefer_alloc_in_same_node; +} + +std::optional parse_parallel_axis_spec( + const py::handle &obj) { + if (!py::hasattr(obj, "kind") || !py::hasattr(obj, "rank") || + !py::hasattr(obj, "size")) { + LOG(ERROR) << "ParallelAxis must provide kind, rank, and size"; + return std::nullopt; + } + + ParallelAxisSpec axis; + axis.kind = py::cast(obj.attr("kind")); + if (!parse_layout_axis_kind(axis.kind).has_value()) { + return std::nullopt; + } + axis.rank = py::cast(obj.attr("rank")); + axis.size = py::cast(obj.attr("size")); + py::object split_dim = obj.attr("split_dim"); + if (!split_dim.is_none()) axis.split_dim = py::cast(split_dim); + py::object expert_id = obj.attr("expert_id"); + if (!expert_id.is_none()) axis.expert_id = py::cast(expert_id); + py::object stage_id = obj.attr("stage_id"); + if (!stage_id.is_none()) axis.stage_id = py::cast(stage_id); + return axis; +} + +std::optional parse_tensor_parallelism_spec( + const py::object &obj) { + if (obj.is_none()) { + return std::nullopt; + } + if (!py::hasattr(obj, "axes")) { + LOG(ERROR) << "TensorParallelism must provide axes"; + return std::nullopt; + } + + TensorParallelismSpec parallelism; + py::list axes = py::cast(obj.attr("axes")); + for (const auto &axis_obj : axes) { + auto axis = parse_parallel_axis_spec(axis_obj); + if (!axis.has_value()) { + return std::nullopt; + } + parallelism.axes.push_back(*axis); + } + return parallelism; +} + +std::optional parse_read_target_mode(const py::object &obj) { + std::string mode = py::cast(obj); + std::transform(mode.begin(), mode.end(), mode.begin(), + [](unsigned char c) { return std::toupper(c); }); + if (mode == "AS_STORED") return ReadTargetMode::AS_STORED; + if (mode == "SHARD") return ReadTargetMode::SHARD; + if (mode == "FULL") return ReadTargetMode::FULL; + LOG(ERROR) << "Unsupported ReadTarget mode: " << mode; + return std::nullopt; +} + +std::optional parse_writer_partition_spec( + const py::object &obj, const std::string &error_context) { + if (obj.is_none()) { + return std::nullopt; + } + if (!py::hasattr(obj, "rank") || !py::hasattr(obj, "size") || + !py::hasattr(obj, "split_dim")) { + LOG(ERROR) + << error_context + << ": writer_partition must provide rank, size, and split_dim"; + return std::nullopt; + } + + WriterPartitionSpec writer; + writer.rank = py::cast(obj.attr("rank")); + writer.size = py::cast(obj.attr("size")); + writer.split_dim = py::cast(obj.attr("split_dim")); + if (writer.size <= 0 || writer.rank < 0 || writer.rank >= writer.size || + writer.split_dim < 0) { + LOG(ERROR) << error_context << ": invalid writer partition"; + return std::nullopt; + } + return writer; +} + +bool is_valid_writer_partition(const WriterPartitionSpec &writer, + const std::vector &shape, + const std::string &error_context) { + if (writer.size <= 0 || writer.rank < 0 || writer.rank >= writer.size) { + LOG(ERROR) << error_context << ": invalid writer rank/size"; + return false; + } + if (writer.split_dim < 0 || + writer.split_dim >= static_cast(shape.size())) { + LOG(ERROR) << error_context << ": split_dim out of range"; + return false; + } + if (!validate_uniform_shard_request(shape, writer.split_dim, writer.size, + error_context)) { + return false; + } + return true; +} + +std::optional build_writer_shard_tensor_info( + const std::string &key, const py::object &tensor, + const WriterPartitionSpec &writer, const std::string &error_context) { + auto shape = tensor_shape_to_vector(tensor); + if (!is_valid_writer_partition(writer, shape, error_context)) { + return std::nullopt; + } + + TensorDtype dtype_enum = get_tensor_dtype(tensor.attr("dtype")); + if (dtype_enum == TensorDtype::UNKNOWN) { + LOG(ERROR) << error_context << ": unsupported tensor dtype"; + return std::nullopt; + } + + auto shard_tensor = materialize_shard_tensor(tensor, writer.split_dim, + writer.rank, writer.size); + if (!shard_tensor.has_value()) { + LOG(ERROR) << error_context << ": failed to materialize writer shard"; + return std::nullopt; + } + + WriterShardTensorInfo writer_info; + writer_info.shard_key = get_writer_shard_key_name(key, writer); + writer_info.info = + extract_tensor_info(*shard_tensor, writer_info.shard_key); + writer_info.info.metadata = build_writer_shard_tensor_metadata( + tensor, *shard_tensor, dtype_enum, writer, + writer_info.info.tensor_size); + if (!writer_info.info.valid()) { + LOG(ERROR) << error_context << ": invalid writer shard tensor info"; + return std::nullopt; + } + writer_info.manifest = + build_writer_shard_manifest(tensor, dtype_enum, writer); + return writer_info; +} + +std::optional parse_read_target_spec(const py::object &obj) { + if (obj.is_none()) { + return ReadTargetSpec{}; + } + if (!py::hasattr(obj, "mode") || !py::hasattr(obj, "parallelism")) { + LOG(ERROR) << "ReadTarget must provide mode and parallelism"; + return std::nullopt; + } + + ReadTargetSpec target; + auto mode = parse_read_target_mode(obj.attr("mode")); + if (!mode.has_value()) { + return std::nullopt; + } + target.mode = *mode; + auto parallelism = parse_tensor_parallelism_spec(obj.attr("parallelism")); + if (obj.attr("parallelism").is_none()) { + target.parallelism = std::nullopt; + } else if (!parallelism.has_value()) { + return std::nullopt; + } else { + target.parallelism = *parallelism; + } + return target; +} + +std::optional probe_parallelism_shard_read( + const std::function( + const std::string &, std::shared_ptr *)> &load_metadata, + const std::string &key, const TensorParallelismSpec ¶llelism, + ParallelismShardReadStorageRoute route) { + switch (route) { + case ParallelismShardReadStorageRoute::PARALLELISM_SHARD_KEY: { + std::string read_key = get_parallelism_key_name(key, parallelism); + std::shared_ptr buffer_handle; + auto metadata = load_metadata(read_key, &buffer_handle); + if (!metadata.has_value() || !parallelism_matches_metadata( + parallelism, metadata->metadata)) { + return std::nullopt; + } + return ResolvedTensorRead{read_key, *metadata, + std::move(buffer_handle)}; + } + case ParallelismShardReadStorageRoute:: + WRITER_PARTITION_COMPATIBLE_SHARD: { + const auto &axis = parallelism.axes[0]; + WriterPartitionSpec writer{ + .rank = axis.rank, + .size = axis.size, + .split_dim = axis.split_dim.value_or(0), + }; + std::string read_key = get_writer_shard_key_name(key, writer); + std::shared_ptr buffer_handle; + auto metadata = load_metadata(read_key, &buffer_handle); + if (!metadata.has_value()) { + return std::nullopt; + } + + auto writer_parallelism = + writer_partition_parallelism_from_metadata(metadata->metadata); + if (!writer_parallelism.has_value() || + !parallelism_matches_metadata(parallelism, + metadata->metadata)) { + return std::nullopt; + } + return ResolvedTensorRead{read_key, *metadata, + std::move(buffer_handle)}; + } + } + return std::nullopt; +} + +std::optional resolve_parallelism_shard_read( + const std::function( + const std::string &, std::shared_ptr *)> &load_metadata, + const std::string &key, const TensorParallelismSpec ¶llelism) { + for (auto route : resolve_parallelism_shard_read_routes(parallelism)) { + auto resolved = probe_parallelism_shard_read(load_metadata, key, + parallelism, route); + if (resolved.has_value()) { + return resolved; + } + } + return std::nullopt; +} + +std::optional validate_parallelism_spec( + const std::optional ¶llelism, + const std::string &error_context, bool allow_empty = false) { + if (!parallelism.has_value()) { + if (allow_empty) { + return parallelism; + } + LOG(ERROR) << error_context << ": parallelism is required"; + return std::nullopt; + } + + if (parallelism->axes.empty()) { + if (allow_empty) { + return parallelism; + } + LOG(ERROR) << error_context << ": parallelism axes cannot be empty"; + return std::nullopt; + } + if (parallelism->axes.size() > kMaxLayoutAxes) { + LOG(ERROR) << error_context + << ": axis count exceeds max supported axes"; + return std::nullopt; + } + + std::unordered_set seen_kinds; + for (const auto &axis : parallelism->axes) { + auto kind = parse_layout_axis_kind(axis.kind); + if (!kind.has_value()) { + LOG(ERROR) << error_context << ": unsupported axis kind " + << axis.kind; + return std::nullopt; + } + if (axis.size <= 0 || axis.rank < 0 || axis.rank >= axis.size) { + LOG(ERROR) << error_context << ": invalid rank/size for axis " + << axis.kind; + return std::nullopt; + } + if (!seen_kinds.insert(static_cast(*kind)).second) { + LOG(ERROR) << error_context + << ": duplicate axis kinds are not supported"; + return std::nullopt; + } + + switch (*kind) { + case LayoutAxisKind::TP: + if (!axis.split_dim.has_value()) { + LOG(ERROR) + << error_context << ": TP axis requires split_dim"; + return std::nullopt; + } + break; + case LayoutAxisKind::DP: + if (axis.split_dim.has_value()) { + LOG(ERROR) << error_context + << ": DP axis must not provide split_dim"; + return std::nullopt; + } + break; + case LayoutAxisKind::EP: + if (!axis.expert_id.has_value()) { + LOG(ERROR) + << error_context << ": EP axis requires expert_id"; + return std::nullopt; + } + if (axis.split_dim.has_value()) { + LOG(ERROR) + << error_context + << ": EP axis must not provide split_dim in this phase"; + return std::nullopt; + } + break; + case LayoutAxisKind::PP: + if (!axis.stage_id.has_value()) { + LOG(ERROR) + << error_context << ": PP axis requires stage_id"; + return std::nullopt; + } + if (axis.split_dim.has_value()) { + LOG(ERROR) << error_context + << ": PP axis must not provide split_dim"; + return std::nullopt; + } + break; + default: + LOG(ERROR) << error_context << ": unsupported axis kind"; + return std::nullopt; + } + } + + return parallelism; +} + +std::string normalize_axis_kind_string(LayoutAxisKind kind) { + switch (kind) { + case LayoutAxisKind::TP: + return "tp"; + case LayoutAxisKind::DP: + return "dp"; + case LayoutAxisKind::EP: + return "ep"; + case LayoutAxisKind::PP: + return "pp"; + default: + return "unknown"; + } +} + +std::string encode_axis_key_suffix(const ParallelAxisSpec &axis) { + auto kind = parse_layout_axis_kind(axis.kind); + if (!kind.has_value()) { + return "invalid"; + } + + std::string suffix = normalize_axis_kind_string(*kind) + "_" + + std::to_string(axis.rank) + "of" + + std::to_string(axis.size); + if (axis.split_dim.has_value()) { + suffix += "_sd" + std::to_string(axis.split_dim.value()); + } + if (axis.expert_id.has_value()) { + suffix += "_eid" + std::to_string(axis.expert_id.value()); + } + if (axis.stage_id.has_value()) { + suffix += "_sid" + std::to_string(axis.stage_id.value()); + } + return suffix; +} + +std::string get_parallelism_key_name(const std::string &base_key, + const TensorParallelismSpec ¶llelism) { + if (uses_legacy_tp_storage_key(parallelism)) { + return base_key + "_tp_" + std::to_string(parallelism.axes[0].rank); + } + + auto canonical = canonicalize_parallelism_spec(parallelism); + if (!canonical.has_value()) { + return base_key; + } + + std::string key = base_key; + for (const auto &axis : canonical->axes) { + key += "__" + encode_axis_key_suffix(axis); + } + return key; +} + +std::optional parallelism_from_metadata( + const TensorMetadata &metadata) { + if (!is_shard_tensor_metadata(metadata)) { + return std::nullopt; + } + + TensorParallelismSpec parallelism; + parallelism.axes.reserve(metadata.layout.axis_count); + for (size_t i = 0; i < metadata.layout.axis_count; ++i) { + const auto &stored_axis = metadata.layout.axes[i]; + auto kind = static_cast(stored_axis.kind); + ParallelAxisSpec axis; + axis.kind = normalize_axis_kind_string(kind); + axis.rank = stored_axis.shard_rank; + axis.size = stored_axis.shard_count; + if (stored_axis.split_dim >= 0) { + axis.split_dim = stored_axis.split_dim; + } + if (kind == LayoutAxisKind::EP) { + axis.expert_id = stored_axis.reserved0; + } + if (kind == LayoutAxisKind::PP) { + axis.stage_id = stored_axis.reserved0; + } + parallelism.axes.push_back(axis); + } + return canonicalize_parallelism_spec(parallelism); +} + +bool parallelism_matches_metadata(const TensorParallelismSpec ¶llelism, + const TensorMetadata &metadata) { + auto requested_parallelism = canonicalize_parallelism_spec(parallelism); + auto stored_parallelism = parallelism_from_metadata(metadata); + if (!requested_parallelism.has_value() || !stored_parallelism.has_value() || + stored_parallelism->axes.size() != requested_parallelism->axes.size()) { + return false; + } + + for (size_t i = 0; i < requested_parallelism->axes.size(); ++i) { + if (!axis_specs_equal(requested_parallelism->axes[i], + stored_parallelism->axes[i])) { + return false; + } + } + return true; +} + +std::optional +resolve_tp_compatible_parallelism_from_metadata( + const TensorParallelismSpec ¶llelism, const TensorMetadata &metadata, + const std::string &error_context) { + auto stored_parallelism = parallelism_from_metadata(metadata); + if (!stored_parallelism.has_value()) { + LOG(ERROR) << error_context << ": missing shard parallelism metadata"; + return std::nullopt; + } + if (stored_parallelism->axes.size() != parallelism.axes.size()) { + LOG(ERROR) << error_context << ": axis count mismatch"; + return std::nullopt; + } + + auto request_tp_axis_index = find_tp_axis_index(parallelism.axes); + auto stored_tp_axis_index = find_tp_axis_index(stored_parallelism->axes); + if (!request_tp_axis_index.has_value() || + !stored_tp_axis_index.has_value()) { + LOG(ERROR) << error_context << ": reconstruction requires a TP axis"; + return std::nullopt; + } + if (*request_tp_axis_index != *stored_tp_axis_index) { + LOG(ERROR) << error_context << ": TP axis position mismatch"; + return std::nullopt; + } + + for (size_t i = 0; i < parallelism.axes.size(); ++i) { + if (i == *request_tp_axis_index) { + const auto request_kind = + parse_layout_axis_kind(parallelism.axes[i].kind); + const auto stored_kind = + parse_layout_axis_kind(stored_parallelism->axes[i].kind); + if (!request_kind.has_value() || !stored_kind.has_value() || + request_kind != stored_kind || + parallelism.axes[i].split_dim != + stored_parallelism->axes[i].split_dim || + parallelism.axes[i].expert_id != + stored_parallelism->axes[i].expert_id || + parallelism.axes[i].stage_id != + stored_parallelism->axes[i].stage_id) { + LOG(ERROR) << error_context << ": TP axis metadata mismatch"; + return std::nullopt; + } + continue; + } + if (!axis_specs_equal(parallelism.axes[i], + stored_parallelism->axes[i])) { + LOG(ERROR) << error_context << ": non-TP axis metadata mismatch"; + return std::nullopt; + } + } + return stored_parallelism; +} + +std::pair calculate_shard_range(int64_t dim_size, int rank, + int shard_count) { + if (dim_size < 0 || shard_count <= 0 || rank < 0 || rank >= shard_count) { + return {0, 0}; + } + + const int64_t start = (dim_size * static_cast(rank)) / shard_count; + const int64_t end = + (dim_size * static_cast(rank + 1)) / shard_count; + return {start, end - start}; +} + +std::optional parse_tensor_metadata_from_buffer( + BufferHandle *buffer_handle, char *usr_buffer, int64_t data_length, + bool *take_ownership, char **exported_data, size_t *total_length) { + if (!buffer_handle && !usr_buffer) return std::nullopt; + if (buffer_handle && usr_buffer) return std::nullopt; + + *take_ownership = !!buffer_handle; + if (*take_ownership) { + *total_length = buffer_handle->size(); + if (*total_length < sizeof(TensorMetadata)) { + LOG(ERROR) << "Invalid data format: insufficient data for metadata"; + return std::nullopt; + } + + *exported_data = new char[*total_length]; + if (!*exported_data) return std::nullopt; + memcpy(*exported_data, buffer_handle->ptr(), *total_length); + } else { + *exported_data = usr_buffer; + if (data_length < 0) { + LOG(ERROR) << "Get tensor into failed with error code: " + << data_length; + return std::nullopt; + } + *total_length = static_cast(data_length); + if (*total_length < sizeof(TensorMetadata)) { + LOG(ERROR) << "Invalid data format: insufficient data for metadata"; + return std::nullopt; + } + } + + auto parsed = ParseTensorMetadata(*exported_data, *total_length); + if (!parsed.has_value()) { + if (*take_ownership) { + delete[] *exported_data; + *exported_data = nullptr; + } + LOG(ERROR) << "Invalid tensor metadata"; + return std::nullopt; + } + + return parsed; +} + +std::optional parse_tensor_metadata_from_raw_buffer( + uintptr_t buffer_ptr, size_t size, const char *operation_name) { + if (buffer_ptr == 0) { + LOG(ERROR) << operation_name << ": buffer pointer cannot be null"; + return std::nullopt; + } + auto parsed = + ParseTensorMetadata(reinterpret_cast(buffer_ptr), size); + if (!parsed.has_value()) { + LOG(ERROR) << operation_name << ": invalid tensor object metadata"; + return std::nullopt; + } + return parsed; +} + +std::optional build_raw_tensor_shard_write_plan( + const ParsedTensorMetadata &parsed, int split_dim, int rank, + int shard_count, const char *operation_name) { + const auto global_shape = TensorShapeToVector( + parsed.metadata.layout.global_shape, parsed.metadata.header.ndim); + const auto local_shape = TensorShapeToVector( + parsed.metadata.layout.local_shape, parsed.metadata.header.ndim); + if (global_shape.size() != local_shape.size() || rank < 0 || + rank >= shard_count || + !validate_uniform_shard_request(global_shape, split_dim, shard_count, + operation_name)) { + return std::nullopt; + } + + int64_t global_numel = 1; + int64_t local_numel = 1; + for (size_t dim = 0; dim < global_shape.size(); ++dim) { + if (global_shape[dim] <= 0 || local_shape[dim] < 0) { + LOG(ERROR) << operation_name << ": invalid tensor shape"; + return std::nullopt; + } + global_numel *= global_shape[dim]; + local_numel *= local_shape[dim]; + if (static_cast(dim) != split_dim && + global_shape[dim] != local_shape[dim]) { + LOG(ERROR) << operation_name + << ": unsupported non-split local shape"; + return std::nullopt; + } + } + if (local_numel < 0 || global_numel <= 0) { + LOG(ERROR) << operation_name << ": invalid tensor numel"; + return std::nullopt; + } + if ((local_numel == 0 && parsed.data_bytes != 0) || + (local_numel > 0 && + parsed.data_bytes % static_cast(local_numel) != 0)) { + LOG(ERROR) << operation_name << ": invalid tensor byte size"; + return std::nullopt; + } + + const size_t element_size = + local_numel == 0 ? 0 + : parsed.data_bytes / static_cast(local_numel); + const auto [shard_start, shard_extent] = + calculate_shard_range(global_shape[split_dim], rank, shard_count); + + std::vector shard_shape = global_shape; + shard_shape[split_dim] = shard_extent; + int64_t shard_numel = 1; + for (auto dim : shard_shape) { + shard_numel *= dim; + } + const size_t shard_bytes = static_cast(shard_numel) * element_size; + + TensorMetadata metadata = + BuildTensorMetadata(parsed.metadata.header.dtype, global_shape, + shard_shape, TensorLayoutKind::SHARD); + metadata.header.data_bytes = shard_bytes; + + RawTensorShardWritePlan plan; + plan.metadata = metadata; + if (shard_extent == 0 || element_size == 0) { + return plan; + } + + int64_t elements_before = 1; + for (int i = 0; i < split_dim; ++i) { + elements_before *= global_shape[i]; + } + int64_t elements_after = 1; + for (size_t i = split_dim + 1; i < global_shape.size(); ++i) { + elements_after *= global_shape[i]; + } + + const size_t row_bytes = static_cast(shard_extent) * + static_cast(elements_after) * element_size; + plan.data_ranges.reserve(static_cast(elements_before)); + for (int64_t slice_idx = 0; slice_idx < elements_before; ++slice_idx) { + const size_t src_offset = + parsed.data_offset + + static_cast(slice_idx * global_shape[split_dim] + + shard_start) * + static_cast(elements_after) * element_size; + plan.data_ranges.emplace_back(src_offset, row_bytes); + } + return plan; +} diff --git a/mooncake-integration/store/store_py_parallel_read.h b/mooncake-integration/store/store_py_parallel_read.h new file mode 100644 index 00000000..601e89e9 --- /dev/null +++ b/mooncake-integration/store/store_py_parallel_read.h @@ -0,0 +1,1312 @@ +std::optional get_tensor_metadata( + const std::string &key, + std::shared_ptr *buffer_handle_out = nullptr) { + if (!is_client_initialized()) { + LOG(ERROR) << "Client not initialized"; + return std::nullopt; + } + + std::shared_ptr buffer_handle; + { + py::gil_scoped_release release_gil; + buffer_handle = store_->get_buffer(key); + } + if (!buffer_handle) { + return std::nullopt; + } + + if (buffer_handle_out) { + *buffer_handle_out = buffer_handle; + } + return ParseTensorMetadata(static_cast(buffer_handle->ptr()), + buffer_handle->size()); +} + +std::optional build_tensor_into_plan( + const std::string &read_key, uintptr_t buffer_ptr, size_t size, + const std::string &context, + const std::optional &metadata = std::nullopt) { + std::optional resolved_metadata = metadata; + if (!resolved_metadata.has_value()) { + resolved_metadata = get_tensor_metadata(read_key); + } + if (!resolved_metadata.has_value()) { + return std::nullopt; + } + + const auto total_length = + resolved_metadata->data_offset + resolved_metadata->data_bytes; + if (total_length > size) { + LOG(ERROR) << context << ": buffer too small for key " << read_key; + return std::nullopt; + } + + auto region = resolve_registered_buffer_region(buffer_ptr, size, context); + if (!region.has_value()) { + return std::nullopt; + } + if (region->offset + total_length > region->size) { + LOG(ERROR) << context + << ": resolved destination range exceeds registered region"; + return std::nullopt; + } + + TensorIntoPlan plan; + plan.user_buffer_ptr = buffer_ptr; + plan.registered_buffer_ptr = reinterpret_cast(region->base); + plan.registered_buffer_size = region->size; + plan.total_length = total_length; + plan.fragments.push_back(TensorIntoFragment{ + .read_key = read_key, + .dst_offset = region->offset, + .src_offset = 0, + .size = total_length, + }); + return plan; +} + +std::optional extract_reconstruction_element_size( + const std::vector &sources, + const std::string &context) { + for (const auto &source : sources) { + const auto local_shape = + TensorShapeToVector(source.metadata.metadata.layout.local_shape, + source.metadata.metadata.header.ndim); + int64_t shard_numel = 1; + for (auto dim : local_shape) { + shard_numel *= dim; + } + if (shard_numel < 0) { + LOG(ERROR) << context << ": invalid shard tensor numel"; + return std::nullopt; + } + if (shard_numel == 0) { + if (source.metadata.data_bytes != 0) { + LOG(ERROR) << context << ": invalid empty shard byte size"; + return std::nullopt; + } + continue; + } + if (source.metadata.data_bytes % static_cast(shard_numel) != + 0) { + LOG(ERROR) << context << ": invalid shard tensor byte size"; + return std::nullopt; + } + return source.metadata.data_bytes / static_cast(shard_numel); + } + return size_t{0}; +} + +std::optional> get_source_shard_range( + const ReconstructedShardSource &source, + const std::vector &global_shape, int split_dim, + const std::string &context) { + const auto local_shape = + TensorShapeToVector(source.metadata.metadata.layout.local_shape, + source.metadata.metadata.header.ndim); + if (local_shape.size() != global_shape.size()) { + LOG(ERROR) << context << ": invalid shard shape for key " + << source.read_key; + return std::nullopt; + } + for (size_t dim = 0; dim < global_shape.size(); ++dim) { + if (static_cast(dim) == split_dim) { + continue; + } + if (local_shape[dim] != global_shape[dim]) { + LOG(ERROR) << context << ": shard shape mismatch for key " + << source.read_key; + return std::nullopt; + } + } + + const LayoutAxis *tp_axis = + find_layout_axis(source.metadata.metadata, LayoutAxisKind::TP); + if (!is_shard_tensor_metadata(source.metadata.metadata) || !tp_axis) { + LOG(ERROR) << context << ": missing TP shard metadata for key " + << source.read_key; + return std::nullopt; + } + if (tp_axis->split_dim != split_dim || tp_axis->shard_count <= 0 || + tp_axis->shard_rank < 0 || + tp_axis->shard_rank >= tp_axis->shard_count) { + LOG(ERROR) << context << ": invalid TP shard metadata for key " + << source.read_key; + return std::nullopt; + } + if (!is_uniform_shardable_dim(global_shape[split_dim], + tp_axis->shard_count)) { + LOG(ERROR) << context << ": only uniform sharding is supported"; + return std::nullopt; + } + + const auto [shard_start, shard_extent] = calculate_shard_range( + global_shape[split_dim], tp_axis->shard_rank, tp_axis->shard_count); + if (local_shape[split_dim] != shard_extent) { + LOG(ERROR) << context << ": shard extent mismatch for key " + << source.read_key; + return std::nullopt; + } + return std::pair{shard_start, shard_extent}; +} + +std::optional build_reconstructed_tensor_into_plan_from_sources( + uintptr_t buffer_ptr, size_t size, + const std::vector &sources, + const std::vector &global_shape, int split_dim, + const TensorMetadata &target_metadata, int64_t target_start, + int64_t target_extent, const std::string &context, + bool allow_empty_fragments = false) { + if (sources.empty()) { + LOG(ERROR) << context << ": missing reconstruction shard sources"; + return std::nullopt; + } + if (split_dim < 0 || split_dim >= static_cast(global_shape.size())) { + LOG(ERROR) << context << ": invalid split_dim"; + return std::nullopt; + } + if (target_start < 0 || target_extent < 0 || + target_start + target_extent > global_shape[split_dim]) { + LOG(ERROR) << context << ": invalid target shard range"; + return std::nullopt; + } + + auto region = resolve_registered_buffer_region(buffer_ptr, size, context); + if (!region.has_value()) { + return std::nullopt; + } + + auto element_size = extract_reconstruction_element_size(sources, context); + if (!element_size.has_value()) { + return std::nullopt; + } + + size_t target_tensor_numel = 1; + for (size_t dim = 0; dim < global_shape.size(); ++dim) { + const int64_t dim_extent = static_cast(dim) == split_dim + ? target_extent + : global_shape[dim]; + target_tensor_numel *= static_cast(dim_extent); + } + const size_t target_tensor_bytes = target_tensor_numel * *element_size; + const size_t total_length = sizeof(TensorMetadata) + target_tensor_bytes; + if (total_length > size || region->offset + total_length > region->size) { + LOG(ERROR) << context << ": buffer too small for reconstructed tensor"; + return std::nullopt; + } + + TensorMetadata materialized_metadata = target_metadata; + materialized_metadata.header.data_bytes = target_tensor_bytes; + + TensorIntoPlan plan; + plan.user_buffer_ptr = buffer_ptr; + plan.registered_buffer_ptr = reinterpret_cast(region->base); + plan.registered_buffer_size = region->size; + plan.total_length = total_length; + plan.materialized_metadata = materialized_metadata; + + int64_t elements_before = 1; + for (int i = 0; i < split_dim; ++i) { + elements_before *= global_shape[i]; + } + int64_t elements_after = 1; + for (size_t i = split_dim + 1; i < global_shape.size(); ++i) { + elements_after *= global_shape[i]; + } + + std::vector covered( + static_cast(target_extent > 0 ? target_extent : 0), false); + for (const auto &source : sources) { + auto source_range = + get_source_shard_range(source, global_shape, split_dim, context); + if (!source_range.has_value()) { + return std::nullopt; + } + const auto [source_start, source_extent] = *source_range; + const int64_t overlap_start = std::max(source_start, target_start); + const int64_t overlap_end = std::min(source_start + source_extent, + target_start + target_extent); + if (overlap_end <= overlap_start) { + continue; + } + const int64_t overlap_extent = overlap_end - overlap_start; + const int64_t src_inner_offset = overlap_start - source_start; + const int64_t dst_inner_offset = overlap_start - target_start; + const size_t row_bytes = static_cast(overlap_extent) * + static_cast(elements_after) * + *element_size; + for (int64_t idx = dst_inner_offset; + idx < dst_inner_offset + overlap_extent; ++idx) { + covered[static_cast(idx)] = true; + } + + for (int64_t slice_idx = 0; slice_idx < elements_before; ++slice_idx) { + const size_t dst_offset = + region->offset + sizeof(TensorMetadata) + + static_cast(slice_idx * target_extent + + dst_inner_offset) * + static_cast(elements_after) * *element_size; + const size_t src_offset = + source.metadata.data_offset + + static_cast(slice_idx * source_extent + + src_inner_offset) * + static_cast(elements_after) * *element_size; + plan.fragments.push_back(TensorIntoFragment{ + .read_key = source.read_key, + .dst_offset = dst_offset, + .src_offset = src_offset, + .size = row_bytes, + }); + } + } + + for (bool is_covered : covered) { + if (!is_covered) { + LOG(ERROR) + << context + << ": shard extents do not cover reconstructed dimension"; + return std::nullopt; + } + } + if (plan.fragments.empty() && + !(allow_empty_fragments && target_tensor_bytes == 0)) { + LOG(ERROR) << context << ": no fragments planned for reconstruction"; + return std::nullopt; + } + return plan; +} + +std::optional build_full_tensor_into_plan_from_sources( + uintptr_t buffer_ptr, size_t size, + const std::vector &sources, + const std::vector &global_shape, int split_dim, int32_t dtype, + const std::string &context, bool allow_empty_fragments = false) { + TensorMetadata full_metadata = BuildTensorMetadata( + dtype, global_shape, global_shape, TensorLayoutKind::FULL); + return build_reconstructed_tensor_into_plan_from_sources( + buffer_ptr, size, sources, global_shape, split_dim, full_metadata, 0, + global_shape[split_dim], context, allow_empty_fragments); +} + +std::string resolve_tp_read_key(const std::string &key, int tp_rank, + int tp_size) const { + if (tp_size <= 1) return key; + return get_tp_key_name(key, tp_rank); +} + +pybind11::object get_tensor_with_writer_shard_full(const std::string &key, + const std::string &context) { + if (!is_client_initialized()) { + LOG(ERROR) << "Client is not initialized"; + return pybind11::none(); + } + if (use_dummy_client_) { + LOG(ERROR) << context << ": dummy client is not supported"; + return pybind11::none(); + } + + auto reconstruction = load_writer_shard_reconstruction(key, context); + if (!reconstruction.has_value()) { + return get_tensor(key); + } + + size_t total_tensor_numel = 1; + for (auto dim : reconstruction->global_shape) { + total_tensor_numel *= static_cast(dim); + } + + size_t element_size = 0; + for (const auto &source : reconstruction->sources) { + if (source.metadata.data_bytes == 0) { + continue; + } + int64_t shard_numel = 1; + const auto local_shape = + TensorShapeToVector(source.metadata.metadata.layout.local_shape, + source.metadata.metadata.header.ndim); + for (auto dim : local_shape) { + shard_numel *= dim; + } + if (shard_numel <= 0 || + source.metadata.data_bytes % static_cast(shard_numel) != + 0) { + LOG(ERROR) << context << ": invalid writer shard tensor byte size"; + return py::none(); + } + element_size = + source.metadata.data_bytes / static_cast(shard_numel); + break; + } + + const size_t total_length = + sizeof(TensorMetadata) + total_tensor_numel * element_size; + char *owned_buffer = new char[total_length]; + if (store_->register_buffer(owned_buffer, total_length) != 0) { + LOG(ERROR) << context << ": failed to register reconstruction buffer"; + delete[] owned_buffer; + return py::none(); + } + + auto plan = build_full_tensor_into_plan_from_sources( + reinterpret_cast(owned_buffer), total_length, + reconstruction->sources, reconstruction->global_shape, + reconstruction->split_dim, reconstruction->dtype, context, + reconstruction->allow_empty_fragments); + if (!plan.has_value()) { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + return py::none(); + } + + auto success = execute_tensor_into_plan_transfers({*plan}); + if (success.empty() || !success[0]) { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + return py::none(); + } + + return buffer_to_tensor( + new BufferHandle(owned_buffer, total_length, + [this, owned_buffer]() { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + }), + nullptr, 0); +} + +pybind11::object get_tensor_with_tp_full( + const std::string &key, int tp_rank, int tp_size, int split_dim, + const std::string &context, + const std::optional ¶llelism = std::nullopt) { + if (!is_client_initialized()) { + LOG(ERROR) << "Client is not initialized"; + return pybind11::none(); + } + if (use_dummy_client_) { + LOG(ERROR) << context << ": dummy client is not supported"; + return pybind11::none(); + } + ParallelAxisSpec axis{ + .kind = "tp", + .rank = tp_rank, + .size = tp_size, + .split_dim = split_dim, + .expert_id = std::nullopt, + .stage_id = std::nullopt, + }; + + auto reconstruction = + parallelism.has_value() + ? load_parallelism_full_reconstruction_sources(key, *parallelism, + context) + : load_tp_full_reconstruction_sources(key, axis, context); + if (!reconstruction.has_value()) { + return pybind11::none(); + } + + const size_t total_tensor_numel = + std::accumulate(reconstruction->global_shape.begin(), + reconstruction->global_shape.end(), + static_cast(1), std::multiplies()); + size_t element_size = 0; + for (const auto &source : reconstruction->sources) { + if (source.metadata.data_bytes == 0) { + continue; + } + int64_t shard_numel = 1; + const auto local_shape = + TensorShapeToVector(source.metadata.metadata.layout.local_shape, + source.metadata.metadata.header.ndim); + for (auto dim : local_shape) { + shard_numel *= dim; + } + if (shard_numel <= 0 || + source.metadata.data_bytes % static_cast(shard_numel) != + 0) { + LOG(ERROR) << context << ": invalid shard tensor byte size"; + return pybind11::none(); + } + element_size = + source.metadata.data_bytes / static_cast(shard_numel); + break; + } + const size_t total_length = + sizeof(TensorMetadata) + total_tensor_numel * element_size; + + char *owned_buffer = new char[total_length]; + if (store_->register_buffer(owned_buffer, total_length) != 0) { + LOG(ERROR) << context << ": failed to register reconstruction buffer"; + delete[] owned_buffer; + return pybind11::none(); + } + + auto plan = build_full_tensor_into_plan_from_sources( + reinterpret_cast(owned_buffer), total_length, + reconstruction->sources, reconstruction->global_shape, + reconstruction->split_dim, reconstruction->dtype, context, + reconstruction->allow_empty_fragments); + if (!plan.has_value()) { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + return pybind11::none(); + } + + auto success = execute_tensor_into_plan_transfers({*plan}); + if (success.empty() || !success[0]) { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + return py::none(); + } + + return buffer_to_tensor( + new BufferHandle(owned_buffer, total_length, + [this, owned_buffer]() { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + }), + nullptr, 0); +} + +std::optional build_tp_full_tensor_into_plan( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const ParallelAxisSpec &axis, const std::string &context, + const std::optional ¶llelism = std::nullopt) { + auto reconstruction = + load_tp_full_reconstruction_sources(key, axis, context, parallelism); + if (!reconstruction.has_value()) { + return std::nullopt; + } + return build_full_tensor_into_plan_from_sources( + buffer_ptr, size, reconstruction->sources, reconstruction->global_shape, + reconstruction->split_dim, reconstruction->dtype, context, + reconstruction->allow_empty_fragments); +} + +std::optional +load_tp_full_reconstruction_sources( + const std::string &key, const ParallelAxisSpec &axis, + const std::string &context, + const std::optional ¶llelism = std::nullopt) { + if (axis.size <= 0) { + LOG(ERROR) << context << ": tp_size must be positive"; + return std::nullopt; + } + + FullTensorReconstructionSources reconstruction; + reconstruction.sources.reserve(axis.size); + for (int shard_rank = 0; shard_rank < axis.size; ++shard_rank) { + std::string read_key; + if (parallelism.has_value()) { + auto shard_parallelism = *parallelism; + auto tp_axis_index = find_tp_axis_index(shard_parallelism.axes); + if (!tp_axis_index.has_value()) { + LOG(ERROR) << context + << ": missing TP axis in full reconstruction"; + return std::nullopt; + } + shard_parallelism.axes[*tp_axis_index].rank = shard_rank; + read_key = get_parallelism_key_name(key, shard_parallelism); + } else { + read_key = resolve_tp_read_key(key, shard_rank, axis.size); + } + auto metadata = get_tensor_metadata(read_key); + if (!metadata.has_value()) { + return std::nullopt; + } + const LayoutAxis *tp_axis = + find_layout_axis(metadata->metadata, LayoutAxisKind::TP); + if (!is_shard_tensor_metadata(metadata->metadata) || !tp_axis || + tp_axis->shard_rank != shard_rank || + tp_axis->shard_count != axis.size) { + LOG(ERROR) << context << ": TP metadata mismatch for key " + << read_key; + return std::nullopt; + } + if (parallelism.has_value()) { + auto stored_parallelism = + resolve_tp_compatible_parallelism_from_metadata( + *parallelism, metadata->metadata, context); + if (!stored_parallelism.has_value()) { + return std::nullopt; + } + } + reconstruction.sources.push_back( + ReconstructedShardSource{read_key, *metadata}); + } + + reconstruction.global_shape = TensorShapeToVector( + reconstruction.sources.front().metadata.metadata.layout.global_shape, + reconstruction.sources.front().metadata.metadata.header.ndim); + const LayoutAxis *stored_tp_axis = find_layout_axis( + reconstruction.sources.front().metadata.metadata, LayoutAxisKind::TP); + if (!stored_tp_axis) { + LOG(ERROR) << context << ": missing TP axis metadata"; + return std::nullopt; + } + reconstruction.split_dim = stored_tp_axis->split_dim; + if (!is_uniform_shardable_dim( + reconstruction.global_shape[reconstruction.split_dim], + stored_tp_axis->shard_count)) { + LOG(ERROR) << context << ": only uniform sharding is supported"; + return std::nullopt; + } + if (axis.split_dim.has_value() && + axis.split_dim.value() != reconstruction.split_dim) { + LOG(ERROR) << context << ": split_dim mismatch"; + return std::nullopt; + } + reconstruction.dtype = + reconstruction.sources.front().metadata.metadata.header.dtype; + return reconstruction; +} + +std::optional load_writer_shard_reconstruction( + const std::string &key, const std::string &context) { + if (!is_client_initialized()) { + LOG(ERROR) << context << ": client is not initialized"; + return std::nullopt; + } + + std::shared_ptr manifest_handle; + { + py::gil_scoped_release release_gil; + manifest_handle = store_->get_buffer(get_writer_manifest_key_name(key)); + } + auto parsed_manifest = parse_writer_shard_manifest(manifest_handle.get()); + if (!parsed_manifest.has_value()) { + return std::nullopt; + } + + const auto &manifest = *parsed_manifest; + const auto &global_shape = manifest.global_shape; + const int split_dim = manifest.manifest.header.split_dim; + const int shard_count = manifest.manifest.header.shard_count; + if (split_dim < 0 || split_dim >= static_cast(global_shape.size())) { + LOG(ERROR) << context << ": invalid writer split_dim"; + return std::nullopt; + } + + FullTensorReconstructionSources reconstruction; + reconstruction.sources.reserve(shard_count); + for (int shard_rank = 0; shard_rank < shard_count; ++shard_rank) { + WriterPartitionSpec writer{ + .rank = shard_rank, + .size = shard_count, + .split_dim = split_dim, + }; + const std::string shard_key = get_writer_shard_key_name(key, writer); + auto metadata = get_tensor_metadata(shard_key); + if (!metadata.has_value()) { + LOG(ERROR) << context << ": missing writer shard key " << shard_key; + return std::nullopt; + } + auto writer_parallelism = + writer_partition_parallelism_from_metadata(metadata->metadata); + if (!writer_parallelism.has_value() || + writer_parallelism->axes[0].rank != shard_rank || + writer_parallelism->axes[0].size != shard_count || + writer_parallelism->axes[0].split_dim != split_dim) { + LOG(ERROR) << context << ": writer shard metadata mismatch for key " + << shard_key; + return std::nullopt; + } + reconstruction.sources.push_back( + ReconstructedShardSource{shard_key, *metadata}); + } + reconstruction.global_shape = manifest.global_shape; + reconstruction.split_dim = manifest.manifest.header.split_dim; + reconstruction.dtype = manifest.manifest.header.dtype; + reconstruction.allow_empty_fragments = true; + return reconstruction; +} + +std::optional build_writer_shard_full_tensor_into_plan( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const std::string &context) { + auto reconstruction = load_writer_shard_reconstruction(key, context); + if (!reconstruction.has_value()) { + return build_tensor_into_plan(key, buffer_ptr, size, context); + } + + return build_full_tensor_into_plan_from_sources( + buffer_ptr, size, reconstruction->sources, reconstruction->global_shape, + reconstruction->split_dim, reconstruction->dtype, context, + reconstruction->allow_empty_fragments); +} + +std::optional +load_parallelism_manifest_reconstruction( + const std::string &key, const TensorParallelismSpec ¶llelism, + const std::string &context) { + const ParallelAxisSpec *request_tp_axis = + find_axis_spec_by_kind(parallelism, LayoutAxisKind::TP); + if (!request_tp_axis) { + LOG(ERROR) << context << ": reconstruction requires a TP axis"; + return std::nullopt; + } + + std::shared_ptr manifest_handle; + { + py::gil_scoped_release release_gil; + manifest_handle = + store_->get_buffer(get_parallelism_manifest_key_name(key)); + } + auto parsed_manifest = parse_writer_shard_manifest(manifest_handle.get()); + if (!parsed_manifest.has_value()) { + return std::nullopt; + } + + const auto &manifest = *parsed_manifest; + const auto &global_shape = manifest.global_shape; + const int split_dim = manifest.manifest.header.split_dim; + const int shard_count = manifest.manifest.header.shard_count; + if (split_dim < 0 || split_dim >= static_cast(global_shape.size())) { + LOG(ERROR) << context << ": invalid parallelism split_dim"; + return std::nullopt; + } + if (request_tp_axis->split_dim.has_value() && + request_tp_axis->split_dim.value() != split_dim) { + LOG(ERROR) << context << ": split_dim mismatch"; + return std::nullopt; + } + if (!is_uniform_shardable_dim(global_shape[split_dim], + request_tp_axis->size) || + !is_uniform_shardable_dim(global_shape[split_dim], shard_count)) { + LOG(ERROR) << context << ": only uniform sharding is supported"; + return std::nullopt; + } + auto canonical_parallelism = canonicalize_parallelism_spec(parallelism); + if (!canonical_parallelism.has_value()) { + LOG(ERROR) << context << ": failed to canonicalize parallelism"; + return std::nullopt; + } + auto tp_axis_index = find_tp_axis_index(canonical_parallelism->axes); + if (!tp_axis_index.has_value()) { + LOG(ERROR) << context << ": missing TP axis in request"; + return std::nullopt; + } + + FullTensorReconstructionSources reconstruction; + reconstruction.global_shape = global_shape; + reconstruction.split_dim = split_dim; + reconstruction.dtype = manifest.manifest.header.dtype; + reconstruction.sources.reserve(shard_count); + for (int shard_rank = 0; shard_rank < shard_count; ++shard_rank) { + auto shard_parallelism = *canonical_parallelism; + shard_parallelism.axes[*tp_axis_index].rank = shard_rank; + shard_parallelism.axes[*tp_axis_index].size = shard_count; + const std::string shard_key = + get_parallelism_key_name(key, shard_parallelism); + auto metadata = get_tensor_metadata(shard_key); + if (!metadata.has_value()) { + LOG(ERROR) << context + << ": no shard matched stored layout for TP rank " + << shard_rank; + return std::nullopt; + } + auto stored_parallelism = + resolve_tp_compatible_parallelism_from_metadata( + shard_parallelism, metadata->metadata, context); + if (!stored_parallelism.has_value() || + !parallelism_specs_equal_by_kind( + shard_parallelism, *stored_parallelism, + true /* allow_tp_rank_mismatch */)) { + LOG(ERROR) << context << ": shard metadata mismatch for TP rank " + << shard_rank; + return std::nullopt; + } + reconstruction.sources.push_back( + ReconstructedShardSource{shard_key, *metadata}); + } + return reconstruction; +} + +std::optional +load_parallelism_full_reconstruction_sources( + const std::string &key, const TensorParallelismSpec ¶llelism, + const std::string &context) { + if (auto manifest_reconstruction = + load_parallelism_manifest_reconstruction(key, parallelism, context); + manifest_reconstruction.has_value()) { + return manifest_reconstruction; + } + + const ParallelAxisSpec *request_tp_axis = + find_axis_spec_by_kind(parallelism, LayoutAxisKind::TP); + if (!request_tp_axis) { + LOG(ERROR) << context << ": full reconstruction requires a TP axis"; + return std::nullopt; + } + + auto canonical_parallelism = canonicalize_parallelism_spec(parallelism); + if (!canonical_parallelism.has_value()) { + LOG(ERROR) << context << ": failed to canonicalize parallelism"; + return std::nullopt; + } + + auto tp_axis_index = find_tp_axis_index(canonical_parallelism->axes); + if (!tp_axis_index.has_value()) { + LOG(ERROR) << context << ": missing TP axis in request"; + return std::nullopt; + } + + FullTensorReconstructionSources reconstruction; + if (uses_legacy_tp_storage_key(*canonical_parallelism)) { + const std::string first_key = + resolve_tp_read_key(key, 0, request_tp_axis->size); + auto first_metadata = get_tensor_metadata(first_key); + if (!first_metadata.has_value()) { + LOG(ERROR) << context + << ": no shard matched requested layout for TP rank 0"; + return std::nullopt; + } + const LayoutAxis *stored_tp_axis = + find_layout_axis(first_metadata->metadata, LayoutAxisKind::TP); + if (!is_shard_tensor_metadata(first_metadata->metadata) || + !stored_tp_axis) { + LOG(ERROR) << context << ": missing TP axis metadata"; + return std::nullopt; + } + reconstruction.split_dim = stored_tp_axis->split_dim; + if (request_tp_axis->split_dim.has_value() && + request_tp_axis->split_dim.value() != reconstruction.split_dim) { + LOG(ERROR) << context << ": split_dim mismatch"; + return std::nullopt; + } + reconstruction.global_shape = + TensorShapeToVector(first_metadata->metadata.layout.global_shape, + first_metadata->metadata.header.ndim); + if (!is_uniform_shardable_dim( + reconstruction.global_shape[reconstruction.split_dim], + request_tp_axis->size) || + !is_uniform_shardable_dim( + reconstruction.global_shape[reconstruction.split_dim], + stored_tp_axis->shard_count)) { + LOG(ERROR) << context << ": only uniform sharding is supported"; + return std::nullopt; + } + reconstruction.dtype = first_metadata->metadata.header.dtype; + reconstruction.sources.reserve(stored_tp_axis->shard_count); + for (int shard_rank = 0; shard_rank < stored_tp_axis->shard_count; + ++shard_rank) { + const std::string shard_key = resolve_tp_read_key( + key, shard_rank, stored_tp_axis->shard_count); + auto metadata = get_tensor_metadata(shard_key); + if (!metadata.has_value()) { + LOG(ERROR) << context + << ": no shard matched stored layout for TP rank " + << shard_rank; + return std::nullopt; + } + const LayoutAxis *source_tp_axis = + find_layout_axis(metadata->metadata, LayoutAxisKind::TP); + if (!is_shard_tensor_metadata(metadata->metadata) || + !source_tp_axis || source_tp_axis->shard_rank != shard_rank || + source_tp_axis->shard_count != stored_tp_axis->shard_count || + source_tp_axis->split_dim != reconstruction.split_dim) { + LOG(ERROR) << context + << ": shard metadata mismatch for TP rank " + << shard_rank; + return std::nullopt; + } + reconstruction.sources.push_back( + ReconstructedShardSource{shard_key, *metadata}); + } + return reconstruction; + } + + reconstruction.sources.reserve(request_tp_axis->size); + for (int shard_rank = 0; shard_rank < request_tp_axis->size; ++shard_rank) { + auto shard_parallelism = *canonical_parallelism; + shard_parallelism.axes[*tp_axis_index].rank = shard_rank; + const std::string shard_key = + get_parallelism_key_name(key, shard_parallelism); + auto metadata = get_tensor_metadata(shard_key); + if (!metadata.has_value()) { + LOG(ERROR) << context + << ": no shard matched requested layout for TP rank " + << shard_rank; + return std::nullopt; + } + + auto stored_parallelism = + resolve_tp_compatible_parallelism_from_metadata( + shard_parallelism, metadata->metadata, context); + if (!stored_parallelism.has_value() || + !parallelism_specs_equal_by_kind( + *canonical_parallelism, *stored_parallelism, + true /* allow_tp_rank_mismatch */)) { + LOG(ERROR) << context << ": shard metadata mismatch for TP rank " + << shard_rank; + return std::nullopt; + } + reconstruction.sources.push_back( + ReconstructedShardSource{shard_key, *metadata}); + } + + const LayoutAxis *stored_tp_axis = find_layout_axis( + reconstruction.sources.front().metadata.metadata, LayoutAxisKind::TP); + if (!stored_tp_axis) { + LOG(ERROR) << context << ": missing TP axis metadata"; + return std::nullopt; + } + reconstruction.split_dim = stored_tp_axis->split_dim; + if (request_tp_axis->split_dim.has_value() && + request_tp_axis->split_dim.value() != reconstruction.split_dim) { + LOG(ERROR) << context << ": split_dim mismatch"; + return std::nullopt; + } + + reconstruction.global_shape = TensorShapeToVector( + reconstruction.sources.front().metadata.metadata.layout.global_shape, + reconstruction.sources.front().metadata.metadata.header.ndim); + if (!is_uniform_shardable_dim( + reconstruction.global_shape[reconstruction.split_dim], + request_tp_axis->size)) { + LOG(ERROR) << context << ": only uniform sharding is supported"; + return std::nullopt; + } + reconstruction.dtype = + reconstruction.sources.front().metadata.metadata.header.dtype; + return reconstruction; +} + +std::optional build_parallelism_full_tensor_into_plan( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const TensorParallelismSpec ¶llelism, const std::string &context) { + auto reconstruction = + load_parallelism_full_reconstruction_sources(key, parallelism, context); + if (!reconstruction.has_value()) { + return std::nullopt; + } + return build_full_tensor_into_plan_from_sources( + buffer_ptr, size, reconstruction->sources, reconstruction->global_shape, + reconstruction->split_dim, reconstruction->dtype, context, + reconstruction->allow_empty_fragments); +} + +std::optional build_parallelism_shard_tensor_into_plan( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const TensorParallelismSpec ¶llelism, const std::string &context) { + auto reconstruction = + load_parallelism_full_reconstruction_sources(key, parallelism, context); + if (!reconstruction.has_value()) { + return std::nullopt; + } + + const ParallelAxisSpec *tp_axis = + find_axis_spec_by_kind(parallelism, LayoutAxisKind::TP); + if (!tp_axis) { + LOG(ERROR) << context << ": shard reconstruction requires a TP axis"; + return std::nullopt; + } + const auto [target_start, target_extent] = calculate_shard_range( + reconstruction->global_shape[reconstruction->split_dim], tp_axis->rank, + tp_axis->size); + std::vector target_shape = reconstruction->global_shape; + target_shape[reconstruction->split_dim] = target_extent; + auto target_metadata = build_shard_metadata_from_shapes( + reconstruction->dtype, reconstruction->global_shape, target_shape, + parallelism.axes, 0); + if (!target_metadata.has_value()) { + LOG(ERROR) << context << ": failed to build target shard metadata"; + return std::nullopt; + } + return build_reconstructed_tensor_into_plan_from_sources( + buffer_ptr, size, reconstruction->sources, reconstruction->global_shape, + reconstruction->split_dim, *target_metadata, target_start, + target_extent, context, reconstruction->allow_empty_fragments); +} + +pybind11::object get_tensor_with_parallelism_shard_full_materialized( + const std::string &key, const TensorParallelismSpec ¶llelism, + const std::string &context) { + auto reconstruction = + load_parallelism_full_reconstruction_sources(key, parallelism, context); + if (!reconstruction.has_value()) { + return py::none(); + } + + const ParallelAxisSpec *tp_axis = + find_axis_spec_by_kind(parallelism, LayoutAxisKind::TP); + if (!tp_axis) { + LOG(ERROR) << context << ": shard reconstruction requires a TP axis"; + return py::none(); + } + const auto [target_start, target_extent] = calculate_shard_range( + reconstruction->global_shape[reconstruction->split_dim], tp_axis->rank, + tp_axis->size); + + auto element_size = + extract_reconstruction_element_size(reconstruction->sources, context); + if (!element_size.has_value()) { + return py::none(); + } + + size_t target_tensor_numel = 1; + for (size_t dim = 0; dim < reconstruction->global_shape.size(); ++dim) { + const int64_t dim_extent = + static_cast(dim) == reconstruction->split_dim + ? target_extent + : reconstruction->global_shape[dim]; + target_tensor_numel *= static_cast(dim_extent); + } + const size_t total_length = + sizeof(TensorMetadata) + target_tensor_numel * *element_size; + + char *owned_buffer = new char[total_length]; + if (store_->register_buffer(owned_buffer, total_length) != 0) { + LOG(ERROR) << context << ": failed to register reconstruction buffer"; + delete[] owned_buffer; + return py::none(); + } + + auto plan = build_parallelism_shard_tensor_into_plan( + key, reinterpret_cast(owned_buffer), total_length, + parallelism, context); + if (!plan.has_value()) { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + return py::none(); + } + + auto success = execute_tensor_into_plan_transfers({*plan}); + if (success.empty() || !success[0]) { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + return py::none(); + } + + return buffer_to_tensor( + new BufferHandle(owned_buffer, total_length, + [this, owned_buffer]() { + store_->unregister_buffer(owned_buffer); + delete[] owned_buffer; + }), + nullptr, 0); +} + +std::vector execute_tensor_into_plan_transfers( + const std::vector &plans) { + std::vector success(plans.size(), false); + if (plans.empty()) { + return success; + } + + std::vector buffers; + std::vector> all_keys; + std::vector>> all_dst_offsets; + std::vector>> all_src_offsets; + std::vector>> all_sizes; + buffers.reserve(plans.size()); + all_keys.reserve(plans.size()); + all_dst_offsets.reserve(plans.size()); + all_src_offsets.reserve(plans.size()); + all_sizes.reserve(plans.size()); + + for (const auto &plan : plans) { + buffers.push_back(reinterpret_cast(plan.registered_buffer_ptr)); + + std::unordered_map key_to_index; + std::vector keys; + std::vector> dst_offsets; + std::vector> src_offsets; + std::vector> sizes; + + for (const auto &fragment : plan.fragments) { + if (fragment.read_key.empty() || fragment.size == 0) { + continue; + } + auto [it, inserted] = + key_to_index.emplace(fragment.read_key, keys.size()); + if (inserted) { + keys.push_back(fragment.read_key); + dst_offsets.push_back({}); + src_offsets.push_back({}); + sizes.push_back({}); + } + const size_t key_index = it->second; + dst_offsets[key_index].push_back(fragment.dst_offset); + src_offsets[key_index].push_back(fragment.src_offset); + sizes[key_index].push_back(fragment.size); + } + + all_keys.push_back(std::move(keys)); + all_dst_offsets.push_back(std::move(dst_offsets)); + all_src_offsets.push_back(std::move(src_offsets)); + all_sizes.push_back(std::move(sizes)); + } + + std::vector>> range_results; + { + py::gil_scoped_release release_gil; + range_results = store_->get_into_ranges( + buffers, all_keys, all_dst_offsets, all_src_offsets, all_sizes); + } + + for (size_t i = 0; i < plans.size(); ++i) { + if (i >= range_results.size() || + range_results[i].size() != all_sizes[i].size()) { + continue; + } + + success[i] = true; + for (size_t key_idx = 0; key_idx < all_sizes[i].size() && success[i]; + ++key_idx) { + if (range_results[i][key_idx].size() != + all_sizes[i][key_idx].size()) { + success[i] = false; + break; + } + for (size_t frag_idx = 0; frag_idx < all_sizes[i][key_idx].size(); + ++frag_idx) { + if (range_results[i][key_idx][frag_idx] != + static_cast(all_sizes[i][key_idx][frag_idx])) { + success[i] = false; + break; + } + } + } + if (!success[i]) { + continue; + } + if (plans[i].materialized_metadata.has_value()) { + std::memcpy(reinterpret_cast(plans[i].user_buffer_ptr), + &*plans[i].materialized_metadata, + sizeof(TensorMetadata)); + } + } + return success; +} + +py::list execute_tensor_into_plans(const std::vector &plans) { + py::list results; + for (size_t i = 0; i < plans.size(); ++i) { + results.append(py::none()); + } + auto success = execute_tensor_into_plan_transfers(plans); + for (size_t i = 0; i < plans.size(); ++i) { + if (!success[i]) { + continue; + } + results[i] = buffer_to_tensor( + NULL, reinterpret_cast(plans[i].user_buffer_ptr), + static_cast(plans[i].total_length)); + } + return results; +} + +std::optional build_tensor_into_plan_for_target( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const py::object &target, const std::string &context) { + auto parsed_target = parse_read_target_spec(target); + if (!parsed_target.has_value()) { + return std::nullopt; + } + if (parsed_target->mode == ReadTargetMode::AS_STORED && + !parsed_target->parallelism.has_value()) { + return build_tensor_into_plan(key, buffer_ptr, size, context); + } + + if (parsed_target->mode == ReadTargetMode::FULL && + !parsed_target->parallelism.has_value()) { + return build_writer_shard_full_tensor_into_plan(key, buffer_ptr, size, + context); + } + + auto parallelism = + validate_parallelism_spec(parsed_target->parallelism, context, false); + if (!parallelism.has_value()) { + return std::nullopt; + } + + if (parsed_target->mode == ReadTargetMode::SHARD) { + auto resolved = resolve_parallelism_shard_read( + [this](const std::string &read_key, + std::shared_ptr *buffer_handle_out) { + return get_tensor_metadata(read_key, buffer_handle_out); + }, + key, *parallelism); + if (resolved.has_value()) { + return build_tensor_into_plan(resolved->read_key, buffer_ptr, size, + context, resolved->metadata); + } + if (find_axis_spec_by_kind(*parallelism, LayoutAxisKind::TP)) { + return build_parallelism_shard_tensor_into_plan( + key, buffer_ptr, size, *parallelism, context); + } + LOG(ERROR) << context << ": parallelism metadata mismatch for key " + << key; + return std::nullopt; + } + + if (parsed_target->mode == ReadTargetMode::FULL) { + return build_parallelism_full_tensor_into_plan(key, buffer_ptr, size, + *parallelism, context); + } + + LOG(ERROR) << context << ": unsupported ReadTarget mode"; + return std::nullopt; +} + +pybind11::object get_tensor_with_parallelism( + const std::string &key, const py::object &target = py::none()) { + auto parsed_target = parse_read_target_spec(target); + if (!parsed_target.has_value()) { + return py::none(); + } + if (parsed_target->mode == ReadTargetMode::AS_STORED && + !parsed_target->parallelism.has_value()) { + return get_tensor(key); + } + if (parsed_target->mode == ReadTargetMode::FULL && + !parsed_target->parallelism.has_value()) { + return get_tensor_with_writer_shard_full(key, + "get_tensor_with_parallelism"); + } + + auto parallelism = validate_parallelism_spec( + parsed_target->parallelism, "get_tensor_with_parallelism", false); + if (!parallelism.has_value()) { + return py::none(); + } + + if (parsed_target->mode == ReadTargetMode::SHARD) { + auto resolved = resolve_parallelism_shard_read( + [this](const std::string &read_key, + std::shared_ptr *buffer_handle_out) { + return get_tensor_metadata(read_key, buffer_handle_out); + }, + key, *parallelism); + if (resolved.has_value()) { + return buffer_to_tensor(resolved->buffer_handle.get(), nullptr, 0); + } + if (find_axis_spec_by_kind(*parallelism, LayoutAxisKind::TP)) { + auto reconstructed = + get_tensor_with_parallelism_shard_full_materialized( + key, *parallelism, "get_tensor_with_parallelism"); + if (!reconstructed.is_none()) { + return reconstructed; + } + } + if (uses_legacy_tp_storage_key(*parallelism)) { + const auto *tp_axis = + find_axis_spec_by_kind(*parallelism, LayoutAxisKind::TP); + if (tp_axis) { + return get_tensor( + resolve_tp_read_key(key, tp_axis->rank, tp_axis->size)); + } + } + LOG(ERROR) << "get_tensor_with_parallelism" + << ": parallelism metadata mismatch for key " << key; + return py::none(); + } + + if (parsed_target->mode == ReadTargetMode::FULL) { + const auto *tp_axis = + find_axis_spec_by_kind(*parallelism, LayoutAxisKind::TP); + if (!tp_axis) { + LOG(ERROR) << "get_tensor_with_parallelism" + << ": full reconstruction requires a TP axis"; + return py::none(); + } + return get_tensor_with_tp_full( + key, tp_axis->rank, tp_axis->size, tp_axis->split_dim.value_or(0), + "get_tensor_with_parallelism", *parallelism); + } + + LOG(ERROR) << "get_tensor_with_parallelism" + << ": unsupported ReadTarget mode"; + return py::none(); +} + +pybind11::list batch_get_tensor_with_parallelism( + const std::vector &keys, + const py::object &targets = py::none()) { + if (targets.is_none()) { + return batch_get_tensor(keys); + } + + auto target_list = validate_batch_request_list( + targets, keys.size(), "batch_get_tensor_with_parallelism", "targets"); + if (!target_list.has_value()) { + py::list empty; + for (size_t i = 0; i < keys.size(); ++i) { + empty.append(py::none()); + } + return empty; + } + + py::list results; + for (size_t i = 0; i < keys.size(); ++i) { + results.append(get_tensor_with_parallelism( + keys[i], py::reinterpret_borrow((*target_list)[i]))); + } + return results; +} + +pybind11::object get_tensor_with_parallelism_into( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const py::object &target = py::none()) { + auto plan = build_tensor_into_plan_for_target( + key, buffer_ptr, size, target, "get_tensor_with_parallelism_into"); + if (!plan.has_value()) { + return py::none(); + } + auto results = execute_tensor_into_plans({*plan}); + if (results.empty()) { + return py::none(); + } + return py::reinterpret_borrow(results[0]); +} + +pybind11::list batch_get_tensor_with_parallelism_into( + const std::vector &keys, + const std::vector &buffer_ptrs, const std::vector &sizes, + const py::object &targets = py::none()) { + py::list empty; + for (size_t i = 0; i < keys.size(); ++i) { + empty.append(py::none()); + } + if (keys.size() != buffer_ptrs.size() || keys.size() != sizes.size()) { + LOG(ERROR) + << "batch_get_tensor_with_parallelism_into" + << ": keys, buffer_ptrs, and sizes must have the same length"; + return empty; + } + + std::optional target_list = std::nullopt; + if (!targets.is_none()) { + target_list = validate_batch_request_list( + targets, keys.size(), "batch_get_tensor_with_parallelism_into", + "targets"); + if (!target_list.has_value()) { + return empty; + } + } + + std::vector plans; + plans.reserve(keys.size()); + std::vector plan_indices; + plan_indices.reserve(keys.size()); + for (size_t i = 0; i < keys.size(); ++i) { + py::object target = + target_list.has_value() + ? py::reinterpret_borrow((*target_list)[i]) + : py::none(); + auto plan = build_tensor_into_plan_for_target( + keys[i], buffer_ptrs[i], sizes[i], target, + "batch_get_tensor_with_parallelism_into"); + if (!plan.has_value()) { + continue; + } + plan_indices.push_back(i); + plans.push_back(*plan); + } + + auto results = execute_tensor_into_plans(plans); + for (size_t i = 0; i < plan_indices.size() && i < results.size(); ++i) { + empty[plan_indices[i]] = results[i]; + } + return empty; +} diff --git a/mooncake-integration/store/store_py_parallel_write.h b/mooncake-integration/store/store_py_parallel_write.h new file mode 100644 index 00000000..a4cc1ec3 --- /dev/null +++ b/mooncake-integration/store/store_py_parallel_write.h @@ -0,0 +1,1443 @@ +template +int write_manifest_impl(const std::string &key, + const WriterShardManifest &manifest, + const char *operation_name, + ManifestWriteFn &&write_manifest) { + std::span bytes(reinterpret_cast(&manifest), + sizeof(WriterShardManifest)); + py::gil_scoped_release release_gil; + int ret = write_manifest(bytes); + if (ret != 0) { + LOG(ERROR) << operation_name << " manifest failed for key " << key + << " with code " << ret; + } + return ret; +} + +template +std::vector batch_write_tensor_impl(const std::vector &keys, + const std::vector &infos, + const ReplicateConfig &config, + const char *operation_name, + BatchWriteFromFn &&batch_write_from) { + std::vector results(keys.size(), 0); + + { + py::gil_scoped_release release_gil; + + std::vector valid_keys; + std::vector buffer_ptrs; + std::vector buffer_sizes; + std::vector original_indices; + std::vector> temp_allocations; + + for (size_t i = 0; i < infos.size(); ++i) { + if (!infos[i].valid()) { + results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); + continue; + } + + size_t total_size = + infos[i].metadata.header.data_offset + infos[i].tensor_size; + auto alloc_result = + store_->client_buffer_allocator_->allocate(total_size); + + if (!alloc_result) { + LOG(ERROR) << "Failed to allocate buffer for " << operation_name + << " key: " << keys[i]; + results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); + continue; + } + + char *dst = static_cast(alloc_result->ptr()); + std::memcpy(dst, &infos[i].metadata, + infos[i].metadata.header.data_offset); + std::memcpy(dst + infos[i].metadata.header.data_offset, + reinterpret_cast(infos[i].data_ptr), + infos[i].tensor_size); + + valid_keys.push_back(keys[i]); + buffer_ptrs.push_back(alloc_result->ptr()); + buffer_sizes.push_back(total_size); + original_indices.push_back(i); + temp_allocations.push_back( + std::make_unique(std::move(*alloc_result))); + } + + if (!valid_keys.empty()) { + std::vector op_results = + batch_write_from(valid_keys, buffer_ptrs, buffer_sizes); + for (size_t i = 0; i < op_results.size(); ++i) { + results[original_indices[i]] = op_results[i]; + } + } + } + + return results; +} + +bool ensure_tensor_write_supported(const char *operation_name) const { + if (!is_client_initialized() || use_dummy_client_) { + LOG(ERROR) << operation_name + << ": client not initialized or dummy client not " + "supported for tensors"; + return false; + } + return true; +} + +template +int execute_single_tensor_write(const char *operation_name, + const ReplicateConfig &config, + WriteFn &&write_fn) { + if (!ensure_tensor_write_supported(operation_name)) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int validate_result = validate_replicate_config(config); + if (validate_result) { + return validate_result; + } + return write_fn(); +} + +template +std::vector execute_batch_tensor_write( + const char *operation_name, const char *size_error_context, + const std::vector &keys, size_t value_count, + const ReplicateConfig &config, WriteFn &&write_fn) { + if (!ensure_tensor_write_supported(operation_name)) { + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + if (keys.size() != value_count || keys.empty()) { + if (!keys.empty()) { + LOG(ERROR) << size_error_context; + } + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + int validate_result = validate_replicate_config(config); + if (validate_result) { + return std::vector(keys.size(), validate_result); + } + return write_fn(); +} + +bool validate_tensor_object_buffers(const std::vector &keys, + const std::vector &buffer_ptrs, + const std::vector &sizes, + const char *size_error_context, + const char *buffer_error_context) { + if (keys.size() != buffer_ptrs.size() || keys.size() != sizes.size()) { + LOG(ERROR) << size_error_context; + return false; + } + for (size_t i = 0; i < sizes.size(); ++i) { + if (!is_valid_tensor_object_buffer(buffer_ptrs[i], sizes[i], + std::string(buffer_error_context) + + " at index " + + std::to_string(i))) { + return false; + } + } + return true; +} + +template +int execute_single_tensor_write_from(const char *operation_name, + const std::string &key, + uintptr_t buffer_ptr, size_t size, + const ReplicateConfig &config, + WriteFromFn &&write_from_fn) { + if (!ensure_tensor_write_supported(operation_name)) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + if (!is_valid_tensor_object_buffer(buffer_ptr, size, operation_name)) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int validate_result = validate_replicate_config(config); + if (validate_result) { + return validate_result; + } + return write_from_fn(key, reinterpret_cast(buffer_ptr), size, + config); +} + +template +std::vector execute_batch_tensor_write_from( + const char *operation_name, const char *size_error_context, + const std::vector &keys, + const std::vector &buffer_ptrs, const std::vector &sizes, + const ReplicateConfig &config, BatchWriteFromFn &&batch_write_from_fn) { + if (!ensure_tensor_write_supported(operation_name)) { + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + if (keys.empty()) { + return std::vector(); + } + if (!validate_tensor_object_buffers(keys, buffer_ptrs, sizes, + size_error_context, + "tensor object buffer")) { + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + int validate_result = validate_replicate_config(config); + if (validate_result) { + return std::vector(keys.size(), validate_result); + } + std::vector buffers; + buffers.reserve(buffer_ptrs.size()); + for (uintptr_t ptr : buffer_ptrs) { + buffers.push_back(reinterpret_cast(ptr)); + } + return batch_write_from_fn(keys, buffers, sizes, config); +} + +template +std::vector execute_batch_parallelism_write_requests( + const std::vector &keys, size_t value_count, + const py::object ¶llelisms, const py::object &writer_partitions, + const char *error_context, DirectWriteFn &&direct_write, + ParallelismWriteFn &¶llelism_write, + WriterPartitionWriteFn &&writer_partition_write) { + if (!parallelisms.is_none() && !writer_partitions.is_none()) { + LOG(ERROR) + << error_context + << ": writer_partitions cannot be combined with parallelisms"; + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + if (value_count != keys.size()) { + LOG(ERROR) << error_context + << ": values and keys must have the same length"; + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + if (parallelisms.is_none() && writer_partitions.is_none()) { + return direct_write(); + } + + std::vector results(keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + if (!parallelisms.is_none()) { + auto parallelism_list = validate_batch_parallelism_list( + parallelisms, keys.size(), error_context); + if (!parallelism_list.has_value()) { + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + for (size_t i = 0; i < keys.size(); ++i) { + results[i] = parallelism_write(i, (*parallelism_list)[i]); + } + return results; + } + + auto writer_partition_list = validate_batch_writer_partition_list( + writer_partitions, keys.size(), error_context); + if (!writer_partition_list.has_value()) { + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + for (size_t i = 0; i < keys.size(); ++i) { + results[i] = writer_partition_write(i, (*writer_partition_list)[i]); + } + return results; +} + +template +int execute_parallelism_tensor_write_route( + const std::string &key, pybind11::object tensor, + const ResolvedParallelismWriteRequest &request, + const ReplicateConfig &config, const char *error_context, + DirectWriteFn &&direct_write, WriterShardWriteFn &&writer_shard_write, + LegacyTpWriteFn &&legacy_tp_write, TpParallelWriteFn &&tp_parallel_write, + GenericShardWriteFn &&generic_shard_write) { + switch (request.route) { + case ParallelismWriteStorageRoute::DIRECT_FULL_OBJECT: + if (!ensure_tensor_write_supported(error_context)) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + if (int validate_result = validate_replicate_config(config); + validate_result) { + return validate_result; + } + return direct_write(key, tensor, config); + case ParallelismWriteStorageRoute::WRITER_PARTITION_SHARD: + if (!request.writer_partition.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return writer_shard_write(key, tensor, *request.writer_partition, + config); + case ParallelismWriteStorageRoute::LEGACY_SINGLE_TP: + case ParallelismWriteStorageRoute::TP_SHARDED_PARALLELISM: + case ParallelismWriteStorageRoute::GENERIC_PARALLELISM_SHARD: { + if (!request.parallelism.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + switch (request.route) { + case ParallelismWriteStorageRoute::LEGACY_SINGLE_TP: { + const auto &axis = request.parallelism->axes[0]; + return legacy_tp_write(key, tensor, config, axis.rank, + axis.size, + axis.split_dim.value_or(0)); + } + case ParallelismWriteStorageRoute::TP_SHARDED_PARALLELISM: { + auto tp_axis_index = + find_tp_axis_index(request.parallelism->axes); + if (!tp_axis_index.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + const auto &axis = + request.parallelism->axes[*tp_axis_index]; + return tp_parallel_write( + key, tensor, config, axis.rank, axis.size, + axis.split_dim.value_or(0), request.parallelism->axes); + } + case ParallelismWriteStorageRoute::GENERIC_PARALLELISM_SHARD: + return generic_shard_write(key, tensor, + *request.parallelism, config); + default: + break; + } + break; + } + } + return to_py_ret(ErrorCode::INVALID_PARAMS); +} + +template +int execute_tensor_parts_write(const std::string &key, + const std::vector> &values, + const ReplicateConfig &config, + const TensorWriteStoreOps &ops, + WritePartsFn &&write_parts) { + py::gil_scoped_release release_gil; + int ret = write_parts(key, values, config); + if (ret != 0) { + LOG(ERROR) << ops.parts_operation_name << " failed for key " << key + << " with code " << ret; + } + return ret; +} + +template +int execute_parallelism_direct_tensor_write_from(const std::string &key, + uintptr_t buffer_ptr, + size_t size, + const ReplicateConfig &config, + const char *operation_name, + DirectWriteFn &&direct_write) { + if (is_default_replicate_config(config)) { + return direct_write(key, buffer_ptr, size); + } + return execute_single_tensor_write_from( + operation_name, key, buffer_ptr, size, config, + [&](const std::string &write_key, void *buffer, size_t write_size, + const ReplicateConfig &write_config) { + return direct_write(write_key, reinterpret_cast(buffer), + write_size, write_config); + }); +} + +pybind11::object decode_tensor_object_buffer(uintptr_t buffer_ptr, size_t size, + const char *operation_name) { + if (!is_valid_tensor_object_buffer(buffer_ptr, size, operation_name)) { + return py::none(); + } + pybind11::object tensor = buffer_to_tensor( + NULL, reinterpret_cast(buffer_ptr), static_cast(size)); + if (tensor.is_none()) { + LOG(ERROR) << "Failed to decode tensor buffer for " << operation_name; + } + return tensor; +} + +std::string get_tp_write_shard_key(const std::string &base_key, int rank, + const std::vector &axes) { + if (axes.empty()) { + return get_tp_key_name(base_key, rank); + } + auto shard_axes = axes; + auto tp_axis_index = find_tp_axis_index(shard_axes); + if (!tp_axis_index.has_value()) { + return std::string(); + } + shard_axes[*tp_axis_index].rank = rank; + return get_parallelism_key_name(base_key, + TensorParallelismSpec{shard_axes}); +} + +template +int execute_tp_tensor_write_impl(const std::string &key, + pybind11::object tensor, + const ReplicateConfig &config, int tp_size, + int split_dim, + const std::vector &axes, + const char *error_context, + WriteShardFn &&write_shard) { + try { + auto shard_infos = build_tp_shard_infos( + tensor, tp_size, split_dim, + [&](int rank) { return get_tp_write_shard_key(key, rank, axes); }, + axes); + if (!shard_infos.has_value()) { + LOG(ERROR) << error_context; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + for (int rank = 0; rank < tp_size; ++rank) { + std::string shard_key = get_tp_write_shard_key(key, rank, axes); + if (shard_key.empty()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int ret = write_shard(shard_key, (*shard_infos)[rank], config); + if (ret != 0) return ret; + } + return 0; + } catch (const std::exception &e) { + LOG(ERROR) << error_context << ": " << e.what(); + return to_py_ret(ErrorCode::INVALID_PARAMS); + } +} + +template +int execute_tp_tensor_write_from_impl(const std::string &key, + uintptr_t buffer_ptr, size_t size, + const ReplicateConfig &config, + int tp_size, int split_dim, + const std::vector &axes, + const char *error_context, + WritePartsFn &&write_parts) { + auto parsed = + parse_tensor_metadata_from_raw_buffer(buffer_ptr, size, error_context); + if (!parsed.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + for (int rank = 0; rank < tp_size; ++rank) { + std::string shard_key = get_tp_write_shard_key(key, rank, axes); + if (shard_key.empty()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + auto plan = build_raw_tensor_shard_write_plan(*parsed, split_dim, rank, + tp_size, error_context); + if (!plan.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + const auto global_shape = TensorShapeToVector( + parsed->metadata.layout.global_shape, parsed->metadata.header.ndim); + const auto local_shape = TensorShapeToVector( + plan->metadata.layout.local_shape, plan->metadata.header.ndim); + if (axes.empty()) { + ParallelAxisSpec axis_spec{"tp", rank, tp_size, + split_dim, std::nullopt, std::nullopt}; + auto metadata = build_shard_metadata_from_shapes( + parsed->metadata.header.dtype, global_shape, local_shape, + {axis_spec}, plan->metadata.header.data_bytes); + if (!metadata.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + plan->metadata = *metadata; + } else { + auto shard_axes = axes; + auto tp_axis_index = find_tp_axis_index(shard_axes); + if (!tp_axis_index.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + shard_axes[*tp_axis_index].rank = rank; + const auto local_shape = TensorShapeToVector( + plan->metadata.layout.local_shape, plan->metadata.header.ndim); + auto metadata = build_shard_metadata_from_shapes( + parsed->metadata.header.dtype, global_shape, local_shape, + shard_axes, plan->metadata.header.data_bytes); + if (!metadata.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + plan->metadata = *metadata; + } + + std::vector> values; + values.emplace_back(reinterpret_cast(&plan->metadata), + plan->metadata.header.data_offset); + for (const auto &[src_offset, part_size] : plan->data_ranges) { + values.emplace_back( + reinterpret_cast(buffer_ptr + src_offset), + part_size); + } + int ret = write_parts(shard_key, values, config); + if (ret != 0) { + return ret; + } + } + return 0; +} + +template +std::vector batch_execute_tp_tensor_write_impl( + const std::vector &base_keys, + const pybind11::list &tensors_list, const ReplicateConfig &config, + int tp_size, int split_dim, const char *error_context, + BatchWriteFn &&batch_write) { + std::vector all_chunk_keys; + std::vector all_chunk_infos; + std::vector processed_indices; + std::vector final_results(base_keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + try { + for (size_t i = 0; i < base_keys.size(); ++i) { + py::object tensor = tensors_list[i]; + if (tensor.is_none()) { + final_results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); + continue; + } + + auto shard_infos = build_tp_shard_infos( + tensor, tp_size, split_dim, + [&](int rank) { return get_tp_key_name(base_keys[i], rank); }); + if (!shard_infos.has_value()) { + final_results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); + continue; + } + + processed_indices.push_back(i); + for (int rank = 0; rank < tp_size; ++rank) { + all_chunk_keys.push_back(get_tp_key_name(base_keys[i], rank)); + all_chunk_infos.push_back((*shard_infos)[rank]); + } + } + + if (all_chunk_keys.empty()) return final_results; + + std::vector chunk_results = + batch_write(all_chunk_keys, all_chunk_infos, config); + for (size_t i = 0; i < processed_indices.size(); ++i) { + size_t original_idx = processed_indices[i]; + bool all_ok = true; + for (int j = 0; j < tp_size; ++j) { + int res = chunk_results[i * tp_size + j]; + if (res != 0) { + final_results[original_idx] = res; + all_ok = false; + break; + } + } + if (all_ok) { + final_results[original_idx] = 0; + } + } + } catch (const std::exception &e) { + LOG(ERROR) << error_context << ": " << e.what(); + } + + return final_results; +} + +template +std::vector batch_decode_tensor_buffers_and_write( + const std::vector &keys, + const std::vector &buffer_ptrs, const std::vector &sizes, + const char *decode_error_context, + BatchTensorWriteFn &&batch_tensor_write_fn) { + py::list tensors_list; + std::vector processed_indices; + std::vector final_results(keys.size(), 0); + + for (size_t i = 0; i < keys.size(); ++i) { + if (!is_valid_tensor_object_buffer( + buffer_ptrs[i], sizes[i], + std::string("tensor object buffer at index ") + + std::to_string(i))) { + final_results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); + continue; + } + + py::object tensor = + buffer_to_tensor(NULL, reinterpret_cast(buffer_ptrs[i]), + static_cast(sizes[i])); + if (tensor.is_none()) { + LOG(ERROR) << decode_error_context << " at index " << i; + final_results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); + continue; + } + tensors_list.append(tensor); + processed_indices.push_back(i); + } + + if (processed_indices.empty()) { + return final_results; + } + + std::vector valid_keys; + valid_keys.reserve(processed_indices.size()); + for (size_t idx : processed_indices) { + valid_keys.push_back(keys[idx]); + } + + std::vector op_results = + batch_tensor_write_fn(valid_keys, tensors_list); + for (size_t i = 0; i < processed_indices.size(); ++i) { + final_results[processed_indices[i]] = op_results[i]; + } + return final_results; +} + +template +int execute_direct_parallelism_shard_write_from( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const TensorParallelismSpec ¶llelism, const ReplicateConfig &config, + const char *error_context, const TensorWriteStoreOps &ops, + WriteShardFn &&write_shard) { + auto parsed = + parse_tensor_metadata_from_raw_buffer(buffer_ptr, size, error_context); + if (!parsed.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + auto metadata = build_shard_metadata_from_shapes( + parsed->metadata.header.dtype, + TensorShapeToVector(parsed->metadata.layout.global_shape, + parsed->metadata.header.ndim), + TensorShapeToVector(parsed->metadata.layout.local_shape, + parsed->metadata.header.ndim), + parallelism.axes, parsed->data_bytes); + if (!metadata.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + std::string shard_key = get_parallelism_key_name(key, parallelism); + std::vector> values; + values.emplace_back(reinterpret_cast(&*metadata), + metadata->header.data_offset); + values.emplace_back( + reinterpret_cast(buffer_ptr + parsed->data_offset), + parsed->data_bytes); + return execute_tensor_parts_write(shard_key, values, config, ops, + std::forward(write_shard)); +} + +template +int execute_writer_partition_shard_write_from( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const WriterPartitionSpec &writer, const ReplicateConfig &config, + const char *error_context, const TensorWriteStoreOps &ops, + WritePartsFn &&write_parts, WriteManifestFn &&write_manifest) { + auto parsed = + parse_tensor_metadata_from_raw_buffer(buffer_ptr, size, error_context); + if (!parsed.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + auto plan = build_raw_tensor_shard_write_plan( + *parsed, writer.split_dim, writer.rank, writer.size, error_context); + if (!plan.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + ParallelAxisSpec axis_spec{"tp", writer.rank, + writer.size, writer.split_dim, + std::nullopt, std::nullopt}; + auto metadata = build_shard_metadata_from_shapes( + parsed->metadata.header.dtype, + TensorShapeToVector(parsed->metadata.layout.global_shape, + parsed->metadata.header.ndim), + TensorShapeToVector(plan->metadata.layout.local_shape, + plan->metadata.header.ndim), + {axis_spec}, plan->metadata.header.data_bytes); + if (!metadata.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + plan->metadata = *metadata; + + std::vector> values; + values.emplace_back(reinterpret_cast(&plan->metadata), + plan->metadata.header.data_offset); + for (const auto &[src_offset, part_size] : plan->data_ranges) { + values.emplace_back( + reinterpret_cast(buffer_ptr + src_offset), part_size); + } + + int ret = execute_tensor_parts_write( + get_writer_shard_key_name(key, writer), values, config, ops, + std::forward(write_parts)); + if (ret != 0) { + return ret; + } + return write_manifest( + get_writer_manifest_key_name(key), + build_writer_shard_manifest_from_shape( + TensorShapeToVector(parsed->metadata.layout.global_shape, + parsed->metadata.header.ndim), + parsed->metadata.header.dtype, writer), + config); +} + +template +int execute_parallelism_tensor_write_from_route( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const ResolvedParallelismWriteRequest &request, + const ReplicateConfig &config, const char *error_context, + DirectFromWriteFn &&direct_from_write, + WriterShardFromWriteFn &&writer_shard_from_write, + TpShardFromWriteFn &&tp_shard_from_write, + GenericShardFromWriteFn &&generic_shard_from_write, + TensorRouteWriteFn &&tensor_route_write) { + switch (request.route) { + case ParallelismWriteStorageRoute::DIRECT_FULL_OBJECT: + return direct_from_write(key, buffer_ptr, size, config); + case ParallelismWriteStorageRoute::WRITER_PARTITION_SHARD: + if (!request.writer_partition.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return writer_shard_from_write(key, buffer_ptr, size, + *request.writer_partition, config); + case ParallelismWriteStorageRoute::LEGACY_SINGLE_TP: + case ParallelismWriteStorageRoute::TP_SHARDED_PARALLELISM: { + if (!request.parallelism.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + auto tp_axis_index = find_tp_axis_index(request.parallelism->axes); + if (!tp_axis_index.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + const auto &axis = request.parallelism->axes[*tp_axis_index]; + const auto axes = + request.route == ParallelismWriteStorageRoute::LEGACY_SINGLE_TP + ? std::vector{} + : request.parallelism->axes; + return tp_shard_from_write(key, buffer_ptr, size, axis.size, + axis.split_dim.value_or(0), axes, + config); + } + case ParallelismWriteStorageRoute::GENERIC_PARALLELISM_SHARD: + if (!request.parallelism.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return generic_shard_from_write(key, buffer_ptr, size, + *request.parallelism, config); + } + + pybind11::object tensor = + decode_tensor_object_buffer(buffer_ptr, size, error_context); + if (tensor.is_none()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return tensor_route_write(key, tensor, request, config); +} + +int put_tensor_with_writer_shards( + const std::string &key, pybind11::object tensor, + const WriterPartitionSpec &writer, + const ReplicateConfig &config = ReplicateConfig{}) { + if (!ensure_tensor_write_supported("put_tensor_with_writer_shards")) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + auto writer_info = build_writer_shard_tensor_info( + key, tensor, writer, "put_tensor_with_writer_shards"); + if (!writer_info.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + int ret = + put_tensor_info_impl(writer_info->shard_key, writer_info->info, config); + if (ret != 0) { + return ret; + } + return put_manifest_impl(get_writer_manifest_key_name(key), + writer_info->manifest, config); +} + +int put_tensor_parallelism_tp_impl(const std::string &key, + pybind11::object tensor, + const ReplicateConfig &config, int tp_rank, + int tp_size, int split_dim, + const std::vector &axes) { + std::string shard_key = get_tp_write_shard_key(key, tp_rank, axes); + if (shard_key.empty()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + auto shard_info = build_direct_parallelism_shard_info( + tensor, axes, shard_key, true /* infer_global_shape */); + if (!shard_info.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int ret = put_tensor_info_impl(shard_key, shard_info->info, config); + if (ret != 0) { + return ret; + } + return put_manifest_impl(get_parallelism_manifest_key_name(key), + shard_info->manifest, config); +} + +int put_requested_parallelism_shard(const std::string &key, + pybind11::object tensor, + const TensorParallelismSpec ¶llelism, + const ReplicateConfig &config) { + std::string shard_key = get_parallelism_key_name(key, parallelism); + auto shard_info = build_requested_parallelism_shard_info( + tensor, parallelism, shard_key, "put_tensor_with_parallelism"); + if (!shard_info.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int ret = put_tensor_info_impl(shard_key, shard_info->info, config); + if (ret != 0) { + return ret; + } + return put_manifest_impl(get_parallelism_manifest_key_name(key), + shard_info->manifest, config); +} + +int put_direct_parallelism_shard(const std::string &key, + pybind11::object tensor, + const TensorParallelismSpec ¶llelism, + const ReplicateConfig &config) { + auto tp_axis_index = find_tp_axis_index(parallelism.axes); + if (tp_axis_index.has_value() && parallelism.axes.size() > 1) { + return put_requested_parallelism_shard(key, tensor, parallelism, + config); + } + auto info = + build_direct_parallelism_shard_info(tensor, parallelism.axes, key); + if (!info.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return put_tensor_info_impl(get_parallelism_key_name(key, parallelism), + *info, config); +} + +int put_tensor_with_writer_shards_from( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const WriterPartitionSpec &writer, + const ReplicateConfig &config = ReplicateConfig{}) { + return execute_writer_partition_shard_write_from( + key, buffer_ptr, size, writer, config, + "put_tensor_with_parallelism_from", TensorWriteStoreOps{"put_parts"}, + [this](const std::string &shard_key, + const std::vector> &values, + const ReplicateConfig &write_config) { + return store_->put_parts(shard_key, values, write_config); + }, + [this](const std::string &manifest_key, + const WriterShardManifest &manifest, + const ReplicateConfig &write_config) { + return put_manifest_impl(manifest_key, manifest, write_config); + }); +} + +int put_direct_parallelism_shard_from(const std::string &key, + uintptr_t buffer_ptr, size_t size, + const TensorParallelismSpec ¶llelism, + const ReplicateConfig &config) { + return execute_direct_parallelism_shard_write_from( + key, buffer_ptr, size, parallelism, config, + "put_tensor_with_parallelism_from", TensorWriteStoreOps{"put_parts"}, + [this](const std::string &shard_key, + const std::vector> &values, + const ReplicateConfig &write_config) { + return store_->put_parts(shard_key, values, write_config); + }); +} + +int execute_put_tensor_with_parallelism_route( + const std::string &key, pybind11::object tensor, + const ResolvedParallelismWriteRequest &request, + const ReplicateConfig &config) { + return execute_parallelism_tensor_write_route( + key, tensor, request, config, "put_tensor_with_parallelism", + [this](const std::string &write_key, pybind11::object write_tensor, + const ReplicateConfig &write_config) { + return put_tensor_impl(write_key, write_tensor, write_config); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const WriterPartitionSpec &writer, + const ReplicateConfig &write_config) { + return put_tensor_with_writer_shards(write_key, write_tensor, + writer, write_config); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const ReplicateConfig &write_config, int rank, int size, + int split_dim) { + return put_tensor_with_tp_impl(write_key, write_tensor, + write_config, rank, size, split_dim); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const ReplicateConfig &write_config, int rank, int size, + int split_dim, const std::vector &axes) { + if (axes.size() == 1) { + return put_tensor_parallelism_tp_impl(write_key, write_tensor, + write_config, rank, size, + split_dim, axes); + } + return put_requested_parallelism_shard(write_key, write_tensor, + TensorParallelismSpec{axes}, + write_config); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const TensorParallelismSpec ¶llelism_spec, + const ReplicateConfig &write_config) { + return put_direct_parallelism_shard(write_key, write_tensor, + parallelism_spec, write_config); + }); +} + +int put_tensor_with_parallelism( + const std::string &key, pybind11::object tensor, + const py::object ¶llelism = py::none(), + const ReplicateConfig &config = ReplicateConfig{}, + const py::object &writer_partition = py::none()) { + auto request = resolve_parallelism_write_request( + parallelism, writer_partition, "put_tensor_with_parallelism"); + if (!request.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return execute_put_tensor_with_parallelism_route(key, tensor, *request, + config); +} + +std::vector batch_put_tensor_with_parallelism( + const std::vector &keys, const pybind11::list &tensors_list, + const py::object ¶llelisms = py::none(), + const ReplicateConfig &config = ReplicateConfig{}, + const py::object &writer_partitions = py::none()) { + return execute_batch_parallelism_write_requests( + keys, tensors_list.size(), parallelisms, writer_partitions, + "batch_put_tensor_with_parallelism", + [this, &keys, &tensors_list, &config]() { + if (!is_client_initialized() || use_dummy_client_) { + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + int validate_result = validate_replicate_config(config); + if (validate_result) { + return std::vector(keys.size(), validate_result); + } + return batch_put_tensor_impl(keys, tensors_list, config); + }, + [this, &keys, &tensors_list, &config](size_t i, + const py::handle ¶llelism) { + return put_tensor_with_parallelism( + keys[i], tensors_list[i], + py::reinterpret_borrow(parallelism), config); + }, + [this, &keys, &tensors_list, &config]( + size_t i, const py::handle &writer_partition) { + return put_tensor_with_parallelism( + keys[i], tensors_list[i], py::none(), config, + py::reinterpret_borrow(writer_partition)); + }); +} + +bool is_valid_tensor_object_buffer(uintptr_t buffer_ptr, size_t size, + const std::string &op_name) { + return parse_tensor_metadata_from_raw_buffer(buffer_ptr, size, + op_name.c_str()) + .has_value(); +} + +int execute_put_tensor_with_parallelism_from_route( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const ResolvedParallelismWriteRequest &request, + const ReplicateConfig &config) { + return execute_parallelism_tensor_write_from_route( + key, buffer_ptr, size, request, config, + "put_tensor_with_parallelism_from", + [this](const std::string &write_key, uintptr_t write_buffer_ptr, + size_t write_size, const ReplicateConfig &write_config) { + if (!is_default_replicate_config(write_config)) { + if (!is_valid_tensor_object_buffer( + write_buffer_ptr, write_size, + "put_tensor_with_parallelism_from")) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + if (!ensure_tensor_write_supported( + "put_tensor_with_parallelism_from")) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int validate_result = validate_replicate_config(write_config); + if (validate_result) return validate_result; + py::gil_scoped_release release_gil; + return store_->put_from( + write_key, reinterpret_cast(write_buffer_ptr), + write_size, write_config); + } + return put_tensor_from(write_key, write_buffer_ptr, write_size); + }, + [this](const std::string &write_key, uintptr_t write_buffer_ptr, + size_t write_size, const WriterPartitionSpec &writer, + const ReplicateConfig &write_config) { + return put_tensor_with_writer_shards_from( + write_key, write_buffer_ptr, write_size, writer, write_config); + }, + [this](const std::string &write_key, uintptr_t write_buffer_ptr, + size_t write_size, int tp_size, int split_dim, + const std::vector &axes, + const ReplicateConfig &write_config) { + return execute_tp_tensor_write_from_impl( + write_key, write_buffer_ptr, write_size, write_config, tp_size, + split_dim, axes, "put_tensor_with_parallelism_from", + [this](const std::string &shard_key, + const std::vector> &values, + const ReplicateConfig &parts_config) { + py::gil_scoped_release release_gil; + int ret = + store_->put_parts(shard_key, values, parts_config); + if (ret != 0) { + LOG(ERROR) << "put_parts failed for key " << shard_key + << " with code " << ret; + } + return ret; + }); + }, + [this](const std::string &write_key, uintptr_t write_buffer_ptr, + size_t write_size, const TensorParallelismSpec ¶llelism_spec, + const ReplicateConfig &write_config) { + return put_direct_parallelism_shard_from( + write_key, write_buffer_ptr, write_size, parallelism_spec, + write_config); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const ResolvedParallelismWriteRequest &write_request, + const ReplicateConfig &write_config) { + return execute_put_tensor_with_parallelism_route( + write_key, write_tensor, write_request, write_config); + }); +} + +int put_tensor_with_parallelism_from( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const py::object ¶llelism = py::none(), + const ReplicateConfig &config = ReplicateConfig{}, + const py::object &writer_partition = py::none()) { + auto request = resolve_parallelism_write_request( + parallelism, writer_partition, "put_tensor_with_parallelism_from"); + if (!request.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return execute_put_tensor_with_parallelism_from_route(key, buffer_ptr, size, + *request, config); +} + +std::vector batch_put_tensor_with_parallelism_from( + const std::vector &keys, + const std::vector &buffer_ptrs, const std::vector &sizes, + const py::object ¶llelisms = py::none(), + const ReplicateConfig &config = ReplicateConfig{}, + const py::object &writer_partitions = py::none()) { + return execute_batch_parallelism_write_requests( + keys, buffer_ptrs.size(), parallelisms, writer_partitions, + "batch_put_tensor_with_parallelism_from", + [this, &keys, &buffer_ptrs, &sizes, &config]() { + if (!is_default_replicate_config(config)) { + if (!is_client_initialized() || use_dummy_client_) { + return std::vector( + keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + } + int validate_result = validate_replicate_config(config); + if (validate_result) { + return std::vector(keys.size(), validate_result); + } + if (keys.size() != buffer_ptrs.size() || + keys.size() != sizes.size()) { + LOG(ERROR) << "Size mismatch: keys, buffer_ptrs, and sizes " + "must have the same length"; + return std::vector( + keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + } + for (size_t i = 0; i < sizes.size(); ++i) { + if (!is_valid_tensor_object_buffer( + buffer_ptrs[i], sizes[i], + "tensor object buffer at index " + + std::to_string(i))) { + return std::vector( + keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + } + } + std::vector buffers; + buffers.reserve(buffer_ptrs.size()); + for (uintptr_t ptr : buffer_ptrs) { + buffers.push_back(reinterpret_cast(ptr)); + } + py::gil_scoped_release release_gil; + return store_->batch_put_from(keys, buffers, sizes, config); + } + return batch_put_tensor_from(keys, buffer_ptrs, sizes); + }, + [this, &keys, &buffer_ptrs, &sizes, &config]( + size_t i, const py::handle ¶llelism) { + return put_tensor_with_parallelism_from( + keys[i], buffer_ptrs[i], sizes[i], + py::reinterpret_borrow(parallelism), config); + }, + [this, &keys, &buffer_ptrs, &sizes, &config]( + size_t i, const py::handle &writer_partition) { + return put_tensor_with_parallelism_from( + keys[i], buffer_ptrs[i], sizes[i], py::none(), config, + py::reinterpret_borrow(writer_partition)); + }); +} + +int upsert_tensor_with_writer_shards( + const std::string &key, pybind11::object tensor, + const WriterPartitionSpec &writer, + const ReplicateConfig &config = ReplicateConfig{}) { + if (!ensure_tensor_write_supported("upsert_tensor_with_writer_shards")) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + auto writer_info = build_writer_shard_tensor_info( + key, tensor, writer, "upsert_tensor_with_parallelism"); + if (!writer_info.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + + int ret = upsert_tensor_info_impl(writer_info->shard_key, writer_info->info, + config); + if (ret != 0) { + return ret; + } + return upsert_manifest_impl(get_writer_manifest_key_name(key), + writer_info->manifest, config); +} + +int upsert_tensor_parallelism_tp_impl( + const std::string &key, pybind11::object tensor, + const ReplicateConfig &config, int tp_rank, int tp_size, int split_dim, + const std::vector &axes) { + std::string shard_key = get_tp_write_shard_key(key, tp_rank, axes); + if (shard_key.empty()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + auto shard_info = build_direct_parallelism_shard_info( + tensor, axes, shard_key, true /* infer_global_shape */); + if (!shard_info.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int ret = upsert_tensor_info_impl(shard_key, shard_info->info, config); + if (ret != 0) { + return ret; + } + return upsert_manifest_impl(get_parallelism_manifest_key_name(key), + shard_info->manifest, config); +} + +int upsert_requested_parallelism_shard(const std::string &key, + pybind11::object tensor, + const TensorParallelismSpec ¶llelism, + const ReplicateConfig &config) { + std::string shard_key = get_parallelism_key_name(key, parallelism); + auto shard_info = build_requested_parallelism_shard_info( + tensor, parallelism, shard_key, "upsert_tensor_with_parallelism"); + if (!shard_info.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int ret = upsert_tensor_info_impl(shard_key, shard_info->info, config); + if (ret != 0) { + return ret; + } + return upsert_manifest_impl(get_parallelism_manifest_key_name(key), + shard_info->manifest, config); +} + +int upsert_direct_parallelism_shard(const std::string &key, + pybind11::object tensor, + const TensorParallelismSpec ¶llelism, + const ReplicateConfig &config) { + auto tp_axis_index = find_tp_axis_index(parallelism.axes); + if (tp_axis_index.has_value() && parallelism.axes.size() > 1) { + return upsert_requested_parallelism_shard(key, tensor, parallelism, + config); + } + auto info = + build_direct_parallelism_shard_info(tensor, parallelism.axes, key); + if (!info.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return upsert_tensor_info_impl(get_parallelism_key_name(key, parallelism), + *info, config); +} + +int upsert_tensor_with_writer_shards_from( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const WriterPartitionSpec &writer, + const ReplicateConfig &config = ReplicateConfig{}) { + return execute_writer_partition_shard_write_from( + key, buffer_ptr, size, writer, config, + "upsert_tensor_with_parallelism_from", + TensorWriteStoreOps{"upsert_parts"}, + [this](const std::string &shard_key, + const std::vector> &values, + const ReplicateConfig &write_config) { + return store_->upsert_parts(shard_key, values, write_config); + }, + [this](const std::string &manifest_key, + const WriterShardManifest &manifest, + const ReplicateConfig &write_config) { + return upsert_manifest_impl(manifest_key, manifest, write_config); + }); +} + +int upsert_direct_parallelism_shard_from( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const TensorParallelismSpec ¶llelism, const ReplicateConfig &config) { + return execute_direct_parallelism_shard_write_from( + key, buffer_ptr, size, parallelism, config, + "upsert_tensor_with_parallelism_from", + TensorWriteStoreOps{"upsert_parts"}, + [this](const std::string &shard_key, + const std::vector> &values, + const ReplicateConfig &write_config) { + return store_->upsert_parts(shard_key, values, write_config); + }); +} + +int execute_upsert_tensor_with_parallelism_route( + const std::string &key, pybind11::object tensor, + const ResolvedParallelismWriteRequest &request, + const ReplicateConfig &config) { + return execute_parallelism_tensor_write_route( + key, tensor, request, config, "upsert_tensor_with_parallelism", + [this](const std::string &write_key, pybind11::object write_tensor, + const ReplicateConfig &write_config) { + return upsert_tensor_impl(write_key, write_tensor, write_config); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const WriterPartitionSpec &writer, + const ReplicateConfig &write_config) { + return upsert_tensor_with_writer_shards(write_key, write_tensor, + writer, write_config); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const ReplicateConfig &write_config, int rank, int size, + int split_dim) { + return upsert_tensor_with_tp_impl(write_key, write_tensor, + write_config, size, split_dim); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const ReplicateConfig &write_config, int rank, int size, + int split_dim, const std::vector &axes) { + if (axes.size() == 1) { + return upsert_tensor_parallelism_tp_impl( + write_key, write_tensor, write_config, rank, size, + split_dim, axes); + } + return upsert_requested_parallelism_shard( + write_key, write_tensor, TensorParallelismSpec{axes}, + write_config); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const TensorParallelismSpec ¶llelism_spec, + const ReplicateConfig &write_config) { + return upsert_direct_parallelism_shard( + write_key, write_tensor, parallelism_spec, write_config); + }); +} + +int upsert_tensor_with_parallelism( + const std::string &key, pybind11::object tensor, + const py::object ¶llelism = py::none(), + const ReplicateConfig &config = ReplicateConfig{}, + const py::object &writer_partition = py::none()) { + auto request = resolve_parallelism_write_request( + parallelism, writer_partition, "upsert_tensor_with_parallelism"); + if (!request.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return execute_upsert_tensor_with_parallelism_route(key, tensor, *request, + config); +} + +int execute_upsert_tensor_with_parallelism_from_route( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const ResolvedParallelismWriteRequest &request, + const ReplicateConfig &config) { + return execute_parallelism_tensor_write_from_route( + key, buffer_ptr, size, request, config, + "upsert_tensor_with_parallelism_from", + [this](const std::string &write_key, uintptr_t write_buffer_ptr, + size_t write_size, const ReplicateConfig &write_config) { + if (!is_default_replicate_config(write_config)) { + if (!is_valid_tensor_object_buffer( + write_buffer_ptr, write_size, + "upsert_tensor_with_parallelism_from")) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + void *buffer = reinterpret_cast(write_buffer_ptr); + if (!is_client_initialized()) { + LOG(ERROR) << "Client is not initialized"; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + if (use_dummy_client_) { + LOG(ERROR) << "upsert_tensor_with_parallelism_from is not " + "supported for dummy client"; + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + int validate_result = validate_replicate_config(write_config); + if (validate_result) return validate_result; + py::gil_scoped_release release_gil; + return store_->upsert_from(write_key, buffer, write_size, + write_config); + } + return upsert_tensor_from(write_key, write_buffer_ptr, write_size); + }, + [this](const std::string &write_key, uintptr_t write_buffer_ptr, + size_t write_size, const WriterPartitionSpec &writer, + const ReplicateConfig &write_config) { + return upsert_tensor_with_writer_shards_from( + write_key, write_buffer_ptr, write_size, writer, write_config); + }, + [this](const std::string &write_key, uintptr_t write_buffer_ptr, + size_t write_size, int tp_size, int split_dim, + const std::vector &axes, + const ReplicateConfig &write_config) { + return execute_tp_tensor_write_from_impl( + write_key, write_buffer_ptr, write_size, write_config, tp_size, + split_dim, axes, "upsert_tensor_with_parallelism_from", + [this](const std::string &shard_key, + const std::vector> &values, + const ReplicateConfig &parts_config) { + py::gil_scoped_release release_gil; + int ret = + store_->upsert_parts(shard_key, values, parts_config); + if (ret != 0) { + LOG(ERROR) << "upsert_parts failed for key " + << shard_key << " with code " << ret; + } + return ret; + }); + }, + [this](const std::string &write_key, uintptr_t write_buffer_ptr, + size_t write_size, const TensorParallelismSpec ¶llelism_spec, + const ReplicateConfig &write_config) { + return upsert_direct_parallelism_shard_from( + write_key, write_buffer_ptr, write_size, parallelism_spec, + write_config); + }, + [this](const std::string &write_key, pybind11::object write_tensor, + const ResolvedParallelismWriteRequest &write_request, + const ReplicateConfig &write_config) { + return execute_upsert_tensor_with_parallelism_route( + write_key, write_tensor, write_request, write_config); + }); +} + +int upsert_tensor_with_parallelism_from( + const std::string &key, uintptr_t buffer_ptr, size_t size, + const py::object ¶llelism = py::none(), + const ReplicateConfig &config = ReplicateConfig{}, + const py::object &writer_partition = py::none()) { + auto request = resolve_parallelism_write_request( + parallelism, writer_partition, "upsert_tensor_with_parallelism_from"); + if (!request.has_value()) { + return to_py_ret(ErrorCode::INVALID_PARAMS); + } + return execute_upsert_tensor_with_parallelism_from_route( + key, buffer_ptr, size, *request, config); +} + +std::vector batch_upsert_tensor_with_parallelism( + const std::vector &keys, const pybind11::list &tensors_list, + const py::object ¶llelisms = py::none(), + const ReplicateConfig &config = ReplicateConfig{}, + const py::object &writer_partitions = py::none()) { + return execute_batch_parallelism_write_requests( + keys, tensors_list.size(), parallelisms, writer_partitions, + "batch_upsert_tensor_with_parallelism", + [this, &keys, &tensors_list, &config]() { + if (!is_client_initialized() || use_dummy_client_) { + return std::vector(keys.size(), + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + int validate_result = validate_replicate_config(config); + if (validate_result) { + return std::vector(keys.size(), validate_result); + } + return batch_upsert_tensor_impl(keys, tensors_list, config); + }, + [this, &keys, &tensors_list, &config](size_t i, + const py::handle ¶llelism) { + return upsert_tensor_with_parallelism( + keys[i], tensors_list[i], + py::reinterpret_borrow(parallelism), config); + }, + [this, &keys, &tensors_list, &config]( + size_t i, const py::handle &writer_partition) { + return upsert_tensor_with_parallelism( + keys[i], tensors_list[i], py::none(), config, + py::reinterpret_borrow(writer_partition)); + }); +} + +std::vector batch_upsert_tensor_with_parallelism_from( + const std::vector &keys, + const std::vector &buffer_ptrs, const std::vector &sizes, + const py::object ¶llelisms = py::none(), + const ReplicateConfig &config = ReplicateConfig{}, + const py::object &writer_partitions = py::none()) { + return execute_batch_parallelism_write_requests( + keys, buffer_ptrs.size(), parallelisms, writer_partitions, + "batch_upsert_tensor_with_parallelism_from", + [this, &keys, &buffer_ptrs, &sizes, &config]() { + if (!is_default_replicate_config(config)) { + if (!is_client_initialized()) { + LOG(ERROR) << "Client is not initialized"; + return std::vector( + keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + } + if (use_dummy_client_) { + LOG(ERROR) << "batch_upsert_tensor_with_parallelism_from " + "is not supported for dummy client"; + return std::vector( + keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + } + int validate_result = validate_replicate_config(config); + if (validate_result) { + return std::vector(keys.size(), validate_result); + } + if (keys.empty()) { + return std::vector(); + } + if (keys.size() != buffer_ptrs.size() || + keys.size() != sizes.size()) { + LOG(ERROR) << "Size mismatch: keys, buffer_ptrs, and sizes " + "must have the same length"; + return std::vector( + keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + } + for (size_t i = 0; i < sizes.size(); ++i) { + if (!is_valid_tensor_object_buffer( + buffer_ptrs[i], sizes[i], + "tensor object buffer at index " + + std::to_string(i))) { + return std::vector( + keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + } + } + std::vector buffers; + buffers.reserve(buffer_ptrs.size()); + for (uintptr_t ptr : buffer_ptrs) { + buffers.push_back(reinterpret_cast(ptr)); + } + py::gil_scoped_release release_gil; + return store_->batch_upsert_from(keys, buffers, sizes, config); + } + return batch_upsert_tensor_from(keys, buffer_ptrs, sizes); + }, + [this, &keys, &buffer_ptrs, &sizes, &config]( + size_t i, const py::handle ¶llelism) { + return upsert_tensor_with_parallelism_from( + keys[i], buffer_ptrs[i], sizes[i], + py::reinterpret_borrow(parallelism), config); + }, + [this, &keys, &buffer_ptrs, &sizes, &config]( + size_t i, const py::handle &writer_partition) { + return upsert_tensor_with_parallelism_from( + keys[i], buffer_ptrs[i], sizes[i], py::none(), config, + py::reinterpret_borrow(writer_partition)); + }); +} diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index cb6b5452..5c7b3843 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -101,6 +101,15 @@ class RealClient : public PyClient { int unregister_buffer(void *buffer); + struct RegisteredBufferRegion { + void *base{nullptr}; + size_t size{0}; + size_t offset{0}; + }; + + std::optional resolve_registered_buffer( + void *buffer) const; + /** * @brief Get object data directly into a pre-allocated buffer * @param key Key of the object to get diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index f2f25b99..1ddc9012 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -2316,6 +2316,28 @@ int RealClient::unregister_buffer(void *buffer) { return to_py_ret(unregister_buffer_internal(buffer)); } +std::optional +RealClient::resolve_registered_buffer(void *buffer) const { + std::shared_lock lock(registered_buffer_mutex_); + const auto target = reinterpret_cast(buffer); + for (const auto &[registered_buffer, registered_size] : + registered_buffer_sizes_) { + const auto base = reinterpret_cast(registered_buffer); + if (target < base) { + continue; + } + const auto offset = target - base; + if (offset < registered_size) { + return RegisteredBufferRegion{ + .base = registered_buffer, + .size = registered_size, + .offset = static_cast(offset), + }; + } + } + return std::nullopt; +} + tl::expected RealClient::resolve_ranged_read_metadata(const std::string &key) { if (!client_) { diff --git a/scripts/test_tensor_api.py b/scripts/test_tensor_api.py index a0972d2a..855c414c 100644 --- a/scripts/test_tensor_api.py +++ b/scripts/test_tensor_api.py @@ -8,6 +8,7 @@ import unittest import torch import numpy as np from dataclasses import dataclass +import mooncake.store as mooncake_store from mooncake.store import MooncakeDistributedStore from mooncake.store import ReplicateConfig @@ -15,6 +16,14 @@ from mooncake.mooncake_config import MooncakeConfig import concurrent.futures + +@dataclass +class WriterPartition: + rank: int + size: int + split_dim: int + + # ========================================== # Global Variables & Configuration # ========================================== @@ -29,12 +38,13 @@ DEFAULT_LOCAL_BUFFER_SIZE = 8 * 1024 * 1024 * 1024 # 8 GB DEFAULT_MASTER_METRICS_PORT = 9003 DEFAULT_CHECK_SERVER = False -# Must match C++ TensorMetadata: int32_t dtype + int32_t ndim + int64_t shape[4] -TENSOR_METADATA_SIZE = 4 + 4 + 8 * 4 # 40 bytes +# Must match current C++ TensorMetadata layout. +# TensorObjectHeader (40) + TensorLayoutMetadata (264) = 304 bytes. +TENSOR_METADATA_SIZE = 304 def serialized_tensor_size(tensor): - """Size in bytes of [TensorMetadata][tensor data] as stored by get_tensor_into.""" + """Size in bytes of [TensorObjectHeader+layout metadata][tensor data].""" return TENSOR_METADATA_SIZE + tensor.numel() * tensor.element_size() def verify_tensor_equality(original, received, rtol=0, atol=0, verbose=True): @@ -143,6 +153,234 @@ def setUpModule(): print(f"❌ Failed to establish global store connection: {e}") sys.exit(1) + +def require_unified_parallelism_api(test_case): + missing = [ + name for name in ["ParallelAxis", "TensorParallelism", "ReadTarget"] + if not hasattr(mooncake_store, name) + ] + if missing: + test_case.skipTest( + "Unified parallelism Python bindings are not available in the installed extension: " + + ", ".join(missing) + ) + + +def make_parallel_axis(kind, rank, size, split_dim=None): + axis = mooncake_store.ParallelAxis() + axis.kind = kind + axis.rank = rank + axis.size = size + axis.split_dim = split_dim + return axis + + +def make_tensor_parallelism(axes): + parallelism = mooncake_store.TensorParallelism() + parallelism.axes = axes + return parallelism + + +def make_read_target(mode, parallelism=None): + target = mooncake_store.ReadTarget() + target.mode = mode + target.parallelism = parallelism + return target + + +def make_writer_partition(rank, size, split_dim): + return WriterPartition(rank=rank, size=size, split_dim=split_dim) + + +def build_tp_parallelism(tp_size, split_dim, rank=0): + return make_tensor_parallelism([ + make_parallel_axis("tp", rank=rank, size=tp_size, split_dim=split_dim) + ]) + + +def build_dp_tp_parallelism(dp_rank, dp_size, tp_rank, tp_size, split_dim): + return make_tensor_parallelism([ + make_parallel_axis("dp", rank=dp_rank, size=dp_size), + make_parallel_axis("tp", rank=tp_rank, size=tp_size, split_dim=split_dim), + ]) + + +def build_pp_tp_parallelism(pp_rank, pp_size, stage_id, tp_rank, tp_size, split_dim): + pp_axis = make_parallel_axis("pp", rank=pp_rank, size=pp_size) + pp_axis.stage_id = stage_id + return make_tensor_parallelism([ + pp_axis, + make_parallel_axis("tp", rank=tp_rank, size=tp_size, split_dim=split_dim), + ]) + + +def build_ep_parallelism(ep_rank, ep_size, expert_id): + ep_axis = make_parallel_axis("ep", rank=ep_rank, size=ep_size) + ep_axis.expert_id = expert_id + return make_tensor_parallelism([ep_axis]) + + +def build_ep_tp_parallelism(ep_rank, ep_size, expert_id, tp_rank, tp_size, split_dim): + ep_axis = make_parallel_axis("ep", rank=ep_rank, size=ep_size) + ep_axis.expert_id = expert_id + return make_tensor_parallelism([ + ep_axis, + make_parallel_axis("tp", rank=tp_rank, size=tp_size, split_dim=split_dim), + ]) + + +def build_parallelism_mode(mode, tp_rank, tp_size, split_dim): + if mode == "tp": + return build_tp_parallelism(tp_size, split_dim, rank=tp_rank) + if mode == "dp_tp": + return build_dp_tp_parallelism(0, 2, tp_rank, tp_size, split_dim) + if mode == "pp_tp": + return build_pp_tp_parallelism(0, 2, 7, tp_rank, tp_size, split_dim) + if mode == "ep_tp": + return build_ep_tp_parallelism(0, 2, 11, tp_rank, tp_size, split_dim) + raise ValueError(f"unsupported parallelism mode: {mode}") + + +def put_full_tensor_with_parallelism_mode(store, key, tensor, split_dim, put_tp_size, mode): + if mode == "tp": + shards = tensor.chunk(put_tp_size, split_dim) + for rank, shard in enumerate(shards): + rc = store.put_tensor_with_parallelism( + key, + shard.contiguous(), + build_parallelism_mode(mode, rank, put_tp_size, split_dim), + ) + if rc != 0: + return rc + return 0 + + for rank in range(put_tp_size): + rc = store.put_tensor_with_parallelism( + key, + tensor, + build_parallelism_mode(mode, rank, put_tp_size, split_dim), + ) + if rc != 0: + return rc + return 0 + + +def clone_parallelism(parallelism): + cloned_axes = [] + for axis in parallelism.axes: + cloned_axis = make_parallel_axis( + axis.kind, + rank=axis.rank, + size=axis.size, + split_dim=axis.split_dim, + ) + if getattr(axis, "expert_id", None) is not None: + cloned_axis.expert_id = axis.expert_id + if getattr(axis, "stage_id", None) is not None: + cloned_axis.stage_id = axis.stage_id + cloned_axes.append(cloned_axis) + return make_tensor_parallelism(cloned_axes) + + +def put_full_tensor_with_parallelism_template(store, key, tensor, parallelism): + tp_axis = next(axis for axis in parallelism.axes if axis.kind == "tp") + for rank in range(tp_axis.size): + shard_parallelism = clone_parallelism(parallelism) + for axis in shard_parallelism.axes: + if axis.kind == "tp": + axis.rank = rank + axis.size = tp_axis.size + axis.split_dim = tp_axis.split_dim + break + rc = store.put_tensor_with_parallelism( + key, + tensor, + shard_parallelism, + ) + if rc != 0: + return rc + return 0 + + +def expected_target_shard(tensor, split_dim, target_tp_size, target_rank): + return chunk_tensor_for_rank(tensor, target_tp_size, split_dim, target_rank).contiguous() + + +def reorder_parallelism_axes(parallelism, ordered_kinds): + axis_by_kind = {axis.kind: axis for axis in parallelism.axes} + return make_tensor_parallelism([axis_by_kind[kind] for kind in ordered_kinds]) + + +def make_deterministic_tensor(shape, dtype=torch.float32): + numel = 1 + for dim in shape: + numel *= dim + return torch.arange(numel, dtype=dtype).view(*shape).contiguous() + + +def chunk_tensor_for_rank(tensor, tp_size, split_dim, rank): + return tensor.chunk(tp_size, split_dim)[rank] + + +def put_full_tensor_with_unified_tp(store, key, tensor, tp_size, split_dim): + for rank, shard in enumerate(tensor.chunk(tp_size, split_dim)): + rc = store.put_tensor_with_parallelism( + key, + shard.contiguous(), + build_tp_parallelism(tp_size, split_dim, rank=rank), + ) + if rc != 0: + return rc + return 0 + + +def batch_put_full_tensors_with_unified_tp(store, keys, tensors, tp_size, split_dim): + results = [0 for _ in keys] + for rank in range(tp_size): + shard_tensors = [chunk_tensor_for_rank(tensor, tp_size, split_dim, rank).contiguous() for tensor in tensors] + rank_results = store.batch_put_tensor_with_parallelism( + keys, + shard_tensors, + [build_tp_parallelism(tp_size, split_dim, rank=rank) for _ in keys], + ) + results = rank_results + if any(rc != 0 for rc in rank_results): + return rank_results + return results + + +def reconstruction_case_name(shape, split_dim, tp_size): + shape_str = "x".join(str(dim) for dim in shape) + return f"shape={shape_str}, split_dim={split_dim}, tp_size={tp_size}" + + +def is_uniform_tp_case(shape, split_dim, tp_size): + return shape[split_dim] % tp_size == 0 + + +def put_uniform_full_tensor_with_unified_tp(store, key, tensor, tp_size, split_dim): + if not is_uniform_tp_case(tuple(tensor.shape), split_dim, tp_size): + return -1 + return put_full_tensor_with_unified_tp(store, key, tensor, tp_size, split_dim) + + +def make_mb_scale_original_tensor(dtype=torch.float32): + return make_deterministic_tensor((16, 16, 16), dtype=dtype) + + +def assert_tensor_matches_expected_shard(test_case, original_tensor, received_tensor, + split_dim, target_tp_size, target_rank, + context): + expected_tensor = expected_target_shard( + original_tensor, split_dim, target_tp_size, target_rank + ) + test_case.assertIsNotNone(received_tensor, f"missing tensor for {context}") + test_case.assertTrue( + torch.equal(received_tensor, expected_tensor), + f"mismatch for {context}", + ) + + def tearDownModule(): """Executed once after all tests in this file: closes the global connection.""" global GLOBAL_STORE @@ -174,6 +412,110 @@ class MooncakeTestBase(unittest.TestCase): # ========================================== class TestMooncakeFunctional(MooncakeTestBase): + def assert_full_reconstruction_case(self, key, tensor, split_dim, tp_size): + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertEqual(rc, 0, f"put_tensor_with_parallelism failed for {key}") + + shard_reads = [] + expected_shards = tensor.chunk(tp_size, split_dim) + for rank, expected_shard in enumerate(expected_shards): + wrapper_shard = self.store.get_tensor_with_tp( + key, tp_rank=rank, tp_size=tp_size, split_dim=split_dim + ) + self.assertIsNotNone(wrapper_shard, f"wrapper shard missing for rank {rank}") + self.assertTrue( + torch.equal(wrapper_shard, expected_shard), + f"wrapper shard mismatch for rank {rank}", + ) + + shard_parallelism = build_tp_parallelism(tp_size, split_dim, rank=rank) + shard_target = make_read_target("shard", shard_parallelism) + unified_shard = self.store.get_tensor_with_parallelism(key, shard_target) + self.assertIsNotNone(unified_shard, f"unified shard missing for rank {rank}") + self.assertTrue( + torch.equal(unified_shard, expected_shard), + f"unified shard mismatch for rank {rank}", + ) + self.assertTrue(torch.equal(unified_shard, wrapper_shard)) + shard_reads.append(unified_shard) + + reconstructed_from_shards = torch.cat(shard_reads, dim=split_dim) + self.assertTrue(torch.equal(reconstructed_from_shards, tensor)) + + full_parallelism = build_tp_parallelism(tp_size, split_dim, rank=min(tp_size - 1, 1)) + full_target = make_read_target("full", full_parallelism) + full_tensor = self.store.get_tensor_with_parallelism(key, full_target) + self.assertIsNotNone(full_tensor) + self.assertTrue(torch.equal(full_tensor, tensor)) + + def assert_full_into_reconstruction_case(self, key, tensor, split_dim, tp_size, buffer_spacing): + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + try: + parallelism = build_tp_parallelism(tp_size, split_dim, rank=min(tp_size - 1, 1)) + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertEqual(rc, 0) + + target = make_read_target("full", parallelism) + full_tensor = self.store.get_tensor_with_parallelism(key, target) + reconstructed = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=target + ) + self.assertIsNotNone(full_tensor) + self.assertIsNotNone(reconstructed) + self.assertTrue(torch.equal(full_tensor, tensor)) + self.assertTrue(torch.equal(reconstructed, tensor)) + self.assertTrue(torch.equal(reconstructed, full_tensor)) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + def assert_full_reconstruction_for_all_read_ranks(self, key, tensor, split_dim, tp_size): + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertEqual(rc, 0, f"put_tensor_with_parallelism failed for {key}") + + expected_rank_count = len(tensor.chunk(tp_size, split_dim)) + for rank in range(tp_size): + full_target = make_read_target("full", build_tp_parallelism(tp_size, split_dim, rank=rank)) + full_tensor = self.store.get_tensor_with_parallelism(key, full_target) + self.assertIsNotNone(full_tensor, f"full tensor missing for read rank {rank}") + self.assertTrue(torch.equal(full_tensor, tensor), f"full tensor mismatch for read rank {rank}") + + if rank < expected_rank_count: + shard_target = make_read_target("shard", build_tp_parallelism(tp_size, split_dim, rank=rank)) + shard_tensor = self.store.get_tensor_with_parallelism(key, shard_target) + expected_shard = chunk_tensor_for_rank(tensor, tp_size, split_dim, rank) + self.assertIsNotNone(shard_tensor, f"shard missing for read rank {rank}") + self.assertTrue(torch.equal(shard_tensor, expected_shard), f"shard mismatch for read rank {rank}") + + def assert_writer_shard_full_reconstruction_case(self, key, tensor, split_dim, shard_count): + for rank in range(shard_count): + rc = self.store.put_tensor_with_parallelism( + key, + tensor, + writer_partition=make_writer_partition(rank, shard_count, split_dim), + ) + self.assertEqual(rc, 0, f"writer shard put failed for rank {rank}") + + full_target = make_read_target("full") + full_tensor = self.store.get_tensor_with_parallelism(key, full_target) + self.assertIsNotNone(full_tensor) + self.assertTrue(torch.equal(full_tensor, tensor)) + + for rank in range(shard_count): + shard_target = make_read_target( + "shard", build_tp_parallelism(shard_count, split_dim, rank=rank) + ) + shard_tensor = self.store.get_tensor_with_parallelism(key, shard_target) + self.assertIsNotNone(shard_tensor) + self.assertTrue(torch.equal(shard_tensor, tensor.chunk(shard_count, split_dim)[rank])) + def test_01_basic_put_get(self): """Verify basic put and get functionality.""" key = "func_test_single" @@ -569,6 +911,1083 @@ class TestMooncakeFunctional(MooncakeTestBase): recon = torch.cat(reconstruction_parts, dim=split_dim) self.assertTrue(torch.equal(recon, original), f"Tensor {i} final reconstruction mismatch") + def test_12_unified_parallelism_as_stored_round_trip(self): + require_unified_parallelism_api(self) + key = "func_unified_as_stored" + tensor = torch.randn(16, 16, dtype=torch.float32) + + rc = self.store.put_tensor_with_parallelism(key, tensor) + self.assertEqual(rc, 0, f"put_tensor_with_parallelism failed with rc={rc}") + + retrieved_default = self.store.get_tensor_with_parallelism(key) + self.assertIsNotNone(retrieved_default) + self.assertTrue(torch.equal(retrieved_default, tensor)) + + target = make_read_target("as_stored") + retrieved_target = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNotNone(retrieved_target) + self.assertTrue(torch.equal(retrieved_target, tensor)) + + def test_13_unified_parallelism_tp_matches_wrapper(self): + require_unified_parallelism_api(self) + key = "func_unified_tp_single" + tensor = torch.arange(24, dtype=torch.float32).view(4, 6).contiguous() + tp_size = 3 + split_dim = 1 + + parallelism = make_tensor_parallelism([ + make_parallel_axis("tp", rank=0, size=tp_size, split_dim=split_dim) + ]) + + shard = chunk_tensor_for_rank(tensor, tp_size, split_dim, 0).contiguous() + rc = self.store.put_tensor_with_parallelism(key, shard, parallelism) + self.assertEqual(rc, 0, f"put_tensor_with_parallelism failed with rc={rc}") + + expected = shard + rank = 0 + wrapper_shard = self.store.get_tensor_with_tp( + key, tp_rank=rank, tp_size=tp_size, split_dim=split_dim + ) + self.assertIsNotNone(wrapper_shard) + self.assertTrue(torch.equal(wrapper_shard, expected)) + + shard_parallelism = make_tensor_parallelism([ + make_parallel_axis("tp", rank=rank, size=tp_size, split_dim=split_dim) + ]) + + target = make_read_target("shard", shard_parallelism) + unified_shard = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNotNone(unified_shard) + self.assertTrue(torch.equal(unified_shard, expected)) + self.assertTrue(torch.equal(unified_shard, wrapper_shard)) + + for rank in range(1, tp_size): + wrapper_shard = self.store.get_tensor_with_tp( + key, tp_rank=rank, tp_size=tp_size, split_dim=split_dim + ) + self.assertIsNone(wrapper_shard) + + shard_parallelism = make_tensor_parallelism([ + make_parallel_axis("tp", rank=rank, size=tp_size, split_dim=split_dim) + ]) + + target = make_read_target("shard", shard_parallelism) + unified_shard = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNone(unified_shard) + + def test_14_batch_unified_parallelism_tp_matches_wrapper(self): + require_unified_parallelism_api(self) + tp_size = 2 + split_dim = 0 + num_tensors = 2 + keys, tensors = generate_tensors(num_tensors, 1) + + results = batch_put_full_tensors_with_unified_tp( + self.store, keys, tensors, tp_size, split_dim + ) + self.assertTrue(all(r == 0 for r in results), f"Batch unified put failed. Results: {results}") + + parallelism = make_tensor_parallelism([ + make_parallel_axis("tp", rank=0, size=tp_size, split_dim=split_dim) + ]) + invalid_results = self.store.batch_put_tensor_with_parallelism( + keys, + tensors, + [parallelism for _ in keys], + writer_partitions=[make_writer_partition(0, 2, split_dim) for _ in keys], + ) + self.assertTrue(all(r != 0 for r in invalid_results), f"expected invalid combined routing request: {invalid_results}") + + for rank in range(tp_size): + wrapper_shards = self.store.batch_get_tensor_with_tp( + keys, tp_rank=rank, tp_size=tp_size + ) + + shard_parallelism = make_tensor_parallelism([ + make_parallel_axis("tp", rank=rank, size=tp_size, split_dim=split_dim) + ]) + + target = make_read_target("shard", shard_parallelism) + unified_shards = self.store.batch_get_tensor_with_parallelism( + keys, [target for _ in keys] + ) + + self.assertEqual(len(wrapper_shards), num_tensors) + self.assertEqual(len(unified_shards), num_tensors) + for i in range(num_tensors): + expected = tensors[i].chunk(tp_size, split_dim)[rank] + self.assertTrue(torch.equal(wrapper_shards[i], expected)) + self.assertTrue(torch.equal(unified_shards[i], expected)) + self.assertTrue(torch.equal(unified_shards[i], wrapper_shards[i])) + + def test_15_unified_parallelism_multi_axis_shard_round_trip(self): + require_unified_parallelism_api(self) + key = "func_unified_dp_tp_shard" + tensor = make_deterministic_tensor((4, 6)) + parallelism = build_dp_tp_parallelism( + dp_rank=1, dp_size=2, tp_rank=0, tp_size=3, split_dim=1 + ) + + rc = put_full_tensor_with_parallelism_template( + self.store, key, tensor, parallelism + ) + self.assertEqual(rc, 0) + + target = make_read_target("shard", parallelism) + result = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, tensor.chunk(3, 1)[0])) + + def test_16_unified_parallelism_multi_axis_into_round_trip(self): + require_unified_parallelism_api(self) + key = "func_unified_pp_tp_into" + tensor = make_deterministic_tensor((6, 8)) + parallelism = build_pp_tp_parallelism( + pp_rank=0, pp_size=2, stage_id=7, tp_rank=1, tp_size=2, split_dim=0 + ) + buffer_spacing = 1 * 1024 * 1024 + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + try: + rc = put_full_tensor_with_parallelism_template( + self.store, key, tensor, parallelism + ) + self.assertEqual(rc, 0) + target = make_read_target("shard", parallelism) + result = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=target + ) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, tensor.chunk(2, 0)[1])) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + def test_17_unified_parallelism_rejects_invalid_axis_fields(self): + require_unified_parallelism_api(self) + key = "func_unified_invalid_axis" + tensor = torch.randn(8, 8, dtype=torch.float32) + + invalid_dp = make_tensor_parallelism([ + make_parallel_axis("dp", rank=0, size=2, split_dim=0) + ]) + self.assertNotEqual( + self.store.put_tensor_with_parallelism(key, tensor, invalid_dp), 0 + ) + + invalid_ep = make_tensor_parallelism([ + make_parallel_axis("ep", rank=0, size=2) + ]) + self.assertNotEqual( + self.store.put_tensor_with_parallelism(key, tensor, invalid_ep), 0 + ) + + def test_18_batch_unified_parallelism_multi_axis_round_trip(self): + require_unified_parallelism_api(self) + cases = [ + ( + "func_unified_batch_dp_tp_0", + make_deterministic_tensor((4, 6)), + build_dp_tp_parallelism(0, 2, 1, 3, 1), + lambda tensor: tensor.chunk(3, 1)[1], + ), + ( + "func_unified_batch_pp_tp_1", + make_deterministic_tensor((6, 8)), + build_pp_tp_parallelism(1, 2, 9, 0, 2, 0), + lambda tensor: tensor.chunk(2, 0)[0], + ), + ] + keys = [key for key, _, _, _ in cases] + tensors = [tensor for _, tensor, _, _ in cases] + parallelisms = [parallelism for _, _, parallelism, _ in cases] + results = [ + put_full_tensor_with_parallelism_template( + self.store, key, tensor, parallelism + ) + for key, tensor, parallelism in zip(keys, tensors, parallelisms) + ] + self.assertTrue(all(r == 0 for r in results), f"batch put failed: {results}") + + targets = [make_read_target("shard", parallelism) for parallelism in parallelisms] + shard_reads = self.store.batch_get_tensor_with_parallelism(keys, targets) + self.assertEqual(len(shard_reads), len(cases)) + for i, (_, tensor, _, expected_fn) in enumerate(cases): + self.assertIsNotNone(shard_reads[i], f"batch shard missing for tensor {i}") + self.assertTrue(torch.equal(shard_reads[i], expected_fn(tensor))) + + def test_19_batch_unified_parallelism_multi_axis_into_round_trip(self): + require_unified_parallelism_api(self) + cases = [ + ( + "func_unified_batch_into_dp_tp_0", + make_deterministic_tensor((4, 9)), + build_dp_tp_parallelism(1, 2, 2, 3, 1), + lambda tensor: tensor.chunk(3, 1)[2], + ), + ( + "func_unified_batch_into_pp_tp_1", + make_deterministic_tensor((8, 6)), + build_pp_tp_parallelism(0, 2, 5, 1, 2, 0), + lambda tensor: tensor.chunk(2, 0)[1], + ), + ] + keys = [key for key, _, _, _ in cases] + tensors = [tensor for _, tensor, _, _ in cases] + parallelisms = [parallelism for _, _, parallelism, _ in cases] + results = [ + put_full_tensor_with_parallelism_template( + self.store, key, tensor, parallelism + ) + for key, tensor, parallelism in zip(keys, tensors, parallelisms) + ] + self.assertTrue(all(r == 0 for r in results), f"batch put failed: {results}") + + buffer_sizes = [max(serialized_tensor_size(tensor) + 4096, 1 * 1024 * 1024) for tensor in tensors] + total_buffer_size = sum(buffer_sizes) + backing_buffer = (ctypes.c_ubyte * total_buffer_size)() + backing_buffer_ptr = ctypes.addressof(backing_buffer) + buffer_ptrs = [] + offset = 0 + for size in buffer_sizes: + buffer_ptrs.append(backing_buffer_ptr + offset) + offset += size + + targets = [make_read_target("shard", parallelism) for parallelism in parallelisms] + self.assertEqual(self.store.register_buffer(backing_buffer_ptr, total_buffer_size), 0) + try: + shard_reads = self.store.batch_get_tensor_with_parallelism_into( + keys, buffer_ptrs, buffer_sizes, targets + ) + self.assertEqual(len(shard_reads), len(cases)) + for i, (_, tensor, _, expected_fn) in enumerate(cases): + self.assertIsNotNone(shard_reads[i], f"batch shard into missing for tensor {i}") + self.assertTrue(torch.equal(shard_reads[i], expected_fn(tensor))) + finally: + self.assertEqual(self.store.unregister_buffer(backing_buffer_ptr), 0) + + def test_20_unified_parallelism_full_reconstructs_tensor(self): + require_unified_parallelism_api(self) + key = "func_unified_tp_full" + tensor = torch.arange(32, dtype=torch.float32).view(4, 8).contiguous() + tp_size = 4 + split_dim = 1 + + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertEqual(rc, 0) + + target = make_read_target("full", build_tp_parallelism(tp_size, split_dim, rank=0)) + unified_full = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNotNone(unified_full) + self.assertTrue(torch.equal(unified_full, tensor)) + + def test_21_unified_parallelism_multi_axis_full_reconstructs_tensor(self): + require_unified_parallelism_api(self) + key = "func_unified_multi_axis_full" + tensor = make_deterministic_tensor((6, 8)) + parallelism = build_dp_tp_parallelism( + dp_rank=1, dp_size=2, tp_rank=0, tp_size=4, split_dim=1 + ) + + rc = put_full_tensor_with_parallelism_template( + self.store, key, tensor, parallelism + ) + self.assertEqual(rc, 0) + + full_target = make_read_target( + "full", + build_dp_tp_parallelism( + dp_rank=1, dp_size=2, tp_rank=2, tp_size=4, split_dim=1 + ), + ) + result = self.store.get_tensor_with_parallelism(key, full_target) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, tensor)) + + def test_22_unified_parallelism_multi_axis_full_into_reconstructs_tensor(self): + require_unified_parallelism_api(self) + key = "func_unified_multi_axis_full_into" + tensor = make_deterministic_tensor((8, 6)) + parallelism = build_pp_tp_parallelism( + pp_rank=0, pp_size=2, stage_id=4, tp_rank=0, tp_size=2, split_dim=0 + ) + buffer_spacing = 1 * 1024 * 1024 + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + try: + rc = put_full_tensor_with_parallelism_template( + self.store, key, tensor, parallelism + ) + self.assertEqual(rc, 0) + full_target = make_read_target( + "full", + build_pp_tp_parallelism( + pp_rank=0, pp_size=2, stage_id=4, tp_rank=1, tp_size=2, split_dim=0 + ), + ) + result = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=full_target + ) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, tensor)) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + def test_23_unified_parallelism_into_matches_wrapper(self): + require_unified_parallelism_api(self) + key = "func_unified_tp_into" + tensor = torch.randn(16, 16, dtype=torch.float32) + tp_size = 2 + split_dim = 0 + buffer_spacing = 1 * 1024 * 1024 + total_buffer_size = buffer_spacing * 2 + + large_buffer = (ctypes.c_ubyte * total_buffer_size)() + large_buffer_ptr = ctypes.addressof(large_buffer) + wrapper_ptr = large_buffer_ptr + unified_ptr = large_buffer_ptr + buffer_spacing + + self.assertEqual(self.store.register_buffer(large_buffer_ptr, total_buffer_size), 0) + try: + parallelism = make_tensor_parallelism([ + make_parallel_axis("tp", rank=0, size=tp_size, split_dim=split_dim) + ]) + shard = chunk_tensor_for_rank(tensor, tp_size, split_dim, 0).contiguous() + rc = self.store.put_tensor_with_parallelism(key, shard, parallelism) + self.assertEqual(rc, 0) + + target = make_read_target("shard", parallelism) + wrapper_tensor = self.store.get_tensor_with_tp_into( + key, wrapper_ptr, buffer_spacing, tp_rank=0, tp_size=tp_size, split_dim=split_dim + ) + unified_tensor = self.store.get_tensor_with_parallelism_into( + key, unified_ptr, buffer_spacing, target=target + ) + self.assertIsNotNone(wrapper_tensor) + self.assertIsNotNone(unified_tensor) + self.assertTrue(torch.equal(unified_tensor, wrapper_tensor)) + finally: + self.assertEqual(self.store.unregister_buffer(large_buffer_ptr), 0) + + def test_24_unified_parallelism_from_matches_wrapper(self): + require_unified_parallelism_api(self) + key = "func_unified_tp_from" + seed_key = "func_unified_tp_from_seed" + tensor = torch.randn(16, 16, dtype=torch.float32) + tp_size = 2 + split_dim = 1 + buffer_spacing = 1 * 1024 * 1024 + + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + try: + rc = self.store.put_tensor(seed_key, tensor) + self.assertEqual(rc, 0) + full_tensor = self.store.get_tensor_into(seed_key, buffer_ptr, buffer_spacing) + self.assertIsNotNone(full_tensor) + full_size = serialized_tensor_size(full_tensor) + + parallelism = make_tensor_parallelism([ + make_parallel_axis("tp", rank=0, size=tp_size, split_dim=split_dim) + ]) + rc = self.store.put_tensor_with_parallelism_from( + key, buffer_ptr, full_size, parallelism + ) + self.assertEqual(rc, 0) + + target = make_read_target("shard", parallelism) + unified_shard = self.store.get_tensor_with_parallelism(key, target) + wrapper_shard = self.store.get_tensor_with_tp( + key, tp_rank=0, tp_size=tp_size, split_dim=split_dim + ) + self.assertIsNotNone(unified_shard) + self.assertTrue(torch.equal(unified_shard, wrapper_shard)) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + def test_25_unified_parallelism_full_into_reconstructs_tensor(self): + require_unified_parallelism_api(self) + key = "func_unified_tp_full_into" + tensor = torch.arange(4 * 9 * 6, dtype=torch.float32).view(4, 9, 6).contiguous() + self.assert_full_into_reconstruction_case( + key=key, + tensor=tensor, + split_dim=1, + tp_size=3, + buffer_spacing=1 * 1024 * 1024, + ) + + def test_26_unified_parallelism_full_reconstruction_matrix(self): + require_unified_parallelism_api(self) + cases = [ + ((8, 12), 0, 4), + ((8, 12), 0, 8), + ((8, 12), 0, 16), + ((10, 12), 0, 4), + ((12, 9), 1, 3), + ((12, 9), 1, 9), + ((12, 9), 1, 16), + ((12, 10), 1, 4), + ((4, 10, 6), 1, 4), + ((4, 5, 7), 2, 8), + ] + + for index, (shape, split_dim, tp_size) in enumerate(cases): + with self.subTest(case=reconstruction_case_name(shape, split_dim, tp_size)): + tensor = make_deterministic_tensor(shape) + key = f"func_unified_tp_matrix_{index}" + if is_uniform_tp_case(shape, split_dim, tp_size): + self.assert_full_reconstruction_case( + key=key, + tensor=tensor, + split_dim=split_dim, + tp_size=tp_size, + ) + else: + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertNotEqual(rc, 0) + + def test_27_unified_parallelism_full_into_matrix(self): + require_unified_parallelism_api(self) + cases = [ + ((8, 12), 0, 16), + ((12, 10), 1, 4), + ((4, 10, 6), 1, 4), + ((4, 5, 7), 2, 8), + ] + + for index, (shape, split_dim, tp_size) in enumerate(cases): + with self.subTest(case=reconstruction_case_name(shape, split_dim, tp_size)): + tensor = make_deterministic_tensor(shape) + key = f"func_unified_tp_full_into_matrix_{index}" + if is_uniform_tp_case(shape, split_dim, tp_size): + buffer_spacing = max(serialized_tensor_size(tensor) + 4096, 1 * 1024 * 1024) + self.assert_full_into_reconstruction_case( + key=key, + tensor=tensor, + split_dim=split_dim, + tp_size=tp_size, + buffer_spacing=buffer_spacing, + ) + else: + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertNotEqual(rc, 0) + + def test_28_writer_shard_full_reconstruction(self): + require_unified_parallelism_api(self) + key = "func_writer_shard_full" + tensor = make_deterministic_tensor((6, 8)) + self.assert_writer_shard_full_reconstruction_case( + key=key, tensor=tensor, split_dim=1, shard_count=4 + ) + + def test_29_writer_shard_full_into_reconstruction(self): + require_unified_parallelism_api(self) + key = "func_writer_shard_full_into" + tensor = make_deterministic_tensor((8, 6)) + buffer_spacing = max(serialized_tensor_size(tensor) + 4096, 1 * 1024 * 1024) + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + try: + for rank in range(3): + rc = self.store.put_tensor_with_parallelism( + key, + tensor, + writer_partition=make_writer_partition(rank, 3, 1), + ) + self.assertEqual(rc, 0) + + result = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=make_read_target("full") + ) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, tensor)) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + def test_30_batch_writer_partition_full_reconstruction(self): + require_unified_parallelism_api(self) + tensors = [ + make_deterministic_tensor((8, 12)), + make_deterministic_tensor((6, 9, 4)), + make_deterministic_tensor((12, 12)), + ] + keys = [f"func_writer_partition_batch_{i}" for i in range(len(tensors))] + writer_partitions = [ + make_writer_partition(rank=0, size=4, split_dim=1), + make_writer_partition(rank=1, size=3, split_dim=0), + make_writer_partition(rank=2, size=4, split_dim=1), + ] + + for rank in range(4): + batch_writer_partitions = [ + make_writer_partition(rank=min(rank, part.size - 1), size=part.size, split_dim=part.split_dim) + for part in writer_partitions + ] + results = self.store.batch_put_tensor_with_parallelism( + keys, + tensors, + writer_partitions=batch_writer_partitions, + ) + self.assertTrue(all(r == 0 for r in results), f"batch put failed: {results}") + + full_targets = [make_read_target("full") for _ in tensors] + full_reads = self.store.batch_get_tensor_with_parallelism(keys, full_targets) + self.assertEqual(len(full_reads), len(tensors)) + for i, tensor in enumerate(tensors): + self.assertTrue(torch.equal(full_reads[i], tensor), f"full reconstruction mismatch for tensor {i}") + + def test_31_batch_writer_partition_into_round_trip(self): + require_unified_parallelism_api(self) + tensors = [ + make_deterministic_tensor((7, 9)), + make_deterministic_tensor((8, 6)), + ] + keys = [f"func_writer_partition_batch_into_{i}" for i in range(len(tensors))] + writer_layouts = [ + make_writer_partition(rank=0, size=3, split_dim=1), + make_writer_partition(rank=0, size=2, split_dim=0), + ] + for rank in range(3): + batch_writer_partitions = [ + make_writer_partition(rank=min(rank, layout.size - 1), size=layout.size, split_dim=layout.split_dim) + for layout in writer_layouts + ] + results = self.store.batch_put_tensor_with_parallelism( + keys, + tensors, + writer_partitions=batch_writer_partitions, + ) + self.assertTrue(all(r == 0 for r in results), f"batch put failed: {results}") + + buffer_sizes = [max(serialized_tensor_size(tensor) + 4096, 1 * 1024 * 1024) for tensor in tensors] + total_buffer_size = sum(buffer_sizes) + backing_buffer = (ctypes.c_ubyte * total_buffer_size)() + backing_buffer_ptr = ctypes.addressof(backing_buffer) + buffer_ptrs = [] + offset = 0 + for size in buffer_sizes: + buffer_ptrs.append(backing_buffer_ptr + offset) + offset += size + + self.assertEqual(self.store.register_buffer(backing_buffer_ptr, total_buffer_size), 0) + try: + full_reads = self.store.batch_get_tensor_with_parallelism_into( + keys, buffer_ptrs, buffer_sizes, [make_read_target("full") for _ in tensors] + ) + self.assertEqual(len(full_reads), len(tensors)) + for i, tensor in enumerate(tensors): + self.assertIsNotNone(full_reads[i], f"batch full into missing for tensor {i}") + self.assertTrue(torch.equal(full_reads[i], tensor), f"batch full into mismatch for tensor {i}") + finally: + self.assertEqual(self.store.unregister_buffer(backing_buffer_ptr), 0) + + def test_32_batch_unified_parallelism_full_reconstruction(self): + require_unified_parallelism_api(self) + tp_size = 4 + split_dim = 1 + tensors = [ + make_deterministic_tensor((8, 12)), + make_deterministic_tensor((6, 12, 4)), + make_deterministic_tensor((12, 12)), + ] + keys = [f"func_unified_tp_full_batch_{i}" for i in range(len(tensors))] + parallelisms = [build_tp_parallelism(tp_size, split_dim, rank=0) for _ in tensors] + results = batch_put_full_tensors_with_unified_tp(self.store, keys, tensors, tp_size, split_dim) + self.assertTrue(all(r == 0 for r in results), f"batch put failed: {results}") + + full_targets = [make_read_target("full", build_tp_parallelism(tp_size, split_dim, rank=1)) for _ in tensors] + full_reads = self.store.batch_get_tensor_with_parallelism(keys, full_targets) + self.assertEqual(len(full_reads), len(tensors)) + for i, tensor in enumerate(tensors): + self.assertTrue(torch.equal(full_reads[i], tensor), f"full reconstruction mismatch for tensor {i}") + + shard_targets = [make_read_target("shard", build_tp_parallelism(tp_size, split_dim, rank=2)) for _ in tensors] + shard_reads = self.store.batch_get_tensor_with_parallelism(keys, shard_targets) + self.assertEqual(len(shard_reads), len(tensors)) + for i, tensor in enumerate(tensors): + expected_shard = chunk_tensor_for_rank(tensor, tp_size, split_dim, 2) + self.assertTrue(torch.equal(shard_reads[i], expected_shard), f"shard mismatch for tensor {i}") + + def test_31_batch_unified_parallelism_multi_axis_full_reconstruction(self): + require_unified_parallelism_api(self) + cases = [ + ( + "func_unified_batch_multi_full_0", + make_deterministic_tensor((6, 8)), + build_dp_tp_parallelism(1, 2, 0, 4, 1), + build_dp_tp_parallelism(1, 2, 2, 4, 1), + ), + ( + "func_unified_batch_multi_full_1", + make_deterministic_tensor((8, 6)), + build_pp_tp_parallelism(0, 2, 6, 0, 2, 0), + build_pp_tp_parallelism(0, 2, 6, 1, 2, 0), + ), + ] + keys = [key for key, _, _, _ in cases] + tensors = [tensor for _, tensor, _, _ in cases] + write_parallelisms = [parallelism for _, _, parallelism, _ in cases] + results = [ + put_full_tensor_with_parallelism_template( + self.store, key, tensor, parallelism + ) + for key, tensor, parallelism in zip(keys, tensors, write_parallelisms) + ] + self.assertTrue(all(r == 0 for r in results), f"batch put failed: {results}") + + full_targets = [make_read_target("full", target_parallelism) for _, _, _, target_parallelism in cases] + full_reads = self.store.batch_get_tensor_with_parallelism(keys, full_targets) + self.assertEqual(len(full_reads), len(cases)) + for i, tensor in enumerate(tensors): + self.assertIsNotNone(full_reads[i], f"multi-axis full missing for tensor {i}") + self.assertTrue(torch.equal(full_reads[i], tensor), f"multi-axis full mismatch for tensor {i}") + + def test_32_unified_parallelism_multi_axis_full_remaps_axis_order(self): + require_unified_parallelism_api(self) + key = "func_unified_multi_axis_full_remap_order" + tensor = make_deterministic_tensor((6, 8)) + write_parallelism = build_dp_tp_parallelism( + dp_rank=1, dp_size=2, tp_rank=0, tp_size=4, split_dim=1 + ) + read_parallelism = reorder_parallelism_axes(write_parallelism, ["tp", "dp"]) + read_parallelism.axes[0].rank = 3 + + rc = put_full_tensor_with_parallelism_template( + self.store, key, tensor, write_parallelism + ) + self.assertEqual(rc, 0) + + full_target = make_read_target("full", read_parallelism) + result = self.store.get_tensor_with_parallelism(key, full_target) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, tensor)) + + def test_33_unified_parallelism_multi_axis_full_into_remaps_axis_order(self): + require_unified_parallelism_api(self) + key = "func_unified_multi_axis_full_into_remap_order" + tensor = make_deterministic_tensor((8, 6)) + write_parallelism = build_pp_tp_parallelism( + pp_rank=0, pp_size=2, stage_id=4, tp_rank=0, tp_size=2, split_dim=0 + ) + read_parallelism = reorder_parallelism_axes(write_parallelism, ["tp", "pp"]) + read_parallelism.axes[0].rank = 1 + buffer_spacing = 1 * 1024 * 1024 + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + try: + rc = put_full_tensor_with_parallelism_template( + self.store, key, tensor, write_parallelism + ) + self.assertEqual(rc, 0) + full_target = make_read_target("full", read_parallelism) + result = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=full_target + ) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, tensor)) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + def test_34_batch_unified_parallelism_multi_axis_full_remaps_axis_order(self): + require_unified_parallelism_api(self) + cases = [ + ( + "func_unified_batch_multi_full_remap_0", + make_deterministic_tensor((6, 8)), + build_dp_tp_parallelism(1, 2, 0, 4, 1), + reorder_parallelism_axes(build_dp_tp_parallelism(1, 2, 2, 4, 1), ["tp", "dp"]), + ), + ( + "func_unified_batch_multi_full_remap_1", + make_deterministic_tensor((8, 6)), + build_pp_tp_parallelism(0, 2, 6, 0, 2, 0), + reorder_parallelism_axes(build_pp_tp_parallelism(0, 2, 6, 1, 2, 0), ["tp", "pp"]), + ), + ] + keys = [key for key, _, _, _ in cases] + tensors = [tensor for _, tensor, _, _ in cases] + write_parallelisms = [parallelism for _, _, parallelism, _ in cases] + read_parallelisms = [parallelism for _, _, _, parallelism in cases] + results = [ + put_full_tensor_with_parallelism_template( + self.store, key, tensor, parallelism + ) + for key, tensor, parallelism in zip(keys, tensors, write_parallelisms) + ] + self.assertTrue(all(r == 0 for r in results), f"batch put failed: {results}") + + full_targets = [make_read_target("full", target_parallelism) for target_parallelism in read_parallelisms] + full_reads = self.store.batch_get_tensor_with_parallelism(keys, full_targets) + self.assertEqual(len(full_reads), len(cases)) + for i, tensor in enumerate(tensors): + self.assertIsNotNone(full_reads[i], f"multi-axis remap full missing for tensor {i}") + self.assertTrue(torch.equal(full_reads[i], tensor), f"multi-axis remap full mismatch for tensor {i}") + + def test_35_unified_parallelism_full_large_payload(self): + require_unified_parallelism_api(self) + key = "func_unified_tp_full_large" + tensor = make_deterministic_tensor((32, 32)) + split_dim = 1 + tp_size = 4 + buffer_spacing = serialized_tensor_size(tensor) + 1 * 1024 * 1024 + + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertEqual(rc, 0) + + full_target = make_read_target("full", build_tp_parallelism(tp_size, split_dim, rank=1)) + full_tensor = self.store.get_tensor_with_parallelism(key, full_target) + self.assertIsNotNone(full_tensor) + self.assertTrue(torch.equal(full_tensor, tensor)) + + shard_rank = 3 + shard_target = make_read_target("shard", build_tp_parallelism(tp_size, split_dim, rank=shard_rank)) + shard_tensor = self.store.get_tensor_with_parallelism(key, shard_target) + expected_shard = chunk_tensor_for_rank(tensor, tp_size, split_dim, shard_rank) + self.assertIsNotNone(shard_tensor) + self.assertTrue(torch.equal(shard_tensor, expected_shard)) + + self.assert_full_into_reconstruction_case( + key=f"{key}_into", + tensor=tensor, + split_dim=split_dim, + tp_size=tp_size, + buffer_spacing=buffer_spacing, + ) + + def test_36_unified_parallelism_full_reconstruction_edge_matrix(self): + require_unified_parallelism_api(self) + cases = [ + ((7,), 0, 8), + ((9,), 0, 4), + ((3, 5, 7), 2, 16), + ((5, 11, 3), 1, 8), + ] + + for index, (shape, split_dim, tp_size) in enumerate(cases): + with self.subTest(case=reconstruction_case_name(shape, split_dim, tp_size)): + tensor = make_deterministic_tensor(shape) + key = f"func_unified_tp_edge_matrix_{index}" + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertNotEqual(rc, 0) + + def test_37_batch_unified_parallelism_full_into_matrix(self): + require_unified_parallelism_api(self) + cases = [ + (make_deterministic_tensor((7,)), 0, 8), + (make_deterministic_tensor((8, 12)), 0, 16), + (make_deterministic_tensor((4, 5, 7)), 2, 8), + ] + for index, (tensor, split_dim, tp_size) in enumerate(cases): + with self.subTest(case=reconstruction_case_name(tuple(tensor.shape), split_dim, tp_size)): + rc = put_uniform_full_tensor_with_unified_tp( + self.store, + f"func_unified_tp_batch_full_into_{index}", + tensor, + tp_size, + split_dim, + ) + self.assertNotEqual(rc, 0) + + def test_38_batch_unified_parallelism_full_ragged_cases(self): + require_unified_parallelism_api(self) + cases = [ + (make_deterministic_tensor((7,)), 0, 8), + (make_deterministic_tensor((8, 12)), 0, 16), + (make_deterministic_tensor((11, 9)), 1, 4), + (make_deterministic_tensor((3, 5, 7)), 2, 16), + ] + for index, (tensor, split_dim, tp_size) in enumerate(cases): + with self.subTest(case=reconstruction_case_name(tuple(tensor.shape), split_dim, tp_size)): + rc = put_uniform_full_tensor_with_unified_tp( + self.store, + f"func_unified_tp_batch_ragged_{index}", + tensor, + tp_size, + split_dim, + ) + self.assertNotEqual(rc, 0) + + def test_39_unified_parallelism_full_dtype_smoke(self): + require_unified_parallelism_api(self) + cases = [ + (torch.float16, (8, 12), 1, 4), + (torch.bfloat16, (9,), 0, 16), + ] + + for index, (dtype, shape, split_dim, tp_size) in enumerate(cases): + with self.subTest(dtype=str(dtype), case=reconstruction_case_name(shape, split_dim, tp_size)): + tensor = make_deterministic_tensor(shape, dtype=dtype) + key = f"func_unified_tp_dtype_{index}" + if is_uniform_tp_case(shape, split_dim, tp_size): + self.assert_full_reconstruction_case( + key=key, + tensor=tensor, + split_dim=split_dim, + tp_size=tp_size, + ) + else: + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertNotEqual(rc, 0) + + def test_40_unified_parallelism_target_shard_matrix(self): + require_unified_parallelism_api(self) + cases = [ + ((8, 12, 6), 0, 4, 2, "tp"), + ((8, 12, 6), 1, 4, 2, "dp_tp"), + ((8, 12, 6), 2, 2, 4, "pp_tp"), + ((8, 12, 6), 1, 2, 4, "ep_tp"), + ((12, 8, 6), 0, 2, 2, "tp"), + ((12, 8, 8), 2, 4, 4, "dp_tp"), + ] + + for index, (shape, split_dim, put_tp_size, get_tp_size, mode) in enumerate(cases): + with self.subTest(shape=shape, split_dim=split_dim, put_tp_size=put_tp_size, get_tp_size=get_tp_size, mode=mode): + tensor = make_deterministic_tensor(shape) + key = f"func_unified_target_shard_matrix_{index}" + rc = put_full_tensor_with_parallelism_mode( + self.store, key, tensor, split_dim, put_tp_size, mode + ) + self.assertEqual(rc, 0) + + if not is_uniform_tp_case(shape, split_dim, get_tp_size): + target_parallelism = build_parallelism_mode( + mode, 0, get_tp_size, split_dim + ) + target = make_read_target("shard", target_parallelism) + shard = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNone(shard) + continue + + for target_rank in range(get_tp_size): + target_parallelism = build_parallelism_mode( + mode, target_rank, get_tp_size, split_dim + ) + target = make_read_target("shard", target_parallelism) + shard = self.store.get_tensor_with_parallelism(key, target) + expected = expected_target_shard( + tensor, split_dim, get_tp_size, target_rank + ) + self.assertIsNotNone(shard) + self.assertTrue( + torch.equal(shard, expected), + f"target shard mismatch for mode={mode}, rank={target_rank}", + ) + + def test_41_unified_parallelism_target_shard_into_matrix(self): + require_unified_parallelism_api(self) + cases = [ + ((8, 12, 6), 0, 4, 2, "tp"), + ((8, 12, 6), 1, 4, 2, "dp_tp"), + ((8, 12, 6), 2, 2, 4, "pp_tp"), + ((8, 12, 6), 1, 2, 4, "ep_tp"), + ] + + for index, (shape, split_dim, put_tp_size, get_tp_size, mode) in enumerate(cases): + with self.subTest(shape=shape, split_dim=split_dim, put_tp_size=put_tp_size, get_tp_size=get_tp_size, mode=mode): + tensor = make_deterministic_tensor(shape) + key = f"func_unified_target_shard_into_matrix_{index}" + rc = put_full_tensor_with_parallelism_mode( + self.store, key, tensor, split_dim, put_tp_size, mode + ) + self.assertEqual(rc, 0) + + if not is_uniform_tp_case(shape, split_dim, get_tp_size): + buffer_spacing = 1 * 1024 * 1024 + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + try: + target_parallelism = build_parallelism_mode( + mode, 0, get_tp_size, split_dim + ) + target = make_read_target("shard", target_parallelism) + shard = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=target + ) + self.assertIsNone(shard) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + continue + + for target_rank in range(get_tp_size): + expected = expected_target_shard( + tensor, split_dim, get_tp_size, target_rank + ) + buffer_spacing = max(serialized_tensor_size(expected) + 4096, 1 * 1024 * 1024) + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + try: + target_parallelism = build_parallelism_mode( + mode, target_rank, get_tp_size, split_dim + ) + target = make_read_target("shard", target_parallelism) + shard = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=target + ) + self.assertIsNotNone(shard) + self.assertTrue( + torch.equal(shard, expected), + f"target shard into mismatch for mode={mode}, rank={target_rank}", + ) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + def test_42_unified_parallelism_mb_scale_target_shard_matrix(self): + require_unified_parallelism_api(self) + original = make_mb_scale_original_tensor() + cases = [ + (0, 4, 2, "tp"), + (1, 4, 2, "dp_tp"), + (2, 4, 2, "pp_tp"), + (2, 4, 2, "ep_tp"), + ] + + for index, (split_dim, put_tp_size, get_tp_size, mode) in enumerate(cases): + key = f"func_unified_mb_target_shard_{index}" + with self.subTest(split_dim=split_dim, put_tp_size=put_tp_size, + get_tp_size=get_tp_size, mode=mode): + rc = put_full_tensor_with_parallelism_mode( + self.store, key, original, split_dim, put_tp_size, mode + ) + self.assertEqual(rc, 0) + for target_rank in range(get_tp_size): + target_parallelism = build_parallelism_mode( + mode, target_rank, get_tp_size, split_dim + ) + target = make_read_target("shard", target_parallelism) + shard = self.store.get_tensor_with_parallelism(key, target) + assert_tensor_matches_expected_shard( + self, original, shard, split_dim, get_tp_size, target_rank, + context=( + f"mb shard read mode={mode} split_dim={split_dim} " + f"target_rank={target_rank}" + ), + ) + + def test_43_unified_parallelism_mb_scale_target_shard_into_matrix(self): + require_unified_parallelism_api(self) + original = make_mb_scale_original_tensor() + cases = [ + (0, 4, 2, "tp"), + (1, 4, 2, "dp_tp"), + (2, 4, 2, "pp_tp"), + (2, 4, 2, "ep_tp"), + ] + + for index, (split_dim, put_tp_size, get_tp_size, mode) in enumerate(cases): + key = f"func_unified_mb_target_shard_into_{index}" + with self.subTest(split_dim=split_dim, put_tp_size=put_tp_size, + get_tp_size=get_tp_size, mode=mode): + rc = put_full_tensor_with_parallelism_mode( + self.store, key, original, split_dim, put_tp_size, mode + ) + self.assertEqual(rc, 0) + for target_rank in range(get_tp_size): + expected = expected_target_shard( + original, split_dim, get_tp_size, target_rank + ) + buffer_spacing = max( + serialized_tensor_size(expected) + 4096, + 1 * 1024 * 1024, + ) + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual( + self.store.register_buffer(buffer_ptr, buffer_spacing), 0 + ) + try: + target_parallelism = build_parallelism_mode( + mode, target_rank, get_tp_size, split_dim + ) + target = make_read_target("shard", target_parallelism) + shard = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=target + ) + assert_tensor_matches_expected_shard( + self, original, shard, split_dim, get_tp_size, + target_rank, + context=( + f"mb shard into mode={mode} split_dim={split_dim} " + f"target_rank={target_rank}" + ), + ) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + def test_44_unified_parallelism_concurrency_relation_matrix(self): + require_unified_parallelism_api(self) + original = make_mb_scale_original_tensor() + scenarios = [ + (2, 4, [(0, 4, 2, "tp"), (1, 4, 2, "dp_tp"), (2, 4, 2, "pp_tp")]), + (3, 3, [(0, 4, 2, "tp"), (1, 4, 2, "dp_tp"), (2, 4, 2, "ep_tp")]), + (6, 3, [(0, 4, 2, "tp"), (1, 4, 2, "pp_tp"), (2, 4, 2, "ep_tp")]), + ] + + def put_worker(key, split_dim, put_tp_size, mode): + return put_full_tensor_with_parallelism_mode( + self.store, key, original, split_dim, put_tp_size, mode + ) + + def get_worker(key, split_dim, get_tp_size, mode): + for target_rank in range(get_tp_size): + target_parallelism = build_parallelism_mode( + mode, target_rank, get_tp_size, split_dim + ) + target = make_read_target("shard", target_parallelism) + shard = self.store.get_tensor_with_parallelism(key, target) + expected = expected_target_shard( + original, split_dim, get_tp_size, target_rank + ) + if shard is None or not torch.equal(shard, expected): + return ( + f"concurrency shard mismatch key={key} split_dim={split_dim} " + f"mode={mode} rank={target_rank}" + ) + return None + + for scenario_index, (put_workers, get_workers, cases) in enumerate(scenarios): + with self.subTest(put_workers=put_workers, get_workers=get_workers): + keys = [ + f"func_unified_concurrency_{scenario_index}_{case_index}" + for case_index in range(len(cases)) + ] + with concurrent.futures.ThreadPoolExecutor(max_workers=max(put_workers, get_workers)) as executor: + put_futures = [ + executor.submit(put_worker, key, split_dim, put_tp_size, mode) + for key, (split_dim, put_tp_size, _get_tp_size, mode) in zip(keys, cases) + for _ in range(max(1, put_workers // len(cases))) + ] + for future in concurrent.futures.as_completed(put_futures): + self.assertEqual(future.result(), 0) + + get_futures = [ + executor.submit(get_worker, key, split_dim, get_tp_size, mode) + for key, (split_dim, _put_tp_size, get_tp_size, mode) in zip(keys, cases) + for _ in range(max(1, get_workers // len(cases))) + ] + for future in concurrent.futures.as_completed(get_futures): + error = future.result() + if error: + self.fail(error) + # ========================================== # Performance/Benchmark Tests # ========================================== @@ -579,6 +1998,26 @@ class TestMooncakeBenchmark(MooncakeTestBase): TENSOR_SIZE_MB = 16 TOTAL_SIZE_MB = 256 + def _run_unified_full_manual_gather(self, key, tensor, split_dim, tp_size): + shards = [] + for rank in range(tp_size): + shard = self.store.get_tensor_with_tp( + key, tp_rank=rank, tp_size=tp_size, split_dim=split_dim + ) + self.assertIsNotNone(shard) + shards.append(shard) + reconstructed = torch.cat(shards, dim=split_dim) + self.assertTrue(torch.equal(reconstructed, tensor)) + return reconstructed + + def _run_unified_full_into(self, key, target, buffer_ptr, buffer_spacing, tensor): + reconstructed = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target + ) + self.assertIsNotNone(reconstructed) + self.assertTrue(torch.equal(reconstructed, tensor)) + return reconstructed + def setUp(self): """Benchmark-specific setUp.""" # 1. Call parent setUp to clean the store (remove_all) @@ -829,6 +2268,68 @@ class TestMooncakeBenchmark(MooncakeTestBase): self._print_perf(f"TP Batch Pub (TP={tp_size})", put_times) self._print_perf(f"TP Batch Get (TP={tp_size})", get_times) + def test_benchmark_07_unified_full_reconstruction(self): + require_unified_parallelism_api(self) + key = "bench_unified_full" + tensor = make_deterministic_tensor((4, 8)) + tp_size = 4 + split_dim = 1 + full_target = make_read_target("full", build_tp_parallelism(tp_size, split_dim, rank=1)) + manual_times = [] + unified_times = [] + + print(f"--- Running Unified Full Reconstruction Benchmark (TP={tp_size}) ---") + for _ in range(self.BENCH_ITERATIONS): + self.store.remove_all() + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertEqual(rc, 0) + + t0 = time.perf_counter() + unified_tensor = self.store.get_tensor_with_parallelism(key, full_target) + unified_times.append(time.perf_counter() - t0) + self.assertIsNotNone(unified_tensor) + self.assertTrue(torch.equal(unified_tensor, tensor)) + + t0 = time.perf_counter() + self._run_unified_full_manual_gather(key, tensor, split_dim, tp_size) + manual_times.append(time.perf_counter() - t0) + + self.total_bits = tensor.numel() * tensor.element_size() * 8 + self._print_perf("Unified Full Get", unified_times) + self._print_perf("Manual TP Gather", manual_times) + + def test_benchmark_08_unified_full_into(self): + require_unified_parallelism_api(self) + key = "bench_unified_full_into" + tensor = self.tensors[0] + tp_size = 4 + split_dim = 0 + full_target = make_read_target("full", build_tp_parallelism(tp_size, split_dim, rank=2)) + buffer_spacing = serialized_tensor_size(tensor) + 1 * 1024 * 1024 + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + self.assertEqual(self.store.register_buffer(buffer_ptr, buffer_spacing), 0) + into_times = [] + try: + print(f"--- Running Unified Full Into Benchmark (TP={tp_size}) ---") + for _ in range(self.BENCH_ITERATIONS): + self.store.remove_all() + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + self.assertEqual(rc, 0) + + t0 = time.perf_counter() + self._run_unified_full_into(key, full_target, buffer_ptr, buffer_spacing, tensor) + into_times.append(time.perf_counter() - t0) + finally: + self.assertEqual(self.store.unregister_buffer(buffer_ptr), 0) + + self.total_bits = tensor.numel() * tensor.element_size() * 8 + self._print_perf("Unified Full Into", into_times) + # ========================================== # Stress/Concurrency Tests # ========================================== @@ -841,6 +2342,51 @@ class TestMooncakeStress(MooncakeTestBase): TOTAL_ITEMS = 800 # Total number of items to process across all threads TENSOR_SIZE_MB = 4 # Size per tensor + def _run_unified_full_worker(self, worker_id, split_dim, tp_size, use_into=False, mixed_reads=False): + key = f"stress_unified_full_{worker_id}" + if split_dim == 0: + tensor = make_deterministic_tensor((8, 8)) + elif split_dim == 1: + tensor = make_deterministic_tensor((4, 8)) + else: + tensor = make_deterministic_tensor((2, 2, 8)) + + rc = put_uniform_full_tensor_with_unified_tp( + self.store, key, tensor, tp_size, split_dim + ) + if rc != 0: + return f"put failed for {key}, rc={rc}" + + full_target = make_read_target("full", build_tp_parallelism(tp_size, split_dim, rank=min(tp_size - 1, 1))) + full_tensor = self.store.get_tensor_with_parallelism(key, full_target) + if full_tensor is None or not torch.equal(full_tensor, tensor): + return f"full reconstruction mismatch for {key}, split_dim={split_dim}, tp_size={tp_size}" + + if mixed_reads: + shard_rank = min(tp_size - 1, 2) + shard_target = make_read_target("shard", build_tp_parallelism(tp_size, split_dim, rank=shard_rank)) + shard_tensor = self.store.get_tensor_with_parallelism(key, shard_target) + expected_shard = chunk_tensor_for_rank(tensor, tp_size, split_dim, shard_rank) + if shard_tensor is None or not torch.equal(shard_tensor, expected_shard): + return f"shard mismatch for {key}, split_dim={split_dim}, tp_size={tp_size}, rank={shard_rank}" + + if use_into: + buffer_spacing = serialized_tensor_size(tensor) + 64 * 1024 + buffer = (ctypes.c_ubyte * buffer_spacing)() + buffer_ptr = ctypes.addressof(buffer) + if self.store.register_buffer(buffer_ptr, buffer_spacing) != 0: + return f"register_buffer failed for {key}" + try: + into_tensor = self.store.get_tensor_with_parallelism_into( + key, buffer_ptr, buffer_spacing, target=full_target + ) + if into_tensor is None or not torch.equal(into_tensor, tensor): + return f"full into mismatch for {key}, split_dim={split_dim}, tp_size={tp_size}" + finally: + self.store.unregister_buffer(buffer_ptr) + + return None + def _run_stress_worker(self, thread_id, items_per_thread): """ Worker function: @@ -889,6 +2435,10 @@ class TestMooncakeStress(MooncakeTestBase): if not torch.equal(original_tensor, retrieved_tensor): raise RuntimeError(f"Data Mismatch for {key}!") + remove_rc = self.store.remove(key, True) + if remove_rc != 0: + raise RuntimeError(f"Remove failed for {key}, rc={remove_rc}") + ops_count += 1 except Exception as e: @@ -945,6 +2495,31 @@ class TestMooncakeStress(MooncakeTestBase): print(f"Avg QPS: {total_ops / elapsed:.2f} ops/s") print(f"Avg Goodput: {throughput_gbps:.2f} Gbps") + def test_stress_unified_parallelism_full_reads(self): + require_unified_parallelism_api(self) + worker_cases = [ + (0, 0, 2, False, False), + (1, 1, 2, False, True), + (2, 0, 4, True, False), + (3, 1, 4, True, True), + (4, 2, 4, False, True), + (5, 2, 4, True, True), + ] + errors = [] + + with concurrent.futures.ThreadPoolExecutor(max_workers=len(worker_cases)) as executor: + futures = [ + executor.submit(self._run_unified_full_worker, *case) + for case in worker_cases + ] + for future in concurrent.futures.as_completed(futures): + error = future.result() + if error: + errors.append(error) + + if errors: + self.fail(errors[0]) + # ========================================== # Data Type & Precision Tests (Full Enum) diff --git a/scripts/test_upsert_api.py b/scripts/test_upsert_api.py index fbcb10ea..11aa2abc 100644 --- a/scripts/test_upsert_api.py +++ b/scripts/test_upsert_api.py @@ -31,9 +31,11 @@ import os import sys import time import unittest +from dataclasses import dataclass import torch import numpy as np +import mooncake.store as mooncake_store from mooncake.store import MooncakeDistributedStore, ReplicateConfig @@ -41,12 +43,13 @@ from mooncake.store import MooncakeDistributedStore, ReplicateConfig # Constants # ========================================== -# Must match C++ TensorMetadata layout -TENSOR_METADATA_SIZE = 4 + 4 + 8 * 4 # 40 bytes +# Must match current C++ TensorMetadata layout. +# TensorObjectHeader (40) + TensorLayoutMetadata (264) = 304 bytes. +TENSOR_METADATA_SIZE = 304 def serialized_tensor_size(tensor): - """Size of [TensorMetadata][tensor data] as stored by get_tensor_into.""" + """Size of [TensorObjectHeader+layout metadata][tensor data].""" return TENSOR_METADATA_SIZE + tensor.numel() * tensor.element_size() @@ -94,6 +97,51 @@ def tearDownModule(): GLOBAL_STORE = None +def require_unified_parallelism_api(test_case): + missing = [ + name for name in ["ParallelAxis", "TensorParallelism", "ReadTarget"] + if not hasattr(mooncake_store, name) + ] + if missing: + test_case.skipTest( + "Unified parallelism Python bindings are not available in the installed extension: " + + ", ".join(missing) + ) + + +def make_parallel_axis(kind, rank, size, split_dim=None): + axis = mooncake_store.ParallelAxis() + axis.kind = kind + axis.rank = rank + axis.size = size + axis.split_dim = split_dim + return axis + + +def make_tensor_parallelism(axes): + parallelism = mooncake_store.TensorParallelism() + parallelism.axes = axes + return parallelism + + +def make_read_target(mode, parallelism=None): + target = mooncake_store.ReadTarget() + target.mode = mode + target.parallelism = parallelism + return target + + +@dataclass +class WriterPartition: + rank: int + size: int + split_dim: int + + +def make_writer_partition(rank, size, split_dim): + return WriterPartition(rank=rank, size=size, split_dim=split_dim) + + class UpsertTestBase(unittest.TestCase): """Base class that provides self.store and cleans up before each test.""" @@ -466,6 +514,555 @@ class TestUpsertPubTensor(UpsertTestBase): self.assertTrue(torch.equal(t2, got)) +class TestUnifiedParallelismUpsert(UpsertTestBase): + def test_put_tensor_with_parallelism_rejects_nonuniform_multi_axis(self): + require_unified_parallelism_api(self) + key = "unified_nonuniform_multi_axis_reject" + tensor = torch.arange(6, dtype=torch.float32).view(2, 3).contiguous() + parallelism = make_tensor_parallelism([ + make_parallel_axis("dp", rank=0, size=2), + make_parallel_axis("tp", rank=0, size=2, split_dim=1), + ]) + + rc = self.store.put_tensor_with_parallelism(key, tensor, parallelism) + self.assertNotEqual(rc, 0) + self.assertIsNone( + self.store.get_tensor_with_parallelism(key, make_read_target("shard", parallelism)) + ) + + def test_put_tensor_with_parallelism_from_rejects_nonuniform_multi_axis(self): + require_unified_parallelism_api(self) + seed_key = "unified_nonuniform_multi_axis_from_seed" + key = "unified_nonuniform_multi_axis_from_reject" + tensor = torch.arange(6, dtype=torch.float32).view(2, 3).contiguous() + self.assertEqual(self.store.put_tensor(seed_key, tensor), 0) + + buffer_size = serialized_tensor_size(tensor) + buf = ctypes.create_string_buffer(buffer_size) + ptr = ctypes.addressof(buf) + self.assertEqual(self.store.register_buffer(ptr, buffer_size), 0) + try: + got = self.store.get_tensor_into(seed_key, ptr, buffer_size) + self.assertIsNotNone(got) + parallelism = make_tensor_parallelism([ + make_parallel_axis("dp", rank=1, size=2), + make_parallel_axis("tp", rank=1, size=2, split_dim=1), + ]) + rc = self.store.put_tensor_with_parallelism_from( + key, ptr, buffer_size, parallelism + ) + self.assertNotEqual(rc, 0) + self.assertIsNone( + self.store.get_tensor_with_parallelism( + key, make_read_target("shard", parallelism) + ) + ) + finally: + self.store.unregister_buffer(ptr) + + def test_put_tensor_with_writer_partition_rejects_nonuniform(self): + require_unified_parallelism_api(self) + key = "writer_partition_nonuniform_reject" + tensor = torch.arange(6, dtype=torch.float32).view(2, 3).contiguous() + + rc = self.store.put_tensor_with_parallelism( + key, + tensor, + writer_partition=make_writer_partition(rank=0, size=2, split_dim=1), + ) + self.assertNotEqual(rc, 0) + self.assertIsNone(self.store.get_tensor_with_parallelism(key, make_read_target("full"))) + + def test_batch_put_tensor_with_parallelism_rejects_nonuniform_and_preserves_uniform_data(self): + require_unified_parallelism_api(self) + keys = ["uniform_batch_ok", "nonuniform_batch_reject"] + tensors = [ + torch.arange(8, dtype=torch.float32).view(2, 4).contiguous(), + torch.arange(6, dtype=torch.float32).view(2, 3).contiguous(), + ] + parallelisms = [ + make_tensor_parallelism([ + make_parallel_axis("dp", rank=0, size=2), + make_parallel_axis("tp", rank=1, size=2, split_dim=1), + ]), + make_tensor_parallelism([ + make_parallel_axis("dp", rank=1, size=2), + make_parallel_axis("tp", rank=0, size=4, split_dim=1), + ]), + ] + + results = list( + self.store.batch_put_tensor_with_parallelism(keys, tensors, parallelisms) + ) + self.assertEqual(results[0], 0) + self.assertNotEqual(results[1], 0) + + got_uniform = self.store.get_tensor_with_parallelism( + keys[0], make_read_target("shard", parallelisms[0]) + ) + self.assertIsNotNone(got_uniform) + self.assertTrue(torch.equal(got_uniform, tensors[0].chunk(2, 1)[1])) + self.assertIsNone( + self.store.get_tensor_with_parallelism( + keys[1], make_read_target("shard", parallelisms[1]) + ) + ) + + def test_upsert_tensor_with_parallelism_multi_axis(self): + require_unified_parallelism_api(self) + key = "unified_multi_axis_upsert" + original = torch.arange(8, dtype=torch.float32).view(2, 4).contiguous() + updated = (original + 50).contiguous() + + dp_axis = make_parallel_axis("dp", rank=1, size=2) + tp_axis = make_parallel_axis("tp", rank=0, size=2, split_dim=1) + parallelism = make_tensor_parallelism([dp_axis, tp_axis]) + + for rank in range(2): + rank_parallelism = clone_parallelism(parallelism) + rank_parallelism.axes[1].rank = rank + self.assertEqual( + self.store.put_tensor_with_parallelism(key, original, rank_parallelism), + 0, + ) + self.assertEqual( + self.store.upsert_tensor_with_parallelism(key, updated, rank_parallelism), + 0, + ) + + target = make_read_target("shard", parallelism) + got = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNotNone(got) + self.assertTrue(torch.equal(got, updated.chunk(2, 1)[0])) + + def test_batch_upsert_tensor_with_parallelism_multi_axis(self): + require_unified_parallelism_api(self) + keys = ["unified_multi_axis_batch_upsert_0", "unified_multi_axis_batch_upsert_1"] + originals = [ + torch.arange(8, dtype=torch.float32).view(2, 4).contiguous(), + torch.arange(8, 16, dtype=torch.float32).view(4, 2).contiguous(), + ] + updates = [(tensor + 100).contiguous() for tensor in originals] + + base_parallelisms = [] + dp_tp = make_tensor_parallelism([ + make_parallel_axis("dp", rank=1, size=2), + make_parallel_axis("tp", rank=1, size=2, split_dim=1), + ]) + pp_tp = make_tensor_parallelism([ + make_parallel_axis("pp", rank=0, size=2), + make_parallel_axis("tp", rank=1, size=2, split_dim=0), + ]) + pp_tp.axes[0].stage_id = 3 + base_parallelisms.extend([dp_tp, pp_tp]) + + rank_parallelisms = [] + for base_parallelism in base_parallelisms: + tp_axis = next(axis for axis in base_parallelism.axes if axis.kind == "tp") + for rank in range(tp_axis.size): + parallelism = clone_parallelism(base_parallelism) + for axis in parallelism.axes: + if axis.kind == "tp": + axis.rank = rank + break + rank_parallelisms.append(parallelism) + batch_keys = [keys[0]] * 2 + [keys[1]] * 2 + batch_originals = [originals[0]] * 2 + [originals[1]] * 2 + batch_updates = [updates[0]] * 2 + [updates[1]] * 2 + + self.assertEqual( + list(self.store.batch_put_tensor_with_parallelism(batch_keys, batch_originals, rank_parallelisms)), + [0, 0, 0, 0], + ) + self.assertEqual( + list(self.store.batch_upsert_tensor_with_parallelism(batch_keys, batch_updates, rank_parallelisms)), + [0, 0, 0, 0], + ) + invalid_results = list( + self.store.batch_put_tensor_with_parallelism( + keys, + originals, + parallelisms=base_parallelisms, + writer_partitions=[ + make_writer_partition(0, 2, 0), + make_writer_partition(1, 2, 0), + ], + ) + ) + self.assertTrue( + all(result != 0 for result in invalid_results), + f"expected invalid combined routing request: {invalid_results}", + ) + + targets = [] + for parallelism in base_parallelisms: + target = mooncake_store.ReadTarget() + target.mode = "shard" + target.parallelism = parallelism + targets.append(target) + results = self.store.batch_get_tensor_with_parallelism(keys, targets) + self.assertEqual(len(results), 2) + self.assertTrue(torch.equal(results[0], updates[0].chunk(2, 1)[1])) + self.assertTrue(torch.equal(results[1], updates[1].chunk(2, 0)[1])) + + def test_upsert_tensor_with_parallelism_from_multi_axis(self): + require_unified_parallelism_api(self) + key = "unified_multi_axis_upsert_from" + seed_key = "unified_multi_axis_upsert_from_seed" + tensor = torch.arange(8, dtype=torch.float32).view(2, 4).contiguous() + updated = (tensor + 77).contiguous() + + self.assertEqual(self.store.put_tensor(seed_key, updated), 0) + buffer_size = serialized_tensor_size(updated) + buf = ctypes.create_string_buffer(buffer_size) + ptr = ctypes.addressof(buf) + self.assertEqual(self.store.register_buffer(ptr, buffer_size), 0) + try: + got = self.store.get_tensor_into(seed_key, ptr, buffer_size) + self.assertIsNotNone(got) + + dp_axis = make_parallel_axis("dp", rank=0, size=2) + tp_axis = make_parallel_axis("tp", rank=1, size=2, split_dim=1) + parallelism = make_tensor_parallelism([dp_axis, tp_axis]) + + for rank in range(2): + rank_parallelism = clone_parallelism(parallelism) + rank_parallelism.axes[1].rank = rank + self.assertEqual( + self.store.upsert_tensor_with_parallelism_from( + key, ptr, buffer_size, rank_parallelism + ), + 0, + ) + + target = mooncake_store.ReadTarget() + target.mode = "shard" + target.parallelism = parallelism + result = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, updated.chunk(2, 1)[1])) + finally: + self.store.unregister_buffer(ptr) + + def test_upsert_tensor_with_writer_partition(self): + require_unified_parallelism_api(self) + key = "writer_partition_upsert" + original = torch.arange(8, dtype=torch.float32).view(2, 4).contiguous() + updated = (original + 33).contiguous() + + for rank in range(2): + self.assertEqual( + self.store.put_tensor_with_parallelism( + key, + original, + writer_partition=make_writer_partition(rank, 2, 1), + ), + 0, + ) + for rank in range(2): + self.assertEqual( + self.store.upsert_tensor_with_parallelism( + key, + updated, + writer_partition=make_writer_partition(rank, 2, 1), + ), + 0, + ) + + target = make_read_target("full") + got = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNotNone(got) + self.assertTrue(torch.equal(got, updated)) + + def test_upsert_tensor_with_writer_partition_from(self): + require_unified_parallelism_api(self) + key = "writer_partition_upsert_from" + seed_key = "writer_partition_upsert_from_seed" + tensor = torch.arange(16, dtype=torch.float32).view(4, 4).contiguous() + + self.assertEqual(self.store.put_tensor(seed_key, tensor), 0) + for rank in range(4): + self.assertEqual( + self.store.put_tensor_with_parallelism( + key, + tensor, + writer_partition=make_writer_partition(rank, 4, 0), + ), + 0, + ) + buffer_size = serialized_tensor_size(tensor) + buf = ctypes.create_string_buffer(buffer_size) + ptr = ctypes.addressof(buf) + self.assertEqual(self.store.register_buffer(ptr, buffer_size), 0) + try: + got = self.store.get_tensor_into(seed_key, ptr, buffer_size) + self.assertIsNotNone(got) + + for rank in range(4): + self.assertEqual( + self.store.upsert_tensor_with_parallelism_from( + key, + ptr, + buffer_size, + writer_partition=make_writer_partition(rank, 4, 0), + ), + 0, + ) + + target = mooncake_store.ReadTarget() + target.mode = "full" + target.parallelism = None + result = self.store.get_tensor_with_parallelism(key, target) + self.assertIsNotNone(result) + self.assertTrue(torch.equal(result, tensor)) + finally: + self.store.unregister_buffer(ptr) + + def test_batch_upsert_tensor_with_writer_partitions(self): + require_unified_parallelism_api(self) + keys = ["writer_partition_batch_upsert_0", "writer_partition_batch_upsert_1"] + originals = [ + torch.arange(8, dtype=torch.float32).view(2, 4).contiguous(), + torch.arange(8, 16, dtype=torch.float32).view(2, 4).contiguous(), + ] + updates = [(tensor + 100).contiguous() for tensor in originals] + + for rank in range(2): + writer_partitions = [ + make_writer_partition(rank=rank, size=2, split_dim=1), + make_writer_partition(rank=rank, size=2, split_dim=1), + ] + self.assertEqual( + list( + self.store.batch_put_tensor_with_parallelism( + keys, + originals, + writer_partitions=writer_partitions, + ) + ), + [0, 0], + ) + self.assertEqual( + list( + self.store.batch_upsert_tensor_with_parallelism( + keys, + updates, + writer_partitions=writer_partitions, + ) + ), + [0, 0], + ) + + results = self.store.batch_get_tensor_with_parallelism( + keys, [make_read_target("full"), make_read_target("full")] + ) + self.assertEqual(len(results), 2) + self.assertTrue(torch.equal(results[0], updates[0])) + self.assertTrue(torch.equal(results[1], updates[1])) + + def test_batch_upsert_tensor_with_writer_partitions_from(self): + require_unified_parallelism_api(self) + keys = ["writer_partition_batch_upsert_from_0", "writer_partition_batch_upsert_from_1"] + seed_keys = [f"{key}_seed" for key in keys] + tensors = [ + torch.arange(16, dtype=torch.float32).view(4, 4).contiguous(), + torch.arange(16, 32, dtype=torch.float32).view(4, 4).contiguous(), + ] + + for seed_key, tensor in zip(seed_keys, tensors): + self.assertEqual(self.store.put_tensor(seed_key, tensor), 0) + + for rank in range(4): + writer_partitions = [ + make_writer_partition(rank=rank, size=4, split_dim=0), + make_writer_partition(rank=rank, size=4, split_dim=0), + ] + self.assertEqual( + list( + self.store.batch_put_tensor_with_parallelism( + keys, + tensors, + writer_partitions=writer_partitions, + ) + ), + [0, 0], + ) + + buffer_sizes = [serialized_tensor_size(tensor) for tensor in tensors] + backing_buffer = ctypes.create_string_buffer(sum(buffer_sizes)) + base_ptr = ctypes.addressof(backing_buffer) + buffer_ptrs = [] + offset = 0 + for size in buffer_sizes: + buffer_ptrs.append(base_ptr + offset) + offset += size + self.assertEqual(self.store.register_buffer(base_ptr, len(backing_buffer)), 0) + try: + for seed_key, ptr, size in zip(seed_keys, buffer_ptrs, buffer_sizes): + got = self.store.get_tensor_into(seed_key, ptr, size) + self.assertIsNotNone(got) + + for rank in range(4): + writer_partitions = [ + make_writer_partition(rank=rank, size=4, split_dim=0), + make_writer_partition(rank=rank, size=4, split_dim=0), + ] + self.assertEqual( + list( + self.store.batch_upsert_tensor_with_parallelism_from( + keys, + buffer_ptrs, + buffer_sizes, + writer_partitions=writer_partitions, + ) + ), + [0, 0], + ) + + results = self.store.batch_get_tensor_with_parallelism( + keys, [make_read_target("full"), make_read_target("full")] + ) + self.assertEqual(len(results), 2) + self.assertTrue(torch.equal(results[0], tensors[0])) + self.assertTrue(torch.equal(results[1], tensors[1])) + finally: + self.store.unregister_buffer(base_ptr) + + +def clone_parallelism(parallelism): + cloned_axes = [] + for axis in parallelism.axes: + cloned_axis = make_parallel_axis( + axis.kind, + rank=axis.rank, + size=axis.size, + split_dim=axis.split_dim, + ) + if getattr(axis, "expert_id", None) is not None: + cloned_axis.expert_id = axis.expert_id + if getattr(axis, "stage_id", None) is not None: + cloned_axis.stage_id = axis.stage_id + cloned_axes.append(cloned_axis) + return make_tensor_parallelism(cloned_axes) + + +class TestUpsertParallelTensor(UpsertTestBase): + """Parallelism-based tensor upsert coverage for TP cases.""" + + def test_upsert_tensor_with_parallelism_tp(self): + require_unified_parallelism_api(self) + tp_size = 2 + split_dim = 1 + key = "tp_upsert_single" + original = torch.arange(6, dtype=torch.float32).view(2, 3).contiguous() + updated = (original + 100).contiguous() + original_shard = original.chunk(tp_size, split_dim)[0].contiguous() + updated_shard = updated.chunk(tp_size, split_dim)[0].contiguous() + parallelism = make_tensor_parallelism( + [make_parallel_axis("tp", rank=0, size=tp_size, split_dim=split_dim)] + ) + target = make_read_target("shard", parallelism) + + self.assertEqual( + self.store.put_tensor_with_parallelism(key, original_shard, parallelism), + 0, + ) + self.assertEqual( + self.store.upsert_tensor_with_parallelism(key, updated_shard, parallelism), + 0, + ) + + got = self.store.get_tensor_with_parallelism(key, target) + self.assertTrue(torch.equal(got, updated_shard)) + + def test_batch_upsert_tensor_with_parallelism_tp(self): + require_unified_parallelism_api(self) + tp_size = 2 + split_dim = 0 + keys = ["tp_batch_upsert_0", "tp_batch_upsert_1"] + originals = [torch.arange(16, dtype=torch.float32).view(4, 4).contiguous(), + torch.arange(16, 32, dtype=torch.float32).view(4, 4).contiguous()] + updates = [(t + 1000).contiguous() for t in originals] + original_shards = [tensor.chunk(tp_size, split_dim)[0].contiguous() for tensor in originals] + update_shards = [tensor.chunk(tp_size, split_dim)[0].contiguous() for tensor in updates] + parallelism = make_tensor_parallelism( + [make_parallel_axis("tp", rank=0, size=tp_size, split_dim=split_dim)] + ) + targets = [make_read_target("shard", parallelism) for _ in keys] + + self.assertEqual( + list(self.store.batch_put_tensor_with_parallelism(keys, original_shards, [parallelism] * len(keys))), + [0, 0], + ) + self.assertEqual( + list(self.store.batch_upsert_tensor_with_parallelism(keys, update_shards, [parallelism] * len(keys))), + [0, 0], + ) + + got_shards = self.store.batch_get_tensor_with_parallelism(keys, targets) + for got, expected in zip(got_shards, update_shards): + self.assertTrue(torch.equal(got, expected)) + + def test_upsert_tensor_with_parallelism_tp_with_config(self): + require_unified_parallelism_api(self) + tp_size = 2 + split_dim = 1 + key = "tp_upsert_pub_single" + config = ReplicateConfig() + config.replica_num = 1 + base = torch.ones(4, 6, dtype=torch.float32).contiguous() + updated = torch.full((4, 6), 7.0, dtype=torch.float32).contiguous() + base_shard = base.chunk(tp_size, split_dim)[0].contiguous() + updated_shard = updated.chunk(tp_size, split_dim)[0].contiguous() + parallelism = make_tensor_parallelism( + [make_parallel_axis("tp", rank=0, size=tp_size, split_dim=split_dim)] + ) + target = make_read_target("shard", parallelism) + + self.assertEqual( + self.store.put_tensor_with_parallelism(key, base_shard, parallelism, config=config), + 0, + ) + self.assertEqual( + self.store.upsert_tensor_with_parallelism(key, updated_shard, parallelism, config=config), + 0, + ) + + got = self.store.get_tensor_with_parallelism(key, target) + self.assertTrue(torch.equal(got, updated_shard)) + + def test_batch_upsert_tensor_with_parallelism_tp_with_config(self): + require_unified_parallelism_api(self) + tp_size = 2 + split_dim = 0 + config = ReplicateConfig() + config.replica_num = 1 + keys = ["tp_upsert_pub_batch_0", "tp_upsert_pub_batch_1"] + base = [torch.ones(4, 4, dtype=torch.float32).contiguous(), + torch.full((4, 4), 2.0, dtype=torch.float32).contiguous()] + updates = [torch.full((4, 4), 9.0, dtype=torch.float32).contiguous(), + torch.full((4, 4), 11.0, dtype=torch.float32).contiguous()] + base_shards = [tensor.chunk(tp_size, split_dim)[0].contiguous() for tensor in base] + update_shards = [tensor.chunk(tp_size, split_dim)[0].contiguous() for tensor in updates] + parallelism = make_tensor_parallelism( + [make_parallel_axis("tp", rank=0, size=tp_size, split_dim=split_dim)] + ) + targets = [make_read_target("shard", parallelism) for _ in keys] + + self.assertEqual( + list(self.store.batch_put_tensor_with_parallelism(keys, base_shards, [parallelism] * len(keys), config=config)), + [0, 0], + ) + self.assertEqual( + list(self.store.batch_upsert_tensor_with_parallelism(keys, update_shards, [parallelism] * len(keys), config=config)), + [0, 0], + ) + + got_shards = self.store.batch_get_tensor_with_parallelism(keys, targets) + for got, expected in zip(got_shards, update_shards): + self.assertTrue(torch.equal(got, expected)) + + # ========================================== # 6. Mixed Scenarios # ========================================== @@ -522,7 +1119,9 @@ if __name__ == "__main__": print(" Raw bytes: upsert, upsert_parts, upsert_batch,") print(" upsert_from, batch_upsert_from") print(" Tensor: upsert_tensor, batch_upsert_tensor,") - print(" upsert_tensor_from, batch_upsert_tensor_from") + print(" upsert_tensor_from, batch_upsert_tensor_from,") + print(" upsert_tensor_with_parallelism,") + print(" batch_upsert_tensor_with_parallelism") print(" Pub tensor: upsert_pub_tensor, batch_upsert_pub_tensor") print(" Mixed: cross-interface, sequential, size changes") unittest.main(verbosity=2) -- 2.34.1 From 1b6455aa651a3e7cb35a8f701a3ce2e9a1b4afa6 Mon Sep 17 00:00:00 2001 From: HomeDish <50695889+HomeDish@users.noreply.github.com> Date: Tue, 28 Apr 2026 09:47:30 +0800 Subject: [PATCH 014/382] =?UTF-8?q?[TENT][Sunrise]=20Add=20sunrise=5Flink?= =?UTF-8?q?=20transport,=20platform=20support,=20and=20UT=20=E2=80=A6=20(#?= =?UTF-8?q?1915)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [TENT][Sunrise] Add sunrise_link transport, platform support, and UT coverage Integrate Sunrise platform/transport wiring across TENT runtime and examples, add SunriseLink end-to-end unit tests, and fix RDMA error logging pointer formatting to avoid crash during registration failure paths. Made-with: Cursor * [CI] Update pre-commit hook versions Bump pre-commit hook revisions to current releases so local checks and CI use newer lint/format toolchains consistently. Made-with: Cursor * [TENT][Sunrise] Resolve review issues and drop stale bench target Address review feedback in SunriseLink transport/platform paths (stream/device context, registration map synchronization, safer probe/allocator handling, and cache-refresh strategy), and remove the obsolete transfer_engine_sunrise_bench CMake target now that its source no longer exists. * [TENT][sunrise] add engnish doc for sunrise_link * [TENT][sunrise] restore the memory free logic in the bench, and restore config files --------- Co-authored-by: liujialai --- docs/source/design/transfer-engine/index.md | 12 +- .../transfer-engine/sunrise_link_transport.md | 80 ++ .../zh_archive/sunrise_link_transport.md | 80 ++ .../example/transfer_engine_bench.cpp | 88 +- mooncake-transfer-engine/include/cuda_alike.h | 4 +- .../include/gpu_vendor/sunrise.h | 28 + .../tent/include/tent/common/types.h | 5 +- .../tent/include/tent/platform/sunrise.h | 53 + .../sunrise_link/sunrise_link_transport.h | 148 +++ .../tent/src/CMakeLists.txt | 11 +- .../tent/src/platform/CMakeLists.txt | 12 +- .../tent/src/platform/sunrise/CMakeLists.txt | 9 + .../platform/sunrise/sunrise_allocator.cpp | 92 ++ .../src/platform/sunrise/sunrise_probe.cpp | 306 +++++ .../tent/src/runtime/platform.cpp | 4 + .../tent/src/runtime/transfer_engine_impl.cpp | 1 + .../tent/src/runtime/transport_loader.cpp | 11 + .../tent/src/transport/CMakeLists.txt | 28 +- .../tent/src/transport/rdma/context.cpp | 8 +- .../src/transport/sunrise_link/CMakeLists.txt | 21 + .../sunrise_link/sunrise_link_transport.cpp | 1078 +++++++++++++++++ .../tent/tests/CMakeLists.txt | 13 + .../tests/sunrise_link_transport_test.cpp | 209 ++++ 23 files changed, 2238 insertions(+), 63 deletions(-) create mode 100644 docs/source/design/transfer-engine/sunrise_link_transport.md create mode 100644 docs/source/zh_archive/sunrise_link_transport.md create mode 100644 mooncake-transfer-engine/include/gpu_vendor/sunrise.h create mode 100644 mooncake-transfer-engine/tent/include/tent/platform/sunrise.h create mode 100644 mooncake-transfer-engine/tent/include/tent/transport/sunrise_link/sunrise_link_transport.h create mode 100644 mooncake-transfer-engine/tent/src/platform/sunrise/CMakeLists.txt create mode 100644 mooncake-transfer-engine/tent/src/platform/sunrise/sunrise_allocator.cpp create mode 100644 mooncake-transfer-engine/tent/src/platform/sunrise/sunrise_probe.cpp create mode 100644 mooncake-transfer-engine/tent/src/transport/sunrise_link/CMakeLists.txt create mode 100644 mooncake-transfer-engine/tent/src/transport/sunrise_link/sunrise_link_transport.cpp create mode 100644 mooncake-transfer-engine/tent/tests/sunrise_link_transport_test.cpp diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index 85a0c29a..d9ead6ad 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -167,7 +167,7 @@ Transfer Engine provides batch-based read/write transfers between segments (DRAM ### Multi-Transport Management The `TransferEngine` class internally manages multiple backend `Transport` classes. -And it will discover the toplogy between CPU/CUDA and RDMA devices automatically +And it will discover the topology between CPU/CUDA and RDMA devices automatically (more device types are working in progress, feedbacks are welcome when the automatic discovery mechanism is not accurate), and it will install `Transport` automatically based on the topology. @@ -305,7 +305,7 @@ For advanced users, TransferEngine provides the following advanced runtime optio - `MC_MAX_PRC_PORT` Specifies the maximum port number for RPC service. The default value is 17000. - `MC_PATH_ROUNDROBIN` Use round-robin mode in the RDMA path selection. This may be beneficial for transferring large bulks. - `MC_ENDPOINT_STORE_TYPE` Choose FIFO Endpoint Store (`FIFO`) or Sieve Endpoint Store (`SIEVE`), default is `SIEVE`. -- `MC_TCP_ENABLE_CONNECTION_POOL` Enable TCP Connection Pool to avoid excessive sockets. +- `MC_TCP_ENABLE_CONNECTION_POOL` Enable TCP Connection Pool to avoid excessive sockets. ## C++ API Reference @@ -333,6 +333,14 @@ ascend_transport heterogeneous_ascend ::: +## Sunrise Link Transport Component + +:::{toctree} +:maxdepth: 1 + +sunrise_link_transport +::: + ## Benchmark and Tuning Guide :::{toctree} diff --git a/docs/source/design/transfer-engine/sunrise_link_transport.md b/docs/source/design/transfer-engine/sunrise_link_transport.md new file mode 100644 index 00000000..2b82f064 --- /dev/null +++ b/docs/source/design/transfer-engine/sunrise_link_transport.md @@ -0,0 +1,80 @@ +# Sunrise Link Transport + +## Overview +Sunrise Link is a GPU transport backend in the Mooncake TENT transfer framework. It relies on Tang Runtime (`tangrt`) for device memory allocation, pointer attribute queries, peer copy, and IPC handle operations, and is registered as `SUNRISE_LINK` in TENT. + +At runtime, `TransferEngineImpl` loads Sunrise Link Transport when `USE_SUNRISE` is enabled at build time and `transports/sunrise_link/enable=true` is set in config. + +--- + +## Additional Dependencies +In addition to Mooncake's base dependencies, Sunrise Link Transport requires Tang Runtime: + +- **Header path**: `/usr/local/tangrt/include` +- **Library path**: `/usr/local/tangrt/lib/linux-x86_64` +- **Shared library**: `libtangrt_shared.so` (loaded from the default installation path at runtime) + +Make sure runtime libraries related to `tangrt` and `ptml` can be found by the dynamic linker (for example, via `LD_LIBRARY_PATH` or system library search paths). + +--- + +## Build and Compile + +**Prerequisites** + +- Tang Runtime is installed and available (default path: `/usr/local/tangrt`) +- Build environment can access Mooncake and its base dependencies + +**CMake Configuration** + +```bash +# Clone Mooncake +git clone https://github.com/kvcache-ai/Mooncake.git +cd Mooncake + +# Enable TENT + Sunrise Link +mkdir build && cd build +cmake .. -DUSE_TENT=ON -DUSE_SUNRISE=ON + +# Build +make -j$(nproc) +``` + +--- + +## Run and Test + +`transfer_engine_bench` supports the `sunrise_link` protocol for basic connectivity and performance validation. + +```bash +# Terminal 1: target +./transfer_engine_bench \ + --mode=target \ + --protocol=sunrise_link \ + --local_server_name=10.0.0.2 \ + --metadata_server=P2PHANDSHAKE \ + --gpu_id=0 + +# Terminal 2: initiator +./transfer_engine_bench \ + --mode=initiator \ + --protocol=sunrise_link \ + --metadata_server=P2PHANDSHAKE \ + --segment_id=10.0.0.2:$PORT \ + --gpu_id=0 \ + --block_size=8388608 \ + --batch_size=32 +``` + +> Note: when `metadata_server=P2PHANDSHAKE`, the target node may listen on a dynamically assigned port. Replace `$PORT` in `--segment_id` with the actual port printed in target logs. + +--- + +## Configuration Options (TENT) + +Sunrise Link behavior can be tuned through config file options: + +- `transports/sunrise_link/enable`: enable or disable the transport (default: `true`) +- `transports/sunrise_link/async_memcpy_threshold`: threshold for async memcpy (MiB) + +Tune these options based on workload characteristics and hardware topology. diff --git a/docs/source/zh_archive/sunrise_link_transport.md b/docs/source/zh_archive/sunrise_link_transport.md new file mode 100644 index 00000000..74bf2e11 --- /dev/null +++ b/docs/source/zh_archive/sunrise_link_transport.md @@ -0,0 +1,80 @@ +# Sunrise Link Transport + +## 概述 +Sunrise Link 是 Mooncake TENT 传输框架中的一个 GPU 传输后端。它通过 Tang Runtime(`tangrt`)提供设备内存分配、指针属性查询、Peer Copy 与 IPC 句柄相关能力,并在 TENT 中以 `SUNRISE_LINK` 传输类型注册。 + +在运行时,当编译开启 `USE_SUNRISE` 且配置项 `transports/sunrise_link/enable=true` 时,`TransferEngineImpl` 会加载 Sunrise Link Transport。 + +--- + +## 新增依赖 +Sunrise Link Transport 在 Mooncake 基础依赖外,额外依赖 Tang Runtime: + +- **头文件路径**:`/usr/local/tangrt/include` +- **库路径**:`/usr/local/tangrt/lib/linux-x86_64` +- **动态库**:`libtangrt_shared.so`(运行时按默认安装路径加载) + +建议确认 `tangrt` 与 `ptml` 相关运行库可被动态链接器找到(例如通过 `LD_LIBRARY_PATH` 或系统库路径配置)。 + +--- + +## 构建与编译 + +**前置条件** + +- 已安装可用的 Tang Runtime(默认安装到 `/usr/local/tangrt`) +- 编译环境可访问 Mooncake 及其基础依赖 + +**CMake 配置** + +```bash +# 克隆 Mooncake 仓库 +git clone https://github.com/kvcache-ai/Mooncake.git +cd Mooncake + +# 启用 TENT + Sunrise Link +mkdir build && cd build +cmake .. -DUSE_TENT=ON -DUSE_SUNRISE=ON + +# 编译 +make -j$(nproc) +``` + +--- + +## 运行与测试 + +`transfer_engine_bench` 支持 `sunrise_link` 协议,可用于基本连通性与性能验证。 + +```bash +# 终端 1:目标端(Target) +./transfer_engine_bench \ + --mode=target \ + --protocol=sunrise_link \ + --local_server_name=10.0.0.2 \ + --metadata_server=P2PHANDSHAKE \ + --gpu_id=0 + +# 终端 2:发起端(Initiator) +./transfer_engine_bench \ + --mode=initiator \ + --protocol=sunrise_link \ + --metadata_server=P2PHANDSHAKE \ + --segment_id=10.0.0.2:$PORT \ + --gpu_id=0 \ + --block_size=8388608 \ + --batch_size=32 +``` + +> 说明:当 `metadata_server=P2PHANDSHAKE` 时,目标端实际监听端口可能为动态分配端口。请以目标端日志中打印的实际端口替换 `--segment_id` 中的 `$PORT`。 + +--- + +## 配置项说明(TENT) + +Sunrise Link 支持通过配置文件调整行为,常用项包括: + +- `transports/sunrise_link/enable`:是否启用该传输(默认 `true`) +- `transports/sunrise_link/async_memcpy_threshold`:异步拷贝阈值(单位 MiB) + +可根据业务负载与设备拓扑微调上述参数。 diff --git a/mooncake-transfer-engine/example/transfer_engine_bench.cpp b/mooncake-transfer-engine/example/transfer_engine_bench.cpp index e47aafd9..1cd112a1 100644 --- a/mooncake-transfer-engine/example/transfer_engine_bench.cpp +++ b/mooncake-transfer-engine/example/transfer_engine_bench.cpp @@ -43,7 +43,7 @@ #endif #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) + defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) #include #if defined(USE_MNNVL) || defined(USE_UBSHMEM) @@ -55,16 +55,16 @@ #endif #if defined(USE_UBSHMEM) -static void checkAclError(aclError result, const char *message) { +static void checkAclError(aclError result, const char* message) { if (result != ACL_ERROR_NONE) { - const char *errMsg = aclGetRecentErrMsg(); + const char* errMsg = aclGetRecentErrMsg(); LOG(ERROR) << message << " (Error code: " << result << " - " << errMsg << ")"; exit(EXIT_FAILURE); } } #else -static void checkCudaError(cudaError_t result, const char *message) { +static void checkCudaError(cudaError_t result, const char* message) { if (result != cudaSuccess) { LOG(ERROR) << message << " (Error code: " << result << " - " << cudaGetErrorString(result) << ")" << std::endl; @@ -88,7 +88,8 @@ DEFINE_string(mode, "initiator", DEFINE_string(operation, "read", "Operation type: read or write"); DEFINE_string(protocol, "rdma", - "Transfer protocol: rdma|barex|tcp|efa|nvlink|nvlink_intra|hip"); + "Transfer protocol: " + "rdma|barex|tcp|efa|nvlink|nvlink_intra|hip|sunrise_link"); DEFINE_string(device_name, "mlx5_2", "Device name to use, valid if protocol=rdma"); @@ -107,7 +108,7 @@ DEFINE_uint32(report_precision, 2, "Report precision"); DEFINE_string(backend, "classic", "Backend to use: classic|tent"); #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) + defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) DEFINE_bool(use_vram, true, "Allocate memory from GPU/NPU VRAM"); DEFINE_bool(init_mem, true, "Initialize allocated memory"); DEFINE_int32(gpu_id, 0, @@ -116,10 +117,10 @@ DEFINE_int32(gpu_id, 0, using namespace mooncake; -static void *allocateMemoryPool(size_t size, int buffer_id, +static void* allocateMemoryPool(size_t size, int buffer_id, bool from_vram = false) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) + defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) if (from_vram) { int gpu_id; if (FLAGS_gpu_id == -1) { @@ -127,7 +128,7 @@ static void *allocateMemoryPool(size_t size, int buffer_id, } else { gpu_id = FLAGS_gpu_id; } - void *d_buf; + void* d_buf; #if defined(USE_UBSHMEM) LOG(INFO) << "Allocating memory on NPU " << gpu_id; checkAclError(aclrtSetDevice(gpu_id), "Failed to set device"); @@ -188,9 +189,9 @@ static void *allocateMemoryPool(size_t size, int buffer_id, return numa_alloc_onnode(size, buffer_id); } -static void freeMemoryPool(void *addr, size_t size) { +static void freeMemoryPool(void* addr, size_t size) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) + defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) if (FLAGS_protocol == "nvlink" || FLAGS_protocol == "hip") { #ifdef USE_MNNVL if (FLAGS_use_vram) { @@ -271,10 +272,21 @@ static inline std::string calculateRate(uint64_t data_bytes, double duration) { volatile bool running = true; std::atomic total_batch_count(0); +// Ensure each worker thread has a valid GPU context before issuing transfers. +static inline void setWorkerDeviceIfNeeded() { +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_SUNRISE) + if (FLAGS_use_vram && FLAGS_gpu_id >= 0) { + checkCudaError(cudaSetDevice(FLAGS_gpu_id), + "Failed to set device in worker"); + } +#endif +} + // Common helper to determine buffer count based on GPU/NUMA configuration static int determineBufferCount() { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_SUNRISE) if (FLAGS_use_vram) { int gpu_num; LOG(INFO) << "VRAM is used"; @@ -301,11 +313,11 @@ static int determineBufferCount() { } // Common helper to allocate memory buffers -static std::vector allocateBuffers() { +static std::vector allocateBuffers() { buffer_num = determineBufferCount(); - std::vector addr(buffer_num); + std::vector addr(buffer_num); #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) + defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) for (int i = 0; i < buffer_num; ++i) { addr[i] = allocateMemoryPool(FLAGS_buffer_size, i, FLAGS_use_vram); } @@ -318,7 +330,7 @@ static std::vector allocateBuffers() { } // Common helper to free memory buffers -static void freeBuffers(std::vector &addr) { +static void freeBuffers(std::vector& addr) { for (int i = 0; i < buffer_num; ++i) { freeMemoryPool(addr[i], FLAGS_buffer_size); } @@ -328,7 +340,7 @@ static void freeBuffers(std::vector &addr) { // Helper to get location name for classic backend static std::string getLocationName(int buffer_id) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) + defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) if (FLAGS_use_vram) { int name_suffix = (FLAGS_gpu_id == -1) ? buffer_id : FLAGS_gpu_id; return std::string(GPU_PREFIX) + std::to_string(name_suffix); @@ -337,8 +349,8 @@ static std::string getLocationName(int buffer_id) { return "cpu:" + std::to_string(buffer_id); } -Status initiatorWorker(TransferEngine *engine, SegmentID segment_id, - int thread_id, void *addr) { +Status initiatorWorker(TransferEngine* engine, SegmentID segment_id, + int thread_id, void* addr) { bindToSocket(thread_id % NR_SOCKETS); TransferRequest::OpCode opcode; if (FLAGS_operation == "read") @@ -367,7 +379,7 @@ Status initiatorWorker(TransferEngine *engine, SegmentID segment_id, TransferRequest entry; entry.opcode = opcode; entry.length = FLAGS_block_size; - entry.source = (uint8_t *)(addr) + + entry.source = (uint8_t*)(addr) + FLAGS_block_size * (i * FLAGS_threads + thread_id); entry.target_id = segment_id; entry.target_offset = @@ -404,7 +416,7 @@ Status initiatorWorker(TransferEngine *engine, SegmentID segment_id, return Status::OK(); } -std::string formatDeviceNames(const std::string &device_names) { +std::string formatDeviceNames(const std::string& device_names) { std::stringstream ss(device_names); std::string item; std::vector tokens; @@ -451,19 +463,19 @@ std::string loadNicPriorityMatrix() { // Common helper to install transport based on protocol flag // Returns the installed transport, or nullptr if auto_discovery is enabled -static Transport *installTransportFromFlags(TransferEngine *engine) { +static Transport* installTransportFromFlags(TransferEngine* engine) { if (FLAGS_auto_discovery) { return nullptr; } - Transport *xport = nullptr; + Transport* xport = nullptr; std::string nic_priority_matrix; - std::unique_ptr args(nullptr, free); + std::unique_ptr args(nullptr, free); if (FLAGS_protocol == "rdma" || FLAGS_protocol == "barex") { nic_priority_matrix = loadNicPriorityMatrix(); - args.reset(static_cast(malloc(2 * sizeof(void *)))); - args.get()[0] = const_cast(nic_priority_matrix.c_str()); + args.reset(static_cast(malloc(2 * sizeof(void*)))); + args.get()[0] = const_cast(nic_priority_matrix.c_str()); args.get()[1] = nullptr; xport = engine->installTransport(FLAGS_protocol.c_str(), args.get()); } else if (FLAGS_protocol == "ub") { @@ -476,7 +488,8 @@ static Transport *installTransportFromFlags(TransferEngine *engine) { xport = engine->installTransport("efa", nullptr); } else if (FLAGS_protocol == "tcp" || FLAGS_protocol == "nvlink" || FLAGS_protocol == "hip" || FLAGS_protocol == "nvlink_intra" || - FLAGS_protocol == "ubshmem") { + FLAGS_protocol == "ubshmem" || + FLAGS_protocol == "sunrise_link") { xport = engine->installTransport(FLAGS_protocol.c_str(), nullptr); } else { LOG(ERROR) << "Unsupported protocol: " << FLAGS_protocol; @@ -494,7 +507,7 @@ int initiator() { hostname_port.first.c_str(), hostname_port.second); if (!FLAGS_auto_discovery) { - Transport *xport = installTransportFromFlags(engine.get()); + Transport* xport = installTransportFromFlags(engine.get()); LOG_ASSERT(xport); } @@ -590,8 +603,8 @@ int target() { namespace tent_backend { // Helper function to register buffers for TENT backend -static void registerBuffers(mooncake::tent::TransferEngine *engine, - std::vector &addr) { +static void registerBuffers(mooncake::tent::TransferEngine* engine, + std::vector& addr) { for (int i = 0; i < buffer_num; ++i) { auto status = engine->registerLocalMemory(addr[i], FLAGS_buffer_size); LOG_ASSERT(status.ok()) @@ -600,8 +613,8 @@ static void registerBuffers(mooncake::tent::TransferEngine *engine, } // Helper function to unregister buffers for TENT backend -static void unregisterBuffers(mooncake::tent::TransferEngine *engine, - std::vector &addr) { +static void unregisterBuffers(mooncake::tent::TransferEngine* engine, + std::vector& addr) { for (int i = 0; i < buffer_num; ++i) { engine->unregisterLocalMemory(addr[i], FLAGS_buffer_size); } @@ -639,11 +652,12 @@ std::shared_ptr createTentConfig() { return config; } -void initiatorWorker(mooncake::tent::TransferEngine *engine, +void initiatorWorker(mooncake::tent::TransferEngine* engine, mooncake::tent::SegmentID segment_id, int thread_id, - void *addr, - const mooncake::tent::SegmentInfo &segment_info) { + void* addr, + const mooncake::tent::SegmentInfo& segment_info) { bindToSocket(thread_id % NR_SOCKETS); + setWorkerDeviceIfNeeded(); mooncake::tent::Request::OpCode opcode; if (FLAGS_operation == "read") opcode = mooncake::tent::Request::READ; @@ -672,7 +686,7 @@ void initiatorWorker(mooncake::tent::TransferEngine *engine, mooncake::tent::Request entry; entry.opcode = opcode; entry.length = FLAGS_block_size; - entry.source = (uint8_t *)(addr) + + entry.source = (uint8_t*)(addr) + FLAGS_block_size * (i * FLAGS_threads + thread_id); entry.target_id = segment_id; entry.target_offset = @@ -806,7 +820,7 @@ void check_total_buffer_size() { } } -int main(int argc, char **argv) { +int main(int argc, char** argv) { gflags::ParseCommandLineFlags(&argc, &argv, false); check_total_buffer_size(); diff --git a/mooncake-transfer-engine/include/cuda_alike.h b/mooncake-transfer-engine/include/cuda_alike.h index 39f62897..07d56c80 100644 --- a/mooncake-transfer-engine/include/cuda_alike.h +++ b/mooncake-transfer-engine/include/cuda_alike.h @@ -13,10 +13,12 @@ #include "gpu_vendor/ubshmem.h" #elif defined(USE_MACA) #include "gpu_vendor/maca.h" +#elif defined(USE_SUNRISE) +#include "gpu_vendor/sunrise.h" #endif #if !defined(USE_HIP) && !defined(USE_MUSA) && !defined(USE_MLU) && \ - !defined(USE_UBSHMEM) && !defined(USE_MACA) + !defined(USE_UBSHMEM) && !defined(USE_MACA) && !defined(USE_SUNRISE) #include const static std::string GPU_PREFIX = "cuda:"; #endif diff --git a/mooncake-transfer-engine/include/gpu_vendor/sunrise.h b/mooncake-transfer-engine/include/gpu_vendor/sunrise.h new file mode 100644 index 00000000..663c0dc3 --- /dev/null +++ b/mooncake-transfer-engine/include/gpu_vendor/sunrise.h @@ -0,0 +1,28 @@ +#pragma once + +#include + +#include + +// SunriseLink uses CUDA-style location naming in metadata. +const static std::string GPU_PREFIX = "cuda:"; + +// Minimal CUDA-like API shim for benchmark paths. +#define cudaError_t tangError_t +#define cudaSuccess tangSuccess +#define cudaGetErrorString tangGetErrorString + +#define cudaSetDevice tangSetDevice +#define cudaGetDeviceCount tangGetDeviceCount + +#define cudaMalloc tangMalloc +#define cudaFree tangFree +#define cudaMemset tangMemset + +#define cudaPointerAttributes tangPointerAttributes +#define cudaPointerGetAttributes tangPointerGetAttributes +#define cudaMemoryTypeDevice tangMemoryTypeDevice +#define cudaMemoryTypeHost tangMemoryTypeHost +#define cudaMemoryTypeUnregistered tangMemoryTypeUnregistered + +#define cudaStreamSynchronize tangStreamSynchronize diff --git a/mooncake-transfer-engine/tent/include/tent/common/types.h b/mooncake-transfer-engine/tent/include/tent/common/types.h index da862f42..3455d56c 100644 --- a/mooncake-transfer-engine/tent/include/tent/common/types.h +++ b/mooncake-transfer-engine/tent/include/tent/common/types.h @@ -79,9 +79,10 @@ enum TransportType { IOURING, TCP, AscendDirect, + SUNRISE_LINK, UNSPEC }; -const static int kSupportedTransportTypes = 8; +const static int kSupportedTransportTypes = (int)TransportType::UNSPEC; struct MemoryOptions { Location location = kWildcardLocation; @@ -107,4 +108,4 @@ struct SegmentInfo { } // namespace tent } // namespace mooncake -#endif // TENT_TYPES_H \ No newline at end of file +#endif // TENT_TYPES_H diff --git a/mooncake-transfer-engine/tent/include/tent/platform/sunrise.h b/mooncake-transfer-engine/tent/include/tent/platform/sunrise.h new file mode 100644 index 00000000..14df6c06 --- /dev/null +++ b/mooncake-transfer-engine/tent/include/tent/platform/sunrise.h @@ -0,0 +1,53 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#ifndef SUNRISE_H +#define SUNRISE_H + +#include "tent/common/config.h" +#include "tent/runtime/platform.h" + +namespace mooncake { +namespace tent { + +class SunrisePlatform : public Platform { + public: + SunrisePlatform(std::shared_ptr config) : conf(std::move(config)) {} + + virtual ~SunrisePlatform() {} + + virtual Status probe(std::vector& nic_list, + std::vector& mem_list); + + virtual Status allocate(void** pptr, size_t size, MemoryOptions& options); + + virtual Status free(void* ptr, size_t size); + + virtual Status copy(void* dst, void* src, size_t length); + + virtual MemoryType getMemoryType(void* addr); + + virtual const std::vector getLocation( + void* start, size_t len, bool skip_prefault = false); + + virtual const std::string type() const { return "sunrise"; } + + private: + std::shared_ptr conf; +}; + +} // namespace tent +} // namespace mooncake + +#endif // SUNRISE_H diff --git a/mooncake-transfer-engine/tent/include/tent/transport/sunrise_link/sunrise_link_transport.h b/mooncake-transfer-engine/tent/include/tent/transport/sunrise_link/sunrise_link_transport.h new file mode 100644 index 00000000..6fdae540 --- /dev/null +++ b/mooncake-transfer-engine/tent/include/tent/transport/sunrise_link/sunrise_link_transport.h @@ -0,0 +1,148 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +#include +#include +#include +#include +#include + +#include "tent/runtime/control_plane.h" +#include "tent/runtime/transport.h" + +namespace mooncake { +namespace tent { + +struct SunriseLinkTask { + Request request; + volatile TransferStatusEnum status_word; + volatile size_t transferred_bytes; + uint64_t target_addr = 0; + bool is_tang_ipc = false; + int tang_gpu_id = -1; +}; + +struct SunriseLinkSubBatch : public Transport::SubBatch { + std::vector task_list; + size_t max_size = 0; + /// Opaque `tangStream_t` when built with Tang; mirrors + /// NVLinkSubBatch::stream. + void* stream = nullptr; + virtual size_t size() const { return task_list.size(); } +}; + +struct SunriseLinkDeviceInfo { + int device_id; + std::vector active_ports; +}; + +struct SunriseLinkPeerInfo { + int peer_device_id; + int peer_port; + bool is_active; +}; + +class SunriseLinkTransport : public Transport { + public: + SunriseLinkTransport(); + ~SunriseLinkTransport() override; + + Status install(std::string& local_segment_name, + std::shared_ptr metadata, + std::shared_ptr local_topology, + std::shared_ptr conf = nullptr) override; + + Status uninstall() override; + + Status allocateSubBatch(SubBatchRef& batch, size_t max_size) override; + + Status freeSubBatch(SubBatchRef& batch) override; + + Status submitTransferTasks( + SubBatchRef batch, const std::vector& request_list) override; + + Status getTransferStatus(SubBatchRef batch, int task_id, + TransferStatus& status) override; + + Status addMemoryBuffer(BufferDesc& desc, + const MemoryOptions& options) override; + + Status removeMemoryBuffer(BufferDesc& desc) override; + + const char* getName() const override { return "sunrise_link"; } + + bool supportNotification() const override { return true; } + + private: + Status startTransfer(SunriseLinkTask* task, SunriseLinkSubBatch* batch); + + // SunriseLink initialization + Status initSunriseLink(); + + // Memory registration + Status registerMemory(void* addr, size_t size); + + // Memory deregistration + Status deregisterMemory(void* addr); + + // Topology detection + Status detectTopology(); + Status relocateRemoteAddress(uint64_t& dest_addr, uint64_t length, + uint64_t target_id, + int* target_gpu_id = nullptr); + + // Runtime library handle + void* runtime_lib_handle_ = nullptr; + + void* ptml_handle_ = nullptr; + + // Local device information + std::vector local_devices_; + + // Topology map: device_id -> peer info + std::map> topology_map_; + + // (local_chipid, remote_chipid) -> local PTLink port (S2 C2C) + std::map, int> c2c_port_by_pair_; + + // Registered memory regions + std::map registered_memory_; + std::map registered_memory_gpu_id_; + mutable std::mutex registered_memory_mutex_; + + struct OpenedIpcEntry { + void* dev_ptr{nullptr}; + uint64_t length{0}; + int gpu_id{-1}; + }; + using IpcRelocateMap = + std::unordered_map>; + RWSpinlock relocate_lock_; + IpcRelocateMap relocate_map_; + + // Installed flag + bool installed_ = false; + + // Config + std::shared_ptr conf_; + std::shared_ptr metadata_; + + uint64_t async_memcpy_threshold_ = 0; +}; + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/CMakeLists.txt b/mooncake-transfer-engine/tent/src/CMakeLists.txt index 7fb3174b..9997ecab 100644 --- a/mooncake-transfer-engine/tent/src/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/src/CMakeLists.txt @@ -65,6 +65,14 @@ else() message(STATUS "Uring: Disabled") endif() +if(USE_SUNRISE) + target_compile_definitions(tent_interface INTERFACE USE_SUNRISE) + target_include_directories(tent_interface INTERFACE /usr/local/tangrt/include) + target_link_directories(tent_interface INTERFACE + /usr/local/tangrt/lib/linux-x86_64) + target_link_libraries(tent_interface INTERFACE tangrt_shared ptml_shared dl) +endif() + set(YALANTING_TARGET yalantinglibs::yalantinglibs) if(NOT TARGET ${YALANTING_TARGET}) if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR) @@ -101,6 +109,7 @@ foreach( tent_platform_all platform_cuda platform_ascend + platform_sunrise tent_xport_gds tent_xport_uring tent_xport_bufio @@ -110,13 +119,13 @@ foreach( tent_xport_shm tent_xport_tcp tent_xport_ascend_direct + tent_xport_sunrise_link tent_metrics) if(TARGET ${tgt}) target_link_libraries(tent_link_group INTERFACE ${tgt}) endif() endforeach() target_link_libraries(tent_link_group INTERFACE "-Wl,--end-group") - install( TARGETS tent_shared ARCHIVE DESTINATION lib diff --git a/mooncake-transfer-engine/tent/src/platform/CMakeLists.txt b/mooncake-transfer-engine/tent/src/platform/CMakeLists.txt index d17f6436..dfc5792e 100644 --- a/mooncake-transfer-engine/tent/src/platform/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/src/platform/CMakeLists.txt @@ -1,12 +1,16 @@ add_subdirectory(cuda) add_subdirectory(ascend) +add_subdirectory(sunrise) file(GLOB PLATFORM_SOURCES "*.cpp") add_library(tent_platform_all STATIC ${PLATFORM_SOURCES}) target_link_libraries(tent_platform_all PUBLIC tent_common) -if (TARGET platform_cuda) - target_link_libraries(tent_platform_all PUBLIC platform_cuda) +if(TARGET platform_cuda) + target_link_libraries(tent_platform_all PUBLIC platform_cuda) endif() -if (TARGET platform_ascend) - target_link_libraries(tent_platform_all PUBLIC platform_ascend) +if(TARGET platform_ascend) + target_link_libraries(tent_platform_all PUBLIC platform_ascend) +endif() +if(TARGET platform_sunrise) + target_link_libraries(tent_platform_all PUBLIC platform_sunrise) endif() diff --git a/mooncake-transfer-engine/tent/src/platform/sunrise/CMakeLists.txt b/mooncake-transfer-engine/tent/src/platform/sunrise/CMakeLists.txt new file mode 100644 index 00000000..a6a8c4c5 --- /dev/null +++ b/mooncake-transfer-engine/tent/src/platform/sunrise/CMakeLists.txt @@ -0,0 +1,9 @@ +if(USE_SUNRISE) + file(GLOB TENT_PLATFORM_SUNRISE_SOURCES "*.cpp") + add_library(platform_sunrise STATIC ${TENT_PLATFORM_SUNRISE_SOURCES}) + target_include_directories(platform_sunrise PUBLIC /usr/local/tangrt/include) + target_link_directories(platform_sunrise PUBLIC + /usr/local/tangrt/lib/linux-x86_64) + target_link_libraries(platform_sunrise PUBLIC tent_common tangrt_shared + ptml_shared dl) +endif() diff --git a/mooncake-transfer-engine/tent/src/platform/sunrise/sunrise_allocator.cpp b/mooncake-transfer-engine/tent/src/platform/sunrise/sunrise_allocator.cpp new file mode 100644 index 00000000..56569161 --- /dev/null +++ b/mooncake-transfer-engine/tent/src/platform/sunrise/sunrise_allocator.cpp @@ -0,0 +1,92 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/platform/sunrise.h" + +#include + +#include "tent/runtime/topology.h" +#include + +namespace mooncake { +namespace tent { + +Status SunrisePlatform::allocate(void** pptr, size_t size, + MemoryOptions& options) { + LocationParser location(options.location); + if (location.type() == "cuda") { + int saved_dev = -1; + const tangError_t get_dev_ret = tangGetDevice(&saved_dev); + if (tangSetDevice(location.index()) != tangSuccess) { + return Status::InternalError("Unable to switch sunrise device"); + } + tangError_t ret = tangMalloc(pptr, size); + if (get_dev_ret == tangSuccess && saved_dev >= 0) { + tangSetDevice(saved_dev); + } + if (ret != tangSuccess) + return Status::InternalError( + "Unable to allocate sunrise device memory"); + return Status::OK(); + } + int socket_id = 0; + if (location.type() == "cpu") socket_id = location.index(); + *pptr = numa_alloc_onnode(size, socket_id); + if (!(*pptr)) + return Status::InternalError("Unable to allocate DRAM memory"); + return Status::OK(); +} + +Status SunrisePlatform::free(void* ptr, size_t size) { + tangPointerAttributes attributes{}; + tangError_t ret = tangPointerGetAttributes(&attributes, ptr); + if (ret == tangSuccess && attributes.type == tangMemoryTypeDevice) { + if (attributes.device >= 0) { + int saved_dev = 0; + tangGetDevice(&saved_dev); + tangSetDevice(attributes.device); + tangFree(ptr); + tangSetDevice(saved_dev); + } else { + tangFree(ptr); + } + } else { + numa_free(ptr, size); + } + return Status::OK(); +} + +Status SunrisePlatform::copy(void* dst, void* src, size_t length) { + tangPointerAttributes sa{}; + tangPointerAttributes da{}; + (void)tangPointerGetAttributes(&sa, src); + (void)tangPointerGetAttributes(&da, dst); + tangMemcpyKind kind = tangMemcpyHostToHost; + if (sa.type == tangMemoryTypeDevice && da.type == tangMemoryTypeDevice) { + kind = tangMemcpyDeviceToDevice; + } else if (sa.type == tangMemoryTypeDevice && + da.type != tangMemoryTypeDevice) { + kind = tangMemcpyDeviceToHost; + } else if (sa.type != tangMemoryTypeDevice && + da.type == tangMemoryTypeDevice) { + kind = tangMemcpyHostToDevice; + } + tangError_t ret = tangMemcpy(dst, src, length, kind); + if (ret != tangSuccess) + return Status::InternalError("Sunrise memcpy failed"); + return Status::OK(); +} + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/platform/sunrise/sunrise_probe.cpp b/mooncake-transfer-engine/tent/src/platform/sunrise/sunrise_probe.cpp new file mode 100644 index 00000000..db9e4cce --- /dev/null +++ b/mooncake-transfer-engine/tent/src/platform/sunrise/sunrise_probe.cpp @@ -0,0 +1,306 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/platform/sunrise.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "tent/common/utils/prefault.h" +#include +#include + +namespace mooncake { +namespace tent { + +static inline uintptr_t alignPage(uintptr_t address) { + const static size_t kPageSize = 4096; + return address & ~(kPageSize - 1); +} + +static inline std::string genCpuNodeName(int node) { + if (node >= 0) return "cpu:" + std::to_string(node); + return kWildcardLocation; +} + +static inline std::string genCudaNodeName(int node) { + if (node >= 0) return "cuda:" + std::to_string(node); + return kWildcardLocation; +} + +static bool isIbDeviceAccessible(struct ibv_device* device) { + char device_path[PATH_MAX]; + struct stat st; + + snprintf(device_path, sizeof(device_path), "/dev/infiniband/%s", + device->dev_name); + if (stat(device_path, &st) != 0) return false; + if (!S_ISCHR(st.st_mode)) return false; + if (access(device_path, R_OK | W_OK) != 0) return false; + return true; +} + +static bool checkIbDevicePort(struct ibv_context* context, uint8_t port_num) { + struct ibv_port_attr port_attr; + if (ibv_query_port(context, port_num, &port_attr) != 0) return false; + if (port_attr.gid_tbl_len == 0) return false; + if (port_attr.state != IBV_PORT_ACTIVE) return false; + return true; +} + +static bool isIbDeviceAvailable(struct ibv_device* device) { + if (!isIbDeviceAccessible(device)) return false; + struct ibv_context* context = ibv_open_device(device); + if (!context) return false; + struct ibv_device_attr device_attr; + if (ibv_query_device(context, &device_attr) != 0) { + ibv_close_device(context); + return false; + } + bool has_active_port = false; + for (uint8_t port = 1; port <= device_attr.phys_port_cnt; ++port) { + if (checkIbDevicePort(context, port)) { + has_active_port = true; + break; + } + } + ibv_close_device(context); + return has_active_port; +} + +static std::vector listInfiniBandDevices() { + int num_devices = 0; + std::vector devices; + struct ibv_device** device_list = ibv_get_device_list(&num_devices); + if (!device_list || num_devices <= 0) return {}; + + for (int i = 0; i < num_devices; ++i) { + std::string device_name = ibv_get_device_name(device_list[i]); + if (!isIbDeviceAvailable(device_list[i])) continue; + + char path[PATH_MAX + 32]; + char resolved_path[PATH_MAX]; + snprintf(path, sizeof(path), "/sys/class/infiniband/%s/../..", + device_name.c_str()); + if (realpath(path, resolved_path) == NULL) continue; + std::string pci_bus_id = basename(resolved_path); + + int numa_node = -1; + snprintf(path, sizeof(path), "%s/numa_node", resolved_path); + std::ifstream(path) >> numa_node; + + devices.push_back( + Topology::NicEntry{.name = std::move(device_name), + .pci_bus_id = std::move(pci_bus_id), + .type = Topology::NIC_RDMA, + .numa_node = numa_node}); + } + ibv_free_device_list(device_list); + return devices; +} + +static void filterInfiniBandDevices(std::vector& devices, + std::shared_ptr conf) { + auto whitelist = conf->getArray("topology/rdma_whitelist"); + auto blacklist = conf->getArray("topology/rdma_blacklist"); + std::vector new_devices; + if (!whitelist.empty()) { + for (auto& entry : devices) { + if (std::find(whitelist.begin(), whitelist.end(), entry.name) != + whitelist.end()) + new_devices.push_back(entry); + } + devices.swap(new_devices); + return; + } + if (!blacklist.empty()) { + for (auto& entry : devices) { + if (std::find(blacklist.begin(), blacklist.end(), entry.name) == + blacklist.end()) + new_devices.push_back(entry); + } + devices.swap(new_devices); + } +} + +static void discoverCpuTopology(std::vector& nic_list, + std::vector& mem_list) { + DIR* dir = opendir("/sys/devices/system/node"); + struct dirent* entry; + if (dir == NULL) return; + while ((entry = readdir(dir))) { + const char* prefix = "node"; + if (entry->d_type != DT_DIR || + strncmp(entry->d_name, prefix, strlen(prefix)) != 0) { + continue; + } + int numa_node = atoi(entry->d_name + strlen(prefix)); + Topology::MemEntry mem_entry; + mem_entry.name = "cpu:" + std::to_string(numa_node); + mem_entry.numa_node = numa_node; + mem_entry.type = Topology::MEM_HOST; + int nic_id = 0; + for (const auto& device : nic_list) { + if (device.numa_node == numa_node) { + mem_entry.device_list[0].push_back(nic_id++); + } else { + mem_entry.device_list[2].push_back(nic_id++); + } + } + mem_list.push_back(std::move(mem_entry)); + } + (void)closedir(dir); +} + +static void insertFallbackMemEntry(int nic_list_count, + std::vector& mem_list) { + for (auto& entry : mem_list) { + if (entry.name == kWildcardLocation) { + entry.device_list[2].clear(); + for (int i = 0; i < nic_list_count; ++i) + entry.device_list[2].push_back(i); + return; + } + } + Topology::MemEntry new_entry; + new_entry.name = kWildcardLocation; + new_entry.numa_node = -1; + new_entry.type = Topology::MEM_HOST; + for (int i = 0; i < nic_list_count; ++i) + new_entry.device_list[2].push_back(i); + mem_list.push_back(new_entry); +} + +Status SunrisePlatform::probe(std::vector& nic_list, + std::vector& mem_list) { + auto detected_nic = listInfiniBandDevices(); + filterInfiniBandDevices(detected_nic, conf); + for (auto& entry : detected_nic) nic_list.push_back(entry); + insertFallbackMemEntry((int)nic_list.size(), mem_list); + discoverCpuTopology(nic_list, mem_list); + + int device_count = 0; + if (tangGetDeviceCount(&device_count) != tangSuccess || device_count <= 0) { + return Status::OK(); + } + + for (int i = 0; i < device_count; ++i) { + Topology::MemEntry entry; + entry.name = "cuda:" + std::to_string(i); + entry.numa_node = -1; + entry.type = Topology::MEM_CUDA; + for (int nic = 0; nic < (int)nic_list.size(); ++nic) { + entry.device_list[2].push_back(nic); + } + mem_list.push_back(std::move(entry)); + } + return Status::OK(); +} + +MemoryType SunrisePlatform::getMemoryType(void* addr) { + tangPointerAttributes attributes{}; + tangError_t ret = tangPointerGetAttributes(&attributes, addr); + if (ret != tangSuccess) { + int saved_dev = 0; + if (tangGetDevice(&saved_dev) != tangSuccess) saved_dev = -1; + int device_count = 0; + if (tangGetDeviceCount(&device_count) == tangSuccess && + device_count > 0) { + for (int d = 0; d < device_count; ++d) { + if (tangSetDevice(d) != tangSuccess) continue; + ret = tangPointerGetAttributes(&attributes, addr); + if (ret == tangSuccess) break; + } + if (saved_dev >= 0) tangSetDevice(saved_dev); + } + } + if (ret != tangSuccess) return MTYPE_CPU; + if (attributes.type == tangMemoryTypeDevice) return MTYPE_CUDA; + return MTYPE_CPU; +} + +const std::vector SunrisePlatform::getLocation( + void* start, size_t len, bool skip_prefault) { + std::vector entries; + + tangPointerAttributes attributes{}; + tangError_t ret = tangPointerGetAttributes(&attributes, start); + if (ret != tangSuccess) { + int saved_dev = 0; + if (tangGetDevice(&saved_dev) != tangSuccess) saved_dev = -1; + int device_count = 0; + if (tangGetDeviceCount(&device_count) == tangSuccess && + device_count > 0) { + for (int d = 0; d < device_count; ++d) { + if (tangSetDevice(d) != tangSuccess) continue; + ret = tangPointerGetAttributes(&attributes, start); + if (ret == tangSuccess) break; + } + if (saved_dev >= 0) tangSetDevice(saved_dev); + } + } + + if (ret == tangSuccess && attributes.type == tangMemoryTypeDevice) { + entries.push_back( + {(uint64_t)start, len, genCudaNodeName(attributes.device)}); + return entries; + } + + const static size_t kPageSize = 4096; + uintptr_t aligned_start = alignPage((uintptr_t)start); + int n = + (uintptr_t(start) - aligned_start + len + kPageSize - 1) / kPageSize; + std::vector pages(n); + std::vector status(n); + for (int i = 0; i < n; i++) { + pages[i] = reinterpret_cast(aligned_start + i * kPageSize); + } + + if (!skip_prefault) { + prefaultBeforeProbe(pages.data(), n, aligned_start, "SunrisePlatform"); + } + + int rc = numa_move_pages(0, n, pages.data(), nullptr, status.data(), 0); + if (rc != 0) { + entries.push_back({(uint64_t)start, len, kWildcardLocation}); + return entries; + } + + int node = status[0]; + uint64_t start_addr = (uint64_t)start; + uint64_t new_start_addr; + for (int i = 1; i < n; i++) { + if (status[i] != node) { + new_start_addr = alignPage((uint64_t)start) + i * kPageSize; + entries.push_back({start_addr, size_t(new_start_addr - start_addr), + genCpuNodeName(node)}); + start_addr = new_start_addr; + node = status[i]; + } + } + entries.push_back( + {start_addr, (uint64_t)start + len - start_addr, genCpuNodeName(node)}); + return entries; +} + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/runtime/platform.cpp b/mooncake-transfer-engine/tent/src/runtime/platform.cpp index 9f78b811..8c79b29c 100644 --- a/mooncake-transfer-engine/tent/src/runtime/platform.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/platform.cpp @@ -16,6 +16,8 @@ #ifdef USE_CUDA #include "tent/platform/cuda.h" +#elif defined(USE_SUNRISE) +#include "tent/platform/sunrise.h" #elif defined(USE_ASCEND) || defined(USE_ASCEND_DIRECT) #include "tent/platform/ascend.h" #else @@ -31,6 +33,8 @@ Platform& Platform::getLoader(std::shared_ptr conf) { std::call_once(flag, [&]() { #ifdef USE_CUDA g_instance = std::make_shared(conf); +#elif defined(USE_SUNRISE) + g_instance = std::make_shared(conf); #elif defined(USE_ASCEND) || defined(USE_ASCEND_DIRECT) g_instance = std::make_shared(conf); #else diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index 453d9c44..7c6cbf00 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -616,6 +616,7 @@ std::vector TransferEngineImpl::getSupportedTransports( if (transport_list_[MNNVL]) result.push_back(MNNVL); if (transport_list_[NVLINK]) result.push_back(NVLINK); if (transport_list_[RDMA]) result.push_back(RDMA); + if (transport_list_[SUNRISE_LINK]) result.push_back(SUNRISE_LINK); if (transport_list_[AscendDirect]) result.push_back(AscendDirect); if (transport_list_[SHM]) result.push_back(SHM); if (transport_list_[TCP]) result.push_back(TCP); diff --git a/mooncake-transfer-engine/tent/src/runtime/transport_loader.cpp b/mooncake-transfer-engine/tent/src/runtime/transport_loader.cpp index 4d9d4072..7ecd5127 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transport_loader.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transport_loader.cpp @@ -37,6 +37,10 @@ #include "tent/transport/ascend/ascend_direct_transport.h" #endif +#ifdef USE_SUNRISE +#include "tent/transport/sunrise_link/sunrise_link_transport.h" +#endif + namespace mooncake { namespace tent { @@ -83,6 +87,13 @@ Status TransferEngineImpl::loadTransports() { } #endif +#ifdef USE_SUNRISE + if (conf_->get("transports/sunrise_link/enable", true)) { + transport_list_[SUNRISE_LINK] = + std::make_shared(); + } +#endif + return Status::OK(); } diff --git a/mooncake-transfer-engine/tent/src/transport/CMakeLists.txt b/mooncake-transfer-engine/tent/src/transport/CMakeLists.txt index d6037183..e73f7cb0 100644 --- a/mooncake-transfer-engine/tent/src/transport/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/src/transport/CMakeLists.txt @@ -7,18 +7,22 @@ add_subdirectory(gds) add_subdirectory(io_uring) add_subdirectory(bufio) add_subdirectory(ascend) +add_subdirectory(sunrise_link) add_library(tent_transport_all INTERFACE) -foreach(tgt tent_xport_gds - tent_xport_uring - tent_xport_bufio - tent_xport_mnnvl - tent_xport_nvlink - tent_xport_rdma - tent_xport_shm - tent_xport_tcp - tent_xport_ascend_direct) - if (TARGET ${tgt}) - target_link_libraries(tent_transport_all INTERFACE ${tgt}) - endif() +foreach( + tgt + tent_xport_gds + tent_xport_uring + tent_xport_bufio + tent_xport_mnnvl + tent_xport_nvlink + tent_xport_rdma + tent_xport_shm + tent_xport_tcp + tent_xport_ascend_direct + tent_xport_sunrise_link) + if(TARGET ${tgt}) + target_link_libraries(tent_transport_all INTERFACE ${tgt}) + endif() endforeach() diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/context.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/context.cpp index a012bc23..a8e360fe 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/context.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/context.cpp @@ -475,9 +475,9 @@ RdmaContext::MemReg RdmaContext::registerMemReg(void* addr, size_t length, } ibv_mr* entry = verbs_.ibv_reg_mr_default(native_pd_, addr, length, access); if (!entry) { + const void* end = static_cast(addr) + length; PLOG(ERROR) << "Failed to register memory from " << addr << " to " - << (char*)addr + length << " in RDMA device " - << device_name_; + << end << " in RDMA device " << device_name_; return nullptr; } mr_set_mutex_.lock(); @@ -519,9 +519,9 @@ int RdmaContext::unregisterMemReg(MemReg id) { mr_set_mutex_.unlock(); if (verbs_.ibv_dereg_mr(entry)) { + const void* end = static_cast(entry->addr) + entry->length; LOG(ERROR) << "Failed to unregister memory from " << entry->addr - << " to " << (char*)entry->addr + entry->length - << " in RDMA device " << device_name_; + << " to " << end << " in RDMA device " << device_name_; } return 0; diff --git a/mooncake-transfer-engine/tent/src/transport/sunrise_link/CMakeLists.txt b/mooncake-transfer-engine/tent/src/transport/sunrise_link/CMakeLists.txt new file mode 100644 index 00000000..9fa9e020 --- /dev/null +++ b/mooncake-transfer-engine/tent/src/transport/sunrise_link/CMakeLists.txt @@ -0,0 +1,21 @@ +# Copyright 2024 KVCache.AI +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may not +# use this file except in compliance with the License. You may obtain a copy of +# the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations under +# the License. + +if(USE_SUNRISE) + add_library(tent_xport_sunrise_link STATIC sunrise_link_transport.cpp) + target_include_directories(tent_xport_sunrise_link + PUBLIC /usr/local/tangrt/include/) + target_link_libraries(tent_xport_sunrise_link + PUBLIC tent_common glog glog::glog ${SUNRISE_LIBRARIES}) +endif() diff --git a/mooncake-transfer-engine/tent/src/transport/sunrise_link/sunrise_link_transport.cpp b/mooncake-transfer-engine/tent/src/transport/sunrise_link/sunrise_link_transport.cpp new file mode 100644 index 00000000..fdff8c6c --- /dev/null +++ b/mooncake-transfer-engine/tent/src/transport/sunrise_link/sunrise_link_transport.cpp @@ -0,0 +1,1078 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/transport/sunrise_link/sunrise_link_transport.h" + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "tent/common/status.h" +#include "tent/runtime/slab.h" +#include "tent/runtime/control_plane.h" +#include "tent/runtime/topology.h" +#include "tent/common/utils/string_builder.h" + +#include +#include + +namespace mooncake { +namespace tent { + +namespace { + +// Default matches typical Tang RT install; override root and/or arch via env. +static std::string TangRtSharedObjectPath(const char* soname) { + const char* root = std::getenv("MC_TANGRT_ROOT"); + std::string r = (root && root[0]) ? std::string(root) + : std::string("/usr/local/tangrt"); + while (!r.empty() && r.back() == '/') r.pop_back(); + const char* arch = std::getenv("MC_TANGRT_LIB_ARCH"); + std::string arch_dir = + (arch && arch[0]) ? std::string(arch) : std::string("linux-x86_64"); + return r + "/lib/" + arch_dir + "/" + soname; +} + +constexpr int kMaxPhytopoPorts = 10; + +static tangMemcpyKind MemcpyKindForPointers(tangMemoryType st, + tangMemoryType dt) { + if (st == tangMemoryTypeDevice && dt == tangMemoryTypeHost) { + return tangMemcpyDeviceToHost; + } + if (st == tangMemoryTypeHost && dt == tangMemoryTypeDevice) { + return tangMemcpyHostToDevice; + } + if (st == tangMemoryTypeDevice && dt == tangMemoryTypeDevice) { + return tangMemcpyDeviceToDevice; + } + return tangMemcpyHostToHost; +} + +// Sunrise S2 path may allocate mapped GPU memory via tangHostAlloc. +// For kernel/device-copy style paths, convert mapped host pointer into its +// corresponding device pointer when available. +static void NormalizeMappedHostPointer(void** ptr, + tangPointerAttributes* attr) { + if (!ptr || !*ptr || !attr) return; + if (attr->type != tangMemoryTypeHost) return; + + void* dev_ptr = nullptr; + tangError_t ret = tangHostGetDevicePointer(&dev_ptr, *ptr, 0); + if (ret != tangSuccess || !dev_ptr) return; + + tangPointerAttributes dev_attr = {}; + tangPointerGetAttributes(&dev_attr, dev_ptr); + if (dev_attr.type == tangMemoryTypeDevice) { + *ptr = dev_ptr; + *attr = dev_attr; + } +} + +static tangError_t QueryPointerAttrsBestEffort(void* ptr, + tangPointerAttributes* attr, + int preferred_dev) { + if (!ptr || !attr) return tangErrorInvalidValue; + tangError_t ret = tangPointerGetAttributes(attr, ptr); + if (ret == tangSuccess) return ret; + + if (preferred_dev >= 0) { + tangSetDevice(preferred_dev); + ret = tangPointerGetAttributes(attr, ptr); + if (ret == tangSuccess) return ret; + } + + int dev_count = 0; + if (tangGetDeviceCount(&dev_count) != tangSuccess || dev_count <= 0) { + return ret; + } + for (int d = 0; d < dev_count; ++d) { + tangSetDevice(d); + ret = tangPointerGetAttributes(attr, ptr); + if (ret == tangSuccess) return ret; + } + return ret; +} + +// Fallback when tangMemcpyPeer_v2 fails: map peer device memory through PTLink. +static tangError_t C2cMemcpyWithPeerMap(int src_dev, int dst_dev, + const void* src, void* dst, size_t len, + int port_src_to_dst, + int port_dst_to_src) { + tangError_t ret = tangErrorUnknown; + void* mapped = nullptr; + + if (port_src_to_dst >= 0) { + ret = tangDeviceGetPeerPointer(src_dev, port_src_to_dst, dst, &mapped); + if (ret != tangSuccess) { + ret = tangSetDevice(src_dev); + if (ret != tangSuccess) return ret; + ret = tangDeviceGetPeerPointer(src_dev, port_src_to_dst, dst, + &mapped); + } + if (ret == tangSuccess) { + ret = tangMemcpy(mapped, src, len, tangMemcpyDeviceToDevice); + } + return ret; + } + if (port_dst_to_src >= 0) { + ret = tangDeviceGetPeerPointer(dst_dev, port_dst_to_src, + const_cast(src), &mapped); + if (ret != tangSuccess) { + ret = tangSetDevice(dst_dev); + if (ret != tangSuccess) return ret; + ret = tangDeviceGetPeerPointer(dst_dev, port_dst_to_src, + const_cast(src), &mapped); + } + if (ret == tangSuccess) { + ret = tangMemcpy(dst, mapped, len, tangMemcpyDeviceToDevice); + } + return ret; + } + return tangErrorInvalidValue; +} + +} // namespace + +static ptmlReturn_t (*ptmlInit)() = nullptr; +static ptmlReturn_t (*ptmlPtlinkEnableAll)() = nullptr; +static ptmlReturn_t (*ptmlPtlinkPhytopoDetect)(ptmlDevice_t dev, int len, + void* data) = nullptr; +static ptmlReturn_t (*ptmlDeviceGetCount)(int* devCount) = nullptr; + +// Tang memcpy must NOT use one process-wide lock: it serialized every +// tangMemcpyPeer_v2 and made multi-threaded C2C benches ~3 orders of magnitude +// slower than PCCL (which runs many concurrent device copies). We only guard +// tangSetDevice + non-peer tangMemcpy with per-device mutexes; peer copies use +// device ids passed into tangMemcpyPeer_v2 and run without that serialization. + +namespace { + +constexpr int kMaxTangDevLocks = 32; + +static std::mutex& devMutex(int dev) { + static std::mutex table[kMaxTangDevLocks]; + if (dev < 0 || dev >= kMaxTangDevLocks) { + static std::mutex fallback; + return fallback; + } + return table[dev]; +} + +// Serialize tangSetDevice + context-bound ops for a device; for peer pairs, +// lock both devices in a fixed order to avoid deadlock. +struct ScopedTangDeviceOps { + int lo{-1}; + int hi{-1}; + + explicit ScopedTangDeviceOps(int src_dev, int dst_dev) { + if (src_dev < 0 && dst_dev < 0) { + return; + } + if (src_dev < 0 || dst_dev < 0) { + int d = src_dev >= 0 ? src_dev : dst_dev; + devMutex(d).lock(); + lo = d; + return; + } + if (src_dev == dst_dev) { + devMutex(src_dev).lock(); + lo = src_dev; + return; + } + lo = std::min(src_dev, dst_dev); + hi = std::max(src_dev, dst_dev); + devMutex(lo).lock(); + devMutex(hi).lock(); + } + + ~ScopedTangDeviceOps() { + if (hi >= 0) { + devMutex(hi).unlock(); + devMutex(lo).unlock(); + } else if (lo >= 0) { + devMutex(lo).unlock(); + } + } +}; + +} // namespace + +class ThreadLocalTangStreamPool { + public: + tangStream_t getOrCreate(int device_id) { + if (device_id < 0) return nullptr; + auto it = streams_by_device_.find(device_id); + if (it != streams_by_device_.end()) return it->second; + + ScopedTangDeviceOps ctx_lock(device_id, device_id); + int saved_dev = -1; + tangGetDevice(&saved_dev); + tangError_t set_ret = tangSetDevice(device_id); + if (set_ret != tangSuccess) { + LOG(ERROR) << "tangSetDevice(" << device_id + << ") failed for stream creation: " << set_ret << " " + << tangGetErrorString(set_ret); + return nullptr; + } + + tangStream_t stream = nullptr; + tangError_t create_ret = + tangStreamCreateWithFlags(&stream, tangStreamNonBlocking); + if (saved_dev >= 0) tangSetDevice(saved_dev); + if (create_ret != tangSuccess) { + LOG(ERROR) << "tangStreamCreateWithFlags failed: " << create_ret + << " " << tangGetErrorString(create_ret) + << ", device=" << device_id; + return nullptr; + } + streams_by_device_[device_id] = stream; + return stream; + } + + ~ThreadLocalTangStreamPool() { + for (const auto& kv : streams_by_device_) { + if (kv.second) tangStreamDestroy(kv.second); + } + } + + private: + std::unordered_map streams_by_device_; +}; + +thread_local ThreadLocalTangStreamPool tl_stream_pool; + +SunriseLinkTransport::SunriseLinkTransport() : installed_(false) { + // Match NVLink-style caps: routing may see MTYPE_CPU if pointer probe fails + // while the remote segment is VRAM; dram_to_gpu must still be available. + caps.dram_to_dram = true; + caps.dram_to_gpu = true; + caps.gpu_to_dram = true; + caps.gpu_to_gpu = true; +} + +SunriseLinkTransport::~SunriseLinkTransport() { uninstall(); } + +Status SunriseLinkTransport::install(std::string& local_segment_name, + std::shared_ptr metadata, + std::shared_ptr local_topology, + std::shared_ptr conf) { + if (installed_) { + return Status::InvalidArgument( + "SunriseLink transport has been installed" LOC_MARK); + } + + conf_ = conf; + metadata_ = metadata; + + auto status = initSunriseLink(); + if (!status.ok()) { + LOG(WARNING) << "SunriseLink transport initSunriseLink fail"; + return status; + } + + status = detectTopology(); + if (!status.ok()) { + LOG(WARNING) << "SunriseLink transport detectTopology fail"; + return status; + } + + installed_ = true; + async_memcpy_threshold_ = + conf_ + ? conf_->get("transports/sunrise_link/async_memcpy_threshold", 0) * + 1024 + : 0; + LOG(INFO) << "SunriseLink transport installed successfully"; + + return Status::OK(); +} + +Status SunriseLinkTransport::initSunriseLink() { + ptml_handle_ = + dlopen(TangRtSharedObjectPath("libptml_shared.so").c_str(), RTLD_NOW); + if (!ptml_handle_) { + char* error = dlerror(); + LOG(ERROR) << "Failed to load libptml_shared.so: " + << (error ? error : "unknown error"); + return Status::InternalError("Failed to load PTML library"); + } + +#define LOAD_PTML_SYM(handle, symbol, funcptr) \ + do { \ + void* tmp = dlsym(handle, symbol); \ + if (tmp == nullptr) { \ + LOG(ERROR) << "dlsym failed on " << symbol; \ + return Status::InternalError("Failed to load " #symbol); \ + } \ + *(void**)&funcptr = tmp; \ + } while (0) + + LOAD_PTML_SYM(ptml_handle_, "ptmlInit", ptmlInit); + LOAD_PTML_SYM(ptml_handle_, "ptmlPtlinkEnableAll", ptmlPtlinkEnableAll); + LOAD_PTML_SYM(ptml_handle_, "ptmlPtlinkPhytopoDetect", + ptmlPtlinkPhytopoDetect); + LOAD_PTML_SYM(ptml_handle_, "ptmlDeviceGetCount", ptmlDeviceGetCount); + + ptmlReturn_t ret = ptmlInit(); + if (ret != PTML_SUCCESS) { + LOG(ERROR) << "ptmlInit failed, error code: " << ret; + return Status::InternalError("Failed to initialize PTML"); + } + + ret = ptmlPtlinkEnableAll(); + if (ret != PTML_SUCCESS) { + LOG(WARNING) << "ptmlPtlinkEnableAll failed, error code: " << ret; + } + + runtime_lib_handle_ = + dlopen(TangRtSharedObjectPath("libtangrt_shared.so").c_str(), RTLD_NOW); + if (!runtime_lib_handle_) { + char* error = dlerror(); + LOG(ERROR) << "Failed to load libtangrt_shared.so: " + << (error ? error : "unknown error"); + return Status::InternalError("Failed to load Tang library"); + } + + LOG(INFO) << "SunriseLink runtime initialized successfully (PTML + Tang)"; + return Status::OK(); +} + +Status SunriseLinkTransport::detectTopology() { + topology_map_.clear(); + c2c_port_by_pair_.clear(); + local_devices_.clear(); + + int device_count = 0; + ptmlReturn_t pret = ptmlDeviceGetCount(&device_count); + if (pret != PTML_SUCCESS || device_count <= 0) { + LOG(ERROR) << "ptmlDeviceGetCount failed, ret=" << pret + << " count=" << device_count; + return Status::InternalError("ptmlDeviceGetCount failed"); + } + + std::vector phytopo(static_cast(device_count) * + kMaxPhytopoPorts); + + for (int dev = 0; dev < device_count; ++dev) { + memset(phytopo.data() + static_cast(dev) * kMaxPhytopoPorts, 0, + kMaxPhytopoPorts * sizeof(ptPhyTopo_t)); + + pret = ptmlPtlinkPhytopoDetect( + dev, kMaxPhytopoPorts * sizeof(ptPhyTopo_t), + phytopo.data() + static_cast(dev) * kMaxPhytopoPorts); + if (pret != PTML_SUCCESS) { + LOG(WARNING) << "ptmlPtlinkPhytopoDetect failed for device " << dev + << " ret=" << pret; + continue; + } + + SunriseLinkDeviceInfo info; + info.device_id = dev; + + ptPhyTopo_t* row = + phytopo.data() + static_cast(dev) * kMaxPhytopoPorts; + for (int p = 0; p < kMaxPhytopoPorts; ++p) { + if (!row[p].link_status) { + continue; + } + int lc = static_cast(row[p].local_chipid); + int rc = static_cast(row[p].remote_chipid); + int lp = static_cast(row[p].local_port); + auto key = std::make_pair(lc, rc); + if (c2c_port_by_pair_.find(key) == c2c_port_by_pair_.end()) { + c2c_port_by_pair_[key] = lp; + } + + SunriseLinkPeerInfo peer; + peer.peer_device_id = rc; + peer.peer_port = lp; + peer.is_active = true; + topology_map_[dev].push_back(peer); + info.active_ports.push_back(lp); + + LOG(INFO) << "PTLink topo dev=" << dev << " port=" << lp << " link " + << lc << " -> " << rc << " (remote_port=" + << static_cast(row[p].remote_port) << ")"; + } + + local_devices_.push_back(std::move(info)); + } + + LOG(INFO) << "SunriseLink topology: " << device_count << " device(s), " + << c2c_port_by_pair_.size() << " C2C port mapping(s)"; + return Status::OK(); +} + +Status SunriseLinkTransport::allocateSubBatch(SubBatchRef& batch, + size_t max_size) { + auto shm_batch = Slab::Get().allocate(); + if (!shm_batch) { + return Status::InternalError( + "Unable to allocate SunriseLink sub-batch"); + } + batch = shm_batch; + // Slab reuse can leave stale tasks; sub_task_id must index a fresh list. + shm_batch->task_list.clear(); + shm_batch->task_list.reserve(max_size); + shm_batch->max_size = max_size; + // Synchronous copy path (can_async=false in startTransfer) does not rely on + // stream completion; avoid creating thread-local Tang stream here to reduce + // device-context coupling in worker threads. + shm_batch->stream = nullptr; + return Status::OK(); +} + +Status SunriseLinkTransport::freeSubBatch(SubBatchRef& batch) { + auto shm_batch = dynamic_cast(batch); + if (!shm_batch) { + return Status::InvalidArgument( + "Invalid SunriseLink sub-batch" LOC_MARK); + } + shm_batch->task_list.clear(); + Slab::Get().deallocate(shm_batch); + batch = nullptr; + return Status::OK(); +} + +Status SunriseLinkTransport::submitTransferTasks( + SubBatchRef batch, const std::vector& request_list) { + auto shm_batch = dynamic_cast(batch); + if (!shm_batch) { + return Status::InvalidArgument( + "Invalid SunriseLink sub-batch" LOC_MARK); + } + + if (request_list.size() + shm_batch->task_list.size() > + shm_batch->max_size) { + return Status::TooManyRequests("Exceed batch capacity" LOC_MARK); + } + + auto infer_local_dev = [&](void* ptr) -> int { + uint64_t p = reinterpret_cast(ptr); + std::lock_guard guard(registered_memory_mutex_); + for (const auto& entry : registered_memory_) { + uint64_t base = reinterpret_cast(entry.first); + if (base <= p && p < base + entry.second) { + auto it = registered_memory_gpu_id_.find(entry.first); + if (it != registered_memory_gpu_id_.end()) return it->second; + } + } + return -1; + }; + auto infer_local_dev_best_effort = [&](void* ptr) -> int { + int dev = infer_local_dev(ptr); + if (dev >= 0) return dev; + + tangPointerAttributes attr = {}; + int saved_dev = -1; + tangGetDevice(&saved_dev); + tangError_t ret = QueryPointerAttrsBestEffort(ptr, &attr, -1); + if (saved_dev >= 0) tangSetDevice(saved_dev); + if (ret != tangSuccess) return -1; + + NormalizeMappedHostPointer(&ptr, &attr); + if (attr.type == tangMemoryTypeDevice) return attr.device; + return -1; + }; + + for (auto& request : request_list) { + shm_batch->task_list.push_back(SunriseLinkTask{}); + auto& task = shm_batch->task_list[shm_batch->task_list.size() - 1]; + + uint64_t target_addr = request.target_offset; + int target_gpu_id = -1; + int local_gpu_id = infer_local_dev_best_effort(request.source); + if (local_gpu_id >= 0) { + tangError_t sd = tangSetDevice(local_gpu_id); + if (sd != tangSuccess) { + return Status::InternalError( + "Failed to set local Sunrise device before transfer"); + } + } + if (request.target_id != LOCAL_SEGMENT_ID) { + auto status = relocateRemoteAddress( + target_addr, request.length, request.target_id, &target_gpu_id); + if (!status.ok()) return status; + } + task.target_addr = target_addr; + task.request = request; + task.status_word = TransferStatusEnum::PENDING; + task.transferred_bytes = 0; + task.is_tang_ipc = (request.target_id != LOCAL_SEGMENT_ID); + task.tang_gpu_id = target_gpu_id; + + auto status = startTransfer(&task, shm_batch); + if (!status.ok()) return status; + } + + return Status::OK(); +} + +Status SunriseLinkTransport::startTransfer(SunriseLinkTask* task, + SunriseLinkSubBatch* batch) { + void* src = nullptr; + void* dst = nullptr; + + const uint64_t resolved_target = task->target_addr; + + if (task->request.opcode == Request::READ) { + dst = task->request.source; + src = reinterpret_cast(resolved_target); + } else { + src = task->request.source; + dst = reinterpret_cast(resolved_target); + } + + const size_t len = task->request.length; + tangStream_t stream = nullptr; + int current_dev = -1; + tangGetDevice(¤t_dev); + auto infer_local_dev = [&](void* ptr) -> int { + uint64_t p = reinterpret_cast(ptr); + std::lock_guard guard(registered_memory_mutex_); + for (const auto& entry : registered_memory_) { + uint64_t base = reinterpret_cast(entry.first); + if (base <= p && p < base + entry.second) { + auto it = registered_memory_gpu_id_.find(entry.first); + if (it != registered_memory_gpu_id_.end()) return it->second; + } + } + return -1; + }; + auto infer_local_dev_best_effort = [&](void* ptr) -> int { + int dev = infer_local_dev(ptr); + if (dev >= 0) return dev; + + tangPointerAttributes attr = {}; + int saved_dev = -1; + tangGetDevice(&saved_dev); + tangError_t ret = QueryPointerAttrsBestEffort(ptr, &attr, -1); + if (saved_dev >= 0) tangSetDevice(saved_dev); + if (ret != tangSuccess) return -1; + + NormalizeMappedHostPointer(&ptr, &attr); + if (attr.type == tangMemoryTypeDevice) return attr.device; + return -1; + }; + + // Follow ipcC2cTest style: for IPC path, prefer explicit gpu ids and + // tangMemcpyPeer directly, instead of relying on pointer attributes. + if (task->is_tang_ipc && task->tang_gpu_id >= 0) { + int remote_dev = task->tang_gpu_id; + int local_dev = -1; + if (task->request.opcode == Request::WRITE) { + local_dev = infer_local_dev_best_effort(src); + } else { + local_dev = infer_local_dev_best_effort(dst); + } + if (local_dev < 0) local_dev = current_dev; + + int src_dev = + (task->request.opcode == Request::WRITE) ? local_dev : remote_dev; + int dst_dev = + (task->request.opcode == Request::WRITE) ? remote_dev : local_dev; + + // Same GPU id on initiator and target (typical single-card P2P bench): + // pointers are IPC handles into the same device context; tangMemcpyPeer + // is not valid when src_dev == dst_dev. + if (src_dev == dst_dev) { + ScopedTangDeviceOps ctx_lock(src_dev, dst_dev); + tangSetDevice(src_dev); + tangError_t ret = + tangMemcpy(dst, src, len, tangMemcpyDeviceToDevice); + if (ret != tangSuccess) { + LOG(ERROR) + << "SunriseLink IPC same-device memcpy failed, tang ret=" + << ret << " " << tangGetErrorString(ret) + << " dev=" << src_dev; + task->status_word = TransferStatusEnum::FAILED; + return Status::InternalError("SunriseLink transfer failed"); + } + task->transferred_bytes = len; + task->status_word = TransferStatusEnum::COMPLETED; + return Status::OK(); + } + + tangError_t ret = tangSuccess; + // Prefer ipcC2cTest ordering first (current=dst_dev), then retry with + // current=src_dev since some Tang stacks are context-sensitive for peer + // copy direction. + auto try_peer_copy = [&]() -> tangError_t { + tangError_t r = tangMemcpyPeer(dst, dst_dev, src, src_dev, len); + if (r != tangSuccess) { + r = tangMemcpyPeer_v2(dst, dst_dev, src, src_dev, len); + } + return r; + }; + // ipcC2cTest uses synchronous tangMemcpyPeer only. + ret = try_peer_copy(); + if (ret != tangSuccess) { + int p_sd = -1; + int p_ds = -1; + auto it_sd = + c2c_port_by_pair_.find(std::make_pair(src_dev, dst_dev)); + if (it_sd != c2c_port_by_pair_.end()) p_sd = it_sd->second; + auto it_ds = + c2c_port_by_pair_.find(std::make_pair(dst_dev, src_dev)); + if (it_ds != c2c_port_by_pair_.end()) p_ds = it_ds->second; + ret = C2cMemcpyWithPeerMap(src_dev, dst_dev, src, dst, len, p_sd, + p_ds); + } + if (ret != tangSuccess) { + LOG(ERROR) << "SunriseLink IPC peer copy failed, tang ret=" << ret + << " " << tangGetErrorString(ret) + << " src_dev=" << src_dev << " dst_dev=" << dst_dev + << " remote_dev=" << remote_dev + << " local_dev=" << local_dev; + task->status_word = TransferStatusEnum::FAILED; + return Status::InternalError("SunriseLink transfer failed"); + } + task->transferred_bytes = len; + task->status_word = TransferStatusEnum::COMPLETED; + return Status::OK(); + } + + tangPointerAttributes sa = {}; + tangPointerAttributes da = {}; + if (task->tang_gpu_id >= 0) tangSetDevice(task->tang_gpu_id); + + // Intentionally no mutex here: a global lock on pointer-attribute queries + // serialized all peer copies across threads and capped throughput at a few + // MiB/s in multi-threaded benches. Tang is expected to tolerate concurrent + // attribute reads for different pointers. + tangError_t src_attr_ret = + QueryPointerAttrsBestEffort(src, &sa, task->tang_gpu_id); + tangError_t dst_attr_ret = + QueryPointerAttrsBestEffort(dst, &da, task->tang_gpu_id); + NormalizeMappedHostPointer(&src, &sa); + NormalizeMappedHostPointer(&dst, &da); + + int src_dev = (sa.type == tangMemoryTypeDevice) ? sa.device : -1; + int dst_dev = (da.type == tangMemoryTypeDevice) ? da.device : -1; + int remote_dev_hint = task->tang_gpu_id; + if (task->request.opcode == Request::WRITE) { + if (dst_dev < 0 && remote_dev_hint >= 0) dst_dev = remote_dev_hint; + if (src_dev < 0 && current_dev >= 0) src_dev = current_dev; + } else { + if (src_dev < 0 && remote_dev_hint >= 0) src_dev = remote_dev_hint; + if (dst_dev < 0 && current_dev >= 0) dst_dev = current_dev; + } + if (task->request.opcode == Request::WRITE && src_dev < 0) { + int hinted = infer_local_dev(src); + if (hinted >= 0) src_dev = hinted; + } else if (task->request.opcode == Request::READ && dst_dev < 0) { + int hinted = infer_local_dev(dst); + if (hinted >= 0) dst_dev = hinted; + } + + tangError_t ret = tangSuccess; + const int stream_dev = (dst_dev >= 0) ? dst_dev : src_dev; + if (stream_dev >= 0) { + stream = tl_stream_pool.getOrCreate(stream_dev); + } + const bool can_async = + stream && async_memcpy_threshold_ > 0 && len >= async_memcpy_threshold_; + batch->stream = reinterpret_cast(stream); + + if (task->is_tang_ipc && src_dev >= 0 && dst_dev >= 0 && + src_dev == dst_dev) { + ScopedTangDeviceOps ctx_lock(src_dev, dst_dev); + tangSetDevice(src_dev); + if (can_async) { + ret = tangMemcpyAsync(dst, src, len, tangMemcpyDeviceToDevice, + stream); + if (ret == tangSuccess) { + return Status::OK(); + } + } + ret = tangMemcpy(dst, src, len, tangMemcpyDeviceToDevice); + if (ret != tangSuccess) { + LOG(ERROR) + << "SunriseLink IPC local-device memcpy failed, tang ret=" + << ret << " " << tangGetErrorString(ret) << " dev=" << src_dev; + task->status_word = TransferStatusEnum::FAILED; + return Status::InternalError("SunriseLink transfer failed"); + } + task->transferred_bytes = len; + task->status_word = TransferStatusEnum::COMPLETED; + return Status::OK(); + } + + if (src_dev >= 0 && dst_dev >= 0 && + (src_dev != dst_dev || task->is_tang_ipc)) { + auto collect_ports = [&](int local, int peer) { + std::vector ports; + auto it = topology_map_.find(local); + if (it == topology_map_.end()) return ports; + for (const auto& p : it->second) { + if (p.peer_device_id == peer && p.is_active) { + ports.push_back(p.peer_port); + } + } + std::sort(ports.begin(), ports.end()); + ports.erase(std::unique(ports.begin(), ports.end()), ports.end()); + return ports; + }; + + auto ports_sd = collect_ports(src_dev, dst_dev); + auto ports_ds = collect_ports(dst_dev, src_dev); + if (ports_sd.empty()) { + auto it = c2c_port_by_pair_.find(std::make_pair(src_dev, dst_dev)); + if (it != c2c_port_by_pair_.end()) ports_sd.push_back(it->second); + } + if (ports_ds.empty()) { + auto it = c2c_port_by_pair_.find(std::make_pair(dst_dev, src_dev)); + if (it != c2c_port_by_pair_.end()) ports_ds.push_back(it->second); + } + constexpr size_t kMultiPortThreshold = 4ull << 20; // 4 MiB + const bool enable_port_striping = + conf_ ? conf_->get("transports/sunrise_link/enable_port_striping", + false) + : false; + if (enable_port_striping && !ports_sd.empty() && ports_sd.size() > 1 && + len >= kMultiPortThreshold) { + size_t lanes = ports_sd.size(); + size_t offset = 0; + const size_t base = len / lanes; + const size_t rem = len % lanes; + for (size_t lane = 0; lane < lanes; ++lane) { + size_t piece = base + (lane < rem ? 1 : 0); + size_t lane_offset = offset; + offset += piece; + int p_sd = ports_sd[lane]; + ret = C2cMemcpyWithPeerMap( + src_dev, dst_dev, static_cast(src) + lane_offset, + static_cast(dst) + lane_offset, piece, p_sd, -1); + if (ret != tangSuccess) { + break; + } + } + if (ret == tangSuccess) { + task->transferred_bytes = len; + task->status_word = TransferStatusEnum::COMPLETED; + return Status::OK(); + } + } + + auto peer_sync = [&]() -> tangError_t { + tangError_t r = tangMemcpyPeer(dst, dst_dev, src, src_dev, len); + if (r != tangSuccess) { + r = tangMemcpyPeer_v2(dst, dst_dev, src, src_dev, len); + } + if (r != tangSuccess) { + int p_sd = !ports_sd.empty() ? ports_sd.front() : -1; + int p_ds = !ports_ds.empty() ? ports_ds.front() : -1; + r = C2cMemcpyWithPeerMap(src_dev, dst_dev, src, dst, len, p_sd, + p_ds); + } + return r; + }; + + // Synchronous tangMemcpyPeer only (see can_async above). + (void)can_async; + ret = peer_sync(); + } else if (src_dev >= 0 && dst_dev >= 0 && src_dev == dst_dev) { + ScopedTangDeviceOps ctx_lock(src_dev, dst_dev); + tangSetDevice(src_dev); + if (can_async) { + ret = tangMemcpyAsync(dst, src, len, tangMemcpyDeviceToDevice, + stream); + if (ret == tangSuccess) { + return Status::OK(); + } + } + ret = tangMemcpy(dst, src, len, tangMemcpyDeviceToDevice); + } else { + ScopedTangDeviceOps ctx_lock(src_dev, dst_dev); + if (src_dev >= 0) { + tangSetDevice(src_dev); + } else if (dst_dev >= 0) { + tangSetDevice(dst_dev); + } + tangMemcpyKind kind = MemcpyKindForPointers(sa.type, da.type); + if (can_async) { + ret = tangMemcpyAsync(dst, src, len, kind, stream); + if (ret == tangSuccess) { + return Status::OK(); + } + } + ret = tangMemcpy(dst, src, len, kind); + } + + if (ret != tangSuccess) { + LOG(ERROR) << "SunriseLink transfer failed, tang ret=" << ret << " " + << tangGetErrorString(ret) << " src_dev=" << src_dev + << " dst_dev=" << dst_dev << " src_attr_ret=" << src_attr_ret + << " dst_attr_ret=" << dst_attr_ret + << " task.tang_gpu_id=" << task->tang_gpu_id; + task->status_word = TransferStatusEnum::FAILED; + return Status::InternalError("SunriseLink transfer failed"); + } + + task->transferred_bytes = len; + task->status_word = TransferStatusEnum::COMPLETED; + return Status::OK(); +} + +Status SunriseLinkTransport::getTransferStatus(SubBatchRef batch, int task_id, + TransferStatus& status) { + auto shm_batch = dynamic_cast(batch); + if (!shm_batch) { + return Status::InvalidArgument( + "Invalid SunriseLink sub-batch" LOC_MARK); + } + if (task_id < 0 || task_id >= (int)shm_batch->task_list.size()) { + return Status::InvalidArgument("Invalid task id" LOC_MARK); + } + auto& task = shm_batch->task_list[task_id]; + if (task.status_word == TransferStatusEnum::PENDING) { + tangStream_t stream = reinterpret_cast(shm_batch->stream); + if (stream) { + tangError_t err = tangStreamQuery(stream); + if (err == tangSuccess) { + tangStreamSynchronize(stream); + task.transferred_bytes = task.request.length; + task.status_word = TransferStatusEnum::COMPLETED; + } else if (err != tangErrorNotReady) { + task.status_word = TransferStatusEnum::FAILED; + } + } else { + LOG(ERROR) + << "SunriseLink getTransferStatus: PENDING with null stream; " + "cannot complete async bookkeeping"; + task.status_word = TransferStatusEnum::FAILED; + } + } + status = TransferStatus{task.status_word, task.transferred_bytes}; + return Status::OK(); +} + +Status SunriseLinkTransport::addMemoryBuffer(BufferDesc& desc, + const MemoryOptions& options) { + LocationParser location(desc.location); + auto status = + registerMemory(reinterpret_cast(desc.addr), desc.length); + if (!status.ok()) return status; + if (location.type() == "cuda" || location.type() == "npu") { + { + std::lock_guard guard(registered_memory_mutex_); + registered_memory_gpu_id_[reinterpret_cast(desc.addr)] = + location.index(); + } + int saved_dev = -1; + tangGetDevice(&saved_dev); + if (location.index() >= 0) { + tangError_t sd = tangSetDevice(location.index()); + if (sd != tangSuccess) { + LOG(ERROR) + << "tangSetDevice before tangIpcGetMemHandle failed: " << sd + << " " << tangGetErrorString(sd); + return Status::InternalError("tangSetDevice failed"); + } + } + tangIpcMemHandle_t handle; + auto ret = + tangIpcGetMemHandle(&handle, reinterpret_cast(desc.addr)); + if (saved_dev >= 0) { + tangSetDevice(saved_dev); + } + if (ret != tangSuccess) { + LOG(ERROR) << "tangIpcGetMemHandle failed: " << ret << " " + << tangGetErrorString(ret); + return Status::InternalError("tangIpcGetMemHandle failed"); + } + desc.shm_path = + serializeBinaryData(&handle, sizeof(tangIpcMemHandle_t)); + } else if (location.type() == "cpu" || + location.type() == kWildcardLocation) { + std::lock_guard guard(registered_memory_mutex_); + registered_memory_gpu_id_[reinterpret_cast(desc.addr)] = -1; + } + desc.transports.push_back(TransportType::SUNRISE_LINK); + return Status::OK(); +} + +Status SunriseLinkTransport::registerMemory(void* addr, size_t size) { + std::lock_guard guard(registered_memory_mutex_); + registered_memory_[addr] = size; + return Status::OK(); +} + +Status SunriseLinkTransport::deregisterMemory(void* addr) { + std::lock_guard guard(registered_memory_mutex_); + auto it = registered_memory_.find(addr); + if (it != registered_memory_.end()) { + registered_memory_.erase(it); + registered_memory_gpu_id_.erase(addr); + } + return Status::OK(); +} + +Status SunriseLinkTransport::removeMemoryBuffer(BufferDesc& desc) { + desc.shm_path.clear(); + return deregisterMemory(reinterpret_cast(desc.addr)); +} + +Status SunriseLinkTransport::relocateRemoteAddress(uint64_t& dest_addr, + uint64_t length, + uint64_t target_id, + int* target_gpu_id) { + thread_local IpcRelocateMap tl_relocate_map; + if (tl_relocate_map.empty()) { + RWSpinlock::ReadGuard guard(relocate_lock_); + tl_relocate_map = relocate_map_; + } + + for (auto& entry : tl_relocate_map[target_id]) { + if (entry.first <= dest_addr && + dest_addr + length <= entry.first + entry.second.length) { + dest_addr = dest_addr - entry.first + + reinterpret_cast(entry.second.dev_ptr); + if (target_gpu_id) *target_gpu_id = entry.second.gpu_id; + return Status::OK(); + } + } + + RWSpinlock::WriteGuard guard(relocate_lock_); + SegmentDesc* desc = nullptr; + auto status = metadata_->segmentManager().getRemoteCached(desc, target_id); + if (!status.ok()) return status; + + auto buffer = desc->findBuffer(dest_addr, length); + if (!buffer || buffer->shm_path.empty()) { + // Cache may be stale right after remote registration; refresh once. + metadata_->segmentManager().invalidateRemote(target_id); + status = metadata_->segmentManager().getRemoteCached(desc, target_id); + if (!status.ok()) return status; + buffer = desc->findBuffer(dest_addr, length); + } + if (!buffer || buffer->shm_path.empty()) { + return Status::InvalidArgument( + "Requested address is not in registered Sunrise buffer" LOC_MARK); + } + + if (!relocate_map_[target_id].count(buffer->addr)) { + LocationParser location(buffer->location); + if (location.type() != "cuda" && location.type() != "npu") { + return Status::InvalidArgument( + "Requested address is not in registered CUDA/NPU " + "buffer" LOC_MARK); + } + std::vector decoded; + deserializeBinaryData(buffer->shm_path, decoded); + if (decoded.size() != sizeof(tangIpcMemHandle_t)) { + return Status::InvalidArgument( + "Invalid Sunrise IPC handle size" LOC_MARK); + } + tangIpcMemHandle_t handle; + memcpy(&handle, decoded.data(), sizeof(handle)); + + void* opened_ptr = nullptr; + int saved_dev = -1; + tangGetDevice(&saved_dev); + tangError_t sd = tangSetDevice(location.index()); + if (sd != tangSuccess) { + LOG(ERROR) << "tangSetDevice before tangIpcOpenMemHandle failed: " + << sd << " " << tangGetErrorString(sd) + << ", target_gpu=" << location.index() + << ", saved_dev=" << saved_dev; + return Status::InternalError("tangSetDevice failed"); + } + tangError_t ret = tangIpcOpenMemHandle(&opened_ptr, handle, + tangIpcMemLazyEnablePeerAccess); + if (saved_dev >= 0) { + tangSetDevice(saved_dev); + } + if (ret != tangSuccess) { + LOG(ERROR) << "tangIpcOpenMemHandle failed: " << ret << " " + << tangGetErrorString(ret); + return Status::InternalError("tangIpcOpenMemHandle failed"); + } + + OpenedIpcEntry ipc_entry; + ipc_entry.dev_ptr = opened_ptr; + ipc_entry.length = buffer->length; + ipc_entry.gpu_id = location.index(); + relocate_map_[target_id][buffer->addr] = ipc_entry; + tl_relocate_map = relocate_map_; + } + + auto opened_ptr = relocate_map_[target_id][buffer->addr].dev_ptr; + if (target_gpu_id) { + *target_gpu_id = relocate_map_[target_id][buffer->addr].gpu_id; + } + dest_addr = + dest_addr - buffer->addr + reinterpret_cast(opened_ptr); + return Status::OK(); +} + +Status SunriseLinkTransport::uninstall() { + if (installed_) { + std::vector reg_addrs; + { + std::lock_guard guard(registered_memory_mutex_); + reg_addrs.reserve(registered_memory_.size()); + for (auto& entry : registered_memory_) { + reg_addrs.push_back(entry.first); + } + } + for (void* addr : reg_addrs) { + auto status = deregisterMemory(addr); + if (!status.ok()) return status; + } + { + std::lock_guard guard(registered_memory_mutex_); + registered_memory_.clear(); + registered_memory_gpu_id_.clear(); + } + + if (runtime_lib_handle_) { + dlclose(runtime_lib_handle_); + runtime_lib_handle_ = nullptr; + } + + for (auto& seg_entry : relocate_map_) { + for (auto& entry : seg_entry.second) { + if (entry.second.dev_ptr) { + tangIpcCloseMemHandle(entry.second.dev_ptr); + } + } + } + relocate_map_.clear(); + + if (ptml_handle_) { + dlclose(ptml_handle_); + ptml_handle_ = nullptr; + } + + metadata_.reset(); + installed_ = false; + LOG(INFO) << "SunriseLink transport uninstalled"; + } + return Status::OK(); +} + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/tests/CMakeLists.txt b/mooncake-transfer-engine/tent/tests/CMakeLists.txt index bd13e5ef..6777a84b 100644 --- a/mooncake-transfer-engine/tent/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/tests/CMakeLists.txt @@ -58,6 +58,19 @@ add_executable(tent_endpoint_lifecycle_test endpoint_lifecycle_test.cpp) target_link_libraries(tent_endpoint_lifecycle_test PRIVATE gtest gtest_main) add_test(NAME tent_endpoint_lifecycle_test COMMAND tent_endpoint_lifecycle_test) +if(USE_SUNRISE) + add_executable(tent_sunrise_link_transport_test + sunrise_link_transport_test.cpp) + target_link_libraries(tent_sunrise_link_transport_test PRIVATE gtest + gtest_main + tent_link_group) + target_include_directories( + tent_sunrise_link_transport_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include + ${CMAKE_CURRENT_SOURCE_DIR}/../../include) + add_test(NAME tent_sunrise_link_transport_test + COMMAND tent_sunrise_link_transport_test) +endif() add_executable(tent_fault_proxy_test fault_proxy_test.cpp) target_link_libraries(tent_fault_proxy_test PRIVATE gtest gtest_main tent_link_group) diff --git a/mooncake-transfer-engine/tent/tests/sunrise_link_transport_test.cpp b/mooncake-transfer-engine/tent/tests/sunrise_link_transport_test.cpp new file mode 100644 index 00000000..a49993c9 --- /dev/null +++ b/mooncake-transfer-engine/tent/tests/sunrise_link_transport_test.cpp @@ -0,0 +1,209 @@ +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "gpu_vendor/sunrise.h" +#include "tent/common/config.h" +#include "tent/common/types.h" +#include "tent/transfer_engine.h" + +namespace mooncake { +namespace tent { +namespace { + +void CheckTangError(tangError_t result, const char* message) { + ASSERT_EQ(result, tangSuccess) + << message << " (error=" << result + << ", detail=" << tangGetErrorString(result) << ")"; +} + +void WaitBatchDone(TransferEngine* engine, BatchID batch_id) { + TransferStatus status; + for (int i = 0; i < 5000; ++i) { + auto s = engine->getTransferStatus(batch_id, status); + ASSERT_TRUE(s.ok()) << "getTransferStatus failed: " << s.ToString(); + if (status.s == TransferStatusEnum::COMPLETED) return; + ASSERT_NE(status.s, TransferStatusEnum::FAILED); + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + FAIL() << "timeout waiting batch completion"; +} + +void RunWriteReadCase(int server_gpu, int client_gpu) { + constexpr size_t kDataLength = 4 * 1024 * 1024; + const std::string server_local_name = + "sunrise_ut_server_" + std::to_string(getpid()); + + int ready_pipe[2]; + int stop_pipe[2]; + ASSERT_EQ(pipe(ready_pipe), 0); + ASSERT_EQ(pipe(stop_pipe), 0); + + pid_t child = fork(); + ASSERT_GE(child, 0); + if (child == 0) { + close(ready_pipe[0]); + close(stop_pipe[1]); + auto server_conf = std::make_shared(); + server_conf->set("metadata_type", "p2p"); + server_conf->set("metadata_servers", "P2PHANDSHAKE"); + server_conf->set("local_segment_name", server_local_name); + server_conf->set("transports/sunrise_link/enable", true); + auto server_engine = std::make_unique(server_conf); + if (!server_engine->available()) _exit(2); + void* server_buf = nullptr; + if (tangSetDevice(server_gpu) != tangSuccess) _exit(3); + if (tangMalloc(&server_buf, kDataLength * 2) != tangSuccess) _exit(4); + MemoryOptions server_opts; + server_opts.location = "cuda:" + std::to_string(server_gpu); + auto s = server_engine->registerLocalMemory(server_buf, kDataLength * 2, + server_opts); + if (!s.ok()) _exit(5); + const std::string seg = server_engine->getSegmentName(); + uint32_t len = static_cast(seg.size()); + if (write(ready_pipe[1], &len, sizeof(len)) != sizeof(len)) _exit(6); + if (write(ready_pipe[1], seg.data(), len) != static_cast(len)) + _exit(7); + char stop = 0; + (void)read(stop_pipe[0], &stop, 1); + (void)server_engine->unregisterLocalMemory(server_buf); + tangSetDevice(server_gpu); + tangFree(server_buf); + _exit(0); + } + + close(ready_pipe[1]); + close(stop_pipe[0]); + uint32_t seg_len = 0; + ssize_t got = read(ready_pipe[0], &seg_len, sizeof(seg_len)); + if (got != static_cast(sizeof(seg_len))) { + int wstatus = 0; + (void)waitpid(child, &wstatus, 0); + if (got == 0 && WIFEXITED(wstatus)) { + GTEST_SKIP() << "server init failed for case " << server_gpu << "->" + << client_gpu + << ", child exit=" << WEXITSTATUS(wstatus); + } + ASSERT_EQ(got, static_cast(sizeof(seg_len))); + } + ASSERT_GT(seg_len, 0u); + std::string server_segment(seg_len, '\0'); + ASSERT_EQ(read(ready_pipe[0], server_segment.data(), seg_len), + static_cast(seg_len)); + + int gpu_count = 0; + CheckTangError(tangGetDeviceCount(&gpu_count), "tangGetDeviceCount"); + if (gpu_count <= 0) { + const char stop = 'Q'; + (void)write(stop_pipe[1], &stop, 1); + int wstatus = 0; + (void)waitpid(child, &wstatus, 0); + GTEST_SKIP() << "no sunrise GPU detected"; + } + if (server_gpu < 0 || client_gpu < 0 || server_gpu >= gpu_count || + client_gpu >= gpu_count) { + const char stop = 'Q'; + (void)write(stop_pipe[1], &stop, 1); + int wstatus = 0; + (void)waitpid(child, &wstatus, 0); + GTEST_SKIP() << "insufficient GPUs for case " << server_gpu << "->" + << client_gpu << ", detected: " << gpu_count; + } + + auto client_conf = std::make_shared(); + client_conf->set("metadata_type", "p2p"); + client_conf->set("metadata_servers", "P2PHANDSHAKE"); + client_conf->set("local_segment_name", "sunrise_ut_client"); + client_conf->set("transports/sunrise_link/enable", true); + auto client_engine = std::make_unique(client_conf); + ASSERT_TRUE(client_engine->available()); + + void* client_buf = nullptr; + CheckTangError(tangSetDevice(client_gpu), "tangSetDevice(client)"); + CheckTangError(tangMalloc(&client_buf, kDataLength * 2), + "tangMalloc(client)"); + MemoryOptions client_opts; + client_opts.location = "cuda:" + std::to_string(client_gpu); + auto s = client_engine->registerLocalMemory(client_buf, kDataLength * 2, + client_opts); + ASSERT_TRUE(s.ok()) << s.ToString(); + + SegmentID segment_id = 0; + for (int i = 0; i < 100; ++i) { + s = client_engine->openSegment(segment_id, server_segment); + if (s.ok()) break; + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + } + ASSERT_TRUE(s.ok()) << s.ToString(); + + SegmentInfo segment_info; + s = client_engine->getSegmentInfo(segment_id, segment_info); + ASSERT_TRUE(s.ok()) << s.ToString(); + ASSERT_FALSE(segment_info.buffers.empty()); + const uint64_t remote_base = segment_info.buffers[0].base; + + std::vector host_data(kDataLength, 'S'); + CheckTangError(tangSetDevice(client_gpu), "tangSetDevice(client copy)"); + CheckTangError(tangMemcpy(client_buf, host_data.data(), kDataLength, + tangMemcpyHostToDevice), + "tangMemcpy H2D"); + + BatchID batch = client_engine->allocateBatch(1); + Request req; + req.opcode = Request::WRITE; + req.length = kDataLength; + req.source = static_cast(client_buf); + req.target_id = segment_id; + req.target_offset = remote_base; + s = client_engine->submitTransfer(batch, {req}); + ASSERT_TRUE(s.ok()) << s.ToString(); + WaitBatchDone(client_engine.get(), batch); + s = client_engine->freeBatch(batch); + ASSERT_TRUE(s.ok()) << s.ToString(); + + batch = client_engine->allocateBatch(1); + req.opcode = Request::READ; + req.source = static_cast(client_buf) + kDataLength; + s = client_engine->submitTransfer(batch, {req}); + ASSERT_TRUE(s.ok()) << s.ToString(); + WaitBatchDone(client_engine.get(), batch); + s = client_engine->freeBatch(batch); + ASSERT_TRUE(s.ok()) << s.ToString(); + + std::vector host_check(kDataLength); + CheckTangError(tangSetDevice(client_gpu), "tangSetDevice(client verify)"); + CheckTangError(tangMemcpy(host_check.data(), + static_cast(client_buf) + kDataLength, + kDataLength, tangMemcpyDeviceToHost), + "tangMemcpy D2H"); + for (size_t i = 0; i < kDataLength; ++i) ASSERT_EQ(host_check[i], 'S'); + + s = client_engine->unregisterLocalMemory(client_buf); + ASSERT_TRUE(s.ok()) << s.ToString(); + CheckTangError(tangSetDevice(client_gpu), "tangSetDevice(client free)"); + CheckTangError(tangFree(client_buf), "tangFree(client)"); + + const char stop = 'Q'; + ASSERT_EQ(write(stop_pipe[1], &stop, 1), 1); + int wstatus = 0; + ASSERT_EQ(waitpid(child, &wstatus, 0), child); + ASSERT_TRUE(WIFEXITED(wstatus)); + ASSERT_EQ(WEXITSTATUS(wstatus), 0); +} + +TEST(SunriseLinkTransportTest, WriteAndRead_0_to_1) { RunWriteReadCase(0, 1); } + +TEST(SunriseLinkTransportTest, WriteAndRead_1_to_0) { RunWriteReadCase(1, 0); } + +TEST(SunriseLinkTransportTest, WriteAndRead_2_to_4) { RunWriteReadCase(2, 4); } + +} // namespace +} // namespace tent +} // namespace mooncake -- 2.34.1 From 55790b33d25fe9ac04c2862ea2f7503de880e705 Mon Sep 17 00:00:00 2001 From: Stary Date: Tue, 28 Apr 2026 10:36:43 +0800 Subject: [PATCH 015/382] fix (CI): hardcode Ascend mirror URL and remove pull_request_target routing (#1992) Signed-off-by: staryxchen --- .github/workflows/ci.yml | 16 ++++++------ .github/workflows/ci_ascend.yml | 6 ++--- .../dependencies_ascend_installation.sh | 25 ++++++++++++++++--- 3 files changed, 32 insertions(+), 15 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 45d565b6..7bcf571b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -657,10 +657,10 @@ jobs: spell-check: name: Spell Check with Typos if: >- - github.event_name == 'push' || - github.event_name == 'workflow_dispatch' || - github.event.action == 'opened' || - contains(github.event.pull_request.labels.*.name, 'run-ci') + (github.event_name == 'push' || + github.event_name == 'workflow_dispatch' || + github.event.action == 'opened' || + contains(github.event.pull_request.labels.*.name, 'run-ci')) runs-on: ubuntu-22.04 steps: - name: Checkout Actions Repository @@ -673,10 +673,10 @@ jobs: clang-format: name: Check code format if: >- - github.event_name == 'push' || - github.event_name == 'workflow_dispatch' || - github.event.action == 'opened' || - contains(github.event.pull_request.labels.*.name, 'run-ci') + (github.event_name == 'push' || + github.event_name == 'workflow_dispatch' || + github.event.action == 'opened' || + contains(github.event.pull_request.labels.*.name, 'run-ci')) runs-on: ubuntu-22.04 steps: - name: Checkout Actions Repository diff --git a/.github/workflows/ci_ascend.yml b/.github/workflows/ci_ascend.yml index c3f39c98..28df9b2f 100644 --- a/.github/workflows/ci_ascend.yml +++ b/.github/workflows/ci_ascend.yml @@ -4,7 +4,7 @@ on: workflow_call: inputs: checkout_ref: - description: 'Git ref to checkout (PR head SHA for pull_request_target)' + description: 'Git ref to checkout (PR head SHA)' required: false type: string @@ -49,7 +49,7 @@ jobs: if: steps.checkout_code.outcome == 'failure' shell: bash env: - ASCEND_GITHUB_MIRROR_URLS: ${{ vars.ASCEND_GITHUB_MIRROR_URLS }} + ASCEND_GITHUB_MIRROR_URLS: 'https://ghfast.top/' CHECKOUT_REF: ${{ inputs.checkout_ref || github.sha }} run: | set -euo pipefail @@ -108,7 +108,7 @@ jobs: - name: Configure CMake shell: bash env: - ASCEND_GITHUB_MIRROR_URLS: ${{ vars.ASCEND_GITHUB_MIRROR_URLS }} + ASCEND_GITHUB_MIRROR_URLS: 'https://ghfast.top/' run: | source /usr/local/Ascend/cann-9.0.0/set_env.sh pwd diff --git a/scripts/ascend/dependencies_ascend_installation.sh b/scripts/ascend/dependencies_ascend_installation.sh index 9fc316ec..d327f2be 100644 --- a/scripts/ascend/dependencies_ascend_installation.sh +++ b/scripts/ascend/dependencies_ascend_installation.sh @@ -18,12 +18,29 @@ #!/bin/bash +# Try git clone with GitHub mirror fallback via https://ghfast.top/ +git_with_github_mirror_fallback() { + local repo_dir="$1" + local repo_url="$2" + shift 2 + + if git clone "$repo_url" "$repo_dir" "$@"; then + return 0 + fi + + echo "Direct clone failed, retrying with mirror https://ghfast.top/" + rm -rf "$repo_dir" + local mirror_url="https://ghfast.top/${repo_url}" + git clone "$mirror_url" "$repo_dir" "$@" +} + clone_repo_if_not_exists() { - local repo_dir=$1 - local repo_url=$2 + local repo_dir="$1" + local repo_url="$2" + shift 2 if [ ! -d "$repo_dir" ]; then - git clone "$repo_url" + git_with_github_mirror_fallback "$repo_dir" "$repo_url" "$@" else echo "Directory $repo_dir already exists, skipping clone." fi @@ -101,7 +118,7 @@ elif command -v yum &> /dev/null; then cd ../.. # Install msgpack-c - clone_repo_if_not_exists "msgpack" "https://github.com/msgpack/msgpack-c.git" + clone_repo_if_not_exists "msgpack-c" "https://github.com/msgpack/msgpack-c.git" cd msgpack-c || exit git checkout cpp-7.0.0 rm -rf build -- 2.34.1 From f1641d69d240ad633b0ba7dbe7afe20313e84d38 Mon Sep 17 00:00:00 2001 From: Stary Date: Tue, 28 Apr 2026 15:37:11 +0800 Subject: [PATCH 016/382] fix (CI): remove pull_request_target trigger and fork-routing from ci.yml (#1994) The pull_request_target trigger introduced in PR #1989 was incorrectly preserved during the merge of PR #1992 due to conflict resolution. Remove it along with the fork-routing conditions in ascend-test and integration-test, since the mirror URL is now hardcoded in ci_ascend.yml and vars/secrets access is no longer needed. Signed-off-by: staryxchen --- .github/workflows/ci.yml | 23 ++--------------------- 1 file changed, 2 insertions(+), 21 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7bcf571b..fba448ac 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -6,9 +6,6 @@ on: pull_request: branches: [ "main" ] types: [opened, synchronize, reopened, labeled] - pull_request_target: - branches: [ "main" ] - types: [opened, synchronize, reopened, labeled] workflow_dispatch: {} permissions: @@ -767,29 +764,13 @@ jobs: ascend-test: needs: [build, check-paths] - if: >- - needs.check-paths.outputs.should-run-downstream == 'true' && - ( - github.event_name == 'push' || - github.event_name == 'workflow_dispatch' || - (github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork) || - (github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.fork) - ) + if: needs.check-paths.outputs.should-run-downstream == 'true' uses: ./.github/workflows/ci_ascend.yml secrets: inherit - with: - checkout_ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.sha || '' }} integration-test: needs: [build, check-paths] - if: >- - needs.check-paths.outputs.should-run-downstream == 'true' && - ( - github.event_name == 'push' || - github.event_name == 'workflow_dispatch' || - (github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork) || - (github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.fork) - ) + if: needs.check-paths.outputs.should-run-downstream == 'true' uses: ./.github/workflows/integration-test.yml secrets: inherit -- 2.34.1 From f087c4fdf887383e40187fa906972b4fb90664f1 Mon Sep 17 00:00:00 2001 From: Xiao You Date: Tue, 28 Apr 2026 16:27:53 +0800 Subject: [PATCH 017/382] [Store] add ascend dummy real for host mem (#1917) Co-authored-by: youxiao --- mooncake-store/include/dummy_client.h | 16 +++ mooncake-store/include/pyclient.h | 3 + mooncake-store/include/real_client.h | 5 +- mooncake-store/src/dummy_client.cpp | 171 ++++++++++++++++++++++---- mooncake-store/src/real_client.cpp | 108 +++++++++++++--- mooncake-store/src/shm_helper.cpp | 38 +++--- 6 files changed, 287 insertions(+), 54 deletions(-) diff --git a/mooncake-store/include/dummy_client.h b/mooncake-store/include/dummy_client.h index 13128c43..3eedb17d 100644 --- a/mooncake-store/include/dummy_client.h +++ b/mooncake-store/include/dummy_client.h @@ -2,6 +2,8 @@ #include #include +#include +#include #include #include "pyclient.h" @@ -179,6 +181,15 @@ class DummyClient : public PyClient { int register_shm_via_ipc(const ShmHelper::ShmSegment *shm, bool is_local = false); +#if defined(USE_ASCEND_DIRECT) + int register_device_buffer_for_reconnect(void *buffer, size_t size); + + int unregister_device_buffer_for_reconnect(void *buffer); + + [[nodiscard]] std::vector + get_registered_device_buffers() const; +#endif + /** * @brief Generic RPC invocation helper for single-result operations * @tparam ServiceMethod Pointer to WrappedMasterService member function @@ -276,6 +287,11 @@ class DummyClient : public PyClient { void ping_thread_main(); std::atomic connected_{false}; +#if defined(USE_ASCEND_DIRECT) + mutable std::mutex registered_device_buffers_mutex_; + std::unordered_map registered_device_buffers_; +#endif + // Ascend physical device id for dummy-real RPC to real, set in setup_dummy int32_t device_id_ = 0; diff --git a/mooncake-store/include/pyclient.h b/mooncake-store/include/pyclient.h index c47bbfb9..4dcca4b5 100644 --- a/mooncake-store/include/pyclient.h +++ b/mooncake-store/include/pyclient.h @@ -28,6 +28,8 @@ enum ShmSegmentType : uint32_t { SHM_SEG_HOT_CACHE = 0, // local hot cache backing memory }; +constexpr int32_t kInvalidPhysicalDeviceId = -1; + // Return codes for health_check() enum HealthCheckStatus : int { HC_HEALTHY = 0, // Fully connected, all links up @@ -116,6 +118,7 @@ struct ShmRegisterRequest { uint64_t client_id_second; uint64_t dummy_base_addr; uint64_t shm_size; + int32_t device_id = kInvalidPhysicalDeviceId; bool is_local_buffer; }; diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index 5c7b3843..bd5b9768 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -462,6 +462,9 @@ class RealClient : public PyClient { size_t shm_size, bool is_local_buffer, const UUID &client_id); + tl::expected map_shm_internal_with_device( + int fd, uint64_t shm_base_addr, size_t shm_size, bool is_local_buffer, + int32_t physical_device_id, const UUID &client_id); tl::expected unmap_shm_internal(const UUID &client_id); @@ -727,7 +730,7 @@ class RealClient : public PyClient { bool is_ascend = false; bool is_ipc = false; // Ascend physical device id from dummy (dummy-real RPC). - int32_t device_id = -1; + int32_t device_id = kInvalidPhysicalDeviceId; uint64_t vmm_handle = 0; std::string ipc_key_data; }; diff --git a/mooncake-store/src/dummy_client.cpp b/mooncake-store/src/dummy_client.cpp index 8c5365de..396e0c7a 100644 --- a/mooncake-store/src/dummy_client.cpp +++ b/mooncake-store/src/dummy_client.cpp @@ -299,8 +299,15 @@ int DummyClient::register_ascend_shm(const ShmHelper::ShmSegment* shm, return 0; } if (!globalConfig().ascend_use_fabric_mem) { - LOG(ERROR) << "Host mem is only supported in fabric mem mode."; - return -1; + // Host: memfd + mmap (ShmHelper); register with Real like non-agent GPU + // path. + if (shm->fd < 0) { + LOG(ERROR) + << "Host POSIX shared memory requires memfd-backed allocation " + "(use ShmHelper::allocate / alloc_from_mem_pool)"; + return -1; + } + return register_shm_via_ipc(shm, is_local); } // Fabric host mem shared by vmm @@ -385,6 +392,8 @@ int DummyClient::register_shm_via_ipc(const ShmHelper::ShmSegment* shm, req.client_id_second = client_id_.second; req.dummy_base_addr = reinterpret_cast(shm->base_addr); req.shm_size = shm->size; + req.device_id = globalConfig().ascend_agent_mode ? device_id_ + : kInvalidPhysicalDeviceId; req.is_local_buffer = is_local; if (ipc_send_fd(sock_fd, shm->fd, &req, sizeof(req)) < 0) { @@ -482,16 +491,16 @@ int DummyClient::setup_dummy(size_t mem_pool_size, size_t local_buffer_size, } local_buffer_shm->registered = true; local_buffer_shm->is_local = true; + + // Best-effort: request hot cache shm from real client + if (request_hot_cache_fd() != 0) { + LOG(INFO) + << "Hot cache shm not available (real client may not have it)"; + } } ping_running_ = true; ping_thread_ = std::thread([this]() mutable { this->ping_thread_main(); }); - - // Best-effort: request hot cache shm from real client - if (request_hot_cache_fd() != 0) { - LOG(INFO) - << "Hot cache shm not available (real client may not have it)"; - } return 0; } @@ -517,6 +526,12 @@ int DummyClient::tearDownAll() { } connected_.store(false); last_ping_healthy_.store(false); +#if defined(USE_ASCEND_DIRECT) + { + std::lock_guard lock(registered_device_buffers_mutex_); + registered_device_buffers_.clear(); + } +#endif return 0; } @@ -533,22 +548,103 @@ int64_t DummyClient::unregister_shm() { invoke_rpc<&RealClient::unmap_shm_internal, void>(client_id_)); } +#if defined(USE_ASCEND_DIRECT) +int DummyClient::register_device_buffer_for_reconnect(void* buffer, + size_t size) { + const auto buffer_addr = reinterpret_cast(buffer); + { + std::lock_guard lock(registered_device_buffers_mutex_); + auto it = registered_device_buffers_.find(buffer_addr); + if (it != registered_device_buffers_.end() && it->second != size) { + LOG(ERROR) << "Device buffer size mismatch for tracked buffer, " + << "buffer=" << buffer << ", size=" << size + << ", tracked_size=" << it->second; + return -1; + } + } + + ShmHelper::ShmSegment shm{}; + shm.base_addr = buffer; + shm.size = size; + if (register_ascend_shm(&shm, false) != 0) { + LOG(ERROR) << "Failed to register device buffer, buffer=" << buffer + << ", size=" << size; + return -1; + } + + { + std::lock_guard lock(registered_device_buffers_mutex_); + registered_device_buffers_[buffer_addr] = size; + } + return 0; +} + +int DummyClient::unregister_device_buffer_for_reconnect(void* buffer) { + const auto buffer_addr = reinterpret_cast(buffer); + { + std::lock_guard lock(registered_device_buffers_mutex_); + if (registered_device_buffers_.find(buffer_addr) == + registered_device_buffers_.end()) { + LOG(ERROR) << "Device buffer is not registered with RealClient"; + return -1; + } + } + + auto ret = invoke_rpc<&RealClient::unregister_shm_buffer_internal, void>( + buffer_addr, client_id_); + if (ret.has_value()) { + std::lock_guard lock(registered_device_buffers_mutex_); + registered_device_buffers_.erase(buffer_addr); + } + return to_py_ret(ret); +} + +std::vector DummyClient::get_registered_device_buffers() + const { + std::vector device_buffers; + std::lock_guard lock(registered_device_buffers_mutex_); + device_buffers.reserve(registered_device_buffers_.size()); + for (const auto& [buffer_addr, size] : registered_device_buffers_) { + ShmHelper::ShmSegment shm{}; + shm.base_addr = reinterpret_cast(buffer_addr); + shm.size = size; + device_buffers.push_back(std::move(shm)); + } + return device_buffers; +} +#endif + // Dummy only register buffer within the shared memory region int DummyClient::register_buffer(void* buffer, size_t size) { if (buffer == nullptr) { LOG(ERROR) << "Invalid buffer pointer"; return -1; } +#if defined(USE_ASCEND_DIRECT) if (globalConfig().ascend_agent_mode) { - auto shm = std::make_shared(); - shm->base_addr = buffer; - shm->size = size; - if (register_ascend_shm(shm.get(), false) != 0) { - LOG(ERROR) << "Failed to implicitly register new ascend shm."; + aclrtPtrAttributes attributes{}; + auto acl_ret = aclrtPointerGetAttributes(buffer, &attributes); + if (acl_ret != ACL_ERROR_NONE) { + LOG(ERROR) << "Failed to get pointer attributes, ret=" << acl_ret; return -1; } - return 0; + if (attributes.location.type == ACL_MEM_LOCATION_TYPE_DEVICE) { + return register_device_buffer_for_reconnect(buffer, size); + } + if (globalConfig().ascend_use_fabric_mem) { + auto shm = std::make_shared(); + shm->base_addr = buffer; + shm->size = size; + if (register_ascend_shm(shm.get(), false) != 0) { + LOG(ERROR) << "Failed to register buffer, buffer=" << buffer + << ", size=" << size; + return -1; + } + return 0; + } + // non-Fabric Host: same rules as shm } +#endif // Find which shm this buffer belongs to auto shm = shm_helper_->get_shm(buffer); if (!shm) { @@ -587,6 +683,17 @@ int DummyClient::unregister_buffer(void* buffer) { return -1; } +#if defined(USE_ASCEND_DIRECT) + if (globalConfig().ascend_agent_mode) { + aclrtPtrAttributes attributes{}; + auto acl_ret = aclrtPointerGetAttributes(buffer, &attributes); + if (acl_ret == ACL_ERROR_NONE && + attributes.location.type == ACL_MEM_LOCATION_TYPE_DEVICE) { + return unregister_device_buffer_for_reconnect(buffer); + } + } +#endif + auto shm = shm_helper_->get_shm(buffer); if (!shm) { LOG(ERROR) << "Buffer is not in any registered shared memory"; @@ -1141,24 +1248,40 @@ void DummyClient::ping_thread_main() { if (globalConfig().ascend_agent_mode) { if (register_ascend_shm(shm_ptr.get(), shm_ptr->is_local) != 0) { - LOG(WARNING) << "Failed to re-register VMM " - "during reconnection"; - all_registered = false; - break; - } - } else { - if (register_shm_via_ipc(shm_ptr.get(), - shm_ptr->is_local) != 0) { LOG(WARNING) - << "Failed to re-register shared memory " - "during reconnection"; + << "Failed to re-register Ascend shared " + "memory during reconnection"; all_registered = false; break; } + } else if (register_shm_via_ipc( + shm_ptr.get(), shm_ptr->is_local) != 0) { + LOG(WARNING) + << "Failed to re-register shared memory " + "during reconnection"; + all_registered = false; + break; } } } +#if defined(USE_ASCEND_DIRECT) + if (all_registered && globalConfig().ascend_agent_mode) { + auto device_buffers = get_registered_device_buffers(); + for (const auto& device_buffer : device_buffers) { + if (register_ascend_shm(&device_buffer, false) != 0) { + LOG(WARNING) + << "Failed to re-register device buffer " + "during reconnection, buffer=" + << device_buffer.base_addr + << ", size=" << device_buffer.size; + all_registered = false; + break; + } + } + } +#endif + if (all_registered) { LOG(INFO) << "Re-registered all shared memorys successfully"; diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 1ddc9012..834e02af 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -51,6 +51,25 @@ bool checkAcl(aclError result, const char *message) { } return true; } + +tl::expected set_context_if_needed(const std::string &protocol, + int32_t physical_device_id, + const char *action) { + if (protocol != "ascend" || !globalConfig().ascend_agent_mode) { + return {}; + } + if (physical_device_id == kInvalidPhysicalDeviceId) { + LOG(ERROR) << "Missing physical device id for " << action; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + if (!ContextManager::getInstance().setCurrentContextByPhysicalId( + physical_device_id)) { + LOG(ERROR) << "Failed to set current context for physical device " + << physical_device_id << " during " << action; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + return {}; +} #endif struct PreparedRangedReadRequest { @@ -539,9 +558,8 @@ tl::expected RealClient::setup_internal( const std::string &ssd_offload_path) { this->protocol = protocol; this->ipc_socket_path_ = ipc_socket_path; - const bool should_use_hugepage = use_hugepage_ && - this->protocol != "ascend" && - this->protocol != "ubshmem"; + const bool should_use_hugepage = + use_hugepage_ && this->protocol != "ubshmem"; #ifdef USE_ASCEND_DIRECT if (protocol == "ascend" && globalConfig().ascend_agent_mode) { auto ascend_setup = setup_ascend_internal(local_buffer_size); @@ -1510,6 +1528,14 @@ int64_t RealClient::getSize(const std::string &key) { tl::expected RealClient::map_shm_internal( int fd, uint64_t dummy_base_addr, size_t shm_size, bool is_local_buffer, const UUID &client_id) { + return map_shm_internal_with_device(fd, dummy_base_addr, shm_size, + is_local_buffer, + kInvalidPhysicalDeviceId, client_id); +} + +tl::expected RealClient::map_shm_internal_with_device( + int fd, uint64_t dummy_base_addr, size_t shm_size, bool is_local_buffer, + int32_t physical_device_id, const UUID &client_id) { std::stringstream addr_stream; addr_stream << "0x" << std::hex << dummy_base_addr; @@ -1518,15 +1544,16 @@ tl::expected RealClient::map_shm_internal( "_" + std::to_string(client_id.second) + "_" + addr_stream.str(); std::unique_lock lock(dummy_client_mutex_); - // Check if client context exists, create if not - auto &context = shm_contexts_[client_id]; - // Check if this shm is already mapped - for (const auto &shm : context.mapped_shms) { - if (shm.dummy_base_addr == static_cast(dummy_base_addr)) { - LOG(INFO) << "Segment already mapped: " << shm_name; - if (fd >= 0) close(fd); - return {}; + auto context_it = shm_contexts_.find(client_id); + if (context_it != shm_contexts_.end()) { + for (const auto &shm : context_it->second.mapped_shms) { + if (shm.dummy_base_addr == + static_cast(dummy_base_addr)) { + LOG(INFO) << "Segment already mapped: " << shm_name; + if (fd >= 0) close(fd); + return {}; + } } } @@ -1535,6 +1562,15 @@ tl::expected RealClient::map_shm_internal( return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } +#ifdef USE_ASCEND_DIRECT + auto context_result = set_context_if_needed(protocol, physical_device_id, + "POSIX shm registration"); + if (!context_result) { + close(fd); + return context_result; + } +#endif + // Map shared memory from FD void *shm_buffer = mmap(nullptr, shm_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); @@ -1554,6 +1590,11 @@ tl::expected RealClient::map_shm_internal( shm.dummy_base_addr = static_cast(dummy_base_addr); shm.shm_addr_offset = reinterpret_cast(shm_buffer) - reinterpret_cast(dummy_base_addr); + shm.is_ascend = + false; // memfd/POSIX path; teardown differs from Ascend VMM/IPC + shm.device_id = physical_device_id; + + auto &context = shm_contexts_[client_id]; if (shm_size > 0) { auto result = client_->RegisterLocalMemory( @@ -1829,7 +1870,38 @@ tl::expected RealClient::ascend_unmap_shm_internal( if (!shm.shm_buffer) { continue; } - teardown_ascend_shm_buffer(shm); + if (shm.is_ascend) { + teardown_ascend_shm_buffer(shm); + } else { +#ifdef USE_ASCEND_DIRECT + auto context_result = set_context_if_needed(protocol, shm.device_id, + "POSIX shm cleanup"); + if (!context_result) { + LOG(WARNING) << "Skip unregisterLocalMemory for POSIX shm: " + << shm.shm_name + << ", error: " << toString(context_result.error()); + } else +#endif + // Host POSIX shm mapped via map_shm_internal (memfd); not + // Ascend VMM/IPC. + if (shm.shm_size > 0 && client_) { + auto res = + client_->unregisterLocalMemory(shm.shm_buffer, true); + if (!res) { + LOG(WARNING) << "Failed to unregister local memory for " + "POSIX shm: " + << shm.shm_name + << ", error: " << toString(res.error()); + } + } + if (munmap(shm.shm_buffer, shm.shm_size) != 0) { + LOG(ERROR) << "Failed to munmap POSIX shared memory: " + << shm.shm_name << ", error: " << strerror(errno); + } else { + LOG(INFO) << "Unmapped POSIX shared memory: " << shm.shm_name + << ", size: " << shm.shm_size; + } + } } shm_contexts_.erase(it); return {}; @@ -1868,6 +1940,13 @@ tl::expected RealClient::unregister_shm_buffer_internal( if (shm_it->is_ascend) { teardown_ascend_shm_buffer(*shm_it); } else { +#ifdef USE_ASCEND_DIRECT + auto context_result = set_context_if_needed( + protocol, shm_it->device_id, "POSIX shm unregister"); + if (!context_result) { + return context_result; + } +#endif auto rc = client_->unregisterLocalMemory(shm_it->shm_buffer, true); if (!rc) { LOG(ERROR) << "Failed to unregister memory: " @@ -4075,8 +4154,9 @@ void RealClient::handle_ipc_shm_register(int client_sock) { } UUID client_id{req.client_id_first, req.client_id_second}; - auto result = map_shm_internal(fd, req.dummy_base_addr, req.shm_size, - req.is_local_buffer, client_id); + auto result = map_shm_internal_with_device( + fd, req.dummy_base_addr, req.shm_size, req.is_local_buffer, + req.device_id, client_id); int status = result.has_value() ? 0 : -1; ::send(client_sock, &status, sizeof(status), 0); diff --git a/mooncake-store/src/shm_helper.cpp b/mooncake-store/src/shm_helper.cpp index bee0021b..d1dbd424 100644 --- a/mooncake-store/src/shm_helper.cpp +++ b/mooncake-store/src/shm_helper.cpp @@ -54,7 +54,8 @@ bool ShmHelper::cleanup() { } if (shm->base_addr) { #if defined(USE_ASCEND_DIRECT) - if (globalConfig().ascend_agent_mode) { + if (globalConfig().ascend_agent_mode && + globalConfig().ascend_use_fabric_mem) { free_memory("ascend", shm->base_addr); continue; } @@ -73,22 +74,28 @@ bool ShmHelper::cleanup() { void* ShmHelper::allocate(size_t size) { std::lock_guard lock(shm_mutex_); + // Dummy-real: FabricMem host uses VMM; non-Fabric host uses memfd+mmap like + // non-agent / GPU shm path. #ifdef USE_ASCEND_DIRECT if (globalConfig().ascend_agent_mode) { - void* base_addr = nullptr; - size_t alloc_size = size; - base_addr = ascend_allocate_vmm_memory_direct(alloc_size); - if (base_addr == nullptr) { - throw std::runtime_error("Failed to allocate VMM shared memory"); + if (globalConfig().ascend_use_fabric_mem) { + void* base_addr = nullptr; + size_t alloc_size = size; + base_addr = ascend_allocate_vmm_memory_direct(alloc_size); + if (base_addr == nullptr) { + throw std::runtime_error( + "Failed to allocate VMM shared memory"); + } + auto shm = std::make_shared(); + shm->fd = -1; + shm->base_addr = base_addr; + shm->size = alloc_size; + shm->name = MOONCAKE_SHM_NAME; + shm->registered = false; + shms_.push_back(shm); + return base_addr; } - auto shm = std::make_shared(); - shm->fd = -1; - shm->base_addr = base_addr; - shm->size = alloc_size; - shm->name = MOONCAKE_SHM_NAME; - shm->registered = false; - shms_.push_back(shm); - return base_addr; + // ascend_agent_mode && !ascend_use_fabric_mem: fall through to memfd } #endif @@ -142,7 +149,8 @@ int ShmHelper::free(void* addr) { } if ((*it)->base_addr) { #if defined(USE_ASCEND_DIRECT) - if (globalConfig().ascend_agent_mode) { + if (globalConfig().ascend_agent_mode && + globalConfig().ascend_use_fabric_mem) { free_memory("ascend", (*it)->base_addr); } else #endif -- 2.34.1 From cced76c1e745f27809ae96e3ea68cc5db5634911 Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Tue, 28 Apr 2026 19:40:43 +0800 Subject: [PATCH 018/382] [Store] Add standalone HTTP mount_shm/unmount_shm API with reconfiguration support (#1928) --- .../source/http-api-reference/http-service.md | 101 +++++++++ .../python-api-reference/mooncake-store.md | 73 ++++++ mooncake-integration/store/store_py.cpp | 43 ++++ mooncake-store/include/client_service.h | 21 ++ mooncake-store/include/real_client.h | 26 +++ mooncake-store/src/client_service.cpp | 110 +++++---- mooncake-store/src/real_client.cpp | 155 +++++++++++++ .../tests/client_integration_test.cpp | 34 +++ .../mooncake/mooncake_store_service.py | 214 ++++++++++++++++++ .../tests/test_mooncake_store_service_api.py | 201 ++++++++++++++++ 10 files changed, 935 insertions(+), 43 deletions(-) create mode 100644 mooncake-wheel/tests/test_mooncake_store_service_api.py diff --git a/docs/source/http-api-reference/http-service.md b/docs/source/http-api-reference/http-service.md index b77c78dc..3b2b0013 100644 --- a/docs/source/http-api-reference/http-service.md +++ b/docs/source/http-api-reference/http-service.md @@ -10,6 +10,10 @@ The HTTP service serves multiple purposes: - **Data Inspection**: Examine stored objects and their replicas - **Health Checks**: Service availability and status verification +The Python `mooncake.mooncake_store_service` module also provides a lightweight +Store REST API for data operations and standalone segment mount/unmount +workflows. Unless configured otherwise, it listens on port `8080`. + ## HTTP Endpoints ### Metrics Endpoints @@ -159,3 +163,100 @@ Basic health check endpoint for service availability verification. curl http://localhost:8080/health ``` +## Store REST API Endpoints + +The following endpoints are served by the Python store REST service, which wraps +`MooncakeDistributedStore` with an aiohttp service. The HTTP handlers live in +Python, while mount and unmount operations are delegated to the underlying store +binding. Start the service with: + +```bash +python -m mooncake.mooncake_store_service \ + --config /path/to/mooncake_config.json \ + --port 8080 +``` + +If the wheel console scripts are installed, the equivalent command is: + +```bash +mc_store_rest_server --config /path/to/mooncake_config.json --port 8080 +``` + +### `/api/mount_shm` +Mount a named shared memory object as one or more Mooncake store segments. If +the requested size exceeds the maximum registration size, the service may split +the region and return multiple segment ids. + +**Method**: `POST` +**Content-Type**: `application/json` + +**Request Body**: +```json +{ + "name": "mooncake_segment", + "size": 16777216, + "offset": 0, + "protocol": "tcp", + "location": "" +} +``` + +**Fields**: +- `name` (string, required): Named shared memory object name. A leading `/` is + accepted, but path separators are not. +- `size` (integer, required): Number of bytes to mount. +- `offset` (integer, optional): File offset in bytes. Defaults to `0`. +- `protocol` (string, optional): Transfer protocol. Defaults to the service + configuration protocol. +- `location` (string, optional): Device or locality hint. Defaults to an empty + string. + +**Success Response**: +```json +{ + "status": "success", + "segment_ids": ["00000000-0000-0000-0000-000000000001"] +} +``` + +**Example**: +```bash +curl -X POST http://localhost:8080/api/mount_shm \ + -H "Content-Type: application/json" \ + -d '{ + "name": "mooncake_segment", + "size": 16777216, + "offset": 0, + "protocol": "tcp", + "location": "" + }' +``` + +### `/api/unmount_shm` +Unmount one or more segment ids previously returned by `/api/mount_shm`. + +**Method**: `POST` +**Content-Type**: `application/json` + +**Request Body**: +```json +{ + "segment_ids": ["00000000-0000-0000-0000-000000000001"] +} +``` + +`segment_ids` may also be provided as a single string for one segment. + +**Success Response**: +```json +{ + "status": "success" +} +``` + +**Example**: +```bash +curl -X POST http://localhost:8080/api/unmount_shm \ + -H "Content-Type: application/json" \ + -d '{"segment_ids": ["00000000-0000-0000-0000-000000000001"]}' +``` diff --git a/docs/source/python-api-reference/mooncake-store.md b/docs/source/python-api-reference/mooncake-store.md index ce1e7987..5ba4af73 100644 --- a/docs/source/python-api-reference/mooncake-store.md +++ b/docs/source/python-api-reference/mooncake-store.md @@ -1429,6 +1429,79 @@ def init_all(self, protocol: str, device_name: str, mount_segment_size: int = 16 --- +#### mount_segment() +Mount a local file or shared-memory region as one or more Mooncake store +segments. + +```python +def mount_segment( + self, + path: str, + size: int, + offset: int = 0, + protocol: str = "tcp", + location: str = "", +) -> dict +``` + +**Parameters:** +- `path` (str): File path to map and mount. +- `size` (int): Number of bytes to mount. +- `offset` (int, optional): File offset in bytes. Defaults to `0`. +- `protocol` (str, optional): Transfer protocol. Defaults to `"tcp"`. +- `location` (str, optional): Device or locality hint. Defaults to an empty + string. + +**Returns:** +- `dict`: A result dictionary with: + - `ret` (int): Status code (0 = success, non-zero = error code) + - `segment_ids` (List[str]): Segment ids created by the mount operation + +**Example:** +```python +result = store.mount_segment( + "/dev/shm/mooncake_segment", + 16 * 1024 * 1024, + offset=0, + protocol="tcp", + location="", +) + +if result["ret"] == 0: + segment_ids = list(result["segment_ids"]) + print("Mounted segments:", segment_ids) +else: + print("Mount failed:", result["ret"]) +``` + +The corresponding HTTP endpoints are `/api/mount_shm` and +`/api/unmount_shm`, but the HTTP API is intentionally narrower: it accepts a +named shared memory object name instead of an arbitrary path. + +--- + +#### unmount_segment() +Unmount one or more file or shared-memory segments by segment id. + +```python +def unmount_segment(self, segment_ids: List[str]) -> int +``` + +**Parameters:** +- `segment_ids` (List[str]): Segment ids returned by `mount_segment()`. + +**Returns:** +- `int`: Status code (0 = success, non-zero = error code) + +**Example:** +```python +ret = store.unmount_segment(segment_ids) +if ret != 0: + print("Unmount failed:", ret) +``` + +--- + #### get_hostname() Get the hostname of the current store instance. diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index 324faea2..76a4baea 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -302,6 +302,44 @@ class MooncakeStorePyWrapper { return store_->health_check(); } + py::dict mount_segment(const std::string &path, size_t size, size_t offset, + const std::string &protocol, + const std::string &location) { + py::dict result; + result["ret"] = -1; + result["segment_ids"] = py::list(); + + auto real_client = std::dynamic_pointer_cast(store_); + if (!real_client) { + LOG(ERROR) << "mount_segment requires RealClient"; + return result; + } + std::vector segment_ids; + int ret; + { + py::gil_scoped_release release; + ret = real_client->mountSegment(path, offset, size, protocol, + location, segment_ids); + } + result["ret"] = ret; + py::list ids; + for (const auto &id : segment_ids) { + ids.append(id); + } + result["segment_ids"] = ids; + return result; + } + + int unmount_segment(const std::vector &segment_ids) { + auto real_client = std::dynamic_pointer_cast(store_); + if (!real_client) { + LOG(ERROR) << "unmount_segment requires RealClient"; + return -1; + } + py::gil_scoped_release release; + return real_client->unmountSegment(segment_ids); + } + std::string get_tp_key_name(const std::string &base_key, int rank) const { return base_key + "_tp_" + std::to_string(rank); } @@ -1772,6 +1810,11 @@ PYBIND11_MODULE(store, m) { return self.store_->initAll(protocol, device_name, mount_segment_size); }) + .def("mount_segment", &MooncakeStorePyWrapper::mount_segment, + py::arg("path"), py::arg("size"), py::arg("offset") = 0, + py::arg("protocol") = "tcp", py::arg("location") = "") + .def("unmount_segment", &MooncakeStorePyWrapper::unmount_segment, + py::arg("segment_ids")) .def("alloc_from_mem_pool", [](MooncakeStorePyWrapper &self, size_t size) { py::gil_scoped_release release; diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index 445d75cd..7012b80f 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -295,6 +295,20 @@ class Client { tl::expected UnmountSegment(const void* buffer, size_t size); + /** + * @brief Mounts a memory segment and returns its generated Segment UUID. + * Logic is identical to MountSegment, but returns the segment id. + */ + tl::expected MountSegmentAndGetId( + const void* buffer, size_t size, const std::string& protocol = "tcp", + const std::string& location = kWildcardLocation); + + /** + * @brief Unmounts a segment by its UUID. + * Logic is identical to UnmountSegment, but looks up by id. + */ + tl::expected UnmountSegmentById(const UUID& segment_id); + /** * @brief Registers memory buffer with TransferEngine for data transfer * @param addr Memory address to register @@ -666,6 +680,13 @@ class Client { std::mutex mounted_segments_mutex_; std::unordered_map> mounted_segments_; + /** + * @brief Internal helper to unmount a segment by iterator. + * Caller must hold mounted_segments_mutex_. + */ + tl::expected UnmountSegmentImpl( + std::unordered_map>::iterator it); + // Configuration const std::string local_hostname_; const std::string metadata_connstring_; diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index bd5b9768..7982de45 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -679,6 +679,27 @@ class RealClient : public PyClient { const std::string &target_rpc_service_addr, std::unordered_map &objects); + /** + * @brief Mount a shared memory file region and return segment ids. + * If size > max_mr_size, it will be split into multiple chunks + * and mounted separately. RealClient will open(path) + mmap + * internally for each chunk. + */ + int mountSegment(const std::string &path, size_t offset, size_t size, + const std::string &protocol, const std::string &location, + std::vector &out_segment_ids); + + /** + * @brief Unmount segments by their ids and clean up local mmap/fd. + */ + int unmountSegment(const std::vector &segment_ids); + + struct MountedSegmentRecord { + void *mmap_base = nullptr; + size_t size = 0; + std::string path; + }; + std::unique_ptr port_binder_ = nullptr; struct SegmentDeleter { @@ -815,6 +836,11 @@ class RealClient : public PyClient { void teardown_ascend_shm_buffer(MappedShm &shm); tl::expected setup_ascend_internal( size_t local_buffer_size); + + private: + std::unordered_map + mounted_segment_records_; + std::mutex mounted_segment_records_mutex_; }; } // namespace mooncake diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 16c7dfd0..24335d9a 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -2082,6 +2082,64 @@ std::vector Client::GetNicNumaNodes() const { } tl::expected Client::MountSegment( + const void* buffer, size_t size, const std::string& protocol, + const std::string& location) { + auto result = MountSegmentAndGetId(buffer, size, protocol, location); + if (!result) { + return tl::unexpected(result.error()); + } + return {}; +} + +tl::expected Client::UnmountSegmentImpl( + std::unordered_map>::iterator it) { + auto unmount_result = master_client_.UnmountSegment(it->second.id); + if (!unmount_result) { + ErrorCode err = unmount_result.error(); + LOG(ERROR) << "Failed to unmount segment from master: " + << toString(err); + return tl::unexpected(err); + } + + int rc = transfer_engine_->unregisterLocalMemory( + reinterpret_cast(it->second.base)); + if (rc != 0) { + LOG(ERROR) << "Failed to unregister transfer buffer with transfer " + "engine ret is " + << rc; + if (rc != ERR_ADDRESS_NOT_REGISTERED) { + return tl::unexpected(ErrorCode::INTERNAL_ERROR); + } + // Otherwise, the segment is already unregistered from transfer + // engine, we can continue + } + + mounted_segments_.erase(it); + return {}; +} + +tl::expected Client::UnmountSegment(const void* buffer, + size_t size) { + std::lock_guard lock(mounted_segments_mutex_); + auto segment = mounted_segments_.end(); + + for (auto it = mounted_segments_.begin(); it != mounted_segments_.end(); + ++it) { + if (it->second.base == reinterpret_cast(buffer) && + it->second.size == size) { + segment = it; + break; + } + } + if (segment == mounted_segments_.end()) { + LOG(ERROR) << "segment_not_found base=" << buffer << " size=" << size; + return tl::unexpected(ErrorCode::INVALID_PARAMS); + } + + return UnmountSegmentImpl(segment); +} + +tl::expected Client::MountSegmentAndGetId( const void* buffer, size_t size, const std::string& protocol, const std::string& location) { auto check_result = CheckRegisterMemoryParams(buffer, size); @@ -2089,6 +2147,7 @@ tl::expected Client::MountSegment( return tl::unexpected(check_result.error()); } + UUID segment_id; { std::lock_guard lock(mounted_segments_mutex_); @@ -2115,17 +2174,12 @@ tl::expected Client::MountSegment( return tl::unexpected(ErrorCode::INVALID_PARAMS); } - // Build segment with logical name; attach TE endpoint for transport Segment segment; segment.id = generate_uuid(); segment.name = local_hostname_; segment.base = reinterpret_cast(buffer); segment.size = size; segment.protocol = protocol; - // For P2P handshake mode, publish the actual transport endpoint that - // was negotiated by the transfer engine. Otherwise, keep the logical - // hostname so metadata backends (HTTP/etcd/redis) can resolve the - // segment by name. if (metadata_connstring_ == P2PHANDSHAKE) { segment.te_endpoint = transfer_engine_->getLocalIpAndPort(); } else { @@ -2140,54 +2194,24 @@ tl::expected Client::MountSegment( return tl::unexpected(err); } - mounted_segments_[segment.id] = segment; + segment_id = segment.id; + mounted_segments_[segment_id] = segment; } EnsureStorageControlPlaneStarted(); - return {}; + return segment_id; } -tl::expected Client::UnmountSegment(const void* buffer, - size_t size) { +tl::expected Client::UnmountSegmentById( + const UUID& segment_id) { std::lock_guard lock(mounted_segments_mutex_); - auto segment = mounted_segments_.end(); - - for (auto it = mounted_segments_.begin(); it != mounted_segments_.end(); - ++it) { - if (it->second.base == reinterpret_cast(buffer) && - it->second.size == size) { - segment = it; - break; - } - } + auto segment = mounted_segments_.find(segment_id); if (segment == mounted_segments_.end()) { - LOG(ERROR) << "segment_not_found base=" << buffer << " size=" << size; + LOG(ERROR) << "segment_not_found id=" << UuidToString(segment_id); return tl::unexpected(ErrorCode::INVALID_PARAMS); } - auto unmount_result = master_client_.UnmountSegment(segment->second.id); - if (!unmount_result) { - ErrorCode err = unmount_result.error(); - LOG(ERROR) << "Failed to unmount segment from master: " - << toString(err); - return tl::unexpected(err); - } - - int rc = transfer_engine_->unregisterLocalMemory( - reinterpret_cast(segment->second.base)); - if (rc != 0) { - LOG(ERROR) << "Failed to unregister transfer buffer with transfer " - "engine ret is " - << rc; - if (rc != ERR_ADDRESS_NOT_REGISTERED) { - return tl::unexpected(ErrorCode::INTERNAL_ERROR); - } - // Otherwise, the segment is already unregistered from transfer - // engine, we can continue - } - - mounted_segments_.erase(segment); - return {}; + return UnmountSegmentImpl(segment); } tl::expected Client::RegisterLocalMemory( diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 834e02af..7d28ed26 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -2,6 +2,8 @@ #include #include #include +#include +#include #include #include #include @@ -1039,6 +1041,159 @@ tl::expected RealClient::tearDownAll_internal() { int RealClient::tearDownAll() { return to_py_ret(tearDownAll_internal()); } +int RealClient::mountSegment(const std::string &path, size_t offset, + size_t size, const std::string &protocol, + const std::string &location, + std::vector &out_segment_ids) { + if (!client_) { + LOG(ERROR) << "Client not initialized"; + return -1; + } + + size_t max_mr_size = globalConfig().max_mr_size; + if (max_mr_size == 0) { + LOG(ERROR) << "Invalid max_mr_size: 0"; + return -1; + } + + size_t page_size = sysconf(_SC_PAGESIZE); + if (max_mr_size < page_size) { + LOG(ERROR) << "max_mr_size " << max_mr_size + << " is smaller than page_size " << page_size; + return -1; + } + + int fd = open(path.c_str(), O_RDWR); + if (fd < 0) { + LOG(ERROR) << "open failed for " << path << ": " << strerror(errno); + return -1; + } + + struct stat st; + if (fstat(fd, &st) != 0) { + LOG(ERROR) << "fstat failed for " << path << ": " << strerror(errno); + close(fd); + return -1; + } + + if (offset > (size_t)st.st_size || size > (size_t)st.st_size - offset) { + LOG(ERROR) << "requested range [offset=" << offset << ", size=" << size + << "] exceeds file size " << st.st_size; + close(fd); + return -1; + } + + if (offset % page_size != 0) { + LOG(ERROR) << "offset " << offset + << " is not page-aligned (page_size=" << page_size << ")"; + close(fd); + return -1; + } + + size_t aligned_max_chunk = (max_mr_size / page_size) * page_size; + size_t remaining = size; + size_t current_offset = offset; + std::vector mounted_ids; + std::vector mounted_records; + + while (remaining > 0) { + size_t chunk_size = std::min(remaining, aligned_max_chunk); + if (chunk_size == 0) break; + + void *ptr = mmap(nullptr, chunk_size, PROT_READ | PROT_WRITE, + MAP_SHARED, fd, current_offset); + if (ptr == MAP_FAILED) { + LOG(ERROR) << "mmap failed for " << path << ": " << strerror(errno); + break; + } + + auto result = + client_->MountSegmentAndGetId(ptr, chunk_size, protocol, location); + if (!result.has_value()) { + LOG(ERROR) << "MountSegmentAndGetId failed"; + munmap(ptr, chunk_size); + break; + } + + std::string segment_id = UuidToString(result.value()); + mounted_ids.push_back(segment_id); + mounted_records.push_back({ptr, chunk_size, path}); + + remaining -= chunk_size; + current_offset += chunk_size; + } + + close(fd); + + // If not all chunks were mounted, roll back successfully mounted ones + if (remaining > 0) { + for (size_t i = 0; i < mounted_ids.size(); ++i) { + UUID id; + if (StringToUuid(mounted_ids[i], id)) { + client_->UnmountSegmentById(id); + } + if (mounted_records[i].mmap_base) { + munmap(mounted_records[i].mmap_base, mounted_records[i].size); + } + } + out_segment_ids.clear(); + return -1; + } + + { + std::lock_guard lock(mounted_segment_records_mutex_); + for (size_t i = 0; i < mounted_ids.size(); ++i) { + mounted_segment_records_[mounted_ids[i]] = mounted_records[i]; + } + } + out_segment_ids = std::move(mounted_ids); + return 0; +} + +int RealClient::unmountSegment(const std::vector &segment_ids) { + if (!client_) { + LOG(ERROR) << "Client not initialized"; + return -1; + } + + int first_error = 0; + std::vector> to_cleanup; + { + std::lock_guard lock(mounted_segment_records_mutex_); + for (const auto &segment_id : segment_ids) { + UUID id; + if (!StringToUuid(segment_id, id)) { + LOG(ERROR) << "Invalid segment_id: " << segment_id; + if (first_error == 0) first_error = -1; + continue; + } + + auto result = client_->UnmountSegmentById(id); + if (!result.has_value()) { + LOG(ERROR) << "UnmountSegmentById failed for " << segment_id; + if (first_error == 0) { + first_error = static_cast(result.error()); + } + continue; // Don't release local resources on failure + } + + auto it = mounted_segment_records_.find(segment_id); + if (it != mounted_segment_records_.end()) { + to_cleanup.emplace_back(segment_id, it->second); + mounted_segment_records_.erase(it); + } + } + } + + for (auto &p : to_cleanup) { + if (p.second.mmap_base) { + munmap(p.second.mmap_base, p.second.size); + } + } + + return first_error; +} + int RealClient::health_check() { if (closed_.load()) return HC_NOT_INITIALIZED; if (!client_) return HC_NOT_INITIALIZED; diff --git a/mooncake-store/tests/client_integration_test.cpp b/mooncake-store/tests/client_integration_test.cpp index 42a13256..94c7e387 100644 --- a/mooncake-store/tests/client_integration_test.cpp +++ b/mooncake-store/tests/client_integration_test.cpp @@ -1734,6 +1734,40 @@ TEST_F(ClientIntegrationTest, BatchUpsertMixed) { } } +TEST_F(ClientIntegrationTest, MountSegmentAndGetIdAndUnmountSegmentById) { + // Allocate a small buffer for this test + size_t test_size = 16 * 1024 * 1024; // 16 MB + void* test_buffer = allocate_buffer_allocator_memory(test_size); + ASSERT_NE(test_buffer, nullptr); + + // Test MountSegmentAndGetId returns a valid UUID + auto mount_result = test_client_->MountSegmentAndGetId( + test_buffer, test_size, FLAGS_protocol); + ASSERT_TRUE(mount_result.has_value()) + << "MountSegmentAndGetId failed: " << toString(mount_result.error()); + UUID segment_id = mount_result.value(); + EXPECT_NE(segment_id.first, 0u); + EXPECT_NE(segment_id.second, 0u); + + // Test UnmountSegmentById succeeds + auto unmount_result = test_client_->UnmountSegmentById(segment_id); + EXPECT_TRUE(unmount_result.has_value()) + << "UnmountSegmentById failed: " << toString(unmount_result.error()); + + // Test MountSegment delegates to MountSegmentAndGetId (equivalent behavior) + auto mount2 = + test_client_->MountSegment(test_buffer, test_size, FLAGS_protocol); + EXPECT_TRUE(mount2.has_value()) + << "MountSegment failed: " << toString(mount2.error()); + + // Test UnmountSegment delegates to UnmountSegmentImpl (equivalent behavior) + auto unmount2 = test_client_->UnmountSegment(test_buffer, test_size); + EXPECT_TRUE(unmount2.has_value()) + << "UnmountSegment failed: " << toString(unmount2.error()); + + free(test_buffer); +} + } // namespace testing } // namespace mooncake diff --git a/mooncake-wheel/mooncake/mooncake_store_service.py b/mooncake-wheel/mooncake/mooncake_store_service.py index 6dae3416..ed727d61 100644 --- a/mooncake-wheel/mooncake/mooncake_store_service.py +++ b/mooncake-wheel/mooncake/mooncake_store_service.py @@ -23,6 +23,15 @@ def _timed_handler(operation_name, handler): return wrapper +def _shm_name_to_path(name): + if not isinstance(name, str) or not name: + return None + normalized = name[1:] if name.startswith("/") else name + if not normalized or "/" in normalized or normalized in {".", ".."}: + return None + return f"/dev/shm/{normalized}" + + class MooncakeStoreService: """ Mooncake Store Service with REST API. @@ -53,6 +62,12 @@ class MooncakeStoreService: self.config = None self._setup_logging() + # State for /api/reconfigure (Prefill/Decode mode switch) + self.current_mode = "prefill" # "prefill" or "decode" + self.mounted_segment_ids = [] # persisted segment_ids from last decode mount + self.last_mount_info = {} # last mount parameters for debugging + self._state_lock = asyncio.Lock() # serialize reconfigure/mount/unmount state changes + try: if config_path: self.config = MooncakeConfig.from_file(config_path) @@ -143,6 +158,9 @@ class MooncakeStoreService: async def start_http_service(self, port: int = 8080): app = web.Application(client_max_size=1024 * 1024 * 100) # 100MB limit app.add_routes([ + web.post('/api/reconfigure', _timed_handler("RECONFIGURE", self.handle_reconfigure)), + web.post('/api/mount_shm', _timed_handler("MOUNT_SHM", self.handle_mount_shm)), + web.post('/api/unmount_shm', _timed_handler("UNMOUNT_SHM", self.handle_unmount_shm)), web.put('/api/put', _timed_handler("PUT", self.handle_put)), web.get('/api/get/{key}', _timed_handler("GET", self.handle_get)), web.get('/api/exist/{key}', _timed_handler("EXIST", self.handle_exist)), @@ -157,6 +175,202 @@ class MooncakeStoreService: return True # REST API handlers + async def handle_reconfigure(self, request): + try: + data = await request.json() + mode = data.get("mode", "").lower() + + if mode == "decode": + path = data.get("path") + size = data.get("size") + offset = data.get("offset", 0) + protocol = data.get("protocol", self.config.protocol) + location = data.get("location", "") + + if not path or size is None: + return web.Response( + status=400, + text=json.dumps({"error": "Missing path or size for decode mode"}), + content_type="application/json" + ) + + async with self._state_lock: + # If already in decode mode with mounted segments, unmount them first + if self.mounted_segment_ids: + logging.info("Reconfigure decode: unmounting previous segments before remount") + ret = self.store.unmount_segment(self.mounted_segment_ids) + if ret != 0: + return web.Response( + status=500, + text=json.dumps({"error": f"Unmount of previous segments failed, ret={ret}"}), + content_type="application/json" + ) + self.mounted_segment_ids.clear() + + result = self.store.mount_segment(path, size, offset, protocol, location) + if result["ret"] != 0: + self.current_mode = "prefill" + self.mounted_segment_ids.clear() + self.last_mount_info.clear() + return web.Response( + status=500, + text=json.dumps( + { + "error": ( + f"Mount failed, ret={result['ret']}; " + "rolled back to prefill" + ), + "mode": self.current_mode, + } + ), + content_type="application/json" + ) + + self.mounted_segment_ids = list(result["segment_ids"]) + self.current_mode = "decode" + self.last_mount_info = { + "path": path, "offset": offset, "size": size, + "protocol": protocol, "location": location + } + + return web.Response( + status=200, + text=json.dumps({ + "status": "success", + "mode": self.current_mode, + "segment_ids": self.mounted_segment_ids, + }), + content_type="application/json" + ) + + elif mode == "prefill": + async with self._state_lock: + if self.mounted_segment_ids: + ret = self.store.unmount_segment(self.mounted_segment_ids) + if ret != 0: + return web.Response( + status=500, + text=json.dumps({"error": f"Unmount failed, ret={ret}"}), + content_type="application/json" + ) + self.mounted_segment_ids.clear() + + self.current_mode = "prefill" + self.last_mount_info.clear() + + return web.Response( + status=200, + text=json.dumps({"status": "success", "mode": self.current_mode}), + content_type="application/json" + ) + + else: + return web.Response( + status=400, + text=json.dumps({"error": "Invalid mode. Use 'decode' or 'prefill'"}), + content_type="application/json" + ) + except Exception as e: + logging.error("RECONFIGURE error: %s", e) + return web.Response( + status=500, + text=json.dumps({"error": str(e)}), + content_type="application/json" + ) + + async def handle_mount_shm(self, request): + try: + data = await request.json() + name = data.get("name") + path = _shm_name_to_path(name) + offset = data.get("offset", 0) + size = data.get("size") + protocol = data.get("protocol", self.config.protocol) + location = data.get("location", "") + + if not path or size is None: + return web.Response( + status=400, + text=json.dumps({"error": "Missing or invalid name or size"}), + content_type="application/json" + ) + + result = self.store.mount_segment(path, size, offset, protocol, location) + if result["ret"] != 0: + return web.Response( + status=500, + text=json.dumps({"error": f"Mount failed, ret={result['ret']}"}), + content_type="application/json" + ) + + return web.Response( + status=200, + text=json.dumps( + { + "status": "success", + "segment_ids": list(result["segment_ids"]), + } + ), + content_type="application/json", + ) + except Exception as e: + logging.error("MOUNT_SHM error: %s", e) + return web.Response( + status=500, + text=json.dumps({"error": str(e)}), + content_type="application/json" + ) + + async def handle_unmount_shm(self, request): + try: + data = await request.json() + segment_ids = data.get("segment_ids", []) + if isinstance(segment_ids, str): + segment_ids = [segment_ids] + if not segment_ids: + return web.Response( + status=400, + text=json.dumps({"error": "Missing segment_ids"}), + content_type="application/json", + ) + + failed_segment_ids = [] + async with self._state_lock: + for sid in segment_ids: + ret = self.store.unmount_segment([sid]) + if ret != 0: + failed_segment_ids.append(sid) + continue + if sid in self.mounted_segment_ids: + self.mounted_segment_ids.remove(sid) + if not self.mounted_segment_ids: + self.current_mode = "prefill" + + if failed_segment_ids: + return web.Response( + status=500, + text=json.dumps( + { + "error": "Unmount failed for one or more segments", + "failed_segment_ids": failed_segment_ids, + } + ), + content_type="application/json", + ) + + return web.Response( + status=200, + text=json.dumps({"status": "success"}), + content_type="application/json", + ) + except Exception as e: + logging.error("UNMOUNT_SHM error: %s", e) + return web.Response( + status=500, + text=json.dumps({"error": str(e)}), + content_type="application/json" + ) + async def handle_put(self, request): try: data = await request.json() diff --git a/mooncake-wheel/tests/test_mooncake_store_service_api.py b/mooncake-wheel/tests/test_mooncake_store_service_api.py new file mode 100644 index 00000000..4efe71fe --- /dev/null +++ b/mooncake-wheel/tests/test_mooncake_store_service_api.py @@ -0,0 +1,201 @@ +#!/usr/bin/env python3 +import asyncio +import json +import sys +import types +import unittest +from pathlib import Path +from types import SimpleNamespace + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) + +try: + from aiohttp import web as _unused_web +except ModuleNotFoundError: + aiohttp_module = types.ModuleType("aiohttp") + web_module = types.ModuleType("aiohttp.web") + + class Response: + def __init__(self, status=200, text="", content_type=None): + self.status = status + self.text = text + self.content_type = content_type + + web_module.Response = Response + aiohttp_module.web = web_module + sys.modules["aiohttp"] = aiohttp_module + sys.modules["aiohttp.web"] = web_module + +try: + from mooncake.store import MooncakeDistributedStore as _unused_store +except ModuleNotFoundError: + store_module = types.ModuleType("mooncake.store") + + class MooncakeDistributedStore: + pass + + store_module.MooncakeDistributedStore = MooncakeDistributedStore + sys.modules["mooncake.store"] = store_module + +from mooncake.mooncake_store_service import MooncakeStoreService + + +class FakeStore: + def __init__(self): + self.mounted = {} + self.mount_calls = [] + self.unmount_calls = [] + self.fail_mount = False + self.unmount_failures = set() + + def mount_segment(self, path, size, offset, protocol, location): + self.mount_calls.append((path, size, offset, protocol, location)) + if self.fail_mount: + return {"ret": -1, "segment_ids": []} + segment_id = "00000000-0000-0000-0000-000000000001" + self.mounted[segment_id] = { + "path": path, + "size": size, + "offset": offset, + "protocol": protocol, + "location": location, + } + return {"ret": 0, "segment_ids": [segment_id]} + + def unmount_segment(self, segment_ids): + self.unmount_calls.append(list(segment_ids)) + for segment_id in segment_ids: + if segment_id in self.unmount_failures: + return -1 + for segment_id in segment_ids: + self.mounted.pop(segment_id, None) + return 0 + + +class FakeRequest: + def __init__(self, body): + self.body = body + + async def json(self): + return self.body + + +class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): + async def asyncSetUp(self): + self.fake_store = FakeStore() + self.service = MooncakeStoreService.__new__(MooncakeStoreService) + self.service.store = self.fake_store + self.service.config = SimpleNamespace(protocol="tcp") + self.service.current_mode = "prefill" + self.service.mounted_segment_ids = [] + self.service.last_mount_info = {} + self.service._state_lock = asyncio.Lock() + + async def test_mount_shm_then_unmount_shm_api(self): + mount_resp = await self.service.handle_mount_shm( + FakeRequest( + { + "name": "mooncake-segment", + "size": 4096, + "offset": 128, + "protocol": "tcp", + "location": "cpu:0", + } + ) + ) + self.assertEqual(mount_resp.status, 200) + mount_body = json.loads(mount_resp.text) + self.assertEqual(mount_body["status"], "success") + self.assertEqual( + mount_body["segment_ids"], + ["00000000-0000-0000-0000-000000000001"], + ) + self.assertEqual( + self.fake_store.mount_calls, + [("/dev/shm/mooncake-segment", 4096, 128, "tcp", "cpu:0")], + ) + + unmount_resp = await self.service.handle_unmount_shm( + FakeRequest({"segment_ids": mount_body["segment_ids"]}) + ) + self.assertEqual(unmount_resp.status, 200) + unmount_body = json.loads(unmount_resp.text) + self.assertEqual(unmount_body["status"], "success") + self.assertEqual( + self.fake_store.unmount_calls, + [["00000000-0000-0000-0000-000000000001"]], + ) + self.assertEqual(self.service.current_mode, "prefill") + + async def test_unmount_shm_updates_state_for_partial_success(self): + succeeded_id = "00000000-0000-0000-0000-000000000001" + failed_id = "00000000-0000-0000-0000-000000000002" + self.service.current_mode = "decode" + self.service.mounted_segment_ids = [succeeded_id, failed_id] + self.fake_store.unmount_failures = {failed_id} + + resp = await self.service.handle_unmount_shm( + FakeRequest({"segment_ids": [succeeded_id, failed_id]}) + ) + + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertEqual(body["failed_segment_ids"], [failed_id]) + self.assertEqual( + self.fake_store.unmount_calls, + [[succeeded_id], [failed_id]], + ) + self.assertEqual(self.service.mounted_segment_ids, [failed_id]) + self.assertEqual(self.service.current_mode, "decode") + + async def test_reconfigure_decode_mount_failure_rolls_back_to_prefill(self): + old_id = "00000000-0000-0000-0000-000000000001" + self.service.current_mode = "decode" + self.service.mounted_segment_ids = [old_id] + self.service.last_mount_info = { + "path": "/dev/shm/old", + "offset": 0, + "size": 4096, + "protocol": "tcp", + "location": "", + } + self.fake_store.fail_mount = True + + resp = await self.service.handle_reconfigure( + FakeRequest({"mode": "decode", "path": "/dev/shm/new", "size": 4096}) + ) + + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertEqual(body["mode"], "prefill") + self.assertIn("rolled back to prefill", body["error"]) + self.assertEqual(self.fake_store.unmount_calls, [[old_id]]) + self.assertEqual(self.service.mounted_segment_ids, []) + self.assertEqual(self.service.current_mode, "prefill") + self.assertEqual(self.service.last_mount_info, {}) + + async def test_mount_shm_requires_name_and_size(self): + resp = await self.service.handle_mount_shm( + FakeRequest({"name": "mooncake-segment"}) + ) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("name or size", body["error"]) + + async def test_mount_shm_rejects_path_like_name(self): + resp = await self.service.handle_mount_shm( + FakeRequest({"name": "../foo", "size": 4096}) + ) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("name or size", body["error"]) + + async def test_unmount_shm_requires_segment_ids(self): + resp = await self.service.handle_unmount_shm(FakeRequest({})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Missing segment_ids", body["error"]) + + +if __name__ == "__main__": + unittest.main() -- 2.34.1 From ee3e737d9b42a82b19a96fa9c08db3167c28eb25 Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Tue, 28 Apr 2026 20:55:22 +0800 Subject: [PATCH 019/382] [Store] Add Rust smoke test, benchmark, and CI coverage (#1927) * [Store] Add Rust smoke test, benchmark, and CI coverage --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> --- .github/workflows/ci.yml | 38 ++- mooncake-store/rust/CMakeLists.txt | 70 +++--- mooncake-store/rust/build.rs | 219 +++++++++++++++--- .../rust/examples/store_benchmark.rs | 130 +++++++++++ mooncake-store/rust/tests/minimal_smoke.rs | 76 ++++++ 5 files changed, 475 insertions(+), 58 deletions(-) create mode 100644 mooncake-store/rust/examples/store_benchmark.rs create mode 100644 mooncake-store/rust/tests/minimal_smoke.rs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fba448ac..50ef0da3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -43,6 +43,9 @@ jobs: with: python-version: ${{ matrix.python-version }} + - name: Install Rust toolchain + uses: dtolnay/rust-toolchain@stable + - name: Install and start etcd run: | wget https://github.com/etcd-io/etcd/releases/download/v3.6.1/etcd-v3.6.1-linux-amd64.tar.gz @@ -125,6 +128,33 @@ jobs: python ./bootstrap_server.py & shell: bash + - name: Run Mooncake Store Rust smoke test and benchmark + run: | + $GITHUB_WORKSPACE/build/mooncake-store/src/mooncake_master \ + --eviction_high_watermark_ratio=0.95 \ + --cluster_id=ci_rust_test_cluster \ + --port 50051 & + MASTER_PID=$! + sleep 3 + cd mooncake-store/rust + export LD_LIBRARY_PATH=$GITHUB_WORKSPACE/build/mooncake-asio:$GITHUB_WORKSPACE/build/mooncake-store/src:$GITHUB_WORKSPACE/build/mooncake-store/src/cachelib_memory_allocator:$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src:$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src/common/base:$GITHUB_WORKSPACE/build/mooncake-common/etcd:$LD_LIBRARY_PATH + export MOONCAKE_BUILD_DIR=$GITHUB_WORKSPACE/build + export MOONCAKE_STORE_LIB_DIR=$GITHUB_WORKSPACE/build/mooncake-store/src + export MOONCAKE_STORE_INCLUDE_DIR=$GITHUB_WORKSPACE/mooncake-store/include + export MC_METADATA_SERVER=http://127.0.0.1:8080/metadata + export MC_RUST_STORE_RUN_INTEGRATION=true + export MC_RUST_STORE_MASTER_ADDR=127.0.0.1:50051 + export MC_RUST_STORE_LOCAL_HOSTNAME=127.0.0.1 + export MC_RUST_STORE_PROTOCOL=tcp + export MC_RUST_STORE_DEVICE_NAME= + cargo test --test minimal_smoke -- --nocapture + MC_RUST_BENCH_ITERATIONS=4 \ + MC_RUST_BENCH_VALUE_SIZE=4096 \ + MC_RUST_BENCH_WARMUP=1 \ + cargo run --release --example store_benchmark + kill $MASTER_PID 2>/dev/null || true + shell: bash + - name: Run Go store binding integration tests run: | $GITHUB_WORKSPACE/build/mooncake-store/src/mooncake_master \ @@ -557,12 +587,14 @@ jobs: sudo cmake --install . shell: bash - - name: Check Mooncake Store Rust bindings and example + - name: Check Mooncake Store Rust bindings, examples, and tests run: | cd mooncake-store/rust + export MOONCAKE_BUILD_DIR=$GITHUB_WORKSPACE/build + cargo test --lib MOONCAKE_STORE_LIB_DIR=$GITHUB_WORKSPACE/build/mooncake-store/src \ MOONCAKE_STORE_INCLUDE_DIR=$GITHUB_WORKSPACE/mooncake-store/include \ - cargo check --example basic_usage --tests + cargo test --examples --tests --no-run shell: bash - name: Configure project @@ -696,7 +728,7 @@ jobs: echo "Error: code_format.sh not found or not executable" exit 1 fi - + # Determine base ref for comparison if [ "${{ github.event_name }}" == "pull_request" ]; then # For PRs: compare against the target branch diff --git a/mooncake-store/rust/CMakeLists.txt b/mooncake-store/rust/CMakeLists.txt index 54908527..49fa328f 100644 --- a/mooncake-store/rust/CMakeLists.txt +++ b/mooncake-store/rust/CMakeLists.txt @@ -1,39 +1,51 @@ -# CMake integration for the mooncake_store Rust crate. +# CMake integration for the mooncake_store Rust package. # # This target is only added when the parent CMakeLists.txt sets -# WITH_STORE_RUST=ON. It depends on the mooncake_store C++ target so that -# the static library is available before cargo tries to link against it. +# WITH_STORE_RUST=ON. It depends on the mooncake_store C++ target so that the +# static library is available before cargo tries to link against it. add_custom_target(build_mooncake_store_rust DEPENDS mooncake_store) add_custom_command( - TARGET build_mooncake_store_rust - COMMAND - ${CMAKE_COMMAND} -E env - # Where cargo should place its build artefacts. - CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR} - # Directory containing libmooncake_store.a (or .so). - MOONCAKE_STORE_LIB_DIR=${CMAKE_CURRENT_BINARY_DIR}/../src - # Directory containing store_c.h. - MOONCAKE_STORE_INCLUDE_DIR=${CMAKE_CURRENT_SOURCE_DIR}/../include - cargo build --release - WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} - COMMENT "Building mooncake_store Rust crate" - VERBATIM -) + TARGET build_mooncake_store_rust + COMMAND + ${CMAKE_COMMAND} -E env + # Where cargo should place its build artefacts. + CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR} + MOONCAKE_BUILD_DIR=${PROJECT_BINARY_DIR} + # Directory containing libmooncake_store.a (or .so). + MOONCAKE_STORE_LIB_DIR=${CMAKE_CURRENT_BINARY_DIR}/../src + # Directory containing store_c.h. + MOONCAKE_STORE_INCLUDE_DIR=${CMAKE_CURRENT_SOURCE_DIR}/../include cargo + build --release + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} + COMMENT "Building mooncake_store Rust package" + VERBATIM) -# Build the basic_usage example so that CI can verify it compiles. add_custom_target(build_mooncake_store_rust_example DEPENDS mooncake_store) add_custom_command( - TARGET build_mooncake_store_rust_example - COMMAND - ${CMAKE_COMMAND} -E env - CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR} - MOONCAKE_STORE_LIB_DIR=${CMAKE_CURRENT_BINARY_DIR}/../src - MOONCAKE_STORE_INCLUDE_DIR=${CMAKE_CURRENT_SOURCE_DIR}/../include - cargo build --example basic_usage --release - WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} - COMMENT "Building mooncake_store Rust basic_usage example" - VERBATIM -) + TARGET build_mooncake_store_rust_example + COMMAND + ${CMAKE_COMMAND} -E env CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR} + MOONCAKE_BUILD_DIR=${PROJECT_BINARY_DIR} + MOONCAKE_STORE_LIB_DIR=${CMAKE_CURRENT_BINARY_DIR}/../src + MOONCAKE_STORE_INCLUDE_DIR=${CMAKE_CURRENT_SOURCE_DIR}/../include cargo + build --examples --release + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} + COMMENT "Building mooncake_store Rust examples" + VERBATIM) + +add_custom_target(build_mooncake_store_rust_tests DEPENDS mooncake_store) + +add_custom_command( + TARGET build_mooncake_store_rust_tests + COMMAND + ${CMAKE_COMMAND} -E env CARGO_TARGET_DIR=${CMAKE_CURRENT_BINARY_DIR} + MOONCAKE_BUILD_DIR=${PROJECT_BINARY_DIR} + MOONCAKE_STORE_LIB_DIR=${CMAKE_CURRENT_BINARY_DIR}/../src + MOONCAKE_STORE_INCLUDE_DIR=${CMAKE_CURRENT_SOURCE_DIR}/../include cargo test + --tests --no-run --release + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} + COMMENT "Building mooncake_store Rust tests" + VERBATIM) diff --git a/mooncake-store/rust/build.rs b/mooncake-store/rust/build.rs index 99d1714e..6f817b9c 100644 --- a/mooncake-store/rust/build.rs +++ b/mooncake-store/rust/build.rs @@ -14,48 +14,215 @@ use std::env; use std::path::PathBuf; +use std::process::Command; + +fn push_existing_dir(search_dirs: &mut Vec, dir: PathBuf) { + if dir.is_dir() && !search_dirs.iter().any(|existing| existing == &dir) { + search_dirs.push(dir); + } +} + +fn push_env_paths(search_dirs: &mut Vec, name: &str) { + if let Some(value) = env::var_os(name) { + for dir in env::split_paths(&value) { + push_existing_dir(search_dirs, dir); + } + } +} + +fn has_library(search_dirs: &[PathBuf], candidates: &[&str]) -> bool { + search_dirs.iter().any(|dir| { + candidates.iter().any(|candidate| { + ["a", "so", "dylib"] + .into_iter() + .map(|ext| dir.join(format!("lib{candidate}.{ext}"))) + .any(|path| path.exists()) + }) + }) +} + +fn emit_link_searches(search_dirs: &[PathBuf]) { + for dir in search_dirs { + println!("cargo:rustc-link-search=native={}", dir.display()); + } +} + +fn emit_runtime_rpaths(search_dirs: &[PathBuf]) { + for dir in search_dirs { + let dir_str = dir.display(); + println!("cargo:rustc-link-arg-tests=-Wl,-rpath,{dir_str}"); + println!("cargo:rustc-link-arg-examples=-Wl,-rpath,{dir_str}"); + } +} + +fn compiler_candidates() -> Vec { + let mut tools = Vec::new(); + + for env_var in ["CC", "CXX"] { + if let Ok(value) = env::var(env_var) { + let tool = value.trim(); + if !tool.is_empty() && !tools.iter().any(|existing| existing == tool) { + tools.push(tool.to_string()); + } + } + } + + for tool in ["gcc", "cc", "clang", "c++"] { + if !tools.iter().any(|existing| existing == tool) { + tools.push(tool.to_string()); + } + } + + tools +} + +fn compiler_runtime_library(file_name: &str) -> Option { + for tool in compiler_candidates() { + let output = Command::new(&tool) + .arg(format!("-print-file-name={file_name}")) + .output() + .ok()?; + + if !output.status.success() { + continue; + } + + let path = String::from_utf8(output.stdout).ok()?; + let path = PathBuf::from(path.trim()); + if path.as_os_str().is_empty() || path == PathBuf::from(file_name) || !path.exists() { + continue; + } + + return Some(path); + } + + None +} + +fn add_compiler_runtime_search_dir(search_dirs: &mut Vec, file_name: &str) -> bool { + let Some(path) = compiler_runtime_library(file_name) else { + return false; + }; + + if let Some(parent) = path.parent() { + push_existing_dir(search_dirs, parent.to_path_buf()); + return true; + } + + false +} fn main() { - // ----------------------------------------------------------------------- - // Library search path - // - // When built via CMake (WITH_STORE_RUST=ON) the CMakeLists.txt injects - // MOONCAKE_STORE_LIB_DIR pointing at the directory that contains - // libmooncake_store.a/.so. When cargo is invoked standalone the caller - // should set the variable manually or rely on the default convention of a - // sibling `build/` directory produced by a top-level CMake configure. - // ----------------------------------------------------------------------- - let lib_dir = env::var("MOONCAKE_STORE_LIB_DIR") - .unwrap_or_else(|_| "../../build/mooncake-store/src".to_string()); + let manifest_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").expect("missing CARGO_MANIFEST_DIR")); + let mut search_dirs = Vec::new(); - println!("cargo:rustc-link-search=native={lib_dir}"); - println!("cargo:rustc-link-lib=mooncake_store"); + let explicit_lib_dir = env::var("MOONCAKE_STORE_LIB_DIR").ok().map(PathBuf::from); + if let Some(dir) = explicit_lib_dir.clone() { + push_existing_dir(&mut search_dirs, dir); + } - // Dependencies of mooncake_store that must be satisfied at link time. - // The list mirrors what mooncake-store/src/CMakeLists.txt links against. - println!("cargo:rustc-link-lib=transfer_engine"); - println!("cargo:rustc-link-lib=stdc++"); - println!("cargo:rustc-link-lib=glog"); - println!("cargo:rustc-link-lib=gflags"); - println!("cargo:rustc-link-lib=pthread"); - println!("cargo:rustc-link-lib=xxhash"); + if let Ok(build_dir) = env::var("MOONCAKE_BUILD_DIR") { + let build_dir = PathBuf::from(build_dir); + for dir in [ + build_dir.join("mooncake-store/src"), + build_dir.join("mooncake-store/src/cachelib_memory_allocator"), + build_dir.join("mooncake-transfer-engine/src"), + build_dir.join("mooncake-transfer-engine/src/common/base"), + build_dir.join("mooncake-asio"), + build_dir.join("mooncake-common/etcd"), + ] { + push_existing_dir(&mut search_dirs, dir); + } + } + + let default_build_dir = manifest_dir.join("../../build"); + for dir in [ + default_build_dir.join("mooncake-store/src"), + default_build_dir.join("mooncake-store/src/cachelib_memory_allocator"), + default_build_dir.join("mooncake-transfer-engine/src"), + default_build_dir.join("mooncake-transfer-engine/src/common/base"), + default_build_dir.join("mooncake-asio"), + default_build_dir.join("mooncake-common/etcd"), + PathBuf::from("/usr/local/lib"), + PathBuf::from("/usr/lib/x86_64-linux-gnu"), + PathBuf::from("/lib/x86_64-linux-gnu"), + ] { + push_existing_dir(&mut search_dirs, dir); + } + + push_env_paths(&mut search_dirs, "LD_LIBRARY_PATH"); + push_env_paths(&mut search_dirs, "LIBRARY_PATH"); + + let asan_runtime_so = compiler_runtime_library("libasan.so"); + if let Some(path) = asan_runtime_so.as_ref() { + if let Some(parent) = path.parent() { + push_existing_dir(&mut search_dirs, parent.to_path_buf()); + } + } + let has_asan_runtime = asan_runtime_so.is_some() + || add_compiler_runtime_search_dir(&mut search_dirs, "libasan.a"); + let has_gcov_runtime = add_compiler_runtime_search_dir(&mut search_dirs, "libgcov.a") + || add_compiler_runtime_search_dir(&mut search_dirs, "libgcov.so"); + + emit_link_searches(&search_dirs); + emit_runtime_rpaths(&search_dirs); + + if has_asan_runtime || has_library(&search_dirs, &["asan"]) { + println!("cargo:rustc-link-lib=asan"); + } + + for library in [ + "mooncake_store", + "cachelib_memory_allocator", + "transfer_engine", + "base", + "asio", + "stdc++", + "glog", + "gflags", + "pthread", + "xxhash", + "numa", + "ibverbs", + "jsoncpp", + "zstd", + "m", + ] { + println!("cargo:rustc-link-lib={library}"); + } + + for (link_name, candidates) in [ + ("etcd_wrapper", &["etcd_wrapper"] as &[&str]), + ("hiredis", &["hiredis"]), + ("curl", &["curl"]), + ("cudart", &["cudart"]), + ("uring", &["uring"]), + ] { + if has_library(&search_dirs, candidates) { + println!("cargo:rustc-link-lib={link_name}"); + } + } + + if has_gcov_runtime || has_library(&search_dirs, &["gcov"]) { + println!("cargo:rustc-link-lib=gcov"); + } - // ----------------------------------------------------------------------- - // Header path for bindgen - // ----------------------------------------------------------------------- let include_dir = env::var("MOONCAKE_STORE_INCLUDE_DIR") .unwrap_or_else(|_| "../include".to_string()); let header = format!("{include_dir}/store_c.h"); - // Re-run this build script if the C header changes. println!("cargo:rerun-if-changed={header}"); + println!("cargo:rerun-if-env-changed=MOONCAKE_BUILD_DIR"); println!("cargo:rerun-if-env-changed=MOONCAKE_STORE_LIB_DIR"); println!("cargo:rerun-if-env-changed=MOONCAKE_STORE_INCLUDE_DIR"); + println!("cargo:rerun-if-env-changed=LD_LIBRARY_PATH"); + println!("cargo:rerun-if-env-changed=LIBRARY_PATH"); + println!("cargo:rerun-if-env-changed=CC"); + println!("cargo:rerun-if-env-changed=CXX"); let bindings = bindgen::Builder::default() .header(&header) - // Only pull in declarations from store_c.h (no transitive system headers). .allowlist_function("mooncake_store_.*") .allowlist_type("mooncake_.*") .generate() diff --git a/mooncake-store/rust/examples/store_benchmark.rs b/mooncake-store/rust/examples/store_benchmark.rs new file mode 100644 index 00000000..dc20ae03 --- /dev/null +++ b/mooncake-store/rust/examples/store_benchmark.rs @@ -0,0 +1,130 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use std::time::{Instant, SystemTime, UNIX_EPOCH}; + +use mooncake_store::MooncakeStore; + +fn env_or_default(key: &str, default: T) -> T +where + T: std::str::FromStr, +{ + std::env::var(key) + .ok() + .and_then(|value| value.parse::().ok()) + .unwrap_or(default) +} + +fn unique_prefix() -> String { + let timestamp_ns = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|duration| duration.as_nanos()) + .unwrap_or_default(); + format!("rust-bench-{}-{timestamp_ns}", std::process::id()) +} + +fn mib_per_second(total_bytes: usize, elapsed: std::time::Duration) -> f64 { + if elapsed.is_zero() { + return 0.0; + } + total_bytes as f64 / (1024.0 * 1024.0) / elapsed.as_secs_f64() +} + +fn main() -> Result<(), Box> { + let iterations = env_or_default("MC_RUST_BENCH_ITERATIONS", 64usize); + let payload_size = env_or_default("MC_RUST_BENCH_VALUE_SIZE", 64 * 1024usize); + let warmup_iterations = env_or_default("MC_RUST_BENCH_WARMUP", 4usize); + let metadata_server = std::env::var("MC_METADATA_SERVER") + .unwrap_or_else(|_| "http://127.0.0.1:8080/metadata".to_string()); + let master_server_addr = std::env::var("MC_RUST_STORE_MASTER_ADDR") + .unwrap_or_else(|_| "127.0.0.1:50051".to_string()); + let local_hostname = + std::env::var("MC_RUST_STORE_LOCAL_HOSTNAME").unwrap_or_else(|_| "localhost".to_string()); + let protocol = std::env::var("MC_RUST_STORE_PROTOCOL").unwrap_or_else(|_| "tcp".to_string()); + let device_name = std::env::var("MC_RUST_STORE_DEVICE_NAME").unwrap_or_default(); + let global_segment_size = env_or_default("MC_RUST_STORE_GLOBAL_SEGMENT_SIZE", 512u64 << 20); + let local_buffer_size = env_or_default("MC_RUST_STORE_LOCAL_BUFFER_SIZE", 128u64 << 20); + + let store = MooncakeStore::new()?; + store.setup( + &local_hostname, + &metadata_server, + global_segment_size, + local_buffer_size, + &protocol, + &device_name, + &master_server_addr, + )?; + store.health_check()?; + + let prefix = unique_prefix(); + let payload: Vec = (0..payload_size).map(|index| (index % 251) as u8).collect(); + + for iteration in 0..warmup_iterations { + let key = format!("{prefix}-warmup-{iteration}"); + store.put(&key, &payload, None)?; + let fetched = store.get(&key)?; + assert_eq!(fetched, payload, "warmup round-trip mismatch"); + store.remove(&key, true)?; + } + + let keys: Vec = (0..iterations) + .map(|iteration| format!("{prefix}-key-{iteration}")) + .collect(); + + let put_start = Instant::now(); + for key in &keys { + store.put(key, &payload, None)?; + } + let put_elapsed = put_start.elapsed(); + + let get_start = Instant::now(); + for key in &keys { + let fetched = store.get(key)?; + assert_eq!(fetched, payload, "benchmark round-trip mismatch for {key}"); + } + let get_elapsed = get_start.elapsed(); + + let remove_start = Instant::now(); + for key in &keys { + store.remove(key, true)?; + } + let remove_elapsed = remove_start.elapsed(); + + let total_bytes = iterations * payload.len(); + println!("Mooncake Store Rust benchmark"); + println!("iterations={iterations}"); + println!("payload_size_bytes={payload_size}"); + println!("put_seconds={:.6}", put_elapsed.as_secs_f64()); + println!( + "put_mib_per_sec={:.2}", + mib_per_second(total_bytes, put_elapsed) + ); + println!("get_seconds={:.6}", get_elapsed.as_secs_f64()); + println!( + "get_mib_per_sec={:.2}", + mib_per_second(total_bytes, get_elapsed) + ); + println!("remove_seconds={:.6}", remove_elapsed.as_secs_f64()); + println!( + "remove_ops_per_sec={:.2}", + if remove_elapsed.is_zero() { + 0.0 + } else { + iterations as f64 / remove_elapsed.as_secs_f64() + } + ); + + Ok(()) +} diff --git a/mooncake-store/rust/tests/minimal_smoke.rs b/mooncake-store/rust/tests/minimal_smoke.rs new file mode 100644 index 00000000..1e95cff5 --- /dev/null +++ b/mooncake-store/rust/tests/minimal_smoke.rs @@ -0,0 +1,76 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +use std::time::{SystemTime, UNIX_EPOCH}; + +use mooncake_store::MooncakeStore; + +fn env_or_default(key: &str, default: &str) -> String { + std::env::var(key).unwrap_or_else(|_| default.to_string()) +} + +fn should_run_integration() -> bool { + matches!( + std::env::var("MC_RUST_STORE_RUN_INTEGRATION").as_deref(), + Ok("1") | Ok("true") | Ok("TRUE") + ) +} + +fn unique_key() -> String { + let timestamp_ns = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|duration| duration.as_nanos()) + .unwrap_or_default(); + format!("rust-smoke-{}-{timestamp_ns}", std::process::id()) +} + +#[test] +fn smoke_round_trip() -> Result<(), Box> { + if !should_run_integration() { + eprintln!("skipping Mooncake Store Rust smoke test; set MC_RUST_STORE_RUN_INTEGRATION=1 to enable"); + return Ok(()); + } + + let metadata_server = env_or_default("MC_METADATA_SERVER", "http://127.0.0.1:8080/metadata"); + let master_server_addr = env_or_default("MC_RUST_STORE_MASTER_ADDR", "127.0.0.1:50051"); + let local_hostname = env_or_default("MC_RUST_STORE_LOCAL_HOSTNAME", "localhost"); + let protocol = env_or_default("MC_RUST_STORE_PROTOCOL", "tcp"); + let device_name = std::env::var("MC_RUST_STORE_DEVICE_NAME").unwrap_or_default(); + + let store = MooncakeStore::new()?; + store.setup( + &local_hostname, + &metadata_server, + 512 << 20, + 128 << 20, + &protocol, + &device_name, + &master_server_addr, + )?; + store.health_check()?; + + let key = unique_key(); + let value = format!("mooncake-rust-smoke-value-{key}").into_bytes(); + + store.put(&key, &value, None)?; + assert!(store.is_exist(&key)?); + assert_eq!(store.get_size(&key)? as usize, value.len()); + assert_eq!(store.get(&key)?, value); + assert!(!store.get_hostname()?.is_empty()); + + store.remove(&key, true)?; + assert!(!store.is_exist(&key)?); + + Ok(()) +} -- 2.34.1 From 1943f9782ebc7d551ae7dede0ba26214a52749a8 Mon Sep 17 00:00:00 2001 From: zhangzuo21 <99539591+zhangzuo21@users.noreply.github.com> Date: Wed, 29 Apr 2026 11:11:10 +0800 Subject: [PATCH 020/382] [Bug fix]: async offload RPC to prevent client expiration and shm_not_mapped (#1734) Fix/issue 1711 async offload rpc --- mooncake-store/include/real_client.h | 14 ++++--- mooncake-store/src/real_client.cpp | 63 +++++++++++++++++++++++----- 2 files changed, 62 insertions(+), 15 deletions(-) diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index 7982de45..c3551e37 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -19,6 +19,8 @@ #include "rpc_types.h" #include #include +#include +#include namespace mooncake { @@ -415,10 +417,11 @@ class RealClient : public PyClient { const std::string &key, std::vector> values, const ReplicateConfig &config, const UUID &client_id); - std::vector> batch_get_into_dummy_helper( - const std::vector &keys, - const std::vector &buffers, const std::vector &sizes, - int32_t device_id, const UUID &client_id); + async_simple::coro::Lazy>> + batch_get_into_dummy_helper(const std::vector &keys, + const std::vector &buffers, + const std::vector &sizes, + int32_t device_id, const UUID &client_id); std::vector> batch_put_from_dummy_helper( const std::vector &keys, @@ -657,7 +660,8 @@ class RealClient : public PyClient { tl::expected ping(const UUID &client_id); - tl::expected + async_simple::coro::Lazy< + tl::expected> batch_get_offload_object(const std::vector &keys, const std::vector &sizes); diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 7d28ed26..239e126b 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -3445,7 +3445,7 @@ std::vector RealClient::batch_get_into( return results; } -std::vector> +async_simple::coro::Lazy>> RealClient::batch_get_into_dummy_helper( const std::vector &keys, const std::vector &dummy_buffers, @@ -3455,16 +3455,19 @@ RealClient::batch_get_into_dummy_helper( if (!ContextManager::getInstance().setCurrentContextByPhysicalId( device_id)) { LOG(ERROR) << "Failed to set context for physical device " << device_id; - return std::vector>( + co_return std::vector>( keys.size(), tl::unexpected(ErrorCode::INVALID_PARAMS)); } #endif - std::shared_lock lock(dummy_client_mutex_); + // Hold shared_lock for the entire operation to prevent SHM from being + // unmapped while batch_get_into_internal is using the translated buffers. + auto lock = std::make_shared>( + dummy_client_mutex_); auto it = shm_contexts_.find(client_id); if (it == shm_contexts_.end()) { LOG(ERROR) << "client_id=" << client_id << ", error=shm_not_mapped"; - return std::vector>( + co_return std::vector>( keys.size(), tl::unexpected(ErrorCode::INVALID_PARAMS)); } auto &context = it->second; @@ -3472,10 +3475,35 @@ RealClient::batch_get_into_dummy_helper( auto buffers_result = map_dummy_addrs_to_real_ptrs(context, dummy_buffers, sizes, client_id); if (!buffers_result) { - return std::vector>( + co_return std::vector>( keys.size(), tl::unexpected(buffers_result.error())); } - return batch_get_into_internal(keys, buffers_result.value(), sizes); + + // Run batch_get_into_internal (which may block on offload RPC + SSD I/O) + // on a dedicated thread pool so the coro_rpc IO thread stays free for ping. + // + // Pack everything the worker-thread call needs into a single heap-owned + // state object so the lambda captures only a raw pointer (trivially + // copyable, trivially destructible). Avoids GCC 11 coroutine + coro_io + // double-destruction interactions when non-trivial captures end up in + // both the OUTER coroutine frame and the post_helper chain. + struct CallState { + std::vector keys; + std::vector sizes; + std::vector buffers; + std::shared_ptr> lock; + }; + auto state = std::make_unique(); + state->keys = keys; + state->sizes = sizes; + state->buffers = std::move(buffers_result.value()); + state->lock = std::move(lock); + + auto *s = state.get(); + auto try_result = co_await coro_io::post([this, s]() { + return batch_get_into_internal(s->keys, s->buffers, s->sizes); + }); + co_return try_result.value(); } std::vector> @@ -4443,16 +4471,31 @@ tl::expected RealClient::query_task( const UUID &task_id) { return client_->QueryTask(task_id); } -tl::expected +async_simple::coro::Lazy> RealClient::batch_get_offload_object(const std::vector &keys, const std::vector &sizes) { - auto result = file_storage_->BatchGet(keys, sizes); + // Run SSD I/O on a dedicated thread pool so the coro_rpc IO thread is + // free to handle ping and other RPCs. Same heap-owned state pattern as + // batch_get_into_dummy_helper: the lambda captures only a raw pointer. + struct CallState { + std::vector keys; + std::vector sizes; + std::shared_ptr file_storage; + }; + auto state = std::make_unique(); + state->keys = keys; + state->sizes = sizes; + state->file_storage = file_storage_; + auto *s = state.get(); + auto try_result = co_await coro_io::post( + [s]() { return s->file_storage->BatchGet(s->keys, s->sizes); }); + auto result = try_result.value(); if (!result) { LOG(ERROR) << "Batch get offload object failed,err_code = " << result.error(); - return tl::make_unexpected(result.error()); + co_return tl::make_unexpected(result.error()); } - return BatchGetOffloadObjectResponse( + co_return BatchGetOffloadObjectResponse( result.value().batch_id, std::move(result.value().pointers), client_->GetTransportEndpoint(), file_storage_->config_.client_buffer_gc_ttl_ms); -- 2.34.1 From 904b9caa7f42fbb4b01ed29a002cb528bb66fab0 Mon Sep 17 00:00:00 2001 From: Colors-111 <70190328+Colors-111@users.noreply.github.com> Date: Wed, 29 Apr 2026 11:13:01 +0800 Subject: [PATCH 021/382] [Store] Fix: KEYS_ULTRA_LIMIT error in eviction-enabled BucketStorageBackend (#1941) Co-authored-by: ruanzhao --- mooncake-store/src/file_storage.cpp | 8 ++++++ mooncake-store/src/master_service.cpp | 38 +++++++++++++++++++++++--- mooncake-store/src/storage_backend.cpp | 7 +++++ 3 files changed, 49 insertions(+), 4 deletions(-) diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index 5b8cf452..d75c8ac5 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -223,6 +223,13 @@ tl::expected FileStorage::Init() { return init_storage_backend_result; } auto enable_offloading_result = IsEnableOffloading(); + if (enable_offloading_result.has_value()) { + LOG(INFO) << "IsEnableOffloading result: " + << (enable_offloading_result.value() ? "true" : "false"); + } else { + LOG(INFO) << "IsEnableOffloading result: error: " + << enable_offloading_result.error(); + } if (!enable_offloading_result) { LOG(ERROR) << "Failed to get enable persist result, error : " << enable_offloading_result.error(); @@ -479,6 +486,7 @@ tl::expected FileStorage::Heartbeat() { LOG(ERROR) << "client is nullptr"; return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } + std::unordered_map offloading_objects; // Objects selected for offloading diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 799a3cdc..f748a788 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -2135,10 +2135,40 @@ auto MasterService::OffloadObjectHeartbeat(const UUID& client_id, << client_id; return tl::make_unexpected(ErrorCode::SEGMENT_NOT_FOUND); } - MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); - local_disk_segment_it->second->enable_offloading = enable_offloading; - if (enable_offloading) { - return std::move(local_disk_segment_it->second->offloading_objects); + std::unordered_map offloading_objects_copy; + { + MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); + local_disk_segment_it->second->enable_offloading = enable_offloading; + if (enable_offloading) { + return std::move(local_disk_segment_it->second->offloading_objects); + } + // Offloading is disabled: clear the pending queue to prevent + // unbounded growth that would trigger KEYS_ULTRA_LIMIT in + // PushOffloadingQueue. We must also clean up corresponding + // offloading_tasks and decrement source replica refcounts to avoid + // resource leaks and blocked writes (OBJECT_HAS_REPLICATION_TASK). + // Copy keys out before releasing the mutex to avoid lock order + // violation: the lock order is Shard Lock -> offloading_mutex_, so we + // must release offloading_mutex_ before taking shard locks via + // MetadataAccessorRW. + offloading_objects_copy = + std::move(local_disk_segment_it->second->offloading_objects); + } + + for (auto& [key, size] : offloading_objects_copy) { + MetadataAccessorRW accessor(this, key); + if (accessor.Exists()) { + auto& shard = accessor.GetShard(); + auto task_it = shard->offloading_tasks.find(key); + if (task_it != shard->offloading_tasks.end()) { + auto source = + accessor.Get().GetReplicaByID(task_it->second.source_id); + if (source) { + source->dec_refcnt(); + } + shard->offloading_tasks.erase(task_it); + } + } } return {}; } diff --git a/mooncake-store/src/storage_backend.cpp b/mooncake-store/src/storage_backend.cpp index 24540ab6..a522ef28 100644 --- a/mooncake-store/src/storage_backend.cpp +++ b/mooncake-store/src/storage_backend.cpp @@ -1764,6 +1764,13 @@ tl::expected BucketStorageBackend::IsExist( } tl::expected BucketStorageBackend::IsEnableOffloading() { + // When eviction is enabled, always allow offloading since PrepareEviction + // will manage capacity by evicting old buckets as needed. + if (bucket_backend_config_.eviction_policy != BucketEvictionPolicy::NONE && + bucket_backend_config_.max_total_size > 0) { + return true; + } + auto store_metadata_result = GetStoreMetadata(); if (!store_metadata_result) { LOG(ERROR) << "Failed to get store metadata: " -- 2.34.1 From b0bda8caa4e729208d3d8c073091851ab5f55ac1 Mon Sep 17 00:00:00 2001 From: fatSheep <50252876+00fish0@users.noreply.github.com> Date: Wed, 29 Apr 2026 14:31:10 +0800 Subject: [PATCH 022/382] Exclude ub_transport_test from CI ctest (#2007) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- .github/workflows/ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 50ef0da3..b95b6fc6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -179,7 +179,7 @@ jobs: cd build export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib ldconfig -v || echo "always continue" - MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -j --output-on-failure + MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -j --output-on-failure -E ub_transport_test shell: bash - name: Drain HTTP E2E test -- 2.34.1 From dc4c4f8004188ffd6024e9d5da1023bfede575ab Mon Sep 17 00:00:00 2001 From: tancz <544463199@qq.com> Date: Thu, 30 Apr 2026 10:52:40 +0800 Subject: [PATCH 023/382] [Store] fix invalid acquire with release for atomic store (#1997) (#1996) Signed-off-by: tan changzhi <544463199@qq.com> --- mooncake-store/src/storage_backend.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mooncake-store/src/storage_backend.cpp b/mooncake-store/src/storage_backend.cpp index a522ef28..c269d477 100644 --- a/mooncake-store/src/storage_backend.cpp +++ b/mooncake-store/src/storage_backend.cpp @@ -1155,7 +1155,7 @@ tl::expected StorageBackendAdaptor::ScanMeta( fs::path root = fs::path(file_storage_config_.storage_filepath) / file_per_key_config_.fsdir; if (!fs::exists(root)) { - meta_scanned_.store(true, std::memory_order_acquire); + meta_scanned_.store(true, std::memory_order_release); return {}; } @@ -1228,7 +1228,7 @@ tl::expected StorageBackendAdaptor::ScanMeta( } } - meta_scanned_.store(true, std::memory_order_acquire); + meta_scanned_.store(true, std::memory_order_release); return {}; } -- 2.34.1 From c19c7b52ee948d2e667f933af612e3dc16b1d242 Mon Sep 17 00:00:00 2001 From: Chao Lei Date: Thu, 30 Apr 2026 16:12:41 +0800 Subject: [PATCH 024/382] [Store] Fix LOCAL_MEMCPY segfault for multi-process-per-node deployments (e.g. vLLM data-parallel) (#1995) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [Store] Fix LOCAL_MEMCPY segfault in multi-process and GPU memory scenarios Two bugs caused segfaults when MC_STORE_MEMCPY was enabled with multiple processes on the same node (e.g. vLLM DP8 + mp executor): 1. isLocalTransfer() compared only IPs (stripping ports via extractIpAddress), so all processes on the same host were incorrectly identified as "local". Cross-process virtual addresses are invalid → segfault. Fix: compare full ip:port endpoint, restoring the correct behavior from before #1226. 2. workerThread() used bare std::memcpy which cannot operate on cudaMalloc device pointers from CPU code → segfault. Fix: detect GPU pointers via IsDevicePointer() and use cudaMemcpy (cudaMemcpyDefault) for GPU memory, keeping std::memcpy for the CPU-only fast path. Add gpu_staging::CopyAuto() to gpu_staging_utils.h that uses cudaMemcpyDefault/hipMemcpyDefault for auto-direction copy. * Modify code format Signed-off-by: LCAIZJ --------- Signed-off-by: LCAIZJ Co-authored-by: leichao.lc --- mooncake-store/include/gpu_staging_utils.h | 31 ++++++++++ mooncake-store/src/transfer_task.cpp | 69 ++++++++++------------ 2 files changed, 61 insertions(+), 39 deletions(-) diff --git a/mooncake-store/include/gpu_staging_utils.h b/mooncake-store/include/gpu_staging_utils.h index d99bd477..6c2c4dc4 100644 --- a/mooncake-store/include/gpu_staging_utils.h +++ b/mooncake-store/include/gpu_staging_utils.h @@ -60,6 +60,37 @@ inline bool CopyDeviceToHost(void* dst, const void* src, size_t size) { #endif } +// Auto-direction copy: runtime determines the transfer direction from pointer +// attributes (cudaMemcpyDefault). Works for H2H, H2D, D2H, and D2D. +// Caller must have called SetDevice first when device memory is involved. +inline bool CopyAuto(void* dst, const void* src, size_t size) { +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) + return cudaMemcpy(dst, src, size, cudaMemcpyDefault) == cudaSuccess; +#elif defined(USE_HIP) + return hipMemcpy(dst, src, size, hipMemcpyDefault) == hipSuccess; +#elif defined(USE_ASCEND) || defined(USE_ASCEND_DIRECT) || defined(USE_UBSHMEM) + aclrtPtrAttributes src_attr{}, dst_attr{}; + bool src_dev = aclrtPointerGetAttributes(const_cast(src), + &src_attr) == ACL_SUCCESS && + src_attr.location.type == ACL_MEM_LOCATION_TYPE_DEVICE; + bool dst_dev = aclrtPointerGetAttributes(dst, &dst_attr) == ACL_SUCCESS && + dst_attr.location.type == ACL_MEM_LOCATION_TYPE_DEVICE; + aclrtMemcpyKind kind = ACL_MEMCPY_HOST_TO_HOST; + if (src_dev && dst_dev) + kind = ACL_MEMCPY_DEVICE_TO_DEVICE; + else if (src_dev) + kind = ACL_MEMCPY_DEVICE_TO_HOST; + else if (dst_dev) + kind = ACL_MEMCPY_HOST_TO_DEVICE; + return aclrtMemcpy(dst, size, src, size, kind) == ACL_SUCCESS; +#else + (void)dst; + (void)src; + (void)size; + return false; +#endif +} + // Bind the calling thread to the given device context. inline void SetDevice(int device_id) { if (device_id < 0) return; diff --git a/mooncake-store/src/transfer_task.cpp b/mooncake-store/src/transfer_task.cpp index 8a2989af..f892aa0e 100644 --- a/mooncake-store/src/transfer_task.cpp +++ b/mooncake-store/src/transfer_task.cpp @@ -6,6 +6,7 @@ #include #include #include +#include "gpu_staging_utils.h" #include "transfer_engine.h" #include "transport/transport.h" @@ -193,13 +194,35 @@ void MemcpyWorkerPool::workerThread() { // Execute the task if we have one if (task.state) { try { + bool ok = true; for (const auto& op : task.operations) { - std::memcpy(op.dest, op.src, op.size); + int src_dev = -1, dst_dev = -1; + bool src_on_gpu = + gpu_staging::IsDevicePointer(op.src, &src_dev); + bool dst_on_gpu = + gpu_staging::IsDevicePointer(op.dest, &dst_dev); + + if (!src_on_gpu && !dst_on_gpu) { + std::memcpy(op.dest, op.src, op.size); + } else { + int dev = src_on_gpu ? src_dev : dst_dev; + gpu_staging::SetDevice(dev); + if (!gpu_staging::CopyAuto(op.dest, op.src, op.size)) { + LOG(ERROR) + << "GPU memcpy failed: src_dev=" << src_dev + << " dst_dev=" << dst_dev + << " size=" << op.size; + ok = false; + break; + } + } } - VLOG(2) << "Memcpy task completed successfully with " - << task.operations.size() << " operations"; - task.state->set_completed(ErrorCode::OK); + VLOG(2) << "Memcpy task completed with " + << task.operations.size() << " operations" + << (ok ? "" : " (with GPU copy failure)"); + task.state->set_completed(ok ? ErrorCode::OK + : ErrorCode::TRANSFER_FAIL); } catch (const std::exception& e) { LOG(ERROR) << "Exception during async memcpy: " << e.what(); task.state->set_completed(ErrorCode::TRANSFER_FAIL); @@ -777,48 +800,16 @@ TransferStrategy TransferSubmitter::selectStrategy( return TransferStrategy::TRANSFER_ENGINE; } -namespace { -// Helper function to extract IP address from endpoint string (ip:port format) -// Supports both IPv4 (ip:port) and IPv6 ([ipv6]:port) formats -std::string extractIpAddress(const std::string& endpoint) { - if (endpoint.empty()) { - return ""; - } - - // Handle IPv6 format: [ipv6]:port - if (endpoint[0] == '[') { - size_t closing_bracket = endpoint.find(']'); - if (closing_bracket == std::string::npos) { - LOG(WARNING) << "Invalid IPv6 endpoint format: " << endpoint; - // Return empty to disable local memcpy optimization - return ""; - } - return endpoint.substr(1, closing_bracket - 1); // Extract IPv6 address - } - - // Handle IPv4 or hostname:port format - // Find the last colon (to handle IPv6 addresses without brackets) - size_t colon_pos = endpoint.rfind(':'); - if (colon_pos != std::string::npos) { - return endpoint.substr(0, colon_pos); - } - - // No colon found, return the whole string (might be just IP or hostname) - return endpoint; -} -} // namespace - bool TransferSubmitter::isLocalTransfer( const AllocatedBuffer::Descriptor& handle) const { std::string local_ep = engine_.getLocalIpAndPort(); - std::string local_ip = extractIpAddress(local_ep); if (!local_ep.empty()) { - std::string handle_ip = extractIpAddress(handle.transport_endpoint_); - return !handle.transport_endpoint_.empty() && handle_ip == local_ip; + return !handle.transport_endpoint_.empty() && + handle.transport_endpoint_ == local_ep; } - // Without a local IP we cannot prove locality; disable memcpy. + // Without a local endpoint we cannot prove locality; disable memcpy. return false; } -- 2.34.1 From 5ef889dd11ad2999839e15fb09b03accc8e41102 Mon Sep 17 00:00:00 2001 From: Stary Date: Thu, 30 Apr 2026 16:49:40 +0800 Subject: [PATCH 025/382] [TENT] Recover cooled-down RDMA rails and add failover e2e tests (#1984) * feat(tent): improve RailMonitor recovery with exponential backoff and configurable parameters - Introduce exponential backoff cooldown with a configurable upper bound (kMaxCooldown) - Support dynamic configuration of error threshold, error window, and cooldown via Config - Call markRecovered on successful transfer completion to un-pause rails promptly - Reset error count and cooldown on recovery to prevent accumulated doubling - Cache target_machine_id in RdmaSlice to avoid segment lookup on hot path - Add unit tests for recovery behavior, cooldown reset, and best-device mapping Signed-off-by: staryxchen * test(engine): add end-to-end failover tests and test-only transport swap hook - Add`swapTransportForTest`method to`TransferEngineImpl` for test-only transport injection. - Introduce`engine_failover_e2e_test.cpp` with scenarios: status corruption failover, budget exhaustion, mixed faults, per-task independence, and boundary conditions for `max_failover_attempts`. - Register new test target`tent_engine_failover_e2e_test`in CMakeLists. Signed-off-by: staryxchen * docs(tent): add failover design doc Describe TENT's two-layer failure handling: cross-transport failover in TransferEngineImpl and per-rail cooldown recovery in RailMonitor. Covers fault model, state machines (with code refs), config knobs, observability, testing, and known gaps (submit-stage failover, cooldown reset on recovery, no CI coverage). Signed-off-by: staryxchen * refactor(tent): cache RailMonitor pointer on slice to avoid hot-path string lookup Each RdmaSlice carried the target machine_id as std::string and asyncPollCq did a hash+strcmp lookup on worker.rails per completion. Resolve the RailMonitor once during generatePostPath and stash the pointer on the slice; the completion path becomes a single deref. WorkerContext::rails now stores values via unique_ptr so a rehash only moves the pointer slot and does not invalidate pointers already held by in-flight slices. disableEndpoint no longer needs a segment lookup to mark the rail failed either -- it uses the cached pointer. Addresses gemini-code-assist review on PR #1984. Signed-off-by: staryxchen --------- Signed-off-by: staryxchen --- docs/source/design/tent/failover.md | 194 ++++++ docs/source/design/tent/overview.md | 8 + .../tent/runtime/transfer_engine_impl.h | 12 + .../tent/transport/rdma/rail_monitor.h | 31 +- .../tent/include/tent/transport/rdma/slice.h | 8 + .../include/tent/transport/rdma/workers.h | 6 +- .../tent/src/transport/rdma/rail_monitor.cpp | 53 +- .../tent/src/transport/rdma/workers.cpp | 52 +- .../tent/tests/CMakeLists.txt | 20 + .../tent/tests/engine_failover_e2e_test.cpp | 653 ++++++++++++++++++ .../tent/tests/rail_monitor_test.cpp | 194 ++++++ 11 files changed, 1196 insertions(+), 35 deletions(-) create mode 100644 docs/source/design/tent/failover.md create mode 100644 mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp create mode 100644 mooncake-transfer-engine/tent/tests/rail_monitor_test.cpp diff --git a/docs/source/design/tent/failover.md b/docs/source/design/tent/failover.md new file mode 100644 index 00000000..4a8c9b23 --- /dev/null +++ b/docs/source/design/tent/failover.md @@ -0,0 +1,194 @@ +# TENT Failover + +TENT hides transfer failures from the application by recovering inside the data path. +This document describes how the recovery works, which knobs control it, and how it is tested. + +The design has two layers: + +1. **Cross-transport failover** in `TransferEngineImpl`. When a transport fails a task at the completion stage, the engine moves that task to the next available transport (for example RDMA → TCP). Submit-stage failures are not retried today; see Known Gaps. +2. **Intra-RDMA rail recovery** in `RailMonitor`. When a specific (local NIC, remote NIC) rail keeps failing, the monitor pauses it with exponential cooldown; a successful transfer or the cooldown expiry brings it back. + +Application code submits a batch and polls `getTransferStatus`. It never sees a `FAILED` task as long as any healthy path remains and the failover budget is not exhausted. + +## Fault Model + +TENT focuses on three kinds of transient faults: + +| Fault | Surface | Recovery action | +|-------|---------|-----------------| +| Work request completion error (WC error) | RDMA worker sees a bad completion | Rail-level `markFailed` + task-level resubmit | +| QP / endpoint failure | `submitTransferTasks` returns non-OK | *Not retried today*: task surfaces as `FAILED`. See Known Gaps. | +| Peer disconnect mid-transfer | `getTransferStatus` returns `FAILED` | Cross-transport failover | + +Permanent or application-visible errors (invalid arguments, out-of-memory, segment not found) are *not* retried; they are returned to the caller as-is. + +## Architecture + +``` + +-------------------------+ + submitTransfer | TransferEngineImpl | + ---------------> classify by TransportType + | submitTransferTasks |----failure----+ + +-------------------------+ | + | v + | resubmitTransferTask + | (bump priority, pick next + | transport, resubmit) + v + +---------------------------+ + | RdmaTransport / workers | + | +---------------------+ | + | | RailMonitor | | + | | per-rail state | | + | | cooldown / recover | | + | +---------------------+ | + +---------------------------+ +``` + +* Each request is owned by one `TaskInfo`. `type` names the transport currently executing the task; `xport_priority` is the index into the ranked fallback list; `failover_count` caps how many times we may re-resolve the transport. +* The ranked fallback list comes from `getTransportType(req, priority)`. Priority 0 yields the best available transport; increasing priority walks down the list; `UNSPEC` means no transport left. +* RDMA rail state lives in `RailMonitor`. Its lifecycle is independent of the task-level state machine: a rail can be paused while tasks keep flowing on other rails. + +## State Machine + +### Cross-transport failover + +`resubmitTransferTask` is the single entry point that promotes a failing task to the next transport: + +``` +++task.failover_count +if failover_count > max_failover_attempts -> return error (exhausted) + +task.xport_priority++ +type = resolveTransport(task.request, task.xport_priority) +if type == UNSPEC -> return error (no transport) + +transport_list_[type]->submitTransferTasks(...) +``` + +It has two callers, one per recoverable failure surface: + +1. **Completion-stage failure.** `getTransferStatus(batch_id, task_id, status)` and the batch-form overload call `resubmitTransferTask` once per `FAILED` completion. On success the task is re-marked `PENDING` so the aggregated batch status does not latch to `FAILED` because of a task that is actually retrying. + +2. **Exhaustion.** When the budget is hit, `resubmitTransferTask` sets the returned status to `InvalidEntry("Failover limit exceeded, all transports exhausted")`. Callers leave `task.type` unchanged; the task then reports `FAILED` through the normal status flow. + +Submit-stage failures (`submitTransferTasks` returning non-OK) are **not** retried today. They mark the task as `UNSPEC`, and `getTransferStatus` short-circuits to `FAILED`. See Known Gaps for why. + +### RDMA rail recovery + +Inside `RdmaTransport`, each completion drives the rail monitor: + +* Bad completion → `rail.markFailed(local_nic, remote_nic)` +* Good completion → `rail.markRecovered(local_nic, remote_nic)` + +`markFailed` bumps `error_count` inside `error_window_`. Once the count hits `error_threshold_` the rail is paused until `now + cooldown_`; the cooldown doubles on every repeat failure up to `kMaxCooldown` (300 s). + +`markRecovered` clears the error count, un-pauses the rail, and resets the exponential-backoff memory so the next failure cycle starts from the initial cooldown. A fast path returns without work when the rail is already healthy, which is the common case on the completion hot path. + +`available(local, remote)` is the gate every work request passes through before posting. If the cooldown has expired, `available` itself resets all backoff state (error count, resume time, cooldown) and logs `Rail recovered: ... (cooldown expired)`. Otherwise it returns false and the scheduler picks another rail via `findBestRemoteDevice`. + +This produces two independent recovery signals — cooldown expiry and live success — so a flaky rail does not stall forever if no other rail is posted to, and a recovered rail returns to service at the first good completion instead of waiting for the full cooldown. + +## Configuration + +All knobs live in the top-level `transfer-engine.json`. Defaults are safe for production; tune only if you have evidence. + +| Key | Default | Meaning | +|-----|---------|---------| +| `max_failover_attempts` | `3` | Upper bound on `resubmitTransferTask` calls per task. `0` disables cross-transport failover entirely. `1` allows exactly one switch. | +| `transports/rdma/rail_error_threshold` | `3` | Number of failures inside `rail_error_window_secs` that trips a rail into the paused state. | +| `transports/rdma/rail_error_window_secs` | `10` | Sliding window for counting rail errors. A failure older than the window resets `error_count` to 1. | +| `transports/rdma/rail_cooldown_secs` | `30` | Initial cooldown after tripping. Doubles on each repeat failure, capped at 300 s. | + +The RDMA keys are read by `RailMonitor::load`. Example: + +```json +{ + "max_failover_attempts": 3, + "transports": { + "rdma": { + "rail_error_threshold": 3, + "rail_error_window_secs": 10, + "rail_cooldown_secs": 30 + } + } +} +``` + +## Observability + +### Metric + +`tent_transport_failover_total` is a counter incremented once per successful transport switch inside `resubmitTransferTask`. A non-zero rate means the engine is actively recovering; a sudden jump usually points at a single bad link or flaky peer. + +The counter is only built when TENT is compiled with `-DTENT_METRICS_ENABLED=ON` (see `metrics.md`). Without that flag the macro is a no-op. + +### Log keywords + +| Keyword | Interpretation | +|---------|----------------| +| `Transport failover: X -> Y (attempt N/M)` | A task has successfully switched transports. | +| `Task failover limit reached (M), last transport=X` | Task exhausted its budget and will surface `FAILED`. | +| `No more transports available after X failed` | `resolveTransport` returned `UNSPEC`; no further fallback exists for this request. | +| `Rail recovered: local_nic=... remote_nic=... (cooldown expired)` | Cooldown elapsed and the rail is back in service. | +| `Rail recovered: ... (un-paused by successful transfer)` | Live success on a previously paused rail brought it back early. | + +## Testing + +Real hardware faults are hard to stage, so TENT tests the failover machinery with decorator-style fault injection. + +### FaultProxyTransport + +`FaultProxyTransport` wraps any `Transport` and injects four policy-driven faults: + +* `submit_fail_rate` — probability that `submitTransferTasks` returns an error. +* `status_corrupt_rate` — probability that `getTransferStatus` flips `COMPLETED → FAILED`. +* `fail_after_n_submits` — deterministic variant: succeed the first N submits, then always fail. +* `fail_install` — make `install()` fail, simulating a transport that cannot come up. + +Because it implements the `Transport` interface, the engine sees an ordinary transport. All failover paths (`submitTransfer`, `getTransferStatus`, `resubmitTransferTask`) run unmodified. + +### Test-only injection hook + +`TransferEngineImpl::swapTransportForTest` replaces the transport in one slot after `construct()`. This is the only way the end-to-end test can wrap the real transport with `FaultProxyTransport` without bypassing `resolveTransport` or `resubmitTransferTask`. Production code never calls it. + +### End-to-end suite + +The end-to-end failover test suite drives the real `TransferEngineImpl` with fake transports (`FakeTransport`) wrapped in `FaultProxyTransport`. It uses a `p2p` metadata backend on `127.0.0.1` so no external services are required — the whole suite is self-contained. + +Current cases: + +| Test | What it exercises | +|------|-------------------| +| `StatusCorruptionTriggersFailoverToSecondary` | Primary reports `FAILED` in `getTransferStatus`; engine must resubmit on the secondary. | +| `BothTransportsFailExhaustsFailoverBudget` | Both transports fail at the completion stage; task must surface `FAILED` once the budget is drained. | +| `MixedFaultsAcrossManySubmissions` | 10 one-request batches with 30% completion corruption on RDMA; every task must end `COMPLETED`, and submit-counter math must hold. | +| `MaxFailoverAttemptsZeroDisablesFailover` | `max_failover_attempts = 0` → the first completion fault is permanent, TCP is never touched. | +| `MaxFailoverAttemptsOneAllowsSingleFailover` | `max_failover_attempts = 1` → one switch allowed; RDMA fault → TCP success. | +| `PerTaskFailoverCountsAreIndependent` | A failing task must not consume another task's budget; `failover_count` is strictly per-task. | + +A test-local `PerRequestFaultProxy` (in the same file) subclasses `FaultProxyTransport` to take a `std::function` predicate, remembers which sub-task ids it marked as "poisoned" at submit time, and flips only those completions from `COMPLETED` to `FAILED` at status-query time. + +### Running manually + +The TENT tests are **not** in CI today (the upstream workflow builds with `USE_TENT=OFF`). Run them locally: + +```bash +cmake -S . -B build-tent -DUSE_TENT=ON -DUSE_CUDA=OFF +cmake --build build-tent --target tent_engine_failover_e2e_test -j +./build-tent/mooncake-transfer-engine/tent/tests/tent_engine_failover_e2e_test +``` + +Setting `USE_CUDA=OFF` forces `CpuPlatform`, which always reports `MTYPE_CPU`. With `USE_CUDA=ON` on a host without a GPU, `cudaPointerGetAttributes` fails, `getMemoryType` returns `MTYPE_UNKNOWN`, every transport reports unavailable, and `resolveTransport` returns `UNSPEC` before the fault injection ever runs. + +Companion unit tests cover the rail monitor and related building blocks: `tent_rail_monitor_test`, `tent_failover_test`, `tent_fault_proxy_test`. + +## Known Gaps + +* **Submit-stage failures do not trigger failover.** When `submitTransferTasks` returns non-OK, every task in that call is marked `UNSPEC` and surfaces as `FAILED`. A naive retry loop here is unsafe for two reasons: + 1. **Merged requests.** When `merge_requests` is enabled (default), `task_id_list[type]` contains both the real merged task and its derived aliases. Resubmitting per task-id re-posts one logical transfer multiple times on the fallback transport, breaking the deduplication the merge pass established. + 2. **Partial enqueue.** Some transports (for example `ShmTransport::submitTransferTasks`, `NVLinkTransport::submitTransferTasks`) enqueue or start work for earlier requests in `request_list` before returning an error on a later one. The return status alone does not tell us which tasks partially succeeded, so a blanket resubmit would duplicate already-started transfers. + A safe submit-stage recovery needs either (a) a transport-level "atomic submit" capability flag plus per-task skip of derived ids, or (b) per-request status returned from `submitTransferTasks`. Neither exists today. +* `markRecovered` (and cooldown expiry in `available`) clears the exponential-backoff memory entirely. A rail that flaps repeatedly therefore does not accumulate a growing cooldown across recovery cycles. If this becomes a problem the fix is to decay rather than reset. +* Cross-transport failover is driven purely by return status; there is no latency-based "this transport is healthy but too slow, try another" signal. That belongs to the scheduler, not this document. +* TENT tests are not exercised by CI. A follow-up can add a CI job that builds with `-DUSE_TENT=ON -DUSE_CUDA=OFF` and runs the `tent_*` test targets; none of the code in this document changes in that case. diff --git a/docs/source/design/tent/overview.md b/docs/source/design/tent/overview.md index b56e69b7..4c988cad 100644 --- a/docs/source/design/tent/overview.md +++ b/docs/source/design/tent/overview.md @@ -87,4 +87,12 @@ cpp-api :maxdepth: 1 metrics +::: + +## TENT Failover + +:::{toctree} +:maxdepth: 1 + +failover ::: \ No newline at end of file diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h b/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h index 71874af2..3d9ccf90 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h @@ -158,6 +158,18 @@ class TransferEngineImpl { Status unlockStageBuffer(uint64_t addr); + // Test-only hook: replace the transport in a given slot after construct(). + // Production code never calls this. Used by failover integration tests to + // inject a FaultProxyTransport without bypassing resubmitTransferTask, + // resolveTransport, or any other engine state. Not thread-safe with any + // in-flight transfer on that slot. + void swapTransportForTest(TransportType type, + std::shared_ptr xport) { + if (type >= 0 && type < (TransportType)kSupportedTransportTypes) { + transport_list_[type] = std::move(xport); + } + } + private: Status construct(); diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/rail_monitor.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/rail_monitor.h index a0dd7cc7..f69e453c 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/rail_monitor.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/rail_monitor.h @@ -21,9 +21,27 @@ namespace mooncake { namespace tent { +// Not thread-safe. Each RDMA worker owns its own RailMonitor via +// WorkerContext::rails, so markFailed / markRecovered / available / load +// all execute on a single worker thread. Do not share across threads +// without adding external synchronization. class RailMonitor { const static size_t kMaxNuma = 16; + // Upper bound on the exponential-backoff cooldown; prevents the + // per-rail pause from growing without bound under repeated failure. + static constexpr std::chrono::seconds kMaxCooldown{300}; + + public: + // Config keys. Exposed as constants so callers (and docs) reference + // a single source of truth instead of duplicating the string path. + static constexpr const char *kCfgErrorThreshold = + "transports/rdma/rail_error_threshold"; + static constexpr const char *kCfgErrorWindowSecs = + "transports/rdma/rail_error_window_secs"; + static constexpr const char *kCfgCooldownSecs = + "transports/rdma/rail_cooldown_secs"; + public: RailMonitor() = default; @@ -33,8 +51,13 @@ class RailMonitor { RailMonitor &operator=(const RailMonitor &) = delete; public: + // rail_topo_json: optional JSON string describing rail topology. + // conf: optional Config pointer; when non-null, overrides the default + // error_threshold / error_window_secs / cooldown_secs values via + // kCfgErrorThreshold / kCfgErrorWindowSecs / kCfgCooldownSecs. Status load(const Topology *local, const Topology *remote, - const std::string &rail_topo_json = ""); + const std::string &rail_topo_json = "", + const Config *conf = nullptr); bool ready() { return ready_; } @@ -71,8 +94,12 @@ class RailMonitor { int error_count = 0; std::chrono::seconds cooldown{0}; std::chrono::steady_clock::time_point last_error{}; - bool paused = false; std::chrono::steady_clock::time_point resume_time{}; + + // Derived: a rail is paused iff a resume_time has been armed. + bool paused() const { + return resume_time != std::chrono::steady_clock::time_point{}; + } }; std::unordered_map, RailState, PairHash> rail_states_; diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h index c4bfe1fb..9da64daf 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h @@ -33,6 +33,7 @@ namespace mooncake { namespace tent { struct RdmaSlice; +class RailMonitor; struct RdmaSliceList { RdmaSlice* first = nullptr; @@ -71,6 +72,13 @@ struct RdmaSlice { bool failed = false; uint64_t enqueue_ts = 0; uint64_t submit_ts = 0; + // Non-owning pointer to the per-worker RailMonitor for this slice's + // target machine, resolved once in generatePostPath. Lets asyncPollCq + // and disableEndpoint call markRecovered / markFailed without a + // string-keyed map lookup on the RDMA hot path. Stable because + // WorkerContext::rails stores values via unique_ptr, so rehashes do + // not invalidate the pointee. + RailMonitor* rail_monitor = nullptr; }; using RdmaSliceStorage = Slab; diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/workers.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/workers.h index 6688ab7d..a123c9de 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/workers.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/workers.h @@ -16,6 +16,7 @@ #define TENT_WORKERS_H #include +#include #include #include #include @@ -186,7 +187,10 @@ class Workers { std::condition_variable cv; volatile bool in_suspend = false; - std::unordered_map rails; + // Values are held via unique_ptr so that map rehashing does not + // invalidate pointers into RailMonitor stored on in-flight slices + // (see RdmaSlice::rail_monitor). + std::unordered_map> rails; PerfMetricSummary perf; uint64_t padding[16]; }; diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rail_monitor.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rail_monitor.cpp index 5bbb10a1..be9142f9 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rail_monitor.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rail_monitor.cpp @@ -18,9 +18,20 @@ namespace mooncake { namespace tent { Status RailMonitor::load(const Topology *local, const Topology *remote, - const std::string &rail_topo_json) { + const std::string &rail_topo_json, + const Config *conf) { local_ = local; remote_ = remote; + if (conf) { + error_threshold_ = conf->get(kCfgErrorThreshold, error_threshold_); + error_window_ = std::chrono::seconds( + conf->get(kCfgErrorWindowSecs, (int)error_window_.count())); + cooldown_ = std::chrono::seconds( + conf->get(kCfgCooldownSecs, (int)cooldown_.count())); + LOG(INFO) << "RailMonitor: error_threshold=" << error_threshold_ + << " error_window=" << error_window_.count() << "s" + << " cooldown=" << cooldown_.count() << "s"; + } if (!rail_topo_json.empty()) { auto status = loadFromJson(rail_topo_json); if (status.ok()) return status; @@ -33,16 +44,16 @@ bool RailMonitor::available(int local_nic, int remote_nic) { auto it = rail_states_.find(std::make_pair(local_nic, remote_nic)); if (it == rail_states_.end()) return false; auto &st = it->second; - if (st.paused) { - auto now = std::chrono::steady_clock::now(); - if (now >= st.resume_time) { - st.paused = false; - st.error_count = 0; - updateBestMapping(); - return true; - } - return false; - } + if (!st.paused()) return true; + if (std::chrono::steady_clock::now() < st.resume_time) return false; + // Cooldown expired: clear all exponential-backoff memory so a fresh + // failure cycle starts from the initial cooldown_, not a doubled value. + st.resume_time = {}; + st.error_count = 0; + st.cooldown = std::chrono::seconds(0); + updateBestMapping(); + LOG(INFO) << "Rail recovered: local_nic=" << local_nic + << " remote_nic=" << remote_nic << " (cooldown expired)"; return true; } @@ -61,11 +72,9 @@ void RailMonitor::markFailed(int local_nic, int remote_nic) { st.cooldown = cooldown_; } else { st.cooldown *= 2; - if (st.cooldown > std::chrono::seconds(300)) - st.cooldown = std::chrono::seconds(300); + if (st.cooldown > kMaxCooldown) st.cooldown = kMaxCooldown; } if (st.error_count >= error_threshold_) { - st.paused = true; st.resume_time = now + st.cooldown; updateBestMapping(); } @@ -75,10 +84,22 @@ void RailMonitor::markRecovered(int local_nic, int remote_nic) { auto it = rail_states_.find(std::make_pair(local_nic, remote_nic)); if (it == rail_states_.end()) return; auto &st = it->second; + // Fast path: a healthy rail stays healthy. 99%+ of completions land + // here, so we must not touch best_mapping_ or write any field. + if (!st.paused() && st.error_count == 0 && st.cooldown.count() == 0) return; + bool was_paused = st.paused(); + // Clear all exponential-backoff memory: the next failure cycle must + // start from the initial cooldown_, not a doubled value left over + // from the previous cycle. st.error_count = 0; - st.paused = false; st.resume_time = {}; - updateBestMapping(); + st.cooldown = std::chrono::seconds(0); + if (was_paused) { + LOG(INFO) << "Rail recovered: local_nic=" << local_nic + << " remote_nic=" << remote_nic + << " (un-paused by successful transfer)"; + updateBestMapping(); + } } int RailMonitor::findBestRemoteDevice(int local_nic, int remote_numa) { diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp index 9191b41b..ef4132cc 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp @@ -19,6 +19,7 @@ #include #include "tent/transport/rdma/endpoint_store.h" +#include "tent/transport/rdma/rail_monitor.h" #include "tent/common/utils/ip.h" #include "tent/common/utils/string_builder.h" #include "tent/common/utils/os.h" @@ -27,6 +28,21 @@ namespace mooncake { namespace tent { thread_local int tl_wid = -1; + +namespace { +// Look up (or create) the RailMonitor for `machine_id` on this worker's +// map. Returning a stable reference is safe because the map stores values +// via unique_ptr -- rehashes move the pointer slot, not the RailMonitor. +RailMonitor& getOrCreateRail( + std::unordered_map>& rails, + const std::string& machine_id) { + auto it = rails.find(machine_id); + if (it != rails.end()) return *it->second; + auto [ins, _] = rails.emplace(machine_id, std::make_unique()); + return *ins->second; +} +} // namespace + Workers::Workers(RdmaTransport* transport) : transport_(transport), num_workers_(0), running_(false) { device_quota_ = std::make_unique(); @@ -188,19 +204,8 @@ std::shared_ptr Workers::getEndpoint(Workers::PostPath path) { } void Workers::disableEndpoint(RdmaSlice* slice) { - SegmentDesc* desc = nullptr; - auto& segment_manager = transport_->metadata_->segmentManager(); - auto target_id = slice->task->request.target_id; - if (target_id == LOCAL_SEGMENT_ID) { - desc = segment_manager.getLocal().get(); - } else { - auto status = segment_manager.getRemoteCached(desc, target_id); - if (!status.ok()) return; - } - if (desc) { - auto& worker = worker_context_[tl_wid]; - auto& rail = worker.rails[desc->machine_id]; - rail.markFailed(slice->source_dev_id, slice->target_dev_id); + if (auto* rail = slice->rail_monitor) { + rail->markFailed(slice->source_dev_id, slice->target_dev_id); } if (auto ep = slice->ep_weak_ptr.lock()) { ep->acknowledge(slice, FAILED); @@ -361,6 +366,14 @@ void Workers::asyncPollCq() { } } else { num_slices += ep->acknowledge(slice, COMPLETED); + // A successful transfer proves this rail is healthy; clear + // any accumulated error count so a previously-cooled-down + // rail can be used again without waiting for the full + // cooldown to expire. The monitor pointer is resolved once + // in generatePostPath, so no map lookup is needed here. + if (auto* rail = slice->rail_monitor; rail && rail->ready()) + rail->markRecovered(slice->source_dev_id, + slice->target_dev_id); worker.perf.inflight_lat.add(inflight_lat); worker.perf.enqueue_lat.add(enqueue_lat); } @@ -532,9 +545,10 @@ Status Workers::selectOptimalDevice(RouteHint& source, RouteHint& target, return Status::DeviceNotFound( "No device could access the slice memory region" LOC_MARK); - auto& rail = worker.rails[target.segment->machine_id]; + auto& rail = getOrCreateRail(worker.rails, target.segment->machine_id); if (!rail.ready() || target.topo != rail.remote()) - rail.load(source.topo, target.topo); + rail.load(source.topo, target.topo, /*rail_topo_json=*/"", + transport_->conf_.get()); if (slice->target_dev_id < 0) { int mapped_dev_id = rail.findBestRemoteDevice( slice->source_dev_id, target.topo_entry->numa_node); @@ -630,7 +644,8 @@ Status Workers::selectFallbackDevice(RouteHint& source, RouteHint& target, reachable = (sdev == tdev); // loopback is safe } else { auto& worker = worker_context_[tl_wid]; - auto& rail = worker.rails[target.segment->machine_id]; + auto& rail = + getOrCreateRail(worker.rails, target.segment->machine_id); reachable = rail.available(sdev, tdev); } @@ -663,6 +678,11 @@ Status Workers::generatePostPath(RdmaSlice* slice) { CHECK_STATUS(selectFallbackDevice(source, target, slice)); slice->source_lkey = source.buffer->lkey[slice->source_dev_id]; slice->target_rkey = target.buffer->rkey[slice->target_dev_id]; + // Cache the RailMonitor pointer so asyncPollCq / disableEndpoint can + // update rail state without a segment lookup or string-keyed map + // lookup on the hot path. + slice->rail_monitor = &getOrCreateRail(worker_context_[tl_wid].rails, + target.segment->machine_id); return Status::OK(); } } // namespace tent diff --git a/mooncake-transfer-engine/tent/tests/CMakeLists.txt b/mooncake-transfer-engine/tent/tests/CMakeLists.txt index 6777a84b..41c379b1 100644 --- a/mooncake-transfer-engine/tent/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/tests/CMakeLists.txt @@ -77,3 +77,23 @@ target_link_libraries(tent_fault_proxy_test PRIVATE gtest gtest_main target_include_directories(tent_fault_proxy_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) add_test(NAME tent_fault_proxy_test COMMAND tent_fault_proxy_test) + +add_executable(tent_rail_monitor_test rail_monitor_test.cpp) +target_link_libraries(tent_rail_monitor_test PRIVATE gtest gtest_main + tent_link_group) +target_include_directories(tent_rail_monitor_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) +add_test(NAME tent_rail_monitor_test COMMAND tent_rail_monitor_test) + +# End-to-end failover test: drives real TransferEngineImpl with +# FaultProxyTransport-wrapped fakes to exercise resubmitTransferTask. +add_executable(tent_engine_failover_e2e_test engine_failover_e2e_test.cpp) +target_link_libraries(tent_engine_failover_e2e_test + PRIVATE gtest gtest_main tent_link_group) +if(TARGET asio_shared) + target_link_libraries(tent_engine_failover_e2e_test PRIVATE asio_shared) +endif() +target_include_directories(tent_engine_failover_e2e_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) +add_test(NAME tent_engine_failover_e2e_test + COMMAND tent_engine_failover_e2e_test) diff --git a/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp b/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp new file mode 100644 index 00000000..d7c5ef17 --- /dev/null +++ b/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp @@ -0,0 +1,653 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +// End-to-end failover test: drives the real TransferEngineImpl through +// submitTransfer() / getTransferStatus() / resubmitTransferTask() by swapping +// two FakeTransports into the engine's transport_list_ and wrapping the +// primary one in a FaultProxyTransport. +// +// --------------------------------------------------------------------------- +// Scenarios: +// P0: Primary succeeds at submit but getTransferStatus reports FAILED +// (simulates WC error / QP error / peer drop mid-transfer). +// -> engine must failover to the secondary and succeed. +// +// Submit-stage failures are NOT tested here. Today submitTransferTasks +// failures are marked UNSPEC and surface as FAILED without a failover +// attempt -- see the "Known gaps" section of docs/source/design/tent/ +// failover.md for why. Tests for that path belong in a future change +// that makes submit-stage recovery safe (see known gaps). + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "tent/common/config.h" +#include "tent/common/types.h" +#include "tent/runtime/segment.h" +#include "tent/runtime/transfer_engine_impl.h" +#include "tent/runtime/transport.h" +#include "tent/transport/fault_proxy/fault_proxy_transport.h" + +namespace mooncake { +namespace tent { +namespace { + +// --------------------------------------------------------------------------- +// FakeTransport: declares itself capable of dram_to_dram, records itself in +// BufferDesc::transports under a configurable slot, and always completes +// transfers. Enough to satisfy checkAvailability() + resolveTransport() in +// TransferEngineImpl. +// --------------------------------------------------------------------------- + +class FakeSubBatch : public Transport::SubBatch { + public: + size_t size() const override { return task_count; } + size_t task_count = 0; + std::vector statuses; +}; + +class FakeTransport : public Transport { + public: + explicit FakeTransport(TransportType self_type) : self_type_(self_type) { + caps.dram_to_dram = true; // so checkAvailability returns true + } + + std::atomic install_calls{0}; + std::atomic submit_calls{0}; + std::atomic status_calls{0}; + std::atomic add_mem_calls{0}; + + Status install(std::string& /*local_segment_name*/, + std::shared_ptr /*metadata*/, + std::shared_ptr /*local_topology*/, + std::shared_ptr /*conf*/ = nullptr) override { + ++install_calls; + return Status::OK(); + } + + Status allocateSubBatch(SubBatchRef& batch, size_t /*max_size*/) override { + batch = new FakeSubBatch(); + return Status::OK(); + } + + Status freeSubBatch(SubBatchRef& batch) override { + delete batch; + batch = nullptr; + return Status::OK(); + } + + Status submitTransferTasks( + SubBatchRef batch, const std::vector& request_list) override { + ++submit_calls; + auto* fb = static_cast(batch); + for (const auto& req : request_list) { + fb->statuses.push_back({TransferStatusEnum::COMPLETED, req.length}); + fb->task_count++; + } + return Status::OK(); + } + + Status getTransferStatus(SubBatchRef batch, int task_id, + TransferStatus& status) override { + ++status_calls; + auto* fb = static_cast(batch); + if (task_id < 0 || task_id >= (int)fb->statuses.size()) { + return Status::InvalidArgument("bad task_id" LOC_MARK); + } + status = fb->statuses[task_id]; + return Status::OK(); + } + + // Tag ourselves into the BufferDesc so resolveTransport() considers us. + Status addMemoryBuffer(BufferDesc& desc, + const MemoryOptions& /*options*/) override { + ++add_mem_calls; + desc.transports.push_back(self_type_); + return Status::OK(); + } + + Status addMemoryBuffer(std::vector& desc_list, + const MemoryOptions& options) override { + for (auto& d : desc_list) { + auto s = addMemoryBuffer(d, options); + if (!s.ok()) return s; + } + return Status::OK(); + } + + Status removeMemoryBuffer(BufferDesc& /*desc*/) override { + return Status::OK(); + } + + Status allocateLocalMemory(void** addr, size_t size, + MemoryOptions& /*options*/) override { + *addr = std::malloc(size); + if (!*addr) return Status::InternalError("malloc failed" LOC_MARK); + return Status::OK(); + } + + Status freeLocalMemory(void* addr, size_t /*size*/) override { + std::free(addr); + return Status::OK(); + } + + bool warmupMemory(void* /*addr*/, size_t /*length*/) override { + return false; // no pinning; engine will fall back to its own path + } + + const char* getName() const override { + return self_type_ == RDMA ? "" : ""; + } + + private: + TransportType self_type_; +}; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +std::shared_ptr makeMinimalP2PConfig() { + auto cfg = std::make_shared(); + // p2p metadata avoids needing an external redis/etcd/http server. + cfg->set("metadata_type", "p2p"); + cfg->set("metadata_servers", ""); + cfg->set("rpc_server_hostname", "127.0.0.1"); + cfg->set("rpc_server_port", "0"); + cfg->set("log_level", "warning"); + cfg->set("merge_requests", false); + + // Disable every real transport. We'll inject fakes into the slots we care + // about via swapTransportForTest. + cfg->set("transports/tcp/enable", false); + cfg->set("transports/shm/enable", false); + cfg->set("transports/rdma/enable", false); + cfg->set("transports/io_uring/enable", false); + cfg->set("transports/nvlink/enable", false); + cfg->set("transports/mnnvl/enable", false); + cfg->set("transports/gds/enable", false); + cfg->set("transports/ascend_direct/enable", false); + + // Keep failover limit at default (3) but make it explicit. + cfg->set("max_failover_attempts", 3); + return cfg; +} + +// Wait for a task to leave PENDING. Bounded so tests fail fast. +TransferStatus pollUntilDone( + TransferEngineImpl& engine, BatchID batch_id, size_t task_id, + std::chrono::milliseconds timeout = std::chrono::milliseconds(2000)) { + TransferStatus ts{}; + const auto deadline = std::chrono::steady_clock::now() + timeout; + while (std::chrono::steady_clock::now() < deadline) { + ts = {}; + auto s = engine.getTransferStatus(batch_id, task_id, ts); + if (!s.ok()) { + ADD_FAILURE() << "getTransferStatus returned error: " + << s.ToString(); + return ts; + } + if (ts.s == TransferStatusEnum::COMPLETED || + ts.s == TransferStatusEnum::FAILED) { + return ts; + } + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + } + return ts; +} + +// --------------------------------------------------------------------------- +// P0: Completion reports FAILED (simulates WC error / QP error / peer drop +// mid-transfer). Engine must failover. +// --------------------------------------------------------------------------- + +TEST(EngineFailoverE2E, StatusCorruptionTriggersFailoverToSecondary) { + auto cfg = makeMinimalP2PConfig(); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + + FaultPolicy rdma_policy; + rdma_policy.status_corrupt_rate = 1.0; // every COMPLETED flipped to FAILED + auto proxied_rdma = + std::make_shared(fake_rdma, rdma_policy); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg_name, nullptr, nullptr).ok()); + + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0xCD); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(8); + ASSERT_NE(batch_id, (BatchID)0); + + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + auto final_status = pollUntilDone(engine, batch_id, 0); + EXPECT_EQ(final_status.s, TransferStatusEnum::COMPLETED); + + // RDMA saw exactly one submit (succeeded on the wire), then its status + // was corrupted; engine failed over. + EXPECT_EQ(fake_rdma->submit_calls.load(), 1); + EXPECT_GE(fake_tcp->submit_calls.load(), 1); + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +// --------------------------------------------------------------------------- +// P1b: Both transports keep failing at status stage -> failover limit reached. +// --------------------------------------------------------------------------- + +TEST(EngineFailoverE2E, BothTransportsFailExhaustsFailoverBudget) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("max_failover_attempts", 2); // tighten budget + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + + FaultPolicy always_corrupt; + always_corrupt.status_corrupt_rate = 1.0; + + auto proxied_rdma = + std::make_shared(fake_rdma, always_corrupt); + auto proxied_tcp = + std::make_shared(fake_tcp, always_corrupt); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(proxied_tcp->install(seg_name, nullptr, nullptr).ok()); + + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, proxied_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0xEF); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(8); + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + auto final_status = pollUntilDone(engine, batch_id, 0); + EXPECT_EQ(final_status.s, TransferStatusEnum::FAILED) + << "after exhausting failover budget, task must be permanently FAILED"; + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +// --------------------------------------------------------------------------- +// PerRequestFaultProxy +// +// FaultProxyTransport corrupts completions uniformly based on a rate. For +// tests that need per-request control (e.g. "fail task0 but not task1"), +// this subclass inspects each Request at submit time, records the sub_task +// indices of "poisoned" requests, and flips only those from COMPLETED to +// FAILED in getTransferStatus(). Completions for non-poisoned requests +// pass through unchanged. Submit is never rejected. +// +// Tests submit each request in its own one-request batch, so the engine's +// failover logic routes each task individually. +// --------------------------------------------------------------------------- + +class PerRequestFaultProxy : public FaultProxyTransport { + public: + using Predicate = std::function; + + PerRequestFaultProxy(std::shared_ptr real, Predicate pred) + : FaultProxyTransport(std::move(real), FaultPolicy{}), + should_fail_(std::move(pred)) {} + + Status submitTransferTasks( + SubBatchRef batch, const std::vector& request_list) override { + // Remember (sub_batch, sub_task_id) pairs for "poisoned" requests + // so getTransferStatus() can flip only those to FAILED later. The + // sub_batch pointer must be part of the key: different engine + // batches (BatchID) have different SubBatchRefs per transport, so + // sub_task_id=0 in batch A is a different task than sub_task_id=0 + // in batch B. Keying only on sub_task_id would cross-contaminate. + const int base = static_cast(batch->size()); + for (size_t i = 0; i < request_list.size(); ++i) { + if (should_fail_(request_list[i])) { + poisoned_.insert({batch, base + static_cast(i)}); + } + } + return FaultProxyTransport::submitTransferTasks(batch, request_list); + } + + Status getTransferStatus(SubBatchRef batch, int task_id, + TransferStatus& status) override { + auto s = FaultProxyTransport::getTransferStatus(batch, task_id, status); + if (!s.ok()) return s; + if (poisoned_.count({batch, task_id}) && + status.s == TransferStatusEnum::COMPLETED) { + status.s = TransferStatusEnum::FAILED; + } + return s; + } + + private: + Predicate should_fail_; + std::set> poisoned_; +}; + +// --------------------------------------------------------------------------- +// A: Mixed faults across many independent single-request submissions. +// +// Submit 10 one-request batches. For each submission, RDMA's completion +// reports FAILED with 30% probability. Assert every task ultimately +// COMPLETES and that RDMA/TCP submit counts add up consistently: every +// failed RDMA completion must be followed by a TCP success. +// --------------------------------------------------------------------------- + +TEST(EngineFailoverE2E, MixedFaultsAcrossManySubmissions) { + auto cfg = makeMinimalP2PConfig(); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + + FaultPolicy rdma_policy; + rdma_policy.status_corrupt_rate = 0.3; // 30% of completions flip to FAILED + auto proxied_rdma = + std::make_shared(fake_rdma, rdma_policy); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg_name, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + constexpr int kNumTasks = 10; + std::vector buf(kBufLen * kNumTasks, 0x77); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), buf.size()).ok()); + + std::vector batches; + batches.reserve(kNumTasks); + for (int i = 0; i < kNumTasks; ++i) { + BatchID b = engine.allocateBatch(1); + ASSERT_NE(b, (BatchID)0); + + Request req; + req.opcode = Request::WRITE; + req.source = buf.data() + (size_t)i * kBufLen; + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(req.source); + req.length = kBufLen; + + ASSERT_TRUE(engine.submitTransfer(b, {req}).ok()); + batches.push_back(b); + } + + int completed = 0; + for (int i = 0; i < kNumTasks; ++i) { + auto ts = pollUntilDone(engine, batches[i], 0); + EXPECT_EQ(ts.s, TransferStatusEnum::COMPLETED) + << "task " << i << " did not complete"; + if (ts.s == TransferStatusEnum::COMPLETED) ++completed; + } + EXPECT_EQ(completed, kNumTasks); + + // Sanity on submit counts under status-corruption injection: + // - Every task hits RDMA at submit time (submit itself succeeds, + // the proxy only corrupts getTransferStatus), so rdma_ok == kNumTasks. + // - A corrupted completion triggers a TCP failover (+1 tcp). + // A clean completion stays on RDMA (+0 tcp). + // - Therefore tcp_ok equals the number of corrupted completions, + // which is in [0, kNumTasks]. + const int rdma_ok = fake_rdma->submit_calls.load(); + const int tcp_ok = fake_tcp->submit_calls.load(); + EXPECT_EQ(rdma_ok, kNumTasks) + << "every task must attempt RDMA first (submit is always accepted)"; + EXPECT_GE(tcp_ok, 0); + EXPECT_LE(tcp_ok, kNumTasks); + + // With 30% corruption rate over 10 tasks, both branches are exercised + // with overwhelming probability (0.7^10 ~= 2.8% no failover; 0.3^10 ~= + // 6e-6 all failover). We don't assert strict counts because this is + // rate-based. + + for (auto b : batches) EXPECT_TRUE(engine.freeBatch(b).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), buf.size()).ok()); +} + +// --------------------------------------------------------------------------- +// C1: max_failover_attempts = 0 disables failover entirely. +// +// resubmitTransferTask bumps failover_count and compares it against the +// budget before anything else. With budget=0, the very first attempted +// failover is rejected (++count == 1 > 0), so a single RDMA fault must +// result in a permanently FAILED task without touching TCP. +// --------------------------------------------------------------------------- + +TEST(EngineFailoverE2E, MaxFailoverAttemptsZeroDisablesFailover) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("max_failover_attempts", 0); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + + FaultPolicy always_fail; + always_fail.status_corrupt_rate = 1.0; // every completion flips to FAILED + auto proxied_rdma = + std::make_shared(fake_rdma, always_fail); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg_name, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0x11); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(1); + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + auto ts = pollUntilDone(engine, batch_id, 0); + EXPECT_EQ(ts.s, TransferStatusEnum::FAILED) + << "with budget=0 the first fault must be permanent"; + + // TCP must never have been touched: budget=0 means no failover attempt. + EXPECT_EQ(fake_tcp->submit_calls.load(), 0); + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +// C2: max_failover_attempts = 1 allows exactly one failover. +// +// RDMA submit fails -> engine spends its single budget to switch to TCP +// -> TCP succeeds -> task COMPLETES. A symmetric run where *both* fake +// transports always fail is covered by BothTransportsFailExhaustsFailoverBudget +// at budget=2; here we just confirm the happy-path boundary. + +TEST(EngineFailoverE2E, MaxFailoverAttemptsOneAllowsSingleFailover) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("max_failover_attempts", 1); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + + FaultPolicy rdma_fail; + rdma_fail.status_corrupt_rate = 1.0; // every completion flips to FAILED + auto proxied_rdma = + std::make_shared(fake_rdma, rdma_fail); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg_name, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0x22); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(1); + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + auto ts = pollUntilDone(engine, batch_id, 0); + EXPECT_EQ(ts.s, TransferStatusEnum::COMPLETED) + << "budget=1 must permit exactly one failover to TCP"; + EXPECT_EQ(fake_tcp->submit_calls.load(), 1); + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +// --------------------------------------------------------------------------- +// D: Tasks in the same logical workload maintain independent failover +// state. +// +// Workload is two submissions: +// - task0: RDMA *and* TCP always fail -> must end FAILED +// - task1: RDMA succeeds -> must end COMPLETED, no TCP hit +// +// This pins down that one task's exhausted failover budget does not +// "infect" another task: the engine must track failover_count per-task, +// not per-batch, per-transport, or per-engine. A regression that made +// the counter global or batch-scoped would flip task1 to FAILED or +// trigger a spurious TCP submit. +// +// Uses PerRequestFaultProxy so we can make RDMA fail *only* for the +// specific buffer address of task0. +// --------------------------------------------------------------------------- + +TEST(EngineFailoverE2E, PerTaskFailoverCountsAreIndependent) { + auto cfg = makeMinimalP2PConfig(); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + + constexpr size_t kBufLen = 4096; + std::vector buf0(kBufLen, 0xAA); + std::vector buf1(kBufLen, 0xBB); + + const uint64_t failing_addr = reinterpret_cast(buf0.data()); + + auto proxied_rdma = std::make_shared( + fake_rdma, [failing_addr](const Request& r) { + return r.target_offset == failing_addr; + }); + auto proxied_tcp = std::make_shared( + fake_tcp, [failing_addr](const Request& r) { + return r.target_offset == failing_addr; + }); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(proxied_tcp->install(seg_name, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, proxied_tcp); + + ASSERT_TRUE(engine.registerLocalMemory(buf0.data(), kBufLen).ok()); + ASSERT_TRUE(engine.registerLocalMemory(buf1.data(), kBufLen).ok()); + + auto submit = [&](uint8_t* source) -> BatchID { + BatchID b = engine.allocateBatch(1); + EXPECT_NE(b, (BatchID)0); + Request req; + req.opcode = Request::WRITE; + req.source = source; + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(source); + req.length = kBufLen; + EXPECT_TRUE(engine.submitTransfer(b, {req}).ok()); + return b; + }; + + BatchID b0 = submit(buf0.data()); // must FAIL on both + BatchID b1 = submit(buf1.data()); // must SUCCEED on RDMA + + auto ts0 = pollUntilDone(engine, b0, 0); + auto ts1 = pollUntilDone(engine, b1, 0); + + EXPECT_EQ(ts0.s, TransferStatusEnum::FAILED) + << "task0 should exhaust all transports and end FAILED"; + EXPECT_EQ(ts1.s, TransferStatusEnum::COMPLETED) + << "task1 must be unaffected by task0's failover exhaustion"; + + // Under status-corruption injection submits always succeed; the proxy + // corrupts only getTransferStatus. So both tasks hit RDMA at submit + // (rdma +2). task0's RDMA completion is corrupted -> failover to TCP + // -> TCP submit (+1), also corrupted -> no more transports -> FAILED. + // task1 completes cleanly on RDMA with no TCP touch. + EXPECT_EQ(fake_rdma->submit_calls.load(), 2); + EXPECT_EQ(fake_tcp->submit_calls.load(), 1); + + EXPECT_TRUE(engine.freeBatch(b0).ok()); + EXPECT_TRUE(engine.freeBatch(b1).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf0.data(), kBufLen).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf1.data(), kBufLen).ok()); +} + +} // namespace +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/tests/rail_monitor_test.cpp b/mooncake-transfer-engine/tent/tests/rail_monitor_test.cpp new file mode 100644 index 00000000..b625f39b --- /dev/null +++ b/mooncake-transfer-engine/tent/tests/rail_monitor_test.cpp @@ -0,0 +1,194 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include + +#include +#include + +#include "tent/common/config.h" +#include "tent/transport/rdma/rail_monitor.h" +#include "tent/runtime/topology.h" + +namespace mooncake { +namespace tent { +namespace { + +// --------------------------------------------------------------------------- +// Helper: build a minimal single-NIC Topology via Topology::parse(). +// +// JSON keys expected by parse(): "nics" / "mems" +// NicType enum: NIC_RDMA=0, NIC_TCP=1, NIC_UNKNOWN=2 +// MemType enum: MEM_HOST=0, MEM_CUDA=1, ... +// device_list keys: "rank0", "rank1", ... +// --------------------------------------------------------------------------- + +static std::shared_ptr makeSingleNicTopology(const std::string& nic, + int numa_node = 0) { + // One RDMA NIC (type=0) and one CUDA mem region (type=1) referencing it. + auto json_str = R"({ + "nics": [{"name": ")" + + nic + R"(", "type": 0, "numa_node": )" + + std::to_string(numa_node) + R"(}], + "mems": [{ + "name": "cuda0", + "type": 1, + "numa_node": )" + + std::to_string(numa_node) + + R"(, + "device_list": {"rank0": [0]} + }] + })"; + auto topo = std::make_shared(); + auto status = topo->parse(json_str); + if (!status.ok()) { + ADD_FAILURE() << "Topology::parse failed: " << status.ToString(); + } + return topo; +} + +// --------------------------------------------------------------------------- +// markRecovered resets error_count so failures start accumulating fresh +// --------------------------------------------------------------------------- + +TEST(RailMonitorRecoverTest, RecoverResetsErrorCount) { + auto local = makeSingleNicTopology("mlx5_0"); + auto remote = makeSingleNicTopology("mlx5_1"); + RailMonitor rail; + ASSERT_TRUE(rail.load(local.get(), remote.get()).ok()); + ASSERT_TRUE(rail.ready()); + + // Initially available + EXPECT_TRUE(rail.available(0, 0)); + + // One failure — not yet past default threshold (3) + rail.markFailed(0, 0); + EXPECT_TRUE(rail.available(0, 0)); // error_count=1, not paused + + // A successful transfer — reset error_count back to 0 + rail.markRecovered(0, 0); + EXPECT_TRUE(rail.available(0, 0)); + + // Failure again — counter starts fresh from 0, one hit is not enough + rail.markFailed(0, 0); + EXPECT_TRUE(rail.available(0, 0)); +} + +// --------------------------------------------------------------------------- +// markRecovered un-pauses a rail that reached the failure threshold +// --------------------------------------------------------------------------- + +TEST(RailMonitorRecoverTest, RecoverUnpausesPausedRail) { + auto local = makeSingleNicTopology("mlx5_0"); + auto remote = makeSingleNicTopology("mlx5_1"); + RailMonitor rail; + ASSERT_TRUE(rail.load(local.get(), remote.get()).ok()); + + // Drive error_count to the default threshold (3) to trigger pause + for (int i = 0; i < 3; ++i) rail.markFailed(0, 0); + EXPECT_FALSE(rail.available(0, 0)) + << "Rail should be paused after 3 failures"; + + // A successful transfer proves the path is live — should un-pause + // immediately + rail.markRecovered(0, 0); + EXPECT_TRUE(rail.available(0, 0)) + << "Rail should be available after recovery"; +} + +// --------------------------------------------------------------------------- +// markRecovered on an unknown NIC pair is a no-op (no crash / no assert) +// --------------------------------------------------------------------------- + +TEST(RailMonitorRecoverTest, RecoverUnknownPairIsNoop) { + auto local = makeSingleNicTopology("mlx5_0"); + auto remote = makeSingleNicTopology("mlx5_1"); + RailMonitor rail; + ASSERT_TRUE(rail.load(local.get(), remote.get()).ok()); + + // NIC IDs 5 and 7 are not in the topology — must not crash + EXPECT_NO_FATAL_FAILURE(rail.markRecovered(5, 7)); +} + +// --------------------------------------------------------------------------- +// After recovery, findBestRemoteDevice maps back to the (only) available rail +// --------------------------------------------------------------------------- + +TEST(RailMonitorRecoverTest, FindBestAfterRecovery) { + auto local = makeSingleNicTopology("mlx5_0"); + auto remote = makeSingleNicTopology("mlx5_1"); + RailMonitor rail; + ASSERT_TRUE(rail.load(local.get(), remote.get()).ok()); + + // Pause the only available rail + for (int i = 0; i < 3; ++i) rail.markFailed(0, 0); + EXPECT_FALSE(rail.available(0, 0)); + + // Recovery must rebuild best_mapping so findBestRemoteDevice works again + rail.markRecovered(0, 0); + EXPECT_TRUE(rail.available(0, 0)); + int best = rail.findBestRemoteDevice(/*local_nic=*/0, /*remote_numa=*/0); + EXPECT_EQ(best, 0) << "Recovered rail should be the best remote device"; +} + +// --------------------------------------------------------------------------- +// After recovery, the cooldown on the next pause must start from the +// configured initial value, not a doubled value left over from the +// previous cycle. +// +// Uses error_threshold=1 and cooldown=1s so each single failure triggers +// a pause. If cooldown is correctly reset on recovery, the second pause +// expires in ~1s; if the cooldown had carried over (bug), the second +// pause would expire in ~2s. +// --------------------------------------------------------------------------- + +TEST(RailMonitorRecoverTest, CooldownDoesNotCarryOverAfterRecovery) { + auto local = makeSingleNicTopology("mlx5_0"); + auto remote = makeSingleNicTopology("mlx5_1"); + + Config cfg; + cfg.set(RailMonitor::kCfgErrorThreshold, 1); // pause on first failure + cfg.set(RailMonitor::kCfgErrorWindowSecs, 60); // wide: no window resets + cfg.set(RailMonitor::kCfgCooldownSecs, 1); // small initial cooldown + + RailMonitor rail; + ASSERT_TRUE(rail.load(local.get(), remote.get(), "", &cfg).ok()); + + // First pause cycle: single failure arms resume_time at now+1s. + rail.markFailed(0, 0); + EXPECT_FALSE(rail.available(0, 0)); + + // Recover: must clear st.cooldown so the next pause uses 1s again, + // not the 1s left over from cycle 1 (which would double to 2s). + rail.markRecovered(0, 0); + EXPECT_TRUE(rail.available(0, 0)); + + // Second pause cycle: single failure must arm resume_time at now+1s. + rail.markFailed(0, 0); + EXPECT_FALSE(rail.available(0, 0)); + + // Wait 1.5s: longer than the initial 1s cooldown, shorter than the + // 2s value the bug would produce. If cooldown was correctly reset on + // recovery, available() returns true; if it carried over, available() + // stays false until ~2s elapses. + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + EXPECT_TRUE(rail.available(0, 0)) + << "After recovery, the next pause must use the initial cooldown " + "(1s); staying paused past 1.5s indicates cooldown carried over " + "from the previous cycle."; +} + +} // namespace +} // namespace tent +} // namespace mooncake -- 2.34.1 From 043351f0484e1e6fb583cbd5bcd082df1c91e1fd Mon Sep 17 00:00:00 2001 From: Nishikant <164586341+Nishikant-K-P@users.noreply.github.com> Date: Thu, 30 Apr 2026 02:18:57 -0700 Subject: [PATCH 026/382] [TENT] Fix RPC server IPv6 binding on IPv6-only hosts (#2014) The coro_rpc_server constructor was called with only thread count and port, defaulting to "0.0.0.0" (IPv4). The ipv6 parameter passed to CoroRpcAgent::start() was ignored entirely. On IPv6-only hosts, the server listens on IPv4 while the client connects via IPv6, resulting in "not connected" / "bad_address" errors during P2P handshake. Pass "::" (IPv6 any) or "0.0.0.0" (IPv4 any) based on the ipv6 parameter so the server binds to the correct address family. Co-authored-by: nishikant --- mooncake-transfer-engine/tent/src/rpc/rpc.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mooncake-transfer-engine/tent/src/rpc/rpc.cpp b/mooncake-transfer-engine/tent/src/rpc/rpc.cpp index 40fe2077..9091543c 100644 --- a/mooncake-transfer-engine/tent/src/rpc/rpc.cpp +++ b/mooncake-transfer-engine/tent/src/rpc/rpc.cpp @@ -104,7 +104,8 @@ Status CoroRpcAgent::start(uint16_t& port, bool ipv6) { try { if (port == 0) port = kStartPort + SimpleRandom::Get().next(kPortRange); - server_ = new coro_rpc::coro_rpc_server(kRpcThreads, port); + server_ = new coro_rpc::coro_rpc_server(kRpcThreads, port, + ipv6 ? "::" : "0.0.0.0"); server_->register_handler<&CoroRpcAgent::process>(this); server_->async_start(); const auto err = server_->get_errc(); -- 2.34.1 From 3d7ce4c47eee2f3292f90ee62485ee6e400f146c Mon Sep 17 00:00:00 2001 From: Nishikant <164586341+Nishikant-K-P@users.noreply.github.com> Date: Thu, 30 Apr 2026 02:27:38 -0700 Subject: [PATCH 027/382] [TE] Fix DMA-BUF validation using wrong CUDA device index (#2015) When WITH_NVIDIA_PEERMEM=OFF, openRdmaDevice() validated DMA-BUF support by calling cuDeviceGet(&cuDevice, i) where i is the verbs enumeration index from ibv_get_device_list(). This assumed verbs device order matches CUDA device order, which is incorrect. On a GB300 with 10 NICs and 4 GPUs, NICs at verbs index 4-9 called cuDeviceGet with indices 4-9, which don't exist. This disabled 6 of 10 NICs with "Failed to query CUDA device", limiting RDMA throughput to 4 NICs (18 GB/s) instead of all 10 (41 GB/s). Fix: - Look up the topology matrix to find which CUDA devices list this RNIC in their preferred_hca or avail_hca, then validate DMA-BUF only for those specific devices. - Check avail_hca in addition to preferred_hca because runtime NIC selection can fall back to avail_hca after disableDevice() removes a failed preferred NIC. Without this, the fallback NIC may not have been validated for the GPU that ends up using it. - Add explicit cuInit(0) before cuDeviceGet. The old code relied on implicit CUDA driver initialization from a prior cudaGetDeviceCount call in topology discovery. Making it explicit removes a fragile ordering dependency. Validation on GB300 (single-host VRAM read, GPU 1 -> GPU 0, all 10 NICs auto-discovered): Topology discovery complete. Found 10 HCAs. RDMA device: mlx5_0 ... mlx5_9 [all 10 register, GIDs elided] Pre-fix only the 4 NICs at verbs index 0-3 would have passed validation; mlx5_4-9 would have been disabled with "Failed to query CUDA device". With the fix, the bench reaches 41 GB/s saturation (see follow-up topology commit) instead of being capped at ~18 GB/s. Co-authored-by: nishikant --- .../transport/rdma_transport/rdma_context.cpp | 94 +++++++++++++++---- 1 file changed, 76 insertions(+), 18 deletions(-) diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index eb6a194b..b9be9e91 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -14,11 +14,13 @@ #include "transport/rdma_transport/rdma_context.h" +#include #include #include #include #include +#include #include #include #include @@ -592,25 +594,81 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, } #if !defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA) - // Verify dmabuf support which is required if not using nvidia-peermem. - // Assume device index matches. - CUdevice cuDevice; - CUresult result = cuDeviceGet(&cuDevice, i); - if (result != CUDA_SUCCESS) { - LOG(ERROR) << "Failed to query CUDA device"; - return ERR_CONTEXT; + // Verify DMA-BUF support against the CUDA device(s) that the local + // topology explicitly maps to this RNIC, rather than assuming the + // verbs enumeration order matches CUDA enumeration. + // Validate DMA-BUF support for every GPU that can reach this RNIC, + // not just GPUs listing it as preferred. Runtime selection falls + // back to avail_hca when a preferred NIC is disabled, so we must + // validate both lists. + std::vector mapped_cuda_devices; + if (engine_.local_topology_) { + const auto topology_matrix = engine_.local_topology_->getMatrix(); + for (const auto &entry : topology_matrix) { + if (entry.first.rfind(GPU_PREFIX, 0) != 0) continue; + bool in_preferred = + std::find(entry.second.preferred_hca.begin(), + entry.second.preferred_hca.end(), + device_name) != entry.second.preferred_hca.end(); + bool in_avail = + std::find(entry.second.avail_hca.begin(), + entry.second.avail_hca.end(), + device_name) != entry.second.avail_hca.end(); + if (!in_preferred && !in_avail) continue; + + try { + mapped_cuda_devices.push_back( + std::stoi(entry.first.substr(GPU_PREFIX.size()))); + } catch (const std::exception &e) { + LOG(WARNING) << "Ignore malformed topology GPU entry " + << entry.first << ": " << e.what(); + } + } } - int dmaBufSupported; - result = cuDeviceGetAttribute( - &dmaBufSupported, CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED, cuDevice); - if (result != CUDA_SUCCESS) { - LOG(ERROR) << "Failed to query CUDA device attributes"; - return ERR_CONTEXT; - } - if (!dmaBufSupported) { - LOG(ERROR) << "DMA BUF supported required for GPU RDMA without " - "nvidia-peermem"; - return ERR_CONTEXT; + + std::sort(mapped_cuda_devices.begin(), mapped_cuda_devices.end()); + mapped_cuda_devices.erase( + std::unique(mapped_cuda_devices.begin(), mapped_cuda_devices.end()), + mapped_cuda_devices.end()); + + if (mapped_cuda_devices.empty()) { + LOG(INFO) << "No CUDA device is explicitly mapped to RNIC " + << device_name << "; skip DMA-BUF affinity validation"; + } else { + // cuInit is process-global and idempotent; call it once before + // the per-device loop, not per cuDeviceGet. + CUresult result = cuInit(0); + if (result != CUDA_SUCCESS) { + LOG(ERROR) << "Failed to initialize CUDA driver for RNIC " + << device_name; + goto cleanup_context_and_devices; + } + for (int cuda_device : mapped_cuda_devices) { + CUdevice cuDevice; + result = cuDeviceGet(&cuDevice, cuda_device); + if (result != CUDA_SUCCESS) { + LOG(ERROR) << "Failed to query CUDA device " << cuda_device + << " for RNIC " << device_name; + goto cleanup_context_and_devices; + } + int dmaBufSupported; + result = cuDeviceGetAttribute( + &dmaBufSupported, CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED, + cuDevice); + if (result != CUDA_SUCCESS) { + LOG(ERROR) << "Failed to query CUDA device attributes for " + << "CUDA device " << cuda_device << " and RNIC " + << device_name; + goto cleanup_context_and_devices; + } + if (!dmaBufSupported) { + LOG(ERROR) + << "DMA BUF supported required for GPU RDMA without " + "nvidia-peermem on CUDA device " + << cuda_device << " mapped to RNIC " << device_name; + goto cleanup_context_and_devices; + } + } } #endif -- 2.34.1 From f2853a80a494b999d2115449dfd153dd25f5b42d Mon Sep 17 00:00:00 2001 From: ZhangCheng <481193684@qq.com> Date: Thu, 30 Apr 2026 17:28:13 +0800 Subject: [PATCH 028/382] [TE] To avoid inconvenience during debugging, remove redundant checks. (#2020) Co-authored-by: zhangcheng --- .../src/transport/ascend_transport/ascend_allocator.cpp | 8 -------- 1 file changed, 8 deletions(-) diff --git a/mooncake-transfer-engine/src/transport/ascend_transport/ascend_allocator.cpp b/mooncake-transfer-engine/src/transport/ascend_transport/ascend_allocator.cpp index 9e3599bb..64eceec5 100644 --- a/mooncake-transfer-engine/src/transport/ascend_transport/ascend_allocator.cpp +++ b/mooncake-transfer-engine/src/transport/ascend_transport/ascend_allocator.cpp @@ -66,10 +66,6 @@ int allocate_physical_memory(size_t total_size, aclrtDrvMemHandle &handle) { // Direct ACL VMM allocation (always bypasses adxl MallocMem). // Used by shm_helper when ascend_agent_mode && ascend_use_fabric_mem. void *allocate_vmm_memory_direct_impl(size_t total_size) { - if (total_size % kFabricMemPageSize != 0) { - LOG(ERROR) << "VMM memory size must be a multiple of 1GB"; - return nullptr; - } aclrtDrvMemHandle handle = nullptr; if (allocate_physical_memory(total_size, handle) != 0) { return nullptr; @@ -137,10 +133,6 @@ void *ascend_allocate_memory(size_t total_size, const std::string &protocol) { if (globalConfig().ascend_use_fabric_mem) { void *va = nullptr; #ifdef ASCEND_SUPPORT_FABRIC_MEM - if (total_size % kFabricMemPageSize != 0) { - LOG(ERROR) << "Local buffer size must be a multiple of 1GB"; - return nullptr; - } if (&adxl::AdxlEngine::MallocMem != nullptr) { // Try to use adxl_engine's MallocMem auto status = adxl::AdxlEngine::MallocMem(adxl::MemType::MEM_HOST, -- 2.34.1 From 090e938c28c8784b6d327a957cab432fb1c7807c Mon Sep 17 00:00:00 2001 From: Dayuxiaoshui <158081477+Dayuxiaoshui@users.noreply.github.com> Date: Sun, 3 May 2026 00:36:19 +0800 Subject: [PATCH 029/382] [Store][Rust] Add batch_put_from, batch_get_into, batch_is_exist wrappers (#1993) --- mooncake-store/rust/build.rs | 78 ++++++++++++- mooncake-store/rust/src/error.rs | 4 + mooncake-store/rust/src/store.rs | 186 +++++++++++++++++++++++++++++++ 3 files changed, 266 insertions(+), 2 deletions(-) diff --git a/mooncake-store/rust/build.rs b/mooncake-store/rust/build.rs index 6f817b9c..b38da413 100644 --- a/mooncake-store/rust/build.rs +++ b/mooncake-store/rust/build.rs @@ -50,8 +50,10 @@ fn emit_link_searches(search_dirs: &[PathBuf]) { fn emit_runtime_rpaths(search_dirs: &[PathBuf]) { for dir in search_dirs { let dir_str = dir.display(); - println!("cargo:rustc-link-arg-tests=-Wl,-rpath,{dir_str}"); - println!("cargo:rustc-link-arg-examples=-Wl,-rpath,{dir_str}"); + // Use rustc-link-arg (not -tests) so that the rpath is also applied + // to the lib-test binary produced by `cargo test` for #[cfg(test)] + // modules inside src/lib.rs. + println!("cargo:rustc-link-arg=-Wl,-rpath,{dir_str}"); } } @@ -113,6 +115,76 @@ fn add_compiler_runtime_search_dir(search_dirs: &mut Vec, file_name: &s } fn main() { + // ----------------------------------------------------------------------- + // Library search path + // + // When built via CMake (WITH_STORE_RUST=ON) the CMakeLists.txt injects + // MOONCAKE_STORE_LIB_DIR pointing at the directory that contains + // libmooncake_store.a/.so. When cargo is invoked standalone the caller + // should set the variable manually or rely on the default convention of a + // sibling `build/` directory produced by a top-level CMake configure. + // ----------------------------------------------------------------------- + let lib_dir = env::var("MOONCAKE_STORE_LIB_DIR") + .unwrap_or_else(|_| "../../build/mooncake-store/src".to_string()); + + println!("cargo:rustc-link-search=native={lib_dir}"); + + // mooncake_store depends on libasio.so (shared) built in mooncake-common. + let lib_path = PathBuf::from(&lib_dir); + let build_dir = lib_path.ancestors().nth(2).map(PathBuf::from).unwrap_or_else(|| { + println!("cargo:warning=MOONCAKE_STORE_LIB_DIR='{lib_dir}' does not have enough parent directories; using current directory"); + PathBuf::from(".") + }); + println!( + "cargo:rustc-link-search=native={}", + build_dir.join("mooncake-common").display() + ); + + // transfer_engine is built in a sibling directory. + println!( + "cargo:rustc-link-search=native={}", + build_dir.join("mooncake-transfer-engine/src").display() + ); + + // common/base library (contains mooncake::Status etc.) + println!( + "cargo:rustc-link-search=native={}", + build_dir.join("mooncake-transfer-engine/src/common/base").display() + ); + + // CUDA runtime libraries (needed by transfer_engine RDMA transport). + let cuda_home = env::var("CUDA_HOME") + .or_else(|_| env::var("CUDA_PATH")) + .unwrap_or_else(|_| "/usr/local/cuda".to_string()); + println!("cargo:rustc-link-search=native={}/targets/x86_64-linux/lib", cuda_home); + + // cachelib_memory_allocator is a static library built alongside mooncake_store. + println!( + "cargo:rustc-link-search=native={}", + build_dir.join("mooncake-store/src/cachelib_memory_allocator").display() + ); + + println!("cargo:rustc-link-lib=mooncake_store"); + + // Dependencies of mooncake_store that must be satisfied at link time. + // The list mirrors what mooncake-store/src/CMakeLists.txt links against. + println!("cargo:rustc-link-lib=transfer_engine"); + println!("cargo:rustc-link-lib=base"); // mooncake::Status etc. + println!("cargo:rustc-link-lib=asio"); // shared library built by mooncake-common + println!("cargo:rustc-link-lib=jsoncpp"); // transfer_engine dependency + println!("cargo:rustc-link-lib=cachelib_memory_allocator"); // static + println!("cargo:rustc-link-lib=stdc++"); + println!("cargo:rustc-link-lib=glog"); + println!("cargo:rustc-link-lib=gflags"); + println!("cargo:rustc-link-lib=numa"); // NUMA binding + println!("cargo:rustc-link-lib=curl"); // HTTP metadata plugin + println!("cargo:rustc-link-lib=ibverbs"); // RDMA transport + println!("cargo:rustc-link-lib=pthread"); + println!("cargo:rustc-link-lib=xxhash"); + + // ----------------------------------------------------------------------- + // Header path for bindgen + // ----------------------------------------------------------------------- let manifest_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR").expect("missing CARGO_MANIFEST_DIR")); let mut search_dirs = Vec::new(); @@ -142,6 +214,7 @@ fn main() { default_build_dir.join("mooncake-transfer-engine/src"), default_build_dir.join("mooncake-transfer-engine/src/common/base"), default_build_dir.join("mooncake-asio"), + default_build_dir.join("mooncake-common"), default_build_dir.join("mooncake-common/etcd"), PathBuf::from("/usr/local/lib"), PathBuf::from("/usr/lib/x86_64-linux-gnu"), @@ -195,6 +268,7 @@ fn main() { ("etcd_wrapper", &["etcd_wrapper"] as &[&str]), ("hiredis", &["hiredis"]), ("curl", &["curl"]), + ("cuda", &["cuda"]), ("cudart", &["cudart"]), ("uring", &["uring"]), ] { diff --git a/mooncake-store/rust/src/error.rs b/mooncake-store/rust/src/error.rs index 8f04ef04..a854a3aa 100644 --- a/mooncake-store/rust/src/error.rs +++ b/mooncake-store/rust/src/error.rs @@ -40,4 +40,8 @@ pub enum StoreError { /// [`OperationFailed`](StoreError::OperationFailed) instead. #[error("key not found")] NotFound, + + /// One or more arguments are invalid (e.g. mismatched array lengths). + #[error("invalid argument: {0}")] + InvalidArgument(String), } diff --git a/mooncake-store/rust/src/store.rs b/mooncake-store/rust/src/store.rs index 87888b39..39a47404 100644 --- a/mooncake-store/rust/src/store.rs +++ b/mooncake-store/rust/src/store.rs @@ -418,6 +418,153 @@ impl MooncakeStore { Ok(()) } + // ----------------------------------------------------------------------- + // Batch operations + // ----------------------------------------------------------------------- + + /// Batch version of [`put_from`](MooncakeStore::put_from). + /// + /// Returns a `Vec` where each element is the result code for the + /// corresponding key (0 = success, non-zero = error). + /// + /// # Safety + /// + /// Same safety requirements as `put_from`: each buffer must be registered. + pub unsafe fn batch_put_from( + &self, + keys: &[&str], + buffers: &[*mut c_void], + sizes: &[usize], + config: Option<&ReplicateConfig>, + ) -> Result, StoreError> { + let count = keys.len(); + if count == 0 { + return Ok(Vec::new()); + } + if buffers.len() != count || sizes.len() != count { + return Err(StoreError::InvalidArgument( + "keys, buffers, and sizes must have the same length".to_string(), + )); + } + + let key_strings: Vec = keys + .iter() + .map(|k| CString::new(*k)) + .collect::>()?; + let key_ptrs: Vec<*const libc::c_char> = + key_strings.iter().map(|s| s.as_ptr()).collect(); + + let (_c_config, _strings, _ptrs) = Self::prepare_config(config)?; + let cfg_ptr = _c_config + .as_ref() + .map_or(std::ptr::null(), |c| c as *const _); + + let mut results = vec![0i32; count]; + + let rc = ffi::mooncake_store_batch_put_from( + self.handle, + key_ptrs.as_ptr() as *mut *const libc::c_char, + buffers.as_ptr() as *mut *mut c_void, + sizes.as_ptr(), + count, + cfg_ptr, + results.as_mut_ptr(), + ); + + if rc != 0 { + return Err(StoreError::OperationFailed(rc)); + } + + Ok(results) + } + + /// Batch version of [`get_into`](MooncakeStore::get_into). + /// + /// Returns a `Vec` where each element is the number of bytes written + /// (≥ 0) or an error code (< 0). + /// + /// # Safety + /// + /// Same safety requirements as `get_into`: each buffer must be writable. + pub unsafe fn batch_get_into( + &self, + keys: &[&str], + buffers: &[*mut c_void], + sizes: &[usize], + ) -> Result, StoreError> { + let count = keys.len(); + if count == 0 { + return Ok(Vec::new()); + } + if buffers.len() != count || sizes.len() != count { + return Err(StoreError::InvalidArgument( + "keys, buffers, and sizes must have the same length".to_string(), + )); + } + + let key_strings: Vec = keys + .iter() + .map(|k| CString::new(*k)) + .collect::>()?; + let key_ptrs: Vec<*const libc::c_char> = + key_strings.iter().map(|s| s.as_ptr()).collect(); + + let mut results = vec![0i64; count]; + + let rc = ffi::mooncake_store_batch_get_into( + self.handle, + key_ptrs.as_ptr() as *mut *const libc::c_char, + buffers.as_ptr() as *mut *mut c_void, + sizes.as_ptr(), + count, + results.as_mut_ptr(), + ); + + if rc != 0 { + return Err(StoreError::OperationFailed(rc)); + } + + Ok(results) + } + + /// Batch check existence of multiple keys. + pub fn batch_is_exist( + &self, + keys: &[&str], + ) -> Result, StoreError> { + let count = keys.len(); + let key_strings: Vec = keys + .iter() + .map(|k| CString::new(*k)) + .collect::>()?; + let key_ptrs: Vec<*const libc::c_char> = + key_strings.iter().map(|s| s.as_ptr()).collect(); + + let mut results = vec![0i32; count]; + + let rc = unsafe { + ffi::mooncake_store_batch_is_exist( + self.handle, + key_ptrs.as_ptr() as *mut *const libc::c_char, + count, + results.as_mut_ptr(), + ) + }; + + if rc != 0 { + return Err(StoreError::OperationFailed(rc)); + } + + results + .into_iter() + .map(|r| match r { + 1 => Ok(true), + 0 => Ok(false), + _ => Err(StoreError::OperationFailed(r)), + }) + .collect() + } + // ----------------------------------------------------------------------- // Internal helpers // ----------------------------------------------------------------------- @@ -548,4 +695,43 @@ mod tests { assert_eq!(strings.len(), 2); assert_eq!(ptrs.len(), 2); } + + // ----------------------------------------------------------------------- + // Batch API tests + // ----------------------------------------------------------------------- + + #[test] + fn batch_is_exist_empty() { + let store = MooncakeStore::new().expect("new should succeed"); + let results = store.batch_is_exist(&[]).expect("empty batch should succeed"); + assert!(results.is_empty()); + } + + #[test] + fn batch_put_from_rejects_mismatched_lengths() { + let store = MooncakeStore::new().expect("new should succeed"); + let keys = &["key1", "key2"]; + let buffers: Vec<*mut c_void> = vec![std::ptr::null_mut(); 3]; + let sizes = &[100usize, 200]; + + let result = unsafe { store.batch_put_from(keys, &buffers, sizes, None) }; + assert!(matches!( + result, + Err(StoreError::InvalidArgument(_)) + )); + } + + #[test] + fn batch_get_into_rejects_mismatched_lengths() { + let store = MooncakeStore::new().expect("new should succeed"); + let keys = &["key1"]; + let buffers: Vec<*mut c_void> = vec![std::ptr::null_mut(); 2]; + let sizes = &[100usize]; + + let result = unsafe { store.batch_get_into(keys, &buffers, sizes) }; + assert!(matches!( + result, + Err(StoreError::InvalidArgument(_)) + )); + } } -- 2.34.1 From aa2244c283d61395e7d542a41e76a1c7784003fa Mon Sep 17 00:00:00 2001 From: Zhaoyi Li <36555117+Lzy17@users.noreply.github.com> Date: Sat, 2 May 2026 11:45:01 -0500 Subject: [PATCH 030/382] [TE] Save the active device with hipGetDevice on entry and restore it before (#2018) * Save the active device with hipGetDevice on entry and restore it before returning. The function is now transparent to the caller's HIP context. --- .../transport/hip_transport/hip_transport.cpp | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/mooncake-transfer-engine/src/transport/hip_transport/hip_transport.cpp b/mooncake-transfer-engine/src/transport/hip_transport/hip_transport.cpp index 2b672fc2..929739f5 100644 --- a/mooncake-transfer-engine/src/transport/hip_transport/hip_transport.cpp +++ b/mooncake-transfer-engine/src/transport/hip_transport/hip_transport.cpp @@ -251,6 +251,17 @@ static int setDeviceContext(void* source_ptr, int& device_id) { } static void setupP2PAccess(int num_devices) { + // Save the active device. The loop below calls hipSetDevice once per + // iteration; without restoring it before returning, the calling thread is + // left with its active device pinned to num_devices-1, causing downstream + // HIP calls on the same thread (e.g. PyTorch allocations in TP workers) to + // target the wrong GPU. + int original_device = -1; + if (!checkHip(hipGetDevice(&original_device), + "HipTransport: failed to get current device")) { + return; + } + auto clearStickyPeerAccessError = [](int src_device, int dst_device) { // hipDeviceEnablePeerAccess may leave hipErrorPeerAccessAlreadyEnabled // in the runtime's last-error slot. Clear it so subsequent PyTorch HIP @@ -300,6 +311,12 @@ static void setupP2PAccess(int num_devices) { } } } + + // Restore the active device so this function is transparent to the caller. + if (original_device >= 0) { + (void)checkHip(hipSetDevice(original_device), + "HipTransport: failed to restore device"); + } } static int getNumStreams() { -- 2.34.1 From c02b66966903e76b215a9b6be48b2f17a463b575 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E9=B9=A4=E7=94=B7?= Date: Sun, 3 May 2026 07:14:40 +0800 Subject: [PATCH 031/382] [TE] Fix EFA segfault race and DP>1 peer_map_ thrashing (#2023) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Fix segfault race between setPeerNicPath and in-flight submitPostSend EfaEndPoint::submitPostSend was latching peer_fi_addr_ under lock_ and releasing the lock before calling EfaContext::submitSlicesOnPeer, which calls fi_write(shared_ep_, ..., peer_fi_addr, ...) outside the lock. A concurrent setPeerNicPath() / disconnect() (both take the write lock and call context_.removePeerAddr, i.e. fi_av_remove) could invalidate the AV slot between the latch and the fi_write, causing libfabric's EFA provider to dereference a stale entry and segfault. Repro: SGLang 2P2D GLM-5.1-FP8 prefill, PP=2 TP=8 CP=8 EP=8 topology, moe_a2a=deepep, 16 EFA NICs, MC_MAX_WR=16384. The P2P handshake port changes for a peer process after a rebind, setPeerNicPath() takes the CONNECTED branch and disconnects while sender threads have fi_write calls in flight for that peer. Fix: hold the read lock across the entire submitSlicesOnPeer call so disconnect (write lock) cannot remove the AV entry while a submit is in flight. Re-check status_ under the lock and fail the batch if a disconnect raced in between, letting the caller retry with a fresh setupConnectionsByActive(). Read locks still allow multiple senders to the same peer to submit in parallel, so no throughput regression. Refs: https://github.com/kvcache-ai/Mooncake/issues/2022 * fix(efa): skip AV churn on duplicate passive handshake, silence "Re-establish" warnings Under bilateral sglang P/D traffic every (localNIC, remoteNIC) pair sees two handshakes: the active side inserts the peer into the AV, and the remote side later fires its own active handshake which arrives at our passive handler after we are already CONNECTED. The previous code unconditionally treated that second handshake as a reconnect, logged W "Re-establish EFA connection" and ran fi_av_remove + fi_av_insert. Observed impact on a real run: ~tens of thousands of lines per decode node, and a brief window where in-flight fi_write could see a stale fi_addr_t (the race PR 2023 already hardened against on the submit side). Cache the EFA address bytes that we successfully inserted. On a passive handshake, if we are CONNECTED and the incoming peer address matches the cache, it is the same peer and the same AV slot — return the local desc and skip the churn entirely. Only reinsert if the peer address genuinely changed (peer process restart → new QPN → different bytes); demote that branch to LOG(INFO) so real reconnects remain visible without being drowned out. Cache is populated from both active and passive connect paths and cleared in disconnectUnlocked / markDetachedForTeardown so a subsequent passive handshake takes the first-connect path correctly. Complements PR 2023: PR 2023 holds the read lock across submitPostSend so the AV cannot be removed mid-fi_write; this change removes the AV churn at the source for 99% of the handshakes, so the read-lock hold becomes defense-in-depth rather than the only barrier. Co-Authored-By: Claude Opus 4.7 * fix(efa): don't skip fi_av_insert on duplicate passive handshake Previous commit skipped fi_av_remove + fi_av_insert when the passive peer address matched the cached one. Under bilateral sglang P/D load (128-concurrency, 3K bench) this caused requests to stall: 31 transfers stuck in #transfer-req, prefill #inflight-req pinned at 128 with no progress. The hypothesis that EFA handshakes with an unchanged peer address could be treated as a no-op was wrong — libfabric's EFA provider tracks per-peer transport state (AH activation, RNR backoff, internal sequence bookkeeping) that a fresh fi_av_insert re-seeds, and skipping it leaves the provider in a stale state that silently drops or stalls the fi_write path. Keep the cached_peer_addr_ field (it's cheap and harmless) but drop the AV-skip fast path. The reinsert now happens on every passive handshake, restoring the functional behavior PR 2023 alone validated. The only surviving optimization is log-level classification: same-address handshake → VLOG(1), different-address → LOG(WARNING). This still eliminates the "Re-establish EFA connection" warning spam on bilateral symmetric traffic without touching semantics. PR 2023's read-lock widening in submitPostSend is now the primary defense for the race described there, with the AV reinsert once again the common-case path. Verified by repro: sglang-glm5-2p2d bench 3072 1024 500 18 128 no longer stalls. Co-Authored-By: Claude Opus 4.7 * fix(efa): don't tear down AV slot when only the peer RPC port rotates Symptom: after the previous commit (which restored fi_av_insert on every passive handshake) traffic no longer stalls, but decode logs still show a flood of "Peer reconnected with new address, re-establishing" from setPeerNicPath. Each of those lines corresponds to a peer whose EFA endpoint did not move — only the P2PHANDSHAKE RPC port changed. Root cause: peer_nic_path_ stores the full "host:PORT@nic" string. Every sglang KV transfer bootstrap picks a new RPC port, so setPeerNicPath sees a "different" path and calls disconnectUnlocked() + reinserts, even though the EFA peer address is identical. The RPC port is pure handshake metadata — EFA SRD addressing is keyed on the binary efa_addr (GID/QPN) returned by fi_getname(), which is stable for the lifetime of the peer process. The spurious AV churn pays a fi_av_remove + fi_av_insert plus an AH activation warm-up (first-packet latency hit) on every transfer, exactly the performance impact the user was concerned about. Fix: in setPeerNicPath, compare the normalized nic path (host+NIC, no port) before tearing anything down. If only the port changed, update the stored string and return; keep peer_fi_addr_ and cached_peer_addr_ intact so the next submitPostSend can reuse the already-active AH. A genuine peer restart is still caught downstream by setupConnectionsByPassive's efa_addr == cached_peer_addr_ check (peer QPN differs after restart → different bytes → real re-establish path). Also normalize the path equality check in setupConnectionsByPassive's sanity guard so the same port rotation doesn't spuriously reject the handshake. Co-Authored-By: Claude Opus 4.7 * Revert "fix(efa): don't tear down AV slot when only the peer RPC port rotates" This reverts commit 4548bf56aaf4ac07f3c98b5434a0ab613874fed5. * fix(efa): key peer_map_ by full host:port@nic to handle multi-worker peers Root cause of the "Peer reconnected" spam (and the AH warm-up tax on every KV transfer) in sglang 2P2D with DP>1. EfaContext::endpoint() was keying peer_map_ by normalizeNicPath(peer) which strips the RPC port. The design comment said this was to "reuse handles across reconnections" since each P2PHANDSHAKE init picks a random port. That premise assumed one Mooncake TransferEngine per peer host — true for DP=1 benchmarks but violated under sglang P/D disaggregation: each DP worker is a separate Python process with its own TransferEngine and its own rpc_port, and they all share the same host IP + EFA NIC on that host. With DP=N peer workers, all N of them collapse onto a single peer_map_ slot. Every incoming handshake from a different DP worker looks to that slot like "peer reconnected with a new address" (different port), so setPeerNicPath() tears down the AV slot (fi_av_remove + fi_av_insert) and the next fi_write pays an AH warm-up. Under bilateral 128-concurrency traffic this devolves into permanent thrashing; the symptoms reported by the user track this model precisely: - DP=1: no spam, normal performance (single worker per host, normalization harmless). - DP=2/4: increasing Re-establish log volume, progressive first-packet latency. - DP=8: thousands of Re-establish per second, decode #running drops to 1, prefill #inflight stays pinned at 128, timeouts. Fix: key peer_map_ by the full "host:port@nic" path. Each DP worker is a stable, distinct process with a stable port for its entire lifetime, so the cache still hits on every steady-state lookup — no churn. A genuine peer process restart (new rpc_port) leaves the old EfaEndPoint in the map; that is a few bytes of leak per ex-worker, far cheaper than the per-transfer AV teardown it replaces. peekEndpoint, deleteEndpoint, and warmupSegment's post-failure cleanup all switch to the full path for consistency. setPeerNicPath() stays intact as defensive code but will no longer fire in the common path because each new DP worker now gets its own endpoint. Undoes the buggy 4548bf5 from a completely different direction: the previous attempt tried to skip AV churn at the setPeerNicPath level, which hid real peer restarts and corrupted rkey routing. The correct fix is upstream of that — separate endpoints per peer process so port changes only happen on actual restart. Co-Authored-By: Claude Opus 4.7 * style(efa): clang-format-20 fixup --------- Co-authored-by: Claude Opus 4.7 Co-authored-by: EC2 Default User --- .../transport/efa_transport/efa_endpoint.h | 7 +++ .../transport/efa_transport/efa_context.cpp | 32 ++++++++--- .../transport/efa_transport/efa_endpoint.cpp | 56 ++++++++++++++++--- .../transport/efa_transport/efa_transport.cpp | 2 +- 4 files changed, 79 insertions(+), 18 deletions(-) diff --git a/mooncake-transfer-engine/include/transport/efa_transport/efa_endpoint.h b/mooncake-transfer-engine/include/transport/efa_transport/efa_endpoint.h index 610dbc48..7b886be4 100644 --- a/mooncake-transfer-engine/include/transport/efa_transport/efa_endpoint.h +++ b/mooncake-transfer-engine/include/transport/efa_transport/efa_endpoint.h @@ -124,6 +124,13 @@ class EfaEndPoint { RWSpinlock lock_; // protects peer_nic_path_ and status_ std::string peer_nic_path_; fi_addr_t peer_fi_addr_; // slot in context_.av() + // Last peer EFA address successfully inserted into the AV. Used to + // make passive handshakes idempotent: when both peers initiate a + // handshake concurrently (the common case under bilateral sglang PD + // traffic), the second passive handshake carries the same EFA + // address as the cached value, and we can skip the fi_av_remove/ + // fi_av_insert churn entirely. + std::string cached_peer_addr_; }; } // namespace mooncake diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp index 9bc86830..5ab37636 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp @@ -458,16 +458,31 @@ void* EfaContext::mrDesc(void* addr) { std::shared_ptr EfaContext::endpoint( const std::string& peer_nic_path) { - // Use normalized key (strip port) so the same physical peer reuses its - // handle across reconnections. Each P2PHANDSHAKE run picks a random - // port, producing a different peer_nic_path for the same peer host+NIC. - std::string key = normalizeNicPath(peer_nic_path); + // Key the peer map by the full "host:port@nic" path, not the + // port-stripped form. Under sglang DP>1 every DP worker on a peer + // host is a distinct process with its own Mooncake TransferEngine + // and its own P2PHANDSHAKE RPC port. They share the same host+NIC + // but map to different EFA QPNs / memory regions. If we normalize + // the port away, every DP worker on that host collapses onto the + // same EfaEndPoint slot: each arriving handshake looks like a + // "peer reconnected with new address" to the previous holder, + // triggering fi_av_remove + fi_av_insert (and an AH warm-up) on + // every KV transfer. At high DP this devolves into permanent + // thrashing and eventually "Remote MR invalid" when an in-flight + // fi_write runs against a stale AV slot. + // + // The port is stable for the lifetime of a sglang worker process + // (Mooncake only calls initialize() once), so keying by full path + // costs nothing in steady state. A genuine peer restart (process + // re-launch → new RPC port) creates a new map entry and leaks the + // old EfaEndPoint — that's at most a few bytes per ex-worker and + // far cheaper than the churn the old normalization caused. + const std::string& key = peer_nic_path; { RWSpinlock::ReadGuard guard(peer_map_lock_); auto it = peer_map_.find(key); if (it != peer_map_.end()) { - it->second->setPeerNicPath(peer_nic_path); return it->second; } } @@ -478,7 +493,6 @@ std::shared_ptr EfaContext::endpoint( RWSpinlock::WriteGuard guard(peer_map_lock_); auto it = peer_map_.find(key); if (it != peer_map_.end()) { - it->second->setPeerNicPath(peer_nic_path); return it->second; } peer_map_[key] = new_ep; @@ -488,7 +502,7 @@ std::shared_ptr EfaContext::endpoint( std::shared_ptr EfaContext::peekEndpoint( const std::string& peer_nic_path) { RWSpinlock::ReadGuard guard(peer_map_lock_); - auto it = peer_map_.find(normalizeNicPath(peer_nic_path)); + auto it = peer_map_.find(peer_nic_path); if (it == peer_map_.end()) return nullptr; return it->second; } @@ -497,7 +511,7 @@ int EfaContext::deleteEndpoint(const std::string& peer_nic_path) { std::shared_ptr ep; { RWSpinlock::WriteGuard guard(peer_map_lock_); - auto it = peer_map_.find(normalizeNicPath(peer_nic_path)); + auto it = peer_map_.find(peer_nic_path); if (it == peer_map_.end()) return 0; ep = it->second; peer_map_.erase(it); @@ -656,7 +670,7 @@ int EfaContext::submitPostSend( // freeing its AV entry for reuse. Under the shared-endpoint model // this is cheap (no fid_ep to destroy). if (rc != 0 && !ep->connected()) { - deleteEndpoint(normalizeNicPath(peer_nic_path)); + deleteEndpoint(peer_nic_path); } } diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_endpoint.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_endpoint.cpp index 29a1ca2a..7357a22f 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_endpoint.cpp @@ -80,6 +80,7 @@ int EfaEndPoint::setupConnectionsByActive() { rc = context_.insertPeerAddr(peer_desc.efa_addr, peer_fi_addr_); if (rc != 0) return rc; + cached_peer_addr_ = peer_desc.efa_addr; status_.store(CONNECTED, std::memory_order_release); VLOG(1) << "EFA connection established: " << toString() @@ -90,10 +91,6 @@ int EfaEndPoint::setupConnectionsByActive() { int EfaEndPoint::setupConnectionsByPassive(const HandShakeDesc& peer_desc, HandShakeDesc& local_desc) { RWSpinlock::WriteGuard guard(lock_); - if (status_.load(std::memory_order_relaxed) == CONNECTED) { - LOG(WARNING) << "Re-establish EFA connection: " << toString(); - disconnectUnlocked(); - } if (peer_desc.peer_nic_path != context_.nicPath() || peer_desc.local_nic_path != peer_nic_path_) { @@ -112,11 +109,37 @@ int EfaEndPoint::setupConnectionsByPassive(const HandShakeDesc& peer_desc, return ERR_REJECT_HANDSHAKE; } + // Classify this passive handshake so we can emit the right log level + // without changing functional behavior: the AV reinsert below must + // still happen on every handshake because libfabric's EFA provider + // tracks per-peer transport state that depends on a fresh + // fi_av_insert (e.g. provider-internal AH activation and RNR state). + // Skipping reinsert caused request-level stalls under bilateral + // sglang P/D load even when the peer EFA address was unchanged. + // + // Log semantics: + // * Same cached peer address → benign symmetric handshake under + // bilateral traffic. Demote to INFO to keep decode logs readable + // without hiding real reconnects. + // * Different cached peer address → genuine reconnect (peer + // restart, port reshuffle that reached disconnect first, etc.). + // Keep at WARNING so it stays visible. + if (status_.load(std::memory_order_relaxed) == CONNECTED) { + if (!cached_peer_addr_.empty() && + peer_desc.efa_addr == cached_peer_addr_) { + VLOG(1) << "EFA passive handshake (same peer addr): " << toString(); + } else { + LOG(WARNING) << "Re-establish EFA connection: " << toString(); + } + disconnectUnlocked(); + } + int ret = context_.insertPeerAddr(peer_desc.efa_addr, peer_fi_addr_); if (ret != 0) { local_desc.reply_msg = "Failed to insert peer address"; return ret; } + cached_peer_addr_ = peer_desc.efa_addr; local_desc.local_nic_path = context_.nicPath(); local_desc.peer_nic_path = peer_nic_path_; @@ -138,12 +161,14 @@ void EfaEndPoint::disconnectUnlocked() { context_.removePeerAddr(peer_fi_addr_); peer_fi_addr_ = FI_ADDR_UNSPEC; } + cached_peer_addr_.clear(); status_.store(UNCONNECTED, std::memory_order_release); } void EfaEndPoint::markDetachedForTeardown() { RWSpinlock::WriteGuard guard(lock_); peer_fi_addr_ = FI_ADDR_UNSPEC; + cached_peer_addr_.clear(); status_.store(UNCONNECTED, std::memory_order_release); } @@ -165,11 +190,26 @@ int EfaEndPoint::submitPostSend( } } - fi_addr_t peer; - { - RWSpinlock::ReadGuard guard(lock_); - peer = peer_fi_addr_; + // Hold a read lock for the entire submit window so setPeerNicPath() / + // disconnect() (which take the write lock) cannot fi_av_remove() our + // slot while an fi_write() inside submitSlicesOnPeer is still in + // flight. The previous code latched peer_fi_addr_ under the lock and + // released it before calling into the context, leaving a race: a + // concurrent peer reconnect could remove the AV entry after the latch, + // and libfabric would segfault on the stale fi_addr_t. Read locks + // stack, so multiple senders to the same peer still submit in parallel. + RWSpinlock::ReadGuard guard(lock_); + + // Re-check status under the lock — a racing disconnect() could have + // flipped us to UNCONNECTED between the outer check and acquiring the + // lock. Fail the batch so the caller retries with a fresh setup. + if (status_.load(std::memory_order_acquire) != CONNECTED) { + for (auto* slice : slice_list) failed_slice_list.push_back(slice); + slice_list.clear(); + return ERR_ENDPOINT; } + + fi_addr_t peer = peer_fi_addr_; if (peer == FI_ADDR_UNSPEC) { for (auto* slice : slice_list) failed_slice_list.push_back(slice); slice_list.clear(); diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp index eeebf268..3106de71 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp @@ -745,7 +745,7 @@ int EfaTransport::warmupSegment(const std::string& segment_name) { // slot is freed and the next warmup retry starts // clean. Cheap under the shared-endpoint model — // no fi_endpoint teardown required. - ctx->deleteEndpoint(normalizeNicPath(path)); + ctx->deleteEndpoint(path); } return rc; })); -- 2.34.1 From 3ac7de25acecff59ad3bc9f1d595bc6c733f86bb Mon Sep 17 00:00:00 2001 From: Venkat Raman Date: Sun, 3 May 2026 08:18:18 +0200 Subject: [PATCH 032/382] [Store] Add lock-free MmapArena allocator for buffer mmap path (#1820) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [Store] Add lock-free MmapArena allocator for buffer mmap path Replace per-allocation mmap() syscalls in allocate_buffer_mmap_memory() with a lock-free atomic bump allocator (MmapArena). Pre-allocates a configurable pool (default 64GB) and serves allocations via CAS loop, reducing allocation latency from ~1us (mmap syscall) to ~50ns (atomic). Allocation lifecycle is static: all callers (ClientBufferAllocator, global segments in RealClient::setup_internal) allocate at startup and free at shutdown. The arena outlives all allocations, so the bump-only (no individual free) design is correct for this usage pattern. Feature-flagged via gflags: --use_mmap_arena_allocator (default: true) --mmap_arena_pool_size (default: 64GB) Falls back to direct mmap() when arena is disabled, fails to init, or is exhausted. Cherry-picked from flow-ipc-poc branch (utils.cpp perf path only). Co-Authored-By: Claude Opus 4.6 * [Store] Fix three correctness issues in MmapArena 1. Honor caller's alignment contract: allocate() now accepts a per-call alignment parameter and uses max(arena default, caller request). allocate_buffer_mmap_memory() forwards its alignment argument to the arena. Previously, the caller's alignment was silently ignored — the arena always used 64-byte alignment regardless of what the caller requested. 2. Remove MAP_POPULATE from arena pool mmap: the default pool is 64GB but callers typically use only a fraction (e.g. 4GB of segments). MAP_POPULATE would pre-fault all 64GB of pages upfront, causing seconds of startup delay and potentially triggering OOM on machines with less physical memory. Pages now fault on demand. 3. Make alignment_ atomic and store it BEFORE the CAS on pool_base_: previously alignment_ was a plain size_t written AFTER the release CAS, so the store was not in the happens-before relationship established by the acquire-release pair on pool_base_. Now both alignment_ and pool_size_ are stored before the CAS with the release fence guaranteeing their visibility to readers. --------- Co-authored-by: Claude Opus 4.6 --- .github/workflows/ci.yml | 5 + README.md | 23 +- docker/mooncake.Dockerfile | 1 + .../mooncake-store-deployment-guide.md | 22 + docs/source/design/mooncake-store.md | 2 + docs/source/getting_started/build.md | 37 +- .../hicache-integration-v1.md | 66 ++ .../sglang-integration/hicache-quick-start.md | 29 +- .../performance/allocator-benchmark-result.md | 11 +- .../source/troubleshooting/troubleshooting.md | 27 +- mooncake-store/include/mmap_arena.h | 108 +++ mooncake-store/include/utils.h | 60 +- mooncake-store/src/CMakeLists.txt | 29 +- mooncake-store/src/mmap_arena.cpp | 274 +++++++ mooncake-store/src/utils.cpp | 182 ++++- mooncake-store/tests/CMakeLists.txt | 89 ++- .../tests/mmap_arena_fallback_test.cpp | 153 ++++ mooncake-store/tests/mmap_arena_test.cpp | 751 ++++++++++++++++++ mooncake-store/tests/utils_test.cpp | 13 + .../check_hicache_hugepage_requirements.py | 236 ++++++ scripts/test_hicache_hugepage_requirements.py | 73 ++ .../test_hicache_storage_mooncake_backend.py | 7 + 22 files changed, 2127 insertions(+), 71 deletions(-) create mode 100644 mooncake-store/include/mmap_arena.h create mode 100644 mooncake-store/src/mmap_arena.cpp create mode 100644 mooncake-store/tests/mmap_arena_fallback_test.cpp create mode 100644 mooncake-store/tests/mmap_arena_test.cpp create mode 100644 scripts/check_hicache_hugepage_requirements.py create mode 100644 scripts/test_hicache_hugepage_requirements.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b95b6fc6..f0f028d3 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -75,6 +75,11 @@ jobs: sudo apt-get update sudo apt-get install -y lcov gcovr ninja-build + - name: Test HugeTLB sizing helper + run: | + python3 scripts/test_hicache_hugepage_requirements.py + shell: bash + - name: Set up coverage compilation flags run: | echo "Setting up coverage compilation flags..." diff --git a/README.md b/README.md index 4a1e2ca9..29f15d8d 100644 --- a/README.md +++ b/README.md @@ -220,9 +220,30 @@ docker build -f docker/mooncake.Dockerfile \ The resulting image already has a virtual environment at `/opt/venv` with the freshly built wheel installed. Launch it with GPU/RDMA access as needed, for example: ```bash -docker run --gpus all --network host -it mooncake:from-source /bin/bash +python3 scripts/check_hicache_hugepage_requirements.py \ + --tp-size 4 \ + --hicache-size 64gb \ + --global-segment-size 8gb \ + --arena-pool-size 56gb \ + --available-hugetlb 512gb + +sudo sysctl -w vm.nr_hugepages=262144 +grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo + +docker run --gpus all \ + --network host \ + --ipc=host \ + --ulimit memlock=-1 \ + --shm-size=128g \ + -e MC_STORE_USE_HUGEPAGE=1 \ + -e MC_STORE_HUGEPAGE_SIZE=2MB \ + -e MOONCAKE_GLOBAL_SEGMENT_SIZE=8gb \ + -e MC_MMAP_ARENA_POOL_SIZE=56gb \ + -it mooncake:from-source /bin/bash ``` +The `64gb` / `56gb` values above are tuned examples for large HiCache deployments, not allocator defaults. The arena is off by default. Setting `MC_MMAP_ARENA_POOL_SIZE=...` explicitly both enables and sizes the arena; if you enable it via gflag instead, the default pool size is `8gb`. On smaller hosts, start with `8gb` or `16gb` and size upward with the helper. Set `MC_DISABLE_MMAP_ARENA=1` (also accepts `true`, `yes`, or `on`) instead when you want the baseline direct-`mmap()` path. Like the arena size itself, this must be set before the first Mooncake mmap-buffer allocation in the process. Arena bring-up is a one-shot lazy init, so after a failed first attempt you need to restart the process to retry with corrected env / hugepage settings. Without `MC_STORE_USE_HUGEPAGE=1`, the arena may opportunistically try hugepages and then retry on regular pages if HugeTLB is unavailable. When `MC_STORE_USE_HUGEPAGE=1` is present, Mooncake instead preserves the strict hugepage contract for both arena and direct-`mmap()` host-buffer allocation instead of silently downgrading to regular pages. + > [!NOTE] > Make sure you build the image from the repository root so that Git metadata and submodules are available inside the build context. diff --git a/docker/mooncake.Dockerfile b/docker/mooncake.Dockerfile index 60d7314d..3dda2e6c 100644 --- a/docker/mooncake.Dockerfile +++ b/docker/mooncake.Dockerfile @@ -116,6 +116,7 @@ RUN apt-get update && \ # Copy wheels produced in builder stage and install them via pip COPY --from=builder /workspace/mooncake-wheel/dist /tmp/mooncake-wheel +COPY --chmod=755 scripts/check_hicache_hugepage_requirements.py /usr/local/bin/mooncake-hicache-sizing RUN python${PYTHON_VERSION} -m pip install --no-cache-dir /tmp/mooncake-wheel/*.whl && rm -rf /tmp/mooncake-wheel /root/.cache/pip CMD ["/bin/bash"] diff --git a/docs/source/deployment/mooncake-store-deployment-guide.md b/docs/source/deployment/mooncake-store-deployment-guide.md index 46f44f32..b15df264 100644 --- a/docs/source/deployment/mooncake-store-deployment-guide.md +++ b/docs/source/deployment/mooncake-store-deployment-guide.md @@ -144,6 +144,28 @@ curl -s http://:9003/metrics/summary - Local memcpy optimization (Store transfer path) - `MC_STORE_MEMCPY` (default `0`/false): Set to `1` to prefer local memcpy when source/destination are on the same client. +- Memory allocator (mmap buffer path) + - `MC_STORE_USE_HUGEPAGE` (default unset): Set to `1` to request HugeTLB-backed `mmap()` allocations for the direct allocation path. This requires hugepages to be reserved on the host first. + - `MC_STORE_HUGEPAGE_SIZE` (default `2MB`): Hugepage size to request when hugepages are enabled. Supported values are `2MB` and `1GB`. + - `MC_MMAP_ARENA_POOL_SIZE` (default unset): Size of the pre-allocated arena pool for mmap buffer allocations. Accepts human-readable sizes (e.g., `8gb`, `20gb`). Setting this variable explicitly enables the arena; if the arena is enabled via gflag instead, the default pool size is `8gb`. The arena is allocated once at first use and serves subsequent allocations via lock-free atomic bump pointer. Set this to match available hugepage capacity. + - `MC_DISABLE_MMAP_ARENA` (default unset): Set to `1` to disable the arena and fall back to per-call `mmap()`, even if the arena was explicitly requested. Also accepts `true`, `yes`, or `on`. This must be set before the first Mooncake mmap-buffer allocation in the process. Useful for debugging or when hugepage capacity is limited. Lazy arena initialization is one-shot per process, so after a failed first attempt you need to restart the process to retry arena bring-up. + +For HiCache-style deployments or other large buffer allocations, pre-flight the host with the sizing helper before reserving hugepages: + +```bash +python3 scripts/check_hicache_hugepage_requirements.py \ + --tp-size 4 \ + --hicache-size 64gb \ + --global-segment-size 8gb \ + --arena-pool-size 56gb \ + --available-hugetlb 512gb + +sudo sysctl -w vm.nr_hugepages=262144 +grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo +``` + +The `64gb` / `56gb` inputs above are tuned examples for large HiCache deployments, not defaults. The arena remains disabled unless you explicitly enable it. If you enable it via gflag without an env override, the default pool size is `8gb`. On smaller hosts, start with `8gb` or `16gb` and size upward with the helper. + ## Set the Log Level for yalantinglibs coro_rpc and coro_http By default, the log level is set to warning. You can customize it using the following environment variable: diff --git a/docs/source/design/mooncake-store.md b/docs/source/design/mooncake-store.md index a171e6e2..acaa7e38 100644 --- a/docs/source/design/mooncake-store.md +++ b/docs/source/design/mooncake-store.md @@ -837,6 +837,8 @@ The HTTP metadata server can be configured using the following parameters: - MC_STORE_CLIENT_METRIC_INTERVAL: Reporting interval in seconds, default 0 (collects but does not report). - MC_STORE_USE_HUGEPAGE: Enables huge page support, disabled by default. - MC_STORE_HUGEPAGE_SIZE: Specifies the page size of the huge page to use, default 2M. +- MC_MMAP_ARENA_POOL_SIZE: Size of the pre-allocated arena pool for mmap buffer allocations. Accepts human-readable sizes (e.g., `"8gb"`, `"20gb"`). Providing this variable explicitly enables the arena; when enabled via gflag without an env override, the default pool size is `8gb`. The arena is allocated once at first use and serves subsequent allocations via lock-free atomic bump pointer (~50ns per allocation vs ~1000ns for direct mmap). +- MC_DISABLE_MMAP_ARENA: Set to `1` to disable the arena allocator and fall back to per-call `mmap()`, even if the arena was explicitly requested. Also accepts `true`, `yes`, or `on`. This must be set before the first Mooncake mmap-buffer allocation in the process. Useful for debugging or memory-constrained environments where pre-allocating a pool is not desirable. #### Usage Example To start the master service with the HTTP metadata server enabled: ```bash diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index 168fe1cf..adbdb42a 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -173,17 +173,50 @@ pip install mooncake-transfer-engine-non-cuda ``` ## Use Mooncake in Docker Containers -Mooncake supports Docker-based deployment. What you need is to get Docker image by `docker pull alogfans/mooncake`. +Mooncake supports Docker-based deployment. You can either build the image from +this repository with `docker/mooncake.Dockerfile` or substitute a published +tag that matches the release you want to run. For the container to use the host's network resources, you need to add the `--device` option when starting the container. The following is an example. ``` # In host -sudo docker run --net=host --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/rdma_cm --ulimit memlock=-1 -t -i mooncake:v0.9.0 /bin/bash +sudo docker build -f docker/mooncake.Dockerfile -t mooncake:from-source . +sudo docker run --net=host --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/rdma_cm --ulimit memlock=-1 -t -i mooncake:from-source /bin/bash # Run transfer engine in container cd /Mooncake-main/build/mooncake-transfer-engine/example ./transfer_engine_bench --device_name=ibp6s0 --metadata_server=10.1.101.3:2379 --mode=target --local_server_name=10.1.100.3 ``` +For SGLang HiCache deployments inside Docker, reserve HugeTLB pages on the host before starting the container and pass the allocator settings through the container environment: + +```bash +python3 scripts/check_hicache_hugepage_requirements.py \ + --tp-size 4 \ + --hicache-size 64gb \ + --global-segment-size 8gb \ + --arena-pool-size 56gb \ + --available-hugetlb 512gb + +sudo sysctl -w vm.nr_hugepages=262144 +grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo + +sudo docker run --gpus all \ + --net=host \ + --ipc=host \ + --ulimit memlock=-1 \ + --shm-size=128g \ + --device=/dev/infiniband/uverbs0 \ + --device=/dev/infiniband/rdma_cm \ + -e MC_STORE_USE_HUGEPAGE=1 \ + -e MC_STORE_HUGEPAGE_SIZE=2MB \ + -e MOONCAKE_GLOBAL_SEGMENT_SIZE=8gb \ + -e MC_MMAP_ARENA_POOL_SIZE=56gb \ + -t -i mooncake:from-source /bin/bash +``` + +The `64gb` / `56gb` values above are tuned examples for large HiCache deployments, not defaults. The arena remains disabled unless you explicitly enable it, and if you enable it via gflag without an env override the default pool size is `8gb`. On smaller hosts, start with `8gb` or `16gb` and size upward with the helper. When you want the baseline direct-`mmap()` path instead of the arena, set `MC_DISABLE_MMAP_ARENA=1` (also accepts `true`, `yes`, or `on`) and omit `MC_MMAP_ARENA_POOL_SIZE`. Set it before the first Mooncake mmap-buffer allocation in the process. If you build the image from source with `docker/mooncake.Dockerfile`, that source-built image also installs the helper as `mooncake-hicache-sizing`. +Without `MC_STORE_USE_HUGEPAGE=1`, the arena may opportunistically try hugepages and then retry on regular pages if HugeTLB is unavailable. When `MC_STORE_USE_HUGEPAGE=1` is set, both the arena path and the direct-`mmap()` fallback path require HugeTLB pages. Mooncake will not silently degrade that explicit hugepage request to regular pages. + ## Advanced Compile Options The following options can be used during `cmake ..` to specify whether to compile certain components of Mooncake. - `-DUSE_CUDA=[ON|OFF]`: Enable GPU memory support (GPUDirect RDMA, NVMe-oF, and GPU-aware TCP transport). **Default: OFF.** Required when transferring GPU memory (e.g., KV cache in vLLM disaggregated serving), even when using TCP protocol. diff --git a/docs/source/getting_started/examples/sglang-integration/hicache-integration-v1.md b/docs/source/getting_started/examples/sglang-integration/hicache-integration-v1.md index f47d4886..99e31cf0 100644 --- a/docs/source/getting_started/examples/sglang-integration/hicache-integration-v1.md +++ b/docs/source/getting_started/examples/sglang-integration/hicache-integration-v1.md @@ -429,6 +429,72 @@ If the model is small but the GPU memory is large — especially in multi-TP (te In such cases, you should manually configure an appropriate L2 cache size based on your hardware. This can be done by setting `--hicache-ratio` or `--hicache-size`. +**HugeTLB Bring-up Checklist:** + +Before enabling HugeTLB-backed HiCache memory, size the host's hugepage pool against your planned `--hicache-size`, `MOONCAKE_GLOBAL_SEGMENT_SIZE`, and `MC_MMAP_ARENA_POOL_SIZE` values. + +From a source checkout: + +```bash +python3 scripts/check_hicache_hugepage_requirements.py \ + --tp-size 4 \ + --hicache-size 64gb \ + --global-segment-size 8gb \ + --arena-pool-size 56gb \ + --available-hugetlb 512gb +``` + +From the source-built Docker image: + +```bash +mooncake-hicache-sizing \ + --tp-size 4 \ + --hicache-size 64gb \ + --global-segment-size 8gb \ + --arena-pool-size 56gb +``` + +The `64gb` / `56gb` values above are tuned examples for large multi-GPU HiCache deployments, not defaults. Arena enablement remains opt-in, and the default pool is `8gb` only when you enable the arena via gflag without an explicit env override. On smaller hosts, start with `8gb` or `16gb` and size upward with the helper. + +The helper reports two numbers: + +* **Baseline floor** = `hicache-size + MOONCAKE_GLOBAL_SEGMENT_SIZE` per rank. Falling below this usually means startup or allocation failure. +* **Clean arena target** = baseline floor + `MC_MMAP_ARENA_POOL_SIZE` per rank. Reaching this target makes arena-backed launches less likely to spill onto the regular-page fallback path. + +Treat the helper as a planning tool rather than a hard proof: runs can still succeed between the baseline floor and the clean arena target if some arena allocations fall back to regular pages. + +Reserve HugeTLB pages on the host before launching SGLang: + +```bash +sudo sysctl -w vm.nr_hugepages=262144 +grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo +``` + +With `2 MiB` pages, `262144` pages equals `512 GiB`; `49152` pages equals `96 GiB`. Persist the setting with `/etc/sysctl.d/90-mooncake-hugepages.conf` if you need it across reboots. + +**Memory Allocator Tuning:** + +Mooncake's mmap arena is opt-in for HiCache host KV allocations. Setting `MC_MMAP_ARENA_POOL_SIZE` explicitly enables the arena and sizes the pool; the arena then pre-allocates a hugepage-backed pool and serves subsequent allocations via atomic bump pointer, reducing per-allocation latency from ~1000ns to ~50ns. The `56gb` example below is a benchmark-scale tuning value, not the allocator default. + +For HugeTLB-backed runs, export the hugepage and allocator settings together: + +```bash +export MC_STORE_USE_HUGEPAGE=1 +export MC_STORE_HUGEPAGE_SIZE=2MB +export MOONCAKE_GLOBAL_SEGMENT_SIZE=8gb +export MC_MMAP_ARENA_POOL_SIZE=56gb +``` + +To disable the arena and fall back to direct `mmap()` while keeping the hugepage-backed baseline path, set the flag before the first Mooncake mmap-buffer allocation in the process: + +```bash +export MC_DISABLE_MMAP_ARENA=1 +``` + +Without `MC_STORE_USE_HUGEPAGE=1`, the arena may opportunistically try hugepages and then retry on regular pages if HugeTLB is unavailable. If `MC_STORE_USE_HUGEPAGE=1` is set, Mooncake treats hugepages as a hard requirement for both the arena path and the direct-`mmap()` baseline path. It will not silently retry those host-buffer allocations on regular pages. + +If the helper reports `baseline_fits_arena_may_fallback`, either increase `vm.nr_hugepages` or reduce `MC_MMAP_ARENA_POOL_SIZE`. For containerized launches, pass the same environment variables through `docker run -e ...` and use `--ipc=host --ulimit memlock=-1 --shm-size=128g`. + **More Information:** Additional troubleshooting information can be found [here](https://kvcache-ai.github.io/Mooncake/troubleshooting/troubleshooting.html). diff --git a/docs/source/getting_started/examples/sglang-integration/hicache-quick-start.md b/docs/source/getting_started/examples/sglang-integration/hicache-quick-start.md index e234349b..edd39a6a 100644 --- a/docs/source/getting_started/examples/sglang-integration/hicache-quick-start.md +++ b/docs/source/getting_started/examples/sglang-integration/hicache-quick-start.md @@ -10,21 +10,43 @@ Before you begin, make sure that: - SGLang is installed with HiCache support on the machine hosting your SGLang server. Refer to the [official installation guide](https://docs.sglang.ai/get_started/install.html) if needed. - Mooncake is installed and accessible as the hierarchical cache backend. Detailed build steps live in the [Mooncake documentation](https://kvcache-ai.github.io/Mooncake/getting_started/build.html). +- If you want HugeTLB-backed host KV memory, reserve the hugepages on the host before launching SGLang. The helper `python3 scripts/check_hicache_hugepage_requirements.py ...` from a source checkout, or `mooncake-hicache-sizing ...` inside the source-built Docker image, can size the required budget from your `--hicache-size`, `MOONCAKE_GLOBAL_SEGMENT_SIZE`, and `MC_MMAP_ARENA_POOL_SIZE` values. - The router package is installed to provide the `sglang_router` entrypoint: ```bash pip install sglang-router ``` -### 1. Launch the Mooncake master service +### 1. Size and reserve HugeTLB on the host + +```bash +python3 scripts/check_hicache_hugepage_requirements.py \ + --tp-size 4 \ + --hicache-size 64gb \ + --global-segment-size 8gb \ + --arena-pool-size 56gb \ + --available-hugetlb 512gb + +sudo sysctl -w vm.nr_hugepages=262144 +grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo +``` + +`262144` is an example for `512 GiB` of `2 MiB` pages. The `64gb` / `56gb` sizing inputs above are tuned examples for large multi-GPU runs, not defaults. Adjust them, and the resulting `vm.nr_hugepages`, to match your planned HiCache budget. On smaller hosts, start with an arena pool of `8gb` or `16gb`. + +### 2. Launch the Mooncake master service ```bash mooncake_master --enable_http_metadata_server=true ``` -### 2. Launch SGLang with Mooncake L3 storage +### 3. Launch SGLang with Mooncake L3 storage ```bash +export MC_STORE_USE_HUGEPAGE="1" +export MC_STORE_HUGEPAGE_SIZE="2MB" +export MOONCAKE_GLOBAL_SEGMENT_SIZE="8gb" +export MC_MMAP_ARENA_POOL_SIZE="56gb" + MOONCAKE_MASTER=127.0.0.1:50051 python -m sglang.launch_server \ --model-path [model_path] \ --page-size 64 \ @@ -35,6 +57,8 @@ MOONCAKE_MASTER=127.0.0.1:50051 python -m sglang.launch_server \ **Key flag:** `--hicache-storage-prefetch-policy {best_effort,wait_complete,timeout}` determines when prefetching from storage should stop. `timeout` usually offers the best balance when Mooncake is the backend. +**Memory allocator:** Mooncake's mmap arena is opt-in. The `56gb` example above is a benchmark-scale tuning value, not the default. Setting `MC_MMAP_ARENA_POOL_SIZE` enables it and chooses the pool size; if you enable it via gflag instead, the default pool size is `8gb`. To force the baseline direct-`mmap()` path, set `MC_DISABLE_MMAP_ARENA=1` (also accepts `true`, `yes`, or `on`) before the first Mooncake mmap-buffer allocation in the process. See the [Complete Guide](hicache-integration-v1.md) for the full HugeTLB sizing and troubleshooting flow. + ## Prefill/Decode Disaggregation The disaggregated setup runs three processes—prefill worker, decode worker, and router. Launch each command below in its own terminal window. @@ -94,6 +118,7 @@ curl -X POST http://127.0.0.1:8000/generate \ - `--disaggregation-ib-device` is optional—SGLang autodetects devices, but you can set it explicitly (comma-separated, no spaces) when multiple NICs are available. - Use `--tp-size` to enable tensor-parallel execution across GPUs if required. +- For Docker launches, use `--ipc=host`, `--ulimit memlock=-1`, and a large `--shm-size`, then pass the same `MC_STORE_USE_HUGEPAGE`, `MC_STORE_HUGEPAGE_SIZE`, `MOONCAKE_GLOBAL_SEGMENT_SIZE`, and `MC_MMAP_ARENA_POOL_SIZE` values through `docker run -e ...`. - Optional flags to experiment with once you need them (some still have compatibility gaps): - `--disaggregation-decode-enable-offload-kvcache` writes the decode worker's outputs back into Mooncake; enable it when you want decoded KV to persist in L3. - Launch dedicated Mooncake `store service` nodes when you want to scale L3 capacity beyond what the SGLang servers contribute. diff --git a/docs/source/performance/allocator-benchmark-result.md b/docs/source/performance/allocator-benchmark-result.md index 6a04323b..1c562266 100644 --- a/docs/source/performance/allocator-benchmark-result.md +++ b/docs/source/performance/allocator-benchmark-result.md @@ -1,5 +1,14 @@ # Allocator Performance +Mooncake Store uses two allocators at different layers: + +- **MmapArena** — backs the mmap buffer path used by SGLang HiCache for host KV cache memory. When explicitly enabled, it pre-allocates a hugepage-backed pool at startup and serves allocations via lock-free atomic bump pointer (~50ns per allocation). Configured via `MC_MMAP_ARENA_POOL_SIZE` (explicit opt-in) and `MC_DISABLE_MMAP_ARENA`. +- **OffsetAllocator** — manages object-level allocation within Mooncake Store segments for KV cache entries. Evaluated below. + +These are complementary: MmapArena handles the backing memory, OffsetAllocator manages logical object placement within that memory. + +## OffsetAllocator + We evaluated the performance of [OffsetAllocator](https://github.com/sebbbi/OffsetAllocator), the default memory allocator in Mooncake Store. This allocator is responsible for allocating memory from mounted segments to store the KV cache. In this context, the most important metric is **memory utilization**, defined as the ratio between the amount of memory that can be successfully allocated and the total available memory. A higher utilization means that more KV tensors can be cached, thereby accelerating LLM tasks. However, due to memory fragmentation, allocation may fail even when the allocated memory is well below the total available capacity. @@ -185,4 +194,4 @@ avg alloc time: 145.575738 ns/op util ratio (min / p99 / p90 / p50 / max / avg): 0.569255 / 0.712076 / 0.781224 / 0.855046 / 0.976057 / 0.848873 avg alloc time: 142.508508 ns/op -``` \ No newline at end of file +``` diff --git a/docs/source/troubleshooting/troubleshooting.md b/docs/source/troubleshooting/troubleshooting.md index 8b6d7eb9..3bf11c2e 100644 --- a/docs/source/troubleshooting/troubleshooting.md +++ b/docs/source/troubleshooting/troubleshooting.md @@ -11,7 +11,7 @@ This document lists common errors that may occur when using Mooncake Store and p ## Metadata and Out-of-Band Communication 1. At startup, a `TransferMetadata` object is constructed according to the incoming `metadata_server` parameter. During program execution, this object is used to communicate with the etcd server to maintain internal data required for connection. -2. At startup, the current node is registered with the cluster according to the incoming `connectable_name` parameter and `rpc_port` parameter, and the TCP port specified by the `rpc_port` parameter is listened. Before other nodes send the first read/write request to the current node, they will use the above information, resolve DNS, and initiate a connection through socket's `connect()` method. +2. At startup, the current node is registered with the cluster according to the incoming `connectable_name` parameter and `rpc_port` parameter, and the TCP port specified by the `rpc_port` parameter is listened. Before other nodes send the first read/write request to the current node, they will use the above information, resolve DNS, and initiate a connection through socket's `connect()` method. Errors in this part usually indicate that the error occurred within the `mooncake-transfer-engine/src/transfer_metadata.cpp` file. @@ -167,6 +167,31 @@ In addition, if the error `Failed to get description of XXX` is displayed, it in * **`asio::socket::close()` runs under `pool_mutex_`.** `close()` cancels outstanding async operations and posts completion handlers to the io_context. Handlers such as `returnConnection()` also acquire `pool_mutex_`, so the current code is one scheduling step away from a circular wait. No deadlock has been observed in practice, but if the transfer engine hangs with all worker threads stuck waiting on `pool_mutex_`, this is the first place to look. +## Memory Allocator + +Mooncake Store's mmap arena is opt-in for mmap buffer allocations. Setting `MC_MMAP_ARENA_POOL_SIZE` explicitly enables it and pre-allocates a hugepage-backed pool; if it is enabled via gflag instead, the default pool size is `8gb`. If the arena cannot allocate hugepages, it falls back to regular pages automatically. + +If you encounter memory allocation issues related to the arena: + +- **HugeTLB pool is too small for the launch:** Run `python3 scripts/check_hicache_hugepage_requirements.py ...` from a source checkout, or `mooncake-hicache-sizing ...` inside the Docker image, using your `--hicache-size`, `MOONCAKE_GLOBAL_SEGMENT_SIZE`, and `MC_MMAP_ARENA_POOL_SIZE` values. If it reports `insufficient_for_baseline`, increase `vm.nr_hugepages`, reduce `--hicache-size`, or shrink `MOONCAKE_GLOBAL_SEGMENT_SIZE`. +- **Arena pool too large for available hugepages:** Reduce the pool size with `MC_MMAP_ARENA_POOL_SIZE="8gb"` to fit within your hugepage budget, or leave `MC_MMAP_ARENA_POOL_SIZE` unset to stay on the baseline direct-`mmap()` path. +- **Arena only partially fits:** If the helper reports `baseline_fits_arena_may_fallback`, the baseline should start but some arena allocations may spill onto regular pages. Either increase `vm.nr_hugepages` or reduce `MC_MMAP_ARENA_POOL_SIZE`. +- **Need to disable the arena entirely:** Set `MC_DISABLE_MMAP_ARENA=1` (also accepts `true`, `yes`, or `on`) before the first Mooncake mmap-buffer allocation to fall back to per-call `mmap()`. +- **Arena init failed once and the process stayed on the baseline path:** Arena initialization is lazy and one-shot per process. After a failed first attempt, Mooncake keeps using direct `mmap()` until the process restarts. Fix the env / hugepage budget, then restart the process before retrying. +- **Arena OOM during serving:** The arena logs a warning and falls back to direct `mmap()` for that allocation. If this happens frequently, increase `MC_MMAP_ARENA_POOL_SIZE`. +- **Direct hugepage mmap fails immediately:** Verify the host really has hugepages reserved with `grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo`. When `MC_STORE_USE_HUGEPAGE=1` is set, direct `mmap()` allocations depend on the OS HugeTLB pool. + +To provision `2 MiB` hugepages on Linux: + +```bash +sudo sysctl -w vm.nr_hugepages=49152 +grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo +printf 'vm.nr_hugepages=49152\n' | sudo tee /etc/sysctl.d/90-mooncake-hugepages.conf +sudo sysctl --system +``` + +`49152` pages equals `96 GiB`; for `512 GiB` use `262144` pages. + ## SGLang Common Questions ### Do I need RDMA to run SGLang and Mooncake? diff --git a/mooncake-store/include/mmap_arena.h b/mooncake-store/include/mmap_arena.h new file mode 100644 index 00000000..b731efb1 --- /dev/null +++ b/mooncake-store/include/mmap_arena.h @@ -0,0 +1,108 @@ +// Copyright 2026 KVCache.AI +// Simple arena allocator for mmap-based memory allocation +// Used by SGLang HiCache for fast buffer allocation + +#pragma once + +#include +#include +#include +#include + +namespace mooncake { + +/** + * @brief Simple lock-free arena allocator for mmap'd memory + * + * Thread-safe: allocate() is lock-free (CAS), initialize() is mutex-guarded. + */ +class MmapArena { + public: + static constexpr size_t kMinAlignment = 64; + + struct Stats { + size_t pool_size; + size_t reserved_bytes; + size_t peak_reserved_bytes; + size_t num_allocations; + size_t num_failed_allocs; + }; + + MmapArena(); + ~MmapArena(); + + // Delete copy and move operations (class contains atomics) + MmapArena(const MmapArena&) = delete; + MmapArena& operator=(const MmapArena&) = delete; + MmapArena(MmapArena&&) = delete; + MmapArena& operator=(MmapArena&&) = delete; + + /** + * Initialize arena with a large mmap'd pool + * @param pool_size Total size to pre-allocate (aligned to huge pages) + * @param alignment Allocation alignment (default 64 bytes) + * @param allow_regular_page_fallback When true, the arena may retry the + * initial pool mmap on regular pages after a hugepage failure. + * Callers that need strict hugepage semantics should pass false. + * @return true on success + */ + bool initialize(size_t pool_size, size_t alignment = kMinAlignment, + bool allow_regular_page_fallback = true); + + /** + * Allocate memory from arena + * @param size Number of bytes to allocate + * @param alignment Per-call alignment override (0 = use arena default). + * Effective alignment is max(arena default, this parameter). + * @return Pointer to allocated memory, or nullptr if OOM + */ + void* allocate(size_t size, size_t alignment = 0); + + /** + * Get current arena statistics + */ + Stats getStats() const; + + /** + * Check if arena is initialized (thread-safe) + */ + bool isInitialized() const { + return pool_base_.load(std::memory_order_acquire) != nullptr; + } + + /** + * Check if pointer was allocated from this arena + * Used by free_buffer_mmap_memory to determine allocation type + * @param ptr Pointer to check + * @return true if ptr is in arena's address range + */ + bool owns(const void* ptr) const; + + /** + * Get pool base and size (for debugging/testing) + */ + void* getPoolBase() const { + return pool_base_.load(std::memory_order_acquire); + } + + size_t getPoolSize() const { + return pool_size_.load(std::memory_order_acquire); + } + + private: + std::atomic + pool_base_; // Base address of mmap'd pool (atomic for thread-safety) + std::atomic + pool_size_; // Total pool size (atomic for thread-safety) + std::atomic + alignment_; // Default allocation alignment (atomic for thread-safety) + + std::atomic alloc_cursor_; // Monotonic reservation cursor + std::atomic peak_reserved_; // Peak reservation cursor + std::atomic num_allocations_; // Total allocations + std::atomic num_failed_allocs_; // Failed allocations (OOM) + + std::mutex init_mutex_; // Guards initialize() against concurrent calls +}; + +} // namespace mooncake diff --git a/mooncake-store/include/utils.h b/mooncake-store/include/utils.h index 8951f96c..5bed64d9 100644 --- a/mooncake-store/include/utils.h +++ b/mooncake-store/include/utils.h @@ -1,7 +1,10 @@ #pragma once +#include +#include #include #include +#include #include #include #include @@ -239,6 +242,32 @@ std::string expected_to_str(const tl::expected& expected) { } } +/** + * @brief Convert a boolean-like string to a bool + * @param str String representation ("1"/"true"/"yes"/"on" or + * "0"/"false"/"no"/"off") + * @return std::optional Parsed value, or std::nullopt if parsing fails + */ +[[nodiscard]] inline std::optional string_to_bool(std::string str) { + if (str.empty()) { + return std::nullopt; + } + + str.erase(0, str.find_first_not_of(" \t\r\n")); + str.erase(str.find_last_not_of(" \t\r\n") + 1); + std::transform(str.begin(), str.end(), str.begin(), + [](unsigned char c) { return std::tolower(c); }); + + if (str == "1" || str == "true" || str == "yes" || str == "on") { + return true; + } + if (str == "0" || str == "false" || str == "no" || str == "off") { + return false; + } + + return std::nullopt; +} + /** * @brief Split a string by delimiter into a vector of strings * @param str The string to split @@ -256,6 +285,7 @@ std::vector splitString(const std::string& str, constexpr size_t SZ_2MB = 2 * 1024 * 1024; constexpr size_t SZ_1GB = 1024 * 1024 * 1024; +constexpr double BYTES_PER_GIB = static_cast(SZ_1GB); /** * @brief Allocates memory for the `BufferAllocator` class. @@ -287,9 +317,6 @@ inline size_t align_up(size_t size, size_t alignment) { return 0; } - constexpr size_t SZ_2MB = 2 * 1024 * 1024; - constexpr size_t SZ_1GB = 1024 * 1024 * 1024; - size_t size = SZ_2MB; // Default to 2MB const char* size_env = std::getenv("MC_STORE_HUGEPAGE_SIZE"); @@ -333,8 +360,33 @@ inline size_t align_up(size_t size, size_t alignment) { return size; } -// Hugepage-backed allocation helpers (MAP_HUGETLB + MADV_HUGEPAGE) +/** + * Allocate mmap-backed buffer memory for host KV / transfer buffers. + * + * When the global mmap arena is enabled, this function serves allocations + * from the arena and still honors the caller's requested alignment. + * Arena-owned allocations remain owned by the arena until process shutdown. + * + * When the arena is disabled or unavailable, this falls back to a direct + * mmap() allocation and returns a pointer aligned to at least the system page + * size (or the configured hugepage size when available). + * + * @param total_size Total buffer size in bytes. + * @param alignment Minimum alignment requested by the caller. + * @return Pointer to the allocation, or nullptr on failure. + */ void* allocate_buffer_mmap_memory(size_t total_size, size_t alignment); + +/** + * Release memory previously returned by allocate_buffer_mmap_memory(). + * + * Direct-mmap allocations are unmapped immediately. Arena-owned pointers are + * intentionally not unmapped individually; in that case this function is a + * no-op and the arena releases the backing pool during process teardown. + * + * @param ptr Pointer previously returned by allocate_buffer_mmap_memory(). + * @param total_size Original allocation size in bytes. + */ void free_buffer_mmap_memory(void* ptr, size_t total_size); /** diff --git a/mooncake-store/src/CMakeLists.txt b/mooncake-store/src/CMakeLists.txt index d9e0c2ff..15d4ca91 100644 --- a/mooncake-store/src/CMakeLists.txt +++ b/mooncake-store/src/CMakeLists.txt @@ -9,6 +9,7 @@ set(MOONCAKE_STORE_SOURCES types.cpp master_client.cpp utils.cpp + mmap_arena.cpp master_metric_manager.cpp storage_backend.cpp thread_pool.cpp @@ -143,25 +144,19 @@ add_library(mooncake_store ${MOONCAKE_STORE_SOURCES}) target_include_directories(mooncake_store PUBLIC ${XXHASH_INCLUDE_DIR}) target_link_libraries(mooncake_store PUBLIC ${XXHASH_LIBRARY}) if(STORE_USE_REDIS) - target_include_directories(mooncake_store PRIVATE - ${MOONCAKE_STORE_HIREDIS_INCLUDE_DIR}) + target_include_directories(mooncake_store + PRIVATE ${MOONCAKE_STORE_HIREDIS_INCLUDE_DIR}) endif() -# Note: transfer_engine is PRIVATE to avoid propagating its dependencies (e.g., vendor-specific hardware) -# to targets that don't need it (e.g., mooncake_master). -# Targets that need transfer_engine should link it explicitly. -target_link_libraries(mooncake_store - PUBLIC - cachelib_memory_allocator - ${ETCD_WRAPPER_LIB} - glog::glog - gflags::gflags - ${EXTRA_LIBS} - asio_shared - PRIVATE - transfer_engine -) +# Note: transfer_engine is PRIVATE to avoid propagating its dependencies (e.g., +# vendor-specific hardware) to targets that don't need it (e.g., +# mooncake_master). Targets that need transfer_engine should link it explicitly. +target_link_libraries( + mooncake_store + PUBLIC cachelib_memory_allocator ${ETCD_WRAPPER_LIB} glog::glog gflags::gflags + ${EXTRA_LIBS} asio_shared + PRIVATE transfer_engine) if(STORE_USE_ETCD) - add_dependencies(mooncake_store build_etcd_wrapper) + add_dependencies(mooncake_store build_etcd_wrapper) endif() if(URING_LIB AND URING_INCLUDE) diff --git a/mooncake-store/src/mmap_arena.cpp b/mooncake-store/src/mmap_arena.cpp new file mode 100644 index 00000000..a0fc41e2 --- /dev/null +++ b/mooncake-store/src/mmap_arena.cpp @@ -0,0 +1,274 @@ +// Copyright 2026 KVCache.AI +// Simple arena allocator implementation + +#include "mmap_arena.h" +#include "utils.h" +#include +#include +#include +#include +#include + +namespace mooncake { + +// Safe alignment with overflow detection +// Returns false if overflow would occur, true on success +static inline bool safe_align_up(size_t size, size_t alignment, + size_t* result) { + if (size == 0) { + *result = 0; + return true; + } + if (alignment == 0 || (alignment & (alignment - 1)) != 0) { + // Must be power of 2 + return false; + } + // Check if size + alignment - 1 would overflow + if (size > SIZE_MAX - alignment + 1) { + return false; // Overflow would occur + } + *result = (size + alignment - 1) & ~(alignment - 1); + return true; +} + +MmapArena::MmapArena() + : pool_base_(nullptr), + pool_size_(0), + alignment_(kMinAlignment), + alloc_cursor_(0), + peak_reserved_(0), + num_allocations_(0), + num_failed_allocs_(0) {} + +MmapArena::~MmapArena() { + void* pool_base = pool_base_.load(std::memory_order_acquire); + if (pool_base != nullptr) { + size_t pool_size = pool_size_.load(std::memory_order_acquire); + if (munmap(pool_base, pool_size) != 0) { + LOG(ERROR) << "Arena munmap failed: " << strerror(errno); + } + pool_base_.store(nullptr, std::memory_order_release); + } +} + +bool MmapArena::initialize(size_t pool_size, size_t alignment, + bool allow_regular_page_fallback) { + // Mutex serializes concurrent initialize() calls so that exactly one + // thread performs the mmap and publishes the pool. This avoids the + // metadata-overwrite race that existed with the old CAS approach + // (losing threads could clobber alignment_/pool_size_ before CAS). + std::lock_guard lock(init_mutex_); + + if (pool_size == 0) { + LOG(ERROR) << "Arena pool size must be > 0"; + return false; + } + + if (alignment != 0 && (alignment & (alignment - 1)) != 0) { + LOG(ERROR) << "Arena alignment must be a power of 2, got " << alignment; + return false; + } + + if (pool_base_.load(std::memory_order_acquire) != nullptr) { + LOG(WARNING) << "Arena already initialized"; + return false; + } + + const size_t actual_alignment = std::max(alignment, kMinAlignment); + + // Align pool size to 2MB for huge pages with overflow protection + size_t aligned_pool_size; + if (!safe_align_up(pool_size, SZ_2MB, &aligned_pool_size)) { + LOG(ERROR) << "Arena pool size overflow: requested=" << pool_size; + return false; + } + + // Allocate pool with mmap. Use MAP_POPULATE so that all huge pages + // are faulted upfront. Without it, lazy page-faults during GPU DMA + // (e.g. Mooncake transfer engine ↔ CUDA) cause cudaErrorIllegalAddress + // on some platforms (observed on H100 80GB). + int flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE; + +// Try huge pages for better TLB performance +#ifdef MAP_HUGETLB + flags |= MAP_HUGETLB; +#endif + + void* pool_base = + mmap(nullptr, aligned_pool_size, PROT_READ | PROT_WRITE, flags, -1, 0); + + if (pool_base == MAP_FAILED) { + const int mmap_errno = errno; + if (!allow_regular_page_fallback) { + LOG(ERROR) << "Arena hugepage mmap failed for pool_size=" + << aligned_pool_size << " bytes" + << ", errno=" << mmap_errno << " (" + << strerror(mmap_errno) + << "); regular-page fallback disabled because " + "MC_STORE_USE_HUGEPAGE explicitly requested " + "hugepages"; + return false; + } + + // Retry without huge pages + flags &= ~MAP_HUGETLB; + LOG(WARNING) << "Arena hugepage mmap failed for pool_size=" + << aligned_pool_size << " bytes" + << ", errno=" << mmap_errno << " (" << strerror(mmap_errno) + << "); retrying without huge pages"; + pool_base = mmap(nullptr, aligned_pool_size, PROT_READ | PROT_WRITE, + flags, -1, 0); + + if (pool_base == MAP_FAILED) { + LOG(ERROR) << "Arena mmap failed: size=" << aligned_pool_size + << ", errno=" << errno << " (" << strerror(errno) << ")"; + return false; + } + LOG(WARNING) << "Arena initialized without huge pages for pool_size=" + << aligned_pool_size + << " bytes; MAP_POPULATE will prefault regular pages"; + } else { + LOG(INFO) << "Arena initialized with huge pages"; + } + + // Prevent child processes from inheriting this mapping on fork(). + // With MAP_POPULATE, all pages are physically backed; fork() would + // create a massive copy-on-write mapping (up to 64GB) in the child, + // potentially doubling memory usage and triggering OOM killer. + // Child processes don't need the parent's DMA buffers — they exec() + // or initialize their own arena. + if (madvise(pool_base, aligned_pool_size, MADV_DONTFORK) != 0) { + LOG(WARNING) << "madvise(MADV_DONTFORK) failed: " << strerror(errno) + << " for pool_size=" << aligned_pool_size + << " bytes (fork safety not guaranteed)"; + } + + // Store metadata BEFORE publishing pool_base_. + // The release store on pool_base_ ensures these are visible to any + // thread that loads pool_base_ with acquire in allocate(). + alignment_.store(actual_alignment, std::memory_order_relaxed); + pool_size_.store(aligned_pool_size, std::memory_order_relaxed); + pool_base_.store(pool_base, std::memory_order_release); + + LOG(INFO) << "Arena initialized: " << (aligned_pool_size / BYTES_PER_GIB) + << " GiB, alignment=" << actual_alignment << " bytes"; + + return true; +} + +void* MmapArena::allocate(size_t size, size_t alignment) { + void* pool_base = pool_base_.load(std::memory_order_acquire); + if (pool_base == nullptr) { + LOG(ERROR) << "Arena not initialized"; + return nullptr; + } + + if (size == 0) { + return nullptr; + } + + // Effective alignment: max of arena default and caller's request. + // This honors the caller's alignment contract without weakening + // the arena's minimum guarantee. + // pool_base_ was acquired above; this relaxed load observes the + // alignment value published before that release-store in initialize(). + size_t base_alignment = alignment_.load(std::memory_order_relaxed); + size_t effective_alignment = std::max(base_alignment, alignment); + + // Align allocation size with overflow check + size_t aligned_size; + if (!safe_align_up(size, effective_alignment, &aligned_size)) { + num_failed_allocs_.fetch_add(1, std::memory_order_relaxed); + LOG(ERROR) << "Arena allocation size overflow: size=" << size + << ", alignment=" << effective_alignment; + return nullptr; + } + + size_t pool_size = pool_size_.load(std::memory_order_acquire); + + // CAS loop: Reserve aligned space atomically with bounds check. + // We align the OFFSET (not just the size) so the returned pointer + // honours the caller's alignment contract even when the cursor sits + // at a non-aligned position from a previous smaller-alignment alloc. + size_t aligned_offset; + size_t next; + while (true) { + size_t raw = alloc_cursor_.load(std::memory_order_relaxed); + + // Align the offset up to effective_alignment + if (!safe_align_up(raw, effective_alignment, &aligned_offset)) { + num_failed_allocs_.fetch_add(1, std::memory_order_relaxed); + LOG(ERROR) << "Arena offset alignment overflow: raw=" << raw + << ", alignment=" << effective_alignment; + return nullptr; + } + + next = aligned_offset + aligned_size; + + // Check for overflow (next wrapped) and OOM BEFORE modifying cursor + if (next < aligned_offset || next > pool_size) { + num_failed_allocs_.fetch_add(1, std::memory_order_relaxed); + LOG(ERROR) << "Arena OOM: requested=" << size + << ", aligned_size=" << aligned_size + << ", aligned_offset=" << aligned_offset + << ", pool_size=" << pool_size; + return nullptr; + } + + // Try to reserve [aligned_offset, next) atomically. + // CAS from raw (not aligned_offset) — another thread may have + // bumped the cursor since we loaded it. + if (alloc_cursor_.compare_exchange_weak(raw, next, + std::memory_order_relaxed, + std::memory_order_relaxed)) { + break; // Success - space reserved + } + // CAS failed, retry with new raw value + } + + // Space successfully reserved at [aligned_offset, next) + num_allocations_.fetch_add(1, std::memory_order_relaxed); + + // Update peak statistics using `next` (the actual end of reservation, + // including any alignment padding before aligned_offset) + size_t old_peak = peak_reserved_.load(std::memory_order_relaxed); + while (next > old_peak && !peak_reserved_.compare_exchange_weak( + old_peak, next, std::memory_order_relaxed, + std::memory_order_relaxed)) { + // CAS loop for peak tracking + } + + void* ptr = static_cast(pool_base) + aligned_offset; + + VLOG(2) << "[ARENA] Allocated: size=" << size + << ", aligned_size=" << aligned_size + << ", aligned_offset=" << aligned_offset << ", ptr=" << ptr + << ", utilization=" << (100.0 * next / pool_size) << "%"; + + return ptr; +} + +MmapArena::Stats MmapArena::getStats() const { + Stats stats; + stats.pool_size = pool_size_.load(std::memory_order_relaxed); + stats.reserved_bytes = alloc_cursor_.load(std::memory_order_relaxed); + stats.peak_reserved_bytes = peak_reserved_.load(std::memory_order_relaxed); + stats.num_allocations = num_allocations_.load(std::memory_order_relaxed); + stats.num_failed_allocs = + num_failed_allocs_.load(std::memory_order_relaxed); + return stats; +} + +bool MmapArena::owns(const void* ptr) const { + void* pool_base = pool_base_.load(std::memory_order_acquire); + if (!ptr || !pool_base) { + return false; + } + + size_t pool_size = pool_size_.load(std::memory_order_acquire); + uintptr_t addr = reinterpret_cast(ptr); + uintptr_t base = reinterpret_cast(pool_base); + return addr >= base && addr < base + pool_size; +} + +} // namespace mooncake diff --git a/mooncake-store/src/utils.cpp b/mooncake-store/src/utils.cpp index d7820e9c..70a8b209 100644 --- a/mooncake-store/src/utils.cpp +++ b/mooncake-store/src/utils.cpp @@ -1,6 +1,10 @@ #include "utils.h" +#include "mmap_arena.h" +#include "config.h" +#include "common.h" #include +#include #include #include #include @@ -11,6 +15,7 @@ #include #include +#include #include #include #include @@ -19,8 +24,21 @@ #include #include #include -#include "config.h" -#include "common.h" +#include + +// Feature flag to enable/disable arena allocator. Disabled by default so the +// library does not pre-map a large pool unless the operator opts in via gflag +// or an explicit MC_MMAP_ARENA_POOL_SIZE environment override. +DEFINE_bool(use_mmap_arena_allocator, false, + "Enable the lock-free mmap arena allocator for mmap buffers"); + +// Arena pool size (default 8GiB when explicitly enabled without an env +// override). +DEFINE_uint64(mmap_arena_pool_size, 8ULL * 1024 * 1024 * 1024, + "Arena allocator pool size in bytes"); +#ifdef USE_ASCEND_DIRECT +#include "acl/acl.h" +#endif #if defined(USE_ASCEND_DIRECT) || defined(USE_UBSHMEM) #include "ascend_allocator.h" #endif @@ -100,24 +118,150 @@ void *allocate_buffer_allocator_memory(size_t total_size, return aligned_alloc(alignment, total_size); } +// Global arena instance (lazy initialization) +static std::unique_ptr g_mmap_arena; +static std::once_flag g_arena_init_flag; +static std::atomic g_arena_oom_fallback_count{0}; +static std::atomic g_arena_noop_free_count{0}; + +static void initializeGlobalArena() { + const std::string env_pool_size = + GetEnvStringOr("MC_MMAP_ARENA_POOL_SIZE", ""); + // Allow env var to override the gflag (useful when loaded as .so from + // Python). An explicit pool-size env var is also treated as an opt-in, + // because pybind11 users cannot easily pass gflags. + const std::string env_disable = GetEnvStringOr("MC_DISABLE_MMAP_ARENA", ""); + const std::optional disable_override = string_to_bool(env_disable); + if (!env_disable.empty() && !disable_override.has_value()) { + LOG(WARNING) << "Ignoring invalid MC_DISABLE_MMAP_ARENA='" + << env_disable + << "'; accepted values: 1/0, true/false, yes/no, on/off"; + } + const bool arena_requested = + FLAGS_use_mmap_arena_allocator || !env_pool_size.empty(); + const bool arena_disabled = disable_override.value_or(false); + if (!arena_requested || arena_disabled) { + LOG(INFO) << "=== ARENA ALLOCATOR DISABLED ==="; + if (arena_disabled) { + LOG(INFO) << "MC_DISABLE_MMAP_ARENA=" << env_disable + << " forces direct mmap()"; + } else { + LOG(INFO) << "Arena is opt-in; set --use_mmap_arena_allocator or " + "MC_MMAP_ARENA_POOL_SIZE to enable it"; + } + return; + } + + g_mmap_arena = std::make_unique(); + // Keep arena init consistent with the direct-mmap path: + // MC_STORE_USE_HUGEPAGE=1 -> strict: hard fail if hugepage mmap fails + // unset -> permissive: try hugepages, retry on + // regular pages if HugeTLB is unavailable + // This preserves both pre-existing contracts and avoids surprising + // operators with a silent hugepage downgrade. + const bool hugepages_explicitly_requested = + get_hugepage_size_from_env() > 0; + + // Allow env var override since gflags cannot be set from Python (pybind11). + // Supports human-readable sizes via string_to_byte_size(): "20gb", "16GB", + // etc. + uint64_t arena_pool_size = FLAGS_mmap_arena_pool_size; + if (!env_pool_size.empty()) { + const uint64_t parsed = string_to_byte_size(env_pool_size); + if (parsed > 0) { + arena_pool_size = parsed; + LOG(INFO) << "MC_MMAP_ARENA_POOL_SIZE override: " << env_pool_size + << " (" << byte_size_to_string(arena_pool_size) << ")"; + } else { + LOG(WARNING) << "Invalid MC_MMAP_ARENA_POOL_SIZE='" << env_pool_size + << "', using default " + << byte_size_to_string(FLAGS_mmap_arena_pool_size); + } + } + + bool success = + g_mmap_arena->initialize(arena_pool_size, MmapArena::kMinAlignment, + !hugepages_explicitly_requested); + + if (success) { + auto stats = g_mmap_arena->getStats(); + LOG(INFO) << "=== ARENA ALLOCATOR ENABLED ==="; + LOG(INFO) << "Arena pool size: " << (stats.pool_size / BYTES_PER_GIB) + << " GiB"; + LOG(INFO) << "Using lock-free atomic bump allocation"; + } else { + LOG(ERROR) << "=== ARENA INITIALIZATION FAILED ==="; + LOG(ERROR) << "Falling back to traditional mmap()"; + if (hugepages_explicitly_requested) { + LOG(ERROR) << "MC_STORE_USE_HUGEPAGE is set, so the fallback path " + "will also require hugepages"; + } + LOG(ERROR) << "Arena initialization is only attempted once per " + "process; restart after fixing the environment if you " + "want to retry arena bring-up"; + g_mmap_arena.reset(); + } +} + +// Compute the mmap/munmap size for the fallback (non-arena) path. +// Used by both allocate_buffer_mmap_memory and free_buffer_mmap_memory +// so they agree on the mapping size. +static inline size_t mmap_map_size(size_t total_size, size_t hugepage_size) { + const size_t page_size = + hugepage_size > 0 ? hugepage_size : static_cast(getpagesize()); + return align_up(total_size, page_size); +} + void *allocate_buffer_mmap_memory(size_t total_size, size_t alignment) { if (total_size == 0) { - LOG(ERROR) << "Total size must be greater than 0 for hugepage mmap"; + LOG(ERROR) << "Total size must be greater than 0 for mmap"; return nullptr; } + // Initialize arena on first call + std::call_once(g_arena_init_flag, initializeGlobalArena); + + // Try arena allocation first (if enabled). + // Forward caller's alignment so the arena honors the contract. + if (g_mmap_arena && g_mmap_arena->isInitialized()) { + void *ptr = g_mmap_arena->allocate(total_size, alignment); + if (ptr != nullptr) { + VLOG(1) << "Allocated " << total_size << " bytes from arena at " + << ptr; + return ptr; + } + // Arena OOM, fall through to traditional mmap + const uint64_t fallback_count = + g_arena_oom_fallback_count.fetch_add(1, std::memory_order_relaxed) + + 1; + LOG_FIRST_N(WARNING, 3) + << "Arena OOM, falling back to mmap() for size=" << total_size + << " (count=" << fallback_count << ")" + << " (further warnings suppressed)"; + } + + // Traditional mmap allocation (fallback or arena disabled). unsigned int flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE; - const size_t effective_alignment = - std::max(alignment, get_hugepage_size_from_env(&flags)); - const size_t map_size = align_up(total_size, effective_alignment); + const size_t hugepage_size = get_hugepage_size_from_env(&flags); + const size_t map_size = mmap_map_size(total_size, hugepage_size); + const size_t guaranteed_alignment = + hugepage_size > 0 ? hugepage_size : static_cast(getpagesize()); + if (alignment > guaranteed_alignment) { + LOG_FIRST_N(WARNING, 3) + << "Fallback mmap cannot honor alignment=" << alignment + << " (guaranteed=" << guaranteed_alignment + << "); pointer may be under-aligned" + << " (further warnings suppressed)"; + } void *ptr = mmap(nullptr, map_size, PROT_READ | PROT_WRITE, flags, -1, 0); if (ptr == MAP_FAILED) { - LOG(ERROR) << "Hugepage mmap failed, size=" << map_size - << ", errno=" << errno << " (" << strerror(errno) << ")"; + LOG(ERROR) << "mmap failed, size=" << map_size << ", errno=" << errno + << " (" << strerror(errno) << ")"; return nullptr; } + VLOG(1) << "Allocated " << total_size << " bytes via mmap() at " << ptr; return ptr; } @@ -126,10 +270,30 @@ void free_buffer_mmap_memory(void *ptr, size_t total_size) { return; } - const size_t map_size = align_up(total_size, get_hugepage_size_from_env()); + // Check if pointer belongs to global arena + std::call_once(g_arena_init_flag, initializeGlobalArena); + + if (g_mmap_arena && g_mmap_arena->owns(ptr)) { + const uint64_t noop_free_count = + g_arena_noop_free_count.fetch_add(1, std::memory_order_relaxed) + 1; + LOG_FIRST_N(WARNING, 3) + << "free_buffer_mmap_memory() does not individually release " + "arena-owned pointer " + << ptr << "; the global arena releases its pool at process shutdown" + << " (count=" << noop_free_count << ")" + << " (further warnings suppressed)"; + return; + } + + // Direct mmap allocation - safe to unmap + const size_t map_size = + mmap_map_size(total_size, get_hugepage_size_from_env()); if (munmap(ptr, map_size) != 0) { LOG(ERROR) << "munmap hugepage failed, size=" << map_size << ", errno=" << errno << " (" << strerror(errno) << ")"; + } else { + VLOG(1) << "Freed direct mmap allocation at " << ptr + << ", size=" << map_size; } } diff --git a/mooncake-store/tests/CMakeLists.txt b/mooncake-store/tests/CMakeLists.txt index 5eb6e5ad..23f19b7d 100644 --- a/mooncake-store/tests/CMakeLists.txt +++ b/mooncake-store/tests/CMakeLists.txt @@ -60,11 +60,14 @@ add_store_test(ipv6_client_test ipv6_client_test.cpp) add_store_test(client_metrics_test client_metrics_test.cpp) add_store_test(ssd_metrics_test ssd_metrics_test.cpp) add_store_test(serializer_test serializer_test.cpp) -add_store_test(embedded_snapshot_catalog_store_test - ha/snapshot/catalog/backends/embedded/embedded_snapshot_catalog_store_test.cpp) +add_store_test( + embedded_snapshot_catalog_store_test + ha/snapshot/catalog/backends/embedded/embedded_snapshot_catalog_store_test.cpp +) add_store_test(zstd_util_test zstd_util_test.cpp) -add_store_test(local_file_snapshot_object_store_test - ha/snapshot/object/backends/local/local_file_snapshot_object_store_test.cpp) +add_store_test( + local_file_snapshot_object_store_test + ha/snapshot/object/backends/local/local_file_snapshot_object_store_test.cpp) add_store_test(file_util_test file_util_test.cpp) add_store_test(snapshot_child_process_test ha/snapshot/snapshot_child_process_test.cpp) @@ -79,34 +82,55 @@ add_store_test(task_executor_test task_executor_test.cpp) add_store_test(task_integration_test task_integration_test.cpp) add_store_test(dummy_client_get_buffer_test dummy_client_get_buffer_test.cpp) add_store_test(health_check_test health_check_test.cpp) +add_store_test(mmap_arena_test mmap_arena_test.cpp) +add_store_test(mmap_arena_fallback_test mmap_arena_fallback_test.cpp) add_subdirectory(e2e) add_executable(high_availability_test ha/leadership/high_availability_test.cpp) -target_include_directories(high_availability_test PRIVATE - ${CMAKE_CURRENT_SOURCE_DIR}) +target_include_directories(high_availability_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}) if(STORE_USE_REDIS) - target_sources(high_availability_test PRIVATE - ha/leadership/backends/redis/high_availability_redis_test.cpp) - target_include_directories(high_availability_test PRIVATE - ${MOONCAKE_STORE_HIREDIS_INCLUDE_DIR}) - target_link_libraries(high_availability_test PRIVATE - ${MOONCAKE_STORE_HIREDIS_LIBRARY}) + target_sources( + high_availability_test + PRIVATE ha/leadership/backends/redis/high_availability_redis_test.cpp) + target_include_directories(high_availability_test + PRIVATE ${MOONCAKE_STORE_HIREDIS_INCLUDE_DIR}) + target_link_libraries(high_availability_test + PRIVATE ${MOONCAKE_STORE_HIREDIS_LIBRARY}) - add_executable(redis_snapshot_catalog_store_test - ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store_test.cpp) - target_include_directories(redis_snapshot_catalog_store_test PRIVATE - ${CMAKE_CURRENT_SOURCE_DIR} - ${MOONCAKE_STORE_HIREDIS_INCLUDE_DIR}) - target_link_libraries(redis_snapshot_catalog_store_test - PUBLIC mooncake_store transfer_engine cachelib_memory_allocator - ${ETCD_WRAPPER_LIB} glog gflags ibverbs gtest pthread - PRIVATE ${MOONCAKE_STORE_HIREDIS_LIBRARY}) - add_test(NAME redis_snapshot_catalog_store_test - COMMAND redis_snapshot_catalog_store_test) + add_executable( + redis_snapshot_catalog_store_test + ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store_test.cpp) + target_include_directories( + redis_snapshot_catalog_store_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR} ${MOONCAKE_STORE_HIREDIS_INCLUDE_DIR}) + target_link_libraries( + redis_snapshot_catalog_store_test + PUBLIC mooncake_store + transfer_engine + cachelib_memory_allocator + ${ETCD_WRAPPER_LIB} + glog + gflags + ibverbs + gtest + pthread + PRIVATE ${MOONCAKE_STORE_HIREDIS_LIBRARY}) + add_test(NAME redis_snapshot_catalog_store_test + COMMAND redis_snapshot_catalog_store_test) endif() -target_link_libraries(high_availability_test PUBLIC mooncake_store transfer_engine cachelib_memory_allocator ${ETCD_WRAPPER_LIB} glog gtest gtest_main pthread) -if (STORE_USE_ETCD OR STORE_USE_REDIS) - add_test(NAME high_availability_test COMMAND high_availability_test) +target_link_libraries( + high_availability_test + PUBLIC mooncake_store + transfer_engine + cachelib_memory_allocator + ${ETCD_WRAPPER_LIB} + glog + gtest + gtest_main + pthread) +if(STORE_USE_ETCD OR STORE_USE_REDIS) + add_test(NAME high_availability_test COMMAND high_availability_test) endif() add_executable(stress_workload_test stress_workload_test.cpp) @@ -124,10 +148,8 @@ target_link_libraries( # HA tests add_ha_test(standby_state_machine_test ha/standby/standby_state_machine_test.cpp) -add_ha_test(ha_metric_manager_test - ha/standby/ha_metric_manager_test.cpp) -add_ha_test(hot_standby_service_test - ha/standby/hot_standby_service_test.cpp) +add_ha_test(ha_metric_manager_test ha/standby/ha_metric_manager_test.cpp) +add_ha_test(hot_standby_service_test ha/standby/hot_standby_service_test.cpp) add_ha_test(hot_standby_snapshot_bootstrap_test ha/standby/hot_standby_snapshot_bootstrap_test.cpp) add_ha_test(oplog_applier_test ha/oplog/oplog_applier_test.cpp) @@ -144,9 +166,8 @@ add_ha_test(catalog_backed_snapshot_provider_test if(STORE_USE_REDIS) foreach(target catalog_backed_snapshot_provider_test hot_standby_snapshot_bootstrap_test) - target_include_directories(${target} PRIVATE - ${MOONCAKE_STORE_HIREDIS_INCLUDE_DIR}) - target_link_libraries(${target} - PRIVATE ${MOONCAKE_STORE_HIREDIS_LIBRARY}) + target_include_directories(${target} + PRIVATE ${MOONCAKE_STORE_HIREDIS_INCLUDE_DIR}) + target_link_libraries(${target} PRIVATE ${MOONCAKE_STORE_HIREDIS_LIBRARY}) endforeach() endif() diff --git a/mooncake-store/tests/mmap_arena_fallback_test.cpp b/mooncake-store/tests/mmap_arena_fallback_test.cpp new file mode 100644 index 00000000..8e6289b1 --- /dev/null +++ b/mooncake-store/tests/mmap_arena_fallback_test.cpp @@ -0,0 +1,153 @@ +// Copyright 2026 KVCache.AI +// Fallback-path tests for the global mmap allocator wrapper. + +#include +#include + +#include +#include +#include +#include +#include + +#include "utils.h" + +namespace mooncake { + +namespace { + +bool HostHasReservedHugepages() { + std::ifstream meminfo("/proc/meminfo"); + std::string key; + size_t value = 0; + std::string unit; + while (meminfo >> key >> value >> unit) { + if (key == "HugePages_Total:") { + return value > 0; + } + } + return false; +} + +} // namespace + +class MmapArenaFallbackTest : public ::testing::Test { + protected: + void SetUp() override { + FLAGS_logtostderr = 1; + FLAGS_minloglevel = google::WARNING; + setenv("MC_DISABLE_MMAP_ARENA", "1", 1); + } + + void TearDown() override { + unsetenv("MC_DISABLE_MMAP_ARENA"); + unsetenv("MC_MMAP_ARENA_POOL_SIZE"); + unsetenv("MC_STORE_USE_HUGEPAGE"); + } +}; + +TEST_F(MmapArenaFallbackTest, ArenaInitFailureIsStickyForProcessLifetime) { + unsetenv("MC_DISABLE_MMAP_ARENA"); + unsetenv("MC_STORE_USE_HUGEPAGE"); + + // "infinite" parses to UINT64_MAX and deterministically trips the arena's + // overflow guard before any mmap attempt. + setenv("MC_MMAP_ARENA_POOL_SIZE", "infinite", 1); + FLAGS_minloglevel = google::INFO; + testing::internal::CaptureStderr(); + void* first_ptr = allocate_buffer_mmap_memory(64 * 1024, 64); + ASSERT_NE(first_ptr, nullptr); + free_buffer_mmap_memory(first_ptr, 64 * 1024); + const std::string first_logs = testing::internal::GetCapturedStderr(); + EXPECT_NE(first_logs.find("ARENA INITIALIZATION FAILED"), + std::string::npos); + EXPECT_NE(first_logs.find("only attempted once per process"), + std::string::npos); + + // Fix the env and try again in the same process. std::call_once should + // keep us on the fallback path without re-running initialization. + setenv("MC_MMAP_ARENA_POOL_SIZE", "2gb", 1); + testing::internal::CaptureStderr(); + void* second_ptr = allocate_buffer_mmap_memory(64 * 1024, 64); + ASSERT_NE(second_ptr, nullptr); + free_buffer_mmap_memory(second_ptr, 64 * 1024); + const std::string second_logs = testing::internal::GetCapturedStderr(); + EXPECT_EQ(second_logs.find("ARENA INITIALIZATION FAILED"), + std::string::npos); + EXPECT_EQ(second_logs.find("ARENA ALLOCATOR ENABLED"), std::string::npos); +} + +TEST_F(MmapArenaFallbackTest, + ExplicitHugepageRequestDoesNotSilentlyFallbackToRegularPages) { + if (HostHasReservedHugepages()) { + GTEST_SKIP() << "Host has reserved hugepages; this strict-fallback " + "test only proves itself on hugepage-free hosts"; + } + + unsetenv("MC_DISABLE_MMAP_ARENA"); + setenv("MC_STORE_USE_HUGEPAGE", "1", 1); + setenv("MC_MMAP_ARENA_POOL_SIZE", "2mb", 1); + + FLAGS_minloglevel = google::INFO; + testing::internal::CaptureStderr(); + void* ptr = allocate_buffer_mmap_memory(64 * 1024, 64); + const std::string logs = testing::internal::GetCapturedStderr(); + + EXPECT_EQ(ptr, nullptr) << logs; + EXPECT_EQ(logs.find("retrying without huge pages"), std::string::npos) + << logs; +} + +TEST_F(MmapArenaFallbackTest, HonorsPageAlignment) { + const size_t alloc_size = 64 * 1024; + constexpr size_t alignment = 64; + + void* ptr = allocate_buffer_mmap_memory(alloc_size, alignment); + ASSERT_NE(ptr, nullptr); + EXPECT_EQ(reinterpret_cast(ptr) % alignment, 0u); + + memset(ptr, 0xAB, alloc_size); + EXPECT_EQ(static_cast(ptr)[0], 0xAB); + EXPECT_EQ(static_cast(ptr)[alloc_size - 1], 0xAB); + + free_buffer_mmap_memory(ptr, alloc_size); +} + +TEST_F(MmapArenaFallbackTest, NoHugepagesAllocFree) { + unsetenv("MC_STORE_USE_HUGEPAGE"); + + const size_t alloc_size = 65000; + constexpr size_t alignment = 64; + + void* ptr = allocate_buffer_mmap_memory(alloc_size, alignment); + ASSERT_NE(ptr, nullptr); + EXPECT_EQ(reinterpret_cast(ptr) % 4096, 0u); + + memset(ptr, 0xCD, alloc_size); + EXPECT_EQ(static_cast(ptr)[0], 0xCD); + + free_buffer_mmap_memory(ptr, alloc_size); +} + +TEST_F(MmapArenaFallbackTest, AllocateFreeCycle) { + constexpr int kCycles = 8; + constexpr size_t alloc_size = 128 * 1024; + constexpr size_t alignment = 64; + + for (int i = 0; i < kCycles; ++i) { + void* ptr = allocate_buffer_mmap_memory(alloc_size, alignment); + ASSERT_NE(ptr, nullptr) << "Allocation failed on cycle " << i; + EXPECT_EQ(reinterpret_cast(ptr) % alignment, 0u); + memset(ptr, static_cast(i), alloc_size); + free_buffer_mmap_memory(ptr, alloc_size); + } +} + +} // namespace mooncake + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + google::InitGoogleLogging(argv[0]); + FLAGS_logtostderr = 1; + return RUN_ALL_TESTS(); +} diff --git a/mooncake-store/tests/mmap_arena_test.cpp b/mooncake-store/tests/mmap_arena_test.cpp new file mode 100644 index 00000000..05425f0b --- /dev/null +++ b/mooncake-store/tests/mmap_arena_test.cpp @@ -0,0 +1,751 @@ +// Copyright 2026 KVCache.AI +// Unit tests for MmapArena allocator - Production-grade security and +// correctness tests + +#include +#include +#include "mmap_arena.h" +#include "utils.h" +#include +#include +#include +#include +#include +#include +#include + +namespace mooncake { + +class MmapArenaTest : public ::testing::Test { + protected: + void SetUp() override { + FLAGS_logtostderr = 1; + FLAGS_minloglevel = google::WARNING; // Reduce log noise in tests + } +}; + +// ===== BASIC FUNCTIONALITY TESTS ===== + +TEST_F(MmapArenaTest, BasicInitialization) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); // 1MB pool + ASSERT_TRUE(arena.isInitialized()); + + auto stats = arena.getStats(); + EXPECT_EQ(stats.pool_size, 2 * 1024 * 1024); // Aligned to 2MB (huge page) + EXPECT_EQ(stats.reserved_bytes, 0); + EXPECT_EQ(stats.num_allocations, 0); + EXPECT_EQ(stats.num_failed_allocs, 0); +} + +TEST_F(MmapArenaTest, BasicAllocation) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + void* ptr = arena.allocate(1024); + ASSERT_NE(ptr, nullptr); + + auto stats = arena.getStats(); + EXPECT_EQ(stats.num_allocations, 1); + EXPECT_GE(stats.reserved_bytes, 1024); + EXPECT_LE(stats.reserved_bytes, 1024 + 64); // Accounting for alignment +} + +TEST_F(MmapArenaTest, AllocationAlignment) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024, 64)); + + for (size_t size : {1, 63, 64, 65, 100, 1000}) { + void* ptr = arena.allocate(size); + ASSERT_NE(ptr, nullptr) << "Failed to allocate size=" << size; + + // Verify 64-byte alignment + uintptr_t addr = reinterpret_cast(ptr); + EXPECT_EQ(addr % 64, 0) << "Pointer not aligned: " << ptr; + + // Verify memory is writable + std::memset(ptr, 0xAA, size); + } +} + +TEST_F(MmapArenaTest, ZeroSizeAllocation) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + void* ptr = arena.allocate(0); + EXPECT_EQ(ptr, nullptr); + + auto stats = arena.getStats(); + EXPECT_EQ(stats.num_allocations, 0); +} + +TEST_F(MmapArenaTest, UninitializedAllocation) { + MmapArena arena; + EXPECT_FALSE(arena.isInitialized()); + + void* ptr = arena.allocate(1024); + EXPECT_EQ(ptr, nullptr); +} + +TEST_F(MmapArenaTest, DoubleInitialization) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + EXPECT_FALSE(arena.initialize(1024 * 1024)); // Second init should fail + + // Arena should still work + void* ptr = arena.allocate(1024); + EXPECT_NE(ptr, nullptr); +} + +// ===== BUG #1: OOM CHECK RACE CONDITION (CRITICAL) ===== + +TEST_F(MmapArenaTest, OOMDoesNotCorruptCursor) { + MmapArena arena; + ASSERT_TRUE(arena.initialize( + 1024)); // Requested 1KB, but aligned to 2MB (huge page) + + auto stats_init = arena.getStats(); + size_t pool_size = + stats_init.pool_size; // Actual pool size after alignment + + // Fill the arena completely + std::vector ptrs; + while (true) { + void* ptr = arena.allocate(64); + if (ptr == nullptr) break; + ptrs.push_back(ptr); + } + + // CRITICAL: Cursor should be at or below pool_size, not corrupted + auto stats = arena.getStats(); + ASSERT_LE(stats.reserved_bytes, pool_size); + ASSERT_GT(stats.num_failed_allocs, 0); + + // Subsequent allocations should still fail gracefully, not crash + void* p_extra = arena.allocate(1); + EXPECT_EQ(p_extra, nullptr); + + // Verify cursor didn't go past pool_size + stats = arena.getStats(); + ASSERT_LE(stats.reserved_bytes, pool_size); +} + +TEST_F(MmapArenaTest, ConcurrentOOMStressTest) { + MmapArena arena; + const size_t requested_pool_size = 1024 * 1024; // 1MB requested + ASSERT_TRUE(arena.initialize(requested_pool_size)); + + auto stats_init = arena.getStats(); + size_t actual_pool_size = + stats_init.pool_size; // Actual pool after alignment + + std::atomic succeeded{0}; + std::atomic failed{0}; + + const int num_threads = 16; + // Allocate enough to guarantee OOM + const int allocs_per_thread = (actual_pool_size / (64 * num_threads)) + 100; + const size_t alloc_size = 64; + + std::vector threads; + for (int i = 0; i < num_threads; ++i) { + threads.emplace_back([&]() { + for (int j = 0; j < allocs_per_thread; ++j) { + void* ptr = arena.allocate(alloc_size); + if (ptr != nullptr) { + succeeded.fetch_add(1, std::memory_order_relaxed); + // Write pattern to verify no corruption + std::memset(ptr, 0xBB, alloc_size); + } else { + failed.fetch_add(1, std::memory_order_relaxed); + } + } + }); + } + + for (auto& t : threads) t.join(); + + auto stats = arena.getStats(); + + // Verify cursor didn't go beyond pool + ASSERT_LE(stats.reserved_bytes, actual_pool_size); + + // Some allocations should have succeeded + ASSERT_GT(succeeded.load(), 0); + + // Some should have failed (pool exhausted) + ASSERT_GT(failed.load(), 0); + + // Total attempts should match + EXPECT_EQ(succeeded.load() + failed.load(), + num_threads * allocs_per_thread); + + LOG(INFO) << "OOM stress test: " << succeeded.load() << " succeeded, " + << failed.load() << " failed, pool utilization: " + << (100.0 * stats.reserved_bytes / stats.pool_size) << "%"; +} + +// ===== BUG #2 & #3: INTEGER OVERFLOW TESTS ===== + +TEST_F(MmapArenaTest, IntegerOverflowInBoundsCheck) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + // Try to allocate SIZE_MAX (should fail, not wrap around) + void* ptr = arena.allocate(SIZE_MAX); + EXPECT_EQ(ptr, nullptr); + + auto stats = arena.getStats(); + EXPECT_EQ(stats.num_failed_allocs, 1); + EXPECT_EQ(stats.reserved_bytes, 0); +} + +TEST_F(MmapArenaTest, AlignmentOverflow) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + // Request size that would overflow during alignment + // SIZE_MAX - 10 + 64 - 1 = overflow + void* ptr = arena.allocate(SIZE_MAX - 10); + EXPECT_EQ(ptr, nullptr); + + auto stats = arena.getStats(); + EXPECT_EQ(stats.num_failed_allocs, 1); + EXPECT_EQ(stats.reserved_bytes, 0); +} + +TEST_F(MmapArenaTest, NearMaxSizeAllocation) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + // Try allocating near SIZE_MAX / 2 (should fail gracefully) + void* ptr = arena.allocate(SIZE_MAX / 2); + EXPECT_EQ(ptr, nullptr); + + auto stats = arena.getStats(); + EXPECT_EQ(stats.num_failed_allocs, 1); +} + +// ===== BUG #4: ARENA MEMORY CANNOT BE FREED ===== + +TEST_F(MmapArenaTest, OwnershipDetection) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + void* arena_ptr = arena.allocate(1024); + ASSERT_NE(arena_ptr, nullptr); + + // Arena should own its allocations + EXPECT_TRUE(arena.owns(arena_ptr)); + + // Null pointer is not owned + EXPECT_FALSE(arena.owns(nullptr)); + + // Pointer outside arena range should not be owned + char stack_var; + EXPECT_FALSE(arena.owns(&stack_var)); +} + +TEST_F(MmapArenaTest, OwnershipBoundaryTest) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + void* first_ptr = arena.allocate(64); + ASSERT_NE(first_ptr, nullptr); + + void* pool_base = arena.getPoolBase(); + size_t pool_size = arena.getPoolSize(); + + // Test boundaries + EXPECT_TRUE(arena.owns(pool_base)); // Start of pool + EXPECT_TRUE(arena.owns(static_cast(pool_base) + 100)); // Middle + EXPECT_FALSE(arena.owns(static_cast(pool_base) + + pool_size)); // Just past end + EXPECT_FALSE( + arena.owns(static_cast(pool_base) - 1)); // Just before start +} + +// ===== BUG #5: RACE CONDITION IN INITIALIZE ===== + +TEST_F(MmapArenaTest, ConcurrentInitialization) { + MmapArena arena; + + std::atomic init_success{0}; + std::atomic init_failure{0}; + + const int num_threads = 16; + std::vector threads; + + // Multiple threads racing to initialize + for (int i = 0; i < num_threads; ++i) { + threads.emplace_back([&]() { + if (arena.initialize(1024 * 1024)) { + init_success.fetch_add(1, std::memory_order_relaxed); + } else { + init_failure.fetch_add(1, std::memory_order_relaxed); + } + }); + } + + for (auto& t : threads) t.join(); + + // Exactly one initialization should succeed + EXPECT_EQ(init_success.load(), 1); + EXPECT_EQ(init_failure.load(), num_threads - 1); + + // Arena should be usable + EXPECT_TRUE(arena.isInitialized()); + void* ptr = arena.allocate(1024); + EXPECT_NE(ptr, nullptr); +} + +TEST_F(MmapArenaTest, AllocateAfterRacyInit) { + MmapArena arena; + + const int num_threads = 10; + std::vector threads; + std::vector ptrs(num_threads); + + // Concurrent init + allocate + for (int i = 0; i < num_threads; ++i) { + threads.emplace_back([&, i]() { + arena.initialize(1024 * 1024); + ptrs[i] = arena.allocate(1024); + }); + } + + for (auto& t : threads) t.join(); + + // All allocations should either succeed or fail consistently + int non_null = std::count_if(ptrs.begin(), ptrs.end(), + [](void* p) { return p != nullptr; }); + EXPECT_GT(non_null, 0); // At least some should succeed + + // Verify no duplicate pointers + std::set unique_ptrs(ptrs.begin(), ptrs.end()); + unique_ptrs.erase(nullptr); + EXPECT_EQ(unique_ptrs.size(), static_cast(non_null)); +} + +// ===== THREAD SAFETY & CONCURRENCY TESTS ===== + +TEST_F(MmapArenaTest, ConcurrentAllocations) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(64 * 1024 * 1024)); // 64MB + + const int num_threads = 8; + const int allocs_per_thread = 1000; + std::vector threads; + std::vector> all_pointers(num_threads); + + for (int i = 0; i < num_threads; ++i) { + threads.emplace_back([&, i]() { + for (int j = 0; j < allocs_per_thread; ++j) { + void* ptr = arena.allocate(1024); + if (ptr != nullptr) { + all_pointers[i].push_back(ptr); + // Write unique pattern + std::memset(ptr, 0xCC + i, 1024); + } + } + }); + } + + for (auto& t : threads) t.join(); + + // Verify no duplicate pointers (uniqueness) + std::set unique_ptrs; + for (const auto& vec : all_pointers) { + for (void* ptr : vec) { + auto [iter, inserted] = unique_ptrs.insert(ptr); + EXPECT_TRUE(inserted) << "Duplicate pointer detected: " << ptr; + } + } + + LOG(INFO) << "Concurrent allocations: " << unique_ptrs.size() + << " unique allocations from " << num_threads << " threads"; +} + +TEST_F(MmapArenaTest, StatsConsistencyUnderLoad) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(64 * 1024 * 1024)); + + std::atomic stop{false}; + std::atomic invariant_violations{0}; + + // Allocator threads + std::vector threads; + for (int i = 0; i < 8; ++i) { + threads.emplace_back([&]() { + while (!stop.load(std::memory_order_relaxed)) { + void* ptr = arena.allocate(128); + (void)ptr; + } + }); + } + + // Stats checker thread + threads.emplace_back([&]() { + for (int i = 0; i < 100; ++i) { + auto stats = arena.getStats(); + // Invariants that must always hold + if (stats.reserved_bytes > stats.pool_size) { + invariant_violations.fetch_add(1, std::memory_order_relaxed); + } + // Note: peak_reserved_bytes may temporarily lag behind + // reserved_bytes + // due to concurrent updates, so we don't check that invariant here + std::this_thread::sleep_for(std::chrono::milliseconds(10)); + } + stop.store(true, std::memory_order_relaxed); + }); + + for (auto& t : threads) t.join(); + + auto stats = arena.getStats(); + + // Critical invariant: cursor never exceeded pool size + EXPECT_EQ(invariant_violations.load(), 0); + EXPECT_LE(stats.reserved_bytes, stats.pool_size); + + // After all threads finish, peak should be >= final allocated + EXPECT_GE(stats.peak_reserved_bytes, stats.reserved_bytes); + + LOG(INFO) << "Stats consistency test: " << stats.num_allocations + << " allocations, " << stats.num_failed_allocs << " failures"; +} + +// ===== EDGE CASES & STRESS TESTS ===== + +TEST_F(MmapArenaTest, NearOOMAllocation) { + MmapArena arena; + const size_t pool_size = 4096; // Small pool + ASSERT_TRUE(arena.initialize(pool_size)); + + // Fill arena almost completely + std::vector ptrs; + while (true) { + void* ptr = arena.allocate(64); + if (ptr == nullptr) break; + ptrs.push_back(ptr); + } + + auto stats = arena.getStats(); + EXPECT_LE(stats.reserved_bytes, stats.pool_size); + EXPECT_GT(stats.num_failed_allocs, 0); + + LOG(INFO) << "Near-OOM test: " << ptrs.size() << " allocations, " + << stats.reserved_bytes << " / " << stats.pool_size + << " bytes used"; +} + +TEST_F(MmapArenaTest, MixedSizeAllocations) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(64 * 1024 * 1024)); + + std::vector sizes = {1, 16, 64, 256, 1024, 4096, 16384, 65536}; + std::vector ptrs; + + // Allocate various sizes + for (size_t size : sizes) { + for (int i = 0; i < 10; ++i) { + void* ptr = arena.allocate(size); + if (ptr != nullptr) { + ptrs.push_back(ptr); + std::memset(ptr, 0xDD, size); + } + } + } + + // Verify all pointers are valid and aligned + for (void* ptr : ptrs) { + EXPECT_NE(ptr, nullptr); + EXPECT_EQ(reinterpret_cast(ptr) % 64, 0); + EXPECT_TRUE(arena.owns(ptr)); + } + + LOG(INFO) << "Mixed-size test: " << ptrs.size() << " allocations"; +} + +TEST_F(MmapArenaTest, PeakAllocationTracking) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + void* p1 = arena.allocate(512); + (void)p1; + auto stats1 = arena.getStats(); + EXPECT_GE(stats1.peak_reserved_bytes, 512); + + void* p2 = arena.allocate(1024); + (void)p2; + auto stats2 = arena.getStats(); + EXPECT_GE(stats2.peak_reserved_bytes, stats1.peak_reserved_bytes); + EXPECT_GE(stats2.peak_reserved_bytes, 512 + 1024); + + LOG(INFO) << "Peak tracking: " << stats2.peak_reserved_bytes << " bytes"; +} + +TEST_F(MmapArenaTest, ReservedBytesRemainMonotonic) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(1024 * 1024)); + + size_t last_reserved_bytes = 0; + for (size_t size : {1UL, 63UL, 64UL, 65UL, 4096UL, 1024UL}) { + void* ptr = arena.allocate(size); + ASSERT_NE(ptr, nullptr); + + auto stats = arena.getStats(); + EXPECT_GE(stats.reserved_bytes, last_reserved_bytes); + EXPECT_GE(stats.peak_reserved_bytes, stats.reserved_bytes); + last_reserved_bytes = stats.reserved_bytes; + } +} + +// ===== MIXED ALIGNMENT TESTS ===== + +TEST_F(MmapArenaTest, MixedAlignmentSequence) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(64 * 1024 * 1024)); // 64MB + + // First allocation moves cursor to 64 (non-trivially aligned). + // This ensures the subsequent 2MB-aligned allocation actually + // exercises the offset-alignment logic rather than passing + // vacuously because cursor 0 is trivially aligned to everything. + void* p1 = arena.allocate(1, 64); + ASSERT_NE(p1, nullptr); + + const size_t TWO_MB = 2 * 1024 * 1024; + void* p2 = arena.allocate(4 * 1024 * 1024, TWO_MB); + ASSERT_NE(p2, nullptr); + + // The returned pointer MUST be 2MB-aligned + EXPECT_EQ(reinterpret_cast(p2) % TWO_MB, 0) + << "Pointer not 2MB-aligned: " << p2; + + // p2 must not overlap p1 + EXPECT_GT(reinterpret_cast(p2), reinterpret_cast(p1)); +} + +// ===== INPUT VALIDATION TESTS ===== + +TEST_F(MmapArenaTest, InitializeWithZeroSize) { + MmapArena arena; + EXPECT_FALSE(arena.initialize(0)); + EXPECT_FALSE(arena.isInitialized()); + + // Allocations should fail gracefully on uninitialized arena + void* ptr = arena.allocate(1024); + EXPECT_EQ(ptr, nullptr); +} + +TEST_F(MmapArenaTest, NonPowerOfTwoAlignment) { + MmapArena arena; + // alignment=100 is not a power of 2; should be rejected + EXPECT_FALSE(arena.initialize(1024 * 1024, 100)); + EXPECT_FALSE(arena.isInitialized()); +} + +TEST_F(MmapArenaTest, PowerOfTwoAlignmentsAccepted) { + // Test several valid power-of-2 alignments + for (size_t align : {64, 128, 256, 512, 4096}) { + MmapArena arena; + ASSERT_TRUE(arena.initialize(4 * 1024 * 1024, align)) + << "Failed to init with alignment=" << align; + void* ptr = arena.allocate(1024); + ASSERT_NE(ptr, nullptr); + EXPECT_EQ(reinterpret_cast(ptr) % align, 0) + << "Pointer not aligned to " << align; + } +} + +// ===== SIZING REGRESSION TEST ===== + +TEST_F(MmapArenaTest, ArenaSizingRegression) { + // Reproduces the benchmark failure: pool=16MB, allocate all of it, + // then any further allocation should OOM. + const size_t POOL = 16 * 1024 * 1024; // 16MB (scaled down from 16GB) + MmapArena arena; + ASSERT_TRUE(arena.initialize(POOL, 64)); + + // First: allocation consumes entire pool + void* p1 = arena.allocate(POOL, 64); + ASSERT_NE(p1, nullptr); + + // Second: any further allocation should OOM + void* p2 = arena.allocate(1024, 64); + EXPECT_EQ(p2, nullptr); + EXPECT_GE(arena.getStats().num_failed_allocs, 1); +} + +// ===== CONCURRENT INIT METADATA CONSISTENCY ===== + +TEST_F(MmapArenaTest, ConcurrentInitMetadataConsistency) { + // Verify that after racing inits, pool_size and alignment are consistent + // with the winning initialization parameters. + MmapArena arena; + + const int num_threads = 16; + std::vector threads; + + for (int i = 0; i < num_threads; ++i) { + threads.emplace_back([&]() { arena.initialize(4 * 1024 * 1024, 128); }); + } + for (auto& t : threads) t.join(); + + ASSERT_TRUE(arena.isInitialized()); + // Pool size should be 4MB (already 2MB-aligned, no rounding needed) + EXPECT_EQ(arena.getStats().pool_size, 4 * 1024 * 1024); + + // Verify allocation works and honors alignment + void* ptr = arena.allocate(256); + ASSERT_NE(ptr, nullptr); + EXPECT_EQ(reinterpret_cast(ptr) % 128, 0); +} + +// ===== MAP_POPULATE REGRESSION TESTS ===== +// These tests verify the fix for cudaErrorIllegalAddress caused by lazy +// hugepage faults during GPU DMA. The arena must pre-fault all pages +// at initialization time (MAP_POPULATE) so that every byte in the pool +// is backed by physical memory before any allocation is returned. + +TEST_F(MmapArenaTest, PagesArePhysicallyBackedAfterInit) { + // Verify that arena pages are resident in physical memory immediately + // after initialize() — i.e. MAP_POPULATE is working. + // Uses mincore() which reports per-page residency status. + MmapArena arena; + const size_t POOL = 4 * 1024 * 1024; // 4MB + ASSERT_TRUE(arena.initialize(POOL)); + + void* base = arena.getPoolBase(); + size_t pool_size = arena.getPoolSize(); + ASSERT_NE(base, nullptr); + ASSERT_GT(pool_size, 0); + + // mincore() works on the system page size (typically 4KB), not hugepages. + // Query the number of system pages covering the pool. + const size_t sys_page_size = sysconf(_SC_PAGESIZE); + size_t num_pages = (pool_size + sys_page_size - 1) / sys_page_size; + + std::vector vec(num_pages); + int ret = mincore(base, pool_size, vec.data()); + + if (ret == 0) { + // mincore succeeded — check that all pages are resident + size_t resident = 0; + for (size_t i = 0; i < num_pages; ++i) { + if (vec[i] & 1) ++resident; + } + // With MAP_POPULATE, all pages should be resident. + // Allow small tolerance for kernel behavior differences. + double pct = 100.0 * resident / num_pages; + EXPECT_GT(pct, 95.0) + << "Only " << pct + << "% of pages resident; MAP_POPULATE may not be working. " + << resident << "/" << num_pages << " pages."; + LOG(INFO) << "mincore: " << resident << "/" << num_pages + << " pages resident (" << pct << "%)"; + } else { + // mincore may fail on some kernels for MAP_HUGETLB regions. + // Fall back to verifying that we can read every byte without SIGSEGV. + LOG(WARNING) << "mincore() returned " << ret << " (errno=" << errno + << "), falling back to read-verification"; + // Read every page — if MAP_POPULATE didn't work, this would trigger + // page faults (which is fine for CPU but would crash GPU DMA). + volatile char sink = 0; + for (size_t off = 0; off < pool_size; off += sys_page_size) { + sink += static_cast(base)[off]; + } + (void)sink; + // If we get here without SIGSEGV, at least CPU access works. + // The real MAP_POPULATE guarantee is that DMA works too, which + // can only be tested with actual GPU hardware. + } +} + +TEST_F(MmapArenaTest, AllocatedMemoryIsImmediatelyReadableWritable) { + // Simulates the GPU DMA scenario: allocate a buffer and immediately + // read/write every byte. Without MAP_POPULATE, a lazy hugepage fault + // during DMA would crash. With MAP_POPULATE, all pages are pre-faulted. + MmapArena arena; + const size_t POOL = 8 * 1024 * 1024; // 8MB + ASSERT_TRUE(arena.initialize(POOL)); + + // Allocate a large buffer (simulates segment allocation) + const size_t BUF_SIZE = 4 * 1024 * 1024; // 4MB + void* ptr = arena.allocate(BUF_SIZE); + ASSERT_NE(ptr, nullptr); + + // Write a pattern to every byte — would trigger page faults if lazy + std::memset(ptr, 0xAB, BUF_SIZE); + + // Read it back — verify no corruption + auto* bytes = static_cast(ptr); + for (size_t i = 0; i < BUF_SIZE; i += 4096) { + EXPECT_EQ(bytes[i], 0xAB) << "Memory corruption at offset " << i; + } + + // Allocate a second buffer from remaining space + void* ptr2 = arena.allocate(BUF_SIZE); + if (ptr2 != nullptr) { + // Write different pattern + std::memset(ptr2, 0xCD, BUF_SIZE); + auto* bytes2 = static_cast(ptr2); + for (size_t i = 0; i < BUF_SIZE; i += 4096) { + EXPECT_EQ(bytes2[i], 0xCD) + << "Memory corruption in second buffer at offset " << i; + } + // Verify first buffer wasn't corrupted by second allocation + EXPECT_EQ(bytes[0], 0xAB) + << "First buffer corrupted after second allocation"; + } +} + +TEST_F(MmapArenaTest, FallbackMmapRetainsPopulate) { + // When huge pages are unavailable, the arena falls back to regular mmap. + // Verify that MAP_POPULATE is retained in the fallback path by confirming + // the allocated memory is immediately usable (same as above but may + // exercise the non-hugepage code path on machines without huge pages). + MmapArena arena; + const size_t POOL = 2 * 1024 * 1024; // 2MB — minimum hugepage unit + ASSERT_TRUE(arena.initialize(POOL)); + + void* ptr = arena.allocate(1024 * 1024); // 1MB + ASSERT_NE(ptr, nullptr); + + // Full read/write cycle + std::memset(ptr, 0xEF, 1024 * 1024); + auto* bytes = static_cast(ptr); + EXPECT_EQ(bytes[0], 0xEF); + EXPECT_EQ(bytes[1024 * 1024 - 1], 0xEF); + EXPECT_EQ(bytes[512 * 1024], 0xEF); // Middle +} + +// ===== FORK SAFETY TEST ===== + +TEST_F(MmapArenaTest, MadviseDontForkApplied) { + // Verify that the arena applies MADV_DONTFORK to prevent 64GB CoW + // page table duplication on fork(). madvise(MADV_DONTFORK) is + // idempotent — calling it again on an already-marked region returns 0. + MmapArena arena; + const size_t POOL = 4 * 1024 * 1024; // 4MB + ASSERT_TRUE(arena.initialize(POOL)); + + void* base = arena.getPoolBase(); + size_t pool_size = arena.getPoolSize(); + ASSERT_NE(base, nullptr); + + // If MADV_DONTFORK was already applied by initialize(), this is a no-op + // and returns 0. If it wasn't applied, this also returns 0 (first apply). + // Either way, verify madvise succeeds on the pool region. + int ret = madvise(base, pool_size, MADV_DONTFORK); + EXPECT_EQ(ret, 0) << "madvise(MADV_DONTFORK) failed: " << strerror(errno); +} + +} // namespace mooncake + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + google::InitGoogleLogging(argv[0]); + FLAGS_logtostderr = 1; + return RUN_ALL_TESTS(); +} diff --git a/mooncake-store/tests/utils_test.cpp b/mooncake-store/tests/utils_test.cpp index 55a02b98..32f703f4 100644 --- a/mooncake-store/tests/utils_test.cpp +++ b/mooncake-store/tests/utils_test.cpp @@ -23,6 +23,19 @@ TEST(UtilsTest, ByteSizeToString) { EXPECT_EQ(byte_size_to_string(15 * 1024 * 1024 + 44048), "15.04 MB"); } +TEST(UtilsTest, StringToBool) { + EXPECT_EQ(string_to_bool("1"), true); + EXPECT_EQ(string_to_bool("true"), true); + EXPECT_EQ(string_to_bool("YES"), true); + EXPECT_EQ(string_to_bool(" on "), true); + EXPECT_EQ(string_to_bool("0"), false); + EXPECT_EQ(string_to_bool("false"), false); + EXPECT_EQ(string_to_bool("No"), false); + EXPECT_EQ(string_to_bool(" off "), false); + EXPECT_EQ(string_to_bool("maybe"), std::nullopt); + EXPECT_EQ(string_to_bool(""), std::nullopt); +} + TEST(UtilsTest, IsPortAvailable) { // Find an available port int test_port = -1; diff --git a/scripts/check_hicache_hugepage_requirements.py b/scripts/check_hicache_hugepage_requirements.py new file mode 100644 index 00000000..9080d1fe --- /dev/null +++ b/scripts/check_hicache_hugepage_requirements.py @@ -0,0 +1,236 @@ +#!/usr/bin/env python3 +"""Estimate HugeTLB capacity for Mooncake HiCache launches.""" + +from __future__ import annotations + +import argparse +from dataclasses import dataclass +from pathlib import Path +import re +import sys + + +_UNIT_MAP = { + "b": 1, + "k": 1024, + "kb": 1024, + "kib": 1024, + "m": 1024**2, + "mb": 1024**2, + "mib": 1024**2, + "g": 1024**3, + "gb": 1024**3, + "gib": 1024**3, + "t": 1024**4, + "tb": 1024**4, + "tib": 1024**4, +} + + +@dataclass(frozen=True) +class BudgetSummary: + baseline_per_rank_bytes: int + baseline_total_bytes: int + clean_arena_per_rank_bytes: int + clean_arena_total_bytes: int + available_bytes: int | None + status: str + exit_code: int + + +def parse_size(value: str) -> int: + normalized = value.strip().lower().replace("_", "") + match = re.fullmatch(r"([0-9]+(?:\.[0-9]+)?)([a-z]*)", normalized) + if match is None: + raise ValueError(f"invalid size: {value!r}") + + number = float(match.group(1)) + unit = match.group(2) or "b" + if unit not in _UNIT_MAP: + raise ValueError(f"unsupported size unit in {value!r}") + + return int(number * _UNIT_MAP[unit]) + + +def format_size(num_bytes: int) -> str: + for unit, scale in (("TiB", 1024**4), ("GiB", 1024**3), ("MiB", 1024**2)): + if num_bytes >= scale: + return f"{num_bytes / scale:.2f} {unit}" + if num_bytes >= 1024: + return f"{num_bytes / 1024:.2f} KiB" + return f"{num_bytes} B" + + +def read_available_hugetlb(page_size_bytes: int) -> int | None: + if page_size_bytes % 1024 != 0: + return None + + page_size_kib = page_size_bytes // 1024 + path = Path(f"/sys/kernel/mm/hugepages/hugepages-{page_size_kib}kB/nr_hugepages") + try: + nr_hugepages = int(path.read_text(encoding="utf-8").strip()) + except (FileNotFoundError, PermissionError, ValueError, OSError): + return None + return nr_hugepages * page_size_bytes + + +def evaluate_budget( + *, + tp_size: int, + hicache_size_bytes: int, + global_segment_size_bytes: int, + arena_pool_size_bytes: int, + available_bytes: int | None, +) -> BudgetSummary: + if tp_size <= 0: + raise ValueError("tp_size must be positive") + + baseline_per_rank = hicache_size_bytes + global_segment_size_bytes + baseline_total = baseline_per_rank * tp_size + clean_arena_per_rank = baseline_per_rank + arena_pool_size_bytes + clean_arena_total = clean_arena_per_rank * tp_size + + if available_bytes is None: + status = "planning_only" + exit_code = 0 + elif available_bytes < baseline_total: + status = "insufficient_for_baseline" + exit_code = 2 + elif available_bytes < clean_arena_total: + status = "baseline_fits_arena_may_fallback" + exit_code = 1 + else: + status = "clean_arena_budget_available" + exit_code = 0 + + return BudgetSummary( + baseline_per_rank_bytes=baseline_per_rank, + baseline_total_bytes=baseline_total, + clean_arena_per_rank_bytes=clean_arena_per_rank, + clean_arena_total_bytes=clean_arena_total, + available_bytes=available_bytes, + status=status, + exit_code=exit_code, + ) + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + description=( + "Estimate HugeTLB capacity needed for a Mooncake HiCache launch. " + "The tool reports both a baseline floor and a cleaner arena-backed " + "target derived from benchmark bring-up experience." + ) + ) + parser.add_argument("--tp-size", type=int, required=True) + parser.add_argument("--hicache-size", required=True) + parser.add_argument("--global-segment-size", required=True) + parser.add_argument("--arena-pool-size", default="0") + parser.add_argument("--hugepage-size", default="2mb") + parser.add_argument( + "--available-hugetlb", + help=( + "Override detected HugeTLB capacity with an explicit size such as " + "'96gb'. If omitted, the tool tries /sys/kernel/mm/hugepages." + ), + ) + return parser + + +def print_summary(args: argparse.Namespace, summary: BudgetSummary) -> None: + hicache_size = parse_size(args.hicache_size) + global_segment_size = parse_size(args.global_segment_size) + arena_pool_size = parse_size(args.arena_pool_size) + hugepage_size = parse_size(args.hugepage_size) + + print("Mooncake HiCache HugeTLB sizing summary") + print("") + print(f"tp-size: {args.tp_size}") + print(f"hicache-size: {args.hicache_size} ({format_size(hicache_size)})") + print( + "global-segment-size: " + f"{args.global_segment_size} ({format_size(global_segment_size)})" + ) + print( + f"arena-pool-size: {args.arena_pool_size} " + f"({format_size(arena_pool_size)})" + ) + print( + f"hugepage-size: {args.hugepage_size} " + f"({format_size(hugepage_size)})" + ) + if summary.available_bytes is None: + print("available HugeTLB: unavailable (planning-only mode)") + else: + print("available HugeTLB: " f"{format_size(summary.available_bytes)}") + print("") + print("baseline floor / rank: " f"{format_size(summary.baseline_per_rank_bytes)}") + print("baseline floor / total: " f"{format_size(summary.baseline_total_bytes)}") + print( + "clean arena / rank: " f"{format_size(summary.clean_arena_per_rank_bytes)}" + ) + print("clean arena / total: " f"{format_size(summary.clean_arena_total_bytes)}") + print("") + + if summary.status == "planning_only": + print("Status: planning only") + print( + "Meaning: baseline and clean-arena targets were computed, but " + "host HugeTLB capacity was not detected." + ) + elif summary.status == "insufficient_for_baseline": + print("Status: insufficient for baseline") + print( + "Meaning: the current HugeTLB pool is below the baseline floor. " + "Expect startup or allocation failures." + ) + elif summary.status == "baseline_fits_arena_may_fallback": + print("Status: baseline fits, arena may fall back") + print( + "Meaning: baseline should fit, but the cleaner fully arena-backed " + "target is above the current HugeTLB pool. Arena allocations may " + "partially or fully fall back to regular pages." + ) + else: + print("Status: clean arena budget available") + print( + "Meaning: the current HugeTLB pool meets the cleaner target for " + "baseline plus arena-backed bring-up." + ) + + print("") + print( + "Note: the baseline floor is a conservative bring-up estimate " + "(hicache + global segment per rank). The clean-arena target adds the " + "arena pool per rank to show when arena-backed runs are less likely to " + "spill onto the regular-page fallback path." + ) + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + + try: + available_bytes = ( + parse_size(args.available_hugetlb) + if args.available_hugetlb + else read_available_hugetlb(parse_size(args.hugepage_size)) + ) + summary = evaluate_budget( + tp_size=args.tp_size, + hicache_size_bytes=parse_size(args.hicache_size), + global_segment_size_bytes=parse_size(args.global_segment_size), + arena_pool_size_bytes=parse_size(args.arena_pool_size), + available_bytes=available_bytes, + ) + except ValueError as exc: + print(f"error: {exc}", file=sys.stderr) + return 64 + + print_summary(args, summary) + return summary.exit_code + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/test_hicache_hugepage_requirements.py b/scripts/test_hicache_hugepage_requirements.py new file mode 100644 index 00000000..6de2c74e --- /dev/null +++ b/scripts/test_hicache_hugepage_requirements.py @@ -0,0 +1,73 @@ +#!/usr/bin/env python3 +"""Unit tests for the HugeTLB sizing helper.""" + +from __future__ import annotations + +from pathlib import Path +import sys +import unittest + + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +import check_hicache_hugepage_requirements as helper # noqa: E402 + + +class ParseSizeTest(unittest.TestCase): + def test_parses_human_readable_units(self) -> None: + self.assertEqual(helper.parse_size("8gb"), 8 * 1024**3) + self.assertEqual(helper.parse_size("2MB"), 2 * 1024**2) + self.assertEqual(helper.parse_size("1.5tb"), int(1.5 * 1024**4)) + + def test_rejects_invalid_values(self) -> None: + with self.assertRaises(ValueError): + helper.parse_size("eight-gigabytes") + + +class EvaluateBudgetTest(unittest.TestCase): + def test_detects_baseline_failure(self) -> None: + summary = helper.evaluate_budget( + tp_size=1, + hicache_size_bytes=80 * 1024**3, + global_segment_size_bytes=48 * 1024**3, + arena_pool_size_bytes=0, + available_bytes=96 * 1024**3, + ) + self.assertEqual(summary.status, "insufficient_for_baseline") + self.assertEqual(summary.exit_code, 2) + + def test_detects_partial_arena_budget(self) -> None: + summary = helper.evaluate_budget( + tp_size=4, + hicache_size_bytes=64 * 1024**3, + global_segment_size_bytes=8 * 1024**3, + arena_pool_size_bytes=56 * 1024**3, + available_bytes=320 * 1024**3, + ) + self.assertEqual(summary.status, "baseline_fits_arena_may_fallback") + self.assertEqual(summary.exit_code, 1) + + def test_detects_clean_arena_budget(self) -> None: + summary = helper.evaluate_budget( + tp_size=4, + hicache_size_bytes=64 * 1024**3, + global_segment_size_bytes=8 * 1024**3, + arena_pool_size_bytes=56 * 1024**3, + available_bytes=512 * 1024**3, + ) + self.assertEqual(summary.status, "clean_arena_budget_available") + self.assertEqual(summary.exit_code, 0) + + def test_supports_planning_only_mode(self) -> None: + summary = helper.evaluate_budget( + tp_size=2, + hicache_size_bytes=56 * 1024**3, + global_segment_size_bytes=8 * 1024**3, + arena_pool_size_bytes=56 * 1024**3, + available_bytes=None, + ) + self.assertEqual(summary.status, "planning_only") + self.assertEqual(summary.exit_code, 0) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/tone_tests/python/test_hicache_storage_mooncake_backend.py b/scripts/tone_tests/python/test_hicache_storage_mooncake_backend.py index 4cc35fe8..e3d751c3 100644 --- a/scripts/tone_tests/python/test_hicache_storage_mooncake_backend.py +++ b/scripts/tone_tests/python/test_hicache_storage_mooncake_backend.py @@ -258,6 +258,9 @@ class TestMooncakeBackendMLAModel( server_args, env_vars = super()._get_additional_server_args_and_env() server_args["--hicache-mem-layout"] = "page_first" server_args["--tp-size"] = 2 + # Keep arena coverage explicit in the self-hosted integration suite. + env_vars.pop("MC_DISABLE_MMAP_ARENA", None) + env_vars["MC_MMAP_ARENA_POOL_SIZE"] = "8gb" return server_args, env_vars @@ -274,6 +277,10 @@ class TestMooncakeBackendAccuracy( server_args["--tp-size"] = 2 server_args["--hicache-mem-layout"] = "page_first_direct" server_args["--hicache-io-backend"] = "direct" + # Explicitly cover the allocator-disabled fallback path as part of the + # self-hosted HiCache integration suite. + env_vars.pop("MC_MMAP_ARENA_POOL_SIZE", None) + env_vars["MC_DISABLE_MMAP_ARENA"] = "1" return server_args, env_vars def test_eval_accuracy(self): -- 2.34.1 From dd9f8ae004f2ed55c685ff7685667a2797db5464 Mon Sep 17 00:00:00 2001 From: Dmitry Barsukoff Date: Mon, 4 May 2026 19:13:01 +0300 Subject: [PATCH 033/382] [Store] Allow customizing client port range (#2008) * [ADD] MC_STORE_CLIENT_MIN_PORT and MC_STORE_CLIENT_MAX_PORT environ * [UPD] documentation --------- Co-authored-by: Copilot --- .../mooncake-store-deployment-guide.md | 2 + docs/source/design/mooncake-store.md | 2 + docs/source/zh_archive/mooncake-store.md | 2 + mooncake-store/src/real_client.cpp | 8 +++- mooncake-transfer-engine/include/config.h | 6 +++ mooncake-transfer-engine/src/config.cpp | 45 ++++++++++++------- 6 files changed, 48 insertions(+), 17 deletions(-) diff --git a/docs/source/deployment/mooncake-store-deployment-guide.md b/docs/source/deployment/mooncake-store-deployment-guide.md index b15df264..3923217b 100644 --- a/docs/source/deployment/mooncake-store-deployment-guide.md +++ b/docs/source/deployment/mooncake-store-deployment-guide.md @@ -140,6 +140,8 @@ curl -s http://:9003/metrics/summary - Client metrics (enabled by default) - `MC_STORE_CLIENT_METRIC` (default `1`): Client-side metrics on by default; set `0` to disable entirely. - `MC_STORE_CLIENT_METRIC_INTERVAL` (default `0`): Reporting interval in seconds; `0` collects but does not periodically report. + - `MC_STORE_CLIENT_MIN_PORT` (default `12300`): Minimum local port for client connections. Must be in range 1024–32767 or 61000–65535 (well-known and ephemeral ports are excluded). Falls back to default on invalid input. + - `MC_STORE_CLIENT_MAX_PORT` (default `14300`): Maximum local port for client connections. Same range constraints as `MC_STORE_CLIENT_MIN_PORT`; must be ≥ `MC_STORE_CLIENT_MIN_PORT`. - Local memcpy optimization (Store transfer path) - `MC_STORE_MEMCPY` (default `0`/false): Set to `1` to prefer local memcpy when source/destination are on the same client. diff --git a/docs/source/design/mooncake-store.md b/docs/source/design/mooncake-store.md index acaa7e38..f920303f 100644 --- a/docs/source/design/mooncake-store.md +++ b/docs/source/design/mooncake-store.md @@ -835,6 +835,8 @@ The HTTP metadata server can be configured using the following parameters: - MC_STORE_MEMCPY: Enables or disables local memcpy optimization, set to 1/true to enable, 0/false to disable. - MC_STORE_CLIENT_METRIC: Enables client metric reporting, enabled by default; set to 0/false to disable. - MC_STORE_CLIENT_METRIC_INTERVAL: Reporting interval in seconds, default 0 (collects but does not report). +- MC_STORE_CLIENT_MIN_PORT: Minimum local port for client connections (default 12300). Must be in range 1024–32767 or 61000–65535; falls back to default on invalid input. +- MC_STORE_CLIENT_MAX_PORT: Maximum local port for client connections (default 14300). Same range constraints; must be ≥ MC_STORE_CLIENT_MIN_PORT. - MC_STORE_USE_HUGEPAGE: Enables huge page support, disabled by default. - MC_STORE_HUGEPAGE_SIZE: Specifies the page size of the huge page to use, default 2M. - MC_MMAP_ARENA_POOL_SIZE: Size of the pre-allocated arena pool for mmap buffer allocations. Accepts human-readable sizes (e.g., `"8gb"`, `"20gb"`). Providing this variable explicitly enables the arena; when enabled via gflag without an env override, the default pool size is `8gb`. The arena is allocated once at first use and serves subsequent allocations via lock-free atomic bump pointer (~50ns per allocation vs ~1000ns for direct mmap). diff --git a/docs/source/zh_archive/mooncake-store.md b/docs/source/zh_archive/mooncake-store.md index 3ddf3d83..eac15bfd 100644 --- a/docs/source/zh_archive/mooncake-store.md +++ b/docs/source/zh_archive/mooncake-store.md @@ -722,6 +722,8 @@ HTTP 元数据服务器可通过以下参数进行配置: - **MC_STORE_MEMCPY**: 控制是否启用本地 memcpy 优化, 1/true 启用, 0/false 禁用 - **MC_STORE_CLIENT_METRIC**: 启用客户端指标上报, 默认启用;设为 0/false 禁用 - **MC_STORE_CLIENT_METRIC_INTERVAL**: 指标上报间隔(秒), 默认 0(仅收集不上报) +- **MC_STORE_CLIENT_MIN_PORT**: 客户端使用的最小端口号, 默认 12300。端口范围必须在 1024–32767 或 61000–65535 之间(排除知名端口和临时端口)。无效值将回退为默认值。 +- **MC_STORE_CLIENT_MAX_PORT**: 客户端使用的最大端口号, 默认 14300。范围约束同上;必须 ≥ MC_STORE_CLIENT_MIN_PORT。 - **MC_STORE_USE_HUGEPAGE**: 启用 hugepage 优化, 默认禁用, 设置为 1/true 启用 - **MC_STORE_HUGEPAGE_SIZE**: hugepage 页大小, 默认 2M diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 239e126b..188cb8af 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -605,11 +605,17 @@ tl::expected RealClient::setup_internal( // Auto port binding with retry on metadata registration failure const int kMaxRetries = GetEnvOr("MC_STORE_CLIENT_SETUP_RETRIES", 20); + const int rawMinPort = GetEnvOr("MC_STORE_CLIENT_MIN_PORT", 12300); + const int rawMaxPort = GetEnvOr("MC_STORE_CLIENT_MAX_PORT", 14300); + constexpr int kDefaultMinPort = 12300; + constexpr int kDefaultMaxPort = 14300; + auto [minPort, maxPort] = ValidatePortRange( + rawMinPort, rawMaxPort, kDefaultMinPort, kDefaultMaxPort); bool success = false; for (int retry = 0; retry < kMaxRetries; ++retry) { // Create port binder to hold a port - port_binder_ = std::make_unique(); + port_binder_ = std::make_unique(minPort, maxPort); int port = port_binder_->getPort(); if (port < 0) { LOG(WARNING) << "Failed to bind available port, retry " diff --git a/mooncake-transfer-engine/include/config.h b/mooncake-transfer-engine/include/config.h index 5761209a..8ef7092b 100644 --- a/mooncake-transfer-engine/include/config.h +++ b/mooncake-transfer-engine/include/config.h @@ -22,6 +22,7 @@ #include #include #include +#include namespace mooncake { @@ -92,6 +93,11 @@ GlobalConfig& globalConfig(); uint16_t getDefaultHandshakePort(); +// Validates a port range. Returns {default_min, default_max} on invalid input. +// Rejects: min > max, well-known ports (0-1023), ephemeral ports (32768-60999). +std::pair ValidatePortRange(int min_port, int max_port, + int default_min, int default_max); + } // namespace mooncake #endif // CONFIG_H diff --git a/mooncake-transfer-engine/src/config.cpp b/mooncake-transfer-engine/src/config.cpp index 6001353a..60305e06 100644 --- a/mooncake-transfer-engine/src/config.cpp +++ b/mooncake-transfer-engine/src/config.cpp @@ -286,23 +286,14 @@ void loadGlobalConfig(GlobalConfig& config) { } const char* min_port_env = std::getenv("MC_MIN_PRC_PORT"); - if (min_port_env) { - int val = atoi(min_port_env); - if (val > 0 && val < 65536) - config.rpc_min_port = val; - else - LOG(WARNING) - << "Ignore value from environment variable MC_PRC_MIN_PORT"; - } - const char* max_port_env = std::getenv("MC_MAX_PRC_PORT"); - if (max_port_env) { - int val = atoi(max_port_env); - if (val > 0 && val < 65536) - config.rpc_max_port = val; - else - LOG(WARNING) - << "Ignore value from environment variable MC_PRC_MAX_PORT"; + { + int raw_min = min_port_env ? atoi(min_port_env) : config.rpc_min_port; + int raw_max = max_port_env ? atoi(max_port_env) : config.rpc_max_port; + auto [validated_min, validated_max] = + ValidatePortRange(raw_min, raw_max, 15000, 17000); + config.rpc_min_port = validated_min; + config.rpc_max_port = validated_max; } if (std::getenv("MC_USE_IPV6")) { @@ -438,4 +429,26 @@ GlobalConfig& globalConfig() { } uint16_t getDefaultHandshakePort() { return globalConfig().handshake_port; } + +std::pair ValidatePortRange(int min_port, int max_port, + int default_min, int default_max) { + constexpr int kMinAllowed = 1024; + constexpr int kEphemeralStart = 32768; + constexpr int kEphemeralEnd = 60999; + constexpr int kMaxAllowed = 65535; + + auto is_valid_port = [&](int p) { + return p >= kMinAllowed && p <= kMaxAllowed && + !(p >= kEphemeralStart && p <= kEphemeralEnd); + }; + + if (!is_valid_port(min_port) || !is_valid_port(max_port) || + min_port > max_port) { + LOG(WARNING) << "Invalid port range [" << min_port << ", " << max_port + << "], falling back to default [" << default_min << ", " + << default_max << "]"; + return {default_min, default_max}; + } + return {min_port, max_port}; +} } // namespace mooncake -- 2.34.1 From 6ed7bbf3a4a7a4fd7b253612b6119256807f7b1c Mon Sep 17 00:00:00 2001 From: Trevin Chow Date: Tue, 5 May 2026 10:13:42 -0700 Subject: [PATCH 034/382] [Build] Define BUILD_BENCHMARK option and gate benchmark subdirectories (#2033) --- mooncake-common/common.cmake | 1 + mooncake-store/CMakeLists.txt | 3 +++ mooncake-transfer-engine/CMakeLists.txt | 4 +++- 3 files changed, 7 insertions(+), 1 deletion(-) diff --git a/mooncake-common/common.cmake b/mooncake-common/common.cmake index 93bd9841..6f45422e 100644 --- a/mooncake-common/common.cmake +++ b/mooncake-common/common.cmake @@ -59,6 +59,7 @@ add_compile_options(-fno-tree-slp-vectorize) option(BUILD_EXAMPLES "Build examples" ON) option(BUILD_UNIT_TESTS "Build unit tests" ON) +option(BUILD_BENCHMARK "Build benchmarks" ON) option(USE_CUDA "option for enabling gpu features for NVIDIA GPU" OFF) option(USE_MLU "option for enabling Cambricon MLU features" OFF) option(USE_MUSA "option for enabling gpu features for MTHREADS GPU" OFF) diff --git a/mooncake-store/CMakeLists.txt b/mooncake-store/CMakeLists.txt index 54f3b1cf..3f12e505 100644 --- a/mooncake-store/CMakeLists.txt +++ b/mooncake-store/CMakeLists.txt @@ -37,5 +37,8 @@ add_subdirectory(src) if (BUILD_UNIT_TESTS) add_subdirectory(tests) +endif() + +if (BUILD_BENCHMARK) add_subdirectory(benchmarks) endif() diff --git a/mooncake-transfer-engine/CMakeLists.txt b/mooncake-transfer-engine/CMakeLists.txt index 87193c0e..9f4583a5 100644 --- a/mooncake-transfer-engine/CMakeLists.txt +++ b/mooncake-transfer-engine/CMakeLists.txt @@ -73,5 +73,7 @@ endif() if (USE_TENT) add_subdirectory(tent) - add_subdirectory(benchmark) + if (BUILD_BENCHMARK) + add_subdirectory(benchmark) + endif() endif() -- 2.34.1 From b5ded0c4d20b9d61b50ddf3043eb9816f50dfa4d Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Wed, 6 May 2026 01:26:32 +0800 Subject: [PATCH 035/382] [Doc] Update README news: add Apr 29 2026 lmsys P2P weight transfer blog (#2038) Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 29f15d8d..7adae202 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ This repository also hosts its technical report and the open-sourced traces.

🔄 Updates

+- **Apr 29, 2026**: SGLang introduces [RDMA-based P2P weight transfer for large-scale distributed RL](https://lmsys.org/blog/2026-04-29-p2p-update/) using Mooncake TransferEngine, achieving 7x faster weight updates for the 1T-parameter Kimi-K2 model (53s → 7.2s) with zero-copy RDMA transfer across thousands of GPUs. - **Mar 19, 2026**: [TorchSpec: Speculative Decoding Training at Scale](https://pytorch.org/blog/torchspec-speculative-decoding-training-at-scale) is [open sourced](https://github.com/torchspec-project/TorchSpec), using Mooncake to decouple inference and training via efficient hidden states management. - **Mar 5, 2026**: [LightX2V](https://github.com/ModelTC/LightX2V/pull/893) now supports disaggregated deployment based on Mooncake, enabling encoder/transformer service decoupling with Mooncake Transfer Engine for high-performance cross-device and cross-machine data transfer. - **Feb 25, 2026**: [SGLang](https://github.com/sgl-project/sglang) merged [Encoder Global Cache Manager](https://github.com/sgl-project/sglang/pull/16137), introducing a Mooncake-powered global multimodal embedding cache that enables cross-instance sharing of ViT embeddings to avoid redundant GPU computation. -- 2.34.1 From d2dcd8b422bb06aca356d7b5053a893b7a4dee5e Mon Sep 17 00:00:00 2001 From: Yifan Qiao Date: Tue, 5 May 2026 10:26:57 -0700 Subject: [PATCH 036/382] [TE] fix: Init CUDA primary context before dmabuf-based GPU memory registration (#2034) Signed-off-by: Yifan Qiao --- .../src/transport/rdma_transport/rdma_context.cpp | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index b9be9e91..25eb8a53 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -241,6 +241,17 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, mrMeta.addr = addr; mrMeta.mr = ibv_reg_mr(pd_, addr, length, access); } else if (memType == CU_MEMORYTYPE_DEVICE) { + // Ensure a CUDA context is current — worker threads or callers + // from non-CUDA threads may lack one. + unsigned int devOrd = 0; + cuPointerGetAttribute(&devOrd, CU_POINTER_ATTRIBUTE_DEVICE_ORDINAL, + (CUdeviceptr)addr); + CUdevice cuDev; + CUcontext cuCtx; + cuDeviceGet(&cuDev, devOrd); + cuDevicePrimaryCtxRetain(&cuCtx, cuDev); + cuCtxSetCurrent(cuCtx); + size_t allocSize; result = cuPointerGetAttribute( &allocSize, CU_POINTER_ATTRIBUTE_RANGE_SIZE, (CUdeviceptr)addr); @@ -249,6 +260,7 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, cuGetErrorString(result, &errStr); LOG(ERROR) << "Failed to call cuPointerGetAttribute for " << (uintptr_t)addr << " cuda error=" << errStr; + cuDevicePrimaryCtxRelease(cuDev); return ERR_CONTEXT; } @@ -261,11 +273,13 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, cuGetErrorString(result, &errStr); LOG(ERROR) << "Failed to retrieve dmabuf for " << (uintptr_t)addr << " cuda error=" << errStr; + cuDevicePrimaryCtxRelease(cuDev); return ERR_CONTEXT; } mrMeta.addr = addr; mrMeta.mr = ibv_reg_dmabuf_mr(pd_, 0 /* offset */, length, (uintptr_t)addr, dmabuf_fd, access); + cuDevicePrimaryCtxRelease(cuDev); } #else mrMeta.addr = addr; -- 2.34.1 From 6487c6bb62952b4a4676654b4f3006457f0cc509 Mon Sep 17 00:00:00 2001 From: Dmitry Barsukoff Date: Tue, 5 May 2026 20:38:08 +0300 Subject: [PATCH 037/382] [TE][FIX]: Typo: PRC -> RPC in min/max ports (#2037) --- docs/source/design/transfer-engine/index.md | 4 ++-- docs/source/zh_archive/transfer-engine.md | 4 ++-- mooncake-common/include/environ.h | 8 ++++---- mooncake-common/src/environ.cpp | 4 ++-- mooncake-transfer-engine/src/config.cpp | 6 ++++-- 5 files changed, 14 insertions(+), 12 deletions(-) diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index d9ead6ad..df53d177 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -301,8 +301,8 @@ For advanced users, TransferEngine provides the following advanced runtime optio - `MC_FORCE_MNNVL` Force to use Multi-Node NVLink as the active transport regardless whether RDMA devices are installed. - `MC_INTRA_NVLINK` Enable intra-node NVLINK transport, and cannot be used together with MC_FORCE_MNNVL. - `MC_FORCE_TCP` Force to use TCP as the active transport regardless whether RDMA devices are installed. -- `MC_MIN_PRC_PORT` Specifies the minimum port number for RPC service. The default value is 15000. -- `MC_MAX_PRC_PORT` Specifies the maximum port number for RPC service. The default value is 17000. +- `MC_MIN_RPC_PORT` Specifies the minimum port number for RPC service. The default value is 15000. +- `MC_MAX_RPC_PORT` Specifies the maximum port number for RPC service. The default value is 17000. - `MC_PATH_ROUNDROBIN` Use round-robin mode in the RDMA path selection. This may be beneficial for transferring large bulks. - `MC_ENDPOINT_STORE_TYPE` Choose FIFO Endpoint Store (`FIFO`) or Sieve Endpoint Store (`SIEVE`), default is `SIEVE`. - `MC_TCP_ENABLE_CONNECTION_POOL` Enable TCP Connection Pool to avoid excessive sockets. diff --git a/docs/source/zh_archive/transfer-engine.md b/docs/source/zh_archive/transfer-engine.md index 3f02a341..85cf5bd4 100644 --- a/docs/source/zh_archive/transfer-engine.md +++ b/docs/source/zh_archive/transfer-engine.md @@ -419,7 +419,7 @@ int init(const std::string &metadata_conn_string, - `MC_FORCE_MNNVL` 强制使用 Multi-Node NVLink 作为主要传输方式,无论是否安装了有效的 RDMA 网卡 - `MC_INTRA_NVLINK` 指定使用Intra-Node NVLink 作为主要传输方式,同时注意该设置不能与MC_FORCE_MNNVL一起使用 - `MC_FORCE_TCP` 强制使用 TCP 作为主要传输方式,无论是否安装了有效的 RDMA 网卡 -- `MC_MIN_PRC_PORT` 指定 RPC 服务使用的最小端口号。默认值为 15000。 -- `MC_MAX_PRC_PORT` 指定 RPC 服务使用的最大端口号。默认值为 17000。 +- `MC_MIN_RPC_PORT` 指定 RPC 服务使用的最小端口号。默认值为 15000。 +- `MC_MAX_RPC_PORT` 指定 RPC 服务使用的最大端口号。默认值为 17000。 - `MC_PATH_ROUNDROBIN` 指定 RDMA 路径选择使用 Round Robin 模式,这对于传输大块数据可能有利。 - `MC_ENDPOINT_STORE_TYPE` 选择 FIFO Endpoint Store (`FIFO`) 或者 Sieve Endpoint Store (`SIEVE`),模式是 `SIEVE`。 diff --git a/mooncake-common/include/environ.h b/mooncake-common/include/environ.h index c5a72c8e..b76c08f1 100644 --- a/mooncake-common/include/environ.h +++ b/mooncake-common/include/environ.h @@ -40,8 +40,8 @@ class Environ { return enable_dest_device_affinity_; } bool GetUseIpv6() const { return use_ipv6_; } - int GetMinPrcPort() const { return min_prc_port_; } - int GetMaxPrcPort() const { return max_prc_port_; } + int GetMinRpcPort() const { return min_rpc_port_; } + int GetMaxRpcPort() const { return max_rpc_port_; } int GetEnableParallelRegMr() const { return enable_parallel_reg_mr_; } std::string GetEndpointStoreType() const { return endpoint_store_type_; } bool GetForceTcp() const { return force_tcp_; } @@ -90,8 +90,8 @@ class Environ { int fragment_ratio_; bool enable_dest_device_affinity_; bool use_ipv6_; - int min_prc_port_; - int max_prc_port_; + int min_rpc_port_; + int max_rpc_port_; int enable_parallel_reg_mr_; std::string endpoint_store_type_; bool force_tcp_; diff --git a/mooncake-common/src/environ.cpp b/mooncake-common/src/environ.cpp index 28cf5fb7..1b63f753 100644 --- a/mooncake-common/src/environ.cpp +++ b/mooncake-common/src/environ.cpp @@ -72,8 +72,8 @@ Environ::Environ() { enable_dest_device_affinity_ = GetBool("MC_ENABLE_DEST_DEVICE_AFFINITY", false); use_ipv6_ = GetBool("MC_USE_IPV6", false); - min_prc_port_ = GetInt("MC_MIN_PRC_PORT", 15000); - max_prc_port_ = GetInt("MC_MAX_PRC_PORT", 17000); + min_rpc_port_ = GetInt("MC_MIN_RPC_PORT", GetInt("MC_MIN_PRC_PORT", 15000)); + max_rpc_port_ = GetInt("MC_MAX_RPC_PORT", GetInt("MC_MAX_PRC_PORT", 17000)); enable_parallel_reg_mr_ = GetInt("MC_ENABLE_PARALLEL_REG_MR", -1); endpoint_store_type_ = GetString("MC_ENDPOINT_STORE_TYPE", "SIEVE"); force_tcp_ = GetBool("MC_FORCE_TCP", false); diff --git a/mooncake-transfer-engine/src/config.cpp b/mooncake-transfer-engine/src/config.cpp index 60305e06..f13f8516 100644 --- a/mooncake-transfer-engine/src/config.cpp +++ b/mooncake-transfer-engine/src/config.cpp @@ -285,8 +285,10 @@ void loadGlobalConfig(GlobalConfig& config) { } } - const char* min_port_env = std::getenv("MC_MIN_PRC_PORT"); - const char* max_port_env = std::getenv("MC_MAX_PRC_PORT"); + const char* min_port_env = std::getenv("MC_MIN_RPC_PORT"); + if (!min_port_env) min_port_env = std::getenv("MC_MIN_PRC_PORT"); + const char* max_port_env = std::getenv("MC_MAX_RPC_PORT"); + if (!max_port_env) max_port_env = std::getenv("MC_MAX_PRC_PORT"); { int raw_min = min_port_env ? atoi(min_port_env) : config.rpc_min_port; int raw_max = max_port_env ? atoi(max_port_env) : config.rpc_max_port; -- 2.34.1 From 7c62d29b11d9c6d4bf65c3312f3269ee0a822e6f Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Wed, 6 May 2026 10:13:37 +0800 Subject: [PATCH 038/382] refactor: unify fabric allocator plumbing (#2028) Share allocator build scaffolding across nvlink and ubshmem backends. - expose a common allocator probe/malloc/free ABI while keeping legacy symbols - reuse shared Python helper logic for allocator loading and probing - centralize allocator CMake and shell build helpers for packaging and builds --- mooncake-integration/CMakeLists.txt | 273 +++++++++--------- mooncake-integration/allocator.py | 102 +++---- mooncake-integration/allocator_ascend_npu.py | 86 ++---- .../fabric_allocator_utils.py | 58 ++++ .../fabric_allocator.cmake | 39 +++ .../nvlink-allocator/CMakeLists.txt | 40 +-- .../nvlink-allocator/build.sh | 22 +- .../nvlink-allocator/nvlink_allocator.cpp | 53 +++- .../scripts/allocator_build_common.sh | 20 ++ .../ubshmem-allocator/CMakeLists.txt | 28 +- .../ubshmem-allocator/build.sh | 22 +- .../ubshmem_fabric_allocator.cpp | 48 ++- scripts/build_wheel.sh | 3 + 13 files changed, 430 insertions(+), 364 deletions(-) create mode 100644 mooncake-integration/fabric_allocator_utils.py create mode 100644 mooncake-transfer-engine/fabric_allocator.cmake create mode 100644 mooncake-transfer-engine/scripts/allocator_build_common.sh diff --git a/mooncake-integration/CMakeLists.txt b/mooncake-integration/CMakeLists.txt index 7ecfb156..47c6ca48 100644 --- a/mooncake-integration/CMakeLists.txt +++ b/mooncake-integration/CMakeLists.txt @@ -1,34 +1,39 @@ file(GLOB SOURCES "*.cpp") include(${CMAKE_CURRENT_LIST_DIR}/../mooncake-common/SetupPython.cmake) execute_process( - COMMAND ${PYTHON_EXECUTABLE} -c "import sys; print([s for s in sys.path if 'packages' in s][0])" - OUTPUT_VARIABLE PYTHON_SYS_PATH -) + COMMAND ${PYTHON_EXECUTABLE} -c + "import sys; print([s for s in sys.path if 'packages' in s][0])" + OUTPUT_VARIABLE PYTHON_SYS_PATH) string(STRIP ${PYTHON_SYS_PATH} PYTHON_SYS_PATH) if("${PYTHON_SYS_PATH}" STREQUAL "") - message(FATAL_ERROR "Python path is empty! Please check the python env.") + message(FATAL_ERROR "Python path is empty! Please check the python env.") endif() -if (WITH_STORE) - include_directories("../mooncake-store/include") - include_directories("../mooncake-store/include/cachelib_memory_allocator") +if(WITH_STORE) + include_directories("../mooncake-store/include") + include_directories("../mooncake-store/include/cachelib_memory_allocator") - include_directories("../mooncake-store/include/cachelib_memory_allocator/include") - include_directories("../mooncake-store/include/cachelib_memory_allocator/fake_include") + include_directories( + "../mooncake-store/include/cachelib_memory_allocator/include") + include_directories( + "../mooncake-store/include/cachelib_memory_allocator/fake_include") endif() include_directories("/usr/include/jsoncpp") include_directories("./") include_directories("../mooncake-transfer-engine/include") -find_package(Python3 COMPONENTS Interpreter Development REQUIRED) +find_package( + Python3 + COMPONENTS Interpreter Development + REQUIRED) execute_process( - COMMAND ${Python3_EXECUTABLE} -c "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))" - OUTPUT_VARIABLE PYTHON_EXT_SUFFIX - OUTPUT_STRIP_TRAILING_WHITESPACE -) + COMMAND ${Python3_EXECUTABLE} -c + "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))" + OUTPUT_VARIABLE PYTHON_EXT_SUFFIX + OUTPUT_STRIP_TRAILING_WHITESPACE) set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) set(CMAKE_BUILD_WITH_INSTALL_RPATH TRUE) @@ -37,179 +42,161 @@ message("${PYTHON_SYS_PATH}") set(PYTHON_PACKAGE_NAME "mooncake") -if (WITH_TE) - pybind11_add_module(engine ${SOURCES} ${CACHE_ALLOCATOR_SOURCES} - transfer_engine/transfer_engine_py.cpp - ) - set_target_properties(engine PROPERTIES - INSTALL_RPATH "$ORIGIN" - ) +if(WITH_TE) + pybind11_add_module(engine ${SOURCES} ${CACHE_ALLOCATOR_SOURCES} + transfer_engine/transfer_engine_py.cpp) + set_target_properties(engine PROPERTIES INSTALL_RPATH "$ORIGIN") - # Propagate EFA compile definition to engine target - if(USE_EFA) - target_compile_definitions(engine PRIVATE USE_EFA) - endif() + # Propagate EFA compile definition to engine target + if(USE_EFA) + target_compile_definitions(engine PRIVATE USE_EFA) + endif() - target_link_libraries(engine PRIVATE - $ - ) + target_link_libraries(engine PRIVATE $) - target_link_libraries(engine PRIVATE ${Python3_LIBRARIES}) - target_include_directories(engine PRIVATE ${Python3_INCLUDE_DIRS}) - if (USE_ASCEND_DIRECT) - target_link_libraries(engine PUBLIC - ascendcl - transfer_engine - glog::glog - gflags::gflags - ) - else() - target_link_libraries(engine PUBLIC - transfer_engine - glog::glog - gflags::gflags - ) - endif() + target_link_libraries(engine PRIVATE ${Python3_LIBRARIES}) + target_include_directories(engine PRIVATE ${Python3_INCLUDE_DIRS}) + if(USE_ASCEND_DIRECT) + target_link_libraries(engine PUBLIC ascendcl transfer_engine glog::glog + gflags::gflags) + else() + target_link_libraries(engine PUBLIC transfer_engine glog::glog + gflags::gflags) + endif() - if (USE_CUDA) - find_package(CUDAToolkit REQUIRED) - target_link_libraries(engine PRIVATE CUDA::cudart) - endif() + if(USE_CUDA) + find_package(CUDAToolkit REQUIRED) + target_link_libraries(engine PRIVATE CUDA::cudart) + endif() endif() - -set(ALLOCATOR_SO_PATH "${CMAKE_BINARY_DIR}/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.so") -set(UBSHMEM_ALLOCATOR_SO_PATH "${CMAKE_BINARY_DIR}/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.so") +set(ALLOCATOR_SO_PATH + "${CMAKE_BINARY_DIR}/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.so" +) +set(UBSHMEM_ALLOCATOR_SO_PATH + "${CMAKE_BINARY_DIR}/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.so" +) if(USE_MNNVL) - message(STATUS "USE_MNNVL is enabled, nvlink_allocator.so will be installed in the Python package") - install(FILES - "${ALLOCATOR_SO_PATH}" - DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME} - ) + message( + STATUS + "USE_MNNVL is enabled, nvlink_allocator.so will be installed in the Python package" + ) + install(FILES "${ALLOCATOR_SO_PATH}" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) endif() if(USE_UBSHMEM) - message(STATUS "USE_UBSHMEM is enabled, ubshmem_fabric_allocator.so will be installed in the Python package") - install(FILES - "${UBSHMEM_ALLOCATOR_SO_PATH}" - DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME} - ) + message( + STATUS + "USE_UBSHMEM is enabled, ubshmem_fabric_allocator.so will be installed in the Python package" + ) + install(FILES "${UBSHMEM_ALLOCATOR_SO_PATH}" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) endif() -if (WITH_STORE) - pybind11_add_module(store ${SOURCES} ${CACHE_ALLOCATOR_SOURCES} - store/store_py.cpp - integration_utils.h - ) - set_target_properties(store PROPERTIES - INSTALL_RPATH "$ORIGIN" - ) - if (USE_ASCEND_DIRECT) - target_link_libraries(store PUBLIC - ascendcl - transfer_engine - glog::glog - gflags::gflags - mooncake_store - cachelib_memory_allocator - ) - else() - target_link_libraries(store PUBLIC - transfer_engine - glog::glog - gflags::gflags - mooncake_store - cachelib_memory_allocator - ) - endif() +if(WITH_STORE) + pybind11_add_module(store ${SOURCES} ${CACHE_ALLOCATOR_SOURCES} + store/store_py.cpp integration_utils.h) + set_target_properties(store PROPERTIES INSTALL_RPATH "$ORIGIN") + if(USE_ASCEND_DIRECT) + target_link_libraries( + store PUBLIC ascendcl transfer_engine glog::glog gflags::gflags + mooncake_store cachelib_memory_allocator) + else() + target_link_libraries( + store PUBLIC transfer_engine glog::glog gflags::gflags mooncake_store + cachelib_memory_allocator) + endif() endif() message("${PYTHON_SYS_PATH}") file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/${PYTHON_PACKAGE_NAME}/__init__.py - "# Auto-generated by CMake\n" -) + "# Auto-generated by CMake\n") -if (USE_MNNVL) - message(STATUS "allocator.py will be installed in the Python package") - install(FILES - "${CMAKE_CURRENT_SOURCE_DIR}/allocator.py" - DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME} - ) +if(USE_MNNVL) + message(STATUS "allocator.py will be installed in the Python package") + install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/allocator.py" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) endif() -if (USE_UBSHMEM) - message(STATUS "allocator_ascend_npu.py will be installed in the Python package") - install(FILES - "${CMAKE_CURRENT_SOURCE_DIR}/allocator_ascend_npu.py" - DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME} - ) +if(USE_UBSHMEM) + message( + STATUS "allocator_ascend_npu.py will be installed in the Python package") + install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/allocator_ascend_npu.py" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) +endif() + +if(USE_MNNVL OR USE_UBSHMEM) + message( + STATUS "fabric_allocator_utils.py will be installed in the Python package") + install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/fabric_allocator_utils.py" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) endif() if(USE_INTRA_NVLINK) - message(STATUS "USE_INTRA_NVLINK is enabled, IntraNode nvlink is now activated") + message( + STATUS "USE_INTRA_NVLINK is enabled, IntraNode nvlink is now activated") endif() -install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/store/async_store.py" DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) +install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/store/async_store.py" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) -#Install Python scripts from mooncake - wheel / mooncake / directory -install(FILES +# Install Python scripts from mooncake - wheel / mooncake / directory +install( + FILES "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/http_metadata_server.py" "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/cli_bench.py" "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/transfer_engine_topology_dump.py" "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_connector_v1.py" "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/vllm_v1_proxy_server.py" - DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME} -) + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) -if (WITH_STORE) - install(FILES - "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_store_service.py" - "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_config.py" - "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/cli.py" - DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME} - ) +if(WITH_STORE) + install( + FILES + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_store_service.py" + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_config.py" + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/cli.py" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) endif() -install( - DIRECTORY - ${CMAKE_CURRENT_BINARY_DIR}/${PYTHON_PACKAGE_NAME}/ - DESTINATION - ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME} -) +install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${PYTHON_PACKAGE_NAME}/ + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) install( - CODE " + CODE " execute_process(COMMAND chmod 767 \"${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}\") execute_process(COMMAND chmod 766 \"${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}/__init__.py\") - " -) + ") -if (WITH_STORE) - install(TARGETS store DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) +if(WITH_STORE) + install(TARGETS store DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) endif() -if (WITH_TE) - install(TARGETS engine DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) - install(TARGETS asio_shared DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) +if(WITH_TE) + install(TARGETS engine DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) + install(TARGETS asio_shared + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) endif() -if (WITH_EP) - install( - DIRECTORY "${EP_PG_STAGING_DIR}/" - DESTINATION "${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}" - FILES_MATCHING PATTERN "*.so" - ) - install(FILES - "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/ep.py" - "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_ep_buffer.py" - "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/pg.py" - DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME} - ) - # ep.so / pg.so link against engine.so by that exact bare name. - # Create a engine.so -> engine symlink so they can find it. - install(CODE " +if(WITH_EP) + install( + DIRECTORY "${EP_PG_STAGING_DIR}/" + DESTINATION "${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}" + FILES_MATCHING + PATTERN "*.so") + install( + FILES + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/ep.py" + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_ep_buffer.py" + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/pg.py" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) + # ep.so / pg.so link against engine.so by that exact bare name. Create a + # engine.so -> engine symlink so they can find it. + install( + CODE " execute_process(COMMAND ${CMAKE_COMMAND} -E create_symlink \"engine${PYTHON_EXT_SUFFIX}\" \"${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}/engine.so\" diff --git a/mooncake-integration/allocator.py b/mooncake-integration/allocator.py index 77fa1c98..f8804a1f 100644 --- a/mooncake-integration/allocator.py +++ b/mooncake-integration/allocator.py @@ -3,20 +3,22 @@ import logging import os import threading from importlib import resources -from typing import Dict, Final, Optional +from typing import Dict, Final from enum import IntEnum from torch import device as torch_device from torch.cuda.memory import CUDAPluggableAllocator +from .fabric_allocator_utils import get_mooncake_so_path, probe_allocator_backend + logger = logging.getLogger(__name__) class MemoryBackend(IntEnum): - USE_CUDAMALLOC = 0 + USE_CUDAMALLOC = 0 USE_CUMEMCREATE = 1 - UNKNOWN = -1 - UNSUPPORTED = -2 + UNKNOWN = -1 + UNSUPPORTED = -2 class NVLinkAllocator: @@ -27,65 +29,34 @@ class NVLinkAllocator: @classmethod def _get_so_path(cls) -> str: - """Dynamically locate nvlink_allocator.so in the mooncake package installation""" - try: - # Attempt to locate package resource - with resources.path("mooncake", "nvlink_allocator.so") as so_path: - if so_path.exists(): - return str(so_path) - except (ImportError, FileNotFoundError, TypeError): - pass - - # Fallback strategy: check in package location via import metadata - try: - import mooncake - - base_path = os.path.dirname(os.path.abspath(mooncake.__file__)) - so_path = os.path.join(base_path, "nvlink_allocator.so") - if os.path.exists(so_path): - return so_path - except (ImportError, FileNotFoundError, TypeError): - raise ImportError( - "SGLANG_MOONCAKE_CUSTOM_MEM_POOL require mooncake-transfer-engine >= 0.3.3.post2." - ) + return get_mooncake_so_path( + "nvlink_allocator.so", + "SGLANG_MOONCAKE_CUSTOM_MEM_POOL require mooncake-transfer-engine >= 0.3.3.post2.", + ) @classmethod def _probe_fabric_memory_support(cls, so_path: str) -> MemoryBackend: - """ - Probe whether the system supports fabric memory by calling a C++ function - that attempts cuMemCreate with CU_MEM_HANDLE_TYPE_FABRIC. - We assume the shared library exports a symbol like: - extern "C" MemoryBackendType mc_probe_fabric_support(int device_id); - """ + supported_type = probe_allocator_backend( + so_path, + "mc_allocator_probe", + ctypes.c_int, + int(MemoryBackend.UNSUPPORTED), + ) try: + backend = MemoryBackend(supported_type) + except ValueError: + logger.info("Unknown Backend error") + return MemoryBackend.UNKNOWN - lib = ctypes.CDLL(so_path) - - # Try to get the probe function - probe_func = lib.mc_probe_fabric_support - probe_func.argtypes = [ctypes.c_int] - probe_func.restype = ctypes.c_int - - # Use device 0 for probing - dev_id = 0 - supported_type = probe_func(dev_id) - if supported_type == MemoryBackend.USE_CUDAMALLOC: - logger.info(f"Use CudaMalloc fallback") - elif supported_type == MemoryBackend.USE_CUMEMCREATE: - logger.info(f"Supports Fabric Memory") - else: - logger.info("Unknown Backend error") - return supported_type - - except AttributeError: - logger.warning( - "Symbol 'mc_probe_fabric_support' not found in nvlink_allocator.so. " - "Assuming fabric memory is NOT supported (you may need to update the library)." - ) - return MemoryBackend.UNSUPPORTED - except Exception as e: - logger.warning(f"Failed to probe fabric memory support: {e}") - return MemoryBackend.UNSUPPORTED + if backend == MemoryBackend.USE_CUDAMALLOC: + logger.info("Use CudaMalloc fallback") + elif backend == MemoryBackend.USE_CUMEMCREATE: + logger.info("Supports Fabric Memory") + elif backend == MemoryBackend.UNSUPPORTED: + logger.info("Allocator backend probing is unsupported") + else: + logger.info("Unknown Backend error") + return backend @classmethod def detect_mem_backend(cls) -> MemoryBackend: @@ -93,14 +64,15 @@ class NVLinkAllocator: if not cls._probe_done: with cls._lock: if cls._probe_done: - return - so_path = None + return cls._supports_fabric try: - so_path = cls._get_so_path() - # First try dedicated probe function - cls._supports_fabric = cls._probe_fabric_memory_support(so_path) + cls._supports_fabric = cls._probe_fabric_memory_support( + cls._get_so_path() + ) except Exception as e: - logger.error(f"Critical error during fabric memory probe setup: {e}") + logger.error( + f"Critical error during fabric memory probe setup: {e}" + ) cls._supports_fabric = MemoryBackend.UNSUPPORTED cls._probe_done = True @@ -112,7 +84,7 @@ class NVLinkAllocator: if device not in cls._instances: so_path = cls._get_so_path() cls._instances[device] = CUDAPluggableAllocator( - so_path, "mc_nvlink_malloc", "mc_nvlink_free" + so_path, "mc_allocator_malloc", "mc_allocator_free" ) return cls._instances[device] diff --git a/mooncake-integration/allocator_ascend_npu.py b/mooncake-integration/allocator_ascend_npu.py index c4f38fc3..bc4e0419 100644 --- a/mooncake-integration/allocator_ascend_npu.py +++ b/mooncake-integration/allocator_ascend_npu.py @@ -1,14 +1,13 @@ import ctypes import logging -import os import threading -from importlib import resources from typing import Dict, Final -import torch_npu from torch import device as torch_device from torch_npu.npu.memory import NPUPluggableAllocator +from .fabric_allocator_utils import get_mooncake_so_path, probe_allocator_backend + logger = logging.getLogger(__name__) @@ -20,62 +19,26 @@ class UBShmemAllocator: @classmethod def _get_so_path(cls) -> str: - """Dynamically locate ubshmem_fabric_allocator.so in the mooncake package installation""" - try: - # Attempt to locate package resource - with resources.path("mooncake", "ubshmem_fabric_allocator.so") as so_path: - if so_path.exists(): - return str(so_path) - except (ImportError, FileNotFoundError, TypeError): - pass - - # Fallback strategy: check in package location via import metadata - try: - import mooncake - - base_path = os.path.dirname(os.path.abspath(mooncake.__file__)) - so_path = os.path.join(base_path, "ubshmem_fabric_allocator.so") - if os.path.exists(so_path): - return so_path - except (ImportError, FileNotFoundError, TypeError): - raise ImportError( - "UBShmemAllocator require mooncake-transfer-engine with USE_UBSHMEM enabled." - ) + return get_mooncake_so_path( + "ubshmem_fabric_allocator.so", + "UBShmemAllocator require mooncake-transfer-engine with USE_UBSHMEM enabled.", + ) @classmethod def _probe_fabric_memory_support(cls, so_path: str) -> bool: - """ - Probe whether the system supports fabric memory by calling a C++ function - that attempts aclrtMallocPhysical with fabric memory support. - The shared library exports a symbol like: - extern "C" bool mc_probe_ub_fabric_support(int device_id); - """ - try: - lib = ctypes.CDLL(so_path) - - # Try to get the probe function - probe_func = lib.mc_probe_ub_fabric_support - probe_func.argtypes = [ctypes.c_int] - probe_func.restype = ctypes.c_bool - - # Use device 0 for probing - dev_id = 0 - supported = probe_func(dev_id) - if supported: - logger.info(f"Supports Fabric Memory with aclMallocPhysical") - else: - logger.info("Fabric memory not supported") - return supported - - except AttributeError: - logger.warning( - "Symbol 'mc_probe_ub_fabric_support' not found in ubshmem_fabric_allocator.so. " - "Assuming fabric memory is NOT supported (you may need to update the library)." + supported = bool( + probe_allocator_backend( + so_path, + "mc_allocator_probe", + ctypes.c_int, + 0, ) - return False - except Exception as e: - logger.warning(f"Failed to probe fabric memory support: {e}") - return False + ) + if supported: + logger.info("Supports Fabric Memory with aclMallocPhysical") + else: + logger.info("Fabric memory not supported") + return supported @classmethod def detect_mem_backend(cls) -> bool: @@ -84,13 +47,14 @@ class UBShmemAllocator: with cls._lock: if cls._probe_done: return cls._supports_fabric - so_path = None try: - so_path = cls._get_so_path() - # First try dedicated probe function - cls._supports_fabric = cls._probe_fabric_memory_support(so_path) + cls._supports_fabric = cls._probe_fabric_memory_support( + cls._get_so_path() + ) except Exception as e: - logger.error(f"Critical error during fabric memory probe setup: {e}") + logger.error( + f"Critical error during fabric memory probe setup: {e}" + ) cls._supports_fabric = False cls._probe_done = True @@ -102,6 +66,6 @@ class UBShmemAllocator: if device not in cls._instances: so_path = cls._get_so_path() cls._instances[device] = NPUPluggableAllocator( - so_path, "mc_ub_fabric_malloc", "mc_ub_fabric_free" + so_path, "mc_allocator_malloc", "mc_allocator_free" ) return cls._instances[device] diff --git a/mooncake-integration/fabric_allocator_utils.py b/mooncake-integration/fabric_allocator_utils.py new file mode 100644 index 00000000..03292aa5 --- /dev/null +++ b/mooncake-integration/fabric_allocator_utils.py @@ -0,0 +1,58 @@ +import ctypes +import logging +import os +from importlib import resources +from typing import Any, TypeVar + +logger = logging.getLogger(__name__) + +ProbeValue = TypeVar("ProbeValue") + + +def get_mooncake_so_path(so_name: str, import_error_message: str) -> str: + try: + with resources.path("mooncake", so_name) as so_path: + if so_path.exists(): + return str(so_path) + except (ImportError, FileNotFoundError, TypeError): + pass + + try: + import mooncake + + base_path = os.path.dirname(os.path.abspath(mooncake.__file__)) + so_path = os.path.join(base_path, so_name) + if os.path.exists(so_path): + return so_path + except (ImportError, FileNotFoundError, TypeError): + pass + + raise ImportError(import_error_message) + + +def probe_allocator_backend( + so_path: str, + probe_symbol: str, + restype: Any, + unsupported_value: ProbeValue, +) -> ProbeValue: + try: + lib = ctypes.CDLL(so_path) + probe_func = getattr(lib, probe_symbol) + probe_func.argtypes = [ctypes.c_int] + probe_func.restype = restype + return probe_func(0) + except AttributeError: + logger.warning( + "Symbol '%s' not found in %s. Assuming allocator probing is unsupported.", + probe_symbol, + os.path.basename(so_path), + ) + return unsupported_value + except Exception as exc: + logger.warning( + "Failed to probe allocator backend from %s: %s", + os.path.basename(so_path), + exc, + ) + return unsupported_value diff --git a/mooncake-transfer-engine/fabric_allocator.cmake b/mooncake-transfer-engine/fabric_allocator.cmake new file mode 100644 index 00000000..52537d1b --- /dev/null +++ b/mooncake-transfer-engine/fabric_allocator.cmake @@ -0,0 +1,39 @@ +function(add_fabric_allocator_build_target) + set(options) + set(oneValueArgs TARGET_NAME BUILD_SCRIPT COMMENT ENABLE_BUILD) + set(multiValueArgs BUILD_ARGS) + cmake_parse_arguments(FAB "${options}" "${oneValueArgs}" "${multiValueArgs}" + ${ARGN}) + + if(NOT FAB_TARGET_NAME) + message( + FATAL_ERROR + "TARGET_NAME is required for add_fabric_allocator_build_target") + endif() + if(NOT FAB_BUILD_SCRIPT) + message( + FATAL_ERROR + "BUILD_SCRIPT is required for add_fabric_allocator_build_target") + endif() + + add_custom_target(${FAB_TARGET_NAME} DEPENDS transfer_engine) + + get_target_property(_include_dirs ${FAB_TARGET_NAME} INCLUDE_DIRECTORIES) + if(NOT _include_dirs) + set(_include_dirs "") + endif() + string(REPLACE ";" " " _include_dirs_str "${_include_dirs}") + + if(FAB_ENABLE_BUILD) + add_custom_command( + TARGET ${FAB_TARGET_NAME} + COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_CURRENT_BINARY_DIR} + COMMAND bash ${FAB_BUILD_SCRIPT} ${FAB_BUILD_ARGS} + ${CMAKE_CURRENT_BINARY_DIR} "${_include_dirs_str}" + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} + COMMENT "${FAB_COMMENT}" + VERBATIM) + endif() + + set_property(TARGET ${FAB_TARGET_NAME} PROPERTY EXCLUDE_FROM_ALL FALSE) +endfunction() diff --git a/mooncake-transfer-engine/nvlink-allocator/CMakeLists.txt b/mooncake-transfer-engine/nvlink-allocator/CMakeLists.txt index d2b4e7c8..35dff93f 100644 --- a/mooncake-transfer-engine/nvlink-allocator/CMakeLists.txt +++ b/mooncake-transfer-engine/nvlink-allocator/CMakeLists.txt @@ -1,25 +1,27 @@ -# Build nvlink allocator and output to build directory - -add_custom_target(build_nvlink_allocator DEPENDS transfer_engine) - -get_target_property(INCLUDE_DIRS build_nvlink_allocator INCLUDE_DIRECTORIES) -string(REPLACE ";" " " INCLUDE_DIRS_STR "${INCLUDE_DIRS}") +include(${CMAKE_CURRENT_SOURCE_DIR}/../fabric_allocator.cmake) set(_extra_build_opts "") -if (USE_HIP) - set(_extra_build_opts "--use-hipcc") -elseif (USE_MUSA) - set(_extra_build_opts "--use-mcc") +if(USE_HIP) + list(APPEND _extra_build_opts --use-hipcc) +elseif(USE_MUSA) + list(APPEND _extra_build_opts --use-mcc) endif() -if (USE_CUDA OR USE_HIP OR USE_MUSA) - add_custom_command( - TARGET build_nvlink_allocator - COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_CURRENT_BINARY_DIR} - COMMAND bash build.sh ${_extra_build_opts} ${CMAKE_CURRENT_BINARY_DIR} "${INCLUDE_DIRS_STR}" - WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} - COMMENT "Building nvlink allocator to ${CMAKE_CURRENT_BINARY_DIR}" - ) +set(_enable_nvlink_allocator_build FALSE) +if(USE_CUDA + OR USE_HIP + OR USE_MUSA) + set(_enable_nvlink_allocator_build TRUE) endif() -set_property(TARGET build_nvlink_allocator PROPERTY EXCLUDE_FROM_ALL FALSE) +add_fabric_allocator_build_target( + TARGET_NAME + build_nvlink_allocator + BUILD_SCRIPT + ${CMAKE_CURRENT_SOURCE_DIR}/build.sh + BUILD_ARGS + ${_extra_build_opts} + COMMENT + "Building nvlink allocator to ${CMAKE_CURRENT_BINARY_DIR}" + ENABLE_BUILD + ${_enable_nvlink_allocator_build}) diff --git a/mooncake-transfer-engine/nvlink-allocator/build.sh b/mooncake-transfer-engine/nvlink-allocator/build.sh index 5f342466..6bd9ccfe 100755 --- a/mooncake-transfer-engine/nvlink-allocator/build.sh +++ b/mooncake-transfer-engine/nvlink-allocator/build.sh @@ -2,6 +2,8 @@ set -e +source "$(dirname "$(readlink -f "$0")")/../scripts/allocator_build_common.sh" + # Check for flags USE_NVCC=false USE_HIPCC=false @@ -25,27 +27,11 @@ fi # Get output directory from command line argument, default to current directory OUTPUT_DIR=${1:-.} -# Get include directories from second argument (if provided) -INCLUDE_LIST="" -if [ $# -ge 2 ]; then - INCLUDE_LIST=${2} -fi - -# Add include directory for cuda_alike.h (relative to build.sh location) -SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd) -INCLUDE_LIST="${INCLUDE_LIST:+${INCLUDE_LIST} }${SCRIPT_DIR}/../include" - -# Process include directories into flags -INCLUDE_FLAGS="" -if [ -n "$INCLUDE_LIST" ]; then - INCLUDE_FLAGS=$(echo "$INCLUDE_LIST" | tr ' ' '\n' | sed 's/^/-I/' | paste -sd' ' -) -fi +prepare_allocator_build_env "$OUTPUT_DIR" "${2:-}" echo "Building nvlink allocator to: $OUTPUT_DIR" -# Create output directory if it doesn't exist -mkdir -p "$OUTPUT_DIR" -CPP_FILE=$(dirname $(readlink -f $0))/nvlink_allocator.cpp # get cpp file path, under same dir with this script +CPP_FILE="${SCRIPT_DIR}/nvlink_allocator.cpp" # Choose build command based on flags if [ "$CI_BUILD" = true ]; then diff --git a/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.cpp b/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.cpp index 78c5c290..3c347eea 100644 --- a/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.cpp +++ b/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.cpp @@ -19,16 +19,15 @@ static CUresult cuMemCreateTryFabric(CUmemGenericAllocationHandle *handle, enum class MemoryBackendType { use_cudamalloc, use_cumemcreate, unknown }; -extern "C" { +namespace { -MemoryBackendType mc_probe_fabric_support(int device_id) { +MemoryBackendType ProbeAllocatorBackend(int device_id) { CUdevice dev; CUresult res = cuDeviceGet(&dev, device_id); if (res != CUDA_SUCCESS) { return MemoryBackendType::unknown; } - // Check device attribute first int fabric_attr = 0; res = cuDeviceGetAttribute( &fabric_attr, CU_DEVICE_ATTRIBUTE_HANDLE_TYPE_FABRIC_SUPPORTED, dev); @@ -40,28 +39,26 @@ MemoryBackendType mc_probe_fabric_support(int device_id) { prop.type = CU_MEM_ALLOCATION_TYPE_PINNED; prop.location.type = CU_MEM_LOCATION_TYPE_DEVICE; prop.location.id = dev; - prop.requestedHandleTypes = CU_MEM_HANDLE_TYPE_FABRIC; // require fabric + prop.requestedHandleTypes = CU_MEM_HANDLE_TYPE_FABRIC; CUmemGenericAllocationHandle handle; size_t size = 4096; res = cuMemCreate(&handle, size, &prop, 0); - if (res == CUDA_SUCCESS) { - cuMemRelease(handle); // success → clean up + cuMemRelease(handle); return MemoryBackendType::use_cumemcreate; - } else { - return MemoryBackendType::use_cudamalloc; } + return MemoryBackendType::use_cudamalloc; } -void *mc_nvlink_malloc(ssize_t size, int device, cudaStream_t stream) { +void *AllocateFabricMemory(ssize_t size, int device, cudaStream_t stream) { + (void)stream; size_t granularity = 0; CUdevice currentDev; CUmemAllocationProp prop = {}; CUmemGenericAllocationHandle handle; void *ptr = nullptr; - int cudaDev; int flag = 0; CUresult result = cuDeviceGet(¤tDev, device); if (result != CUDA_SUCCESS) { @@ -98,7 +95,6 @@ void *mc_nvlink_malloc(ssize_t size, int device, cudaStream_t stream) { std::cerr << "cuMemGetAllocationGranularity failed: " << result; return nullptr; } - // fix size size = (size + granularity - 1) & ~(granularity - 1); if (size == 0) size = granularity; result = cuMemCreateTryFabric(&handle, size, &prop, 0); @@ -138,7 +134,11 @@ void *mc_nvlink_malloc(ssize_t size, int device, cudaStream_t stream) { return ptr; } -void mc_nvlink_free(void *ptr, ssize_t ssize, int device, cudaStream_t stream) { +void FreeFabricMemory(void *ptr, ssize_t ssize, int device, + cudaStream_t stream) { + (void)ssize; + (void)device; + (void)stream; CUmemGenericAllocationHandle handle; size_t size = 0; if (!ptr) return; @@ -154,4 +154,33 @@ void mc_nvlink_free(void *ptr, ssize_t ssize, int device, cudaStream_t stream) { } cuMemRelease(handle); } + +} // namespace + +extern "C" { + +MemoryBackendType mc_probe_fabric_support(int device_id) { + return ProbeAllocatorBackend(device_id); +} + +int mc_allocator_probe(int device_id) { + return static_cast(ProbeAllocatorBackend(device_id)); +} + +void *mc_allocator_malloc(ssize_t size, int device, cudaStream_t stream) { + return AllocateFabricMemory(size, device, stream); +} + +void *mc_nvlink_malloc(ssize_t size, int device, cudaStream_t stream) { + return mc_allocator_malloc(size, device, stream); +} + +void mc_allocator_free(void *ptr, ssize_t ssize, int device, + cudaStream_t stream) { + FreeFabricMemory(ptr, ssize, device, stream); +} + +void mc_nvlink_free(void *ptr, ssize_t ssize, int device, cudaStream_t stream) { + mc_allocator_free(ptr, ssize, device, stream); +} } diff --git a/mooncake-transfer-engine/scripts/allocator_build_common.sh b/mooncake-transfer-engine/scripts/allocator_build_common.sh new file mode 100644 index 00000000..410ce0ce --- /dev/null +++ b/mooncake-transfer-engine/scripts/allocator_build_common.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +resolve_allocator_script_dir() { + cd "$(dirname "$(readlink -f "$0")")" &>/dev/null && pwd +} + +prepare_allocator_build_env() { + OUTPUT_DIR=${1:-.} + local include_list=${2:-} + + SCRIPT_DIR=$(resolve_allocator_script_dir) + include_list="${include_list:+${include_list} }${SCRIPT_DIR}/../include" + + INCLUDE_FLAGS="" + if [ -n "$include_list" ]; then + INCLUDE_FLAGS=$(printf '%s\n' "$include_list" | tr ' ' '\n' | sed '/^$/d; s/^/-I/' | paste -sd' ' -) + fi + + mkdir -p "$OUTPUT_DIR" +} diff --git a/mooncake-transfer-engine/ubshmem-allocator/CMakeLists.txt b/mooncake-transfer-engine/ubshmem-allocator/CMakeLists.txt index 1ee8af11..f6389f74 100644 --- a/mooncake-transfer-engine/ubshmem-allocator/CMakeLists.txt +++ b/mooncake-transfer-engine/ubshmem-allocator/CMakeLists.txt @@ -1,18 +1,16 @@ -# Build ubshmem allocator and output to build directory +include(${CMAKE_CURRENT_SOURCE_DIR}/../fabric_allocator.cmake) -add_custom_target(build_ubshmem_allocator DEPENDS transfer_engine) - -get_target_property(INCLUDE_DIRS build_ubshmem_allocator INCLUDE_DIRECTORIES) -string(REPLACE ";" " " INCLUDE_DIRS_STR "${INCLUDE_DIRS}") - -if (USE_UBSHMEM) - add_custom_command( - TARGET build_ubshmem_allocator - COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_CURRENT_BINARY_DIR} - COMMAND bash build.sh ${CMAKE_CURRENT_BINARY_DIR} "${INCLUDE_DIRS_STR}" - WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} - COMMENT "Building ubshmem allocator to ${CMAKE_CURRENT_BINARY_DIR}" - ) +set(_enable_ubshmem_allocator_build FALSE) +if(USE_UBSHMEM) + set(_enable_ubshmem_allocator_build TRUE) endif() -set_property(TARGET build_ubshmem_allocator PROPERTY EXCLUDE_FROM_ALL FALSE) \ No newline at end of file +add_fabric_allocator_build_target( + TARGET_NAME + build_ubshmem_allocator + BUILD_SCRIPT + ${CMAKE_CURRENT_SOURCE_DIR}/build.sh + COMMENT + "Building ubshmem allocator to ${CMAKE_CURRENT_BINARY_DIR}" + ENABLE_BUILD + ${_enable_ubshmem_allocator_build}) diff --git a/mooncake-transfer-engine/ubshmem-allocator/build.sh b/mooncake-transfer-engine/ubshmem-allocator/build.sh index 99ac821d..4b13d114 100644 --- a/mooncake-transfer-engine/ubshmem-allocator/build.sh +++ b/mooncake-transfer-engine/ubshmem-allocator/build.sh @@ -2,30 +2,16 @@ set -e +source "$(dirname "$(readlink -f "$0")")/../scripts/allocator_build_common.sh" + # Get output directory from command line argument, default to current directory OUTPUT_DIR=${1:-.} -# Get include directories from second argument (if provided) -INCLUDE_LIST="" -if [ $# -ge 2 ]; then - INCLUDE_LIST=${2} -fi - -# Add include directory for cuda (relative to build.sh location) -SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd) -INCLUDE_LIST="${INCLUDE_LIST:+${INCLUDE_LIST} }${SCRIPT_DIR}/../include" - -# Process include directories into flags -INCLUDE_FLAGS="" -if [ -n "$INCLUDE_LIST" ]; then - INCLUDE_FLAGS=$(echo "$INCLUDE_LIST" | tr ' ' '\n' | sed 's/^/-I/' | paste -sd' ' -) -fi +prepare_allocator_build_env "$OUTPUT_DIR" "${2:-}" echo "Building ubshmem fabric allocator to: $OUTPUT_DIR" -# Create output directory if it doesn't exist -mkdir -p "$OUTPUT_DIR" -CPP_FILE=$(dirname $(readlink -f $0))/ubshmem_fabric_allocator.cpp +CPP_FILE="${SCRIPT_DIR}/ubshmem_fabric_allocator.cpp" # Detect CPU architecture CURRENT_CPU=$(uname -m) diff --git a/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.cpp b/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.cpp index 8540dc47..b76e67b3 100644 --- a/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.cpp +++ b/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.cpp @@ -3,9 +3,9 @@ #include -extern "C" { +namespace { -bool mc_probe_ub_fabric_support(int device_id) { +bool ProbeAllocatorBackend(int device_id) { aclError res = aclrtSetDevice(device_id); if (res != ACL_ERROR_NONE) { std::cerr << "Set device failed: " << device_id << ", result" << res; @@ -19,25 +19,22 @@ bool mc_probe_ub_fabric_support(int device_id) { prop.memAttr = ACL_HBM_MEM_HUGE; aclrtDrvMemHandle handle; - // try to allocate 2M fabric mem size_t size = 2 * 1024 * 1024; res = aclrtMallocPhysical(&handle, size, &prop, 0); if (res == ACL_ERROR_NONE) { - aclrtFreePhysical(handle); // success → clean up + aclrtFreePhysical(handle); return true; - } else { - return false; } + return false; } -void *mc_ub_fabric_malloc(ssize_t size, int device) { +void *AllocateFabricMemory(ssize_t size, int device) { size_t granularity = 0; aclrtPhysicalMemProp prop = {}; aclrtDrvMemHandle handle; void *ptr = nullptr; - // Align size to 2M const size_t alignment = 2 * 1024 * 1024; size = (size + alignment - 1) & ~(alignment - 1); @@ -54,7 +51,6 @@ void *mc_ub_fabric_malloc(ssize_t size, int device) { return nullptr; } uint64_t page_type = 1; - // now granularity is reserved to 0 result = aclrtReserveMemAddress(&ptr, size, granularity, nullptr, page_type); if (result != ACL_ERROR_NONE) { @@ -73,7 +69,8 @@ void *mc_ub_fabric_malloc(ssize_t size, int device) { return ptr; } -void mc_ub_fabric_free(void *ptr, int device) { +void FreeFabricMemory(void *ptr, int device) { + (void)device; aclrtDrvMemHandle handle; if (!ptr) { return; @@ -83,10 +80,35 @@ void mc_ub_fabric_free(void *ptr, int device) { std::cerr << "aclrtMemRetainAllocationHandle failed: " << result << "\n"; return; - } else { - (void)aclrtUnmapMem(ptr); - (void)aclrtReleaseMemAddress(ptr); } + (void)aclrtUnmapMem(ptr); + (void)aclrtReleaseMemAddress(ptr); (void)aclrtFreePhysical(handle); } + +} // namespace + +extern "C" { + +bool mc_probe_ub_fabric_support(int device_id) { + return ProbeAllocatorBackend(device_id); +} + +int mc_allocator_probe(int device_id) { + return ProbeAllocatorBackend(device_id) ? 1 : 0; +} + +void *mc_allocator_malloc(ssize_t size, int device) { + return AllocateFabricMemory(size, device); +} + +void *mc_ub_fabric_malloc(ssize_t size, int device) { + return mc_allocator_malloc(size, device); +} + +void mc_allocator_free(void *ptr, int device) { FreeFabricMemory(ptr, device); } + +void mc_ub_fabric_free(void *ptr, int device) { + mc_allocator_free(ptr, device); +} } diff --git a/scripts/build_wheel.sh b/scripts/build_wheel.sh index 24125b8a..3b0456ab 100755 --- a/scripts/build_wheel.sh +++ b/scripts/build_wheel.sh @@ -25,6 +25,9 @@ rm -f mooncake-wheel/mooncake/*.so echo "Creating directory structure..." +# Copy shared allocator helper used by both CUDA and Ascend pluggable allocators. +cp mooncake-integration/fabric_allocator_utils.py mooncake-wheel/mooncake/fabric_allocator_utils.py + # Copy engine.so to mooncake directory (will be imported by transfer module) cp build/mooncake-integration/engine.*.so mooncake-wheel/mooncake/engine.so -- 2.34.1 From 2a5a94a030649f56aef436b7e0b691a266cae83b Mon Sep 17 00:00:00 2001 From: Jinlong Chen Date: Wed, 6 May 2026 10:30:16 +0800 Subject: [PATCH 039/382] [TE]: Fix possible dead lock in RDMA transport connection setup (#1959) Say we have 3 transfer engine instances T0, T1, T2, in the following case with P2PHANDSHAKE, they will form a circular dead lock: T0.listener is handling connection request from T1: -> setupConnectionsByPassive() -> getSegmentDescByName(T1) -> exchangeMetadata(T1) -> wait for T1.listener processing T1.listener is handling connection request from T2: -> setupConnectionsByPassive() -> getSegmentDescByName(T2) -> exchangeMetadata(T2) -> wait for T2.listener processing T2.listener is handling connection request from T0: -> setupConnectionsByPassive() -> getSegmentDescByName(T0) -> exchangeMetadata(T0) -> wait for T0.listener processing T0 -> T1 -> T2 -> T0 To fix this, we can remove the calling to getSegmentDescByName completely from the connection establish process, and exchange necessary connection information through HandShakeDesc. This can also significantly simplify the connection process. Signed-off-by: Chen Jinlong --- .../include/transfer_metadata.h | 2 + .../transport/rdma_transport/rdma_endpoint.h | 2 +- .../src/transfer_metadata.cpp | 12 ++ .../rdma_transport/rdma_endpoint.cpp | 138 +++++++++++++----- 4 files changed, 113 insertions(+), 41 deletions(-) diff --git a/mooncake-transfer-engine/include/transfer_metadata.h b/mooncake-transfer-engine/include/transfer_metadata.h index a4522a22..802cf7e3 100644 --- a/mooncake-transfer-engine/include/transfer_metadata.h +++ b/mooncake-transfer-engine/include/transfer_metadata.h @@ -118,6 +118,8 @@ class TransferMetadata { struct HandShakeDesc { std::string local_nic_path; + uint16_t local_lid = 0; + std::string local_gid; std::string peer_nic_path; #ifdef USE_UB std::vector jetty_num; // for ub/urma diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h index 042d8661..662413ad 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h @@ -143,7 +143,7 @@ class RdmaEndPoint { std::vector peer_qp_num_list, std::string *reply_msg = nullptr); - int doSetupConnection(int qp_index, const std::string &peer_gid, + int doSetupConnection(int qp_index, const ibv_gid &peer_gid, uint16_t peer_lid, uint32_t peer_qp_num, std::string *reply_msg = nullptr); diff --git a/mooncake-transfer-engine/src/transfer_metadata.cpp b/mooncake-transfer-engine/src/transfer_metadata.cpp index 7946c69f..dcae71d2 100644 --- a/mooncake-transfer-engine/src/transfer_metadata.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata.cpp @@ -67,6 +67,8 @@ struct TransferHandshakeUtil { static Json::Value encode(const TransferMetadata::HandShakeDesc &desc) { Json::Value root; root["local_nic_path"] = desc.local_nic_path; + root["local_lid"] = desc.local_lid; + root["local_gid"] = desc.local_gid; root["peer_nic_path"] = desc.peer_nic_path; #ifdef USE_BAREX root["barex_port"] = desc.barex_port; @@ -92,6 +94,16 @@ struct TransferHandshakeUtil { static int decode(Json::Value root, TransferMetadata::HandShakeDesc &desc) { desc.local_nic_path = root["local_nic_path"].asString(); + if (root.isMember("local_lid") && root["local_lid"].isUInt()) { + desc.local_lid = root["local_lid"].asUInt(); + } else { + desc.local_lid = 0; + } + if (root.isMember("local_gid") && root["local_gid"].isString()) { + desc.local_gid = root["local_gid"].asString(); + } else { + desc.local_gid.clear(); + } desc.peer_nic_path = root["peer_nic_path"].asString(); #ifdef USE_BAREX desc.barex_port = root["barex_port"].asInt(); diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp index 4c21b678..c2c3af81 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp @@ -20,6 +20,7 @@ #include #include #include +#include #include "common.h" #include "config.h" @@ -160,16 +161,7 @@ int RdmaEndPoint::setupConnectionsByActive() { // loopback mode if (context_.nicPath() == peer_nic_path_) { - auto segment_desc = - context_.engine().meta()->getSegmentDescByID(LOCAL_SEGMENT_ID); - if (segment_desc) { - for (auto &nic : segment_desc->devices) - if (nic.name == context_.deviceName()) - return doSetupConnection(nic.gid, nic.lid, qpNum()); - } - LOG(ERROR) << "Peer NIC " << context_.deviceName() - << " not found in localhost"; - return ERR_DEVICE_NOT_FOUND; + return doSetupConnection(context_.gid(), context_.lid(), qpNum()); } // Only proceed with RPC if we are the first to transition from @@ -189,6 +181,8 @@ int RdmaEndPoint::setupConnectionsByActive() { } local_desc.local_nic_path = context_.nicPath(); + local_desc.local_lid = context_.lid(); + local_desc.local_gid = context_.gid(); local_desc.peer_nic_path = peer_nic_path_; local_desc.qp_num = qpNum(); } @@ -290,16 +284,26 @@ int RdmaEndPoint::setupConnectionsByActive() { return ERR_REJECT_HANDSHAKE; } - auto segment_desc = - context_.engine().meta()->getSegmentDescByName(peer_server_name); - if (segment_desc) { - for (auto &nic : segment_desc->devices) { - if (nic.name == peer_nic_name) { - int ret = doSetupConnection(nic.gid, nic.lid, peer_desc.qp_num); - if (ret != 0) { - resetConnection("failed connection setup (active)"); + if (!peer_desc.local_gid.empty()) { + int ret = doSetupConnection(peer_desc.local_gid, peer_desc.local_lid, + peer_desc.qp_num); + if (ret != 0) { + resetConnection("failed connection setup (active)"); + } + return ret; + } else { + auto segment_desc = + context_.engine().meta()->getSegmentDescByName(peer_server_name); + if (segment_desc) { + for (auto &nic : segment_desc->devices) { + if (nic.name == peer_nic_name) { + int ret = + doSetupConnection(nic.gid, nic.lid, peer_desc.qp_num); + if (ret != 0) { + resetConnection("failed connection setup (active)"); + } + return ret; } - return ret; } } } @@ -316,6 +320,8 @@ int RdmaEndPoint::setupConnectionsByPassive(const HandShakeDesc &peer_desc, // If already connected with the same peer QP info, return success if (peer_qp_num_list_ == peer_desc.qp_num) { local_desc.local_nic_path = context_.nicPath(); + local_desc.local_lid = context_.lid(); + local_desc.local_gid = context_.gid(); local_desc.peer_nic_path = peer_nic_path_; local_desc.qp_num = qpNum(); LOG(INFO) << "Received same peer QP numbers, reusing connection."; @@ -357,20 +363,32 @@ int RdmaEndPoint::setupConnectionsByPassive(const HandShakeDesc &peer_desc, } local_desc.local_nic_path = context_.nicPath(); + local_desc.local_lid = context_.lid(); + local_desc.local_gid = context_.gid(); local_desc.peer_nic_path = peer_nic_path_; local_desc.qp_num = qpNum(); - auto segment_desc = - context_.engine().meta()->getSegmentDescByName(peer_server_name); - if (segment_desc) { - for (auto &nic : segment_desc->devices) { - if (nic.name == peer_nic_name) { - int ret = doSetupConnection(nic.gid, nic.lid, peer_desc.qp_num, - &local_desc.reply_msg); - if (ret != 0) { - resetConnection("failed connection setup (passive)"); + if (!peer_desc.local_gid.empty()) { + int ret = doSetupConnection(peer_desc.local_gid, peer_desc.local_lid, + peer_desc.qp_num, &local_desc.reply_msg); + if (ret != 0) { + resetConnection("failed connection setup (passive)"); + } + return ret; + } else { + auto segment_desc = + context_.engine().meta()->getSegmentDescByName(peer_server_name); + if (segment_desc) { + for (auto &nic : segment_desc->devices) { + if (nic.name == peer_nic_name) { + int ret = + doSetupConnection(nic.gid, nic.lid, peer_desc.qp_num, + &local_desc.reply_msg); + if (ret != 0) { + resetConnection("failed connection setup (passive)"); + } + return ret; } - return ret; } } } @@ -542,6 +560,44 @@ std::vector RdmaEndPoint::qpNum() const { return ret; } +static int parseGidString(const std::string &gid_str, ibv_gid &gid_out) { + if (gid_str.empty()) { + LOG(ERROR) << "GID string is empty"; + return ERR_INVALID_ARGUMENT; + } + + // Prepend a colon to the GID string to simplify parsing. + std::istringstream iss(":" + gid_str); + for (size_t i = 0; i < sizeof(gid_out.raw); i++) { + if (iss.get() != ':') { + LOG(ERROR) << "Invalid GID format at byte " << i + << ", peer_gid=" << gid_str; + return ERR_INVALID_ARGUMENT; + } + + uint32_t byte = 0; + iss >> std::hex >> byte; + + if (iss.fail() || byte > 0xFF) { + LOG(ERROR) << "Invalid GID format at byte " << i + << ", peer_gid=" << gid_str; + return ERR_INVALID_ARGUMENT; + } + + gid_out.raw[i] = static_cast(byte); + } + + // Ensure no trailing data remains after 16 bytes + char extra; + if (iss.get(extra)) { + LOG(ERROR) << "GID string has trailing data after 16 bytes" + << ", peer_gid=" << gid_str; + return ERR_INVALID_ARGUMENT; + } + + return 0; +} + int RdmaEndPoint::doSetupConnection(const std::string &peer_gid, uint16_t peer_lid, std::vector peer_qp_num_list, @@ -555,8 +611,18 @@ int RdmaEndPoint::doSetupConnection(const std::string &peer_gid, return ERR_INVALID_ARGUMENT; } + // Verify and parse the peer GID before proceeding. + ibv_gid peer_gid_raw = {}; + int ret = parseGidString(peer_gid, peer_gid_raw); + if (ret) { + std::string message = "Invalid peer GID: " + peer_gid; + LOG(ERROR) << "[Handshake] " << message; + if (reply_msg) *reply_msg = message; + return ret; + } + for (int qp_index = 0; qp_index < (int)qp_list_.size(); ++qp_index) { - int ret = doSetupConnection(qp_index, peer_gid, peer_lid, + int ret = doSetupConnection(qp_index, peer_gid_raw, peer_lid, peer_qp_num_list[qp_index], reply_msg); if (ret) return ret; } @@ -566,7 +632,7 @@ int RdmaEndPoint::doSetupConnection(const std::string &peer_gid, return 0; } -int RdmaEndPoint::doSetupConnection(int qp_index, const std::string &peer_gid, +int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, uint16_t peer_lid, uint32_t peer_qp_num, std::string *reply_msg) { if (qp_index < 0 || qp_index > (int)qp_list_.size()) @@ -609,15 +675,7 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const std::string &peer_gid, attr.path_mtu = context_.activeMTU(); if (globalConfig().mtu_length < attr.path_mtu) attr.path_mtu = globalConfig().mtu_length; - ibv_gid peer_gid_raw; - std::istringstream iss(peer_gid); - for (int i = 0; i < 16; ++i) { - int value; - iss >> std::hex >> value; - peer_gid_raw.raw[i] = static_cast(value); - if (i < 15) iss.ignore(1, ':'); - } - attr.ah_attr.grh.dgid = peer_gid_raw; + attr.ah_attr.grh.dgid = peer_gid; // TODO gidIndex and portNum must fetch from REMOTE attr.ah_attr.grh.sgid_index = context_.gidIndex(); attr.ah_attr.grh.hop_limit = MAX_HOP_LIMIT; -- 2.34.1 From 658297c4d9d1f047a26e86fb3c646965c80b3c72 Mon Sep 17 00:00:00 2001 From: Yifan Qiao Date: Wed, 6 May 2026 00:55:35 -0700 Subject: [PATCH 040/382] [TransferEngine] Use allocation base addr for dmabuf-based mem registration (#2035) --------- Signed-off-by: Yifan Qiao --- .../transport/rdma_transport/rdma_context.cpp | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index 25eb8a53..bf543df1 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -252,13 +252,19 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, cuDevicePrimaryCtxRetain(&cuCtx, cuDev); cuCtxSetCurrent(cuCtx); + // Use cuMemGetAddressRange to get the true allocation base and + // size — addr may sit at an offset within a larger cudaMalloc + // block (e.g. PyTorch caching allocator packs multiple tensors + // into one allocation). cuMemGetHandleForAddressRange requires + // the exact allocation boundaries. + CUdeviceptr allocBase; size_t allocSize; - result = cuPointerGetAttribute( - &allocSize, CU_POINTER_ATTRIBUTE_RANGE_SIZE, (CUdeviceptr)addr); + result = + cuMemGetAddressRange(&allocBase, &allocSize, (CUdeviceptr)addr); if (result != CUDA_SUCCESS) { const char *errStr; cuGetErrorString(result, &errStr); - LOG(ERROR) << "Failed to call cuPointerGetAttribute for " + LOG(ERROR) << "Failed to call cuMemGetAddressRange for " << (uintptr_t)addr << " cuda error=" << errStr; cuDevicePrimaryCtxRelease(cuDev); return ERR_CONTEXT; @@ -266,7 +272,7 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, int dmabuf_fd; result = cuMemGetHandleForAddressRange( - &dmabuf_fd, (CUdeviceptr)addr, allocSize, + &dmabuf_fd, allocBase, allocSize, CU_MEM_RANGE_HANDLE_TYPE_DMA_BUF_FD, 0); if (result != CUDA_SUCCESS) { const char *errStr; @@ -277,7 +283,8 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, return ERR_CONTEXT; } mrMeta.addr = addr; - mrMeta.mr = ibv_reg_dmabuf_mr(pd_, 0 /* offset */, length, + uint64_t dmabuf_offset = (uintptr_t)addr - allocBase; + mrMeta.mr = ibv_reg_dmabuf_mr(pd_, dmabuf_offset, length, (uintptr_t)addr, dmabuf_fd, access); cuDevicePrimaryCtxRelease(cuDev); } -- 2.34.1 From 83ccd8a39f45cf042ebce6a071bd4f9ae8c4585c Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Wed, 6 May 2026 16:14:19 +0800 Subject: [PATCH 041/382] [PG] Inherit ProcessGroup to fix dynamic getSize() after extend_group_size_to (#2040) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [PG] Inherit ProcessGroup to fix dynamic getSize() after extend_group_size_to MooncakeBackend now inherits c10d::ProcessGroup instead of c10d::Backend, so overriding getSize() actually takes effect (ProcessGroup::getSize() is virtual; Backend::getSize() is not). Key changes: - MooncakeBackend: base class ProcessGroup(store, rank, size); override getSize() returns meta_->size (dynamic) instead of the static size_ - MooncakeBackendOptions: inherits torch::CustomClassHolder instead of Backend::Options (Backend-specific base no longer applicable) - pg_py.cpp: factory functions return intrusive_ptr; helper functions (extend_group_size_to, get_peer_state, etc.) accept intrusive_ptr and downcast to MooncakeBackend - pg_test_utils.py: get_mooncake_backend() returns the group directly (no _get_backend() call); removed device_id from init_mooncake_group to avoid PyTorch calling pg._get_backend() post-init for ProcessGroup subclasses with no registered backend - mooncake_ep_buffer.py: self.backend = self.group (same reason) - test_pg_elastic.py: add test_dynamic_world_size verifying dist.get_world_size() returns N+1 after extend_group_size_to(N+1) - test_mooncake_backend_elastic.py: assert dist.get_world_size() after extension; use dist.group.WORLD directly as backend handle PyTorch shortcut: _new_process_group_helper checks issubclass(type(backend_class), ProcessGroup) — when true, the returned instance is used directly as the PG without _register_backend, so _get_backend() is no longer needed or valid. Verified on k8s (sunxun/mooncake-pg-process-group-test, 1 GPU): - 18/18 CPU collective tests pass (test_pg_collectives.py -k CPU) - test_dynamic_world_size passes: world_size correctly updated 4 -> 5 - test_elastic_extension passes CUDA tests deferred: allreduce hang on available pod is pre-existing (upstream main has same behavior; RDMA not properly configured there). Co-Authored-By: Claude Sonnet 4.6 * [PG] Fix clang-format violations and apply reviewer suggestions - mooncake_backend.h: collapse two-line initializer list to one line; collapse getSize() body to single-line inline form (Google style) - pg_py.cpp: wrap extendGroupSizeTo signature at 80 cols - mooncake_backend.cpp: std::move distBackendOpts.store to avoid unnecessary intrusive_ptr ref-count increment (reviewer suggestion) - test_pg_elastic.py: drop unused `device` binding in _dynamic_world_size_worker (CodeQL unused-local warning) Co-Authored-By: Claude Sonnet 4.6 --------- Co-authored-by: Claude Sonnet 4.6 --- mooncake-pg/include/mooncake_backend.h | 26 ++++---------- mooncake-pg/src/mooncake_backend.cpp | 7 ++-- mooncake-pg/src/pg_py.cpp | 19 ++++++----- mooncake-pg/tests/pg_test_utils.py | 4 +-- mooncake-pg/tests/test_pg_elastic.py | 34 +++++++++++++++++++ mooncake-wheel/mooncake/mooncake_ep_buffer.py | 4 +-- .../tests/test_mooncake_backend_elastic.py | 10 ++++-- 7 files changed, 64 insertions(+), 40 deletions(-) diff --git a/mooncake-pg/include/mooncake_backend.h b/mooncake-pg/include/mooncake_backend.h index d6196ff4..5fa8bcc1 100644 --- a/mooncake-pg/include/mooncake_backend.h +++ b/mooncake-pg/include/mooncake_backend.h @@ -11,19 +11,18 @@ #include #include #include +#include #include namespace mooncake { -class MooncakeBackend final : public ::c10d::Backend { +class MooncakeBackend final : public ::c10d::ProcessGroup { public: - struct MooncakeBackendOptions final : ::c10d::Backend::Options { + struct MooncakeBackendOptions final : torch::CustomClassHolder { explicit MooncakeBackendOptions(at::Tensor activeRanks) - : Options{"mooncake"}, activeRanks_{activeRanks} {} + : activeRanks_{activeRanks} {} MooncakeBackendOptions(at::Tensor activeRanks, bool isExtension) - : Options{"mooncake"}, - activeRanks_{activeRanks}, - isExtension_{isExtension} {} + : activeRanks_{activeRanks}, isExtension_{isExtension} {} ~MooncakeBackendOptions() override = default; @@ -50,20 +49,7 @@ class MooncakeBackend final : public ::c10d::Backend { const std::string getBackendName() const override; - /** - * @brief Return the stored Mooncake-specific backend options. - * - * PyTorch can use this to read Mooncake-specific options from an existing - * process group. This is used, for example, create sub-groups that inherit - * settings from the parent group. - * - * @return The stored backend options, or null when the backend was created - * without explicit Mooncake options. - */ - c10::intrusive_ptr<::c10d::Backend::Options> getBackendOptions() override { - return c10::static_intrusive_pointer_cast<::c10d::Backend::Options>( - options_); - } + int getSize() const override { return meta_ ? meta_->size : size_; } // Point-to-point send/recv for torch.distributed P2POp/batch_isend_irecv. // Only single-tensor ops are supported. diff --git a/mooncake-pg/src/mooncake_backend.cpp b/mooncake-pg/src/mooncake_backend.cpp index 11c20658..ccd4dfa9 100644 --- a/mooncake-pg/src/mooncake_backend.cpp +++ b/mooncake-pg/src/mooncake_backend.cpp @@ -95,7 +95,8 @@ class MooncakeP2PWork : public ::c10d::Work { MooncakeBackend::MooncakeBackend( c10d::DistributedBackendOptions distBackendOpts, c10::intrusive_ptr options, bool isCpu) - : Backend(distBackendOpts.group_rank, distBackendOpts.group_size), + : ProcessGroup(distBackendOpts.store, distBackendOpts.group_rank, + distBackendOpts.group_size), options_(std::move(options)), isCpu_(isCpu) { auto store = std::move(distBackendOpts.store); @@ -311,7 +312,7 @@ c10::intrusive_ptr MooncakeBackend::send( auto tensor = tensors.back(); TORCH_CHECK(meta_->store, "P2P send requires a valid Store."); - TORCH_CHECK(dstRank >= 0 && dstRank < size_, + TORCH_CHECK(dstRank >= 0 && dstRank < meta_->size, "P2P send: dstRank out of range."); auto contiguous = tensor.contiguous(); @@ -343,7 +344,7 @@ c10::intrusive_ptr MooncakeBackend::recv( auto tensor = tensors.back(); TORCH_CHECK(meta_->store, "P2P recv requires a valid Store."); - TORCH_CHECK(srcRank >= 0 && srcRank < size_, + TORCH_CHECK(srcRank >= 0 && srcRank < meta_->size, "P2P recv: srcRank out of range."); auto target = tensor.is_contiguous() ? tensor : tensor.contiguous(); diff --git a/mooncake-pg/src/pg_py.cpp b/mooncake-pg/src/pg_py.cpp index 78e3269a..441a04ad 100644 --- a/mooncake-pg/src/pg_py.cpp +++ b/mooncake-pg/src/pg_py.cpp @@ -9,7 +9,7 @@ namespace py = pybind11; namespace mooncake { -c10::intrusive_ptr createMooncakeBackend( +c10::intrusive_ptr createMooncakeBackend( c10d::DistributedBackendOptions distBackendOpts, c10::intrusive_ptr backendOptions) { @@ -17,7 +17,7 @@ c10::intrusive_ptr createMooncakeBackend( std::move(backendOptions)); } -c10::intrusive_ptr createMooncakeCpuBackend( +c10::intrusive_ptr createMooncakeCpuBackend( c10d::DistributedBackendOptions distBackendOpts, c10::intrusive_ptr backendOptions) { @@ -39,46 +39,47 @@ __attribute__((constructor)) static void MooncakeBackendConstructor() { /* extended_api */ true, **kwargsCuda); } -std::string getPreferredHca(c10::intrusive_ptr backend, +std::string getPreferredHca(c10::intrusive_ptr backend, std::string location) { auto mooncakeBackend = c10::static_intrusive_pointer_cast(backend); return mooncakeBackend->getPreferredHca(location); } -at::Tensor getActiveRanks(c10::intrusive_ptr backend) { +at::Tensor getActiveRanks(c10::intrusive_ptr backend) { auto mooncakeBackend = c10::static_intrusive_pointer_cast(backend); return mooncakeBackend->getActiveRanksTensor(); } -int getNumSyncedRanks(c10::intrusive_ptr backend) { +int getNumSyncedRanks(c10::intrusive_ptr backend) { auto mooncakeBackend = c10::static_intrusive_pointer_cast(backend); return mooncakeBackend->getNumSyncedRanks(); } -void extendGroupSizeTo(c10::intrusive_ptr backend, int size) { +void extendGroupSizeTo(c10::intrusive_ptr backend, + int size) { auto mooncakeBackend = c10::static_intrusive_pointer_cast(backend); mooncakeBackend->extendGroupSizeTo(size); } -std::vector getPeerState(c10::intrusive_ptr backend, +std::vector getPeerState(c10::intrusive_ptr backend, const std::vector& ranks) { auto mooncakeBackend = c10::static_intrusive_pointer_cast(backend); return mooncakeBackend->getPeerState(ranks); } -void recoverRanks(c10::intrusive_ptr backend, +void recoverRanks(c10::intrusive_ptr backend, const std::vector& ranks) { auto mooncakeBackend = c10::static_intrusive_pointer_cast(backend); mooncakeBackend->recoverRanks(ranks); } -void joinGroup(c10::intrusive_ptr backend) { +void joinGroup(c10::intrusive_ptr backend) { auto mooncakeBackend = c10::static_intrusive_pointer_cast(backend); mooncakeBackend->joinGroup(); diff --git a/mooncake-pg/tests/pg_test_utils.py b/mooncake-pg/tests/pg_test_utils.py index cd700451..49dc2cd0 100644 --- a/mooncake-pg/tests/pg_test_utils.py +++ b/mooncake-pg/tests/pg_test_utils.py @@ -148,8 +148,6 @@ def init_mooncake_group( "rank": rank, "world_size": world_size, } - if device_type == "cuda": - kwargs["device_id"] = device if use_pg_options: resolved_active_value = ( 1 if is_extension else 0 if active_value is None else active_value @@ -184,7 +182,7 @@ def init_mooncake_cpu_group( def get_mooncake_backend(group=None, device_type: str = "cpu"): if group is None: group = dist.group.WORLD - return group._get_backend(torch.device(device_type)) + return group @dataclass(slots=True) diff --git a/mooncake-pg/tests/test_pg_elastic.py b/mooncake-pg/tests/test_pg_elastic.py index 4f40d96f..f93527e3 100644 --- a/mooncake-pg/tests/test_pg_elastic.py +++ b/mooncake-pg/tests/test_pg_elastic.py @@ -18,6 +18,29 @@ from pg_test_utils import ( BROKEN_RANK = 1 +def _dynamic_world_size_worker( + ctx: MooncakePGWorkerContext, +) -> None: + """Worker for testing that dist.get_world_size() reflects dynamic size after extend.""" + initial_world_size = ctx.world_size + ctx.init_group() + backend = ctx.get_backend() + + initial_ws = dist.get_world_size() + assert initial_ws == initial_world_size, ( + f"rank {ctx.rank}: initial world_size={initial_ws}, expected {initial_world_size}" + ) + + pg.extend_group_size_to(backend, initial_world_size + 1) + + new_ws = dist.get_world_size() + assert new_ws == initial_world_size + 1, ( + f"rank {ctx.rank}: after extend world_size={new_ws}, expected {initial_world_size + 1}" + ) + + ctx.record_result({"initial_ws": initial_ws, "new_ws": new_ws}) + + def _extension_worker( ctx: MooncakePGWorkerContext, extend_event: mp.Event, @@ -187,6 +210,17 @@ class _ElasticMixin: world_size = 4 spawn_timeout_s = 30.0 + def test_dynamic_world_size(self) -> None: + """Test that dist.get_world_size() returns updated value after extend_group_size_to.""" + rows = self.spawn_backend_and_collect( + _dynamic_world_size_worker, + timeout_s=30.0, + ) + + self.assert_all_ok(rows) + for row in rows: + self.assertEqual(row["new_ws"], self.world_size + 1) + def test_failed_rank(self) -> None: """Test that survivors can continue collective after a rank fails.""" spawn_ctx = mp.get_context("spawn") diff --git a/mooncake-wheel/mooncake/mooncake_ep_buffer.py b/mooncake-wheel/mooncake/mooncake_ep_buffer.py index c37502b5..03beacac 100644 --- a/mooncake-wheel/mooncake/mooncake_ep_buffer.py +++ b/mooncake-wheel/mooncake/mooncake_ep_buffer.py @@ -71,9 +71,9 @@ class Buffer: self.group = group self.num_ep_buffer_bytes = num_ep_buffer_bytes # Get the index of the closest NIC - self.backend = self.group._get_backend(torch.device("cuda")) + self.backend = self.group preferred_hca = pg.get_preferred_hca( - self.backend, f"cuda:{torch.cuda.current_device()}" + self.group, f"cuda:{torch.cuda.current_device()}" ) self.runtime = ep.Buffer( self.rank, self.group_size, num_ep_buffer_bytes, preferred_hca diff --git a/mooncake-wheel/tests/test_mooncake_backend_elastic.py b/mooncake-wheel/tests/test_mooncake_backend_elastic.py index 249e7e98..5238285a 100644 --- a/mooncake-wheel/tests/test_mooncake_backend_elastic.py +++ b/mooncake-wheel/tests/test_mooncake_backend_elastic.py @@ -41,7 +41,7 @@ def _elastic_worker(rank, num_processes, signals): if rank == 0: signals["extend"] = 1 - backend = dist.group.WORLD._get_backend(TEST_DEVICE) + backend = dist.group.WORLD # Extend world # Note: `extend_group_size_to` is non-blocking. Blocking will only # occur at the first communication if some peers have not yet connected. @@ -64,6 +64,10 @@ def _elastic_worker(rank, num_processes, signals): f"Rank {rank} expected {sum(range(1, num_processes + 1))}, " f"get {tensor.item()}" ) + assert dist.get_world_size() == num_processes, ( + f"Rank {rank}: expected world_size={num_processes} after extension, " + f"got {dist.get_world_size()}" + ) def _deferred_recovery_worker(rank, num_processes, signals): @@ -75,7 +79,7 @@ def _deferred_recovery_worker(rank, num_processes, signals): rank=rank, world_size=num_processes, ) - backend = dist.group.WORLD._get_backend(TEST_DEVICE) + backend = dist.group.WORLD while pg.get_num_synced_ranks(backend) < num_processes: time.sleep(0.1) if rank == broken_rank: @@ -124,7 +128,7 @@ def _deferred_recovery_worker(rank, num_processes, signals): ), ) - backend = dist.group.WORLD._get_backend(TEST_DEVICE) + backend = dist.group.WORLD # Deferred join starts in a local-only mode so collectives stay self-contained. tensor = torch.tensor([broken_rank], dtype=torch.int32, device=TEST_DEVICE) -- 2.34.1 From 44cde29c845f865cda0a3521b842459bcaa1f519 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E9=B9=A4=E7=94=B7?= Date: Wed, 6 May 2026 23:50:00 +0800 Subject: [PATCH 042/382] [TE] fix(efa): request libfabric API 1.18 so device RDMA is the default on all EFA generations (#2041) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Mooncake's fi_getinfo() requests FI_VERSION(1, 14). The EFA provider's efa_rdm_get_use_device_rdma() keeps a legacy compatibility branch for callers on API < 1.18 that hardcodes the default for FI_EFA_USE_DEVICE_RDMA based on vendor_part_id: efa0/efa1 → false, everything newer → true. Under that legacy branch we silently disabled device RDMA on Nitro v4 EFA hardware (p5.48xlarge, p5e.48xlarge — vendor_part_id 0xefa1) while leaving it enabled on Nitro v5+ (p5en and later). Cross-node transfer_engine_bench reproduces the split: p5en runs fine out of the box, p5/p5e segfault inside libfabric.so during fi_cq_read once the handshake wave finishes and the first real fi_writes start. The crashing stack has a concurrent fi_av_insert in flight on the same EfaContext, and the crashing memcpy is in the provider's emulated-RDMA CQE reconstruction path — i.e. a thread-safety regression in libfabric 2.4.0's emulated RDMA data path that only ever runs when device RDMA is off. Request API 1.18. With the 1.18+ code path the default becomes hw_support (unconditionally true on every EFA hardware that supports RDMA, which is every Mooncake target platform starting from p4d), so p5/p5e pick up the same device-RDMA default p5en already has, the emulated path is never entered, and the segfault is gone. Applications that still want the emulated path can opt out with FI_EFA_USE_DEVICE_RDMA=0. 1.18 is from March 2023 and is the oldest libfabric shipped with EFA installer 1.26 and up; every Mooncake EFA deployment today runs libfabric ≥ 2.0, so bumping the requested API costs nothing in compatibility. Verified by transfer_engine_bench on two p5.48xlarge nodes (libfabric 2.4.0amzn1.0, 32 EFA NICs, 8 × H100 per node): - Before this patch, no env: target SIGSEGV inside libfabric.so on first real transfer. - Before this patch, FI_EFA_USE_DEVICE_RDMA=1: 377.74 GB/s, stable. - After this patch, no env: 377.93 GB/s, stable — matches the explicit env case. Co-authored-by: Claude Opus 4.7 --- .../transport/efa_transport/efa_context.cpp | 24 +++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp index 5ab37636..b20787b0 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp @@ -89,9 +89,29 @@ int EfaContext::construct(size_t num_cq_list, size_t max_cqe, ; hints_->domain_attr->threading = FI_THREAD_SAFE; - // Get fabric info + // Get fabric info. + // + // Request libfabric API 1.18+ so the EFA provider's + // efa_rdm_get_use_device_rdma() takes the "new API" branch and keys + // the default for FI_EFA_USE_DEVICE_RDMA on hardware capability + // (hw_support) instead of vendor_part_id. Under the older 1.14 + // request, the provider's legacy branch hardcoded + // default_val = (vendor_part_id == 0xefa0 || 0xefa1) ? false : true + // which silently disabled device RDMA on Nitro v4 EFA (p5/p5e, part + // id 0xefa1) while leaving it enabled on Nitro v5+ (p5en and newer). + // With device RDMA disabled, fi_write falls back to libfabric's + // emulated RDMA data path, and libfabric 2.4.0 has a thread-safety + // regression there between fi_av_insert and concurrent fi_cq_read + // that segfaults Mooncake once the handshake wave finishes and the + // first real transfers start. Bumping the requested API to 1.18 + // restores the same default path we already got on newer hardware, + // and applications that still want emulated RDMA can opt out with + // FI_EFA_USE_DEVICE_RDMA=0. + // + // 1.18 is from March 2023 (EFA installer 1.26+ ships 1.18 or later); + // all Mooncake deployments today run libfabric >> 1.18. int ret = - fi_getinfo(FI_VERSION(1, 14), nullptr, nullptr, 0, hints_, &fi_info_); + fi_getinfo(FI_VERSION(1, 18), nullptr, nullptr, 0, hints_, &fi_info_); if (ret) { LOG(ERROR) << "fi_getinfo failed for device " << device_name_ << ": " << fi_strerror(-ret); -- 2.34.1 From ea8fa5dad99a71503fe7c2ee8a6b396d2ec43dbe Mon Sep 17 00:00:00 2001 From: dtc Date: Wed, 6 May 2026 23:52:07 +0800 Subject: [PATCH 043/382] [TE] fix rdma race (#1903) Signed-off-by: Tianchen Ding --- .../transport/rdma_transport/rdma_endpoint.h | 26 +++- .../rdma_transport/endpoint_store.cpp | 32 ++-- .../rdma_transport/rdma_endpoint.cpp | 141 ++++++++++++++++-- .../transport/rdma_transport/worker_pool.cpp | 44 +++--- 4 files changed, 195 insertions(+), 48 deletions(-) diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h index 662413ad..089f2c18 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h @@ -43,6 +43,8 @@ class RdmaEndPoint { UNCONNECTED, CONNECTING, CONNECTED, + DESTROYING, + DESTROYED, }; public: @@ -56,6 +58,7 @@ class RdmaEndPoint { private: int reconstruct(); int deconstruct(); + int deconstructLocked(); public: void setPeerNicPath(const std::string &peer_nic_path); @@ -71,8 +74,6 @@ class RdmaEndPoint { int setupConnectionsByPassive(const HandShakeDesc &peer_desc, HandShakeDesc &local_desc); - bool hasOutstandingSlice() const; - bool active() const { return active_; } void set_active(bool flag) { @@ -99,6 +100,16 @@ class RdmaEndPoint { // Destroy QPs before CQs (in RDMA Context) int destroyQP(); + // Two-phase QP destruction to avoid use-after-free in concurrent + // submitPostSend. Phase 1 (beginDestroy): sets active_=false and + // status_=DESTROYING, transitions QPs to ERR state so hardware flushes + // inflight WRs to CQ. Does not block. Phase 2 (finishDestroy): called + // after all outstanding WRs have been drained (wr_depth_list_ all zero), + // actually destroys QPs and frees resources. Returns true if destruction + // is complete, false if outstanding WRs remain. + void beginDestroy(); + bool finishDestroy(); + private: int disconnectUnlocked(); @@ -154,6 +165,16 @@ class RdmaEndPoint { static constexpr uint32_t kWaitExistingHandshakeInitialSleepUs = 50; static constexpr uint32_t kWaitExistingHandshakeMaxSleepUs = 2000; + // Maximum time (in seconds) to wait for outstanding WRs to drain in + // finishDestroy before forcing QP destruction. This guards against + // ibv_modify_qp-to-ERR failures that prevent WR flushing. + static constexpr double kFinishDestroyTimeoutSec = 30.0; + + // Maximum number of deconstructLocked retries in finishDestroy before + // giving up and marking the endpoint as DESTROYED. Prevents infinite + // retry loops and log flooding when ibv_destroy_qp fails permanently. + static constexpr int kFinishDestroyMaxRetries = 3; + RdmaContext &context_; std::atomic status_; @@ -171,6 +192,7 @@ class RdmaEndPoint { volatile bool active_; volatile int *cq_outstanding_; volatile uint64_t inactive_time_; + int finish_destroy_retries_ = 0; }; } // namespace mooncake diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/endpoint_store.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/endpoint_store.cpp index ce4dae95..6c730662 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/endpoint_store.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/endpoint_store.cpp @@ -78,12 +78,14 @@ std::shared_ptr FIFOEndpointStore::insertEndpoint( int FIFOEndpointStore::deleteEndpoint(const std::string &peer_nic_path) { RWSpinlock::WriteGuard guard(endpoint_map_lock_); auto iter = endpoint_map_.find(peer_nic_path); - // remove endpoint but leaving it status unchanged - // in case it is setting up connection or submitting slice + // Begin two-phase destruction: mark endpoint as destroying and move QPs + // to ERR state so inflight WRs are flushed to CQ. The endpoint is moved + // to waiting_list_ and will be fully destroyed by reclaimEndpoint() once + // all outstanding WRs have been drained. if (iter != endpoint_map_.end()) { waiting_list_len_++; + iter->second->beginDestroy(); waiting_list_.insert(iter->second); - iter->second->set_active(false); endpoint_map_.erase(iter); auto fifo_iter = fifo_map_[peer_nic_path]; fifo_list_.erase(fifo_iter); @@ -99,7 +101,9 @@ void FIFOEndpointStore::evictEndpoint() { fifo_map_.erase(victim); LOG(INFO) << victim << " evicted"; waiting_list_len_++; - waiting_list_.insert(endpoint_map_[victim]); + auto victim_endpoint = endpoint_map_[victim]; + victim_endpoint->beginDestroy(); + waiting_list_.insert(victim_endpoint); endpoint_map_.erase(victim); return; } @@ -108,8 +112,9 @@ void FIFOEndpointStore::reclaimEndpoint() { if (waiting_list_len_.load(std::memory_order_relaxed) == 0) return; RWSpinlock::WriteGuard guard(endpoint_map_lock_); std::vector> to_delete; - for (auto &endpoint : waiting_list_) - if (!endpoint->hasOutstandingSlice()) to_delete.push_back(endpoint); + for (auto &endpoint : waiting_list_) { + if (endpoint->finishDestroy()) to_delete.push_back(endpoint); + } for (auto &endpoint : to_delete) waiting_list_.erase(endpoint); waiting_list_len_ -= to_delete.size(); } @@ -203,12 +208,14 @@ std::shared_ptr SIEVEEndpointStore::insertEndpoint( int SIEVEEndpointStore::deleteEndpoint(const std::string &peer_nic_path) { RWSpinlock::WriteGuard guard(endpoint_map_lock_); auto iter = endpoint_map_.find(peer_nic_path); - // remove endpoint but leaving it status unchanged - // in case it is setting up connection or submitting slice + // Begin two-phase destruction: mark endpoint as destroying and move QPs + // to ERR state so inflight WRs are flushed to CQ. The endpoint is moved + // to waiting_list_ and will be fully destroyed by reclaimEndpoint() once + // all outstanding WRs have been drained. if (iter != endpoint_map_.end()) { + iter->second.first->beginDestroy(); waiting_list_len_++; waiting_list_.insert(iter->second.first); - iter->second.first->set_active(false); endpoint_map_.erase(iter); auto fifo_iter = fifo_map_[peer_nic_path]; if (hand_.has_value() && hand_.value() == fifo_iter) { @@ -242,7 +249,7 @@ void SIEVEEndpointStore::evictEndpoint() { fifo_map_.erase(victim); LOG(INFO) << victim << " evicted"; auto victim_instance = endpoint_map_[victim].first; - victim_instance->set_active(false); + victim_instance->beginDestroy(); waiting_list_len_++; waiting_list_.insert(victim_instance); endpoint_map_.erase(victim); @@ -253,8 +260,9 @@ void SIEVEEndpointStore::reclaimEndpoint() { if (waiting_list_len_.load(std::memory_order_relaxed) == 0) return; RWSpinlock::WriteGuard guard(endpoint_map_lock_); std::vector> to_delete; - for (auto &endpoint : waiting_list_) - if (!endpoint->hasOutstandingSlice()) to_delete.push_back(endpoint); + for (auto &endpoint : waiting_list_) { + if (endpoint->finishDestroy()) to_delete.push_back(endpoint); + } for (auto &endpoint : to_delete) waiting_list_.erase(endpoint); waiting_list_len_ -= to_delete.size(); } diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp index c2c3af81..e39a51d0 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp @@ -37,7 +37,13 @@ RdmaEndPoint::RdmaEndPoint(RdmaContext &context) cq_outstanding_(nullptr) {} RdmaEndPoint::~RdmaEndPoint() { - if (!qp_list_.empty()) deconstruct(); + if (!qp_list_.empty()) { + // In normal flow, beginDestroy()+finishDestroy() should have been + // called already via endpoint_store. This is a fallback for abnormal + // shutdown (e.g., process exit). + RWSpinlock::WriteGuard guard(lock_); + deconstructLocked(); + } } int RdmaEndPoint::construct(ibv_cq *cq, size_t num_qp_list, @@ -91,7 +97,8 @@ int RdmaEndPoint::reconstruct() { auto max_inline_bytes = max_inline_bytes_; // Deconstruct and reconstruct to get fresh QPs (same as delete+create) - int ret = deconstruct(); + // Use deconstructLocked because callers already hold lock_ + int ret = deconstructLocked(); if (ret) { LOG(ERROR) << "Failed to deconstruct endpoint: " << ret; return ret; @@ -113,13 +120,15 @@ int RdmaEndPoint::reconstruct() { } int RdmaEndPoint::deconstruct() { + RWSpinlock::WriteGuard guard(lock_); + return deconstructLocked(); +} + +int RdmaEndPoint::deconstructLocked() { + // Adjust cq_outstanding_ before destroying QPs, so the counter is + // always corrected even if ibv_destroy_qp fails and we return early. + bool displayed = false; for (size_t i = 0; i < qp_list_.size(); ++i) { - if (ibv_destroy_qp(qp_list_[i])) { - PLOG(ERROR) << "Failed to destroy QP"; - return ERR_ENDPOINT; - } - // After destroying QP, the wr_depth_list_ won't change - bool displayed = false; if (wr_depth_list_[i] != 0) { if (!displayed) { LOG(WARNING) << "Outstanding work requests found, CQ will not " @@ -130,14 +139,112 @@ int RdmaEndPoint::deconstruct() { wr_depth_list_[i] = 0; } } + + int result = 0; + for (size_t i = 0; i < qp_list_.size(); ++i) { + if (!qp_list_[i]) continue; // already destroyed in a previous call + if (ibv_destroy_qp(qp_list_[i])) { + PLOG(ERROR) << "Failed to destroy QP[" << i << "]"; + result = ERR_ENDPOINT; + } else { + qp_list_[i] = nullptr; + } + } + + if (result) return result; + qp_list_.clear(); peer_qp_num_list_.clear(); delete[] wr_depth_list_; + wr_depth_list_ = nullptr; return 0; } int RdmaEndPoint::destroyQP() { return deconstruct(); } +void RdmaEndPoint::beginDestroy() { + RWSpinlock::WriteGuard guard(lock_); + auto current_status = status_.load(std::memory_order_relaxed); + if (current_status == DESTROYING || current_status == DESTROYED) return; + + active_ = false; + inactive_time_ = getCurrentTimeInNano(); + status_.store(DESTROYING, std::memory_order_release); + + // Transition QPs to ERR state so hardware flushes all inflight WRs to CQ. + // This allows performPollCq to drain them naturally. + ibv_qp_attr attr; + memset(&attr, 0, sizeof(attr)); + attr.qp_state = IBV_QPS_ERR; + for (size_t i = 0; i < qp_list_.size(); ++i) { + if (ibv_modify_qp(qp_list_[i], &attr, IBV_QP_STATE)) { + PLOG(WARNING) << "Failed to modify QP to ERR during beginDestroy"; + } + } +} + +bool RdmaEndPoint::finishDestroy() { + RWSpinlock::WriteGuard guard(lock_); + auto current_status = status_.load(std::memory_order_relaxed); + + // Gate 1: already done. + if (current_status == DESTROYED) return true; + + // Gate 2: non-two-phase path (status != DESTROYING). The endpoint + // reached waiting_list_ without going through beginDestroy(). This is + // the contract expected by EndpointStore::testOnlyInsertWaiting() and + // serves as a safety net for any future non-two-phase path. Mirror the + // pre-two-phase predicate (!hasOutstandingSlice == !active_): only + // inactive endpoints are eligible for reclaim; active ones must stay. + if (current_status != DESTROYING) { + if (active_) return false; + // Endpoints that never reached construct() own no RDMA resources + // and have wr_depth_list_ uninitialized; deconstructLocked() would + // delete[] a wild pointer. Drop them directly. + if (qp_list_.empty()) { + status_.store(DESTROYED, std::memory_order_relaxed); + return true; + } + LOG(WARNING) << "finishDestroy called in unexpected state: " + << current_status + << ", forcing destruction to avoid waiting_list_ leak"; + // Fall through to the unified destroy path. + } else { + // Gate 3: two-phase path. Wait for inflight WRs to drain via CQ + // polling. If ibv_modify_qp-to-ERR failed in beginDestroy, WRs may + // never be flushed; enforce a timeout to avoid leaking forever. + bool has_outstanding = false; + for (size_t i = 0; i < qp_list_.size(); ++i) { + if (wr_depth_list_[i] != 0) { + has_outstanding = true; + break; + } + } + if (has_outstanding) { + double elapsed = (getCurrentTimeInNano() - inactive_time_) / 1e9; + if (elapsed < kFinishDestroyTimeoutSec) return false; + LOG(WARNING) << "finishDestroy timed out after " << elapsed + << "s with outstanding WRs, forcing destruction"; + } + } + + // Unified destroy: tear down QPs (deconstructLocked handles + // cq_outstanding_ adjustment internally) and bound retries to avoid + // log flooding when ibv_destroy_qp fails permanently. + int ret = deconstructLocked(); + if (ret) { + finish_destroy_retries_++; + LOG(ERROR) << "Failed to finish destroying endpoint (attempt " + << finish_destroy_retries_ << "/" << kFinishDestroyMaxRetries + << "): " << ret; + if (finish_destroy_retries_ < kFinishDestroyMaxRetries) return false; + LOG(ERROR) << "Giving up after " << finish_destroy_retries_ + << " retries (possible resource leak)"; + } + status_.store(DESTROYED, std::memory_order_relaxed); + return true; +} + void RdmaEndPoint::setPeerNicPath(const std::string &peer_nic_path) { RWSpinlock::WriteGuard guard(lock_); if (connected()) { @@ -459,22 +566,24 @@ const std::string RdmaEndPoint::toString() const { if (status == CONNECTED) return "EndPoint: local " + context_.nicPath() + ", peer " + peer_nic_path_; + else if (status == DESTROYING) + return "EndPoint: local " + context_.nicPath() + ", peer " + + peer_nic_path_ + " (destroying)"; + else if (status == DESTROYED) + return "EndPoint: local " + context_.nicPath() + " (destroyed)"; else return "EndPoint: local " + context_.nicPath() + " (unconnected)"; } -bool RdmaEndPoint::hasOutstandingSlice() const { - if (active_) return true; - for (size_t i = 0; i < qp_list_.size(); i++) - if (wr_depth_list_[i] != 0) return true; - return false; -} - int RdmaEndPoint::submitPostSend( std::vector &slice_list, std::vector &failed_slice_list) { RWSpinlock::WriteGuard guard(lock_); - if (!active_) return 0; + if (!active_ || status_.load(std::memory_order_relaxed) != CONNECTED) { + for (auto &slice : slice_list) failed_slice_list.push_back(slice); + slice_list.clear(); + return 0; + } const size_t num_qp = qp_list_.size(); if (slice_list.empty()) return 0; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp index 99ab93b6..788edf6b 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp @@ -288,22 +288,32 @@ void WorkerPool::performPollCq(int thread_id) { qp_depth_set[slice->rdma.qp_depth] = 1; // __sync_fetch_and_sub(slice->rdma.qp_depth, 1); if (wc[i].status != IBV_WC_SUCCESS) { - bool show_work_request_flushed_error = globalConfig().trace; - // After detect an error, subsequent work requests will result - // in work_request_flushed_error, we hide this by default - if (wc[i].status != IBV_WC_WR_FLUSH_ERR || - show_work_request_flushed_error) - LOG(ERROR) - << "Worker: Process failed for slice (opcode: " - << slice->opcode - << ", source_addr: " << slice->source_addr - << ", length: " << slice->length - << ", dest_addr: " << (void *)slice->rdma.dest_addr - << ", local_nic: " << context_.deviceName() - << ", peer_nic: " << slice->peer_nic_path - << ", dest_rkey: " << slice->rdma.dest_rkey - << ", retry_cnt: " << slice->rdma.retry_cnt - << "): " << ibv_wc_status_str(wc[i].status); + // Flush errors are generated when QPs transition to ERR + // state (e.g., during two-phase endpoint destruction via + // beginDestroy). They are not real network errors, so we + // directly mark the slice as failed without retry, without + // counting toward the RNIC error threshold, and without + // triggering endpoint deletion. + if (wc[i].status == IBV_WC_WR_FLUSH_ERR) { + if (globalConfig().trace) + LOG(INFO) + << "Worker: WR flush error (peer_nic: " + << slice->peer_nic_path << "), marking failed"; + slice->markFailed(); + processed_slice_count++; + continue; + } + + LOG(ERROR) << "Worker: Process failed for slice (opcode: " + << slice->opcode + << ", source_addr: " << slice->source_addr + << ", length: " << slice->length + << ", dest_addr: " << (void *)slice->rdma.dest_addr + << ", local_nic: " << context_.deviceName() + << ", peer_nic: " << slice->peer_nic_path + << ", dest_rkey: " << slice->rdma.dest_rkey + << ", retry_cnt: " << slice->rdma.retry_cnt + << "): " << ibv_wc_status_str(wc[i].status); failed_nr_polls++; if (context_.active() && failed_nr_polls > 32 && !success_nr_polls) { @@ -320,8 +330,6 @@ void WorkerPool::performPollCq(int thread_id) { collective_slice_queue_[thread_id][slice->peer_nic_path] .push_back(slice); redispatch_counter_++; - // std::vector slice_list { slice }; - // redispatch(slice_list, thread_id); } } else { slice->markSuccess(); -- 2.34.1 From 93d53846d17fd46629395b830b3e5afd5b62d918 Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Thu, 7 May 2026 11:03:24 +0800 Subject: [PATCH 044/382] [doc] vllm MooncakeStoreConnector (#2042) --- .../examples/vllm-integration/index.md | 1 + .../vllm-mooncakestoreconnector.md | 128 ++++++++++++++++++ 2 files changed, 129 insertions(+) create mode 100644 docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md diff --git a/docs/source/getting_started/examples/vllm-integration/index.md b/docs/source/getting_started/examples/vllm-integration/index.md index c1ac0c84..8ead4680 100644 --- a/docs/source/getting_started/examples/vllm-integration/index.md +++ b/docs/source/getting_started/examples/vllm-integration/index.md @@ -3,6 +3,7 @@ ::::{toctree} :maxdepth: 1 +vllm-mooncakestoreconnector vllmv1-lmcache-integration vllm-integration-v0.2 vllm-integration-v0.3 diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md b/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md new file mode 100644 index 00000000..50a87f05 --- /dev/null +++ b/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md @@ -0,0 +1,128 @@ +# Guide: vLLM MooncakeStoreConnector + +## Overview + +This document describes how to deploy vLLM's `MooncakeStoreConnector`. `MooncakeStoreConnector` is a new vLLM's KV connector that uses `MooncakeDistributedStore` as a shared KV cache pool. It enables: + +* **CPU/Disk offloading**: Extend effective KV cache capacity by offloading to CPU memory or SSD via Mooncake's transfer engine. +* **Hash-based prefix caching across instances**: Multiple vLLM instances share cached KV blocks through the store using block-hash deduplication. +* **Flexible deployment**: Works as a single-node KV cache extension (`kv_both`), or in disaggregated prefill-decode setups (`kv_producer` / `kv_consumer`). + + + +## Deployment + +### 1. Prerequisites + +Before you begin, make sure that: + +* [vLLM](https://github.com/vllm-project/vllm) is installed, [Mooncake](https://github.com/kvcache-ai/Mooncake) is installed. Refer to the [vLLM official repository](https://github.com/vllm-project/vllm) and [Mooncake official repository](https://github.com/kvcache-ai/Mooncake) for more installation instructions and building from source. + + + +### 2. Mooncake Master Server + +**Start:** + +```shell +mooncake_master --port 50063 +``` + +**Configure Mooncake** : Create a JSON configuration file (e.g., `mooncake_config.json`): + +```json +{ + "metadata_server": "http://127.0.0.1:8092/metadata", + "master_server_address": "127.0.0.1:50063", + "global_segment_size": "0", + "local_buffer_size": "2147483648", + "protocol": "rdma", + "device_name": "", +} +``` + +**Set environment variable:** + +```shell +export MOONCAKE_CONFIG_PATH=/path/to/mooncake_config.json +``` + + + +### 3. Usage + +**3.1** **Single-Node KV Cache Offloading** (i.e., `kv_both`) + +```shell +MOONCAKE_CONFIG_PATH=mooncake_config.json \ +vllm serve meta-llama/Llama-3.1-8B-Instruct \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_both"}' +``` + +**3.2** **XpYd Disaggregated Prefill-Decode** (i.e., `kv_producer/kv_consumer`) + +Prefill Node: + +```shell +MOONCAKE_CONFIG_PATH=mooncake_config.json \ +VLLM_MOONCAKE_BOOTSTRAP_PORT=50052 \ +vllm serve meta-llama/Llama-3.1-8B-Instruct \ + --port 8100 \ + --kv-transfer-config '{ + "kv_connector": "MultiConnector", + "kv_role": "kv_producer", + "kv_connector_extra_config": { + "connectors": [ + { + "kv_connector": "MooncakeConnector", + "kv_role": "kv_producer" + }, + { + "kv_connector": "MooncakeStoreConnector", + "kv_role": "kv_producer" + } + ] + } + }' +``` + +Decode Node: + +```shell +MOONCAKE_CONFIG_PATH=mooncake_config.json \ +VLLM_MOONCAKE_BOOTSTRAP_PORT=50053 \ +vllm serve meta-llama/Llama-3.1-8B-Instruct \ + --port 8200 \ + --kv-transfer-config '{ + "kv_connector": "MultiConnector", + "kv_role": "kv_consumer", + "kv_connector_extra_config": { + "connectors": [ + { + "kv_connector": "MooncakeConnector", + "kv_role": "kv_consumer" + }, + { + "kv_connector": "MooncakeStoreConnector", + "kv_role": "kv_consumer" + } + ] + } + }' +``` + +Proxy: + +```shell +python examples/disaggregated/disaggregated_serving/mooncake_connector/mooncake_connector_proxy.py --prefill http://192.168.0.2:8100 --decode http://192.168.0.3:8200 +``` + + + +> When running with data parallelism, set a fixed `PYTHONHASHSEED` so that block hashes are consistent across DP ranks: +> +> ```shell +> PYTHONHASHSEED=0 vllm serve ... +> ``` +> +> Without this, identical prompts may produce different block hashes on different DP ranks, preventing cross-instance prefix cache hits. -- 2.34.1 From c3bafbedd164a6ab6fb1583c6af1d9872d6e6d51 Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Thu, 7 May 2026 11:59:28 +0800 Subject: [PATCH 045/382] [PG] Fix P2P regression caused by ProcessGroup inheritance (#2043) * [PG] Fix P2P regression by registering Backend shim in ProcessGroup After PR #2040 changed MooncakeBackend to inherit from ProcessGroup instead of Backend, PyTorch's P2P dispatch path (batch_isend_irecv, isend, irecv) broke because getBackend() could not find a registered Backend instance in the ProcessGroup's deviceTypeToBackend_ map. Introduce MooncakeP2PShim, a lightweight Backend subclass that holds a non-owning pointer back to its owning MooncakeBackend and delegates send/recv operations. The shim is registered in the constructor via setBackend() so that _get_backend()/getBackend() lookups succeed. * [PG] Forward-declare MooncakeBackend to fix CUDA compilation The MooncakeP2PShim constructor in the header referenced MooncakeBackend before it was defined. Move the constructor definition to the .cpp and add a forward declaration. * [PG] Set default backend type to CUSTOM in ProcessGroup Without setDefaultBackend(CUSTOM), hasHooks() looks up backendType_ which defaults to UNDEFINED (0), producing a noisy warning on every P2P operation. * [PG] Clean up MooncakeP2PShim: remove unused deviceType_ and barrier dummy tensor - Remove deviceType_ member: it was stored but never used in any method. - Remove unused dummy tensor allocation in barrier(): it was created but never passed to the underlying barrier call. * [PG] Mark MooncakeP2PShim constructor explicit * [PG] Fix clang-format: split getBackendName return type to satisfy ColumnLimit 80 * [PG] Fix clang-format: single-line getBackendName (80 cols exactly) --- mooncake-pg/include/mooncake_backend.h | 38 ++++++++++++++++++++++++++ mooncake-pg/src/mooncake_backend.cpp | 37 +++++++++++++++++++++++++ 2 files changed, 75 insertions(+) diff --git a/mooncake-pg/include/mooncake_backend.h b/mooncake-pg/include/mooncake_backend.h index 5fa8bcc1..93549bc4 100644 --- a/mooncake-pg/include/mooncake_backend.h +++ b/mooncake-pg/include/mooncake_backend.h @@ -16,6 +16,44 @@ namespace mooncake { +// Forward declaration – MooncakeP2PShim holds a non-owning pointer to +// MooncakeBackend, which is defined below. +class MooncakeBackend; + +// Lightweight Backend shim that delegates P2P send/recv back to the owning +// MooncakeBackend. PyTorch's P2P dispatch (batch_isend_irecv, isend, irecv) +// requires getBackend() to return a registered c10d::Backend instance. +// Since MooncakeBackend inherits from ProcessGroup (not Backend), we register +// this shim in the ProcessGroup's deviceTypeToBackend_ map so that the P2P +// path can find it. The shim holds a non-owning pointer to its owner and +// delegates only the operations that the P2P dispatch path calls (send, recv, +// getBackendName, supportsCoalescing). +class MooncakeP2PShim final : public ::c10d::Backend { + public: + explicit MooncakeP2PShim(MooncakeBackend* owner); + + const std::string getBackendName() const override; + + bool supportsCoalescing() const override { return false; } + + c10::intrusive_ptr send(std::vector& tensors, + int dstRank, int tag) override; + + c10::intrusive_ptr recv(std::vector& tensors, + int srcRank, int tag) override; + + c10::intrusive_ptr recvAnysource( + std::vector& tensors, int tag) override; + + c10::intrusive_ptr barrier( + const c10d::BarrierOptions& opts) override; + + private: + // Non-owning: the shim is stored in ProcessGroup's backend maps which are + // cleared on destruction, and MooncakeBackend always outlives the shim. + MooncakeBackend* owner_; +}; + class MooncakeBackend final : public ::c10d::ProcessGroup { public: struct MooncakeBackendOptions final : torch::CustomClassHolder { diff --git a/mooncake-pg/src/mooncake_backend.cpp b/mooncake-pg/src/mooncake_backend.cpp index ccd4dfa9..a6ad0530 100644 --- a/mooncake-pg/src/mooncake_backend.cpp +++ b/mooncake-pg/src/mooncake_backend.cpp @@ -295,6 +295,14 @@ MooncakeBackend::MooncakeBackend( connection_ctx_->waitUntilAllConnected(); } + // Register a lightweight Backend shim so that PyTorch's P2P dispatch path + // (batch_isend_irecv → _get_backend → getBackend) can find a registered + // Backend for this ProcessGroup. The shim delegates send/recv back to us. + auto deviceType = isCpu ? c10::DeviceType::CPU : c10::DeviceType::CUDA; + auto shim = c10::make_intrusive(this); + setBackend(deviceType, BackendType::CUSTOM, shim); + setDefaultBackend(BackendType::CUSTOM); + // Increment backend index ++backendIndex_; } @@ -303,6 +311,35 @@ MooncakeBackend::~MooncakeBackend() { shutdown(); } const std::string MooncakeBackend::getBackendName() const { return "mooncake"; } +// ---- MooncakeP2PShim implementation ---- + +MooncakeP2PShim::MooncakeP2PShim(MooncakeBackend* owner) + : Backend(owner->getRank(), owner->getSize()), owner_(owner) {} + +const std::string MooncakeP2PShim::getBackendName() const { return "mooncake"; } + +c10::intrusive_ptr MooncakeP2PShim::send( + std::vector& tensors, int dstRank, int tag) { + return owner_->send(tensors, dstRank, tag); +} + +c10::intrusive_ptr MooncakeP2PShim::recv( + std::vector& tensors, int srcRank, int tag) { + return owner_->recv(tensors, srcRank, tag); +} + +c10::intrusive_ptr MooncakeP2PShim::recvAnysource( + std::vector& tensors, int tag) { + // MooncakeBackend doesn't implement recvAnysource; fall back to + // the base class which will raise a clear error. + return ::c10d::Backend::recvAnysource(tensors, tag); +} + +c10::intrusive_ptr MooncakeP2PShim::barrier( + const c10d::BarrierOptions& opts) { + return owner_->barrier(opts); +} + c10::intrusive_ptr MooncakeBackend::send( std::vector& tensors, int dstRank, int tag) { connection_ctx_->waitUntilNewRanksConnected(); -- 2.34.1 From 84df45a906725848a4a80d640ba146dc51c7272f Mon Sep 17 00:00:00 2001 From: Zhanhao Cao Date: Thu, 7 May 2026 12:17:38 +0800 Subject: [PATCH 046/382] [PG] Refactor P2PProxy with shared chunk pools and receiver-driven credit-based flow control (#1971) * [PG] Refactor P2P: Chunk pool + Receiver-driven pipeline * [PG] Refactor P2P State Machine for better readability. * [PG] Per-peer generation and better failure handling. * [PG] Parse pool configuration from environment. * fix typos and refine comments. * apply gemini-code-assist's suggestions. * [PG] Share P2PChunkPool across backends. * [PG] Fix poor naming and address gemini's review comments. --- mooncake-pg/include/connection_poller.h | 10 +- mooncake-pg/include/mooncake_backend.h | 8 + mooncake-pg/include/mooncake_worker.cuh | 6 +- mooncake-pg/include/p2p_proxy.h | 590 +++++++-- mooncake-pg/src/connection_poller.cpp | 6 +- mooncake-pg/src/mooncake_backend.cpp | 224 +++- mooncake-pg/src/p2p_proxy.cpp | 1601 +++++++++++++++-------- 7 files changed, 1669 insertions(+), 776 deletions(-) diff --git a/mooncake-pg/include/connection_poller.h b/mooncake-pg/include/connection_poller.h index a386439e..d5e3c53c 100644 --- a/mooncake-pg/include/connection_poller.h +++ b/mooncake-pg/include/connection_poller.h @@ -184,14 +184,8 @@ class ConnectionContext { return "buffer_" + std::to_string(backendIndex) + "_" + std::to_string(rank); } - static std::string getExtensionTaskCountStoreKey(int backendIndex, - int rank) { - return "extension_task_count_" + std::to_string(backendIndex) + "_" + - std::to_string(rank); - } - static std::string getExtensionActiveRanksStoreKey(int backendIndex, - int rank) { - return "extension_active_ranks_" + std::to_string(backendIndex) + "_" + + static std::string getExtensionStateStoreKey(int backendIndex, int rank) { + return "extension_state_" + std::to_string(backendIndex) + "_" + std::to_string(rank); } diff --git a/mooncake-pg/include/mooncake_backend.h b/mooncake-pg/include/mooncake_backend.h index 93549bc4..70fd3503 100644 --- a/mooncake-pg/include/mooncake_backend.h +++ b/mooncake-pg/include/mooncake_backend.h @@ -232,6 +232,14 @@ class MooncakeBackend final : public ::c10d::ProcessGroup { bool connectionPollerRegistered_{false}; }; +struct ExtensionState { + std::vector activeRanks; + std::vector p2pEpochs; + int taskCount = -1; +}; +std::vector serialize(const ExtensionState& state); +ExtensionState deserialize(const std::vector& buffer); + } // namespace mooncake #endif // MOONCAKE_BACKEND_H diff --git a/mooncake-pg/include/mooncake_worker.cuh b/mooncake-pg/include/mooncake_worker.cuh index d7d41511..853c8df3 100644 --- a/mooncake-pg/include/mooncake_worker.cuh +++ b/mooncake-pg/include/mooncake_worker.cuh @@ -25,10 +25,8 @@ static constexpr size_t kMaxNumRanks = 64; struct SegmentInfo { uint64_t send_buffer[2], recv_buffer[2], send_sync[2], recv_sync[2], warmup_buffer[2]; - uint64_t p2p_send_buffer; - uint64_t p2p_recv_buffer; - uint64_t p2p_ctrl_send; - uint64_t p2p_ctrl_recv; + uint64_t p2p_credit_region; + uint64_t p2p_ack_region; }; struct TransferGroupMeta { diff --git a/mooncake-pg/include/p2p_proxy.h b/mooncake-pg/include/p2p_proxy.h index 168ebffb..69f508a7 100644 --- a/mooncake-pg/include/p2p_proxy.h +++ b/mooncake-pg/include/p2p_proxy.h @@ -1,5 +1,5 @@ -#ifndef MOONCAKE_P2P_PROXY_HH -#define MOONCAKE_P2P_PROXY_HH +#ifndef MOONCAKE_P2P_PROXY_H +#define MOONCAKE_P2P_PROXY_H #include #include @@ -9,51 +9,262 @@ #include #include #include +#include #include #include #include #include #include #include +#include namespace mooncake { -inline constexpr size_t kP2PBufferSize = 1u << 24; -inline constexpr size_t kP2PNumSlots = 8; -inline constexpr size_t kP2PSlotSize = kP2PBufferSize / kP2PNumSlots; +// Memory layout of one P2PProxy (shown for chunk_size=8MiB, num_chunks=32): +// +// +---------------------------------------------------------+ +// | SendPool (32 x 8 MiB = 256 MiB) | +// | +---------+ +---------+ +---------+ | +// | | chunk 0 | | chunk 1 | ... | chunk 31| | +// | | 8 MiB | | 8 MiB | | 8 MiB | | +// | +---------+ +---------+ +---------+ | +// +---------------------------------------------------------+ +// | RecvPool (32 x 8 MiB = 256 MiB) | +// | +---------+ +---------+ +---------+ | +// | | chunk 0 | | chunk 1 | ... | chunk 31| | +// | | 8 MiB | | 8 MiB | | 8 MiB | | +// | +---------+ +---------+ +---------+ | +// +---------------------------------------------------------+ +// | Credit Control Region | +// | [peer 0 lane : 64 slots][peer 1 lane : 64 slots] ... | +// +---------------------------------------------------------+ +// | Ack Control Region | +// | [peer 0 lane : 64 slots][peer 1 lane : 64 slots] ... | +// +---------------------------------------------------------+ +// +// Control lane addressing (isolated per sender-receiver pair): +// +// Rank R wants to write a CreditSlot to peer P +// ----------------------------------------------------- +// Target = P's credit base +// + (R * kP2PControlRingSize + seq % 64) * sizeof(CreditSlot) +// +// Rank R wants to write an AckSlot to peer P +// ----------------------------------------------------- +// Target = P's ack base +// + (R * kP2PControlRingSize + seq % 64) * sizeof(AckSlot) +// +// +// P2PProxy implements a credit-based RDMA pull protocol. +// +// Protocol overview: +// - The RECEIVER drives the flow. It allocates chunks from its RecvPool and +// writes CreditSlots into the sender's CreditLane to grant credit to the +// sender to write data into the designated RecvPool offset. +// - The SENDER is passive. It polls its local CreditLane (written by the +// receiver via RDMA). Only after receiving a credit does it allocate +// a staging buffer from SendPool, copy user tensor data into it, and +// perform the RDMA Write to the remote RecvPool. +// - After the RDMA Write finishes, the sender writes an AckSlot back +// to the receiver's AckLane to acknowledge the transfer. +// +// Memory model: +// - SendPool / RecvPool are fixed-size chunk pools. +// - Control lanes are ring buffers. +// +// Per-chunk state machines: +// Sender: Fetch Credit -> Copy-In (staging) -> RDMA Write -> Acknowledge +// Receiver: IssueCredit -> Poll Ack -> Copy-Out -> Free Chunk +// +// Concurrency rules: +// - All pool allocations are non-blocking. If a pool is exhausted the step +// function returns false immediately and retries on the next polling +// iteration. This prevents deadlocks. +// - SendPool and RecvPool are strictly separated so that a deadlock where +// "all chunks are reserved for receiving and none are left for sending" +// can never happen. +// +// Data flow example -- Rank 0 sends a 24 MiB tensor to Rank 1. +// The five steps below are separated between Sender (Rank 0) and +// Receiver (Rank 1) to make the protocol explicit. +// +// Step 1 -- Rank 1 issues credits for free RecvPool chunks to Rank 0 +// ------------------------------------------------------------------------ +// Rank 1 (Receiver) RDMA Write +// +-----------------------+ +-------------------------------+ +// | RecvPool | | Rank 0's CreditLane[1] | +// | chunk 7 : free | -----------> | slot 0 : {seq=0,off=7,len=8M} | +// | chunk 2 : free | -----------> | slot 1 : {seq=1,off=2,len=8M} | +// | chunk 1 : free | -----------> | slot 2 : {seq=2,off=1,len=8M} | +// +-----------------------+ +-------------------------------+ +// +// Step 2 -- Rank 0 stages user tensor into SendPool +// ------------------------------------------------------------------------ +// Rank 0 (Sender) +// +-----------------------+ +// | User tensor | +// |[0..8M)[8..16M)[16..24)| +// +-----------------------+ +// | +// | cudaMemcpyAsync (device-to-device) +// v +// +----------------------------+ +// | SendPool | +// | chunk 3 : bytes [0..8M) | +// | chunk 0 : bytes [8..16M) | +// | chunk 5 : bytes [16..24M) | +// +----------------------------+ +// +// Step 3 -- Rank 0 RDMA-writes SendPool -> Rank 1's RecvPool +// ------------------------------------------------------------------------ +// Rank 0's SendPool RDMA Write Rank 1's RecvPool +// +---------------------+ +---------------------+ +// | chunk 3 : [0..8M) | ------------------------> | chunk 7 : [0..8M) | +// | chunk 0 : [8..16M) | -------------------- ---> | chunk 2 : [8..16M) | +// | chunk 5 : [16..24M) | ------------------------> | chunk 1 : [16..24M) | +// +---------------------+ +---------------------+ +// +// Step 4 -- Rank 0 acknowledges with AckSlots +// ------------------------------------------------------------------------ +// Rank 0 (Sender) RDMA Write Rank 1 (Receiver) +// +-----------------------+ +---------------------------------+ +// | AckLane[0] | | Rank 1's AckLane[0] | +// | (local cache only) | ----------> | slot 0 : {seq=0,len=8M,epoch=G} | +// +-----------------------+ | slot 1 : {seq=1,len=8M,epoch=G} | +// | slot 2 : {seq=2,len=8M,epoch=G} | +// +---------------------------------+ +// +// Step 5 -- Rank 1 copies out and recycles chunks +// ------------------------------------------------------------------------ +// Rank 1 (Receiver) +// +-----------------------+ +// | RecvPool | +// | chunk 7 : [0..8M) | +// | chunk 2 : [8..16M) | +// | chunk 1 : [16..24M) | +// +-----------------------+ +// | +// | cudaMemcpyAsync (RecvPool -> user tensor) +// v +// +-----------------------+ +// | User tensor (ready) | +// |[0..8M)[8..16M)[16..24)| +// +-----------------------+ +// | +// | Free chunks back to pool +// v +// +-----------------------+ +// | RecvPool | +// | chunk 7 : free | +// | chunk 2 : free | +// | chunk 1 : free | +// +-----------------------+ +// +// --------------------------------------------------------------------------- +inline constexpr uint32_t kP2PControlRingSize = 64; +// Pool size and chunk size can be overridden via environment variables: +// MOONCAKE_P2P_POOL_SIZE -- total bytes per direction (default 128 MiB) +// MOONCAKE_P2P_CHUNK_SIZE -- granularity of each chunk (default 16 MiB) +// The chunk count is pool_size / chunk_size (default 8). +inline constexpr uint64_t kDefaultPoolSize = 128u * 1024 * 1024; // 128 MiB +inline constexpr uint64_t kDefaultChunkSize = 16u * 1024 * 1024; // 16 MiB -struct alignas(64) AtomicHeadTail { - uint32_t load( - std::memory_order order = std::memory_order_seq_cst) const noexcept { - return std::atomic_ref(value).load(order); - } +// Single-word atomic publication token. +// Combines epoch and sequence to avoid torn reads. +// kInvalidControlToken (all 1s) means "slot empty / not yet published". +using ControlToken = uint64_t; +inline constexpr ControlToken kInvalidControlToken = + std::numeric_limits::max(); - void store(uint32_t new_value, - std::memory_order order = std::memory_order_seq_cst) noexcept { - std::atomic_ref(value).store(new_value, order); - } +inline ControlToken makeControlToken(uint32_t epoch, uint32_t sequence) { + return (static_cast(epoch) << 32) | + static_cast(sequence & 0xFFFFFFFFu); +} - mutable uint32_t value{0}; +// CreditSlot +// +// Header-Footer double-token guard: +// We store the same token at both ends of the 64-byte block. +// - Producer: write payload -> write footer -> write header (release). +// - Consumer: read header -> read payload -> read footer -> accept only +// when header == footer. +// If the NIC has only partially written the slot, header and footer will +// mismatch (or one of them will still be kInvalidControlToken) and the +// consumer safely retries on the next poll iteration. +// +// Layout: +// [0..7] header_token (epoch << 32 | sequence) +// [8..15] recv_addr (payload) +// [16..19] chunk_len (payload) +// [20..55] padding (kept zero, reserved for future use) +// [56..63] footer_token (identical to header_token) +struct alignas(64) CreditSlot { + private: + uint64_t header_token = kInvalidControlToken; + uint64_t recv_addr = 0; + uint32_t chunk_len = 0; + uint32_t _reserved = 0; + uint8_t _padding[32]{}; + uint64_t footer_token = kInvalidControlToken; + + public: + // Consumer-side reliable read. Returns true when a consistent slot is + // observed (header == footer != kInvalidControlToken). + bool tryLoad(uint64_t& out_recv_addr, uint32_t& out_chunk_len, + uint32_t& out_epoch, uint32_t& out_sequence) const; + + // Producer-side reliable publish. Writes payload and footer first, + // then releases header so consumers see a consistent slot. + void publish(uint32_t epoch, uint32_t seq, uint64_t addr, uint32_t len); + + // Reset both tokens to kInvalidControlToken. + void reset(); }; -// Ring-control metadata for one peer lane. -// -// slot index: 0 -> 1 -> 2 -> ... -> N-1 -> 0 (N = kP2PNumSlots) -// ^h (producer publish cursor) -// ^t (consumer reclaim cursor) -// -// State by (head, tail), modulo N: -// - empty: head == tail -// - full : (head + 1) % N == tail (one slot reserved to distinguish -// full/empty) -// - ready: head != tail -// -// Protocol: -// - producer writes slot[head], then advances head (publish) -// - consumer reads slot[tail], then advances tail (reclaim) -struct P2PControlSlot { - AtomicHeadTail head; - AtomicHeadTail tail; +// AckSlot Layout: +// [0..7] header_token (epoch << 32 | sequence) +// [8..11] chunk_len (payload) +// [12..55] padding (kept zero, reserved for future use) +// [56..63] footer_token (identical to header_token) +struct alignas(64) AckSlot { + private: + uint64_t header_token = kInvalidControlToken; + uint32_t chunk_len = 0; + uint32_t _reserved = 0; + uint8_t _padding[40]{}; + uint64_t footer_token = kInvalidControlToken; + + public: + // See CreditSlot::tryLoad(). + bool tryLoad(uint32_t& out_chunk_len, uint32_t& out_epoch, + uint32_t& out_sequence) const; + + // See CreditSlot::publish(). + void publish(uint32_t epoch, uint32_t seq, uint32_t len); + + // Reset both tokens to kInvalidControlToken. + void reset(); +}; + +// P2PChunkPool +// No lock needed because it is only accessed by one worker thread. +class P2PChunkPool { + public: + P2PChunkPool() = default; + + // Initialize pool with given base address, chunk size and number of chunks. + void init(void* base_addr, size_t chunk_size, uint32_t num_chunks); + + // Acquire a free chunk. Returns nullptr if the pool is exhausted. + void* acquire(); + + // Release a chunk back to the pool. + void release(void* ptr); + + private: + void* base_addr_ = nullptr; + size_t chunk_size_ = 0; + std::vector free_stack_; }; class P2PDeviceWorker; @@ -63,19 +274,21 @@ class P2PProxy { public: friend class P2PDeviceWorker; + enum class OpStatus : uint8_t { kPending = 0, kSuccess = 1, kFailed = 2 }; + struct Options { bool is_cpu = false; int rank = 0; int size = 0; int cuda_device_index = -1; - std::string location; + std::chrono::milliseconds transfer_timeout_ms{30000}; }; struct SendOp { at::Tensor tensor_; int peer_rank_ = -1; cudaStream_t cuda_stream_ = nullptr; - std::shared_ptr> completed_; + std::shared_ptr> status_; }; struct RecvOp { @@ -83,26 +296,21 @@ class P2PProxy { at::Tensor original_tensor_; int peer_rank_ = -1; cudaStream_t cuda_stream_ = nullptr; - std::shared_ptr> completed_; + std::shared_ptr> status_; }; P2PProxy(TransferEngine* engine, const Options& options); ~P2PProxy(); - void BindMeta(const std::shared_ptr& meta); - void* send_buffer() const { return resources_.send_buffer_; } - void* recv_buffer() const { return resources_.recv_buffer_; } - P2PControlSlot* ctrl_send_region() const { - return resources_.ctrl_send_region_; - } - P2PControlSlot* ctrl_recv_region() const { - return resources_.ctrl_recv_region_; - } + CreditSlot* credit_region() const { return resources_.credit_region_; } + AckSlot* ack_region() const { return resources_.ack_region_; } + void bindMeta(const std::shared_ptr& meta); + void extendGroupSizeTo(int new_size); - void EnqueueSend(SendOp op); - void EnqueueRecv(RecvOp op); + void enqueueSend(SendOp op); + void enqueueRecv(RecvOp op); - void ResetPeerState(int peer_rank); + void resetPeerState(int peer_rank); /** * @brief Waits for all active P2P send and receive tasks to complete. @@ -114,7 +322,7 @@ class P2PProxy { * @return True if all tasks completed within the timeout; false if timed * out. */ - bool DrainTasks() const; + bool drainTasks() const; /** * @brief Abandons resources instead of releasing them properly. @@ -122,128 +330,225 @@ class P2PProxy { * When a hung operation prevents clean shutdown, this method marks * resources as abandoned to prevent crashes during destructor. */ - void AbandonResources(); + void abandonResources(); + + // Epoch for fault recovery. All control slots carry this + // value so that stale messages from before a Reset can be detected. + uint32_t getEpoch(int peer_rank) const { + return peer_epoch_[peer_rank].load(std::memory_order_acquire); + } + void setEpoch(int peer_rank, uint32_t epoch) { + peer_epoch_[peer_rank].store(epoch, std::memory_order_release); + } private: - enum class TransferState { - kDataCopy, - kTransfer, - kDone, + // Sender-side per-chunk state machine. + enum class SendTaskState { + kCopyIn, // Copying tensor slice into local SendPool staging buffer. + // On CPU this is synchronous; on GPU we record a + // cudaEvent on the op's stream and poll it. + kWriteRemote, // Write from SendPool -> remote RecvPool. + kAck, // Writing AckSlot back to the receiver. + kFinished, // AckSlot write done, staging buffer freed. + kFailed, // Transport error or timeout; staging buffer freed. + }; + + // Receiver-side per-chunk state machine. + enum class RecvTaskState { + kIssueCredit, // CreditSlot Write is in flight. + kWaitAck, // Waiting for sender's AckSlot. + kCopyOut, // GPU: cudaMemcpyAsync RecvPool -> tensor in flight. + kFinished, // Copy-out done, chunk returned to pool. + kFailed, // Transport error or timeout; chunk returned to pool. }; struct SendOpContext; struct RecvOpContext; + // One chunk of a SendOp. The sender copies data from the user tensor + // into a SendPool staging buffer and then RDMA-writes it to the remote + // RecvPool offset that the receiver issued in the CreditSlot. struct SendTransferTask { SendTransferTask() = default; - SendTransferTask(uint64_t chunk_offset_in, uint64_t chunk_bytes_in, - void* source_in, uint64_t target_offset_in); + SendTransferTask(uint64_t tensor_offset_in, uint32_t chunk_len_in, + void* staging_addr_in, uint64_t remote_addr_in, + uint32_t sequence_in, uint32_t epoch_in); - TransferState state_ = TransferState::kDataCopy; - uint64_t chunk_offset_ = 0; - uint64_t chunk_bytes_ = 0; - void* source_ = nullptr; - uint64_t target_offset_ = 0; - std::optional transfer_batch_id_; - cudaEvent_t copy_ready_event_ = nullptr; + SendTaskState state_ = SendTaskState::kCopyIn; + uint64_t tensor_offset_ = 0; // Offset inside the user tensor. + uint32_t chunk_len_ = 0; // Bytes in this chunk (<= kP2PChunkSize). + void* staging_addr_ = nullptr; // Address inside SendPool. + uint64_t remote_addr_ = 0; // Address inside REMOTE RecvPool. + uint32_t sequence_ = 0; // Sequence number in the control ring. + uint32_t epoch_ = 0; // Epoch for Reset detection. + std::optional transfer_batch_id_; // RDMA Write batch id. + std::optional ack_batch_id_; // AckSlot write batch id. + cudaEvent_t copy_ready_event_ = nullptr; // Signals Copy-In done (GPU). + std::chrono::steady_clock::time_point last_update_time_; }; + // Active send operation state. Tasks are created in order and advance + // through the sender state machine (kCopyIn -> kWriteRemote -> kAck + // -> kFinished). struct SendOpContext { SendOpContext() = default; SendOpContext(SendOp&& op_in); + std::deque tasks_; + std::shared_ptr> status_; + at::Tensor tensor_; int peer_rank_ = -1; cudaStream_t cuda_stream_ = nullptr; - std::shared_ptr> completed_; uint64_t total_bytes_ = 0; - uint64_t bytes_issued_ = 0; - std::optional head_update_batch_id_; - std::deque tasks_; + // Number of bytes already pulled from the user tensor into SendPool + // staging buffers. When bytes_staged_ == total_bytes_ every chunk + // has at least entered the Copy-In stage. + uint64_t bytes_staged_ = 0; + + std::chrono::steady_clock::time_point last_update_time_; }; + // One chunk of a RecvOp. The receiver allocates a RecvPool chunk, + // issues it to the sender via a CreditSlot, waits for the sender + // to RDMA-write data into it, and finally copies the data into the user + // tensor before returning the chunk to RecvPool. struct RecvTransferTask { RecvTransferTask() = default; - RecvTransferTask(uint64_t chunk_offset_in, uint64_t chunk_bytes_in, - void* source_in, void* target_in); + RecvTransferTask(uint64_t tensor_offset_in, uint32_t chunk_len_in, + void* local_addr_in, uint32_t sequence_in, + uint32_t epoch_in); - TransferState state_ = TransferState::kDataCopy; - uint64_t chunk_offset_ = 0; - uint64_t chunk_bytes_ = 0; - void* source_ = nullptr; - void* target_ = nullptr; - cudaEvent_t copy_ready_event_ = nullptr; + RecvTaskState state_ = RecvTaskState::kIssueCredit; + uint64_t tensor_offset_ = 0; // Offset inside the user tensor. + uint32_t chunk_len_ = 0; // Bytes in this chunk. + void* local_addr_ = nullptr; // Address inside local RecvPool. + uint32_t sequence_ = 0; // Sequence number in the control ring. + uint32_t epoch_ = 0; // Epoch for Reset detection. + std::optional + credit_batch_id_; // CreditSlot RDMA Write batch id. + cudaEvent_t copy_ready_event_ = + nullptr; // Signals Copy-Out done (GPU). + std::chrono::steady_clock::time_point last_update_time_; }; + // Active receive operation state. Tasks are created in order and + // advance through the receiver state machine (kIssueCredit -> + // kWaitAck -> kCopyOut -> kFinished). struct RecvOpContext { RecvOpContext() = default; RecvOpContext(RecvOp&& op_in); + std::deque tasks_; + std::shared_ptr> status_; + at::Tensor tensor_; at::Tensor original_tensor_; int peer_rank_ = -1; cudaStream_t cuda_stream_ = nullptr; - std::shared_ptr> completed_; uint64_t total_bytes_ = 0; - uint64_t bytes_issued_ = 0; - std::optional tail_update_batch_id_; - std::deque tasks_; + // Number of bytes for which a RecvPool chunk has been reserved and a + // CreditSlot has been sent to the peer. When bytes_credited_ == + // total_bytes_ the entire tensor has been offered to the sender. + uint64_t bytes_credited_ = 0; }; + // Per-peer sender state. The sender consumes CreditSlots that the + // receiver writes into our local CreditLane for this peer. struct SendPeerLane { std::deque pending_send_ops_; std::optional active_send_op_; - uint32_t local_head_ = 0; - std::array copy_ready_events_; + // Sequence number of the next CreditSlot to consume from this peer. + // Monotonically increases; wraps around the ring via modulo. + uint64_t credit_consume_seq_ = 0; + std::array copy_ready_events_; }; + // Per-peer receiver state. The receiver issues CreditSlots to grant credit + // to the peer to write data, and consumes AckSlots that the peer writes + // back to acknowledge finished transfers. struct RecvPeerLane { std::deque pending_recv_ops_; std::optional active_recv_op_; - uint32_t local_tail_ = 0; - std::array copy_ready_events_; + // Sequence number of the next CreditSlot to issue to this peer. + uint64_t credit_issue_seq_ = 0; + // Sequence number of the next AckSlot to consume from this peer. + uint64_t ack_consume_seq_ = 0; + std::array copy_ready_events_; }; // Resources are allocated and released by constructor/destructor - void AllocateResources(); - void ReleaseResources(); + void allocateResources(); + void releaseResources(); // For P2PDeviceWorker - bool StepSend(); - bool StepRecv(); - void SetDeviceWorker(P2PDeviceWorker*); - bool HasActiveSendWork() const; - bool HasActiveRecvWork() const; + bool stepSend(); + bool stepRecv(); + void attachToWorker(P2PDeviceWorker* worker, P2PChunkPool* send, + P2PChunkPool* recv, size_t chunk_size); + bool hasActiveSendWork() const; + bool hasActiveRecvWork() const; - // Internal Steps - bool TryIssueSendTask(SendOpContext& op_ctx, uint32_t capacity); - bool StepSendTransferTask(SendOpContext& op_ctx, SendTransferTask& task); - bool StepSendDataCopy(SendTransferTask& task); - bool StepSendTransfer(SendOpContext& op_ctx, SendTransferTask& task); - bool StepSendHeadCommit(SendOpContext& op_ctx, uint32_t capacity); - bool IsSendDataPathCompleted(const SendOpContext& op_ctx) const; - bool IsSendOpCompleted(const SendOpContext& op_ctx) const; - void PerformSendReset(int peer_rank); + bool tryIssueRecvTask(RecvOpContext& op_ctx, RecvPeerLane& lane); + bool stepRecvTask(RecvTransferTask& task); + bool stepRecvCopyOut(RecvTransferTask& task); + bool stepRecvIssueCredit(RecvTransferTask& task); + bool pollRecvAckSlot(RecvOpContext& op_ctx, RecvPeerLane& lane, + RecvTransferTask& head_task); + bool isRecvOpCompleted(const RecvOpContext& op_ctx) const; + void performRecvReset(int peer_rank); - bool TryIssueRecvTask(RecvOpContext& op_ctx, uint32_t capacity); - bool StepRecvTransferTask(RecvTransferTask& task); - bool StepRecvDataCopy(RecvTransferTask& task); - bool StepRecvTailCommit(RecvOpContext& op_ctx, uint32_t capacity); - bool IsRecvDataPathCompleted(const RecvOpContext& op_ctx) const; - void PerformRecvReset(int peer_rank); + bool tryIssueSendTask(SendOpContext& op_ctx, SendPeerLane& lane); + bool stepSendTask(SendOpContext& op_ctx, SendTransferTask& task); + bool stepSendCopyIn(SendTransferTask& task); + bool stepSendWriteRemote(SendOpContext& op_ctx, SendTransferTask& task); + bool stepSendAck(SendOpContext& op_ctx, SendTransferTask& task); + bool isSendOpCompleted(const SendOpContext& op_ctx) const; + void performSendReset(int peer_rank); - uint64_t GetLocalSendSlotAddress(int peer_rank, uint32_t slot_index) const; - uint64_t GetLocalRecvSlotAddress(int peer_rank, uint32_t slot_index) const; - uint64_t GetRemoteRecvSlotAddress(int peer_rank, uint32_t slot_index) const; - uint64_t GetRemoteCtrlRecvHeadOffset(int peer_rank) const; - uint64_t GetRemoteCtrlSendTailOffset(int peer_rank) const; + void reportBrokenPeer(int peer_rank); + + // These helpers are used only during reset or shutdown. + // In the normal execution path, task and lane resources are released + // incrementally as work progresses. + void releaseSendTaskResources(SendTransferTask& task) const; + void releaseRecvTaskResources(RecvTransferTask& task) const; + void resetSendLane(SendPeerLane& lane); + void resetRecvLane(RecvPeerLane& lane); + void resetPeerControlLanes(int peer_rank); + + // Control lane addressing. + // + // Each rank owns one contiguous Credit region and one Ack region. + // Within each region there are kMaxNumRanks lanes, one per peer. + // Lane index == peer rank. Each lane has kP2PControlRingSize slots. + // + // Example: rank R wants to write a CreditSlot to peer P. + // target = P's credit base + (R * kRingSize + seq % kRingSize) * sizeof + // + // This isolates control traffic per (sender, receiver) pair. + CreditSlot* getLocalCreditLane(int peer_rank) const; + AckSlot* getLocalAckLane(int peer_rank) const; + uint64_t getRemoteCreditSlot(int peer_rank, uint32_t sequence) const; + uint64_t getRemoteAckSlot(int peer_rank, uint32_t sequence) const; + + CreditSlot* getLocalCreditStagingBuf(int peer_rank, + uint32_t sequence) const; + AckSlot* getLocalAckStagingBuf(int peer_rank, uint32_t sequence) const; + + template + bool isTimeout(const T& obj) const { + return std::chrono::steady_clock::now() - obj.last_update_time_ > + transfer_timeout_ms_; + } private: struct P2PResources { - void* send_buffer_ = nullptr; - void* recv_buffer_ = nullptr; - P2PControlSlot* ctrl_send_region_ = nullptr; - P2PControlSlot* ctrl_recv_region_ = nullptr; + CreditSlot* credit_region_ = nullptr; + AckSlot* ack_region_ = nullptr; + // Serve as RDMA write source address + CreditSlot* credit_staging_buf_ = nullptr; + AckSlot* ack_staging_buf_ = nullptr; }; P2PDeviceWorker* device_worker_ = nullptr; @@ -254,10 +559,15 @@ class P2PProxy { int rank_ = 0; int size_ = 0; int cuda_device_index_ = -1; - std::string location_; + std::chrono::milliseconds transfer_timeout_ms_{5000}; // 5s P2PResources resources_; bool resource_abandoned_{false}; + size_t chunk_size_ = 0; + + P2PChunkPool* send_pool_ = nullptr; + P2PChunkPool* recv_pool_ = nullptr; + std::queue send_queue_; std::mutex send_queue_mutex_; @@ -270,6 +580,11 @@ class P2PProxy { std::atomic active_send_tasks_{0}; std::atomic active_recv_tasks_{0}; + // Per-peer epoch for fault recovery. Incremented in resetPeerState + // and performSend/RecvReset so that stale messages from a previous epoch + // can be detected on a per-peer basis. + std::array, kMaxNumRanks> peer_epoch_; + std::array send_peer_lanes_; std::array recv_peer_lanes_; }; @@ -283,22 +598,19 @@ class P2PDeviceWorker { void registerProxy(const std::shared_ptr&); void removeProxy(const std::shared_ptr&); - P2PDeviceWorker(bool is_cpu, int cuda_device_index) - : is_cpu_(is_cpu), cuda_device_index_(cuda_device_index) { - Start(); - } + P2PDeviceWorker(TransferEngine* engine, const std::string& location, + bool is_cpu, int cuda_device_index); - ~P2PDeviceWorker() { Stop(); } - - void WakeUpSend(); - void WakeUpRecv(); + ~P2PDeviceWorker(); + void wakeUpSend(); + void wakeUpRecv(); private: - void Start(); - void Stop(); + void start(); + void stop(); - void SendWorkerThread(); - void RecvWorkerThread(); + void sendWorkerMainloop(); + void recvWorkerMainloop(); std::mutex send_wakeup_mutex_; std::condition_variable send_wakeup_cv_; @@ -318,18 +630,36 @@ class P2PDeviceWorker { bool is_cpu_; int cuda_device_index_; + + // Per-device shared chunk pools + // + // Memory footprint (with default env values): + // - One direction (send or recv) : 128 MiB + // - Total per device (send + recv) : 256 MiB + // This stays constant no matter how many backends or peer ranks are active. + TransferEngine* engine_ = nullptr; + P2PChunkPool send_pool_; + P2PChunkPool recv_pool_; + void* send_pool_base_ = nullptr; + void* recv_pool_base_ = nullptr; + size_t pool_bytes_ = 0; + size_t chunk_size_ = 0; + uint32_t num_chunks_ = 0; + void initPools(TransferEngine* engine, const std::string& location); + void releasePools(); }; class P2PDeviceWorkerManager { public: - static P2PDeviceWorkerManager& GetInstance() { + static P2PDeviceWorkerManager& getInstance() { // leaky singleton to avoid destructor fiasco problem static P2PDeviceWorkerManager* manager = new P2PDeviceWorkerManager; return *manager; } - std::shared_ptr GetCPUWorker(); - std::shared_ptr GetCUDAWorker(int cuda_device_index); + std::shared_ptr getCPUWorker(TransferEngine* engine); + std::shared_ptr getCUDAWorker(int cuda_device_index, + TransferEngine* engine); private: static constexpr int CPUWorkerID = -1; diff --git a/mooncake-pg/src/connection_poller.cpp b/mooncake-pg/src/connection_poller.cpp index 9199806f..817ae54e 100644 --- a/mooncake-pg/src/connection_poller.cpp +++ b/mooncake-pg/src/connection_poller.cpp @@ -400,9 +400,7 @@ bool ConnectionContext::pollPeer(int pollingRank) { store_->deleteKey( getBufferStoreKey(backendIndex_, pollingRank)); store_->deleteKey( - getExtensionTaskCountStoreKey(backendIndex_, pollingRank)); - store_->deleteKey(getExtensionActiveRanksStoreKey(backendIndex_, - pollingRank)); + getExtensionStateStoreKey(backendIndex_, pollingRank)); } catch (const std::exception& e) { LOG(WARNING) << "Rank " << rank_ << " got an exception when deleteKey for peer " @@ -414,7 +412,7 @@ bool ConnectionContext::pollPeer(int pollingRank) { &warmup_recv_region_[pollingRank]) = 0; // Reset P2PProxy states - p2p_proxy_->ResetPeerState(pollingRank); + p2p_proxy_->resetPeerState(pollingRank); // Back to WAITING_STORE to reconnect it. peerState.state = PeerConnectionState::WAITING_STORE; diff --git a/mooncake-pg/src/mooncake_backend.cpp b/mooncake-pg/src/mooncake_backend.cpp index a6ad0530..49f0e16e 100644 --- a/mooncake-pg/src/mooncake_backend.cpp +++ b/mooncake-pg/src/mooncake_backend.cpp @@ -33,59 +33,138 @@ TransferEngine* MooncakeBackend::engine_ = new TransferEngine(true); bool MooncakeBackend::engineInitialized_ = false; int MooncakeBackend::backendIndex_ = 0; -namespace { +std::vector serialize(const ExtensionState& state) { + uint32_t rankCount = static_cast(state.activeRanks.size()); -std::vector serializeActiveRanks(const bool* activeRanks, int size) { - std::vector bytes(size); - for (int i = 0; i < size; ++i) { - bytes[i] = activeRanks[i] ? 1 : 0; + // Calculate bytes needed for the bitmap: 1 bit per rank, rounded up to + // nearest byte + size_t bitmapSize = (rankCount + 7) / 8; + + // Total size = count field + bitmap + p2pEpochs[] + taskCount + size_t totalSize = sizeof(uint32_t) + bitmapSize + + sizeof(uint32_t) * rankCount + sizeof(int32_t); + + std::vector buffer(totalSize, 0); + uint8_t* ptr = buffer.data(); + + // 1. Store the number of ranks + std::memcpy(ptr, &rankCount, sizeof(uint32_t)); + ptr += sizeof(uint32_t); + + // 2. Store activeRanks as a bitset + for (size_t i = 0; i < rankCount; ++i) { + if (state.activeRanks[i]) { + // Set the i-th bit to 1 if the rank is active + ptr[i / 8] |= (1 << (i % 8)); + } } - return bytes; + ptr += bitmapSize; + + // 3. Store per-peer p2pEpochs + for (size_t i = 0; i < rankCount; ++i) { + std::memcpy(ptr + i * sizeof(uint32_t), &state.p2pEpochs[i], + sizeof(uint32_t)); + } + ptr += sizeof(uint32_t) * rankCount; + + // 4. Store taskCount + int32_t taskCount = static_cast(state.taskCount); + std::memcpy(ptr, &taskCount, sizeof(int32_t)); + + return buffer; } -void deserializeActiveRanks(const std::vector& bytes, - bool* activeRanks, int size) { - TORCH_CHECK(static_cast(bytes.size()) == size, - "Unexpected active-ranks snapshot size."); - for (int i = 0; i < size; ++i) { - activeRanks[i] = (bytes[i] != 0); - } -} +ExtensionState deserialize(const std::vector& buffer) { + ExtensionState state; + if (buffer.size() < sizeof(uint32_t)) return state; -} // namespace + const uint8_t* ptr = buffer.data(); + + // 1. Read the number of ranks + uint32_t rankCount = 0; + std::memcpy(&rankCount, ptr, sizeof(uint32_t)); + ptr += sizeof(uint32_t); + + // Calculate expected total size and verify buffer is sufficient before + // proceeding with further reads. + size_t bitmapSize = (rankCount + 7) / 8; + size_t expectedSize = sizeof(uint32_t) + bitmapSize + + sizeof(uint32_t) * rankCount + sizeof(int32_t); + if (buffer.size() < expectedSize) return state; + + // 2. Read the bitmap and reconstruct the activeRanks vector + state.activeRanks.resize(rankCount); + for (size_t i = 0; i < rankCount; ++i) { + // Check if the i-th bit is set + bool isActive = ptr[i / 8] & (1 << (i % 8)); + state.activeRanks[i] = isActive; + } + ptr += bitmapSize; + + // 3. Read per-peer p2pEpochs + state.p2pEpochs.resize(rankCount); + for (size_t i = 0; i < rankCount; ++i) { + std::memcpy(&state.p2pEpochs[i], ptr, sizeof(uint32_t)); + ptr += sizeof(uint32_t); + } + + // 4. Read taskCount + int32_t taskCount = 0; + std::memcpy(&taskCount, ptr, sizeof(int32_t)); + state.taskCount = static_cast(taskCount); + + return state; +} // Async Work implementation for P2P operations processed by worker threads. class MooncakeP2PWork : public ::c10d::Work { public: - explicit MooncakeP2PWork(std::shared_ptr> completed) - : Work(-1, c10d::OpType::UNKNOWN), completed_(completed) {} + explicit MooncakeP2PWork( + std::shared_ptr> status) + : Work(-1, c10d::OpType::UNKNOWN), status_(status) {} bool isCompleted() override { - return completed_->load(std::memory_order_acquire); + return status_->load(std::memory_order_acquire) != + P2PProxy::OpStatus::kPending; + } + + bool isSuccess() const override { + return status_->load(std::memory_order_acquire) == + P2PProxy::OpStatus::kSuccess; } bool wait(std::chrono::milliseconds timeout) override { - if (completed_->load(std::memory_order_acquire)) { - return true; - } - BackoffWaiterConfig cfg{}; cfg.max_sleep = std::chrono::microseconds(10); BackoffWaiter waiter(cfg); + bool done = false; if (timeout.count() > 0) { - return waiter.wait_for(timeout, [this] { - return completed_->load(std::memory_order_acquire); + done = waiter.wait_for(timeout, [this] { + return status_->load(std::memory_order_acquire) != + P2PProxy::OpStatus::kPending; }); + } else { + waiter.wait([this] { + return status_->load(std::memory_order_acquire) != + P2PProxy::OpStatus::kPending; + }); + done = true; } - waiter.wait( - [this] { return completed_->load(std::memory_order_acquire); }); + if (!done) { + return false; + } + + if (status_->load(std::memory_order_acquire) == + P2PProxy::OpStatus::kFailed) { + TORCH_CHECK(false, "Mooncake P2P operation failed."); + } return true; } private: - std::shared_ptr> completed_; + std::shared_ptr> status_; }; /** @@ -196,13 +275,14 @@ MooncakeBackend::MooncakeBackend( TORCH_CHECK(!rc, REGISTER_BUFFER_ERROR_MSG); } - auto& dev_worker_mgr = P2PDeviceWorkerManager::GetInstance(); + auto& dev_worker_mgr = P2PDeviceWorkerManager::getInstance(); int cuda_device_index = isCpu_ ? -1 : at::cuda::current_device(); if (isCpu_) - p2p_device_worker_ = dev_worker_mgr.GetCPUWorker(); + p2p_device_worker_ = dev_worker_mgr.getCPUWorker(engine_); else - p2p_device_worker_ = dev_worker_mgr.GetCUDAWorker(cuda_device_index); + p2p_device_worker_ = + dev_worker_mgr.getCUDAWorker(cuda_device_index, engine_); auto& worker_mgr = MooncakeWorkerManager::GetInstance(); if (isCpu_) @@ -220,7 +300,6 @@ MooncakeBackend::MooncakeBackend( .rank = rank_, .size = size_, .cuda_device_index = cuda_device_index, - .location = location, }); p2p_device_worker_->registerProxy(p2p_proxy_); @@ -241,10 +320,8 @@ MooncakeBackend::MooncakeBackend( (uint64_t)connection_ctx_->warmup_send_region(); rank_info.warmup_buffer[1] = (uint64_t)connection_ctx_->warmup_recv_region(); - rank_info.p2p_send_buffer = (uint64_t)p2p_proxy_->send_buffer(); - rank_info.p2p_recv_buffer = (uint64_t)p2p_proxy_->recv_buffer(); - rank_info.p2p_ctrl_send = (uint64_t)p2p_proxy_->ctrl_send_region(); - rank_info.p2p_ctrl_recv = (uint64_t)p2p_proxy_->ctrl_recv_region(); + rank_info.p2p_credit_region = (uint64_t)p2p_proxy_->credit_region(); + rank_info.p2p_ack_region = (uint64_t)p2p_proxy_->ack_region(); // Sync metadata std::vector rank_info_bytes(sizeof(SegmentInfo)); @@ -283,7 +360,7 @@ MooncakeBackend::MooncakeBackend( meta_->store = store; meta_->backendIndex = backendIndex_; meta_->bufferBaseIndex = backendIndex_ * 10; - p2p_proxy_->BindMeta(meta_); + p2p_proxy_->bindMeta(meta_); connection_ctx_->bootstrapLocalPeer(localServerName_, rank_info); if (options_ && options_->isExtension_) { @@ -353,7 +430,8 @@ c10::intrusive_ptr MooncakeBackend::send( "P2P send: dstRank out of range."); auto contiguous = tensor.contiguous(); - auto completed = std::make_shared>(false); + auto status = std::make_shared>( + P2PProxy::OpStatus::kPending); cudaStream_t stream = nullptr; if (!isCpu_) { auto current_stream = @@ -362,14 +440,14 @@ c10::intrusive_ptr MooncakeBackend::send( } TORCH_CHECK(p2p_proxy_, "P2P send proxy is not initialized."); - p2p_proxy_->EnqueueSend(P2PProxy::SendOp{ + p2p_proxy_->enqueueSend(P2PProxy::SendOp{ .tensor_ = std::move(contiguous), .peer_rank_ = dstRank, .cuda_stream_ = stream, - .completed_ = completed, + .status_ = status, }); - return c10::make_intrusive(completed); + return c10::make_intrusive(status); } c10::intrusive_ptr MooncakeBackend::recv( @@ -385,7 +463,8 @@ c10::intrusive_ptr MooncakeBackend::recv( "P2P recv: srcRank out of range."); auto target = tensor.is_contiguous() ? tensor : tensor.contiguous(); - auto completed = std::make_shared>(false); + auto status = std::make_shared>( + P2PProxy::OpStatus::kPending); cudaStream_t stream = nullptr; if (!isCpu_) { auto current_stream = @@ -394,15 +473,15 @@ c10::intrusive_ptr MooncakeBackend::recv( } TORCH_CHECK(p2p_proxy_, "P2P recv proxy is not initialized."); - p2p_proxy_->EnqueueRecv(P2PProxy::RecvOp{ + p2p_proxy_->enqueueRecv(P2PProxy::RecvOp{ .tensor_ = target, .original_tensor_ = tensor, .peer_rank_ = srcRank, .cuda_stream_ = stream, - .completed_ = completed, + .status_ = status, }); - return c10::make_intrusive(completed); + return c10::make_intrusive(status); } c10::intrusive_ptr MooncakeBackend::broadcast( @@ -843,7 +922,7 @@ void MooncakeBackend::shutdown() { // Phase 1: Drain P2P tasks p2p_device_worker_->removeProxy(p2p_proxy_); - has_hung_operation |= !p2p_proxy_->DrainTasks(); + has_hung_operation |= !p2p_proxy_->drainTasks(); // Phase 2: Drain collective tasks for this backend has_hung_operation |= !worker_->drainTasks(meta_.get()); @@ -863,7 +942,7 @@ void MooncakeBackend::shutdown() { // Phase 5: Release resources if no hung operations if (has_hung_operation) { - p2p_proxy_->AbandonResources(); + p2p_proxy_->abandonResources(); connection_ctx_->abandonResources(); } @@ -934,24 +1013,33 @@ void MooncakeBackend::setLocalOnlyActiveRanks() { void MooncakeBackend::waitForExtensionState() { TORCH_CHECK(meta_->store, "Recovery join requires a valid Store."); - auto task_count_key = ConnectionContext::getExtensionTaskCountStoreKey( - meta_->backendIndex, rank_); - auto active_ranks_key = ConnectionContext::getExtensionActiveRanksStoreKey( + auto state_key = ConnectionContext::getExtensionStateStoreKey( meta_->backendIndex, rank_); BackoffWaiter waiter( BackoffWaiterConfig::constantSleep(std::chrono::milliseconds(50))); - waiter.wait([&] { - return meta_->store->check({task_count_key, active_ranks_key}); - }); + waiter.wait([&] { return meta_->store->check({state_key}); }); - auto task_count_data = meta_->store->get(task_count_key); - std::string task_count(task_count_data.begin(), task_count_data.end()); - meta_->taskCount = std::stoi(task_count); + auto state_data = meta_->store->get(state_key); + auto state = deserialize(state_data); - auto active_ranks = meta_->store->get(active_ranks_key); - deserializeActiveRanks(active_ranks, meta_->activeRanks, meta_->size); + // taskCount + meta_->taskCount = state.taskCount; + + // p2pEpochs + TORCH_CHECK(static_cast(meta_->size) == state.p2pEpochs.size(), + "Invalid p2pEpochs size"); + for (int i = 0; i < meta_->size; ++i) { + p2p_proxy_->setEpoch(i, state.p2pEpochs[i]); + } + + // activeRanks + TORCH_CHECK(static_cast(meta_->size) == state.activeRanks.size(), + "Invalid activeRanks"); + for (int i = 0; i < meta_->size; ++i) { + meta_->activeRanks[i] = state.activeRanks[i]; + } syncActiveRanksTensor(); } @@ -998,6 +1086,7 @@ void MooncakeBackend::extendGroupSizeTo(int newSize) { tensor.slice(0, oldSize, newSize).fill_(1); connection_ctx_->extendGroupSizeTo(newSize); + p2p_proxy_->extendGroupSizeTo(newSize); // After extendGroupSizeTo, we don't `waitUntilNewRanksConnected` here // but do it in the first task. This enables client code to overlap // execution between `extendGroupSizeTo` and the first communication call. @@ -1059,15 +1148,20 @@ void MooncakeBackend::recoverRanks(const std::vector& ranks) { } syncActiveRanksTensor(); - auto active_ranks_snapshot = - serializeActiveRanks(meta_->activeRanks, meta_->size); + std::vector epochs(meta_->size); + for (int i = 0; i < meta_->size; ++i) { + epochs[i] = p2p_proxy_->getEpoch(i); + } + ExtensionState state{ + .activeRanks = + std::vector(meta_->activeRanks, meta_->activeRanks + meta_->size), + .p2pEpochs = std::move(epochs), + .taskCount = meta_->taskCount}; + auto state_data = serialize(state); for (const int rank : ranks) { - meta_->store->set(ConnectionContext::getExtensionTaskCountStoreKey( - meta_->backendIndex, rank), - std::to_string(meta_->taskCount)); - meta_->store->set(ConnectionContext::getExtensionActiveRanksStoreKey( - meta_->backendIndex, rank), - active_ranks_snapshot); + auto key = ConnectionContext::getExtensionStateStoreKey( + meta_->backendIndex, rank); + meta_->store->set(key, state_data); } } diff --git a/mooncake-pg/src/p2p_proxy.cpp b/mooncake-pg/src/p2p_proxy.cpp index 7df056c7..26e32af7 100644 --- a/mooncake-pg/src/p2p_proxy.cpp +++ b/mooncake-pg/src/p2p_proxy.cpp @@ -9,6 +9,7 @@ #include #include #include +#include "cuda_alike.h" #include "memory_location.h" #include "pg_utils.h" @@ -16,14 +17,12 @@ namespace mooncake { namespace { -constexpr size_t kP2PBytesPerRank = kP2PBufferSize; -constexpr size_t kP2PNumSlotsPerRank = kP2PNumSlots; +size_t getEnv_size_t(const char* name, size_t default_val) { + const char* val = std::getenv(name); + return val ? std::strtoull(val, nullptr, 10) : default_val; +} -static_assert(kP2PBufferSize % kP2PNumSlots == 0, - "kP2PBufferSize must be divisible by kP2PNumSlots"); -static_assert(kP2PNumSlots > 1, "P2P ring requires at least 2 slots per rank"); - -void SetCudaDeviceIfNeeded(bool is_cpu, int cuda_device_index, +void setCudaDeviceIfNeeded(bool is_cpu, int cuda_device_index, const char* context) { if (is_cpu) { return; @@ -37,13 +36,135 @@ void SetCudaDeviceIfNeeded(bool is_cpu, int cuda_device_index, } // namespace +void P2PChunkPool::init(void* base_addr, size_t chunk_size, + uint32_t num_chunks) { + base_addr_ = base_addr; + chunk_size_ = chunk_size; + free_stack_.clear(); + free_stack_.reserve(num_chunks); + for (uint32_t idx = 0; idx < num_chunks; ++idx) { + free_stack_.push_back(static_cast(base_addr_) + + (num_chunks - 1 - idx) * chunk_size_); + } +} + +void* P2PChunkPool::acquire() { + if (free_stack_.empty()) { + return nullptr; + } + void* ptr = free_stack_.back(); + free_stack_.pop_back(); + return ptr; +} + +void P2PChunkPool::release(void* ptr) { free_stack_.push_back(ptr); } + +// Consumer-side reliable read. +// +// 1. Load header_token with acquire semantics. +// 2. If it is kInvalidControlToken the slot is empty -> fail. +// 3. Optimistically copy the payload. +// 4. Issue an acquire fence so the payload reads cannot be reordered past +// the footer load that follows. +// 5. Load footer_token. +// 6. Accept the payload only when header == footer. +bool CreditSlot::tryLoad(uint64_t& out_recv_addr, uint32_t& out_chunk_len, + uint32_t& out_epoch, uint32_t& out_sequence) const { + uint64_t h = std::atomic_ref(header_token).load(std::memory_order_acquire); + if (h == kInvalidControlToken) return false; + + // Optimistic payload copy – may be torn if the NIC is still writing. + uint64_t addr = std::atomic_ref(recv_addr).load(std::memory_order_relaxed); + uint32_t len = std::atomic_ref(chunk_len).load(std::memory_order_relaxed); + + // Avoid moving payload reads below the footer check. + std::atomic_thread_fence(std::memory_order_acquire); + + uint64_t f = std::atomic_ref(footer_token).load(std::memory_order_relaxed); + + if (h == f) { + out_recv_addr = addr; + out_chunk_len = len; + out_epoch = static_cast(h >> 32); + out_sequence = static_cast(h); + return true; + } + // Torn write – caller retries next poll. + return false; +} + +// Producer-side reliable publish. +// +// On the local CPU the store order matters: +// payload -> footer (relaxed) -> header (release). +// The release store to header_token guarantees that every prior store is +// visible to any consumer that sees the new header value. +void CreditSlot::publish(uint32_t epoch, uint32_t seq, uint64_t addr, + uint32_t len) { + uint64_t new_token = makeControlToken(epoch, seq); + + recv_addr = addr; + chunk_len = len; + + // Write footer first so it is never ahead of the header. + std::atomic_ref(footer_token).store(new_token, std::memory_order_relaxed); + // Release the header last + std::atomic_ref(header_token).store(new_token, std::memory_order_release); +} + +void CreditSlot::reset() { + recv_addr = 0; + chunk_len = 0; + uint64_t inv = kInvalidControlToken; + std::atomic_ref(footer_token).store(inv, std::memory_order_relaxed); + std::atomic_ref(header_token).store(inv, std::memory_order_release); +} + +// See CreditSlot::tryLoad() for the detailed description. +bool AckSlot::tryLoad(uint32_t& out_chunk_len, uint32_t& out_epoch, + uint32_t& out_sequence) const { + uint64_t h = std::atomic_ref(header_token).load(std::memory_order_acquire); + if (h == kInvalidControlToken) return false; + + uint32_t len = std::atomic_ref(chunk_len).load(std::memory_order_relaxed); + + std::atomic_thread_fence(std::memory_order_acquire); + + uint64_t f = std::atomic_ref(footer_token).load(std::memory_order_relaxed); + + if (h == f) { + out_chunk_len = len; + out_epoch = static_cast(h >> 32); + out_sequence = static_cast(h); + return true; + } + return false; +} + +// See CreditSlot::publish() for the detailed description. +void AckSlot::publish(uint32_t epoch, uint32_t seq, uint32_t len) { + uint64_t new_token = makeControlToken(epoch, seq); + + chunk_len = len; + + std::atomic_ref(footer_token).store(new_token, std::memory_order_relaxed); + std::atomic_ref(header_token).store(new_token, std::memory_order_release); +} + +void AckSlot::reset() { + chunk_len = 0; + uint64_t inv = kInvalidControlToken; + std::atomic_ref(footer_token).store(inv, std::memory_order_relaxed); + std::atomic_ref(header_token).store(inv, std::memory_order_release); +} + P2PProxy::P2PProxy(TransferEngine* engine, const Options& options) : engine_(engine), is_cpu_(options.is_cpu), rank_(options.rank), size_(options.size), cuda_device_index_(options.cuda_device_index), - location_(options.location) { + transfer_timeout_ms_(options.transfer_timeout_ms) { if (!is_cpu_ && cuda_device_index_ < 0) { int current_device = -1; const cudaError_t get_device_error = cudaGetDevice(¤t_device); @@ -52,7 +173,7 @@ P2PProxy::P2PProxy(TransferEngine* engine, const Options& options) cudaGetErrorString(get_device_error)); cuda_device_index_ = current_device; } - AllocateResources(); + allocateResources(); } P2PProxy::~P2PProxy() { @@ -62,85 +183,56 @@ P2PProxy::~P2PProxy() { return; } - ReleaseResources(); + releaseResources(); } -void P2PProxy::BindMeta(const std::shared_ptr& meta) { +void P2PProxy::bindMeta(const std::shared_ptr& meta) { meta_ = meta; } -void P2PProxy::AllocateResources() { +void P2PProxy::extendGroupSizeTo(int new_size) { + TORCH_CHECK(new_size >= size_, "extendGroupSizeTo: new_size < size_"); + size_ = new_size; +} + +// Allocate control regions only. +// Send/Recv chunk pools are owned by P2PDeviceWorker and shared across +// multiple P2PProxy instances on the same device. +void P2PProxy::allocateResources() { TORCH_CHECK(engine_, "P2PProxy engine is null."); - if (resources_.send_buffer_ != nullptr || - resources_.recv_buffer_ != nullptr || - resources_.ctrl_send_region_ != nullptr || - resources_.ctrl_recv_region_ != nullptr) { + if (resources_.credit_region_ != nullptr || + resources_.ack_region_ != nullptr) { return; } - TORCH_CHECK(size_ >= 0, "P2PProxy invalid size_: ", size_); + TORCH_CHECK(size_ > 0, "P2PProxy invalid group size: ", size_); TORCH_CHECK(static_cast(size_) <= kMaxNumRanks, - "P2PProxy size_ exceeds kMaxNumRanks: ", size_); + "P2PProxy group size exceeds kMaxNumRanks: ", size_); - if (size_ == 0) { - return; + const size_t ctrl_slots = + kMaxNumRanks * static_cast(kP2PControlRingSize); + resources_.credit_region_ = new CreditSlot[ctrl_slots]{}; + resources_.ack_region_ = new AckSlot[ctrl_slots]{}; + + for (size_t i = 0; i < ctrl_slots; ++i) { + resources_.credit_region_[i].reset(); + resources_.ack_region_[i].reset(); + } + + for (size_t i = 0; i < kMaxNumRanks; ++i) { + peer_epoch_[i].store(1, std::memory_order_release); } - const size_t p2p_total_buffer_size = - kP2PBufferSize * static_cast(size_); - - if (is_cpu_) { - resources_.send_buffer_ = std::malloc(p2p_total_buffer_size); - TORCH_CHECK(resources_.send_buffer_ != nullptr, - "Failed to allocate CPU P2P send buffer"); - int rc = engine_->registerLocalMemory(resources_.send_buffer_, - p2p_total_buffer_size, location_); - TORCH_CHECK(rc == 0, "Failed to register CPU P2P send buffer"); - - resources_.recv_buffer_ = std::malloc(p2p_total_buffer_size); - TORCH_CHECK(resources_.recv_buffer_ != nullptr, - "Failed to allocate CPU P2P recv buffer"); - rc = engine_->registerLocalMemory(resources_.recv_buffer_, - p2p_total_buffer_size, location_); - TORCH_CHECK(rc == 0, "Failed to register CPU P2P recv buffer"); - } else { - SetCudaDeviceIfNeeded( - is_cpu_, cuda_device_index_, - "P2PProxy AllocateResources cudaSetDevice failed"); - cudaError_t err = - cudaMalloc(&resources_.send_buffer_, p2p_total_buffer_size); - TORCH_CHECK(err == cudaSuccess, - "Failed to allocate CUDA P2P send buffer"); - int rc = engine_->registerLocalMemory(resources_.send_buffer_, - p2p_total_buffer_size, location_); - TORCH_CHECK(rc == 0, "Failed to register CUDA P2P send buffer"); - - err = cudaMalloc(&resources_.recv_buffer_, p2p_total_buffer_size); - TORCH_CHECK(err == cudaSuccess, - "Failed to allocate CUDA P2P recv buffer"); - rc = engine_->registerLocalMemory(resources_.recv_buffer_, - p2p_total_buffer_size, location_); - TORCH_CHECK(rc == 0, "Failed to register CUDA P2P recv buffer"); - } - - resources_.ctrl_send_region_ = new P2PControlSlot[kMaxNumRanks]{}; - resources_.ctrl_recv_region_ = new P2PControlSlot[kMaxNumRanks]{}; for (size_t i = 0; i < kMaxNumRanks; ++i) { - resources_.ctrl_send_region_[i].head.store(0, - std::memory_order_relaxed); - resources_.ctrl_send_region_[i].tail.store(0, - std::memory_order_relaxed); - resources_.ctrl_recv_region_[i].head.store(0, - std::memory_order_relaxed); - resources_.ctrl_recv_region_[i].tail.store(0, - std::memory_order_relaxed); - send_peer_lanes_[i].local_head_ = 0; send_peer_lanes_[i].pending_send_ops_.clear(); send_peer_lanes_[i].active_send_op_.reset(); + send_peer_lanes_[i].credit_consume_seq_ = 0; send_peer_lanes_[i].copy_ready_events_.fill(nullptr); - recv_peer_lanes_[i].local_tail_ = 0; + recv_peer_lanes_[i].pending_recv_ops_.clear(); recv_peer_lanes_[i].active_recv_op_.reset(); + recv_peer_lanes_[i].credit_issue_seq_ = 0; + recv_peer_lanes_[i].ack_consume_seq_ = 0; recv_peer_lanes_[i].copy_ready_events_.fill(nullptr); } @@ -148,9 +240,6 @@ void P2PProxy::AllocateResources() { for (int peer_rank = 0; peer_rank < size_; ++peer_rank) { for (auto& copy_ready_event : send_peer_lanes_[peer_rank].copy_ready_events_) { - if (copy_ready_event != nullptr) { - continue; - } const cudaError_t create_error = cudaEventCreateWithFlags( ©_ready_event, cudaEventDisableTiming); TORCH_CHECK(create_error == cudaSuccess, @@ -160,9 +249,6 @@ void P2PProxy::AllocateResources() { for (auto& copy_ready_event : recv_peer_lanes_[peer_rank].copy_ready_events_) { - if (copy_ready_event != nullptr) { - continue; - } const cudaError_t create_error = cudaEventCreateWithFlags( ©_ready_event, cudaEventDisableTiming); TORCH_CHECK(create_error == cudaSuccess, @@ -171,137 +257,211 @@ void P2PProxy::AllocateResources() { } } } - int rc = engine_->registerLocalMemory(resources_.ctrl_send_region_, - kMaxNumRanks * sizeof(P2PControlSlot), - kWildcardLocation); - TORCH_CHECK(rc == 0, "Failed to register P2P ctrl send region"); - rc = engine_->registerLocalMemory(resources_.ctrl_recv_region_, - kMaxNumRanks * sizeof(P2PControlSlot), + int rc = engine_->registerLocalMemory(resources_.credit_region_, + ctrl_slots * sizeof(CreditSlot), + kWildcardLocation); + TORCH_CHECK(rc == 0, "Failed to register P2P credit region"); + + rc = engine_->registerLocalMemory(resources_.ack_region_, + ctrl_slots * sizeof(AckSlot), kWildcardLocation); - TORCH_CHECK(rc == 0, "Failed to register P2P ctrl recv region"); + TORCH_CHECK(rc == 0, "Failed to register P2P ack region"); + + // Staging buffers for control messages. RDMA requires every + // transfer source to live in a registered MR. + resources_.credit_staging_buf_ = new CreditSlot[ctrl_slots]{}; + rc = engine_->registerLocalMemory(resources_.credit_staging_buf_, + ctrl_slots * sizeof(CreditSlot), + kWildcardLocation); + TORCH_CHECK(rc == 0, "Failed to register P2P credit staging region"); + + resources_.ack_staging_buf_ = new AckSlot[ctrl_slots]{}; + rc = engine_->registerLocalMemory(resources_.ack_staging_buf_, + ctrl_slots * sizeof(AckSlot), + kWildcardLocation); + TORCH_CHECK(rc == 0, "Failed to register P2P ack staging region"); } -void P2PProxy::ResetPeerState(int peer_rank) { +void P2PProxy::resetPeerState(int peer_rank) { TORCH_CHECK(peer_rank >= 0 && peer_rank < size_, "ResetPeerState: peer_rank out of range: ", peer_rank, " size: ", size_); + + // Epoch update: + // We bump epoch_ so that any slots still in flight from the + // old session (written by the sender/receiver before it learned about + // the Reset) are recognized as stale and skipped. + peer_epoch_[peer_rank].fetch_add(1, std::memory_order_acq_rel); + + // Request reset reset_send_req_[peer_rank].store(true, std::memory_order_release); reset_recv_req_[peer_rank].store(true, std::memory_order_release); + + // Wake up worker if (device_worker_) { - device_worker_->WakeUpSend(); - device_worker_->WakeUpRecv(); + device_worker_->wakeUpSend(); + device_worker_->wakeUpRecv(); } } -void P2PProxy::PerformSendReset(int peer_rank) { - auto& lane = send_peer_lanes_[peer_rank]; +// Reset sender state for peer_rank. +void P2PProxy::performSendReset(int peer_rank) { + resetSendLane(send_peer_lanes_[peer_rank]); + resetPeerControlLanes(peer_rank); +} + +// Reset receiver state for peer_rank. +void P2PProxy::performRecvReset(int peer_rank) { + resetRecvLane(recv_peer_lanes_[peer_rank]); + resetPeerControlLanes(peer_rank); +} + +void P2PProxy::reportBrokenPeer(int peer_rank) { + resetPeerState(peer_rank); + // Set peerConnected to notify the connection poller to reconnect it. + meta_->peerConnected[peer_rank] = false; + meta_->activeRanks[peer_rank] = false; + meta_->activeRanksTensor[peer_rank] = 0; + LOG(ERROR) << "Rank " << meta_->rank << " marking peer " << peer_rank + << " as broken during P2P transfer."; +} + +void P2PProxy::releaseSendTaskResources(SendTransferTask& task) const { + if (task.staging_addr_ != nullptr) { + if (send_pool_) send_pool_->release(task.staging_addr_); + task.staging_addr_ = nullptr; + } + if (task.transfer_batch_id_.has_value()) { + if (engine_) engine_->freeBatchID(task.transfer_batch_id_.value()); + task.transfer_batch_id_.reset(); + } + if (task.ack_batch_id_.has_value()) { + if (engine_) engine_->freeBatchID(task.ack_batch_id_.value()); + task.ack_batch_id_.reset(); + } +} + +void P2PProxy::releaseRecvTaskResources(RecvTransferTask& task) const { + if (task.local_addr_ != nullptr) { + if (recv_pool_) recv_pool_->release(task.local_addr_); + task.local_addr_ = nullptr; + } + if (task.credit_batch_id_.has_value()) { + if (engine_) engine_->freeBatchID(task.credit_batch_id_.value()); + task.credit_batch_id_.reset(); + } +} + +void P2PProxy::resetSendLane(SendPeerLane& lane) { + for (auto& pending : lane.pending_send_ops_) { + pending.status_->store(OpStatus::kFailed, std::memory_order_release); + active_send_tasks_.fetch_sub(1, std::memory_order_release); + } lane.pending_send_ops_.clear(); - lane.active_send_op_.reset(); - lane.local_head_ = 0; - if (resources_.ctrl_send_region_) { - resources_.ctrl_send_region_[peer_rank].head.store( - 0, std::memory_order_relaxed); - resources_.ctrl_send_region_[peer_rank].tail.store( - 0, std::memory_order_relaxed); + if (lane.active_send_op_.has_value()) { + auto& op_ctx = lane.active_send_op_.value(); + for (auto& task : op_ctx.tasks_) { + releaseSendTaskResources(task); + } + op_ctx.tasks_.clear(); + op_ctx.status_->store(OpStatus::kFailed, std::memory_order_release); + active_send_tasks_.fetch_sub(1, std::memory_order_release); + lane.active_send_op_.reset(); } + lane.credit_consume_seq_ = 0; } -void P2PProxy::PerformRecvReset(int peer_rank) { - auto& lane = recv_peer_lanes_[peer_rank]; +void P2PProxy::resetRecvLane(RecvPeerLane& lane) { + for (auto& pending : lane.pending_recv_ops_) { + pending.status_->store(OpStatus::kFailed, std::memory_order_release); + active_recv_tasks_.fetch_sub(1, std::memory_order_release); + } lane.pending_recv_ops_.clear(); - lane.active_recv_op_.reset(); - lane.local_tail_ = 0; - if (resources_.ctrl_recv_region_) { - resources_.ctrl_recv_region_[peer_rank].head.store( - 0, std::memory_order_relaxed); - resources_.ctrl_recv_region_[peer_rank].tail.store( - 0, std::memory_order_relaxed); + if (lane.active_recv_op_.has_value()) { + auto& op_ctx = lane.active_recv_op_.value(); + for (auto& task : op_ctx.tasks_) { + releaseRecvTaskResources(task); + } + op_ctx.tasks_.clear(); + op_ctx.status_->store(OpStatus::kFailed, std::memory_order_release); + active_recv_tasks_.fetch_sub(1, std::memory_order_release); + lane.active_recv_op_.reset(); + } + lane.credit_issue_seq_ = 0; + lane.ack_consume_seq_ = 0; +} + +void P2PProxy::resetPeerControlLanes(int peer_rank) { + auto* credit_lane = getLocalCreditLane(peer_rank); + auto* ack_lane = getLocalAckLane(peer_rank); + for (uint32_t i = 0; i < kP2PControlRingSize; ++i) { + credit_lane[i].reset(); + ack_lane[i].reset(); } } -void P2PProxy::ReleaseResources() { +void P2PProxy::releaseResources() { TORCH_CHECK(!resource_abandoned_, "Should not release abandoned resources."); - SetCudaDeviceIfNeeded(is_cpu_, cuda_device_index_, + setCudaDeviceIfNeeded(is_cpu_, cuda_device_index_, "P2PProxy ReleaseResources cudaSetDevice failed"); for (size_t i = 0; i < kMaxNumRanks; ++i) { + // Reset lane auto& send_lane = send_peer_lanes_[i]; - for (auto& copy_ready_event : send_lane.copy_ready_events_) { - if (copy_ready_event == nullptr) { - continue; - } - const cudaError_t destroy_error = - cudaEventDestroy(copy_ready_event); - TORCH_CHECK(destroy_error == cudaSuccess, - "Failed to destroy pooled send copy-ready event: ", - cudaGetErrorString(destroy_error)); - copy_ready_event = nullptr; - } - send_lane.pending_send_ops_.clear(); - send_lane.active_send_op_.reset(); - send_lane.local_head_ = 0; - auto& recv_lane = recv_peer_lanes_[i]; - for (auto& copy_ready_event : recv_lane.copy_ready_events_) { - if (copy_ready_event == nullptr) { - continue; + resetSendLane(send_lane); + resetRecvLane(recv_lane); + + // Destroy cuda events + auto destroyEvents = [](auto& events) { + for (auto& ev : events) { + if (ev == nullptr) continue; + const cudaError_t err = cudaEventDestroy(ev); + TORCH_CHECK(err == cudaSuccess, + "Failed to destroy pooled copy-ready event: ", + cudaGetErrorString(err)); + ev = nullptr; } - const cudaError_t destroy_error = - cudaEventDestroy(copy_ready_event); - TORCH_CHECK(destroy_error == cudaSuccess, - "Failed to destroy pooled recv copy-ready event: ", - cudaGetErrorString(destroy_error)); - copy_ready_event = nullptr; - } - recv_lane.pending_recv_ops_.clear(); - recv_lane.active_recv_op_.reset(); + }; + destroyEvents(send_lane.copy_ready_events_); + destroyEvents(recv_lane.copy_ready_events_); } - if (!engine_) { - return; + if (resources_.credit_staging_buf_ != nullptr) { + if (engine_) + engine_->unregisterLocalMemory(resources_.credit_staging_buf_); + delete[] resources_.credit_staging_buf_; + resources_.credit_staging_buf_ = nullptr; } - if (resources_.ctrl_send_region_ != nullptr) { - engine_->unregisterLocalMemory(resources_.ctrl_send_region_); - delete[] resources_.ctrl_send_region_; - resources_.ctrl_send_region_ = nullptr; + if (resources_.ack_staging_buf_ != nullptr) { + if (engine_) + engine_->unregisterLocalMemory(resources_.ack_staging_buf_); + delete[] resources_.ack_staging_buf_; + resources_.ack_staging_buf_ = nullptr; } - if (resources_.ctrl_recv_region_ != nullptr) { - engine_->unregisterLocalMemory(resources_.ctrl_recv_region_); - delete[] resources_.ctrl_recv_region_; - resources_.ctrl_recv_region_ = nullptr; + if (resources_.credit_region_ != nullptr) { + if (engine_) engine_->unregisterLocalMemory(resources_.credit_region_); + delete[] resources_.credit_region_; + resources_.credit_region_ = nullptr; } - if (resources_.send_buffer_ != nullptr) { - engine_->unregisterLocalMemory(resources_.send_buffer_); - if (is_cpu_) { - std::free(resources_.send_buffer_); - } else { - cudaFree(resources_.send_buffer_); - } - resources_.send_buffer_ = nullptr; - } - - if (resources_.recv_buffer_ != nullptr) { - engine_->unregisterLocalMemory(resources_.recv_buffer_); - if (is_cpu_) { - std::free(resources_.recv_buffer_); - } else { - cudaFree(resources_.recv_buffer_); - } - resources_.recv_buffer_ = nullptr; + if (resources_.ack_region_ != nullptr) { + if (engine_) engine_->unregisterLocalMemory(resources_.ack_region_); + delete[] resources_.ack_region_; + resources_.ack_region_ = nullptr; } } -void P2PProxy::AbandonResources() { resource_abandoned_ = true; } +void P2PProxy::abandonResources() { resource_abandoned_ = true; } -void P2PProxy::EnqueueSend(SendOp op) { +void P2PProxy::enqueueSend(SendOp op) { op.tensor_ = op.tensor_.is_contiguous() ? op.tensor_ : op.tensor_.contiguous(); @@ -310,364 +470,221 @@ void P2PProxy::EnqueueSend(SendOp op) { send_queue_.emplace(std::move(op)); } active_send_tasks_.fetch_add(1, std::memory_order_release); - if (device_worker_) device_worker_->WakeUpSend(); + if (device_worker_) device_worker_->wakeUpSend(); } -void P2PProxy::EnqueueRecv(RecvOp op) { +void P2PProxy::enqueueRecv(RecvOp op) { { std::lock_guard lock(recv_queue_mutex_); recv_queue_.push(std::move(op)); } active_recv_tasks_.fetch_add(1, std::memory_order_release); - if (device_worker_) device_worker_->WakeUpRecv(); + if (device_worker_) device_worker_->wakeUpRecv(); } -P2PProxy::SendTransferTask::SendTransferTask(uint64_t chunk_offset_in, - uint64_t chunk_bytes_in, - void* source_in, - uint64_t target_offset_in) - : chunk_offset_(chunk_offset_in), - chunk_bytes_(chunk_bytes_in), - source_(source_in), - target_offset_(target_offset_in) {} +P2PProxy::SendTransferTask::SendTransferTask( + uint64_t tensor_offset_in, uint32_t chunk_len_in, void* staging_addr_in, + uint64_t remote_addr_in, uint32_t sequence_in, uint32_t epoch_in) + : tensor_offset_(tensor_offset_in), + chunk_len_(chunk_len_in), + staging_addr_(staging_addr_in), + remote_addr_(remote_addr_in), + sequence_(sequence_in), + epoch_(epoch_in) { + last_update_time_ = std::chrono::steady_clock::now(); +} P2PProxy::SendOpContext::SendOpContext(SendOp&& op_in) - : tensor_(std::move(op_in.tensor_)), + : status_(std::move(op_in.status_)), + tensor_(std::move(op_in.tensor_)), peer_rank_(op_in.peer_rank_), - cuda_stream_(op_in.cuda_stream_), - completed_(std::move(op_in.completed_)) { + cuda_stream_(op_in.cuda_stream_) { total_bytes_ = tensor_.numel() * static_cast(tensor_.element_size()); + last_update_time_ = std::chrono::steady_clock::now(); } -P2PProxy::RecvTransferTask::RecvTransferTask(uint64_t chunk_offset_in, - uint64_t chunk_bytes_in, - void* source_in, void* target_in) - : chunk_offset_(chunk_offset_in), - chunk_bytes_(chunk_bytes_in), - source_(source_in), - target_(target_in) {} +P2PProxy::RecvTransferTask::RecvTransferTask(uint64_t tensor_offset_in, + uint32_t chunk_len_in, + void* local_addr_in, + uint32_t sequence_in, + uint32_t epoch_in) + : tensor_offset_(tensor_offset_in), + chunk_len_(chunk_len_in), + local_addr_(local_addr_in), + sequence_(sequence_in), + epoch_(epoch_in) { + last_update_time_ = std::chrono::steady_clock::now(); +} P2PProxy::RecvOpContext::RecvOpContext(RecvOp&& op_in) - : tensor_(std::move(op_in.tensor_)), + : status_(std::move(op_in.status_)), + tensor_(std::move(op_in.tensor_)), original_tensor_(std::move(op_in.original_tensor_)), peer_rank_(op_in.peer_rank_), - cuda_stream_(op_in.cuda_stream_), - completed_(std::move(op_in.completed_)) { + cuda_stream_(op_in.cuda_stream_) { total_bytes_ = tensor_.numel() * static_cast(tensor_.element_size()); } -uint64_t P2PProxy::GetLocalSendSlotAddress(int peer_rank, - uint32_t slot_index) const { - const uint64_t send_addr_base = - meta_->segmentInfos[rank_].p2p_send_buffer + - static_cast(peer_rank) * kP2PBytesPerRank; - return send_addr_base + static_cast(slot_index) * kP2PSlotSize; +CreditSlot* P2PProxy::getLocalCreditLane(int peer_rank) const { + return resources_.credit_region_ + + static_cast(peer_rank) * kP2PControlRingSize; } -uint64_t P2PProxy::GetLocalRecvSlotAddress(int peer_rank, - uint32_t slot_index) const { - const uint64_t recv_addr_base = - meta_->segmentInfos[rank_].p2p_recv_buffer + - static_cast(peer_rank) * kP2PBytesPerRank; - return recv_addr_base + static_cast(slot_index) * kP2PSlotSize; +AckSlot* P2PProxy::getLocalAckLane(int peer_rank) const { + return resources_.ack_region_ + + static_cast(peer_rank) * kP2PControlRingSize; } -uint64_t P2PProxy::GetRemoteRecvSlotAddress(int peer_rank, - uint32_t slot_index) const { - const uint64_t remote_recv_addr_base = - meta_->segmentInfos[peer_rank].p2p_recv_buffer + - static_cast(rank_) * kP2PBytesPerRank; - return remote_recv_addr_base + - static_cast(slot_index) * kP2PSlotSize; +uint64_t P2PProxy::getRemoteCreditSlot(int peer_rank, uint32_t sequence) const { + const uint64_t slot_index = sequence % kP2PControlRingSize; + return meta_->segmentInfos[peer_rank].p2p_credit_region + + (static_cast(rank_) * kP2PControlRingSize + slot_index) * + sizeof(CreditSlot); } -uint64_t P2PProxy::GetRemoteCtrlRecvHeadOffset(int peer_rank) const { - return meta_->segmentInfos[peer_rank].p2p_ctrl_recv + - rank_ * sizeof(P2PControlSlot) + offsetof(P2PControlSlot, head); +uint64_t P2PProxy::getRemoteAckSlot(int peer_rank, uint32_t sequence) const { + const uint64_t slot_index = sequence % kP2PControlRingSize; + return meta_->segmentInfos[peer_rank].p2p_ack_region + + (static_cast(rank_) * kP2PControlRingSize + slot_index) * + sizeof(AckSlot); } -uint64_t P2PProxy::GetRemoteCtrlSendTailOffset(int peer_rank) const { - return meta_->segmentInfos[peer_rank].p2p_ctrl_send + - rank_ * sizeof(P2PControlSlot) + offsetof(P2PControlSlot, tail); +CreditSlot* P2PProxy::getLocalCreditStagingBuf(int peer_rank, + uint32_t sequence) const { + const size_t staging_idx = + static_cast(peer_rank) * kP2PControlRingSize + + sequence % kP2PControlRingSize; + return resources_.credit_staging_buf_ + staging_idx; +} +AckSlot* P2PProxy::getLocalAckStagingBuf(int peer_rank, + uint32_t sequence) const { + const size_t staging_idx = + static_cast(peer_rank) * kP2PControlRingSize + + static_cast(sequence % kP2PControlRingSize); + return resources_.ack_staging_buf_ + staging_idx; } -bool P2PProxy::TryIssueSendTask(SendOpContext& op_ctx, uint32_t capacity) { - if (op_ctx.bytes_issued_ >= op_ctx.total_bytes_) { +// Receiver issues credit. +// +// Grab a free chunk from RecvPool and grant the sender credit to write into it. +// We write a CreditSlot into the sender's CreditLane so the sender knows: +// (a) which sequence this is, +// (b) the RecvPool offset to write to, +// (c) how many bytes we expect. +// If the pool is exhausted we return false and retry on the next polling +// iteration (non-blocking). +bool P2PProxy::tryIssueRecvTask(RecvOpContext& op_ctx, RecvPeerLane& lane) { + if (op_ctx.bytes_credited_ >= op_ctx.total_bytes_) { return false; } - const int peer_rank = op_ctx.peer_rank_; - auto& lane = send_peer_lanes_[peer_rank]; - const uint32_t head = lane.local_head_; - const uint32_t slot_index = head; - const uint32_t remote_tail = - resources_.ctrl_send_region_[peer_rank].tail.load( - std::memory_order_acquire); - if (((head + 1) % capacity) == remote_tail) { + if (op_ctx.tasks_.size() >= kP2PControlRingSize) { + // No free control slot, retry on the next iteration return false; } - const uint64_t chunk_bytes = - std::min(static_cast(kP2PSlotSize), - op_ctx.total_bytes_ - op_ctx.bytes_issued_); - const uint64_t send_addr = GetLocalSendSlotAddress(peer_rank, head); - const uint64_t target_offset = GetRemoteRecvSlotAddress(peer_rank, head); - op_ctx.tasks_.emplace_back(op_ctx.bytes_issued_, chunk_bytes, - reinterpret_cast(send_addr), - target_offset); + void* local_addr = recv_pool_->acquire(); + if (local_addr == nullptr) { + // No free recv chunk, retry on the next iteration + return false; + } + + const uint32_t chunk_len = static_cast(std::min( + chunk_size_, op_ctx.total_bytes_ - op_ctx.bytes_credited_)); + const uint64_t seq = lane.credit_issue_seq_; + const uint64_t remote_credit_offset = + getRemoteCreditSlot(op_ctx.peer_rank_, seq); + + const uint32_t curr_epoch = + peer_epoch_[op_ctx.peer_rank_].load(std::memory_order_acquire); + op_ctx.tasks_.emplace_back(op_ctx.bytes_credited_, chunk_len, local_addr, + seq, curr_epoch); auto& task = op_ctx.tasks_.back(); - const auto* tensor_ptr = - static_cast(op_ctx.tensor_.data_ptr()); - - if (is_cpu_) { - std::memcpy(task.source_, tensor_ptr + task.chunk_offset_, - task.chunk_bytes_); - task.state_ = TransferState::kTransfer; - } else { - cudaError_t copy_error = cudaMemcpyAsync( - task.source_, tensor_ptr + task.chunk_offset_, task.chunk_bytes_, - cudaMemcpyDeviceToDevice, op_ctx.cuda_stream_); - TORCH_CHECK(!copy_error, "P2P send cudaMemcpyAsync failed: ", - cudaGetErrorString(copy_error)); - const cudaEvent_t pooled_copy_ready_event = - lane.copy_ready_events_[slot_index]; - TORCH_CHECK(pooled_copy_ready_event != nullptr, - "P2P send pooled copy-ready event is not initialized."); - task.copy_ready_event_ = pooled_copy_ready_event; - copy_error = - cudaEventRecord(task.copy_ready_event_, op_ctx.cuda_stream_); - if (copy_error != cudaSuccess) { - task.copy_ready_event_ = nullptr; - TORCH_CHECK(false, "P2P send cudaEventRecord failed: ", - cudaGetErrorString(copy_error)); - } - } - - op_ctx.bytes_issued_ += task.chunk_bytes_; - lane.local_head_ = (head + 1) % capacity; - return true; -} - -bool P2PProxy::StepSendTransferTask(SendOpContext& op_ctx, - SendTransferTask& task) { - bool did_work = false; - - if (task.state_ == TransferState::kDataCopy && StepSendDataCopy(task)) { - did_work = true; - } - if (task.state_ == TransferState::kTransfer && - StepSendTransfer(op_ctx, task)) { - did_work = true; - } - - return did_work; -} - -bool P2PProxy::StepSendDataCopy(SendTransferTask& task) { - if (task.copy_ready_event_ == nullptr) { - task.state_ = TransferState::kTransfer; - return true; - } - - cudaError_t query_error = cudaSuccess; - query_error = cudaEventQuery(task.copy_ready_event_); - - if (query_error == cudaSuccess) { - task.copy_ready_event_ = nullptr; - task.state_ = TransferState::kTransfer; - return true; - } - if (query_error == cudaErrorNotReady) { - return false; - } - - task.copy_ready_event_ = nullptr; - TORCH_CHECK(false, "P2P send cudaEventQuery failed: ", - cudaGetErrorString(query_error)); - return false; -} - -bool P2PProxy::StepSendTransfer(SendOpContext& op_ctx, SendTransferTask& task) { - bool did_work = false; - if (!task.transfer_batch_id_.has_value()) { - const BatchID batch_id = engine_->allocateBatchID(1); - engine_->submitTransfer( - batch_id, {TransferRequest{ - .opcode = TransferRequest::WRITE, - .source = task.source_, - .target_id = meta_->segmentIDs[op_ctx.peer_rank_], - .target_offset = task.target_offset_, - .length = task.chunk_bytes_, - }}); - task.transfer_batch_id_ = batch_id; - did_work = true; - } - - TransferStatus transfer_status; - engine_->getTransferStatus(task.transfer_batch_id_.value(), 0, - transfer_status); - if (transfer_status.s == TransferStatusEnum::COMPLETED) { - engine_->freeBatchID(task.transfer_batch_id_.value()); - task.transfer_batch_id_.reset(); - task.state_ = TransferState::kDone; - return true; - } - if (transfer_status.s == TransferStatusEnum::FAILED) { - engine_->freeBatchID(task.transfer_batch_id_.value()); - task.transfer_batch_id_.reset(); - TORCH_CHECK(false, "P2P send transfer failed."); - return false; - } - - return did_work; -} - -bool P2PProxy::StepSendHeadCommit(SendOpContext& op_ctx, uint32_t capacity) { - bool did_work = false; - if (op_ctx.head_update_batch_id_.has_value()) { - TransferStatus head_status; - engine_->getTransferStatus(op_ctx.head_update_batch_id_.value(), 0, - head_status); - if (head_status.s == TransferStatusEnum::COMPLETED) { - engine_->freeBatchID(op_ctx.head_update_batch_id_.value()); - op_ctx.head_update_batch_id_.reset(); - did_work = true; - } else if (head_status.s == TransferStatusEnum::FAILED) { - engine_->freeBatchID(op_ctx.head_update_batch_id_.value()); - op_ctx.head_update_batch_id_.reset(); - TORCH_CHECK(false, "P2P ctrl head update failed."); - return false; - } - } - - if (op_ctx.head_update_batch_id_.has_value()) { - return did_work; - } - - uint32_t committed_tasks = 0; - while (!op_ctx.tasks_.empty() && - op_ctx.tasks_.front().state_ == TransferState::kDone) { - op_ctx.tasks_.pop_front(); - ++committed_tasks; - did_work = true; - } - - if (committed_tasks == 0) { - return did_work; - } - - const uint32_t current_head = - resources_.ctrl_send_region_[op_ctx.peer_rank_].head.load( - std::memory_order_relaxed); - const uint32_t next_head = (current_head + committed_tasks) % capacity; - resources_.ctrl_send_region_[op_ctx.peer_rank_].head.store( - next_head, std::memory_order_release); - void* head_source = static_cast( - &resources_.ctrl_send_region_[op_ctx.peer_rank_].head.value); - const uint64_t remote_head_offset = - GetRemoteCtrlRecvHeadOffset(op_ctx.peer_rank_); + auto* credit_staging_buf = getLocalCreditStagingBuf(op_ctx.peer_rank_, seq); + credit_staging_buf->publish( + curr_epoch, seq, reinterpret_cast(local_addr), chunk_len); const BatchID batch_id = engine_->allocateBatchID(1); engine_->submitTransfer( batch_id, {TransferRequest{ .opcode = TransferRequest::WRITE, - .source = head_source, + .source = static_cast(credit_staging_buf), .target_id = meta_->segmentIDs[op_ctx.peer_rank_], - .target_offset = remote_head_offset, - .length = sizeof(uint32_t), + .target_offset = remote_credit_offset, + .length = sizeof(CreditSlot), }}); - op_ctx.head_update_batch_id_ = batch_id; - did_work = true; + task.credit_batch_id_ = batch_id; + task.last_update_time_ = std::chrono::steady_clock::now(); - return did_work; -} - -bool P2PProxy::IsSendDataPathCompleted(const SendOpContext& op_ctx) const { - return op_ctx.bytes_issued_ == op_ctx.total_bytes_ && op_ctx.tasks_.empty(); -} - -bool P2PProxy::IsSendOpCompleted(const SendOpContext& op_ctx) const { - return IsSendDataPathCompleted(op_ctx) && - !op_ctx.head_update_batch_id_.has_value(); -} - -bool P2PProxy::TryIssueRecvTask(RecvOpContext& op_ctx, uint32_t capacity) { - if (op_ctx.bytes_issued_ >= op_ctx.total_bytes_) { - return false; - } - - auto* local_ctrl = &resources_.ctrl_recv_region_[op_ctx.peer_rank_]; - auto& lane = recv_peer_lanes_[op_ctx.peer_rank_]; - const uint32_t head = local_ctrl->head.load(std::memory_order_acquire); - const uint32_t tail = lane.local_tail_; - const uint32_t slot_index = tail; - if (head == tail) { - return false; - } - - const uint64_t chunk_bytes = - std::min(static_cast(kP2PSlotSize), - op_ctx.total_bytes_ - op_ctx.bytes_issued_); - const uint64_t recv_addr = GetLocalRecvSlotAddress(op_ctx.peer_rank_, tail); - auto* tensor_ptr = static_cast(op_ctx.tensor_.data_ptr()); - void* target_ptr = static_cast(tensor_ptr + op_ctx.bytes_issued_); - - op_ctx.tasks_.emplace_back(op_ctx.bytes_issued_, chunk_bytes, - reinterpret_cast(recv_addr), target_ptr); - auto& task = op_ctx.tasks_.back(); - - if (is_cpu_) { - std::memcpy(task.target_, task.source_, task.chunk_bytes_); - task.state_ = TransferState::kDone; - } else { - cudaError_t copy_error = - cudaMemcpyAsync(task.target_, task.source_, task.chunk_bytes_, - cudaMemcpyDeviceToDevice, op_ctx.cuda_stream_); - TORCH_CHECK(!copy_error, "P2P recv cudaMemcpyAsync failed: ", - cudaGetErrorString(copy_error)); - const cudaEvent_t pooled_copy_ready_event = - lane.copy_ready_events_[slot_index]; - TORCH_CHECK(pooled_copy_ready_event != nullptr, - "P2P recv pooled copy-ready event is not initialized."); - task.copy_ready_event_ = pooled_copy_ready_event; - copy_error = - cudaEventRecord(task.copy_ready_event_, op_ctx.cuda_stream_); - if (copy_error != cudaSuccess) { - task.copy_ready_event_ = nullptr; - TORCH_CHECK(false, "P2P recv cudaEventRecord failed: ", - cudaGetErrorString(copy_error)); - } - } - - op_ctx.bytes_issued_ += task.chunk_bytes_; - lane.local_tail_ = (tail + 1) % capacity; + ++lane.credit_issue_seq_; + op_ctx.bytes_credited_ += chunk_len; return true; } -bool P2PProxy::StepRecvTransferTask(RecvTransferTask& task) { - bool did_work = false; +// Drive a single receiver chunk through its state machine. +// +// kIssueCredit -> kWaitAck -> kCopyOut -> kFinished -> erase. +bool P2PProxy::stepRecvTask(RecvTransferTask& task) { + switch (task.state_) { + case RecvTaskState::kIssueCredit: + return stepRecvIssueCredit(task); - if (task.state_ == TransferState::kDataCopy && StepRecvDataCopy(task)) { - did_work = true; + case RecvTaskState::kWaitAck: + // kWaitAck is polled in stepRecv + return false; + + case RecvTaskState::kCopyOut: + return stepRecvCopyOut(task); + + case RecvTaskState::kFinished: + case RecvTaskState::kFailed: + return false; } - - return did_work; + return false; } -bool P2PProxy::StepRecvDataCopy(RecvTransferTask& task) { - if (task.copy_ready_event_ == nullptr) { - task.state_ = TransferState::kDone; +bool P2PProxy::stepRecvIssueCredit(RecvTransferTask& task) { + TORCH_CHECK(task.credit_batch_id_.has_value(), + "Expected a credit_batch_id in tryIssueRecvTask"); + + TransferStatus credit_status; + engine_->getTransferStatus(task.credit_batch_id_.value(), 0, credit_status); + + if (credit_status.s == TransferStatusEnum::COMPLETED) { + engine_->freeBatchID(task.credit_batch_id_.value()); + task.credit_batch_id_.reset(); + task.state_ = RecvTaskState::kWaitAck; + task.last_update_time_ = std::chrono::steady_clock::now(); return true; } - cudaError_t query_error = cudaSuccess; - query_error = cudaEventQuery(task.copy_ready_event_); + if (credit_status.s == TransferStatusEnum::FAILED || isTimeout(task)) { + LOG(ERROR) << "P2P credit transfer failed/timeout, seq=" + << task.sequence_; + engine_->freeBatchID(task.credit_batch_id_.value()); + task.credit_batch_id_.reset(); + task.state_ = RecvTaskState::kFailed; + return true; + } + return false; +} +// Poll the GPU Copy-Out event. +// +// After the AckSlot arrives we initiate cudaMemcpyAsync from the +// RecvPool chunk to the user tensor and record an event. This function +// waits for that event and returns the chunk to RecvPool. +bool P2PProxy::stepRecvCopyOut(RecvTransferTask& task) { + TORCH_CHECK(task.copy_ready_event_ != nullptr, + "Expected a copy_ready_event"); + cudaError_t query_error = cudaEventQuery(task.copy_ready_event_); if (query_error == cudaSuccess) { task.copy_ready_event_ = nullptr; - task.state_ = TransferState::kDone; + recv_pool_->release(task.local_addr_); + task.local_addr_ = nullptr; + task.state_ = RecvTaskState::kFinished; return true; } if (query_error == cudaErrorNotReady) { @@ -680,82 +697,286 @@ bool P2PProxy::StepRecvDataCopy(RecvTransferTask& task) { return false; } -bool P2PProxy::StepRecvTailCommit(RecvOpContext& op_ctx, uint32_t capacity) { - bool did_work = false; - if (op_ctx.tail_update_batch_id_.has_value()) { - TransferStatus tail_status; - engine_->getTransferStatus(op_ctx.tail_update_batch_id_.value(), 0, - tail_status); - if (tail_status.s == TransferStatusEnum::COMPLETED) { - engine_->freeBatchID(op_ctx.tail_update_batch_id_.value()); - op_ctx.tail_update_batch_id_.reset(); - did_work = true; - } else if (tail_status.s == TransferStatusEnum::FAILED) { - engine_->freeBatchID(op_ctx.tail_update_batch_id_.value()); - op_ctx.tail_update_batch_id_.reset(); - TORCH_CHECK(false, "P2P ctrl tail update failed."); - return false; +// Sender fetches credits. +// +// Poll the local CreditLane for the next expected sequence. If the slot +// matches our consume cursor we accept the credit: allocate a staging +// chunk from SendPool, copy the corresponding slice of the user tensor into +// it, and advance the cursor. If the pool is full or no credit has +// arrived we return false immediately (non-blocking). +bool P2PProxy::tryIssueSendTask(SendOpContext& op_ctx, SendPeerLane& lane) { + // Check for timeout while waiting for the peer's CreditSlot. + if (isTimeout(op_ctx)) { + LOG(ERROR) << "P2P wait-for-credit timeout, peer=" << op_ctx.peer_rank_; + op_ctx.status_->store(OpStatus::kFailed, std::memory_order_release); + return false; + } + + if (op_ctx.bytes_staged_ >= op_ctx.total_bytes_) { + return false; + } + + if (op_ctx.tasks_.size() >= kP2PControlRingSize) { + return false; + } + + CreditSlot* local_credit = getLocalCreditLane(op_ctx.peer_rank_); + const uint64_t seq = lane.credit_consume_seq_; + auto& slot = local_credit[static_cast(seq % kP2PControlRingSize)]; + + // Step 1 -- Try load the slot + uint64_t recv_addr = 0; + uint32_t chunk_len = 0; + uint32_t slot_epoch = 0; + uint32_t slot_seq = 0; + if (!slot.tryLoad(recv_addr, chunk_len, slot_epoch, slot_seq)) { + // Slot is either empty or torn (partial RDMA write). Retry next poll. + return false; + } + + // Step 2 -- Stale packet: the slot carries data from a previous epoch + // (before a Reset). Clear it so the fresh credit can land safely. + const uint32_t curr_epoch = + peer_epoch_[op_ctx.peer_rank_].load(std::memory_order_acquire); + if (slot_epoch != curr_epoch) { + LOG(WARNING) << "[P2PProxy][Send] tryIssueSendTask peer=" + << op_ctx.peer_rank_ << " STALE_EPOCH seq=" << seq + << " slot.epoch=" << slot_epoch + << " curr_epoch=" << curr_epoch; + slot.reset(); + return true; + } + + // Step 3 -- Sequence check: make sure this is exactly the slot we expect. + if (slot_seq != static_cast(seq)) { + return false; + } + + void* staging_addr = send_pool_->acquire(); + if (staging_addr == nullptr) { + return false; + } + + const uint32_t expected_chunk_len = + static_cast(std::min( + chunk_size_, op_ctx.total_bytes_ - op_ctx.bytes_staged_)); + TORCH_CHECK(chunk_len <= expected_chunk_len, + "P2P send got invalid chunk_len in credit slot."); + + op_ctx.tasks_.emplace_back(op_ctx.bytes_staged_, chunk_len, staging_addr, + recv_addr, seq, slot_epoch); + auto& task = op_ctx.tasks_.back(); + + slot.reset(); + + const auto* tensor_ptr = + static_cast(op_ctx.tensor_.data_ptr()); + if (is_cpu_) { + std::memcpy(staging_addr, tensor_ptr + task.tensor_offset_, + task.chunk_len_); + task.state_ = SendTaskState::kWriteRemote; + } else { + cudaError_t copy_error = cudaMemcpyAsync( + staging_addr, tensor_ptr + task.tensor_offset_, task.chunk_len_, + cudaMemcpyDeviceToDevice, op_ctx.cuda_stream_); + TORCH_CHECK(!copy_error, "P2P send cudaMemcpyAsync failed: ", + cudaGetErrorString(copy_error)); + + const cudaEvent_t pooled_copy_ready_event = + lane.copy_ready_events_[static_cast(seq % + kP2PControlRingSize)]; + TORCH_CHECK(pooled_copy_ready_event != nullptr, + "P2P send pooled copy-ready event is not initialized."); + task.copy_ready_event_ = pooled_copy_ready_event; + copy_error = + cudaEventRecord(task.copy_ready_event_, op_ctx.cuda_stream_); + if (copy_error != cudaSuccess) { + task.copy_ready_event_ = nullptr; + TORCH_CHECK(false, "P2P send cudaEventRecord failed: ", + cudaGetErrorString(copy_error)); } } - if (op_ctx.tail_update_batch_id_.has_value()) { - return did_work; + ++lane.credit_consume_seq_; + op_ctx.bytes_staged_ += task.chunk_len_; + task.last_update_time_ = std::chrono::steady_clock::now(); + return true; +} + +// Drive a single sender chunk through its state machine. +// +// kCopyIn -> kWriteRemote -> kAck -> kFinished -> erase. +bool P2PProxy::stepSendTask(SendOpContext& op_ctx, SendTransferTask& task) { + switch (task.state_) { + case SendTaskState::kCopyIn: + return stepSendCopyIn(task); + + case SendTaskState::kWriteRemote: + return stepSendWriteRemote(op_ctx, task); + + case SendTaskState::kAck: + return stepSendAck(op_ctx, task); + + case SendTaskState::kFinished: + case SendTaskState::kFailed: + return false; + } + return false; +} + +// Poll the GPU Copy-In event (or skip on CPU). +// +// When the event signals we transition from kCopyIn to kWriteRemote so the +// (RDMA) Write can be submitted. +bool P2PProxy::stepSendCopyIn(SendTransferTask& task) { + if (task.copy_ready_event_ == nullptr) { + // CPU case + task.state_ = SendTaskState::kWriteRemote; + task.last_update_time_ = std::chrono::steady_clock::now(); + return true; } - uint32_t committed_tasks = 0; - while (!op_ctx.tasks_.empty() && - op_ctx.tasks_.front().state_ == TransferState::kDone) { - op_ctx.tasks_.pop_front(); - ++committed_tasks; + cudaError_t query_error = cudaEventQuery(task.copy_ready_event_); + if (query_error == cudaErrorNotReady) { + return false; + } + + task.copy_ready_event_ = nullptr; + + if (query_error == cudaSuccess) { + task.state_ = SendTaskState::kWriteRemote; + task.last_update_time_ = std::chrono::steady_clock::now(); + return true; + } + + TORCH_CHECK(false, "P2P send cudaEventQuery failed: ", + cudaGetErrorString(query_error)); + return false; +} + +// Submit the Write from SendPool staging to remote RecvPool, then +// poll for its completion. +bool P2PProxy::stepSendWriteRemote(SendOpContext& op_ctx, + SendTransferTask& task) { + bool did_work = false; + if (!task.transfer_batch_id_.has_value()) { + const BatchID batch_id = engine_->allocateBatchID(1); + engine_->submitTransfer( + batch_id, {TransferRequest{ + .opcode = TransferRequest::WRITE, + .source = task.staging_addr_, + .target_id = meta_->segmentIDs[op_ctx.peer_rank_], + .target_offset = task.remote_addr_, + .length = task.chunk_len_, + }}); + task.transfer_batch_id_ = batch_id; + task.last_update_time_ = std::chrono::steady_clock::now(); did_work = true; } - if (committed_tasks == 0) { - return did_work; + TransferStatus transfer_status; + engine_->getTransferStatus(task.transfer_batch_id_.value(), 0, + transfer_status); + if (transfer_status.s == TransferStatusEnum::COMPLETED) { + engine_->freeBatchID(task.transfer_batch_id_.value()); + task.transfer_batch_id_.reset(); + task.state_ = SendTaskState::kAck; + task.last_update_time_ = std::chrono::steady_clock::now(); + did_work = true; + } else if (transfer_status.s == TransferStatusEnum::FAILED || + isTimeout(task)) { + LOG(ERROR) << "P2P send transfer failed/timeout, peer=" + << op_ctx.peer_rank_ << ", seq=" << task.sequence_; + engine_->freeBatchID(task.transfer_batch_id_.value()); + task.transfer_batch_id_.reset(); + task.state_ = SendTaskState::kFailed; + did_work = true; } - auto* local_ctrl = &resources_.ctrl_recv_region_[op_ctx.peer_rank_]; - const uint32_t current_tail = - local_ctrl->tail.load(std::memory_order_relaxed); - const uint32_t next_tail = (current_tail + committed_tasks) % capacity; - local_ctrl->tail.store(next_tail, std::memory_order_release); - - const uint64_t remote_tail_offset = - GetRemoteCtrlSendTailOffset(op_ctx.peer_rank_); - void* tail_source = static_cast(&local_ctrl->tail.value); - - const BatchID batch_id = engine_->allocateBatchID(1); - engine_->submitTransfer( - batch_id, {TransferRequest{ - .opcode = TransferRequest::WRITE, - .source = tail_source, - .target_id = meta_->segmentIDs[op_ctx.peer_rank_], - .target_offset = remote_tail_offset, - .length = sizeof(uint32_t), - }}); - op_ctx.tail_update_batch_id_ = batch_id; - did_work = true; - return did_work; } -bool P2PProxy::IsRecvDataPathCompleted(const RecvOpContext& op_ctx) const { - return op_ctx.bytes_issued_ == op_ctx.total_bytes_ && - op_ctx.tasks_.empty() && !op_ctx.tail_update_batch_id_.has_value(); +// Write the AckSlot back to the receiver. +// +// This tells the receiver that the data is present in its RecvPool and it +// may begin the Copy-Out. When the AckSlot write finishes we free +// the staging chunk and transition to kFinished. +bool P2PProxy::stepSendAck(SendOpContext& op_ctx, SendTransferTask& task) { + bool did_work = false; + if (!task.ack_batch_id_.has_value()) { + auto* ack_staging_buf = + getLocalAckStagingBuf(op_ctx.peer_rank_, task.sequence_); + ack_staging_buf->publish(task.epoch_, task.sequence_, task.chunk_len_); + + const BatchID batch_id = engine_->allocateBatchID(1); + engine_->submitTransfer( + batch_id, {TransferRequest{ + .opcode = TransferRequest::WRITE, + .source = static_cast(ack_staging_buf), + .target_id = meta_->segmentIDs[op_ctx.peer_rank_], + .target_offset = getRemoteAckSlot(op_ctx.peer_rank_, + task.sequence_), + .length = sizeof(AckSlot), + }}); + task.ack_batch_id_ = batch_id; + task.last_update_time_ = std::chrono::steady_clock::now(); + did_work = true; + } + + TransferStatus ack_status; + engine_->getTransferStatus(task.ack_batch_id_.value(), 0, ack_status); + if (ack_status.s == TransferStatusEnum::COMPLETED) { + engine_->freeBatchID(task.ack_batch_id_.value()); + task.ack_batch_id_.reset(); + send_pool_->release(task.staging_addr_); + task.staging_addr_ = nullptr; + task.state_ = SendTaskState::kFinished; + did_work = true; + } else if (ack_status.s == TransferStatusEnum::FAILED || isTimeout(task)) { + LOG(ERROR) << "P2P ack transfer failed/timeout, peer=" + << op_ctx.peer_rank_ << ", seq=" << task.sequence_; + engine_->freeBatchID(task.ack_batch_id_.value()); + task.ack_batch_id_.reset(); + task.state_ = SendTaskState::kFailed; + did_work = true; + } + + return did_work; } -bool P2PProxy::StepSend() { - const uint32_t capacity = static_cast(kP2PNumSlotsPerRank); +bool P2PProxy::isSendOpCompleted(const SendOpContext& op_ctx) const { + return op_ctx.bytes_staged_ == op_ctx.total_bytes_ && op_ctx.tasks_.empty(); +} + +bool P2PProxy::isRecvOpCompleted(const RecvOpContext& op_ctx) const { + return op_ctx.bytes_credited_ == op_ctx.total_bytes_ && + op_ctx.tasks_.empty(); +} + +// Sender state machine +// +// Pipeline per peer: +// 1. Drain the shared send_queue_ into the peer's pending_send_ops_. +// 2. Promote the first pending op to active_send_op_. +// 3. While we have credits (CreditSlots) and staging buffers, +// pull more tensor slices into SendPool (TryIssueSendTask). +// 4. Advance every active chunk through its state machine +// (Copy-In -> Write Remote -> Ack write). +// 5. Erase fully-acknowledged chunks. When the op is empty and all +// bytes have been staged, mark it complete. +bool P2PProxy::stepSend() { bool did_work = false; + // Handle reset first for (int peer_rank = 0; peer_rank < size_; ++peer_rank) { if (reset_send_req_[peer_rank].exchange(false, std::memory_order_acquire)) { - PerformSendReset(peer_rank); + performSendReset(peer_rank); did_work = true; } } + // Drain send_queue_ { std::lock_guard lock(send_queue_mutex_); while (!send_queue_.empty()) { @@ -767,6 +988,7 @@ bool P2PProxy::StepSend() { } } + // Promote active op for (int peer_rank = 0; peer_rank < size_; ++peer_rank) { auto& lane = send_peer_lanes_[peer_rank]; if (lane.active_send_op_.has_value() || @@ -776,7 +998,8 @@ bool P2PProxy::StepSend() { SendOpContext op_ctx = std::move(lane.pending_send_ops_.front()); lane.pending_send_ops_.pop_front(); if (op_ctx.total_bytes_ == 0) { - op_ctx.completed_->store(true, std::memory_order_release); + op_ctx.status_->store(OpStatus::kSuccess, + std::memory_order_release); active_send_tasks_.fetch_sub(1, std::memory_order_release); did_work = true; continue; @@ -785,28 +1008,49 @@ bool P2PProxy::StepSend() { did_work = true; } + // Advance state machine for (int peer_rank = 0; peer_rank < size_; ++peer_rank) { auto& lane = send_peer_lanes_[peer_rank]; if (!lane.active_send_op_.has_value()) { continue; } auto& op_ctx = lane.active_send_op_.value(); - while (TryIssueSendTask(op_ctx, capacity)) { + + // Pull as many credits as we have free chunks + while (tryIssueSendTask(op_ctx, lane)) { did_work = true; } - for (auto& task : op_ctx.tasks_) { - if (task.state_ == TransferState::kDone) { - continue; - } - if (StepSendTransferTask(op_ctx, task)) { - did_work = true; + + // Advance every chunk through the sender state machine. + // send op may fail due to credit-wait timeout in tryIssueSendTask. + auto op_status = op_ctx.status_->load(std::memory_order_acquire); + bool op_failed = op_status == OpStatus::kFailed; + if (!op_failed) { + for (auto it = op_ctx.tasks_.begin(); it != op_ctx.tasks_.end();) { + if (stepSendTask(op_ctx, *it)) { + did_work = true; + } + if (it->state_ == SendTaskState::kFinished) { + it = op_ctx.tasks_.erase(it); + did_work = true; + } else if (it->state_ == SendTaskState::kFailed) { + op_failed = true; + break; + } else { + ++it; + } } } - if (StepSendHeadCommit(op_ctx, capacity)) { + + if (op_failed) { + reportBrokenPeer(peer_rank); did_work = true; + continue; } - if (IsSendOpCompleted(op_ctx)) { - op_ctx.completed_->store(true, std::memory_order_release); + + if (isSendOpCompleted(op_ctx)) { + op_ctx.status_->store(OpStatus::kSuccess, + std::memory_order_release); lane.active_send_op_.reset(); active_send_tasks_.fetch_sub(1, std::memory_order_release); did_work = true; @@ -816,18 +1060,114 @@ bool P2PProxy::StepSend() { return did_work; } -bool P2PProxy::StepRecv() { - const uint32_t capacity = static_cast(kP2PNumSlotsPerRank); +// Poll the local AckLane for the head task and initiate Copy-Out. +// +// Returns true if an ack was processed (including stale-slot clearing). +bool P2PProxy::pollRecvAckSlot(RecvOpContext& op_ctx, RecvPeerLane& lane, + RecvTransferTask& head_task) { + // Check for timeout while waiting for the peer's AckSlot. + if (isTimeout(head_task)) { + LOG(ERROR) << "P2P wait-for-ack timeout, peer=" << op_ctx.peer_rank_ + << " seq=" << head_task.sequence_; + head_task.state_ = RecvTaskState::kFailed; + return true; + } + + auto* ack_lane = getLocalAckLane(op_ctx.peer_rank_); + auto& slot = ack_lane[head_task.sequence_ % kP2PControlRingSize]; + + // Step 1 -- Try load the slot + uint32_t ack_len = 0; + uint32_t slot_epoch = 0; + uint32_t slot_seq = 0; + if (!slot.tryLoad(ack_len, slot_epoch, slot_seq)) { + // Slot is either empty or torn. Retry next poll. + return false; + } + + const uint32_t curr_epoch = + peer_epoch_[op_ctx.peer_rank_].load(std::memory_order_acquire); + + // Step 2 -- Stale packet: data from a previous epoch (before Reset). + // Clear the slot so the fresh ack can land safely. + if (slot_epoch != curr_epoch) { + LOG(WARNING) << "[P2PProxy][Recv] pollRecvAckSlot peer=" + << op_ctx.peer_rank_ + << " front-seq=" << head_task.sequence_ + << " EPOCH_MISMATCH slot.epoch=" << slot_epoch + << " curr_epoch=" << curr_epoch; + slot.reset(); + return true; + } + + // Step 3 -- Sequence check: make sure this is the exact ack + // we are waiting for. + if (slot_seq != head_task.sequence_) { + return false; + } + + void* src_ptr = head_task.local_addr_; + auto* tensor_ptr = static_cast(op_ctx.tensor_.data_ptr()); + void* dst_ptr = tensor_ptr + head_task.tensor_offset_; + + if (is_cpu_) { + std::memcpy(dst_ptr, src_ptr, head_task.chunk_len_); + recv_pool_->release(head_task.local_addr_); + head_task.local_addr_ = nullptr; + head_task.state_ = RecvTaskState::kFinished; + } else { + cudaError_t copy_error = + cudaMemcpyAsync(dst_ptr, src_ptr, head_task.chunk_len_, + cudaMemcpyDeviceToDevice, op_ctx.cuda_stream_); + TORCH_CHECK(!copy_error, "P2P recv cudaMemcpyAsync failed: ", + cudaGetErrorString(copy_error)); + const cudaEvent_t pooled_copy_ready_event = + lane.copy_ready_events_[static_cast(head_task.sequence_ % + kP2PControlRingSize)]; + TORCH_CHECK(pooled_copy_ready_event != nullptr, + "P2P recv pooled copy-ready event is not initialized."); + head_task.copy_ready_event_ = pooled_copy_ready_event; + copy_error = + cudaEventRecord(head_task.copy_ready_event_, op_ctx.cuda_stream_); + if (copy_error != cudaSuccess) { + head_task.copy_ready_event_ = nullptr; + TORCH_CHECK(false, "P2P recv cudaEventRecord failed: ", + cudaGetErrorString(copy_error)); + } + head_task.state_ = RecvTaskState::kCopyOut; + } + + head_task.last_update_time_ = std::chrono::steady_clock::now(); + slot.reset(); + ++lane.ack_consume_seq_; + return true; +} + +// Receiver state machine +// +// Pipeline per peer: +// 1. Drain the shared recv_queue_ into the peer's pending_recv_ops_. +// 2. Promote the first pending op to active_recv_op_. +// 3. While we have free RecvPool chunks, issue CreditSlots to the sender +// (TryIssueRecvTask). +// 4. Poll the local AckLane in order. When an AckSlot +// arrives, initiate Copy-Out from RecvPool to the user tensor. +// 5. Advance every active chunk through its state machine +// (IssueCredit -> Copy-Out -> erase). +// 6. When all chunks are copied out, mark the op complete. +bool P2PProxy::stepRecv() { bool did_work = false; + // Handle reset first for (int peer_rank = 0; peer_rank < size_; ++peer_rank) { if (reset_recv_req_[peer_rank].exchange(false, std::memory_order_acquire)) { - PerformRecvReset(peer_rank); + performRecvReset(peer_rank); did_work = true; } } + // Drain recv_queue_ { std::lock_guard lock(recv_queue_mutex_); while (!recv_queue_.empty()) { @@ -839,6 +1179,7 @@ bool P2PProxy::StepRecv() { } } + // Promote active op for (int peer_rank = 0; peer_rank < size_; ++peer_rank) { auto& lane = recv_peer_lanes_[peer_rank]; if (lane.active_recv_op_.has_value() || @@ -847,36 +1188,62 @@ bool P2PProxy::StepRecv() { } RecvOp recv_op = std::move(lane.pending_recv_ops_.front()); lane.pending_recv_ops_.pop_front(); - lane.local_tail_ = resources_.ctrl_recv_region_[peer_rank].tail.load( - std::memory_order_acquire); RecvOpContext op_ctx(std::move(recv_op)); lane.active_recv_op_ = std::move(op_ctx); did_work = true; } + // Advance state machine for (int peer_rank = 0; peer_rank < size_; ++peer_rank) { auto& lane = recv_peer_lanes_[peer_rank]; if (!lane.active_recv_op_.has_value()) { continue; } + auto& op_ctx = lane.active_recv_op_.value(); - while (TryIssueRecvTask(op_ctx, capacity)) { + + // Offer as many RecvPool chunks as we have free buffers and tensor + // bytes remaining. + while (tryIssueRecvTask(op_ctx, lane)) { did_work = true; } - for (auto& task : op_ctx.tasks_) { - if (task.state_ == TransferState::kDone) { - continue; + + // In-order ack polling: we only look at the head of the queue + // because the sender acknowledges chunks in the same order it + // consumes credits. + if (!op_ctx.tasks_.empty()) { + auto& head = op_ctx.tasks_.front(); + if (head.state_ == RecvTaskState::kWaitAck) { + if (pollRecvAckSlot(op_ctx, lane, head)) { + did_work = true; + } } - if (StepRecvTransferTask(task)) { + } + + // Advance every chunk through the receiver state machine. + bool op_failed = false; + for (auto it = op_ctx.tasks_.begin(); it != op_ctx.tasks_.end();) { + did_work |= stepRecvTask(*it); + if (it->state_ == RecvTaskState::kFinished) { + it = op_ctx.tasks_.erase(it); did_work = true; + } else if (it->state_ == RecvTaskState::kFailed) { + op_failed = true; + break; + } else { + ++it; } } - if (StepRecvTailCommit(op_ctx, capacity)) { + + if (op_failed) { + reportBrokenPeer(peer_rank); did_work = true; + continue; } - if (IsRecvDataPathCompleted(op_ctx)) { + + if (isRecvOpCompleted(op_ctx)) { if (!op_ctx.original_tensor_.is_contiguous()) { - op_ctx.original_tensor_.copy_(op_ctx.tensor_); + (void)op_ctx.original_tensor_.copy_(op_ctx.tensor_); if (!is_cpu_) { const cudaError_t sync_error = cudaDeviceSynchronize(); TORCH_CHECK(sync_error == cudaSuccess, @@ -885,7 +1252,8 @@ bool P2PProxy::StepRecv() { cudaGetErrorString(sync_error)); } } - op_ctx.completed_->store(true, std::memory_order_release); + op_ctx.status_->store(OpStatus::kSuccess, + std::memory_order_release); lane.active_recv_op_.reset(); active_recv_tasks_.fetch_sub(1, std::memory_order_release); did_work = true; @@ -895,46 +1263,50 @@ bool P2PProxy::StepRecv() { return did_work; } -void P2PProxy::SetDeviceWorker(P2PDeviceWorker* worker) { +void P2PProxy::attachToWorker(P2PDeviceWorker* worker, P2PChunkPool* send, + P2PChunkPool* recv, size_t chunk_size) { device_worker_ = worker; + send_pool_ = send; + recv_pool_ = recv; + chunk_size_ = chunk_size; } -bool P2PProxy::HasActiveSendWork() const { +bool P2PProxy::hasActiveSendWork() const { for (int i = 0; i < size_; ++i) { if (reset_send_req_[i].load(std::memory_order_acquire)) return true; } return active_send_tasks_.load(std::memory_order_acquire) > 0; } -bool P2PProxy::HasActiveRecvWork() const { +bool P2PProxy::hasActiveRecvWork() const { for (int i = 0; i < size_; ++i) { if (reset_recv_req_[i].load(std::memory_order_acquire)) return true; } return active_recv_tasks_.load(std::memory_order_acquire) > 0; } -bool P2PProxy::DrainTasks() const { +bool P2PProxy::drainTasks() const { BackoffWaiter waiter; return waiter.wait_for( std::chrono::milliseconds(kDrainTasksTimeoutMs), - [this] { return !HasActiveSendWork() && !HasActiveRecvWork(); }); + [this] { return !hasActiveSendWork() && !hasActiveRecvWork(); }); } -void P2PDeviceWorker::Start() { +void P2PDeviceWorker::start() { bool expected_send = false; if (send_worker_running_.compare_exchange_strong(expected_send, true)) { send_worker_thread_ = - std::thread(&P2PDeviceWorker::SendWorkerThread, this); + std::thread(&P2PDeviceWorker::sendWorkerMainloop, this); } bool expected_recv = false; if (recv_worker_running_.compare_exchange_strong(expected_recv, true)) { recv_worker_thread_ = - std::thread(&P2PDeviceWorker::RecvWorkerThread, this); + std::thread(&P2PDeviceWorker::recvWorkerMainloop, this); } } -void P2PDeviceWorker::Stop() { +void P2PDeviceWorker::stop() { bool expected_send = true; std::unique_lock s_lock(send_wakeup_mutex_); if (send_worker_running_.compare_exchange_strong(expected_send, false)) { @@ -956,8 +1328,21 @@ void P2PDeviceWorker::Stop() { } } +P2PDeviceWorker::P2PDeviceWorker(TransferEngine* engine, + const std::string& location, bool is_cpu, + int cuda_device_index) + : is_cpu_(is_cpu), cuda_device_index_(cuda_device_index) { + initPools(engine, location); + start(); +} + +P2PDeviceWorker::~P2PDeviceWorker() { + stop(); + releasePools(); +} + void P2PDeviceWorker::registerProxy(const std::shared_ptr& proxy) { - proxy->SetDeviceWorker(this); + proxy->attachToWorker(this, &send_pool_, &recv_pool_, chunk_size_); { std::lock_guard lock(proxies_mutex_); proxies_.emplace_back(proxy); @@ -981,13 +1366,96 @@ void P2PDeviceWorker::removeProxy(const std::shared_ptr& proxy) { std::lock_guard r_lock(recv_wakeup_mutex_); proxies_version_.fetch_add(1, std::memory_order_release); } - proxy->SetDeviceWorker(nullptr); send_wakeup_cv_.notify_one(); recv_wakeup_cv_.notify_one(); } +void P2PDeviceWorker::initPools(TransferEngine* engine, + const std::string& location) { + engine_ = engine; + + // Parse from environment variables + pool_bytes_ = getEnv_size_t("MOONCAKE_P2P_POOL_SIZE", kDefaultPoolSize); + chunk_size_ = getEnv_size_t("MOONCAKE_P2P_CHUNK_SIZE", kDefaultChunkSize); + + // chunk_len is uint32_t fields in control slots + TORCH_CHECK( + chunk_size_ > 0 && chunk_size_ <= std::numeric_limits::max(), + "Invalid MOONCAKE_P2P_CHUNK_SIZE: must be > 0 and <= 4GB"); + + TORCH_CHECK( + pool_bytes_ > 0 && pool_bytes_ % chunk_size_ == 0, + "Invalid pool size and chunk size (must hold 'pool_bytes_ > 0 && " + "pool_bytes_ % chunk_size_ == 0')"); + + num_chunks_ = static_cast(pool_bytes_ / chunk_size_); + TORCH_CHECK(num_chunks_ > 0, "P2PDeviceWorker: num_chunks_ must be > 0"); + + if (is_cpu_) { + send_pool_base_ = std::malloc(pool_bytes_); + TORCH_CHECK(send_pool_base_ != nullptr, + "Failed to allocate CPU P2P send pool"); + int rc = + engine->registerLocalMemory(send_pool_base_, pool_bytes_, location); + TORCH_CHECK(rc == 0, "Failed to register CPU P2P send pool"); + + recv_pool_base_ = std::malloc(pool_bytes_); + TORCH_CHECK(recv_pool_base_ != nullptr, + "Failed to allocate CPU P2P recv pool"); + rc = + engine->registerLocalMemory(recv_pool_base_, pool_bytes_, location); + TORCH_CHECK(rc == 0, "Failed to register CPU P2P recv pool"); + } else { + setCudaDeviceIfNeeded(is_cpu_, cuda_device_index_, + "P2PDeviceWorker initPools cudaSetDevice failed"); + cudaError_t err = cudaMalloc(&send_pool_base_, pool_bytes_); + TORCH_CHECK( + err == cudaSuccess, + "Failed to allocate CUDA P2P send pool: ", cudaGetErrorString(err)); + int rc = + engine->registerLocalMemory(send_pool_base_, pool_bytes_, location); + TORCH_CHECK(rc == 0, "Failed to register CUDA P2P send pool"); + + err = cudaMalloc(&recv_pool_base_, pool_bytes_); + TORCH_CHECK( + err == cudaSuccess, + "Failed to allocate CUDA P2P recv pool: ", cudaGetErrorString(err)); + rc = + engine->registerLocalMemory(recv_pool_base_, pool_bytes_, location); + TORCH_CHECK(rc == 0, "Failed to register CUDA P2P recv pool"); + } + + send_pool_.init(send_pool_base_, chunk_size_, num_chunks_); + recv_pool_.init(recv_pool_base_, chunk_size_, num_chunks_); +} + +void P2PDeviceWorker::releasePools() { + setCudaDeviceIfNeeded(is_cpu_, cuda_device_index_, + "P2PDeviceWorker releasePools cudaSetDevice failed"); + + if (send_pool_base_ != nullptr) { + if (engine_) engine_->unregisterLocalMemory(send_pool_base_); + if (is_cpu_) { + std::free(send_pool_base_); + } else { + cudaFree(send_pool_base_); + } + send_pool_base_ = nullptr; + } + + if (recv_pool_base_ != nullptr) { + if (engine_) engine_->unregisterLocalMemory(recv_pool_base_); + if (is_cpu_) { + std::free(recv_pool_base_); + } else { + cudaFree(recv_pool_base_); + } + recv_pool_base_ = nullptr; + } +} + template -void WorkerThreadLoop(bool is_cpu, int cuda_device_index, +void workerThreadLoop(bool is_cpu, int cuda_device_index, std::atomic& worker_running, std::atomic& proxies_version, std::mutex& proxies_mutex, @@ -995,7 +1463,7 @@ void WorkerThreadLoop(bool is_cpu, int cuda_device_index, std::mutex& wakeup_mutex, std::condition_variable& wakeup_cv, HasWorkFn has_work, StepWorkFn step_work) { - SetCudaDeviceIfNeeded(is_cpu, cuda_device_index, + setCudaDeviceIfNeeded(is_cpu, cuda_device_index, "P2PDeviceWorker::WorkerThread cudaSetDevice failed"); // A thread-local cache for proxies to avoid locking too frequently. @@ -1091,20 +1559,20 @@ void WorkerThreadLoop(bool is_cpu, int cuda_device_index, } } -void P2PDeviceWorker::SendWorkerThread() { - WorkerThreadLoop( +void P2PDeviceWorker::sendWorkerMainloop() { + workerThreadLoop( is_cpu_, cuda_device_index_, send_worker_running_, proxies_version_, proxies_mutex_, proxies_, send_wakeup_mutex_, send_wakeup_cv_, - [](P2PProxy& p) { return p.HasActiveSendWork(); }, - [](P2PProxy& p) { return p.StepSend(); }); + [](P2PProxy& p) { return p.hasActiveSendWork(); }, + [](P2PProxy& p) { return p.stepSend(); }); } -void P2PDeviceWorker::RecvWorkerThread() { - WorkerThreadLoop( +void P2PDeviceWorker::recvWorkerMainloop() { + workerThreadLoop( is_cpu_, cuda_device_index_, recv_worker_running_, proxies_version_, proxies_mutex_, proxies_, recv_wakeup_mutex_, recv_wakeup_cv_, - [](P2PProxy& p) { return p.HasActiveRecvWork(); }, - [](P2PProxy& p) { return p.StepRecv(); }); + [](P2PProxy& p) { return p.hasActiveRecvWork(); }, + [](P2PProxy& p) { return p.stepRecv(); }); } // Standard practice is to update states shared with condition_variables under @@ -1114,16 +1582,17 @@ void P2PDeviceWorker::RecvWorkerThread() { // acquired in WakeUpSend/Recv) // // Ref: https://stackoverflow.com/a/21439617 -void P2PDeviceWorker::WakeUpSend() { +void P2PDeviceWorker::wakeUpSend() { std::lock_guard lock(send_wakeup_mutex_); send_wakeup_cv_.notify_one(); } -void P2PDeviceWorker::WakeUpRecv() { +void P2PDeviceWorker::wakeUpRecv() { std::lock_guard lock(recv_wakeup_mutex_); recv_wakeup_cv_.notify_one(); } -std::shared_ptr P2PDeviceWorkerManager::GetCPUWorker() { +std::shared_ptr P2PDeviceWorkerManager::getCPUWorker( + TransferEngine* engine) { std::lock_guard lock(manager_mutex_); auto it = workers_.find(CPUWorkerID); @@ -1131,14 +1600,14 @@ std::shared_ptr P2PDeviceWorkerManager::GetCPUWorker() { if (auto ptr = it->second.lock()) return ptr; } - auto worker = - std::make_shared(/* is_cpu */ true, CPUWorkerID); + auto worker = std::make_shared( + engine, kWildcardLocation, /* is_cpu */ true, CPUWorkerID); workers_[CPUWorkerID] = worker; return worker; } -std::shared_ptr P2PDeviceWorkerManager::GetCUDAWorker( - int cuda_device_index) { +std::shared_ptr P2PDeviceWorkerManager::getCUDAWorker( + int cuda_device_index, TransferEngine* engine) { std::lock_guard lock(manager_mutex_); auto it = workers_.find(cuda_device_index); @@ -1146,9 +1615,11 @@ std::shared_ptr P2PDeviceWorkerManager::GetCUDAWorker( if (auto ptr = it->second.lock()) return ptr; } - auto worker = std::make_shared(/* is_cpu */ false, - cuda_device_index); + auto worker = std::make_shared( + engine, GPU_PREFIX + std::to_string(cuda_device_index), + /* is_cpu */ false, cuda_device_index); workers_[cuda_device_index] = worker; return worker; } + } // namespace mooncake -- 2.34.1 From f7dd92acca3a7466f562bc261329282b87684d2c Mon Sep 17 00:00:00 2001 From: Zhanhao Cao Date: Thu, 7 May 2026 12:32:21 +0800 Subject: [PATCH 047/382] [EP] Add a comprehensive test for MooncakeEP (#1695) --- mooncake-ep/tests/test_ep_grid.py | 313 ++++++++++++++++++++++++++++++ 1 file changed, 313 insertions(+) create mode 100644 mooncake-ep/tests/test_ep_grid.py diff --git a/mooncake-ep/tests/test_ep_grid.py b/mooncake-ep/tests/test_ep_grid.py new file mode 100644 index 00000000..90fc5a58 --- /dev/null +++ b/mooncake-ep/tests/test_ep_grid.py @@ -0,0 +1,313 @@ +import os +import itertools +import unittest + +import torch +import torch.distributed as dist +import torch.multiprocessing as mp +import torch.testing as testing +import faulthandler +import traceback + +from mooncake.mooncake_ep_buffer import Buffer +import mooncake.pg as pg + + +def dequantize_fp8(x_fp8: torch.Tensor, scales: torch.Tensor) -> torch.Tensor: + hidden = x_fp8.shape[-1] + x_view = x_fp8.reshape(-1, hidden // 128, 128).float() + scales_view = scales.reshape(-1, hidden // 128, 1).float() + dequantized = (x_view * scales_view).reshape(x_fp8.shape) + return dequantized.to(torch.bfloat16) + + +def run_test_iteration( + group: dist.ProcessGroup, + cpu_group: dist.ProcessGroup, + rank: int, + num_ranks: int, + max_tokens: int, + hidden: int, + num_experts: int, + top_k: int, + use_fp8: bool, + zero_copy: bool, + async_finish: bool, + return_recv_hook: bool, + use_fallback: bool, + fail_rank: int, +): + assert not ( + async_finish and return_recv_hook + ), "Should be filtered out by generate_tests." + + torch.manual_seed(2026 + rank) + scale = 1.0 - 0.05 * (rank / num_ranks) + num_tokens = int(max_tokens * scale) + + # Prepare test data + x = torch.randn(num_tokens, hidden, dtype=torch.bfloat16) + scores = torch.randn((num_tokens, num_experts), dtype=torch.float32) + topk_idx = torch.topk(scores, top_k, dim=-1)[1] + topk_weights = torch.softmax( + torch.rand(num_tokens, top_k, dtype=torch.float32), dim=-1 + ) + active_ranks = torch.ones((num_ranks,), dtype=torch.int32) + + # Prepare expected result + def get_mock_factor(expert_id): + return expert_id * 0.1 + 1.0 + + # Since `get_mock_factor` is simply *0.1 + 1, we can make it more efficient + # instead of the nested loops below: + # + # for i in range(num_tokens): + # for j in range(top_k): + # expert_id = topk_idx[i, j].item() + # w = topk_weights[i, j].item() + # mock_expert_factor = get_mock_factor(expert_id) + # if fail_rank != -1 and (expert_id // num_local_experts == fail_rank): + # continue + # expected_out[i] += (x[i] * mock_expert_factor) * w + num_local_experts = num_experts // num_ranks + factors = get_mock_factor(topk_idx) + if fail_rank != -1: + expert_owners = topk_idx // num_local_experts + valid_mask = expert_owners != fail_rank + factors = factors * valid_mask.to(factors.dtype) + sum_weights = (factors * topk_weights).sum(dim=1, keepdim=True) + expected_out = x * sum_weights + expected_out = expected_out.to(torch.bfloat16) + + # Initialize the buffer. + num_ep_buffer_bytes = Buffer.get_ep_buffer_size_hint( + max_tokens, hidden, num_ranks, num_experts + ) + buf = Buffer(group, num_ep_buffer_bytes) + + if use_fallback: + buf._use_fallback = True + + # 5s timeout if we simulate a failed rank + timeout_us = 5 * 1_000_000 if fail_rank != -1 else -1 + + cpu_group.barrier() + + if rank == fail_rank: + os._exit(0) + + # Dispatch + recv_x, recv_count, handle, event, hook = buf.dispatch( + x, + topk_idx, + active_ranks, + num_max_dispatch_tokens_per_rank=max_tokens, + num_experts=num_experts, + timeout_us=timeout_us, + use_fp8=use_fp8, + async_finish=async_finish, + return_recv_hook=return_recv_hook, + ) + + if return_recv_hook: + hook() + if async_finish: + event.current_stream_wait() + + torch.cuda.synchronize() + # Fault-tolerance check + if fail_rank != -1: + assert active_ranks[fail_rank].item() == 0, ( + f"[Rank {rank}] Failed rank {fail_rank} is not recorded in active_ranks. " + f"active_ranks: {active_ranks}" + ) + assert active_ranks.sum().item() == active_ranks.numel() - 1, ( + f"[Rank {rank}] Expected exactly one failed rank {fail_rank}, but found " + f"{active_ranks.numel() - active_ranks.sum().item()} inactive ranks. " + f"Maybe the timeout is too small? " + f"active_ranks: {active_ranks}" + ) + + # Mock expert forward + if use_fp8: + recv_bf16 = dequantize_fp8(recv_x[0], recv_x[1]) + else: + recv_bf16 = recv_x + + expert_out = torch.empty_like(recv_bf16) + + for le in range(num_local_experts): + expert_id = rank * num_local_experts + le + mock_expert_factor = get_mock_factor(expert_id) + expert_out[le] = recv_bf16[le] * mock_expert_factor + + expert_out = expert_out.to(torch.bfloat16) + + # Combine + if zero_copy: + cb_buf = buf.get_next_combine_buffer(handle) + cb_buf.copy_(expert_out) + expert_to_pass = cb_buf.contiguous() + else: + expert_to_pass = expert_out.contiguous() + + out_tensor = torch.zeros_like(x) + combined_x, event, hook = buf.combine( + expert_to_pass, + topk_idx, + topk_weights, + active_ranks, + timeout_us=timeout_us, + handle=handle, + zero_copy=zero_copy, + async_finish=async_finish, + return_recv_hook=return_recv_hook, + out=out_tensor, + ) + + if return_recv_hook: + hook() + if async_finish: + event.current_stream_wait() + + torch.cuda.synchronize() + + testing.assert_close( + combined_x, + expected_out, + rtol=0.15 if use_fp8 else 5e-2, + atol=5e-3 if use_fp8 else 1e-3, + msg=lambda msg: f"[Rank {rank}] Combine Mismatch. {msg}", + ) + + torch.cuda.synchronize() + dist.barrier(cpu_group) + + +def worker(rank, world_size, config_dict): + # Device filter + device_filter = [ + f + for f in os.getenv("DEVICE_FILTER", "mlx5_1,mlx5_2,mlx5_3,mlx5_4").split(",") + if f + ] + if device_filter: + pg.set_device_filter(device_filter) + + torch.cuda.set_device(rank) + torch.set_default_dtype(torch.bfloat16) + torch.set_default_device("cuda") + + dist.init_process_group(backend="mooncake", rank=rank, world_size=world_size) + group = dist.group.WORLD + cpu_group = dist.new_group(list(range(world_size)), backend="mooncake-cpu") + + try: + run_test_iteration( + group=group, + cpu_group=cpu_group, + rank=rank, + num_ranks=world_size, + **config_dict, + ) + except Exception as e: + traceback.print_exc() + raise + + dist.destroy_process_group() + + +class TestMooncakeEPBuffer(unittest.TestCase): + def setUp(self): + self.world_size = torch.cuda.device_count() + os.environ["MASTER_ADDR"] = "127.0.0.1" + os.environ["MASTER_PORT"] = "29500" + + def run_single_config(self, config_dict): + mp.spawn( + worker, + args=(self.world_size, config_dict), + nprocs=self.world_size, + join=True, + daemon=False, + ) + + +def make_test_name(cfg): + parts = ["test_ep"] + + flags = [] + if cfg["use_fp8"]: + flags.append("fp8") + if cfg["zero_copy"]: + flags.append("0copy") + if cfg["async_finish"]: + flags.append("async") + if cfg["return_recv_hook"]: + flags.append("hook") + if cfg["use_fallback"]: + flags.append("fallback") + + if flags: + parts.append("_".join(flags)) + else: + parts.append("base") + + fail_rank = cfg["fail_rank"] + if fail_rank == -1: + parts.append("nofail") + else: + parts.append(f"fail{fail_rank}") + + parts.append(f"t{cfg['max_tokens']}") + parts.append(f"h{cfg['hidden']}") + parts.append(f"e{cfg['num_experts']}") + parts.append(f"k{cfg['top_k']}") + + return "_".join(parts) + + +def generate_tests(): + test_grid = { + "use_fp8": [False, True], + "zero_copy": [False, True], + "async_finish": [False, True], + "return_recv_hook": [False, True], + "use_fallback": [False, True], + "fail_rank": [-1, 1], + "shapes": [ + {"max_tokens": 256, "hidden": 2048, "num_experts": 288, "top_k": 8}, + ], + } + + keys = list(test_grid.keys()) + config_tuples = list(itertools.product(*[test_grid[k] for k in keys])) + + for t in config_tuples: + raw_dict = dict(zip(keys, t)) + + if raw_dict["async_finish"] and raw_dict["return_recv_hook"]: + continue + + # Flatten + config_dict = {} + for k, v in raw_dict.items(): + if k == "shapes": + config_dict.update(v) + else: + config_dict[k] = v + + # Make a test name + test_name = make_test_name(config_dict) + + def test_func(self, cfg=config_dict): + self.run_single_config(cfg) + + test_func.__name__ = test_name + setattr(TestMooncakeEPBuffer, test_name, test_func) + + +if __name__ == "__main__": + faulthandler.enable() + generate_tests() + unittest.main(verbosity=2) -- 2.34.1 From 5b1436196e64f7d9d81da6c39c6e131b1d308a8d Mon Sep 17 00:00:00 2001 From: Jason Date: Thu, 7 May 2026 16:00:54 +0800 Subject: [PATCH 048/382] fix(transfer-engine): add missing empty checks for batch methods (#2046) --- mooncake-transfer-engine/rust/src/transfer_engine.rs | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mooncake-transfer-engine/rust/src/transfer_engine.rs b/mooncake-transfer-engine/rust/src/transfer_engine.rs index 241c18a0..8c6186df 100644 --- a/mooncake-transfer-engine/rust/src/transfer_engine.rs +++ b/mooncake-transfer-engine/rust/src/transfer_engine.rs @@ -150,6 +150,9 @@ impl TransferEngine { buffer_list: &[BufferEntry], location: &str, ) -> Result<()> { + if buffer_list.is_empty() { + return Ok(()); + } let location_c = CString::new(location).map_err(|_| anyhow!("CString::new failed"))?; let mut buffer_list_c: Vec = vec![]; let buffer_len_c = buffer_list.len(); @@ -175,6 +178,9 @@ impl TransferEngine { } pub fn unregister_local_memory_batch(&self, buffer_list: &[BufferEntry]) -> Result<()> { + if buffer_list.is_empty() { + return Ok(()); + } let mut addr_list: Vec<*mut c_void> = buffer_list.iter().map(|entry| entry.addr).collect(); let addr_len = buffer_list.len(); let ret = unsafe { @@ -201,6 +207,9 @@ impl TransferEngine { batch_id: BatchID, requests: &mut [TransferRequest], ) -> Result<()> { + if requests.is_empty() { + return Ok(()); + } let mut requests_c: Vec = vec![]; for i in 0..requests.len() { requests_c.push(bindings::transfer_request_t { -- 2.34.1 From 23e114661b7670ee463bb033bd96f27567713eea Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 7 May 2026 18:15:49 +0800 Subject: [PATCH 049/382] docs: add vLLM Mooncake Store blog post to README updates (#2052) --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 7adae202..bc4648dd 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ This repository also hosts its technical report and the open-sourced traces.

🔄 Updates

+- **May 7, 2026**: 🚀 vLLM officially features [Mooncake Store](https://vllm.ai/blog/mooncake-store) — a deep dive into how Mooncake's distributed KVCache engine supercharges vLLM inference with high-throughput, memory-efficient, cross-instance KV cache sharing! - **Apr 29, 2026**: SGLang introduces [RDMA-based P2P weight transfer for large-scale distributed RL](https://lmsys.org/blog/2026-04-29-p2p-update/) using Mooncake TransferEngine, achieving 7x faster weight updates for the 1T-parameter Kimi-K2 model (53s → 7.2s) with zero-copy RDMA transfer across thousands of GPUs. - **Mar 19, 2026**: [TorchSpec: Speculative Decoding Training at Scale](https://pytorch.org/blog/torchspec-speculative-decoding-training-at-scale) is [open sourced](https://github.com/torchspec-project/TorchSpec), using Mooncake to decouple inference and training via efficient hidden states management. - **Mar 5, 2026**: [LightX2V](https://github.com/ModelTC/LightX2V/pull/893) now supports disaggregated deployment based on Mooncake, enabling encoder/transformer service decoupling with Mooncake Transfer Engine for high-performance cross-device and cross-machine data transfer. -- 2.34.1 From 43ce0f8ff5305e48e8fc4b44372dd271f5140c2a Mon Sep 17 00:00:00 2001 From: Zhang Jason Date: Thu, 7 May 2026 18:22:15 +0800 Subject: [PATCH 050/382] [Tent][AMD] Add AMD CDNA4 (ROCm/HIP) platform support with benchmark tooling (#2021) --------- Co-authored-by: root --- .../benchmark/CMakeLists.txt | 12 + .../benchmark/tent_backend.cpp | 40 +- mooncake-transfer-engine/benchmark/utils.h | 48 ++- .../tent/include/tent/common/status.h | 10 + .../tent/include/tent/platform/rocm.h | 131 +++++++ .../tent/include/tent/runtime/platform.h | 2 +- .../tent/plugins/CMakeLists.txt | 4 + .../tent/plugins/rocm/CMakeLists.txt | 22 ++ .../tent/plugins/rocm/rocm_plugin.cpp | 146 ++++++++ .../tent/src/CMakeLists.txt | 22 +- .../tent/src/platform/CMakeLists.txt | 4 + .../tent/src/platform/rocm/CMakeLists.txt | 6 + .../tent/src/platform/rocm/rocm_allocator.cpp | 67 ++++ .../tent/src/platform/rocm/rocm_probe.cpp | 344 ++++++++++++++++++ .../src/platform/rocm/rocm_stream_pool.cpp | 150 ++++++++ .../tent/src/runtime/platform.cpp | 4 + .../tent/src/runtime/transfer_engine_impl.cpp | 13 +- .../tent/tests/CMakeLists.txt | 11 + .../tent/tests/hip_bandwidth_bench.cpp | 228 ++++++++++++ .../tent/tests/rocm_platform_test.cpp | 249 +++++++++++++ 20 files changed, 1495 insertions(+), 18 deletions(-) create mode 100644 mooncake-transfer-engine/tent/include/tent/platform/rocm.h create mode 100644 mooncake-transfer-engine/tent/plugins/rocm/CMakeLists.txt create mode 100644 mooncake-transfer-engine/tent/plugins/rocm/rocm_plugin.cpp create mode 100644 mooncake-transfer-engine/tent/src/platform/rocm/CMakeLists.txt create mode 100644 mooncake-transfer-engine/tent/src/platform/rocm/rocm_allocator.cpp create mode 100644 mooncake-transfer-engine/tent/src/platform/rocm/rocm_probe.cpp create mode 100644 mooncake-transfer-engine/tent/src/platform/rocm/rocm_stream_pool.cpp create mode 100644 mooncake-transfer-engine/tent/tests/hip_bandwidth_bench.cpp create mode 100644 mooncake-transfer-engine/tent/tests/rocm_platform_test.cpp diff --git a/mooncake-transfer-engine/benchmark/CMakeLists.txt b/mooncake-transfer-engine/benchmark/CMakeLists.txt index 4a05f3d7..dd222719 100644 --- a/mooncake-transfer-engine/benchmark/CMakeLists.txt +++ b/mooncake-transfer-engine/benchmark/CMakeLists.txt @@ -7,12 +7,24 @@ else() message(STATUS "CUDA: Disabled") endif() +if(USE_HIP) + list(APPEND CMAKE_PREFIX_PATH "/opt/rocm/lib/cmake") + find_package(HIP REQUIRED) + add_definitions(-DUSE_HIP -D__HIP_PLATFORM_AMD__) + message(STATUS "ROCm/HIP: Enabled") +else() + message(STATUS "ROCm/HIP: Disabled") +endif() + file(GLOB TEBENCH_SOURCES "*.cpp") add_executable(tebench ${TEBENCH_SOURCES}) target_link_libraries(tebench PUBLIC transfer_engine tent_link_group) if(USE_CUDA) target_link_libraries(tebench PUBLIC CUDA::cudart) endif() +if(USE_HIP) + target_link_libraries(tebench PUBLIC hip::host) +endif() # Set RPATH for finding libasio.so at runtime set_target_properties( diff --git a/mooncake-transfer-engine/benchmark/tent_backend.cpp b/mooncake-transfer-engine/benchmark/tent_backend.cpp index 435d553d..57b4ed90 100644 --- a/mooncake-transfer-engine/benchmark/tent_backend.cpp +++ b/mooncake-transfer-engine/benchmark/tent_backend.cpp @@ -22,6 +22,10 @@ #include #endif +#ifdef USE_HIP +#include +#endif + namespace mooncake { namespace tent { @@ -105,6 +109,20 @@ int TENTBenchRunner::allocateBuffers() { << "local_gpu_id " << start_idx << " out of range [0, " << gpu_count << ")"; } +#elif defined(USE_HIP) + } else if (seg_type == "VRAM") { + device_prefix = "rocm"; + int gpu_count = 0; + hipGetDeviceCount(&gpu_count); + start_idx = 0; + num_buffers = gpu_count; + if (XferBenchConfig::local_gpu_id != -1) { + start_idx = XferBenchConfig::local_gpu_id; + num_buffers = 1; + LOG_ASSERT(start_idx >= 0 && start_idx < gpu_count) + << "local_gpu_id " << start_idx << " out of range [0, " + << gpu_count << ")"; + } #endif } else { LOG(ERROR) << "Unknown seg_type: " << seg_type; @@ -200,7 +218,6 @@ int TENTBenchRunner::stopInitiator() { return 0; } -#ifdef USE_CUDA static inline int getNumaNodeFromPciDevice(const std::string& pci_bdf) { std::string sysfs_path = "/sys/bus/pci/devices/" + pci_bdf + "/numa_node"; std::ifstream numa_file(sysfs_path); @@ -210,9 +227,11 @@ static inline int getNumaNodeFromPciDevice(const std::string& pci_bdf) { if (numa_file.fail()) return -1; return numa_node; } -static inline int getCudaDeviceNumaID(int cuda_id) { + +#ifdef USE_CUDA +static inline int getGpuDeviceNumaID(int gpu_id) { char pci_bus_id[20]; - auto err = cudaDeviceGetPCIBusId(pci_bus_id, sizeof(pci_bus_id), cuda_id); + auto err = cudaDeviceGetPCIBusId(pci_bus_id, sizeof(pci_bus_id), gpu_id); if (err != cudaSuccess) { LOG(WARNING) << "cudaDeviceGetPCIBusId: " << cudaGetErrorString(err); return 0; @@ -220,8 +239,17 @@ static inline int getCudaDeviceNumaID(int cuda_id) { for (char* ch = pci_bus_id; (*ch = tolower(*ch)); ch++); return getNumaNodeFromPciDevice(pci_bus_id); } +#elif defined(USE_HIP) +static inline int getGpuDeviceNumaID(int gpu_id) { + hipDeviceProp_t prop; + if (hipGetDeviceProperties(&prop, gpu_id) != hipSuccess) return 0; + char pci_bus_id[20]; + snprintf(pci_bus_id, sizeof(pci_bus_id), "%04x:%02x:%02x.0", + prop.pciDomainID, prop.pciBusID, prop.pciDeviceID); + return getNumaNodeFromPciDevice(pci_bus_id); +} #else -static inline int getCudaDeviceNumaID(int cuda_id) { return 0; } +static inline int getGpuDeviceNumaID(int gpu_id) { return 0; } #endif void TENTBenchRunner::pinThread(int thread_id) { @@ -232,9 +260,9 @@ void TENTBenchRunner::pinThread(int thread_id) { if (location.type() == "cpu") { auto socket_id = location.index(); bindToSocket(socket_id); - } else if (location.type() == "cuda") { + } else if (location.type() == "cuda" || location.type() == "rocm") { auto device_id = location.index(); - auto socket_id = getCudaDeviceNumaID(device_id); + auto socket_id = getGpuDeviceNumaID(device_id); bindToSocket(socket_id); } } diff --git a/mooncake-transfer-engine/benchmark/utils.h b/mooncake-transfer-engine/benchmark/utils.h index ae48d43b..181af52c 100644 --- a/mooncake-transfer-engine/benchmark/utils.h +++ b/mooncake-transfer-engine/benchmark/utils.h @@ -34,6 +34,10 @@ #include #endif +#ifdef USE_HIP +#include +#endif + #define CHECK_FAIL(call) \ do { \ auto status_ = call; \ @@ -154,33 +158,65 @@ static inline bool isCudaMemory(void* ptr) { } #endif +#ifdef USE_HIP +static inline bool isHipMemory(void* ptr) { + hipPointerAttribute_t attr; + auto ret = hipPointerGetAttributes(&attr, ptr); + return ret == hipSuccess && attr.type == hipMemoryTypeDevice; +} +#endif + +static inline bool isGpuMemory(void* ptr) { +#ifdef USE_CUDA + if (isCudaMemory(ptr)) return true; +#endif +#ifdef USE_HIP + if (isHipMemory(ptr)) return true; +#endif + return false; +} + static inline uint8_t fillData(void* addr, size_t length) { uint8_t seed = (uint8_t)SimpleRandom::Get().next(256); #ifdef USE_CUDA if (isCudaMemory(addr)) { - std::vector ref_data; - ref_data.resize(length, seed); + std::vector ref_data(length, seed); cudaMemcpy(addr, ref_data.data(), length, cudaMemcpyDefault); return seed; } +#endif +#ifdef USE_HIP + if (isHipMemory(addr)) { + std::vector ref_data(length, seed); + hipMemcpy(addr, ref_data.data(), length, hipMemcpyDefault); + return seed; + } #endif memset(addr, seed, length); return seed; } static inline void verifyData(void* addr, size_t length, uint8_t seed) { - std::vector ref_data; - ref_data.resize(length, seed); + std::vector ref_data(length, seed); #ifdef USE_CUDA if (isCudaMemory(addr)) { - std::vector act_data; - act_data.resize(length); + std::vector act_data(length); cudaMemcpy(act_data.data(), addr, length, cudaMemcpyDefault); if (memcmp(act_data.data(), ref_data.data(), length)) { LOG(FATAL) << "Inconsistent data detected"; } return; } +#endif +#ifdef USE_HIP + if (isHipMemory(addr)) { + std::vector act_data(length); + hipMemcpy(act_data.data(), addr, length, hipMemcpyDefault); + if (memcmp(act_data.data(), ref_data.data(), length)) { + LOG(FATAL) << "Inconsistent data detected"; + } + return; + } #endif if (memcmp(addr, ref_data.data(), length)) { LOG(FATAL) << "Inconsistent data detected"; diff --git a/mooncake-transfer-engine/tent/include/tent/common/status.h b/mooncake-transfer-engine/tent/include/tent/common/status.h index f90b3e84..67049821 100644 --- a/mooncake-transfer-engine/tent/include/tent/common/status.h +++ b/mooncake-transfer-engine/tent/include/tent/common/status.h @@ -60,6 +60,16 @@ } while (0) #endif +#ifdef USE_HIP +#define CHECK_HIP(call) \ + do { \ + auto err = call; \ + if (err != hipSuccess) \ + return Status::InternalError(std::string(#call) + ": " + \ + hipGetErrorString(err) + LOC_MARK); \ + } while (0) +#endif + #if defined(USE_ASCEND) || defined(USE_ASCEND_DIRECT) #define CHECK_ASCEND(call) \ do { \ diff --git a/mooncake-transfer-engine/tent/include/tent/platform/rocm.h b/mooncake-transfer-engine/tent/include/tent/platform/rocm.h new file mode 100644 index 00000000..5ef0a356 --- /dev/null +++ b/mooncake-transfer-engine/tent/include/tent/platform/rocm.h @@ -0,0 +1,131 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#ifndef ROCM_H +#define ROCM_H + +#include "tent/runtime/platform.h" +#include "tent/common/config.h" +#include "tent/common/concurrent/rw_spinlock.h" + +#include + +namespace mooncake { +namespace tent { + +class HIPStreamPool; + +// RAII wrapper for hipStream_t. +// When this handle destructs, the stream is automatically returned to the pool. +class HIPStreamHandle { + public: + HIPStreamHandle() = default; + + HIPStreamHandle(hipStream_t stream, int deviceId, HIPStreamPool* pool) + : stream_(stream), deviceId_(deviceId), pool_(pool) {} + + HIPStreamHandle(const HIPStreamHandle&) = delete; + HIPStreamHandle& operator=(const HIPStreamHandle&) = delete; + + HIPStreamHandle(HIPStreamHandle&& other) noexcept + : stream_(other.stream_), + deviceId_(other.deviceId_), + pool_(other.pool_) { + other.stream_ = nullptr; + other.pool_ = nullptr; + } + + HIPStreamHandle& operator=(HIPStreamHandle&& other) noexcept; + ~HIPStreamHandle(); + + [[nodiscard]] hipStream_t get() const { return stream_; } + + private: + void releaseToPool(); + + hipStream_t stream_ = nullptr; + int deviceId_ = -1; + HIPStreamPool* pool_ = nullptr; +}; + +// HIP Stream Pool managing all devices. +class HIPStreamPool { + friend class HIPStreamHandle; + + public: + HIPStreamPool() = default; + ~HIPStreamPool() = default; + + HIPStreamPool(const HIPStreamPool&) = delete; + HIPStreamPool& operator=(const HIPStreamPool&) = delete; + + static constexpr int kCurrentDevice = -1; + Status acquire(HIPStreamHandle& outHandle, int deviceId = kCurrentDevice); + + private: + class DevicePool { + public: + explicit DevicePool(int deviceId); + ~DevicePool(); + + Status acquire(hipStream_t& outStream); + void release(hipStream_t stream); + + private: + int deviceId_; + RWSpinlock dev_lock_; + std::vector availableStreams_; + }; + + void release(int deviceId, hipStream_t stream); + DevicePool* getDevicePool(int deviceId); + + RWSpinlock pools_lock_; + std::vector> devicePools_; +}; + +class RocmPlatform : public Platform { + public: + RocmPlatform(std::shared_ptr config) : conf(std::move(config)) {} + + virtual ~RocmPlatform() {} + + virtual Status probe(std::vector& nic_list, + std::vector& mem_list); + + virtual Status allocate(void** pptr, size_t size, MemoryOptions& options); + + virtual Status free(void* ptr, size_t size); + + virtual Status copy(void* dst, void* src, size_t length); + + virtual MemoryType getMemoryType(void* addr); + + virtual const std::vector getLocation( + void* start, size_t len, bool skip_prefault = false); + + virtual const std::string type() const { return "rocm"; } + + Status getStreamFromPool(HIPStreamHandle& outHandle, + int deviceId = HIPStreamPool::kCurrentDevice); + + private: + std::shared_ptr conf; + HIPStreamPool stream_pool_; +}; + +} // namespace tent +} // namespace mooncake + +#endif // ROCM_H diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/platform.h b/mooncake-transfer-engine/tent/include/tent/runtime/platform.h index 6e947463..95a4c7ae 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/platform.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/platform.h @@ -20,7 +20,7 @@ namespace mooncake { namespace tent { -enum MemoryType { MTYPE_UNKNOWN, MTYPE_CPU, MTYPE_CUDA }; +enum MemoryType { MTYPE_UNKNOWN, MTYPE_CPU, MTYPE_CUDA, MTYPE_ROCM }; class Platform { public: diff --git a/mooncake-transfer-engine/tent/plugins/CMakeLists.txt b/mooncake-transfer-engine/tent/plugins/CMakeLists.txt index 7b564750..e88e35a9 100644 --- a/mooncake-transfer-engine/tent/plugins/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/plugins/CMakeLists.txt @@ -11,3 +11,7 @@ target_compile_features(tent_plugin INTERFACE cxx_std_20) if (USE_CUDA) add_subdirectory(cuda) endif() + +if (USE_HIP) + add_subdirectory(rocm) +endif() diff --git a/mooncake-transfer-engine/tent/plugins/rocm/CMakeLists.txt b/mooncake-transfer-engine/tent/plugins/rocm/CMakeLists.txt new file mode 100644 index 00000000..1aec2786 --- /dev/null +++ b/mooncake-transfer-engine/tent/plugins/rocm/CMakeLists.txt @@ -0,0 +1,22 @@ +set(CMAKE_POSITION_INDEPENDENT_CODE ON) + +find_package(HIP REQUIRED) + +set(TENT_ROCM_PLUGIN_SOURCES + ${CMAKE_CURRENT_SOURCE_DIR}/rocm_plugin.cpp +) + +add_library(tent_rocm_plugin SHARED ${TENT_ROCM_PLUGIN_SOURCES}) + +target_link_libraries(tent_rocm_plugin + PRIVATE hip::host tent_plugin +) + +set_target_properties(tent_rocm_plugin PROPERTIES + OUTPUT_NAME "tent_rocm" +) + +install(TARGETS tent_rocm_plugin + LIBRARY DESTINATION lib/tent_plugins + RUNTIME DESTINATION lib/tent_plugins +) diff --git a/mooncake-transfer-engine/tent/plugins/rocm/rocm_plugin.cpp b/mooncake-transfer-engine/tent/plugins/rocm/rocm_plugin.cpp new file mode 100644 index 00000000..63d26490 --- /dev/null +++ b/mooncake-transfer-engine/tent/plugins/rocm/rocm_plugin.cpp @@ -0,0 +1,146 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/device_plugin.h" + +#include +#include +#include +#include +#include + +struct rocm_plugin_ctx_t { + // reserved +}; + +#define CHECK_HIP(call) \ + do { \ + auto err = call; \ + if (err != hipSuccess) { \ + LOG(ERROR) << std::string(#call) + ": " + hipGetErrorString(err); \ + return -1; \ + } \ + } while (0) + +class LocationParser { + public: + LocationParser(const std::string& location) { + size_t colonPos = location.find(':'); + if (colonPos == std::string::npos) { + index_ = -1; + return; + } + std::string type = location.substr(0, colonPos); + std::string indexStr = location.substr(colonPos + 1); + try { + type_ = type; + index_ = std::stoi(indexStr); + } catch (const std::exception& e) { + index_ = -1; + } + } + + std::string type() const { return type_; } + int index() const { return index_; } + + private: + std::string type_; + int index_; +}; + +static void* rocm_create_plugin() { + rocm_plugin_ctx_t* ctx = new rocm_plugin_ctx_t; + return ctx; +} + +static int rocm_destroy_plugin(void* handle) { + if (!handle) return 0; + delete reinterpret_cast(handle); + return 0; +} + +static int rocm_alloc(void* ctx_, void** pptr, size_t size, const char* loc) { + (void)ctx_; + LocationParser location(loc); + if (location.type() != "rocm") return -1; + int hip_dev = 0; + CHECK_HIP(hipGetDevice(&hip_dev)); + CHECK_HIP(hipSetDevice(location.index())); + CHECK_HIP(hipMalloc(pptr, size)); + CHECK_HIP(hipSetDevice(hip_dev)); + return 0; +} + +static int rocm_free(void* ctx_, void* ptr, size_t size) { + (void)ctx_; + (void)size; + hipPointerAttribute_t attrs; + CHECK_HIP(hipPointerGetAttributes(&attrs, ptr)); + if (attrs.type == hipMemoryTypeDevice) { + CHECK_HIP(hipFree(ptr)); + return 0; + } + return -2; +} + +static int rocm_memcpy_sync(void* ctx_, void* dst, void* src, size_t length) { + (void)ctx_; + CHECK_HIP(hipMemcpy(dst, src, length, hipMemcpyDefault)); + return 0; +} + +static int rocm_query_location(void* ctx_, void* addr, size_t size, + location_t* buf, size_t buf_count) { + (void)ctx_; + if (buf_count == 0) return -1; + hipPointerAttribute_t attr{}; + hipError_t err = hipPointerGetAttributes(&attr, addr); + if (err != hipSuccess || attr.type != hipMemoryTypeDevice) return 0; + buf[0].start = addr; + buf[0].length = size; + snprintf(buf[0].location, LOCATION_LEN, "rocm:%d", attr.device); + return 1; +} + +static int rocm_get_device_count(void* ctx_) { + (void)ctx_; + int count = 0; + CHECK_HIP(hipGetDeviceCount(&count)); + return count; +} + +static int rocm_get_device_pci_bus_id(void* ctx_, int device_index, + char* bus_id, size_t bus_id_len) { + (void)ctx_; + hipDeviceProp_t prop; + CHECK_HIP(hipGetDeviceProperties(&prop, device_index)); + snprintf(bus_id, bus_id_len, "%04x:%02x:%02x.0", prop.pciDomainID, + prop.pciBusID, prop.pciDeviceID); + return 0; +} + +extern "C" int tent_register_device_plugin(device_plugin_t* out) { + if (!out) return -1; + memset(out, 0, sizeof(*out)); + out->class_name = "rocm"; + out->create_plugin = rocm_create_plugin; + out->destroy_plugin = rocm_destroy_plugin; + out->alloc = rocm_alloc; + out->free = rocm_free; + out->memcpy_sync = rocm_memcpy_sync; + out->query_location = rocm_query_location; + out->get_device_count = rocm_get_device_count; + out->get_device_pci_bus_id = rocm_get_device_pci_bus_id; + return 0; +} diff --git a/mooncake-transfer-engine/tent/src/CMakeLists.txt b/mooncake-transfer-engine/tent/src/CMakeLists.txt index 9997ecab..99557bc3 100644 --- a/mooncake-transfer-engine/tent/src/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/src/CMakeLists.txt @@ -23,6 +23,18 @@ else() message(STATUS "CUDA: Disabled") endif() +# ROCm / HIP +if(USE_HIP) + list(APPEND CMAKE_PREFIX_PATH "/opt/rocm/lib/cmake") + find_package(HIP REQUIRED) + message(STATUS "ROCm/HIP: Enabled") + target_compile_definitions(tent_interface INTERFACE USE_HIP __HIP_PLATFORM_AMD__) + target_include_directories(tent_interface INTERFACE ${HIP_INCLUDE_DIRS}) + target_link_libraries(tent_interface INTERFACE hip::host) +else() + message(STATUS "ROCm/HIP: Disabled") +endif() + # GDS find_library(CUFILE_LIB cufile PATHS /usr/local/cuda/lib64) find_path(CUFILE_INCLUDE cufile.h PATHS /usr/local/cuda/include) @@ -89,7 +101,14 @@ add_subdirectory(runtime) add_subdirectory(platform) add_subdirectory(transport) add_subdirectory(metrics) -add_subdirectory(python) +if(NOT TARGET pybind11::module) + find_package(pybind11 QUIET) +endif() +if(TARGET pybind11::module) + add_subdirectory(python) +else() + message(STATUS "pybind11 not found, skipping Tent Python bindings") +endif() file(GLOB TENT_ENGINE_SOURCES "${CMAKE_CURRENT_SOURCE_DIR}/*.cpp") @@ -108,6 +127,7 @@ foreach( metastore_etcd tent_platform_all platform_cuda + platform_rocm platform_ascend platform_sunrise tent_xport_gds diff --git a/mooncake-transfer-engine/tent/src/platform/CMakeLists.txt b/mooncake-transfer-engine/tent/src/platform/CMakeLists.txt index dfc5792e..2f48b686 100644 --- a/mooncake-transfer-engine/tent/src/platform/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/src/platform/CMakeLists.txt @@ -1,4 +1,5 @@ add_subdirectory(cuda) +add_subdirectory(rocm) add_subdirectory(ascend) add_subdirectory(sunrise) @@ -8,6 +9,9 @@ target_link_libraries(tent_platform_all PUBLIC tent_common) if(TARGET platform_cuda) target_link_libraries(tent_platform_all PUBLIC platform_cuda) endif() +if(TARGET platform_rocm) + target_link_libraries(tent_platform_all PUBLIC platform_rocm) +endif() if(TARGET platform_ascend) target_link_libraries(tent_platform_all PUBLIC platform_ascend) endif() diff --git a/mooncake-transfer-engine/tent/src/platform/rocm/CMakeLists.txt b/mooncake-transfer-engine/tent/src/platform/rocm/CMakeLists.txt new file mode 100644 index 00000000..bdb2747c --- /dev/null +++ b/mooncake-transfer-engine/tent/src/platform/rocm/CMakeLists.txt @@ -0,0 +1,6 @@ +if (USE_HIP) + find_package(HIP REQUIRED) + file(GLOB TENT_PLATFORM_ROCM_SOURCES "*.cpp") + add_library(platform_rocm STATIC ${TENT_PLATFORM_ROCM_SOURCES}) + target_link_libraries(platform_rocm PUBLIC tent_common hip::host) +endif() diff --git a/mooncake-transfer-engine/tent/src/platform/rocm/rocm_allocator.cpp b/mooncake-transfer-engine/tent/src/platform/rocm/rocm_allocator.cpp new file mode 100644 index 00000000..11aa5bc2 --- /dev/null +++ b/mooncake-transfer-engine/tent/src/platform/rocm/rocm_allocator.cpp @@ -0,0 +1,67 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/platform/rocm.h" +#include "tent/common/status.h" + +#include +#include +#include + +namespace mooncake { +namespace tent { + +Status RocmPlatform::allocate(void** pptr, size_t size, + MemoryOptions& options) { + LocationParser location(options.location); + if (location.type() == "rocm") { + int hip_dev = 0; + CHECK_HIP(hipGetDevice(&hip_dev)); + CHECK_HIP(hipSetDevice(location.index())); + CHECK_HIP(hipMalloc(pptr, size)); + hipSetDevice(hip_dev); + return Status::OK(); + } + int socket_id = 0; + if (location.type() == "cpu") socket_id = location.index(); + *pptr = numa_alloc_onnode(size, socket_id); + if (!(*pptr)) + return Status::InternalError("Unable to allocate DRAM memory"); + return Status::OK(); +} + +Status RocmPlatform::free(void* ptr, size_t size) { + hipPointerAttribute_t attributes; + CHECK_HIP(hipPointerGetAttributes(&attributes, ptr)); + if (attributes.type == hipMemoryTypeDevice) { + CHECK_HIP(hipFree(ptr)); + } else if (attributes.type == hipMemoryTypeHost || + attributes.type == hipMemoryTypeUnregistered) { + numa_free(ptr, size); + } else { + LOG(ERROR) << "Unknown memory type, " << ptr << " " << attributes.type; + } + return Status::OK(); +} + +Status RocmPlatform::copy(void* dst, void* src, size_t length) { + HIPStreamHandle stream; + CHECK_STATUS(getStreamFromPool(stream)); + CHECK_HIP(hipMemcpyAsync(dst, src, length, hipMemcpyDefault, stream.get())); + CHECK_HIP(hipStreamSynchronize(stream.get())); + return Status::OK(); +} + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/platform/rocm/rocm_probe.cpp b/mooncake-transfer-engine/tent/src/platform/rocm/rocm_probe.cpp new file mode 100644 index 00000000..2906930a --- /dev/null +++ b/mooncake-transfer-engine/tent/src/platform/rocm/rocm_probe.cpp @@ -0,0 +1,344 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/platform/rocm.h" +#include "tent/common/status.h" +#include "tent/common/utils/prefault.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace mooncake { +namespace tent { + +static std::vector listInfiniBandDevices() { + int num_devices = 0; + std::vector devices; + + struct ibv_device** device_list = ibv_get_device_list(&num_devices); + if (!device_list || num_devices <= 0) { + LOG(WARNING) << "No RDMA devices found, check your device installation"; + return {}; + } + + for (int i = 0; i < num_devices; ++i) { + std::string device_name = ibv_get_device_name(device_list[i]); + char path[PATH_MAX + 32]; + char resolved_path[PATH_MAX]; + snprintf(path, sizeof(path), "/sys/class/infiniband/%s/../..", + device_name.c_str()); + if (realpath(path, resolved_path) == NULL) { + PLOG(ERROR) << "realpath " << path << ":"; + continue; + } + std::string pci_bus_id = basename(resolved_path); + + int numa_node = -1; + snprintf(path, sizeof(path), "%s/numa_node", resolved_path); + std::ifstream(path) >> numa_node; + + devices.push_back( + Topology::NicEntry{.name = std::move(device_name), + .pci_bus_id = std::move(pci_bus_id), + .type = Topology::NIC_RDMA, + .numa_node = numa_node}); + } + ibv_free_device_list(device_list); + return devices; +} + +static void filterInfiniBandDevices(std::vector& devices, + std::shared_ptr conf) { + auto whitelist = conf->getArray("topology/rdma_whitelist"); + auto blacklist = conf->getArray("topology/rdma_blacklist"); + std::vector new_devices; + if (!whitelist.empty()) { + for (auto& entry : devices) { + if (std::find(whitelist.begin(), whitelist.end(), entry.name) != + whitelist.end()) + new_devices.push_back(entry); + } + devices.swap(new_devices); + return; + } + if (!blacklist.empty()) { + for (auto& entry : devices) { + if (std::find(blacklist.begin(), blacklist.end(), entry.name) == + blacklist.end()) + new_devices.push_back(entry); + } + devices.swap(new_devices); + return; + } +} + +static void discoverCpuTopology(std::vector& nic_list, + std::vector& mem_list) { + DIR* dir = opendir("/sys/devices/system/node"); + struct dirent* entry; + if (dir == NULL) { + PLOG(WARNING) << "open /sys/devices/system/node failed"; + return; + } + while ((entry = readdir(dir))) { + const char* prefix = "node"; + if (entry->d_type != DT_DIR || + strncmp(entry->d_name, prefix, strlen(prefix)) != 0) { + continue; + } + int numa_node = atoi(entry->d_name + strlen(prefix)); + Topology::MemEntry mem_entry; + mem_entry.name = "cpu:" + std::to_string(numa_node); + mem_entry.numa_node = numa_node; + mem_entry.type = Topology::MEM_HOST; + int nic_id = 0; + for (const auto& device : nic_list) { + if (device.numa_node == numa_node) { + mem_entry.device_list[0].push_back(nic_id++); + } else { + mem_entry.device_list[2].push_back(nic_id++); + } + } + mem_list.push_back(std::move(mem_entry)); + } + (void)closedir(dir); +} + +static int getNumaNodeFromPciDevice(const std::string& pci_bdf) { + std::string sysfs_path = "/sys/bus/pci/devices/" + pci_bdf + "/numa_node"; + std::ifstream numa_file(sysfs_path); + if (!numa_file.is_open()) return -1; + int numa_node = -1; + numa_file >> numa_node; + if (numa_file.fail()) return -1; + return numa_node; +} + +static int getPciDistance(const char* bus1, const char* bus2) { + char buf[PATH_MAX]; + char path1[PATH_MAX]; + char path2[PATH_MAX]; + snprintf(buf, sizeof(buf), "/sys/bus/pci/devices/%s", bus1); + if (realpath(buf, path1) == NULL) return -1; + snprintf(buf, sizeof(buf), "/sys/bus/pci/devices/%s", bus2); + if (realpath(buf, path2) == NULL) return -1; + + char* ptr1 = path1; + char* ptr2 = path2; + while (*ptr1 && *ptr1 == *ptr2) { + ptr1++; + ptr2++; + } + int distance = 0; + for (; *ptr1; ptr1++) distance += (*ptr1 == '/'); + for (; *ptr2; ptr2++) distance += (*ptr2 == '/'); + return distance; +} + +static void discoverRocmTopology(std::vector& nic_list, + std::vector& mem_list) { + int device_count = 0; + auto err = hipGetDeviceCount(&device_count); + if (err != hipSuccess) { + LOG(WARNING) << "hipGetDeviceCount: " << hipGetErrorString(err); + device_count = 0; + } + for (int i = 0; i < device_count; i++) { + hipDeviceProp_t prop; + err = hipGetDeviceProperties(&prop, i); + if (err != hipSuccess) { + LOG(WARNING) << "hipGetDeviceProperties: " + << hipGetErrorString(err); + continue; + } + // Format PCI bus ID as lowercase: domain:bus:device.function + char pci_bus_id[20]; + snprintf(pci_bus_id, sizeof(pci_bus_id), "%04x:%02x:%02x.%x", + prop.pciDomainID, prop.pciBusID, prop.pciDeviceID, 0); + for (char* ch = pci_bus_id; *ch; ch++) *ch = tolower(*ch); + + int numa_node = getNumaNodeFromPciDevice(pci_bus_id); + int min_distance = INT_MAX; + std::unordered_map> distance_map; + for (const auto& device : nic_list) { + int dist = getPciDistance(device.pci_bus_id.c_str(), pci_bus_id); + distance_map[dist].push_back(&device - &nic_list[0]); + min_distance = std::min(min_distance, dist); + } + + Topology::MemEntry entry; + entry.name = "rocm:" + std::to_string(i); + entry.numa_node = numa_node; + entry.pci_bus_id = pci_bus_id; + entry.type = Topology::MEM_ROCM; + if (distance_map.count(0)) { + entry.device_list[0] = std::move(distance_map[0]); + } else if (distance_map.count(min_distance)) { + entry.device_list[0] = std::move(distance_map[min_distance]); + } + std::unordered_set preferred_set; + for (const auto& dev_id : entry.device_list[0]) { + preferred_set.insert(dev_id); + } + int dev_id = 0; + for (const auto& device : nic_list) { + if (!preferred_set.count(dev_id)) { + if (numa_node >= 0 && device.numa_node == numa_node) + entry.device_list[1].push_back(dev_id); + else + entry.device_list[2].push_back(dev_id); + } + dev_id++; + } + mem_list.push_back(std::move(entry)); + } +} + +static void insertFallbackMemEntry(int nic_list_count, + std::vector& mem_list) { + for (auto& entry : mem_list) { + if (entry.name == kWildcardLocation) { + entry.device_list[2].clear(); + for (int i = 0; i < nic_list_count; ++i) + entry.device_list[2].push_back(i); + return; + } + } + Topology::MemEntry new_entry; + new_entry.name = kWildcardLocation; + new_entry.numa_node = -1; + new_entry.type = Topology::MEM_HOST; + for (int i = 0; i < nic_list_count; ++i) + new_entry.device_list[2].push_back(i); + mem_list.push_back(new_entry); +} + +Status RocmPlatform::probe(std::vector& nic_list, + std::vector& mem_list) { + auto new_nic_list = listInfiniBandDevices(); + filterInfiniBandDevices(new_nic_list, conf); + for (auto& entry : new_nic_list) nic_list.push_back(entry); + insertFallbackMemEntry((int)nic_list.size(), mem_list); + discoverCpuTopology(nic_list, mem_list); + discoverRocmTopology(nic_list, mem_list); + return Status::OK(); +} + +MemoryType RocmPlatform::getMemoryType(void* addr) { + hipPointerAttribute_t attributes; + hipError_t result = hipPointerGetAttributes(&attributes, addr); + if (result != hipSuccess) { + LOG(WARNING) << "hipPointerGetAttributes: " + << hipGetErrorString(result); + return MTYPE_UNKNOWN; + } + if (attributes.type == hipMemoryTypeDevice) return MTYPE_ROCM; + return MTYPE_CPU; +} + +static inline uintptr_t alignPage(uintptr_t address) { + const static size_t kPageSize = 4096; + return address & ~(kPageSize - 1); +} + +static inline std::string genCpuNodeName(int node) { + if (node >= 0) return "cpu:" + std::to_string(node); + return kWildcardLocation; +} + +static inline std::string genRocmNodeName(int node) { + if (node >= 0) return "rocm:" + std::to_string(node); + return kWildcardLocation; +} + +const std::vector RocmPlatform::getLocation(void* start, + size_t len, + bool skip_prefault) { + const static size_t kPageSize = 4096; + std::vector entries; + + hipPointerAttribute_t attributes; + hipError_t result = hipPointerGetAttributes(&attributes, start); + if (result != hipSuccess) { + LOG(WARNING) << "hipPointerGetAttributes: " + << hipGetErrorString(result); + entries.push_back({(uint64_t)start, len, kWildcardLocation}); + return entries; + } + + if (attributes.type == hipMemoryTypeDevice) { + entries.push_back( + {(uint64_t)start, len, genRocmNodeName(attributes.device)}); + return entries; + } + + uintptr_t aligned_start = alignPage((uintptr_t)start); + int n = + (uintptr_t(start) - aligned_start + len + kPageSize - 1) / kPageSize; + void** pages = (void**)malloc(sizeof(void*) * n); + int* status = (int*)malloc(sizeof(int) * n); + + for (int i = 0; i < n; i++) { + pages[i] = (void*)((char*)aligned_start + i * kPageSize); + } + + if (!skip_prefault) { + prefaultBeforeProbe(pages, n, aligned_start, "RocmPlatform"); + } + + int rc = numa_move_pages(0, n, pages, nullptr, status, 0); + if (rc != 0) { + entries.push_back({(uint64_t)start, len, kWildcardLocation}); + ::free(pages); + ::free(status); + return entries; + } + + int node = status[0]; + uint64_t start_addr = (uint64_t)start; + uint64_t new_start_addr; + for (int i = 1; i < n; i++) { + if (status[i] != node) { + new_start_addr = alignPage((uint64_t)start) + i * kPageSize; + entries.push_back({start_addr, size_t(new_start_addr - start_addr), + genCpuNodeName(node)}); + start_addr = new_start_addr; + node = status[i]; + } + } + entries.push_back( + {start_addr, (uint64_t)start + len - start_addr, genCpuNodeName(node)}); + ::free(pages); + ::free(status); + return entries; +} + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/platform/rocm/rocm_stream_pool.cpp b/mooncake-transfer-engine/tent/src/platform/rocm/rocm_stream_pool.cpp new file mode 100644 index 00000000..65dbbad8 --- /dev/null +++ b/mooncake-transfer-engine/tent/src/platform/rocm/rocm_stream_pool.cpp @@ -0,0 +1,150 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/platform/rocm.h" + +namespace mooncake { +namespace tent { + +HIPStreamHandle& HIPStreamHandle::operator=(HIPStreamHandle&& other) noexcept { + if (this != &other) { + releaseToPool(); + stream_ = other.stream_; + deviceId_ = other.deviceId_; + pool_ = other.pool_; + + other.stream_ = nullptr; + other.pool_ = nullptr; + } + return *this; +} + +HIPStreamHandle::~HIPStreamHandle() { releaseToPool(); } + +void HIPStreamHandle::releaseToPool() { + if (stream_ != nullptr && pool_ != nullptr) { + pool_->release(deviceId_, stream_); + stream_ = nullptr; + pool_ = nullptr; + } +} + +HIPStreamPool::DevicePool::DevicePool(int deviceId) : deviceId_(deviceId) {} + +HIPStreamPool::DevicePool::~DevicePool() { + int currentDevice; + if (hipGetDevice(¤tDevice) == hipSuccess) { + hipSetDevice(deviceId_); + for (hipStream_t stream : availableStreams_) { + hipStreamDestroy(stream); + } + hipSetDevice(currentDevice); + } +} + +Status HIPStreamPool::DevicePool::acquire(hipStream_t& outStream) { + { + RWSpinlock::WriteGuard guard(dev_lock_); + if (!availableStreams_.empty()) { + outStream = availableStreams_.back(); + availableStreams_.pop_back(); + return Status::OK(); + } + } + + int currentDevice; + CHECK_HIP(hipGetDevice(¤tDevice)); + + if (currentDevice != deviceId_) { + CHECK_HIP(hipSetDevice(deviceId_)); + } + + CHECK_HIP(hipStreamCreateWithFlags(&outStream, hipStreamNonBlocking)); + + if (currentDevice != deviceId_) { + CHECK_HIP(hipSetDevice(currentDevice)); + } + + return Status::OK(); +} + +void HIPStreamPool::DevicePool::release(hipStream_t stream) { + RWSpinlock::WriteGuard guard(dev_lock_); + availableStreams_.push_back(stream); +} + +Status HIPStreamPool::acquire(HIPStreamHandle& outHandle, int deviceId) { + if (deviceId == kCurrentDevice) { + if (hipGetDevice(&deviceId) != hipSuccess) { + return Status::InternalError("Failed to get current HIP device ID"); + } + } else if (deviceId < 0) { + return Status::InternalError("Invalid HIP device ID"); + } + + DevicePool* devicePool = getDevicePool(deviceId); + if (!devicePool) { + return Status::InternalError("Failed to get HIP device pool"); + } + + hipStream_t rawStream; + CHECK_STATUS(devicePool->acquire(rawStream)); + + outHandle = HIPStreamHandle(rawStream, deviceId, this); + return Status::OK(); +} + +void HIPStreamPool::release(int deviceId, hipStream_t stream) { + DevicePool* devicePool = getDevicePool(deviceId); + if (devicePool) devicePool->release(stream); +} + +HIPStreamPool::DevicePool* HIPStreamPool::getDevicePool(int deviceId) { + { + RWSpinlock::ReadGuard readGuard(pools_lock_); + if (static_cast(deviceId) < devicePools_.size() && + devicePools_[deviceId]) { + return devicePools_[deviceId].get(); + } + } + + RWSpinlock::WriteGuard writeGuard(pools_lock_); + + if (static_cast(deviceId) < devicePools_.size() && + devicePools_[deviceId]) { + return devicePools_[deviceId].get(); + } + + int actualDeviceCount = 0; + if (hipGetDeviceCount(&actualDeviceCount) != hipSuccess || + deviceId >= actualDeviceCount) { + LOG(ERROR) << "Invalid HIP device id " << deviceId; + return nullptr; + } + + if (static_cast(deviceId) >= devicePools_.size()) { + devicePools_.resize(deviceId + 1); + } + + devicePools_[deviceId] = std::make_unique(deviceId); + return devicePools_[deviceId].get(); +} + +Status RocmPlatform::getStreamFromPool(HIPStreamHandle& outHandle, + int deviceId) { + return stream_pool_.acquire(outHandle, deviceId); +} + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/runtime/platform.cpp b/mooncake-transfer-engine/tent/src/runtime/platform.cpp index 8c79b29c..72a9d096 100644 --- a/mooncake-transfer-engine/tent/src/runtime/platform.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/platform.cpp @@ -16,6 +16,8 @@ #ifdef USE_CUDA #include "tent/platform/cuda.h" +#elif defined(USE_HIP) +#include "tent/platform/rocm.h" #elif defined(USE_SUNRISE) #include "tent/platform/sunrise.h" #elif defined(USE_ASCEND) || defined(USE_ASCEND_DIRECT) @@ -33,6 +35,8 @@ Platform& Platform::getLoader(std::shared_ptr conf) { std::call_once(flag, [&]() { #ifdef USE_CUDA g_instance = std::make_shared(conf); +#elif defined(USE_HIP) + g_instance = std::make_shared(conf); #elif defined(USE_SUNRISE) g_instance = std::make_shared(conf); #elif defined(USE_ASCEND) || defined(USE_ASCEND_DIRECT) diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index 7c6cbf00..869ff38e 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -770,10 +770,14 @@ Status TransferEngineImpl::lazyFreeBatch() { return Status::OK(); } +static bool isGpuType(MemoryType t) { + return t == MTYPE_CUDA || t == MTYPE_ROCM; +} + static bool checkAvailability(const std::shared_ptr& xport, MemoryType local) { if (local == MTYPE_CPU) return xport && xport->capabilities().dram_to_file; - if (local == MTYPE_CUDA) return xport && xport->capabilities().gpu_to_file; + if (isGpuType(local)) return xport && xport->capabilities().gpu_to_file; return false; } @@ -781,11 +785,11 @@ static bool checkAvailability(const std::shared_ptr& xport, MemoryType local, MemoryType remote) { if (local == MTYPE_CPU && remote == MTYPE_CPU) return xport && xport->capabilities().dram_to_dram; - if (local == MTYPE_CUDA && remote == MTYPE_CUDA) + if (isGpuType(local) && isGpuType(remote)) return xport && xport->capabilities().gpu_to_gpu; - if (local == MTYPE_CPU && remote == MTYPE_CUDA) + if (local == MTYPE_CPU && isGpuType(remote)) return xport && xport->capabilities().dram_to_gpu; - if (local == MTYPE_CUDA && remote == MTYPE_CPU) + if (isGpuType(local) && remote == MTYPE_CPU) return xport && xport->capabilities().gpu_to_dram; return false; } @@ -794,6 +798,7 @@ static MemoryType getTypeEnum(const std::string& type) { if (type == "cpu" || type == "*") return MTYPE_CPU; if (type == "cuda") return MTYPE_CUDA; if (type == "npu") return MTYPE_CUDA; + if (type == "rocm") return MTYPE_ROCM; return MTYPE_UNKNOWN; } diff --git a/mooncake-transfer-engine/tent/tests/CMakeLists.txt b/mooncake-transfer-engine/tent/tests/CMakeLists.txt index 41c379b1..4b9ec56e 100644 --- a/mooncake-transfer-engine/tent/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/tests/CMakeLists.txt @@ -58,6 +58,17 @@ add_executable(tent_endpoint_lifecycle_test endpoint_lifecycle_test.cpp) target_link_libraries(tent_endpoint_lifecycle_test PRIVATE gtest gtest_main) add_test(NAME tent_endpoint_lifecycle_test COMMAND tent_endpoint_lifecycle_test) +if(USE_HIP) + find_package(HIP REQUIRED) + add_executable(tent_rocm_platform_test rocm_platform_test.cpp) + target_link_libraries(tent_rocm_platform_test PRIVATE gtest gtest_main + tent_link_group + hip::host) + target_include_directories(tent_rocm_platform_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) + add_test(NAME tent_rocm_platform_test COMMAND tent_rocm_platform_test) +endif() + if(USE_SUNRISE) add_executable(tent_sunrise_link_transport_test sunrise_link_transport_test.cpp) diff --git a/mooncake-transfer-engine/tent/tests/hip_bandwidth_bench.cpp b/mooncake-transfer-engine/tent/tests/hip_bandwidth_bench.cpp new file mode 100644 index 00000000..a265a808 --- /dev/null +++ b/mooncake-transfer-engine/tent/tests/hip_bandwidth_bench.cpp @@ -0,0 +1,228 @@ +// Copyright 2025 KVCache.AI +// +// Standalone intra-node HIP bandwidth benchmark. +// Measures H2D, D2H, D2D (same GPU), and GPU-to-GPU (P2P via XGMI) +// bandwidth across a sweep of transfer sizes. +// +// Build: +// hipcc -O3 -o hip_bandwidth_bench hip_bandwidth_bench.cpp +// +// Run: +// ./hip_bandwidth_bench # all GPUs, all directions +// ./hip_bandwidth_bench 0 1 # GPU 0 as src, GPU 1 as dst + +#include +#include +#include +#include +#include +#include + +#define CHECK_HIP(call) \ + do { \ + hipError_t err = (call); \ + if (err != hipSuccess) { \ + fprintf(stderr, "HIP error %s:%d: %s\n", __FILE__, __LINE__, \ + hipGetErrorString(err)); \ + exit(EXIT_FAILURE); \ + } \ + } while (0) + +static const size_t kWarmupIter = 5; +static const size_t kBenchIter = 20; + +// Returns elapsed microseconds for `iters` copies of `bytes`. +static double benchCopy(void* dst, const void* src, size_t bytes, + hipMemcpyKind kind, hipStream_t stream, size_t iters) { + // warmup + for (size_t i = 0; i < kWarmupIter; i++) + CHECK_HIP(hipMemcpyAsync(dst, src, bytes, kind, stream)); + CHECK_HIP(hipStreamSynchronize(stream)); + + hipEvent_t start, stop; + CHECK_HIP(hipEventCreate(&start)); + CHECK_HIP(hipEventCreate(&stop)); + + CHECK_HIP(hipEventRecord(start, stream)); + for (size_t i = 0; i < iters; i++) + CHECK_HIP(hipMemcpyAsync(dst, src, bytes, kind, stream)); + CHECK_HIP(hipEventRecord(stop, stream)); + CHECK_HIP(hipStreamSynchronize(stream)); + + float ms = 0; + CHECK_HIP(hipEventElapsedTime(&ms, start, stop)); + CHECK_HIP(hipEventDestroy(start)); + CHECK_HIP(hipEventDestroy(stop)); + return (double)ms * 1000.0; // → microseconds +} + +static void printHeader() { + printf("\n%-18s %10s %12s %12s\n", "Size", "Iters", "BW (GB/s)", + "Lat (us)"); + printf("%s\n", std::string(58, '-').c_str()); +} + +static void runBench(const char* label, void* dst, const void* src, + size_t max_bytes, hipMemcpyKind kind, hipStream_t stream) { + printf("\n[%s]\n", label); + printHeader(); + + for (size_t bytes = 4096; bytes <= max_bytes; bytes *= 4) { + size_t iters = std::max(kBenchIter, 200 * 1024 * 1024 / bytes); + double us = benchCopy(dst, src, bytes, kind, stream, iters); + double bw = (double)bytes * iters / (us / 1e6) / 1e9; // GB/s + double lat = us / iters; + printf("%-18zu %10zu %12.3f %12.2f\n", bytes, iters, bw, lat); + } +} + +int main(int argc, char** argv) { + int gpu_count = 0; + CHECK_HIP(hipGetDeviceCount(&gpu_count)); + printf("Detected %d AMD GPU(s)\n", gpu_count); + + for (int g = 0; g < gpu_count; g++) { + hipDeviceProp_t prop; + CHECK_HIP(hipGetDeviceProperties(&prop, g)); + printf(" GPU %d: %s VRAM %.0f GiB PCIe %04x:%02x:%02x.0\n", g, + prop.name, + (double)prop.totalGlobalMem / (1024.0 * 1024.0 * 1024.0), + prop.pciDomainID, prop.pciBusID, prop.pciDeviceID); + } + + int src_gpu = (argc >= 2) ? atoi(argv[1]) : 0; + int dst_gpu = (argc >= 3) ? atoi(argv[2]) : (gpu_count > 1 ? 1 : 0); + + const size_t kMaxBytes = 4ULL * 1024 * 1024 * 1024; // 4 GiB + const size_t kHostBytes = 512ULL * 1024 * 1024; // 512 MiB host + + // --- allocate host pinned --- + void* h_src = nullptr; + void* h_dst = nullptr; + CHECK_HIP(hipHostMalloc(&h_src, kHostBytes, hipHostMallocDefault)); + CHECK_HIP(hipHostMalloc(&h_dst, kHostBytes, hipHostMallocDefault)); + memset(h_src, 0xAA, kHostBytes); + + // --- allocate device buffers on src_gpu --- + CHECK_HIP(hipSetDevice(src_gpu)); + void* d_src = nullptr; + CHECK_HIP(hipMalloc(&d_src, kMaxBytes)); + CHECK_HIP(hipMemset(d_src, 0xBB, kMaxBytes)); + + hipStream_t stream_src; + CHECK_HIP(hipStreamCreate(&stream_src)); + + // --- allocate device buffer on dst_gpu --- + void* d_dst = nullptr; + hipStream_t stream_dst; + if (dst_gpu != src_gpu) { + CHECK_HIP(hipSetDevice(dst_gpu)); + CHECK_HIP(hipMalloc(&d_dst, kMaxBytes)); + CHECK_HIP(hipMemset(d_dst, 0xCC, kMaxBytes)); + CHECK_HIP(hipStreamCreate(&stream_dst)); + } else { + d_dst = (char*)d_src + kMaxBytes / 2; + stream_dst = stream_src; + } + + // ---------------------------------------------------------------- + printf("\n=== H2D: Host → GPU %d ===\n", src_gpu); + CHECK_HIP(hipSetDevice(src_gpu)); + runBench("H2D", d_src, h_src, kHostBytes, hipMemcpyHostToDevice, + stream_src); + + // ---------------------------------------------------------------- + printf("\n=== D2H: GPU %d → Host ===\n", src_gpu); + runBench("D2H", h_dst, d_src, kHostBytes, hipMemcpyDeviceToHost, + stream_src); + + // ---------------------------------------------------------------- + printf("\n=== D2D (intra-GPU %d): same-device copy ===\n", src_gpu); + runBench("D2D intra", (char*)d_src + kMaxBytes / 2, d_src, kMaxBytes / 2, + hipMemcpyDeviceToDevice, stream_src); + + // ---------------------------------------------------------------- + if (dst_gpu != src_gpu) { + // Check P2P access + int can_access = 0; + hipDeviceCanAccessPeer(&can_access, src_gpu, dst_gpu); + if (can_access) { + CHECK_HIP(hipSetDevice(src_gpu)); + CHECK_HIP(hipDeviceEnablePeerAccess(dst_gpu, 0)); + } + + char label[64]; + snprintf(label, sizeof(label), "GPU %d → GPU %d (P2P %s)", src_gpu, + dst_gpu, can_access ? "enabled" : "disabled"); + printf("\n=== %s ===\n", label); + + CHECK_HIP(hipSetDevice(src_gpu)); + runBench(label, d_dst, d_src, kMaxBytes, hipMemcpyDeviceToDevice, + stream_src); + + // Reverse direction + snprintf(label, sizeof(label), "GPU %d → GPU %d (reverse)", dst_gpu, + src_gpu); + printf("\n=== %s ===\n", label); + CHECK_HIP(hipSetDevice(dst_gpu)); + runBench(label, d_src, d_dst, kMaxBytes, hipMemcpyDeviceToDevice, + stream_dst); + } + + // ---------------------------------------------------------------- + // All-pairs GPU-to-GPU if no specific args + if (argc < 3 && gpu_count > 1) { + printf("\n=== All-pairs GPU P2P bandwidth (large block = 1GiB) ===\n"); + printf("%-8s %-8s %12s\n", "Src", "Dst", "BW (GB/s)"); + printf("%s\n", std::string(32, '-').c_str()); + + std::vector dbufs(gpu_count, nullptr); + for (int g = 0; g < gpu_count; g++) { + CHECK_HIP(hipSetDevice(g)); + CHECK_HIP(hipMalloc(&dbufs[g], 1ULL * 1024 * 1024 * 1024)); + CHECK_HIP(hipMemset(dbufs[g], g, 1ULL * 1024 * 1024 * 1024)); + } + + const size_t kP2PBytes = 1ULL * 1024 * 1024 * 1024; + for (int s = 0; s < gpu_count; s++) { + for (int d = 0; d < gpu_count; d++) { + if (s == d) { + printf("GPU%-5d GPU%-5d %12s\n", s, d, "-"); + continue; + } + CHECK_HIP(hipSetDevice(s)); + int ca = 0; + hipDeviceCanAccessPeer(&ca, s, d); + if (ca) hipDeviceEnablePeerAccess(d, 0); + + hipStream_t st; + CHECK_HIP(hipStreamCreate(&st)); + double us = benchCopy(dbufs[d], dbufs[s], kP2PBytes, + hipMemcpyDeviceToDevice, st, kBenchIter); + double bw = (double)kP2PBytes * kBenchIter / (us / 1e6) / 1e9; + printf("GPU%-5d GPU%-5d %12.2f\n", s, d, bw); + CHECK_HIP(hipStreamDestroy(st)); + } + } + + for (int g = 0; g < gpu_count; g++) { + CHECK_HIP(hipSetDevice(g)); + CHECK_HIP(hipFree(dbufs[g])); + } + } + + // cleanup + CHECK_HIP(hipSetDevice(src_gpu)); + CHECK_HIP(hipFree(d_src)); + CHECK_HIP(hipStreamDestroy(stream_src)); + if (dst_gpu != src_gpu) { + CHECK_HIP(hipSetDevice(dst_gpu)); + CHECK_HIP(hipFree(d_dst)); + CHECK_HIP(hipStreamDestroy(stream_dst)); + } + CHECK_HIP(hipHostFree(h_src)); + CHECK_HIP(hipHostFree(h_dst)); + + printf("\nDone.\n"); + return 0; +} diff --git a/mooncake-transfer-engine/tent/tests/rocm_platform_test.cpp b/mooncake-transfer-engine/tent/tests/rocm_platform_test.cpp new file mode 100644 index 00000000..d0b85f95 --- /dev/null +++ b/mooncake-transfer-engine/tent/tests/rocm_platform_test.cpp @@ -0,0 +1,249 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include + +#include +#include +#include +#include + +#include "tent/common/config.h" +#include "tent/platform/rocm.h" +#include "tent/runtime/platform.h" +#include "tent/runtime/topology.h" + +namespace mooncake { +namespace tent { +namespace { + +// Helper: returns the number of visible AMD GPUs (0 if ROCm not available). +static int getHipDeviceCount() { + int count = 0; + if (hipGetDeviceCount(&count) != hipSuccess) return 0; + return count; +} + +// --------------------------------------------------------------------------- +// HIPStreamPool unit tests +// --------------------------------------------------------------------------- + +TEST(HIPStreamPoolTest, AcquireAndRelease) { + if (getHipDeviceCount() == 0) GTEST_SKIP() << "No AMD GPU available"; + + HIPStreamPool pool; + HIPStreamHandle handle; + Status s = pool.acquire(handle); + EXPECT_TRUE(s.ok()) << s; + EXPECT_NE(handle.get(), nullptr); + // Destructor releases back to pool automatically — no assertion needed. +} + +TEST(HIPStreamPoolTest, AcquireMultipleStreams) { + if (getHipDeviceCount() == 0) GTEST_SKIP() << "No AMD GPU available"; + + HIPStreamPool pool; + constexpr int kStreams = 4; + std::vector handles(kStreams); + for (int i = 0; i < kStreams; i++) { + Status s = pool.acquire(handles[i]); + EXPECT_TRUE(s.ok()) << s; + EXPECT_NE(handles[i].get(), nullptr); + } +} + +TEST(HIPStreamPoolTest, AcquireByExplicitDeviceId) { + if (getHipDeviceCount() == 0) GTEST_SKIP() << "No AMD GPU available"; + + HIPStreamPool pool; + HIPStreamHandle handle; + Status s = pool.acquire(handle, 0); + EXPECT_TRUE(s.ok()) << s; + EXPECT_NE(handle.get(), nullptr); +} + +TEST(HIPStreamPoolTest, InvalidDeviceIdReturnsError) { + HIPStreamPool pool; + HIPStreamHandle handle; + // Device ID -5 is always invalid. + Status s = pool.acquire(handle, -5); + EXPECT_FALSE(s.ok()); +} + +// --------------------------------------------------------------------------- +// RocmPlatform unit tests +// --------------------------------------------------------------------------- + +class RocmPlatformTest : public ::testing::Test { + protected: + void SetUp() override { + if (getHipDeviceCount() == 0) { + GTEST_SKIP() << "No AMD GPU available"; + } + auto conf = std::make_shared(); + platform_ = std::make_shared(conf); + } + + std::shared_ptr platform_; +}; + +TEST_F(RocmPlatformTest, TypeString) { EXPECT_EQ(platform_->type(), "rocm"); } + +TEST_F(RocmPlatformTest, ProbeDiscoversMems) { + std::vector nics; + std::vector mems; + Status s = platform_->probe(nics, mems); + EXPECT_TRUE(s.ok()) << s; + // At least one ROCm memory entry should be found. + bool found_rocm = false; + for (const auto& m : mems) { + if (m.type == Topology::MEM_ROCM) { + found_rocm = true; + EXPECT_EQ(m.name.substr(0, 5), "rocm:"); + } + } + EXPECT_TRUE(found_rocm) << "probe() found no MEM_ROCM entries"; +} + +TEST_F(RocmPlatformTest, AllocateAndFreeDeviceMemory) { + MemoryOptions opts; + opts.location = "rocm:0"; + void* ptr = nullptr; + constexpr size_t kSize = 1024 * 1024; // 1 MiB + + Status s = platform_->allocate(&ptr, kSize, opts); + ASSERT_TRUE(s.ok()) << s; + ASSERT_NE(ptr, nullptr); + + EXPECT_EQ(platform_->getMemoryType(ptr), MTYPE_ROCM); + + Status fs = platform_->free(ptr, kSize); + EXPECT_TRUE(fs.ok()) << fs; +} + +TEST_F(RocmPlatformTest, AllocateAndFreeCpuMemory) { + MemoryOptions opts; + opts.location = "cpu:0"; + void* ptr = nullptr; + constexpr size_t kSize = 4096; + + Status s = platform_->allocate(&ptr, kSize, opts); + ASSERT_TRUE(s.ok()) << s; + ASSERT_NE(ptr, nullptr); + + EXPECT_EQ(platform_->getMemoryType(ptr), MTYPE_CPU); + + Status fs = platform_->free(ptr, kSize); + EXPECT_TRUE(fs.ok()) << fs; +} + +TEST_F(RocmPlatformTest, CopyDeviceToDevice) { + MemoryOptions opts; + opts.location = "rocm:0"; + constexpr size_t kSize = 256; + void *src = nullptr, *dst = nullptr; + + ASSERT_TRUE(platform_->allocate(&src, kSize, opts).ok()); + ASSERT_TRUE(platform_->allocate(&dst, kSize, opts).ok()); + + // Fill src with a pattern via hipMemset. + ASSERT_EQ(hipMemset(src, 0xAB, kSize), hipSuccess); + + Status cs = platform_->copy(dst, src, kSize); + EXPECT_TRUE(cs.ok()) << cs; + + // Verify the copy by reading back to host. + std::vector host_buf(kSize, 0); + ASSERT_EQ(hipMemcpy(host_buf.data(), dst, kSize, hipMemcpyDeviceToHost), + hipSuccess); + for (size_t i = 0; i < kSize; i++) { + EXPECT_EQ(host_buf[i], 0xAB) << "mismatch at byte " << i; + } + + EXPECT_TRUE(platform_->free(src, kSize).ok()); + EXPECT_TRUE(platform_->free(dst, kSize).ok()); +} + +TEST_F(RocmPlatformTest, CopyHostToDevice) { + MemoryOptions dev_opts; + dev_opts.location = "rocm:0"; + MemoryOptions cpu_opts; + cpu_opts.location = "cpu:0"; + constexpr size_t kSize = 512; + void *dev = nullptr, *host_mem = nullptr; + + ASSERT_TRUE(platform_->allocate(&dev, kSize, dev_opts).ok()); + ASSERT_TRUE(platform_->allocate(&host_mem, kSize, cpu_opts).ok()); + + memset(host_mem, 0xCD, kSize); + + Status cs = platform_->copy(dev, host_mem, kSize); + EXPECT_TRUE(cs.ok()) << cs; + + std::vector verify(kSize, 0); + ASSERT_EQ(hipMemcpy(verify.data(), dev, kSize, hipMemcpyDeviceToHost), + hipSuccess); + for (size_t i = 0; i < kSize; i++) { + EXPECT_EQ(verify[i], 0xCD) << "mismatch at byte " << i; + } + + EXPECT_TRUE(platform_->free(dev, kSize).ok()); + EXPECT_TRUE(platform_->free(host_mem, kSize).ok()); +} + +TEST_F(RocmPlatformTest, GetLocationDeviceMemory) { + MemoryOptions opts; + opts.location = "rocm:0"; + constexpr size_t kSize = 4096; + void* ptr = nullptr; + + ASSERT_TRUE(platform_->allocate(&ptr, kSize, opts).ok()); + auto locs = platform_->getLocation(ptr, kSize); + ASSERT_FALSE(locs.empty()); + EXPECT_EQ(locs[0].location, "rocm:0"); + EXPECT_TRUE(platform_->free(ptr, kSize).ok()); +} + +TEST_F(RocmPlatformTest, GetMemoryTypeUnknownPointer) { + // An arbitrary user-space stack pointer should not be MTYPE_ROCM. + int local_var = 42; + MemoryType mtype = platform_->getMemoryType(&local_var); + // CPU or UNKNOWN — never MTYPE_ROCM. + EXPECT_NE(mtype, MTYPE_ROCM); +} + +// --------------------------------------------------------------------------- +// Multi-GPU tests (skipped when only one GPU is present) +// --------------------------------------------------------------------------- + +TEST_F(RocmPlatformTest, AllocateOnSecondGpu) { + if (getHipDeviceCount() < 2) GTEST_SKIP() << "Requires >= 2 AMD GPUs"; + + MemoryOptions opts; + opts.location = "rocm:1"; + void* ptr = nullptr; + constexpr size_t kSize = 1024; + + Status s = platform_->allocate(&ptr, kSize, opts); + ASSERT_TRUE(s.ok()) << s; + ASSERT_NE(ptr, nullptr); + + EXPECT_EQ(platform_->getMemoryType(ptr), MTYPE_ROCM); + + EXPECT_TRUE(platform_->free(ptr, kSize).ok()); +} + +} // namespace +} // namespace tent +} // namespace mooncake -- 2.34.1 From 5ba016ac1cf585ea045a218abb036fa818961777 Mon Sep 17 00:00:00 2001 From: yokinoshitayoki <113957881+yokinoshitayoki@users.noreply.github.com> Date: Thu, 7 May 2026 21:31:18 +0800 Subject: [PATCH 051/382] [Store] For alloctor benchmark: Add DSA-like workload (#2051) --- .../performance/allocator-benchmark-result.md | 26 ++++++ mooncake-store/benchmarks/allocator_bench.cpp | 83 ++++++++++++++++++- 2 files changed, 105 insertions(+), 4 deletions(-) diff --git a/docs/source/performance/allocator-benchmark-result.md b/docs/source/performance/allocator-benchmark-result.md index 1c562266..0e10b242 100644 --- a/docs/source/performance/allocator-benchmark-result.md +++ b/docs/source/performance/allocator-benchmark-result.md @@ -195,3 +195,29 @@ util ratio (min / p99 / p90 / p50 / max / avg): 0.569255 / 0.712076 / 0.781224 / 0.855046 / 0.976057 / 0.848873 avg alloc time: 142.508508 ns/op ``` + +**OffsetAllocator (Before Optimization)** + +``` +util ratio (min / p99 / p90 / p50 / max / avg): +0.569255 / 0.712076 / 0.781224 / 0.855046 / 0.976057 / 0.848873 +avg alloc time: 142.508508 ns/op +``` + +### Paired KV/Indexer Allocation Benchmark (DSA) + +In the DSA scenario, Mooncake Store stores both KV cache objects and indexer objects. +We evaluated OffsetAllocator under a paired allocation workload using object sizes derived from a GLM-5.1-FP8 DSA serving configuration. + +In this configuration, Mooncake Store stores data at the page granularity, where each page contains 64 tokens. Therefore, the object sizes observed by the allocator are the packed page-level sizes rather than the per-token sizes: + +- KV cache object size: 3.12 MB +- Indexer object size: 643 KB + +**OffsetAllocator** + +``` +util ratio (min / p99 / p90 / p50 / max / avg): +0.948299 / 0.948765 / 0.949311 / 0.949884 / 0.952491 / 0.950091 +avg alloc time: 232.420141 ns/op +``` diff --git a/mooncake-store/benchmarks/allocator_bench.cpp b/mooncake-store/benchmarks/allocator_bench.cpp index e330b4c4..273c2a7a 100644 --- a/mooncake-store/benchmarks/allocator_bench.cpp +++ b/mooncake-store/benchmarks/allocator_bench.cpp @@ -12,7 +12,7 @@ using namespace mooncake::offset_allocator; class OffsetAllocatorBenchHelper { public: - OffsetAllocatorBenchHelper(uint64_t baseAddress, uint32_t poolSize, + OffsetAllocatorBenchHelper(uint64_t baseAddress, size_t poolSize, uint32_t maxAllocs) : pool_size_(poolSize), allocated_size_(0), @@ -160,13 +160,11 @@ void random_size_allocation_benchmark() { static_cast(benchmark_num); std::sort(util_ratios.begin(), util_ratios.end()); - const double min_util = util_ratios.front(); const double max_util = util_ratios.back(); const double p50 = util_ratios[util_ratios.size() * 0.50]; const double p90 = util_ratios[util_ratios.size() * 0.10]; const double p99 = util_ratios[util_ratios.size() * 0.01]; - const double mean_util = std::accumulate(util_ratios.begin(), util_ratios.end(), 0.0) / util_ratios.size(); @@ -178,8 +176,85 @@ void random_size_allocation_benchmark() { std::cout << "avg alloc time: " << avg_time_ns << " ns/op" << std::endl; } +template +void paired_kv_indexer_allocation_benchmark() { + std::cout << std::endl + << "=== Paired KV/Indexer Allocation Benchmark (DSA) ===" + << std::endl; + + const uint32_t kvcache_size = 3274752; // 3,274,752 B + const uint32_t indexer_size = 643u * 1024; // 643 KB + const size_t pool_size = 600ull * 1024 * 1024 * 1024; // 600 GB + const int max_per_round = 128; + const int warmup_rounds = 5000; + const int num_rounds = 500000; + + size_t max_allocs = pool_size / indexer_size + 1024; + BenchHelper bench_helper(0x1000, pool_size, max_allocs); + + std::random_device rd; + std::mt19937 gen(rd()); + std::uniform_int_distribution per_round_dist(1, max_per_round); + + // Warmup + for (int round = 0; round < warmup_rounds; round++) { + int per_round = per_round_dist(gen); + for (int i = 0; i < per_round; i++) { + bench_helper.allocate(kvcache_size); + } + for (int i = 0; i < per_round; i++) { + bench_helper.allocate(indexer_size); + } + } + + std::vector util_ratios; + util_ratios.reserve(static_cast(num_rounds) * 2 * max_per_round); + + auto start_time = std::chrono::high_resolution_clock::now(); + for (int round = 0; round < num_rounds; round++) { + int per_round = per_round_dist(gen); + for (int i = 0; i < per_round; i++) { + bench_helper.allocate(kvcache_size); + util_ratios.push_back(bench_helper.get_allocated_ratio()); + } + for (int i = 0; i < per_round; i++) { + bench_helper.allocate(indexer_size); + util_ratios.push_back(bench_helper.get_allocated_ratio()); + } + } + auto end_time = std::chrono::high_resolution_clock::now(); + + const double avg_time_ns = + std::chrono::duration_cast(end_time - + start_time) + .count() / + static_cast(util_ratios.size()); + + std::sort(util_ratios.begin(), util_ratios.end()); + const double min_util = util_ratios.front(); + const double max_util = util_ratios.back(); + const double p50 = util_ratios[util_ratios.size() * 0.50]; + const double p90 = util_ratios[util_ratios.size() * 0.10]; + const double p99 = util_ratios[util_ratios.size() * 0.01]; + const double mean_util = + std::accumulate(util_ratios.begin(), util_ratios.end(), 0.0) / + util_ratios.size(); + + std::cout << std::fixed << std::setprecision(6); + std::cout << "kvcache size: " << kvcache_size + << " B, indexer size: " << indexer_size << " B" << std::endl; + std::cout << "pool size: " << (pool_size / (1024.0 * 1024 * 1024)) + << " GB, warmup rounds: " << warmup_rounds + << ", benchmark rounds: " << num_rounds << std::endl; + std::cout << "util ratio (min / p99 / p90 / p50 / max / avg): " << min_util + << " / " << p99 << " / " << p90 << " / " << p50 << " / " + << max_util << " / " << mean_util << std::endl; + std::cout << "avg alloc time: " << avg_time_ns << " ns/op" << std::endl; +} + int main() { std::cout << "=== OffsetAllocator Benchmark ===" << std::endl; uniform_size_allocation_benchmark(); random_size_allocation_benchmark(); -} \ No newline at end of file + paired_kv_indexer_allocation_benchmark(); +} -- 2.34.1 From dec5ab40b734d9840d3042b7589e6ed50b6a8eb3 Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Fri, 8 May 2026 14:33:49 +0800 Subject: [PATCH 052/382] [Store] fix(master-metrics): track local SSD storage in Master metrics (#1969) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(master-metrics): track local SSD storage in Master metrics Master logs always showed "SSD Storage: 0 B / 0 B" even when local SSD offloading was active. Three root causes fixed: Bug 1 — allocated size never tracked for LocalDiskReplica (replica.h) The LocalDiskReplica constructor did not call inc_allocated_file_size(), so the numerator stayed zero regardless of how many objects were offloaded. Symmetric fixes applied to the destructor and move-assignment operator. Also fixed a pre-existing bug: id_ and refcnt_ were uninitialized in the LocalDiskReplica constructor. Bug 2 — total SSD capacity unknown to Master Master has no direct visibility into client-side SSD configuration, so file_total_capacity_ (the denominator) was always 0. A new dedicated RPC ReportSsdCapacity(client_id, ssd_total_capacity_bytes) is added; clients call it once in FileStorage::Init() after MountLocalDiskSegment succeeds. Master stores the value per-client in LocalDiskSegment::ssd_total_capacity_bytes and updates MasterMetricManager via inc/dec on change. Old clients that lack this RPC simply never call it — OffloadObjectHeartbeat signature is unchanged, ensuring backward compatibility. Bug 3 — data race in UnmountLocalDiskSegment (segment.cpp) ssd_total_capacity_bytes was read without holding offloading_mutex_ while OffloadObjectHeartbeat writes it under that lock (C++ UB). Fixed by reading inside a scoped lock block, then releasing the lock before erase() to avoid unlocking an already-destroyed mutex. * Update mooncake-store/src/master_service.cpp Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- mooncake-store/include/client_service.h | 3 + mooncake-store/include/master_client.h | 3 + mooncake-store/include/master_service.h | 4 + mooncake-store/include/replica.h | 30 ++++-- mooncake-store/include/rpc_service.h | 3 + mooncake-store/include/segment.h | 1 + mooncake-store/src/client_service.cpp | 12 +++ mooncake-store/src/file_storage.cpp | 11 ++ mooncake-store/src/master_client.cpp | 14 +++ mooncake-store/src/master_service.cpp | 35 +++++++ mooncake-store/src/rpc_service.cpp | 11 ++ mooncake-store/src/segment.cpp | 14 +++ mooncake-store/tests/master_metrics_test.cpp | 100 +++++++++++++++++++ 13 files changed, 232 insertions(+), 9 deletions(-) diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index 7012b80f..d79ba05c 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -402,6 +402,9 @@ class Client { bool enable_offloading, std::unordered_map& offloading_objects); + tl::expected ReportSsdCapacity( + int64_t ssd_total_capacity_bytes); + /** * @brief Performs a batched read of multiple objects using a * high-throughput Transfer Engine. diff --git a/mooncake-store/include/master_client.h b/mooncake-store/include/master_client.h index 40ebb55e..bda524b5 100644 --- a/mooncake-store/include/master_client.h +++ b/mooncake-store/include/master_client.h @@ -349,6 +349,9 @@ class MasterClient { ErrorCode> OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading); + [[nodiscard]] tl::expected ReportSsdCapacity( + const UUID& client_id, int64_t ssd_total_capacity_bytes); + /** * @brief Adds multiple new objects to a specified client in batch. * @param keys A list of object keys (names) that were successfully diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index fc4f2c32..c1cddd27 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -439,6 +439,10 @@ class MasterService { auto OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading) -> tl::expected, ErrorCode>; + auto ReportSsdCapacity(const UUID& client_id, + int64_t ssd_total_capacity_bytes) + -> tl::expected; + /** * @brief Notifies the master that offloading of specified objects has * succeeded. diff --git a/mooncake-store/include/replica.h b/mooncake-store/include/replica.h index 0e776818..e7434c23 100644 --- a/mooncake-store/include/replica.h +++ b/mooncake-store/include/replica.h @@ -169,17 +169,25 @@ class Replica { MasterMetricManager::instance().inc_allocated_file_size(object_size); } + // local disk replica constructor Replica(UUID client_id, uint64_t object_size, std::string transport_endpoint, ReplicaStatus status) - : data_(LocalDiskReplicaData{client_id, object_size, + : id_(next_id_.fetch_add(1)), + data_(LocalDiskReplicaData{client_id, object_size, std::move(transport_endpoint)}), - status_(status) {} + status_(status), + refcnt_(0) { + MasterMetricManager::instance().inc_allocated_file_size(object_size); + } ~Replica() { - if (status_ != ReplicaStatus::UNDEFINED && is_disk_replica()) { - const auto& disk_data = std::get(data_); + if (status_ == ReplicaStatus::UNDEFINED) return; + if (is_disk_replica()) { MasterMetricManager::instance().dec_allocated_file_size( - disk_data.object_size); + std::get(data_).object_size); + } else if (is_local_disk_replica()) { + MasterMetricManager::instance().dec_allocated_file_size( + std::get(data_).object_size); } } @@ -205,10 +213,14 @@ class Replica { } // Decrement metric for the current object before overwriting. - if (status_ != ReplicaStatus::UNDEFINED && is_disk_replica()) { - const auto& disk_data = std::get(data_); - MasterMetricManager::instance().dec_allocated_file_size( - disk_data.object_size); + if (status_ != ReplicaStatus::UNDEFINED) { + if (is_disk_replica()) { + MasterMetricManager::instance().dec_allocated_file_size( + std::get(data_).object_size); + } else if (is_local_disk_replica()) { + MasterMetricManager::instance().dec_allocated_file_size( + std::get(data_).object_size); + } } id_ = src.id_; diff --git a/mooncake-store/include/rpc_service.h b/mooncake-store/include/rpc_service.h index f771774f..07670f86 100644 --- a/mooncake-store/include/rpc_service.h +++ b/mooncake-store/include/rpc_service.h @@ -145,6 +145,9 @@ class WrappedMasterService { tl::expected, ErrorCode> OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading); + tl::expected ReportSsdCapacity( + const UUID& client_id, int64_t ssd_total_capacity_bytes); + tl::expected NotifyOffloadSuccess( const UUID& client_id, const std::vector& keys, const std::vector& metadatas); diff --git a/mooncake-store/include/segment.h b/mooncake-store/include/segment.h index 0feb9633..da90ecc3 100644 --- a/mooncake-store/include/segment.h +++ b/mooncake-store/include/segment.h @@ -50,6 +50,7 @@ struct MountedSegment { struct LocalDiskSegment { mutable Mutex offloading_mutex_; bool enable_offloading; + int64_t ssd_total_capacity_bytes = 0; // last reported by client heartbeat std::unordered_map GUARDED_BY(offloading_mutex_) offloading_objects; explicit LocalDiskSegment(bool enable_offloading) diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 24335d9a..076cac61 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -2295,6 +2295,18 @@ tl::expected Client::OffloadObjectHeartbeat( return {}; } +tl::expected Client::ReportSsdCapacity( + int64_t ssd_total_capacity_bytes) { + auto response = + master_client_.ReportSsdCapacity(client_id_, ssd_total_capacity_bytes); + if (!response) { + LOG(ERROR) << "ReportSsdCapacity failed, error code is " + << response.error(); + return tl::make_unexpected(response.error()); + } + return {}; +} + tl::expected Client::BatchGetOffloadObject( const std::string& transfer_engine_addr, const std::vector& keys, diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index d75c8ac5..cefac1a1 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -246,6 +246,17 @@ tl::expected FileStorage::Init() { return mount_file_storage_result; } } + // Report configured SSD capacity to Master so it can populate + // file_total_capacity_ (the denominator in "SSD Storage: X / Y"). + // Called once at init; old Masters that lack this RPC will log an error + // but FileStorage continues normally. + if (config_.total_size_limit > 0) { + auto cap_result = client_->ReportSsdCapacity(config_.total_size_limit); + if (!cap_result) { + LOG(WARNING) << "ReportSsdCapacity failed (old Master?): " + << cap_result.error(); + } + } auto scan_meta_result = storage_backend_->ScanMeta( [this](const std::vector& keys, diff --git a/mooncake-store/src/master_client.cpp b/mooncake-store/src/master_client.cpp index 44f562cc..dfbde59c 100644 --- a/mooncake-store/src/master_client.cpp +++ b/mooncake-store/src/master_client.cpp @@ -187,6 +187,11 @@ struct RpcNameTraits<&WrappedMasterService::OffloadObjectHeartbeat> { static constexpr const char* value = "OffloadObjectHeartbeat"; }; +template <> +struct RpcNameTraits<&WrappedMasterService::ReportSsdCapacity> { + static constexpr const char* value = "ReportSsdCapacity"; +}; + template <> struct RpcNameTraits<&WrappedMasterService::NotifyOffloadSuccess> { static constexpr const char* value = "NotifyOffloadSuccess"; @@ -808,6 +813,15 @@ MasterClient::OffloadObjectHeartbeat(const UUID& client_id, return result; } +tl::expected MasterClient::ReportSsdCapacity( + const UUID& client_id, int64_t ssd_total_capacity_bytes) { + ScopedVLogTimer timer(1, "MasterClient::ReportSsdCapacity"); + timer.LogRequest("client_id=", client_id, + ", ssd_total_capacity_bytes=", ssd_total_capacity_bytes); + return invoke_rpc<&WrappedMasterService::ReportSsdCapacity, void>( + client_id, ssd_total_capacity_bytes); +} + tl::expected MasterClient::NotifyOffloadSuccess( const UUID& client_id, const std::vector& keys, const std::vector& metadatas) { diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index f748a788..5fbcb0d8 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -2173,6 +2173,41 @@ auto MasterService::OffloadObjectHeartbeat(const UUID& client_id, return {}; } +auto MasterService::ReportSsdCapacity(const UUID& client_id, + int64_t ssd_total_capacity_bytes) + -> tl::expected { + if (ssd_total_capacity_bytes < 0) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + std::shared_lock shared_lock(snapshot_mutex_); + ScopedLocalDiskSegmentAccess local_disk_segment_access = + segment_manager_.getLocalDiskSegmentAccess(); + auto& client_local_disk_segment = + local_disk_segment_access.getClientLocalDiskSegment(); + auto local_disk_segment_it = client_local_disk_segment.find(client_id); + if (local_disk_segment_it == client_local_disk_segment.end()) { + LOG(ERROR) << "Local disk segment not found with client id = " + << client_id; + return tl::make_unexpected(ErrorCode::SEGMENT_NOT_FOUND); + } + MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); + int64_t old_capacity = + local_disk_segment_it->second->ssd_total_capacity_bytes; + if (ssd_total_capacity_bytes != old_capacity) { + local_disk_segment_it->second->ssd_total_capacity_bytes = + ssd_total_capacity_bytes; + if (old_capacity > 0) { + MasterMetricManager::instance().dec_total_file_capacity( + old_capacity); + } + if (ssd_total_capacity_bytes > 0) { + MasterMetricManager::instance().inc_total_file_capacity( + ssd_total_capacity_bytes); + } + } + return {}; +} + auto MasterService::NotifyOffloadSuccess( const UUID& client_id, const std::vector& keys, const std::vector& metadatas) diff --git a/mooncake-store/src/rpc_service.cpp b/mooncake-store/src/rpc_service.cpp index 4c15549a..05a59af1 100644 --- a/mooncake-store/src/rpc_service.cpp +++ b/mooncake-store/src/rpc_service.cpp @@ -1651,6 +1651,15 @@ WrappedMasterService::OffloadObjectHeartbeat(const UUID& client_id, return result; } +tl::expected WrappedMasterService::ReportSsdCapacity( + const UUID& client_id, int64_t ssd_total_capacity_bytes) { + ScopedVLogTimer timer(1, "ReportSsdCapacity"); + timer.LogRequest("client_id=", client_id, + ", ssd_total_capacity_bytes=", ssd_total_capacity_bytes); + return master_service_.ReportSsdCapacity(client_id, + ssd_total_capacity_bytes); +} + tl::expected WrappedMasterService::NotifyOffloadSuccess( const UUID& client_id, const std::vector& keys, const std::vector& metadatas) { @@ -1754,6 +1763,8 @@ void RegisterRpcService( server.register_handler< &mooncake::WrappedMasterService::OffloadObjectHeartbeat>( &wrapped_master_service); + server.register_handler<&mooncake::WrappedMasterService::ReportSsdCapacity>( + &wrapped_master_service); server.register_handler< &mooncake::WrappedMasterService::NotifyOffloadSuccess>( &wrapped_master_service); diff --git a/mooncake-store/src/segment.cpp b/mooncake-store/src/segment.cpp index b96845eb..eaaebd21 100644 --- a/mooncake-store/src/segment.cpp +++ b/mooncake-store/src/segment.cpp @@ -276,6 +276,20 @@ ErrorCode ScopedSegmentAccess::GetClientSegments( void ScopedSegmentAccess::UnmountLocalDiskSegment(const UUID& client_id) { auto it = segment_manager_->client_local_disk_segment_.find(client_id); if (it != segment_manager_->client_local_disk_segment_.end()) { + // Hold offloading_mutex_ while reading ssd_total_capacity_bytes to + // avoid a data race with OffloadObjectHeartbeat, which writes the + // field under the same lock. Release the lock before erase() so we + // don't unlock an already-destroyed mutex (erase destroys the + // LocalDiskSegment, including its mutex). + int64_t reported_capacity = 0; + { + MutexLocker locker(&it->second->offloading_mutex_); + reported_capacity = it->second->ssd_total_capacity_bytes; + } + if (reported_capacity > 0) { + MasterMetricManager::instance().dec_total_file_capacity( + reported_capacity); + } segment_manager_->client_local_disk_segment_.erase(it); LOG(INFO) << "client_id=" << client_id << ", action=unmount_local_disk_segment"; diff --git a/mooncake-store/tests/master_metrics_test.cpp b/mooncake-store/tests/master_metrics_test.cpp index 6e110fd8..65e02cfa 100644 --- a/mooncake-store/tests/master_metrics_test.cpp +++ b/mooncake-store/tests/master_metrics_test.cpp @@ -8,6 +8,7 @@ #include #include "utils.h" +#include "master_service.h" #include "rpc_service.h" #include "types.h" #include "master_config.h" @@ -511,6 +512,105 @@ TEST_F(MasterMetricsTest, BatchRequestTest) { ASSERT_EQ(metrics.get_batch_put_start_failed_items(), 3); } +// --------------------------------------------------------------------------- +// Tests for local SSD metrics (Bug 1 fix) +// --------------------------------------------------------------------------- + +// Helper: put a key into a mem segment and immediately notify offload success, +// simulating the client-side SSD write completing. +static std::string PutKeyAndOffload(MasterService& svc, const UUID& client_id, + const std::string& segment_name, + uint64_t value_size, + const std::string& key) { + ReplicateConfig cfg; + cfg.replica_num = 1; + auto put_start = svc.PutStart(client_id, key, value_size, cfg); + if (!put_start) return ""; + svc.PutEnd(client_id, key, ReplicaType::MEMORY); + + StorageObjectMetadata meta; + meta.data_size = static_cast(value_size); + meta.transport_endpoint = "127.0.0.1:9999"; + svc.NotifyOffloadSuccess(client_id, {key}, {meta}); + return key; +} + +// Verify that creating a LocalDiskReplica (via NotifyOffloadSuccess) increments +// file_allocated_size, and that removing the key decrements it back to zero. +TEST_F(MasterMetricsTest, LocalDiskReplicaAllocatedSize) { + auto& metrics = MasterMetricManager::instance(); + MasterServiceConfig config; + config.enable_offload = true; + MasterService svc(config); + + constexpr size_t kBuf = 0x400000000; + constexpr size_t kSegSize = 64 * 1024 * 1024; + constexpr uint64_t kValueSize = 4096; + + UUID client_id = generate_uuid(); + Segment seg; + seg.id = generate_uuid(); + seg.name = "ssd_alloc_test_segment"; + seg.base = kBuf; + seg.size = kSegSize; + seg.te_endpoint = seg.name; + + ASSERT_TRUE(svc.MountSegment(seg, client_id).has_value()); + ASSERT_TRUE(svc.MountLocalDiskSegment(client_id, true).has_value()); + + const int64_t baseline = metrics.get_allocated_file_size(); + + // After NotifyOffloadSuccess, a LocalDiskReplica is created. + std::string key = PutKeyAndOffload(svc, client_id, seg.name, kValueSize, + "ssd_alloc_test_key"); + ASSERT_FALSE(key.empty()); + EXPECT_EQ(metrics.get_allocated_file_size(), baseline + kValueSize); + + // After removing the key the LocalDiskReplica is destroyed; gauge resets. + ASSERT_TRUE(svc.Remove(key).has_value()); + EXPECT_EQ(metrics.get_allocated_file_size(), baseline); +} + +// Verify that OffloadObjectHeartbeat updates total_file_capacity correctly, +// including when a client reports a changed capacity on a subsequent heartbeat. +TEST_F(MasterMetricsTest, LocalDiskSegmentCapacityHeartbeat) { + auto& metrics = MasterMetricManager::instance(); + MasterServiceConfig config; + config.enable_offload = true; + MasterService svc(config); + + constexpr size_t kBuf = 0x500000000; + constexpr size_t kSegSize = 64 * 1024 * 1024; + constexpr int64_t kCap1 = 800LL * 1024 * 1024 * 1024; // 800 GB + constexpr int64_t kCap2 = 400LL * 1024 * 1024 * 1024; // 400 GB + + UUID client_id = generate_uuid(); + Segment seg; + seg.id = generate_uuid(); + seg.name = "ssd_capacity_test_segment"; + seg.base = kBuf; + seg.size = kSegSize; + seg.te_endpoint = seg.name; + + ASSERT_TRUE(svc.MountSegment(seg, client_id).has_value()); + ASSERT_TRUE(svc.MountLocalDiskSegment(client_id, true).has_value()); + + const int64_t baseline = metrics.get_total_file_capacity(); + + // ReportSsdCapacity: client reports 800 GB. + ASSERT_TRUE(svc.ReportSsdCapacity(client_id, kCap1).has_value()); + EXPECT_EQ(metrics.get_total_file_capacity(), baseline + kCap1); + + // Client reports 400 GB (e.g. config changed). + // Gauge must be updated to reflect the new value, not double-counted. + ASSERT_TRUE(svc.ReportSsdCapacity(client_id, kCap2).has_value()); + EXPECT_EQ(metrics.get_total_file_capacity(), baseline + kCap2); + + // Idempotent: same capacity reported again — gauge must not change. + ASSERT_TRUE(svc.ReportSsdCapacity(client_id, kCap2).has_value()); + EXPECT_EQ(metrics.get_total_file_capacity(), baseline + kCap2); +} + } // namespace mooncake::test int main(int argc, char** argv) { -- 2.34.1 From bd7ed210171ffee1e187d03e78f4027d23c75be1 Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Fri, 8 May 2026 16:02:29 +0800 Subject: [PATCH 053/382] [Store] feat(store): add allocate-backed segment mount and unmount APIs (#2006) --- .../source/http-api-reference/http-service.md | 71 +++++++ .../python-api-reference/mooncake-store.md | 69 ++++++- mooncake-integration/store/store_py.cpp | 44 +++++ mooncake-store/include/real_client.h | 27 +++ mooncake-store/src/real_client.cpp | 180 +++++++++++++++++- mooncake-store/tests/pybind_client_test.cpp | 117 ++++++++++++ .../mooncake/mooncake_store_service.py | 79 ++++++++ .../tests/test_mooncake_store_service_api.py | 47 +++++ 8 files changed, 628 insertions(+), 6 deletions(-) diff --git a/docs/source/http-api-reference/http-service.md b/docs/source/http-api-reference/http-service.md index 3b2b0013..b415c262 100644 --- a/docs/source/http-api-reference/http-service.md +++ b/docs/source/http-api-reference/http-service.md @@ -260,3 +260,74 @@ curl -X POST http://localhost:8080/api/unmount_shm \ -H "Content-Type: application/json" \ -d '{"segment_ids": ["00000000-0000-0000-0000-000000000001"]}' ``` + +### `/api/mount` +Allocate memory inside the store process and mount it as one or more Mooncake +store segments. If the requested size exceeds the maximum registration size, +the service may split it and return multiple segment ids. The response includes +the actual allocated size after alignment. + +**Method**: `POST` +**Content-Type**: `application/json` + +**Request Body**: +```json +{ + "size": 16777216, + "protocol": "tcp", + "location": "" +} +``` + +**Fields**: +- `size` (integer, required): Number of bytes requested. Must be positive. +- `protocol` (string, optional): Transfer protocol. Defaults to the service + configuration protocol. +- `location` (string, optional): Device or locality hint. Defaults to an empty + string. + +**Success Response**: +```json +{ + "status": "success", + "segment_ids": ["00000000-0000-0000-0000-000000000002"], + "allocated_size": 16777216 +} +``` + +**Example**: +```bash +curl -X POST http://localhost:8080/api/mount \ + -H "Content-Type: application/json" \ + -d '{"size": 16777216, "protocol": "tcp", "location": ""}' +``` + +### `/api/unmount` +Unmount one or more segment ids previously returned by `/api/mount` and free +the memory allocated by the store process. + +**Method**: `POST` +**Content-Type**: `application/json` + +**Request Body**: +```json +{ + "segment_ids": ["00000000-0000-0000-0000-000000000002"] +} +``` + +`segment_ids` may also be provided as a single string for one segment. + +**Success Response**: +```json +{ + "status": "success" +} +``` + +**Example**: +```bash +curl -X POST http://localhost:8080/api/unmount \ + -H "Content-Type: application/json" \ + -d '{"segment_ids": ["00000000-0000-0000-0000-000000000002"]}' +``` diff --git a/docs/source/python-api-reference/mooncake-store.md b/docs/source/python-api-reference/mooncake-store.md index 5ba4af73..60352a13 100644 --- a/docs/source/python-api-reference/mooncake-store.md +++ b/docs/source/python-api-reference/mooncake-store.md @@ -1445,7 +1445,7 @@ def mount_segment( ``` **Parameters:** -- `path` (str): File path to map and mount. +- `path` (str): File or shared-memory path to map and mount. - `size` (int): Number of bytes to mount. - `offset` (int, optional): File offset in bytes. Defaults to `0`. - `protocol` (str, optional): Transfer protocol. Defaults to `"tcp"`. @@ -1502,6 +1502,73 @@ if ret != 0: --- +#### allocate_and_mount_segment() +Allocate memory inside the store process and mount it as one or more Mooncake +store segments. + +```python +def allocate_and_mount_segment( + self, + size: int, + protocol: str = "tcp", + location: str = "", +) -> dict +``` + +**Parameters:** +- `size` (int): Number of bytes requested. The allocated size may be rounded up + for alignment. +- `protocol` (str, optional): Transfer protocol. Defaults to `"tcp"`. +- `location` (str, optional): Device or locality hint. Defaults to an empty + string. + +**Returns:** +- `dict`: A result dictionary with: + - `ret` (int): Status code (0 = success, non-zero = error code) + - `segment_ids` (List[str]): Segment ids created by the mount operation + - `allocated_size` (int): Actual allocated size in bytes + +**Example:** +```python +result = store.allocate_and_mount_segment( + 16 * 1024 * 1024, + protocol="tcp", + location="", +) + +if result["ret"] == 0: + segment_ids = list(result["segment_ids"]) + allocated_size = result["allocated_size"] +``` + +The corresponding HTTP endpoints are `/api/mount` and `/api/unmount`. + +--- + +#### unmount_and_free_segment() +Unmount one or more internally allocated segments by segment id and free their +local memory. + +```python +def unmount_and_free_segment(self, segment_ids: List[str]) -> int +``` + +**Parameters:** +- `segment_ids` (List[str]): Segment ids returned by + `allocate_and_mount_segment()`. + +**Returns:** +- `int`: Status code (0 = success, non-zero = error code) + +**Example:** +```python +ret = store.unmount_and_free_segment(segment_ids) +if ret != 0: + print("Unmount and free failed:", ret) +``` + +--- + #### get_hostname() Get the hostname of the current store instance. diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index 76a4baea..9b9a734d 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -340,6 +340,43 @@ class MooncakeStorePyWrapper { return real_client->unmountSegment(segment_ids); } + py::dict allocate_and_mount_segment(size_t size, + const std::string &protocol, + const std::string &location) { + py::dict result; + result["ret"] = -1; + result["segment_ids"] = py::list(); + result["allocated_size"] = 0; + + auto real_client = std::dynamic_pointer_cast(store_); + if (!real_client) { + LOG(ERROR) << "allocate_and_mount_segment requires RealClient"; + return result; + } + std::vector segment_ids; + size_t allocated_size = 0; + int ret; + { + py::gil_scoped_release release; + ret = real_client->allocateAndMountSegment( + size, protocol, location, segment_ids, &allocated_size); + } + result["ret"] = ret; + result["segment_ids"] = py::cast(segment_ids); + result["allocated_size"] = allocated_size; + return result; + } + + int unmount_and_free_segment(const std::vector &segment_ids) { + auto real_client = std::dynamic_pointer_cast(store_); + if (!real_client) { + LOG(ERROR) << "unmount_and_free_segment requires RealClient"; + return -1; + } + py::gil_scoped_release release; + return real_client->unmountAndFreeSegment(segment_ids); + } + std::string get_tp_key_name(const std::string &base_key, int rank) const { return base_key + "_tp_" + std::to_string(rank); } @@ -1815,6 +1852,13 @@ PYBIND11_MODULE(store, m) { py::arg("protocol") = "tcp", py::arg("location") = "") .def("unmount_segment", &MooncakeStorePyWrapper::unmount_segment, py::arg("segment_ids")) + .def("allocate_and_mount_segment", + &MooncakeStorePyWrapper::allocate_and_mount_segment, + py::arg("size"), py::arg("protocol") = "tcp", + py::arg("location") = "") + .def("unmount_and_free_segment", + &MooncakeStorePyWrapper::unmount_and_free_segment, + py::arg("segment_ids")) .def("alloc_from_mem_pool", [](MooncakeStorePyWrapper &self, size_t size) { py::gil_scoped_release release; diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index c3551e37..cf84427b 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -698,12 +698,35 @@ class RealClient : public PyClient { */ int unmountSegment(const std::vector &segment_ids); + /** + * @brief Allocate memory internally and mount segments to master. + * If size > max_mr_size, it will be split into multiple chunks. + * Memory is allocated via allocate_buffer_allocator_memory. + * The actual allocated size (aligned up to Slab::kSize) is written + * to out_allocated_size if non-null. + */ + int allocateAndMountSegment(size_t size, const std::string &protocol, + const std::string &location, + std::vector &out_segment_ids, + size_t *out_allocated_size = nullptr); + + /** + * @brief Unmount segments by their ids and free locally allocated memory. + */ + int unmountAndFreeSegment(const std::vector &segment_ids); + struct MountedSegmentRecord { void *mmap_base = nullptr; size_t size = 0; std::string path; }; + struct AllocatedSegmentRecord { + void *base = nullptr; + size_t size = 0; + std::string protocol; + }; + std::unique_ptr port_binder_ = nullptr; struct SegmentDeleter { @@ -845,6 +868,10 @@ class RealClient : public PyClient { std::unordered_map mounted_segment_records_; std::mutex mounted_segment_records_mutex_; + + std::unordered_map + allocated_segment_records_; + std::mutex allocated_segment_records_mutex_; }; } // namespace mooncake diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 188cb8af..27f5f0a9 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -1010,6 +1010,18 @@ tl::expected RealClient::tearDownAll_internal() { << toString(unregister_result.error()); } } + + std::unordered_map records_to_free; + { + std::lock_guard lock(allocated_segment_records_mutex_); + records_to_free.swap(allocated_segment_records_); + } + for (const auto &entry : records_to_free) { + if (entry.second.base) { + free_memory(entry.second.protocol, entry.second.base); + } + } + // Reset all resources client_.reset(); client_buffer_allocator_.reset(); @@ -1174,6 +1186,14 @@ int RealClient::unmountSegment(const std::vector &segment_ids) { continue; } + auto it = mounted_segment_records_.find(segment_id); + if (it == mounted_segment_records_.end()) { + LOG(ERROR) << "segment_id not found in mounted records: " + << segment_id; + if (first_error == 0) first_error = -1; + continue; + } + auto result = client_->UnmountSegmentById(id); if (!result.has_value()) { LOG(ERROR) << "UnmountSegmentById failed for " << segment_id; @@ -1183,11 +1203,8 @@ int RealClient::unmountSegment(const std::vector &segment_ids) { continue; // Don't release local resources on failure } - auto it = mounted_segment_records_.find(segment_id); - if (it != mounted_segment_records_.end()) { - to_cleanup.emplace_back(segment_id, it->second); - mounted_segment_records_.erase(it); - } + to_cleanup.emplace_back(segment_id, it->second); + mounted_segment_records_.erase(it); } } @@ -1200,6 +1217,159 @@ int RealClient::unmountSegment(const std::vector &segment_ids) { return first_error; } +int RealClient::allocateAndMountSegment( + size_t size, const std::string &protocol, const std::string &location, + std::vector &out_segment_ids, size_t *out_allocated_size) { + if (!client_) { + LOG(ERROR) << "Client not initialized"; + return -1; + } + + size_t max_mr_size = globalConfig().max_mr_size; + if (max_mr_size == 0) { + LOG(ERROR) << "Invalid max_mr_size: 0"; + return -1; + } + + if (size == 0) { + LOG(ERROR) << "size is 0"; + return -1; + } + + const size_t slab_size = facebook::cachelib::Slab::kSize; + size_t page_size = sysconf(_SC_PAGESIZE); + if (max_mr_size < page_size) { + LOG(ERROR) << "max_mr_size " << max_mr_size + << " is smaller than page_size " << page_size; + return -1; + } + + size_t aligned_max_chunk = (max_mr_size / page_size) * page_size; + if (aligned_max_chunk < slab_size) { + LOG(ERROR) << "max_mr_size " << max_mr_size + << " is smaller than slab_size " << slab_size; + return -1; + } + // Round down chunk size to slab_size multiple + aligned_max_chunk = (aligned_max_chunk / slab_size) * slab_size; + + // Check overflow before aligning up to slab_size + if (size > std::numeric_limits::max() - (slab_size - 1)) { + LOG(ERROR) << "size " << size + << " overflows when aligning to slab_size"; + return -1; + } + // Round up total size to slab_size multiple + size_t aligned_total_size = + ((size + slab_size - 1) / slab_size) * slab_size; + size_t remaining = aligned_total_size; + std::vector mounted_ids; + std::vector allocated_records; + + while (remaining > 0) { + size_t chunk_size = std::min(remaining, aligned_max_chunk); + if (chunk_size == 0) break; + + void *ptr = allocate_buffer_allocator_memory(chunk_size, protocol); + if (!ptr) { + LOG(ERROR) << "allocate_buffer_allocator_memory failed for size " + << chunk_size; + break; + } + + auto result = + client_->MountSegmentAndGetId(ptr, chunk_size, protocol, location); + if (!result.has_value()) { + LOG(ERROR) << "MountSegmentAndGetId failed"; + free_memory(protocol, ptr); + break; + } + + std::string segment_id = UuidToString(result.value()); + mounted_ids.push_back(segment_id); + allocated_records.push_back({ptr, chunk_size, protocol}); + + remaining -= chunk_size; + } + + if (remaining > 0) { + for (size_t i = 0; i < mounted_ids.size(); ++i) { + UUID id; + if (StringToUuid(mounted_ids[i], id)) { + client_->UnmountSegmentById(id); + } + if (allocated_records[i].base) { + free_memory(allocated_records[i].protocol, + allocated_records[i].base); + } + } + out_segment_ids.clear(); + return -1; + } + + { + std::lock_guard lock(allocated_segment_records_mutex_); + for (size_t i = 0; i < mounted_ids.size(); ++i) { + allocated_segment_records_[mounted_ids[i]] = allocated_records[i]; + } + } + if (out_allocated_size) { + *out_allocated_size = aligned_total_size; + } + out_segment_ids = std::move(mounted_ids); + return 0; +} + +int RealClient::unmountAndFreeSegment( + const std::vector &segment_ids) { + if (!client_) { + LOG(ERROR) << "Client not initialized"; + return -1; + } + + int first_error = 0; + std::vector> to_cleanup; + { + std::lock_guard lock(allocated_segment_records_mutex_); + for (const auto &segment_id : segment_ids) { + UUID id; + if (!StringToUuid(segment_id, id)) { + LOG(ERROR) << "Invalid segment_id: " << segment_id; + if (first_error == 0) first_error = -1; + continue; + } + + auto it = allocated_segment_records_.find(segment_id); + if (it == allocated_segment_records_.end()) { + LOG(ERROR) << "segment_id not found in allocated records: " + << segment_id; + if (first_error == 0) first_error = -1; + continue; + } + + auto result = client_->UnmountSegmentById(id); + if (!result.has_value()) { + LOG(ERROR) << "UnmountSegmentById failed for " << segment_id; + if (first_error == 0) { + first_error = static_cast(result.error()); + } + continue; // Don't release local resources on failure + } + + to_cleanup.emplace_back(segment_id, it->second); + allocated_segment_records_.erase(it); + } + } + + for (auto &p : to_cleanup) { + if (p.second.base) { + free_memory(p.second.protocol, p.second.base); + } + } + + return first_error; +} + int RealClient::health_check() { if (closed_.load()) return HC_NOT_INITIALIZED; if (!client_) return HC_NOT_INITIALIZED; diff --git a/mooncake-store/tests/pybind_client_test.cpp b/mooncake-store/tests/pybind_client_test.cpp index b9bc26b6..bd4d9aad 100644 --- a/mooncake-store/tests/pybind_client_test.cpp +++ b/mooncake-store/tests/pybind_client_test.cpp @@ -6,6 +6,10 @@ #include #include #include +#include +#include +#include +#include #include "real_client.h" #include "test_server_helpers.h" @@ -63,8 +67,121 @@ class RealClientTest : public ::testing::Test { // In-proc master for tests mooncake::testing::InProcMaster master_; std::string master_address_; + + void StartMasterAndSetupClient() { + ASSERT_TRUE(master_.Start(InProcMasterConfigBuilder().build())) + << "Failed to start in-proc master"; + master_address_ = master_.master_address(); + + const std::string rdma_devices = (FLAGS_protocol == std::string("rdma")) + ? FLAGS_device_name + : std::string(""); + ASSERT_EQ(py_client_->setup_real("localhost:17813", "P2PHANDSHAKE", + 16 * 1024 * 1024, 16 * 1024 * 1024, + FLAGS_protocol, rdma_devices, + master_address_), + 0); + } + + std::string CreateTempSegmentFile(size_t size) { + std::string path = "/tmp/mooncake_real_client_segment_XXXXXX"; + int fd = mkstemp(path.data()); + EXPECT_GE(fd, 0) << "Failed to create temp segment file"; + if (fd < 0) { + return ""; + } + EXPECT_EQ(ftruncate(fd, size), 0) + << "Failed to resize temp segment file"; + close(fd); + return path; + } }; +TEST_F(RealClientTest, AllocateAndMountSegmentAlignsAndUnmounts) { + StartMasterAndSetupClient(); + + const size_t slab_size = facebook::cachelib::Slab::kSize; + std::vector segment_ids; + size_t allocated_size = 0; + + ASSERT_EQ(py_client_->allocateAndMountSegment(1, FLAGS_protocol, "", + segment_ids, &allocated_size), + 0); + EXPECT_EQ(allocated_size, slab_size); + ASSERT_FALSE(segment_ids.empty()); + EXPECT_EQ(py_client_->unmountAndFreeSegment(segment_ids), 0); + + segment_ids.clear(); + allocated_size = 0; + ASSERT_EQ( + py_client_->allocateAndMountSegment(slab_size + 1, FLAGS_protocol, "", + segment_ids, &allocated_size), + 0); + EXPECT_EQ(allocated_size, slab_size * 2); + ASSERT_FALSE(segment_ids.empty()); + EXPECT_EQ(py_client_->unmountAndFreeSegment(segment_ids), 0); +} + +TEST_F(RealClientTest, AllocateAndMountSegmentRejectsOverflowSize) { + StartMasterAndSetupClient(); + + std::vector segment_ids; + size_t allocated_size = 0; + + EXPECT_NE(py_client_->allocateAndMountSegment( + std::numeric_limits::max(), FLAGS_protocol, "", + segment_ids, &allocated_size), + 0); + EXPECT_TRUE(segment_ids.empty()); + EXPECT_EQ(allocated_size, 0); +} + +TEST_F(RealClientTest, AllocateAndMountSegmentFreesOnTearDown) { + StartMasterAndSetupClient(); + + std::vector segment_ids; + size_t allocated_size = 0; + ASSERT_EQ(py_client_->allocateAndMountSegment(1, FLAGS_protocol, "", + segment_ids, &allocated_size), + 0); + ASSERT_FALSE(segment_ids.empty()); + EXPECT_GT(allocated_size, 0); + + EXPECT_EQ(py_client_->tearDownAll(), 0); + + GLogMuter muter; + EXPECT_NE(py_client_->unmountAndFreeSegment(segment_ids), 0); +} + +TEST_F(RealClientTest, MountAndAllocateUnmountApisRejectForeignSegments) { + StartMasterAndSetupClient(); + + const size_t slab_size = facebook::cachelib::Slab::kSize; + std::vector allocated_segment_ids; + size_t allocated_size = 0; + + ASSERT_EQ( + py_client_->allocateAndMountSegment( + 1, FLAGS_protocol, "", allocated_segment_ids, &allocated_size), + 0); + ASSERT_FALSE(allocated_segment_ids.empty()); + EXPECT_NE(py_client_->unmountSegment(allocated_segment_ids), 0); + EXPECT_EQ(py_client_->unmountAndFreeSegment(allocated_segment_ids), 0); + + std::string path = CreateTempSegmentFile(slab_size); + ASSERT_FALSE(path.empty()); + + std::vector mounted_segment_ids; + ASSERT_EQ(py_client_->mountSegment(path, 0, slab_size, FLAGS_protocol, "", + mounted_segment_ids), + 0); + ASSERT_FALSE(mounted_segment_ids.empty()); + EXPECT_NE(py_client_->unmountAndFreeSegment(mounted_segment_ids), 0); + EXPECT_EQ(py_client_->unmountSegment(mounted_segment_ids), 0); + + EXPECT_EQ(std::remove(path.c_str()), 0); +} + // Test basic Put and Get operations TEST_F(RealClientTest, BasicPutGetOperations) { // Start in-proc master diff --git a/mooncake-wheel/mooncake/mooncake_store_service.py b/mooncake-wheel/mooncake/mooncake_store_service.py index ed727d61..7191ec8a 100644 --- a/mooncake-wheel/mooncake/mooncake_store_service.py +++ b/mooncake-wheel/mooncake/mooncake_store_service.py @@ -161,6 +161,8 @@ class MooncakeStoreService: web.post('/api/reconfigure', _timed_handler("RECONFIGURE", self.handle_reconfigure)), web.post('/api/mount_shm', _timed_handler("MOUNT_SHM", self.handle_mount_shm)), web.post('/api/unmount_shm', _timed_handler("UNMOUNT_SHM", self.handle_unmount_shm)), + web.post('/api/mount', _timed_handler("MOUNT", self.handle_mount)), + web.post('/api/unmount', _timed_handler("UNMOUNT", self.handle_unmount)), web.put('/api/put', _timed_handler("PUT", self.handle_put)), web.get('/api/get/{key}', _timed_handler("GET", self.handle_get)), web.get('/api/exist/{key}', _timed_handler("EXIST", self.handle_exist)), @@ -371,6 +373,83 @@ class MooncakeStoreService: content_type="application/json" ) + async def handle_mount(self, request): + try: + data = await request.json() + size = data.get("size") + protocol = data.get("protocol", self.config.protocol) + location = data.get("location", "") + + if type(size) is not int or size <= 0: + return web.Response( + status=400, + text=json.dumps({"error": "Invalid size, must be a positive integer"}), + content_type="application/json" + ) + + result = self.store.allocate_and_mount_segment(size, protocol, location) + if result["ret"] != 0: + return web.Response( + status=500, + text=json.dumps({"error": f"Allocate and mount failed, ret={result['ret']}"}), + content_type="application/json" + ) + + return web.Response( + status=200, + text=json.dumps( + { + "status": "success", + "segment_ids": list(result["segment_ids"]), + "allocated_size": result["allocated_size"], + } + ), + content_type="application/json", + ) + except Exception as e: + logging.error("MOUNT error: %s", e) + return web.Response( + status=500, + text=json.dumps({"error": str(e)}), + content_type="application/json" + ) + + async def handle_unmount(self, request): + try: + data = await request.json() + segment_ids = data.get("segment_ids", []) + if isinstance(segment_ids, str): + segment_ids = [segment_ids] + if not segment_ids: + return web.Response( + status=400, + text=json.dumps({"error": "Missing segment_ids"}), + content_type="application/json", + ) + + ret = self.store.unmount_and_free_segment(segment_ids) + if ret != 0: + return web.Response( + status=500, + text=json.dumps( + {"error": f"Unmount and free failed, ret={ret}"} + ), + content_type="application/json", + ) + + return web.Response( + status=200, + text=json.dumps({"status": "success"}), + content_type="application/json", + ) + except Exception as e: + logging.error("UNMOUNT error: %s", e) + return web.Response( + status=500, + text=json.dumps({"error": str(e)}), + content_type="application/json" + ) + async def handle_put(self, request): try: data = await request.json() diff --git a/mooncake-wheel/tests/test_mooncake_store_service_api.py b/mooncake-wheel/tests/test_mooncake_store_service_api.py index 4efe71fe..23b2f229 100644 --- a/mooncake-wheel/tests/test_mooncake_store_service_api.py +++ b/mooncake-wheel/tests/test_mooncake_store_service_api.py @@ -47,6 +47,8 @@ class FakeStore: self.unmount_calls = [] self.fail_mount = False self.unmount_failures = set() + self.allocated_mount_calls = [] + self.free_unmount_calls = [] def mount_segment(self, path, size, offset, protocol, location): self.mount_calls.append((path, size, offset, protocol, location)) @@ -71,6 +73,19 @@ class FakeStore: self.mounted.pop(segment_id, None) return 0 + def allocate_and_mount_segment(self, size, protocol, location): + self.allocated_mount_calls.append((size, protocol, location)) + segment_id = "00000000-0000-0000-0000-000000000002" + return { + "ret": 0, + "segment_ids": [segment_id], + "allocated_size": 4096, + } + + def unmount_and_free_segment(self, segment_ids): + self.free_unmount_calls.append(list(segment_ids)) + return 0 + class FakeRequest: def __init__(self, body): @@ -174,6 +189,32 @@ class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(self.service.current_mode, "prefill") self.assertEqual(self.service.last_mount_info, {}) + async def test_mount_allocates_and_frees_on_unmount(self): + mount_resp = await self.service.handle_mount( + FakeRequest({"size": 1, "protocol": "tcp", "location": "cpu:0"}) + ) + self.assertEqual(mount_resp.status, 200) + mount_body = json.loads(mount_resp.text) + self.assertEqual(mount_body["status"], "success") + self.assertEqual( + mount_body["segment_ids"], + ["00000000-0000-0000-0000-000000000002"], + ) + self.assertEqual(mount_body["allocated_size"], 4096) + self.assertEqual(self.fake_store.allocated_mount_calls, + [(1, "tcp", "cpu:0")]) + + unmount_resp = await self.service.handle_unmount( + FakeRequest({"segment_ids": mount_body["segment_ids"]}) + ) + self.assertEqual(unmount_resp.status, 200) + unmount_body = json.loads(unmount_resp.text) + self.assertEqual(unmount_body["status"], "success") + self.assertEqual( + self.fake_store.free_unmount_calls, + [["00000000-0000-0000-0000-000000000002"]], + ) + async def test_mount_shm_requires_name_and_size(self): resp = await self.service.handle_mount_shm( FakeRequest({"name": "mooncake-segment"}) @@ -190,6 +231,12 @@ class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): body = json.loads(resp.text) self.assertIn("name or size", body["error"]) + async def test_mount_requires_positive_size(self): + resp = await self.service.handle_mount(FakeRequest({"size": 0})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Invalid size", body["error"]) + async def test_unmount_shm_requires_segment_ids(self): resp = await self.service.handle_unmount_shm(FakeRequest({})) self.assertEqual(resp.status, 400) -- 2.34.1 From 32471c9b9e77efcc2cabc7ca178b626ecdbfd28f Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Fri, 8 May 2026 16:09:36 +0800 Subject: [PATCH 054/382] [Doc] feat: readme hardware (#2053) * [Misc] Add hardware summary and Copilot worktree hook * [Doc] Polish hardware support section and drop worktree hook * [Doc] Replace generated hardware badges with real logos * [Doc] Refine hardware support matrix * [Doc] Restore vendor icon strip --- README.md | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 56 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index bc4648dd..ffe3f186 100644 --- a/README.md +++ b/README.md @@ -95,11 +95,66 @@ Mooncake adds elasticity and fault tolerance support for MoE model inference, en **Tensor-Centric Ecosystem** Mooncake establishes a full-stack, Tensor-oriented AI infrastructure where Tensors serve as the fundamental data carrier. The ecosystem spans from the Transfer Engine, which accelerates Tensor data movement across heterogeneous storage (DRAM/VRAM/NVMe), to the P2P Store and Mooncake Store for distributed management of Tensor objects (e.g., Checkpoints and KVCache), up to the Mooncake Backend enabling Tensor-based elastic distributed computing. This architecture is designed to maximize Tensor processing efficiency for large-scale model inference and training. +

🖥️ Supported Hardware

+ +Mooncake supports heterogeneous accelerators, NICs, and specialized transport paths. The summary below focuses on runtime and transport coverage that is already exposed through build options, documented protocols, or dedicated examples in this repository. + +
+ + + + + + + + + + + + +
Huawei
Huawei
Cambricon
Cambricon
Moore Threads
Moore Threads
MetaX
MetaX
T-Head
T-Head
NVIDIA
NVIDIA
AMD
AMD
Alibaba Cloud
Alibaba Cloud
AWS
AWS
+
+ +#### Accelerator runtimes + +| Vendor / Platform | Hardware / Runtime | Current support in Mooncake | How it is exposed | +|-------------------|--------------------|-----------------------------|-------------------| +| Huawei Ascend | Ascend NPUs | Supported | `-DUSE_ASCEND=ON`, `-DUSE_ASCEND_DIRECT=ON`, `-DUSE_UBSHMEM=ON`, `-DUSE_ASCEND_HETEROGENEOUS=ON`; covers HCCL transport, Ascend Direct transport, UBShmem transport, and heterogeneous Ascend-GPU transport | +| Cambricon | MLU + Neuware | Supported | `-DUSE_MLU=ON`; MLU memory detection, topology discovery, and registration reuse the standard `rdma` data path | +| Moore Threads | MUSA GPUs | Supported | `-DUSE_MUSA=ON`; accelerator-aware data transfer with MUSA runtime integration | +| MetaX (Muxi) | MACA GPUs | Supported | `-DUSE_MACA=ON`; source build support through the MACA SDK | +| T-Head | PPU / Barex | Supported | T-Head PPU deployments are represented here through Barex-based transport support | +| NVIDIA | CUDA GPUs / NVLink | Supported | `-DUSE_CUDA=ON`, `-DUSE_INTRA_NVLINK=ON`, `-DUSE_MNNVL=ON`; covers CUDA memory, GPUDirect RDMA, GPUDirect Storage, intra-node NVLink, and multi-node NVLink | +| AMD | ROCm / HIP GPUs | Supported | `-DUSE_HIP=ON`; HIP transport for AMD GPU communication | + +#### Network and fabric support + +| Vendor / Fabric | Hardware / Transport | Current support in Mooncake | How it is exposed | +|-----------------|----------------------|-----------------------------|-------------------| +| Alibaba Cloud | eRDMA NICs | Supported | `rdma` data path with eRDMA devices such as `erdma_0`; the build also enables `CONFIG_ERDMA` | +| Standard RDMA ecosystem | InfiniBand / RoCE NICs | Supported | Available through the standard `rdma` protocol path with topology-aware NIC selection | +| AWS | Elastic Fabric Adapter (EFA) | Supported | `-DUSE_EFA=ON`; EFA transport built on libfabric SRD | +| Storage disaggregation | NVMe-oF | Supported | Enabled with `-DUSE_NVMEOF=ON` | +| Memory pooling | CXL | Supported | Enabled with `-DUSE_CXL=ON` | +| Baseline networking | TCP/IP | Supported | `tcp` works in all environments | + +#### Specialized transport paths + +| Transport path | Current support in Mooncake | How it is exposed | +|----------------|-----------------------------|-------------------| +| Ascend HCCL transport | Supported | Enabled by `-DUSE_ASCEND=ON`; examples use `hccl` for Ascend NPU data movement | +| Ascend Direct transport | Supported | Enabled by `-DUSE_ASCEND_DIRECT=ON`; dedicated Ascend Direct examples and docs are included | +| UBShmem transport | Supported | Enabled by `-DUSE_UBSHMEM=ON`; Transfer Engine examples accept `--protocol=ubshmem` | +| Heterogeneous Ascend transport | Supported | Enabled by `-DUSE_ASCEND_HETEROGENEOUS=ON`; used for Ascend-GPU heterogeneous transfer | +| Barex transport | Supported | Enabled by `-DUSE_BAREX=ON`; documented as the `barex` advanced transport | +| Sunrise Transport | Supported | Included here as an additional specialized transport path to reflect current hardware support positioning | +| T-Head PPU / Barex | Supported | Barex-based transport coverage is available for T-Head PPU deployments | +

🔥 Show Cases

### Use Transfer Engine Standalone ([Guide](https://kvcache-ai.github.io/Mooncake/design/transfer-engine/index.html)) -Transfer Engine is a high-performance data transfer framework. Transfer Engine provides a unified interface to transfer data from DRAM, VRAM or NVMe, while the technical details related to hardware are hidden. Transfer Engine supports multiple communication protocols including TCP, RDMA (InfiniBand/RoCEv2/eRDMA/NVIDIA GPUDirect), NVMe over Fabric (NVMe-of), NVLink, HIP, CXL, and Ascend. When built with the corresponding runtime, Transfer Engine can also detect and route accelerator memory on CUDA, MUSA, HIP, and Cambricon MLU devices. For a complete list of supported protocols and configuration guide, see the [Supported Protocols Documentation](https://kvcache-ai.github.io/Mooncake/getting_started/supported-protocols.html). +Transfer Engine is a high-performance data transfer framework. Transfer Engine provides a unified interface to transfer data from DRAM, VRAM or NVMe, while the technical details related to hardware are hidden. Transfer Engine supports multiple communication protocols including TCP, RDMA (InfiniBand/RoCEv2/eRDMA/NVIDIA GPUDirect), AWS EFA, NVMe over Fabric (NVMe-of), NVLink, HIP, Barex, CXL, and Ascend-family transports. When built with the corresponding runtime, Transfer Engine can also detect and route accelerator memory on CUDA, MUSA, HIP, MACA, Cambricon MLU, and Ascend-enabled environments. For a complete list of supported protocols and configuration guide, see the [Supported Protocols Documentation](https://kvcache-ai.github.io/Mooncake/getting_started/supported-protocols.html). #### Highlights - **Efficient use of multiple RDMA NIC devices.** Transfer Engine supports the use of multiple RDMA NIC devices to achieve the *aggregation of transfer bandwidth*. -- 2.34.1 From b34fd159b824f0e4f9250ddb510b6cee88252417 Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Fri, 8 May 2026 16:29:39 +0800 Subject: [PATCH 055/382] [TENT] Add reference counting to RdmaTask to prevent UAF (#2047) * Add reference counting to RdmaTask to prevent UAF - Convert RdmaSubBatch::task_list from value to pointer storage - Add atomic reference counting to RdmaTask with Slab allocator integration - Properly dereference tasks in freeSubBatch cleanup path - Each slice holds a reference to its parent task Author: Feng Ren * Add paired ref/deref * remove task->ref_count assignment --- .../tent/transport/rdma/rdma_transport.h | 2 +- .../tent/include/tent/transport/rdma/slice.h | 20 ++++++++++++-- .../src/transport/rdma/rdma_transport.cpp | 26 ++++++++++++------- 3 files changed, 35 insertions(+), 13 deletions(-) diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h index cabbaac4..0761d57e 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h @@ -47,7 +47,7 @@ class LocalBuffers; using RdmaContextSet = std::vector>; struct RdmaSubBatch : public Transport::SubBatch { - std::vector task_list; + std::vector task_list; std::vector slice_chain; size_t max_size; virtual size_t size() const { return task_list.size(); } diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h index 9da64daf..13272bda 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h @@ -40,6 +40,13 @@ struct RdmaSliceList { int num_slices = 0; }; +// Forward declarations +class RdmaEndPoint; +struct RdmaTask; + +using RdmaSliceStorage = Slab; +using RdmaTaskStorage = Slab; + struct RdmaTask { int num_slices; Request request; @@ -48,6 +55,16 @@ struct RdmaTask { volatile int success_slices; volatile int resolved_slices; volatile TransferStatusEnum first_error = PENDING; + + // Reference counting for UAF protection + std::atomic ref_count{0}; + + void ref() { ref_count.fetch_add(1, std::memory_order_relaxed); } + void deref() { + if (ref_count.fetch_sub(1, std::memory_order_acq_rel) == 1) { + RdmaTaskStorage::Get().deallocate(this); + } + } }; class RdmaEndPoint; @@ -81,8 +98,6 @@ struct RdmaSlice { RailMonitor* rail_monitor = nullptr; }; -using RdmaSliceStorage = Slab; - static inline void updateSliceStatus(RdmaSlice* slice, TransferStatusEnum status) { if (status == PENDING) return; @@ -102,6 +117,7 @@ static inline void updateSliceStatus(RdmaSlice* slice, if (final_st == PENDING) final_st = FAILED; __sync_bool_compare_and_swap(&task->status_word, PENDING, final_st); } + task->deref(); } } // namespace tent diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp index dad739f4..b054233b 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp @@ -293,6 +293,10 @@ Status RdmaTransport::freeSubBatch(SubBatchRef& batch) { auto rdma_batch = dynamic_cast(batch); if (!rdma_batch) return Status::InvalidArgument("Invalid RDMA sub-batch" LOC_MARK); + for (auto* task : rdma_batch->task_list) { + task->deref(); // Release batch's reference to the task + } + rdma_batch->task_list.clear(); for (auto slice : rdma_batch->slice_chain) { while (slice) { auto next = slice->next; @@ -336,12 +340,13 @@ Status RdmaTransport::submitTransferTasks( size_t max_slice_count = 64; if (type == MTYPE_CUDA || opcode == Request::WRITE) max_slice_count = 32; - rdma_batch->task_list.push_back(RdmaTask{}); - auto& task = rdma_batch->task_list.back(); - task.request = request; - task.num_slices = 0; - task.status_word = PENDING; - task.transferred_bytes = 0; + auto* task = RdmaTaskStorage::Get().allocate(); + rdma_batch->task_list.push_back(task); + task->request = request; + task->num_slices = 0; + task->status_word = PENDING; + task->transferred_bytes = 0; + task->ref(); // Batch holds a reference to the task const double merge_ratio = 0.25; uint64_t base_block = default_block_size; @@ -371,13 +376,14 @@ Status RdmaTransport::submitTransferTasks( slice->source_addr = (char*)request.source + offset; slice->target_addr = request.target_offset + offset; slice->length = length; - slice->task = &task; + slice->task = task; slice->retry_count = 0; slice->ep_weak_ptr.reset(); slice->word = PENDING; slice->next = nullptr; slice->enqueue_ts = enqueue_ts; - task.num_slices++; + task->num_slices++; + task->ref(); // Each slice holds a reference to the task offset += length; int part_id = ((enable_spray ? submit_slices : static_cast(slice_idx)) / @@ -411,8 +417,8 @@ Status RdmaTransport::getTransferStatus(SubBatchRef batch, int task_id, if (task_id < 0 || task_id >= (int)rdma_batch->task_list.size()) { return Status::InvalidArgument("Invalid task ID" LOC_MARK); } - auto& task = rdma_batch->task_list[task_id]; - status = TransferStatus{task.status_word, task.transferred_bytes}; + auto* task = rdma_batch->task_list[task_id]; + status = TransferStatus{task->status_word, task->transferred_bytes}; return Status::OK(); } -- 2.34.1 From a422a0caafbc50b3a96f968948acebe199b849aa Mon Sep 17 00:00:00 2001 From: Colors-111 <70190328+Colors-111@users.noreply.github.com> Date: Fri, 8 May 2026 18:49:16 +0800 Subject: [PATCH 056/382] [Store]Fix SSD offload in Metadata Server mode: sync ClientBuffer to metadata server (#1939) Co-authored-by: ruanzhao --- mooncake-store/include/client_service.h | 11 +++++++++++ mooncake-store/src/file_storage.cpp | 2 +- mooncake-store/src/real_client.cpp | 2 +- 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index d79ba05c..1c49c113 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -491,6 +491,17 @@ class Client { return transfer_engine_->getLocalIpAndPort(); } + /** + * @brief Get the endpoint address for segment operations. + * @return For P2PHANDSHAKE mode, returns the actual RPC endpoint (IP:Port). + * For other modes, returns the logical local hostname used for + * segment registration. + */ + [[nodiscard]] std::string GetSegmentEndpoint() { + return (metadata_connstring_ == P2PHANDSHAKE) ? GetTransportEndpoint() + : local_hostname_; + } + // Return sorted NUMA node IDs that have at least one RDMA NIC. [[nodiscard]] std::vector GetNicNumaNodes() const; diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index cefac1a1..1dff43d3 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -592,7 +592,7 @@ tl::expected FileStorage::BatchQuerySegmentSlices( tl::expected FileStorage::RegisterLocalMemory() { auto error_code = client_->RegisterLocalMemory( client_buffer_allocator_->getBase(), config_.local_buffer_size, - kWildcardLocation, false, false); + kWildcardLocation, false, true); if (!error_code) { LOG(ERROR) << "Failed to register local memory: " << error_code.error(); return error_code; diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 27f5f0a9..4afbdd32 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -4673,7 +4673,7 @@ RealClient::batch_get_offload_object(const std::vector &keys, } co_return BatchGetOffloadObjectResponse( result.value().batch_id, std::move(result.value().pointers), - client_->GetTransportEndpoint(), + client_->GetSegmentEndpoint(), file_storage_->config_.client_buffer_gc_ttl_ms); } -- 2.34.1 From 4326adbd519f83536a36b19a971359265eb46e00 Mon Sep 17 00:00:00 2001 From: Cruz Zhao Date: Fri, 8 May 2026 19:52:35 +0800 Subject: [PATCH 057/382] [engram] support engram (#1483) * support engram Signed-off-by: Cruz Zhao * add test case for engram Signed-off-by: Cruz Zhao * add docs for engram Signed-off-by: Cruz Zhao Co-authored-by: Claude Opus 4.6 --- docs/source/design/engram.md | 144 ++++++ docs/source/index.md | 3 +- mooncake-integration/CMakeLists.txt | 3 +- .../store/engram_store_py.cpp | 257 +++++++++++ mooncake-integration/store/store_py.cpp | 61 +++ mooncake-store/include/engram/engram_store.h | 83 ++++ .../include/engram/engram_store_config.h | 22 + mooncake-store/src/CMakeLists.txt | 4 +- mooncake-store/src/engram/engram_store.cpp | 322 ++++++++++++++ mooncake-store/tests/pybind_client_test.cpp | 32 +- scripts/bench_engram_store_27b.py | 281 ++++++++++++ scripts/test_engram_store.py | 419 ++++++++++++++++++ 12 files changed, 1621 insertions(+), 10 deletions(-) create mode 100644 docs/source/design/engram.md create mode 100644 mooncake-integration/store/engram_store_py.cpp create mode 100644 mooncake-store/include/engram/engram_store.h create mode 100644 mooncake-store/include/engram/engram_store_config.h create mode 100644 mooncake-store/src/engram/engram_store.cpp create mode 100644 scripts/bench_engram_store_27b.py create mode 100755 scripts/test_engram_store.py diff --git a/docs/source/design/engram.md b/docs/source/design/engram.md new file mode 100644 index 00000000..0372ee44 --- /dev/null +++ b/docs/source/design/engram.md @@ -0,0 +1,144 @@ +# EngramStore Backend + +Mooncake provides EngramStore as the storage backend for Engram embedding tables. + +The scope is intentionally narrow: + +- the caller defines the physical table layout +- the caller uploads one table per head +- the caller provides precomputed row ids with shape `[B, L, H]` +- Mooncake returns the selected rows as `[B, L, H, D]` + +Mooncake does not implement tokenizer compression, N-gram hashing, query logic, +or any other model-side Engram algorithm. + +## Current Backend Boundary + +The current implementation is intentionally conservative. It keeps EngramStore +on top of the existing Store interfaces and does not depend on: + +- transfer scatter read +- grouped transfer task +- `get_into_range` +- `batch_query` +- local direct mapping +- query cache +- remote gather control-plane changes + +Those optimizations are deferred to follow-up PRs so that the EngramStore backend can +land first as a small, reviewable unit. + +## Configuration + +`EngramStoreConfig` contains the physical layout for one EngramStore layer: + +- `table_vocab_sizes`: per-head table sizes `[N_0, N_1, ..., N_{H-1}]` +- `embedding_dim`: row width `D` + +For `layer_id`, Mooncake generates one store key per head: + +```text +engram:l{layer_id}:h{head_idx} +``` + +Each key stores a `float32` table with shape `[N_h, D]`. + +## Public Interface + +Python: + +- `EngramStore(layer_id, config, store=None)` +- `populate(embedding_buffers)` +- `lookup(row_ids)` +- `remove_from_store(force=False)` +- `get_table_vocab_sizes()` +- `get_store_keys()` +- `get_num_heads()` +- `get_embedding_dim()` + +The Python `store` argument accepts the existing `MooncakeDistributedStore` +wrapper, or `None` for metadata-only construction. + +C++: + +- constructor `EngramStore(int layer_id, const EngramStoreConfig&, std::shared_ptr)` +- `populate(...)` +- `lookup_rows(...)` +- `lookup_rows_contiguous(...)` +- `remove_from_store(...)` +- metadata getters matching the Python surface + +## Data Contract + +Populate expects one NumPy `float32` array per head: + +```text +embedding_buffers[h].shape == [N_h, D] +``` + +Lookup accepts either: + +- nested Python lists with logical shape `[B, L, H]`, or +- a contiguous NumPy `int64` array with shape `[B, L, H]` + +Lookup returns: + +```text +output.shape == [B, L, H, D] +``` + +## Populate Flow + +Populate follows the existing Store write path: + +1. validate that exactly one table is provided for each head +2. validate that every table matches `[N_h, D]` +3. verify that the target head-table keys do not already exist +4. register each embedding table buffer +5. upload all head tables with `batch_put_from(...)` +6. unregister the staging buffers + +`populate(...)` is defined as a create-only operation for one EngramStore layer. To +reuse a `layer_id`, first remove the old tables with `remove_from_store(...)`. + +If upload fails after some head tables have already been written, or if publish +finishes but post-write buffer cleanup fails, the backend best-effort removes +the keys written by the failed populate attempt before returning an error. + +## Lookup Flow + +Each lookup follows the same simplified backend flow: + +1. validate the `row_ids` shape and bounds +2. build per-head byte ranges for the requested rows +3. issue one `get_into_ranges(...)` call to materialize those rows into the output buffer + +For NumPy `row_ids`, the binding uses a contiguous fast path and builds ranges +directly from the input tensor without first converting the entire input into a +nested C++ container. + +## Validation + +The backend enforces these invariants: + +- `table_vocab_sizes` is non-empty and every entry is positive +- `embedding_dim` is positive +- `populate(...)` receives exactly one table per head +- every populated table matches `[N_h, D]` +- `lookup(...)` receives a non-empty `[B, L, H]` input +- every row id satisfies `0 <= row_ids[..., h] < N_h` + +## Validation Status + +This backend is covered by: + +- correctness tests in `scripts/test_engram_store.py` +- benchmark coverage in `scripts/bench_engram_store_27b.py` + +`scripts/test_engram_store.py` can run against an existing Mooncake deployment through +`MOONCAKE_CONFIG_PATH` / `MOONCAKE_MASTER`, or it can start a local +`mooncake_master` instance automatically for a self-contained TCP test run. + +By default, the benchmark exercises `engram_store.populate(...)` directly. Its +fallback populate paths are gated behind `ENGRAM_ALLOW_POPULATE_FALLBACK=1` so +they do not silently mask regressions in the current implementation. diff --git a/docs/source/index.md b/docs/source/index.md index 1a171d97..1d8d9702 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -111,9 +111,10 @@ design/architecture design/mooncake-store design/p2p-store design/transfer-engine/index +design/hicache-design +design/engram design/tent/overview design/tent/tebench -design/hicache-design design/conductor/conductor-architecture-design design/conductor/indexer-api-design ::: diff --git a/mooncake-integration/CMakeLists.txt b/mooncake-integration/CMakeLists.txt index 47c6ca48..379a5e03 100644 --- a/mooncake-integration/CMakeLists.txt +++ b/mooncake-integration/CMakeLists.txt @@ -13,6 +13,7 @@ endif() if(WITH_STORE) include_directories("../mooncake-store/include") include_directories("../mooncake-store/include/cachelib_memory_allocator") + include_directories("../mooncake-store/include/engram") include_directories( "../mooncake-store/include/cachelib_memory_allocator/include") @@ -97,7 +98,7 @@ endif() if(WITH_STORE) pybind11_add_module(store ${SOURCES} ${CACHE_ALLOCATOR_SOURCES} - store/store_py.cpp integration_utils.h) + store/store_py.cpp store/engram_store_py.cpp integration_utils.h) set_target_properties(store PROPERTIES INSTALL_RPATH "$ORIGIN") if(USE_ASCEND_DIRECT) target_link_libraries( diff --git a/mooncake-integration/store/engram_store_py.cpp b/mooncake-integration/store/engram_store_py.cpp new file mode 100644 index 00000000..3f434736 --- /dev/null +++ b/mooncake-integration/store/engram_store_py.cpp @@ -0,0 +1,257 @@ +#include +#include +#include + +#include + +#include "engram/engram_store.h" +#include "engram/engram_store_config.h" +#include "pyclient.h" + +namespace py = pybind11; +using namespace mooncake; +using namespace mooncake::engram; + +namespace { + +constexpr char kPyClientCapsuleName[] = "mooncake.PyClient.shared_ptr"; +constexpr char kPyClientCapsuleMethod[] = "_get_pyclient_capsule"; + +std::vector>> py_obj_to_vec3d(py::object obj) { + std::vector>> result; + for (auto batch_item : obj) { + std::vector> batch_vec; + for (auto token_item : batch_item) { + std::vector token_vec; + for (auto head_item : token_item) { + token_vec.push_back(head_item.cast()); + } + batch_vec.push_back(std::move(token_vec)); + } + result.push_back(std::move(batch_vec)); + } + return result; +} + +std::shared_ptr unwrap_pyclient_capsule(py::object capsule) { + if (capsule.is_none()) { + return nullptr; + } + + if (!PyCapsule_CheckExact(capsule.ptr())) { + throw std::runtime_error( + "store wrapper returned a non-capsule PyClient handle"); + } + + py::capsule py_client_capsule(capsule); + const char* capsule_name = py_client_capsule.name(); + if (capsule_name == nullptr || + std::strcmp(capsule_name, kPyClientCapsuleName) != 0) { + throw std::runtime_error( + "store wrapper returned an unexpected PyClient capsule type"); + } + + auto* ptr = static_cast*>( + py_client_capsule.get_pointer()); + if (ptr == nullptr) { + throw std::runtime_error( + "store wrapper returned an empty PyClient capsule"); + } + return *ptr; +} + +std::shared_ptr unwrap_store(py::object store_obj) { + if (store_obj.is_none()) { + return nullptr; + } + + try { + return store_obj.cast>(); + } catch (const py::cast_error&) { + } + + if (!py::hasattr(store_obj, kPyClientCapsuleMethod)) { + throw std::runtime_error( + "EngramStore store parameter must be a PyClient or store wrapper " + "that " + "implements _get_pyclient_capsule()"); + } + + try { + py::object capsule = store_obj.attr(kPyClientCapsuleMethod)(); + return unwrap_pyclient_capsule(capsule); + } catch (const py::error_already_set& e) { + throw std::runtime_error( + "Failed to unwrap store wrapper for EngramStore: " + + std::string(e.what())); + } +} + +py::array_t require_embedding_buffer(py::handle buf, + int64_t expected_rows, + int expected_cols) { + if (!py::isinstance(buf)) { + throw std::runtime_error("embedding_buffers must be NumPy arrays"); + } + + auto arr = + py::array_t::ensure( + buf); + if (!arr) { + throw std::runtime_error("embedding_buffers must be float32 arrays"); + } + + auto req = arr.request(); + if (req.ndim != 2) { + throw std::runtime_error("each embedding buffer must be 2D [N_h, D]"); + } + if (req.shape[0] != expected_rows || req.shape[1] != expected_cols) { + throw std::runtime_error( + "embedding buffer shape does not match " + "get_table_vocab_sizes()/get_embedding_dim()"); + } + return arr; +} + +py::array_t lookup_to_numpy( + EngramStore& self, + const std::vector>>& row_ids) { + if (row_ids.empty() || row_ids[0].empty()) { + throw std::runtime_error("row_ids must not be empty"); + } + + const int B = static_cast(row_ids.size()); + const int L = static_cast(row_ids[0].size()); + const int H = self.get_num_heads(); + const int D = self.get_embedding_dim(); + py::array_t output({B, L, H, D}); + auto out_buf = output.request(); + + int ret = + self.lookup_rows(row_ids, out_buf.ptr, out_buf.size * sizeof(float)); + if (ret != 0) { + throw std::runtime_error("EngramStore lookup failed"); + } + return output; +} + +py::array_t lookup_array_to_numpy( + EngramStore& self, + const py::array_t& + row_ids) { + auto req = row_ids.request(); + if (req.ndim != 3) { + throw std::runtime_error("row_ids array must have shape [B, L, H]"); + } + + const int B = static_cast(req.shape[0]); + const int L = static_cast(req.shape[1]); + const int H = static_cast(req.shape[2]); + if (H != self.get_num_heads()) { + throw std::runtime_error("row_ids last dimension must match num_heads"); + } + + const int D = self.get_embedding_dim(); + py::array_t output({B, L, H, D}); + auto out_buf = output.request(); + + int ret = + self.lookup_rows_contiguous(static_cast(req.ptr), B, L, + out_buf.ptr, out_buf.size * sizeof(float)); + if (ret != 0) { + throw std::runtime_error("EngramStore lookup failed"); + } + return output; +} + +} // namespace + +namespace mooncake { +namespace engram { + +void bind_engram_store(py::module& m) { + py::class_(m, "EngramStoreConfig") + .def(py::init<>()) + .def_readwrite("table_vocab_sizes", + &EngramStoreConfig::table_vocab_sizes) + .def_readwrite("embedding_dim", &EngramStoreConfig::embedding_dim); + + py::class_(m, "EngramStore") + .def("get_table_vocab_sizes", &EngramStore::get_table_vocab_sizes) + .def("get_store_keys", &EngramStore::get_store_keys) + .def("get_num_heads", &EngramStore::get_num_heads) + .def("get_embedding_dim", &EngramStore::get_embedding_dim) + .def( + "remove_from_store", + [](EngramStore& self, bool force) { + int ret = self.remove_from_store(force); + if (ret < 0) { + throw std::runtime_error("remove_from_store failed, rc=" + + std::to_string(ret)); + } + return ret; + }, + py::arg("force") = false, + "Remove all Mooncake Store tables owned by this EngramStore layer. " + "Returns the number of removed head tables; missing keys are " + "ignored.") + .def(py::init([](int layer_id, const EngramStoreConfig& cfg, + py::object store_obj) { + std::shared_ptr store = unwrap_store(store_obj); + return new EngramStore(layer_id, cfg, store); + }), + py::arg("layer_id"), py::arg("config"), + py::arg("store") = py::none()) + .def( + "lookup", + [](EngramStore& self, py::object row_ids_obj) { + if (py::isinstance(row_ids_obj)) { + auto row_ids_array = py::array_t< + int64_t, py::array::c_style | + py::array::forcecast>::ensure(row_ids_obj); + if (!row_ids_array) { + throw std::runtime_error( + "row_ids array must be convertible to int64"); + } + return lookup_array_to_numpy(self, row_ids_array); + } + return lookup_to_numpy(self, py_obj_to_vec3d(row_ids_obj)); + }, + py::arg("row_ids"), + "Lookup embeddings by precomputed row IDs. Returns [B, L, H, D].") + .def( + "populate", + [](EngramStore& self, py::list embedding_buffers) { + const std::vector vocab_sizes = + self.get_table_vocab_sizes(); + const int embed_dim = self.get_embedding_dim(); + if (static_cast(py::len(embedding_buffers)) != + vocab_sizes.size()) { + throw std::runtime_error( + "embedding_buffers size must match num_heads"); + } + + std::vector> arrays; + std::vector bufs; + std::vector sizes; + arrays.reserve(vocab_sizes.size()); + bufs.reserve(vocab_sizes.size()); + sizes.reserve(vocab_sizes.size()); + for (size_t i = 0; i < vocab_sizes.size(); ++i) { + auto arr = require_embedding_buffer( + embedding_buffers[i], vocab_sizes[i], embed_dim); + auto req = arr.request(); + arrays.push_back(arr); + bufs.push_back(req.ptr); + sizes.push_back(req.size * sizeof(float)); + } + int ret = self.populate(bufs, sizes); + if (ret != 0) { + throw std::runtime_error("populate failed"); + } + }, + py::arg("embedding_buffers")); +} + +} // namespace engram +} // namespace mooncake diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index 9b9a734d..fb13243b 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -13,9 +13,19 @@ #include "types.h" #include // for atexit +#include #include "integration_utils.h" +// Forward declaration for EngramStore bindings +namespace mooncake { +namespace engram { +void bind_engram_store(py::module &m); +} +} // namespace mooncake +#include "engram/engram_store.h" +#include "engram/engram_store_config.h" + namespace py = pybind11; namespace mooncake { @@ -1730,6 +1740,21 @@ PYBIND11_MODULE(store, m) { .def_readwrite("expert_id", &ParallelAxisSpec::expert_id) .def_readwrite("stage_id", &ParallelAxisSpec::stage_id); + auto make_pyclient_capsule = + [](const std::shared_ptr &store) -> py::object { + if (!store) { + return py::none(); + } + auto ptr = std::make_unique>(store); + std::shared_ptr *raw_ptr = ptr.get(); + py::object cap = + py::capsule(raw_ptr, "mooncake.PyClient.shared_ptr", [](void *p) { + delete static_cast *>(p); + }); + (void)ptr.release(); // Transfer ownership to capsule + return cap; + }; + py::class_(m, "TensorParallelism") .def(py::init<>()) .def_readwrite("axes", &TensorParallelismSpec::axes); @@ -1760,8 +1785,41 @@ PYBIND11_MODULE(store, m) { // Create a wrapper that exposes DistributedObjectStore with Python-specific // methods + // Helper function to extract PyClient shared_ptr from + // MooncakeStorePyWrapper This is used by EngramStore to get the underlying + // PyClient We return it as a Python capsule to avoid type registration + // issues + m.def( + "_get_pyclient_from_wrapper", + [](MooncakeStorePyWrapper &wrapper) -> py::object { + if (!wrapper.store_) { + return py::none(); + } + // Return as a capsule containing the shared_ptr + // The caller (engram_store_py.cpp) will extract it + // Use unique_ptr for RAII: if py::capsule throws, the pointer is + // freed; otherwise release() transfers ownership to the capsule. + auto ptr = + std::make_unique>(wrapper.store_); + std::shared_ptr *raw_ptr = ptr.get(); + py::object cap = py::capsule(raw_ptr, [](void *p) { + delete static_cast *>(p); + }); + (void)ptr.release(); // Transfer ownership to capsule + return cap; + }, + py::arg("wrapper"), + "Get PyClient from MooncakeDistributedStore (internal use, returns " + "capsule)"); + py::class_(m, "MooncakeDistributedStore") .def(py::init<>()) + .def( + "_get_pyclient_capsule", + [make_pyclient_capsule](MooncakeStorePyWrapper &self) + -> py::object { return make_pyclient_capsule(self.store_); }, + "Internal use: expose the underlying PyClient handle as a typed " + "capsule") .def( "setup", [](MooncakeStorePyWrapper &self, const std::string &local_hostname, @@ -2690,6 +2748,9 @@ PYBIND11_MODULE(store, m) { py::arg("node"), "Bind the current thread and memory allocation preference to the " "specified NUMA node"); + + // Add EngramStore bindings + mooncake::engram::bind_engram_store(m); } } // namespace mooncake diff --git a/mooncake-store/include/engram/engram_store.h b/mooncake-store/include/engram/engram_store.h new file mode 100644 index 00000000..b655bd60 --- /dev/null +++ b/mooncake-store/include/engram/engram_store.h @@ -0,0 +1,83 @@ +#pragma once + +#include +#include +#include +#include + +#include "engram/engram_store_config.h" + +namespace mooncake { + +class PyClient; + +namespace engram { + +/** + * Mooncake backend for Engram embedding tables. + * + * This class intentionally owns only storage-side concerns: + * - per-head table naming/layout in Mooncake Store + * - batch populate / remove + * - row-id based embedding lookup + * + * It does not implement tokenizer compression, N-gram hashing, routing, + * gating, convolution, or any other model-side Engram logic. + */ +class EngramStore { + public: + EngramStore(int layer_id, const EngramStoreConfig& config, + std::shared_ptr store = nullptr); + + ~EngramStore() = default; + + /** + * Lookup embedding rows for a batch of precomputed row IDs. + * @param row_ids [B, L, H] precomputed row IDs, where H == num_heads + * @param output [B, L, H, D] output buffer + * @param output_size Size of output buffer in bytes + * @return 0 on success, negative on error + */ + int lookup_rows( + const std::vector>>& row_ids, + void* output, size_t output_size) const; + + /** + * Fast path for contiguous row-id buffers with shape [B, L, H]. + */ + int lookup_rows_contiguous(const int64_t* row_ids, int B, int L, + void* output, size_t output_size) const; + + std::vector get_table_vocab_sizes() const; + std::vector get_store_keys() const; + int get_num_heads() const; + int get_embedding_dim() const; + + /** + * Remove all head tables owned by this EngramStore layer from Mooncake + * Store. Missing keys are ignored. Returns the number of removed tables on + * success, or a negative error code on failure. + */ + int remove_from_store(bool force = false); + + /** + * Populate Store with per-head embedding tensors. + * @param embedding_buffers Buffers for each head [N_h, D] + * @param buffer_sizes Size in bytes for each buffer + * @return 0 on success, negative on error + */ + int populate(const std::vector& embedding_buffers, + const std::vector& buffer_sizes); + + private: + int lookup_rows_flat(const int64_t* row_ids, int B, int L, void* output, + size_t output_size) const; + + std::shared_ptr store_; + std::vector table_vocab_sizes_; + int embedding_dim_; + std::vector embed_keys_; +}; + +} // namespace engram +} // namespace mooncake diff --git a/mooncake-store/include/engram/engram_store_config.h b/mooncake-store/include/engram/engram_store_config.h new file mode 100644 index 00000000..09578d4d --- /dev/null +++ b/mooncake-store/include/engram/engram_store_config.h @@ -0,0 +1,22 @@ +#pragma once + +#include +#include + +namespace mooncake { +namespace engram { + +/** + * Physical table layout for Mooncake's EngramStore backend. + * + * Mooncake does not derive table sizes from tokenizer/hash/model config. The + * caller must provide the final per-head table sizes and the per-row embedding + * width it wants Mooncake to store and query. + */ +struct EngramStoreConfig { + std::vector table_vocab_sizes = {1024}; + int embedding_dim = 64; +}; + +} // namespace engram +} // namespace mooncake diff --git a/mooncake-store/src/CMakeLists.txt b/mooncake-store/src/CMakeLists.txt index 15d4ca91..5d4c9c3c 100644 --- a/mooncake-store/src/CMakeLists.txt +++ b/mooncake-store/src/CMakeLists.txt @@ -55,7 +55,9 @@ set(MOONCAKE_STORE_SOURCES hot_standby_service.cpp standby_state_machine.cpp ha_metric_manager.cpp - store_c.cpp) + store_c.cpp + engram/engram_store.cpp +) set(EXTRA_LIBS "") diff --git a/mooncake-store/src/engram/engram_store.cpp b/mooncake-store/src/engram/engram_store.cpp new file mode 100644 index 00000000..1ece43d3 --- /dev/null +++ b/mooncake-store/src/engram/engram_store.cpp @@ -0,0 +1,322 @@ +#include "engram/engram_store.h" + +#include +#include +#include +#include +#include + +#include "pyclient.h" + +namespace mooncake { +namespace engram { + +EngramStore::EngramStore(int layer_id, const EngramStoreConfig& config, + std::shared_ptr store) + : store_(std::move(store)), + table_vocab_sizes_(config.table_vocab_sizes), + embedding_dim_(config.embedding_dim) { + if (table_vocab_sizes_.empty()) { + throw std::invalid_argument( + "EngramStoreConfig.table_vocab_sizes must not be empty"); + } + if (embedding_dim_ <= 0) { + throw std::invalid_argument( + "EngramStoreConfig.embedding_dim must be positive"); + } + for (int64_t vocab_size : table_vocab_sizes_) { + if (vocab_size <= 0) { + throw std::invalid_argument( + "EngramStoreConfig.table_vocab_sizes must contain only " + "positive values"); + } + } + + embed_keys_.reserve(table_vocab_sizes_.size()); + for (size_t h = 0; h < table_vocab_sizes_.size(); ++h) { + std::ostringstream oss; + oss << "engram:l" << layer_id << ":h" << h; + embed_keys_.push_back(oss.str()); + } +} + +int EngramStore::lookup_rows_flat(const int64_t* row_ids, int B, int L, + void* output_buffer, + size_t output_size) const { + if (store_ == nullptr || row_ids == nullptr || output_buffer == nullptr || + B <= 0 || L <= 0) { + return -1; + } + + const int num_heads = static_cast(table_vocab_sizes_.size()); + const size_t row_bytes = + static_cast(embedding_dim_) * sizeof(float); + const size_t max_size = std::numeric_limits::max(); + if (static_cast(B) > max_size / static_cast(L)) { + return -1; + } + const size_t token_count = static_cast(B) * static_cast(L); + if (token_count > max_size / static_cast(num_heads) || + token_count * static_cast(num_heads) > max_size / row_bytes) { + return -1; + } + const size_t expected_size = + token_count * static_cast(num_heads) * row_bytes; + if (output_size < expected_size) { + return -1; + } + + auto fail_lookup = [&]() { + std::memset(output_buffer, 0, expected_size); + return -1; + }; + + std::vector buffers{output_buffer}; + std::vector> all_keys(1); + std::vector>> all_dst_offsets(1); + std::vector>> all_src_offsets(1); + std::vector>> all_sizes(1); + + all_keys[0].reserve(static_cast(num_heads)); + all_dst_offsets[0].reserve(static_cast(num_heads)); + all_src_offsets[0].reserve(static_cast(num_heads)); + all_sizes[0].reserve(static_cast(num_heads)); + + for (int h = 0; h < num_heads; ++h) { + all_keys[0].push_back(embed_keys_[h]); + all_dst_offsets[0].emplace_back(); + all_src_offsets[0].emplace_back(); + all_sizes[0].emplace_back(); + all_dst_offsets[0].back().reserve(static_cast(B) * L); + all_src_offsets[0].back().reserve(static_cast(B) * L); + all_sizes[0].back().reserve(static_cast(B) * L); + } + + for (int b = 0; b < B; ++b) { + for (int l = 0; l < L; ++l) { + const size_t token_index = static_cast(b) * L + l; + const size_t row_offset = + token_index * static_cast(num_heads); + for (int h = 0; h < num_heads; ++h) { + const int64_t idx = + row_ids[row_offset + static_cast(h)]; + if (idx < 0 || idx >= table_vocab_sizes_[h]) { + return fail_lookup(); + } + all_dst_offsets[0][h].push_back( + (row_offset + static_cast(h)) * row_bytes); + all_src_offsets[0][h].push_back(static_cast(idx) * + row_bytes); + all_sizes[0][h].push_back(row_bytes); + } + } + } + + const int register_ret = + store_->register_buffer(output_buffer, expected_size); + if (register_ret != 0) { + return fail_lookup(); + } + + auto results = store_->get_into_ranges(buffers, all_keys, all_dst_offsets, + all_src_offsets, all_sizes); + const int unregister_ret = store_->unregister_buffer(output_buffer); + if (unregister_ret != 0) { + return fail_lookup(); + } + if (results.size() != 1 || + results[0].size() != static_cast(num_heads)) { + return fail_lookup(); + } + for (int h = 0; h < num_heads; ++h) { + if (results[0][h].size() != all_sizes[0][h].size()) { + return fail_lookup(); + } + for (int64_t bytes_read : results[0][h]) { + if (bytes_read != static_cast(row_bytes)) { + return fail_lookup(); + } + } + } + + return 0; +} + +int EngramStore::lookup_rows_contiguous(const int64_t* row_ids, int B, int L, + void* output_buffer, + size_t output_size) const { + return lookup_rows_flat(row_ids, B, L, output_buffer, output_size); +} + +int EngramStore::lookup_rows( + const std::vector>>& row_ids, + void* output_buffer, size_t output_size) const { + if (store_ == nullptr || output_buffer == nullptr || row_ids.empty() || + row_ids[0].empty()) { + return -1; + } + + const int B = static_cast(row_ids.size()); + const int L = static_cast(row_ids[0].size()); + const int num_heads = static_cast(table_vocab_sizes_.size()); + std::vector flat_row_ids; + flat_row_ids.reserve(static_cast(B) * L * num_heads); + for (int b = 0; b < B; ++b) { + if (static_cast(row_ids[b].size()) != L) { + return -1; + } + for (int l = 0; l < L; ++l) { + if (static_cast(row_ids[b][l].size()) != num_heads) { + return -1; + } + flat_row_ids.insert(flat_row_ids.end(), row_ids[b][l].begin(), + row_ids[b][l].end()); + } + } + + return lookup_rows_flat(flat_row_ids.data(), B, L, output_buffer, + output_size); +} + +std::vector EngramStore::get_table_vocab_sizes() const { + return table_vocab_sizes_; +} + +std::vector EngramStore::get_store_keys() const { + return embed_keys_; +} + +int EngramStore::get_num_heads() const { + return static_cast(table_vocab_sizes_.size()); +} + +int EngramStore::get_embedding_dim() const { return embedding_dim_; } + +int EngramStore::remove_from_store(bool force) { + if (store_ == nullptr) { + return static_cast(ErrorCode::INVALID_PARAMS); + } + + constexpr int kObjectNotFound = + static_cast(ErrorCode::OBJECT_NOT_FOUND); + int removed = 0; + int first_error = 0; + + for (const auto& key : embed_keys_) { + int rc = store_->remove(key, force); + if (rc == 0) { + ++removed; + continue; + } + if (rc == kObjectNotFound) { + continue; + } + if (first_error == 0) { + first_error = rc; + } + } + + return first_error != 0 ? first_error : removed; +} + +int EngramStore::populate(const std::vector& embedding_buffers, + const std::vector& buffer_sizes) { + if (store_ == nullptr) { + return -1; + } + if (embedding_buffers.size() != embed_keys_.size() || + buffer_sizes.size() != embed_keys_.size()) { + return -1; + } + + for (size_t i = 0; i < buffer_sizes.size(); ++i) { + const size_t expected = static_cast(table_vocab_sizes_[i]) * + embedding_dim_ * sizeof(float); + if (embedding_buffers[i] == nullptr || buffer_sizes[i] != expected) { + return -1; + } + } + + std::vector exists_results = store_->batchIsExist(embed_keys_); + if (exists_results.size() != embed_keys_.size()) { + LOG(ERROR) << "Failed to preflight EngramStore populate key existence"; + return -1; + } + for (size_t i = 0; i < exists_results.size(); ++i) { + const int exists = exists_results[i]; + if (exists < 0) { + LOG(ERROR) << "Failed to query EngramStore key '" << embed_keys_[i] + << "' before populate, rc=" << exists; + return -1; + } + if (exists != 0) { + LOG(ERROR) + << "EngramStore populate requires empty destination key '" + << embed_keys_[i] << "'. Remove the existing layer first."; + return -1; + } + } + + auto cleanup_registered_buffers = [&](size_t count) { + bool cleanup_failed = false; + for (size_t i = 0; i < count; ++i) { + int rc = store_->unregister_buffer(embedding_buffers[i]); + if (rc != 0) { + cleanup_failed = true; + LOG(ERROR) << "Failed to unregister embedding buffer at index " + << i << ", rc=" << rc; + } + } + return cleanup_failed; + }; + + for (size_t i = 0; i < embedding_buffers.size(); ++i) { + int ret = + store_->register_buffer(embedding_buffers[i], buffer_sizes[i]); + if (ret != 0) { + if (cleanup_registered_buffers(i)) { + LOG(ERROR) << "Failed to clean up registered embedding buffers " + "after register_buffer error"; + } + return -1; + } + } + + std::vector put_results = + store_->batch_put_from(embed_keys_, embedding_buffers, buffer_sizes); + const bool put_succeeded = + put_results.size() == embed_keys_.size() && + std::all_of(put_results.begin(), put_results.end(), + [](int result) { return result == 0; }); + + const bool unregister_failed = + cleanup_registered_buffers(embedding_buffers.size()); + + if (!put_succeeded || unregister_failed) { + const bool put_results_complete = + put_results.size() == embed_keys_.size(); + for (size_t i = 0; i < embed_keys_.size(); ++i) { + if (put_results_complete && put_results[i] != 0) { + continue; + } + int rc = store_->remove(embed_keys_[i], true); + if (rc != 0 && + rc != static_cast(ErrorCode::OBJECT_NOT_FOUND)) { + LOG(ERROR) + << "Failed to roll back partially populated EngramStore " + << "key '" << embed_keys_[i] << "', rc=" << rc; + } + } + if (unregister_failed) { + LOG(ERROR) + << "Rolling back EngramStore populate because buffer cleanup " + "failed after publish"; + } + return -1; + } + + return 0; +} + +} // namespace engram +} // namespace mooncake diff --git a/mooncake-store/tests/pybind_client_test.cpp b/mooncake-store/tests/pybind_client_test.cpp index bd4d9aad..182b3f21 100644 --- a/mooncake-store/tests/pybind_client_test.cpp +++ b/mooncake-store/tests/pybind_client_test.cpp @@ -2,13 +2,15 @@ #include #include -#include -#include -#include #include +#include #include #include #include +#include +#include +#include +#include #include #include "real_client.h" @@ -811,16 +813,31 @@ TEST_F(RealClientTest, TestCopyMoveQueryTask) { config.preferred_segment = client1_addr; ASSERT_EQ(py_client_->put(key, data_span, config), 0); + auto wait_for_task = [this](const UUID& task_id) { + const auto deadline = + std::chrono::steady_clock::now() + std::chrono::seconds(10); + while (std::chrono::steady_clock::now() < deadline) { + auto query_res = py_client_->query_task(task_id); + if (query_res.has_value() && + is_finished_status(query_res->status)) { + return query_res; + } + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + } + return py_client_->query_task(task_id); + }; + // Test create copy task from client 1 to client 2 auto copy_res = py_client_->create_copy_task(key, {client2_addr}); ASSERT_TRUE(copy_res.has_value()) << "Copy should return a task ID"; UUID copy_task_id = copy_res.value(); - // Query Copy Task - auto query_copy_res = py_client_->query_task(copy_task_id); + auto query_copy_res = wait_for_task(copy_task_id); ASSERT_TRUE(query_copy_res.has_value()) << "QueryTask should succeed"; EXPECT_EQ(query_copy_res->id, copy_task_id); EXPECT_EQ(query_copy_res->type, TaskType::REPLICA_COPY); + EXPECT_EQ(query_copy_res->status, TaskStatus::SUCCESS) + << query_copy_res->message; // Test create move task from client 1 to client 2 auto move_res = @@ -828,11 +845,12 @@ TEST_F(RealClientTest, TestCopyMoveQueryTask) { ASSERT_TRUE(move_res.has_value()) << "Move should return a task ID"; UUID move_task_id = move_res.value(); - // Query Move Task - auto query_move_res = py_client_->query_task(move_task_id); + auto query_move_res = wait_for_task(move_task_id); ASSERT_TRUE(query_move_res.has_value()) << "QueryTask should succeed"; EXPECT_EQ(query_move_res->id, move_task_id); EXPECT_EQ(query_move_res->type, TaskType::REPLICA_MOVE); + EXPECT_EQ(query_move_res->status, TaskStatus::SUCCESS) + << query_move_res->message; py_client2->tearDownAll(); } diff --git a/scripts/bench_engram_store_27b.py b/scripts/bench_engram_store_27b.py new file mode 100644 index 00000000..7a961ca3 --- /dev/null +++ b/scripts/bench_engram_store_27b.py @@ -0,0 +1,281 @@ +#!/usr/bin/env python3 +""" +EngramStore backend benchmark. + +Scenario: + - 1 token per request (L=1) + - Batch sizes: 1, 4, 16, 64, 128, 256 + - Benchmarks Mooncake's row-id lookup backend only + +Usage: + # Start master first: + # mooncake_master --default_kv_lease_ttl=500 --enable_http_metadata_server=true + + export MOONCAKE_MASTER=127.0.0.1:50051 + export MOONCAKE_TE_META_DATA_SERVER=http://127.0.0.1:8080/metadata + python scripts/bench_engram_store_27b.py + +Optional: + export ENGRAM_STORE_ALLOW_POPULATE_FALLBACK=1 +""" + +import ctypes +import os +import sys +import time +import uuid + +import numpy as np + +repo_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +build_dir = os.environ.get("MOONCAKE_BUILD_DIR", "build") +build_store = os.path.join(repo_root, build_dir, "mooncake-integration") +wheel_dir = os.path.join(repo_root, "mooncake-wheel") +for d in [build_store, wheel_dir]: + if os.path.isdir(d) and d not in sys.path: + sys.path.insert(0, d) + +try: + import store + from mooncake.mooncake_config import MooncakeConfig +except ImportError as e: + print(f"❌ Import failed: {e}") + sys.exit(1) + +BATCH_SIZES = [1, 4, 16, 64, 128, 256] +NUM_WARMUP = 10 +NUM_ITER = 50 +TABLE_VOCAB_SIZES = [50000] * 8 +EMBEDDING_DIM = 16 +ALLOW_POPULATE_FALLBACK = ( + os.environ.get("ENGRAM_STORE_ALLOW_POPULATE_FALLBACK", "0") == "1" +) + + +def create_engram_store_config(): + cfg = store.EngramStoreConfig() + cfg.table_vocab_sizes = TABLE_VOCAB_SIZES + cfg.embedding_dim = EMBEDDING_DIM + return cfg + + +def rollback_keys(store_obj, keys): + for key in keys: + rc = store_obj.remove(key, True) + if rc not in (0, -704): + print(f"Warning: failed to remove benchmark key {key}, rc={rc}") + + +def populate_store_via_cxl_segment(store_obj, keys, embedding_buffers): + cxl_path = os.environ.get("MC_CXL_DEV_PATH") + if not cxl_path: + raise RuntimeError("MC_CXL_DEV_PATH is required for CXL fallback") + + real_cxl_path = os.path.realpath(cxl_path) + base_addr = None + limit_addr = None + with open("/proc/self/maps", "r", encoding="utf-8") as fin: + for line in fin: + if real_cxl_path not in line: + continue + addr_range = line.split()[0] + start_hex, end_hex = addr_range.split("-") + base_addr = int(start_hex, 16) + limit_addr = int(end_hex, 16) + break + + if base_addr is None or limit_addr is None: + raise RuntimeError(f"failed to find CXL mapping for {real_cxl_path}") + + page_size = 4096 + aligned_sizes = [] + total_aligned_bytes = 0 + for emb in embedding_buffers: + nbytes = emb.nbytes + aligned = (nbytes + page_size - 1) // page_size * page_size + aligned_sizes.append(aligned) + total_aligned_bytes += aligned + + cursor = limit_addr - total_aligned_bytes + if cursor < base_addr: + raise RuntimeError("CXL mapping is too small for staging region") + + published_keys = [] + try: + for head_idx, emb in enumerate(embedding_buffers): + nbytes = emb.nbytes + aligned = aligned_sizes[head_idx] + if cursor + aligned > limit_addr: + raise RuntimeError("CXL mapping is too small for benchmark tables") + ctypes.memmove(cursor, emb.ctypes.data, nbytes) + key = keys[head_idx] + rc = store_obj.put_from(key, cursor, nbytes) + if rc != 0: + raise RuntimeError( + f"CXL put_from fallback failed for {key}, rc={rc}" + ) + published_keys.append(key) + cursor += aligned + except Exception: + rollback_keys(store_obj, published_keys) + raise + + +def populate_store(engram_store, store_obj): + keys = engram_store.get_store_keys() + embedding_buffers = [] + for vocab_size in engram_store.get_table_vocab_sizes(): + emb = np.random.randn(vocab_size, engram_store.get_embedding_dim()).astype(np.float32) + embedding_buffers.append(emb) + + t0 = time.perf_counter() + mode = "engram_store.populate" + try: + engram_store.populate(embedding_buffers) + except RuntimeError: + if not ALLOW_POPULATE_FALLBACK: + raise + protocol = os.environ.get("MOONCAKE_PROTOCOL", "") + if protocol == "cxl": + mode = "cxl put_from fallback" + populate_store_via_cxl_segment(store_obj, keys, embedding_buffers) + else: + mode = "store.put fallback" + published_keys = [] + try: + for head_idx, emb in enumerate(embedding_buffers): + key = keys[head_idx] + rc = store_obj.put(key, emb) + if rc != 0: + raise RuntimeError( + f"fallback populate failed for {key}, rc={rc}" + ) + published_keys.append(key) + except Exception: + rollback_keys(store_obj, published_keys) + raise + populate_ms = (time.perf_counter() - t0) * 1000 + return embedding_buffers, populate_ms, mode + + +def make_row_ids(engram_store, batch_size, seq_len): + row_ids = [] + for b in range(batch_size): + batch = [] + for l in range(seq_len): + token_rows = [] + for head, vocab_size in enumerate(engram_store.get_table_vocab_sizes()): + token_rows.append((b * seq_len + l + head) % vocab_size) + batch.append(token_rows) + row_ids.append(batch) + return row_ids + + +def run_benchmark(engram_store, batch_size, num_warmup, num_iter): + seq_len = 1 + row_ids = make_row_ids(engram_store, batch_size, seq_len) + + for _ in range(num_warmup): + engram_store.lookup(row_ids) + + total_ms_list = [] + for _ in range(num_iter): + t0 = time.perf_counter() + engram_store.lookup(row_ids) + t1 = time.perf_counter() + total_ms_list.append((t1 - t0) * 1000) + + total_ms = np.array(total_ms_list) + mean_total = np.mean(total_ms) + p50_total = np.percentile(total_ms, 50) + p99_total = np.percentile(total_ms, 99) + + tokens_per_sec = batch_size / (mean_total / 1000) + bytes_per_request = ( + batch_size + * seq_len + * engram_store.get_num_heads() + * engram_store.get_embedding_dim() + * 4 + ) + gbps = (bytes_per_request / 1e9) / (mean_total / 1000) + + return { + "batch_size": batch_size, + "mean_total_ms": mean_total, + "p50_ms": p50_total, + "p99_ms": p99_total, + "tokens_per_sec": tokens_per_sec, + "gbps": gbps, + } + + +def validate_lookup_correctness(engram_store, embedding_buffers): + row_ids = make_row_ids(engram_store, batch_size=1, seq_len=1) + output = np.asarray(engram_store.lookup(row_ids)) + + max_abs_err = 0.0 + for head_idx in range(output.shape[2]): + row_idx = row_ids[0][0][head_idx] + expected = embedding_buffers[head_idx][row_idx] + actual = output[0, 0, head_idx] + max_abs_err = max(max_abs_err, float(np.max(np.abs(actual - expected)))) + return max_abs_err + + +def main(): + print("=" * 60) + print("EngramStore Backend Benchmark") + print(" Config: 1 token, Mooncake row-id lookup only") + print(" Batch sizes: 1, 4, 16, 64, 128, 256") + print(f" Build dir: {build_dir}") + print(f" Protocol: {os.environ.get('MOONCAKE_PROTOCOL', '')}") + print(f" Allow populate fallback: {ALLOW_POPULATE_FALLBACK}") + print("=" * 60) + + config = MooncakeConfig.load_from_env() + store_obj = store.MooncakeDistributedStore() + rc = store_obj.setup( + config.local_hostname, + config.metadata_server, + config.global_segment_size, + config.local_buffer_size, + config.protocol, + config.device_name, + config.master_server_address, + ) + if rc != 0: + raise RuntimeError(f"Failed to setup Mooncake store, rc={rc}") + + engram_store = None + try: + cfg = create_engram_store_config() + layer_id = uuid.uuid4().int & 0x7FFFFFFF + engram_store = store.EngramStore(layer_id=layer_id, config=cfg, store=store_obj) + embedding_buffers, populate_ms, mode = populate_store(engram_store, store_obj) + print(f"Populate mode: {mode}, took {populate_ms:.2f} ms") + + max_abs_err = validate_lookup_correctness(engram_store, embedding_buffers) + print(f"Max abs error: {max_abs_err:.6f}") + + print("\nResults:") + print( + f"{'Batch':>8} {'Mean(ms)':>10} {'P50(ms)':>10} {'P99(ms)':>10} {'Tok/s':>12} {'GB/s':>10}" + ) + for batch_size in BATCH_SIZES: + result = run_benchmark(engram_store, batch_size, NUM_WARMUP, NUM_ITER) + print( + f"{result['batch_size']:>8} {result['mean_total_ms']:>10.3f} {result['p50_ms']:>10.3f} " + f"{result['p99_ms']:>10.3f} {result['tokens_per_sec']:>12.1f} {result['gbps']:>10.3f}" + ) + finally: + if engram_store is not None: + try: + engram_store.remove_from_store(force=True) + except Exception as exc: + print(f"Warning: failed to clean up benchmark layer: {exc}") + store_obj.close() + + +if __name__ == "__main__": + main() diff --git a/scripts/test_engram_store.py b/scripts/test_engram_store.py new file mode 100755 index 00000000..759fcd7f --- /dev/null +++ b/scripts/test_engram_store.py @@ -0,0 +1,419 @@ +#!/usr/bin/env python3 +""" +Integration tests for the Mooncake EngramStore backend. + +This suite validates: +1. metadata-only construction and store-backed construction +2. populate / lookup / remove on the simplified Store interface +3. Python-list and NumPy row-id lookup paths +4. error handling and cleanup behavior +""" + +import importlib +import os +import shutil +import socket +import subprocess +import sys +import tempfile +import time +import unittest +import urllib.error +import urllib.request +import uuid +from pathlib import Path + +import numpy as np + +REPO_ROOT = Path(__file__).resolve().parents[1] +BUILD_DIR = os.environ.get("MOONCAKE_BUILD_DIR", "build") +BUILD_STORE = REPO_ROOT / BUILD_DIR / "mooncake-integration" +WHEEL_DIR = REPO_ROOT / "mooncake-wheel" +MASTER_BINARY = REPO_ROOT / BUILD_DIR / "mooncake-store" / "src" / "mooncake_master" + +for path in (BUILD_STORE, WHEEL_DIR): + if path.is_dir() and str(path) not in sys.path: + sys.path.insert(0, str(path)) + +from mooncake.mooncake_config import MooncakeConfig + +GLOBAL_STORE = None +STORE_MODULE = None +TEST_CONFIG = None +MASTER_PROCESS = None +MASTER_LOG_PATH = None +MASTER_LOG_FILE = None + + +def import_store_module(): + if not BUILD_STORE.is_dir(): + raise ImportError( + f"{BUILD_STORE} not found. Build Mooncake with store support: " + "cd build && cmake .. -DWITH_STORE=ON && make -j 128" + ) + + store_module = importlib.import_module("store") + print(f"✅ store.so imported successfully from {BUILD_STORE}") + return store_module + + +def find_free_port(): + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: + sock.bind(("127.0.0.1", 0)) + return sock.getsockname()[1] + + +def resolve_master_binary(): + if MASTER_BINARY.is_file(): + return str(MASTER_BINARY) + + binary = shutil.which("mooncake_master") + if binary: + return binary + + raise FileNotFoundError( + "Cannot find mooncake_master. Build it first or install it into PATH." + ) + + +def wait_for_tcp_port(host, port, timeout=20.0): + deadline = time.time() + timeout + while time.time() < deadline: + try: + with socket.create_connection((host, port), timeout=0.5): + return + except OSError: + time.sleep(0.1) + raise RuntimeError(f"Timed out waiting for TCP port {host}:{port}") + + +def wait_for_metadata_server(metadata_url, timeout=20.0): + deadline = time.time() + timeout + while time.time() < deadline: + try: + with urllib.request.urlopen( + metadata_url + "?key=engram_store_test_probe", timeout=1.0 + ): + return + except urllib.error.HTTPError as exc: + if exc.code in (200, 400, 404): + return + except urllib.error.URLError: + time.sleep(0.1) + else: + return + raise RuntimeError(f"Timed out waiting for metadata server {metadata_url}") + + +def read_master_log(): + if MASTER_LOG_PATH is None or not os.path.exists(MASTER_LOG_PATH): + return "" + with open(MASTER_LOG_PATH, "r", encoding="utf-8", errors="replace") as fin: + return fin.read() + + +def start_local_master(): + global MASTER_PROCESS, MASTER_LOG_FILE, MASTER_LOG_PATH + + rpc_port = find_free_port() + http_port = find_free_port() + metrics_port = find_free_port() + master_binary = resolve_master_binary() + + fd, MASTER_LOG_PATH = tempfile.mkstemp(prefix="engram_store-master-", suffix=".log") + os.close(fd) + + MASTER_LOG_FILE = open(MASTER_LOG_PATH, "w", encoding="utf-8") + cmd = [ + master_binary, + "--default_kv_lease_ttl=500", + "--enable_http_metadata_server=true", + "--rpc_address=127.0.0.1", + f"--rpc_port={rpc_port}", + "--http_metadata_server_host=127.0.0.1", + f"--http_metadata_server_port={http_port}", + f"--metrics_port={metrics_port}", + ] + MASTER_PROCESS = subprocess.Popen( + cmd, + cwd=REPO_ROOT, + stdout=MASTER_LOG_FILE, + stderr=subprocess.STDOUT, + text=True, + ) + + metadata_url = f"http://127.0.0.1:{http_port}/metadata" + try: + wait_for_tcp_port("127.0.0.1", rpc_port) + wait_for_metadata_server(metadata_url) + except Exception: + log_output = read_master_log() + stop_local_master() + raise RuntimeError( + "Failed to start local mooncake_master for EngramStore tests.\n" + f"Log output:\n{log_output}" + ) + + print( + "✅ Started local mooncake_master for EngramStore tests at " + f"127.0.0.1:{rpc_port} with metadata {metadata_url}" + ) + return MooncakeConfig( + local_hostname="127.0.0.1", + metadata_server=metadata_url, + global_segment_size=128 * 1024 * 1024, + local_buffer_size=64 * 1024 * 1024, + protocol="tcp", + device_name="", + master_server_address=f"127.0.0.1:{rpc_port}", + ) + + +def stop_local_master(): + global MASTER_PROCESS, MASTER_LOG_FILE, MASTER_LOG_PATH + + if MASTER_PROCESS is not None: + if MASTER_PROCESS.poll() is None: + MASTER_PROCESS.terminate() + try: + MASTER_PROCESS.wait(timeout=10) + except subprocess.TimeoutExpired: + MASTER_PROCESS.kill() + MASTER_PROCESS.wait(timeout=5) + MASTER_PROCESS = None + + if MASTER_LOG_FILE is not None: + MASTER_LOG_FILE.close() + MASTER_LOG_FILE = None + + if MASTER_LOG_PATH and os.path.exists(MASTER_LOG_PATH): + os.remove(MASTER_LOG_PATH) + MASTER_LOG_PATH = None + + +def load_test_config(): + if os.getenv("MOONCAKE_CONFIG_PATH") or os.getenv("MOONCAKE_MASTER"): + print("Using Mooncake configuration from environment") + return MooncakeConfig.load_from_env() + + return start_local_master() + + +def create_store_connection(store_module, config): + store = store_module.MooncakeDistributedStore() + print( + f"[{os.getpid()}] Connecting to Mooncake Master at " + f"{config.master_server_address}..." + ) + + rc = store.setup( + config.local_hostname, + config.metadata_server, + config.global_segment_size, + config.local_buffer_size, + config.protocol, + config.device_name, + config.master_server_address, + ) + if rc != 0: + raise RuntimeError(f"Failed to setup mooncake store, error code: {rc}") + + print("✅ Store connection established") + return store + + +def setUpModule(): + global GLOBAL_STORE, STORE_MODULE, TEST_CONFIG + STORE_MODULE = import_store_module() + try: + TEST_CONFIG = load_test_config() + GLOBAL_STORE = create_store_connection(STORE_MODULE, TEST_CONFIG) + except Exception: + stop_local_master() + raise + + +def tearDownModule(): + global GLOBAL_STORE + if GLOBAL_STORE: + print("\nClosing global store connection...") + GLOBAL_STORE.close() + GLOBAL_STORE = None + stop_local_master() + + +class EngramStoreTestBase(unittest.TestCase): + def setUp(self): + if GLOBAL_STORE is None: + self.skipTest("Store not initialized") + + self.store = GLOBAL_STORE + self.EngramStore = STORE_MODULE.EngramStore + self.EngramStoreConfig = STORE_MODULE.EngramStoreConfig + self._created_engram_stores = [] + self._next_layer_id = uuid.uuid4().int & 0x7FFFFFFF + + def tearDown(self): + for engram_store in reversed(self._created_engram_stores): + try: + engram_store.remove_from_store(force=True) + except Exception as exc: + print(f"Warning: failed to clean up EngramStore test layer: {exc}") + + def create_config(self): + cfg = self.EngramStoreConfig() + cfg.table_vocab_sizes = [17, 19, 23, 29] + cfg.embedding_dim = 8 + return cfg + + def create_engram_store(self, layer_id=None, store_marker=Ellipsis): + if layer_id is None: + layer_id = self._next_layer_id + self._next_layer_id += 1 + + cfg = self.create_config() + if store_marker is Ellipsis: + engram_store = self.EngramStore(layer_id=layer_id, config=cfg, store=self.store) + self._created_engram_stores.append(engram_store) + elif store_marker is None: + engram_store = self.EngramStore(layer_id=layer_id, config=cfg) + else: + engram_store = self.EngramStore(layer_id=layer_id, config=cfg, store=store_marker) + self._created_engram_stores.append(engram_store) + return cfg, engram_store + + def make_embedding_tables(self, engram_store): + embed_dim = engram_store.get_embedding_dim() + tables = [] + for head_idx, vocab_size in enumerate(engram_store.get_table_vocab_sizes()): + base = np.arange(vocab_size * embed_dim, dtype=np.float32).reshape( + vocab_size, embed_dim + ) + tables.append(base + head_idx * 1000) + return tables + + def populate_store(self, engram_store): + tables = self.make_embedding_tables(engram_store) + engram_store.populate(tables) + return tables + + +class TestEngramStoreMetadata(EngramStoreTestBase): + def test_creation_and_metadata(self): + cfg, engram_store = self.create_engram_store() + self.assertEqual(engram_store.get_num_heads(), len(cfg.table_vocab_sizes)) + self.assertEqual(engram_store.get_embedding_dim(), cfg.embedding_dim) + self.assertEqual(engram_store.get_table_vocab_sizes(), cfg.table_vocab_sizes) + self.assertEqual(len(engram_store.get_store_keys()), len(cfg.table_vocab_sizes)) + + def test_creation_without_store_keeps_metadata_accessible(self): + layer_id = self._next_layer_id + cfg, engram_store = self.create_engram_store(layer_id=layer_id, store_marker=None) + self.assertEqual(engram_store.get_num_heads(), len(cfg.table_vocab_sizes)) + self.assertEqual(engram_store.get_embedding_dim(), cfg.embedding_dim) + self.assertEqual(engram_store.get_store_keys()[0], f"engram:l{layer_id}:h0") + + +class TestStorePopulateAndLookup(EngramStoreTestBase): + def test_populate_and_lookup_shape_with_python_lists(self): + _, engram_store = self.create_engram_store() + self.populate_store(engram_store) + + row_ids = [ + [[0, 1, 2, 3], [4, 5, 6, 7]], + [[1, 2, 3, 4], [8, 9, 10, 11]], + ] + output = engram_store.lookup(row_ids) + + expected_shape = ( + len(row_ids), + len(row_ids[0]), + engram_store.get_num_heads(), + engram_store.get_embedding_dim(), + ) + self.assertEqual(output.shape, expected_shape) + self.assertFalse(np.any(np.isnan(output))) + self.assertFalse(np.any(np.isinf(output))) + + def test_lookup_matches_stored_rows_from_numpy_ids(self): + _, engram_store = self.create_engram_store() + tables = self.populate_store(engram_store) + + row_ids = np.array( + [[[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11]]], dtype=np.int64 + ) + output = np.asarray(engram_store.lookup(row_ids)) + + for pos in range(row_ids.shape[1]): + for head in range(engram_store.get_num_heads()): + idx = row_ids[0, pos, head] + np.testing.assert_allclose(output[0, pos, head], tables[head][idx]) + + def test_lookup_list_and_numpy_paths_match(self): + _, engram_store = self.create_engram_store() + self.populate_store(engram_store) + + row_ids_list = [ + [[0, 1, 2, 3], [4, 5, 6, 7]], + [[1, 2, 3, 4], [8, 9, 10, 11]], + ] + row_ids_numpy = np.asarray(row_ids_list, dtype=np.int64) + + output_from_list = np.asarray(engram_store.lookup(row_ids_list)) + output_from_numpy = np.asarray(engram_store.lookup(row_ids_numpy)) + np.testing.assert_allclose(output_from_list, output_from_numpy) + + def test_remove_from_store(self): + _, engram_store = self.create_engram_store() + self.populate_store(engram_store) + + removed = engram_store.remove_from_store(force=True) + self.assertEqual(removed, engram_store.get_num_heads()) + + for key in engram_store.get_store_keys(): + self.assertEqual(self.store.is_exist(key), 0) + + self.assertEqual(engram_store.remove_from_store(force=True), 0) + + +class TestErrorHandling(EngramStoreTestBase): + def test_populate_rejects_reusing_existing_layer_keys(self): + _, engram_store = self.create_engram_store() + self.populate_store(engram_store) + with self.assertRaises(Exception): + engram_store.populate(self.make_embedding_tables(engram_store)) + + def test_lookup_rejects_missing_tables(self): + _, engram_store = self.create_engram_store() + with self.assertRaises(Exception): + engram_store.lookup([[[0, 1, 2, 3]]]) + + def test_populate_rejects_wrong_table_shape(self): + _, engram_store = self.create_engram_store() + tables = self.make_embedding_tables(engram_store) + tables[0] = tables[0][:-1] + with self.assertRaises(Exception): + engram_store.populate(tables) + + def test_lookup_rejects_empty_input(self): + _, engram_store = self.create_engram_store() + self.populate_store(engram_store) + with self.assertRaises(Exception): + engram_store.lookup([]) + + def test_lookup_rejects_wrong_head_dimension(self): + _, engram_store = self.create_engram_store() + self.populate_store(engram_store) + row_ids = np.zeros((1, 1, engram_store.get_num_heads() - 1), dtype=np.int64) + with self.assertRaises(Exception): + engram_store.lookup(row_ids) + + def test_lookup_rejects_out_of_range_row_id(self): + _, engram_store = self.create_engram_store() + self.populate_store(engram_store) + with self.assertRaises(Exception): + engram_store.lookup([[[999, 1, 2, 3]]]) + + +if __name__ == "__main__": + unittest.main(verbosity=2) -- 2.34.1 From fed78a7133f2d3ed2ebac1254f92ed8bc92aa4ba Mon Sep 17 00:00:00 2001 From: Jason Date: Sat, 9 May 2026 02:10:00 +0800 Subject: [PATCH 058/382] [Store] fix(rust): add missing empty check for batch_is_exist (#2045) --- mooncake-store/rust/src/store.rs | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mooncake-store/rust/src/store.rs b/mooncake-store/rust/src/store.rs index 39a47404..cb59439c 100644 --- a/mooncake-store/rust/src/store.rs +++ b/mooncake-store/rust/src/store.rs @@ -533,6 +533,9 @@ impl MooncakeStore { keys: &[&str], ) -> Result, StoreError> { let count = keys.len(); + if count == 0 { + return Ok(Vec::new()); + } let key_strings: Vec = keys .iter() .map(|k| CString::new(*k)) -- 2.34.1 From 98333ad42ee1c770324bc6b27c40ef9bcfc772df Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Sat, 9 May 2026 11:17:48 +0800 Subject: [PATCH 059/382] [Store] Fix disk replica read paths for GPU KV cache (LOCAL_DISK zero-copy, DISK temp-buf scatter) (#2004) --- mooncake-store/include/client_service.h | 15 +- mooncake-store/include/gpu_staging_utils.h | 53 ++ mooncake-store/include/real_client.h | 2 +- mooncake-store/include/transfer_task.h | 3 +- mooncake-store/src/client_service.cpp | 2 +- mooncake-store/src/real_client.cpp | 828 ++++++++++++++++++--- mooncake-store/src/transfer_task.cpp | 43 +- 7 files changed, 832 insertions(+), 114 deletions(-) diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index 1c49c113..a2e71eb5 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -420,7 +420,8 @@ class Client { const std::string& transfer_engine_addr, const std::vector& keys, const std::vector& pointers, - const std::unordered_map& batch_slices); + const std::unordered_map>& + batch_slices); /** * @brief Notifies the master that offloading of specified objects has @@ -507,6 +508,16 @@ class Client { tl::expected GetPreferredReplica( const std::vector& replica_list); + + std::unordered_set GetLocalEndpoints() const { + std::lock_guard lock(mounted_segments_mutex_); + std::unordered_set endpoints; + for (const auto& [segment_id, segment] : mounted_segments_) { + endpoints.insert(segment.te_endpoint); + } + return endpoints; + } + /** * @brief Check if local hot cache is enabled * @return true if hot cache is enabled, false otherwise @@ -691,7 +702,7 @@ class Client { std::unique_ptr transfer_submitter_; // Mutex to protect mounted_segments_ - std::mutex mounted_segments_mutex_; + mutable std::mutex mounted_segments_mutex_; std::unordered_map> mounted_segments_; /** diff --git a/mooncake-store/include/gpu_staging_utils.h b/mooncake-store/include/gpu_staging_utils.h index 6c2c4dc4..375687d0 100644 --- a/mooncake-store/include/gpu_staging_utils.h +++ b/mooncake-store/include/gpu_staging_utils.h @@ -103,5 +103,58 @@ inline void SetDevice(int device_id) { #endif } +// Copy host memory to device. Caller must have called SetDevice first. +inline bool CopyHostToDevice(void* dst, const void* src, size_t size) { +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) + return cudaMemcpy(dst, src, size, cudaMemcpyHostToDevice) == cudaSuccess; +#elif defined(USE_HIP) + return hipMemcpy(dst, src, size, hipMemcpyHostToDevice) == hipSuccess; +#elif defined(USE_ASCEND) || defined(USE_ASCEND_DIRECT) || defined(USE_UBSHMEM) + return aclrtMemcpy(dst, size, src, size, ACL_MEMCPY_HOST_TO_DEVICE) == + ACL_SUCCESS; +#else + (void)dst; + (void)src; + (void)size; + return false; +#endif +} + +// Detect whether ptr resides in host (CPU) memory. +// Used together with IsDevicePointer for safe pointer-type dispatching: +// if IsDevicePointer -> CopyHostToDevice / CopyDeviceToHost +// else if IsHostPointer -> memcpy +// else -> reject (unknown type, e.g. non-standard allocator) +// +// Pageable host memory (not tracked by CUDA runtime) is treated as host. +inline bool IsHostPointer(const void* ptr) { +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) + cudaPointerAttributes attr{}; + if (cudaPointerGetAttributes(&attr, ptr) != cudaSuccess) { + // Query failed: pageable host memory not tracked by the runtime. + cudaGetLastError(); // clear sticky error + return true; + } + return attr.type != cudaMemoryTypeDevice; +#elif defined(USE_HIP) + hipPointerAttribute_t attr{}; + if (hipPointerGetAttributes(&attr, ptr) != hipSuccess) { + hipGetLastError(); // clear sticky error + return true; + } + return attr.type != hipMemoryTypeDevice; +#elif defined(USE_ASCEND) || defined(USE_ASCEND_DIRECT) || defined(USE_UBSHMEM) + aclrtPtrAttributes attr{}; + if (aclrtPointerGetAttributes(const_cast(ptr), &attr) != + ACL_SUCCESS) { + // Query failed: likely pageable host memory not tracked by the runtime. + return true; + } + return attr.location.type != ACL_MEM_LOCATION_TYPE_DEVICE; +#else + (void)ptr; + return true; // CPU-only build: all pointers are host +#endif +} } // namespace gpu_staging } // namespace mooncake diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index cf84427b..0c6088e5 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -681,7 +681,7 @@ class RealClient : public PyClient { */ tl::expected batch_get_into_offload_object_internal( const std::string &target_rpc_service_addr, - std::unordered_map &objects); + std::unordered_map> &objects); /** * @brief Mount a shared memory file region and return segment ids. diff --git a/mooncake-store/include/transfer_task.h b/mooncake-store/include/transfer_task.h index 75a96ac0..4dbb9fc1 100644 --- a/mooncake-store/include/transfer_task.h +++ b/mooncake-store/include/transfer_task.h @@ -406,7 +406,8 @@ class TransferSubmitter { const std::string& transfer_engine_addr, const std::vector& keys, const std::vector& pointers, - const std::unordered_map& batched_slices); + const std::unordered_map>& + batched_slices); private: TransferEngine& engine_; diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 076cac61..1373f590 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -2311,7 +2311,7 @@ tl::expected Client::BatchGetOffloadObject( const std::string& transfer_engine_addr, const std::vector& keys, const std::vector& pointers, - const std::unordered_map& batch_slices) { + const std::unordered_map>& batch_slices) { auto future = transfer_submitter_->submit_batch_get_offload_object( transfer_engine_addr, keys, pointers, batch_slices); if (!future) { diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 4afbdd32..33031689 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -27,6 +27,7 @@ #include "utils.h" #include "rpc_types.h" #include "file_storage.h" +#include "gpu_staging_utils.h" #include "default_config.h" #include "shm_helper.h" #include "memory_location.h" @@ -256,6 +257,67 @@ void fill_ranged_read_results_with_error( } } } + +// Scatter host (CPU) memory to a destination that may be GPU or host. +// Returns tl::expected for use in functions returning +// tl::expected. +inline tl::expected scatter_host_to_maybe_device( + void *dst, const void *src, size_t size, const std::string &context) { + int device_id = -1; + if (gpu_staging::IsDevicePointer(dst, &device_id)) { + gpu_staging::SetDevice(device_id); + if (!gpu_staging::CopyHostToDevice(dst, src, size)) { + LOG(ERROR) << "H2D copy failed: " << context; + return tl::unexpected(ErrorCode::TRANSFER_FAIL); + } + } else if (gpu_staging::IsHostPointer(dst)) { + memcpy(dst, src, size); + } else { + LOG(ERROR) << "Unknown memory type for dst buffer: " << context; + return tl::unexpected(ErrorCode::INVALID_PARAMS); + } + return {}; +} + +// Select the best replica from a list: prefer local MEMORY, then any +// MEMORY, then LOCAL_DISK, then DISK. Master may return replicas in any +// order, so we always scan. +inline const Replica::Descriptor *SelectBestReplica( + const std::vector &replicas, + const std::unordered_set &local_endpoints) { + const Replica::Descriptor *first_memory = nullptr; + for (const auto &r : replicas) { + if (r.status != ReplicaStatus::COMPLETE) continue; + if (r.is_memory_replica()) { + if (local_endpoints.count( + r.get_memory_descriptor() + .buffer_descriptor.transport_endpoint_)) { + return &r; // local MEMORY — best case + } + if (!first_memory) first_memory = &r; + } + } + if (first_memory) return first_memory; + + const Replica::Descriptor *best = nullptr; + for (const auto &r : replicas) { + if (r.status != ReplicaStatus::COMPLETE) continue; + if (r.is_local_disk_replica()) { + best = &r; // LOCAL_DISK always overrides DISK + } else if (r.is_disk_replica() && !best) { + best = &r; + } + } + return best; +} + +// Build a QueryResult containing only the chosen replica so that +// Client::Get / Client::BatchGet (which internally call +// FindFirstCompleteReplica) cannot pick a different replica type. +inline QueryResult FilterQueryResult(const QueryResult &qr, + const Replica::Descriptor &replica) { + return QueryResult({replica}, qr.lease_timeout); +} } // namespace PyClient::~PyClient() {} @@ -2333,43 +2395,73 @@ std::shared_ptr RealClient::get_buffer_internal( return nullptr; } - const auto &res = client_->GetPreferredReplica(replica_list); - if (!res) { - LOG(ERROR) << "Empty replica list for key: " << key; + // Select best replica: prefer local MEMORY, then any MEMORY, + // then LOCAL_DISK, then DISK. + // LOCAL_DISK data is on a remote node's SSD — must use offload RPC. + // MEMORY / DISK are handled via client_->Get below. + auto local_endpoints = client_->GetLocalEndpoints(); + const auto *best_replica = SelectBestReplica(replica_list, local_endpoints); + if (!best_replica) { + LOG(ERROR) << "No usable replica for key: " << key; return nullptr; } - const auto &replica = res.value(); + const auto &replica = *best_replica; uint64_t total_length = calculate_total_size(replica); if (total_length == 0) { return nullptr; } - // Normal allocation path + // Allocate buffer auto alloc_result = client_buffer_allocator->allocate(total_length); if (!alloc_result) { LOG(ERROR) << "Failed to allocate buffer for get_buffer, key: " << key; return nullptr; } - auto &buffer_handle = *alloc_result; + auto buffer_handle = + std::make_shared(std::move(*alloc_result)); + + if (best_replica->is_local_disk_replica()) { + // LOCAL_DISK: data is on remote node's SSD. Use offload RPC. + const auto &endpoint = + best_replica->get_local_disk_descriptor().transport_endpoint; + std::unordered_map> objects; + objects.emplace( + key, std::vector{{buffer_handle->ptr(), total_length}}); + auto read_result = + batch_get_into_offload_object_internal(endpoint, objects); + if (!read_result) { + LOG(ERROR) << "SSD read failed for key '" << key + << "': " << toString(read_result.error()); + return nullptr; + } + return buffer_handle; + } + + // MEMORY / DISK: use client_->Get. FilterQueryResult ensures + // Client::Get's internal FindFirstCompleteReplica can only see + // the replica we selected, preventing accidental LOCAL_DISK picks. + if (replica.is_disk_replica() && + gpu_staging::IsDevicePointer(buffer_handle->ptr(), nullptr)) { + LOG(WARNING) << "DISK replica for key '" << key + << "' received a device pointer from the allocator; " + << "file I/O cannot write to GPU memory — read will fail. " + << "Ensure client_buffer_allocator_ returns host memory."; + } - // Create slices for the allocated buffer std::vector slices; - allocateSlices(slices, replica, buffer_handle.ptr()); - - // Get the object data - auto get_result = client_->Get(key, query_result.value(), slices); + allocateSlices(slices, replica, buffer_handle->ptr()); + auto filtered_qr = FilterQueryResult(query_result.value(), replica); + auto get_result = client_->Get(key, filtered_qr, slices); if (!get_result) { LOG(ERROR) << "Get failed for key: " << key << " with error: " << toString(get_result.error()); return nullptr; } - // Create BufferHandle with the allocated memory - // The buffer will be managed by the BufferHandle's shared_ptr - return std::make_shared(std::move(buffer_handle)); + return buffer_handle; } // Implementation of get_buffer method @@ -2585,9 +2677,18 @@ RealClient::batch_get_buffer_internal( std::unique_ptr buffer_handle; std::vector slices; }; + struct DiskKeyOp { + size_t original_index; + std::string key; + QueryResult query_result; + std::unique_ptr buffer_handle; + uint64_t total_size; + }; std::vector valid_ops; + std::vector disk_ops; valid_ops.reserve(keys.size()); + auto local_endpoints = client_->GetLocalEndpoints(); for (size_t i = 0; i < keys.size(); ++i) { const auto &key = keys[i]; @@ -2606,7 +2707,15 @@ RealClient::batch_get_buffer_internal( continue; } - const auto &replica = query_result_values.replicas[0]; + // Select best replica: prefer local MEMORY, then any MEMORY, + // then LOCAL_DISK, then DISK. + const auto *best_replica = + SelectBestReplica(query_result_values.replicas, local_endpoints); + if (!best_replica) { + LOG(ERROR) << "No usable replica for key: " << key; + continue; + } + const auto replica = *best_replica; uint64_t total_size = calculate_total_size(replica); if (total_size == 0) { continue; @@ -2625,43 +2734,120 @@ RealClient::batch_get_buffer_internal( std::vector slices; allocateSlices(slices, replica, buffer_handle->ptr()); - valid_ops.emplace_back( - KeyOp{.original_index = i, - .key = key, - .query_result = std::move(query_result_values), - .buffer_handle = std::move(buffer_handle), - .slices = std::move(slices)}); + if (replica.is_local_disk_replica()) { + // LOCAL_DISK: buffer is allocated and registered via + // client_buffer_allocator_, route to SSD RPC path below. + disk_ops.emplace_back(DiskKeyOp{ + .original_index = i, + .key = key, + .query_result = FilterQueryResult(query_result_values, replica), + .buffer_handle = std::move(buffer_handle), + .total_size = total_size}); + continue; + } + + // DISK replicas use storage_backend::vector_read (file I/O) which + // can only write to CPU-addressable memory. If the allocator ever + // returns device memory for DISK, the read will silently fail. + if (replica.is_disk_replica() && + gpu_staging::IsDevicePointer(buffer_handle->ptr(), nullptr)) { + LOG(WARNING) + << "DISK replica for key '" << key + << "' received a device pointer from the allocator; " + << "file I/O cannot write to GPU memory — read will fail. " + << "Ensure client_buffer_allocator_ returns host memory."; + } + valid_ops.emplace_back(KeyOp{ + .original_index = i, + .key = key, + .query_result = FilterQueryResult(query_result_values, replica), + .buffer_handle = std::move(buffer_handle), + .slices = std::move(slices)}); } - if (valid_ops.empty()) { + if (valid_ops.empty() && disk_ops.empty()) { return final_results; } - // 3. Execute batch get - std::vector batch_keys; - std::vector batch_query_results; - std::unordered_map> batch_slices; - batch_keys.reserve(valid_ops.size()); - batch_query_results.reserve(valid_ops.size()); + // 3. Execute batch get for memory/disk replicas + if (!valid_ops.empty()) { + std::vector batch_keys; + std::vector batch_query_results; + std::unordered_map> batch_slices; + batch_keys.reserve(valid_ops.size()); + batch_query_results.reserve(valid_ops.size()); - for (auto &op : valid_ops) { - batch_keys.push_back(op.key); - batch_query_results.push_back(op.query_result); - batch_slices[op.key] = op.slices; + for (auto &op : valid_ops) { + batch_keys.push_back(op.key); + batch_query_results.push_back(op.query_result); + batch_slices[op.key] = op.slices; + } + + auto batch_get_results = + client_->BatchGet(batch_keys, batch_query_results, batch_slices); + + // 4. Process results and create BufferHandles + for (size_t i = 0; i < valid_ops.size(); ++i) { + if (batch_get_results[i]) { + auto &op = valid_ops[i]; + final_results[op.original_index] = + std::make_shared( + std::move(*op.buffer_handle)); + } else { + LOG(ERROR) << "BatchGet failed for key '" << valid_ops[i].key + << "': " << toString(batch_get_results[i].error()); + } + } } - auto batch_get_results = - client_->BatchGet(batch_keys, batch_query_results, batch_slices); + // 5. Execute batch get for LOCAL_DISK replicas via SSD RPC + if (!disk_ops.empty()) { + // Group by transport endpoint + std::unordered_map>> + offload_objects; + // Build key -> disk_ops index for result lookup + std::unordered_map disk_key_to_idx; - // 4. Process results and create BufferHandles - for (size_t i = 0; i < valid_ops.size(); ++i) { - if (batch_get_results[i]) { - auto &op = valid_ops[i]; - final_results[op.original_index] = - std::make_shared(std::move(*op.buffer_handle)); - } else { - LOG(ERROR) << "BatchGet failed for key '" << valid_ops[i].key - << "': " << toString(batch_get_results[i].error()); + for (size_t idx = 0; idx < disk_ops.size(); ++idx) { + auto &op = disk_ops[idx]; + // Find the LOCAL_DISK replica — replicas may be in any order. + const Replica::Descriptor *replica_ptr = nullptr; + for (const auto &r : op.query_result.replicas) { + if (r.is_local_disk_replica()) { + replica_ptr = &r; + break; + } + } + if (!replica_ptr) { + LOG(ERROR) << "No LOCAL_DISK replica found for key: " << op.key; + continue; + } + const auto &replica = *replica_ptr; + offload_objects[replica.get_local_disk_descriptor() + .transport_endpoint] + .emplace(op.key, std::vector{ + {op.buffer_handle->ptr(), op.total_size}}); + disk_key_to_idx[op.key] = idx; + } + + for (auto &[endpoint, objects] : offload_objects) { + if (objects.empty()) continue; + auto read_result = + batch_get_into_offload_object_internal(endpoint, objects); + for (auto &[key, slices] : objects) { + auto idx_it = disk_key_to_idx.find(key); + if (idx_it == disk_key_to_idx.end()) continue; + auto &op = disk_ops[idx_it->second]; + if (read_result) { + final_results[op.original_index] = + std::make_shared( + std::move(*op.buffer_handle)); + } else { + LOG(ERROR) << "SSD read failed for key '" << key + << "': " << toString(read_result.error()); + } + } } } @@ -2772,14 +2958,17 @@ RealClient::resolve_ranged_read_metadata(const std::string &key) { return tl::unexpected(ErrorCode::INVALID_PARAMS); } - const auto &res = client_->GetPreferredReplica(replica_list); - if (!res) { - LOG(ERROR) << "Internal error: replica_list is empty"; - return tl::unexpected(ErrorCode::INVALID_PARAMS); + // Select best replica: prefer local MEMORY, then any MEMORY, + // then LOCAL_DISK, then DISK. + auto local_endpoints = client_->GetLocalEndpoints(); + const auto *best_replica = SelectBestReplica(replica_list, local_endpoints); + if (!best_replica) { + LOG(ERROR) << "No usable replica for key: " << key; + return tl::unexpected(ErrorCode::INVALID_REPLICA); } auto query_value = std::move(query_result.value()); - auto replica = res.value(); + auto replica = *best_replica; return RangedReadMetadata{.query_result = std::move(query_value), .replica = std::move(replica), .total_size = calculate_total_size(replica)}; @@ -2807,11 +2996,57 @@ tl::expected RealClient::execute_ranged_read( } if (src_offset == 0 && size == total_size) { + // LOCAL_DISK full-object read: destination buffer is passed + // directly to SSD RPC (same pattern as single-buffer batch_get_into; + // GPU buffers are pre-registered by vLLM via register_buffer). + if (replica.is_local_disk_replica()) { + const auto &endpoint = + replica.get_local_disk_descriptor().transport_endpoint; + std::unordered_map> objects; + objects.emplace( + key, std::vector{ + {static_cast(buffer) + dst_offset, size}}); + auto result = + batch_get_into_offload_object_internal(endpoint, objects); + if (!result) return tl::unexpected(result.error()); + return static_cast(total_size); + } + + if (replica.is_disk_replica()) { + // DISK full read: local file I/O (vector_read) cannot write to + // GPU memory. Use temp CPU buffer, then scatter to dst. + auto alloc_result = client_buffer_allocator_->allocate(total_size); + if (!alloc_result) { + LOG(ERROR) << "Failed to allocate temp buffer for DISK full " + << "read, key: " << key << ", size: " << total_size; + return tl::unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + } + BufferHandle tmp_handle(std::move(*alloc_result)); + std::vector tmp_slices; + allocateSlices(tmp_slices, replica, tmp_handle.ptr()); + auto filtered_qr = FilterQueryResult(query_result, replica); + auto get_result = client_->Get(key, filtered_qr, tmp_slices); + if (!get_result) { + LOG(ERROR) << "DISK Get failed for key: " << key + << " with error: " << toString(get_result.error()); + return tl::unexpected(get_result.error()); + } + void *dst = static_cast(buffer) + dst_offset; + const void *src = tmp_handle.ptr(); + if (auto r = scatter_host_to_maybe_device( + dst, src, total_size, "DISK full read, key: " + key); + !r) { + return tl::unexpected(r.error()); + } + return static_cast(total_size); + } + std::vector slices; allocateSlices(slices, replica, static_cast(buffer) + dst_offset); - auto get_result = client_->Get(key, query_result, slices); + auto filtered_qr = FilterQueryResult(query_result, replica); + auto get_result = client_->Get(key, filtered_qr, slices); if (!get_result) { LOG(ERROR) << "Get failed for key: " << key << " with error: " << toString(get_result.error()); @@ -2820,8 +3055,75 @@ tl::expected RealClient::execute_ranged_read( return static_cast(total_size); } + // Partial disk read: allocate temp CPU buffer, invoke read_op to + // fill it, then scatter [src_offset, src_offset+size) to dst. + // + // buf_size controls how much to allocate / read. DISK must use + // total_size (allocateSlices requires full-object slices). + // LOCAL_DISK can use src_offset + size (offload RPC transfers + // sequentially from remote offset 0). + auto partial_disk_read = + [&](auto &&read_op, + size_t buf_size) -> tl::expected { + auto alloc_result = client_buffer_allocator_->allocate(buf_size); + if (!alloc_result) { + LOG(ERROR) << "Failed to allocate temp buffer for ranged disk " + << "read, key: " << key << ", size: " << buf_size; + return tl::unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + } + BufferHandle tmp_handle(std::move(*alloc_result)); + auto read_result = read_op(tmp_handle.ptr()); + if (!read_result) return tl::unexpected(read_result.error()); + void *dst = static_cast(buffer) + dst_offset; + const void *src = + static_cast(tmp_handle.ptr()) + src_offset; + if (auto r = scatter_host_to_maybe_device( + dst, src, size, "ranged disk read, key: " + key); + !r) { + return tl::unexpected(r.error()); + } + return static_cast(size); + }; + + if (replica.is_local_disk_replica()) { + // LOCAL_DISK: offload RPC transfers sequentially from remote offset + // 0, so we only need src_offset + size bytes (not total_size). + return partial_disk_read( + [&](void *tmp_buf) -> tl::expected { + const auto &endpoint = + replica.get_local_disk_descriptor().transport_endpoint; + std::unordered_map> objects; + objects.emplace( + key, std::vector{{static_cast(tmp_buf), + src_offset + size}}); + return batch_get_into_offload_object_internal(endpoint, + objects); + }, + src_offset + size); + } + + if (replica.is_disk_replica()) { + // DISK: client_->Get + allocateSlices requires full-object slices, + // so we must allocate total_size. + return partial_disk_read( + [&](void *tmp_buf) -> tl::expected { + std::vector tmp_slices; + allocateSlices(tmp_slices, replica, tmp_buf); + auto filtered_qr = FilterQueryResult(query_result, replica); + auto get_result = client_->Get(key, filtered_qr, tmp_slices); + if (!get_result) { + LOG(ERROR) + << "DISK Get failed for key: " << key + << " with error: " << toString(get_result.error()); + return tl::unexpected(get_result.error()); + } + return {}; + }, + total_size); + } + if (!replica.is_memory_replica()) { - LOG(ERROR) << "ranged reads only support memory replicas"; + LOG(ERROR) << "ranged reads only support memory/disk replicas"; return tl::unexpected(ErrorCode::INVALID_REPLICA); } @@ -3871,11 +4173,20 @@ RealClient::batch_get_into_internal(const std::vector &keys, std::vector slices; uint64_t total_size; }; + struct DiskKeyInfo { + std::string key; + size_t original_index; + QueryResult query_result; + void *dst_buffer; + uint64_t total_size; + }; std::vector valid_operations; std::unordered_map valid_local_disk_operations; + std::vector disk_operations; valid_operations.reserve(num_keys); + auto local_endpoints = client_->GetLocalEndpoints(); for (size_t i = 0; i < num_keys; ++i) { const auto &key = keys[i]; @@ -3899,8 +4210,18 @@ RealClient::batch_get_into_internal(const std::vector &keys, continue; } + // Select best replica: prefer local MEMORY, then any MEMORY, + // then LOCAL_DISK, then DISK. + const auto *best_replica = + SelectBestReplica(query_result_values.replicas, local_endpoints); + if (!best_replica) { + LOG(ERROR) << "No usable replica for key: " << key; + results[i] = tl::unexpected(ErrorCode::INVALID_REPLICA); + continue; + } + // Calculate required buffer size - const auto &replica = query_result_values.replicas[0]; + const auto replica = *best_replica; uint64_t total_size = calculate_total_size(replica); // Validate buffer capacity @@ -3912,12 +4233,9 @@ RealClient::batch_get_into_internal(const std::vector &keys, continue; } - // Create slices for this key's buffer - std::vector key_slices; - allocateSlices(key_slices, replica, buffers[i]); - - if (query_result_values.replicas.size() == 1 && - query_result_values.replicas.at(0).is_local_disk_replica()) { + if (replica.is_local_disk_replica()) { + std::vector key_slices; + allocateSlices(key_slices, replica, buffers[i]); valid_local_disk_operations.emplace( key, ValidKeyInfo{.key = key, @@ -3928,11 +4246,25 @@ RealClient::batch_get_into_internal(const std::vector &keys, results[i] = static_cast(total_size); continue; } - // Store operation info for batch processing + if (replica.is_disk_replica()) { + // DISK: file I/O (vector_read) cannot write to user GPU buffer. + // Defer — allocate CPU temp buffer, BatchGet, then scatter. + disk_operations.emplace_back( + DiskKeyInfo{.key = key, + .original_index = i, + .query_result = std::move(query_result_values), + .dst_buffer = buffers[i], + .total_size = total_size}); + results[i] = static_cast(total_size); + continue; + } + // MEMORY: RDMA directly to user buffer. + std::vector key_slices; + allocateSlices(key_slices, replica, buffers[i]); valid_operations.push_back( {.key = key, .original_index = i, - .query_result = std::move(query_result_values), + .query_result = FilterQueryResult(query_result_values, replica), .slices = std::move(key_slices), .total_size = total_size}); @@ -3941,7 +4273,8 @@ RealClient::batch_get_into_internal(const std::vector &keys, } // Early return if no valid operations - if (valid_operations.empty() && valid_local_disk_operations.empty()) { + if (valid_operations.empty() && valid_local_disk_operations.empty() && + disk_operations.empty()) { return results; } @@ -3976,16 +4309,103 @@ RealClient::batch_get_into_internal(const std::vector &keys, } } + // ---- DISK replicas: BatchGet into CPU temp buffers, then scatter ---- + if (!disk_operations.empty()) { + std::vector disk_batch_keys; + std::vector disk_batch_qrs; + std::unordered_map> disk_batch_slices; + std::vector disk_batch_indices; + std::unordered_map> + disk_temp_handles; + + for (size_t di = 0; di < disk_operations.size(); ++di) { + auto &op = disk_operations[di]; + // Find the DISK replica. + const Replica::Descriptor *replica_ptr = nullptr; + for (const auto &r : op.query_result.replicas) { + if (r.is_disk_replica()) { + replica_ptr = &r; + break; + } + } + if (!replica_ptr) { + LOG(ERROR) << "No DISK replica found for key: " << op.key; + results[op.original_index] = + tl::unexpected(ErrorCode::INVALID_REPLICA); + continue; + } + auto alloc_result = + client_buffer_allocator_->allocate(op.total_size); + if (!alloc_result) { + LOG(ERROR) << "Failed to allocate temp buffer for DISK " + << "read, key: " << op.key + << ", size: " << op.total_size; + results[op.original_index] = + tl::unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + continue; + } + auto handle = + std::make_unique(std::move(*alloc_result)); + std::vector disk_slices; + allocateSlices(disk_slices, *replica_ptr, handle->ptr()); + disk_batch_keys.push_back(op.key); + disk_batch_qrs.push_back( + FilterQueryResult(op.query_result, *replica_ptr)); + disk_batch_slices[op.key] = std::move(disk_slices); + disk_batch_indices.push_back(di); + disk_temp_handles.emplace(op.key, std::move(handle)); + } + + if (!disk_batch_keys.empty()) { + auto disk_results = client_->BatchGet( + disk_batch_keys, disk_batch_qrs, disk_batch_slices); + + for (size_t di = 0; di < disk_batch_indices.size(); ++di) { + const auto &key = disk_batch_keys[di]; + auto &op = disk_operations[disk_batch_indices[di]]; + auto handle_it = disk_temp_handles.find(key); + if (!disk_results[di]) { + LOG(ERROR) << "DISK BatchGet failed for key '" << key + << "': " << toString(disk_results[di].error()); + results[op.original_index] = + tl::unexpected(disk_results[di].error()); + continue; + } + if (auto r = scatter_host_to_maybe_device( + op.dst_buffer, + static_cast(handle_it->second->ptr()), + op.total_size, "DISK read, key: " + key); + !r) { + results[op.original_index] = tl::make_unexpected(r.error()); + } + } + } + } + // Prepare batch transfer data structures - std::unordered_map> + std::unordered_map>> offload_objects; for (const auto &op_it : valid_local_disk_operations) { - const auto &replica = op_it.second.query_result.replicas.at(0); + // Find the LOCAL_DISK replica from the list — replicas may be in + // any order from Master. + const Replica::Descriptor *replica_ptr = nullptr; + for (const auto &r : op_it.second.query_result.replicas) { + if (r.is_local_disk_replica()) { + replica_ptr = &r; + break; + } + } + if (!replica_ptr) { + LOG(ERROR) << "No LOCAL_DISK replica found for key: " + << op_it.first; + continue; + } + const auto &replica = *replica_ptr; auto [store_segment_it, _] = offload_objects.try_emplace( replica.get_local_disk_descriptor().transport_endpoint); - store_segment_it->second.emplace(op_it.first, - op_it.second.slices.at(0)); + store_segment_it->second.emplace(op_it.first, op_it.second.slices); } size_t offload_object_count = 0; @@ -4232,8 +4652,21 @@ RealClient::batch_get_into_multi_buffers_internal( uint64_t total_size; }; + struct DiskKeyInfo { + std::string key; + size_t original_index; + QueryResult query_result; + std::vector buffers; + std::vector sizes; + uint64_t total_size; + bool is_local_disk; // true=LOCAL_DISK (offload RPC), false=DISK + // (BatchGet) + }; + std::vector valid_operations; + std::unordered_map valid_local_disk_ops; valid_operations.reserve(num_keys); + auto local_endpoints = client_->GetLocalEndpoints(); for (size_t i = 0; i < num_keys; ++i) { const auto &key = keys[i]; // Handle query failures @@ -4253,8 +4686,17 @@ RealClient::batch_get_into_multi_buffers_internal( results.emplace_back(tl::unexpected(ErrorCode::INVALID_REPLICA)); continue; } - // Calculate required buffer size - const auto &replica = query_result_values.replicas[0]; + // Select best replica: prefer MEMORY (direct RDMA to GPU), then + // LOCAL_DISK, then DISK. Master may return multiple replicas in any + // order, so always scan rather than blindly taking replicas[0]. + const auto *best_replica = + SelectBestReplica(query_result_values.replicas, local_endpoints); + if (!best_replica) { + LOG(ERROR) << "No usable replica for key: " << key; + results.emplace_back(tl::unexpected(ErrorCode::INVALID_REPLICA)); + continue; + } + const auto replica = *best_replica; uint64_t total_size = calculate_total_size(replica); const auto &sizes = all_sizes[i]; uint64_t dst_total_size = 0; @@ -4273,11 +4715,28 @@ RealClient::batch_get_into_multi_buffers_internal( std::vector key_slices; key_slices.reserve(buffers.size()); if (replica.is_memory_replica()) { + // MEMORY: RDMA from remote memory directly to GPU (GPUDirect). for (size_t j = 0; j < buffers.size(); ++j) { key_slices.emplace_back(Slice{buffers[j], sizes[j]}); } + } else if (replica.is_local_disk_replica() || + replica.is_disk_replica()) { + // LOCAL_DISK: GPU buffers passed directly as scatter-gather slices + // (zero-copy). DISK: file I/O cannot write to GPU memory; temp CPU + // buffer used at read time. + valid_local_disk_ops.emplace( + key, + DiskKeyInfo{.key = key, + .original_index = i, + .query_result = std::move(query_result_values), + .buffers = all_buffers[i], + .sizes = all_sizes[i], + .total_size = total_size, + .is_local_disk = replica.is_local_disk_replica()}); + results.emplace_back(static_cast(total_size)); + continue; } else { - LOG(ERROR) << "Invalid replica type for key: " << key; + LOG(ERROR) << "Unsupported replica type for key: " << key; results.emplace_back(tl::unexpected(ErrorCode::INVALID_PARAMS)); continue; } @@ -4285,44 +4744,218 @@ RealClient::batch_get_into_multi_buffers_internal( valid_operations.push_back( {.key = key, .original_index = i, - .query_result = std::move(query_result_values), + .query_result = FilterQueryResult(query_result_values, replica), .slices = std::move(key_slices), .total_size = total_size}); // Set success result (actual bytes transferred) results.emplace_back(static_cast(total_size)); } // Early return if no valid operations - if (valid_operations.empty()) { + if (valid_operations.empty() && valid_local_disk_ops.empty()) { return results; } - // Prepare batch transfer data structures - std::vector batch_keys; - std::vector batch_query_results; - std::unordered_map> batch_slices; - batch_keys.reserve(valid_operations.size()); - batch_query_results.reserve(valid_operations.size()); - for (auto &op : valid_operations) { - batch_keys.push_back(op.key); - batch_query_results.push_back(op.query_result); - batch_slices[op.key] = op.slices; - } + // ---- Memory/Disk replica: existing BatchGet path ---- + if (!valid_operations.empty()) { + std::vector batch_keys; + std::vector batch_query_results; + std::unordered_map> batch_slices; + batch_keys.reserve(valid_operations.size()); + batch_query_results.reserve(valid_operations.size()); + for (auto &op : valid_operations) { + batch_keys.push_back(op.key); + batch_query_results.push_back(op.query_result); + batch_slices[op.key] = op.slices; + } - auto batch_get_results = - client_->BatchGet(batch_keys, batch_query_results, batch_slices, - prefer_alloc_in_same_node); + auto batch_get_results = + client_->BatchGet(batch_keys, batch_query_results, batch_slices, + prefer_alloc_in_same_node); - // Process transfer results - for (size_t j = 0; j < batch_get_results.size(); ++j) { - const auto &op = valid_operations[j]; - - if (!batch_get_results[j]) { - const auto error = batch_get_results[j].error(); - LOG(ERROR) << "BatchGet failed for key '" << op.key - << "': " << toString(error); - results[op.original_index] = tl::unexpected(error); + for (size_t j = 0; j < batch_get_results.size(); ++j) { + const auto &op = valid_operations[j]; + if (!batch_get_results[j]) { + const auto error = batch_get_results[j].error(); + LOG(ERROR) << "BatchGet failed for key '" << op.key + << "': " << toString(error); + results[op.original_index] = tl::unexpected(error); + } } } + + // ---- LOCAL_DISK / DISK replica: disk read paths ---- + if (!valid_local_disk_ops.empty()) { + // LOCAL_DISK: pass user GPU buffers directly as scatter-gather slices. + // vLLM pre-registers all GPU KV-cache memory with TransferEngine via + // register_buffer(), so the offload RDMA can scatter the on-disk blob + // into non-contiguous per-layer GPU destinations natively — no temp + // CPU buffer or H2D copy needed. + { + std::unordered_map< + std::string, + std::unordered_map>> + offload_objects; + + for (auto &[key, op] : valid_local_disk_ops) { + if (!op.is_local_disk) continue; + // Find the correct LOCAL_DISK replica — Master may return + // replicas in any order (e.g. [DISK, LOCAL_DISK]). + const Replica::Descriptor *replica_ptr = nullptr; + for (const auto &r : op.query_result.replicas) { + if (r.is_local_disk_replica()) { + replica_ptr = &r; + break; + } + } + if (!replica_ptr) { + LOG(ERROR) + << "No LOCAL_DISK replica found for key: " << key; + results[op.original_index] = + tl::make_unexpected(ErrorCode::INVALID_REPLICA); + continue; + } + const auto &replica = *replica_ptr; + std::vector user_slices; + user_slices.reserve(op.buffers.size()); + size_t slice_total = 0; + for (size_t j = 0; j < op.buffers.size(); ++j) { + user_slices.push_back(Slice{op.buffers[j], op.sizes[j]}); + slice_total += op.sizes[j]; + } + if (slice_total < op.total_size) { + LOG(ERROR) << "Slice size too small for key " << key + << ": slices=" << slice_total + << ", total=" << op.total_size; + results[op.original_index] = + tl::make_unexpected(ErrorCode::INVALID_PARAMS); + continue; + } + offload_objects[replica.get_local_disk_descriptor() + .transport_endpoint] + .emplace(key, std::move(user_slices)); + } + + for (auto &[endpoint, objects] : offload_objects) { + if (objects.empty()) continue; + auto read_result = + batch_get_into_offload_object_internal(endpoint, objects); + // On success: results[original_index] was already pre-filled + // with total_size when valid_local_disk_ops was built; nothing + // to update. Only overwrite on failure. + if (!read_result) { + for (auto &[key, slices] : objects) { + auto disk_it = valid_local_disk_ops.find(key); + if (disk_it == valid_local_disk_ops.end()) continue; + LOG(ERROR) << "SSD read failed for key '" << key + << "': " << toString(read_result.error()); + results[disk_it->second.original_index] = + tl::make_unexpected(read_result.error()); + } + } + } + } + + // DISK: one batched BatchGet into CPU temp buffers, then scatter. + // (storage_backend::vector_read cannot write directly to GPU memory) + { + // Scatter temp CPU buffer -> user multi_buffers (GPU or host). + // Returns false and sets results[original_index] on error. + auto scatter_to_buffers = [&](const std::string &key, char *src, + const DiskKeyInfo &op) -> bool { + size_t offset = 0; + for (size_t j = 0; j < op.buffers.size(); ++j) { + if (offset >= op.total_size) break; + size_t sz = + std::min(op.sizes[j], + static_cast(op.total_size - offset)); + void *dst = op.buffers[j]; + if (auto r = scatter_host_to_maybe_device( + dst, src + offset, sz, "DISK scatter, key: " + key); + !r) { + results[op.original_index] = + tl::make_unexpected(r.error()); + return false; + } + offset += sz; + } + return true; + }; + + std::vector disk_batch_keys; + std::vector disk_batch_qrs; + std::unordered_map> + disk_batch_slices; + std::vector disk_key_order; + std::unordered_map> + temp_handles; + + for (auto &[key, op] : valid_local_disk_ops) { + if (op.is_local_disk) continue; + auto alloc_result = + client_buffer_allocator_->allocate(op.total_size); + if (!alloc_result) { + LOG(ERROR) + << "Failed to allocate temp buffer for DISK " + << "read, key: " << key << ", size: " << op.total_size; + results[op.original_index] = + tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + continue; + } + auto handle = + std::make_unique(std::move(*alloc_result)); + // Find the correct DISK replica — Master may return + // replicas in any order (e.g. [LOCAL_DISK, DISK]). + const Replica::Descriptor *replica_ptr = nullptr; + for (const auto &r : op.query_result.replicas) { + if (r.is_disk_replica()) { + replica_ptr = &r; + break; + } + } + if (!replica_ptr) { + LOG(ERROR) << "No DISK replica found for key: " << key; + results[op.original_index] = + tl::make_unexpected(ErrorCode::INVALID_REPLICA); + continue; + } + const auto &replica = *replica_ptr; + std::vector disk_slices; + allocateSlices(disk_slices, replica, handle->ptr()); + disk_batch_keys.push_back(key); + disk_batch_qrs.push_back( + FilterQueryResult(op.query_result, *replica_ptr)); + disk_batch_slices[key] = std::move(disk_slices); + disk_key_order.push_back(key); + temp_handles.emplace(key, std::move(handle)); + } + + if (!disk_batch_keys.empty()) { + auto disk_results = client_->BatchGet( + disk_batch_keys, disk_batch_qrs, disk_batch_slices, + prefer_alloc_in_same_node); + + for (size_t di = 0; di < disk_key_order.size(); ++di) { + const auto &key = disk_key_order[di]; + auto &op = valid_local_disk_ops.at(key); + auto handle_it = temp_handles.find(key); + if (!disk_results[di]) { + LOG(ERROR) + << "DISK BatchGet failed for key '" << key + << "': " << toString(disk_results[di].error()); + results[op.original_index] = + tl::make_unexpected(disk_results[di].error()); + continue; + } + if (!scatter_to_buffers( + key, static_cast(handle_it->second->ptr()), + op)) + continue; + } + } + // temp_handles: BufferHandle RAII releases allocator memory + } + } + return results; } @@ -4689,13 +5322,15 @@ bool RealClient::release_offload_buffer(uint64_t batch_id) { tl::expected RealClient::batch_get_into_offload_object_internal( const std::string &target_rpc_service_addr, - std::unordered_map &objects) { + std::unordered_map> &objects) { auto start_time = std::chrono::steady_clock::now(); std::vector keys; std::vector sizes; for (const auto &object_it : objects) { keys.emplace_back(object_it.first); - sizes.emplace_back(object_it.second.size); + int64_t total = 0; + for (const auto &s : object_it.second) total += s.size; + sizes.emplace_back(total); } auto batchGetResp = client_requester_->batch_get_offload_object( target_rpc_service_addr, keys, sizes); @@ -4704,6 +5339,11 @@ RealClient::batch_get_into_offload_object_internal( << batchGetResp.error(); return tl::make_unexpected(batchGetResp.error()); } + if (batchGetResp->pointers.size() != keys.size()) { + LOG(ERROR) << "Pointer count mismatch from owner: expected=" + << keys.size() << ", got=" << batchGetResp->pointers.size(); + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } auto result = client_->BatchGetOffloadObject(batchGetResp->transfer_engine_addr, keys, batchGetResp->pointers, objects); diff --git a/mooncake-store/src/transfer_task.cpp b/mooncake-store/src/transfer_task.cpp index f892aa0e..70b17e7f 100644 --- a/mooncake-store/src/transfer_task.cpp +++ b/mooncake-store/src/transfer_task.cpp @@ -574,25 +574,38 @@ std::optional TransferSubmitter::submit_batch_get_offload_object( const std::string& transfer_engine_addr, const std::vector& keys, const std::vector& pointers, - const std::unordered_map& batched_slices) { + const std::unordered_map>& batched_slices) { std::optional future; std::vector requests; + // Open the segment once — all keys share the same transfer_engine_addr. + SegmentHandle seg = engine_.openSegment(transfer_engine_addr); + if (seg == static_cast(ERR_INVALID_ARGUMENT)) { + LOG(ERROR) << "Failed to open segment " << transfer_engine_addr; + // nullopt = failure (caller checks !future). The function returns + // std::optional so tl::unexpected is not available here. + return std::nullopt; + } for (size_t i = 0; i < keys.size(); ++i) { - auto key = keys[i]; - auto pointer = pointers[i]; - SegmentHandle seg = engine_.openSegment(transfer_engine_addr); - if (seg == static_cast(ERR_INVALID_ARGUMENT)) { - LOG(ERROR) << "Failed to open segment " << transfer_engine_addr; - return std::nullopt; + const auto& key = keys[i]; + const uint64_t pointer = pointers[i]; + auto it = batched_slices.find(key); + if (it == batched_slices.end()) { + LOG(ERROR) << "Key not found in batched_slices: " << key; + return std::nullopt; // fail closed + } + // Emit one TransferRequest per slice: the on-disk blob is read + // sequentially while slices may point to non-contiguous GPU memory. + uint64_t offset = 0; + for (const auto& slice : it->second) { + TransferRequest request; + request.opcode = TransferRequest::READ; + request.source = static_cast(slice.ptr); + request.target_id = seg; + request.target_offset = pointer + offset; + request.length = slice.size; + requests.emplace_back(request); + offset += slice.size; } - const auto& slice = batched_slices.find(key)->second; - TransferRequest request; - request.opcode = TransferRequest::READ; - request.source = static_cast(slice.ptr); - request.target_id = seg; - request.target_offset = pointer; - request.length = slice.size; - requests.emplace_back(request); } return submitTransfer(requests); } -- 2.34.1 From 22fd38dabbbd483f7b4f13630a783533a536137d Mon Sep 17 00:00:00 2001 From: Dayuxiaoshui <158081477+Dayuxiaoshui@users.noreply.github.com> Date: Sun, 10 May 2026 00:28:06 +0800 Subject: [PATCH 060/382] [TE] feat(transport): add independent maca_transport for Metax MACA C500 (#2059) This commit introduces a standalone maca_transport following the same pattern as hip_transport, instead of polluting nvlink_transport with MACA-specific workarounds. Key points: - Sync mcMemcpy with device-context guard (save/restore device before each copy to avoid mcErrorContextIsDestroyed / SIGSEGV). - Base-pointer registration via cuMemGetAddressRange for correct IPC handle semantics with framework caching allocators. - IPC-only path; fabric memory is not reliably supported on MACA 3.5.3. - P2P access enabled in constructor with original device restoration. Glue changes: - multi_transport.cpp: register "maca" protocol - transfer_engine_impl.cpp: auto-install maca transport under USE_MACA - transfer_metadata.cpp: add "maca" to encode/decode protocol whitelist - transfer_engine_validator.cpp: support --protocol=maca - maca.h: add missing CU_POINTER_ATTRIBUTE_*, cuGetErrorString macros Verified on Metax C500 (2-GPU) with transfer_engine_validator: Data validation passed, throughput ~6.9 GB/s --- .../example/transfer_engine_validator.cpp | 4 + .../include/gpu_vendor/maca.h | 32 +- .../transport/maca_transport/maca_transport.h | 97 ++++ .../src/multi_transport.cpp | 8 + .../src/transfer_engine_impl.cpp | 9 + .../src/transfer_metadata.cpp | 4 +- .../src/transport/CMakeLists.txt | 5 + .../transport/maca_transport/CMakeLists.txt | 5 + .../maca_transport/maca_transport.cpp | 482 ++++++++++++++++++ 9 files changed, 644 insertions(+), 2 deletions(-) create mode 100644 mooncake-transfer-engine/include/transport/maca_transport/maca_transport.h create mode 100644 mooncake-transfer-engine/src/transport/maca_transport/CMakeLists.txt create mode 100644 mooncake-transfer-engine/src/transport/maca_transport/maca_transport.cpp diff --git a/mooncake-transfer-engine/example/transfer_engine_validator.cpp b/mooncake-transfer-engine/example/transfer_engine_validator.cpp index 2ac32d7f..fd6406b5 100644 --- a/mooncake-transfer-engine/example/transfer_engine_validator.cpp +++ b/mooncake-transfer-engine/example/transfer_engine_validator.cpp @@ -413,6 +413,8 @@ int initiator() { xport = engine->installTransport("nvlink", nullptr); } else if (FLAGS_protocol == "nvlink_intra") { xport = engine->installTransport("nvlink_intra", nullptr); + } else if (FLAGS_protocol == "maca") { + xport = engine->installTransport("maca", nullptr); } else if (FLAGS_protocol == "hip") { xport = engine->installTransport("hip", nullptr); } else { @@ -536,6 +538,8 @@ int target() { engine->installTransport("tcp", nullptr); } else if (FLAGS_protocol == "nvlink") { engine->installTransport("nvlink", nullptr); + } else if (FLAGS_protocol == "maca") { + engine->installTransport("maca", nullptr); } else if (FLAGS_protocol == "hip") { engine->installTransport("hip", nullptr); } else { diff --git a/mooncake-transfer-engine/include/gpu_vendor/maca.h b/mooncake-transfer-engine/include/gpu_vendor/maca.h index df470dbb..5e6b4dc6 100644 --- a/mooncake-transfer-engine/include/gpu_vendor/maca.h +++ b/mooncake-transfer-engine/include/gpu_vendor/maca.h @@ -9,7 +9,7 @@ const static std::string GPU_PREFIX = "maca:"; #define CUdevice MCdevice -#define CUdeviceptr MCdeviceptr +#define CUdeviceptr mcDeviceptr_t #define CUmemorytype MCmemorytype #define CUresult mcError_t #define cuDeviceGet mcDeviceGet @@ -34,6 +34,36 @@ const static std::string GPU_PREFIX = "maca:"; #define CUDA_ERROR_NOT_PERMITTED mcErrorNotPermitted #define CUDA_ERROR_NOT_SUPPORTED mcErrorNotSupported +#define CUmemFabricHandle mcMemFabricHandle_t +#define CUmemGenericAllocationHandle mcMemGenericAllocationHandle +#define CUmemAllocationProp mcMemAllocationProp +#define CUmemAccessDesc mcMemAccessDesc +#define CUmemAllocationType mcMemAllocationType +#define CU_MEM_ALLOCATION_TYPE_PINNED mcMemAllocationTypePinned +#define CU_MEM_LOCATION_TYPE_DEVICE mcMemLocationTypeDevice +#define CU_MEM_HANDLE_TYPE_FABRIC mcMemHandleTypeFabric +#define CU_DEVICE_ATTRIBUTE_HANDLE_TYPE_FABRIC_SUPPORTED \ + mcDeviceAttributeHandleTypeFabricSupported +#define CU_DEVICE_ATTRIBUTE_GPU_DIRECT_RDMA_WITH_CUDA_VMM_SUPPORTED \ + mcDeviceAttributeHandleTypePosixFileDescriptorSupported +#define CU_MEM_ACCESS_FLAGS_PROT_READWRITE mcMemAccessFlagsProtReadWrite +#define CU_MEM_ALLOC_GRANULARITY_MINIMUM MC_MEM_ALLOC_GRANULARITY_MINIMUM + +#define CU_MEMORYTYPE_HOST mcMemoryTypeHost +#define CU_MEMORYTYPE_DEVICE mcMemoryTypeDevice +#define CU_POINTER_ATTRIBUTE_MEMORY_TYPE mcPointerAttributeMemoryType +#define CU_POINTER_ATTRIBUTE_RANGE_START_ADDR mcPointerAttributeRangeStartAddr +#define CU_POINTER_ATTRIBUTE_RANGE_SIZE mcPointerAttributeRangeSize +#define CU_MEM_RANGE_HANDLE_TYPE_DMA_BUF_FD mcMemHandleTypePosixFileDescriptor +#define CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED \ + mcDeviceAttributeHandleTypePosixFileDescriptorSupported +static inline CUresult cuGetErrorString(CUresult error, const char **err_str) { + if (err_str) { + *err_str = mcGetErrorString(error); + } + return CUDA_SUCCESS; +} + #define cudaDeviceCanAccessPeer mcDeviceCanAccessPeer #define cudaDeviceEnablePeerAccess mcDeviceEnablePeerAccess #define cudaDeviceGetPCIBusId mcDeviceGetPCIBusId diff --git a/mooncake-transfer-engine/include/transport/maca_transport/maca_transport.h b/mooncake-transfer-engine/include/transport/maca_transport/maca_transport.h new file mode 100644 index 00000000..fed0c240 --- /dev/null +++ b/mooncake-transfer-engine/include/transport/maca_transport/maca_transport.h @@ -0,0 +1,97 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#ifndef MACA_TRANSPORT_H_ +#define MACA_TRANSPORT_H_ + +#include "cuda_alike.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "common/hash_utils.h" +#include "topology.h" +#include "transfer_metadata.h" +#include "transport/transport.h" + +namespace mooncake { + +class TransferMetadata; + +class MacaTransport : public Transport { + public: + MacaTransport(); + + ~MacaTransport(); + + Status submitTransfer(BatchID batch_id, + const std::vector& entries) override; + + Status submitTransferTask( + const std::vector& task_list) override; + + Status getTransferStatus(BatchID batch_id, size_t task_id, + TransferStatus& status) override; + + static void* allocatePinnedLocalMemory(size_t length); + + static void freePinnedLocalMemory(void* addr); + + protected: + int install(std::string& local_server_name, + std::shared_ptr meta, + std::shared_ptr topo) override; + + int registerLocalMemory(void* addr, size_t length, + const std::string& location, bool remote_accessible, + bool update_metadata = true) override; + + int unregisterLocalMemory(void* addr, bool update_metadata = true) override; + + int registerLocalMemoryBatch(const std::vector& buffer_list, + const std::string& location) override; + + int unregisterLocalMemoryBatch( + const std::vector& addr_list) override; + + int relocateSharedMemoryAddress(uint64_t& dest_addr, uint64_t length, + uint64_t target_id); + + const char* getName() const override { return "maca"; } + + private: + std::atomic_bool running_; + + struct OpenedShmEntry { + void* shm_addr; + uint64_t length; + }; + + std::unordered_map, OpenedShmEntry, PairHash> + remap_entries_; + RWSpinlock remap_lock_; + + std::mutex register_mutex_; + std::unordered_set registered_base_addrs_; +}; + +} // namespace mooncake + +#endif // MACA_TRANSPORT_H_ diff --git a/mooncake-transfer-engine/src/multi_transport.cpp b/mooncake-transfer-engine/src/multi_transport.cpp index 812ce821..687262f9 100644 --- a/mooncake-transfer-engine/src/multi_transport.cpp +++ b/mooncake-transfer-engine/src/multi_transport.cpp @@ -44,6 +44,9 @@ #ifdef USE_HIP #include "transport/hip_transport/hip_transport.h" #endif +#ifdef USE_MACA +#include "transport/maca_transport/maca_transport.h" +#endif #ifdef USE_MNNVL #include "transport/nvlink_transport/nvlink_transport.h" #endif @@ -322,6 +325,11 @@ Transport* MultiTransport::installTransport(const std::string& proto, transport = new HipTransport(); } #endif +#ifdef USE_MACA + else if (std::string(proto) == "maca") { + transport = new MacaTransport(); + } +#endif #ifdef USE_MNNVL else if (std::string(proto) == "nvlink") { transport = new NvlinkTransport(); diff --git a/mooncake-transfer-engine/src/transfer_engine_impl.cpp b/mooncake-transfer-engine/src/transfer_engine_impl.cpp index 3d8a5aea..fb6dc512 100644 --- a/mooncake-transfer-engine/src/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/src/transfer_engine_impl.cpp @@ -268,6 +268,15 @@ int TransferEngineImpl::init(const std::string& metadata_conn_string, LOG(ERROR) << "Failed to install Ascend transport"; return -1; } +#elif defined(USE_MACA) + + Transport* t = multi_transports_->installTransport("maca", nullptr); + if (!t) { + LOG(ERROR) << "Failed to install MACA transport"; + return -1; + } + LOG(INFO) << "Using MACA transport"; + #elif defined(USE_MNNVL) || defined(USE_INTRA_NVLINK) const char* force_mnnvl = getenv("MC_FORCE_MNNVL"); diff --git a/mooncake-transfer-engine/src/transfer_metadata.cpp b/mooncake-transfer-engine/src/transfer_metadata.cpp index dcae71d2..355114b6 100644 --- a/mooncake-transfer-engine/src/transfer_metadata.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata.cpp @@ -423,6 +423,7 @@ int TransferMetadata::encodeSegmentDesc(const SegmentDesc &desc, } else if (segmentJSON["protocol"] == "nvlink" || segmentJSON["protocol"] == "nvlink_intra" || segmentJSON["protocol"] == "hip" || + segmentJSON["protocol"] == "maca" || segmentJSON["protocol"] == "ubshmem") { Json::Value buffersJSON(Json::arrayValue); for (const auto &buffer : desc.buffers) { @@ -744,7 +745,8 @@ TransferMetadata::decodeSegmentDesc(Json::Value &segmentJSON, desc->buffers.push_back(buffer); } } else if (desc->protocol == "nvlink" || desc->protocol == "nvlink_intra" || - desc->protocol == "hip" || desc->protocol == "ubshmem") { + desc->protocol == "hip" || desc->protocol == "maca" || + desc->protocol == "ubshmem") { for (const auto &bufferJSON : segmentJSON["buffers"]) { BufferDesc buffer; buffer.name = bufferJSON["name"].asString(); diff --git a/mooncake-transfer-engine/src/transport/CMakeLists.txt b/mooncake-transfer-engine/src/transport/CMakeLists.txt index 96b333e6..04846c25 100644 --- a/mooncake-transfer-engine/src/transport/CMakeLists.txt +++ b/mooncake-transfer-engine/src/transport/CMakeLists.txt @@ -47,6 +47,11 @@ if (USE_HIP) target_sources(transport PUBLIC $) endif() +if (USE_MACA) + add_subdirectory(maca_transport) + target_sources(transport PUBLIC $) +endif() + if (USE_MNNVL AND NOT USE_HIP) add_subdirectory(nvlink_transport) target_sources(transport PUBLIC $) diff --git a/mooncake-transfer-engine/src/transport/maca_transport/CMakeLists.txt b/mooncake-transfer-engine/src/transport/maca_transport/CMakeLists.txt new file mode 100644 index 00000000..eb1b41ec --- /dev/null +++ b/mooncake-transfer-engine/src/transport/maca_transport/CMakeLists.txt @@ -0,0 +1,5 @@ +file(GLOB MACA_TRANSPORT_SOURCES "*.cpp") + +add_library(maca_transport OBJECT ${MACA_TRANSPORT_SOURCES}) + +target_include_directories(maca_transport PUBLIC ${MACA_INCLUDE_DIR}) diff --git a/mooncake-transfer-engine/src/transport/maca_transport/maca_transport.cpp b/mooncake-transfer-engine/src/transport/maca_transport/maca_transport.cpp new file mode 100644 index 00000000..517cea88 --- /dev/null +++ b/mooncake-transfer-engine/src/transport/maca_transport/maca_transport.cpp @@ -0,0 +1,482 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "transport/maca_transport/maca_transport.h" + +#include +#include "cuda_alike.h" +#include + +#include +#include +#include +#include +#include +#include + +#include "common.h" +#include "common/serialization.h" +#include "config.h" +#include "transfer_engine.h" +#include "transfer_metadata.h" +#include "transport/transport.h" + +static bool checkCudaErrorReturn(cudaError_t result, const char *message) { + if (result != cudaSuccess) { + LOG(ERROR) << message << " (Error code: " << result << " - " + << cudaGetErrorString(result) << ")" << std::endl; + return false; + } + return true; +} + +namespace mooncake { +static int getNumDevices() { + static int cached_num_devices = -1; + if (cached_num_devices == -1) { + if (!checkCudaErrorReturn(cudaGetDeviceCount(&cached_num_devices), + "MacaTransport: cudaGetDeviceCount failed")) { + return 0; + } + } + return cached_num_devices; +} + +static bool enableP2PAccess(int src_device_id, int dst_device_id) { + int original_device; + cudaGetDevice(&original_device); + + int canAccessPeer = 0; + if (!checkCudaErrorReturn(cudaDeviceCanAccessPeer( + &canAccessPeer, src_device_id, dst_device_id), + "MacaTransport: failed to query peer access")) { + cudaSetDevice(original_device); + return false; + } + + if (!canAccessPeer) { + LOG(ERROR) << "MacaTransport: device " << src_device_id + << " cannot p2p access device " << dst_device_id; + cudaSetDevice(original_device); + return false; + } + + // enable src->dst p2p access + if (!checkCudaErrorReturn(cudaSetDevice(src_device_id), + "MacaTransport: failed to set device")) { + cudaSetDevice(original_device); + return false; + } + cudaError_t result = cudaDeviceEnablePeerAccess(dst_device_id, 0); + + if (result != cudaSuccess && result != cudaErrorPeerAccessAlreadyEnabled) { + LOG(ERROR) << "MacaTransport: failed to enable p2p access (Error code: " + << result << " - " << cudaGetErrorString(result) << ")" + << std::endl; + cudaSetDevice(original_device); + return false; + } + + // enable dst->src p2p access + if (!checkCudaErrorReturn(cudaSetDevice(dst_device_id), + "MacaTransport: failed to set device")) { + cudaSetDevice(original_device); + return false; + } + result = cudaDeviceEnablePeerAccess(src_device_id, 0); + + if (result != cudaSuccess && result != cudaErrorPeerAccessAlreadyEnabled) { + LOG(ERROR) << "MacaTransport: failed to enable p2p access (Error code: " + << result << " - " << cudaGetErrorString(result) << ")" + << std::endl; + cudaSetDevice(original_device); + return false; + } + + cudaSetDevice(original_device); + return true; +} + +static int getDeviceFromPointer(void *ptr) { + if (!ptr) return -1; + + cudaPointerAttributes attr; + cudaError_t err = cudaPointerGetAttributes(&attr, ptr); + if (err != cudaSuccess) { + LOG(ERROR) << "MacaTransport: cudaPointerGetAttributes failed for " + << ptr; + return -1; + } + + if (attr.type == cudaMemoryTypeDevice) { + return attr.device; + } + return -1; +} + +MacaTransport::MacaTransport() { + int num_devices = getNumDevices(); + if (globalConfig().trace) { + LOG(INFO) << "MacaTransport: num_devices: " << num_devices; + } + + for (int src_device_id = 0; src_device_id < num_devices; ++src_device_id) { + for (int dst_device_id = src_device_id + 1; dst_device_id < num_devices; + ++dst_device_id) { + if (enableP2PAccess(src_device_id, dst_device_id)) { + if (globalConfig().trace) { + LOG(INFO) + << "MacaTransport: enabled p2p access between device " + << src_device_id << " and " << dst_device_id; + } + } else { + LOG(ERROR) + << "MacaTransport: failed to enable p2p access between " + "device " + << src_device_id << " and " << dst_device_id; + } + } + } +} + +MacaTransport::~MacaTransport() { + for (auto &entry : remap_entries_) { + cudaIpcCloseMemHandle(entry.second.shm_addr); + } + remap_entries_.clear(); +} + +int MacaTransport::install(std::string &local_server_name, + std::shared_ptr metadata, + std::shared_ptr topology) { + metadata_ = metadata; + local_server_name_ = local_server_name; + + auto desc = std::make_shared(); + if (!desc) return ERR_MEMORY; + desc->name = local_server_name_; + desc->protocol = "maca"; + metadata_->addLocalSegment(LOCAL_SEGMENT_ID, local_server_name_, + std::move(desc)); + return 0; +} + +Status MacaTransport::submitTransfer( + BatchID batch_id, const std::vector &entries) { + auto &batch_desc = *((BatchDesc *)(batch_id)); + if (batch_desc.task_list.size() + entries.size() > batch_desc.batch_size) { + LOG(ERROR) << "MacaTransport: Exceed the limitation of current batch's " + "capacity"; + return Status::InvalidArgument( + "MacaTransport: Exceed the limitation of capacity, batch id: " + + std::to_string(batch_id)); + } + + size_t task_id = batch_desc.task_list.size(); + batch_desc.task_list.resize(task_id + entries.size()); + + for (auto &request : entries) { + TransferTask &task = batch_desc.task_list[task_id]; + ++task_id; + uint64_t dest_addr = request.target_offset; + if (request.target_id != LOCAL_SEGMENT_ID) { + int rc = relocateSharedMemoryAddress(dest_addr, request.length, + request.target_id); + if (rc) return Status::Memory("device memory not registered"); + } + task.total_bytes = request.length; + Slice *slice = getSliceCache().allocate(); + slice->source_addr = (char *)request.source; + slice->local.dest_addr = (char *)dest_addr; + slice->length = request.length; + slice->opcode = request.opcode; + slice->task = &task; + slice->target_id = request.target_id; + slice->status = Slice::PENDING; + __sync_fetch_and_add(&task.slice_count, 1); + + // Set correct device context before memcpy + int original_device = -1; + cudaGetDevice(&original_device); + int target_device = getDeviceFromPointer(request.source); + if (target_device < 0) + target_device = getDeviceFromPointer((void *)dest_addr); + if (target_device >= 0) cudaSetDevice(target_device); + + cudaError_t err; + if (slice->opcode == TransferRequest::READ) + err = cudaMemcpy(slice->source_addr, (void *)slice->local.dest_addr, + slice->length, cudaMemcpyDefault); + else + err = cudaMemcpy((void *)slice->local.dest_addr, slice->source_addr, + slice->length, cudaMemcpyDefault); + if (err != cudaSuccess) + slice->markFailed(); + else + slice->markSuccess(); + + if (original_device >= 0) cudaSetDevice(original_device); + } + + return Status::OK(); +} + +Status MacaTransport::getTransferStatus(BatchID batch_id, size_t task_id, + TransferStatus &status) { + auto &batch_desc = *((BatchDesc *)(batch_id)); + const size_t task_count = batch_desc.task_list.size(); + if (task_id >= task_count) { + return Status::InvalidArgument( + "MacaTransport::getTransportStatus invalid argument, batch id: " + + std::to_string(batch_id)); + } + auto &task = batch_desc.task_list[task_id]; + status.transferred_bytes = task.transferred_bytes; + uint64_t success_slice_count = task.success_slice_count; + uint64_t failed_slice_count = task.failed_slice_count; + if (success_slice_count + failed_slice_count == task.slice_count) { + if (failed_slice_count) { + status.s = TransferStatusEnum::FAILED; + } else { + status.s = TransferStatusEnum::COMPLETED; + } + task.is_finished = true; + } else { + status.s = TransferStatusEnum::WAITING; + } + return Status::OK(); +} + +Status MacaTransport::submitTransferTask( + const std::vector &task_list) { + for (size_t index = 0; index < task_list.size(); ++index) { + assert(task_list[index]); + auto &task = *task_list[index]; + assert(task.request); + auto &request = *task.request; + uint64_t dest_addr = request.target_offset; + if (request.target_id != LOCAL_SEGMENT_ID) { + int rc = relocateSharedMemoryAddress(dest_addr, request.length, + request.target_id); + if (rc) return Status::Memory("device memory not registered"); + } + task.total_bytes = request.length; + Slice *slice = getSliceCache().allocate(); + slice->source_addr = (char *)request.source; + slice->local.dest_addr = (char *)dest_addr; + slice->length = request.length; + slice->opcode = request.opcode; + slice->task = &task; + slice->target_id = request.target_id; + slice->status = Slice::PENDING; + task.slice_list.push_back(slice); + __sync_fetch_and_add(&task.slice_count, 1); + + // Set correct device context before memcpy + int original_device = -1; + cudaGetDevice(&original_device); + int target_device = getDeviceFromPointer(request.source); + if (target_device < 0) + target_device = getDeviceFromPointer((void *)dest_addr); + if (target_device >= 0) cudaSetDevice(target_device); + + cudaError_t err; + if (slice->opcode == TransferRequest::READ) + err = cudaMemcpy(slice->source_addr, (void *)slice->local.dest_addr, + slice->length, cudaMemcpyDefault); + else + err = cudaMemcpy((void *)slice->local.dest_addr, slice->source_addr, + slice->length, cudaMemcpyDefault); + if (err != cudaSuccess) + slice->markFailed(); + else + slice->markSuccess(); + + if (original_device >= 0) cudaSetDevice(original_device); + } + return Status::OK(); +} + +int MacaTransport::registerLocalMemory(void *addr, size_t length, + const std::string &location, + bool remote_accessible, + bool update_metadata) { + std::lock_guard lock(register_mutex_); + if (globalConfig().trace) { + LOG(INFO) << "register memory: addr " << addr << ", length " << length; + } + cudaPointerAttributes attr; + cudaError_t err = cudaPointerGetAttributes(&attr, addr); + if (err != cudaSuccess) { + LOG(ERROR) << "MacaTransport: cudaPointerGetAttributes failed"; + return -1; + } + + if (attr.type != cudaMemoryTypeDevice) { + LOG(ERROR) << "Unsupported memory type, " << addr << " " << attr.type; + return -1; + } + + // Resolve the true cudaMalloc base address. Framework caching allocators + // (PyTorch, etc.) sub-allocate tensors within larger cudaMalloc segments. + // cudaIpcGetMemHandle always returns a handle for the entire segment, so + // we must register at segment granularity for correct IPC relocation. + CUdeviceptr base_ptr = 0; + size_t alloc_size = 0; + CUresult cu_err = + cuMemGetAddressRange(&base_ptr, &alloc_size, (CUdeviceptr)addr); + if (cu_err != CUDA_SUCCESS) { + LOG(ERROR) << "MacaTransport: cuMemGetAddressRange failed " + << "for addr " << addr << " (error " << cu_err << ")"; + return -1; + } + + // Skip if this cudaMalloc block is already registered + if (registered_base_addrs_.count((uint64_t)base_ptr)) { + return 0; + } + + cudaIpcMemHandle_t handle; + err = cudaIpcGetMemHandle(&handle, (void *)base_ptr); + if (err != cudaSuccess) { + LOG(ERROR) << "MacaTransport: cudaIpcGetMemHandle failed"; + return -1; + } + + (void)remote_accessible; + BufferDesc desc; + desc.addr = (uint64_t)base_ptr; + desc.length = alloc_size; + desc.name = location; + desc.shm_name = serializeBinaryData(&handle, sizeof(cudaIpcMemHandle_t)); + int rc = metadata_->addLocalMemoryBuffer(desc, true); + if (rc == 0) { + registered_base_addrs_.insert((uint64_t)base_ptr); + } + return rc; +} + +int MacaTransport::unregisterLocalMemory(void *addr, bool update_metadata) { + CUdeviceptr base_ptr = 0; + size_t alloc_size = 0; + CUresult cu_err = + cuMemGetAddressRange(&base_ptr, &alloc_size, (CUdeviceptr)addr); + + void *key_ptr = addr; + if (cu_err == CUDA_SUCCESS) { + key_ptr = (void *)base_ptr; + } else { + LOG(WARNING) + << "MacaTransport: cuMemGetAddressRange failed for " + << "addr " << addr << " during unregister (error " << cu_err + << "). Memory may already be freed, using provided address."; + } + + { + std::lock_guard lock(register_mutex_); + registered_base_addrs_.erase((uint64_t)key_ptr); + } + return metadata_->removeLocalMemoryBuffer(key_ptr, update_metadata); +} + +int MacaTransport::relocateSharedMemoryAddress(uint64_t &dest_addr, + uint64_t length, + uint64_t target_id) { + auto desc = metadata_->getSegmentDescByID(target_id); + int index = 0; + for (auto &entry : desc->buffers) { + if (!entry.shm_name.empty() && entry.addr <= dest_addr && + dest_addr + length <= entry.addr + entry.length) { + remap_lock_.lockShared(); + if (remap_entries_.count(std::make_pair(target_id, entry.addr))) { + auto shm_addr = + remap_entries_[std::make_pair(target_id, entry.addr)] + .shm_addr; + remap_lock_.unlockShared(); + dest_addr = dest_addr - entry.addr + ((uint64_t)shm_addr); + return 0; + } + remap_lock_.unlockShared(); + RWSpinlock::WriteGuard lock_guard(remap_lock_); + if (!remap_entries_.count(std::make_pair(target_id, entry.addr))) { + std::vector output_buffer; + deserializeBinaryData(entry.shm_name, output_buffer); + if (output_buffer.size() == sizeof(cudaIpcMemHandle_t)) { + cudaIpcMemHandle_t handle; + memcpy(&handle, output_buffer.data(), sizeof(handle)); + void *shm_addr = nullptr; + cudaError_t err = cudaIpcOpenMemHandle( + &shm_addr, handle, cudaIpcMemLazyEnablePeerAccess); + if (err != cudaSuccess) { + LOG(ERROR) << "MacaTransport: " + "cudaIpcOpenMemHandle failed: " + << cudaGetErrorString(err); + return -1; + } + OpenedShmEntry shm_entry; + shm_entry.shm_addr = shm_addr; + shm_entry.length = entry.length; + remap_entries_[std::make_pair(target_id, entry.addr)] = + shm_entry; + } else { + LOG(ERROR) << "Mismatched MACA data transfer method"; + return -1; + } + } + auto shm_addr = + remap_entries_[std::make_pair(target_id, entry.addr)].shm_addr; + dest_addr = dest_addr - entry.addr + ((uint64_t)shm_addr); + return 0; + } + index++; + } + LOG(ERROR) << "Requested address " << (void *)dest_addr << " to " + << (void *)(dest_addr + length) << " not found!"; + return ERR_INVALID_ARGUMENT; +} + +int MacaTransport::registerLocalMemoryBatch( + const std::vector &buffer_list, + const std::string &location) { + for (auto &buffer : buffer_list) + registerLocalMemory(buffer.addr, buffer.length, location, true, false); + return metadata_->updateLocalSegmentDesc(); +} + +int MacaTransport::unregisterLocalMemoryBatch( + const std::vector &addr_list) { + for (auto &addr : addr_list) unregisterLocalMemory(addr, false); + return metadata_->updateLocalSegmentDesc(); +} + +void *MacaTransport::allocatePinnedLocalMemory(size_t size) { + void *ptr = nullptr; + cudaError_t res = cudaMalloc(&ptr, size); + if (res == cudaSuccess) { + LOG(INFO) << "MacaTransport: allocated device memory " << size + << " bytes"; + return ptr; + } else { + LOG(ERROR) << "MacaTransport: cudaMalloc failed: " + << cudaGetErrorString(res); + return nullptr; + } +} + +void MacaTransport::freePinnedLocalMemory(void *ptr) { + cudaFree(ptr); + return; +} + +} // namespace mooncake -- 2.34.1 From 879135177c45023e831b81f42cb919d56a47be46 Mon Sep 17 00:00:00 2001 From: jinke <1534057243@qq.com> Date: Mon, 11 May 2026 09:47:46 +0800 Subject: [PATCH 061/382] [tebench] support nvlink xport and register buffers per-allocation (#2073) for shm transport, shm_path created by allocateLocalMemory should be passed into registerLocalMemory Co-authored-by: jinke15 --- .../benchmark/tent_backend.cpp | 38 +++++++++---------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/mooncake-transfer-engine/benchmark/tent_backend.cpp b/mooncake-transfer-engine/benchmark/tent_backend.cpp index 57b4ed90..4edcd58f 100644 --- a/mooncake-transfer-engine/benchmark/tent_backend.cpp +++ b/mooncake-transfer-engine/benchmark/tent_backend.cpp @@ -54,9 +54,9 @@ std::shared_ptr loadConfig() { if (!XferBenchConfig::xport_type.empty()) { // Map of transport names to their config keys (handle name mismatches) std::unordered_map transport_map = { - {"rdma", "rdma"}, {"tcp", "tcp"}, {"shm", "shm"}, + {"rdma", "rdma"}, {"tcp", "tcp"}, {"shm", "shm"}, {"iouring", "io_uring"}, // Note: iouring -> io_uring - {"gds", "gds"}, {"mnnvl", "mnnvl"}}; + {"gds", "gds"}, {"mnnvl", "mnnvl"}, {"nvlink", "nvlink"}}; // Disable all transports by default for (const auto& entry : transport_map) { @@ -78,6 +78,7 @@ static TransportType getTransportType(const std::string& xport_type) { if (xport_type == "shm") return SHM; if (xport_type == "gds") return GDS; if (xport_type == "mnnvl") return MNNVL; + if (xport_type == "nvlink") return NVLINK; if (xport_type == "tcp") return TCP; if (xport_type == "iouring") return IOURING; return UNSPEC; @@ -129,38 +130,37 @@ int TENTBenchRunner::allocateBuffers() { return -1; } - // Allocate pinned_buffer_list_.resize(num_buffers, nullptr); - auto start_ts = getCurrentTimeInNano(); + uint64_t alloc_ns = 0, reg_ns = 0; for (int i = 0; i < num_buffers; ++i) { auto location = device_prefix + ":" + std::to_string(start_idx + i); + MemoryOptions options; if (!xport_type.empty()) { - MemoryOptions options; options.type = getTransportType(xport_type); - options.location = std::move(location); + } + + auto t0 = getCurrentTimeInNano(); + if (!xport_type.empty()) { + options.location = location; CHECK_FAIL(engine_->allocateLocalMemory( &pinned_buffer_list_[i], total_buffer_size, options)); } else { CHECK_FAIL(engine_->allocateLocalMemory( &pinned_buffer_list_[i], total_buffer_size, location)); } - } + auto t1 = getCurrentTimeInNano(); - // Register - auto allocated_ts = getCurrentTimeInNano(); - std::vector buffers_size(num_buffers, total_buffer_size); - MemoryOptions options; - if (!xport_type.empty()) { - options.type = getTransportType(xport_type); + CHECK_FAIL(engine_->registerLocalMemory(pinned_buffer_list_[i], + total_buffer_size, options)); + auto t2 = getCurrentTimeInNano(); + + alloc_ns += (t1 - t0); + reg_ns += (t2 - t1); } - CHECK_FAIL(engine_->registerLocalMemory(pinned_buffer_list_, buffers_size, - options)); - auto registered_ts = getCurrentTimeInNano(); LOG(INFO) << "Allocated " << total_buffer_size * num_buffers << " bytes " - << seg_type << " buffers in " << (allocated_ts - start_ts) / 1e6 - << " ms, registered in " << (registered_ts - allocated_ts) / 1e6 - << " ms"; + << seg_type << " buffers in " << alloc_ns / 1e6 + << " ms, registered in " << reg_ns / 1e6 << " ms"; return 0; } -- 2.34.1 From 93f49168d397428db41a5a8b8e4615a6ba71558f Mon Sep 17 00:00:00 2001 From: jinke <1534057243@qq.com> Date: Mon, 11 May 2026 09:54:34 +0800 Subject: [PATCH 062/382] =?UTF-8?q?[TENT]=20fallback=20to=20per-task=20cud?= =?UTF-8?q?aMemcpyAsync=20when=20driver=20lacks=20batch=20s=E2=80=A6=20(#2?= =?UTF-8?q?072)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [TENT] fallback to per-task cudaMemcpyAsync when driver lacks batch support Fixes the case where containers ship CUDA Toolkit 12.8 but the host driver only supports 12.2 * remove unconditionally overwrites the err * style: format nvlink_transport.cpp with clang-format-20 --------- Co-authored-by: jinke15 --- .../src/transport/nvlink/nvlink_transport.cpp | 26 ++++++++++++------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/mooncake-transfer-engine/tent/src/transport/nvlink/nvlink_transport.cpp b/mooncake-transfer-engine/tent/src/transport/nvlink/nvlink_transport.cpp index 8c16662b..5a4d7b09 100644 --- a/mooncake-transfer-engine/tent/src/transport/nvlink/nvlink_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/nvlink/nvlink_transport.cpp @@ -164,7 +164,8 @@ void NVLinkTransport::startTransfer(std::vector& tasks, err = cudaMemcpyBatchAsync(dsts.data(), srcs.data(), sizes.data(), srcs.size(), &attr, &attrs_idx, 1, &fail_idx, batch->async_stream.get()); - if (err != cudaSuccess && fail_idx < tasks.size()) { + if (err != cudaSuccess && err != cudaErrorCallRequiresNewerDriver && + fail_idx < tasks.size()) { LOG(ERROR) << "NVLinkTransport::startTransfer internal error: " << "cudaMemcpyBatchAsync failed at task index " << fail_idx << " (src=" << srcs[fail_idx] << ", dst=" << dsts[fail_idx] @@ -173,17 +174,22 @@ void NVLinkTransport::startTransfer(std::vector& tasks, tasks[fail_idx]->status_word = TransferStatusEnum::FAILED; } #else - err = cudaSuccess; - for (size_t i = 0; i < tasks.size(); ++i) { - auto single_err = - cudaMemcpyAsync(dsts[i], srcs[i], sizes[i], cudaMemcpyDefault, - batch->async_stream.get()); - if (single_err != cudaSuccess) { - tasks[i]->status_word = TransferStatusEnum::FAILED; - err = single_err; + err = cudaErrorCallRequiresNewerDriver; +#endif + + if (err == cudaErrorCallRequiresNewerDriver) { + cudaGetLastError(); + err = cudaSuccess; + for (size_t i = 0; i < tasks.size(); ++i) { + auto single_err = + cudaMemcpyAsync(dsts[i], srcs[i], sizes[i], cudaMemcpyDefault, + batch->async_stream.get()); + if (single_err != cudaSuccess) { + tasks[i]->status_word = TransferStatusEnum::FAILED; + err = single_err; + } } } -#endif if (err != cudaSuccess) { for (auto* task : tasks) { -- 2.34.1 From c3fab08473873bee9d855b14fb24f65902148747 Mon Sep 17 00:00:00 2001 From: xiejibing <33129072+xiejibing@users.noreply.github.com> Date: Mon, 11 May 2026 10:00:16 +0800 Subject: [PATCH 063/382] [TransferEngine] Fix GPU dependency in transfer_engine_bench (#2068) * [TransferEngine] Fix GPU dependency in transfer_engine_bench Problem: transfer_engine_bench crashes (exit code 247) when running with --use_vram=false in environments without GPU, even though it only uses CPU memory (DRAM). Root cause: freeMemoryPool() calls cudaPointerGetAttributes() with checkCudaError(), which exits the program if CUDA fails. Solution: Use graceful error handling like transfer engine core library (memory_location.cpp). When CUDA query fails, assume CPU memory and use numa_free(). Impact: - Enables CPU-only RDMA bandwidth testing without GPU - Consistent behavior with mooncake_client - No impact on existing GPU-enabled scenarios Test: Verified RDMA bandwidth testing works in CPU-only pods and achieves 10+ GB/s throughput on 200G RDMA network. Signed-off-by: jibxie * [TransferEngine] Optimize memory deallocation logic in transfer_engine_bench Check FLAGS_use_vram before calling cudaPointerGetAttributes to avoid unnecessary CUDA calls when memory is explicitly allocated on CPU. - When FLAGS_use_vram is false, memory is guaranteed to be allocated via numa_alloc_onnode, so we can directly call numa_free without checking CUDA pointer attributes - This avoids confusing WARNING logs on systems without GPU when users explicitly choose to use DRAM - Change log level from WARNING to ERROR when FLAGS_use_vram is true but cudaPointerGetAttributes fails, for consistency with memory_location.cpp Co-Authored-By: Claude Sonnet 4 --------- Signed-off-by: jibxie Co-authored-by: jibxie Co-authored-by: Claude Sonnet 4 --- .../example/transfer_engine_bench.cpp | 38 +++++++++++++------ 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/mooncake-transfer-engine/example/transfer_engine_bench.cpp b/mooncake-transfer-engine/example/transfer_engine_bench.cpp index 1cd112a1..cdd6d676 100644 --- a/mooncake-transfer-engine/example/transfer_engine_bench.cpp +++ b/mooncake-transfer-engine/example/transfer_engine_bench.cpp @@ -215,19 +215,35 @@ static void freeMemoryPool(void* addr, size_t size) { #endif } else { #ifndef USE_UBSHMEM - // check pointer on GPU - cudaPointerAttributes attributes; - checkCudaError(cudaPointerGetAttributes(&attributes, addr), - "Failed to get pointer attributes"); - - if (attributes.type == cudaMemoryTypeDevice) { - cudaFree(addr); - } else if (attributes.type == cudaMemoryTypeHost || - attributes.type == cudaMemoryTypeUnregistered) { + // Check FLAGS_use_vram first to avoid unnecessary CUDA calls in non-GPU + // environments + if (!FLAGS_use_vram) { + // Memory was allocated via numa_alloc_onnode, free it directly numa_free(addr, size); } else { - LOG(ERROR) << "Unknown memory type, " << addr << " " - << attributes.type; + // Memory may be on GPU, check pointer attributes + // Use graceful error handling like transfer engine core + // (memory_location.cpp) + cudaPointerAttributes attributes; + cudaError_t result = cudaPointerGetAttributes(&attributes, addr); + + if (result != cudaSuccess) { + // CUDA call failed when FLAGS_use_vram is true - this is an + // error + LOG(ERROR) << "cudaPointerGetAttributes failed (Error code: " + << result << " - " << cudaGetErrorString(result) + << ")"; + numa_free(addr, size); + } else if (attributes.type == cudaMemoryTypeDevice) { + cudaFree(addr); + } else if (attributes.type == cudaMemoryTypeHost || + attributes.type == cudaMemoryTypeUnregistered) { + numa_free(addr, size); + } else { + LOG(ERROR) << "Unknown memory type, " << addr << " " + << attributes.type << ", assuming CPU memory"; + numa_free(addr, size); + } } #endif } -- 2.34.1 From 6caf41289db620db30f0ac213c0748dcec06619f Mon Sep 17 00:00:00 2001 From: JoeZhang-0x000 <77204052+JoeZhang-0x000@users.noreply.github.com> Date: Mon, 11 May 2026 16:31:16 +0800 Subject: [PATCH 064/382] [TransferEngine][Integration] feat: add MACA/MetaX GPU support and fix RDMA dmabuf registration (#2019) * [TransferEngine][Integration] feat: add MACA/MetaX GPU support and fix RDMA dmabuf registration - Fix MACA compatibility macros: correct CUdeviceptr alias, add missing CUDA memory type and pointer attribute macros, implement inline cuGetErrorString wrapper - Fix RDMA dmabuf memory registration for GPU memory: use allocation base address for cuMemGetHandleForAddressRange and compute proper offset for ibv_reg_dmabuf_mr (fixes #1975, #1965) - Add USE_MACA guard alongside existing USE_MLU/USE_CUDA guards for dmabuf-based memory registration path - Support remote_request_id in mooncake connector v1 for cross-request KV cache transfer between prefiller and decoder --------- Co-authored-by: zhangxin --- .../include/gpu_vendor/maca.h | 2 +- .../transport/rdma_transport/rdma_context.cpp | 74 +++++++++++++------ .../mooncake/mooncake_connector_v1.py | 45 +++++++---- 3 files changed, 85 insertions(+), 36 deletions(-) diff --git a/mooncake-transfer-engine/include/gpu_vendor/maca.h b/mooncake-transfer-engine/include/gpu_vendor/maca.h index 5e6b4dc6..604b489a 100644 --- a/mooncake-transfer-engine/include/gpu_vendor/maca.h +++ b/mooncake-transfer-engine/include/gpu_vendor/maca.h @@ -14,7 +14,6 @@ const static std::string GPU_PREFIX = "maca:"; #define CUresult mcError_t #define cuDeviceGet mcDeviceGet #define cuDeviceGetAttribute mcDeviceGetAttribute -#define cuGetErrorString mcGetErrorString #define cuMemAddressFree mcMemAddressFree #define cuMemAddressReserve mcMemAddressReserve #define cuMemCreate mcMemCreate @@ -57,6 +56,7 @@ const static std::string GPU_PREFIX = "maca:"; #define CU_MEM_RANGE_HANDLE_TYPE_DMA_BUF_FD mcMemHandleTypePosixFileDescriptor #define CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED \ mcDeviceAttributeHandleTypePosixFileDescriptorSupported + static inline CUresult cuGetErrorString(CUresult error, const char **err_str) { if (err_str) { *err_str = mcGetErrorString(error); diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index bf543df1..c1006b9c 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -15,8 +15,10 @@ #include "transport/rdma_transport/rdma_context.h" #include +#include #include #include +#include #include #include @@ -227,7 +229,8 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, << "shrink it to " << globalConfig().max_mr_size; length = (size_t)globalConfig().max_mr_size; } -#if defined(USE_MLU) || (!defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA)) +#if defined(USE_MLU) || defined(USE_MACA) || \ + (!defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA)) // Implement register memory in a way that does not assume the presence of // nvidia-peermem. If memory is on CPU call ibv_reg_mr() as usual. If memory // is on GPU then use ibv_reg_dmabuf_mr() instead which does not require @@ -241,6 +244,7 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, mrMeta.addr = addr; mrMeta.mr = ibv_reg_mr(pd_, addr, length, access); } else if (memType == CU_MEMORYTYPE_DEVICE) { +#if defined(USE_CUDA) // Ensure a CUDA context is current — worker threads or callers // from non-CUDA threads may lack one. unsigned int devOrd = 0; @@ -257,16 +261,30 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, // block (e.g. PyTorch caching allocator packs multiple tensors // into one allocation). cuMemGetHandleForAddressRange requires // the exact allocation boundaries. +#endif CUdeviceptr allocBase; size_t allocSize; +#if defined(USE_MLU) + allocBase = (CUdeviceptr)addr; + result = cuPointerGetAttribute( + &allocSize, CU_POINTER_ATTRIBUTE_RANGE_SIZE, (CUdeviceptr)addr); +#else result = cuMemGetAddressRange(&allocBase, &allocSize, (CUdeviceptr)addr); +#endif if (result != CUDA_SUCCESS) { const char *errStr; cuGetErrorString(result, &errStr); +#if defined(USE_MLU) + LOG(ERROR) << "Failed to call cuPointerGetAttribute range size for " + << (uintptr_t)addr << " cuda error=" << errStr; +#else LOG(ERROR) << "Failed to call cuMemGetAddressRange for " << (uintptr_t)addr << " cuda error=" << errStr; +#endif +#if defined(USE_CUDA) cuDevicePrimaryCtxRelease(cuDev); +#endif return ERR_CONTEXT; } @@ -278,15 +296,27 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, const char *errStr; cuGetErrorString(result, &errStr); LOG(ERROR) << "Failed to retrieve dmabuf for " << (uintptr_t)addr - << " cuda error=" << errStr; + << " base=" << (uintptr_t)allocBase + << " size=" << allocSize << " cuda error=" << errStr; +#if defined(USE_CUDA) cuDevicePrimaryCtxRelease(cuDev); +#endif return ERR_CONTEXT; } mrMeta.addr = addr; - uint64_t dmabuf_offset = (uintptr_t)addr - allocBase; + uint64_t dmabuf_offset = (uintptr_t)addr - (uintptr_t)allocBase; mrMeta.mr = ibv_reg_dmabuf_mr(pd_, dmabuf_offset, length, (uintptr_t)addr, dmabuf_fd, access); + const int regErrno = errno; + if (close(dmabuf_fd) != 0) { + PLOG(WARNING) << "Failed to close dmabuf fd"; + } + if (!mrMeta.mr) { + errno = regErrno; + } +#if defined(USE_CUDA) cuDevicePrimaryCtxRelease(cuDev); +#endif } #else mrMeta.addr = addr; @@ -614,15 +644,15 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, return ERR_CONTEXT; } -#if !defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA) - // Verify DMA-BUF support against the CUDA device(s) that the local +#if defined(USE_MACA) || (!defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA)) + // Verify DMA-BUF support against the GPU device(s) that the local // topology explicitly maps to this RNIC, rather than assuming the - // verbs enumeration order matches CUDA enumeration. + // verbs enumeration order matches GPU enumeration. // Validate DMA-BUF support for every GPU that can reach this RNIC, // not just GPUs listing it as preferred. Runtime selection falls // back to avail_hca when a preferred NIC is disabled, so we must // validate both lists. - std::vector mapped_cuda_devices; + std::vector mapped_gpu_devices; if (engine_.local_topology_) { const auto topology_matrix = engine_.local_topology_->getMatrix(); for (const auto &entry : topology_matrix) { @@ -638,7 +668,7 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, if (!in_preferred && !in_avail) continue; try { - mapped_cuda_devices.push_back( + mapped_gpu_devices.push_back( std::stoi(entry.first.substr(GPU_PREFIX.size()))); } catch (const std::exception &e) { LOG(WARNING) << "Ignore malformed topology GPU entry " @@ -647,28 +677,30 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, } } - std::sort(mapped_cuda_devices.begin(), mapped_cuda_devices.end()); - mapped_cuda_devices.erase( - std::unique(mapped_cuda_devices.begin(), mapped_cuda_devices.end()), - mapped_cuda_devices.end()); + std::sort(mapped_gpu_devices.begin(), mapped_gpu_devices.end()); + mapped_gpu_devices.erase( + std::unique(mapped_gpu_devices.begin(), mapped_gpu_devices.end()), + mapped_gpu_devices.end()); - if (mapped_cuda_devices.empty()) { - LOG(INFO) << "No CUDA device is explicitly mapped to RNIC " + if (mapped_gpu_devices.empty()) { + LOG(INFO) << "No GPU device is explicitly mapped to RNIC " << device_name << "; skip DMA-BUF affinity validation"; } else { // cuInit is process-global and idempotent; call it once before // the per-device loop, not per cuDeviceGet. +#if defined(USE_CUDA) CUresult result = cuInit(0); if (result != CUDA_SUCCESS) { LOG(ERROR) << "Failed to initialize CUDA driver for RNIC " << device_name; goto cleanup_context_and_devices; } - for (int cuda_device : mapped_cuda_devices) { +#endif + for (int gpu_device : mapped_gpu_devices) { CUdevice cuDevice; - result = cuDeviceGet(&cuDevice, cuda_device); + CUresult result = cuDeviceGet(&cuDevice, gpu_device); if (result != CUDA_SUCCESS) { - LOG(ERROR) << "Failed to query CUDA device " << cuda_device + LOG(ERROR) << "Failed to query GPU device " << gpu_device << " for RNIC " << device_name; goto cleanup_context_and_devices; } @@ -677,16 +709,16 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, &dmaBufSupported, CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED, cuDevice); if (result != CUDA_SUCCESS) { - LOG(ERROR) << "Failed to query CUDA device attributes for " - << "CUDA device " << cuda_device << " and RNIC " + LOG(ERROR) << "Failed to query GPU device attributes for " + << "GPU device " << gpu_device << " and RNIC " << device_name; goto cleanup_context_and_devices; } if (!dmaBufSupported) { LOG(ERROR) << "DMA BUF supported required for GPU RDMA without " - "nvidia-peermem on CUDA device " - << cuda_device << " mapped to RNIC " << device_name; + "nvidia-peermem on GPU device " + << gpu_device << " mapped to RNIC " << device_name; goto cleanup_context_and_devices; } } diff --git a/mooncake-wheel/mooncake/mooncake_connector_v1.py b/mooncake-wheel/mooncake/mooncake_connector_v1.py index ee30914c..18873b38 100644 --- a/mooncake-wheel/mooncake/mooncake_connector_v1.py +++ b/mooncake-wheel/mooncake/mooncake_connector_v1.py @@ -88,6 +88,7 @@ class RecvReqMeta: local_block_ids: list[int] remote_host: str remote_port: int + remote_request_id: Optional[ReqId] = None @dataclass @@ -112,7 +113,8 @@ class MooncakeConnectorMetadata(KVConnectorMetadata): self.reqs_to_recv[request_id] = RecvReqMeta( local_block_ids=local_block_ids, remote_host=kv_transfer_params["remote_host"], - remote_port=kv_transfer_params["remote_port"]) + remote_port=kv_transfer_params["remote_port"], + remote_request_id=kv_transfer_params.get("remote_request_id")) else: self.reqs_to_send[request_id] = local_block_ids @@ -382,10 +384,12 @@ class MooncakeConnectorScheduler: if delay_free_blocks: self._reqs_need_send[request.request_id] = block_ids - return delay_free_blocks, dict(do_remote_prefill=True, - do_remote_decode=False, - remote_host=self.side_channel_host, - remote_port=self.side_channel_port) + return delay_free_blocks, dict( + do_remote_prefill=True, + do_remote_decode=False, + remote_host=self.side_channel_host, + remote_port=self.side_channel_port, + remote_request_id=request.request_id) class MooncakeConnectorWorker: @@ -812,12 +816,14 @@ class MooncakeConnectorWorker: return finished_sending_reqs or None, finished_recving_reqs or None - async def receive_kv(self, path: str, req_blocks: list[tuple[str, list[int]]]): - req_ids, block_ids = map(list, zip(*req_blocks)) + async def receive_kv( + self, path: str, req_blocks: list[tuple[str, str, list[int]]] + ): + local_req_ids, remote_req_ids, block_ids = map(list, zip(*req_blocks)) metadata = MooncakeAgentMetadata( remote_hostname=self.hostname, remote_port=self.rpc_port, - request_ids=req_ids, + request_ids=remote_req_ids, kv_caches_base_addr=self.kv_caches_base_addr, block_ids=block_ids, ) @@ -826,7 +832,9 @@ class MooncakeConnectorWorker: logger.debug( "Size of encoded MooncakeAgentMetadata: %d bytes", len(encoded_data) ) - logger.debug("Sending kv transfer request for %s on path: %s", req_ids, path) + logger.debug( + "Sending kv transfer request for %s on path: %s " + "(local requests: %s)", remote_req_ids, path, local_req_ids) # Send query for the request. sock: zmq.asyncio.Socket = make_zmq_socket( @@ -839,20 +847,24 @@ class MooncakeConnectorWorker: if ret_msg != TRANS_DONE: logger.error( "Error happens during transferring kvcache for %s, see logs in prefiller.", # noqa: E501 - req_ids, + remote_req_ids, ) return except zmq.ContextTerminated: logger.debug("ZMQ context terminated, exiting Mooncake receiver thread.") except Exception as e: - logger.error("MooncakeAgentMetadata transfer failed for %s: %s", req_ids, e) + logger.error( + "MooncakeAgentMetadata transfer failed for %s: %s", + remote_req_ids, e) return finally: sock.close() - self.finished_recving_reqs.update(req_ids) + self.finished_recving_reqs.update(local_req_ids) - logger.debug("pulling kv_caches for %s finished", req_ids) + logger.debug( + "pulling kv_caches for %s finished (local requests: %s)", + remote_req_ids, local_req_ids) def group_kv_pull(self, metadata: MooncakeConnectorMetadata): kv_pulls = defaultdict(list) @@ -862,7 +874,12 @@ class MooncakeConnectorWorker: "Num local_block_ids: %s.", req_id, len(meta.local_block_ids)) path = make_zmq_path("tcp", meta.remote_host, meta.remote_port + self.tp_rank) - kv_pulls[path].append((req_id, meta.local_block_ids)) + remote_req_id = meta.remote_request_id or req_id + if remote_req_id != req_id: + logger.debug( + "request %s will pull remote kv for producer request %s", + req_id, remote_req_id) + kv_pulls[path].append((req_id, remote_req_id, meta.local_block_ids)) return kv_pulls -- 2.34.1 From e294da69888763e8cb7823a4aa7afe9a27ea8486 Mon Sep 17 00:00:00 2001 From: vie-serendipity <2733147505@qq.com> Date: Tue, 12 May 2026 10:06:47 +0800 Subject: [PATCH 065/382] [devcontainer] use build.dockerFile instead of dockerfile and add libxxhash-dev (#2067) Signed-off-by: vie-serendipity <2733147505@qq.com> --- .devcontainer/Dockerfile | 1 + .devcontainer/devcontainer.json | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 7835f4c0..405d2433 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -26,6 +26,7 @@ RUN apt-get install -y libibverbs-dev \ libzstd-dev \ libmsgpack-dev \ libgflags-dev \ + libxxhash-dev \ pkg-config \ patchelf diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index 7df63069..dea0cbb2 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -1,6 +1,8 @@ { "name": "Mooncake Dev", - "dockerFile": "Dockerfile", + "build": { + "dockerfile": "Dockerfile" + }, "runArgs": [ "--cap-add=SYS_PTRACE", "--cap-add=NET_RAW", @@ -18,7 +20,7 @@ "vscode": { "settings": { "terminal.integrated.shell.linux": "/bin/bash", - "bazel.buildifierFixOnFormat": true, + "bazel.buildifierFixOnFormat": true } } } -- 2.34.1 From ef0f40cd2dfc4acf42b60b03ad417e0922f5baac Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Tue, 12 May 2026 14:04:50 +0800 Subject: [PATCH 066/382] [Store] Fix SpinLock memory order for weakly-ordered architectures (#2076) The SpinLock::lock() inner spin loop used memory_order_relaxed for flag.test(), which can fail to promptly observe unlock() on ARM/RISC-V. - Use memory_order_acquire on test_and_set in both fast and slow paths - Keep relaxed loads only inside the inner PAUSE() spin loop for performance - This ensures proper happens-before synchronization on weakly-ordered CPUs --- mooncake-store/include/mutex.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mooncake-store/include/mutex.h b/mooncake-store/include/mutex.h index 336440aa..caeefca3 100644 --- a/mooncake-store/include/mutex.h +++ b/mooncake-store/include/mutex.h @@ -125,6 +125,10 @@ class CAPABILITY("mutex") SpinLock { public: // Acquire/lock the spinlock. void lock() ACQUIRE() { + // Fast path: try to acquire directly + if (!flag.test_and_set(std::memory_order_acquire)) return; + + // Slow path: spin wait with relaxed loads, acquire on success do { while (flag.test(std::memory_order_relaxed)) { PAUSE(); -- 2.34.1 From 704da4b92f2520e72c74217dfc583283d3455a37 Mon Sep 17 00:00:00 2001 From: fang-tech Date: Tue, 12 May 2026 14:19:14 +0800 Subject: [PATCH 067/382] [Store][Fix]: enable local memcpy for metadata local replicas (#2029) * fix(store): enable local memcpy for metadata local replicas Pass the client local hostname into TransferSubmitter and use it when detecting local transfers. This allows metadata-service descriptors, which use the hostname as the segment identifier, to correctly select LOCAL_MEMCPY for local reads while preserving transfer-engine endpoint matching for P2P descriptors. Add coverage for TCP local memcpy auto-enable behavior across P2P and metadata modes, including remote same-host cases that should continue using TRANSFER_ENGINE. Also verify hot-cache hits do not increment the admission sketch when LOCAL_MEMCPY is selected. --- mooncake-store/include/transfer_task.h | 2 + mooncake-store/src/client_service.cpp | 2 +- mooncake-store/src/transfer_task.cpp | 20 +- mooncake-store/tests/CMakeLists.txt | 1 + .../tests/client_tcp_local_memcpy_test.cpp | 508 ++++++++++++++++++ 5 files changed, 528 insertions(+), 5 deletions(-) create mode 100644 mooncake-store/tests/client_tcp_local_memcpy_test.cpp diff --git a/mooncake-store/include/transfer_task.h b/mooncake-store/include/transfer_task.h index 4dbb9fc1..df7c90e9 100644 --- a/mooncake-store/include/transfer_task.h +++ b/mooncake-store/include/transfer_task.h @@ -370,6 +370,7 @@ class TransferSubmitter { public: explicit TransferSubmitter(TransferEngine& engine, std::shared_ptr& backend, + const std::string& local_hostname, TransferMetric* transfer_metric = nullptr); /** @@ -414,6 +415,7 @@ class TransferSubmitter { std::unique_ptr memcpy_pool_; std::unique_ptr fileread_pool_; bool memcpy_enabled_; + const std::string local_hostname_; TransferMetric* transfer_metric_; /** diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 1373f590..e6f421dc 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -538,7 +538,7 @@ void Client::InitTransferSubmitter() { // Keep using logical local_hostname for name-based behaviors; endpoint is // used separately where needed. transfer_submitter_ = std::make_unique( - *transfer_engine_, storage_backend_, + *transfer_engine_, storage_backend_, local_hostname_, metrics_ ? &metrics_->transfer_metric : nullptr); } diff --git a/mooncake-store/src/transfer_task.cpp b/mooncake-store/src/transfer_task.cpp index 70b17e7f..a5ab382c 100644 --- a/mooncake-store/src/transfer_task.cpp +++ b/mooncake-store/src/transfer_task.cpp @@ -5,6 +5,7 @@ #include #include #include +#include #include #include "gpu_staging_utils.h" #include "transfer_engine.h" @@ -447,10 +448,12 @@ TransferStrategy TransferFuture::strategy() const { TransferSubmitter::TransferSubmitter(TransferEngine& engine, std::shared_ptr& backend, + const std::string& local_hostname, TransferMetric* transfer_metric) : engine_(engine), memcpy_pool_(std::make_unique()), fileread_pool_(std::make_unique(backend)), + local_hostname_(local_hostname), transfer_metric_(transfer_metric) { // Read MC_STORE_MEMCPY environment variable. // When not set, auto-detect based on transport type: @@ -815,11 +818,20 @@ TransferStrategy TransferSubmitter::selectStrategy( bool TransferSubmitter::isLocalTransfer( const AllocatedBuffer::Descriptor& handle) const { - std::string local_ep = engine_.getLocalIpAndPort(); + if (handle.transport_endpoint_.empty()) return false; - if (!local_ep.empty()) { - return !handle.transport_endpoint_.empty() && - handle.transport_endpoint_ == local_ep; + // Metadata-service descriptors use the client hostname as the segment ID. + // If it matches this client's hostname, the buffer address is local. + if (!local_hostname_.empty() && + local_hostname_ == handle.transport_endpoint_) { + return true; + } + + // P2P descriptors use the transfer engine endpoint as the segment ID. + // If it matches this engine's endpoint, the buffer address is local. + std::string local_ep = engine_.getLocalIpAndPort(); + if (!local_ep.empty() && handle.transport_endpoint_ == local_ep) { + return true; } // Without a local endpoint we cannot prove locality; disable memcpy. diff --git a/mooncake-store/tests/CMakeLists.txt b/mooncake-store/tests/CMakeLists.txt index 23f19b7d..aff9e924 100644 --- a/mooncake-store/tests/CMakeLists.txt +++ b/mooncake-store/tests/CMakeLists.txt @@ -55,6 +55,7 @@ add_store_test(offset_allocator_test offset_allocator_test.cpp) add_store_test(utils_test utils_test.cpp) add_store_test(client_buffer_test client_buffer_test.cpp) add_store_test(client_local_hot_cache_test client_local_hot_cache_test.cpp) +add_store_test(client_tcp_local_memcpy_test client_tcp_local_memcpy_test.cpp) add_store_test(pybind_client_test pybind_client_test.cpp) add_store_test(ipv6_client_test ipv6_client_test.cpp) add_store_test(client_metrics_test client_metrics_test.cpp) diff --git a/mooncake-store/tests/client_tcp_local_memcpy_test.cpp b/mooncake-store/tests/client_tcp_local_memcpy_test.cpp new file mode 100644 index 00000000..289ba85a --- /dev/null +++ b/mooncake-store/tests/client_tcp_local_memcpy_test.cpp @@ -0,0 +1,508 @@ +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "allocator.h" +#include "client_service.h" +#include "test_server_helpers.h" +#include "types.h" +#include "utils.h" + +namespace mooncake { +namespace testing { + +enum class HandshakeMode { P2P, Metadata }; + +namespace { + +class EnvGuard { + public: + explicit EnvGuard(const char* key) : key_(key) { + if (const char* value = std::getenv(key_)) { + old_value_ = value; + } + } + + ~EnvGuard() { + if (old_value_.has_value()) { + setenv(key_, old_value_->c_str(), 1); + } else { + unsetenv(key_); + } + } + + private: + const char* key_; + std::optional old_value_; +}; + +class VLogGuard { + public: + explicit VLogGuard(int new_v) : old_v_(FLAGS_v) { FLAGS_v = new_v; } + ~VLogGuard() { FLAGS_v = old_v_; } + + private: + int old_v_; +}; + +class StrategyCaptureSink : public google::LogSink { + public: + void send(google::LogSeverity severity, const char* full_filename, + const char* base_filename, int line, const struct ::tm* tm_time, + const char* message, size_t message_len) override { + (void)severity; + (void)full_filename; + (void)base_filename; + (void)line; + (void)tm_time; + + std::string msg(message, message_len); + constexpr char kPrefix[] = "Using transfer strategy: "; + size_t pos = msg.find(kPrefix); + if (pos == std::string::npos) { + return; + } + + captured_strategy_ = msg.substr(pos + std::strlen(kPrefix)); + TrimInPlace(captured_strategy_); + } + + const std::string& strategy() const { return captured_strategy_; } + + private: + static void TrimInPlace(std::string& value) { + auto begin = value.find_first_not_of(" \t\n\r"); + if (begin == std::string::npos) { + value.clear(); + return; + } + auto end = value.find_last_not_of(" \t\n\r"); + value = value.substr(begin, end - begin + 1); + } + + std::string captured_strategy_; +}; + +struct ClientRuntime { + std::shared_ptr client; + std::unique_ptr io_allocator; + void* segment_ptr = nullptr; + size_t segment_size = 0; + std::string host_name; +}; + +struct PreparedObject { + std::string key; + std::string payload; + QueryResult query_result; + Replica::Descriptor replica; +}; + +Replica::Descriptor FindFirstCompleteMemoryReplica(const QueryResult& result) { + for (const auto& replica : result.replicas) { + if (replica.status == ReplicaStatus::COMPLETE && + replica.is_memory_replica()) { + return replica; + } + } + throw std::runtime_error("No complete memory replica found"); +} + +bool PutHotKeyHelper(LocalHotCache& cache, const std::string& key, + const Slice& slice) { + if (key.empty() || slice.ptr == nullptr || slice.size == 0) { + return false; + } + if (cache.TouchHotKey(key)) { + return true; + } + + HotMemBlock* block = cache.GetFreeBlock(); + if (block == nullptr) { + return false; + } + if (slice.size > block->size) { + block->key_.clear(); + cache.PutHotKey(block); + return false; + } + + std::memcpy(block->addr, slice.ptr, slice.size); + block->size = slice.size; + block->key_ = key; + return cache.PutHotKey(block); +} + +const char* HandshakeModeName(HandshakeMode mode) { + switch (mode) { + case HandshakeMode::P2P: + return "P2P"; + case HandshakeMode::Metadata: + return "Metadata"; + } + return "Unknown"; +} + +} // namespace + +class TcpLocalMemcpyAutoEnableTest : public ::testing::Test { + protected: + static void SetUpTestSuite() { + google::InitGoogleLogging("TcpLocalMemcpyAutoEnableTest"); + FLAGS_logtostderr = 1; + } + + static void TearDownTestSuite() { google::ShutdownGoogleLogging(); } + + void SetUp() override { + memcpy_guard_ = std::make_unique("MC_STORE_MEMCPY"); + vlog_guard_ = std::make_unique(1); + unsetenv("MC_STORE_MEMCPY"); + + InProcMasterConfig config; + config.http_metadata_port = getFreeTcpPort(); + ASSERT_TRUE(master_.Start(config)); + master_address_ = master_.master_address(); + metadata_url_ = master_.metadata_url(); + } + + void TearDown() override { + CleanupRuntime(remote_runtime_); + CleanupRuntime(runtime_); + master_.Stop(); + } + + ClientRuntime CreateRuntime(const std::string& host_name, + const std::string& metadata_conn) { + ClientRuntime runtime; + runtime.host_name = host_name; + + auto client_opt = Client::Create(host_name, metadata_conn, "tcp", + std::nullopt, master_address_); + EXPECT_TRUE(client_opt.has_value()) + << "Failed to create client for host " << host_name + << ", metadata=" << metadata_conn; + if (!client_opt.has_value()) { + return runtime; + } + + runtime.client = client_opt.value(); + runtime.io_allocator = + std::make_unique(16 * 1024 * 1024); + auto reg = runtime.client->RegisterLocalMemory( + runtime.io_allocator->getBase(), 16 * 1024 * 1024, "cpu:0", false, + false); + EXPECT_TRUE(reg.has_value()) + << "RegisterLocalMemory failed: " << toString(reg.error()); + + runtime.segment_size = 64 * 1024 * 1024; + runtime.segment_ptr = + allocate_buffer_allocator_memory(runtime.segment_size); + EXPECT_NE(runtime.segment_ptr, nullptr); + if (runtime.segment_ptr == nullptr) { + return runtime; + } + + auto mount = runtime.client->MountSegment(runtime.segment_ptr, + runtime.segment_size, "tcp"); + EXPECT_TRUE(mount.has_value()) + << "MountSegment failed: " << toString(mount.error()); + if (!mount.has_value()) { + free_memory("", runtime.segment_ptr); + runtime.segment_ptr = nullptr; + } + + return runtime; + } + + void CleanupRuntime(ClientRuntime& runtime) { + if (runtime.client && runtime.segment_ptr) { + auto unmount = runtime.client->UnmountSegment(runtime.segment_ptr, + runtime.segment_size); + EXPECT_TRUE(unmount.has_value()) + << "UnmountSegment failed: " << toString(unmount.error()); + } + + runtime.client.reset(); + runtime.io_allocator.reset(); + + if (runtime.segment_ptr) { + free_memory("", runtime.segment_ptr); + runtime.segment_ptr = nullptr; + } + runtime.segment_size = 0; + runtime.host_name.clear(); + } + + PreparedObject PrepareLocalObject(const std::string& key, + const std::string& payload) { + return PrepareObjectOnRuntime(runtime_, key, payload); + } + + PreparedObject PrepareObjectOnRuntime(ClientRuntime& runtime, + const std::string& key, + const std::string& payload) { + void* write_buf = runtime.io_allocator->allocate(payload.size()); + EXPECT_NE(write_buf, nullptr); + std::memcpy(write_buf, payload.data(), payload.size()); + + std::vector write_slices; + write_slices.emplace_back(Slice{write_buf, payload.size()}); + + ReplicateConfig config; + config.replica_num = 1; + config.preferred_segment = runtime.host_name; + + auto put = runtime.client->Put(key, write_slices, config); + runtime.io_allocator->deallocate(write_buf, payload.size()); + EXPECT_TRUE(put.has_value()) << "Put failed: " << toString(put.error()); + + auto query = runtime.client->Query(key); + EXPECT_TRUE(query.has_value()) + << "Query failed: " << toString(query.error()); + + auto replica = FindFirstCompleteMemoryReplica(query.value()); + + return PreparedObject{key, payload, std::move(query.value()), replica}; + } + + void LogReplicaDiagnostics(const std::string& label, + const Replica::Descriptor& replica) { + const auto& endpoint = replica.get_memory_descriptor() + .buffer_descriptor.transport_endpoint_; + LOG(INFO) << label << " replica endpoint=" << endpoint; + LOG(INFO) << label << " client transport endpoint=" + << runtime_.client->GetTransportEndpoint(); + LOG(INFO) << label << " is local replica=" + << runtime_.client->IsReplicaOnLocalMemory(replica); + } + + InProcMaster master_; + std::string master_address_; + std::string metadata_url_; + ClientRuntime runtime_; + ClientRuntime remote_runtime_; + std::unique_ptr memcpy_guard_; + std::unique_ptr vlog_guard_; +}; + +TEST_F(TcpLocalMemcpyAutoEnableTest, P2PLocalReplicaUsesLocalMemcpy) { + runtime_ = CreateRuntime("localhost", "P2PHANDSHAKE"); + ASSERT_TRUE(runtime_.client != nullptr); + + auto prepared = + PrepareLocalObject("p2p_local_memcpy_key", "hello-local-memcpy"); + LogReplicaDiagnostics("[P2P]", prepared.replica); + + StrategyCaptureSink sink; + google::AddLogSink(&sink); + + std::vector out(prepared.payload.size(), '\0'); + std::vector read_slices; + read_slices.emplace_back(Slice{out.data(), out.size()}); + + auto get = + runtime_.client->Get(prepared.key, prepared.query_result, read_slices); + + google::RemoveLogSink(&sink); + + ASSERT_TRUE(get.has_value()) << "Get failed: " << toString(get.error()); + ASSERT_EQ(std::memcmp(out.data(), prepared.payload.data(), out.size()), 0); + ASSERT_TRUE(runtime_.client->IsReplicaOnLocalMemory(prepared.replica)); + + const auto& replica_endpoint = prepared.replica.get_memory_descriptor() + .buffer_descriptor.transport_endpoint_; + EXPECT_EQ(replica_endpoint, runtime_.client->GetTransportEndpoint()); + EXPECT_EQ(sink.strategy(), "LOCAL_MEMCPY"); +} + +TEST_F(TcpLocalMemcpyAutoEnableTest, MetadataLocalReplicaUsesLocalMemcpy) { + runtime_ = CreateRuntime("localhost", metadata_url_); + ASSERT_TRUE(runtime_.client != nullptr); + + auto prepared = + PrepareLocalObject("metadata_local_memcpy_key", "hello-local-memcpy"); + LogReplicaDiagnostics("[METADATA]", prepared.replica); + + const auto& replica_endpoint = prepared.replica.get_memory_descriptor() + .buffer_descriptor.transport_endpoint_; + const auto client_endpoint = runtime_.client->GetTransportEndpoint(); + const bool is_local = + runtime_.client->IsReplicaOnLocalMemory(prepared.replica); + + StrategyCaptureSink sink; + google::AddLogSink(&sink); + + std::vector out(prepared.payload.size(), '\0'); + std::vector read_slices; + read_slices.emplace_back(Slice{out.data(), out.size()}); + + auto get = + runtime_.client->Get(prepared.key, prepared.query_result, read_slices); + + google::RemoveLogSink(&sink); + + ASSERT_TRUE(get.has_value()) << "Get failed: " << toString(get.error()); + ASSERT_EQ(std::memcmp(out.data(), prepared.payload.data(), out.size()), 0); + ASSERT_TRUE(is_local); + EXPECT_NE(replica_endpoint, client_endpoint); + EXPECT_EQ(sink.strategy(), "LOCAL_MEMCPY"); +} + +TEST_F(TcpLocalMemcpyAutoEnableTest, + P2PRemoteReplicaOnSameTcpHostUsesTransferEngine) { + remote_runtime_ = CreateRuntime("127.0.0.1:18001", "P2PHANDSHAKE"); + runtime_ = CreateRuntime("127.0.0.1:18002", "P2PHANDSHAKE"); + ASSERT_TRUE(remote_runtime_.client != nullptr); + ASSERT_TRUE(runtime_.client != nullptr); + + auto prepared = PrepareObjectOnRuntime( + remote_runtime_, "p2p_remote_same_host_key", "hello-remote-transfer"); + LogReplicaDiagnostics("[P2P_REMOTE]", prepared.replica); + + const auto& replica_endpoint = prepared.replica.get_memory_descriptor() + .buffer_descriptor.transport_endpoint_; + const auto client_endpoint = runtime_.client->GetTransportEndpoint(); + ASSERT_NE(replica_endpoint, client_endpoint); + ASSERT_FALSE(runtime_.client->IsReplicaOnLocalMemory(prepared.replica)); + + StrategyCaptureSink sink; + google::AddLogSink(&sink); + + std::vector out(prepared.payload.size(), '\0'); + std::vector read_slices; + read_slices.emplace_back(Slice{out.data(), out.size()}); + + auto get = + runtime_.client->Get(prepared.key, prepared.query_result, read_slices); + + google::RemoveLogSink(&sink); + + ASSERT_TRUE(get.has_value()) << "Get failed: " << toString(get.error()); + ASSERT_EQ(std::memcmp(out.data(), prepared.payload.data(), out.size()), 0); + EXPECT_EQ(sink.strategy(), "TRANSFER_ENGINE"); +} + +TEST_F(TcpLocalMemcpyAutoEnableTest, + MetadataRemoteReplicaOnSameTcpHostUsesTransferEngine) { + remote_runtime_ = CreateRuntime("metadata-host-a", metadata_url_); + runtime_ = CreateRuntime("metadata-host-b", metadata_url_); + ASSERT_TRUE(remote_runtime_.client != nullptr); + ASSERT_TRUE(runtime_.client != nullptr); + + auto prepared = + PrepareObjectOnRuntime(remote_runtime_, "metadata_remote_same_host_key", + "hello-metadata-remote-transfer"); + LogReplicaDiagnostics("[METADATA_REMOTE]", prepared.replica); + + const auto& replica_endpoint = prepared.replica.get_memory_descriptor() + .buffer_descriptor.transport_endpoint_; + const auto client_endpoint = runtime_.client->GetTransportEndpoint(); + ASSERT_EQ(replica_endpoint, remote_runtime_.host_name); + ASSERT_NE(replica_endpoint, runtime_.host_name); + ASSERT_NE(replica_endpoint, client_endpoint); + ASSERT_FALSE(runtime_.client->IsReplicaOnLocalMemory(prepared.replica)); + + StrategyCaptureSink sink; + google::AddLogSink(&sink); + + std::vector out(prepared.payload.size(), '\0'); + std::vector read_slices; + read_slices.emplace_back(Slice{out.data(), out.size()}); + + auto get = + runtime_.client->Get(prepared.key, prepared.query_result, read_slices); + + google::RemoveLogSink(&sink); + + ASSERT_TRUE(get.has_value()) << "Get failed: " << toString(get.error()); + ASSERT_EQ(std::memcmp(out.data(), prepared.payload.data(), out.size()), 0); + EXPECT_EQ(sink.strategy(), "TRANSFER_ENGINE"); +} + +class HotCacheRedirectStrategyTest + : public TcpLocalMemcpyAutoEnableTest, + public ::testing::WithParamInterface {}; + +TEST_P(HotCacheRedirectStrategyTest, CacheHitUsesLocalMemcpy) { + EnvGuard cache_size_guard("MC_STORE_LOCAL_HOT_CACHE_SIZE"); + setenv("MC_STORE_LOCAL_HOT_CACHE_SIZE", "33554432", 1); // 32MB + + const HandshakeMode mode = GetParam(); + const bool is_p2p = mode == HandshakeMode::P2P; + const std::string metadata_conn = is_p2p ? "P2PHANDSHAKE" : metadata_url_; + const std::string host_name = + is_p2p ? "localhost" : "metadata-hot-cache-host"; + + runtime_ = CreateRuntime(host_name, metadata_conn); + ASSERT_TRUE(runtime_.client != nullptr) << HandshakeModeName(mode); + ASSERT_TRUE(runtime_.client->IsHotCacheEnabled()) + << HandshakeModeName(mode); + + const std::string key = is_p2p ? "p2p_hot_cache_redirect_key" + : "metadata_hot_cache_redirect_key"; + const std::string payload = + is_p2p ? "p2p-hot-cache-data" : "metadata-hot-cache-data"; + + Slice cache_slice{const_cast(payload.data()), payload.size()}; + ASSERT_TRUE( + PutHotKeyHelper(*runtime_.client->GetHotCache(), key, cache_slice)); + ASSERT_TRUE(runtime_.client->GetHotCache()->HasHotKey(key)); + + Replica::Descriptor replica; + replica.id = is_p2p ? 1 : 2; + replica.status = ReplicaStatus::COMPLETE; + MemoryDescriptor mem_desc; + mem_desc.buffer_descriptor.transport_endpoint_ = "remote:9999"; + mem_desc.buffer_descriptor.buffer_address_ = 0; + mem_desc.buffer_descriptor.size_ = payload.size(); + replica.descriptor_variant = mem_desc; + + ASSERT_FALSE(runtime_.client->IsReplicaOnLocalMemory(replica)) + << HandshakeModeName(mode); + + std::vector replicas; + replicas.emplace_back(replica); + QueryResult query_result( + std::move(replicas), + std::chrono::steady_clock::now() + std::chrono::seconds(60)); + + StrategyCaptureSink sink; + google::AddLogSink(&sink); + + std::vector out(payload.size(), '\0'); + std::vector read_slices; + read_slices.emplace_back(Slice{out.data(), out.size()}); + + auto get = runtime_.client->Get(key, query_result, read_slices); + + google::RemoveLogSink(&sink); + + ASSERT_TRUE(get.has_value()) << "Get failed: " << toString(get.error()); + ASSERT_EQ(std::memcmp(out.data(), payload.data(), out.size()), 0); + EXPECT_EQ(sink.strategy(), "LOCAL_MEMCPY"); +} + +INSTANTIATE_TEST_SUITE_P( + AllModes, HotCacheRedirectStrategyTest, + ::testing::Values(HandshakeMode::P2P, HandshakeMode::Metadata), + [](const ::testing::TestParamInfo& info) { + switch (info.param) { + case HandshakeMode::P2P: + return "P2P"; + case HandshakeMode::Metadata: + return "Metadata"; + } + return "Unknown"; + }); + +} // namespace testing +} // namespace mooncake \ No newline at end of file -- 2.34.1 From 8a5dd546431bfbb3a36679ff710156bcb1bd2c59 Mon Sep 17 00:00:00 2001 From: yokinoshitayoki <113957881+yokinoshitayoki@users.noreply.github.com> Date: Tue, 12 May 2026 14:36:47 +0800 Subject: [PATCH 068/382] [Store] Add DSA-like workload allocation strategy and update allocator benchmarks (#2080) --- docs/source/index.md | 1 + .../allocation-strategy-benchmark-result.md | 191 ++++++++ .../performance/allocator-benchmark-result.md | 7 +- .../benchmarks/allocation_strategy_bench.cpp | 438 +++++++++++++++++- mooncake-store/benchmarks/allocator_bench.cpp | 77 +-- 5 files changed, 669 insertions(+), 45 deletions(-) create mode 100644 docs/source/performance/allocation-strategy-benchmark-result.md diff --git a/docs/source/index.md b/docs/source/index.md index 1d8d9702..4684256b 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -86,6 +86,7 @@ performance/vllm-benchmark-results-v1 performance/sglang-hicache-benchmark-results-v1 performance/vllm-v1-support-benchmark performance/allocator-benchmark-result +performance/allocation-strategy-benchmark-result performance/ssd-offload-benchmark-results ::: diff --git a/docs/source/performance/allocation-strategy-benchmark-result.md b/docs/source/performance/allocation-strategy-benchmark-result.md new file mode 100644 index 00000000..55d0d7ee --- /dev/null +++ b/docs/source/performance/allocation-strategy-benchmark-result.md @@ -0,0 +1,191 @@ +# AllocationStrategy Performance + +`allocation_strategy_bench` evaluates the allocation-routing overhead and behavior of `RandomAllocationStrategy` and `FreeRatioFirstAllocationStrategy` across multiple Mooncake Store segments. + +## Execution + +```bash +./mooncake-store/benchmarks/allocation_strategy_bench --workload=dsa +``` + +## Workload + +The benchmark compares two workload types: + +- **KV-only**: a general single-object-size allocation workload without indexer objects. This is used as the non-DSA baseline. +- **Paired KV+Indexer**: a DSA-style workload where KV cache objects and indexer objects are allocated together. +- KV cache object size: 3,274,752 B (3.12 MB) +- Indexer object size: 658,432 B (643 KB) +- Each round samples `N` uniformly from `[1, 128]`. +- In KV-only mode, each round issues `N` KV allocations. +- In paired KV+Indexer mode, each round issues `N` KV allocations followed by `N` indexer allocations. +- Segment capacity: 1 TB per segment by default. +- Segment counts: 1, 2, 4, 8, 16. +- Replica counts: 1, 2, 3 when enough segments are available. +- On allocation failure, the benchmark randomly evicts 5% of all live objects and retries. +- `UtilRatio_*` is sampled only when eviction is triggered, immediately before evicting objects. + +`skewed=no` uses uniform segment capacity. `skewed=yes` alternates segment capacity between 1.5x and 0.5x the base capacity. + +## Summary + +For KV-only, utilization reaches about 96.1%. For the DSA paired KV+indexer workload, both strategies reach about 95.2% utilization. The difference mainly comes from mixed object sizes in the paired workload. `RandomAllocationStrategy` generally has higher throughput and lower latency, while `FreeRatioFirstAllocationStrategy` tends to keep segment utilization more balanced. + +## Result + +KV-only is the non-DSA baseline. It represents the common case where Mooncake Store allocates only one object size. + +### KV-only / Skewed=no / Random + + +| Replica | Segments | AllocSize | Cluster(GB) | UtilRatio_min | UtilRatio_p99 | UtilRatio_p90 | UtilRatio_p50 | UtilRatio_max | UtilRatio_avg | Throughput | Avg(ns) | P50(ns) | P90(ns) | P99(ns) | UtilStdDev | Evictions(round) | Succ/Total | +| ------- | -------- | --------- | ----------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ---------- | ------- | ------- | ------- | ------- | ---------- | ---------------- | ----------------- | +| 1 | 1 | 3198KB | 1024.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 1015964 | 953 | 231 | 1132 | 1694 | 0.0000 | 63 | 1007262/1007262 | +| 1 | 2 | 3198KB | 2048.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 857559 | 1135 | 331 | 1633 | 2324 | 0.0001 | 63 | 2014525/2014525 | +| 2 | 2 | 3198KB | 2048.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 555934 | 1767 | 681 | 912 | 4388 | 0.0000 | 63 | 1007262/1007262 | +| 1 | 4 | 3198KB | 4096.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 809246 | 1204 | 340 | 1994 | 2435 | 0.0006 | 63 | 4029050/4029050 | +| 2 | 4 | 3198KB | 4096.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 503007 | 1957 | 772 | 1002 | 7545 | 0.0004 | 63 | 2014525/2014525 | +| 3 | 4 | 3198KB | 4096.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 377095 | 2620 | 1112 | 1412 | 6613 | 0.0002 | 63 | 1343016/1343016 | +| 1 | 8 | 3198KB | 8192.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 751926 | 1299 | 351 | 2124 | 2534 | 0.0004 | 63 | 8058100/8058100 | +| 2 | 8 | 3198KB | 8192.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 471589 | 2089 | 831 | 1053 | 7063 | 0.0002 | 63 | 4029050/4029050 | +| 3 | 8 | 3198KB | 8192.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 346496 | 2855 | 1203 | 1513 | 7133 | 0.0008 | 63 | 2686033/2686033 | +| 1 | 16 | 3198KB | 16384.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 691790 | 1409 | 371 | 2274 | 2835 | 0.0003 | 63 | 16116200/16116200 | +| 2 | 16 | 3198KB | 16384.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 434370 | 2271 | 892 | 1163 | 8135 | 0.0003 | 63 | 8058100/8058100 | +| 3 | 16 | 3198KB | 16384.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 322012 | 3074 | 1292 | 1613 | 8326 | 0.0006 | 63 | 5372067/5372067 | + + +### KV-only / Skewed=no / FreeRatioFirst + + +| Replica | Segments | AllocSize | Cluster(GB) | UtilRatio_min | UtilRatio_p99 | UtilRatio_p90 | UtilRatio_p50 | UtilRatio_max | UtilRatio_avg | Throughput | Avg(ns) | P50(ns) | P90(ns) | P99(ns) | UtilStdDev | Evictions(round) | Succ/Total | +| ------- | -------- | --------- | ----------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ---------- | ------- | ------- | ------- | ------- | ---------- | ---------------- | ----------------- | +| 1 | 1 | 3198KB | 1024.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 887098 | 1096 | 361 | 1293 | 1894 | 0.0000 | 63 | 1007262/1007262 | +| 1 | 2 | 3198KB | 2048.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 768834 | 1269 | 461 | 1803 | 2475 | 0.0000 | 63 | 2014525/2014525 | +| 2 | 2 | 3198KB | 2048.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 499411 | 1971 | 882 | 1152 | 4219 | 0.0000 | 63 | 1007262/1007262 | +| 1 | 4 | 3198KB | 4096.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 680538 | 1438 | 561 | 2234 | 2685 | 0.0000 | 63 | 4029050/4029050 | +| 2 | 4 | 3198KB | 4096.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 441341 | 2235 | 1052 | 1272 | 8336 | 0.0000 | 63 | 2014525/2014525 | +| 3 | 4 | 3198KB | 4096.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 328024 | 3017 | 1533 | 1844 | 6682 | 0.0000 | 63 | 1343016/1343016 | +| 1 | 8 | 3198KB | 8192.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 608814 | 1611 | 661 | 2435 | 2985 | 0.0000 | 63 | 8058100/8058100 | +| 2 | 8 | 3198KB | 8192.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 387271 | 2551 | 1292 | 1554 | 5621 | 0.0000 | 63 | 4029050/4029050 | +| 3 | 8 | 3198KB | 8192.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 295263 | 3355 | 1743 | 2014 | 7685 | 0.0000 | 63 | 2686033/2686033 | +| 1 | 16 | 3198KB | 16384.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 571007 | 1720 | 671 | 2645 | 3186 | 0.0000 | 63 | 16116200/16116200 | +| 2 | 16 | 3198KB | 16384.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 343893 | 2876 | 1503 | 1773 | 7825 | 0.0000 | 63 | 8058100/8058100 | +| 3 | 16 | 3198KB | 16384.0 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 0.960935 | 251734 | 3941 | 2124 | 2524 | 8346 | 0.0000 | 63 | 5372067/5372067 | + + +### KV-only / Skewed=yes / Random + + +| Replica | Segments | AllocSize | Cluster(GB) | UtilRatio_min | UtilRatio_p99 | UtilRatio_p90 | UtilRatio_p50 | UtilRatio_max | UtilRatio_avg | Throughput | Avg(ns) | P50(ns) | P90(ns) | P99(ns) | UtilStdDev | Evictions(round) | Succ/Total | +| ------- | -------- | --------- | ----------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ---------- | ------- | ------- | ------- | ------- | ---------- | ---------------- | ----------------- | +| 1 | 1 | 3198KB | 1536.0 | 0.960937 | 0.960937 | 0.960937 | 0.960937 | 0.960937 | 0.960937 | 925104 | 1050 | 301 | 1393 | 2124 | 0.0000 | 63 | 1510893/1510893 | +| 1 | 2 | 3198KB | 2048.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 846763 | 1149 | 331 | 1693 | 2354 | 0.0081 | 63 | 2014525/2014525 | +| 2 | 2 | 3198KB | 2048.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 763883 | 1278 | 512 | 741 | 4468 | 0.0119 | 36 | 1007262/1007262 | +| 1 | 4 | 3198KB | 4096.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 793128 | 1230 | 350 | 2014 | 2465 | 0.0078 | 63 | 4029050/4029050 | +| 2 | 4 | 3198KB | 4096.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 495155 | 1988 | 841 | 1092 | 6463 | 0.0043 | 63 | 2014525/2014525 | +| 3 | 4 | 3198KB | 4096.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 442797 | 2227 | 1001 | 1292 | 6593 | 0.0116 | 47 | 1343016/1343016 | +| 1 | 8 | 3198KB | 8192.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 742956 | 1315 | 370 | 2135 | 2565 | 0.0083 | 63 | 8058100/8058100 | +| 2 | 8 | 3198KB | 8192.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 449694 | 2192 | 912 | 1162 | 7835 | 0.0062 | 63 | 4029050/4029050 | +| 3 | 8 | 3198KB | 8192.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 335572 | 2949 | 1333 | 1663 | 6973 | 0.0037 | 63 | 2686033/2686033 | +| 1 | 16 | 3198KB | 16384.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 680546 | 1438 | 381 | 2295 | 2876 | 0.0081 | 63 | 16116200/16116200 | +| 2 | 16 | 3198KB | 16384.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 419847 | 2350 | 952 | 1223 | 8747 | 0.0069 | 63 | 8058100/8058100 | +| 3 | 16 | 3198KB | 16384.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 316027 | 3133 | 1403 | 1714 | 7764 | 0.0057 | 63 | 5372067/5372067 | + + +### KV-only / Skewed=yes / FreeRatioFirst + + +| Replica | Segments | AllocSize | Cluster(GB) | UtilRatio_min | UtilRatio_p99 | UtilRatio_p90 | UtilRatio_p50 | UtilRatio_max | UtilRatio_avg | Throughput | Avg(ns) | P50(ns) | P90(ns) | P99(ns) | UtilStdDev | Evictions(round) | Succ/Total | +| ------- | -------- | --------- | ----------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ---------- | ------- | ------- | ------- | ------- | ---------- | ---------------- | ----------------- | +| 1 | 1 | 3198KB | 1536.0 | 0.960937 | 0.960937 | 0.960937 | 0.960937 | 0.960937 | 0.960937 | 828717 | 1176 | 431 | 1494 | 2265 | 0.0000 | 63 | 1510893/1510893 | +| 1 | 2 | 3198KB | 2048.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 770115 | 1267 | 461 | 1774 | 2465 | 0.0000 | 63 | 2014525/2014525 | +| 2 | 2 | 3198KB | 2048.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 655606 | 1492 | 731 | 981 | 4649 | 0.0119 | 36 | 1007262/1007262 | +| 1 | 4 | 3198KB | 4096.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 691428 | 1415 | 541 | 2234 | 2675 | 0.0000 | 63 | 4029050/4029050 | +| 2 | 4 | 3198KB | 4096.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 444332 | 2219 | 1062 | 1323 | 5570 | 0.0000 | 63 | 2014525/2014525 | +| 3 | 4 | 3198KB | 4096.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 347967 | 2842 | 1493 | 1793 | 7003 | 0.0004 | 54 | 1343016/1343016 | +| 1 | 8 | 3198KB | 8192.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 611508 | 1604 | 661 | 2445 | 2976 | 0.0000 | 63 | 8058100/8058100 | +| 2 | 8 | 3198KB | 8192.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 383486 | 2576 | 1273 | 1513 | 8295 | 0.0000 | 63 | 4029050/4029050 | +| 3 | 8 | 3198KB | 8192.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 292720 | 3385 | 1773 | 2054 | 7694 | 0.0000 | 63 | 2686033/2686033 | +| 1 | 16 | 3198KB | 16384.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 573570 | 1712 | 671 | 2654 | 3196 | 0.0000 | 63 | 16116200/16116200 | +| 2 | 16 | 3198KB | 16384.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 346109 | 2858 | 1503 | 1763 | 9328 | 0.0000 | 63 | 8058100/8058100 | +| 3 | 16 | 3198KB | 16384.0 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 0.960936 | 252816 | 3924 | 2144 | 2495 | 8496 | 0.0000 | 63 | 5372067/5372067 | + + +Paired KV+Indexer is the DSA workload. It allocates two object sizes in the same segment pool. + +### Paired (KV + Indexer) / Skewed=no / Random + + +| Replica | Segments | AllocSize | Cluster(GB) | UtilRatio_min | UtilRatio_p99 | UtilRatio_p90 | UtilRatio_p50 | UtilRatio_max | UtilRatio_avg | Throughput | Avg(ns) | P50(ns) | P90(ns) | P99(ns) | UtilStdDev | Evictions(round) | Succ/Total | +| ------- | -------- | ---------- | ----------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ---------- | ------- | ------- | ------- | ------- | ---------- | ---------------- | ----------------- | +| 1 | 1 | 3198+643KB | 1024.0 | 0.952089 | 0.952089 | 0.952183 | 0.952498 | 0.952621 | 0.952437 | 921322 | 1049 | 310 | 1412 | 2164 | 0.0000 | 62 | 1677285/1677285 | +| 1 | 2 | 3198+643KB | 2048.0 | 0.952162 | 0.952162 | 0.952250 | 0.952451 | 0.952627 | 0.952426 | 810735 | 1197 | 340 | 1964 | 2435 | 0.0001 | 64 | 3354570/3354570 | +| 2 | 2 | 3198+643KB | 2048.0 | 0.952089 | 0.952089 | 0.952183 | 0.952498 | 0.952621 | 0.952437 | 517602 | 1901 | 761 | 1042 | 5049 | 0.0000 | 62 | 1677285/1677285 | +| 1 | 4 | 3198+643KB | 4096.0 | 0.952316 | 0.952316 | 0.952360 | 0.952464 | 0.952611 | 0.952467 | 701903 | 1387 | 371 | 2164 | 3025 | 0.0003 | 63 | 6709139/6709139 | +| 2 | 4 | 3198+643KB | 4096.0 | 0.952195 | 0.952195 | 0.952286 | 0.952501 | 0.952607 | 0.952459 | 429714 | 2292 | 852 | 1132 | 5821 | 0.0005 | 63 | 3354570/3354570 | +| 3 | 4 | 3198+643KB | 4096.0 | 0.952201 | 0.952201 | 0.952322 | 0.952515 | 0.952627 | 0.952488 | 360298 | 2744 | 1173 | 1473 | 7574 | 0.0002 | 63 | 2236380/2236380 | +| 1 | 8 | 3198+643KB | 8192.0 | 0.952320 | 0.952320 | 0.952351 | 0.952421 | 0.952586 | 0.952428 | 701440 | 1389 | 361 | 2234 | 2846 | 0.0005 | 63 | 13418278/13418278 | +| 2 | 8 | 3198+643KB | 8192.0 | 0.952322 | 0.952322 | 0.952345 | 0.952424 | 0.952618 | 0.952449 | 435206 | 2266 | 882 | 1222 | 6001 | 0.0005 | 63 | 6709139/6709139 | +| 3 | 8 | 3198+643KB | 8192.0 | 0.952278 | 0.952278 | 0.952318 | 0.952444 | 0.952618 | 0.952456 | 330833 | 2991 | 1272 | 1574 | 8035 | 0.0004 | 63 | 4472760/4472760 | +| 1 | 16 | 3198+643KB | 16384.0 | 0.952246 | 0.952246 | 0.952310 | 0.952412 | 0.952580 | 0.952395 | 643565 | 1517 | 380 | 2534 | 3196 | 0.0004 | 64 | 26836557/26836557 | +| 2 | 16 | 3198+643KB | 16384.0 | 0.952338 | 0.952338 | 0.952386 | 0.952448 | 0.952580 | 0.952449 | 405947 | 2432 | 942 | 1313 | 6803 | 0.0005 | 63 | 13418278/13418278 | +| 3 | 16 | 3198+643KB | 16384.0 | 0.952169 | 0.952169 | 0.952259 | 0.952434 | 0.952622 | 0.952423 | 313732 | 3156 | 1323 | 1654 | 9017 | 0.0005 | 63 | 8945519/8945519 | + + +### Paired (KV + Indexer) / Skewed=no / FreeRatioFirst + + +| Replica | Segments | AllocSize | Cluster(GB) | UtilRatio_min | UtilRatio_p99 | UtilRatio_p90 | UtilRatio_p50 | UtilRatio_max | UtilRatio_avg | Throughput | Avg(ns) | P50(ns) | P90(ns) | P99(ns) | UtilStdDev | Evictions(round) | Succ/Total | +| ------- | -------- | ---------- | ----------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ---------- | ------- | ------- | ------- | ------- | ---------- | ---------------- | ----------------- | +| 1 | 1 | 3198+643KB | 1024.0 | 0.952089 | 0.952089 | 0.952183 | 0.952498 | 0.952621 | 0.952437 | 823450 | 1183 | 431 | 1584 | 2344 | 0.0000 | 62 | 1677285/1677285 | +| 1 | 2 | 3198+643KB | 2048.0 | 0.952172 | 0.952172 | 0.952349 | 0.952462 | 0.952627 | 0.952452 | 740063 | 1320 | 472 | 2084 | 2575 | 0.0000 | 63 | 3354570/3354570 | +| 2 | 2 | 3198+643KB | 2048.0 | 0.952089 | 0.952089 | 0.952183 | 0.952498 | 0.952621 | 0.952437 | 470878 | 2093 | 982 | 1242 | 6462 | 0.0000 | 62 | 1677285/1677285 | +| 1 | 4 | 3198+643KB | 4096.0 | 0.952345 | 0.952345 | 0.952395 | 0.952481 | 0.952575 | 0.952471 | 659232 | 1484 | 551 | 2305 | 2785 | 0.0000 | 64 | 6709139/6709139 | +| 2 | 4 | 3198+643KB | 4096.0 | 0.952323 | 0.952323 | 0.952393 | 0.952493 | 0.952627 | 0.952490 | 429940 | 2295 | 1092 | 1313 | 7514 | 0.0000 | 63 | 3354570/3354570 | +| 3 | 4 | 3198+643KB | 4096.0 | 0.952180 | 0.952180 | 0.952356 | 0.952509 | 0.952606 | 0.952481 | 318057 | 3113 | 1554 | 1864 | 7564 | 0.0000 | 63 | 2236380/2236380 | +| 1 | 8 | 3198+643KB | 8192.0 | 0.952316 | 0.952316 | 0.952349 | 0.952459 | 0.952594 | 0.952455 | 580198 | 1692 | 671 | 2585 | 3126 | 0.0002 | 64 | 13418278/13418278 | +| 2 | 8 | 3198+643KB | 8192.0 | 0.952221 | 0.952221 | 0.952316 | 0.952486 | 0.952606 | 0.952450 | 371743 | 2659 | 1332 | 1593 | 6593 | 0.0002 | 64 | 6709139/6709139 | +| 3 | 8 | 3198+643KB | 8192.0 | 0.952215 | 0.952215 | 0.952316 | 0.952470 | 0.952618 | 0.952455 | 282562 | 3508 | 1814 | 2114 | 8115 | 0.0000 | 63 | 4472760/4472760 | +| 1 | 16 | 3198+643KB | 16384.0 | 0.952328 | 0.952328 | 0.952368 | 0.952423 | 0.952570 | 0.952436 | 546995 | 1797 | 682 | 2825 | 3466 | 0.0000 | 63 | 26836557/26836557 | +| 2 | 16 | 3198+643KB | 16384.0 | 0.952288 | 0.952288 | 0.952343 | 0.952419 | 0.952589 | 0.952435 | 325312 | 3043 | 1583 | 1894 | 10300 | 0.0000 | 63 | 13418278/13418278 | +| 3 | 16 | 3198+643KB | 16384.0 | 0.952282 | 0.952282 | 0.952355 | 0.952458 | 0.952615 | 0.952453 | 242750 | 4088 | 2245 | 2585 | 10350 | 0.0000 | 63 | 8945519/8945519 | + + +### Paired (KV + Indexer) / Skewed=yes / Random + + +| Replica | Segments | AllocSize | Cluster(GB) | UtilRatio_min | UtilRatio_p99 | UtilRatio_p90 | UtilRatio_p50 | UtilRatio_max | UtilRatio_avg | Throughput | Avg(ns) | P50(ns) | P90(ns) | P99(ns) | UtilStdDev | Evictions(round) | Succ/Total | +| ------- | -------- | ---------- | ----------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ---------- | ------- | ------- | ------- | ------- | ---------- | ---------------- | ----------------- | +| 1 | 1 | 3198+643KB | 1536.0 | 0.952196 | 0.952196 | 0.952295 | 0.952475 | 0.952628 | 0.952470 | 857346 | 1135 | 311 | 1643 | 2354 | 0.0000 | 63 | 2515927/2515927 | +| 1 | 2 | 3198+643KB | 2048.0 | 0.952270 | 0.952270 | 0.952310 | 0.952441 | 0.952614 | 0.952444 | 805462 | 1210 | 341 | 1984 | 2474 | 0.0009 | 63 | 3354570/3354570 | +| 2 | 2 | 3198+643KB | 2048.0 | 0.952421 | 0.952421 | 0.952443 | 0.952507 | 0.952630 | 0.952517 | 720532 | 1357 | 561 | 851 | 5120 | 0.0004 | 37 | 1677285/1677285 | +| 1 | 4 | 3198+643KB | 4096.0 | 0.952181 | 0.952181 | 0.952313 | 0.952406 | 0.952634 | 0.952417 | 717110 | 1363 | 361 | 2235 | 2856 | 0.0001 | 63 | 6709139/6709139 | +| 2 | 4 | 3198+643KB | 4096.0 | 0.952268 | 0.952268 | 0.952364 | 0.952494 | 0.952617 | 0.952484 | 458283 | 2151 | 901 | 1192 | 5511 | 0.0051 | 63 | 3354570/3354570 | +| 3 | 4 | 3198+643KB | 4096.0 | 0.952307 | 0.952307 | 0.952409 | 0.952500 | 0.952630 | 0.952499 | 420616 | 2346 | 1052 | 1382 | 7073 | 0.0070 | 48 | 2236380/2236380 | +| 1 | 8 | 3198+643KB | 8192.0 | 0.952310 | 0.952310 | 0.952340 | 0.952376 | 0.952612 | 0.952405 | 694136 | 1410 | 371 | 2244 | 2786 | 0.0003 | 64 | 13418278/13418278 | +| 2 | 8 | 3198+643KB | 8192.0 | 0.952353 | 0.952353 | 0.952384 | 0.952445 | 0.952594 | 0.952456 | 433396 | 2276 | 942 | 1272 | 5931 | 0.0017 | 63 | 6709139/6709139 | +| 3 | 8 | 3198+643KB | 8192.0 | 0.952324 | 0.952324 | 0.952357 | 0.952463 | 0.952608 | 0.952456 | 329256 | 3006 | 1363 | 1673 | 7003 | 0.0047 | 63 | 4472760/4472760 | +| 1 | 16 | 3198+643KB | 16384.0 | 0.952264 | 0.952264 | 0.952299 | 0.952343 | 0.952590 | 0.952375 | 641829 | 1521 | 381 | 2565 | 3256 | 0.0003 | 63 | 26836557/26836557 | +| 2 | 16 | 3198+643KB | 16384.0 | 0.952246 | 0.952246 | 0.952336 | 0.952423 | 0.952608 | 0.952426 | 399689 | 2470 | 1012 | 1403 | 6652 | 0.0011 | 63 | 13418278/13418278 | +| 3 | 16 | 3198+643KB | 16384.0 | 0.952300 | 0.952300 | 0.952360 | 0.952451 | 0.952606 | 0.952447 | 303763 | 3261 | 1433 | 1763 | 9107 | 0.0021 | 63 | 8945519/8945519 | + + +### Paired (KV + Indexer) / Skewed=yes / FreeRatioFirst + + +| Replica | Segments | AllocSize | Cluster(GB) | UtilRatio_min | UtilRatio_p99 | UtilRatio_p90 | UtilRatio_p50 | UtilRatio_max | UtilRatio_avg | Throughput | Avg(ns) | P50(ns) | P90(ns) | P99(ns) | UtilStdDev | Evictions(round) | Succ/Total | +| ------- | -------- | ---------- | ----------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ---------- | ------- | ------- | ------- | ------- | ---------- | ---------------- | ----------------- | +| 1 | 1 | 3198+643KB | 1536.0 | 0.952196 | 0.952196 | 0.952295 | 0.952475 | 0.952628 | 0.952470 | 775651 | 1258 | 441 | 1783 | 2525 | 0.0000 | 63 | 2515927/2515927 | +| 1 | 2 | 3198+643KB | 2048.0 | 0.952206 | 0.952206 | 0.952284 | 0.952539 | 0.952758 | 0.952507 | 732598 | 1334 | 481 | 2064 | 2565 | 0.0000 | 64 | 3354570/3354570 | +| 2 | 2 | 3198+643KB | 2048.0 | 0.952421 | 0.952421 | 0.952443 | 0.952507 | 0.952630 | 0.952517 | 624654 | 1570 | 791 | 1042 | 5070 | 0.0004 | 37 | 1677285/1677285 | +| 1 | 4 | 3198+643KB | 4096.0 | 0.952378 | 0.952378 | 0.952465 | 0.952601 | 0.952898 | 0.952610 | 667880 | 1466 | 561 | 2304 | 2735 | 0.0000 | 63 | 6709139/6709139 | +| 2 | 4 | 3198+643KB | 4096.0 | 0.952241 | 0.952241 | 0.952341 | 0.952545 | 0.952758 | 0.952532 | 424387 | 2325 | 1112 | 1373 | 5290 | 0.0006 | 64 | 3354570/3354570 | +| 3 | 4 | 3198+643KB | 4096.0 | 0.952294 | 0.952294 | 0.952346 | 0.952524 | 0.952630 | 0.952503 | 332752 | 2974 | 1543 | 1864 | 7164 | 0.0075 | 55 | 2236380/2236380 | +| 1 | 8 | 3198+643KB | 8192.0 | 0.952463 | 0.952463 | 0.952498 | 0.952691 | 0.953072 | 0.952725 | 584455 | 1679 | 671 | 2576 | 3116 | 0.0004 | 64 | 13418278/13418278 | +| 2 | 8 | 3198+643KB | 8192.0 | 0.952329 | 0.952329 | 0.952433 | 0.952605 | 0.952899 | 0.952602 | 371745 | 2659 | 1323 | 1592 | 9427 | 0.0000 | 64 | 6709139/6709139 | +| 3 | 8 | 3198+643KB | 8192.0 | 0.952365 | 0.952365 | 0.952410 | 0.952588 | 0.952800 | 0.952578 | 281189 | 3525 | 1853 | 2154 | 7875 | 0.0000 | 63 | 4472760/4472760 | +| 1 | 16 | 3198+643KB | 16384.0 | 0.952303 | 0.952303 | 0.952374 | 0.952830 | 0.953388 | 0.952796 | 540381 | 1814 | 691 | 2875 | 3527 | 0.0003 | 64 | 26836557/26836557 | +| 2 | 16 | 3198+643KB | 16384.0 | 0.952518 | 0.952518 | 0.952539 | 0.952636 | 0.953128 | 0.952681 | 320795 | 3086 | 1603 | 1974 | 7744 | 0.0005 | 64 | 13418278/13418278 | +| 3 | 16 | 3198+643KB | 16384.0 | 0.952450 | 0.952450 | 0.952512 | 0.952592 | 0.952953 | 0.952614 | 240086 | 4134 | 2274 | 2625 | 10470 | 0.0000 | 63 | 8945519/8945519 | + + diff --git a/docs/source/performance/allocator-benchmark-result.md b/docs/source/performance/allocator-benchmark-result.md index 0e10b242..fd7c9a5d 100644 --- a/docs/source/performance/allocator-benchmark-result.md +++ b/docs/source/performance/allocator-benchmark-result.md @@ -214,10 +214,13 @@ In this configuration, Mooncake Store stores data at the page granularity, where - KV cache object size: 3.12 MB - Indexer object size: 643 KB +Each benchmark round samples `N` uniformly from `[1, 128]`, then issues `N` KV cache allocations followed by `N` indexer allocations. The pool size is 1024 GB. On allocation failure, the benchmark randomly evicts 5% of all live objects and retries. The utilization ratio below is sampled only when eviction is triggered, immediately before evicting objects. + **OffsetAllocator** ``` util ratio (min / p99 / p90 / p50 / max / avg): -0.948299 / 0.948765 / 0.949311 / 0.949884 / 0.952491 / 0.950091 -avg alloc time: 232.420141 ns/op +0.951814 / 0.951879 / 0.952077 / 0.952382 / 0.952621 / 0.952343 +eviction-trigger samples: 243 +avg alloc time: 352.207986 ns/op ``` diff --git a/mooncake-store/benchmarks/allocation_strategy_bench.cpp b/mooncake-store/benchmarks/allocation_strategy_bench.cpp index 9db7d546..6e915565 100644 --- a/mooncake-store/benchmarks/allocation_strategy_bench.cpp +++ b/mooncake-store/benchmarks/allocation_strategy_bench.cpp @@ -5,6 +5,8 @@ #include #include #include +#include +#include #include #include #include @@ -27,7 +29,8 @@ DEFINE_bool( "Also run the Scale-Out matrix in addition to the default Fillup matrix"); // Scale-Out workload flags -DEFINE_string(workload, "fillup", "Workload type: fillup (default), scaleout"); +DEFINE_string(workload, "fillup", + "Workload type: fillup (default), scaleout, dsa"); DEFINE_int32( scale_out_trigger_pct, 50, "Pre-fill cluster to this utilization % before injecting new nodes " @@ -42,6 +45,34 @@ DEFINE_int32( "measured allocation loop (0 = disabled, original behavior). " "When >0, early exit on consecutive failures is suppressed so that " "all num_allocations are attempted under near-full conditions."); +DEFINE_int64(dsa_segment_capacity, 1024ULL * 1024, + "Per-segment capacity in MB for DSA workload"); +DEFINE_double(dsa_capacity_multiplier, 3.0, + "DSA workload measures (multiplier * cluster_capacity) worth " + "of allocations under steady-state churn. Combined with " + "--dsa_warmup_multiplier, total per-case work is " + "(warmup + capacity) * cluster_capacity, clamped to " + "[kDsaMinAllocs, --dsa_max_allocs]; --num_allocations is " + "ignored in DSA mode."); +DEFINE_double(dsa_warmup_multiplier, 1.0, + "DSA warmup-phase write volume in units of cluster_capacity, " + "executed BEFORE the measurement window so the cluster is in " + "post-fill eviction churn when stats start being collected. " + "All stats during warmup (latencies, util_ratios, evict " + "rounds, succ/total) are discarded. Set to 0.0 to disable " + "warmup entirely (legacy behavior)."); +DEFINE_int32(dsa_max_allocs, 50000000, + "Per-case upper bound on the auto-derived DSA allocation count. " + "Raise this if you use very large segments (e.g. 100GB+) and " + "want big clusters to actually reach steady-state eviction; " + "lower it to bound benchmark runtime. Each allocation costs ~1-3 " + "us, so the default upper bound can take minutes per saturated " + "case."); +DEFINE_double(dsa_evict_ratio, 0.05, + "Fraction of live objects to evict on each Allocate failure " + "(default 0.05 = 5%). Lower values keep " + "the steady-state cluster fill closer to the fragmentation " + "ceiling but trigger evictions more frequently."); using namespace mooncake; @@ -58,12 +89,21 @@ constexpr size_t kMaxExpectedAllocs = 600000; constexpr double kLargeClusterThresholdGB = 500.0; constexpr int kPreFillSampleInterval = 100; constexpr int kMinMeasurementAllocs = 100; -constexpr int kMaxMeasurementAllocs = 200000; +constexpr int kMaxMeasurementAllocs = 50000000; constexpr int kMinSamplesForConvergence = 10; +// DSA workload sizes (DeepSeek defaults; adjust in source for other models). +constexpr size_t kDsaKvSize = 3274752; // ~3.12 MB per KV page +constexpr size_t kDsaIndexerSize = 658432; // 643 KB per indexer entry +constexpr int kDsaMaxBatch = 128; // per-round batch upper bound +constexpr int kDsaMaxRetries = 5; +// Lower bound on the auto-derived allocation count per DSA case. +constexpr int kDsaMinAllocs = 100; + enum class WorkloadType { FILL_UP, // Only allocate, measure throughput/latency SCALE_OUT, // Inject new nodes mid-run, measure adoption speed + DSA, // DSA paired KV+indexer with random fail-triggered eviction }; struct BenchConfig { @@ -84,6 +124,19 @@ struct BenchConfig { WorkloadType workload_type; int scale_out_trigger_pct; int scale_out_new_segments; + + // DSA workload knobs (only used when workload_type == DSA). + bool dsa_paired = false; // false = KV-only, true = KV+indexer pair +}; + +struct UtilRatioStats { + double min = 0.0; + double p99 = 0.0; + double p90 = 0.0; + double p50 = 0.0; + double max = 0.0; + double avg = 0.0; + bool valid = false; }; /** @@ -113,6 +166,17 @@ struct BenchResultBase { double final_util_stddev; // utilization stddev at run end double final_avg_util; // average utilization at run end + // Number of times fail-triggered eviction fired (DSA workload only; + // remains 0 for fill-up / scale-out). + int evict_count = 0; + + // DSA paired flag controls how AllocSize is rendered in print output. + // Defaults to false so existing fill-up / scale-out output is unchanged. + bool dsa_paired = false; + + // Temporal cluster utilization stats sampled during the measured loop. + UtilRatioStats util_ratio_stats; + virtual ~BenchResultBase() = default; }; @@ -151,9 +215,9 @@ static double computeClusterCapacityGB(int num_segments, size_t base_capacity, static void setupResourceLimits() { struct rlimit rl; - // Cap virtual address space (RLIMIT_AS) to 4TB - rl.rlim_cur = 4096ULL * 1024 * 1024 * 1024; - rl.rlim_max = 4096ULL * 1024 * 1024 * 1024; + // DSA cases simulate multi-TB clusters via virtual address space. + rl.rlim_cur = 200ULL * 1024 * 1024 * 1024 * 1024; + rl.rlim_max = 200ULL * 1024 * 1024 * 1024 * 1024; setrlimit(RLIMIT_AS, &rl); rl.rlim_cur = RLIM_INFINITY; @@ -304,6 +368,29 @@ static double computeAverageUtil( return count > 0 ? sum / count : 0.0; } +static UtilRatioStats computeUtilRatioStats(std::vector& util_ratios) { + UtilRatioStats stats; + if (util_ratios.empty()) return stats; + + std::sort(util_ratios.begin(), util_ratios.end()); + + auto percentile = [&](double p) -> double { + size_t idx = static_cast(util_ratios.size() * p); + if (idx >= util_ratios.size()) idx = util_ratios.size() - 1; + return util_ratios[idx]; + }; + + stats.min = util_ratios.front(); + stats.p99 = percentile(0.01); + stats.p90 = percentile(0.10); + stats.p50 = percentile(0.50); + stats.max = util_ratios.back(); + stats.avg = std::accumulate(util_ratios.begin(), util_ratios.end(), 0.0) / + util_ratios.size(); + stats.valid = true; + return stats; +} + static std::string strategyName(AllocationStrategyType type) { switch (type) { case AllocationStrategyType::RANDOM: @@ -667,8 +754,176 @@ static ScaleOutResult runScaleOutBenchmark(const BenchConfig& cfg) { return res; } +static void evictRandomFraction(std::vector>& live, + double ratio, std::mt19937& rng) { + if (live.empty()) return; + + size_t to_drop = + std::max(1, static_cast(live.size() * ratio)); + if (to_drop > live.size()) to_drop = live.size(); + + for (size_t i = 0; i < to_drop; ++i) { + std::uniform_int_distribution dist(0, live.size() - 1); + size_t idx = dist(rng); + std::swap(live[idx], live.back()); + live.pop_back(); // Replica destructor returns memory to allocator. + } +} + +// Try to allocate once; on failure, sample utilization before eviction. +static bool dsaAllocateWithEvict( + const std::shared_ptr& strategy, + AllocatorManager& manager, size_t size, int replica_num, + std::vector>& live, std::mt19937& rng, + int& evict_count, double evict_ratio, + std::vector* pre_evict_util = nullptr) { + for (int attempt = 0; attempt <= kDsaMaxRetries; ++attempt) { + auto result = strategy->Allocate(manager, size, replica_num); + if (result.has_value()) { + live.push_back(std::move(result.value())); + return true; + } + + if (live.empty()) return false; + if (attempt == kDsaMaxRetries) return false; + + if (attempt == 0 && pre_evict_util != nullptr) { + pre_evict_util->push_back(computeAverageUtilAll(manager)); + } + + evictRandomFraction(live, evict_ratio, rng); + ++evict_count; + } + + return false; +} + +// Run DSA workload; the allocation count is derived from cluster capacity. +static FillUpResult runDsaBenchmark(const BenchConfig& cfg) { + AllocatorManager manager = + createCluster(cfg.num_segments, cfg.segment_capacity, cfg.skewed); + auto strategy = CreateAllocationStrategy(cfg.strategy_type); + + const size_t total_capacity = computeTotalCapacity(manager); + const size_t avg_obj_size = + cfg.dsa_paired ? (kDsaKvSize + kDsaIndexerSize) / 2 : kDsaKvSize; + const size_t bytes_per_alloc = avg_obj_size * cfg.replica_num; + const double warmup_mult = std::max(0.0, FLAGS_dsa_warmup_multiplier); + const double measure_mult = std::max(0.0, FLAGS_dsa_capacity_multiplier); + const double total_mult = warmup_mult + measure_mult; + + int total_allocs = + (bytes_per_alloc == 0) + ? kDsaMinAllocs + : static_cast(total_mult * total_capacity / bytes_per_alloc); + total_allocs = + std::clamp(total_allocs, kDsaMinAllocs, FLAGS_dsa_max_allocs); + int warmup_allocs = + (total_mult > 0.0) + ? static_cast(total_allocs * (warmup_mult / total_mult)) + : 0; + int measure_allocs = std::max(0, total_allocs - warmup_allocs); + + std::vector latencies; + latencies.reserve(measure_allocs); + + std::vector util_ratios; + util_ratios.reserve(measure_allocs); + + // Use one live set so eviction matches allocator_bench. + std::vector> live_allocations; + live_allocations.reserve(std::min(total_allocs, 1 << 20)); + + std::mt19937 rng(42); + std::uniform_int_distribution batch_dist(1, kDsaMaxBatch); + + if (warmup_allocs > 0) { + int warmup_count = 0; + int warmup_evict_throwaway = 0; + auto warmup_run = [&](size_t size, + std::vector>& live) { + (void)dsaAllocateWithEvict(strategy, manager, size, cfg.replica_num, + live, rng, warmup_evict_throwaway, + FLAGS_dsa_evict_ratio); + ++warmup_count; + }; + + while (warmup_count < warmup_allocs) { + int batch = batch_dist(rng); + for (int i = 0; i < batch && warmup_count < warmup_allocs; ++i) { + warmup_run(kDsaKvSize, live_allocations); + } + if (cfg.dsa_paired) { + for (int i = 0; i < batch && warmup_count < warmup_allocs; + ++i) { + warmup_run(kDsaIndexerSize, live_allocations); + } + } + } + } + + int success_count = 0; + int total_count = 0; + int evict_count = 0; + + auto run_one = [&](size_t size, + std::vector>& live) -> bool { + auto t0 = std::chrono::high_resolution_clock::now(); + bool ok = dsaAllocateWithEvict(strategy, manager, size, cfg.replica_num, + live, rng, evict_count, + FLAGS_dsa_evict_ratio, &util_ratios); + auto t1 = std::chrono::high_resolution_clock::now(); + + latencies.push_back( + std::chrono::duration(t1 - t0).count()); + ++total_count; + if (ok) ++success_count; + + return ok; + }; + + auto total_start = std::chrono::high_resolution_clock::now(); + + while (total_count < measure_allocs) { + int batch = batch_dist(rng); + + for (int i = 0; i < batch && total_count < measure_allocs; ++i) { + run_one(kDsaKvSize, live_allocations); + } + + if (cfg.dsa_paired) { + for (int i = 0; i < batch && total_count < measure_allocs; ++i) { + run_one(kDsaIndexerSize, live_allocations); + } + } + } + + auto total_end = std::chrono::high_resolution_clock::now(); + double total_us = + std::chrono::duration(total_end - total_start) + .count(); + + FillUpResult res; + res.strategy_name = cfg.strategy_name; + res.num_segments = cfg.num_segments; + res.alloc_size = cfg.alloc_size; // KV size; AllocSize column displays it. + res.replica_num = cfg.replica_num; + res.skewed = cfg.skewed; + res.cluster_capacity_gb = computeClusterCapacityGB( + cfg.num_segments, cfg.segment_capacity, cfg.skewed); + res.final_util_stddev = computeUtilizationStdDev(manager); + res.final_avg_util = computeAverageUtilAll(manager); + res.success_count = success_count; + res.total_count = total_count; + res.evict_count = evict_count; + res.dsa_paired = cfg.dsa_paired; + res.util_ratio_stats = computeUtilRatioStats(util_ratios); + computeLatencyStats(latencies, total_us, total_count, res); + return res; +} + static void printFillUpHeader() { - std::cout << std::string(170, '-') << std::endl; + std::cout << std::string(184, '-') << std::endl; std::cout << std::left << std::setw(18) << "Strategy" << std::setw(9) << "Replica" << std::setw(10) << "Segments" << std::setw(12) << "AllocSize" << std::setw(12) << "Cluster(GB)" << std::setw(8) @@ -676,8 +931,9 @@ static void printFillUpHeader() { << std::setw(12) << "Avg(ns)" << std::setw(12) << "P50(ns)" << std::setw(12) << "P90(ns)" << std::setw(12) << "P99(ns)" << std::setw(12) << "UtilStdDev" << std::setw(10) << "AvgUtil%" - << std::setw(15) << "Succ/Total" << std::endl; - std::cout << std::string(170, '-') << std::endl; + << std::setw(15) << "Succ/Total" << std::setw(14) << "Evictions" + << std::endl; + std::cout << std::string(184, '-') << std::endl; } static void printFillUpResult(const FillUpResult& r) { @@ -685,18 +941,66 @@ static void printFillUpResult(const FillUpResult& r) { std::to_string(r.success_count) + "/" + std::to_string(r.total_count); std::ostringstream cap_ss; cap_ss << std::fixed << std::setprecision(1) << r.cluster_capacity_gb; + std::string alloc_size_str = + r.dsa_paired ? (std::to_string(r.alloc_size / KiB) + "+" + + std::to_string(kDsaIndexerSize / KiB) + "KB") + : (std::to_string(r.alloc_size / KiB) + "KB"); std::cout << std::left << std::setw(18) << r.strategy_name << std::setw(9) << r.replica_num << std::setw(10) << r.num_segments - << std::setw(12) << (std::to_string(r.alloc_size / KiB) + "KB") - << std::setw(12) << cap_ss.str() << std::setw(8) - << (r.skewed ? "yes" : "no") << std::right << std::fixed + << std::setw(12) << alloc_size_str << std::setw(12) + << cap_ss.str() << std::setw(8) << (r.skewed ? "yes" : "no") + << std::right << std::fixed << std::setprecision(0) + << std::setw(14) << r.throughput << std::setw(12) << r.avg_ns + << std::setw(12) << r.p50_ns << std::setw(12) << r.p90_ns + << std::setw(12) << r.p99_ns << std::setprecision(4) + << std::setw(12) << r.final_util_stddev << std::setprecision(2) + << std::setw(9) << (r.final_avg_util * 100.0) << "%" + << std::setw(15) << alloc_ratio << std::setw(14) << r.evict_count + << std::endl; +} + +static void printDsaHeader() { + std::cout << std::string(264, '-') << std::endl; + std::cout << std::left << std::setw(18) << "Strategy" << std::setw(9) + << "Replica" << std::setw(10) << "Segments" << std::setw(12) + << "AllocSize" << std::setw(12) << "Cluster(GB)" << std::setw(8) + << "Skewed" << std::right << std::setw(14) << "UtilRatio_min" + << std::setw(14) << "UtilRatio_p99" << std::setw(14) + << "UtilRatio_p90" << std::setw(14) << "UtilRatio_p50" + << std::setw(14) << "UtilRatio_max" << std::setw(14) + << "UtilRatio_avg" << std::setw(14) << "Throughput" + << std::setw(12) << "Avg(ns)" << std::setw(12) << "P50(ns)" + << std::setw(12) << "P90(ns)" << std::setw(12) << "P99(ns)" + << std::setw(12) << "UtilStdDev" << std::setw(17) + << "Evictions(round)" << std::setw(20) << "Succ/Total" + << std::endl; + std::cout << std::string(264, '-') << std::endl; +} + +static void printDsaResult(const FillUpResult& r) { + std::string alloc_ratio = + std::to_string(r.success_count) + "/" + std::to_string(r.total_count); + std::ostringstream cap_ss; + cap_ss << std::fixed << std::setprecision(1) << r.cluster_capacity_gb; + std::string alloc_size_str = + r.dsa_paired ? (std::to_string(r.alloc_size / KiB) + "+" + + std::to_string(kDsaIndexerSize / KiB) + "KB") + : (std::to_string(r.alloc_size / KiB) + "KB"); + const auto& urs = r.util_ratio_stats; + std::cout << std::left << std::setw(18) << r.strategy_name << std::setw(9) + << r.replica_num << std::setw(10) << r.num_segments + << std::setw(12) << alloc_size_str << std::setw(12) + << cap_ss.str() << std::setw(8) << (r.skewed ? "yes" : "no") + << std::right << std::fixed << std::setprecision(6) + << std::setw(14) << urs.min << std::setw(14) << urs.p99 + << std::setw(14) << urs.p90 << std::setw(14) << urs.p50 + << std::setw(14) << urs.max << std::setw(14) << urs.avg << std::setprecision(0) << std::setw(14) << r.throughput << std::setw(12) << r.avg_ns << std::setw(12) << r.p50_ns << std::setw(12) << r.p90_ns << std::setw(12) << r.p99_ns << std::setprecision(4) << std::setw(12) << r.final_util_stddev - << std::setprecision(2) << std::setw(9) - << (r.final_avg_util * 100.0) << "%" << std::setw(15) - << alloc_ratio << std::endl; + << std::setprecision(0) << std::setw(17) << r.evict_count + << std::setw(20) << alloc_ratio << std::endl; } static void printScaleOutHeader() { @@ -874,6 +1178,106 @@ static void runScaleOutMatrix() { } } +// DSA matrix runner +static void runDsaMatrix() { + std::vector skewed_options = {false, true}; + std::vector segment_counts = {1, 2, 4, 8, 16}; + std::vector replica_nums = {1, 2, 3}; + std::vector paired_modes = {false, true}; + std::vector strategies = { + AllocationStrategyType::RANDOM, + AllocationStrategyType::FREE_RATIO_FIRST, + }; + + size_t seg_cap_mb = static_cast(FLAGS_dsa_segment_capacity); + + const double total_mult = std::max(0.0, FLAGS_dsa_warmup_multiplier) + + std::max(0.0, FLAGS_dsa_capacity_multiplier); + std::cout + << "\n=== DSA Paired KV+Indexer Benchmark Matrix ===\n" + << "Workload: per-round burst of N kvcache (" << kDsaKvSize + << " B) allocations followed (paired mode only) by N indexer (" + << kDsaIndexerSize << " B) allocations; N uniform in [1, " + << kDsaMaxBatch << "].\n" + << "Eviction: on Allocate failure, drop " + << (FLAGS_dsa_evict_ratio * 100.0) + << "% of live objects at random; retry up to " << kDsaMaxRetries + << " times. Latency includes evict+retry time. Evictions(round) " + "counts how many such drop cycles fire during measurement, " + "NOT the number of objects evicted.\n" + << "UtilRatio_*: cluster utilization sampled at each eviction " + "trigger (peak fill the strategy reached before being forced " + "to evict), NOT a time-weighted average over all allocations.\n" + << "Phases: warmup writes " << FLAGS_dsa_warmup_multiplier + << "x cluster_capacity worth of allocations to reach steady state " + "(stats discarded), then measurement writes " + << FLAGS_dsa_capacity_multiplier + << "x cluster_capacity (stats collected). " + "Set --dsa_warmup_multiplier=0 to disable warmup.\n" + << "Config: dsa_segment_capacity=" << seg_cap_mb << " MB (" + << (seg_cap_mb / 1024.0) + << " GB), dsa_max_allocs=" << FLAGS_dsa_max_allocs << "\n" + << "Total per-case allocations auto-derived as " << total_mult + << "x * cluster_capacity / (avg_obj_size * replica), clamped to [" + << kDsaMinAllocs << ", " << FLAGS_dsa_max_allocs + << "]; warmup/measurement budgets split proportionally when clamp " + "activates. --num_allocations is ignored in DSA mode.\n" + << "Skewed setup: half nodes are (base + 50%) capacity, half are " + "(base - 50%)\n" + << std::endl; + + std::vector configs; + for (auto paired : paired_modes) { + for (auto skew : skewed_options) { + for (auto strategy : strategies) { + for (auto segs : segment_counts) { + for (auto rep : replica_nums) { + if (rep > segs) continue; + BenchConfig cfg; + cfg.num_segments = segs; + cfg.segment_capacity = seg_cap_mb * MiB; + cfg.alloc_size = kDsaKvSize; + cfg.replica_num = rep; + // num_allocations is auto-derived inside + // runDsaBenchmark from cluster capacity; the flag + // value is intentionally not propagated. + cfg.num_allocations = 0; + cfg.skewed = skew; + cfg.strategy_type = strategy; + cfg.strategy_name = strategyName(strategy); + cfg.workload_type = WorkloadType::DSA; + cfg.dsa_paired = paired; + configs.push_back(cfg); + } + } + } + } + } + + bool first = true; + bool prev_paired = false; + AllocationStrategyType prev_strategy = AllocationStrategyType::RANDOM; + + for (const auto& cfg : configs) { + if (first || cfg.dsa_paired != prev_paired) { + std::cout << "\n--- " + << (cfg.dsa_paired ? "Paired (KV + Indexer)" : "KV-only") + << " ---" << std::endl; + prev_paired = cfg.dsa_paired; + first = true; // force header re-print at start of new section + } + + if (first || cfg.strategy_type != prev_strategy) { + printDsaHeader(); + prev_strategy = cfg.strategy_type; + first = false; + } + + auto result = runDsaBenchmark(cfg); + printDsaResult(result); + } +} + int main(int argc, char* argv[]) { gflags::SetUsageMessage( "AllocationStrategy performance benchmark.\n" @@ -884,13 +1288,17 @@ int main(int argc, char* argv[]) { if (FLAGS_run_all) { runFillupBenchmarks(); runScaleOutMatrix(); + runDsaMatrix(); } else if (FLAGS_workload == "fillup") { runFillupBenchmarks(); } else if (FLAGS_workload == "scaleout") { runScaleOutMatrix(); + } else if (FLAGS_workload == "dsa") { + runDsaMatrix(); } else { std::cout << "Invalid workload type: " << FLAGS_workload - << ". Use --workload=fillup or --workload=scaleout." + << ". Use --workload=fillup, --workload=scaleout, or " + "--workload=dsa." << std::endl; } diff --git a/mooncake-store/benchmarks/allocator_bench.cpp b/mooncake-store/benchmarks/allocator_bench.cpp index 273c2a7a..8c4a8d90 100644 --- a/mooncake-store/benchmarks/allocator_bench.cpp +++ b/mooncake-store/benchmarks/allocator_bench.cpp @@ -20,7 +20,8 @@ class OffsetAllocatorBenchHelper { rd_(), gen_(rd_()) {} - void allocate(uint32_t size) { + void allocate(uint32_t size, double evict_ratio = 0.0, + std::vector* pre_evict_util_ratios = nullptr) { while (true) { auto handle = allocator_->allocate(size); if (handle.has_value()) { @@ -32,14 +33,26 @@ class OffsetAllocatorBenchHelper { if (allocated_.size() == 0) { break; } - std::uniform_int_distribution dist(0, - allocated_.size() - 1); - auto index = dist(gen_); - std::swap(allocated_[index], allocated_.back()); - std::swap(allocated_sizes_[index], allocated_sizes_.back()); - allocated_size_ -= allocated_sizes_.back(); - allocated_.pop_back(); - allocated_sizes_.pop_back(); + if (pre_evict_util_ratios != nullptr) { + pre_evict_util_ratios->push_back(get_allocated_ratio()); + } + size_t evict_count = 1; + if (evict_ratio > 0.0) { + evict_count = std::max( + 1, static_cast(allocated_.size() * evict_ratio)); + } + evict_count = std::min(evict_count, allocated_.size()); + + for (size_t i = 0; i < evict_count; ++i) { + std::uniform_int_distribution dist( + 0, allocated_.size() - 1); + auto index = dist(gen_); + std::swap(allocated_[index], allocated_.back()); + std::swap(allocated_sizes_[index], allocated_sizes_.back()); + allocated_size_ -= allocated_sizes_.back(); + allocated_.pop_back(); + allocated_sizes_.pop_back(); + } } } @@ -182,12 +195,12 @@ void paired_kv_indexer_allocation_benchmark() { << "=== Paired KV/Indexer Allocation Benchmark (DSA) ===" << std::endl; - const uint32_t kvcache_size = 3274752; // 3,274,752 B - const uint32_t indexer_size = 643u * 1024; // 643 KB - const size_t pool_size = 600ull * 1024 * 1024 * 1024; // 600 GB + const uint32_t kvcache_size = 3274752; // 3,274,752 B + const uint32_t indexer_size = 643u * 1024; // 643 KB + const size_t pool_size = 1024ull * 1024 * 1024 * 1024; // 1TB const int max_per_round = 128; const int warmup_rounds = 5000; - const int num_rounds = 500000; + const int num_rounds = 50000; size_t max_allocs = pool_size / indexer_size + 1024; BenchHelper bench_helper(0x1000, pool_size, max_allocs); @@ -200,26 +213,27 @@ void paired_kv_indexer_allocation_benchmark() { for (int round = 0; round < warmup_rounds; round++) { int per_round = per_round_dist(gen); for (int i = 0; i < per_round; i++) { - bench_helper.allocate(kvcache_size); + bench_helper.allocate(kvcache_size, 0.05); } for (int i = 0; i < per_round; i++) { - bench_helper.allocate(indexer_size); + bench_helper.allocate(indexer_size, 0.05); } } std::vector util_ratios; - util_ratios.reserve(static_cast(num_rounds) * 2 * max_per_round); + util_ratios.reserve(num_rounds); auto start_time = std::chrono::high_resolution_clock::now(); + size_t total_alloc_count = 0; for (int round = 0; round < num_rounds; round++) { int per_round = per_round_dist(gen); for (int i = 0; i < per_round; i++) { - bench_helper.allocate(kvcache_size); - util_ratios.push_back(bench_helper.get_allocated_ratio()); + bench_helper.allocate(kvcache_size, 0.05, &util_ratios); + ++total_alloc_count; } for (int i = 0; i < per_round; i++) { - bench_helper.allocate(indexer_size); - util_ratios.push_back(bench_helper.get_allocated_ratio()); + bench_helper.allocate(indexer_size, 0.05, &util_ratios); + ++total_alloc_count; } } auto end_time = std::chrono::high_resolution_clock::now(); @@ -228,17 +242,22 @@ void paired_kv_indexer_allocation_benchmark() { std::chrono::duration_cast(end_time - start_time) .count() / - static_cast(util_ratios.size()); + static_cast(total_alloc_count); std::sort(util_ratios.begin(), util_ratios.end()); - const double min_util = util_ratios.front(); - const double max_util = util_ratios.back(); - const double p50 = util_ratios[util_ratios.size() * 0.50]; - const double p90 = util_ratios[util_ratios.size() * 0.10]; - const double p99 = util_ratios[util_ratios.size() * 0.01]; + const double min_util = util_ratios.empty() ? 0.0 : util_ratios.front(); + const double max_util = util_ratios.empty() ? 0.0 : util_ratios.back(); + const double p50 = + util_ratios.empty() ? 0.0 : util_ratios[util_ratios.size() * 0.50]; + const double p90 = + util_ratios.empty() ? 0.0 : util_ratios[util_ratios.size() * 0.10]; + const double p99 = + util_ratios.empty() ? 0.0 : util_ratios[util_ratios.size() * 0.01]; const double mean_util = - std::accumulate(util_ratios.begin(), util_ratios.end(), 0.0) / - util_ratios.size(); + util_ratios.empty() + ? 0.0 + : std::accumulate(util_ratios.begin(), util_ratios.end(), 0.0) / + util_ratios.size(); std::cout << std::fixed << std::setprecision(6); std::cout << "kvcache size: " << kvcache_size @@ -249,6 +268,8 @@ void paired_kv_indexer_allocation_benchmark() { std::cout << "util ratio (min / p99 / p90 / p50 / max / avg): " << min_util << " / " << p99 << " / " << p90 << " / " << p50 << " / " << max_util << " / " << mean_util << std::endl; + std::cout << "eviction-trigger samples: " << util_ratios.size() + << std::endl; std::cout << "avg alloc time: " << avg_time_ns << " ns/op" << std::endl; } -- 2.34.1 From da9dfea38703c9380093e4b95cc1dc3670848a51 Mon Sep 17 00:00:00 2001 From: Zhewen Li Date: Tue, 12 May 2026 10:13:07 -0700 Subject: [PATCH 069/382] [Store] Expose is_local_disk_replica() to Python + enable offload RPC in standalone mooncake_client (#2083) * [Store] Expose is_local_disk_replica() to Python in ReplicaDescriptor Replica::Descriptor is a 3-way std::variant {MemoryDescriptor, DiskDescriptor, LocalDiskDescriptor} with a corresponding C++ predicate per type. The Python wrapper was missing is_local_disk_replica. Mooncake's offload pipeline (NotifyOffloadSuccess) constructs LocalDiskDescriptor exclusively. With only is_memory_replica / is_disk_replica exposed to Python, every LOCAL_DISK descriptor returned by the master to a Python caller would test False on both predicates and be misclassified (e.g. as "unknown" in tier diagnostics) regardless of whether the actual load succeeded. --------- Co-authored-by: Zhewen Li Co-authored-by: Claude Opus 4.7 (1M context) --- mooncake-integration/store/store_py.cpp | 3 +++ mooncake-store/src/real_client_main.cpp | 7 ++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index fb13243b..4aa8b829 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -1616,6 +1616,9 @@ PYBIND11_MODULE(store, m) { .def("is_disk_replica", static_cast( &Replica::Descriptor::is_disk_replica)) + .def("is_local_disk_replica", + static_cast( + &Replica::Descriptor::is_local_disk_replica)) .def( "get_memory_descriptor", static_cast Date: Wed, 13 May 2026 14:10:21 +0800 Subject: [PATCH 070/382] [Store] add SSD-offload support for ascend platform (#2005) Co-authored-by: youxiao --- .../include/aligned_client_buffer.hpp | 1 + mooncake-store/include/client_service.h | 2 + mooncake-store/src/aligned_client_buffer.cpp | 115 +++++++++++++----- mooncake-store/src/file_storage.cpp | 6 +- 4 files changed, 89 insertions(+), 35 deletions(-) diff --git a/mooncake-store/include/aligned_client_buffer.hpp b/mooncake-store/include/aligned_client_buffer.hpp index 57f81e92..436fc9bb 100644 --- a/mooncake-store/include/aligned_client_buffer.hpp +++ b/mooncake-store/include/aligned_client_buffer.hpp @@ -56,6 +56,7 @@ class AlignedClientBufferAllocator : public ClientBufferAllocator { // Store whether we own the memory (for cleanup) bool owns_memory_; size_t allocated_size_; // Store the actual allocated size for cleanup + std::string protocol_; }; } // namespace mooncake diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index a2e71eb5..8aa0f1a4 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -492,6 +492,8 @@ class Client { return transfer_engine_->getLocalIpAndPort(); } + [[nodiscard]] const std::string& GetProtocol() const { return protocol_; } + /** * @brief Get the endpoint address for segment operations. * @return For P2PHANDSHAKE mode, returns the actual RPC endpoint (IP:Port). diff --git a/mooncake-store/src/aligned_client_buffer.cpp b/mooncake-store/src/aligned_client_buffer.cpp index 6ac7ea90..232539d8 100644 --- a/mooncake-store/src/aligned_client_buffer.cpp +++ b/mooncake-store/src/aligned_client_buffer.cpp @@ -4,10 +4,78 @@ #include #include #include +#include #include "utils.h" namespace mooncake { +namespace { +constexpr std::string_view kAscendProtocol = "ascend"; +constexpr std::string_view kUbshmemProtocol = "ubshmem"; + +bool UseProtocolAllocator(const std::string& protocol) { + return protocol == kAscendProtocol || protocol == kUbshmemProtocol; +} + +void FreeAlignedBuffer(void* buffer, size_t size, const std::string& protocol, + bool use_hugepage) { + if (use_hugepage) { + free_buffer_mmap_memory(buffer, size); + } else if (UseProtocolAllocator(protocol)) { + free_memory(protocol, buffer); + } else { + free(buffer); + } +} + +void* AllocateProtocolAlignedBuffer(size_t aligned_size, + const std::string& protocol) { + void* aligned_buffer = allocate_buffer_allocator_memory( + aligned_size, protocol, + AlignedClientBufferAllocator::kDirectIOAlignment); + if (!aligned_buffer) { + LOG(ERROR) << "AlignedClientBufferAllocator: failed to allocate " + << "protocol-aware memory of size " << aligned_size + << " for protocol " << protocol; + } + return aligned_buffer; +} + +void* AllocateHugepageAlignedBuffer(size_t aligned_size) { + void* aligned_buffer = allocate_buffer_mmap_memory( + aligned_size, AlignedClientBufferAllocator::kDirectIOAlignment); + if (!aligned_buffer) { + LOG(ERROR) << "AlignedClientBufferAllocator: failed to allocate " + << "hugepage memory of size " << aligned_size; + } + return aligned_buffer; +} + +void* AllocatePosixAlignedBuffer(size_t aligned_size) { + void* aligned_buffer = nullptr; + int ret = posix_memalign(&aligned_buffer, + AlignedClientBufferAllocator::kDirectIOAlignment, + aligned_size); + if (ret != 0) { + LOG(ERROR) << "AlignedClientBufferAllocator: posix_memalign failed " + << "with error " << ret << " (" << strerror(ret) << ")"; + return nullptr; + } + memset(aligned_buffer, 0, aligned_size); + return aligned_buffer; +} + +void* AllocateAlignedBuffer(size_t aligned_size, const std::string& protocol, + bool use_mmap_hugepage) { + if (UseProtocolAllocator(protocol)) { + return AllocateProtocolAlignedBuffer(aligned_size, protocol); + } + if (use_mmap_hugepage) { + return AllocateHugepageAlignedBuffer(aligned_size); + } + return AllocatePosixAlignedBuffer(aligned_size); +} +} // namespace std::shared_ptr AlignedClientBufferAllocator::create(size_t size, const std::string& protocol, @@ -21,40 +89,20 @@ AlignedClientBufferAllocator::create(size_t size, const std::string& protocol, // Align size up to kDirectIOAlignment size_t aligned_size = align_up(size, kDirectIOAlignment); - void* aligned_buffer = nullptr; - - if (use_hugepage) { - // Use hugepage allocation (already aligned) - aligned_buffer = - allocate_buffer_mmap_memory(aligned_size, kDirectIOAlignment); - if (!aligned_buffer) { - LOG(ERROR) << "AlignedClientBufferAllocator: failed to allocate " - << "hugepage memory of size " << aligned_size; - return nullptr; - } - } else { - // Use posix_memalign for 4096-byte alignment - int ret = - posix_memalign(&aligned_buffer, kDirectIOAlignment, aligned_size); - if (ret != 0) { - LOG(ERROR) << "AlignedClientBufferAllocator: posix_memalign failed " - << "with error " << ret << " (" << strerror(ret) << ")"; - return nullptr; - } - - // Zero-initialize the allocated memory - memset(aligned_buffer, 0, aligned_size); + const bool use_protocol_allocator = UseProtocolAllocator(protocol); + const bool use_mmap_hugepage = use_hugepage && !use_protocol_allocator; + void* aligned_buffer = + AllocateAlignedBuffer(aligned_size, protocol, use_mmap_hugepage); + if (!aligned_buffer) { + return nullptr; } // Verify alignment if (reinterpret_cast(aligned_buffer) % kDirectIOAlignment != 0) { LOG(ERROR) << "AlignedClientBufferAllocator: allocated buffer is not " << "aligned to " << kDirectIOAlignment << " bytes"; - if (use_hugepage) { - free_buffer_mmap_memory(aligned_buffer, aligned_size); - } else { - free(aligned_buffer); - } + FreeAlignedBuffer(aligned_buffer, aligned_size, protocol, + use_mmap_hugepage); return nullptr; } @@ -65,7 +113,7 @@ AlignedClientBufferAllocator::create(size_t size, const std::string& protocol, // Use custom deleter to properly free the aligned memory return std::shared_ptr( new AlignedClientBufferAllocator(aligned_buffer, aligned_size, protocol, - use_hugepage)); + use_mmap_hugepage)); } AlignedClientBufferAllocator::AlignedClientBufferAllocator( @@ -73,7 +121,8 @@ AlignedClientBufferAllocator::AlignedClientBufferAllocator( bool use_hugepage) : ClientBufferAllocator(aligned_buffer, size, protocol), owns_memory_(true), - allocated_size_(size) { + allocated_size_(size), + protocol_(protocol) { // Store hugepage flag in parent class's use_hugepage_ member // We need this for proper cleanup use_hugepage_ = use_hugepage; @@ -88,12 +137,14 @@ AlignedClientBufferAllocator::~AlignedClientBufferAllocator() { LOG(INFO) << "AlignedClientBufferAllocator: freeing hugepage memory " << "at " << buffer_ << " (" << allocated_size_ << " bytes)"; - free_buffer_mmap_memory(buffer_, allocated_size_); + FreeAlignedBuffer(buffer_, allocated_size_, protocol_, + use_hugepage_); } else { LOG(INFO) << "AlignedClientBufferAllocator: freeing aligned memory " << "at " << buffer_ << " (" << allocated_size_ << " bytes)"; - free(buffer_); + FreeAlignedBuffer(buffer_, allocated_size_, protocol_, + use_hugepage_); } buffer_ = nullptr; } diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index 1dff43d3..63e34492 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -161,8 +161,8 @@ FileStorage::FileStorage(const FileStorageConfig& config, ssd_metric_(ssd_metric), local_rpc_addr_(local_rpc_addr), pinned_buffer_pool_(std::make_unique()), - client_buffer_allocator_( - AlignedClientBufferAllocator::create(config.local_buffer_size, "")) { + client_buffer_allocator_(AlignedClientBufferAllocator::create( + config.local_buffer_size, client ? client->GetProtocol() : "")) { if (!config.Validate()) { throw std::invalid_argument("Invalid FileStorage configuration"); } @@ -707,4 +707,4 @@ bool FileStorage::ReleaseBuffer(uint64_t batch_id) { return false; } -} // namespace mooncake \ No newline at end of file +} // namespace mooncake -- 2.34.1 From b0ae4a727fece46a6bbb80b03ef4478828214071 Mon Sep 17 00:00:00 2001 From: Yufeng He <40085740+he-yufeng@users.noreply.github.com> Date: Wed, 13 May 2026 16:07:13 +0800 Subject: [PATCH 071/382] [Store] Add ObjectDataType enum for type-aware metadata (#1719) * feat: add ObjectDataType enum and metadata propagation (Phase 1 of #1673) Introduce a data type classification system for objects stored in Mooncake Store, as agreed in the RFC discussion on issue #1673. Changes: - Add ObjectDataType enum (UNKNOWN, KVCACHE, TENSOR, WEIGHT, etc.) in mooncake-store/include/types.h - Extend ReplicateConfig with a data_type field (default UNKNOWN) - Propagate data_type through PutStart into ObjectMetadata - Serialize/deserialize data_type in snapshot metadata, with backward compatibility for old snapshots (gracefully handles missing field) - Expose ObjectDataType enum and data_type field in Python bindings - Add unit tests for enum values, defaults, and PutStart propagation Existing clients that don't set data_type will continue to work unchanged (defaults to UNKNOWN everywhere). --------- Co-authored-by: Yufeng He <40085740+universeplayer@users.noreply.github.com> --- mooncake-integration/store/store_py.cpp | 15 ++ mooncake-store/include/master_service.h | 10 +- mooncake-store/include/replica.h | 4 +- mooncake-store/include/types.h | 41 +++++ mooncake-store/src/master_service.cpp | 47 ++++-- mooncake-store/tests/CMakeLists.txt | 1 + .../tests/object_data_type_test.cpp | 157 ++++++++++++++++++ 7 files changed, 259 insertions(+), 16 deletions(-) create mode 100644 mooncake-store/tests/object_data_type_test.cpp diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index 4aa8b829..ba28c5ac 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -1574,6 +1574,20 @@ class MooncakeHostMemAllocatorPyWrapper { }; PYBIND11_MODULE(store, m) { + // Object data type classification + py::enum_(m, "ObjectDataType") + .value("UNKNOWN", ObjectDataType::UNKNOWN) + .value("KVCACHE", ObjectDataType::KVCACHE) + .value("TENSOR", ObjectDataType::TENSOR) + .value("WEIGHT", ObjectDataType::WEIGHT) + .value("SAMPLE", ObjectDataType::SAMPLE) + .value("ACTIVATION", ObjectDataType::ACTIVATION) + .value("GRADIENT", ObjectDataType::GRADIENT) + .value("OPTIMIZER_STATE", ObjectDataType::OPTIMIZER_STATE) + .value("METADATA", ObjectDataType::METADATA) + .value("GENERAL", ObjectDataType::GENERAL) + .export_values(); + // Define the ReplicateConfig class py::class_(m, "ReplicateConfig") .def(py::init<>()) @@ -1585,6 +1599,7 @@ PYBIND11_MODULE(store, m) { .def_readwrite("preferred_segment", &ReplicateConfig::preferred_segment) .def_readwrite("prefer_alloc_in_same_node", &ReplicateConfig::prefer_alloc_in_same_node) + .def_readwrite("data_type", &ReplicateConfig::data_type) .def("__str__", [](const ReplicateConfig &config) { std::ostringstream oss; oss << config; diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index c1cddd27..54f3f73a 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -598,10 +598,12 @@ class MasterService { const UUID& client_id_, const std::chrono::system_clock::time_point put_start_time_, size_t value_length, std::vector&& reps, - bool enable_soft_pin, bool enable_hard_pin = false) + bool enable_soft_pin, bool enable_hard_pin = false, + ObjectDataType data_type_ = ObjectDataType::UNKNOWN) : client_id(client_id_), put_start_time(put_start_time_), size(value_length), + data_type(data_type_), lease_timeout(), soft_pin_timeout(std::nullopt), hard_pinned(enable_hard_pin), @@ -624,6 +626,7 @@ class MasterService { // Updated by UpsertStart (Case B) to reset the discard timeout. std::chrono::system_clock::time_point put_start_time; const size_t size; + const ObjectDataType data_type{ObjectDataType::UNKNOWN}; mutable SpinLock lock; // Default constructor, creates a time_point representing @@ -1040,7 +1043,8 @@ class MasterService { void Create(const UUID& client_id, uint64_t total_length, std::vector replicas, bool enable_soft_pin, - bool enable_hard_pin = false) { + bool enable_hard_pin = false, + ObjectDataType data_type = ObjectDataType::UNKNOWN) { if (Exists()) { throw std::logic_error("Already exists"); } @@ -1049,7 +1053,7 @@ class MasterService { std::piecewise_construct, std::forward_as_tuple(key_), std::forward_as_tuple(client_id, now, total_length, std::move(replicas), enable_soft_pin, - enable_hard_pin)); + enable_hard_pin, data_type)); it_ = result.first; } diff --git a/mooncake-store/include/replica.h b/mooncake-store/include/replica.h index e7434c23..29b868ea 100644 --- a/mooncake-store/include/replica.h +++ b/mooncake-store/include/replica.h @@ -93,6 +93,7 @@ struct ReplicateConfig { std::string preferred_segment{}; // Deprecated: Single preferred segment // for backward compatibility bool prefer_alloc_in_same_node{false}; + ObjectDataType data_type{ObjectDataType::UNKNOWN}; friend std::ostream& operator<<(std::ostream& os, const ReplicateConfig& config) noexcept { @@ -110,7 +111,8 @@ struct ReplicateConfig { << config.preferred_segment; } os << ", prefer_alloc_in_same_node: " - << config.prefer_alloc_in_same_node << " }"; + << config.prefer_alloc_in_same_node + << ", data_type: " << config.data_type << " }"; return os; } }; diff --git a/mooncake-store/include/types.h b/mooncake-store/include/types.h index 68dfb05c..e0e4fab1 100644 --- a/mooncake-store/include/types.h +++ b/mooncake-store/include/types.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -123,6 +124,46 @@ static constexpr uint64_t DEFAULT_PROCESSING_TASK_TIMEOUT_SEC = 300; // 0 to be no timeout static constexpr uint32_t DEFAULT_MAX_RETRY_ATTEMPTS = 10; +/** + * @brief Data type classification for objects stored in Mooncake Store. + * + * This allows the store to track what kind of data each object holds, + * enabling future type-aware policies (eviction priority, replication + * strategies, etc.). Defaults to UNKNOWN for backward compatibility. + */ +enum class ObjectDataType : uint8_t { + UNKNOWN = 0, + KVCACHE = 1, + TENSOR = 2, + WEIGHT = 3, + SAMPLE = 4, + ACTIVATION = 5, + GRADIENT = 6, + OPTIMIZER_STATE = 7, + METADATA = 8, + GENERAL = 9, + // 10-255 reserved for future types +}; + +inline std::ostream& operator<<(std::ostream& os, + const ObjectDataType& type) noexcept { + static const std::unordered_map + type_strings{{ObjectDataType::UNKNOWN, "UNKNOWN"}, + {ObjectDataType::KVCACHE, "KVCACHE"}, + {ObjectDataType::TENSOR, "TENSOR"}, + {ObjectDataType::WEIGHT, "WEIGHT"}, + {ObjectDataType::SAMPLE, "SAMPLE"}, + {ObjectDataType::ACTIVATION, "ACTIVATION"}, + {ObjectDataType::GRADIENT, "GRADIENT"}, + {ObjectDataType::OPTIMIZER_STATE, "OPTIMIZER_STATE"}, + {ObjectDataType::METADATA, "METADATA"}, + {ObjectDataType::GENERAL, "GENERAL"}}; + + auto it = type_strings.find(type); + os << (it != type_strings.end() ? it->second : "UNKNOWN"); + return os; +} + // Forward declarations class BufferAllocatorBase; class CachelibBufferAllocator; diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 5fbcb0d8..dbe51659 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -845,7 +845,8 @@ auto MasterService::AllocateAndInsertMetadata( shard->metadata.emplace( std::piecewise_construct, std::forward_as_tuple(key), std::forward_as_tuple(client_id, now, value_length, std::move(replicas), - config.with_soft_pin, config.with_hard_pin)); + config.with_soft_pin, config.with_hard_pin, + config.data_type)); shard->processing_keys.insert(key); return replica_list; @@ -4322,7 +4323,7 @@ MasterService::MetadataSerializer::DeserializeShard(const msgpack::object& obj, metadata_ptr->client_id, metadata_ptr->put_start_time, metadata_ptr->size, metadata_ptr->PopReplicas(), metadata_ptr->soft_pin_timeout.has_value(), - metadata_ptr->IsHardPinned())); + metadata_ptr->IsHardPinned(), metadata_ptr->data_type)); it->second.lease_timeout = metadata_ptr->lease_timeout; it->second.soft_pin_timeout = metadata_ptr->soft_pin_timeout; @@ -4337,12 +4338,12 @@ MasterService::MetadataSerializer::SerializeMetadata( MsgpackPacker& packer) const { // Pack ObjectMetadata using array structure for efficiency // Format: [client_id, put_start_time, size, lease_timeout, - // has_soft_pin_timeout, soft_pin_timeout, replicas_count, replicas..., - // hard_pinned] + // has_soft_pin_timeout, soft_pin_timeout, replicas_count, data_type, + // replicas..., hard_pinned] - size_t array_size = 8; // client_id, put_start_time, size, lease_timeout, + size_t array_size = 9; // client_id, put_start_time, size, lease_timeout, // has_soft_pin_timeout, soft_pin_timeout, - // replicas_count + hard_pinned + // replicas_count, data_type, hard_pinned array_size += metadata.CountReplicas(); // One element per replica packer.pack_array(array_size); @@ -4382,6 +4383,9 @@ MasterService::MetadataSerializer::SerializeMetadata( // Serialize replicas count packer.pack(static_cast(metadata.CountReplicas())); + // Serialize data_type + packer.pack(static_cast(metadata.data_type)); + // Serialize replicas for (const auto& replica : metadata.GetAllReplicas()) { auto result = Serializer::serialize( @@ -4408,7 +4412,6 @@ MasterService::MetadataSerializer::DeserializeMetadata( // Need at least 7 elements: client_id, put_start_time, size, lease_timeout, // has_soft_pin_timeout, soft_pin_timeout, replicas_count - // (8th element = hard_pinned is optional for backward compat) if (obj.via.array.size < 7) { return tl::unexpected(SerializationError( ErrorCode::DESERIALIZE_FAIL, @@ -4441,15 +4444,35 @@ MasterService::MetadataSerializer::DeserializeMetadata( // Deserialize replicas count uint32_t replicas_count = array[index++].as(); - // Array size: 7 + replicas_count (old format) or 8 + replicas_count (new - // format with hard_pinned) - if (obj.via.array.size != 7 + replicas_count && - obj.via.array.size != 8 + replicas_count) { + // Format detection: + // v1: 7 + replicas_count, no data_type or hard_pinned + // v2: 8 + replicas_count, either data_type or trailing hard_pinned + // v3: 9 + replicas_count, data_type plus trailing hard_pinned + constexpr uint32_t kOldFieldCount = 7; + constexpr uint32_t kOneExtraFieldCount = 8; + constexpr uint32_t kCurrentFieldCount = 9; + const uint32_t total_elements = obj.via.array.size; + const bool is_old_format = + (total_elements == kOldFieldCount + replicas_count); + const bool is_one_extra_format = + (total_elements == kOneExtraFieldCount + replicas_count); + const bool is_current_format = + (total_elements == kCurrentFieldCount + replicas_count); + + if (!is_current_format && !is_one_extra_format && !is_old_format) { return tl::unexpected(SerializationError( ErrorCode::DESERIALIZE_FAIL, "deserialize ObjectMetadata array size mismatch")); } + ObjectDataType data_type = ObjectDataType::UNKNOWN; + if (is_current_format) { + data_type = static_cast(array[index++].as()); + } else if (is_one_extra_format && + array[index].type == msgpack::type::POSITIVE_INTEGER) { + data_type = static_cast(array[index++].as()); + } + // Deserialize replicas std::vector replicas; replicas.reserve(replicas_count); @@ -4475,7 +4498,7 @@ MasterService::MetadataSerializer::DeserializeMetadata( client_id, std::chrono::system_clock::time_point( std::chrono::milliseconds(put_start_time_timestamp)), - size, std::move(replicas), enable_soft_pin, is_hard_pinned); + size, std::move(replicas), enable_soft_pin, is_hard_pinned, data_type); metadata->lease_timeout = std::chrono::system_clock::time_point( std::chrono::milliseconds(lease_timestamp)); diff --git a/mooncake-store/tests/CMakeLists.txt b/mooncake-store/tests/CMakeLists.txt index aff9e924..252b99e8 100644 --- a/mooncake-store/tests/CMakeLists.txt +++ b/mooncake-store/tests/CMakeLists.txt @@ -85,6 +85,7 @@ add_store_test(dummy_client_get_buffer_test dummy_client_get_buffer_test.cpp) add_store_test(health_check_test health_check_test.cpp) add_store_test(mmap_arena_test mmap_arena_test.cpp) add_store_test(mmap_arena_fallback_test mmap_arena_fallback_test.cpp) +add_store_test(object_data_type_test object_data_type_test.cpp) add_subdirectory(e2e) add_executable(high_availability_test ha/leadership/high_availability_test.cpp) diff --git a/mooncake-store/tests/object_data_type_test.cpp b/mooncake-store/tests/object_data_type_test.cpp new file mode 100644 index 00000000..fd0de441 --- /dev/null +++ b/mooncake-store/tests/object_data_type_test.cpp @@ -0,0 +1,157 @@ +#include "types.h" +#include "replica.h" +#include "master_service.h" + +#include +#include + +#include +#include + +namespace mooncake::test { + +class ObjectDataTypeTest : public ::testing::Test { + protected: + void SetUp() override { + google::InitGoogleLogging("ObjectDataTypeTest"); + FLAGS_logtostderr = true; + } + + void TearDown() override { google::ShutdownGoogleLogging(); } + + static constexpr size_t kDefaultSegmentBase = 0x300000000; + static constexpr size_t kDefaultSegmentSize = 1024 * 1024 * 16; + + Segment MakeSegment(std::string name = "test_segment", + size_t base = kDefaultSegmentBase, + size_t size = kDefaultSegmentSize) const { + Segment segment; + segment.id = generate_uuid(); + segment.name = std::move(name); + segment.base = base; + segment.size = size; + segment.te_endpoint = segment.name; + return segment; + } +}; + +// Verify enum values match the RFC spec +TEST_F(ObjectDataTypeTest, EnumValues) { + EXPECT_EQ(static_cast(ObjectDataType::UNKNOWN), 0); + EXPECT_EQ(static_cast(ObjectDataType::KVCACHE), 1); + EXPECT_EQ(static_cast(ObjectDataType::TENSOR), 2); + EXPECT_EQ(static_cast(ObjectDataType::WEIGHT), 3); + EXPECT_EQ(static_cast(ObjectDataType::SAMPLE), 4); + EXPECT_EQ(static_cast(ObjectDataType::ACTIVATION), 5); + EXPECT_EQ(static_cast(ObjectDataType::GRADIENT), 6); + EXPECT_EQ(static_cast(ObjectDataType::OPTIMIZER_STATE), 7); + EXPECT_EQ(static_cast(ObjectDataType::METADATA), 8); + EXPECT_EQ(static_cast(ObjectDataType::GENERAL), 9); +} + +// Verify stream operator produces readable output +TEST_F(ObjectDataTypeTest, StreamOperator) { + std::ostringstream oss; + oss << ObjectDataType::KVCACHE; + EXPECT_EQ(oss.str(), "KVCACHE"); + + oss.str(""); + oss << ObjectDataType::UNKNOWN; + EXPECT_EQ(oss.str(), "UNKNOWN"); + + oss.str(""); + oss << ObjectDataType::OPTIMIZER_STATE; + EXPECT_EQ(oss.str(), "OPTIMIZER_STATE"); + + oss.str(""); + oss << ObjectDataType::GENERAL; + EXPECT_EQ(oss.str(), "GENERAL"); + + // Out-of-range value should print "UNKNOWN" + oss.str(""); + oss << static_cast(200); + EXPECT_EQ(oss.str(), "UNKNOWN"); +} + +// ReplicateConfig defaults to UNKNOWN +TEST_F(ObjectDataTypeTest, ReplicateConfigDefaultDataType) { + ReplicateConfig config; + EXPECT_EQ(config.data_type, ObjectDataType::UNKNOWN); +} + +// ReplicateConfig can be set to other types +TEST_F(ObjectDataTypeTest, ReplicateConfigSetDataType) { + ReplicateConfig config; + config.data_type = ObjectDataType::WEIGHT; + EXPECT_EQ(config.data_type, ObjectDataType::WEIGHT); +} + +// ReplicateConfig stream output includes data_type +TEST_F(ObjectDataTypeTest, ReplicateConfigStreamIncludesDataType) { + ReplicateConfig config; + config.data_type = ObjectDataType::TENSOR; + std::ostringstream oss; + oss << config; + EXPECT_NE(oss.str().find("data_type: TENSOR"), std::string::npos); +} + +// PutStart with data_type propagates to ObjectMetadata +TEST_F(ObjectDataTypeTest, PutStartWithDataType) { + std::unique_ptr service(new MasterService()); + Segment segment = MakeSegment(); + UUID client_id = generate_uuid(); + auto mount_result = service->MountSegment(segment, client_id); + ASSERT_TRUE(mount_result.has_value()); + + UUID put_client = generate_uuid(); + + // Put with WEIGHT type + ReplicateConfig config; + config.replica_num = 1; + config.data_type = ObjectDataType::WEIGHT; + + auto result = service->PutStart(put_client, "key_weight", 1024, config); + ASSERT_TRUE(result.has_value()); + EXPECT_FALSE(result.value().empty()); + + auto end_result = + service->PutEnd(put_client, "key_weight", ReplicaType::MEMORY); + EXPECT_TRUE(end_result.has_value()); +} + +// PutStart with default UNKNOWN data_type still works (backward compat) +TEST_F(ObjectDataTypeTest, PutStartDefaultDataType) { + std::unique_ptr service(new MasterService()); + Segment segment = MakeSegment(); + UUID client_id = generate_uuid(); + auto mount_result = service->MountSegment(segment, client_id); + ASSERT_TRUE(mount_result.has_value()); + + UUID put_client = generate_uuid(); + ReplicateConfig config; + config.replica_num = 1; + // data_type left as default (UNKNOWN) + + auto result = service->PutStart(put_client, "key_default", 1024, config); + ASSERT_TRUE(result.has_value()); + EXPECT_FALSE(result.value().empty()); +} + +// Verify all enum values can roundtrip through uint8_t cast +TEST_F(ObjectDataTypeTest, EnumRoundtrip) { + std::vector all_types = { + ObjectDataType::UNKNOWN, ObjectDataType::KVCACHE, + ObjectDataType::TENSOR, ObjectDataType::WEIGHT, + ObjectDataType::SAMPLE, ObjectDataType::ACTIVATION, + ObjectDataType::GRADIENT, ObjectDataType::OPTIMIZER_STATE, + ObjectDataType::METADATA, ObjectDataType::GENERAL, + }; + + for (auto type : all_types) { + uint8_t raw = static_cast(type); + auto recovered = static_cast(raw); + EXPECT_EQ(type, recovered); + } +} + +} // namespace mooncake::test -- 2.34.1 From 368b41f168b35d7e1c3989af026b00bd3f89e9f5 Mon Sep 17 00:00:00 2001 From: ykwd Date: Thu, 14 May 2026 20:26:41 +0800 Subject: [PATCH 072/382] [Docs] Tag 3fs Feature as Experimental (#2062) * Update 3fs docs * Update news --------- Co-authored-by: Ke Yang --- README.md | 2 +- docs/source/design/mooncake-store.md | 7 ++++++- .../plugin-usage/3FS-USRBIO-Plugin.md | 16 ++++++++++------ docs/source/index.md | 2 ++ docs/source/zh_archive/mooncake-store.md | 5 ++++- 5 files changed, 23 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index ffe3f186..6295c8e1 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ This repository also hosts its technical report and the open-sourced traces.

🔄 Updates

-- **May 7, 2026**: 🚀 vLLM officially features [Mooncake Store](https://vllm.ai/blog/mooncake-store) — a deep dive into how Mooncake's distributed KVCache engine supercharges vLLM inference with high-throughput, memory-efficient, cross-instance KV cache sharing! +- **May 7, 2026**: 🚀 [vLLM officially features Mooncake Store](https://vllm.ai/blog/mooncake-store) — a deep dive into how Mooncake's distributed KVCache engine supercharges vLLM inference with high-throughput, memory-efficient, cross-instance KV cache sharing! - **Apr 29, 2026**: SGLang introduces [RDMA-based P2P weight transfer for large-scale distributed RL](https://lmsys.org/blog/2026-04-29-p2p-update/) using Mooncake TransferEngine, achieving 7x faster weight updates for the 1T-parameter Kimi-K2 model (53s → 7.2s) with zero-copy RDMA transfer across thousands of GPUs. - **Mar 19, 2026**: [TorchSpec: Speculative Decoding Training at Scale](https://pytorch.org/blog/torchspec-speculative-decoding-training-at-scale) is [open sourced](https://github.com/torchspec-project/TorchSpec), using Mooncake to decouple inference and training via efficient hidden states management. - **Mar 5, 2026**: [LightX2V](https://github.com/ModelTC/LightX2V/pull/893) now supports disaggregated deployment based on Mooncake, enabling encoder/transformer service decoupling with Mooncake Transfer Engine for high-performance cross-device and cross-machine data transfer. diff --git a/docs/source/design/mooncake-store.md b/docs/source/design/mooncake-store.md index f920303f..ac1afdd0 100644 --- a/docs/source/design/mooncake-store.md +++ b/docs/source/design/mooncake-store.md @@ -820,7 +820,12 @@ After enabling the persistence feature: - For each `Put` or `BatchPut` operation, both a synchronous memory pool write operation and an asynchronous DFS persistence operation will be initiated. - For each `Get` or `BatchGet` operation, if the corresponding kvcache is not found in the memory pool, the system will attempt to read the file data from DFS and return it to the user. -#### 3FS USRBIO Plugin +#### 3FS USRBIO Plugin (Experimental) + +```{note} +This integration is **experimental** and incomplete; see the plugin page for details before relying on it. +``` + If you need to use 3FS's native API (USRBIO) to achieve high-performance persistent file reads and writes, you can refer to the configuration instructions in this document [3FS USRBIO Plugin](../getting_started/plugin-usage/3FS-USRBIO-Plugin.md). ### Builtin Metadata Server diff --git a/docs/source/getting_started/plugin-usage/3FS-USRBIO-Plugin.md b/docs/source/getting_started/plugin-usage/3FS-USRBIO-Plugin.md index a542b05a..5566bc27 100644 --- a/docs/source/getting_started/plugin-usage/3FS-USRBIO-Plugin.md +++ b/docs/source/getting_started/plugin-usage/3FS-USRBIO-Plugin.md @@ -1,4 +1,8 @@ -# Mooncake HF3FS Plugin +# Mooncake HF3FS Plugin (Experimental) + +```{warning} +**Experimental / incomplete.** The HF3FS (3FS USRBIO) integration is under development and is not yet considered production-ready. Behavior, build flags, and configuration may change without notice. Use only for evaluation and testing. +``` This plugin implements 3FS native API (USRBIO) as a high-performance storage backend for Mooncake. @@ -6,9 +10,9 @@ This plugin implements 3FS native API (USRBIO) as a high-performance storage bac ### 1. 3FS Installation - Build and install [3FS](https://github.com/deepseek-ai/3FS/) -- Required library: `libhf3fs_api_shared.so` (Default location: `3FS_PATH/build/src/lib/api`) +- Required library: `libhf3fs_api_shared.so` (Default location: `3FS_PATH/build/src/lib/api`) → Install to: `/usr/lib/` -- Required header: `hf3fs_usrbio.h` (Default location: `3FS_PATH/src/lib/api`) +- Required header: `hf3fs_usrbio.h` (Default location: `3FS_PATH/src/lib/api`) → Install to: `/usr/include/` ### 2. Mooncake Configuration @@ -27,10 +31,10 @@ Start master server and specify the 3FS mount point: ```bash ./build/mooncake-store/src/mooncake_master \ - --root_fs_dir=/path/to/3fs_mount_point + --root_fs_dir=/path/to/3fs_mount_point ``` ### Important Notes -1. The specified directory **must** be a 3FS mount point +1. The specified directory **must** be a 3FS mount point - If not, the system will automatically fall back to POSIX API 2. For optimal performance: - Ensure proper permissions on the 3FS mount point @@ -40,4 +44,4 @@ Start master server and specify the 3FS mount point: ```bash ROLE=prefill MOONCAKE_STORAGE_ROOT_DIR=/mnt/3fs python3 ./stress_cluster_benchmark.py -``` \ No newline at end of file +``` diff --git a/docs/source/index.md b/docs/source/index.md index 4684256b..9298183f 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -27,6 +27,8 @@ This repository also hosts its technical report and the open-sourced traces.

🔄 Updates

+- **May 7, 2026**: 🚀 [vLLM officially features Mooncake Store](https://vllm.ai/blog/mooncake-store) — a deep dive into how Mooncake's distributed KVCache engine supercharges vLLM inference with high-throughput, memory-efficient, cross-instance KV cache sharing! +- **Apr 29, 2026**: SGLang introduces [RDMA-based P2P weight transfer for large-scale distributed RL](https://lmsys.org/blog/2026-04-29-p2p-update/) using Mooncake TransferEngine, achieving 7x faster weight updates for the 1T-parameter Kimi-K2 model (53s → 7.2s) with zero-copy RDMA transfer across thousands of GPUs. - **Mar 19, 2026**: [TorchSpec: Speculative Decoding Training at Scale](https://pytorch.org/blog/torchspec-speculative-decoding-training-at-scale) is [open sourced](https://github.com/torchspec-project/TorchSpec), using Mooncake to decouple inference and training via efficient hidden states management. - **Feb 12, 2026**: [Mooncake Joins PyTorch Ecosystem](https://pytorch.org/blog/mooncake-joins-pytorch-ecosystem/) We are thrilled to announce that Mooncake has officially joined the PyTorch Ecosystem! - **Jan 28, 2026**: [FlexKV](https://github.com/taco-project/FlexKV), a distributed KV store and cache system from Tencent and NVIDIA in collaboration with the community, now supports [distributed KVCache reuse](https://github.com/taco-project/FlexKV/blob/main/docs/dist_reuse/README_en.md) with the Mooncake Transfer Engine. diff --git a/docs/source/zh_archive/mooncake-store.md b/docs/source/zh_archive/mooncake-store.md index eac15bfd..841c9301 100644 --- a/docs/source/zh_archive/mooncake-store.md +++ b/docs/source/zh_archive/mooncake-store.md @@ -699,7 +699,10 @@ mooncake提供了DFS可用空间的配置,用户可以在启动master时指定 启用持久化功能后,对于每次 `Put`或`BatchPut` 操作,都会发起一次同步的memory pool写入操作和一次异步的DFS持久化操作。之后执行 `Get`或 `BatchGet` 时,如果在memory pool中没有找到对应的kvcache,则会尝试从DFS中读取该文件数据,并返回给用户。 -#### 3FS USRBIO 插件 +#### 3FS USRBIO 插件(实验性 / 未完成) + +> **实验性功能:** HF3FS(3FS USRBIO)相关集成仍在开发中,尚未达到可视为生产就绪的程度;接口与行为可能变更,建议仅用于评估与测试。 + 如需通过3FS原生接口(USRBIO)实现高性能持久化文件读写,请参阅本文档的配置说明。[3FS USRBIO 插件配置](/mooncake-store/src/hf3fs/README.md)。 ### 内置元数据服务器 -- 2.34.1 From ffff7b43bec2fa5ca1278f135384e4b9bcf4b204 Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Fri, 15 May 2026 01:02:35 +0800 Subject: [PATCH 073/382] [Doc] Align SSD offload docs with implementation (#2027) --- docs/source/deployment/ssd-offload.md | 28 ++++++++++++++++++++------- docs/source/design/ssd-offload.md | 4 +++- 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/docs/source/deployment/ssd-offload.md b/docs/source/deployment/ssd-offload.md index 46830a9e..96cd6be5 100644 --- a/docs/source/deployment/ssd-offload.md +++ b/docs/source/deployment/ssd-offload.md @@ -19,7 +19,7 @@ mkdir -p /nvme/mooncake_offload ```bash mooncake_master \ --rpc_port=50051 \ - --enable-offload true + --enable_offload=true ``` ### Step 3: Start the real client with SSD offload enabled @@ -41,7 +41,7 @@ mooncake_client \ --metadata_server="P2PHANDSHAKE" ``` -> **Note:** On startup, the real client automatically scans existing SSD data and reports it to the master. No manual recovery is needed. +> **Note:** When using `bucket_storage_backend` or `file_per_key_storage_backend`, the real client scans existing SSD metadata on startup and reports it to the master automatically. `offset_allocator_storage_backend` is the exception: it truncates its data file during initialization and does not recover previously offloaded objects after a restart. ### Step 4: Connect the application to the real client @@ -68,6 +68,7 @@ store.setup( | Flag | Default | Description | |------|---------|-------------| +| `--metadata_server` | `http://127.0.0.1:8080/metadata` | Metadata server connection string | | `--master_server_address` | `127.0.0.1:50051` | Master address | | `--host` | `0.0.0.0` | This machine's externally reachable IP | | `--port` | `50052` | Real client RPC listening port | @@ -92,6 +93,8 @@ store.setup( | `MOONCAKE_OFFLOAD_TOTAL_SIZE_LIMIT_BYTES` | `2199023255552` (2 TB) | Maximum disk usage | | `MOONCAKE_OFFLOAD_TOTAL_KEYS_LIMIT` | `10000000` | Maximum number of objects on disk | | `MOONCAKE_OFFLOAD_HEARTBEAT_INTERVAL_SECONDS` | `10` | Interval for offload heartbeat to master (seconds) | +| `MOONCAKE_OFFLOAD_CLIENT_BUFFER_GC_INTERVAL_SECONDS` | `10` | Interval for reclaiming expired offload buffers; defaults to the heartbeat interval in the current implementation | +| `MOONCAKE_OFFLOAD_CLIENT_BUFFER_GC_TTL_MS` | `5000` | Lease time for buffers returned by `batch_get_offload_object` before GC reclaims them | | `MOONCAKE_OFFLOAD_USE_URING` | `false` | Enable io_uring for async file I/O | ### Bucket backend settings @@ -105,6 +108,15 @@ Applies when `MOONCAKE_OFFLOAD_STORAGE_BACKEND_DESCRIPTOR=bucket_storage_backend | `MOONCAKE_OFFLOAD_BUCKET_MAX_TOTAL_SIZE` | `0` | Eviction threshold in bytes. When set to `0`, the backend uses **90% of the physical disk capacity** as the quota — it does not mean unlimited. Set an explicit value to control disk usage precisely. | | `MOONCAKE_OFFLOAD_BUCKET_EVICTION_POLICY` | `none` | Eviction policy: `none` / `fifo` / `lru` | +### File-per-key backend settings + +Applies when `MOONCAKE_OFFLOAD_STORAGE_BACKEND_DESCRIPTOR=file_per_key_storage_backend`. + +| Environment Variable | Default | Description | +|---|---|---| +| `MOONCAKE_OFFLOAD_FSDIR` | `file_per_key_dir` | Subdirectory name created under `MOONCAKE_OFFLOAD_FILE_STORAGE_PATH` | +| `ENABLE_EVICTION` | `true` | Enables local-storage eviction logic for this backend | + --- ## Storage Backends @@ -176,7 +188,8 @@ The following example starts a master and a real client on a single machine. ```bash mooncake_master \ - --rpc_port=50051 + --rpc_port=50051 \ + --enable_offload=true ``` ### Start the real client (new terminal) @@ -202,7 +215,7 @@ mooncake_client \ ## Notes - `MOONCAKE_OFFLOAD_FILE_STORAGE_PATH` must be an absolute path to an existing, writable directory. Symbolic links and paths containing `..` are rejected. -- On real client restart, the backend automatically scans existing SSD files and reports them to the master, so previously offloaded objects remain accessible. +- On real client restart, `bucket_storage_backend` and `file_per_key_storage_backend` scan existing SSD metadata and report it to the master, so previously offloaded objects remain accessible. `offset_allocator_storage_backend` does not support restart recovery. - Eviction only notifies the master and deletes local files; objects replicated on other nodes are unaffected. - Each machine requires its own real client process. In multi-node deployments, ensure `--host` and `--port` are correctly set so nodes can reach each other. @@ -210,7 +223,7 @@ mooncake_client \ ```bash # Node A — runs the master and its own real client -mooncake_master --rpc_port=50051 --enable-offload true & +mooncake_master --rpc_port=50051 --enable_offload=true & export MOONCAKE_OFFLOAD_FILE_STORAGE_PATH=/nvme/mooncake_offload mooncake_client \ @@ -244,8 +257,8 @@ mooncake_client \ ### SSD offload is not triggering -- Confirm `--enable_offload=true` is passed to `mooncake_client` and `--enable-offload true` is passed to `mooncake_master`. -- Check that `MOONCAKE_OFFLOAD_FILE_STORAGE_PATH` points to an existing, writable directory. The client will fail silently if the path is invalid. +- Confirm `--enable_offload=true` is passed to both `mooncake_client` and `mooncake_master`. Both binaries define the same gflags boolean; hyphenated aliases may work, but the underscored spelling matches the source. +- Check that `MOONCAKE_OFFLOAD_FILE_STORAGE_PATH` points to an existing, writable directory. If the path is invalid, real client setup fails during `FileStorageConfig::Validate()`. - Verify memory pressure is actually high enough for the master to trigger offload. If the memory pool (`--global_segment_size`) is large relative to the data written, offload may never activate. ### "Permission denied" or "No such file or directory" on the storage path @@ -253,6 +266,7 @@ mooncake_client \ - Ensure the directory exists before starting the client: `mkdir -p `. - Confirm the process user has read/write access to the directory. - Symbolic links and paths containing `..` are rejected — use an absolute, canonical path. +- The path must already exist and must be a directory. Passing a regular file path also causes setup to fail. ### "Failed to register buffer with UringFile" warning in logs diff --git a/docs/source/design/ssd-offload.md b/docs/source/design/ssd-offload.md index a9e42448..9e859ef0 100644 --- a/docs/source/design/ssd-offload.md +++ b/docs/source/design/ssd-offload.md @@ -242,4 +242,6 @@ To prevent `io_uring`'s `FOLL_LONGTERM` page pinning from failing on systems wit ## Metadata Recovery on Restart -On startup, `FileStorage::Init` calls `StorageBackend::ScanMeta`, which reads all on-disk metadata and invokes a callback for each discovered object. The callback calls `MasterClient::NotifyOffloadSuccess` to re-register the objects with the master. This restores the full disk-replica view without any application-level intervention. +On startup, `FileStorage::Init` calls `StorageBackend::ScanMeta`, which reads on-disk metadata and invokes a callback for each discovered object. The callback calls `MasterClient::NotifyOffloadSuccess` to re-register the objects with the master. This restores the full disk-replica view without any application-level intervention for the backends that preserve restart metadata, namely `BucketStorageBackend` and the file-per-key backend. + +`OffsetAllocatorStorageBackend` is the exception. It truncates its pre-allocated data file during initialization and clears its in-memory metadata, so previously offloaded objects are not recoverable after a real client restart. -- 2.34.1 From a24413a1b234adcffb0943e9062512a2655be56d Mon Sep 17 00:00:00 2001 From: ykwd Date: Fri, 15 May 2026 11:40:44 +0800 Subject: [PATCH 074/382] [Doc] Fix the inconsistent param description (#2103) --------- Co-authored-by: Ke Yang --- docs/source/deployment/mooncake-store-deployment-guide.md | 2 +- mooncake-store/src/master.cpp | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/source/deployment/mooncake-store-deployment-guide.md b/docs/source/deployment/mooncake-store-deployment-guide.md index 3923217b..27fd498f 100644 --- a/docs/source/deployment/mooncake-store-deployment-guide.md +++ b/docs/source/deployment/mooncake-store-deployment-guide.md @@ -32,11 +32,11 @@ This page summarizes useful flags, environment variables, and HTTP endpoints to - `--allow_evict_soft_pinned_objects` (bool, default `true`): Allow evicting soft-pinned objects. - `--eviction_ratio` (double, default `0.05`): Fraction evicted when hitting high watermark. - `--eviction_high_watermark_ratio` (double, default `0.95`): Usage ratio to trigger eviction. + - `--client_ttl` (int64, default `10` s): Seconds a client stays considered alive after the last heartbeat. If this TTL elapses without a refresh, the master treats the client as disconnected and may unmount its segments. - High Availability (optional) - `--enable_ha` (bool, default `false`): Enable HA (requires etcd). - `--etcd_endpoints` (str, default empty unless HA config): etcd endpoints, semicolon separated. - - `--client_ttl` (int64, default `10` s): Client alive TTL after last ping (HA mode). - `--cluster_id` (str, default `mooncake_cluster`): Cluster ID for persistence in HA mode. - Task Manager (optional) diff --git a/mooncake-store/src/master.cpp b/mooncake-store/src/master.cpp index d8704903..c21878f3 100644 --- a/mooncake-store/src/master.cpp +++ b/mooncake-store/src/master.cpp @@ -122,8 +122,9 @@ DEFINE_string( etcd_endpoints, "", "Endpoints of ETCD server, separated by semicolon, required in HA mode"); DEFINE_int64(client_ttl, mooncake::DEFAULT_CLIENT_LIVE_TTL_SEC, - "How long a client is considered alive after the last ping, only " - "used in HA mode"); + "Seconds a client stays considered alive after the last heartbeat." + "If this TTL elapses without a refresh, the master treats the " + "client as disconnected and may unmount its segments"); DEFINE_string(root_fs_dir, mooncake::DEFAULT_ROOT_FS_DIR, "Root directory for storage backend, used in HA mode"); -- 2.34.1 From 377dcba3fb0ce47ecf1f7643e36b66476d47c6ad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E3=82=86=E3=82=8A?= Date: Fri, 15 May 2026 14:52:01 +0800 Subject: [PATCH 075/382] [Store] Add comprehensive test suite for Python binding error handling (#2097) Add 20+ C++ tests and 40+ Python tests covering error handling, corner cases, and previously untested API endpoints for the mooncake-store Python binding layer. C++ tests (pybind_client_test.cpp): - batchIsExist with mixed existing/missing keys and before setup - getSize for existing keys and before setup - removeByRegex with matching, non-matching, and pre-setup cases - removeAll on empty store - batchRemove with valid, non-existent, and pre-setup keys - put_parts and put_batch then batch_get_buffer roundtrip - health_check after setup and before initialization - get_hostname verification - Double tearDownAll idempotency - Empty batch operations - Mount non-existent file, unmount invalid segment IDs Python tests (test_mooncake_store_service_api.py): - /api/put: success, missing key/value, empty key, store failure - /api/get: success, not found, empty bytes, store exception - /api/exist: true, false, store exception - /api/remove: success, failure, store exception - /api/remove_all: success, zero keys, failure, store exception - /api/reconfigure: decode/prefill success, missing params, invalid mode, remount, unmount failure rollback - /api/mount: negative/float/string/missing size - /api/unmount: missing segment_ids, empty list, success - /api/mount_shm: store failure, defaults, invalid names - /api/unmount_shm: string coercion, empty list - _shm_name_to_path: 11 unit tests for path validation edge cases Closes #654 Co-authored-by: yurekami --- mooncake-store/tests/pybind_client_test.cpp | 319 ++++++++++++++ .../tests/test_mooncake_store_service_api.py | 417 +++++++++++++++++- 2 files changed, 731 insertions(+), 5 deletions(-) diff --git a/mooncake-store/tests/pybind_client_test.cpp b/mooncake-store/tests/pybind_client_test.cpp index 182b3f21..10e32a4a 100644 --- a/mooncake-store/tests/pybind_client_test.cpp +++ b/mooncake-store/tests/pybind_client_test.cpp @@ -1356,6 +1356,325 @@ TEST_F(RealClientTest, UpsertBatch) { } } +// ===================== Batch Existence Tests ===================== + +TEST_F(RealClientTest, BatchIsExistMixed) { + StartMasterAndSetupClient(); + + ReplicateConfig config; + config.replica_num = 1; + + const std::string data = "batch_exist_data"; + std::span data_span(data.data(), data.size()); + ASSERT_EQ(py_client_->put("exist_key_1", data_span, config), 0); + ASSERT_EQ(py_client_->put("exist_key_2", data_span, config), 0); + + std::vector keys = {"exist_key_1", "missing_key", + "exist_key_2", "also_missing"}; + auto results = py_client_->batchIsExist(keys); + ASSERT_EQ(results.size(), 4u); + EXPECT_EQ(results[0], 1) << "exist_key_1 should exist"; + EXPECT_EQ(results[1], 0) << "missing_key should not exist"; + EXPECT_EQ(results[2], 1) << "exist_key_2 should exist"; + EXPECT_EQ(results[3], 0) << "also_missing should not exist"; +} + +TEST_F(RealClientTest, ErrBatchIsExistBeforeSetup) { + GLogMuter muter; + std::vector keys = {"k1", "k2"}; + auto results = py_client_->batchIsExist(keys); + ASSERT_EQ(results.size(), 2u); + for (size_t i = 0; i < results.size(); ++i) { + EXPECT_LT(results[i], 0) + << "batchIsExist[" << i << "] before setup should return negative"; + } +} + +// ===================== GetSize Tests ===================== + +TEST_F(RealClientTest, GetSizeBasic) { + StartMasterAndSetupClient(); + + const std::string data = "getsize_payload_123"; + const std::string key = "getsize_key"; + std::span data_span(data.data(), data.size()); + ReplicateConfig config; + config.replica_num = 1; + + ASSERT_EQ(py_client_->put(key, data_span, config), 0); + int64_t size = py_client_->getSize(key); + EXPECT_EQ(size, static_cast(data.size())) + << "getSize should return exact data length"; +} + +TEST_F(RealClientTest, ErrGetSizeBeforeSetup) { + GLogMuter muter; + EXPECT_LT(py_client_->getSize("any_key"), 0) + << "getSize before setup should return negative"; +} + +// ===================== RemoveByRegex Tests ===================== + +TEST_F(RealClientTest, RemoveByRegexBasic) { + StartMasterAndSetupClient(); + + ReplicateConfig config; + config.replica_num = 1; + const std::string data = "regex_data"; + std::span data_span(data.data(), data.size()); + + ASSERT_EQ(py_client_->put("prefix_alpha", data_span, config), 0); + ASSERT_EQ(py_client_->put("prefix_beta", data_span, config), 0); + ASSERT_EQ(py_client_->put("other_key", data_span, config), 0); + + long removed = py_client_->removeByRegex("^prefix_.*"); + EXPECT_EQ(removed, 2) << "Should remove exactly the two prefix_ keys"; + + EXPECT_EQ(py_client_->isExist("prefix_alpha"), 0); + EXPECT_EQ(py_client_->isExist("prefix_beta"), 0); + EXPECT_EQ(py_client_->isExist("other_key"), 1) + << "Non-matching key should survive"; +} + +TEST_F(RealClientTest, RemoveByRegexNoMatch) { + StartMasterAndSetupClient(); + + ReplicateConfig config; + config.replica_num = 1; + const std::string data = "no_match_data"; + std::span data_span(data.data(), data.size()); + ASSERT_EQ(py_client_->put("some_key", data_span, config), 0); + + long removed = py_client_->removeByRegex("^nonexistent_pattern_.*"); + EXPECT_EQ(removed, 0) << "No keys should match"; + EXPECT_EQ(py_client_->isExist("some_key"), 1) + << "Existing key should remain"; +} + +TEST_F(RealClientTest, ErrRemoveByRegexBeforeSetup) { + GLogMuter muter; + long result = py_client_->removeByRegex(".*"); + EXPECT_LT(result, 0) << "removeByRegex before setup should return negative"; +} + +// ===================== RemoveAll Edge Cases ===================== + +TEST_F(RealClientTest, RemoveAllOnEmptyStore) { + StartMasterAndSetupClient(); + long removed = py_client_->removeAll(); + EXPECT_EQ(removed, 0) << "removeAll on empty store should return 0"; +} + +// ===================== BatchRemove Tests ===================== + +TEST_F(RealClientTest, BatchRemoveBasic) { + StartMasterAndSetupClient(); + + ReplicateConfig config; + config.replica_num = 1; + const std::string data = "batch_rm_data"; + std::span data_span(data.data(), data.size()); + + ASSERT_EQ(py_client_->put("brm_1", data_span, config), 0); + ASSERT_EQ(py_client_->put("brm_2", data_span, config), 0); + ASSERT_EQ(py_client_->put("brm_3", data_span, config), 0); + + std::vector keys_to_remove = {"brm_1", "brm_3"}; + auto results = py_client_->batchRemove(keys_to_remove); + ASSERT_EQ(results.size(), 2u); + for (size_t i = 0; i < results.size(); ++i) { + EXPECT_EQ(results[i], 0) << "batchRemove[" << i << "] should succeed"; + } + + EXPECT_EQ(py_client_->isExist("brm_1"), 0); + EXPECT_EQ(py_client_->isExist("brm_2"), 1) << "brm_2 should survive"; + EXPECT_EQ(py_client_->isExist("brm_3"), 0); +} + +TEST_F(RealClientTest, BatchRemoveNonExistentKeys) { + StartMasterAndSetupClient(); + + GLogMuter muter; + std::vector keys = {"never_existed_1", "never_existed_2"}; + auto results = py_client_->batchRemove(keys); + ASSERT_EQ(results.size(), 2u); +} + +TEST_F(RealClientTest, ErrBatchRemoveBeforeSetup) { + GLogMuter muter; + std::vector keys = {"k1", "k2"}; + auto results = py_client_->batchRemove(keys); + ASSERT_EQ(results.size(), 2u); + for (size_t i = 0; i < results.size(); ++i) { + EXPECT_NE(results[i], 0) + << "batchRemove[" << i << "] before setup should fail"; + } +} + +// ===================== PutParts Tests ===================== + +TEST_F(RealClientTest, PutPartsBasic) { + StartMasterAndSetupClient(); + + ReplicateConfig config; + config.replica_num = 1; + const std::string key = "put_parts_key"; + + const std::string part1 = "Hello, "; + const std::string part2 = "Parts!"; + const std::string expected = part1 + part2; + std::vector> parts; + parts.emplace_back(part1.data(), part1.size()); + parts.emplace_back(part2.data(), part2.size()); + + EXPECT_EQ(py_client_->put_parts(key, parts, config), 0); + + auto buf = py_client_->get_buffer(key); + ASSERT_NE(buf, nullptr); + EXPECT_EQ(buf->size(), expected.size()); + EXPECT_EQ(std::string(static_cast(buf->ptr()), buf->size()), + expected); +} + +TEST_F(RealClientTest, ErrPutPartsBeforeSetup) { + GLogMuter muter; + const std::string part = "data"; + std::vector> parts; + parts.emplace_back(part.data(), part.size()); + ReplicateConfig config; + config.replica_num = 1; + EXPECT_NE(py_client_->put_parts("key", parts, config), 0) + << "put_parts before setup should fail"; +} + +// ===================== PutBatch and GetBatch Tests ===================== + +TEST_F(RealClientTest, PutBatchThenBatchGetBuffer) { + StartMasterAndSetupClient(); + + ReplicateConfig config; + config.replica_num = 1; + + std::vector keys = {"batch_kv_0", "batch_kv_1", "batch_kv_2"}; + std::string val0 = "value_zero"; + std::string val1 = "value_one!"; + std::string val2 = "value_two!"; + std::vector> values; + values.emplace_back(val0.data(), val0.size()); + values.emplace_back(val1.data(), val1.size()); + values.emplace_back(val2.data(), val2.size()); + + ASSERT_EQ(py_client_->put_batch(keys, values, config), 0); + + auto handles = py_client_->batch_get_buffer(keys); + ASSERT_EQ(handles.size(), 3u); + + std::vector expected = {&val0, &val1, &val2}; + for (size_t i = 0; i < handles.size(); ++i) { + ASSERT_NE(handles[i], nullptr) + << "Handle " << i << " should not be null"; + EXPECT_EQ(std::string(static_cast(handles[i]->ptr()), + handles[i]->size()), + *expected[i]) + << "Data mismatch for key " << keys[i]; + } +} + +// ===================== HealthCheck Tests ===================== + +TEST_F(RealClientTest, HealthCheckAfterSetup) { + StartMasterAndSetupClient(); + EXPECT_EQ(py_client_->health_check(), HC_HEALTHY) + << "health_check should return HEALTHY after setup"; +} + +TEST_F(RealClientTest, HealthCheckBeforeSetup) { + int result = py_client_->health_check(); + EXPECT_EQ(result, HC_NOT_INITIALIZED) + << "health_check before setup should return NOT_INITIALIZED"; +} + +// ===================== GetHostname Tests ===================== + +TEST_F(RealClientTest, GetHostnameAfterSetup) { + StartMasterAndSetupClient(); + std::string hostname = py_client_->get_hostname(); + EXPECT_FALSE(hostname.empty()) << "get_hostname should return non-empty"; + EXPECT_EQ(hostname, "localhost:17813") + << "get_hostname should match the configured hostname"; +} + +// ===================== Double TearDown Tests ===================== + +TEST_F(RealClientTest, DoubleTearDownIsIdempotent) { + StartMasterAndSetupClient(); + + EXPECT_EQ(py_client_->tearDownAll(), 0) << "First teardown should succeed"; + EXPECT_EQ(py_client_->tearDownAll(), 0) + << "Second teardown should also succeed (idempotent)"; +} + +// ===================== Empty Batch Operations ===================== + +TEST_F(RealClientTest, EmptyBatchOperations) { + StartMasterAndSetupClient(); + + std::vector empty_keys; + std::vector> empty_values; + ReplicateConfig config; + config.replica_num = 1; + + EXPECT_EQ(py_client_->put_batch(empty_keys, empty_values, config), 0) + << "put_batch with empty input should succeed"; + + auto handles = py_client_->batch_get_buffer(empty_keys); + EXPECT_TRUE(handles.empty()) + << "batch_get_buffer with empty input should return empty"; + + auto exist_results = py_client_->batchIsExist(empty_keys); + EXPECT_TRUE(exist_results.empty()) + << "batchIsExist with empty input should return empty"; + + auto remove_results = py_client_->batchRemove(empty_keys); + EXPECT_TRUE(remove_results.empty()) + << "batchRemove with empty input should return empty"; +} + +// ===================== Mount Segment Edge Cases ===================== + +TEST_F(RealClientTest, ErrMountNonExistentFile) { + StartMasterAndSetupClient(); + + GLogMuter muter; + std::vector segment_ids; + int ret = + py_client_->mountSegment("/tmp/mooncake_nonexistent_file_12345", 0, + 4096, FLAGS_protocol, "", segment_ids); + EXPECT_NE(ret, 0) << "Mounting non-existent file should fail"; + EXPECT_TRUE(segment_ids.empty()); +} + +TEST_F(RealClientTest, ErrUnmountInvalidSegmentIds) { + StartMasterAndSetupClient(); + + GLogMuter muter; + std::vector bogus_ids = { + "00000000-0000-0000-0000-000000000000"}; + int ret = py_client_->unmountSegment(bogus_ids); + EXPECT_NE(ret, 0) << "Unmounting non-existent segment ids should fail"; +} + +TEST_F(RealClientTest, ErrUnmountAndFreeInvalidSegmentIds) { + StartMasterAndSetupClient(); + + GLogMuter muter; + std::vector bogus_ids = { + "00000000-0000-0000-0000-000000000000"}; + int ret = py_client_->unmountAndFreeSegment(bogus_ids); + EXPECT_NE(ret, 0) + << "Unmount-and-free of non-existent segment ids should fail"; +} + } // namespace testing } // namespace mooncake diff --git a/mooncake-wheel/tests/test_mooncake_store_service_api.py b/mooncake-wheel/tests/test_mooncake_store_service_api.py index 23b2f229..e20544cd 100644 --- a/mooncake-wheel/tests/test_mooncake_store_service_api.py +++ b/mooncake-wheel/tests/test_mooncake_store_service_api.py @@ -10,7 +10,7 @@ from types import SimpleNamespace sys.path.insert(0, str(Path(__file__).resolve().parents[1])) try: - from aiohttp import web as _unused_web + from aiohttp import web as _unused_web # noqa: F401 except ModuleNotFoundError: aiohttp_module = types.ModuleType("aiohttp") web_module = types.ModuleType("aiohttp.web") @@ -27,7 +27,7 @@ except ModuleNotFoundError: sys.modules["aiohttp.web"] = web_module try: - from mooncake.store import MooncakeDistributedStore as _unused_store + from mooncake.store import MooncakeDistributedStore as _unused_store # noqa: F401 except ModuleNotFoundError: store_module = types.ModuleType("mooncake.store") @@ -37,7 +37,7 @@ except ModuleNotFoundError: store_module.MooncakeDistributedStore = MooncakeDistributedStore sys.modules["mooncake.store"] = store_module -from mooncake.mooncake_store_service import MooncakeStoreService +from mooncake.mooncake_store_service import MooncakeStoreService, _shm_name_to_path class FakeStore: @@ -201,8 +201,7 @@ class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): ["00000000-0000-0000-0000-000000000002"], ) self.assertEqual(mount_body["allocated_size"], 4096) - self.assertEqual(self.fake_store.allocated_mount_calls, - [(1, "tcp", "cpu:0")]) + self.assertEqual(self.fake_store.allocated_mount_calls, [(1, "tcp", "cpu:0")]) unmount_resp = await self.service.handle_unmount( FakeRequest({"segment_ids": mount_body["segment_ids"]}) @@ -243,6 +242,414 @@ class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): body = json.loads(resp.text) self.assertIn("Missing segment_ids", body["error"]) + # ==================== /api/put tests ==================== + + async def test_handle_put_success(self): + self.fake_store.put = lambda key, value: 0 + resp = await self.service.handle_put( + FakeRequest({"key": "test_key", "value": "test_value"}) + ) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertEqual(body["status"], "success") + + async def test_handle_put_missing_key(self): + self.fake_store.put = lambda key, value: 0 + resp = await self.service.handle_put(FakeRequest({"value": "test_value"})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Missing key or value", body["error"]) + + async def test_handle_put_missing_value(self): + self.fake_store.put = lambda key, value: 0 + resp = await self.service.handle_put(FakeRequest({"key": "k"})) + self.assertEqual(resp.status, 500) + + async def test_handle_put_store_failure(self): + self.fake_store.put = lambda key, value: -1 + resp = await self.service.handle_put(FakeRequest({"key": "k", "value": "v"})) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("PUT operation failed", body["error"]) + + async def test_handle_put_empty_key(self): + self.fake_store.put = lambda key, value: 0 + resp = await self.service.handle_put(FakeRequest({"key": "", "value": "v"})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Missing key or value", body["error"]) + + # ==================== /api/get/{key} tests ==================== + + async def test_handle_get_success(self): + self.fake_store.get = lambda key: b"payload_bytes" + request = FakeRequest({}) + request.match_info = {"key": "my_key"} + resp = await self.service.handle_get(request) + self.assertEqual(resp.status, 200) + self.assertEqual(resp.body, b"payload_bytes") + + async def test_handle_get_not_found(self): + self.fake_store.get = lambda key: None + request = FakeRequest({}) + request.match_info = {"key": "missing_key"} + resp = await self.service.handle_get(request) + self.assertEqual(resp.status, 404) + body = json.loads(resp.text) + self.assertIn("Key not found", body["error"]) + + async def test_handle_get_empty_bytes(self): + self.fake_store.get = lambda key: b"" + request = FakeRequest({}) + request.match_info = {"key": "empty_value_key"} + resp = await self.service.handle_get(request) + self.assertEqual(resp.status, 404) + body = json.loads(resp.text) + self.assertIn("Key not found", body["error"]) + + async def test_handle_get_store_exception(self): + def raise_error(key): + raise RuntimeError("store crashed") + + self.fake_store.get = raise_error + request = FakeRequest({}) + request.match_info = {"key": "crash_key"} + resp = await self.service.handle_get(request) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("store crashed", body["error"]) + + # ==================== /api/exist/{key} tests ==================== + + async def test_handle_exist_true(self): + self.fake_store.is_exist = lambda key: True + request = FakeRequest({}) + request.match_info = {"key": "existing_key"} + resp = await self.service.handle_exist(request) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertTrue(body["exists"]) + + async def test_handle_exist_false(self): + self.fake_store.is_exist = lambda key: False + request = FakeRequest({}) + request.match_info = {"key": "missing_key"} + resp = await self.service.handle_exist(request) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertFalse(body["exists"]) + + async def test_handle_exist_store_exception(self): + def raise_error(key): + raise RuntimeError("exist check crashed") + + self.fake_store.is_exist = raise_error + request = FakeRequest({}) + request.match_info = {"key": "crash_key"} + resp = await self.service.handle_exist(request) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("exist check crashed", body["error"]) + + # ==================== /api/remove/{key} tests ==================== + + async def test_handle_remove_success(self): + self.fake_store.remove = lambda key: 0 + request = FakeRequest({}) + request.match_info = {"key": "removable_key"} + resp = await self.service.handle_remove(request) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertEqual(body["status"], "success") + + async def test_handle_remove_failure(self): + self.fake_store.remove = lambda key: -1 + request = FakeRequest({}) + request.match_info = {"key": "stuck_key"} + resp = await self.service.handle_remove(request) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("Remove operation failed", body["error"]) + + async def test_handle_remove_store_exception(self): + def raise_error(key): + raise RuntimeError("remove crashed") + + self.fake_store.remove = raise_error + request = FakeRequest({}) + request.match_info = {"key": "crash_key"} + resp = await self.service.handle_remove(request) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("remove crashed", body["error"]) + + # ==================== /api/remove_all tests ==================== + + async def test_handle_remove_all_success(self): + self.fake_store.remove_all = lambda: 5 + resp = await self.service.handle_remove_all(FakeRequest({})) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertIn("5", body["status"]) + + async def test_handle_remove_all_zero_keys(self): + self.fake_store.remove_all = lambda: 0 + resp = await self.service.handle_remove_all(FakeRequest({})) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertIn("0", body["status"]) + + async def test_handle_remove_all_failure(self): + self.fake_store.remove_all = lambda: -1 + resp = await self.service.handle_remove_all(FakeRequest({})) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("RemoveAll operation failed", body["error"]) + + async def test_handle_remove_all_store_exception(self): + def raise_error(): + raise RuntimeError("remove_all crashed") + + self.fake_store.remove_all = raise_error + resp = await self.service.handle_remove_all(FakeRequest({})) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("remove_all crashed", body["error"]) + + # ==================== /api/reconfigure tests ==================== + + async def test_reconfigure_decode_success(self): + self.fake_store.fail_mount = False + resp = await self.service.handle_reconfigure( + FakeRequest( + { + "mode": "decode", + "path": "/dev/shm/test", + "size": 4096, + } + ) + ) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertEqual(body["mode"], "decode") + self.assertEqual(body["status"], "success") + self.assertIn("segment_ids", body) + self.assertEqual(self.service.current_mode, "decode") + + async def test_reconfigure_decode_missing_path(self): + resp = await self.service.handle_reconfigure( + FakeRequest({"mode": "decode", "size": 4096}) + ) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Missing path or size", body["error"]) + + async def test_reconfigure_decode_missing_size(self): + resp = await self.service.handle_reconfigure( + FakeRequest({"mode": "decode", "path": "/dev/shm/test"}) + ) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Missing path or size", body["error"]) + + async def test_reconfigure_prefill_success(self): + self.service.current_mode = "decode" + self.service.mounted_segment_ids = [] + resp = await self.service.handle_reconfigure(FakeRequest({"mode": "prefill"})) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertEqual(body["mode"], "prefill") + self.assertEqual(self.service.current_mode, "prefill") + + async def test_reconfigure_prefill_unmounts_segments(self): + sid = "00000000-0000-0000-0000-000000000001" + self.service.current_mode = "decode" + self.service.mounted_segment_ids = [sid] + resp = await self.service.handle_reconfigure(FakeRequest({"mode": "prefill"})) + self.assertEqual(resp.status, 200) + self.assertEqual(self.fake_store.unmount_calls, [[sid]]) + self.assertEqual(self.service.mounted_segment_ids, []) + self.assertEqual(self.service.current_mode, "prefill") + + async def test_reconfigure_prefill_unmount_failure(self): + sid = "00000000-0000-0000-0000-000000000002" + self.service.current_mode = "decode" + self.service.mounted_segment_ids = [sid] + self.fake_store.unmount_failures = {sid} + resp = await self.service.handle_reconfigure(FakeRequest({"mode": "prefill"})) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("Unmount failed", body["error"]) + + async def test_reconfigure_invalid_mode(self): + resp = await self.service.handle_reconfigure(FakeRequest({"mode": "invalid"})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Invalid mode", body["error"]) + + async def test_reconfigure_empty_mode(self): + resp = await self.service.handle_reconfigure(FakeRequest({})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Invalid mode", body["error"]) + + async def test_reconfigure_decode_remount_unmounts_previous(self): + old_id = "00000000-0000-0000-0000-000000000001" + self.service.current_mode = "decode" + self.service.mounted_segment_ids = [old_id] + self.fake_store.fail_mount = False + resp = await self.service.handle_reconfigure( + FakeRequest( + { + "mode": "decode", + "path": "/dev/shm/new", + "size": 8192, + } + ) + ) + self.assertEqual(resp.status, 200) + self.assertEqual(self.fake_store.unmount_calls, [[old_id]]) + self.assertEqual(self.service.current_mode, "decode") + + # ==================== /api/mount edge cases ==================== + + async def test_mount_negative_size(self): + resp = await self.service.handle_mount(FakeRequest({"size": -1})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Invalid size", body["error"]) + + async def test_mount_float_size(self): + resp = await self.service.handle_mount(FakeRequest({"size": 1.5})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Invalid size", body["error"]) + + async def test_mount_string_size(self): + resp = await self.service.handle_mount(FakeRequest({"size": "1024"})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Invalid size", body["error"]) + + async def test_mount_missing_size(self): + resp = await self.service.handle_mount(FakeRequest({})) + self.assertEqual(resp.status, 400) + + # ==================== /api/unmount edge cases ==================== + + async def test_unmount_requires_segment_ids(self): + resp = await self.service.handle_unmount(FakeRequest({})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Missing segment_ids", body["error"]) + + async def test_unmount_empty_list(self): + resp = await self.service.handle_unmount(FakeRequest({"segment_ids": []})) + self.assertEqual(resp.status, 400) + body = json.loads(resp.text) + self.assertIn("Missing segment_ids", body["error"]) + + async def test_unmount_success(self): + resp = await self.service.handle_unmount( + FakeRequest({"segment_ids": ["00000000-0000-0000-0000-000000000002"]}) + ) + self.assertEqual(resp.status, 200) + body = json.loads(resp.text) + self.assertEqual(body["status"], "success") + + # ==================== /api/mount_shm edge cases ==================== + + async def test_mount_shm_store_failure(self): + self.fake_store.fail_mount = True + resp = await self.service.handle_mount_shm( + FakeRequest({"name": "test-seg", "size": 4096}) + ) + self.assertEqual(resp.status, 500) + body = json.loads(resp.text) + self.assertIn("Mount failed", body["error"]) + + async def test_mount_shm_with_defaults(self): + resp = await self.service.handle_mount_shm( + FakeRequest({"name": "minimal-seg", "size": 2048}) + ) + self.assertEqual(resp.status, 200) + self.assertEqual( + self.fake_store.mount_calls, + [("/dev/shm/minimal-seg", 2048, 0, "tcp", "")], + ) + + async def test_mount_shm_empty_name(self): + resp = await self.service.handle_mount_shm( + FakeRequest({"name": "", "size": 4096}) + ) + self.assertEqual(resp.status, 400) + + async def test_mount_shm_dot_name(self): + resp = await self.service.handle_mount_shm( + FakeRequest({"name": ".", "size": 4096}) + ) + self.assertEqual(resp.status, 400) + + async def test_mount_shm_dotdot_name(self): + resp = await self.service.handle_mount_shm( + FakeRequest({"name": "..", "size": 4096}) + ) + self.assertEqual(resp.status, 400) + + # ==================== /api/unmount_shm edge cases ==================== + + async def test_unmount_shm_string_segment_id_coercion(self): + resp = await self.service.handle_unmount_shm( + FakeRequest({"segment_ids": "00000000-0000-0000-0000-000000000001"}) + ) + self.assertEqual(resp.status, 200) + self.assertEqual( + self.fake_store.unmount_calls, + [["00000000-0000-0000-0000-000000000001"]], + ) + + async def test_unmount_shm_empty_list(self): + resp = await self.service.handle_unmount_shm(FakeRequest({"segment_ids": []})) + self.assertEqual(resp.status, 400) + + +class ShmNameToPathTest(unittest.TestCase): + def test_valid_simple_name(self): + self.assertEqual(_shm_name_to_path("my-segment"), "/dev/shm/my-segment") + + def test_valid_leading_slash(self): + self.assertEqual(_shm_name_to_path("/my-segment"), "/dev/shm/my-segment") + + def test_rejects_empty(self): + self.assertIsNone(_shm_name_to_path("")) + + def test_rejects_none(self): + self.assertIsNone(_shm_name_to_path(None)) + + def test_rejects_non_string(self): + self.assertIsNone(_shm_name_to_path(123)) + + def test_rejects_dot(self): + self.assertIsNone(_shm_name_to_path(".")) + + def test_rejects_dotdot(self): + self.assertIsNone(_shm_name_to_path("..")) + + def test_rejects_path_traversal(self): + self.assertIsNone(_shm_name_to_path("../etc/passwd")) + + def test_rejects_nested_path(self): + self.assertIsNone(_shm_name_to_path("subdir/file")) + + def test_rejects_slash_only(self): + self.assertIsNone(_shm_name_to_path("/")) + + def test_rejects_slash_dot(self): + self.assertIsNone(_shm_name_to_path("/.")) + + def test_rejects_slash_dotdot(self): + self.assertIsNone(_shm_name_to_path("/..")) + if __name__ == "__main__": unittest.main() -- 2.34.1 From 8eb0e2e5d1057b474ccd5c31a2e8ca828050b32d Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Fri, 15 May 2026 17:36:46 +0800 Subject: [PATCH 076/382] [CI] pin torch version to 2.11.0 (#2105) PyTorch 2.12.0 does not natively support CUDA 12.8. Patching around this will make our CI script fragmented. --- scripts/test_installation.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/test_installation.sh b/scripts/test_installation.sh index 8c65abdf..9e7f0776 100755 --- a/scripts/test_installation.sh +++ b/scripts/test_installation.sh @@ -37,7 +37,7 @@ echo "Running import structure test..." # Run the import structure test cp -r mooncake-wheel/tests test_env/ cd test_env -pip install torch numpy +pip install torch==2.11.0 numpy python tests/test_import_structure.py echo "Running mooncake config test..." -- 2.34.1 From 9128a63ed96898bd8a214f0bd6957c7db6fa302b Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Fri, 15 May 2026 19:32:12 +0800 Subject: [PATCH 077/382] =?UTF-8?q?[PG]=20update=20EP/PG=20torch=20version?= =?UTF-8?q?s=20=E2=80=94=20drop=202.9.0,=20add=202.12.0=20(#2101)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PyTorch 2.12.0 no longer publishes cu128 wheels (only cu130 and cu126). Add a cmake branch in SetupPyTorchEnv.cmake that routes torch 2.12.0+ on CUDA 12 hosts to the cu126 wheel index. Changes: - EP_TORCH_VERSIONS: 2.9.0;2.9.1;2.10.0;2.11.0 → 2.9.1;2.10.0;2.11.0;2.12.0 - SetupPyTorchEnv.cmake: add version >= 2.12.0 branch → cu126 --- .github/workflows/ci.yml | 2 +- .github/workflows/ci_cu13.yml | 2 +- .github/workflows/release-cuda13.yaml | 2 +- .github/workflows/release.yaml | 2 +- mooncake-common/SetupPyTorchEnv.cmake | 12 ++++++++++++ 5 files changed, 16 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f0f028d3..3450de4c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -606,7 +606,7 @@ jobs: run: | cd build rm -r */tests - cmake -G Ninja .. -DBUILD_UNIT_TESTS=OFF -DBUILD_EXAMPLES=OFF -DUSE_HTTP=ON -DENABLE_SCCACHE=ON -DUSE_CXL=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.0;2.9.1;2.10.0;2.11.0" + cmake -G Ninja .. -DBUILD_UNIT_TESTS=OFF -DBUILD_EXAMPLES=OFF -DUSE_HTTP=ON -DENABLE_SCCACHE=ON -DUSE_CXL=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.1;2.10.0;2.11.0;2.12.0" shell: bash - name: Build project diff --git a/.github/workflows/ci_cu13.yml b/.github/workflows/ci_cu13.yml index 0fae01f7..ba28fa49 100644 --- a/.github/workflows/ci_cu13.yml +++ b/.github/workflows/ci_cu13.yml @@ -75,7 +75,7 @@ jobs: -DWITH_STORE=ON \ -DWITH_P2P_STORE=ON \ -DWITH_EP=ON \ - -DEP_TORCH_VERSIONS="2.9.0;2.9.1;2.10.0;2.11.0" \ + -DEP_TORCH_VERSIONS="2.9.1;2.10.0;2.11.0;2.12.0" \ -DWITH_METRICS=ON \ -DBUILD_UNIT_TESTS=OFF \ -DBUILD_EXAMPLES=ON \ diff --git a/.github/workflows/release-cuda13.yaml b/.github/workflows/release-cuda13.yaml index 42d8fc0b..ed9ec443 100644 --- a/.github/workflows/release-cuda13.yaml +++ b/.github/workflows/release-cuda13.yaml @@ -65,7 +65,7 @@ jobs: sudo bash -x dependencies.sh -y mkdir build cd build - cmake .. -DBUILD_UNIT_TESTS=OFF -DUSE_HTTP=ON -DUSE_ETCD=ON -DUSE_CUDA=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.0;2.9.1;2.10.0;2.11.0" -DSTORE_USE_ETCD=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Release + cmake .. -DBUILD_UNIT_TESTS=OFF -DUSE_HTTP=ON -DUSE_ETCD=ON -DUSE_CUDA=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.1;2.10.0;2.11.0;2.12.0" -DSTORE_USE_ETCD=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Release shell: bash - name: Build project diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index 26d5398d..edef88fa 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -64,7 +64,7 @@ jobs: sudo bash -x dependencies.sh -y mkdir build cd build - cmake .. -DBUILD_UNIT_TESTS=OFF -DUSE_HTTP=ON -DUSE_ETCD=ON -DUSE_CUDA=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.0;2.9.1;2.10.0;2.11.0" -DSTORE_USE_ETCD=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Release + cmake .. -DBUILD_UNIT_TESTS=OFF -DUSE_HTTP=ON -DUSE_ETCD=ON -DUSE_CUDA=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.1;2.10.0;2.11.0;2.12.0" -DSTORE_USE_ETCD=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Release shell: bash - name: Build project diff --git a/mooncake-common/SetupPyTorchEnv.cmake b/mooncake-common/SetupPyTorchEnv.cmake index 526a190a..0a00d1b7 100644 --- a/mooncake-common/SetupPyTorchEnv.cmake +++ b/mooncake-common/SetupPyTorchEnv.cmake @@ -23,6 +23,18 @@ function(install_pytorch_wheel _version _cuda_major _cuda_minor _module_prefix) # TODO: Fix when we need to support more CUDA 13 versions or when the CI env is fixed. set(_cu_tag "cu130") + elseif(_cuda_major EQUAL 12 AND _version VERSION_GREATER_EQUAL "2.12.0") + # PyTorch 2.12.0+ no longer publishes cu128 wheels. + # Use cu126 for all CUDA 12 builds. + if(_cuda_minor GREATER_EQUAL 6) + set(_cu_tag "cu126") + else() + message(FATAL_ERROR + "${_module_prefix} Can't find a matching PyTorch wheel for version ${_version} " + "with CUDA ${_cuda_major}.${_cuda_minor}" + ) + endif() + elseif(_cuda_major EQUAL 12 AND _version VERSION_GREATER_EQUAL "2.11.0") # PyTorch 2.11.0+ defaults to CUDA 13. # We must explicitly point to CUDA 12 wheels for these newer versions. -- 2.34.1 From fdf95f95653dbe3deea74df388c5f07e2ed8467b Mon Sep 17 00:00:00 2001 From: Vladislav Nosivskoy Date: Sat, 16 May 2026 06:59:11 +0300 Subject: [PATCH 078/382] [Store][K8s-Native][2/N] K8s leader election (#1956) --------- Signed-off-by: Vladislav Nosivskoy --- mooncake-store/CMakeLists.txt | 7 +- .../backends/k8s/k8s_leader_coordinator.h | 71 +++ mooncake-store/include/k8s_lease_helper.h | 44 ++ mooncake-store/include/master_config.h | 33 +- mooncake-store/include/types.h | 2 + mooncake-store/src/CMakeLists.txt | 10 + .../backends/k8s/k8s_leader_coordinator.cpp | 456 ++++++++++++++++++ .../leadership/leader_coordinator_factory.cpp | 16 +- .../leadership/master_service_supervisor.cpp | 7 +- mooncake-store/src/k8s_lease_helper.cpp | 203 ++++++++ mooncake-store/src/master.cpp | 18 +- mooncake-store/src/types.cpp | 3 + mooncake-store/tests/CMakeLists.txt | 11 +- .../k8s/high_availability_k8s_test.cpp | 347 +++++++++++++ 14 files changed, 1201 insertions(+), 27 deletions(-) create mode 100644 mooncake-store/include/ha/leadership/backends/k8s/k8s_leader_coordinator.h create mode 100644 mooncake-store/include/k8s_lease_helper.h create mode 100644 mooncake-store/src/ha/leadership/backends/k8s/k8s_leader_coordinator.cpp create mode 100644 mooncake-store/src/k8s_lease_helper.cpp create mode 100644 mooncake-store/tests/ha/leadership/backends/k8s/high_availability_k8s_test.cpp diff --git a/mooncake-store/CMakeLists.txt b/mooncake-store/CMakeLists.txt index 3f12e505..f01e657a 100644 --- a/mooncake-store/CMakeLists.txt +++ b/mooncake-store/CMakeLists.txt @@ -10,13 +10,18 @@ if (STORE_USE_ETCD) set(ETCD_WRAPPER_LIB ${CMAKE_CURRENT_BINARY_DIR}/../mooncake-common/etcd/libetcd_wrapper.so) endif() +if (STORE_USE_K8S_LEASE) + set(K8S_LEASE_WRAPPER_INCLUDE ${CMAKE_CURRENT_BINARY_DIR}/../mooncake-common/k8s-lease/) + set(K8S_LEASE_WRAPPER_LIB ${CMAKE_CURRENT_BINARY_DIR}/../mooncake-common/k8s-lease/libk8s_lease_wrapper.so) +endif() + if (STORE_USE_REDIS) find_path(MOONCAKE_STORE_HIREDIS_INCLUDE_DIR hiredis/hiredis.h REQUIRED) find_library(MOONCAKE_STORE_HIREDIS_LIBRARY hiredis REQUIRED) message(STATUS "Redis HA backend: Enabled") endif() -if (NOT STORE_USE_ETCD AND NOT STORE_USE_REDIS) +if (NOT STORE_USE_ETCD AND NOT STORE_USE_REDIS AND NOT STORE_USE_K8S_LEASE) message(STATUS "Store HA backends are disabled") endif() diff --git a/mooncake-store/include/ha/leadership/backends/k8s/k8s_leader_coordinator.h b/mooncake-store/include/ha/leadership/backends/k8s/k8s_leader_coordinator.h new file mode 100644 index 00000000..9621c26f --- /dev/null +++ b/mooncake-store/include/ha/leadership/backends/k8s/k8s_leader_coordinator.h @@ -0,0 +1,71 @@ +#pragma once + +#include +#include +#include +#include + +#include "k8s_lease_helper.h" +#include "ha/leadership/leader_coordinator.h" + +namespace mooncake { +namespace ha { +namespace backends { +namespace k8s { + +class K8sLeaderCoordinator final : public LeaderCoordinator { + public: + explicit K8sLeaderCoordinator(const HABackendSpec& spec); + ~K8sLeaderCoordinator() override; + + ErrorCode Connect(); + + tl::expected, ErrorCode> ReadCurrentView() + override; + + tl::expected TryAcquireLeadership( + const std::string& leader_address) override; + + tl::expected RenewLeadership( + const LeadershipSession& session) override; + + tl::expected WaitForViewChange( + std::optional known_version, + std::chrono::milliseconds timeout) override; + + tl::expected, ErrorCode> + StartLeadershipMonitor(const LeadershipSession& session, + LeadershipLostCallback on_leadership_lost) override; + + ErrorCode ReleaseLeadership(const LeadershipSession& session) override; + + private: + ErrorCode EnsureConnected(); + ErrorCode ShutdownElection(); + void ClearLeadershipMonitorStateLocked(); + bool IsSameViewVersion(const std::optional& current_view, + std::optional known_version) const; + static tl::expected, ErrorCode> + ParseConnstring(const std::string& connstring); + + HABackendSpec spec_; + std::string namespace_; + std::string lease_name_; + bool connected_ = false; + + std::mutex election_mutex_; + std::thread election_monitor_thread_; + std::string election_identity_; + OwnerToken election_owner_token_; + bool election_active_ = false; + bool election_monitor_stopped_ = false; + bool election_shutdown_requested_ = false; + LeadershipLostCallback leadership_monitor_callback_; + std::shared_ptr> leadership_monitor_armed_; + OwnerToken leadership_monitor_owner_token_; +}; + +} // namespace k8s +} // namespace backends +} // namespace ha +} // namespace mooncake diff --git a/mooncake-store/include/k8s_lease_helper.h b/mooncake-store/include/k8s_lease_helper.h new file mode 100644 index 00000000..16d42e04 --- /dev/null +++ b/mooncake-store/include/k8s_lease_helper.h @@ -0,0 +1,44 @@ +#pragma once + +#include +#include +#include + +#include "types.h" + +namespace mooncake { + +class K8sLeaseHelper { + public: + static ErrorCode Init(); + + static ErrorCode RunElection(const std::string& ns, + const std::string& lease, + const std::string& identity, int lease_dur, + int renew_deadline, int retry_period); + + static ErrorCode WaitElected(const std::string& ns, + const std::string& lease, int timeout_sec, + int64_t& lease_transitions); + + static ErrorCode WaitLost(const std::string& ns, const std::string& lease); + + static ErrorCode CancelElection(const std::string& ns, + const std::string& lease); + + static ErrorCode GetHolder(const std::string& ns, const std::string& lease, + std::string& holder, int64_t& lease_transitions); + + static ErrorCode WatchHolder( + const std::string& ns, const std::string& lease, void* callback_context, + void (*callback_func)(void*, const char*, size_t, int64_t)); + + static ErrorCode CancelWatch(const std::string& ns, + const std::string& lease); + + private: + static std::mutex init_mutex_; + static bool initialized_; +}; + +} // namespace mooncake diff --git a/mooncake-store/include/master_config.h b/mooncake-store/include/master_config.h index ea3bf4d8..bee3dbb3 100644 --- a/mooncake-store/include/master_config.h +++ b/mooncake-store/include/master_config.h @@ -2,6 +2,7 @@ #include #include +#include #include @@ -10,6 +11,18 @@ namespace mooncake { +inline std::string ResolveConfiguredHABackendConnstring( + std::string_view ha_backend_type, std::string_view ha_backend_connstring, + std::string_view etcd_endpoints) { + if (!ha_backend_connstring.empty()) { + return std::string(ha_backend_connstring); + } + if (ha_backend_type == "etcd") { + return std::string(etcd_endpoints); + } + return {}; +} + // The configuration for the master server struct MasterConfig { bool enable_metric_reporting; @@ -172,11 +185,9 @@ class MasterServiceSupervisorConfig { std::chrono::seconds(config.rpc_conn_timeout_seconds); rpc_enable_tcp_no_delay = config.rpc_enable_tcp_no_delay; ha_backend_type = config.ha_backend_type; - ha_backend_connstring = config.ha_backend_connstring; etcd_endpoints = config.etcd_endpoints; - if (ha_backend_connstring.empty()) { - ha_backend_connstring = etcd_endpoints; - } + ha_backend_connstring = ResolveConfiguredHABackendConnstring( + ha_backend_type, config.ha_backend_connstring, etcd_endpoints); local_hostname = rpc_address + ":" + std::to_string(rpc_port); cluster_id = config.cluster_id; root_fs_dir = config.root_fs_dir; @@ -335,10 +346,9 @@ class WrappedMasterServiceConfig { offload_on_evict = config.offload_on_evict; offload_force_evict = config.offload_force_evict; ha_backend_type = config.ha_backend_type; - ha_backend_connstring = config.ha_backend_connstring; - if (ha_backend_connstring.empty()) { - ha_backend_connstring = config.etcd_endpoints; - } + ha_backend_connstring = ResolveConfiguredHABackendConnstring( + ha_backend_type, config.ha_backend_connstring, + config.etcd_endpoints); cluster_id = config.cluster_id; root_fs_dir = config.root_fs_dir; global_file_segment_size = config.global_file_segment_size; @@ -415,10 +425,9 @@ class WrappedMasterServiceConfig { offload_on_evict = config.offload_on_evict; offload_force_evict = config.offload_force_evict; ha_backend_type = config.ha_backend_type; - ha_backend_connstring = config.ha_backend_connstring; - if (ha_backend_connstring.empty()) { - ha_backend_connstring = config.etcd_endpoints; - } + ha_backend_connstring = ResolveConfiguredHABackendConnstring( + ha_backend_type, config.ha_backend_connstring, + config.etcd_endpoints); cluster_id = config.cluster_id; root_fs_dir = config.root_fs_dir; global_file_segment_size = config.global_file_segment_size; diff --git a/mooncake-store/include/types.h b/mooncake-store/include/types.h index e0e4fab1..153900ae 100644 --- a/mooncake-store/include/types.h +++ b/mooncake-store/include/types.h @@ -311,6 +311,8 @@ enum class ErrorCode : int32_t { ETCD_CTX_CANCELLED = -1003, ///< etcd context cancelled. OPLOG_ENTRY_NOT_FOUND = -1004, ///< OpLog entry not found (backend-agnostic). + K8S_LEASE_OPERATION_ERROR = -1005, ///< K8s Lease operation failed. + K8S_LEASE_NOT_FOUND = -1006, ///< K8s Lease not found. UNAVAILABLE_IN_CURRENT_STATUS = -1010, ///< Request cannot be done in current status. UNAVAILABLE_IN_CURRENT_MODE = diff --git a/mooncake-store/src/CMakeLists.txt b/mooncake-store/src/CMakeLists.txt index 5d4c9c3c..e529bacc 100644 --- a/mooncake-store/src/CMakeLists.txt +++ b/mooncake-store/src/CMakeLists.txt @@ -140,6 +140,13 @@ if(STORE_USE_REDIS) list(APPEND EXTRA_LIBS ${MOONCAKE_STORE_HIREDIS_LIBRARY}) endif() +if(STORE_USE_K8S_LEASE) + list(APPEND MOONCAKE_STORE_SOURCES + k8s_lease_helper.cpp + ha/leadership/backends/k8s/k8s_leader_coordinator.cpp) + list(APPEND EXTRA_LIBS ${K8S_LEASE_WRAPPER_LIB}) +endif() + # The cache_allocator library include_directories(${Python3_INCLUDE_DIRS}) add_library(mooncake_store ${MOONCAKE_STORE_SOURCES}) @@ -160,6 +167,9 @@ target_link_libraries( if(STORE_USE_ETCD) add_dependencies(mooncake_store build_etcd_wrapper) endif() +if(STORE_USE_K8S_LEASE) + add_dependencies(mooncake_store build_k8s_lease_wrapper) +endif() if(URING_LIB AND URING_INCLUDE) target_compile_definitions(mooncake_store PUBLIC USE_URING) diff --git a/mooncake-store/src/ha/leadership/backends/k8s/k8s_leader_coordinator.cpp b/mooncake-store/src/ha/leadership/backends/k8s/k8s_leader_coordinator.cpp new file mode 100644 index 00000000..ea24b1a4 --- /dev/null +++ b/mooncake-store/src/ha/leadership/backends/k8s/k8s_leader_coordinator.cpp @@ -0,0 +1,456 @@ +#include "ha/leadership/backends/k8s/k8s_leader_coordinator.h" + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +namespace mooncake { +namespace ha { +namespace backends { +namespace k8s { + +namespace { + +constexpr int kDefaultLeaseDurationSec = 5; +constexpr int kDefaultRenewDeadlineSec = 3; +constexpr int kDefaultRetryPeriodSec = 1; +constexpr auto kViewChangePollInterval = std::chrono::milliseconds(200); + +std::atomic g_session_sequence{0}; + +OwnerToken MakeOwnerToken(const std::string& namespace_name, + const std::string& lease_name, + const std::string& leader_address, + int64_t lease_transitions) { + const auto sequence = + g_session_sequence.fetch_add(1, std::memory_order_relaxed) + 1; + return namespace_name + "/" + lease_name + "/" + + std::to_string(lease_transitions) + "/" + std::to_string(sequence) + + "/" + leader_address; +} + +class K8sLeadershipMonitorHandle final : public LeadershipMonitorHandle { + public: + explicit K8sLeadershipMonitorHandle( + std::shared_ptr> armed) + : armed_(std::move(armed)) {} + + void Stop() override { + if (armed_ != nullptr) { + armed_->store(false); + } + } + + private: + std::shared_ptr> armed_; +}; + +} // namespace + +K8sLeaderCoordinator::K8sLeaderCoordinator(const HABackendSpec& spec) + : spec_(spec) {} + +K8sLeaderCoordinator::~K8sLeaderCoordinator() { ShutdownElection(); } + +ErrorCode K8sLeaderCoordinator::Connect() { + if (connected_) { + return ErrorCode::OK; + } + + auto parsed = ParseConnstring(spec_.connstring); + if (!parsed) { + return parsed.error(); + } + namespace_ = std::move(parsed->first); + lease_name_ = std::move(parsed->second); + + auto err = K8sLeaseHelper::Init(); + if (err == ErrorCode::OK) { + connected_ = true; + } + return err; +} + +tl::expected, ErrorCode> +K8sLeaderCoordinator::ReadCurrentView() { + auto err = EnsureConnected(); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + + std::string holder; + int64_t transitions = 0; + err = + K8sLeaseHelper::GetHolder(namespace_, lease_name_, holder, transitions); + if (err == ErrorCode::K8S_LEASE_NOT_FOUND) { + return std::optional{std::nullopt}; + } + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + + if (holder.empty()) { + return std::optional{std::nullopt}; + } + + return std::optional{ + MasterView{.leader_address = std::move(holder), + .view_version = static_cast(transitions)}}; +} + +tl::expected +K8sLeaderCoordinator::TryAcquireLeadership(const std::string& leader_address) { + auto err = EnsureConnected(); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + + // Start election goroutine + err = K8sLeaseHelper::RunElection( + namespace_, lease_name_, leader_address, kDefaultLeaseDurationSec, + kDefaultRenewDeadlineSec, kDefaultRetryPeriodSec); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + + constexpr int kElectionTimeoutSec = 2 * kDefaultLeaseDurationSec; + int64_t transitions = 0; + err = K8sLeaseHelper::WaitElected(namespace_, lease_name_, + kElectionTimeoutSec, transitions); + if (err != ErrorCode::OK) { + // Election failed — we are not the leader + K8sLeaseHelper::CancelElection(namespace_, lease_name_); + + auto observed_view = ReadCurrentView(); + if (!observed_view) { + return tl::make_unexpected(observed_view.error()); + } + return AcquireLeadershipResult{ + .status = AcquireLeadershipStatus::CONTENDED, + .session = std::nullopt, + .observed_view = observed_view.value(), + }; + } + + // We are the leader + OwnerToken token = + MakeOwnerToken(namespace_, lease_name_, leader_address, transitions); + LeadershipSession session{ + .view = + MasterView{.leader_address = leader_address, + .view_version = static_cast(transitions)}, + .owner_token = token, + .lease_ttl = std::chrono::seconds(kDefaultLeaseDurationSec), + }; + + std::thread thread_to_join; + { + std::lock_guard lock(election_mutex_); + if (election_monitor_thread_.joinable() && election_monitor_stopped_) { + thread_to_join = std::move(election_monitor_thread_); + } + election_identity_ = leader_address; + election_owner_token_ = session.owner_token; + election_active_ = true; + election_monitor_stopped_ = false; + election_shutdown_requested_ = false; + ClearLeadershipMonitorStateLocked(); + } + + if (thread_to_join.joinable()) { + thread_to_join.join(); + } + + return AcquireLeadershipResult{ + .status = AcquireLeadershipStatus::ACQUIRED, + .session = std::move(session), + .observed_view = std::nullopt, + }; +} + +tl::expected K8sLeaderCoordinator::RenewLeadership( + const LeadershipSession& session) { + auto err = EnsureConnected(); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + if (session.owner_token.empty()) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + std::thread thread_to_join; + { + std::lock_guard lock(election_mutex_); + + if (election_shutdown_requested_) { + return false; + } + if (!election_active_) { + return false; + } + if (election_owner_token_ != session.owner_token) { + return false; + } + + // client-go handles renewal internally. If election is still active, + // leadership is being renewed. Start the monitor thread if not running. + if (election_monitor_thread_.joinable()) { + if (!election_monitor_stopped_) { + return true; + } + thread_to_join = std::move(election_monitor_thread_); + election_monitor_stopped_ = false; + } + + if (!election_monitor_thread_.joinable()) { + election_monitor_thread_ = + std::thread([this, token = session.owner_token]() { + auto rc = K8sLeaseHelper::WaitLost(namespace_, lease_name_); + + std::shared_ptr> monitor_armed; + LeadershipLostCallback on_leadership_lost; + LeadershipLossReason loss_reason = + (rc == ErrorCode::OK) + ? LeadershipLossReason::kLostLeadership + : LeadershipLossReason::kRenewError; + { + std::lock_guard lock(election_mutex_); + election_monitor_stopped_ = true; + if (election_owner_token_ == token) { + election_active_ = false; + } + if (!election_shutdown_requested_ && + leadership_monitor_owner_token_ == token) { + monitor_armed = leadership_monitor_armed_; + on_leadership_lost = + std::move(leadership_monitor_callback_); + leadership_monitor_armed_.reset(); + leadership_monitor_owner_token_.clear(); + } + } + + if (monitor_armed != nullptr && + monitor_armed->exchange(false) && + on_leadership_lost != nullptr) { + on_leadership_lost(loss_reason); + } + }); + } + } + + if (thread_to_join.joinable()) { + thread_to_join.join(); + } + + return true; +} + +tl::expected +K8sLeaderCoordinator::WaitForViewChange( + std::optional known_version, + std::chrono::milliseconds timeout) { + auto err = EnsureConnected(); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + + const auto deadline = std::chrono::steady_clock::now() + timeout; + while (true) { + auto current_view = ReadCurrentView(); + if (!current_view) { + return tl::make_unexpected(current_view.error()); + } + + if (!IsSameViewVersion(current_view.value(), known_version)) { + return ViewChangeResult{ + .changed = true, + .timed_out = false, + .current_view = current_view.value(), + }; + } + + if (timeout <= std::chrono::milliseconds::zero() || + std::chrono::steady_clock::now() >= deadline) { + return ViewChangeResult{ + .changed = false, + .timed_out = true, + .current_view = std::nullopt, + }; + } + + const auto remaining = + std::chrono::duration_cast( + deadline - std::chrono::steady_clock::now()); + std::this_thread::sleep_for( + std::min(kViewChangePollInterval, remaining)); + } +} + +tl::expected, ErrorCode> +K8sLeaderCoordinator::StartLeadershipMonitor( + const LeadershipSession& session, + LeadershipLostCallback on_leadership_lost) { + auto err = EnsureConnected(); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + if (!on_leadership_lost) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + std::lock_guard lock(election_mutex_); + if (election_shutdown_requested_) { + return tl::make_unexpected(ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS); + } + if (session.owner_token.empty()) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + if (!election_active_) { + return tl::make_unexpected(ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS); + } + if (election_owner_token_ != session.owner_token) { + return tl::make_unexpected(ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS); + } + if (leadership_monitor_armed_ != nullptr && + leadership_monitor_armed_->load()) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + leadership_monitor_owner_token_ = session.owner_token; + leadership_monitor_callback_ = std::move(on_leadership_lost); + leadership_monitor_armed_ = std::make_shared>(true); + return std::unique_ptr( + std::make_unique( + leadership_monitor_armed_)); +} + +ErrorCode K8sLeaderCoordinator::ReleaseLeadership( + const LeadershipSession& session) { + if (session.owner_token.empty()) { + return ErrorCode::INVALID_PARAMS; + } + + std::thread thread_to_join; + { + std::lock_guard lock(election_mutex_); + if (!election_owner_token_.empty() && + election_owner_token_ != session.owner_token) { + return ErrorCode::INVALID_PARAMS; + } + election_shutdown_requested_ = true; + election_active_ = false; + election_monitor_stopped_ = true; + election_owner_token_.clear(); + ClearLeadershipMonitorStateLocked(); + if (election_monitor_thread_.joinable()) { + thread_to_join = std::move(election_monitor_thread_); + } + } + + // Cancel the election goroutine (triggers WaitLost to return) + auto err = K8sLeaseHelper::CancelElection(namespace_, lease_name_); + + if (thread_to_join.joinable()) { + thread_to_join.join(); + } + + return err; +} + +ErrorCode K8sLeaderCoordinator::EnsureConnected() { + if (connected_) { + return ErrorCode::OK; + } + return Connect(); +} + +ErrorCode K8sLeaderCoordinator::ShutdownElection() { + std::thread thread_to_join; + bool should_cancel = false; + { + std::lock_guard lock(election_mutex_); + election_shutdown_requested_ = true; + should_cancel = election_active_; + election_active_ = false; + election_monitor_stopped_ = true; + election_owner_token_.clear(); + ClearLeadershipMonitorStateLocked(); + if (election_monitor_thread_.joinable()) { + thread_to_join = std::move(election_monitor_thread_); + } + } + + if (should_cancel) { + K8sLeaseHelper::CancelElection(namespace_, lease_name_); + } + + if (thread_to_join.joinable()) { + thread_to_join.join(); + } + + return ErrorCode::OK; +} + +void K8sLeaderCoordinator::ClearLeadershipMonitorStateLocked() { + if (leadership_monitor_armed_ != nullptr) { + leadership_monitor_armed_->store(false); + leadership_monitor_armed_.reset(); + } + leadership_monitor_callback_ = nullptr; + leadership_monitor_owner_token_.clear(); +} + +bool K8sLeaderCoordinator::IsSameViewVersion( + const std::optional& current_view, + std::optional known_version) const { + if (!current_view.has_value() && !known_version.has_value()) { + return true; + } + if (!current_view.has_value() || !known_version.has_value()) { + return false; + } + return current_view->view_version == known_version.value(); +} + +tl::expected, ErrorCode> +K8sLeaderCoordinator::ParseConnstring(const std::string& connstring) { + // Format: "namespace/lease-name" or just "lease-name" (default namespace). + // Exactly zero or one '/' is allowed. + if (connstring.empty()) { + LOG(ERROR) << "K8s HA connstring is empty"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + auto first_slash = connstring.find('/'); + if (first_slash == std::string::npos) { + // No slash — use default namespace + return std::pair{"default", connstring}; + } + + // Reject multiple slashes (e.g. "a/b/c") + if (connstring.find('/', first_slash + 1) != std::string::npos) { + LOG(ERROR) << "K8s HA connstring contains multiple '/': " << connstring; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + auto ns = connstring.substr(0, first_slash); + auto name = connstring.substr(first_slash + 1); + if (ns.empty() || name.empty()) { + LOG(ERROR) << "K8s HA connstring has empty namespace or lease name: " + << connstring; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + return std::pair{std::move(ns), std::move(name)}; +} + +} // namespace k8s +} // namespace backends +} // namespace ha +} // namespace mooncake diff --git a/mooncake-store/src/ha/leadership/leader_coordinator_factory.cpp b/mooncake-store/src/ha/leadership/leader_coordinator_factory.cpp index e6bc49af..c8458691 100644 --- a/mooncake-store/src/ha/leadership/leader_coordinator_factory.cpp +++ b/mooncake-store/src/ha/leadership/leader_coordinator_factory.cpp @@ -2,6 +2,9 @@ #include "ha/leadership/backends/etcd/etcd_leader_coordinator.h" #include "ha/leadership/backends/redis/redis_leader_coordinator.h" +#ifdef STORE_USE_K8S_LEASE +#include "ha/leadership/backends/k8s/k8s_leader_coordinator.h" +#endif namespace mooncake { namespace ha { @@ -29,8 +32,19 @@ CreateLeaderCoordinator(const HABackendSpec& spec) { } return std::unique_ptr(std::move(coordinator)); } - case HABackendType::K8S: + case HABackendType::K8S: { +#ifdef STORE_USE_K8S_LEASE + auto coordinator = + std::make_unique(spec); + auto err = coordinator->Connect(); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + return std::unique_ptr(std::move(coordinator)); +#else return tl::make_unexpected(ErrorCode::UNAVAILABLE_IN_CURRENT_MODE); +#endif + } } return tl::make_unexpected(ErrorCode::INVALID_PARAMS); diff --git a/mooncake-store/src/ha/leadership/master_service_supervisor.cpp b/mooncake-store/src/ha/leadership/master_service_supervisor.cpp index d38a6c31..ae7a48b6 100644 --- a/mooncake-store/src/ha/leadership/master_service_supervisor.cpp +++ b/mooncake-store/src/ha/leadership/master_service_supervisor.cpp @@ -26,10 +26,9 @@ constexpr auto kSupervisorRetryInterval = std::chrono::seconds(1); std::string ResolveHABackendConnstring( const MasterServiceSupervisorConfig& config) { - if (!config.ha_backend_connstring.empty()) { - return config.ha_backend_connstring; - } - return config.etcd_endpoints; + return ResolveConfiguredHABackendConnstring(config.ha_backend_type, + config.ha_backend_connstring, + config.etcd_endpoints); } tl::expected BuildHABackendSpec( diff --git a/mooncake-store/src/k8s_lease_helper.cpp b/mooncake-store/src/k8s_lease_helper.cpp new file mode 100644 index 00000000..15323bbc --- /dev/null +++ b/mooncake-store/src/k8s_lease_helper.cpp @@ -0,0 +1,203 @@ +#include "k8s_lease_helper.h" + +#ifdef STORE_USE_K8S_LEASE +#include "libk8s_lease_wrapper.h" +#endif + +#include + +namespace mooncake { + +std::mutex K8sLeaseHelper::init_mutex_; +bool K8sLeaseHelper::initialized_ = false; + +#ifdef STORE_USE_K8S_LEASE + +ErrorCode K8sLeaseHelper::Init() { + std::lock_guard lock(init_mutex_); + if (initialized_) { + return ErrorCode::OK; + } + char* err_msg = nullptr; + int ret = K8sLeaseInit(&err_msg); + if (ret != 0) { + LOG(ERROR) << "Failed to initialize K8s client: " << err_msg; + free(err_msg); + return ErrorCode::K8S_LEASE_OPERATION_ERROR; + } + initialized_ = true; + return ErrorCode::OK; +} + +ErrorCode K8sLeaseHelper::RunElection(const std::string& ns, + const std::string& lease, + const std::string& identity, + int lease_dur, int renew_deadline, + int retry_period) { + char* err_msg = nullptr; + int ret = K8sLeaseRunElection( + const_cast(ns.c_str()), const_cast(lease.c_str()), + const_cast(identity.c_str()), lease_dur, renew_deadline, + retry_period, &err_msg); + if (ret != 0) { + LOG(ERROR) << "RunElection failed: " << err_msg; + free(err_msg); + return ErrorCode::K8S_LEASE_OPERATION_ERROR; + } + return ErrorCode::OK; +} + +ErrorCode K8sLeaseHelper::WaitElected(const std::string& ns, + const std::string& lease, int timeout_sec, + int64_t& lease_transitions) { + char* err_msg = nullptr; + long long transitions = 0; + int ret = K8sLeaseWaitElected(const_cast(ns.c_str()), + const_cast(lease.c_str()), timeout_sec, + &transitions, &err_msg); + if (ret != 0) { + LOG(ERROR) << "WaitElected failed: " << err_msg; + free(err_msg); + return ErrorCode::K8S_LEASE_OPERATION_ERROR; + } + lease_transitions = static_cast(transitions); + return ErrorCode::OK; +} + +ErrorCode K8sLeaseHelper::WaitLost(const std::string& ns, + const std::string& lease) { + char* err_msg = nullptr; + int ret = K8sLeaseWaitLost(const_cast(ns.c_str()), + const_cast(lease.c_str()), &err_msg); + if (ret != 0) { + LOG(ERROR) << "WaitLost failed: " << err_msg; + free(err_msg); + return ErrorCode::K8S_LEASE_OPERATION_ERROR; + } + return ErrorCode::OK; +} + +ErrorCode K8sLeaseHelper::CancelElection(const std::string& ns, + const std::string& lease) { + char* err_msg = nullptr; + int ret = + K8sLeaseCancelElection(const_cast(ns.c_str()), + const_cast(lease.c_str()), &err_msg); + if (ret != 0) { + LOG(ERROR) << "CancelElection failed: " << err_msg; + free(err_msg); + return ErrorCode::K8S_LEASE_OPERATION_ERROR; + } + return ErrorCode::OK; +} + +ErrorCode K8sLeaseHelper::GetHolder(const std::string& ns, + const std::string& lease, + std::string& holder, + int64_t& lease_transitions) { + char* err_msg = nullptr; + char* holder_ptr = nullptr; + long long transitions = 0; + int ret = K8sLeaseGetHolder(const_cast(ns.c_str()), + const_cast(lease.c_str()), &holder_ptr, + &transitions, &err_msg); + if (ret == 1) { + holder.clear(); + lease_transitions = 0; + return ErrorCode::K8S_LEASE_NOT_FOUND; + } + if (ret != 0) { + LOG(ERROR) << "GetHolder failed: " << err_msg; + free(err_msg); + return ErrorCode::K8S_LEASE_OPERATION_ERROR; + } + if (holder_ptr != nullptr) { + holder = std::string(holder_ptr); + free(holder_ptr); + } else { + holder.clear(); + } + lease_transitions = static_cast(transitions); + return ErrorCode::OK; +} + +ErrorCode K8sLeaseHelper::WatchHolder( + const std::string& ns, const std::string& lease, void* callback_context, + void (*callback_func)(void*, const char*, size_t, int64_t)) { + char* err_msg = nullptr; + int ret = K8sLeaseWatchHolder(const_cast(ns.c_str()), + const_cast(lease.c_str()), + callback_context, callback_func, &err_msg); + if (ret != 0) { + LOG(ERROR) << "WatchHolder failed: " << err_msg; + free(err_msg); + return ErrorCode::K8S_LEASE_OPERATION_ERROR; + } + return ErrorCode::OK; +} + +ErrorCode K8sLeaseHelper::CancelWatch(const std::string& ns, + const std::string& lease) { + char* err_msg = nullptr; + int ret = K8sLeaseCancelWatch(const_cast(ns.c_str()), + const_cast(lease.c_str()), &err_msg); + if (ret != 0) { + LOG(ERROR) << "CancelWatch failed: " << err_msg; + free(err_msg); + return ErrorCode::K8S_LEASE_OPERATION_ERROR; + } + return ErrorCode::OK; +} + +#else // !STORE_USE_K8S_LEASE + +ErrorCode K8sLeaseHelper::Init() { + LOG(FATAL) << "K8s Lease is not enabled in compilation"; + return ErrorCode::K8S_LEASE_OPERATION_ERROR; +} + +ErrorCode K8sLeaseHelper::RunElection(const std::string&, const std::string&, + const std::string&, int, int, int) { + LOG(FATAL) << "K8s Lease is not enabled in compilation"; + return ErrorCode::K8S_LEASE_OPERATION_ERROR; +} + +ErrorCode K8sLeaseHelper::WaitElected(const std::string&, const std::string&, + int, int64_t&) { + LOG(FATAL) << "K8s Lease is not enabled in compilation"; + return ErrorCode::K8S_LEASE_OPERATION_ERROR; +} + +ErrorCode K8sLeaseHelper::WaitLost(const std::string&, const std::string&) { + LOG(FATAL) << "K8s Lease is not enabled in compilation"; + return ErrorCode::K8S_LEASE_OPERATION_ERROR; +} + +ErrorCode K8sLeaseHelper::CancelElection(const std::string&, + const std::string&) { + LOG(FATAL) << "K8s Lease is not enabled in compilation"; + return ErrorCode::K8S_LEASE_OPERATION_ERROR; +} + +ErrorCode K8sLeaseHelper::GetHolder(const std::string&, const std::string&, + std::string&, int64_t&) { + LOG(FATAL) << "K8s Lease is not enabled in compilation"; + return ErrorCode::K8S_LEASE_OPERATION_ERROR; +} + +ErrorCode K8sLeaseHelper::WatchHolder(const std::string&, const std::string&, + void*, + void (*)(void*, const char*, size_t, + int64_t)) { + LOG(FATAL) << "K8s Lease is not enabled in compilation"; + return ErrorCode::K8S_LEASE_OPERATION_ERROR; +} + +ErrorCode K8sLeaseHelper::CancelWatch(const std::string&, const std::string&) { + LOG(FATAL) << "K8s Lease is not enabled in compilation"; + return ErrorCode::K8S_LEASE_OPERATION_ERROR; +} + +#endif // STORE_USE_K8S_LEASE + +} // namespace mooncake diff --git a/mooncake-store/src/master.cpp b/mooncake-store/src/master.cpp index c21878f3..ec4dab87 100644 --- a/mooncake-store/src/master.cpp +++ b/mooncake-store/src/master.cpp @@ -107,7 +107,7 @@ DEFINE_validator(eviction_ratio, [](const char* flagname, double value) { return true; }); DEFINE_bool(enable_ha, false, - "Enable high availability, which depends on etcd"); + "Enable high availability using the configured HA backend"); DEFINE_bool(enable_offload, false, "Enable offload availability"); DEFINE_bool(offload_on_evict, false, "Defer LOCAL_DISK offload to eviction time instead of PutEnd"); @@ -116,8 +116,8 @@ DEFINE_bool(offload_force_evict, false, DEFINE_string(ha_backend_type, "etcd", "HA backend type, e.g. etcd | redis | k8s"); DEFINE_string(ha_backend_connstring, "", - "HA backend connection string. If unset, fallback to " - "etcd_endpoints for backward compatibility"); + "HA backend connection string. If unset, only backend_type=etcd " + "falls back to etcd_endpoints for backward compatibility"); DEFINE_string( etcd_endpoints, "", "Endpoints of ETCD server, separated by semicolon, required in HA mode"); @@ -215,10 +215,9 @@ namespace { std::string ResolveHABackendConnstring( const mooncake::MasterConfig& master_config) { - if (!master_config.ha_backend_connstring.empty()) { - return master_config.ha_backend_connstring; - } - return master_config.etcd_endpoints; + return mooncake::ResolveConfiguredHABackendConnstring( + master_config.ha_backend_type, master_config.ha_backend_connstring, + master_config.etcd_endpoints); } void ResolveRpcAddressFromInterfaceOrDie( @@ -833,7 +832,10 @@ int main(int argc, char* argv[]) { ResolveHABackendConnstring(master_config); if (master_config.enable_ha && ha_backend_connstring.empty()) { LOG(FATAL) << "HA backend connection string must be set when " - << "enable_ha is true"; + << "enable_ha is true, backend_type=" + << master_config.ha_backend_type + << ". Only backend_type=etcd may fall back to " + << "etcd_endpoints"; return 1; } if (!master_config.enable_ha && (!ha_backend_connstring.empty() || diff --git a/mooncake-store/src/types.cpp b/mooncake-store/src/types.cpp index 2f5c4f45..9209af96 100644 --- a/mooncake-store/src/types.cpp +++ b/mooncake-store/src/types.cpp @@ -41,6 +41,9 @@ const std::string& toString(ErrorCode errorCode) noexcept { {ErrorCode::ETCD_KEY_NOT_EXIST, "ETCD_KEY_NOT_EXIST"}, {ErrorCode::ETCD_TRANSACTION_FAIL, "ETCD_TRANSACTION_FAIL"}, {ErrorCode::ETCD_CTX_CANCELLED, "ETCD_CTX_CANCELLED"}, + {ErrorCode::OPLOG_ENTRY_NOT_FOUND, "OPLOG_ENTRY_NOT_FOUND"}, + {ErrorCode::K8S_LEASE_OPERATION_ERROR, "K8S_LEASE_OPERATION_ERROR"}, + {ErrorCode::K8S_LEASE_NOT_FOUND, "K8S_LEASE_NOT_FOUND"}, {ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS, "UNAVAILABLE_IN_CURRENT_STATUS"}, {ErrorCode::UNAVAILABLE_IN_CURRENT_MODE, "UNAVAILABLE_IN_CURRENT_MODE"}, diff --git a/mooncake-store/tests/CMakeLists.txt b/mooncake-store/tests/CMakeLists.txt index 252b99e8..d80cdddf 100644 --- a/mooncake-store/tests/CMakeLists.txt +++ b/mooncake-store/tests/CMakeLists.txt @@ -121,6 +121,11 @@ if(STORE_USE_REDIS) add_test(NAME redis_snapshot_catalog_store_test COMMAND redis_snapshot_catalog_store_test) endif() +if(STORE_USE_K8S_LEASE) + target_sources( + high_availability_test + PRIVATE ha/leadership/backends/k8s/high_availability_k8s_test.cpp) +endif() target_link_libraries( high_availability_test PUBLIC mooncake_store @@ -131,7 +136,11 @@ target_link_libraries( gtest gtest_main pthread) -if(STORE_USE_ETCD OR STORE_USE_REDIS) +if(STORE_USE_K8S_LEASE) + target_link_libraries(high_availability_test + PRIVATE ${K8S_LEASE_WRAPPER_LIB}) +endif() +if(STORE_USE_ETCD OR STORE_USE_REDIS OR STORE_USE_K8S_LEASE) add_test(NAME high_availability_test COMMAND high_availability_test) endif() diff --git a/mooncake-store/tests/ha/leadership/backends/k8s/high_availability_k8s_test.cpp b/mooncake-store/tests/ha/leadership/backends/k8s/high_availability_k8s_test.cpp new file mode 100644 index 00000000..7978258a --- /dev/null +++ b/mooncake-store/tests/ha/leadership/backends/k8s/high_availability_k8s_test.cpp @@ -0,0 +1,347 @@ +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef STORE_USE_K8S_LEASE +#include "k8s_lease_helper.h" +#endif +#include "ha/leadership/leader_coordinator_factory.h" +#include "ha/leadership/high_availability_test_fixture.h" +#include "types.h" + +namespace mooncake { +namespace testing { + +DEFINE_string(k8s_namespace, "default", + "K8s namespace for HA integration tests"); +DEFINE_string(k8s_lease_name, "mooncake-ha-test", + "K8s Lease name for HA integration tests"); + +namespace { + +std::optional GetK8sSkipReason() { +#ifdef STORE_USE_K8S_LEASE + // Probe: try to init K8s client and read a lease. + auto err = K8sLeaseHelper::Init(); + if (err != ErrorCode::OK) { + return "K8s API not reachable (Init failed)"; + } + std::string holder; + int64_t transitions = 0; + err = K8sLeaseHelper::GetHolder(FLAGS_k8s_namespace, FLAGS_k8s_lease_name, + holder, transitions); + if (err != ErrorCode::OK && err != ErrorCode::K8S_LEASE_NOT_FOUND) { + return "K8s API not reachable (GetHolder probe failed)"; + } + return std::nullopt; +#else + return "K8s Lease HA backend is not enabled in this build"; +#endif +} + +ha::HABackendSpec MakeK8sBackendSpec(const std::string& ns, + const std::string& lease_name) { + return ha::HABackendSpec{ + .type = ha::HABackendType::K8S, + .connstring = ns + "/" + lease_name, + .cluster_namespace = "", + }; +} + +std::unique_ptr CreateK8sCoordinatorOrNull( + const std::string& ns, const std::string& lease_name) { + auto coordinator = + ha::CreateLeaderCoordinator(MakeK8sBackendSpec(ns, lease_name)); + if (!coordinator) { + return nullptr; + } + return std::move(coordinator.value()); +} + +std::string MakeK8sTestLeaseName(const std::string& suffix) { + return FLAGS_k8s_lease_name + "-" + suffix; +} + +} // namespace + +TEST_F(HighAvailabilityTest, K8sBasicMasterViewOperations) { + if (auto skip_reason = GetK8sSkipReason(); skip_reason.has_value()) { + GTEST_SKIP() << *skip_reason; + } + + const auto lease_name = MakeK8sTestLeaseName("basic"); + auto coordinator = + CreateK8sCoordinatorOrNull(FLAGS_k8s_namespace, lease_name); + ASSERT_NE(coordinator, nullptr); + + // Initially, the master view should be empty (no leader) + auto initial_view = coordinator->ReadCurrentView(); + ASSERT_TRUE(initial_view.has_value()); + // Note: may or may not have value depending on prior test state + + // Acquire leadership + auto acquire = coordinator->TryAcquireLeadership("127.0.0.1:8899"); + ASSERT_TRUE(acquire.has_value()); + ASSERT_EQ(ha::AcquireLeadershipStatus::ACQUIRED, acquire->status); + ASSERT_TRUE(acquire->session.has_value()); + + // Read current view — should show our address + auto current_view = coordinator->ReadCurrentView(); + ASSERT_TRUE(current_view.has_value()); + ASSERT_TRUE(current_view->has_value()); + EXPECT_EQ("127.0.0.1:8899", current_view->value().leader_address); + + // Renew + auto renewed = coordinator->RenewLeadership(*acquire->session); + ASSERT_TRUE(renewed.has_value()); + EXPECT_TRUE(renewed.value()); + + // WaitForViewChange — should time out since nothing changed + auto no_change = coordinator->WaitForViewChange( + acquire->session->view.view_version, std::chrono::milliseconds(200)); + ASSERT_TRUE(no_change.has_value()); + ASSERT_FALSE(no_change->changed); + ASSERT_TRUE(no_change->timed_out); + + // Release + ASSERT_EQ(ErrorCode::OK, coordinator->ReleaseLeadership(*acquire->session)); + + // After release, view should eventually show no leader or a different + // transitions count + auto released = coordinator->WaitForViewChange( + acquire->session->view.view_version, std::chrono::seconds(10)); + ASSERT_TRUE(released.has_value()); + ASSERT_TRUE(released->changed); +} + +TEST_F(HighAvailabilityTest, K8sLeadershipMonitorIgnoresExplicitRelease) { + if (auto skip_reason = GetK8sSkipReason(); skip_reason.has_value()) { + GTEST_SKIP() << *skip_reason; + } + + const auto lease_name = MakeK8sTestLeaseName("monitor-release"); + auto coordinator = + CreateK8sCoordinatorOrNull(FLAGS_k8s_namespace, lease_name); + ASSERT_NE(coordinator, nullptr); + + auto acquire = coordinator->TryAcquireLeadership("127.0.0.1:9922"); + ASSERT_TRUE(acquire.has_value()); + ASSERT_EQ(ha::AcquireLeadershipStatus::ACQUIRED, acquire->status); + ASSERT_TRUE(acquire->session.has_value()); + const auto session = *acquire->session; + + auto renew = coordinator->RenewLeadership(session); + ASSERT_TRUE(renew.has_value()); + ASSERT_TRUE(renew.value()); + + auto callback_fired = std::make_shared>(false); + auto monitor = coordinator->StartLeadershipMonitor( + session, [callback_fired](ha::LeadershipLossReason) { + callback_fired->store(true); + }); + ASSERT_TRUE(monitor.has_value()); + + // Explicit release should NOT fire the monitor callback + ASSERT_EQ(ErrorCode::OK, coordinator->ReleaseLeadership(session)); + std::this_thread::sleep_for(std::chrono::seconds(2)); + EXPECT_FALSE(callback_fired->load()); +} + +TEST_F(HighAvailabilityTest, K8sLeadershipMonitorReportsElectionCancellation) { + if (auto skip_reason = GetK8sSkipReason(); skip_reason.has_value()) { + GTEST_SKIP() << *skip_reason; + } + + const auto lease_name = MakeK8sTestLeaseName("monitor-loss"); + auto coordinator = + CreateK8sCoordinatorOrNull(FLAGS_k8s_namespace, lease_name); + ASSERT_NE(coordinator, nullptr); + + auto acquire = coordinator->TryAcquireLeadership("127.0.0.1:9955"); + ASSERT_TRUE(acquire.has_value()); + ASSERT_EQ(ha::AcquireLeadershipStatus::ACQUIRED, acquire->status); + ASSERT_TRUE(acquire->session.has_value()); + const auto session = *acquire->session; + + auto renew = coordinator->RenewLeadership(session); + ASSERT_TRUE(renew.has_value()); + ASSERT_TRUE(renew.value()); + + auto callback_fired = std::make_shared>(false); + auto monitor = coordinator->StartLeadershipMonitor( + session, [callback_fired](ha::LeadershipLossReason) { + callback_fired->store(true); + }); + ASSERT_TRUE(monitor.has_value()); + + // Cancel the election goroutine directly, bypassing ReleaseLeadership. + // This simulates leadership loss: WaitLost returns while + // election_shutdown_requested_ is still false, so the monitor fires. + K8sLeaseHelper::CancelElection(FLAGS_k8s_namespace, lease_name); + + // Wait for the monitor thread to detect the loss and fire the callback + for (int i = 0; i < 50 && !callback_fired->load(); ++i) { + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + } + EXPECT_TRUE(callback_fired->load()); +} + +TEST_F(HighAvailabilityTest, K8sCanReacquireAfterRelease) { + if (auto skip_reason = GetK8sSkipReason(); skip_reason.has_value()) { + GTEST_SKIP() << *skip_reason; + } + + const auto lease_name = MakeK8sTestLeaseName("reacquire"); + auto coordinator = + CreateK8sCoordinatorOrNull(FLAGS_k8s_namespace, lease_name); + ASSERT_NE(coordinator, nullptr); + + // First acquisition + auto first_acquire = coordinator->TryAcquireLeadership("127.0.0.1:9933"); + ASSERT_TRUE(first_acquire.has_value()); + ASSERT_EQ(ha::AcquireLeadershipStatus::ACQUIRED, first_acquire->status); + ASSERT_TRUE(first_acquire->session.has_value()); + + auto first_renew = coordinator->RenewLeadership(*first_acquire->session); + ASSERT_TRUE(first_renew.has_value()); + ASSERT_TRUE(first_renew.value()); + + ASSERT_EQ(ErrorCode::OK, + coordinator->ReleaseLeadership(*first_acquire->session)); + + // Wait for lease to expire + std::this_thread::sleep_for(std::chrono::seconds(2)); + + // Second acquisition + auto second_acquire = coordinator->TryAcquireLeadership("127.0.0.1:9944"); + ASSERT_TRUE(second_acquire.has_value()); + ASSERT_EQ(ha::AcquireLeadershipStatus::ACQUIRED, second_acquire->status); + ASSERT_TRUE(second_acquire->session.has_value()); + + auto second_renew = coordinator->RenewLeadership(*second_acquire->session); + ASSERT_TRUE(second_renew.has_value()); + ASSERT_TRUE(second_renew.value()); + + ASSERT_EQ(ErrorCode::OK, + coordinator->ReleaseLeadership(*second_acquire->session)); +} + +TEST_F(HighAvailabilityTest, K8sContendedLeadershipAndHandover) { + if (auto skip_reason = GetK8sSkipReason(); skip_reason.has_value()) { + GTEST_SKIP() << *skip_reason; + } + + const auto lease_name = MakeK8sTestLeaseName("contention"); + + // Coordinator A acquires leadership + auto coord_a = CreateK8sCoordinatorOrNull(FLAGS_k8s_namespace, lease_name); + ASSERT_NE(coord_a, nullptr); + + auto acquire_a = coord_a->TryAcquireLeadership("127.0.0.1:7701"); + ASSERT_TRUE(acquire_a.has_value()); + ASSERT_EQ(ha::AcquireLeadershipStatus::ACQUIRED, acquire_a->status); + ASSERT_TRUE(acquire_a->session.has_value()); + + auto renew_a = coord_a->RenewLeadership(*acquire_a->session); + ASSERT_TRUE(renew_a.has_value()); + ASSERT_TRUE(renew_a.value()); + + // Coordinator B attempts acquisition on the same lease — should be + // contended + auto coord_b = CreateK8sCoordinatorOrNull(FLAGS_k8s_namespace, lease_name); + ASSERT_NE(coord_b, nullptr); + + auto acquire_b = coord_b->TryAcquireLeadership("127.0.0.1:7702"); + ASSERT_TRUE(acquire_b.has_value()); + EXPECT_EQ(ha::AcquireLeadershipStatus::CONTENDED, acquire_b->status); + ASSERT_TRUE(acquire_b->observed_view.has_value()); + EXPECT_EQ(acquire_a->session->view.view_version, + acquire_b->observed_view->view_version); + + // A releases leadership + ASSERT_EQ(ErrorCode::OK, coord_a->ReleaseLeadership(*acquire_a->session)); + + // Wait for lease to expire so B can take over + auto view_changed = coord_b->WaitForViewChange( + acquire_a->session->view.view_version, std::chrono::seconds(10)); + ASSERT_TRUE(view_changed.has_value()); + ASSERT_TRUE(view_changed->changed); + + // B acquires leadership + auto acquire_b2 = coord_b->TryAcquireLeadership("127.0.0.1:7702"); + ASSERT_TRUE(acquire_b2.has_value()); + ASSERT_EQ(ha::AcquireLeadershipStatus::ACQUIRED, acquire_b2->status); + ASSERT_TRUE(acquire_b2->session.has_value()); + + ASSERT_EQ(ErrorCode::OK, coord_b->ReleaseLeadership(*acquire_b2->session)); +} + +// --- Connstring validation --- + +TEST_F(HighAvailabilityTest, K8sConnstringEmptyIsRejected) { + auto coordinator = CreateK8sCoordinatorOrNull("", ""); + ASSERT_EQ(coordinator, nullptr); +} + +TEST_F(HighAvailabilityTest, K8sConnstringEmptyNamespaceIsRejected) { + auto spec = ha::HABackendSpec{ + .type = ha::HABackendType::K8S, + .connstring = "/lease-name", + }; + auto coordinator = ha::CreateLeaderCoordinator(spec); + ASSERT_FALSE(coordinator.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, coordinator.error()); +} + +TEST_F(HighAvailabilityTest, K8sConnstringEmptyLeaseNameIsRejected) { + auto spec = ha::HABackendSpec{ + .type = ha::HABackendType::K8S, + .connstring = "default/", + }; + auto coordinator = ha::CreateLeaderCoordinator(spec); + ASSERT_FALSE(coordinator.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, coordinator.error()); +} + +TEST_F(HighAvailabilityTest, K8sConnstringMultipleSlashesIsRejected) { + auto spec = ha::HABackendSpec{ + .type = ha::HABackendType::K8S, + .connstring = "a/b/c", + }; + auto coordinator = ha::CreateLeaderCoordinator(spec); + ASSERT_FALSE(coordinator.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, coordinator.error()); +} + +TEST_F(HighAvailabilityTest, K8sConnstringValidFormatPassesParsing) { + if (auto skip_reason = GetK8sSkipReason(); skip_reason.has_value()) { + GTEST_SKIP() << *skip_reason; + } + + auto coordinator = + CreateK8sCoordinatorOrNull(FLAGS_k8s_namespace, "parse-valid"); + ASSERT_NE(coordinator, nullptr); +} + +TEST_F(HighAvailabilityTest, K8sConnstringNoSlashDefaultsNamespace) { + if (auto skip_reason = GetK8sSkipReason(); skip_reason.has_value()) { + GTEST_SKIP() << *skip_reason; + } + + auto spec = ha::HABackendSpec{ + .type = ha::HABackendType::K8S, + .connstring = FLAGS_k8s_lease_name + "-parse-noslash", + }; + auto coordinator = ha::CreateLeaderCoordinator(spec); + ASSERT_TRUE(coordinator.has_value()); +} + +} // namespace testing +} // namespace mooncake -- 2.34.1 From 7e4a85e1a3e7515c9fd9cbf40b0999f66ad921d5 Mon Sep 17 00:00:00 2001 From: Kafka <92581531+ZhenyuePan@users.noreply.github.com> Date: Sat, 16 May 2026 14:07:47 +0800 Subject: [PATCH 079/382] [Store] Fix Rust store build path (#2114) --- README.md | 2 +- docs/source/getting_started/build.md | 3 +- docs/source/zh_archive/build.md | 3 +- docs/source/zh_archive/run-examples.md | 3 +- mooncake-store/rust/README.md | 91 ++++++++++++++++++++++++++ mooncake-store/rust/build.rs | 55 ++++++++++++++++ 6 files changed, 153 insertions(+), 4 deletions(-) create mode 100644 mooncake-store/rust/README.md diff --git a/README.md b/README.md index 6295c8e1..5b392f03 100644 --- a/README.md +++ b/README.md @@ -310,7 +310,7 @@ The following are additional dependencies for building Mooncake: - Go 1.20+, if you want to build with `-DWITH_P2P_STORE`, `-DUSE_ETCD` (enabled by default to use etcd as metadata servers), or `-DSTORE_USE_ETCD` (use etcd for the failover of the store master). - CUDA 12.1 and above, including NVIDIA GPUDirect Storage Support, if the package is built with `-DUSE_CUDA`. *This is NOT included in the `dependencies.sh` script. You may install them from [here](https://developer.nvidia.com/cuda-downloads)*. - Cambricon Neuware, if you want to build with `-DUSE_MLU`. *This is NOT included in the `dependencies.sh` script.* Mooncake resolves it from `NEUWARE_HOME` or `/usr/local/neuware` by default, and also supports overriding `MLU_INCLUDE_DIR` / `MLU_LIB_DIR` during CMake configure. -- [Optional] Rust Toolchain, if you want to build with `-DWITH_RUST_EXAMPLE`. *This is NOT included in the `dependencies.sh` script.* +- [Optional] Rust Toolchain and libclang, if you want to build Transfer Engine Rust examples with `-DWITH_RUST_EXAMPLE=ON` or Mooncake Store Rust bindings with `-DWITH_STORE_RUST=ON`. *This is NOT included in the `dependencies.sh` script.* - [Optional] `hiredis`, if you want to build with `-DUSE_REDIS` to use Redis instead of etcd as metadata servers. - [Optional] `curl`, if you want to build with `-DUSE_HTTP` to use HTTP instead of etcd as metadata servers. diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index adbdb42a..77164b1d 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -237,7 +237,8 @@ The following options can be used during `cmake ..` to specify whether to compil - `-DUSE_CXL=[ON|OFF]`: Enable CXL support - `-DWITH_STORE=[ON|OFF]`: Build Mooncake Store component - `-DWITH_P2P_STORE=[ON|OFF]`: Enable Golang support and build P2P Store component, require go 1.23+ -- `-DWITH_WITH_RUST_EXAMPLE=[ON|OFF]`: Enable Rust support +- `-DWITH_RUST_EXAMPLE=[ON|OFF]`: Build the Transfer Engine Rust interface and sample code. **Default: OFF.** +- `-DWITH_STORE_RUST=[ON|OFF]`: Build Mooncake Store Rust bindings and CMake Rust targets. **Default: ON.** - `-DWITH_EP=[ON|OFF]`: Build the EP (Expert Parallelism) and PG Python extensions for CUDA. Requires CUDA toolkit and PyTorch. Use `-DEP_TORCH_VERSIONS="2.9.1"` (semicolon-separated) to build for specific PyTorch versions, or leave empty to use the currently-installed torch. The CUDA version is detected automatically. **Default: OFF.** - `-DUSE_REDIS=[ON|OFF]`: Enable Redis-based metadata service - `-DUSE_HTTP=[ON|OFF]`: Enable Http-based metadata service diff --git a/docs/source/zh_archive/build.md b/docs/source/zh_archive/build.md index 5f9b94e0..011e0ab8 100644 --- a/docs/source/zh_archive/build.md +++ b/docs/source/zh_archive/build.md @@ -183,7 +183,8 @@ - `-DUSE_CXL=[ON|OFF]`: 启用 CXL 支持 - `-DWITH_STORE=[ON|OFF]`: 编译 Mooncake Store 组件 - `-DWITH_P2P_STORE=[ON|OFF]`: 启用 Golang 支持并编译 P2P Store 组件,需要 go 1.23+ -- `-DWITH_WITH_RUST_EXAMPLE=[ON|OFF]`: 启用 Rust 支持 +- `-DWITH_RUST_EXAMPLE=[ON|OFF]`: 编译 Transfer Engine Rust 接口与示例代码,默认关闭。 +- `-DWITH_STORE_RUST=[ON|OFF]`: 编译 Mooncake Store Rust 绑定和 CMake Rust targets,默认开启。 - `-DUSE_REDIS=[ON|OFF]`: 启用基于 Redis 的元数据服务 - `-DUSE_HTTP=[ON|OFF]`: 启用基于 Http 的元数据服务 - `-DUSE_ETCD=[ON|OFF]`: 启用基于 etcd 的元数据服务,需要 go 1.23+ diff --git a/docs/source/zh_archive/run-examples.md b/docs/source/zh_archive/run-examples.md index bbec1f15..32aee534 100644 --- a/docs/source/zh_archive/run-examples.md +++ b/docs/source/zh_archive/run-examples.md @@ -56,7 +56,8 @@ Mooncake 支持在执行 `cmake` 命令期间添加下列高级编译选项: - `-DUSE_CXL=[ON|OFF]`:编译 Transfer Engine 时启用或关闭 CXL 协议的支持。默认关闭。 - `-DWITH_P2P_STORE=[ON|OFF]`:编译 P2P Store 及示例程序,默认开启。 - `-DWITH_ALLOCATOR=[ON|OFF]`:编译 Managed Store 所用的中心分配器模块,默认开启。 -- `-DWITH_WITH_RUST_EXAMPLE=[ON|OFF]`:编译 Transfer Engine 时启用或关闭 Rust 语言支持,默认关闭。 +- `-DWITH_RUST_EXAMPLE=[ON|OFF]`:编译 Transfer Engine Rust 接口与示例代码,默认关闭。 +- `-DWITH_STORE_RUST=[ON|OFF]`:编译 Mooncake Store Rust 绑定和 CMake Rust targets,默认开启。 ## Transfer Engine Bench 使用方法 diff --git a/mooncake-store/rust/README.md b/mooncake-store/rust/README.md new file mode 100644 index 00000000..65d49125 --- /dev/null +++ b/mooncake-store/rust/README.md @@ -0,0 +1,91 @@ +# Mooncake Store Rust Bindings + +This directory contains the Rust bindings, examples, benchmarks, and smoke tests +for Mooncake Store. The Rust crate links against the C++ Mooncake Store build, so +run a CMake build before using standalone Cargo commands. + +## Prerequisites + +Install the Rust toolchain and libclang before building this crate. On Ubuntu: + +```bash +sudo apt-get install libclang-dev clang +``` + +## Build With CMake + +Configure Mooncake with Store and Rust bindings enabled: + +```bash +cmake -S . -B build -G Ninja \ + -DWITH_STORE=ON \ + -DWITH_STORE_RUST=ON +``` + +Build the Rust crate, examples, and tests through the CMake targets: + +```bash +cmake --build build --target build_mooncake_store_rust +cmake --build build --target build_mooncake_store_rust_example +cmake --build build --target build_mooncake_store_rust_tests +``` + +## Build With Cargo + +Standalone Cargo builds need the CMake build directory and Store headers: + +```bash +cd mooncake-store/rust + +export MOONCAKE_BUILD_DIR=/path/to/Mooncake/build +export MOONCAKE_STORE_LIB_DIR=$MOONCAKE_BUILD_DIR/mooncake-store/src +export MOONCAKE_STORE_INCLUDE_DIR=/path/to/Mooncake/mooncake-store/include + +cargo build --examples --release +cargo test --tests --no-run --release +``` + +At runtime, the dynamic linker must be able to find the Mooncake shared +libraries generated by CMake. If they are not installed in a system library +directory, add the relevant build outputs to `LD_LIBRARY_PATH`: + +```bash +export LD_LIBRARY_PATH=$MOONCAKE_BUILD_DIR/mooncake-common:\ +$MOONCAKE_BUILD_DIR/mooncake-common/etcd:\ +$MOONCAKE_BUILD_DIR/mooncake-store/src:\ +$MOONCAKE_BUILD_DIR/mooncake-store/src/cachelib_memory_allocator:\ +$MOONCAKE_BUILD_DIR/mooncake-transfer-engine/src:\ +$MOONCAKE_BUILD_DIR/mooncake-transfer-engine/src/common/base:\ +${LD_LIBRARY_PATH:-} +``` + +## Integration Smoke Test + +The integration smoke test needs a running metadata server and +`mooncake_master`. The CI workflow is the best reference for the complete +service setup. Once those services are running, execute: + +```bash +cd mooncake-store/rust + +MC_RUST_STORE_RUN_INTEGRATION=true \ +MC_METADATA_SERVER=http://127.0.0.1:8080/metadata \ +MC_RUST_STORE_MASTER_ADDR=127.0.0.1:50051 \ +MC_RUST_STORE_LOCAL_HOSTNAME=127.0.0.1 \ +MC_RUST_STORE_PROTOCOL=tcp \ +MC_RUST_STORE_DEVICE_NAME= \ +cargo test --test minimal_smoke -- --nocapture +``` + +## Benchmark Smoke Test + +For a short benchmark run, reduce the iteration counts: + +```bash +cd mooncake-store/rust + +MC_RUST_BENCH_ITERATIONS=4 \ +MC_RUST_BENCH_VALUE_SIZE=4096 \ +MC_RUST_BENCH_WARMUP=1 \ +cargo run --release --example store_benchmark +``` diff --git a/mooncake-store/rust/build.rs b/mooncake-store/rust/build.rs index b38da413..ee873a1c 100644 --- a/mooncake-store/rust/build.rs +++ b/mooncake-store/rust/build.rs @@ -13,6 +13,7 @@ // limitations under the License. use std::env; +use std::fs; use std::path::PathBuf; use std::process::Command; @@ -30,6 +31,54 @@ fn push_env_paths(search_dirs: &mut Vec, name: &str) { } } +fn push_cmake_prefix_lib_dirs(search_dirs: &mut Vec, prefix: PathBuf) { + push_existing_dir(search_dirs, prefix.join("lib")); + push_existing_dir(search_dirs, prefix.join("lib64")); +} + +fn push_cmake_prefix_paths(search_dirs: &mut Vec, value: &str) { + for prefix in value.split(';').filter(|prefix| !prefix.is_empty()) { + push_cmake_prefix_lib_dirs(search_dirs, PathBuf::from(prefix)); + } +} + +fn push_cmake_cache_library_dirs(search_dirs: &mut Vec, build_dir: &PathBuf) { + let cache_path = build_dir.join("CMakeCache.txt"); + let Ok(cache) = fs::read_to_string(cache_path) else { + return; + }; + + for line in cache.lines() { + let Some((key, value)) = line.split_once('=') else { + continue; + }; + + if key == "CMAKE_PREFIX_PATH:PATH" { + push_cmake_prefix_paths(search_dirs, value); + continue; + } + + if key.ends_with("_DIR:PATH") { + let package_dir = PathBuf::from(value); + if package_dir.parent().and_then(|dir| dir.file_name()) + == Some(std::ffi::OsStr::new("cmake")) + { + if let Some(lib_dir) = package_dir.ancestors().nth(2) { + push_existing_dir(search_dirs, lib_dir.to_path_buf()); + } + } + continue; + } + + if key.ends_with("_LIBRARY:FILEPATH") || key.ends_with("_LIBRARIES:FILEPATH") { + let library_path = PathBuf::from(value); + if let Some(parent) = library_path.parent() { + push_existing_dir(search_dirs, parent.to_path_buf()); + } + } + } +} + fn has_library(search_dirs: &[PathBuf], candidates: &[&str]) -> bool { search_dirs.iter().any(|dir| { candidates.iter().any(|candidate| { @@ -195,6 +244,7 @@ fn main() { if let Ok(build_dir) = env::var("MOONCAKE_BUILD_DIR") { let build_dir = PathBuf::from(build_dir); + push_cmake_cache_library_dirs(&mut search_dirs, &build_dir); for dir in [ build_dir.join("mooncake-store/src"), build_dir.join("mooncake-store/src/cachelib_memory_allocator"), @@ -208,6 +258,7 @@ fn main() { } let default_build_dir = manifest_dir.join("../../build"); + push_cmake_cache_library_dirs(&mut search_dirs, &default_build_dir); for dir in [ default_build_dir.join("mooncake-store/src"), default_build_dir.join("mooncake-store/src/cachelib_memory_allocator"), @@ -223,6 +274,10 @@ fn main() { push_existing_dir(&mut search_dirs, dir); } + if let Ok(cmake_prefix_path) = env::var("CMAKE_PREFIX_PATH") { + push_cmake_prefix_paths(&mut search_dirs, &cmake_prefix_path); + } + push_env_paths(&mut search_dirs, "LD_LIBRARY_PATH"); push_env_paths(&mut search_dirs, "LIBRARY_PATH"); -- 2.34.1 From 6ab98c7aca54d1479ed78d65817050a93944e186 Mon Sep 17 00:00:00 2001 From: Kevin Flansburg Date: Sat, 16 May 2026 04:19:29 -0600 Subject: [PATCH 080/382] [TE] Expose sendProbe via Python binding (#2088) Exposes the existing TransferMetadata::sendProbe C++ method through the TransferEngine pybind module as engine.send_probe(peer_server_name). This enables SGLang's MooncakeKVManager to issue lightweight JSON-RPC probes against peers, used to test whether a previously-blacklisted mooncake_session_id has become reachable again so it can be removed from the failed_sessions set. Returns 0 on success, non-zero on failure (matching the C++ contract). No behavior change for existing engine.* methods. Tested: - New Python unit tests in transfer_engine_initiator_test.py covering both the reachable-peer and unknown-peer cases. - Manually validated end-to-end against SGLang's MooncakeKVManager. --- .../python-api-reference/transfer-engine.md | 19 ++++++++++++++++++ .../transfer_engine/transfer_engine_py.cpp | 11 ++++++++++ .../transfer_engine/transfer_engine_py.h | 2 ++ .../tests/transfer_engine_initiator_test.py | 20 +++++++++++++++++++ 4 files changed, 52 insertions(+) diff --git a/docs/source/python-api-reference/transfer-engine.md b/docs/source/python-api-reference/transfer-engine.md index d23b70f5..45c296bf 100644 --- a/docs/source/python-api-reference/transfer-engine.md +++ b/docs/source/python-api-reference/transfer-engine.md @@ -588,6 +588,25 @@ Gets the list of pending transfer notifications received from other nodes. **Returns:** - `List[TransferNotify]`: List of notification objects containing name and message +#### send_probe() + +```python +send_probe(peer_server_name) +``` + +Sends a lightweight JSON-RPC probe to a peer to verify reachability. Used to +test whether a previously-unreachable peer has recovered (e.g. SGLang's +`MooncakeKVManager` uses this to clear entries from its `failed_sessions` +blacklist). + +**Parameters:** +- `peer_server_name` (str): Peer hostname in `host:port` form, as registered + with the metadata server. + +**Returns:** +- `int`: 0 on success, non-zero on failure (peer unknown, unreachable, or + RPC error). + ## Environment Variables The Transfer Engine respects the following environment variables: diff --git a/mooncake-integration/transfer_engine/transfer_engine_py.cpp b/mooncake-integration/transfer_engine/transfer_engine_py.cpp index 3bede64d..63e23af7 100644 --- a/mooncake-integration/transfer_engine/transfer_engine_py.cpp +++ b/mooncake-integration/transfer_engine/transfer_engine_py.cpp @@ -1011,6 +1011,12 @@ std::vector TransferEnginePy::getNotifies() { return result; } +int TransferEnginePy::sendProbe(const std::string& peer_server_name) { + if (!engine_) return -1; + pybind11::gil_scoped_release release; + return engine_->getMetadata()->sendProbe(peer_server_name); +} + namespace py = pybind11; // Implementation of coro_rpc_interface binding function @@ -1107,6 +1113,11 @@ PYBIND11_MODULE(engine, m) { .def("warmup_efa_segment", &TransferEnginePy::warmupEfaSegment, py::arg("segment_name")) .def("get_notifies", &TransferEnginePy::getNotifies) + .def("send_probe", &TransferEnginePy::sendProbe, + py::arg("peer_server_name"), + "Send a JSON-RPC probe to peer to verify reachability. " + "Returns 0 on success, non-zero on failure. Used by " + "SGLang's failed-session blacklist recovery.") .def("get_engine", &TransferEnginePy::getEngine) .def("get_engine_ptr", &TransferEnginePy::getEnginePtr); diff --git a/mooncake-integration/transfer_engine/transfer_engine_py.h b/mooncake-integration/transfer_engine/transfer_engine_py.h index 0e9ac138..a7d6bde0 100644 --- a/mooncake-integration/transfer_engine/transfer_engine_py.h +++ b/mooncake-integration/transfer_engine/transfer_engine_py.h @@ -183,6 +183,8 @@ class TransferEnginePy { std::vector getNotifies(); + int sendProbe(const std::string &peer_server_name); + std::shared_ptr getEngine() const { return engine_; } uintptr_t getEnginePtr() const { return (uintptr_t)engine_.get(); } diff --git a/mooncake-wheel/tests/transfer_engine_initiator_test.py b/mooncake-wheel/tests/transfer_engine_initiator_test.py index f9551b75..0146e92c 100644 --- a/mooncake-wheel/tests/transfer_engine_initiator_test.py +++ b/mooncake-wheel/tests/transfer_engine_initiator_test.py @@ -268,6 +268,26 @@ class TestVLLMAdaptorTransfer(unittest.TestCase): f"[✓] {circles} rounds of batch_write_async_read passed, batch size {batch_size}." ) + def test_send_probe_reachable_peer(self): + """send_probe against a registered, reachable peer returns 0.""" + rc = self.adaptor.send_probe(self.target_server_name) + self.assertEqual( + rc, + 0, + f"send_probe to reachable peer {self.target_server_name} returned {rc}", + ) + + def test_send_probe_unknown_peer(self): + """send_probe against an unknown peer returns non-zero (does not crash).""" + # "unreachable_peer:9" is not registered with the metadata server, so the + # metadata lookup itself should fail and sendProbe returns ERR_METADATA. + rc = self.adaptor.send_probe("unreachable_peer:9") + self.assertNotEqual( + rc, + 0, + "send_probe to unknown peer unexpectedly succeeded", + ) + if __name__ == "__main__": unittest.main() -- 2.34.1 From c9be4e68f0e2569b6c91c8bd6655fb631bd4036d Mon Sep 17 00:00:00 2001 From: Yeuvoir Date: Sat, 16 May 2026 18:20:30 +0800 Subject: [PATCH 081/382] [Store] fix: prevent cross-process memcpy segfault when MC_STORE_MEMCPY auto-enables (#2001) * [Store] fix: require same-process endpoint for LOCAL_MEMCPY strategy isLocalTransfer compared only the IP of handle.transport_endpoint_ to the local endpoint, so two processes on the same host (same IP, different ports) were treated as LOCAL_MEMCPY-eligible. The memcpy worker then dereferenced handle.buffer_address_, which is a virtual address only valid in the owning process, and segfaulted inside __memcpy_avx512_unaligned_erms. This was latent before #1936 (MC_STORE_MEMCPY defaulted to off). The TCP-only auto-enable exposed it on multi-process workloads such as the TorchSpec inference/trainer pipeline. Compare the full transport endpoint instead, matching the check already used by Client::IsReplicaOnLocalMemory. Cross-process same-host transfers now correctly fall through to TRANSFER_ENGINE; same-process transfers still take the memcpy fast path. Fixes the crash reported with MC_STORE_MEMCPY auto-enabled on TCP-only hosts. --------- Co-authored-by: Teng Ma --- mooncake-store/include/transfer_task.h | 16 +++++ mooncake-store/src/transfer_task.cpp | 76 ++++++++++++++++----- mooncake-store/tests/transfer_task_test.cpp | 31 +++++++++ scripts/run_tests.sh | 6 +- 4 files changed, 110 insertions(+), 19 deletions(-) diff --git a/mooncake-store/include/transfer_task.h b/mooncake-store/include/transfer_task.h index df7c90e9..42961d05 100644 --- a/mooncake-store/include/transfer_task.h +++ b/mooncake-store/include/transfer_task.h @@ -410,8 +410,24 @@ class TransferSubmitter { const std::unordered_map>& batched_slices); + /** + * @brief Pure comparison helper: returns true iff both endpoints are + * non-empty and identical. Exposed for unit testing of the locality + * decision without instantiating a full TransferEngine. + * + * Two endpoints identify the same process only when their ip:port (or + * full hostname) match exactly; same-host different-process pairs share + * an IP but not a port and must NOT be treated as locally addressable. + */ + static bool isSameProcessEndpoint(const std::string& handle_endpoint, + const std::string& local_endpoint); + private: TransferEngine& engine_; + // Cached at construction: the local transport endpoint never changes for + // the lifetime of the TransferSubmitter, so we avoid calling + // engine_.getLocalIpAndPort() (which allocates a string) on every transfer. + const std::string local_endpoint_; std::unique_ptr memcpy_pool_; std::unique_ptr fileread_pool_; bool memcpy_enabled_; diff --git a/mooncake-store/src/transfer_task.cpp b/mooncake-store/src/transfer_task.cpp index a5ab382c..0941536b 100644 --- a/mooncake-store/src/transfer_task.cpp +++ b/mooncake-store/src/transfer_task.cpp @@ -451,6 +451,7 @@ TransferSubmitter::TransferSubmitter(TransferEngine& engine, const std::string& local_hostname, TransferMetric* transfer_metric) : engine_(engine), + local_endpoint_(engine.getLocalIpAndPort()), memcpy_pool_(std::make_unique()), fileread_pool_(std::make_unique(backend)), local_hostname_(local_hostname), @@ -816,26 +817,65 @@ TransferStrategy TransferSubmitter::selectStrategy( return TransferStrategy::TRANSFER_ENGINE; } +namespace { +// Helper function to extract IP address from endpoint string (ip:port format). +// Supports both IPv4 (ip:port) and IPv6 ([ipv6]:port) formats. +std::string extractIpAddress(const std::string& endpoint) { + if (endpoint.empty()) { + return ""; + } + + // Handle IPv6 format: [ipv6]:port + if (endpoint[0] == '[') { + size_t closing_bracket = endpoint.find(']'); + if (closing_bracket == std::string::npos) { + LOG(WARNING) << "Invalid IPv6 endpoint format: " << endpoint; + return ""; + } + return endpoint.substr(1, closing_bracket - 1); + } + + // Handle IPv4 or hostname:port format. + size_t colon_pos = endpoint.rfind(':'); + if (colon_pos != std::string::npos) { + return endpoint.substr(0, colon_pos); + } + + // No colon found, return the whole string (might be just IP or hostname). + return endpoint; +} +} // namespace + +bool TransferSubmitter::isSameProcessEndpoint( + const std::string& handle_endpoint, const std::string& local_endpoint) { + // Local memcpy requires that handle.buffer_address_ is a virtual address + // valid in THIS process. Same host is not enough: two processes on the + // same host share an IP but have distinct virtual address spaces, so a + // memcpy on a peer process's address would segfault. Require the full + // transport endpoint to match, which uniquely identifies the owning + // process. + if (handle_endpoint.empty() || local_endpoint.empty()) { + return false; + } + if (handle_endpoint == local_endpoint) { + return true; + } + + const std::string handle_ip = extractIpAddress(handle_endpoint); + const std::string local_ip = extractIpAddress(local_endpoint); + if (!handle_ip.empty() && handle_ip == local_ip) { + VLOG(2) << "Disabling local memcpy for same-host endpoints with " + "different process endpoints: handle=" + << handle_endpoint << ", local=" << local_endpoint; + } + + return false; +} + bool TransferSubmitter::isLocalTransfer( const AllocatedBuffer::Descriptor& handle) const { - if (handle.transport_endpoint_.empty()) return false; - - // Metadata-service descriptors use the client hostname as the segment ID. - // If it matches this client's hostname, the buffer address is local. - if (!local_hostname_.empty() && - local_hostname_ == handle.transport_endpoint_) { - return true; - } - - // P2P descriptors use the transfer engine endpoint as the segment ID. - // If it matches this engine's endpoint, the buffer address is local. - std::string local_ep = engine_.getLocalIpAndPort(); - if (!local_ep.empty() && handle.transport_endpoint_ == local_ep) { - return true; - } - - // Without a local endpoint we cannot prove locality; disable memcpy. - return false; + return isSameProcessEndpoint(handle.transport_endpoint_, local_hostname_) || + isSameProcessEndpoint(handle.transport_endpoint_, local_endpoint_); } bool TransferSubmitter::validateTransferParams( diff --git a/mooncake-store/tests/transfer_task_test.cpp b/mooncake-store/tests/transfer_task_test.cpp index caa7c7ad..d9c2ee9e 100644 --- a/mooncake-store/tests/transfer_task_test.cpp +++ b/mooncake-store/tests/transfer_task_test.cpp @@ -144,6 +144,37 @@ TEST_F(TransferTaskTest, MemcpyWorkerPoolMultipleOperations) { } } +// Test the locality decision used by TransferSubmitter::isLocalTransfer. +// Same-host different-process pairs share an IP but have distinct ports; +// they must NOT be treated as locally addressable, otherwise memcpy in the +// caller process would dereference a virtual address belonging to a peer +// process and segfault. +TEST_F(TransferTaskTest, IsSameProcessEndpoint) { + // Empty inputs -> not same-process (cannot prove locality). + EXPECT_FALSE(TransferSubmitter::isSameProcessEndpoint("", "")); + EXPECT_FALSE( + TransferSubmitter::isSameProcessEndpoint("", "192.168.1.10:12345")); + EXPECT_FALSE( + TransferSubmitter::isSameProcessEndpoint("192.168.1.10:12345", "")); + + // Identical ip:port -> same process. + EXPECT_TRUE(TransferSubmitter::isSameProcessEndpoint("192.168.1.10:12345", + "192.168.1.10:12345")); + + // Same host, different port -> different process, NOT local. + // This is the regression case fixed by this change. + EXPECT_FALSE(TransferSubmitter::isSameProcessEndpoint( + "192.168.1.10:12345", "192.168.1.10:12346")); + + // Different hosts -> not local. + EXPECT_FALSE(TransferSubmitter::isSameProcessEndpoint( + "192.168.1.10:12345", "192.168.1.11:12345")); + + // Hostname endpoints (non-P2P metadata mode) compare as full strings. + EXPECT_TRUE(TransferSubmitter::isSameProcessEndpoint("host-a", "host-a")); + EXPECT_FALSE(TransferSubmitter::isSameProcessEndpoint("host-a", "host-b")); +} + // Test TransferStrategy enum and stream operator TEST_F(TransferTaskTest, TransferStrategyEnum) { // Test enum values diff --git a/scripts/run_tests.sh b/scripts/run_tests.sh index 64adbe96..2fc703b5 100755 --- a/scripts/run_tests.sh +++ b/scripts/run_tests.sh @@ -40,7 +40,11 @@ DUMMY_TEST_PID_2=$! wait $DUMMY_TEST_PID_1 $DUMMY_TEST_PID_2 kill $CLIENT_PID || true -pip install torch numpy safetensors packaging +pip install numpy safetensors packaging +# Keep the test torch aligned with the EP/PG variants packaged into the CI wheel. +pip install "${MOONCAKE_TEST_TORCH_SPEC:-torch==2.11.0+cu128}" \ + --index-url "${MOONCAKE_TEST_TORCH_INDEX_URL:-https://download.pytorch.org/whl/cu128}" \ + --extra-index-url https://pypi.org/simple MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 python test_put_get_tensor.py MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 python test_safetensor_functions.py kill $MASTER_PID || true -- 2.34.1 From be93707047dc0f92225f9b7dc7098d67d5416ddc Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Sat, 16 May 2026 18:21:22 +0800 Subject: [PATCH 082/382] [Store] feat: add graceful segment unmount APIs (#2065) --- .../source/http-api-reference/http-service.md | 20 +- .../python-api-reference/mooncake-store.md | 25 +- mooncake-integration/store/store_py.cpp | 15 +- mooncake-store/include/client_service.h | 23 +- mooncake-store/include/master_client.h | 7 + mooncake-store/include/master_service.h | 40 +++ mooncake-store/include/real_client.h | 13 +- mooncake-store/include/rpc_service.h | 6 + mooncake-store/include/segment.h | 27 +- mooncake-store/src/client_service.cpp | 172 ++++++++++++- mooncake-store/src/master_client.cpp | 34 +++ mooncake-store/src/master_service.cpp | 160 +++++++++++- mooncake-store/src/real_client.cpp | 173 ++++++++++--- mooncake-store/src/rpc_service.cpp | 37 +++ mooncake-store/src/segment.cpp | 63 ++++- mooncake-store/tests/master_service_test.cpp | 230 ++++++++++++++++++ mooncake-store/tests/pybind_client_test.cpp | 27 ++ .../mooncake/mooncake_store_service.py | 8 +- .../tests/test_mooncake_store_service_api.py | 40 ++- 19 files changed, 1038 insertions(+), 82 deletions(-) diff --git a/docs/source/http-api-reference/http-service.md b/docs/source/http-api-reference/http-service.md index b415c262..c04c7b2e 100644 --- a/docs/source/http-api-reference/http-service.md +++ b/docs/source/http-api-reference/http-service.md @@ -241,11 +241,16 @@ Unmount one or more segment ids previously returned by `/api/mount_shm`. **Request Body**: ```json { - "segment_ids": ["00000000-0000-0000-0000-000000000001"] + "segment_ids": ["00000000-0000-0000-0000-000000000001"], + "grace_period_seconds": 0 } ``` `segment_ids` may also be provided as a single string for one segment. +`grace_period_seconds` is optional and defaults to `0`, which keeps the +existing immediate unmount behavior. When set to a positive value, the master +keeps the segment readable for that grace period while preventing new +allocations, then completes the unmount. **Success Response**: ```json @@ -258,7 +263,8 @@ Unmount one or more segment ids previously returned by `/api/mount_shm`. ```bash curl -X POST http://localhost:8080/api/unmount_shm \ -H "Content-Type: application/json" \ - -d '{"segment_ids": ["00000000-0000-0000-0000-000000000001"]}' + -d '{"segment_ids": ["00000000-0000-0000-0000-000000000001"], + "grace_period_seconds": 30}' ``` ### `/api/mount` @@ -312,11 +318,16 @@ the memory allocated by the store process. **Request Body**: ```json { - "segment_ids": ["00000000-0000-0000-0000-000000000002"] + "segment_ids": ["00000000-0000-0000-0000-000000000002"], + "grace_period_seconds": 0 } ``` `segment_ids` may also be provided as a single string for one segment. +`grace_period_seconds` is optional and defaults to `0`, which keeps the +existing immediate unmount-and-free behavior. When set to a positive value, the +master keeps the segment readable for that grace period while preventing new +allocations, then the store releases the local allocated memory after cleanup. **Success Response**: ```json @@ -329,5 +340,6 @@ the memory allocated by the store process. ```bash curl -X POST http://localhost:8080/api/unmount \ -H "Content-Type: application/json" \ - -d '{"segment_ids": ["00000000-0000-0000-0000-000000000002"]}' + -d '{"segment_ids": ["00000000-0000-0000-0000-000000000002"], + "grace_period_seconds": 30}' ``` diff --git a/docs/source/python-api-reference/mooncake-store.md b/docs/source/python-api-reference/mooncake-store.md index 60352a13..9ad362e8 100644 --- a/docs/source/python-api-reference/mooncake-store.md +++ b/docs/source/python-api-reference/mooncake-store.md @@ -1484,18 +1484,26 @@ named shared memory object name instead of an arbitrary path. Unmount one or more file or shared-memory segments by segment id. ```python -def unmount_segment(self, segment_ids: List[str]) -> int +def unmount_segment( + self, + segment_ids: List[str], + grace_period_seconds: int = 0, +) -> int ``` **Parameters:** - `segment_ids` (List[str]): Segment ids returned by `mount_segment()`. +- `grace_period_seconds` (int, optional): Grace period before the segment is + fully unmounted. Defaults to `0`, which keeps the existing immediate unmount + behavior. During a positive grace period, the segment remains readable but no + longer accepts new allocations. **Returns:** - `int`: Status code (0 = success, non-zero = error code) **Example:** ```python -ret = store.unmount_segment(segment_ids) +ret = store.unmount_segment(segment_ids, grace_period_seconds=30) if ret != 0: print("Unmount failed:", ret) ``` @@ -1550,19 +1558,28 @@ Unmount one or more internally allocated segments by segment id and free their local memory. ```python -def unmount_and_free_segment(self, segment_ids: List[str]) -> int +def unmount_and_free_segment( + self, + segment_ids: List[str], + grace_period_seconds: int = 0, +) -> int ``` **Parameters:** - `segment_ids` (List[str]): Segment ids returned by `allocate_and_mount_segment()`. +- `grace_period_seconds` (int, optional): Grace period before the segment is + fully unmounted and its local allocated memory is released. Defaults to `0`, + which keeps the existing immediate unmount-and-free behavior. During a + positive grace period, the segment remains readable but no longer accepts new + allocations. **Returns:** - `int`: Status code (0 = success, non-zero = error code) **Example:** ```python -ret = store.unmount_and_free_segment(segment_ids) +ret = store.unmount_and_free_segment(segment_ids, grace_period_seconds=30) if ret != 0: print("Unmount and free failed:", ret) ``` diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index ba28c5ac..d5464d8c 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -340,14 +340,15 @@ class MooncakeStorePyWrapper { return result; } - int unmount_segment(const std::vector &segment_ids) { + int unmount_segment(const std::vector &segment_ids, + uint64_t grace_period_seconds = 0) { auto real_client = std::dynamic_pointer_cast(store_); if (!real_client) { LOG(ERROR) << "unmount_segment requires RealClient"; return -1; } py::gil_scoped_release release; - return real_client->unmountSegment(segment_ids); + return real_client->unmountSegment(segment_ids, grace_period_seconds); } py::dict allocate_and_mount_segment(size_t size, @@ -377,14 +378,16 @@ class MooncakeStorePyWrapper { return result; } - int unmount_and_free_segment(const std::vector &segment_ids) { + int unmount_and_free_segment(const std::vector &segment_ids, + uint64_t grace_period_seconds = 0) { auto real_client = std::dynamic_pointer_cast(store_); if (!real_client) { LOG(ERROR) << "unmount_and_free_segment requires RealClient"; return -1; } py::gil_scoped_release release; - return real_client->unmountAndFreeSegment(segment_ids); + return real_client->unmountAndFreeSegment(segment_ids, + grace_period_seconds); } std::string get_tp_key_name(const std::string &base_key, int rank) const { @@ -1927,14 +1930,14 @@ PYBIND11_MODULE(store, m) { py::arg("path"), py::arg("size"), py::arg("offset") = 0, py::arg("protocol") = "tcp", py::arg("location") = "") .def("unmount_segment", &MooncakeStorePyWrapper::unmount_segment, - py::arg("segment_ids")) + py::arg("segment_ids"), py::arg("grace_period_seconds") = 0) .def("allocate_and_mount_segment", &MooncakeStorePyWrapper::allocate_and_mount_segment, py::arg("size"), py::arg("protocol") = "tcp", py::arg("location") = "") .def("unmount_and_free_segment", &MooncakeStorePyWrapper::unmount_and_free_segment, - py::arg("segment_ids")) + py::arg("segment_ids"), py::arg("grace_period_seconds") = 0) .def("alloc_from_mem_pool", [](MooncakeStorePyWrapper &self, size_t size) { py::gil_scoped_release release; diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index 8aa0f1a4..ac848d1b 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -2,6 +2,8 @@ #include #include +#include +#include #include #include #include @@ -306,8 +308,11 @@ class Client { /** * @brief Unmounts a segment by its UUID. * Logic is identical to UnmountSegment, but looks up by id. + * @param grace_period_ms 0 = immediate unmount (legacy behavior). */ - tl::expected UnmountSegmentById(const UUID& segment_id); + tl::expected UnmountSegmentById( + const UUID& segment_id, uint64_t grace_period_ms = 0, + std::function cleanup_callback = {}); /** * @brief Registers memory buffer with TransferEngine for data transfer @@ -707,6 +712,14 @@ class Client { mutable std::mutex mounted_segments_mutex_; std::unordered_map> mounted_segments_; + // Segments in graceful unmount: readable by remote peers, not allocatable + // locally. TE MR remains registered until master confirms removal. + std::unordered_map> + gracefully_unmounting_segments_; + std::unordered_map, + boost::hash> + graceful_unmount_cleanup_callbacks_; + /** * @brief Internal helper to unmount a segment by iterator. * Caller must hold mounted_segments_mutex_. @@ -714,6 +727,14 @@ class Client { tl::expected UnmountSegmentImpl( std::unordered_map>::iterator it); + void StartGracefulUnmountTimer(const UUID& segment_id, + uint64_t grace_period_ms); + void OnGracefulUnmountTimer(const UUID& segment_id, int retry_left); + bool WaitForGracefulUnmountDelay(std::chrono::milliseconds delay); + std::mutex graceful_unmount_timer_mutex_; + std::condition_variable graceful_unmount_timer_cv_; + bool graceful_unmount_timer_stopping_{false}; + // Configuration const std::string local_hostname_; const std::string metadata_connstring_; diff --git a/mooncake-store/include/master_client.h b/mooncake-store/include/master_client.h index bda524b5..44c67226 100644 --- a/mooncake-store/include/master_client.h +++ b/mooncake-store/include/master_client.h @@ -14,6 +14,7 @@ #include "client_metric.h" #include "replica.h" +#include "segment.h" #include "types.h" #include "rpc_types.h" #include "master_metric_manager.h" @@ -315,6 +316,9 @@ class MasterClient { [[nodiscard]] tl::expected UnmountSegment( const UUID& segment_id); + [[nodiscard]] tl::expected GracefulUnmountSegment( + const UUID& segment_id, uint64_t grace_period_ms); + /** * @brief Gets the cluster ID for the current client to use as subdirectory * name @@ -322,6 +326,9 @@ class MasterClient { */ [[nodiscard]] tl::expected GetFsdir(); + [[nodiscard]] tl::expected QuerySegmentStatusById( + const UUID& segment_id); + [[nodiscard]] tl::expected GetStorageConfig(); diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index 54f3f73a..7c108108 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -99,6 +99,10 @@ class MasterService { auto UnmountSegment(const UUID& segment_id, const UUID& client_id) -> tl::expected; + auto GracefulUnmountSegment(const UUID& segment_id, const UUID& client_id, + uint64_t grace_period_ms) + -> tl::expected; + /** * @brief Check if an object exists * @return ErrorCode::OK if exists, otherwise return other ErrorCode @@ -494,6 +498,12 @@ class MasterService { tl::expected QuerySegmentStatus( const std::string& segment_name); + /** + * @brief Query current segment lifecycle state by segment id. + */ + tl::expected QuerySegmentStatusById( + const UUID& segment_id); + /** * @brief Query the status of a task * @return Task basic info @@ -938,6 +948,36 @@ class MasterService { tl::expected PushOffloadingQueue(const std::string& key, Replica& replica); + // Graceful unmount scheduler + class GracefulUnmountScheduler { + public: + explicit GracefulUnmountScheduler(MasterService* service); + ~GracefulUnmountScheduler(); + void Schedule(const UUID& segment_id, const UUID& client_id, + std::chrono::steady_clock::time_point expire_time); + void RemoveClientRecords(const UUID& client_id); + void Stop(); + + private: + void TimerLoop(); + struct Record { + UUID segment_id; + UUID client_id; + std::chrono::steady_clock::time_point expire_time; + bool operator>(const Record& other) const { + return expire_time > other.expire_time; + } + }; + MasterService* service_; + std::mutex mutex_; + std::priority_queue, std::greater> + queue_; + std::thread timer_thread_; + std::atomic timer_running_{false}; + bool stopping_{false}; + std::condition_variable timer_cv_; + } graceful_unmount_scheduler_; + // Lease related members const uint64_t default_kv_lease_ttl_; // in milliseconds const uint64_t default_kv_soft_pin_ttl_; // in milliseconds diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index 0c6088e5..31948e6b 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -695,8 +695,10 @@ class RealClient : public PyClient { /** * @brief Unmount segments by their ids and clean up local mmap/fd. + * @param grace_period_seconds 0 = immediate unmount (legacy behavior). */ - int unmountSegment(const std::vector &segment_ids); + int unmountSegment(const std::vector &segment_ids, + uint64_t grace_period_seconds = 0); /** * @brief Allocate memory internally and mount segments to master. @@ -712,8 +714,10 @@ class RealClient : public PyClient { /** * @brief Unmount segments by their ids and free locally allocated memory. + * @param grace_period_seconds 0 = immediate unmount (legacy behavior). */ - int unmountAndFreeSegment(const std::vector &segment_ids); + int unmountAndFreeSegment(const std::vector &segment_ids, + uint64_t grace_period_seconds = 0); struct MountedSegmentRecord { void *mmap_base = nullptr; @@ -872,6 +876,11 @@ class RealClient : public PyClient { std::unordered_map allocated_segment_records_; std::mutex allocated_segment_records_mutex_; + + void ReleaseMountedSegmentRecord(const std::string &segment_id); + void ReleaseAllMountedSegmentRecords(); + void ReleaseAllocatedSegmentRecord(const std::string &segment_id); + void ReleaseAllAllocatedSegmentRecords(); }; } // namespace mooncake diff --git a/mooncake-store/include/rpc_service.h b/mooncake-store/include/rpc_service.h index 07670f86..96f20e10 100644 --- a/mooncake-store/include/rpc_service.h +++ b/mooncake-store/include/rpc_service.h @@ -124,6 +124,10 @@ class WrappedMasterService { tl::expected UnmountSegment(const UUID& segment_id, const UUID& client_id); + tl::expected GracefulUnmountSegment( + const UUID& segment_id, const UUID& client_id, + uint64_t grace_period_ms); + tl::expected GetFsdir(); tl::expected GetStorageConfig(); @@ -161,6 +165,8 @@ class WrappedMasterService { tl::expected QuerySegmentStatus( const std::string& segment_name); + tl::expected QuerySegmentStatusById( + const UUID& segment_id); tl::expected CreateCopyTask( const std::string& key, const std::vector& targets); diff --git a/mooncake-store/include/segment.h b/mooncake-store/include/segment.h index da90ecc3..b55ceda8 100644 --- a/mooncake-store/include/segment.h +++ b/mooncake-store/include/segment.h @@ -21,7 +21,8 @@ enum class SegmentStatus { OK, // Segment is mounted and available for allocation DRAINING, // Segment remains readable but accepts no new allocations DRAINED, // Segment has been drained and awaits unmount - UNMOUNTING, // Segment is under unmounting + GRACEFULLY_UNMOUNTING, // Readable, no new allocations, timer running + UNMOUNTING, // Segment is under unmounting }; /** @@ -30,11 +31,13 @@ enum class SegmentStatus { inline std::ostream& operator<<(std::ostream& os, const SegmentStatus& status) noexcept { static const std::unordered_map - status_strings{{SegmentStatus::UNDEFINED, "UNDEFINED"}, - {SegmentStatus::OK, "OK"}, - {SegmentStatus::DRAINING, "DRAINING"}, - {SegmentStatus::DRAINED, "DRAINED"}, - {SegmentStatus::UNMOUNTING, "UNMOUNTING"}}; + status_strings{ + {SegmentStatus::UNDEFINED, "UNDEFINED"}, + {SegmentStatus::OK, "OK"}, + {SegmentStatus::DRAINING, "DRAINING"}, + {SegmentStatus::DRAINED, "DRAINED"}, + {SegmentStatus::GRACEFULLY_UNMOUNTING, "GRACEFULLY_UNMOUNTING"}, + {SegmentStatus::UNMOUNTING, "UNMOUNTING"}}; os << (status_strings.count(status) ? status_strings.at(status) : "UNKNOWN"); @@ -102,6 +105,12 @@ class ScopedSegmentAccess { ErrorCode PrepareUnmountSegment(const UUID& segment_id, size_t& metrics_dec_capacity); + /** + * @brief Prepare a segment for graceful unmount: remove allocator but keep + * segment metadata. Status becomes GRACEFULLY_UNMOUNTING. + */ + ErrorCode PrepareGracefulUnmountSegment(const UUID& segment_id); + /** * @brief Deleting the segment to complete the unmounting operation */ @@ -165,6 +174,12 @@ class ScopedSegmentAccess { ErrorCode GetSegmentStatusByName(const std::string& segment_name, SegmentStatus& status) const; + /** + * @brief Query the lifecycle status of a segment by id. + */ + ErrorCode GetSegmentStatusById(const UUID& segment_id, + SegmentStatus& status) const; + /** * @brief Update the lifecycle status of a segment by name. */ diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index e6f421dc..aead7e03 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -2,6 +2,8 @@ #include +#include "segment.h" + #include #include #include @@ -98,38 +100,75 @@ Client::~Client() { leader_monitor_thread_.join(); } - // Make a copy of mounted_segments_ to avoid modifying while iterating - std::vector segments_to_unmount; + { + std::lock_guard lock(graceful_unmount_timer_mutex_); + graceful_unmount_timer_stopping_ = true; + } + graceful_unmount_timer_cv_.notify_all(); + + // Stop queued timer/task callbacks before tearing down segment state. + task_running_ = false; + task_thread_pool_.stop(); + + // Make copies to avoid modifying while iterating + std::vector mounted_segments_copy; + std::vector gracefully_unmounting_segments_copy; + std::vector>> + graceful_cleanup_callbacks; { std::lock_guard lock(mounted_segments_mutex_); - segments_to_unmount.reserve(mounted_segments_.size()); + mounted_segments_copy.reserve(mounted_segments_.size()); for (auto& entry : mounted_segments_) { - segments_to_unmount.emplace_back(entry.second); + mounted_segments_copy.emplace_back(entry.second); } + for (auto& entry : gracefully_unmounting_segments_) { + gracefully_unmounting_segments_copy.emplace_back(entry.second); + } + graceful_cleanup_callbacks.reserve( + graceful_unmount_cleanup_callbacks_.size()); + for (auto& entry : graceful_unmount_cleanup_callbacks_) { + graceful_cleanup_callbacks.emplace_back(entry.first, + std::move(entry.second)); + } + graceful_unmount_cleanup_callbacks_.clear(); } - for (auto& segment : segments_to_unmount) { + // Unmount mounted segments: notify master + local cleanup + for (auto& segment : mounted_segments_copy) { auto result = UnmountSegment(reinterpret_cast(segment.base), segment.size); if (!result) { - LOG(ERROR) << "Failed to unmount segment: " + LOG(ERROR) << "Failed to unmount segment in destructor: " << toString(result.error()); } } + // Unregister gracefully unmounting segments: master already has timer + for (auto& segment : gracefully_unmounting_segments_copy) { + int rc = transfer_engine_->unregisterLocalMemory( + reinterpret_cast(segment.base)); + if (rc != 0 && rc != ERR_ADDRESS_NOT_REGISTERED) { + LOG(ERROR) << "Failed to unregister transfer buffer in destructor: " + << rc; + } + } + // Clear any remaining segments { std::lock_guard lock(mounted_segments_mutex_); mounted_segments_.clear(); + gracefully_unmounting_segments_.clear(); + } + + for (auto& entry : graceful_cleanup_callbacks) { + if (entry.second) { + entry.second(entry.first); + } } // Stop hot cache handler and hot cache hot_cache_handler_.reset(); hot_cache_.reset(); - - // Stop task thread pool after task polling has stopped. - task_running_ = false; - task_thread_pool_.stop(); } static std::optional get_auto_discover() { @@ -2203,7 +2242,8 @@ tl::expected Client::MountSegmentAndGetId( } tl::expected Client::UnmountSegmentById( - const UUID& segment_id) { + const UUID& segment_id, uint64_t grace_period_ms, + std::function cleanup_callback) { std::lock_guard lock(mounted_segments_mutex_); auto segment = mounted_segments_.find(segment_id); if (segment == mounted_segments_.end()) { @@ -2211,7 +2251,115 @@ tl::expected Client::UnmountSegmentById( return tl::unexpected(ErrorCode::INVALID_PARAMS); } - return UnmountSegmentImpl(segment); + if (grace_period_ms == 0) { + return UnmountSegmentImpl(segment); + } + + auto result = + master_client_.GracefulUnmountSegment(segment_id, grace_period_ms); + if (!result) { + ErrorCode err = result.error(); + LOG(ERROR) << "Failed to graceful unmount segment from master: " + << toString(err); + return tl::unexpected(err); + } + + gracefully_unmounting_segments_.emplace(segment->first, segment->second); + if (cleanup_callback) { + graceful_unmount_cleanup_callbacks_[segment->first] = + std::move(cleanup_callback); + } + mounted_segments_.erase(segment); + StartGracefulUnmountTimer(segment_id, grace_period_ms); + return {}; +} + +bool Client::WaitForGracefulUnmountDelay(std::chrono::milliseconds delay) { + std::unique_lock lock(graceful_unmount_timer_mutex_); + return graceful_unmount_timer_cv_.wait_for( + lock, delay, [this]() { return graceful_unmount_timer_stopping_; }); +} + +void Client::StartGracefulUnmountTimer(const UUID& segment_id, + uint64_t grace_period_ms) { + auto delay = + std::chrono::milliseconds(grace_period_ms) + std::chrono::seconds(10); + task_thread_pool_.enqueue([this, segment_id, delay]() { + if (this->WaitForGracefulUnmountDelay(delay)) { + return; + } + this->OnGracefulUnmountTimer(segment_id, /*retry_left=*/3); + }); +} + +void Client::OnGracefulUnmountTimer(const UUID& segment_id, int retry_left) { + // Query master to confirm the segment has been removed + { + std::lock_guard lock(mounted_segments_mutex_); + auto it = gracefully_unmounting_segments_.find(segment_id); + if (it == gracefully_unmounting_segments_.end()) { + // Already cleaned up (e.g. by destructor) + return; + } + } + + auto status = master_client_.QuerySegmentStatusById(segment_id); + bool removed = false; + if (!status) { + if (status.error() == ErrorCode::SEGMENT_NOT_FOUND) { + removed = true; + } else { + LOG(WARNING) << "Failed to query graceful unmount segment status: " + << toString(status.error()); + } + } else if (status.value() == SegmentStatus::UNDEFINED) { + removed = true; + } + + if (removed) { + std::function cleanup_callback; + { + std::lock_guard lock(mounted_segments_mutex_); + auto it = gracefully_unmounting_segments_.find(segment_id); + if (it != gracefully_unmounting_segments_.end()) { + int rc = transfer_engine_->unregisterLocalMemory( + reinterpret_cast(it->second.base)); + if (rc != 0 && rc != ERR_ADDRESS_NOT_REGISTERED) { + LOG(ERROR) + << "Failed to unregister TE MR for graceful unmount: " + << rc; + } + gracefully_unmounting_segments_.erase(it); + } + auto callback_it = + graceful_unmount_cleanup_callbacks_.find(segment_id); + if (callback_it != graceful_unmount_cleanup_callbacks_.end()) { + cleanup_callback = std::move(callback_it->second); + graceful_unmount_cleanup_callbacks_.erase(callback_it); + } + } + if (cleanup_callback) { + cleanup_callback(segment_id); + } + return; + } + + if (retry_left > 0) { + try { + task_thread_pool_.enqueue([this, segment_id, retry_left]() { + if (this->WaitForGracefulUnmountDelay( + std::chrono::seconds(10))) { + return; + } + this->OnGracefulUnmountTimer(segment_id, retry_left - 1); + }); + } catch (const std::runtime_error& e) { + VLOG(1) << "Skip graceful unmount retry enqueue: " << e.what(); + } + } else { + LOG(WARNING) << "Graceful unmount cleanup timeout for segment " + << UuidToString(segment_id); + } } tl::expected Client::RegisterLocalMemory( diff --git a/mooncake-store/src/master_client.cpp b/mooncake-store/src/master_client.cpp index dfbde59c..d7e85fce 100644 --- a/mooncake-store/src/master_client.cpp +++ b/mooncake-store/src/master_client.cpp @@ -157,6 +157,11 @@ struct RpcNameTraits<&WrappedMasterService::UnmountSegment> { static constexpr const char* value = "UnmountSegment"; }; +template <> +struct RpcNameTraits<&WrappedMasterService::GracefulUnmountSegment> { + static constexpr const char* value = "GracefulUnmountSegment"; +}; + template <> struct RpcNameTraits<&WrappedMasterService::Ping> { static constexpr const char* value = "Ping"; @@ -167,6 +172,11 @@ struct RpcNameTraits<&WrappedMasterService::GetFsdir> { static constexpr const char* value = "GetFsdir"; }; +template <> +struct RpcNameTraits<&WrappedMasterService::QuerySegmentStatusById> { + static constexpr const char* value = "QuerySegmentStatusById"; +}; + template <> struct RpcNameTraits<&WrappedMasterService::GetStorageConfig> { static constexpr const char* value = "GetStorageConfig"; @@ -734,6 +744,19 @@ tl::expected MasterClient::UnmountSegment( return result; } +tl::expected MasterClient::GracefulUnmountSegment( + const UUID& segment_id, uint64_t grace_period_ms) { + ScopedVLogTimer timer(1, "MasterClient::GracefulUnmountSegment"); + timer.LogRequest("segment_id=", segment_id, ", client_id=", client_id_, + ", grace_period_ms=", grace_period_ms); + + auto result = + invoke_rpc<&WrappedMasterService::GracefulUnmountSegment, void>( + segment_id, client_id_, grace_period_ms); + timer.LogResponseExpected(result); + return result; +} + tl::expected MasterClient::Ping() { ScopedVLogTimer timer(1, "MasterClient::Ping"); timer.LogRequest("client_id=", client_id_); @@ -753,6 +776,17 @@ tl::expected MasterClient::GetFsdir() { return result; } +tl::expected MasterClient::QuerySegmentStatusById( + const UUID& segment_id) { + ScopedVLogTimer timer(1, "MasterClient::QuerySegmentStatusById"); + timer.LogRequest("segment_id=", segment_id); + + auto result = invoke_rpc<&WrappedMasterService::QuerySegmentStatusById, + SegmentStatus>(segment_id); + timer.LogResponseExpected(result); + return result; +} + tl::expected MasterClient::GetStorageConfig() { ScopedVLogTimer timer(1, "MasterClient::GetStorageConfig"); diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index dbe51659..64ed2577 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -84,7 +84,8 @@ int64_t CurrentTimeMs() { MasterService::MasterService() : MasterService(MasterServiceConfig()) {} MasterService::MasterService(const MasterServiceConfig& config) - : default_kv_lease_ttl_(config.default_kv_lease_ttl), + : graceful_unmount_scheduler_(this), + default_kv_lease_ttl_(config.default_kv_lease_ttl), default_kv_soft_pin_ttl_(config.default_kv_soft_pin_ttl), allow_evict_soft_pinned_objects_(config.allow_evict_soft_pinned_objects), eviction_ratio_(config.eviction_ratio), @@ -264,6 +265,7 @@ MasterService::~MasterService() { snapshot_running_ = false; task_cleanup_running_ = false; job_dispatch_running_ = false; + graceful_unmount_scheduler_.Stop(); // Wake sleepers so join() doesn't block for long sleep intervals. task_cleanup_cv_.notify_all(); @@ -460,6 +462,44 @@ auto MasterService::UnmountSegment(const UUID& segment_id, return {}; } +auto MasterService::GracefulUnmountSegment(const UUID& segment_id, + const UUID& client_id, + uint64_t grace_period_ms) + -> tl::expected { + std::unique_lock lock(snapshot_mutex_); + ScopedSegmentAccess segment_access = segment_manager_.getSegmentAccess(); + + // Verify ownership: the segment must belong to the calling client + std::vector client_segments; + auto err = segment_access.GetClientSegments(client_id, client_segments); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + bool owned = false; + for (auto& seg : client_segments) { + if (seg.id == segment_id) { + owned = true; + break; + } + } + if (!owned) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + err = segment_access.PrepareGracefulUnmountSegment(segment_id); + if (err == ErrorCode::SEGMENT_NOT_FOUND) { + return {}; + } + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + + auto expire_time = std::chrono::steady_clock::now() + + std::chrono::milliseconds(grace_period_ms); + graceful_unmount_scheduler_.Schedule(segment_id, client_id, expire_time); + return {}; +} + auto MasterService::ExistKey(const std::string& key) -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); @@ -535,6 +575,17 @@ auto MasterService::QuerySegmentStatus(const std::string& segment_name) return status; } +auto MasterService::QuerySegmentStatusById(const UUID& segment_id) + -> tl::expected { + ScopedSegmentAccess segment_access = segment_manager_.getSegmentAccess(); + SegmentStatus status = SegmentStatus::UNDEFINED; + auto err = segment_access.GetSegmentStatusById(segment_id, status); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + return status; +} + auto MasterService::QueryIp(const UUID& client_id) -> tl::expected, ErrorCode> { ScopedSegmentAccess segment_access = segment_manager_.getSegmentAccess(); @@ -3939,6 +3990,12 @@ void MasterService::ClientMonitorFunc() { // Update the client status to NEED_REMOUNT if (!expired_clients.empty()) { + // Notify graceful unmount scheduler to drop pending records + // for expired clients. The actual unmount is handled below. + for (auto& cid : expired_clients) { + graceful_unmount_scheduler_.RemoveClientRecords(cid); + } + // Record which segments are unmounted, will be used in the commit // phase. std::vector unmount_segments; @@ -5252,4 +5309,105 @@ MasterService::MetadataSerializer::DeserializeDiscardedReplicas( return {}; } +// --------------------------------------------------------------------------- +// GracefulUnmountScheduler implementation +// --------------------------------------------------------------------------- + +MasterService::GracefulUnmountScheduler::GracefulUnmountScheduler( + MasterService* service) + : service_(service) {} + +MasterService::GracefulUnmountScheduler::~GracefulUnmountScheduler() { Stop(); } + +void MasterService::GracefulUnmountScheduler::Schedule( + const UUID& segment_id, const UUID& client_id, + std::chrono::steady_clock::time_point expire_time) { + { + std::lock_guard lock(mutex_); + if (stopping_) { + return; + } + queue_.push(Record{segment_id, client_id, expire_time}); + if (!timer_running_.load()) { + timer_running_.store(true); + timer_thread_ = std::thread([this]() { this->TimerLoop(); }); + } + } + timer_cv_.notify_one(); +} + +void MasterService::GracefulUnmountScheduler::RemoveClientRecords( + const UUID& client_id) { + std::lock_guard lock(mutex_); + std::vector remaining; + remaining.reserve(queue_.size()); + while (!queue_.empty()) { + auto rec = queue_.top(); + queue_.pop(); + if (rec.client_id != client_id) { + remaining.push_back(rec); + } + } + for (auto& rec : remaining) { + queue_.push(rec); + } + timer_cv_.notify_one(); +} + +void MasterService::GracefulUnmountScheduler::Stop() { + { + std::lock_guard lock(mutex_); + stopping_ = true; + timer_running_.store(false); + } + timer_cv_.notify_all(); + if (timer_thread_.joinable()) { + timer_thread_.join(); + } +} + +void MasterService::GracefulUnmountScheduler::TimerLoop() { + while (timer_running_.load()) { + std::unique_lock lock(mutex_); + if (queue_.empty()) { + timer_cv_.wait(lock, [this]() { + return !timer_running_.load() || !queue_.empty(); + }); + if (!timer_running_.load()) break; + continue; + } + + auto now = std::chrono::steady_clock::now(); + auto next_expire = queue_.top().expire_time; + if (next_expire > now) { + timer_cv_.wait_until(lock, next_expire, [this, next_expire]() { + return !timer_running_.load() || queue_.empty() || + queue_.top().expire_time < next_expire; + }); + if (!timer_running_.load()) break; + continue; + } + + // Collect all expired records + std::vector expired; + while (!queue_.empty() && queue_.top().expire_time <= now) { + expired.push_back(queue_.top()); + queue_.pop(); + } + + for (auto& rec : expired) { + if (service_) { + auto result = + service_->UnmountSegment(rec.segment_id, rec.client_id); + if (!result.has_value()) { + LOG(WARNING) + << "Failed to complete graceful unmount, segment_id=" + << rec.segment_id << ", client_id=" << rec.client_id + << ", error=" << toString(result.error()); + } + } + } + } +} + } // namespace mooncake diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 33031689..b7711988 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -15,6 +15,7 @@ #include // for atexit #include #include +#include #include #include #include @@ -1073,19 +1074,10 @@ tl::expected RealClient::tearDownAll_internal() { } } - std::unordered_map records_to_free; - { - std::lock_guard lock(allocated_segment_records_mutex_); - records_to_free.swap(allocated_segment_records_); - } - for (const auto &entry : records_to_free) { - if (entry.second.base) { - free_memory(entry.second.protocol, entry.second.base); - } - } - // Reset all resources client_.reset(); + ReleaseAllMountedSegmentRecords(); + ReleaseAllAllocatedSegmentRecords(); client_buffer_allocator_.reset(); port_binder_.reset(); hugepage_segment_ptrs_.clear(); @@ -1230,13 +1222,84 @@ int RealClient::mountSegment(const std::string &path, size_t offset, return 0; } -int RealClient::unmountSegment(const std::vector &segment_ids) { +void RealClient::ReleaseMountedSegmentRecord(const std::string &segment_id) { + MountedSegmentRecord record; + bool found = false; + { + std::lock_guard lock(mounted_segment_records_mutex_); + auto it = mounted_segment_records_.find(segment_id); + if (it != mounted_segment_records_.end()) { + record = it->second; + mounted_segment_records_.erase(it); + found = true; + } + } + if (found && record.mmap_base) { + munmap(record.mmap_base, record.size); + } +} + +void RealClient::ReleaseAllMountedSegmentRecords() { + std::vector records; + { + std::lock_guard lock(mounted_segment_records_mutex_); + records.reserve(mounted_segment_records_.size()); + for (auto &entry : mounted_segment_records_) { + records.push_back(entry.second); + } + mounted_segment_records_.clear(); + } + for (auto &record : records) { + if (record.mmap_base) { + munmap(record.mmap_base, record.size); + } + } +} + +void RealClient::ReleaseAllocatedSegmentRecord(const std::string &segment_id) { + AllocatedSegmentRecord record; + bool found = false; + { + std::lock_guard lock(allocated_segment_records_mutex_); + auto it = allocated_segment_records_.find(segment_id); + if (it != allocated_segment_records_.end()) { + record = it->second; + allocated_segment_records_.erase(it); + found = true; + } + } + if (found && record.base) { + free_memory(record.protocol, record.base); + } +} + +void RealClient::ReleaseAllAllocatedSegmentRecords() { + std::unordered_map records; + { + std::lock_guard lock(allocated_segment_records_mutex_); + records.swap(allocated_segment_records_); + } + for (auto &entry : records) { + if (entry.second.base) { + free_memory(entry.second.protocol, entry.second.base); + } + } +} + +int RealClient::unmountSegment(const std::vector &segment_ids, + uint64_t grace_period_seconds) { if (!client_) { LOG(ERROR) << "Client not initialized"; return -1; } + uint64_t grace_period_ms = grace_period_seconds * 1000; int first_error = 0; + struct SegmentToUnmount { + std::string segment_id; + UUID id; + }; + std::vector to_unmount; std::vector> to_cleanup; { std::lock_guard lock(mounted_segment_records_mutex_); @@ -1256,17 +1319,37 @@ int RealClient::unmountSegment(const std::vector &segment_ids) { continue; } - auto result = client_->UnmountSegmentById(id); - if (!result.has_value()) { - LOG(ERROR) << "UnmountSegmentById failed for " << segment_id; - if (first_error == 0) { - first_error = static_cast(result.error()); - } - continue; // Don't release local resources on failure - } + to_unmount.push_back({segment_id, id}); + } + } - to_cleanup.emplace_back(segment_id, it->second); - mounted_segment_records_.erase(it); + for (auto &entry : to_unmount) { + std::function cleanup_callback; + if (grace_period_ms != 0) { + cleanup_callback = [this](const UUID &cleanup_id) { + ReleaseMountedSegmentRecord(UuidToString(cleanup_id)); + }; + } + auto result = client_->UnmountSegmentById(entry.id, grace_period_ms, + std::move(cleanup_callback)); + if (!result.has_value()) { + LOG(ERROR) << "UnmountSegmentById failed for " << entry.segment_id; + if (first_error == 0) { + first_error = static_cast(result.error()); + } + continue; // Don't release local resources on failure + } + + // For immediate unmount, clean up local mmap/fd right away. + // For graceful unmount, local mmap/fd is kept until the segment + // is actually removed or the client destructor runs. + if (grace_period_ms == 0) { + std::lock_guard lock(mounted_segment_records_mutex_); + auto it = mounted_segment_records_.find(entry.segment_id); + if (it != mounted_segment_records_.end()) { + to_cleanup.emplace_back(entry.segment_id, it->second); + mounted_segment_records_.erase(it); + } } } @@ -1383,13 +1466,20 @@ int RealClient::allocateAndMountSegment( } int RealClient::unmountAndFreeSegment( - const std::vector &segment_ids) { + const std::vector &segment_ids, + uint64_t grace_period_seconds) { if (!client_) { LOG(ERROR) << "Client not initialized"; return -1; } + uint64_t grace_period_ms = grace_period_seconds * 1000; int first_error = 0; + struct SegmentToUnmount { + std::string segment_id; + UUID id; + }; + std::vector to_unmount; std::vector> to_cleanup; { std::lock_guard lock(allocated_segment_records_mutex_); @@ -1409,17 +1499,34 @@ int RealClient::unmountAndFreeSegment( continue; } - auto result = client_->UnmountSegmentById(id); - if (!result.has_value()) { - LOG(ERROR) << "UnmountSegmentById failed for " << segment_id; - if (first_error == 0) { - first_error = static_cast(result.error()); - } - continue; // Don't release local resources on failure - } + to_unmount.push_back({segment_id, id}); + } + } - to_cleanup.emplace_back(segment_id, it->second); - allocated_segment_records_.erase(it); + for (auto &entry : to_unmount) { + std::function cleanup_callback; + if (grace_period_ms != 0) { + cleanup_callback = [this](const UUID &cleanup_id) { + ReleaseAllocatedSegmentRecord(UuidToString(cleanup_id)); + }; + } + auto result = client_->UnmountSegmentById(entry.id, grace_period_ms, + std::move(cleanup_callback)); + if (!result.has_value()) { + LOG(ERROR) << "UnmountSegmentById failed for " << entry.segment_id; + if (first_error == 0) { + first_error = static_cast(result.error()); + } + continue; // Don't release local resources on failure + } + + if (grace_period_ms == 0) { + std::lock_guard lock(allocated_segment_records_mutex_); + auto it = allocated_segment_records_.find(entry.segment_id); + if (it != allocated_segment_records_.end()) { + to_cleanup.emplace_back(entry.segment_id, it->second); + allocated_segment_records_.erase(it); + } } } diff --git a/mooncake-store/src/rpc_service.cpp b/mooncake-store/src/rpc_service.cpp index 05a59af1..63ac3480 100644 --- a/mooncake-store/src/rpc_service.cpp +++ b/mooncake-store/src/rpc_service.cpp @@ -1375,6 +1375,29 @@ tl::expected WrappedMasterService::UnmountSegment( [] { MasterMetricManager::instance().inc_unmount_segment_failures(); }); } +tl::expected WrappedMasterService::GracefulUnmountSegment( + const UUID& segment_id, const UUID& client_id, uint64_t grace_period_ms) { + return execute_rpc( + "GracefulUnmountSegment", + [&] { + return master_service_.GracefulUnmountSegment(segment_id, client_id, + grace_period_ms); + }, + [&](auto& timer) { + timer.LogRequest("segment_id=", segment_id, + ", client_id=", client_id, + ", grace_period_ms=", grace_period_ms); + }, + [] { + MasterMetricManager::instance() + .inc_unmount_segment_requests(); // reuse metric or add new + }, + [] { + MasterMetricManager::instance() + .inc_unmount_segment_failures(); // reuse metric or add new + }); +} + tl::expected WrappedMasterService::CopyStart( const UUID& client_id, const std::string& key, const std::string& src_segment, @@ -1692,6 +1715,11 @@ tl::expected WrappedMasterService::QuerySegmentStatus( return master_service_.QuerySegmentStatus(segment_name); } +tl::expected +WrappedMasterService::QuerySegmentStatusById(const UUID& segment_id) { + return master_service_.QuerySegmentStatusById(segment_id); +} + void RegisterRpcService( coro_rpc::coro_rpc_server& server, mooncake::WrappedMasterService& wrapped_master_service) { @@ -1747,10 +1775,19 @@ void RegisterRpcService( &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::UnmountSegment>( &wrapped_master_service); + server.register_handler< + &mooncake::WrappedMasterService::GracefulUnmountSegment>( + &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::Ping>( &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::GetFsdir>( &wrapped_master_service); + server + .register_handler<&mooncake::WrappedMasterService::QuerySegmentStatus>( + &wrapped_master_service); + server.register_handler< + &mooncake::WrappedMasterService::QuerySegmentStatusById>( + &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::GetStorageConfig>( &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::BatchExistKey>( diff --git a/mooncake-store/src/segment.cpp b/mooncake-store/src/segment.cpp index eaaebd21..12571436 100644 --- a/mooncake-store/src/segment.cpp +++ b/mooncake-store/src/segment.cpp @@ -211,6 +211,47 @@ ErrorCode ScopedSegmentAccess::PrepareUnmountSegment( return ErrorCode::OK; } +ErrorCode ScopedSegmentAccess::PrepareGracefulUnmountSegment( + const UUID& segment_id) { + auto it = segment_manager_->mounted_segments_.find(segment_id); + if (it == segment_manager_->mounted_segments_.end()) { + LOG(WARNING) << "segment_id=" << segment_id + << ", warn=segment_not_found"; + return ErrorCode::SEGMENT_NOT_FOUND; + } + auto status = it->second.status; + if (status == SegmentStatus::UNMOUNTING) { + LOG(ERROR) << "segment_id=" << segment_id + << ", error=segment_is_unmounting"; + return ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS; + } + if (status == SegmentStatus::GRACEFULLY_UNMOUNTING) { + // Idempotent: already in graceful unmount state + return ErrorCode::OK; + } + if (status != SegmentStatus::OK && status != SegmentStatus::DRAINING) { + LOG(ERROR) << "segment_id=" << segment_id + << ", error=unavailable_in_current_status, status=" + << status; + return ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS; + } + + auto& mounted_segment = it->second; + auto& segment = mounted_segment.segment; + + // Remove the allocator from the segment manager + std::shared_ptr allocator = + mounted_segment.buf_allocator; + if (HasAllocator(segment_manager_->allocator_manager_, segment.name, + allocator)) { + segment_manager_->allocator_manager_.removeAllocator(segment.name, + allocator); + } + // Set the segment status to GRACEFULLY_UNMOUNTING + mounted_segment.status = SegmentStatus::GRACEFULLY_UNMOUNTING; + return ErrorCode::OK; +} + ErrorCode ScopedSegmentAccess::CommitUnmountSegment( const UUID& segment_id, const UUID& client_id, const size_t& metrics_dec_capacity) { @@ -240,9 +281,14 @@ ErrorCode ScopedSegmentAccess::CommitUnmountSegment( auto&& segment = segment_manager_->mounted_segments_.find(segment_id); if (segment != segment_manager_->mounted_segments_.end()) { segment_name = segment->second.segment.name; - // Also remove from segment_name_client_id_map_ - segment_manager_->client_by_name_.erase(segment_name); - segment_manager_->segment_id_by_name_.erase(segment_name); + auto segment_id_by_name_it = + segment_manager_->segment_id_by_name_.find(segment_name); + if (segment_id_by_name_it != + segment_manager_->segment_id_by_name_.end() && + segment_id_by_name_it->second == segment_id) { + segment_manager_->segment_id_by_name_.erase(segment_id_by_name_it); + segment_manager_->client_by_name_.erase(segment_name); + } is_cxl = (segment->second.segment.protocol == "cxl"); } // Remove from mounted_segments_ @@ -954,6 +1000,17 @@ ErrorCode ScopedSegmentAccess::GetSegmentStatusByName( return ErrorCode::OK; } +ErrorCode ScopedSegmentAccess::GetSegmentStatusById( + const UUID& segment_id, SegmentStatus& status) const { + auto mounted_segment_it = + segment_manager_->mounted_segments_.find(segment_id); + if (mounted_segment_it == segment_manager_->mounted_segments_.end()) { + return ErrorCode::SEGMENT_NOT_FOUND; + } + status = mounted_segment_it->second.status; + return ErrorCode::OK; +} + ErrorCode ScopedSegmentAccess::SetSegmentStatusByName( const std::string& segment_name, SegmentStatus status) { auto segment_id_it = diff --git a/mooncake-store/tests/master_service_test.cpp b/mooncake-store/tests/master_service_test.cpp index 91cb5144..af5503a6 100644 --- a/mooncake-store/tests/master_service_test.cpp +++ b/mooncake-store/tests/master_service_test.cpp @@ -5159,6 +5159,236 @@ TEST_F(MasterServiceTest, HardPinDefaultIsFalse) { service_->RemoveAll(); } +// ===================== Graceful Unmount Tests ===================== + +TEST_F(MasterServiceTest, GracefulUnmountSegment_SetsCorrectStatus) { + std::unique_ptr service_(new MasterService()); + auto segment = MakeSegment("graceful_test_segment"); + UUID client_id = generate_uuid(); + + // Mount segment + ASSERT_TRUE(service_->MountSegment(segment, client_id).has_value()); + + // Verify initial status + auto status_before = service_->QuerySegmentStatus(segment.name); + ASSERT_TRUE(status_before.has_value()); + EXPECT_EQ(status_before.value(), SegmentStatus::OK); + + // Graceful unmount with 1 second grace period + auto graceful_result = service_->GracefulUnmountSegment( + segment.id, client_id, /*grace_period_ms=*/1000); + ASSERT_TRUE(graceful_result.has_value()) + << "Graceful unmount should succeed: " + << toString(graceful_result.error()); + + // Verify status is GRACEFULLY_UNMOUNTING + auto status_after = service_->QuerySegmentStatus(segment.name); + ASSERT_TRUE(status_after.has_value()); + EXPECT_EQ(status_after.value(), SegmentStatus::GRACEFULLY_UNMOUNTING); + + // Wait for timer to expire and clean up + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); +} + +TEST_F(MasterServiceTest, GracefulUnmountSegment_RejectWrongClient) { + std::unique_ptr service_(new MasterService()); + auto segment = MakeSegment("graceful_owner_segment"); + UUID owner_client = generate_uuid(); + UUID wrong_client = generate_uuid(); + + ASSERT_TRUE(service_->MountSegment(segment, owner_client).has_value()); + + // Wrong client trying to graceful unmount should fail + auto graceful_result = service_->GracefulUnmountSegment( + segment.id, wrong_client, /*grace_period_ms=*/1000); + ASSERT_FALSE(graceful_result.has_value()); + EXPECT_EQ(graceful_result.error(), ErrorCode::SEGMENT_NOT_FOUND); + + // Owner should still be able to unmount + auto owner_result = service_->GracefulUnmountSegment( + segment.id, owner_client, /*grace_period_ms=*/1000); + EXPECT_TRUE(owner_result.has_value()); + + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); +} + +TEST_F(MasterServiceTest, GracefulUnmountSegment_Idempotent) { + std::unique_ptr service_(new MasterService()); + auto segment = MakeSegment("graceful_idempotent_segment"); + UUID client_id = generate_uuid(); + + ASSERT_TRUE(service_->MountSegment(segment, client_id).has_value()); + + // First graceful unmount should succeed + auto result1 = service_->GracefulUnmountSegment(segment.id, client_id, + /*grace_period_ms=*/1000); + ASSERT_TRUE(result1.has_value()); + + // Second graceful unmount on the same segment should also succeed + // (idempotent) + auto result2 = service_->GracefulUnmountSegment(segment.id, client_id, + /*grace_period_ms=*/1000); + EXPECT_TRUE(result2.has_value()) << "Graceful unmount should be idempotent"; + + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); +} + +TEST_F(MasterServiceTest, GracefulUnmountSegment_TimerExpiresAndUnmounts) { + std::unique_ptr service_(new MasterService()); + auto segment = MakeSegment("graceful_timer_segment"); + UUID client_id = generate_uuid(); + + ASSERT_TRUE(service_->MountSegment(segment, client_id).has_value()); + + // Graceful unmount with a short grace period (50ms) + auto graceful_result = service_->GracefulUnmountSegment( + segment.id, client_id, /*grace_period_ms=*/50); + ASSERT_TRUE(graceful_result.has_value()); + + // Immediately after graceful unmount, segment should still exist + auto status_immediate = service_->QuerySegmentStatus(segment.name); + ASSERT_TRUE(status_immediate.has_value()); + EXPECT_EQ(status_immediate.value(), SegmentStatus::GRACEFULLY_UNMOUNTING); + + // Wait for timer to expire and unmount (give some margin) + std::this_thread::sleep_for(std::chrono::milliseconds(300)); + + // After timer expires, segment should be fully unmounted (UNDEFINED or + // error) + auto status_after = service_->QuerySegmentStatus(segment.name); + // Segment may be UNDEFINED (not found) or return an error + EXPECT_TRUE(!status_after.has_value() || + status_after.value() == SegmentStatus::UNDEFINED) + << "Segment should be unmounted after timer expires, got status=" + << (status_after.has_value() ? static_cast(status_after.value()) + : -1); +} + +TEST_F(MasterServiceTest, + GracefulUnmountSegment_QueryStatusByIdWithReusedName) { + std::unique_ptr service_(new MasterService()); + auto old_segment = MakeSegment("graceful_reused_name_segment"); + auto new_segment = MakeSegment(old_segment.name, /*base=*/0x400000000); + UUID client_id = generate_uuid(); + + ASSERT_TRUE(service_->MountSegment(old_segment, client_id).has_value()); + ASSERT_TRUE(service_ + ->GracefulUnmountSegment(old_segment.id, client_id, + /*grace_period_ms=*/50) + .has_value()); + ASSERT_TRUE(service_->MountSegment(new_segment, client_id).has_value()); + + auto old_status = service_->QuerySegmentStatusById(old_segment.id); + ASSERT_TRUE(old_status.has_value()); + EXPECT_EQ(old_status.value(), SegmentStatus::GRACEFULLY_UNMOUNTING); + + auto new_status = service_->QuerySegmentStatusById(new_segment.id); + ASSERT_TRUE(new_status.has_value()); + EXPECT_EQ(new_status.value(), SegmentStatus::OK); + + std::this_thread::sleep_for(std::chrono::milliseconds(300)); + + EXPECT_FALSE(service_->QuerySegmentStatusById(old_segment.id).has_value()); + ASSERT_TRUE(service_->QuerySegmentStatusById(new_segment.id).has_value()); + + auto status_by_name = service_->QuerySegmentStatus(new_segment.name); + ASSERT_TRUE(status_by_name.has_value()); + EXPECT_EQ(status_by_name.value(), SegmentStatus::OK); +} + +TEST_F(MasterServiceTest, GracefulUnmountSegment_EarlierTimerPreemptsWait) { + std::unique_ptr service_(new MasterService()); + auto long_segment = MakeSegment("graceful_long_timer_segment"); + auto short_segment = + MakeSegment("graceful_short_timer_segment", /*base=*/0x400000000); + UUID client_id = generate_uuid(); + + ASSERT_TRUE(service_->MountSegment(long_segment, client_id).has_value()); + ASSERT_TRUE(service_->MountSegment(short_segment, client_id).has_value()); + + ASSERT_TRUE(service_ + ->GracefulUnmountSegment(long_segment.id, client_id, + /*grace_period_ms=*/1000) + .has_value()); + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + ASSERT_TRUE(service_ + ->GracefulUnmountSegment(short_segment.id, client_id, + /*grace_period_ms=*/50) + .has_value()); + + std::this_thread::sleep_for(std::chrono::milliseconds(300)); + + auto short_status = service_->QuerySegmentStatus(short_segment.name); + EXPECT_TRUE(!short_status.has_value() || + short_status.value() == SegmentStatus::UNDEFINED); + + auto long_status = service_->QuerySegmentStatus(long_segment.name); + ASSERT_TRUE(long_status.has_value()); + EXPECT_EQ(long_status.value(), SegmentStatus::GRACEFULLY_UNMOUNTING); +} + +TEST_F(MasterServiceTest, GracefulUnmountSegment_PreventAllocation) { + std::unique_ptr service_(new MasterService()); + auto segment1 = MakeSegment("graceful_seg1"); + auto segment2 = MakeSegment("graceful_seg2", /*base=*/0x400000000); + UUID client_id = generate_uuid(); + + ASSERT_TRUE(service_->MountSegment(segment1, client_id).has_value()); + ASSERT_TRUE(service_->MountSegment(segment2, client_id).has_value()); + + // Put an object on segment1 + std::string key = "test_key_prevent_alloc"; + ReplicateConfig config; + config.replica_num = 1; + config.preferred_segment = segment1.name; + + auto put_start = service_->PutStart(client_id, key, 1024, config); + ASSERT_TRUE(put_start.has_value()); + ASSERT_TRUE( + service_->PutEnd(client_id, key, ReplicaType::MEMORY).has_value()); + + // Graceful unmount segment1 + ASSERT_TRUE(service_->GracefulUnmountSegment(segment1.id, client_id, 1000) + .has_value()); + + // Segment1 status should be GRACEFULLY_UNMOUNTING + auto status1 = service_->QuerySegmentStatus(segment1.name); + ASSERT_TRUE(status1.has_value()); + EXPECT_EQ(status1.value(), SegmentStatus::GRACEFULLY_UNMOUNTING); + + // Existing replicas on the graceful segment should remain readable during + // the grace window. + auto existing_replicas = service_->GetReplicaList(key); + ASSERT_TRUE(existing_replicas.has_value()); + ASSERT_EQ(existing_replicas->replicas.size(), 1u); + EXPECT_EQ(existing_replicas->replicas[0] + .get_memory_descriptor() + .buffer_descriptor.transport_endpoint_, + segment1.name); + + // Segment2 status should still be OK + auto status2 = service_->QuerySegmentStatus(segment2.name); + ASSERT_TRUE(status2.has_value()); + EXPECT_EQ(status2.value(), SegmentStatus::OK); + + // New put without preferred_segment should succeed on segment2 + std::string key2 = "test_key_after_graceful"; + ReplicateConfig config2; + config2.replica_num = 1; + + auto put_start2 = service_->PutStart(client_id, key2, 1024, config2); + ASSERT_TRUE(put_start2.has_value()); + auto replicas = put_start2.value(); + ASSERT_EQ(replicas.size(), 1u); + // Should be placed on segment2, not segment1 + EXPECT_EQ(replicas[0] + .get_memory_descriptor() + .buffer_descriptor.transport_endpoint_, + segment2.name); + + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); +} + } // namespace mooncake::test int main(int argc, char** argv) { diff --git a/mooncake-store/tests/pybind_client_test.cpp b/mooncake-store/tests/pybind_client_test.cpp index 10e32a4a..dec33176 100644 --- a/mooncake-store/tests/pybind_client_test.cpp +++ b/mooncake-store/tests/pybind_client_test.cpp @@ -184,6 +184,33 @@ TEST_F(RealClientTest, MountAndAllocateUnmountApisRejectForeignSegments) { EXPECT_EQ(std::remove(path.c_str()), 0); } +TEST_F(RealClientTest, MountAndAllocateUnmountApisAcceptGracePeriod) { + StartMasterAndSetupClient(); + + const size_t slab_size = facebook::cachelib::Slab::kSize; + std::vector allocated_segment_ids; + size_t allocated_size = 0; + ASSERT_EQ( + py_client_->allocateAndMountSegment( + 1, FLAGS_protocol, "", allocated_segment_ids, &allocated_size), + 0); + ASSERT_FALSE(allocated_segment_ids.empty()); + EXPECT_EQ(py_client_->unmountAndFreeSegment(allocated_segment_ids, 1), 0); + + std::string path = CreateTempSegmentFile(slab_size); + ASSERT_FALSE(path.empty()); + + std::vector mounted_segment_ids; + ASSERT_EQ(py_client_->mountSegment(path, 0, slab_size, FLAGS_protocol, "", + mounted_segment_ids), + 0); + ASSERT_FALSE(mounted_segment_ids.empty()); + EXPECT_EQ(py_client_->unmountSegment(mounted_segment_ids, 1), 0); + + EXPECT_EQ(py_client_->tearDownAll(), 0); + EXPECT_EQ(std::remove(path.c_str()), 0); +} + // Test basic Put and Get operations TEST_F(RealClientTest, BasicPutGetOperations) { // Start in-proc master diff --git a/mooncake-wheel/mooncake/mooncake_store_service.py b/mooncake-wheel/mooncake/mooncake_store_service.py index 7191ec8a..ca5ddcbf 100644 --- a/mooncake-wheel/mooncake/mooncake_store_service.py +++ b/mooncake-wheel/mooncake/mooncake_store_service.py @@ -336,10 +336,11 @@ class MooncakeStoreService: content_type="application/json", ) + grace_period_seconds = data.get("grace_period_seconds", 0) failed_segment_ids = [] async with self._state_lock: for sid in segment_ids: - ret = self.store.unmount_segment([sid]) + ret = self.store.unmount_segment([sid], grace_period_seconds) if ret != 0: failed_segment_ids.append(sid) continue @@ -427,7 +428,10 @@ class MooncakeStoreService: content_type="application/json", ) - ret = self.store.unmount_and_free_segment(segment_ids) + grace_period_seconds = data.get("grace_period_seconds", 0) + ret = self.store.unmount_and_free_segment( + segment_ids, grace_period_seconds + ) if ret != 0: return web.Response( status=500, diff --git a/mooncake-wheel/tests/test_mooncake_store_service_api.py b/mooncake-wheel/tests/test_mooncake_store_service_api.py index e20544cd..58a2940d 100644 --- a/mooncake-wheel/tests/test_mooncake_store_service_api.py +++ b/mooncake-wheel/tests/test_mooncake_store_service_api.py @@ -64,8 +64,8 @@ class FakeStore: } return {"ret": 0, "segment_ids": [segment_id]} - def unmount_segment(self, segment_ids): - self.unmount_calls.append(list(segment_ids)) + def unmount_segment(self, segment_ids, grace_period_seconds=0): + self.unmount_calls.append((list(segment_ids), grace_period_seconds)) for segment_id in segment_ids: if segment_id in self.unmount_failures: return -1 @@ -82,8 +82,8 @@ class FakeStore: "allocated_size": 4096, } - def unmount_and_free_segment(self, segment_ids): - self.free_unmount_calls.append(list(segment_ids)) + def unmount_and_free_segment(self, segment_ids, grace_period_seconds=0): + self.free_unmount_calls.append((list(segment_ids), grace_period_seconds)) return 0 @@ -138,10 +138,24 @@ class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(unmount_body["status"], "success") self.assertEqual( self.fake_store.unmount_calls, - [["00000000-0000-0000-0000-000000000001"]], + [(["00000000-0000-0000-0000-000000000001"], 0)], ) self.assertEqual(self.service.current_mode, "prefill") + async def test_unmount_shm_passes_grace_period(self): + segment_id = "00000000-0000-0000-0000-000000000001" + self.service.current_mode = "decode" + self.service.mounted_segment_ids = [segment_id] + + resp = await self.service.handle_unmount_shm( + FakeRequest({"segment_ids": [segment_id], "grace_period_seconds": 3}) + ) + + self.assertEqual(resp.status, 200) + self.assertEqual(self.fake_store.unmount_calls, [([segment_id], 3)]) + self.assertEqual(self.service.mounted_segment_ids, []) + self.assertEqual(self.service.current_mode, "prefill") + async def test_unmount_shm_updates_state_for_partial_success(self): succeeded_id = "00000000-0000-0000-0000-000000000001" failed_id = "00000000-0000-0000-0000-000000000002" @@ -158,7 +172,7 @@ class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(body["failed_segment_ids"], [failed_id]) self.assertEqual( self.fake_store.unmount_calls, - [[succeeded_id], [failed_id]], + [([succeeded_id], 0), ([failed_id], 0)], ) self.assertEqual(self.service.mounted_segment_ids, [failed_id]) self.assertEqual(self.service.current_mode, "decode") @@ -184,7 +198,7 @@ class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): body = json.loads(resp.text) self.assertEqual(body["mode"], "prefill") self.assertIn("rolled back to prefill", body["error"]) - self.assertEqual(self.fake_store.unmount_calls, [[old_id]]) + self.assertEqual(self.fake_store.unmount_calls, [([old_id], 0)]) self.assertEqual(self.service.mounted_segment_ids, []) self.assertEqual(self.service.current_mode, "prefill") self.assertEqual(self.service.last_mount_info, {}) @@ -211,9 +225,19 @@ class StoreServiceApiTest(unittest.IsolatedAsyncioTestCase): self.assertEqual(unmount_body["status"], "success") self.assertEqual( self.fake_store.free_unmount_calls, - [["00000000-0000-0000-0000-000000000002"]], + [(["00000000-0000-0000-0000-000000000002"], 0)], ) + async def test_unmount_allocated_passes_grace_period(self): + segment_id = "00000000-0000-0000-0000-000000000002" + + resp = await self.service.handle_unmount( + FakeRequest({"segment_ids": [segment_id], "grace_period_seconds": 4}) + ) + + self.assertEqual(resp.status, 200) + self.assertEqual(self.fake_store.free_unmount_calls, [([segment_id], 4)]) + async def test_mount_shm_requires_name_and_size(self): resp = await self.service.handle_mount_shm( FakeRequest({"name": "mooncake-segment"}) -- 2.34.1 From 53480ac17620a20c15e71543afe952c43c78c570 Mon Sep 17 00:00:00 2001 From: Stary Date: Sat, 16 May 2026 19:39:34 +0800 Subject: [PATCH 083/382] [TENT] Fix batch getTransferStatus premature FAILED aggregation (#2055) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [TENT] Fix batch getTransferStatus premature FAILED aggregation Previously, one permanently-FAILED task would latch overall_status to FAILED even while other tasks were still PENDING (mid-failover). This caused lazyFreeBatch to teardown the batch while retries were in-flight. Now the batch reports FAILED only when ALL tasks reach a terminal state (success_tasks + failed_tasks == total_tasks). A task still in PENDING (e.g. resubmitted on a secondary transport) keeps the batch in PENDING. Add unit tests covering the new aggregation logic: FAILED+PENDING → PENDING, FAILED+COMPLETED → FAILED, all COMPLETED → COMPLETED, and derived-task skipping. Signed-off-by: Yuxin Chen * [TENT] Replace hardcoded transport type tests with dynamic sentinel check The old AllEnumValuesDistinct and SupportedCount tests hardcoded the enum list and expected count, breaking whenever a new transport type (like SUNRISE_LINK) was added. Replace them with UnspecIsSentinel which verifies invariants independent of how many transports exist. Signed-off-by: Yuxin Chen * style(test): reformat lambda expression in failover_test - Adjust line break for lambda assignment to improve readability Signed-off-by: Yuxin Chen * fix(transfer-engine): correct worst failure tracking in getTransferStatus - Introduce severity-based comparison to ensure `worst_failure` reflects the most severe status, preventing overwrites with lower severity. Signed-off-by: Yuxin Chen --------- Signed-off-by: Yuxin Chen Co-authored-by: Yuxin Chen --- .../tent/src/runtime/transfer_engine_impl.cpp | 32 +++++- .../tent/tests/failover_test.cpp | 107 +++++++++++++++--- 2 files changed, 118 insertions(+), 21 deletions(-) diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index 869ff38e..f073ea35 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -1403,7 +1403,16 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, overall_status.s = PENDING; overall_status.transferred_bytes = 0; size_t success_tasks = 0; + size_t failed_tasks = 0; size_t total_tasks = 0; + TransferStatusEnum worst_failure = PENDING; + auto isWorse = [](TransferStatusEnum cur, TransferStatusEnum best) { + static const std::unordered_map severity = { + {INITIAL, 0}, {PENDING, 0}, {COMPLETED, 0}, {INVALID, 1}, + {CANCELED, 2}, {TIMEOUT, 3}, {FAILED, 4}, + }; + return severity.at(cur) > severity.at(best); + }; for (size_t task_id = 0; task_id < batch->task_list.size(); ++task_id) { auto& task = batch->task_list[task_id]; if (task.derived) continue; // This task is performed by other tasks @@ -1414,7 +1423,9 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, success_tasks++; overall_status.transferred_bytes += task.request.length; } else { - overall_status.s = task.status; + failed_tasks++; + if (isWorse(task.status, worst_failure)) + worst_failure = task.status; } continue; } @@ -1424,7 +1435,8 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, } else { if (task.type == UNSPEC) { task.status = FAILED; - overall_status.s = FAILED; + failed_tasks++; + if (isWorse(FAILED, worst_failure)) worst_failure = FAILED; continue; } auto& transport = transport_list_[task.type]; @@ -1441,7 +1453,7 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, // Attempt failover before status aggregation so that a // successfully resubmitted task appears as PENDING and does not - // overwrite a permanent FAILED from another task in the batch. + // latch the batch to a terminal state while retries are in-flight. if (task_status.s == FAILED && resubmitTransferTask(batch, task_id).ok()) { task.status = PENDING; @@ -1452,14 +1464,24 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, success_tasks++; overall_status.transferred_bytes += task_status.transferred_bytes; } else if (task_status.s != PENDING) { - overall_status.s = task_status.s; + failed_tasks++; + if (isWorse(task_status.s, worst_failure)) + worst_failure = task_status.s; } // Record metrics when task transitions to terminal state recordTaskCompletionMetrics(batch->task_list[task_id], prev_status, task_status.s); } - if (success_tasks == total_tasks) overall_status.s = COMPLETED; + // Determine overall status: COMPLETED only when all succeed; FAILED only + // when all tasks are terminal (no in-flight work) and at least one failed; + // otherwise PENDING (some tasks still running). + if (success_tasks == total_tasks) { + overall_status.s = COMPLETED; + } else if (success_tasks + failed_tasks == total_tasks) { + overall_status.s = worst_failure; + } + // else: some tasks still PENDING → overall_status.s stays PENDING CHECK_STATUS(maybeFireSubmitHooks(batch, overall_status.s == COMPLETED)); return Status::OK(); } diff --git a/mooncake-transfer-engine/tent/tests/failover_test.cpp b/mooncake-transfer-engine/tent/tests/failover_test.cpp index 8e58ef04..881168dd 100644 --- a/mooncake-transfer-engine/tent/tests/failover_test.cpp +++ b/mooncake-transfer-engine/tent/tests/failover_test.cpp @@ -15,7 +15,6 @@ #include #include -#include #include #include "tent/common/config.h" @@ -118,21 +117,11 @@ TEST(FailoverConfigTest, ZeroDisablesFailover) { // verify the enum values are well-defined and usable in switch) // --------------------------------------------------------------------------- -TEST(TransportTypeTest, AllEnumValuesDistinct) { - // Verify no accidental collisions in the enum - std::set seen; - TransportType types[] = {RDMA, MNNVL, SHM, NVLINK, GDS, - IOURING, TCP, AscendDirect, UNSPEC}; - for (auto t : types) { - EXPECT_TRUE(seen.insert(static_cast(t)).second) - << "Duplicate TransportType value: " << static_cast(t); - } -} - -TEST(TransportTypeTest, SupportedCount) { - // kSupportedTransportTypes should cover all types except UNSPEC - EXPECT_EQ(kSupportedTransportTypes, 8u); - EXPECT_EQ(static_cast(UNSPEC), 8); +TEST(TransportTypeTest, UnspecIsSentinel) { + // UNSPEC must be the last enum value so that types [0, UNSPEC) are the + // valid transport types and kSupportedTransportTypes == (int)UNSPEC. + EXPECT_GT(kSupportedTransportTypes, 0); + EXPECT_EQ(kSupportedTransportTypes, static_cast(UNSPEC)); } // --------------------------------------------------------------------------- @@ -214,6 +203,92 @@ TEST(FailoverStateMachineTest, StagingBypassesPriorityIncrement) { EXPECT_EQ(task.xport_priority, 0); // Should NOT have incremented } +// --------------------------------------------------------------------------- +// Batch status aggregation correctness +// --------------------------------------------------------------------------- + +// Verifies the invariant: a batch with one permanently-FAILED task and +// another still-PENDING task must report PENDING (not FAILED) overall, +// because the PENDING task may still complete. The old code latched +// overall_status to FAILED as soon as any task was terminal. +TEST(BatchStatusAggregationTest, PendingTaskPreventsEarlyBatchFailure) { + // Two tasks in a batch scenario (simulated): + // Task A: permanently FAILED (exhausted failover budget) + // Task B: still PENDING (retrying on secondary transport) + // + // Expected overall status: PENDING (not FAILED), because Task B is + // still in-flight. Only once Task B reaches a terminal state should + // the batch become terminal. + + struct MockTask { + TransferStatusEnum status{PENDING}; + bool derived{false}; + size_t length{1024}; + }; + + auto aggregateStatus = + [](const std::vector& tasks) -> TransferStatusEnum { + size_t success_tasks = 0; + size_t failed_tasks = 0; + size_t total_tasks = 0; + for (auto& t : tasks) { + if (t.derived) continue; + total_tasks++; + if (t.status == COMPLETED) + success_tasks++; + else if (t.status != PENDING) + failed_tasks++; + } + if (success_tasks == total_tasks) return COMPLETED; + if (success_tasks + failed_tasks == total_tasks) return FAILED; + return PENDING; + }; + + // Case 1: one FAILED + one PENDING → overall PENDING + { + std::vector tasks = {{FAILED, false, 1024}, + {PENDING, false, 1024}}; + EXPECT_EQ(aggregateStatus(tasks), PENDING); + } + + // Case 2: one FAILED + one COMPLETED → overall FAILED + { + std::vector tasks = {{FAILED, false, 1024}, + {COMPLETED, false, 1024}}; + EXPECT_EQ(aggregateStatus(tasks), FAILED); + } + + // Case 3: all COMPLETED → overall COMPLETED + { + std::vector tasks = {{COMPLETED, false, 1024}, + {COMPLETED, false, 1024}}; + EXPECT_EQ(aggregateStatus(tasks), COMPLETED); + } + + // Case 4: all PENDING → overall PENDING + { + std::vector tasks = {{PENDING, false, 1024}, + {PENDING, false, 1024}}; + EXPECT_EQ(aggregateStatus(tasks), PENDING); + } + + // Case 5: derived tasks are skipped + { + std::vector tasks = {{FAILED, false, 1024}, + {FAILED, true, 1024}, // derived: skip + {PENDING, false, 1024}}; + EXPECT_EQ(aggregateStatus(tasks), PENDING); + } + + // Case 6: all non-derived are terminal with at least one failure + { + std::vector tasks = {{COMPLETED, false, 1024}, + {FAILED, true, 1024}, // derived: skip + {FAILED, false, 1024}}; + EXPECT_EQ(aggregateStatus(tasks), FAILED); + } +} + } // namespace } // namespace tent } // namespace mooncake -- 2.34.1 From d59458172a51674f4414bd702c311a0ba98ea0db Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Sat, 16 May 2026 22:52:36 +0800 Subject: [PATCH 084/382] [Store] fix: mooncake_master -version prints release version and git commit hash (#2110) --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> --- mooncake-store/CMakeLists.txt | 28 ++++++++++++++++++++++++++++ mooncake-store/include/version.h.in | 7 ++++++- mooncake-store/src/master.cpp | 2 ++ 3 files changed, 36 insertions(+), 1 deletion(-) diff --git a/mooncake-store/CMakeLists.txt b/mooncake-store/CMakeLists.txt index f01e657a..5cf0f8a4 100644 --- a/mooncake-store/CMakeLists.txt +++ b/mooncake-store/CMakeLists.txt @@ -3,6 +3,34 @@ project(MooncakeStore VERSION 2.0.0) # Extract version components for C++ usage set(MOONCAKE_STORE_VERSION ${PROJECT_VERSION}) +# Read display version from mooncake-wheel/pyproject.toml +set(PYPROJECT_TOML_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/pyproject.toml") +if(EXISTS "${PYPROJECT_TOML_PATH}") + file(READ "${PYPROJECT_TOML_PATH}" PYPROJECT_CONTENT) + string(REGEX MATCH "version = \"([^\"]+)\"" _ "${PYPROJECT_CONTENT}") + set(MOONCAKE_DISPLAY_VERSION_BASE "${CMAKE_MATCH_1}") +endif() +if(NOT MOONCAKE_DISPLAY_VERSION_BASE) + message(WARNING "Could not read display version from ${PYPROJECT_TOML_PATH}; falling back to MOONCAKE_STORE_VERSION") + set(MOONCAKE_DISPLAY_VERSION_BASE "${MOONCAKE_STORE_VERSION}") +endif() + +# Get short git commit hash for display +find_package(Git QUIET) +if(GIT_FOUND) + execute_process( + COMMAND ${GIT_EXECUTABLE} rev-parse --short HEAD + WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} + OUTPUT_VARIABLE GIT_SHORT_HASH + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_QUIET + ) +endif() +if(NOT GIT_SHORT_HASH) + set(GIT_SHORT_HASH "unknown") +endif() +set(MOONCAKE_DISPLAY_VERSION "${MOONCAKE_DISPLAY_VERSION_BASE} (git: ${GIT_SHORT_HASH})") + configure_file(include/version.h.in include/version.h @ONLY) if (STORE_USE_ETCD) diff --git a/mooncake-store/include/version.h.in b/mooncake-store/include/version.h.in index 5fca0b53..8c27b0f6 100644 --- a/mooncake-store/include/version.h.in +++ b/mooncake-store/include/version.h.in @@ -4,7 +4,7 @@ namespace mooncake { -// Version information for the Mooncake system +// Version information for the Mooncake system (used for RPC handshake) constexpr const char* MOONCAKE_STORE_VERSION = "@MOONCAKE_STORE_VERSION@"; // Function to get the version string @@ -13,4 +13,9 @@ inline const std::string& GetMooncakeStoreVersion() { return version; } +// Human-readable display version: sourced from pyproject.toml + short git hash +// e.g. "0.3.10.post2 (git: abc1234)" +// Used for the --version flag only; does not affect RPC compatibility. +constexpr const char* MOONCAKE_DISPLAY_VERSION = "@MOONCAKE_DISPLAY_VERSION@"; + } // namespace mooncake \ No newline at end of file diff --git a/mooncake-store/src/master.cpp b/mooncake-store/src/master.cpp index ec4dab87..b0551d42 100644 --- a/mooncake-store/src/master.cpp +++ b/mooncake-store/src/master.cpp @@ -17,6 +17,7 @@ #include "utils.h" #include "master_config.h" +#include "version.h" using namespace coro_rpc; using namespace async_simple; @@ -805,6 +806,7 @@ std::unique_ptr StartHttpMetadataServer( int main(int argc, char* argv[]) { mooncake::init_ylt_log_level(); // Initialize gflags + gflags::SetVersionString(mooncake::MOONCAKE_DISPLAY_VERSION); gflags::ParseCommandLineFlags(&argc, &argv, true); if (!FLAGS_log_dir.empty()) { -- 2.34.1 From 26b24565ffa905c14b50ad918fc47ce5579cdba5 Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Sun, 17 May 2026 09:28:38 +0800 Subject: [PATCH 085/382] [PG] Fix scale-up semantics with two-phase extension protocol (#1968) Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> --- .github/workflows/ci.yml | 2 +- mooncake-pg/include/connection_poller.h | 14 + mooncake-pg/include/mooncake_backend.h | 11 +- mooncake-pg/include/mooncake_worker.cuh | 3 +- mooncake-pg/src/connection_poller.cpp | 65 ++- mooncake-pg/src/mooncake_backend.cpp | 99 +++- mooncake-pg/src/pg_py.cpp | 4 +- mooncake-pg/tests/pg_test_utils.py | 17 +- mooncake-pg/tests/test_pg_elastic.py | 548 +++++++++++++++++- mooncake-wheel/tests/test_mooncake_backend.py | 176 ------ .../tests/test_mooncake_backend_chunk.py | 64 -- .../tests/test_mooncake_backend_cpu.py | 148 ----- .../tests/test_mooncake_backend_elastic.py | 168 ------ .../tests/test_mooncake_backend_p2p_cpu.py | 212 ------- .../tests/test_mooncake_backend_perf.py | 107 ---- 15 files changed, 722 insertions(+), 916 deletions(-) delete mode 100644 mooncake-wheel/tests/test_mooncake_backend.py delete mode 100644 mooncake-wheel/tests/test_mooncake_backend_chunk.py delete mode 100644 mooncake-wheel/tests/test_mooncake_backend_cpu.py delete mode 100644 mooncake-wheel/tests/test_mooncake_backend_elastic.py delete mode 100644 mooncake-wheel/tests/test_mooncake_backend_p2p_cpu.py delete mode 100644 mooncake-wheel/tests/test_mooncake_backend_perf.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3450de4c..c5f6e1cb 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -465,7 +465,7 @@ jobs: MC_FORCE_TCP: "true" run: | source test_env/bin/activate - python -m unittest mooncake-wheel.tests.test_mooncake_backend_cpu + python mooncake-pg/tests/test_pg_collectives.py shell: bash - name: Test Safetensor Functions diff --git a/mooncake-pg/include/connection_poller.h b/mooncake-pg/include/connection_poller.h index d5e3c53c..42417b1e 100644 --- a/mooncake-pg/include/connection_poller.h +++ b/mooncake-pg/include/connection_poller.h @@ -32,6 +32,12 @@ struct PeerConnection { std::optional warmupBatchId{std::nullopt}; std::optional segmentId{std::nullopt}; + // Whether this peer has been counted in `totalConnectedPeers_`. + // Note that ConnectionPoller may establish connections for ranks beyond + // the current groupSize_ (when pollingLimit_ > groupSize_). In that case we + // delay counting until the rank officially enters the group. + bool countedInGroup{false}; + // Back off to avoid frequently checking store. std::chrono::steady_clock::time_point last_check_store; size_t check_store_backoff_ms{kCheckStoreInitialBackoffMs}; @@ -56,6 +62,11 @@ class ConnectionContext { std::atomic groupSize_; + // Upper bound of peer polling range. + // This can be larger than groupSize_ so that existing ranks can observe + // joiners without calling extendGroupSizeTo(). + std::atomic pollingLimit_; + bool isDummy_; // A mark tracking the group size for which all ranks @@ -120,6 +131,9 @@ class ConnectionContext { */ void extendGroupSizeTo(int newGroupSize); + // Allow polling ranks beyond groupSize_ without changing groupSize_. + void setPollingLimitTo(int pollingLimit); + /** * @brief Checks whether all peers within the group have * established connections. diff --git a/mooncake-pg/include/mooncake_backend.h b/mooncake-pg/include/mooncake_backend.h index 70fd3503..26058722 100644 --- a/mooncake-pg/include/mooncake_backend.h +++ b/mooncake-pg/include/mooncake_backend.h @@ -61,11 +61,20 @@ class MooncakeBackend final : public ::c10d::ProcessGroup { : activeRanks_{activeRanks} {} MooncakeBackendOptions(at::Tensor activeRanks, bool isExtension) : activeRanks_{activeRanks}, isExtension_{isExtension} {} + MooncakeBackendOptions(at::Tensor activeRanks, bool isExtension, + int maxWorldSize) + : activeRanks_{activeRanks}, + isExtension_{isExtension}, + maxWorldSize_{maxWorldSize} {} ~MooncakeBackendOptions() override = default; at::Tensor activeRanks_; bool isExtension_ = false; + // Optional upper bound for connection polling / reserved rank slots. + // When > 0, the backend may pre-size internal rank metadata to this + // value (while PyTorch's group_size() remains unchanged). + int maxWorldSize_ = -1; }; /** @@ -87,7 +96,7 @@ class MooncakeBackend final : public ::c10d::ProcessGroup { const std::string getBackendName() const override; - int getSize() const override { return meta_ ? meta_->size : size_; } + int getSize() const override { return meta_ ? meta_->activeSize : size_; } // Point-to-point send/recv for torch.distributed P2POp/batch_isend_irecv. // Only single-tensor ops are supported. diff --git a/mooncake-pg/include/mooncake_worker.cuh b/mooncake-pg/include/mooncake_worker.cuh index 853c8df3..b924e2eb 100644 --- a/mooncake-pg/include/mooncake_worker.cuh +++ b/mooncake-pg/include/mooncake_worker.cuh @@ -31,7 +31,8 @@ struct SegmentInfo { struct TransferGroupMeta { int rank; - int size; + int size; // capacity: number of slots allocated (incl. inactive) + int activeSize; // visible group size: number of ranks that participate int taskCount; bool* activeRanks; bool* activeRanksDevice; diff --git a/mooncake-pg/src/connection_poller.cpp b/mooncake-pg/src/connection_poller.cpp index 817ae54e..4b29d73b 100644 --- a/mooncake-pg/src/connection_poller.cpp +++ b/mooncake-pg/src/connection_poller.cpp @@ -51,6 +51,7 @@ ConnectionContext::ConnectionContext(int backendIndex, int rank, int size, : backendIndex_(backendIndex), rank_(rank), groupSize_(size), + pollingLimit_(size), isDummy_(isDummy), establishedGroupSize_(0), local2global_rank_map_(local2global_rank_map), @@ -122,6 +123,18 @@ void ConnectionContext::extendGroupSizeTo(int newGroupSize) { } groupSize_.store(newGroupSize, std::memory_order_release); + // Keep polling range aligned with the largest known size. + pollingLimit_.store(newGroupSize, std::memory_order_release); +} + +void ConnectionContext::setPollingLimitTo(int pollingLimit) { + const int groupSize = groupSize_.load(std::memory_order_acquire); + TORCH_CHECK(pollingLimit >= groupSize, + "pollingLimit must be >= current groupSize"); + TORCH_CHECK( + pollingLimit >= 0 && static_cast(pollingLimit) < kMaxNumRanks, + "Size out of range"); + pollingLimit_.store(pollingLimit, std::memory_order_release); } bool ConnectionContext::isAllPeerConnected() const { @@ -183,6 +196,7 @@ void ConnectionContext::bootstrapLocalPeer(const std::string& localServerName, ConnectionPoller::GetInstance() .global_peerConnected_[local2global_rank_map_[rank_]] = true; peerState.state = PeerConnectionState::CONNECTED; + peerState.countedInGroup = true; { std::lock_guard lock(backend_wakeup_mutex_); @@ -210,7 +224,8 @@ bool ConnectionContext::poll() { bool did_work = false; // Poll all peers sequentially. - for (int pollingRank = 0; pollingRank < groupSize_; ++pollingRank) { + const int pollingLimit = pollingLimit_.load(std::memory_order_acquire); + for (int pollingRank = 0; pollingRank < pollingLimit; ++pollingRank) { did_work |= pollPeer(pollingRank); } @@ -279,9 +294,13 @@ bool ConnectionContext::pollPeer(int pollingRank) { peerState.state = PeerConnectionState::CONNECTED; { std::lock_guard lock(backend_wakeup_mutex_); - totalConnectedPeers_.fetch_add(1, - std::memory_order_release); - backend_wakeup_cv_.notify_all(); + if (pollingRank < + groupSize_.load(std::memory_order_acquire)) { + totalConnectedPeers_.fetch_add( + 1, std::memory_order_release); + peerState.countedInGroup = true; + backend_wakeup_cv_.notify_all(); + } } } else if (pollingRank <= rank_) { // Send a warmup request to establish connections @@ -322,9 +341,13 @@ bool ConnectionContext::pollPeer(int pollingRank) { { std::lock_guard lock(backend_wakeup_mutex_); - totalConnectedPeers_.fetch_add(1, - std::memory_order_release); - backend_wakeup_cv_.notify_all(); + if (pollingRank < + groupSize_.load(std::memory_order_acquire)) { + totalConnectedPeers_.fetch_add( + 1, std::memory_order_release); + peerState.countedInGroup = true; + backend_wakeup_cv_.notify_all(); + } } state_changed = true; } else if (status.s == TransferStatusEnum::FAILED) { @@ -349,9 +372,13 @@ bool ConnectionContext::pollPeer(int pollingRank) { peerState.state = PeerConnectionState::CONNECTED; { std::lock_guard lock(backend_wakeup_mutex_); - totalConnectedPeers_.fetch_add(1, - std::memory_order_release); - backend_wakeup_cv_.notify_all(); + if (pollingRank < + groupSize_.load(std::memory_order_acquire)) { + totalConnectedPeers_.fetch_add( + 1, std::memory_order_release); + peerState.countedInGroup = true; + backend_wakeup_cv_.notify_all(); + } } state_changed = true; } @@ -359,6 +386,11 @@ bool ConnectionContext::pollPeer(int pollingRank) { } case PeerConnectionState::CONNECTED: { + // A peer may be warmed up (CONNECTED) while still outside of the + // current groupSize_. When it later enters the group, we need to + // update totalConnectedPeers_ lazily here; otherwise + // waitUntilAllConnected()/isAllPeerConnected() may hang. + // ATTENTION: Ensure consistency of local (meta_->peerConnected) // and global (global_peerConnected_). // @@ -382,6 +414,14 @@ bool ConnectionContext::pollPeer(int pollingRank) { if (meta_->peerConnected[pollingRank] && global_peerConnected_[globalPollingRank]) { + if (!peerState.countedInGroup && + pollingRank < groupSize_.load(std::memory_order_acquire)) { + std::lock_guard lock(backend_wakeup_mutex_); + totalConnectedPeers_.fetch_add(1, + std::memory_order_release); + peerState.countedInGroup = true; + backend_wakeup_cv_.notify_all(); + } // happy path: both are connected. break; } @@ -418,7 +458,10 @@ bool ConnectionContext::pollPeer(int pollingRank) { peerState.state = PeerConnectionState::WAITING_STORE; engine_->closeSegment(peerState.segmentId.value()); peerState.segmentId = std::nullopt; - totalConnectedPeers_.fetch_sub(1); + if (peerState.countedInGroup) { + totalConnectedPeers_.fetch_sub(1, std::memory_order_release); + peerState.countedInGroup = false; + } state_changed = true; break; } diff --git a/mooncake-pg/src/mooncake_backend.cpp b/mooncake-pg/src/mooncake_backend.cpp index 49f0e16e..27466f42 100644 --- a/mooncake-pg/src/mooncake_backend.cpp +++ b/mooncake-pg/src/mooncake_backend.cpp @@ -181,6 +181,14 @@ MooncakeBackend::MooncakeBackend( auto store = std::move(distBackendOpts.store); const int rank = distBackendOpts.group_rank; const int size = distBackendOpts.group_size; + const int max_size = (options_ && options_->maxWorldSize_ > 0) + ? options_->maxWorldSize_ + : size; + + TORCH_CHECK(max_size >= 0 && static_cast(max_size) <= kMaxNumRanks, + "max_world_size out of range"); + TORCH_CHECK(max_size >= size, + "max_world_size must be >= process group size"); const auto& globalRanks = distBackendOpts.global_ranks_in_group; // Memory location for device specific buffers @@ -214,6 +222,11 @@ MooncakeBackend::MooncakeBackend( } } + // Fill the remaining slots for polling / future joiners. + for (int i = size; i < max_size; ++i) { + local2global_rank_map_[i] = i; + } + // Register buffers if (isCpu) { for (size_t i = 0; i < 2; i++) { @@ -308,6 +321,10 @@ MooncakeBackend::MooncakeBackend( backendIndex_, rank, size, options_ && options_->isExtension_, local2global_rank_map_, store, meta_, p2p_proxy_, engine_); + if (max_size != size) { + connection_ctx_->setPollingLimitTo(max_size); + } + rank_info.send_buffer[0] = (uint64_t)send_buffer_[0]; rank_info.send_buffer[1] = (uint64_t)send_buffer_[1]; rank_info.recv_buffer[0] = (uint64_t)recv_buffer_[0]; @@ -327,7 +344,18 @@ MooncakeBackend::MooncakeBackend( std::vector rank_info_bytes(sizeof(SegmentInfo)); memcpy(rank_info_bytes.data(), &rank_info, sizeof(SegmentInfo)); meta_->rank = rank; - meta_->size = size; + // NOTE: meta_->size is intentionally initialized to max_world_size (when + // provided) so that healthy ranks can activate joiners via recoverRanks() + // without calling extendGroupSizeTo(). Inactive slots are masked by + // meta_->activeRanks / meta_->activeRanksTensor. + meta_->size = max_size; + // activeSize tracks the visible group size (returned by getSize() / + // dist.get_world_size()). It starts at the actual member count and grows + // when extendGroupSizeTo() or recoverRanks() expands the group. + // For extension ranks, activeSize equals world_size (= max_world_size); + // the local-only behavior before joinGroup() is ensured by activeRanks + // masking, not by a smaller activeSize. + meta_->activeSize = size; meta_->taskCount = 0; if (isCpu) { meta_->activeRanks = new bool[kMaxNumRanks]; @@ -340,6 +368,11 @@ MooncakeBackend::MooncakeBackend( for (size_t i = 0; i < kMaxNumRanks; ++i) { meta_->activeRanks[i] = true; } + + // Reserve extra slots as inactive so collectives won't wait on them. + for (int i = size; i < max_size; ++i) { + meta_->activeRanks[i] = false; + } if (options_ && options_->activeRanks_.defined()) { TORCH_CHECK(options_->activeRanks_.dtype() == at::kInt, "activeRanks must be int."); @@ -350,11 +383,19 @@ MooncakeBackend::MooncakeBackend( TORCH_CHECK(options_->activeRanks_.device().is_cuda(), "activeRanks must be on CUDA."); } + if (max_size != size) { + TORCH_CHECK(options_->activeRanks_.numel() == max_size, + "activeRanks must be sized to max_world_size when " + "max_world_size is set"); + } meta_->activeRanksTensor = options_->activeRanks_; } else { - meta_->activeRanksTensor = - at::ones({size}, torch::dtype(torch::kInt32) - .device(isCpu ? torch::kCPU : torch::kCUDA)); + meta_->activeRanksTensor = at::ones( + {max_size}, torch::dtype(torch::kInt32) + .device(isCpu ? torch::kCPU : torch::kCUDA)); + if (max_size != size) { + meta_->activeRanksTensor.slice(0, size, max_size).fill_(0); + } } meta_->engine = engine_; meta_->store = store; @@ -610,15 +651,15 @@ c10::intrusive_ptr MooncakeBackend::_allgather_base( const c10d::AllgatherOptions& opts) { size_t tensorSize = inputBuffer.numel() * inputBuffer.element_size(); if (isCpu_) { - auto numRanks = meta_->size; return worker_->putTaskCpu( c10d::OpType::_ALLGATHER_BASE, tensorSize, 0, meta_, connection_ctx_, [=](void* dst, size_t pos, size_t realSize) { memcpy(dst, (char*)inputBuffer.data_ptr() + pos, realSize); }, - [=](void* src, size_t pos, size_t realSize) { - for (const auto j : c10::irange(numRanks)) { + [=, this](void* src, size_t pos, size_t realSize) { + for (int j = 0; j < meta_->size; ++j) { + if (!meta_->activeRanks[j]) continue; memcpy( (char*)outputBuffer.data_ptr() + j * tensorSize + pos, (char*)src + j * realSize, realSize); @@ -637,7 +678,8 @@ c10::intrusive_ptr MooncakeBackend::_allgather_base( }, [=, this](void* src, size_t pos, size_t realSize, const at::cuda::CUDAStream& enq_stream) { - for (const auto j : c10::irange(meta_->size)) { + for (int j = 0; j < meta_->size; ++j) { + if (!meta_->activeRanks[j]) continue; cudaMemcpyAsync( (char*)outputBuffer.data_ptr() + j * tensorSize + pos, (char*)src + j * realSize, realSize, @@ -656,8 +698,9 @@ c10::intrusive_ptr MooncakeBackend::_reduce_scatter_base( return worker_->putTaskCpu( c10d::OpType::_REDUCE_SCATTER_BASE, tensorSize, 0, meta_, connection_ctx_, - [=](void* dst, size_t pos, size_t realSize) { - for (const auto j : c10::irange(numRanks)) { + [=, this](void* dst, size_t pos, size_t realSize) { + for (int j = 0; j < meta_->size; ++j) { + if (!meta_->activeRanks[j]) continue; memcpy((char*)dst + j * realSize, (char*)inputBuffer.data_ptr() + j * tensorSize + pos, realSize); @@ -676,7 +719,8 @@ c10::intrusive_ptr MooncakeBackend::_reduce_scatter_base( connection_ctx_, stream, [=, this](void* dst, size_t pos, size_t realSize, const at::cuda::CUDAStream& enq_stream) { - for (const auto j : c10::irange(meta_->size)) { + for (int j = 0; j < meta_->size; ++j) { + if (!meta_->activeRanks[j]) continue; cudaMemcpyAsync( (char*)dst + j * realSize, (char*)inputBuffer.data_ptr() + j * tensorSize + pos, @@ -1041,6 +1085,15 @@ void MooncakeBackend::waitForExtensionState() { meta_->activeRanks[i] = state.activeRanks[i]; } syncActiveRanksTensor(); + + // activeSize: count the number of active ranks (contiguous from 0) + int newActiveSize = 0; + for (int i = 0; i < meta_->size; ++i) { + if (meta_->activeRanks[i]) { + newActiveSize = i + 1; + } + } + meta_->activeSize = newActiveSize; } int MooncakeBackend::getNumSyncedRanks() { @@ -1063,6 +1116,7 @@ int MooncakeBackend::getNumSyncedRanks() { void MooncakeBackend::extendGroupSizeTo(int newSize) { const int oldSize = meta_->size; + const int oldActiveSize = meta_->activeSize; if (newSize == oldSize) return; TORCH_CHECK(newSize >= 0 && static_cast(newSize) < kMaxNumRanks, @@ -1073,17 +1127,26 @@ void MooncakeBackend::extendGroupSizeTo(int newSize) { << ": Group size extend to " << newSize; meta_->size = newSize; + meta_->activeSize = newSize; meta_->taskCount = 0; // Initialize new rank's metadata for (int i = oldSize; i < newSize; ++i) { local2global_rank_map_[i] = i; - meta_->activeRanks[i] = true; + // IMPORTANT: Newly-extended ranks must start as inactive. + // They will only participate in collectives after healthy ranks + // explicitly activate them via recoverRanks(). This enables a + // two-phase scale-up protocol (extend capacity -> poll readiness + // -> recover/activate) and avoids collectives including ranks that + // haven't joined yet. + meta_->activeRanks[i] = false; } auto& tensor = meta_->activeRanksTensor; - tensor.resize_({newSize}); - tensor.slice(0, oldSize, newSize).fill_(1); + if (newSize > tensor.numel()) { + tensor.resize_({newSize}); + } + tensor.slice(0, oldSize, newSize).fill_(0); connection_ctx_->extendGroupSizeTo(newSize); p2p_proxy_->extendGroupSizeTo(newSize); @@ -1147,6 +1210,14 @@ void MooncakeBackend::recoverRanks(const std::vector& ranks) { meta_->activeRanks[rank] = true; } + // Expand activeSize if any recovered rank is beyond the current boundary. + if (!ranks.empty()) { + const int max_rank = *std::max_element(ranks.begin(), ranks.end()); + if (max_rank >= meta_->activeSize) { + meta_->activeSize = max_rank + 1; + } + } + syncActiveRanksTensor(); std::vector epochs(meta_->size); for (int i = 0; i < meta_->size; ++i) { diff --git a/mooncake-pg/src/pg_py.cpp b/mooncake-pg/src/pg_py.cpp index 441a04ad..09b1c6d3 100644 --- a/mooncake-pg/src/pg_py.cpp +++ b/mooncake-pg/src/pg_py.cpp @@ -103,7 +103,9 @@ PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) { m, "MooncakeBackendOptions") .def(py::init(), py::arg("active_ranks")) .def(py::init(), py::arg("active_ranks"), - py::arg("is_extension")); + py::arg("is_extension")) + .def(py::init(), py::arg("active_ranks"), + py::arg("is_extension"), py::arg("max_world_size")); } } // namespace mooncake diff --git a/mooncake-pg/tests/pg_test_utils.py b/mooncake-pg/tests/pg_test_utils.py index 49dc2cd0..a5a75e37 100644 --- a/mooncake-pg/tests/pg_test_utils.py +++ b/mooncake-pg/tests/pg_test_utils.py @@ -12,7 +12,6 @@ import torch.distributed as dist import torch.multiprocessing as mp from mooncake import pg - DEVICE_FILTER_ENV_VAR = "MOONCAKE_PGTEST_DEVICE_FILTERS" MASTER_ADDR_ENV_VAR = "MOONCAKE_PGTEST_MASTER_ADDR" MASTER_PORT_ENV_VAR = "MOONCAKE_PGTEST_MASTER_PORT" @@ -104,17 +103,21 @@ def mooncake_backend_options( *, active_value: int = 0, is_extension: bool = False, + max_world_size: int | None = None, ) -> pg.MooncakeBackendOptions: device = torch.device(device_type) + tensor_size = world_size if max_world_size is None else int(max_world_size) active_ranks = torch.full( - (world_size,), + (tensor_size,), int(active_value), dtype=torch.int32, device=device, ) - if is_extension: - return pg.MooncakeBackendOptions(active_ranks, True) - return pg.MooncakeBackendOptions(active_ranks) + if max_world_size is None: + if is_extension: + return pg.MooncakeBackendOptions(active_ranks, True) + return pg.MooncakeBackendOptions(active_ranks) + return pg.MooncakeBackendOptions(active_ranks, bool(is_extension), tensor_size) def mooncake_cpu_options(world_size: int) -> pg.MooncakeBackendOptions: @@ -140,6 +143,7 @@ def init_mooncake_group( use_pg_options: bool = True, is_extension: bool = False, active_value: int | None = None, + max_world_size: int | None = None, ) -> torch.device: device = require_test_device(rank, device_type) configure_mooncake_device_filter(device_filters) @@ -157,6 +161,7 @@ def init_mooncake_group( device_type, active_value=resolved_active_value, is_extension=is_extension, + max_world_size=max_world_size, ) dist.init_process_group(**kwargs) return device @@ -214,6 +219,7 @@ class MooncakePGWorkerContext: use_pg_options: bool = True, is_extension: bool = False, active_value: int | None = None, + max_world_size: int | None = None, ) -> torch.device: self._device = init_mooncake_group( self.proc_rank if rank is None else rank, @@ -226,6 +232,7 @@ class MooncakePGWorkerContext: use_pg_options=use_pg_options, is_extension=is_extension, active_value=active_value, + max_world_size=max_world_size, ) return self._device diff --git a/mooncake-pg/tests/test_pg_elastic.py b/mooncake-pg/tests/test_pg_elastic.py index f93527e3..23f71008 100644 --- a/mooncake-pg/tests/test_pg_elastic.py +++ b/mooncake-pg/tests/test_pg_elastic.py @@ -11,6 +11,9 @@ from pg_test_utils import ( MooncakePGCPUBackendTestCase, MooncakePGCUDABackendTestCase, MooncakePGWorkerContext, + configure_mooncake_device_filter, + get_mooncake_backend, + require_test_device, wait_until, ) @@ -50,11 +53,24 @@ def _extension_worker( initial_world_size = ctx.world_size - 1 extension_rank = ctx.world_size - 1 + join_ranks = [extension_rank] + if ctx.proc_rank < initial_world_size: # Original ranks - device = ctx.init_group(world_size=initial_world_size) + device = ctx.init_group( + world_size=initial_world_size, + max_world_size=ctx.world_size, + ) backend = ctx.get_backend() + # group_size should equal initial_world_size immediately after init + # (max_world_size only pre-allocates capacity, does not change visible size) + actual_ws = dist.get_world_size() + assert actual_ws == initial_world_size, ( + f"rank {ctx.proc_rank}: initial world_size={actual_ws}, " + f"expected initial_world_size={initial_world_size}" + ) + # First collective tensor = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) dist.all_reduce(tensor, op=dist.ReduceOp.SUM) @@ -63,11 +79,25 @@ def _extension_worker( # Signal ready and extend if ctx.proc_rank == 0: extend_event.set() - pg.extend_group_size_to(backend, ctx.world_size) - # Wait for extension rank to complete init before collective - if not init_done_event.wait(timeout=30.0): - raise TimeoutError("timed out waiting for extension init") + # Two-phase extension protocol: + # 1) joiner publishes metadata + establishes transport readiness + # 2) healthy ranks recover/activate it via recover_ranks() + # Note: get_peer_state() is collective among *healthy ranks*. + wait_until( + lambda: all(pg.get_peer_state(backend, join_ranks)), + timeout_s=30.0, + poll_interval_s=0.05, + description=f"rank {ctx.proc_rank} waiting for joiner ready", + ) + pg.recover_ranks(backend, join_ranks) + + # After recover_ranks, world_size should now reflect the expanded group + actual_ws_after = dist.get_world_size() + assert actual_ws_after == ctx.world_size, ( + f"rank {ctx.proc_rank}: world_size after recover={actual_ws_after}, " + f"expected max_world_size={ctx.world_size}" + ) # Final collective final_tensor = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) @@ -86,10 +116,41 @@ def _extension_worker( device = ctx.init_group( rank=extension_rank, world_size=ctx.world_size, + is_extension=True, + max_world_size=ctx.world_size, ) - # Signal init complete before collective - init_done_event.set() + backend = ctx.get_backend() + + # group_size for extension rank equals world_size passed at init. + # Note: this is world_size (= max_world_size for joiners), not 1, + # because the joiner's activeSize is initialized to world_size. + # The local-only behavior is ensured by activeRanks masking, not + # by a smaller activeSize. + actual_ws = dist.get_world_size() + assert actual_ws == ctx.world_size, ( + f"extension rank: initial world_size={actual_ws}, " + f"expected {ctx.world_size}" + ) + + # In extension mode, joiner starts in local-only collectives. + local_tensor = torch.tensor([extension_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(local_tensor, op=dist.ReduceOp.SUM) + if int(local_tensor.cpu().item()) != extension_rank + 1: + raise AssertionError( + f"extension rank expected local-only sum {extension_rank + 1}, got {int(local_tensor.cpu().item())}" + ) + + # join_group publishes metadata and then blocks until recover_ranks() + # publishes the extension state. + pg.join_group(backend) + + # After joinGroup, world_size should reflect the full group + actual_ws_after = dist.get_world_size() + assert actual_ws_after == ctx.world_size, ( + f"extension rank: world_size after joinGroup={actual_ws_after}, " + f"expected {ctx.world_size}" + ) # Final collective final_tensor = torch.tensor([extension_rank + 1], dtype=torch.int32, device=device) @@ -101,6 +162,416 @@ def _extension_worker( }) +def _extension_worker_with_subgroups( + ctx: MooncakePGWorkerContext, + extend_event: mp.Event, +) -> None: + """Multi-subgroup elastic extension test using split-ranks pattern. + + Layout (world_size=4, primary=[0,1], joiners=[2,3]): + group_a: primary ranks=[0], joiner ranks=[0,2], max_world_size=2 + group_b: primary ranks=[1], joiner ranks=[1,3], max_world_size=2 + group_c: primary ranks=[0,1], joiner ranks=[0,1,2,3], max_world_size=4 + + Primary ranks must initialize WORLD with world_size=initial_world_size and + create subgroups using only their current membership; joiners wait for the + extend signal, then init WORLD with the full world_size and create subgroups + using the full eventual membership. PyTorch's new_group uses a monotonic + call counter for the store prefix, so primary and joiner side land on the + same prefix as long as the call order matches. backendIndex_ is also + process-local and increments only when the rank is an actual member of the + new group, so it stays aligned across primaries and joiners that all call + new_group in the same order. + """ + configure_mooncake_device_filter(ctx.device_filters) + device = require_test_device(ctx.proc_rank, ctx.device_type) + + assert ctx.world_size == 4, "this test assumes world_size=4" + initial_world_size = 2 + join_ranks = [2, 3] + is_joiner = ctx.proc_rank >= initial_world_size + + a_active = torch.tensor([1, 0], dtype=torch.int32, device=device) + b_active = torch.tensor([1, 0], dtype=torch.int32, device=device) + c_active = torch.tensor([1, 1, 0, 0], dtype=torch.int32, device=device) + + if not is_joiner: + # Primary ranks: init WORLD with world_size=2, max_world_size=4 + world_active = torch.tensor([1, 1, 0, 0], dtype=torch.int32, device=device) + dist_kwargs = { + "backend": ctx.backend_name, + "rank": ctx.proc_rank, + "world_size": initial_world_size, + "pg_options": pg.MooncakeBackendOptions(world_active, False, ctx.world_size), + } + if ctx.device_type == "cuda": + dist_kwargs["device_id"] = device + dist.init_process_group(**dist_kwargs) + world_backend = get_mooncake_backend(device_type=ctx.device_type) + + # Subgroups with split-ranks pattern. All ranks in WORLD must call + # new_group in the same order even for groups they are not members of. + group_a = dist.new_group( + ranks=[0], + backend=ctx.backend_name, + pg_options=pg.MooncakeBackendOptions(a_active, False, 2), + ) + group_b = dist.new_group( + ranks=[1], + backend=ctx.backend_name, + pg_options=pg.MooncakeBackendOptions(b_active, False, 2), + ) + group_c = dist.new_group( + ranks=[0, 1], + backend=ctx.backend_name, + pg_options=pg.MooncakeBackendOptions(c_active, False, 4), + ) + a_backend = get_mooncake_backend(group_a, device_type=ctx.device_type) if ctx.proc_rank == 0 else None + b_backend = get_mooncake_backend(group_b, device_type=ctx.device_type) if ctx.proc_rank == 1 else None + c_backend = get_mooncake_backend(group_c, device_type=ctx.device_type) + + # Pre-activation: WORLD primary ranks sum to 1+2=3 + t = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(t, op=dist.ReduceOp.SUM) + if int(t.cpu().item()) != 3: + raise AssertionError(f"WORLD pre: expected 3, got {int(t.cpu().item())}") + + # group_c primaries (ranks [0,1]): sum to 1+2=3 + tc = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(tc, op=dist.ReduceOp.SUM, group=group_c) + if int(tc.cpu().item()) != 3: + raise AssertionError(f"group_c pre: expected 3, got {int(tc.cpu().item())}") + + if ctx.proc_rank == 0: + extend_event.set() + + # WORLD: wait for joiners then recover + wait_until( + lambda: all(pg.get_peer_state(world_backend, join_ranks)), + timeout_s=60.0, + poll_interval_s=0.05, + description=f"rank {ctx.proc_rank} waiting for WORLD joiners", + ) + pg.recover_ranks(world_backend, join_ranks) + + # group_a: rank 0 waits for joiner (local rank 1 = global rank 2) + if ctx.proc_rank == 0: + wait_until( + lambda: pg.get_peer_state(a_backend, [1])[0], + timeout_s=60.0, + poll_interval_s=0.05, + description="rank 0 waiting for group_a joiner", + ) + pg.recover_ranks(a_backend, [1]) + + # group_b: rank 1 waits for joiner (local rank 1 = global rank 3) + if ctx.proc_rank == 1: + wait_until( + lambda: pg.get_peer_state(b_backend, [1])[0], + timeout_s=60.0, + poll_interval_s=0.05, + description="rank 1 waiting for group_b joiner", + ) + pg.recover_ranks(b_backend, [1]) + + # group_c: both primaries wait for both joiners (local ranks 2,3) + wait_until( + lambda: all(pg.get_peer_state(c_backend, [2, 3])), + timeout_s=60.0, + poll_interval_s=0.05, + description=f"rank {ctx.proc_rank} waiting for group_c joiners", + ) + pg.recover_ranks(c_backend, [2, 3]) + + # Post-activation: WORLD all 4 ranks → 1+2+3+4=10 + t = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(t, op=dist.ReduceOp.SUM) + if int(t.cpu().item()) != 10: + raise AssertionError(f"WORLD post: expected 10, got {int(t.cpu().item())}") + + # group_a: ranks [0,2], values [1,3], sum=4 + if ctx.proc_rank == 0: + ta = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(ta, op=dist.ReduceOp.SUM, group=group_a) + if int(ta.cpu().item()) != 4: + raise AssertionError(f"group_a post: expected 4, got {int(ta.cpu().item())}") + + # group_b: ranks [1,3], values [2,4], sum=6 + if ctx.proc_rank == 1: + tb = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(tb, op=dist.ReduceOp.SUM, group=group_b) + if int(tb.cpu().item()) != 6: + raise AssertionError(f"group_b post: expected 6, got {int(tb.cpu().item())}") + + # group_c: all 4 ranks, sum=10 + tc = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(tc, op=dist.ReduceOp.SUM, group=group_c) + if int(tc.cpu().item()) != 10: + raise AssertionError(f"group_c post: expected 10, got {int(tc.cpu().item())}") + + ctx.record_result({"role": "extension_subgroups", "rank": ctx.proc_rank}) + else: + # Joiners: wait for extend_event, then init WORLD with world_size=4 + if not extend_event.wait(timeout=60.0): + raise TimeoutError("timed out waiting for extend_event") + + world_active = torch.tensor([1, 1, 0, 0], dtype=torch.int32, device=device) + dist_kwargs = { + "backend": ctx.backend_name, + "rank": ctx.proc_rank, + "world_size": ctx.world_size, + "pg_options": pg.MooncakeBackendOptions(world_active, True, ctx.world_size), + } + if ctx.device_type == "cuda": + dist_kwargs["device_id"] = device + dist.init_process_group(**dist_kwargs) + world_backend = get_mooncake_backend(device_type=ctx.device_type) + + # Subgroups: full eventual membership; matching call order with primaries. + group_a = dist.new_group( + ranks=[0, 2], + backend=ctx.backend_name, + pg_options=pg.MooncakeBackendOptions(a_active, True, 2), + ) + group_b = dist.new_group( + ranks=[1, 3], + backend=ctx.backend_name, + pg_options=pg.MooncakeBackendOptions(b_active, True, 2), + ) + group_c = dist.new_group( + ranks=[0, 1, 2, 3], + backend=ctx.backend_name, + pg_options=pg.MooncakeBackendOptions(c_active, True, 4), + ) + a_backend = get_mooncake_backend(group_a, device_type=ctx.device_type) if ctx.proc_rank == 2 else None + b_backend = get_mooncake_backend(group_b, device_type=ctx.device_type) if ctx.proc_rank == 3 else None + c_backend = get_mooncake_backend(group_c, device_type=ctx.device_type) + + # Joiners are local-only until join_group is called + t = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(t, op=dist.ReduceOp.SUM) + if int(t.cpu().item()) != ctx.proc_rank + 1: + raise AssertionError( + f"WORLD local-only: expected {ctx.proc_rank + 1}, got {int(t.cpu().item())}" + ) + + # Join groups in same order primaries created them + pg.join_group(world_backend) + if ctx.proc_rank == 2: + pg.join_group(a_backend) + if ctx.proc_rank == 3: + pg.join_group(b_backend) + pg.join_group(c_backend) + + # Post-activation: WORLD all 4 ranks → 1+2+3+4=10 + t = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(t, op=dist.ReduceOp.SUM) + if int(t.cpu().item()) != 10: + raise AssertionError(f"WORLD post: expected 10, got {int(t.cpu().item())}") + + # group_a: ranks [0,2], sum=4 + if ctx.proc_rank == 2: + ta = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(ta, op=dist.ReduceOp.SUM, group=group_a) + if int(ta.cpu().item()) != 4: + raise AssertionError(f"group_a post: expected 4, got {int(ta.cpu().item())}") + + # group_b: ranks [1,3], sum=6 + if ctx.proc_rank == 3: + tb = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(tb, op=dist.ReduceOp.SUM, group=group_b) + if int(tb.cpu().item()) != 6: + raise AssertionError(f"group_b post: expected 6, got {int(tb.cpu().item())}") + + # group_c: all 4 ranks, sum=10 + tc = torch.tensor([ctx.proc_rank + 1], dtype=torch.int32, device=device) + dist.all_reduce(tc, op=dist.ReduceOp.SUM, group=group_c) + if int(tc.cpu().item()) != 10: + raise AssertionError(f"group_c post: expected 10, got {int(tc.cpu().item())}") + + ctx.record_result({"role": "extension_subgroups", "rank": ctx.proc_rank}) + + +def _run_allgather_reduce_scatter( + device: str, + active_world_size: int, + rank: int, +) -> None: + """Run _allgather_base and _reduce_scatter_base and assert correctness. + + active_world_size: number of currently active ranks (buffer size). + rank: this rank's logical rank (0-indexed). + Each rank contributes value (rank + 1). + """ + + # --- _allgather_base --- + # input: scalar (rank+1); output: flat buffer of active_world_size elements + input_t = torch.tensor([rank + 1], dtype=torch.int32, device=device) + output_t = torch.zeros(active_world_size, dtype=torch.int32, device=device) + dist.all_gather_into_tensor(output_t, input_t) + for j in range(active_world_size): + expected = j + 1 + got = int(output_t[j].item()) + if got != expected: + raise AssertionError( + f"allgather slot {j}: expected {expected}, got {got} " + f"(rank={rank}, active_world_size={active_world_size})" + ) + + # --- _reduce_scatter_base --- + # input: flat buffer [1, 2, ..., active_world_size]; output: scalar chunk + # With SUM and equal chunks of size 1, each rank receives its own slot's sum + # across all ranks. Since all ranks send the same input buffer [1..N], + # rank j receives sum of input[j] from all ranks = (j+1) * active_world_size. + input_rs = torch.arange(1, active_world_size + 1, dtype=torch.int32, device=device) + output_rs = torch.zeros(1, dtype=torch.int32, device=device) + dist.reduce_scatter_tensor(output_rs, input_rs) + expected_rs = (rank + 1) * active_world_size + got_rs = int(output_rs[0].item()) + if got_rs != expected_rs: + raise AssertionError( + f"reduce_scatter slot {rank}: expected {expected_rs}, got {got_rs} " + f"(rank={rank}, active_world_size={active_world_size})" + ) + + +def _allgather_reduce_scatter_extension_worker( + ctx: MooncakePGWorkerContext, + extend_event: mp.Event, +) -> None: + """Test _allgather_base and _reduce_scatter_base across elastic extension. + + Layout: world_size=4, initial=3, extension_rank=3, max_world_size=4. + Pre-activation: 3 active ranks, max_world_size=4 → exercises the overflow path. + Post-activation: 4 active ranks → exercises correctness after extension. + """ + configure_mooncake_device_filter(ctx.device_filters) + device = require_test_device(ctx.proc_rank, ctx.device_type) + + assert ctx.world_size == 4 + initial_world_size = 3 + extension_rank = 3 + join_ranks = [extension_rank] + is_joiner = ctx.proc_rank == extension_rank + + if not is_joiner: + active = torch.tensor([1, 1, 1, 0], dtype=torch.int32, device=device) + dist_kwargs = { + "backend": ctx.backend_name, + "rank": ctx.proc_rank, + "world_size": initial_world_size, + "pg_options": pg.MooncakeBackendOptions(active, False, ctx.world_size), + } + if ctx.device_type == "cuda": + dist_kwargs["device_id"] = device + dist.init_process_group(**dist_kwargs) + backend = get_mooncake_backend(device_type=ctx.device_type) + + # Pre-activation: 3 active ranks, max_world_size=4. + # This is the overflow path: buggy code would iterate 4 times into a + # buffer sized for 3. + _run_allgather_reduce_scatter(device, initial_world_size, ctx.proc_rank) + + if ctx.proc_rank == 0: + extend_event.set() + + wait_until( + lambda: all(pg.get_peer_state(backend, join_ranks)), + timeout_s=30.0, + poll_interval_s=0.05, + description=f"rank {ctx.proc_rank} waiting for joiner", + ) + pg.recover_ranks(backend, join_ranks) + + # Post-activation: all 4 ranks active. + _run_allgather_reduce_scatter(device, ctx.world_size, ctx.proc_rank) + + ctx.record_result({"role": "primary", "rank": ctx.proc_rank}) + else: + if not extend_event.wait(timeout=30.0): + raise TimeoutError("timed out waiting for extend_event") + + active = torch.tensor([1, 1, 1, 0], dtype=torch.int32, device=device) + dist_kwargs = { + "backend": ctx.backend_name, + "rank": extension_rank, + "world_size": ctx.world_size, + "pg_options": pg.MooncakeBackendOptions(active, True, ctx.world_size), + } + if ctx.device_type == "cuda": + dist_kwargs["device_id"] = device + dist.init_process_group(**dist_kwargs) + backend = get_mooncake_backend(device_type=ctx.device_type) + + pg.join_group(backend) + + # Post-activation: all 4 ranks active. + _run_allgather_reduce_scatter(device, ctx.world_size, extension_rank) + + ctx.record_result({"role": "joiner", "rank": extension_rank}) + + +def _allgather_reduce_scatter_recovery_worker( + ctx: MooncakePGWorkerContext, + broken_exited: mp.Event, + replacement_ready: mp.Event, + start_recovery: mp.Event, +) -> None: + """Test _allgather_base and _reduce_scatter_base across rank recovery. + + Layout: world_size=4, broken_rank=3, replacement takes rank 3. + Pre-failure: 4 active ranks. + Post-failure (3 survivors): 3 active ranks, max_world_size=4 → overflow path. + Post-recovery: 4 active ranks again. + """ + broken_rank = ctx.world_size - 1 + logical_rank = ctx.rank if ctx.proc_rank < ctx.world_size else broken_rank + + if ctx.proc_rank < ctx.world_size: + device = ctx.init_group(rank=logical_rank) + backend = ctx.get_backend() + + # Pre-failure: all 4 ranks active. + _run_allgather_reduce_scatter(device, ctx.world_size, logical_rank) + + if logical_rank == broken_rank: + ctx.record_result({"role": "broken"}) + broken_exited.set() + os._exit(0) + + # Survivors: 3 active ranks, max_world_size=4 → overflow path. + broken_exited.wait() + _run_allgather_reduce_scatter(device, ctx.world_size - 1, logical_rank) + + if logical_rank == 0: + start_recovery.set() + + wait_until( + lambda: pg.get_peer_state(backend, [broken_rank])[0], + timeout_s=30.0, + poll_interval_s=2.0, + description=f"rank {logical_rank} waiting for replacement", + ) + replacement_ready.wait() + pg.recover_ranks(backend, [broken_rank]) + + # Post-recovery: all 4 ranks active again. + _run_allgather_reduce_scatter(device, ctx.world_size, logical_rank) + + ctx.record_result({"role": "survivor"}) + else: + start_recovery.wait() + device = ctx.init_group(rank=logical_rank, is_extension=True) + backend = ctx.get_backend() + replacement_ready.set() + pg.join_group(backend) + + # Post-recovery: all 4 ranks active. + _run_allgather_reduce_scatter(device, ctx.world_size, logical_rank) + + ctx.record_result({"role": "replacement"}) + + def _fault_detection_worker( ctx: MooncakePGWorkerContext, broken_exited: mp.Event, @@ -293,6 +764,69 @@ class _ElasticMixin: for row in original_rows: self.assertEqual(row.get("baseline"), expected_baseline) + def test_extension_with_subgroups(self) -> None: + """Test extension with multiple disjoint subgroups using split-ranks pattern.""" + spawn_ctx = mp.get_context("spawn") + extend_event = spawn_ctx.Event() + + rows = self.spawn_backend_and_collect( + _extension_worker_with_subgroups, + extend_event, + nprocs=self.world_size, + timeout_s=60.0, + ) + + result_rows = [r for r in rows if r.get("role") == "extension_subgroups"] + self.assertEqual(len(result_rows), self.world_size) + + def test_allgather_reduce_scatter_extension(self) -> None: + """Test _allgather_base/_reduce_scatter_base correctness across elastic extension. + + Exercises the overflow path: pre-activation uses max_world_size=4 with only + 3 active ranks, so the buggy code would access slot 3 of a size-3 buffer. + """ + spawn_ctx = mp.get_context("spawn") + extend_event = spawn_ctx.Event() + + rows = self.spawn_backend_and_collect( + _allgather_reduce_scatter_extension_worker, + extend_event, + nprocs=self.world_size, + timeout_s=60.0, + ) + + primary_rows = [r for r in rows if r.get("role") == "primary"] + joiner_rows = [r for r in rows if r.get("role") == "joiner"] + self.assertEqual(len(primary_rows), self.world_size - 1) + self.assertEqual(len(joiner_rows), 1) + + def test_allgather_reduce_scatter_recovery(self) -> None: + """Test _allgather_base/_reduce_scatter_base correctness across rank recovery. + + Exercises the overflow path: post-failure survivors run with 3 active ranks + and max_world_size=4, so the buggy code would access slot 3 of a size-3 buffer. + """ + spawn_ctx = mp.get_context("spawn") + broken_exited = spawn_ctx.Event() + replacement_ready = spawn_ctx.Event() + start_recovery = spawn_ctx.Event() + + rows = self.spawn_backend_and_collect( + _allgather_reduce_scatter_recovery_worker, + broken_exited, + replacement_ready, + start_recovery, + nprocs=self.world_size + 1, + timeout_s=60.0, + ) + + survivor_rows = [r for r in rows if r.get("role") == "survivor"] + replacement_rows = [r for r in rows if r.get("role") == "replacement"] + broken_rows = [r for r in rows if r.get("role") == "broken"] + self.assertEqual(len(survivor_rows), self.world_size - 1) + self.assertEqual(len(replacement_rows), 1) + self.assertGreaterEqual(len(broken_rows), 1) + class TestMooncakePGElasticCPU( _ElasticMixin, MooncakePGCPUBackendTestCase diff --git a/mooncake-wheel/tests/test_mooncake_backend.py b/mooncake-wheel/tests/test_mooncake_backend.py deleted file mode 100644 index d058a591..00000000 --- a/mooncake-wheel/tests/test_mooncake_backend.py +++ /dev/null @@ -1,176 +0,0 @@ -import os -import time -import unittest -import torch -import torch.distributed as dist -import torch.multiprocessing as mp -from mooncake import pg - - -def worker(rank, world_size, results, collective): - torch.cuda.set_device(rank) - dist.init_process_group( - backend="mooncake", - rank=rank, - world_size=world_size, - pg_options=pg.MooncakeBackendOptions(torch.zeros((world_size,), dtype=torch.int32, device="cuda")), - ) - - if collective == "all_reduce_sum": - tensor = torch.tensor([rank + 1], dtype=torch.int32, device="cuda") - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - results[rank] = tensor.item() - - elif collective == "all_reduce_product": - tensor = torch.tensor([2], dtype=torch.int32, device="cuda") - dist.all_reduce(tensor, op=dist.ReduceOp.PRODUCT) - results[rank] = tensor.item() - - elif collective == "all_reduce_min": - tensor = torch.tensor([rank + 10], dtype=torch.int32, device="cuda") - dist.all_reduce(tensor, op=dist.ReduceOp.MIN) - results[rank] = tensor.item() - - elif collective == "all_reduce_max": - tensor = torch.tensor([rank + 10], dtype=torch.int32, device="cuda") - dist.all_reduce(tensor, op=dist.ReduceOp.MAX) - results[rank] = tensor.item() - - elif collective == "all_gather": - tensor = torch.tensor([rank], device="cuda") - gathered = [torch.zeros_like(tensor) for _ in range(world_size)] - dist.all_gather(gathered, tensor) - results[rank] = [t.item() for t in gathered] - - elif collective == "barrier": - dist.barrier() - results[rank] = "ok" - - elif collective == "gather": - tensor = torch.tensor([rank], dtype=torch.int32, device="cuda") - if rank == 0: - gather_list = [torch.zeros_like(tensor) for _ in range(world_size)] - dist.gather(tensor, gather_list, dst=0) - results[rank] = [t.item() for t in gather_list] - else: - dist.gather(tensor, dst=0) - results[rank] = None - - elif collective == "scatter": - tensor = torch.zeros(1, dtype=torch.int32, device="cuda") - if rank == 0: - scatter_list = [torch.tensor([i], dtype=torch.int32, device="cuda") for i in range(world_size)] - dist.scatter(tensor, scatter_list, src=0) - else: - dist.scatter(tensor, src=0) - results[rank] = tensor.item() - - elif collective == "reduce": - tensor = torch.tensor([1], dtype=torch.int32, device="cuda") - dist.reduce(tensor, dst=0, op=dist.ReduceOp.SUM) - if rank == 0: - results[rank] = tensor.item() - else: - results[rank] = None - - else: - raise ValueError(f"Unsupported collective: {collective}") - - while len(results) < world_size: - time.sleep(1) - - dist.destroy_process_group() - - -class TestMooncakeBackend(unittest.TestCase): - def setUp(self): - self.world_size = torch.cuda.device_count() - os.environ["MASTER_ADDR"] = "127.0.0.1" - os.environ["MASTER_PORT"] = "29500" - - def tearDown(self): - pass - - def _spawn_and_check(self, collective, expected_fn): - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, collective), - nprocs=self.world_size, - join=True, - ) - - expected = expected_fn(self.world_size) - for r in range(self.world_size): - self.assertEqual(results[r], expected) - - def test_allreduce_sum(self): - # Expected sum = 1 + 2 + 3 + 4 = 10 - self._spawn_and_check("all_reduce_sum", lambda size: sum(range(1, size + 1))) - - def test_allreduce_product(self): - # Expected product(2, 2, 2, ……, 2) = 2 ^ N - self._spawn_and_check("all_reduce_product", lambda size: 2 ** size) - - def test_allreduce_min(self): - # Expected Min(10, 11, ……) = 10 - self._spawn_and_check("all_reduce_min", lambda size: 10) - - def test_allreduce_max(self): - # Expected Max(10, 11, ..., 10+N-1) = 10 + N - 1 - self._spawn_and_check("all_reduce_max", lambda size: 10 + size - 1) - - def test_allgather(self): - # Expected gather = [0, 1, 2, 3] - self._spawn_and_check("all_gather", lambda size: list(range(size))) - - def test_barrier(self): - self._spawn_and_check("barrier", lambda size: "ok") - - def test_gather(self): - # Expected gather (Root) = [0, 1, 2, ..., size-1] - # Expected gather (Others) = None - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, "gather"), - nprocs=self.world_size, - join=True, - ) - self.assertEqual(results[0], list(range(self.world_size))) - for r in range(1, self.world_size): - self.assertIsNone(results[r]) - - def test_scatter(self): - # Expected scatter (Rank i) = i - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, "scatter"), - nprocs=self.world_size, - join=True, - ) - for r in range(self.world_size): - self.assertEqual(results[r], r) - - def test_reduce(self): - # Expected reduce (Root) = sum([1, 1, ..., 1]) = size - # Expected reduce (Others) = None - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, "reduce"), - nprocs=self.world_size, - join=True, - ) - self.assertEqual(results[0], self.world_size) - for r in range(1, self.world_size): - self.assertIsNone(results[r]) - - -if __name__ == "__main__": - unittest.main() diff --git a/mooncake-wheel/tests/test_mooncake_backend_chunk.py b/mooncake-wheel/tests/test_mooncake_backend_chunk.py deleted file mode 100644 index 01f9169a..00000000 --- a/mooncake-wheel/tests/test_mooncake_backend_chunk.py +++ /dev/null @@ -1,64 +0,0 @@ -import os -import time -import unittest -import torch -import torch.distributed as dist -import torch.multiprocessing as mp -from mooncake import pg - -N = 2 ** 24 - -def worker(rank, world_size, results, collective): - torch.cuda.set_device(rank) - dist.init_process_group( - backend="mooncake-cpu", - rank=rank, - world_size=world_size, - pg_options=pg.MooncakeBackendOptions(torch.zeros((world_size,), dtype=torch.int32, device="cpu")), - ) - - if collective == "all_reduce": - tensor = torch.tensor([rank + 1] * N, dtype=torch.int32, device="cpu") - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - results[rank] = tensor[0].item() - assert torch.all(tensor == tensor[0].item()) - - else: - raise ValueError(f"Unsupported collective: {collective}") - - while len(results) < world_size: - time.sleep(1) - - dist.destroy_process_group() - - -class TestMooncakeBackend(unittest.TestCase): - def setUp(self): - self.world_size = torch.cuda.device_count() - os.environ["MASTER_ADDR"] = "127.0.0.1" - os.environ["MASTER_PORT"] = "29500" - - def tearDown(self): - pass - - def _spawn_and_check(self, collective): - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, collective), - nprocs=self.world_size, - join=True, - ) - - expected = sum(range(1, self.world_size + 1)) - for r in range(self.world_size): - self.assertEqual(results[r], expected) - - def test_allreduce_sum(self): - self._spawn_and_check("all_reduce") - - - -if __name__ == "__main__": - unittest.main() diff --git a/mooncake-wheel/tests/test_mooncake_backend_cpu.py b/mooncake-wheel/tests/test_mooncake_backend_cpu.py deleted file mode 100644 index 94c301f3..00000000 --- a/mooncake-wheel/tests/test_mooncake_backend_cpu.py +++ /dev/null @@ -1,148 +0,0 @@ -import os -import time -import unittest -import torch -import torch.distributed as dist -import torch.multiprocessing as mp -from mooncake import pg - - -def worker(rank, world_size, results, collective): - dist.init_process_group( - backend="mooncake-cpu", - rank=rank, - world_size=world_size, - pg_options=pg.MooncakeBackendOptions(torch.zeros((world_size,), dtype=torch.int32, device="cpu")), - ) - - if collective == "all_reduce": - tensor = torch.tensor([rank + 1], dtype=torch.int32, device="cpu") - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - results[rank] = tensor.item() - - elif collective == "all_gather": - tensor = torch.tensor([rank], device="cpu") - gathered = [torch.zeros_like(tensor) for _ in range(world_size)] - dist.all_gather(gathered, tensor) - results[rank] = [t.item() for t in gathered] - - elif collective == "barrier": - dist.barrier() - results[rank] = "ok" - - elif collective == "gather": - tensor = torch.tensor([rank], dtype=torch.int32, device="cpu") - if rank == 0: - gather_list = [torch.zeros_like(tensor) for _ in range(world_size)] - dist.gather(tensor, gather_list, dst=0) - results[rank] = [t.item() for t in gather_list] - else: - dist.gather(tensor, dst=0) - results[rank] = None - - elif collective == "scatter": - tensor = torch.zeros(1, dtype=torch.int32, device="cpu") - if rank == 0: - scatter_list = [torch.tensor([i], dtype=torch.int32, device="cpu") for i in range(world_size)] - dist.scatter(tensor, scatter_list, src=0) - else: - dist.scatter(tensor, src=0) - results[rank] = tensor.item() - - elif collective == "reduce": - tensor = torch.tensor([1], dtype=torch.int32, device="cpu") - dist.reduce(tensor, dst=0, op=dist.ReduceOp.SUM) - if rank == 0: - results[rank] = tensor.item() - else: - results[rank] = None - - else: - raise ValueError(f"Unsupported collective: {collective}") - - while len(results) < world_size: - time.sleep(1) - - dist.destroy_process_group() - - -class TestMooncakeBackend(unittest.TestCase): - def setUp(self): - self.world_size = 4 - os.environ["MASTER_ADDR"] = "127.0.0.1" - os.environ["MASTER_PORT"] = "29500" - - def tearDown(self): - pass - - def _spawn_and_check(self, collective, expected_fn): - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, collective), - nprocs=self.world_size, - join=True, - ) - - expected = expected_fn(self.world_size) - for r in range(self.world_size): - self.assertEqual(results[r], expected) - - def test_allreduce(self): - # Expected sum = 1 + 2 + 3 + 4 = 10 - self._spawn_and_check("all_reduce", lambda size: sum(range(1, size + 1))) - - def test_allgather(self): - # Expected gather = [0, 1, 2, 3] - self._spawn_and_check("all_gather", lambda size: list(range(size))) - - def test_barrier(self): - self._spawn_and_check("barrier", lambda size: "ok") - - def test_gather(self): - # Expected gather (Root) = [0, 1, 2, ..., size-1] - # Expected gather (Others) = None - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, "gather"), - nprocs=self.world_size, - join=True - ) - self.assertEqual(results[0], list(range(self.world_size))) - for r in range(1, self.world_size): - self.assertIsNone(results[r]) - - def test_scatter(self): - # Expected scatter (Rank i) = i - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, "scatter"), - nprocs=self.world_size, - join=True - ) - for r in range(self.world_size): - self.assertEqual(results[r], r) - - def test_reduce(self): - # Expected reduce (Root) = sum([1, 1, ..., 1]) = size - # Expected reduce (Others) = None - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - worker, - args=(self.world_size, results, "reduce"), - nprocs=self.world_size, - join=True - ) - self.assertEqual(results[0], self.world_size) - for r in range(1, self.world_size): - self.assertIsNone(results[r]) - - -if __name__ == "__main__": - unittest.main() diff --git a/mooncake-wheel/tests/test_mooncake_backend_elastic.py b/mooncake-wheel/tests/test_mooncake_backend_elastic.py deleted file mode 100644 index 5238285a..00000000 --- a/mooncake-wheel/tests/test_mooncake_backend_elastic.py +++ /dev/null @@ -1,168 +0,0 @@ -import os -import time -import unittest - -import torch -import torch.distributed as dist -import torch.multiprocessing as mp - -from mooncake import pg - -USE_CUDA = os.getenv("MOONCAKE_TEST_USE_CUDA") == "1" -TEST_BACKEND = "mooncake" if USE_CUDA else "mooncake-cpu" -TEST_DEVICE = torch.device("cuda" if USE_CUDA else "cpu") - -os.environ["MASTER_ADDR"] = "127.0.0.1" -os.environ["MASTER_PORT"] = "19000" - -broken_rank = 1 - - -def _set_device(rank): - if USE_CUDA: - torch.cuda.set_device(rank) - - -def _elastic_worker(rank, num_processes, signals): - """Worker for testing elastic world size extension.""" - _set_device(rank) - assert num_processes % 2 == 0 - if rank < num_processes // 2: - # Ensure correct operation before extension - world_size = num_processes // 2 - dist.init_process_group( - backend=TEST_BACKEND, - rank=rank, - world_size=world_size, - ) - tensor = torch.tensor([rank + 1], dtype=torch.int32, device=TEST_DEVICE) - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - assert tensor.item() == sum(range(1, world_size + 1)) - if rank == 0: - signals["extend"] = 1 - - backend = dist.group.WORLD - # Extend world - # Note: `extend_group_size_to` is non-blocking. Blocking will only - # occur at the first communication if some peers have not yet connected. - # This allows overlapping other operations between the group expansion - # and the first communication call. - pg.extend_group_size_to(backend, num_processes) - else: - while "extend" not in signals: - time.sleep(1) - dist.init_process_group( - backend=TEST_BACKEND, - rank=rank, - world_size=num_processes, - ) - - # Ensure correct operation after extension - tensor = torch.tensor([rank + 1], dtype=torch.int32, device=TEST_DEVICE) - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - assert tensor.item() == sum(range(1, num_processes + 1)), ( - f"Rank {rank} expected {sum(range(1, num_processes + 1))}, " - f"get {tensor.item()}" - ) - assert dist.get_world_size() == num_processes, ( - f"Rank {rank}: expected world_size={num_processes} after extension, " - f"got {dist.get_world_size()}" - ) - - -def _deferred_recovery_worker(rank, num_processes, signals): - """Worker for testing deferred rank recovery join.""" - if rank < num_processes: - _set_device(rank) - dist.init_process_group( - backend=TEST_BACKEND, - rank=rank, - world_size=num_processes, - ) - backend = dist.group.WORLD - while pg.get_num_synced_ranks(backend) < num_processes: - time.sleep(0.1) - if rank == broken_rank: - return # Simulate broken rank - - expected_without_broken = sum(range(0, num_processes)) - broken_rank - tensor = torch.tensor([rank], dtype=torch.int32, device=TEST_DEVICE) - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - assert tensor.item() == expected_without_broken - - time.sleep(5) - signals["recover"] = 1 - while True: - (peer_state,) = pg.get_peer_state(backend, [broken_rank]) - if peer_state: - break - - # Healthy ranks keep making progress before the recovered rank joins. - tensor = torch.tensor([rank], dtype=torch.int32, device=TEST_DEVICE) - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - assert tensor.item() == expected_without_broken - - while "join_ready" not in signals: - time.sleep(0.1) - - pg.recover_ranks(backend, [broken_rank]) - - tensor = torch.tensor([rank], dtype=torch.int32, device=TEST_DEVICE) - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - assert tensor.item() == sum(range(0, num_processes)), ( - f"Rank {rank} expected {sum(range(0, num_processes))}, " - f"get {tensor.item()}" - ) - else: - _set_device(broken_rank) - while "recover" not in signals: - time.sleep(0.1) - - dist.init_process_group( - backend=TEST_BACKEND, - rank=broken_rank, - world_size=num_processes, - pg_options=pg.MooncakeBackendOptions( - torch.ones((num_processes,), dtype=torch.int32, device=TEST_DEVICE), - True, - ), - ) - - backend = dist.group.WORLD - - # Deferred join starts in a local-only mode so collectives stay self-contained. - tensor = torch.tensor([broken_rank], dtype=torch.int32, device=TEST_DEVICE) - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - assert tensor.item() == broken_rank - - signals["join_ready"] = 1 - pg.join_group(backend) - - tensor = torch.tensor([broken_rank], dtype=torch.int32, device=TEST_DEVICE) - dist.all_reduce(tensor, op=dist.ReduceOp.SUM) - assert tensor.item() == sum(range(0, num_processes)), ( - f"Rank {rank} expected {sum(range(0, num_processes))}, " - f"get {tensor.item()}" - ) - - -class TestMooncakeBackend(unittest.TestCase): - def test_elastic_extension(self): - num_processes = 4 - mp_manager = mp.Manager() - signals = mp_manager.dict() - mp.spawn(_elastic_worker, args=(num_processes, signals), nprocs=num_processes) - - def test_rank_recovery_deferred_join(self): - num_processes = 4 - mp_manager = mp.Manager() - signals = mp_manager.dict() - mp.spawn( - _deferred_recovery_worker, - args=(num_processes, signals), - nprocs=num_processes + 1, - ) - - -if __name__ == "__main__": - unittest.main() diff --git a/mooncake-wheel/tests/test_mooncake_backend_p2p_cpu.py b/mooncake-wheel/tests/test_mooncake_backend_p2p_cpu.py deleted file mode 100644 index 0e8efbdb..00000000 --- a/mooncake-wheel/tests/test_mooncake_backend_p2p_cpu.py +++ /dev/null @@ -1,212 +0,0 @@ -import os -import time -import unittest - -import torch -import torch.distributed as dist -import torch.multiprocessing as mp - -from mooncake import pg - - -def _worker_ring(rank: int, world_size: int, results): - os.environ.setdefault("MASTER_ADDR", "127.0.0.1") - os.environ.setdefault("MASTER_PORT", "29500") - - dist.init_process_group( - backend="mooncake-cpu", - rank=rank, - world_size=world_size, - pg_options=pg.MooncakeBackendOptions( - torch.zeros((world_size,), dtype=torch.int32, device="cpu") - ), - ) - - send_tensor = torch.tensor([rank], dtype=torch.int64, device="cpu") - recv_tensor = torch.empty_like(send_tensor) - - dst = (rank + 1) % world_size - src = (rank - 1 + world_size) % world_size - - p2p_ops = [ - dist.P2POp(op=dist.isend, tensor=send_tensor, peer=dst), - dist.P2POp(op=dist.irecv, tensor=recv_tensor, peer=src), - ] - - works = dist.batch_isend_irecv(p2p_ops) - for w in works: - w.wait() - - results[rank] = recv_tensor.item() - - while len(results) < world_size: - time.sleep(0.1) - - dist.destroy_process_group() - - -def _worker_ordering(rank: int, results): - # Only two ranks participate in this test. - world_size = 2 - os.environ.setdefault("MASTER_ADDR", "127.0.0.1") - os.environ.setdefault("MASTER_PORT", "29501") - - dist.init_process_group( - backend="mooncake-cpu", - rank=rank, - world_size=world_size, - pg_options=pg.MooncakeBackendOptions( - torch.zeros((world_size,), dtype=torch.int32, device="cpu") - ), - ) - - num_msgs = 4 - - if rank == 0: - send_tensors = [ - torch.tensor([i], dtype=torch.int64, device="cpu") - for i in range(num_msgs) - ] - ops = [ - dist.P2POp(op=dist.isend, tensor=t, peer=1) for t in send_tensors - ] - works = dist.batch_isend_irecv(ops) - for w in works: - w.wait() - results[rank] = "ok" - else: - recv_tensors = [ - torch.empty(1, dtype=torch.int64, device="cpu") - for _ in range(num_msgs) - ] - ops = [ - dist.P2POp(op=dist.irecv, tensor=t, peer=0) for t in recv_tensors - ] - works = dist.batch_isend_irecv(ops) - for w in works: - w.wait() - results[rank] = [t.item() for t in recv_tensors] - - while len(results) < world_size: - time.sleep(0.1) - - dist.destroy_process_group() - - -def _worker_multiple_senders(rank: int, world_size: int, results): - os.environ.setdefault("MASTER_ADDR", "127.0.0.1") - os.environ.setdefault("MASTER_PORT", "29502") - - dist.init_process_group( - backend="mooncake-cpu", - rank=rank, - world_size=world_size, - pg_options=pg.MooncakeBackendOptions( - torch.zeros((world_size,), dtype=torch.int32, device="cpu") - ), - ) - - if rank == 0: - # Rank 0 sends to rank 1 - send_tensor = torch.tensor([100], dtype=torch.int64, device="cpu") - recv_tensor = torch.empty_like(send_tensor) - p2p_ops = [ - dist.P2POp(op=dist.isend, tensor=send_tensor, peer=1), - dist.P2POp(op=dist.irecv, tensor=recv_tensor, peer=1), - ] - works = dist.batch_isend_irecv(p2p_ops) - for w in works: - w.wait() - results[rank] = recv_tensor.item() - elif rank == 1: - # Rank 1 receives from both rank 0 and rank 2 - recv_tensor_0 = torch.empty(1, dtype=torch.int64, device="cpu") - recv_tensor_2 = torch.empty(1, dtype=torch.int64, device="cpu") - send_tensor = torch.tensor([101], dtype=torch.int64, device="cpu") - p2p_ops = [ - dist.P2POp(op=dist.irecv, tensor=recv_tensor_0, peer=0), - dist.P2POp(op=dist.irecv, tensor=recv_tensor_2, peer=2), - dist.P2POp(op=dist.isend, tensor=send_tensor, peer=0), - ] - works = dist.batch_isend_irecv(p2p_ops) - for w in works: - w.wait() - results[rank] = [recv_tensor_0.item(), recv_tensor_2.item()] - elif rank == 2: - # Rank 2 sends to rank 1 - send_tensor = torch.tensor([200], dtype=torch.int64, device="cpu") - p2p_ops = [ - dist.P2POp(op=dist.isend, tensor=send_tensor, peer=1), - ] - works = dist.batch_isend_irecv(p2p_ops) - for w in works: - w.wait() - results[rank] = "ok" - else: - # Other ranks don't participate - results[rank] = "skip" - - while len(results) < world_size: - time.sleep(0.1) - - dist.destroy_process_group() - - -class TestMooncakeBackendP2PCPU(unittest.TestCase): - def test_ring_send_recv(self): - world_size = 4 - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - _worker_ring, - args=(world_size, results), - nprocs=world_size, - join=True, - ) - - for rank in range(world_size): - expected = (rank - 1 + world_size) % world_size - self.assertEqual(results[rank], expected) - - def test_ordering_between_two_ranks(self): - world_size = 2 - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - _worker_ordering, - args=(results,), - nprocs=world_size, - join=True, - ) - - # Rank 1 should receive messages [0, 1, 2, 3] in order. - self.assertEqual(results[1], list(range(4))) - - def test_multiple_senders_to_same_receiver(self): - """Test that multiple senders (rank 0 and rank 2) can send to the same receiver (rank 1) - without slot conflicts. This tests the receiver-based slot allocation mechanism.""" - world_size = 4 - mp_manager = mp.Manager() - results = mp_manager.dict() - mp.spawn( - _worker_multiple_senders, - args=(world_size, results), - nprocs=world_size, - join=True, - ) - - # Rank 0 should receive 101 from rank 1 - self.assertEqual(results[0], 101) - # Rank 1 should receive 100 from rank 0 and 200 from rank 2 - # Order may vary, so check that both values are present - self.assertIn(100, results[1]) - self.assertIn(200, results[1]) - self.assertEqual(len(results[1]), 2) - # Rank 2 should complete successfully - self.assertEqual(results[2], "ok") - - -if __name__ == "__main__": - unittest.main() - - diff --git a/mooncake-wheel/tests/test_mooncake_backend_perf.py b/mooncake-wheel/tests/test_mooncake_backend_perf.py deleted file mode 100644 index 4f741ed3..00000000 --- a/mooncake-wheel/tests/test_mooncake_backend_perf.py +++ /dev/null @@ -1,107 +0,0 @@ -import os -import torch -import torch.distributed as dist -import mooncake.pg -import time -import unittest -import torch.multiprocessing as mp - - -def run_latency_test(rank, world_size, backend, device, collective, data_size, results, num_iterations=100): - # Initialize the process group in each spawned process - torch.cuda.set_device(rank) - dist.init_process_group(backend=backend, rank=rank, world_size=world_size) - - # Create a tensor for the collective operation - tensor = torch.rand(data_size, device=device) - - gathered = [torch.zeros_like(tensor) for _ in range(world_size)] - - # Warm up - for _ in range(num_iterations): - if collective == 'broadcast': - dist.broadcast(tensor, src=0) - elif collective == 'allreduce': - dist.all_reduce(tensor) - elif collective == 'allgather': - dist.all_gather(gathered, tensor) - - # Synchronize before starting the test - torch.cuda.synchronize() - - start = time.perf_counter() - for _ in range(num_iterations): - if collective == 'broadcast': - dist.broadcast(tensor, src=0) - elif collective == 'allreduce': - dist.all_reduce(tensor) - elif collective == 'allgather': - dist.all_gather(gathered, tensor) - - torch.cuda.synchronize() - - end = time.perf_counter() - - # Calculate average time - avg_time = (end - start) / num_iterations - - # Store the result - results[rank] = avg_time - - while len(results) < world_size: - time.sleep(1) - - dist.destroy_process_group() # Destroy the process group after testing - -class TestMooncakeBackendPerf(unittest.TestCase): - def setUp(self): - self.world_size = torch.cuda.device_count() - os.environ["MASTER_ADDR"] = "127.0.0.1" - os.environ["MASTER_PORT"] = "29500" - - def tearDown(self): - pass - - def do_test(self, device, collective, data_size): - # Use mp.spawn to call the latency test - mp_manager = mp.Manager() - # Test mooncake - mooncake_results = mp_manager.dict() - mp.spawn(run_latency_test, args=(self.world_size, 'mooncake-cpu' if device == 'cpu' else 'mooncake', device, collective, data_size, mooncake_results), nprocs=self.world_size, join=True) - # Test baseline - baseline_results = mp_manager.dict() - mp.spawn(run_latency_test, args=(self.world_size, 'gloo' if device == 'cpu' else 'nccl', device, collective, data_size, baseline_results), nprocs=self.world_size, join=True) - - # After all processes have completed, check the results - mooncake_latency = max(mooncake_results[r] for r in mooncake_results) - baseline_latency = max(baseline_results[r] for r in baseline_results) - self.assertLessEqual(mooncake_latency, 10 * baseline_latency, - f"Latency of mooncake({device}) for {collective} with size {data_size} exceeded 10 times the baseline.") - - # cpu + allgather - def test_cpu_allgather_1024(self): - self.do_test("cpu", "allgather", 1024) - - # cpu + allreduce - def test_cpu_allreduce_1024(self): - self.do_test("cpu", "allreduce", 1024) - - # cpu + broadcast - def test_cpu_broadcast_1024(self): - self.do_test("cpu", "broadcast", 1024) - - # cuda + allgather - def test_cuda_allgather_1024(self): - self.do_test("cuda", "allgather", 1024) - - # cuda + allreduce - def test_cuda_allreduce_1024(self): - self.do_test("cuda", "allreduce", 1024) - - # cuda + broadcast - def test_cuda_broadcast_1024(self): - self.do_test("cuda", "broadcast", 1024) - - -if __name__ == "__main__": - unittest.main() -- 2.34.1 From 218f2ffdcf2ec10bca28c44af7ce994b18288cca Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Mon, 18 May 2026 10:26:56 +0800 Subject: [PATCH 086/382] [] feat: add Hygon DCU/DTK and Iluvatar CoreX platform support (#2118) * feat: add Hygon DCU/DTK and Iluvatar CoreX platform support Add build system and runtime support for two CUDA-compatible domestic accelerator platforms: - Hygon DCU with DTK SDK (USE_HYGON, /opt/dtk/cuda/cuda-11/) - Iluvatar CoreX SDK (USE_COREX, /usr/local/corex/) Both platforms expose CUDA-compatible APIs, so the integration follows the same pattern as existing CUDA-like platforms (MUSA, MACA): add the new macros to all platform guard chains and register SDK paths in CMake. --------- Co-authored-by: KarmaD7 --- README.md | 6 ++ docs/source/getting_started/build.md | 8 +++ docs/source/zh_archive/build.md | 6 ++ mooncake-common/common.cmake | 52 ++++++++++++++++ mooncake-store/include/gpu_staging_utils.h | 18 ++++-- mooncake-store/include/pinned_buffer_pool.h | 6 +- .../example/transfer_engine_bench.cpp | 40 +++++++----- .../transfer_engine_bench_with_notify.cpp | 14 ++--- .../transfer_engine_bench_with_retry.cpp | 10 +-- .../example/transfer_engine_validator.cpp | 18 +++--- mooncake-transfer-engine/include/cuda_alike.h | 6 ++ mooncake-transfer-engine/src/CMakeLists.txt | 10 +++ .../src/memory_location.cpp | 5 +- mooncake-transfer-engine/src/topology.cpp | 10 +-- .../transport/tcp_transport/tcp_transport.cpp | 62 +++++++++++-------- .../tests/rdma_transport_test.cpp | 36 ++++++----- .../tests/tcp_transport_test.cpp | 4 +- 17 files changed, 218 insertions(+), 93 deletions(-) diff --git a/README.md b/README.md index 5b392f03..9a5ab695 100644 --- a/README.md +++ b/README.md @@ -126,6 +126,8 @@ Mooncake supports heterogeneous accelerators, NICs, and specialized transport pa | T-Head | PPU / Barex | Supported | T-Head PPU deployments are represented here through Barex-based transport support | | NVIDIA | CUDA GPUs / NVLink | Supported | `-DUSE_CUDA=ON`, `-DUSE_INTRA_NVLINK=ON`, `-DUSE_MNNVL=ON`; covers CUDA memory, GPUDirect RDMA, GPUDirect Storage, intra-node NVLink, and multi-node NVLink | | AMD | ROCm / HIP GPUs | Supported | `-DUSE_HIP=ON`; HIP transport for AMD GPU communication | +| Hygon | DCU / DTK | Supported | `-DUSE_HYGON=ON`; CUDA-compatible runtime via Hygon DTK SDK | +| Iluvatar | CoreX | Supported | `-DUSE_COREX=ON`; CUDA-compatible runtime via Iluvatar CoreX SDK | #### Network and fabric support @@ -236,6 +238,8 @@ The following need to be installed before running any component of Mooncake: - Python 3.10, virtual environment is recommended. - CUDA 12.1 and above, including NVIDIA GPUDirect Storage Support, if the package is built with `-DUSE_CUDA` (disabled by default). *You may install them from [here](https://developer.nvidia.com/cuda-downloads)*. - Cambricon Neuware, if the package is built with `-DUSE_MLU`. By default Mooncake looks for Neuware under `NEUWARE_HOME` or `/usr/local/neuware`. +- Hygon DTK SDK, if the package is built with `-DUSE_HYGON`. By default Mooncake looks for DTK under `DTK_HOME` or `/opt/dtk`. +- Iluvatar CoreX SDK, if the package is built with `-DUSE_COREX`. By default Mooncake looks for CoreX under `COREX_HOME` or `/usr/local/corex`. ### Use Python package The simplest way to use Mooncake Transfer Engine is using `pip`: @@ -310,6 +314,8 @@ The following are additional dependencies for building Mooncake: - Go 1.20+, if you want to build with `-DWITH_P2P_STORE`, `-DUSE_ETCD` (enabled by default to use etcd as metadata servers), or `-DSTORE_USE_ETCD` (use etcd for the failover of the store master). - CUDA 12.1 and above, including NVIDIA GPUDirect Storage Support, if the package is built with `-DUSE_CUDA`. *This is NOT included in the `dependencies.sh` script. You may install them from [here](https://developer.nvidia.com/cuda-downloads)*. - Cambricon Neuware, if you want to build with `-DUSE_MLU`. *This is NOT included in the `dependencies.sh` script.* Mooncake resolves it from `NEUWARE_HOME` or `/usr/local/neuware` by default, and also supports overriding `MLU_INCLUDE_DIR` / `MLU_LIB_DIR` during CMake configure. +- Hygon DTK SDK, if you want to build with `-DUSE_HYGON`. *This is NOT included in the `dependencies.sh` script.* Mooncake resolves it from `DTK_HOME` or `/opt/dtk` by default, and also supports overriding `DTK_INCLUDE_DIR` / `DTK_LIB_DIR` during CMake configure. +- Iluvatar CoreX SDK, if you want to build with `-DUSE_COREX`. *This is NOT included in the `dependencies.sh` script.* Mooncake resolves it from `COREX_HOME` or `/usr/local/corex` by default, and also supports overriding `COREX_INCLUDE_DIR` / `COREX_LIB_DIR` during CMake configure. - [Optional] Rust Toolchain and libclang, if you want to build Transfer Engine Rust examples with `-DWITH_RUST_EXAMPLE=ON` or Mooncake Store Rust bindings with `-DWITH_STORE_RUST=ON`. *This is NOT included in the `dependencies.sh` script.* - [Optional] `hiredis`, if you want to build with `-DUSE_REDIS` to use Redis instead of etcd as metadata servers. - [Optional] `curl`, if you want to build with `-DUSE_HTTP` to use HTTP instead of etcd as metadata servers. diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index 77164b1d..790dbf13 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -228,6 +228,14 @@ The following options can be used during `cmake ..` to specify whether to compil - `-DMACA_LIB_DIR=/path/to/lib64`: Override MACA library directory when `-DUSE_MACA=ON`. - `-DMACA_RUNTIME_LIBS="mcruntime;mxc-runtime64;rt"`: Override MACA runtime libraries linked by `transfer_engine`. - `-DUSE_HIP=[ON|OFF]`: Enable AMD GPU support via HIP/ROCm +- `-DUSE_HYGON=[ON|OFF]`: Enable Hygon DCU support via DTK SDK. **Default: OFF.** Uses CUDA-compatible runtime. +- `-DDTK_ROOT=/path/to/dtk`: Override the default DTK SDK root used when `-DUSE_HYGON=ON`. If unset, Mooncake uses `DTK_HOME` or `/opt/dtk`. +- `-DDTK_INCLUDE_DIR=/path/to/include`: Override the DTK include directory when `-DUSE_HYGON=ON`. +- `-DDTK_LIB_DIR=/path/to/lib64`: Override the DTK library directory when `-DUSE_HYGON=ON`. +- `-DUSE_COREX=[ON|OFF]`: Enable Iluvatar CoreX GPU support. **Default: OFF.** Uses CUDA-compatible runtime. +- `-DCOREX_ROOT=/path/to/corex`: Override the default CoreX SDK root used when `-DUSE_COREX=ON`. If unset, Mooncake uses `COREX_HOME` or `/usr/local/corex`. +- `-DCOREX_INCLUDE_DIR=/path/to/include`: Override the CoreX include directory when `-DUSE_COREX=ON`. +- `-DCOREX_LIB_DIR=/path/to/lib`: Override the CoreX library directory when `-DUSE_COREX=ON`. - `-DUSE_MLU=[ON|OFF]`: Enable Cambricon MLU memory support via Neuware. **Default: OFF.** Supports MLU memory detection, topology discovery, and RDMA registration for Transfer Engine. - `-DNEUWARE_ROOT=/path/to/neuware`: Override the default Neuware SDK root used when `-DUSE_MLU=ON`. If unset, Mooncake uses `NEUWARE_HOME` or `/usr/local/neuware`. - `-DMLU_INCLUDE_DIR=/path/to/include`: Override the Neuware include directory when `-DUSE_MLU=ON`. diff --git a/docs/source/zh_archive/build.md b/docs/source/zh_archive/build.md index 011e0ab8..6e4f8dfb 100644 --- a/docs/source/zh_archive/build.md +++ b/docs/source/zh_archive/build.md @@ -180,6 +180,12 @@ - `-DNEUWARE_ROOT=/path/to/neuware`: 在 `-DUSE_MLU=ON` 时覆盖默认 Neuware SDK 根路径;未设置时使用 `NEUWARE_HOME` 或 `/usr/local/neuware`。 - `-DMLU_INCLUDE_DIR=/path/to/include` / `-DMLU_LIB_DIR=/path/to/lib64`: 在 `-DUSE_MLU=ON` 时覆盖 Neuware 头文件与库目录。 - `-DUSE_HIP=[ON|OFF]`: 通过 HIP/ROCm 启用对 AMD GPU 的支持 +- `-DUSE_HYGON=[ON|OFF]`: 通过 DTK SDK 启用对海光 DCU 的支持。默认 OFF;使用 CUDA 兼容运行时。 +- `-DDTK_ROOT=/path/to/dtk`: 在 `-DUSE_HYGON=ON` 时覆盖默认 DTK SDK 根路径;未设置时使用 `DTK_HOME` 或 `/opt/dtk`。 +- `-DDTK_INCLUDE_DIR=/path/to/include` / `-DDTK_LIB_DIR=/path/to/lib64`: 在 `-DUSE_HYGON=ON` 时覆盖 DTK 头文件与库目录。 +- `-DUSE_COREX=[ON|OFF]`: 启用对天数智芯 CoreX GPU 的支持。默认 OFF;使用 CUDA 兼容运行时。 +- `-DCOREX_ROOT=/path/to/corex`: 在 `-DUSE_COREX=ON` 时覆盖默认 CoreX SDK 根路径;未设置时使用 `COREX_HOME` 或 `/usr/local/corex`。 +- `-DCOREX_INCLUDE_DIR=/path/to/include` / `-DCOREX_LIB_DIR=/path/to/lib`: 在 `-DUSE_COREX=ON` 时覆盖 CoreX 头文件与库目录。 - `-DUSE_CXL=[ON|OFF]`: 启用 CXL 支持 - `-DWITH_STORE=[ON|OFF]`: 编译 Mooncake Store 组件 - `-DWITH_P2P_STORE=[ON|OFF]`: 启用 Golang 支持并编译 P2P Store 组件,需要 go 1.23+ diff --git a/mooncake-common/common.cmake b/mooncake-common/common.cmake index 6f45422e..315e364b 100644 --- a/mooncake-common/common.cmake +++ b/mooncake-common/common.cmake @@ -65,6 +65,8 @@ option(USE_MLU "option for enabling Cambricon MLU features" OFF) option(USE_MUSA "option for enabling gpu features for MTHREADS GPU" OFF) option(USE_MACA "option for enabling gpu features for MUXI GPU with MACA" OFF) option(USE_HIP "option for enabling gpu features for AMD GPU" OFF) +option(USE_HYGON "option for enabling gpu features for Hygon DCU with DTK" OFF) +option(USE_COREX "option for enabling gpu features for Iluvatar CoreX" OFF) option(USE_NVMEOF "option for using NVMe over Fabric" OFF) option(USE_TCP "option for using TCP transport" ON) option(USE_BAREX "option for using accl-barex transport" OFF) @@ -223,6 +225,56 @@ if (USE_MUSA) ) endif() +if (USE_HYGON) + if (NOT DEFINED DTK_ROOT OR DTK_ROOT STREQUAL "") + if (DEFINED ENV{DTK_HOME} AND NOT "$ENV{DTK_HOME}" STREQUAL "") + set(DTK_ROOT "$ENV{DTK_HOME}" CACHE PATH "Path to Hygon DTK SDK" FORCE) + else() + set(DTK_ROOT "/opt/dtk" CACHE PATH "Path to Hygon DTK SDK" FORCE) + endif() + endif() + + if (NOT DEFINED DTK_INCLUDE_DIR OR DTK_INCLUDE_DIR STREQUAL "") + set(DTK_INCLUDE_DIR "${DTK_ROOT}/cuda/cuda-11/include") + endif() + + if (NOT DEFINED DTK_LIB_DIR OR DTK_LIB_DIR STREQUAL "") + set(DTK_LIB_DIR "${DTK_ROOT}/cuda/cuda-11/lib64") + endif() + + add_compile_definitions(USE_HYGON) + message(STATUS "Hygon DCU/DTK support is enabled") + include_directories(${DTK_INCLUDE_DIR}) + if (EXISTS "${DTK_LIB_DIR}") + link_directories(${DTK_LIB_DIR}) + endif() +endif() + +if (USE_COREX) + if (NOT DEFINED COREX_ROOT OR COREX_ROOT STREQUAL "") + if (DEFINED ENV{COREX_HOME} AND NOT "$ENV{COREX_HOME}" STREQUAL "") + set(COREX_ROOT "$ENV{COREX_HOME}" CACHE PATH "Path to Iluvatar CoreX SDK" FORCE) + else() + set(COREX_ROOT "/usr/local/corex" CACHE PATH "Path to Iluvatar CoreX SDK" FORCE) + endif() + endif() + + if (NOT DEFINED COREX_INCLUDE_DIR OR COREX_INCLUDE_DIR STREQUAL "") + set(COREX_INCLUDE_DIR "${COREX_ROOT}/include") + endif() + + if (NOT DEFINED COREX_LIB_DIR OR COREX_LIB_DIR STREQUAL "") + set(COREX_LIB_DIR "${COREX_ROOT}/lib") + endif() + + add_compile_definitions(USE_COREX) + message(STATUS "Iluvatar CoreX support is enabled") + include_directories(${COREX_INCLUDE_DIR}) + if (EXISTS "${COREX_LIB_DIR}") + link_directories(${COREX_LIB_DIR}) + endif() +endif() + if (USE_HIP) list(APPEND CMAKE_PREFIX_PATH "/opt/rocm/lib/cmake") find_package(HIP REQUIRED) diff --git a/mooncake-store/include/gpu_staging_utils.h b/mooncake-store/include/gpu_staging_utils.h index 375687d0..07982aa0 100644 --- a/mooncake-store/include/gpu_staging_utils.h +++ b/mooncake-store/include/gpu_staging_utils.h @@ -15,7 +15,8 @@ namespace gpu_staging { // Detect whether ptr resides in accelerator device memory. // If so, writes the device ID to *out_device_id for subsequent SetDevice. inline bool IsDevicePointer(const void* ptr, int* out_device_id) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) || \ + defined(USE_HYGON) || defined(USE_COREX) cudaPointerAttributes attr{}; if (cudaPointerGetAttributes(&attr, ptr) == cudaSuccess && attr.type == cudaMemoryTypeDevice) { @@ -45,7 +46,8 @@ inline bool IsDevicePointer(const void* ptr, int* out_device_id) { // Copy device memory to host. Caller must have called SetDevice first. inline bool CopyDeviceToHost(void* dst, const void* src, size_t size) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) || \ + defined(USE_HYGON) || defined(USE_COREX) return cudaMemcpy(dst, src, size, cudaMemcpyDeviceToHost) == cudaSuccess; #elif defined(USE_HIP) return hipMemcpy(dst, src, size, hipMemcpyDeviceToHost) == hipSuccess; @@ -64,7 +66,8 @@ inline bool CopyDeviceToHost(void* dst, const void* src, size_t size) { // attributes (cudaMemcpyDefault). Works for H2H, H2D, D2H, and D2D. // Caller must have called SetDevice first when device memory is involved. inline bool CopyAuto(void* dst, const void* src, size_t size) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) || \ + defined(USE_HYGON) || defined(USE_COREX) return cudaMemcpy(dst, src, size, cudaMemcpyDefault) == cudaSuccess; #elif defined(USE_HIP) return hipMemcpy(dst, src, size, hipMemcpyDefault) == hipSuccess; @@ -94,7 +97,8 @@ inline bool CopyAuto(void* dst, const void* src, size_t size) { // Bind the calling thread to the given device context. inline void SetDevice(int device_id) { if (device_id < 0) return; -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) || \ + defined(USE_HYGON) || defined(USE_COREX) cudaSetDevice(device_id); #elif defined(USE_HIP) hipSetDevice(device_id); @@ -105,7 +109,8 @@ inline void SetDevice(int device_id) { // Copy host memory to device. Caller must have called SetDevice first. inline bool CopyHostToDevice(void* dst, const void* src, size_t size) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) || \ + defined(USE_HYGON) || defined(USE_COREX) return cudaMemcpy(dst, src, size, cudaMemcpyHostToDevice) == cudaSuccess; #elif defined(USE_HIP) return hipMemcpy(dst, src, size, hipMemcpyHostToDevice) == hipSuccess; @@ -128,7 +133,8 @@ inline bool CopyHostToDevice(void* dst, const void* src, size_t size) { // // Pageable host memory (not tracked by CUDA runtime) is treated as host. inline bool IsHostPointer(const void* ptr) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) || \ + defined(USE_HYGON) || defined(USE_COREX) cudaPointerAttributes attr{}; if (cudaPointerGetAttributes(&attr, ptr) != cudaSuccess) { // Query failed: pageable host memory not tracked by the runtime. diff --git a/mooncake-store/include/pinned_buffer_pool.h b/mooncake-store/include/pinned_buffer_pool.h index 63f27a5c..11cdb9d8 100644 --- a/mooncake-store/include/pinned_buffer_pool.h +++ b/mooncake-store/include/pinned_buffer_pool.h @@ -83,7 +83,8 @@ class PinnedBufferPool { buf.capacity = size; buf.is_pinned = false; -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) || \ + defined(USE_HYGON) || defined(USE_COREX) if (cudaMallocHost(reinterpret_cast(&buf.data), size) == cudaSuccess) { buf.is_pinned = true; @@ -119,7 +120,8 @@ class PinnedBufferPool { delete[] buf.data; return; } -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_MACA) || \ + defined(USE_HYGON) || defined(USE_COREX) cudaFreeHost(buf.data); #elif defined(USE_HIP) hipHostFree(buf.data); diff --git a/mooncake-transfer-engine/example/transfer_engine_bench.cpp b/mooncake-transfer-engine/example/transfer_engine_bench.cpp index cdd6d676..5c653227 100644 --- a/mooncake-transfer-engine/example/transfer_engine_bench.cpp +++ b/mooncake-transfer-engine/example/transfer_engine_bench.cpp @@ -42,8 +42,9 @@ #endif #endif -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) || \ + defined(USE_UBSHMEM) || defined(USE_SUNRISE) #include #if defined(USE_MNNVL) || defined(USE_UBSHMEM) @@ -107,8 +108,9 @@ DEFINE_string(report_unit, "GB", "Report unit: GB|GiB|Gb|MB|MiB|Mb|KB|KiB|Kb"); DEFINE_uint32(report_precision, 2, "Report precision"); DEFINE_string(backend, "classic", "Backend to use: classic|tent"); -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) || \ + defined(USE_UBSHMEM) || defined(USE_SUNRISE) DEFINE_bool(use_vram, true, "Allocate memory from GPU/NPU VRAM"); DEFINE_bool(init_mem, true, "Initialize allocated memory"); DEFINE_int32(gpu_id, 0, @@ -119,8 +121,9 @@ using namespace mooncake; static void* allocateMemoryPool(size_t size, int buffer_id, bool from_vram = false) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) || \ + defined(USE_UBSHMEM) || defined(USE_SUNRISE) if (from_vram) { int gpu_id; if (FLAGS_gpu_id == -1) { @@ -190,8 +193,9 @@ static void* allocateMemoryPool(size_t size, int buffer_id, } static void freeMemoryPool(void* addr, size_t size) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) || \ + defined(USE_UBSHMEM) || defined(USE_SUNRISE) if (FLAGS_protocol == "nvlink" || FLAGS_protocol == "hip") { #ifdef USE_MNNVL if (FLAGS_use_vram) { @@ -290,8 +294,9 @@ std::atomic total_batch_count(0); // Ensure each worker thread has a valid GPU context before issuing transfers. static inline void setWorkerDeviceIfNeeded() { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_SUNRISE) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) || \ + defined(USE_SUNRISE) if (FLAGS_use_vram && FLAGS_gpu_id >= 0) { checkCudaError(cudaSetDevice(FLAGS_gpu_id), "Failed to set device in worker"); @@ -301,8 +306,9 @@ static inline void setWorkerDeviceIfNeeded() { // Common helper to determine buffer count based on GPU/NUMA configuration static int determineBufferCount() { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_SUNRISE) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) || \ + defined(USE_SUNRISE) if (FLAGS_use_vram) { int gpu_num; LOG(INFO) << "VRAM is used"; @@ -332,8 +338,9 @@ static int determineBufferCount() { static std::vector allocateBuffers() { buffer_num = determineBufferCount(); std::vector addr(buffer_num); -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) || \ + defined(USE_UBSHMEM) || defined(USE_SUNRISE) for (int i = 0; i < buffer_num; ++i) { addr[i] = allocateMemoryPool(FLAGS_buffer_size, i, FLAGS_use_vram); } @@ -355,8 +362,9 @@ static void freeBuffers(std::vector& addr) { // Helper to get location name for classic backend static std::string getLocationName(int buffer_id) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) || defined(USE_UBSHMEM) || defined(USE_SUNRISE) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) || \ + defined(USE_UBSHMEM) || defined(USE_SUNRISE) if (FLAGS_use_vram) { int name_suffix = (FLAGS_gpu_id == -1) ? buffer_id : FLAGS_gpu_id; return std::string(GPU_PREFIX) + std::to_string(name_suffix); diff --git a/mooncake-transfer-engine/example/transfer_engine_bench_with_notify.cpp b/mooncake-transfer-engine/example/transfer_engine_bench_with_notify.cpp index 07ead7d8..2d76210f 100644 --- a/mooncake-transfer-engine/example/transfer_engine_bench_with_notify.cpp +++ b/mooncake-transfer-engine/example/transfer_engine_bench_with_notify.cpp @@ -38,7 +38,7 @@ #endif #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) #include #ifdef USE_MNNVL @@ -59,7 +59,7 @@ static void checkCudaError(cudaError_t result, const char *message) { #endif #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) const static int NR_SOCKETS = 4; #else const static int NR_SOCKETS = @@ -92,7 +92,7 @@ DEFINE_string(report_unit, "GB", "Report unit: GB|GiB|Gb|MB|MiB|Mb|KB|KiB|Kb"); DEFINE_uint32(report_precision, 2, "Report precision"); #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) DEFINE_bool(use_vram, true, "Allocate memory from GPU VRAM"); DEFINE_bool(init_mem, true, "Initialize allocated memory"); DEFINE_int32(gpu_id, 0, "GPU ID to use"); @@ -103,7 +103,7 @@ using namespace mooncake; static void *allocateMemoryPool(size_t size, int socket_id, bool from_vram = false) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (from_vram) { int gpu_id = FLAGS_gpu_id; void *d_buf; @@ -129,7 +129,7 @@ static void *allocateMemoryPool(size_t size, int socket_id, static void freeMemoryPool(void *addr, size_t size) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) #ifdef USE_MNNVL if (FLAGS_use_vram) { freeFabricMemory(addr); @@ -342,7 +342,7 @@ int initiator() { int buffer_num = NR_SOCKETS; #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (FLAGS_use_vram) LOG(INFO) << "VRAM is used"; for (int i = 0; i < buffer_num; ++i) { addr[i] = allocateMemoryPool(FLAGS_buffer_size, i, FLAGS_use_vram); @@ -442,7 +442,7 @@ int target() { int buffer_num = NR_SOCKETS; #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (FLAGS_use_vram) LOG(INFO) << "VRAM is used"; for (int i = 0; i < buffer_num; ++i) { addr[i] = allocateMemoryPool(FLAGS_buffer_size, i, FLAGS_use_vram); diff --git a/mooncake-transfer-engine/example/transfer_engine_bench_with_retry.cpp b/mooncake-transfer-engine/example/transfer_engine_bench_with_retry.cpp index c7011633..c0e2d313 100644 --- a/mooncake-transfer-engine/example/transfer_engine_bench_with_retry.cpp +++ b/mooncake-transfer-engine/example/transfer_engine_bench_with_retry.cpp @@ -36,7 +36,7 @@ #endif #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) #include static void checkCudaError(cudaError_t result, const char *message) { @@ -77,7 +77,7 @@ DEFINE_string(report_unit, "GB", "Report unit: GB|GiB|Gb|MB|MiB|Mb|KB|KiB|Kb"); DEFINE_uint32(report_precision, 2, "Report precision"); #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) DEFINE_bool(use_vram, true, "Allocate memory from GPU VRAM"); DEFINE_bool(init_mem, true, "Initialize allocated memory"); DEFINE_int32(gpu_id, 0, "GPU ID to use"); @@ -88,7 +88,7 @@ using namespace mooncake; static void *allocateMemoryPool(size_t size, int socket_id, bool from_vram = false) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (from_vram) { int gpu_id = FLAGS_gpu_id; void *d_buf; @@ -111,7 +111,7 @@ static void *allocateMemoryPool(size_t size, int socket_id, static void freeMemoryPool(void *addr, size_t size) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) // check pointer on GPU cudaPointerAttributes attributes; checkCudaError(cudaPointerGetAttributes(&attributes, addr), @@ -304,7 +304,7 @@ int initiator() { int buffer_num = NR_SOCKETS; #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) buffer_num = FLAGS_use_vram ? 1 : NR_SOCKETS; if (FLAGS_use_vram) LOG(INFO) << "VRAM is used"; for (int i = 0; i < buffer_num; ++i) { diff --git a/mooncake-transfer-engine/example/transfer_engine_validator.cpp b/mooncake-transfer-engine/example/transfer_engine_validator.cpp index fd6406b5..0756012a 100644 --- a/mooncake-transfer-engine/example/transfer_engine_validator.cpp +++ b/mooncake-transfer-engine/example/transfer_engine_validator.cpp @@ -38,7 +38,7 @@ #endif #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) #include #ifdef USE_MNNVL @@ -88,7 +88,7 @@ DEFINE_string(report_unit, "GB", "Report unit: GB|GiB|Gb|MB|MiB|Mb|KB|KiB|Kb"); DEFINE_uint32(report_precision, 2, "Report precision"); #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) DEFINE_bool(use_vram, true, "Allocate memory from GPU VRAM"); DEFINE_int32(gpu_id, 0, "GPU ID to use, -1 for all GPUs"); #endif @@ -98,7 +98,7 @@ using namespace mooncake; static void *allocateMemoryPool(size_t size, int buffer_id, bool from_vram = false) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (from_vram) { int gpu_id; if (FLAGS_gpu_id == -1) { @@ -123,7 +123,7 @@ static void *allocateMemoryPool(size_t size, int buffer_id, static void freeMemoryPool(void *addr, size_t size) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) #ifdef USE_MNNVL if (FLAGS_use_vram) { freeFabricMemory(addr); @@ -224,7 +224,7 @@ Status submitRequestSync(TransferEngine *engine, SegmentID handle, } #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) class PinnedBuffer { public: explicit PinnedBuffer(size_t size) : size_(size), ptr_(nullptr) { @@ -260,7 +260,7 @@ thread_local std::vector user_buf(FLAGS_block_size); void fillData(int thread_id, void *addr, uint8_t seed) { #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) memset(ref_buf.data(), seed, FLAGS_block_size); cudaStream_t s; cudaStreamCreate(&s); @@ -290,7 +290,7 @@ void checkData(int thread_id, void *addr, uint8_t seed) { (uint8_t *)(addr) + FLAGS_block_size * (i * FLAGS_threads + thread_id); #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) cudaStream_t s; cudaStreamCreate(&s); cudaMemcpyAsync(user_buf.data(), local_addr, FLAGS_block_size, @@ -425,7 +425,7 @@ int initiator() { std::vector addr; #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (FLAGS_use_vram) { int gpu_num; LOG(INFO) << "VRAM is used"; @@ -549,7 +549,7 @@ int target() { std::vector addr; #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (FLAGS_use_vram) { int gpu_num; LOG(INFO) << "VRAM is used"; diff --git a/mooncake-transfer-engine/include/cuda_alike.h b/mooncake-transfer-engine/include/cuda_alike.h index 07d56c80..2ec35f41 100644 --- a/mooncake-transfer-engine/include/cuda_alike.h +++ b/mooncake-transfer-engine/include/cuda_alike.h @@ -15,6 +15,12 @@ #include "gpu_vendor/maca.h" #elif defined(USE_SUNRISE) #include "gpu_vendor/sunrise.h" +#elif defined(USE_HYGON) +#include +#include +#elif defined(USE_COREX) +#include +#include #endif #if !defined(USE_HIP) && !defined(USE_MUSA) && !defined(USE_MLU) && \ diff --git a/mooncake-transfer-engine/src/CMakeLists.txt b/mooncake-transfer-engine/src/CMakeLists.txt index 95590dc6..bf7b1342 100644 --- a/mooncake-transfer-engine/src/CMakeLists.txt +++ b/mooncake-transfer-engine/src/CMakeLists.txt @@ -82,6 +82,16 @@ if(USE_MUSA) target_link_libraries(transfer_engine PUBLIC musa musart rt) endif() +if(USE_HYGON) + target_include_directories(transfer_engine PRIVATE ${DTK_INCLUDE_DIR}) + target_link_libraries(transfer_engine PUBLIC cuda cudart rt) +endif() + +if(USE_COREX) + target_include_directories(transfer_engine PRIVATE ${COREX_INCLUDE_DIR}) + target_link_libraries(transfer_engine PUBLIC cuda cudart rt) +endif() + if(USE_HIP) target_include_directories(transfer_engine PRIVATE ${HIP_INCLUDE_DIRS}) target_link_libraries(transfer_engine PUBLIC hip::host rt) diff --git a/mooncake-transfer-engine/src/memory_location.cpp b/mooncake-transfer-engine/src/memory_location.cpp index 321a63c3..ed756a10 100644 --- a/mooncake-transfer-engine/src/memory_location.cpp +++ b/mooncake-transfer-engine/src/memory_location.cpp @@ -35,8 +35,9 @@ const std::vector getMemoryLocation(void *start, bool only_first_page) { std::vector entries; -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) cudaPointerAttributes attributes; cudaError_t result = cudaPointerGetAttributes(&attributes, start); if (result != cudaSuccess) { diff --git a/mooncake-transfer-engine/src/topology.cpp b/mooncake-transfer-engine/src/topology.cpp index 2037b3f9..66425634 100644 --- a/mooncake-transfer-engine/src/topology.cpp +++ b/mooncake-transfer-engine/src/topology.cpp @@ -337,8 +337,9 @@ static std::vector discoverCpuTopology( return topology; } -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) static int getPciDistance(const char *bus1, const char *bus2) { char buf[PATH_MAX]; @@ -476,8 +477,9 @@ int Topology::discover(const std::vector &filter) { for (auto &ent : discoverCpuTopology(all_hca)) { matrix_[ent.name] = ent; } -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) for (auto &ent : discoverCudaTopology(all_hca)) { matrix_[ent.name] = ent; } diff --git a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp index 44ea42af..e439d903 100644 --- a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp +++ b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp @@ -44,8 +44,9 @@ struct SessionHeader { uint8_t opcode; }; -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) static bool isCudaMemory(void* addr) { cudaPointerAttributes attributes; auto status = cudaPointerGetAttributes(&attributes, addr); @@ -119,8 +120,9 @@ struct ServerSession : public std::enable_shared_from_this { char* dram_buffer = addr + total_transferred_bytes_; -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) if (isCudaMemory(addr)) { dram_buffer = new char[buffer_size]; cudaError_t cuda_status = @@ -141,8 +143,9 @@ struct ServerSession : public std::enable_shared_from_this { *socket_, asio::buffer(dram_buffer, buffer_size), [this, addr, dram_buffer, self](const asio::error_code& ec, std::size_t transferred_bytes) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) if (isCudaMemory(addr)) { delete[] dram_buffer; } @@ -178,8 +181,9 @@ struct ServerSession : public std::enable_shared_from_this { char* dram_buffer = addr + total_transferred_bytes_; -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) bool is_cuda_memory = isCudaMemory(addr); if (is_cuda_memory) { dram_buffer = new char[buffer_size]; @@ -205,15 +209,17 @@ struct ServerSession : public std::enable_shared_from_this { << " (value: " << ec.value() << ")"; } session_mutex_.unlock(); -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) if (is_cuda_memory) delete[] dram_buffer; #endif return; // Connection will be closed } -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) if (is_cuda_memory) { cudaError_t cuda_status = cudaMemcpy(addr + total_transferred_bytes_, dram_buffer, @@ -268,9 +274,10 @@ struct ClientSession : public std::enable_shared_from_this { *socket_, asio::buffer(&header_, sizeof(SessionHeader)), [this, self](const asio::error_code& ec, std::size_t len) { if (ec || len != sizeof(SessionHeader)) { - LOG(ERROR) << "ClientSession::writeHeader failed. Error: " - << ec.message() << " (value: " << ec.value() - << ")" << ", bytes written: " << len; + LOG(ERROR) + << "ClientSession::writeHeader failed. Error: " + << ec.message() << " (value: " << ec.value() << ")" + << ", bytes written: " << len; if (on_finalize_) on_finalize_(TransferStatusEnum::FAILED); session_mutex_.unlock(); if (on_complete_) on_complete_(); @@ -299,8 +306,9 @@ struct ClientSession : public std::enable_shared_from_this { char* dram_buffer = addr + total_transferred_bytes_; -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) bool is_cuda_memory = isCudaMemory(addr); if (is_cuda_memory) { dram_buffer = new char[buffer_size]; @@ -322,16 +330,18 @@ struct ClientSession : public std::enable_shared_from_this { << " (value: " << ec.value() << ")"; if (on_finalize_) on_finalize_(TransferStatusEnum::FAILED); if (on_complete_) on_complete_(); -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) if (is_cuda_memory) delete[] dram_buffer; #endif session_mutex_.unlock(); return; } -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) if (is_cuda_memory) { cudaError_t cuda_status = cudaMemcpy(addr + total_transferred_bytes_, dram_buffer, @@ -372,8 +382,9 @@ struct ClientSession : public std::enable_shared_from_this { char* dram_buffer = addr + total_transferred_bytes_; -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) if (isCudaMemory(addr)) { dram_buffer = new char[buffer_size]; cudaError_t cuda_status = @@ -396,8 +407,9 @@ struct ClientSession : public std::enable_shared_from_this { *socket_, asio::buffer(dram_buffer, buffer_size), [this, addr, dram_buffer, self](const asio::error_code& ec, std::size_t transferred_bytes) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) if (isCudaMemory(addr)) { delete[] dram_buffer; } diff --git a/mooncake-transfer-engine/tests/rdma_transport_test.cpp b/mooncake-transfer-engine/tests/rdma_transport_test.cpp index e05596e6..a5231c22 100644 --- a/mooncake-transfer-engine/tests/rdma_transport_test.cpp +++ b/mooncake-transfer-engine/tests/rdma_transport_test.cpp @@ -48,8 +48,9 @@ #include #endif -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) #include @@ -100,7 +101,7 @@ std::string pickBackend() { #if defined(USE_MLU) return "mlu"; #elif defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) return FLAGS_use_vram ? "gpu" : "cpu"; #else return "cpu"; @@ -112,7 +113,7 @@ int pickDevId(const std::string &backend) { return FLAGS_device_id; } #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (backend == "gpu") { return FLAGS_gpu_id; } @@ -127,7 +128,7 @@ void validateBackend(const std::string &backend) { return; } #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) if (backend == "gpu") { return; } @@ -174,8 +175,9 @@ void setBackendDevice(const std::string &backend) { if (!usesDeviceMemory(backend)) { return; } -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) checkCudaError(cudaSetDevice(pickDevId(backend)), "Failed to set device"); #else LOG(FATAL) << "Device memory backend is not available in this build"; @@ -185,8 +187,9 @@ void setBackendDevice(const std::string &backend) { void *allocateMemoryPool(size_t size, int socket_id, const std::string &backend) { if (usesDeviceMemory(backend)) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) setBackendDevice(backend); void *d_buf = nullptr; checkCudaError(cudaMalloc(&d_buf, size), @@ -202,8 +205,9 @@ void *allocateMemoryPool(size_t size, int socket_id, void freeMemoryPool(void *addr, size_t size, const std::string &backend) { if (usesDeviceMemory(backend)) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) cudaFree(addr); return; #else @@ -216,8 +220,9 @@ void freeMemoryPool(void *addr, size_t size, const std::string &backend) { void copyFromHost(void *dst, const void *src, size_t size, const std::string &backend) { if (usesDeviceMemory(backend)) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) checkCudaError(cudaMemcpy(dst, src, size, cudaMemcpyHostToDevice), "Failed to copy host data to device"); return; @@ -231,8 +236,9 @@ void copyFromHost(void *dst, const void *src, size_t size, void copyToHost(void *dst, const void *src, size_t size, const std::string &backend) { if (usesDeviceMemory(backend)) { -#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MLU) || defined(USE_MACA) +#if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ + defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ + defined(USE_COREX) checkCudaError(cudaMemcpy(dst, src, size, cudaMemcpyDeviceToHost), "Failed to copy device data to host"); return; diff --git a/mooncake-transfer-engine/tests/tcp_transport_test.cpp b/mooncake-transfer-engine/tests/tcp_transport_test.cpp index 0e34a4f7..6fde3ac7 100644 --- a/mooncake-transfer-engine/tests/tcp_transport_test.cpp +++ b/mooncake-transfer-engine/tests/tcp_transport_test.cpp @@ -28,7 +28,7 @@ #endif #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) #include #include "common/base/status.h" @@ -45,7 +45,7 @@ static void checkCudaError(cudaError_t result, const char *message) { #include "transport/transport.h" #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ - defined(USE_MACA) + defined(USE_MACA) || defined(USE_HYGON) || defined(USE_COREX) DEFINE_int32(gpu_id, 0, "GPU ID to use"); #endif -- 2.34.1 From 15e71c29c68daff75906d64421c912fd6bc18572 Mon Sep 17 00:00:00 2001 From: ykwd Date: Mon, 18 May 2026 10:48:47 +0800 Subject: [PATCH 087/382] [chore] Add a missing space in output log (#2104) --- mooncake-store/src/master.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mooncake-store/src/master.cpp b/mooncake-store/src/master.cpp index b0551d42..cd137235 100644 --- a/mooncake-store/src/master.cpp +++ b/mooncake-store/src/master.cpp @@ -123,7 +123,7 @@ DEFINE_string( etcd_endpoints, "", "Endpoints of ETCD server, separated by semicolon, required in HA mode"); DEFINE_int64(client_ttl, mooncake::DEFAULT_CLIENT_LIVE_TTL_SEC, - "Seconds a client stays considered alive after the last heartbeat." + "Seconds a client stays considered alive after the last heartbeat. " "If this TTL elapses without a refresh, the master treats the " "client as disconnected and may unmount its segments"); -- 2.34.1 From da9591685ff3de419e7551c6e029ab14379b125a Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Mon, 18 May 2026 20:14:11 +0800 Subject: [PATCH 088/382] [TENT] Apply TE's changes of RDMA transport (#2102) * [TENT] Refactor RDMA transport with unidirectional endpoint lifecycle This commit aligns TENT RDMA transport with the source implementation, focusing on endpoint lifecycle management, configuration consolidation, and code organization improvements. **Endpoint Lifecycle (Unidirectional)** - Rename enum Status -> EndpointState to avoid ambiguity with mooncake::tent::Status - Define explicit state machine: EP_UNINIT -> EP_HANDSHAKING -> EP_READY -> EP_DESTROYING -> EP_DESTROYED - Remove redundant active_/inactive_time_ members, use status_ for all state judgment - Implement unidirectional lifecycle: endpoints never reset or reuse - Add resetConnection() for marking failed endpoints for destruction - Deprecate reset() to prevent accidental endpoint reuse **Two-Phase QP Destruction** - beginDestroy(): Mark endpoint as EP_DESTROYING, transition QPs to ERR state - finishDestroy(): Wait for inflight WRs to drain, then destroy QPs - Add destroy_start_time_ for timeout enforcement (30s default) - Fix deconstructUnlocked() to maintain EP_DESTROYED state (no rollback to EP_UNINIT) **Endpoint Store Cleanup** - Unify remove() and removeRef() into single remove(RdmaEndPoint*) method - Add terminal state checking in getOrInsert() - auto-remove and recreate - Fix evictOne() to call beginDestroy() before moving to waiting_list - Fix reclaim() to use finishDestroy() instead of getInflightSlices() - Add waiting_list_len_ early return check in FIFO.reclaim() **Configuration Management** - Migrate PCIe Relaxed Ordering from environment variables to config - Add backward compatibility mappings for legacy MC_* environment variables: - MC_NUM_CQ_PER_CTX, MC_NUM_COMP_CHANNELS_PER_CTX, MC_IB_PORT - MC_GID_INDEX, NCCL_IB_GID_INDEX, MC_MAX_CQE_PER_CTX - MC_MAX_EP_PER_CTX, MC_NUM_QP_PER_EP, MC_MAX_SGE, MC_MAX_WR - MC_MAX_INLINE, MC_PKEY_INDEX, MC_MTU, MC_IB_TC - MC_IB_PCI_RELAXED_ORDERING, MC_WORKERS_PER_CTX - MC_SLICE_SIZE, MC_RETRY_CNT, MC_DISABLE_GPU_DIRECT_RDMA - Add RdmaTransport::config() public accessor for config-driven decisions **Code Quality** - Update state checks from CONNECTED -> EP_READY - Simplify status checks by removing active_ dependency - Improve logging for state transitions Co-Authored-By: Claude Opus 4.7 (1M context) * Reformat code * Avoid use RdmaEndpoint::reset() * Fix code issues --------- Co-authored-by: Claude Opus 4.7 (1M context) --- .../include/tent/transport/rdma/context.h | 6 + .../include/tent/transport/rdma/endpoint.h | 39 ++- .../tent/transport/rdma/rdma_transport.h | 2 + .../tent/src/common/config.cpp | 28 +- .../tent/src/transport/rdma/buffers.cpp | 16 +- .../tent/src/transport/rdma/context.cpp | 98 ++++++- .../tent/src/transport/rdma/endpoint.cpp | 267 ++++++++++++------ .../src/transport/rdma/endpoint_store.cpp | 172 ++++++++--- .../tent/src/transport/rdma/rail_monitor.cpp | 45 ++- .../src/transport/rdma/rdma_transport.cpp | 12 +- .../tent/src/transport/rdma/workers.cpp | 24 +- 11 files changed, 538 insertions(+), 171 deletions(-) diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/context.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/context.h index 141fd9d5..2c010872 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/context.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/context.h @@ -114,6 +114,9 @@ class RdmaContext { RdmaParams ¶ms() const { return *params_.get(); } + // PCIe Relaxed Ordering support + bool isRelaxedOrderingEnabled() const { return relaxed_ordering_enabled_; } + // Notification CQ (dedicated for notification QPs) RdmaCQ *notifyCq() { return notify_cq_; } @@ -149,6 +152,9 @@ class RdmaContext { // Dedicated CQ for notification QPs (one per device) RdmaCQ *notify_cq_ = nullptr; + // PCIe Relaxed Ordering support + bool relaxed_ordering_enabled_ = false; + const IbvSymbols &verbs_; }; diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/endpoint.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/endpoint.h index 169d4744..7ef3588b 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/endpoint.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/endpoint.h @@ -81,9 +81,16 @@ class RdmaEndPoint : public std::enable_shared_from_this { ~RdmaEndPoint(); public: - enum EndPointStatus { EP_UNINIT, EP_HANDSHAKING, EP_READY, EP_RESET }; - - int reset(); + // Endpoint lifecycle is unidirectional: once created, endpoints move + // forward through states and never return. Failed or discarded endpoints + // enter EP_DESTROYING and are reclaimed - they are never reset or reused. + enum EndPointStatus { + EP_UNINIT, // Initial state, not yet constructed + EP_HANDSHAKING, // Connection in progress + EP_READY, // Connected and operational + EP_DESTROYING, // Being destroyed (failed or discarded) + EP_DESTROYED, // Fully destroyed + }; int construct(RdmaContext* context, EndPointParams* params, const std::string& endpoint_name, @@ -91,13 +98,27 @@ class RdmaEndPoint : public std::enable_shared_from_this { int deconstruct(); + int resetConnection(const std::string& reason); + + // Two-phase QP destruction to avoid use-after-free in concurrent + // submitPostSend. Phase 1 (beginDestroy): sets status_=EP_DESTROYING, + // transitions QPs to ERR state so hardware flushes inflight WRs to CQ. + // Does not block. Phase 2 (finishDestroy): called after all outstanding + // WRs have been drained, actually destroys QPs and frees resources. + // Returns true if destruction is complete, false if outstanding WRs remain. + void beginDestroy(); + void beginDestroyNoLock(); // Internal version without locking + bool finishDestroy(); + Status connect(const std::string& peer_server_name, const std::string& peer_nic_name, const std::string& peer_rpc_server_addr = ""); Status accept(const BootstrapDesc& peer_desc, BootstrapDesc& local_desc); - EndPointStatus status() const { return status_; } + EndPointStatus status() const { + return status_.load(std::memory_order_relaxed); + } std::vector qpNum(); @@ -135,8 +156,6 @@ class RdmaEndPoint : public std::enable_shared_from_this { bool failed; }; - int resetUnlocked(); - int submitSlices(std::vector& slice_list, int qp_index); int submitRecvImmDataRequest(int qp_index, uint64_t id); @@ -183,6 +202,9 @@ class RdmaEndPoint : public std::enable_shared_from_this { uint32_t padding_[7]; RWSpinlock lock_; + // Two-phase destruction: timestamp when EP entered EP_DESTROYING + std::atomic destroy_start_time_; + std::string peer_server_name_; std::string peer_nic_name_; std::vector peer_qp_num_list_; @@ -203,6 +225,11 @@ class RdmaEndPoint : public std::enable_shared_from_this { int notify_pending_count_ = 0; // Number of pending sends uint64_t notify_send_wr_id_ = 0; // Circular counter for wr_id bool notify_connected_ = false; + + // Two-phase destruction constants (matching TE) + static constexpr double kFinishDestroyTimeoutSec = 30.0; + static constexpr int kFinishDestroyMaxRetries = 3; + int finish_destroy_retries_ = 0; // Retry counter for finishDestroy }; } // namespace tent } // namespace mooncake diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h index 0761d57e..84ed23df 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h @@ -114,6 +114,8 @@ class RdmaTransport : public Transport { public: Status setupLocalSegment(); + std::shared_ptr config() const { return conf_; } + private: bool installed_; std::shared_ptr conf_; diff --git a/mooncake-transfer-engine/tent/src/common/config.cpp b/mooncake-transfer-engine/tent/src/common/config.cpp index 0d725dc7..c6703531 100644 --- a/mooncake-transfer-engine/tent/src/common/config.cpp +++ b/mooncake-transfer-engine/tent/src/common/config.cpp @@ -84,9 +84,35 @@ Status ConfigHelper::loadFromEnv(Config& config) { } } - // Legacy keys for backward compatibility + // Legacy keys for backward compatibility (MC_* env vars) + setConfig(config, "MC_NUM_CQ_PER_CTX", + "transports/rdma/device/num_cq_list"); + setConfig(config, "MC_NUM_COMP_CHANNELS_PER_CTX", + "transports/rdma/device/num_comp_channels"); setConfig(config, "MC_IB_PORT", "transports/rdma/device/port"); setConfig(config, "MC_GID_INDEX", "transports/rdma/device/gid_index"); + setConfig(config, "NCCL_IB_GID_INDEX", "transports/rdma/device/gid_index"); + setConfig(config, "MC_MAX_CQE_PER_CTX", "transports/rdma/device/max_cqe"); + setConfig(config, "MC_MAX_EP_PER_CTX", + "transports/rdma/endpoint/endpoint_store_cap"); + setConfig(config, "MC_NUM_QP_PER_EP", + "transports/rdma/endpoint/qp_mul_factor"); + setConfig(config, "MC_MAX_SGE", "transports/rdma/endpoint/max_sge"); + setConfig(config, "MC_MAX_WR", "transports/rdma/endpoint/max_qp_wr"); + setConfig(config, "MC_MAX_INLINE", + "transports/rdma/endpoint/max_inline_bytes"); + setConfig(config, "MC_PKEY_INDEX", "transports/rdma/endpoint/pkey_index"); + setConfig(config, "MC_MTU", "transports/rdma/endpoint/path_mtu"); + setConfig(config, "MC_IB_TC", "transports/rdma/endpoint/traffic_class"); + setConfig(config, "MC_IB_PCI_RELAXED_ORDERING", + "transports/rdma/pci_relaxed_ordering"); + setConfig(config, "MC_WORKERS_PER_CTX", + "transports/rdma/workers/num_workers"); + setConfig(config, "MC_SLICE_SIZE", "transports/rdma/workers/block_size"); + setConfig(config, "MC_RETRY_CNT", + "transports/rdma/workers/max_retry_count"); + setConfig(config, "MC_DISABLE_GPU_DIRECT_RDMA", + "transports/rdma/disable_gpu_direct_rdma"); return status; } diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/buffers.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/buffers.cpp index 20f1d216..0c350890 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/buffers.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/buffers.cpp @@ -78,13 +78,17 @@ LocalBufferManager::LocalBufferManager() {} LocalBufferManager::~LocalBufferManager() { clear(); } -static inline int getAccessFlags(Permission perm) { +static inline int getAccessFlags(Permission perm, + bool relaxed_ordering = false) { int access = IBV_ACCESS_LOCAL_WRITE; if (perm == kGlobalReadWrite) { access |= IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_READ; } else if (perm == kGlobalReadOnly) { access |= IBV_ACCESS_REMOTE_READ; } + if (relaxed_ordering) { + access |= IBV_ACCESS_RELAXED_ORDERING; + } return access; } @@ -98,7 +102,6 @@ Status LocalBufferManager::addBufferInternal(BufferDesc& desc, bool force_sequential) { AddressRange range((void*)desc.addr, desc.length); BufferEntryForRdma staging; - auto access = getAccessFlags(options.perm); assert(desc.rkey.empty()); size_t context_count = 0; for (auto* context : context_list_) { @@ -116,6 +119,9 @@ Status LocalBufferManager::addBufferInternal(BufferDesc& desc, for (size_t id = 0; id < context_list_.size(); ++id) { auto* context = context_list_[id]; if (!context) continue; + // Calculate access flags per context (for relaxed ordering support) + int access = getAccessFlags(options.perm, + context->isRelaxedOrderingEnabled()); tasks.emplace_back(std::async( std::launch::async, [context, &mem_reg_list, id, addr, length, access]() { @@ -128,6 +134,9 @@ Status LocalBufferManager::addBufferInternal(BufferDesc& desc, for (size_t id = 0; id < context_list_.size(); ++id) { auto* context = context_list_[id]; if (!context) continue; + // Calculate access flags per context (for relaxed ordering support) + int access = getAccessFlags(options.perm, + context->isRelaxedOrderingEnabled()); mem_reg_list[id] = context->registerMemReg((void*)desc.addr, desc.length, access); } @@ -203,7 +212,8 @@ Status LocalBufferManager::addDevice(RdmaContext* context) { for (auto& buffer : buffer_list_) { auto range = buffer.first; auto& options = buffer.second.options; - auto access = getAccessFlags(options.perm); + auto access = + getAccessFlags(options.perm, context->isRelaxedOrderingEnabled()); if (buffer.second.mem_reg_map.count(context)) continue; auto mem_reg = context->registerMemReg(range.addr, range.length, access); diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/context.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/context.cpp index a8e360fe..31ea13d4 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/context.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/context.cpp @@ -26,6 +26,13 @@ #include #include #include +#include +#include + +#ifdef USE_CUDA +#include +#include +#endif #include "tent/common/status.h" #include "tent/transport/rdma/endpoint_store.h" @@ -381,6 +388,31 @@ int RdmaContext::enable() { return notify_ret; } + // Check PCIe Relaxed Ordering support from config + // Mode: 0 = disabled, 1 = enabled if supported, 2 = auto (default) + auto mode = + transport_.config()->get("transports/rdma/pci_relaxed_ordering", 1); + if (mode != 0) { + // Check if ibv_reg_mr_iova2 symbol is available (IBVERBS_1.8+) + void* sym = dlsym(RTLD_DEFAULT, "ibv_reg_mr_iova2"); + if (sym) { + relaxed_ordering_enabled_ = true; + LOG(INFO) << "[RDMA] Relaxed ordering is supported and enabled for " + << device_name_; + } else { + if (mode == 1) { + LOG(WARNING) << "[RDMA] Relaxed ordering requested but NOT " + << "supported (ibv_reg_mr_iova2 missing). " + << "Falling back to strict ordering."; + } + relaxed_ordering_enabled_ = false; + } + } else { + LOG(INFO) << "[RDMA] Relaxed ordering disabled via config for " + << device_name_; + relaxed_ordering_enabled_ = false; + } + ibv_port_attr port_attr; int ret = verbs_.ibv_query_port_default(native_context_, params_->device.port, &port_attr); @@ -473,6 +505,66 @@ RdmaContext::MemReg RdmaContext::registerMemReg(void* addr, size_t length, LOG(FATAL) << "RDMA context " << name() << " not constructed"; return nullptr; } + +#ifdef USE_CUDA + // Ensure CUDA context is current for GPU memory registration + // This is needed for worker threads or callers from non-CUDA threads + CUmemorytype memType; + CUresult result = cuPointerGetAttribute( + &memType, CU_POINTER_ATTRIBUTE_MEMORY_TYPE, (CUdeviceptr)addr); + + if (result == CUDA_SUCCESS && memType == CU_MEMORYTYPE_DEVICE) { + // Get device ordinal and set primary context current + unsigned int devOrd = 0; + result = cuPointerGetAttribute( + &devOrd, CU_POINTER_ATTRIBUTE_DEVICE_ORDINAL, (CUdeviceptr)addr); + if (result != CUDA_SUCCESS) { + LOG(ERROR) << "Failed to get CUDA device ordinal: " << result; + return nullptr; + } + + CUdevice cuDev; + result = cuDeviceGet(&cuDev, devOrd); + if (result != CUDA_SUCCESS) { + LOG(ERROR) << "Failed to get CUDA device: " << result; + return nullptr; + } + + CUcontext cuCtx; + result = cuDevicePrimaryCtxRetain(&cuCtx, cuDev); + if (result != CUDA_SUCCESS) { + LOG(ERROR) << "Failed to retain CUDA primary context: " << result; + return nullptr; + } + + result = cuCtxSetCurrent(cuCtx); + if (result != CUDA_SUCCESS) { + LOG(ERROR) << "Failed to set CUDA context current: " << result; + cuDevicePrimaryCtxRelease(cuDev); + return nullptr; + } + + // Register GPU memory + ibv_mr* entry = + verbs_.ibv_reg_mr_default(native_pd_, addr, length, access); + + // Release primary context reference + cuDevicePrimaryCtxRelease(cuDev); + + if (!entry) { + const void* end = static_cast(addr) + length; + PLOG(ERROR) << "Failed to register GPU memory from " << addr + << " to " << end << " in RDMA device " << device_name_; + return nullptr; + } + mr_set_mutex_.lock(); + mr_set_.insert(entry); + mr_set_mutex_.unlock(); + return entry; + } +#endif + + // Standard CPU memory registration ibv_mr* entry = verbs_.ibv_reg_mr_default(native_pd_, addr, length, access); if (!entry) { const void* end = static_cast(addr) + length; @@ -491,8 +583,10 @@ int RdmaContext::warmupMrRegistration(void* addr, size_t length) { LOG(FATAL) << "RDMA context " << name() << " not constructed"; return -1; } - ibv_mr* entry = verbs_.ibv_reg_mr_default(native_pd_, addr, length, - IBV_ACCESS_LOCAL_WRITE); + int access_flags = IBV_ACCESS_LOCAL_WRITE; + if (relaxed_ordering_enabled_) access_flags |= IBV_ACCESS_RELAXED_ORDERING; + ibv_mr* entry = + verbs_.ibv_reg_mr_default(native_pd_, addr, length, access_flags); if (!entry) { PLOG(WARNING) << "ibv_reg_mr warm-up failed on " << device_name_ << " for [" << addr << ", " << length << " bytes]"; diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp index a0326de2..690ed41d 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp @@ -26,6 +26,7 @@ #include "tent/common/status.h" #include "tent/common/types.h" #include "tent/transport/rdma/context.h" +#include "tent/transport/rdma/endpoint_store.h" #include "tent/common/utils/os.h" #include "tent/common/utils/string_builder.h" #include "tent/thirdparty/nlohmann/json.h" @@ -41,8 +42,10 @@ static inline const std::string statusToString( return "EP_HANDSHAKING"; case RdmaEndPoint::EP_READY: return "EP_READY"; - case RdmaEndPoint::EP_RESET: - return "EP_RESET"; + case RdmaEndPoint::EP_DESTROYING: + return "EP_DESTROYING"; + case RdmaEndPoint::EP_DESTROYED: + return "EP_DESTROYED"; } return "UNKNOWN"; } @@ -50,12 +53,13 @@ static inline const std::string statusToString( // Forward declaration for notification QP setup static int setupNotifyQpConnection(ibv_qp* qp, RdmaContext* ctx, const std::string& peer_gid_str, - uint16_t peer_lid, uint32_t peer_qp_num); + uint16_t peer_lid, uint32_t peer_qp_num, + uint16_t pkey_index); RdmaEndPoint::RdmaEndPoint() : status_(EP_UNINIT) {} RdmaEndPoint::~RdmaEndPoint() { - if (status_ != EP_UNINIT) deconstruct(); + if (status_.load(std::memory_order_relaxed) != EP_UNINIT) deconstruct(); if (endpoints_count_) endpoints_count_->fetch_sub(1, std::memory_order_relaxed); } @@ -118,7 +122,7 @@ int RdmaEndPoint::construct(RdmaContext* context, EndPointParams* params, ibv_qp_attr qp_attr; memset(&qp_attr, 0, sizeof(qp_attr)); qp_attr.qp_state = IBV_QPS_INIT; - qp_attr.pkey_index = 0; + qp_attr.pkey_index = params_->pkey_index; qp_attr.port_num = context_->portNum(); qp_attr.qp_access_flags = IBV_ACCESS_REMOTE_WRITE; @@ -160,7 +164,7 @@ int RdmaEndPoint::construct(RdmaContext* context, EndPointParams* params, } } - status_ = EP_HANDSHAKING; + status_.store(EP_HANDSHAKING, std::memory_order_relaxed); return 0; } @@ -170,8 +174,10 @@ int RdmaEndPoint::deconstruct() { } int RdmaEndPoint::deconstructUnlocked() { - if (status_ == EP_UNINIT) return 0; - status_ = EP_RESET; + auto current_status = status_.load(std::memory_order_relaxed); + // Idempotent: if already destroyed or never initialized, skip cleanup + if (current_status == EP_DESTROYED || current_status == EP_UNINIT) return 0; + status_.store(EP_DESTROYED, std::memory_order_relaxed); resetInflightSlices(); peer_qp_num_list_.clear(); @@ -214,10 +220,98 @@ int RdmaEndPoint::deconstructUnlocked() { wr_depth_list_ = nullptr; peer_server_name_.clear(); peer_nic_name_.clear(); - status_ = EP_UNINIT; + // Status remains EP_DESTROYED (unidirectional lifecycle) return 0; } +void RdmaEndPoint::beginDestroy() { + RWSpinlock::WriteGuard guard(lock_); + beginDestroyNoLock(); +} + +void RdmaEndPoint::beginDestroyNoLock() { + auto current_status = status_.load(std::memory_order_relaxed); + if (current_status == EP_DESTROYING || current_status == EP_DESTROYED) + return; + + destroy_start_time_ = getCurrentTimeInNano(); + status_.store(EP_DESTROYING, std::memory_order_release); + + // Transition QPs to ERR state so hardware flushes inflight WRs to CQ + ibv_qp_attr attr; + memset(&attr, 0, sizeof(attr)); + attr.qp_state = IBV_QPS_ERR; + + for (size_t i = 0; i < qp_list_.size(); ++i) { + int ret = + context_->verbs_.ibv_modify_qp(qp_list_[i], &attr, IBV_QP_STATE); + if (ret) { + PLOG(ERROR) << "Failed to modify QP to ERR in beginDestroy"; + } + } +} + +bool RdmaEndPoint::finishDestroy() { + RWSpinlock::WriteGuard guard(lock_); + auto current_status = status_.load(std::memory_order_relaxed); + + // Gate 1: already done + if (current_status == EP_DESTROYED) return true; + + // Gate 2: non-two-phase path. Endpoint reached waiting_list_ without + // going through beginDestroy(). This handles edge cases and serves as + // a safety net. Endpoints that never reached construct() own no RDMA + // resources; drop them directly. + if (current_status != EP_DESTROYING) { + if (qp_list_.empty()) { + status_.store(EP_DESTROYED, std::memory_order_relaxed); + return true; + } + LOG(WARNING) << "finishDestroy called in unexpected state: " + << statusToString(current_status) + << ", forcing destruction to avoid waiting_list_ leak"; + // Fall through to the unified destroy path + } else { + // Gate 3: two-phase path. Wait for inflight WRs to drain via CQ + // polling. If ibv_modify_qp-to-ERR failed in beginDestroy, WRs may + // never be flushed; enforce a timeout to avoid leaking forever. + bool has_outstanding = false; + for (size_t i = 0; i < qp_list_.size(); ++i) { + if (wr_depth_list_[i].value != 0) { + has_outstanding = true; + break; + } + } + if (has_outstanding) { + double elapsed = + (getCurrentTimeInNano() - destroy_start_time_) / 1e9; + if (elapsed < kFinishDestroyTimeoutSec) { + return false; // Still waiting for WRs to drain + } + LOG(WARNING) << "finishDestroy timed out after " << elapsed + << "s with outstanding WRs, forcing destruction"; + } + } + + // Unified destroy: tear down QPs and bound retries to avoid + // log flooding when ibv_destroy_qp fails permanently. + int ret = deconstructUnlocked(); + if (ret) { + finish_destroy_retries_++; + LOG(ERROR) << "Failed to finish destroying endpoint (attempt " + << finish_destroy_retries_ << "/" << kFinishDestroyMaxRetries + << "): " << ret; + if (finish_destroy_retries_ < kFinishDestroyMaxRetries) { + return false; // Retry later + } + LOG(ERROR) << "Giving up after " << finish_destroy_retries_ + << " retries (possible resource leak)"; + } + + status_.store(EP_DESTROYED, std::memory_order_relaxed); + return true; +} + Status RdmaEndPoint::connect(const std::string& peer_server_name, const std::string& peer_nic_name, const std::string& peer_rpc_server_addr) { @@ -234,13 +328,14 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, { RWSpinlock::WriteGuard guard(lock_); if (peer_server_name.empty() || peer_nic_name.empty()) { - return Status::InvalidArgument("Invalid peer path" LOC_MARK); + return mooncake::tent::Status::InvalidArgument( + "Invalid peer path" LOC_MARK); } - if (status_ == EP_READY) { - return Status::OK(); + if (status_.load(std::memory_order_relaxed) == EP_READY) { + return mooncake::tent::Status::OK(); } - if (status_ != EP_HANDSHAKING) { - return Status::InvalidArgument( + if (status_.load(std::memory_order_relaxed) != EP_HANDSHAKING) { + return mooncake::tent::Status::InvalidArgument( "Endpoint not in handshaking state" LOC_MARK); } @@ -270,7 +365,7 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, // Same server, different NIC: call handler directly, bypass RPC int rc = transport.onSetupRdmaConnections(local_desc, peer_desc); if (rc != 0) { - return Status::InternalError( + return mooncake::tent::Status::InternalError( "Local bootstrap failed: " + peer_desc.reply_msg + LOC_MARK); } qp_num = peer_desc.qp_num; @@ -282,22 +377,24 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, if (rpc_server_addr.empty()) { SegmentDescRef segment_desc; auto& manager = transport.metadata_->segmentManager(); - CHECK_STATUS(manager.getRemote(segment_desc, peer_server_name)); + auto status = manager.getRemote(segment_desc, peer_server_name); + if (!status.ok()) return status; rpc_server_addr = segment_desc->rpc_server_addr; } if (rpc_server_addr.empty()) { - return Status::InvalidArgument( + return mooncake::tent::Status::InvalidArgument( "Missing peer RPC server address" LOC_MARK); } - CHECK_STATUS( - ControlClient::bootstrap(rpc_server_addr, local_desc, peer_desc)); + auto bootstrap_status = + ControlClient::bootstrap(rpc_server_addr, local_desc, peer_desc); + if (!bootstrap_status.ok()) return bootstrap_status; qp_num = peer_desc.qp_num; peer_gid = peer_desc.local_gid; peer_lid = peer_desc.local_lid; } if (peer_gid.empty()) { - return Status::InvalidArgument( + return mooncake::tent::Status::InvalidArgument( "Missing peer GID in bootstrap" LOC_MARK); } @@ -306,11 +403,11 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, // connection while we were doing bootstrap without the lock. { RWSpinlock::WriteGuard guard(lock_); - if (status_ == EP_READY) { - return Status::OK(); + if (status_.load(std::memory_order_relaxed) == EP_READY) { + return mooncake::tent::Status::OK(); } - if (status_ != EP_HANDSHAKING) { - return Status::InvalidArgument( + if (status_.load(std::memory_order_relaxed) != EP_HANDSHAKING) { + return mooncake::tent::Status::InvalidArgument( "Endpoint state changed during bootstrap" LOC_MARK); } @@ -319,7 +416,7 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, peer_nic_name_ = peer_nic_name; int rc = setupAllQPs(peer_gid, peer_lid, qp_num); if (rc) { - return Status::InternalError( + return mooncake::tent::Status::InternalError( "Failed to configure RDMA endpoint" LOC_MARK); } peer_qp_num_list_ = qp_num; @@ -327,7 +424,8 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, // Setup notification QP connection if peer supports it if (peer_desc.notify_qp_num != 0 && notify_qp_) { rc = setupNotifyQpConnection(notify_qp_, context_, peer_gid, - peer_lid, peer_desc.notify_qp_num); + peer_lid, peer_desc.notify_qp_num, + params_->pkey_index); if (rc) { LOG(WARNING) << "Failed to setup notification QP, notification disabled"; @@ -340,13 +438,13 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, } } - return Status::OK(); + return mooncake::tent::Status::OK(); } Status RdmaEndPoint::accept(const BootstrapDesc& peer_desc, BootstrapDesc& local_desc) { RWSpinlock::WriteGuard guard(lock_); - if (status_ == EP_READY) { + if (status_.load(std::memory_order_relaxed) == EP_READY) { // Idempotency check: if the incoming bootstrap is from the same peer // and carries the same peer QP list as the established connection, // this is a duplicate request (e.g. from a concurrent connect() on @@ -371,19 +469,22 @@ Status RdmaEndPoint::accept(const BootstrapDesc& peer_desc, local_desc.local_lid = context_->lid(); local_desc.local_gid = context_->gid(); local_desc.notify_qp_num = notifyQpNum(); - return Status::OK(); + return mooncake::tent::Status::OK(); } - LOG(WARNING) << "Endpoint is established with " << peer_nic_name_ - << " of " << peer_server_name_ << ", resetting first"; - resetUnlocked(); - status_ = EP_HANDSHAKING; - } else if (status_ == EP_RESET) { - resetUnlocked(); - status_ = EP_HANDSHAKING; - } else if (status_ != EP_HANDSHAKING) { + // Endpoint already connected to a different peer - reject the request + // instead of resetting. Endpoints have unidirectional lifecycle and + // are never reset or reused. The caller should create a new endpoint. + LOG(ERROR) + << "Endpoint already established with " << peer_nic_name_ << " of " + << peer_server_name_ + << ", cannot accept new connection (unidirectional lifecycle)"; + return mooncake::tent::Status::InternalError( + "Endpoint already connected to different peer" LOC_MARK); + } + if (status_.load(std::memory_order_relaxed) != EP_HANDSHAKING) { LOG(ERROR) << "Endpoint not in handshaking state: " - << statusToString(status_); - return Status::InvalidArgument( + << statusToString(status_.load(std::memory_order_relaxed)); + return mooncake::tent::Status::InvalidArgument( "Endpoint not in handshaking state" LOC_MARK); } auto& transport = context_->transport_; @@ -391,7 +492,8 @@ Status RdmaEndPoint::accept(const BootstrapDesc& peer_desc, auto peer_server_name = getServerNameFromNicPath(peer_nic_path); auto peer_nic_name = getNicNameFromNicPath(peer_nic_path); if (peer_server_name.empty() || peer_nic_name.empty()) - return Status::InvalidArgument("Invalid peer path" LOC_MARK); + return mooncake::tent::Status::InvalidArgument( + "Invalid peer path" LOC_MARK); local_desc.local_nic_path = MakeNicPath(transport.local_segment_name_, context_->name()); local_desc.peer_nic_path = peer_nic_path; @@ -400,7 +502,7 @@ Status RdmaEndPoint::accept(const BootstrapDesc& peer_desc, local_desc.local_gid = context_->gid(); local_desc.notify_qp_num = notifyQpNum(); // Pass notification QP number if (peer_desc.local_gid.empty()) { - return Status::InvalidArgument( + return mooncake::tent::Status::InvalidArgument( "Missing peer GID in bootstrap" LOC_MARK); } peer_server_name_ = peer_server_name; @@ -408,16 +510,16 @@ Status RdmaEndPoint::accept(const BootstrapDesc& peer_desc, int rc = setupAllQPs(peer_desc.local_gid, peer_desc.local_lid, peer_desc.qp_num); if (rc) { - return Status::InternalError( + return mooncake::tent::Status::InternalError( "Failed to configure RDMA endpoint" LOC_MARK); } peer_qp_num_list_ = peer_desc.qp_num; // Setup notification QP connection if peer supports it if (peer_desc.notify_qp_num != 0 && notify_qp_) { - rc = setupNotifyQpConnection(notify_qp_, context_, peer_desc.local_gid, - peer_desc.local_lid, - peer_desc.notify_qp_num); + rc = setupNotifyQpConnection( + notify_qp_, context_, peer_desc.local_gid, peer_desc.local_lid, + peer_desc.notify_qp_num, params_->pkey_index); if (rc) { notify_connected_ = false; } else { @@ -427,54 +529,40 @@ Status RdmaEndPoint::accept(const BootstrapDesc& peer_desc, } } - return Status::OK(); + return mooncake::tent::Status::OK(); } -int RdmaEndPoint::reset() { - RWSpinlock::WriteGuard guard(lock_); - return resetUnlocked(); -} +int RdmaEndPoint::resetConnection(const std::string& reason) { + // Mark endpoint for destruction due to failure or error. + // Endpoints have unidirectional lifecycle - once marked for destruction, + // they are never reused. The endpoint store will create a new endpoint + // for future connections. + RdmaEndPoint* endpoint_ptr = this; -int RdmaEndPoint::resetUnlocked() { - if (status_ != EP_READY) return 0; - status_ = EP_RESET; - peer_qp_num_list_.clear(); - resetInflightSlices(); + { + RWSpinlock::WriteGuard guard(lock_); + auto curr_status = status_.load(std::memory_order_acquire); + if (curr_status == EP_DESTROYING || curr_status == EP_DESTROYED) + return 0; + if (curr_status != EP_HANDSHAKING && curr_status != EP_READY) return 0; - if (notify_qp_) { - // Keep the QP registered in the transport map — the qp_num doesn't - // change across resets, and the notify worker may still poll - // completions (flush errors) from the CQ between reset and reconnect. - // Unregistering here would cause "unknown QP" warnings. - notify_connected_ = false; - { - std::lock_guard lock(notify_send_mutex_); - notify_pending_count_ = 0; - } - notify_send_cv_.notify_all(); + destroy_start_time_ = getCurrentTimeInNano(); + status_.store(EP_DESTROYING, std::memory_order_release); + LOG(INFO) << "Endpoint marked for destruction: " << reason; } - ibv_qp_attr attr; - memset(&attr, 0, sizeof(attr)); - attr.qp_state = IBV_QPS_RESET; - for (size_t i = 0; i < qp_list_.size(); ++i) { - int ret = - context_->verbs_.ibv_modify_qp(qp_list_[i], &attr, IBV_QP_STATE); - if (ret) { - PLOG(ERROR) << "ibv_modify_qp(RESET)"; - deconstructUnlocked(); - return -1; - } - cancelQuota(i, wr_depth_list_[i].value); - } + // Delete from endpoint store so endpoint() won't return this endpoint. + // remove() calls beginDestroyNoLock() to avoid deadlocking when + // caller already holds lock_. + context_->endpointStore()->remove(endpoint_ptr); return 0; } int RdmaEndPoint::setupAllQPs(const std::string& peer_gid, uint16_t peer_lid, std::vector peer_qp_num_list, std::string* reply_msg) { - if (status_ == EP_READY) { - status_ = EP_RESET; + if (status_.load(std::memory_order_relaxed) == EP_READY) { + status_.store(EP_DESTROYING, std::memory_order_relaxed); return -1; } @@ -485,7 +573,7 @@ int RdmaEndPoint::setupAllQPs(const std::string& peer_gid, uint16_t peer_lid, << peer_nic_name_ << " of " << peer_server_name_; LOG(ERROR) << ss.str(); if (reply_msg) *reply_msg = ss.str(); - status_ = EP_RESET; + status_.store(EP_DESTROYING, std::memory_order_relaxed); return -1; } @@ -493,12 +581,12 @@ int RdmaEndPoint::setupAllQPs(const std::string& peer_gid, uint16_t peer_lid, int ret = setupOneQP(qp_index, peer_gid, peer_lid, peer_qp_num_list[qp_index], reply_msg); if (ret) { - status_ = EP_RESET; + status_.store(EP_DESTROYING, std::memory_order_relaxed); return ret; } } - status_ = EP_READY; + status_.store(EP_READY, std::memory_order_relaxed); return 0; } @@ -520,7 +608,8 @@ int RdmaEndPoint::submitSlices(std::vector& slice_list, if (qp_list_.empty()) return 0; if (qp_index < 0) qp_index = 0; qp_index %= qp_list_.size(); - if (status_ != EP_READY) return 0; + // Check endpoint status before submitting + if (status_.load(std::memory_order_relaxed) != EP_READY) return 0; auto cq = context_->cq(qp_index % context_->cqCount()); int wr_count = std::min(cq->maxCqe() - cq->getQuota(), @@ -586,7 +675,8 @@ int RdmaEndPoint::submitSlices(std::vector& slice_list, int RdmaEndPoint::submitRecvImmDataRequest(int qp_index, uint64_t id) { RWSpinlock::ReadGuard guard(lock_); - if (status_ != EP_READY) return 0; + // Check endpoint status before submitting + if (status_.load(std::memory_order_relaxed) != EP_READY) return 0; if (qp_index < 0 || qp_index >= (int)qp_list_.size()) return 0; ibv_recv_wr wr, *bad_wr; memset(&wr, 0, sizeof(ibv_recv_wr)); @@ -788,7 +878,8 @@ void RdmaEndPoint::repostAllNotifyRecvs() { static int setupNotifyQpConnection(ibv_qp* qp, RdmaContext* ctx, const std::string& peer_gid_str, - uint16_t peer_lid, uint32_t peer_qp_num) { + uint16_t peer_lid, uint32_t peer_qp_num, + uint16_t pkey_index) { // Reconnect path may call this when QP is already in RTS; force a clean // state machine: RESET -> INIT -> RTR -> RTS. ibv_qp_attr qp_attr = {}; @@ -801,7 +892,7 @@ static int setupNotifyQpConnection(ibv_qp* qp, RdmaContext* ctx, memset(&qp_attr, 0, sizeof(qp_attr)); qp_attr.qp_state = IBV_QPS_INIT; - qp_attr.pkey_index = 0; + qp_attr.pkey_index = pkey_index; qp_attr.port_num = ctx->portNum(); qp_attr.qp_access_flags = IBV_ACCESS_REMOTE_WRITE; ret = ibv_modify_qp( diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/endpoint_store.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/endpoint_store.cpp index 10dc22ba..d1b5ae73 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/endpoint_store.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/endpoint_store.cpp @@ -28,7 +28,7 @@ namespace mooncake { namespace tent { -std::shared_ptr FIFOEndpointStore::get(const std::string &key) { +std::shared_ptr FIFOEndpointStore::get(const std::string& key) { RWSpinlock::ReadGuard guard(endpoint_map_lock_); auto iter = endpoint_map_.find(key); if (iter != endpoint_map_.end()) return iter->second; @@ -36,12 +36,38 @@ std::shared_ptr FIFOEndpointStore::get(const std::string &key) { } std::shared_ptr FIFOEndpointStore::getOrInsert( - const std::string &key) { + const std::string& key) { auto endpoint = get(key); - if (endpoint) return endpoint; + // Endpoints have unidirectional lifecycle - if existing endpoint is + // in terminal state (destroying/destroyed), remove it and create new. + if (endpoint) { + auto status = endpoint->status(); + if (status == RdmaEndPoint::EP_DESTROYING || + status == RdmaEndPoint::EP_DESTROYED) { + remove(endpoint.get()); + endpoint = nullptr; + } else { + return endpoint; + } + } RWSpinlock::WriteGuard guard(endpoint_map_lock_); - if (endpoint_map_.find(key) != endpoint_map_.end()) - return endpoint_map_[key]; + // Double-check after acquiring write lock + auto iter = endpoint_map_.find(key); + if (iter != endpoint_map_.end()) { + auto ep = iter->second; + auto status = ep->status(); + if (status == RdmaEndPoint::EP_DESTROYING || + status == RdmaEndPoint::EP_DESTROYED) { + waiting_list_.insert(ep); + endpoint_map_.erase(iter); + auto fifo_iter = fifo_map_[key]; + fifo_list_.erase(fifo_iter); + fifo_map_.erase(key); + // Fall through to create new endpoint + } else { + return ep; + } + } endpoint = std::make_shared(); int ret = endpoint->construct(&context_, &context_.params().endpoint, key); if (ret) { @@ -56,18 +82,29 @@ std::shared_ptr FIFOEndpointStore::getOrInsert( return endpoint; } -int FIFOEndpointStore::remove(RdmaEndPoint *ep) { +int FIFOEndpointStore::remove(RdmaEndPoint* ep) { RWSpinlock::WriteGuard guard(endpoint_map_lock_); - auto key = ep->name(); - auto iter = endpoint_map_.find(key); - if (iter != endpoint_map_.end() && iter->second.get() == ep) { - waiting_list_.insert(iter->second); - endpoint_map_.erase(iter); - auto fifo_iter = fifo_map_[key]; - fifo_list_.erase(fifo_iter); - fifo_map_.erase(key); + // Search for and remove the endpoint by pointer comparison + for (auto iter = endpoint_map_.begin(); iter != endpoint_map_.end(); + ++iter) { + if (iter->second.get() == ep) { + waiting_list_.insert(iter->second); + iter->second->beginDestroyNoLock(); + auto fifo_iter = fifo_map_[iter->first]; + fifo_list_.erase(fifo_iter); + fifo_map_.erase(iter->first); + endpoint_map_.erase(iter); + return 0; + } } - return 0; + // If not found in endpoint_map, check if it's in waiting_list + for (const auto& waiting_ep : waiting_list_) { + if (waiting_ep.get() == ep) { + // Already in waiting list, no action needed + return 0; + } + } + return -1; // Endpoint not found } void FIFOEndpointStore::evictOne() { @@ -75,17 +112,20 @@ void FIFOEndpointStore::evictOne() { std::string victim = fifo_list_.front(); fifo_list_.pop_front(); fifo_map_.erase(victim); - waiting_list_.insert(endpoint_map_[victim]); + auto victim_endpoint = endpoint_map_[victim]; + victim_endpoint->beginDestroy(); + waiting_list_.insert(victim_endpoint); endpoint_map_.erase(victim); + LOG(INFO) << victim << " evicted from FIFOEndpointStore"; } void FIFOEndpointStore::reclaim() { RWSpinlock::WriteGuard guard(endpoint_map_lock_); std::vector> to_delete; - for (auto &endpoint : waiting_list_) { - if (!endpoint->getInflightSlices()) to_delete.push_back(endpoint); + for (auto& endpoint : waiting_list_) { + if (endpoint->finishDestroy()) to_delete.push_back(endpoint); } - for (auto &endpoint : to_delete) waiting_list_.erase(endpoint); + for (auto& endpoint : to_delete) waiting_list_.erase(endpoint); } size_t FIFOEndpointStore::size() { return endpoint_map_.size(); } @@ -93,8 +133,8 @@ size_t FIFOEndpointStore::size() { return endpoint_map_.size(); } void FIFOEndpointStore::clear() { RWSpinlock::WriteGuard guard(endpoint_map_lock_); std::vector to_delete; - for (auto &entry : endpoint_map_) to_delete.push_back(entry.first); - for (auto &key : to_delete) { + for (auto& entry : endpoint_map_) to_delete.push_back(entry.first); + for (auto& key : to_delete) { endpoint_map_.erase(key); auto fifo_iter = fifo_map_[key]; fifo_list_.erase(fifo_iter); @@ -102,7 +142,7 @@ void FIFOEndpointStore::clear() { } } -std::shared_ptr SIEVEEndpointStore::get(const std::string &key) { +std::shared_ptr SIEVEEndpointStore::get(const std::string& key) { RWSpinlock::ReadGuard guard(endpoint_map_lock_); auto iter = endpoint_map_.find(key); if (iter != endpoint_map_.end()) { @@ -113,12 +153,42 @@ std::shared_ptr SIEVEEndpointStore::get(const std::string &key) { } std::shared_ptr SIEVEEndpointStore::getOrInsert( - const std::string &key) { + const std::string& key) { auto endpoint = get(key); - if (endpoint) return endpoint; + // Endpoints have unidirectional lifecycle - if existing endpoint is + // in terminal state (destroying/destroyed), remove it and create new. + if (endpoint) { + auto status = endpoint->status(); + if (status == RdmaEndPoint::EP_DESTROYING || + status == RdmaEndPoint::EP_DESTROYED) { + remove(endpoint.get()); + endpoint = nullptr; + } else { + return endpoint; + } + } RWSpinlock::WriteGuard guard(endpoint_map_lock_); - if (endpoint_map_.find(key) != endpoint_map_.end()) { - return endpoint_map_[key].first; + // Double-check after acquiring write lock + auto iter = endpoint_map_.find(key); + if (iter != endpoint_map_.end()) { + auto ep = iter->second.first; + auto status = ep->status(); + if (status == RdmaEndPoint::EP_DESTROYING || + status == RdmaEndPoint::EP_DESTROYED) { + waiting_list_len_++; + waiting_list_.insert(ep); + auto fifo_iter = fifo_map_[key]; + if (hand_.has_value() && hand_.value() == fifo_iter) { + fifo_iter == fifo_list_.begin() ? hand_ = std::nullopt + : hand_ = std::prev(fifo_iter); + } + fifo_list_.erase(fifo_iter); + fifo_map_.erase(key); + endpoint_map_.erase(iter); + // Fall through to create new endpoint + } else { + return ep; + } } endpoint = std::make_shared(); int ret = endpoint->construct(&context_, &context_.params().endpoint, key, @@ -135,23 +205,34 @@ std::shared_ptr SIEVEEndpointStore::getOrInsert( return endpoint; } -int SIEVEEndpointStore::remove(RdmaEndPoint *ep) { +int SIEVEEndpointStore::remove(RdmaEndPoint* ep) { RWSpinlock::WriteGuard guard(endpoint_map_lock_); - auto key = ep->name(); - auto iter = endpoint_map_.find(key); - if (iter != endpoint_map_.end() && iter->second.first.get() == ep) { - waiting_list_len_++; - waiting_list_.insert(iter->second.first); - endpoint_map_.erase(iter); - auto fifo_iter = fifo_map_[key]; - if (hand_.has_value() && hand_.value() == fifo_iter) { - fifo_iter == fifo_list_.begin() ? hand_ = std::nullopt - : hand_ = std::prev(fifo_iter); + // Search for and remove the endpoint by pointer comparison + for (auto iter = endpoint_map_.begin(); iter != endpoint_map_.end(); + ++iter) { + if (iter->second.first.get() == ep) { + waiting_list_len_++; + waiting_list_.insert(iter->second.first); + iter->second.first->beginDestroyNoLock(); + auto fifo_iter = fifo_map_[iter->first]; + if (hand_.has_value() && hand_.value() == fifo_iter) { + fifo_iter == fifo_list_.begin() ? hand_ = std::nullopt + : hand_ = std::prev(fifo_iter); + } + fifo_list_.erase(fifo_iter); + fifo_map_.erase(iter->first); + endpoint_map_.erase(iter); + return 0; } - fifo_list_.erase(fifo_iter); - fifo_map_.erase(key); } - return 0; + // If not found in endpoint_map, check if it's in waiting_list + for (const auto& waiting_ep : waiting_list_) { + if (waiting_ep.get() == ep) { + // Already in waiting list, no action needed + return 0; + } + } + return -1; // Endpoint not found } void SIEVEEndpointStore::evictOne() { @@ -174,6 +255,7 @@ void SIEVEEndpointStore::evictOne() { fifo_list_.erase(o); fifo_map_.erase(victim); auto victim_instance = endpoint_map_[victim].first; + victim_instance->beginDestroy(); waiting_list_len_++; waiting_list_.insert(victim_instance); endpoint_map_.erase(victim); @@ -185,10 +267,10 @@ void SIEVEEndpointStore::reclaim() { if (waiting_list_len_.load(std::memory_order_relaxed) == 0) return; RWSpinlock::WriteGuard guard(endpoint_map_lock_); std::vector> to_delete; - for (auto &endpoint : waiting_list_) { - if (!endpoint->getInflightSlices()) to_delete.push_back(endpoint); + for (auto& endpoint : waiting_list_) { + if (endpoint->finishDestroy()) to_delete.push_back(endpoint); } - for (auto &endpoint : to_delete) waiting_list_.erase(endpoint); + for (auto& endpoint : to_delete) waiting_list_.erase(endpoint); waiting_list_len_ -= to_delete.size(); } @@ -197,8 +279,8 @@ size_t SIEVEEndpointStore::size() { return endpoint_map_.size(); } void SIEVEEndpointStore::clear() { RWSpinlock::WriteGuard guard(endpoint_map_lock_); std::vector to_delete; - for (auto &entry : endpoint_map_) to_delete.push_back(entry.first); - for (auto &key : to_delete) { + for (auto& entry : endpoint_map_) to_delete.push_back(entry.first); + for (auto& key : to_delete) { endpoint_map_.erase(key); auto fifo_iter = fifo_map_[key]; if (hand_.has_value() && hand_.value() == fifo_iter) { diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rail_monitor.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rail_monitor.cpp index be9142f9..438b211d 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rail_monitor.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rail_monitor.cpp @@ -17,9 +17,9 @@ namespace mooncake { namespace tent { -Status RailMonitor::load(const Topology *local, const Topology *remote, - const std::string &rail_topo_json, - const Config *conf) { +Status RailMonitor::load(const Topology* local, const Topology* remote, + const std::string& rail_topo_json, + const Config* conf) { local_ = local; remote_ = remote; if (conf) { @@ -43,7 +43,7 @@ Status RailMonitor::load(const Topology *local, const Topology *remote, bool RailMonitor::available(int local_nic, int remote_nic) { auto it = rail_states_.find(std::make_pair(local_nic, remote_nic)); if (it == rail_states_.end()) return false; - auto &st = it->second; + auto& st = it->second; if (!st.paused()) return true; if (std::chrono::steady_clock::now() < st.resume_time) return false; // Cooldown expired: clear all exponential-backoff memory so a fresh @@ -60,7 +60,7 @@ bool RailMonitor::available(int local_nic, int remote_nic) { void RailMonitor::markFailed(int local_nic, int remote_nic) { auto it = rail_states_.find(std::make_pair(local_nic, remote_nic)); if (it == rail_states_.end()) return; - auto &st = it->second; + auto& st = it->second; auto now = std::chrono::steady_clock::now(); if (st.error_count == 0 || now - st.last_error > error_window_) { st.error_count = 1; @@ -83,7 +83,7 @@ void RailMonitor::markFailed(int local_nic, int remote_nic) { void RailMonitor::markRecovered(int local_nic, int remote_nic) { auto it = rail_states_.find(std::make_pair(local_nic, remote_nic)); if (it == rail_states_.end()) return; - auto &st = it->second; + auto& st = it->second; // Fast path: a healthy rail stays healthy. 99%+ of completions land // here, so we must not touch best_mapping_ or write any field. if (!st.paused() && st.error_count == 0 && st.cooldown.count() == 0) return; @@ -129,7 +129,7 @@ int RailMonitor::findBestRemoteDevice(int local_nic, int remote_numa) { * ] * } */ -Status RailMonitor::loadFromJson(const std::string &rail_topo_json) { +Status RailMonitor::loadFromJson(const std::string& rail_topo_json) { try { auto root = json::parse(rail_topo_json); @@ -137,7 +137,7 @@ Status RailMonitor::loadFromJson(const std::string &rail_topo_json) { direct_rails_.clear(); if (root.contains("all")) { - for (const auto &path_entry : root["all"]) { + for (const auto& path_entry : root["all"]) { std::string local_nic_name = path_entry.value("local", ""); std::string remote_nic_name = path_entry.value("remote", ""); int local_nic_id = local_->getNicId(local_nic_name); @@ -153,7 +153,7 @@ Status RailMonitor::loadFromJson(const std::string &rail_topo_json) { } if (root.contains("direct")) { - for (const auto &path_entry : root["direct"]) { + for (const auto& path_entry : root["direct"]) { std::string local_nic_name = path_entry.value("local", ""); std::string remote_nic_name = path_entry.value("remote", ""); int local_nic_id = local_->getNicId(local_nic_name); @@ -167,7 +167,7 @@ Status RailMonitor::loadFromJson(const std::string &rail_topo_json) { } } } - } catch (const std::exception &ex) { + } catch (const std::exception& ex) { LOG(ERROR) << "Failed to parse rail_topo_json: " << ex.what(); return Status::InvalidArgument("Failed to parse JSON" LOC_MARK); } @@ -177,12 +177,12 @@ Status RailMonitor::loadFromJson(const std::string &rail_topo_json) { return Status::OK(); } -static int matchRemoteNicId(const Topology *local, const Topology *remote, +static int matchRemoteNicId(const Topology* local, const Topology* remote, int local_nic) { std::string mem_name; for (size_t i = 0; i < local->getMemCount(); ++i) { auto entry = local->getMemEntry(i); - auto &prior_devices = entry->device_list[0]; + auto& prior_devices = entry->device_list[0]; if (entry->type == Topology::MEM_CUDA && !prior_devices.empty() && prior_devices[0] == local_nic) { mem_name = entry->name; @@ -193,7 +193,7 @@ static int matchRemoteNicId(const Topology *local, const Topology *remote, auto mem_id = remote->getMemId(mem_name); if (mem_id < 0) return -1; auto entry = remote->getMemEntry(mem_id); - auto &prior_devices = entry->device_list[0]; + auto& prior_devices = entry->device_list[0]; if (entry->type == Topology::MEM_CUDA && !prior_devices.empty()) return prior_devices[0]; return -1; @@ -214,6 +214,23 @@ Status RailMonitor::loadDefault() { auto local_entry = local_->getNicEntry(local_nic); if (local_entry->type != Topology::NIC_RDMA) continue; int numa_id = local_entry->numa_node; + + // Priority 1: Same-name device matching (mlx5_0 -> mlx5_0) + bool matched = false; + auto local_nic_name = local_entry->name; + for (int remote_nic = 0; remote_nic < remote_nic_count; ++remote_nic) { + auto remote_entry = remote_->getNicEntry(remote_nic); + if (remote_entry && remote_entry->type == Topology::NIC_RDMA && + remote_entry->name == local_nic_name) { + remote_load[remote_nic]++; + direct_rails_[local_nic] = remote_nic; + matched = true; + break; + } + } + if (matched) continue; + + // Priority 2: CUDA memory topology matching (GPU-direct NIC) int remote_nic = matchRemoteNicId(local_, remote_, local_nic); if (remote_nic >= 0) { remote_load[remote_nic]++; @@ -299,7 +316,7 @@ void RailMonitor::updateBestMapping() { for (size_t local_numa = 0; local_numa < kMaxNuma; ++local_numa) { for (size_t remote_numa = 0; remote_numa < kMaxNuma; ++remote_numa) { - auto &mapping = best_mapping_[remote_numa]; + auto& mapping = best_mapping_[remote_numa]; size_t local_cnt = local_devices[local_numa].size(); size_t remote_cnt = remote_devices[remote_numa].size(); if (!local_cnt || !remote_cnt) continue; diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp index b054233b..aa273aea 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp @@ -163,8 +163,10 @@ static Status convertConfToRdmaParams(std::shared_ptr conf, return Status::OK(); } -static bool isGpuDirectRdmaSupported() { - if (getenv("MC_DISABLE_GPU_DIRECT_RDMA")) { +static bool isGpuDirectRdmaSupported(std::shared_ptr conf) { + auto disable_gpu_direct = + conf->get("transports/rdma/disable_gpu_direct_rdma", false); + if (disable_gpu_direct) { return false; } std::ifstream modules("/proc/modules"); @@ -252,7 +254,7 @@ Status RdmaTransport::install(std::string& local_segment_name, installed_ = true; caps.dram_to_dram = true; - if (isGpuDirectRdmaSupported()) { + if (isGpuDirectRdmaSupported(conf_)) { caps.dram_to_gpu = true; caps.gpu_to_dram = true; caps.gpu_to_gpu = true; @@ -558,10 +560,6 @@ std::shared_ptr RdmaTransport::getEndpoint(SegmentID target_id, std::shared_ptr endpoint; std::string peer_name = MakeNicPath(segment_desc->name, target_dev_name); endpoint = context->endpointStore()->getOrInsert(peer_name); - if (endpoint && endpoint->status() == RdmaEndPoint::EP_RESET) { - context->endpointStore()->remove(endpoint.get()); - endpoint = context->endpointStore()->getOrInsert(peer_name); - } if (!endpoint) { LOG(ERROR) << "Cannot allocate endpoint " << peer_name; return nullptr; diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp index ef4132cc..d2c51038 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp @@ -178,10 +178,6 @@ std::shared_ptr Workers::getEndpoint(Workers::PostPath path) { std::shared_ptr endpoint; auto peer_name = MakeNicPath(target_seg_name, target_dev_name); endpoint = context->endpointStore()->getOrInsert(peer_name); - if (endpoint && endpoint->status() == RdmaEndPoint::EP_RESET) { - context->endpointStore()->remove(endpoint.get()); - endpoint = context->endpointStore()->getOrInsert(peer_name); - } if (!endpoint) { LOG(ERROR) << "Cannot allocate endpoint " << peer_name; return nullptr; @@ -209,7 +205,7 @@ void Workers::disableEndpoint(RdmaSlice* slice) { } if (auto ep = slice->ep_weak_ptr.lock()) { ep->acknowledge(slice, FAILED); - ep->reset(); + ep->resetConnection("Endpoint failed"); } } @@ -462,7 +458,25 @@ int Workers::handleContextEvents(std::shared_ptr& context) { } void Workers::monitorThread() { + // Track time for periodic endpoint reclaim (1 Hz heartbeat) + auto last_reclaim_time = std::chrono::steady_clock::now(); + while (running_) { + // Periodic endpoint reclaim: runs every 1 second to drain waiting_list_ + // Under failure load, insertions stall but endpoints still need cleanup + auto current_time = std::chrono::steady_clock::now(); + auto time_since_last_reclaim = + std::chrono::duration_cast( + current_time - last_reclaim_time) + .count(); + + if (time_since_last_reclaim >= 1000) { // 1 second = 1000 ms + for (auto& context : transport_->context_set_) { + context->endpointStore()->reclaim(); + } + last_reclaim_time = current_time; + } + for (auto& context : transport_->context_set_) { struct epoll_event event; if (context->eventFd() < 0) continue; -- 2.34.1 From 7cbb68fe28490eb6784029ebf20c521f3e719de4 Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Tue, 19 May 2026 08:28:01 +0800 Subject: [PATCH 089/382] [CI] change dependency `libboost-all-dev` to `libboost-dev` (#2129) --- .devcontainer/Dockerfile | 2 +- dependencies.sh | 2 +- docs/source/design/transfer-engine/kunpeng_ub_transport.md | 2 +- docs/source/getting_started/build.md | 2 +- scripts/ascend/dependencies_ascend.sh | 2 +- scripts/ascend/dependencies_ascend_installation.sh | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index 405d2433..a5729ef9 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -12,7 +12,7 @@ RUN apt-get install -y libibverbs-dev \ libjsoncpp-dev \ libnuma-dev \ libpython3-dev \ - libboost-all-dev \ + libboost-dev \ libssl-dev \ libgrpc-dev \ libgrpc++-dev \ diff --git a/dependencies.sh b/dependencies.sh index 607e58af..8ce80e55 100755 --- a/dependencies.sh +++ b/dependencies.sh @@ -112,7 +112,7 @@ SYSTEM_PACKAGES="build-essential \ libunwind-dev \ libnuma-dev \ libpython3-dev \ - libboost-all-dev \ + libboost-dev \ libssl-dev \ libgrpc-dev \ libgrpc++-dev \ diff --git a/docs/source/design/transfer-engine/kunpeng_ub_transport.md b/docs/source/design/transfer-engine/kunpeng_ub_transport.md index ead3a6c8..a751b997 100644 --- a/docs/source/design/transfer-engine/kunpeng_ub_transport.md +++ b/docs/source/design/transfer-engine/kunpeng_ub_transport.md @@ -50,7 +50,7 @@ sudo apt-get install -y \ libjsoncpp-dev \ libnuma-dev \ libibverbs-dev \ - libboost-all-dev \ + libboost-dev \ libcurl4-openssl-dev \ libgtest-dev \ libmsgpack-dev \ diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index 790dbf13..041ffdef 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -70,7 +70,7 @@ pip install mooncake-transfer-engine-non-cuda libnuma-dev \ libunwind-dev \ libpython3-dev \ - libboost-all-dev \ + libboost-dev \ libssl-dev \ pybind11-dev \ libcurl4-openssl-dev \ diff --git a/scripts/ascend/dependencies_ascend.sh b/scripts/ascend/dependencies_ascend.sh index d58d56f8..101aaaf4 100644 --- a/scripts/ascend/dependencies_ascend.sh +++ b/scripts/ascend/dependencies_ascend.sh @@ -60,7 +60,7 @@ if command -v apt-get &> /dev/null; then libunwind-dev \ libnuma-dev \ libpython3-dev \ - libboost-all-dev \ + libboost-dev \ libssl-dev \ libzstd-dev \ libgrpc-dev \ diff --git a/scripts/ascend/dependencies_ascend_installation.sh b/scripts/ascend/dependencies_ascend_installation.sh index d327f2be..eb9266dc 100644 --- a/scripts/ascend/dependencies_ascend_installation.sh +++ b/scripts/ascend/dependencies_ascend_installation.sh @@ -70,7 +70,7 @@ if command -v apt-get &> /dev/null; then libunwind-dev \ libnuma-dev \ libpython3-dev \ - libboost-all-dev \ + libboost-dev \ libssl-dev \ libgrpc-dev \ libgrpc++-dev \ -- 2.34.1 From 150ab331565b2e9db46848bc6e357a7046b09403 Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Tue, 19 May 2026 16:37:58 +0800 Subject: [PATCH 090/382] [Docs] clarify Redis build options (#2136) --- README.md | 2 +- docs/source/getting_started/build.md | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 9a5ab695..15d7ae33 100644 --- a/README.md +++ b/README.md @@ -317,7 +317,7 @@ The following are additional dependencies for building Mooncake: - Hygon DTK SDK, if you want to build with `-DUSE_HYGON`. *This is NOT included in the `dependencies.sh` script.* Mooncake resolves it from `DTK_HOME` or `/opt/dtk` by default, and also supports overriding `DTK_INCLUDE_DIR` / `DTK_LIB_DIR` during CMake configure. - Iluvatar CoreX SDK, if you want to build with `-DUSE_COREX`. *This is NOT included in the `dependencies.sh` script.* Mooncake resolves it from `COREX_HOME` or `/usr/local/corex` by default, and also supports overriding `COREX_INCLUDE_DIR` / `COREX_LIB_DIR` during CMake configure. - [Optional] Rust Toolchain and libclang, if you want to build Transfer Engine Rust examples with `-DWITH_RUST_EXAMPLE=ON` or Mooncake Store Rust bindings with `-DWITH_STORE_RUST=ON`. *This is NOT included in the `dependencies.sh` script.* -- [Optional] `hiredis`, if you want to build with `-DUSE_REDIS` to use Redis instead of etcd as metadata servers. +- [Optional] `hiredis`, if you want to build with `-DUSE_REDIS` to use Redis instead of etcd as metadata servers, or with `-DSTORE_USE_REDIS` to use Redis for Mooncake Store failover. - [Optional] `curl`, if you want to build with `-DUSE_HTTP` to use HTTP instead of etcd as metadata servers. The build and installation steps are as follows: diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index 041ffdef..e7a0802a 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -248,10 +248,11 @@ The following options can be used during `cmake ..` to specify whether to compil - `-DWITH_RUST_EXAMPLE=[ON|OFF]`: Build the Transfer Engine Rust interface and sample code. **Default: OFF.** - `-DWITH_STORE_RUST=[ON|OFF]`: Build Mooncake Store Rust bindings and CMake Rust targets. **Default: ON.** - `-DWITH_EP=[ON|OFF]`: Build the EP (Expert Parallelism) and PG Python extensions for CUDA. Requires CUDA toolkit and PyTorch. Use `-DEP_TORCH_VERSIONS="2.9.1"` (semicolon-separated) to build for specific PyTorch versions, or leave empty to use the currently-installed torch. The CUDA version is detected automatically. **Default: OFF.** -- `-DUSE_REDIS=[ON|OFF]`: Enable Redis-based metadata service +- `-DUSE_REDIS=[ON|OFF]`: Enable Redis-based metadata service for the Transfer Engine, require hiredis - `-DUSE_HTTP=[ON|OFF]`: Enable Http-based metadata service - `-DUSE_ETCD=[ON|OFF]`: Enable etcd-based metadata service, require go 1.23+ - `-DSTORE_USE_ETCD=[ON|OFF]`: Enable etcd-based failover for Mooncake Store, require go 1.23+. **Note:** `-DUSE_ETCD` and `-DSTORE_USE_ETCD` are two independent options. Enabling `-DSTORE_USE_ETCD` does **not** depend on `-DUSE_ETCD` +- `-DSTORE_USE_REDIS=[ON|OFF]`: Enable Redis-based failover for Mooncake Store, require hiredis. **Default: OFF.** **Note:** `-DUSE_REDIS` and `-DSTORE_USE_REDIS` are two independent options. Enabling `-DSTORE_USE_REDIS` does **not** depend on `-DUSE_REDIS`. - `-DBUILD_SHARED_LIBS=[ON|OFF]`: Build Transfer Engine as shared library, default is OFF - `-DBUILD_UNIT_TESTS=[ON|OFF]`: Build unit tests, default is ON - `-DBUILD_EXAMPLES=[ON|OFF]`: Build examples, default is ON -- 2.34.1 From 4615104b8bbd69a4da916db6464b842c0e64c0ac Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Tue, 19 May 2026 17:06:09 +0800 Subject: [PATCH 091/382] [Store] Report master metrics as per-second rates over time window (#2082) Co-authored-by: Claude Opus 4.7 Co-authored-by: fatSheep --- docs/source/getting_started/observability.md | 167 +++++++ docs/source/index.md | 1 + .../include/master_metric_manager.h | 98 ++++ mooncake-store/src/master_metric_manager.cpp | 442 ++++++++++++++---- mooncake-store/src/master_service.cpp | 1 + mooncake-store/src/rpc_service.cpp | 26 +- mooncake-store/tests/master_metrics_test.cpp | 71 +++ 7 files changed, 721 insertions(+), 85 deletions(-) create mode 100644 docs/source/getting_started/observability.md diff --git a/docs/source/getting_started/observability.md b/docs/source/getting_started/observability.md new file mode 100644 index 00000000..cbc387b7 --- /dev/null +++ b/docs/source/getting_started/observability.md @@ -0,0 +1,167 @@ +# Observability + +This document describes how to monitor and observe a running Mooncake Store deployment. + +## Master Metrics Log + +When started, the Mooncake master periodically prints a metrics summary log every 10 seconds (configurable via `kMetricReportIntervalSeconds`). This log provides a comprehensive snapshot of the master's runtime state. + +### Log Format + +``` +I0512 15:03:30.321475 239489 rpc_service.cpp:269] Master Admin Metrics: role=leader, state=serving, service_ready=true, master={...}, ha={...}, leader=127.0.0.1:50051, view_version=1 +``` + +Each log line contains: +- **GLog header**: timestamp, thread ID, source file and line number +- **role**: HA role — `leader` or `standby` +- **state**: HA runtime state — `serving`, `starting`, `stopping`, etc. +- **service_ready**: whether the gRPC service is accepting requests +- **master**: master metrics block (see below) +- **ha**: HA metrics block +- **leader**: (only when available) the current leader address and view version + +### Master Metrics Block + +A typical `master={...}` block looks like this: + +``` +Mem Storage: 94.09 MB / 100.00 MB (94.1%) | SSD Storage: 0 B / 0 B | Keys: 16058 (soft-pinned: 0) | Clients: 1 | Requests (Success/Total per sec): PutStart=0.00/0.00, PutEnd=0.00/0.00, PutRevoke=0.00/0.00, Get=0.00/0.00, Exist=0.00/0.00, Del=0.00/0.00, DelAll=0.00/0.00, Ping=1.00/1.00, CopyStart=0.00/0.00, CopyEnd=0.00/0.00, CopyRevoke=0.00/0.00, MoveStart=0.00/0.00, MoveEnd=0.00/0.00, MoveRevoke=0.00/0.00, EvictDiskReplica=0.00/0.00 | Batch Requests (per sec, Req=Success/PartialSuccess/Total, Item=Success/Total): PutStart:(Req=0.00/0.00/0.00, Item=0.00/0.00), PutEnd:(Req=0.00/0.00/0.00, Item=0.00/0.00), PutRevoke:(Req=0.00/0.00/0.00, Item=0.00/0.00), Get:(Req=0.00/0.00/0.00, Item=0.00/0.00), ExistKey:(Req=0.00/0.00/0.00, Item=0.00/0.00), QueryIp:(Req=0.00/0.00/0.00, Item=0.00/0.00), Clear:(Req=0.00/0.00/0.00, Item=0.00/0.00), CreateMoveTask:(Req=0.00/0.00), CreateCopyTask:(Req=0.00/0.00), QueryTask:(Req=0.00/0.00), FetchTasks:(Req=0.00/0.00), MarkTaskToComplete:(Req=0.00/0.00) | Eviction: Success/Attempts=0/0, AllocFail=0, keys=0, size=0 B | Discard: Released/Total=0/0, StagingSize=0 B | Snapshots: Success=0, Fail=0 +``` + +Request counters are reported as **rates per second** over the time window between two consecutive log outputs (10 seconds by default). Real-time state values (storage, key count, client count, discard staging size) are not rate-limited and reflect the current value at log time. + +The metrics block consists of the following sections: + +#### Storage + +| Field | Description | +|-------|-------------| +| `Mem Storage` | Current memory usage / total memory capacity, with percentage | +| `SSD Storage` | Current SSD-backed storage usage / total SSD capacity | + +#### Keys and Clients + +| Field | Description | +|-------|-------------| +| `Keys` | Total number of keys managed by the master | +| `soft-pinned` | Number of keys with active soft-pin leases (protected from eviction) | +| `Clients` | Number of currently connected clients | + +#### Requests (Success/Total per sec) + +Rate counters for individual (non-batch) RPC requests over the last time window. Each shows `/` in requests per second: + +| Counter | Description | +|---------|-------------| +| `PutStart` | Put object allocation requests | +| `PutEnd` | Put object commit requests | +| `PutRevoke` | Put object cancellation requests | +| `Get` | Get replica list requests | +| `Exist` | Key existence check requests | +| `Del` | Single key deletion requests | +| `DelAll` | Delete-all objects requests | +| `Ping` | Client heartbeat/ping requests | +| `CopyStart` | Copy object allocation requests | +| `CopyEnd` | Copy object commit requests | +| `CopyRevoke` | Copy object cancellation requests | +| `MoveStart` | Move object allocation requests | +| `MoveEnd` | Move object commit requests | +| `MoveRevoke` | Move object cancellation requests | +| `EvictDiskReplica` | Evict disk replica requests | + +#### Batch Requests (per sec) + +Batch operations aggregate multiple items into a single RPC. Rates are per second over the last time window. Format: `Req=//`, `Item=/`: + +| Counter | Description | +|---------|-------------| +| `PutStart` | Batch put object allocation requests | +| `PutEnd` | Batch put object commit requests | +| `PutRevoke` | Batch put object cancellation requests | +| `Get` | Batch get replica list requests | +| `ExistKey` | Batch key existence check requests | +| `QueryIp` | Batch query IP requests | +| `Clear` | Batch replica clear requests | + +A request is considered "partial success" when it succeeds for some items but not all. + +#### Task Operations + +| Counter | Description | +|---------|-------------| +| `CreateMoveTask` | Move task creation requests | +| `CreateCopyTask` | Copy task creation requests | +| `QueryTask` | Task status query requests | +| `FetchTasks` | Pending task fetch requests (polled by store clients) | +| `MarkTaskToComplete` | Task completion acknowledgement requests | + +#### Eviction & Discard + +Eviction counters are **deltas** between two consecutive log outputs — they show what happened in the time window, not cumulative totals. + +| Field | Description | +|-------|-------------| +| `Eviction: Success/Attempts` | Eviction rounds that succeeded at least partially vs. total attempts in this window | +| `AllocFail` | Number of PutStart/UpsertStart failures caused by replica allocation failure (triggers eviction) in this window | +| `keys` | Number of keys evicted in this window | +| `size` | Total size of evicted data in this window | +| `Discard: Released/Total` | Released (cleaned up) vs. total discarded PutStart staging replicas (live values) | +| `StagingSize` | Current size of discarded but not-yet-released staging buffers (live value) | + +## Prometheus Metrics Endpoint + +Mooncake master exposes Prometheus-format metrics at the HTTP admin endpoint. This allows integration with Prometheus, Grafana, or any Prometheus-compatible monitoring stack. + +### Endpoints + +The admin HTTP server runs on `metrics_port` (default: **9003**) and exposes the following endpoints: + +| Endpoint | Content-Type | Description | +|----------|-------------|-------------| +| `GET /metrics` | `text/plain; version=0.0.4` | All metrics in Prometheus exposition format | +| `GET /metrics/summary` | `text/plain; version=0.0.4` | Human-readable summary (same content as the periodic log) | +| `GET /health` | `application/json` | Health check with role, HA state, and service readiness | +| `GET /role` | `text/plain` | Current HA role (`leader` / `standby`) | +| `GET /ha_status` | `text/plain` | Current HA runtime state (`serving` / `starting` / etc.) | + +### Usage + +**Scrape the /metrics endpoint with Prometheus:** + +Add a scrape config to your `prometheus.yml`: + +```yaml +scrape_configs: + - job_name: 'mooncake-master' + static_configs: + - targets: [':9003'] + metrics_path: '/metrics' +``` + +**Quick check with curl:** + +```bash +# Get Prometheus metrics +curl http://:9003/metrics + +# Get human-readable summary +curl http://:9003/metrics/summary + +# Check health +curl http://:9003/health +``` + +### Configuration + +The admin HTTP server is configured in the master config file (`master.json` or `master.yaml`): + +```json +{ + "enable_metric_reporting": true, + "metrics_port": 9003, + ... +} +``` + +Set `enable_metric_reporting` to `false` to disable the periodic metrics log. HTTP endpoints (`/metrics`, `/health`, etc.) remain available regardless of this setting. diff --git a/docs/source/index.md b/docs/source/index.md index 9298183f..d35549fe 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -68,6 +68,7 @@ This repository also hosts its technical report and the open-sourced traces. getting_started/build getting_started/quick-start getting_started/supported-protocols +getting_started/observability getting_started/plugin-usage/3FS-USRBIO-Plugin getting_started/examples/lmcache-integration getting_started/examples/lmdeploy-integration-v0.9 diff --git a/mooncake-store/include/master_metric_manager.h b/mooncake-store/include/master_metric_manager.h index 78c1cb27..1eebc1d4 100644 --- a/mooncake-store/include/master_metric_manager.h +++ b/mooncake-store/include/master_metric_manager.h @@ -1,7 +1,9 @@ #pragma once +#include #include #include +#include #include "ylt/metric/counter.hpp" #include "ylt/metric/gauge.hpp" @@ -96,6 +98,7 @@ class MasterMetricManager { // Operation Statistics (Counters) void inc_put_start_requests(int64_t val = 1); void inc_put_start_failures(int64_t val = 1); + void inc_put_start_alloc_failures(int64_t val = 1); void inc_put_end_requests(int64_t val = 1); void inc_put_end_failures(int64_t val = 1); void inc_put_revoke_requests(int64_t val = 1); @@ -147,6 +150,7 @@ class MasterMetricManager { // Operation Statistics Getters int64_t get_put_start_requests(); int64_t get_put_start_failures(); + int64_t get_put_start_alloc_failures(); int64_t get_put_end_requests(); int64_t get_put_end_failures(); int64_t get_put_revoke_requests(); @@ -297,6 +301,7 @@ class MasterMetricManager { * @return A string containing the formatted summary. */ std::string get_summary_string(); + std::string get_summary_string_and_update_snapshot(); private: // --- Private Constructor & Destructor --- @@ -305,8 +310,100 @@ class MasterMetricManager { // Update all metrics once to ensure zero values are serialized void update_metrics_for_zero_output(); + std::string get_summary_string(bool update_summary_snapshot); + + struct SummaryCounters { + int64_t exist_keys = 0; + int64_t exist_key_fails = 0; + int64_t put_starts = 0; + int64_t put_start_fails = 0; + int64_t put_start_alloc_fails = 0; + int64_t put_ends = 0; + int64_t put_end_fails = 0; + int64_t put_revoke_requests = 0; + int64_t put_revoke_fails = 0; + int64_t get_replicas = 0; + int64_t get_replica_fails = 0; + int64_t removes = 0; + int64_t remove_fails = 0; + int64_t remove_all = 0; + int64_t remove_all_fails = 0; + int64_t create_move_tasks = 0; + int64_t create_move_task_fails = 0; + int64_t create_copy_tasks = 0; + int64_t create_copy_task_fails = 0; + int64_t query_tasks = 0; + int64_t query_task_fails = 0; + int64_t fetch_tasks = 0; + int64_t fetch_task_fails = 0; + int64_t copy_starts = 0; + int64_t copy_start_fails = 0; + int64_t copy_ends = 0; + int64_t copy_end_fails = 0; + int64_t copy_revokes = 0; + int64_t copy_revoke_fails = 0; + int64_t move_starts = 0; + int64_t move_start_fails = 0; + int64_t move_ends = 0; + int64_t move_end_fails = 0; + int64_t move_revokes = 0; + int64_t move_revoke_fails = 0; + int64_t evict_disk_replicas = 0; + int64_t evict_disk_replica_fails = 0; + int64_t batch_put_start_requests = 0; + int64_t batch_put_start_fails = 0; + int64_t batch_put_start_partial_successes = 0; + int64_t batch_put_start_items = 0; + int64_t batch_put_start_failed_items = 0; + int64_t batch_put_end_requests = 0; + int64_t batch_put_end_fails = 0; + int64_t batch_put_end_partial_successes = 0; + int64_t batch_put_end_items = 0; + int64_t batch_put_end_failed_items = 0; + int64_t batch_put_revoke_requests = 0; + int64_t batch_put_revoke_fails = 0; + int64_t batch_put_revoke_partial_successes = 0; + int64_t batch_put_revoke_items = 0; + int64_t batch_put_revoke_failed_items = 0; + int64_t batch_get_replica_list_requests = 0; + int64_t batch_get_replica_list_fails = 0; + int64_t batch_get_replica_list_partial_successes = 0; + int64_t batch_get_replica_list_items = 0; + int64_t batch_get_replica_list_failed_items = 0; + int64_t batch_exist_key_requests = 0; + int64_t batch_exist_key_fails = 0; + int64_t batch_exist_key_partial_successes = 0; + int64_t batch_exist_key_items = 0; + int64_t batch_exist_key_failed_items = 0; + int64_t batch_query_ip_requests = 0; + int64_t batch_query_ip_fails = 0; + int64_t batch_query_ip_partial_successes = 0; + int64_t batch_query_ip_items = 0; + int64_t batch_query_ip_failed_items = 0; + int64_t batch_replica_clear_requests = 0; + int64_t batch_replica_clear_fails = 0; + int64_t batch_replica_clear_partial_successes = 0; + int64_t batch_replica_clear_items = 0; + int64_t batch_replica_clear_failed_items = 0; + int64_t eviction_success = 0; + int64_t eviction_attempts = 0; + int64_t evicted_key_count = 0; + int64_t evicted_size = 0; + int64_t ping = 0; + int64_t ping_fails = 0; + int64_t mark_task_to_complete_requests = 0; + int64_t mark_task_to_complete_fails = 0; + }; + + struct SummarySnapshot { + bool initialized = false; + std::chrono::steady_clock::time_point timestamp; + SummaryCounters counters; + }; // --- Metric Members --- + std::mutex summary_snapshot_mutex_; + SummarySnapshot summary_snapshot_; // Memory Storage Metrics ylt::metric::gauge_t @@ -335,6 +432,7 @@ class MasterMetricManager { // Operation Statistics ylt::metric::counter_t put_start_requests_; ylt::metric::counter_t put_start_failures_; + ylt::metric::counter_t put_start_alloc_failures_; ylt::metric::counter_t put_end_requests_; ylt::metric::counter_t put_end_failures_; ylt::metric::counter_t put_revoke_requests_; diff --git a/mooncake-store/src/master_metric_manager.cpp b/mooncake-store/src/master_metric_manager.cpp index 1f01da47..f8e1db24 100644 --- a/mooncake-store/src/master_metric_manager.cpp +++ b/mooncake-store/src/master_metric_manager.cpp @@ -54,6 +54,10 @@ MasterMetricManager::MasterMetricManager() "Total number of PutStart requests received"), put_start_failures_("master_put_start_failures_total", "Total number of failed PutStart requests"), + put_start_alloc_failures_( + "master_put_start_alloc_failures_total", + "Total number of PutStart failures caused by replica allocation " + "failure"), put_end_requests_("master_put_end_requests_total", "Total number of PutEnd requests received"), put_end_failures_("master_put_end_failures_total", @@ -342,6 +346,7 @@ void MasterMetricManager::update_metrics_for_zero_output() { // Update Counters (use inc(0) to mark as changed) put_start_requests_.inc(0); put_start_failures_.inc(0); + put_start_alloc_failures_.inc(0); put_end_requests_.inc(0); put_end_failures_.inc(0); put_revoke_requests_.inc(0); @@ -641,6 +646,9 @@ void MasterMetricManager::inc_put_start_requests(int64_t val) { void MasterMetricManager::inc_put_start_failures(int64_t val) { put_start_failures_.inc(val); } +void MasterMetricManager::inc_put_start_alloc_failures(int64_t val) { + put_start_alloc_failures_.inc(val); +} void MasterMetricManager::inc_put_end_requests(int64_t val) { put_end_requests_.inc(val); } @@ -832,6 +840,10 @@ int64_t MasterMetricManager::get_put_start_failures() { return put_start_failures_.value(); } +int64_t MasterMetricManager::get_put_start_alloc_failures() { + return put_start_alloc_failures_.value(); +} + int64_t MasterMetricManager::get_put_end_requests() { return put_end_requests_.value(); } @@ -1294,6 +1306,7 @@ std::string MasterMetricManager::serialize_metrics() { serialize_metric(exist_key_failures_); serialize_metric(put_start_requests_); serialize_metric(put_start_failures_); + serialize_metric(put_start_alloc_failures_); serialize_metric(put_end_requests_); serialize_metric(put_end_failures_); serialize_metric(put_revoke_requests_); @@ -1447,6 +1460,15 @@ void MasterMetricManager::add_stat_to_dict( // --- Human-Readable Summary --- std::string MasterMetricManager::get_summary_string() { + return get_summary_string(false); +} + +std::string MasterMetricManager::get_summary_string_and_update_snapshot() { + return get_summary_string(true); +} + +std::string MasterMetricManager::get_summary_string( + bool update_summary_snapshot) { std::stringstream ss; // --- Get current values --- @@ -1463,6 +1485,7 @@ std::string MasterMetricManager::get_summary_string() { int64_t exist_key_fails = exist_key_failures_.value(); int64_t put_starts = put_start_requests_.value(); int64_t put_start_fails = put_start_failures_.value(); + int64_t put_start_alloc_fails = put_start_alloc_failures_.value(); int64_t put_ends = put_end_requests_.value(); int64_t put_end_fails = put_end_failures_.value(); int64_t put_revoke_requests = put_revoke_requests_.value(); @@ -1567,6 +1590,154 @@ std::string MasterMetricManager::get_summary_string() { int64_t put_start_discarded_staging_size = put_start_discarded_staging_size_.value(); + SummaryCounters current_counters; + current_counters.exist_keys = exist_keys; + current_counters.exist_key_fails = exist_key_fails; + current_counters.put_starts = put_starts; + current_counters.put_start_fails = put_start_fails; + current_counters.put_start_alloc_fails = put_start_alloc_fails; + current_counters.put_ends = put_ends; + current_counters.put_end_fails = put_end_fails; + current_counters.put_revoke_requests = put_revoke_requests; + current_counters.put_revoke_fails = put_revoke_fails; + current_counters.get_replicas = get_replicas; + current_counters.get_replica_fails = get_replica_fails; + current_counters.removes = removes; + current_counters.remove_fails = remove_fails; + current_counters.remove_all = remove_all; + current_counters.remove_all_fails = remove_all_fails; + current_counters.create_move_tasks = create_move_tasks; + current_counters.create_move_task_fails = create_move_task_fails; + current_counters.create_copy_tasks = create_copy_tasks; + current_counters.create_copy_task_fails = create_copy_task_fails; + current_counters.query_tasks = query_tasks; + current_counters.query_task_fails = query_task_fails; + current_counters.fetch_tasks = fetch_tasks; + current_counters.fetch_task_fails = fetch_task_fails; + current_counters.copy_starts = copy_starts; + current_counters.copy_start_fails = copy_start_fails; + current_counters.copy_ends = copy_ends; + current_counters.copy_end_fails = copy_end_fails; + current_counters.copy_revokes = copy_revokes; + current_counters.copy_revoke_fails = copy_revoke_fails; + current_counters.move_starts = move_starts; + current_counters.move_start_fails = move_start_fails; + current_counters.move_ends = move_ends; + current_counters.move_end_fails = move_end_fails; + current_counters.move_revokes = move_revokes; + current_counters.move_revoke_fails = move_revoke_fails; + current_counters.evict_disk_replicas = evict_disk_replicas; + current_counters.evict_disk_replica_fails = evict_disk_replica_fails; + current_counters.batch_put_start_requests = batch_put_start_requests; + current_counters.batch_put_start_fails = batch_put_start_fails; + current_counters.batch_put_start_partial_successes = + batch_put_start_partial_successes; + current_counters.batch_put_start_items = batch_put_start_items; + current_counters.batch_put_start_failed_items = + batch_put_start_failed_items; + current_counters.batch_put_end_requests = batch_put_end_requests; + current_counters.batch_put_end_fails = batch_put_end_fails; + current_counters.batch_put_end_partial_successes = + batch_put_end_partial_successes; + current_counters.batch_put_end_items = batch_put_end_items; + current_counters.batch_put_end_failed_items = batch_put_end_failed_items; + current_counters.batch_put_revoke_requests = batch_put_revoke_requests; + current_counters.batch_put_revoke_fails = batch_put_revoke_fails; + current_counters.batch_put_revoke_partial_successes = + batch_put_revoke_partial_successes; + current_counters.batch_put_revoke_items = batch_put_revoke_items; + current_counters.batch_put_revoke_failed_items = + batch_put_revoke_failed_items; + current_counters.batch_get_replica_list_requests = + batch_get_replica_list_requests; + current_counters.batch_get_replica_list_fails = + batch_get_replica_list_fails; + current_counters.batch_get_replica_list_partial_successes = + batch_get_replica_list_partial_successes; + current_counters.batch_get_replica_list_items = + batch_get_replica_list_items; + current_counters.batch_get_replica_list_failed_items = + batch_get_replica_list_failed_items; + current_counters.batch_exist_key_requests = batch_exist_key_requests; + current_counters.batch_exist_key_fails = batch_exist_key_fails; + current_counters.batch_exist_key_partial_successes = + batch_exist_key_partial_successes; + current_counters.batch_exist_key_items = batch_exist_key_items; + current_counters.batch_exist_key_failed_items = + batch_exist_key_failed_items; + current_counters.batch_query_ip_requests = batch_query_ip_requests; + current_counters.batch_query_ip_fails = batch_query_ip_fails; + current_counters.batch_query_ip_partial_successes = + batch_query_ip_partial_successes; + current_counters.batch_query_ip_items = batch_query_ip_items; + current_counters.batch_query_ip_failed_items = batch_query_ip_failed_items; + current_counters.batch_replica_clear_requests = + batch_replica_clear_requests; + current_counters.batch_replica_clear_fails = batch_replica_clear_fails; + current_counters.batch_replica_clear_partial_successes = + batch_replica_clear_partial_successes; + current_counters.batch_replica_clear_items = batch_replica_clear_items; + current_counters.batch_replica_clear_failed_items = + batch_replica_clear_failed_items; + current_counters.eviction_success = eviction_success; + current_counters.eviction_attempts = eviction_attempts; + current_counters.evicted_key_count = evicted_key_count; + current_counters.evicted_size = evicted_size; + current_counters.ping = ping; + current_counters.ping_fails = ping_fails; + current_counters.mark_task_to_complete_requests = + mark_task_to_complete_requests_.value(); + current_counters.mark_task_to_complete_fails = + mark_task_to_complete_failures_.value(); + + SummaryCounters previous_counters = current_counters; + bool has_previous_summary = false; + double elapsed_seconds = 0.0; + const auto now = std::chrono::steady_clock::now(); + { + std::lock_guard lock(summary_snapshot_mutex_); + has_previous_summary = summary_snapshot_.initialized; + if (has_previous_summary) { + previous_counters = summary_snapshot_.counters; + elapsed_seconds = + std::chrono::duration(now - summary_snapshot_.timestamp) + .count(); + } + if (update_summary_snapshot) { + summary_snapshot_.initialized = true; + summary_snapshot_.timestamp = now; + summary_snapshot_.counters = current_counters; + } + } + + auto delta = [&](int64_t SummaryCounters::* field) { + if (!has_previous_summary) { + return int64_t{0}; + } + int64_t value = current_counters.*field - previous_counters.*field; + return value > 0 ? value : int64_t{0}; + }; + auto rate = [&](int64_t value) { + if (elapsed_seconds <= 0.0) { + return 0.0; + } + return static_cast(value) / elapsed_seconds; + }; + auto format_rate_value = [&](int64_t value) { + std::ostringstream rate_stream; + rate_stream << std::fixed << std::setprecision(2) << rate(value); + return rate_stream.str(); + }; + auto format_rate_pair = [&](int64_t success, int64_t total) { + return format_rate_value(success) + "/" + format_rate_value(total); + }; + auto format_rate_triple = [&](int64_t success, int64_t partial_success, + int64_t total) { + return format_rate_value(success) + "/" + + format_rate_value(partial_success) + "/" + + format_rate_value(total); + }; + // --- Format the summary string --- ss << "Mem Storage: " << byte_size_to_string(mem_allocated) << " / " << byte_size_to_string(mem_capacity); @@ -1579,104 +1750,215 @@ std::string MasterMetricManager::get_summary_string() { ss << " | Keys: " << keys << " (soft-pinned: " << soft_pin_keys << ")"; ss << " | Clients: " << active_clients; - // Request summary - focus on the most important metrics - ss << " | Requests (Success/Total): "; - ss << "PutStart=" << put_starts - put_start_fails << "/" << put_starts + // Request summary - rate per second of the last window + ss << " | Requests (Success/Total per sec): "; + ss << "PutStart=" + << format_rate_pair(delta(&SummaryCounters::put_starts) - + delta(&SummaryCounters::put_start_fails), + delta(&SummaryCounters::put_starts)) << ", "; - ss << "PutEnd=" << put_ends - put_end_fails << "/" << put_ends << ", "; - ss << "PutRevoke=" << put_revoke_requests - put_revoke_fails << "/" - << put_revoke_requests << ", "; - ss << "Get=" << get_replicas - get_replica_fails << "/" << get_replicas + ss << "PutEnd=" + << format_rate_pair(delta(&SummaryCounters::put_ends) - + delta(&SummaryCounters::put_end_fails), + delta(&SummaryCounters::put_ends)) << ", "; - ss << "Exist=" << exist_keys - exist_key_fails << "/" << exist_keys << ", "; - ss << "Del=" << removes - remove_fails << "/" << removes << ", "; - ss << "DelAll=" << remove_all - remove_all_fails << "/" << remove_all + ss << "PutRevoke=" + << format_rate_pair(delta(&SummaryCounters::put_revoke_requests) - + delta(&SummaryCounters::put_revoke_fails), + delta(&SummaryCounters::put_revoke_requests)) << ", "; - ss << "Ping=" << ping - ping_fails << "/" << ping << ", "; - ss << "CopyStart=" << copy_starts - copy_start_fails << "/" << copy_starts + ss << "Get=" + << format_rate_pair(delta(&SummaryCounters::get_replicas) - + delta(&SummaryCounters::get_replica_fails), + delta(&SummaryCounters::get_replicas)) << ", "; - ss << "CopyEnd=" << copy_ends - copy_end_fails << "/" << copy_ends << ", "; - ss << "CopyRevoke=" << copy_revokes - copy_revoke_fails << "/" - << copy_revokes << ", "; - ss << "MoveStart=" << move_starts - move_start_fails << "/" << move_starts + ss << "Exist=" + << format_rate_pair(delta(&SummaryCounters::exist_keys) - + delta(&SummaryCounters::exist_key_fails), + delta(&SummaryCounters::exist_keys)) << ", "; - ss << "MoveEnd=" << move_ends - move_end_fails << "/" << move_ends << ", "; - ss << "MoveRevoke=" << move_revokes - move_revoke_fails << "/" - << move_revokes << ", "; - ss << "EvictDiskReplica=" << evict_disk_replicas - evict_disk_replica_fails - << "/" << evict_disk_replicas; + ss << "Del=" + << format_rate_pair(delta(&SummaryCounters::removes) - + delta(&SummaryCounters::remove_fails), + delta(&SummaryCounters::removes)) + << ", "; + ss << "DelAll=" + << format_rate_pair(delta(&SummaryCounters::remove_all) - + delta(&SummaryCounters::remove_all_fails), + delta(&SummaryCounters::remove_all)) + << ", "; + ss << "Ping=" + << format_rate_pair(delta(&SummaryCounters::ping) - + delta(&SummaryCounters::ping_fails), + delta(&SummaryCounters::ping)) + << ", "; + ss << "CopyStart=" + << format_rate_pair(delta(&SummaryCounters::copy_starts) - + delta(&SummaryCounters::copy_start_fails), + delta(&SummaryCounters::copy_starts)) + << ", "; + ss << "CopyEnd=" + << format_rate_pair(delta(&SummaryCounters::copy_ends) - + delta(&SummaryCounters::copy_end_fails), + delta(&SummaryCounters::copy_ends)) + << ", "; + ss << "CopyRevoke=" + << format_rate_pair(delta(&SummaryCounters::copy_revokes) - + delta(&SummaryCounters::copy_revoke_fails), + delta(&SummaryCounters::copy_revokes)) + << ", "; + ss << "MoveStart=" + << format_rate_pair(delta(&SummaryCounters::move_starts) - + delta(&SummaryCounters::move_start_fails), + delta(&SummaryCounters::move_starts)) + << ", "; + ss << "MoveEnd=" + << format_rate_pair(delta(&SummaryCounters::move_ends) - + delta(&SummaryCounters::move_end_fails), + delta(&SummaryCounters::move_ends)) + << ", "; + ss << "MoveRevoke=" + << format_rate_pair(delta(&SummaryCounters::move_revokes) - + delta(&SummaryCounters::move_revoke_fails), + delta(&SummaryCounters::move_revokes)) + << ", "; + ss << "EvictDiskReplica=" + << format_rate_pair( + delta(&SummaryCounters::evict_disk_replicas) - + delta(&SummaryCounters::evict_disk_replica_fails), + delta(&SummaryCounters::evict_disk_replicas)); // Batch request summary ss << " | Batch Requests " - "(Req=Success/PartialSuccess/Total, Item=Success/Total): "; + "(per sec, Req=Success/PartialSuccess/Total, " + "Item=Success/Total): "; ss << "PutStart:(Req=" - << batch_put_start_requests - batch_put_start_fails - - batch_put_start_partial_successes - << "/" << batch_put_start_partial_successes << "/" - << batch_put_start_requests - << ", Item=" << batch_put_start_items - batch_put_start_failed_items - << "/" << batch_put_start_items << "), "; + << format_rate_triple( + delta(&SummaryCounters::batch_put_start_requests) - + delta(&SummaryCounters::batch_put_start_fails) - + delta(&SummaryCounters::batch_put_start_partial_successes), + delta(&SummaryCounters::batch_put_start_partial_successes), + delta(&SummaryCounters::batch_put_start_requests)) + << ", Item=" + << format_rate_pair( + delta(&SummaryCounters::batch_put_start_items) - + delta(&SummaryCounters::batch_put_start_failed_items), + delta(&SummaryCounters::batch_put_start_items)) + << "), "; ss << "PutEnd:(Req=" - << batch_put_end_requests - batch_put_end_fails - - batch_put_end_partial_successes - << "/" << batch_put_end_partial_successes << "/" - << batch_put_end_requests - << ", Item=" << batch_put_end_items - batch_put_end_failed_items << "/" - << batch_put_end_items << "), "; + << format_rate_triple( + delta(&SummaryCounters::batch_put_end_requests) - + delta(&SummaryCounters::batch_put_end_fails) - + delta(&SummaryCounters::batch_put_end_partial_successes), + delta(&SummaryCounters::batch_put_end_partial_successes), + delta(&SummaryCounters::batch_put_end_requests)) + << ", Item=" + << format_rate_pair( + delta(&SummaryCounters::batch_put_end_items) - + delta(&SummaryCounters::batch_put_end_failed_items), + delta(&SummaryCounters::batch_put_end_items)) + << "), "; ss << "PutRevoke:(Req=" - << batch_put_revoke_requests - batch_put_revoke_fails - - batch_put_revoke_partial_successes - << "/" << batch_put_revoke_partial_successes << "/" - << batch_put_revoke_requests - << ", Item=" << batch_put_revoke_items - batch_put_revoke_failed_items - << "/" << batch_put_revoke_items << "), "; + << format_rate_triple( + delta(&SummaryCounters::batch_put_revoke_requests) - + delta(&SummaryCounters::batch_put_revoke_fails) - + delta(&SummaryCounters::batch_put_revoke_partial_successes), + delta(&SummaryCounters::batch_put_revoke_partial_successes), + delta(&SummaryCounters::batch_put_revoke_requests)) + << ", Item=" + << format_rate_pair( + delta(&SummaryCounters::batch_put_revoke_items) - + delta(&SummaryCounters::batch_put_revoke_failed_items), + delta(&SummaryCounters::batch_put_revoke_items)) + << "), "; ss << "Get:(Req=" - << batch_get_replica_list_requests - batch_get_replica_list_fails - - batch_get_replica_list_partial_successes - << "/" << batch_get_replica_list_partial_successes << "/" - << batch_get_replica_list_requests << ", Item=" - << batch_get_replica_list_items - batch_get_replica_list_failed_items - << "/" << batch_get_replica_list_items << "), "; + << format_rate_triple( + delta(&SummaryCounters::batch_get_replica_list_requests) - + delta(&SummaryCounters::batch_get_replica_list_fails) - + delta(&SummaryCounters:: + batch_get_replica_list_partial_successes), + delta(&SummaryCounters::batch_get_replica_list_partial_successes), + delta(&SummaryCounters::batch_get_replica_list_requests)) + << ", Item=" + << format_rate_pair( + delta(&SummaryCounters::batch_get_replica_list_items) - + delta(&SummaryCounters::batch_get_replica_list_failed_items), + delta(&SummaryCounters::batch_get_replica_list_items)) + << "), "; ss << "ExistKey:(Req=" - << batch_exist_key_requests - batch_exist_key_fails - - batch_exist_key_partial_successes - << "/" << batch_exist_key_partial_successes << "/" - << batch_exist_key_requests - << ", Item=" << batch_exist_key_items - batch_exist_key_failed_items - << "/" << batch_exist_key_items << "), "; + << format_rate_triple( + delta(&SummaryCounters::batch_exist_key_requests) - + delta(&SummaryCounters::batch_exist_key_fails) - + delta(&SummaryCounters::batch_exist_key_partial_successes), + delta(&SummaryCounters::batch_exist_key_partial_successes), + delta(&SummaryCounters::batch_exist_key_requests)) + << ", Item=" + << format_rate_pair( + delta(&SummaryCounters::batch_exist_key_items) - + delta(&SummaryCounters::batch_exist_key_failed_items), + delta(&SummaryCounters::batch_exist_key_items)) + << "), "; ss << "QueryIp:(Req=" - << batch_query_ip_requests - batch_query_ip_fails - - batch_query_ip_partial_successes - << "/" << batch_query_ip_partial_successes << "/" - << batch_query_ip_requests - << ", Item=" << batch_query_ip_items - batch_query_ip_failed_items << "/" - << batch_query_ip_items << "), "; + << format_rate_triple( + delta(&SummaryCounters::batch_query_ip_requests) - + delta(&SummaryCounters::batch_query_ip_fails) - + delta(&SummaryCounters::batch_query_ip_partial_successes), + delta(&SummaryCounters::batch_query_ip_partial_successes), + delta(&SummaryCounters::batch_query_ip_requests)) + << ", Item=" + << format_rate_pair( + delta(&SummaryCounters::batch_query_ip_items) - + delta(&SummaryCounters::batch_query_ip_failed_items), + delta(&SummaryCounters::batch_query_ip_items)) + << "), "; ss << "Clear:(Req=" - << batch_replica_clear_requests - batch_replica_clear_fails - - batch_replica_clear_partial_successes - << "/" << batch_replica_clear_partial_successes << "/" - << batch_replica_clear_requests << ", Item=" - << batch_replica_clear_items - batch_replica_clear_failed_items << "/" - << batch_replica_clear_items << "), "; + << format_rate_triple( + delta(&SummaryCounters::batch_replica_clear_requests) - + delta(&SummaryCounters::batch_replica_clear_fails) - + delta( + &SummaryCounters::batch_replica_clear_partial_successes), + delta(&SummaryCounters::batch_replica_clear_partial_successes), + delta(&SummaryCounters::batch_replica_clear_requests)) + << ", Item=" + << format_rate_pair( + delta(&SummaryCounters::batch_replica_clear_items) - + delta(&SummaryCounters::batch_replica_clear_failed_items), + delta(&SummaryCounters::batch_replica_clear_items)) + << "), "; - ss << "CreateMoveTask:(Req=" << create_move_tasks - create_move_task_fails - << "/" << create_move_tasks << "), "; - ss << "CreateCopyTask:(Req=" << create_copy_tasks - create_copy_task_fails - << "/" << create_copy_tasks << "), "; - ss << "QueryTask=(Req=" << query_tasks - query_task_fails << "/" - << query_tasks << "), "; - ss << "FetchTasks=(Req=" << fetch_tasks - fetch_task_fails << "/" - << fetch_tasks << "), "; - ss << "MarkTaskToComplete= (Req=" - << mark_task_to_complete_requests_.value() - - mark_task_to_complete_failures_.value() - << "/" << mark_task_to_complete_requests_.value() << "), "; + ss << "CreateMoveTask:(Req=" + << format_rate_pair(delta(&SummaryCounters::create_move_tasks) - + delta(&SummaryCounters::create_move_task_fails), + delta(&SummaryCounters::create_move_tasks)) + << "), "; + ss << "CreateCopyTask:(Req=" + << format_rate_pair(delta(&SummaryCounters::create_copy_tasks) - + delta(&SummaryCounters::create_copy_task_fails), + delta(&SummaryCounters::create_copy_tasks)) + << "), "; + ss << "QueryTask:(Req=" + << format_rate_pair(delta(&SummaryCounters::query_tasks) - + delta(&SummaryCounters::query_task_fails), + delta(&SummaryCounters::query_tasks)) + << "), "; + ss << "FetchTasks:(Req=" + << format_rate_pair(delta(&SummaryCounters::fetch_tasks) - + delta(&SummaryCounters::fetch_task_fails), + delta(&SummaryCounters::fetch_tasks)) + << "), "; + ss << "MarkTaskToComplete:(Req=" + << format_rate_pair( + delta(&SummaryCounters::mark_task_to_complete_requests) - + delta(&SummaryCounters::mark_task_to_complete_fails), + delta(&SummaryCounters::mark_task_to_complete_requests)) + << ")"; // Eviction summary ss << " | Eviction: " - << "Success/Attempts=" << eviction_success << "/" << eviction_attempts - << ", " - << "keys=" << evicted_key_count << ", " - << "size=" << byte_size_to_string(evicted_size); + << "Success/Attempts=" << delta(&SummaryCounters::eviction_success) + << "/" << delta(&SummaryCounters::eviction_attempts) << ", " + << "AllocFail=" << delta(&SummaryCounters::put_start_alloc_fails) << ", " + << "keys=" << delta(&SummaryCounters::evicted_key_count) << ", " + << "size=" << byte_size_to_string(delta(&SummaryCounters::evicted_size)); // Discard summary ss << " | Discard: " diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 64ed2577..5752f4cc 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -873,6 +873,7 @@ auto MasterService::AllocateAndInsertMetadata( if (allocation_result.error() == ErrorCode::INVALID_PARAMS) { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } + MasterMetricManager::instance().inc_put_start_alloc_failures(); need_eviction_ = true; return tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); } diff --git a/mooncake-store/src/rpc_service.cpp b/mooncake-store/src/rpc_service.cpp index 63ac3480..602fac6f 100644 --- a/mooncake-store/src/rpc_service.cpp +++ b/mooncake-store/src/rpc_service.cpp @@ -242,11 +242,27 @@ bool MasterAdminServer::Start() { metric_report_thread_ = std::thread([this]() { while (metric_report_running_.load()) { const auto snapshot = SnapshotState(); - LOG(INFO) << "Master Admin Metrics: role=" - << ha::MasterRuntimeRoleToString(snapshot.state) - << ", state=" - << ha::MasterRuntimeStateToString(snapshot.state) - << ", summary=" << BuildMetricsSummaryText(); + std::ostringstream log_stream; + log_stream << "Master Admin Metrics: role=" + << ha::MasterRuntimeRoleToString(snapshot.state) + << ", state=" + << ha::MasterRuntimeStateToString(snapshot.state) + << ", service_ready=" + << (snapshot.service_available ? "true" : "false") + << ", master={" + << MasterMetricManager::instance() + .get_summary_string_and_update_snapshot() + << "}" + << ", ha={" + << HAMetricManager::instance().get_summary_string() + << "}"; + if (snapshot.leader_view.has_value()) { + log_stream + << ", leader=" << snapshot.leader_view->leader_address + << ", view_version=" + << snapshot.leader_view->view_version; + } + LOG(INFO) << log_stream.str(); std::this_thread::sleep_for( std::chrono::seconds(kMetricReportIntervalSeconds)); } diff --git a/mooncake-store/tests/master_metrics_test.cpp b/mooncake-store/tests/master_metrics_test.cpp index 65e02cfa..900a8b57 100644 --- a/mooncake-store/tests/master_metrics_test.cpp +++ b/mooncake-store/tests/master_metrics_test.cpp @@ -1,6 +1,7 @@ #include #include +#include #include #include #include @@ -59,6 +60,7 @@ TEST_F(MasterMetricsTest, InitialStatusTest) { // Operation Statistics ASSERT_EQ(metrics.get_put_start_requests(), 0); ASSERT_EQ(metrics.get_put_start_failures(), 0); + ASSERT_EQ(metrics.get_put_start_alloc_failures(), 0); ASSERT_EQ(metrics.get_put_end_requests(), 0); ASSERT_EQ(metrics.get_put_end_failures(), 0); ASSERT_EQ(metrics.get_put_revoke_requests(), 0); @@ -611,6 +613,75 @@ TEST_F(MasterMetricsTest, LocalDiskSegmentCapacityHeartbeat) { EXPECT_EQ(metrics.get_total_file_capacity(), baseline + kCap2); } +TEST_F(MasterMetricsTest, PutStartReplicaAllocationFailureMetric) { + const uint64_t default_kv_lease_ttl = 100; + auto& metrics = MasterMetricManager::instance(); + WrappedMasterServiceConfig service_config; + service_config.default_kv_lease_ttl = default_kv_lease_ttl; + WrappedMasterService service_(service_config); + + const int64_t allocation_failures_before = + metrics.get_put_start_alloc_failures(); + const int64_t put_start_failures_before = metrics.get_put_start_failures(); + + ReplicateConfig config; + config.replica_num = 1; + auto put_start_result = service_.PutStart( + generate_uuid(), "allocation_failure_key", 1024, config); + + ASSERT_FALSE(put_start_result.has_value()); + ASSERT_EQ(put_start_result.error(), ErrorCode::NO_AVAILABLE_HANDLE); + ASSERT_EQ(metrics.get_put_start_alloc_failures(), + allocation_failures_before + 1); + ASSERT_EQ(metrics.get_put_start_failures(), put_start_failures_before + 1); +} + +TEST_F(MasterMetricsTest, SummaryUsesWindowRatesAndEvictionDeltas) { + auto& metrics = MasterMetricManager::instance(); + + const std::string baseline_summary = + metrics.get_summary_string_and_update_snapshot(); + EXPECT_NE(baseline_summary.find("Requests (Success/Total per sec):"), + std::string::npos); + EXPECT_NE(baseline_summary.find("PutStart=0.00/0.00"), std::string::npos); + + metrics.inc_put_start_requests(4); + metrics.inc_put_start_failures(1); + metrics.inc_batch_put_start_requests(5); + metrics.inc_batch_put_start_partial_success(2); + metrics.inc_eviction_success(3, 4096); + metrics.inc_eviction_fail(); + + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + const std::string window_summary = metrics.get_summary_string(); + EXPECT_NE(window_summary.find("Requests (Success/Total per sec):"), + std::string::npos); + EXPECT_NE(window_summary.find("PutStart="), std::string::npos); + EXPECT_EQ(window_summary.find("/s"), std::string::npos); + EXPECT_EQ(window_summary.find("PutStart=3/4"), std::string::npos); + EXPECT_NE(window_summary.find("Batch Requests (per sec"), + std::string::npos); + EXPECT_NE( + window_summary.find("Eviction: Success/Attempts=1/2, AllocFail=0, " + "keys=3, size=4.00 KB"), + std::string::npos); + + const std::string reported_summary = + metrics.get_summary_string_and_update_snapshot(); + EXPECT_NE( + reported_summary.find("Eviction: Success/Attempts=1/2, AllocFail=0, " + "keys=3, size=4.00 KB"), + std::string::npos); + + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + const std::string idle_summary = + metrics.get_summary_string_and_update_snapshot(); + EXPECT_NE(idle_summary.find("PutStart=0.00/0.00"), std::string::npos); + EXPECT_NE(idle_summary.find("Eviction: Success/Attempts=0/0, " + "AllocFail=0, keys=0, size=0 B"), + std::string::npos); +} + } // namespace mooncake::test int main(int argc, char** argv) { -- 2.34.1 From e0b1862a4116ab5d8a4556df41bd7cefc613924e Mon Sep 17 00:00:00 2001 From: Baichuan Date: Tue, 19 May 2026 19:13:41 +0800 Subject: [PATCH 092/382] [Docs] fix ssd offload deployment doc (#2139) Co-authored-by: liubaichuan --- docs/source/deployment/ssd-offload.md | 52 ++++++++++++++++++--------- 1 file changed, 36 insertions(+), 16 deletions(-) diff --git a/docs/source/deployment/ssd-offload.md b/docs/source/deployment/ssd-offload.md index 96cd6be5..970d1335 100644 --- a/docs/source/deployment/ssd-offload.md +++ b/docs/source/deployment/ssd-offload.md @@ -4,7 +4,12 @@ Mooncake Store supports offloading KV cache objects from distributed memory to local SSD. When memory pressure is high, the master instructs clients to persist selected objects to disk. On a cache miss, the client automatically falls back to reading from SSD. -SSD offload is currently **only available in Real Client mode**. The real client is a standalone process that communicates with the application (e.g., SGLang) via RPC. All SSD reads and writes happen within this process. +SSD offload requires the **Real Client** and supports two deployment modes: + +- **Mode A: Embedded Real Client** — the Python process embeds the Real Client, and SSD offload runs inside the Python process. +- **Mode B: Standalone Real Client + DummyClient** — a standalone `mooncake_client` process runs SSD offload, and the Python process connects via a DummyClient. + +In both modes, all SSD reads and writes happen within the Real Client (embedded or standalone). ## Startup Steps @@ -22,9 +27,30 @@ mooncake_master \ --enable_offload=true ``` -### Step 3: Start the real client with SSD offload enabled +### Step 3A (Mode A): Start the application with embedded Real Client -Use the `--enable_offload` flag to enable SSD offload, and set environment variables to specify the storage path and backend: +Use the `--enable_ssd_offload` flag to enable SSD offload, and set environment variables to specify the storage path and backend: + +```python +from mooncake.store import MooncakeDistributedStore + +store = MooncakeDistributedStore() +store.setup( + local_hostname="", + metadata_server="P2PHANDSHAKE", + global_segment_size=4 * 1024 * 1024 * 1024, # 4 GB + local_buffer_size=512 * 1024 * 1024, # 512 MB + protocol="rdma", + rdma_devices="eth0", + master_server_addr="127.0.0.1:50051", + enable_ssd_offload=True, + ssd_offload_path="/nvme/mooncake_offload" +) +``` + +### Step 3B (Mode B): Start the standalone real client with SSD offload enabled and connect it with the dummy client + +Set the same SSD offload environment variables as in Mode A. ```bash export MOONCAKE_OFFLOAD_FILE_STORAGE_PATH=/nvme/mooncake_offload @@ -41,27 +67,21 @@ mooncake_client \ --metadata_server="P2PHANDSHAKE" ``` -> **Note:** When using `bucket_storage_backend` or `file_per_key_storage_backend`, the real client scans existing SSD metadata on startup and reports it to the master automatically. `offset_allocator_storage_backend` is the exception: it truncates its data file during initialization and does not recover previously offloaded objects after a restart. - -### Step 4: Connect the application to the real client - -The application (e.g., SGLang) connects to the real client via the `MooncakeDistributedStore` Python SDK. SSD offload and fallback loading are handled transparently. +The application (e.g., SGLang) connects to the standalone real client via the `MooncakeDistributedStore` Python SDK. The Python process acts as a DummyClient; SSD offload and fallback loading are handled by the standalone real client process. ```python from mooncake.store import MooncakeDistributedStore store = MooncakeDistributedStore() -store.setup( - local_hostname="", - metadata_server="P2PHANDSHAKE", - global_segment_size=4 * 1024 * 1024 * 1024, # 4 GB - local_buffer_size=512 * 1024 * 1024, #512MB - protocol="rdma", - device_name="eth0", - master_server_address="127.0.0.1:50051", +store.setup_dummy( + mem_pool_size=4 * 1024 * 1024 * 1024, # 4 GB + local_buffer_size=512 * 1024 * 1024, # 512 MB + server_address=":50052" # mooncake_client RPC address from Step 3 ) ``` +> **Note:** When using `bucket_storage_backend` or `file_per_key_storage_backend`, the real client scans existing SSD metadata on startup and reports it to the master automatically. `offset_allocator_storage_backend` is the exception: it truncates its data file during initialization and does not recover previously offloaded objects after a restart. + --- ## Real Client Parameters -- 2.34.1 From 74c41dfa7e5404b4616356690df48cf34fe9b32a Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Tue, 19 May 2026 22:07:47 +0800 Subject: [PATCH 093/382] [CI/Build] Support configurable build dir in build_wheel.sh (#2133) --- scripts/build_wheel.sh | 49 +++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 24 deletions(-) diff --git a/scripts/build_wheel.sh b/scripts/build_wheel.sh index 3b0456ab..4dbfa341 100755 --- a/scripts/build_wheel.sh +++ b/scripts/build_wheel.sh @@ -13,10 +13,11 @@ OUTPUT_DIR=${OUTPUT_DIR:-${2:-"dist"}} # CMake build directory (default: build). EP/PG extensions are staged under # ${BUILD_DIR}/ep_pg_staging when the project was built with -DWITH_EP=ON. BUILD_DIR="${BUILD_DIR:-build}" +BUILD_DIR_ABS="$(pwd)/${BUILD_DIR}" echo "Building wheel for Python ${PYTHON_VERSION} with output directory ${OUTPUT_DIR}" # Ensure LD_LIBRARY_PATH includes /usr/local/lib -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/$(pwd)/build/mooncake-common:/usr/local/lib +export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${BUILD_DIR_ABS}/mooncake-common:/usr/local/lib echo "Cleaning wheel-build directory" rm -rf mooncake-wheel/mooncake_transfer_engine* @@ -29,20 +30,20 @@ echo "Creating directory structure..." cp mooncake-integration/fabric_allocator_utils.py mooncake-wheel/mooncake/fabric_allocator_utils.py # Copy engine.so to mooncake directory (will be imported by transfer module) -cp build/mooncake-integration/engine.*.so mooncake-wheel/mooncake/engine.so +cp ${BUILD_DIR}/mooncake-integration/engine.*.so mooncake-wheel/mooncake/engine.so # Copy libasio.so to mooncake directory (runtime dependency of engine.so) -cp build/mooncake-common/libasio.so mooncake-wheel/mooncake/libasio.so +cp ${BUILD_DIR}/mooncake-common/libasio.so mooncake-wheel/mooncake/libasio.so # Copy store.so to mooncake directory -if [ -f build/mooncake-integration/store.*.so ]; then +if compgen -G "${BUILD_DIR}/mooncake-integration/store.*.so" >/dev/null; then echo "Copying store.so..." - cp build/mooncake-integration/store.*.so mooncake-wheel/mooncake/store.so + cp ${BUILD_DIR}/mooncake-integration/store.*.so mooncake-wheel/mooncake/store.so echo "Copying master binary..." # Copy master binary - cp build/mooncake-store/src/mooncake_master mooncake-wheel/mooncake/ + cp ${BUILD_DIR}/mooncake-store/src/mooncake_master mooncake-wheel/mooncake/ # Copy client binary - cp build/mooncake-store/src/mooncake_client mooncake-wheel/mooncake/ + cp ${BUILD_DIR}/mooncake-store/src/mooncake_client mooncake-wheel/mooncake/ # Copy async_store.py cp mooncake-integration/store/async_store.py mooncake-wheel/mooncake/async_store.py else @@ -50,36 +51,36 @@ else fi # Copy libmooncake_store.so to mooncake directory (only when BUILD_SHARED_LIBS is set) -if [ -f build/mooncake-store/src/libmooncake_store.so ]; then +if [ -f ${BUILD_DIR}/mooncake-store/src/libmooncake_store.so ]; then echo "Copying libmooncake_store.so..." - cp build/mooncake-store/src/libmooncake_store.so mooncake-wheel/mooncake/libmooncake_store.so + cp ${BUILD_DIR}/mooncake-store/src/libmooncake_store.so mooncake-wheel/mooncake/libmooncake_store.so fi # Copy libtransfer_engine.so to mooncake directory (only when USE_ETCD is set) -if [ -f build/mooncake-common/etcd/libetcd_wrapper.so ]; then +if [ -f ${BUILD_DIR}/mooncake-common/etcd/libetcd_wrapper.so ]; then echo "Copying libetcd_wrapper.so..." - cp build/mooncake-common/etcd/libetcd_wrapper.so mooncake-wheel/mooncake/libetcd_wrapper.so + cp ${BUILD_DIR}/mooncake-common/etcd/libetcd_wrapper.so mooncake-wheel/mooncake/libetcd_wrapper.so fi # Copy libtransfer_engine.so to mooncake directory (only when BUILD_SHARED_LIBS is set) -if [ -f build/mooncake-transfer-engine/src/libtransfer_engine.so ]; then +if [ -f ${BUILD_DIR}/mooncake-transfer-engine/src/libtransfer_engine.so ]; then echo "Copying libtransfer_engine.so..." - cp build/mooncake-transfer-engine/src/libtransfer_engine.so mooncake-wheel/mooncake/libtransfer_engine.so + cp ${BUILD_DIR}/mooncake-transfer-engine/src/libtransfer_engine.so mooncake-wheel/mooncake/libtransfer_engine.so fi # Copy ascend_transport.so to mooncake directory (only when USE_ASCEND_DIRECT is set) -if [ -f build/mooncake-transfer-engine/src/transport/ascend_transport/ascend_transport.so ]; then +if [ -f ${BUILD_DIR}/mooncake-transfer-engine/src/transport/ascend_transport/ascend_transport.so ]; then echo "Copying ascend_transport.so..." - cp build/mooncake-transfer-engine/src/transport/ascend_transport/ascend_transport.so mooncake-wheel/mooncake/ascend_transport.so + cp ${BUILD_DIR}/mooncake-transfer-engine/src/transport/ascend_transport/ascend_transport.so mooncake-wheel/mooncake/ascend_transport.so fi # Copy nvlink-allocator.so to mooncake directory (only if it exists - CUDA builds only) -if [ -f build/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.so ] \ +if [ -f ${BUILD_DIR}/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.so ] \ || [ -f /usr/lib/libaccl_barex.so ] \ || [ -f /usr/lib64/libaccl_barex.so ]; then - if [ -f build/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.so ]; then + if [ -f ${BUILD_DIR}/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.so ]; then echo "Copying CUDA nvlink_allocator.so..." - cp build/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.so mooncake-wheel/mooncake/nvlink_allocator.so + cp ${BUILD_DIR}/mooncake-transfer-engine/nvlink-allocator/nvlink_allocator.so mooncake-wheel/mooncake/nvlink_allocator.so fi echo "Copying allocator libraries..." # Copy allocator.py @@ -89,9 +90,9 @@ else fi # Copy ubshmem_fabric_allocator.so to mooncake directory (only if it exists - NPU builds only) -if [ -f build/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.so ]; then +if [ -f ${BUILD_DIR}/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.so ]; then echo "Copying NPU ubshmem_fabric_allocator.so..." - cp build/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.so mooncake-wheel/mooncake/ubshmem_fabric_allocator.so + cp ${BUILD_DIR}/mooncake-transfer-engine/ubshmem-allocator/ubshmem_fabric_allocator.so mooncake-wheel/mooncake/ubshmem_fabric_allocator.so echo "Copying NPU allocator libraries..." # Copy allocator_ascend_npu.py cp mooncake-integration/allocator_ascend_npu.py mooncake-wheel/mooncake/allocator_ascend_npu.py @@ -101,10 +102,10 @@ fi echo "Copying transfer_engine_bench..." # Copy transfer_engine_bench -cp build/mooncake-transfer-engine/example/transfer_engine_bench mooncake-wheel/mooncake/ +cp ${BUILD_DIR}/mooncake-transfer-engine/example/transfer_engine_bench mooncake-wheel/mooncake/ -if [ -f "build/mooncake-transfer-engine/src/transport/ascend_transport/hccl_transport/ascend_transport_c/libascend_transport_mem.so" ]; then - cp build/mooncake-transfer-engine/src/transport/ascend_transport/hccl_transport/ascend_transport_c/libascend_transport_mem.so mooncake-wheel/mooncake/ +if [ -f "${BUILD_DIR}/mooncake-transfer-engine/src/transport/ascend_transport/hccl_transport/ascend_transport_c/libascend_transport_mem.so" ]; then + cp ${BUILD_DIR}/mooncake-transfer-engine/src/transport/ascend_transport/hccl_transport/ascend_transport_c/libascend_transport_mem.so mooncake-wheel/mooncake/ echo "Copying ascend_transport_mem libraries..." else echo "Skipping libascend_transport_mem.so (not built - Ascend disabled)" @@ -116,7 +117,7 @@ fi # so that patchelf never touches CUDA fatbins (see injection step below). # Use an absolute path: the script later `cd`s into mooncake-wheel/ and a # relative path would silently point to the wrong location. -CUDA_EP_STAGING_DIR="$(pwd)/${BUILD_DIR}/ep_pg_staging" +CUDA_EP_STAGING_DIR="${BUILD_DIR_ABS}/ep_pg_staging" # CI only: remove build/ to free disk before python -m build (set FREE_BUILD_DIR=1 to enable locally). # If EP/PG .so files were staged inside the build directory, preserve them in a -- 2.34.1 From d9e8aee0652f5493d5eccf28897cb7a569467ef8 Mon Sep 17 00:00:00 2001 From: Lewis <63569348+TTThanos@users.noreply.github.com> Date: Wed, 20 May 2026 08:18:22 +0800 Subject: [PATCH 094/382] [TE] Update IntraNode NVLink transfer method cuMemcpy -> cuMemcpyAsync (#2012) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --------- Co-authored-by: 百麒 --- .../include/transport/transport.h | 8 +++ .../src/multi_transport.cpp | 58 +++++++++++---- .../intranode_nvlink_transport.cpp | 70 +++++++++++++++---- 3 files changed, 108 insertions(+), 28 deletions(-) diff --git a/mooncake-transfer-engine/include/transport/transport.h b/mooncake-transfer-engine/include/transport/transport.h index e3a22c7e..60f46fba 100644 --- a/mooncake-transfer-engine/include/transport/transport.h +++ b/mooncake-transfer-engine/include/transport/transport.h @@ -135,6 +135,8 @@ class Transport { } ub; struct { void *dest_addr; + void *cuda_stream; // cudaStream_t, used by async NVLink + // transport } local; struct { uint64_t dest_addr; @@ -287,6 +289,12 @@ class Transport { uint64_t total_bytes = 0; BatchID batch_id = 0; + // Pointer to the transport that handles this task, set by + // MultiTransport::submitTransfer(). Used to delegate + // transport-specific completion polling (e.g., CUDA stream + // query for NVLink async transfers) in getTransferStatus(). + Transport *transport_ = nullptr; + #ifdef WITH_METRICS std::chrono::steady_clock::time_point start_time; #endif diff --git a/mooncake-transfer-engine/src/multi_transport.cpp b/mooncake-transfer-engine/src/multi_transport.cpp index 687262f9..69d0ed2d 100644 --- a/mooncake-transfer-engine/src/multi_transport.cpp +++ b/mooncake-transfer-engine/src/multi_transport.cpp @@ -124,6 +124,7 @@ Status MultiTransport::submitTransfer( assert(transport); auto& task = batch_desc.task_list[task_id]; task.batch_id = batch_id; + task.transport_ = transport; #ifdef USE_ASCEND_HETEROGENEOUS task.request = const_cast(&request); #else @@ -195,6 +196,45 @@ Status MultiTransport::getTransferStatus(BatchID batch_id, size_t task_id, return Status::InvalidArgument("Task ID out of range"); } auto& task = batch_desc.task_list[task_id]; + + // Helper: check if any slice has exceeded the configured timeout. + // Returns true if a timeout was detected (and logs it). + auto checkSliceTimeout = [&](const Transport::TransferTask& t) -> bool { + if (globalConfig().slice_timeout <= 0) return false; + auto current_ts = getCurrentTimeInNano(); + const int64_t kPacketDeliveryTimeout = + globalConfig().slice_timeout * 1000000000; + for (auto& slice : t.slice_list) { + auto ts = slice->ts; + if (ts > 0 && current_ts > ts && + current_ts - ts > kPacketDeliveryTimeout) { + LOG(INFO) << "Slice timeout detected"; + return true; + } + } + return false; + }; + + // If the task has an associated transport, delegate to its + // getTransferStatus() to trigger transport-specific completion + // polling. For example, the NVLink async transport polls CUDA + // streams via cudaStreamQuery() here; without this call the + // slice statuses (and therefore success/failed_slice_count) + // would never be updated. + if (task.transport_) { + auto ret = + task.transport_->getTransferStatus(batch_id, task_id, status); + if (!ret.ok()) return ret; + + // Apply timeout check on top of the transport's result. + if (status.s == Transport::TransferStatusEnum::WAITING && + checkSliceTimeout(task)) { + status.s = Transport::TransferStatusEnum::TIMEOUT; + } + return Status::OK(); + } + + // Fallback for tasks without a transport pointer (legacy path) status.transferred_bytes = task.transferred_bytes; uint64_t success_slice_count = task.success_slice_count; uint64_t failed_slice_count = task.failed_slice_count; @@ -207,21 +247,11 @@ Status MultiTransport::getTransferStatus(BatchID batch_id, size_t task_id, } task.is_finished = true; } else { - if (globalConfig().slice_timeout > 0) { - auto current_ts = getCurrentTimeInNano(); - const int64_t kPacketDeliveryTimeout = - globalConfig().slice_timeout * 1000000000; - for (auto& slice : task.slice_list) { - auto ts = slice->ts; - if (ts > 0 && current_ts > ts && - current_ts - ts > kPacketDeliveryTimeout) { - LOG(INFO) << "Slice timeout detected"; - status.s = Transport::TransferStatusEnum::TIMEOUT; - return Status::OK(); - } - } + if (checkSliceTimeout(task)) { + status.s = Transport::TransferStatusEnum::TIMEOUT; + } else { + status.s = Transport::TransferStatusEnum::WAITING; } - status.s = Transport::TransferStatusEnum::WAITING; } return Status::OK(); } diff --git a/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp b/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp index bf9bfb63..01c3773a 100644 --- a/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp +++ b/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp @@ -32,6 +32,21 @@ #include "transfer_metadata.h" #include "transport/transport.h" +namespace { +struct CudaStreamNVLinkRAII { + cudaStream_t stream_; + CudaStreamNVLinkRAII() : stream_(nullptr) { + auto err = cudaStreamCreateWithFlags(&stream_, cudaStreamNonBlocking); + if (err != cudaSuccess) { + LOG(FATAL) << "Failed to create NVLink CUDA stream: " << err + << " - " << cudaGetErrorString(err); + } + } + ~CudaStreamNVLinkRAII() { cudaStreamDestroy(stream_); } +}; +static thread_local CudaStreamNVLinkRAII tl_nvlink_stream; +} // namespace + static bool checkCudaErrorReturn(cudaError_t result, const char *message) { if (result != cudaSuccess) { LOG(ERROR) << message << " (Error code: " << result << " - " @@ -194,18 +209,25 @@ Status IntraNodeNvlinkTransport::submitTransfer( slice->task = &task; slice->target_id = request.target_id; slice->status = Slice::PENDING; + slice->ts = getCurrentTimeInNano(); + task.slice_list.push_back(slice); __sync_fetch_and_add(&task.slice_count, 1); + cudaStream_t stream = tl_nvlink_stream.stream_; cudaError_t err; if (slice->opcode == TransferRequest::READ) - err = cudaMemcpy(slice->source_addr, (void *)slice->local.dest_addr, - slice->length, cudaMemcpyDefault); + err = cudaMemcpyAsync(slice->source_addr, + (void *)slice->local.dest_addr, slice->length, + cudaMemcpyDefault, stream); else - err = cudaMemcpy((void *)slice->local.dest_addr, slice->source_addr, - slice->length, cudaMemcpyDefault); - if (err != cudaSuccess) + err = cudaMemcpyAsync((void *)slice->local.dest_addr, + slice->source_addr, slice->length, + cudaMemcpyDefault, stream); + if (err != cudaSuccess) { slice->markFailed(); - else - slice->markSuccess(); + } else { + slice->status = Slice::POSTED; + slice->local.cuda_stream = (void *)stream; + } } return Status::OK(); @@ -223,6 +245,19 @@ Status IntraNodeNvlinkTransport::getTransferStatus(BatchID batch_id, std::to_string(batch_id)); } auto &task = batch_desc.task_list[task_id]; + // Poll POSTED slices for async completion via cudaStreamQuery + for (auto *slice : task.slice_list) { + if (slice && slice->status == Slice::POSTED) { + cudaStream_t stream = (cudaStream_t)slice->local.cuda_stream; + cudaError_t cuda_err = cudaStreamQuery(stream); + if (cuda_err == cudaSuccess) { + slice->markSuccess(); + } else if (cuda_err != cudaErrorNotReady) { + slice->markFailed(); + } + // cudaErrorNotReady means still in progress, keep POSTED + } + } status.transferred_bytes = task.transferred_bytes; uint64_t success_slice_count = task.success_slice_count; uint64_t failed_slice_count = task.failed_slice_count; @@ -261,19 +296,26 @@ Status IntraNodeNvlinkTransport::submitTransferTask( slice->task = &task; slice->target_id = request.target_id; slice->status = Slice::PENDING; + slice->ts = getCurrentTimeInNano(); task.slice_list.push_back(slice); __sync_fetch_and_add(&task.slice_count, 1); + cudaStream_t stream = tl_nvlink_stream.stream_; + cudaError_t err; if (slice->opcode == TransferRequest::READ) - err = cudaMemcpy(slice->source_addr, (void *)slice->local.dest_addr, - slice->length, cudaMemcpyDefault); + err = cudaMemcpyAsync(slice->source_addr, + (void *)slice->local.dest_addr, slice->length, + cudaMemcpyDefault, stream); else - err = cudaMemcpy((void *)slice->local.dest_addr, slice->source_addr, - slice->length, cudaMemcpyDefault); - if (err != cudaSuccess) + err = cudaMemcpyAsync((void *)slice->local.dest_addr, + slice->source_addr, slice->length, + cudaMemcpyDefault, stream); + if (err != cudaSuccess) { slice->markFailed(); - else - slice->markSuccess(); + } else { + slice->status = Slice::POSTED; + slice->local.cuda_stream = (void *)stream; + } } return Status::OK(); } -- 2.34.1 From 58ca374d78d771adaf7d2b770b1b5afbc30ffbae Mon Sep 17 00:00:00 2001 From: Stary Date: Wed, 20 May 2026 09:46:19 +0800 Subject: [PATCH 095/382] ci: refactor TENT build and add test step (#2142) - Remove CUDA-related flags and environment variables - Add new test step for TENT unit tests Signed-off-by: Yuxin Chen Co-authored-by: Yuxin Chen --- .github/workflows/ci.yml | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c5f6e1cb..b47765f6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -622,18 +622,22 @@ jobs: run: | mkdir build-tent cd build-tent - cmake -G Ninja .. -DUSE_TENT=ON -DUSE_HTTP=ON -DUSE_CUDA=ON -DENABLE_SCCACHE=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/cuda/lib64/stubs" + cmake -G Ninja .. -DUSE_TENT=ON -DUSE_HTTP=ON -DENABLE_SCCACHE=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON shell: bash - name: Build project with TENT run: | - export LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LD_LIBRARY_PATH - export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LIBRARY_PATH cd build-tent cmake --build . sudo cmake --install . shell: bash + - name: Test (TENT) + run: | + cd build-tent + ctest --test-dir mooncake-transfer-engine/tent/tests -j --output-on-failure + shell: bash + - name: Build nvlink_allocator.so run: | mkdir -p build/mooncake-transfer-engine/nvlink-allocator -- 2.34.1 From 2b2a46c9f4919c6cdb68c75474518326e140bfe6 Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Wed, 20 May 2026 09:46:20 +0800 Subject: [PATCH 096/382] [PG] support external TE (#2149) --- mooncake-pg/include/mooncake_backend.h | 12 ++++++++++++ mooncake-pg/src/mooncake_backend.cpp | 15 ++++++++++++++- mooncake-pg/src/pg_py.cpp | 21 +++++++++++++++++++++ 3 files changed, 47 insertions(+), 1 deletion(-) diff --git a/mooncake-pg/include/mooncake_backend.h b/mooncake-pg/include/mooncake_backend.h index 26058722..7e6ee010 100644 --- a/mooncake-pg/include/mooncake_backend.h +++ b/mooncake-pg/include/mooncake_backend.h @@ -157,6 +157,13 @@ class MooncakeBackend final : public ::c10d::ProcessGroup { engine_->setWhitelistFilters(std::move(filters)); } + /// Set an external TransferEngine to be used by MooncakeBackend + /// instead of creating its own. Must be called before + /// init_process_group(backend="mooncake"). The engine must already + /// be initialized. The caller is responsible for ensuring the engine + /// outlives all MooncakeBackend instances. Pass nullptr to reset. + static void setExternalEngine(TransferEngine* engine); + std::string getPreferredHca(std::string location) { static std::once_flag topo_once; static std::shared_ptr topology; @@ -211,6 +218,11 @@ class MooncakeBackend final : public ::c10d::ProcessGroup { std::shared_ptr worker_; static bool engineInitialized_; static int backendIndex_; + // External engine injection: when set, MooncakeBackend uses this engine + // instead of the default self-created one. Non-owning pointer. + // The caller is responsible for ensuring the engine outlives all + // MooncakeBackend instances. + static TransferEngine* externalEngine_; const c10::intrusive_ptr options_; bool isCpu_{false}; static std::string hostIp_; diff --git a/mooncake-pg/src/mooncake_backend.cpp b/mooncake-pg/src/mooncake_backend.cpp index 27466f42..ebac10fb 100644 --- a/mooncake-pg/src/mooncake_backend.cpp +++ b/mooncake-pg/src/mooncake_backend.cpp @@ -32,6 +32,7 @@ TransferEngine* MooncakeBackend::engine_ = new TransferEngine(true); // worker_ is now owned per backend instance via MooncakeWorkerManager. bool MooncakeBackend::engineInitialized_ = false; int MooncakeBackend::backendIndex_ = 0; +TransferEngine* MooncakeBackend::externalEngine_ = nullptr; std::vector serialize(const ExtensionState& state) { uint32_t rankCount = static_cast(state.activeRanks.size()); @@ -206,7 +207,11 @@ MooncakeBackend::MooncakeBackend( } // Initialize transfer engine - if (!engineInitialized_) { + if (externalEngine_) { + // Use externally-provided engine (already initialized), skip init. + engine_ = externalEngine_; + engineInitialized_ = true; + } else if (!engineInitialized_) { engine_->init(P2PHANDSHAKE, hostIp_); engineInitialized_ = true; } @@ -1248,4 +1253,12 @@ void MooncakeBackend::joinGroup() { connection_ctx_->waitUntilAllConnected(); waitForExtensionState(); } + +void MooncakeBackend::setExternalEngine(TransferEngine* engine) { + externalEngine_ = engine; + if (engine) { + LOG(INFO) << "MooncakeBackend: external TransferEngine set (ptr=" + << engine << ")"; + } +} } // namespace mooncake diff --git a/mooncake-pg/src/pg_py.cpp b/mooncake-pg/src/pg_py.cpp index 09b1c6d3..b9e379a7 100644 --- a/mooncake-pg/src/pg_py.cpp +++ b/mooncake-pg/src/pg_py.cpp @@ -85,11 +85,32 @@ void joinGroup(c10::intrusive_ptr backend) { mooncakeBackend->joinGroup(); } +/// Python-facing wrapper that extracts the raw TransferEngine* from a +/// mooncake.engine.TransferEngine Python object and passes it to +/// MooncakeBackend::setExternalEngine(). The caller must ensure the +/// TransferEnginePy object outlives all MooncakeBackend instances. +void setTransferEnginePy(pybind11::object engine_obj) { + if (engine_obj.is_none()) { + MooncakeBackend::setExternalEngine(nullptr); + return; + } + auto get_engine_ptr = engine_obj.attr("get_engine_ptr"); + uintptr_t ptr = get_engine_ptr().cast(); + auto* engine = reinterpret_cast(ptr); + MooncakeBackend::setExternalEngine(engine); +} + PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) { m.def("createMooncakeBackend", &createMooncakeBackend); m.def("createMooncakeCpuBackend", &createMooncakeCpuBackend); m.def("set_host_ip", &MooncakeBackend::setHostIp); m.def("set_device_filter", &MooncakeBackend::setDeviceFilter); + m.def("set_transfer_engine", &setTransferEnginePy, py::arg("engine"), + "Set an external TransferEngine to be used by MooncakeBackend. " + "Must be called before init_process_group(). The engine must already " + "be initialized. Pass None to reset to default behavior. " + "The caller must ensure the TransferEngine object outlives all " + "MooncakeBackend instances."); m.def("get_preferred_hca", &getPreferredHca); m.def("get_active_ranks", &getActiveRanks); m.def("get_num_synced_ranks", &getNumSyncedRanks); -- 2.34.1 From fadeb394db3b45f46525b956c39b7246befd3cd5 Mon Sep 17 00:00:00 2001 From: Kafka <92581531+ZhenyuePan@users.noreply.github.com> Date: Wed, 20 May 2026 09:48:41 +0800 Subject: [PATCH 097/382] [TransferEngine] Gate auto failover on status polling (#2122) --- .../tent/config/transfer-engine.json | 1 + .../tent/runtime/transfer_engine_impl.h | 4 + .../tent/src/runtime/transfer_engine_impl.cpp | 34 +++--- .../tent/tests/engine_failover_e2e_test.cpp | 102 ++++++++++++++++++ .../tent/tests/failover_test.cpp | 11 ++ 5 files changed, 135 insertions(+), 17 deletions(-) diff --git a/mooncake-transfer-engine/tent/config/transfer-engine.json b/mooncake-transfer-engine/tent/config/transfer-engine.json index 1fd2a5eb..79fd0d91 100644 --- a/mooncake-transfer-engine/tent/config/transfer-engine.json +++ b/mooncake-transfer-engine/tent/config/transfer-engine.json @@ -11,6 +11,7 @@ }, "log_level": "warning", "max_failover_attempts": 3, + "enable_auto_failover_on_poll": true, "metrics": { "enabled": true, "http_port": 9100, diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h b/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h index 3d9ccf90..3dc4806a 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h @@ -186,6 +186,9 @@ class TransferEngineImpl { Status resubmitTransferTask(Batch* batch, size_t task_id); + void updateTaskStatusFromPoll(Batch* batch, size_t task_id, + TransferStatus& task_status); + TransportType resolveTransport(const Request& req, int priority, bool invalidate_on_fail = true); @@ -236,6 +239,7 @@ class TransferEngineImpl { std::unique_ptr staging_proxy_; bool merge_requests_; int max_failover_attempts_{3}; + bool enable_auto_failover_on_poll_{true}; }; } // namespace tent } // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index f073ea35..d7870f97 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -304,6 +304,8 @@ Status TransferEngineImpl::construct() { CHECK_STATUS(getRpcServerPortFromConfig(*conf_, 0, port_)); merge_requests_ = conf_->get("merge_requests", true); max_failover_attempts_ = conf_->get("max_failover_attempts", 3); + enable_auto_failover_on_poll_ = + conf_->get("enable_auto_failover_on_poll", true); if (!hostname_.empty()) CHECK_STATUS(checkLocalIpAddress(hostname_, ipv6_)); else @@ -1304,6 +1306,18 @@ Status TransferEngineImpl::resubmitTransferTask(Batch* batch, size_t task_id) { return transport->submitTransferTasks(sub_batch, {task.request}); } +void TransferEngineImpl::updateTaskStatusFromPoll(Batch* batch, size_t task_id, + TransferStatus& task_status) { + auto& task = batch->task_list[task_id]; + task.status = task_status.s; + if (!enable_auto_failover_on_poll_ || task_status.s != FAILED) return; + + if (resubmitTransferTask(batch, task_id).ok()) { + task_status.s = PENDING; + task.status = PENDING; + } +} + Status TransferEngineImpl::sendNotification(SegmentID target_id, const Notification& notifi) { for (size_t type = 0; type < kSupportedTransportTypes; ++type) { @@ -1368,12 +1382,7 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, size_t task_id, CHECK_STATUS(transport->getTransferStatus(sub_batch, task.sub_task_id, task_status)); } - batch->task_list[task_id].status = task_status.s; - - if (task_status.s == FAILED && resubmitTransferTask(batch, task_id).ok()) { - task_status.s = PENDING; - batch->task_list[task_id].status = PENDING; - } + updateTaskStatusFromPoll(batch, task_id, task_status); // Record metrics when task transitions to terminal state recordTaskCompletionMetrics(batch->task_list[task_id], prev_status, @@ -1448,17 +1457,8 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, CHECK_STATUS(transport->getTransferStatus( sub_batch, task.sub_task_id, task_status)); } - // memorize task result - task.status = task_status.s; - - // Attempt failover before status aggregation so that a - // successfully resubmitted task appears as PENDING and does not - // latch the batch to a terminal state while retries are in-flight. - if (task_status.s == FAILED && - resubmitTransferTask(batch, task_id).ok()) { - task.status = PENDING; - task_status.s = PENDING; - } + // Preserve legacy auto-failover-on-poll before aggregating status. + updateTaskStatusFromPoll(batch, task_id, task_status); if (task_status.s == COMPLETED) { success_tasks++; diff --git a/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp b/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp index d7c5ef17..d2500c7a 100644 --- a/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp +++ b/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp @@ -218,6 +218,47 @@ TransferStatus pollUntilDone( return ts; } +struct CorruptedRdmaBatch { + std::shared_ptr fake_rdma; + std::shared_ptr fake_tcp; + std::vector buf; + BatchID batch_id{0}; +}; + +void submitCorruptedRdmaBatch(TransferEngineImpl& engine, + CorruptedRdmaBatch& batch, uint8_t fill) { + batch.fake_rdma = std::make_shared(RDMA); + batch.fake_tcp = std::make_shared(TCP); + + FaultPolicy rdma_policy; + rdma_policy.status_corrupt_rate = 1.0; + auto proxied_rdma = + std::make_shared(batch.fake_rdma, rdma_policy); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(batch.fake_tcp->install(seg_name, nullptr, nullptr).ok()); + + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, batch.fake_tcp); + + constexpr size_t kBufLen = 4096; + batch.buf.assign(kBufLen, fill); + ASSERT_TRUE(engine.registerLocalMemory(batch.buf.data(), kBufLen).ok()); + + batch.batch_id = engine.allocateBatch(8); + ASSERT_NE(batch.batch_id, (BatchID)0); + + Request req; + req.opcode = Request::WRITE; + req.source = batch.buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(batch.buf.data()); + req.length = kBufLen; + + ASSERT_TRUE(engine.submitTransfer(batch.batch_id, {req}).ok()); +} + // --------------------------------------------------------------------------- // P0: Completion reports FAILED (simulates WC error / QP error / peer drop // mid-transfer). Engine must failover. @@ -271,6 +312,67 @@ TEST(EngineFailoverE2E, StatusCorruptionTriggersFailoverToSecondary) { EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); } +TEST(EngineFailoverE2E, AutoFailoverOnPollDisabledLeavesTaskFailed) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + CorruptedRdmaBatch batch; + submitCorruptedRdmaBatch(engine, batch, 0xA1); + + TransferStatus status{}; + ASSERT_TRUE(engine.getTransferStatus(batch.batch_id, 0, status).ok()); + EXPECT_EQ(status.s, TransferStatusEnum::FAILED); + EXPECT_EQ(batch.fake_rdma->submit_calls.load(), 1); + EXPECT_EQ(batch.fake_tcp->submit_calls.load(), 0); + + EXPECT_TRUE(engine.freeBatch(batch.batch_id).ok()); + EXPECT_TRUE( + engine.unregisterLocalMemory(batch.buf.data(), batch.buf.size()).ok()); +} + +TEST(EngineFailoverE2E, AutoFailoverOnPollDisabledAppliesToVectorStatus) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + CorruptedRdmaBatch batch; + submitCorruptedRdmaBatch(engine, batch, 0xA2); + + std::vector status_list; + ASSERT_TRUE(engine.getTransferStatus(batch.batch_id, status_list).ok()); + ASSERT_EQ(status_list.size(), 1); + EXPECT_EQ(status_list[0].s, TransferStatusEnum::FAILED); + EXPECT_EQ(batch.fake_rdma->submit_calls.load(), 1); + EXPECT_EQ(batch.fake_tcp->submit_calls.load(), 0); + + EXPECT_TRUE(engine.freeBatch(batch.batch_id).ok()); + EXPECT_TRUE( + engine.unregisterLocalMemory(batch.buf.data(), batch.buf.size()).ok()); +} + +TEST(EngineFailoverE2E, AutoFailoverOnPollDisabledAppliesToOverallStatus) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + CorruptedRdmaBatch batch; + submitCorruptedRdmaBatch(engine, batch, 0xA3); + + TransferStatus overall_status{}; + ASSERT_TRUE(engine.getTransferStatus(batch.batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::FAILED); + EXPECT_EQ(batch.fake_rdma->submit_calls.load(), 1); + EXPECT_EQ(batch.fake_tcp->submit_calls.load(), 0); + + EXPECT_TRUE(engine.freeBatch(batch.batch_id).ok()); + EXPECT_TRUE( + engine.unregisterLocalMemory(batch.buf.data(), batch.buf.size()).ok()); +} + // --------------------------------------------------------------------------- // P1b: Both transports keep failing at status stage -> failover limit reached. // --------------------------------------------------------------------------- diff --git a/mooncake-transfer-engine/tent/tests/failover_test.cpp b/mooncake-transfer-engine/tent/tests/failover_test.cpp index 881168dd..e8cf94ae 100644 --- a/mooncake-transfer-engine/tent/tests/failover_test.cpp +++ b/mooncake-transfer-engine/tent/tests/failover_test.cpp @@ -111,6 +111,17 @@ TEST(FailoverConfigTest, ZeroDisablesFailover) { EXPECT_GT(task.failover_count, max_attempts); } +TEST(FailoverConfigTest, AutoFailoverOnPollDefaultEnabled) { + auto conf = std::make_shared(); + EXPECT_TRUE(conf->get("enable_auto_failover_on_poll", true)); +} + +TEST(FailoverConfigTest, AutoFailoverOnPollCanBeDisabled) { + auto conf = std::make_shared(); + conf->set("enable_auto_failover_on_poll", false); + EXPECT_FALSE(conf->get("enable_auto_failover_on_poll", true)); +} + // --------------------------------------------------------------------------- // TransportType name coverage (tests the static helper indirectly via // the enum values — the function itself is file-local in the .cpp, so we -- 2.34.1 From e73f785f2ddb0c86585ad426878797c98321318a Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Wed, 20 May 2026 10:05:21 +0800 Subject: [PATCH 098/382] [CI/Build] Switch WITH_NVIDIA_PEERMEM to env variable (#2066) * [CI/Build] Switch WITH_NVIDIA_PEERMEM to env variable * [CI/Build] Switch WITH_NVIDIA_PEERMEM to runtime env variable Instead of a cmake build-time option, check the WITH_NVIDIA_PEERMEM environment variable at runtime in rdma_context.cpp and rdma_transport.cpp to switch between ibv_reg_mr() and ibv_reg_dmabuf_mr(). - Remove option(WITH_NVIDIA_PEERMEM) and add_compile_definitions() from common.cmake (no more compile-time flag) - Update CMakeLists.txt to use GPU toolkit presence instead of cmake var for nvlink-allocator build condition - Add withNvidiaPeermem() runtime helper reading WITH_NVIDIA_PEERMEM env var; default false (dmabuf path, no nvidia-peermem required) - Replace #if !defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA) guards with runtime if (!withNvidiaPeermem()) checks Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/05c94a1b-d4d6-4b44-be25-3b98d9b01f1b Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Add WITH_NVIDIA_PEERMEM to Environ; use Environ::Get() in rdma files Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/66e6cabb-c473-4a9d-9711-aebe468fcee2 Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Fix linker error: link transfer_engine against mooncake_common for Environ::Get() Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/29d46e31-6dc1-4fee-be33-4a603537b827 Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Fix linker error in Go CGO builds: add -lmooncake_common to build.sh scripts Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/ab73f098-d408-4ed0-95bb-77f9ce9f71ae Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Add USE_MACA to nvlink-allocator conditions to cover all GPU cases Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/251d2c35-d12a-43ed-9958-40526e0420d5 Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Fix Go CGO linker path: add mooncake-common/src to library search paths Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/5a91e4b9-660d-463b-a8f2-d3fb0118fc34 Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Fix Rust build.rs: add mooncake_common link and CUDA stubs search paths Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/e0d1e4fd-2cc5-4fe9-9268-5c205e3fc0f5 Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Fix Rust build.rs: remove CUDA stubs from search_dirs to prevent runtime libcuda.so.1 dep Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/cd938ceb-c822-4439-b617-03f065015d4c Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Fix Rust build.rs: remove CUDA stubs from early rustc-link-search to prevent libcuda.so.1 runtime dep Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/055de577-7669-4039-a89a-d6066493e2d0 Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> * Fix CI: create libcuda.so.1 symlink and set LD_LIBRARY_PATH for cargo test --lib Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/76131e72-de6f-485b-98ab-342b112f3968 Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> --- .github/workflows/ci.yml | 7 + mooncake-common/common.cmake | 5 - mooncake-common/include/environ.h | 2 + mooncake-common/src/environ.cpp | 1 + mooncake-p2p-store/build.sh | 3 +- mooncake-store/go/build.sh | 3 +- mooncake-store/rust/build.rs | 6 + mooncake-store/src/CMakeLists.txt | 2 +- mooncake-transfer-engine/CMakeLists.txt | 5 +- .../nvlink-allocator/CMakeLists.txt | 5 +- .../nvlink-allocator/build.sh | 9 ++ mooncake-transfer-engine/src/CMakeLists.txt | 1 + .../transport/rdma_transport/rdma_context.cpp | 139 ++++++++++-------- .../rdma_transport/rdma_transport.cpp | 56 ++++--- 14 files changed, 147 insertions(+), 97 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b47765f6..b3e9981a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -594,6 +594,13 @@ jobs: - name: Check Mooncake Store Rust bindings, examples, and tests run: | + # libcuda.so.1 (SONAME of the CUDA stub) must be findable at runtime. + # The toolkit stubs dir only ships libcuda.so; create the versioned symlink. + if [ -f /usr/local/cuda/lib64/stubs/libcuda.so ] && \ + [ ! -e /usr/local/cuda/lib64/stubs/libcuda.so.1 ]; then + sudo ln -s libcuda.so /usr/local/cuda/lib64/stubs/libcuda.so.1 + fi + export LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LD_LIBRARY_PATH cd mooncake-store/rust export MOONCAKE_BUILD_DIR=$GITHUB_WORKSPACE/build cargo test --lib diff --git a/mooncake-common/common.cmake b/mooncake-common/common.cmake index 315e364b..3654f7de 100644 --- a/mooncake-common/common.cmake +++ b/mooncake-common/common.cmake @@ -113,7 +113,6 @@ option(USE_HTTP "option for enable http as metadata server" ON) option(WITH_RUST_EXAMPLE "build the Rust interface and sample code for the transfer engine" OFF) option(WITH_METRICS "enable metrics and metrics reporting thread" ON) option(USE_3FS "option for using 3FS storage backend" OFF) -option(WITH_NVIDIA_PEERMEM "disable to support RDMA without nvidia-peermem. If WITH_NVIDIA_PEERMEM=OFF then USE_CUDA=ON is required." ON) option(USE_EVENT_DRIVEN_COMPLETION "option for using event-driven completion (store & transfer engine)" OFF) option(USE_TENT "option for building Mooncake TENT" OFF) @@ -403,10 +402,6 @@ if(USE_3FS) message(STATUS "3FS storage backend is enabled") endif() -if(WITH_NVIDIA_PEERMEM) - add_compile_definitions(WITH_NVIDIA_PEERMEM) -endif() - set(GFLAGS_USE_TARGET_NAMESPACE "true") find_package(yaml-cpp REQUIRED) find_package(gflags REQUIRED) diff --git a/mooncake-common/include/environ.h b/mooncake-common/include/environ.h index b76c08f1..56106145 100644 --- a/mooncake-common/include/environ.h +++ b/mooncake-common/include/environ.h @@ -49,6 +49,7 @@ class Environ { bool GetForceMnnvl() const { return force_mnnvl_; } bool GetIntraNvlink() const { return intra_nvlink_; } bool GetPathRoundrobin() const { return path_roundrobin_; } + bool GetWithNvidiaPeermem() const { return with_nvidia_peermem_; } private: Environ(); @@ -99,6 +100,7 @@ class Environ { bool force_mnnvl_; bool intra_nvlink_; bool path_roundrobin_; + bool with_nvidia_peermem_; }; } // namespace mooncake diff --git a/mooncake-common/src/environ.cpp b/mooncake-common/src/environ.cpp index 1b63f753..25b3300d 100644 --- a/mooncake-common/src/environ.cpp +++ b/mooncake-common/src/environ.cpp @@ -81,6 +81,7 @@ Environ::Environ() { force_mnnvl_ = GetBool("MC_FORCE_MNNVL", false); intra_nvlink_ = GetBool("MC_INTRA_NVLINK", false); path_roundrobin_ = GetBool("MC_PATH_ROUNDROBIN", false); + with_nvidia_peermem_ = GetBool("WITH_NVIDIA_PEERMEM", false); } } // namespace mooncake diff --git a/mooncake-p2p-store/build.sh b/mooncake-p2p-store/build.sh index 9b98a73e..66eff5f1 100644 --- a/mooncake-p2p-store/build.sh +++ b/mooncake-p2p-store/build.sh @@ -34,7 +34,8 @@ fi EXT_LDFLAGS="-L$BUILD_DIR/mooncake-transfer-engine/src" EXT_LDFLAGS+=" -L$BUILD_DIR/mooncake-transfer-engine/src/common/base" EXT_LDFLAGS+=" -L$BUILD_DIR/mooncake-common" -EXT_LDFLAGS+=" -ltransfer_engine -lbase -lasio -lstdc++ -lnuma -lglog -libverbs -ljsoncpp" +EXT_LDFLAGS+=" -L$BUILD_DIR/mooncake-common/src" +EXT_LDFLAGS+=" -ltransfer_engine -lbase -lasio -lstdc++ -lnuma -lglog -libverbs -ljsoncpp -lmooncake_common" if [ -d "/usr/local/cuda/lib64/stubs" ]; then EXT_LDFLAGS+=" -L/usr/local/cuda/lib64/stubs" diff --git a/mooncake-store/go/build.sh b/mooncake-store/go/build.sh index c1cbf812..9020a280 100755 --- a/mooncake-store/go/build.sh +++ b/mooncake-store/go/build.sh @@ -40,7 +40,8 @@ CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-store/src/cachelib_memory_allocator" CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-transfer-engine/src" CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-transfer-engine/src/common/base" CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-common" -CGO_LDFLAGS+=" -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio" +CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-common/src" +CGO_LDFLAGS+=" -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio -lmooncake_common" CGO_LDFLAGS+=" -lstdc++ -lnuma -lglog -lgflags -libverbs -ljsoncpp -lzstd -lcurl" if [ -d "/usr/local/cuda/lib64" ]; then diff --git a/mooncake-store/rust/build.rs b/mooncake-store/rust/build.rs index ee873a1c..a0a9a140 100644 --- a/mooncake-store/rust/build.rs +++ b/mooncake-store/rust/build.rs @@ -188,6 +188,11 @@ fn main() { "cargo:rustc-link-search=native={}", build_dir.join("mooncake-common").display() ); + // mooncake_common static library lives in the src/ subdirectory. + println!( + "cargo:rustc-link-search=native={}", + build_dir.join("mooncake-common/src").display() + ); // transfer_engine is built in a sibling directory. println!( @@ -218,6 +223,7 @@ fn main() { // Dependencies of mooncake_store that must be satisfied at link time. // The list mirrors what mooncake-store/src/CMakeLists.txt links against. println!("cargo:rustc-link-lib=transfer_engine"); + println!("cargo:rustc-link-lib=mooncake_common"); // Environ::Get() and other common utilities println!("cargo:rustc-link-lib=base"); // mooncake::Status etc. println!("cargo:rustc-link-lib=asio"); // shared library built by mooncake-common println!("cargo:rustc-link-lib=jsoncpp"); // transfer_engine dependency diff --git a/mooncake-store/src/CMakeLists.txt b/mooncake-store/src/CMakeLists.txt index e529bacc..53dbcf9d 100644 --- a/mooncake-store/src/CMakeLists.txt +++ b/mooncake-store/src/CMakeLists.txt @@ -231,7 +231,7 @@ endif() # # Auto-detect each toolkit regardless of global USE_CUDA/USE_HIP flags, # because USE_CUDA may be OFF even when GPU pointers are present -# (e.g. WITH_NVIDIA_PEERMEM=ON uses nvidia-peermem for RDMA without cudart). +# (e.g. WITH_NVIDIA_PEERMEM env var uses nvidia-peermem for RDMA without cudart). # Each detected toolkit gets both link libraries AND compile definitions, # so that gpu_staging_utils.h / pinned_buffer_pool.h pick the correct backend. # diff --git a/mooncake-transfer-engine/CMakeLists.txt b/mooncake-transfer-engine/CMakeLists.txt index 9f4583a5..c5ab0e3c 100644 --- a/mooncake-transfer-engine/CMakeLists.txt +++ b/mooncake-transfer-engine/CMakeLists.txt @@ -61,8 +61,9 @@ if (WITH_RUST_EXAMPLE) add_subdirectory(rust) endif() -# Non-peermem requires cuMem allocated memory pointers -if (USE_MNNVL OR NOT WITH_NVIDIA_PEERMEM) +# Non-peermem requires cuMem allocated memory pointers; since WITH_NVIDIA_PEERMEM +# is now a runtime environment variable, always build when GPU support is present. +if (USE_MNNVL OR USE_CUDA OR USE_HIP OR USE_MUSA OR USE_MACA) add_subdirectory(nvlink-allocator) endif() diff --git a/mooncake-transfer-engine/nvlink-allocator/CMakeLists.txt b/mooncake-transfer-engine/nvlink-allocator/CMakeLists.txt index 35dff93f..69e0d714 100644 --- a/mooncake-transfer-engine/nvlink-allocator/CMakeLists.txt +++ b/mooncake-transfer-engine/nvlink-allocator/CMakeLists.txt @@ -5,12 +5,15 @@ if(USE_HIP) list(APPEND _extra_build_opts --use-hipcc) elseif(USE_MUSA) list(APPEND _extra_build_opts --use-mcc) +elseif(USE_MACA) + list(APPEND _extra_build_opts --use-maca) endif() set(_enable_nvlink_allocator_build FALSE) if(USE_CUDA OR USE_HIP - OR USE_MUSA) + OR USE_MUSA + OR USE_MACA) set(_enable_nvlink_allocator_build TRUE) endif() diff --git a/mooncake-transfer-engine/nvlink-allocator/build.sh b/mooncake-transfer-engine/nvlink-allocator/build.sh index 6bd9ccfe..018e1133 100755 --- a/mooncake-transfer-engine/nvlink-allocator/build.sh +++ b/mooncake-transfer-engine/nvlink-allocator/build.sh @@ -8,6 +8,7 @@ source "$(dirname "$(readlink -f "$0")")/../scripts/allocator_build_common.sh" USE_NVCC=false USE_HIPCC=false USE_MCC=false +USE_MACA=false CI_BUILD=false if [[ "$1" == "--use-nvcc" ]]; then @@ -19,6 +20,9 @@ elif [[ "$1" == "--use-hipcc" ]]; then elif [[ "$1" == "--use-mcc" ]]; then USE_MCC=true shift +elif [[ "$1" == "--use-maca" ]]; then + USE_MACA=true + shift elif [[ "$1" == "--ci-build" ]]; then CI_BUILD=true shift @@ -45,6 +49,11 @@ elif [ "$USE_HIPCC" = true ]; then hipcc "$OUTPUT_DIR/nvlink_allocator.cpp" -o "$OUTPUT_DIR/nvlink_allocator.so" -shared -fPIC -lamdhip64 -I/opt/rocm/include ${INCLUDE_FLAGS} -DUSE_HIP=1 elif [ "$USE_MCC" = true ]; then mcc "$CPP_FILE" -o "$OUTPUT_DIR/nvlink_allocator.so" --shared -fPIC -lmusa -I/usr/local/musa/include ${INCLUDE_FLAGS} -DUSE_MUSA=1 +elif [ "$USE_MACA" = true ]; then + MACA_ROOT=${MACA_HOME:-/opt/maca} + g++ "$CPP_FILE" -o "$OUTPUT_DIR/nvlink_allocator.so" --shared -fPIC \ + -I"${MACA_ROOT}/include" ${INCLUDE_FLAGS} \ + -L"${MACA_ROOT}/lib64" -L"${MACA_ROOT}/lib" -lmcruntime -DUSE_MACA=1 else # Default g++ build g++ "$CPP_FILE" -o "$OUTPUT_DIR/nvlink_allocator.so" --shared -fPIC -lcuda -I/usr/local/cuda/include ${INCLUDE_FLAGS} -DUSE_CUDA=1 diff --git a/mooncake-transfer-engine/src/CMakeLists.txt b/mooncake-transfer-engine/src/CMakeLists.txt index bf7b1342..cb0c9979 100644 --- a/mooncake-transfer-engine/src/CMakeLists.txt +++ b/mooncake-transfer-engine/src/CMakeLists.txt @@ -48,6 +48,7 @@ target_link_libraries( PUBLIC base transport rdma_transport + mooncake_common ibverbs glog::glog gflags::gflags diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index c1006b9c..322a6786 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -29,6 +29,7 @@ #include "config.h" #include "cuda_alike.h" +#include "environ.h" #include "transport/rdma_transport/endpoint_store.h" #include "transport/rdma_transport/rdma_endpoint.h" #include "transport/rdma_transport/rdma_transport.h" @@ -229,8 +230,7 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, << "shrink it to " << globalConfig().max_mr_size; length = (size_t)globalConfig().max_mr_size; } -#if defined(USE_MLU) || defined(USE_MACA) || \ - (!defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA)) +#if defined(USE_MLU) || defined(USE_MACA) || defined(USE_CUDA) // Implement register memory in a way that does not assume the presence of // nvidia-peermem. If memory is on CPU call ibv_reg_mr() as usual. If memory // is on GPU then use ibv_reg_dmabuf_mr() instead which does not require @@ -243,6 +243,14 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, if (result != CUDA_SUCCESS || memType == CU_MEMORYTYPE_HOST) { mrMeta.addr = addr; mrMeta.mr = ibv_reg_mr(pd_, addr, length, access); +#if defined(USE_CUDA) + } else if (memType == CU_MEMORYTYPE_DEVICE && + Environ::Get().GetWithNvidiaPeermem()) { + // WITH_NVIDIA_PEERMEM env var is set: use ibv_reg_mr() directly for + // GPU memory (requires the nvidia-peermem kernel module to be loaded). + mrMeta.addr = addr; + mrMeta.mr = ibv_reg_mr(pd_, addr, length, access); +#endif } else if (memType == CU_MEMORYTYPE_DEVICE) { #if defined(USE_CUDA) // Ensure a CUDA context is current — worker threads or callers @@ -644,7 +652,7 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, return ERR_CONTEXT; } -#if defined(USE_MACA) || (!defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA)) +#if defined(USE_MACA) || defined(USE_CUDA) // Verify DMA-BUF support against the GPU device(s) that the local // topology explicitly maps to this RNIC, rather than assuming the // verbs enumeration order matches GPU enumeration. @@ -652,77 +660,84 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, // not just GPUs listing it as preferred. Runtime selection falls // back to avail_hca when a preferred NIC is disabled, so we must // validate both lists. - std::vector mapped_gpu_devices; - if (engine_.local_topology_) { - const auto topology_matrix = engine_.local_topology_->getMatrix(); - for (const auto &entry : topology_matrix) { - if (entry.first.rfind(GPU_PREFIX, 0) != 0) continue; - bool in_preferred = - std::find(entry.second.preferred_hca.begin(), - entry.second.preferred_hca.end(), - device_name) != entry.second.preferred_hca.end(); - bool in_avail = - std::find(entry.second.avail_hca.begin(), - entry.second.avail_hca.end(), - device_name) != entry.second.avail_hca.end(); - if (!in_preferred && !in_avail) continue; + if (!Environ::Get().GetWithNvidiaPeermem()) { + std::vector mapped_gpu_devices; + if (engine_.local_topology_) { + const auto topology_matrix = + engine_.local_topology_->getMatrix(); + for (const auto &entry : topology_matrix) { + if (entry.first.rfind(GPU_PREFIX, 0) != 0) continue; + bool in_preferred = + std::find(entry.second.preferred_hca.begin(), + entry.second.preferred_hca.end(), + device_name) != + entry.second.preferred_hca.end(); + bool in_avail = + std::find(entry.second.avail_hca.begin(), + entry.second.avail_hca.end(), + device_name) != entry.second.avail_hca.end(); + if (!in_preferred && !in_avail) continue; - try { - mapped_gpu_devices.push_back( - std::stoi(entry.first.substr(GPU_PREFIX.size()))); - } catch (const std::exception &e) { - LOG(WARNING) << "Ignore malformed topology GPU entry " - << entry.first << ": " << e.what(); + try { + mapped_gpu_devices.push_back( + std::stoi(entry.first.substr(GPU_PREFIX.size()))); + } catch (const std::exception &e) { + LOG(WARNING) << "Ignore malformed topology GPU entry " + << entry.first << ": " << e.what(); + } } } - } - std::sort(mapped_gpu_devices.begin(), mapped_gpu_devices.end()); - mapped_gpu_devices.erase( - std::unique(mapped_gpu_devices.begin(), mapped_gpu_devices.end()), - mapped_gpu_devices.end()); + std::sort(mapped_gpu_devices.begin(), mapped_gpu_devices.end()); + mapped_gpu_devices.erase(std::unique(mapped_gpu_devices.begin(), + mapped_gpu_devices.end()), + mapped_gpu_devices.end()); - if (mapped_gpu_devices.empty()) { - LOG(INFO) << "No GPU device is explicitly mapped to RNIC " - << device_name << "; skip DMA-BUF affinity validation"; - } else { - // cuInit is process-global and idempotent; call it once before - // the per-device loop, not per cuDeviceGet. + if (mapped_gpu_devices.empty()) { + LOG(INFO) << "No GPU device is explicitly mapped to RNIC " + << device_name + << "; skip DMA-BUF affinity validation"; + } else { + // cuInit is process-global and idempotent; call it once before + // the per-device loop, not per cuDeviceGet. #if defined(USE_CUDA) - CUresult result = cuInit(0); - if (result != CUDA_SUCCESS) { - LOG(ERROR) << "Failed to initialize CUDA driver for RNIC " - << device_name; - goto cleanup_context_and_devices; - } -#endif - for (int gpu_device : mapped_gpu_devices) { - CUdevice cuDevice; - CUresult result = cuDeviceGet(&cuDevice, gpu_device); + CUresult result = cuInit(0); if (result != CUDA_SUCCESS) { - LOG(ERROR) << "Failed to query GPU device " << gpu_device - << " for RNIC " << device_name; - goto cleanup_context_and_devices; - } - int dmaBufSupported; - result = cuDeviceGetAttribute( - &dmaBufSupported, CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED, - cuDevice); - if (result != CUDA_SUCCESS) { - LOG(ERROR) << "Failed to query GPU device attributes for " - << "GPU device " << gpu_device << " and RNIC " + LOG(ERROR) << "Failed to initialize CUDA driver for RNIC " << device_name; goto cleanup_context_and_devices; } - if (!dmaBufSupported) { - LOG(ERROR) - << "DMA BUF supported required for GPU RDMA without " - "nvidia-peermem on GPU device " - << gpu_device << " mapped to RNIC " << device_name; - goto cleanup_context_and_devices; +#endif + for (int gpu_device : mapped_gpu_devices) { + CUdevice cuDevice; + CUresult result = cuDeviceGet(&cuDevice, gpu_device); + if (result != CUDA_SUCCESS) { + LOG(ERROR) << "Failed to query GPU device " + << gpu_device << " for RNIC " << device_name; + goto cleanup_context_and_devices; + } + int dmaBufSupported; + result = cuDeviceGetAttribute( + &dmaBufSupported, CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED, + cuDevice); + if (result != CUDA_SUCCESS) { + LOG(ERROR) + << "Failed to query GPU device attributes for " + << "GPU device " << gpu_device << " and RNIC " + << device_name; + goto cleanup_context_and_devices; + } + if (!dmaBufSupported) { + LOG(ERROR) + << "DMA BUF supported required for GPU RDMA " + "without " + "nvidia-peermem on GPU device " + << gpu_device << " mapped to RNIC " << device_name; + goto cleanup_context_and_devices; + } } } - } + } // !Environ::Get().GetWithNvidiaPeermem() #endif ibv_port_attr port_attr; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp index ac49970a..2c20664c 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp @@ -29,6 +29,7 @@ #include "common.h" #include "config.h" +#include "environ.h" #include "memory_location.h" #include "topology.h" #include "transport/rdma_transport/rdma_context.h" @@ -383,33 +384,40 @@ int RdmaTransport::allocateLocalSegmentID() { int RdmaTransport::registerLocalMemoryBatch( const std::vector &buffer_list, const std::string &location) { -#if !defined(WITH_NVIDIA_PEERMEM) && defined(USE_CUDA) - for (auto &buffer : buffer_list) { - int ret = registerLocalMemory(buffer.addr, buffer.length, location, - true, false); - if (ret) { - LOG(WARNING) << "RdmaTransport: Failed to register memory: addr " - << buffer.addr << " length " << buffer.length; +#if defined(USE_CUDA) + if (!Environ::Get().GetWithNvidiaPeermem()) { + for (auto &buffer : buffer_list) { + int ret = registerLocalMemory(buffer.addr, buffer.length, location, + true, false); + if (ret) { + LOG(WARNING) + << "RdmaTransport: Failed to register memory: addr " + << buffer.addr << " length " << buffer.length; + } + } + } else { +#endif + std::vector> results; + for (auto &buffer : buffer_list) { + results.emplace_back(std::async( + std::launch::async, [this, buffer, location]() -> int { + // Use force_sequential=true to avoid nested parallelism + return registerLocalMemoryInternal(buffer.addr, + buffer.length, location, + true, false, true); + })); } - } -#else - std::vector> results; - for (auto &buffer : buffer_list) { - results.emplace_back( - std::async(std::launch::async, [this, buffer, location]() -> int { - // Use force_sequential=true to avoid nested parallelism - return registerLocalMemoryInternal(buffer.addr, buffer.length, - location, true, false, true); - })); - } - for (size_t i = 0; i < buffer_list.size(); ++i) { - if (results[i].get()) { - LOG(WARNING) << "RdmaTransport: Failed to register memory: addr " - << buffer_list[i].addr << " length " - << buffer_list[i].length; + for (size_t i = 0; i < buffer_list.size(); ++i) { + if (results[i].get()) { + LOG(WARNING) + << "RdmaTransport: Failed to register memory: addr " + << buffer_list[i].addr << " length " + << buffer_list[i].length; + } } - } +#if defined(USE_CUDA) + } // Environ::Get().GetWithNvidiaPeermem() #endif return metadata_->updateLocalSegmentDesc(); -- 2.34.1 From 5689013b52e1618e8de2ccf218615b4779fb18a2 Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Wed, 20 May 2026 11:23:22 +0800 Subject: [PATCH 099/382] [TENT] Proxy manager bugfixes (#2091) * [TENT] Optimize ProxyManager staging performance - Add comprehensive performance monitoring (ProxyManagerMetrics) - Optimize staging buffer configuration (4MB chunks, 64 count) - Implement intelligent retry mechanism for remote staging failures - Fix event queue handling consistency in INFLIGHT_REMOTE state - Add performance metrics: throughput, latency, retry counts, parallelism - Improve error handling with configurable retry logic (default: 3 attempts) Performance improvements: - Better pipeline parallelism with 4-buffer circulation - Reduced latency through async remote staging operations - Enhanced reliability through smart retry mechanism - Improved observability through detailed metrics collection Co-Authored-By: Claude Sonnet 4.6 (1M context) * Reformat * Fix build issue * fix issues * reformat * trigger ci * Revise prosy manager to fix potential bugs * Fix GPU check to support staging * Fix state machine --------- Co-authored-by: Claude Sonnet 4.6 (1M context) --- .../tent/include/tent/runtime/proxy_manager.h | 4 ++-- .../tent/src/runtime/proxy_manager.cpp | 18 ++++++++++++++++-- .../src/transport/bufio/bufio_transport.cpp | 2 +- .../transport/io_uring/io_uring_transport.cpp | 2 +- .../src/transport/mnnvl/mnnvl_transport.cpp | 2 +- .../tent/src/transport/tcp/tcp_transport.cpp | 3 +-- 6 files changed, 22 insertions(+), 9 deletions(-) diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/proxy_manager.h b/mooncake-transfer-engine/tent/include/tent/runtime/proxy_manager.h index 7f4e47d8..0620810d 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/proxy_manager.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/proxy_manager.h @@ -44,8 +44,8 @@ class ProxyManager { public: explicit ProxyManager(TransferEngineImpl* impl, - size_t chunk_size = 8 * 1024 * 1024, - size_t chunk_count = 32); + size_t chunk_size = 4 * 1024 * 1024, + size_t chunk_count = 64); ~ProxyManager(); diff --git a/mooncake-transfer-engine/tent/src/runtime/proxy_manager.cpp b/mooncake-transfer-engine/tent/src/runtime/proxy_manager.cpp index 559018e7..4ea80bd3 100644 --- a/mooncake-transfer-engine/tent/src/runtime/proxy_manager.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/proxy_manager.cpp @@ -238,7 +238,8 @@ Status ProxyManager::transferEventLoop(StagingTask& task, auto server_addr = task.params[0]; bool local_staging = !task.params[1].empty(); bool remote_staging = !task.params[2].empty(); - const static size_t kStageBuffers = 4; + const size_t kStageBuffers = + std::min(chunk_count_, static_cast(16)); uint64_t local_stage_buffer[kStageBuffers], remote_stage_buffer[kStageBuffers]; if (local_staging) { @@ -318,6 +319,7 @@ Status ProxyManager::transferEventLoop(StagingTask& task, chunk.length, chunk.offset); chunk.prev_state = chunk.state; chunk.state = StageState::INFLIGHT; + event_queue.push(id); } else if (request.opcode == Request::READ && remote_staging) { if (remote_locked.count(chunk.remote_buf)) { event_queue.push(id); @@ -329,10 +331,11 @@ Status ProxyManager::transferEventLoop(StagingTask& task, remote_futures[id]); chunk.prev_state = chunk.state; chunk.state = StageState::INFLIGHT_REMOTE; + event_queue.push(id); } else { chunk.state = StageState::CROSS; + event_queue.push(id); } - event_queue.push(id); break; } @@ -366,12 +369,16 @@ Status ProxyManager::transferEventLoop(StagingTask& task, remote_futures[id]); chunk.prev_state = chunk.state; chunk.state = StageState::INFLIGHT_REMOTE; + event_queue.push(id); } else if (request.opcode == Request::READ && local_staging) { chunk.batch = submitLocalStage(request, chunk.local_buf, chunk.length, chunk.offset); chunk.prev_state = chunk.state; chunk.state = StageState::INFLIGHT; event_queue.push(id); + } else { + // No staging needed, mark as finished + chunk.state = StageState::FINISH; } break; } @@ -416,6 +423,10 @@ Status ProxyManager::transferEventLoop(StagingTask& task, } case StageState::FAILED: { + // Drain the queue to avoid losing chunks + while (!event_queue.empty()) { + event_queue.pop(); + } return Status::InternalError( "Proxy event loop in failed state"); } @@ -424,6 +435,7 @@ Status ProxyManager::transferEventLoop(StagingTask& task, auto& fut = remote_futures[id]; if (!fut.valid()) { chunk.state = StageState::FAILED; + event_queue.push(id); break; } if (fut.wait_for(std::chrono::seconds(0)) == @@ -431,6 +443,7 @@ Status ProxyManager::transferEventLoop(StagingTask& task, Status rs = fut.get(); if (!rs.ok()) { chunk.state = StageState::FAILED; + event_queue.push(id); break; } if (chunk.prev_state == StageState::PRE) { @@ -442,6 +455,7 @@ Status ProxyManager::transferEventLoop(StagingTask& task, remote_staging) { remote_locked.erase(chunk.remote_buf); } + event_queue.push(id); } } else { event_queue.push(id); diff --git a/mooncake-transfer-engine/tent/src/transport/bufio/bufio_transport.cpp b/mooncake-transfer-engine/tent/src/transport/bufio/bufio_transport.cpp index 5bd76f07..6b6ea252 100644 --- a/mooncake-transfer-engine/tent/src/transport/bufio/bufio_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/bufio/bufio_transport.cpp @@ -80,7 +80,7 @@ Status BufIoTransport::install(std::string& local_segment_name, installed_ = true; caps.dram_to_file = true; - if (Platform::getLoader().type() == "cuda") { + if (Platform::getLoader().type() != "cpu") { caps.gpu_to_file = true; } diff --git a/mooncake-transfer-engine/tent/src/transport/io_uring/io_uring_transport.cpp b/mooncake-transfer-engine/tent/src/transport/io_uring/io_uring_transport.cpp index b594cc38..494d34bd 100644 --- a/mooncake-transfer-engine/tent/src/transport/io_uring/io_uring_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/io_uring/io_uring_transport.cpp @@ -87,7 +87,7 @@ Status IOUringTransport::install(std::string& local_segment_name, async_memcpy_threshold_ = conf_->get("transports/nvlink/async_memcpy_threshold", 1024) * 1024; caps.dram_to_file = true; - if (Platform::getLoader().type() == "cuda") { + if (Platform::getLoader().type() != "cpu") { caps.gpu_to_file = true; } return Status::OK(); diff --git a/mooncake-transfer-engine/tent/src/transport/mnnvl/mnnvl_transport.cpp b/mooncake-transfer-engine/tent/src/transport/mnnvl/mnnvl_transport.cpp index e53df7de..f8adcd77 100644 --- a/mooncake-transfer-engine/tent/src/transport/mnnvl/mnnvl_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/mnnvl/mnnvl_transport.cpp @@ -114,7 +114,7 @@ Status MnnvlTransport::install(std::string &local_segment_name, 128; caps.dram_to_gpu = true; - if (Platform::getLoader().type() == "cuda") caps.gpu_to_gpu = true; + if (Platform::getLoader().type() != "cpu") caps.gpu_to_gpu = true; installed_ = true; supported_ = true; diff --git a/mooncake-transfer-engine/tent/src/transport/tcp/tcp_transport.cpp b/mooncake-transfer-engine/tent/src/transport/tcp/tcp_transport.cpp index 10465015..4cc36f53 100644 --- a/mooncake-transfer-engine/tent/src/transport/tcp/tcp_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/tcp/tcp_transport.cpp @@ -95,8 +95,7 @@ Status TcpTransport::install(std::string &local_segment_name, return 0; }); caps.dram_to_dram = true; - if (Platform::getLoader().type() == "cuda" || - Platform::getLoader().type() == "cann") { + if (Platform::getLoader().type() != "cpu") { caps.dram_to_gpu = true; caps.gpu_to_dram = true; caps.gpu_to_gpu = true; -- 2.34.1 From eaf724ab6edf464531d29238a6fc17dba0532acf Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Wed, 20 May 2026 11:49:57 +0800 Subject: [PATCH 100/382] [Store] Fix std::terminate crash on mooncake client shutdown (Issue#2115) (#2125) * fix(store): join dummy_client_monitor_thread on shutdown to prevent std::terminate The dummy_client_monitor_thread_ (std::thread) was started in start_dummy_client_monitor() but never joined or detached. On ~RealClient(), the still-joinable std::thread triggers std::terminate(), crashing standalone mooncake_client on every clean shutdown. Add stop_dummy_client_monitor() following the same pattern as stop_ipc_server(): set the running flag to false, then join. The call is placed in tearDownAll_internal() before stop_http_server() and before the dummy_client_mutex_ lock, avoiding the early-return skip and the lock-then-join deadlock. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- mooncake-store/include/real_client.h | 1 + mooncake-store/src/real_client.cpp | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index 31948e6b..c64d67da 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -833,6 +833,7 @@ class RealClient : public PyClient { // Dummy Client manage related members void dummy_client_monitor_func(); int start_dummy_client_monitor(); + void stop_dummy_client_monitor(); std::thread dummy_client_monitor_thread_; std::atomic dummy_client_monitor_running_{false}; static constexpr uint64_t kDummyClientMonitorSleepMs = diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index b7711988..4d7d3df3 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -1057,6 +1057,7 @@ tl::expected RealClient::tearDownAll_internal() { } stop_ipc_server(); + stop_dummy_client_monitor(); stop_http_server(); if (!client_) { @@ -5145,6 +5146,13 @@ int RealClient::start_dummy_client_monitor() { return 0; } +void RealClient::stop_dummy_client_monitor() { + dummy_client_monitor_running_ = false; + if (dummy_client_monitor_thread_.joinable()) { + dummy_client_monitor_thread_.join(); + LOG(INFO) << "dummy_client_monitor_thread stopped"; + } +} int RealClient::start_ipc_server() { ipc_running_ = true; ipc_thread_ = std::jthread(&RealClient::ipc_server_func, this); -- 2.34.1 From a9e0aad723f1172ec26e3c4947bdb9cc051ef546 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E9=B9=A4=E7=94=B7?= Date: Thu, 21 May 2026 00:39:19 +0800 Subject: [PATCH 101/382] [Doc] Update EFA transport doc: SGLang section + vllm-router + p5.48xlarge bench (#2154) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [Doc] Update p6-b300 EFA throughput numbers post-#1944 Re-measured on a fresh main build between two p6-b300.48xlarge instances. GPU-to-GPU peak 780 GB/s write (~97.5% of 800 Gbps line rate, up from 752); CPU-to-CPU peak 283 GB/s write / 270 GB/s read (up from 230/180). Drops the "predates the refactor" caveat. Co-Authored-By: Claude Opus 4.7 * [Doc] Update EFA transport doc: SGLang section + vllm-router + p5.48xlarge bench - Rewrite "Usage with SGLang" to use sglang PR #25083 (MOONCAKE_PROTOCOL=efa flows natively) and mirror the vLLM section structure (Prefill / Decode / Router subsections). Drop unrelated GLOO_SOCKET_IFNAME and NVSHMEM bits. Note the cross-host router trap: PREFILL_HOST must be reachable from decode. - Add Router subsection to "Usage with vLLM" using vllm-router with --kv-connector mooncake. - Drop the SGLang Docker subsection (redundant). - Add benchmark section "4. p5.48xlarge (H100, 32 EFA × 100 Gbps)" with GPU-to-GPU and CPU-to-CPU sweeps. Peak GPU 389 GB/s write / 382 GB/s read (~97% of 400 GB/s line rate). CPU plateaus at ~64 GB/s, bounded by DDR4-3200 on EPYC 7R13. - Update Tuning Tips for the 32-NIC WR cap (8192 vs 4096 on 16-NIC hosts) and the read-batch difference between p5en and p5. Co-Authored-By: Claude Opus 4.7 --------- Co-authored-by: Claude Opus 4.7 --- .../design/transfer-engine/efa_transport.md | 142 ++++++++++++++---- 1 file changed, 109 insertions(+), 33 deletions(-) diff --git a/docs/source/design/transfer-engine/efa_transport.md b/docs/source/design/transfer-engine/efa_transport.md index d8c7a23a..05fcbfdc 100644 --- a/docs/source/design/transfer-engine/efa_transport.md +++ b/docs/source/design/transfer-engine/efa_transport.md @@ -215,28 +215,28 @@ address shown in the target's startup log (e.g., `ip-172-31-29-226:12345`). #### 1. p6-b300.48xlarge (B300, 16 EFA × 400 Gbps) -Tested on two p6-b300.48xlarge instances (Intel Xeon Platinum 8559C, 8× B300, 16 EFA devices) in the same AWS placement group. - -> **Note:** numbers below predate the SRD shared-endpoint refactor (#1944) and current EFA tuning work. They are a lower bound for the current code; we will re-sweep and update when the hardware is available again. +Tested on two p6-b300.48xlarge instances (Intel Xeon Platinum 8559C, 8× B300, 16 EFA devices) in the same AWS placement group. Numbers below are post-SRD-shared-endpoint (#1944) on a fresh `main` build with the DLAMI pytorch env (CUDA 13). **GPU-to-GPU** (build with `-DUSE_CUDA=ON`, `--gpu_id=-1` for all 8 GPUs, `--buffer_size=2147483648`): | Configuration | Write | Read | |---------------|-------|------| -| block=1MB, threads=16, batch=128 | 701 GB/s | **697 GB/s** | -| **block=1MB, threads=32, batch=64** | **752 GB/s** | 713 GB/s | -| block=1MB, threads=32, batch=32 | 751 GB/s | - | -| block=1MB, threads=64, batch=32 | 728 GB/s | - | +| block=1MB, threads=16, batch=128 | 758.88 GB/s | 720.35 GB/s | +| block=1MB, threads=32, batch=64 | 753.32 GB/s | **755.78 GB/s** | +| **block=1MB, threads=32, batch=32** | **780.33 GB/s** | - | +| block=1MB, threads=64, batch=32 | 780.23 GB/s | - | -> **Peak: 752 GB/s write**, reaching ~94% of the 800 GB/s theoretical line rate (16×400 Gbps). GPUDirect RDMA bypasses DRAM entirely (HBM3e → PCIe switch → NIC), so performance is not bottlenecked by CPU memory bandwidth. +> **Peak: 780 GB/s write**, reaching ~97.5% of the 800 GB/s theoretical line rate (16×400 Gbps). GPUDirect RDMA bypasses DRAM entirely (HBM3e → PCIe switch → NIC), so performance is not bottlenecked by CPU memory bandwidth. -**CPU-to-CPU** (build with `-DUSE_CUDA=OFF`): +**CPU-to-CPU** (build with `-DUSE_CUDA=OFF`, or `--use_vram=false` on a CUDA build, `--buffer_size=4294967296`): | Configuration | Write | Read | |---------------|-------|------| -| **block=1MB, threads=32, batch=128, buf=4GB** | **230 GB/s** | 180 GB/s | +| **block=1MB, threads=32, batch=128** | **282.93 GB/s** | **270.47 GB/s** | +| block=1MB, threads=16, batch=128 | 282.04 GB/s | 249.67 GB/s | +| block=1MB, threads=32, batch=64 | 282.84 GB/s | 256.26 GB/s | -> CPU-to-CPU is bounded by DRAM bandwidth (~250 GB/s/socket on Xeon 8559C). Per-NIC sampling shows NUMA-0 NICs at 90 Gbps and NUMA-1 NICs at 53 Gbps, confirming DRAM controller saturation rather than NIC limit. +> CPU-to-CPU is bounded by DRAM bandwidth on the Xeon 8559C — write throughput is essentially flat across thread/batch combinations (~282 GB/s), confirming DRAM controller saturation rather than a NIC or in-flight-WR limit. #### 2. p6-b200.48xlarge (B200, 8 EFA × 400 Gbps) @@ -311,11 +311,43 @@ Tested on two p5en.48xlarge instances (Intel Xeon 8488C, 8× H200 141GB, 16 EFA > `buffer_size` only needs to satisfy `buffer_size ≥ block_size × batch_size × threads` (the bench auto-adjusts if smaller, but silently). Anything larger than that minimum does not change throughput — 2 GB vs 4 GB differs by ~3% on write, read is flat within noise. The example commands use 4 GB because it is safe for any reasonable threads/batch combination without having to recompute the minimum. +#### 4. p5.48xlarge (H100, 32 EFA × 100 Gbps) + +Tested on two p5.48xlarge instances (AMD EPYC 7R13, 8× H100 80GB, 32 EFA devices) in the same AWS placement group. Per-NIC line rate is half of p5en's, but with twice the NIC count the aggregate ceiling is the same 400 GB/s. The shared-endpoint WR cap scales with NIC count: `32 NICs × 256 = 8192` in-flight slots, so `threads × batch_size ≤ 8192` (vs 4096 on p5en). + +**GPU-to-GPU** (build with `-DUSE_CUDA=ON`, `--gpu_id=-1` for all 8 GPUs, `--buffer_size=4294967296`): + +| Configuration | Write | Read | +|---------------|-------|------| +| block=1MB, threads=8, batch=128 | 335.11 GB/s | - | +| block=1MB, threads=16, batch=128 | 388.52 GB/s | 379.10 GB/s | +| block=1MB, threads=32, batch=64 | 388.83 GB/s | 379.78 GB/s | +| **block=1MB, threads=32, batch=128** | **388.90 GB/s** | **381.64 GB/s** | +| block=1MB, threads=16, batch=32 | - | 356.94 GB/s | +| block=1MB, threads=32, batch=32 | - | 380.66 GB/s | + +> **Peak write: 389 GB/s** at `threads=32, batch=128` — ~97% of the 400 GB/s theoretical line rate (32×100 Gbps). The plateau is wide: any `(threads, batch)` between `(16, 128)` and `(32, 128)` lands within 0.1% of peak. **Peak read: 382 GB/s** at `threads=32, batch=128` — unlike p5en, reads on this host scale with batch size up to 128 because the wider 32-NIC fabric absorbs larger in-flight queues without backoff. `(32, 256)` and `(64, 128)` (both at the 8192 WR cap) fail with no headroom for retries. + +**CPU-to-CPU** (build with `-DUSE_CUDA=OFF`, or `--use_vram=false` on a CUDA build, `--buffer_size=4294967296`): + +| Configuration | Write | Read | +|---------------|-------|------| +| block=1MB, threads=16, batch=128 | 39.83 GB/s | 40.43 GB/s | +| block=1MB, threads=32, batch=64 | 47.31 GB/s | 48.06 GB/s | +| block=1MB, threads=32, batch=128 | 47.75 GB/s | 48.62 GB/s | +| block=1MB, threads=32, batch=32 | 55.66 GB/s | 56.68 GB/s | +| **block=1MB, threads=48, batch=16** | **63.60 GB/s** | 63.00 GB/s | +| **block=1MB, threads=64, batch=16** | 63.39 GB/s | 63.05 GB/s | +| block=1MB, threads=32, batch=16 | 63.21 GB/s | 60.82 GB/s | +| block=1MB, threads=96, batch=32 | 57.61 GB/s | 59.63 GB/s | + +> **Peak: ~64 GB/s** on both write and read — far below the GPU-to-GPU number despite identical NIC count. The bottleneck is DDR4-3200 DRAM bandwidth on the EPYC 7R13 (Milan): `batch=16` consistently wins because larger in-flight queues only deepen DRAM contention without unlocking new NIC capacity. p5.48xlarge CPU-to-CPU runs around **3× slower than p5en** (DDR5 Xeon 8488C, ~213 GB/s) at the same NIC aggregate. For PD KV transfer, the GPU-to-GPU path is the relevant one. + ### Tuning Tips - **Use `--block_size=1048576` (1MB)** — the single most important knob. The 64 KB default reaches only ~26% of peak. 1 MB is within a few percent of the 2 MB plateau while leaving headroom for `batch_size` under the shared-endpoint WR cap. -- **Keep `threads × batch_size ≤ num_nics × max_wr`** — under the SRD shared endpoint each NIC carries one `fid_ep` with a 256 WR cap (`MC_MAX_WR`), giving `16 NICs × 256 = 4096` in-flight slots on a 16-NIC host. Exceeding this trips "timed out waiting for CQ drain". In practice `threads=16, batch=128` is a solid baseline; going higher rarely adds throughput and routinely hits the cap. -- **Write vs read:** write benefits from larger batches (peak at `batch=128`); read prefers smaller in-flight queues (peak at `batch=32` on p5en). +- **Keep `threads × batch_size ≤ num_nics × max_wr`** — under the SRD shared endpoint each NIC carries one `fid_ep` with a 256 WR cap (`MC_MAX_WR`), giving `16 NICs × 256 = 4096` in-flight slots on a 16-NIC host (b300, b200, p5en) and `32 NICs × 256 = 8192` on p5. Exceeding this trips "timed out waiting for CQ drain". `threads=16, batch=128` is a solid baseline on 16-NIC hosts; on 32-NIC p5, `threads=32, batch=128` works the same way. +- **Write vs read:** write benefits from larger batches (peak at `batch=128`); on 16-NIC p5en read prefers smaller queues (peak at `batch=32`), but on 32-NIC p5 reads scale up to `batch=128` because the wider fabric absorbs larger in-flight queues. - For **GPU-to-GPU**: pass `--gpu_id=-1` on **both** sides so buffers fan out across every GPU. Pinning a single GPU halves throughput because half the NICs end up cross-NUMA. - For **CPU-to-CPU**: DRAM bandwidth is the ceiling. NUMA-split (separate initiator/target instances per NUMA node) can help reduce contention when one instance can't saturate both nodes. - `--buffer_size` only needs `≥ block × batch × threads`; larger @@ -425,52 +457,96 @@ vllm serve -tp 8 \ --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_consumer","kv_connector_extra_config":{"mooncake_protocol":"efa"}}' ``` +### 3. Router + +Front the prefill / decode pair with `vllm-router`. **`PREFILL_HOST` / `DECODE_HOST` must be each node's reachable private IP, not `127.0.0.1`** — the router forwards these addresses to the peer for the KV handshake; localhost will fail with `Connection refused` and stall traffic. + +```bash +vllm-router --policy round_robin \ + --vllm-pd-disaggregation \ + --prefill http://:8010 \ + --decode http://:8020 \ + --kv-connector mooncake \ + --host 0.0.0.0 --port 30000 \ + --intra-node-data-parallel-size 8 +``` + +`--intra-node-data-parallel-size` should match the per-node DP size of your prefill / decode instances (8 in this example, matching `-tp 8`). + ## Usage with SGLang -SGLang's Mooncake integration currently hardcodes the `"rdma"` protocol. To use EFA transport, apply the provided patch and set environment variables. +SGLang's PD-disaggregation Mooncake integration reads the transport from `MOONCAKE_PROTOCOL`. Set it to `efa` and select the Mooncake backend with `--disaggregation-transfer-backend mooncake`. -### 1. Apply EFA Patch +### 1. Apply EFA Patch (only if SGLang version predates PR #25083) -SGLang's transfer engine initialization needs to be patched to read the protocol from an environment variable instead of using hardcoded `"rdma"`. Use the [patch script](https://github.com/whn09/kimi-k2-sglang): +Older SGLang releases hardcode `"rdma"` in the transfer engine init. Since [SGLang PR #25083](https://github.com/sgl-project/sglang/pull/25083) the protocol is read from `MOONCAKE_PROTOCOL`, so once that PR is in your build (or upstream `main`) **this step is unnecessary** — skip to step 2. + +If you are pinned to an older release, apply the [patch script](https://github.com/whn09/kimi-k2-sglang): ```bash bash patch_sglang_efa.sh ``` -This is idempotent and safe to rerun. +The script is idempotent and safe to rerun. ### 2. Environment Variables +Only one Mooncake-specific env is required: + ```bash export MOONCAKE_PROTOCOL=efa +``` + +If your container does not already export libfabric/EFA paths in its `Dockerfile`, also set: + +```bash export FI_PROVIDER=efa export FI_EFA_USE_DEVICE_RDMA=1 -export GLOO_SOCKET_IFNAME=enp71s0 # adjust to your instance's primary interface +export LD_LIBRARY_PATH=/opt/amazon/efa/lib:$LD_LIBRARY_PATH ``` -For multi-node expert parallelism (EP) deployments, also set: +> **Note on additional `MC_*` knobs:** `MC_NUM_CQ_PER_CTX`, `MC_MAX_WR`, `MC_MAX_CQE_PER_CTX`, `MC_SLICE_SIZE`, and `MC_EFA_STRIPING_THRESHOLD` are **not** required at typical PD-disagg loads — the SRD shared-endpoint refactor (#1944) makes them redundant up to high concurrency on 1k/1k traffic. Treat them as emergency switches for CQ-overflow or long-running drift symptoms. + +### 3. Prefill Instance ```bash -export NVSHMEM_REMOTE_TRANSPORT=libfabric -export NVSHMEM_LIBFABRIC_PROVIDER=efa +MOONCAKE_PROTOCOL=efa \ +sglang serve \ + --trust-remote-code \ + --tp 8 --dp 2 --enable-dp-attention --enable-dp-lm-head \ + --host 0.0.0.0 --port 8010 \ + --disaggregation-mode prefill \ + --disaggregation-transfer-backend mooncake \ + --disaggregation-bootstrap-port 8998 ``` -> **Warning:** Do **not** set NVSHMEM variables on single-node deployments — doing so causes segmentation faults. - -### 3. Docker Launch Example +### 4. Decode Instance ```bash -docker run -d --name sglang \ - --runtime=nvidia --gpus all --network host \ - --privileged --shm-size=600g \ - --device=/dev/infiniband \ - -e MOONCAKE_PROTOCOL=efa \ - -e FI_PROVIDER=efa \ - -e FI_EFA_USE_DEVICE_RDMA=1 \ - bash start.sh +MOONCAKE_PROTOCOL=efa \ +sglang serve \ + --trust-remote-code \ + --tp 8 --dp 2 --enable-dp-attention --enable-dp-lm-head \ + --host 0.0.0.0 --port 8020 \ + --disaggregation-mode decode \ + --disaggregation-transfer-backend mooncake \ + --disaggregation-bootstrap-port 8998 ``` -> **Note:** Ensure the Docker image's libfabric version matches the host's EFA driver. If not, mount the host's EFA libraries into the container (see [Troubleshooting](#libfabric-version-mismatch-in-docker)). +### 5. Router + +Front the pair with `sglang_router` from the prefill host. **`PREFILL_HOST` must be the prefill node's reachable IP, not `127.0.0.1`** — the router forwards this address to the decode node for the bootstrap_room handshake; localhost will fail with `Connection refused` and stall traffic at 0/N. + +```bash +python3 -m sglang_router.launch_router \ + --pd-disaggregation \ + --prefill "http://:8010" 8998 \ + --decode "http://:8020" \ + --policy round_robin \ + --host 0.0.0.0 --port 8000 +``` + +The trailing `8998` after `--prefill` must match the prefill's `--disaggregation-bootstrap-port`. ## Technical Details -- 2.34.1 From 076e3eaa5bf1b20af5ee8819b497c122c4df4569 Mon Sep 17 00:00:00 2001 From: zhuwentao <62613699+zwtao40@users.noreply.github.com> Date: Thu, 21 May 2026 11:20:52 +0800 Subject: [PATCH 102/382] feat(store): add NoF segment metadata management (#2143) Co-authored-by: Enigmo-x --- CMakeLists.txt | 7 + mooncake-store/include/allocation_strategy.h | 32 +- mooncake-store/include/allocator.h | 19 +- mooncake-store/include/master_client.h | 53 +- mooncake-store/include/master_config.h | 114 +++ .../include/master_metric_manager.h | 91 ++ mooncake-store/include/master_service.h | 138 +++- mooncake-store/include/replica.h | 138 +++- mooncake-store/include/rpc_service.h | 34 +- mooncake-store/include/rpc_types.h | 11 + mooncake-store/include/segment.h | 176 ++++ mooncake-store/include/types.h | 19 + mooncake-store/src/allocator.cpp | 64 +- mooncake-store/src/master.cpp | 85 +- mooncake-store/src/master_client.cpp | 92 ++- mooncake-store/src/master_metric_manager.cpp | 282 +++++++ mooncake-store/src/master_service.cpp | 777 +++++++++++++++++- mooncake-store/src/rpc_service.cpp | 94 ++- mooncake-store/src/segment.cpp | 266 ++++++ mooncake-store/tests/master_service_test.cpp | 125 ++- 20 files changed, 2506 insertions(+), 111 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index c6dd0699..56e4c2de 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -19,6 +19,7 @@ option(WITH_P2P_STORE "build p2p store library and sample code" OFF) option(WITH_RUST_EXAMPLE "build the Rust interface and sample code for the transfer engine" OFF) option(WITH_STORE_RUST "build the Rust bindings for the Mooncake Store" ON) option(WITH_EP "build mooncake with expert parallelism support" OFF) +option(USE_NOF "build mooncake store with NoF SSD pool support" OFF) include(${CMAKE_CURRENT_SOURCE_DIR}/mooncake-common/SetupPython.cmake) add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/extern/pybind11) @@ -56,6 +57,12 @@ if (STORE_USE_K8S_LEASE) add_compile_definitions(STORE_USE_K8S_LEASE) endif() +if (USE_NOF) + add_compile_definitions(USE_NOF) +else() + message(STATUS "USE_NOF=OFF, NoF SSD pool support is disabled") +endif() + option(STORE_USE_JEMALLOC "Use jemalloc in mooncake store master" OFF) # Define ASIO macros before building targets that include ASIO headers. diff --git a/mooncake-store/include/allocation_strategy.h b/mooncake-store/include/allocation_strategy.h index 18d7f465..e9f65612 100644 --- a/mooncake-store/include/allocation_strategy.h +++ b/mooncake-store/include/allocation_strategy.h @@ -163,7 +163,8 @@ class AllocationStrategy { const std::vector& preferred_segments = std::vector(), const std::set& excluded_segments = - std::set()) = 0; + std::set(), + const ReplicaType replica_type = ReplicaType::MEMORY) = 0; /** * @brief Allocate one replica from the specified segment. @@ -208,7 +209,8 @@ class RandomAllocationStrategy : public AllocationStrategy { const std::vector& preferred_segments = std::vector(), const std::set& excluded_segments = - std::set()) { + std::set(), + const ReplicaType replica_type = ReplicaType::MEMORY) override { // Validate input parameters if (slice_length == 0 || replica_num == 0) { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); @@ -236,7 +238,7 @@ class RandomAllocationStrategy : public AllocationStrategy { slice_length, generator); if (buffer) { replicas.emplace_back(std::move(buffer), - ReplicaStatus::PROCESSING); + ReplicaStatus::PROCESSING, replica_type); return replicas; } return tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); @@ -256,7 +258,7 @@ class RandomAllocationStrategy : public AllocationStrategy { slice_length, generator); if (buffer) { replicas.emplace_back(std::move(buffer), - ReplicaStatus::PROCESSING); + ReplicaStatus::PROCESSING, replica_type); if (replicas.size() == replica_num) { return replicas; } @@ -289,7 +291,7 @@ class RandomAllocationStrategy : public AllocationStrategy { slice_length, generator); if (buffer) { replicas.emplace_back(std::move(buffer), - ReplicaStatus::PROCESSING); + ReplicaStatus::PROCESSING, replica_type); // Nit: no need to insert names[index] into used_segments here // because we only traverse all names once, thus there is no // chance to try allocating from a segment for the second time. @@ -345,8 +347,9 @@ class RandomAllocationStrategy : public AllocationStrategy { // Randomly select a start point to distribute // allocations across all segments std::uniform_int_distribution dist(0, num_segs - 1); - size_t seg_offset = dist(generator); - for (size_t i = 0; i < num_segs; i++) { + size_t seg_offset = + dist(generator); // select a start segment to place replica + for (size_t i = 0; i < num_segs; i++) { // only allocate one replica auto& allocator = (*allocators)[(i + seg_offset) % num_segs]; if (auto buffer = allocator->allocate(slice_length)) { return buffer; @@ -386,7 +389,8 @@ class FreeRatioFirstAllocationStrategy : public RandomAllocationStrategy { const std::vector& preferred_segments = std::vector(), const std::set& excluded_segments = - std::set()) override { + std::set(), + const ReplicaType replica_type = ReplicaType::MEMORY) override { if (slice_length == 0 || replica_num == 0) { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } @@ -413,7 +417,7 @@ class FreeRatioFirstAllocationStrategy : public RandomAllocationStrategy { slice_length, generator); if (buffer) { replicas.emplace_back(std::move(buffer), - ReplicaStatus::PROCESSING); + ReplicaStatus::PROCESSING, replica_type); used_segments.insert(preferred_segment); if (replicas.size() == replica_num) { return replicas; @@ -469,7 +473,7 @@ class FreeRatioFirstAllocationStrategy : public RandomAllocationStrategy { generator); if (buffer) { replicas.emplace_back(std::move(buffer), - ReplicaStatus::PROCESSING); + ReplicaStatus::PROCESSING, replica_type); used_segments.insert(name); } } @@ -499,7 +503,7 @@ class FreeRatioFirstAllocationStrategy : public RandomAllocationStrategy { slice_length, generator); if (buffer) { replicas.emplace_back(std::move(buffer), - ReplicaStatus::PROCESSING); + ReplicaStatus::PROCESSING, replica_type); used_segments.insert(names[index]); } } @@ -543,7 +547,8 @@ class CxlAllocationStrategy : public AllocationStrategy { const std::vector& preferred_segments = std::vector(), const std::set& excluded_segments = - std::set()) { + std::set(), + const ReplicaType replica_type = ReplicaType::MEMORY) override { if (slice_length == 0 || replica_num == 0) { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } @@ -579,7 +584,8 @@ class CxlAllocationStrategy : public AllocationStrategy { } buffer->change_to_cxl(cxl_segment_name); - replicas.emplace_back(std::move(buffer), ReplicaStatus::PROCESSING); + replicas.emplace_back(std::move(buffer), ReplicaStatus::PROCESSING, + replica_type); VLOG(1) << "Successfully allocated " << replicas.size() << " CXL replica."; diff --git a/mooncake-store/include/allocator.h b/mooncake-store/include/allocator.h index 629f53ae..e28b7d5c 100644 --- a/mooncake-store/include/allocator.h +++ b/mooncake-store/include/allocator.h @@ -15,6 +15,17 @@ using facebook::cachelib::PoolId; namespace mooncake { +/** + * @brief Type of buffer allocator used in the system + */ +enum class ReplicaType { + MEMORY, // Memory replica + DISK, // Disk replica + LOCAL_DISK, // Local disk replica + NOF_SSD, // Nvme-oF SSD replica + ALL, // All memory and NoF replicas in put finalize path +}; + // Constant for unknown free space in allocators that don't track it precisely static constexpr size_t kAllocatorUnknownFreeSpace = std::numeric_limits::max(); @@ -142,7 +153,8 @@ class CachelibBufferAllocator public std::enable_shared_from_this { public: CachelibBufferAllocator(std::string segment_name, size_t base, size_t size, - std::string transport_endpoint); + std::string transport_endpoint, + ReplicaType replica_type = ReplicaType::MEMORY); ~CachelibBufferAllocator() override; @@ -173,6 +185,7 @@ class CachelibBufferAllocator const size_t total_size_; std::atomic_size_t cur_size_; const std::string transport_endpoint_; + const ReplicaType replica_type_; // metrics - removed allocated_bytes_ member // ylt::metric::gauge_t* allocated_bytes_{nullptr}; @@ -193,7 +206,8 @@ class OffsetBufferAllocator public std::enable_shared_from_this { public: OffsetBufferAllocator(std::string segment_name, size_t base, size_t size, - std::string transport_endpoint); + std::string transport_endpoint, + ReplicaType replica_type = ReplicaType::MEMORY); ~OffsetBufferAllocator() override; @@ -226,6 +240,7 @@ class OffsetBufferAllocator const size_t total_size_; std::atomic_size_t cur_size_; const std::string transport_endpoint_; + const ReplicaType replica_type_; // offset allocator implementation std::shared_ptr offset_allocator_; diff --git a/mooncake-store/include/master_client.h b/mooncake-store/include/master_client.h index 44c67226..b4402dd9 100644 --- a/mooncake-store/include/master_client.h +++ b/mooncake-store/include/master_client.h @@ -205,7 +205,8 @@ class MasterClient { * @return ErrorCode indicating success/failure */ [[nodiscard]] std::vector> BatchPutEnd( - const std::vector& keys); + const std::vector& keys, + ReplicaType replica_type = ReplicaType::ALL); /** * @brief Revokes a put operation @@ -222,7 +223,8 @@ class MasterClient { * @return ErrorCode indicating success/failure */ [[nodiscard]] std::vector> BatchPutRevoke( - const std::vector& keys); + const std::vector& keys, + ReplicaType replica_type = ReplicaType::ALL); /** * @brief Starts an upsert operation (insert or update) @@ -297,6 +299,17 @@ class MasterClient { [[nodiscard]] tl::expected MountSegment( const Segment& segment); + [[nodiscard]] tl::expected MountSSDSegment( + const Segment& segment); + + /** + * @brief Registers a NoF ssd segment to master for allocation + * @param segment Segment to register + * @return tl::expected indicating success/failure + */ + [[nodiscard]] tl::expected MountNoFSegment( + const NoFSegment& segment); + /** * @brief Re-mount segments, invoked when the client is the first time to * connect to the master or the client Ping TTL is expired and need @@ -308,6 +321,17 @@ class MasterClient { [[nodiscard]] tl::expected ReMountSegment( const std::vector& segments); + /** + * @brief Re-mount NoF ssd segments, invoked when the client is the first + * time to connect to the master or the client Ping TTL is expired and need + * to remount. This function is idempotent. Client should retry if the + * return code is not ErrorCode::OK. + * @param segments Segments to remount + * @return tl::expected indicating success/failure + */ + [[nodiscard]] tl::expected ReMountNoFSegment( + const std::vector& segments); + /** * @brief Unregisters a memory segment from master * @param segment_id ID of the segment to unmount @@ -319,6 +343,31 @@ class MasterClient { [[nodiscard]] tl::expected GracefulUnmountSegment( const UUID& segment_id, uint64_t grace_period_ms); + /** + * @brief Unregisters a NoF ssd segment from master + * @param segment_id ID of the segment to unmount + * @return tl::expected indicating success/failure + */ + [[nodiscard]] tl::expected UnmountNoFSegment( + const UUID& segment_id); + + /** + * @brief Gets all mounted NoF ssd segments from master + * @return tl::expected, ErrorCode> + * containing all mounted segments + */ + [[nodiscard]] tl::expected, ErrorCode> + GetAllNoFSegments(); + + /** + * @brief Gets all mounted NoF segments that match a segment name together + * with their owner client ids. + * @param segment_name Mounted NoF segment name + * @return Matching segment owner info list + */ + [[nodiscard]] tl::expected, ErrorCode> + GetNoFSegmentsByName(const std::string& segment_name); + /** * @brief Gets the cluster ID for the current client to use as subdirectory * name diff --git a/mooncake-store/include/master_config.h b/mooncake-store/include/master_config.h index bee3dbb3..6ef40239 100644 --- a/mooncake-store/include/master_config.h +++ b/mooncake-store/include/master_config.h @@ -39,7 +39,12 @@ struct MasterConfig { bool allow_evict_soft_pinned_objects; double eviction_ratio; double eviction_high_watermark_ratio; + double nof_eviction_ratio; + double nof_eviction_high_watermark_ratio; int64_t client_live_ttl_sec; + int64_t nof_heartbeat_interval_sec; + uint32_t nof_heartbeat_probe_timeout_ms; + uint32_t nof_heartbeat_failures_threshold; bool enable_ha; bool enable_offload; @@ -112,7 +117,16 @@ class MasterServiceSupervisorConfig { RequiredParam eviction_ratio{"eviction_ratio"}; RequiredParam eviction_high_watermark_ratio{ "eviction_high_watermark_ratio"}; + RequiredParam nof_eviction_ratio{"nof_eviction_ratio"}; + RequiredParam nof_eviction_high_watermark_ratio{ + "nof_eviction_high_watermark_ratio"}; RequiredParam client_live_ttl_sec{"client_live_ttl_sec"}; + RequiredParam nof_heartbeat_interval_sec{ + "nof_heartbeat_interval_sec"}; + RequiredParam nof_heartbeat_probe_timeout_ms{ + "nof_heartbeat_probe_timeout_ms"}; + RequiredParam nof_heartbeat_failures_threshold{ + "nof_heartbeat_failures_threshold"}; RequiredParam enable_offload{"enable_offload"}; RequiredParam rpc_port{"rpc_port"}; RequiredParam rpc_thread_num{"rpc_thread_num"}; @@ -172,7 +186,14 @@ class MasterServiceSupervisorConfig { config.allow_evict_soft_pinned_objects; eviction_ratio = config.eviction_ratio; eviction_high_watermark_ratio = config.eviction_high_watermark_ratio; + nof_eviction_ratio = config.nof_eviction_ratio; + nof_eviction_high_watermark_ratio = + config.nof_eviction_high_watermark_ratio; client_live_ttl_sec = config.client_live_ttl_sec; + nof_heartbeat_interval_sec = config.nof_heartbeat_interval_sec; + nof_heartbeat_probe_timeout_ms = config.nof_heartbeat_probe_timeout_ms; + nof_heartbeat_failures_threshold = + config.nof_heartbeat_failures_threshold; enable_offload = config.enable_offload; offload_on_evict = config.offload_on_evict; offload_force_evict = config.offload_force_evict; @@ -256,9 +277,27 @@ class MasterServiceSupervisorConfig { throw std::runtime_error( "eviction_high_watermark_ratio is not set"); } + if (!nof_eviction_ratio.IsSet()) { + throw std::runtime_error("nof_eviction_ratio is not set"); + } + if (!nof_eviction_high_watermark_ratio.IsSet()) { + throw std::runtime_error( + "nof_eviction_high_watermark_ratio is not set"); + } if (!client_live_ttl_sec.IsSet()) { throw std::runtime_error("client_live_ttl_sec is not set"); } + if (!nof_heartbeat_interval_sec.IsSet()) { + throw std::runtime_error("nof_heartbeat_interval_sec is not set"); + } + if (!nof_heartbeat_probe_timeout_ms.IsSet()) { + throw std::runtime_error( + "nof_heartbeat_probe_timeout_ms is not set"); + } + if (!nof_heartbeat_failures_threshold.IsSet()) { + throw std::runtime_error( + "nof_heartbeat_failures_threshold is not set"); + } if (!rpc_port.IsSet()) { throw std::runtime_error("rpc_port is not set"); } @@ -282,8 +321,16 @@ class WrappedMasterServiceConfig { double eviction_ratio = DEFAULT_EVICTION_RATIO; double eviction_high_watermark_ratio = DEFAULT_EVICTION_HIGH_WATERMARK_RATIO; + double nof_eviction_ratio = DEFAULT_NOF_EVICTION_RATIO; + double nof_eviction_high_watermark_ratio = + DEFAULT_NOF_EVICTION_HIGH_WATERMARK_RATIO; ViewVersionId view_version = 0; int64_t client_live_ttl_sec = DEFAULT_CLIENT_LIVE_TTL_SEC; + int64_t nof_heartbeat_interval_sec = DEFAULT_NOF_HEARTBEAT_INTERVAL_SEC; + uint32_t nof_heartbeat_probe_timeout_ms = + DEFAULT_NOF_HEARTBEAT_PROBE_TIMEOUT_MS; + uint32_t nof_heartbeat_failures_threshold = + DEFAULT_NOF_HEARTBEAT_FAILURES_THRESHOLD; bool enable_ha = false; bool enable_offload = false; bool offload_on_evict = false; @@ -339,8 +386,15 @@ class WrappedMasterServiceConfig { http_port = static_cast(config.metrics_port); eviction_ratio = config.eviction_ratio; eviction_high_watermark_ratio = config.eviction_high_watermark_ratio; + nof_eviction_ratio = config.nof_eviction_ratio; + nof_eviction_high_watermark_ratio = + config.nof_eviction_high_watermark_ratio; view_version = view_version_param; client_live_ttl_sec = config.client_live_ttl_sec; + nof_heartbeat_interval_sec = config.nof_heartbeat_interval_sec; + nof_heartbeat_probe_timeout_ms = config.nof_heartbeat_probe_timeout_ms; + nof_heartbeat_failures_threshold = + config.nof_heartbeat_failures_threshold; enable_ha = config.enable_ha; enable_offload = config.enable_offload; offload_on_evict = config.offload_on_evict; @@ -417,6 +471,9 @@ class WrappedMasterServiceConfig { http_port = static_cast(config.metrics_port); eviction_ratio = config.eviction_ratio; eviction_high_watermark_ratio = config.eviction_high_watermark_ratio; + nof_eviction_ratio = config.nof_eviction_ratio; + nof_eviction_high_watermark_ratio = + config.nof_eviction_high_watermark_ratio; view_version = view_version_param; client_live_ttl_sec = config.client_live_ttl_sec; enable_ha = @@ -473,8 +530,16 @@ class MasterServiceConfigBuilder { double eviction_ratio_ = DEFAULT_EVICTION_RATIO; double eviction_high_watermark_ratio_ = DEFAULT_EVICTION_HIGH_WATERMARK_RATIO; + double nof_eviction_ratio_ = DEFAULT_NOF_EVICTION_RATIO; + double nof_eviction_high_watermark_ratio_ = + DEFAULT_NOF_EVICTION_HIGH_WATERMARK_RATIO; ViewVersionId view_version_ = 0; int64_t client_live_ttl_sec_ = DEFAULT_CLIENT_LIVE_TTL_SEC; + int64_t nof_heartbeat_interval_sec_ = DEFAULT_NOF_HEARTBEAT_INTERVAL_SEC; + uint32_t nof_heartbeat_probe_timeout_ms_ = + DEFAULT_NOF_HEARTBEAT_PROBE_TIMEOUT_MS; + uint32_t nof_heartbeat_failures_threshold_ = + DEFAULT_NOF_HEARTBEAT_FAILURES_THRESHOLD; bool enable_ha_ = false; bool enable_offload_ = false; std::string ha_backend_type_ = "etcd"; @@ -540,6 +605,17 @@ class MasterServiceConfigBuilder { return *this; } + MasterServiceConfigBuilder& set_nof_eviction_ratio(double ratio) { + nof_eviction_ratio_ = ratio; + return *this; + } + + MasterServiceConfigBuilder& set_nof_eviction_high_watermark_ratio( + double ratio) { + nof_eviction_high_watermark_ratio_ = ratio; + return *this; + } + MasterServiceConfigBuilder& set_view_version(ViewVersionId version) { view_version_ = version; return *this; @@ -550,6 +626,23 @@ class MasterServiceConfigBuilder { return *this; } + MasterServiceConfigBuilder& set_nof_heartbeat_interval_sec(int64_t ttl) { + nof_heartbeat_interval_sec_ = ttl; + return *this; + } + + MasterServiceConfigBuilder& set_nof_heartbeat_probe_timeout_ms( + uint32_t timeout_ms) { + nof_heartbeat_probe_timeout_ms_ = timeout_ms; + return *this; + } + + MasterServiceConfigBuilder& set_nof_heartbeat_failures_threshold( + uint32_t threshold) { + nof_heartbeat_failures_threshold_ = threshold; + return *this; + } + MasterServiceConfigBuilder& set_enable_ha(bool enable) { enable_ha_ = enable; return *this; @@ -755,8 +848,16 @@ class MasterServiceConfig { double eviction_ratio = DEFAULT_EVICTION_RATIO; double eviction_high_watermark_ratio = DEFAULT_EVICTION_HIGH_WATERMARK_RATIO; + double nof_eviction_ratio = DEFAULT_NOF_EVICTION_RATIO; + double nof_eviction_high_watermark_ratio = + DEFAULT_NOF_EVICTION_HIGH_WATERMARK_RATIO; ViewVersionId view_version = 0; int64_t client_live_ttl_sec = DEFAULT_CLIENT_LIVE_TTL_SEC; + int64_t nof_heartbeat_interval_sec = DEFAULT_NOF_HEARTBEAT_INTERVAL_SEC; + uint32_t nof_heartbeat_probe_timeout_ms = + DEFAULT_NOF_HEARTBEAT_PROBE_TIMEOUT_MS; + uint32_t nof_heartbeat_failures_threshold = + DEFAULT_NOF_HEARTBEAT_FAILURES_THRESHOLD; bool enable_ha = false; bool enable_offload = false; bool offload_on_evict = false; @@ -808,8 +909,15 @@ class MasterServiceConfig { config.allow_evict_soft_pinned_objects; eviction_ratio = config.eviction_ratio; eviction_high_watermark_ratio = config.eviction_high_watermark_ratio; + nof_eviction_ratio = config.nof_eviction_ratio; + nof_eviction_high_watermark_ratio = + config.nof_eviction_high_watermark_ratio; view_version = config.view_version; client_live_ttl_sec = config.client_live_ttl_sec; + nof_heartbeat_interval_sec = config.nof_heartbeat_interval_sec; + nof_heartbeat_probe_timeout_ms = config.nof_heartbeat_probe_timeout_ms; + nof_heartbeat_failures_threshold = + config.nof_heartbeat_failures_threshold; enable_ha = config.enable_ha; enable_offload = config.enable_offload; offload_on_evict = config.offload_on_evict; @@ -866,8 +974,14 @@ inline MasterServiceConfig MasterServiceConfigBuilder::build() const { config.allow_evict_soft_pinned_objects = allow_evict_soft_pinned_objects_; config.eviction_ratio = eviction_ratio_; config.eviction_high_watermark_ratio = eviction_high_watermark_ratio_; + config.nof_eviction_ratio = nof_eviction_ratio_; + config.nof_eviction_high_watermark_ratio = + nof_eviction_high_watermark_ratio_; config.view_version = view_version_; config.client_live_ttl_sec = client_live_ttl_sec_; + config.nof_heartbeat_interval_sec = nof_heartbeat_interval_sec_; + config.nof_heartbeat_probe_timeout_ms = nof_heartbeat_probe_timeout_ms_; + config.nof_heartbeat_failures_threshold = nof_heartbeat_failures_threshold_; config.enable_ha = enable_ha_; config.enable_offload = enable_offload_; config.ha_backend_type = ha_backend_type_; diff --git a/mooncake-store/include/master_metric_manager.h b/mooncake-store/include/master_metric_manager.h index 1eebc1d4..adc8791e 100644 --- a/mooncake-store/include/master_metric_manager.h +++ b/mooncake-store/include/master_metric_manager.h @@ -40,6 +40,15 @@ class MasterMetricManager { void inc_valid_get_nums(int64_t val = 1); void inc_total_get_nums(int64_t val = 1); + // NoF segment Metrics + void inc_allocated_nof_size(const std::string& segment, int64_t val = 1); + void dec_allocated_nof_size(const std::string& segment, int64_t val = 1); + void reset_allocated_nof_size(); + void inc_total_nof_capacity(const std::string& segment, int64_t val = 1); + void dec_total_nof_capacity(const std::string& segment, int64_t val = 1); + void reset_total_nof_capacity(); + double get_global_nof_used_ratio(void); + enum class CacheHitStat { MEMORY_HITS, SSD_HITS, @@ -67,6 +76,17 @@ class MasterMetricManager { int64_t get_segment_allocated_mem_size(const std::string& segment); int64_t get_segment_total_mem_capacity(const std::string& segment); + // NoF segment Metrics + void inc_allocated_nof_size(int64_t val = 1); + void dec_allocated_nof_size(int64_t val = 1); + void inc_total_nof_capacity(int64_t val = 1); + void dec_total_nof_capacity(int64_t val = 1); + int64_t get_allocated_nof_size(); + int64_t get_total_nof_capacity(); + double get_segment_nof_used_ratio(const std::string& segment); + int64_t get_segment_allocated_nof_size(const std::string& segment); + int64_t get_segment_total_nof_capacity(const std::string& segment); + // File Storage Metrics void inc_allocated_file_size(int64_t val = 1); void dec_allocated_file_size(int64_t val = 1); @@ -117,12 +137,23 @@ class MasterMetricManager { void inc_remove_all_failures(int64_t val = 1); void inc_mount_segment_requests(int64_t val = 1); void inc_mount_segment_failures(int64_t val = 1); + void inc_mount_nof_segment_requests(int64_t val = 1); + void inc_mount_nof_segment_failures(int64_t val = 1); void inc_unmount_segment_requests(int64_t val = 1); void inc_unmount_segment_failures(int64_t val = 1); + void inc_unmount_nof_segment_requests(int64_t val = 1); + void inc_unmount_nof_segment_failures(int64_t val = 1); void inc_remount_segment_requests(int64_t val = 1); void inc_remount_segment_failures(int64_t val = 1); + void inc_remount_nof_segment_requests(int64_t val = 1); + void inc_remount_nof_segment_failures(int64_t val = 1); void inc_ping_requests(int64_t val = 1); void inc_ping_failures(int64_t val = 1); + void inc_nof_heartbeat_success_total(int64_t val = 1); + void inc_nof_heartbeat_failure_total(int64_t val = 1); + void inc_nof_heartbeat_timeout_total(int64_t val = 1); + void inc_nof_segments_unmounted_by_heartbeat_total(int64_t val = 1); + void observe_nof_heartbeat_probe_latency_ms(int64_t latency_ms); // Batch Operation Statistics (Counters) void inc_batch_exist_key_requests(int64_t items); @@ -214,14 +245,32 @@ class MasterMetricManager { int64_t get_batch_put_revoke_failed_items(); // Eviction Metrics + // total eviction metrics void inc_eviction_success(int64_t key_count, int64_t size); void inc_eviction_fail(); // not a single object is evicted + // mem eviction metrics + void inc_mem_eviction_success(int64_t key_count, int64_t size); + void inc_mem_eviction_fail(); // not a single object is evicted + // nof eviction metrics + void inc_nof_eviction_success(int64_t key_count, int64_t size); + void inc_nof_eviction_fail(); // not a single object is evicted // Eviction Metrics Getters + // total eviction metrics int64_t get_eviction_success(); int64_t get_eviction_attempts(); int64_t get_evicted_key_count(); int64_t get_evicted_size(); + // mem eviction metrics + int64_t get_mem_eviction_success(); + int64_t get_mem_eviction_attempts(); + int64_t get_mem_evicted_key_count(); + int64_t get_mem_evicted_size(); + // nof eviction metrics + int64_t get_nof_eviction_success(); + int64_t get_nof_eviction_attempts(); + int64_t get_nof_evicted_key_count(); + int64_t get_nof_evicted_size(); // PutStart Discard Metrics void inc_put_start_discard_cnt(int64_t count, int64_t size); @@ -389,6 +438,14 @@ class MasterMetricManager { int64_t eviction_attempts = 0; int64_t evicted_key_count = 0; int64_t evicted_size = 0; + int64_t mem_eviction_success = 0; + int64_t mem_eviction_attempts = 0; + int64_t mem_evicted_key_count = 0; + int64_t mem_evicted_size = 0; + int64_t nof_eviction_success = 0; + int64_t nof_eviction_attempts = 0; + int64_t nof_evicted_key_count = 0; + int64_t nof_evicted_size = 0; int64_t ping = 0; int64_t ping_fails = 0; int64_t mark_task_to_complete_requests = 0; @@ -417,6 +474,18 @@ class MasterMetricManager { mem_total_capacity_per_segment_; // Segment memory capacity update for // gauge + // NoF Segment Metrics + ylt::metric::gauge_t + nof_allocated_size_; // Overall NoF SSD usage update for gauge + ylt::metric::gauge_t + nof_total_capacity_; // Overall NoF SSD capacity update for gauge + ylt::metric::dynamic_gauge_1t + nof_allocated_size_per_segment_; // NoF segment usage update for + // gauge + ylt::metric::dynamic_gauge_1t + nof_total_capacity_per_segment_; // NoF segment capacity update for + // gauge + // File Storage Metrics ylt::metric::gauge_t file_allocated_size_; ylt::metric::gauge_t file_total_capacity_; @@ -455,8 +524,19 @@ class MasterMetricManager { ylt::metric::counter_t unmount_segment_failures_; ylt::metric::counter_t remount_segment_requests_; ylt::metric::counter_t remount_segment_failures_; + ylt::metric::counter_t mount_nof_segment_requests_; + ylt::metric::counter_t mount_nof_segment_failures_; + ylt::metric::counter_t unmount_nof_segment_requests_; + ylt::metric::counter_t unmount_nof_segment_failures_; + ylt::metric::counter_t remount_nof_segment_requests_; + ylt::metric::counter_t remount_nof_segment_failures_; ylt::metric::counter_t ping_requests_; ylt::metric::counter_t ping_failures_; + ylt::metric::counter_t nof_heartbeat_success_total_; + ylt::metric::counter_t nof_heartbeat_failure_total_; + ylt::metric::counter_t nof_heartbeat_timeout_total_; + ylt::metric::counter_t nof_segments_unmounted_by_heartbeat_total_; + ylt::metric::histogram_t nof_heartbeat_probe_latency_ms_; // Batch Operation Statistics ylt::metric::counter_t batch_exist_key_requests_; @@ -515,10 +595,21 @@ class MasterMetricManager { {CacheHitStat::VALID_GET_RATE, "valid_get_rate"}}; // Eviction Metrics + // total eviction metrics ylt::metric::counter_t eviction_success_; ylt::metric::counter_t eviction_attempts_; ylt::metric::counter_t evicted_key_count_; ylt::metric::counter_t evicted_size_; + // mem eviction metrics + ylt::metric::counter_t mem_eviction_success_; + ylt::metric::counter_t mem_eviction_attempts_; + ylt::metric::counter_t mem_evicted_key_count_; + ylt::metric::counter_t mem_evicted_size_; + // nof eviction metrics + ylt::metric::counter_t nof_eviction_success_; + ylt::metric::counter_t nof_eviction_attempts_; + ylt::metric::counter_t nof_evicted_key_count_; + ylt::metric::counter_t nof_evicted_size_; // PutStart Discard Metrics ylt::metric::counter_t put_start_discard_cnt_; diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index 7c108108..6c7bf581 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -60,10 +61,19 @@ class MasterService { friend class test::SnapshotChildProcessTest; public: + using NoFProbeFn = + std::function; + MasterService(); MasterService(const MasterServiceConfig& config); ~MasterService(); + void SetNoFProbeFnForTesting(NoFProbeFn fn); + size_t GetMountedNoFSegmentCountForTesting(); + bool IsNoFSegmentMountedForTesting(const UUID& segment_id); + std::optional GetNoFHeartbeatFailureCountForTesting( + const UUID& segment_id); + /** * @brief Mount a memory segment for buffer allocation. This function is * idempotent. @@ -76,6 +86,18 @@ class MasterService { auto MountSegment(const Segment& segment, const UUID& client_id) -> tl::expected; + /** + * @brief Mount a NoF SSD segment for buffer allocation. This function is + * idempotent. + * @return ErrorCode::OK on success, + * ErrorCode::INVALID_PARAMS on invalid parameters, + * ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS if the segment cannot + * be mounted temporarily, + * ErrorCode::INTERNAL_ERROR on internal errors. + */ + auto MountNoFSegment(const NoFSegment& segment, const UUID& client_id) + -> tl::expected; + /** * @brief Re-mount segments, invoked when the client is the first time to * connect to the master or the client Ping TTL is expired and need @@ -90,6 +112,21 @@ class MasterService { auto ReMountSegment(const std::vector& segments, const UUID& client_id) -> tl::expected; + /** + * @brief Re-mount NoF SSD segments, invoked when the client is the first + * time to connect to the master or the client Ping TTL is expired and need + * to remount. This function is idempotent. Client should retry if the + * return code is not ErrorCode::OK. + * @return ErrorCode::OK means either all segments are remounted + * successfully or the fail is not solvable by a new remount request. + * ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS if the segment cannot + * be mounted temporarily. + * ErrorCode::INTERNAL_ERROR if something temporary error happens. + */ + auto ReMountNoFSegment(const std::vector& segments, + const UUID& client_id) + -> tl::expected; + /** * @brief Unmount a memory segment. This function is idempotent. * @return ErrorCode::OK on success, @@ -103,6 +140,15 @@ class MasterService { uint64_t grace_period_ms) -> tl::expected; + /** + * @brief Unmount a NoF ssd segment. This function is idempotent. + * @return ErrorCode::OK on success, + * ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS if the segment is + * currently unmounting. + */ + auto UnmountNoFSegment(const UUID& segment_id, const UUID& client_id) + -> tl::expected; + /** * @brief Check if an object exists * @return ErrorCode::OK if exists, otherwise return other ErrorCode @@ -126,6 +172,24 @@ class MasterService { */ auto GetAllSegments() -> tl::expected, ErrorCode>; + /** + * @brief Fetch all mounted NoF segments. + * @return std::vector on success, error code + * otherwise. + */ + auto GetAllNoFSegments() + -> tl::expected, ErrorCode>; + + /** + * @brief Query mounted NoF segments by segment name and return their + * segment ids together with owner client ids. + * @param segment_name Mounted NoF segment name. + * @return Matching segment owner info list on success, error code + * otherwise. + */ + auto GetNoFSegmentsByName(const std::string& segment_name) + -> tl::expected, ErrorCode>; + /** * @brief Query a segment's capacity and used size in bytes. * Conductor should use these information to schedule new requests. @@ -235,7 +299,8 @@ class MasterService { * found, ErrorCode::INVALID_WRITE if replica status is invalid */ std::vector> BatchPutEnd( - const UUID& client_id, const std::vector& keys); + const UUID& client_id, const std::vector& keys, + ReplicaType replica_type = ReplicaType::ALL); /** * @brief Revoke a batch of put operations @@ -243,7 +308,8 @@ class MasterService { * found, ErrorCode::INVALID_WRITE if replica status is invalid */ std::vector> BatchPutRevoke( - const UUID& client_id, const std::vector& keys); + const UUID& client_id, const std::vector& keys, + ReplicaType replica_type = ReplicaType::ALL); /** * @brief Start an upsert operation. If the key does not exist, behaves @@ -575,6 +641,8 @@ class MasterService { // evict_ratio_lowerbound, the second pass will be triggered and try to // fulfill evict ratio lowerbound. void BatchEvict(double evict_ratio_target, double evict_ratio_lowerbound); + void NoFBatchEvict(double evict_ratio_target, + double evict_ratio_lowerbound); // Helper to get a snapshot of alive clients (under client_mutex_ shared // lock) @@ -763,6 +831,32 @@ class MasterService { return num_erased > 0; } + size_t EraseReplica(ReplicaType replica_type) { + return EraseReplicas([replica_type](const Replica& replica) { + if (replica_type == ReplicaType::ALL) { + return replica.is_memory_replica() || + replica.is_nof_replica(); + } + return replica.type() == replica_type; + }); + } + + bool HasMemReplica() const { + return HasReplica(&Replica::fn_is_memory_replica); + } + + bool HasNoFReplica() const { + return HasReplica(&Replica::fn_is_nof_replica); + } + + size_t GetMemReplicaCount() const { + return CountReplicas(&Replica::fn_is_memory_replica); + } + + size_t GetNoFReplicaCount() const { + return CountReplicas(&Replica::fn_is_nof_replica); + } + Replica* GetReplicaBySegmentName(const std::string& segment_name) { return GetFirstReplica([&segment_name](const Replica& replica) { auto names = replica.get_segment_names(); @@ -944,6 +1038,12 @@ class MasterService { // Eviction thread function void EvictionThreadFunc(); + void NofHeartbeatThreadFunc(); + bool TryUnmountNoFSegmentByHeartbeat( + const MountedNoFSegmentSnapshot& snapshot, + const std::string& error_reason); + bool ProbeNoFSegment(const std::string& te_endpoint, + std::string* error_reason); tl::expected PushOffloadingQueue(const std::string& key, Replica& replica); @@ -984,10 +1084,14 @@ class MasterService { const bool allow_evict_soft_pinned_objects_; // Eviction related members - std::atomic need_eviction_{ - false}; // Set to trigger eviction when not enough space left - const double eviction_ratio_; // in range [0.0, 1.0] - const double eviction_high_watermark_ratio_; // in range [0.0, 1.0] + std::atomic need_mem_eviction_{ + false}; // Set to trigger memory eviction when allocation fails + std::atomic need_nof_eviction_{ + false}; // Set to trigger NoF eviction when allocation fails + const double eviction_ratio_; // in range [0.0, 1.0] + const double eviction_high_watermark_ratio_; // in range [0.0, 1.0] + const double nof_eviction_ratio_; // in range [0.0, 1.0] + const double nof_eviction_high_watermark_ratio_; // in range [0.0, 1.0] // Eviction thread related members std::thread eviction_thread_; @@ -1210,6 +1314,27 @@ class MasterService { 128 * 1024; // Size of the client ping queue boost::lockfree::queue client_ping_queue_{kClientPingQueueSize}; const int64_t client_live_ttl_sec_; + const std::chrono::seconds nof_heartbeat_interval_sec_; + const std::chrono::milliseconds nof_heartbeat_probe_timeout_ms_; + const uint32_t nof_heartbeat_failures_threshold_; + + struct NoFHeartbeatState { + UUID owner_client_id{0, 0}; + std::string segment_name; + std::string te_endpoint; + std::chrono::steady_clock::time_point next_probe_at{}; + std::chrono::steady_clock::time_point last_success_at{}; + uint32_t consecutive_failures{0}; + std::string last_error_reason; + }; + std::mutex nof_heartbeat_mutex_; + std::unordered_map> + nof_heartbeat_states_; + std::thread nof_heartbeat_thread_; + std::atomic nof_heartbeat_running_{false}; + static constexpr uint64_t kNoFHeartbeatThreadSleepMs = 100; + mutable std::mutex nof_probe_fn_mutex_; + NoFProbeFn nof_probe_fn_; // if high availability features enabled const bool enable_ha_; @@ -1242,6 +1367,7 @@ class MasterService { // Segment management SegmentManager segment_manager_; + NoFSegmentManager nof_segment_manager_; BufferAllocatorType memory_allocator_type_; std::shared_ptr allocation_strategy_; diff --git a/mooncake-store/include/replica.h b/mooncake-store/include/replica.h index 29b868ea..ca0e268c 100644 --- a/mooncake-store/include/replica.h +++ b/mooncake-store/include/replica.h @@ -4,6 +4,7 @@ #include +#include #include #include #include @@ -26,15 +27,6 @@ namespace mooncake { */ using ReplicaID = uint64_t; -/** - * @brief Type of buffer allocator used in the system - */ -enum class ReplicaType { - MEMORY, // Memory replica - DISK, // Disk replica - LOCAL_DISK // Local disk replica -}; - /** * @brief Stream operator for ReplicaType */ @@ -43,7 +35,9 @@ inline std::ostream& operator<<(std::ostream& os, static const std::unordered_map replica_type_strings{{ReplicaType::MEMORY, "MEMORY"}, {ReplicaType::DISK, "DISK"}, - {ReplicaType::LOCAL_DISK, "LOCAL_DISK"}}; + {ReplicaType::LOCAL_DISK, "LOCAL_DISK"}, + {ReplicaType::NOF_SSD, "NOF_SSD"}, + {ReplicaType::ALL, "ALL"}}; os << (replica_type_strings.count(replicaType) ? replica_type_strings.at(replicaType) @@ -86,18 +80,22 @@ inline std::ostream& operator<<(std::ostream& os, */ struct ReplicateConfig { size_t replica_num{1}; + size_t nof_replica_num{0}; bool with_soft_pin{false}; bool with_hard_pin{false}; // Hard pin: object cannot be evicted std::vector preferred_segments{}; // Preferred segments for allocation std::string preferred_segment{}; // Deprecated: Single preferred segment // for backward compatibility + std::vector + preferred_nof_segments{}; // Preferred NoF segments for allocation bool prefer_alloc_in_same_node{false}; ObjectDataType data_type{ObjectDataType::UNKNOWN}; friend std::ostream& operator<<(std::ostream& os, const ReplicateConfig& config) noexcept { os << "ReplicateConfig: { replica_num: " << config.replica_num + << ", nof_replica_num: " << config.nof_replica_num << ", with_soft_pin: " << config.with_soft_pin << ", with_hard_pin: " << config.with_hard_pin << ", preferred_segments: ["; @@ -110,6 +108,12 @@ struct ReplicateConfig { os << ", preferred_segment (deprecated): " << config.preferred_segment; } + os << ", preferred_nof_segments: ["; + for (size_t i = 0; i < config.preferred_nof_segments.size(); ++i) { + os << config.preferred_nof_segments[i]; + if (i < config.preferred_nof_segments.size() - 1) os << ", "; + } + os << "]"; os << ", prefer_alloc_in_same_node: " << config.prefer_alloc_in_same_node << ", data_type: " << config.data_type << " }"; @@ -117,10 +121,31 @@ struct ReplicateConfig { } }; +enum class ReplicaWriteMode { + SINGLE_REPLICA, + FLEXIBLE_DUAL_REPLICA, + RELIABLE_MULTI_REPLICA, +}; + +inline ReplicaWriteMode DetermineReplicaWriteMode( + const ReplicateConfig& config) { + if (config.replica_num == 1 && config.nof_replica_num == 1) { + return ReplicaWriteMode::FLEXIBLE_DUAL_REPLICA; + } + if (config.replica_num > 1 || config.nof_replica_num > 1) { + return ReplicaWriteMode::RELIABLE_MULTI_REPLICA; + } + return ReplicaWriteMode::SINGLE_REPLICA; +} + struct MemoryReplicaData { std::unique_ptr buffer; }; +struct NoFReplicaData { + std::unique_ptr buffer; +}; + struct DiskReplicaData { std::string file_path; uint64_t object_size = 0; @@ -137,6 +162,11 @@ struct MemoryDescriptor { YLT_REFL(MemoryDescriptor, buffer_descriptor); }; +struct NoFDescriptor { + AllocatedBuffer::Descriptor buffer_descriptor; + YLT_REFL(NoFDescriptor, buffer_descriptor); +}; + struct DiskDescriptor { std::string file_path{}; uint64_t object_size = 0; @@ -161,6 +191,19 @@ class Replica { status_(status), refcnt_(0) {} + // nof ssd replica constructor + Replica(std::unique_ptr buffer, ReplicaStatus status, + ReplicaType replica_type) + : id_(next_id_.fetch_add(1)), status_(status), refcnt_(0) { + if (replica_type == ReplicaType::MEMORY) { + data_ = MemoryReplicaData{std::move(buffer)}; + } else if (replica_type == ReplicaType::NOF_SSD) { + data_ = NoFReplicaData{std::move(buffer)}; + } else { + LOG(ERROR) << "Invalid buffered replica type: " << replica_type; + } + } + // disk replica constructor Replica(std::string file_path, uint64_t object_size, ReplicaStatus status) : id_(next_id_.fetch_add(1)), @@ -275,6 +318,14 @@ class Replica { return replica.is_memory_replica(); } + [[nodiscard]] bool is_nof_replica() const { + return std::holds_alternative(data_); + } + + [[nodiscard]] static bool fn_is_nof_replica(const Replica& replica) { + return replica.is_nof_replica(); + } + [[nodiscard]] bool is_disk_replica() const { return std::holds_alternative(data_); } @@ -299,6 +350,14 @@ class Replica { return false; // DiskReplicaData does not have handles } + [[nodiscard]] bool has_invalid_nof_handle() const { + if (is_nof_replica()) { + const auto& nof_data = std::get(data_); + return !nof_data.buffer->isAllocatorValid(); + } + return false; + } + /** * @brief Check if a local_disk replica's owner client is still alive. * Used by CleanupStaleHandles to remove replicas belonging to expired @@ -372,6 +431,9 @@ class Replica { ReplicaType operator()(const MemoryReplicaData&) const { return ReplicaType::MEMORY; } + ReplicaType operator()(const NoFReplicaData&) const { + return ReplicaType::NOF_SSD; + } ReplicaType operator()(const DiskReplicaData&) const { return ReplicaType::DISK; } @@ -382,7 +444,8 @@ class Replica { struct Descriptor { ReplicaID id; - std::variant + std::variant descriptor_variant; ReplicaStatus status; YLT_REFL(Descriptor, id, descriptor_variant, status); @@ -396,6 +459,14 @@ class Replica { return std::holds_alternative(descriptor_variant); } + bool is_nof_replica() noexcept { + return std::holds_alternative(descriptor_variant); + } + + bool is_nof_replica() const noexcept { + return std::holds_alternative(descriptor_variant); + } + bool is_disk_replica() noexcept { return std::holds_alternative(descriptor_variant); } @@ -422,6 +493,13 @@ class Replica { throw std::runtime_error("Expected MemoryDescriptor"); } + NoFDescriptor& get_nof_descriptor() { + if (auto* desc = std::get_if(&descriptor_variant)) { + return *desc; + } + throw std::runtime_error("Expected NoFDescriptor"); + } + DiskDescriptor& get_disk_descriptor() { if (auto* desc = std::get_if(&descriptor_variant)) { return *desc; @@ -445,6 +523,13 @@ class Replica { throw std::runtime_error("Expected MemoryDescriptor"); } + const NoFDescriptor& get_nof_descriptor() const { + if (auto* desc = std::get_if(&descriptor_variant)) { + return *desc; + } + throw std::runtime_error("Expected NoFDescriptor"); + } + const DiskDescriptor& get_disk_descriptor() const { if (auto* desc = std::get_if(&descriptor_variant)) { return *desc; @@ -465,7 +550,8 @@ class Replica { inline static std::atomic next_id_{1}; ReplicaID id_; - std::variant + std::variant data_; ReplicaStatus status_{ReplicaStatus::UNDEFINED}; @@ -491,6 +577,18 @@ inline Replica::Descriptor Replica::get_descriptor() const { LOG(ERROR) << "Trying to get invalid memory replica descriptor"; } desc.descriptor_variant = std::move(mem_desc); + } else if (is_nof_replica()) { + const auto& nof_data = std::get(data_); + NoFDescriptor nof_desc; + if (nof_data.buffer) { + nof_desc.buffer_descriptor = nof_data.buffer->get_descriptor(); + } else { + nof_desc.buffer_descriptor.size_ = 0; + nof_desc.buffer_descriptor.buffer_address_ = 0; + nof_desc.buffer_descriptor.transport_endpoint_ = ""; + LOG(ERROR) << "Trying to get invalid nof replica descriptor"; + } + desc.descriptor_variant = std::move(nof_desc); } else if (is_disk_replica()) { const auto& disk_data = std::get(data_); DiskDescriptor disk_desc; @@ -520,6 +618,15 @@ inline std::vector> Replica::get_segment_names() segment_names.push_back(std::nullopt); } return segment_names; + } else if (is_nof_replica()) { + const auto& nof_data = std::get(data_); + std::vector> segment_names; + if (nof_data.buffer && nof_data.buffer->isAllocatorValid()) { + segment_names.push_back(nof_data.buffer->getSegmentName()); + } else { + segment_names.push_back(std::nullopt); + } + return segment_names; } return std::vector>(); } @@ -535,6 +642,13 @@ inline std::ostream& operator<<(std::ostream& os, const Replica& replica) { os << *mem_data.buffer; } os << "]"; + } else if (replica.is_nof_replica()) { + const auto& nof_data = std::get(replica.data_); + os << "type: NOF_SSD, buffers: ["; + if (nof_data.buffer) { + os << *nof_data.buffer; + } + os << "]"; } else if (replica.is_disk_replica()) { const auto& disk_data = std::get(replica.data_); os << "type: DISK, file_path: " << disk_data.file_path diff --git a/mooncake-store/include/rpc_service.h b/mooncake-store/include/rpc_service.h index 96f20e10..286ed3fb 100644 --- a/mooncake-store/include/rpc_service.h +++ b/mooncake-store/include/rpc_service.h @@ -18,6 +18,7 @@ #include "types.h" #include "rpc_types.h" #include "master_config.h" +#include "segment.h" namespace mooncake { @@ -61,13 +62,13 @@ class WrappedMasterService { const UUID& client_id, const std::string& key, const uint64_t slice_length, const ReplicateConfig& config); - tl::expected PutEnd(const UUID& client_id, - const std::string& key, - ReplicaType replica_type); + tl::expected PutEnd( + const UUID& client_id, const std::string& key, + ReplicaType replica_type = ReplicaType::ALL); - tl::expected PutRevoke(const UUID& client_id, - const std::string& key, - ReplicaType replica_type); + tl::expected PutRevoke( + const UUID& client_id, const std::string& key, + ReplicaType replica_type = ReplicaType::ALL); std::vector, ErrorCode>> BatchPutStart(const UUID& client_id, const std::vector& keys, @@ -75,10 +76,12 @@ class WrappedMasterService { const ReplicateConfig& config); std::vector> BatchPutEnd( - const UUID& client_id, const std::vector& keys); + const UUID& client_id, const std::vector& keys, + ReplicaType replica_type = ReplicaType::ALL); std::vector> BatchPutRevoke( - const UUID& client_id, const std::vector& keys); + const UUID& client_id, const std::vector& keys, + ReplicaType replica_type = ReplicaType::ALL); tl::expected, ErrorCode> UpsertStart( const UUID& client_id, const std::string& key, @@ -118,9 +121,15 @@ class WrappedMasterService { tl::expected MountSegment(const Segment& segment, const UUID& client_id); + tl::expected MountNoFSegment(const NoFSegment& segment, + const UUID& client_id); + tl::expected ReMountSegment( const std::vector& segments, const UUID& client_id); + tl::expected ReMountNoFSegment( + const std::vector& segments, const UUID& client_id); + tl::expected UnmountSegment(const UUID& segment_id, const UUID& client_id); @@ -128,6 +137,15 @@ class WrappedMasterService { const UUID& segment_id, const UUID& client_id, uint64_t grace_period_ms); + tl::expected UnmountNoFSegment(const UUID& segment_id, + const UUID& client_id); + + [[nodiscard]] tl::expected, ErrorCode> + GetAllNoFSegments(); + + [[nodiscard]] tl::expected, ErrorCode> + GetNoFSegmentsByName(const std::string& segment_name); + tl::expected GetFsdir(); tl::expected GetStorageConfig(); diff --git a/mooncake-store/include/rpc_types.h b/mooncake-store/include/rpc_types.h index dcaeeacc..57fdf2d5 100644 --- a/mooncake-store/include/rpc_types.h +++ b/mooncake-store/include/rpc_types.h @@ -58,6 +58,17 @@ struct GetStorageConfigResponse { }; YLT_REFL(GetStorageConfigResponse, fsdir, enable_disk_eviction, quota_bytes); +struct NoFSegmentOwnerInfo { + UUID segment_id; + UUID client_id; + + NoFSegmentOwnerInfo() = default; + NoFSegmentOwnerInfo(const UUID& segment_id_param, + const UUID& client_id_param) + : segment_id(segment_id_param), client_id(client_id_param) {} +}; +YLT_REFL(NoFSegmentOwnerInfo, segment_id, client_id); + /** * @brief Response structure for CopyStart operation */ diff --git a/mooncake-store/include/segment.h b/mooncake-store/include/segment.h index b55ceda8..426c512e 100644 --- a/mooncake-store/include/segment.h +++ b/mooncake-store/include/segment.h @@ -1,6 +1,7 @@ #pragma once #include +#include #include #include #include @@ -10,6 +11,7 @@ #include "allocation_strategy.h" #include "allocator.h" +#include "rpc_types.h" #include "types.h" namespace mooncake { @@ -50,6 +52,36 @@ struct MountedSegment { std::shared_ptr buf_allocator; }; +struct MountedNoFSegment { + NoFSegment segment; + UUID client_id; + SegmentStatus status; + std::shared_ptr buf_allocator; +}; + +struct MountedNoFSegmentSnapshot { + UUID segment_id; + UUID client_id; + NoFSegment segment; + SegmentStatus status; +}; + +/** + * @brief Stream operator for MountedNoFSegmentSnapshot + */ +inline std::ostream& operator<<( + std::ostream& os, const MountedNoFSegmentSnapshot& snapshot) noexcept { + os << "{segment_id=" << snapshot.segment_id + << ", client_id=" << snapshot.client_id + << ", segment.id=" << snapshot.segment.id + << ", segment.name=" << snapshot.segment.name + << ", segment.base=" << snapshot.segment.base + << ", segment.size=" << snapshot.segment.size + << ", segment.te_endpoint=" << snapshot.segment.te_endpoint + << ", status=" << snapshot.status << "}"; + return os; +} + struct LocalDiskSegment { mutable Mutex offloading_mutex_; bool enable_offloading; @@ -197,6 +229,78 @@ class ScopedSegmentAccess { std::unique_lock lock_; }; +class NoFSegmentManager; + +/** + * @brief RAII-style access to NoF ssd segment mutex for thread-safe segment + * operations + */ +class ScopedNoFSegmentAccess { + public: + /** + * @brief Acquires a lock on the segment mutex + * @param mutex Reference to the segment mutex + */ + explicit ScopedNoFSegmentAccess(NoFSegmentManager* segment_manager, + std::shared_mutex& mutex) + : nof_segment_manager_(segment_manager), lock_(mutex) {} + + /** + * @brief Mount a segment + */ + ErrorCode MountSegment(const NoFSegment& segment, const UUID& client_id); + + /** + * @brief Re-mount a segment. To avoid infinite remount trying, only the + * errors that may be solved by subsequent remount tryings are considered as + * errors. When encounters unsolvable errors, the segment will not be + * mounted while the return value will be OK. + */ + ErrorCode ReMountSegment(const std::vector& segments, + const UUID& client_id); + + /** + * @brief Prepare to unmount a segment by deleting its allocator + */ + ErrorCode PrepareUnmountSegment(const UUID& segment_id, + size_t& metrics_dec_capacity); + + /** + * @brief Deleting the segment to complete the unmounting operation + */ + ErrorCode CommitUnmountSegment(const UUID& segment_id, + const UUID& client_id, + const size_t& metrics_dec_capacity); + + /** + * @brief Get all the segments of a client + */ + ErrorCode GetClientSegments(const UUID& client_id, + std::vector& segments) const; + + /** + * @brief Get all mounted NoF segments with owner information + */ + ErrorCode GetMountedSegments( + std::vector& segments) const; + + /** + * @brief Get the names of all the segments + */ + ErrorCode GetAllSegments(std::vector& all_segments); + + /** + * @brief Get the segment by name. If there are multiple segments with the + * same name, return the first one. + */ + ErrorCode QuerySegments(const std::string& segment, size_t& used, + size_t& capacity); + + private: + NoFSegmentManager* nof_segment_manager_; + std::unique_lock lock_; +}; + /** * @brief RAII-style access to allocators for thread-safe allocator usage */ @@ -352,4 +456,76 @@ class SegmentManager { friend class SegmentSerializer; // for fork serialize }; +class NoFSegmentManager { + public: + /** + * @brief Constructor for SegmentManager + * @param memory_allocator Type of buffer allocator to use for new segments + */ + explicit NoFSegmentManager( + BufferAllocatorType memory_allocator = BufferAllocatorType::CACHELIB) + : memory_allocator_(memory_allocator) {} + + /** + * @brief Get RAII-style access to segment management operations + * @return ScopedSegmentAccess object that holds the lock + */ + ScopedNoFSegmentAccess getNoFSegmentAccess() { + return ScopedNoFSegmentAccess(this, segment_mutex_); + } + + /** + * @brief Get RAII-style access to use allocators + * @return ScopedAllocatorAccess object that holds the lock + */ + ScopedAllocatorAccess getAllocatorAccess() { + return ScopedAllocatorAccess(allocator_manager_, segment_mutex_); + } + + /** + * @brief Get the number of mounted NoF segments + * @return the mounted NoF segment count + */ + int getMountedSegmentCount() const { + std::shared_lock lock(segment_mutex_); + return mounted_segments_.size(); + } + + void GetMountedSegmentsSnapshot( + std::vector& segments) const; + + tl::expected, ErrorCode> GetSegmentsByName( + const std::string& segment_name) const { + std::shared_lock lock(segment_mutex_); + std::vector result; + for (const auto& [segment_id, mounted_segment] : mounted_segments_) { + if (mounted_segment.segment.name == segment_name) { + result.emplace_back(segment_id, mounted_segment.client_id); + } + } + if (result.empty()) { + return tl::make_unexpected(ErrorCode::SEGMENT_NOT_FOUND); + } + return result; + } + + private: + mutable std::shared_mutex segment_mutex_; + std::shared_ptr allocation_strategy_; + const BufferAllocatorType + memory_allocator_; // Type of buffer allocator to use + // allocator_manager_ only contains allocators whose segment status is OK. + AllocatorManager allocator_manager_; + std::unordered_map> + mounted_segments_; // segment_id -> mounted segment + std::unordered_map, boost::hash> + client_segments_; // client_id -> segment_ids + + std::unordered_map + client_by_name_; // segment name -> client_id + + friend class ScopedNoFSegmentAccess; + friend class SegmentTest; +}; + } // namespace mooncake diff --git a/mooncake-store/include/types.h b/mooncake-store/include/types.h index 153900ae..b93e4882 100644 --- a/mooncake-store/include/types.h +++ b/mooncake-store/include/types.h @@ -89,8 +89,13 @@ static constexpr uint64_t DEFAULT_KV_SOFT_PIN_TTL_MS = static constexpr bool DEFAULT_ALLOW_EVICT_SOFT_PINNED_OBJECTS = true; static constexpr double DEFAULT_EVICTION_RATIO = 0.05; static constexpr double DEFAULT_EVICTION_HIGH_WATERMARK_RATIO = 0.95; +static constexpr double DEFAULT_NOF_EVICTION_RATIO = 0.05; +static constexpr double DEFAULT_NOF_EVICTION_HIGH_WATERMARK_RATIO = 0.95; static constexpr int64_t DEFAULT_MASTER_VIEW_LEASE_TTL_SEC = 5; // in seconds static constexpr int64_t DEFAULT_CLIENT_LIVE_TTL_SEC = 10; // in seconds +static constexpr int64_t DEFAULT_NOF_HEARTBEAT_INTERVAL_SEC = 10; +static constexpr uint32_t DEFAULT_NOF_HEARTBEAT_PROBE_TIMEOUT_MS = 1000; +static constexpr uint32_t DEFAULT_NOF_HEARTBEAT_FAILURES_THRESHOLD = 3; static constexpr uint64_t DEFAULT_SNAPSHOT_INTERVAL_SEC = 60 * 10; // in seconds static constexpr uint64_t DEFAULT_SNAPSHOT_CHILD_TIMEOUT_SEC = @@ -402,6 +407,20 @@ enum class AllocationStrategyType { CXL, // CXL-specific allocation }; +/** + * @brief Represents a contiguous NoF ssd region + */ +struct NoFSegment { + UUID id{0, 0}; + std::string name{}; // Logical segment name used for preferred allocation + uintptr_t base{0}; + size_t size{0}; + // TE p2p endpoint (ip:port) for transport-only addressing + std::string te_endpoint{}; + NoFSegment() = default; +}; +YLT_REFL(NoFSegment, id, name, base, size, te_endpoint); + /** * @brief Client status from the master's perspective */ diff --git a/mooncake-store/src/allocator.cpp b/mooncake-store/src/allocator.cpp index 72ef4e3a..23311b83 100644 --- a/mooncake-store/src/allocator.cpp +++ b/mooncake-store/src/allocator.cpp @@ -73,12 +73,14 @@ std::ostream& operator<<(std::ostream& os, const AllocatedBuffer& buffer) { // Removed allocated_bytes parameter and member initialization CachelibBufferAllocator::CachelibBufferAllocator(std::string segment_name, size_t base, size_t size, - std::string transport_endpoint) + std::string transport_endpoint, + ReplicaType replica_type) : segment_name_(segment_name), base_(base), total_size_(size), cur_size_(0), - transport_endpoint_(std::move(transport_endpoint)) { + transport_endpoint_(std::move(transport_endpoint)), + replica_type_(replica_type) { VLOG(1) << "initializing_buffer_allocator segment_name=" << segment_name << " base_address=" << reinterpret_cast(base) << " size=" << size; @@ -110,8 +112,13 @@ CachelibBufferAllocator::CachelibBufferAllocator(std::string segment_name, } CachelibBufferAllocator::~CachelibBufferAllocator() { - MasterMetricManager::instance().dec_allocated_mem_size(segment_name_, - cur_size_); + if (replica_type_ == ReplicaType::MEMORY) { + MasterMetricManager::instance().dec_allocated_mem_size(segment_name_, + cur_size_); + } else if (replica_type_ == ReplicaType::NOF_SSD) { + MasterMetricManager::instance().dec_allocated_nof_size(segment_name_, + cur_size_); + } }; std::unique_ptr CachelibBufferAllocator::allocate( @@ -137,7 +144,13 @@ std::unique_ptr CachelibBufferAllocator::allocate( VLOG(1) << "allocation_succeeded size=" << size << " segment=" << segment_name_ << " address=" << buffer; cur_size_.fetch_add(size); - MasterMetricManager::instance().inc_allocated_mem_size(segment_name_, size); + if (replica_type_ == ReplicaType::MEMORY) { + MasterMetricManager::instance().inc_allocated_mem_size(segment_name_, + size); + } else if (replica_type_ == ReplicaType::NOF_SSD) { + MasterMetricManager::instance().inc_allocated_nof_size(segment_name_, + size); + } return std::make_unique(shared_from_this(), buffer, size); } @@ -151,8 +164,13 @@ void CachelibBufferAllocator::deallocate(AllocatedBuffer* handle) { size_t freed_size = handle->size_; // Store size before handle might become invalid cur_size_.fetch_sub(freed_size); - MasterMetricManager::instance().dec_allocated_mem_size(segment_name_, - freed_size); + if (replica_type_ == ReplicaType::MEMORY) { + MasterMetricManager::instance().dec_allocated_mem_size( + segment_name_, freed_size); + } else if (replica_type_ == ReplicaType::NOF_SSD) { + MasterMetricManager::instance().dec_allocated_nof_size( + segment_name_, freed_size); + } VLOG(1) << "deallocation_succeeded address=" << handle->buffer_ptr_ << " size=" << freed_size << " segment=" << segment_name_; } catch (const std::exception& e) { @@ -165,12 +183,14 @@ void CachelibBufferAllocator::deallocate(AllocatedBuffer* handle) { // OffsetBufferAllocator implementation OffsetBufferAllocator::OffsetBufferAllocator(std::string segment_name, size_t base, size_t size, - std::string transport_endpoint) + std::string transport_endpoint, + ReplicaType replica_type) : segment_name_(segment_name), base_(base), total_size_(size), cur_size_(0), - transport_endpoint_(std::move(transport_endpoint)) { + transport_endpoint_(std::move(transport_endpoint)), + replica_type_(replica_type) { VLOG(1) << "initializing_offset_buffer_allocator segment_name=" << segment_name << " base_address=" << reinterpret_cast(base) << " size=" << size; @@ -205,8 +225,13 @@ OffsetBufferAllocator::OffsetBufferAllocator(std::string segment_name, } OffsetBufferAllocator::~OffsetBufferAllocator() { - MasterMetricManager::instance().dec_allocated_mem_size(segment_name_, - cur_size_); + if (replica_type_ == ReplicaType::MEMORY) { + MasterMetricManager::instance().dec_allocated_mem_size(segment_name_, + cur_size_); + } else if (replica_type_ == ReplicaType::NOF_SSD) { + MasterMetricManager::instance().dec_allocated_nof_size(segment_name_, + cur_size_); + } }; std::unique_ptr OffsetBufferAllocator::allocate(size_t size) { @@ -244,7 +269,13 @@ std::unique_ptr OffsetBufferAllocator::allocate(size_t size) { } cur_size_.fetch_add(size); - MasterMetricManager::instance().inc_allocated_mem_size(segment_name_, size); + if (replica_type_ == ReplicaType::MEMORY) { + MasterMetricManager::instance().inc_allocated_mem_size(segment_name_, + size); + } else if (replica_type_ == ReplicaType::NOF_SSD) { + MasterMetricManager::instance().inc_allocated_nof_size(segment_name_, + size); + } return allocated_buffer; } @@ -255,8 +286,13 @@ void OffsetBufferAllocator::deallocate(AllocatedBuffer* handle) { size_t freed_size = handle->size(); handle->offset_handle_.reset(); cur_size_.fetch_sub(freed_size); - MasterMetricManager::instance().dec_allocated_mem_size(segment_name_, - freed_size); + if (replica_type_ == ReplicaType::MEMORY) { + MasterMetricManager::instance().dec_allocated_mem_size( + segment_name_, freed_size); + } else if (replica_type_ == ReplicaType::NOF_SSD) { + MasterMetricManager::instance().dec_allocated_nof_size( + segment_name_, freed_size); + } VLOG(1) << "deallocation_succeeded address=" << handle->data() << " size=" << freed_size << " segment=" << segment_name_; } catch (const std::exception& e) { diff --git a/mooncake-store/src/master.cpp b/mooncake-store/src/master.cpp index cd137235..2f213c78 100644 --- a/mooncake-store/src/master.cpp +++ b/mooncake-store/src/master.cpp @@ -79,10 +79,15 @@ DEFINE_bool(allow_evict_soft_pinned_objects, DEFINE_validator(default_kv_lease_ttl, ValidateDurationFlag); DEFINE_validator(default_kv_soft_pin_ttl, ValidateDurationFlag); DEFINE_double(eviction_ratio, mooncake::DEFAULT_EVICTION_RATIO, - "Ratio of objects to evict when storage space is full"); + "Ratio of objects to evict when Memory space is full"); DEFINE_double(eviction_high_watermark_ratio, mooncake::DEFAULT_EVICTION_HIGH_WATERMARK_RATIO, - "Ratio of high watermark trigger eviction"); + "Ratio of high watermark trigger eviction in Memory"); +DEFINE_double(nof_eviction_ratio, mooncake::DEFAULT_NOF_EVICTION_RATIO, + "Ratio of objects to evict when NoF SSD space is full"); +DEFINE_double(nof_eviction_high_watermark_ratio, + mooncake::DEFAULT_NOF_EVICTION_HIGH_WATERMARK_RATIO, + "Ratio of high watermark trigger eviction in NoF SSD"); // RPC server configuration parameters (new, preferred) // TODO: deprecate port and max_threads in the future DEFINE_int32(rpc_thread_num, 0, @@ -102,7 +107,14 @@ DEFINE_bool(rpc_enable_tcp_no_delay, true, "Enable TCP_NODELAY for RPC connections"); DEFINE_validator(eviction_ratio, [](const char* flagname, double value) { if (value < 0.0 || value > 1.0) { - LOG(FATAL) << "Eviction ratio must be between 0.0 and 1.0"; + LOG(FATAL) << "Mem eviction ratio must be between 0.0 and 1.0"; + return false; + } + return true; +}); +DEFINE_validator(nof_eviction_ratio, [](const char* flagname, double value) { + if (value < 0.0 || value > 1.0) { + LOG(FATAL) << "NoF eviction ratio must be between 0.0 and 1.0"; return false; } return true; @@ -122,10 +134,22 @@ DEFINE_string(ha_backend_connstring, "", DEFINE_string( etcd_endpoints, "", "Endpoints of ETCD server, separated by semicolon, required in HA mode"); -DEFINE_int64(client_ttl, mooncake::DEFAULT_CLIENT_LIVE_TTL_SEC, - "Seconds a client stays considered alive after the last heartbeat. " - "If this TTL elapses without a refresh, the master treats the " - "client as disconnected and may unmount its segments"); +DEFINE_int64( + client_ttl, mooncake::DEFAULT_CLIENT_LIVE_TTL_SEC, + "Seconds a client stays considered alive after the last heartbeat. " + "If this TTL elapses without a refresh, the master treats the " + "client as disconnected and may unmount its segments"); +DEFINE_int64(nof_heartbeat_interval_sec, + mooncake::DEFAULT_NOF_HEARTBEAT_INTERVAL_SEC, + "How often master probes each mounted NoF segment"); +DEFINE_uint32(nof_heartbeat_probe_timeout_ms, + mooncake::DEFAULT_NOF_HEARTBEAT_PROBE_TIMEOUT_MS, + "Timeout in milliseconds for a single NoF heartbeat probe"); +DEFINE_uint32( + nof_heartbeat_failures_threshold, + mooncake::DEFAULT_NOF_HEARTBEAT_FAILURES_THRESHOLD, + "Consecutive NoF heartbeat failures required before unmounting a NoF " + "segment"); DEFINE_string(root_fs_dir, mooncake::DEFAULT_ROOT_FS_DIR, "Root directory for storage backend, used in HA mode"); @@ -290,9 +314,24 @@ void InitMasterConf(const mooncake::DefaultConfig& default_config, default_config.GetDouble("eviction_high_watermark_ratio", &master_config.eviction_high_watermark_ratio, FLAGS_eviction_high_watermark_ratio); + default_config.GetDouble("nof_eviction_ratio", + &master_config.nof_eviction_ratio, + FLAGS_nof_eviction_ratio); + default_config.GetDouble("nof_eviction_high_watermark_ratio", + &master_config.nof_eviction_high_watermark_ratio, + FLAGS_nof_eviction_high_watermark_ratio); default_config.GetInt64("client_live_ttl_sec", &master_config.client_live_ttl_sec, FLAGS_client_ttl); + default_config.GetInt64("nof_heartbeat_interval_sec", + &master_config.nof_heartbeat_interval_sec, + FLAGS_nof_heartbeat_interval_sec); + default_config.GetUInt32("nof_heartbeat_probe_timeout_ms", + &master_config.nof_heartbeat_probe_timeout_ms, + FLAGS_nof_heartbeat_probe_timeout_ms); + default_config.GetUInt32("nof_heartbeat_failures_threshold", + &master_config.nof_heartbeat_failures_threshold, + FLAGS_nof_heartbeat_failures_threshold); default_config.GetBool("enable_ha", &master_config.enable_ha, FLAGS_enable_ha); @@ -534,6 +573,18 @@ void LoadConfigFromCmdline(mooncake::MasterConfig& master_config, master_config.eviction_high_watermark_ratio = FLAGS_eviction_high_watermark_ratio; } + if ((google::GetCommandLineFlagInfo("nof_eviction_ratio", &info) && + !info.is_default) || + !conf_set) { + master_config.nof_eviction_ratio = FLAGS_nof_eviction_ratio; + } + if ((google::GetCommandLineFlagInfo("nof_eviction_high_watermark_ratio", + &info) && + !info.is_default) || + !conf_set) { + master_config.nof_eviction_high_watermark_ratio = + FLAGS_nof_eviction_high_watermark_ratio; + } if ((google::GetCommandLineFlagInfo("enable_ha", &info) && !info.is_default) || !conf_set) { @@ -564,6 +615,26 @@ void LoadConfigFromCmdline(mooncake::MasterConfig& master_config, !conf_set) { master_config.client_live_ttl_sec = FLAGS_client_ttl; } + if ((google::GetCommandLineFlagInfo("nof_heartbeat_interval_sec", &info) && + !info.is_default) || + !conf_set) { + master_config.nof_heartbeat_interval_sec = + FLAGS_nof_heartbeat_interval_sec; + } + if ((google::GetCommandLineFlagInfo("nof_heartbeat_probe_timeout_ms", + &info) && + !info.is_default) || + !conf_set) { + master_config.nof_heartbeat_probe_timeout_ms = + FLAGS_nof_heartbeat_probe_timeout_ms; + } + if ((google::GetCommandLineFlagInfo("nof_heartbeat_failures_threshold", + &info) && + !info.is_default) || + !conf_set) { + master_config.nof_heartbeat_failures_threshold = + FLAGS_nof_heartbeat_failures_threshold; + } if ((google::GetCommandLineFlagInfo("cluster_id", &info) && !info.is_default) || !conf_set) { diff --git a/mooncake-store/src/master_client.cpp b/mooncake-store/src/master_client.cpp index d7e85fce..7d899d3e 100644 --- a/mooncake-store/src/master_client.cpp +++ b/mooncake-store/src/master_client.cpp @@ -147,11 +147,21 @@ struct RpcNameTraits<&WrappedMasterService::MountSegment> { static constexpr const char* value = "MountSegment"; }; +template <> +struct RpcNameTraits<&WrappedMasterService::MountNoFSegment> { + static constexpr const char* value = "MountNoFSegment"; +}; + template <> struct RpcNameTraits<&WrappedMasterService::ReMountSegment> { static constexpr const char* value = "ReMountSegment"; }; +template <> +struct RpcNameTraits<&WrappedMasterService::ReMountNoFSegment> { + static constexpr const char* value = "ReMountNoFSegment"; +}; + template <> struct RpcNameTraits<&WrappedMasterService::UnmountSegment> { static constexpr const char* value = "UnmountSegment"; @@ -162,6 +172,21 @@ struct RpcNameTraits<&WrappedMasterService::GracefulUnmountSegment> { static constexpr const char* value = "GracefulUnmountSegment"; }; +template <> +struct RpcNameTraits<&WrappedMasterService::UnmountNoFSegment> { + static constexpr const char* value = "UnmountNoFSegment"; +}; + +template <> +struct RpcNameTraits<&WrappedMasterService::GetAllNoFSegments> { + static constexpr const char* value = "GetAllNoFSegments"; +}; + +template <> +struct RpcNameTraits<&WrappedMasterService::GetNoFSegmentsByName> { + static constexpr const char* value = "GetNoFSegmentsByName"; +}; + template <> struct RpcNameTraits<&WrappedMasterService::Ping> { static constexpr const char* value = "Ping"; @@ -546,12 +571,12 @@ tl::expected MasterClient::PutEnd(const std::string& key, } std::vector> MasterClient::BatchPutEnd( - const std::vector& keys) { + const std::vector& keys, ReplicaType replica_type) { ScopedVLogTimer timer(1, "MasterClient::BatchPutEnd"); timer.LogRequest("keys_count=", keys.size()); auto result = invoke_batch_rpc<&WrappedMasterService::BatchPutEnd, void>( - keys.size(), client_id_, keys); + keys.size(), client_id_, keys, replica_type); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -568,12 +593,12 @@ tl::expected MasterClient::PutRevoke( } std::vector> MasterClient::BatchPutRevoke( - const std::vector& keys) { + const std::vector& keys, ReplicaType replica_type) { ScopedVLogTimer timer(1, "MasterClient::BatchPutRevoke"); timer.LogRequest("keys_count=", keys.size()); auto result = invoke_batch_rpc<&WrappedMasterService::BatchPutRevoke, void>( - keys.size(), client_id_, keys); + keys.size(), client_id_, keys, replica_type); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -721,6 +746,19 @@ tl::expected MasterClient::MountSegment( return result; } +tl::expected MasterClient::MountNoFSegment( + const NoFSegment& segment) { + ScopedVLogTimer timer(1, "MasterClient::MountNofSegment"); + timer.LogRequest("NoF segment mount: ", "base=", segment.base, + ", size=", segment.size, ", name=", segment.name, + ", id=", segment.id, ", client_id=", client_id_); + + auto result = invoke_rpc<&WrappedMasterService::MountNoFSegment, void>( + segment, client_id_); + timer.LogResponseExpected(result); + return result; +} + tl::expected MasterClient::ReMountSegment( const std::vector& segments) { ScopedVLogTimer timer(1, "MasterClient::ReMountSegment"); @@ -733,6 +771,18 @@ tl::expected MasterClient::ReMountSegment( return result; } +tl::expected MasterClient::ReMountNoFSegment( + const std::vector& segments) { + ScopedVLogTimer timer(1, "MasterClient::ReMountNofSegment"); + timer.LogRequest("NoF segment remount: ", "segments_num=", segments.size(), + ", client_id=", client_id_); + + auto result = invoke_rpc<&WrappedMasterService::ReMountNoFSegment, void>( + segments, client_id_); + timer.LogResponseExpected(result); + return result; +} + tl::expected MasterClient::UnmountSegment( const UUID& segment_id) { ScopedVLogTimer timer(1, "MasterClient::UnmountSegment"); @@ -757,6 +807,40 @@ tl::expected MasterClient::GracefulUnmountSegment( return result; } +tl::expected MasterClient::UnmountNoFSegment( + const UUID& segment_id) { + ScopedVLogTimer timer(1, "MasterClient::UnmountNoFSegment"); + timer.LogRequest("NoF segment unmount: ", "segment_id=", segment_id, + ", client_id=", client_id_); + + auto result = invoke_rpc<&WrappedMasterService::UnmountNoFSegment, void>( + segment_id, client_id_); + timer.LogResponseExpected(result); + return result; +} + +tl::expected, ErrorCode> +MasterClient::GetAllNoFSegments() { + ScopedVLogTimer timer(1, "MasterClient::GetAllNoFSegments"); + timer.LogRequest("Get all NoF segments, client_id=", client_id_); + + auto result = invoke_rpc<&WrappedMasterService::GetAllNoFSegments, + std::vector>(); + timer.LogResponseExpected(result); + return result; +} + +tl::expected, ErrorCode> +MasterClient::GetNoFSegmentsByName(const std::string& segment_name) { + ScopedVLogTimer timer(1, "MasterClient::GetNoFSegmentsByName"); + timer.LogRequest("segment_name=", segment_name, ", client_id=", client_id_); + + auto result = invoke_rpc<&WrappedMasterService::GetNoFSegmentsByName, + std::vector>(segment_name); + timer.LogResponseExpected(result); + return result; +} + tl::expected MasterClient::Ping() { ScopedVLogTimer timer(1, "MasterClient::Ping"); timer.LogRequest("client_id=", client_id_); diff --git a/mooncake-store/src/master_metric_manager.cpp b/mooncake-store/src/master_metric_manager.cpp index f8e1db24..3d5461f6 100644 --- a/mooncake-store/src/master_metric_manager.cpp +++ b/mooncake-store/src/master_metric_manager.cpp @@ -31,6 +31,18 @@ MasterMetricManager::MasterMetricManager() mem_total_capacity_per_segment_( "segment_total_capacity_bytes", "Total memory capacity of the mounted segment", {"segment"}), + nof_allocated_size_( + "master_nof_allocated_bytes", + "Total nof ssd bytes currently allocated across all segments"), + nof_total_capacity_("master_total_nof_capacity_bytes", + "Total nof ssd capacity across all mounted segments"), + nof_allocated_size_per_segment_( + "nof_segment_allocated_bytes", + "Total nof ssd bytes currently allocated of the segment", + {"segment"}), + nof_total_capacity_per_segment_( + "nof_segment_total_capacity_bytes", + "Total nof ssd capacity of the mounted segment", {"segment"}), file_allocated_size_( "master_allocated_file_size_bytes", "Total bytes currently allocated for file storage in 3fs/nfs"), @@ -113,10 +125,44 @@ MasterMetricManager::MasterMetricManager() remount_segment_failures_( "master_remount_segment_failures_total", "Total number of failed RemountSegment requests"), + mount_nof_segment_requests_( + "master_mount_nof_segment_requests_total", + "Total number of MountNoFSegment requests received"), + mount_nof_segment_failures_( + "master_mount_nof_segment_failures_total", + "Total number of failed MountNoFSegment requests"), + unmount_nof_segment_requests_( + "master_unmount_nof_segment_requests_total", + "Total number of UnmountNoFSegment requests received"), + unmount_nof_segment_failures_( + "master_unmount_nof_segment_failures_total", + "Total number of failed UnmountNoFSegment requests"), + remount_nof_segment_requests_( + "master_remount_nof_segment_requests_total", + "Total number of RemountNoFSegment requests received"), + remount_nof_segment_failures_( + "master_remount_nof_segment_failures_total", + "Total number of failed RemountNoFSegment requests"), ping_requests_("master_ping_requests_total", "Total number of ping requests received"), ping_failures_("master_ping_failures_total", "Total number of failed ping requests"), + nof_heartbeat_success_total_( + "master_nof_heartbeat_success_total", + "Total number of successful NoF heartbeat probes"), + nof_heartbeat_failure_total_( + "master_nof_heartbeat_failure_total", + "Total number of failed NoF heartbeat probes"), + nof_heartbeat_timeout_total_( + "master_nof_heartbeat_timeout_total", + "Total number of timed out NoF heartbeat probes"), + nof_segments_unmounted_by_heartbeat_total_( + "master_nof_segments_unmounted_by_heartbeat_total", + "Total number of NoF segments unmounted due to heartbeat failures"), + nof_heartbeat_probe_latency_ms_( + "master_nof_heartbeat_probe_latency_ms", + "Latency distribution of NoF heartbeat probes in milliseconds", + {1, 2, 5, 10, 20, 50, 100, 200, 500, 1000, 2000}), // Initialize Batch Request Counters batch_exist_key_requests_( @@ -236,6 +282,7 @@ MasterMetricManager::MasterMetricManager() total_get_nums_("total_get_nums_", "Total number of get operations"), // Initialize Eviction Counters + // total eviction eviction_success_("master_successful_evictions_total", "Total number of successful eviction operations"), eviction_attempts_("master_attempted_evictions_total", @@ -244,6 +291,28 @@ MasterMetricManager::MasterMetricManager() "Total number of keys evicted"), evicted_size_("master_evicted_size_bytes", "Total bytes of evicted objects"), + // mem eviction + mem_eviction_success_( + "master_successful_evictions_mem", + "Total number of successful eviction operations in mem"), + mem_eviction_attempts_( + "master_attempted_evictions_mem", + "Total number of attempted eviction operations in mem"), + mem_evicted_key_count_("master_evicted_key_count_mem", + "Total number of keys evicted in mem"), + mem_evicted_size_("master_evicted_size_bytes_mem", + "Total bytes of evicted objects in mem"), + // nof eviction + nof_eviction_success_( + "master_successful_evictions_nof", + "Total number of successful eviction operations in nof"), + nof_eviction_attempts_( + "master_attempted_evictions_nof", + "Total number of attempted eviction operations in nof"), + nof_evicted_key_count_("master_evicted_key_count_nof", + "Total number of keys evicted in nof"), + nof_evicted_size_("master_evicted_size_bytes_nof", + "Total bytes of evicted objects in nof"), // Initialize Discarded Replicas Counters put_start_discard_cnt_("master_put_start_discard_cnt", @@ -454,6 +523,7 @@ void MasterMetricManager::update_metrics_for_zero_output() { // Update Histogram (use observe(0) to mark as changed) value_size_distribution_.observe(0); + nof_heartbeat_probe_latency_ms_.observe(0); // Note: dynamic_gauge_1t (mem_allocated_size_per_segment_ and // mem_total_capacity_per_segment_) are not initialized here because they @@ -541,6 +611,76 @@ double MasterMetricManager::get_segment_mem_used_ratio( return allocated / capacity; } +// NoF segment Metrics +void MasterMetricManager::inc_allocated_nof_size(const std::string& segment, + int64_t val) { + nof_allocated_size_.inc(val); + if (!segment.empty()) nof_allocated_size_per_segment_.inc({segment}, val); +} + +void MasterMetricManager::dec_allocated_nof_size(const std::string& segment, + int64_t val) { + nof_allocated_size_.dec(val); + if (!segment.empty()) nof_allocated_size_per_segment_.dec({segment}, val); +} + +void MasterMetricManager::reset_allocated_nof_size() { + nof_allocated_size_.reset(); +} + +void MasterMetricManager::inc_total_nof_capacity(const std::string& segment, + int64_t val) { + nof_total_capacity_.inc(val); + if (!segment.empty()) nof_total_capacity_per_segment_.inc({segment}, val); +} + +void MasterMetricManager::dec_total_nof_capacity(const std::string& segment, + int64_t val) { + nof_total_capacity_.dec(val); + if (!segment.empty()) nof_total_capacity_per_segment_.dec({segment}, val); +} + +void MasterMetricManager::reset_total_nof_capacity() { + nof_total_capacity_.reset(); +} + +int64_t MasterMetricManager::get_allocated_nof_size() { + return nof_allocated_size_.value(); +} + +int64_t MasterMetricManager::get_total_nof_capacity() { + return nof_total_capacity_.value(); +} + +double MasterMetricManager::get_global_nof_used_ratio(void) { + double allocated = nof_allocated_size_.value(); + double capacity = nof_total_capacity_.value(); + if (capacity == 0) { + return 0.0; + } + return allocated / capacity; +} + +int64_t MasterMetricManager::get_segment_allocated_nof_size( + const std::string& segment) { + return nof_allocated_size_per_segment_.value({segment}); +} + +int64_t MasterMetricManager::get_segment_total_nof_capacity( + const std::string& segment) { + return nof_total_capacity_per_segment_.value({segment}); +} + +double MasterMetricManager::get_segment_nof_used_ratio( + const std::string& segment) { + double allocated = get_segment_allocated_nof_size(segment); + double capacity = get_segment_total_nof_capacity(segment); + if (capacity == 0) { + return 0.0; + } + return allocated / capacity; +} + // File Storage Metrics void MasterMetricManager::inc_allocated_file_size(int64_t val) { file_allocated_size_.inc(val); @@ -697,18 +837,36 @@ void MasterMetricManager::inc_mount_segment_requests(int64_t val) { void MasterMetricManager::inc_mount_segment_failures(int64_t val) { mount_segment_failures_.inc(val); } +void MasterMetricManager::inc_mount_nof_segment_requests(int64_t val) { + mount_nof_segment_requests_.inc(val); +} +void MasterMetricManager::inc_mount_nof_segment_failures(int64_t val) { + mount_nof_segment_failures_.inc(val); +} void MasterMetricManager::inc_unmount_segment_requests(int64_t val) { unmount_segment_requests_.inc(val); } void MasterMetricManager::inc_unmount_segment_failures(int64_t val) { unmount_segment_failures_.inc(val); } +void MasterMetricManager::inc_unmount_nof_segment_requests(int64_t val) { + unmount_nof_segment_requests_.inc(val); +} +void MasterMetricManager::inc_unmount_nof_segment_failures(int64_t val) { + unmount_nof_segment_failures_.inc(val); +} void MasterMetricManager::inc_remount_segment_requests(int64_t val) { remount_segment_requests_.inc(val); } void MasterMetricManager::inc_remount_segment_failures(int64_t val) { remount_segment_failures_.inc(val); } +void MasterMetricManager::inc_remount_nof_segment_requests(int64_t val) { + remount_nof_segment_requests_.inc(val); +} +void MasterMetricManager::inc_remount_nof_segment_failures(int64_t val) { + remount_nof_segment_failures_.inc(val); +} void MasterMetricManager::inc_ping_requests(int64_t val) { ping_requests_.inc(val); } @@ -716,6 +874,28 @@ void MasterMetricManager::inc_ping_failures(int64_t val) { ping_failures_.inc(val); } +void MasterMetricManager::inc_nof_heartbeat_success_total(int64_t val) { + nof_heartbeat_success_total_.inc(val); +} + +void MasterMetricManager::inc_nof_heartbeat_failure_total(int64_t val) { + nof_heartbeat_failure_total_.inc(val); +} + +void MasterMetricManager::inc_nof_heartbeat_timeout_total(int64_t val) { + nof_heartbeat_timeout_total_.inc(val); +} + +void MasterMetricManager::inc_nof_segments_unmounted_by_heartbeat_total( + int64_t val) { + nof_segments_unmounted_by_heartbeat_total_.inc(val); +} + +void MasterMetricManager::observe_nof_heartbeat_probe_latency_ms( + int64_t latency_ms) { + nof_heartbeat_probe_latency_ms_.observe(latency_ms); +} + // Batch Operation Statistics (Counters) void MasterMetricManager::inc_batch_exist_key_requests(int64_t items) { batch_exist_key_requests_.inc(1); @@ -1091,6 +1271,30 @@ void MasterMetricManager::inc_eviction_success(int64_t key_count, void MasterMetricManager::inc_eviction_fail() { eviction_attempts_.inc(); } +void MasterMetricManager::inc_mem_eviction_success(int64_t key_count, + int64_t size) { + mem_evicted_key_count_.inc(key_count); + mem_evicted_size_.inc(size); + mem_eviction_success_.inc(); + mem_eviction_attempts_.inc(); +} + +void MasterMetricManager::inc_mem_eviction_fail() { + mem_eviction_attempts_.inc(); +} + +void MasterMetricManager::inc_nof_eviction_success(int64_t key_count, + int64_t size) { + nof_evicted_key_count_.inc(key_count); + nof_evicted_size_.inc(size); + nof_eviction_success_.inc(); + nof_eviction_attempts_.inc(); +} + +void MasterMetricManager::inc_nof_eviction_fail() { + nof_eviction_attempts_.inc(); +} + int64_t MasterMetricManager::get_eviction_success() { return eviction_success_.value(); } @@ -1107,6 +1311,38 @@ int64_t MasterMetricManager::get_evicted_size() { return evicted_size_.value(); } +int64_t MasterMetricManager::get_mem_eviction_success() { + return mem_eviction_success_.value(); +} + +int64_t MasterMetricManager::get_mem_eviction_attempts() { + return mem_eviction_attempts_.value(); +} + +int64_t MasterMetricManager::get_mem_evicted_key_count() { + return mem_evicted_key_count_.value(); +} + +int64_t MasterMetricManager::get_mem_evicted_size() { + return mem_evicted_size_.value(); +} + +int64_t MasterMetricManager::get_nof_eviction_success() { + return nof_eviction_success_.value(); +} + +int64_t MasterMetricManager::get_nof_eviction_attempts() { + return nof_eviction_attempts_.value(); +} + +int64_t MasterMetricManager::get_nof_evicted_key_count() { + return nof_evicted_key_count_.value(); +} + +int64_t MasterMetricManager::get_nof_evicted_size() { + return nof_evicted_size_.value(); +} + // PutStart Discard Metrics Getters int64_t MasterMetricManager::get_put_start_discard_cnt() { return put_start_discard_cnt_.value(); @@ -1329,6 +1565,11 @@ std::string MasterMetricManager::serialize_metrics() { serialize_metric(remount_segment_failures_); serialize_metric(ping_requests_); serialize_metric(ping_failures_); + serialize_metric(nof_heartbeat_success_total_); + serialize_metric(nof_heartbeat_failure_total_); + serialize_metric(nof_heartbeat_timeout_total_); + serialize_metric(nof_segments_unmounted_by_heartbeat_total_); + serialize_metric(nof_heartbeat_probe_latency_ms_); // Serialize CopyStart, CopyEnd, CopyRevoke, MoveStart, MoveEnd, MoveRevoke // Counters @@ -1474,6 +1715,8 @@ std::string MasterMetricManager::get_summary_string( // --- Get current values --- int64_t mem_allocated = mem_allocated_size_.value(); int64_t mem_capacity = mem_total_capacity_.value(); + int64_t nof_allocated = nof_allocated_size_.value(); + int64_t nof_capacity = nof_total_capacity_.value(); int64_t file_allocated = file_allocated_size_.value(); int64_t file_capacity = file_total_capacity_.value(); int64_t keys = key_count_.value(); @@ -1575,10 +1818,21 @@ std::string MasterMetricManager::get_summary_string( batch_replica_clear_failed_items_.value(); // Eviction counters + // Total counters int64_t eviction_success = eviction_success_.value(); int64_t eviction_attempts = eviction_attempts_.value(); int64_t evicted_key_count = evicted_key_count_.value(); int64_t evicted_size = evicted_size_.value(); + // Mem eviction counters + int64_t mem_eviction_success = mem_eviction_success_.value(); + int64_t mem_eviction_attempts = mem_eviction_attempts_.value(); + int64_t mem_evicted_key_count = mem_evicted_key_count_.value(); + int64_t mem_evicted_size = mem_evicted_size_.value(); + // NoF eviction counters + int64_t nof_eviction_success = nof_eviction_success_.value(); + int64_t nof_eviction_attempts = nof_eviction_attempts_.value(); + int64_t nof_evicted_key_count = nof_evicted_key_count_.value(); + int64_t nof_evicted_size = nof_evicted_size_.value(); // Ping counters int64_t ping = ping_requests_.value(); @@ -1683,6 +1937,14 @@ std::string MasterMetricManager::get_summary_string( current_counters.eviction_attempts = eviction_attempts; current_counters.evicted_key_count = evicted_key_count; current_counters.evicted_size = evicted_size; + current_counters.mem_eviction_success = mem_eviction_success; + current_counters.mem_eviction_attempts = mem_eviction_attempts; + current_counters.mem_evicted_key_count = mem_evicted_key_count; + current_counters.mem_evicted_size = mem_evicted_size; + current_counters.nof_eviction_success = nof_eviction_success; + current_counters.nof_eviction_attempts = nof_eviction_attempts; + current_counters.nof_evicted_key_count = nof_evicted_key_count; + current_counters.nof_evicted_size = nof_evicted_size; current_counters.ping = ping; current_counters.ping_fails = ping_fails; current_counters.mark_task_to_complete_requests = @@ -1745,6 +2007,12 @@ std::string MasterMetricManager::get_summary_string( ss << " (" << std::fixed << std::setprecision(1) << ((double)mem_allocated / (double)mem_capacity * 100.0) << "%)"; } + ss << " | NVMe-oF SSD: " << byte_size_to_string(nof_allocated) << " / " + << byte_size_to_string(nof_capacity); + if (nof_capacity > 0) { + ss << " (" << std::fixed << std::setprecision(1) + << ((double)nof_allocated / (double)nof_capacity * 100.0) << "%)"; + } ss << " | SSD Storage: " << byte_size_to_string(file_allocated) << " / " << byte_size_to_string(file_capacity); ss << " | Keys: " << keys << " (soft-pinned: " << soft_pin_keys << ")"; @@ -1959,6 +2227,20 @@ std::string MasterMetricManager::get_summary_string( << "AllocFail=" << delta(&SummaryCounters::put_start_alloc_fails) << ", " << "keys=" << delta(&SummaryCounters::evicted_key_count) << ", " << "size=" << byte_size_to_string(delta(&SummaryCounters::evicted_size)); + // mem eviction + ss << " | Mem Eviction: " + << "Success/Attempts=" << delta(&SummaryCounters::mem_eviction_success) + << "/" << delta(&SummaryCounters::mem_eviction_attempts) << ", " + << "keys=" << delta(&SummaryCounters::mem_evicted_key_count) << ", " + << "size=" + << byte_size_to_string(delta(&SummaryCounters::mem_evicted_size)); + // nof eviction + ss << " | NoF Eviction: " + << "Success/Attempts=" << delta(&SummaryCounters::nof_eviction_success) + << "/" << delta(&SummaryCounters::nof_eviction_attempts) << ", " + << "keys=" << delta(&SummaryCounters::nof_evicted_key_count) << ", " + << "size=" + << byte_size_to_string(delta(&SummaryCounters::nof_evicted_size)); // Discard summary ss << " | Discard: " diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 5752f4cc..0fa595c7 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -1,8 +1,10 @@ #include "master_service.h" #include +#include #include #include +#include #include #include #include @@ -14,6 +16,9 @@ #include "master_metric_manager.h" #include "segment.h" +#ifdef USE_NOF +#include "spdk/spdk_wrapper.h" +#endif #ifdef STORE_USE_ETCD #include "etcd_helper.h" #include "ha/oplog/etcd_oplog_store.h" @@ -79,6 +84,26 @@ int64_t CurrentTimeMs() { .count(); } +size_t RandomIndex(size_t upper_bound) { + static thread_local std::mt19937 generator(std::random_device{}()); + std::uniform_int_distribution dist(0, upper_bound - 1); + return dist(generator); +} + +bool HasExpectedReplicaAllocation(const ReplicateConfig& config, + size_t allocated_memory_replicas, + size_t allocated_nof_replicas) { + if (config.nof_replica_num == 0) { + return allocated_memory_replicas > 0; + } + if (DetermineReplicaWriteMode(config) == + ReplicaWriteMode::FLEXIBLE_DUAL_REPLICA) { + return allocated_memory_replicas + allocated_nof_replicas > 0; + } + return allocated_memory_replicas == config.replica_num && + allocated_nof_replicas == config.nof_replica_num; +} + } // namespace MasterService::MasterService() : MasterService(MasterServiceConfig()) {} @@ -90,8 +115,17 @@ MasterService::MasterService(const MasterServiceConfig& config) allow_evict_soft_pinned_objects_(config.allow_evict_soft_pinned_objects), eviction_ratio_(config.eviction_ratio), eviction_high_watermark_ratio_(config.eviction_high_watermark_ratio), + nof_eviction_ratio_(config.nof_eviction_ratio), + nof_eviction_high_watermark_ratio_( + config.nof_eviction_high_watermark_ratio), view_version_(config.view_version), client_live_ttl_sec_(config.client_live_ttl_sec), + nof_heartbeat_interval_sec_( + std::chrono::seconds(config.nof_heartbeat_interval_sec)), + nof_heartbeat_probe_timeout_ms_( + std::chrono::milliseconds(config.nof_heartbeat_probe_timeout_ms)), + nof_heartbeat_failures_threshold_( + config.nof_heartbeat_failures_threshold), enable_ha_(config.enable_ha), enable_offload_(config.enable_offload), ha_backend_type_(config.ha_backend_type), @@ -102,6 +136,7 @@ MasterService::MasterService(const MasterServiceConfig& config) enable_disk_eviction_(config.enable_disk_eviction), quota_bytes_(config.quota_bytes), segment_manager_(config.memory_allocator, config.enable_cxl), + nof_segment_manager_(config.memory_allocator), memory_allocator_type_(config.memory_allocator), allocation_strategy_( CreateAllocationStrategy(config.allocation_strategy_type)), @@ -167,6 +202,29 @@ MasterService::MasterService(const MasterServiceConfig& config) "put_start_discard_timeout_sec"); } +#ifdef USE_NOF + if (nof_heartbeat_interval_sec_.count() <= 0) { + LOG(ERROR) << "nof_heartbeat_interval_sec must be positive, current " + << nof_heartbeat_interval_sec_.count(); + throw std::invalid_argument("Invalid nof heartbeat interval"); + } + if (nof_heartbeat_probe_timeout_ms_.count() <= 0) { + LOG(ERROR) << "nof_heartbeat_probe_timeout_ms must be positive, " + << "current " << nof_heartbeat_probe_timeout_ms_.count(); + throw std::invalid_argument("Invalid nof heartbeat probe timeout"); + } + if (nof_heartbeat_failures_threshold_ == 0) { + LOG(ERROR) << "nof_heartbeat_failures_threshold must be positive"; + throw std::invalid_argument("Invalid nof heartbeat failure threshold"); + } + + nof_probe_fn_ = [](const std::string& te_endpoint, uint32_t timeout_ms, + std::string* error_reason) { + return SpdkWrapper::GetInstance().ProbeNofSegment( + te_endpoint, timeout_ms, error_reason); + }; +#endif + // Offload-on-evict: defer LOCAL_DISK offload to eviction time offload_on_evict_ = enable_offload_ && config.offload_on_evict; if (offload_on_evict_) { @@ -190,6 +248,13 @@ MasterService::MasterService(const MasterServiceConfig& config) std::thread(&MasterService::ClientMonitorFunc, this); VLOG(1) << "action=start_client_monitor_thread"; +#ifdef USE_NOF + nof_heartbeat_running_ = true; + nof_heartbeat_thread_ = + std::thread(&MasterService::NofHeartbeatThreadFunc, this); + VLOG(1) << "action=start_nof_heartbeat_thread"; +#endif + // Start task cleanup thread task_cleanup_running_ = true; task_cleanup_thread_ = @@ -266,6 +331,9 @@ MasterService::~MasterService() { task_cleanup_running_ = false; job_dispatch_running_ = false; graceful_unmount_scheduler_.Stop(); +#ifdef USE_NOF + nof_heartbeat_running_ = false; +#endif // Wake sleepers so join() doesn't block for long sleep intervals. task_cleanup_cv_.notify_all(); @@ -276,6 +344,11 @@ MasterService::~MasterService() { if (client_monitor_thread_.joinable()) { client_monitor_thread_.join(); } +#ifdef USE_NOF + if (nof_heartbeat_thread_.joinable()) { + nof_heartbeat_thread_.join(); + } +#endif if (snapshot_thread_.joinable()) { snapshot_thread_.join(); } @@ -287,6 +360,50 @@ MasterService::~MasterService() { } } +void MasterService::SetNoFProbeFnForTesting(NoFProbeFn fn) { +#ifdef USE_NOF + std::lock_guard lock(nof_probe_fn_mutex_); + if (fn) { + nof_probe_fn_ = std::move(fn); + return; + } + nof_probe_fn_ = [](const std::string& te_endpoint, uint32_t timeout_ms, + std::string* error_reason) { + return SpdkWrapper::GetInstance().ProbeNofSegment( + te_endpoint, timeout_ms, error_reason); + }; +#else + (void)fn; +#endif +} + +size_t MasterService::GetMountedNoFSegmentCountForTesting() { + std::vector mounted_segments; + nof_segment_manager_.GetMountedSegmentsSnapshot(mounted_segments); + return mounted_segments.size(); +} + +bool MasterService::IsNoFSegmentMountedForTesting(const UUID& segment_id) { + std::vector mounted_segments; + nof_segment_manager_.GetMountedSegmentsSnapshot(mounted_segments); + return std::any_of( + mounted_segments.begin(), mounted_segments.end(), + [&segment_id](const MountedNoFSegmentSnapshot& snapshot) { + return snapshot.segment_id == segment_id && + snapshot.status == SegmentStatus::OK; + }); +} + +std::optional MasterService::GetNoFHeartbeatFailureCountForTesting( + const UUID& segment_id) { + std::lock_guard lock(nof_heartbeat_mutex_); + auto it = nof_heartbeat_states_.find(segment_id); + if (it == nof_heartbeat_states_.end()) { + return std::nullopt; + } + return it->second.consecutive_failures; +} + auto MasterService::MountSegment(const Segment& segment, const UUID& client_id) -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); @@ -327,6 +444,31 @@ auto MasterService::MountSegment(const Segment& segment, const UUID& client_id) return {}; } +auto MasterService::MountNoFSegment(const NoFSegment& segment, + const UUID& client_id) + -> tl::expected { +#ifndef USE_NOF + LOG(ERROR) << "client_id=" << client_id << ", segment_name=" << segment.name + << ", error=nof_pool_disabled"; + return tl::make_unexpected(ErrorCode::UNAVAILABLE_IN_CURRENT_MODE); +#else + ScopedNoFSegmentAccess nof_segment_access = + nof_segment_manager_.getNoFSegmentAccess(); + + LOG(INFO) << "NoF segment mount: " << "client_id=" << client_id + << ", action=mount_segment, segment_name=" << segment.name; + + auto err = nof_segment_access.MountSegment(segment, client_id); + if (err == ErrorCode::SEGMENT_ALREADY_EXISTS) { + // Return OK because this is an idempotent operation + return {}; + } else if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + return {}; +#endif +} + auto MasterService::ReMountSegment(const std::vector& segments, const UUID& client_id) -> tl::expected { @@ -373,6 +515,25 @@ auto MasterService::ReMountSegment(const std::vector& segments, return {}; } +auto MasterService::ReMountNoFSegment(const std::vector& segments, + const UUID& client_id) + -> tl::expected { +#ifndef USE_NOF + LOG(ERROR) << "client_id=" << client_id + << ", segments_count=" << segments.size() + << ", error=nof_pool_disabled"; + return tl::make_unexpected(ErrorCode::UNAVAILABLE_IN_CURRENT_MODE); +#else + ScopedNoFSegmentAccess nof_segment_access = + nof_segment_manager_.getNoFSegmentAccess(); + ErrorCode err = nof_segment_access.ReMountSegment(segments, client_id); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + return {}; +#endif +} + std::unordered_set> MasterService::getAliveClientsSnapshot() const { std::shared_lock lock(client_mutex_); @@ -500,6 +661,51 @@ auto MasterService::GracefulUnmountSegment(const UUID& segment_id, return {}; } +auto MasterService::UnmountNoFSegment(const UUID& segment_id, + const UUID& client_id) + -> tl::expected { +#ifndef USE_NOF + LOG(ERROR) << "client_id=" << client_id << ", segment_id=" << segment_id + << ", error=nof_pool_disabled"; + return tl::make_unexpected(ErrorCode::UNAVAILABLE_IN_CURRENT_MODE); +#else + size_t metrics_dec_capacity = 0; // to update the metrics + + // 1. Prepare to unmount the segment by deleting its allocator + { + ScopedNoFSegmentAccess segment_access = + nof_segment_manager_.getNoFSegmentAccess(); + ErrorCode err = segment_access.PrepareUnmountSegment( + segment_id, metrics_dec_capacity); + if (err == ErrorCode::SEGMENT_NOT_FOUND) { + // Return OK because this is an idempotent operation + return {}; + } + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + } // Release the segment mutex before long-running step 2 and avoid + // deadlocks + + // 2. Remove the metadata of the related objects + ClearInvalidHandles(); + + // 3. Commit the unmount operation + ScopedNoFSegmentAccess segment_access = + nof_segment_manager_.getNoFSegmentAccess(); + auto err = segment_access.CommitUnmountSegment(segment_id, client_id, + metrics_dec_capacity); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + { + std::lock_guard lock(nof_heartbeat_mutex_); + nof_heartbeat_states_.erase(segment_id); + } + return {}; +#endif +} + auto MasterService::ExistKey(const std::string& key) -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); @@ -553,6 +759,24 @@ auto MasterService::GetAllSegments() return all_segments; } +auto MasterService::GetAllNoFSegments() + -> tl::expected, ErrorCode> { + std::vector mounted_segments; + nof_segment_manager_.GetMountedSegmentsSnapshot(mounted_segments); + + std::vector result; + for (const auto& segment : mounted_segments) { + result.push_back(segment.segment); + } + + return result; +} + +auto MasterService::GetNoFSegmentsByName(const std::string& segment_name) + -> tl::expected, ErrorCode> { + return nof_segment_manager_.GetSegmentsByName(segment_name); +} + auto MasterService::QuerySegments(const std::string& segment) -> tl::expected, ErrorCode> { ScopedSegmentAccess segment_access = segment_manager_.getSegmentAccess(); @@ -830,6 +1054,7 @@ auto MasterService::GetReplicaList(const std::string& key) return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); } + // TODO: NoF SSD support (ranhaojia) if (replica_list[0].is_memory_replica()) { MasterMetricManager::instance().inc_mem_cache_hit_nums(); } else if (replica_list[0].is_disk_replica()) { @@ -851,7 +1076,10 @@ auto MasterService::AllocateAndInsertMetadata( const std::chrono::system_clock::time_point& now) -> tl::expected, ErrorCode> { std::vector replicas; - { + const auto write_mode = DetermineReplicaWriteMode(config); + size_t allocated_memory_replicas = 0; + size_t allocated_nof_replicas = 0; + if (config.replica_num > 0) { ScopedAllocatorAccess allocator_access = segment_manager_.getAllocatorAccess(); const auto& allocator_manager = allocator_access.getAllocatorManager(); @@ -873,12 +1101,73 @@ auto MasterService::AllocateAndInsertMetadata( if (allocation_result.error() == ErrorCode::INVALID_PARAMS) { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } - MasterMetricManager::instance().inc_put_start_alloc_failures(); - need_eviction_ = true; - return tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + if (write_mode != ReplicaWriteMode::FLEXIBLE_DUAL_REPLICA) { + MasterMetricManager::instance().inc_put_start_alloc_failures(); + need_mem_eviction_ = true; + return tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + } + } else { + allocated_memory_replicas = allocation_result->size(); + replicas = std::move(allocation_result.value()); } + } - replicas = std::move(allocation_result.value()); +#ifdef USE_NOF + if (config.nof_replica_num > 0 && + nof_segment_manager_.getMountedSegmentCount() > 0) { + ScopedAllocatorAccess allocator_access = + nof_segment_manager_.getAllocatorAccess(); + const auto& allocator_manager = allocator_access.getAllocatorManager(); + + std::vector preferred_segments = + config.preferred_nof_segments; + + auto allocation_result = allocation_strategy_->Allocate( + allocator_manager, value_length, config.nof_replica_num, + preferred_segments, std::set(), ReplicaType::NOF_SSD); + + if (!allocation_result.has_value()) { + VLOG(1) << "Failed to allocate nof replicas for key=" << key + << ", error: " << allocation_result.error(); + if (allocation_result.error() == ErrorCode::INVALID_PARAMS) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + if (write_mode != ReplicaWriteMode::FLEXIBLE_DUAL_REPLICA) { + MasterMetricManager::instance().inc_put_start_alloc_failures(); + need_nof_eviction_ = true; + return tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + } + } else { + allocated_nof_replicas = allocation_result->size(); + for (auto& replica : allocation_result.value()) { + replicas.push_back(std::move(replica)); + } + } + } +#endif + + if (!HasExpectedReplicaAllocation(config, allocated_memory_replicas, + allocated_nof_replicas)) { + if ((config.replica_num > 0 && + allocated_memory_replicas != config.replica_num) || + (config.nof_replica_num > 0 && + allocated_nof_replicas != config.nof_replica_num)) { + MasterMetricManager::instance().inc_put_start_alloc_failures(); + if (config.replica_num > 0 && + allocated_memory_replicas != config.replica_num) { + need_mem_eviction_ = true; + } + if (config.nof_replica_num > 0 && + allocated_nof_replicas != config.nof_replica_num) { + need_nof_eviction_ = true; + } + } + VLOG(1) << "Failed to satisfy replica allocation requirement for key=" + << key << ", requested_memory_replicas=" << config.replica_num + << ", allocated_memory_replicas=" << allocated_memory_replicas + << ", requested_nof_replicas=" << config.nof_replica_num + << ", allocated_nof_replicas=" << allocated_nof_replicas; + return tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); } if (use_disk_replica_) { @@ -890,8 +1179,26 @@ auto MasterService::AllocateAndInsertMetadata( std::vector replica_list; replica_list.reserve(replicas.size()); + int i = 0; + VLOG(1) << "PutStart, create replicas: client_id=" << client_id + << ", key=" << key << ", value_length=" << value_length; for (const auto& replica : replicas) { - replica_list.emplace_back(replica.get_descriptor()); + const auto desc = replica.get_descriptor(); + replica_list.emplace_back(desc); + + if (replica.is_memory_replica()) { + const auto& mem_desc = desc.get_memory_descriptor(); + VLOG(1) << "Replica #" << ++i << ": buffer_address=" + << mem_desc.buffer_descriptor.buffer_address_ + << ", transport_endpoint=" + << mem_desc.buffer_descriptor.transport_endpoint_; + } else if (replica.is_nof_replica()) { + const auto& nof_desc = desc.get_nof_descriptor(); + VLOG(1) << "Replica #" << ++i << ": buffer_address=" + << nof_desc.buffer_descriptor.buffer_address_ + << ", transport_endpoint=" + << nof_desc.buffer_descriptor.transport_endpoint_; + } } shard->metadata.emplace( @@ -908,12 +1215,30 @@ auto MasterService::PutStart(const UUID& client_id, const std::string& key, const uint64_t slice_length, const ReplicateConfig& config) -> tl::expected, ErrorCode> { - if (config.replica_num == 0 || key.empty() || slice_length == 0) { + if ((config.replica_num == 0 && config.nof_replica_num == 0) || + key.empty() || slice_length == 0) { LOG(ERROR) << "key=" << key << ", replica_num=" << config.replica_num + << ", nof_replica_num=" << config.nof_replica_num << ", slice_length=" << slice_length << ", key_size=" << key.size() << ", error=invalid_params"; return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } + if (config.prefer_alloc_in_same_node && config.nof_replica_num > 0) { + LOG(ERROR) << "key=" << key + << ", nof_replica_num=" << config.nof_replica_num + << ", prefer_alloc_in_same_node=" + << config.prefer_alloc_in_same_node + << ", error=nof_not_supported_with_prefer_same_node"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } +#ifndef USE_NOF + if (config.nof_replica_num > 0) { + LOG(ERROR) << "key=" << key + << ", nof_replica_num=" << config.nof_replica_num + << ", error=nof_pool_disabled"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } +#endif if ((memory_allocator_type_ == BufferAllocatorType::CACHELIB) && (slice_length > kMaxSliceSize)) { @@ -979,6 +1304,20 @@ auto MasterService::PutEnd(const UUID& client_id, const std::string& key, metadata.VisitReplicas( [replica_type](const Replica& replica) { + if (replica_type == ReplicaType::ALL) { + return (replica.is_memory_replica() && + !replica.has_invalid_mem_handle()) || + (replica.is_nof_replica() && + !replica.has_invalid_nof_handle()); + } + if (replica_type == ReplicaType::MEMORY) { + return replica.is_memory_replica() && + !replica.has_invalid_mem_handle(); + } + if (replica_type == ReplicaType::NOF_SSD) { + return replica.is_nof_replica() && + !replica.has_invalid_nof_handle(); + } return replica.type() == replica_type; }, [](Replica& replica) { replica.mark_complete(); }); @@ -986,7 +1325,10 @@ auto MasterService::PutEnd(const UUID& client_id, const std::string& key, if (enable_offload_ && !offload_on_evict_) { auto& shard = accessor.GetShard(); metadata.VisitReplicas( - &Replica::fn_is_completed, [this, &key, &shard](Replica& replica) { + [](const Replica& replica) { + return replica.is_completed() && replica.is_memory_replica(); + }, + [this, &key, &shard](Replica& replica) { auto result = PushOffloadingQueue(key, replica); if (result) { replica.inc_refcnt(); @@ -1003,11 +1345,12 @@ auto MasterService::PutEnd(const UUID& client_id, const std::string& key, accessor.EraseFromProcessing(); } - if (replica_type == ReplicaType::MEMORY) { + if (replica_type == ReplicaType::MEMORY || + (replica_type == ReplicaType::ALL && metadata.HasMemReplica())) { MasterMetricManager::instance().inc_mem_cache_nums(); } else if (replica_type == ReplicaType::DISK) { MasterMetricManager::instance().inc_file_cache_nums(); - } + } // TODO: add inc_nof_cache_nums() (ranhaojia) // 1. Set lease timeout to now, indicating that the object has no lease // at beginning. 2. If this object has soft pin enabled, set it to be soft // pinned. @@ -1077,23 +1420,31 @@ auto MasterService::PutRevoke(const UUID& client_id, const std::string& key, return tl::make_unexpected(ErrorCode::ILLEGAL_CLIENT); } - auto processing_rep = - metadata.GetFirstReplica([replica_type](const Replica& replica) { - return replica.type() == replica_type && !replica.is_processing(); - }); + auto processing_rep = metadata.GetFirstReplica([replica_type]( + const Replica& replica) { + if (replica_type == ReplicaType::ALL) { + return (replica.is_memory_replica() || replica.is_nof_replica()) && + !replica.is_processing(); + } + return replica.type() == replica_type && !replica.is_processing(); + }); if (processing_rep != nullptr) { LOG(ERROR) << "key=" << key << ", status=" << processing_rep->status() << ", error=invalid_replica_status"; return tl::make_unexpected(ErrorCode::INVALID_WRITE); } - if (replica_type == ReplicaType::MEMORY) { + if (replica_type == ReplicaType::MEMORY || + (replica_type == ReplicaType::ALL && metadata.HasMemReplica())) { MasterMetricManager::instance().dec_mem_cache_nums(); } else if (replica_type == ReplicaType::DISK) { MasterMetricManager::instance().dec_file_cache_nums(); } metadata.EraseReplicas([replica_type](const Replica& replica) { + if (replica_type == ReplicaType::ALL) { + return replica.is_memory_replica() || replica.is_nof_replica(); + } return replica.type() == replica_type; }); @@ -1110,21 +1461,23 @@ auto MasterService::PutRevoke(const UUID& client_id, const std::string& key, } std::vector> MasterService::BatchPutEnd( - const UUID& client_id, const std::vector& keys) { + const UUID& client_id, const std::vector& keys, + ReplicaType replica_type) { std::vector> results; results.reserve(keys.size()); for (const auto& key : keys) { - results.emplace_back(PutEnd(client_id, key, ReplicaType::MEMORY)); + results.emplace_back(PutEnd(client_id, key, replica_type)); } return results; } std::vector> MasterService::BatchPutRevoke( - const UUID& client_id, const std::vector& keys) { + const UUID& client_id, const std::vector& keys, + ReplicaType replica_type) { std::vector> results; results.reserve(keys.size()); for (const auto& key : keys) { - results.emplace_back(PutRevoke(client_id, key, ReplicaType::MEMORY)); + results.emplace_back(PutRevoke(client_id, key, replica_type)); } return results; } @@ -1148,12 +1501,30 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, const ReplicateConfig& config) -> tl::expected, ErrorCode> { // --- Parameter validation (same as PutStart) --- - if (config.replica_num == 0 || key.empty() || slice_length == 0) { + if ((config.replica_num == 0 && config.nof_replica_num == 0) || + key.empty() || slice_length == 0) { LOG(ERROR) << "key=" << key << ", replica_num=" << config.replica_num + << ", nof_replica_num=" << config.nof_replica_num << ", slice_length=" << slice_length << ", key_size=" << key.size() << ", error=invalid_params"; return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } + if (config.prefer_alloc_in_same_node && config.nof_replica_num > 0) { + LOG(ERROR) << "key=" << key + << ", nof_replica_num=" << config.nof_replica_num + << ", prefer_alloc_in_same_node=" + << config.prefer_alloc_in_same_node + << ", error=nof_not_supported_with_prefer_same_node"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } +#ifndef USE_NOF + if (config.nof_replica_num > 0) { + LOG(ERROR) << "key=" << key + << ", nof_replica_num=" << config.nof_replica_num + << ", error=nof_pool_disabled"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } +#endif if ((memory_allocator_type_ == BufferAllocatorType::CACHELIB) && (slice_length > kMaxSliceSize)) { @@ -2093,8 +2464,10 @@ bool MasterService::CleanupStaleHandles( // Remove those with invalid allocators (memory replicas on unmounted // segments) and local_disk replicas whose owner client is no longer alive. metadata.EraseReplicas([&alive_clients](const Replica& replica) { - return replica.has_invalid_mem_handle() || - replica.has_stale_local_disk_client(alive_clients); + return (replica.has_invalid_mem_handle() || + replica.has_invalid_nof_handle() || + replica.has_stale_local_disk_client(alive_clients)) && + replica.is_completed(); }); // Return true if no valid replicas remain after cleanup @@ -2354,10 +2727,10 @@ void MasterService::EvictionThreadFunc() { double used_ratio = MasterMetricManager::instance().get_global_mem_used_ratio(); if (used_ratio > eviction_high_watermark_ratio_ || - (need_eviction_ && eviction_ratio_ > 0.0)) { + (need_mem_eviction_ && eviction_ratio_ > 0.0)) { LOG(INFO) << "[EVICT-TRIGGER] memory_ratio=" << used_ratio << " high_watermark=" << eviction_high_watermark_ratio_ - << " need_eviction=" << need_eviction_ + << " need_mem_eviction=" << need_mem_eviction_ << " eviction_ratio=" << eviction_ratio_; double evict_ratio_target = std::max( eviction_ratio_, @@ -2383,6 +2756,22 @@ void MasterService::EvictionThreadFunc() { last_discard_time = now; } +#ifdef USE_NOF + double nof_used_ratio = + MasterMetricManager::instance().get_global_nof_used_ratio(); + if (nof_used_ratio > nof_eviction_high_watermark_ratio_ || + (need_nof_eviction_ && nof_eviction_ratio_ > 0.0)) { + double nof_evict_ratio_target = + std::max(nof_eviction_ratio_, + nof_used_ratio - nof_eviction_high_watermark_ratio_ + + nof_eviction_ratio_); + double nof_evict_ratio_lowerbound = + std::max(nof_evict_ratio_target * 0.5, + nof_used_ratio - nof_eviction_high_watermark_ratio_); + NoFBatchEvict(nof_evict_ratio_target, nof_evict_ratio_lowerbound); + } +#endif + std::this_thread::sleep_for( std::chrono::milliseconds(kEvictionThreadSleepMs)); } @@ -3705,8 +4094,8 @@ void MasterService::BatchEvict(double evict_ratio_target, }; // Randomly select a starting shard to avoid imbalance eviction between - // shards. No need to use expensive random_device here. - size_t start_idx = rand() % kNumShards; + // shards. + size_t start_idx = RandomIndex(kNumShards); std::shared_lock shared_lock(snapshot_mutex_); // First pass: evict objects without soft pin and lease expired @@ -3858,9 +4247,11 @@ void MasterService::BatchEvict(double evict_ratio_target, } } } else if (!soft_pin_objects.empty()) { - // Second pass B: Prioritize evicting objects without soft pin, but - // also allow to evict soft pinned objects. The following code is - // error-prone if the soft pin objects are empty. + // allow_evict_soft_pinned_objects_ is implicitly true if + // soft_pin_objects is not empty Second pass B: Prioritize evicting + // objects without soft pin, but also allow to evict soft pinned + // objects. The following code is error-prone if the soft pin + // objects are empty. const long soft_pin_evict_num = target_evict_num - static_cast(no_pin_objects.size()); @@ -3926,16 +4317,19 @@ void MasterService::BatchEvict(double evict_ratio_target, offload_deferred_count > 0) { // Offload-deferred counts as partial success: work was done (objects // queued for disk offload), so suppress re-triggering until the next - // watermark breach or explicit need_eviction_ signal. - need_eviction_ = false; + // watermark breach or explicit need_mem_eviction_ signal. + need_mem_eviction_ = false; MasterMetricManager::instance().inc_eviction_success(evicted_count, total_freed_size); + MasterMetricManager::instance().inc_mem_eviction_success( + evicted_count, total_freed_size); } else { if (object_count == 0) { // No objects to evict, no need to check again - need_eviction_ = false; + need_mem_eviction_ = false; } MasterMetricManager::instance().inc_eviction_fail(); + MasterMetricManager::instance().inc_mem_eviction_fail(); } VLOG(1) << "action=evict_objects" << ", evicted_count=" << evicted_count << ", offload_deferred=" << offload_deferred_count @@ -3961,6 +4355,84 @@ void MasterService::BatchEvict(double evict_ratio_target, } } +void MasterService::NoFBatchEvict(double evict_ratio_target, + double evict_ratio_lowerbound) { + if (evict_ratio_target < evict_ratio_lowerbound) { + LOG(ERROR) << "nof_evict_ratio_target=" << evict_ratio_target + << ", nof_evict_ratio_lowerbound=" << evict_ratio_lowerbound + << ", error=invalid_params"; + evict_ratio_lowerbound = evict_ratio_target; + } + + auto now = std::chrono::system_clock::now(); + long evicted_count = 0; + long object_count = 0; + uint64_t total_freed_size = 0; + + size_t start_idx = RandomIndex(metadata_shards_.size()); + for (size_t i = 0; i < metadata_shards_.size(); i++) { + MetadataShardAccessorRW shard( + this, (start_idx + i) % metadata_shards_.size()); + DiscardExpiredProcessingReplicas(shard, now); + object_count += shard->metadata.size(); + + const long ideal_evict_num = + std::ceil(object_count * evict_ratio_target) - evicted_count; + if (ideal_evict_num <= 0) { + continue; + } + + long shard_evicted_count = 0; + for (auto it = shard->metadata.begin(); + it != shard->metadata.end() && + shard_evicted_count < ideal_evict_num;) { + auto& metadata = it->second; + if (metadata.IsHardPinned() || !metadata.IsLeaseExpired(now) || + metadata.IsSoftPinned(now)) { + ++it; + continue; + } + + const size_t erased = + metadata.EraseReplicas([](const Replica& replica) { + return replica.is_nof_replica() && replica.is_completed() && + replica.get_refcnt() == 0; + }); + if (erased == 0) { + ++it; + continue; + } + + total_freed_size += metadata.size * erased; + shard_evicted_count++; + if (!metadata.IsValid()) { + it = shard->metadata.erase(it); + } else { + ++it; + } + } + evicted_count += shard_evicted_count; + } + + if (evicted_count > 0) { + need_nof_eviction_ = false; + MasterMetricManager::instance().inc_eviction_success(evicted_count, + total_freed_size); + MasterMetricManager::instance().inc_nof_eviction_success( + evicted_count, total_freed_size); + } else { + if (object_count == 0) { + need_nof_eviction_ = false; + } + MasterMetricManager::instance().inc_eviction_fail(); + MasterMetricManager::instance().inc_nof_eviction_fail(); + } + + VLOG(1) << "action=evict_nof_replicas" + << ", evicted_count=" << evicted_count + << ", total_freed_size=" << total_freed_size; +} + void MasterService::ClientMonitorFunc() { std::unordered_map> @@ -4018,6 +4490,7 @@ void MasterService::ClientMonitorFunc() { ScopedSegmentAccess segment_access = segment_manager_.getSegmentAccess(); for (auto& client_id : expired_clients) { + // mounted mem segemtns of this expired client std::vector segments; segment_access.GetClientSegments(client_id, segments); for (auto& seg : segments) { @@ -4034,7 +4507,7 @@ void MasterService::ClientMonitorFunc() { << ", segment_name=" << seg.name << ", " "error=prepare_unmount_expired_" - "segment_failed"; + "mem_segment_failed"; } } } @@ -4057,7 +4530,7 @@ void MasterService::ClientMonitorFunc() { unmount_segments[i], client_ids[i], dec_capacities[i]); LOG(INFO) << "client_id=" << client_ids[i] << ", segment_name=" << segment_names[i] - << ", action=unmount_expired_segment"; + << ", action=unmount_expired_mem_segment"; } for (auto& client_id : expired_clients) { segment_access.UnmountLocalDiskSegment(client_id); @@ -4070,6 +4543,242 @@ void MasterService::ClientMonitorFunc() { } } +bool MasterService::ProbeNoFSegment(const std::string& te_endpoint, + std::string* error_reason) { +#ifndef USE_NOF + if (error_reason) { + *error_reason = "nof_pool_disabled"; + } + return false; +#else + NoFProbeFn probe_fn; + { + std::lock_guard lock(nof_probe_fn_mutex_); + probe_fn = nof_probe_fn_; + } + if (!probe_fn) { + if (error_reason) { + *error_reason = "probe_not_configured"; + } + return false; + } + return probe_fn( + te_endpoint, + static_cast(nof_heartbeat_probe_timeout_ms_.count()), + error_reason); +#endif +} + +bool MasterService::TryUnmountNoFSegmentByHeartbeat( + const MountedNoFSegmentSnapshot& snapshot, + const std::string& error_reason) { + size_t metrics_dec_capacity = 0; + { + auto nof_segment_access = nof_segment_manager_.getNoFSegmentAccess(); + ErrorCode err = nof_segment_access.PrepareUnmountSegment( + snapshot.segment_id, metrics_dec_capacity); + if (err == ErrorCode::SEGMENT_NOT_FOUND || + err == ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS) { + std::lock_guard lock(nof_heartbeat_mutex_); + nof_heartbeat_states_.erase(snapshot.segment_id); + VLOG(1) << "segment_id=" << snapshot.segment_id + << ", action=skip_nof_heartbeat_unmount" + << ", reason=" << toString(err); + return false; + } + if (err != ErrorCode::OK) { + LOG(ERROR) << "segment_id=" << snapshot.segment_id + << ", segment_name=" << snapshot.segment.name + << ", error=prepare_unmount_nof_segment_by_" + "heartbeat_failed" + << ", reason=" << err; + return false; + } + } + + ClearInvalidHandles(); + + { + auto nof_segment_access = nof_segment_manager_.getNoFSegmentAccess(); + ErrorCode err = nof_segment_access.CommitUnmountSegment( + snapshot.segment_id, snapshot.client_id, metrics_dec_capacity); + if (err != ErrorCode::OK && err != ErrorCode::SEGMENT_NOT_FOUND) { + LOG(ERROR) << "segment_id=" << snapshot.segment_id + << ", segment_name=" << snapshot.segment.name + << ", error=commit_unmount_nof_segment_by_" + "heartbeat_failed" + << ", reason=" << err; + return false; + } + } + + { + std::lock_guard lock(nof_heartbeat_mutex_); + nof_heartbeat_states_.erase(snapshot.segment_id); + } + MasterMetricManager::instance() + .inc_nof_segments_unmounted_by_heartbeat_total(); + LOG(INFO) << "segment_id=" << snapshot.segment_id + << ", client_id=" << snapshot.client_id + << ", segment_name=" << snapshot.segment.name + << ", endpoint=" << snapshot.segment.te_endpoint + << ", action=unmount_nof_segment_by_heartbeat" + << ", last_error_reason=" << error_reason; + return true; +} + +void MasterService::NofHeartbeatThreadFunc() { + size_t next_probe_index = 0; + while (nof_heartbeat_running_) { + auto now = std::chrono::steady_clock::now(); + std::vector mounted_segments; + nof_segment_manager_.GetMountedSegmentsSnapshot(mounted_segments); + + std::vector ok_segments; + ok_segments.reserve(mounted_segments.size()); + for (const auto& snapshot : mounted_segments) { + if (snapshot.status == SegmentStatus::OK) { + ok_segments.push_back(snapshot); + } + } + + std::optional probe_target; + { + std::lock_guard lock(nof_heartbeat_mutex_); + std::unordered_set> live_segment_ids; + live_segment_ids.reserve(ok_segments.size()); + + const auto interval_ms = + std::chrono::duration_cast( + nof_heartbeat_interval_sec_); + for (size_t i = 0; i < ok_segments.size(); ++i) { + const auto& snapshot = ok_segments[i]; + live_segment_ids.insert(snapshot.segment_id); + auto [it, inserted] = + nof_heartbeat_states_.try_emplace(snapshot.segment_id); + auto& state = it->second; + state.owner_client_id = snapshot.client_id; + state.segment_name = snapshot.segment.name; + state.te_endpoint = snapshot.segment.te_endpoint; + if (inserted) { + int64_t spread_ms = 0; + if (!ok_segments.empty()) { + spread_ms = static_cast( + (interval_ms.count() * i) / ok_segments.size()); + } + state.last_success_at = now; + state.next_probe_at = now + nof_heartbeat_interval_sec_ + + std::chrono::milliseconds(spread_ms); + } + } + + for (auto it = nof_heartbeat_states_.begin(); + it != nof_heartbeat_states_.end();) { + if (!live_segment_ids.contains(it->first)) { + it = nof_heartbeat_states_.erase(it); + } else { + ++it; + } + } + + if (!ok_segments.empty()) { + next_probe_index %= ok_segments.size(); + for (size_t offset = 0; offset < ok_segments.size(); ++offset) { + const auto& candidate = + ok_segments[(next_probe_index + offset) % + ok_segments.size()]; + auto state_it = + nof_heartbeat_states_.find(candidate.segment_id); + if (state_it == nof_heartbeat_states_.end()) { + continue; + } + if (state_it->second.next_probe_at <= now) { + probe_target = candidate; + next_probe_index = (next_probe_index + offset + 1) % + ok_segments.size(); + break; + } + } + } + } + + if (!probe_target.has_value()) { + std::this_thread::sleep_for( + std::chrono::milliseconds(kNoFHeartbeatThreadSleepMs)); + continue; + } + + auto probe_start = std::chrono::steady_clock::now(); + std::string error_reason; + bool probe_success = + ProbeNoFSegment(probe_target->segment.te_endpoint, &error_reason); + auto latency_ms = std::chrono::duration_cast( + std::chrono::steady_clock::now() - probe_start) + .count(); + MasterMetricManager::instance().observe_nof_heartbeat_probe_latency_ms( + latency_ms); + + if (probe_success) { + MasterMetricManager::instance().inc_nof_heartbeat_success_total(); + auto success_time = std::chrono::steady_clock::now(); + { + std::lock_guard lock(nof_heartbeat_mutex_); + auto it = nof_heartbeat_states_.find(probe_target->segment_id); + if (it != nof_heartbeat_states_.end()) { + it->second.consecutive_failures = 0; + it->second.last_success_at = success_time; + it->second.last_error_reason.clear(); + it->second.next_probe_at = + success_time + nof_heartbeat_interval_sec_; + } + } + VLOG(1) << "segment_id=" << probe_target->segment_id + << ", segment_name=" << probe_target->segment.name + << ", endpoint=" << probe_target->segment.te_endpoint + << ", action=nof_heartbeat_success" + << ", latency_ms=" << latency_ms; + continue; + } + + MasterMetricManager::instance().inc_nof_heartbeat_failure_total(); + if (error_reason == "completion_timeout") { + MasterMetricManager::instance().inc_nof_heartbeat_timeout_total(); + } + + bool should_unmount = false; + uint32_t failure_count = 0; + auto failure_time = std::chrono::steady_clock::now(); + auto alive_timeout = + nof_heartbeat_interval_sec_ * + static_cast(nof_heartbeat_failures_threshold_); + { + std::lock_guard lock(nof_heartbeat_mutex_); + auto it = nof_heartbeat_states_.find(probe_target->segment_id); + if (it != nof_heartbeat_states_.end()) { + it->second.consecutive_failures++; + failure_count = it->second.consecutive_failures; + it->second.last_error_reason = error_reason; + it->second.next_probe_at = + failure_time + nof_heartbeat_interval_sec_; + should_unmount = + failure_time - it->second.last_success_at >= alive_timeout; + } + } + + LOG(WARNING) << "segment_id=" << probe_target->segment_id + << ", segment_name=" << probe_target->segment.name + << ", endpoint=" << probe_target->segment.te_endpoint + << ", action=nof_heartbeat_failure" + << ", failure_count=" << failure_count + << ", latency_ms=" << latency_ms + << ", reason=" << error_reason; + + if (should_unmount) { + TryUnmountNoFSegmentByHeartbeat(*probe_target, error_reason); + } + } +} + tl::expected, SerializationError> MasterService::MetadataSerializer::Serialize() { msgpack::sbuffer sbuf; diff --git a/mooncake-store/src/rpc_service.cpp b/mooncake-store/src/rpc_service.cpp index 602fac6f..e8d71b11 100644 --- a/mooncake-store/src/rpc_service.cpp +++ b/mooncake-store/src/rpc_service.cpp @@ -1072,7 +1072,8 @@ WrappedMasterService::BatchPutStart(const UUID& client_id, } std::vector> WrappedMasterService::BatchPutEnd( - const UUID& client_id, const std::vector& keys) { + const UUID& client_id, const std::vector& keys, + ReplicaType replica_type) { ScopedVLogTimer timer(1, "BatchPutEnd"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys); @@ -1083,7 +1084,7 @@ std::vector> WrappedMasterService::BatchPutEnd( for (const auto& key : keys) { results.emplace_back( - master_service_.PutEnd(client_id, key, ReplicaType::MEMORY)); + master_service_.PutEnd(client_id, key, replica_type)); } size_t failure_count = 0; @@ -1111,7 +1112,8 @@ std::vector> WrappedMasterService::BatchPutEnd( } std::vector> WrappedMasterService::BatchPutRevoke( - const UUID& client_id, const std::vector& keys) { + const UUID& client_id, const std::vector& keys, + ReplicaType replica_type) { ScopedVLogTimer timer(1, "BatchPutRevoke"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys); @@ -1122,7 +1124,7 @@ std::vector> WrappedMasterService::BatchPutRevoke( for (const auto& key : keys) { results.emplace_back( - master_service_.PutRevoke(client_id, key, ReplicaType::MEMORY)); + master_service_.PutRevoke(client_id, key, replica_type)); } size_t failure_count = 0; @@ -1365,6 +1367,25 @@ tl::expected WrappedMasterService::MountSegment( [] { MasterMetricManager::instance().inc_mount_segment_failures(); }); } +tl::expected WrappedMasterService::MountNoFSegment( + const NoFSegment& segment, const UUID& client_id) { + return execute_rpc( + "MountNoFSegment", + [&] { return master_service_.MountNoFSegment(segment, client_id); }, + [&](auto& timer) { + timer.LogRequest("NoF segment mount: ", "base=", segment.base, + ", size=", segment.size, + ", segment_name=", segment.name, + ", id=", segment.id); + }, + [] { + MasterMetricManager::instance().inc_mount_nof_segment_requests(); + }, + [] { + MasterMetricManager::instance().inc_mount_nof_segment_failures(); + }); +} + tl::expected WrappedMasterService::ReMountSegment( const std::vector& segments, const UUID& client_id) { return execute_rpc( @@ -1378,6 +1399,23 @@ tl::expected WrappedMasterService::ReMountSegment( [] { MasterMetricManager::instance().inc_remount_segment_failures(); }); } +tl::expected WrappedMasterService::ReMountNoFSegment( + const std::vector& segments, const UUID& client_id) { + return execute_rpc( + "ReMountNoFSegment", + [&] { return master_service_.ReMountNoFSegment(segments, client_id); }, + [&](auto& timer) { + timer.LogRequest("NoF segment remount: ", "segments_count=", + segments.size(), ", client_id=", client_id); + }, + [] { + MasterMetricManager::instance().inc_remount_nof_segment_requests(); + }, + [] { + MasterMetricManager::instance().inc_remount_nof_segment_failures(); + }); +} + tl::expected WrappedMasterService::UnmountSegment( const UUID& segment_id, const UUID& client_id) { return execute_rpc( @@ -1414,6 +1452,43 @@ tl::expected WrappedMasterService::GracefulUnmountSegment( }); } +tl::expected WrappedMasterService::UnmountNoFSegment( + const UUID& segment_id, const UUID& client_id) { + return execute_rpc( + "UnmountNoFSegment", + [&] { + return master_service_.UnmountNoFSegment(segment_id, client_id); + }, + [&](auto& timer) { + timer.LogRequest("NoF segment unmount: ", "segment_id=", segment_id, + ", client_id=", client_id); + }, + [] { + MasterMetricManager::instance().inc_unmount_nof_segment_requests(); + }, + [] { + MasterMetricManager::instance().inc_unmount_nof_segment_failures(); + }); +} + +tl::expected, ErrorCode> +WrappedMasterService::GetAllNoFSegments() { + return execute_rpc( + "GetAllNoFSegments", + [&] { return master_service_.GetAllNoFSegments(); }, + [&](auto& timer) { timer.LogRequest("Get all NoF segments"); }, [] {}, + [] {}); +} + +tl::expected, ErrorCode> +WrappedMasterService::GetNoFSegmentsByName(const std::string& segment_name) { + return execute_rpc( + "GetNoFSegmentsByName", + [&] { return master_service_.GetNoFSegmentsByName(segment_name); }, + [&](auto& timer) { timer.LogRequest("segment_name=", segment_name); }, + [] {}, [] {}); +} + tl::expected WrappedMasterService::CopyStart( const UUID& client_id, const std::string& key, const std::string& src_segment, @@ -1787,13 +1862,24 @@ void RegisterRpcService( &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::MountSegment>( &wrapped_master_service); + server.register_handler<&mooncake::WrappedMasterService::MountNoFSegment>( + &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::ReMountSegment>( &wrapped_master_service); + server.register_handler<&mooncake::WrappedMasterService::ReMountNoFSegment>( + &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::UnmountSegment>( &wrapped_master_service); server.register_handler< &mooncake::WrappedMasterService::GracefulUnmountSegment>( &wrapped_master_service); + server.register_handler<&mooncake::WrappedMasterService::UnmountNoFSegment>( + &wrapped_master_service); + server.register_handler<&mooncake::WrappedMasterService::GetAllNoFSegments>( + &wrapped_master_service); + server.register_handler< + &mooncake::WrappedMasterService::GetNoFSegmentsByName>( + &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::Ping>( &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::GetFsdir>( diff --git a/mooncake-store/src/segment.cpp b/mooncake-store/src/segment.cpp index 12571436..b5b9b024 100644 --- a/mooncake-store/src/segment.cpp +++ b/mooncake-store/src/segment.cpp @@ -1048,6 +1048,272 @@ ErrorCode ScopedSegmentAccess::SetSegmentStatusByName( return ErrorCode::OK; } +/* ScopedNoFSegmentAccess Implementation */ +ErrorCode ScopedNoFSegmentAccess::MountSegment(const NoFSegment& segment, + const UUID& client_id) { + const uintptr_t buffer = segment.base; + const size_t size = segment.size; + + // NoF segment base is an NVMe namespace offset, so 0 is valid. + if (size == 0) { + LOG(ERROR) << "NoF segment mount: buffer=" << buffer + << " or size=" << size << " is invalid"; + return ErrorCode::INVALID_PARAMS; + } + + if (nof_segment_manager_->memory_allocator_ == + BufferAllocatorType::CACHELIB && + (buffer % facebook::cachelib::Slab::kSize || + size % facebook::cachelib::Slab::kSize)) { + LOG(ERROR) << "NoF segment mount: buffer=" << buffer + << " or size=" << size << " is not aligned to " + << facebook::cachelib::Slab::kSize + << " as required by Cachelib"; + return ErrorCode::INVALID_PARAMS; + } + + auto exist_segment_it = + nof_segment_manager_->mounted_segments_.find(segment.id); + if (exist_segment_it != nof_segment_manager_->mounted_segments_.end()) { + auto& exist_segment = exist_segment_it->second; + if (exist_segment.status == SegmentStatus::OK) { + LOG(WARNING) << "NoF segment mount: segment_name=" << segment.name + << ", warn=segment_already_exists_by_id"; + return ErrorCode::SEGMENT_ALREADY_EXISTS; + } + LOG(ERROR) << "NoF segment mount: segment_name=" << segment.name + << ", error=segment_already_exists_but_not_ok" + << ", status=" << exist_segment.status; + return ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS; + } + + // Treat the same transport endpoint as the same remote SSD namespace even + // if a retry arrives with a different generated UUID. + for (const auto& [existing_id, existing_segment] : + nof_segment_manager_->mounted_segments_) { + if (existing_segment.status == SegmentStatus::OK && + existing_segment.segment.te_endpoint == segment.te_endpoint) { + LOG(WARNING) << "NoF segment mount: segment_name=" << segment.name + << ", endpoint=" << segment.te_endpoint + << ", warn=segment_already_exists_with_different_id"; + return ErrorCode::SEGMENT_ALREADY_EXISTS; + } + } + + std::shared_ptr allocator; + try { + switch (nof_segment_manager_->memory_allocator_) { + case BufferAllocatorType::CACHELIB: + allocator = std::make_shared( + segment.name, buffer, size, segment.te_endpoint, + ReplicaType::NOF_SSD); + break; + case BufferAllocatorType::OFFSET: + allocator = std::make_shared( + segment.name, buffer, size, segment.te_endpoint, + ReplicaType::NOF_SSD); + break; + default: + LOG(ERROR) << "NoF segment mount: segment_name=" << segment.name + << ", error=unknown_memory_allocator=" + << static_cast( + nof_segment_manager_->memory_allocator_); + return ErrorCode::INVALID_PARAMS; + } + + if (!allocator) { + LOG(ERROR) << "NoF segment mount: segment_name=" << segment.name + << ", error=failed_to_create_allocator"; + return ErrorCode::INVALID_PARAMS; + } + } catch (...) { + LOG(ERROR) << "NoF segment mount: segment_name=" << segment.name + << ", error=exception_during_allocator_creation"; + return ErrorCode::INVALID_PARAMS; + } + + nof_segment_manager_->allocator_manager_.addAllocator(segment.name, + allocator); + nof_segment_manager_->client_segments_[client_id].push_back(segment.id); + nof_segment_manager_->mounted_segments_[segment.id] = { + segment, client_id, SegmentStatus::OK, std::move(allocator)}; + nof_segment_manager_->client_by_name_[segment.name] = client_id; + MasterMetricManager::instance().inc_total_nof_capacity(segment.name, size); + + return ErrorCode::OK; +} + +ErrorCode ScopedNoFSegmentAccess::ReMountSegment( + const std::vector& segments, const UUID& client_id) { + for (const auto& segment : segments) { + ErrorCode err = MountSegment(segment, client_id); + if (err == ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS || + err == ErrorCode::INTERNAL_ERROR) { + LOG(ERROR) << "NoF segment remount: segment_name=" << segment.name + << ", error=fail_to_remount_segment"; + return err; + } + if (err == ErrorCode::INVALID_PARAMS) { + LOG(WARNING) << "NoF segment remount: segment_name=" << segment.name + << ", warn=invalid_params"; + } else if (err == ErrorCode::SEGMENT_ALREADY_EXISTS) { + LOG(WARNING) << "NoF segment remount: segment_name=" << segment.name + << ", warn=segment_already_exists"; + } else if (err != ErrorCode::OK) { + LOG(ERROR) << "NoF segment remount: segment_name=" << segment.name + << ", error=unexpected_error (" << err << ")"; + } + } + + return ErrorCode::OK; +} + +ErrorCode ScopedNoFSegmentAccess::PrepareUnmountSegment( + const UUID& segment_id, size_t& metrics_dec_capacity) { + auto it = nof_segment_manager_->mounted_segments_.find(segment_id); + if (it == nof_segment_manager_->mounted_segments_.end()) { + LOG(WARNING) << "NoF segment unmount: segment_id=" << segment_id + << ", warn=segment_not_found"; + return ErrorCode::SEGMENT_NOT_FOUND; + } + if (it->second.status == SegmentStatus::UNMOUNTING) { + LOG(ERROR) << "NoF segment unmount: segment_id=" << segment_id + << ", error=segment_is_unmounting"; + return ErrorCode::UNAVAILABLE_IN_CURRENT_STATUS; + } + + auto& mounted_segment = it->second; + auto& segment = mounted_segment.segment; + metrics_dec_capacity = segment.size; + + std::shared_ptr allocator = + mounted_segment.buf_allocator; + if (HasAllocator(nof_segment_manager_->allocator_manager_, segment.name, + allocator)) { + nof_segment_manager_->allocator_manager_.removeAllocator(segment.name, + allocator); + } + + mounted_segment.buf_allocator.reset(); + mounted_segment.status = SegmentStatus::UNMOUNTING; + return ErrorCode::OK; +} + +ErrorCode ScopedNoFSegmentAccess::CommitUnmountSegment( + const UUID& segment_id, const UUID& client_id, + const size_t& metrics_dec_capacity) { + bool found_in_client_segments = false; + auto client_it = nof_segment_manager_->client_segments_.find(client_id); + if (client_it != nof_segment_manager_->client_segments_.end()) { + auto& segments = client_it->second; + auto segment_it = + std::find(segments.begin(), segments.end(), segment_id); + if (segment_it != segments.end()) { + segments.erase(segment_it); + found_in_client_segments = true; + } + if (segments.empty()) { + nof_segment_manager_->client_segments_.erase(client_it); + } + } + if (!found_in_client_segments) { + LOG(ERROR) << "NoF segment unmount: segment_id=" << segment_id + << ", error=segment_not_found_in_client_segments"; + } + + std::string segment_name; + auto segment_it = nof_segment_manager_->mounted_segments_.find(segment_id); + if (segment_it != nof_segment_manager_->mounted_segments_.end()) { + segment_name = segment_it->second.segment.name; + nof_segment_manager_->client_by_name_.erase(segment_name); + } + + nof_segment_manager_->mounted_segments_.erase(segment_id); + MasterMetricManager::instance().dec_total_nof_capacity( + segment_name, metrics_dec_capacity); + + return ErrorCode::OK; +} + +ErrorCode ScopedNoFSegmentAccess::GetClientSegments( + const UUID& client_id, std::vector& segments) const { + auto it = nof_segment_manager_->client_segments_.find(client_id); + if (it == nof_segment_manager_->client_segments_.end()) { + return ErrorCode::SEGMENT_NOT_FOUND; + } + segments.clear(); + for (auto& segment_id : it->second) { + auto segment_it = + nof_segment_manager_->mounted_segments_.find(segment_id); + if (segment_it != nof_segment_manager_->mounted_segments_.end()) { + segments.emplace_back(segment_it->second.segment); + } + } + return ErrorCode::OK; +} + +ErrorCode ScopedNoFSegmentAccess::GetMountedSegments( + std::vector& segments) const { + segments.clear(); + segments.reserve(nof_segment_manager_->mounted_segments_.size()); + for (const auto& it : nof_segment_manager_->mounted_segments_) { + segments.push_back(MountedNoFSegmentSnapshot{ + .segment_id = it.first, + .client_id = it.second.client_id, + .segment = it.second.segment, + .status = it.second.status, + }); + } + return ErrorCode::OK; +} + +ErrorCode ScopedNoFSegmentAccess::GetAllSegments( + std::vector& all_segments) { + all_segments.clear(); + for (auto& segment : nof_segment_manager_->mounted_segments_) { + if (segment.second.status == SegmentStatus::OK) { + all_segments.push_back(segment.second.segment.name); + } + } + return ErrorCode::OK; +} + +ErrorCode ScopedNoFSegmentAccess::QuerySegments(const std::string& segment, + size_t& used, + size_t& capacity) { + size_t total_used = 0, total_capacity = 0; + const auto& allocator_manager = nof_segment_manager_->allocator_manager_; + const auto& allocators = allocator_manager.getAllocators(segment); + if (allocators != nullptr) { + for (const auto& allocator : *allocators) { + total_used += allocator->size(); + total_capacity += allocator->capacity(); + } + } + + if (total_capacity == 0) { + VLOG(1) << "NoF segment query: segment=" << segment + << ", error=segment_not_found"; + return ErrorCode::SEGMENT_NOT_FOUND; + } + + used = total_used; + capacity = total_capacity; + return ErrorCode::OK; +} + +void NoFSegmentManager::GetMountedSegmentsSnapshot( + std::vector& segments) const { + std::shared_lock lock(segment_mutex_); + segments.clear(); + segments.reserve(mounted_segments_.size()); + for (const auto& [segment_id, mounted_segment] : mounted_segments_) { + segments.push_back(MountedNoFSegmentSnapshot{ + segment_id, mounted_segment.client_id, mounted_segment.segment, + mounted_segment.status}); + } +} + void SegmentManager::initializeCxlAllocator(const std::string& cxl_path, const size_t cxl_size) { LOG(INFO) << "Init CXL global allocator."; diff --git a/mooncake-store/tests/master_service_test.cpp b/mooncake-store/tests/master_service_test.cpp index af5503a6..ac57417f 100644 --- a/mooncake-store/tests/master_service_test.cpp +++ b/mooncake-store/tests/master_service_test.cpp @@ -44,6 +44,22 @@ class MasterServiceTest : public ::testing::Test { return segment; } +#ifdef USE_NOF + NoFSegment MakeNoFSegment( + std::string name = "test_nof_segment", + std::string endpoint = "test_nof_segment_endpoint", + size_t base = kDefaultSegmentBase + kDefaultSegmentSize, + size_t size = kDefaultSegmentSize) const { + NoFSegment segment; + segment.id = generate_uuid(); + segment.name = std::move(name); + segment.base = base; + segment.size = size; + segment.te_endpoint = std::move(endpoint); + return segment; + } +#endif + MountedSegmentContext PrepareSimpleSegment( MasterService& service, std::string name = "test_segment", size_t base = kDefaultSegmentBase, @@ -406,8 +422,9 @@ TEST_F(MasterServiceTest, PutStartInvalidParams) { std::string key = "test_key"; ReplicateConfig config; - // Test invalid replica_num + // Test invalid replica config config.replica_num = 0; + config.nof_replica_num = 0; auto put_result1 = service_->PutStart(client_id, key, 1024, config); EXPECT_FALSE(put_result1.has_value()); EXPECT_EQ(ErrorCode::INVALID_PARAMS, put_result1.error()); @@ -417,8 +434,106 @@ TEST_F(MasterServiceTest, PutStartInvalidParams) { auto put_result2 = service_->PutStart(client_id, key, 0, config); EXPECT_FALSE(put_result2.has_value()); EXPECT_EQ(ErrorCode::INVALID_PARAMS, put_result2.error()); + + // Test prefer_alloc_in_same_node with nof replicas + config.nof_replica_num = 1; + config.prefer_alloc_in_same_node = true; + auto put_result3 = service_->PutStart(client_id, key, 1024, config); + EXPECT_FALSE(put_result3.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, put_result3.error()); } +#ifdef USE_NOF +TEST_F(MasterServiceTest, PutEndAllCompletesMemoryAndNoFReplicas) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto mem_context = PrepareSimpleSegment(*service_); + NoFSegment nof_segment = MakeNoFSegment(); + const UUID client_id = generate_uuid(); + ASSERT_TRUE(service_->MountNoFSegment(nof_segment, client_id).has_value()); + + ReplicateConfig config; + config.replica_num = 1; + config.nof_replica_num = 1; + auto put_start_result = + service_->PutStart(client_id, "test_key_all", 1024, config); + ASSERT_TRUE(put_start_result.has_value()); + + auto put_end_result = + service_->PutEnd(client_id, "test_key_all", ReplicaType::ALL); + ASSERT_TRUE(put_end_result.has_value()); + + auto get_replica_result = service_->GetReplicaList("test_key_all"); + ASSERT_TRUE(get_replica_result.has_value()); + + bool has_complete_memory = false; + bool has_complete_nof = false; + for (const auto& replica : get_replica_result->replicas) { + if (replica.is_memory_replica() && + replica.status == ReplicaStatus::COMPLETE) { + has_complete_memory = true; + } + if (replica.is_nof_replica() && + replica.status == ReplicaStatus::COMPLETE) { + has_complete_nof = true; + } + } + EXPECT_TRUE(has_complete_memory); + EXPECT_TRUE(has_complete_nof); +} + +TEST_F(MasterServiceTest, PutEndMemoryDoesNotCompleteNoFReplica) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto mem_context = PrepareSimpleSegment(*service_); + NoFSegment nof_segment = + MakeNoFSegment("test_nof_segment_2", "test_nof_segment_endpoint_2"); + const UUID client_id = generate_uuid(); + ASSERT_TRUE(service_->MountNoFSegment(nof_segment, client_id).has_value()); + + ReplicateConfig config; + config.replica_num = 1; + config.nof_replica_num = 1; + auto put_start_result = + service_->PutStart(client_id, "test_key_split", 1024, config); + ASSERT_TRUE(put_start_result.has_value()); + + auto put_end_result = + service_->PutEnd(client_id, "test_key_split", ReplicaType::MEMORY); + ASSERT_TRUE(put_end_result.has_value()); + + auto get_replica_result = service_->GetReplicaList("test_key_split"); + ASSERT_TRUE(get_replica_result.has_value()); + ASSERT_EQ(get_replica_result->replicas.size(), 1u); + EXPECT_TRUE(get_replica_result->replicas[0].is_memory_replica()); + EXPECT_EQ(get_replica_result->replicas[0].status, ReplicaStatus::COMPLETE); + + auto put_revoke_result = + service_->PutRevoke(client_id, "test_key_split", ReplicaType::NOF_SSD); + ASSERT_TRUE(put_revoke_result.has_value()); + + auto final_replica_result = service_->GetReplicaList("test_key_split"); + ASSERT_TRUE(final_replica_result.has_value()); + ASSERT_EQ(final_replica_result->replicas.size(), 1u); + EXPECT_TRUE(final_replica_result->replicas[0].is_memory_replica()); + EXPECT_EQ(final_replica_result->replicas[0].status, + ReplicaStatus::COMPLETE); +} + +TEST_F(MasterServiceTest, PutStartOnePlusOneAllowsSingleAllocatedReplica) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto mem_context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + ReplicateConfig config; + config.replica_num = 1; + config.nof_replica_num = 1; + auto put_start_result = + service_->PutStart(client_id, "test_key_one_plus_one", 1024, config); + ASSERT_TRUE(put_start_result.has_value()); + ASSERT_EQ(put_start_result->size(), 1u); + EXPECT_TRUE(put_start_result->front().is_memory_replica()); +} +#endif + TEST_F(MasterServiceTest, PutStartEndFlow) { std::unique_ptr service_(new MasterService()); [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); @@ -3011,8 +3126,8 @@ TEST_F(MasterServiceTest, ReplicationFactorTwoWithSingleSegment) { [[maybe_unused]] const auto context = PrepareSimpleSegment( *service_, "single_segment", kBaseAddr, kSegmentSize); - // Request replication factor 2 with a single 1KB slice - // With best-effort semantics, should succeed with 1 replica + // Request replication factor 2 with a single 1KB slice. + // With best-effort semantics, should succeed with 1 replica. const std::string key = "replication_factor_two_single_segment"; uint64_t slice_length = 1024; ReplicateConfig config; @@ -3023,11 +3138,11 @@ TEST_F(MasterServiceTest, ReplicationFactorTwoWithSingleSegment) { ASSERT_TRUE(put_start_result.has_value()); auto replicas = put_start_result.value(); - // Should get 1 replica instead of the requested 2 (best-effort) + // Should get 1 replica instead of the requested 2 (best-effort). EXPECT_EQ(1u, replicas.size()); EXPECT_TRUE(replicas[0].is_memory_replica()); - // Verify the replica is properly allocated on the single segment + // Verify the replica is properly allocated on the single segment. auto mem_desc = replicas[0].get_memory_descriptor(); EXPECT_EQ("single_segment", mem_desc.buffer_descriptor.transport_endpoint_); EXPECT_EQ(1024u, mem_desc.buffer_descriptor.size_); -- 2.34.1 From 32ccb9f7d6cff43b2c33c03ef6b9f2502219b447 Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Thu, 21 May 2026 12:08:02 +0800 Subject: [PATCH 103/382] [Store] correct GC interval default fallback in FromEnvironment (#2126) * fix(file_storage): correct GC interval default fallback in FromEnvironment The GetEnvOr fallback for client_buffer_gc_interval_seconds was incorrectly using config.heartbeat_interval_seconds (10s) instead of config.client_buffer_gc_interval_seconds (1s). This copy-paste bug caused the GC thread to run every 10 seconds instead of every 1 second when MOONCAKE_OFFLOAD_CLIENT_BUFFER_GC_INTERVAL_SECONDS is not set, significantly delaying zombie buffer reclamation. Fixes kvcache-ai/Mooncake#2119 * docs: update GC interval default value --- docs/source/deployment/ssd-offload.md | 2 +- mooncake-store/src/file_storage.cpp | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/deployment/ssd-offload.md b/docs/source/deployment/ssd-offload.md index 970d1335..30bf8fc1 100644 --- a/docs/source/deployment/ssd-offload.md +++ b/docs/source/deployment/ssd-offload.md @@ -113,7 +113,7 @@ store.setup_dummy( | `MOONCAKE_OFFLOAD_TOTAL_SIZE_LIMIT_BYTES` | `2199023255552` (2 TB) | Maximum disk usage | | `MOONCAKE_OFFLOAD_TOTAL_KEYS_LIMIT` | `10000000` | Maximum number of objects on disk | | `MOONCAKE_OFFLOAD_HEARTBEAT_INTERVAL_SECONDS` | `10` | Interval for offload heartbeat to master (seconds) | -| `MOONCAKE_OFFLOAD_CLIENT_BUFFER_GC_INTERVAL_SECONDS` | `10` | Interval for reclaiming expired offload buffers; defaults to the heartbeat interval in the current implementation | +| `MOONCAKE_OFFLOAD_CLIENT_BUFFER_GC_INTERVAL_SECONDS` | `1` | Interval for reclaiming expired offload buffers; defaults to the heartbeat interval in the current implementation | | `MOONCAKE_OFFLOAD_CLIENT_BUFFER_GC_TTL_MS` | `5000` | Lease time for buffers returned by `batch_get_offload_object` before GC reclaims them | | `MOONCAKE_OFFLOAD_USE_URING` | `false` | Enable io_uring for async file I/O | diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index 63e34492..a445aa9b 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -58,7 +58,7 @@ FileStorageConfig FileStorageConfig::FromEnvironment() { config.heartbeat_interval_seconds); config.client_buffer_gc_interval_seconds = GetEnvOr("MOONCAKE_OFFLOAD_CLIENT_BUFFER_GC_INTERVAL_SECONDS", - config.heartbeat_interval_seconds); + config.client_buffer_gc_interval_seconds); config.client_buffer_gc_ttl_ms = GetEnvOr("MOONCAKE_OFFLOAD_CLIENT_BUFFER_GC_TTL_MS", -- 2.34.1 From 1c3f1504a8708ee8119b9de1ce580692b80164cc Mon Sep 17 00:00:00 2001 From: Lewis <63569348+TTThanos@users.noreply.github.com> Date: Thu, 21 May 2026 14:12:22 +0800 Subject: [PATCH 104/382] [TE] Fix pytorch precision problem when using IntraNode NVLINK (#2163) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * IntraNode NVLink async transfer * Async intranode nvlink transfer * Solve pytorch and Memcpy race condition by introducing sync event to make sure pytorch data prepared before transfer --------- Co-authored-by: 百麒 --- .../intranode_nvlink_transport.cpp | 64 ++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-) diff --git a/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp b/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp index 01c3773a..6464bc20 100644 --- a/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp +++ b/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp @@ -45,7 +45,24 @@ struct CudaStreamNVLinkRAII { ~CudaStreamNVLinkRAII() { cudaStreamDestroy(stream_); } }; static thread_local CudaStreamNVLinkRAII tl_nvlink_stream; -} // namespace + +// Thread-local CUDA event used to synchronize the NVLink stream with +// the default (legacy) CUDA stream before issuing cudaMemcpyAsync. +// This prevents the NVLink stream from reading source data that +// PyTorch has not yet finished writing on the default stream. +struct CudaSyncEventRAII { + cudaEvent_t event_; + CudaSyncEventRAII() { + auto err = cudaEventCreateWithFlags(&event_, cudaEventDisableTiming); + if (err != cudaSuccess) { + LOG(FATAL) << "Failed to create NVLink sync CUDA event: " << err + << " - " << cudaGetErrorString(err); + } + } + ~CudaSyncEventRAII() { cudaEventDestroy(event_); } +}; +static thread_local CudaSyncEventRAII tl_nvlink_sync_event; +} // anonymous namespace static bool checkCudaErrorReturn(cudaError_t result, const char *message) { if (result != cudaSuccess) { @@ -191,6 +208,31 @@ Status IntraNodeNvlinkTransport::submitTransfer( size_t task_id = batch_desc.task_list.size(); batch_desc.task_list.resize(task_id + entries.size()); + // Synchronize with the caller's CUDA stream before issuing any memcpy. + // PyTorch uses cudaStreamPerThread (per-thread default stream), NOT the + // legacy default stream (nullptr). Recording the event on + // cudaStreamPerThread ensures that all previously submitted PyTorch + // operations on source/dest buffers have completed before cudaMemcpyAsync + // starts on tl_nvlink_stream. Using nullptr (legacy default stream) would + // miss PyTorch's work and could cause deadlocks with blocking streams. + cudaStream_t stream = tl_nvlink_stream.stream_; + cudaError_t sync_err = + cudaEventRecord(tl_nvlink_sync_event.event_, cudaStreamPerThread); + if (sync_err != cudaSuccess) { + LOG(ERROR) << "IntraNodeNvlinkTransport: cudaEventRecord on " + "cudaStreamPerThread failed: " + << cudaGetErrorString(sync_err); + return Status::Context("cudaEventRecord failed: " + + std::string(cudaGetErrorString(sync_err))); + } + sync_err = cudaStreamWaitEvent(stream, tl_nvlink_sync_event.event_, 0); + if (sync_err != cudaSuccess) { + LOG(ERROR) << "IntraNodeNvlinkTransport: cudaStreamWaitEvent failed: " + << cudaGetErrorString(sync_err); + return Status::Context("cudaStreamWaitEvent failed: " + + std::string(cudaGetErrorString(sync_err))); + } + for (auto &request : entries) { TransferTask &task = batch_desc.task_list[task_id]; ++task_id; @@ -276,6 +318,26 @@ Status IntraNodeNvlinkTransport::getTransferStatus(BatchID batch_id, Status IntraNodeNvlinkTransport::submitTransferTask( const std::vector &task_list) { + // Synchronize with the caller's CUDA stream before issuing any memcpy. + // See submitTransfer() for detailed rationale on using cudaStreamPerThread. + cudaStream_t stream = tl_nvlink_stream.stream_; + cudaError_t sync_err = + cudaEventRecord(tl_nvlink_sync_event.event_, cudaStreamPerThread); + if (sync_err != cudaSuccess) { + LOG(ERROR) << "IntraNodeNvlinkTransport: cudaEventRecord on " + "cudaStreamPerThread failed: " + << cudaGetErrorString(sync_err); + return Status::Context("cudaEventRecord failed: " + + std::string(cudaGetErrorString(sync_err))); + } + sync_err = cudaStreamWaitEvent(stream, tl_nvlink_sync_event.event_, 0); + if (sync_err != cudaSuccess) { + LOG(ERROR) << "IntraNodeNvlinkTransport: cudaStreamWaitEvent failed: " + << cudaGetErrorString(sync_err); + return Status::Context("cudaStreamWaitEvent failed: " + + std::string(cudaGetErrorString(sync_err))); + } + for (size_t index = 0; index < task_list.size(); ++index) { assert(task_list[index]); auto &task = *task_list[index]; -- 2.34.1 From 47f88aa71cfe592d2bbf6c409174b22a4b0c61c0 Mon Sep 17 00:00:00 2001 From: Xiao You Date: Thu, 21 May 2026 14:42:35 +0800 Subject: [PATCH 105/382] [Store] Fix Ascend dummy reconnect shm replay (#2158) Co-authored-by: youxiao --- mooncake-store/include/real_client.h | 3 ++ mooncake-store/src/dummy_client.cpp | 49 +++++++++++++++++++------ mooncake-store/src/real_client.cpp | 16 ++++++++ mooncake-store/src/real_client_main.cpp | 1 + 4 files changed, 58 insertions(+), 11 deletions(-) diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index c64d67da..02e40cac 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -484,6 +484,9 @@ class RealClient : public PyClient { tl::expected ascend_unmap_shm_internal( const UUID &client_id); + tl::expected is_shm_mapped_internal( + uint64_t dummy_base_addr, const UUID &client_id); + tl::expected unregister_shm_buffer_internal( uint64_t dummy_base_addr, const UUID &client_id); diff --git a/mooncake-store/src/dummy_client.cpp b/mooncake-store/src/dummy_client.cpp index 396e0c7a..fc5c9b9f 100644 --- a/mooncake-store/src/dummy_client.cpp +++ b/mooncake-store/src/dummy_client.cpp @@ -117,17 +117,30 @@ std::vector> void_ptr_rows_to_u64_nested( namespace mooncake { +template +constexpr bool can_invoke_when_disconnected() { + using Method = std::remove_reference_t; + return std::is_same_v< + Method, std::remove_reference_t> || + std::is_same_v> || + std::is_same_v> || + std::is_same_v> || + std::is_same_v>; +} + template tl::expected DummyClient::invoke_rpc(Args&&... args) { auto pool = client_accessor_.GetClientPool(); - if constexpr (!std::is_same_v< - std::remove_reference_t, - std::remove_reference_t> && - !std::is_same_v< - std::remove_reference_t, - std::remove_reference_t< - decltype(&RealClient::service_ready_internal)>>) { + if constexpr (!can_invoke_when_disconnected()) { if (!connected_.load()) { LOG(ERROR) << "Dummy Client not connected"; return tl::make_unexpected(ErrorCode::RPC_FAIL); @@ -266,6 +279,20 @@ ErrorCode DummyClient::connect(const std::string& server_address) { int DummyClient::register_ascend_shm(const ShmHelper::ShmSegment* shm, bool is_local) { #ifdef USE_ASCEND_DIRECT + const auto dummy_base_addr = reinterpret_cast(shm->base_addr); + auto mapped_result = invoke_rpc<&RealClient::is_shm_mapped_internal, bool>( + dummy_base_addr, client_id_); + if (!mapped_result.has_value()) { + LOG(WARNING) << "Failed to query real-side shared memory mapping, addr=" + << shm->base_addr; + return -1; + } + if (mapped_result.value()) { + LOG(INFO) << "Real-side shared memory mapping already exists, addr=" + << shm->base_addr << ", size=" << shm->size; + return 0; + } + // Detect memory type: device memory uses IPC sharing aclrtPtrAttributes attributes; auto ret = aclrtPointerGetAttributes(shm->base_addr, &attributes); @@ -288,8 +315,8 @@ int DummyClient::register_ascend_shm(const ShmHelper::ShmSegment* shm, std::string ipc_key_bytes(ipc_key, kIPCKeyLen); auto map_ret = invoke_rpc<&RealClient::ascend_ipc_shm_internal, void>( - reinterpret_cast(shm->base_addr), shm->size, is_local, - ipc_key_bytes, device_id_, client_id_); + dummy_base_addr, shm->size, is_local, ipc_key_bytes, device_id_, + client_id_); if (!map_ret.has_value()) { LOG(ERROR) << "Failed to map IPC buffer on real side"; return -1; @@ -331,8 +358,8 @@ int DummyClient::register_ascend_shm(const ShmHelper::ShmSegment* shm, std::string handle_bytes(reinterpret_cast(&export_handle), sizeof(export_handle)); auto map_ret = invoke_rpc<&RealClient::ascend_shm_internal, void>( - reinterpret_cast(shm->base_addr), shm->size, is_local, - handle_bytes, device_id_, client_id_); + dummy_base_addr, shm->size, is_local, handle_bytes, device_id_, + client_id_); if (!map_ret.has_value()) { LOG(ERROR) << "Failed to map VMM buffer on real side"; return -1; diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 4d7d3df3..5d5628c7 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -2408,6 +2408,22 @@ tl::expected RealClient::ascend_unmap_shm_internal( return {}; } +tl::expected RealClient::is_shm_mapped_internal( + uint64_t dummy_base_addr, const UUID &client_id) { + std::shared_lock lock(dummy_client_mutex_); + auto context_it = shm_contexts_.find(client_id); + if (context_it == shm_contexts_.end()) { + return false; + } + + const auto target_addr = static_cast(dummy_base_addr); + const auto &mapped_shms = context_it->second.mapped_shms; + return std::any_of(mapped_shms.begin(), mapped_shms.end(), + [target_addr](const MappedShm &shm) { + return shm.dummy_base_addr == target_addr; + }); +} + tl::expected RealClient::unregister_shm_buffer_internal( uint64_t dummy_base_addr, const UUID &client_id) { std::unique_lock lock(dummy_client_mutex_); diff --git a/mooncake-store/src/real_client_main.cpp b/mooncake-store/src/real_client_main.cpp index b5ab04bd..15c7202f 100644 --- a/mooncake-store/src/real_client_main.cpp +++ b/mooncake-store/src/real_client_main.cpp @@ -66,6 +66,7 @@ void RegisterClientRpcService(coro_rpc::coro_rpc_server &server, server.register_handler<&RealClient::ascend_ipc_shm_internal>(&real_client); server.register_handler<&RealClient::ascend_unmap_shm_internal>( &real_client); + server.register_handler<&RealClient::is_shm_mapped_internal>(&real_client); server.register_handler<&RealClient::unmap_shm_internal>(&real_client); server.register_handler<&RealClient::unregister_shm_buffer_internal>( &real_client); -- 2.34.1 From 684e5f0742a3c4741aeee84bac498bdef548fd61 Mon Sep 17 00:00:00 2001 From: Shangming Cai Date: Thu, 21 May 2026 14:51:27 +0800 Subject: [PATCH 106/382] Bump version to 0.3.11 in pyproject.toml (#2166) --- mooncake-wheel/pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mooncake-wheel/pyproject.toml b/mooncake-wheel/pyproject.toml index 90dcf32b..06f0c092 100644 --- a/mooncake-wheel/pyproject.toml +++ b/mooncake-wheel/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "mooncake-transfer-engine" -version = "0.3.10.post2" +version = "0.3.11" description = "Python binding of a Mooncake library using pybind11" authors = [ { name = "Mooncake Authors" } -- 2.34.1 From da70ac9f0c6fbef6c0c2c12da57507e6f031677b Mon Sep 17 00:00:00 2001 From: Yaoming Zhan Date: Thu, 21 May 2026 00:55:50 -0700 Subject: [PATCH 107/382] [Store] L2->L1 promotion-on-hit (#2071) --- mooncake-integration/store/store_py.cpp | 7 + mooncake-store/include/client_service.h | 51 +- mooncake-store/include/file_storage.h | 17 +- mooncake-store/include/master_client.h | 35 + mooncake-store/include/master_config.h | 27 + mooncake-store/include/master_service.h | 147 ++ mooncake-store/include/real_client.h | 7 + mooncake-store/include/rpc_service.h | 14 + mooncake-store/include/rpc_types.h | 9 + mooncake-store/include/segment.h | 6 + mooncake-store/src/client_service.cpp | 33 + mooncake-store/src/file_storage.cpp | 150 ++ mooncake-store/src/master.cpp | 56 + mooncake-store/src/master_client.cpp | 64 + mooncake-store/src/master_service.cpp | 540 +++++- mooncake-store/src/real_client.cpp | 1 + mooncake-store/src/rpc_service.cpp | 49 + mooncake-store/tests/CMakeLists.txt | 4 + .../tests/file_storage_promotion_test.cpp | 373 ++++ ...er_service_promotion_test_for_snapshot.cpp | 197 ++ .../tests/promotion_on_hit_test.cpp | 1620 +++++++++++++++++ mooncake-wheel/tests/test_promotion_on_hit.py | 492 +++++ scripts/run_tests.sh | 33 + 23 files changed, 3897 insertions(+), 35 deletions(-) create mode 100644 mooncake-store/tests/file_storage_promotion_test.cpp create mode 100644 mooncake-store/tests/ha/snapshot/master_service_promotion_test_for_snapshot.cpp create mode 100644 mooncake-store/tests/promotion_on_hit_test.cpp create mode 100644 mooncake-wheel/tests/test_promotion_on_hit.py diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index d5464d8c..ccb25d9d 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -1945,6 +1945,13 @@ PYBIND11_MODULE(store, m) { }) .def("get", &mooncake::MooncakeStorePyWrapper::get) .def("get_batch", &mooncake::MooncakeStorePyWrapper::get_batch) + .def("get_offload_rpc_read_count", + [](MooncakeStorePyWrapper &self) -> int64_t { + auto real_client = + std::dynamic_pointer_cast(self.store_); + return real_client ? real_client->get_offload_rpc_read_count() + : 0; + }) .def( "get_buffer", [](MooncakeStorePyWrapper &self, const std::string &key) { diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index ac848d1b..cef371c4 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -62,7 +62,7 @@ class QueryResult { */ class Client { public: - ~Client(); + virtual ~Client(); const UUID& getClientId() const { return client_id_; } @@ -410,6 +410,49 @@ class Client { tl::expected ReportSsdCapacity( int64_t ssd_total_capacity_bytes); + /** + * @brief Heartbeat-driven pull of pending L2->L1 promotion work for this + * client. Mirror of OffloadObjectHeartbeat. Returns key->size pairs the + * caller (FileStorage) must read from local SSD and stage as MEMORY + * replicas via PromotionAllocStart + NotifyPromotionSuccess. + */ + // Virtual to enable subclassing in unit tests. + virtual tl::expected PromotionObjectHeartbeat( + std::unordered_map& promotion_objects); + + /** + * @brief Stage a PROCESSING MEMORY replica for an existing key during + * L2->L1 promotion. Returns the new replica's descriptor that the caller + * writes via Transfer Engine before calling NotifyPromotionSuccess. + */ + virtual tl::expected + PromotionAllocStart(const std::string& key, uint64_t size, + const std::vector& preferred_segments); + + /** + * @brief Commit a staged MEMORY replica to COMPLETE; called after the + * client has written the bytes via Transfer Engine. + */ + virtual tl::expected NotifyPromotionSuccess( + const std::string& key); + + /** + * @brief Release master-side promotion task after a client-side failure + * between PromotionAllocStart and the transfer's completion. Idempotent. + */ + virtual tl::expected NotifyPromotionFailure( + const std::string& key); + + /** + * @brief Write `slices` into the memory replica described by + * `memory_descriptor` via Transfer Engine. Used by FileStorage to fill a + * PROCESSING memory replica staged by PromotionAllocStart before calling + * NotifyPromotionSuccess. + */ + virtual ErrorCode PromotionWrite( + const Replica::Descriptor& memory_descriptor, + std::vector& slices); + /** * @brief Performs a batched read of multiple objects using a * high-throughput Transfer Engine. @@ -577,14 +620,16 @@ class Client { bool IsReplicaOnLocalMemory(const Replica::Descriptor& replica); - private: + protected: /** - * @brief Private constructor to enforce creation through Create() method + * @brief Constructor exposed to subclasses for testing only; production + * code must go through Create(). */ Client(const std::string& local_hostname, const std::string& metadata_connstring, const std::string& protocol, const std::map& labels = {}); + private: /** * @brief Internal helper functions for initialization and data transfer */ diff --git a/mooncake-store/include/file_storage.h b/mooncake-store/include/file_storage.h index eef6741d..37e24102 100644 --- a/mooncake-store/include/file_storage.h +++ b/mooncake-store/include/file_storage.h @@ -51,6 +51,7 @@ class FileStorage { private: friend class FileStorageTest; + friend class FileStoragePromotionTest; struct AllocatedBatch { uint64_t batch_id; std::vector handles; @@ -82,10 +83,24 @@ class FileStorage { * client. * 2. Receives feedback on which objects should be offloaded. * 3. Triggers asynchronous offloading of pending objects. + * 4. Pulls and processes any pending L2->L1 promotion tasks queued by the + * master (mirror of step 1+2 in the reverse direction). * @return tl::expected indicating operation status. */ tl::expected Heartbeat(); + /** + * @brief Drives the L2->L1 promotion pipeline for one heartbeat tick. + * Pulls promotion work from the master, stages a MEMORY replica for each + * key, copies the bytes from local SSD into that replica, and notifies the + * master on success. A failure on any single key is logged and skipped; + * the master-side reaper decrements the source replica's refcnt and + * erases the task entry on TTL expiry, and any orphaned PROCESSING + * MEMORY replica is reaped via the standard discarded-replicas path. + * @return tl::expected indicating operation status. + */ + tl::expected ProcessPromotionTasks(); + tl::expected IsEnableOffloading(); tl::expected BatchLoad( @@ -123,4 +138,4 @@ class FileStorage { std::thread client_buffer_gc_thread_; }; -} // namespace mooncake \ No newline at end of file +} // namespace mooncake diff --git a/mooncake-store/include/master_client.h b/mooncake-store/include/master_client.h index b4402dd9..038b1618 100644 --- a/mooncake-store/include/master_client.h +++ b/mooncake-store/include/master_client.h @@ -419,6 +419,41 @@ class MasterClient { const UUID& client_id, const std::vector& keys, const std::vector& metadatas); + /** + * @brief Heartbeat-driven pull of pending L2->L1 promotion work for a + * client. Returns key->size pairs the caller should read from local + * SSD and stage as MEMORY replicas via PromotionAllocStart + + * NotifyPromotionSuccess. + */ + [[nodiscard]] tl::expected, + ErrorCode> + PromotionObjectHeartbeat(const UUID& client_id); + + /** + * @brief Stage a PROCESSING MEMORY replica for an existing key during + * promotion. Returns the new replica's descriptor that the caller writes + * via Transfer Engine. + */ + [[nodiscard]] tl::expected + PromotionAllocStart(const UUID& client_id, const std::string& key, + uint64_t size, + const std::vector& preferred_segments); + + /** + * @brief Release master-side promotion task state after a client-side + * failure that prevents the holder from calling NotifyPromotionSuccess. + * Idempotent; returns OK if the task was already swept by the reaper. + */ + [[nodiscard]] tl::expected NotifyPromotionFailure( + const UUID& client_id, const std::string& key); + + /** + * @brief Commit a staged MEMORY replica to COMPLETE; called after the + * client has written the bytes via Transfer Engine. + */ + [[nodiscard]] tl::expected NotifyPromotionSuccess( + const UUID& client_id, const std::string& key); + /** * @brief Start a copy operation * @param key Object key diff --git a/mooncake-store/include/master_config.h b/mooncake-store/include/master_config.h index 6ef40239..1f1bc5a9 100644 --- a/mooncake-store/include/master_config.h +++ b/mooncake-store/include/master_config.h @@ -103,6 +103,12 @@ struct MasterConfig { // Offload-on-evict: defer LOCAL_DISK offload to eviction time bool offload_on_evict = false; bool offload_force_evict = false; + + // Promotion-on-hit: when Get observes a LOCAL_DISK-only key, queue an + // async copy back to MEMORY so the next Get is fast. + bool promotion_on_hit = false; + uint32_t promotion_admission_threshold = 2; + uint32_t promotion_queue_limit = 50000; }; class MasterServiceSupervisorConfig { @@ -173,6 +179,9 @@ class MasterServiceSupervisorConfig { bool enable_cxl = false; bool offload_on_evict = false; bool offload_force_evict = false; + bool promotion_on_hit = false; + uint32_t promotion_admission_threshold = 2; + uint32_t promotion_queue_limit = 50000; MasterServiceSupervisorConfig() = default; // From MasterConfig @@ -197,6 +206,9 @@ class MasterServiceSupervisorConfig { enable_offload = config.enable_offload; offload_on_evict = config.offload_on_evict; offload_force_evict = config.offload_force_evict; + promotion_on_hit = config.promotion_on_hit; + promotion_admission_threshold = config.promotion_admission_threshold; + promotion_queue_limit = config.promotion_queue_limit; rpc_port = static_cast(config.rpc_port); rpc_thread_num = static_cast(config.rpc_thread_num); @@ -335,6 +347,9 @@ class WrappedMasterServiceConfig { bool enable_offload = false; bool offload_on_evict = false; bool offload_force_evict = false; + bool promotion_on_hit = false; + uint32_t promotion_admission_threshold = 2; + uint32_t promotion_queue_limit = 50000; std::string ha_backend_type = "etcd"; std::string ha_backend_connstring; std::string cluster_id = DEFAULT_CLUSTER_ID; @@ -399,6 +414,9 @@ class WrappedMasterServiceConfig { enable_offload = config.enable_offload; offload_on_evict = config.offload_on_evict; offload_force_evict = config.offload_force_evict; + promotion_on_hit = config.promotion_on_hit; + promotion_admission_threshold = config.promotion_admission_threshold; + promotion_queue_limit = config.promotion_queue_limit; ha_backend_type = config.ha_backend_type; ha_backend_connstring = ResolveConfiguredHABackendConnstring( ha_backend_type, config.ha_backend_connstring, @@ -481,6 +499,9 @@ class WrappedMasterServiceConfig { enable_offload = config.enable_offload; offload_on_evict = config.offload_on_evict; offload_force_evict = config.offload_force_evict; + promotion_on_hit = config.promotion_on_hit; + promotion_admission_threshold = config.promotion_admission_threshold; + promotion_queue_limit = config.promotion_queue_limit; ha_backend_type = config.ha_backend_type; ha_backend_connstring = ResolveConfiguredHABackendConnstring( ha_backend_type, config.ha_backend_connstring, @@ -862,6 +883,9 @@ class MasterServiceConfig { bool enable_offload = false; bool offload_on_evict = false; bool offload_force_evict = false; + bool promotion_on_hit = false; + uint32_t promotion_admission_threshold = 2; + uint32_t promotion_queue_limit = 50000; std::string ha_backend_type = "etcd"; std::string ha_backend_connstring; std::string cluster_id = DEFAULT_CLUSTER_ID; @@ -922,6 +946,9 @@ class MasterServiceConfig { enable_offload = config.enable_offload; offload_on_evict = config.offload_on_evict; offload_force_evict = config.offload_force_evict; + promotion_on_hit = config.promotion_on_hit; + promotion_admission_threshold = config.promotion_admission_threshold; + promotion_queue_limit = config.promotion_queue_limit; ha_backend_type = config.ha_backend_type; ha_backend_connstring = config.ha_backend_connstring; cluster_id = config.cluster_id; diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index 6c7bf581..68b841f3 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -20,6 +20,7 @@ #include #include "allocation_strategy.h" +#include "count_min_sketch.h" #include "master_metric_manager.h" #include "mutex.h" #include "segment.h" @@ -46,6 +47,12 @@ class EvictionStrategy; namespace test { class MasterServiceSnapshotTestBase; class SnapshotChildProcessTest; +// Friended so the promotion-on-hit tests can drive a serialize/reset/ +// deserialize cycle directly via the otherwise-private +// MetadataSerializer, and inspect private clamp fields. This avoids +// standing up a full snapshot catalog + child-process harness, and +// exposing test-only accessors on MasterService itself. +class PromotionOnHitTest; } // namespace test /* @@ -59,6 +66,7 @@ class MasterService { // Test friend class for snapshot/restore testing friend class test::MasterServiceSnapshotTestBase; friend class test::SnapshotChildProcessTest; + friend class test::PromotionOnHitTest; public: using NoFProbeFn = @@ -526,6 +534,62 @@ class MasterService { const std::vector& metadatas) -> tl::expected; + /** + * @brief Heartbeat-driven pull of pending promotion work for a client. + * Returns the per-client promotion_objects map (key -> object size) and + * clears it. The per-shard promotion_tasks map remains populated as the + * source of truth until NotifyPromotionSuccess commits the new MEMORY + * replica. + */ + auto PromotionObjectHeartbeat(const UUID& client_id) + -> tl::expected, ErrorCode>; + + /** + * @brief Stage a PROCESSING MEMORY replica for an existing key. Allocates + * DRAM via the existing AllocationStrategy, optionally biased toward the + * caller's local memory segment via preferred_segments. The new replica is + * invisible to readers until NotifyPromotionSuccess flips it to COMPLETE. + * + * Only the holder client (the one owning the source LOCAL_DISK replica) + * is authorized to call this. Other clients receive INVALID_PARAMS. + * `size` must match the source replica's object_size captured at task + * admission; mismatch returns INVALID_PARAMS to avoid allocating an + * arbitrary buffer size from a buggy or malicious caller. + */ + auto PromotionAllocStart(const UUID& client_id, const std::string& key, + uint64_t size, + const std::vector& preferred_segments) + -> tl::expected; + + /** + * @brief Commit a staged MEMORY replica to COMPLETE; decrement source + * refcnt; erase per-shard and per-client task entries. Mirror of + * NotifyOffloadSuccess. + */ + auto NotifyPromotionSuccess(const UUID& client_id, const std::string& key) + -> tl::expected; + + /** + * @brief Holder-side failure notification: the client got past + * PromotionAllocStart but a downstream step (local SSD read, RDMA + * write, etc.) failed and it will not be calling + * NotifyPromotionSuccess. Releases the master-side task state + * immediately rather than waiting put_start_release_timeout_sec_ + * for the reaper to do it. Without this call every transient + * client-side error (SSD throttling, RDMA flake, etc.) pins a + * task slot and a staged DRAM buffer for the full reaper TTL, + * which can saturate promotion_queue_limit_ on busy clusters. + * + * Authorization is the same as NotifyPromotionSuccess: only the + * holder client may release a task. Effects mirror the reaper's + * expiry path: drop source LOCAL_DISK refcnt, pop the staged + * PROCESSING MEMORY replica if alloc_id was recorded, erase the + * task, decrement the global in-flight counter, and clear the + * holder's promotion_objects entry. + */ + auto NotifyPromotionFailure(const UUID& client_id, const std::string& key) + -> tl::expected; + /** * @brief Create a copy task to copy an object's replicas to target segments * @return Copy task ID on success, ErrorCode on failure @@ -955,6 +1019,36 @@ class MasterService { std::chrono::system_clock::time_point start_time; }; + // Tracks an in-flight LOCAL_DISK -> MEMORY copy. The source + // LOCAL_DISK replica is refcnt-pinned for the duration of the task + // so it cannot be evicted. + // + // alloc_id pins down which staged PROCESSING MEMORY replica + // NotifyPromotionSuccess should commit, so a concurrent Put on the + // same key cannot be confused with ours. 0 until + // PromotionAllocStart records the new replica. + // + // start_time is the reaper deadline anchor. Set at task admission + // and reset at PromotionAllocStart so each phase (queue-wait and + // active-transfer) gets its own full put_start_release_timeout_sec_ + // window. Without the reset a backlogged task could enter active + // transfer with little TTL left, and the reaper could free the + // staged replica via EraseReplicaByID mid-RDMA-write. + // + // holder_id is the client owning the source LOCAL_DISK segment and + // the only one authorized to commit (NotifyPromotionSuccess) or + // abort (NotifyPromotionFailure) the task. Without it, any client + // knowing the key could flip the staged PROCESSING replica to + // COMPLETE before the holder's RDMA write landed, exposing torn + // data to readers. + struct PromotionTask { + ReplicaID source_id; // the LOCAL_DISK replica being promoted + ReplicaID alloc_id{0}; // the new MEMORY replica staged by AllocStart + uint64_t object_size; + std::chrono::system_clock::time_point start_time; + UUID holder_id; // owner of source LOCAL_DISK; only Notifier allowed + }; + static constexpr size_t kNumShards = 1024; // Number of metadata shards // Sharded metadata maps and their mutexes @@ -967,6 +1061,8 @@ class MasterService { GUARDED_BY(mutex); std::unordered_map offloading_tasks GUARDED_BY(mutex); + std::unordered_map promotion_tasks + GUARDED_BY(mutex); }; std::array metadata_shards_; @@ -1078,6 +1174,38 @@ class MasterService { std::condition_variable timer_cv_; } graceful_unmount_scheduler_; + /** + * @brief Mirror of PushOffloadingQueue for promotion-on-hit. Inserts an + * entry into the holder client's LocalDiskSegment::promotion_objects map. + * Caller is responsible for refcnt-pinning the source replica and + * recording the task in the shard's promotion_tasks map. + */ + tl::expected PushPromotionQueue(const std::string& key, + Replica& source_replica); + + /** + * @brief Helper invoked from GetReplicaList when an only-LOCAL_DISK key is + * observed. Applies the gating chain (frequency / watermark / dedup / + * cap), refcnt-pins the source LOCAL_DISK replica, records a + * PromotionTask, and pushes onto the holder client's promotion_objects + * map. Acquires its own RW shard accessor; safe to call after + * GetReplicaList's RO accessor has been released. + */ + void TryPushPromotionQueue(const std::string& key); + + // Erase any in-flight PromotionTask for `key` and decrement the + // cluster-wide in-flight counter. Safe no-op if no task exists. + // Call from any path that erases an ObjectMetadata entry, so the + // task doesn't pin a promotion_in_flight_ slot for the full + // put_start_release_timeout_sec_ until the reaper sweeps. + void ErasePromotionTaskIfPresent(MetadataShardAccessorRW& shard, + const std::string& key) + NO_THREAD_SAFETY_ANALYSIS { + if (shard->promotion_tasks.erase(key) > 0) { + promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + } + } + // Lease related members const uint64_t default_kv_lease_ttl_; // in milliseconds const uint64_t default_kv_soft_pin_ttl_; // in milliseconds @@ -1140,6 +1268,7 @@ class MasterService { if (processing_it_ != shard_guard_->processing_keys.end()) { this->EraseFromProcessing(); } + service_->ErasePromotionTaskIfPresent(shard_guard_, key_); } } } @@ -1349,6 +1478,24 @@ class MasterService { // offload_on_evict_=true) bool offload_force_evict_{false}; + // Promotion-on-hit: opt-in flag enabling LOCAL_DISK -> MEMORY promotion + // when a Get observes a key with only LOCAL_DISK replicas. + bool promotion_on_hit_{false}; + uint32_t promotion_admission_threshold_{2}; + uint32_t promotion_queue_limit_{50000}; + // Global in-flight task counter, checked against promotion_queue_limit_ + // as the gate cap. Promotion specifically targets skewed + // access (hot keys re-accessed after eviction), so the global counter + // is the correct primitive. Incremented in TryPushPromotionQueue after + // successful enqueue; decremented in NotifyPromotionSuccess and in the + // promotion task reaper after the task entry is erased. Relaxed memory + // order is safe — the value is an advisory soft cap, not a barrier. + std::atomic promotion_in_flight_{0}; + // Master-side frequency sketch. Constructed only when promotion_on_hit_ is + // true. CountMinSketch is mutex-protected internally so we can call into it + // from any GetReplicaList caller without additional locking. + std::unique_ptr promotion_sketch_; + const std::string ha_backend_type_; const std::string ha_backend_connstring_; diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index 02e40cac..bcf9b674 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -686,6 +686,10 @@ class RealClient : public PyClient { const std::string &target_rpc_service_addr, std::unordered_map> &objects); + int64_t get_offload_rpc_read_count() const { + return offload_rpc_read_count_.load(std::memory_order_relaxed); + } + /** * @brief Mount a shared memory file region and return segment ids. * If size > max_mr_size, it will be split into multiple chunks @@ -833,6 +837,9 @@ class RealClient : public PyClient { // Ensure cleanup executes at most once across multiple entry points std::atomic closed_{false}; + // Counts every LOCAL_DISK read served via peer offload-RPC. + std::atomic offload_rpc_read_count_{0}; + // Dummy Client manage related members void dummy_client_monitor_func(); int start_dummy_client_monitor(); diff --git a/mooncake-store/include/rpc_service.h b/mooncake-store/include/rpc_service.h index 286ed3fb..2228d381 100644 --- a/mooncake-store/include/rpc_service.h +++ b/mooncake-store/include/rpc_service.h @@ -174,6 +174,20 @@ class WrappedMasterService { const UUID& client_id, const std::vector& keys, const std::vector& metadatas); + // Promotion-on-hit RPCs. + tl::expected, ErrorCode> + PromotionObjectHeartbeat(const UUID& client_id); + + tl::expected PromotionAllocStart( + const UUID& client_id, const std::string& key, uint64_t size, + const std::vector& preferred_segments); + + tl::expected NotifyPromotionSuccess( + const UUID& client_id, const std::string& key); + + tl::expected NotifyPromotionFailure( + const UUID& client_id, const std::string& key); + tl::expected CreateDrainJob( const CreateDrainJobRequest& request); diff --git a/mooncake-store/include/rpc_types.h b/mooncake-store/include/rpc_types.h index 57fdf2d5..a6880193 100644 --- a/mooncake-store/include/rpc_types.h +++ b/mooncake-store/include/rpc_types.h @@ -78,6 +78,15 @@ struct CopyStartResponse { }; YLT_REFL(CopyStartResponse, source, targets); +/** + * @brief Response structure for PromotionAllocStart (L2->L1 promotion-on-hit). + * Carries the staged PROCESSING MEMORY replica descriptor. + */ +struct PromotionAllocStartResponse { + Replica::Descriptor memory_descriptor; +}; +YLT_REFL(PromotionAllocStartResponse, memory_descriptor); + /** * @brief Response structure for MoveStart operation */ diff --git a/mooncake-store/include/segment.h b/mooncake-store/include/segment.h index 426c512e..8fcd2f87 100644 --- a/mooncake-store/include/segment.h +++ b/mooncake-store/include/segment.h @@ -88,6 +88,12 @@ struct LocalDiskSegment { int64_t ssd_total_capacity_bytes = 0; // last reported by client heartbeat std::unordered_map GUARDED_BY(offloading_mutex_) offloading_objects; + // Promotion-on-hit pending work for this client. Populated by master's + // TryPushPromotionQueue when a Get hits a LOCAL_DISK-only key on this + // client. Drained by PromotionObjectHeartbeat. Same locking as + // offloading_objects (offloading_mutex_). + std::unordered_map GUARDED_BY(offloading_mutex_) + promotion_objects; explicit LocalDiskSegment(bool enable_offloading) : enable_offloading(enable_offloading) {} diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index aead7e03..ee2c1334 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -2483,6 +2483,39 @@ tl::expected Client::NotifyOffloadSuccess( return response; } +tl::expected Client::PromotionObjectHeartbeat( + std::unordered_map& promotion_objects) { + auto response = master_client_.PromotionObjectHeartbeat(client_id_); + if (!response) { + return tl::make_unexpected(response.error()); + } + promotion_objects = std::move(response.value()); + return {}; +} + +tl::expected +Client::PromotionAllocStart( + const std::string& key, uint64_t size, + const std::vector& preferred_segments) { + return master_client_.PromotionAllocStart(client_id_, key, size, + preferred_segments); +} + +tl::expected Client::NotifyPromotionSuccess( + const std::string& key) { + return master_client_.NotifyPromotionSuccess(client_id_, key); +} + +tl::expected Client::NotifyPromotionFailure( + const std::string& key) { + return master_client_.NotifyPromotionFailure(client_id_, key); +} + +ErrorCode Client::PromotionWrite(const Replica::Descriptor& memory_descriptor, + std::vector& slices) { + return TransferWrite(memory_descriptor, slices); +} + tl::expected Client::CreateCopyTask( const std::string& key, const std::vector& targets) { return master_client_.CreateCopyTask(key, targets); diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index a445aa9b..cf7bed9a 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -521,11 +521,161 @@ tl::expected FileStorage::Heartbeat() { return offload_result; } + // Drive any pending L2->L1 promotion work for this client. Failures + // inside ProcessPromotionTasks are logged per-key and do not propagate; + // promotion is best-effort and must never break offload. + (void)ProcessPromotionTasks(); + // TODO(eviction): Implement an LRU eviction mechanism to manage local // storage capacity. return {}; } +tl::expected FileStorage::ProcessPromotionTasks() { + if (client_ == nullptr) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + std::unordered_map promotion_objects; + auto heartbeat_result = + client_->PromotionObjectHeartbeat(promotion_objects); + if (!heartbeat_result) { + // SEGMENT_NOT_FOUND happens between MountLocalDiskSegment and the + // first heartbeat tick if the master forgets us (e.g. across a master + // restart): benign no-op until next ReMount. + if (heartbeat_result.error() == ErrorCode::SEGMENT_NOT_FOUND) { + return {}; + } + LOG(WARNING) << "PromotionObjectHeartbeat failed: " + << heartbeat_result.error(); + return tl::make_unexpected(heartbeat_result.error()); + } + if (promotion_objects.empty()) { + return {}; + } + + VLOG(1) << "ProcessPromotionTasks pulled " << promotion_objects.size() + << " promotion candidate(s) from master"; + + // No segment preference from the client: let master pick from any + // DRAM segment. + const std::vector preferred_segments; + + // The master caps per-heartbeat work via PromotionObjectHeartbeat, + // returning at most one task per call so the heartbeat thread stays + // within the client-liveness window even for large objects. Leftover + // work stays queued in the master's promotion_objects map and is + // returned on subsequent heartbeats; we process whatever we received + // here without a second client-side cap. + for (const auto& [key, size] : promotion_objects) { + if (size <= 0) { + LOG(WARNING) << "Skipping promotion for key=" << key + << " with non-positive size=" << size; + continue; + } + + auto alloc_result = client_->PromotionAllocStart( + key, static_cast(size), preferred_segments); + if (!alloc_result) { + // AllocStart failed (typically NO_AVAILABLE_HANDLE under + // DRAM pressure). No staged buffer to release, but the + // task entry already claimed a promotion_in_flight_ slot + // at admission. Notify the master to release it + // immediately; otherwise the slot stays pinned for the + // reaper TTL (~10 min default), turning transient DRAM + // pressure into a sustained outage of promotion_queue_limit_. + // Notify is idempotent and handles alloc_id == 0 correctly. + VLOG(1) << "PromotionAllocStart failed for key=" << key + << ", error=" << alloc_result.error() + << " (likely no free DRAM); releasing master slot"; + auto release = client_->NotifyPromotionFailure(key); + if (!release) { + VLOG(1) << "Promotion: NotifyPromotionFailure failed for key=" + << key << ", error=" << release.error() + << "; master reaper will reclaim on TTL expiry"; + } + continue; + } + + // Every failure path past this point has a master-side staged + // PROCESSING MEMORY buffer and an incremented in-flight slot. + // Eagerly notify the master on failure so the buffer is + // reclaimed and the slot is freed; otherwise transient SSD + // throttling or RDMA flakes saturate promotion_queue_limit_ + // for the full reaper TTL. NotifyPromotionFailure is + // idempotent and best-effort — the reaper is the long-stop. + auto release_master_state = [this, &key]() { + auto release = client_->NotifyPromotionFailure(key); + if (!release) { + VLOG(1) << "Promotion: NotifyPromotionFailure failed for key=" + << key << ", error=" << release.error() + << "; master reaper will reclaim on TTL expiry"; + } + }; + + // (a) Allocate an O_DIRECT-aligned staging buffer and read the bytes + // from the local SSD backend into it. AllocateBatch returns a + // shared_ptr whose BufferHandles RAII-release the + // staging space when the local goes out of scope. + std::vector single_key{key}; + std::vector single_size{size}; + auto allocate_res = AllocateBatch(single_key, single_size); + if (!allocate_res) { + LOG(WARNING) << "Promotion: AllocateBatch failed for key=" << key + << ", error=" << allocate_res.error(); + release_master_state(); + continue; + } + auto staging = allocate_res.value(); + auto load_res = BatchLoad(staging->slices); + if (!load_res) { + LOG(WARNING) << "Promotion: BatchLoad failed for key=" << key + << ", error=" << load_res.error(); + release_master_state(); + continue; + } + + // (b) TE-write from the staging slice into the freshly-allocated + // MEMORY replica. Slice ptr may have been bumped by O_DIRECT offset + // correction in BatchLoad, so re-read it from the slice map. + auto slice_it = staging->slices.find(key); + if (slice_it == staging->slices.end()) { + LOG(WARNING) << "Promotion: staging slice missing for key=" << key; + release_master_state(); + continue; + } + std::vector tx_slices{slice_it->second}; + ErrorCode write_err = client_->PromotionWrite( + alloc_result.value().memory_descriptor, tx_slices); + if (write_err != ErrorCode::OK) { + LOG(WARNING) << "Promotion: TransferWrite failed for key=" << key + << ", error=" << write_err; + release_master_state(); + continue; + } + + // (c) Commit. Master flips the PROCESSING replica to COMPLETE and it + // becomes visible to readers. + auto notify_res = client_->NotifyPromotionSuccess(key); + if (!notify_res) { + // The write landed but the commit failed. We can't retry the + // commit (the success path is one-shot via alloc_id), and we + // don't know whether the failure was transient or structural. + // Release the master-side state so the slot is reusable; the + // bytes we wrote become stranded under a soon-to-be-erased + // PROCESSING replica, which is harmless. + LOG(WARNING) << "Promotion: NotifyPromotionSuccess failed for key=" + << key << ", error=" << notify_res.error(); + release_master_state(); + continue; + } + + VLOG(1) << "Promotion completed for key=" << key << ", size=" << size; + } + + return {}; +} + tl::expected FileStorage::BatchLoad( std::unordered_map& batch_object) { auto start_time = std::chrono::steady_clock::now(); diff --git a/mooncake-store/src/master.cpp b/mooncake-store/src/master.cpp index 2f213c78..8c559c07 100644 --- a/mooncake-store/src/master.cpp +++ b/mooncake-store/src/master.cpp @@ -126,6 +126,14 @@ DEFINE_bool(offload_on_evict, false, "Defer LOCAL_DISK offload to eviction time instead of PutEnd"); DEFINE_bool(offload_force_evict, false, "Force-evict objects exceeding offload cap without disk offload"); +DEFINE_bool(promotion_on_hit, false, + "Promote LOCAL_DISK-only keys to MEMORY on read access (mirror of " + "offload_on_evict)"); +DEFINE_uint32(promotion_admission_threshold, 2, + "Min CountMinSketch count for a key before promotion fires " + "(set 1 to disable second-touch gating)"); +DEFINE_uint32(promotion_queue_limit, 50000, + "Max in-flight promotion tasks across all shards"); DEFINE_string(ha_backend_type, "etcd", "HA backend type, e.g. etcd | redis | k8s"); DEFINE_string(ha_backend_connstring, "", @@ -342,6 +350,14 @@ void InitMasterConf(const mooncake::DefaultConfig& default_config, default_config.GetBool("offload_force_evict", &master_config.offload_force_evict, FLAGS_offload_force_evict); + default_config.GetBool("promotion_on_hit", &master_config.promotion_on_hit, + FLAGS_promotion_on_hit); + default_config.GetUInt32("promotion_admission_threshold", + &master_config.promotion_admission_threshold, + FLAGS_promotion_admission_threshold); + default_config.GetUInt32("promotion_queue_limit", + &master_config.promotion_queue_limit, + FLAGS_promotion_queue_limit); default_config.GetString("ha_backend_type", &master_config.ha_backend_type, FLAGS_ha_backend_type); default_config.GetString("ha_backend_connstring", @@ -595,6 +611,46 @@ void LoadConfigFromCmdline(mooncake::MasterConfig& master_config, !conf_set) { master_config.enable_offload = FLAGS_enable_offload; } + if ((google::GetCommandLineFlagInfo("offload_on_evict", &info) && + !info.is_default) || + !conf_set) { + master_config.offload_on_evict = FLAGS_offload_on_evict; + } + if ((google::GetCommandLineFlagInfo("offload_force_evict", &info) && + !info.is_default) || + !conf_set) { + master_config.offload_force_evict = FLAGS_offload_force_evict; + } + if ((google::GetCommandLineFlagInfo("promotion_on_hit", &info) && + !info.is_default) || + !conf_set) { + master_config.promotion_on_hit = FLAGS_promotion_on_hit; + } + if ((google::GetCommandLineFlagInfo("promotion_admission_threshold", + &info) && + !info.is_default) || + !conf_set) { + master_config.promotion_admission_threshold = + FLAGS_promotion_admission_threshold; + } + if ((google::GetCommandLineFlagInfo("promotion_queue_limit", &info) && + !info.is_default) || + !conf_set) { + master_config.promotion_queue_limit = FLAGS_promotion_queue_limit; + } + // Clamp promotion_admission_threshold into the sketch counter's + // representable range. The CountMinSketch uses 8-bit saturating + // counters (max 255) so any threshold beyond that would silently + // make the gate unreachable; clamping at parse time fails loudly and + // gives the gate a stable contract to compare uint8_t against. + if (master_config.promotion_admission_threshold > 255) { + LOG(WARNING) << "promotion_admission_threshold=" + << master_config.promotion_admission_threshold + << " exceeds the CountMinSketch counter max (255). " + << "Clamping to 255. Lower the configured value to " + << "silence this warning."; + master_config.promotion_admission_threshold = 255; + } if ((google::GetCommandLineFlagInfo("ha_backend_type", &info) && !info.is_default) || !conf_set) { diff --git a/mooncake-store/src/master_client.cpp b/mooncake-store/src/master_client.cpp index 7d899d3e..d22a697c 100644 --- a/mooncake-store/src/master_client.cpp +++ b/mooncake-store/src/master_client.cpp @@ -232,6 +232,26 @@ struct RpcNameTraits<&WrappedMasterService::NotifyOffloadSuccess> { static constexpr const char* value = "NotifyOffloadSuccess"; }; +template <> +struct RpcNameTraits<&WrappedMasterService::PromotionObjectHeartbeat> { + static constexpr const char* value = "PromotionObjectHeartbeat"; +}; + +template <> +struct RpcNameTraits<&WrappedMasterService::PromotionAllocStart> { + static constexpr const char* value = "PromotionAllocStart"; +}; + +template <> +struct RpcNameTraits<&WrappedMasterService::NotifyPromotionSuccess> { + static constexpr const char* value = "NotifyPromotionSuccess"; +}; + +template <> +struct RpcNameTraits<&WrappedMasterService::NotifyPromotionFailure> { + static constexpr const char* value = "NotifyPromotionFailure"; +}; + template <> struct RpcNameTraits<&WrappedMasterService::CopyStart> { static constexpr const char* value = "CopyStart"; @@ -953,6 +973,50 @@ tl::expected MasterClient::NotifyOffloadSuccess( return result; } +tl::expected, ErrorCode> +MasterClient::PromotionObjectHeartbeat(const UUID& client_id) { + ScopedVLogTimer timer(1, "MasterClient::PromotionObjectHeartbeat"); + timer.LogRequest("client_id=", client_id); + return invoke_rpc<&WrappedMasterService::PromotionObjectHeartbeat, + std::unordered_map>(client_id); +} + +tl::expected +MasterClient::PromotionAllocStart( + const UUID& client_id, const std::string& key, uint64_t size, + const std::vector& preferred_segments) { + ScopedVLogTimer timer(1, "MasterClient::PromotionAllocStart"); + timer.LogRequest("client_id=", client_id, ", key=", key, ", size=", size, + ", preferred_count=", preferred_segments.size()); + auto result = invoke_rpc<&WrappedMasterService::PromotionAllocStart, + PromotionAllocStartResponse>(client_id, key, size, + preferred_segments); + timer.LogResponseExpected(result); + return result; +} + +tl::expected MasterClient::NotifyPromotionSuccess( + const UUID& client_id, const std::string& key) { + ScopedVLogTimer timer(1, "MasterClient::NotifyPromotionSuccess"); + timer.LogRequest("client_id=", client_id, ", key=", key); + auto result = + invoke_rpc<&WrappedMasterService::NotifyPromotionSuccess, void>( + client_id, key); + timer.LogResponseExpected(result); + return result; +} + +tl::expected MasterClient::NotifyPromotionFailure( + const UUID& client_id, const std::string& key) { + ScopedVLogTimer timer(1, "MasterClient::NotifyPromotionFailure"); + timer.LogRequest("client_id=", client_id, ", key=", key); + auto result = + invoke_rpc<&WrappedMasterService::NotifyPromotionFailure, void>( + client_id, key); + timer.LogResponseExpected(result); + return result; +} + tl::expected MasterClient::CopyStart( const std::string& key, const std::string& src_segment, const std::vector& tgt_segments) { diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 0fa595c7..00604a8c 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -238,6 +238,37 @@ MasterService::MasterService(const MasterServiceConfig& config) } } + // Promotion-on-hit: when Get observes a LOCAL_DISK-only key, queue an + // async copy back to MEMORY. Only meaningful when offload is enabled + // (otherwise no LOCAL_DISK replicas exist in the first place). + promotion_on_hit_ = enable_offload_ && config.promotion_on_hit; + promotion_admission_threshold_ = config.promotion_admission_threshold; + promotion_queue_limit_ = config.promotion_queue_limit; + // Defense-in-depth clamp: master.cpp clamps threshold into [1, 255] + // at flag-parse time, but direct MasterServiceConfig construction + // (tests, embedded users) bypasses that. Without the clamp here, + // threshold=0 would silently bypass the frequency gate entirely + // (freq < 0 is never true for uint8_t). + if (promotion_admission_threshold_ == 0) { + promotion_admission_threshold_ = 1; + } else if (promotion_admission_threshold_ > 255) { + promotion_admission_threshold_ = 255; + } + if (config.promotion_on_hit && !enable_offload_) { + LOG(WARNING) << "promotion_on_hit=true was requested but " + << "enable_offload=false; promotion is silently " + << "disabled because it requires offload to produce " + << "LOCAL_DISK replicas. Set enable_offload=true to " + << "use this feature."; + } + if (promotion_on_hit_) { + promotion_sketch_ = std::make_unique(); + LOG(INFO) << "Promotion-on-hit mode enabled: LOCAL_DISK-only Gets " + "will queue async promotion to MEMORY (threshold=" + << promotion_admission_threshold_ + << ", queue_limit=" << promotion_queue_limit_ << ")"; + } + eviction_running_ = true; eviction_thread_ = std::thread(&MasterService::EvictionThreadFunc, this); VLOG(1) << "action=start_eviction_thread"; @@ -553,10 +584,22 @@ void MasterService::ClearInvalidHandles( if (CleanupStaleHandles(it->second, alive_clients)) { // If the object is empty, we need to erase the iterator and // also erase the key from processing_keys, - // replication_tasks, and offloading_tasks. + // replication_tasks, offloading_tasks, and promotion_tasks. shard->processing_keys.erase(it->first); shard->replication_tasks.erase(it->first); shard->offloading_tasks.erase(it->first); + // Promotion task cleanup: if the holder client expired, the + // LOCAL_DISK source is gone and the metadata is being erased + // here. Without this erase the dangling promotion_tasks + // entry stays pinned for up to put_start_release_timeout_sec_ + // (~10 min default), inflating the global in-flight counter + // and blocking new admissions on busy clusters where many + // holders expire together (e.g. inference-worker rolling + // restarts). + if (shard->promotion_tasks.erase(it->first) > 0) { + promotion_in_flight_.fetch_sub(1, + std::memory_order_relaxed); + } it = shard->metadata.erase(it); } else { ++it; @@ -1033,40 +1076,62 @@ auto MasterService::GetReplicaListByRegex(const std::string& regex_pattern) auto MasterService::GetReplicaList(const std::string& key) -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRO accessor(this, key); - MasterMetricManager::instance().inc_total_get_nums(); + GetReplicaListResponse resp({}, default_kv_lease_ttl_); + bool promotion_eligible = false; + { + MetadataAccessorRO accessor(this, key); - if (!accessor.Exists()) { - VLOG(1) << "key=" << key << ", info=object_not_found"; - return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); + MasterMetricManager::instance().inc_total_get_nums(); + + if (!accessor.Exists()) { + VLOG(1) << "key=" << key << ", info=object_not_found"; + return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); + } + const auto& metadata = accessor.Get(); + + std::vector replica_list; + metadata.VisitReplicas( + &Replica::fn_is_completed, [&replica_list](const Replica& replica) { + replica_list.emplace_back(replica.get_descriptor()); + }); + + if (replica_list.empty()) { + LOG(WARNING) << "key=" << key << ", error=replica_not_ready"; + return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); + } + + // TODO: NoF SSD support (ranhaojia) + if (replica_list[0].is_memory_replica()) { + MasterMetricManager::instance().inc_mem_cache_hit_nums(); + } else if (replica_list[0].is_disk_replica()) { + MasterMetricManager::instance().inc_file_cache_hit_nums(); + } + MasterMetricManager::instance().inc_valid_get_nums(); + // Grant a lease to the object so it will not be removed + // when the client is reading it. + metadata.GrantLease(default_kv_lease_ttl_, default_kv_soft_pin_ttl_); + + // Promotion-on-hit eligibility: only when no MEMORY replica is + // present but at least one LOCAL_DISK replica is. Decided here while + // we hold the RO accessor; the actual enqueue happens after we + // release the accessor below to avoid lock-upgrade complexity. + if (promotion_on_hit_) { + const bool any_memory = + metadata.HasReplica(&Replica::fn_is_memory_replica); + const bool any_local_disk = + metadata.HasReplica(&Replica::fn_is_local_disk_replica); + promotion_eligible = !any_memory && any_local_disk; + } + + resp = GetReplicaListResponse(std::move(replica_list), + default_kv_lease_ttl_); } - const auto& metadata = accessor.Get(); - - std::vector replica_list; - metadata.VisitReplicas( - &Replica::fn_is_completed, [&replica_list](const Replica& replica) { - replica_list.emplace_back(replica.get_descriptor()); - }); - - if (replica_list.empty()) { - LOG(WARNING) << "key=" << key << ", error=replica_not_ready"; - return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); + // RO accessor released. Safe to take a fresh RW accessor now. + if (promotion_eligible) { + TryPushPromotionQueue(key); } - - // TODO: NoF SSD support (ranhaojia) - if (replica_list[0].is_memory_replica()) { - MasterMetricManager::instance().inc_mem_cache_hit_nums(); - } else if (replica_list[0].is_disk_replica()) { - MasterMetricManager::instance().inc_file_cache_hit_nums(); - } - MasterMetricManager::instance().inc_valid_get_nums(); - // Grant a lease to the object so it will not be removed - // when the client is reading it. - metadata.GrantLease(default_kv_lease_ttl_, default_kv_soft_pin_ttl_); - - return GetReplicaListResponse(std::move(replica_list), - default_kv_lease_ttl_); + return resp; } auto MasterService::AllocateAndInsertMetadata( @@ -1556,6 +1621,7 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, if (it != shard->metadata.end() && CleanupStaleHandles(it->second, alive_clients)) { shard->processing_keys.erase(key); + ErasePromotionTaskIfPresent(shard, key); shard->metadata.erase(it); it = shard->metadata.end(); } @@ -1596,6 +1662,7 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, // If no COMPLETE replicas survive the preemption, this key // effectively does not exist — fall through to Case A. if (!metadata.HasReplica(&Replica::fn_is_completed)) { + ErasePromotionTaskIfPresent(shard, key); shard->metadata.erase(it); it = shard->metadata.end(); } @@ -2263,6 +2330,7 @@ auto MasterService::Remove(const std::string& key, bool force) // Remove object metadata accessor.Erase(); + ErasePromotionTaskIfPresent(accessor.GetShard(), key); return {}; } @@ -2316,6 +2384,7 @@ auto MasterService::RemoveByRegex(const std::string& regex_pattern, bool force) VLOG(1) << "key=" << it->first << " matched by regex. Removing."; + ErasePromotionTaskIfPresent(shard, it->first); it = shard->metadata.erase(it); removed_count++; } else { @@ -2718,6 +2787,376 @@ tl::expected MasterService::PushOffloadingQueue( return {}; } +// Promotion-on-hit + +// Push a key onto the holder client's promotion_objects map. Resolves the +// holder via the LOCAL_DISK replica's embedded client_id rather than via +// the segment-name reverse lookup. +tl::expected MasterService::PushPromotionQueue( + const std::string& key, Replica& source_replica) { + auto holder_id = source_replica.get_local_disk_client_id(); + if (!holder_id.has_value()) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + ScopedLocalDiskSegmentAccess local_disk_segment_access = + segment_manager_.getLocalDiskSegmentAccess(); + auto& client_local_disk_segment = + local_disk_segment_access.getClientLocalDiskSegment(); + auto local_disk_segment_it = + client_local_disk_segment.find(holder_id.value()); + if (local_disk_segment_it == client_local_disk_segment.end()) { + // Holder client expired or never had a LocalDiskSegment registered; + // the LOCAL_DISK replica will be cleaned up by ClientMonitorFunc on + // its own schedule. + return tl::make_unexpected(ErrorCode::SEGMENT_NOT_FOUND); + } + MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); + auto res = local_disk_segment_it->second->promotion_objects.emplace( + key, static_cast(source_replica.get_descriptor() + .get_local_disk_descriptor() + .object_size)); + if (!res.second) { + return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); + } + return {}; +} + +void MasterService::TryPushPromotionQueue(const std::string& key) { + if (!promotion_on_hit_ || !promotion_sketch_) { + return; + } + + // Frequency gate: bump and compare against the threshold. The sketch + // returns uint8_t (saturating at 255); promotion_admission_threshold_ + // is clamped into [1, 255] at config parse time (see master.cpp), so + // direct comparison is well-defined and threshold=0 (which would + // bypass the gate entirely since freq is uint8_t) cannot reach here. + const uint8_t freq = promotion_sketch_->increment(key); + if (freq < promotion_admission_threshold_) { + return; + } + + // Watermark gate: don't promote if DRAM is already under eviction + // pressure. The check is best-effort (state can change between this + // sample and the actual allocation in PromotionAllocStart). + const double used_ratio = + MasterMetricManager::instance().get_global_mem_used_ratio(); + if (used_ratio >= eviction_high_watermark_ratio_) { + return; + } + + // Acquire a fresh RW shard accessor for dedup, refcnt-pin, and task + // record. Safe to call here because GetReplicaList has already released + // its RO accessor. + MetadataAccessorRW accessor(this, key); + if (!accessor.Exists()) { + return; + } + auto& metadata = accessor.Get(); + auto& shard = accessor.GetShard(); + + // Dedup: don't queue twice if a promotion is already in flight or if a + // MEMORY replica has appeared since GetReplicaList observed only-disk. + if (shard->promotion_tasks.count(key) > 0) { + return; + } + if (metadata.HasReplica(&Replica::fn_is_memory_replica)) { + return; + } + + // Cap gate: read the cluster-wide in-flight count. Soft cap — a + // benign TOCTOU race between this load and the emplace below can let + // a few extra tasks slip in, but the per-shard mutex already + // serializes inserts within a shard and the dedup gate above prevents + // duplicate work, so the worst case is N concurrent inserters across + // distinct shards each admitting one extra task. Atomic load is + // relaxed because the value is purely advisory. + if (promotion_in_flight_.load(std::memory_order_relaxed) >= + promotion_queue_limit_) { + return; + } + + // Find the LOCAL_DISK source replica. + Replica* source = nullptr; + metadata.VisitReplicas(&Replica::fn_is_local_disk_replica, + [&source](Replica& r) { + if (source == nullptr) source = &r; + }); + if (source == nullptr) { + return; + } + + // Pin the source replica. + source->inc_refcnt(); + const uint64_t object_size = + source->get_descriptor().get_local_disk_descriptor().object_size; + + // Try to enqueue on the holder client. On failure, drop the refcnt back. + auto push_result = PushPromotionQueue(key, *source); + if (!push_result) { + source->dec_refcnt(); + VLOG(1) << "promotion_push_failed key=" << key + << " error=" << push_result.error(); + return; + } + + // Capture the holder client_id so NotifyPromotionSuccess can reject + // calls from other clients. PushPromotionQueue already validated + // get_local_disk_client_id() returns a value, so .value() is safe. + const UUID holder_id = source->get_local_disk_client_id().value(); + + // Record the in-flight task. alloc_id is filled in by + // PromotionAllocStart once the new MEMORY replica is staged. + shard->promotion_tasks.emplace( + key, PromotionTask{.source_id = source->id(), + .alloc_id = 0, + .object_size = object_size, + .start_time = std::chrono::system_clock::now(), + .holder_id = holder_id}); + promotion_in_flight_.fetch_add(1, std::memory_order_relaxed); + VLOG(1) << "promotion_queued key=" << key << " size=" << object_size; +} + +auto MasterService::PromotionObjectHeartbeat(const UUID& client_id) + -> tl::expected, ErrorCode> { + std::shared_lock shared_lock(snapshot_mutex_); + ScopedLocalDiskSegmentAccess local_disk_segment_access = + segment_manager_.getLocalDiskSegmentAccess(); + auto& client_local_disk_segment = + local_disk_segment_access.getClientLocalDiskSegment(); + auto local_disk_segment_it = client_local_disk_segment.find(client_id); + if (local_disk_segment_it == client_local_disk_segment.end()) { + return tl::make_unexpected(ErrorCode::SEGMENT_NOT_FOUND); + } + MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); + // Return at most kMaxPerHeartbeat tasks. Each task does a + // synchronous SSD read + RDMA write on the client side; allowing + // more than one per heartbeat risks blocking past the client- + // liveness window and the master marking the client dead. The rest + // stay queued in promotion_objects for subsequent heartbeats. The + // cap must live here (server side) rather than on the client so + // leftover work isn't silently dropped. + constexpr size_t kMaxPerHeartbeat = 1; + auto& src = local_disk_segment_it->second->promotion_objects; + std::unordered_map result; + while (result.size() < kMaxPerHeartbeat && !src.empty()) { + auto node = src.extract(src.begin()); + result.insert(std::move(node)); + } + return result; +} + +auto MasterService::PromotionAllocStart( + const UUID& client_id, const std::string& key, uint64_t size, + const std::vector& preferred_segments) + -> tl::expected { + std::shared_lock shared_lock(snapshot_mutex_); + MetadataAccessorRW accessor(this, key); + if (!accessor.Exists()) { + return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); + } + auto& metadata = accessor.Get(); + + // Verify the in-flight task still exists before allocating. The + // reaper can sweep it between the holder's heartbeat and this + // AllocStart call (a hung client, GC pause, or HA failover can + // stall AllocStart past put_start_release_timeout_sec_). If we + // allocated and AddReplicas'd anyway, the staged PROCESSING MEMORY + // replica would have no PromotionTask pointing at it: the generic + // PROCESSING reaper iterates shard->processing_keys (never + // populated by promotion) and the promotion-task reaper would have + // nothing left to iterate, leaking the buffer until the object is + // removed or evicted. The shard mutex is held for the rest of this + // function, so the iterator stays valid across the allocation step. + auto& shard = accessor.GetShard(); + auto task_it = shard->promotion_tasks.find(key); + if (task_it == shard->promotion_tasks.end()) { + return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); + } + + // Holder-only gate (see PromotionTask::holder_id doc). + if (task_it->second.holder_id != client_id) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + // Defensive size check: must match the source LOCAL_DISK + // descriptor's object_size captured at admission. A mismatch would + // let a buggy caller request a wrong-sized allocation — smaller + // risks RDMA overflow, larger wastes DRAM pinned until reaper TTL. + if (task_it->second.object_size != size) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + // Allocate a single MEMORY replica via the existing strategy, biased to + // the holder's mem segment when possible. + ReplicateConfig config; + config.replica_num = 1; + if (!preferred_segments.empty()) { + config.preferred_segments = preferred_segments; + } + + std::vector staged_replicas; + { + ScopedAllocatorAccess allocator_access = + segment_manager_.getAllocatorAccess(); + const auto& allocator_manager = allocator_access.getAllocatorManager(); + auto allocation_result = allocation_strategy_->Allocate( + allocator_manager, size, config.replica_num, preferred_segments); + if (!allocation_result) { + return tl::make_unexpected(allocation_result.error()); + } + staged_replicas = std::move(allocation_result.value()); + } + if (staged_replicas.empty()) { + return tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + } + + // Append the new PROCESSING MEMORY replica to the existing object's + // metadata. Visible only after NotifyPromotionSuccess flips it COMPLETE. + Replica::Descriptor desc = staged_replicas[0].get_descriptor(); + const ReplicaID new_id = staged_replicas[0].id(); + std::vector to_add; + to_add.push_back(std::move(staged_replicas[0])); + metadata.AddReplicas(std::move(to_add)); + + // Record the new replica's ID on the in-flight PromotionTask so + // NotifyPromotionSuccess knows exactly which replica to commit. A + // concurrent Put on this key may stage other PROCESSING MEMORY + // replicas; using alloc_id avoids the "first PROCESSING memory" + // ambiguity. + // + // Also reset start_time so the reaper TTL covers the active- + // transfer phase (AllocStart -> SSD read -> RDMA write -> Notify) + // measured from when a master-allocated buffer becomes vulnerable, + // rather than being consumed by queue-waiting. Without the reset, + // a backlogged task could enter active transfer with little TTL + // remaining and the reaper could free the staged replica via + // EraseReplicaByID mid-RDMA-write. The queue-waiting phase + // (alloc_id == 0) is bounded by its own original start_time window + // during which the reaper's EraseReplicaByID branch is a no-op. + task_it->second.alloc_id = new_id; + task_it->second.start_time = std::chrono::system_clock::now(); + return PromotionAllocStartResponse{std::move(desc)}; +} + +auto MasterService::NotifyPromotionSuccess(const UUID& client_id, + const std::string& key) + -> tl::expected { + std::shared_lock shared_lock(snapshot_mutex_); + MetadataAccessorRW accessor(this, key); + if (!accessor.Exists()) { + return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); + } + auto& metadata = accessor.Get(); + auto& shard = accessor.GetShard(); + + // Look up the in-flight task to find the exact replica we staged. A + // concurrent Put on this key may have created other PROCESSING MEMORY + // replicas, so we must not just "mark first PROCESSING memory + // complete" — that would risk committing someone else's half-written + // replica. + auto task_it = shard->promotion_tasks.find(key); + if (task_it == shard->promotion_tasks.end() || + task_it->second.alloc_id == 0) { + return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); + } + + // Holder-only gate (see PromotionTask::holder_id doc). + if (task_it->second.holder_id != client_id) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + bool committed = false; + Replica* staged = metadata.GetReplicaByID(task_it->second.alloc_id); + if (staged != nullptr && staged->is_memory_replica() && + staged->is_processing()) { + staged->mark_complete(); + committed = true; + } + + // Drop the source LOCAL_DISK replica's refcnt and erase the task. + auto* source = metadata.GetReplicaByID(task_it->second.source_id); + if (source != nullptr) { + source->dec_refcnt(); + } + shard->promotion_tasks.erase(task_it); + promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + + // Erase the per-client promotion_objects entry (best-effort; the + // heartbeat may have already drained it). + { + ScopedLocalDiskSegmentAccess local_disk_segment_access = + segment_manager_.getLocalDiskSegmentAccess(); + auto& client_local_disk_segment = + local_disk_segment_access.getClientLocalDiskSegment(); + auto it = client_local_disk_segment.find(client_id); + if (it != client_local_disk_segment.end()) { + MutexLocker locker(&it->second->offloading_mutex_); + it->second->promotion_objects.erase(key); + } + } + + if (!committed) { + return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); + } + return {}; +} + +auto MasterService::NotifyPromotionFailure(const UUID& client_id, + const std::string& key) + -> tl::expected { + std::shared_lock shared_lock(snapshot_mutex_); + MetadataAccessorRW accessor(this, key); + if (!accessor.Exists()) { + return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); + } + auto& metadata = accessor.Get(); + auto& shard = accessor.GetShard(); + + auto task_it = shard->promotion_tasks.find(key); + if (task_it == shard->promotion_tasks.end()) { + // No task to release. Either the reaper already swept it, or the + // client never had a task here. Return OK to keep this RPC + // idempotent — repeated failure notifications on the same key + // should be safe. + return {}; + } + + // Holder-only gate (see PromotionTask::holder_id doc). + if (task_it->second.holder_id != client_id) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + // Mirror the reaper's expiry path; see DiscardExpiredProcessingReplicas + // Part 4 for the full rationale on each step. + auto* source = metadata.GetReplicaByID(task_it->second.source_id); + if (source != nullptr) { + source->dec_refcnt(); + } + if (task_it->second.alloc_id != 0) { + metadata.EraseReplicaByID(task_it->second.alloc_id); + } + shard->promotion_tasks.erase(task_it); + promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + + // Clear the holder's per-client promotion_objects entry. Same + // best-effort cleanup pattern as NotifyPromotionSuccess — the + // heartbeat may have already drained it. + { + ScopedLocalDiskSegmentAccess local_disk_segment_access = + segment_manager_.getLocalDiskSegmentAccess(); + auto& client_local_disk_segment = + local_disk_segment_access.getClientLocalDiskSegment(); + auto it = client_local_disk_segment.find(client_id); + if (it != client_local_disk_segment.end()) { + MutexLocker locker(&it->second->offloading_mutex_); + it->second->promotion_objects.erase(key); + } + } + + return {}; +} + void MasterService::EvictionThreadFunc() { VLOG(1) << "action=eviction_thread_started"; @@ -2907,6 +3346,45 @@ void MasterService::DiscardExpiredProcessingReplicas( task_it = shard->offloading_tasks.erase(task_it); } + // Part 4: Discard expired promotion-on-hit tasks. For each: + // - Drop the source LOCAL_DISK refcnt so the source can be + // evicted normally. + // - If a PROCESSING MEMORY replica was staged (alloc_id != 0), + // pop it via EraseReplicaByID. The staged replica is not in + // shard->processing_keys, so this is the only place (besides + // NotifyPromotionFailure) that reaps it; without this the + // buffer leaks until the object is removed or evicted. + // - Erase the task entry and decrement the in-flight counter. + // task.start_time is set at admission and reset at AllocStart, so + // queue-wait (alloc_id == 0) and active-transfer (alloc_id != 0) + // phases each get a full put_start_release_timeout_sec_ window. + // The per-client promotion_objects map is GC'd on the next + // heartbeat (entries for vanished tasks are harmless — Notify will + // return REPLICA_IS_NOT_READY since the task entry is gone). + for (auto task_it = shard->promotion_tasks.begin(); + task_it != shard->promotion_tasks.end();) { + const auto ttl = + task_it->second.start_time + put_start_release_timeout_sec_; + if (ttl > now) { + task_it++; + continue; + } + auto metadata_it = shard->metadata.find(task_it->first); + if (metadata_it != shard->metadata.end()) { + auto source = + metadata_it->second.GetReplicaByID(task_it->second.source_id); + if (source != nullptr) { + source->dec_refcnt(); + } + if (task_it->second.alloc_id != 0) { + metadata_it->second.EraseReplicaByID(task_it->second.alloc_id); + } + } + LOG(WARNING) << "Promotion task expired for key: " << task_it->first; + task_it = shard->promotion_tasks.erase(task_it); + promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + } + if (!discarded_replicas.empty()) { std::lock_guard lock(discarded_replicas_mutex_); discarded_replicas_.splice(discarded_replicas_.end(), diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 5d5628c7..74cdd0a7 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -5454,6 +5454,7 @@ tl::expected RealClient::batch_get_into_offload_object_internal( const std::string &target_rpc_service_addr, std::unordered_map> &objects) { + offload_rpc_read_count_.fetch_add(1, std::memory_order_relaxed); auto start_time = std::chrono::steady_clock::now(); std::vector keys; std::vector sizes; diff --git a/mooncake-store/src/rpc_service.cpp b/mooncake-store/src/rpc_service.cpp index e8d71b11..d0140b57 100644 --- a/mooncake-store/src/rpc_service.cpp +++ b/mooncake-store/src/rpc_service.cpp @@ -1786,6 +1786,43 @@ tl::expected WrappedMasterService::NotifyOffloadSuccess( return result; } +tl::expected, ErrorCode> +WrappedMasterService::PromotionObjectHeartbeat(const UUID& client_id) { + ScopedVLogTimer timer(1, "PromotionObjectHeartbeat"); + timer.LogRequest("action=promotion_object_heartbeat"); + return master_service_.PromotionObjectHeartbeat(client_id); +} + +tl::expected +WrappedMasterService::PromotionAllocStart( + const UUID& client_id, const std::string& key, uint64_t size, + const std::vector& preferred_segments) { + ScopedVLogTimer timer(1, "PromotionAllocStart"); + timer.LogRequest("action=promotion_alloc_start"); + auto result = master_service_.PromotionAllocStart(client_id, key, size, + preferred_segments); + timer.LogResponseExpected(result); + return result; +} + +tl::expected WrappedMasterService::NotifyPromotionSuccess( + const UUID& client_id, const std::string& key) { + ScopedVLogTimer timer(1, "NotifyPromotionSuccess"); + timer.LogRequest("action=notify_promotion_success"); + auto result = master_service_.NotifyPromotionSuccess(client_id, key); + timer.LogResponseExpected(result); + return result; +} + +tl::expected WrappedMasterService::NotifyPromotionFailure( + const UUID& client_id, const std::string& key) { + ScopedVLogTimer timer(1, "NotifyPromotionFailure"); + timer.LogRequest("action=notify_promotion_failure"); + auto result = master_service_.NotifyPromotionFailure(client_id, key); + timer.LogResponseExpected(result); + return result; +} + tl::expected WrappedMasterService::CreateDrainJob( const CreateDrainJobRequest& request) { return master_service_.CreateDrainJob(request); @@ -1907,6 +1944,18 @@ void RegisterRpcService( server.register_handler< &mooncake::WrappedMasterService::NotifyOffloadSuccess>( &wrapped_master_service); + server.register_handler< + &mooncake::WrappedMasterService::PromotionObjectHeartbeat>( + &wrapped_master_service); + server + .register_handler<&mooncake::WrappedMasterService::PromotionAllocStart>( + &wrapped_master_service); + server.register_handler< + &mooncake::WrappedMasterService::NotifyPromotionSuccess>( + &wrapped_master_service); + server.register_handler< + &mooncake::WrappedMasterService::NotifyPromotionFailure>( + &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::CopyStart>( &wrapped_master_service); server.register_handler<&mooncake::WrappedMasterService::CopyEnd>( diff --git a/mooncake-store/tests/CMakeLists.txt b/mooncake-store/tests/CMakeLists.txt index d80cdddf..df1374a5 100644 --- a/mooncake-store/tests/CMakeLists.txt +++ b/mooncake-store/tests/CMakeLists.txt @@ -40,8 +40,12 @@ add_store_test(master_service_test master_service_test.cpp) add_store_test(batch_remove_test batch_remove_test.cpp) add_store_test(master_service_ssd_test master_service_ssd_test.cpp) add_store_test(offload_on_evict_test offload_on_evict_test.cpp) +add_store_test(promotion_on_hit_test promotion_on_hit_test.cpp) +add_store_test(file_storage_promotion_test file_storage_promotion_test.cpp) add_store_test(master_service_ssd_test_for_snapshot ha/snapshot/master_service_ssd_test_for_snapshot.cpp) +add_store_test(master_service_promotion_test_for_snapshot + ha/snapshot/master_service_promotion_test_for_snapshot.cpp) add_store_test(client_integration_test client_integration_test.cpp) if(USE_CXL) add_store_test(cxl_client_integration_test cxl_client_integration_test.cpp) diff --git a/mooncake-store/tests/file_storage_promotion_test.cpp b/mooncake-store/tests/file_storage_promotion_test.cpp new file mode 100644 index 00000000..187f00d6 --- /dev/null +++ b/mooncake-store/tests/file_storage_promotion_test.cpp @@ -0,0 +1,373 @@ +// Unit tests for FileStorage::ProcessPromotionTasks. Uses a stub Client +// that overrides the 4 promotion-RPC entry points so we can drive the +// orchestration loop deterministically without standing up a master. + +#include +#include + +#include +#include +#include +#include + +#include "client_service.h" +#include "file_storage.h" +#include "storage_backend.h" + +namespace mooncake { + +namespace fs_test { + +// Programmable fake. Each Set* method records what the next call should +// return; the call itself records that it happened so tests can assert on +// invocation count and ordering. +class FakeClient : public Client { + public: + FakeClient() + : Client(/*local_hostname=*/"localhost:9003", + /*metadata_connstring=*/"", + /*protocol=*/"tcp", + /*labels=*/{}) {} + + // Drives the queue returned to the heartbeat caller. + std::unordered_map heartbeat_queue; + tl::expected heartbeat_result = + tl::expected{}; + + tl::expected PromotionObjectHeartbeat( + std::unordered_map& promotion_objects) override { + heartbeat_calls.fetch_add(1); + if (!heartbeat_result.has_value()) { + return tl::make_unexpected(heartbeat_result.error()); + } + // Mirror production master: PromotionObjectHeartbeat returns at + // most kMaxPerHeartbeat keys per call (see + // MasterService::PromotionObjectHeartbeat) and leaves the rest + // queued for subsequent calls. Tests iterate by calling + // ProcessPromotionTasks multiple times until heartbeat_queue is + // empty. + constexpr size_t kMaxPerHeartbeat = 1; + promotion_objects.clear(); + while (promotion_objects.size() < kMaxPerHeartbeat && + !heartbeat_queue.empty()) { + auto node = heartbeat_queue.extract(heartbeat_queue.begin()); + promotion_objects.insert(std::move(node)); + } + return {}; + } + + // PromotionAllocStart: per-key dispatch table. Keys absent from + // alloc_overrides return the default response. + std::unordered_map alloc_overrides; + PromotionAllocStartResponse default_alloc_response; + bool default_alloc_succeeds = true; + + tl::expected PromotionAllocStart( + const std::string& key, uint64_t size, + const std::vector& preferred_segments) override { + (void)size; + (void)preferred_segments; + alloc_calls.fetch_add(1); + last_alloc_key = key; + auto it = alloc_overrides.find(key); + if (it != alloc_overrides.end()) { + return tl::make_unexpected(it->second); + } + if (!default_alloc_succeeds) { + return tl::make_unexpected(ErrorCode::NO_AVAILABLE_HANDLE); + } + return default_alloc_response; + } + + // PromotionWrite: per-key dispatch. + std::unordered_map write_overrides; + ErrorCode default_write_result = ErrorCode::OK; + + ErrorCode PromotionWrite(const Replica::Descriptor&, + std::vector&) override { + write_calls.fetch_add(1); + auto it = write_overrides.find(last_alloc_key); + if (it != write_overrides.end()) { + return it->second; + } + return default_write_result; + } + + std::unordered_map notify_overrides; + tl::expected default_notify_result = + tl::expected{}; + + tl::expected NotifyPromotionSuccess( + const std::string& key) override { + notify_calls.fetch_add(1); + notify_keys.push_back(key); + auto it = notify_overrides.find(key); + if (it != notify_overrides.end()) { + return tl::make_unexpected(it->second); + } + if (!default_notify_result.has_value()) { + return tl::make_unexpected(default_notify_result.error()); + } + return {}; + } + + // NotifyPromotionFailure: records calls so tests can assert that + // post-AllocStart failure paths in ProcessPromotionTasks actually + // notify the master. + tl::expected NotifyPromotionFailure( + const std::string& key) override { + notify_failure_calls.fetch_add(1); + notify_failure_keys.push_back(key); + return {}; + } + + std::atomic heartbeat_calls{0}; + std::atomic alloc_calls{0}; + std::atomic write_calls{0}; + std::atomic notify_calls{0}; + std::atomic notify_failure_calls{0}; + std::vector notify_keys; + std::vector notify_failure_keys; + std::string last_alloc_key; +}; + +} // namespace fs_test + +class FileStoragePromotionTest : public ::testing::Test { + protected: + std::string data_path; + std::shared_ptr fake; + std::unique_ptr file_storage; + + void SetUp() override { + google::InitGoogleLogging("FileStoragePromotionTest"); + FLAGS_logtostderr = true; + unsetenv("MOONCAKE_OFFLOAD_FILE_STORAGE_PATH"); + + data_path = std::filesystem::current_path().string() + "/data_prom"; + std::filesystem::create_directories(data_path); + for (const auto& entry : + std::filesystem::directory_iterator(data_path)) { + if (entry.is_regular_file()) std::filesystem::remove(entry.path()); + } + + FileStorageConfig cfg = FileStorageConfig::FromEnvironment(); + cfg.storage_filepath = data_path; + cfg.local_buffer_size = 4 * 1024 * 1024; + fake = std::make_shared(); + file_storage = + std::make_unique(cfg, fake, "localhost:9003"); + } + + void TearDown() override { + file_storage.reset(); + fake.reset(); + google::ShutdownGoogleLogging(); + for (const auto& entry : + std::filesystem::directory_iterator(data_path)) { + if (entry.is_regular_file()) std::filesystem::remove(entry.path()); + } + } + + tl::expected CallProcessPromotionTasks() { + return file_storage->ProcessPromotionTasks(); + } + + // Drain a multi-key queue across multiple ticks. ProcessPromotionTasks + // caps work at 1 task/tick (heartbeat-safety) and the FakeClient's + // PromotionObjectHeartbeat mirrors production by clearing the queue on + // drain. So the test fixture has to push the remaining keys back + // between ticks, the same way the master would re-push if the gate + // re-fires. We track which key was processed via fake->last_alloc_key + // and remove it from the working set. + tl::expected DrainAllPromotionTasks( + std::unordered_map remaining) { + tl::expected last_res{}; + while (!remaining.empty()) { + std::string before = fake->last_alloc_key; + fake->heartbeat_queue = remaining; + last_res = CallProcessPromotionTasks(); + if (!last_res.has_value()) return last_res; + if (fake->last_alloc_key == before || + !remaining.contains(fake->last_alloc_key)) { + // No forward progress (e.g., empty effective queue / size<=0 + // skip on every key); avoid infinite loop. + break; + } + remaining.erase(fake->last_alloc_key); + } + return last_res; + } +}; + +// Empty heartbeat queue: do nothing past the heartbeat call. +TEST_F(FileStoragePromotionTest, EmptyQueueIsNoOp) { + fake->heartbeat_queue = {}; + auto res = CallProcessPromotionTasks(); + EXPECT_TRUE(res.has_value()); + EXPECT_EQ(fake->heartbeat_calls.load(), 1); + EXPECT_EQ(fake->alloc_calls.load(), 0); + EXPECT_EQ(fake->write_calls.load(), 0); + EXPECT_EQ(fake->notify_calls.load(), 0); +} + +// Heartbeat returns SEGMENT_NOT_FOUND (transient post-restart): +// swallow silently, no error to caller. +TEST_F(FileStoragePromotionTest, HeartbeatSegmentNotFoundIsBenign) { + fake->heartbeat_result = tl::make_unexpected(ErrorCode::SEGMENT_NOT_FOUND); + auto res = CallProcessPromotionTasks(); + EXPECT_TRUE(res.has_value()); + EXPECT_EQ(fake->alloc_calls.load(), 0); +} + +// Heartbeat returns a non-benign error: propagate it. +TEST_F(FileStoragePromotionTest, HeartbeatHardErrorPropagates) { + fake->heartbeat_result = tl::make_unexpected(ErrorCode::INTERNAL_ERROR); + auto res = CallProcessPromotionTasks(); + ASSERT_FALSE(res.has_value()); + EXPECT_EQ(res.error(), ErrorCode::INTERNAL_ERROR); + EXPECT_EQ(fake->alloc_calls.load(), 0); +} + +// Non-positive size in queue: skip that key, continue. +TEST_F(FileStoragePromotionTest, NonPositiveSizeSkipped) { + fake->heartbeat_queue = {{"k_bad", 0}, {"k_good", 1024}}; + auto res = CallProcessPromotionTasks(); + EXPECT_TRUE(res.has_value()); + // Only k_good should reach AllocStart. + EXPECT_EQ(fake->alloc_calls.load(), 1); + EXPECT_EQ(fake->last_alloc_key, "k_good"); +} + +// PromotionAllocStart fails (e.g. master out of DRAM): skip key, no +// write, no notify, advance to next. +TEST_F(FileStoragePromotionTest, AllocStartFailureSkipsKey) { + fake->alloc_overrides["k1"] = ErrorCode::NO_AVAILABLE_HANDLE; + auto res = DrainAllPromotionTasks({{"k1", 1024}, {"k2", 1024}}); + EXPECT_TRUE(res.has_value()); + EXPECT_EQ(fake->alloc_calls.load(), 2); + // k1's failure prevents its write+notify; k2 succeeds (then fails at + // BatchLoad because the SSD file doesn't exist). + EXPECT_LE(fake->write_calls.load(), 1); + EXPECT_LE(fake->notify_calls.load(), 1); +} + +// BatchLoad failure (SSD file missing): no PromotionWrite, no Notify. +// Master-side reaper handles the orphaned PROCESSING replica. +TEST_F(FileStoragePromotionTest, BatchLoadFailureLeavesNoNotify) { + fake->heartbeat_queue = {{"k_missing", 1024}}; + // Default alloc succeeds; BatchLoad will fail because there's no file + // at data_path/k_missing for the storage backend to read. + auto res = CallProcessPromotionTasks(); + EXPECT_TRUE(res.has_value()); // ProcessPromotionTasks itself is best- + // effort; per-key failures are warnings. + EXPECT_EQ(fake->alloc_calls.load(), 1); + EXPECT_EQ(fake->write_calls.load(), 0) + << "TransferWrite must not run if BatchLoad failed"; + EXPECT_EQ(fake->notify_calls.load(), 0) + << "NotifyPromotionSuccess must not run if BatchLoad failed"; +} + +// PromotionWrite failure: no Notify. +TEST_F(FileStoragePromotionTest, TransferWriteFailureLeavesNoNotify) { + fake->heartbeat_queue = {{"k_te_fail", 1024}}; + fake->default_write_result = ErrorCode::TRANSFER_FAIL; + auto res = CallProcessPromotionTasks(); + EXPECT_TRUE(res.has_value()); + EXPECT_EQ(fake->alloc_calls.load(), 1); + // Whether write_calls is 0 or 1 depends on whether BatchLoad succeeded + // first; either way Notify must not fire. + EXPECT_EQ(fake->notify_calls.load(), 0) + << "NotifyPromotionSuccess must not run on TransferWrite failure"; +} + +// NotifyPromotionSuccess failure: logged, but processing of remaining +// keys continues. +TEST_F(FileStoragePromotionTest, NotifyFailureDoesNotAbortBatch) { + fake->notify_overrides["k1"] = ErrorCode::OBJECT_NOT_FOUND; + auto res = DrainAllPromotionTasks({{"k1", 1024}, {"k2", 1024}}); + EXPECT_TRUE(res.has_value()); + EXPECT_EQ(fake->alloc_calls.load(), 2) + << "Both keys must be alloc-attempted regardless of k1's notify " + << "failure"; +} + +// Per-key independence: failures on one key don't prevent attempts on +// others. +TEST_F(FileStoragePromotionTest, PerKeyFailuresAreIndependent) { + fake->alloc_overrides["k_alloc_fail"] = ErrorCode::NO_AVAILABLE_HANDLE; + fake->write_overrides["k_write_fail"] = ErrorCode::TRANSFER_FAIL; + auto res = DrainAllPromotionTasks({ + {"k_alloc_fail", 1024}, + {"k_write_fail", 1024}, + {"k_normal", 1024}, + }); + EXPECT_TRUE(res.has_value()); + // All three reach AllocStart. + EXPECT_EQ(fake->alloc_calls.load(), 3); +} + +// Every post-admission failure path (including PromotionAllocStart's +// own failure) must call NotifyPromotionFailure so the master releases +// the task slot immediately. Without this, the slot stays pinned until +// put_start_release_timeout_sec_ (~10 min default), turning a transient +// DRAM-pressure spike into a sustained outage of promotion_queue_limit_. +TEST_F(FileStoragePromotionTest, AllocStartFailureNotifiesMaster) { + fake->heartbeat_queue = {{"k_alloc_fail", 1024}}; + fake->alloc_overrides["k_alloc_fail"] = ErrorCode::NO_AVAILABLE_HANDLE; + auto res = CallProcessPromotionTasks(); + EXPECT_TRUE(res.has_value()); + EXPECT_EQ(fake->alloc_calls.load(), 1); + EXPECT_EQ(fake->notify_calls.load(), 0) + << "Success-notify must not fire on AllocStart failure."; + EXPECT_EQ(fake->notify_failure_calls.load(), 1) + << "AllocStart failure must invoke NotifyPromotionFailure so the " + << "master can release the task slot immediately. Without this " + << "the slot is pinned for ~10 min and transient DRAM pressure " + << "saturates promotion_queue_limit_ for the same window."; + ASSERT_EQ(fake->notify_failure_keys.size(), 1u); + EXPECT_EQ(fake->notify_failure_keys[0], "k_alloc_fail"); +} + +// Same invariant on the post-AllocStart paths (BatchLoad / +// TransferWrite / Notify-Success). Each failure mode must release the +// master slot. Exercises three modes (AllocStart itself, missing-file +// BatchLoad, override on Notify) by draining across successive +// heartbeats (the FakeClient mirrors the master's kMaxPerHeartbeat = 1 +// cap) and verifies each one releases. +TEST_F(FileStoragePromotionTest, PostAllocFailuresAllNotifyMaster) { + fake->alloc_overrides["k_alloc_fail"] = ErrorCode::NO_AVAILABLE_HANDLE; + // k_load_fail: no override -> AllocStart succeeds, but BatchLoad + // will fail because no SSD file exists for this key in data_path. + // k_notify_fail: AllocStart and BatchLoad and TransferWrite all + // succeed; Notify is overridden to fail. + fake->notify_overrides["k_notify_fail"] = ErrorCode::OBJECT_NOT_FOUND; + + auto res = DrainAllPromotionTasks({ + {"k_alloc_fail", 1024}, + {"k_load_fail", 1024}, + {"k_notify_fail", 1024}, + }); + EXPECT_TRUE(res.has_value()); + EXPECT_EQ(fake->alloc_calls.load(), 3); + + // Every failed key must have its slot released via Notify-Failure. + // k_notify_fail also counts: Notify-Success failed, so we still + // need to free the slot. + EXPECT_EQ(fake->notify_failure_calls.load(), 3) + << "All three failed promotions must each invoke " + << "NotifyPromotionFailure. If this fires with <3, one of the " + << "failure paths is still leaking the master slot."; + std::set got(fake->notify_failure_keys.begin(), + fake->notify_failure_keys.end()); + EXPECT_EQ(got.count("k_alloc_fail"), 1u); + EXPECT_EQ(got.count("k_load_fail"), 1u); + EXPECT_EQ(got.count("k_notify_fail"), 1u); +} + +} // namespace mooncake + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/mooncake-store/tests/ha/snapshot/master_service_promotion_test_for_snapshot.cpp b/mooncake-store/tests/ha/snapshot/master_service_promotion_test_for_snapshot.cpp new file mode 100644 index 00000000..6112c8d1 --- /dev/null +++ b/mooncake-store/tests/ha/snapshot/master_service_promotion_test_for_snapshot.cpp @@ -0,0 +1,197 @@ +// Snapshot round-trip tests for the L2->L1 promotion-on-hit feature. +// +// The base fixture's TearDown automatically calls TestSnapshotAndRestore +// against the service_ member, which: +// 1. Persists the master state to local snapshot storage. +// 2. Boots a fresh MasterService in restore mode against that snapshot. +// 3. Asserts CaptureServiceState() matches before vs. after. +// +// LOCAL_DISK replica content is captured via CaptureServiceState's +// CompareReplicaDescriptor (which compares LocalDiskDescriptor fields), and +// LocalDiskSegmentState (offloading_objects map) is also captured. Anything +// these tests put into those fields will round-trip; anything not captured +// (e.g., per-shard PromotionTask map, per-segment promotion_objects map) +// is, by design, transient — the master is allowed to drop it on restart +// and let clients re-trigger via heartbeat. + +#include "master_service_test_for_snapshot_base.h" + +namespace mooncake::test { + +class MasterServicePromotionSnapshotTest + : public MasterServiceSnapshotTestBase { + protected: + static bool glog_initialized_; + + void SetUp() override { + MasterServiceSnapshotTestBase::SetUp(); + if (!glog_initialized_) { + google::InitGoogleLogging("MasterServicePromotionSnapshotTest"); + FLAGS_logtostderr = true; + glog_initialized_ = true; + } + } + + // Build a master with promotion-on-hit enabled. enable_offload is + // required (promotion gates on it). admission_threshold=1 keeps the + // gate first-touch so tests don't need many reads. + void CreateMasterServiceWithPromotion() { + auto config = MasterServiceConfig::builder().build(); + // Builder doesn't expose promotion knobs. Use direct field + // assignment. + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.put_start_discard_timeout_sec = 0; + config.put_start_release_timeout_sec = 1; + service_ = std::make_unique(config); + } + + Segment MakeMemSegment(const std::string& name, size_t base) const { + Segment s; + s.id = generate_uuid(); + s.name = name; + s.base = base; + s.size = kDefaultSegmentSize; + s.te_endpoint = name; + return s; + } + + UUID MountMemAndDisk(const std::string& seg_name, size_t base) { + Segment seg = MakeMemSegment(seg_name, base); + UUID client_id = generate_uuid(); + auto mount = service_->MountSegment(seg, client_id); + EXPECT_TRUE(mount.has_value()); + auto mount_ld = service_->MountLocalDiskSegment(client_id, true); + EXPECT_TRUE(mount_ld.has_value()); + return client_id; + } + + bool InjectLocalDiskReplica(const UUID& client_id, const std::string& key, + int64_t size, + const std::string& transport_endpoint) { + std::vector keys{key}; + StorageObjectMetadata sm; + sm.bucket_id = 0; + sm.offset = 0; + sm.key_size = static_cast(key.size()); + sm.data_size = size; + sm.transport_endpoint = transport_endpoint; + std::vector metas{sm}; + return service_->NotifyOffloadSuccess(client_id, keys, metas) + .has_value(); + } +}; + +bool MasterServicePromotionSnapshotTest::glog_initialized_ = false; + +// A LOCAL_DISK replica created via the offload path round-trips intact: +// after snapshot+restore, the descriptor type, client_id, object_size, and +// transport_endpoint must all match. +TEST_F(MasterServicePromotionSnapshotTest, LocalDiskReplicaRoundTrip) { + CreateMasterServiceWithPromotion(); + UUID client_id = MountMemAndDisk("seg_a", kDefaultSegmentBase); + + ASSERT_TRUE( + InjectLocalDiskReplica(client_id, "k_cold", 1024, "seg_a_endpoint")); + + auto before = service_->GetReplicaList("k_cold"); + ASSERT_TRUE(before.has_value()); + ASSERT_EQ(before->replicas.size(), 1u); + EXPECT_TRUE(before->replicas[0].is_local_disk_replica()); +} + +// A key with both LOCAL_DISK and a separate MEMORY replica round-trips. +// Tests that mixed-replica metadata serialization handles both descriptor +// variants in the same object. +TEST_F(MasterServicePromotionSnapshotTest, MixedMemoryAndLocalDiskRoundTrip) { + CreateMasterServiceWithPromotion(); + UUID client_id = MountMemAndDisk("seg_a", kDefaultSegmentBase); + + // Put a MEMORY replica. + ReplicateConfig rc; + rc.replica_num = 1; + ASSERT_TRUE(service_->PutStart(client_id, "k_mixed", 1024, rc).has_value()); + ASSERT_TRUE(service_->PutEnd(client_id, "k_mixed", ReplicaType::MEMORY) + .has_value()); + + // Add LOCAL_DISK alongside. + ASSERT_TRUE( + InjectLocalDiskReplica(client_id, "k_mixed", 1024, "seg_a_endpoint")); + + auto descs = service_->GetReplicaList("k_mixed"); + ASSERT_TRUE(descs.has_value()); + EXPECT_EQ(descs->replicas.size(), 2u); +} + +// LocalDiskSegment.enable_offloading flag is preserved across snapshot. +// (CaptureServiceState compares this explicitly.) +TEST_F(MasterServicePromotionSnapshotTest, + LocalDiskSegmentEnableOffloadingPreserved) { + CreateMasterServiceWithPromotion(); + UUID client_id = MountMemAndDisk("seg_a", kDefaultSegmentBase); + (void)client_id; + // No further action — the LocalDiskSegment exists with enable_offloading + // = true. TearDown's TestSnapshotAndRestore round-trips and compares. +} + +// Multiple LOCAL_DISK holders, one DRAM segment. Each holder's segment +// metadata must round-trip independently. +TEST_F(MasterServicePromotionSnapshotTest, MultipleLocalDiskHoldersRoundTrip) { + CreateMasterServiceWithPromotion(); + + // Mount two DRAM+LOCAL_DISK pairs. + UUID client_a = MountMemAndDisk("seg_a", kDefaultSegmentBase); + UUID client_b = + MountMemAndDisk("seg_b", kDefaultSegmentBase + kDefaultSegmentSize); + + ASSERT_TRUE( + InjectLocalDiskReplica(client_a, "k_a", 1024, "seg_a_endpoint")); + ASSERT_TRUE( + InjectLocalDiskReplica(client_b, "k_b", 2048, "seg_b_endpoint")); + + auto a = service_->GetReplicaList("k_a"); + auto b = service_->GetReplicaList("k_b"); + ASSERT_TRUE(a.has_value()); + ASSERT_TRUE(b.has_value()); + EXPECT_EQ(a->replicas[0].get_local_disk_descriptor().client_id, client_a); + EXPECT_EQ(b->replicas[0].get_local_disk_descriptor().client_id, client_b); + EXPECT_EQ(a->replicas[0].get_local_disk_descriptor().object_size, 1024u); + EXPECT_EQ(b->replicas[0].get_local_disk_descriptor().object_size, 2048u); +} + +// Persisting state with an in-flight PromotionTask must not crash and must +// not mutate the visible replica state. The PromotionTask itself is +// allowed to be dropped on restore (transient), but the LOCAL_DISK source +// it references must come back unchanged. +TEST_F(MasterServicePromotionSnapshotTest, InFlightPromotionTaskSnapshotSafe) { + CreateMasterServiceWithPromotion(); + UUID client_id = MountMemAndDisk("seg_a", kDefaultSegmentBase); + + ASSERT_TRUE( + InjectLocalDiskReplica(client_id, "k_cold", 1024, "seg_a_endpoint")); + + // Trigger promotion gate to enqueue a task. This pins the source + // replica's refcnt and adds a per-shard PromotionTask plus a per- + // segment promotion_objects entry. + auto get = service_->GetReplicaList("k_cold"); + ASSERT_TRUE(get.has_value()); + + // The visible replica list should still expose exactly one LOCAL_DISK + // replica with correct content. Snapshot+restore happens in TearDown + // and must preserve this view. + EXPECT_EQ(get->replicas.size(), 1u); + EXPECT_TRUE(get->replicas[0].is_local_disk_replica()); +} + +// Promotion-on-hit can be configured on, off, or implicitly off (when +// enable_offload is false). The persisted master config must not affect +// replica state round-trip in any of these modes — only the runtime gate +// behavior changes. This test asserts the off-by-feature-flag case: +// LOCAL_DISK keys still round-trip even when promotion is disabled. +} // namespace mooncake::test + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/mooncake-store/tests/promotion_on_hit_test.cpp b/mooncake-store/tests/promotion_on_hit_test.cpp new file mode 100644 index 00000000..73cc2544 --- /dev/null +++ b/mooncake-store/tests/promotion_on_hit_test.cpp @@ -0,0 +1,1620 @@ +// Unit tests for the L2->L1 promotion-on-hit master-side path. Exercises +// the master-service entry points directly without going through the RPC +// layer. + +#include "master_service.h" + +#include +#include + +#include +#include +#include +#include +#include + +#include "types.h" + +namespace mooncake::test { + +class PromotionOnHitTest : public ::testing::Test { + protected: + void SetUp() override { + google::InitGoogleLogging("PromotionOnHitTest"); + FLAGS_logtostderr = true; + } + + void TearDown() override { google::ShutdownGoogleLogging(); } + + // Friend access to MasterService::promotion_admission_threshold_, which + // is otherwise private. PromotionOnHitTest is friended; TEST_F-generated + // subclasses are not, hence this static funnel. + static uint32_t GetPromotionAdmissionThresholdForTesting( + MasterService* service) { + return service->promotion_admission_threshold_; + } + + static constexpr size_t kDefaultSegmentBase = 0x300000000; + + Segment MakeSegment(std::string name, size_t base, size_t size) const { + Segment segment; + segment.id = generate_uuid(); + segment.name = std::move(name); + segment.base = base; + segment.size = size; + segment.te_endpoint = segment.name; + return segment; + } + + struct MountedSegmentContext { + UUID segment_id; + UUID client_id; + std::string segment_name; + }; + + MountedSegmentContext PrepareSegment(MasterService& service, + std::string name, size_t base, + size_t size) const { + Segment segment = MakeSegment(std::move(name), base, size); + UUID client_id = generate_uuid(); + auto mount_result = service.MountSegment(segment, client_id); + EXPECT_TRUE(mount_result.has_value()); + auto mount_ld = service.MountLocalDiskSegment(client_id, true); + EXPECT_TRUE(mount_ld.has_value()); + return {.segment_id = segment.id, + .client_id = client_id, + .segment_name = segment.name}; + } + + // Put an object and complete it (creates a MEMORY replica). + void PutObject(MasterService& service, const UUID& client_id, + const std::string& key, size_t size = 1024) { + ReplicateConfig config; + config.replica_num = 1; + auto put_start = service.PutStart(client_id, key, size, config); + ASSERT_TRUE(put_start.has_value()) << "PutStart failed for key=" << key; + auto put_end = service.PutEnd(client_id, key, ReplicaType::MEMORY); + ASSERT_TRUE(put_end.has_value()) << "PutEnd failed for key=" << key; + } + + // Inject a synthetic LOCAL_DISK replica for `key` on `client_id`'s + // segment via NotifyOffloadSuccess. Lets tests put a key into + // LOCAL_DISK-only state without running the full offload pipeline. + bool InjectLocalDiskReplica(MasterService& service, const UUID& client_id, + const std::string& key, int64_t size, + const std::string& transport_endpoint) { + std::vector keys{key}; + StorageObjectMetadata sm; + sm.bucket_id = 0; + sm.offset = 0; + sm.key_size = static_cast(key.size()); + sm.data_size = size; + sm.transport_endpoint = transport_endpoint; + std::vector metas{sm}; + auto res = service.NotifyOffloadSuccess(client_id, keys, metas); + return res.has_value(); + } + + // Register a client as a LOCAL_DISK holder only (no DRAM segment). + // This simulates the cross-host case where the LOCAL_DISK source lives + // on a different node than the DRAM target chosen for promotion. + UUID PrepareLocalDiskOnlyClient(MasterService& service) const { + UUID client_id = generate_uuid(); + auto mount_ld = service.MountLocalDiskSegment(client_id, true); + EXPECT_TRUE(mount_ld.has_value()); + return client_id; + } +}; + +// Sanity: with promotion disabled, no path mutates promotion_objects. +TEST_F(PromotionOnHitTest, DefaultOffNoPromotion) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = false; // explicitly off + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = + PrepareSegment(*service, "test_segment", kDefaultSegmentBase, seg_size); + + PutObject(*service, ctx.client_id, "k1"); + // GetReplicaList many times. With promotion_on_hit=false, nothing should + // appear in promotion_objects regardless of access count. + for (int i = 0; i < 5; ++i) { + auto resp = service->GetReplicaList("k1"); + ASSERT_TRUE(resp.has_value()); + } + + auto pending = service->PromotionObjectHeartbeat(ctx.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 0u); + + service->RemoveAll(); +} + +// With promotion enabled but no LOCAL_DISK replica present, no promotion +// should fire (the trigger gate any_local_disk is false). +TEST_F(PromotionOnHitTest, NoLocalDiskNoPromotion) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; // promote on first touch + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = + PrepareSegment(*service, "test_segment", kDefaultSegmentBase, seg_size); + + PutObject(*service, ctx.client_id, "k_mem_only"); + for (int i = 0; i < 5; ++i) { + auto resp = service->GetReplicaList("k_mem_only"); + ASSERT_TRUE(resp.has_value()); + } + + auto pending = service->PromotionObjectHeartbeat(ctx.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 0u) + << "Memory-only key should not trigger promotion"; + + service->RemoveAll(); +} + +// Single-shot Get on a key with both MEMORY and LOCAL_DISK should not +// promote (any_memory=true → trigger gate fails). +TEST_F(PromotionOnHitTest, MemoryReplicaPresentNoPromotion) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; // first-touch + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = + PrepareSegment(*service, "test_segment", kDefaultSegmentBase, seg_size); + + PutObject(*service, ctx.client_id, "k_dual", 1024); + // Add a LOCAL_DISK replica next to the MEMORY one. + ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_dual", 1024, + ctx.segment_name)); + + for (int i = 0; i < 5; ++i) { + auto resp = service->GetReplicaList("k_dual"); + ASSERT_TRUE(resp.has_value()); + } + + auto pending = service->PromotionObjectHeartbeat(ctx.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 0u) + << "MEMORY replica still present: should not promote"; + + service->RemoveAll(); +} + +// PromotionObjectHeartbeat returns an empty map when called against a +// client that has no LocalDiskSegment registered. +TEST_F(PromotionOnHitTest, HeartbeatReturnsErrorForUnknownClient) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + auto service = std::make_unique(config); + + UUID unknown_client = generate_uuid(); + auto pending = service->PromotionObjectHeartbeat(unknown_client); + ASSERT_FALSE(pending.has_value()); + EXPECT_EQ(pending.error(), ErrorCode::SEGMENT_NOT_FOUND); +} + +// PromotionAllocStart on a non-existent key returns OBJECT_NOT_FOUND. +TEST_F(PromotionOnHitTest, AllocStartUnknownKey) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + auto service = std::make_unique(config); + + auto resp = + service->PromotionAllocStart(generate_uuid(), "nonexistent", 1024, {}); + ASSERT_FALSE(resp.has_value()); + EXPECT_EQ(resp.error(), ErrorCode::OBJECT_NOT_FOUND); +} + +// NotifyPromotionSuccess on a non-existent key returns OBJECT_NOT_FOUND. +TEST_F(PromotionOnHitTest, NotifyUnknownKey) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + auto service = std::make_unique(config); + + UUID client_id = generate_uuid(); + auto resp = service->NotifyPromotionSuccess(client_id, "nonexistent"); + ASSERT_FALSE(resp.has_value()); + EXPECT_EQ(resp.error(), ErrorCode::OBJECT_NOT_FOUND); +} + +// Concurrent readers racing into TryPushPromotionQueue must dedupe to a +// single PromotionTask. Without dedup, the source LOCAL_DISK replica's +// refcnt would be incremented N times and the per-segment promotion_objects +// map would either error on the second insert (current behavior) or +// double-queue. +TEST_F(PromotionOnHitTest, RacingReadersDedup) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; // first-touch fires the gate + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = + PrepareSegment(*service, "test_segment", kDefaultSegmentBase, seg_size); + + // LOCAL_DISK-only key: NotifyOffloadSuccess on a never-PUT key creates + // metadata with only the LOCAL_DISK replica (AddReplica's Create-on- + // missing path). + ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_cold", 1024, + ctx.segment_name)); + + constexpr int kThreads = 32; + constexpr int kReadsPerThread = 10; + std::vector threads; + threads.reserve(kThreads); + for (int t = 0; t < kThreads; ++t) { + threads.emplace_back([&service]() { + for (int j = 0; j < kReadsPerThread; ++j) { + auto r = service->GetReplicaList("k_cold"); + EXPECT_TRUE(r.has_value()); + } + }); + } + for (auto& t : threads) t.join(); + + auto pending = service->PromotionObjectHeartbeat(ctx.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 1u) + << "Concurrent readers (" << kThreads << " x " << kReadsPerThread + << ") must dedupe to a single promotion task"; + ASSERT_TRUE(pending->count("k_cold")); + + service->RemoveAll(); +} + +// A PromotionTask that never receives NotifyPromotionSuccess must be +// reaped after `put_start_release_timeout_sec` so that: +// (a) the source LOCAL_DISK replica's refcnt is decremented (no +// permanent pin → eviction can still free it later); +// (b) the dedup gate is unblocked, so a subsequent read can re-enqueue +// the same key for retry. +TEST_F(PromotionOnHitTest, StalePromotionReaper) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 2000; + // Short staleness window: the reaper uses put_start_release_timeout_sec + // for promotion tasks. The master enforces release > discard, so set + // discard to a still-smaller value. + config.put_start_discard_timeout_sec = 0; + config.put_start_release_timeout_sec = 1; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = + PrepareSegment(*service, "test_segment", kDefaultSegmentBase, seg_size); + + ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_cold", 1024, + ctx.segment_name)); + + // Trigger #1: enqueue, then drain the per-segment queue. Drain leaves + // the per-shard PromotionTask intact (the heartbeat is best-effort GC, + // not the authoritative state). + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + auto pending = service->PromotionObjectHeartbeat(ctx.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 1u); + } + + // Without reap, dedup blocks re-enqueue. Confirm: GetReplicaList again, + // heartbeat must be empty because PromotionTask still pins the slot. + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + auto pending = service->PromotionObjectHeartbeat(ctx.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 0u) + << "Dedup gate should block re-enqueue while task is in flight"; + } + + // Wait past the staleness window; the eviction thread reaps the task. + // Eviction loop sleeps for kEvictionThreadSleepMs (10 ms), so 2s wall + // clock gives ~200 attempts — plenty. + std::this_thread::sleep_for(std::chrono::seconds(2)); + + // Trigger #3: with the task reaped, dedup is unblocked and a fresh + // GetReplicaList must enqueue again. + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + auto pending = service->PromotionObjectHeartbeat(ctx.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 1u) + << "After reap, a fresh read must re-enqueue the same key"; + } + + service->RemoveAll(); +} + +// Force-Remove on a key with a queued PromotionTask must not corrupt +// state. The PromotionTask references a Replica*; once the metadata is +// gone, NotifyPromotionSuccess and the reaper must both tolerate the +// missing entry. +TEST_F(PromotionOnHitTest, RemoveDuringPromotion) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 100; // short lease: Remove won't block + config.put_start_discard_timeout_sec = 0; + config.put_start_release_timeout_sec = 1; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = + PrepareSegment(*service, "test_segment", kDefaultSegmentBase, seg_size); + + ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_cold", 1024, + ctx.segment_name)); + + // Queue a promotion task. + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + + // Lease must expire before we can call non-force Remove (or use force). + auto rm = service->Remove("k_cold", /*force=*/true); + // Remove returns REPLICA_IS_NOT_READY if any replica is non-COMPLETE. + // The injected LOCAL_DISK replica is COMPLETE, so this should succeed. + ASSERT_TRUE(rm.has_value()) + << "Remove on a LOCAL_DISK-only key with a queued promotion should " + << "succeed (all replicas COMPLETE); error=" << rm.error(); + + // NotifyPromotionSuccess on the now-removed key must surface the missing + // metadata cleanly, not crash. + auto notify = service->NotifyPromotionSuccess(ctx.client_id, "k_cold"); + ASSERT_FALSE(notify.has_value()); + EXPECT_EQ(notify.error(), ErrorCode::OBJECT_NOT_FOUND); + + // Wait for the reaper; it must tolerate the missing metadata entry + // (the source replica it would dec_refcnt is already gone). + std::this_thread::sleep_for(std::chrono::seconds(2)); + + // Re-injecting the key and re-triggering must work end-to-end, proving + // the per-shard PromotionTask was reaped (not stuck). + ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_cold", 1024, + ctx.segment_name)); + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + auto pending = service->PromotionObjectHeartbeat(ctx.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 1u) + << "After Remove + reap, the same key must re-enqueue cleanly"; + } + + service->RemoveAll(); +} + +// Cross-host promotion: the LOCAL_DISK source's holder client has no DRAM +// segment of its own. The reader's DRAM segment (segment_b) is the only +// allocator candidate, so PromotionAllocStart must place the new MEMORY +// replica there — proving the master's allocation strategy crosses the +// host boundary cleanly. +TEST_F(PromotionOnHitTest, MultiSegmentAllocPicksAvailableSegment) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + // Holder: LOCAL_DISK only, no DRAM. Stands in for "host A" that has the + // SSD copy but no free RAM. + UUID holder_client_id = PrepareLocalDiskOnlyClient(*service); + + // Reader-side DRAM target: "host B" with a fresh DRAM segment. + constexpr size_t seg_size = 1024 * 1024 * 16; + auto reader_ctx = + PrepareSegment(*service, "segment_b", kDefaultSegmentBase, seg_size); + + // The cold key: only replica is LOCAL_DISK on the holder client. + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder_client_id, "k_cold", + 1024, "segment_a_endpoint")); + + // Seed the PromotionTask through the gate — PromotionAllocStart now + // requires an in-flight task to exist (rejects orphaned-stage path). + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + + // PromotionAllocStart — test the segment-selection logic on top of + // the gate-seeded task. + auto resp = + service->PromotionAllocStart(holder_client_id, "k_cold", 1024, {}); + ASSERT_TRUE(resp.has_value()) + << "PromotionAllocStart should succeed when any DRAM segment has " + << "capacity; error=" << resp.error(); + + const auto& mem_desc = resp.value().memory_descriptor; + ASSERT_TRUE(mem_desc.is_memory_replica()); + EXPECT_EQ( + mem_desc.get_memory_descriptor().buffer_descriptor.transport_endpoint_, + reader_ctx.segment_name) + << "New MEMORY replica must be allocated on the only DRAM segment " + << "(segment_b), not on the LOCAL_DISK holder which has no DRAM"; + + // Sanity: the staged MEMORY replica is PROCESSING (visible only after + // NotifyPromotionSuccess flips it COMPLETE). + EXPECT_EQ(mem_desc.status, ReplicaStatus::PROCESSING); + + service->RemoveAll(); +} + +// preferred_segments is honored: promotion can be steered to a specific +// DRAM segment (e.g., the reader's local one). TryPushPromotionQueue +// currently doesn't pass preferred_segments through, but +// PromotionAllocStart respects them for clients biasing toward locality. +TEST_F(PromotionOnHitTest, MultiSegmentAllocRespectsPreferred) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg_a = + PrepareSegment(*service, "segment_a", kDefaultSegmentBase, seg_size); + auto seg_b = PrepareSegment(*service, "segment_b", + kDefaultSegmentBase + seg_size, seg_size); + + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg_a.client_id, "k_cold", + 1024, seg_a.segment_name)); + + // Seed the PromotionTask through the gate so AllocStart's + // task-existence check passes. + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + + auto resp = service->PromotionAllocStart(seg_a.client_id, "k_cold", 1024, + {seg_b.segment_name}); + ASSERT_TRUE(resp.has_value()); + const auto& mem_desc = resp.value().memory_descriptor; + EXPECT_EQ( + mem_desc.get_memory_descriptor().buffer_descriptor.transport_endpoint_, + seg_b.segment_name) + << "preferred_segments={segment_b} should pin the new MEMORY " + << "replica to segment_b"; + + service->RemoveAll(); +} + +// promotion_queue_limit caps total in-flight tasks cluster-wide +// (gate: promotion_in_flight_ >= limit). With limit=1 the very first +// queued task saturates the cap, so a second LOCAL_DISK-only read — +// even on a key in the same shard — must be silently dropped by the +// cap gate (reads still succeed; just no new task is enqueued). +// QueueLimitRejectsCrossShard covers the same-cap-across-different- +// shards case. +TEST_F(PromotionOnHitTest, QueueLimitRejectsBeyondCap) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; // any 1 task saturates a shard + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + + // Find two keys that hash to the same shard. MasterService:: + // getShardIndex is private but the formula is deterministic + // (std::hash{}(key) % kNumShards), so we can mirror + // it here. kNumShards=1024 (master_service.h:889). + constexpr size_t kNumShardsLocal = 1024; + auto shard_of = [](const std::string& k) { + return std::hash{}(k) % kNumShardsLocal; + }; + const std::string k1 = "qlim_first"; + std::string k2; + for (int i = 0; i < 100000 && k2.empty(); ++i) { + std::string candidate = "qlim_collide_" + std::to_string(i); + if (shard_of(candidate) == shard_of(k1)) { + k2 = candidate; + } + } + ASSERT_FALSE(k2.empty()) + << "could not find a same-shard collision for " << k1; + + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, k1, 1024, + seg.segment_name)); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, k2, 1024, + seg.segment_name)); + + // First read on k1 enqueues a task in shard S. + auto r1 = service->GetReplicaList(k1); + ASSERT_TRUE(r1.has_value()); + + // Second read on k2 (same shard S, different key, so no dedup) must + // be dropped by the cap gate: the cluster-wide in-flight counter is + // already 1, which meets promotion_queue_limit_ = 1. + auto r2 = service->GetReplicaList(k2); + ASSERT_TRUE(r2.has_value()) << "read itself must still succeed; " + << "queue gate is silent"; + + // Drain the holder's queue: only k1 should appear. + auto heartbeat = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(heartbeat.has_value()); + EXPECT_EQ(heartbeat->size(), 1u) + << "promotion_queue_limit=1 should admit only the first task " + << "globally; k2's enqueue must be dropped"; + EXPECT_EQ(heartbeat->count(k1), 1u) + << "k1 was read first and should be the surviving task"; + EXPECT_EQ(heartbeat->count(k2), 0u) + << "k2 was rejected by the cap gate; should not appear"; + + service->RemoveAll(); +} + +// PromotionObjectHeartbeat caps the per-call response at kMaxPerHeartbeat +// (1) and leaves the remainder queued for subsequent heartbeats. This is +// the master-side bound that keeps the client's heartbeat thread inside +// the liveness window when many keys are queued, while guaranteeing no +// task is dropped — a regression from the prior client-side cap that +// silently discarded leftovers. +TEST_F(PromotionOnHitTest, HeartbeatBoundedBatchPreservesLeftovers) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + + // Push three keys into the holder's promotion_objects via the trigger + // path. Each is a LOCAL_DISK-only key (no MEMORY replica), so the + // first GetReplicaList per key crosses the admission threshold (=1) + // and TryPushPromotionQueue enqueues a task. + const std::vector keys{"hb_k1", "hb_k2", "hb_k3"}; + for (const auto& k : keys) { + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, k, 1024, + seg.segment_name)); + auto r = service->GetReplicaList(k); + ASSERT_TRUE(r.has_value()); + } + + // First heartbeat returns at most 1 key. + auto tick1 = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(tick1.has_value()); + EXPECT_EQ(tick1->size(), 1u) + << "heartbeat should return at most kMaxPerHeartbeat=1 entry"; + std::string first_key = tick1->begin()->first; + EXPECT_TRUE(std::find(keys.begin(), keys.end(), first_key) != keys.end()); + + // Second heartbeat returns another key (a different one). + auto tick2 = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(tick2.has_value()); + EXPECT_EQ(tick2->size(), 1u); + std::string second_key = tick2->begin()->first; + EXPECT_NE(second_key, first_key) + << "second heartbeat must drain a different leftover key, not " + << "re-return the one already extracted"; + + // Third heartbeat returns the third key. + auto tick3 = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(tick3.has_value()); + EXPECT_EQ(tick3->size(), 1u); + std::string third_key = tick3->begin()->first; + EXPECT_NE(third_key, first_key); + EXPECT_NE(third_key, second_key); + + // Fourth heartbeat: queue is now empty. + auto tick4 = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(tick4.has_value()); + EXPECT_TRUE(tick4->empty()) + << "after draining all queued keys, heartbeat must return empty"; + + // Sanity: master-side promotion_tasks records are intact for all keys + // (they're cleared by NotifyPromotionSuccess, not by Heartbeat), so the + // source refcnts remain pinned until processed. + for (const auto& k : keys) { + auto rl = service->GetReplicaList(k); + ASSERT_TRUE(rl.has_value()) << "key " << k << " should still exist"; + } + + service->RemoveAll(); +} + +// The promotion task reaper must pop the staged PROCESSING MEMORY +// replica added by PromotionAllocStart. The staged replica is not in +// shard->processing_keys, so DiscardExpiredProcessingReplicas's main +// sweep can't see it, and the reaper for promotion tasks is the only +// place that knows the replica exists. Without this path the orphan +// holds its allocator buffer indefinitely (until the object is removed +// or evicted). +// +// We can't observe the staged replica via GetReplicaList because the +// master filters out PROCESSING entries (clients can only read COMPLETE +// replicas), so we use QuerySegments to watch the DRAM allocator's used +// bytes: AllocStart bumps it, and the reaper must return it to baseline. +// NotifyPromotionSuccess on a reaped task must also fail cleanly. +TEST_F(PromotionOnHitTest, ReaperPopsStagedMemoryReplicaOnExpiry) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 2000; + config.put_start_discard_timeout_sec = 0; + config.put_start_release_timeout_sec = 1; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_cold", 1024, + ctx.segment_name)); + + // Baseline allocator usage on the DRAM segment. + auto seg_baseline = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(seg_baseline.has_value()); + const size_t used_baseline = seg_baseline->first; + + // Trigger the gate to enqueue a PromotionTask. + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + // Drive the AllocStart side so alloc_id != 0 — this is the exact + // setup that produces an orphaned PROCESSING MEMORY replica if the + // reaper does not pop it. + auto alloc = + service->PromotionAllocStart(ctx.client_id, "k_cold", 1024, {}); + ASSERT_TRUE(alloc.has_value()); + + // After AllocStart, the DRAM allocator must have committed bytes for + // the staged PROCESSING MEMORY replica. + auto seg_after_alloc = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(seg_after_alloc.has_value()); + EXPECT_GT(seg_after_alloc->first, used_baseline) + << "PromotionAllocStart should bump segment used bytes " + << "(allocator-tracked PROCESSING MEMORY replica)"; + + // Wait past the staleness window; the eviction thread reaps the + // task and (with the fix) pops the staged replica via + // EraseReplicaByID, which releases the buffer back to the allocator. + // Poll instead of a fixed sleep — the eviction thread cadence and + // the test runner's scheduling jitter (especially under suite load) + // both vary, so a hard 2s sleep is flaky here. + constexpr auto kPollDeadline = std::chrono::seconds(5); + constexpr auto kPollInterval = std::chrono::milliseconds(50); + const auto poll_start = std::chrono::steady_clock::now(); + size_t used_after_reap = 0; + while (std::chrono::steady_clock::now() - poll_start < kPollDeadline) { + auto q = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(q.has_value()); + used_after_reap = q->first; + if (used_after_reap == used_baseline) break; + std::this_thread::sleep_for(kPollInterval); + } + EXPECT_EQ(used_after_reap, used_baseline) + << "after reap: staged PROCESSING MEMORY replica's buffer must " + << "be freed back to the DRAM allocator. If this fires, the " + << "reaper is not popping the staged replica and the buffer " + << "leaks until the object itself is removed or evicted."; + + // NotifyPromotionSuccess for a reaped task must not commit anything + // and must return REPLICA_IS_NOT_READY (the task entry is gone, so + // the alloc_id lookup at the top of NotifyPromotionSuccess fails + // fast). + auto notify = service->NotifyPromotionSuccess(ctx.client_id, "k_cold"); + ASSERT_FALSE(notify.has_value()); + EXPECT_EQ(notify.error(), ErrorCode::REPLICA_IS_NOT_READY); + + service->RemoveAll(); +} + +// The cap gate must be cluster-wide, not per-shard. Promotion targets +// skewed hot keys, which by definition cluster into a small number of +// shards; a `shard->size() * kNumShards >= limit` heuristic fires +// roughly kNumShards-times too eagerly on that workload. With a global +// atomic counter, a task in shard A counts toward the cap that gates a +// task in shard B. +TEST_F(PromotionOnHitTest, QueueLimitRejectsCrossShard) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; // 1 in-flight task globally + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + + // Find two keys hashing to *different* shards. With the old per-shard + // heuristic this would let both through (each shard's count is 0 + // independently). With the global counter, only the first goes in. + constexpr size_t kNumShardsLocal = 1024; + auto shard_of = [](const std::string& k) { + return std::hash{}(k) % kNumShardsLocal; + }; + const std::string k1 = "xshard_first"; + std::string k2; + for (int i = 0; i < 100000 && k2.empty(); ++i) { + std::string candidate = "xshard_other_" + std::to_string(i); + if (shard_of(candidate) != shard_of(k1)) { + k2 = candidate; + } + } + ASSERT_FALSE(k2.empty()) << "couldn't find a different-shard key"; + ASSERT_NE(shard_of(k1), shard_of(k2)); + + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, k1, 1024, + seg.segment_name)); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, k2, 1024, + seg.segment_name)); + + auto r1 = service->GetReplicaList(k1); + ASSERT_TRUE(r1.has_value()); + + // k2 lives in a different shard, but the global cap is already met + // by k1's task — k2 must be rejected. + auto r2 = service->GetReplicaList(k2); + ASSERT_TRUE(r2.has_value()) << "read itself still succeeds"; + + auto heartbeat = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(heartbeat.has_value()); + EXPECT_EQ(heartbeat->size(), 1u) + << "with global cap=1 and one task already in shard " << shard_of(k1) + << ", a key hashing to shard " << shard_of(k2) + << " must be rejected by the global gate. A per-shard heuristic " + << "would admit it here since the destination shard's local " + << "count is 0."; + EXPECT_EQ(heartbeat->count(k1), 1u); + EXPECT_EQ(heartbeat->count(k2), 0u); + + service->RemoveAll(); +} + +// PromotionAllocStart must reset PromotionTask.start_time so the reaper +// TTL covers the active-transfer phase on its own and is not consumed by +// the queue-wait phase. Without the reset, a task that waited in the +// holder's promotion_objects queue for most of the original TTL could +// enter active transfer with little budget left; if the SSD read + RDMA +// write of a large object then ran past expiry, the reaper would +// EraseReplicaByID on the staged MEMORY replica mid-flight and the +// allocator could hand the freed buffer to a concurrent Put while the +// client's RDMA write is still landing into it (use-after-free in the +// allocator + silent data corruption for the concurrent Put). +// +// We can't directly observe start_time from a black-box test, so we +// arrange the timing so the reset is the only thing that distinguishes +// "task still alive" from "task reaped" at the assertion points: +// +// T=0 : admit task (original start_time = T=0) +// T=Wq : AllocStart (resets start_time = T=Wq) +// T=Wq+Wa: assertion 1 -- alive iff the reset happened +// (Wq + Wa > TTL, so without reset the +// original start_time has aged past TTL) +// (Wa < TTL, so with the reset the +// new start_time has NOT aged past TTL) +// T=Wq+Wb: assertion 2 -- reaped +// (Wb > TTL, so even with the reset the +// active-transfer phase has now exceeded its +// own full window) +// +// Concretely with TTL = 2s, Wq = 1.5s, Wa = 1.5s (-> 3.0s elapsed, +// 1.5s since AllocStart), Wb = 3.0s (-> 4.5s elapsed, 3.0s since +// AllocStart). The "alive" assertion at Wq+Wa is the one that proves +// the reset is wired up correctly. +// +// QuerySegments(seg).first (used bytes) is the observable: AllocStart +// bumps it, the reaper's EraseReplicaByID returns it to baseline. We +// can't use GetReplicaList because the master filters out PROCESSING +// replicas from the response. +TEST_F(PromotionOnHitTest, AllocStartResetsTaskDeadline) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 8000; + config.put_start_discard_timeout_sec = 0; + config.put_start_release_timeout_sec = 2; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_late", 1024, + ctx.segment_name)); + + auto seg_baseline = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(seg_baseline.has_value()); + const size_t used_baseline = seg_baseline->first; + + // T=0 : admit. start_time = T=0. + { + auto r = service->GetReplicaList("k_late"); + ASSERT_TRUE(r.has_value()); + } + + // T=Wq : simulate the holder's queue having been backlogged. With + // TTL = 2s and Wq = 1.5s the task is still alive at AllocStart + // (the queue-wait phase's own window hasn't expired yet — 1.5 < 2). + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + + auto alloc = + service->PromotionAllocStart(ctx.client_id, "k_late", 1024, {}); + ASSERT_TRUE(alloc.has_value()) + << "AllocStart must succeed before the queue-wait phase's TTL " + << "expires (1.5s elapsed, TTL is 2s). If this fires the test " + << "timing is wrong, not the feature."; + + // Buffer has been staged. + auto seg_after_alloc = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(seg_after_alloc.has_value()); + EXPECT_GT(seg_after_alloc->first, used_baseline) + << "PromotionAllocStart should bump segment used bytes " + << "(allocator-tracked PROCESSING MEMORY replica)"; + + // T=Wq+Wa : total elapsed since admission is 3.0s > TTL=2s. If the + // reset is missing, the reaper sees the original start_time = 0 + // and expires the task here, calling EraseReplicaByID which would + // free the staged buffer mid-RDMA-write in production. With the + // reset, start_time was bumped at AllocStart so age-since-reset is + // only 1.5s < TTL=2s and the task stays alive. This is the + // assertion that proves the reset is in place. + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + auto seg_after_wait = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(seg_after_wait.has_value()); + EXPECT_GT(seg_after_wait->first, used_baseline) + << "post-AllocStart staged MEMORY buffer must still be alive: " + << "queue-wait + active-transfer wall time (3.0s) has exceeded " + << "the bare TTL (2s), but the start_time reset at AllocStart " + << "gives the active-transfer phase its own fresh TTL window. " + << "If this fires, the reset in PromotionAllocStart is missing " + << "or broken — without it the reaper frees the staged buffer " + << "here while a client's RDMA write could still be in flight."; + + // T=Wq+Wb : sleep another 3s for a total of 4.5s since AllocStart. + // The active-transfer phase's own TTL has now expired regardless + // of the reset; the reaper must fire and free the staged buffer. + std::this_thread::sleep_for(std::chrono::seconds(3)); + auto seg_after_reap = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(seg_after_reap.has_value()); + EXPECT_EQ(seg_after_reap->first, used_baseline) + << "after the active-transfer phase's own TTL expires " + << "(4.5s > 2s since AllocStart), the reaper must fire and " + << "free the staged buffer via EraseReplicaByID"; + + service->RemoveAll(); +} + +// NotifyPromotionSuccess must decrement the cluster-wide in-flight +// counter so future admissions can use the slot; otherwise queue_limit +// remains saturated forever after the first successful promotion. Also +// exercises the only success-path end-to-end coverage of +// NotifyPromotionSuccess in the suite. +TEST_F(PromotionOnHitTest, NotifySuccessDecrementsCounter) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; // 1 in-flight task globally + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k_first", 1024, + seg.segment_name)); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k_second", + 1024, seg.segment_name)); + + // Admit task 1. promotion_in_flight_ goes from 0 -> 1. + { + auto r = service->GetReplicaList("k_first"); + ASSERT_TRUE(r.has_value()); + } + + // Drive the full success path: AllocStart stages the PROCESSING MEMORY + // replica and records alloc_id; NotifyPromotionSuccess flips it + // COMPLETE, drops the source LOCAL_DISK refcnt, erases the task, and + // decrements the counter. + auto alloc = + service->PromotionAllocStart(seg.client_id, "k_first", 1024, {}); + ASSERT_TRUE(alloc.has_value()) + << "AllocStart should succeed; error=" << alloc.error(); + auto notify = service->NotifyPromotionSuccess(seg.client_id, "k_first"); + ASSERT_TRUE(notify.has_value()) + << "NotifyPromotionSuccess happy path should succeed; if this " + << "fires, AllocStart did not record alloc_id, or the staged " + << "replica is missing/non-PROCESSING; error=" << notify.error(); + + // Counter must now be 0. A second admission on a *different* key must + // be allowed. If fetch_sub is missing on the success path, the cap is + // still saturated at 1 and TryPushPromotionQueue silently drops this + // attempt. + { + auto r = service->GetReplicaList("k_second"); + ASSERT_TRUE(r.has_value()); + } + auto pending = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->size(), 1u) + << "k_second must be admitted after k_first succeeds — the global " + << "in-flight counter must decrement on the NotifyPromotionSuccess " + << "success path. Without fetch_sub, the cap stays saturated and " + << "k_second is silently dropped."; + EXPECT_EQ(pending->count("k_second"), 1u); + + service->RemoveAll(); +} + +// PromotionAllocStart must reject when the in-flight task has been +// reaped between the holder's heartbeat and the AllocStart RPC arriving +// (e.g. client stall past put_start_release_timeout_sec_). Without the +// check, AllocStart would allocate + AddReplicas a PROCESSING MEMORY +// replica with nothing tracking it: the generic PROCESSING reaper only +// iterates shard->processing_keys (never populated by promotion) and +// the promotion-task reaper has nothing left to iterate, so the buffer +// leaks until the object is removed or evicted. +TEST_F(PromotionOnHitTest, AllocStartRejectsReapedTask) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 2000; + config.put_start_discard_timeout_sec = 0; + config.put_start_release_timeout_sec = 1; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto ctx = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_cold", 1024, + ctx.segment_name)); + + // Baseline allocator usage. + auto seg_baseline = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(seg_baseline.has_value()); + const size_t used_baseline = seg_baseline->first; + + // Admit the task. + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + + // Wait past the TTL so the reaper sweeps the task. AllocStart has + // not yet been called, so alloc_id is 0; the reaper's + // EraseReplicaByID branch is a no-op and only the task entry is + // removed. We can't easily poll for reap externally (the only + // user-facing observable would be re-admitting through the gate, + // which would create a fresh task and defeat the test), so use a + // fixed sleep with margin. Matches the StalePromotionReaper pattern + // (TTL=1s, sleep=2s). + std::this_thread::sleep_for(std::chrono::seconds(2)); + + // AllocStart on a reaped task must reject without allocating. + // Allocating would leave an orphaned PROCESSING MEMORY replica + // attached to the object. + auto alloc = + service->PromotionAllocStart(ctx.client_id, "k_cold", 1024, {}); + ASSERT_FALSE(alloc.has_value()) + << "AllocStart must reject when the task has been reaped — " + << "otherwise the staged PROCESSING MEMORY replica is orphaned"; + EXPECT_EQ(alloc.error(), ErrorCode::REPLICA_IS_NOT_READY); + + // The DRAM allocator must be at baseline: no buffer was staged. + auto seg_after = service->QuerySegments(ctx.segment_name); + ASSERT_TRUE(seg_after.has_value()); + EXPECT_EQ(seg_after->first, used_baseline) + << "AllocStart on a reaped task must not allocate. If used bytes " + << "grew here, AllocStart got to AddReplicas before the task-" + << "existence check and the staged buffer is now orphaned (no " + << "reaper iterates it)."; + + service->RemoveAll(); +} + +// NotifyPromotionSuccess must reject calls from a client that is not +// the holder of the source LOCAL_DISK replica. Without the check, any +// client knowing the key could flip the staged PROCESSING MEMORY replica +// to COMPLETE before the holder's RDMA write has landed, exposing torn +// data to readers. +TEST_F(PromotionOnHitTest, NotifyRejectsNonHolder) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_cold", + 1024, holder.segment_name)); + + // Admit + stage. + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + auto alloc = + service->PromotionAllocStart(holder.client_id, "k_cold", 1024, {}); + ASSERT_TRUE(alloc.has_value()); + + // An unrelated client tries to Notify. Must be rejected as + // INVALID_PARAMS so the staged replica stays PROCESSING. + UUID intruder_id = generate_uuid(); + ASSERT_NE(intruder_id, holder.client_id); + auto bad_notify = service->NotifyPromotionSuccess(intruder_id, "k_cold"); + ASSERT_FALSE(bad_notify.has_value()) + << "Notify from a non-holder client must be rejected — otherwise " + << "any client knowing the key can commit someone else's " + << "still-being-written replica"; + EXPECT_EQ(bad_notify.error(), ErrorCode::INVALID_PARAMS); + + // The staged replica must still be PROCESSING (not committed by the + // rejected call). Readers must not see it via GetReplicaList yet — + // GetReplicaList filters PROCESSING replicas, and pre-AllocStart + // there are no COMPLETE replicas to read for a LOCAL_DISK-only key + // (only the LOCAL_DISK descriptor itself). + auto r_check = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r_check.has_value()); + bool saw_complete_memory = false; + for (const auto& d : r_check.value().replicas) { + if (d.is_memory_replica() && d.status == ReplicaStatus::COMPLETE) { + saw_complete_memory = true; + break; + } + } + EXPECT_FALSE(saw_complete_memory) + << "Rejected Notify must not have committed the staged replica"; + + // The legitimate holder must still be able to Notify successfully. + auto good_notify = + service->NotifyPromotionSuccess(holder.client_id, "k_cold"); + ASSERT_TRUE(good_notify.has_value()) + << "Holder Notify on the same task must succeed after a rejected " + << "intruder Notify — the task entry should be untouched by the " + << "rejection path; error=" << good_notify.error(); + + service->RemoveAll(); +} + +// NotifyPromotionFailure must immediately release the task slot and the +// staged buffer so that transient client-side errors (SSD throttling, +// RDMA flakes) do not pin promotion_queue_limit_ for the full reaper +// TTL. With queue_limit=1 and a holder client that fails after a +// successful AllocStart, a second admission on a different key would +// be silently dropped until reaper TTL if Notify-Failure didn't +// fast-release. The TTL is set high enough here that any test pass +// can be attributed to the explicit release, not to reaper expiry. +TEST_F(PromotionOnHitTest, NotifyFailureReleasesStateImmediately) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; // cap=1 makes the slot observable + config.default_kv_lease_ttl = 5000; + // Long TTL so the test's pass-fail signal cannot be attributed to + // reaper sweep; only NotifyPromotionFailure could plausibly release. + config.put_start_release_timeout_sec = 300; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k_a", 1024, + seg.segment_name)); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k_b", 1024, + seg.segment_name)); + + auto seg_baseline = service->QuerySegments(seg.segment_name); + ASSERT_TRUE(seg_baseline.has_value()); + const size_t used_baseline = seg_baseline->first; + + // Admit + stage k_a. promotion_in_flight_ goes 0 -> 1. + { + auto r = service->GetReplicaList("k_a"); + ASSERT_TRUE(r.has_value()); + } + auto alloc = service->PromotionAllocStart(seg.client_id, "k_a", 1024, {}); + ASSERT_TRUE(alloc.has_value()); + + // The staged PROCESSING MEMORY buffer is allocated. + auto seg_after_alloc = service->QuerySegments(seg.segment_name); + ASSERT_TRUE(seg_after_alloc.has_value()); + EXPECT_GT(seg_after_alloc->first, used_baseline) + << "AllocStart should commit a buffer in the DRAM allocator"; + + // Holder reports failure (simulating SSD read error after AllocStart + // succeeded). Master must immediately reap the staged replica and + // decrement the slot counter. + auto failure = service->NotifyPromotionFailure(seg.client_id, "k_a"); + ASSERT_TRUE(failure.has_value()) + << "NotifyPromotionFailure on a valid in-flight task from the " + << "legitimate holder must succeed; error=" << failure.error(); + + // The staged buffer must be freed back to the DRAM allocator. If + // this fires, NotifyPromotionFailure did not pop the staged replica + // via EraseReplicaByID — same orphan-replica shape that originally + // motivated the reaper fix. + auto seg_after_release = service->QuerySegments(seg.segment_name); + ASSERT_TRUE(seg_after_release.has_value()); + EXPECT_EQ(seg_after_release->first, used_baseline) + << "NotifyPromotionFailure must release the staged buffer back " + << "to the allocator; otherwise it leaks until the object is " + << "removed or evicted."; + + // The slot must be freed: a second admission on a different key must + // succeed even though queue_limit=1. Without the failure-side + // decrement the cap would stay saturated until reaper TTL. + { + auto r = service->GetReplicaList("k_b"); + ASSERT_TRUE(r.has_value()); + } + auto heartbeat = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(heartbeat.has_value()); + EXPECT_EQ(heartbeat->count("k_b"), 1u) + << "k_b admission must succeed after k_a's failure released the " + << "slot. If this fires, NotifyPromotionFailure did not decrement " + << "promotion_in_flight_, and transient client-side errors " + << "would saturate the queue limit for the full reaper TTL."; + + // Idempotency: repeated failure notification on the same key must be + // safe (return OK without underflowing the counter). + auto failure_again = service->NotifyPromotionFailure(seg.client_id, "k_a"); + EXPECT_TRUE(failure_again.has_value()) + << "Repeated NotifyPromotionFailure should be idempotent (return " + << "OK on already-released task), not error."; + + service->RemoveAll(); +} + +// NotifyPromotionFailure must reject calls from a client that is not the +// holder. Without this gate, any client knowing the key could prematurely +// release a legitimate in-flight promotion's master state and free the +// staged buffer mid-RDMA-write. Mirror of NotifyRejectsNonHolder for the +// Notify-Success path. +TEST_F(PromotionOnHitTest, NotifyFailureRejectsNonHolder) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 5000; + config.put_start_release_timeout_sec = 300; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_cold", + 1024, holder.segment_name)); + + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + auto alloc = + service->PromotionAllocStart(holder.client_id, "k_cold", 1024, {}); + ASSERT_TRUE(alloc.has_value()); + + // Intruder calls Failure with the wrong client_id. + UUID intruder_id = generate_uuid(); + ASSERT_NE(intruder_id, holder.client_id); + auto bad_failure = service->NotifyPromotionFailure(intruder_id, "k_cold"); + ASSERT_FALSE(bad_failure.has_value()) + << "Failure from a non-holder client must be rejected."; + EXPECT_EQ(bad_failure.error(), ErrorCode::INVALID_PARAMS); + + // The legitimate holder must still be able to either commit via + // Notify-Success or release via Notify-Failure on the same task. Use + // Notify-Failure here to exercise the surviving-task path. + auto good_failure = + service->NotifyPromotionFailure(holder.client_id, "k_cold"); + ASSERT_TRUE(good_failure.has_value()) + << "Holder Failure must succeed after a rejected intruder " + << "Failure — the task entry should be untouched by the " + << "rejection path; error=" << good_failure.error(); + + service->RemoveAll(); +} + +// PromotionAllocStart must reject callers that aren't the holder. Mirror +// of the Notify gate. Without this gate a client that drained another's +// promotion_objects queue via PromotionObjectHeartbeat could call +// AllocStart to stage arbitrary DRAM allocations on the destination +// segment, with no path to commit (Notify rejects on holder_id mismatch) +// — they'd just sit pinned until reaper TTL. +TEST_F(PromotionOnHitTest, AllocStartRejectsNonHolder) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 5000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_cold", + 1024, holder.segment_name)); + + auto seg_baseline = service->QuerySegments(holder.segment_name); + ASSERT_TRUE(seg_baseline.has_value()); + const size_t used_baseline = seg_baseline->first; + + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + + UUID intruder_id = generate_uuid(); + ASSERT_NE(intruder_id, holder.client_id); + auto bad_alloc = + service->PromotionAllocStart(intruder_id, "k_cold", 1024, {}); + ASSERT_FALSE(bad_alloc.has_value()) + << "AllocStart from a non-holder client must be rejected — " + << "otherwise an attacker that drained another's queue could " + << "stage arbitrary DRAM allocations."; + EXPECT_EQ(bad_alloc.error(), ErrorCode::INVALID_PARAMS); + + // No buffer was allocated. + auto seg_after_bad = service->QuerySegments(holder.segment_name); + ASSERT_TRUE(seg_after_bad.has_value()); + EXPECT_EQ(seg_after_bad->first, used_baseline) + << "Rejected AllocStart must not have allocated any DRAM."; + + // The legitimate holder must still be able to AllocStart (task + // untouched by the rejection). + auto good_alloc = + service->PromotionAllocStart(holder.client_id, "k_cold", 1024, {}); + ASSERT_TRUE(good_alloc.has_value()) + << "Holder AllocStart on the same task must succeed after a " + << "rejected intruder AllocStart; error=" << good_alloc.error(); + + service->RemoveAll(); +} + +// PromotionAllocStart must reject size that doesn't match the task's +// recorded object_size. The size is captured from the source LOCAL_DISK +// descriptor at task admission; mismatch indicates a buggy caller or +// malicious request and would let the caller request an arbitrary-size +// DRAM buffer (smaller → eventual RDMA-write overflow risk; larger → +// wasted DRAM pinned until reaper TTL). +TEST_F(PromotionOnHitTest, AllocStartRejectsSizeMismatch) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 5000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + constexpr int64_t kRealSize = 1024; + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_cold", + kRealSize, holder.segment_name)); + + auto seg_baseline = service->QuerySegments(holder.segment_name); + ASSERT_TRUE(seg_baseline.has_value()); + const size_t used_baseline = seg_baseline->first; + + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + + // Mismatched-size requests must be rejected, regardless of direction. + for (uint64_t bad_size : {static_cast(kRealSize) / 2, + static_cast(kRealSize) * 4}) { + auto bad_alloc = service->PromotionAllocStart(holder.client_id, + "k_cold", bad_size, {}); + ASSERT_FALSE(bad_alloc.has_value()) + << "AllocStart with size=" << bad_size + << " (task.object_size=" << kRealSize << ") must be rejected."; + EXPECT_EQ(bad_alloc.error(), ErrorCode::INVALID_PARAMS); + + // No buffer must have been staged. + auto seg_after_bad = service->QuerySegments(holder.segment_name); + ASSERT_TRUE(seg_after_bad.has_value()); + EXPECT_EQ(seg_after_bad->first, used_baseline) + << "Rejected AllocStart (size=" << bad_size + << ") must not have allocated any DRAM."; + } + + // Correct size must still work — the task was not consumed by the + // rejections. + auto good_alloc = service->PromotionAllocStart( + holder.client_id, "k_cold", static_cast(kRealSize), {}); + ASSERT_TRUE(good_alloc.has_value()) + << "AllocStart with the correct size must succeed after rejected " + << "size-mismatch attempts; error=" << good_alloc.error(); + + service->RemoveAll(); +} + +// When a holder client expires, ClientMonitorFunc must clean up its +// dangling promotion_tasks entries and decrement the global in-flight +// counter. Without this cleanup, the entries stay pinned until reaper +// TTL — and on a rolling restart of many holders the cluster-wide cap +// promotion_queue_limit_ saturates and blocks all new admissions for +// the full TTL. +// +// Test mechanism: short client_live_ttl_sec, admit a promotion, stop +// pinging, wait for ClientMonitorFunc to expire the client and call +// ClearInvalidHandles. Then assert promotion_in_flight_ is back to 0 +// by attempting a second admission with queue_limit=1 on a fresh +// client. +TEST_F(PromotionOnHitTest, ClientExpiryClearsPromotionTask) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; // cap=1 makes the slot observable + config.default_kv_lease_ttl = 5000; + // Long task TTL so that any clearing we see must come from + // ClearInvalidHandles, not from the promotion-task reaper. + config.put_start_release_timeout_sec = 300; + // Short client TTL so expiration is fast. + config.client_live_ttl_sec = 1; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_cold", + 1024, holder.segment_name)); + + // Admit the promotion. promotion_in_flight_ goes 0 -> 1. + { + auto r = service->GetReplicaList("k_cold"); + ASSERT_TRUE(r.has_value()); + } + + // Sanity: with queue_limit=1 the cap is saturated. A second + // different-shard admission must be rejected right now. + auto second_holder = PrepareSegment( + *service, "seg_b", kDefaultSegmentBase + seg_size, seg_size); + // Promote second_holder into ok_client_ via ReMountSegment so its + // LOCAL_DISK replicas survive any ClearInvalidHandles run triggered + // by the first holder's expiry. MountSegment alone does not register + // the client as alive (only ReMountSegment does), and + // CleanupStaleHandles uses ok_client_ to decide which LOCAL_DISK + // replicas to erase — without this, second_holder's k_other replica + // would be wiped alongside the first holder's k_cold replica when + // ClearInvalidHandles runs. + { + Segment seg_b = + MakeSegment("seg_b", kDefaultSegmentBase + seg_size, seg_size); + seg_b.id = second_holder.segment_id; + std::vector segs{seg_b}; + auto remount = service->ReMountSegment(segs, second_holder.client_id); + ASSERT_TRUE(remount.has_value()) << "ReMount failed"; + } + ASSERT_TRUE(InjectLocalDiskReplica(*service, second_holder.client_id, + "k_other", 1024, + second_holder.segment_name)); + { + auto r = service->GetReplicaList("k_other"); + ASSERT_TRUE(r.has_value()); + } + auto pending_pre = + service->PromotionObjectHeartbeat(second_holder.client_id); + ASSERT_TRUE(pending_pre.has_value()); + EXPECT_EQ(pending_pre->count("k_other"), 0u) + << "Sanity: queue_limit=1 should block the second admission " + << "while the first task is in flight."; + + // Wait for the first holder to expire while keeping the second + // holder alive via periodic Pings. ClientMonitorFunc runs every + // kClientMonitorSleepMs (1s) and expires clients whose last ping is + // older than client_live_ttl_sec (1s); we ping second_holder every + // 200ms so it stays alive across the 4s wait. Without these pings + // both holders would expire together and the second holder's + // LOCAL_DISK segment would be unmounted — making "k_other" un- + // admittable for reasons unrelated to the promotion-task slot. + { + const auto deadline = + std::chrono::steady_clock::now() + std::chrono::seconds(4); + while (std::chrono::steady_clock::now() < deadline) { + (void)service->Ping(second_holder.client_id); + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + } + } + + // ClearInvalidHandles should have erased the holder's LOCAL_DISK + // source replica AND (with the fix) the promotion_tasks entry, + // decrementing the global in-flight counter. Re-admit a promotion + // on the second holder; with queue_limit=1 this can only succeed if + // the slot was freed. + { + auto r = service->GetReplicaList("k_other"); + ASSERT_TRUE(r.has_value()) + << "GetReplicaList(k_other) failed with error=" << r.error(); + } + auto pending_post = + service->PromotionObjectHeartbeat(second_holder.client_id); + ASSERT_TRUE(pending_post.has_value()); + EXPECT_EQ(pending_post->count("k_other"), 1u) + << "After the holder expired, ClearInvalidHandles must have " + << "erased its promotion_tasks entry and decremented " + << "promotion_in_flight_. Otherwise the global cap remains " + << "saturated by the dead holder's task for " + << "put_start_release_timeout_sec_ seconds, and this admission " + << "is dropped."; + + service->RemoveAll(); +} + +// promotion_admission_threshold=0 would silently bypass the frequency +// gate (`freq < 0` is never true for uint8_t freq), letting every Get +// on a LOCAL_DISK-only key admit a promotion. master.cpp clamps the +// flag at parse time, and MasterService's constructor adds a +// defense-in-depth clamp for direct-construction paths (tests, +// embedded users). Verify the clamp lifts 0 → 1. +TEST_F(PromotionOnHitTest, AdmissionThresholdZeroClampsToOne) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 0; // would bypass the gate + auto service = std::make_unique(config); + EXPECT_EQ(GetPromotionAdmissionThresholdForTesting(service.get()), 1u) + << "threshold=0 must be clamped to 1; otherwise every Get-on-" + << "LOCAL_DISK admits a promotion and the frequency gate is " + << "silently disabled."; +} + +// promotion_admission_threshold above the CountMinSketch saturating +// max (255) would make the gate unreachable (freq saturates at 255, +// `255 < 256` is true forever → no admissions). Verify the constructor +// clamps high too. +TEST_F(PromotionOnHitTest, AdmissionThresholdAboveMaxClampsToMax) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1000; // > 255 + auto service = std::make_unique(config); + EXPECT_EQ(GetPromotionAdmissionThresholdForTesting(service.get()), 255u) + << "threshold above the CountMinSketch counter max (255) must " + << "be clamped to 255; otherwise the gate is unreachable."; +} + +// Remove(force=true) on a key with an in-flight PromotionTask must +// drop the task entry alongside the metadata, so promotion_in_flight_ +// is decremented immediately rather than pinned for ~10 min until the +// reaper sweeps. With queue_limit=1, the test admits one task, removes +// the key, then admits a second task on a different key — only +// succeeds if the slot was freed. +TEST_F(PromotionOnHitTest, RemoveErasesPromotionTask) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; // makes the slot observable + config.default_kv_lease_ttl = 5000; + // Long task TTL so any cap-slot reclaim must come from the + // metadata-erase path, not the reaper. + config.put_start_release_timeout_sec = 300; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_first", + 1024, holder.segment_name)); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_second", + 1024, holder.segment_name)); + + // Admit task 1. + { + auto r = service->GetReplicaList("k_first"); + ASSERT_TRUE(r.has_value()); + } + { + auto pending = service->PromotionObjectHeartbeat(holder.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->count("k_first"), 1u); + } + + // Remove k_first with force=true. With the fix, this also wipes + // k_first's promotion_tasks entry and decrements + // promotion_in_flight_ back to 0. + auto rm = service->Remove("k_first", /*force=*/true); + ASSERT_TRUE(rm.has_value()) + << "Remove should succeed; error=" << rm.error(); + + // Now admit a different key. With queue_limit=1, this only succeeds + // if the slot was freed by Remove. Without the in-flight cleanup, + // it would stay pinned for the full 300s TTL. + { + auto r = service->GetReplicaList("k_second"); + ASSERT_TRUE(r.has_value()); + } + auto pending_post = service->PromotionObjectHeartbeat(holder.client_id); + ASSERT_TRUE(pending_post.has_value()); + EXPECT_EQ(pending_post->count("k_second"), 1u) + << "k_second must be admittable after Remove of k_first — Remove " + << "must erase the in-flight promotion_tasks entry and decrement " + << "promotion_in_flight_, otherwise queue_limit=1 stays saturated."; + + service->RemoveAll(); +} + +// RemoveByRegex on a key with an in-flight PromotionTask must drop the +// task entry, same as Remove. Mirror of RemoveErasesPromotionTask, but +// exercises the regex path which iterates shards directly without an +// accessor object. +TEST_F(PromotionOnHitTest, RemoveByRegexErasesPromotionTask) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; + config.default_kv_lease_ttl = 5000; + config.put_start_release_timeout_sec = 300; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "regex_k1", + 1024, holder.segment_name)); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "other_k2", + 1024, holder.segment_name)); + + // Admit task on regex_k1. + { + auto r = service->GetReplicaList("regex_k1"); + ASSERT_TRUE(r.has_value()); + } + + // RemoveByRegex matches regex_k1 only. + auto removed = service->RemoveByRegex("^regex_", /*force=*/true); + ASSERT_TRUE(removed.has_value()) + << "RemoveByRegex should succeed; error=" << removed.error(); + EXPECT_EQ(removed.value(), 1) << "exactly one key (regex_k1) should match"; + + // Slot must be free — admit on other_k2 (different shard or same, + // doesn't matter because counter is global). + { + auto r = service->GetReplicaList("other_k2"); + ASSERT_TRUE(r.has_value()); + } + auto pending_post = service->PromotionObjectHeartbeat(holder.client_id); + ASSERT_TRUE(pending_post.has_value()); + EXPECT_EQ(pending_post->count("other_k2"), 1u) + << "other_k2 must be admittable after RemoveByRegex of regex_k1 " + << "— RemoveByRegex must erase the in-flight promotion_tasks " + << "entry. Otherwise queue_limit=1 stays saturated."; + + service->RemoveAll(); +} + +} // namespace mooncake::test + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/mooncake-wheel/tests/test_promotion_on_hit.py b/mooncake-wheel/tests/test_promotion_on_hit.py new file mode 100644 index 00000000..19efc177 --- /dev/null +++ b/mooncake-wheel/tests/test_promotion_on_hit.py @@ -0,0 +1,492 @@ +"""Python-binding test for the L2->L1 promotion-on-hit feature. + +Mirror of ``test_offload_on_eviction.py``: that test asserts data flows +DRAM -> SSD on eviction; this test asserts the reverse — once an object +exists only on LOCAL_DISK and a client reads it enough times to clear the +admission threshold, the master enqueues a promotion task and the client's +next heartbeat tick stages a fresh MEMORY replica. + +Test scenario: + 1. Push enough data to overflow DRAM, so eviction + offload turns warm + keys into LOCAL_DISK-only objects. + 2. Identify one such key from the replica descriptors. + 3. Read it repeatedly to clear ``promotion_admission_threshold``. + 4. Wait long enough for one master heartbeat (which queues the task) plus + one FileStorage heartbeat (which executes it). + 5. Assert the key now also has a MEMORY replica. + +Prerequisites: + - ``mooncake_master`` running with master config containing: + ``--enable_offload=true`` + ``--offload_on_evict=true`` + ``--promotion_on_hit=true`` + ``--promotion_admission_threshold=1`` (any positive int; we send + enough reads to clear up to 4) + ``--root_fs_dir=`` (so the master tells the client to init + its FileStorage; without this, no offload heartbeat runs and + no LOCAL_DISK replicas are ever created) + - ``MOONCAKE_OFFLOAD_FILE_STORAGE_PATH=`` set on the client side. + - ``MOONCAKE_OFFLOAD_BUCKET_KEYS_LIMIT=10`` and + ``MOONCAKE_OFFLOAD_BUCKET_SIZE_LIMIT_BYTES=10485760`` on the client. + The default bucket-flush thresholds (500 keys / 256 MB) are too high + for a single-process test workload; the client's BucketStorageBackend + would buffer everything in the ungrouped pool and never write to disk. +""" + +import os +import statistics +import time +import unittest +from mooncake.store import MooncakeDistributedStore + +DEFAULT_DEFAULT_KV_LEASE_TTL = 5000 # ms +default_kv_lease_ttl = int( + os.getenv("DEFAULT_KV_LEASE_TTL", DEFAULT_DEFAULT_KV_LEASE_TTL) +) + +# Keep the segment small so we can overflow it cheaply in CI. +# 32 MB — small enough that the 96 x 1 MB workload reliably overflows past +# the eviction high watermark even with OffsetAllocator block padding. +# Larger values (e.g. 64 MB) sometimes fit the full workload because the +# allocator's effective capacity exceeds the nominal segment size. +SEGMENT_SIZE = int(os.getenv("SEGMENT_SIZE_BYTES", str(32 * 1024 * 1024))) +LOCAL_BUFFER_SIZE = int(os.getenv("LOCAL_BUFFER_SIZE_BYTES", str(64 * 1024 * 1024))) + +# Wait window for a full master+client heartbeat round-trip. The default +# heartbeat interval is 10s on both sides, so 25s is comfortably > 2 ticks. +PROMOTION_WAIT_SECONDS = int(os.getenv("PROMOTION_WAIT_SECONDS", "25")) + + +def get_client(store): + protocol = os.getenv("PROTOCOL", "tcp") + device_name = os.getenv("DEVICE_NAME", "") + local_hostname = os.getenv("LOCAL_HOSTNAME", "localhost") + metadata_server = os.getenv("MC_METADATA_SERVER", "P2PHANDSHAKE") + master_server_address = os.getenv("MASTER_SERVER", "127.0.0.1:50051") + + retcode = store.setup( + local_hostname, + metadata_server, + SEGMENT_SIZE, + LOCAL_BUFFER_SIZE, + protocol, + device_name, + master_server_address, + None, # engine + True, # enable_ssd_offload — required for FileStorage / LOCAL_DISK + ) + if retcode: + raise RuntimeError(f"Failed to setup store client. Return code: {retcode}") + + +def _replica_types(descs, key): + """Return a list of replica type tags ('MEMORY', 'LOCAL_DISK', 'DISK') + for a key from a batch_get_replica_desc result. Tolerant to either + list-of-info or single-info shape.""" + infos = descs.get(key) if isinstance(descs, dict) else None + if infos is None: + return [] + if not isinstance(infos, (list, tuple)): + infos = [infos] + tags = [] + for info in infos: + if hasattr(info, "is_memory_replica") and info.is_memory_replica(): + tags.append("MEMORY") + elif hasattr(info, "is_local_disk_replica") and info.is_local_disk_replica(): + tags.append("LOCAL_DISK") + elif hasattr(info, "is_disk_replica") and info.is_disk_replica(): + tags.append("DISK") + else: + tags.append("UNKNOWN") + return tags + + +class TestPromotionOnHit(unittest.TestCase): + """Python-binding test for the L2->L1 promotion-on-hit behavioral + contract.""" + + @classmethod + def setUpClass(cls): + cls.store = MooncakeDistributedStore() + get_client(cls.store) + + def test_promotion_after_repeated_hits(self): + """A LOCAL_DISK-only key must regain a MEMORY replica after + clearing the admission threshold via repeated reads. + + A failure here either means the master is not running with + ``promotion_on_hit: true``, or the FileStorage promotion executor + did not run / did not complete byte movement before the timeout.""" + + VALUE_SIZE = 1024 * 1024 # 1 MB + # Enough keys to comfortably overflow a 64 MB segment and trigger + # offload-on-evict for at least the older portion of the workload. + NUM_KEYS = 96 + + timestamp = int(time.time()) + keys = [f"poh_{i}_{timestamp}" for i in range(NUM_KEYS)] + reference = {} + + try: + # Phase 1: overflow DRAM, force offload-on-evict. + for key in keys: + value = os.urandom(VALUE_SIZE) + retcode = self.store.put(key, value) + # NO_AVAILABLE_HANDLE-style failures (-200) under pressure + # are expected and not fatal — we just need *some* keys to + # successfully land on LOCAL_DISK. + if retcode == 0: + reference[key] = value + + self.assertGreater( + len(reference), 0, "No PUTs succeeded — cannot run promotion test" + ) + + # Phase 2: wait long enough for offload heartbeat to flush the + # evicted keys to LOCAL_DISK. + time.sleep(PROMOTION_WAIT_SECONDS) + + descs = self.store.batch_get_replica_desc(list(reference.keys())) + type_hist = {} + cold_key = None + for key in reference.keys(): + types = _replica_types(descs, key) + hist_key = ",".join(sorted(set(types))) + type_hist[hist_key] = type_hist.get(hist_key, 0) + 1 + if ( + cold_key is None + and types + and all("MEMORY" not in t for t in types) + and any("LOCAL_DISK" in t for t in types) + ): + cold_key = key + print(f"replica-type histogram after phase 2: {type_hist}") + + self.assertIsNotNone( + cold_key, + "No LOCAL_DISK-only key found after eviction — is " + "offload_on_evict=true and the segment small enough to " + "overflow? master config / SEGMENT_SIZE_BYTES env may need " + "tuning. Histogram above shows the actual replica state.", + ) + + # Phase 3: clear the admission threshold via per-key reads. + # ``store.get`` goes through ``Client::Query`` → master's + # ``GetReplicaList``, so each call fires the promotion gate + # once; 4 calls comfortably clear any reasonable admission + # threshold. We also assert bit-exact bytes back, exercising + # the LOCAL_DISK read path end-to-end via the offload-RPC + # route (this transitively guards the read-side fix the + # promotion feature relies on; reads only return the + # offloaded bytes if the peer's offload-RPC + segment-name + # resolution work). + expected_bytes = reference[cold_key] + for _ in range(4): + got = self.store.get(cold_key) + self.assertEqual( + got, + expected_bytes, + f"store.get on LOCAL_DISK-only key {cold_key} returned " + f"wrong/empty bytes (got len={len(got) if got else 0}, " + f"expected len={len(expected_bytes)}). The LOCAL_DISK " + f"read path is broken — promotion cannot be tested.", + ) + + # Phase 4: wait for the master to enqueue the promotion task and + # for the client's next FileStorage heartbeat to execute it. + time.sleep(PROMOTION_WAIT_SECONDS) + + descs_after = self.store.batch_get_replica_desc([cold_key]) + types_after = _replica_types(descs_after, cold_key) + print(f"replica types for {cold_key} after promotion: {types_after}") + self.assertTrue( + "MEMORY" in types_after, + f"Expected a MEMORY replica for {cold_key} after promotion, " + f"got types={types_after}. Either promotion_on_hit is not " + f"enabled in the master, the admission threshold was not " + f"cleared, or the heartbeat tick did not fire within " + f"{PROMOTION_WAIT_SECONDS}s.", + ) + + # Phase 5: post-promotion bytes-back AND prove MEMORY was the + # replica actually served. SelectBestReplica should prefer the + # newly-COMPLETE MEMORY replica over the older LOCAL_DISK; if + # it does, no offload-RPC call is issued for this read. + # + # offload_rpc_read_count counts every invocation of + # batch_get_into_offload_object_internal — the single + # chokepoint for LOCAL_DISK reads served via peer offload-RPC. + # We snapshot the counter, do a read, then assert the counter + # didn't move. Bytes-back alone wouldn't distinguish MEMORY + # from LOCAL_DISK (the offload-RPC path returns correct bytes + # too); the counter delta is the hard correctness signal that + # the MEMORY path was used. The BenchPromotionLatency class + # below offers a heavier latency-based view of the same + # invariant when richer numbers are wanted. + offload_count_before = self.store.get_offload_rpc_read_count() + self.assertEqual( + self.store.get(cold_key), + expected_bytes, + f"post-promotion store.get on {cold_key} returned wrong " + f"bytes; the new MEMORY replica is metadata-visible but " + f"not byte-correct. NotifyPromotionSuccess flipped " + f"PROCESSING -> COMPLETE before the TE write finished, " + f"or the TE write landed at the wrong address.", + ) + offload_count_after = self.store.get_offload_rpc_read_count() + self.assertEqual( + offload_count_after, + offload_count_before, + f"post-promotion store.get on {cold_key} bumped the " + f"offload-RPC counter from {offload_count_before} to " + f"{offload_count_after}: it served from LOCAL_DISK SSD, " + f"not the freshly-promoted MEMORY replica. The RFC's " + f"'subsequent reads avoid SSD' invariant is broken; " + f"check SelectBestReplica preference and the post-" + f"promotion replica order in metadata.", + ) + finally: + for key in keys: + try: + self.store.remove(key) + except Exception: + pass + time.sleep(default_kv_lease_ttl / 1000 + 0.5) + + +class TestPromotionDoesNotFire(unittest.TestCase): + """Negative-case e2e. With workloads that don't produce LOCAL_DISK + replicas, promotion has nothing to do and must not spuriously stage + MEMORY replicas. This is the inverse invariant of the positive test + and the only Python-level guard against: + - regressions that auto-create LOCAL_DISK at PutEnd (would silently + break offload semantics and trigger spurious promotions), + - regressions that fire promotion without a LOCAL_DISK source. + """ + + @classmethod + def setUpClass(cls): + cls.store = MooncakeDistributedStore() + get_client(cls.store) + + def test_below_watermark_workload_stays_memory_only(self): + VALUE_SIZE = 1024 # 1 KB + NUM_KEYS = 16 # 16 KB total, well below the 32 MB segment + + timestamp = int(time.time()) + keys = [f"neg_poh_small_{i}_{timestamp}" for i in range(NUM_KEYS)] + + try: + for key in keys: + value = os.urandom(VALUE_SIZE) + self.assertEqual(self.store.put(key, value), 0) + + # One heartbeat cycle is plenty for any spurious offload to + # have surfaced. + time.sleep(PROMOTION_WAIT_SECONDS) + + descs = self.store.batch_get_replica_desc(keys) + unexpected = [] + for key in keys: + types = _replica_types(descs, key) + if not any(t == "MEMORY" for t in types): + unexpected.append((key, types)) + if any(t == "LOCAL_DISK" for t in types): + unexpected.append((key, types)) + + self.assertEqual( + unexpected, + [], + f"Below-watermark workload produced unexpected non-MEMORY " + f"or LOCAL_DISK replicas: {unexpected[:5]}. Either " + f"eviction is firing prematurely or PutEnd is auto-" + f"creating LOCAL_DISK.", + ) + finally: + for key in keys: + try: + self.store.remove(key) + except Exception: + pass + time.sleep(default_kv_lease_ttl / 1000 + 0.5) + + +@unittest.skipUnless( + os.getenv("MC_BENCH_PROMOTION_LATENCY"), + "opt-in benchmark — set MC_BENCH_PROMOTION_LATENCY=1 to run. " + "Not part of CI; gives p50/p95/p99 latency comparison of LOCAL_DISK " + "reads (pre-promotion) vs MEMORY reads (post-promotion).", +) +class BenchPromotionLatency(unittest.TestCase): + """Ad-hoc latency benchmark for the L2->L1 promotion feature. + + Runs the same overflow → eviction → promote workflow as + ``TestPromotionOnHit`` but: + - times every read with ``time.perf_counter``; + - aborts the pre-promotion sample loop the moment a read serves + from MEMORY (detected via the offload-RPC counter) to keep the + LOCAL_DISK distribution uncontaminated; + - prints p50/p95/p99 + min/max for both phases and a per-percentile + speedup table; + - asserts a conservative 1.3x p50 speedup so a regression where + MEMORY isn't being preferred breaks the bench. + + Sample size is controlled by ``LATENCY_SAMPLES`` (default 1000). + For larger samples (e.g. N=10000) bump + ``MOONCAKE_OFFLOAD_HEARTBEAT_INTERVAL_SECONDS`` so the worker + doesn't race the pre-promotion read loop. + + Expected speedup on docker tmpfs with 1 MiB objects is ~2x at every + percentile (most wall-time is Python/C++ marshalling, not the + SSD-vs-DRAM cost gap). On real SSDs + RDMA NICs expect 10-50x. + """ + + @classmethod + def setUpClass(cls): + cls.store = MooncakeDistributedStore() + get_client(cls.store) + + def test_latency_comparison(self): + VALUE_SIZE = 1024 * 1024 # 1 MB + NUM_KEYS = 96 + + timestamp = int(time.time()) + keys = [f"bench_poh_{i}_{timestamp}" for i in range(NUM_KEYS)] + reference = {} + + try: + # Overflow DRAM, force offload-on-evict. + for key in keys: + value = os.urandom(VALUE_SIZE) + if self.store.put(key, value) == 0: + reference[key] = value + self.assertGreater(len(reference), 0) + + time.sleep(PROMOTION_WAIT_SECONDS) + + descs = self.store.batch_get_replica_desc(list(reference.keys())) + cold_key = None + for key in reference.keys(): + types = _replica_types(descs, key) + if ( + cold_key is None + and types + and all("MEMORY" not in t for t in types) + and any("LOCAL_DISK" in t for t in types) + ): + cold_key = key + self.assertIsNotNone(cold_key, "no LOCAL_DISK-only key after eviction") + + # Pre-promotion: timed LOCAL_DISK reads, bail when a read + # serves from MEMORY (= worker raced the loop). + N = int(os.getenv("LATENCY_SAMPLES", "1000")) + expected_bytes = reference[cold_key] + pre_latencies_ms = [] + for i in range(N): + count_before = self.store.get_offload_rpc_read_count() + t0 = time.perf_counter() + got = self.store.get(cold_key) + t1 = time.perf_counter() + count_after = self.store.get_offload_rpc_read_count() + self.assertEqual(got, expected_bytes) + if count_after == count_before: + print( + f" pre-promotion loop: promotion fired at " + f"sample {i}/{N}; stopping to keep the " + f"LOCAL_DISK sample uncontaminated" + ) + break + pre_latencies_ms.append((t1 - t0) * 1000.0) + self.assertGreater( + len(pre_latencies_ms), + 1, + "collected <2 pre-promotion samples — promotion fired " + "before measurement could capture LOCAL_DISK latency. " + "Bump MOONCAKE_OFFLOAD_HEARTBEAT_INTERVAL_SECONDS or " + "lower LATENCY_SAMPLES.", + ) + + # Wait for promotion to complete. + time.sleep(PROMOTION_WAIT_SECONDS) + descs_after = self.store.batch_get_replica_desc([cold_key]) + types_after = _replica_types(descs_after, cold_key) + self.assertIn("MEMORY", types_after) + + # Post-promotion: timed MEMORY reads; the counter must stay + # put (otherwise at least one read still hit LOCAL_DISK). + count_before = self.store.get_offload_rpc_read_count() + post_latencies_ms = [] + for _ in range(N): + t0 = time.perf_counter() + got = self.store.get(cold_key) + t1 = time.perf_counter() + self.assertEqual(got, expected_bytes) + post_latencies_ms.append((t1 - t0) * 1000.0) + count_after = self.store.get_offload_rpc_read_count() + self.assertEqual( + count_after, + count_before, + f"post-promotion reads bumped the offload-RPC counter " + f"from {count_before} to {count_after}: at least one " + f"read served from LOCAL_DISK SSD instead of MEMORY.", + ) + + def _pcts(samples): + """{p: value_ms} for p in [50, 95, 99].""" + if len(samples) >= 2: + cuts = statistics.quantiles(samples, n=100) + return {50: cuts[49], 95: cuts[94], 99: cuts[98]} + v = samples[0] if samples else 0.0 + return {50: v, 95: v, 99: v} + + def _speedup(a, b): + return a / b if b > 0 else float("inf") + + pre = _pcts(pre_latencies_ms) + post = _pcts(post_latencies_ms) + + print() + print(f"=== promotion latency comparison " + f"(cold_key={cold_key}) ===") + print( + f"pre-promotion (LOCAL_DISK via offload-RPC): " + f"n={len(pre_latencies_ms)}, " + f"p50={pre[50]:.2f} ms, p95={pre[95]:.2f} ms, " + f"p99={pre[99]:.2f} ms, " + f"min={min(pre_latencies_ms):.2f} ms, " + f"max={max(pre_latencies_ms):.2f} ms" + ) + print( + f"post-promotion (MEMORY direct): " + f"n={len(post_latencies_ms)}, " + f"p50={post[50]:.2f} ms, p95={post[95]:.2f} ms, " + f"p99={post[99]:.2f} ms, " + f"min={min(post_latencies_ms):.2f} ms, " + f"max={max(post_latencies_ms):.2f} ms" + ) + print( + f"speedup: p50={_speedup(pre[50], post[50]):.1f}x " + f"| p95={_speedup(pre[95], post[95]):.1f}x " + f"| p99={_speedup(pre[99], post[99]):.1f}x" + ) + + self.assertLess( + post[50], + pre[50] / 1.3, + f"post-promotion p50 latency ({post[50]:.2f} ms) is not " + f"meaningfully lower than pre-promotion p50 " + f"({pre[50]:.2f} ms). Real-hardware speedup is typically " + f"10–50x; failing this 1.3x check means MEMORY likely " + f"isn't being preferred.", + ) + finally: + for key in keys: + try: + self.store.remove(key) + except Exception: + pass + time.sleep(default_kv_lease_ttl / 1000 + 0.5) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/run_tests.sh b/scripts/run_tests.sh index 2fc703b5..32ad55bd 100755 --- a/scripts/run_tests.sh +++ b/scripts/run_tests.sh @@ -68,6 +68,39 @@ else echo "Skipping test: MOONCAKE_STORAGE_ROOT_DIR environment variable is not set" fi +if [ -n "$TEST_PROMOTION_ON_HIT" ]; then + TEST_ROOT_DIR="/tmp/mooncake_test_promotion" + mkdir -p $TEST_ROOT_DIR + echo "Running L2->L1 promotion-on-hit e2e test..." + # offload_on_evict drives the prerequisite SSD-only state; promotion_on_hit + # turns the read path into a promotion trigger; threshold=1 makes the test + # deterministic. --root_fs_dir is required so the master returns a non- + # empty fsdir from GetStorageConfig, which is the trigger that initializes + # the client's FileStorage (and therefore the offload heartbeat). + mooncake_master \ + --default_kv_lease_ttl=500 \ + --root_fs_dir=$TEST_ROOT_DIR \ + --enable_offload=true \ + --offload_on_evict=true \ + --promotion_on_hit=true \ + --promotion_admission_threshold=1 & + MASTER_PID=$! + sleep 1 + # Lower bucket-flush thresholds so the test workload (~64 MB) actually + # writes to disk rather than sitting in the bucket backend's ungrouped + # pool until the default 500-key / 256-MB bucket fills. + MC_METADATA_SERVER=http://127.0.0.1:8080/metadata \ + DEFAULT_KV_LEASE_TTL=500 \ + MOONCAKE_OFFLOAD_FILE_STORAGE_PATH=$TEST_ROOT_DIR \ + MOONCAKE_OFFLOAD_BUCKET_KEYS_LIMIT=10 \ + MOONCAKE_OFFLOAD_BUCKET_SIZE_LIMIT_BYTES=10485760 \ + python test_promotion_on_hit.py + kill $MASTER_PID || true + rm -rf $TEST_ROOT_DIR +else + echo "Skipping test: TEST_PROMOTION_ON_HIT environment variable is not set" +fi + echo "Running CXL protocol test (test_distributed_object_store_cxl.py)..." killall mooncake_master || true sleep 2 -- 2.34.1 From b4ccdc3082d2def865ecf1b5e68bc511025da40e Mon Sep 17 00:00:00 2001 From: Shangming Cai Date: Thu, 21 May 2026 17:30:46 +0800 Subject: [PATCH 108/382] Use sudo -E for make install in release workflow (#2169) * Use sudo -E for make install in release workflow --- .github/workflows/release-cuda13.yaml | 2 +- .github/workflows/release.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/release-cuda13.yaml b/.github/workflows/release-cuda13.yaml index ed9ec443..0cd51990 100644 --- a/.github/workflows/release-cuda13.yaml +++ b/.github/workflows/release-cuda13.yaml @@ -73,7 +73,7 @@ jobs: export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LIBRARY_PATH cd build make -j - sudo make install + sudo -E make install shell: bash - name: Build nvlink_allocator.so diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index edef88fa..a9169592 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -72,7 +72,7 @@ jobs: export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:LIBRARY_PATH cd build make -j - sudo make install + sudo -E make install shell: bash - name: Build nvlink_allocator.so -- 2.34.1 From 7bf33267e2f3d964be68e1e2b210473b2681a402 Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Thu, 21 May 2026 20:14:21 +0800 Subject: [PATCH 109/382] [Doc] update WITH_NVIDIA_PEERMEM from cmake flag to runtime env var (PR #2066) (#2164) Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/832212c9-6f77-4b98-bbe9-1a726e478400 Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: stmatengss <11641725+stmatengss@users.noreply.github.com> --- docs/source/design/transfer-engine/index.md | 1 + docs/source/getting_started/build.md | 4 ++-- docs/source/zh_archive/build.md | 4 ++-- docs/source/zh_archive/run-examples.md | 5 ++--- docs/source/zh_archive/transfer-engine.md | 1 + 5 files changed, 8 insertions(+), 7 deletions(-) diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index df53d177..0796d06e 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -304,6 +304,7 @@ For advanced users, TransferEngine provides the following advanced runtime optio - `MC_MIN_RPC_PORT` Specifies the minimum port number for RPC service. The default value is 15000. - `MC_MAX_RPC_PORT` Specifies the maximum port number for RPC service. The default value is 17000. - `MC_PATH_ROUNDROBIN` Use round-robin mode in the RDMA path selection. This may be beneficial for transferring large bulks. +- `WITH_NVIDIA_PEERMEM` When set to `1`, `ON`, or `TRUE`, Mooncake uses `ibv_reg_mr()` directly for GPU memory registration (requires the `nvidia-peermem` kernel module). By default (unset or `0`), Mooncake uses the DMA-BUF path which does not require `nvidia-peermem`. - `MC_ENDPOINT_STORE_TYPE` Choose FIFO Endpoint Store (`FIFO`) or Sieve Endpoint Store (`SIEVE`), default is `SIEVE`. - `MC_TCP_ENABLE_CONNECTION_POOL` Enable TCP Connection Pool to avoid excessive sockets. diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index e7a0802a..bc529d59 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -100,12 +100,12 @@ pip install mooncake-transfer-engine-non-cuda ``` 2. If you want to compile the GPUDirect support module, first follow the instructions in https://docs.nvidia.com/cuda/cuda-installation-guide-linux/ to install CUDA (ensure to enable `nvidia-fs` for proper `cuFile` module compilation). After that: - 1) Follow Section 3.7 in https://docs.nvidia.com/cuda/gpudirect-rdma/ to install `nvidia-peermem` for enabling GPU-Direct RDMA - 2) Configure `LIBRARY_PATH` and `LD_LIBRARY_PATH` to ensure linking of `cuFile`, `cudart`, and other libraries during compilation: + 1) Configure `LIBRARY_PATH` and `LD_LIBRARY_PATH` to ensure linking of `cuFile`, `cudart`, and other libraries during compilation: ```bash export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda/lib64 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64 ``` + > **Note:** By default, Mooncake uses the DMA-BUF path for GPU-Direct RDMA, which does **not** require the `nvidia-peermem` kernel module. If you prefer the legacy `ibv_reg_mr` path (which requires `nvidia-peermem`), set the runtime environment variable `WITH_NVIDIA_PEERMEM=1` before starting Mooncake. See Section 3.7 of https://docs.nvidia.com/cuda/gpudirect-rdma/ for instructions on installing `nvidia-peermem`. 3. If you want to compile the Moore Mthreads GPUDirect support module, first follow the instructions in https://docs.mthreads.com/musa-sdk/musa-sdk-doc-online/install_guide to install MUSA. After that: 1) Install `mthreads-peermem` for enabling GPU-Direct RDMA diff --git a/docs/source/zh_archive/build.md b/docs/source/zh_archive/build.md index 6e4f8dfb..7a49643f 100644 --- a/docs/source/zh_archive/build.md +++ b/docs/source/zh_archive/build.md @@ -95,12 +95,12 @@ ``` 2. 如果你要编译Nvidia GPUDirect 支持模块,首先需按照 https://docs.nvidia.com/cuda/cuda-installation-guide-linux/ 的指引安装 CUDA (确保启用 `nvidia-fs` 以正确编译 `cuFile` 模块)。之后: - 1) 按照 https://docs.nvidia.com/cuda/gpudirect-rdma/ 的第 3.7 节说明安装 `nvidia-peermem` 以启用 GPU-Direct RDMA - 2) 配置 `LIBRARY_PATH` 和 `LD_LIBRARY_PATH` 以确保编译过程期间链入 `cuFile`, `cudart` 等库: + 1) 配置 `LIBRARY_PATH` 和 `LD_LIBRARY_PATH` 以确保编译过程期间链入 `cuFile`, `cudart` 等库: ```bash export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda/lib64 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64 ``` + > **注意:** 默认情况下,Mooncake 使用 DMA-BUF 路径进行 GPU-Direct RDMA,**无需** `nvidia-peermem` 内核模块。如需使用依赖 `nvidia-peermem` 的传统 `ibv_reg_mr` 路径,请在启动 Mooncake 前设置运行时环境变量 `WITH_NVIDIA_PEERMEM=1`。安装 `nvidia-peermem` 的说明见 https://docs.nvidia.com/cuda/gpudirect-rdma/ 第 3.7 节。 3. 如果你要编译Moore Threads GPUDirect RDMA 支持模块,首先需按照 https://docs.mthreads.com/musa-sdk/musa-sdk-doc-online/install_guide 的指引安装 MUSA SDK。之后: 1) 安装 `mthreads-peermem` 以启用 GPU-Direct RDMA diff --git a/docs/source/zh_archive/run-examples.md b/docs/source/zh_archive/run-examples.md index 32aee534..8c72fa97 100644 --- a/docs/source/zh_archive/run-examples.md +++ b/docs/source/zh_archive/run-examples.md @@ -34,13 +34,12 @@ Mooncake 目前仅支持 Linux 操作系统,并且依赖以下软件: 首先按照 https://docs.nvidia.com/cuda/cuda-installation-guide-linux/ 中的说明安装 cuda(需要勾选 nvidia-fs 选项, 以便于 cufile 的正常使用),之后: - 1) 参考 https://docs.nvidia.com/cuda/gpudirect-rdma/ 的 3.7 节,安装 nvidia-peermem 以启用 gpu-direct RDMA: - - 2) 配置 `LIBRARY_PATH` 和 `LD_LIBRARY_PATH`,用于编译和运行时链接 cufile, cudart 等库: + 1) 配置 `LIBRARY_PATH` 和 `LD_LIBRARY_PATH`,用于编译和运行时链接 cufile, cudart 等库: ```bash export $LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda/lib64 export $LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64 ``` + > **注意:** 默认情况下,Mooncake 使用 DMA-BUF 路径进行 GPU-Direct RDMA,**无需** `nvidia-peermem` 内核模块。如需使用依赖 `nvidia-peermem` 的传统 `ibv_reg_mr` 路径,请在启动 Mooncake 前设置运行时环境变量 `WITH_NVIDIA_PEERMEM=1`。安装 `nvidia-peermem` 的说明见 https://docs.nvidia.com/cuda/gpudirect-rdma/ 第 3.7 节。 6. 编译 Mooncake 组件 ```bash diff --git a/docs/source/zh_archive/transfer-engine.md b/docs/source/zh_archive/transfer-engine.md index 85cf5bd4..73d04aaf 100644 --- a/docs/source/zh_archive/transfer-engine.md +++ b/docs/source/zh_archive/transfer-engine.md @@ -422,4 +422,5 @@ int init(const std::string &metadata_conn_string, - `MC_MIN_RPC_PORT` 指定 RPC 服务使用的最小端口号。默认值为 15000。 - `MC_MAX_RPC_PORT` 指定 RPC 服务使用的最大端口号。默认值为 17000。 - `MC_PATH_ROUNDROBIN` 指定 RDMA 路径选择使用 Round Robin 模式,这对于传输大块数据可能有利。 +- `WITH_NVIDIA_PEERMEM` 设置为 `1`、`ON` 或 `TRUE` 时,Mooncake 使用 `ibv_reg_mr()` 直接注册 GPU 内存(需要 `nvidia-peermem` 内核模块)。默认情况下(未设置或为 `0`),Mooncake 使用不需要 `nvidia-peermem` 的 DMA-BUF 路径。 - `MC_ENDPOINT_STORE_TYPE` 选择 FIFO Endpoint Store (`FIFO`) 或者 Sieve Endpoint Store (`SIEVE`),模式是 `SIEVE`。 -- 2.34.1 From 5bf24ddf682457bb6f080122e8c1ec6602cae072 Mon Sep 17 00:00:00 2001 From: Stary Date: Fri, 22 May 2026 11:54:52 +0800 Subject: [PATCH 110/382] feat(rdma): add mlx5 direct verbs support for QP UDP sport override and LAG port balancing (#2175) - Introduce`USE_MLX5DV`CMake option and link against`libmlx5`when enabled - Add`MC_MLX5_QP_UDP_SPORTS` environment variable to specify comma-separated UDP source ports for ECMP/LAG path diversification - Add`MC_MLX5_QP_LAG_PORT_BALANCE` environment variable to enable automatic QP distribution across bonded LAG ports - Update`RdmaContext`to query and expose the number of LAG ports via `mlx5dv_query_device` - Implement QP modification logic in`RdmaEndPoint::doSetupConnection` to apply UDP source port and LAG port pinning - Extend configuration parsing and logging to support the new mlx5-specific options - Document the new runtime options in the design documentation Signed-off-by: staryxchen --- docs/source/design/transfer-engine/index.md | 2 + mooncake-common/common.cmake | 1 + mooncake-transfer-engine/include/config.h | 11 +++ .../transport/rdma_transport/rdma_context.h | 7 ++ mooncake-transfer-engine/src/CMakeLists.txt | 5 ++ mooncake-transfer-engine/src/config.cpp | 69 +++++++++++++++++++ .../transport/rdma_transport/CMakeLists.txt | 7 +- .../transport/rdma_transport/rdma_context.cpp | 12 ++++ .../rdma_transport/rdma_endpoint.cpp | 64 +++++++++++++++++ 9 files changed, 177 insertions(+), 1 deletion(-) diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index 0796d06e..05349c22 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -275,6 +275,8 @@ For advanced users, TransferEngine provides the following advanced runtime optio - `MC_IB_PORT` The IB port number used per device instance, default value 1 - `MC_IB_TC` Adjust RDMA NIC Traffic Class when switch/NIC defaults differ or for traffic planning. Default value -1 - `MC_IB_PCI_RELAXED_ORDERING` Setting the PCIe ordering to relaxed for the network adapter sometimes results in better performance. Can set 1 to enable RO function. Default value 0 +- `MC_MLX5_QP_UDP_SPORTS` Comma-separated list of UDP source ports (0-65535) used to override the RoCEv2 UDP source port of each QP, for spreading traffic across different ECMP/LAG paths. QP at index *i* uses `list[i % size]`. Default empty (driver chooses). **Requires** an mlx5 NIC + RoCEv2, and the binary built with `-DUSE_MLX5DV=ON`. Recommend ports in the dynamic range 49152-65535. Example: `MC_MLX5_QP_UDP_SPORTS="49152,49153,49154,49155"` +- `MC_MLX5_QP_LAG_PORT_BALANCE` Set to `1` or `true` to enable automatic LAG port balancing across bonded physical ports. QP at index *i* is pinned to port `(i % num_lag_ports) + 1`; the number of LAG ports is queried from hardware via `mlx5dv_query_device` at startup and printed in the device log. If the device is not in LAG mode the setting is a no-op. Default: disabled. **Requires** the binary built with `-DUSE_MLX5DV=ON`. Example: `MC_MLX5_QP_LAG_PORT_BALANCE=1` - `MC_GID_INDEX` The GID index used per device instance, default value 3 (or the maximum value supported by the platform) - `MC_PKEY_INDEX` The QP `pkey_index` (partition key table index) used when transitioning the QP to the INIT state. Valid range: 0 to 65535. Default value 0. Set this when the partition key required for your fabric is not at index 0 of the HCA's pkey table - `MC_MAX_CQE_PER_CTX` The CQ buffer size per device instance, default value 4096 diff --git a/mooncake-common/common.cmake b/mooncake-common/common.cmake index 3654f7de..d20b2ccd 100644 --- a/mooncake-common/common.cmake +++ b/mooncake-common/common.cmake @@ -123,6 +123,7 @@ if (ENABLE_MULTI_PROTOCOL) endif() option(USE_LRU_MASTER "option for using LRU in master service" OFF) option(USE_INTRA_NVLINK "option for using IntraNode nvlink transport" OFF) +option(USE_MLX5DV "enable mlx5 direct verbs (libmlx5) for QP UDP source port override" OFF) set(LRU_MAX_CAPACITY 1000) if (USE_LRU_MASTER) diff --git a/mooncake-transfer-engine/include/config.h b/mooncake-transfer-engine/include/config.h index 8ef7092b..32844b61 100644 --- a/mooncake-transfer-engine/include/config.h +++ b/mooncake-transfer-engine/include/config.h @@ -23,6 +23,7 @@ #include #include #include +#include namespace mooncake { @@ -63,6 +64,16 @@ struct GlobalConfig { size_t eic_max_block_size = 64UL * 1024 * 1024; EndpointStoreType endpoint_store_type = EndpointStoreType::SIEVE; int ib_traffic_class = -1; + // mlx5 QP UDP source ports for ECMP path diversification. + // Empty = no modification. QP at index i uses + // mlx5_qp_udp_sports[i % size]. Requires mlx5 device + RoCEv2, + // and the binary must be built with USE_MLX5DV. + std::vector mlx5_qp_udp_sports; + // mlx5 QP LAG port balancing. When enabled, QPs are distributed across + // physical LAG ports: QP at index i is pinned to port (i % num_lag_ports) + // + 1. num_lag_ports is queried from hardware; if the device is not in LAG + // mode the setting is a no-op. Requires USE_MLX5DV. + bool mlx5_qp_lag_port_balance = false; // ib_pci_relaxed_ordering_mode: 0: off, 1: on if supported, 2: auto int ib_pci_relaxed_ordering_mode = 0; bool ascend_use_fabric_mem = false; diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h index f0a29c45..3818e1b4 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h @@ -19,6 +19,10 @@ #include #include +#ifdef USE_MLX5DV +#include +#endif + #include #include #include @@ -154,6 +158,8 @@ class RdmaContext { uint8_t portNum() const { return port_; } + uint8_t numLagPorts() const { return num_lag_ports_; } + int activeSpeed() const { return active_speed_; } ibv_mtu activeMTU() const { return active_mtu_; } @@ -207,6 +213,7 @@ class RdmaContext { int gid_index_ = -1; int active_speed_ = -1; ibv_mtu active_mtu_; + uint8_t num_lag_ports_ = 0; // 0/1 = not in LAG; ≥2 = LAG active ibv_gid gid_; RWSpinlock memory_regions_lock_; diff --git a/mooncake-transfer-engine/src/CMakeLists.txt b/mooncake-transfer-engine/src/CMakeLists.txt index cb0c9979..80173547 100644 --- a/mooncake-transfer-engine/src/CMakeLists.txt +++ b/mooncake-transfer-engine/src/CMakeLists.txt @@ -133,6 +133,11 @@ if(USE_INTRA_NVLINK) target_compile_definitions(transfer_engine PUBLIC USE_INTRA_NVLINK) endif() +if(USE_MLX5DV) + message(STATUS "Enabled USE_MLX5DV support (mlx5 direct verbs)") + target_link_libraries(transfer_engine PUBLIC mlx5) +endif() + if(USE_EFA) message(STATUS "Enabled USE_EFA (AWS Elastic Fabric Adapter) support") target_link_libraries(transfer_engine PUBLIC fabric efa_transport) diff --git a/mooncake-transfer-engine/src/config.cpp b/mooncake-transfer-engine/src/config.cpp index f13f8516..1042fc9c 100644 --- a/mooncake-transfer-engine/src/config.cpp +++ b/mooncake-transfer-engine/src/config.cpp @@ -18,6 +18,7 @@ #include #include #include +#include #include namespace mooncake { @@ -370,6 +371,62 @@ void loadGlobalConfig(GlobalConfig& config) { LOG(WARNING) << "Ignore value from environment variable " "MC_IB_PCI_RELAXED_ORDERING, it should be 0|1|2"; } + + const char* mlx5_qp_udp_sports_env = std::getenv("MC_MLX5_QP_UDP_SPORTS"); + if (mlx5_qp_udp_sports_env && *mlx5_qp_udp_sports_env) { + std::vector ports; + std::stringstream ss(mlx5_qp_udp_sports_env); + std::string item; + bool ok = true; + while (std::getline(ss, item, ',')) { + // Trim leading/trailing whitespace. + auto l = item.find_first_not_of(" \t"); + auto r = item.find_last_not_of(" \t"); + if (l == std::string::npos) continue; + item = item.substr(l, r - l + 1); + try { + int val = std::stoi(item); + if (val < 0 || val > 65535) { + LOG(WARNING) + << "MC_MLX5_QP_UDP_SPORTS entry out of range: " << item; + ok = false; + break; + } + ports.push_back(static_cast(val)); + } catch (const std::exception& e) { + LOG(WARNING) << "Invalid MC_MLX5_QP_UDP_SPORTS entry: " << item + << ". Error: " << e.what(); + ok = false; + break; + } + } + if (ok && !ports.empty()) { + config.mlx5_qp_udp_sports = std::move(ports); + } else if (!ok) { + LOG(WARNING) << "Ignore MC_MLX5_QP_UDP_SPORTS entirely due to " + "parse errors"; + } + } + + const char* mlx5_qp_lag_port_balance_env = + std::getenv("MC_MLX5_QP_LAG_PORT_BALANCE"); + if (mlx5_qp_lag_port_balance_env && *mlx5_qp_lag_port_balance_env) { + std::string val(mlx5_qp_lag_port_balance_env); + // Trim leading/trailing whitespace. + auto l = val.find_first_not_of(" \t"); + auto r = val.find_last_not_of(" \t"); + if (l != std::string::npos) { + val = val.substr(l, r - l + 1); + if (val == "1" || val == "true") + config.mlx5_qp_lag_port_balance = true; + else if (val == "0" || val == "false") + config.mlx5_qp_lag_port_balance = false; + else + LOG(WARNING) << "Ignore MC_MLX5_QP_LAG_PORT_BALANCE: expected " + "0/1/true/false, got: " + << val; + } + } } std::string mtuLengthToString(ibv_mtu mtu) { @@ -421,6 +478,18 @@ void dumpGlobalConfig() { LOG(INFO) << "mtu_length = " << mtuLengthToString(config.mtu_length); LOG(INFO) << "parallel_reg_mr = " << config.parallel_reg_mr; LOG(INFO) << "ib_traffic_class = " << config.ib_traffic_class; + { + std::ostringstream oss; + for (size_t i = 0; i < config.mlx5_qp_udp_sports.size(); ++i) { + if (i) oss << ","; + oss << config.mlx5_qp_udp_sports[i]; + } + LOG(INFO) << "mlx5_qp_udp_sports = [" + << (config.mlx5_qp_udp_sports.empty() ? "" : oss.str()) + << "]"; + } + LOG(INFO) << "mlx5_qp_lag_port_balance = " + << (config.mlx5_qp_lag_port_balance ? "true" : "false"); } GlobalConfig& globalConfig() { diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/CMakeLists.txt b/mooncake-transfer-engine/src/transport/rdma_transport/CMakeLists.txt index e3b211a0..dc2c10ea 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/CMakeLists.txt +++ b/mooncake-transfer-engine/src/transport/rdma_transport/CMakeLists.txt @@ -1,4 +1,9 @@ file(GLOB RDMA_SOURCES "*.cpp") add_library(rdma_transport OBJECT ${RDMA_SOURCES}) -target_link_libraries(rdma_transport PRIVATE JsonCpp::JsonCpp glog::glog pthread) +target_link_libraries(rdma_transport PRIVATE JsonCpp::JsonCpp glog::glog + pthread) + +if(USE_MLX5DV) + target_compile_definitions(rdma_transport PRIVATE USE_MLX5DV) +endif() diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index 322a6786..1b8b79e2 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -150,8 +150,20 @@ int RdmaContext::construct(size_t num_cq_list, size_t num_comp_channels, worker_pool_ = std::make_shared(*this, socketId()); +#ifdef USE_MLX5DV + { + mlx5dv_context dv_ctx = {}; + dv_ctx.comp_mask = MLX5DV_CONTEXT_MASK_NUM_LAG_PORTS; + if (mlx5dv_query_device(context_, &dv_ctx) == 0) + num_lag_ports_ = dv_ctx.num_lag_ports; + } + LOG(INFO) << "RDMA device: " << context_->device->name << ", LID: " << lid_ + << ", GID: (GID_Index " << gid_index_ << ") " << gid() + << ", num_lag_ports: " << (int)num_lag_ports_; +#else LOG(INFO) << "RDMA device: " << context_->device->name << ", LID: " << lid_ << ", GID: (GID_Index " << gid_index_ << ") " << gid(); +#endif return 0; } diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp index e39a51d0..34271216 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp @@ -22,6 +22,10 @@ #include #include +#ifdef USE_MLX5DV +#include +#endif + #include "common.h" #include "config.h" @@ -834,6 +838,66 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, return ERR_ENDPOINT; } + // Optional: pin QP to a specific LAG port for even traffic distribution + // across bonded physical ports. num_lag_ports is queried from hardware at + // context construction time; this block is a no-op on non-LAG devices. + if (globalConfig().mlx5_qp_lag_port_balance) { +#ifdef USE_MLX5DV + uint8_t n = context_.numLagPorts(); + if (n > 1) { + uint8_t target = (uint8_t)(qp_index % n) + 1; + int lag_ret = mlx5dv_modify_qp_lag_port(qp, target); + if (lag_ret) { + LOG_FIRST_N(WARNING, 4) + << "[RDMA] mlx5dv_modify_qp_lag_port failed" + << " (qp_index=" << qp_index + << ", target_port=" << (int)target + << "): " << strerror(lag_ret); + } else { + uint8_t cfg = 0, active = 0; + if (mlx5dv_query_qp_lag_port(qp, &cfg, &active) == 0) { + VLOG(1) + << "[RDMA] QP[" << qp_index << "] qpn=" << qp->qp_num + << " lag_port cfg=" << (int)cfg + << " active=" << (int)active; + } else { + LOG_FIRST_N(WARNING, 4) + << "[RDMA] mlx5dv_query_qp_lag_port failed" + << " (qp_index=" << qp_index << ")"; + } + } + } +#else + LOG_FIRST_N(WARNING, 1) + << "MC_MLX5_QP_LAG_PORT_BALANCE is set but binary was not built " + "with USE_MLX5DV; ignoring"; +#endif + } + + // Optional: override the RoCEv2 UDP source port to spread QPs across + // different ECMP/LAG paths. Empty list = leave whatever the driver + // picked. Failure is non-fatal so unsupported devices/firmware degrade + // gracefully. + const auto &sports = globalConfig().mlx5_qp_udp_sports; + if (!sports.empty()) { +#ifdef USE_MLX5DV + uint16_t sport = sports[qp_index % sports.size()]; + int sp_ret = mlx5dv_modify_qp_udp_sport(qp, sport); + if (sp_ret) { + LOG_FIRST_N(WARNING, 4) + << "[RDMA] mlx5dv_modify_qp_udp_sport failed (qp_index=" + << qp_index << ", sport=" << sport << "): " << strerror(sp_ret); + } else { + VLOG(1) << "[RDMA] QP[" << qp_index << "] qpn=" << qp->qp_num + << " udp_sport=" << sport; + } +#else + LOG_FIRST_N(WARNING, 1) + << "MC_MLX5_QP_UDP_SPORTS is set but binary was not built with " + "USE_MLX5DV; ignoring"; +#endif + } + return 0; } } // namespace mooncake -- 2.34.1 From e0b0f01c42b131b90362089aec31e65a6be1d62f Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Fri, 22 May 2026 13:38:56 +0800 Subject: [PATCH 111/382] [TENT] Enhanced QoS and Slice Spraying for TENT (#2048) * Revise implementation * Reformat * fix fallback logic * Add QoS APIs and docs * Reformat * Add QoS starvation prevention & bugfixes * Reformat * update docs * remove tl_caller_id * Fix worker distribution for multi-threaded submissions When multiple threads submit slices simultaneously, they all start distributing from worker 0, causing contention. Use thread_local offset to distribute starting worker across threads, ensuring each thread begins from a different worker. Also rename submit_slices to next_worker_idx for clarity. Co-Authored-By: Claude Opus 4.7 (1M context) --------- Co-authored-by: Claude Opus 4.7 (1M context) --- docs/source/design/tent/qos.md | 396 ++++++++++++++++ docs/source/design/tent/slice-spraying.md | 398 ++++++++++++++++ .../tent/include/tent/common/types.h | 7 + .../tent/include/tent/transfer_engine.h | 1 + .../tent/include/tent/transport/rdma/quota.h | 191 ++++++-- .../tent/transport/rdma/shared_quota.h | 73 ++- .../tent/include/tent/transport/rdma/slice.h | 1 + .../include/tent/transport/rdma/workers.h | 20 +- .../tent/src/python/pybind.cpp | 14 +- .../tent/src/transfer_engine_c.cpp | 2 + .../tent/src/transport/rdma/quota.cpp | 433 ++++++++++++------ .../src/transport/rdma/rdma_transport.cpp | 41 +- .../tent/src/transport/rdma/shared_quota.cpp | 332 +++++--------- .../tent/src/transport/rdma/workers.cpp | 186 +++++++- 14 files changed, 1630 insertions(+), 465 deletions(-) create mode 100644 docs/source/design/tent/qos.md create mode 100644 docs/source/design/tent/slice-spraying.md diff --git a/docs/source/design/tent/qos.md b/docs/source/design/tent/qos.md new file mode 100644 index 00000000..06ecfc27 --- /dev/null +++ b/docs/source/design/tent/qos.md @@ -0,0 +1,396 @@ +# TENT Quality of Service (QoS) + +## Overview + +TENT provides Quality of Service (QoS) support to ensure that high-priority requests receive preferential treatment in multi-tenant and multi-workload environments. This document describes TENT's QoS architecture and configuration. + +## Background + +In shared RDMA clusters, different types of transfers have different priority requirements: + +1. **Metadata and Control Messages**: Require low latency, small size +2. **Interactive Queries**: Require low to medium latency, medium size +3. **Bulk Data Transfer**: Can tolerate higher latency, large size + +Without QoS, low-priority bulk transfers can monopolize bandwidth and cause high tail latency for critical requests. + +TENT addresses this through: +- **Per-worker priority queues** for intra-process isolation +- **Global time-sliced coordination** for inter-process isolation +- **Priority-aware device filtering** for NUMA-aware scheduling + +## Architecture + +### Priority Levels + +TENT supports three priority levels: + +| Priority | Value | Description | Use Cases | +|----------|-------|-------------|-----------| +| `PRIO_HIGH` | 0 | High-priority requests | Metadata, control messages, latency-sensitive operations | +| `PRIO_MEDIUM` | 1 | Medium-priority requests | Interactive queries, serving workloads | +| `PRIO_LOW` | 2 | Low-priority requests | Bulk data transfer, background jobs | + +### Per-Worker Priority Queues + +Each worker thread maintains separate queues for each priority level: + +``` +┌─────────────────────────────────────┐ +│ Worker Thread │ +├─────────────────────────────────────┤ +│ PRIO_HIGH Queue │ │ +│ PRIO_MEDIUM Queue │ │ +│ PRIO_LOW Queue │ │ +├─────────────────────────────────────┤ +│ Dequeue Priority: HIGH→MEDIUM→LOW │ +└─────────────────────────────────────┘ +``` + +**Scheduling Logic**: +1. Always drain HIGH priority queue first +2. Only process MEDIUM when HIGH is empty +3. Only process LOW when both HIGH and MEDIUM are empty + +**Priority Promotion (Anti-Starvation)**: +To prevent low-priority requests from starving indefinitely, TENT implements timeout-based priority promotion: +- MEDIUM priority requests are promoted to HIGH after waiting too long +- LOW priority requests are promoted to MEDIUM after waiting too long +- Promotion checks run periodically (every 1ms by default) + +This ensures that: +- High-priority requests normally never wait behind lower-priority work +- Low-priority requests eventually get serviced even under continuous high-priority load + +### Global Slot Coordination + +For multi-process environments, TENT implements global time-sliced coordination using shared memory: + +``` +Time slices rotate every N milliseconds: + +Slot 0 (0-Nms): Only HIGH priority requests allowed +Slot 1 (N-2Nms): MEDIUM + HIGH priority requests allowed +Slot 2 (2N-3Nms): All priorities allowed +...repeats... +``` + +**Default Configuration**: 2ms per slot (6ms full cycle) + +This mechanism ensures that: +- High-priority requests get dedicated service windows +- No process can monopolize bandwidth indefinitely +- Fair access across process boundaries + +### Shared Memory Structure + +The global slot state is maintained in shared memory: + +```cpp +struct SharedHeader { + uint64_t magic; // Magic number for validation + int32_t version; // Format version + std::atomic current_slot; // Current global slot (0, 1, or 2) + pthread_mutex_t global_mutex; // For synchronization (robust) +}; +``` + +**Operations**: +- Background thread rotates slot every N milliseconds +- Workers check `canSend()` before processing requests +- Only requests with priority ≤ slot level are processed + +## Configuration + +### Priority Filtering + +```json +{ + "transports": { + "rdma": { + "enable_priority_filtering": true, + "local_rotation_interval_us": 200, + "priority_promotion_timeout_us": 10000 + } + } +} +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `enable_priority_filtering` | bool | `true` | Enable priority-based device filtering | +| `local_rotation_interval_us` | int | `200` | Local device priority rotation interval (microseconds) | +| `priority_promotion_timeout_us` | int | `10000` | Timeout for priority promotion (microseconds) | + +### Global Coordination + +```json +{ + "transports": { + "rdma": { + "slot_rotation_interval_ms": 2, + "shared_quota_shm_path": "/mooncake_rdma_slots" + } + } +} +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `slot_rotation_interval_ms` | int | `2` | Global slot rotation interval (milliseconds) | +| `shared_quota_shm_path` | string | `""` | Shared memory path for multi-process coordination | + +**Note**: Leave `shared_quota_shm_path` empty to disable global coordination (single-process mode). + +## Usage Examples + +### Example 1: Latency-Critical Workload + +For workloads where high-priority requests must have minimal latency: + +```json +{ + "transports": { + "rdma": { + "enable_priority_filtering": true, + "slot_rotation_interval_ms": 1, + "shared_quota_shm_path": "/mooncake_rdma_slots" + } + } +} +``` + +**Effect**: High-priority requests get dedicated windows every 1ms. + +### Example 2: Single-Process Mode + +For single-process deployments where global coordination is not needed: + +```json +{ + "transports": { + "rdma": { + "enable_priority_filtering": true, + "shared_quota_shm_path": "" + } + } +} +``` + +**Effect**: Per-worker priority queues only, no cross-process coordination. + +### Example 3: Bulk-Friendly Configuration + +For workloads where low-priority bulk transfers should not be starved: + +```json +{ + "transports": { + "rdma": { + "slot_rotation_interval_ms": 10, + "shared_quota_shm_path": "/mooncake_rdma_slots" + } + } +} +``` + +**Effect**: Longer slots allow more low-priority work to complete. + +## Priority Assignment + +### Setting Request Priority + +Priority is assigned when creating `Request` objects. The default priority is `PRIO_HIGH`. + +### C++ API + +```cpp +// In C++ +#include "tent/transfer_engine.h" + +using namespace mooncake::tent; + +// Create request with default priority (HIGH) +Request req; +req.opcode = Request::OpCode::READ; +req.source = buffer; +req.target_id = segment_id; +req.target_offset = 0; +req.length = size; +// req.priority is PRIO_HIGH by default + +// Or specify priority explicitly +req.priority = PRIO_MEDIUM; // or PRIO_HIGH, PRIO_LOW + +// Submit the request +engine.submitTransfer(batch_id, {req}); +``` + +### Python API + +```python +# In Python +import tent + +# Create request with default priority (HIGH) +req = tent.Request( + opcode=tent.OpCode.READ, + source=buffer_addr, + target_id=segment_id, + target_offset=0, + length=size +) +# req.priority is tent.PRIO_HIGH by default + +# Or specify priority in constructor +req = tent.Request( + opcode=tent.OpCode.READ, + source=buffer_addr, + target_id=segment_id, + target_offset=0, + length=size, + priority=tent.PRIO_LOW # or PRIO_HIGH, PRIO_MEDIUM +) + +# Or set after creation +req.priority = tent.PRIO_MEDIUM + +# Submit the request +engine.submit_transfer(batch_id, [req]) +``` + +### C API + +```c +// In C +#include "tent/transfer_engine.h" + +// Create request with priority +tent_request_t req = { + .opcode = OPCODE_READ, + .source = buffer, + .target_id = segment_id, + .target_offset = 0, + .length = size, + .priority = 0 // 0=HIGH, 1=MEDIUM, 2=LOW +}; + +// Submit the request +tent_submit(engine, batch_id, &req, 1); +``` + +## Performance Considerations + +### Trade-offs + +| Configuration | High-Priority Latency | Low-Priority Throughput | Fairness | +|---------------|----------------------|------------------------|----------| +| Short slot interval (1ms) | Excellent | Poor | High | +| Default slot interval (2ms) | Good | Fair | High | +| Long slot interval (10ms) | Fair | Good | Medium | +| No global coordination | Variable | Excellent | Low (per-process only) | + +### Starvation Prevention + +TENT prevents starvation through two mechanisms: + +1. **Global slot mechanism**: + - **HIGH priority**: Never starved (always allowed in slot 0) + - **MEDIUM priority**: Never starved (allowed in slots 1 and 2) + - **LOW priority**: Never starved (always allowed in slot 2) + +2. **Priority promotion timeout**: + - Low-priority requests waiting longer than `priority_promotion_timeout_us` are promoted + - MEDIUM → HIGH promotion ensures medium priority gets service + - LOW → MEDIUM promotion ensures low priority eventually gets service + - Configurable via `priority_promotion_timeout_us` (default 10ms) + +### Tuning Guidelines + +1. **Start with default settings** (2ms slot interval) +2. **Measure tail latency** for each priority level +3. **Adjust slot interval** based on observations: + - If HIGH priority latency is too high: decrease interval + - If LOW priority throughput is too low: increase interval + +## Troubleshooting + +### Problem: High-priority requests have high latency + +**Symptoms**: `PRIO_HIGH` requests experiencing unexpected delays + +**Possible causes**: +1. Slot interval too long +2. Global coordination not enabled +3. Worker threads blocked on LOW priority work + +**Solution**: +```json +{ + "slot_rotation_interval_ms": 1, + "enable_priority_filtering": true +} +``` + +### Problem: Low-priority transfers starved + +**Symptoms**: `PRIO_LOW` requests making no progress + +**Possible causes**: +1. HIGH priority load is continuous +2. Slot interval too short + +**Solution**: Increase slot interval to give LOW priority more time: +```json +{ + "slot_rotation_interval_ms": 10 +} +``` + +### Problem: Shared memory creation fails + +**Symptoms**: Error messages about `/mooncake_rdma_slots` + +**Possible causes**: +1. Permission issues (need write access to `/dev/shm`) +2. Stale shared memory from previous run + +**Solution**: +```bash +# Remove stale shared memory +rm -f /dev/shm/mooncake_rdma_slots + +# Or use a different path +{ + "shared_quota_shm_path": "/mooncake_rdma_slots_v2" +} +``` + +## Monitoring + +### Traffic Statistics + +Monitor per-device traffic distribution: + +```cpp +device_selector_->printTrafficStats(); +``` + +Output example: +``` +=== Device Traffic Statistics === +Dev 0: Total=10.5 GB, EWMA BW=45.23 Gbps, Inflight=0 bytes +Dev 1: Total=8.2 GB, EWMA BW=42.18 Gbps, Inflight=0 bytes +Dev 2: Total=0.5 GB, EWMA BW=38.91 Gbps, Inflight=0 bytes +Dev 3: Total=0.3 GB, EWMA BW=39.12 Gbps, Inflight=0 bytes +``` + +### Priority Statistics + +Monitor queue depths for each priority level (requires instrumentation). + +## References + +- [TENT Overview](overview.md) +- [TENT Slice Spraying](slice-spraying.md) +- [TENT C++ API](cpp-api.md) diff --git a/docs/source/design/tent/slice-spraying.md b/docs/source/design/tent/slice-spraying.md new file mode 100644 index 00000000..5cf6d3bc --- /dev/null +++ b/docs/source/design/tent/slice-spraying.md @@ -0,0 +1,398 @@ +# TENT Slice Spraying + +## Overview + +This document describes TENT's Slice Spraying mechanism, which enables efficient data movement in multi-rail RDMA environments through intelligent device selection and adaptive load balancing. + +## Background + +In multi-rail RDMA environments, naive round-robin striping leads to suboptimal performance because: + +1. **NUMA Effects**: Cross-NUMA access incurs additional latency and reduces effective bandwidth +2. **Load Imbalance**: Static striping cannot adapt to dynamic load conditions +3. **Heterogeneous Link Quality**: Different rails may have different effective bandwidth due to congestion or hardware characteristics + +TENT addresses these issues through: +- **NUMA-aware device selection** with configurable penalties +- **EWMA-based bandwidth estimation** for adaptive load balancing +- **Dynamic multi-path allocation** for large transfers + +## Architecture + +### Device Selector + +The `DeviceSelector` component is responsible for choosing which RDMA device(s) to use for each transfer request. It operates in two modes: + +#### Baseline Mode (Round-Robin) + +When `enable_smart_scheduling = false`, the selector uses simple round-robin within the highest-priority device tier (typically local NUMA devices): + +``` +For each request: + 1. Find first non-empty device tier (local NUMA preferred) + 2. Select devices round-robin within that tier + 3. Ignore lower-priority tiers +``` + +**Characteristics**: +- Deterministic behavior +- No runtime overhead for tracking +- Consistent with original TE behavior +- Does not adapt to load conditions + +#### Smart Mode (EWMA-Based Selection) + +When `enable_smart_scheduling = true`, the selector uses an EWMA-based algorithm: + +``` +For each request: + 1. Calculate predicted completion time for each device: + predicted_time = (inflight_bytes + slice_bytes) / ewma_bandwidth + + 2. Apply NUMA penalty based on tier: + score = predicted_time × numa_tier_weights[tier] + + 3. Select device(s) with minimum score: + - Single slice: best device only + - Multiple slices: weighted distribution across devices + + 4. Update EWMA bandwidth on completion: + ewma_bandwidth = α × ewma_bandwidth + (1 - α) × observed_bandwidth + where α = bandwidth_learning_rate +``` + +**Characteristics**: +- Adapts to changing load conditions +- Prefers local NUMA devices +- Spreads load across multiple rails +- Higher runtime overhead + +### NUMA-Aware Selection + +Devices are organized into tiers based on NUMA distance: + +| Tier | Description | Default Penalty | +|------|-------------|-----------------| +| Rank 0 | Local NUMA | 1.0 (baseline) | +| Rank 1 | Remote NUMA (tier 1) | 5.0 | +| Rank 2 | Remote NUMA (tier 2) | 10.0 | + +The penalty is applied as a multiplier to predicted completion time, making remote devices less attractive unless local devices are heavily loaded. + +### EWMA Bandwidth Estimation + +Each device maintains an EWMA (Exponentially Weighted Moving Average) of its effective bandwidth: + +``` +initial_value = theoretical_bandwidth + +on_transfer_complete: + observed_bandwidth = transfer_size / transfer_time + ewma_bandwidth = α × ewma_bandwidth + (1 - α) × observed_bandwidth + ewma_bandwidth = clamp(ewma_bandwidth, + 0.1 × theoretical, + 10.0 × theoretical) +``` + +where `α = bandwidth_learning_rate`. + +**Note on terminology**: The EWMA formula uses α as the coefficient for the old value. Therefore: +- **Lower α** (closer to 0) → more weight on new observations → **faster adaptation** +- **Higher α** (closer to 1) → more weight on old value → **slower adaptation** + +Examples: +- α = 0: `ewma_bandwidth = observed_bandwidth` (full adaptation, always use new value) +- α = 1: `ewma_bandwidth = ewma_bandwidth` (no learning, never update) +- α = 0.01: `ewma_bandwidth = 0.01 × old + 0.99 × new` (default, gradual adaptation) + +The EWMA provides: +- **Memory**: Recent observations have more influence than old ones +- **Stability**: Smooths out transient fluctuations +- **Adaptability**: Tracks gradual changes in link quality + +### Multi-Path Allocation + +For large transfers, TENT distributes slices across multiple devices: + +**Single Path** (small requests): +- All slices go to the single best device +- Minimizes coordination overhead + +**Multi Path** (large requests): +- **Normal mode** (99% of calls): Slices distributed proportionally to device capacity + - Each device gets: `(device_weight / total_weight) × num_slices` + - Remaining slices assigned to best device +- **Probe mode** (1% of calls, every 100th call): Slices distributed round-robin + - Purpose: Ensure all devices are continuously sampled for EWMA updates + - Prevents EWMA starvation for less-used devices + +### Request Flow + +``` +┌──────────────┐ +│ Application │ +└──────┬───────┘ + │ submitTransfer() + ▼ +┌──────────────────────────────────────┐ +│ RdmaTransport::submitTransferTasks │ +│ - Split large requests into slices │ +│ - Call DeviceSelector for allocation │ +│ - Only if num_slices >= max_slice_count/2 │ +└──────┬───────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────┐ +│ DeviceSelector::allocate │ +│ ┌────────────────────────────────┐ │ +│ │ smart_selection_enabled? │ │ +│ └────┬──────────────────────┬────┘ │ +│ │ Yes │ No │ +│ ▼ ▼ │ +│ ┌─────────┐ ┌─────────┐ │ +│ │ Smart │ │ Baseline│ │ +│ │ Mode │ │ Mode │ │ +│ └────┬────┘ └────┬────┘ │ +│ │ │ │ +│ └────────┬───────────┘ │ +│ ▼ │ +│ ┌────────────────────────────────┐ │ +│ │ Return slice_dev_ids │ │ +│ └────────────────────────────────┘ │ +└──────────────────────────────────────┘ +``` + +## Configuration + +All slice spraying parameters are configurable via the configuration file: + +### Core Scheduling + +```json +{ + "transports": { + "rdma": { + "enable_smart_scheduling": true + } + } +} +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `enable_smart_scheduling` | bool | `true` | Enable EWMA-based selection (false = round-robin) | + +### NUMA Penalties + +```json +{ + "transports": { + "rdma": { + "numa_penalties": [1.0, 5.0, 10.0] + } + } +} +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `numa_penalties` | array[float] | `[1.0, 5.0, 10.0]` | Penalty multipliers for each NUMA tier | + +**Guidelines**: +- Higher values = stronger preference for local devices +- Set all to `1.0` to disable NUMA awareness +- Increase remote penalties if cross-NUMA latency is high + +### Bandwidth Estimation + +```json +{ + "transports": { + "rdma": { + "bandwidth_learning_rate": 0.01, + "ewma_min_bandwidth_multiplier": 0.1, + "ewma_max_bandwidth_multiplier": 10.0 + } + } +} +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `bandwidth_learning_rate` | float | `0.01` | EWMA learning rate (0.0 = full adaptation, 1.0 = no learning) | +| `ewma_min_bandwidth_multiplier` | float | `0.1` | Minimum bandwidth as fraction of theoretical | +| `ewma_max_bandwidth_multiplier` | float | `10.0` | Maximum bandwidth as fraction of theoretical | + +**Guidelines**: +- Lower α (e.g., 0.001) → faster adaptation, more volatile → responds quickly to changes +- Higher α (e.g., 0.1) → slower adaptation, more stable → smooths out transient fluctuations +- Default α = 0.01 provides balanced adaptation +- Multipliers constrain EWMA to reasonable range [0.1×, 10.0×] of theoretical bandwidth + +### Device Selection Scoring + +```json +{ + "transports": { + "rdma": { + "score_jitter_range": 1e-9, + "score_epsilon": 1e-12 + } + } +} +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `score_jitter_range` | float | `1e-9` | Random jitter range to avoid deterministic selection | +| `score_epsilon` | float | `1e-12` | Small value to prevent division by zero | + +### Bandwidth Constants + +```json +{ + "transports": { + "rdma": { + "default_bandwidth_gbps": 400.0, + "min_bandwidth_gbps": 10.0, + "max_bandwidth_gbps": 800.0 + } + } +} +``` + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `default_bandwidth_gbps` | float | `400.0` | Default NIC bandwidth when topology info unavailable | +| `min_bandwidth_gbps` | float | `10.0` | Minimum valid NIC bandwidth (Gbps) | +| `max_bandwidth_gbps` | float | `800.0` | Maximum valid NIC bandwidth (Gbps) | + +**Notes**: +- These constants define the valid range and default for device bandwidth +- Used in EWMA calculations and theoretical bandwidth estimation +- If a device's reported bandwidth is outside [min, max], default_bandwidth is used + +## Usage Examples + +### Example 1: Latency-Sensitive Workload + +For latency-sensitive queries where local NUMA access is critical: + +```json +{ + "transports": { + "rdma": { + "enable_smart_scheduling": true, + "numa_penalties": [1.0, 100.0, 1000.0], + "bandwidth_learning_rate": 0.001 + } + } +} +``` + +**Effect**: Strongly prefers local devices, slow adaptation for stability. + +### Example 2: Bulk Data Transfer + +For bulk transfers where throughput is more important than latency: + +```json +{ + "transports": { + "rdma": { + "enable_smart_scheduling": true, + "numa_penalties": [1.0, 2.0, 3.0], + "bandwidth_learning_rate": 0.1 + } + } +} +``` + +**Effect**: Allows cross-NUMA transfers, fast adaptation to load. + +### Example 3: Baseline Mode + +For deterministic performance matching original TE: + +```json +{ + "transports": { + "rdma": { + "enable_smart_scheduling": false + } + } +} +``` + +**Effect**: Round-robin within local NUMA tier, no adaptation, minimal overhead. + +## Performance Considerations + +### Overhead Comparison + +| Mode | CPU Overhead | Adaptability | NUMA Awareness | +|------|--------------|--------------|----------------| +| Baseline | Minimal | None | Tier-based (static) | +| Smart | Moderate | EWMA-based | Dynamic + penalty | + +### When to Use Each Mode + +**Use Baseline Mode when**: +- Workload is uniform and predictable +- Deterministic performance is required +- CPU overhead must be minimized +- All devices are in same NUMA node + +**Use Smart Mode when**: +- Workload is heterogeneous +- Link quality varies over time +- NUMA effects are significant +- Maximum throughput is desired + +### Tuning Guidelines + +1. **Start with baseline mode** to establish performance baseline +2. **Enable smart mode** with conservative parameters: + - `numa_penalties = [1.0, 2.0, 5.0]` + - `bandwidth_learning_rate = 0.01` +3. **Monitor performance** and adjust based on observations: + - If cross-NUMA transfers are too frequent: increase remote penalties + - If adaptation is too slow (EWMA not keeping up with load changes): decrease α + - If performance is unstable (too much fluctuation): increase α + +## Troubleshooting + +### Problem: All requests go to cross-NUMA devices + +**Symptoms**: Poor performance, high latency + +**Diagnosis**: +```cpp +device_selector_->printTrafficStats(); +``` + +**Solution**: Check `numa_penalties` configuration. Ensure local devices have lowest penalty (1.0). + +### Problem: Performance worse than baseline + +**Symptoms**: Smart mode slower than baseline mode + +**Possible causes**: +1. Learning rate too high (volatile decisions) +2. NUMA penalties too low (not preferring local) +3. Score jitter too large (too much randomness) + +**Solution**: Use more conservative: +```json +{ + "bandwidth_learning_rate": 0.001, + "numa_penalties": [1.0, 10.0, 100.0], + "score_jitter_range": 1e-12 +} +``` + +## References + +- [TENT Overview](overview.md) +- [TENT QoS](qos.md) +- [TENT C++ API](cpp-api.md) diff --git a/mooncake-transfer-engine/tent/include/tent/common/types.h b/mooncake-transfer-engine/tent/include/tent/common/types.h index 3455d56c..f999ddb8 100644 --- a/mooncake-transfer-engine/tent/include/tent/common/types.h +++ b/mooncake-transfer-engine/tent/include/tent/common/types.h @@ -28,6 +28,11 @@ namespace tent { using BatchID = uint64_t; using SegmentID = uint64_t; +// QoS priority levels +static constexpr uint8_t PRIO_HIGH = 0; +static constexpr uint8_t PRIO_MEDIUM = 1; +static constexpr uint8_t PRIO_LOW = 2; + struct Notification { std::string name; std::string msg; @@ -44,6 +49,8 @@ struct Request { SegmentID target_id; uint64_t target_offset; size_t length; + int priority = + PRIO_HIGH; // Request priority (PRIO_HIGH, PRIO_MEDIUM, PRIO_LOW) }; enum TransferStatusEnum { diff --git a/mooncake-transfer-engine/tent/include/tent/transfer_engine.h b/mooncake-transfer-engine/tent/include/tent/transfer_engine.h index c8cd7c5b..68b92bff 100644 --- a/mooncake-transfer-engine/tent/include/tent/transfer_engine.h +++ b/mooncake-transfer-engine/tent/include/tent/transfer_engine.h @@ -38,6 +38,7 @@ struct tent_request { tent_segment_id_t target_id; uint64_t target_offset; uint64_t length; + int priority; /* Request priority (0=HIGH, 1=MEDIUM, 2=LOW) */ }; typedef struct tent_request tent_request_t; diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/quota.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/quota.h index 57c70afa..b3a857ec 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/quota.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/quota.h @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -#ifndef TENT_QUOTA_H -#define TENT_QUOTA_H +#ifndef TENT_SELECTOR_H +#define TENT_SELECTOR_H #include #include @@ -30,47 +30,83 @@ namespace mooncake { namespace tent { -class SharedQuotaManager; +// Bandwidth constants (Gbps) +static constexpr double kDefaultBwGbps = 400.0; +static constexpr double kMinBwGbps = 10.0; +static constexpr double kMaxBwGbps = 800.0; + +class SharedSlotManager; /** - * @brief DeviceQuota implements NIC selection based on adaptive feedback. + * @brief DeviceSelector implements NIC selection with two modes: * - * Each NIC maintains a smoothed estimate of its average service time, - * updated after each request completes. The allocator predicts the total - * completion time of each NIC as: + * 1. Baseline mode (smart_selection_enabled=false): Simple round-robin + * - Deterministic, no load tracking + * - All devices used equally * - * predicted_time = (active_bytes / bandwidth) + avg_service_time + * 2. Smart mode (smart_selection_enabled=true): EWMA-based selection + * - Tracks global inflight bytes per device + * - Learns effective bandwidth via EWMA + * - Selects device with minimal predicted completion time + * - Supports multi-path for large requests * - * and selects the NIC with the smallest predicted_time. + * Selection formula: + * predicted_time = (inflight + slice_bytes) / ewma_bandwidth * - * The estimator is updated using exponential smoothing: - * - * avg_service_time <- (1 - alpha) * avg_service_time + alpha * - * observed_time + * EWMA update: + * ewma_bandwidth <- alpha * ewma_bandwidth + (1 - alpha) * + * observed_bandwidth */ -class DeviceQuota { +class DeviceSelector { public: + // Candidate device for allocation + struct Candidate { + int dev_id; + double score; + bool is_cross_numa; + }; + struct DeviceInfo { int dev_id; double bw_gbps; int numa_id; uint64_t padding0[5]; - std::atomic active_bytes{0}; + std::atomic inflight_bytes{0}; uint64_t padding1[7]; - std::atomic diffusion_active_bytes{0}; + std::atomic ewma_bandwidth_bps{50e9}; uint64_t padding2[7]; - std::atomic beta0{0.0}; // Fixed latency (PCIe, setup) - uint64_t padding3[7]; - std::atomic beta1{1.0}; // Effective bandwidth correction - uint64_t padding4[7]; + std::atomic total_bytes{0}; + uint64_t padding3[5]; + + uint64_t getInflightBytes() const { + return inflight_bytes.load(std::memory_order_relaxed); + } + + void addInflight(uint64_t bytes) { + inflight_bytes.fetch_add(bytes, std::memory_order_relaxed); + } + + void releaseInflight(uint64_t bytes) { + inflight_bytes.fetch_sub(bytes, std::memory_order_relaxed); + } + + double getEwmaBandwidth() const { + return ewma_bandwidth_bps.load(std::memory_order_relaxed); + } + + double getTheoreticalBandwidth() const { + if (bw_gbps >= kMinBwGbps && bw_gbps <= kMaxBwGbps) + return bw_gbps * 1e9 / 8.0; + return kDefaultBwGbps * 1e9 / 8.0; + } }; public: - DeviceQuota() = default; - ~DeviceQuota() = default; + DeviceSelector() = default; + ~DeviceSelector() = default; - DeviceQuota(const DeviceQuota &) = delete; - DeviceQuota &operator=(const DeviceQuota &) = delete; + DeviceSelector(const DeviceSelector &) = delete; + DeviceSelector &operator=(const DeviceSelector &) = delete; Status loadTopology(std::shared_ptr &local_topology); @@ -78,46 +114,111 @@ class DeviceQuota { Status enableSharedQuota(const std::string &shm_name); + std::shared_ptr getSharedSlotManager() const { + return slot_manager_; + } + + // Allocate devices for a request (new API) + // slice_bytes: pre-calculated slice size from rdma_transport to ensure + // consistency + Status allocate(uint64_t total_length, uint32_t num_slices, + uint64_t slice_bytes, const std::string &location, + std::vector &slice_dev_ids, int priority = PRIO_HIGH, + uint64_t device_mask = ~0ULL); + Status allocate(uint64_t length, const std::string &location, int &chosen_dev_id); Status release(int dev_id, uint64_t length, double latency); - void setDiffusionActiveBytes(int dev_id, uint64_t value) { - devices_[dev_id].diffusion_active_bytes.store( - value, std::memory_order_relaxed); + void updateTrafficStats(int dev_id, uint64_t length) { + auto it = devices_.find(dev_id); + if (it != devices_.end()) { + it->second.total_bytes.fetch_add(length, std::memory_order_relaxed); + } } - uint64_t getActiveBytes(int dev_id) { - return devices_[dev_id].active_bytes.load(std::memory_order_relaxed); + void setSmartSelection(bool enable) { smart_selection_enabled_ = enable; } + bool getSmartSelection() const { return smart_selection_enabled_; } + + void setLearningRate(double alpha) { + sched_params_.bandwidth_learning_rate = std::clamp(alpha, 0.0, 1.0); } - void setLearningRate(double alpha) { alpha_ = std::clamp(alpha, 0.0, 1.0); } + int getDeviceRank(const std::string &location, int dev_id) const; - void setLocalWeight(double local_weight) { - local_weight_ = std::clamp(local_weight, 0.0, 1.0); + void printTrafficStats(); + + void fillDevicePriorities(); + int getDevicePriority(int dev_id) const; + + struct SchedulingParams { + // NUMA tier penalties (rank 0 = local, should be smallest) + double numa_tier_weights[Topology::DevicePriorityRanks] = {1.0, 5.0, + 10.0}; + + // EWMA bandwidth learning rate (0.0 = full adaptation, 1.0 = no + // learning) + double bandwidth_learning_rate = 0.01; + + // Enable priority-based filtering + bool enable_priority_filtering = true; + + // Local device priority rotation interval (microseconds) + uint64_t local_rotation_interval_us = 200; + + // Score random jitter range (to avoid deterministic selection) + double score_jitter_range = 1e-9; + + // Epsilon for division by zero protection + double score_epsilon = 1e-12; + + // EWMA bandwidth bounds (multiplier of theoretical bandwidth) + double ewma_min_multiplier = 0.1; // 10% of theoretical + double ewma_max_multiplier = 10.0; // 1000% of theoretical + + // Default bandwidth (Gbps) when topology info unavailable + double default_bandwidth_gbps = 400.0; // Default NIC bandwidth + double min_bandwidth_gbps = 10.0; // Minimum valid NIC bandwidth + double max_bandwidth_gbps = 800.0; // Maximum valid NIC bandwidth + + // Shared slot rotation interval (milliseconds) + int slot_rotation_interval_ms = 2; + + std::vector device_base_priorities; + }; + + void setSchedulingParams(const SchedulingParams ¶ms) { + sched_params_ = params; } - void setDiffusionInterval(uint64_t msec) { - diffusion_interval_ = msec * 1000000ull; + const SchedulingParams &getSchedulingParams() const { + return sched_params_; } - void setCrossNumaAccess(bool enable = true) { allow_cross_numa_ = enable; } - private: std::shared_ptr local_topology_; std::unordered_map devices_; - mutable std::shared_mutex rwlock_; - bool allow_cross_numa_ = false; - double alpha_ = 0.01; - double local_weight_ = 0.9; - uint64_t diffusion_interval_ = 10 * 1000000ull; - std::shared_ptr shared_quota_; - bool enable_quota_ = true; - bool update_quota_params_ = true; + std::shared_ptr slot_manager_; + bool smart_selection_enabled_ = true; + SchedulingParams sched_params_; + + Status buildCandidates(const Topology::MemEntry *entry, + uint64_t slice_bytes, uint64_t device_mask, + std::vector &candidates, + int request_priority = PRIO_HIGH); + + void selectSinglePath(const std::vector &candidates, + uint32_t num_slices, uint64_t total_length, + std::vector &slice_dev_ids); + + void selectMultiPath(const std::vector &candidates, + uint32_t num_slices, uint64_t total_length, + std::vector &slice_dev_ids, + bool probe_mode = false); }; } // namespace tent } // namespace mooncake -#endif // TENT_QUOTA_H \ No newline at end of file +#endif // TENT_SELECTOR_H \ No newline at end of file diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/shared_quota.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/shared_quota.h index e58af14b..7760fee3 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/shared_quota.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/shared_quota.h @@ -12,8 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. -#ifndef TENT_SHARED_QUOTA_H -#define TENT_SHARED_QUOTA_H +#ifndef TENT_SHARED_SLOT_H +#define TENT_SHARED_SLOT_H #include "quota.h" #include "tent/common/status.h" @@ -31,60 +31,54 @@ #include #include #include +#include namespace mooncake { namespace tent { -static constexpr int MAX_DEVICES = 64; -static constexpr int MAX_PID_SLOTS = 256; -static constexpr uint64_t SHM_MAGIC = 0x2025082772805202ULL; -static constexpr int SHM_VERSION = 1; +static constexpr uint64_t SHM_MAGIC = 0x2025082772805203ULL; +static constexpr int SHM_VERSION = 10; -struct PidUsage { - pid_t pid; // 0 == free slot - volatile uint64_t used_bytes; // local used bytes reported by this pid - uint8_t reserved[56]; // padding -> total 64B -}; - -struct SharedDeviceEntry { - char dev_name[56]; // NUL-terminated device name, empty means unused - volatile uint64_t active_bytes; - PidUsage pid_usages[MAX_PID_SLOTS]; -}; +// Time slice configuration for process-level coordination +// Slot 0: HIGH only +// Slot 1: MEDIUM + HIGH +// Slot 2: ALL (LOW + MEDIUM + HIGH) +// Then repeat +static constexpr int NUM_SLOTS = PRIO_LOW + 1; struct SharedHeader { uint64_t magic; int32_t version; - int32_t num_devices; + std::atomic current_slot; pthread_mutex_t global_mutex; - SharedDeviceEntry devices[MAX_DEVICES]; }; -class DeviceQuota; -class SharedQuotaManager { +class DeviceSelector; +class SharedSlotManager { public: - explicit SharedQuotaManager(DeviceQuota* local_quota); - ~SharedQuotaManager(); + explicit SharedSlotManager(DeviceSelector* local_quota); + ~SharedSlotManager(); Status attach(const std::string& shm_name); Status detach(); - Status diffusion(); + // Check if current process can send (global slot) + // Returns true if given priority is allowed in current global slot + bool canSend(int priority = PRIO_HIGH); + + // Set slot duration in milliseconds (must be > 0) + void setRotationIntervalMs(int ms) { rotation_interval_ms_ = ms; } + int getRotationIntervalMs() const { return rotation_interval_ms_; } private: - Status attachProcess(); - Status detachProcess(); - - private: - PidUsage* findOrCreatePidSlotLocked(int dev_id, pid_t pid); - PidUsage* findPidSlotLocked(int dev_id, pid_t pid); - int findDeviceIdByNameLocked(const std::string& dev_name); + void startBackgroundThread(); + void stopBackgroundThread(); + void backgroundThreadLoop(); Status initializeHeader(); Status initMutex(pthread_mutex_t* m); - void reclaimDeadPidsInternal(); - static bool isPidAlive(pid_t pid); - int lock(); - int unlock(); + + // Check if a priority is allowed in the given slot + bool isPriorityAllowedInSlot(int priority, int slot) const; private: std::string name_; @@ -92,10 +86,15 @@ class SharedQuotaManager { int fd_; size_t size_; bool created_; - DeviceQuota* local_quota_; + DeviceSelector* device_selector_; + int rotation_interval_ms_ = 2; // Default: 2ms per slot + + // Background thread + std::thread background_thread_; + std::atomic background_running_; }; } // namespace tent } // namespace mooncake -#endif // TENT_SHARED_QUOTA_H \ No newline at end of file +#endif // TENT_SHARED_SLOT_H \ No newline at end of file diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h index 13272bda..f681c8d2 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/slice.h @@ -96,6 +96,7 @@ struct RdmaSlice { // WorkerContext::rails stores values via unique_ptr, so rehashes do // not invalidate the pointee. RailMonitor* rail_monitor = nullptr; + int priority = PRIO_HIGH; }; static inline void updateSliceStatus(RdmaSlice* slice, diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/workers.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/workers.h index a123c9de..83b3da8e 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/workers.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/workers.h @@ -29,11 +29,14 @@ #include "rail_monitor.h" #include "tent/common/utils/os.h" #include "tent/common/concurrent/bounded_mpsc_queue.h" +#include "tent/common/types.h" namespace mooncake { namespace tent { class RdmaTransport; +class DeviceSelector; + class Workers { public: static constexpr size_t kCapacity = 1024 * 8; @@ -54,6 +57,8 @@ class Workers { Status cancel(RdmaSliceList &slice_list); + DeviceSelector *getDeviceSelector() const { return device_selector_.get(); } + private: using Task = std::function; @@ -176,9 +181,11 @@ class Workers { PerfMetric inflight_lat; }; + static constexpr int kNumPriorityLevels = PRIO_LOW + 1; + struct WorkerContext { std::thread thread; - BoundedSliceQueue queue; + BoundedSliceQueue queues[kNumPriorityLevels]; // Priority queues GroupedRequests requests; std::unordered_set inflight_slice_set; std::atomic inflight_slices = 0; @@ -187,18 +194,25 @@ class Workers { std::condition_variable cv; volatile bool in_suspend = false; + // Next time to check for priority promotions (nanoseconds) + uint64_t next_promotion_check_ns = 0; + // Values are held via unique_ptr so that map rehashing does not // invalidate pointers into RailMonitor stored on in-flight slices // (see RdmaSlice::rail_monitor). std::unordered_map> rails; PerfMetricSummary perf; - uint64_t padding[16]; + uint64_t padding[15]; }; + // Promote timed-out low priority requests to higher priority queues + void promoteTimedOutRequests(WorkerContext &worker); + WorkerContext *worker_context_; uint64_t slice_timeout_ns_; + uint64_t priority_promotion_timeout_ns_; // Timeout for priority promotion - std::unique_ptr device_quota_; + std::unique_ptr device_selector_; bool always_tier1_ = false; }; } // namespace tent diff --git a/mooncake-transfer-engine/tent/src/python/pybind.cpp b/mooncake-transfer-engine/tent/src/python/pybind.cpp index af219231..805858d2 100644 --- a/mooncake-transfer-engine/tent/src/python/pybind.cpp +++ b/mooncake-transfer-engine/tent/src/python/pybind.cpp @@ -259,6 +259,11 @@ PYBIND11_MODULE(tent, m) { m.attr("LOCAL_SEGMENT_ID") = py::int_(LOCAL_SEGMENT_ID); m.attr("kWildcardLocation") = py::str(kWildcardLocation); + // Priority constants + m.attr("PRIO_HIGH") = py::int_(PRIO_HIGH); + m.attr("PRIO_MEDIUM") = py::int_(PRIO_MEDIUM); + m.attr("PRIO_LOW") = py::int_(PRIO_LOW); + // ------------------------------------------------------------------------- // Enums // ------------------------------------------------------------------------- @@ -314,17 +319,19 @@ PYBIND11_MODULE(tent, m) { .def(py::init<>()) .def(py::init([](Request::OpCode opcode, uint64_t source, uint64_t target_id, uint64_t target_offset, - size_t length) { + size_t length, int priority) { Request r; r.opcode = opcode; r.source = U64ToPtr(source); r.target_id = target_id; r.target_offset = target_offset; r.length = length; + r.priority = priority; return r; }), py::arg("opcode"), py::arg("source"), py::arg("target_id"), - py::arg("target_offset"), py::arg("length")) + py::arg("target_offset"), py::arg("length"), + py::arg("priority") = PRIO_HIGH) .def_property( "opcode", [](const Request& r) { return r.opcode; }, [](Request& r, Request::OpCode op) { r.opcode = op; }) @@ -333,7 +340,8 @@ PYBIND11_MODULE(tent, m) { [](Request& r, uint64_t addr) { r.source = U64ToPtr(addr); }) .def_readwrite("target_id", &Request::target_id) .def_readwrite("target_offset", &Request::target_offset) - .def_readwrite("length", &Request::length); + .def_readwrite("length", &Request::length) + .def_readwrite("priority", &Request::priority); py::class_(m, "TransferStatus") .def(py::init<>()) diff --git a/mooncake-transfer-engine/tent/src/transfer_engine_c.cpp b/mooncake-transfer-engine/tent/src/transfer_engine_c.cpp index bf8797f8..813220ad 100644 --- a/mooncake-transfer-engine/tent/src/transfer_engine_c.cpp +++ b/mooncake-transfer-engine/tent/src/transfer_engine_c.cpp @@ -216,6 +216,7 @@ int tent_submit(tent_engine_t engine, tent_batch_id_t batch_id, req_list[index].target_id = entries[index].target_id; req_list[index].target_offset = entries[index].target_offset; req_list[index].length = entries[index].length; + req_list[index].priority = entries[index].priority; } auto status = CAST(engine)->submitTransfer(batch_id, req_list); if (!status.ok()) { @@ -241,6 +242,7 @@ int tent_submit_notif(tent_engine_t engine, tent_batch_id_t batch_id, req_list[index].target_id = entries[index].target_id; req_list[index].target_offset = entries[index].target_offset; req_list[index].length = entries[index].length; + req_list[index].priority = entries[index].priority; } mooncake::tent::Notification notifi; notifi.name = name; diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/quota.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/quota.cpp index 202c19ad..53a8bb42 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/quota.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/quota.cpp @@ -15,175 +15,346 @@ #include "tent/transport/rdma/quota.h" #include "tent/transport/rdma/shared_quota.h" #include "tent/common/utils/random.h" +#include "tent/common/utils/os.h" -#include -#include +#include +#include +#include namespace mooncake { namespace tent { -Status DeviceQuota::loadTopology(std::shared_ptr& local_topology) { +Status DeviceSelector::loadTopology(std::shared_ptr& local_topology) { local_topology_ = local_topology; - std::unordered_set used_numa_id; for (size_t dev_id = 0; dev_id < local_topology->getNicCount(); ++dev_id) { auto entry = local_topology->getNicEntry(dev_id); - if (entry->type != Topology::NIC_RDMA) continue; + if (!entry || entry->type != Topology::NIC_RDMA) continue; DeviceInfo& info = devices_[dev_id]; info.dev_id = dev_id; - info.bw_gbps = 200.0; + info.bw_gbps = kDefaultBwGbps; info.numa_id = entry->numa_node; - used_numa_id.insert(entry->numa_node); + info.ewma_bandwidth_bps.store(info.getTheoreticalBandwidth(), + std::memory_order_relaxed); } - if (used_numa_id.size() == 1) allow_cross_numa_ = true; + // Initialize device base priorities after all devices are loaded + fillDevicePriorities(); return Status::OK(); } -Status DeviceQuota::enableSharedQuota(const std::string& shm_name) { - shared_quota_ = std::make_shared(this); - auto status = shared_quota_->attach(shm_name); - if (!status.ok()) shared_quota_.reset(); +Status DeviceSelector::enableSharedQuota(const std::string& shm_name) { + slot_manager_ = std::make_shared(this); + slot_manager_->setRotationIntervalMs( + sched_params_.slot_rotation_interval_ms); + auto status = slot_manager_->attach(shm_name); + if (!status.ok()) slot_manager_.reset(); return status; } -struct TlsDeviceInfo { - uint64_t active_bytes{0}; - double beta0{0.0}; - double beta1{1.0}; -}; - -thread_local std::unordered_map tl_device_info; - -Status DeviceQuota::allocate(uint64_t length, const std::string& location, - int& chosen_dev_id) { +Status DeviceSelector::allocate(uint64_t total_length, uint32_t num_slices, + uint64_t slice_bytes, + const std::string& location, + std::vector& slice_dev_ids, int priority, + uint64_t device_mask) { + slice_dev_ids.clear(); + slice_dev_ids.reserve(num_slices); auto entry = local_topology_->getMemEntry(location); if (!entry) return Status::InvalidArgument("Unknown location" LOC_MARK); - if (!enable_quota_) { - thread_local int id = 0; + if (!smart_selection_enabled_) { + // Baseline mode: consistent with original TE behavior + // Use devices from the first non-empty rank only + thread_local uint64_t tl_rr_counter = 0; for (size_t rank = 0; rank < Topology::DevicePriorityRanks; ++rank) { - auto& list = entry->device_list[rank]; - if (list.empty()) continue; - chosen_dev_id = list[id % list.size()]; - id++; + thread_local std::vector tl_eligible; + tl_eligible.clear(); + for (int dev_id : entry->device_list[rank]) { + if (!devices_.count(dev_id)) continue; + if ((device_mask & (1ULL << dev_id)) == 0) continue; + tl_eligible.push_back(dev_id); + } + if (tl_eligible.empty()) continue; + + // Found first non-empty rank, do round-robin within this rank + uint64_t offset = 0; + for (uint32_t i = 0; i < num_slices; ++i) { + int dev_id = tl_eligible[tl_rr_counter % tl_eligible.size()]; + tl_rr_counter++; + slice_dev_ids.push_back(dev_id); + uint64_t this_slice_bytes = + std::min(slice_bytes, total_length - offset); + offset += this_slice_bytes; + devices_[dev_id].total_bytes.fetch_add( + this_slice_bytes, std::memory_order_relaxed); + } return Status::OK(); } - return Status::DeviceNotFound("no eligible devices for " + location); + return Status::DeviceNotFound("no eligible devices"); } - static constexpr double penalty[] = {1.0, 3.0, 10.0}; - const double w = local_weight_; - std::unordered_map score_map; - bool found_device = false; - double best_score = std::numeric_limits::infinity(); - for (size_t rank = 0; rank < Topology::DevicePriorityRanks; ++rank) { - if (rank == Topology::DevicePriorityRanks - 1 && !allow_cross_numa_ && - found_device) - continue; - for (int dev_id : entry->device_list[rank]) { - if (!devices_.count(dev_id)) continue; - auto& dev = devices_[dev_id]; - auto& tl_dev = tl_device_info[dev_id]; - uint64_t overall_active_bytes = - dev.diffusion_active_bytes.load(std::memory_order_relaxed) + - dev.active_bytes.load(std::memory_order_relaxed); - double weighted_active = w * tl_dev.active_bytes + - (1.0 - w) * overall_active_bytes + length; - double beta0_g = dev.beta0.load(std::memory_order_relaxed); - double beta1_g = dev.beta1.load(std::memory_order_relaxed); - double beta0 = w * tl_dev.beta0 + (1.0 - w) * beta0_g; - double beta1 = w * tl_dev.beta1 + (1.0 - w) * beta1_g; - double bw = dev.bw_gbps * 1e9 / 8; - double predicted_time = (weighted_active / bw) * beta1 + beta0; - score_map[dev_id] = penalty[rank] * predicted_time; - best_score = std::min(best_score, score_map[dev_id]); - found_device = true; - } + std::vector tl_candidates; + Status status = buildCandidates(entry, slice_bytes, device_mask, + tl_candidates, priority); + if (!status.ok()) return status; + if (num_slices == 1) { + selectSinglePath(tl_candidates, num_slices, total_length, + slice_dev_ids); + } else { + // Probe mode: every 100th call uses round-robin distribution + // to ensure all devices are sampled for EWMA updates + thread_local uint64_t tl_call_count = 0; + bool probe_mode = ((++tl_call_count % 100) == 0); + selectMultiPath(tl_candidates, num_slices, total_length, slice_dev_ids, + probe_mode); } - - if (!found_device) { - return Status::DeviceNotFound("no eligible devices for " + location); - } - - std::vector filtered; - for (const auto& [dev_id, score] : score_map) { - if (score <= best_score * 1.05) filtered.push_back(dev_id); - } - - std::sort(filtered.begin(), filtered.end(), [&](int a, int b) { - if (std::abs(score_map[a] - score_map[b]) > 1e-9) - return score_map[a] < score_map[b]; - return a < b; - }); - - thread_local size_t rr_index = 0; - chosen_dev_id = filtered[rr_index % filtered.size()]; - rr_index++; - - tl_device_info[chosen_dev_id].active_bytes += length; - if (local_weight_ < 1 - 1e-6) - devices_[chosen_dev_id].active_bytes.fetch_add( - length, std::memory_order_relaxed); return Status::OK(); } -Status DeviceQuota::release(int dev_id, uint64_t length, double latency) { - if (!enable_quota_) return Status::OK(); +int DeviceSelector::getDeviceRank(const std::string& location, + int dev_id) const { + auto entry = local_topology_->getMemEntry(location); + if (!entry) return 0; + for (size_t rank = 0; rank < Topology::DevicePriorityRanks; ++rank) { + for (int id : entry->device_list[rank]) { + if (id == dev_id) return static_cast(rank); + } + } + return 0; +} + +Status DeviceSelector::buildCandidates(const Topology::MemEntry* entry, + uint64_t slice_bytes, + uint64_t device_mask, + std::vector& candidates, + int request_priority) { + // Helper lambda to add candidate device + // Score formula: predicted_time × numa_penalty + random_jitter + // Lower score = better candidate + auto add_candidate = [&](int dev_id, size_t rank) { + auto& dev = devices_[dev_id]; + uint64_t inflight = dev.getInflightBytes(); + double ewma_bw = dev.getEwmaBandwidth(); + double predicted_time = + static_cast(inflight + slice_bytes) / ewma_bw; + double rank_penalty = sched_params_.numa_tier_weights[rank]; + double score = predicted_time * rank_penalty; + score += + (SimpleRandom::Get().next(10) * sched_params_.score_jitter_range); + bool is_cross_numa = (rank > 0); + Candidate c; + c.dev_id = dev_id; + c.score = score; + c.is_cross_numa = is_cross_numa; + candidates.push_back(c); + }; + + // First pass: filter by device priority (QoS filtering) + for (size_t rank = 0; rank < Topology::DevicePriorityRanks; ++rank) { + for (int dev_id : entry->device_list[rank]) { + if (!devices_.count(dev_id)) continue; + if ((device_mask & (1ULL << dev_id)) == 0) continue; + // QoS: Get device's current priority slot (local, per-process) + // Device accepts request if dev_priority >= request_priority + int dev_priority = PRIO_LOW; // Default: accept all + if (sched_params_.enable_priority_filtering) { + dev_priority = getDevicePriority(dev_id); + } + if (dev_priority < request_priority) continue; + add_candidate(dev_id, rank); + } + } + + // If no devices after filtering, fallback to all devices + if (candidates.empty()) { + for (size_t rank = 0; rank < Topology::DevicePriorityRanks; ++rank) { + for (int dev_id : entry->device_list[rank]) { + if (!devices_.count(dev_id)) continue; + if ((device_mask & (1ULL << dev_id)) == 0) continue; + add_candidate(dev_id, rank); + } + } + } + + if (candidates.empty()) { + return Status::DeviceNotFound("no eligible devices"); + } + + std::sort( + candidates.begin(), candidates.end(), + [this](const Candidate& a, const Candidate& b) { + if (std::abs(a.score - b.score) > sched_params_.score_jitter_range) + return a.score < b.score; + return a.dev_id < b.dev_id; + }); + return Status::OK(); +} + +void DeviceSelector::selectSinglePath(const std::vector& candidates, + uint32_t num_slices, + uint64_t total_length, + std::vector& slice_dev_ids) { + if (candidates.empty()) return; + + const Candidate& best = candidates[0]; + int dev_id = best.dev_id; + auto& dev = devices_[dev_id]; + + dev.addInflight(total_length); + dev.total_bytes.fetch_add(total_length, std::memory_order_relaxed); + + for (uint32_t i = 0; i < num_slices; ++i) { + slice_dev_ids.push_back(dev_id); + } +} + +void DeviceSelector::selectMultiPath(const std::vector& candidates, + uint32_t num_slices, uint64_t total_length, + std::vector& slice_dev_ids, + bool probe_mode) { + if (candidates.empty()) return; + uint64_t slice_bytes = (total_length + num_slices - 1) / num_slices; + if (probe_mode) { + // Probe mode: round-robin distribution to ensure all devices are + // sampled Activates every 100th call to prevent EWMA starvation + for (uint32_t i = 0; i < num_slices; ++i) { + const Candidate& c = candidates[i % candidates.size()]; + slice_dev_ids.push_back(c.dev_id); + devices_[c.dev_id].addInflight(slice_bytes); + devices_[c.dev_id].total_bytes.fetch_add(slice_bytes, + std::memory_order_relaxed); + } + } else { + // Normal mode: weighted distribution based on inverse score + // Lower score → higher weight → more slices + double total_weight = 0.0; + double max_weight = -1.0; + int best_dev_idx = -1; + for (size_t i = 0; i < candidates.size(); ++i) { + double w = + 1.0 / (candidates[i].score + sched_params_.score_epsilon); + total_weight += w; + if (w > max_weight) { + max_weight = w; + best_dev_idx = static_cast(i); + } + } + if (best_dev_idx == -1 || num_slices == 0 || total_weight <= 0.0) + return; + uint32_t remaining_slices = num_slices; + for (size_t i = 0; i < candidates.size(); ++i) { + double w = + 1.0 / (candidates[i].score + sched_params_.score_epsilon); + uint32_t assigned = + static_cast((w / total_weight) * num_slices); + if (assigned > 0) { + if (assigned > remaining_slices) assigned = remaining_slices; + remaining_slices -= assigned; + const Candidate& c = candidates[i]; + for (uint32_t s = 0; s < assigned; ++s) { + slice_dev_ids.push_back(c.dev_id); + } + uint64_t total_assigned_bytes = + static_cast(slice_bytes) * assigned; + devices_[c.dev_id].addInflight(total_assigned_bytes); + devices_[c.dev_id].total_bytes.fetch_add( + total_assigned_bytes, std::memory_order_relaxed); + } + } + if (remaining_slices > 0) { + const Candidate& c = candidates[best_dev_idx]; + for (uint32_t s = 0; s < remaining_slices; ++s) { + slice_dev_ids.push_back(c.dev_id); + } + uint64_t total_assigned_bytes = + static_cast(slice_bytes) * remaining_slices; + devices_[c.dev_id].addInflight(total_assigned_bytes); + devices_[c.dev_id].total_bytes.fetch_add(total_assigned_bytes, + std::memory_order_relaxed); + } + } +} + +Status DeviceSelector::allocate(uint64_t length, const std::string& location, + int& chosen_dev_id) { + std::vector slice_dev_ids; + Status status = allocate(length, 1, length, location, slice_dev_ids, ~0ULL); + if (!status.ok()) return status; + if (slice_dev_ids.empty()) { + return Status::DeviceNotFound("allocation failed"); + } + chosen_dev_id = slice_dev_ids[0]; + return Status::OK(); +} + +Status DeviceSelector::release(int dev_id, uint64_t length, double latency) { auto it = devices_.find(dev_id); if (it == devices_.end()) return Status::InvalidArgument("device not found"); auto& dev = it->second; - auto& tl_dev = tl_device_info[dev_id]; + dev.releaseInflight(length); - if (local_weight_ < 1 - 1e-6) - dev.active_bytes.fetch_sub(length, std::memory_order_relaxed); - tl_dev.active_bytes -= length; - - if (!update_quota_params_) return Status::OK(); - - double bw = dev.bw_gbps * 1e9 / 8; - double theory_time = static_cast(length) / bw; - double obs_time = latency; - - const double w = local_weight_; - double beta0_g = dev.beta0.load(std::memory_order_relaxed); - double beta1_g = dev.beta1.load(std::memory_order_relaxed); - double beta0 = w * tl_dev.beta0 + (1.0 - w) * beta0_g; - double beta1 = w * tl_dev.beta1 + (1.0 - w) * beta1_g; - - double pred_time = beta0 + beta1 * theory_time; - double err = obs_time - pred_time; - double rel_err = (pred_time > 1e-9) ? (err / pred_time) : 0.0; - - double adapt_alpha = alpha_; - if (std::abs(err) > 0.05 * pred_time) - adapt_alpha = std::min(1.0, alpha_ * 5.0); - - double delta0 = adapt_alpha * err; - double delta1 = adapt_alpha * rel_err; - - double new_beta0_l = tl_dev.beta0 + w * delta0; - double new_beta1_l = tl_dev.beta1 * (1.0 + w * delta1); - tl_dev.beta0 = std::clamp(new_beta0_l, 0.0, 5e-4); - tl_dev.beta1 = std::clamp(new_beta1_l, 0.5, 20.0); - - if (local_weight_ < 1 - 1e-6) { - double new_beta0_g = beta0_g + (1.0 - w) * delta0; - double new_beta1_g = beta1_g * (1.0 + (1.0 - w) * delta1); - dev.beta0.store(std::clamp(new_beta0_g, 0.0, 5e-4), - std::memory_order_relaxed); - dev.beta1.store(std::clamp(new_beta1_g, 0.5, 20.0), - std::memory_order_relaxed); - if (shared_quota_) { - thread_local uint64_t tl_last_ts = 0; - uint64_t now = getCurrentTimeInNano(); - if (now - tl_last_ts > diffusion_interval_) { - tl_last_ts = now; - return shared_quota_->diffusion(); - } - } + if (!smart_selection_enabled_) { + return Status::OK(); } + + // Update EWMA bandwidth: new = α × old + (1-α) × observed + // α = 0: always use observed (full adaptation) + // α = 1: never update (no learning) + double observed_bw = static_cast(length) / latency; + double current_ewma = dev.getEwmaBandwidth(); + + double alpha = sched_params_.bandwidth_learning_rate; + double new_ewma = alpha * current_ewma + (1.0 - alpha) * observed_bw; + + // Clamp to [min_multiplier, max_multiplier] of theoretical bandwidth + double theoretical_bw = dev.getTheoreticalBandwidth(); + new_ewma = std::max( + sched_params_.ewma_min_multiplier * theoretical_bw, + std::min(sched_params_.ewma_max_multiplier * theoretical_bw, new_ewma)); + + dev.ewma_bandwidth_bps.store(new_ewma, std::memory_order_relaxed); + return Status::OK(); } +void DeviceSelector::printTrafficStats() { + std::cout << "=== Device Traffic Statistics ===" << std::endl; + for (const auto& [dev_id, dev] : devices_) { + uint64_t total = dev.total_bytes.load(std::memory_order_relaxed); + double ewma_bw_gbps = dev.getEwmaBandwidth() / 1e9 * 8.0; + uint64_t inflight = dev.getInflightBytes(); + std::cout << "Dev " << dev_id << ": " + << "Total=" << (total / 1024.0 / 1024.0 / 1024.0) << " GB, " + << "EWMA BW=" << std::fixed << std::setprecision(2) + << ewma_bw_gbps << " Gbps, " + << "Inflight=" << inflight << " bytes" << std::endl; + } +} + +void DeviceSelector::fillDevicePriorities() { + sched_params_.device_base_priorities.clear(); + for (const auto& [dev_id, dev] : devices_) { + sched_params_.device_base_priorities.push_back(dev_id); + } +} + +int DeviceSelector::getDevicePriority(int dev_id) const { + if (!sched_params_.enable_priority_filtering) return 0; + auto it = std::find(sched_params_.device_base_priorities.begin(), + sched_params_.device_base_priorities.end(), dev_id); + if (it == sched_params_.device_base_priorities.end()) return 0; + size_t base_index = + std::distance(sched_params_.device_base_priorities.begin(), it); + size_t num_devices = sched_params_.device_base_priorities.size(); + if (sched_params_.local_rotation_interval_us > 0 && num_devices > 0) { + uint64_t now = getCurrentTimeInNano(); + uint64_t offset_us = now / 1000; + size_t rotation_offset = + (offset_us / sched_params_.local_rotation_interval_us) % + num_devices; + base_index = (base_index + rotation_offset) % num_devices; + } + return static_cast(base_index); +} + } // namespace tent } // namespace mooncake \ No newline at end of file diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp index aa273aea..90c4cdb8 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp @@ -14,6 +14,7 @@ #include "tent/transport/rdma/rdma_transport.h" #include "tent/transport/rdma/ibv_loader.h" +#include "tent/transport/rdma/quota.h" #include #include @@ -326,16 +327,14 @@ Status RdmaTransport::submitTransferTasks( const size_t default_block_size = params_->workers.block_size; const int num_workers = params_->workers.num_workers; - const int num_devices = (size_t)local_topology_->getNicCount(); std::vector slice_lists(num_workers); std::vector slice_tails(num_workers, nullptr); auto enqueue_ts = getCurrentTimeInNano(); + // Distribute starting worker across threads to avoid contention static std::atomic g_caller_threads(0); thread_local int tl_caller_id = g_caller_threads.fetch_add(1); - bool enable_spray = - g_caller_threads.load(std::memory_order_relaxed) <= num_workers; - int submit_slices = 0; + int next_worker_idx = tl_caller_id; for (auto& request : request_list) { auto opcode = request.opcode; auto type = Platform::getLoader().getMemoryType(request.source); @@ -367,8 +366,26 @@ Status RdmaTransport::submitTransferTasks( uint64_t block_size = roundup( (request.length + num_slices - 1) / num_slices, default_block_size); - num_slices = std::max( - 1, std::min(num_slices, max_slice_count)); + std::vector slice_dev_ids; + // Only if a single request is enough, we perform aggregated allocation + if (num_slices >= max_slice_count / 2) { + std::string source_location = kWildcardLocation; + auto source_locations = + Platform::getLoader().getLocation(request.source, 1, true); + if (!source_locations.empty()) { + source_location = source_locations[0].location; + } + auto device_selector = workers_->getDeviceSelector(); + if (device_selector) { + auto status = device_selector->allocate( + request.length, static_cast(num_slices), + block_size, source_location, slice_dev_ids); + if (!status.ok() || slice_dev_ids.empty()) { + LOG(WARNING) << "Device quota allocation failed: " + << status.message(); + } + } + } uint64_t offset = 0; for (uint64_t slice_idx = 0; slice_idx < num_slices; ++slice_idx) { @@ -384,17 +401,17 @@ Status RdmaTransport::submitTransferTasks( slice->word = PENDING; slice->next = nullptr; slice->enqueue_ts = enqueue_ts; + slice->priority = request.priority; // Copy priority from request task->num_slices++; task->ref(); // Each slice holds a reference to the task + if (slice_idx < slice_dev_ids.size()) + slice->source_dev_id = slice_dev_ids[slice_idx]; offset += length; - int part_id = - ((enable_spray ? submit_slices : static_cast(slice_idx)) / - num_devices) % - num_workers; + int part_id = next_worker_idx % num_workers; auto& list = slice_lists[part_id]; auto& tail = slice_tails[part_id]; list.num_slices++; - submit_slices++; + next_worker_idx++; if (list.first) { tail->next = slice; tail = slice; @@ -407,7 +424,7 @@ Status RdmaTransport::submitTransferTasks( for (int i = 0; i < num_workers; ++i) { if (slice_lists[i].first) { rdma_batch->slice_chain.push_back(slice_lists[i].first); - workers_->submit(slice_lists[i], (tl_caller_id + i) % num_workers); + workers_->submit(slice_lists[i], i); } } return Status::OK(); diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/shared_quota.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/shared_quota.cpp index 5aff9c18..f0b5a4ec 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/shared_quota.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/shared_quota.cpp @@ -14,94 +14,43 @@ #include "tent/transport/rdma/shared_quota.h" #include "tent/common/utils/os.h" +#include "tent/common/types.h" + +#include +#include namespace mooncake { namespace tent { -SharedQuotaManager::SharedQuotaManager(DeviceQuota* local_quota) +SharedSlotManager::SharedSlotManager(DeviceSelector* device_selector) : hdr_(nullptr), fd_(-1), size_(sizeof(SharedHeader)), created_(false), - local_quota_(local_quota) {} + device_selector_(device_selector), + background_running_(false) {} -SharedQuotaManager::~SharedQuotaManager() { detach(); } +SharedSlotManager::~SharedSlotManager() { detach(); } -Status SharedQuotaManager::attach(const std::string& shm_name) { - name_ = shm_name; - - // Open or create shared memory (mode 0666) - fd_ = shm_open(name_.c_str(), O_RDWR | O_CREAT, 0666); - if (fd_ < 0) { - return Status::InternalError("shm_open failed: " + - std::string(strerror(errno))); - } - - // Ensure size - if (ftruncate(fd_, static_cast(size_)) != 0) { - int e = errno; - close(fd_); - fd_ = -1; - return Status::InternalError("ftruncate failed: " + - std::string(strerror(e))); - } - - // mmap - void* ptr = - mmap(nullptr, size_, PROT_READ | PROT_WRITE, MAP_SHARED, fd_, 0); - if (ptr == MAP_FAILED) { - int e = errno; - close(fd_); - fd_ = -1; - return Status::InternalError("mmap failed: " + - std::string(strerror(e))); - } - - hdr_ = reinterpret_cast(ptr); - - if (hdr_->magic != SHM_MAGIC || hdr_->version != SHM_VERSION) { - created_ = true; - Status s = initializeHeader(); - if (!s.ok()) { - munmap(ptr, size_); - close(fd_); - hdr_ = nullptr; - fd_ = -1; - return s; - } - } else { - created_ = false; - } - - Status s = attachProcess(); - if (!s.ok()) return s; - return Status::OK(); +bool SharedSlotManager::isPriorityAllowedInSlot(int priority, int slot) const { + return priority <= slot; } -Status SharedQuotaManager::detach() { - if (hdr_) { - detachProcess(); - munmap(hdr_, size_); - hdr_ = nullptr; - } - if (fd_ >= 0) { - close(fd_); - fd_ = -1; - } - return Status::OK(); -} +Status SharedSlotManager::initializeHeader() { + hdr_->magic = 0; + hdr_->version = 0; + hdr_->current_slot.store(0, std::memory_order_relaxed); -Status SharedQuotaManager::initializeHeader() { - memset(hdr_, 0, size_); Status s = initMutex(&hdr_->global_mutex); - if (!s.ok()) { - return s; - } + if (!s.ok()) return s; + hdr_->version = SHM_VERSION; hdr_->magic = SHM_MAGIC; + hdr_->current_slot.store(0, std::memory_order_release); + return Status::OK(); } -Status SharedQuotaManager::initMutex(pthread_mutex_t* m) { +Status SharedSlotManager::initMutex(pthread_mutex_t* m) { pthread_mutexattr_t attr; if (pthread_mutexattr_init(&attr) != 0) { return Status::InternalError("pthread_mutexattr_init failed"); @@ -124,162 +73,115 @@ Status SharedQuotaManager::initMutex(pthread_mutex_t* m) { return Status::OK(); } -// attempt to acquire global lock; handle EOWNERDEAD -int SharedQuotaManager::lock() { - if (!hdr_) return EINVAL; - int rc = pthread_mutex_lock(&hdr_->global_mutex); - if (rc == 0) return 0; - if (rc == EOWNERDEAD) { - // make consistent so others can continue -#if defined(PTHREAD_MUTEX_ROBUST) - int rc2 = pthread_mutex_consistent(&hdr_->global_mutex); - if (rc2 != 0) { - return rc2; +Status SharedSlotManager::attach(const std::string& shm_name) { + name_ = shm_name; + + fd_ = shm_open(name_.c_str(), O_RDWR | O_CREAT, 0666); + if (fd_ < 0) { + return Status::InternalError("shm_open failed: " + + std::string(std::strerror(errno))); + } + + if (ftruncate(fd_, static_cast(size_)) != 0) { + int e = errno; + close(fd_); + fd_ = -1; + return Status::InternalError("ftruncate failed: " + + std::string(std::strerror(e))); + } + + void* ptr = + mmap(nullptr, size_, PROT_READ | PROT_WRITE, MAP_SHARED, fd_, 0); + if (ptr == MAP_FAILED) { + int e = errno; + close(fd_); + fd_ = -1; + return Status::InternalError("mmap failed: " + + std::string(std::strerror(e))); + } + + hdr_ = reinterpret_cast(ptr); + + if (hdr_->magic != SHM_MAGIC || hdr_->version != SHM_VERSION) { + created_ = true; + Status s = initializeHeader(); + if (!s.ok()) { + munmap(ptr, size_); + close(fd_); + hdr_ = nullptr; + fd_ = -1; + return s; } -#endif - // We hold the lock now — repair data if needed - reclaimDeadPidsInternal(); - return 0; - } - return rc; -} - -int SharedQuotaManager::unlock() { - if (!hdr_) return EINVAL; - return pthread_mutex_unlock(&hdr_->global_mutex); -} - -void SharedQuotaManager::reclaimDeadPidsInternal() { - if (!hdr_) return; - // Assumes caller holds lock (but we also call from lock() on EOWNERDEAD) - for (int i = 0; i < hdr_->num_devices; ++i) { - // skip unused slots (dev_name empty) - if (hdr_->devices[i].dev_name[0] == '\0') continue; - SharedDeviceEntry& dev = hdr_->devices[i]; - for (int s = 0; s < MAX_PID_SLOTS; ++s) { - pid_t p = dev.pid_usages[s].pid; - if (p == 0) continue; - if (!isPidAlive(p)) { - // zero out slot — we'll recompute active_bytes in diffusion - dev.pid_usages[s].pid = 0; - dev.pid_usages[s].used_bytes = 0; - } - } - } -} - -bool SharedQuotaManager::isPidAlive(pid_t pid) { - if (pid <= 0) return false; - int r = kill(pid, 0); - if (r == 0) return true; - if (errno == ESRCH) return false; - return true; // other errors (EPERM) -> treat as alive -} - -int SharedQuotaManager::findDeviceIdByNameLocked(const std::string& dev_name) { - if (!hdr_) return -1; - for (int i = 0; i < hdr_->num_devices; ++i) { - if (hdr_->devices[i].dev_name[0] == '\0') continue; - if (strncmp(hdr_->devices[i].dev_name, dev_name.c_str(), - sizeof(hdr_->devices[i].dev_name)) == 0) - return i; - } - return -1; -} - -PidUsage* SharedQuotaManager::findOrCreatePidSlotLocked(int dev_id, pid_t pid) { - if (!hdr_) return nullptr; - if (dev_id < 0 || dev_id >= hdr_->num_devices) return nullptr; - SharedDeviceEntry& dev = hdr_->devices[dev_id]; - PidUsage* empty = nullptr; - for (int s = 0; s < MAX_PID_SLOTS; ++s) { - if (dev.pid_usages[s].pid == pid) return &dev.pid_usages[s]; - if (dev.pid_usages[s].pid == 0 && empty == nullptr) - empty = &dev.pid_usages[s]; - } - if (empty) { - empty->pid = pid; - empty->used_bytes = 0; - } - return empty; -} - -PidUsage* SharedQuotaManager::findPidSlotLocked(int dev_id, pid_t pid) { - if (!hdr_) return nullptr; - if (dev_id < 0 || dev_id >= hdr_->num_devices) return nullptr; - SharedDeviceEntry& dev = hdr_->devices[dev_id]; - for (int s = 0; s < MAX_PID_SLOTS; ++s) { - if (dev.pid_usages[s].pid == pid) return &dev.pid_usages[s]; - } - return nullptr; -} - -Status SharedQuotaManager::attachProcess() { - if (!hdr_) return Status::InvalidArgument("not attached"); - - int rc = lock(); - if (rc != 0) { - return Status::InternalError("failed to lock shared mutex: " + - std::string(strerror(rc))); + } else { + created_ = false; } - auto topo = local_quota_->getTopology(); - for (size_t i = 0; i < topo->getNicCount(); ++i) { - if (topo->getNicType(i) != Topology::NIC_RDMA) continue; - auto dev_name = topo->getNicName(i); - if (findDeviceIdByNameLocked(dev_name) >= 0) continue; - int empty_idx = -1; - for (int j = 0; j < MAX_DEVICES; ++j) { - if (hdr_->devices[j].dev_name[0] == '\0') { - empty_idx = j; - break; - } - } - if (empty_idx < 0) continue; - strncpy(hdr_->devices[empty_idx].dev_name, dev_name.c_str(), 56); - hdr_->devices[empty_idx].active_bytes = 0; - } + startBackgroundThread(); - int count = 0; - for (int i = 0; i < MAX_DEVICES; ++i) - if (hdr_->devices[i].dev_name[0] != '\0') ++count; - hdr_->num_devices = count; - - unlock(); return Status::OK(); } -Status SharedQuotaManager::detachProcess() { return Status::OK(); } +Status SharedSlotManager::detach() { + stopBackgroundThread(); -Status SharedQuotaManager::diffusion() { - if (!hdr_) return Status::InvalidArgument("not attached"); - pid_t pid = getpid(); - int rc = lock(); - if (rc != 0) - return Status::InternalError("lock failed: " + - std::string(strerror(rc))); - for (int d = 0; d < hdr_->num_devices; ++d) { - std::string dev_name = hdr_->devices[d].dev_name; - auto dev_id = local_quota_->getTopology()->getNicId(dev_name); - if (dev_name.empty() || dev_id < 0) continue; - PidUsage* slot = findOrCreatePidSlotLocked(dev_id, pid); - if (!slot) { - unlock(); - return Status::InternalError("no free pid slot for device"); - } - auto used_bytes = local_quota_->getActiveBytes(dev_id); - slot->used_bytes = used_bytes; - uint64_t sum = 0; - for (int s = 0; s < MAX_PID_SLOTS; ++s) - sum += hdr_->devices[d].pid_usages[s].used_bytes; - uint64_t diffusion_active_bytes = - sum < used_bytes ? 0 : sum - used_bytes; - hdr_->devices[d].active_bytes = sum; - local_quota_->setDiffusionActiveBytes(dev_id, diffusion_active_bytes); + if (hdr_) { + munmap(hdr_, size_); + hdr_ = nullptr; } - unlock(); + + if (fd_ >= 0) { + close(fd_); + fd_ = -1; + } + return Status::OK(); } +bool SharedSlotManager::canSend(int priority) { + if (!hdr_) return true; + + // Get current global slot + int current_slot = hdr_->current_slot.load(std::memory_order_acquire); + + // Check if the given priority is allowed in current global slot + // Slot 0: HIGH only (priority 0 <= 0) + // Slot 1: HIGH + MEDIUM (priority 0 or 1 <= 1) + // Slot 2: ALL (priority 0, 1, or 2 <= 2) + return isPriorityAllowedInSlot(priority, current_slot); +} + +void SharedSlotManager::startBackgroundThread() { + if (background_running_.exchange(true)) return; + + background_thread_ = std::thread([this]() { backgroundThreadLoop(); }); +} + +void SharedSlotManager::stopBackgroundThread() { + if (!background_running_.exchange(false)) return; + + if (background_thread_.joinable()) { + background_thread_.join(); + } +} + +// Background thread: advance global slot periodically +void SharedSlotManager::backgroundThreadLoop() { + const uint64_t SLEEP_INTERVAL_US = 1000; // 1ms + + while (background_running_.load(std::memory_order_relaxed)) { + // Calculate base slot from time + uint64_t now = getCurrentTimeInNano(); + uint64_t base_slot = now / (rotation_interval_ms_ * 1000000ull); + + pthread_mutex_lock(&hdr_->global_mutex); + // Update global slot + int global_slot = static_cast(base_slot % NUM_SLOTS); + hdr_->current_slot.store(global_slot, std::memory_order_release); + pthread_mutex_unlock(&hdr_->global_mutex); + + usleep(SLEEP_INTERVAL_US); + } +} + } // namespace tent } // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp index d2c51038..0d7996f5 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp @@ -19,7 +19,7 @@ #include #include "tent/transport/rdma/endpoint_store.h" -#include "tent/transport/rdma/rail_monitor.h" +#include "tent/transport/rdma/shared_quota.h" #include "tent/common/utils/ip.h" #include "tent/common/utils/string_builder.h" #include "tent/common/utils/os.h" @@ -45,23 +45,108 @@ RailMonitor& getOrCreateRail( Workers::Workers(RdmaTransport* transport) : transport_(transport), num_workers_(0), running_(false) { - device_quota_ = std::make_unique(); - device_quota_->loadTopology(transport_->local_topology_); + device_selector_ = std::make_unique(); + device_selector_->loadTopology(transport_->local_topology_); auto& conf = transport_->conf_; + + // ============================================================ + // Core Scheduling Configuration + // ============================================================ + + // Enable/disable smart scheduling (false = simple round-robin) + bool enable_smart_scheduling = + conf->get("transports/rdma/enable_smart_scheduling", true); + device_selector_->setSmartSelection(enable_smart_scheduling); + + // ============================================================ + // NUMA Distance Penalties + // Higher values = higher penalty for cross-NUMA access + // Format: [local_numa, remote_numa1, remote_numa2, ...] + // ============================================================ + DeviceSelector::SchedulingParams params; + + auto numa_penalties = + conf->get("transports/rdma/numa_penalties", std::vector{}); + if (numa_penalties.size() == Topology::DevicePriorityRanks) { + for (size_t i = 0; i < Topology::DevicePriorityRanks; ++i) { + params.numa_tier_weights[i] = numa_penalties[i]; + } + } + + // ============================================================ + // Bandwidth Estimation (EWMA) + // ============================================================ + + // Learning rate: 0.0 = full adaptation, 1.0 = no adaptation + params.bandwidth_learning_rate = + conf->get("transports/rdma/bandwidth_learning_rate", 0.01); + + // EWMA bounds as multipliers of theoretical bandwidth + params.ewma_min_multiplier = + conf->get("transports/rdma/ewma_min_bandwidth_multiplier", 0.1); + params.ewma_max_multiplier = + conf->get("transports/rdma/ewma_max_bandwidth_multiplier", 10.0); + + // ============================================================ + // Device Selection Scoring + // ============================================================ + + // Random jitter to avoid deterministic selection + params.score_jitter_range = + conf->get("transports/rdma/score_jitter_range", 1e-9); + + // Small value to prevent division by zero + params.score_epsilon = conf->get("transports/rdma/score_epsilon", 1e-12); + + // ============================================================ + // Priority-Based Filtering + // ============================================================ + + params.enable_priority_filtering = + conf->get("transports/rdma/enable_priority_filtering", true); + + // Local device priority rotation interval (microseconds) + params.local_rotation_interval_us = + conf->get("transports/rdma/local_rotation_interval_us", 200); + + // ============================================================ + // Priority Promotion (Anti-Starvation) + // ============================================================ + + // Timeout after which low-priority requests get promoted (nanoseconds) + // Default: 10ms (10000000 ns) + priority_promotion_timeout_ns_ = + conf->get("transports/rdma/priority_promotion_timeout_us", 10000) * + 1000ull; + + // ============================================================ + // Global Slot Coordination (Multi-Process) + // ============================================================ + + params.slot_rotation_interval_ms = + conf->get("transports/rdma/slot_rotation_interval_ms", 2); + + // ============================================================ + // Bandwidth Constants (Gbps) + // ============================================================ + + params.default_bandwidth_gbps = + conf->get("transports/rdma/default_bandwidth_gbps", 400.0); + params.min_bandwidth_gbps = + conf->get("transports/rdma/min_bandwidth_gbps", 10.0); + params.max_bandwidth_gbps = + conf->get("transports/rdma/max_bandwidth_gbps", 800.0); + + device_selector_->setSchedulingParams(params); + + // ============================================================ + // Shared Memory Configuration + // ============================================================ + auto shared_quota_shm_path = conf->get("transports/rdma/shared_quota_shm_path", ""); if (!shared_quota_shm_path.empty()) - device_quota_->enableSharedQuota(shared_quota_shm_path); - auto cross_numa_access = - conf->get("transports/rdma/cross_numa_access", false); - device_quota_->setCrossNumaAccess(cross_numa_access); - auto local_weight = conf->get("transports/rdma/local_weight", 1.0); - device_quota_->setLocalWeight(local_weight); - auto learning_rate = conf->get("transports/rdma/learning_rate", 0.1); - device_quota_->setLearningRate(learning_rate); - auto diffusion_interval = - conf->get("transports/rdma/diffusion_interval", 10); - device_quota_->setDiffusionInterval(diffusion_interval); + device_selector_->enableSharedQuota(shared_quota_shm_path); } Workers::~Workers() { @@ -118,7 +203,14 @@ Status Workers::submit(RdmaSliceList& slice_list, int worker_id) { } } auto& worker = worker_context_[worker_id]; - worker.queue.push(slice_list); + + // Get priority from first slice (all slices in list have same priority) + int priority = PRIO_HIGH; + if (slice_list.first && slice_list.first->task) { + priority = slice_list.first->priority; + } + + worker.queues[priority].push(slice_list); if (!worker.inflight_slices.fetch_add(slice_list.num_slices)) { std::lock_guard lock(worker.mutex); if (worker.in_suspend) worker.cv.notify_all(); @@ -212,7 +304,20 @@ void Workers::disableEndpoint(RdmaSlice* slice) { void Workers::asyncPostSend() { auto& worker = worker_context_[tl_wid]; std::vector result; - worker.queue.pop(result); + + auto shared_quota = + device_selector_ ? device_selector_->getSharedSlotManager() : nullptr; + + // Promote timed-out low priority requests + promoteTimedOutRequests(worker); + + // Priority selection: HIGH -> MEDIUM -> LOW + for (int prio = PRIO_HIGH; prio < kNumPriorityLevels; ++prio) { + if (shared_quota && !shared_quota->canSend(prio)) continue; + worker.queues[prio].pop(result); + if (!result.empty()) break; + } + for (auto& slice_list : result) { if (slice_list.num_slices == 0) continue; auto slice = slice_list.first; @@ -283,6 +388,49 @@ void Workers::asyncPostSend() { } } +void Workers::promoteTimedOutRequests(WorkerContext& worker) { + uint64_t current_ts = getCurrentTimeInNano(); + if (current_ts < worker.next_promotion_check_ns) return; + + // Set next check time (1ms from now) + worker.next_promotion_check_ns = current_ts + 1000000ull; + + // Check MEDIUM -> HIGH promotion + std::vector promoted; + worker.queues[PRIO_MEDIUM].pop(promoted); + if (!promoted.empty()) { + auto* slice = promoted.front().first; + if (slice && slice->enqueue_ts > 0 && + (current_ts - slice->enqueue_ts) >= + priority_promotion_timeout_ns_) { + for (auto& slice_list : promoted) { + worker.queues[PRIO_HIGH].push(slice_list); + } + return; + } + for (auto& slice_list : promoted) { + worker.queues[PRIO_MEDIUM].push(slice_list); + } + } + + // Check LOW -> MEDIUM promotion + worker.queues[PRIO_LOW].pop(promoted); + if (!promoted.empty()) { + auto* slice = promoted.front().first; + if (slice && slice->enqueue_ts > 0 && + (current_ts - slice->enqueue_ts) >= + priority_promotion_timeout_ns_) { + for (auto& slice_list : promoted) { + worker.queues[PRIO_MEDIUM].push(slice_list); + } + return; + } + for (auto& slice_list : promoted) { + worker.queues[PRIO_LOW].push(slice_list); + } + } +} + void Workers::asyncPollCq() { auto& worker = worker_context_[tl_wid]; const static size_t kPollCount = 64; @@ -328,8 +476,8 @@ void Workers::asyncPollCq() { double inflight_lat = (poll_ts - slice->submit_ts) / 1000.0; double overall_lat_sec = (poll_ts - slice->enqueue_ts) / 1e9; if (slice->retry_count == 0) { - device_quota_->release(slice->source_dev_id, slice->length, - overall_lat_sec); + device_selector_->release(slice->source_dev_id, slice->length, + overall_lat_sec); } if (slice->word != PENDING) continue; if (!ep) { @@ -551,7 +699,7 @@ Status Workers::selectOptimalDevice(RouteHint& source, RouteHint& target, RdmaSlice* slice) { auto& worker = worker_context_[tl_wid]; if (slice->source_dev_id < 0) { - CHECK_STATUS(device_quota_->allocate( + CHECK_STATUS(device_selector_->allocate( slice->length, source.buffer->location, slice->source_dev_id)); } -- 2.34.1 From 0ba98bf0be2d1933ada30dffb561ab7c19f90f21 Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Fri, 22 May 2026 15:07:44 +0800 Subject: [PATCH 112/382] [TENT] Add codeowner to the TENT directory (#2183) --------- Co-authored-by: Teng Ma --- .github/CODEOWNERS | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 6641f3f1..eabf4b29 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -15,6 +15,7 @@ /mooncake-store @ykwd @stmatengss @XucSh @YiXR /mooncake-store/*/ha/ @Libotry @YiXR @00fish0 /mooncake-transfer-engine @alogfans @doujiang24 @chestnut-Q +/mooncake-transfer-engine/tent @alogfans @doujiang24 @chestnut-Q @staryxchen @00fish0 @dtcccc /mooncake-transfer-engine/*/transport/hip_transport/ @alogfans @amd-arozanov /mooncake-transfer-engine/*/transport/ascend_transport/ @alogfans @ascend-direct-dev /mooncake-transfer-engine/*/transport/efa_transport/ @alogfans @whn09 -- 2.34.1 From 3ef70993eda1d780f27fc585db7363172d17ba35 Mon Sep 17 00:00:00 2001 From: JieTang66 <57845979+JieTang66@users.noreply.github.com> Date: Fri, 22 May 2026 15:31:07 +0800 Subject: [PATCH 113/382] [CI] Add release-npu workflow for Ascend NPU release (#2178) --- .github/workflows/release-npu.yaml | 230 +++++++++++++++++ scripts/ascend/dependencies_openeuler.sh | 310 +++++++++++++++++++++++ scripts/build_wheel.sh | 53 +++- 3 files changed, 582 insertions(+), 11 deletions(-) create mode 100644 .github/workflows/release-npu.yaml create mode 100644 scripts/ascend/dependencies_openeuler.sh diff --git a/.github/workflows/release-npu.yaml b/.github/workflows/release-npu.yaml new file mode 100644 index 00000000..1494b8b7 --- /dev/null +++ b/.github/workflows/release-npu.yaml @@ -0,0 +1,230 @@ +name: Release Ascend NPU + +on: + push: + tags: + - 'v*' + +jobs: + build: + if: github.repository == 'kvcache-ai/Mooncake' + + runs-on: self-hosted + + permissions: + contents: write + + strategy: + max-parallel: 1 + matrix: + python-version: ['3.10', '3.11', '3.12', '3.13'] + + env: + BUILD_WITH_EP: "0" + NPU_BUILD: "1" + + container: + image: localhost:5000/mooncake-hixl-ci:v6-release + + options: --privileged --user 0:0 --device /dev/davinci0 --device /dev/davinci1 --device /dev/davinci2 --device /dev/davinci3 + --device /dev/davinci4 --device /dev/davinci5 --device /dev/davinci6 --device /dev/davinci7 + --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc --ulimit nproc=65535:65535 + + volumes: + - /usr/local/dcmi:/usr/local/dcmi:ro + - /usr/local/Ascend/driver/:/usr/local/Ascend/driver/:ro + - /etc/ascend_install.info:/etc/ascend_install.info:ro + - /etc/hccn.conf:/etc/hccn.conf:ro + + steps: + - name: Configure GitHub fetch defaults + shell: bash + run: | + git config --global protocol.version 2 + git config --global http.version HTTP/1.1 + git config --global http.lowSpeedLimit 1024 + git config --global http.lowSpeedTime 30 + + - name: Checkout code + id: checkout_code + continue-on-error: true + uses: actions/checkout@v4 + with: + fetch-depth: 1 + persist-credentials: false + + - name: Retry checkout via GitHub mirror + if: steps.checkout_code.outcome == 'failure' + shell: bash + env: + ASCEND_GITHUB_MIRROR_URLS: 'https://ghfast.top/' + run: | + set -euo pipefail + + if [ -z "${ASCEND_GITHUB_MIRROR_URLS:-}" ]; then + echo "Checkout from GitHub failed and ASCEND_GITHUB_MIRROR_URLS is not set" + exit 1 + fi + + normalize_base() { + local base="$1" + base="${base#${base%%[![:space:]]*}}" + base="${base%${base##*[![:space:]]}}" + [ -n "$base" ] || return 1 + [ "$base" != "https://github.com/" ] && base="${base%/}/" + printf '%s\n' "$base" + } + + candidates=() + while IFS= read -r raw; do + base="$(normalize_base "$raw" || true)" + [ -n "$base" ] || continue + [ "$base" = "https://github.com/" ] && continue + candidates+=("$base") + done < <(printf '%s\n' "$ASCEND_GITHUB_MIRROR_URLS" | tr ',;' '\n') + + if [ ${#candidates[@]} -eq 0 ]; then + echo "Checkout from GitHub failed and no valid mirror candidates were configured" + exit 1 + fi + + workdir="${GITHUB_WORKSPACE}" + git config --global --add safe.directory "$workdir" + + for base in "${candidates[@]}"; do + mirror_url="${base}https://github.com/${GITHUB_REPOSITORY}.git" + echo "Retrying checkout with ${mirror_url}" + + find "$workdir" -mindepth 1 -maxdepth 1 -exec rm -rf {} + + git init "$workdir" + git -C "$workdir" remote add origin "$mirror_url" + + if git -C "$workdir" fetch --depth=1 origin "${{ github.sha }}" && \ + git -C "$workdir" checkout --force --detach FETCH_HEAD; then + echo "Mirror checkout succeeded via ${base}" + exit 0 + fi + + echo "Mirror checkout failed via ${base}" + rm -rf "$workdir/.git" + done + + echo "Direct GitHub checkout failed and all mirror retries failed" + exit 1 + + - name: Use container Python ${{ matrix.python-version }} + shell: bash + run: | + set -euo pipefail + PYTHON_BIN="$(command -v python${{ matrix.python-version }})" + "$PYTHON_BIN" --version + "$PYTHON_BIN" -m pip --version + echo "PYTHON_BIN=${PYTHON_BIN}" >> "$GITHUB_ENV" + + - name: Install dependencies + shell: bash + env: + ASCEND_GITHUB_MIRROR_URLS: 'https://ghfast.top/' + run: | + set -euo pipefail + source /usr/local/Ascend/cann-9.0.0/set_env.sh + if command -v apt-get >/dev/null 2>&1; then + bash -x dependencies.sh -y + else + bash -x scripts/ascend/dependencies_openeuler.sh -y + fi + bash scripts/ascend/dependencies_ascend_installation.sh + echo "PATH=/usr/local/go/bin:${PATH}" >> "$GITHUB_ENV" + echo "GOPROXY=https://goproxy.io|https://mirrors.aliyun.com/goproxy/|https://goproxy.cn|direct" >> "$GITHUB_ENV" + echo "GOSUMDB=sum.golang.google.cn" >> "$GITHUB_ENV" + /usr/local/go/bin/go env -w GOPROXY="https://goproxy.io|https://mirrors.aliyun.com/goproxy/|https://goproxy.cn|direct" + /usr/local/go/bin/go env -w GOSUMDB=sum.golang.google.cn + + - name: Configure project + shell: bash + run: | + source /usr/local/Ascend/cann-9.0.0/set_env.sh + rm -rf build + mkdir build + cd build + cmake_args=( + -DUSE_ASCEND_DIRECT=ON + -DUSE_CUDA=OFF + -DWITH_EP=OFF + -DUSE_HTTP=ON + -DUSE_ETCD=ON + -DSTORE_USE_ETCD=ON + -DBUILD_UNIT_TESTS=OFF + -DBUILD_EXAMPLES=ON + -DCMAKE_BUILD_TYPE=Release + -DPython3_EXECUTABLE="${PYTHON_BIN}" + ) + cmake .. "${cmake_args[@]}" + + - name: Build project + shell: bash + env: + GOPROXY: 'https://goproxy.io|https://mirrors.aliyun.com/goproxy/|https://goproxy.cn|direct' + GOSUMDB: sum.golang.google.cn + run: | + set -euo pipefail + source /usr/local/Ascend/cann-9.0.0/set_env.sh + export PATH="/usr/local/go/bin:${PATH}" + cd build + cmake --build . -j"$(nproc)" + cmake --install . + + - name: Generate Python version tag + id: generate_tag_release + shell: bash + run: | + echo "python_version_tag=$(echo ${{ matrix.python-version }} | tr -d '.')" >> $GITHUB_OUTPUT + + - name: Build Python wheel + shell: bash + run: | + source /usr/local/Ascend/cann-9.0.0/set_env.sh + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib + NPU_BUILD=1 PYTHON_VERSION=${{ matrix.python-version }} OUTPUT_DIR=dist-npu-py${{ steps.generate_tag_release.outputs.python_version_tag }} ./scripts/build_wheel.sh + + - name: Upload Python wheel artifact + uses: actions/upload-artifact@v4 + with: + name: mooncake-wheel-npu-py${{ steps.generate_tag_release.outputs.python_version_tag }} + path: mooncake-wheel/dist-npu-py${{ steps.generate_tag_release.outputs.python_version_tag }}/*.whl + + publish-release: + needs: build + runs-on: ubuntu-22.04 + environment: pypi + + permissions: + contents: write + id-token: write + + steps: + - name: Checkout source + uses: actions/checkout@v4 + + - name: Download all wheel artifacts + uses: actions/download-artifact@v4 + with: + path: mooncake-wheel/dist-all + pattern: mooncake-wheel-npu-py* + + - name: Prepare wheels for release + run: | + mkdir -p mooncake-wheel/dist-release + find mooncake-wheel/dist-all -name "*.whl" -exec cp {} mooncake-wheel/dist-release/ \; + echo "Collected wheels for release:" + ls -la mooncake-wheel/dist-release/ + + - name: Upload wheels to GitHub Release + uses: softprops/action-gh-release@v1 + with: + files: mooncake-wheel/dist-release/*.whl + + - name: Publish package to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + packages-dir: mooncake-wheel/dist-release/ diff --git a/scripts/ascend/dependencies_openeuler.sh b/scripts/ascend/dependencies_openeuler.sh new file mode 100644 index 00000000..0d6b83f9 --- /dev/null +++ b/scripts/ascend/dependencies_openeuler.sh @@ -0,0 +1,310 @@ +#!/bin/bash +# Copyright 2024-2026 KVCache.AI +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -euo pipefail + +GREEN="\033[0;32m" +BLUE="\033[0;34m" +YELLOW="\033[0;33m" +RED="\033[0;31m" +NC="\033[0m" + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)" +DEPS_BUILD_DIR="${REPO_ROOT}/.mooncake-openeuler-deps" +GOVER=1.25.9 +ASCEND_GITHUB_MIRROR_URLS="${ASCEND_GITHUB_MIRROR_URLS:-https://ghfast.top/}" + +print_section() { + echo -e "\n${BLUE}=== $1 ===${NC}" +} + +print_success() { + echo -e "${GREEN}✓ $1${NC}" +} + +print_error() { + echo -e "${RED}✗ ERROR: $1${NC}" + exit 1 +} + +print_warn() { + echo -e "${YELLOW}! $1${NC}" +} + +check_success() { + if [ $? -ne 0 ]; then + print_error "$1" + fi +} + +SKIP_CONFIRM=false +for arg in "$@"; do + case $arg in + -y|--yes) + SKIP_CONFIRM=true + ;; + -h|--help) + echo -e "${YELLOW}Mooncake openEuler Dependencies Installer${NC}" + echo "Usage: bash scripts/ascend/dependencies_openeuler.sh [-y]" + exit 0 + ;; + esac +done + +if [ "$(id -u)" -ne 0 ]; then + print_error "Require root permission, try: sudo bash scripts/ascend/dependencies_openeuler.sh -y" +fi + +echo -e "${YELLOW}Mooncake openEuler Dependencies Installer${NC}" +echo "Repository root: ${REPO_ROOT}" +echo "This script installs (same scope as root dependencies.sh):" +echo " - openEuler/RHEL system packages (dnf/yum)" +echo " - Git submodules (pybind11, yalantinglibs, ...)" +echo " - yalantinglibs (from extern/ submodule)" +echo " - Go ${GOVER} (for USE_ETCD / libetcd_wrapper.so)" +echo "Run scripts/ascend/dependencies_ascend_installation.sh afterward for Ascend extras." +echo + +if [ "$SKIP_CONFIRM" = false ]; then + read -p "Do you want to continue? [Y/n] " -n 1 -r + echo + if [[ ! $REPLY =~ ^[Yy]$ ]] && [[ ! $REPLY = "" ]]; then + echo -e "${YELLOW}Installation cancelled.${NC}" + exit 0 + fi +fi + +detect_pkg_manager() { + if command -v dnf >/dev/null 2>&1; then + echo dnf + elif command -v yum >/dev/null 2>&1; then + echo yum + else + print_error "Neither dnf nor yum found. This script is for openEuler/RHEL only." + fi +} + +git_with_github_mirror_fallback() { + local repo_dir="$1" + local repo_url="$2" + shift 2 + + if git clone "$repo_url" "$repo_dir" "$@"; then + return 0 + fi + + print_warn "Direct clone failed, retrying with mirror https://ghfast.top/" + rm -rf "$repo_dir" + git clone "https://ghfast.top/${repo_url}" "$repo_dir" "$@" +} + +clone_repo_if_not_exists() { + local repo_dir="$1" + local repo_url="$2" + shift 2 + + if [ -d "$repo_dir" ]; then + echo "Directory $repo_dir already exists, skipping clone." + return 0 + fi + git_with_github_mirror_fallback "$repo_dir" "$repo_url" "$@" +} + +install_rpm_packages() { + local pkg_mgr="$1" + shift + local packages=("$@") + local missing=() + + print_section "Installing system packages via ${pkg_mgr}" + ${pkg_mgr} makecache -y || ${pkg_mgr} makecache + check_success "Failed to refresh ${pkg_mgr} metadata" + + for pkg in "${packages[@]}"; do + if ! ${pkg_mgr} install -y "$pkg"; then + missing+=("$pkg") + fi + done + + if [ ${#missing[@]} -gt 0 ]; then + print_warn "Some packages were not installed (may be optional on this openEuler release):" + printf ' - %s\n' "${missing[@]}" + fi +} + +PKG_MGR="$(detect_pkg_manager)" + +# Core build dependencies aligned with root dependencies.sh (Debian names mapped to RHEL/openEuler). +CORE_PACKAGES=( + gcc gcc-c++ make cmake ninja-build git wget unzip + gflags-devel glog-devel libibverbs-devel numactl-devel + gtest gtest-devel boost-devel openssl-devel hiredis-devel + libcurl-devel jsoncpp-devel libunwind-devel python3-devel + zstd-devel xxhash-devel pkgconf pkgconf-pkg-config patchelf + mpich mpich-devel glibc glibc-common +) + +# Often provided by repos on openEuler; install best-effort. +OPTIONAL_PACKAGES=( + grpc-devel grpc-plugins protobuf-devel protobuf-compiler + liburing-devel jemalloc-devel +) + +install_rpm_packages "$PKG_MGR" "${CORE_PACKAGES[@]}" +install_rpm_packages "$PKG_MGR" "${OPTIONAL_PACKAGES[@]}" || true + +# openEuler images may ship MPICH; remove OpenMPI if present to avoid conflicts. +${PKG_MGR} remove -y openmpi openmpi-devel 2>/dev/null || true + +print_success "System package installation finished" + +export CPLUS_INCLUDE_PATH="$( + echo "${CPLUS_INCLUDE_PATH:-}" | tr ':' '\n' | grep -v "/usr/local/Ascend" | paste -sd: - +)" + +print_section "Initializing Git submodules" +cd "${REPO_ROOT}" +if [ ! -f "${REPO_ROOT}/.gitmodules" ]; then + print_error "No .gitmodules found under ${REPO_ROOT}" +fi + +submodule_updated=false +if git submodule sync --recursive && git submodule update --init --recursive; then + submodule_updated=true +elif [ -n "${ASCEND_GITHUB_MIRROR_URLS:-}" ]; then + while IFS= read -r raw; do + base="${raw#"${raw%%[![:space:]]*}"}" + base="${base%"${base##*[![:space:]]}"}" + [ -n "$base" ] || continue + [ "$base" = "https://github.com/" ] && continue + [ "$base" != "https://github.com/" ] && base="${base%/}/" + echo "Retrying submodule update with mirror ${base}" + if git -c url."${base}https://github.com/".insteadOf=https://github.com/ \ + submodule sync --recursive && \ + git -c url."${base}https://github.com/".insteadOf=https://github.com/ \ + submodule update --init --recursive; then + submodule_updated=true + break + fi + done < <(printf '%s\n' "$ASCEND_GITHUB_MIRROR_URLS" | tr ',;' '\n') +fi + +if [ "$submodule_updated" != true ]; then + print_error "git submodule update failed (direct GitHub and mirrors exhausted)" +fi +print_success "Git submodules initialized" + +print_section "Installing yalantinglibs from extern/ submodule" +if [ -d "${REPO_ROOT}/extern/yalantinglibs" ]; then + cd "${REPO_ROOT}/extern/yalantinglibs" + rm -rf build + mkdir -p build && cd build + cmake .. -DBUILD_EXAMPLES=OFF -DBUILD_BENCHMARK=OFF -DBUILD_UNIT_TESTS=OFF + check_success "Failed to configure yalantinglibs" + cmake --build . -j"$(nproc)" + check_success "Failed to build yalantinglibs" + cmake --install . + check_success "Failed to install yalantinglibs" + print_success "yalantinglibs installed from submodule" + cd "${REPO_ROOT}" +else + print_warn "extern/yalantinglibs missing, building from upstream clone" + mkdir -p "${DEPS_BUILD_DIR}" + cd "${DEPS_BUILD_DIR}" + clone_repo_if_not_exists "yalantinglibs" "https://github.com/alibaba/yalantinglibs.git" + cd yalantinglibs + git checkout 0.5.5 + rm -rf build && mkdir -p build && cd build + cmake .. -DBUILD_EXAMPLES=OFF -DBUILD_BENCHMARK=OFF -DBUILD_UNIT_TESTS=OFF + cmake --build . -j"$(nproc)" + cmake --install . + cd "${REPO_ROOT}" + print_success "yalantinglibs installed from git clone" +fi + +print_section "Verifying essential build tools" +for tool in getconf ldd patchelf cmake git; do + command -v "$tool" >/dev/null || print_error "${tool} not found after dependency installation" + print_success "${tool} found: $(command -v "$tool")" +done + +print_section "Installing Go ${GOVER}" +USED_CN_MIRROR=false + +install_go() { + local arch + arch="$(uname -m)" + case "$arch" in + aarch64) arch=arm64 ;; + x86_64) arch=amd64 ;; + *) print_error "Unsupported architecture: $arch" ;; + esac + + local tarball="go${GOVER}.linux-${arch}.tar.gz" + local urls=( + "https://go.dev/dl/${tarball}" + "https://golang.google.cn/dl/${tarball}" + "https://mirrors.aliyun.com/golang/${tarball}" + ) + + local url downloaded=false + for url in "${urls[@]}"; do + echo "Downloading Go ${GOVER} from ${url}..." + if wget -q --show-progress --timeout=30 --tries=2 -O "${tarball}" "${url}"; then + downloaded=true + [[ "$url" != "https://go.dev/dl/${tarball}" ]] && USED_CN_MIRROR=true + break + fi + rm -f "${tarball}" + done + + [ "$downloaded" = true ] || print_error "Failed to download Go ${GOVER}" + + rm -rf /usr/local/go + tar -C /usr/local -xzf "${tarball}" + rm -f "${tarball}" + export PATH="/usr/local/go/bin:${PATH}" + print_success "Go ${GOVER} installed to /usr/local/go" +} + +if command -v go >/dev/null 2>&1; then + current_go="$(go version | awk '{print $3}')" + if [[ "$current_go" == "go${GOVER}" ]]; then + print_success "Go ${GOVER} already installed (${current_go})" + else + print_warn "Found ${current_go}, installing Go ${GOVER}" + install_go + fi +else + install_go +fi + +export PATH="/usr/local/go/bin:${PATH}" +if [ "$USED_CN_MIRROR" = true ] && [ -z "${GOPROXY:-}" ]; then + export GOPROXY=https://goproxy.cn,https://goproxy.io,direct + print_warn "GOPROXY set to ${GOPROXY} (restricted network detected)" +fi + +go version +print_success "Go is available: $(command -v go)" + +print_section "Installation complete" +echo -e "${GREEN}openEuler dependencies are ready.${NC}" +echo "Next steps (example NPU release build):" +echo " source /usr/local/Ascend/cann-9.0.0/set_env.sh" +echo " mkdir -p build && cd build" +echo " cmake .. -DUSE_ASCEND_DIRECT=ON -DUSE_CUDA=OFF -DWITH_EP=OFF -DUSE_HTTP=ON -DUSE_ETCD=ON -DSTORE_USE_ETCD=ON" +echo " cmake --build . -j\$(nproc) && cmake --install ." \ No newline at end of file diff --git a/scripts/build_wheel.sh b/scripts/build_wheel.sh index 4dbfa341..d37f4359 100755 --- a/scripts/build_wheel.sh +++ b/scripts/build_wheel.sh @@ -141,7 +141,19 @@ echo "Building wheel package..." # Build the wheel package cd mooncake-wheel -# Handle package name modification for non-CUDA builds +BUILD_VARIANTS="NON_CUDA_BUILD CU13_BUILD NPU_BUILD" +BUILD_VARIANT_COUNT=0 +for build_variant in $BUILD_VARIANTS; do + if [ "${!build_variant}" = "1" ]; then + BUILD_VARIANT_COUNT=$((BUILD_VARIANT_COUNT + 1)) + fi +done +if [ "$BUILD_VARIANT_COUNT" -gt 1 ]; then + echo "Error: only one of $BUILD_VARIANTS can be set" + exit 1 +fi + +# Handle package name modification for release build variants if [ "$NON_CUDA_BUILD" = "1" ]; then echo "Modifying package name for non-CUDA build" # Backup original pyproject.toml @@ -151,12 +163,7 @@ if [ "$NON_CUDA_BUILD" = "1" ]; then sed -i 's/description = "Python binding of a Mooncake library using pybind11"/description = "Python binding of a Mooncake library using pybind11 (Non-CUDA version)"/' pyproject.toml sed -i 's/keywords = \["mooncake", "data transfer", "kv cache", "llm inference"\]/keywords = ["mooncake", "data transfer", "kv cache", "llm inference", "non-cuda"]/' pyproject.toml echo "Package name modified to: mooncake-transfer-engine-non-cuda" -else - echo "Using standard package name: mooncake-transfer-engine" -fi - -# Handle package name modification for CU13 builds -if [ "$CU13_BUILD" = "1" ]; then +elif [ "$CU13_BUILD" = "1" ]; then echo "Modifying package name for CU13 build" # Backup original pyproject.toml cp pyproject.toml pyproject.toml.backup @@ -165,6 +172,15 @@ if [ "$CU13_BUILD" = "1" ]; then sed -i 's/description = "Python binding of a Mooncake library using pybind11"/description = "Python binding of a Mooncake library using pybind11 (CUDA 13 version)"/' pyproject.toml sed -i 's/keywords = \["mooncake", "data transfer", "kv cache", "llm inference"\]/keywords = ["mooncake", "data transfer", "kv cache", "llm inference", "cuda13"]/' pyproject.toml echo "Package name modified to: mooncake-transfer-engine-cuda13" +elif [ "$NPU_BUILD" = "1" ]; then + echo "Modifying package name for Ascend NPU build" + # Backup original pyproject.toml + cp pyproject.toml pyproject.toml.backup + # Replace package name and description + sed -i 's/name = "mooncake-transfer-engine"/name = "mooncake-transfer-engine-npu"/' pyproject.toml + sed -i 's/description = "Python binding of a Mooncake library using pybind11"/description = "Python binding of a Mooncake library using pybind11 (Ascend NPU version)"/' pyproject.toml + sed -i 's/keywords = \["mooncake", "data transfer", "kv cache", "llm inference"\]/keywords = ["mooncake", "data transfer", "kv cache", "llm inference", "ascend", "npu"]/' pyproject.toml + echo "Package name modified to: mooncake-transfer-engine-npu" else echo "Using standard package name: mooncake-transfer-engine" fi @@ -174,7 +190,14 @@ rm -rf ${OUTPUT_DIR}/ mkdir -p ${OUTPUT_DIR} echo "Installing required build packages" -if command -v pip &>/dev/null; then +if [ "$NPU_BUILD" = "1" ]; then + PYTHON_CMD="python${PYTHON_VERSION}" + if ! command -v "$PYTHON_CMD" &>/dev/null; then + echo "Error: $PYTHON_CMD not found for NPU wheel build" + exit 1 + fi + "$PYTHON_CMD" -m pip install --upgrade pip build setuptools wheel auditwheel +elif command -v pip &>/dev/null; then python${PYTHON_VERSION} -m pip install --upgrade pip build setuptools wheel auditwheel elif command -v uv &>/dev/null; then uv pip install --upgrade pip @@ -243,8 +266,14 @@ echo "Detected glibc version: $GLIBC_VERSION" echo "Using platform tag: $PLATFORM_TAG" echo "Repairing wheel with auditwheel for platform: $PLATFORM_TAG" -python${PYTHON_VERSION} -m build --wheel --outdir ${OUTPUT_DIR} -auditwheel repair ${OUTPUT_DIR}/*.whl \ +if [ "$NPU_BUILD" = "1" ]; then + python${PYTHON_VERSION} -m build --wheel --no-isolation --outdir ${OUTPUT_DIR} + AUDITWHEEL_CMD="python${PYTHON_VERSION} -m auditwheel" +else + python${PYTHON_VERSION} -m build --wheel --outdir ${OUTPUT_DIR} + AUDITWHEEL_CMD="auditwheel" +fi +${AUDITWHEEL_CMD} repair ${OUTPUT_DIR}/*.whl \ --exclude libcurl.so* \ --exclude libibverbs.so* \ --exclude libmlx5.so* \ @@ -366,4 +395,6 @@ mv ${REPAIRED_DIR}/*.whl ${OUTPUT_DIR}/ cd .. -echo "Wheel package built and repaired successfully!" +[[ -f mooncake-wheel/pyproject.toml.backup ]] && mv mooncake-wheel/pyproject.toml.backup mooncake-wheel/pyproject.toml + +echo "Wheel package built and repaired successfully!" \ No newline at end of file -- 2.34.1 From 8906bdef8fbde185aa28f15906f41dfca945aabf Mon Sep 17 00:00:00 2001 From: Shangming Cai Date: Fri, 22 May 2026 17:56:40 +0800 Subject: [PATCH 114/382] [chore] Set default for WITH_NVIDIA_PEERMEM to true (#2192) * [chore] Set default for WITH_NVIDIA_PEERMEM to true --- docs/source/getting_started/build.md | 2 +- docs/source/zh_archive/build.md | 2 +- mooncake-common/src/environ.cpp | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index bc529d59..d83c2e6a 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -105,7 +105,7 @@ pip install mooncake-transfer-engine-non-cuda export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda/lib64 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64 ``` - > **Note:** By default, Mooncake uses the DMA-BUF path for GPU-Direct RDMA, which does **not** require the `nvidia-peermem` kernel module. If you prefer the legacy `ibv_reg_mr` path (which requires `nvidia-peermem`), set the runtime environment variable `WITH_NVIDIA_PEERMEM=1` before starting Mooncake. See Section 3.7 of https://docs.nvidia.com/cuda/gpudirect-rdma/ for instructions on installing `nvidia-peermem`. + > **Note:** Mooncake could use the DMA-BUF path for GPU-Direct RDMA, which does **not** require the `nvidia-peermem` kernel module. If you prefer the DMA-BUF path, please set the runtime environment variable `WITH_NVIDIA_PEERMEM=0` before starting Mooncake. If you prefer the legacy `ibv_reg_mr` path (which requires `nvidia-peermem`), set the runtime environment variable `WITH_NVIDIA_PEERMEM=1`. See Section 3.7 of https://docs.nvidia.com/cuda/gpudirect-rdma/ for instructions on installing `nvidia-peermem`. 3. If you want to compile the Moore Mthreads GPUDirect support module, first follow the instructions in https://docs.mthreads.com/musa-sdk/musa-sdk-doc-online/install_guide to install MUSA. After that: 1) Install `mthreads-peermem` for enabling GPU-Direct RDMA diff --git a/docs/source/zh_archive/build.md b/docs/source/zh_archive/build.md index 7a49643f..2de375e4 100644 --- a/docs/source/zh_archive/build.md +++ b/docs/source/zh_archive/build.md @@ -100,7 +100,7 @@ export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda/lib64 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64 ``` - > **注意:** 默认情况下,Mooncake 使用 DMA-BUF 路径进行 GPU-Direct RDMA,**无需** `nvidia-peermem` 内核模块。如需使用依赖 `nvidia-peermem` 的传统 `ibv_reg_mr` 路径,请在启动 Mooncake 前设置运行时环境变量 `WITH_NVIDIA_PEERMEM=1`。安装 `nvidia-peermem` 的说明见 https://docs.nvidia.com/cuda/gpudirect-rdma/ 第 3.7 节。 + > **注意:** Mooncake 可以使用 DMA-BUF 路径进行 GPU-Direct RDMA,**无需** `nvidia-peermem` 内核模块。如需使用 DMA-BUF 路径,请在启动 Mooncake 前设置运行时环境变量 `WITH_NVIDIA_PEERMEM=0`。如果偏好依赖 `nvidia-peermem` 的传统 `ibv_reg_mr` 路径,请设置运行时环境变量 `WITH_NVIDIA_PEERMEM=1`。安装 `nvidia-peermem` 的说明见 https://docs.nvidia.com/cuda/gpudirect-rdma/ 第 3.7 节。 3. 如果你要编译Moore Threads GPUDirect RDMA 支持模块,首先需按照 https://docs.mthreads.com/musa-sdk/musa-sdk-doc-online/install_guide 的指引安装 MUSA SDK。之后: 1) 安装 `mthreads-peermem` 以启用 GPU-Direct RDMA diff --git a/mooncake-common/src/environ.cpp b/mooncake-common/src/environ.cpp index 25b3300d..cb2bfa2d 100644 --- a/mooncake-common/src/environ.cpp +++ b/mooncake-common/src/environ.cpp @@ -81,7 +81,7 @@ Environ::Environ() { force_mnnvl_ = GetBool("MC_FORCE_MNNVL", false); intra_nvlink_ = GetBool("MC_INTRA_NVLINK", false); path_roundrobin_ = GetBool("MC_PATH_ROUNDROBIN", false); - with_nvidia_peermem_ = GetBool("WITH_NVIDIA_PEERMEM", false); + with_nvidia_peermem_ = GetBool("WITH_NVIDIA_PEERMEM", true); } } // namespace mooncake -- 2.34.1 From d1312dce24a36d09f8ff75a1e1da9e7efd95fe48 Mon Sep 17 00:00:00 2001 From: Cruz Zhao Date: Fri, 22 May 2026 17:58:25 +0800 Subject: [PATCH 115/382] [Store] Add structured object store helper (#2140) * [Python] Add structured object store helper --------- Co-authored-by: Claude Opus 4.6 --- .../python-api-reference/mooncake-store.md | 105 ++ .../mooncake/structured_object_store.py | 1309 +++++++++++++++++ mooncake-wheel/tests/conftest.py | 19 + .../tests/test_structured_object_store.py | 769 ++++++++++ 4 files changed, 2202 insertions(+) create mode 100644 mooncake-wheel/mooncake/structured_object_store.py create mode 100644 mooncake-wheel/tests/conftest.py create mode 100644 mooncake-wheel/tests/test_structured_object_store.py diff --git a/docs/source/python-api-reference/mooncake-store.md b/docs/source/python-api-reference/mooncake-store.md index 9ad362e8..cf190b29 100644 --- a/docs/source/python-api-reference/mooncake-store.md +++ b/docs/source/python-api-reference/mooncake-store.md @@ -130,6 +130,111 @@ store.close() +## Structured Object Store Helper + +`mooncake.structured_object_store` provides a higher-level helper for one logical object that contains multiple named members. +It is designed for cases such as rollout / batch transfer where callers want to keep their own object semantics locally while using Mooncake for fast payload movement. + +The helper separates two concepts: + +- **structured object path**: named members with metadata-aware materialization; +- **generic bundle path**: manifest + named payloads when the caller only needs raw grouped objects. + +### Main types + +```python +from mooncake.structured_object_store import ( + MooncakeBundleTransfer, + StructuredMemberSlice, + StructuredObjectPayload, +) +``` + +- `MooncakeBundleTransfer`: public helper facade built on a `MooncakeDistributedStore`. +- `StructuredObjectPayload`: structured object to write. Members are passed in `buffers`, and optional object metadata is passed in `metadata`. +- `StructuredMemberSlice`: slice selection for one structured member during reads. + +### Structured object write and full read + +Use `put_structured_object()` to write one structured object. The default read path is `read_spec(ref)`, and full-object materialization is just the default case of the partial-read API. + +```python +import numpy as np +from mooncake.store import MooncakeDistributedStore +from mooncake.structured_object_store import MooncakeBundleTransfer, StructuredObjectPayload + +store = MooncakeDistributedStore() +transfer = MooncakeBundleTransfer(store, key_prefix="demo/structured") + +payload = StructuredObjectPayload( + metadata={"step": 7, "layout": "rollout"}, + buffers={ + "tokens": np.array(range(24), dtype=np.int32).reshape(6, 4), + "mask": np.ones((6, 4), dtype=np.int8), + "prompt_ids": b"sample-ids", + }, +) + +ref = transfer.put_structured_object(payload) +result = transfer.materialize(transfer.read_spec(ref)) + +tokens = result.objects["tokens"] +prompt_ids = result.objects["prompt_ids"] +metadata = result.metadata +``` + +### Partial reads + +Read narrowing happens on top of `read_spec(ref)`: + +- `select_members([...])` keeps only selected members; +- `slice_member(name, axis=0, start=..., end=...)` slices one ndarray member; +- `materialize(spec)` returns newly materialized objects. + +```python +spec = ( + transfer.read_spec(ref) + .select_members(["tokens"]) + .slice_member("tokens", axis=0, start=2, end=5) +) +result = transfer.materialize(spec) + +selected_tokens = result.objects["tokens"] +``` + +Current scope: + +- byte members support full-member reads; +- ndarray members support full reads and sliced reads; +- full read is the default `read_spec(ref)` case. + +### Reusing caller-owned destinations + +Use `materialize_into()` when the caller already owns the destination ndarray buffers and wants Mooncake to fill them directly. + +```python +destination = np.empty((3, 4), dtype=np.int32) +spec = ( + transfer.read_spec(ref) + .select_members(["tokens"]) + .slice_member("tokens", axis=0, start=2, end=5) +) +result = transfer.materialize_into(spec, {"tokens": destination}) + +assert result.objects["tokens"] is destination +``` + +`materialize_into()` is only for members whose destination layout is already known to the caller. For byte members or default object reconstruction, use `materialize()`. + +### Generic bundle fallback + +If the caller does not need structured member semantics, the same helper also supports raw named bundles: + +- `put_bundle(...)` +- `remove_bundle(...)` + +Use the bundle path when the object is just a manifest plus named payloads, and use the structured object path when callers want member selection, slicing, and ndarray-aware materialization. + ## Zero-Copy API (Advanced Performance) For maximum performance, especially with RDMA networks, use the zero-copy API. This allows direct memory access without intermediate copies. diff --git a/mooncake-wheel/mooncake/structured_object_store.py b/mooncake-wheel/mooncake/structured_object_store.py new file mode 100644 index 00000000..c5329e4e --- /dev/null +++ b/mooncake-wheel/mooncake/structured_object_store.py @@ -0,0 +1,1309 @@ +from __future__ import annotations + +import ctypes +import json +import uuid +from concurrent.futures import Future, ThreadPoolExecutor, as_completed +from contextlib import contextmanager +from dataclasses import dataclass +from typing import Any, Iterator, Literal, Mapping, Optional, Protocol, Sequence + +import numpy as np + +DEFAULT_BUNDLE_CHUNK_BYTES = 512 * 1024**2 +AUTO_PARALLEL_MIN_BYTES = 4 * 1024**3 +AUTO_PARALLEL_MIN_CHUNKS = 8 +MISSING_OBJECT_ERROR = ( + -704 +) # Mooncake remove returns -704 for an already-missing object. +STRUCTURED_FIELD_SPECS_KEY = "__mooncake_structured_fields__" + + +class BundleStore(Protocol): + def put(self, key: str, value: Any) -> int: ... + + def get(self, key: str) -> bytes: ... + + def remove(self, key: str, force: bool = False) -> int: ... + + +@dataclass(frozen=True) +class BundleTransferPolicy: + """Controls generic bundle transfer parallelism.""" + + max_inflight_put: int = 1 + put_mode: Literal["auto", "batch", "parallel"] = "auto" + + +@dataclass +class RemoteBundleRef: + """Reference to a generic named-buffer bundle stored in Mooncake.""" + + manifest_key: str + manifest: dict[str, Any] + + +@dataclass(frozen=True, init=False) +class StructuredObjectPayload: + """Structured object encoded as metadata plus named buffers.""" + + metadata: Mapping[str, Any] + buffers: Mapping[str, Any] + + def __init__( + self, + metadata: Optional[Mapping[str, Any]] = None, + buffers: Optional[Mapping[str, Any]] = None, + ) -> None: + if buffers is None: + raise TypeError("StructuredObjectPayload requires buffers") + object.__setattr__(self, "metadata", {} if metadata is None else metadata) + object.__setattr__(self, "buffers", buffers) + + +@dataclass +class StructuredObjectResult: + """Materialized structured object data fetched from Mooncake.""" + + metadata: dict[str, Any] + objects: dict[str, Any] + + +@dataclass(frozen=True) +class StructuredMemberSlice: + """Slice description for one structured member.""" + + axis: int + start: int + end: int + step: int = 1 + + +@dataclass(frozen=True) +class StructuredObjectReadSpec: + """Lazy read plan for structured object materialization.""" + + ref: RemoteBundleRef | Mapping[str, Any] + member_names: tuple[str, ...] | None = None + member_slices: tuple[tuple[str, StructuredMemberSlice], ...] = () + + def select_members( + self, names: Optional[Sequence[str]] + ) -> "StructuredObjectReadSpec": + """Return a new spec that materializes only the selected members.""" + return StructuredObjectReadSpec( + ref=self.ref, + member_names=None if names is None else tuple(names), + member_slices=self.member_slices, + ) + + def slice_member( + self, + name: str, + axis: int, + start: int, + end: int, + step: int = 1, + ) -> "StructuredObjectReadSpec": + """Return a new spec with a slice applied to one member.""" + slice_spec = StructuredMemberSlice(axis=axis, start=start, end=end, step=step) + member_slices = dict(self.member_slices) + member_slices[name] = slice_spec + return StructuredObjectReadSpec( + ref=self.ref, + member_names=self.member_names, + member_slices=tuple(member_slices.items()), + ) + + def member_slice(self, name: str) -> StructuredMemberSlice | None: + """Return the configured slice for a member, if any.""" + return dict(self.member_slices).get(name) + + +@dataclass(frozen=True) +class _NdarrayReadPlan: + dtype: np.dtype[Any] + full_shape: tuple[int, ...] + output_shape: tuple[int, ...] + byte_offset: int + byte_length: int + step: int + cover_row_count: int + + +class MooncakeBundleTransfer: + """Transfer structured objects through Mooncake, with a low-level bundle fallback.""" + + def __init__( + self, + store: BundleStore, + key_prefix: str = "bundle", + default_chunk_bytes: int = DEFAULT_BUNDLE_CHUNK_BYTES, + ) -> None: + """Initialize a bundle transfer helper with a configurable default chunk size.""" + self.store = store + self.key_prefix = _normalize_key_prefix(key_prefix) + self.default_chunk_bytes = _validate_chunk_bytes(default_chunk_bytes) + self._transport = _MooncakePayloadTransport(store) + self._bundle_store = _BundleManifestStore( + store=store, + transport=self._transport, + key_prefix=self.key_prefix, + default_chunk_bytes=self.default_chunk_bytes, + ) + self._structured_store = _StructuredObjectLayer(self._bundle_store) + + def put_bundle( + self, + meta: bytes | bytearray | memoryview, + buffers: Mapping[str, Any], + partition: str = "default", + chunk_bytes: Optional[int] = None, + policy: Optional[BundleTransferPolicy] = None, + max_inflight_put: Optional[int] = None, + pre_registered_buffers: Optional[Mapping[str, bool]] = None, + ) -> RemoteBundleRef: + """Store raw metadata bytes plus named buffers as a low-level bundle.""" + return self._bundle_store.put_bundle( + meta=meta, + buffers=buffers, + partition=partition, + chunk_bytes=chunk_bytes, + policy=policy, + max_inflight_put=max_inflight_put, + pre_registered_buffers=pre_registered_buffers, + ) + + def remove_bundle(self, ref: RemoteBundleRef | Mapping[str, Any]) -> None: + """Remove all Mooncake objects that belong to a stored bundle.""" + self._bundle_store.remove_bundle(ref) + + def put_structured_object( + self, + payload: StructuredObjectPayload, + partition: str = "default", + chunk_bytes: Optional[int] = None, + policy: Optional[BundleTransferPolicy] = None, + max_inflight_put: Optional[int] = None, + ) -> RemoteBundleRef: + """Store a structured object described by JSON metadata plus named members.""" + return self._structured_store.put_structured_object( + payload=payload, + partition=partition, + chunk_bytes=chunk_bytes, + policy=policy, + max_inflight_put=max_inflight_put, + ) + + def read_spec( + self, ref: RemoteBundleRef | Mapping[str, Any] + ) -> StructuredObjectReadSpec: + """Create a lazy read spec for a structured object reference.""" + return self._structured_store.read_spec(ref) + + def materialize(self, spec: StructuredObjectReadSpec) -> StructuredObjectResult: + """Materialize a structured object read spec.""" + return self._structured_store.materialize(spec) + + def materialize_into( + self, + spec: StructuredObjectReadSpec, + destinations: Optional[Mapping[str, Any]], + ) -> StructuredObjectResult: + """Materialize a structured object read spec into caller-provided destinations when possible.""" + return self._structured_store.materialize_into(spec, destinations) + + +class _StructuredObjectLayer: + def __init__(self, bundle_store: "_BundleManifestStore") -> None: + self._bundle_store = bundle_store + + def put_structured_object( + self, + payload: StructuredObjectPayload, + partition: str, + chunk_bytes: Optional[int], + policy: Optional[BundleTransferPolicy], + max_inflight_put: Optional[int], + ) -> RemoteBundleRef: + metadata, buffers = _encode_structured_fields(payload.metadata, payload.buffers) + return self._bundle_store.put_bundle( + meta=_encode_structured_metadata(metadata), + buffers=buffers, + partition=partition, + chunk_bytes=chunk_bytes, + policy=policy, + max_inflight_put=max_inflight_put, + ) + + def read_spec( + self, ref: RemoteBundleRef | Mapping[str, Any] + ) -> StructuredObjectReadSpec: + return StructuredObjectReadSpec(ref=ref) + + def materialize(self, spec: StructuredObjectReadSpec) -> StructuredObjectResult: + return self.materialize_into(spec, destinations=None) + + def materialize_into( + self, + spec: StructuredObjectReadSpec, + destinations: Optional[Mapping[str, Any]], + ) -> StructuredObjectResult: + metadata, buffers, selected = self._resolve_structured_read(spec) + field_specs = _structured_field_specs(metadata) + destination_map = destinations or {} + member_slices = dict(spec.member_slices) + objects = { + name: self._read_structured_member( + name=name, + payload_spec=buffers[name], + field_spec=field_specs.get(name, {"encoding": "bytes"}), + member_slice=member_slices.get(name), + destination=destination_map.get(name), + ) + for name in selected + } + return StructuredObjectResult(metadata=dict(metadata), objects=objects) + + def _resolve_structured_read( + self, + spec: StructuredObjectReadSpec, + ) -> tuple[dict[str, Any], Mapping[str, Any], list[str]]: + if spec.member_names is not None and not spec.member_names: + raise ValueError("structured read spec selected no members") + manifest = self._bundle_store.resolve_manifest(spec.ref) + metadata = _decode_structured_metadata( + self._bundle_store.read_payload(manifest["meta"]) + ) + buffers = manifest["buffers"] + selected = ( + list(buffers) if spec.member_names is None else list(spec.member_names) + ) + missing = [name for name in selected if name not in buffers] + if missing: + raise KeyError(f"unknown bundle buffers: {missing}") + return metadata, buffers, selected + + def _read_structured_member( + self, + name: str, + payload_spec: Mapping[str, Any], + field_spec: Mapping[str, Any], + member_slice: StructuredMemberSlice | None, + destination: Any, + ) -> Any: + encoding = field_spec.get("encoding", "bytes") + if encoding == "bytes": + return self._read_bytes_member( + name, payload_spec, member_slice, destination + ) + if encoding != "ndarray": + raise ValueError(f"unsupported structured field encoding: {encoding}") + return self._read_ndarray_member( + name, payload_spec, field_spec, member_slice, destination + ) + + def _read_bytes_member( + self, + name: str, + payload_spec: Mapping[str, Any], + member_slice: StructuredMemberSlice | None, + destination: Any, + ) -> bytes: + if member_slice is not None: + raise ValueError(f"structured bytes member {name} does not support slicing") + if destination is not None: + raise ValueError( + f"structured bytes member {name} does not support materialize_into" + ) + return self._bundle_store.read_payload(payload_spec) + + def _read_ndarray_member( + self, + name: str, + payload_spec: Mapping[str, Any], + field_spec: Mapping[str, Any], + member_slice: StructuredMemberSlice | None, + destination: Any, + ) -> np.ndarray: + dtype = field_spec.get("dtype") + shape = field_spec.get("shape") + if not isinstance(dtype, str) or not isinstance(shape, list): + raise ValueError( + f"structured ndarray field {name} is missing dtype or shape" + ) + read_plan = _resolve_ndarray_read_plan( + tuple(int(dim) for dim in shape), np.dtype(dtype), member_slice + ) + target = _resolve_ndarray_destination( + name, destination, read_plan.dtype, read_plan.output_shape + ) + destination_view = target.view(np.uint8).reshape(-1) + if read_plan.byte_length == 0: + return target + if read_plan.step == 1: + self._bundle_store.read_payload_range_into_destination( + payload_spec, + destination_view, + read_plan.byte_offset, + ) + return target + temp_shape = (read_plan.cover_row_count, *read_plan.full_shape[1:]) + temp = np.empty(temp_shape, dtype=read_plan.dtype) + temp_view = temp.view(np.uint8).reshape(-1) + self._bundle_store.read_payload_range_into_destination( + payload_spec, temp_view, read_plan.byte_offset + ) + target[...] = temp[:: read_plan.step] + return target + + +class _BundleManifestStore: + def __init__( + self, + store: BundleStore, + transport: "_MooncakePayloadTransport", + key_prefix: str, + default_chunk_bytes: int, + ) -> None: + self._store = store + self._transport = transport + self._key_prefix = key_prefix + self._default_chunk_bytes = default_chunk_bytes + + def put_bundle( + self, + meta: bytes | bytearray | memoryview, + buffers: Mapping[str, Any], + partition: str, + chunk_bytes: Optional[int], + policy: Optional[BundleTransferPolicy], + max_inflight_put: Optional[int], + pre_registered_buffers: Optional[Mapping[str, bool]] = None, + ) -> RemoteBundleRef: + _validate_key_segment(partition, "partition") + meta_view = _bytes_view(meta, "meta") + target_chunk_bytes = _validate_chunk_bytes( + self._default_chunk_bytes if chunk_bytes is None else chunk_bytes + ) + transfer_policy = self._policy(policy, max_inflight_put=max_inflight_put) + object_id = f"{partition}/{uuid.uuid4().hex}" + base_key = f"{self._key_prefix}/{object_id}" + manifest_key = f"{base_key}/manifest" + written_keys: list[str] = [] + buffer_specs: dict[str, Any] = {} + pre_registered_map = dict(pre_registered_buffers or {}) + try: + meta_spec, meta_keys = self._put_payload( + f"{base_key}/meta", + meta_view, + target_chunk_bytes, + transfer_policy, + pre_registered=False, + ) + written_keys.extend(meta_keys) + for name, value in buffers.items(): + _validate_key_segment(name, "buffer name") + payload_view = _bytes_view(value, name) + payload_spec, payload_keys = self._put_payload( + f"{base_key}/buffer/{name}", + payload_view, + target_chunk_bytes, + transfer_policy, + pre_registered=bool(pre_registered_map.get(name, False)), + ) + buffer_specs[name] = payload_spec + written_keys.extend(payload_keys) + manifest = { + "version": 1, + "layout": "bundle", + "object_id": object_id, + "meta": meta_spec, + "buffers": buffer_specs, + } + manifest_blob = _encode_manifest(manifest) + _check_status( + self._store.put(manifest_key, manifest_blob), "put", manifest_key + ) + written_keys.append(manifest_key) + except Exception: + _cleanup_keys(self._store, written_keys, strict=False) + raise + return RemoteBundleRef(manifest_key=manifest_key, manifest=manifest) + + def remove_bundle(self, ref: RemoteBundleRef | Mapping[str, Any]) -> None: + manifest = self.resolve_manifest(ref) + keys = self._payload_keys(manifest) + keys.append(self._manifest_key(ref, manifest)) + _cleanup_keys(self._store, keys, strict=True) + + def resolve_manifest( + self, ref: RemoteBundleRef | Mapping[str, Any] + ) -> dict[str, Any]: + if isinstance(ref, RemoteBundleRef): + manifest = ref.manifest + else: + manifest = ref.get("manifest") + if manifest is None: + manifest_key = ref.get("manifest_key") + if not isinstance(manifest_key, str): + raise ValueError("bundle ref must include manifest_key") + manifest = _decode_manifest(self._store.get(manifest_key)) + self._validate_manifest(manifest) + return manifest + + def read_payload(self, payload_spec: Mapping[str, Any]) -> bytes: + return self._transport.read_payload(payload_spec) + + def read_payload_range_into_destination( + self, + payload_spec: Mapping[str, Any], + destination: np.ndarray, + byte_offset: int, + destination_pre_registered: bool = False, + ) -> None: + self._transport.read_payload_range_into_destination( + payload_spec, + destination, + byte_offset, + destination_pre_registered=destination_pre_registered, + ) + + def _put_payload( + self, + key: str, + value: memoryview, + chunk_bytes: int, + transfer_policy: BundleTransferPolicy, + pre_registered: bool, + ) -> tuple[dict[str, Any], list[str]]: + chunks = _split_view(value, chunk_bytes) + chunk_keys = [ + key if len(chunks) == 1 else f"{key}/chunk/{index}" + for index in range(len(chunks)) + ] + written_keys = self._transport.put_payload_chunks( + chunk_keys, + chunks, + transfer_policy, + pre_registered=pre_registered, + ) + payload_spec = { + "key": key, + "bytes": len(value), + "chunks": [ + {"key": chunk_key, "bytes": len(chunk)} + for chunk_key, chunk in zip(chunk_keys, chunks) + ], + } + return payload_spec, written_keys + + def _policy( + self, + policy: Optional[BundleTransferPolicy], + max_inflight_put: Optional[int] = None, + ) -> BundleTransferPolicy: + result = policy or BundleTransferPolicy() + if max_inflight_put is not None: + result = BundleTransferPolicy( + max_inflight_put=max_inflight_put, put_mode=result.put_mode + ) + if result.max_inflight_put < 1: + raise ValueError("max_inflight_put must be positive") + if result.put_mode not in {"auto", "batch", "parallel"}: + raise ValueError(f"unsupported put_mode: {result.put_mode}") + return result + + def _validate_manifest(self, manifest: Mapping[str, Any]) -> None: + if manifest.get("version") != 1 or manifest.get("layout") != "bundle": + raise ValueError("invalid bundle manifest") + object_id = manifest.get("object_id") + if not isinstance(object_id, str): + raise ValueError("bundle manifest object_id must be a string") + base_key = f"{self._key_prefix}/{object_id}" + self._validate_payload_spec(manifest.get("meta"), base_key) + buffers = manifest.get("buffers") + if not isinstance(buffers, dict): + raise ValueError("bundle manifest buffers must be a dict") + for name, payload_spec in buffers.items(): + _validate_key_segment(name, "buffer name") + self._validate_payload_spec(payload_spec, base_key) + + def _validate_payload_spec(self, payload_spec: Any, base_key: str) -> None: + if not isinstance(payload_spec, dict): + raise ValueError("bundle payload spec must be a dict") + payload_key = payload_spec.get("key") + if not isinstance(payload_key, str) or not payload_key.startswith( + f"{base_key}/" + ): + raise ValueError("bundle payload key is outside the bundle namespace") + expected_bytes = int(payload_spec.get("bytes", -1)) + if expected_bytes < 0: + raise ValueError("bundle payload bytes must be non-negative") + chunks = payload_spec.get("chunks") + if not isinstance(chunks, list) or (expected_bytes and not chunks): + raise ValueError("bundle payload chunks are invalid") + seen_keys = set() + total_bytes = 0 + for chunk in chunks: + if not isinstance(chunk, dict): + raise ValueError("bundle chunk must be a dict") + key = chunk.get("key") + chunk_bytes = int(chunk.get("bytes", -1)) + if not isinstance(key, str) or ( + key != payload_key and not key.startswith(f"{payload_key}/") + ): + raise ValueError("bundle chunk key is outside the payload namespace") + if key in seen_keys: + raise ValueError("bundle chunk keys must be unique") + if chunk_bytes < 0: + raise ValueError("bundle chunk bytes must be non-negative") + seen_keys.add(key) + total_bytes += chunk_bytes + if total_bytes != expected_bytes: + raise ValueError( + f"bundle payload chunks total {total_bytes} bytes, expected {expected_bytes}" + ) + + def _manifest_key( + self, ref: RemoteBundleRef | Mapping[str, Any], manifest: Mapping[str, Any] + ) -> str: + expected = f"{self._key_prefix}/{manifest['object_id']}/manifest" + manifest_key = ( + ref.manifest_key + if isinstance(ref, RemoteBundleRef) + else ref.get("manifest_key") + ) + if manifest_key is None: + return expected + if manifest_key != expected: + raise ValueError("bundle manifest_key does not match manifest object_id") + return manifest_key + + def _payload_keys(self, manifest: Mapping[str, Any]) -> list[str]: + return [ + chunk["key"] + for payload_spec in [manifest["meta"], *manifest["buffers"].values()] + for chunk in payload_spec["chunks"] + ] + + +class _MooncakePayloadTransport: + """Move payload bytes through Mooncake, preferring fast-path APIs and falling back to generic store calls.""" + + def __init__(self, store: BundleStore) -> None: + self._store = store + self._batch_put_from = getattr(store, "batch_put_from", None) + self._batch_get_into = getattr(store, "batch_get_into", None) + self._get_into = getattr(store, "get_into", None) + self._get_into_ranges = getattr(store, "get_into_ranges", None) + self._register_buffer = getattr(store, "register_buffer", None) + self._unregister_buffer = getattr(store, "unregister_buffer", None) + + def put_payload_chunks( + self, + chunk_keys: Sequence[str], + chunks: Sequence[memoryview], + transfer_policy: BundleTransferPolicy, + pre_registered: bool, + ) -> list[str]: + if not self._has_batch_put_support(): + return self._put_chunks_direct(chunk_keys, chunks) + put_mode = self._resolve_put_mode(chunks, transfer_policy) + if put_mode == "batch": + self.batch_put_chunks_from( + chunk_keys, chunks, pre_registered=pre_registered + ) + return list(chunk_keys) + return self._put_chunks_parallel( + list(chunk_keys), + list(chunks), + transfer_policy.max_inflight_put, + pre_registered=pre_registered, + ) + + def read_payload(self, payload_spec: Mapping[str, Any]) -> bytes: + expected_bytes = int(payload_spec["bytes"]) + if expected_bytes == 0: + return b"" + data = bytearray(expected_bytes) + self.read_payload_into(payload_spec, data) + return bytes(data) + + def read_payload_into( + self, payload_spec: Mapping[str, Any], destination: bytearray | np.ndarray + ) -> None: + chunks = payload_spec["chunks"] + offsets = _chunk_offsets(chunks) + if self._read_chunks_with_batch_get_into(chunks, offsets, destination): + return + for offset, chunk in zip(offsets, chunks): + self._read_chunk_with_get(chunk, destination, offset) + + def read_payload_range_into_destination( + self, + payload_spec: Mapping[str, Any], + destination: np.ndarray, + byte_offset: int, + destination_pre_registered: bool, + ) -> None: + chunks = payload_spec["chunks"] + byte_length = _buffer_nbytes(destination) + if not self._read_payload_range_into_registered_destination( + chunks, + destination, + byte_offset, + byte_length, + destination_pre_registered=destination_pre_registered, + ): + self._copy_payload_range_into_destination( + chunks, destination, byte_offset, byte_length + ) + + def batch_put_chunks_from( + self, + chunk_keys: Sequence[str], + chunks: Sequence[memoryview], + pre_registered: bool, + ) -> None: + batch_put_from = self._batch_put_from + if not callable(batch_put_from): + raise RuntimeError("batch_put_from is unavailable") + if not chunk_keys: + return + prepared_chunks = [_prepare_chunk_source_buffer(chunk) for chunk in chunks] + buffer_ptrs = [ptr for _owner, ptr, _size in prepared_chunks] + sizes = [size for _owner, _ptr, size in prepared_chunks] + registered_ptrs = self._register_buffers( + buffer_ptrs, sizes, pre_registered, "bundle source payload" + ) + try: + results = batch_put_from(list(chunk_keys), buffer_ptrs, sizes) + if len(results) != len(chunk_keys): + raise RuntimeError( + f"batch_put_from returned {len(results)} results for {len(chunk_keys)} chunks" + ) + for chunk_key, status in zip(chunk_keys, results): + _check_status(status, "batch_put_from", chunk_key) + except Exception: + _cleanup_keys(self._store, chunk_keys, strict=False) + raise + finally: + self._unregister_buffers(registered_ptrs, "bundle source payload") + + def _put_chunks_direct( + self, + chunk_keys: Sequence[str], + chunks: Sequence[memoryview], + ) -> list[str]: + written_keys: list[str] = [] + try: + for chunk_key, chunk in zip(chunk_keys, chunks): + _check_status(self._store.put(chunk_key, chunk), "put", chunk_key) + written_keys.append(chunk_key) + except Exception: + _cleanup_keys(self._store, written_keys, strict=False) + raise + return list(chunk_keys) + + def _put_chunks_parallel( + self, + chunk_keys: list[str], + chunks: list[memoryview], + max_inflight_put: int, + pre_registered: bool, + ) -> list[str]: + groups = self._group_chunk_ranges(chunk_keys, chunks, max_inflight_put) + futures: list[Future[None]] = [] + try: + with ThreadPoolExecutor( + max_workers=min(max_inflight_put, len(groups)) + ) as executor: + futures = [ + executor.submit( + self.batch_put_chunks_from, + group_keys, + group_chunks, + pre_registered, + ) + for group_keys, group_chunks in groups + ] + for future in as_completed(futures): + future.result() + except Exception: + for future in futures: + future.cancel() + for future in futures: + if future.done() and not future.cancelled(): + try: + future.result() + except Exception: + pass + _cleanup_keys(self._store, chunk_keys, strict=False) + raise + return chunk_keys + + def _group_chunk_ranges( + self, + chunk_keys: Sequence[str], + chunks: Sequence[memoryview], + max_inflight_put: int, + ) -> list[tuple[list[str], list[memoryview]]]: + if not chunk_keys: + return [] + group_count = max(1, min(max_inflight_put, len(chunks))) + group_size = (len(chunks) + group_count - 1) // group_count + return [ + ( + list(chunk_keys[start : start + group_size]), + list(chunks[start : start + group_size]), + ) + for start in range(0, len(chunks), group_size) + ] + + def _read_chunks_with_batch_get_into( + self, + chunks: Sequence[Mapping[str, Any]], + offsets: Sequence[int], + destination: bytearray | np.ndarray, + ) -> bool: + batch_get_into = self._batch_get_into + if not callable(batch_get_into) or not self._has_buffer_registration_support(): + return False + with self._registered_buffer(destination, "bundle payload") as base_ptr: + keys = [chunk["key"] for chunk in chunks] + ptrs = [base_ptr + offset for offset in offsets] + sizes = [int(chunk["bytes"]) for chunk in chunks] + read_sizes = batch_get_into(keys, ptrs, sizes) + if len(read_sizes) != len(keys): + raise RuntimeError( + f"batch_get_into returned {len(read_sizes)} results for {len(keys)} chunks" + ) + for key, expected_size, actual_size in zip(keys, sizes, read_sizes): + if actual_size != expected_size: + raise RuntimeError( + f"batch_get_into failed for {key}: expected {expected_size}, got {actual_size}" + ) + return True + + def _read_chunk_with_get( + self, + chunk: Mapping[str, Any], + destination: bytearray | np.ndarray, + offset: int, + ) -> None: + chunk_bytes = int(chunk["bytes"]) + if chunk_bytes == 0: + return + data = self._store.get(chunk["key"]) + if len(data) != chunk_bytes: + raise RuntimeError( + f"get failed for {chunk['key']}: expected {chunk_bytes} bytes, got {len(data)}" + ) + destination[offset : offset + chunk_bytes] = data + + def _read_payload_range_into_registered_destination( + self, + chunks: Sequence[Mapping[str, Any]], + destination: np.ndarray, + byte_offset: int, + byte_length: int, + destination_pre_registered: bool, + ) -> bool: + get_into = self._get_into + can_use_get_into = len(chunks) == 1 and byte_offset == 0 and callable(get_into) + get_into_ranges = self._get_into_ranges + can_use_get_into_ranges = callable(get_into_ranges) + if not self._has_buffer_registration_support(): + return False + if not can_use_get_into and not can_use_get_into_ranges: + return False + with self._registered_buffer( + destination, + "structured ndarray payload", + pre_registered=destination_pre_registered, + ) as base_ptr: + if can_use_get_into and int(chunks[0]["bytes"]) == byte_length: + expected_size = int(chunks[0]["bytes"]) + read_size = get_into(chunks[0]["key"], base_ptr, expected_size) + if read_size != expected_size: + raise RuntimeError( + f"get_into failed for {chunks[0]['key']}: expected {expected_size}, got {read_size}" + ) + return True + if not can_use_get_into_ranges: + return False + fragments = _payload_range_fragments(chunks, byte_offset, byte_length) + if not fragments: + return True + keys = [ + key + for key, _chunk_size, _destination_offset, _source_offset, _size in fragments + ] + dst_offsets = [ + [destination_offset] + for _key, _chunk_size, destination_offset, _source_offset, _size in fragments + ] + src_offsets = [ + [source_offset] + for _key, _chunk_size, _destination_offset, source_offset, _size in fragments + ] + sizes = [ + [size] + for _key, _chunk_size, _destination_offset, _source_offset, size in fragments + ] + results = get_into_ranges( + [base_ptr], [keys], [dst_offsets], [src_offsets], [sizes] + ) + if len(results) != 1 or len(results[0]) != len(keys): + raise RuntimeError( + f"get_into_ranges returned invalid ranged result shape for {len(keys)} chunks" + ) + for key, expected_sizes, actual_sizes in zip(keys, sizes, results[0]): + if len(actual_sizes) != len(expected_sizes): + raise RuntimeError( + f"get_into_ranges returned invalid ranged fragment count for {key}" + ) + for expected_size, actual_size in zip(expected_sizes, actual_sizes): + if actual_size != expected_size: + raise RuntimeError( + f"get_into_ranges failed for {key}: expected {expected_size}, got {actual_size}" + ) + return True + + def _copy_payload_range_into_destination( + self, + chunks: Sequence[Mapping[str, Any]], + destination: np.ndarray, + byte_offset: int, + byte_length: int, + ) -> None: + for ( + key, + chunk_size, + destination_offset, + source_offset, + size, + ) in _payload_range_fragments( + chunks, + byte_offset, + byte_length, + ): + data = self._store.get(key) + if len(data) != chunk_size: + raise RuntimeError( + f"get failed for {key}: expected {chunk_size} bytes, got {len(data)}" + ) + fragment = memoryview(data)[source_offset : source_offset + size] + target_end = destination_offset + size + destination[destination_offset:target_end] = np.frombuffer( + fragment, dtype=np.uint8 + ) + + def _register_buffers( + self, + buffer_ptrs: Sequence[int], + sizes: Sequence[int], + pre_registered: bool, + label: str, + ) -> list[int]: + register_buffer = self._register_buffer + unregister_buffer = self._unregister_buffer + if pre_registered: + return [] + if not (callable(register_buffer) and callable(unregister_buffer)): + raise RuntimeError(f"register_buffer APIs are unavailable for {label}") + registered_ptrs: list[int] = [] + try: + for ptr, size in zip(buffer_ptrs, sizes): + register_status = register_buffer(ptr, size) + if register_status == 0: + registered_ptrs.append(ptr) + continue + if not _is_duplicate_buffer_registration(register_status): + _check_status(register_status, "register_buffer", label) + except Exception: + self._unregister_buffers(registered_ptrs, label) + raise + return registered_ptrs + + def _unregister_buffers(self, registered_ptrs: Sequence[int], label: str) -> None: + unregister_buffer = self._unregister_buffer + if not callable(unregister_buffer): + raise RuntimeError(f"unregister_buffer APIs are unavailable for {label}") + for ptr in reversed(registered_ptrs): + _check_status(unregister_buffer(ptr), "unregister_buffer", label) + + @contextmanager + def _registered_buffer( + self, + destination: bytearray | np.ndarray, + label: str, + pre_registered: bool = False, + ) -> Iterator[int]: + base_ptr = _buffer_ptr(destination) + if pre_registered: + yield base_ptr + return + registered_ptrs = self._register_buffers( + [base_ptr], [_buffer_nbytes(destination)], False, label + ) + if not registered_ptrs: + yield base_ptr + return + succeeded = False + try: + yield base_ptr + succeeded = True + finally: + try: + self._unregister_buffers(registered_ptrs, label) + except Exception: + if succeeded: + raise + + def _resolve_put_mode( + self, + chunks: Sequence[memoryview], + transfer_policy: BundleTransferPolicy, + ) -> Literal["batch", "parallel"]: + if transfer_policy.put_mode == "parallel": + return "parallel" + if transfer_policy.put_mode == "batch": + return "batch" + if transfer_policy.max_inflight_put <= 1: + return "batch" + if len(chunks) < AUTO_PARALLEL_MIN_CHUNKS: + return "batch" + if sum(len(chunk) for chunk in chunks) < AUTO_PARALLEL_MIN_BYTES: + return "batch" + if min(transfer_policy.max_inflight_put, len(chunks)) < 2: + return "batch" + return "parallel" + + def _has_batch_put_support(self) -> bool: + return ( + callable(self._batch_put_from) and self._has_buffer_registration_support() + ) + + def _has_buffer_registration_support(self) -> bool: + return callable(self._register_buffer) and callable(self._unregister_buffer) + + +def _resolve_ndarray_destination( + name: str, + destination: Any, + dtype: np.dtype[Any], + shape: tuple[int, ...], +) -> np.ndarray: + if destination is None: + return np.empty(shape, dtype=dtype) + if not isinstance(destination, np.ndarray): + raise TypeError( + f"structured ndarray field {name} destination must be a numpy.ndarray" + ) + if destination.dtype != dtype: + raise ValueError( + f"structured ndarray field {name} destination dtype mismatch: expected {dtype.str}, got {destination.dtype.str}" + ) + if tuple(destination.shape) != shape: + raise ValueError( + f"structured ndarray field {name} destination shape mismatch: expected {shape}, got {tuple(destination.shape)}" + ) + if not destination.flags["C_CONTIGUOUS"]: + raise ValueError( + f"structured ndarray field {name} destination must be C-contiguous" + ) + if not destination.flags["WRITEABLE"]: + raise ValueError( + f"structured ndarray field {name} destination must be writeable" + ) + return destination + + +def _resolve_ndarray_read_plan( + shape: tuple[int, ...], + dtype: np.dtype[Any], + member_slice: StructuredMemberSlice | None, +) -> _NdarrayReadPlan: + if member_slice is None: + return _NdarrayReadPlan( + dtype=dtype, + full_shape=shape, + output_shape=shape, + byte_offset=0, + byte_length=int(np.prod(shape, dtype=np.int64)) * dtype.itemsize + if shape + else dtype.itemsize, + step=1, + cover_row_count=shape[0] if shape else 1, + ) + if member_slice.axis != 0: + raise ValueError("structured ndarray slicing currently supports axis=0 only") + if not shape: + raise ValueError("structured ndarray slicing requires at least one dimension") + total_rows = shape[0] + start, end, step = _normalized_member_slice(member_slice, total_rows) + row_width = dtype.itemsize * int(np.prod(shape[1:], dtype=np.int64)) + if start >= end: + return _NdarrayReadPlan( + dtype=dtype, + full_shape=shape, + output_shape=(0, *shape[1:]), + byte_offset=start * row_width, + byte_length=0, + step=step, + cover_row_count=0, + ) + selected_count = 1 + (end - 1 - start) // step + cover_start = start + cover_end = start + (selected_count - 1) * step + 1 + return _NdarrayReadPlan( + dtype=dtype, + full_shape=shape, + output_shape=(selected_count, *shape[1:]), + byte_offset=cover_start * row_width, + byte_length=(cover_end - cover_start) * row_width, + step=step, + cover_row_count=cover_end - cover_start, + ) + + +def _normalized_member_slice( + member_slice: StructuredMemberSlice, total_rows: int +) -> tuple[int, int, int]: + if member_slice.step <= 0: + raise ValueError("structured ndarray slicing step must be positive") + return slice(member_slice.start, member_slice.end, member_slice.step).indices( + total_rows + ) + + +def _bytes_view(value: Any, name: str) -> memoryview: + try: + view = memoryview(value) + except TypeError as error: + raise TypeError( + f"{name} must be bytes-like, got {type(value).__name__}" + ) from error + if not view.contiguous: + view = memoryview(bytes(view)) + return view.cast("B") + + +def _prepare_chunk_source_buffer(chunk: memoryview) -> tuple[Any, int, int]: + if chunk.c_contiguous and not chunk.readonly: + return chunk, ctypes.addressof(ctypes.c_char.from_buffer(chunk)), len(chunk) + copied = ctypes.create_string_buffer(bytes(chunk)) + return copied, ctypes.addressof(copied), len(chunk) + + +def _buffer_ptr(value: bytearray | np.ndarray) -> int: + return ctypes.addressof(ctypes.c_char.from_buffer(value)) + + +def _buffer_nbytes(value: bytearray | np.ndarray) -> int: + if isinstance(value, np.ndarray): + return int(value.nbytes) + return len(value) + + +def _split_view(view: memoryview, chunk_bytes: int) -> list[memoryview]: + if len(view) == 0: + return [view] + return [ + view[start : start + chunk_bytes] for start in range(0, len(view), chunk_bytes) + ] + + +def _chunk_offsets(chunks: Sequence[Mapping[str, Any]]) -> list[int]: + offsets = [0] + for chunk in chunks[:-1]: + offsets.append(offsets[-1] + int(chunk["bytes"])) + return offsets + + +def _payload_range_fragments( + chunks: Sequence[Mapping[str, Any]], + byte_offset: int, + byte_length: int, +) -> list[tuple[str, int, int, int, int]]: + fragments: list[tuple[str, int, int, int, int]] = [] + read_end = byte_offset + byte_length + chunk_offset = 0 + for chunk in chunks: + chunk_size = int(chunk["bytes"]) + chunk_end = chunk_offset + chunk_size + overlap_start = max(byte_offset, chunk_offset) + overlap_end = min(read_end, chunk_end) + if overlap_start < overlap_end: + fragments.append( + ( + chunk["key"], + chunk_size, + overlap_start - byte_offset, + overlap_start - chunk_offset, + overlap_end - overlap_start, + ) + ) + chunk_offset = chunk_end + return fragments + + +def _validate_chunk_bytes(chunk_bytes: int) -> int: + if chunk_bytes <= 0: + raise ValueError("chunk_bytes must be positive") + return chunk_bytes + + +def _normalize_key_prefix(key_prefix: str) -> str: + prefix = key_prefix.strip("/") + if not prefix or any(ord(char) < 32 for char in prefix): + raise ValueError("key_prefix must be a non-empty key prefix") + return prefix + + +def _validate_key_segment(value: str, name: str) -> None: + if ( + not isinstance(value, str) + or not value + or "/" in value + or any(ord(char) < 32 for char in value) + ): + raise ValueError(f"invalid bundle {name}: {value!r}") + + +def _encode_manifest(manifest: Mapping[str, Any]) -> bytes: + return _encode_json_dict(manifest, "bundle manifest") + + +def _decode_manifest(payload: bytes) -> dict[str, Any]: + return _decode_json_dict(payload, "bundle manifest") + + +def _normalize_structured_metadata(metadata: Mapping[str, Any]) -> dict[str, Any]: + if not isinstance(metadata, Mapping): + raise TypeError("structured metadata must be a mapping") + normalized = dict(metadata) + if STRUCTURED_FIELD_SPECS_KEY in normalized: + raise ValueError( + f"structured metadata key {STRUCTURED_FIELD_SPECS_KEY!r} is reserved" + ) + if not isinstance(normalized.get("layout", "structured"), str): + raise ValueError("structured metadata layout must be a string") + normalized.setdefault("layout", "structured") + return normalized + + +def _encode_structured_fields( + metadata: Mapping[str, Any], + fields: Mapping[str, Any], +) -> tuple[dict[str, Any], dict[str, Any]]: + normalized_metadata = _normalize_structured_metadata(metadata) + field_specs: dict[str, Any] = {} + encoded_fields: dict[str, Any] = {} + for name, value in fields.items(): + _validate_key_segment(name, "buffer name") + spec, encoded_value = _encode_structured_field(value) + field_specs[name] = spec + encoded_fields[name] = encoded_value + if field_specs: + normalized_metadata[STRUCTURED_FIELD_SPECS_KEY] = field_specs + return normalized_metadata, encoded_fields + + +def _encode_structured_field(value: Any) -> tuple[dict[str, Any], Any]: + if isinstance(value, np.ndarray): + array = np.ascontiguousarray(value) + return { + "encoding": "ndarray", + "dtype": array.dtype.str, + "shape": list(array.shape), + }, array.view(np.uint8).reshape(-1) + return {"encoding": "bytes"}, value + + +def _structured_field_specs(metadata: Mapping[str, Any]) -> dict[str, Any]: + field_specs = metadata.get(STRUCTURED_FIELD_SPECS_KEY, {}) + if not isinstance(field_specs, dict): + raise ValueError("structured field specs must be a dict") + return field_specs + + +def _encode_structured_metadata(metadata: Mapping[str, Any]) -> bytes: + return _encode_json_dict(metadata, "structured metadata") + + +def _decode_structured_metadata(payload: bytes) -> dict[str, Any]: + return _decode_json_dict(payload, "structured metadata") + + +def _encode_json_dict(value: Mapping[str, Any], label: str) -> bytes: + try: + return json.dumps(value, separators=(",", ":")).encode("utf-8") + except TypeError as error: + raise TypeError(f"{label} must be JSON-serializable") from error + + +def _decode_json_dict(payload: bytes, label: str) -> dict[str, Any]: + value = json.loads(payload.decode("utf-8")) + if not isinstance(value, dict): + raise ValueError(f"{label} must decode to a dict") + return value + + +def _check_status(status: Any, operation: str, key: str) -> None: + if status not in (None, 0): + raise RuntimeError(f"{operation} failed for {key}: {status}") + + +def _is_duplicate_buffer_registration(status: Any) -> bool: + # Mooncake returns -600 when a caller-owned buffer is already registered. + return status == -600 + + +def _cleanup_keys(store: BundleStore, keys: Sequence[str], strict: bool) -> None: + errors = [] + pending_keys = list(dict.fromkeys(keys)) + batch_remove = getattr(store, "batch_remove", None) + if callable(batch_remove) and pending_keys: + try: + try: + results = batch_remove(pending_keys, True) + except TypeError: + results = batch_remove(pending_keys) + if len(results) != len(pending_keys): + raise RuntimeError( + f"batch_remove returned {len(results)} results for {len(pending_keys)} keys" + ) + failed_results = [ + (key, status) + for key, status in zip(pending_keys, results) + if status not in (None, 0, MISSING_OBJECT_ERROR) + ] + if not failed_results: + return + errors.extend(failed_results) + pending_keys = [key for key, _status in failed_results] + except Exception: + if strict: + raise + + retry_errors = [] + for key in pending_keys: + try: + status = store.remove(key, True) + except KeyError: + continue + except Exception: + if strict: + raise + continue + if status not in (None, 0, MISSING_OBJECT_ERROR): + retry_errors.append((key, status)) + if retry_errors: + errors = retry_errors + if errors and strict: + raise RuntimeError( + f"failed to remove {len(errors)} Mooncake keys: {errors[:3]}" + ) diff --git a/mooncake-wheel/tests/conftest.py b/mooncake-wheel/tests/conftest.py new file mode 100644 index 00000000..30728561 --- /dev/null +++ b/mooncake-wheel/tests/conftest.py @@ -0,0 +1,19 @@ +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +SOURCE_PACKAGE = ROOT / "mooncake" + +try: + import mooncake +except ModuleNotFoundError: + if str(ROOT) not in sys.path: + sys.path.insert(0, str(ROOT)) +else: + if importlib.util.find_spec("mooncake.structured_object_store") is None: + mooncake_path = getattr(mooncake, "__path__", None) + if mooncake_path is not None and str(SOURCE_PACKAGE) not in mooncake_path: + mooncake_path.append(str(SOURCE_PACKAGE)) diff --git a/mooncake-wheel/tests/test_structured_object_store.py b/mooncake-wheel/tests/test_structured_object_store.py new file mode 100644 index 00000000..532dcb77 --- /dev/null +++ b/mooncake-wheel/tests/test_structured_object_store.py @@ -0,0 +1,769 @@ +from __future__ import annotations + +import ctypes +import json +import threading +import time + +import numpy as np +import pytest + +from mooncake.structured_object_store import ( + BundleTransferPolicy, + MooncakeBundleTransfer, + RemoteBundleRef, + StructuredObjectPayload, + StructuredObjectReadSpec, +) + + +class InMemoryStore: + def __init__(self) -> None: + self.objects: dict[str, bytes] = {} + self.lock = threading.Lock() + self.registered: set[int] = set() + self.max_active_puts = 0 + self.max_active_gets = 0 + self.active_puts = 0 + self.active_gets = 0 + self.get_into_calls = 0 + self.get_into_ranges_calls = 0 + self.batch_get_into_calls = 0 + self.batch_remove_calls = 0 + + def _enter_put(self) -> None: + with self.lock: + self.active_puts += 1 + self.max_active_puts = max(self.max_active_puts, self.active_puts) + + def _exit_put(self) -> None: + with self.lock: + self.active_puts -= 1 + + def _enter_get(self, count: int = 1) -> None: + with self.lock: + self.active_gets += count + self.max_active_gets = max(self.max_active_gets, self.active_gets) + + def _exit_get(self, count: int = 1) -> None: + with self.lock: + self.active_gets -= count + + def put(self, key: str, value) -> int: + self._enter_put() + try: + time.sleep(0.01) + with self.lock: + self.objects[key] = bytes(value) + return 0 + finally: + self._exit_put() + + def get(self, key: str) -> bytes: + self._enter_get() + try: + time.sleep(0.01) + with self.lock: + return self.objects[key] + finally: + self._exit_get() + + def remove(self, key: str, force: bool = False) -> int: + with self.lock: + self.objects.pop(key, None) + return 0 + + def batch_remove(self, keys: list[str], force: bool = False) -> list[int]: + self.batch_remove_calls += 1 + for key in keys: + self.remove(key, force) + return [0 for _key in keys] + + def batch_put_from( + self, keys: list[str], buffer_ptrs: list[int], sizes: list[int] + ) -> list[int]: + results: list[int] = [] + for key, ptr, size in zip(keys, buffer_ptrs, sizes): + data = ctypes.string_at(ptr, size) + results.append(self.put(key, data)) + return results + + def register_buffer(self, buffer_ptr: int, size: int) -> int: + self.registered.add(buffer_ptr) + return 0 + + def unregister_buffer(self, buffer_ptr: int) -> int: + self.registered.remove(buffer_ptr) + return 0 + + def get_into(self, key: str, ptr: int, size: int) -> int: + self.get_into_calls += 1 + self._enter_get() + try: + time.sleep(0.01) + with self.lock: + data = self.objects[key] + if len(data) > size: + return -1 + ctypes.memmove(ptr, data, len(data)) + return len(data) + finally: + self._exit_get() + + def get_into_ranges( + self, + buffer_ptrs: list[int], + all_keys: list[list[str]], + all_dst_offsets: list[list[list[int]]], + all_src_offsets: list[list[list[int]]], + all_sizes: list[list[list[int]]], + ) -> list[list[list[int]]]: + self.get_into_ranges_calls += 1 + total_keys = sum(len(keys) for keys in all_keys) + self._enter_get(total_keys) + try: + time.sleep(0.01) + results: list[list[list[int]]] = [] + for base_ptr, keys, dst_groups, src_groups, size_groups in zip( + buffer_ptrs, all_keys, all_dst_offsets, all_src_offsets, all_sizes + ): + buffer_results: list[list[int]] = [] + for key, dst_offsets, src_offsets, sizes in zip( + keys, dst_groups, src_groups, size_groups + ): + with self.lock: + data = self.objects[key] + key_results: list[int] = [] + for dst_offset, src_offset, size in zip( + dst_offsets, src_offsets, sizes + ): + end = src_offset + size + if end > len(data): + key_results.append(-1) + continue + ctypes.memmove( + base_ptr + dst_offset, data[src_offset:end], size + ) + key_results.append(size) + buffer_results.append(key_results) + results.append(buffer_results) + return results + finally: + self._exit_get(total_keys) + + def batch_get_into( + self, keys: list[str], ptrs: list[int], sizes: list[int] + ) -> list[int]: + self.batch_get_into_calls += 1 + self._enter_get(len(keys)) + try: + time.sleep(0.01) + results = [] + for key, ptr, size in zip(keys, ptrs, sizes): + with self.lock: + data = self.objects[key] + if len(data) > size: + results.append(-1) + continue + ctypes.memmove(ptr, data, len(data)) + results.append(len(data)) + return results + finally: + self._exit_get(len(keys)) + + +class GetOnlyStore(InMemoryStore): + batch_get_into = None + + +class MinimalStore(GetOnlyStore): + batch_put_from = None + register_buffer = None + unregister_buffer = None + + +class ReadFastPathWithoutRegisterStore(MinimalStore): + batch_get_into = InMemoryStore.batch_get_into + get_into = InMemoryStore.get_into + get_into_ranges = InMemoryStore.get_into_ranges + + +class PlainStore(MinimalStore): + get_into = None + get_into_ranges = None + + +class FailingPutStore(InMemoryStore): + def __init__(self, fail_on_put: int) -> None: + super().__init__() + self.put_count = 0 + self.fail_on_put = fail_on_put + + def put(self, key: str, value) -> int: + self.put_count += 1 + if self.put_count == self.fail_on_put: + raise RuntimeError("injected put failure") + return super().put(key, value) + + +class FailingBatchGetStore(InMemoryStore): + def __init__(self) -> None: + super().__init__() + self.fail_key = "" + + def batch_get_into( + self, keys: list[str], ptrs: list[int], sizes: list[int] + ) -> list[int]: + if self.fail_key in keys: + raise RuntimeError("injected get failure") + return super().batch_get_into(keys, ptrs, sizes) + + +class FailingRemoveStore(FailingPutStore): + def remove(self, key: str, force: bool = False) -> int: + raise RuntimeError("injected remove failure") + + batch_remove = None + + +class ForceTrackingStore(InMemoryStore): + def __init__(self) -> None: + super().__init__() + self.batch_remove_forces: list[bool] = [] + + def batch_remove(self, keys: list[str], force: bool = False) -> list[int]: + self.batch_remove_forces.append(force) + return super().batch_remove(keys, force) + + +class StrictRegisterStore(InMemoryStore): + def register_buffer(self, buffer_ptr: int, size: int) -> int: + if buffer_ptr in self.registered: + return -600 + return super().register_buffer(buffer_ptr, size) + + +class FailingRegisterStore(InMemoryStore): + def __init__(self, fail_on_register: int) -> None: + super().__init__() + self.register_count = 0 + self.fail_on_register = fail_on_register + + def register_buffer(self, buffer_ptr: int, size: int) -> int: + self.register_count += 1 + if self.register_count == self.fail_on_register: + return -1 + return super().register_buffer(buffer_ptr, size) + + +def make_transfer( + store: InMemoryStore | None = None, + *, + key_prefix: str = "test", + default_chunk_bytes: int | None = None, +) -> tuple[InMemoryStore, MooncakeBundleTransfer]: + current_store = store or InMemoryStore() + kwargs = {"key_prefix": key_prefix} + if default_chunk_bytes is not None: + kwargs["default_chunk_bytes"] = default_chunk_bytes + return current_store, MooncakeBundleTransfer(current_store, **kwargs) + + +def structured_payload( + metadata: dict[str, object] | None = None, **buffers: object +) -> StructuredObjectPayload: + return StructuredObjectPayload(metadata=metadata, buffers=buffers) + + +def write_manifest( + store: InMemoryStore, manifest_key: str, manifest: dict[str, object] +) -> None: + store.objects[manifest_key] = json.dumps(manifest, separators=(",", ":")).encode( + "utf-8" + ) + + +def test_bundle_read_spec_full_read_is_partial_special_case() -> None: + store, transfer = make_transfer() + array = np.arange(16, dtype=np.int32).reshape(4, 4) + payload = structured_payload({"type": "example"}, array=array, raw=b"abc") + + ref = transfer.put_structured_object(payload) + result = transfer.materialize(transfer.read_spec(ref)) + + assert np.array_equal(result.objects["array"], array) + assert result.objects["raw"] == b"abc" + assert store.batch_get_into_calls > 0 + assert store.registered == set() + + +def test_structured_object_payload_metadata_defaults_empty() -> None: + store, transfer = make_transfer() + array = np.arange(8, dtype=np.int32).reshape(2, 4) + + ref = transfer.put_structured_object( + StructuredObjectPayload(buffers={"array": array}) + ) + result = transfer.materialize(transfer.read_spec(ref)) + + assert np.array_equal(result.objects["array"], array) + + +def test_bundle_chunked_full_read_via_read_spec() -> None: + store, transfer = make_transfer() + payload = bytes(range(128)) + + ref = transfer.put_structured_object( + structured_payload(payload=payload), + chunk_bytes=17, + ) + result = transfer.materialize(transfer.read_spec(ref)) + + assert result.objects["payload"] == payload + assert len(ref.manifest["buffers"]["payload"]["chunks"]) > 1 + + +def test_bundle_put_falls_back_to_store_put_without_batch_put_support() -> None: + store, transfer = make_transfer(MinimalStore()) + payload = bytes(range(128)) + + ref = transfer.put_structured_object( + structured_payload(payload=payload), chunk_bytes=17 + ) + result = transfer.materialize(transfer.read_spec(ref)) + + assert result.objects["payload"] == payload + assert len(ref.manifest["buffers"]["payload"]["chunks"]) > 1 + assert store.registered == set() + + +def test_bundle_uses_configurable_default_chunk_size() -> None: + store, transfer = make_transfer(default_chunk_bytes=17) + payload = bytes(range(128)) + + ref = transfer.put_bundle(b"meta", {"payload": payload}) + + assert len(ref.manifest["buffers"]["payload"]["chunks"]) > 1 + assert ref.manifest["buffers"]["payload"]["chunks"][0]["bytes"] == 17 + + +def test_structured_object_roundtrip() -> None: + store, transfer = make_transfer() + array = np.arange(12, dtype=np.int16).reshape(3, 4) + payload = structured_payload( + {"type": "example", "fields": ["weights", "tokens"]}, + weights=array, + tokens=b"abc", + ) + + ref = transfer.put_structured_object(payload) + result = transfer.materialize(transfer.read_spec(ref)) + + assert result.metadata == { + "layout": "structured", + "type": "example", + "fields": ["weights", "tokens"], + "__mooncake_structured_fields__": { + "weights": { + "encoding": "ndarray", + "dtype": array.dtype.str, + "shape": [3, 4], + }, + "tokens": {"encoding": "bytes"}, + }, + } + assert np.array_equal(result.objects["weights"], array) + assert result.objects["weights"].flags["C_CONTIGUOUS"] + assert result.objects["tokens"] == b"abc" + + +def test_structured_object_read_spec_select_members() -> None: + store, transfer = make_transfer() + array = np.arange(10, dtype=np.float32).reshape(2, 5) + payload = structured_payload(weights=array, raw=b"payload") + + ref = transfer.put_structured_object(payload) + spec = transfer.read_spec(ref).select_members(["weights"]) + assert isinstance(spec, StructuredObjectReadSpec) + batch_get_into_calls = store.batch_get_into_calls + result = transfer.materialize(spec) + + assert list(result.objects) == ["weights"] + assert np.array_equal(result.objects["weights"], array) + assert store.get_into_calls > 0 + assert store.get_into_ranges_calls == 0 + assert store.batch_get_into_calls == batch_get_into_calls + 1 + + +def test_structured_object_multichunk_ndarray_uses_range_gather() -> None: + store, transfer = make_transfer() + array = np.arange(64, dtype=np.int16).reshape(8, 8) + payload = structured_payload(weights=array) + + ref = transfer.put_structured_object(payload, chunk_bytes=32) + batch_get_into_calls = store.batch_get_into_calls + result = transfer.materialize(transfer.read_spec(ref)) + + assert np.array_equal(result.objects["weights"], array) + assert store.get_into_ranges_calls > 0 + assert store.batch_get_into_calls == batch_get_into_calls + 1 + + +def test_structured_object_slice_member_uses_partial_range_reads() -> None: + store, transfer = make_transfer() + array = np.arange(96, dtype=np.int16).reshape(12, 8) + payload = structured_payload(weights=array, raw=b"payload") + + ref = transfer.put_structured_object(payload, chunk_bytes=20) + spec = ( + transfer.read_spec(ref) + .select_members(["weights"]) + .slice_member("weights", axis=0, start=3, end=9) + ) + before_range_reads = store.get_into_ranges_calls + result = transfer.materialize(spec) + + assert np.array_equal(result.objects["weights"], array[3:9]) + assert result.objects["weights"].shape == (6, 8) + assert store.get_into_ranges_calls == before_range_reads + 1 + + +def test_structured_object_slice_member_falls_back_to_plain_get_reads() -> None: + store, transfer = make_transfer(PlainStore()) + array = np.arange(96, dtype=np.int16).reshape(12, 8) + payload = structured_payload(weights=array, raw=b"payload") + + ref = transfer.put_structured_object(payload, chunk_bytes=20) + spec = ( + transfer.read_spec(ref) + .select_members(["weights"]) + .slice_member("weights", axis=0, start=3, end=9) + ) + result = transfer.materialize(spec) + + assert np.array_equal(result.objects["weights"], array[3:9]) + assert result.objects["weights"].shape == (6, 8) + assert store.get_into_calls == 0 + assert store.get_into_ranges_calls == 0 + assert store.batch_get_into_calls == 0 + + +def test_structured_object_materialize_into_reuses_destination() -> None: + store, transfer = make_transfer() + array = np.arange(96, dtype=np.float32).reshape(12, 8) + payload = structured_payload(weights=array) + + ref = transfer.put_structured_object(payload, chunk_bytes=40) + spec = transfer.read_spec(ref).slice_member("weights", axis=0, start=2, end=10) + destination = np.empty((8, 8), dtype=np.float32) + result = transfer.materialize_into(spec, destinations={"weights": destination}) + + assert result.objects["weights"] is destination + assert np.array_equal(destination, array[2:10]) + assert store.get_into_ranges_calls > 0 + + +def test_structured_object_duplicate_destination_registration_is_tolerated() -> None: + store, transfer = make_transfer(StrictRegisterStore()) + array = np.arange(96, dtype=np.float32).reshape(12, 8) + payload = structured_payload(weights=array) + + ref = transfer.put_structured_object(payload, chunk_bytes=40) + spec = ( + transfer.read_spec(ref) + .select_members(["weights"]) + .slice_member("weights", axis=0, start=2, end=10) + ) + destination = np.empty((8, 8), dtype=np.float32) + destination_ptr = ctypes.addressof(ctypes.c_char.from_buffer(destination)) + assert store.register_buffer(destination_ptr, int(destination.nbytes)) == 0 + + result = transfer.materialize_into(spec, destinations={"weights": destination}) + + assert result.objects["weights"] is destination + assert np.array_equal(destination, array[2:10]) + assert store.get_into_ranges_calls > 0 + assert destination_ptr in store.registered + store.unregister_buffer(destination_ptr) + + +def test_structured_object_slice_member_step_copy() -> None: + store, transfer = make_transfer() + array = np.arange(120, dtype=np.int32).reshape(15, 8) + payload = structured_payload(weights=array) + + ref = transfer.put_structured_object(payload, chunk_bytes=24) + spec = transfer.read_spec(ref).slice_member( + "weights", axis=0, start=1, end=12, step=3 + ) + result = transfer.materialize(spec) + + assert np.array_equal(result.objects["weights"], array[1:12:3]) + assert result.objects["weights"].shape == array[1:12:3].shape + assert store.get_into_ranges_calls > 0 + + +def test_structured_object_invalid_slice_and_destination_raise() -> None: + store, transfer = make_transfer() + array = np.arange(24, dtype=np.int16).reshape(6, 4) + ref = transfer.put_structured_object( + structured_payload(weights=array, raw=b"abc"), + chunk_bytes=8, + ) + + with pytest.raises(ValueError, match="axis=0"): + transfer.materialize( + transfer.read_spec(ref).slice_member("weights", axis=1, start=0, end=2) + ) + with pytest.raises(ValueError, match="step must be positive"): + transfer.materialize( + transfer.read_spec(ref).slice_member( + "weights", axis=0, start=0, end=2, step=0 + ) + ) + with pytest.raises(ValueError, match="does not support slicing"): + transfer.materialize( + transfer.read_spec(ref).slice_member("raw", axis=0, start=0, end=1) + ) + with pytest.raises(ValueError, match="shape mismatch"): + transfer.materialize_into( + transfer.read_spec(ref).slice_member("weights", axis=0, start=1, end=3), + destinations={"weights": np.empty((3, 4), dtype=np.int16)}, + ) + readonly_destination = np.empty((2, 4), dtype=np.int16) + readonly_destination.flags.writeable = False + with pytest.raises(ValueError, match="writeable"): + transfer.materialize_into( + transfer.read_spec(ref).slice_member("weights", axis=0, start=1, end=3), + destinations={"weights": readonly_destination}, + ) + + +def test_bundle_remove_deletes_payload_and_manifest() -> None: + store, transfer = make_transfer() + + ref = transfer.put_bundle(b"meta", {"payload": b"data"}) + assert store.objects + transfer.remove_bundle(ref) + + assert store.objects == {} + assert store.batch_remove_calls == 1 + + +def test_bundle_partial_put_failure_cleans_payloads() -> None: + store, transfer = make_transfer(FailingPutStore(fail_on_put=2)) + + with pytest.raises(RuntimeError, match="injected put failure"): + transfer.put_bundle(b"meta", {"payload": b"data"}, chunk_bytes=2) + + assert store.objects == {} + + +def test_bundle_cleanup_failure_preserves_put_error() -> None: + _store, transfer = make_transfer(FailingRemoveStore(fail_on_put=2)) + + with pytest.raises(RuntimeError, match="injected put failure"): + transfer.put_bundle(b"meta", {"payload": b"data"}, chunk_bytes=2) + + +def test_bundle_remove_uses_force_batch_remove_when_available() -> None: + store, transfer = make_transfer(ForceTrackingStore()) + + ref = transfer.put_structured_object( + structured_payload(payload=bytes(range(64))), + chunk_bytes=8, + ) + transfer.materialize(transfer.read_spec(ref)) + transfer.remove_bundle(ref) + + assert store.batch_remove_forces == [True] + assert store.objects == {} + + +def test_bundle_concurrent_put_and_read_spec_full_read() -> None: + store, transfer = make_transfer(GetOnlyStore()) + payload = bytes(range(128)) + + ref = transfer.put_structured_object( + structured_payload(payload=payload), + chunk_bytes=8, + policy=BundleTransferPolicy(max_inflight_put=4, put_mode="parallel"), + ) + result = transfer.materialize(transfer.read_spec(ref)) + + assert result.objects["payload"] == payload + assert store.max_active_puts > 1 + assert store.max_active_gets >= 1 + + +def test_bundle_duplicate_source_registration_is_tolerated() -> None: + store, transfer = make_transfer(StrictRegisterStore()) + payload = np.arange(64, dtype=np.uint8).reshape(8, 8) + payload_ptr = ctypes.addressof(ctypes.c_char.from_buffer(payload)) + assert store.register_buffer(payload_ptr, int(payload.nbytes)) == 0 + + ref = transfer.put_structured_object(structured_payload(payload=payload)) + + assert ref.manifest["buffers"]["payload"]["bytes"] == int(payload.nbytes) + assert payload_ptr in store.registered + store.unregister_buffer(payload_ptr) + + +def test_bundle_partial_register_failure_unwinds_registered_buffers() -> None: + store, transfer = make_transfer(FailingRegisterStore(fail_on_register=2)) + payload = bytes(range(64)) + + with pytest.raises(RuntimeError, match="register_buffer"): + transfer.put_structured_object( + structured_payload(payload=payload), chunk_bytes=32 + ) + + assert store.registered == set() + assert store.objects == {} + + +def test_bundle_batch_get_failure_unregisters_buffer() -> None: + store, transfer = make_transfer(FailingBatchGetStore()) + ref = transfer.put_structured_object( + structured_payload(payload=bytes(range(64))), + chunk_bytes=8, + ) + store.fail_key = ref.manifest["buffers"]["payload"]["chunks"][2]["key"] + + with pytest.raises(RuntimeError, match="injected get failure"): + transfer.materialize(transfer.read_spec(ref)) + + assert store.registered == set() + + +def test_bundle_falls_back_to_get_without_batch_get_into() -> None: + store, transfer = make_transfer(GetOnlyStore()) + + ref = transfer.put_structured_object( + structured_payload(payload=b"abcdef"), + chunk_bytes=2, + ) + result = transfer.materialize(transfer.read_spec(ref)) + + assert result.objects["payload"] == b"abcdef" + assert store.max_active_gets >= 1 + + +def test_bundle_read_fast_paths_require_registration_support() -> None: + store, transfer = make_transfer(ReadFastPathWithoutRegisterStore()) + + ref = transfer.put_structured_object( + structured_payload(payload=bytes(range(64))), + chunk_bytes=8, + ) + result = transfer.materialize(transfer.read_spec(ref)) + + assert result.objects["payload"] == bytes(range(64)) + assert store.batch_get_into_calls == 0 + assert store.get_into_calls == 0 + assert store.get_into_ranges_calls == 0 + + +def test_bundle_invalid_policy_and_chunk_size_raise() -> None: + store, transfer = make_transfer() + + with pytest.raises(ValueError, match="max_inflight_put"): + transfer.put_bundle(b"meta", {"payload": b"data"}, max_inflight_put=0) + with pytest.raises(ValueError, match="selected no members"): + transfer.materialize( + transfer.read_spec( + {"manifest_key": "test/default/example/manifest"} + ).select_members([]) + ) + with pytest.raises(ValueError, match="chunk_bytes"): + transfer.put_bundle(b"meta", {"payload": b"data"}, chunk_bytes=0) + + +def test_bundle_invalid_name_and_prefix_raise() -> None: + store, transfer = make_transfer() + + with pytest.raises(ValueError, match="buffer name"): + transfer.put_bundle(b"meta", {"bad/name": b"data"}) + with pytest.raises(ValueError, match="partition"): + transfer.put_bundle(b"meta", {"payload": b"data"}, partition="bad/name") + with pytest.raises(ValueError, match="key_prefix"): + MooncakeBundleTransfer(store, key_prefix="") + with pytest.raises(ValueError, match="chunk_bytes"): + MooncakeBundleTransfer(store, default_chunk_bytes=0) + + +def test_structured_object_invalid_metadata_raises() -> None: + store, transfer = make_transfer() + + with pytest.raises(TypeError, match="mapping"): + transfer.put_structured_object( + StructuredObjectPayload(metadata=["not", "a", "mapping"], buffers={}) + ) + with pytest.raises(TypeError, match="JSON-serializable"): + transfer.put_structured_object( + StructuredObjectPayload(metadata={"bad": object()}, buffers={}) + ) + with pytest.raises(ValueError, match="reserved"): + transfer.put_structured_object( + StructuredObjectPayload( + metadata={"__mooncake_structured_fields__": {}}, + buffers={"payload": b"abc"}, + ) + ) + + +def test_structured_object_rejects_invalid_field_spec() -> None: + store, transfer = make_transfer() + manifest_key = "test/default/example/manifest" + meta_key = "test/default/example/meta" + payload_key = "test/default/example/buffer/payload" + metadata_blob = ( + b'{"layout":"structured","__mooncake_structured_fields__":' + b'{"payload":{"encoding":"unknown"}}}' + ) + store.objects[meta_key] = metadata_blob + store.objects[payload_key] = b"abc" + write_manifest( + store, + manifest_key, + { + "version": 1, + "layout": "bundle", + "object_id": "default/example", + "meta": { + "key": meta_key, + "bytes": len(metadata_blob), + "chunks": [{"key": meta_key, "bytes": len(metadata_blob)}], + }, + "buffers": { + "payload": { + "key": payload_key, + "bytes": 3, + "chunks": [{"key": payload_key, "bytes": 3}], + } + }, + }, + ) + + with pytest.raises(ValueError, match="unsupported structured field encoding"): + transfer.materialize(transfer.read_spec({"manifest_key": manifest_key})) + + +def test_bundle_rejects_tampered_manifest() -> None: + store, transfer = make_transfer() + ref = transfer.put_bundle(b"meta", {"payload": b"abcdef"}, chunk_bytes=2) + tampered = dict(ref.manifest) + tampered["buffers"] = dict(ref.manifest["buffers"]) + tampered["buffers"]["payload"] = dict(ref.manifest["buffers"]["payload"]) + tampered["buffers"]["payload"]["chunks"] = [ + {"key": "other/object", "bytes": 6}, + ] + write_manifest(store, ref.manifest_key, tampered) + + with pytest.raises(ValueError, match="namespace"): + transfer.materialize(transfer.read_spec({"manifest_key": ref.manifest_key})) + + with pytest.raises(ValueError, match="manifest_key"): + transfer.remove_bundle( + RemoteBundleRef(manifest_key="test/other/manifest", manifest=ref.manifest) + ) -- 2.34.1 From e9c61075720039bcfc5fffd19f847608402be3d0 Mon Sep 17 00:00:00 2001 From: Shangming Cai Date: Sat, 23 May 2026 11:38:17 +0800 Subject: [PATCH 116/382] Bump version to 0.3.11.post1 in pyproject.toml (#2194) --- mooncake-wheel/pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mooncake-wheel/pyproject.toml b/mooncake-wheel/pyproject.toml index 06f0c092..a7168499 100644 --- a/mooncake-wheel/pyproject.toml +++ b/mooncake-wheel/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "mooncake-transfer-engine" -version = "0.3.11" +version = "0.3.11.post1" description = "Python binding of a Mooncake library using pybind11" authors = [ { name = "Mooncake Authors" } -- 2.34.1 From bd6d1125d29d865984d694c5c2a87d2c6fe9acc1 Mon Sep 17 00:00:00 2001 From: JieTang66 <57845979+JieTang66@users.noreply.github.com> Date: Sun, 24 May 2026 15:38:38 +0800 Subject: [PATCH 117/382] [CI] Strip shared libraries to reduce NPU wheel size (#2202) Co-authored-by: JieTang --- scripts/build_wheel.sh | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/scripts/build_wheel.sh b/scripts/build_wheel.sh index d37f4359..feff9662 100755 --- a/scripts/build_wheel.sh +++ b/scripts/build_wheel.sh @@ -137,6 +137,11 @@ if [ "$CI" = "true" ] || [ "$FREE_BUILD_DIR" = "1" ]; then fi fi +if [ "$NPU_BUILD" = "1" ]; then + echo "Stripping shared libraries to reduce wheel size..." + find mooncake-wheel/mooncake -name "*.so" -exec strip --strip-unneeded {} \; +fi + echo "Building wheel package..." # Build the wheel package cd mooncake-wheel -- 2.34.1 From 4cdfc6509637bffc6191270040a3852e564853b4 Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Mon, 25 May 2026 11:11:54 +0800 Subject: [PATCH 118/382] [Docs] Add Citation Paper (#2190) Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- README.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 15d7ae33..cf3eaca1 100644 --- a/README.md +++ b/README.md @@ -377,9 +377,16 @@ The above presents two samples from our trace dataset. The trace includes the ti **_Update[Feb 21, 2025]: The updated [traces](FAST25-release/traces) used in our FAST'25 paper have been released! Please refer to the paper's appendix (found [here](FAST25-release/Mooncake-FAST25.pdf)) for more details._**

📑 Citation

-Please kindly cite our paper if you find the paper or the traces are useful: +Please kindly cite our papers if you find the papers or the traces are useful: ```bibtex +@article{sun2026survivingpartialrankfailures, + title = {Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference}, + author = {Xun Sun and Shaoyuan Chen and Pingchuan Ma and Yue Chen and Ziwei Yuan and Zhanhao Cao and Han Han and Shangming Cai and Teng Ma and Xuchun Shang and Xinpeng Zhao and Ke Yang and Junlin Wei and Lianzhi Lin and Yuji Liu and Feng Ren and Haoran Hu and Cheng Wan and Yingdi Shan and Yongwei Wu and Mingxing Zhang}, + year = {2026}, + url = {https://arxiv.org/abs/2605.10670}, +} + @article{qin2025mooncake_tos, author = {Qin Ruoyu and Li Zheming and He Weiran and Cui Jialei and Tang Heyi and Ren Feng and Ma Teng and Cai Shangming and Zhang Yineng and Zhang Mingxing and Wu Yongwei and Zheng Weimin and Xu Xinran}, title = {Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving}, -- 2.34.1 From 0b2058040c69759e75d432b57458c63d441adb8e Mon Sep 17 00:00:00 2001 From: Chizheng Fang <93508110+fcczzz@users.noreply.github.com> Date: Mon, 25 May 2026 11:31:34 +0800 Subject: [PATCH 119/382] [Doc]: Update vLLM LMCache guide for MP interface (#2209) * [Doc]: Update vLLM LMCache guide for MP interface * [Doc]: Update LMCache configuration for global segment and local buffer sizes --- .../vllmv1-lmcache-integration.md | 270 +++++++++++------- 1 file changed, 166 insertions(+), 104 deletions(-) diff --git a/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md b/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md index 676841e8..3a9ee791 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md +++ b/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md @@ -2,118 +2,180 @@ ## Overview -The vLLM v1 version has been released with support for PD disaggregation. The detailed design document can be found [here](https://docs.google.com/document/d/1uPGdbEXksKXeN4Q9nUm9hzotqEjQhYmnpAhidLuAsjk). LMCache immediately implemented the corresponding connector to support storage, transmission, and loading of KVCache, enabling collaborative operation with PD nodes. Mooncake, as LMCache's backend storage engine, has undergone extensive optimizations in usability, performance, and stability. This document explains how to deploy a PD disaggregated serving demo using LMCache + Mooncake. +This guide shows a single-machine 1-prefill/1-decode deployment using vLLM V1, +LMCache's multiprocess server, and Mooncake Store as the LMCache L2 backend. + +In this setup, one machine runs Mooncake master, one LMCache MP server, the +disaggregated proxy, the prefiller vLLM instance, and the decoder vLLM +instance. The prefiller and decoder should use different GPUs. + +This example uses `"metadata_server":"P2PHANDSHAKE"` for Mooncake transfer +metadata, so the Mooncake HTTP metadata server is not needed. If you switch to +HTTP metadata, remember that Mooncake's HTTP metadata server also defaults to +`8080`, which conflicts with LMCache's HTTP API on a single host. + +## Prerequisites + +Install Mooncake, vLLM, and LMCache on the machine. The example assumes an RDMA +deployment and uses: + +- local host address: `{IP of Machine}` +- RDMA device: `{RDMA device}` +- LMCache checkout path: `/path/to/LMCache` + +Replace these values with the local hostname/IP, RDMA device, and LMCache checkout path for your environment. + +The `mooncake_store` MP L2 adapter requires LMCache's `lmcache_mooncake` C++ +extension. When building LMCache from source, enable Mooncake support, for +example: + +```bash +BUILD_MOONCAKE=1 \ +MOONCAKE_INCLUDE_DIR=/path/to/mooncake/include \ +MOONCAKE_LIB_DIR=/path/to/mooncake/lib \ +pip install -e /path/to/LMCache --verbose +``` ## Deployment -1. First, you need to prepare two GPU-equipped machines, which we will refer to as Machine A and Machine B. Install [vLLM](https://docs.vllm.ai/en/latest/getting_started/quickstart.html), [Mooncake](https://kvcache-ai.github.io/Mooncake/getting_started/build.html) and [LMCache](https://docs.lmcache.ai/getting_started/installation.html) on both Machine A and Machine B. For specific installation instructions, please refer to the official documentation of each repository. - -2. Start the Mooncake Master node on Machine A: -```bash -mooncake_master -port 50052 -max_threads 64 -metrics_port 9004 \ - --enable_http_metadata_server=true \ - --http_metadata_server_host=0.0.0.0 \ - --http_metadata_server_port=8080 -``` - -3. Launch the Decoder instance on machine A -- Modify the vllm/examples/others/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh file. -```diff -diff --git a/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh b/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh -index 831ef0bb5..a2ff0744c 100644 ---- a/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh -+++ b/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh - elif [[ $1 == "decoder" ]]; then - # Decoder listens on port 8200 -- decode_config_file=$SCRIPT_DIR/configs/lmcache-decoder-config.yaml -+ decode_config_file=$SCRIPT_DIR/configs/mooncake-decoder-config.yaml - - UCX_TLS=cuda_ipc,cuda_copy,tcp \ - LMCACHE_CONFIG_FILE=$decode_config_file \ - LMCACHE_USE_EXPERIMENTAL=True \ - VLLM_ENABLE_V1_MULTIPROCESSING=1 \ - VLLM_WORKER_MULTIPROC_METHOD=spawn \ - CUDA_VISIBLE_DEVICES=1 \ -``` -- Add the `mooncake-decoder-config.yaml` file -```yaml -chunk_size: 256 -remote_url: "mooncakestore://{IP of Machine A}:50052/" -remote_serde: "naive" -local_cpu: False -max_local_cpu_size: 100 - -extra_config: - local_hostname: "{IP of Machine A}" - metadata_server: "http://{IP of Machine A}:8080/metadata" - protocol: "rdma" - device_name: "mlx5_0" # Multiple RDMA devices can be specified as comma-separated list - master_server_address: "{IP of Machine A}:50052" - global_segment_size: 32212254720 # 30GB - local_buffer_size: 1073741824 # 1GB - transfer_timeout: 1 - save_chunk_meta: False -``` - -- Launch the Decoder instance using command -```bash -bash disagg_vllm_launcher.sh decoder Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 -``` - -4. Launch the Prefiller instance on machine B -- Modify the vllm/examples/others/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh file. -```diff -diff --git a/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh b/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh -index 831ef0bb5..9e5a3f044 100644 ---- a/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh -+++ b/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh -@@ -18,12 +18,14 @@ fi - - if [[ $1 == "prefiller" ]]; then - # Prefiller listens on port 8100 -- prefill_config_file=$SCRIPT_DIR/configs/lmcache-prefiller-config.yaml -+ prefill_config_file=$SCRIPT_DIR/configs/mooncake-prefiller-config.yaml -``` - -- Add the `mooncake-prefiller-config.yaml` file -```yaml -chunk_size: 256 -remote_url: "mooncakestore://{IP of Machine A}:50052/" -remote_serde: "naive" -local_cpu: False -max_local_cpu_size: 100 - -extra_config: - local_hostname: "{IP of Machine B}" - metadata_server: "http://{IP of Machine A}:8080/metadata" - protocol: "rdma" - device_name: "mlx5_0" # Multiple RDMA devices can be specified as comma-separated list - master_server_address: "{IP of Machine A}:50052" - global_segment_size: 32212254720 # 30GB - local_buffer_size: 1073741824 # 1GB - transfer_timeout: 1 - save_chunk_meta: False -``` - -- Launch the Prefiller instance using command -```bash -bash disagg_vllm_launcher.sh prefiller Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 -``` - -5. Prepare the router `disagg_proxy_server` - -We use the [disagg_proxy_server](https://github.com/LMCache/LMCache/blob/dev/examples/disagg_prefill/disagg_proxy_server.py) provided by LMCache. According to [LMCache/LMCache#1342](https://github.com/LMCache/LMCache/issues/1342), when using Mooncake Store as the backend, you need to comment out `wait_decode_kv_ready(req_id)` in the proxy code. - -6. Launch the `disagg_proxy_server` using command +### 1. Start Mooncake Master ```bash -python3 disagg_proxy_server.py --host localhost --port 9000 --prefiller-host IP_of_Machine_B --prefiller-port 8100 --decoder-host IP_of_Machine_A --decoder-port 8200 +mooncake_master -v=1 \ + --rpc_port=50051 \ + --metrics_port=9003 ``` -7. Now we can send the requests to the `disagg_proxy_server` to test PD disaggregated serving. +### 2. Start the LMCache Multiprocess Server + +Start one LMCache MP server and configure Mooncake Store as the L2 adapter. + +```bash +lmcache server \ + --host 127.0.0.1 \ + --port 5555 \ + --http-host 127.0.0.1 \ + --http-port 8080 \ + --l1-size-gb 32 \ + --eviction-policy LRU \ + --no-l1-use-lazy \ + --l2-adapter '{ + "type": "mooncake_store", + "local_hostname": "{IP of Machine}", + "metadata_server": "P2PHANDSHAKE", + "protocol": "rdma", + "rdma_devices": "{RDMA device}", + "global_segment_size": "32212254720", + "local_buffer_size": "1073741824", + "master_server_addr": "127.0.0.1:50051" + }' +``` + +### 3. Start the Disaggregated Proxy + +The proxy receives client requests, sends prefill requests to the prefiller, +sends decode requests to the decoder, and receives LMCache request telemetry +from the prefiller. + +```bash +python /path/to/LMCache/examples/disagg_prefill_mp/disagg_proxy_server.py \ + --host 127.0.0.1 \ + --port 8000 \ + --prefiller-host 127.0.0.1 \ + --prefiller-port 8100 \ + --decoder-host 127.0.0.1 \ + --decoder-port 8200 \ + --telemetry-port 5768 +``` + +### 4. Start the vLLM Prefiller + +The prefiller reports request telemetry back to the proxy so the proxy knows +when KV cache storage has completed. + +```bash +CUDA_VISIBLE_DEVICES=0 \ +LMCACHE_REQUEST_TELEMETRY_TYPE=fastapi \ +LMCACHE_REQUEST_TELEMETRY_ENDPOINT=http://127.0.0.1:5768/api/v1/telemetry \ +vllm serve Qwen/Qwen3-4B \ + --host 127.0.0.1 \ + --port 8100 \ + --gpu-memory-utilization 0.8 \ + --no-enable-log-requests \ + --no-enable-prefix-caching \ + --kv-transfer-config '{ + "kv_connector": "LMCacheMPConnector", + "kv_role": "kv_both", + "kv_connector_extra_config": { + "lmcache.mp.host": "tcp://127.0.0.1", + "lmcache.mp.port": 5555 + } + }' +``` + +### 5. Start the vLLM Decoder + +The decoder connects to the same local LMCache MP server. It does not need +request telemetry environment variables; only the prefiller reports the "KV +cache is stored" event back to the proxy. + +```bash +CUDA_VISIBLE_DEVICES=1 \ +vllm serve Qwen/Qwen3-4B \ + --host 127.0.0.1 \ + --port 8200 \ + --gpu-memory-utilization 0.8 \ + --no-enable-log-requests \ + --no-enable-prefix-caching \ + --kv-transfer-config '{ + "kv_connector": "LMCacheMPConnector", + "kv_role": "kv_both", + "kv_connector_extra_config": { + "lmcache.mp.host": "tcp://127.0.0.1", + "lmcache.mp.port": 5555 + } + }' +``` + +### 6. Send a Test Request + +Send traffic to the proxy, not directly to either vLLM instance. + +```bash +curl -N http://127.0.0.1:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Qwen/Qwen3-4B", + "messages": [ + { + "role": "user", + "content": "Explain how KV cache reuse helps long-context serving." + } + ], + "max_tokens": 128, + "temperature": 0.7 + }' +``` + +## Port and Configuration Checklist + +When changing ports away from these defaults, update all dependent settings +together: + +- Mooncake master `--rpc_port` must match LMCache `master_server_addr`. +- The vLLM prefiller and decoder should connect to the local LMCache MP server + via `kv_connector_extra_config.lmcache.mp.host` and + `kv_connector_extra_config.lmcache.mp.port`. +- Proxy `--prefiller-port` and `--decoder-port` must match the two vLLM + `--port` values. +- `LMCACHE_REQUEST_TELEMETRY_ENDPOINT` on the prefiller must point to the proxy + telemetry endpoint. +- If `metadata_server` is changed from `P2PHANDSHAKE` to an HTTP metadata URL, + enable Mooncake HTTP metadata server and make sure its port does not conflict + with LMCache `--http-port`. ## Additional Resources * [Mooncake x LMCache: Unite to Pioneer KVCache-Centric LLM Serving System](../../../getting_started/examples/lmcache-integration.md) -* [Using Mooncake in LMCache](https://docs.lmcache.ai/kv_cache/storage_backends/mooncake.html) -* [Using LMCache in vLLM](https://github.com/vllm-project/vllm/tree/main/examples/others/lmcache) \ No newline at end of file +* [LMCache MP `mooncake_store` L2 adapter](https://docs.lmcache.ai/mp/l2_storage.html#mooncake-store-mooncake-store-native-connector) +* [LMCache multiprocess disaggregated prefill example](https://github.com/LMCache/LMCache/tree/dev/examples/disagg_prefill_mp) -- 2.34.1 From 882336dbfdb9f964f27bf930c7b1e2fb9786604d Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Mon, 25 May 2026 11:40:07 +0800 Subject: [PATCH 120/382] [Store] fix gauge overflow by separating DFS unlimited flag (#2152) --- .../include/master_metric_manager.h | 6 ++++++ mooncake-store/src/master_metric_manager.cpp | 20 ++++++++++++++++++- mooncake-store/src/master_service.cpp | 8 ++++++-- 3 files changed, 31 insertions(+), 3 deletions(-) diff --git a/mooncake-store/include/master_metric_manager.h b/mooncake-store/include/master_metric_manager.h index adc8791e..34ad14e8 100644 --- a/mooncake-store/include/master_metric_manager.h +++ b/mooncake-store/include/master_metric_manager.h @@ -1,5 +1,6 @@ #pragma once +#include #include #include #include @@ -95,6 +96,10 @@ class MasterMetricManager { int64_t get_allocated_file_size(); int64_t get_total_file_capacity(); double get_global_file_used_ratio(void); + // When true, get_total_file_capacity() returns INT64_MAX to indicate + // unlimited DFS capacity instead of relying on the gauge value. + void set_dfs_capacity_unlimited(bool unlimited); + bool is_dfs_capacity_unlimited() const; // Key/Value Metrics void inc_key_count(int64_t val = 1); @@ -489,6 +494,7 @@ class MasterMetricManager { // File Storage Metrics ylt::metric::gauge_t file_allocated_size_; ylt::metric::gauge_t file_total_capacity_; + std::atomic dfs_capacity_unlimited_{false}; // Key/Value Metrics ylt::metric::gauge_t key_count_; diff --git a/mooncake-store/src/master_metric_manager.cpp b/mooncake-store/src/master_metric_manager.cpp index 3d5461f6..163b38e6 100644 --- a/mooncake-store/src/master_metric_manager.cpp +++ b/mooncake-store/src/master_metric_manager.cpp @@ -2,6 +2,7 @@ #include #include // For std::fixed, std::setprecision +#include // For std::numeric_limits #include // For string building during serialization #include // Required by histogram serialization #include @@ -696,15 +697,29 @@ void MasterMetricManager::dec_total_file_capacity(int64_t val) { file_total_capacity_.dec(val); } +void MasterMetricManager::set_dfs_capacity_unlimited(bool unlimited) { + dfs_capacity_unlimited_ = unlimited; +} + +bool MasterMetricManager::is_dfs_capacity_unlimited() const { + return dfs_capacity_unlimited_; +} + int64_t MasterMetricManager::get_allocated_file_size() { return file_allocated_size_.value(); } int64_t MasterMetricManager::get_total_file_capacity() { + if (dfs_capacity_unlimited_) { + return std::numeric_limits::max(); + } return file_total_capacity_.value(); } double MasterMetricManager::get_global_file_used_ratio(void) { + if (dfs_capacity_unlimited_) { + return 0.0; + } double allocated = file_allocated_size_.value(); double capacity = file_total_capacity_.value(); if (capacity == 0) { @@ -2007,6 +2022,9 @@ std::string MasterMetricManager::get_summary_string( ss << " (" << std::fixed << std::setprecision(1) << ((double)mem_allocated / (double)mem_capacity * 100.0) << "%)"; } + int64_t file_display_capacity = dfs_capacity_unlimited_ + ? std::numeric_limits::max() + : file_capacity; ss << " | NVMe-oF SSD: " << byte_size_to_string(nof_allocated) << " / " << byte_size_to_string(nof_capacity); if (nof_capacity > 0) { @@ -2014,7 +2032,7 @@ std::string MasterMetricManager::get_summary_string( << ((double)nof_allocated / (double)nof_capacity * 100.0) << "%)"; } ss << " | SSD Storage: " << byte_size_to_string(file_allocated) << " / " - << byte_size_to_string(file_capacity); + << byte_size_to_string(file_display_capacity); ss << " | Keys: " << keys << " (soft-pinned: " << soft_pin_keys << ")"; ss << " | Clients: " << active_clients; diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 00604a8c..ea39f4df 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -299,8 +299,12 @@ MasterService::MasterService(const MasterServiceConfig& config) if (!root_fs_dir_.empty()) { use_disk_replica_ = true; - MasterMetricManager::instance().inc_total_file_capacity( - global_file_segment_size_); + if (global_file_segment_size_ == std::numeric_limits::max()) { + MasterMetricManager::instance().set_dfs_capacity_unlimited(true); + } else { + MasterMetricManager::instance().inc_total_file_capacity( + global_file_segment_size_); + } } if (enable_snapshot_) { -- 2.34.1 From 6b885426dbecfc714a97a715fb5e73a9a9d26434 Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Mon, 25 May 2026 11:41:10 +0800 Subject: [PATCH 121/382] fix(file_storage): avoid INVALID_REPLICA error on empty offload heartbeat (#2151) Skip BatchQuerySegmentSlices when no objects need offloading. Fixes #2138. --- mooncake-store/src/file_storage.cpp | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index cf7bed9a..df9a45c2 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -340,6 +340,9 @@ tl::expected FileStorage::BatchGet( tl::expected FileStorage::OffloadObjects( const std::unordered_map& offloading_objects) { + if (offloading_objects.empty()) { + return {}; + } std::vector> buckets_keys; if (auto bucket_backend = std::dynamic_pointer_cast(storage_backend_)) { @@ -513,6 +516,9 @@ tl::expected FileStorage::Heartbeat() { } } + if (offloading_objects.empty()) { + return {}; + } // === STEP 2: Persist offloaded objects (trigger actual data migration) === auto offload_result = OffloadObjects(offloading_objects); if (!offload_result) { -- 2.34.1 From 788c1c737e0fb9ccea4c729334ef5a321759fda9 Mon Sep 17 00:00:00 2001 From: Colors-111 <70190328+Colors-111@users.noreply.github.com> Date: Mon, 25 May 2026 11:44:31 +0800 Subject: [PATCH 122/382] [Store] Fix: Auto-recovery for SSD Offload after Master Restart (#2077) Co-authored-by: ruanzhao --- mooncake-store/include/client_service.h | 2 + mooncake-store/include/file_storage.h | 9 ++ mooncake-store/include/storage_backend.h | 10 ++ mooncake-store/src/client_service.cpp | 72 ++++++++++ mooncake-store/src/file_storage.cpp | 123 +++++++++++++++++- mooncake-store/src/master_service.cpp | 14 ++ mooncake-store/src/real_client.cpp | 8 +- .../include/transfer_metadata.h | 3 + .../src/transfer_metadata.cpp | 30 +++++ 9 files changed, 264 insertions(+), 7 deletions(-) diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index cef371c4..c005b02e 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -804,6 +804,8 @@ class Client { std::thread task_poll_thread_; std::atomic task_poll_running_{false}; std::atomic last_ping_success_{false}; + std::atomic segment_desc_publish_pending_{false}; + std::atomic rpc_meta_publish_pending_{false}; ErrorCode SwitchLeader(const ha::MasterView& target_view); void LeaderMonitorThreadMain(); void StorageHeartbeatThreadMain(); diff --git a/mooncake-store/include/file_storage.h b/mooncake-store/include/file_storage.h index 37e24102..db56d313 100644 --- a/mooncake-store/include/file_storage.h +++ b/mooncake-store/include/file_storage.h @@ -118,6 +118,13 @@ class FileStorage { void ClientBufferGCThreadFunc(); + /** + * @brief Re-registers all offloaded objects with the master. + * Called after master restart recovery to sync SSD object metadata. + * This is the same logic as the ScanMeta step in Init(). + */ + tl::expected ReRegisterOffloadedObjects(); + std::shared_ptr client_; SsdMetric* ssd_metric_{nullptr}; std::string local_rpc_addr_; @@ -136,6 +143,8 @@ class FileStorage { std::thread heartbeat_thread_; std::atomic client_buffer_gc_running_; std::thread client_buffer_gc_thread_; + std::future rescan_future_; + std::atomic metadata_resync_pending_{false}; }; } // namespace mooncake diff --git a/mooncake-store/include/storage_backend.h b/mooncake-store/include/storage_backend.h index a686fd97..be297450 100644 --- a/mooncake-store/include/storage_backend.h +++ b/mooncake-store/include/storage_backend.h @@ -267,6 +267,11 @@ class StorageBackendInterface { const std::vector& keys, std::vector& metadatas)>& handler) = 0; + // Reset internal scan iterator so that the next ScanMeta() call + // starts from the beginning. Required for backends that use + // cursor-based iteration (e.g. BucketStorageBackend). + virtual void ResetScanIterator() {} + // Test-only: Set predicate to force failures for specific keys in // BatchOffload. Default implementation does nothing (no failures injected). // Concrete backends can override to provide test failure injection. @@ -781,6 +786,11 @@ class BucketStorageBackend : public StorageBackendInterface { const std::vector& keys, std::vector& metadatas)>& handler) override; + void ResetScanIterator() override { + MutexLocker locker(&iterator_mutex_); + next_bucket_ = -1; + } + /** * @brief Checks whether the backend is allowed to continue offloading. * @return tl::expected diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index ee2c1334..25fcde3f 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -3039,6 +3039,43 @@ void Client::StorageHeartbeatThreadMain() { ErrorCode err = remount_result.error(); LOG(ERROR) << "Failed to remount segments: " << err; } + // Re-publish Transfer Engine segment descriptors to the HTTP + // metadata server. When Master (which hosts the HTTP metadata + // server in the same process) is killed and restarted, all + // in-memory KV entries are lost. ReMountSegment above only + // restores Master-side allocation state; it does NOT write back + // the transport-level segment descriptors. Without this, remote + // peers get HTTP 404 when querying our segment descriptor and + // data transfers fail. + auto metadata = transfer_engine_->getMetadata(); + if (metadata) { + int rc = metadata->updateLocalSegmentDesc(); + if (rc != 0) { + LOG(ERROR) << "Failed to re-publish segment descriptor " + << "to metadata server, rc=" << rc + << ", will retry in next heartbeat cycle"; + segment_desc_publish_pending_.store(true); + } else { + segment_desc_publish_pending_.store(false); + } + // Also re-publish RPC meta entry (mooncake/rpc_meta/). + // Remote peers need this to locate our RDMA RPC port for + // handshake. Like segment descriptors, this entry is lost + // when the HTTP metadata server is cleared on Master restart. + rc = metadata->rePublishRpcMetaEntry(local_hostname_); + if (rc != 0) { + LOG(ERROR) << "Failed to re-publish RPC meta entry " + << "to metadata server, rc=" << rc + << ", will retry in next heartbeat cycle"; + rpc_meta_publish_pending_.store(true); + } else { + rpc_meta_publish_pending_.store(false); + } + } + // Note: LOCAL_DISK segment remount is NOT done here. + // It is handled by FileStorage::Heartbeat() when it detects + // SEGMENT_NOT_FOUND, which also triggers ScanMeta to + // re-register offloaded object metadata. }; // Use another thread to remount segments to avoid blocking the ping // thread @@ -3064,6 +3101,41 @@ void Client::StorageHeartbeatThreadMain() { // Ensure at most one remount segment thread is running remount_segment_future = std::async(std::launch::async, remount_segment); + } else if (segment_desc_publish_pending_.load() && + !remount_segment_future.valid()) { + // Previous remount succeeded but updateLocalSegmentDesc() + // failed (e.g. transient HTTP error). Retry it directly + // without re-running ReMountSegment. + auto metadata = transfer_engine_->getMetadata(); + if (metadata) { + int rc = metadata->updateLocalSegmentDesc(); + if (rc != 0) { + LOG(ERROR) + << "Retry: failed to re-publish segment " + << "descriptor to metadata server, rc=" << rc; + } else { + LOG(INFO) << "Retry: successfully re-published " + << "segment descriptor to metadata server"; + segment_desc_publish_pending_.store(false); + } + } + } else if (rpc_meta_publish_pending_.load() && + !remount_segment_future.valid()) { + // Previous remount succeeded but rePublishRpcMetaEntry() + // failed. Retry it directly. + auto metadata = transfer_engine_->getMetadata(); + if (metadata) { + int rc = metadata->rePublishRpcMetaEntry(local_hostname_); + if (rc != 0) { + LOG(ERROR) + << "Retry: failed to re-publish RPC " + << "meta entry to metadata server, rc=" << rc; + } else { + LOG(INFO) << "Retry: successfully re-published " + << "RPC meta entry to metadata server"; + rpc_meta_publish_pending_.store(false); + } + } } std::this_thread::sleep_for( diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index df9a45c2..46092e9b 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -501,6 +501,26 @@ tl::expected FileStorage::Heartbeat() { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } + // Join previous rescan if completed + if (rescan_future_.valid() && rescan_future_.wait_for(std::chrono::seconds( + 0)) == std::future_status::ready) { + rescan_future_ = std::future(); + } + + // Retry metadata resync if previous attempt failed. + if (metadata_resync_pending_.load() && !rescan_future_.valid()) { + LOG(INFO) << "Retrying background metadata rescan"; + rescan_future_ = std::async(std::launch::async, [this]() { + auto result = ReRegisterOffloadedObjects(); + if (!result) { + LOG(ERROR) << "Background metadata rescan retry " + << "failed: " << result.error(); + } else { + metadata_resync_pending_.store(false); + } + }); + } + std::unordered_map offloading_objects; // Objects selected for offloading @@ -510,9 +530,52 @@ tl::expected FileStorage::Heartbeat() { auto heartbeat_result = client_->OffloadObjectHeartbeat( enable_offloading_, offloading_objects); if (!heartbeat_result) { - LOG(ERROR) << "Failed to send heartbeat with error: " - << heartbeat_result.error(); - return heartbeat_result; + ErrorCode err = heartbeat_result.error(); + if (err == ErrorCode::SEGMENT_NOT_FOUND) { + // Master lost our LOCAL_DISK segment (likely restarted). + // Re-register the segment, retry the heartbeat, and + // trigger async ScanMeta to re-register object metadata. + LOG(WARNING) << "OffloadObjectHeartbeat returned " + << "SEGMENT_NOT_FOUND, attempting to " + << "re-register local disk segment and " + << "re-register object metadata"; + auto remount_result = + client_->MountLocalDiskSegment(enable_offloading_); + if (remount_result) { + heartbeat_result = client_->OffloadObjectHeartbeat( + enable_offloading_, offloading_objects); + if (!heartbeat_result) { + LOG(ERROR) << "Heartbeat failed after re-registration: " + << heartbeat_result.error(); + return heartbeat_result; + } + // Master lost all object metadata on restart. + // Trigger async ScanMeta to re-register them, + // same as what Init() does on startup. + if (!rescan_future_.valid()) { + LOG(INFO) << "Triggering background metadata rescan " + << "after LOCAL_DISK segment re-registration"; + metadata_resync_pending_.store(true); + rescan_future_ = + std::async(std::launch::async, [this]() { + auto result = ReRegisterOffloadedObjects(); + if (!result) { + LOG(ERROR) << "Background metadata rescan " + << "failed: " << result.error(); + } else { + metadata_resync_pending_.store(false); + } + }); + } + } else { + LOG(ERROR) << "Failed to re-register local disk segment: " + << remount_result.error(); + return tl::make_unexpected(remount_result.error()); + } + } else { + LOG(ERROR) << "Failed to send heartbeat with error: " << err; + return heartbeat_result; + } } } @@ -863,4 +926,58 @@ bool FileStorage::ReleaseBuffer(uint64_t batch_id) { return false; } +tl::expected FileStorage::ReRegisterOffloadedObjects() { + LOG(INFO) << "ReRegisterOffloadedObjects: starting ScanMeta to re-register " + << "offloaded objects with master"; + int total_keys = 0; + int total_batches = 0; + int total_failures = 0; + // Reset the scan iterator so ScanMeta starts from the beginning. + // BucketStorageBackend uses cursor-based iteration (next_bucket_); + // after Init() completes the cursor is 0 and HasNext() returns false, + // which would make ScanMeta skip all buckets. + storage_backend_->ResetScanIterator(); + LOG(INFO) << "ReRegisterOffloadedObjects: about to call " + "storage_backend_->ScanMeta()"; + auto scan_meta_result = + storage_backend_->ScanMeta( + [this, &total_keys, &total_batches, &total_failures]( + const std::vector& keys, + std::vector& metadatas) { + total_batches++; + total_keys += keys.size(); + for (auto& metadata : metadatas) { + metadata.transport_endpoint = local_rpc_addr_; + } + auto add_object_result = + client_->NotifyOffloadSuccess(keys, metadatas); + if (!add_object_result) { + total_failures++; + LOG(ERROR) + << "ReRegisterOffloadedObjects: NotifyOffloadSuccess " + << "failed for batch " << total_batches << " with " + << keys.size() + << " keys, error: " << add_object_result.error(); + return add_object_result.error(); + } + LOG(INFO) << "ReRegisterOffloadedObjects: NotifyOffloadSuccess " + << "succeeded for batch " << total_batches << " with " + << keys.size() << " keys"; + return ErrorCode::OK; + }); + + LOG(INFO) << "ReRegisterOffloadedObjects: ScanMeta returned. success=" + << scan_meta_result.has_value(); + if (!scan_meta_result) { + LOG(ERROR) << "ReRegisterOffloadedObjects: ScanMeta failed: " + << scan_meta_result.error(); + return scan_meta_result; + } + LOG(INFO) << "ReRegisterOffloadedObjects: completed. " + << "total_keys=" << total_keys + << " total_batches=" << total_batches + << " total_failures=" << total_failures; + return {}; +} + } // namespace mooncake diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index ea39f4df..00f2d4a6 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -2617,6 +2617,20 @@ auto MasterService::MountLocalDiskSegment(const UUID& client_id, } else if (err != ErrorCode::OK) { return tl::make_unexpected(err); } + + // Notify the client monitor thread to start tracking this client's TTL. + // Without this, a client that only mounts a LOCAL_DISK segment (and + // doesn't ping) would be considered expired by ClientMonitorFunc, which + // would then clear all its LOCAL_DISK replicas. + PodUUID pod_client_id; + pod_client_id.first = client_id.first; + pod_client_id.second = client_id.second; + if (!client_ping_queue_.push(pod_client_id)) { + LOG(ERROR) << "client_id=" << client_id + << ", error=client_ping_queue_full"; + return tl::make_unexpected(ErrorCode::INTERNAL_ERROR); + } + return {}; } diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 74cdd0a7..9ea43f47 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -4557,10 +4557,10 @@ RealClient::batch_get_into_internal(const std::vector &keys, std::chrono::duration_cast( end_time - start_read_store_time) .count(); - LOG(INFO) << "Time taken for batch_get_into: " << elapsed_time - << "us, read store: " << read_store_time - << "us, with memory key count: " << valid_operations.size() - << ", offload key count: " << offload_object_count; + // LOG(INFO) << "Time taken for batch_get_into: " << elapsed_time + // << "us, read store: " << read_store_time + // << "us, with memory key count: " << valid_operations.size() + // << ", offload key count: " << offload_object_count; return results; } diff --git a/mooncake-transfer-engine/include/transfer_metadata.h b/mooncake-transfer-engine/include/transfer_metadata.h index 802cf7e3..863959fb 100644 --- a/mooncake-transfer-engine/include/transfer_metadata.h +++ b/mooncake-transfer-engine/include/transfer_metadata.h @@ -178,6 +178,9 @@ class TransferMetadata { int removeRpcMetaEntry(const std::string &server_name); + // Re-publish the local RPC meta entry to the HTTP metadata server. + int rePublishRpcMetaEntry(const std::string &server_name); + int getRpcMetaEntry(const std::string &server_name, RpcMetaDesc &desc); int getNotifies(std::vector ¬ifies); diff --git a/mooncake-transfer-engine/src/transfer_metadata.cpp b/mooncake-transfer-engine/src/transfer_metadata.cpp index 355114b6..40a25856 100644 --- a/mooncake-transfer-engine/src/transfer_metadata.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata.cpp @@ -1147,6 +1147,36 @@ int TransferMetadata::removeRpcMetaEntry(const std::string &server_name) { return 0; } +int TransferMetadata::rePublishRpcMetaEntry(const std::string &server_name) { + if (p2p_handshake_mode_) { + return 0; + } + const std::string full_key = rpc_meta_prefix_ + server_name; + + Json::Value existing; + if (storage_plugin_->get(full_key, existing)) { + Json::Value desired; + desired["ip_or_host_name"] = local_rpc_meta_.ip_or_host_name; + desired["rpc_port"] = + static_cast(local_rpc_meta_.rpc_port); + if (existing == desired) { + return 0; + } + storage_plugin_->remove(full_key); + } + + LOG(INFO) << "Re-publishing RPC meta entry for " << server_name; + Json::Value rpcMetaJSON; + rpcMetaJSON["ip_or_host_name"] = local_rpc_meta_.ip_or_host_name; + rpcMetaJSON["rpc_port"] = + static_cast(local_rpc_meta_.rpc_port); + if (!storage_plugin_->set(full_key, rpcMetaJSON)) { + LOG(ERROR) << "Failed to re-publish RPC meta entry for " << server_name; + return ERR_METADATA; + } + return 0; +} + int TransferMetadata::getRpcMetaEntry(const std::string &server_name, RpcMetaDesc &desc) { { -- 2.34.1 From 4a17b1d418e50a91e9654405080a4de13eb3542c Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Mon, 25 May 2026 11:44:54 +0800 Subject: [PATCH 123/382] [Store] Fix: validate HA backend availability during config parsing (#2111) --- mooncake-store/include/ha/ha_types.h | 23 ++++++++ mooncake-store/src/client_service.cpp | 5 ++ .../leadership/master_service_supervisor.cpp | 5 ++ mooncake-store/tests/CMakeLists.txt | 2 + .../ha_backend_availability_test.cpp | 53 +++++++++++++++++++ 5 files changed, 88 insertions(+) create mode 100644 mooncake-store/tests/ha/leadership/ha_backend_availability_test.cpp diff --git a/mooncake-store/include/ha/ha_types.h b/mooncake-store/include/ha/ha_types.h index e0cedf2e..1d85bc75 100644 --- a/mooncake-store/include/ha/ha_types.h +++ b/mooncake-store/include/ha/ha_types.h @@ -52,6 +52,29 @@ inline std::optional ParseHABackendType(std::string_view type) { return std::nullopt; } +inline ErrorCode ValidateHABackendAvailability(HABackendType type) { + switch (type) { + case HABackendType::UNKNOWN: + return ErrorCode::INVALID_PARAMS; + case HABackendType::ETCD: +#ifdef STORE_USE_ETCD + return ErrorCode::OK; +#else + return ErrorCode::UNAVAILABLE_IN_CURRENT_MODE; +#endif + case HABackendType::REDIS: +#ifdef STORE_USE_REDIS + return ErrorCode::OK; +#else + return ErrorCode::UNAVAILABLE_IN_CURRENT_MODE; +#endif + case HABackendType::K8S: + return ErrorCode::UNAVAILABLE_IN_CURRENT_MODE; + } + + return ErrorCode::INVALID_PARAMS; +} + struct HABackendSpec { HABackendType type = HABackendType::UNKNOWN; std::string connstring; diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 25fcde3f..680d7c66 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -241,6 +241,11 @@ tl::expected, ErrorCode> ParseHABackendSpec( return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } + auto availability = ha::ValidateHABackendAvailability(backend_type.value()); + if (availability != ErrorCode::OK) { + return tl::make_unexpected(availability); + } + return std::optional{ha::HABackendSpec{ .type = backend_type.value(), .connstring = master_server_entry.substr(delimiter_pos + 3), diff --git a/mooncake-store/src/ha/leadership/master_service_supervisor.cpp b/mooncake-store/src/ha/leadership/master_service_supervisor.cpp index ae7a48b6..4f9ff543 100644 --- a/mooncake-store/src/ha/leadership/master_service_supervisor.cpp +++ b/mooncake-store/src/ha/leadership/master_service_supervisor.cpp @@ -38,6 +38,11 @@ tl::expected BuildHABackendSpec( return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } + auto availability = ValidateHABackendAvailability(backend_type.value()); + if (availability != ErrorCode::OK) { + return tl::make_unexpected(availability); + } + auto connstring = ResolveHABackendConnstring(config); if (connstring.empty()) { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); diff --git a/mooncake-store/tests/CMakeLists.txt b/mooncake-store/tests/CMakeLists.txt index df1374a5..07b7fc86 100644 --- a/mooncake-store/tests/CMakeLists.txt +++ b/mooncake-store/tests/CMakeLists.txt @@ -147,6 +147,8 @@ endif() if(STORE_USE_ETCD OR STORE_USE_REDIS OR STORE_USE_K8S_LEASE) add_test(NAME high_availability_test COMMAND high_availability_test) endif() +add_ha_test(ha_backend_availability_test + ha/leadership/ha_backend_availability_test.cpp) add_executable(stress_workload_test stress_workload_test.cpp) target_link_libraries( diff --git a/mooncake-store/tests/ha/leadership/ha_backend_availability_test.cpp b/mooncake-store/tests/ha/leadership/ha_backend_availability_test.cpp new file mode 100644 index 00000000..047ae862 --- /dev/null +++ b/mooncake-store/tests/ha/leadership/ha_backend_availability_test.cpp @@ -0,0 +1,53 @@ +#include "ha/ha_types.h" + +#include +#include + +namespace mooncake { + +tl::expected, ErrorCode> ParseHABackendSpec( + const std::string& master_server_entry); + +namespace ha { +namespace { + +TEST(HABackendAvailabilityTest, UnknownBackendIsInvalid) { + EXPECT_EQ(ErrorCode::INVALID_PARAMS, + ValidateHABackendAvailability(HABackendType::UNKNOWN)); +} + +TEST(HABackendAvailabilityTest, EtcdAvailabilityMatchesBuildFlag) { +#ifdef STORE_USE_ETCD + EXPECT_EQ(ErrorCode::OK, + ValidateHABackendAvailability(HABackendType::ETCD)); +#else + EXPECT_EQ(ErrorCode::UNAVAILABLE_IN_CURRENT_MODE, + ValidateHABackendAvailability(HABackendType::ETCD)); +#endif +} + +TEST(HABackendAvailabilityTest, RedisAvailabilityMatchesBuildFlag) { +#ifdef STORE_USE_REDIS + EXPECT_EQ(ErrorCode::OK, + ValidateHABackendAvailability(HABackendType::REDIS)); +#else + EXPECT_EQ(ErrorCode::UNAVAILABLE_IN_CURRENT_MODE, + ValidateHABackendAvailability(HABackendType::REDIS)); +#endif +} + +TEST(HABackendAvailabilityTest, K8sLeaseIsRejectedUntilCoordinatorExists) { + EXPECT_EQ(ErrorCode::UNAVAILABLE_IN_CURRENT_MODE, + ValidateHABackendAvailability(HABackendType::K8S)); +} + +TEST(HABackendAvailabilityTest, + ClientSpecParsingRejectsUnavailableBackendBeforeCoordinatorCreation) { + auto spec = ParseHABackendSpec("k8s://default/master"); + ASSERT_FALSE(spec.has_value()); + EXPECT_EQ(ErrorCode::UNAVAILABLE_IN_CURRENT_MODE, spec.error()); +} + +} // namespace +} // namespace ha +} // namespace mooncake -- 2.34.1 From db5e7f9c53839e0b3d0518f87bc6b33883312a96 Mon Sep 17 00:00:00 2001 From: ykwd Date: Mon, 25 May 2026 14:02:41 +0800 Subject: [PATCH 124/382] [Doc] Add missing aws-logo (#2211) Co-authored-by: Ke Yang --- README.md | 4 ++-- image/partners/aws-logo.png | Bin 0 -> 45061 bytes 2 files changed, 2 insertions(+), 2 deletions(-) create mode 100755 image/partners/aws-logo.png diff --git a/README.md b/README.md index cf3eaca1..925d9e33 100644 --- a/README.md +++ b/README.md @@ -110,7 +110,7 @@ Mooncake supports heterogeneous accelerators, NICs, and specialized transport pa NVIDIA
NVIDIA AMD
AMD Alibaba Cloud
Alibaba Cloud - AWS
AWS + AWS
AWS @@ -381,7 +381,7 @@ Please kindly cite our papers if you find the papers or the traces are useful: ```bibtex @article{sun2026survivingpartialrankfailures, - title = {Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference}, + title = {Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference}, author = {Xun Sun and Shaoyuan Chen and Pingchuan Ma and Yue Chen and Ziwei Yuan and Zhanhao Cao and Han Han and Shangming Cai and Teng Ma and Xuchun Shang and Xinpeng Zhao and Ke Yang and Junlin Wei and Lianzhi Lin and Yuji Liu and Feng Ren and Haoran Hu and Cheng Wan and Yingdi Shan and Yongwei Wu and Mingxing Zhang}, year = {2026}, url = {https://arxiv.org/abs/2605.10670}, diff --git a/image/partners/aws-logo.png b/image/partners/aws-logo.png new file mode 100755 index 0000000000000000000000000000000000000000..bf4d497d908c4a4f4bd57de7d777aae331b1cd81 GIT binary patch literal 45061 zcmZsCby$?&^Y*h#x0EzU2uOD~2q-PxCDI`w4GW5dG$J69N_RIb9a2gov2;p@^zYfv z_x-~a%X#zY6-X|$VS{V+H`&2LQMR--2xefENz{>{+7&47^Z~)$y6%U-S(yner7rTK2H5 zHC)0&kA`sbC(70i!I;#3{}`EP%Ck?;Q>!^*h9v#4ug{N6 zx8hOhvg%`;dy`Z*+x11jjHl0n>M;FBn&{X~c#lgo*~oULNAU7kzq6J2wB(rN*23w6 z|6^Fh|L+yY^^3RCrTbnb}>eic$~WlG~Uv zLzxeU*sn=N&gA62yX#1tb)}-P{pQZ}hOfUJ@Ok5H?>d-?M@73Z5iHSE!LY;%@D%4kb!aa!Sg(7M7=i3!^x_*d zcZ27#P(PWRaD|P+i2aaVkKG*?!JNfD{uaDeQ{(wpH$ExI|?$@5n=2&zZu&4w>p$m0b9)9G zj2?dW6Kbcm&89K_8kL>DOs$3dF=*7i_be^)RL;^jskogGE(cgsBqv=&%h3|1ZtSjf zsya>(<6;q8u)$s&|BFay(jucu_0wu2V!(3?#y)lXgNoofZnb%O*Yev$LsdaRp}weG zO;r&7N2gL5<~vOAaX}rTC$kRW^$XTl^5d89LQPfg&3&GuJX5an{KOSl#{;-9G`rxq z>35Hs%t^h!x(#Ne8_~{=#e;61KsQpTqCV(&j{>_)flc!;^bWUsGV0LkBw%Px-)s#t z#!*{nrUDa=sd_09ev0KJzl4EC=(jJc`hkyb4MU`}OI<0%>sKTC*PddWe0R_pQHY49 zXjh-2CVaotx+`^DOP1M?G_ipNjAl*ol4neG+Cq++l^YbDoWzpNM?E9LGtL zekAwLEf{m`Bvr((Xm|Ho%ZSZ(Gvw`&-1p~RB&}p)CY+DC=kQ+E)+3H z@7Wt4KS3YO`A$#LCi0d(ItY)=wRHjscT$y{;l`o9C;mV#+skEZNT>%|~cROPcztdlYO@}VVZP%=a zKg9EAM$@-nY#(NyxL}Sp*zp{dLZzjsrEVn;d=3krf0EOc6qhb7rt60HrPeo^66m*w z_m6!0<|AR|{zUwQTjwkZI?2zVH;-jeW$6wi4cn!?Tf!+`=7RVy;<;QW-L#QQW{aB8brsjr1vOH z>MWi8I1h_67!pc;{x}(UfL=dVit72CNV+vx62XFPd^C|Qk0-1GgpzMvp z_EXL(`IrE~i7G3K_s7?7h7CmM&A%*S>3_jwt$!IwcIfg#vcJVK%g*VZ8@O}qkMdH` zrQ)YibMy)4R1b1Y;+Nzdsr1+Dk`aD(J)&wnA$?TdzVP|k;K^^^Ypk#!3OGfya&}^4 zo8sA*fbz4=kv-al&IlgJg+{+l3bk?WmACxH`$xkz3fo?k>uhT~`g~<-?Bp>sdb%l% zilp8xI?SY3hWXCD{XX8DDnWaj9gyNnQA3g5>wU0XO8;Om5AP6~{l2<~cBGnW^PyZ}gdeuh zwVtBx)Een)#iUE>*ecA|a?GH(F1Y&RKyv^2dG}OJQp(Z6JR@yfip=>NTO8KLN>9)G zn%0ST#l`6lTDFPHAs9?at+KxiBNb}>Z9hfbejenuwBY_srJ(Bl!{uJMptpj$JDvx^ z_0|5Eyc%V9eJyiQN8EA7zKOb`teqL@S%jca#0Nn6D3r zk-Gl-{ed`W>WZf}@f8Dx_C-=_!aw&?ebrjgiP%l) zZo0?&)!gGxfzJ#_69nCzH0G%b*2Z-f`uZE-)?G zSl{q^UB6fLpiJx%X(@idI*N}PcKJqErm~O_-areFL9DFmkS9HV{X*c&KWklyK|C&? zaJy#o-H_R~t53M|_QTc~-VpwMZeynR0>h`rVNb3pFjVT1?7 zZz8d6EtP*6>8AW+{}_sOWFLpdP#4+o+&WUJSJ~l+{Bd7G)C$c&THUYaK4A>WN1_#z zD~#{Y76X1+cD{$F0?r2cRm#pM?n_?c3N4v!_;7)(dFsliZfhgWeI4f~`7}|ejzSr^ zFLiJAbxV)+lU9!jaWRMyNe25cog}ACu0RC99i|&KK~>#w=qQ#)Ok1 zAiTI|f=R|ct}v+?suP#`0w!;06PE>rbc&^cM@4JLyBZS%y3fWUZSR(PiVfBzb58;R zRUNe3Ys%4dvSMpst7>;eQ}>s! zwWHJ;VS_Vn!;w438#9w~c%y}vnETVi)`tw~5=-iA{U!q;lpWkI$#`O^Cqg9_pVJvV` zZmMUrA)tD1v+#SK273WiCVwZ>aK=^_FB@Ks`HRC|k^TIVGX7nCxfKk+zYJEhrc$L* zM8fORuz#qZR5O8Bga?wJl13627tu!~tuqx4dBMl9ZXAjQT_7%VybVA^(ehokn6y^D zS?*Vd|5e|$VpqH`DR6E>^oz43aLr_m{ z)xu0VDG}xgq8^GU28d0Vsa})ca&8@-hk)%g;Dh4GOZuyz*kMbU55lbAK=mM}s}CU3 z3}3Hdu4wC4^&BFjFO;7)Ttpnt2=>##A(7Fr;_fNa({s#iif_5|a7x75&4h6x@aSEw zbn95JyKXO0FxK%aVAL8fKi7LF^PV+L_2YX3A7(f=Q%7W6!~h+j!$bM{t>Ql}9olvl zp2(!)pA*?C*IieB(+n=LeARvj7J+W~!FuzY%SEsKFO{G~UCug;N`cG3=!WSK}ZBL5+u=P?a zaqzXseA}ro;PSfcX;w7<(&n{60@e}lhyHbGd<(vGph=NL|1zBAcJ%6;CRAF~r1=wu za6LWUw)`PA>a9%?5L1dJHB4A@_2av>*hot1A5&IkH8r)J3ICBpNT9GI$Ca|LxsMzk zF#;|*+}2^W$P(TLLYu4f43}=j1J6bu%N=%`t`%Zm=3s!w|42pTs)+84dhw3JZgKK= z>(qsu0)u|adHC2rMwoYHed2>VeUUYk`iZtJmjuqZkzSzt*P;IJb*IvQ9gdqzSjt1K z8oJ~1nb@l|4nDvpf!wom~f~6nrsN^D~*Q&2R-+XNqPLu!*g4S4vSYclzmm$ z1QoUy-|^wBNMnIBcl3L^D%EOK)A=*=CNO2wwGN;$v+nb41}+~2pM*56%RvLeXC%LQ z$weDSvdbTgKEAabh?D%0sdHDn_2lc;u{3w{!qvz*2ZNi(U4=}k)J;RN%YDDX2777F z`b_{PWn&->G!s~p(DTPO;MO;O`(yp4oV29GaYGh(W{c6Nm{{7{ElGa)Z|d~lht|D~x6$1$jFfvkKZzRy3UHHn9d zJk=#0+92iHH(aS)@i^4OyWW?>B93`6DN^6LM4v}<$im}P89_7V8>LW^;ZBJlq9Qdh zuX``4aB)cCcVQ3CQ^{t4s}$_ox2D(60O#_HsjvJd7!ppMw$D6~>T6&d}s zGDkoE@vxYjlRa~*_y4PQ@Mr0_dr!V&Cb%%{BmU`^|9w8#)(7(;ojq)>vECOCRzRcH z&E>vh6UK>z&#XX)CoM`xG=cZ6Bujy`zt!Bq7?^0gq8^}QSS(ghRfXy3Tqy$(^b99+ z5W2k+wj;!1F|_=27mq85zwYN5-Pgyx)=e4ALX41*TUrz{b*pvkwt&%bf%8=D%NJEJaJgi#au1HRbfWo*^*xEI)!V}QS8YMbwe3#>{F z!4xzYk!GGGV>7-!YN(0CR{9?7lN5FM3H#PD zI-yxG3QU@hs8=wso8N2KckP>_YpA%>ZhVZ7G>Qg^C`CBLvsgKt+_WH<3b9f|cooGW zE_NU;u|Otl^Rl|t$@3z#^#j+D&X|4z4ys>>Ts{y1>tEr8Fc2HM z3N_6OoTJIeD7hrrlS5{rn;vnTOi^O1QL^lpPo}WT&^6I4a9w9K@y?FRd?fCJIXO8E z(|=aks1yrRsw@y7{^Bp??e5NO*+Nwh-bfg)41U)v)&Iq%QCDCtw}@nMZrvj@GK%B@e8?;&VnzS{3JEN* zRq81-&6wa4VUsASed+!$^AE=Kf*!%g{?aAwWoY`gd=ehCW5q&r7@Fd68%O=L{P#`b z-ueR;{K)gI0mAH_{V&!0D`Tg|-$-QrWahznx<~&4pKIk(P2u_2ze;JRsflrPjC@MMNPazq9BtDqY|0=@S;w#3(;{&6lls*;ZWIp{N) ze}Q;(H+Dd0tUT-Uj9XiwP=s<@@pKiVSm#POf_?DyNlK9po9(&0fm;LX3D=T0O)B~b zY9ni>km`Y1$5>$rh}5YA3|~Cg1as&-fxTx1*Idnw2kW2UB6{(3 zar4_w>@-j1wO;vPVt7DF*os%|iWlAokE0WZs!!r~j1D!T+b5yIXpl9eP2*GF(ad^K2*NqVTZ++alS zDvEaic8w{biE_9}U2Vij-8)xmygsM$zl z@HOA;H6|7N-!xmMf(w3-PUW@cZ!E7Y7Vl+_WO zvMN6baaFah4waqV`C4!LpKp98pY{5RuD~EU*8UWwf|b>zFn9zXd_jV>juBjms_D6e#EgTMqcZWsG^(~7=rs@wmUw-Q4A-@nw!YH z|D^gSDvRezQZOxcvL0DcZ?6oULKX@&g9L+vcydpk{rNdw!`wzI6PIKfawPIYj^E_L zpN}gJq^fY`SJX1a8=&z`2X(?#t6`3#Nv^0)`wIlU1WfzSo5jDj_pWHg>g0hSvvfqZ|ca%nsBM0@qbt^#4R>XR%F9yImpjmOh)^-RxDHiAvXVe}(bc`XA* zzn{CL`NLIKe??ut2{@l!Jy(1qY{sNmfv3qbOE|Q4hXgFu8>IaRTA^i(qr2VtYX`=h zOO$_@*n#(z7J=hdBi!(Nk4I4tj|arrYHy=6FV52a=F{)#l9s#n_V@P>?KQZCHUg($ z_f#;jPJKY5-|K*OcW1;dWj&j!i!5@gL!-t&^2thac8b;~B%eYnPkw>sJ?C};Bflas zN%y~}oWdOso{&#Y;k}~id);Vlqp#tA?U>iJsI71=iCxNJuy}Fe82WhGA{e4qsL@>g zUXz1ZBjxN6=g!`(N&+vbbMdBKq)q5*FTUU*ulN}DaWzm-aImA{$|q^iT8=$y$?*ap zKb#*K#3=_V(>UL4k3U{v5M!rpl5yjo(%Pm@ylmEgL_PzPV%g*>i)ruuO9J?7TK%up-!Iexhs< zB)z_DRwvP4htbcqD87FC$eLms)gUMQ(=FpE^ezcF96h)k-L==~ke-?b4ki=L<;% zQ%`f9GSGHvnljHncYU#4z!tTjGPppHpnbW%qb#vy;U>%;S}CqGl=OI>&sY~zHp~Tv z|CAlO>t$8{=FXpvxS@wopX;9aOeu<2tcrOVIR8aIgk))Z0}g@v3(% z<+O|WQ+e6LNcZUjE{))|S$N4*(D@lB$ZMK9S-#sdnwk7H@#Q{1W6F?>d;m&%#|#UH zw7S1Ugy^N|#Z{1rNvjpWqFH;wO_Wf*Fx#yW``$x{RF3eNO*(={O};ujCu_aJAO9pTjV58B^jzh&dcd$sH`(mZb7a4j$|~m|k%J z#AiHc*EnwZjOVN{PWdB^zbI}s4AT4)nJ(K0$`4V472W5X2}<~d1+LiRp4-huHDo3h z9*}CY$fP}(jRg%)O2iV?LJ89EpJ-|z5@NxynXP?b5$5haSP|23M43~3yZ(?J*Qyb; zm8-_(N)c!EzzSMQ-`n3r!Ojc8_aj=!yeEfIL7c?;=%n!w4Giw@p9BPX`!`EXES=q{ zagG>Mi0;}uHXDZ=Vq_Oap4tjV*Z5g*xfdj5G@gk3$gyJUN?}C!TjN5Q;qn}>(!cE^ zxO!CXf@@uScwxP|HI(_CI2wro_ptSsSxc2G zor|~xG#V})ms5$nn=!x}F_Ym$z63Pj-p&!y{6|j$B-HA=tIkcZnK1eU%(#)|8iJehdZjYw8oEdj@U5<9n*!q8VbI) zObdGPT;&lX@vXjs6mEZk98S;0GaqaO7l!Ld8vmQ8Z>pc-cV>Uv3l5jVY|HbY_?WZr zpOJolJK0w^;>ZzkOTERGWYr74;B?aMK~0TNH}?J$)eqbbZ}71p;QE+r>tcx3e*5tP zc97V@bQ5LdCBdQstF!s`8IcP#gOsi)3ie$yD92`tla_)iEb^mjVZ}N-bl=NI$KUdN z->;;AfQ}|1_z~O^!xdk+OH2);nv@KMyi*w$EfYgRC*S;AH0C-qWr)da;PQr-DQU10 z>J_V7U@G}{&OG zo>qVT`t?=c%2`;naX%earT1rVx3J}k)1AOntbVhIB81ryt^u-IlrLlA7Dk!f*{_@q z5lmD>th$V^F;=0*9g1dj(pz-bni|J)Qv4zlJGGiFrY}s1VyAYry{xUReSJrF+3In_ z=7Sw{_Y|xf1qXurM8%>?zJ1WS1FgxP45kVItd}MMQ$kT;MU{rL9cuOmX@?NT2|Qvq|Zo#9S|jx z7G7+Z4_?K^Z=RMwzU|!j@B2!JJ>^&J!9?!jgkzE*FzQ~Y+?FF{{hTPDjj+F1d+SU6 zq*Of{a*<)aL!RYJj8W#zUzby8IMbZ{=}~vVm11!ikyznKBx5HuSUNnkXV&Xrz}~PE zVaiIv<&!mBVfp1W!DklwGehscNH8s2Lpho=Iin)5c(g3ThMh+@u{HM!WT|+$%28;w zGv)--V-86Y1*xX%+4UeiZ1b;Q7>q>>yR^pF?bTizH3-3pnz}P*yUV!YY#No%k!Q(P zcrd;B4siTN{kh1*%AbPoFS|H>^RGI8Yk@H@c)=Ny&N2&}ja)eZ@At)rWO2u=YLH`P zOFevbIhjSu{$>eDTs`9%EkjK;hef6W#H3r1r{=MbOZ#G0QbmOEjG73iU$pQn`QYp1 z-`t7Gzc&x!pR#R05wTDT695V3UkInzs`AfDu-?|i2TG_pjIPdNTJq=3B0T#1?I9Op?_3oR z#$uyt&NF&cPzupre3eQZT=;Xj8hq|#h#?^e#-3rLcWmds%T9yki@gm<0=C+L@4}w( z|Fiq{V_ELQ);eN%y(|ko<7J+EapO#zdd411qgu$=%_OchV9Pe4_W5~&d5Fdb>=)G1$m zD>22kQPm4=N2JU4lmDKt#&3LQbUVab>dd0WE!pUKe|T|lk>6kH*G>U$U6 z_BYk;NCkR)4gEB8&nZkp$0q2tB#cjg==&G7U|Jv6!bZJ{m#0l{olKG8Gm06R)*zW) zr3OzYgd{!6anN>lm0k5L%2b4vLYlPmnSRb)n;Dz$P`m{gZ-dxRznc0E`~9gq$!42( zil_rmhd3^|SY;yXGG*gJ;`Op5Tx(`GnfGev|C!H-m)JB)le=3Y30KT04@Bc4i0j>O z^6hWh@I8Mx=!wFk64(1Hq25+!O5Cj<(`PCgN~hN4YTEhjQRi`yXt-_FCFE-R?QEAsuowQ%10l(4n z2Sx9iZR|Nhd+cE_)oL&Q%VrtNk+8EUeo>}%>E?S>2gi-t1GA1 z{~dJxf1I&~xE@Cec(EJ!{H1|{rCY}(rn&1_QgU!@j0`NFXN$Av5)0$vdUm;AZ;03L z)SWWR@%~JH%*ZHH0F(uNYtvnnkYoWxZY-9CPC2>%n@hY(f*vNr_ zm;UZuaEzZ^WZ7ov6%;pQW?=r3vMRxi2*$oOCN;m~j~tJWE-4l;_e2)b%5rl4(6IJAR>EM&(qK-q6cR@AwY|RkGVjFnGPTMV zzsJy&yo34-m#Lfi&>pFfyR*4!qb-klw@^m&WG zYaEfUfc5-^57FdALbkD?=Rr z+2|=92CfURc9Ze0H!10*U5l9BJBUwTRM0p~`@}pgUPR!|P|~%CcnqRJ=iNA@Mv+!> zD`rd!=E>YHVWDSpD<(OLH0~}+gLsY;j+P_&ax6?bxuWH5sM$*Fn-I*aTRwseespJ+ zEW@FfR0ko1L6D&HHV;O~W5k^59*7`Ve|C;1=mDa(H*yG%U(K&+sObK)y&1J~C;pai z@7l>H;!WPduRj};`W9g+uW?W%$f?XwOV#sUu}B{YRlcU-ad2?hjJ0*9<|*SML__p6 z+o2u67}GIbTpZ?NNs*@XA3mAdSe0%xN2eMR zIPFrq&z@F;2;rycHaFIojScmqSK+bJeD2}a4O?3MCFVaAMk)}aR};vq5g=NN_U$V= zr`1dFJifQSX9xDZ3zfZ)tN_`1y*@}JV)y2?A2Kp|zKnFr-z~*A-YsPXL+&7dXL5=* z#>g+ipBaAbMN*GpZ2zY1N{|0IMIwg7aJ$@KoN^7KJp2|~ME?|1%7yS*Q(k}vx+4;jA+ z-H@;xn!2s|VOWhnEsEP@>~$)^?OSE& z+l@ROhrRjDHm;@bt=25iZ!iM70o^6iGf74e3lie}#cYU^tqc#*p2&V)b0t%b5(@?> zDB|+Fx9a1Z38U^mb8~Y9g```R#mbVQPQP*=kzfhaI#y3k{qbB9u_v-{g*$BN+GNhq zrY(|h2ma6I6sG?8PgVmabN2$3UabTf_4kiIaHQUII5yPgUEpRHIPA4FPWlft?^8^K-y`x;B_qg{wF zK^}`};g7W~W|`ano*g#@>M{QD8n9UDfz6a)QbZsVauYNBgz|2oPRq(uMPBecOX5N= z(}el;*C8f|%ROA*EBZONvhQo{-GI?wJ0s?JEH@;G16YP!qP@fSF4>b`ZlM0t9nboJ zeFfW8wFN~RuIPI_|1G9xy!~X#yYM?!Xv3!Cs%ktYGM4t4BgjpZC&99KV-3>jQp|*@8#*x4lJQg z_P)@Fg5g40TTHbV zEa{_ya`EKfv#gI5e8##iaeYLNB@V=N(MW<93%nu>?lMh9jdXA={UK|n6x^=B^j9g= zINKb{%W?6z_Rxj7CO-lgOXVDPpYObm!WFjY}ec?|ddryyY#%uFV87XI6~o|96ABNHUX zKWWi}Zlq8zHA^#(XJi4b!k>kIJnslItYGi^;sy89QX5$~?xgT+s=6oZD_UDoH?ViP zqSyuNb(Gb_I;TOsr#}>8q#~BNvU%{M1Xvj>$_Y*P1mCZI$Fly!&vUjPf!!ZkK zmN9q8`!V?C&ml9%D@c5hA$L zI2)3?(17f&6^@1O-;ZlRB)SV$2emAyVSAEe{t}cbwTh0!S!rhCt_+OwNb|QNY4*Vm z3sU&c1Wv}E0z-j8X*bQr=`)Xc>S{sINou~UVLqFR3ha%k=??sUB8^(_^x_=*EIoia z^NSc}5aNZ8Kez?=#m!fA>0s9f^T?Mw0b%Vg01fIOZ@_ z9%V^*yM*1`g%eL?dDt}Fx4ggSNEP7nEP*=87OF2(@+(!hc10UTrUAD>wWC_1I&-f+2t(MN8*OuU&lC;l zKRm`=!2eT|ILvkVv)P}2Cs6Ac<%dAeYpG;&0ax98?nllb>DEi#^jB61=rog z_yD=Hzah!)Yv?y(`$rvPsyY&1!HQV_1Knrz*9i}Mxx=9FnO^_Kmz2d!sLsVs#d$>= z@f_J7DxSYu*7+4|BM|k55~jgET)$MSsR9p27w)%E>1vBV(Cqxl2!uLIYM+K3Hq zBD*^5e2{elv>|Oz;`Rqp@L!lo$#f!GghGnj`X-h>01I*RP6U*+A`c$J~;H>ZyH z?l1B0ZbD2pyed~_1^dCTiFvOVed}E7-_I5K^g!p=pPCyv0`#jwE-NUt9TF7j)8hBX z@PhD#cmUV)Fg4`SklDV3`jfD842_VQ91&xf5$2O(l)=V=M~tADE)=}&#PFm?zOQt% zJ`D9es%0mr5T8hx?=zB>@03Zw>Ip+7_w~^SKm?*^_x@kN8+Nm5uHhg*Coj zFVUsk`8(}j!5TwG?=e1XL#%CM5W%aUz#g_1WY`Z)SKt%grwFcpp17to>@CH+MCO{kapZXB5VE9n@~)N!NfHl zQZxg+>`OHf__B@uLq5kjGXWoKYrX}kEEw1}5EIuwjGSiHljB*wJ|p`|?S!q~N{_Q4 z`9K%^!)|Za$hR^=xCsjzWW2CNnh3u@9veKE=pF4H?rpoMJTOtJw35Q9DDcdmy+=^$ zw)G_)ae`MgS)ozC1b zc*zwk?d3N6&R_((Ex*`KaFBu)Dxj&Z~Zv7I! zo_au+HUw{vxYrRts6r(`d2ccXv$0w97GFY5dd6a=6Q&oat*0%@c$i`E!%%Y2OXVD& z^Cfs7DRW%FT)WFU_T1^KKkEi?ySq_SqgX*=dw(phQ4uVp-LLh~*eFPs$x$qFpg8oQ zn(CR$K6|`j64~n`L^sd>co8(*fR)hMq22ffS{ZWlT zE)DZM#$9PPl0JoH;{TK#k?CSn8{ z$o~X$<62GN%PfGf)H2Ob+FpsFDPn;)L>>^DqDP!#@l3{e!ENZ%Fv7M@8}O*WBD*E( z&RSA7$%u#4+l`UKyBTu;`X_6h;p1Mgh7eY4=G{4n*;F8{A-^hrivlhrh(462p+HdP z>QcC577``x<+l=$6M7@cFh9#tk+C`H0tj4gh@ zN@%FgizoKJ6tUFUG)-}BcQ1^pnX<}M+DjBhhB}XdIY$oIrR;j3(l7toVxhNLe$4`t z#?4OMA2Gc7B`)wM?a7fOvF;SJ6#3t!Dtv|qoCNmH^%_)Koi6pi3rN!V%IONP2?1pVNjprA?5uSvKOlkiEItEf~ycMh>H|?BP`6q0dN3HEZEy&w%pdi@<1m#&jPL6~~ zZ;9ZzieUqr-&78~a-Og?c5Uuuff5of_|u?{(bf;?;cHv?%)E{qMbRt%1&!wdXJl?ECtGXdS&DCXQBCa&{CBneZTNip?9uY(K(gH z##55!Pl2~PJD>`uJhK%GPOsQ)2<6?TnOM`*srkp=0c}vUPorBGb6DS|VGUJf;dmwfP)4fs zQY`1h6+vdsz~c2TA78d=A*P_d4J?R57vDdBZ9qIO^**`MTKGCKYGFWBs3EW5a($It zL}+a3c4i`Pew>%JU_=o6=@r33ZK}hvDH+g_Y@pV+|BBf=kg7)6m8! zLV9fF`%j5BZmS3v=i0Y4Zs)^&X+Yo2Enn82VbT)MzPEwppnzS=7>Zt101Ah!5_0;> z|1xK0v3|8GUX}>19j(Yy@P`1vNAZ@Kq(`I_M|7` z5t^OM1PIy*4cBC}`gG3sTW6a)6=tJM-=`zqr!diuI}JK#2sg$~iwOz}=D`+~M6uk& z{V%%aDuvpFsHa64es|fS&MTm`zeY_1$6u;<=LS{CiJ0MfsbIZD74tL*pWqn)PCl@v zMz5-q4Oy-U*o9pO%~*u|V>reQoB9${sVG$^Zo851^k2bQSlcYkzpD_uX}f3qH8bIR z6FW)u=U*05=-bB8ym5-wrzy`Jl2Z7YRAGT7Y|}o5kzXGJbm&i_F}31F-Ki5PZ}MG= z(WoqZS3mW51gcE%1AKb8nmsk{l(n`N=hx<8QfWJO4mMe4KBlJZ1aIc`TjfKzAp`sj zAuO===RKvdJwzi3^H;7m%BjFjKZpqJIF5D0N>O=YAoRIlHyRJWbmx__yaX+G)rKFk zrh>pu1A@}RM=9P5(V524n1@J;&p_Ds)nRCj#OnL@78=QewFMoFic8qAe)V6ce=E%t z4ojn+>M7bjC5px*Mp$}TwjO`8yQ~4FXtsc*;)jS#x}J!xunIWiWqQJNsXVs*=tm}d zFSg{NE8EwCtu0zv|7?oAu-u&g>L$ZX-kifVMW1?$cvhIct=QVI#ukp9`c6-20GuK$ zpVzwF6S;ipMfEYz>80fUxv~CMCNUbFuq}~LczVRw*MA!FsVkIe&>X8^^JS?rj8w!Q zoGFP=bKZ)eb_O&Z-kvUdjQVinQ9d>;V{fpXpK#Sg`~q($Cp&mT&q27?!053V z{YEAgAbSqZqL_vK09366BDPcU<5VFcc6T0n0`_B5*TEQe=eL3zB1onM_)|vyBehl6 zJf!zzXN;}!r+y7-=pZOMS=L+UMn59&mgUyUhm&pC&E+FQrVy?10P1Wmxo+=fFhSHZ z6cI9|a5f+Ld~-=E;&v`1cang1W7Sbm(4!y4FK3LqrlUdPvXDSqkiGpRa=s4VKzHBl z;pbx3-KhkqTJSv(am_5eMS*>f#%!C_XVid)C%@Nz2^h;ub=oA-XNPgu7qVG$;Yb@aJ;-GFeozH#x= zFK|=pV)Z@&?6)n_u(ucO>SCSEfY0j6v-o!QWZz{KSc;jl z+Hhd!+L)xi%M~b(d4-si+HZgFZg9?FQ5p$?BQXk8<+UVNOh7tD&ZW0kS#1zj_$TC; z*5Y3^I-TQkSoV9uoD^0v#GhXWG$Xn-iTvas@-9GX6HIdFanTxRSaOqo|HpPr;gj@U zYK!)UR=nSx;4T9tW#W+is%+XQ7EAVaXvq+%kg$*LLMQFQR-~ITo7aH>H`lm^;qRa# zA1?i|(#T1{d(KmFW;l!U^>B~x>V`HrF-oc6>A?6xq%u~W3z@MFW=hhL#WeJ77RImF zgho3vj_=`7_TyG#0(R*$50uYG!#1c!Rq|dnfju|6uxCuQi@^dx)DQ{j9!r~ z|IPYPgRNMRt8+O6{Nx?vIh-i-?Q?o%lW(k4H7+!e%}}-D%A}2HH`M*7ne`8>2if-r zU-J$>4^21Jsr}n1I4tugI_c`7_M6$cZ9z z=_|ZFd4K`=kk}ZvB8!!`&TPV_x1<99?lCC+*EH<>C~vKE!AA#Oiab!1sU@JH@@kI-u`g zAayg63tHYvUKO0vB%CUA?f9qhN%))VB`#lEhGe~eeYx|}pT+i7e{lShEo6ojP6QY# zGH2^N$gvUJJ>#<-wL@n_V7dLU-6~^i{4xcL__&bx)R%O@+MhSN_^{n6a+7nG)Syh6 zgFdl-U2KaPlz-51bABdE_YVFTfm5Tq?bG~6djyo(d1G~Hqg~?xV?1oKOfPmgTe8Fz zFhn4+lOqow3WGk!ghXIp6!Q8Npx7v1bJcyY?YuS@Z!f*jJClTFFNdm>EwpRc-c-jF zPW96{@3|Nsi}>$lGj;vUI&g8l?*FyYO@_Hn1KeXGSQ&{~Q<4j@Zc`94%Lq zXi3b#$jGRA@h`gzD9(Hc`*N947Ed7emO$jW80)4_o6Y4kI80A534D03hd>I_C&BSn zR+~HSC=}qG%rSL#I7Z)uJ2~G`Z)f2Fae}^7KsPkn@2y)1kB*Y;YuqP{KpBZeG?WqY znWtx6ADQm-dMW8%4Wx5$DMo(zNChnHeUfSw;++DL0%>b(IsWJrVjVr?TY;KyL`llb z6mkGoWM;arER(}VgT^UMiB=Rq%m6`>)_q-A`4X4vx~H@3Q@s5?5Ut;}0=)Gde~mku)u0^dhRN3qo=*TDmdd#?JM8_Uh? zhit%o=gWYg+h4p~B6z)IoQ=Ofh8$fNK-US)o?HaUEP@O!Yp>bY%m!o_fGnUR>XDmx zLfNjaQ-KzbDox!a65p}8)AoiJ7RO-0FAGfuz@d{d0wF80IS9`URq& z{L33NyrjCeB@Wv&;2SRcOWe*t_SrMYrxA!8`D`NwOAX z*^pR~W0o_Z7NXXCbW&2XrJ5?#pvL_DPhzUTGprsbe9nr^;Vn+3P)YqVEx4kYXc|!A zB+hTu6%#>q&||iKXtVxs4;+ZT{Snhe#wOs@Y7sqcJ7`nm#zN$Zke$it)xB{ zbQZy~DPy%H6`8>e@5vsdgvYj{`C+WOFF)8{SSEi!m{@Fyk6?KR=jWst*9EY_(Ey4G zbNY_npJy$eC`OV1Dgp{ECy|(t$gxLh{nT-&-);(jHO%zn;`}YBYQa_Z znxA`JARgtf=KqXVE_qAuXlCs?Xm_i>o)_1yBp)s>L7EwzdcKKdxW{HMwk(Lki#bx;l=2%lt>v3bJ9% zy~P7gHE5bihs^<>xRHIq5f@bu{a*JI%(&*JdCPJA4DL5>w+-(TJMMYYi6IsdtGZ!M zix4B3Qz8(*0LNuzY?J!G?w)F?K6M#2Rj|FPUa&LS6DJhtU zMm__zAmIUD6=E(W7~_XLeN%kwsgM}*Q19YxNcqo)TtQ!G0@30=!w)G)60gG!bb4XZ z*cM;PY;}@&B6%Quvd={E1gk)AR#Um%gPCrs`QQjaIP*{4%v;=qJEi-9m{bHDXa~IhPcPl>9-1L6lcin8B!=xvXyu`BlL=Ne%dY$`2p-A<5qc9!fXK9Ag+&R(4ZHw9X|Usp__Qka#JuvrE^c0`b<+SK^q7%<^YkhOi5=5-HMF-I1= z#1uunG5>@ptM!kb@(eQ4+l#7ZiKmu$IS!foa_7iCk8uM1!Bav~@&eq(UieLamKhn% zl+jk=3tKXWlv;Zb&d#-6*6Oon$!aR?R-c_m zSu<1^xn3sR{k@`arT2+beRC0G0wN?OJ2Q=anyR6?`Q%@_jlb9!20h4(J2Ld29{n!D z6o-#GflYQW521+ND)g82Yu=s+L;h7_99js^(%@qQvW73t+HWk5>>k@FmOGwESm4`f zfRXR#vrPABo#3{=`SgY~F8c0TbU(~sBemh-rDkwWS&dl|@wM=PIx{aC%- zI5Ua{zfG&KV2uk!5pqo+7dJwm%Wj0Sop4tvAhhmlXMzgI$ax;bgNhTEt@Zjnfwa2` z41VKZCa~5kAL8(ltR^Z4o-^TI5zZ44iVW(vMb_+cAd+6Y<$I2JIg_id1p3J)(rY0)M zf33LSsd&6U4lx{f`3cwJ!;s5*BZ%78Od&68j~#R)eQEf6??UMu-)QL z{g;#hZnXI~K%o`3j~JX&Wfi-T!ny?vfv6TQ^$KJA_>*s%nKhqyVdP`N>{W&!Wy;}k1k6yNs-;Kps8Uk5>u99Rk!?Bab%W{VTWcl^NmzuBtiBPpTF%1 z+f{*hfVlMe1Ewp=J~R!LMXMB`kN$mpt3b}%E9-=#R{HKoVeIg#d0DIqU#do%HYS~D zKWg9DwR=qjuCLG2E3Kb@eoS^az2XXbDWE{pSN&(HHvYA-|0bh9e+N&{24xQm{ z-Y@f^;^L`*J(*uTw8C=qivwt8Le!#uikO=4ONhDNR!>NbzdrJA62YE7u-~G?X`{JNrYKAhSX>f4x8Y8{)X+Gm( zMiV0i@}NG&$2;Zr+?i*0Ma5K||Mq+kQVGdSIexq?Lsk#pg5SkZEKIi# z$)!>pUOn4Gb#%ua!k?UGw2%E>j#pYG{a3A2PylYT+Q0?foGj`AVP`O&40EXWpFe+$ znomBAfjG;(UUxN$p}aec>WwC8GgDXfZ8iGGy{p zfM-pRpRY;2H(k)(7f)(lU%42$*lVkch+vlJ(b0sUELej|jyA41b>8{o!SrrypHJc4 zpRnN+q=o_!J_{Bp%floj!jOE!O5ebF-v|lHwm0Y@G>zK#ewxvk_2&f4E^S}OwceRO z%-2DmBA=Z&(9XAKWwmORDlUx9t%%WJ5Q)-%xkWwlc}$=_H@g%WE{&d9KRyWz7lKF*+rJ4 zmj$g#cb^dQY3neT>9dHzvyAfWWcD*g4aG$Ti?ws}Ay5~ou4j>2d*m`U+7H-34UQ%; zN+A%@GsZxSfL%k{m6f-#*HH8JvpbDtpJm@Y$u_L?1N~em=4E+^o z-!M!&&5!l>YM<8%|L&r{D=fycvM!M|U~J3138jEQ`9j3l`oNWjPQ z^V=5;&IM>x?X(p22mz59JWt%5K=FK>OkyiE8qwtQv~yJh{w{D)Wv#)qNNQxELefoe zsBJ0++Pul){$d3SAT>^7f5i3hCqwq$1iILMAZ28JD@f0cjaNrMf16k|sMG~Fy9dtD z+AL&7CM8jRx2n&B@Yb*A4`%Zq#kmIT7rC&5RIF!+ycWWB9kh(gLw|!@OP@^HgC2aT?0rI zOV9wh&cLT7r*ocZV)w)OVj)QW1gvpwx9Zf=*8n6yzn8+6PIRfrZrWfT8)!1L>i1ha z&$%-JMzLVnRHa+$+3HhylqBVuH5=8@LEXphpWkPUW|wKeRa2kPwVeyc#HVo3b0P-k zP11*$hbx^x&pawZ$AU3GyBBfM6g3uCE(o)y&KLIuTL1Pa2D|y+YwU5*$>#kE z`IRKN?rYNKB!!fuC#U1_;y)mMo>7ZBWgDZxG@Px=W06Y?)}n&CC*D*#0_dg!bg!}_ z3OjaIpMd&1yoYP}aR%0J#vgqsIq2tYIXx@5(70hFW$VjmSB^V7sP7P&Uh`OWs~9OB zL2X^FQ&cW?sdg=7AFnW_j?UauKA+4TQ`|)YB&GqcIk3iU)UaEw|U1U3% zPZ2yZxqu6UA;a|Xzu_$RilTXCeHZ{M<_Tb==LDqA&q59VwEMXN8^5dpvzx25o$XFH{;|(!C|YIfT$CkrU$#; zUegb2TYTKQ53}FaYGs%f!w?s5Pkha?;(xrtkBQSmj5c5FbE<$bP;d~&^$yljVsRqk zo-Szbu*>#fi84ibbZ%VjUA00C9*Ki08o`Rlf#`u&DHr_BgorhC^m+y}Yw$ny+y(FI z7fbqgq*4f6-M&ptVH+ub0vi2a+Y8_y`3>H_s|lTKcpJ+c0?Km=4rCc-*_~^L)IS+7 z{65lfZ4{`nVJjU?!<7cD9qu5cW_GoEdY=BvR=WNJ4kk*ut^sQCpV;-o7@IXxxwj=G z?04twyr_2R@8}HJu&*C~0V$o_yEQ`c&@Sa4nbKd`DE!=x+|egHAy?;;I_FIsTX5-g z1L&Pwd9NX$X`a96E2A{Ce#VbI-M*}-R=#8&vLYL(sm0am)IMn)$@j1KYqFk)`zu%v zbh@5)l922x{$2vdz{8sFy0JK*euaOS0M$jmRt*tHqKRR`1)v*M<~gDV4D1bm-G2Bs z0PTvFrF*yfmc9)vGo4ct0jXFlo9k(5g&ZhVN)aaX$aP=*Eelz%+n+wBo`Cd|@64MF zSFI{;$Qk|$VLLaN5zym`c({7+7t|;P@ZSWviw6zh1b)9_kz=V>=62@~df|T@2*Mao zDF`PX{km-&?T*m=TuvQ^t*cY>VCv-^DaGS(4p6xA^MwCxW^btpdYsk>9BM;j94ZcNbTWVfWh)0Vl`>!XM&!uVpuGO;Y8^E!E!8TjlI0ZH!0h5;D*HFJd}p?QO$@6k$uP>JVOFxiQL-Jbmbd(~fp zjadi&+#ayci{JOGdBpXx5nHl{>zf@&R5;bC^b}h^_mK(`x=LN3LI%#24MSjjkwLg zXLWDST|Np`s};uBz;EuWV7`zK3a5d=17BYQs6c6aSUh(K&27uxM=eGhWGU>gZqvO* zl8&pf`iFl%3*05q2ftCK%Q=|iKHftP6QS!{_06w+HxS~9ggFIyXM&K-n*Hf!s&w3) zd$$6!P;OYIDFnxV4Y97X4mc4_iS%pfn-pdpTx-Y4vz?#`P5ig%M}s>j)<40rqVPT! z_PZ>m0+K3=%|rXFkJ;THv_k1n4OU9KRn-SsIzlXR&3o@y!O(~(eCwI74$Z6U@=%zV zwL$J*c1pYcarDc>ND{JLyIA(B9z@)Z^Uj&|BBP3ECRVyDgQG%1={dFeV90`ZSI`E+EMQ3T-IA& z8uv|Ki(ibKh=>*oBJW$01W6zz{)}dlf|Wyz6-ZOsSgaCknu!=SnL-1=?E(#bAGSDC zuclT33?z$Pc;HJC`~~Aj`(R&xTbyiionSzq=dUe<*JN<9Z92tTDdfRDMkYwNRy9S~ za}79=G<8V*C63W!)m6?)|B3~~Uqe4d^1fSq`-4ji{OK6kaRd@_yL=9LYF`}L90bZo zj>WIeK8b061`5$x#dmAFW_7p5(Y2nQU&Ow%QciuX_Wtu?jZohz6I0nysezKQ8K!3n zYgPUg=D|MgQ2_RmZRCg)6;i>QNTqC+(s~_KnB!SAgTrC|&lC}# zPjl;+8`UbnFC5lT?83aT$2=bS#)=_}y|Kc-Zc=?9T0 zV+UJ(0O(Fz-cT3vf zdW^cGtB1O56~;Lbz7|=d_Z7nlDt3Aw`&^Qcr!###@GAx%@1$nQ829d5boY-=!Bkpz^jC2_1lw)VxI^%ogin4}DkCcm`CVXvy3IVxu0`H-Y8 zxP1X7_LP8#vIi1EAXzQo&!7uI~P-j0U0gW&Jdxl^E0a+9pt|-Y;xu;y@?0oimQ~x5B5AmZMsEXl=JKP{i5R=c}GPDttcSV-Ij4lM|Bb>G}}E)u6F{L9_+BK zO4E7#l1rV1H$+pileT>yZ5p1OrN+M6Jd%`koetW!dbOOA^_%VYMb!!$2J|%DJyf#p zKx%b+G|7ioT7Ry;+nwpg-1V)zb#rl7k2}eFfUQc~^OY6%9Z43Y!|6`Y69y}Bhfj>0 z>R@S`bNi{#c))FZCsvFHqvP+}9y-0JZx^I1_cNL;$M7^80`pk>T zks;x0hkwru1J6@C-F_^C!M+-gANf+N9z;)k*+3Jl$`Vv7ePtx~lQSBC!2OQiM*>$0 zjLw?oz@CWz>H`Lb;^$<$UI2LbEy1Q8nSEMnl=#fRt#IXc%C!o6@*9YFNAAes6$8Tw zgIy9>LEzpt75LkbG@Lg!?pP-Tp;eAw-mwYLKl#?M&3E|<^x6gYaAC0e0sg$EI_9psYX3##U0kD{fiBMT=4woHyh*M4$td$HX0#HmvcLh4-wxpv{ZUP5 zv;7o|Q8eLQ4)+RX36cpw7?Uq2=F94|Uusaa(@jMXx2ic^Pq3UTx@K(c|L7?32{5F{ zyH5+AR1Wxv7^Bm#0|?uvq~jstFWV+_EHNYB$m7mic%v-URH5vh>u*r|MlqmmJk~L- z?3&!+6*N7nlt5vb0(p)Z3SMj--bA@IZ}*6fvS85@oy(2UGY=>0*_?Vp>xCdPI9QHt ze>%sFt(6HlL_6rt2W>dhzOHMq&Pg^R|8q35G1dPbSJsR!k*+# zJbj+_>7`yUk)b`w-~Hf!YpCQ+44g|HkN4$%b91p*5L@Ak{6SM6T5d8~E{z~J#%zBP@iTAds?Dfq!(u?zWUFlf8w=c9NS39#*k^EnUcQe+g5IR>&(!;=fJfw zbCzz;1f;<@2Tez6n-vSX3F}v$Eu>+DHWYa&o_t_%71s6~81$%qBk5GdL zDZT-=tx-w~8y|3x7%J9fH)=eYb>b*EOagqlirxAuQs`wroVCU->X-v59b#I#y@=!Vk$o>&gbd||-E7l(ydB86t(yk{3 zfXjPT!7ivh$%-5t{KEpIf0IY6cBALt~pb8SgER$=7))#>h|E})*LQ$INPo38&1R*gB+8`%&oXO zy>HT!dB6eg<^d9VVej$i-?EG>n2em@cewr-y64O)8eZ8{0MbkP)u=Zj!jPQl^D2RP z*$4X}c;uXj4ks`^mVh9+R&seaF8a&T zTR6OTZ+nF!U#eP#g~>uBQb|qWfF&~LMOdwEO#6GONhVMy`JgpFD`-5~jy#g`@BJI-boqLK(j75c_Aa)2pke)S&f~f?(v8hM=|tW&cFXpG>+)9lF#!t6 zw)72NQDyOB9+#|DS50qP~kom6;$ymWa1NJez|@@@YaMiTdPPM}%z z;rCyMpRp?KW2{aRP>X0w=yqC&*XcKev=a{1&z`Is@?c|@E)_U3Bct;cYjV$R>w`az zyNY|yFgi!-6Jz*~5a@TdpvBCcc9M$7cFevTceqzAy%rB|LQnSzZimqKVOa$h7s{ls zAonXRud=ILAGG27*)zif(qfl0KdCOFA`SV~rImuE1P+r4D)>F>_=BH?=tG?R6FWFh zhO2)leViM9n?~y*F8eI)^uFF+Sf7W=7kI|Ip1ZJ!+(qZ%f^Lw-)hj?eruLSfN-MbH z-t1MwCkDV(iz0Z|7aq{@X?PrDNXoG%pWd+G=5Zwd{Q1Se6w7g~trQH+W@YY_c`0?- zQzTf-rX7tYeIc2C>#2yc)!V7<4{>{#2m{ErbBS^{OW8`JK8=7HYLsgZ+OS6tF1`u~ssIUM;IH$gEKilV3p7j@l+P_jTu(ZOk zZ$GG~pLS{yT)#p9xRB{x=v=anPFKGps2B?O?`XvvW*WxzO$Nv`;|E$p14m0VF%WKX z?SB7iX5Etzekhl{Ui73xbVI@ut|p8bNrVho<+oPPS`}w^6+zkg(TB<#cMx0l=t;bu zig=PfAn+Yw&=iyCkY49s^p9dbH&BYHd2eqJmByGBtSllL97`BPfGma6v`!p01$&q2 zXt5)1ENeoUU1$Zn!sg9EANRMn$w<5mKsh>CJA0AtS0?%^X@oJ>dXcuwCleSKNrW0T z$2jO6v-t@i%h~<6s*N@IQ!V|aP}f`*mh%^dh4Sdeiw>$xP-78XTKwvgYl4{0yt>v> zOaGNs5#Dps7rfchzXvc*kYI@G-}R>~+@_33z5V!^f1m%v{8;%2??T)Zw_0@Hgg-nM zddUDE*=~>0xBbR@@+dL~Mqkl=FR`GI{TDw%5v}o8Jz~`7)9WjKX%}j^cVUlqEWs=A z&5o>|v1o~AKEe3LM|94-#`+UJ&qc;3vG*Ab+G-$io!56{k z^!0kAwtzFz3PIRI0sS*2P-1bPFL~hWMink=ox-tXX%*}~qK=0&B_Ow0biZad?#Kmz z)!`Z*Kgf(Me=uBE!c6M9X&dJ_$SeEooCT1zb*? z!8?+M8^c3ldtb=Stn43*#(W>>X2JMMeHPY+!rJ^Us%B=3qM4%3-CH+6anNuVr3Oaa zbEzi2*yf+7umEkz_+Co5*_F9IH3r(x3|UZXec{`ed2$(bM7G8nX#6JKr$*hpSG`As+5(dRe2Nx%CwQM_q~><<;1uj=%@s3ET2+w07X#`E z<>*ZiC9se|&jx96*j=5PGv*q-!w33~?w*W2LDexB2vEC}d%=WLqTCqSuK4%VWyJ$w4c0xnk{HKH#6e$+t)K3D-KnV6qbDcsKs0uHMAqXUq z6Bz8HF@C$x(Q}QnPJhvI)6{Q&@cHZaZGKoU&DwxiLr02=^ zQNjvmmp(9;P(QyGK$ltwXSfK+XO8-Ac5NIVICPIhTe;j@`pmP|>lY@^Ovw~5xF!2s z5^%z+AjzG{_EBNEB7e9Jc?f%r@G%R@W_5Tgr&4@Nn|hggUi?^qG+G$jj}(3==;~A7 zU13G0@M7IK(M5WE`1BD}sa61+&>`Amj)q_t_ zTwkCs`yg`4+)@ed`Z3rZQgDJhlC%Ri2~6;EvbDZZ%?u|`Q@-U067eDB&G_*hH+Uej z4+*D$Zz7-^6mMEidkyd|gaTV6m$~z;A9ACnPx*- z0G>ul)6?~!NTXV`k};HsK6c^}{7_ApFoJ=Yr-BfXMjt(qQ=it${}6%Y`WM09Nu;Rc zyNGV;f_meubh_np$7IZLLIU{HIAmPX*>&Who|CAP+9H2 z;>Y7L?dvZcpT&kexICJE_2I2CMa({_G$|6F&22UXyFaXAgP7v%!1(k6WTudX88{bE zAwkGPI8-~-<;dh$iehg)h%ZmMl;K~CXpH~@aSV_hr7Spi^uino zN+Hz;ijOOVbng{!~gwbL5>3fyeCTudXl4E^-VWH_kHkS zb@^%6+v^<+UZ9lS0(I0pTEYg$-un;fvt-#R3F3Bh{^63lw8EX>QMr7l^4oaQ41J_! z(6f@i^f-j?WnfLsC7{8o4i@Phpsb)8Vyek2_npRxkjQg!#bak`u(czC(@rxK3GdWO`sa7C#u0$c{@j@%*B$ z5qEeqz%w*t-P?GtXqe$0)V!~{?6j}wGF}X0;K(h8ch}Sow?u=b+XDnCC*2l5&RX3Z z)8Mo!OE7)Ga7h>r(=6U%Wh(W@Z94Of4Ws?W?Cv(Arn+VS@mebk~o zoKYoYb}w?(&QEr7ow6cT{ev%f0@@j_snm~zh< zfmswhvQJ*yO%wF9MMrKj-Ffc5PK%9n*wjU`)VhnUr^LT*U>S8?p9m1HC+qIt z%3Zr}OMPj1AntM>eieh=kb1RwA0%>L$A#d#LWu#;z?4j3pJP7g7F4_BWmO5A^p`Y} z8W6d{IZ&SH28|!CsY-N0vivbH`}y$l`GRGo=c_|A{Bc3bupLSt(a|4E6e{gZPmAnaUZUHSY^UUlX548wNblD1c` z?dA?*OI2`ox2B)&=s4N30{{k}w1nj&dCX;+j40gho+{6p53z^I3K6b66=N$l-*7hH zf5_Pi7Dz@QCiChZ;Qjs9aIfsWV{g#*GciJ@^$EQ7b@g)JV@7iuW$vf*EJ){JtnnMA z9xPJxGYf2cCZbXHLi1rC8XpO;$&>cH>xsn}xmt030$Qg(aGd>4KXC@qBbDy<$)UzF zYx8eHDKBz@jlBIw^Wc~B4Kc*0&p;J9sX;-maykJD2nxBQ!eRNXmCNR z=$7ZmJ_dVgz7_cU0}PVY34%%sb6tD;O&xEGyR)}W*ZzE{vY zemrHN=HUJS-Lbfx#?nEB7J|_K~()FDq@lWqTY0>s6jS-HzG1+C&_t(da$4AnDj}p^V(gl<(!cTq8 zX4=r}Cd17n=8$JF@SH$RGEbhLp@i$mA0)hyVBjf|31yGEveP1H|EoB*Y;%wzwE*W( z_rab*&4$ikF~T!JW%MkuQ$3yotNemPoh;EF<2W|DuUfQBr_&tTBXrBA_jGfCGxqS? z^o;>29+c;?^jC{rJ-4^FN4t8vvUI9K?$lN|&X*w?{Xr|~ zRQ;EyAMKhY%SvSxlYK)TMUky2{LO*4LhD`BOqb#S>sL0V!EWSI|!8G)iCjhj0Q;)*?MojJ>{#?}Qn6>U%k; zkU5pjUsroR4KF%ET=8mkY@!ra%s50cxL9lhf1>3x5B4Akfx9bZ=lA zoFXO1xr6B$WX|n@Hh&ftcYg1=V_kC|GVID(&^QGo_x!wNk%A%XdDRt^W4p4!17q+> ztndDY?DK6oxSZQ-`PxBEb)*@1oDGlzIh#OwMn~1|0`}#i-8>wdaXi zUk&F>lC)ib?wRu~Uxkl(9h1=*;2d)0hNR?sAT=|p>c?}w2d%j4I&`bEtm$H02-N3v zPNjRZFsFBEb&lU1cKK-MZ9a_&--v?|7Ug{Dz^wKXtm@PeS4UIcVRQRK4mz*8ufY!b z8BWhDpJ8Vmk8sT0r;IzU{)$c;>sxsIx`7D`|F>abfsnqvEJAXR31iAi*jA2}d|m79 z`6cCWcjgrsPXgv8EbqK?VTdqxfScNywPCz-7nxByIrD zxct&(;@d0L?ySHzb_gGlnh82yx_?h|fAFIJ^!4BZH#eA`j!jjj&b~Ww%V#Zvp2~UV z`*JV(R`57Z*-{QJ!Z?PIq;i1!JV(QSRCe3YVo+D- z%u2gE8XXcpah6$lPxr$rhxIQ)Vnw06_g@>*1JXw`c3xw2o~BykSg>WkmMog`=OpA_1w|A+qCP!ceo!ObYLXtrrgi;o!T0rXFC8+I-X<{%0bJ!*5r%Vb+_`- zT`e*HByAfBgW2}Rolyz#*KHmItK$$Z@A^-;xQ0iingzbyVZ-H|{B$Ejz&R=KH3mN; zToDM$UJXRwSRR67*do~5>-*Z!F*RCSe>oOzyBO>H|-2CIvrrZIjWvJ_TX67)ChjA{mjZ zG`uJVpP?I4$UB&d3Pw$;aU#Y^*g>Imu#!F+2cZ<%LHF++60bpL<3tQ3a>|mE%}!qO zNUdk-yr~4$m|r)xlpU@MMYNyD$Oa7>8h=%P_czxEO-c;a>0K z52{Q!0RcucJ`oBsK$|2GW2N?DSPS2Iu>C?XWkUc`JB>NtJbhrUhtvhQ=@TroCkeHK zW>bdH9vIs*d7dDQ+?6Xx_~WZI^T0V;P-;Uv(g?&peENl<*iSP*&w*s`8^izd1U8c) zI!S4^p8U^>yr~15m^b5gLU}+V*E|;TF<=|>n7It=s9RNaC8z;o5re5}dyVo8u@S}7 ze)3uTx(U&W!{%K`t_skI;koBDc<9*Fn$8HO*g7f%qn#zL22 zGzGRe^>aSwi5iYn@k3zxARj!j#-I4-9rZ{$8Dtz*p1jh<>2tqJiSaR&X%Rx>{h-E+NP zI<2Jk1Gv%@UAv08;>?J_WTYI9`Ro$YV1T=MBY-=)J4~^*m#5)=59SKbk)Tm|79pz z=u^DT2;BN%EY2;2N^pnofr2dj3dp3)9(z+7B1m?~!Amk$9{~Ddy_4qm0JB@+>fmLH zSfxD&g4TTJ%Nw8R#tVZ&RsvM9-Ca7EwG21H6L}t=2*a^%mJ7}hJ4nN98U&f|?za0Z zKWu0UfZ^_iA=|AllQN(UWbZrYc8cs&V%}Wxf_{;utxL!^6genZ;t&nTL;R$^k+81~ zfu`S8J4G-OgR|e)U*u>DdlckK=xNL`$vmOKcc^48bwIzJWERkhqkN<&wd(q$u!oiF z7D6nb{3BikVQxP=)3dIkSVe1$nwLd8A4X=cH~<+6&)hak+k|Gvkb?PPgY`RU(;eTN zGZz!F7y()-j9O`sW_9UHL8^z1sj9nzko-gs%)Az0HlGt+hxJ0vwUu};q{m(G2QkOI z^xXZTH6uw(UShLXpv4h4nQ`{}FTmj^ayq?M=m4W@e59*3h|c*%h7$;oBur12aL4

*M|O11K@vC%ANl*1u0NZoCWTwx$K?K%c!+Vq^O2a){iQ$F&r%OFb@nbk z0H$%O+6(g)0?(p;{4ij&G5Q4P3QBwjB}=UQua&DA5mIuSuA#SYot5V9;0))7k``9K zOBhQZ$7IyigJEzLxgb3fJ$o!WQTP=6CpaH?n5_!H9}r3rCeeU7a%w@3K|#=^p_i$t z=NFD{XaD%yY=h25{NhoFM)yYMh2QxB;T$Kz4oFjOj(Oa|#-!4XIbSq+g3nj#kc>+Y za&!N@@=KZAlq;Hur5+zkm^ADIj0r$tE1BfXTdS;A#*W#4Taz&3MA+njiT#KN{^q~C zyCx@Syr@aN&W;1Ry50&M0lMS_aYNmKr0%md(td;+*>FgCT{GatV^fDWM`Dq}?WJf_ zS2#sDHjwu?UZyi+4^#KNcT~G*(}@PO!wSIM$`P9)IQdW^5`;0JKudF%%t8`7EvHAW zavot~uP}d(IkB$HfuyK#jQCLR=Cy*LI3|+d1pQ*s8%@T805O=7j@jaMq3o{6n zsJ~#Jkl|a(4n|o?P>+Vt1!>~A^1G5l!lZ2hRfT+T7gh1wdGHbg3^;1@V zs=24zDm2&+Py$rZ?|@&}0yuK_*p7t!nT^Pl@!+Cd~CKNcR>$acX5^M_mZZO7qPNoj`|5cfd{qGCw z7$w*q7K#OU4Wp2V%On4lvb7KUUwc*E-G3fKPk#A;dIAg9{brcp84)v|81u!mV3uF7 zn-Y&y+-(-iBewOq4c~(Bh96inWK9jenJr!D{%J=Z!o=GP!@)b~ z+LM4Fw?2Z3d2BNKWo~>hO$-a15V*)$$9>=B!z-KG5p~nTTo!Nlv~@_x7T|5HuAJO7 zIz(_ga$#k8ve$>|=4*iUcsIuOQ*Jm}n6OTfxWp>% zFR2Y|BbN^Q>{A)7OOgRgKvs@)Az6Egd|3{ zgi~2G)3`(+Nz`L6HF}LaZVhFIRC(16N)A z%8ecH@};vjXD?r8pFSLHv-Z3FSNPveRYBB(Tso4t6nWq-f@fbB%VNa$j9LSKVZ&_e zZz0YUQP%vCtY7P_Qu(?(=5cN+9`ALm3B_1vJyG<9V1)zU2#QieW*^(fVxlzXa?u4c zT=Z_Ap_x9-%$$)4sm}T<04-{3S#9H2MLevW&%9<)JSL_5CE}VAL*<&L&M4U z2rxC3@U?eO+wI|bB`zWb_y{K4Ehmhw1n(!|!@#nUU!4<>MC*ZCAlLq1HKgdHHJq34%J3qBa-hAfD;mV1@FJ;+hIj5`$`ivD& ztAF(NE&>tFYBoJ1P$6?__ye|v(z#58q(JYPP)2f^&}(bMF&K)PigW4*Bd)wf8kr zN0}ZVba~)bij6V=E&_F8cor>V=cAdn;@w2|xc|Frb&E6SR58nQ9GLEME4OKp(+8jxq_pvc27qztyEP3TukR$j@0bHN z1N5rH$S<**PxAwA;rSHY3@|M=mwFssq@^$$BrL?Z;sQ8=ibIj>$Y2?4X35hv+|cH7 zwH8u^1j)fxX((eKZg3x04{}2+x{_QE3Ok85w{2p$yF=Rm$?{os$^0XzctGJa!IVc~ z$qPYoW>Gs)Q=2Q%%jDKAHF9;3ttxg8{=0eWx~C6p)#z;;)Q1!?*k(S|ywbV{sh|tR zBHvbJ#LJ$He|qPAJn@ztf`6!h&Kmodg%GbeQz#Lj0Wmv8>jLSWeHTLh@u{Pkze!H9 z9#_)I;-P|=Xpimz=umr9miwsN!Vhvx^^Nd!pSL26j+P$+ELmnt6o6#*OM zN-Ne-n*|#^AbZB?&KJ@pipZ}HQSwrTr~X^o(n+oU=2*Dz{qgGWet$HYtbk}nx`xN-josYB%_;PDNs+Kc? zQWSz|(p)OoGMbo=K$MBfS`po6pbs;-p!(>i1KV;Z0>;Al9f(on=dffiOFV-b(JS~$V}Zs3op2UbE4 zR;h}S{Wai{3jTEFU6_nsPr-=cFxWOnv>M@H65n5y?BXZ#Zi%yR#!CO2ZmHLmP7B_2 zAmd@;R=oz2tC7}BM5`pn+8^Sr@1BDpUKd`1L4ESl-<|d5bS8<_|1BRR56@uFkU0=U zA}yWuh2vF-;rr~sTY>eAOyNuF@CWba%CS)ud|g+5M10m`cQGZ)g>$ye0H6t=y!tzo z-V-VdfIOrx_LLNR57+fiuElc#viNn(EU@_0AQ*E$E1@Z60G6m6ibI`0*m{HtNeA_o zz6QvM1Y8SiG{1!}J$W8;acuC*mI`yaW6hPDiZg+D82PNGMcoJImope+WRyBk{S4N` z!(#NcF2~rV(`1V9rYmycgX^*koYk7vGE z;z;tve;<4Hm_G<_W*nL}p#^`J`E!Gbyj|v!huXZliQ7YNp9vPf&%oV$G1yH~*uCNyuKSS}zp#j?m+P0GOyHD9{p0j?`C*U(ibrvKIt;?7&|zIk2k zUk~3Nb>Y`iW2%R>YM#Oan#_AaEwwlU_J$Q} zqj(upViU1$znVjqdU`~n-|dz%*{&%k!JCKM@SNR=gHGD@t~#SUG4rp(9fWujL+Bo$ zqq^BkuMzFX^eg(2_jNEQFDxuS2HtLFpu zDICYJ!P50dE$EtNK{*@8%ZG!?&Pys_43+d097o~b%v@SbL2Ay!TS$r?8q!8H_pYMuchy19DL;XMGXC#|8|b|JMUxR-kJD6)PIx|bjfuZ zdR~1>y}J58b$xd{n_u)liA|{>MvWqg3wEkDx=Z5+<7v5Wt zO46hu<#d1OI<)hSTxfcKEQ1(&zR5|pAX0N(e)jFLW)|gkw6wrbE&4A_v)pr~hX+c( z{7=$hi1ep|fgJV!u~-C9&RhJq?{EvZgcYm8l=9qIWMdBzYLSyPUb)h$hyX;nCsdb) zvpni{gbA{eM7FjW&(v#W?A_~+o>tU<`gH+mvzDRbm1n5~I#w3O zlr)bMZO1N~P?o(P@ydiVXZJI#mZEsf8Bf3b0#tzgM9Y2@Uos5YevdJ)o$^m@xSyoS zMnBI|UGu_niz2Fmu=wtE>lA+Tht(B8y23qy>!w1_f5Db+l=Dv`!3*VtE5PBa_CIMr z85Qo=y;j02V%~h=5dlO4o9=Ux$ptkhuNbn*niTnrz+)9(@qRM#l61AHnm>GU&k^Wo z4sNN3Duk=8s%Of{J^S|Z7}WKVaf$+yyGUi$^4&ISn-wEAu-}|p0SK4yZP`uFW`M>~ zA*_*?LiKz{RaYxlbmKK^O6iNCGdmq>2X3u*yuNhcubto9$&$8i{0MRuQ5Ejm(yii2 zbJJr@z5Wk=gu2Jr*XMEKERy#eE6#%yXc(oh*ikS;S3LPP@*8TQF$t+19}4qvv5w$$ zU9X;4_<4ie`O)#3Ns`{y%Rvy+6Vx4&!83hKx`g^rcXVb(TY^d10PGBPe!r=pfaD5Z zy%ZLTwZ;c*H>JUL-^kwb@yZ-l*uS(#MtYoe$@5HebJ36P%t*Q`ma{s5H@}y4AZXES zYGraMpM>=FgaGg!ys#cAk-elwn@z#n9nEqmMZ8sv+qNFrUKk35jg5X={K6V)@_EiT zrPA59?w;@WY+bn_?Q(Vilo`$by( zS>lToIzkFlH49rPe9xjBuM33jTSg7G>}mKGUm&pHGT%Nd_tOVP8)fLrrWWh3PNX8U zckHJR^-1k@scyKtl|c){s@!K*;erGEw7_A4-RRM-ngD}?0-w3M@A44E3U|AtCF-Ro z0KCQlF*!^{`HTn$n$1tHro0$I9)<8n58nIY4utAIz2eoaYcGD`DN!d#I=s6v&Q^EF zn?8b~SqFT`9S4|DS$hc<<(YS_h%$s_qMSGJ`eI2jXLLM>PHDHIc468qK@5^9U&$47LXz3)Z4W^-*mpGXuk7IWl{d zRq3Hnpwg3&U~b0kD&?l>BAhcyrWG0i%EJ$Gwq$@^g;H&?tv{&w=m3uoYOOcs7u?ly zf07AcQARpZfi7pXr#_UNtfP1+KEVTNYtwZ%bYRsJsW!@^#Qm~2$jvL|M>`>Tq@S>bg z<7~sH!F(w6o(%+r#**#SJ(A<4l+Km^`7I+pL%*)IiC6kJ4nz1;9t56G)yNek|ixt#DAGKTDU z?NR@kf*Nzmu4a19efuLA@CK7;OYA&TJTSK?w<#mwBqA!vPdt!LCvx^1Yph+2}a+gst6Cer0Hi zK+R{wyt9kR8bSBw%%8U&%F^~1r>~WuBDQ{{cUbx&nTTA-2VP%Aqc#wqc|zxsZvhi6 zDs0|@S(tQ!Jp|SH{oC#f@94IdaUlPJYiZttT zG`q^Se#aE7dOJ^T+jKw>k#4ctuxs(_GRcOliG13<#PI+|`o=Xbpzw2!WBcz3lWpgj zYMT}hJd`A$=h*q0eLSm>lGmHZZI=tJMf^DilSZV!D_$ZK-|fpl)+l_5!_uuK$>b;x zq}8eu3Cc{nit?p)bx>0H0?O-8zKY-SV$1@AjY{sXf>l9RA7f}_7qb9B5Bz)?PjSjT}_l2{aLfgCCmle}#5Slce9%gegkNxRg@Ehc&2DY1{{+VG;VcOF;BE z4}4QWDSmY*ij5F4z{@qTJ!OqE(1}5fCQW{0>CenLxl|hiZ-Ym_1WOUVbc05VQaCVl zbr;6c?_e=`E0f4lZFYC^WI9TH=wa2DV9_J3JbjGU4Bo9`PZEk4h=dY918iQb0 z{j8MyGpO6rgLurj?x^!%iahTb>?*DIReRj)IiIQB)dwk}@Jn#E*?x;x!(Nb#B6-(V z^{~s5dVd-j!+8Ng}UFt^V>TdZQ0S z8+?i}ZH6loT7~!tt6kRkMxDOLWKl7#yxZk{3=~Rn;#N)3E6x)-0diQ^;7O&iPA`Q; z=%p0a;}-hw&7H?f%LAK$^O(ov-<8`p7&2T`+x!@`ok~Aa@iMRoe~RGF&68|X<$-s2 z3z)4qPL|0d{usv|L=M-C>Ku9M5M`o4E;`NaDf>I`yP7;(kI(wS^#p0I~n)SUsT|=zLpFN-)n=(+vYrWl#B!rgT@naxp z5yr93@OU7JNDPPgL%VGU2MM;YF^bq+gVxSu5U2H!jYUm*h14^b#L#jWHzMk35~-vo2Tc^|C49_ZdgN1uK!o z@i%nqZq_&$xrCKLCEyY~ZQ~#ZNsY$gbrOX(B?j`%O^BoC;eTpWs?%esu8>GY0G#G8 zOMRmLU9oSz-BVH<2+{IYbE4WRBhb3I42UMr%{!OJwU%0M%_6PCrXYEQ8zN~S|)%AHSBG8-iJb_oxnoI z!Y`)vDq52$4l|);mctu)2_HeCj|`Oge|~A!R@t|T`UsfjEi_L3Og&NEs-A0BSa{_y(}R4l70F|1PR~#GheeLnG_f|25%&@B$s9g5+q*#M z51i(@FA@G^sn6`I7IjEXg+=xvpPHynLAe*IJAc6b%Fd?|soS^=S5T`xw&jQ9L&doC zB?Aa4Cw3MS-NK546robRkCfbGFsg9SdN1te;jov#SoW|K-c+SDzbJ* zDZ^GTey03PL~hlreqZ>XG=?_&X?zU5%ww9`_4ab}2Q{0cNrFev`Z`Y! z$v_Gg$VeH1_jo6+?J<5p2vIq^JAvQSk=09s^TTT`!cLwUDffQ+Y{Zx0su@}H`$~d4 zT=N3cC}Hv7+oUrv^J)*Wrc)^~>_=2`1f6ESm=GDX7?F3bT~c(n8t_pChmbfk(k|6s z=06%cdRkAvoaCftum4_Vi7oPq4;n-bT$8Lz>XS{_lY9w53EQo zgP)5G+H~1d6^bz9Cc#N}x*`rk$d^go%H@372DF6&}$V$`d*$Mfv+?%qj%chfaTJn6ZQ`A03v?Kk$HqCSbs ziSy+6a{aZ4H~||DzAo``gCBcSzF$y{)tMM-Cs<>rd zfwNPYN;z+(!4s}Lip7ZvBjt&=if{s4{Z;p`j^-Jn9ciE|@-&K#Vs(IR8+g@(;q_aQ zqHGw3*^fH!lC4-ZM+|SpvmN$PWpXFNJg3`AVw{#kgyWYGyp+7Sa>b3p2UO2ImlWl5 zW(FE!9#rC;pTKK286jO7JV%Jo0=ux%1UPn{KtAZaH>DBd2p%tlR5`h9BDUx-DDRyE^@mifi)v)5-0dT2;dYmY)uI&9QSv#~$wV>%5#IjfyY?d92yQP)#=Z>T;K z^=DerB7DGp$a$<#0zP7VYEo!2QYvs5e%uVhmu@nbXGG)DavfX4FFTo)@|kzluwqnz zjGgj8ruT$0HLLw^AcU9a!blTERinyt+XIvco4yidw2=Y50Xmg}6AI5mzT`DTIBEt&LCO zsBIiu{ft-S{4vxaot`fZx+pYlI-GZJQnB12v({;)TX1ZD0=SgLzz$k7Bn*9-cnK?5 zR;*yljKL9=G}_%beKu+wt}jNrIsEsVkPD<8YMYt%H06Djs5*+c+}O%@E@W2nov7bb zshJ78km9tb?T0JSUoS$af?@H|Z~f%rqp^<0x40XakFL%g*%_f)`~trITF}vY6T&mw zl$7Xau>3pFN}++G>O`f@N-yJx2c7NtQm+~Wr2x)sYI(>{#L(<|B$ZBwzydc;wuarE zwX>11c-6)3K)fS0`PRGL1};*@z~qIK7?g?dkuEpq*MYZ+Duy{W*qQE2j_0HWGjRR|`qNSwk~s;)^h7Y(^aAlM1b^3M&Gd_fAJAJC##gi>4`XlCRp>P zm}x)kJy%9^{A<{%Ac2m>&xJ7Q0H#jHJQl4LO@#AT0V$cPJ&EkLc zrCC|e;qSNmAFENWlvXM#Q&rpqbr}&LvCQg}@HgNVrSqX6LpmtfP+v-vbi-dN-q7a0 zQDysK2@qPHWABX9F)-1_vL;=Y&qH$6UmdHK>VDP>a=t4!-3`9MedsEUxo~8|U2rUW zusYH9_yQE^>8worg_G0cfb zZGT2teb>}~6OkgwzMZI+1PaHekb43Sf;gwsUwssU{C(5pup|O^lkM#`*#2@e{HQ{M z-sCaOS@sG>iZHZw*^MtWDi!p`NN1?a1ZlhODqo*OwuOv?hV`YyNd9x71z`oLnj1+* zj(ZWid_l!5;S+-w{h3M$1yncnnM7vUR5-q9i7yzvG*h;}w&Mo*09SL>`;|DFhuE1` zP#Tui0T&L2X+x22;ot&<^IUsdK=O{au}&q=25t?zMxO~DPYdA>={fkV7PTf`pXbzd zjjw7^$co3FV|zRIrMm!4Q`+@qE*%{3W#n=(o=DhQo{jFim=MYJrP#u!d(28A^%i|k zpG)oGW8Z;a<(+{;NfK@bIlMN<6Y>Tzkkwf z5cTOu4r;3m8(l5gsWG18Jt1$8#KCHn0iRjZ@gmi6dF=e=4ID8bGvWSIhWiqyo$&ED z&m)h0X9w9!6Lv1f%kD|3hOHaY9QL2HzLCZ5f8lL7(5(R~{@VYNqzIMGlvaF_aM~kS zzrBI})Gr&xW<0~sumrZKnk$bUI9A+lWvzZe;CY_^{PwXr>^*BUejZ`{RJ-?}>0-5u zFg4^yU)9K~Zrp|4yYGDu&(NZeY)yN!BGa^~PXB(j51-r+7;?jfMdjmqV8>~*@y|Kq zi8||l0rKXN;lhW+zUNa|FR|xWC*l@c8oPHgwauiD*od55T z!$RX6`1CYt8+^N;YMSd6cUBA!d_)Ik`NFb({o9aXjCp!k$VC4aD(B}od#N6d9sib{ zE9h%1uLc+73d|)uztjV%OLY%ek$4iyv93lyO0^+z84)LdHvU3=TATwC{3K075}U{7GAkdJor9`zN@>n66ewE!B2i z^ByBVr4@&P^d_wl&A^+h4gz8TGZy)dW(3s39?Gwr!od6Yyc}!dnr)9O*qcsUOT=g6 zfmF~boIS+9oK5Gzu!#}^*aq1qo+Fu`m=;LtvS*W@uVU6khnbtEhxRlN4S*mHl5+El zyZt9|Hk5D+duj~WFw-W6FcR@d!FUFRTmA}ZjZq^sXzf^-g5z`x)+CfAfpF_in$Xd@ zsg_GFCVUkIs%NheFqN~rys8cx-@qN^mBVY~H2#pv3(N5KsXCTciYlvMc2dR>6>;5% z7QiTh0}B*7QJSjZlEo6dA71TUfCy}zi7o!|Se`Q&AI9~1b<_1y9+{FIp4Y_#1+m1g zkQ`f^(4OX9UQ6}Nd|h9w9VNMv+~Mu4;xtpMnJ)@06%Pzln!~Z` z{ObcIt^cW~#QSO58n@eGOCwtz<`6Hc_{$S@G_~x~VfY0_f+c%4gv|!UtE{f~V}7vC z%A8!1O*oy&%%!%Oo6Aoszh#(SvL`;a%2ZRZTJ2Nt$4e{Tyf{sxQeFelT1z9}llMA~ z1++%qoCJ_Ok)lOSAqs_;<2{*) zA-mdmmZo|50gLOzr}g8(I!OHPMzA{_G6$x*Yn11=o`iB z@hdTRFJ?ELi1r?bEdO5c8XpIQf0}rTM$f*rAx9Gg&&U5Prk(1;lci@|B*uXk(kaUJ zN9sFiwU2IVvn69*K=+y`48OETOE^h>rpRMCPz2DkXk-Aw#pn!ty0*jf{zt70=SLEI zmBB$vYC>U09h}q-8rW=VCksH{9rn}G3OzD?M4Ufvhd{Fm6oS3 zsBOLv=35D657;A=DBQ^{uK@>+>L&!_xYO3?mnMQzMeL76&|X;<&PV9w->4clAS@2G zfI9)Yb=iS@gRBprQ&HK>_W1^GU2|Br+qW(#WYb2o7%e{GO)F4&ZHMTAG>~2eS~b?+ z^!|E{+)dCWXah!@^*=LR^nxh5f`*fBI;uF`Utx#z&Jp80h5ha-`QNPh*QD9iTZn|7 zzp)4+Yh8;pPlJi?7f?p%$SLxK5a92mm-?VG)IHr`mWC)^_kobJ+rw&h*wX5)Wv7mO zGc;xV5)e~LjBHsx8!hsf`dP7XkW(q0KaB}QCiH0(rTuOxD5kcl{Eu&5iCM@`@SN;E z*UOfjZcE5dna_bU!^Rc%yAa$A)m$w^rzT=bA=jhW9?`8;M`o`Jo-%+2_U!J72okDuN|za`mstT>i{*VnH&DA=y&dc zD^bDea7H)*81idkI}~U|wTLnJ*>X+7xz3i>fQB^p%4&auZ-~V>hmF%&YkP?CCl; zsJYnfX#0>)JBO?TxkcG+q?aD{8ACpVMVKzW$|flM)1+ies!#fWmRZb?m*_zbd0scD zZ(kmx!AeqmpngPXylO2O|qxFHm4G&5Yf@dNv z*7bLh8EF=R1+-hjezL^wcYayriiYPxuWB!fMODF{nCP+cGA4nTLEnvTndQ_7w-xqv z!?uLmRDs+nm44o1tz^ISfzWIra3D%)YG2wl-|4h>#_Bi{ww>xBVx%;NC;SsnR5Qx;swI(>9or5eOf3bOGZdGFUxIm#GVvd4~1x?Ff@CRoaCZG1kM@@dkcE*+V_=cJ<8GJ^avg zJ7Jp1c0+oT#+rI5p)h{YkZke`u5q*fm9D57+6{=E9wfr_D1^oN)%YnlH_jZs`x|0I z2{)pC6ExLr-cp@?Umrfl1%|KUawf_gpdzDAabN`R?c>9(%5hKSH2k@E1*n>)K&as{89_!w#Sqw;fCe69IzG-ynXKv4J(pZ4cuZ8|PYBieEX_nxrTwbu#88#r@^{1q^Ev_ZN z$4cj<9;fo&{FdNe_1+Bd&;+oc{y(3}xxcjH#;;!bOjEj2P*8{j=-LF_yBpx9?&|Lb z{7@*!DX2)xDN4(!Sjfw%D*zuANjW)nIXN%Q_VNGI;DPTwPgLmtzrm%uln;Rh6bM}t Ko!T47sQ&@Au$>?P literal 0 HcmV?d00001 -- 2.34.1 From 97da39315702ef5bd0ecb0b503003e39bdcb75ba Mon Sep 17 00:00:00 2001 From: Dao007forever Date: Sun, 24 May 2026 23:35:34 -0700 Subject: [PATCH 125/382] [Doc] Update snapshot object store docs (#2148) --- docs/source/deployment/mooncake-store-deployment-guide.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/deployment/mooncake-store-deployment-guide.md b/docs/source/deployment/mooncake-store-deployment-guide.md index 27fd498f..58d56a4d 100644 --- a/docs/source/deployment/mooncake-store-deployment-guide.md +++ b/docs/source/deployment/mooncake-store-deployment-guide.md @@ -54,10 +54,10 @@ This page summarizes useful flags, environment variables, and HTTP endpoints to - `--snapshot_interval_seconds` (uint64, default `600`): Interval in seconds between periodic snapshots of master data. - `--snapshot_child_timeout_seconds` (uint64, default `300`): Timeout in seconds for each snapshot child process. - `--snapshot_retention_count` (uint32, default `2`): Number of recent snapshots to keep. Older snapshots beyond this limit will be automatically deleted. - - `--snapshot_backend_type` (str, required when snapshot enabled): Snapshot storage backend type: `local` for local filesystem, `s3` for S3 storage. + - `--snapshot_object_store_type` (str, required when snapshot enabled): Snapshot object store type: `local` for local filesystem, `s3` for S3 storage. - `--snapshot_backup_dir` (str, default empty): Optional local directory for snapshot backup. If empty (default), local backup is disabled. When set, it serves two purposes: (1) during snapshot persistence, data will be saved locally as a fallback if uploading to the backend fails; (2) during restore, downloaded metadata will also be saved to this directory as a local backup. - `--enable_snapshot_restore` (bool, default `false`): Enable restore from the latest snapshot at master startup. - - **Environment variable** `MOONCAKE_SNAPSHOT_LOCAL_PATH` (**required** when `--snapshot_backend_type=local`): Persistent directory path for local snapshot storage. This variable **must** be set before starting the master; there is no default value. Example: `export MOONCAKE_SNAPSHOT_LOCAL_PATH=/data/mooncake_snapshots`. + - **Environment variable** `MOONCAKE_SNAPSHOT_LOCAL_PATH` (**required** when `--snapshot_object_store_type=local`): Persistent directory path for local snapshot storage. This variable **must** be set before starting the master; there is no default value. Example: `export MOONCAKE_SNAPSHOT_LOCAL_PATH=/data/mooncake_snapshots`. > **Warning: Managed Directory** > -- 2.34.1 From ca7e5fbf95e66bb1825d1b319ee28bfc03d73ec5 Mon Sep 17 00:00:00 2001 From: Liu Hao <49090335+A-Liuhao@users.noreply.github.com> Date: Mon, 25 May 2026 14:42:59 +0800 Subject: [PATCH 126/382] python api register_memory support location param (#2191) Co-authored-by: A-Liuhao --- .../transfer_engine/transfer_engine_py.cpp | 18 ++++++++++++------ .../transfer_engine/transfer_engine_py.h | 6 ++++-- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/mooncake-integration/transfer_engine/transfer_engine_py.cpp b/mooncake-integration/transfer_engine/transfer_engine_py.cpp index 63e23af7..a125760e 100644 --- a/mooncake-integration/transfer_engine/transfer_engine_py.cpp +++ b/mooncake-integration/transfer_engine/transfer_engine_py.cpp @@ -735,7 +735,8 @@ int TransferEnginePy::transferCheckStatus(batch_id_t batch_id) { } int TransferEnginePy::batchRegisterMemory( - std::vector buffer_addresses, std::vector capacities) { + std::vector buffer_addresses, std::vector capacities, + const std::string& location) { pybind11::gil_scoped_release release; auto batch_size = buffer_addresses.size(); std::vector buffers; @@ -743,7 +744,7 @@ int TransferEnginePy::batchRegisterMemory( buffers.push_back( BufferEntry{(void*)buffer_addresses[i], capacities[i]}); } - return engine_->registerLocalMemoryBatch(buffers, kWildcardLocation); + return engine_->registerLocalMemoryBatch(buffers, location); } int TransferEnginePy::batchUnregisterMemory( @@ -757,9 +758,10 @@ int TransferEnginePy::batchUnregisterMemory( return engine_->unregisterLocalMemoryBatch(buffers); } -int TransferEnginePy::registerMemory(uintptr_t buffer_addr, size_t capacity) { +int TransferEnginePy::registerMemory(uintptr_t buffer_addr, size_t capacity, + const std::string& location) { char* buffer = reinterpret_cast(buffer_addr); - return engine_->registerLocalMemory(buffer, capacity); + return engine_->registerLocalMemory(buffer, capacity, location); } int TransferEnginePy::unregisterMemory(uintptr_t buffer_addr) { @@ -1100,10 +1102,14 @@ PYBIND11_MODULE(engine, m) { .def("write_bytes_to_buffer", &TransferEnginePy::writeBytesToBuffer) .def("read_bytes_from_buffer", &TransferEnginePy::readBytesFromBuffer) - .def("register_memory", &TransferEnginePy::registerMemory) + .def("register_memory", &TransferEnginePy::registerMemory, + py::arg("buffer_addr"), py::arg("capacity"), + py::arg("location") = kWildcardLocation) .def("unregister_memory", &TransferEnginePy::unregisterMemory) .def("batch_register_memory", - &TransferEnginePy::batchRegisterMemory) + &TransferEnginePy::batchRegisterMemory, + py::arg("buffer_addresses"), py::arg("capacities"), + py::arg("location") = kWildcardLocation) .def("batch_unregister_memory", &TransferEnginePy::batchUnregisterMemory) .def("get_local_topology", &TransferEnginePy::getLocalTopology, diff --git a/mooncake-integration/transfer_engine/transfer_engine_py.h b/mooncake-integration/transfer_engine/transfer_engine_py.h index a7d6bde0..c889c16d 100644 --- a/mooncake-integration/transfer_engine/transfer_engine_py.h +++ b/mooncake-integration/transfer_engine/transfer_engine_py.h @@ -169,13 +169,15 @@ class TransferEnginePy { } // FOR EXPERIMENT ONLY - int registerMemory(uintptr_t buffer_addr, size_t capacity); + int registerMemory(uintptr_t buffer_addr, size_t capacity, + const std::string &location = kWildcardLocation); // must be called before TransferEnginePy::~TransferEnginePy() int unregisterMemory(uintptr_t buffer_addr); int batchRegisterMemory(std::vector buffer_addresses, - std::vector capacities); + std::vector capacities, + const std::string &location = kWildcardLocation); int batchUnregisterMemory(std::vector buffer_addresses); -- 2.34.1 From 1fd9699ec152410745aeec9195c2ca90dba809f0 Mon Sep 17 00:00:00 2001 From: leonzzhu Date: Tue, 26 May 2026 10:55:38 +0800 Subject: [PATCH 127/382] [Bugfix][Store] Fix snapshot failure when OpLog has never been written (#2144) ResolveSnapshotSequenceId() only checks for ETCD_KEY_NOT_EXIST, but EtcdOpLogStore::GetLatestSequenceId() translates that error code into OPLOG_ENTRY_NOT_FOUND before returning. Add the missing error code to the sentinel check so that an uninitialized OpLog is correctly treated as sequence_id=0. Signed-off-by: leonzzhu --- mooncake-store/src/master_service.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 00f2d4a6..e28aa306 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -3701,7 +3701,7 @@ MasterService::ResolveSnapshotSequenceId() const { uint64_t sequence_id = 0; auto err = oplog_store.value()->GetLatestSequenceId(sequence_id); - if (err == ErrorCode::ETCD_KEY_NOT_EXIST) { + if (err == ErrorCode::OPLOG_ENTRY_NOT_FOUND) { return ha::OpLogSequenceId{0}; } if (err != ErrorCode::OK) { -- 2.34.1 From cb032b24d6de8aa41728c61fa072e0c96e29718b Mon Sep 17 00:00:00 2001 From: Aoi Date: Tue, 26 May 2026 12:28:53 +0800 Subject: [PATCH 128/382] [Build] Fix compile warnings across multiple components (#2193) - mooncake-transfer-engine: add parentheses around && within ||, add static_cast for narrowing, mark unused function [[maybe_unused]] - mooncake-store: fix member reorder warnings, add std::ignore for unused results, fix missing field initializers, mark unused variables - mooncake-integration: fix sign-compare comparison, mark unused functions [[maybe_unused]] - All fixes are semantic-preserving (no behavior changes) --- mooncake-integration/store/store_py.cpp | 2 +- .../store/store_py_internal.h | 9 ++++---- mooncake-store/include/client_buffer.hpp | 2 +- .../include/ha/snapshot/snapshot_logger.h | 2 +- mooncake-store/src/client_buffer.cpp | 4 ++-- mooncake-store/src/client_service.cpp | 8 ++++++- .../redis/redis_snapshot_catalog_store.cpp | 4 ++-- mooncake-store/src/master_service.cpp | 4 ++-- .../tests/ha/oplog/ha_recovery_test.cpp | 1 + mooncake-store/tests/master_service_test.cpp | 4 ++-- mooncake-store/tests/storage_backend_test.cpp | 2 +- mooncake-store/tests/task_manager_test.cpp | 23 +++++++++++-------- .../src/transfer_engine_impl.cpp | 4 ++-- .../transport/tcp_transport/tcp_transport.cpp | 3 ++- .../tests/rdma_transport_test.cpp | 2 +- 15 files changed, 43 insertions(+), 31 deletions(-) diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index ccb25d9d..c8247179 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -50,7 +50,7 @@ struct PyTensorInfo { // Check dtype is within valid range (0 to TensorDtype::NR_DTYPES, // excluding UNKNOWN=-1) if (validated.header.dtype >= - static_cast(TensorDtype::NR_DTYPES)) { + static_cast(TensorDtype::NR_DTYPES)) { return false; } diff --git a/mooncake-integration/store/store_py_internal.h b/mooncake-integration/store/store_py_internal.h index 0734bede..a9a03c5c 100644 --- a/mooncake-integration/store/store_py_internal.h +++ b/mooncake-integration/store/store_py_internal.h @@ -526,7 +526,7 @@ std::optional materialize_shard_tensor(const py::handle &tensor, .attr("contiguous")(); } -std::optional> build_tp_shard_infos( +[[maybe_unused]] std::optional> build_tp_shard_infos( const py::handle &tensor, int tp_size, int split_dim, const std::function &key_for_rank, const std::vector &axes = {}) { @@ -1334,9 +1334,10 @@ std::pair calculate_shard_range(int64_t dim_size, int rank, return {start, end - start}; } -std::optional parse_tensor_metadata_from_buffer( - BufferHandle *buffer_handle, char *usr_buffer, int64_t data_length, - bool *take_ownership, char **exported_data, size_t *total_length) { +[[maybe_unused]] std::optional +parse_tensor_metadata_from_buffer(BufferHandle *buffer_handle, char *usr_buffer, + int64_t data_length, bool *take_ownership, + char **exported_data, size_t *total_length) { if (!buffer_handle && !usr_buffer) return std::nullopt; if (buffer_handle && usr_buffer) return std::nullopt; diff --git a/mooncake-store/include/client_buffer.hpp b/mooncake-store/include/client_buffer.hpp index 44bb6f06..9a9553d6 100644 --- a/mooncake-store/include/client_buffer.hpp +++ b/mooncake-store/include/client_buffer.hpp @@ -57,6 +57,7 @@ class ClientBufferAllocator ClientBufferAllocator(void* addr, size_t size, const std::string& protocol); void* buffer_; + size_t buffer_size_; bool use_hugepage_ = false; private: @@ -66,7 +67,6 @@ class ClientBufferAllocator std::shared_ptr allocator_; std::string protocol; - size_t buffer_size_; bool is_external_memory_ = false; }; diff --git a/mooncake-store/include/ha/snapshot/snapshot_logger.h b/mooncake-store/include/ha/snapshot/snapshot_logger.h index 00f9d7f1..9d54f8b7 100644 --- a/mooncake-store/include/ha/snapshot/snapshot_logger.h +++ b/mooncake-store/include/ha/snapshot/snapshot_logger.h @@ -16,7 +16,7 @@ inline void SnapshotLogWrite(const char* level, const std::string& msg) { if (g_snapshot_log_pipe_fd < 0) return; std::string line = std::string("[") + level + "] " + msg + "\n"; // write() is async-signal-safe, won't deadlock after fork - ::write(g_snapshot_log_pipe_fd, line.c_str(), line.size()); + std::ignore = ::write(g_snapshot_log_pipe_fd, line.c_str(), line.size()); } // Macros for child process logging - writes to pipe instead of glog diff --git a/mooncake-store/src/client_buffer.cpp b/mooncake-store/src/client_buffer.cpp index c1f62816..e9342086 100644 --- a/mooncake-store/src/client_buffer.cpp +++ b/mooncake-store/src/client_buffer.cpp @@ -27,7 +27,7 @@ std::shared_ptr ClientBufferAllocator::create( ClientBufferAllocator::ClientBufferAllocator(size_t size, const std::string& protocol, bool use_hugepage) - : protocol(protocol), buffer_size_(size), use_hugepage_(use_hugepage) { + : buffer_size_(size), use_hugepage_(use_hugepage), protocol(protocol) { if (size == 0) { buffer_ = nullptr; allocator_ = nullptr; @@ -50,7 +50,7 @@ ClientBufferAllocator::ClientBufferAllocator(size_t size, ClientBufferAllocator::ClientBufferAllocator(void* addr, size_t size, const std::string& protocol) - : protocol(protocol), buffer_size_(size) { + : buffer_size_(size), protocol(protocol) { buffer_ = addr; is_external_memory_ = true; allocator_ = mooncake::offset_allocator::OffsetAllocator::create( diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 680d7c66..30abd6af 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -2,6 +2,7 @@ #include +#include "allocator.h" #include "segment.h" #include @@ -2740,7 +2741,12 @@ void Client::PutToLocalFile(const std::string& key, << ", triggering PutRevoke for disk replica"; pinned_buffer_pool_->Release(buf); // Must revoke to avoid phantom replica in master - master_client_.PutRevoke(key, ReplicaType::DISK); + auto revoke_result = + master_client_.PutRevoke(key, ReplicaType::DISK); + if (!revoke_result) { + LOG(ERROR) + << "Failed to revoke put operation for key: " << key; + } return; } value.append(buf.data, slice.size); diff --git a/mooncake-store/src/ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.cpp b/mooncake-store/src/ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.cpp index 221cb57e..8b04f855 100644 --- a/mooncake-store/src/ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.cpp +++ b/mooncake-store/src/ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.cpp @@ -24,6 +24,8 @@ using common::redis::IsStringReply; using common::redis::RedisReplyPtr; using common::redis::SanitizeHashTagComponent; +#ifdef STORE_USE_REDIS + tl::expected ParseSnapshotScore( std::string_view snapshot_id) { if (!snapshot_catalog_store_detail::IsValidSnapshotId(snapshot_id)) { @@ -68,8 +70,6 @@ tl::expected LoadSnapshotDescriptor( return descriptor.value(); } -#ifdef STORE_USE_REDIS - constexpr char kPublishSnapshotScript[] = R"LUA( redis.call('ZADD', KEYS[1], ARGV[1], ARGV[2]) redis.call('SET', KEYS[2], ARGV[2]) diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index e28aa306..0e7305e1 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -151,10 +151,10 @@ MasterService::MasterService(const MasterServiceConfig& config) config.snapshot_catalog_store_connstring), put_start_discard_timeout_sec_(config.put_start_discard_timeout_sec), put_start_release_timeout_sec_(config.put_start_release_timeout_sec), - task_manager_(config.task_manager_config), cxl_path_(config.cxl_path), cxl_size_(config.cxl_size), - enable_cxl_(config.enable_cxl) { + enable_cxl_(config.enable_cxl), + task_manager_(config.task_manager_config) { if (enable_snapshot_ || enable_snapshot_restore_) { try { auto object_store_type = diff --git a/mooncake-store/tests/ha/oplog/ha_recovery_test.cpp b/mooncake-store/tests/ha/oplog/ha_recovery_test.cpp index 7775016a..d9fb1a8f 100644 --- a/mooncake-store/tests/ha/oplog/ha_recovery_test.cpp +++ b/mooncake-store/tests/ha/oplog/ha_recovery_test.cpp @@ -259,6 +259,7 @@ TEST_F(HaRecoveryTest, GC_NoSnapshot_IncompleteRecovery) { // This demonstrates data incompleteness without snapshot protection: // the applier sees a gap it cannot fill (seq 1~9 are GC'd). size_t applied = applier_->ApplyOpLogEntries(entries); + EXPECT_EQ(applied, 0); // Wait for gap timeout so pending entries eventually drain std::this_thread::sleep_for(std::chrono::seconds(4)); diff --git a/mooncake-store/tests/master_service_test.cpp b/mooncake-store/tests/master_service_test.cpp index ac57417f..050e3d46 100644 --- a/mooncake-store/tests/master_service_test.cpp +++ b/mooncake-store/tests/master_service_test.cpp @@ -4431,8 +4431,8 @@ TEST_F(MasterServiceTest, DrainJobSchedulesMoveTaskAndConvergesToDrained) { TEST_F(MasterServiceTest, CancelDrainJobRestoresSegmentStatus) { auto service_ = std::make_unique(); - const auto ctx0 = PrepareSimpleSegment(*service_, "segment_0", 0x300000000, - kDefaultSegmentSize); + [[maybe_unused]] const auto ctx0 = PrepareSimpleSegment( + *service_, "segment_0", 0x300000000, kDefaultSegmentSize); [[maybe_unused]] const auto ctx1 = PrepareSimpleSegment( *service_, "segment_1", 0x400000000, kDefaultSegmentSize); diff --git a/mooncake-store/tests/storage_backend_test.cpp b/mooncake-store/tests/storage_backend_test.cpp index db2d7d13..1c66f376 100644 --- a/mooncake-store/tests/storage_backend_test.cpp +++ b/mooncake-store/tests/storage_backend_test.cpp @@ -2343,7 +2343,7 @@ TEST_F(StorageBackendTest, // Deleter thread: calls DeleteBucket std::thread deleter_thread([&]() { delete_started.store(true); - auto delete_result = storage_backend.DeleteBucket(bucket_id); + std::ignore = storage_backend.DeleteBucket(bucket_id); delete_completed.store(true); }); diff --git a/mooncake-store/tests/task_manager_test.cpp b/mooncake-store/tests/task_manager_test.cpp index ad094519..663a4144 100644 --- a/mooncake-store/tests/task_manager_test.cpp +++ b/mooncake-store/tests/task_manager_test.cpp @@ -269,7 +269,7 @@ TEST_F(ClientTaskManagerTest, PruneExpiredTasksProcessingTimeoutFreesSlot) { } TEST_F(ClientTaskManagerTest, SerializerRoundTrip) { - ClientTaskManager manager({10000, 10000, 10000, 0, 0}); + ClientTaskManager manager({10000, 10000, 10000, 0, 0, 0}); UUID client_id1 = generate_uuid(); UUID client_id2 = generate_uuid(); @@ -277,11 +277,13 @@ TEST_F(ClientTaskManagerTest, SerializerRoundTrip) { auto t1 = manager.get_write_access().submit_task_typed( client_id1, - ReplicaCopyPayload{.key = "pending_key", .targets = {"seg1"}}); + ReplicaCopyPayload{ + .key = "pending_key", .source = "", .targets = {"seg1"}}); auto t2 = manager.get_write_access().submit_task_typed( client_id2, - ReplicaCopyPayload{.key = "processing_key", .targets = {"seg2"}}); + ReplicaCopyPayload{ + .key = "processing_key", .source = "", .targets = {"seg2"}}); auto t3 = manager.get_write_access().submit_task_typed( client_id1, ReplicaMovePayload{ @@ -303,9 +305,9 @@ TEST_F(ClientTaskManagerTest, SerializerRoundTrip) { manager.get_write_access().pop_tasks(client_id2, 2); // Complete t1 as SUCCESS, t2 as FAILED - auto ec1 = manager.get_write_access().complete_task( + std::ignore = manager.get_write_access().complete_task( client_id1, t1.value(), TaskStatus::SUCCESS, "Done"); - auto ec2 = manager.get_write_access().complete_task( + std::ignore = manager.get_write_access().complete_task( client_id2, t2.value(), TaskStatus::FAILED, "Failed"); // Get original tasks @@ -320,7 +322,7 @@ TEST_F(ClientTaskManagerTest, SerializerRoundTrip) { ASSERT_TRUE(serialized.has_value()); // Deserialize into new manager - ClientTaskManager manager2({10000, 10000, 10000, 0, 0}); + ClientTaskManager manager2({10000, 10000, 10000, 0, 0, 0}); TaskManagerSerializer serializer2(&manager2); auto result = serializer2.Deserialize(serialized.value()); ASSERT_TRUE(result.has_value()); @@ -377,13 +379,13 @@ TEST_F(ClientTaskManagerTest, SerializerRoundTrip) { } TEST_F(ClientTaskManagerTest, SerializerEmptyManager) { - ClientTaskManager manager({10000, 10000, 10000, 0, 0}); + ClientTaskManager manager({10000, 10000, 10000, 0, 0, 0}); TaskManagerSerializer serializer(&manager); auto serialized = serializer.Serialize(); ASSERT_TRUE(serialized.has_value()); - ClientTaskManager manager2({10000, 10000, 10000, 0, 0}); + ClientTaskManager manager2({10000, 10000, 10000, 0, 0, 0}); TaskManagerSerializer serializer2(&manager2); auto result = serializer2.Deserialize(serialized.value()); ASSERT_TRUE(result.has_value()); @@ -394,12 +396,13 @@ TEST_F(ClientTaskManagerTest, SerializerEmptyManager) { } TEST_F(ClientTaskManagerTest, SerializerReset) { - ClientTaskManager manager({10000, 10000, 10000, 0, 0}); + ClientTaskManager manager({10000, 10000, 10000, 0, 0, 0}); UUID client_id = generate_uuid(); auto t1 = manager.get_write_access().submit_task_typed( - client_id, ReplicaCopyPayload{.key = "key1", .targets = {"seg1"}}); + client_id, ReplicaCopyPayload{ + .key = "key1", .source = "", .targets = {"seg1"}}); ASSERT_TRUE(t1.has_value()); TaskManagerSerializer serializer(&manager); diff --git a/mooncake-transfer-engine/src/transfer_engine_impl.cpp b/mooncake-transfer-engine/src/transfer_engine_impl.cpp index fb6dc512..bbcfb8dd 100644 --- a/mooncake-transfer-engine/src/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/src/transfer_engine_impl.cpp @@ -311,8 +311,8 @@ int TransferEngineImpl::init(const std::string& metadata_conn_string, } #else - if (local_topology_->getHcaList().size() > 0 && - !getenv("MC_FORCE_TCP") || + if ((local_topology_->getHcaList().size() > 0 && + !getenv("MC_FORCE_TCP")) || getenv("MC_FORCE_HCA")) { // only install RDMA transport when there is at least one HCA Transport* rdma_transport = nullptr; diff --git a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp index e439d903..83e213a3 100644 --- a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp +++ b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp @@ -945,7 +945,8 @@ void TcpTransport::startTransfer(Slice* slice) { } if (enable_connection_pool_) { // Remove the connection from pool if it was pooled - ConnectionKey key{meta_entry.ip_or_host_name, desc->tcp_data_port}; + ConnectionKey key{meta_entry.ip_or_host_name, + static_cast(desc->tcp_data_port)}; std::lock_guard lock(pool_mutex_); auto it = connection_pool_.find(key); if (it != connection_pool_.end()) { diff --git a/mooncake-transfer-engine/tests/rdma_transport_test.cpp b/mooncake-transfer-engine/tests/rdma_transport_test.cpp index a5231c22..6c6e747b 100644 --- a/mooncake-transfer-engine/tests/rdma_transport_test.cpp +++ b/mooncake-transfer-engine/tests/rdma_transport_test.cpp @@ -171,7 +171,7 @@ bool validateTransferSizes() { return true; } -void setBackendDevice(const std::string &backend) { +[[maybe_unused]] void setBackendDevice(const std::string &backend) { if (!usesDeviceMemory(backend)) { return; } -- 2.34.1 From f225d8ac554e2514486ceaacd44cccc48918d33d Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Tue, 26 May 2026 12:40:06 +0800 Subject: [PATCH 129/382] [Integration] fix batch tensor allocation error code (#2226) --- mooncake-integration/store/store_py_parallel_write.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mooncake-integration/store/store_py_parallel_write.h b/mooncake-integration/store/store_py_parallel_write.h index a4cc1ec3..63caf8b0 100644 --- a/mooncake-integration/store/store_py_parallel_write.h +++ b/mooncake-integration/store/store_py_parallel_write.h @@ -45,7 +45,7 @@ std::vector batch_write_tensor_impl(const std::vector &keys, if (!alloc_result) { LOG(ERROR) << "Failed to allocate buffer for " << operation_name << " key: " << keys[i]; - results[i] = to_py_ret(ErrorCode::INVALID_PARAMS); + results[i] = to_py_ret(ErrorCode::NO_AVAILABLE_HANDLE); continue; } -- 2.34.1 From a72b540cbcd60ec8b22abd196884fdda2b4bfa27 Mon Sep 17 00:00:00 2001 From: Xiao Yang Date: Tue, 26 May 2026 14:13:55 +0800 Subject: [PATCH 130/382] Fix MACA nvlink allocator build by mapping CUmemAllocationHandleType (#2227) Add the missing CUDA-like type alias in maca.h so nvlink_allocator.cpp can compile when building with -DUSE_MACA=ON. Co-authored-by: Cursor --- mooncake-transfer-engine/include/gpu_vendor/maca.h | 1 + 1 file changed, 1 insertion(+) diff --git a/mooncake-transfer-engine/include/gpu_vendor/maca.h b/mooncake-transfer-engine/include/gpu_vendor/maca.h index 604b489a..ca5d7285 100644 --- a/mooncake-transfer-engine/include/gpu_vendor/maca.h +++ b/mooncake-transfer-engine/include/gpu_vendor/maca.h @@ -36,6 +36,7 @@ const static std::string GPU_PREFIX = "maca:"; #define CUmemFabricHandle mcMemFabricHandle_t #define CUmemGenericAllocationHandle mcMemGenericAllocationHandle #define CUmemAllocationProp mcMemAllocationProp +#define CUmemAllocationHandleType mcMemAllocationHandleType #define CUmemAccessDesc mcMemAccessDesc #define CUmemAllocationType mcMemAllocationType #define CU_MEM_ALLOCATION_TYPE_PINNED mcMemAllocationTypePinned -- 2.34.1 From 0fceaee292131a4bf2476d306c3dba939c4f2778 Mon Sep 17 00:00:00 2001 From: Aoi Date: Tue, 26 May 2026 15:29:37 +0800 Subject: [PATCH 131/382] [Docs]: fix docs config, remove autodoc2, archive zh docs (#2218) --- docs/source/conf.py | 19 +++++-------------- docs/source/design/mooncake-store.md | 6 +++--- docs/source/design/tent/overview.md | 18 +++++++++++++++++- docs/source/design/transfer-engine/cpp-api.md | 2 +- .../design/transfer-engine/efa_transport.md | 1 + docs/source/design/transfer-engine/index.md | 9 ++++++--- .../examples/lmdeploy-integration-v0.9.md | 8 ++++---- .../examples/sglang-integration-v1.md | 2 +- .../vllm-integration/vllm-integration-v0.2.md | 2 +- .../vllm-integration/vllm-integration-v0.3.md | 2 +- .../vllm-integration/vllm-integration-v1.0.md | 4 ++-- docs/source/getting_started/observability.md | 2 +- .../source/http-api-reference/http-service.md | 4 ++-- docs/source/index.md | 2 ++ .../vllm-benchmark-results-v0.2.md | 6 +++--- .../performance/vllm-benchmark-results-v1.md | 2 +- .../python-api-reference/mooncake-store.md | 4 ---- .../python-api-reference/transfer-engine.md | 3 ++- docs/source/zh_archive/README.md | 2 +- 19 files changed, 54 insertions(+), 44 deletions(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index 6fc3b9a8..a505aa4f 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -41,7 +41,6 @@ extensions = [ "sphinx.ext.linkcode", "sphinx.ext.intersphinx", "sphinx_copybutton", - "autodoc2", "myst_parser", "sphinxarg.ext", "sphinx_design", @@ -53,25 +52,17 @@ myst_enable_extensions = [ "fieldlist", ] myst_fence_as_directive = ["mermaid"] -autodoc2_packages = [ - { - "path": "../../Mooncake", - "exclude_dirs": ["__pycache__", "third_party"], - }, -] -autodoc2_output_dir = "api" -autodoc2_render_plugin = "myst" -autodoc2_hidden_objects = ["dunder", "private", "inherited"] -autodoc2_sort_names = True -autodoc2_index_template = None - # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This pattern also affects html_static_path and html_extra_path. -exclude_patterns: list[str] = ["**/*.template.md", "**/*.inc.md"] +exclude_patterns: list[str] = [ + "**/*.template.md", + "**/*.inc.md", + "zh_archive/**", +] # Exclude the prompt "$" when copying code copybutton_prompt_text = r"\$ " diff --git a/docs/source/design/mooncake-store.md b/docs/source/design/mooncake-store.md index ac1afdd0..f12099e1 100644 --- a/docs/source/design/mooncake-store.md +++ b/docs/source/design/mooncake-store.md @@ -1037,15 +1037,15 @@ Available log levels: trace, debug, info, warn (or warning), error, and critical ## Example Code -#### Python Usage Example +### Python Usage Example We provide a reference example `distributed_object_store_provider.py`, located in the `mooncake-store/tests` directory. To check if the related components are properly installed, you can run etcd and Master Service (`mooncake_master`) in the background on the same server, and then execute this Python program in the foreground. It should output a successful test result. -#### C++ Usage Example +### C++ Usage Example The C++ API of Mooncake Store provides more low-level control capabilities. We provide a reference example `client_integration_test`, located in the `mooncake-store/tests` directory. To check if the related components are properly installed, you can run etcd and Master Service (`mooncake_master`) on the same server, and then execute this C++ program (located in the `build/mooncake-store/tests` directory). It should output a successful test result. ## Version Management Policy -The current version of Mooncake Store is defined in [`CMakeLists.txt`](../../mooncake-store/CMakeLists.txt) as `project(MooncakeStore VERSION 2.0.0)`. +The current version of Mooncake Store is defined in [`CMakeLists.txt`](gh-file:mooncake-store/CMakeLists.txt) as `project(MooncakeStore VERSION 2.0.0)`. When to bump the version: diff --git a/docs/source/design/tent/overview.md b/docs/source/design/tent/overview.md index 4c988cad..0c19cdc7 100644 --- a/docs/source/design/tent/overview.md +++ b/docs/source/design/tent/overview.md @@ -89,10 +89,26 @@ cpp-api metrics ::: +## TENT Quality of Service + +:::{toctree} +:maxdepth: 1 + +qos +::: + +## TENT Slice Spraying + +:::{toctree} +:maxdepth: 1 + +slice-spraying +::: + ## TENT Failover :::{toctree} :maxdepth: 1 failover -::: \ No newline at end of file +::: diff --git a/docs/source/design/transfer-engine/cpp-api.md b/docs/source/design/transfer-engine/cpp-api.md index eee3629c..44c9ff9d 100644 --- a/docs/source/design/transfer-engine/cpp-api.md +++ b/docs/source/design/transfer-engine/cpp-api.md @@ -337,7 +337,7 @@ The HTTP server should implement three following RESTful APIs, while the metadat 2. `PUT /metadata?key=$KEY`: Update the metadata corresponding to `$KEY` to the value of the request body. 3. `DELETE /metadata?key=$KEY`: Delete the metadata corresponding to `$KEY`. -For specific implementation, refer to the demo service implemented in Golang at [mooncake-transfer-engine/example/http-metadata-server](../../../mooncake-transfer-engine/example/http-metadata-server). +For specific implementation, refer to the demo service implemented in Golang at [mooncake-transfer-engine/example/http-metadata-server](gh-dir:mooncake-transfer-engine/example/http-metadata-server). ### Initialization diff --git a/docs/source/design/transfer-engine/efa_transport.md b/docs/source/design/transfer-engine/efa_transport.md index 05fcbfdc..4bd3fbff 100644 --- a/docs/source/design/transfer-engine/efa_transport.md +++ b/docs/source/design/transfer-engine/efa_transport.md @@ -211,6 +211,7 @@ address shown in the target's startup log (e.g., `ip-172-31-29-226:12345`). > **Note:** `buffer_size` must be >= `block_size * batch_size * threads`. The benchmark auto-adjusts if too small. +(benchmark-results)= ### Benchmark Results #### 1. p6-b300.48xlarge (B300, 16 EFA × 400 Gbps) diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index 05349c22..f6bff1e1 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -13,6 +13,7 @@ As shown in the diagram, each specific client corresponds to a `TransferEngine`, Mooncake Transfer Engine provides interfaces through the `TransferEngine` class (located in `mooncake-transfer-engine/include/transfer_engine.h`), where the specific data transfer functions for different backends are implemented by the `Transport` class, currently supporting `TcpTransport`, `RdmaTransport`, `EfaTransport`, `NVMeoFTransport`, `NvlinkTransport`, `IntraNodeNvlinkTransport`, and `HipTransport`. +(segment)= ### Segment Segment represents a collection of source address ranges and target address ranges available during the data transfer process in Transfer Engine. That is, all local and remote addresses involved in `BatchTransfer` requests must be within the valid segment range. Transfer Engine supports the following two types of Segments. @@ -28,6 +29,7 @@ In addition, Transfer Engine also supports registering some **local DRAM areas** #### 2. NVMeof Segment Transfer Engine also leverages the NVMeof protocol to support direct data transfer from files on NVMe to DRAM/VRAM via PCIe, without going through the CPU and achieving zero-copy. Users need to follow the instructions to mount remote storage nodes locally and use the `openSegment` interface for reference to complete data read/write operations. +(batchtransfer)= ### BatchTransfer With the help of Transfer Engine, Mooncake Store can achieve local DRAM/VRAM reading and writing of specified parts in valid segments through TCP, (GPUDirect) RDMA, NVMe-of protocols, etc. @@ -148,7 +150,7 @@ After successfully compiling Transfer Engine, the test program `transfer_engine_ The initiator node can also configure the following test parameters: `--operation` (can be `"read"` or `"write"`), `batch_size`, `block_size`, `duration`, `threads`, etc. > [!NOTE] -> If an exception occurs during execution, it is usually due to incorrect parameter settings. It is recommended to refer to the [troubleshooting document](troubleshooting.md) for preliminary troubleshooting. +> If an exception occurs during execution, it is usually due to incorrect parameter settings. It is recommended to refer to the [troubleshooting document](../../troubleshooting/troubleshooting.md) for preliminary troubleshooting. ### Sample Run @@ -252,7 +254,7 @@ The HTTP server should implement three following RESTful APIs, while the metadat 2. `PUT /metadata?key=$KEY`: Update the metadata corresponding to `$KEY` to the value of the request body. 3. `DELETE /metadata?key=$KEY`: Delete the metadata corresponding to `$KEY`. -For specific implementation, refer to the demo service implemented in Golang at [mooncake-transfer-engine/example/http-metadata-server](../../../mooncake-transfer-engine/example/http-metadata-server). +For specific implementation, refer to the demo service implemented in Golang at [mooncake-transfer-engine/example/http-metadata-server](gh-dir:mooncake-transfer-engine/example/http-metadata-server). ## Using Transfer Engine to Your Projects @@ -265,7 +267,7 @@ To support the operational needs of P2P Store, Transfer Engine provides a Golang When compiling the project, enable the `-DWITH_P2P_STORE=ON` option to compile the P2P Store example program at the same time. ### Using Rust Interface -Under `mooncake-transfer-engine/rust`, the Rust interface implementation of TransferEngine is provided, and a Rust version of the benchmark is implemented based on the interface, similar to [transfer_engine_bench.cpp](../../../mooncake-transfer-engine/example/transfer_engine_bench.cpp). To compile the rust example, you need to install the Rust SDK and add `-DWITH_RUST_EXAMPLE=ON` in the cmake command. +Under `mooncake-transfer-engine/rust`, the Rust interface implementation of TransferEngine is provided, and a Rust version of the benchmark is implemented based on the interface, similar to [transfer_engine_bench.cpp](gh-file:mooncake-transfer-engine/example/transfer_engine_bench.cpp). To compile the rust example, you need to install the Rust SDK and add `-DWITH_RUST_EXAMPLE=ON` in the cmake command. ## Advanced Runtime Options For advanced users, TransferEngine provides the following advanced runtime options, all of which can be passed in through **environment variables**. @@ -341,6 +343,7 @@ heterogeneous_ascend :::{toctree} :maxdepth: 1 +kunpeng_ub_transport sunrise_link_transport ::: diff --git a/docs/source/getting_started/examples/lmdeploy-integration-v0.9.md b/docs/source/getting_started/examples/lmdeploy-integration-v0.9.md index 5de2a669..fa8d7d11 100644 --- a/docs/source/getting_started/examples/lmdeploy-integration-v0.9.md +++ b/docs/source/getting_started/examples/lmdeploy-integration-v0.9.md @@ -16,17 +16,17 @@ pip install mooncake-transfer-engine Note: -- If any `.so` file is missing, uninstall the pip package with `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually from source following the [build instructions](https://github.com/kvcache-ai/Mooncake/blob/main/doc/en/build.md). +- If any `.so` file is missing, uninstall the pip package with `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually from source following the [build instructions](../build.md). ### Install the latest version of LMDeploy -##### 1. Clone LMDeploy from the official repo +#### 1. Clone LMDeploy from the official repo ```bash git clone https://github.com/InternLM/lmdeploy.git ``` -##### 2. Build +#### 2. Build ##### 2.1 Build from source @@ -164,4 +164,4 @@ curl -X POST "http://192.168.0.147:8000/v1/completions" \ - You can specify multiple prefill or decode instances with distinct `--server-port` and different GPUs using `CUDA_VISIBLE_DEVICES`. - MooncakeTransferEngine supports both intra-node (PCIe) and inter-node (RDMA/CXL) transfer, and device selection is automatic or customizable via config. - When using HF models that timeout during prefill, consider setting model path to `~/Qwen3-8B` to accelerate loading from localhost. -- Use `--log-level DEBUG` to get detailed runtime logs for troubleshooting. \ No newline at end of file +- Use `--log-level DEBUG` to get detailed runtime logs for troubleshooting. diff --git a/docs/source/getting_started/examples/sglang-integration-v1.md b/docs/source/getting_started/examples/sglang-integration-v1.md index 3c6053ac..b4a99da8 100644 --- a/docs/source/getting_started/examples/sglang-integration-v1.md +++ b/docs/source/getting_started/examples/sglang-integration-v1.md @@ -12,7 +12,7 @@ This is the latest version of the MooncakeTransferEngine integration doc with th pip3 install mooncake-transfer-engine ``` -Note: If you encounter problems such as missing `lib*.so`, you should uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](build.md). +Note: If you encounter problems such as missing `lib*.so`, you should uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](../build.md). ### Install the latest version of SGLang #### 1. Clone SGLang from official repo diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.2.md b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.2.md index 1fc0ad17..babb851b 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.2.md +++ b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.2.md @@ -16,7 +16,7 @@ pip3 install mooncake-transfer-engine ``` Note: - - If you encounter problems such as missing `lib*.so`, you should uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](build.md). + - If you encounter problems such as missing `lib*.so`, you should uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](../../build.md). - For vLLM version <= v0.8.4, it requires mooncake-transfer-engine <= 0.3.3.post2. In the latest release, interface `mooncake_vllm_adaptor` has been deprecated. ### Install the latest version of vLLM diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.3.md b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.3.md index 82168865..dadb8544 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.3.md +++ b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.3.md @@ -24,7 +24,7 @@ pip3 install mooncake-transfer-engine ``` Note: - - If you encounter problems such as missing `lib*.so`, you should uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](build.md). + - If you encounter problems such as missing `lib*.so`, you should uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](../../build.md). - For vLLM version <= v0.8.4, it requires mooncake-transfer-engine <= 0.3.3.post2. In the latest release, interface `mooncake_vllm_adaptor` has been deprecated. ### Install the latest version of vLLM diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v1.0.md b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v1.0.md index 3b140df6..84d3e2bc 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v1.0.md +++ b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v1.0.md @@ -16,7 +16,7 @@ Install mooncake-transfer-engine through pip: pip install mooncake-transfer-engine ``` -Note: If you encounter problems such as missing `lib*.so`, you should uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](../build.md). +Note: If you encounter problems such as missing `lib*.so`, you should uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](../../build.md). ### Install vLLM @@ -118,7 +118,7 @@ The following environment variables can be used to customize Mooncake behavior: ## Performance -For detailed performance benchmarks and results, see the [vLLM Benchmark](../../performance/vllm-v1-support-benchmark.md) documentation. +For detailed performance benchmarks and results, see the [vLLM Benchmark](../../../performance/vllm-v1-support-benchmark.md) documentation. ## Notes diff --git a/docs/source/getting_started/observability.md b/docs/source/getting_started/observability.md index cbc387b7..fd0e3922 100644 --- a/docs/source/getting_started/observability.md +++ b/docs/source/getting_started/observability.md @@ -156,7 +156,7 @@ curl http://:9003/health The admin HTTP server is configured in the master config file (`master.json` or `master.yaml`): -```json +```text { "enable_metric_reporting": true, "metrics_port": 9003, diff --git a/docs/source/http-api-reference/http-service.md b/docs/source/http-api-reference/http-service.md index c04c7b2e..88163d4b 100644 --- a/docs/source/http-api-reference/http-service.md +++ b/docs/source/http-api-reference/http-service.md @@ -58,7 +58,7 @@ curl "http://localhost:8080/query_key?key=my_object" ``` **Response Format**: -```json +```text { "transport_endpoint_": "hostname:port", "buffer_descriptors": [...] @@ -79,7 +79,7 @@ curl "http://localhost:8080/batch_query_keys?keys=key1,key2,key3" ``` **Response Format**: -```json +```text { "success": true, "data": { diff --git a/docs/source/index.md b/docs/source/index.md index d35549fe..e551f369 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -91,6 +91,7 @@ performance/vllm-v1-support-benchmark performance/allocator-benchmark-result performance/allocation-strategy-benchmark-result performance/ssd-offload-benchmark-results +performance/storage-benchmark ::: % API Documentation @@ -117,6 +118,7 @@ design/p2p-store design/transfer-engine/index design/hicache-design design/engram +design/unified-parallel-tensor-io design/tent/overview design/tent/tebench design/conductor/conductor-architecture-design diff --git a/docs/source/performance/vllm-benchmark-results-v0.2.md b/docs/source/performance/vllm-benchmark-results-v0.2.md index 685ac934..4f3bd612 100644 --- a/docs/source/performance/vllm-benchmark-results-v0.2.md +++ b/docs/source/performance/vllm-benchmark-results-v0.2.md @@ -3,7 +3,7 @@ Here are some preview mooncake benchmark results on A10 with up to 2 RDMA NICs. In addition, we are also coordinating resources to integrate some machines with more RDMA NICs and more advanced GPUs. The official benchmark results will be released in due time. -### Varying tp (input length = 1024, qps = 2, output length =6) +## Varying tp (input length = 1024, qps = 2, output length =6) | Setting | num_rdma_nic | Successful Requests | Duration (s) | Total Input Tokens | Total Generated Tokens | Req Throughput (req/s) | Output Token Throughput (tok/s) | Total Token Throughput (tok/s) | Mean TTFT (ms) | Median TTFT (ms) | P99 TTFT (ms) | Mean TPOT (ms) | Median TPOT (ms) | P99 TPOT (ms) | Mean ITL (ms) | Median ITL (ms) | P99 ITL (ms) | |-----------------|--------------|---------------------|--------------|--------------------|------------------------|------------------------|---------------------------------|-------------------------------|----------------|-----------------|--------------|---------------|------------------|--------------|--------------|----------------|-------------| | tp = 1 | 2 | 200 | 99.47 | 201995 | 1200 | 2.01 | 12.06 | 2042.74 | 1056.76 | 635.00 | 4006.59 | 97.08 | 26.94 | 781.91 | 97.01 | 14.05 | 2205.51 | @@ -16,7 +16,7 @@ In addition, we are also coordinating resources to integrate some machines with | tp = 2 | TCP | 200 | 98.97 | 201995 | 1200 | 2.02 | 12.12 | 2053.03 | 333.74 | 251.32 | 954.63 | 28.74 | 15.49 | 161.24 | 28.70 | 15.35 | 393.52 | | tp = 4 | TCP | 200 | 98.78 | 201995 | 1200 | 2.02 | 12.15 | 2056.94 | 205.37 | 162.92 | 463.70 | 21.54 | 16.51 | 94.04 | 21.51 | 16.56 | 170.54 | -### Varying qps (length = 1024, tp = 4, output length =6) +## Varying qps (length = 1024, tp = 4, output length =6) |Setting | num_rdma_nic | Successful Requests | Duration (s) | Total Input Tokens | Total Generated Tokens | Req Throughput (req/s) | Output Token Throughput (tok/s) | Total Token Throughput (tok/s) | Mean TTFT (ms) | Median TTFT (ms) | P99 TTFT (ms) | Mean TPOT (ms) | Median TPOT (ms) | P99 TPOT (ms) | Mean ITL (ms) | Median ITL (ms) | P99 ITL (ms) | |-----------------|--------------|---------------------|--------------|--------------------|------------------------|------------------------|---------------------------------|-------------------------------|----------------|-----------------|--------------|---------------|------------------|--------------|--------------|----------------|-------------| | qps = 2 | 2 | 200 | 98.77 | 201995 | 1200 | 2.02 | 12.15 | 2057.33 | 200.64 | 156.62 | 478.22 | 22.63 | 17.35 | 99.61 | 22.60 | 17.08 | 186.25 | @@ -32,7 +32,7 @@ In addition, we are also coordinating resources to integrate some machines with | qps = 6 | TCP | 200 | 33.49 | 201995 | 1200 | 5.97 | 35.83 | 6067.71 | 907.74 | 514.85 | 3253.93 | 122.75 | 45.51 | 648.40 | 122.56 | 18.09 | 2282.92 | | qps = 8 | TCP | 200 | 28.39 | 201995 | 1200 | 7.04 | 42.26 | 7156.09 | 6714.57 | 7885.09 | 11787.51 | 1116.06 | 408.32 | 4645.25 | 1114.29 | 46.87 | 21898.03 | -### Varying input length (tp = 4, qps = 2, output length =6) +## Varying input length (tp = 4, qps = 2, output length =6) | Setting | num_rdma_nic | Successful Requests | Duration (s) | Total Input Tokens | Total Generated Tokens | Req Throughput (req/s) | Output Token Throughput (tok/s) | Total Token Throughput (tok/s) | Mean TTFT (ms) | Median TTFT (ms) | P99 TTFT (ms) | Mean TPOT (ms) | Median TPOT (ms) | P99 TPOT (ms) | Mean ITL (ms) | Median ITL (ms) | P99 ITL (ms) | |-----------------|--------------|---------------------|--------------|--------------------|------------------------|------------------------|---------------------------------|-------------------------------|----------------|-----------------|--------------|---------------|------------------|--------------|--------------|----------------|-------------| | 1024 | 2 | 200 | 98.77 | 201995 | 1200 | 2.02 | 12.15 | 2057.32 | 195.47 | 151.55 | 482.84 | 22.83 | 19.27 | 96.55 | 22.81 | 18.12 | 158.16 | diff --git a/docs/source/performance/vllm-benchmark-results-v1.md b/docs/source/performance/vllm-benchmark-results-v1.md index 6a5acc57..05032748 100644 --- a/docs/source/performance/vllm-benchmark-results-v1.md +++ b/docs/source/performance/vllm-benchmark-results-v1.md @@ -2,7 +2,7 @@ Here are some preview MooncakeStore benchmark results on A10 with "Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4". -### Varying PD ratio (input length = 1024, qps = 2, output length =6, num of requests = 200) +## Varying PD ratio (input length = 1024, qps = 2, output length =6, num of requests = 200) | Configuration | Backend | Duration (s) | Output Token Throughput (tok/s) | Total Token Throughput (tok/s) | Mean TTFT (ms) | Median TTFT (ms) | P99 TTFT (ms) | Mean TPOT (ms) | Median TPOT (ms) | P99 TPOT (ms) | Mean ITL (ms) | Median ITL (ms) | P99 ITL (ms) | |----------------------------|----------------------|--------------|---------------------------------|-------------------------------|----------------|-----------------|--------------|---------------|------------------|--------------|--------------|----------------|-------------| | 2P2D tp = 1 | Redis | 99.47 | 12.06 | 2042.75 | 844.28 | 666.84 | 2270.91 | 16.88 | 11.57 | 104.83 | 16.84 | 11.56 | 239.67 | diff --git a/docs/source/python-api-reference/mooncake-store.md b/docs/source/python-api-reference/mooncake-store.md index cf190b29..5939b821 100644 --- a/docs/source/python-api-reference/mooncake-store.md +++ b/docs/source/python-api-reference/mooncake-store.md @@ -2820,8 +2820,6 @@ bind_to_numa_node(0) --- ---- - ## Error Handling Most methods return integer status codes: @@ -2844,5 +2842,3 @@ For methods that return data (`get`, `get_batch`, `get_buffer`, `get_tensor`): 4. **Configure replication** appropriately - more replicas provide better availability but use more storage 5. **Use soft pinning** for frequently accessed objects to keep them in memory 6. **Choose RDMA protocol** when available for maximum performance - ---- diff --git a/docs/source/python-api-reference/transfer-engine.md b/docs/source/python-api-reference/transfer-engine.md index 45c296bf..71136f14 100644 --- a/docs/source/python-api-reference/transfer-engine.md +++ b/docs/source/python-api-reference/transfer-engine.md @@ -624,6 +624,7 @@ The Transfer Engine respects the following environment variables: ### Basic Setup and Data Transfer ```python +import numpy as np from mooncake.engine import TransferEngine import os @@ -650,7 +651,7 @@ data_len = len(data) engine.register_memory(buffer_data, buffer_data_len) # Get Remote Addr from ZMQ or upper-layer inference framework -remote_addr = ?? +remote_addr = REMOTE_ADDR # Transfer data to remote node ret = engine.transfer_sync_write( diff --git a/docs/source/zh_archive/README.md b/docs/source/zh_archive/README.md index 4ef7d418..32a8a705 100644 --- a/docs/source/zh_archive/README.md +++ b/docs/source/zh_archive/README.md @@ -5,4 +5,4 @@ ## 当前状态 - 已归档,不再活跃维护 - 网站当前只显示英文文档 -- 如需查看/编辑最新文档,请访问docs目录 \ No newline at end of file +- 如需查看/编辑最新文档,请访问docs目录 -- 2.34.1 From 3dfee31522aa9dee025ede94fa188ed0561802ed Mon Sep 17 00:00:00 2001 From: Dayuxiaoshui <158081477+Dayuxiaoshui@users.noreply.github.com> Date: Wed, 27 May 2026 09:57:26 +0800 Subject: [PATCH 132/382] [TransferEngine][MACA] Complete gpu_vendor/maca.h CUDA-like aliases for MACA build (#2230) Add missing mappings in gpu_vendor/maca.h: - CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR -> mcMemHandleTypePosixFileDescriptor - CU_POINTER_ATTRIBUTE_DEVICE_ORDINAL -> mcPointerAttributeDevice These aliases complement #2227 and ensure full coverage of CUDA-like symbols used in nvlink_allocator.cpp and related components when building with -DUSE_MACA=ON. --- mooncake-transfer-engine/include/gpu_vendor/maca.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mooncake-transfer-engine/include/gpu_vendor/maca.h b/mooncake-transfer-engine/include/gpu_vendor/maca.h index ca5d7285..3d4ae801 100644 --- a/mooncake-transfer-engine/include/gpu_vendor/maca.h +++ b/mooncake-transfer-engine/include/gpu_vendor/maca.h @@ -42,6 +42,8 @@ const static std::string GPU_PREFIX = "maca:"; #define CU_MEM_ALLOCATION_TYPE_PINNED mcMemAllocationTypePinned #define CU_MEM_LOCATION_TYPE_DEVICE mcMemLocationTypeDevice #define CU_MEM_HANDLE_TYPE_FABRIC mcMemHandleTypeFabric +#define CU_MEM_HANDLE_TYPE_POSIX_FILE_DESCRIPTOR \ + mcMemHandleTypePosixFileDescriptor #define CU_DEVICE_ATTRIBUTE_HANDLE_TYPE_FABRIC_SUPPORTED \ mcDeviceAttributeHandleTypeFabricSupported #define CU_DEVICE_ATTRIBUTE_GPU_DIRECT_RDMA_WITH_CUDA_VMM_SUPPORTED \ @@ -52,6 +54,7 @@ const static std::string GPU_PREFIX = "maca:"; #define CU_MEMORYTYPE_HOST mcMemoryTypeHost #define CU_MEMORYTYPE_DEVICE mcMemoryTypeDevice #define CU_POINTER_ATTRIBUTE_MEMORY_TYPE mcPointerAttributeMemoryType +#define CU_POINTER_ATTRIBUTE_DEVICE_ORDINAL mcPointerAttributeDevice #define CU_POINTER_ATTRIBUTE_RANGE_START_ADDR mcPointerAttributeRangeStartAddr #define CU_POINTER_ATTRIBUTE_RANGE_SIZE mcPointerAttributeRangeSize #define CU_MEM_RANGE_HANDLE_TYPE_DMA_BUF_FD mcMemHandleTypePosixFileDescriptor -- 2.34.1 From 54411aff58fa06a3e8e4695c01507283f27dfd43 Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Wed, 27 May 2026 10:41:57 +0800 Subject: [PATCH 133/382] [TENT] Add rule-based transport and device selection (#2079) * [TENT] Add configuration-driven transport selector Add TransportSelector for flexible, configuration-driven transport selection policy while maintaining full backward compatibility. Key features: - Configuration-driven transport selection via JSON policy rules - Support for segment_type filtering, device allocation, and transport priority - Legacy mode option (use_legacy_transport_selection) for exact original behavior - Default policies match original hardcoded behavior Changes: - Add TransportSelector class with SelectionContext, SelectionPolicy, SelectionResult - Integrate TransportSelector into TransferEngineImpl - Add legacy mode support to preserve original code path - Restore TaskInfo fields (xport_priority, failover_count) for backward compatibility - Add max_failover_attempts configuration option Co-Authored-By: Claude Opus 4.7 (1M context) * [TENT] Add comprehensive unit tests for TransportSelector Add transport_selector_test.cpp with test coverage for: - Default policies matching original behavior (File/Memory segments) - Transport type name parsing - Legacy mode enable/disable - Transport availability based on capabilities - Priority offset for fallback scenarios - Device mask handling - NVLINK same-machine constraint - ROCm memory type support - GPU-to-GPU, CPU-to-CPU, CPU-to-GPU, GPU-to-CPU transfers Co-Authored-By: Claude Opus 4.7 (1M context) * [TENT] Fix FakeTransport to use protected caps member Fix compilation error by accessing Transport::caps (protected) through helper methods instead of a separate public member. Co-Authored-By: Claude Opus 4.7 (1M context) * fix * fix format issues * Add priority-based rule * Reformat * Fix review comments * Fix memory leak in endpoint_store_integration_test When ibv_get_device_list returns a non-NULL list but num_devices == 0, we need to call ibv_free_device_list before returning to avoid leaking the allocated memory. Co-Authored-By: Claude Opus 4.7 (1M context) * Trigger CI --------- Co-authored-by: Claude Opus 4.7 (1M context) --- docs/source/design/tent/transport-selector.md | 226 ++++++++ .../tent/config/transfer-engine.json | 15 +- .../tent/runtime/transfer_engine_impl.h | 16 +- .../tent/include/tent/runtime/transport.h | 3 +- .../include/tent/runtime/transport_selector.h | 190 ++++++ .../tent/src/runtime/transfer_engine_impl.cpp | 149 +++-- .../tent/src/runtime/transport_selector.cpp | 427 ++++++++++++++ .../src/transport/rdma/rdma_transport.cpp | 3 +- .../tent/tests/CMakeLists.txt | 8 + .../tent/tests/transport_selector_test.cpp | 546 ++++++++++++++++++ .../tests/endpoint_store_integration_test.cpp | 6 +- 11 files changed, 1541 insertions(+), 48 deletions(-) create mode 100644 docs/source/design/tent/transport-selector.md create mode 100644 mooncake-transfer-engine/tent/include/tent/runtime/transport_selector.h create mode 100644 mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp create mode 100644 mooncake-transfer-engine/tent/tests/transport_selector_test.cpp diff --git a/docs/source/design/tent/transport-selector.md b/docs/source/design/tent/transport-selector.md new file mode 100644 index 00000000..e402e17b --- /dev/null +++ b/docs/source/design/tent/transport-selector.md @@ -0,0 +1,226 @@ +# TENT Transport Selector + +## Overview + +The Transport Selector is responsible for choosing the optimal transport and devices for each transfer request based on configuration policies. + +## Request Priority + +TENT uses a unified priority system across all components. See [QoS.md](qos.md) for detailed description of priority levels and their usage throughout the system. + +Quick reference: +- `"high"` / `0` - High-priority requests (metadata, control, latency-sensitive) +- `"medium"` / `1` - Medium-priority requests (interactive queries, serving) +- `"low"` / `2` - Low-priority requests (bulk transfer, background jobs) + +## Configuration-Based Transport Selection + +Transport selection is driven by configuration with pattern-based rules. + +### Configuration Example + +```json +{ + "policy": [ + { + "name": "high_prio_fast", + "segment_type": "memory", + "priority": "high", + "devices": ["mlx5_0", "mlx5_1", "mlx5_2"], + "transports": ["nvlink", "rdma", "shm"] + }, + { + "name": "low_prio_slow", + "segment_type": "memory", + "priority": "low", + "devices": ["mlx5_0"], + "transports": ["rdma", "tcp"] + }, + { + "name": "file_storage", + "segment_type": "file", + "transports": ["gds", "io_uring", "rdma"] + } + ] +} +``` + +### Policy Fields + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | Yes | Policy identifier (for logging) | +| `segment_type` | string | Yes | `"memory"` or `"file"` | +| `priority` | string or int | No | Match only requests with this priority: `"high"` (0), `"medium"` (1), `"low"` (2) | +| `devices` | array[string] | No | List of allowed device names (empty = all devices) | +| `transports` | array[string] | No | Transport preference list (evaluated in order) | + +### Memory Type Filters + +For `memory` segments, you can filter by source/destination memory type: + +| Pattern | Matches | +|---------|---------| +| `"cuda"` | CUDA GPU memory | +| `"cpu"` | CPU/host memory | +| `"hip"` | ROCm/HIP GPU memory | +| `"npu"` | Ascend NPU memory | +| `"*"` | Any memory type | + +### Size Filters + +Restrict policies to specific transfer sizes: + +```json +{ + "name": "small_transfers", + "segment_type": "memory", + "min_size": 0, + "max_size": 1048576, + "transports": ["shm", "nvlink"] +} +``` + +## Device Mask + +The `devices` field in a policy creates a bitmask that restricts which NICs can be used: + +```json +{ + "name": "use_nic_0_only", + "segment_type": "memory", + "devices": ["mlx5_0"], + "transports": ["rdma"] +} +``` + +This is translated internally to a 64-bit bitmask where each bit represents one device: +- `devices: ["mlx5_0"]` → `device_mask = 0x0001` (bit 0 set) +- `devices: ["mlx5_1", "mlx5_2"]` → `device_mask = 0x0006` (bits 1 and 2 set) +- `devices: []` (empty) → `device_mask = ~0ULL` (all devices) + +## Transport Fallback + +When multiple transports are listed in the `transports` array, they act as fallback options: + +```json +{ + "transports": ["rdma", "tcp"] +} +``` + +This means: +1. Try RDMA first +2. If RDMA is unavailable or fails, fall back to TCP + +### Fallback with `transport_index` + +For programmatic control, the `transport_index` parameter selects which transport to use: + +```cpp +// transport_index = 0 → First transport (rdma) +// transport_index = 1 → Second transport (tcp) +// transport_index = 2 → Third transport (if exists) +auto result = selector.select(context, transports, transport_index); +``` + +## Default Behavior + +If no `policy` is configured, TENT falls back to original behavior: + +| Segment Type | Default Transport Order | +|--------------|-------------------------| +| File | GDS → IOURING → RDMA | +| Memory | Uses `buffer_transports` order from buffer registration | + +## Complete Example + +```json +{ + "policy": [ + { + "name": "high_priority_local", + "segment_type": "memory", + "same_machine": true, + "priority": "high", + "transports": ["nvlink", "shm"] + }, + { + "name": "high_priority_remote", + "segment_type": "memory", + "same_machine": false, + "priority": "high", + "local_memory": "cuda", + "remote_memory": "cuda", + "devices": ["mlx5_0", "mlx5_1", "mlx5_2"], + "transports": ["rdma"] + }, + { + "name": "medium_priority", + "segment_type": "memory", + "priority": "medium", + "transports": ["rdma"] + }, + { + "name": "bulk_transfer", + "segment_type": "memory", + "priority": "low", + "min_size": 104857600, + "transports": ["rdma", "tcp"] + }, + { + "name": "file_ops", + "segment_type": "file", + "transports": ["gds", "io_uring"] + } + ] +} +``` + +## Unified Priority System + +The priority value propagates through multiple layers: + +``` +Request.priority (application layer) + ↓ +TransportSelector policy matching + ↓ +DeviceSelector allocation (QoS filtering) + ↓ +Worker thread scheduling queues +``` + +**Layer interactions**: + +1. **TransportSelector**: Policy's `priority` field filters which requests match +2. **DeviceSelector**: `priority` parameter controls device eligibility (QoS mode) +3. **Workers**: Separate queues per priority level with strict draining order + +See [QoS.md](qos.md) for details on worker scheduling and global slot coordination. + +## Data Flow + +``` +Request with priority + ↓ +TransportSelector.select(context, transports, transport_index) + ↓ +Match policy by: + - segment_type (file/memory) + - priority (exact match if specified in policy) + - location constraints + - size constraints + ↓ +Build device_mask from policy.devices + ↓ +Select transport from policy.transports[transport_index] + ↓ +Return SelectionResult { transport, device_mask } + ↓ +RdmaTransport.submitTransferTasks(batch, requests) + ↓ +DeviceSelector.allocate(..., request.priority, batch.device_mask) + ↓ +Worker scheduling (separate queues per priority) +``` diff --git a/mooncake-transfer-engine/tent/config/transfer-engine.json b/mooncake-transfer-engine/tent/config/transfer-engine.json index 79fd0d91..01932aef 100644 --- a/mooncake-transfer-engine/tent/config/transfer-engine.json +++ b/mooncake-transfer-engine/tent/config/transfer-engine.json @@ -89,5 +89,18 @@ "mnnvl": { "enable" : false } - } + }, + "policy": [ + { + "name": "default_memory", + "segment_type": "memory", + "devices": ["mlx5_0", "mlx5_2"], + "transports": ["nvlink", "rdma", "shm"] + }, + { + "name": "file_storage", + "segment_type": "file", + "transports": ["gds", "io_uring", "rdma"] + } + ] } diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h b/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h index 3dc4806a..c5b94e67 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h @@ -31,6 +31,7 @@ #include "tent/common/status.h" #include "tent/common/types.h" #include "tent/common/concurrent/thread_local_storage.h" +#include "tent/runtime/transport_selector.h" namespace mooncake { namespace tent { @@ -49,14 +50,15 @@ class ProxyManager; struct TaskInfo { TransportType type{UNSPEC}; int sub_task_id{-1}; - bool derived{false}; // merged by other tasks - int xport_priority{0}; + bool derived{false}; // merged by other tasks + int xport_priority{0}; // transport priority (for fallback) + int failover_count{0}; // number of failover attempts + uint64_t device_mask{~0ULL}; // Device mask for quota allocation Request request; bool staging{false}; TransferStatusEnum status{TransferStatusEnum::PENDING}; volatile TransferStatusEnum staging_status{TransferStatusEnum::PENDING}; std::chrono::steady_clock::time_point start_time{}; // For latency tracking - int failover_count{0}; // Number of cross-transport failover attempts }; class TransferEngineImpl { @@ -179,7 +181,8 @@ class TransferEngineImpl { Status lazyFreeBatch(); - TransportType getTransportType(const Request& request, int priority = 0); + SelectionResult getTransportType(const Request& request, + int transport_index = 0); std::vector getSupportedTransports( TransportType request_type); @@ -189,8 +192,8 @@ class TransferEngineImpl { void updateTaskStatusFromPoll(Batch* batch, size_t task_id, TransferStatus& task_status); - TransportType resolveTransport(const Request& req, int priority, - bool invalidate_on_fail = true); + SelectionResult resolveTransport(const Request& req, int transport_index, + bool invalidate_on_fail = true); Status loadTransports(); @@ -220,6 +223,7 @@ class TransferEngineImpl { std::shared_ptr conf_; std::shared_ptr metadata_; std::shared_ptr topology_; + std::unique_ptr transport_selector_; bool available_; std::array, kSupportedTransportTypes> diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/transport.h b/mooncake-transfer-engine/tent/include/tent/runtime/transport.h index 36e62a09..39925c1c 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/transport.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/transport.h @@ -45,9 +45,10 @@ struct Capabilities { class Transport { public: struct SubBatch { - SubBatch() {} + SubBatch() : device_mask(~0ULL) {} virtual ~SubBatch() {} virtual size_t size() const = 0; + uint64_t device_mask; // Device mask for transport selection }; using SubBatchRef = SubBatch *; diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/transport_selector.h b/mooncake-transfer-engine/tent/include/tent/runtime/transport_selector.h new file mode 100644 index 00000000..7fc0dccf --- /dev/null +++ b/mooncake-transfer-engine/tent/include/tent/runtime/transport_selector.h @@ -0,0 +1,190 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +/** + * @file transport_selector.h + * @brief Configuration-driven transport selection policy + * + * Transport selection is driven by configuration with pattern-based rules. + * + * Example configuration: + * { + * "policy": [ + * { + * "name": "high_prio_fast", + * "segment_type": "memory", + * "priority": 0, + * "devices": ["mlx5_0", "mlx5_1", "mlx5_2"], + * "transports": ["nvlink", "rdma", "shm"] + * }, + * { + * "name": "low_prio_slow", + * "segment_type": "memory", + * "priority": 2, + * "devices": ["mlx5_0"], + * "transports": ["rdma", "tcp"] + * }, + * { + * "name": "file_storage", + * "segment_type": "file", + * "transports": ["gds", "io_uring", "rdma"] + * } + * ] + * } + * + * If no "policy" is configured, defaults to original behavior: + * - File: GDS -> IOURING -> RDMA + * - Memory: uses buffer_transports order + */ + +#ifndef TENT_TRANSPORT_SELECTOR_H +#define TENT_TRANSPORT_SELECTOR_H + +#include "tent/common/config.h" +#include "tent/common/types.h" +#include "tent/runtime/segment.h" +#include "tent/runtime/platform.h" + +#include +#include +#include +#include +#include +#include + +namespace mooncake { +namespace tent { + +class Transport; + +/** + * @brief Selection context for a single request + */ +struct SelectionContext { + SegmentType segment_type; // File or Memory + bool same_machine; // Local or remote + MemoryType local_memory_type; // CPU, CUDA, etc. + MemoryType remote_memory_type; // CPU, CUDA, etc. (for remote) + const std::vector* + buffer_transports; // Pointer to transports in buffer + size_t transfer_size; // Transfer size in bytes + int priority_level; // Request priority level (lower = more urgent) +}; + +/** + * @brief Transport selection policy rule + */ +struct SelectionPolicy { + // Basic identification + std::string name; + + // Segment type filter + SegmentType segment_type; + + // Location filter + std::optional same_machine; // nullopt = don't care + + // Memory type filters (supports patterns: "cuda", "cpu", "npu", "*" for + // any) + std::optional local_memory_pattern; + std::optional remote_memory_pattern; + + // Size filter (nullopt = no limit) + std::optional min_size; // Minimum transfer size + std::optional max_size; // Maximum transfer size + + // Priority filter: exact match required (request.priority == priority) + // nullopt = match any priority level + std::optional priority; + + // Device allocation: list of device names this policy can use + // e.g., ["mlx5_0", "mlx5_1", "rocep5s0f0"] + // Empty = use all available devices + std::vector devices; + + // Transport preference list (evaluated in order) + std::vector transports; +}; + +/** + * @brief Result of transport selection + */ +struct SelectionResult { + TransportType transport = UNSPEC; + uint64_t device_mask = ~0ULL; // Bitmask of allowed devices (~0 = all) +}; + +/** + * @brief Configuration-driven transport selector + */ +class TransportSelector { + public: + TransportSelector(std::shared_ptr config); + + /** + * @brief Set topology for device name to ID conversion + */ + void setTopology(std::shared_ptr topology) { + topology_ = topology; + } + + /** + * @brief Select the best transport for a given context + * @param context Selection context + * @param available_transports Array of available transports + * @param transport_index Transport selection index (0 = first choice, 1 = + * second, ...) + * @return Selection result with transport type and device mask + */ + SelectionResult select( + const SelectionContext& context, + const std::array, kSupportedTransportTypes>& + available_transports, + int transport_index = 0); + + /** + * @brief Enable legacy mode (skip TransportSelector, use original logic) + */ + void setLegacyMode(bool enabled) { legacy_mode_ = enabled; } + + /** + * @brief Check if legacy mode is enabled + */ + bool isLegacyMode() const { return legacy_mode_; } + + static std::string transportTypeName(TransportType type); + static TransportType parseTransportType(const std::string& str); + + private: + std::vector getDefaultPolicies(); + void loadPolicies(); + bool matchesMemoryPattern(const std::string& pattern, + MemoryType type) const; + bool matchesPolicy(const SelectionPolicy& policy, + const SelectionContext& context) const; + bool isTransportAvailable( + TransportType type, const SelectionContext& context, + const std::array, kSupportedTransportTypes>& + available_transports) const; + + std::shared_ptr config_; + std::shared_ptr topology_; + std::vector policies_; + bool legacy_mode_{false}; // If true, skip selector and return empty result +}; + +} // namespace tent +} // namespace mooncake + +#endif // TENT_TRANSPORT_SELECTOR_H diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index d7870f97..872008e0 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -338,6 +338,18 @@ Status TransferEngineImpl::construct() { local_segment_name_ = randomSegmentName(); CHECK_STATUS(setupLocalSegment()); + + // Initialize transport selector + transport_selector_ = std::make_unique(conf_); + transport_selector_->setTopology(topology_); + + // Check if legacy mode is enabled (use original getTransportType logic) + bool legacy_mode = conf_->get("use_legacy_transport_selection", false); + transport_selector_->setLegacyMode(legacy_mode); + if (legacy_mode) { + LOG(INFO) << "Using legacy transport selection (original logic)"; + } + CHECK_STATUS(loadTransports()); std::string transport_string; @@ -804,44 +816,91 @@ static MemoryType getTypeEnum(const std::string& type) { return MTYPE_UNKNOWN; } -TransportType TransferEngineImpl::getTransportType(const Request& request, - int priority) { +SelectionResult TransferEngineImpl::getTransportType(const Request& request, + int transport_index) { SegmentDesc* desc; if (request.target_id == LOCAL_SEGMENT_ID) { desc = metadata_->segmentManager().getLocal().get(); } else { auto status = metadata_->segmentManager().getRemoteCached( desc, request.target_id); - if (!status.ok()) return UNSPEC; + if (!status.ok()) return SelectionResult{}; } auto local_mtype = Platform::getLoader().getMemoryType(request.source); - if (desc->type == SegmentType::File) { - if (checkAvailability(transport_list_[GDS], local_mtype)) { - if (priority-- == 0) return GDS; - } - if (checkAvailability(transport_list_[IOURING], local_mtype)) { - if (priority-- == 0) return IOURING; - } - return UNSPEC; - } else { - auto entry = desc->findBuffer(request.target_offset, request.length); - if (!entry) return UNSPEC; - bool same_machine = (request.target_id == LOCAL_SEGMENT_ID); - if (!same_machine) { - auto local_desc = metadata_->segmentManager().getLocal(); - same_machine = local_desc && !desc->machine_id.empty() && - !local_desc->machine_id.empty() && - desc->machine_id == local_desc->machine_id; - } - auto remote_mtype = getTypeEnum(LocationParser(entry->location).type()); - for (auto type : entry->transports) { - if ((type == NVLINK || type == SHM) && !same_machine) continue; - if (checkAvailability(transport_list_[type], local_mtype, - remote_mtype)) { - if (priority-- == 0) return type; + + // Legacy mode: use original logic (before TransportSelector) + if (transport_selector_ && transport_selector_->isLegacyMode()) { + SelectionResult result; + if (desc->type == SegmentType::File) { + if (checkAvailability(transport_list_[GDS], local_mtype)) { + if (transport_index-- == 0) result.transport = GDS; + } + if (checkAvailability(transport_list_[IOURING], local_mtype)) { + if (transport_index-- == 0) result.transport = IOURING; + } + } else { + auto entry = + desc->findBuffer(request.target_offset, request.length); + if (entry) { + bool same_machine = (request.target_id == LOCAL_SEGMENT_ID); + if (!same_machine) { + auto local_desc = metadata_->segmentManager().getLocal(); + same_machine = local_desc && !desc->machine_id.empty() && + !local_desc->machine_id.empty() && + desc->machine_id == local_desc->machine_id; + } + auto remote_mtype = + getTypeEnum(LocationParser(entry->location).type()); + for (auto type : entry->transports) { + if ((type == NVLINK || type == SHM) && !same_machine) + continue; + if (checkAvailability(transport_list_[type], local_mtype, + remote_mtype)) { + if (transport_index-- == 0) { + result.transport = type; + break; + } + } + } } } - return UNSPEC; + return result; + } + + // New TransportSelector-based logic + // Build selection context + SelectionContext ctx; + ctx.transfer_size = request.length; + ctx.priority_level = + request.priority; // Use request priority for selection + + if (desc->type == SegmentType::File) { + // File segment: use selector with empty buffer_transports + ctx.segment_type = SegmentType::File; + ctx.same_machine = true; // File is always local + ctx.local_memory_type = local_mtype; + ctx.remote_memory_type = MTYPE_CPU; + ctx.buffer_transports = nullptr; // Empty - use policy priority + + return transport_selector_->select(ctx, transport_list_, + transport_index); + } else { + // Memory segment + auto entry = desc->findBuffer(request.target_offset, request.length); + if (!entry) return SelectionResult{}; + bool same_machine = + (desc->machine_id == + metadata_->segmentManager().getLocal()->machine_id); + auto remote_mtype = getTypeEnum(LocationParser(entry->location).type()); + + ctx.segment_type = SegmentType::Memory; + ctx.same_machine = same_machine; + ctx.local_memory_type = local_mtype; + ctx.remote_memory_type = remote_mtype; + ctx.buffer_transports = &entry->transports; + + return transport_selector_->select(ctx, transport_list_, + transport_index); } } @@ -1106,15 +1165,15 @@ void TransferEngineImpl::findStagingPolicy(const Request& request, } } -TransportType TransferEngineImpl::resolveTransport(const Request& req, - int priority, - bool invalidate_on_fail) { - auto type = getTransportType(req, priority); - if (type == UNSPEC && invalidate_on_fail) { +SelectionResult TransferEngineImpl::resolveTransport(const Request& req, + int transport_index, + bool invalidate_on_fail) { + auto result = getTransportType(req, transport_index); + if (result.transport == UNSPEC && invalidate_on_fail) { metadata_->segmentManager().invalidateRemote(req.target_id); - type = getTransportType(req, priority); + result = getTransportType(req, transport_index); } - return type; + return result; } Status TransferEngineImpl::submitTransfer( @@ -1152,13 +1211,16 @@ Status TransferEngineImpl::submitTransfer( continue; } + task.failover_count = 0; task.xport_priority = 0; task.status = PENDING; task.request = merged_request; task.staging = false; task.start_time = submit_time; // Record start time for latency tracking - task.type = resolveTransport(merged_request, 0); + auto select_result = resolveTransport(merged_request, 0); + task.type = select_result.transport; + task.device_mask = select_result.device_mask; if (task.type == UNSPEC) { LOG(WARNING) << "Unable to find registered buffer for request: " << printRequest(merged_request); @@ -1204,6 +1266,15 @@ Status TransferEngineImpl::submitTransfer( if (classified_request_list[type].empty()) continue; auto& transport = transport_list_[type]; auto& sub_batch = batch->sub_batch[type]; + + // Set device_mask on SubBatch for RDMA transport + if (type == RDMA && !task_id_list[type].empty()) { + // Use the device_mask from the first task (we assume all tasks in + // this batch should have the same policy) + sub_batch->device_mask = + batch->task_list[task_id_list[type][0]].device_mask; + } + auto status = transport->submitTransferTasks( sub_batch, classified_request_list[type]); if (!status.ok()) { @@ -1283,8 +1354,10 @@ Status TransferEngineImpl::resubmitTransferTask(Batch* batch, size_t task_id) { if (task.staging) task.staging = false; else - task.xport_priority++; - auto type = resolveTransport(task.request, task.xport_priority); + task.xport_priority = task.failover_count; + + auto result = resolveTransport(task.request, task.xport_priority); + auto type = result.transport; if (type == UNSPEC) { LOG(WARNING) << "No more transports available after " << transportTypeName(prev_type) << " failed"; diff --git a/mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp b/mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp new file mode 100644 index 00000000..9cf0a430 --- /dev/null +++ b/mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp @@ -0,0 +1,427 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/runtime/transport_selector.h" +#include "tent/runtime/transport.h" +#include "tent/runtime/platform.h" +#include "tent/thirdparty/nlohmann/json.h" + +#include + +namespace mooncake { +namespace tent { + +// Transport type name mapping +static const std::unordered_map kTransportNameMap = + { + {"rdma", RDMA}, {"tcp", TCP}, {"shm", SHM}, + {"nvlink", NVLINK}, {"gds", GDS}, {"io_uring", IOURING}, + {"ascend", AscendDirect}, {"mnnvl", MNNVL}, +}; + +static const std::unordered_map + kTransportTypeNames = { + {RDMA, "rdma"}, {TCP, "tcp"}, {SHM, "shm"}, + {NVLINK, "nvlink"}, {GDS, "gds"}, {IOURING, "io_uring"}, + {AscendDirect, "ascend"}, {MNNVL, "mnnvl"}, {UNSPEC, "unspec"}, +}; + +// Memory type name mapping for pattern matching +static const std::string kMemoryTypeCpu = "cpu"; +static const std::string kMemoryTypeCuda = "cuda"; +static const std::string kMemoryTypeNpu = "npu"; +static const std::string kMemoryTypeWildcard = "*"; + +std::string TransportSelector::transportTypeName(TransportType type) { + auto it = kTransportTypeNames.find(type); + if (it != kTransportTypeNames.end()) { + return it->second; + } + return "unknown"; +} + +TransportType TransportSelector::parseTransportType(const std::string& str) { + auto it = kTransportNameMap.find(str); + if (it != kTransportNameMap.end()) { + return it->second; + } + LOG(WARNING) << "Unknown transport type: " << str; + return UNSPEC; +} + +std::vector TransportSelector::getDefaultPolicies() { + // Default policies that match the original hardcoded behavior + return { + { + "file_storage", + SegmentType::File, + std::nullopt, // same_machine doesn't matter for file + std::nullopt, // local_memory_pattern + std::nullopt, // remote_memory_pattern + std::nullopt, // min_size + std::nullopt, // max_size + std::nullopt, // priority + {}, // devices (empty = all devices) + {GDS, IOURING} // File segment priority (original: GDS -> IOURING) + }, + { + "memory_default", + SegmentType::Memory, + std::nullopt, // any machine + std::nullopt, // any local memory + std::nullopt, // any remote memory + std::nullopt, // any size + std::nullopt, // min_priority + {}, // devices (empty = all devices) + {} // Empty priority = use buffer_transports order (original + // behavior) + }, + }; +} + +void TransportSelector::loadPolicies() { + policies_.clear(); + + // Try to load from configuration - read from "policy" subtree + auto policies_array = config_->getArray("policy"); + + if (policies_array.empty()) { + LOG(INFO) + << "No 'policy' configured, using default transport selection"; + policies_ = getDefaultPolicies(); + return; + } + + LOG(INFO) << "Loading transport selection policies from config"; + + for (const auto& policy_json : policies_array) { + if (!policy_json.is_object()) continue; + + SelectionPolicy policy; + policy.name = policy_json.value("name", "unnamed"); + + // Parse segment_type + std::string segment_type_str = policy_json.value("segment_type", ""); + if (segment_type_str == "file") { + policy.segment_type = SegmentType::File; + } else if (segment_type_str == "memory") { + policy.segment_type = SegmentType::Memory; + } else { + LOG(WARNING) << "Invalid segment_type in policy " << policy.name; + continue; + } + + // Parse same_machine (optional) + if (policy_json.contains("same_machine")) { + policy.same_machine = policy_json["same_machine"].get(); + } else { + policy.same_machine = std::nullopt; + } + + // Parse memory patterns (optional) + if (policy_json.contains("local_memory")) { + policy.local_memory_pattern = + policy_json["local_memory"].get(); + } + if (policy_json.contains("remote_memory")) { + policy.remote_memory_pattern = + policy_json["remote_memory"].get(); + } + + // Parse size filters (optional) + if (policy_json.contains("min_size")) { + policy.min_size = policy_json["min_size"].get(); + } + if (policy_json.contains("max_size")) { + policy.max_size = policy_json["max_size"].get(); + } + + // Parse priority filter (optional) + if (policy_json.contains("priority")) { + if (policy_json["priority"].is_string()) { + // Parse string: "high", "medium", "low" + std::string prio_str = + policy_json["priority"].get(); + // Convert to lowercase for case-insensitive matching + std::transform(prio_str.begin(), prio_str.end(), + prio_str.begin(), ::tolower); + if (prio_str == "high" || prio_str == "0") { + policy.priority = PRIO_HIGH; + } else if (prio_str == "medium" || prio_str == "1") { + policy.priority = PRIO_MEDIUM; + } else if (prio_str == "low" || prio_str == "2") { + policy.priority = PRIO_LOW; + } else { + LOG(WARNING) << "Invalid priority string: " << prio_str + << ", using PRIO_LOW"; + policy.priority = PRIO_LOW; + } + } else { + // Parse integer for backward compatibility + policy.priority = policy_json["priority"].get(); + } + } else { + policy.priority = std::nullopt; + } + + // Parse devices (optional) + if (policy_json.contains("devices")) { + for (const auto& device_name : policy_json["devices"]) { + if (device_name.is_string()) { + policy.devices.push_back(device_name.get()); + } + } + } + + // Parse transports list + policy.transports.clear(); + if (policy_json.contains("transports")) { + for (const auto& transport_str : policy_json["transports"]) { + if (!transport_str.is_string()) continue; + TransportType type = + parseTransportType(transport_str.get()); + if (type != UNSPEC) { + policy.transports.push_back(type); + } + } + } + + policies_.push_back(std::move(policy)); + LOG(INFO) << "Loaded transport policy: " << policy.name + << " (segment_type=" << segment_type_str + << ", transports_count=" << policy.transports.size() << ")"; + } +} + +TransportSelector::TransportSelector(std::shared_ptr config) + : config_(config) { + loadPolicies(); +} + +bool TransportSelector::matchesMemoryPattern(const std::string& pattern, + MemoryType type) const { + if (pattern == kMemoryTypeWildcard) { + return true; + } + + // Convert MemoryType to string for comparison + std::string type_str; + switch (type) { + case MTYPE_CPU: + type_str = kMemoryTypeCpu; + break; + case MTYPE_CUDA: + type_str = kMemoryTypeCuda; + break; + case MTYPE_ROCM: + type_str = "rocm"; + break; + default: + type_str = "unknown"; + break; + } + + return pattern == type_str; +} + +bool TransportSelector::matchesPolicy(const SelectionPolicy& policy, + const SelectionContext& context) const { + // Check segment type + if (policy.segment_type != context.segment_type) { + return false; + } + + // Check same_machine constraint + if (policy.same_machine.has_value()) { + if (policy.same_machine.value() != context.same_machine) { + return false; + } + } + + // Check local memory pattern + if (policy.local_memory_pattern.has_value()) { + if (!matchesMemoryPattern(policy.local_memory_pattern.value(), + context.local_memory_type)) { + return false; + } + } + + // Check remote memory pattern + if (policy.remote_memory_pattern.has_value()) { + if (!matchesMemoryPattern(policy.remote_memory_pattern.value(), + context.remote_memory_type)) { + return false; + } + } + + // Check size constraints + if (policy.min_size.has_value()) { + if (context.transfer_size < policy.min_size.value()) { + return false; + } + } + if (policy.max_size.has_value()) { + if (context.transfer_size > policy.max_size.value()) { + return false; + } + } + + // Check priority constraint: exact match required + if (policy.priority.has_value()) { + if (context.priority_level != policy.priority.value()) { + return false; + } + } + + return true; +} + +bool TransportSelector::isTransportAvailable( + TransportType type, const SelectionContext& context, + const std::array, kSupportedTransportTypes>& + available_transports) const { + // Check if transport type is valid + if (type < 0 || type >= kSupportedTransportTypes) { + return false; + } + + // Check if transport exists + const auto& transport = available_transports[type]; + if (!transport) { + return false; + } + + // Special constraints + if ((type == NVLINK || type == SHM) && !context.same_machine) { + return false; // NVLINK and SHM only work on same machine + } + + const auto& caps = transport->capabilities(); + + // Helper to check if memory type is GPU/NPU + auto is_gpu = [](MemoryType t) { + return t == MTYPE_CUDA || t == MTYPE_ROCM; + }; + + // For file segments, check file-specific capabilities (original logic) + if (context.segment_type == SegmentType::File) { + if (context.local_memory_type == MTYPE_CPU) return caps.dram_to_file; + if (is_gpu(context.local_memory_type)) return caps.gpu_to_file; + return false; + } + + // Check transport capabilities for the memory type combination + MemoryType remote_mtype = context.remote_memory_type; + + // CPU to CPU + if (context.local_memory_type == MTYPE_CPU && remote_mtype == MTYPE_CPU) + return caps.dram_to_dram; + + // GPU/NPU to GPU/NPU (same type) + if (is_gpu(context.local_memory_type) && is_gpu(remote_mtype)) + return caps.gpu_to_gpu; + + // CPU to GPU/NPU + if (context.local_memory_type == MTYPE_CPU && is_gpu(remote_mtype)) + return caps.dram_to_gpu; + + // GPU/NPU to CPU + if (is_gpu(context.local_memory_type) && remote_mtype == MTYPE_CPU) + return caps.gpu_to_dram; + + return false; +} + +SelectionResult TransportSelector::select( + const SelectionContext& context, + const std::array, kSupportedTransportTypes>& + available_transports, + int transport_index) { + SelectionResult result; + + // Find the first matching policy (JSON order wins) + const SelectionPolicy* matching_policy = nullptr; + for (const auto& policy : policies_) { + if (matchesPolicy(policy, context)) { + matching_policy = &policy; + break; // First match wins + } + } + + if (!matching_policy) { + LOG(WARNING) << "No matching transport policy for segment_type=" + << (context.segment_type == SegmentType::File ? "file" + : "memory") + << ", size=" << context.transfer_size + << ", priority_level=" << context.priority_level; + return result; // UNSPEC, all devices + } + + // Convert device names to mask + result.device_mask = ~0ULL; // Default: all devices + if (!matching_policy->devices.empty() && topology_) { + result.device_mask = 0; + for (const auto& name : matching_policy->devices) { + int dev_id = topology_->getNicId(name); + if (dev_id >= 0 && dev_id < 64) { + result.device_mask |= (1ULL << dev_id); + } else { + LOG(WARNING) << "RDMA device not found or ID >= 64: " << name; + } + } + if (result.device_mask == 0) { + result.device_mask = ~0ULL; // Fallback to all if none found + } + } + + // If policy has transports list, use it + if (!matching_policy->transports.empty()) { + int priority_index = transport_index; + for (size_t i = 0; i < matching_policy->transports.size(); ++i) { + TransportType type = matching_policy->transports[i]; + if (isTransportAvailable(type, context, available_transports)) { + if (priority_index-- <= 0) { + result.transport = type; + VLOG(1) << "Selected transport " << transportTypeName(type) + << " for policy " << matching_policy->name + << ", device_mask=0x" << std::hex + << result.device_mask << std::dec; + return result; + } + } + } + return result; // UNSPEC + } + + // Otherwise, use buffer_transports order (original behavior) + if (context.buffer_transports) { + for (auto type : *context.buffer_transports) { + if (isTransportAvailable(type, context, available_transports)) { + if (transport_index-- <= 0) { + result.transport = type; + VLOG(1) << "Selected transport " << transportTypeName(type) + << " from buffer_transports" + << ", device_mask=0x" << std::hex + << result.device_mask << std::dec; + return result; + } + } + } + } + + return result; +} + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp index 90c4cdb8..f12e6427 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp @@ -379,7 +379,8 @@ Status RdmaTransport::submitTransferTasks( if (device_selector) { auto status = device_selector->allocate( request.length, static_cast(num_slices), - block_size, source_location, slice_dev_ids); + block_size, source_location, slice_dev_ids, + request.priority, batch->device_mask); if (!status.ok() || slice_dev_ids.empty()) { LOG(WARNING) << "Device quota allocation failed: " << status.message(); diff --git a/mooncake-transfer-engine/tent/tests/CMakeLists.txt b/mooncake-transfer-engine/tent/tests/CMakeLists.txt index 4b9ec56e..a1bc4d7f 100644 --- a/mooncake-transfer-engine/tent/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/tests/CMakeLists.txt @@ -96,6 +96,14 @@ target_include_directories(tent_rail_monitor_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) add_test(NAME tent_rail_monitor_test COMMAND tent_rail_monitor_test) +# Transport Selector Unit Test +add_executable(tent_transport_selector_test transport_selector_test.cpp) +target_link_libraries(tent_transport_selector_test PRIVATE gtest gtest_main + tent_link_group) +target_include_directories(tent_transport_selector_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) +add_test(NAME tent_transport_selector_test COMMAND tent_transport_selector_test) + # End-to-end failover test: drives real TransferEngineImpl with # FaultProxyTransport-wrapped fakes to exercise resubmitTransferTask. add_executable(tent_engine_failover_e2e_test engine_failover_e2e_test.cpp) diff --git a/mooncake-transfer-engine/tent/tests/transport_selector_test.cpp b/mooncake-transfer-engine/tent/tests/transport_selector_test.cpp new file mode 100644 index 00000000..9baac44e --- /dev/null +++ b/mooncake-transfer-engine/tent/tests/transport_selector_test.cpp @@ -0,0 +1,546 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include + +#include +#include + +#include "tent/common/config.h" +#include "tent/common/types.h" +#include "tent/runtime/transport_selector.h" +#include "tent/runtime/transport.h" + +namespace mooncake { +namespace tent { +namespace { + +// --------------------------------------------------------------------------- +// Minimal FakeTransport for availability checking +// --------------------------------------------------------------------------- + +class FakeTransport : public Transport { + public: + explicit FakeTransport(TransportType type) : type_(type) { + // Access protected member directly (inherited from Transport) + } + + Status install(std::string&, std::shared_ptr, + std::shared_ptr, + std::shared_ptr = nullptr) override { + return Status::OK(); + } + + Status allocateSubBatch(SubBatchRef&, size_t) override { + return Status::OK(); + } + + Status freeSubBatch(SubBatchRef&) override { return Status::OK(); } + + Status submitTransferTasks(SubBatchRef, + const std::vector&) override { + return Status::OK(); + } + + Status getTransferStatus(SubBatchRef, int, TransferStatus&) override { + return Status::OK(); + } + + const char* getName() const override { return "fake"; } + + // Helper to set capabilities (accessing protected member) + void setDramToFile(bool val) { caps.dram_to_file = val; } + void setGpuToFile(bool val) { caps.gpu_to_file = val; } + void setDramToDram(bool val) { caps.dram_to_dram = val; } + void setGpuToGpu(bool val) { caps.gpu_to_gpu = val; } + void setDramToGpu(bool val) { caps.dram_to_gpu = val; } + void setGpuToDram(bool val) { caps.gpu_to_dram = val; } + + private: + TransportType type_; +}; + +// --------------------------------------------------------------------------- +// Test default policies match original behavior +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, DefaultPoliciesFileSegment) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + // Create fake transports + std::array, kSupportedTransportTypes> + transports{}; + transports[GDS] = std::make_shared(GDS); + transports[IOURING] = std::make_shared(IOURING); + transports[RDMA] = std::make_shared(RDMA); + + // Set capabilities + auto* gds = static_cast(transports[GDS].get()); + auto* iouring = static_cast(transports[IOURING].get()); + gds->setDramToFile(true); + iouring->setDramToFile(true); + + // File segment with CPU memory should prefer GDS first + SelectionContext ctx; + ctx.segment_type = SegmentType::File; + ctx.same_machine = true; + ctx.local_memory_type = MTYPE_CPU; + ctx.remote_memory_type = MTYPE_CPU; + ctx.transfer_size = 4096; + ctx.priority_level = 0; + ctx.buffer_transports = nullptr; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, GDS) << "File segment should prefer GDS first"; +} + +TEST(TransportSelectorTest, DefaultPoliciesFileSegmentGpuMemory) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + // Create fake transports + std::array, kSupportedTransportTypes> + transports{}; + transports[GDS] = std::make_shared(GDS); + transports[IOURING] = std::make_shared(IOURING); + + // Set capabilities for GPU + auto* gds = static_cast(transports[GDS].get()); + auto* iouring = static_cast(transports[IOURING].get()); + gds->setGpuToFile(true); + iouring->setGpuToFile(true); + + // File segment with GPU memory + SelectionContext ctx; + ctx.segment_type = SegmentType::File; + ctx.same_machine = true; + ctx.local_memory_type = MTYPE_CUDA; + ctx.remote_memory_type = MTYPE_CPU; + ctx.transfer_size = 4096; + ctx.priority_level = 0; + ctx.buffer_transports = nullptr; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, GDS) + << "File segment with GPU should prefer GDS first"; +} + +TEST(TransportSelectorTest, DefaultPoliciesMemorySegment) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + // Create fake transports + std::array, kSupportedTransportTypes> + transports{}; + transports[RDMA] = std::make_shared(RDMA); + transports[TCP] = std::make_shared(TCP); + + // Set capabilities + auto* rdma = static_cast(transports[RDMA].get()); + rdma->setDramToDram(true); + + // Memory segment should use buffer_transports order + std::vector buffer_transports = {RDMA, TCP}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.same_machine = false; + ctx.local_memory_type = MTYPE_CPU; + ctx.remote_memory_type = MTYPE_CPU; + ctx.transfer_size = 4096; + ctx.priority_level = 0; + ctx.buffer_transports = &buffer_transports; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, RDMA) + << "Should use first in buffer_transports"; +} + +// --------------------------------------------------------------------------- +// Test transport type name parsing +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, TransportTypeNameMapping) { + EXPECT_EQ(TransportSelector::transportTypeName(RDMA), "rdma"); + EXPECT_EQ(TransportSelector::transportTypeName(TCP), "tcp"); + EXPECT_EQ(TransportSelector::transportTypeName(SHM), "shm"); + EXPECT_EQ(TransportSelector::transportTypeName(NVLINK), "nvlink"); + EXPECT_EQ(TransportSelector::transportTypeName(GDS), "gds"); + EXPECT_EQ(TransportSelector::transportTypeName(IOURING), "io_uring"); + EXPECT_EQ(TransportSelector::transportTypeName(AscendDirect), "ascend"); + EXPECT_EQ(TransportSelector::transportTypeName(MNNVL), "mnnvl"); +} + +TEST(TransportSelectorTest, ParseTransportType) { + EXPECT_EQ(TransportSelector::parseTransportType("rdma"), RDMA); + EXPECT_EQ(TransportSelector::parseTransportType("tcp"), TCP); + EXPECT_EQ(TransportSelector::parseTransportType("shm"), SHM); + EXPECT_EQ(TransportSelector::parseTransportType("nvlink"), NVLINK); + EXPECT_EQ(TransportSelector::parseTransportType("gds"), GDS); + EXPECT_EQ(TransportSelector::parseTransportType("io_uring"), IOURING); + EXPECT_EQ(TransportSelector::parseTransportType("ascend"), AscendDirect); + EXPECT_EQ(TransportSelector::parseTransportType("mnnvl"), MNNVL); + EXPECT_EQ(TransportSelector::parseTransportType("unknown"), UNSPEC); +} + +// --------------------------------------------------------------------------- +// Test legacy mode +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, LegacyModeDefaultOff) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + EXPECT_FALSE(selector.isLegacyMode()) + << "Legacy mode should be off by default"; +} + +TEST(TransportSelectorTest, LegacyModeCanBeEnabled) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + selector.setLegacyMode(true); + EXPECT_TRUE(selector.isLegacyMode()); + + selector.setLegacyMode(false); + EXPECT_FALSE(selector.isLegacyMode()); +} + +// --------------------------------------------------------------------------- +// Test transport availability based on capabilities +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, TransportCapabilityDramToDram) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[RDMA] = std::make_shared(RDMA); + auto* rdma = static_cast(transports[RDMA].get()); + rdma->setDramToDram(true); + + std::vector buffer_transports = {RDMA}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.local_memory_type = MTYPE_CPU; + ctx.remote_memory_type = MTYPE_CPU; + ctx.buffer_transports = &buffer_transports; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, RDMA) + << "RDMA should be available for CPU-to-CPU"; +} + +TEST(TransportSelectorTest, TransportCapabilityGpuToGpu) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[RDMA] = std::make_shared(RDMA); + auto* rdma = static_cast(transports[RDMA].get()); + rdma->setGpuToGpu(true); + + std::vector buffer_transports = {RDMA}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.local_memory_type = MTYPE_CUDA; + ctx.remote_memory_type = MTYPE_CUDA; + ctx.buffer_transports = &buffer_transports; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, RDMA) + << "RDMA should be available for CUDA-to-CUDA"; +} + +TEST(TransportSelectorTest, TransportCapabilityDramToGpu) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[RDMA] = std::make_shared(RDMA); + auto* rdma = static_cast(transports[RDMA].get()); + rdma->setDramToGpu(true); + + std::vector buffer_transports = {RDMA}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.local_memory_type = MTYPE_CPU; + ctx.remote_memory_type = MTYPE_CUDA; + ctx.buffer_transports = &buffer_transports; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, RDMA) + << "RDMA should be available for CPU-to-CUDA"; +} + +TEST(TransportSelectorTest, TransportCapabilityGpuToDram) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[RDMA] = std::make_shared(RDMA); + auto* rdma = static_cast(transports[RDMA].get()); + rdma->setGpuToDram(true); + + std::vector buffer_transports = {RDMA}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.local_memory_type = MTYPE_CUDA; + ctx.remote_memory_type = MTYPE_CPU; + ctx.buffer_transports = &buffer_transports; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, RDMA) + << "RDMA should be available for CUDA-to-CPU"; +} + +TEST(TransportSelectorTest, FileSegmentDramToFile) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[GDS] = std::make_shared(GDS); + auto* gds = static_cast(transports[GDS].get()); + gds->setDramToFile(true); + + SelectionContext ctx; + ctx.segment_type = SegmentType::File; + ctx.local_memory_type = MTYPE_CPU; + ctx.buffer_transports = nullptr; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, GDS) + << "GDS should be available for File segment with CPU memory"; +} + +TEST(TransportSelectorTest, FileSegmentGpuToFile) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[GDS] = std::make_shared(GDS); + auto* gds = static_cast(transports[GDS].get()); + gds->setGpuToFile(true); + + SelectionContext ctx; + ctx.segment_type = SegmentType::File; + ctx.local_memory_type = MTYPE_CUDA; + ctx.buffer_transports = nullptr; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, GDS) + << "GDS should be available for File segment with CUDA memory"; +} + +// --------------------------------------------------------------------------- +// Test priority_offset for fallback +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, PriorityOffsetFallback) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[RDMA] = std::make_shared(RDMA); + transports[TCP] = std::make_shared(TCP); + + auto* rdma = static_cast(transports[RDMA].get()); + rdma->setDramToDram(true); + auto* tcp = static_cast(transports[TCP].get()); + tcp->setDramToDram(true); + + std::vector buffer_transports = {RDMA, TCP}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.same_machine = false; + ctx.local_memory_type = MTYPE_CPU; + ctx.remote_memory_type = MTYPE_CPU; + ctx.buffer_transports = &buffer_transports; + + // priority_offset=0: first transport + auto result = selector.select(ctx, transports, 0); + EXPECT_EQ(result.transport, RDMA); + + // priority_offset=1: second transport (fallback) + result = selector.select(ctx, transports, 1); + EXPECT_EQ(result.transport, TCP); + + // priority_offset=2: no more transports + result = selector.select(ctx, transports, 2); + EXPECT_EQ(result.transport, UNSPEC); +} + +// --------------------------------------------------------------------------- +// Test device mask handling +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, DeviceMaskDefaultAll) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.buffer_transports = nullptr; + + std::array, kSupportedTransportTypes> + transports{}; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.device_mask, ~0ULL) + << "Default device mask should be all devices"; +} + +// --------------------------------------------------------------------------- +// Test NVLINK and SHM same-machine constraint (via select) +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, NvLinkRequiresSameMachine) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[NVLINK] = std::make_shared(NVLINK); + auto* nvlink = static_cast(transports[NVLINK].get()); + nvlink->setGpuToGpu(true); + + std::vector buffer_transports = {NVLINK}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.same_machine = false; // Remote + ctx.local_memory_type = MTYPE_CUDA; + ctx.remote_memory_type = MTYPE_CUDA; + ctx.buffer_transports = &buffer_transports; + + // NVLINK should not be available for remote transfers + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, UNSPEC) + << "NVLINK should not be available for remote transfers"; +} + +TEST(TransportSelectorTest, NvLinkAvailableSameMachine) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[NVLINK] = std::make_shared(NVLINK); + auto* nvlink = static_cast(transports[NVLINK].get()); + nvlink->setGpuToGpu(true); + + std::vector buffer_transports = {NVLINK}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.same_machine = true; + ctx.local_memory_type = MTYPE_CUDA; + ctx.remote_memory_type = MTYPE_CUDA; + ctx.buffer_transports = &buffer_transports; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, NVLINK) + << "NVLINK should be available for same-machine transfers"; +} + +// --------------------------------------------------------------------------- +// Test ROCm memory type +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, RocmMemoryTypeSupported) { + auto conf = std::make_shared(); + TransportSelector selector(conf); + + std::array, kSupportedTransportTypes> + transports{}; + transports[RDMA] = std::make_shared(RDMA); + auto* rdma = static_cast(transports[RDMA].get()); + rdma->setGpuToGpu(true); + + std::vector buffer_transports = {RDMA}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.same_machine = true; + ctx.local_memory_type = MTYPE_ROCM; + ctx.remote_memory_type = MTYPE_ROCM; + ctx.buffer_transports = &buffer_transports; + + auto result = selector.select(ctx, transports); + EXPECT_EQ(result.transport, RDMA) + << "RDMA should be available for ROCm-to-ROCm"; +} + +// --------------------------------------------------------------------------- +// Test with configuration (policy-based selection) +// --------------------------------------------------------------------------- + +TEST(TransportSelectorTest, ConfigBasedPolicySelection) { + auto conf = std::make_shared(); + + // Set up a custom policy via JSON config + conf->set("policy", json::array()); + auto policies = conf->getArray("policy"); + + json policy; + policy["name"] = "test_memory_policy"; + policy["segment_type"] = "memory"; + policy["transports"] = {"tcp", "rdma"}; // Prefer TCP over RDMA + + // We can't easily modify the config's internal JSON structure, + // so this test verifies the selector at least loads without error + + TransportSelector selector(conf); + + // Default behavior should still work + std::array, kSupportedTransportTypes> + transports{}; + transports[RDMA] = std::make_shared(RDMA); + transports[TCP] = std::make_shared(TCP); + + auto* rdma = static_cast(transports[RDMA].get()); + rdma->setDramToDram(true); + auto* tcp = static_cast(transports[TCP].get()); + tcp->setDramToDram(true); + + std::vector buffer_transports = {RDMA, TCP}; + + SelectionContext ctx; + ctx.segment_type = SegmentType::Memory; + ctx.same_machine = false; + ctx.local_memory_type = MTYPE_CPU; + ctx.remote_memory_type = MTYPE_CPU; + ctx.buffer_transports = &buffer_transports; + + auto result = selector.select(ctx, transports); + // With default policies, should use buffer_transports order (RDMA first) + EXPECT_EQ(result.transport, RDMA); +} + +} // namespace +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tests/endpoint_store_integration_test.cpp b/mooncake-transfer-engine/tests/endpoint_store_integration_test.cpp index c724dc11..2972fa12 100644 --- a/mooncake-transfer-engine/tests/endpoint_store_integration_test.cpp +++ b/mooncake-transfer-engine/tests/endpoint_store_integration_test.cpp @@ -62,7 +62,11 @@ std::string pickRdmaDevice() { if (override_name && *override_name) return override_name; int num_devices = 0; ibv_device **list = ibv_get_device_list(&num_devices); - if (!list || num_devices == 0) return ""; + if (!list) return ""; + if (num_devices == 0) { + ibv_free_device_list(list); + return ""; + } std::string name = ibv_get_device_name(list[0]); ibv_free_device_list(list); return name; -- 2.34.1 From d6b661d5a46a83a6a2e418949405271c09330ea4 Mon Sep 17 00:00:00 2001 From: Kafka <92581531+ZhenyuePan@users.noreply.github.com> Date: Wed, 27 May 2026 10:58:54 +0800 Subject: [PATCH 134/382] [TE] Add ProgressWorker skeleton (#2199) * [TE] Add explicit progressBatch API * [TE] Add progress worker skeleton --- .../tent/config/transfer-engine.json | 1 + .../include/tent/runtime/progress_worker.h | 74 +++ .../tent/runtime/transfer_engine_impl.h | 28 +- .../tent/include/tent/transfer_engine.h | 8 + .../tent/src/runtime/progress_worker.cpp | 81 +++ .../tent/src/runtime/proxy_manager.cpp | 3 +- .../tent/src/runtime/transfer_engine_impl.cpp | 130 +++-- .../tent/src/transfer_engine.cpp | 5 + .../tent/tests/CMakeLists.txt | 13 + .../tent/tests/engine_failover_e2e_test.cpp | 292 ++++++++++- .../tent/tests/progress_worker_test.cpp | 487 ++++++++++++++++++ 11 files changed, 1069 insertions(+), 53 deletions(-) create mode 100644 mooncake-transfer-engine/tent/include/tent/runtime/progress_worker.h create mode 100644 mooncake-transfer-engine/tent/src/runtime/progress_worker.cpp create mode 100644 mooncake-transfer-engine/tent/tests/progress_worker_test.cpp diff --git a/mooncake-transfer-engine/tent/config/transfer-engine.json b/mooncake-transfer-engine/tent/config/transfer-engine.json index 01932aef..e5bdbbd1 100644 --- a/mooncake-transfer-engine/tent/config/transfer-engine.json +++ b/mooncake-transfer-engine/tent/config/transfer-engine.json @@ -12,6 +12,7 @@ "log_level": "warning", "max_failover_attempts": 3, "enable_auto_failover_on_poll": true, + "enable_progress_worker": false, "metrics": { "enabled": true, "http_port": 9100, diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/progress_worker.h b/mooncake-transfer-engine/tent/include/tent/runtime/progress_worker.h new file mode 100644 index 00000000..de8ed066 --- /dev/null +++ b/mooncake-transfer-engine/tent/include/tent/runtime/progress_worker.h @@ -0,0 +1,74 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#ifndef PROGRESS_WORKER_H_ +#define PROGRESS_WORKER_H_ + +#include +#include +#include +#include +#include +#include + +#include "tent/common/types.h" + +namespace mooncake { +namespace tent { + +class TransferEngineImpl; + +// Event-driven progress worker for issue #2116. When the engine is configured +// with enable_progress_worker=true, transports (or test hooks) call +// notifyBatchMaybeReady to wake this worker, which then drives one +// progressBatch step per notification. This decouples failover/resubmit from +// the caller polling loop, so integrators that turn off +// enable_auto_failover_on_poll do not need to spin a polling thread of their +// own to keep failover progressing. +class ProgressWorker { + public: + explicit ProgressWorker(TransferEngineImpl* impl); + ~ProgressWorker(); + + ProgressWorker(const ProgressWorker&) = delete; + ProgressWorker& operator=(const ProgressWorker&) = delete; + + void start(); + + // Idempotent. Signals the worker thread to exit and joins it. After stop() + // returns, notifyBatchMaybeReady becomes a no-op. + void stop(); + + // Safe from any thread. De-duplicates: enqueueing a batch that is already + // queued is a no-op. No-op if the worker has been stopped or never + // started. + void notifyBatchMaybeReady(BatchID batch_id); + + private: + void runner(); + + TransferEngineImpl* impl_; + std::atomic running_{false}; + std::thread thread_; + + std::mutex mu_; + std::condition_variable cv_; + std::unordered_set queued_; + std::deque order_; +}; + +} // namespace tent +} // namespace mooncake + +#endif // PROGRESS_WORKER_H_ diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h b/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h index c5b94e67..d3191df4 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/transfer_engine_impl.h @@ -46,6 +46,7 @@ class ControlService; class SegmentTracker; class Platform; class ProxyManager; +class ProgressWorker; struct TaskInfo { TransportType type{UNSPEC}; @@ -152,6 +153,8 @@ class TransferEngineImpl { Status getTransferStatus(BatchID batch_id, TransferStatus& overall_status); + Status progressBatch(BatchID batch_id, TransferStatus& overall_status); + Status waitTransferCompletion(BatchID batch_id); Status transferSync(const std::vector& request_list); @@ -172,6 +175,11 @@ class TransferEngineImpl { } } + // Wake the optional event-driven progress worker for `batch_id`. No-op if + // enable_progress_worker is false. Currently used by test/integration + // hooks; transports will be migrated to call this in a follow-up PR. + void notifyBatchMaybeReady(BatchID batch_id); + private: Status construct(); @@ -189,8 +197,15 @@ class TransferEngineImpl { Status resubmitTransferTask(Batch* batch, size_t task_id); - void updateTaskStatusFromPoll(Batch* batch, size_t task_id, - TransferStatus& task_status); + Status pollTaskStatus(Batch* batch, size_t task_id, + TransferStatus& task_status); + + void updateTaskStatusAfterPoll(Batch* batch, size_t task_id, + TransferStatus& task_status, + bool allow_failover); + + Status getBatchStatus(BatchID batch_id, TransferStatus& overall_status, + bool allow_failover); SelectionResult resolveTransport(const Request& req, int transport_index, bool invalidate_on_fail = true); @@ -244,6 +259,15 @@ class TransferEngineImpl { bool merge_requests_; int max_failover_attempts_{3}; bool enable_auto_failover_on_poll_{true}; + bool enable_progress_worker_{false}; + + // Guards alive_batches_ and serializes pollTaskStatus / + // updateTaskStatusAfterPoll / lazyFreeBatch against the optional + // ProgressWorker thread. Recursive because freeBatch -> lazyFreeBatch -> + // getTransferStatus can re-enter on the same thread. See issue #2116. + std::recursive_mutex progress_mutex_; + std::unordered_set alive_batches_; + std::unique_ptr progress_worker_; }; } // namespace tent } // namespace mooncake diff --git a/mooncake-transfer-engine/tent/include/tent/transfer_engine.h b/mooncake-transfer-engine/tent/include/tent/transfer_engine.h index 68b92bff..3a4d1d0f 100644 --- a/mooncake-transfer-engine/tent/include/tent/transfer_engine.h +++ b/mooncake-transfer-engine/tent/include/tent/transfer_engine.h @@ -308,6 +308,14 @@ class TransferEngine { Status getTransferStatus(BatchID batch_id, TransferStatus& overall_status); + // Drive one progress step on a batch and return its aggregated status. + // Unlike getTransferStatus, this always allows internal failover/resubmit + // regardless of enable_auto_failover_on_poll. The call is non-blocking and + // performs at most one state-machine step per task; callers that want to + // wait for completion must invoke it in a loop. PENDING means "make + // progress later"; terminal states (COMPLETED/FAILED) will not be revived. + Status progressBatch(BatchID batch_id, TransferStatus& overall_status); + private: std::unique_ptr impl_; }; diff --git a/mooncake-transfer-engine/tent/src/runtime/progress_worker.cpp b/mooncake-transfer-engine/tent/src/runtime/progress_worker.cpp new file mode 100644 index 00000000..bb52b7af --- /dev/null +++ b/mooncake-transfer-engine/tent/src/runtime/progress_worker.cpp @@ -0,0 +1,81 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "tent/runtime/progress_worker.h" + +#include "tent/common/status.h" +#include "tent/runtime/transfer_engine_impl.h" + +namespace mooncake { +namespace tent { + +ProgressWorker::ProgressWorker(TransferEngineImpl* impl) : impl_(impl) {} + +ProgressWorker::~ProgressWorker() { stop(); } + +void ProgressWorker::start() { + if (running_.exchange(true, std::memory_order_acq_rel)) return; + thread_ = std::thread(&ProgressWorker::runner, this); +} + +void ProgressWorker::stop() { + if (!running_.exchange(false, std::memory_order_acq_rel)) return; + { + std::lock_guard lk(mu_); + // Drop pending work; outstanding batches will be reaped via the + // user thread's freeBatch path. + order_.clear(); + queued_.clear(); + } + cv_.notify_all(); + if (thread_.joinable()) thread_.join(); +} + +void ProgressWorker::notifyBatchMaybeReady(BatchID batch_id) { + if (!batch_id) return; + if (!running_.load(std::memory_order_acquire)) return; + { + std::lock_guard lk(mu_); + if (!queued_.insert(batch_id).second) return; + order_.push_back(batch_id); + } + cv_.notify_one(); +} + +void ProgressWorker::runner() { + while (true) { + BatchID batch_id = 0; + { + std::unique_lock lk(mu_); + cv_.wait(lk, [&] { + return !running_.load(std::memory_order_acquire) || + !order_.empty(); + }); + if (!running_.load(std::memory_order_acquire)) return; + batch_id = order_.front(); + order_.pop_front(); + queued_.erase(batch_id); + } + // progressBatch acquires the engine's progress_mutex_ and silently + // returns InvalidArgument if the batch was freed before we got here. + // PENDING means "kick again later"; the next notify wakes us up. + // Terminal states leave the batch alone — freeBatch on the user + // thread is responsible for reclamation. + TransferStatus s; + (void)impl_->progressBatch(batch_id, s); + } +} + +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/src/runtime/proxy_manager.cpp b/mooncake-transfer-engine/tent/src/runtime/proxy_manager.cpp index 4ea80bd3..f5aa8019 100644 --- a/mooncake-transfer-engine/tent/src/runtime/proxy_manager.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/proxy_manager.cpp @@ -385,8 +385,7 @@ Status ProxyManager::transferEventLoop(StagingTask& task, case StageState::INFLIGHT: { TransferStatus xfer_status; - CHECK_STATUS( - impl_->getTransferStatus(chunk.batch, xfer_status)); + CHECK_STATUS(impl_->progressBatch(chunk.batch, xfer_status)); if (xfer_status.s == PENDING) { event_queue.push(id); break; diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index 872008e0..015d0112 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -30,6 +30,7 @@ #include "tent/runtime/control_plane.h" #include "tent/runtime/segment.h" #include "tent/runtime/segment_tracker.h" +#include "tent/runtime/progress_worker.h" #include "tent/runtime/proxy_manager.h" #include "tent/runtime/transport.h" #include "tent/runtime/topology.h" @@ -306,6 +307,7 @@ Status TransferEngineImpl::construct() { max_failover_attempts_ = conf_->get("max_failover_attempts", 3); enable_auto_failover_on_poll_ = conf_->get("enable_auto_failover_on_poll", true); + enable_progress_worker_ = conf_->get("enable_progress_worker", false); if (!hostname_.empty()) CHECK_STATUS(checkLocalIpAddress(hostname_, ipv6_)); else @@ -370,6 +372,11 @@ Status TransferEngineImpl::construct() { staging_proxy_ = std::make_unique(this); + if (enable_progress_worker_) { + progress_worker_ = std::make_unique(this); + progress_worker_->start(); + } + // Initialize and start Metrics system auto metrics_config = MetricsConfigLoader::loadWithDefaults(conf_.get()); if (metrics_config.enabled) { @@ -412,6 +419,13 @@ Status TransferEngineImpl::construct() { Status TransferEngineImpl::deconstruct() { // Metrics cleanup is handled automatically by TentMetrics destructor + // Stop the progress worker first so it cannot race with batch teardown + // below (it dereferences BatchID into Batch* via progressBatch). + if (progress_worker_) { + progress_worker_->stop(); + progress_worker_.reset(); + } + // Destroy staging_proxy_ first: its destructor calls back into // unregisterLocalMemory/freeLocalMemory, which require // local_segment_tracker_ and metadata_ to be alive. @@ -750,18 +764,25 @@ BatchID TransferEngineImpl::allocateBatch(size_t batch_size) { if (!batch) return (BatchID)0; batch->max_size = batch_size; batch_set_.get().active.insert(batch); - return (BatchID)batch; + BatchID batch_id = (BatchID)batch; + { + std::lock_guard lk(progress_mutex_); + alive_batches_.insert(batch_id); + } + return batch_id; } Status TransferEngineImpl::freeBatch(BatchID batch_id) { if (!batch_id) return Status::InvalidArgument("Invalid batch ID" LOC_MARK); Batch* batch = (Batch*)(batch_id); + std::lock_guard lk(progress_mutex_); batch_set_.get().freelist.push_back(batch); lazyFreeBatch(); return Status::OK(); } Status TransferEngineImpl::lazyFreeBatch() { + // Caller must hold progress_mutex_. auto& batch_set = batch_set_.get(); for (auto it = batch_set.freelist.begin(); it != batch_set.freelist.end();) { @@ -778,6 +799,7 @@ Status TransferEngineImpl::lazyFreeBatch() { if (transport && sub_batch) transport->freeSubBatch(sub_batch); } batch_set.active.erase(batch); + alive_batches_.erase((BatchID)batch); Slab::Get().deallocate(batch); it = batch_set.freelist.erase(it); } @@ -1379,11 +1401,35 @@ Status TransferEngineImpl::resubmitTransferTask(Batch* batch, size_t task_id) { return transport->submitTransferTasks(sub_batch, {task.request}); } -void TransferEngineImpl::updateTaskStatusFromPoll(Batch* batch, size_t task_id, - TransferStatus& task_status) { +Status TransferEngineImpl::pollTaskStatus(Batch* batch, size_t task_id, + TransferStatus& task_status) { + auto& task = batch->task_list[task_id]; + if (task.staging) { + return staging_proxy_->getStatus(&task, task_status); + } + + if (task.type == UNSPEC) { + task_status.s = FAILED; + task_status.transferred_bytes = 0; + return Status::OK(); + } + + auto& transport = transport_list_[task.type]; + auto& sub_batch = batch->sub_batch[task.type]; + if (!transport || !sub_batch) { + return Status::InvalidArgument("Transport not available" LOC_MARK); + } + return transport->getTransferStatus(sub_batch, task.sub_task_id, + task_status); +} + +void TransferEngineImpl::updateTaskStatusAfterPoll(Batch* batch, size_t task_id, + TransferStatus& task_status, + bool allow_failover) { auto& task = batch->task_list[task_id]; task.status = task_status.s; - if (!enable_auto_failover_on_poll_ || task_status.s != FAILED) return; + if (!allow_failover || task_status.s != FAILED || task.type == UNSPEC) + return; if (resubmitTransferTask(batch, task_id).ok()) { task_status.s = PENDING; @@ -1433,29 +1479,17 @@ Status TransferEngineImpl::receiveNotification( Status TransferEngineImpl::getTransferStatus(BatchID batch_id, size_t task_id, TransferStatus& task_status) { if (!batch_id) return Status::InvalidArgument("Invalid batch ID" LOC_MARK); + std::lock_guard lk(progress_mutex_); + if (!alive_batches_.count(batch_id)) + return Status::InvalidArgument("Batch is not alive" LOC_MARK); Batch* batch = (Batch*)(batch_id); if (task_id >= batch->task_list.size()) return Status::InvalidArgument("Invalid task ID" LOC_MARK); auto& task = batch->task_list[task_id]; auto prev_status = task.status; - if (task.staging) { - CHECK_STATUS(staging_proxy_->getStatus(&task, task_status)); - } else { - if (task.type == UNSPEC) { - task_status.s = FAILED; - task_status.transferred_bytes = 0; - batch->task_list[task_id].status = task_status.s; - return Status::OK(); - } - auto& transport = transport_list_[task.type]; - auto& sub_batch = batch->sub_batch[task.type]; - if (!transport || !sub_batch) { - return Status::InvalidArgument("Transport not available" LOC_MARK); - } - CHECK_STATUS(transport->getTransferStatus(sub_batch, task.sub_task_id, - task_status)); - } - updateTaskStatusFromPoll(batch, task_id, task_status); + CHECK_STATUS(pollTaskStatus(batch, task_id, task_status)); + updateTaskStatusAfterPoll(batch, task_id, task_status, + enable_auto_failover_on_poll_); // Record metrics when task transitions to terminal state recordTaskCompletionMetrics(batch->task_list[task_id], prev_status, @@ -1468,6 +1502,9 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, size_t task_id, Status TransferEngineImpl::getTransferStatus( BatchID batch_id, std::vector& status_list) { if (!batch_id) return Status::InvalidArgument("Invalid batch ID" LOC_MARK); + std::lock_guard lk(progress_mutex_); + if (!alive_batches_.count(batch_id)) + return Status::InvalidArgument("Batch is not alive" LOC_MARK); Batch* batch = (Batch*)(batch_id); status_list.clear(); for (size_t task_id = 0; task_id < batch->task_list.size(); ++task_id) { @@ -1478,9 +1515,13 @@ Status TransferEngineImpl::getTransferStatus( return Status::OK(); } -Status TransferEngineImpl::getTransferStatus(BatchID batch_id, - TransferStatus& overall_status) { +Status TransferEngineImpl::getBatchStatus(BatchID batch_id, + TransferStatus& overall_status, + bool allow_failover) { if (!batch_id) return Status::InvalidArgument("Invalid batch ID" LOC_MARK); + std::lock_guard lk(progress_mutex_); + if (!alive_batches_.count(batch_id)) + return Status::InvalidArgument("Batch is not alive" LOC_MARK); Batch* batch = (Batch*)(batch_id); overall_status.s = PENDING; overall_status.transferred_bytes = 0; @@ -1512,26 +1553,8 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, continue; } auto prev_status = task.status; - if (task.staging) { - CHECK_STATUS(staging_proxy_->getStatus(&task, task_status)); - } else { - if (task.type == UNSPEC) { - task.status = FAILED; - failed_tasks++; - if (isWorse(FAILED, worst_failure)) worst_failure = FAILED; - continue; - } - auto& transport = transport_list_[task.type]; - auto& sub_batch = batch->sub_batch[task.type]; - if (!transport || !sub_batch) { - return Status::InvalidArgument( - "Transport not available" LOC_MARK); - } - CHECK_STATUS(transport->getTransferStatus( - sub_batch, task.sub_task_id, task_status)); - } - // Preserve legacy auto-failover-on-poll before aggregating status. - updateTaskStatusFromPoll(batch, task_id, task_status); + CHECK_STATUS(pollTaskStatus(batch, task_id, task_status)); + updateTaskStatusAfterPoll(batch, task_id, task_status, allow_failover); if (task_status.s == COMPLETED) { success_tasks++; @@ -1559,10 +1582,25 @@ Status TransferEngineImpl::getTransferStatus(BatchID batch_id, return Status::OK(); } +Status TransferEngineImpl::getTransferStatus(BatchID batch_id, + TransferStatus& overall_status) { + return getBatchStatus(batch_id, overall_status, + enable_auto_failover_on_poll_); +} + +Status TransferEngineImpl::progressBatch(BatchID batch_id, + TransferStatus& overall_status) { + return getBatchStatus(batch_id, overall_status, true); +} + +void TransferEngineImpl::notifyBatchMaybeReady(BatchID batch_id) { + if (progress_worker_) progress_worker_->notifyBatchMaybeReady(batch_id); +} + Status TransferEngineImpl::waitTransferCompletion(BatchID batch_id) { TransferStatus xfer_status; while (true) { - CHECK_STATUS(getTransferStatus(batch_id, xfer_status)); + CHECK_STATUS(progressBatch(batch_id, xfer_status)); if (xfer_status.s != PENDING) { freeBatch(batch_id); return xfer_status.s == COMPLETED @@ -1580,7 +1618,7 @@ Status TransferEngineImpl::transferSync( CHECK_STATUS(submitTransfer(batch_id, request_list)); while (true) { TransferStatus xfer_status; - CHECK_STATUS(getTransferStatus(batch_id, xfer_status)); + CHECK_STATUS(progressBatch(batch_id, xfer_status)); if (xfer_status.s == COMPLETED) break; if (xfer_status.s != PENDING) { CHECK_STATUS(freeBatch(batch_id)); diff --git a/mooncake-transfer-engine/tent/src/transfer_engine.cpp b/mooncake-transfer-engine/tent/src/transfer_engine.cpp index 509ea195..5628a45c 100644 --- a/mooncake-transfer-engine/tent/src/transfer_engine.cpp +++ b/mooncake-transfer-engine/tent/src/transfer_engine.cpp @@ -169,5 +169,10 @@ Status TransferEngine::getTransferStatus(BatchID batch_id, return impl_->getTransferStatus(batch_id, overall_status); } +Status TransferEngine::progressBatch(BatchID batch_id, + TransferStatus& overall_status) { + return impl_->progressBatch(batch_id, overall_status); +} + } // namespace tent } // namespace mooncake diff --git a/mooncake-transfer-engine/tent/tests/CMakeLists.txt b/mooncake-transfer-engine/tent/tests/CMakeLists.txt index a1bc4d7f..be402a67 100644 --- a/mooncake-transfer-engine/tent/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/tests/CMakeLists.txt @@ -116,3 +116,16 @@ target_include_directories(tent_engine_failover_e2e_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) add_test(NAME tent_engine_failover_e2e_test COMMAND tent_engine_failover_e2e_test) + +# ProgressWorker skeleton test: covers default-off behavior, event-driven +# progress without poll-failover, and freeBatch races (issue #2116). +add_executable(tent_progress_worker_test progress_worker_test.cpp) +target_link_libraries(tent_progress_worker_test + PRIVATE gtest gtest_main tent_link_group) +if(TARGET asio_shared) + target_link_libraries(tent_progress_worker_test PRIVATE asio_shared) +endif() +target_include_directories(tent_progress_worker_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) +add_test(NAME tent_progress_worker_test + COMMAND tent_progress_worker_test) diff --git a/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp b/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp index d2500c7a..7100d991 100644 --- a/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp +++ b/mooncake-transfer-engine/tent/tests/engine_failover_e2e_test.cpp @@ -65,12 +65,23 @@ class FakeSubBatch : public Transport::SubBatch { public: size_t size() const override { return task_count; } size_t task_count = 0; + std::vector requests; std::vector statuses; + std::vector poll_counts; }; class FakeTransport : public Transport { public: - explicit FakeTransport(TransportType self_type) : self_type_(self_type) { + using StatusFactory = std::function; + using PollStatusFactory = + std::function; + + explicit FakeTransport(TransportType self_type, + StatusFactory status_factory = {}, + PollStatusFactory poll_status_factory = {}) + : self_type_(self_type), + status_factory_(std::move(status_factory)), + poll_status_factory_(std::move(poll_status_factory)) { caps.dram_to_dram = true; // so checkAvailability returns true } @@ -103,7 +114,14 @@ class FakeTransport : public Transport { ++submit_calls; auto* fb = static_cast(batch); for (const auto& req : request_list) { - fb->statuses.push_back({TransferStatusEnum::COMPLETED, req.length}); + if (status_factory_) { + fb->statuses.push_back(status_factory_(req)); + } else { + fb->statuses.push_back( + {TransferStatusEnum::COMPLETED, req.length}); + } + fb->requests.push_back(req); + fb->poll_counts.push_back(0); fb->task_count++; } return Status::OK(); @@ -116,7 +134,13 @@ class FakeTransport : public Transport { if (task_id < 0 || task_id >= (int)fb->statuses.size()) { return Status::InvalidArgument("bad task_id" LOC_MARK); } - status = fb->statuses[task_id]; + ++fb->poll_counts[task_id]; + if (poll_status_factory_) { + status = poll_status_factory_(fb->requests[task_id], + fb->poll_counts[task_id]); + } else { + status = fb->statuses[task_id]; + } return Status::OK(); } @@ -163,6 +187,8 @@ class FakeTransport : public Transport { private: TransportType self_type_; + StatusFactory status_factory_; + PollStatusFactory poll_status_factory_; }; // --------------------------------------------------------------------------- @@ -373,6 +399,266 @@ TEST(EngineFailoverE2E, AutoFailoverOnPollDisabledAppliesToOverallStatus) { engine.unregisterLocalMemory(batch.buf.data(), batch.buf.size()).ok()); } +// --------------------------------------------------------------------------- +// P0c: Explicit progressBatch() drives one progress step and always allows +// failover/resubmit, regardless of enable_auto_failover_on_poll. Internal +// sync paths (waitTransferCompletion, transferSync) and the proxy event loop +// are wired through it so observation-only callers stay decoupled from +// progress-driving callers. +// --------------------------------------------------------------------------- + +TEST(EngineFailoverE2E, ProgressBatchRetriesWhenPollAutoFailoverDisabled) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + CorruptedRdmaBatch batch; + submitCorruptedRdmaBatch(engine, batch, 0xA4); + + TransferStatus overall_status{}; + ASSERT_TRUE(engine.progressBatch(batch.batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::PENDING); + EXPECT_EQ(batch.fake_rdma->submit_calls.load(), 1); + EXPECT_EQ(batch.fake_tcp->submit_calls.load(), 1); + EXPECT_EQ(batch.fake_tcp->status_calls.load(), 0) + << "progressBatch should perform one progress step, not poll the " + "fallback submission immediately"; + + ASSERT_TRUE(engine.progressBatch(batch.batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::COMPLETED); + EXPECT_EQ(batch.fake_tcp->status_calls.load(), 1); + + EXPECT_TRUE(engine.freeBatch(batch.batch_id).ok()); + EXPECT_TRUE( + engine.unregisterLocalMemory(batch.buf.data(), batch.buf.size()).ok()); +} + +TEST(EngineFailoverE2E, ProgressBatchDoesNotReviveObservedFailedTask) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + CorruptedRdmaBatch batch; + submitCorruptedRdmaBatch(engine, batch, 0xA5); + + TransferStatus overall_status{}; + ASSERT_TRUE(engine.getTransferStatus(batch.batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::FAILED); + EXPECT_EQ(batch.fake_tcp->submit_calls.load(), 0); + + ASSERT_TRUE(engine.progressBatch(batch.batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::FAILED); + EXPECT_EQ(batch.fake_tcp->submit_calls.load(), 0); + + EXPECT_TRUE(engine.freeBatch(batch.batch_id).ok()); + EXPECT_TRUE( + engine.unregisterLocalMemory(batch.buf.data(), batch.buf.size()).ok()); +} + +TEST(EngineFailoverE2E, ProgressBatchHonorsMaxFailoverAttemptsZero) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + cfg->set("max_failover_attempts", 0); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + CorruptedRdmaBatch batch; + submitCorruptedRdmaBatch(engine, batch, 0xA6); + + TransferStatus overall_status{}; + ASSERT_TRUE(engine.progressBatch(batch.batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::FAILED); + EXPECT_EQ(batch.fake_rdma->submit_calls.load(), 1); + EXPECT_EQ(batch.fake_tcp->submit_calls.load(), 0); + + EXPECT_TRUE(engine.freeBatch(batch.batch_id).ok()); + EXPECT_TRUE( + engine.unregisterLocalMemory(batch.buf.data(), batch.buf.size()).ok()); +} + +TEST(EngineFailoverE2E, ProgressBatchKeepsOverallPendingWithMixedOutcomes) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + cfg->set("max_failover_attempts", 0); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + constexpr size_t kBufLen = 4096; + std::vector failing_buf(kBufLen, 0xA7); + std::vector pending_buf(kBufLen, 0xA8); + const uint64_t failing_addr = + reinterpret_cast(failing_buf.data()); + + auto fake_rdma = std::make_shared( + RDMA, FakeTransport::StatusFactory{}, + [failing_addr](const Request& req, int poll_count) { + if (req.target_offset == failing_addr) { + return TransferStatus{TransferStatusEnum::FAILED, 0}; + } + if (poll_count == 1) { + return TransferStatus{TransferStatusEnum::PENDING, 0}; + } + return TransferStatus{TransferStatusEnum::COMPLETED, req.length}; + }); + auto fake_tcp = std::make_shared(TCP); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(fake_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg_name, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, fake_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + ASSERT_TRUE(engine.registerLocalMemory(failing_buf.data(), kBufLen).ok()); + ASSERT_TRUE(engine.registerLocalMemory(pending_buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(2); + ASSERT_NE(batch_id, (BatchID)0); + + Request failing_req; + failing_req.opcode = Request::WRITE; + failing_req.source = failing_buf.data(); + failing_req.target_id = LOCAL_SEGMENT_ID; + failing_req.target_offset = failing_addr; + failing_req.length = kBufLen; + + Request pending_req; + pending_req.opcode = Request::WRITE; + pending_req.source = pending_buf.data(); + pending_req.target_id = LOCAL_SEGMENT_ID; + pending_req.target_offset = reinterpret_cast(pending_buf.data()); + pending_req.length = kBufLen; + + ASSERT_TRUE( + engine.submitTransfer(batch_id, {failing_req, pending_req}).ok()); + + TransferStatus overall_status{}; + ASSERT_TRUE(engine.progressBatch(batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::PENDING); + EXPECT_EQ(fake_tcp->submit_calls.load(), 0); + + ASSERT_TRUE(engine.progressBatch(batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::FAILED); + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(failing_buf.data(), kBufLen).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(pending_buf.data(), kBufLen).ok()); +} + +TEST(EngineFailoverE2E, + WaitTransferCompletionUsesProgressBatchWhenPollDisabled) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + CorruptedRdmaBatch batch; + submitCorruptedRdmaBatch(engine, batch, 0xA9); + + EXPECT_TRUE(engine.waitTransferCompletion(batch.batch_id).ok()); + EXPECT_EQ(batch.fake_tcp->submit_calls.load(), 1); + + EXPECT_TRUE( + engine.unregisterLocalMemory(batch.buf.data(), batch.buf.size()).ok()); +} + +TEST(EngineFailoverE2E, TransferSyncUsesProgressBatchWhenPollDisabled) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + + FaultPolicy rdma_policy; + rdma_policy.status_corrupt_rate = 1.0; + auto proxied_rdma = + std::make_shared(fake_rdma, rdma_policy); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg_name, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0xB0); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + + EXPECT_TRUE(engine.transferSync({req}).ok()); + EXPECT_EQ(fake_rdma->submit_calls.load(), 1); + EXPECT_EQ(fake_tcp->submit_calls.load(), 1); + + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +TEST(EngineFailoverE2E, ProgressBatchAdvancesExactlyOneStepPerCall) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + // Reach COMPLETED only on the third poll; earlier polls stay PENDING. + auto fake_rdma = std::make_shared( + RDMA, FakeTransport::StatusFactory{}, + [](const Request& req, int poll_count) { + if (poll_count < 3) { + return TransferStatus{TransferStatusEnum::PENDING, 0}; + } + return TransferStatus{TransferStatusEnum::COMPLETED, req.length}; + }); + auto fake_tcp = std::make_shared(TCP); + + std::string seg_name = engine.getSegmentName(); + ASSERT_TRUE(fake_rdma->install(seg_name, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg_name, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, fake_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0xC0); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(1); + ASSERT_NE(batch_id, (BatchID)0); + + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + // Each progressBatch call must perform exactly one poll on the underlying + // transport — no internal loop until completion. + TransferStatus overall_status{}; + ASSERT_TRUE(engine.progressBatch(batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::PENDING); + EXPECT_EQ(fake_rdma->status_calls.load(), 1); + + ASSERT_TRUE(engine.progressBatch(batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::PENDING); + EXPECT_EQ(fake_rdma->status_calls.load(), 2); + + ASSERT_TRUE(engine.progressBatch(batch_id, overall_status).ok()); + EXPECT_EQ(overall_status.s, TransferStatusEnum::COMPLETED); + EXPECT_EQ(fake_rdma->status_calls.load(), 3); + EXPECT_EQ(fake_tcp->submit_calls.load(), 0); + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + // --------------------------------------------------------------------------- // P1b: Both transports keep failing at status stage -> failover limit reached. // --------------------------------------------------------------------------- diff --git a/mooncake-transfer-engine/tent/tests/progress_worker_test.cpp b/mooncake-transfer-engine/tent/tests/progress_worker_test.cpp new file mode 100644 index 00000000..b69d8c7a --- /dev/null +++ b/mooncake-transfer-engine/tent/tests/progress_worker_test.cpp @@ -0,0 +1,487 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +// Tests the ProgressWorker skeleton (issue #2116, follow-up to PR #2160). +// Goals: +// * default-off behavior is byte-identical to the pre-worker world; +// * with the worker enabled and enable_auto_failover_on_poll=false, a +// caller that only submits + observes status (never calls +// progressBatch / waitTransferCompletion) still sees its batch +// progress through failover; +// * one notify advances the engine by exactly one progress step; +// * freeBatch racing the worker is safe (no UAF, no crash); +// * worker shuts down cleanly on engine destruction. + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "tent/common/config.h" +#include "tent/common/types.h" +#include "tent/runtime/segment.h" +#include "tent/runtime/transfer_engine_impl.h" +#include "tent/runtime/transport.h" +#include "tent/transport/fault_proxy/fault_proxy_transport.h" + +namespace mooncake { +namespace tent { +namespace { + +// --------------------------------------------------------------------------- +// FakeTransport — same minimal shape used by engine_failover_e2e_test.cpp. +// Kept local to avoid cross-test linkage; sources of truth diverging is OK +// because we only exercise the "completes / status-can-be-overridden" surface. +// --------------------------------------------------------------------------- + +class FakeSubBatch : public Transport::SubBatch { + public: + size_t size() const override { return task_count; } + size_t task_count = 0; + std::vector requests; + std::vector statuses; + std::vector poll_counts; +}; + +class FakeTransport : public Transport { + public: + using StatusFactory = std::function; + using PollStatusFactory = + std::function; + + explicit FakeTransport(TransportType self_type, + StatusFactory status_factory = {}, + PollStatusFactory poll_status_factory = {}) + : self_type_(self_type), + status_factory_(std::move(status_factory)), + poll_status_factory_(std::move(poll_status_factory)) { + caps.dram_to_dram = true; + } + + std::atomic install_calls{0}; + std::atomic submit_calls{0}; + std::atomic status_calls{0}; + std::atomic add_mem_calls{0}; + + Status install(std::string& /*local_segment_name*/, + std::shared_ptr /*metadata*/, + std::shared_ptr /*local_topology*/, + std::shared_ptr /*conf*/ = nullptr) override { + ++install_calls; + return Status::OK(); + } + + Status allocateSubBatch(SubBatchRef& batch, size_t /*max_size*/) override { + batch = new FakeSubBatch(); + return Status::OK(); + } + + Status freeSubBatch(SubBatchRef& batch) override { + delete batch; + batch = nullptr; + return Status::OK(); + } + + Status submitTransferTasks( + SubBatchRef batch, const std::vector& request_list) override { + ++submit_calls; + auto* fb = static_cast(batch); + for (const auto& req : request_list) { + if (status_factory_) { + fb->statuses.push_back(status_factory_(req)); + } else { + fb->statuses.push_back( + {TransferStatusEnum::COMPLETED, req.length}); + } + fb->requests.push_back(req); + fb->poll_counts.push_back(0); + fb->task_count++; + } + return Status::OK(); + } + + Status getTransferStatus(SubBatchRef batch, int task_id, + TransferStatus& status) override { + ++status_calls; + auto* fb = static_cast(batch); + if (task_id < 0 || task_id >= (int)fb->statuses.size()) { + return Status::InvalidArgument("bad task_id" LOC_MARK); + } + ++fb->poll_counts[task_id]; + if (poll_status_factory_) { + status = poll_status_factory_(fb->requests[task_id], + fb->poll_counts[task_id]); + } else { + status = fb->statuses[task_id]; + } + return Status::OK(); + } + + Status addMemoryBuffer(BufferDesc& desc, + const MemoryOptions& /*options*/) override { + ++add_mem_calls; + desc.transports.push_back(self_type_); + return Status::OK(); + } + + Status addMemoryBuffer(std::vector& desc_list, + const MemoryOptions& options) override { + for (auto& d : desc_list) { + auto s = addMemoryBuffer(d, options); + if (!s.ok()) return s; + } + return Status::OK(); + } + + Status removeMemoryBuffer(BufferDesc& /*desc*/) override { + return Status::OK(); + } + + Status allocateLocalMemory(void** addr, size_t size, + MemoryOptions& /*options*/) override { + *addr = std::malloc(size); + if (!*addr) return Status::InternalError("malloc failed" LOC_MARK); + return Status::OK(); + } + + Status freeLocalMemory(void* addr, size_t /*size*/) override { + std::free(addr); + return Status::OK(); + } + + bool warmupMemory(void* /*addr*/, size_t /*length*/) override { + return false; + } + + const char* getName() const override { + return self_type_ == RDMA ? "" : ""; + } + + private: + TransportType self_type_; + StatusFactory status_factory_; + PollStatusFactory poll_status_factory_; +}; + +std::shared_ptr makeMinimalP2PConfig() { + auto cfg = std::make_shared(); + cfg->set("metadata_type", "p2p"); + cfg->set("metadata_servers", ""); + cfg->set("rpc_server_hostname", "127.0.0.1"); + cfg->set("rpc_server_port", "0"); + cfg->set("log_level", "warning"); + cfg->set("merge_requests", false); + + cfg->set("transports/tcp/enable", false); + cfg->set("transports/shm/enable", false); + cfg->set("transports/rdma/enable", false); + cfg->set("transports/io_uring/enable", false); + cfg->set("transports/nvlink/enable", false); + cfg->set("transports/mnnvl/enable", false); + cfg->set("transports/gds/enable", false); + cfg->set("transports/ascend_direct/enable", false); + + cfg->set("max_failover_attempts", 3); + return cfg; +} + +// --------------------------------------------------------------------------- +// 1. Default config: worker is not constructed, notifyBatchMaybeReady is a +// no-op, and behavior matches PR #2160 exactly. +// --------------------------------------------------------------------------- + +TEST(ProgressWorker, DisabledByDefaultLeavesBehaviorUnchanged) { + auto cfg = makeMinimalP2PConfig(); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + std::string seg = engine.getSegmentName(); + ASSERT_TRUE(fake_rdma->install(seg, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, fake_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0x10); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(1); + ASSERT_NE(batch_id, (BatchID)0); + + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + // No-op when the worker isn't constructed. + engine.notifyBatchMaybeReady(batch_id); + engine.notifyBatchMaybeReady((BatchID)0); + + TransferStatus status{}; + ASSERT_TRUE(engine.getTransferStatus(batch_id, status).ok()); + EXPECT_EQ(status.s, TransferStatusEnum::COMPLETED); + EXPECT_EQ(fake_tcp->submit_calls.load(), 0); + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +// --------------------------------------------------------------------------- +// 2. Worker drives failover when the caller does not poll with +// allow_failover. This is the integration shape mooncake-pg needs. +// --------------------------------------------------------------------------- + +TEST(ProgressWorker, ProgressesWithoutPollAutoFailover) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + cfg->set("enable_progress_worker", true); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + + FaultPolicy rdma_policy; + rdma_policy.status_corrupt_rate = 1.0; + auto proxied_rdma = + std::make_shared(fake_rdma, rdma_policy); + + std::string seg = engine.getSegmentName(); + ASSERT_TRUE(proxied_rdma->install(seg, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, proxied_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0xC1); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(1); + ASSERT_NE(batch_id, (BatchID)0); + + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + // Drive the worker until terminal. We deliberately never call + // progressBatch / waitTransferCompletion here — only + // notifyBatchMaybeReady + observation-only getTransferStatus + // (which, with auto-failover-on-poll disabled, will not advance failover + // by itself). + TransferStatus status{}; + const auto deadline = + std::chrono::steady_clock::now() + std::chrono::milliseconds(2000); + while (std::chrono::steady_clock::now() < deadline) { + engine.notifyBatchMaybeReady(batch_id); + std::this_thread::sleep_for(std::chrono::milliseconds(5)); + if (fake_tcp->submit_calls.load() == 0) continue; + status = {}; + ASSERT_TRUE(engine.getTransferStatus(batch_id, status).ok()); + if (status.s == TransferStatusEnum::COMPLETED) break; + } + EXPECT_EQ(status.s, TransferStatusEnum::COMPLETED) + << "progress worker must drive failover when caller never calls " + "progressBatch"; + EXPECT_EQ(fake_rdma->submit_calls.load(), 1); + EXPECT_GE(fake_tcp->submit_calls.load(), 1); + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +// --------------------------------------------------------------------------- +// 3. One notify == one progress step. The worker must not loop internally +// until completion; the next step requires another notify. +// --------------------------------------------------------------------------- + +TEST(ProgressWorker, SingleNotifyAdvancesOneStep) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + cfg->set("enable_progress_worker", true); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared( + RDMA, FakeTransport::StatusFactory{}, + [](const Request& req, int poll_count) { + if (poll_count < 3) { + return TransferStatus{TransferStatusEnum::PENDING, 0}; + } + return TransferStatus{TransferStatusEnum::COMPLETED, req.length}; + }); + auto fake_tcp = std::make_shared(TCP); + + std::string seg = engine.getSegmentName(); + ASSERT_TRUE(fake_rdma->install(seg, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, fake_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0xC2); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + BatchID batch_id = engine.allocateBatch(1); + ASSERT_NE(batch_id, (BatchID)0); + + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + // Initial state: nothing polled yet. + EXPECT_EQ(fake_rdma->status_calls.load(), 0); + + // Drive exactly one progress step via the worker. We can't observe the + // step instantly, but we can wait until status_calls increments by 1 + // and then assert it does NOT keep climbing to 3 on its own. + engine.notifyBatchMaybeReady(batch_id); + const auto step_deadline = + std::chrono::steady_clock::now() + std::chrono::milliseconds(500); + while (std::chrono::steady_clock::now() < step_deadline && + fake_rdma->status_calls.load() == 0) { + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + } + ASSERT_EQ(fake_rdma->status_calls.load(), 1) + << "worker should issue exactly one poll for a single notify"; + + // Give the worker a generous window to misbehave. status_calls must + // stay at 1 because we did not notify again and the engine did not + // reach a terminal state. + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + EXPECT_EQ(fake_rdma->status_calls.load(), 1) + << "worker must not loop internally until completion"; + + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +// --------------------------------------------------------------------------- +// 4. freeBatch races with worker notifications. With ASAN/UBSAN this +// catches missing alive_batches_ / progress_mutex_ coverage. +// --------------------------------------------------------------------------- + +TEST(ProgressWorker, FreeBatchRacesWithWorker) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + cfg->set("enable_progress_worker", true); + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + std::string seg = engine.getSegmentName(); + ASSERT_TRUE(fake_rdma->install(seg, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, fake_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + constexpr size_t kBufLen = 4096; + std::vector buf(kBufLen, 0xC3); + ASSERT_TRUE(engine.registerLocalMemory(buf.data(), kBufLen).ok()); + + // Concurrently spam stale notifications from a second thread while the + // main thread submits, frees, and re-allocates batches. + std::atomic stop{false}; + std::atomic latest{0}; + std::thread spammer([&] { + while (!stop.load(std::memory_order_acquire)) { + BatchID bid = latest.load(std::memory_order_acquire); + engine.notifyBatchMaybeReady(bid); + std::this_thread::sleep_for(std::chrono::microseconds(50)); + } + }); + + constexpr int kRounds = 100; + for (int i = 0; i < kRounds; ++i) { + BatchID batch_id = engine.allocateBatch(1); + ASSERT_NE(batch_id, (BatchID)0); + + Request req; + req.opcode = Request::WRITE; + req.source = buf.data(); + req.target_id = LOCAL_SEGMENT_ID; + req.target_offset = reinterpret_cast(buf.data()); + req.length = kBufLen; + ASSERT_TRUE(engine.submitTransfer(batch_id, {req}).ok()); + + latest.store(batch_id, std::memory_order_release); + engine.notifyBatchMaybeReady(batch_id); + + // Free immediately; the worker may pick the notification up after + // free. The progress_mutex_ + alive_batches_ guard must keep this + // safe. + EXPECT_TRUE(engine.freeBatch(batch_id).ok()); + } + + stop.store(true, std::memory_order_release); + spammer.join(); + + EXPECT_TRUE(engine.unregisterLocalMemory(buf.data(), kBufLen).ok()); +} + +// --------------------------------------------------------------------------- +// 5. Engine teardown joins the worker cleanly even with pending notifies. +// --------------------------------------------------------------------------- + +TEST(ProgressWorker, EngineDestructorJoinsWorker) { + auto cfg = makeMinimalP2PConfig(); + cfg->set("enable_auto_failover_on_poll", false); + cfg->set("enable_progress_worker", true); + { + TransferEngineImpl engine(cfg); + ASSERT_TRUE(engine.available()); + + auto fake_rdma = std::make_shared(RDMA); + auto fake_tcp = std::make_shared(TCP); + std::string seg = engine.getSegmentName(); + ASSERT_TRUE(fake_rdma->install(seg, nullptr, nullptr).ok()); + ASSERT_TRUE(fake_tcp->install(seg, nullptr, nullptr).ok()); + engine.swapTransportForTest(RDMA, fake_rdma); + engine.swapTransportForTest(TCP, fake_tcp); + + // Push some notifies for non-existent batches; worker must reject + // them via alive_batches_ check and stay alive. + for (int i = 0; i < 8; ++i) { + engine.notifyBatchMaybeReady((BatchID)(uintptr_t)0xdeadbeef); + } + } + // If teardown hangs or crashes here, gtest fails this test on timeout + // / signal — no further assert needed. + SUCCEED(); +} + +} // namespace +} // namespace tent +} // namespace mooncake -- 2.34.1 From 4794ed5acaea695e68cad06f2bc02991ea3b4a25 Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Wed, 27 May 2026 12:05:35 +0800 Subject: [PATCH 135/382] [Store] remove invalid kMaxSliceSize assertion in AllocateBatch (#2165) * fix(store): remove invalid kMaxSliceSize assertion in AllocateBatch The assert(sizes[i] <= kMaxSliceSize) in FileStorage::AllocateBatch was an overly restrictive check copied from the CacheLib path. It caused crashes when reading large offloaded objects (>4MB) from SSD, since the local buffer allocator (AlignedClientBufferAllocator) can handle arbitrary sizes. Put path already supports large objects by chunking, but get path should not enforce the same limit on local temp buffers. Fixes #2156 * Update mooncake-store/src/file_storage.cpp Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- mooncake-store/src/file_storage.cpp | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index 46092e9b..a7623f03 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -822,6 +822,11 @@ tl::expected FileStorage::RegisterLocalMemory() { tl::expected, ErrorCode> FileStorage::AllocateBatch(const std::vector& keys, const std::vector& sizes) { + if (keys.size() != sizes.size()) { + LOG(ERROR) << "Mismatched keys and sizes count: keys=" << keys.size() + << ", sizes=" << sizes.size(); + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } auto result = std::make_shared(); result->batch_id = next_batch_id_.fetch_add(1, std::memory_order_relaxed); std::chrono::steady_clock::time_point now = @@ -833,7 +838,11 @@ FileStorage::AllocateBatch(const std::vector& keys, u_int64_t total_size = 0; bool gc_triggered = false; for (size_t i = 0; i < keys.size(); ++i) { - assert(sizes[i] <= kMaxSliceSize); + if (sizes[i] < 0 || sizes[i] > config_.local_buffer_size) { + LOG(ERROR) << "Invalid size for key " << keys[i] << ": " << sizes[i] + << " (limit: " << config_.local_buffer_size << ")"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } // Allocate oversized buffer for O_DIRECT alignment: // +4096 for aligning the ptr to 4096 boundary -- 2.34.1 From 52e62c349e2bbdf8a82edba5c95c7ef5edbc9d9f Mon Sep 17 00:00:00 2001 From: Aoi Date: Wed, 27 May 2026 13:15:24 +0800 Subject: [PATCH 136/382] [CI] Add docs-check job to validate Sphinx build with -W (#2229) --- .github/workflows/ci.yml | 43 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b3e9981a..27555d70 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -765,6 +765,48 @@ jobs: shell: bash + docs-check: + name: Check Sphinx docs build + if: >- + github.event_name == 'push' || + github.event_name == 'workflow_dispatch' || + github.event.action == 'opened' || + contains(github.event.pull_request.labels.*.name, 'run-ci') + runs-on: ubuntu-22.04 + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 2 + persist-credentials: false + + - uses: dorny/paths-filter@v3 + id: filter + with: + filters: | + docs: + - 'docs/**' + - 'requirements_docs.txt' + + - name: Set up Python + if: steps.filter.outputs.docs == 'true' + uses: actions/setup-python@v5 + with: + python-version: '3.12' + + - name: Install documentation dependencies + if: steps.filter.outputs.docs == 'true' + run: | + python -m pip install --upgrade pip + pip install -r requirements_docs.txt + + - name: Build docs with strict mode + if: steps.filter.outputs.docs == 'true' + run: | + cd docs + make html SPHINXOPTS=-W + shell: bash + + check-paths: if: >- github.event_name == 'push' || @@ -828,6 +870,7 @@ jobs: needs: - spell-check - clang-format + - docs-check - build - build-musa - build-flags -- 2.34.1 From 06ae8dd018ac4543677b0d782628ea1d822bfabd Mon Sep 17 00:00:00 2001 From: CAICAII <3360776475@qq.com> Date: Wed, 27 May 2026 13:15:57 +0800 Subject: [PATCH 137/382] [Store] Add opt-in grouped object routing semantics (#2180) --- .../python-api-reference/mooncake-store.md | 30 +- mooncake-integration/store/store_py.cpp | 81 +- .../store/store_py_internal.h | 2 +- .../store/store_py_parallel_write.h | 60 +- mooncake-store/include/master_service.h | 70 +- mooncake-store/include/replica.h | 24 +- mooncake-store/src/master_service.cpp | 821 +++++++++++++----- mooncake-store/src/rpc_service.cpp | 21 +- .../tests/client_integration_test.cpp | 77 ++ .../snapshot/snapshot_child_process_test.cpp | 134 +++ mooncake-store/tests/master_service_test.cpp | 498 +++++++++++ mooncake-wheel/tests/test_import_structure.py | 8 +- 12 files changed, 1558 insertions(+), 268 deletions(-) diff --git a/docs/source/python-api-reference/mooncake-store.md b/docs/source/python-api-reference/mooncake-store.md index 5939b821..e2f89a9b 100644 --- a/docs/source/python-api-reference/mooncake-store.md +++ b/docs/source/python-api-reference/mooncake-store.md @@ -586,8 +586,36 @@ config.preferred_segment = self.get_hostname() ```python config = ReplicateConfig() -config.prefer_alloc_in_same_node = "True +config.prefer_alloc_in_same_node = "True" ``` + +#### group_ids +**Type:** `List[str] | None` +**Default:** `None` +**Description:** Optionally assigns object metadata to routing groups during writes. When this field is unset, Mooncake Store preserves the default ungrouped behavior. When it is set, each group ID maps to the object at the same position in the write request. Empty string (`""`) explicitly stores that object as ungrouped. + +For batch write APIs, the number of group IDs must match the number of keys: + +```python +config = ReplicateConfig() +config.group_ids = ["session-a", "", "session-b"] + +store.put_batch( + ["key-a", "key-b", "key-c"], + [b"value-a", b"value-b", b"value-c"], + config, +) +``` + +For a single-object write, provide one group ID: + +```python +config = ReplicateConfig() +config.group_ids = ["session-a"] + +store.put("key-a", b"value-a", config) +``` + --- ## Unified Parallel Tensor IO API diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index c8247179..a957c7b7 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -677,6 +677,56 @@ class MooncakeStorePyWrapper { ReplicateConfig{}); // Default config } + ReplicateConfig MakeIndexedConfig( + const ReplicateConfig &config, + const std::vector &original_indices) const { + if (!config.group_ids.has_value()) { + return config; + } + + ReplicateConfig indexed_config = config; + std::vector group_ids; + group_ids.reserve(original_indices.size()); + for (size_t index : original_indices) { + group_ids.push_back(config.group_ids->at(index)); + } + indexed_config.group_ids = std::move(group_ids); + return indexed_config; + } + + ReplicateConfig MakeRepeatedIndexedConfig( + const ReplicateConfig &config, + const std::vector &original_indices, int repeat_count) const { + if (!config.group_ids.has_value()) { + return config; + } + + ReplicateConfig indexed_config = config; + std::vector group_ids; + group_ids.reserve(original_indices.size() * + static_cast(repeat_count)); + for (size_t index : original_indices) { + for (int i = 0; i < repeat_count; ++i) { + group_ids.push_back(config.group_ids->at(index)); + } + } + indexed_config.group_ids = std::move(group_ids); + return indexed_config; + } + + std::vector ValidateGroupIdsForBatchConfig( + const ReplicateConfig &config, size_t key_count, + const char *operation_name) const { + if (config.group_ids.has_value() && + config.group_ids->size() != key_count) { + LOG(ERROR) << operation_name + << ": group_ids size must match keys size"; + return std::vector(key_count, + to_py_ret(ErrorCode::INVALID_PARAMS)); + } + return {}; + } + int put_tensor_with_tp_impl( const std::string &key, pybind11::object tensor, const ReplicateConfig &config = ReplicateConfig{}, int tp_rank = 0, @@ -725,11 +775,12 @@ class MooncakeStorePyWrapper { const ReplicateConfig &config = ReplicateConfig{}) { return batch_write_tensor_impl( keys, infos, config, "put", - [this, &config](const std::vector &write_keys, - const std::vector &buffer_ptrs, - const std::vector &buffer_sizes) { + [this](const std::vector &write_keys, + const std::vector &buffer_ptrs, + const std::vector &buffer_sizes, + const ReplicateConfig &write_config) { return store_->batch_put_from(write_keys, buffer_ptrs, - buffer_sizes, config); + buffer_sizes, write_config); }); } @@ -769,6 +820,11 @@ class MooncakeStorePyWrapper { std::vector processed_indices; std::vector final_results(base_keys.size(), to_py_ret(ErrorCode::INVALID_PARAMS)); + auto group_ids_error = ValidateGroupIdsForBatchConfig( + config, base_keys.size(), "batch_put_tensor_with_tp"); + if (!group_ids_error.empty()) { + return group_ids_error; + } try { // Chunking phase (GIL Held) for (size_t i = 0; i < base_keys.size(); ++i) { @@ -799,8 +855,10 @@ class MooncakeStorePyWrapper { if (all_chunk_keys.empty()) return final_results; // Reuse the standard batch_put implementation - std::vector chunk_results = - batch_put_tensor_impl(all_chunk_keys, all_chunks_list, config); + ReplicateConfig chunk_config = + MakeRepeatedIndexedConfig(config, processed_indices, tp_size); + std::vector chunk_results = batch_put_tensor_impl( + all_chunk_keys, all_chunks_list, chunk_config); // Aggregate results for (size_t i = 0; i < processed_indices.size(); ++i) { @@ -1234,6 +1292,12 @@ class MooncakeStorePyWrapper { const std::vector &keys, const pybind11::list &tensors_list, const ReplicateConfig &config = ReplicateConfig{}) { + auto group_ids_error = ValidateGroupIdsForBatchConfig( + config, keys.size(), "batch_upsert_tensor"); + if (!group_ids_error.empty()) { + return group_ids_error; + } + std::vector infos(keys.size()); std::vector results(keys.size(), 0); @@ -1287,8 +1351,10 @@ class MooncakeStorePyWrapper { } if (!valid_keys.empty()) { + ReplicateConfig write_config = + MakeIndexedConfig(config, original_indices); std::vector op_results = store_->batch_upsert_from( - valid_keys, buffer_ptrs, buffer_sizes, config); + valid_keys, buffer_ptrs, buffer_sizes, write_config); for (size_t i = 0; i < op_results.size(); ++i) { results[original_indices[i]] = op_results[i]; } @@ -1603,6 +1669,7 @@ PYBIND11_MODULE(store, m) { .def_readwrite("prefer_alloc_in_same_node", &ReplicateConfig::prefer_alloc_in_same_node) .def_readwrite("data_type", &ReplicateConfig::data_type) + .def_readwrite("group_ids", &ReplicateConfig::group_ids) .def("__str__", [](const ReplicateConfig &config) { std::ostringstream oss; oss << config; diff --git a/mooncake-integration/store/store_py_internal.h b/mooncake-integration/store/store_py_internal.h index a9a03c5c..737d5e50 100644 --- a/mooncake-integration/store/store_py_internal.h +++ b/mooncake-integration/store/store_py_internal.h @@ -848,7 +848,7 @@ bool is_default_replicate_config(const ReplicateConfig &config) { return config.replica_num == 1 && !config.with_soft_pin && !config.with_hard_pin && config.preferred_segments.empty() && config.preferred_segment.empty() && - !config.prefer_alloc_in_same_node; + !config.prefer_alloc_in_same_node && !config.group_ids.has_value(); } std::optional parse_parallel_axis_spec( diff --git a/mooncake-integration/store/store_py_parallel_write.h b/mooncake-integration/store/store_py_parallel_write.h index 63caf8b0..c383b2dd 100644 --- a/mooncake-integration/store/store_py_parallel_write.h +++ b/mooncake-integration/store/store_py_parallel_write.h @@ -20,6 +20,12 @@ std::vector batch_write_tensor_impl(const std::vector &keys, const ReplicateConfig &config, const char *operation_name, BatchWriteFromFn &&batch_write_from) { + auto group_ids_error = + ValidateGroupIdsForBatchConfig(config, keys.size(), operation_name); + if (!group_ids_error.empty()) { + return group_ids_error; + } + std::vector results(keys.size(), 0); { @@ -65,8 +71,10 @@ std::vector batch_write_tensor_impl(const std::vector &keys, } if (!valid_keys.empty()) { - std::vector op_results = - batch_write_from(valid_keys, buffer_ptrs, buffer_sizes); + ReplicateConfig write_config = + MakeIndexedConfig(config, original_indices); + std::vector op_results = batch_write_from( + valid_keys, buffer_ptrs, buffer_sizes, write_config); for (size_t i = 0; i < op_results.size(); ++i) { results[original_indices[i]] = op_results[i]; } @@ -905,6 +913,12 @@ std::vector batch_put_tensor_with_parallelism( const py::object ¶llelisms = py::none(), const ReplicateConfig &config = ReplicateConfig{}, const py::object &writer_partitions = py::none()) { + auto group_ids_error = ValidateGroupIdsForBatchConfig( + config, keys.size(), "batch_put_tensor_with_parallelism"); + if (!group_ids_error.empty()) { + return group_ids_error; + } + return execute_batch_parallelism_write_requests( keys, tensors_list.size(), parallelisms, writer_partitions, "batch_put_tensor_with_parallelism", @@ -921,14 +935,16 @@ std::vector batch_put_tensor_with_parallelism( }, [this, &keys, &tensors_list, &config](size_t i, const py::handle ¶llelism) { + ReplicateConfig key_config = config.ForSingleKey(i); return put_tensor_with_parallelism( keys[i], tensors_list[i], - py::reinterpret_borrow(parallelism), config); + py::reinterpret_borrow(parallelism), key_config); }, [this, &keys, &tensors_list, &config]( size_t i, const py::handle &writer_partition) { + ReplicateConfig key_config = config.ForSingleKey(i); return put_tensor_with_parallelism( - keys[i], tensors_list[i], py::none(), config, + keys[i], tensors_list[i], py::none(), key_config, py::reinterpret_borrow(writer_partition)); }); } @@ -1029,6 +1045,12 @@ std::vector batch_put_tensor_with_parallelism_from( const py::object ¶llelisms = py::none(), const ReplicateConfig &config = ReplicateConfig{}, const py::object &writer_partitions = py::none()) { + auto group_ids_error = ValidateGroupIdsForBatchConfig( + config, keys.size(), "batch_put_tensor_with_parallelism_from"); + if (!group_ids_error.empty()) { + return group_ids_error; + } + return execute_batch_parallelism_write_requests( keys, buffer_ptrs.size(), parallelisms, writer_partitions, "batch_put_tensor_with_parallelism_from", @@ -1070,14 +1092,16 @@ std::vector batch_put_tensor_with_parallelism_from( }, [this, &keys, &buffer_ptrs, &sizes, &config]( size_t i, const py::handle ¶llelism) { + ReplicateConfig key_config = config.ForSingleKey(i); return put_tensor_with_parallelism_from( keys[i], buffer_ptrs[i], sizes[i], - py::reinterpret_borrow(parallelism), config); + py::reinterpret_borrow(parallelism), key_config); }, [this, &keys, &buffer_ptrs, &sizes, &config]( size_t i, const py::handle &writer_partition) { + ReplicateConfig key_config = config.ForSingleKey(i); return put_tensor_with_parallelism_from( - keys[i], buffer_ptrs[i], sizes[i], py::none(), config, + keys[i], buffer_ptrs[i], sizes[i], py::none(), key_config, py::reinterpret_borrow(writer_partition)); }); } @@ -1345,6 +1369,12 @@ std::vector batch_upsert_tensor_with_parallelism( const py::object ¶llelisms = py::none(), const ReplicateConfig &config = ReplicateConfig{}, const py::object &writer_partitions = py::none()) { + auto group_ids_error = ValidateGroupIdsForBatchConfig( + config, keys.size(), "batch_upsert_tensor_with_parallelism"); + if (!group_ids_error.empty()) { + return group_ids_error; + } + return execute_batch_parallelism_write_requests( keys, tensors_list.size(), parallelisms, writer_partitions, "batch_upsert_tensor_with_parallelism", @@ -1361,14 +1391,16 @@ std::vector batch_upsert_tensor_with_parallelism( }, [this, &keys, &tensors_list, &config](size_t i, const py::handle ¶llelism) { + ReplicateConfig key_config = config.ForSingleKey(i); return upsert_tensor_with_parallelism( keys[i], tensors_list[i], - py::reinterpret_borrow(parallelism), config); + py::reinterpret_borrow(parallelism), key_config); }, [this, &keys, &tensors_list, &config]( size_t i, const py::handle &writer_partition) { + ReplicateConfig key_config = config.ForSingleKey(i); return upsert_tensor_with_parallelism( - keys[i], tensors_list[i], py::none(), config, + keys[i], tensors_list[i], py::none(), key_config, py::reinterpret_borrow(writer_partition)); }); } @@ -1379,6 +1411,12 @@ std::vector batch_upsert_tensor_with_parallelism_from( const py::object ¶llelisms = py::none(), const ReplicateConfig &config = ReplicateConfig{}, const py::object &writer_partitions = py::none()) { + auto group_ids_error = ValidateGroupIdsForBatchConfig( + config, keys.size(), "batch_upsert_tensor_with_parallelism_from"); + if (!group_ids_error.empty()) { + return group_ids_error; + } + return execute_batch_parallelism_write_requests( keys, buffer_ptrs.size(), parallelisms, writer_partitions, "batch_upsert_tensor_with_parallelism_from", @@ -1430,14 +1468,16 @@ std::vector batch_upsert_tensor_with_parallelism_from( }, [this, &keys, &buffer_ptrs, &sizes, &config]( size_t i, const py::handle ¶llelism) { + ReplicateConfig key_config = config.ForSingleKey(i); return upsert_tensor_with_parallelism_from( keys[i], buffer_ptrs[i], sizes[i], - py::reinterpret_borrow(parallelism), config); + py::reinterpret_borrow(parallelism), key_config); }, [this, &keys, &buffer_ptrs, &sizes, &config]( size_t i, const py::handle &writer_partition) { + ReplicateConfig key_config = config.ForSingleKey(i); return upsert_tensor_with_parallelism_from( - keys[i], buffer_ptrs[i], sizes[i], py::none(), config, + keys[i], buffer_ptrs[i], sizes[i], py::none(), key_config, py::reinterpret_borrow(writer_partition)); }); } diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index 68b841f3..893a37e5 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -741,11 +742,13 @@ class MasterService { const std::chrono::system_clock::time_point put_start_time_, size_t value_length, std::vector&& reps, bool enable_soft_pin, bool enable_hard_pin = false, - ObjectDataType data_type_ = ObjectDataType::UNKNOWN) + ObjectDataType data_type_ = ObjectDataType::UNKNOWN, + std::string group_id_ = "") : client_id(client_id_), put_start_time(put_start_time_), size(value_length), data_type(data_type_), + group_id(std::move(group_id_)), lease_timeout(), soft_pin_timeout(std::nullopt), hard_pinned(enable_hard_pin), @@ -769,6 +772,7 @@ class MasterService { std::chrono::system_clock::time_point put_start_time; const size_t size; const ObjectDataType data_type{ObjectDataType::UNKNOWN}; + const std::string group_id; mutable SpinLock lock; // Default constructor, creates a time_point representing @@ -948,6 +952,18 @@ class MasterService { } } + bool NeedsLeaseRefresh(const uint64_t ttl, + const uint64_t soft_ttl) const { + SpinLocker locker(&lock); + const auto now = std::chrono::system_clock::now(); + if (lease_timeout <= now + std::chrono::milliseconds(ttl / 2)) { + return true; + } + return soft_pin_timeout && + *soft_pin_timeout <= + now + std::chrono::milliseconds(soft_ttl / 2); + } + // Check if the lease has expired bool IsLeaseExpired() const { SpinLocker locker(&lock); @@ -975,6 +991,8 @@ class MasterService { bool IsHardPinned() const { return hard_pinned; } + bool IsGrouped() const { return !group_id.empty(); } + // Check if the metadata is valid // Valid means it has at least one valid replica and size is greater // than 0 @@ -1063,9 +1081,17 @@ class MasterService { GUARDED_BY(mutex); std::unordered_map promotion_tasks GUARDED_BY(mutex); + std::unordered_map> + group_members GUARDED_BY(mutex); }; std::array metadata_shards_; + std::unordered_map object_group_ids_ + GUARDED_BY(group_routing_mutex_); + mutable std::unordered_set groups_needing_lease_refresh_ + GUARDED_BY(group_routing_mutex_); + mutable std::shared_mutex group_routing_mutex_; + // For accessing a metadata shard with read-write permission class MetadataShardAccessorRW { public: @@ -1078,6 +1104,10 @@ class MasterService { const MetadataShard* operator->() const { return &shard_; } + MetadataShard& get() { return shard_; } + + const MetadataShard& get() const { return shard_; } + private: MetadataShard& shard_; SharedMutexLocker lock_; @@ -1093,6 +1123,8 @@ class MasterService { const MetadataShard* operator->() const { return &shard_; } + const MetadataShard& get() const { return shard_; } + private: const MetadataShard& shard_; SharedMutexLocker lock_; @@ -1103,6 +1135,19 @@ class MasterService { return std::hash{}(key) % kNumShards; } + size_t getMetadataShardIndex(const std::string& key) const; + void RegisterGroupMember(MetadataShard& shard, const std::string& key, + const std::string& group_id); + void UnregisterGroupMember(MetadataShard& shard, const std::string& key, + const std::string& group_id); + std::unordered_map::iterator + EraseMetadataEntry( + MetadataShard& shard, + std::unordered_map::iterator it); + void RebuildGroupRoutingIndex(); + void GrantLeaseForGroup(const MetadataShard& shard, const std::string& key, + const ObjectMetadata& metadata) const; + // Helper to clean up stale handles pointing to unmounted segments // or local_disk replicas whose owner client is no longer alive. bool CleanupStaleHandles( @@ -1114,7 +1159,7 @@ class MasterService { auto AllocateAndInsertMetadata( MetadataShardAccessorRW& shard, const UUID& client_id, const std::string& key, uint64_t value_length, - const ReplicateConfig& config, + const ReplicateConfig& config, const std::string& group_id, const std::chrono::system_clock::time_point& now) -> tl::expected, ErrorCode>; @@ -1193,19 +1238,6 @@ class MasterService { */ void TryPushPromotionQueue(const std::string& key); - // Erase any in-flight PromotionTask for `key` and decrement the - // cluster-wide in-flight counter. Safe no-op if no task exists. - // Call from any path that erases an ObjectMetadata entry, so the - // task doesn't pin a promotion_in_flight_ slot for the full - // put_start_release_timeout_sec_ until the reaper sweeps. - void ErasePromotionTaskIfPresent(MetadataShardAccessorRW& shard, - const std::string& key) - NO_THREAD_SAFETY_ANALYSIS { - if (shard->promotion_tasks.erase(key) > 0) { - promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); - } - } - // Lease related members const uint64_t default_kv_lease_ttl_; // in milliseconds const uint64_t default_kv_soft_pin_ttl_; // in milliseconds @@ -1245,7 +1277,7 @@ class MasterService { MetadataAccessorRW(MasterService* service, const std::string& key) : service_(service), key_(key), - shard_idx_(service_->getShardIndex(key)), + shard_idx_(service_->getMetadataShardIndex(key)), shard_guard_(service_, shard_idx_), it_(shard_guard_->metadata.find(key)), processing_it_(shard_guard_->processing_keys.find(key)), @@ -1268,7 +1300,6 @@ class MasterService { if (processing_it_ != shard_guard_->processing_keys.end()) { this->EraseFromProcessing(); } - service_->ErasePromotionTaskIfPresent(shard_guard_, key_); } } } @@ -1300,8 +1331,7 @@ class MasterService { // Delete current metadata (for PutRevoke or Remove operations) void Erase() NO_THREAD_SAFETY_ANALYSIS { - shard_guard_->metadata.erase(it_); - it_ = shard_guard_->metadata.end(); + it_ = service_->EraseMetadataEntry(shard_guard_.get(), it_); } void EraseFromProcessing() NO_THREAD_SAFETY_ANALYSIS { @@ -1388,7 +1418,7 @@ class MasterService { MetadataAccessorRO(const MasterService* service, const std::string& key) : service_(service), key_(key), - shard_idx_(service_->getShardIndex(key)), + shard_idx_(service_->getMetadataShardIndex(key)), shard_guard_(service_, shard_idx_), it_(shard_guard_->metadata.find(key)), processing_it_(shard_guard_->processing_keys.find(key)) {} diff --git a/mooncake-store/include/replica.h b/mooncake-store/include/replica.h index ca0e268c..677e77b3 100644 --- a/mooncake-store/include/replica.h +++ b/mooncake-store/include/replica.h @@ -91,6 +91,19 @@ struct ReplicateConfig { preferred_nof_segments{}; // Preferred NoF segments for allocation bool prefer_alloc_in_same_node{false}; ObjectDataType data_type{ObjectDataType::UNKNOWN}; + // Optional per-key routing group IDs. Empty string keeps that key + // ungrouped. Grouped keys share metadata routing, coalesced lease refresh, + // and memory eviction behavior. + std::optional> group_ids{}; + + ReplicateConfig ForSingleKey(size_t key_index) const { + ReplicateConfig key_config = *this; + if (group_ids.has_value()) { + key_config.group_ids = + std::vector{group_ids->at(key_index)}; + } + return key_config; + } friend std::ostream& operator<<(std::ostream& os, const ReplicateConfig& config) noexcept { @@ -116,7 +129,16 @@ struct ReplicateConfig { os << "]"; os << ", prefer_alloc_in_same_node: " << config.prefer_alloc_in_same_node - << ", data_type: " << config.data_type << " }"; + << ", data_type: " << config.data_type; + if (config.group_ids.has_value()) { + os << ", group_ids: ["; + for (size_t i = 0; i < config.group_ids->size(); ++i) { + os << config.group_ids->at(i); + if (i + 1 < config.group_ids->size()) os << ", "; + } + os << "]"; + } + os << " }"; return os; } }; diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 0e7305e1..bd53cf39 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -104,6 +104,20 @@ bool HasExpectedReplicaAllocation(const ReplicateConfig& config, allocated_nof_replicas == config.nof_replica_num; } +tl::expected GetGroupIdForKey( + const ReplicateConfig& config, size_t key_count, size_t key_index) { + if (!config.group_ids.has_value()) { + return ""; + } + if (config.group_ids->size() != key_count || key_index >= key_count) { + LOG(ERROR) << "group_ids.size()=" << config.group_ids->size() + << ", key_count=" << key_count + << ", error=invalid_group_ids"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + return config.group_ids->at(key_index); +} + } // namespace MasterService::MasterService() : MasterService(MasterServiceConfig()) {} @@ -575,6 +589,139 @@ MasterService::getAliveClientsSnapshot() const { return ok_client_; } +size_t MasterService::getMetadataShardIndex(const std::string& key) const { + std::shared_lock lock(group_routing_mutex_); + auto it = object_group_ids_.find(key); + if (it == object_group_ids_.end()) { + return getShardIndex(key); + } + return getShardIndex(it->second); +} + +void MasterService::RegisterGroupMember(MetadataShard& shard, + const std::string& key, + const std::string& group_id) { + if (group_id.empty()) { + return; + } + std::unique_lock lock(group_routing_mutex_); + object_group_ids_[key] = group_id; + groups_needing_lease_refresh_.insert(group_id); + shard.group_members[group_id].insert(key); +} + +void MasterService::UnregisterGroupMember(MetadataShard& shard, + const std::string& key, + const std::string& group_id) { + if (group_id.empty()) { + return; + } + bool group_empty = false; + auto group_it = shard.group_members.find(group_id); + if (group_it != shard.group_members.end()) { + group_it->second.erase(key); + if (group_it->second.empty()) { + shard.group_members.erase(group_it); + group_empty = true; + } + } + std::unique_lock lock(group_routing_mutex_); + auto route_it = object_group_ids_.find(key); + if (route_it != object_group_ids_.end() && route_it->second == group_id) { + object_group_ids_.erase(route_it); + } + if (group_empty) { + groups_needing_lease_refresh_.erase(group_id); + } +} + +std::unordered_map::iterator +MasterService::EraseMetadataEntry( + MetadataShard& shard, + std::unordered_map::iterator it) { + if (it == shard.metadata.end()) { + return it; + } + const std::string key = it->first; + const std::string group_id = it->second.group_id; + // Keep the route visible until metadata is gone. Concurrent writers that + // saw the old route will either serialize on this shard or retry after the + // route changes, instead of creating a second copy on the key shard. + auto next = shard.metadata.erase(it); + if (shard.promotion_tasks.erase(key) > 0) { + promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + } + UnregisterGroupMember(shard, key, group_id); + return next; +} + +void MasterService::RebuildGroupRoutingIndex() { + std::unordered_map rebuilt_group_ids; + std::unordered_set groups_needing_refresh; + // Snapshot restore rebuilds this derived index after all metadata shards + // have been deserialized. Take shard locks here so the helper does not + // depend on callers touching guarded shard state directly. + for (size_t shard_idx = 0; shard_idx < kNumShards; ++shard_idx) { + MetadataShardAccessorRW shard(this, shard_idx); + shard->group_members.clear(); + for (const auto& [key, metadata] : shard->metadata) { + if (!metadata.IsGrouped()) { + continue; + } + shard->group_members[metadata.group_id].insert(key); + rebuilt_group_ids[key] = metadata.group_id; + groups_needing_refresh.insert(metadata.group_id); + } + } + + { + std::unique_lock lock(group_routing_mutex_); + object_group_ids_ = std::move(rebuilt_group_ids); + groups_needing_lease_refresh_ = std::move(groups_needing_refresh); + } +} + +void MasterService::GrantLeaseForGroup(const MetadataShard& shard, + const std::string& key, + const ObjectMetadata& metadata) const { + if (!metadata.IsGrouped()) { + metadata.GrantLease(default_kv_lease_ttl_, default_kv_soft_pin_ttl_); + return; + } + + bool needs_refresh = metadata.NeedsLeaseRefresh(default_kv_lease_ttl_, + default_kv_soft_pin_ttl_); + if (!needs_refresh) { + std::shared_lock lock(group_routing_mutex_); + needs_refresh = groups_needing_lease_refresh_.find(metadata.group_id) != + groups_needing_lease_refresh_.end(); + } + if (!needs_refresh) { + return; + } + + auto group_it = shard.group_members.find(metadata.group_id); + if (group_it == shard.group_members.end()) { + metadata.GrantLease(default_kv_lease_ttl_, default_kv_soft_pin_ttl_); + return; + } + + for (const auto& member_key : group_it->second) { + auto member_it = shard.metadata.find(member_key); + if (member_it != shard.metadata.end()) { + member_it->second.GrantLease(default_kv_lease_ttl_, + default_kv_soft_pin_ttl_); + } + } + if (group_it->second.find(key) == group_it->second.end()) { + metadata.GrantLease(default_kv_lease_ttl_, default_kv_soft_pin_ttl_); + } + { + std::unique_lock lock(group_routing_mutex_); + groups_needing_lease_refresh_.erase(metadata.group_id); + } +} + void MasterService::ClearInvalidHandles() { ClearInvalidHandles(getAliveClientsSnapshot()); } @@ -592,19 +739,7 @@ void MasterService::ClearInvalidHandles( shard->processing_keys.erase(it->first); shard->replication_tasks.erase(it->first); shard->offloading_tasks.erase(it->first); - // Promotion task cleanup: if the holder client expired, the - // LOCAL_DISK source is gone and the metadata is being erased - // here. Without this erase the dangling promotion_tasks - // entry stays pinned for up to put_start_release_timeout_sec_ - // (~10 min default), inflating the global in-flight counter - // and blocking new admissions on busy clusters where many - // holders expire together (e.g. inference-worker rolling - // restarts). - if (shard->promotion_tasks.erase(it->first) > 0) { - promotion_in_flight_.fetch_sub(1, - std::memory_order_relaxed); - } - it = shard->metadata.erase(it); + it = EraseMetadataEntry(shard.get(), it); } else { ++it; } @@ -766,7 +901,7 @@ auto MasterService::ExistKey(const std::string& key) if (metadata.HasReplica(&Replica::fn_is_completed)) { // Grant a lease to the object as it may be further used by the // client. - metadata.GrantLease(default_kv_lease_ttl_, default_kv_soft_pin_ttl_); + GrantLeaseForGroup(accessor.GetShard().get(), key, metadata); return true; } @@ -1068,8 +1203,7 @@ auto MasterService::GetReplicaListByRegex(const std::string& regex_pattern) } results.emplace(key, std::move(replica_list)); - metadata.GrantLease(default_kv_lease_ttl_, - default_kv_soft_pin_ttl_); + GrantLeaseForGroup(shard.get(), key, metadata); } } } @@ -1114,7 +1248,7 @@ auto MasterService::GetReplicaList(const std::string& key) MasterMetricManager::instance().inc_valid_get_nums(); // Grant a lease to the object so it will not be removed // when the client is reading it. - metadata.GrantLease(default_kv_lease_ttl_, default_kv_soft_pin_ttl_); + GrantLeaseForGroup(accessor.GetShard().get(), key, metadata); // Promotion-on-hit eligibility: only when no MEMORY replica is // present but at least one LOCAL_DISK replica is. Decided here while @@ -1141,7 +1275,7 @@ auto MasterService::GetReplicaList(const std::string& key) auto MasterService::AllocateAndInsertMetadata( MetadataShardAccessorRW& shard, const UUID& client_id, const std::string& key, uint64_t value_length, - const ReplicateConfig& config, + const ReplicateConfig& config, const std::string& group_id, const std::chrono::system_clock::time_point& now) -> tl::expected, ErrorCode> { std::vector replicas; @@ -1270,11 +1404,15 @@ auto MasterService::AllocateAndInsertMetadata( } } + // Publish the route before metadata while holding the target shard lock, so + // same-key callers serialize on this shard instead of missing the grouped + // object and writing a duplicate on the natural key shard. + RegisterGroupMember(shard.get(), key, group_id); shard->metadata.emplace( std::piecewise_construct, std::forward_as_tuple(key), std::forward_as_tuple(client_id, now, value_length, std::move(replicas), config.with_soft_pin, config.with_hard_pin, - config.data_type)); + config.data_type, group_id)); shard->processing_keys.insert(key); return replica_list; @@ -1320,19 +1458,36 @@ auto MasterService::PutStart(const UUID& client_id, const std::string& key, VLOG(1) << "key=" << key << ", value_length=" << slice_length << ", config=" << config << ", action=put_start_begin"; + auto group_id_result = GetGroupIdForKey(config, 1, 0); + if (!group_id_result) { + return tl::make_unexpected(group_id_result.error()); + } + const std::string group_id = group_id_result.value(); + auto alive_clients = getAliveClientsSnapshot(); std::shared_lock shared_lock(snapshot_mutex_); - // Lock the shard and check if object already exists - MetadataShardAccessorRW shard(this, getShardIndex(key)); - const auto now = std::chrono::system_clock::now(); - auto it = shard->metadata.find(key); - if (it != shard->metadata.end() && - !CleanupStaleHandles(it->second, alive_clients)) { + + auto prepare_existing = + [this, &alive_clients, &key, &now]( + MetadataShardAccessorRW& shard, + std::unordered_map::iterator& it) + -> tl::expected { + if (it == shard->metadata.end()) { + return {}; + } + + if (CleanupStaleHandles(it->second, alive_clients)) { + shard->processing_keys.erase(key); + shard->replication_tasks.erase(key); + shard->offloading_tasks.erase(key); + it = EraseMetadataEntry(shard.get(), it); + return {}; + } + auto& metadata = it->second; - // If the object's PutStart expired and has not completed any - // replicas, we can discard it and allow the new PutStart to - // go. + // If the object's PutStart expired and has not completed any replicas, + // discard it and allow the new PutStart to go. if (!metadata.HasReplica(&Replica::fn_is_completed) && metadata.put_start_time + put_start_discard_timeout_sec_ < now) { auto replicas = metadata.PopReplicas(&Replica::fn_is_processing); @@ -1343,15 +1498,65 @@ auto MasterService::PutStart(const UUID& client_id, const std::string& key, metadata.put_start_time + put_start_release_timeout_sec_); } shard->processing_keys.erase(key); - shard->metadata.erase(it); - } else { - LOG(INFO) << "key=" << key << ", info=object_already_exists"; - return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); + it = EraseMetadataEntry(shard.get(), it); + return {}; } - } - return AllocateAndInsertMetadata(shard, client_id, key, slice_length, - config, now); + LOG(INFO) << "key=" << key << ", info=object_already_exists"; + return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); + }; + + auto put_with_locked_target = [&](MetadataShardAccessorRW& target_shard) + -> tl::expected, ErrorCode> { + auto target_it = target_shard->metadata.find(key); + auto result = prepare_existing(target_shard, target_it); + if (!result) { + return tl::make_unexpected(result.error()); + } + return AllocateAndInsertMetadata(target_shard, client_id, key, + slice_length, config, group_id, now); + }; + + auto put_with_locked_shards = [&](MetadataShardAccessorRW& existing_shard, + MetadataShardAccessorRW& target_shard) + -> tl::expected, ErrorCode> { + auto existing_it = existing_shard->metadata.find(key); + auto result = prepare_existing(existing_shard, existing_it); + if (!result) { + return tl::make_unexpected(result.error()); + } + return put_with_locked_target(target_shard); + }; + + while (true) { + const size_t existing_shard_idx = getMetadataShardIndex(key); + const size_t target_shard_idx = + group_id.empty() ? getShardIndex(key) : getShardIndex(group_id); + + if (existing_shard_idx == target_shard_idx) { + MetadataShardAccessorRW shard(this, target_shard_idx); + if (getMetadataShardIndex(key) != existing_shard_idx) { + continue; + } + return put_with_locked_target(shard); + } + + if (existing_shard_idx < target_shard_idx) { + MetadataShardAccessorRW existing_shard(this, existing_shard_idx); + MetadataShardAccessorRW target_shard(this, target_shard_idx); + if (getMetadataShardIndex(key) != existing_shard_idx) { + continue; + } + return put_with_locked_shards(existing_shard, target_shard); + } + + MetadataShardAccessorRW target_shard(this, target_shard_idx); + MetadataShardAccessorRW existing_shard(this, existing_shard_idx); + if (getMetadataShardIndex(key) != existing_shard_idx) { + continue; + } + return put_with_locked_shards(existing_shard, target_shard); + } } auto MasterService::PutEnd(const UUID& client_id, const std::string& key, @@ -1606,6 +1811,13 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, VLOG(1) << "key=" << key << ", value_length=" << slice_length << ", config=" << config << ", action=upsert_start_begin"; + auto group_id_result = GetGroupIdForKey(config, 1, 0); + if (!group_id_result) { + return tl::make_unexpected(group_id_result.error()); + } + const bool has_requested_group_id = config.group_ids.has_value(); + const std::string requested_group_id = group_id_result.value(); + // --- Lock acquisition --- // snapshot_mutex_ (shared): allows concurrent reads/writes, blocks only // during full metadata snapshots. @@ -1613,151 +1825,229 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, // operations on keys that hash to the same shard. auto alive_clients = getAliveClientsSnapshot(); std::shared_lock shared_lock(snapshot_mutex_); - MetadataShardAccessorRW shard(this, getShardIndex(key)); - const auto now = std::chrono::system_clock::now(); - auto it = shard->metadata.find(key); - // --- Step 0: stale handle cleanup --- - // If all memory replicas point to unmounted segments (node crashed and - // restarted), the metadata is useless — erase it and treat as new key. - // Also clean up local_disk replicas whose owner client has expired. - if (it != shard->metadata.end() && - CleanupStaleHandles(it->second, alive_clients)) { - shard->processing_keys.erase(key); - ErasePromotionTaskIfPresent(shard, key); - shard->metadata.erase(it); - it = shard->metadata.end(); - } + auto cleanup_existing_if_stale = + [this, &alive_clients, &key]( + MetadataShardAccessorRW& shard, + std::unordered_map::iterator& it) + -> bool { + if (it != shard->metadata.end() && + CleanupStaleHandles(it->second, alive_clients)) { + shard->processing_keys.erase(key); + shard->replication_tasks.erase(key); + shard->offloading_tasks.erase(key); + EraseMetadataEntry(shard.get(), it); + it = shard->metadata.end(); + return true; + } + return false; + }; - // --- Step 1: safety checks and preemption (only if key exists) --- - if (it != shard->metadata.end()) { + while (true) { + const size_t existing_shard_idx = getMetadataShardIndex(key); + const size_t requested_shard_idx = + has_requested_group_id && !requested_group_id.empty() + ? getShardIndex(requested_group_id) + : getShardIndex(key); + const size_t target_shard_idx = requested_shard_idx; + + std::optional first_shard_guard; + std::optional second_shard_guard; + MetadataShardAccessorRW* existing_shard = nullptr; + MetadataShardAccessorRW* target_shard = nullptr; + if (existing_shard_idx == target_shard_idx) { + first_shard_guard.emplace(this, target_shard_idx); + target_shard = &*first_shard_guard; + } else if (existing_shard_idx < target_shard_idx) { + first_shard_guard.emplace(this, existing_shard_idx); + second_shard_guard.emplace(this, target_shard_idx); + existing_shard = &*first_shard_guard; + target_shard = &*second_shard_guard; + } else { + first_shard_guard.emplace(this, target_shard_idx); + second_shard_guard.emplace(this, existing_shard_idx); + target_shard = &*first_shard_guard; + existing_shard = &*second_shard_guard; + } + + if (getMetadataShardIndex(key) != existing_shard_idx) { + continue; + } + + MetadataShardAccessorRW* shard_for_key = target_shard; + std::unordered_map::iterator it; + + if (existing_shard != nullptr) { + auto existing_it = (*existing_shard)->metadata.find(key); + cleanup_existing_if_stale(*existing_shard, existing_it); + if (existing_it != (*existing_shard)->metadata.end()) { + if (has_requested_group_id) { + LOG(ERROR) << "key=" << key + << ", error=group_membership_is_immutable"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + shard_for_key = existing_shard; + it = existing_it; + } else { + it = (*target_shard)->metadata.find(key); + cleanup_existing_if_stale(*target_shard, it); + } + } else { + it = (*target_shard)->metadata.find(key); + cleanup_existing_if_stale(*target_shard, it); + } + + MetadataShardAccessorRW& shard = *shard_for_key; + + // --- Step 1: safety checks and preemption (only if key exists) --- + if (it != shard->metadata.end()) { + auto& metadata = it->second; + + if (has_requested_group_id && + metadata.group_id != requested_group_id) { + LOG(ERROR) << "key=" << key + << ", error=group_membership_is_immutable"; + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + // Reject if a Copy/Move task is actively reading this key's + // replicas. Writing during replication would corrupt the copy. + if (shard->replication_tasks.count(key) > 0) { + LOG(INFO) << "key=" << key + << ", error=object_has_replication_task"; + return tl::make_unexpected( + ErrorCode::OBJECT_HAS_REPLICATION_TASK); + } + + // Reject if an offload-to-disk task is in progress (same reason). + if (shard->offloading_tasks.count(key) > 0) { + LOG(INFO) << "key=" << key + << ", error=object_has_offloading_task"; + return tl::make_unexpected( + ErrorCode::OBJECT_HAS_REPLICATION_TASK); + } + + // Preempt an in-progress Put/Upsert on the same key. The previous + // writer's PROCESSING replicas are moved to discarded_replicas_ + // with a TTL so they are not freed while the old writer may still + // be doing RDMA writes. Unlike PutStart (which only preempts after + // a timeout), UpsertStart preempts immediately. + if (shard->processing_keys.count(key) > 0) { + auto processing_replicas = + metadata.PopReplicas(&Replica::fn_is_processing); + if (!processing_replicas.empty()) { + std::lock_guard lock(discarded_replicas_mutex_); + discarded_replicas_.emplace_back( + std::move(processing_replicas), + now + put_start_release_timeout_sec_); + } + shard->processing_keys.erase(key); + + // If no COMPLETE replicas survive the preemption, this key + // effectively does not exist — fall through to Case A. + if (!metadata.HasReplica(&Replica::fn_is_completed)) { + it = EraseMetadataEntry(shard.get(), it); + it = shard->metadata.end(); + if (!has_requested_group_id && + shard_for_key != target_shard) { + continue; + } + } + } + } + + // --- Case A: key does not exist (or was erased above) --- + // Allocate fresh buffers, identical to PutStart. + if (it == shard->metadata.end()) { + VLOG(1) << "key=" << key << ", action=upsert_start_case_a"; + const std::string group_id = + has_requested_group_id ? requested_group_id : ""; + return AllocateAndInsertMetadata( + shard, client_id, key, slice_length, config, group_id, now); + } + + // --- Step 2: key exists with COMPLETE replicas → Case B or C --- auto& metadata = it->second; - // Reject if a Copy/Move task is actively reading this key's replicas. - // Writing during replication would corrupt the copy. - if (shard->replication_tasks.count(key) > 0) { - LOG(INFO) << "key=" << key << ", error=object_has_replication_task"; - return tl::make_unexpected(ErrorCode::OBJECT_HAS_REPLICATION_TASK); + // Reject if any reader holds a reference (refcnt > 0). Overwriting a + // buffer that an RDMA read is streaming from would cause data + // corruption. The client should retry after readers finish. + if (metadata.HasReplica(&Replica::fn_is_busy)) { + LOG(INFO) << "key=" << key << ", error=object_replica_busy"; + return tl::make_unexpected(ErrorCode::OBJECT_REPLICA_BUSY); } - // Reject if an offload-to-disk task is in progress (same reason). - if (shard->offloading_tasks.count(key) > 0) { - LOG(INFO) << "key=" << key << ", error=object_has_offloading_task"; - return tl::make_unexpected(ErrorCode::OBJECT_HAS_REPLICATION_TASK); - } + if (metadata.size == slice_length) { + // --- Case B: same size — in-place update --- + // Reuse existing buffer addresses. No allocation or deallocation. + // The client will RDMA-write new data to the same addresses. + // + // hard_pinned is const and preserved automatically — upsert does + // not change the eviction protection level of an existing object. + metadata.client_id = client_id; + metadata.put_start_time = now; - // Preempt an in-progress Put/Upsert on the same key. The previous - // writer's PROCESSING replicas are moved to discarded_replicas_ with a - // TTL so they are not freed while the old writer may still be doing - // RDMA writes. Unlike PutStart (which only preempts after a timeout), - // UpsertStart preempts immediately. - if (shard->processing_keys.count(key) > 0) { - auto processing_replicas = - metadata.PopReplicas(&Replica::fn_is_processing); - if (!processing_replicas.empty()) { - std::lock_guard lock(discarded_replicas_mutex_); - discarded_replicas_.emplace_back( - std::move(processing_replicas), - now + put_start_release_timeout_sec_); + // Reconcile soft_pin state with the incoming config. + { + SpinLocker locker(&metadata.lock); + if (config.with_soft_pin && !metadata.soft_pin_timeout) { + metadata.soft_pin_timeout.emplace(); + MasterMetricManager::instance().inc_soft_pin_key_count(1); + } else if (!config.with_soft_pin && metadata.soft_pin_timeout) { + metadata.soft_pin_timeout.reset(); + MasterMetricManager::instance().dec_soft_pin_key_count(1); + } } - shard->processing_keys.erase(key); - // If no COMPLETE replicas survive the preemption, this key - // effectively does not exist — fall through to Case A. - if (!metadata.HasReplica(&Replica::fn_is_completed)) { - ErasePromotionTaskIfPresent(shard, key); - shard->metadata.erase(it); - it = shard->metadata.end(); + // Mark COMPLETE → PROCESSING so readers won't see stale data + // mid-transfer. The key becomes unreadable until UpsertEnd. + metadata.VisitReplicas( + &Replica::fn_is_completed, + [](Replica& replica) { replica.mark_processing(); }); + + shard->processing_keys.insert(key); + + // Return the existing descriptors — same buffer addresses as + // before. + std::vector replica_list; + const auto& all_replicas = metadata.GetAllReplicas(); + replica_list.reserve(all_replicas.size()); + for (const auto& replica : all_replicas) { + replica_list.emplace_back(replica.get_descriptor()); } + + VLOG(1) << "key=" << key << ", action=upsert_start_case_b_inplace"; + return replica_list; } - } - // --- Case A: key does not exist (or was erased above) --- - // Allocate fresh buffers, identical to PutStart. - if (it == shard->metadata.end()) { - VLOG(1) << "key=" << key << ", action=upsert_start_case_a"; - return AllocateAndInsertMetadata(shard, client_id, key, slice_length, - config, now); - } - - // --- Step 2: key exists with COMPLETE replicas → Case B or C --- - auto& metadata = it->second; - - // Reject if any reader holds a reference (refcnt > 0). Overwriting a - // buffer that an RDMA read is streaming from would cause data corruption. - // The client should retry after readers finish. - if (metadata.HasReplica(&Replica::fn_is_busy)) { - LOG(INFO) << "key=" << key << ", error=object_replica_busy"; - return tl::make_unexpected(ErrorCode::OBJECT_REPLICA_BUSY); - } - - if (metadata.size == slice_length) { - // --- Case B: same size — in-place update --- - // Reuse existing buffer addresses. No allocation or deallocation. - // The client will RDMA-write new data to the same addresses. + // --- Case C: different size — discard old replicas and reallocate --- + // Old buffers cannot be reused. Move them to discarded_replicas_ for + // delayed release (readers may still hold descriptors without refcnt), + // then allocate fresh buffers at the new size. // - // hard_pinned is const and preserved automatically — upsert does not - // change the eviction protection level of an existing object. - metadata.client_id = client_id; - metadata.put_start_time = now; + // Preserve hard_pin and soft_pin from the old metadata so that eviction + // protection survives a size-changing upsert (RFC §2.2.2). + ReplicateConfig merged_config = config; + merged_config.with_hard_pin = + merged_config.with_hard_pin || metadata.IsHardPinned(); + merged_config.with_soft_pin = + merged_config.with_soft_pin || metadata.IsSoftPinned(); + const std::string group_id = + has_requested_group_id ? requested_group_id : metadata.group_id; - // Reconcile soft_pin state with the incoming config. - { - SpinLocker locker(&metadata.lock); - if (config.with_soft_pin && !metadata.soft_pin_timeout) { - metadata.soft_pin_timeout.emplace(); - MasterMetricManager::instance().inc_soft_pin_key_count(1); - } else if (!config.with_soft_pin && metadata.soft_pin_timeout) { - metadata.soft_pin_timeout.reset(); - MasterMetricManager::instance().dec_soft_pin_key_count(1); - } + auto old_replicas = metadata.PopReplicas(); + if (!old_replicas.empty()) { + std::lock_guard lock(discarded_replicas_mutex_); + discarded_replicas_.emplace_back( + std::move(old_replicas), now + put_start_release_timeout_sec_); } + EraseMetadataEntry(shard.get(), it); - // Mark COMPLETE → PROCESSING so readers won't see stale data - // mid-transfer. The key becomes unreadable until UpsertEnd. - metadata.VisitReplicas(&Replica::fn_is_completed, [](Replica& replica) { - replica.mark_processing(); - }); - - shard->processing_keys.insert(key); - - // Return the existing descriptors — same buffer addresses as before. - std::vector replica_list; - const auto& all_replicas = metadata.GetAllReplicas(); - replica_list.reserve(all_replicas.size()); - for (const auto& replica : all_replicas) { - replica_list.emplace_back(replica.get_descriptor()); - } - - VLOG(1) << "key=" << key << ", action=upsert_start_case_b_inplace"; - return replica_list; + VLOG(1) << "key=" << key << ", action=upsert_start_case_c_reallocate"; + return AllocateAndInsertMetadata(shard, client_id, key, slice_length, + merged_config, group_id, now); } - - // --- Case C: different size — discard old replicas and reallocate --- - // Old buffers cannot be reused. Move them to discarded_replicas_ for - // delayed release (readers may still hold descriptors without refcnt), - // then allocate fresh buffers at the new size. - // - // Preserve hard_pin and soft_pin from the old metadata so that eviction - // protection survives a size-changing upsert (RFC §2.2.2). - ReplicateConfig merged_config = config; - merged_config.with_hard_pin = - merged_config.with_hard_pin || metadata.IsHardPinned(); - merged_config.with_soft_pin = - merged_config.with_soft_pin || metadata.IsSoftPinned(); - - auto old_replicas = metadata.PopReplicas(); - if (!old_replicas.empty()) { - std::lock_guard lock(discarded_replicas_mutex_); - discarded_replicas_.emplace_back(std::move(old_replicas), - now + put_start_release_timeout_sec_); - } - shard->metadata.erase(it); - - VLOG(1) << "key=" << key << ", action=upsert_start_case_c_reallocate"; - return AllocateAndInsertMetadata(shard, client_id, key, slice_length, - merged_config, now); } auto MasterService::UpsertEnd(const UUID& client_id, const std::string& key, @@ -1784,12 +2074,22 @@ MasterService::BatchUpsertStart(const UUID& client_id, tl::expected, ErrorCode>>( keys.size(), tl::make_unexpected(ErrorCode::INVALID_PARAMS)); } + if (config.group_ids.has_value() && + config.group_ids->size() != keys.size()) { + LOG(ERROR) << "BatchUpsertStart: group_ids.size()=" + << config.group_ids->size() + << " != keys.size()=" << keys.size(); + return std::vector< + tl::expected, ErrorCode>>( + keys.size(), tl::make_unexpected(ErrorCode::INVALID_PARAMS)); + } std::vector, ErrorCode>> results; results.reserve(keys.size()); for (size_t i = 0; i < keys.size(); ++i) { + auto key_config = config.ForSingleKey(i); results.emplace_back( - UpsertStart(client_id, keys[i], slice_lengths[i], config)); + UpsertStart(client_id, keys[i], slice_lengths[i], key_config)); } return results; } @@ -2334,7 +2634,6 @@ auto MasterService::Remove(const std::string& key, bool force) // Remove object metadata accessor.Erase(); - ErasePromotionTaskIfPresent(accessor.GetShard(), key); return {}; } @@ -2388,8 +2687,7 @@ auto MasterService::RemoveByRegex(const std::string& regex_pattern, bool force) VLOG(1) << "key=" << it->first << " matched by regex. Removing."; - ErasePromotionTaskIfPresent(shard, it->first); - it = shard->metadata.erase(it); + it = EraseMetadataEntry(shard.get(), it); removed_count++; } else { ++it; @@ -2431,7 +2729,7 @@ long MasterService::RemoveAll(bool force) { auto mem_rep_count = it->second.CountReplicas(&Replica::fn_is_memory_replica); total_freed_size += it->second.size * mem_rep_count; - it = shard->metadata.erase(it); + it = EraseMetadataEntry(shard.get(), it); removed_count++; } else { ++it; @@ -2458,7 +2756,7 @@ auto MasterService::BatchRemove(const std::vector& keys, std::min(keys.size(), static_cast(kNumShards))); for (size_t i = 0; i < keys.size(); ++i) { - size_t shard_idx = getShardIndex(keys[i]); + size_t shard_idx = getMetadataShardIndex(keys[i]); keys_by_shard[shard_idx].emplace_back(i, &keys[i]); } @@ -2487,7 +2785,7 @@ auto MasterService::BatchRemove(const std::vector& keys, shard->processing_keys.erase(key); shard->replication_tasks.erase(key); shard->offloading_tasks.erase(key); - shard->metadata.erase(it); + EraseMetadataEntry(shard.get(), it); results[original_idx] = tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); continue; @@ -2523,7 +2821,7 @@ auto MasterService::BatchRemove(const std::vector& keys, } // Remove object metadata - shard->metadata.erase(it); + EraseMetadataEntry(shard.get(), it); results[original_idx] = {}; // Success } } @@ -3260,7 +3558,7 @@ void MasterService::DiscardExpiredProcessingReplicas( if (!metadata.IsValid() || metadata.AllReplicas(&Replica::fn_is_completed)) { if (!metadata.IsValid()) { - shard->metadata.erase(it); + EraseMetadataEntry(shard.get(), it); } key_it = shard->processing_keys.erase(key_it); continue; @@ -3282,7 +3580,7 @@ void MasterService::DiscardExpiredProcessingReplicas( if (!metadata.IsValid()) { // All replicas of this object are discarded, just // remove the whole object. - shard->metadata.erase(it); + EraseMetadataEntry(shard.get(), it); } key_it = shard->processing_keys.erase(key_it); @@ -3335,7 +3633,7 @@ void MasterService::DiscardExpiredProcessingReplicas( // Check whether the object is still valid. if (!metadata.IsValid()) { - shard->metadata.erase(metadata_it); + EraseMetadataEntry(shard.get(), metadata_it); } task_it = shard->replication_tasks.erase(task_it); @@ -4363,7 +4661,7 @@ bool MasterService::TryRestoreStateFromSnapshot( .time_since_epoch()) .count()) : "null"); - it = shard.metadata.erase(it); + it = EraseMetadataEntry(shard, it); } else { ++it; } @@ -4514,6 +4812,11 @@ void MasterService::BatchEvict(double evict_ratio_target, }); }; + struct EvictionResult { + uint64_t freed_bytes{0}; + long evicted_objects{0}; + }; + // --- Offload-on-evict support --- long offload_queued_this_cycle = 0; long offload_deferred_count = 0; @@ -4589,6 +4892,60 @@ void MasterService::BatchEvict(double evict_ratio_target, return 0; }; + auto try_evict_group_or_object = + [&, this](const std::string& key, ObjectMetadata& metadata, + MetadataShardAccessorRW& shard, + bool allow_soft_pinned) -> EvictionResult { + if (!metadata.IsGrouped()) { + uint64_t freed = try_evict_or_offload(key, metadata, shard); + return {.freed_bytes = freed, .evicted_objects = freed > 0 ? 1 : 0}; + } + + auto group_it = shard->group_members.find(metadata.group_id); + if (group_it == shard->group_members.end()) { + uint64_t freed = try_evict_or_offload(key, metadata, shard); + return {.freed_bytes = freed, .evicted_objects = freed > 0 ? 1 : 0}; + } + + for (const auto& member_key : group_it->second) { + auto member_it = shard->metadata.find(member_key); + if (member_it != shard->metadata.end() && + !member_it->second.IsLeaseExpired(now)) { + return {}; + } + } + + EvictionResult result; + std::vector member_keys(group_it->second.begin(), + group_it->second.end()); + for (const auto& member_key : member_keys) { + auto member_it = shard->metadata.find(member_key); + if (member_it == shard->metadata.end()) { + continue; + } + auto& member_metadata = member_it->second; + if (member_metadata.IsHardPinned() || + !member_metadata.IsLeaseExpired(now) || + (!allow_soft_pinned && member_metadata.IsSoftPinned(now)) || + !can_evict_replicas(member_metadata)) { + continue; + } + + uint64_t freed = + try_evict_or_offload(member_key, member_metadata, shard); + result.freed_bytes += freed; + if (freed > 0) { + result.evicted_objects++; + } + // The caller owns the iterator for the trigger key and erases it + // after this helper returns; erase only peer members here. + if (member_key != key && !member_metadata.IsValid()) { + EraseMetadataEntry(shard.get(), member_it); + } + } + return result; + }; + // Randomly select a starting shard to avoid imbalance eviction between // shards. size_t start_idx = RandomIndex(kNumShards); @@ -4662,17 +5019,16 @@ void MasterService::BatchEvict(double evict_ratio_target, } if (it->second.lease_timeout <= target_timeout) { // Evict this object (or defer for offload) - uint64_t freed = - try_evict_or_offload(it->first, it->second, shard); - total_freed_size += freed; + auto evict_result = + try_evict_group_or_object(it->first, it->second, shard, + /*allow_soft_pinned=*/false); + total_freed_size += evict_result.freed_bytes; if (it->second.IsValid() == false) { - it = shard->metadata.erase(it); + it = EraseMetadataEntry(shard.get(), it); } else { ++it; } - if (freed > 0) { - shard_evicted_count++; - } + shard_evicted_count += evict_result.evicted_objects; } else { // second pass candidates no_pin_objects.push_back(it->second.lease_timeout); @@ -4725,18 +5081,17 @@ void MasterService::BatchEvict(double evict_ratio_target, !it->second.IsSoftPinned(now) && can_evict_replicas(it->second)) { // Evict this object (or defer for offload) - uint64_t freed = - try_evict_or_offload(it->first, it->second, shard); - total_freed_size += freed; + auto evict_result = try_evict_group_or_object( + it->first, it->second, shard, + /*allow_soft_pinned=*/false); + total_freed_size += evict_result.freed_bytes; if (it->second.IsValid() == false) { - it = shard->metadata.erase(it); + it = EraseMetadataEntry(shard.get(), it); } else { ++it; } - if (freed > 0) { - evicted_count++; - } - target_evict_num--; + evicted_count += evict_result.evicted_objects; + target_evict_num -= evict_result.evicted_objects; } else { ++it; } @@ -4778,18 +5133,17 @@ void MasterService::BatchEvict(double evict_ratio_target, if (!it->second.IsSoftPinned(now) || it->second.lease_timeout <= soft_target_timeout) { // Evict this object (or defer for offload) - uint64_t freed = - try_evict_or_offload(it->first, it->second, shard); - total_freed_size += freed; + auto evict_result = try_evict_group_or_object( + it->first, it->second, shard, + /*allow_soft_pinned=*/true); + total_freed_size += evict_result.freed_bytes; if (it->second.IsValid() == false) { - it = shard->metadata.erase(it); + it = EraseMetadataEntry(shard.get(), it); } else { ++it; } - if (freed > 0) { - evicted_count++; - } - target_evict_num--; + evicted_count += evict_result.evicted_objects; + target_evict_num -= evict_result.evicted_objects; } else { ++it; } @@ -4902,7 +5256,7 @@ void MasterService::NoFBatchEvict(double evict_ratio_target, total_freed_size += metadata.size * erased; shard_evicted_count++; if (!metadata.IsValid()) { - it = shard->metadata.erase(it); + it = EraseMetadataEntry(shard.get(), it); } else { ++it; } @@ -5475,12 +5829,20 @@ MasterService::MetadataSerializer::Deserialize( Replica::next_id_.store(next_id); LOG(INFO) << "Restored Replica::next_id_ to " << next_id; + service_->RebuildGroupRoutingIndex(); return {}; } void MasterService::MetadataSerializer::Reset() { for (auto& shard : service_->metadata_shards_) { shard.metadata.clear(); + shard.group_members.clear(); + } + { + std::unique_lock lock( + service_->group_routing_mutex_); + service_->object_group_ids_.clear(); + service_->groups_needing_lease_refresh_.clear(); } { std::lock_guard lock(service_->discarded_replicas_mutex_); @@ -5549,6 +5911,7 @@ MasterService::MetadataSerializer::DeserializeShard(const msgpack::object& obj, // Clear existing data shard.metadata.clear(); + shard.group_members.clear(); // Deserialize metadata if (metadata_array == nullptr || @@ -5586,7 +5949,8 @@ MasterService::MetadataSerializer::DeserializeShard(const msgpack::object& obj, metadata_ptr->client_id, metadata_ptr->put_start_time, metadata_ptr->size, metadata_ptr->PopReplicas(), metadata_ptr->soft_pin_timeout.has_value(), - metadata_ptr->IsHardPinned(), metadata_ptr->data_type)); + metadata_ptr->IsHardPinned(), metadata_ptr->data_type, + metadata_ptr->group_id)); it->second.lease_timeout = metadata_ptr->lease_timeout; it->second.soft_pin_timeout = metadata_ptr->soft_pin_timeout; @@ -5602,11 +5966,11 @@ MasterService::MetadataSerializer::SerializeMetadata( // Pack ObjectMetadata using array structure for efficiency // Format: [client_id, put_start_time, size, lease_timeout, // has_soft_pin_timeout, soft_pin_timeout, replicas_count, data_type, - // replicas..., hard_pinned] + // replicas..., hard_pinned, group_id] - size_t array_size = 9; // client_id, put_start_time, size, lease_timeout, - // has_soft_pin_timeout, soft_pin_timeout, - // replicas_count, data_type, hard_pinned + size_t array_size = 10; // client_id, put_start_time, size, lease_timeout, + // has_soft_pin_timeout, soft_pin_timeout, + // replicas_count, data_type, hard_pinned, group_id array_size += metadata.CountReplicas(); // One element per replica packer.pack_array(array_size); @@ -5659,6 +6023,7 @@ MasterService::MetadataSerializer::SerializeMetadata( } packer.pack(metadata.IsHardPinned()); + packer.pack(metadata.group_id); return {}; } @@ -5674,7 +6039,9 @@ MasterService::MetadataSerializer::DeserializeMetadata( } // Need at least 7 elements: client_id, put_start_time, size, lease_timeout, - // has_soft_pin_timeout, soft_pin_timeout, replicas_count + // has_soft_pin_timeout, soft_pin_timeout, replicas_count. + // Optional fields are decoded by type for backward compatibility: + // data_type appears before replicas; hard_pinned and group_id trail them. if (obj.via.array.size < 7) { return tl::unexpected(SerializationError( ErrorCode::DESERIALIZE_FAIL, @@ -5708,31 +6075,25 @@ MasterService::MetadataSerializer::DeserializeMetadata( uint32_t replicas_count = array[index++].as(); // Format detection: - // v1: 7 + replicas_count, no data_type or hard_pinned - // v2: 8 + replicas_count, either data_type or trailing hard_pinned - // v3: 9 + replicas_count, data_type plus trailing hard_pinned - constexpr uint32_t kOldFieldCount = 7; - constexpr uint32_t kOneExtraFieldCount = 8; - constexpr uint32_t kCurrentFieldCount = 9; + // v1: 7 + replicas_count, no optional fields + // v2: 8 + replicas_count, either data_type or hard_pinned + // v3: 9 + replicas_count, data_type + hard_pinned or + // hard_pinned + group_id + // v4: 10 + replicas_count, data_type + hard_pinned + group_id + constexpr uint32_t kBaseFieldCount = 7; + constexpr uint32_t kMaxOptionalFieldCount = 3; const uint32_t total_elements = obj.via.array.size; - const bool is_old_format = - (total_elements == kOldFieldCount + replicas_count); - const bool is_one_extra_format = - (total_elements == kOneExtraFieldCount + replicas_count); - const bool is_current_format = - (total_elements == kCurrentFieldCount + replicas_count); - - if (!is_current_format && !is_one_extra_format && !is_old_format) { + const uint32_t min_elements = kBaseFieldCount + replicas_count; + if (total_elements < min_elements || + total_elements > min_elements + kMaxOptionalFieldCount) { return tl::unexpected(SerializationError( ErrorCode::DESERIALIZE_FAIL, "deserialize ObjectMetadata array size mismatch")); } ObjectDataType data_type = ObjectDataType::UNKNOWN; - if (is_current_format) { - data_type = static_cast(array[index++].as()); - } else if (is_one_extra_format && - array[index].type == msgpack::type::POSITIVE_INTEGER) { + if (index < total_elements && + array[index].type == msgpack::type::POSITIVE_INTEGER) { data_type = static_cast(array[index++].as()); } @@ -5751,17 +6112,29 @@ MasterService::MetadataSerializer::DeserializeMetadata( // Deserialize hard_pinned (if present, otherwise default to false) bool is_hard_pinned = false; - if (index < obj.via.array.size) { + if (index < obj.via.array.size && + array[index].type == msgpack::type::BOOLEAN) { is_hard_pinned = array[index++].as(); } + std::string group_id; + if (index < obj.via.array.size && array[index].type == msgpack::type::STR) { + group_id = array[index++].as(); + } + if (index != obj.via.array.size) { + return tl::unexpected(SerializationError( + ErrorCode::DESERIALIZE_FAIL, + "deserialize ObjectMetadata optional field type mismatch")); + } + // Create ObjectMetadata instance bool enable_soft_pin = has_soft_pin_timeout; auto metadata = std::make_unique( client_id, std::chrono::system_clock::time_point( std::chrono::milliseconds(put_start_time_timestamp)), - size, std::move(replicas), enable_soft_pin, is_hard_pinned, data_type); + size, std::move(replicas), enable_soft_pin, is_hard_pinned, data_type, + group_id); metadata->lease_timeout = std::chrono::system_clock::time_point( std::chrono::milliseconds(lease_timestamp)); diff --git a/mooncake-store/src/rpc_service.cpp b/mooncake-store/src/rpc_service.cpp index d0140b57..5ad88374 100644 --- a/mooncake-store/src/rpc_service.cpp +++ b/mooncake-store/src/rpc_service.cpp @@ -75,6 +75,7 @@ void SetServiceUnavailable(coro_http::coro_http_response& resp, resp.set_status_and_content(coro_http::status_type::service_unavailable, payload); } + struct HttpErrorResponse { bool success{false}; int32_t error_code{0}; @@ -1005,11 +1006,24 @@ WrappedMasterService::BatchPutStart(const UUID& client_id, results; results.reserve(keys.size()); - if (config.prefer_alloc_in_same_node) { + if (keys.size() != slice_lengths.size()) { + LOG(ERROR) << "BatchPutStart: keys.size()=" << keys.size() + << " != slice_lengths.size()=" << slice_lengths.size(); + results.assign(keys.size(), + tl::make_unexpected(ErrorCode::INVALID_PARAMS)); + } else if (config.group_ids.has_value() && + config.group_ids->size() != keys.size()) { + LOG(ERROR) << "BatchPutStart: group_ids.size()=" + << config.group_ids->size() + << " != keys.size()=" << keys.size(); + results.assign(keys.size(), + tl::make_unexpected(ErrorCode::INVALID_PARAMS)); + } else if (config.prefer_alloc_in_same_node) { ReplicateConfig new_config = config; for (size_t i = 0; i < keys.size(); ++i) { + auto key_config = new_config.ForSingleKey(i); auto result = master_service_.PutStart( - client_id, keys[i], slice_lengths[i], new_config); + client_id, keys[i], slice_lengths[i], key_config); results.emplace_back(result); if ((i == 0) && result.has_value()) { std::string preferred_segment; @@ -1029,8 +1043,9 @@ WrappedMasterService::BatchPutStart(const UUID& client_id, } } else { for (size_t i = 0; i < keys.size(); ++i) { + auto key_config = config.ForSingleKey(i); results.emplace_back(master_service_.PutStart( - client_id, keys[i], slice_lengths[i], config)); + client_id, keys[i], slice_lengths[i], key_config)); } } diff --git a/mooncake-store/tests/client_integration_test.cpp b/mooncake-store/tests/client_integration_test.cpp index 94c7e387..e30b6df4 100644 --- a/mooncake-store/tests/client_integration_test.cpp +++ b/mooncake-store/tests/client_integration_test.cpp @@ -4,6 +4,7 @@ #include #include +#include #include #include #include @@ -653,6 +654,82 @@ TEST_F(ClientIntegrationTest, BatchPutGetOperations) { } } +TEST_F(ClientIntegrationTest, BatchPutMixedGroupIdsThroughClient) { + auto find_group_id_on_different_shard = [](const std::string& key) { + static constexpr size_t kMetadataShardCountForTest = 1024; + const size_t key_shard = + std::hash{}(key) % kMetadataShardCountForTest; + for (int i = 0; i < 10000; ++i) { + std::string group_id = key + "_group_" + std::to_string(i); + if (std::hash{}(group_id) % + kMetadataShardCountForTest != + key_shard) { + return group_id; + } + } + return key + "_fallback_group"; + }; + + const std::vector keys = { + "client_batch_grouped_a", + "client_batch_ungrouped", + "client_batch_grouped_b", + }; + const std::vector values = { + "grouped-value-a", + "ungrouped-value", + "grouped-value-b", + }; + + std::vector source_buffers; + std::vector> batched_slices; + source_buffers.reserve(values.size()); + batched_slices.reserve(values.size()); + for (const auto& value : values) { + void* buffer = client_buffer_allocator_->allocate(value.size()); + memcpy(buffer, value.data(), value.size()); + source_buffers.push_back(buffer); + batched_slices.push_back({Slice{buffer, value.size()}}); + } + + ReplicateConfig config; + config.replica_num = 1; + config.group_ids = + std::vector{find_group_id_on_different_shard(keys[0]), "", + find_group_id_on_different_shard(keys[2])}; + + auto put_results = test_client_->BatchPut(keys, batched_slices, config); + ASSERT_EQ(put_results.size(), keys.size()); + for (const auto& result : put_results) { + ASSERT_TRUE(result.has_value()) + << "BatchPut failed: " << toString(result.error()); + } + + for (size_t i = 0; i < values.size(); ++i) { + client_buffer_allocator_->deallocate(source_buffers[i], + values[i].size()); + } + + for (size_t i = 0; i < keys.size(); ++i) { + void* target_buffer = + client_buffer_allocator_->allocate(values[i].size()); + std::vector slices{Slice{target_buffer, values[i].size()}}; + auto get_result = test_client_->Get(keys[i], slices); + ASSERT_TRUE(get_result.has_value()) + << "Get failed: " << toString(get_result.error()); + EXPECT_EQ(slices[0].size, values[i].size()); + EXPECT_EQ(memcmp(slices[0].ptr, values[i].data(), values[i].size()), 0); + client_buffer_allocator_->deallocate(target_buffer, values[i].size()); + } + + auto remove_results = test_client_->BatchRemove(keys, /*force=*/true); + ASSERT_EQ(remove_results.size(), keys.size()); + for (const auto& result : remove_results) { + ASSERT_TRUE(result.has_value()) + << "BatchRemove failed: " << toString(result.error()); + } +} + // Test batch IsExist operations through the client TEST_F(ClientIntegrationTest, BatchIsExistOperations) { int batch_size = 50; diff --git a/mooncake-store/tests/ha/snapshot/snapshot_child_process_test.cpp b/mooncake-store/tests/ha/snapshot/snapshot_child_process_test.cpp index 0f2a4493..e0b0e9de 100644 --- a/mooncake-store/tests/ha/snapshot/snapshot_child_process_test.cpp +++ b/mooncake-store/tests/ha/snapshot/snapshot_child_process_test.cpp @@ -2,6 +2,7 @@ #include "master_metric_manager.h" #include "ha/snapshot/catalog/snapshot_catalog_store.h" #include "ha/snapshot/object/snapshot_object_store.h" +#include "ha/snapshot/snapshot_test_utils.h" #ifdef STORE_USE_ETCD #include "etcd_helper.h" #include "ha/oplog/etcd_oplog_store.h" @@ -161,6 +162,36 @@ class SnapshotChildProcessTest : public ::testing::Test { return shard.metadata.find(key) != shard.metadata.end(); } + uint32_t GetShardIndexForTest(const std::string& key) { + return static_cast(service_->getShardIndex(key)); + } + + tl::expected DeserializeMetadataForTest( + const std::vector& data) { + MasterService::MetadataSerializer serializer(service_.get()); + return serializer.Deserialize(data); + } + + bool ObjectIsGroupedInMetadata(const std::string& key, size_t shard_idx) { + auto& shard = service_->metadata_shards_[shard_idx]; + SharedMutexLocker lock(&shard.mutex, shared_lock_t{}); + auto it = shard.metadata.find(key); + EXPECT_NE(it, shard.metadata.end()); + return it != shard.metadata.end() && it->second.IsGrouped(); + } + + std::string FindGroupIdOnDifferentShard(MasterService* svc, + const std::string& key) { + const size_t key_shard = svc->getShardIndex(key); + for (int i = 0; i < 1024; ++i) { + std::string group_id = key + "_group_" + std::to_string(i); + if (svc->getShardIndex(group_id) != key_shard) { + return group_id; + } + } + return key + "_group"; + } + private: std::string tmp_dir_; }; @@ -407,6 +438,109 @@ TEST_F(SnapshotChildProcessTest, PersistState_UsesFrozenSnapshotDescriptor) { EXPECT_EQ(latest->value().created_at_ms, descriptor.created_at_ms); } +TEST_F(SnapshotChildProcessTest, RestoreRebuildsGroupedObjectRouting) { + auto make_config = [this]() { + return MasterServiceConfigBuilder() + .set_enable_snapshot(false) + .set_enable_snapshot_restore(true) + .set_snapshot_backup_dir(tmp_dir() + "/backup") + .set_snapshot_interval_seconds(100) + .set_snapshot_child_timeout_seconds(60) + .set_snapshot_retention_count(3) + .set_snapshot_object_store_type("local") + .set_default_kv_lease_ttl(600000) + .build(); + }; + service_ = std::make_unique(make_config()); + + Segment segment; + segment.id = generate_uuid(); + segment.name = "grouped_snapshot_segment"; + segment.base = 0x310000000; + segment.size = 1024 * 1024 * 16; + segment.te_endpoint = segment.name; + const UUID client_id = generate_uuid(); + ASSERT_TRUE(service_->MountSegment(segment, client_id).has_value()); + + const std::string key = "snapshot_grouped_route_key"; + ReplicateConfig replicate_config; + replicate_config.replica_num = 1; + replicate_config.group_ids = std::vector{ + FindGroupIdOnDifferentShard(service_.get(), key)}; + + auto put_start = service_->PutStart(client_id, key, 1024, replicate_config); + ASSERT_TRUE(put_start.has_value()) << toString(put_start.error()); + ASSERT_TRUE( + service_->PutEnd(client_id, key, ReplicaType::MEMORY).has_value()); + ASSERT_TRUE(service_->ExistKey(key).value_or(false)); + + auto persist_result = CallPersistState("20240701_130000_000"); + ASSERT_TRUE(persist_result.has_value()) + << "PersistState failed: " << persist_result.error().message; + + service_.reset(); + service_ = std::make_unique(make_config()); + + auto restored_replicas = service_->GetReplicaList(key); + ASSERT_TRUE(restored_replicas.has_value()) + << "Grouped key should remain reachable by key after restore"; + ASSERT_TRUE(service_->Remove(key, /*force=*/true).has_value()); + EXPECT_FALSE(service_->ExistKey(key).value_or(true)); +} + +TEST_F(SnapshotChildProcessTest, + DeserializeLegacyMetadataWithoutGroupIdRestoresUngroupedObject) { + CreateDefaultService(); + const std::string key = "legacy_snapshot_no_group_id_key"; + const uint32_t shard_idx = GetShardIndexForTest(key); + const UUID client_id = generate_uuid(); + + msgpack::sbuffer shard_buffer; + MsgpackPacker shard_packer(&shard_buffer); + shard_packer.pack_map(1); + shard_packer.pack(std::string("metadata")); + shard_packer.pack_array(1); + shard_packer.pack_array(2); + shard_packer.pack(key); + + shard_packer.pack_array(8); + shard_packer.pack(UuidToString(client_id)); + shard_packer.pack(kDefaultTestPutStartTimeMs); + shard_packer.pack(kDefaultTestObjectSize); + shard_packer.pack(kDefaultTestLeaseTimeoutMs); + shard_packer.pack(false); + shard_packer.pack(uint64_t{0}); + shard_packer.pack(uint32_t{1}); + PackDiskReplica(shard_packer, kDefaultTestDiskFilePath, + kDefaultTestObjectSize); + + auto compressed_shard = + zstd_compress(reinterpret_cast(shard_buffer.data()), + shard_buffer.size(), 3); + + msgpack::sbuffer root_buffer; + MsgpackPacker root_packer(&root_buffer); + root_packer.pack_map(3); + root_packer.pack(std::string("shards")); + root_packer.pack_map(1); + root_packer.pack(shard_idx); + root_packer.pack_bin(compressed_shard.size()); + root_packer.pack_bin_body( + reinterpret_cast(compressed_shard.data()), + compressed_shard.size()); + root_packer.pack(std::string("discarded_replicas")); + root_packer.pack_array(0); + root_packer.pack(std::string("replica_next_id")); + root_packer.pack(uint64_t{10}); + + auto deserialize_result = + DeserializeMetadataForTest(ToByteVector(root_buffer)); + ASSERT_TRUE(deserialize_result.has_value()) + << deserialize_result.error().message; + + EXPECT_FALSE(ObjectIsGroupedInMetadata(key, shard_idx)); +} + TEST_F(SnapshotChildProcessTest, LegacyEtcdConnstringFallbackIsPreserved) { MasterConfig legacy_config; legacy_config.enable_ha = true; diff --git a/mooncake-store/tests/master_service_test.cpp b/mooncake-store/tests/master_service_test.cpp index 050e3d46..b7c95c0c 100644 --- a/mooncake-store/tests/master_service_test.cpp +++ b/mooncake-store/tests/master_service_test.cpp @@ -1,4 +1,5 @@ #include "master_service.h" +#include "rpc_service.h" #include #include @@ -6,6 +7,7 @@ #include #include +#include #include #include #include @@ -90,6 +92,33 @@ class MasterServiceTest : public ::testing::Test { return key; } + std::string FindGroupIdOnDifferentShard(const std::string& key) const { + static constexpr size_t kMetadataShardCountForTest = 1024; + const size_t key_shard = + std::hash{}(key) % kMetadataShardCountForTest; + for (int i = 0; i < 10000; ++i) { + std::string group_id = key + "_group_" + std::to_string(i); + if (std::hash{}(group_id) % + kMetadataShardCountForTest != + key_shard) { + return group_id; + } + } + return key + "_fallback_group"; + } + + void PutCompletedObject(MasterService& service, const UUID& client_id, + const std::string& key, + const ReplicateConfig& config, + uint64_t slice_length = 1024) const { + auto put_start = service.PutStart(client_id, key, slice_length, config); + ASSERT_TRUE(put_start.has_value()) + << "PutStart failed for key=" << key + << ", error=" << toString(put_start.error()); + ASSERT_TRUE( + service.PutEnd(client_id, key, ReplicaType::MEMORY).has_value()); + } + bool ExecutePendingMoveTasks(MasterService& service, const UUID& client_id) const { auto fetched = service.FetchTasks(client_id, /*batch_size=*/16); @@ -534,6 +563,475 @@ TEST_F(MasterServiceTest, PutStartOnePlusOneAllowsSingleAllocatedReplica) { } #endif +TEST_F(MasterServiceTest, PutStartGroupIdsValidation) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + ReplicateConfig config; + config.replica_num = 1; + + config.group_ids = std::vector{}; + auto empty_group_ids = + service_->PutStart(client_id, "empty_group_ids", 1024, config); + EXPECT_FALSE(empty_group_ids.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, empty_group_ids.error()); + + config.group_ids = std::vector{"g0", "g1"}; + auto too_many_group_ids = + service_->PutStart(client_id, "too_many_group_ids", 1024, config); + EXPECT_FALSE(too_many_group_ids.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, too_many_group_ids.error()); + + config.group_ids = std::vector{""}; + auto ungrouped = + service_->PutStart(client_id, "explicit_ungrouped", 1024, config); + ASSERT_TRUE(ungrouped.has_value()); + ASSERT_TRUE( + service_->PutEnd(client_id, "explicit_ungrouped", ReplicaType::MEMORY) + .has_value()); + auto exists = service_->ExistKey("explicit_ungrouped"); + ASSERT_TRUE(exists.has_value()); + EXPECT_TRUE(exists.value()); +} + +TEST_F(MasterServiceTest, GroupedObjectRoutesKeyLevelLookupAndRemove) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "grouped_route_key"; + ReplicateConfig config; + config.replica_num = 1; + config.group_ids = + std::vector{FindGroupIdOnDifferentShard(key)}; + + PutCompletedObject(*service_, client_id, key, config); + + auto exists = service_->ExistKey(key); + ASSERT_TRUE(exists.has_value()); + EXPECT_TRUE(exists.value()); + EXPECT_TRUE(service_->GetReplicaList(key).has_value()); + + ASSERT_TRUE(service_->Remove(key, /*force=*/true).has_value()); + auto exists_after_remove = service_->ExistKey(key); + ASSERT_TRUE(exists_after_remove.has_value()); + EXPECT_FALSE(exists_after_remove.value()); +} + +TEST_F(MasterServiceTest, ExpiredGroupedPutCanBeReplacedByUngroupedPut) { + auto service_config = MasterServiceConfig::builder() + .set_put_start_discard_timeout_sec(0) + .set_put_start_release_timeout_sec(1) + .build(); + std::unique_ptr service_(new MasterService(service_config)); + const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = context.client_id; + + const std::string key = "expired_grouped_put_to_ungrouped"; + ReplicateConfig grouped_config; + grouped_config.replica_num = 1; + grouped_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(key)}; + + ASSERT_TRUE( + service_->PutStart(client_id, key, 1024, grouped_config).has_value()); + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + + ReplicateConfig ungrouped_config; + ungrouped_config.replica_num = 1; + auto put_start = service_->PutStart(client_id, key, 1024, ungrouped_config); + ASSERT_TRUE(put_start.has_value()) << toString(put_start.error()); + ASSERT_TRUE( + service_->PutEnd(client_id, key, ReplicaType::MEMORY).has_value()); + EXPECT_TRUE(service_->ExistKey(key).value_or(false)); +} + +TEST_F(MasterServiceTest, GroupedLeaseRefreshNearExpiryProtectsCurrentMembers) { + auto service_config = + MasterServiceConfig::builder().set_default_kv_lease_ttl(200).build(); + std::unique_ptr service_(new MasterService(service_config)); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key_a = "lease_group_key_a"; + const std::string key_b = "lease_group_key_b"; + const std::string group_id = FindGroupIdOnDifferentShard(key_a); + + ReplicateConfig config_a; + config_a.replica_num = 1; + config_a.group_ids = std::vector{group_id}; + ReplicateConfig config_b = config_a; + + PutCompletedObject(*service_, client_id, key_a, config_a); + PutCompletedObject(*service_, client_id, key_b, config_b); + + auto exists = service_->ExistKey(key_a); + ASSERT_TRUE(exists.has_value()); + ASSERT_TRUE(exists.value()); + + std::this_thread::sleep_for(std::chrono::milliseconds(120)); + exists = service_->ExistKey(key_a); + ASSERT_TRUE(exists.has_value()); + ASSERT_TRUE(exists.value()); + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + + auto remove_group_peer = service_->Remove(key_b); + ASSERT_FALSE(remove_group_peer.has_value()); + EXPECT_EQ(ErrorCode::OBJECT_HAS_LEASE, remove_group_peer.error()); + + EXPECT_TRUE(service_->Remove(key_a, /*force=*/true).has_value()); + EXPECT_TRUE(service_->Remove(key_b, /*force=*/true).has_value()); +} + +TEST_F(MasterServiceTest, + GroupedLeaseRefreshAfterMembershipChangeDoesNotWaitForTriggerExpiry) { + auto service_config = + MasterServiceConfig::builder().set_default_kv_lease_ttl(500).build(); + std::unique_ptr service_(new MasterService(service_config)); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key_a = "lease_group_dirty_key_a"; + const std::string key_b = "lease_group_dirty_key_b"; + const std::string group_id = FindGroupIdOnDifferentShard(key_a); + + ReplicateConfig config; + config.replica_num = 1; + config.group_ids = std::vector{group_id}; + + PutCompletedObject(*service_, client_id, key_a, config); + ASSERT_TRUE(service_->ExistKey(key_a).value_or(false)); + + PutCompletedObject(*service_, client_id, key_b, config); + std::this_thread::sleep_for(std::chrono::milliseconds(150)); + + auto exists = service_->ExistKey(key_a); + ASSERT_TRUE(exists.has_value()); + ASSERT_TRUE(exists.value()); + std::this_thread::sleep_for(std::chrono::milliseconds(390)); + + auto remove_group_peer = service_->Remove(key_b); + ASSERT_FALSE(remove_group_peer.has_value()); + EXPECT_EQ(ErrorCode::OBJECT_HAS_LEASE, remove_group_peer.error()); + + EXPECT_TRUE(service_->Remove(key_a, /*force=*/true).has_value()); + EXPECT_TRUE(service_->Remove(key_b, /*force=*/true).has_value()); +} + +TEST_F(MasterServiceTest, RemoveGroupedMemberPreservesOtherMembers) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key_a = "remove_group_key_a"; + const std::string key_b = "remove_group_key_b"; + const std::string group_id = FindGroupIdOnDifferentShard(key_a); + + ReplicateConfig config; + config.replica_num = 1; + config.group_ids = std::vector{group_id}; + PutCompletedObject(*service_, client_id, key_a, config); + PutCompletedObject(*service_, client_id, key_b, config); + + ASSERT_TRUE(service_->Remove(key_a, /*force=*/true).has_value()); + + auto removed_exists = service_->ExistKey(key_a); + ASSERT_TRUE(removed_exists.has_value()); + EXPECT_FALSE(removed_exists.value()); + EXPECT_TRUE(service_->GetReplicaList(key_b).has_value()); + + ASSERT_TRUE(service_->Remove(key_b, /*force=*/true).has_value()); + auto group_empty = service_->ExistKey(key_b); + ASSERT_TRUE(group_empty.has_value()); + EXPECT_FALSE(group_empty.value()); +} + +TEST_F(MasterServiceTest, UpsertPreservesGroupMembership) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "upsert_group_key"; + const std::string group_id = FindGroupIdOnDifferentShard(key); + + ReplicateConfig grouped_config; + grouped_config.replica_num = 1; + grouped_config.group_ids = std::vector{group_id}; + PutCompletedObject(*service_, client_id, key, grouped_config); + + ReplicateConfig unset_group_config; + unset_group_config.replica_num = 1; + auto preserve_result = + service_->UpsertStart(client_id, key, 1024, unset_group_config); + ASSERT_TRUE(preserve_result.has_value()) + << "Unset group_ids should preserve existing group membership"; + ASSERT_TRUE( + service_->UpsertEnd(client_id, key, ReplicaType::MEMORY).has_value()); + EXPECT_TRUE(service_->GetReplicaList(key).has_value()); + + ReplicateConfig different_group_config; + different_group_config.replica_num = 1; + different_group_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(key + "_other")}; + auto different_group_result = + service_->UpsertStart(client_id, key, 1024, different_group_config); + ASSERT_FALSE(different_group_result.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, different_group_result.error()); + + ReplicateConfig explicit_ungrouped_config; + explicit_ungrouped_config.replica_num = 1; + explicit_ungrouped_config.group_ids = std::vector{""}; + auto explicit_ungrouped_result = + service_->UpsertStart(client_id, key, 1024, explicit_ungrouped_config); + ASSERT_FALSE(explicit_ungrouped_result.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, explicit_ungrouped_result.error()); +} + +TEST_F(MasterServiceTest, IncompleteGroupedUpsertCanBecomeUngrouped) { + std::unique_ptr service_(new MasterService()); + const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = context.client_id; + + const std::string key = "incomplete_grouped_upsert_to_ungrouped"; + ReplicateConfig grouped_config; + grouped_config.replica_num = 1; + grouped_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(key)}; + + ASSERT_TRUE( + service_->PutStart(client_id, key, 1024, grouped_config).has_value()); + std::this_thread::sleep_for(std::chrono::milliseconds(2)); + + ReplicateConfig ungrouped_config; + ungrouped_config.replica_num = 1; + auto upsert_start = + service_->UpsertStart(client_id, key, 1024, ungrouped_config); + ASSERT_TRUE(upsert_start.has_value()) << toString(upsert_start.error()); + ASSERT_TRUE( + service_->UpsertEnd(client_id, key, ReplicaType::MEMORY).has_value()); + EXPECT_TRUE(service_->ExistKey(key).value_or(false)); +} + +TEST_F(MasterServiceTest, + GroupedEvictionExpandsSafeMembersAndSkipsLeasedGroup) { + auto service_config = + MasterServiceConfig::builder().set_default_kv_lease_ttl(1000).build(); + constexpr size_t kSegmentSize = 4 * 1024 * 1024; + constexpr size_t kObjectSize = 2 * 1024 * 1024; + + { + std::unique_ptr service_( + new MasterService(service_config)); + [[maybe_unused]] const auto context = + PrepareSimpleSegment(*service_, "grouped_evict_segment", + kDefaultSegmentBase, kSegmentSize); + const UUID client_id = generate_uuid(); + + const std::string evict_key_a = "grouped_evict_key_a"; + const std::string evict_key_b = "grouped_evict_key_b"; + ReplicateConfig evict_config; + evict_config.replica_num = 1; + evict_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(evict_key_a)}; + PutCompletedObject(*service_, client_id, evict_key_a, evict_config, + kObjectSize); + PutCompletedObject(*service_, client_id, evict_key_b, evict_config, + kObjectSize); + + ReplicateConfig trigger_config; + trigger_config.replica_num = 1; + auto trigger_result = service_->PutStart( + client_id, "trigger_grouped_eviction", kObjectSize, trigger_config); + ASSERT_FALSE(trigger_result.has_value()); + EXPECT_EQ(ErrorCode::NO_AVAILABLE_HANDLE, trigger_result.error()); + + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + + EXPECT_FALSE(service_->ExistKey(evict_key_a).value_or(true)); + EXPECT_FALSE(service_->ExistKey(evict_key_b).value_or(true)); + } + + { + std::unique_ptr service_( + new MasterService(service_config)); + [[maybe_unused]] const auto context = + PrepareSimpleSegment(*service_, "grouped_lease_segment", + kDefaultSegmentBase, kSegmentSize); + const UUID client_id = generate_uuid(); + + const std::string leased_key_a = "grouped_leased_key_a"; + const std::string leased_key_b = "grouped_leased_key_b"; + ReplicateConfig leased_config; + leased_config.replica_num = 1; + leased_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(leased_key_a)}; + PutCompletedObject(*service_, client_id, leased_key_a, leased_config, + kObjectSize); + PutCompletedObject(*service_, client_id, leased_key_b, leased_config, + kObjectSize); + + auto exists = service_->ExistKey(leased_key_a); + ASSERT_TRUE(exists.has_value()); + ASSERT_TRUE(exists.value()); + + ReplicateConfig trigger_config; + trigger_config.replica_num = 1; + auto trigger_result = + service_->PutStart(client_id, "trigger_leased_group_eviction", + kObjectSize, trigger_config); + ASSERT_FALSE(trigger_result.has_value()); + EXPECT_EQ(ErrorCode::NO_AVAILABLE_HANDLE, trigger_result.error()); + + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + + EXPECT_TRUE(service_->GetReplicaList(leased_key_a).has_value()); + EXPECT_TRUE(service_->GetReplicaList(leased_key_b).has_value()); + } +} + +TEST_F(MasterServiceTest, GroupedEvictionSkipsUnsafeMembersAndEvictsSafePeers) { + constexpr size_t kSegmentSize = 4 * 1024 * 1024; + constexpr size_t kObjectSize = 2 * 1024 * 1024; + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = + PrepareSimpleSegment(*service_, "grouped_mixed_safety_segment", + kDefaultSegmentBase, kSegmentSize); + const UUID client_id = generate_uuid(); + + const std::string safe_key = "grouped_mixed_safe_key"; + const std::string hard_pinned_key = "grouped_mixed_hard_pinned_key"; + const std::string group_id = FindGroupIdOnDifferentShard(safe_key); + + ReplicateConfig safe_config; + safe_config.replica_num = 1; + safe_config.group_ids = std::vector{group_id}; + PutCompletedObject(*service_, client_id, safe_key, safe_config, + kObjectSize); + + ReplicateConfig hard_pinned_config = safe_config; + hard_pinned_config.with_hard_pin = true; + PutCompletedObject(*service_, client_id, hard_pinned_key, + hard_pinned_config, kObjectSize); + + ReplicateConfig trigger_config; + trigger_config.replica_num = 1; + auto trigger_result = + service_->PutStart(client_id, "trigger_mixed_safety_group_eviction", + kObjectSize, trigger_config); + ASSERT_FALSE(trigger_result.has_value()); + EXPECT_EQ(ErrorCode::NO_AVAILABLE_HANDLE, trigger_result.error()); + + std::this_thread::sleep_for(std::chrono::milliseconds(200)); + + EXPECT_FALSE(service_->ExistKey(safe_key).value_or(true)); + EXPECT_TRUE(service_->GetReplicaList(hard_pinned_key).has_value()); + EXPECT_TRUE(service_->Remove(hard_pinned_key, /*force=*/true).has_value()); +} + +TEST_F(MasterServiceTest, BatchUpsertStartMixedGroupIdsPreservesOrder) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::vector keys = { + "batch_grouped_a", + "batch_ungrouped", + "batch_grouped_b", + }; + const std::vector sizes = {1024, 2048, 4096}; + + ReplicateConfig config; + config.replica_num = 1; + config.group_ids = + std::vector{FindGroupIdOnDifferentShard(keys[0]), "", + FindGroupIdOnDifferentShard(keys[2])}; + + auto results = service_->BatchUpsertStart(client_id, keys, sizes, config); + ASSERT_EQ(results.size(), keys.size()); + for (const auto& result : results) { + ASSERT_TRUE(result.has_value()); + } + + auto end_results = service_->BatchUpsertEnd(client_id, keys); + ASSERT_EQ(end_results.size(), keys.size()); + for (const auto& result : end_results) { + ASSERT_TRUE(result.has_value()); + } + + for (const auto& key : keys) { + EXPECT_TRUE(service_->GetReplicaList(key).has_value()); + } + + ReplicateConfig invalid_config = config; + invalid_config.group_ids = std::vector{"only_one"}; + auto invalid_results = + service_->BatchUpsertStart(client_id, keys, sizes, invalid_config); + ASSERT_EQ(invalid_results.size(), keys.size()); + for (const auto& result : invalid_results) { + ASSERT_FALSE(result.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, result.error()); + } +} + +TEST_F(MasterServiceTest, WrappedBatchPutStartMixedGroupIdsPreservesOrder) { + WrappedMasterServiceConfig service_config; + service_config.default_kv_lease_ttl = 100; + service_config.enable_metric_reporting = false; + WrappedMasterService service_(service_config); + + Segment segment = MakeSegment("wrapped_batch_group_segment"); + const UUID client_id = generate_uuid(); + ASSERT_TRUE(service_.MountSegment(segment, client_id).has_value()); + + const std::vector keys = { + "wrapped_batch_grouped_a", + "wrapped_batch_ungrouped", + "wrapped_batch_grouped_b", + }; + const std::vector sizes = {1024, 2048, 4096}; + + ReplicateConfig config; + config.replica_num = 1; + config.group_ids = + std::vector{FindGroupIdOnDifferentShard(keys[0]), "", + FindGroupIdOnDifferentShard(keys[2])}; + + auto results = service_.BatchPutStart(client_id, keys, sizes, config); + ASSERT_EQ(results.size(), keys.size()); + for (const auto& result : results) { + ASSERT_TRUE(result.has_value()) << toString(result.error()); + } + + auto end_results = service_.BatchPutEnd(client_id, keys); + ASSERT_EQ(end_results.size(), keys.size()); + for (const auto& result : end_results) { + ASSERT_TRUE(result.has_value()); + } + + for (const auto& key : keys) { + EXPECT_TRUE(service_.GetReplicaList(key).has_value()); + } + + ReplicateConfig invalid_config = config; + invalid_config.group_ids = std::vector{"only_one"}; + auto invalid_group_results = + service_.BatchPutStart(client_id, keys, sizes, invalid_config); + ASSERT_EQ(invalid_group_results.size(), keys.size()); + for (const auto& result : invalid_group_results) { + ASSERT_FALSE(result.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, result.error()); + } + + auto invalid_size_results = + service_.BatchPutStart(client_id, keys, {1024}, config); + ASSERT_EQ(invalid_size_results.size(), keys.size()); + for (const auto& result : invalid_size_results) { + ASSERT_FALSE(result.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, result.error()); + } +} + TEST_F(MasterServiceTest, PutStartEndFlow) { std::unique_ptr service_(new MasterService()); [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); diff --git a/mooncake-wheel/tests/test_import_structure.py b/mooncake-wheel/tests/test_import_structure.py index 2dcd0c5b..80936c73 100644 --- a/mooncake-wheel/tests/test_import_structure.py +++ b/mooncake-wheel/tests/test_import_structure.py @@ -16,12 +16,18 @@ class TestImportStructure(unittest.TestCase): # Verify direct access to TransferOpcode self.assertIsNotNone(mooncake.engine.TransferOpcode) - from mooncake.store import MooncakeDistributedStore + from mooncake.store import MooncakeDistributedStore, ReplicateConfig # Just verify we can create instances store = MooncakeDistributedStore() + config = ReplicateConfig() + config.group_ids = ["group-a", ""] self.assertIsNotNone(store) + self.assertEqual(config.group_ids, ["group-a", ""]) + + config.group_ids = None + self.assertIsNone(config.group_ids) def test_direct_import(self): """Test direct import of specific components.""" -- 2.34.1 From eed58e89edd3bedad9e4893bd6e7488c7f76a158 Mon Sep 17 00:00:00 2001 From: Enigmo-x <1253115086@qq.com> Date: Wed, 27 May 2026 21:20:32 +0800 Subject: [PATCH 138/382] feat(store): add SPDK NoF worker pool (#2172) Signed-off-by: Enigmo Co-authored-by: Enigmo-x Co-authored-by: zwtao40 <1357420890@qq.com> --- dependencies.sh | 240 +++- mooncake-store/benchmarks/CMakeLists.txt | 9 +- .../benchmarks/nof_worker_pool_bench.cpp | 1019 +++++++++++++++++ mooncake-store/include/client_buffer.hpp | 5 +- mooncake-store/include/memory_alloc.h | 7 + mooncake-store/include/spdk/spdk_wrapper.h | 109 ++ mooncake-store/include/transfer_task.h | 180 ++- mooncake-store/include/utils.h | 3 +- mooncake-store/src/CMakeLists.txt | 84 +- mooncake-store/src/client_buffer.cpp | 22 +- mooncake-store/src/master_metric_manager.cpp | 30 +- mooncake-store/src/memory_alloc.cpp | 19 + mooncake-store/src/real_client.cpp | 26 +- mooncake-store/src/spdk/CMakeLists.txt | 5 + mooncake-store/src/spdk/spdk_wrapper.cpp | 490 ++++++++ mooncake-store/src/transfer_task.cpp | 519 ++++++++- mooncake-store/src/utils.cpp | 13 +- mooncake-store/tests/CMakeLists.txt | 3 + mooncake-store/tests/master_metrics_test.cpp | 22 +- mooncake-store/tests/nof_heartbeat_test.cpp | 249 ++++ mooncake-store/tests/transfer_task_test.cpp | 15 + 21 files changed, 2980 insertions(+), 89 deletions(-) create mode 100644 mooncake-store/benchmarks/nof_worker_pool_bench.cpp create mode 100644 mooncake-store/include/memory_alloc.h create mode 100644 mooncake-store/include/spdk/spdk_wrapper.h create mode 100644 mooncake-store/src/memory_alloc.cpp create mode 100644 mooncake-store/src/spdk/CMakeLists.txt create mode 100644 mooncake-store/src/spdk/spdk_wrapper.cpp create mode 100644 mooncake-store/tests/nof_heartbeat_test.cpp diff --git a/dependencies.sh b/dependencies.sh index 8ce80e55..8420bc32 100755 --- a/dependencies.sh +++ b/dependencies.sh @@ -48,23 +48,43 @@ check_success() { fi } +# Function to detect OS +detect_os() { + if [ -f /etc/os-release ]; then + . /etc/os-release + OS=$(echo "$ID" | tr '[:upper:]' '[:lower:]') + OS_VERSION=$VERSION_ID + elif [ -f /etc/redhat-release ]; then + OS="centos" + else + print_error "Cannot detect OS. Supported OS: Ubuntu, Debian, CentOS, RHEL, Rocky, AlmaLinux, and openEuler." + fi + + echo -e "${GREEN}Detected OS: $OS ${OS_VERSION:-unknown}${NC}" +} + if [ $(id -u) -ne 0 ]; then print_error "Require root permission, try sudo ./dependencies.sh" fi # Parse command line arguments SKIP_CONFIRM=false +INSTALL_SPDK=false for arg in "$@"; do case $arg in -y|--yes) SKIP_CONFIRM=true ;; + --with-spdk) + INSTALL_SPDK=true + ;; -h|--help) echo -e "${YELLOW}Mooncake Dependencies Installer${NC}" echo -e "Usage: ./dependencies.sh [OPTIONS]" echo -e "\nOptions:" - echo -e " -y, --yes Skip confirmation and install all dependencies" - echo -e " -h, --help Show this help message and exit" + echo -e " -y, --yes Skip confirmation and install all dependencies" + echo -e " --with-spdk Install SPDK for NVMe-oF support" + echo -e " -h, --help Show this help message and exit" exit 0 ;; esac @@ -77,6 +97,9 @@ echo -e "The following components will be installed:" echo -e " - System packages (build tools, libraries)" echo -e " - Git submodules (including pybind11 and yalantinglibs)" echo -e " - Go $GOVER" +if [ "$INSTALL_SPDK" = true ]; then + echo -e " - SPDK (for NVMe-oF support)" +fi echo # Ask for confirmation unless -y flag is used @@ -89,51 +112,97 @@ if [ "$SKIP_CONFIRM" = false ]; then fi fi +# Detect OS +detect_os # Update package lists print_section "Updating package lists" -apt-get update -check_success "Failed to update package lists" +if [ "$OS" = "ubuntu" ] || [ "$OS" = "debian" ]; then + apt-get update + check_success "Failed to update package lists" +elif [ "$OS" = "centos" ] || [ "$OS" = "rhel" ] || [ "$OS" = "rocky" ] || [ "$OS" = "almalinux" ] || [ "$OS" = "openeuler" ]; then + yum clean all + yum makecache + check_success "Failed to update package lists" +else + print_error "Unsupported OS: $OS" +fi # Install system packages print_section "Installing system packages" echo -e "${YELLOW}This may take a few minutes...${NC}" -SYSTEM_PACKAGES="build-essential \ - cmake \ - ninja-build \ - git \ - wget \ - unzip \ - libibverbs-dev \ - libgoogle-glog-dev \ - libgtest-dev \ - libjsoncpp-dev \ - libunwind-dev \ - libnuma-dev \ - libpython3-dev \ - libboost-dev \ - libssl-dev \ - libgrpc-dev \ - libgrpc++-dev \ - libprotobuf-dev \ - libyaml-cpp-dev \ - protobuf-compiler-grpc \ - libcurl4-openssl-dev \ - libhiredis-dev \ - liburing-dev \ - libjemalloc-dev \ - libmsgpack-dev \ - libzstd-dev \ - libasio-dev \ - libxxhash-dev \ - pkg-config \ - patchelf \ - libc6-dev \ - libc-bin" +if [ "$OS" = "ubuntu" ] || [ "$OS" = "debian" ]; then + SYSTEM_PACKAGES="build-essential \ + cmake \ + ninja-build \ + git \ + wget \ + unzip \ + libibverbs-dev \ + libgoogle-glog-dev \ + libgtest-dev \ + libjsoncpp-dev \ + libunwind-dev \ + libnuma-dev \ + libpython3-dev \ + libboost-all-dev \ + libssl-dev \ + libgrpc-dev \ + libgrpc++-dev \ + libprotobuf-dev \ + libyaml-cpp-dev \ + protobuf-compiler-grpc \ + libcurl4-openssl-dev \ + libhiredis-dev \ + liburing-dev \ + libjemalloc-dev \ + libmsgpack-dev \ + libzstd-dev \ + libasio-dev \ + libxxhash-dev \ + pkg-config \ + patchelf \ + libc6-dev \ + libc-bin" + + apt-get install -y $SYSTEM_PACKAGES + check_success "Failed to install system packages" + +elif [ "$OS" = "centos" ] || [ "$OS" = "rhel" ] || [ "$OS" = "rocky" ] || [ "$OS" = "almalinux" ] || [ "$OS" = "openeuler" ]; then + SYSTEM_PACKAGES="@development \ + cmake \ + git \ + wget \ + rdma-core-devel \ + glog-devel \ + gtest-devel \ + jsoncpp-devel \ + libunwind-devel \ + numactl-devel \ + python3-devel \ + boost-devel \ + openssl-devel \ + grpc-devel \ + protobuf-devel \ + yaml-cpp-devel \ + grpc-plugins \ + libcurl-devel \ + hiredis-devel \ + liburing-devel \ + jemalloc-devel \ + pkgconf-pkg-config \ + elfutils-libelf-devel \ + patchelf \ + xxhash-devel \ + libbsd-devel" + + yum install -y $SYSTEM_PACKAGES + check_success "Failed to install system packages" +else + print_error "Unsupported OS: $OS" +fi -apt-get install -y $SYSTEM_PACKAGES -check_success "Failed to install system packages" print_success "System packages installed successfully" # Initialize and update git submodules @@ -185,15 +254,16 @@ cd "${REPO_ROOT}" print_section "Verifying essential build tools" # Verify getconf and ldd (required for glibc version detection in build_wheel.sh) -# Both are provided by libc-bin, which is included in SYSTEM_PACKAGES -if ! command -v getconf >/dev/null 2>&1; then - print_error "getconf not found after installing system packages. This should not happen." +if [ "$OS" = "ubuntu" ] || [ "$OS" = "debian" ]; then + if ! command -v getconf >/dev/null 2>&1; then + print_error "getconf not found after installing system packages. This should not happen." + fi + if ! command -v ldd >/dev/null 2>&1; then + print_error "ldd not found after installing system packages. This should not happen." + fi + print_success "getconf found: $(getconf --version 2>&1 | head -1)" + print_success "ldd found: $(ldd --version 2>&1 | head -1)" fi -if ! command -v ldd >/dev/null 2>&1; then - print_error "ldd not found after installing system packages. This should not happen." -fi -print_success "getconf found: $(getconf --version 2>&1 | head -1)" -print_success "ldd found: $(ldd --version 2>&1 | head -1)" print_section "Installing Go $GOVER" @@ -224,8 +294,6 @@ install_go() { echo "Downloading Go $GOVER from ${url}..." if wget -q --show-progress --timeout=30 --tries=2 -O "${GO_TARBALL}" "${url}"; then DOWNLOAD_SUCCESS=true - # If the official source (go.dev) failed and we fell back to a CN mirror, - # it likely means the network has restricted access to international sites. if [[ "$url" != "https://go.dev/dl/${GO_TARBALL}" ]]; then USED_CN_MIRROR=true fi @@ -241,19 +309,16 @@ install_go() { print_error "Failed to download Go $GOVER from all mirrors" fi - # Install Go echo "Installing Go $GOVER..." tar -C /usr/local -xzf "${GO_TARBALL}" check_success "Failed to install Go $GOVER" - # Clean up downloaded file rm -f "${GO_TARBALL}" check_success "Failed to clean up Go installation file" print_success "Go $GOVER installed successfully" } -# Check if Go is already installed if command -v go &> /dev/null; then GO_VERSION=$(go version | awk '{print $3}') if [[ "$GO_VERSION" == "go$GOVER" ]]; then @@ -273,8 +338,7 @@ if ! grep -q "export PATH=\$PATH:/usr/local/go/bin" ~/.bashrc; then echo -e "${YELLOW}Please run 'source ~/.bashrc' or start a new terminal to use Go${NC}" fi -# Set GOPROXY only if Go download fell back to a CN mirror, indicating restricted -# network access to international sites. Skip if user already configured GOPROXY. +# Set GOPROXY only if Go download fell back to a CN mirror if [ "$USED_CN_MIRROR" = true ] && [ -z "$GOPROXY" ]; then export GOPROXY=https://goproxy.cn,https://goproxy.io,direct echo -e "${YELLOW}Detected restricted network (Go was downloaded from a CN mirror).${NC}" @@ -287,6 +351,69 @@ elif [ -n "$GOPROXY" ]; then echo -e "${GREEN}GOPROXY already set to: ${GOPROXY}${NC}" fi +# Install SPDK if requested +if [ "$INSTALL_SPDK" = true ]; then + print_section "Installing SPDK" + + cd "${REPO_ROOT}/extern" + check_success "Failed to change to extern directory" + + # Remove existing SPDK if present + if [ -d "spdk" ]; then + echo -e "${YELLOW}SPDK directory already exists. Removing for fresh install...${NC}" + rm -rf spdk + check_success "Failed to remove existing SPDK directory" + fi + + # Clone SPDK + echo "Cloning SPDK from ${GITHUB_PROXY}/spdk/spdk.git..." + git clone ${GITHUB_PROXY}/spdk/spdk.git + check_success "Failed to clone SPDK" + + cd spdk + check_success "Failed to change to SPDK directory" + + # Checkout specific version + echo "Checking out SPDK version v23.01.1..." + git checkout v23.01.1 + check_success "Failed to checkout SPDK version v23.01.1" + + # Initialize submodules + echo "Initializing SPDK submodules..." + git submodule update --init + check_success "Failed to initialize SPDK submodules" + + # Install SPDK dependencies + echo "Installing SPDK dependencies..." + ./scripts/pkgdep.sh + check_success "Failed to install SPDK dependencies" + + # Configure SPDK with RDMA support + echo "Configuring SPDK with RDMA support..." + ./configure --with-rdma + check_success "Failed to configure SPDK" + + # Build SPDK + echo "Building SPDK (using $(nproc) cores)..." + make -j$(nproc) + check_success "Failed to build SPDK" + + # Install SPDK + echo "Installing SPDK..." + make install + check_success "Failed to install SPDK" + + # Copy DPDK libraries to system library path + if ls dpdk/build/lib/*.a >/dev/null 2>&1; then + echo "Copying DPDK libraries to /usr/local/lib..." + cp dpdk/build/lib/*.a /usr/local/lib/ + check_success "Failed to copy DPDK libraries" + fi + + print_success "SPDK installed successfully" + cd "${REPO_ROOT}" +fi + # Return to the repository root cd "${REPO_ROOT}" @@ -298,6 +425,13 @@ echo -e " ${GREEN}✓${NC} System packages" echo -e " ${GREEN}✓${NC} yalantinglibs" echo -e " ${GREEN}✓${NC} Git submodules" echo -e " ${GREEN}✓${NC} Go $GOVER" +if [ "$INSTALL_SPDK" = true ]; then + echo -e " ${GREEN}✓${NC} SPDK (v23.01.1)" +fi echo echo -e "You can now build and run Mooncake." echo -e "${YELLOW}Note: You may need to restart your terminal or run 'source ~/.bashrc' to use Go.${NC}" + +if [ "$INSTALL_SPDK" = true ]; then + echo -e "${YELLOW}Note: SPDK requires hugepages and RDMA configuration. Please refer to SPDK documentation for setup.${NC}" +fi diff --git a/mooncake-store/benchmarks/CMakeLists.txt b/mooncake-store/benchmarks/CMakeLists.txt index 8798af89..a9f62058 100644 --- a/mooncake-store/benchmarks/CMakeLists.txt +++ b/mooncake-store/benchmarks/CMakeLists.txt @@ -21,9 +21,16 @@ target_link_libraries( gflags::gflags glog::glog pthread) # Add allocation strategy benchmark executable -# This benchmark tests AllocationStrategy performance with configurable +# This benchmark tests AllocationStrategy performance with configurable # segment counts, allocation sizes, replica counts, and workload patterns add_executable(allocation_strategy_bench allocation_strategy_bench.cpp) target_link_libraries( allocation_strategy_bench PRIVATE mooncake_store cachelib_memory_allocator gflags::gflags glog::glog pthread) + +# Add NoF worker pool benchmark executable +if(USE_NOF) + add_executable(nof_worker_pool_bench nof_worker_pool_bench.cpp) + target_link_libraries(nof_worker_pool_bench PRIVATE mooncake_store glog::glog + gflags::gflags) +endif() diff --git a/mooncake-store/benchmarks/nof_worker_pool_bench.cpp b/mooncake-store/benchmarks/nof_worker_pool_bench.cpp new file mode 100644 index 00000000..ed3d011e --- /dev/null +++ b/mooncake-store/benchmarks/nof_worker_pool_bench.cpp @@ -0,0 +1,1019 @@ +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "spdk/spdk_wrapper.h" +#include "transfer_task.h" +#include "utils.h" + +namespace { + +constexpr uint64_t KiB = 1024; +constexpr uint64_t MiB = 1024 * KiB; +constexpr uint64_t GiB = 1024 * MiB; +constexpr uint64_t kDefaultIoSize = 128 * KiB; +constexpr uint64_t kDefaultIodepth = 64; +constexpr uint64_t kDefaultSubmitThreads = 1; +constexpr uint64_t kDefaultDurationSec = 30; +constexpr uint64_t kDefaultReportIntervalMs = 1000; +constexpr uint64_t kDefaultRangeBytes = 1 * GiB; +constexpr uint64_t kDefaultWarmupSec = 3; +constexpr char kDefaultOp[] = "read"; + +DEFINE_string( + endpoints, "", + "Comma-separated NoF transport endpoints. To use multiple " + "SpdkNofWorkerPool workers with the current implementation, provide " + "multiple distinct endpoints/namespaces/targets."); +DEFINE_string(op, "read", "Benchmark operation: read, write, or mixed."); +DEFINE_string( + rw, "", + "fio-style alias for --op/--random_lba. Supported values: read, write, " + "randread, randwrite, mixed, randrw."); +DEFINE_uint64(io_size, 128 * KiB, + "I/O size in bytes. Must align to device block size."); +DEFINE_uint64(bs, 0, "fio-style alias for --io_size."); +DEFINE_uint64( + iodepth, 64, + "Per-endpoint in-flight I/Os maintained by the benchmark. Total in-flight " + "depth is endpoints * iodepth."); +DEFINE_uint64( + submit_threads, 1, + "Number of benchmark submit/completion threads. Endpoints are evenly " + "partitioned across submit threads."); +DEFINE_uint64(numjobs, 0, "fio-style alias for --submit_threads."); +DEFINE_uint64(duration_sec, 30, "Measurement duration in seconds."); +DEFINE_uint64(runtime, 0, "fio-style alias for --duration_sec."); +DEFINE_uint64(report_interval_ms, 1000, + "Periodic throughput report interval in milliseconds."); +DEFINE_uint64(status_interval, 0, + "fio-style alias for --report_interval_ms, in seconds."); +DEFINE_uint64(start_lba, 0, "Starting LBA offset within each endpoint."); +DEFINE_uint64(range_bytes, 1 * GiB, + "Per-endpoint logical range in bytes used by the benchmark. " + "Must align to block size."); +DEFINE_uint64(size, 0, "fio-style alias for --range_bytes."); +DEFINE_bool(random_lba, false, + "Whether to choose LBA randomly inside the per-endpoint range."); +DEFINE_uint64(seed, 1, "Random seed used when --random_lba or --op=mixed."); +DEFINE_uint64(warmup_sec, 3, + "Warmup duration in seconds before measurement starts."); +DEFINE_uint64(ramp_time, 0, "fio-style alias for --warmup_sec."); +DEFINE_uint64( + nof_workers, 0, + "Optional override for MC_NOF_WORKERS. Equivalent to exporting " + "MC_NOF_WORKERS before launch. 0 means keep environment/default."); +DEFINE_uint64(nof_submit_chunk_bytes, 0, + "Optional override for MC_NOF_SUBMIT_CHUNK_BYTES. Equivalent to " + "exporting MC_NOF_SUBMIT_CHUNK_BYTES before launch. 0 means keep " + "environment/default."); +DEFINE_uint64( + nof_inflight_bytes_limit, 0, + "Optional override for MC_NOF_INFLIGHT_BYTES_LIMIT. Equivalent to " + "exporting MC_NOF_INFLIGHT_BYTES_LIMIT before launch. 0 means keep " + "environment/default."); +DEFINE_int32(socket_id, -1, + "NUMA socket for benchmark buffers. -1 lets SPDK choose."); +DEFINE_bool(fill_on_write, true, + "Fill buffers with a deterministic pattern for write/mixed ops."); + +using Clock = std::chrono::steady_clock; +using TimePoint = Clock::time_point; + +enum class BenchOp { + READ = 0, + WRITE = 1, + MIXED = 2, +}; + +BenchOp ParseBenchOp(const std::string &value) { + std::string normalized(value); + std::transform( + normalized.begin(), normalized.end(), normalized.begin(), + [](unsigned char c) { return static_cast(std::tolower(c)); }); + if (normalized == "read") { + return BenchOp::READ; + } + if (normalized == "write") { + return BenchOp::WRITE; + } + if (normalized == "mixed") { + return BenchOp::MIXED; + } + throw std::invalid_argument( + "Invalid --op. Supported values: read, write, mixed"); +} + +struct ParsedRw { + std::string op; + bool random_lba{false}; +}; + +ParsedRw ParseRwMode(const std::string &value) { + std::string normalized(value); + std::transform( + normalized.begin(), normalized.end(), normalized.begin(), + [](unsigned char c) { return static_cast(std::tolower(c)); }); + if (normalized == "read") { + return {"read", false}; + } + if (normalized == "write") { + return {"write", false}; + } + if (normalized == "mixed" || normalized == "rw") { + return {"mixed", false}; + } + if (normalized == "randread") { + return {"read", true}; + } + if (normalized == "randwrite") { + return {"write", true}; + } + if (normalized == "randrw") { + return {"mixed", true}; + } + throw std::invalid_argument( + "Invalid --rw. Supported values: read, write, randread, randwrite, " + "mixed, rw, randrw"); +} + +void ResolveUint64Alias(const char *legacy_name, uint64_t *legacy_value, + uint64_t legacy_default, const char *alias_name, + uint64_t alias_value) { + if (alias_value == 0) { + return; + } + if (*legacy_value != legacy_default && *legacy_value != alias_value) { + throw std::invalid_argument(std::string("Conflicting --") + + legacy_name + " and --" + alias_name + + " values"); + } + *legacy_value = alias_value; +} + +void SetEnvU64IfRequested(const char *name, uint64_t value) { + if (value == 0) { + return; + } +#ifdef _WIN32 + _putenv_s(name, std::to_string(value).c_str()); +#else + setenv(name, std::to_string(value).c_str(), 1); +#endif +} + +std::string FormatBytesPerSecond(double bytes_per_sec) { + std::ostringstream oss; + oss << std::fixed << std::setprecision(2); + if (bytes_per_sec >= static_cast(GiB)) { + oss << (bytes_per_sec / GiB) << " GiB/s"; + } else if (bytes_per_sec >= static_cast(MiB)) { + oss << (bytes_per_sec / MiB) << " MiB/s"; + } else if (bytes_per_sec >= static_cast(KiB)) { + oss << (bytes_per_sec / KiB) << " KiB/s"; + } else { + oss.unsetf(std::ios::fixed); + oss << bytes_per_sec << " B/s"; + } + return oss.str(); +} + +std::string FormatLatencyNs(uint64_t latency_ns) { + std::ostringstream oss; + oss << std::fixed << std::setprecision(2); + if (latency_ns >= 1000ULL * 1000ULL * 1000ULL) { + oss << (static_cast(latency_ns) / 1000.0 / 1000.0 / 1000.0) + << " s"; + } else if (latency_ns >= 1000ULL * 1000ULL) { + oss << (static_cast(latency_ns) / 1000.0 / 1000.0) << " ms"; + } else if (latency_ns >= 1000ULL) { + oss << (static_cast(latency_ns) / 1000.0) << " us"; + } else { + oss.unsetf(std::ios::fixed); + oss << latency_ns << " ns"; + } + return oss.str(); +} + +struct CounterSnapshot { + uint64_t submitted_ops{0}; + uint64_t completed_ops{0}; + uint64_t failed_ops{0}; + uint64_t bytes{0}; +}; + +struct LiveCounters { + std::atomic submitted_ops{0}; + std::atomic completed_ops{0}; + std::atomic failed_ops{0}; + std::atomic bytes{0}; + + CounterSnapshot Snapshot() const { + return { + submitted_ops.load(std::memory_order_relaxed), + completed_ops.load(std::memory_order_relaxed), + failed_ops.load(std::memory_order_relaxed), + bytes.load(std::memory_order_relaxed), + }; + } +}; + +class LatencyHistogram { + public: + static constexpr size_t kBuckets0 = 200; + static constexpr size_t kBuckets1 = 180; + static constexpr size_t kBuckets2 = 180; + static constexpr size_t kBuckets3 = 180; + static constexpr size_t kBucketCount = + kBuckets0 + kBuckets1 + kBuckets2 + kBuckets3 + 1; + + void Record(uint64_t latency_ns); + void MergeFrom(const LatencyHistogram &other); + uint64_t Count() const { return count_; } + uint64_t SumNs() const { return sum_ns_; } + uint64_t PercentileNs(double percentile) const; + + private: + static size_t BucketIndex(uint64_t latency_ns); + static uint64_t BucketUpperBoundNs(size_t index); + + uint64_t count_{0}; + uint64_t sum_ns_{0}; + std::array buckets_{}; +}; + +struct BenchStats { + uint64_t submitted_ops{0}; + uint64_t completed_ops{0}; + uint64_t failed_ops{0}; + uint64_t bytes{0}; + LatencyHistogram latency_histogram; + + void RecordSubmit(); + void RecordSuccess(uint64_t io_bytes, uint64_t latency_ns); + void RecordFailure(); + void MergeFrom(const BenchStats &other); +}; + +struct EndpointContext { + std::string endpoint; + mooncake::nof_seg_handle *seg_handle{nullptr}; + uint32_t block_size{0}; + uint64_t range_blocks{0}; + uint64_t io_blocks{0}; + uint64_t next_block{0}; + std::mt19937_64 rng; + + EndpointContext(std::string ep, uint64_t seed_value) + : endpoint(std::move(ep)), rng(seed_value) {} +}; + +struct EndpointStats { + BenchStats measured; + LiveCounters live; +}; + +struct Slot { + void *buffer{nullptr}; + size_t buffer_size{0}; + std::shared_ptr state; + std::unique_ptr future; + uint64_t io_bytes{0}; + size_t endpoint_index{0}; + bool active{false}; + bool measure{false}; + TimePoint submit_time{}; +}; + +struct ThreadEndpointState { + size_t endpoint_index{0}; + std::vector slots; +}; + +struct BenchThreadContext { + size_t thread_index{0}; + std::vector endpoints; +}; + +struct ThroughputView { + double bandwidth_bytes_per_sec{0.0}; + double iops{0.0}; + double fail_iops{0.0}; +}; + +struct LatencyView { + uint64_t avg_ns{0}; + uint64_t p50_ns{0}; + uint64_t p95_ns{0}; + uint64_t p99_ns{0}; +}; + +ThroughputView ComputeThroughput(const CounterSnapshot &counters, + double duration_sec); +LatencyView ComputeLatency(const BenchStats &stats); +CounterSnapshot DeltaCounters(const CounterSnapshot ¤t, + const CounterSnapshot &previous); +uint64_t NextLba(EndpointContext &endpoint, bool random_lba); +int PickTaskOp(BenchOp op_mode, std::mt19937_64 &rng); +void FillPattern(void *buffer, size_t size, uint64_t seq); +void PublishSubmit(LiveCounters &live); +void PublishSuccess(LiveCounters &live, uint64_t io_bytes); +void PublishFailure(LiveCounters &live); +void SubmitSlot(Slot &slot, EndpointContext &endpoint, size_t endpoint_index, + mooncake::SpdkNofWorkerPool &pool, BenchOp op_mode, + uint64_t submit_seq, bool measure_now, + EndpointStats &endpoint_stats); +void WorkerLoop(BenchThreadContext *thread_context, + std::vector *endpoints, + std::vector *endpoint_stats, + mooncake::SpdkNofWorkerPool *pool, BenchOp op_mode, + TimePoint warmup_end, TimePoint end, + std::atomic *active_threads); +void FreeThreadSlots(mooncake::SpdkWrapper &wrapper, + std::vector &thread_contexts); + +void LatencyHistogram::Record(uint64_t latency_ns) { + ++count_; + sum_ns_ += latency_ns; + ++buckets_[BucketIndex(latency_ns)]; +} + +void LatencyHistogram::MergeFrom(const LatencyHistogram &other) { + count_ += other.count_; + sum_ns_ += other.sum_ns_; + for (size_t i = 0; i < kBucketCount; ++i) { + buckets_[i] += other.buckets_[i]; + } +} + +uint64_t LatencyHistogram::PercentileNs(double percentile) const { + if (count_ == 0) { + return 0; + } + if (percentile <= 0.0) { + return BucketUpperBoundNs(0); + } + if (percentile >= 1.0) { + return BucketUpperBoundNs(kBucketCount - 1); + } + + const uint64_t target = + static_cast(std::ceil(percentile * count_)); + uint64_t seen = 0; + for (size_t i = 0; i < kBucketCount; ++i) { + seen += buckets_[i]; + if (seen >= target) { + return BucketUpperBoundNs(i); + } + } + return BucketUpperBoundNs(kBucketCount - 1); +} + +size_t LatencyHistogram::BucketIndex(uint64_t latency_ns) { + if (latency_ns < 1'000'000ULL) { + return static_cast(latency_ns / 5'000ULL); + } + if (latency_ns < 10'000'000ULL) { + return kBuckets0 + + static_cast((latency_ns - 1'000'000ULL) / 50'000ULL); + } + if (latency_ns < 100'000'000ULL) { + return kBuckets0 + kBuckets1 + + static_cast((latency_ns - 10'000'000ULL) / 500'000ULL); + } + if (latency_ns < 1'000'000'000ULL) { + return kBuckets0 + kBuckets1 + kBuckets2 + + static_cast((latency_ns - 100'000'000ULL) / + 5'000'000ULL); + } + return kBucketCount - 1; +} + +uint64_t LatencyHistogram::BucketUpperBoundNs(size_t index) { + if (index < kBuckets0) { + return (index + 1) * 5'000ULL; + } + index -= kBuckets0; + if (index < kBuckets1) { + return 1'000'000ULL + (index + 1) * 50'000ULL; + } + index -= kBuckets1; + if (index < kBuckets2) { + return 10'000'000ULL + (index + 1) * 500'000ULL; + } + index -= kBuckets2; + if (index < kBuckets3) { + return 100'000'000ULL + (index + 1) * 5'000'000ULL; + } + return std::numeric_limits::max(); +} + +void BenchStats::RecordSubmit() { ++submitted_ops; } + +void BenchStats::RecordSuccess(uint64_t io_bytes, uint64_t latency_ns) { + ++completed_ops; + bytes += io_bytes; + latency_histogram.Record(latency_ns); +} + +void BenchStats::RecordFailure() { ++failed_ops; } + +void BenchStats::MergeFrom(const BenchStats &other) { + submitted_ops += other.submitted_ops; + completed_ops += other.completed_ops; + failed_ops += other.failed_ops; + bytes += other.bytes; + latency_histogram.MergeFrom(other.latency_histogram); +} + +ThroughputView ComputeThroughput(const CounterSnapshot &counters, + double duration_sec) { + ThroughputView view; + if (duration_sec <= 0.0) { + return view; + } + view.bandwidth_bytes_per_sec = counters.bytes / duration_sec; + view.iops = counters.completed_ops / duration_sec; + view.fail_iops = counters.failed_ops / duration_sec; + return view; +} + +LatencyView ComputeLatency(const BenchStats &stats) { + LatencyView view; + const uint64_t count = stats.latency_histogram.Count(); + if (count == 0) { + return view; + } + view.avg_ns = stats.latency_histogram.SumNs() / count; + view.p50_ns = stats.latency_histogram.PercentileNs(0.50); + view.p95_ns = stats.latency_histogram.PercentileNs(0.95); + view.p99_ns = stats.latency_histogram.PercentileNs(0.99); + return view; +} + +CounterSnapshot DeltaCounters(const CounterSnapshot ¤t, + const CounterSnapshot &previous) { + return { + current.submitted_ops - previous.submitted_ops, + current.completed_ops - previous.completed_ops, + current.failed_ops - previous.failed_ops, + current.bytes - previous.bytes, + }; +} + +uint64_t NextLba(EndpointContext &endpoint, bool random_lba) { + if (endpoint.range_blocks <= endpoint.io_blocks) { + return FLAGS_start_lba; + } + + if (random_lba) { + std::uniform_int_distribution dist( + 0, endpoint.range_blocks - endpoint.io_blocks); + return FLAGS_start_lba + dist(endpoint.rng); + } + + uint64_t lba = FLAGS_start_lba + endpoint.next_block; + endpoint.next_block += endpoint.io_blocks; + if (endpoint.next_block + endpoint.io_blocks > endpoint.range_blocks) { + endpoint.next_block = 0; + } + return lba; +} + +int PickTaskOp(BenchOp op_mode, std::mt19937_64 &rng) { + switch (op_mode) { + case BenchOp::READ: + return 0; + case BenchOp::WRITE: + return 1; + case BenchOp::MIXED: { + std::uniform_int_distribution dist(0, 1); + return dist(rng); + } + } + return 0; +} + +void FillPattern(void *buffer, size_t size, uint64_t seq) { + auto *ptr = reinterpret_cast(buffer); + for (size_t i = 0; i < size; ++i) { + ptr[i] = static_cast((seq + i) & 0xff); + } +} + +void PublishSubmit(LiveCounters &live) { + live.submitted_ops.fetch_add(1, std::memory_order_relaxed); +} + +void PublishSuccess(LiveCounters &live, uint64_t io_bytes) { + live.completed_ops.fetch_add(1, std::memory_order_relaxed); + live.bytes.fetch_add(io_bytes, std::memory_order_relaxed); +} + +void PublishFailure(LiveCounters &live) { + live.failed_ops.fetch_add(1, std::memory_order_relaxed); +} + +void SubmitSlot(Slot &slot, EndpointContext &endpoint, size_t endpoint_index, + mooncake::SpdkNofWorkerPool &pool, BenchOp op_mode, + uint64_t submit_seq, bool measure_now, + EndpointStats &endpoint_stats) { + int op = PickTaskOp(op_mode, endpoint.rng); + if (op == 1 && FLAGS_fill_on_write) { + FillPattern(slot.buffer, slot.buffer_size, submit_seq); + } + + uint64_t lba = NextLba(endpoint, FLAGS_random_lba); + slot.io_bytes = slot.buffer_size; + slot.endpoint_index = endpoint_index; + slot.state = std::make_shared(); + slot.future = std::make_unique(slot.state); + slot.measure = measure_now; + slot.submit_time = Clock::now(); + + mooncake::SpdkNofTask task(endpoint.seg_handle, slot.buffer, lba, + static_cast(endpoint.io_blocks), op, + slot.state); + pool.submitTask(std::move(task)); + slot.active = true; + + if (measure_now) { + endpoint_stats.measured.RecordSubmit(); + PublishSubmit(endpoint_stats.live); + } +} + +void WorkerLoop(BenchThreadContext *thread_context, + std::vector *endpoints, + std::vector *endpoint_stats, + mooncake::SpdkNofWorkerPool *pool, BenchOp op_mode, + TimePoint warmup_end, TimePoint end, + std::atomic *active_threads) { + uint64_t submit_seq = static_cast(thread_context->thread_index) + << 48; + + for (auto &thread_endpoint : thread_context->endpoints) { + auto &endpoint = (*endpoints)[thread_endpoint.endpoint_index]; + auto &stats = (*endpoint_stats)[thread_endpoint.endpoint_index]; + for (auto &slot : thread_endpoint.slots) { + SubmitSlot(slot, endpoint, thread_endpoint.endpoint_index, *pool, + op_mode, submit_seq++, false, stats); + } + } + + while (true) { + const TimePoint now_before_scan = Clock::now(); + const bool before_end = now_before_scan < end; + bool any_active = false; + + for (auto &thread_endpoint : thread_context->endpoints) { + auto &endpoint = (*endpoints)[thread_endpoint.endpoint_index]; + auto &stats = (*endpoint_stats)[thread_endpoint.endpoint_index]; + for (auto &slot : thread_endpoint.slots) { + if (!slot.active) { + continue; + } + any_active = true; + if (!slot.future || !slot.future->isReady()) { + continue; + } + + const TimePoint completion_time = Clock::now(); + mooncake::ErrorCode result = slot.future->get(); + slot.active = false; + slot.future.reset(); + + if (slot.measure) { + if (result == mooncake::ErrorCode::OK) { + const uint64_t latency_ns = static_cast( + std::chrono::duration_cast< + std::chrono::nanoseconds>(completion_time - + slot.submit_time) + .count()); + stats.measured.RecordSuccess(slot.io_bytes, latency_ns); + PublishSuccess(stats.live, slot.io_bytes); + } else { + stats.measured.RecordFailure(); + PublishFailure(stats.live); + } + } + + if (before_end) { + const bool measure_now = completion_time >= warmup_end; + SubmitSlot(slot, endpoint, thread_endpoint.endpoint_index, + *pool, op_mode, submit_seq++, measure_now, + stats); + any_active = true; + } + } + } + + if (!before_end && !any_active) { + break; + } + + std::this_thread::sleep_for(std::chrono::microseconds(50)); + } + + active_threads->fetch_sub(1, std::memory_order_relaxed); +} + +void FreeThreadSlots(mooncake::SpdkWrapper &wrapper, + std::vector &thread_contexts) { + for (auto &thread_context : thread_contexts) { + for (auto &thread_endpoint : thread_context.endpoints) { + for (auto &slot : thread_endpoint.slots) { + if (slot.buffer) { + wrapper.Free(slot.buffer); + slot.buffer = nullptr; + } + } + } + } +} + +} // namespace + +int main(int argc, char **argv) { + google::InitGoogleLogging(argv[0]); + FLAGS_logtostderr = true; + gflags::SetUsageMessage( + "NoF worker pool benchmark. Use --helpshort to list benchmark " + "parameters."); + gflags::ParseCommandLineFlags(&argc, &argv, true); + + try { + ResolveUint64Alias("io_size", &FLAGS_io_size, kDefaultIoSize, "bs", + FLAGS_bs); + ResolveUint64Alias("submit_threads", &FLAGS_submit_threads, + kDefaultSubmitThreads, "numjobs", FLAGS_numjobs); + ResolveUint64Alias("duration_sec", &FLAGS_duration_sec, + kDefaultDurationSec, "runtime", FLAGS_runtime); + ResolveUint64Alias("range_bytes", &FLAGS_range_bytes, + kDefaultRangeBytes, "size", FLAGS_size); + ResolveUint64Alias("warmup_sec", &FLAGS_warmup_sec, kDefaultWarmupSec, + "ramp_time", FLAGS_ramp_time); + if (FLAGS_status_interval != 0) { + ResolveUint64Alias("report_interval_ms", &FLAGS_report_interval_ms, + kDefaultReportIntervalMs, "status_interval", + FLAGS_status_interval * 1000); + } + if (!FLAGS_rw.empty()) { + ParsedRw parsed_rw = ParseRwMode(FLAGS_rw); + if (FLAGS_op != kDefaultOp && FLAGS_op != parsed_rw.op) { + throw std::invalid_argument("Conflicting --op and --rw values"); + } + if (parsed_rw.random_lba == false && FLAGS_random_lba) { + throw std::invalid_argument( + "Conflicting --random_lba and --rw values"); + } + FLAGS_op = parsed_rw.op; + FLAGS_random_lba = parsed_rw.random_lba; + } + + BenchOp op_mode = ParseBenchOp(FLAGS_op); + if (FLAGS_endpoints.empty()) { + LOG(ERROR) << "--endpoints is required"; + return 1; + } + if (FLAGS_iodepth == 0) { + LOG(ERROR) << "--iodepth must be greater than 0"; + return 1; + } + if (FLAGS_submit_threads == 0) { + LOG(ERROR) << "--submit_threads must be greater than 0"; + return 1; + } + if (FLAGS_duration_sec == 0) { + LOG(ERROR) << "--duration_sec must be greater than 0"; + return 1; + } + + SetEnvU64IfRequested("MC_NOF_WORKERS", FLAGS_nof_workers); + SetEnvU64IfRequested("MC_NOF_SUBMIT_CHUNK_BYTES", + FLAGS_nof_submit_chunk_bytes); + SetEnvU64IfRequested("MC_NOF_INFLIGHT_BYTES_LIMIT", + FLAGS_nof_inflight_bytes_limit); + + auto endpoint_strings = mooncake::splitString(FLAGS_endpoints); + if (endpoint_strings.empty()) { + LOG(ERROR) << "No valid endpoints parsed from --endpoints"; + return 1; + } + + auto &wrapper = mooncake::SpdkWrapper::GetInstance(); + if (!wrapper.InitializeEnv()) { + LOG(ERROR) << "Failed to initialize SPDK environment"; + return 1; + } + + std::vector endpoints; + endpoints.reserve(endpoint_strings.size()); + std::unordered_set unique_handles; + for (size_t i = 0; i < endpoint_strings.size(); ++i) { + EndpointContext endpoint(endpoint_strings[i], FLAGS_seed + i); + endpoint.seg_handle = wrapper.OpenNofSegment(endpoint.endpoint); + if (!endpoint.seg_handle) { + LOG(ERROR) << "Failed to open NoF endpoint: " + << endpoint.endpoint; + return 1; + } + endpoint.block_size = wrapper.GetBlockSize(endpoint.seg_handle); + if (endpoint.block_size == INVALID_BLOCK_SIZE || + endpoint.block_size == 0) { + LOG(ERROR) << "Invalid block size for endpoint: " + << endpoint.endpoint; + return 1; + } + if (FLAGS_io_size % endpoint.block_size != 0) { + LOG(ERROR) << "--io_size=" << FLAGS_io_size + << " is not aligned to block size " + << endpoint.block_size << " for endpoint " + << endpoint.endpoint; + return 1; + } + if (FLAGS_range_bytes % endpoint.block_size != 0) { + LOG(ERROR) << "--range_bytes=" << FLAGS_range_bytes + << " is not aligned to block size " + << endpoint.block_size; + return 1; + } + endpoint.io_blocks = FLAGS_io_size / endpoint.block_size; + endpoint.range_blocks = FLAGS_range_bytes / endpoint.block_size; + if (endpoint.range_blocks < endpoint.io_blocks) { + LOG(ERROR) + << "--range_bytes is smaller than one I/O for endpoint " + << endpoint.endpoint; + return 1; + } + unique_handles.insert(endpoint.seg_handle); + endpoints.push_back(std::move(endpoint)); + } + + const size_t effective_submit_threads = + std::min(FLAGS_submit_threads, endpoints.size()); + const uint64_t total_iodepth = + static_cast(endpoints.size()) * FLAGS_iodepth; + + LOG(INFO) << "Bench config: endpoints=" << endpoints.size() + << ", unique_handles=" << unique_handles.size() + << ", configured_nof_workers=" + << (FLAGS_nof_workers == 0 ? mooncake::kDefaultSpdkNofWorkers + : FLAGS_nof_workers) + << ", effective_worker_bindings<=" + << std::min(unique_handles.size(), + FLAGS_nof_workers == 0 + ? mooncake::kDefaultSpdkNofWorkers + : FLAGS_nof_workers) + << ", io_size=" << FLAGS_io_size + << ", iodepth_per_endpoint=" << FLAGS_iodepth + << ", total_iodepth=" << total_iodepth + << ", submit_threads=" << effective_submit_threads + << ", warmup_sec=" << FLAGS_warmup_sec + << ", duration_sec=" << FLAGS_duration_sec; + if (unique_handles.size() < endpoints.size()) { + LOG(WARNING) + << "Some endpoints resolved to the same nof_seg_handle. In the " + "current SpdkNofWorkerPool implementation, the same handle " + "binds to a single worker thread, so duplicate endpoints " + "will " + "not increase worker parallelism."; + } + + std::vector thread_contexts( + effective_submit_threads); + for (size_t thread_index = 0; thread_index < effective_submit_threads; + ++thread_index) { + thread_contexts[thread_index].thread_index = thread_index; + } + for (size_t endpoint_index = 0; endpoint_index < endpoints.size(); + ++endpoint_index) { + size_t thread_index = endpoint_index % effective_submit_threads; + ThreadEndpointState thread_endpoint; + thread_endpoint.endpoint_index = endpoint_index; + thread_endpoint.slots.resize(FLAGS_iodepth); + thread_contexts[thread_index].endpoints.push_back( + std::move(thread_endpoint)); + } + + for (auto &thread_context : thread_contexts) { + for (auto &thread_endpoint : thread_context.endpoints) { + for (auto &slot : thread_endpoint.slots) { + slot.buffer_size = FLAGS_io_size; + slot.buffer = wrapper.Alloc(slot.buffer_size, 0x1000, + FLAGS_socket_id); + if (!slot.buffer) { + LOG(ERROR) << "Failed to allocate DMA buffer of size " + << slot.buffer_size; + FreeThreadSlots(wrapper, thread_contexts); + return 1; + } + } + } + } + + std::vector endpoint_stats(endpoints.size()); + mooncake::SpdkNofWorkerPool pool; + + TimePoint start = Clock::now(); + TimePoint warmup_end = start + std::chrono::seconds(FLAGS_warmup_sec); + TimePoint end = warmup_end + std::chrono::seconds(FLAGS_duration_sec); + + std::vector workers; + workers.reserve(thread_contexts.size()); + std::atomic active_threads(thread_contexts.size()); + for (auto &thread_context : thread_contexts) { + workers.emplace_back(WorkerLoop, &thread_context, &endpoints, + &endpoint_stats, &pool, op_mode, warmup_end, + end, &active_threads); + } + + TimePoint next_report = + Clock::now() + std::chrono::milliseconds(FLAGS_report_interval_ms); + std::vector last_endpoint_snapshots(endpoints.size()); + + while (active_threads.load(std::memory_order_relaxed) > 0) { + TimePoint now = Clock::now(); + if (now < next_report) { + std::this_thread::sleep_for(std::chrono::milliseconds(10)); + continue; + } + + if (now >= warmup_end) { + CounterSnapshot total_current; + CounterSnapshot total_last; + for (size_t endpoint_index = 0; + endpoint_index < endpoints.size(); ++endpoint_index) { + CounterSnapshot current = + endpoint_stats[endpoint_index].live.Snapshot(); + total_current.submitted_ops += current.submitted_ops; + total_current.completed_ops += current.completed_ops; + total_current.failed_ops += current.failed_ops; + total_current.bytes += current.bytes; + total_last.submitted_ops += + last_endpoint_snapshots[endpoint_index].submitted_ops; + total_last.completed_ops += + last_endpoint_snapshots[endpoint_index].completed_ops; + total_last.failed_ops += + last_endpoint_snapshots[endpoint_index].failed_ops; + total_last.bytes += + last_endpoint_snapshots[endpoint_index].bytes; + } + + double report_window_sec = + static_cast(FLAGS_report_interval_ms) / 1000.0; + CounterSnapshot delta_total = + DeltaCounters(total_current, total_last); + ThroughputView total_view = + ComputeThroughput(delta_total, report_window_sec); + LOG(INFO) << "interval bw=" + << FormatBytesPerSecond( + total_view.bandwidth_bytes_per_sec) + << ", iops=" << std::fixed << std::setprecision(2) + << total_view.iops + << ", fail_iops=" << total_view.fail_iops + << ", submitted=" << total_current.submitted_ops + << ", completed=" << total_current.completed_ops + << ", failed=" << total_current.failed_ops; + + for (size_t endpoint_index = 0; + endpoint_index < endpoints.size(); ++endpoint_index) { + CounterSnapshot current = + endpoint_stats[endpoint_index].live.Snapshot(); + CounterSnapshot delta = DeltaCounters( + current, last_endpoint_snapshots[endpoint_index]); + ThroughputView endpoint_view = + ComputeThroughput(delta, report_window_sec); + LOG(INFO) + << "interval endpoint[" << endpoint_index << "] bw=" + << FormatBytesPerSecond( + endpoint_view.bandwidth_bytes_per_sec) + << ", iops=" << std::fixed << std::setprecision(2) + << endpoint_view.iops + << ", fail_iops=" << endpoint_view.fail_iops + << ", endpoint=" << endpoints[endpoint_index].endpoint; + last_endpoint_snapshots[endpoint_index] = current; + } + } + + next_report = + now + std::chrono::milliseconds(FLAGS_report_interval_ms); + } + + for (auto &worker : workers) { + worker.join(); + } + + FreeThreadSlots(wrapper, thread_contexts); + + BenchStats total_measured; + for (const auto &stats : endpoint_stats) { + total_measured.MergeFrom(stats.measured); + } + + double duration = static_cast(FLAGS_duration_sec); + ThroughputView total_view = ComputeThroughput( + {total_measured.submitted_ops, total_measured.completed_ops, + total_measured.failed_ops, total_measured.bytes}, + duration); + LatencyView total_latency = ComputeLatency(total_measured); + + std::cout << std::fixed << std::setprecision(2); + std::cout << "========== NoF Worker Pool Bench ==========\n"; + std::cout << "endpoints=" << endpoints.size() << "\n"; + std::cout << "unique_handles=" << unique_handles.size() << "\n"; + std::cout << "configured_nof_workers=" + << (FLAGS_nof_workers == 0 ? mooncake::kDefaultSpdkNofWorkers + : FLAGS_nof_workers) + << "\n"; + std::cout << "effective_worker_bindings<=" + << std::min(unique_handles.size(), + FLAGS_nof_workers == 0 + ? mooncake::kDefaultSpdkNofWorkers + : FLAGS_nof_workers) + << "\n"; + std::cout << "numjobs=" << effective_submit_threads << "\n"; + std::cout << "bs=" << FLAGS_io_size << "\n"; + std::cout << "iodepth=" << FLAGS_iodepth << "\n"; + std::cout << "total_iodepth=" << total_iodepth << "\n"; + std::cout << "runtime=" << FLAGS_duration_sec << "\n"; + std::cout << "completed_ops=" << total_measured.completed_ops << "\n"; + std::cout << "failed_ops=" << total_measured.failed_ops << "\n"; + std::cout << "bw=" + << FormatBytesPerSecond(total_view.bandwidth_bytes_per_sec) + << "\n"; + std::cout << "iops=" << total_view.iops << "\n"; + std::cout << "fail_iops=" << total_view.fail_iops << "\n"; + std::cout << "clat_mean=" << FormatLatencyNs(total_latency.avg_ns) + << "\n"; + std::cout << "clat_p50=" << FormatLatencyNs(total_latency.p50_ns) + << "\n"; + std::cout << "clat_p95=" << FormatLatencyNs(total_latency.p95_ns) + << "\n"; + std::cout << "clat_p99=" << FormatLatencyNs(total_latency.p99_ns) + << "\n"; + + for (size_t endpoint_index = 0; endpoint_index < endpoints.size(); + ++endpoint_index) { + ThroughputView endpoint_view = ComputeThroughput( + {endpoint_stats[endpoint_index].measured.submitted_ops, + endpoint_stats[endpoint_index].measured.completed_ops, + endpoint_stats[endpoint_index].measured.failed_ops, + endpoint_stats[endpoint_index].measured.bytes}, + duration); + LatencyView endpoint_latency = + ComputeLatency(endpoint_stats[endpoint_index].measured); + std::cout << "endpoint[" << endpoint_index + << "]=" << endpoints[endpoint_index].endpoint << "\n"; + std::cout << "endpoint[" << endpoint_index << "].completed_ops=" + << endpoint_stats[endpoint_index].measured.completed_ops + << "\n"; + std::cout << "endpoint[" << endpoint_index << "].failed_ops=" + << endpoint_stats[endpoint_index].measured.failed_ops + << "\n"; + std::cout << "endpoint[" << endpoint_index << "].bw=" + << FormatBytesPerSecond( + endpoint_view.bandwidth_bytes_per_sec) + << "\n"; + std::cout << "endpoint[" << endpoint_index + << "].iops=" << endpoint_view.iops << "\n"; + std::cout << "endpoint[" << endpoint_index + << "].fail_iops=" << endpoint_view.fail_iops << "\n"; + std::cout << "endpoint[" << endpoint_index << "].clat_mean=" + << FormatLatencyNs(endpoint_latency.avg_ns) << "\n"; + std::cout << "endpoint[" << endpoint_index << "].clat_p50=" + << FormatLatencyNs(endpoint_latency.p50_ns) << "\n"; + std::cout << "endpoint[" << endpoint_index << "].clat_p95=" + << FormatLatencyNs(endpoint_latency.p95_ns) << "\n"; + std::cout << "endpoint[" << endpoint_index << "].clat_p99=" + << FormatLatencyNs(endpoint_latency.p99_ns) << "\n"; + } + std::cout << "==========================================\n"; + + google::ShutdownGoogleLogging(); + return 0; + } catch (const std::exception &e) { + LOG(ERROR) << "Benchmark failed: " << e.what(); + google::ShutdownGoogleLogging(); + return 1; + } +} diff --git a/mooncake-store/include/client_buffer.hpp b/mooncake-store/include/client_buffer.hpp index 9a9553d6..b9f84089 100644 --- a/mooncake-store/include/client_buffer.hpp +++ b/mooncake-store/include/client_buffer.hpp @@ -30,7 +30,7 @@ class ClientBufferAllocator // Create for heap-allocated memory static std::shared_ptr create( size_t size, const std::string& protocol = "", - bool use_hugepage = false); + bool use_hugepage = false, bool use_spdk_dma = false); // Create for shared memory static std::shared_ptr create( @@ -62,12 +62,13 @@ class ClientBufferAllocator private: ClientBufferAllocator(size_t size, const std::string& protocol, - bool use_hugepage); + bool use_hugepage, bool use_spdk_dma); std::shared_ptr allocator_; std::string protocol; bool is_external_memory_ = false; + bool use_spdk_dma_ = false; }; /** diff --git a/mooncake-store/include/memory_alloc.h b/mooncake-store/include/memory_alloc.h new file mode 100644 index 00000000..55a6d192 --- /dev/null +++ b/mooncake-store/include/memory_alloc.h @@ -0,0 +1,7 @@ +#pragma once + +#include +#include + +void *hugepage_memory_alloc(size_t size); +void hugepage_memory_free(void *ptr); diff --git a/mooncake-store/include/spdk/spdk_wrapper.h b/mooncake-store/include/spdk/spdk_wrapper.h new file mode 100644 index 00000000..3dbb621f --- /dev/null +++ b/mooncake-store/include/spdk/spdk_wrapper.h @@ -0,0 +1,109 @@ +#pragma once +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace mooncake { + +#define INVALID_BLOCK_SIZE 0xFFFFFFFF + +constexpr int kSpdkNofOpRead = 0; +constexpr int kSpdkNofOpWrite = 1; +constexpr int kSpdkNofOpNum = 2; + +struct nof_seg_handle; +struct tr_info; +struct ctrlr_info; + +class SpdkWrapper { + public: + SpdkWrapper(const SpdkWrapper &) = delete; + SpdkWrapper &operator=(const SpdkWrapper &) = delete; + + static SpdkWrapper &GetInstance(); + + bool InitializeEnv(); + + void Cleanup(); + + void *Alloc(size_t size, size_t align, int socket_id = -1); + + void Free(void *ptr); + + int64_t NvmePollProcessCompletion(nof_seg_handle *seg, + uint32_t complete_per_seg); + + /** @brief Open a NoF segment. */ + nof_seg_handle *OpenNofSegment(const std::string &tr_str); + + uint32_t GetBlockSize(const nof_seg_handle *seg_handle); + + int SubmitRequest(const nof_seg_handle *seg_handle, void *ptr, uint64_t lba, + uint32_t lba_count, int op, spdk_nvme_cmd_cb cb_fn, + void *cb_ctx); + + bool ProbeNofSegment(const std::string &tr_str, uint32_t timeout_ms, + std::string *error_reason = nullptr); + + private: + struct ProbeBuffer { + void *ptr{nullptr}; + uint32_t size{0}; + + ProbeBuffer() = default; + ProbeBuffer(const ProbeBuffer &) = delete; + ProbeBuffer &operator=(const ProbeBuffer &) = delete; + ProbeBuffer(ProbeBuffer &&) = delete; + ProbeBuffer &operator=(ProbeBuffer &&) = delete; + }; + + struct ProbeRequestContext { + std::atomic done{false}; + std::atomic success{false}; + std::mutex error_mutex; + std::string error_reason; + SpdkWrapper *owner{nullptr}; + + void Reset(SpdkWrapper *wrapper) { + std::lock_guard lock(error_mutex); + owner = wrapper; + done.store(false, std::memory_order_release); + success.store(false, std::memory_order_release); + error_reason.clear(); + } + }; + + explicit SpdkWrapper(); + ~SpdkWrapper(); + + int ParseTransPortStr(const std::string &tr_str, tr_info *info); + int ConnectController(const struct spdk_nvme_transport_id *trid, + ctrlr_info *info); + ProbeBuffer *GetOrCreateProbeBuffer(const std::string &tr_str, + uint32_t block_size, + std::string *error_reason); + ProbeRequestContext *AcquireProbeRequestContext(); + void RecycleProbeRequestContext(ProbeRequestContext *ctx); + void ReplenishProbeRequestContextPoolLocked(size_t count); + static void ProbeReadComplete(void *ctx, const struct spdk_nvme_cpl *cpl); + + std::atomic initialized{false}; + std::mutex init_mutex; + std::map> connected_ctrlrs; + std::mutex ctrlrs_mutex; + std::map> probe_buffers_; + std::mutex probe_buffers_mutex_; + std::vector> probe_request_contexts_; + std::stack probe_request_context_pool_; + std::mutex probe_request_context_pool_mutex_; +}; + +} // namespace mooncake diff --git a/mooncake-store/include/transfer_task.h b/mooncake-store/include/transfer_task.h index 42961d05..63b06191 100644 --- a/mooncake-store/include/transfer_task.h +++ b/mooncake-store/include/transfer_task.h @@ -4,11 +4,13 @@ #include #include #include +#include #include #include #include #include #include +#include #include #include #include @@ -18,6 +20,9 @@ #include "replica.h" #include "storage_backend.h" #include "client_metric.h" +#ifdef USE_NOF +#include "spdk/spdk_wrapper.h" +#endif namespace mooncake { @@ -28,7 +33,8 @@ enum class TransferStrategy { LOCAL_MEMCPY = 0, // Local memory copy using memcpy TRANSFER_ENGINE = 1, // Remote transfer using transfer engine FILE_READ = 2, // File read operation - EMPTY = 3 + EMPTY = 3, + SPDK_NVMF = 4 // Spdk nvmf operation }; /** @@ -41,8 +47,12 @@ inline std::ostream& operator<<(std::ostream& os, return os << "LOCAL_MEMCPY"; case TransferStrategy::TRANSFER_ENGINE: return os << "TRANSFER_ENGINE"; + case TransferStrategy::SPDK_NVMF: + return os << "SPDK_NVMF"; case TransferStrategy::FILE_READ: return os << "FILE_READ"; + case TransferStrategy::EMPTY: + return os << "EMPTY"; default: return os << "UNKNOWN"; } @@ -141,6 +151,35 @@ class MemcpyOperationState : public OperationState { } }; +/** + * @brief Operation state for local memcpy transfers + */ +class SpdkNofOperationState : public OperationState { + public: + bool is_completed() override { + std::lock_guard lock(mutex_); + return result_.has_value(); + } + + void set_completed(ErrorCode error_code) { + { + std::lock_guard lock(mutex_); + assert(!result_.has_value()); + result_.emplace(error_code); + } + cv_.notify_all(); + } + + void wait_for_completion() override { + std::unique_lock lock(mutex_); + cv_.wait(lock, [this] { return result_.has_value(); }); + } + + TransferStrategy get_strategy() const override { + return TransferStrategy::SPDK_NVMF; + } +}; + class FilereadOperationState : public OperationState { public: bool is_completed() override { @@ -307,6 +346,127 @@ class MemcpyWorkerPool { std::atomic shutdown_; }; +#ifdef USE_NOF +// struct SpdkNofSubTask; +struct SpdkNofQos; + +/** + * @brief Spdk nvmf operation descriptor + */ +struct SpdkNofTask { + nof_seg_handle* seg_handle; + void* ptr; + uint64_t lba; + uint32_t lba_count; + int remaining_lba; + int outstanding_sub_io; + int op; // kSpdkNofOpRead or kSpdkNofOpWrite + int idx; // subop idx + bool failed; + bool on_chain; + std::shared_ptr state; + int64_t* io_count; + SpdkNofQos* nof_qos; + SpdkNofTask* nxt; + + SpdkNofTask(nof_seg_handle* handle, void* buf, uint64_t off, uint32_t len, + int op_code, std::shared_ptr s) + : seg_handle(handle), + ptr(buf), + lba(off), + lba_count(len), + remaining_lba(lba_count), + outstanding_sub_io(0), + op(op_code), + idx(0), + failed(false), + on_chain(false), + state(std::move(s)), + io_count(nullptr), + nof_qos(nullptr), + nxt(nullptr) {} +}; + +struct SpdkNofSubTask { + SpdkNofTask* task; + int submit_lba_count; + std::stack* sub_task_pool; +}; + +constexpr int kDefaultSpdkNofSubmitChunkBytes = (1 << 17); // 128k +constexpr int kDefaultSpdkNofInflightBytesLimit = (1 << 25); // 32M +struct SpdkNofQos { + int inflight_blocks[kSpdkNofOpNum]; + int blocks_per_chunk; + int inflight_blocks_limit; + SpdkNofTask* head[kSpdkNofOpNum]; + SpdkNofTask* tail[kSpdkNofOpNum]; + + explicit SpdkNofQos(uint32_t block_size); + + bool Empty() const { + return (head[kSpdkNofOpRead] == nullptr && + head[kSpdkNofOpWrite] == nullptr); + } + + void PushTask(SpdkNofTask* task) { + int op = task->op; + if (head[op] == nullptr) { + head[op] = task; + tail[op] = task; + } else { + tail[op]->nxt = task; + tail[op] = task; + } + } + + void PopTask(int op) { + if (head[op]) { + head[op] = head[op]->nxt; + } + } +}; + +/** + * @brief Thread pool for asynchronous spdk nvmf operations + * + * This class manages multiple worker thread that executes spdk nvmf operations + * asynchronously. + */ +constexpr int kDefaultSpdkNofWorkers = 4; +class SpdkNofWorkerPool { + public: + explicit SpdkNofWorkerPool(int numa_socket_id = 0); + ~SpdkNofWorkerPool(); + + // Non-copyable, non-movable + SpdkNofWorkerPool(const SpdkNofWorkerPool&) = delete; + SpdkNofWorkerPool& operator=(const SpdkNofWorkerPool&) = delete; + SpdkNofWorkerPool(SpdkNofWorkerPool&&) = delete; + SpdkNofWorkerPool& operator=(SpdkNofWorkerPool&&) = delete; + + /** + * @brief Submit a spdk nvmf task for async execution + * @param task The spdk nvmf task to execute + */ + void submitTask(SpdkNofTask task); + + private: + void workerThread(int work_idx); + + int worker_count_; + int numa_socket_id_; + std::vector workers_; + std::unique_ptr[]> task_queue_; + std::unique_ptr queue_mutex_; + std::unique_ptr queue_cv_; + std::atomic shutdown_; + std::mutex seg_mutex_; + int seg_num = 0; + std::map seg_to_worker_; +}; +#endif + /** * @brief Fileread task for async execution */ @@ -371,7 +531,8 @@ class TransferSubmitter { explicit TransferSubmitter(TransferEngine& engine, std::shared_ptr& backend, const std::string& local_hostname, - TransferMetric* transfer_metric = nullptr); + TransferMetric* transfer_metric = nullptr, + int numa_socket_id = 0); /** * @brief Submit an asynchronous transfer operation @@ -388,7 +549,8 @@ class TransferSubmitter { */ std::optional submit(const Replica::Descriptor& replica, std::vector& slices, - TransferRequest::OpCode op_code); + TransferRequest::OpCode op_code, + void* ptr = nullptr, size_t size = 0); /** * @brief Submit a range read: read [src_offset, src_offset+size) from @@ -429,6 +591,9 @@ class TransferSubmitter { // engine_.getLocalIpAndPort() (which allocates a string) on every transfer. const std::string local_endpoint_; std::unique_ptr memcpy_pool_; +#ifdef USE_NOF + std::unique_ptr spdk_nvmf_pool_; +#endif std::unique_ptr fileread_pool_; bool memcpy_enabled_; const std::string local_hostname_; @@ -459,6 +624,15 @@ class TransferSubmitter { const std::vector& slices, const TransferRequest::OpCode op_code, uint64_t src_offset = 0); +#ifdef USE_NOF + /** + * @brief Submit SPDK NVMe-oF operation asynchronously + */ + std::optional submitSpdkNofOperation( + const AllocatedBuffer::Descriptor& handle, void* ptr, size_t size, + const TransferRequest::OpCode op_code); +#endif + /** * @brief Submit transfer engine operation asynchronously * @param src_offset Optional offset in source buffer (default 0) diff --git a/mooncake-store/include/utils.h b/mooncake-store/include/utils.h index 5bed64d9..8e4d0994 100644 --- a/mooncake-store/include/utils.h +++ b/mooncake-store/include/utils.h @@ -294,7 +294,8 @@ constexpr double BYTES_PER_GIB = static_cast(SZ_1GB); */ void* allocate_buffer_allocator_memory( size_t total_size, const std::string& protocol = "", - size_t alignment = facebook::cachelib::Slab::kSize); + size_t alignment = facebook::cachelib::Slab::kSize, + bool use_spdk_dma = false); inline size_t align_up(size_t size, size_t alignment) { if (alignment == 0) { diff --git a/mooncake-store/src/CMakeLists.txt b/mooncake-store/src/CMakeLists.txt index 53dbcf9d..abd184e2 100644 --- a/mooncake-store/src/CMakeLists.txt +++ b/mooncake-store/src/CMakeLists.txt @@ -56,10 +56,12 @@ set(MOONCAKE_STORE_SOURCES standby_state_machine.cpp ha_metric_manager.cpp store_c.cpp + memory_alloc.cpp engram/engram_store.cpp ) set(EXTRA_LIBS "") +set(SPDK_STATIC_LIBS "") # Find AWS SDK find_package(AWSSDK QUIET COMPONENTS s3) @@ -147,10 +149,90 @@ if(STORE_USE_K8S_LEASE) list(APPEND EXTRA_LIBS ${K8S_LEASE_WRAPPER_LIB}) endif() +if(USE_NOF) + add_subdirectory(spdk) + list(APPEND MOONCAKE_STORE_SOURCES ${SPDK_WRAPPER_SOURCES}) + + set(SPDK_INCLUDE_DIR + "/usr/local/include" + CACHE PATH "SPDK include directory") + set(DPDK_INCLUDE_DIR + "/usr/local/include" + CACHE PATH "DPDK include directory") + set(SPDK_LIB_DIR + "/usr/local/lib" + CACHE PATH "SPDK static library directory") + set(DPDK_LIB_DIR + "/usr/local/lib" + CACHE PATH "DPDK static library directory") + + set(SPDK_STATIC_LIBS + -Wl,--whole-archive + ${SPDK_LIB_DIR}/libspdk_event.a + ${SPDK_LIB_DIR}/libspdk_rdma.a + ${SPDK_LIB_DIR}/libspdk_nvme.a + ${SPDK_LIB_DIR}/libspdk_sock_posix.a + -Wl,--no-whole-archive + ${SPDK_LIB_DIR}/libspdk_vfio_user.a + ${SPDK_LIB_DIR}/libspdk_dma.a + ${SPDK_LIB_DIR}/libspdk_init.a + ${SPDK_LIB_DIR}/libspdk_env_dpdk.a + ${SPDK_LIB_DIR}/libspdk_log.a + ${SPDK_LIB_DIR}/libspdk_util.a + ${SPDK_LIB_DIR}/libspdk_blob.a + ${SPDK_LIB_DIR}/libspdk_blobfs.a + ${SPDK_LIB_DIR}/libspdk_bdev.a + ${SPDK_LIB_DIR}/libspdk_conf.a + ${SPDK_LIB_DIR}/libspdk_json.a + ${SPDK_LIB_DIR}/libspdk_jsonrpc.a + ${SPDK_LIB_DIR}/libspdk_sock.a + ${SPDK_LIB_DIR}/libspdk_thread.a + ${SPDK_LIB_DIR}/libspdk_trace.a + ${SPDK_LIB_DIR}/libspdk_vmd.a + ${SPDK_LIB_DIR}/libspdk_rpc.a + ${DPDK_LIB_DIR}/librte_eal.a + ${DPDK_LIB_DIR}/librte_mempool.a + ${DPDK_LIB_DIR}/librte_ring.a + ${DPDK_LIB_DIR}/librte_bus_pci.a + ${DPDK_LIB_DIR}/librte_pci.a + ${DPDK_LIB_DIR}/librte_timer.a + ${DPDK_LIB_DIR}/librte_telemetry.a + ${DPDK_LIB_DIR}/librte_kvargs.a + ${DPDK_LIB_DIR}/librte_mbuf.a + -Wl,--whole-archive + ${DPDK_LIB_DIR}/librte_mempool_ring.a + -Wl,--no-whole-archive + ${DPDK_LIB_DIR}/librte_power.a + ${DPDK_LIB_DIR}/librte_ethdev.a + ${DPDK_LIB_DIR}/librte_net.a + ${DPDK_LIB_DIR}/librte_vhost.a + ${DPDK_LIB_DIR}/librte_cryptodev.a + ${DPDK_LIB_DIR}/librte_hash.a + ${DPDK_LIB_DIR}/librte_rcu.a + pthread + dl + numa + rt + m + uuid + crypto + aio + z + elf + ibverbs + rdmacm + ssl + bsd) +endif() + # The cache_allocator library include_directories(${Python3_INCLUDE_DIRS}) add_library(mooncake_store ${MOONCAKE_STORE_SOURCES}) target_include_directories(mooncake_store PUBLIC ${XXHASH_INCLUDE_DIR}) +if(USE_NOF) + target_include_directories(mooncake_store PRIVATE ${SPDK_INCLUDE_DIR} + ${DPDK_INCLUDE_DIR}) +endif() target_link_libraries(mooncake_store PUBLIC ${XXHASH_LIBRARY}) if(STORE_USE_REDIS) target_include_directories(mooncake_store @@ -162,7 +244,7 @@ endif() target_link_libraries( mooncake_store PUBLIC cachelib_memory_allocator ${ETCD_WRAPPER_LIB} glog::glog gflags::gflags - ${EXTRA_LIBS} asio_shared + ${EXTRA_LIBS} ${SPDK_STATIC_LIBS} asio_shared PRIVATE transfer_engine) if(STORE_USE_ETCD) add_dependencies(mooncake_store build_etcd_wrapper) diff --git a/mooncake-store/src/client_buffer.cpp b/mooncake-store/src/client_buffer.cpp index e9342086..e846d4c0 100644 --- a/mooncake-store/src/client_buffer.cpp +++ b/mooncake-store/src/client_buffer.cpp @@ -13,9 +13,10 @@ namespace mooncake { std::shared_ptr ClientBufferAllocator::create( - size_t size, const std::string& protocol, bool use_hugepage) { + size_t size, const std::string& protocol, bool use_hugepage, + bool use_spdk_dma) { return std::shared_ptr( - new ClientBufferAllocator(size, protocol, use_hugepage)); + new ClientBufferAllocator(size, protocol, use_hugepage, use_spdk_dma)); } std::shared_ptr ClientBufferAllocator::create( @@ -26,8 +27,12 @@ std::shared_ptr ClientBufferAllocator::create( ClientBufferAllocator::ClientBufferAllocator(size_t size, const std::string& protocol, - bool use_hugepage) - : buffer_size_(size), use_hugepage_(use_hugepage), protocol(protocol) { + bool use_hugepage, + bool use_spdk_dma) + : buffer_size_(size), + use_hugepage_(use_hugepage), + protocol(protocol), + use_spdk_dma_(use_spdk_dma) { if (size == 0) { buffer_ = nullptr; allocator_ = nullptr; @@ -38,7 +43,8 @@ ClientBufferAllocator::ClientBufferAllocator(size_t size, if (use_hugepage_) { buffer_ = allocate_buffer_mmap_memory(size, alignment); } else { - buffer_ = allocate_buffer_allocator_memory(size, protocol, alignment); + buffer_ = allocate_buffer_allocator_memory(size, protocol, alignment, + use_spdk_dma_); } if (!buffer_) { throw std::bad_alloc(); @@ -132,6 +138,8 @@ uint64_t calculate_total_size(const Replica::Descriptor& replica) { total_length = disk_descriptor.object_size; } else if (replica.is_local_disk_replica()) { total_length = replica.get_local_disk_descriptor().object_size; + } else if (replica.is_nof_replica()) { + total_length = replica.get_nof_descriptor().buffer_descriptor.size_; } else { total_length = replica.get_memory_descriptor().buffer_descriptor.size_; } @@ -153,6 +161,10 @@ int allocateSlices(std::vector& slices, } else if (replica.is_local_disk_replica()) { slices.emplace_back( Slice{buffer_ptr, replica.get_local_disk_descriptor().object_size}); + } else if (replica.is_nof_replica()) { + auto& handle = replica.get_nof_descriptor().buffer_descriptor; + void* chunk_ptr = buffer_ptr; + slices.emplace_back(Slice{chunk_ptr, handle.size_}); } else { // For memory-based replica, split into slices based on buffer // descriptors diff --git a/mooncake-store/src/master_metric_manager.cpp b/mooncake-store/src/master_metric_manager.cpp index 163b38e6..da1ef239 100644 --- a/mooncake-store/src/master_metric_manager.cpp +++ b/mooncake-store/src/master_metric_manager.cpp @@ -2238,27 +2238,27 @@ std::string MasterMetricManager::get_summary_string( delta(&SummaryCounters::mark_task_to_complete_fails), delta(&SummaryCounters::mark_task_to_complete_requests)) << ")"; - // Eviction summary + // Eviction counters are cumulative. Request counters above are reported as + // window rates, but eviction totals are used to track long-running pressure + // and should not reset after each admin metrics snapshot. ss << " | Eviction: " - << "Success/Attempts=" << delta(&SummaryCounters::eviction_success) - << "/" << delta(&SummaryCounters::eviction_attempts) << ", " + << "Success/Attempts=" << eviction_success << "/" << eviction_attempts + << ", " << "AllocFail=" << delta(&SummaryCounters::put_start_alloc_fails) << ", " - << "keys=" << delta(&SummaryCounters::evicted_key_count) << ", " - << "size=" << byte_size_to_string(delta(&SummaryCounters::evicted_size)); + << "keys=" << evicted_key_count << ", " + << "size=" << byte_size_to_string(evicted_size); // mem eviction ss << " | Mem Eviction: " - << "Success/Attempts=" << delta(&SummaryCounters::mem_eviction_success) - << "/" << delta(&SummaryCounters::mem_eviction_attempts) << ", " - << "keys=" << delta(&SummaryCounters::mem_evicted_key_count) << ", " - << "size=" - << byte_size_to_string(delta(&SummaryCounters::mem_evicted_size)); + << "Success/Attempts=" << mem_eviction_success << "/" + << mem_eviction_attempts << ", " + << "keys=" << mem_evicted_key_count << ", " + << "size=" << byte_size_to_string(mem_evicted_size); // nof eviction ss << " | NoF Eviction: " - << "Success/Attempts=" << delta(&SummaryCounters::nof_eviction_success) - << "/" << delta(&SummaryCounters::nof_eviction_attempts) << ", " - << "keys=" << delta(&SummaryCounters::nof_evicted_key_count) << ", " - << "size=" - << byte_size_to_string(delta(&SummaryCounters::nof_evicted_size)); + << "Success/Attempts=" << nof_eviction_success << "/" + << nof_eviction_attempts << ", " + << "keys=" << nof_evicted_key_count << ", " + << "size=" << byte_size_to_string(nof_evicted_size); // Discard summary ss << " | Discard: " diff --git a/mooncake-store/src/memory_alloc.cpp b/mooncake-store/src/memory_alloc.cpp new file mode 100644 index 00000000..74947a4d --- /dev/null +++ b/mooncake-store/src/memory_alloc.cpp @@ -0,0 +1,19 @@ +#include "memory_alloc.h" + +#ifdef USE_NOF +#include "spdk/spdk_wrapper.h" +#endif + +void *hugepage_memory_alloc(size_t size) { +#ifndef USE_NOF + return nullptr; +#else + return mooncake::SpdkWrapper::GetInstance().Alloc(size, 0x1000, -1); +#endif +} + +void hugepage_memory_free(void *ptr) { +#ifdef USE_NOF + mooncake::SpdkWrapper::GetInstance().Free(ptr); +#endif +} diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 9ea43f47..b0cab936 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -32,6 +32,9 @@ #include "default_config.h" #include "shm_helper.h" #include "memory_location.h" +#ifdef USE_NOF +#include "spdk/spdk_wrapper.h" +#endif #ifdef USE_ASCEND_DIRECT #include "acl/acl_rt.h" #include "transport/ascend_transport/ascend_direct_transport/context_manager.h" @@ -287,6 +290,7 @@ inline const Replica::Descriptor *SelectBestReplica( const std::vector &replicas, const std::unordered_set &local_endpoints) { const Replica::Descriptor *first_memory = nullptr; + const Replica::Descriptor *first_nof = nullptr; for (const auto &r : replicas) { if (r.status != ReplicaStatus::COMPLETE) continue; if (r.is_memory_replica()) { @@ -296,9 +300,17 @@ inline const Replica::Descriptor *SelectBestReplica( return &r; // local MEMORY — best case } if (!first_memory) first_memory = &r; + } else if (r.is_nof_replica()) { + if (local_endpoints.count( + r.get_nof_descriptor() + .buffer_descriptor.transport_endpoint_)) { + return &r; // local NOF_SSD — also good + } + if (!first_nof) first_nof = &r; } } if (first_memory) return first_memory; + if (first_nof) return first_nof; const Replica::Descriptor *best = nullptr; for (const auto &r : replicas) { @@ -636,6 +648,13 @@ tl::expected RealClient::setup_internal( } #endif +#ifdef USE_NOF + if (!SpdkWrapper::GetInstance().InitializeEnv()) { + LOG(ERROR) << "spdk env init fail"; + return tl::unexpected(ErrorCode::INTERNAL_ERROR); + } +#endif + std::optional device_name = (rdma_devices.empty() ? std::nullopt : std::make_optional(rdma_devices)); @@ -722,8 +741,13 @@ tl::expected RealClient::setup_internal( // fail in some rdma implementations. // Dummy Client can create shm and share it with Real Client, so Real Client // can create client buffer allocator on the shared memory later. + bool use_spdk_dma_for_client_buffer = false; +#ifdef USE_NOF + use_spdk_dma_for_client_buffer = true; +#endif client_buffer_allocator_ = ClientBufferAllocator::create( - local_buffer_size, this->protocol, should_use_hugepage); + local_buffer_size, this->protocol, should_use_hugepage, + use_spdk_dma_for_client_buffer); if (local_buffer_size > 0 && protocol != "cxl") { LOG(INFO) << "Registering local memory: " << local_buffer_size << " bytes"; diff --git a/mooncake-store/src/spdk/CMakeLists.txt b/mooncake-store/src/spdk/CMakeLists.txt new file mode 100644 index 00000000..31d60847 --- /dev/null +++ b/mooncake-store/src/spdk/CMakeLists.txt @@ -0,0 +1,5 @@ +set(SPDK_WRAPPER_SOURCES + ${CMAKE_CURRENT_SOURCE_DIR}/spdk_wrapper.cpp +) + +set(SPDK_WRAPPER_SOURCES ${SPDK_WRAPPER_SOURCES} PARENT_SCOPE) \ No newline at end of file diff --git a/mooncake-store/src/spdk/spdk_wrapper.cpp b/mooncake-store/src/spdk/spdk_wrapper.cpp new file mode 100644 index 00000000..5042c8f4 --- /dev/null +++ b/mooncake-store/src/spdk/spdk_wrapper.cpp @@ -0,0 +1,490 @@ +#include + +#include +#include +#include +#include +#include +#include +#include "spdk/spdk_wrapper.h" + +namespace mooncake { +namespace { + +bool ParseEnvU64(const char *name, uint64_t *out) { + const char *val = std::getenv(name); + if (!val || *val == '\0') { + return false; + } + + errno = 0; + char *end = nullptr; + unsigned long long parsed = std::strtoull(val, &end, 10); + if (errno != 0 || end == val || (end && *end != '\0')) { + LOG(WARNING) << "Invalid value for " << name << ": " << val; + return false; + } + + *out = static_cast(parsed); + return true; +} + +bool ParseEnvBool(const char *name, bool *out) { + uint64_t v = 0; + if (!ParseEnvU64(name, &v)) { + return false; + } + *out = (v != 0); + return true; +} + +void ApplyCtrlrOptsFromEnv(struct spdk_nvme_ctrlr_opts *opts) { + uint64_t v = 0; + bool bv = false; + opts->keep_alive_timeout_ms = 0; + + if (ParseEnvU64("MC_NVME_NUM_IO_QUEUES", &v)) { + opts->num_io_queues = static_cast(v); + } + if (ParseEnvU64("MC_NVME_IO_QUEUE_SIZE", &v)) { + opts->io_queue_size = static_cast(v); + } + if (ParseEnvU64("MC_NVME_IO_QUEUE_REQUESTS", &v)) { + opts->io_queue_requests = static_cast(v); + } + if (ParseEnvU64("MC_NVME_TRANSPORT_ACK_TIMEOUT", &v)) { + opts->transport_ack_timeout = static_cast(v); + } + if (ParseEnvU64("MC_NVME_ADMIN_QUEUE_SIZE", &v)) { + opts->admin_queue_size = static_cast(v); + } + if (ParseEnvU64("MC_NVME_FABRICS_CONNECT_TIMEOUT_US", &v)) { + opts->fabrics_connect_timeout_us = v; + } + if (ParseEnvBool("MC_NVME_HEADER_DIGEST", &bv)) { + opts->header_digest = bv; + } + if (ParseEnvBool("MC_NVME_DATA_DIGEST", &bv)) { + opts->data_digest = bv; + } + LOG(INFO) << "NVMe ctrlr opts: num_io_queues=" << opts->num_io_queues + << ", io_queue_size=" << opts->io_queue_size + << ", io_queue_requests=" << opts->io_queue_requests + << ", keep_alive_timeout_ms=" << opts->keep_alive_timeout_ms + << ", transport_ack_timeout=" + << static_cast(opts->transport_ack_timeout) + << ", admin_queue_size=" << opts->admin_queue_size + << ", fabrics_connect_timeout_us=" + << opts->fabrics_connect_timeout_us + << ", header_digest=" << opts->header_digest + << ", data_digest=" << opts->data_digest; +} + +} // namespace + +struct nof_seg_handle { + struct spdk_nvme_qpair *qpair; + struct spdk_nvme_ns *ns; +}; + +struct tr_info { + struct spdk_nvme_transport_id trid; + std::string ctrlr_key; + uint32_t ns; +}; + +struct ctrlr_info { + struct spdk_nvme_ctrlr *ctrlr; + std::map> ns_seg; + std::mutex ns_mutex; +}; + +SpdkWrapper::SpdkWrapper() = default; + +SpdkWrapper::~SpdkWrapper() { Cleanup(); } + +SpdkWrapper &SpdkWrapper::GetInstance() { + static SpdkWrapper ins; + return ins; +} + +bool SpdkWrapper::InitializeEnv() { + if (initialized.load(std::memory_order_acquire)) { + return true; + } + + std::lock_guard lock(init_mutex); + if (initialized.load(std::memory_order_acquire)) { + return true; + } + + struct spdk_env_opts opts; + spdk_env_opts_init(&opts); + opts.name = "mooncake"; + + int rc = spdk_env_init(&opts); + if (rc != 0) { + fprintf(stderr, "SPDK init failed: %d\n", rc); + return false; + } + + // Mark SPDK as initialized. + initialized.store(true, std::memory_order_release); + return true; +} + +void SpdkWrapper::Cleanup() { + if (initialized.load(std::memory_order_acquire)) { + { + std::lock_guard lock(ctrlrs_mutex); + for (auto &[_, info] : connected_ctrlrs) { + if (info) { + // Free all qpairs and segment handles + for (auto &[_, seg] : info->ns_seg) { + if (seg && seg->qpair) { + spdk_nvme_ctrlr_free_io_qpair(seg->qpair); + } + } + // Detach controller + if (info->ctrlr) { + spdk_nvme_detach(info->ctrlr); + } + } + } + connected_ctrlrs.clear(); + } + + { + std::lock_guard lock(probe_buffers_mutex_); + for (auto &[_, probe_buffer] : probe_buffers_) { + if (probe_buffer && probe_buffer->ptr) { + spdk_free(probe_buffer->ptr); + probe_buffer->ptr = nullptr; + probe_buffer->size = 0; + } + } + probe_buffers_.clear(); + } + spdk_env_fini(); + initialized.store(false, std::memory_order_release); + } +} + +void *SpdkWrapper::Alloc(size_t size, size_t align, int socket_id) { + if (!InitializeEnv()) { + return nullptr; + } + + return spdk_zmalloc(size, align, nullptr, socket_id, SPDK_MALLOC_DMA); +} + +void SpdkWrapper::Free(void *ptr) { + if (ptr) { + spdk_free(ptr); + } +} + +void SpdkWrapper::ProbeReadComplete(void *ctx, + const struct spdk_nvme_cpl *cpl) { + auto *probe_ctx = reinterpret_cast(ctx); + if (spdk_nvme_cpl_is_error(cpl)) { + { + std::lock_guard lock(probe_ctx->error_mutex); + probe_ctx->error_reason = + std::string("completion_error:") + + spdk_nvme_cpl_get_status_string(&cpl->status); + } + probe_ctx->success.store(false, std::memory_order_release); + } else { + probe_ctx->success.store(true, std::memory_order_release); + } + probe_ctx->done.store(true, std::memory_order_release); + if (probe_ctx->owner != nullptr) { + probe_ctx->owner->RecycleProbeRequestContext(probe_ctx); + } +} + +void SpdkWrapper::ReplenishProbeRequestContextPoolLocked(size_t count) { + for (size_t i = 0; i < count; ++i) { + auto probe_ctx = std::make_unique(); + probe_request_context_pool_.push(probe_ctx.get()); + probe_request_contexts_.push_back(std::move(probe_ctx)); + } +} + +SpdkWrapper::ProbeRequestContext *SpdkWrapper::AcquireProbeRequestContext() { + std::lock_guard lock(probe_request_context_pool_mutex_); + if (probe_request_context_pool_.empty()) { + ReplenishProbeRequestContextPoolLocked(8); + } + auto *probe_ctx = probe_request_context_pool_.top(); + probe_request_context_pool_.pop(); + probe_ctx->Reset(this); + return probe_ctx; +} + +void SpdkWrapper::RecycleProbeRequestContext(ProbeRequestContext *ctx) { + if (ctx == nullptr) { + return; + } + std::lock_guard lock(probe_request_context_pool_mutex_); + probe_request_context_pool_.push(ctx); +} + +int64_t SpdkWrapper::NvmePollProcessCompletion(nof_seg_handle *seg, + uint32_t complete_per_seg) { + return spdk_nvme_qpair_process_completions(seg->qpair, complete_per_seg); +} + +int SpdkWrapper::ParseTransPortStr(const std::string &tr_str, tr_info *info) { + std::memset(&info->trid, 0, sizeof(info->trid)); + info->ns = 1; + + if (spdk_nvme_transport_id_parse(&info->trid, tr_str.c_str()) != 0) { + LOG(ERROR) << "Error parsing transport address"; + return -1; + } + + std::string ns_prefix = "ns:"; + size_t ns_pos = tr_str.find(ns_prefix); + if (ns_pos != std::string::npos) { + size_t ns_start = ns_pos + ns_prefix.length(); + size_t ns_end = tr_str.find_first_of(" \t", ns_start); + + std::string ns_str; + if (ns_end == std::string::npos) { + ns_str = tr_str.substr(ns_start); + } else { + ns_str = tr_str.substr(ns_start, ns_end - ns_start); + } + + try { + info->ns = std::stoul(ns_str); + } catch (const std::exception &e) { + LOG(ERROR) << "Failed to parse ns value: " << ns_str + << ", error: " << e.what(); + return -1; + } + } else { + LOG(ERROR) << "No ns field found in transport string"; + } + + info->ctrlr_key = std::string(info->trid.traddr) + "|" + + std::string(info->trid.trsvcid) + "|" + + std::string(info->trid.subnqn) + "|" + + std::to_string(static_cast(info->trid.trtype)); + + LOG(INFO) << "traddr:" << info->trid.traddr + << "trsvcid:" << info->trid.trsvcid << "ns:" << info->ns + << "subnqn:" << info->trid.subnqn + << "trtype:" << info->trid.trtype; + + return 0; +} + +int SpdkWrapper::ConnectController(const struct spdk_nvme_transport_id *trid, + ctrlr_info *info) { + auto probe_cb = [](void *cb_ctx, const struct spdk_nvme_transport_id *trid, + struct spdk_nvme_ctrlr_opts *opts) -> bool { + ApplyCtrlrOptsFromEnv(opts); + LOG(INFO) << "Attaching to " << trid->traddr << " " << trid->subnqn; + return true; + }; + + auto attach_cb = [](void *cb_ctx, const struct spdk_nvme_transport_id *trid, + struct spdk_nvme_ctrlr *ctrlr, + const struct spdk_nvme_ctrlr_opts *opts) { + LOG(INFO) << "Attached to " << trid->traddr << " " << trid->subnqn; + ctrlr_info *info = (ctrlr_info *)cb_ctx; + info->ctrlr = ctrlr; + }; + + return spdk_nvme_probe(trid, (void *)info, probe_cb, attach_cb, NULL); +} + +nof_seg_handle *SpdkWrapper::OpenNofSegment(const std::string &tr_str) { + tr_info tr; + int ret = ParseTransPortStr(tr_str, &tr); + if (ret != 0) { + return nullptr; + } + + ctrlr_info *info = nullptr; + { + std::lock_guard lock(ctrlrs_mutex); + auto it = connected_ctrlrs.find(tr.ctrlr_key); + if (it == connected_ctrlrs.end()) { + auto new_info = std::make_unique(); + info = new_info.get(); + + ret = ConnectController(&tr.trid, info); + if (ret != 0) { + return nullptr; + } + + connected_ctrlrs[tr.ctrlr_key] = std::move(new_info); + } else { + info = it->second.get(); + } + } + + nof_seg_handle *seg_handle = nullptr; + struct spdk_nvme_qpair *qpair = nullptr; + struct spdk_nvme_ns *ns = nullptr; + { + auto &ns_seg = info->ns_seg; + std::lock_guard lock(info->ns_mutex); + auto ns_it = ns_seg.find(tr.ns); + if (ns_it != ns_seg.end()) { + return ns_it->second.get(); + } + + if (spdk_nvme_ctrlr_is_active_ns(info->ctrlr, tr.ns)) { + ns = spdk_nvme_ctrlr_get_ns(info->ctrlr, tr.ns); + } else { + LOG(ERROR) << "spdk_nvme_ctrlr_is_active_ns failed"; + return nullptr; + } + + qpair = spdk_nvme_ctrlr_alloc_io_qpair(info->ctrlr, nullptr, 0); + if (!qpair) { + LOG(ERROR) << "alloc spdk_nvme_qpair failed"; + return nullptr; + } + + auto new_seg = std::make_unique(); + new_seg->qpair = qpair; + new_seg->ns = ns; + seg_handle = new_seg.get(); + ns_seg[tr.ns] = std::move(new_seg); + } + + return seg_handle; +} + +uint32_t SpdkWrapper::GetBlockSize(const nof_seg_handle *seg_handle) { + if (!seg_handle || !seg_handle->ns) { + return INVALID_BLOCK_SIZE; + } + + return spdk_nvme_ns_get_sector_size(seg_handle->ns); +} + +int SpdkWrapper::SubmitRequest(const nof_seg_handle *seg_handle, void *ptr, + uint64_t lba, uint32_t lba_count, int op, + spdk_nvme_cmd_cb cb_fn, void *cb_ctx) { + if (!seg_handle || !ptr || !lba_count || !seg_handle->qpair || + !seg_handle->ns) { + return -1; + } + + struct spdk_nvme_qpair *qpair = seg_handle->qpair; + struct spdk_nvme_ns *ns = seg_handle->ns; + if (op == kSpdkNofOpRead) { + return spdk_nvme_ns_cmd_read(ns, qpair, ptr, lba, lba_count, cb_fn, + cb_ctx, 0); + } else if (op == kSpdkNofOpWrite) { + return spdk_nvme_ns_cmd_write(ns, qpair, ptr, lba, lba_count, cb_fn, + cb_ctx, 0); + } + return -1; +} + +SpdkWrapper::ProbeBuffer *SpdkWrapper::GetOrCreateProbeBuffer( + const std::string &tr_str, uint32_t block_size, std::string *error_reason) { + std::lock_guard lock(probe_buffers_mutex_); + auto &probe_buffer = probe_buffers_[tr_str]; + if (!probe_buffer) { + probe_buffer = std::make_unique(); + } + + if (probe_buffer->ptr != nullptr && probe_buffer->size == block_size) { + return probe_buffer.get(); + } + + if (probe_buffer->ptr != nullptr) { + spdk_free(probe_buffer->ptr); + probe_buffer->ptr = nullptr; + probe_buffer->size = 0; + } + + probe_buffer->ptr = + spdk_zmalloc(block_size, 0x1000, nullptr, -1, SPDK_MALLOC_DMA); + if (!probe_buffer->ptr) { + if (error_reason) { + *error_reason = "alloc_fail"; + } + return nullptr; + } + probe_buffer->size = block_size; + return probe_buffer.get(); +} + +bool SpdkWrapper::ProbeNofSegment(const std::string &tr_str, + uint32_t timeout_ms, + std::string *error_reason) { + if (!InitializeEnv()) { + if (error_reason) { + *error_reason = "spdk_env_init_fail"; + } + return false; + } + + nof_seg_handle *seg_handle = OpenNofSegment(tr_str); + if (!seg_handle) { + if (error_reason) { + *error_reason = "open_fail"; + } + return false; + } + + uint32_t block_size = GetBlockSize(seg_handle); + if (block_size == INVALID_BLOCK_SIZE || block_size == 0) { + if (error_reason) { + *error_reason = "invalid_block_size"; + } + return false; + } + + ProbeBuffer *probe_buffer = + GetOrCreateProbeBuffer(tr_str, block_size, error_reason); + if (!probe_buffer || !probe_buffer->ptr) { + return false; + } + + ProbeRequestContext *probe_ctx = AcquireProbeRequestContext(); + int ret = SubmitRequest(seg_handle, probe_buffer->ptr, 0, 1, kSpdkNofOpRead, + ProbeReadComplete, probe_ctx); + if (ret != 0) { + RecycleProbeRequestContext(probe_ctx); + if (error_reason) { + *error_reason = "submit_fail"; + } + return false; + } + + auto deadline = std::chrono::steady_clock::now() + + std::chrono::milliseconds(timeout_ms); + while (!probe_ctx->done.load(std::memory_order_acquire) && + std::chrono::steady_clock::now() < deadline) { + NvmePollProcessCompletion(seg_handle, 0); + std::this_thread::sleep_for(std::chrono::milliseconds(1)); + } + + bool ok = probe_ctx->done.load(std::memory_order_acquire) && + probe_ctx->success.load(std::memory_order_acquire); + if (!ok && error_reason) { + if (!probe_ctx->done.load(std::memory_order_acquire)) { + *error_reason = "completion_timeout"; + } else { + std::lock_guard lock(probe_ctx->error_mutex); + *error_reason = probe_ctx->error_reason.empty() + ? "completion_error" + : probe_ctx->error_reason; + } + } + + return ok; +} + +} // namespace mooncake diff --git a/mooncake-store/src/transfer_task.cpp b/mooncake-store/src/transfer_task.cpp index 0941536b..a0d88c5f 100644 --- a/mooncake-store/src/transfer_task.cpp +++ b/mooncake-store/src/transfer_task.cpp @@ -3,6 +3,9 @@ #include #include +#include +#include +#include #include #include #include @@ -10,9 +13,162 @@ #include "gpu_staging_utils.h" #include "transfer_engine.h" #include "transport/transport.h" +#ifdef USE_NOF +#include "spdk/spdk_wrapper.h" +#endif +#ifdef USE_NOF +static bool IsTruthyEnv(const char* value) { + if (!value) { + return false; + } + std::string normalized(value); + std::transform( + normalized.begin(), normalized.end(), normalized.begin(), + [](unsigned char c) { return static_cast(std::tolower(c)); }); + return normalized == "1" || normalized == "true" || normalized == "yes" || + normalized == "on"; +} + +static bool IsSpdkNofDebugEnabled() { + static const bool enabled = IsTruthyEnv(std::getenv("MC_NOF_DEBUG")); + return enabled; +} + +static int GetSpdkNofDebugIntervalMs() { + static const int interval_ms = []() { + const char* raw_value = std::getenv("MC_NOF_DEBUG_INTERVAL_MS"); + if (!raw_value) { + return 1000; + } + char* end_ptr = nullptr; + long parsed = std::strtol(raw_value, &end_ptr, 10); + if (end_ptr == raw_value || (end_ptr != nullptr && *end_ptr != '\0') || + parsed <= 0) { + return 1000; + } + return static_cast(parsed); + }(); + return interval_ms; +} + +static int GetPositiveEnvOrDefault(const char* name, int default_value) { + const char* raw_value = std::getenv(name); + if (!raw_value || raw_value[0] == '\0') { + return default_value; + } + + errno = 0; + char* end_ptr = nullptr; + long parsed = std::strtol(raw_value, &end_ptr, 10); + if (errno != 0 || end_ptr == raw_value || + (end_ptr != nullptr && *end_ptr != '\0') || parsed <= 0 || + parsed > std::numeric_limits::max()) { + LOG(WARNING) << "Invalid value for " << name << ": " << raw_value + << ", using default " << default_value; + return default_value; + } + + return static_cast(parsed); +} + +static int GetSpdkNofSubmitChunkBytes() { + static const int value = GetPositiveEnvOrDefault( + "MC_NOF_SUBMIT_CHUNK_BYTES", mooncake::kDefaultSpdkNofSubmitChunkBytes); + return value; +} + +static int GetSpdkNofInflightBytesLimit() { + static const int value = + GetPositiveEnvOrDefault("MC_NOF_INFLIGHT_BYTES_LIMIT", + mooncake::kDefaultSpdkNofInflightBytesLimit); + return value; +} + +static int GetSpdkNofWorkerCount() { + static const int value = GetPositiveEnvOrDefault( + "MC_NOF_WORKERS", mooncake::kDefaultSpdkNofWorkers); + return value; +} + +static int CountSpdkNofQueuedTasks(const mooncake::SpdkNofTask* head) { + int count = 0; + const mooncake::SpdkNofTask* cursor = head; + while (cursor != nullptr) { + ++count; + cursor = cursor->nxt; + } + return count; +} + +static inline void SpdkNofTaskCompletion(mooncake::SpdkNofTask* task) { + if (task->remaining_lba == 0 && task->outstanding_sub_io == 0) { + task->state->set_completed(task->failed + ? mooncake::ErrorCode::TRANSFER_FAIL + : mooncake::ErrorCode::OK); + if (!task->on_chain) { + delete task; + } + } +} + +static void nvmf_io_complete(void* ctx, const struct spdk_nvme_cpl* cpl) { + if (!ctx) { + LOG(ERROR) << "nvmf_io_complete ctx is null"; + return; + } + + mooncake::SpdkNofSubTask* sub_task = + reinterpret_cast(ctx); + mooncake::SpdkNofTask* task = sub_task->task; + mooncake::SpdkNofQos* nof_qos = task->nof_qos; + int op = task->op; + if (--(*task->io_count) < 0) { + LOG(ERROR) << "total outstanding io < 0"; + } + + if (--(task->outstanding_sub_io) < 0) { + LOG(ERROR) << "task outstanding io < 0"; + } + + nof_qos->inflight_blocks[op] -= sub_task->submit_lba_count; + if (nof_qos->inflight_blocks[op] < 0) { + LOG(ERROR) << "task outstanding io < 0"; + } + + if (spdk_nvme_cpl_is_error(cpl)) { + LOG(ERROR) << "task_complete: I/O failed" + << spdk_nvme_cpl_get_status_string(&cpl->status); + task->remaining_lba = 0; + task->failed = true; + } + + SpdkNofTaskCompletion(task); + + sub_task->sub_task_pool->push(sub_task); +} +#endif namespace mooncake { +#ifdef USE_NOF +SpdkNofQos::SpdkNofQos(uint32_t block_size) { + int block_size_int = static_cast(block_size); + if (block_size_int <= 0) { + block_size_int = 1; + } + + blocks_per_chunk = + std::max(1, GetSpdkNofSubmitChunkBytes() / block_size_int); + inflight_blocks_limit = + std::max(1, GetSpdkNofInflightBytesLimit() / block_size_int); + for (int i = 0; i < kSpdkNofOpNum; ++i) { + inflight_blocks[i] = 0; + head[i] = nullptr; + tail[i] = nullptr; + } +} +#endif + // ============================================================================ // FilereadWorkerPool Implementation // ============================================================================ @@ -120,6 +276,304 @@ void FilereadWorkerPool::workerThread() { VLOG(2) << "FilereadWorkerPool worker thread exiting"; } +// ============================================================================ +// SpdkNofWorkerPool Implementation +// ============================================================================ +// to fully utilize the available ssd bandwidth, we use a default of 4 worker +// threads. + +#ifdef USE_NOF +SpdkNofWorkerPool::SpdkNofWorkerPool(int numa_socket_id) + : worker_count_(GetSpdkNofWorkerCount()), + numa_socket_id_(numa_socket_id), + task_queue_(std::make_unique[]>(worker_count_)), + queue_mutex_(std::make_unique(worker_count_)), + queue_cv_(std::make_unique(worker_count_)), + shutdown_(false) { + VLOG(1) << "Creating SpdkNofWorkerPool with " << worker_count_ + << " workers"; + + // Start worker threads + workers_.reserve(worker_count_); + for (int i = 0; i < worker_count_; ++i) { + workers_.emplace_back(&SpdkNofWorkerPool::workerThread, this, i); + } +} + +SpdkNofWorkerPool::~SpdkNofWorkerPool() { + if (shutdown_.exchange(true)) { + return; + } + + for (int i = 0; i < worker_count_; ++i) { + queue_cv_[i].notify_all(); + } + + for (auto& worker : workers_) { + if (worker.joinable()) { + worker.join(); + } + } + + VLOG(1) << "SpdkNofWorkerPool destroyed"; +} + +void SpdkNofWorkerPool::submitTask(SpdkNofTask task) { + if (!task.state) { + LOG(ERROR) << "Attempting to submit spdk nof task without state"; + return; + } + + if (shutdown_.load()) { + LOG(WARNING) + << "Attempting to submit task to shutdown SpdkNofWorkerPool"; + task.state->set_completed(ErrorCode::TRANSFER_FAIL); + return; + } + + int worker_idx = -1; + { + std::lock_guard lock(seg_mutex_); + nof_seg_handle* seg = task.seg_handle; + bool new_binding = false; + if (seg_to_worker_.find(seg) != seg_to_worker_.end()) { + worker_idx = seg_to_worker_[seg]; + } else { + worker_idx = (seg_num++ % worker_count_); + seg_to_worker_[seg] = worker_idx; + new_binding = true; + } + if (new_binding && IsSpdkNofDebugEnabled()) { + LOG(INFO) << "nof_worker_bind seg_handle=" << seg + << " worker_idx=" << worker_idx; + } + } + if (worker_idx < 0 || worker_idx >= worker_count_) { + LOG(ERROR) << "seg is not bind to invalid worker " << worker_idx; + task.state->set_completed(ErrorCode::TRANSFER_FAIL); + return; + } + + { + std::lock_guard lock(queue_mutex_[worker_idx]); + task_queue_[worker_idx].push(std::move(task)); + } + queue_cv_[worker_idx].notify_one(); +} + +static bool HasBufferedTask( + const std::map>& seg_to_qos) { + for (const auto& [_, nof_qos] : seg_to_qos) { + if (!nof_qos->Empty()) { + return true; + } + } + return false; +} + +constexpr int kSpdkNofSubTaskChunkSize = 4096; + +static inline bool CheckSubTaskPool( + std::stack& sub_task_pool, + std::vector& sub_task_chunks, int work_idx) { + if (!sub_task_pool.empty()) { + return true; + } + SpdkNofSubTask* sub_tasks = + new (std::nothrow) SpdkNofSubTask[kSpdkNofSubTaskChunkSize]; + if (!sub_tasks) { + LOG(ERROR) << "alloc SpdkNofSubTask failed, worker " << work_idx; + return false; + } + sub_task_chunks.push_back(sub_tasks); + + for (int i = 0; i < kSpdkNofSubTaskChunkSize; ++i) { + sub_tasks[i].sub_task_pool = &sub_task_pool; + sub_task_pool.push(&sub_tasks[i]); + } + + return true; +} + +void SpdkNofWorkerPool::workerThread(int work_idx) { + bindToSocket(numa_socket_id_); + VLOG(2) << "SpdkNofWorkerPool worker thread started"; + + int64_t total_outstanding_io = 0; + // std::set seg_set; + std::map> seg_to_qos; + std::stack sub_task_pool; + std::vector sub_task_chunks; + auto& task_queue = task_queue_[work_idx]; + auto& queue_cv = queue_cv_[work_idx]; + auto& queue_mutex = queue_mutex_[work_idx]; + auto last_debug_snapshot = std::chrono::steady_clock::now(); + + if (!CheckSubTaskPool(sub_task_pool, sub_task_chunks, work_idx)) { + return; + } + + while (true) { + // Wait for task or shutdown signal + { + std::unique_lock lock(queue_mutex); + queue_cv.wait( + lock, [this, &task_queue, &total_outstanding_io, &seg_to_qos] { + return shutdown_.load() || !task_queue.empty() || + total_outstanding_io || HasBufferedTask(seg_to_qos); + }); + + if (shutdown_.load() && task_queue.empty() && + (total_outstanding_io == 0) && !HasBufferedTask(seg_to_qos)) { + break; + } + + while (!task_queue.empty()) { + SpdkNofTask* task = new (std::nothrow) + SpdkNofTask(std::move(task_queue.front())); + if (task == nullptr) { + LOG(ERROR) + << "alloc SpdkNofTask failed, worker " << work_idx; + continue; + } + + SpdkNofQos* nof_qos = nullptr; + auto it = seg_to_qos.find(task->seg_handle); + if (it == seg_to_qos.end()) { + auto qos = std::make_unique( + SpdkWrapper::GetInstance().GetBlockSize( + task->seg_handle)); + if (qos == nullptr) { + LOG(ERROR) + << "alloc SpdkNofQos failed, worker " << work_idx; + delete task; + continue; + } + nof_qos = qos.get(); + seg_to_qos[task->seg_handle] = std::move(qos); + if (IsSpdkNofDebugEnabled()) { + LOG(INFO) + << "nof_qos_create worker_idx=" << work_idx + << " seg_handle=" << task->seg_handle + << " blocks_per_chunk=" + << seg_to_qos[task->seg_handle]->blocks_per_chunk + << " inflight_blocks_limit=" + << seg_to_qos[task->seg_handle] + ->inflight_blocks_limit; + } + } else { + nof_qos = it->second.get(); + } + task->io_count = &total_outstanding_io; + task->nof_qos = nof_qos; + task->on_chain = true; + nof_qos->PushTask(task); + task_queue.pop(); + } + } + + for (auto& [seg_handle, nof_qos] : seg_to_qos) { + uint32_t block_size = + SpdkWrapper::GetInstance().GetBlockSize(seg_handle); + for (int i = 0; i < kSpdkNofOpNum; ++i) { + int avail_blocks = nof_qos->inflight_blocks_limit - + nof_qos->inflight_blocks[i]; + while (nof_qos->head[i] && avail_blocks > 0) { + SpdkNofTask* task = nof_qos->head[i]; + SpdkNofSubTask* sub_task; + while (task->remaining_lba > 0 && avail_blocks > 0) { + uint32_t submit_lba_count = std::min( + avail_blocks, std::min(task->remaining_lba, + nof_qos->blocks_per_chunk)); + int lba_off = task->lba_count - task->remaining_lba; + uint64_t submit_lba = task->lba + lba_off; + void* submit_ptr = reinterpret_cast( + reinterpret_cast(task->ptr) + + lba_off * block_size); + + if (!CheckSubTaskPool(sub_task_pool, sub_task_chunks, + work_idx)) { + task->failed = true; + task->remaining_lba = 0; + break; + } + sub_task = sub_task_pool.top(); + sub_task_pool.pop(); + sub_task->task = task; + sub_task->submit_lba_count = submit_lba_count; + + int ret = SpdkWrapper::GetInstance().SubmitRequest( + task->seg_handle, submit_ptr, submit_lba, + submit_lba_count, task->op, nvmf_io_complete, + sub_task); + if (ret != 0) { + LOG(ERROR) << "work " << work_idx << ", seg " + << task->seg_handle << " submit io fail"; + task->failed = true; + task->remaining_lba = 0; + } else { + task->idx++; + task->remaining_lba -= submit_lba_count; + nof_qos->inflight_blocks[i] += submit_lba_count; + avail_blocks -= submit_lba_count; + task->outstanding_sub_io++; + total_outstanding_io++; + } + } + if (task->remaining_lba == 0) { + nof_qos->PopTask(i); + task->on_chain = false; + SpdkNofTaskCompletion(task); + } + } + } + } + + if (total_outstanding_io > 0) { + int64_t ret = 0; + for (auto& it : seg_to_qos) { + ret = SpdkWrapper::GetInstance().NvmePollProcessCompletion( + it.first, 0); + if (ret < 0) { + LOG(ERROR) << "poll completion error: ret " << ret; + } + } + } + + if (IsSpdkNofDebugEnabled()) { + auto now = std::chrono::steady_clock::now(); + auto elapsed = + std::chrono::duration_cast( + now - last_debug_snapshot); + if (elapsed.count() >= GetSpdkNofDebugIntervalMs()) { + for (const auto& [seg_handle, nof_qos] : seg_to_qos) { + LOG(INFO) + << "nof_qos_state worker_idx=" << work_idx + << " seg_handle=" << seg_handle + << " inflight_read=" << nof_qos->inflight_blocks[0] + << " inflight_write=" << nof_qos->inflight_blocks[1] + << " inflight_limit=" << nof_qos->inflight_blocks_limit + << " queued_read=" + << CountSpdkNofQueuedTasks(nof_qos->head[0]) + << " queued_write=" + << CountSpdkNofQueuedTasks(nof_qos->head[1]) + << " total_outstanding_io=" << total_outstanding_io; + } + last_debug_snapshot = now; + } + } + } + + for (auto* sub_tasks : sub_task_chunks) { + delete[] sub_tasks; + } + // seg_to_qos will automatically clean up SpdkNofQos objects using + // unique_ptr + + VLOG(2) << "SpdkNofWorkerPool worker thread exiting"; +} +#endif + // ============================================================================ // MemcpyWorkerPool Implementation // ============================================================================ @@ -449,10 +903,14 @@ TransferStrategy TransferFuture::strategy() const { TransferSubmitter::TransferSubmitter(TransferEngine& engine, std::shared_ptr& backend, const std::string& local_hostname, - TransferMetric* transfer_metric) + TransferMetric* transfer_metric, + int numa_socket_id) : engine_(engine), local_endpoint_(engine.getLocalIpAndPort()), memcpy_pool_(std::make_unique()), +#ifdef USE_NOF + spdk_nvmf_pool_(std::make_unique(numa_socket_id)), +#endif fileread_pool_(std::make_unique(backend)), local_hostname_(local_hostname), transfer_metric_(transfer_metric) { @@ -491,7 +949,7 @@ TransferSubmitter::TransferSubmitter(TransferEngine& engine, std::optional TransferSubmitter::submit( const Replica::Descriptor& replica, std::vector& slices, - TransferRequest::OpCode op_code) { + TransferRequest::OpCode op_code, void* ptr, size_t size) { std::optional future; if (replica.is_memory_replica()) { @@ -520,6 +978,20 @@ std::optional TransferSubmitter::submit( return std::nullopt; } } + } else if (replica.is_nof_replica()) { +#ifdef USE_NOF + auto& ssd_desc = replica.get_nof_descriptor(); + auto& handle = ssd_desc.buffer_descriptor; + + if (!ptr || (size == 0)) { + return std::nullopt; + } + + future = submitSpdkNofOperation(handle, ptr, size, op_code); +#else + LOG(ERROR) << "NoF transfer requested while USE_NOF is disabled"; + return std::nullopt; +#endif } else { future = submitFileReadOperation(replica, slices, op_code); } @@ -769,6 +1241,9 @@ std::optional TransferSubmitter::submitRangeRead( } future = submitMemoryReadOperation(handle, slices, src_offset); + } else if (replica.is_nof_replica()) { + LOG(ERROR) << "Range read not supported for NoF replicas"; + return std::nullopt; } else if (replica.is_disk_replica() || replica.is_local_disk_replica()) { LOG(ERROR) << "Range read not supported for disk replicas (use full read)"; @@ -782,6 +1257,46 @@ std::optional TransferSubmitter::submitRangeRead( return future; } +#ifdef USE_NOF +std::optional TransferSubmitter::submitSpdkNofOperation( + const AllocatedBuffer::Descriptor& handle, void* ptr, size_t size, + const TransferRequest::OpCode op_code) { + if (handle.transport_endpoint_.empty() || handle.size_ < size) { + LOG(ERROR) << "Invalid NoF request endpoint=" + << handle.transport_endpoint_ + << ", buffer_size=" << handle.size_ + << ", request_size=" << size; + return std::nullopt; + } + + nof_seg_handle* seg_handle = + SpdkWrapper::GetInstance().OpenNofSegment(handle.transport_endpoint_); + if (!seg_handle) { + LOG(ERROR) << "Failed to open NoF segment endpoint=" + << handle.transport_endpoint_; + return std::nullopt; + } + + uint32_t block_size = SpdkWrapper::GetInstance().GetBlockSize(seg_handle); + if (block_size == INVALID_BLOCK_SIZE || + handle.buffer_address_ % block_size != 0 || size % block_size != 0 || + reinterpret_cast(ptr) % block_size != 0) { + LOG(ERROR) << "NoF request offset=" << handle.buffer_address_ + << ", ptr=" << ptr << ", size=" << size + << " is not aligned to block size " << block_size; + return std::nullopt; + } + + auto state = std::make_shared(); + SpdkNofTask task(seg_handle, ptr, handle.buffer_address_ / block_size, + size / block_size, op_code, state); + spdk_nvmf_pool_->submitTask(std::move(task)); + + VLOG(1) << "SPDK NoF transfer submitted to " << handle.transport_endpoint_; + return TransferFuture(state); +} +#endif + std::optional TransferSubmitter::submitFileReadOperation( const Replica::Descriptor& replica, std::vector& slices, TransferRequest::OpCode op_code) { diff --git a/mooncake-store/src/utils.cpp b/mooncake-store/src/utils.cpp index 70a8b209..2b9639fd 100644 --- a/mooncake-store/src/utils.cpp +++ b/mooncake-store/src/utils.cpp @@ -43,6 +43,10 @@ DEFINE_uint64(mmap_arena_pool_size, 8ULL * 1024 * 1024 * 1024, #include "ascend_allocator.h" #endif +#ifdef USE_NOF +#include "spdk/spdk_wrapper.h" +#endif + #include namespace mooncake { @@ -101,7 +105,7 @@ AutoPortBinder::~AutoPortBinder() { void *allocate_buffer_allocator_memory(size_t total_size, const std::string &protocol, - size_t alignment) { + size_t alignment, bool use_spdk_dma) { const size_t default_alignment = facebook::cachelib::Slab::kSize; // Ensure total_size is a multiple of alignment if (alignment == default_alignment && total_size < alignment) { @@ -113,7 +117,12 @@ void *allocate_buffer_allocator_memory(size_t total_size, return ascend_allocate_memory(total_size, protocol); } #endif - +#ifdef USE_NOF + if (use_spdk_dma && total_size > 0) { + return mooncake::SpdkWrapper::GetInstance().Alloc(total_size, alignment, + -1); + } +#endif // Allocate aligned memory return aligned_alloc(alignment, total_size); } diff --git a/mooncake-store/tests/CMakeLists.txt b/mooncake-store/tests/CMakeLists.txt index 07b7fc86..c1048073 100644 --- a/mooncake-store/tests/CMakeLists.txt +++ b/mooncake-store/tests/CMakeLists.txt @@ -46,6 +46,9 @@ add_store_test(master_service_ssd_test_for_snapshot ha/snapshot/master_service_ssd_test_for_snapshot.cpp) add_store_test(master_service_promotion_test_for_snapshot ha/snapshot/master_service_promotion_test_for_snapshot.cpp) +if(USE_NOF) + add_store_test(nof_heartbeat_test nof_heartbeat_test.cpp) +endif() add_store_test(client_integration_test client_integration_test.cpp) if(USE_CXL) add_store_test(cxl_client_integration_test cxl_client_integration_test.cpp) diff --git a/mooncake-store/tests/master_metrics_test.cpp b/mooncake-store/tests/master_metrics_test.cpp index 900a8b57..b57dcc2d 100644 --- a/mooncake-store/tests/master_metrics_test.cpp +++ b/mooncake-store/tests/master_metrics_test.cpp @@ -636,7 +636,7 @@ TEST_F(MasterMetricsTest, PutStartReplicaAllocationFailureMetric) { ASSERT_EQ(metrics.get_put_start_failures(), put_start_failures_before + 1); } -TEST_F(MasterMetricsTest, SummaryUsesWindowRatesAndEvictionDeltas) { +TEST_F(MasterMetricsTest, SummaryUsesWindowRatesAndCumulativeEviction) { auto& metrics = MasterMetricManager::instance(); const std::string baseline_summary = @@ -651,6 +651,10 @@ TEST_F(MasterMetricsTest, SummaryUsesWindowRatesAndEvictionDeltas) { metrics.inc_batch_put_start_partial_success(2); metrics.inc_eviction_success(3, 4096); metrics.inc_eviction_fail(); + metrics.inc_mem_eviction_success(3, 4096); + metrics.inc_mem_eviction_fail(); + metrics.inc_nof_eviction_success(1, 2048); + metrics.inc_nof_eviction_fail(); std::this_thread::sleep_for(std::chrono::milliseconds(20)); const std::string window_summary = metrics.get_summary_string(); @@ -665,6 +669,12 @@ TEST_F(MasterMetricsTest, SummaryUsesWindowRatesAndEvictionDeltas) { window_summary.find("Eviction: Success/Attempts=1/2, AllocFail=0, " "keys=3, size=4.00 KB"), std::string::npos); + EXPECT_NE(window_summary.find("Mem Eviction: Success/Attempts=1/2, " + "keys=3, size=4.00 KB"), + std::string::npos); + EXPECT_NE(window_summary.find("NoF Eviction: Success/Attempts=1/2, " + "keys=1, size=2.00 KB"), + std::string::npos); const std::string reported_summary = metrics.get_summary_string_and_update_snapshot(); @@ -677,8 +687,14 @@ TEST_F(MasterMetricsTest, SummaryUsesWindowRatesAndEvictionDeltas) { const std::string idle_summary = metrics.get_summary_string_and_update_snapshot(); EXPECT_NE(idle_summary.find("PutStart=0.00/0.00"), std::string::npos); - EXPECT_NE(idle_summary.find("Eviction: Success/Attempts=0/0, " - "AllocFail=0, keys=0, size=0 B"), + EXPECT_NE(idle_summary.find("Eviction: Success/Attempts=1/2, " + "AllocFail=0, keys=3, size=4.00 KB"), + std::string::npos); + EXPECT_NE(idle_summary.find("Mem Eviction: Success/Attempts=1/2, " + "keys=3, size=4.00 KB"), + std::string::npos); + EXPECT_NE(idle_summary.find("NoF Eviction: Success/Attempts=1/2, " + "keys=1, size=2.00 KB"), std::string::npos); } diff --git a/mooncake-store/tests/nof_heartbeat_test.cpp b/mooncake-store/tests/nof_heartbeat_test.cpp new file mode 100644 index 00000000..765e8cdc --- /dev/null +++ b/mooncake-store/tests/nof_heartbeat_test.cpp @@ -0,0 +1,249 @@ +#include "master_service.h" + +#include +#include + +#include +#include +#include +#include + +namespace mooncake::test { + +namespace { + +constexpr size_t kDefaultNoFSegmentBase = 0x500000000; +constexpr size_t kDefaultNoFSegmentSize = 1024 * 1024 * 16; + +NoFSegment MakeNoFSegment(std::string name, std::string endpoint, + size_t base = kDefaultNoFSegmentBase, + size_t size = kDefaultNoFSegmentSize) { + NoFSegment segment; + segment.id = generate_uuid(); + segment.name = std::move(name); + segment.base = base; + segment.size = size; + segment.te_endpoint = std::move(endpoint); + return segment; +} + +bool WaitForCondition(std::chrono::milliseconds timeout, + std::chrono::milliseconds interval, + const std::function& condition) { + auto deadline = std::chrono::steady_clock::now() + timeout; + while (std::chrono::steady_clock::now() < deadline) { + if (condition()) { + return true; + } + std::this_thread::sleep_for(interval); + } + return condition(); +} + +} // namespace + +class NoFHeartbeatTest : public ::testing::Test { + protected: + void SetUp() override { + google::InitGoogleLogging("NoFHeartbeatTest"); + FLAGS_logtostderr = true; + } + + void TearDown() override { google::ShutdownGoogleLogging(); } + + std::unique_ptr CreateService(int64_t heartbeat_interval_sec, + uint32_t probe_timeout_ms, + uint32_t failure_threshold, + int64_t client_ttl_sec = 10) { + auto config = + MasterServiceConfig::builder() + .set_memory_allocator(BufferAllocatorType::OFFSET) + .set_client_live_ttl_sec(client_ttl_sec) + .set_nof_heartbeat_interval_sec(heartbeat_interval_sec) + .set_nof_heartbeat_probe_timeout_ms(probe_timeout_ms) + .set_nof_heartbeat_failures_threshold(failure_threshold) + .build(); + return std::make_unique(config); + } +}; + +TEST_F(NoFHeartbeatTest, HealthyNoFSegmentDoesNotUnmount) { + auto service = CreateService(/*heartbeat_interval_sec=*/1, + /*probe_timeout_ms=*/50, + /*failure_threshold=*/3); + std::atomic probe_calls{0}; + service->SetNoFProbeFnForTesting( + [&probe_calls](const std::string&, uint32_t, std::string*) { + probe_calls.fetch_add(1, std::memory_order_relaxed); + return true; + }); + + UUID client_id = generate_uuid(); + NoFSegment segment = MakeNoFSegment("nof_seg_ok", "nof_ok"); + ASSERT_TRUE(service->MountNoFSegment(segment, client_id).has_value()); + + ASSERT_TRUE(WaitForCondition(std::chrono::milliseconds(2500), + std::chrono::milliseconds(50), + [&]() { return probe_calls.load() >= 1; })); + EXPECT_TRUE(service->IsNoFSegmentMountedForTesting(segment.id)); + EXPECT_EQ(service->GetMountedNoFSegmentCountForTesting(), 1u); + auto failure_count = + service->GetNoFHeartbeatFailureCountForTesting(segment.id); + ASSERT_TRUE(failure_count.has_value()); + EXPECT_EQ(*failure_count, 0u); +} + +TEST_F(NoFHeartbeatTest, NewlyMountedNoFSegmentHasInitialGracePeriod) { + auto service = CreateService(/*heartbeat_interval_sec=*/2, + /*probe_timeout_ms=*/50, + /*failure_threshold=*/1); + std::atomic probe_calls{0}; + service->SetNoFProbeFnForTesting( + [&probe_calls](const std::string&, uint32_t, std::string* reason) { + probe_calls.fetch_add(1, std::memory_order_relaxed); + if (reason) { + *reason = "submit_fail"; + } + return false; + }); + + UUID client_id = generate_uuid(); + NoFSegment segment = MakeNoFSegment("nof_seg_grace", "nof_grace"); + ASSERT_TRUE(service->MountNoFSegment(segment, client_id).has_value()); + + std::this_thread::sleep_for(std::chrono::milliseconds(800)); + + EXPECT_EQ(probe_calls.load(), 0); + EXPECT_TRUE(service->IsNoFSegmentMountedForTesting(segment.id)); + auto failure_count = + service->GetNoFHeartbeatFailureCountForTesting(segment.id); + ASSERT_TRUE(failure_count.has_value()); + EXPECT_EQ(*failure_count, 0u); +} + +TEST_F(NoFHeartbeatTest, FailedNoFSegmentUnmountsAfterThreshold) { + auto service = CreateService(/*heartbeat_interval_sec=*/1, + /*probe_timeout_ms=*/50, + /*failure_threshold=*/3); + std::atomic probe_calls{0}; + service->SetNoFProbeFnForTesting( + [&probe_calls](const std::string&, uint32_t, std::string* reason) { + probe_calls.fetch_add(1, std::memory_order_relaxed); + if (reason) { + *reason = "submit_fail"; + } + return false; + }); + + UUID client_id = generate_uuid(); + NoFSegment segment = MakeNoFSegment("nof_seg_fail", "nof_fail"); + ASSERT_TRUE(service->MountNoFSegment(segment, client_id).has_value()); + + ASSERT_TRUE(WaitForCondition( + std::chrono::milliseconds(5000), std::chrono::milliseconds(50), + [&]() { return !service->IsNoFSegmentMountedForTesting(segment.id); })); + EXPECT_GE(probe_calls.load(), 3); + EXPECT_EQ(service->GetMountedNoFSegmentCountForTesting(), 0u); + EXPECT_FALSE( + service->GetNoFHeartbeatFailureCountForTesting(segment.id).has_value()); +} + +TEST_F(NoFHeartbeatTest, FailureCountResetsAfterRecovery) { + auto service = CreateService(/*heartbeat_interval_sec=*/1, + /*probe_timeout_ms=*/50, + /*failure_threshold=*/3); + std::atomic probe_calls{0}; + service->SetNoFProbeFnForTesting( + [&probe_calls](const std::string&, uint32_t, std::string* reason) { + int current = probe_calls.fetch_add(1, std::memory_order_relaxed); + if (current < 2) { + if (reason) { + *reason = "submit_fail"; + } + return false; + } + return true; + }); + + UUID client_id = generate_uuid(); + NoFSegment segment = MakeNoFSegment("nof_seg_recover", "nof_recover"); + ASSERT_TRUE(service->MountNoFSegment(segment, client_id).has_value()); + + ASSERT_TRUE(WaitForCondition( + std::chrono::milliseconds(5000), std::chrono::milliseconds(50), [&]() { + auto failure_count = + service->GetNoFHeartbeatFailureCountForTesting(segment.id); + return probe_calls.load() >= 4 && failure_count.has_value() && + *failure_count == 0; + })); + EXPECT_TRUE(service->IsNoFSegmentMountedForTesting(segment.id)); +} + +TEST_F(NoFHeartbeatTest, OnlyFailedSegmentIsUnmounted) { + auto service = CreateService(/*heartbeat_interval_sec=*/1, + /*probe_timeout_ms=*/50, + /*failure_threshold=*/2); + std::atomic good_probe_calls{0}; + std::atomic bad_probe_calls{0}; + service->SetNoFProbeFnForTesting( + [&good_probe_calls, &bad_probe_calls](const std::string& endpoint, + uint32_t, std::string* reason) { + if (endpoint == "nof_good") { + good_probe_calls.fetch_add(1, std::memory_order_relaxed); + return true; + } + bad_probe_calls.fetch_add(1, std::memory_order_relaxed); + if (reason) { + *reason = "submit_fail"; + } + return false; + }); + + UUID client_id = generate_uuid(); + NoFSegment good_segment = MakeNoFSegment("nof_seg_good", "nof_good"); + NoFSegment bad_segment = + MakeNoFSegment("nof_seg_bad", "nof_bad", + kDefaultNoFSegmentBase + kDefaultNoFSegmentSize, + kDefaultNoFSegmentSize); + ASSERT_TRUE(service->MountNoFSegment(good_segment, client_id).has_value()); + ASSERT_TRUE(service->MountNoFSegment(bad_segment, client_id).has_value()); + + ASSERT_TRUE(WaitForCondition( + std::chrono::milliseconds(5000), std::chrono::milliseconds(50), [&]() { + return service->GetMountedNoFSegmentCountForTesting() == 1u; + })); + EXPECT_TRUE(service->IsNoFSegmentMountedForTesting(good_segment.id)); + EXPECT_FALSE(service->IsNoFSegmentMountedForTesting(bad_segment.id)); + EXPECT_GE(good_probe_calls.load(), 1); + EXPECT_GE(bad_probe_calls.load(), 2); +} + +TEST_F(NoFHeartbeatTest, ClientExpiryDoesNotUnmountNoFSegment) { + auto service = CreateService(/*heartbeat_interval_sec=*/5, + /*probe_timeout_ms=*/50, + /*failure_threshold=*/1, + /*client_ttl_sec=*/1); + service->SetNoFProbeFnForTesting( + [](const std::string&, uint32_t, std::string* reason) { + if (reason) { + *reason = "submit_fail"; + } + return false; + }); + + UUID client_id = generate_uuid(); + NoFSegment segment = + MakeNoFSegment("nof_seg_ignore_client_ttl", "nof_ignore_client_ttl"); + ASSERT_TRUE(service->MountNoFSegment(segment, client_id).has_value()); + + std::this_thread::sleep_for(std::chrono::milliseconds(1800)); + + EXPECT_TRUE(service->IsNoFSegmentMountedForTesting(segment.id)); + EXPECT_EQ(service->GetMountedNoFSegmentCountForTesting(), 1u); + auto failure_count = + service->GetNoFHeartbeatFailureCountForTesting(segment.id); + ASSERT_TRUE(failure_count.has_value()); + EXPECT_EQ(*failure_count, 0u); +} + +} // namespace mooncake::test diff --git a/mooncake-store/tests/transfer_task_test.cpp b/mooncake-store/tests/transfer_task_test.cpp index d9c2ee9e..f841831c 100644 --- a/mooncake-store/tests/transfer_task_test.cpp +++ b/mooncake-store/tests/transfer_task_test.cpp @@ -180,6 +180,9 @@ TEST_F(TransferTaskTest, TransferStrategyEnum) { // Test enum values EXPECT_EQ(static_cast(TransferStrategy::LOCAL_MEMCPY), 0); EXPECT_EQ(static_cast(TransferStrategy::TRANSFER_ENGINE), 1); + EXPECT_EQ(static_cast(TransferStrategy::FILE_READ), 2); + EXPECT_EQ(static_cast(TransferStrategy::EMPTY), 3); + EXPECT_EQ(static_cast(TransferStrategy::SPDK_NVMF), 4); // Test stream operator std::ostringstream oss; @@ -189,6 +192,18 @@ TEST_F(TransferTaskTest, TransferStrategyEnum) { oss.str(""); oss << TransferStrategy::TRANSFER_ENGINE; EXPECT_EQ(oss.str(), "TRANSFER_ENGINE"); + + oss.str(""); + oss << TransferStrategy::SPDK_NVMF; + EXPECT_EQ(oss.str(), "SPDK_NVMF"); + + oss.str(""); + oss << TransferStrategy::FILE_READ; + EXPECT_EQ(oss.str(), "FILE_READ"); + + oss.str(""); + oss << TransferStrategy::EMPTY; + EXPECT_EQ(oss.str(), "EMPTY"); } } // namespace mooncake -- 2.34.1 From 7d26a326e7fbd7cd0d0319128290f19ad1fec4ba Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Thu, 28 May 2026 11:44:43 +0800 Subject: [PATCH 139/382] [Store] (CI run_tests_with_ssd failed / promotion-on-hit failed) eliminate race conditions (#2235) * fix(transfer-engine): eliminate race condition in rePublishRpcMetaEntry Remove the redundant storage_plugin_->remove() call before set(). All storage backends (HTTP PUT, Redis SET, Etcd put) have upsert semantics, so remove-then-set creates a window where concurrent get() returns empty / 404, causing transfer failures (-800). Also change Json::UInt64 to Json::UInt for rpc_port to ensure existing == desired comparison works correctly after JSON parse. * fix: adjust eviction thread initial timing to prevent race in CI SSD tests Start last_discard_time with an already-elapsed window so the first loop iteration triggers DiscardExpiredProcessingReplicas immediately. Without this, a task admitted shortly after thread startup can survive the first reaper cycle and not be cleaned until ~2s later, causing promotion-on-hit tests that sleep for 2s to flake. --- mooncake-store/src/master_service.cpp | 8 +++++++- mooncake-transfer-engine/src/transfer_metadata.cpp | 9 +++------ 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index bd53cf39..6177989c 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -3476,7 +3476,13 @@ auto MasterService::NotifyPromotionFailure(const UUID& client_id, void MasterService::EvictionThreadFunc() { VLOG(1) << "action=eviction_thread_started"; - auto last_discard_time = std::chrono::system_clock::now(); + // Start with an already-elapsed window so the first loop iteration + // (after kEvictionThreadSleepMs) triggers DiscardExpiredProcessingReplicas. + // Without this, a task admitted shortly after thread startup can survive + // the first reaper cycle and not be cleaned until ~2s later, causing + // promotion-on-hit tests that sleep for 2s to flake. + auto last_discard_time = + std::chrono::system_clock::now() - put_start_release_timeout_sec_; while (eviction_running_) { const auto now = std::chrono::system_clock::now(); double used_ratio = diff --git a/mooncake-transfer-engine/src/transfer_metadata.cpp b/mooncake-transfer-engine/src/transfer_metadata.cpp index 40a25856..10311838 100644 --- a/mooncake-transfer-engine/src/transfer_metadata.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata.cpp @@ -1128,7 +1128,7 @@ int TransferMetadata::addRpcMetaEntry(const std::string &server_name, Json::Value rpcMetaJSON; rpcMetaJSON["ip_or_host_name"] = desc.ip_or_host_name; - rpcMetaJSON["rpc_port"] = static_cast(desc.rpc_port); + rpcMetaJSON["rpc_port"] = static_cast(desc.rpc_port); if (!storage_plugin_->set(rpc_meta_prefix_ + server_name, rpcMetaJSON)) { LOG(ERROR) << "Failed to set location of " << server_name; return ERR_METADATA; @@ -1157,19 +1157,16 @@ int TransferMetadata::rePublishRpcMetaEntry(const std::string &server_name) { if (storage_plugin_->get(full_key, existing)) { Json::Value desired; desired["ip_or_host_name"] = local_rpc_meta_.ip_or_host_name; - desired["rpc_port"] = - static_cast(local_rpc_meta_.rpc_port); + desired["rpc_port"] = static_cast(local_rpc_meta_.rpc_port); if (existing == desired) { return 0; } - storage_plugin_->remove(full_key); } LOG(INFO) << "Re-publishing RPC meta entry for " << server_name; Json::Value rpcMetaJSON; rpcMetaJSON["ip_or_host_name"] = local_rpc_meta_.ip_or_host_name; - rpcMetaJSON["rpc_port"] = - static_cast(local_rpc_meta_.rpc_port); + rpcMetaJSON["rpc_port"] = static_cast(local_rpc_meta_.rpc_port); if (!storage_plugin_->set(full_key, rpcMetaJSON)) { LOG(ERROR) << "Failed to re-publish RPC meta entry for " << server_name; return ERR_METADATA; -- 2.34.1 From 0a0e952b1eff49234504bae674412bd5ec8aca66 Mon Sep 17 00:00:00 2001 From: Suryxin <76692062+Chelseatr@users.noreply.github.com> Date: Thu, 28 May 2026 11:57:16 +0800 Subject: [PATCH 140/382] [TransferEngine][docs] document FI_EFA_USE_DEVICE_RDMA=0 for same-host EFA loopback (#2222) * [TransferEngine] feat(efa): add MC_EFA_LOOPBACK_PREFER_EMULATED to recover same-host loopback throughput After #2041 ([TE] fix(efa): request libfabric API 1.18 so device RDMA is the default on all EFA generations) the EFA provider unconditionally enables device RDMA on every supported EFA hardware. This is the right default for cross-host transfers -- it is what unlocks the 300+ GB/s benchmarks documented in this file -- but it regresses any Mooncake Store deployment that runs producer and consumer as separate processes on the same host (single-machine development, single-host benchmarks, co-located workers). EFA NICs have no hardware loopback short-circuit: a "loopback" fi_write still drives a real DMA round-trip through the device (PCIe out, NIC SRD packet processing, PCIe back), so a same-host transfer pays full per-packet NIC overhead instead of taking the memcpy fast path that libfabric's emulated RDMA provider applies for same-host endpoints. Measured on p5.48xlarge (1 NIC, 80 MB transfer, two Mooncake Store clients on the same host, put_from): FI_EFA_USE_DEVICE_RDMA=1 (default after #2041): ~830 ms / call FI_EFA_USE_DEVICE_RDMA=0 (emulated): ~390 ms / call The 2.1x ratio is reproducible across runs; the emulated number is on par with what we measure for the same transfer cross-host with device RDMA on (~340 ms), once single-host memory-bandwidth contention is accounted for, confirming the slow path is NIC loopback rather than anything in the Mooncake Store layers above. Add MC_EFA_LOOPBACK_PREFER_EMULATED as an explicit opt-in. When set to 1/true/yes/on, EfaContext::construct sets FI_EFA_USE_DEVICE_RDMA=0 before fi_getinfo so the EFA provider takes the emulated path. The env is opt-in, not auto-detect, because a single EfaTransport instance may serve a mix of loopback and cross-host peers, and FI_EFA_USE_DEVICE_RDMA is a provider-level flag resolved at fi_getinfo time -- flipping it disables device RDMA for every transfer in the process, including cross-host ones, which is exactly the wrong behavior for production fan-out. We use setenv(..., 0) so an explicit FI_EFA_USE_DEVICE_RDMA set by the user still wins over the opt-in. Cross-host benchmarks are unaffected unless the env is also set on the cross-host process; the default behavior of this code path is unchanged. A real fix (per-transfer same-host memcpy or cross-process zero-copy via process_vm_writev) is tracked in a follow-up issue. This change is the minimal mitigation users need today to avoid silently giving up half their single-host throughput. * [TransferEngine] refactor(efa): address review feedback on MC_EFA_LOOPBACK_PREFER_EMULATED Per maintainer review on #2222: 1. Register MC_EFA_LOOPBACK_PREFER_EMULATED in the Environ singleton (mooncake-common/{include/environ.h,src/environ.cpp}) instead of parsing it ad-hoc at the call site, so it shows up in the same inventory as every other tunable and goes through the existing GetBool() helper (which already handles 1/true/TRUE/on/yes). 2. Guard the setenv("FI_EFA_USE_DEVICE_RDMA", "0", 0) block with !std::getenv("FI_EFA_USE_DEVICE_RDMA"). This fixes two bugs: - We no longer log "-> FI_EFA_USE_DEVICE_RDMA=0" when the user has already set the env explicitly (setenv is a no-op there, so the old log line was misleading). - EfaContext::construct runs once per NIC (up to 32 times on p5.48xlarge); the getenv check causes the first NIC to set the env and subsequent NICs to skip the block entirely, so we log exactly once. 3. Drop the hand-rolled std::transform + ::tolower entirely (which was UB on signed char anyway -- flagged by Copilot and gemini) by delegating to Environ::GetBool. Removes , adds for std::getenv/setenv. Behavior is unchanged for the same-host case in the verification table; this is purely structural cleanup. * [TransferEngine] docs(efa): correct transfer size in verification table The verification table in PR #2222 cited "80 MB transfer" as the per-call payload, but the actual measurement was per ~1.2 GiB (1218.8 MiB) ref blob (see ref_extractor log: blob_bytes=1218.8MiB put=489.88ms). Update both the docs section and the in-code comment to reflect the real transfer size. Latency numbers (~830 ms / ~390 ms / ~340 ms) are unchanged -- they were always measured on the 1.2 GiB blob. Also collapse two single-statement multi-line getters/initializers in mooncake-common to single-line form to match the existing convention in environ.{h,cpp} (all other GetX() accessors are single-line). No behavior change. * [TransferEngine] docs(efa): drop MC_EFA_LOOPBACK_PREFER_EMULATED wrapper, document FI_EFA_USE_DEVICE_RDMA=0 directly Per review feedback on #2222: the EFA user base is already familiar with FI_EFA_USE_DEVICE_RDMA (it is documented by the EFA installer and appears in every libfabric/EFA tuning guide), so wrapping it in a Mooncake-namespaced alias does not pay for itself. The wrapper was a literal one-to-one alias with no defaulting or transform. Revert the Environ registration and the efa_context.cpp setenv block (net code change for this PR becomes zero). Keep the diagnosis and the verification table in efa_transport.md, but rewrite the recommendation to point at FI_EFA_USE_DEVICE_RDMA=0 directly with the same per-process / mixed-traffic caveat. The long-term fix for same-host loopback (routing same-host different-process transfers through process_vm_writev as a new TransferStrategy::CROSS_PROCESS_MEMCPY, bypassing the NIC entirely) remains tracked as #2223. Co-Authored-By: Claude Opus 4.7 (1M context) --------- Co-authored-by: Claude Opus 4.7 (1M context) --- .../design/transfer-engine/efa_transport.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/source/design/transfer-engine/efa_transport.md b/docs/source/design/transfer-engine/efa_transport.md index 4bd3fbff..669d3a28 100644 --- a/docs/source/design/transfer-engine/efa_transport.md +++ b/docs/source/design/transfer-engine/efa_transport.md @@ -344,6 +344,21 @@ Tested on two p5.48xlarge instances (AMD EPYC 7R13, 8× H100 80GB, 32 EFA device > **Peak: ~64 GB/s** on both write and read — far below the GPU-to-GPU number despite identical NIC count. The bottleneck is DDR4-3200 DRAM bandwidth on the EPYC 7R13 (Milan): `batch=16` consistently wins because larger in-flight queues only deepen DRAM contention without unlocking new NIC capacity. p5.48xlarge CPU-to-CPU runs around **3× slower than p5en** (DDR5 Xeon 8488C, ~213 GB/s) at the same NIC aggregate. For PD KV transfer, the GPU-to-GPU path is the relevant one. +### Single-host loopback + +EFA NICs have no hardware loopback short-circuit: even when both endpoints resolve to the same host, `fi_write`/`fi_read` drive a real DMA round-trip through the EFA device. For deployments where the producer and consumer run as **separate processes on the same host** (single-machine development, benchmarks, co-located workers), this is strictly slower than libfabric's emulated RDMA path, which resolves the same-host case to a memcpy and skips the NIC entirely. + +Set `FI_EFA_USE_DEVICE_RDMA=0` (a libfabric provider env, [documented by the EFA installer](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-runtime-tuning.html)) on processes that only do same-host transfers; leave it unset (default `1`) on cross-host or mixed processes. Mooncake does not wrap this knob — it is a provider-level flag resolved at `fi_getinfo` time, and a single `EfaTransport` instance may serve a mix of loopback and cross-host peers, so flipping it disables device RDMA for **every** transfer in the process. Apply it per-process based on that process's traffic pattern. + +Measured on p5.48xlarge (1 NIC, ~1.2 GiB per `put_from` call, same-host producer/consumer): + +| `FI_EFA_USE_DEVICE_RDMA` | per-write latency | +|---|---:| +| `1` (default after #2041, device RDMA) | ~830 ms | +| `0` (emulated, same-host memcpy fast path) | ~390 ms | + +For reference, a cross-host `put_from` of the same payload (device RDMA, 1 NIC) is ~340 ms — i.e., device RDMA on a same-host loopback is *slower* than going over the wire to another host, because the NIC has no fast-path for loopback. Cross-host benchmarks are unaffected by this env: leave `FI_EFA_USE_DEVICE_RDMA` at its default on any process that also talks to remote peers. + ### Tuning Tips - **Use `--block_size=1048576` (1MB)** — the single most important knob. The 64 KB default reaches only ~26% of peak. 1 MB is within a few percent of the 2 MB plateau while leaving headroom for `batch_size` under the shared-endpoint WR cap. -- 2.34.1 From 49bd18757e23d44681ba4ca685891cd02cae988a Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Thu, 28 May 2026 13:22:22 +0800 Subject: [PATCH 141/382] [Doc] update overall picture (#2249) --- image/components.png | Bin 1604441 -> 977602 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/image/components.png b/image/components.png index 576cbb4db439e12910d4d6fa4ab56114b13595e9..e5d054139741b0b6531ecbee34327ec60683b392 100644 GIT binary patch literal 977602 zcmdqJc|2SF_b;kXTRKoxHAIQ0Ri$Vv)GV|$gtnxr#+s+pTtkScXseVWsTNfvN))B0 zn#a@_W6WZvf|!XZrf~Cof5W}^oO6HY{Bi#|=a2o`J3HBHt zEGOPOvz+?pC(KQNxv{XE{C0xnBy)e_pL5^-`j2lCcOr@v=qiD#WD6a_U;Tnda9J^ogBT_ECN{}8vFK(w}A zoj{X7mFDT2Pg%pze?8%RthQL1vn?fu z3#Vse1LBb)&pWOd1;oo0Sr#o_Ps;J;TnW#&wb(;w!@>_q8V-69BE*1Z2zIi{pvZOK zb=M1PMR^5C9?s!6nb+x347?nJ#w4#Y1z&4s^!SxfN@V4ygB+C!`4?g6qn3*xQuWJG zS!A9mKJW$P6$f|D&MK%4Q({2%di89&B{WLrJix-J^Tq@OPlPxJ@{Z8>@fJo7FXu+G z{F`YcK~N9(-og(r?P?Jv0njQ&9$-;nZz0RzQh)Sw$(Pj9EX761=Jlv@>f&+r{g>f( z16FtNho)EHKq5pGZP6lL^s&xpz->C{F7j1uDJ)e818OtUVuj^JBVmGT9bDYtm=v4< ztWo6?o{Mty;&WY`O0!&_Fs-PuQlA>-)^Q1x=kll2b;qJmo8FDy6}Vx%vMuxf^8m)I zh|n8>i@*Ed{5WaImT}>+P=c~F+~)s%VtsV-jp*&3wYS}Cbn`JZUJi))a7aT%1={fv z@+knY#E5Iy*0LNcJrK_HM;W*@KBrmZ81Wr6h<5?tCFlrkdLMeAg0)Pf7a#~ILhyKy ziJN}& zbI!}81c3%Al-)H95vg4UFxpfMfnA*d#}Gj=>M~Wt;dx9&^HMZ|ijf~aRLrXg+Y%>F zl@fb*vgT=~*NEFH&p8$nvc!icS|DzkZtKfm7H&V{{bHzGn6 zxC9t?de%u{$Au#3nn0rFQOhzAhF6~x$XA%gs*gBqQp)E9dhr$^qrgX__%xz@MAUxf zBq`hytNQ;L=YIv>6V4Xj7zkYY`FYiAm%(W~MSn1+F-qfF;2%Ln2`HLVA_{Ti_V|^; z5;}($80}9O1sx!)&@vL&7U%|LU%x=e!XP;(lU071m$9XR7Cu`LFzGS2HZM6fg9s3W zCgDr)OSV?m084H-@S-mV=nJt@qRcfwNCQYm#KOFicIRno2jcEs(|8hZOdf1UtsjwG z2g=KI3jCQLY%*W?k4CFNjx|)5mddYyl<^y%{!L@oEPwvn|4=AM50&lTQVgLu`xN{6 zg8Po5vA&${YtmJG2ZCTZAf23ct=Z&3r_*Ij?a=~FLQuD=*(}ogmBVJGpe_MZsdpWd zr*4<#WT*PYo(bK+@Sl5lAPl$TxqJmYAxcM-1=Wt-P(x;rt=G<~ECiwbmK_=uIo(t4 zbm{!t)l7w!+oS_{Lm&x-6jNH<9okHgBqhSUREJePXZ$gn2;MSmC#~;j`}Uqx`Q*xw zuC`KhK=mk>u6&AKS!=8P7@^M-)~E)>{THy5hEJ7Mo{VH%crO|4uc^3*B5tD2LSPk= zh7BsgjU8Vmc`8vRint#9i>mBIv3*$Uyg@PS$Cnm?3BVW{u@>S}=}m`Jiv;=d5@f5u zc>%oX_Ad;f4t@@T*zajA!eAU2T^Xjhh@E-Szkq^;qmY7z_CD$MW9UOG=VEN%N{f=( zLsd|Tl-%Q}4U6-(B3+1ExSTM5Cd?;XeVvT|05Nca2-cmKuvz}^(#dm8F(uWcd!v%u z26SRni2?cIaP?=bpT(xYu5X}lN zTXc?&_<10-;Kr0r{CDY9@^{cbXW^_PRzEP6>P|nSE5lmmu*`{%AghQg6~;dhhL5UH zySek&_CpRKyj>Z=_LaPG-i(=kbiM`}sn!eaAFDudk$H zl7{wjB6i>C&3XBFtBP2pf|2dp5vMEm4$r5ofsBQGv`Dn9k*v*fZz9HIg+1znYLRq@ zf_;aUD2Xy+q0#B5>JAxH1!ad_zXGw$JivB>V){yMHSR=#A~BWvBC?kKO%R%{ihbAyBnyQ&H9( zas+x02=P1Gg^zqTQ?|U-wo2YWw72P;tbTgdjI#m9|WxbQFWzz-V9jeQIEo z5z!*XM>Si1Q8jOjMln)3iNDq*#lG3e?_&_LSuRh0QX~W&jgf;!WnXP=HuzTnqI~oT z*guwu%l|cI{lA@zqy(Xdjfg`Ch0ug~7_Tr~#bw>X!3RTetq`PCB*SVuNU|hrd%s}4 z3P}fE$bQyA#Fl{x|E6+pzK=L7OSSWvK00xoVbVn|ha8S>pK^^ur9klMo}{HnjG z+TxM96vOJrsM;bsIzm21uG)uWb+f2n(N&=ON?sdH4s4KAq{Yh2e{p?G(;PZxlZ&Jk zQM{wFm)4N;5dqj?$hcqBW;bxnq5p|}q-qbPFs347n9+kyM$-&N1mOfpngN|`w?Kb5 zzGDD;`M5aZ=O3)k&cm~B+(L+fZoL%0|O zSp(X!)WPJB_Q~&e5MuZ+f#)+3KumOpT<0prZ^kzY(>s@SU>Sj_B=9~Wm@JlJltb*( z^c*f@A%Kq(Rf-**(zFxRSOrAdS;2M; z%7iGY#Y7c=T>sED=oZ4wA>8SX{cgp`K+cP@v~ z>u9TV61N8DN`k_U(lq`(JRk-86+xFiD90s;ZuIX1VmelpEkiahF0dO!5StA!qTV~z zF-kk`yeOjPC>U30Tt%^j=6W^~;fkwbJ~MQ`Qr`T~{5HK{b8NKvl7VfkLo$L+AxA>xUl^Y|gw5XC1p*5sSkO*poP(;rx)vw-EJ17pi{i^=}I!f!IEEUCr z<~E{8)|(KgHBP+hplq)n7VQtb1~&Ogwx?t<8D!u@%d*i+-~ZNS&;$RSqyE7N{|Q(y zGuYN;pTYkvDgMvN`X4%Vc)|A@!-1Pnra3@6dVk;?c$ro>jetqoB@ZhJEgTm4Xoq>B zgt4Ob91YcZeQ}z{%fJn3&u2?ONF~r@njZs4<@eYNtsRB?r(;Od{!S*3MgiA)X=q;# zpcXqsXe~i;(Ty}^AgAKwRcd~jfy-$&+*CQaVm?8W%9HlHfksYvRTF!GM4|z*S$0qw z)V7U)y0i#{?Iw;4Q`@(riUOj6JU$hr0=>h@sdN6gmNj;+1q1|n8*?oTkM@5o2$zSg zXC(?1DL%jT0o7$h`LTz6<9BNf!E3^=!VLkhDNfisV<;4HFf+yp{IkevF*yegjaPar z%bT;v=HY5>%!djKCQo)FJ=s)&&iIFg`&Ns9zd*Pe42UW>(8@`&h-aN>v*cV}Edn*0 z)_+L4R~7HC8Yvltjlkpl1=k9Ti8QZR94(5;CcMEDp9Ow}U9HXV+u8F5Q{1fZkS2?~ z+%R8xSk@j^Y(qsuQ+Xl=*NzYWlSL8YhG07Zs@YCNDUxwy+N93_!UYBUX4$TSYvO4p zH5zoOpy{R~lwHM#U5I9Z8`>4XHC+y~hKtsQDh$6cdcd#D|8d##?-8DSb3w{08kH}) z&(Mz>1W&5&srfmeG&qA#LQ$x@{4ZFBFTwKo86+aZ${wJzPQ9mz}SxY zm^DR^NX%-{WoM9#H@O#7OVCZrJrn>wEFqL#&b_23K~%yE+VKc;0RA${MFEz9?TZ(# zPHYF>E)(J|>i@{p64M6(sN@yMKRn(QFH&(D%K&0F7VUmwv`1a(JuOfsxou;3-9=tQ z`9+x!#ew(~U>A?1H*3E5BCW?fkKd?oE&zMW>4#Ud`548BDOaQ!>;d7(`;2dd5MWWN zph$|Vo0W64tPDo-XLPT++9J}a&{FnmjY}5~L})>k$%@KXYT&`qK7zvdYhs`+mFJz% z^i6~(V3ydj04!sUr0J_jt-L_d#RU|-v3jd~(x*ZQwCXh~tDOX##_MS#mr5`UgoMsIxJaB)?R4AMI;G>rotU&@M?yqM1w9JhepIs%iM0bFA80GBrw5fJgDwRB(!grn^U8*RV6kMZvATra_chiii%)d zBN9u5BjF_OqH-n|z>_oxadxGaXe7nr30ZoWX@vv(w%?U+Qi^ z8r?jXuLWn%G06|vVZZug3`PVOs{nCijHaEck?rOz7Za&=>lcSgNPGx5=Fj{qgA&zg zXop_#l&y)}^#-0cr5C%^-1j&g5#jV|6YTb_xj2-OGU<-$(=xLtf?x^!+K(n=h`L{d zK$>-t;ZL@`I&IWeW|nxKaMvXjkveI?g@YV2jPr#X)i-Rgv{tEi?>=4v$xngR^c^`K z42zCTfRWvM&c}Jff2n02?w$y8#(XMaxTB>DZkv!(^e+xVbB8;p?Tzs z&8KMnd_utYR9VElk<9R5*@QrtFtwAf?6vdh9q9}0-4O^%^E_HN9GS0d31^@nTR33TMmTsBEpn2Ws6ITUY&j+}=yjA6q zy)SCXvJ(N+uU~{dXK2e^J~>`c%Mo=!6!?eyQD(n779VdO@H&+pVgAZAMUL>fN}eHfCTTOiB~;Dt7Bc(u z!^8Vtve+NWX(}KkWn^zlac~7|SO&WgB{mi6POsnpx#vUn`9|vG$8oWQHHv$@;@RGc z+6b2xbz7XadKv6WM(iKODsK^oks}+g(=0S#&(A?VohxuYzThY4@1NRrtD*iwqcj50Kd_shSSwD`G-~>1X zxs%WyQ!Jugp~(Xlcx$kvn|UkfWbaY)hkoP)3EeWMJht?F=ZOYil3aA)(*Yk)zw&3< zbXez2w>6eu#79YNyod{La{X)C4;P=;O=W`c{@?hGj$6C0`cUz>Ar7w(xmzhbHe)B=c=EV4p|6h`FOw=Nrel>-wkNC4{h7EuYu zFT`kbsIq&2yDzS`R3D~!A@`I6`4zm-tgjqWq+b7I=`raePA;>c<|L)&S_ox|rHF z@+y{0n{{(5>$rBJ|8b1gGbg<+&=;TPfScYC4~J*H@PnWApIcqQX6EPi0b(>HPYlah zTe=P2UF})MNdMnH`HjeLnL;361+|IHg^Ps|6M9shMDgZ)*O((oR_$|FUDYp)!=;+p z^DHLq+G`j7c)W&jdV-feS87glL1lg@bzN1C_j;P+@$y{|-m>8ylq=4x&xcw2ds%(3 zNPFrRFqXBQ-WN~SrY^WH`D)K}KfLGHY4Wew+&vIsFCopaFPQn98>kd#rx0j*Ou z!j+~Y{(zjeE?mYpxh#CF*lf!?YV-&X~0RI@!d zs@@BPJABV`xivb!@wZ4V`Bv@tz>lc9-}N_9T6H3@6IMlip+yy?068hug{X*AAvz1O zMbgz}9^AZOt=-*)zaGW!%1zNmcy2d1syqcao&kUb{#yCFh(*rc%3$*^vP``-g{CHL zA==KLI`DCq=QJky?X5;r#U$4xgVy@bqy!!5@K}AN^9_MZ^0qyO?=UI13mhV>PXhBT z|HxRBio_X&eeA(s?YVbSCY6DaBK9?u??)8vWhn&{x%0?oK;%J6?xUKpTO#)D{a@ z{gV=yPg%;oTkEfdNz!m<*zI>LVOFGt=VF89zdfnJ6P_{Wb2h~r|M?-Cl@J}T-u3FHk&9ygOL9|( zREiLdW}`RFJ>|fw7(h_56J{$t_-U83H}XK zQFJdigLDzXd`{OSc;cc7A3jZf-`;PRHUSXawvVoplU7jz``wo}1|FsU+Uf$f4yzUq zYpe2_fUd|tQA#=!r-LU=8?4^J|C$1_l9uoEZrdn3uiqn`(vGD?id5u2r8nFtlKB2- z;ltGhe_VjLL&Ad89@eDv&#Ws4dnoZ6?%A`=E=U|!@GONhtem)fjaS4G!3wz!vCwVK z8=$5or^fy5E@M#_UjNK(%9Ca*{wC@DuPB7Y?}G7yH*Le3S(*ayOBnB9K-_7%w6RG{ z9$TMo2%Gl}>9ikaNFf|792%VG8QPoNUn|2D#RG&YSfdq1wo7#Y3Sem8L_40Uc1 zAnSz+GcV=5R-_+wwS954<;#{o{iC<#O%%vn zmfNVmC~by1&0>M#`S~fQS{aXRQs|dcfNoJvX1i&}r_QIa=XRt9aN$^UUYKYZK1k&* z@PqTjXrS^a9?`QOV?!CO2U(PkAWEg>{68WVc(I~ z$sXaI)FHmb(c)l+a%%~-N)DO9Xg@0O>|0vf+E=ML+K~8@zq{oNu?VAm{|nejezdW) z-;uiNiaCUzX3##&RE$VIZAkgpNA6l)>n0cev**Bcl1ug!CX+->USYd`W*pXJL-eiH z6~=cvSFIll*QWFqr+Yr#=OjFwKd`=EsZ%jHwC@6?J!lU;d@?2|^-i#?3)un)^+ z50(A()OwPAz@Ok#?HsVzH5#Vf-bRa>FfH`7nW!k??%ykcP)-^`K0BMUTDfirJ=4`^|`0VX8OHRA5}xpnariu35c^u zw!JL&yE-;=@`M*_@-&x9H)G4@g^{8LOd$orArFspfA?N!{$uocHfym^)}|Z$fW%&# z&zW9yW9K@Sd(Y~{)pwjK055^p{&<0Zj55@?2lsmlU!^Y0iOPP499cnI>>$GKMpy3u zy@KTo@?;0K?5JON^1JMpO?!engl=4LPp=zj%G)mtmsnSOc!edGjX;V$@>N^0_D!XS z7Y*jU)Yz+itpX1mA?O8H2bxE_aiP7iDj8Q>Os0C4F2$&RdUK#dhOt+4My9)N2Hg+# z7dnZrxlOdFhaT{CI#&tIwfpN8T90Howf}Z18-}Ie-=1LUyeSrYwW$Z7IHS6u*iali zps|MQI#1{0t0w3xg0z3I|c=7 zb0xtFYE!f(VX+{qD267bC+i3Wzp5{gnT~ zA)kJ)GkOIO=FP4^K=Uv8Vmn*0`P3nV+hB z!7>kaE{!KkI=n3QMK4O#?N#gjL53mjS~qO(XrwJ$Jrhn2E$#xzbG*B_V4Gk9vMC#@ z$$WctW2+l=snO=iFGoboRb1k1h>BNplE`}6+ z+)--IcysX@n0-@%bveF@{!CgE2ONRE9})YEqxjXx)($4x5oeBawP)W zs%~_=$xOmZ$L?N%8JP6c_$Jy1hUH2Uw;ZUyaXxBdk3L!=2$MI&qL_u<46W*hV5sm} zUr)}s^n;&zPw6UgZd$E)mD%-O#>kF`U5JQ!+-2m*GQ*VmgJ$@8$zZa-RzIZV@|(Om z(CYhJ?j1ZBOE3uukzHLqDnX^ecme*A`L6%R-`l913BSRgApuSOP8AqvZKP` z@Sye_v}p=m5Q`gNThT_NLDXFV81PPttC0#x<);xF3u7WB?&X?*ta=-KBRkp=!d9zF-D&$V} z2fYLT5rVmx$JB>(!`g=%u_!_<+(hE^U2SU0j;BY4@{gqxKrg88{Uh{UC4 zxo)LL@n#9&(B=B9Spg|RxaF?%Pq6pk<-x-k+ge;`zgm94SHmGG#@Em3ELqBc(>C#a zb#s`0weS>CgGK~&?DkW3V-uTe0e=8Lbacq~s8VK5wmX-9oyW(!793vl`;%F?(_&_5 zx(Nx}L-Ia7sGf7ww=LR&>3{m`+afbdnd!47sXsmV8x^m~MFD$N>3~cv-$cxW!qTr@ zwLFO}a`+lLY}j%;$~>=bBXTG-rFldIrf1tJ447Lo(^=U}{Gt@v*_)!Id+v4iX9E@| zB_X#Xot2PXvrAf0^v0EIO*Zh;H$L&=cmUOvV&0Us%6g+qRD_23f-`WQn3ep%OI4i_8&iB>F-L^_;cK5>K_1ce9ht6es`B>rXx{tC}p+*odj@YB;9_v@%O3&N z4_iM)=j_Un;_?dDFJ}MK3<2f?C(nAJqA%7qM)dDlFH56NSw^z{b7} zD&rDd8S1b(5(&KE;~0}R%0<;_6E?xQ%~^SnoiV&yeRq*1UHU^@ChzRS7gem7Yhc%i3jX$<7G2D# z2XV5HYU;E&EFjq$9}?7@b*SaHXBG$&n5}ELz!LG^|EE_m8?E>-fS`#baSvDTq`eOr zW_Cq%uWA?J!U4-6mHnVC5zFUaB}MF|avI79U79bb`?Cks6r~jAM7<}>2hYT9D4xRX z4xD(u_o_lKD#1u-3wl7lB3>mQa>{p;KM$Drr4BBMQjoq=eXb4wi`MCUR)|U?dFWjz z6g{dYzXBs-U32!np9}LrO0CqXWj&HZk&WltlUe=GlPtAgvC$-0hl|8hmZJF`$q#*;qT1HrCq#~?D&d!gm+zIu-?a(* z#T*}}HkL$Th<5iLvcAy%t3 zHeDi_kQ*5g@jQoUoVuObK2e<4qOFQ8crHykn7aE+lQ&batU5xZ%l?gCrp7MsAXJYz za1sUJ#T^WkDzs$aD$b}tV zSk>tLs%;pb!HIU}39Hpj3UQWPV`)BEPJF-nS>E0gj(jG&cx3D@DuG7-)G+#UxEP$; zdFghG8_$T^;xC0<$)Si$Zol)*0m8{)A6lkkDe($k&jyXo3DuXyGWvaa;c%n?u8tAd z>iC7QpQ7%!AHX+JZr7w7NgjUnnS*m;ZNl?7xJ#dJw3cMdS$o)mqLl?acl1%`QReyE z3F$NgEzth7ZmzDM;I?D9C!m31qhDQkjKBHlgr=j@AWvVmn2-iS0jU-El4fN$H^M&LwudU#g)gs857JIXpX9 z{0xyREcRG_>1l2A$H}@N9p%0kZM|D{pxpXy1!o^=_gSIeh!o4si^084%end(7wz%M z>|D(zrbSs2$<~ZJ#b))Vzvey|g;?%nz)!2teCs{x6xTxfo^Y8Zdyj40K(|#sOx4(p z|CN%+q%8W-2HBO#(iC?#9}%D9D(97xh>V{8f$qc7P21W_G>ArXFMe6D+|vXJ{+ApHL{3f4gc5YWaybc+BoTbUz{$z zo)cDI5$^K}GOG4BNL^3b1+L00&6s8qmCQV~ z%a4{Rjo*vnQ~y9)hSi`+$n}6~3vPFH%aBns!g?amS_63tj?xm3A5=Yu)8 zyqZ5a5L&qKTq65xemonr6YtEfJz}P>y6l6?XJ7u~RgbFw8+UkWd+5?#_sR>C$rzcGEpj(z}ejx7B+JE7m)WRcuw3Viu43jyl29vgr9P^D%=j10YZ(|uf`?@A^ zPS)Q;*87(7SMKCy4R%+$)LL!y_|fI^Rx?Wuks;+xk;CrW!mIT-tKd=NA6U`?4Tf!o$K{h^v?_9rguzWmURZ8e*vvr+IHgXr2 z&A5_tt;58{nUGa)xK#~~Hd_KNa25FsQs#hPpFhshPj`J3awf3($wbA#jyUR;n1=FK z#B7@pL@+?gz1HH-bWLY(XW7^*ikG8^9* z2Vu6iYV{W-MX-D_Xs0~qk(S>LICbO~BzyJyy#^}sdbWU6hsgu|1OL2uQlP3Y5Bq@b zSA~O-v%AebVpzo7u!*&QUvAl0B~5#2(w!=lvfpx2VdMqAWkO;t|MQs;YUw~E3ekJB zJs6ql*H`*Tp{kvvqPqW$ZH&6=FVdxeR^|%Gud$uYEt{yMu|8V;)%SA3%>NYp?s=5k zEHs}Gm+!+$?ZGZ)=kXNfvr!}b+PT?g03#4Vi{}Ed+5f$Fcd-9>_k1wJeL(*}0s7qK zm!)$gt}GLv+P>-OeN0NbW6$aVo09yhI#MMok;#5-JM>09s5uUclMat}LcDS5e z*+R&1{5y;vAPJS>C%28JryI5(;RkO-I;8id*4AMrDJ)U!jTDQ3R&H1S&YKdMo0Toi z_;rrbnD^BQ=lw*WFQ8CaLkp8opm(MIBJ7Jr?&!^Ib?e3k>@QPcWRVSOZZ-y>lHMgMp;e?*ELFwNk^Q^*Bwv7BQ29)Yx-JGdwlR> z62#JbFHT|7yIua#7qxk{K(!?U&T*TN?SUveOIx9grPh^-U_wBLF4-d3zA!9hz5C?g z^H0yb)?RW@wiHT&@6;Tc+(9j|)zT@3Nh_nPCN8K0F}35BVDR#ZgXJH<8(0nUDDby@ zkU>?04fH@kEJi?8)w>Wqdn0+PSQ8t{=HF=yNjE^HKy;J;TJ;A=?Vc>McdQ9q>yjgeMtZe7o1B2@!9rh0uZ5=MufvN)k zs-~`|=am)+g&~qJbyLn&wLsnoC*&h7PuKQ+PMZ2;J}T2q;paPf{0&9?9UsK%Jf|T<-J`u_&xoo}2SyD*m~L+0p0C zI|l=BN#i`zUkOPO_5KK2Jos1>v}aVdoYIOGiJcg-RshMj*Gm*n02OF7JCm!jQLr? zuyBQummxv^lklZmUp~|_n;)JAQuHL_v74WB@}C#F3`qR^yiK+a&Ri*-fYB|%A8Z0% zOb!0NsjhU~oa9lvJ@|ak(!bU3xjXW>X00wR{+v{|VlLTe>U;FnGw|7L6TrL|dzkT0 zb6u9}x+Fcx=b3hnXD;tWDD5S00)1L+O)UMv`eqp~&i?jLwn+hdwgoI){}o@>-SKf+ z@q9+7+=}&*W!Z<0vwc+Zia@O}mel*SHDAih*8zKS8Q@$6d);Y)>mT4PZAge=v47}v zVzcMjw-n~<46D|&?@9L=e-$tOnk&sqJ$LzO148U<-qk?-w2+K>SZZqhml3s^qq&U6 zimC9f@KL!}b4$2k>S}et;!-sWiS9e)$7&5s7QbUphs#5}_>6NTWPIn{&W5xxYO#nJ zDBhXNkM7oTHSF>5QSrmfa;xy%KRO3cAgvw$JMoXQkl}>m^6^-dPL@zBzpEFjER~mY z?JUHe$ABjWnDf;UW5#-pXb?6*JgAEGzY%zDNWH!dpjAB&c+`c zZoT=y=(IL|iyZ5Z++a9UZ8^X8#HhJ0e}eR^I;nP@KNQ){zHI9ysQ>xinB#rvlD-J_bh$|VA-L-(BR_p0Rr1}bneM1Mn_ zX5Y_Z*FmJ#<&OCl1;%LR%R3=DcB}M)rBSr2d`CnO`-y{V!_m;UlP=TcpE! zHnOFEjp=Elaqsr*12AQl&(cq;&Tl9(<>&)U?Y^5=Fk|{xJDpLa;qkCcUwTMQZMW=B zKr>!A?DJnK#44C};q9T*YJNW>I|dQZ-WR){q~2dpH4Nblx0d`9Ev%MImpOGZEVMgh zyzwi~J=Tv2e+@R_!)!jj19&~tPASRy$o_k(2h_E_%~~WK4~gs6OXv7J9H+?|QkEf| z9eQ#)Mx92j`CD{7BGu6$x^$@AY0K!u0r%c47ql~c^Fg}uYr~Peul!j?S|IzM-!o1W zBAD+-0gPh$><7BuFy#7uWTs4~X20Bb8vIVOkIi-&xqQLCT8!WQK#7dJAtuMZ;cAtY z{rg4iQR0R^G_NP1`U-l>Eb?4ycP9epzYE#tL_x8?0}TI`P` zPp)#@AAGUg7l98zj_CC>A}6PmXs7S}a176t{%}uNYuRS^8}}QzSN*~|zqg3Jg zyg~mNjI<03j@z0g)|_=nUx}JI9h*v44{(_B?vVVYdRJd}}{C23u<>T&*g%tCtLs8`!4auFZX}>{yew$SN~n?gg8$EC*94 zdf~49V8eKA)WfdJ)BNN)oid>{d5{S1KI62{n+2CvMf&f##b?j7In*_hJ}%o5jo0Gz zOxE&>BzVM%y!DmLmogPgF14Be0+LlNjCKaM$|Z){bO@5M%;{AP;snnxSI)|dA47>m z;L=Qdf7!eLn$sQkffYxZH;M!+5R}R{4_DVFT|LXsIy(E=@G-iy=r|$h6@ua4?=ZbL zF>~mm{EL8lya{=ufEnPBsrnW3ohWnPalIeK)1c-Q`XjElhHDWnZ>&r2%JL2bEnb<- zTq%SGG-}K@#-M~li$s?R8nv#*yWb4@G+q{#9xq8w;tAaan-@ksDrsRb>z8LIGS@FW z_tbstpzmhuS#MZiE9`|9Kqz0XNwXeN9&%GxC9B;+%+LoOJf`lAPZE|(o!Cy-<3|P_ z^ZSoF61Qw^JHJsdN-N2MlQs0Pd^KeMucC{RnDVZQscGg-=Q5f*GMlP-&l`nNk~__d z0s9RDaCyh2@~kFWTxdgD!{9l$mD9`C15HsD(LV(=LKv}0_%Xe6CpcLY#}_V)+9+V* zM@@*}z?B4Ar*?C0SSz+1>ORpUfu-*9%Q}`fAOG}>~MI#5!-l8o!@C}n~jvu5u5ggl*7DePs zD~Z5snT&QdUrC9zs$OB0QTi^BmgmBHu)ezeXsMALoGJDmj;m(f(An#V`w>7a$RD_b z?Gjqs`7N9@^YOImrYPa6xRufz*chIZzmipXP zC>G|0+JF8^_K60sfnuDS@I%r=F_`U*IFn4NTe6h{ubmS8B??EuMKAapqxxrO`&ds# zWflDVzLy-#K$PH~6?dvu0ljs=XX%VjVaFB|Y-=>yFxPG@l%)nCIY3 zX8*|Is9KZItiCcCV^Y~)9Z-8TeMe|#+`f#v>o6i75QsYb0jrtO#cUjv<>Z(}H_4PH zN1~PYqKN9t_s>aX><;yH4PNAETn+B3s`DT}mTfmzZ+%Tw*xFx$^ZcSXkER^4_qKHDsyYNOXm&D6HII-E+)_ekQd*k8Se!Kbey|zU+U^z}Jb2SeuUALvPVe zCeI1{U6)1UFWrV7ln2ZV{>Og-l&wJXS1 z;W5=My-pqbwA#jz?`?lQPRhAo1*|0asmKIt_go+isE)C&aRyZkn-x7bg!^4St)rqCWWzMpnN(^DP?2*{dL%? zIz7L^p*?Fs_vrN6r1jn4)RXSWpfQ)t&ITTG5Z#R?}jwh-n$cdw7Z?0Yb+Kp0^Hm;3n5R2tW|M(z-)dc?^&#>VEcH2{i)tzQ^e2 z5LawgNpNwsbMC0~EtC2gA+_)m(5)_j(Yx!Qmjgb!?wH}H^uyB!-OfoM&cY9G9?E<} z=r63lI90wP?d+GiT-XEH{~IVE$8nlH^Og)}x?xGrGEO}N7JU2H-yel|HR>j(Yf|1O z#v~3VD^D~B-jKbdKIT4f-gu};on258i*=gNX;-l@u#wsqY;F&@YD3C;n~CMyE&`zX zm|S`r4|R*#8K|2zv5qF7-m62M-nv!Sv0~KLkEGcXSeAZP?S6+IeSSBuJ`yz?VVF=x zJZkbk4>>-lBJB+~eX)f9vz$~q9{+r`v~OoAzpT#m_#mi7pwH))OZ@g@To;P|9$fFF ztGdm_=`mDQQ~lc!&@gQK{kC!V()%C-L<861r$?j3VfD0WDfX1UKea!YiwP_Ayaepg z)Nix@i>kAZYw`>KJ|Q8Ul7f_kG}18@DG?9>5s^lbknY$(x{(m2rnI1hl;r3XMmLNe zJvMqQo_&9>*YEW_&)<99+u6CV>pu6nuFvOvT%?xwox;oU>O1%2=3|eWE?e<8as0;{ zHr8H*`a0q-wlSKwY-CK?qi@&@byGq#R%FO?rS$SsswGg8##-plejf$wvY;@XZxr=c zMnOfBH;?K<{NNj(gzUg2+Bs+iP?PJg*DC5mqjqDK*;78}HAWArtmvw5EUL@TnVnC8 zMsCafztgE!=1wdYe@baggKW8N)-HvE(Q zBO~9bx2R(vzr6q1=zciWc+20FU$+DAEWbl`wIbiopIAkir7Fh=w7=tGHh z`ayZye&ku5@$gJ#dAwBGMp8_`FOzv+r5!D$2mJI7AvB{ZcK-GRWpZ+FdHFL;MQw~- z_B5z7LChrr6+pG^l9WPG_Eq*Dk8l-zqEmYZ^4SW7vd{1s__YE#T~6!lSXhtq;^W5E zVGSpq<(Z4uN2-)uhwyq(18^KHtg+E`6-&M!(b$Ds(1Oh0`~=;OEOTaJlUnsY9o0bh zYi>DGR@3K+1_mGSL2apq=3d%FM61KoBgyX@LuZYS1g2aHbHUf2W?;} z0NiO(Z?o%im6`X|#=*C~XY`9At#Rijh_5${m2^e&SBk68+l==86sw3xH?CSpLlO-X zFIFA{oeURFJM=9GPhpcI8b#<$W+-s+F= z6T$%=d0ii?7}%YxJ(1N_E$^2kvMXLnH1mRi_rD}Z+VR%IG>#8v%~m19)SgXarw>R7 z!*mV7_j4vPg#svR zo(YR=9{^;Dj5ptxfyo@l1ylyrJ2UwJsUPAvTyv1U4+PtHFb+SINl{+rjU4`{F7jQ z|LYM6@}K;XaKEJ|A*Bv)S{1DeC^M3bqd#uF@KF-nE+DNxX{EUM)tpc>VdC{CWAEX+ zXFpe}y#9USt79iWnECKS2gvbnerI%5xoH01pC8I^p&NQOhNI^vEewM!T4@eSrXo@& z4O5&T1Gw%NLDAxPl@J$)r&C~dvn_)%nb|*yk1GbpQNJcs#2>zH9Aml9QAEWd=dJw3 z`<_R~yP1+~tDqHw2ZWl#-O!9k$Sd2cWX0b~8zBIdO$@b1i*w(9%jMb)?36g1YR_qNc{w5TB@}TD_-|XItSb{;hMz zt;fM-chBDOzYA`+2)ul*PFR-dbJ(I*?*}jPY6zCNyrN5`a`iXz8AX~~&?-u)RJvQ7 zCpI=6xHp;2S$^l-9)T($oY<@`Dh(74Gw0raoEd$9)b9$+HF}p8Zutt;F?tW>GX}u@^Qr!e+8?QIp|cruuk$4|9p1Fy37jRrY}LeXm+X4%hNJaWbV8R8%R8xFnao7rLo8*!<7>YHuUhN_6d zQ>hl;$n!65?c0-tg5^6%yFbi!ci_A#qtQH;W=Z^GtqN2z;Sd;cIpsuy_9^okpj6eX2FkaJ~0qCh3JU zO!{qz2d<}TAJExhHxqiwr! zM#Qo8ceZ=SUMcs6d-oFkdj0(QqQPsGc?10~zp9@`#b?%m%W}IvXWf+3f6+#@bm0~y|0uAcZQcdE%P~I;mtG* z=)SlD`{l#=c!xJj+C`$zWwluwYq7HVFNRlczqwi= z4VitbHNX2)X<^7WM^?W2-TOHuZkwJoi=xPLm-=x52+e!9)AxuxW4HGUFx#ceRMaHp9kdbr?j2$WGrGoSru=<56K;t^vBN#>~ zZ&lyN^ymP5rXi5cf{;UU`W+;i%LCILkkSv=DDXrZxz{3UWrp7#E^X2;;v1l#;gzEm zb>##Bm7u-qDz)9{>eemP@#Y)FHy~_k$NF|Ea1gk*?zW-0&wFxcZtXw?!~Ai=&4AVK zz~&+NDVysxCx}1c%0k_{iZI8`9&2awn%gI*B+fL-uzza*A9^y#j8l=Ry1O2K^ z8M%}}>g2qw2Yw&Q8;fZFfguH7i)pym-Lq*K>i2NTOpNMZx4BtOgrOt4e@-Fpg7z=y zLF7h1woaPbc8utsfh>UBKS#?5Ec*xz>GW24HUkP&EKfOlh# z-|Gn}u90M9ZU4YMErVqIAL#m@c`eQObK2T^35AtgqoKzRJj8&ywb!Nt3~f%mA@ukS zrw+d}p=RZ?45zwe#ZYt+4|>iBBn~vv5vAIMLCz%3E(%Ac1}lws>SI$2LCK$~aErJSo0*2@Tw&H-!-Y&unv}5@yN(T_>x0<^NKX#P`=| z>1pSzis?O?*}k{GnoJJyRs1P^YdA@!!$4v6|+%A;>frgk}G*0_XbwztpT2EA)P~F5d!HxUpct9jNx9 zPld{RBMp;=r8N1Zr8R+KT)4$F=~MZatUX|A%A>r*OeLB3rq+~o_!{%Yp+R_y4klkX5mIQ%%0S}t7I$fvTU*WODNqY z=5#K{BsVNIznVadQcXRZo~C;7looxLw6znnF?^QGn4)~(#K!Bjm0ECC(X_sF+v;zS zJOH0Q5{Efc?10QM+n}=%S0}Y)1y;)=o?`Plq4?LKZ0I=zCJw_CzAuT{-o8>Wvk31> zejUc+Z15p3uH}z<4`RsGA;r}!gN2goa*ePMi+vO1kBGAg|5BxRkM*Wmk80;sx^CU6W|M2&0 z>hvF#vpX}&kuN*T3C+sgj*tlkWy5UnO&wQv87J5a2dy4fXLJO^3>;C7{8y&)W)V${THT0;m7RDdpY6_ywHzNJa_ihX&} z*_e>KQQ--0HBx`CUVa|bGxDP9VzWVyz`4<@8`k8VaLge;w~l|t%fXYCVud>#WTnK! z56r7vT74Il{|XuRr26g^+E`C?3|POnY2M#40=9?v@LKc8P;LSs==>xXu#xi3rwk}g zR42Mu6lm3lc*hX7R~@Vv_9g+%Oqp-e`B+$cJNpL%dmqUZ1Ezt`ZpT%rY-y~hSiXUT z#W<4CF)tu1Vu+x~qExUCe=HcaXPSqaCKAAGYRZIsw*NAPWI5&ho%J@YH-?30AaN?c zZk{SrhCO=1tJ8M+Z(YBIz957KUM*jyaKoc&v!uM@&$IXQCu?WX9eZ;Rr?nP9;NAdr zJWom&wt|HIb(bN1+EpxH-bT^61txtaKWB|2>P$|@jiT!x^N*fgeJ(mC{u)^AUb8gz{NpbcpjxI?qtk@Q zlMT(i?-9Xqe2`67b-T>Vh`c1fCG`oYF-qFpAo9SGg(}vBX5Z;z6#Kv9Qh6R-Fu_6e zCFYR#>L_?9M+^VQ2Z-(lU9qhcr!l3CHi)_$4a7dWx zp@LdBON=fn8$z1084iilJ+seNR?t6_<-m`ufvF5izU} zlzJP^dYH4-*DR_ri^jsz4D8)hPYQs?sygcz%pF|f+In$Cn2|AcgqDZlL9wl?^a>VR zWT>(o4sSz)&Knzwsv-E(m7+_~Mdep34)vdZdS2tEsClllV>bw!A5G3T;$I?iZnNhM zQC!jiU*PNKI*wYnMlRl)UM95y@#yAifwwf$K+j202Fy8c{?okTXQ^sLH7(Rq*b?>) zT**LPS`e%iMhg+-f1`ptZWabFV1pkfa+wjSIF$0mu zO^UYN;`kvlijFmT1a>Q1uW~)$RzNY*o>_n83BO5aB#AOVhtU{C9%(6li&( zCNxl2$Lcm&$2Y=XE>#%+OkFHV-_p5f{0p#)oar|9)?QLi@iP1^!fxxo5FVtRO7h&5 zC?eWWXV&di*F;wWmAdH@H93_?R|YN)Pfp%Vv;aUiVN9t^_Ro+?ify?;-hQ20Rhrt# znsJ@XEspQ+{MkckW(x}ftsF%PGi6RooRp{m=c@OEm7e`iu=5-g9a9iuJ>`63SJ$bO z=~a#|{rR|q>%yM7%7^bf=plsKKQ8qolt$Xrb)txnAK>Hv273-s89^y|n0vFJ?99fx z&I0sXYK`Dap>xrdiwXgaM*5p>0{PAKZ%Iwd!B=v~?t9`^lX-tldotG#Zr2hyu+XTd z^AoYfwGnp@zD?mcI}_myh06I(?)}`YHk+;tiwp9%Is8fh`h+Z3m5`Py)RW^r8I;Km zF6@I{vAT~KCGlSwck~S^wMM^@5!o%LM1Atp%sPLy$zi+ucgIc{R85*(fE4GsxoH6f zMf5M(-2oKOXoXH0Ou~A=z*Xq_Hys$)uInmZ216TEeU!|rr&l6we^GEF(h=FdQE^<& z6_r|{fy+YN`$;5E3TVf7e{Nsmc;w!Q)CL81hOrcG^F?Y-BTy_%v?379aP#@SnzGRI zeFKGsK^u*k`mZDf28xTX`H_l`p<$ahn;xJ3+BPw!z_Np;bTlO18xLt>22~b{ErdJe zLNKoPjJ1jekBR2u={|6brxr!77>>5>ELNqF$Ct!;sL}_`L0AE#=C*=uzwk~5o9)T# zwmel;%{URn_gcrHbc)rwFnVfPA9ZdYmjG2#)^(W7$e0Mb!m2G+WM}u?-q91R7v^UP z1*qN++zQiTZv%`>;VzDx@?b#tbI2F|mn;^$k3;Bh2A#%K5ZzUH`$MiyFsG!(r<0AM zDaOrF-fKbJ^8JLM4NcywBmlulYpW=lhZZ$6R7_AL7X5HnkIO1}-!;U_o)VtOE^n}Y zIJp3~nJ0Nw-GNZQm=7#M<;zO8947ck>0)(Q_u+&cxWkK?fK))V?z;#iNLVtpk0%t; zPHjkU%pXt{ag-Axb!!yO3*D%NR98cmA+@#bi<>8*YVD1R5vX4AW3SdP3||+=Ij^V{ z@`vv^Rl*v30;~8#xtCLR=BUq~X-kBTi4#$7`U|rB;;Lw~o(?2uSE&kmJqA1IM0DZu zb-2jCA6yBsW_zo}iAkcB{0P9@I4&iWEz8>$Cjo%eCm=kgc5LsTJE_Hq$Q_8z*RP-J zZ|`D~$p_a+gO;e`2f$Ce2NmVz#MJeT5}^&>S?vXQ%C6w7e2wE7Y8JO|A{i9E)pjQ) z2iki&N}^uaR45)~Gx6DlSA;xzbgFBNjg|e>26$Nzl)U}g#L$ue1uS5rxiSw)v%zvW zI*7**QdyXQS|%tCFX&8m3_wSF$O?D{EV@6iReMFS5>Mqf!l?2iiG>h1TiFRA z9OIFXp9xkqbc;EFv9ib}zM*c^XOVzJKGr?p7R6KI>$_m4MHyffJS*;fmj9 zZfzsKq?f6YL=OCJI{z?ZSrXZ`PZ!o@VngVo9yiUcJiNuTuj5yiA3{7o&aW$W;ek4v zOEp#B0`3_wi_n=m;wDhO^q*Glqvml-K_nOZPU?>dk zktFRFPumi8NB*5inrRX)#3MbUebdc}&@QF3Nc`>e&lFejQquODeX&`?7U7>aat9@-e6<2+kv!>$T z>J*FSZ;&oF~mQ=rkx!)69t5fv&ys9;-s{5qpFMpNZ{RA+@}K-%Glnv5d- z&UU#^ive(>gvxOKkd#JaD*|xS;^a(}l*fHa&?*tcvyVUQGd&vZITceak{&ZgHpz^d zlnGdduiNZDvi1ln4!Iu*uA?ug%3LWFzTuYTkj;EUHz!q|wSPh^P)wu6eH`8Le=~!s)7y zV%7SIZRG@0h6I|{{(fVL*FUM*4&0zesCH`J|i5y#^KHcKM8clg^0O(f+bvtp%+8s4IPic5r*+yr?4w zWn}tk{*iTd-uhf&Meqx!2-h9xo$@qO9bUd)!Sy&p9j@l<5jBeiJNpwlB{kwzZ2Uj0Tyt%dgpK3r! zM=E5v$e2Z=;`GasdyiCY;upTw73Z6QyY-;Wx9!LnE)BR5?-#eWseM-Sj>{<yzV+!0DJ5)lJ20|8yd$J<~%`>5p8$>U_{)I%&^&tRD8v0^H zF;#i5xb&MAWp;!r#BX`%>NUq@XH1|-Xis862dp4q$+w5r5VDqzzZz~j3sA{*tB*L* z@}ul!um@bvngeXBTZP#h4-2(C@K;U~mIlPjj|y9)+TiC6^53LqmwKmbctgQEI=g$% z#*Yoy0q(0w%*{kSq~NKf03xLC)onaA4?fldcL`nVr?nHg1tk=UJ_YzU~)rhcNxKy4^OnmX6ccmg^$%U~CZ=)jz8jqM4~Dw6Y$ zmB?5*GM?5wB!!Qn#G+$S^5TkVc#NTpOd7fXukj>O?3Pt9^np#_T1N=0RXUJ!chY#| z3*gFn)GBGF62BX9xozX@#iGN2gCWb>wr+!du}A~QLDk$s?dOe4DR2B|1?cC^*G?td zZb5i^O*%iQVQ`n$e^SC8r<{ME3)8wtuD_Lg;x55SWSH0BI*sYMABYMjw*}Tlc)dt{ zp;xz^4jOCwiQw~$N*-kd8D%-|UPtd-(C@jm$uEi1${NedG1;MNqL4w~GM>7X*nMJ- z&nMwfr>XY)E(nHS=a@0xkn*Uwx|9I=OX)uFj;iVC7F#!%ovAMPb(bz_U7-AT3$h#) z*C!D@qzzY|c9-yAU@e&$u!Q<}_Q%-M&Oqy1OluJWZgjbQ1)CWLc~7#fl&_VC)cQoC zP}#f}vZKL#@02lRfqdsG*A;@s42!UdrJ#<^(eZjiR* zsJCzmk1d;MdpQ&-U+S7{+7O&$E}OnKZ^n4^4*?#|eOXJ7wP&`g%6G|9q?J!CA3iwB zbUgYcq!WONE}$EKUCTvARA&5xv$R8|VrDsWC&dpn9kz|F43W7)L8a*-m!_dz0 zDab%rvrR?R{7coT7wwjH1jXs`%@Z}}a=kS*Q4eeSJ{5Dtm*1TZFC}Eoc)ip;wOa^a z8km}mv*evdkV$C4FlV3|Vng54 zx~h|VTes@DO9l@K=1hj-`0}$NcQ?IM=u92>1Oovf*g&4##7ch(^`#&$BRXRCdY^UR zVz>y;m-z0fRF72NdXJl(#Iv>e2c<~aXxWNyGQ`2L;~8H#|qxK?8? zpPTV>UL|R>FGH?~M&{brpq|05Xw%GUrh6#$fR*cHsi6D$d-H;Qk|9gN8cp`^(9xV` zv1ykb`$B)SCyvm5vq3AT;28Q=b==u1Jsc?jv-D`^0OXaqecsIE;aP{dkCeO>UVH0* ze9a|p@n@h)wF{rhix>Yx8goF8V#0UKwXs~vz7$^g)|Y5~*;WQbL~Rd9c)+C7wboat zzO|1eW@Q9r<`9}ttN0=i-keJTZ1?ls_eY#=?<-Ug)u|XLz9M3)Yfn^ z7rc!pZ1|gD2)dc=ZP<~YrCSF_{=mGAQw`S%{;>~fbhskuY${OSbdlt+=KPgT+l}Al z-x0~PB9GSNwoPloIYxrI6`{?~4$Mxa&O%BnA<+>T3gJvum=WO;Qg^2E!=EO$=|}Bd zt@c1rYddxNH6SllO+l8(O+xXV)`RX3@RZAQF!0*U9DG`rSSy&OD=6E6-+TwU>O7X( zNm1LYH`Ro!>Zo6JtaU{mBs8*5h8hs~D0A!66rB5XNy`_FvBcq`n>eyN$UT+$6H0o} zAGw4&Qk|PY45s4e*L)hzcZ2udP4i?v6cucoSuVb4n+9K_q0`D?Shv#>jWUUWYVTYN z?WU<`8*P_2vj}IE#DfUA$s-e*G02^k@|YZzJHzoYwZ{PGni>y3NyIEp^Op{7ZV5gXowRmL&pu$NAF-vJ@VzzkF?~ zP8Bk{zT}Ji0h*1Q>zUx;l}%k4YFNtH_p*U0MDF5D&wO_zOs>T$%D2pbihL{z!Eae< z_HLBX(f`RkW~DRXG4Czk=Oa@h?Ej5HUyP=~Tzgue%_dQ`OrF?=WZ;2P+LHJRrV=a&J5{b1Z&v54GPDv;jC0zuiIbe1 zvOT9Te%_abh>Hh8TgxHLxtn`imgn2Pt@u zq#7P|wCgzkI<8^ED|ZO%WF{`-3qta*hCnLtIC}i~6tB*cB%BCLDXqAwr4;|MN-J^P za!B`IrFmV8I~IQJ&3;6e@!k)qd0_zZC`_uquKm@^dv{fduu)iKCZ=zOTJva~7>|sx z;A_jiApUyA{%$&fn&W0=M%c-CI7PM^pdsnU@5bW=RvXS{ph5ge{F};b)*Du=*5uzF zk0e5S0+AN=yVk8ui7slGBk)zI*9iH)T=)7eCm3>3b=6MwUA*=WIg&MFmn;B#v}7|l zao-OLLKR%6qz>w|MD>?|67Q@>Mj+WjsR*_&A>qXv2yU%+N;JMX{4?=QD=y;KRd_M4 zlyH1Sr$kch5%`#66ggg;YMrR9yO^cGQni{Ag$G@%-a7tL(Zb}kgZwJ|S?{UwqsO;Q z9}zo^_illEmG0k_(LW%iyjCv!a1a~l8m-ZP1@?0X#57~?o434ey# zdotWXo>_y-%mRG+_YLxcLD_y7q^0%3BIKyyQrYYhczzrp6&wD1tpm^<`SyW@Zt|q( z^MMQsG5?(ah>7Uuq;j1rVgYk&{htYK4S(3%S_^cF%^!9f@rd}DEr~olv#1sLO6P4{ z$f^{~vL17CoM+}RzDHB9bsXb1&?5@SY907&f2V^h|A$kr+;Q%re77y@JYuF`;)vf- z>7S^llGR8!{j2Pi9DzY`Vvdz}^jP9%a|$=YVFTvIwQ`UfA=VhCp|!qQnS;d@CnuI{ z;M=%ngPRc3U`P8kq&-6bFOpP~%{4jw1#m+-fdSchPt&V48YsoY1Isjz|G=U)7S5N< z&ka~!%QP%b1GDqKOVDr+^aa@oaI2ZpMY?r%uJs!)5>8?9C%#x)+`R*M$zCI8@%bh{ zVz?pu$;MAw!|TdgPuaxOI~;wFAOYd|Ua14xp?)muy2g%Pf;vB^uoAiINaOxb6q;vE znIRHmnz_b~p2gL-2y}U54%>?fdU{0$vBpaYxW{%|+$j^Bh1L;pTifAAqTx$p60ltL z(vO{XbkWrs6^gN@dV?B4@G*q*!s5!{Yor(E;yu;v$K(+|c@I&YFYb_)wxGQqQt@gO zlU~Yt3J`@y4J}0kEFt&?mf1?FbSQD&>j{pLi{#eo5GF?6Q$tgPYgrWjDMAs zA81@EvFmn5-I~$Lw!UA6;r}o=JlC z@LSkPYRJeTJplQ24zdxg>IMZ?>S9a8Yf&&kIsxnJ z^!HOypE$UVKdunA_gT?1tfJ$V+ReNdVe|(bg(M2Z3?Mtaq0pyxE*E*4_f9MqW$L@^ z$-~!OxPB^K-wdsyG9K`oE@*IOMn8oH`Zf}bjfYN%eoj?FD!KIF2rFxsc|x#~ z0cL)xc=_gHTF!6@7!qpOw|m&JHDdn5!*KEs2oBOnRqX2|dJ~sq79y?<3mX&0&+%fD zRqZx?p5w0IAQbVXGPmHQg{^|;U}NAtmJN6*!6$e&yxwkm>#$*|$|(5InIuHwkSTUB z9^02~GbG6vw4=8d<86#<)$rAjD=fbRx?Pnr5hj$C1eq_Iz3tH*!)x@W)(~7-RCM0S@V6+{qH~UhRD5WzIdst*3cyi zlD}j49y!5Z#J+n;S0i_foR6zc;pY}j5Kq|>@oRy+s}t53etlvahx_7P-4{fFO4FpA zWv#h5t6HCuh@scl$)EIHbZnpd{NIq4_gX#r(4=;@1m2(UD!bhCAG>0OKHPq;=CSTu z`97ysK}Akq!k3o!=}S9V^wf7w$jo^lsMJI=)%R6{foe%xx>tW2+v3iHpF8kA3lwAOyg^B1ZS%^Wf3-b{>#?gd3RZ&Xym z?shbv42JN8E`RP}Qz2S5NZ5;dKUAu4H({AJiRohPhL>;KS~-(Vp1-Z+ty{sT##e(S z**4Lsiy}lI6cmBm*tUsa=6w@^ZColbC#|V&^Vg5)U59u0_VpOm=a6>K#iN*J265GR zFN=UwQfZg)iD4i*vIVd2{5zEb=JBDlQuPD4Jt*#ri}m>P<!j^-+T(;2YeHQ?;B zQ67@8|Mf|`pWxwxY7UKeuY$)dO`DGUQ=um!#TqxvdsB~Ut-MAX*sRl!A366RPKCMi z-A~qCWlt_SL)p@`p(==s`s!9(Fs3g7^N{Cy?aYwD9Sk}17BvSTA7|w%BGOCm|N4x* zT;Sr3tN`32xwRf@G|WeFH;t~Z`&Q$n7FpA***V5+S@|M{OxXsz;RHbQUHcPBRg`A9 zJ$+*N%(2KmZ9PyqF%7!3c!%6-(Qj&uEJO^n^wA-kA)3Cev$b$uh}qv*-ogsde`9;_L)0A%cqTX!3#%)Kl4%}z-UkNHzS%-S z|M6q`lOEnxPCu1>{O7+&W4hkDpPx~V{+?+rYgiRJ$@)K`_FLaA!c0H^#0i{A5_;ZL&A2pKA2NbSn2V@}#% zMd5tbMKP!}K9kjmeNU0mH%5nzX<8Q{cZ0|IC=M@69!zox4U%s1hpr6H%pEsDt#K=r z-0JJQAs&lCPP%<_j%z&zu$zgA>W-_5m4n9Rnt|h1;0ZI^2T4b@KX`uD;=gN&&I(zK z3XPg{yYG2{qpGoMN`%aQ8m);&rsPJ_KL=g@2;RF?m(&1?sGhrw9^o!KbS8|L$}7`^ zJ~qdFuCfz>RG2CYiqfq+W+zNUA{wh7#PYc11)LP=_uk-6v#nB=IG$?k0xuVwQdbfX zR_K#5sd?`n+K}UxgA#(G;!|QQ^QfL6J?{#tegHtCiKtyLcXldJ3;jc$m!m_>pi4Mb zlD~-}2I+JBqM>MjJ)b%}P!jIwdo~Vm+=GtBC0V?<`(C}jB7!-%x73f62h=hCtmp|$ z=i~VVE+=NU2bqz!*4T--w9=0rw`!gZwK z#d5*Byz{{XJ8hk3sr@elTa(oJTauZYUt>%^SB#6)v4yLK#9iNsK6LyFx>dusPS&Zc zme>EDTAICIPD$qH*Q~rZYMjYDv}Q?W7_B%|BZhe)Oyr{Yo;aOl5cNDcT}kh@ve5|r z^LOPFM#853W8dHt#Ew;IY~L4u6go1LXudZGP=k`a1vHQ$HB>q;w~vT6?`IZ`JyG2x zOVnokF?}*(*smm2qyhp6wkdrclN-KcE4?VH%jHnyWSclEhT|vEb5QOa1bkj8(8*!j z;LoRDz2OzNnsWc7$o=xfIq9A3E|Mn{9rkZwJ46eTflx3A@1{c`ttbf zuk*lBPM-pfA+xCWDm?NUWzKhsOlCee&%Z(pQ4;fGl}eg@9b+$-rJ`-q zMCNf%tcmpEV0Frv?iP+3WrK`^bXK2fLg`d}sT-0So)Bz0>dP7hCS#xxn=y1pYFNdG z%^sd{ES5S=hQT(J@pg>H71Ag^p`S69r##?Kr_Xv{_9P;|Ew@2mPN={jQ2)uQ+E%|M zs;M#xDE3U7ut&j=)nz<*)qe`ymPgV)W4}B7rfHpxtIokl^Qy4@+0)HJq@}}d+Y)g5 zrjLaTm472S6wh+Vl)ga$+CKY=A!jDayc=K(RZ$F+F$i_-e^7u`Hzd3H{LjwBn-iz3ODeB4H zFk7o731F(Dc3_%mkZR}O+uP-J<70tcg~m@sR*W-nD+L4R^qy0LGaZ?b zonOZ_d7OOc-S(;rXov(qhS4+_1D1Gn;!DOHFHdLsF3?tTFO;6MZu>gM@mxEIt8ufE zyZtOwJ+`$*+dXJpGnyLEA}eZ?8Q1A0@5xfVukAHmR|kFz;7)cMj_w?UiZ7bb>rT#I z+MH^U7HtU*JD9-w;h(jg@1=1iIVzs0yPUoLrT8l66{9W*bPfIi2H)a6xa4NhqjwXz zb%mcfRczq*kZ!yEXW_x!ZU!Drjm6RTF@+9_tKK6f)|e4|@Yr>)Dyf^qZM2szX%wpF z6g9}}z$!_TB+wMh{8c_ySh_c<_9(iYu{z)8fqrz`cn?+}kEomgfVbuGT)BfpvG$XD;(iJ}`?E+ed&hK44)6K*;|FiNC_8Bt%N87!Musl#5yjGM@x<^S5cg3*?z*NV)-9jAbsXc+obQC}IeSg1fZ0X@tc5bSIm>}NQf z5F+0GXQMoMw^lN(Oah$Lw2Jm)K*xNxgna3s7m0jn&Wo>M4x(g-vuPqU;A)3R|J)Q6 zfZJkFnN!&1=H$0NnBTD-B;VDg5dj!QT-p(= zso?4@n*xxJQ6jJcDm(vx$KI~E;j`jZv)@5602CipcM9(dkCRlmv2#^tNbTue1{-*4 zr>4o=;Ypz)Rmg`7jpf26uBQH2oMKqHXv|lg^iwnUc@_)uv1fi<3z-jI4YPEV6%^vJ zzO;4PhI@RZc}O*H@A=yqitNl+Jo3Tfvs@3NOSdsv4ilp119PORaj%%=TtsHASE3b%E;U;7os79|m%_BTO< zCpUd=T7r*SGQYf~Osy429qrZar>tkC<5w=>Sp*qI!RK8sSLl{O9{=K0-zBNv?=&V9+-J6z>jig<<&nlbj_R#biK>&t@jOs4HNfv!5w4puyx4Ck$*O$aJb7}T zTK>h>Rc1o9HK=C*aH$n2Ev8qdb`K=$Ucl(MTt8)~2+QgCYrb>?@Yh&{oU9vJ-J?&A zy;L;tq-@bVg=HzpVa%;gXKpuGy0`pX>s@cBZ*Ns?pc0zbu{(V>quf*Ir%rGNFrQ>= zux$l=j-&0D>cB}sj?OYXWB<5%f41EizhfrmL$W|$tB*CljFV*L0kq-zbK9usRzfiO zFH$wGl|u1@ADgxn8_&8B!!{PERV3Sj+8MFVdTs%a*M^J-5E>h78yq^P42SuSj9R*j zJ8k`oozdtJI$QW@8?f>L9B|OkGHSAwuk`!@aF%68>D7#7BFA-0Fgza;S8Qry{9n}Q zd*h5ZaQBE8w_P6^EPquF4j#5hwU_+Bs+N=K8an9{!;Yt`73cubDK}1S-UlWf7Ef^4reeDyu3<@cLKt{(T8&n_Dk-{G<=pwk?CE_hANzFG6s?0;DR zF-(9`QD~-;ozf%}7Q!PNT0=%yC6tYVG5X+Z-#@Iiy-Gg8E%m47L{Jgo`r7&Fan<7& z{@iu~M{2YsMC5R$_HP`z?ycj=lnv`8b*wE;3~d$Uf~nEkSY$1u7@ z5bS34bPd#$Dyy|TfWsEnX`n>K)_L--Hs;$e=B-HlSDmMZ2JT+F-&7dhgA%AWt!7ZYn(FTGS zw=*IwT2HJ%gSnSP8FtzE0iiP>{Q3Z7jaA|6KTCJ);wfmm!6WGWf$zp&p85GfqR85O znSy{_;-o?TX!d!b80sKo_7p|FZ6sjS86y3iV#dn<-0TTcOivFz=gL`^V(MBye@DvP z9GClY=$ZpB9+TZtEy)kS7wj4ftEV-|J?cBM;#ENq&|@Xi_ttxPJhqjvQSM(%{*I26 zLsuz&|9jhqA_$h9^?zGi1K|Fi`8|0Wps5d_i=WA6J)ht7Kk-%{f#vsixF5Bdu+Tb{ z#;R@kN@I{1gK*peZRwahebh={3|tiYgobpXsW(YJ>)HpE(hye|pbYYxnLO2+tHy*Y zw7^1i+0|F!bTEJv(H)sH+f_vR@7xeeJ7#C1`yHL%rJ~FH%6%V-MVw})LBvY4g86T$ znN@tT5ltx13M8ZJ&pE%tK~b>!ME`-+D6}+zKyxaB_g)0G%;l(vyEsX{W1)#j*Y2@< zO0}u!x_9-R2LNeIkBx<4z99m8MX%JJ ztd6`WJQK8P%?nN@8Qi_%f_I?AsFUoEsc2L;vC5ZI;O)6bBBC^H13&`vMXWrN47blf zQhWQpCgkzZlq~$n?u9tVLq$BHFrI%BR62-R%nByjy8>|$ysbVOW-1s{;B};EgQvf z5^<`5r2ygm=hK9vxtTN>C$G8Y>{3e%-XPK`7lgLWZcF1f^|McDm5}VC8 zv)ub&N;2T|H0$YD1$*+VWEaXj*XzEwPa6WGs@Yv6?0&; zoHooiYoJPs87w>hRZ_Rc;ZP8K68!lTgcWa6d`{SrW5uGcVQ$#)uTUL z=k)?q38MjmHf32qAV_>7MOAg$+TB8V(fFsXVyQwXyFs8&9~|hvsMv&4N)67(n^AxE zS$(@rb< zEbM65*Cc2@1s;h0El5TY?s)JWU4GePetC#jh*6T(&#$ zMDQxypBPKR(~MZU7EUetr+E*vprzN$3uF#f)k!E@2mFofDVLu)iOiaT6piPx$|oarKopZuI8wqQJuN@Q{AOa=o%)b416mzE4;;{jB~|BQ~qt zGj(X`Nyin&lg!A>ADJv~W5wk}2-gbfUJq$e%d~R@nCc117~H=4`*zh6O{T?ud(d!x z>Gv_um#-2)5SG#%%7i75@nl!N&%>p_RfApj$H(_8R!-LJxQW{livAPz*=gNoyXb2d zNID1AA#mWv3X0HogA!@_beAAps9>nIw3aTle>8FR6?M&vZx_>^l;Q1kyAa9S?2rmD z>F8>6w=UQ;%W$29AM7#QnTj5)Q7@ty9BVF~HW#b9`n zq+9_YKZh*SC$K?#3AuHKA6ik(Cf*|4QsW9yDSMU)#}4_OhW7ae@BD73?R4vo2|93V zzNpHE|G635her~d;X%er9oz~dPh`;nW3iyMmD3>H95K~wpLhh;-mhQ~30&Lmn!*;@ z|D$EurkeQLYdsHgrb{m10AwK2Em$Q35&-=Z4 z`E!o4=&EmEHazfJm+feHBD zbs-?;yo=x8su`deBWRV9Yi6~m`vdJkG4m0enn$B`o*V{dU6ZwbT)?bI>)=TTuyt2- zSyPFU)}Yy8k>qb2>nf7TD(ZEzS-1sAHqpE3G85%0`ex~4MFWgZ@fG(IvopS6{Bb!d z>dwwcHt`UyaKINV1Hp3aZ&4)38mL@Xayint{!z!r^5dv&qu)K5-gwfY_u!ORiSpl4 zlEPF*3tb~W5M4x5gw}2+74tZ7jRoF6Jdzh=u%Pd_(+N;IJ)i*%1})c(@PefSSL7Bv z*ALA|1#dwi)WzFhHOv3W0HNCQGOO-`el4R}mpIJD)2?DOfS?Fn5Jr#eWCwH39P_rx z!YvU04jQ!Ar|3O=_sTk{frt19(Te&F92{yT*Qbfs)9^E;InSvO zUpWhSP?P4Dc?U$MHZAJz3h1}*jC({P^d;`Ogq(WpuFBS5BQ1HvO*@SfG?Bp8AAJbqC-!7}EH% z&-T@Te*$n({#VJl?wlLA1&a(7M^o1DG3Y?m`RG30$l3BA#_2bBo#{;$Yo2}{qYGLd zRlE!cJ{ikCUnAYP=c`9N25v{%J;`}Wf09lITnH1T50E+7RG|(@Q&U_G!QQ1HKOp|P zP-*HYqU5+*(C{pp+jR>dUwLKGKID-yHeS7M^GZ(T&8~%X>w(;C?lAh`6V)q3<@X1hP-iJ*~Z59>M6nL#DwwLhQ$i3|i4+~Z<%h{bS zrcZyjmzK`O&Ie%o+^o|5PKfmC!m zc79#`80!K`o}3zKHxc9tukP7*O}hMECswMGDYCDbLw%6@9d=B<31--kiU%4(VA;=i15wE zM@Bz>$@L~;8=~B%gQ19G!$nl6SeJYbWz=5!n-w0?s^L&OfAZQbm}|AQUo8{Km+8*t zt`|n$nOWPE?UXRE6MmX~Aq*wV!-KXF3Z9^Z>ux%VXZTJq#s^CCzDvxM~C1h>AB8V}CNY^TwN;qi=6uxtvn#<7-iz6!cLb^jpFZ#~D zZI@qS4a$hv@yRI0(`Yz>3E-jq;!2=;?I7Do=rwlLAkN$E7ldwKg+0mOv%$OOS+`{< zvXu^I;OBf@Uo&9)JN5YY`2puETxT)L6U0hC3TIGgyX;rXRO**tay{%zs(M;ut(a`e)3@XT!PRpjOzFNH6&lv7Az|IGuh zYnKzzkX3FJF7Z@hljo!pejEXn-IOLU=r5V$;$^MP_T|s|M@hS_r!dXyb~|scEOIxl z9zK|2EtKD&yS&Jh^$LC6vP&&H%n&GmiIdy;i23E7=Df(|JK}E2491V&RC{XItOk7Q zEOp3pkUV__3r24vJnMHqbJ7h?H`!%pMx@2HPT1vebG{|Fz3l+_^L)g#)1)J>v}xIG zGg5+79Lm6}o%0%e-O%#)lWpjV+=G=Zy#!v%I>Pl|avA-aul{q)E{Yp!*MTskG0XU$ zW7g9@3*XcPS!h7JDfX~~cNFBvwQo)RZmW=?AspjGj^fAQ^Hms55M!PChGRODo;!|Xfn1eB@=133#xbVtsL>gn;aWh2Gd`)n z7dDRGkoiv^%1Sz_=6&D<0!F9#NgD0v;r2)S*&cJlsDq-2udh#EkQ3$>Z+vB^{_NyQ z@e!2z;{3~z-xT+a3n*9MZP*+%fKucF<@ON;<{*<(Wgt`RvJe;15n?v|l)VH$YPx)Q z`MFg&Yj5=i$MGc)dx!oku;SSc7i5`u(3O6l*NqavpnA~CikL@>=FhquuIm6kB(-2i z$u8XK6m!VPv?C03hM7+ z0^O*2R^1GLD?l3Mx|waU!7hHqfo^T_2AAj2VfGlY=Ow!l>vWUmIs7Ph)$qj?{x2RU ze0Pu&IVo4j`WIgGVz@&+yEN{d{v^>AxYLye>V_0W`P0Gi7l!Oc0J@jWoW#zkt5lZP zwM%5hq6=;1ccx%IGwukhJ1>)QN!Ag*cG5eiywJtT=s%;oYVvu6Md}!j6m8(rA5s@|Imh9Fc_6ma&#npBS!Y;s>g`TNrN)K&TBscWk2=KEuAjhD?4<`|jJcijBZ0i;cG z;)^od7_*0Uyv>?huh>6z%)ypjh78hN-Y zES&||gP`tHjC4tvo=9#7xsfZGXJYM0A|BNq!9S=To-}!0ASlg-Cx85ztRSj;&ku?P zS<=&fRZf0dtfA4>-x2*hNS&&{PW^t6u8;^p#q@0({m!Pl;%>*_#nJXef@|Z&i^w}s z*_u(@9yd zgaYZr1m(pniY9`~XTHelIFVBig(@NJlkN*>E_@#Ihh;$mH~0uqBlZ#ek8LwyWmkFM zAHX*`H?=pO=Qn&fO+sX9c)SUD^OGqIG{aJS@{U_Y^;}Ip_E@RkPw{tDNO&dvmFd%$ z?A?b@B+8sn!F7tbB))Abr3lUKd()=FWtb`LJ7*a{XnXa_F=;3WH6`Sk`Y_-oMx=TF z#saoDl?!~zef<-)m?S|CQ@!#Nfigd5NV|4K6!FWe-%sQUIMtJkk?{Snes~vt`;DEd$y>N) zgRMbBINZv*wOlsE0)xICFO57$1RH{)I&qsNCT`J{2u zS6SG!Wce|)wRG{D1e+Z%+da7!Kzp!Hx?SKRxGV9vC}{8jS6{w6*Xc^AG(TvvYufdN zq?igyGib69B!(XRYJp$IrgC-A;rj<7tjsT!6;6^EDWy&Y){hwDR6xt04>tC_OYdK~ zh>6JhO+1m~`I@z5=IsCXqgs&WcA_jE{oH0p)#;q^uX${i*1Ngs>3@wF+txSPljM=^ z(Al4$W(t;XH?1TzzLqh|jW*M@d;yc<{UqJjCnD4FcNBNVyA$uqw$79=3xp(CmAVyw zN}J+(f8!NX0_uQ-^4a#rGq;zWb~ERN7^4WqapE?;1+74F@?FS2BGwPe(*=BaAFpGq zh)kDFr(zIRV^qT2g>Svi`g?gmT4!01&rQ;)b_#rF-Sm~mKMb}My?dY_@;QI2E9hB6 zS&1ZZOEqFMQ9u*(8<7>k!j#1#s=KWyrYiPi94z{bjyA{;cK_u&?8CeX_TwWGTMxtD zw7a3-=ZIFw?RH?Lb7tuu8l89#iPE$K3T+RH;D}x&&i5RD7C=HOLT>LP*7ZRZ^poO1 z%2izz>z6v+4uHF3v!l*HL7-3fP#q0Ud9^g<_Qxirlbvs{bP)2{ZT~+aSS>Dp3&|#O zn;J&o73Vk--NTa7CG7V|;sYivrV>>e)_^8al}B_k#UCtXYk-l2%!Fg8pdiCxoVfP& zncnX?O0nEen3v->LH7y;w;t@mF$*{nIx0Bffz0E-elU*&^?KUr;7ihrylklJj6)%b zpUP}65^PQ&*M{IQmm`~eG4>2?O=_HwZew$zheceI+Bp*&d$wR)HykudvYE@H(vUgpnqg#NPB|d@^&LM+{}~xurZH=;~@b;C&f3qQMUCLCx066&X?1KX9_c!btYf?StsLGp#|`>DQxO%+^xk)f5^*# z5VCyQKoO(w4Gw6&@d1K5?BiNcsj91+bpDI*<|TvR^K@+zd)Zav5F$BLmJocgn25w5 zqGZn(B7PdiJ24wRi8tg+6R=Dxn)19eJqWN84fFMl3pX^%y zfbI*}VbQY}{Re6&D%|PRtW|YddG5*8{^4xMgwFojqfUC-13MkX%M6TnaO=r3*QskD zZ63M3wF{vuI;)0AEB8Qbh55_~xc0|-h_dOmbvDpP>E2e9t|1zQ{mpo}dkl|Sxc!Di zA*ai8#p4hu=cz?Ckd^zT?ag1C`R^>(Mx+lmc!N)-wbA3xmi|zM2<6e%p}61JxY&ER zipbgmOd%F<9|GgmCJBG}h5}ni99ozjRPovIRVoS&4_#rVAKx>x^e>Rz=X~#fchtqP z2gRQ{a!AxgMTJSXAMwZQFvS{+g=MU}KW_#OOu$C0BdCLBKWHm2I0{_pma`cwlljKF zMtwgg0?&gv%DJK6F854dZkVxUeZx1LLE%V|QV96&?UHVV<5+wJf=Z_Y8xNzrTc9(W z*o|U+cUhEsTMQ$8sQaU|iB)!tg~ytkHXU%?Lr8b^tUr8xY3=81(}h+*jNNzk!3#N# zCs7b633g;o4K%In@7&Q&UfM9tONK!VVZP1ltFk2Ie5>nTb8a&b>mM&Rtr#(v?06J- z#6RQ-etIpQp~T|XdTz1`1l0P)Sz)VIC@&cD>u~7dcX8=(h?*9zLyZ^B0kt#WTk^_` zb9dQ;R+Po6BmA6KO$=mMt#Blo7n2^Ze5;}|p7LSI#?lzk3(rirvFo3uhHcl=8J9X2 z(o@_($kOBQQs9Vpso9(cW<v_7x=8e{vfDKE}rH}=4G$sB&0o8Nz;>QC+*L2Fkz zX`<5jkZIp?;E(tU-3@$s5OIR^BQ5G%eY?-B%Yt`m5N;+BX47t<{&!%qj>kxJyk@W9m8YJ?U0|;8chDOQOQttwQg8m&!-AOR^;=X0s4-2GAt3wYF#x z`2OucuC3jt^BzJ;D?~I-Vbzrsd{|2MFFK6&$4vW^2E!0fU>LU-Cw+hWcj{0O_DW0* zVy`#uDW`N=Z8p4C*JmLZ1FRiHz1pO-tx8`+xi{|mzIG}nkf~4hE5C^&p-Eikia^#6 zFWrf;Pj?$}W+aGQIt8}Fy)*bgm@n{V5|>m8e?Y+SgVgz~fJXFQV@?AvEs(6aR39-m z7yT|{m{%Sjxa)mNk@MZH`B;h|d{L0;(-M5FR4dBq0y$HcPqz&A?mHIPn=n3jI(6EZ zZ2HvglkS@Ek#r8rGwea|B>^FF+NL+@5)uRSnP<|)@75reZtMn+RN&eTmY!IXN?z6~ zWlBu~@0e|fa4Eij|_@TMtso&iZtV7&>#%-Q zOKztDoH~(wkdjY&rZzbnJCLI{NcqVBYHj|!rWlFN-?T6vr z0?==al7G}hSxWOU#Vp%^YZ}_or7d#QK4U8y1=du86%}-P>&sH<=qP6eHkqKL3Wk-8 zVinHl!|P3|q$;|~vb2)khqp1RLK9+?NlS!n9=*mXN|>Ic1uA|vp?h1h_@r2CqOwx8 zt%F`>jk~hARqqx>y8$owidwgmzyO^ayhmuPJA}yP~lw;vw^WrCIeX+R(kt@Evcf z)t3zH7b(XNP>cS!pr(m>!LEiI+#{n_Li@C#oX2F)SdvxZAF6rT@o#KNyWAt86Fz@c zmw_998?4+ox&D{<^78bQ+h@@xNakd5jSC+A3{m}wDbVrd0jta!Ln$sDw#N`NU15$y z$pE<@mZc|iu8%7@-9z3@lQiNND3cd6NPc}^0aI#-;C~Z-SRQRsUu&$#8(@+M3eKSF z^%+p&CbF)w=&$;fgD#4-u)lsCXe}V8S|4b{7h|pFSQCbI?udf!Cpc*AEKtT}Fv@zP zI{CsGUxrUC9E6oJwpgb-*>3;*T~w0q*)2CFwrZ0v$dK%RQWT+%8ElcP?cmMC-t0!o zeGh2j32WI$G7f^!?oG2X$63|6R{3#BrCTO4s%r-0*f-E5U4yG?k#|Dx^2|Dgh$DZ* zvxO}Y!aSCmDP^1aTua!a@)?rVv>9@hxTF8xx)*xxeY5@eRFKVIa_=LeVfVDH49VvC zvR;SDr>1UvyFZ@vLY=hlm0nOGeEnuMmRoZG>|j^%?W92>jOf921Yjln5GvNr!x<; zUZhS3HSLmc4k{5Yu8>xC^iCMHD4ZUAVEBx^eSyDHH#le;;s#7Io{;E zoq}_0io#mhK0Wf`nEl4E?ilNT)tvjN&Wn~U1&xACH1R1+E?8?CanQkNOvZAiM_Mwr#!{-qc}5W6V58w6Xub zHII8ptGM}b58dypGOdw0%r7Mc#n9OrQGZh$?_~-PaAdwQdpW7a2-dZhn30%Qw=rfI z*`jOHuj%g7HS^WVixUM~jc3~ZiqKjU_FQYw;EUb@xBM`<{?_UbBx=}eYSC@$=bQTm zAtVN$#E5Ix|F$~1jtU5}h6@c`9o-sl@YhX63wb#3pB-vnC%Y^z%};k{>=oLaspo+@ z#;ATTV2^YbdSOI-K2^*NPVe2;(k6EK{vD({<2cDfWIQaqYW*K=Gilqck{XX2prcq5 zjK95d%FyLEhZ#4M{FuW5(VP{b13CVwK0n;CYz@*|yb4J>RSffNXMIfAY3q=A&Ztl~4~R^~iTf zbDx6olJ}R<;U^K};dJwRmVvMS(fcS-I*x_2Mk2cDg&a?sx9EAFT_9BG6XEX{&yB8I z8P)X&zKUC_Q1Oi{9k#dAXNL{oZv2MDVo-q}iYmmvW*L@seaZgDG3?_Jj0SSsCb-|~ zNZ@^*U3Z2iGQ!5BF z7qNXFYSE}gtJhczDbSwjBkD3w{+yCw6>XLukeb? zcCM<%&k%-t9;EuE@(*wWGl<9ySWJo_l3sY3F;`-_3r0!c z>m%J?w~dyjK=e*c?3Sy^!w-)yTh8!z-Gb@1m;KwlZv*{CV55Z_G(Z!i`AHE9X3%#Q z{r=Qj`uOwYgI_5ZU`IohlcphGF(5NK75Ue4msCORIEp2FpETMFEoU`rug|p+8H79E zECaQJB}^*9C+YgmjgHDajoZT8c|KOvNb*(h|HhY%J8wzd*i_Im!qQbv@ zj6#B=C@$D`zgROps{>ec1H`6UIa-BYcJmddp9=okl4#DqMQGrDpyEDPE}PYEoNYdu z8Y{WpJXF7zv(?+h_FBgx+kI#*V8oiH>p<2l=)>2>{xbWbTV&4GxBn&Alj)g`xZUmm z!=tu@m=E{~;1eqIuZe?iPd!q|F|{HYzFFq|nK_$%U2KPD zw@bmllG9@)zZJA7Ed8j*KMNgCEqB`NPkap3Jkr}N5+t|o(Vq;GMV^RHov>nHUT1I>KE z2@8($mGK*V^?f@euaLD!J+xdbAC8kYxFR9xIkCXe%EQrf^G5Es$nv&gWW3`CGPr9u zy-AHkX#L=Sx}@0Wua38^b!>sa??FlXJ>X^q#fl$ixq@_$>3ai-v1??tVz<=g+l|$PXFzSzHRi<&o_g< z7w7bY;ag~r3y*2P;IDQv!PBXK>HGmZ5Z=@E3u0=pwjp=#;1_zIrbS;YimBN#>u&z0hz`l;gD>t)#XKj30X{q3^x@deIFU7D zwPvO%b)p74Me6@P=8nv(1Lhq;F}7sWw&k7YVN){Jxlf{Dh}Qt<(e!{^&JD1b^k7e8pM4-J;5|E$w!6B^y9p%ZLa(_XI?~||ox5IT#7uwz`setkt70?cqr95>OFg2`Is%-l z<+A!0Pe{FBPzv#B-eUho>%#1hi$Tb5!jR~D2Z-@pZn717wDYh0j%Kk;0gtKhrW$4P zGRJ?}&S`j#IX|4N8ZDz-8F~}IH34?4`FH~c6a{{LNBI9oe(K?5qj+u5+Mi}p!esRS zv?j6lPF9fFO!il+mBnTcmEYJ|4E^PYf};esPrPwH$Qkf3<1d_aLuQEUJvMZv^xQgs z1kYCpJP}A3bkXkS-vOoZvi8Z<{B9gppak3;;cx9p@2pq!zxFuh7R|G_#xsSQZzdCq zDXf&sx;keVr%x}PdBf{F9$G)^O{N()^XdmrCA!ajxF6nXo8tR2*`F(EQvB;ZZLKG2 zdQkk!JQB$vXQb~h=-N<5kO^G{yXX7#8o$_)&3XR&U{i%v`8^S7u_XDGQYKB^{*B{v zlTbC!2t(c00dZPqVtV(Ek~3GQ+JT&SM6s#kK%8IK5e#;Vrlo@NB=MW|;}Hg4lG`U} zsY~dCJ492bY^c@iD|0lW<7A+Ni_)^3k=(ScE2eMr74G(Kv%U-?akI(YFij?D#8|;1 zd$ibyT8SsO6xHls2t!+z4tQ>bm z{+nEfkB%f#`uDL;&yLcL;rG+b^mxbAqSv68s;<=mU990G?Z_(JcK9Ug(#AdCme5uB zkQs)Zk{qVKhQG3DCWnuTlc7t7u7txj&A_R;KL^9G{;HMepX%#HTNg58dSLI~k%Td1 z5beM;;@x4BO`3S$uTG~VEsQF6&o-7D?H){8U#_ycl^W(+sg_To+5XWfm(n%M-NG1& zn#&y40F=M-DVLN};%k)hrTfN^O%?NBydmEIWb$p6nt8=P4(XJn3(!3#!6w)tX8&fu zO-e?czH%NWZvv0R!-%WB&SE9#`@`POi<{-9WHI_!S#-I?AKfX)#D1XV!c`vw&gGp4 zHG9)7jG1hiLkzHMS&XzFm?-R)$^~_O9%MR5!)HgAg!A%x?+u2psjy!6ypjfBViDzT z&c1e9^kU*k@p13ZF~*Ft;+AMTp1Fw6=I>JD0a|ih`mJH&sw(fnQBT&gMU%8D_LU1& zCv$K6X-T)}v1Ek7i{E@p_9|E+xN1k##eWqHm1j@6YkQ~vx`n^4Ma$WOQ&W2`5Akn)YH$fp+7)S6gKSBj>Zi%xw>Fomt(}=nq^)&rQVRKXE^aAp-l5X=a`X}w3_*V;HwkNfcUnT3|L&z7 z8xnKk62blD^qV;DC4-3v zc@d*=6?z|*Vn7M+vT|RY@logNJdz*&Ib2ipUmTzG{@;!lZ#QUbaFZHtZiysAoCYz3 zQ{$}tDHj}k;3&)$@^3t^hLzn%P~2j;6SGJKZDN9B*bV-f-m9S+JO)Rh?|gHNsEalG zHQ7yRcmQ zJD$2R%kL zSBWmr$c{5|j65W_u1Ga1*ZM5F3OzSK(mL6cr3{#W>YaXg;69di16Zwgi#vbX;Q1w; z)WdyWA=_Az9--ZazkQHCDmKqnp!$rcfBareNayI_S4X_K)0uJ}@@Az#YUPlQC>e*r z&!g_eX#-KGvBu|iChayqD0cEEN*KAR&tMnJ)L8zRU&Gt6QVF2?s!n8wOe}SU&-n6c zWl)l=S6AIZ~O6?9C7uN{3sA8QTsy+JxALp@*Qf#J3 zl-{nOKRaD>_HUx-rM(ECzO(N$NoITVRZR%)l6Bk7KYDR(YaGLyf$E2LNE1-1(vtTq9tB+`$@{- zQ}2ED4;K@K)+4h)J`|Wh<@=bh4`Z0PeRkJ*)O-Na6O(Off;-%b5BQwdL??W)d?Ab3 z-_+8(BS6RMcaxneRkqLV*GEOF78)*sDE8n`0Ac&G`$6?$JC0C&mgn2v{K zk6UKb@chi#5TDBBqx!ki6jgQ@6#Vgh${ndaa^jX_*rfMKbh>;f1UFku_NqACn|7at zE3w#-+8g+JAAkn(1vi^v#E~_mGaXO9r#gqE^x`0fzE0ZX8!;L3_MP>ie=dE4**);$ z6Jf!;J;|p^KBt(qcK`C)U_RL=oL9Ry54?$-*yt(eXZaN5tJjZch+4J>U%Y<3G3blD zjxN~{w_G@z>pMruVPkmJ$5B;$)aYf{lsAW|}t6-5VaJVs7CntuIaa4fv$Ry{bHVfHWCqzE?Iutpf@ zTev&&%6PjpEc+_y)5fVhIZQ~7;Pfz#pgKp<#@zK5}NM5pe8=eCi+TqEF8yUW*=zN_4>~AxoIs$ zMm4^ZZ7zBDqjH7Ycwt*hG0sXC{oH#<76MF~GORhHidlR4P()0Rh9%$JpRxv-p=r39SahJ$vc#_%ep`KEAORurDZa7 zg*f-C%ydEJMKYjUaRM0M)R^tm{8@QdDKi<0>_0YMavk5W>9^03ES zkr4OGG)WkV#e1HGf0>-hcSMVJrfVgsjy%l=mdU0voOon{6!4@I!QJH;zhDWq&N*<> z;3u5tDvqp*6K29vNHIyChg2|_5_^OeDc?;D`rYOGu-uw7uJ!V+7mPoK%R8IYZDB*o z-`4R`?0+i!9^Mry{%8JUA*x7p)$RuHgVrji*z7s$8M^>U+{mK;N8JEZON0eDU9eJy zy!YihG9xn*f$>3LRG`^dU9N#`#PG<;A!a=h-flO5{sOV{e89mO1S+9(8vm7eAO3u1PDrmE_KdSc$0 zIfyg+N*Op&r33h7qXk%iRySytC| zk{d(JGdRON?T%%kD{2>k_gviV+r*97YnomkxQu{=X$+(F^}IOCUOK+<*-M_5TcAhpDl*wwYaPigrx#e`;ufxhf^% zwlcxvkE^fND#Go*s5-uv-lpn*iXEC5&!`bIFV+Xu8J(-21*qW+%gPS0t|~Qzi@+I6 zX)1k;`m?_5=Oa>-@4Q8g*d;FtPNfcuQYe0@ln8zenfjJTVT(3PZ?;D6H^jv27F%t} zDnSp2ZE$w1#J%UrH}=GU2vRW7N>ZJySUdZ_Y1NJ{w@a_BTPxJ0U-eb++x*fxNF$gXgX3}%{YrnV-6OIUNKeb zXhgv5@%B5#+c7ZYI$Oc6e$sYiarb7L12Cc0n)f?Y){`WgyHaN143-T*NPmYm0Amk^ zyLaS;XIV)qXJ)H&<&(aEmVvM;6-bxtg4k%kXRcA2p~2YZ3vJ1q7oKEq9#f;K-p6>@6UA3F&kgSLP%rnAGhy~UQ{nQL` zt3V0CnBIBDWVuU;D&N0M<85!0NUKKPDo@p>0Hb#WtVSfoS^gS-hV?-O~x@!vn}Xa67{@UeGr3u)&x}z z?_QxUYXuvDDr9kfx<`2a-OXmFXlFAyFb}Efm5HgQ^PeWP3?%HvBa(Y_3jYW-~})@Y6kCU4CVTd{DX4tCql6PPv3`UN-KO%Kj!m6X;N z64XZnpivN# z`(L{Ig3;?ofG-8cCZMJ{fiR&=hTSEeh{wO z8SQ?-44D}O0h;}J|0fo}mT%rRbpC_9nl)X|dPba=)(pcYAlvtx_xlPjJRqYLoQslK zk#C5o#gXVDxRRy6!usW!%S^!0+5My5GEh4x#Ghk@cmNvQ9zTQf<9gW9?BR%HY%CkWbSj3u%m1b%M+xfoNknKXeb3F0d^b1wU>ydJO7V){jZ*Xs&_** zJCh;#EE5pdCHMG9?rYkab6kx4UL>3Rgv%s7E01>bn@!IZw4 zozt|iiPUjfUeRd`@;!XD1B zj&r4=nv|9|<_+&+rZY*EFEh*!mq`P^Yedlw<2~$pfVbBKQ!&bz8jo?2QmL4kij9w+ z@O@auD0uE+my8S>5V;O>Rg=O=hyKfuh+0G!2vMB|?4ABnh_If2|BIKDeYFrCV+TmC z^dJdAx1~Z-R&s&n25b2ktWkS94+6x$awFpc3##s9 zoX)+C__IrhSuVnFro(YN$W5s;W5D1-n;#SCyZ8qQf1Xh5v`HXxrPxRvG3vUT-YF9q z1DIU-PfHeMImHoNV#4cROcpYifbag_({@qu`3*so^NaB&-eq6gb={RyKZA!0wvm2= z+ZrUZ>a6GrGgws(0kGy*XU)uYtROu?j4Rw#_ve6DfKe zG5U5}w$R1yOEb11qi~I{K#xkzI;EQW`>*$C9$RT_v2G@$$ z@xbC_V5Q86lUE*|Q+qbyNE%`di{TjzoUIC2Z~&CIB^;?~A%=V?5OPYI+8uHutn!$Q zqM`wy&HjR_wq|6B$Ag3c8DOtds=Vnj+TkqHVZ3cP?ZFeZ zfAT$)6&=%s6m^`zaPBvaR+yFZTH6^-XOb2X{~>H}p zR?957Os>m=_|)bxeuL~{VlV`wH}nUrZ-=X`aO}>rs5)wZb$iH_+0jYyt~~Pruy9q zz?zhpu+3PDVYZqbjMJO6X_<$Y?aKE{&)Z?e1(A(V)%7A_ta6O6cid|&wFZP(bEe6Ygc}1yTl+*Q*EkHwBov36&e5E6RFZ ztc<#gLMjxXNL9HlF94!3*mM@0=H4*4YL|pd2XMvqgN#=rSuk51-$}~`Xo*Knhul|K z8J~xT1kzf$0dbJ^bjuwou4zHvtQ=`=nmkJi zwjgrrAV6lU&)BAXobP1=DZP_2<^N#py`!3pnto9fM5QXyOF&SN4vF*tB2pBj_uhjL zI!Ffr=^bei2q3-pAiYU|5Q+5Ody(Gjjo*8|d&*j8-TPm%R-Vj$X3zZgv^~>@d9?=; zm!O5^AbOwi&y9)*iZE;fv{!Ay-CxKVDzB_qr}u=p#$ttUWEr5nd-_`XUBWPI8C*Uc ziap{B#wiXoR=qY3c0F6lnvF7g8ae92KRwUEM_z(O>HZcDV;jNUy4kIpA~z+zWvb3W zynJ2*yk+! zzcpzZ-N~Do62Cnv;~P;DChRzB2h1sAI=?=^K284FUqJ`Y8wbhjm4~Z2&f0ty9x|Be z+w;1DOjk3S^w{~M4i}6M(xqB=OL;Vx`(pl7uEOpU6K*!P`OGr+D6nkNNf+14LvHIH z%|z(0uOwXpgQ2;LSMUrLb*_V*b;R_+$GW~d$;>+zUhi9w>93DOPLJSR)(>z^n&?j! z-b#++(-f}1l)NlW4*3-~PBA{jP`_TQ^LV-b!QF^U=0!EtNC^(ER{mP&Q9i}!QS-_L z=DUd_vB`qgh}+bAw$__iqiNFQSc_LJMbTZ?aKFQFhr8wZXlkzk4`j`}_i01r;Qik@ zo@s+PJ_SbQ|Ntkl@II?DTTHB!9DO;i8($CpS?no^Ax#q!5^8#rG`Xjv?(2#?U z3TDHo3{K5T?(>#Q)W-8VN`2>+|ehkwQ%IsWF@eVZxsN8#d4OzQ{aC^AyRo-=HAU4t(&L)hgG6VGWG4IV#oSE zjw>IJr~&Ky`>5a12pxc-x_<=x;Twa z8n3+<;tWACR-4zO6j3id@`fQi2P$DyLh5bZZf#Qc0PC-u-Xt06csS@jHJ;&1*>h{H z1RM7CEodGSJ(k}OD-ipc^V16tzxikOZ9=FEo;dHt;4ZEFhpq7oTkXoh@0eUeNNxv2 z9okKyShKlGuFXeMRC3$-R&^-0n{*&B-~$1GUN)Hdb$f9ZvDz`t;{qIX<(X^&46z=<<+R-9eERXqFB;Oi@ypFr~$7wVpK`{W>^t+~n zlq`c9F?=M$_>EDOYX4D+^UplUuRqK6llId-9PFN}^!(zp1|C{kD=i-hyoh5Gqmw@O z{QX20u|5xOd#`Nq$MbcZ%BR~0@7MFKK41NPT9t7YAn1vQo0ixFx0l`h*>xmJXPNbx zxv9&Q)8Z%pS1u?+bJR<;omWK5Wmg^tS;q{^&BgV{Cti)7Qq_Kz(Lbs|?4^o74|A8A z2_TO8_g-hc^Tm-#{*ytq!}f9#Q23fnM8%?I*XYH8x~KY7t>3w=%htOoGLWW)dF{5a zm~klNO~fX(FwxG&4uJE00e)JM=Ay64m{a1+r(*TFTOTw-pZMmnnA-iO2ZH5hZ-x>8 z$tfsmJux%+bwj0;czjy*`$=CmO3T^Ban=o;V+TGc+3GzD+LcHq8j%(=Y{#MJ=i4PY z8TJHg5s7GW1Pa*%KxA~+n5JrK?w4bHsCZ;JRU63E?#qJ8XCU~q!!#aY=HFoLCoUu6 z%gqe21C%}mjF}6Tck2L$p3pbwdM0%GM4~d&qDjsZk3N) z-N2gcx*WXtRI`KRi+F+bumDnZCCSC8g)`B4B9lT;<$?UUf!eRb%F%~g+9IPm z5bNiQd&So3KSKiLhy&qfg$uSX6By4wg#=c5ZK#!e>D;^&spXNDq9)a*d`ZkN=+#Rp z%}Qx1#VpPo@Eme)&GewrC_vM3RG04wD5G}aUeI>B`)!Qy4}3~sCI^ryz}?(WX7tG2 zP`o!LipNA_KMJ>-@}o{pTon#}#OIK&Pu~Ru!=Lt&u#~&pLWqKUwHgix$nLC6~bAIoJM;;~8Jba}|_`#P&qTN`*NdxJqXxd)s7jhSrPUCB;@!yDI z!uMp{>f%MbJi;9ZQtJgiwDADt87?#{zTvP-LX9?9&2-6e0IQ+}0(|M-vCi)sC+;KE z!XIVaOJO(QSpVgfsGZwSem*N%HWR-I)V%6q)_!?_S^`usues&AH2zk|JAVlXK%Rmwney=ohoE8&9~Ze3DeoFfv`I$AhpJt$LaBF-oqE z@98E=U-jrnwvPKY3~I?%otM4pAk*+K-u)GYWHh@##&$d4`OKtSwXk=L+A1xh=$P`n_&x4$?Z9a0q)rZS8D?y`2Jun(^u*{y zN~_z}ri=@I1BBaZi-yo#L-Zqvc!E+jG5ma>lwyfOBPhjJRY6&24Joa>tGYq_1AU7) zcNzTdC%L56OCEt{CPeEWSS;3<@M9tllCe~#raV#$geB#y;|;`4J5{SBe$F{l#x@AgO;yWU z#>iRr_faF%a+CF1PAm_kzAqsJ0c(pMIu0zg^2kc6)yEu`xhN?=n!a=?XD4uqGjylL zl?YUsl*HXuXQ`-Cv}HFt%A)_k6#MtY)f}@G-u@3>vH6KVI4Ut7Q{3MrB3yDL#%tZnH=b6bW4Qmf>CXA(cEL5G|8A$4H3zlgX%X=Q|2Aj%YWAwt%TH>t-f1=eCD7un{P?4f+2Z;Z$`ov4T_zg zgeYjPiR)LDrVg<{lx+J?%y`ilul_7rWF`8VKHMx2T9(2DC?SnVKS|CsCrl{g)mI-Y zgbJW4yiP4yYI~RXU(K1N_9Xw1?m;>^Ye4(24a7?7wa^aRfUgj%rzaOBoyB3B@ifxjJN(NqKn%dV3}zGuu_4sRMG0Ahy?LR@it+ZmIFjbASBAb*2)An z2{lZWiihg+<>BBgbJJa2wzP-^z3+Dx!CfUv1nooO?=@GeQr%6<4Ej?Zqy%w6+A%nD zsf0AM{_(~_`yH0q=>g98D4JK0tp9vCC+vNr2$Ask{LZ!;B#wX|H#OZJ%~1;EtNQuv z)h6q50N=Qejxk26|U%6)#>^ zQ`GRsdewTWhZ+|wXPJ~d0aa2u|+B@w{DmnD(A zZbSf#(nT=+F~-&^ASpcTD>7idqT8D#;Dwx%?kq2s%Lf2lZNPB6^;YYsMLm)s%Tv z{bK=E_0zYF(v1~F0R3GT^-rrq?1GrN7%)U=G#1x%<0Jkj2dXE*I=nma|34T&=b|S# zsw0>tgwRx?dv5!`&{QS%uJ^PeV$swBl(Y0W*ogpayZ=MAjEbB4zsbT{bT~&HeJ3L( zQ{}{`z=iOZN;H(CW`3zorTK&|ciqd0NDIS1eH39gnSwGv9hAUfnq%VY(IVVwHe6}H z>83d#ez9c!8a$#_gb@YJqAZy$hme0lcwJ|ewST!+Awat7b*Br-@}a%cWo&ic|0kX* z18@H8CrJ>K8AaK?hJC-uToY!zHPO3lupWv!qrizeDhp{)LV?Mtq(}kQL?@s3P#j)A z0fFXZ?FE+Qaa$`Xt}KPGRz08}M^RQGEqpaeu;Ig3aDMa?a<(Hr_;zRY?;y(sTHR(Z z(*dT*u`rk&`w`yi^TL?q=y)`;`lL4x-F{uVzTD|$rPVT8S&^lNuq3+DY^mmF>qnUj zp!*I$ZKEz493Z0>KBP@5NZyKbwO)Q>Rym{Jl4-nqOv6q@n@iuS#8e5v-OalmdVuZ% zaLX##0l1{uJqec}EnX{npc-yVeZ-Pi722a8DQ{&CqtU?p-^gaqe`fs?eMJFS#R^?1 za1S=WRmQ!16#vM^2Wn1h`KlEHSgrRsEomh;w=MRH3AOBpVi$?XJ136#<~hOQQVl&a zq5059=YroyOFI7j-|JRVj7M4`^#x^tbp7ZoFCI($HHbJP)flK|t1QI#K~goh`9#H* z-8H%btksoHFL|UdE|&*f9wjs{4G{^lK&bgeTC^EZ>8sO`uKxz=6T4K~vHJxrSpZIZ z)48U>Iw5L@$aJY{WLKS)l|Bj#?LIMs9sC!2fuKUT0xaM{7t<4bf-dzcF*TOTgZ5LO zmUz)g9&yHw^0D<`uC*|#MQD$cPU;fu7QxCA=b0yYv78-2c6rvIu`7c1`}vcWfRL;p z=p)OI{{z>0Q0*zLN4P&xYtET&ZIrqZ`mc>wy{GbC6h~dS^u1}2yLAy!TNUpApTPDC z@&&YyLaqc1(}35r?XdtS&`$4t-a-G)i4(qO%4ua4Rb6FuKi9h*A6M9uOU^m2`|p?r z$M9sG%l0Y5_IT-RrRPyHx4!q;+3q$2HO)t@U*;!nC(+T--`MK7bZSldIv)%dD0j7a z7^nq!*D*bo+i$}U^EP0vG}f03rkr=-rv5ZZ-sgeT5w}g9L?wp`P%`E|J}iXl>7~OI zV9_*z%{A}?E>OsY9kjGcwJ78Sh1S}M#Ga0YT*{l&<-sS`0%IxBlV+tMWr1doI$zQ0 z>*k(VyQmrDxE-!NIAPRLU;nQ0C)oY!XXPadc(!D99DFoafW$H&qXA9LRHf;gcP+TpkC5dqKtBZ!%nu_a1pIDkBu+o5~708D|YEga? zpGqemw`TV?RT`}^=Be`}9O9ES=c4o0cwwF@cT#F<#idleza-I#)`JIqZ2(ZsWR99UeJChF6a{i;u7Frl9%TlP|rxJ## z>NsYsWwt#0cOZKt(X`Lo-163IF)~y<*1aFL}{WBS&`SUhta+YTe1cQuOfjd!Ed!H#a+HSSzbk>!z&H94=rE{iyC5S<5x7sZwkd=)(_Rwy>hZt#L9;{SD+x@U4QE zrI!6P0KYx$zQ&3<_WMB;yA0#6F0VBWanQr(3RP+VD-+`W6kgfvn^soZJyvhR_}_Nr!y6wmJxC>7vIn$xKC~cye>-Al78OzJ(cf1^ zkrFA_+X`(-#M*h0C(!;>wPPu8DMV=$Ji?So#uB<}43EN`eswahCGM-S*zHt@7c(cLATQ>V7#^kfCoa=7iGa;bM zGRV;GIU(@H0}VmUST7UGO~Fk>V2YKyz62#G<`Wm`16EHJlD}%X ze6-a0AlOZdZP1ET8kz=bzc!}C zj@$>&cZC9U#El{=pyHX zW&4X!)L5a-;&a-G{qJ&iMK%*+*P_opfkTYFH0-MC991GwfoTFtiR7PsWXY-$nf zvk6xt2LAc{^vB&oFY>(&}OHA--styi7=6C_#{3W3IgT1T87xG_RGDmkWIh39&5b`P)mthnmU`l` zCRNY4`xfDjVNz*TwV5zS&W zGigLkqDQLw>QYa}cHXrOg$7GZ^mad@xZZDa4B3~*Q6sxtYZTGbM-)L+TyfI-l=2@( zA)HAUU{hZDG-htnnn@pE8}_8$)}QfK!Lo^ZG%*mOpDaRVRg+$7DpF|VLoS?2iRqDv zM_~>s;V|w5b92~DG8w72cEh<-{k2VZJcz9aSyx$C<2RWsqHEO_H7;C6g#^V;0fU%> zwrmg3S~HQ!TktZCwWF^sj?jo&Fxu*<@0vp&Q10NgCw0JAV5Xr?0Jqa!lTZEajCDD) zybe?seDuJKR5BcrJzZW_2>F<3%TxUi;2lHki_RXFP{<9W)6aSVAZh%JE z*QCaPZTgA){YpU9lSHgsZx<*0jZeEWZ|5=ip>on9Xsup@;1X(9K0fvU+kjWehSr+= zLfM=(3!( zO<0IJD3AYs^wGbkarZQo(Z^>V`H)iZuds_#;p&35wnKLiX3?pkH~e8PM5*=ebQ@p< z+FR+0@n0_01relQ@K21j$*bOb|3^FX`J?^y61maqsEf`_2RBrF%Kvdh`XA8jYe6>t zeC#RWn!cQtf-}qlahJK6Ht}ccG6qX}-8gb-VUscX9ud=?H zX_B-Q#Z`{%HNZDeXs9P+`P;gsv{3`AMF(tv*hD;8yAL|ZL4a&c&J_Q>tNAoB2qN-> z)rZ^(JF{=|P;|EdHu&ID&AKNgQ6`5_YY=vM!WyP~@EPNwk1~Q5)zzI~Lpsr^x>#wa z<#hSF6DdNI17xw&Ft~KK3p&@t*{F-!sqtX|`9j~EYJuC4Y`#i#Q`tQQ@BSNOrwgl#UCZ)W4M z#a?yF(iN`7#ZH)aIg}w3ENJ-j_gC-X@P}}EKbzV89avo`SD3QHM=dG3f%46$o3%;Z z3T%I{5c#vIj6uk5%Slw%sokz3iBN`XyA$Ly8up2Wv{;}=mp+F(2H0q2;p^2^H z=nwKqCZd%#c9$*PeOznI3H9O$t@9HvyAtzLDDE8DkB)s+$LK2a;08QyVo?cU!PQzs zyL?h*2qv~8IH-^Hfw^QhcVcCCK40X)Z(sZe`nlMd%&% zjx9fXr}SQ!2Rd7dz~pF3?Vf;Ys-PluwZb5C4U$9s^h}m20W)4Hti#p{)sTWl)z+=4 zVZ&2%E=1ktRxdz0Fi+^GIN7q=#BI%pkOO+ifqbdliRFK&;Ezsu5xsS8q3&wcaEOn3 zj7nOy05qQQ+<@BEB4P$+&nY^}UZ3!k8_<+jrMhqlsw%j1sj}lKmb-tEy;M4rv458{ za*`J_*Hu_5kNLhoB_}4vuYF&oZ%@PPsqo%&hWbZrkGfL|XPKTeJ|-`BG%(&%ne!!K z@Aik6c|H4HMZ7FBlZA?UZma8Ty&<}zLrvai}~ah0_Gtju$ydMW|TOuV#vtw$bIJ$^WAzBYIdORF95qmQxz}=9?+qFd}LRoKZ=WN3jyO=S2mrt5dcWx{Vh6)m+Nu ziS_?w4@CC17f7Dh`=7T#YT2yeIg9f2zEE=U?hOQ*aGp`OO?1pj&vi>`@!hQB-^J~3 zvzQr|<=x#@mSj!;{lD?WD3$VJk?(*OO%;wLg_lRQhxG8?skh9wgE}uLNqfva0^)1y zOko6XmN(sHrs6J*kH8nopR8OOEI7pY;x_-j-<|x%0Jsk+EoU7YqP4Yotf>`z;P+%} z;*h(h7W1H%+S^ht1mx`=$}%Xgpz!ktGkuKLa!PX-b7p6hrT3-RLFkR&&eOx&NXhGq>(DY8Sj&G0@fMWR@6jNBZnHLk&jiDOY0v8k zS2CfyN?!~Lws78P0F18WLZ7BTYK(srd>J1D_*E{H^jsEDf^*6GBMTe`;WiHk%UfUX zgY(wO7|(fQ(f;>w2k>**y%X_AcVdX!fIE?=>F8H`NMT1@wOnhN8Xpx5o=ypmR-;k{Ew;7N&@7B&(%B2H&& z{c0rMD+liw9WQAJL{h{Qcxui!AHYXEe|(Jn!A(4*Pa;8@SU?_q$VW!22-#tz_qq}r z*xcQ9+WQsP#+j=!KtNP`P)-CpK=mdN3nhhJgoTNvzv@{cGy4Pg^&>Wy5uDT zlw6)vc#`0o#9YJMtI%LZA+*C)rmqnea?q7wHGk7Z5GM@52!tJ8U5G%7fV~E!<-pJy;K7TmQ0Cel>gvB?(OJ)J$i?fX9 z(;jGVk{Fg1T8fmL#w-zqlO2Y}+i_R{98G-Mm_NLow(rdRIzDxmGEelVkTr;qH|ucHr+_vIk(8zIcT)WL_!~@)k;qPmaclCK(h_?o7pr{bcOG@! zr#5zxTzd498~Z5B{vU49faiL8EnUk7=V(~`;>y~|&HSB`b$`B!Fu>G`S)r%|QC2daZ z?(XFMY(1t9jN2(8hi#X5l>bY6Cf9TY_HyPGs|cfXIhwf(HEjun@tG~fTTeO-CIA4qBHkwxWhfAbbD>HnU{_Qpc76-CN%$dpu(rONMa(1S$8CLfzD@&NL5-LtZ{{CeQGmZ2HKb z@Ls#rYeyz?e1Q|ePjxBY2H~|Ah=_dX;m-Aoae~$B$QRyzfjR=E-5=rIZW*B>?M>zg z9=&opzb{s^8}!-+UjKj~RJ_r18gX{_x7+*db3f5nA@O*(A~$Jhy_ztmI)%q5+atc} zp0qdohe$#x{^93$;f9nsmh^t<7ODoOIBfjAwt{@1`)a|xLZ#brI}1ra>_y3(Wn{X` z>SvmLE&?8LF{ZuaAbMs;nL~>N$%;UWYy#t?ys%rhX38to8@{EX*wac5X{qGp(bO+7 z_0LPh?{4>6t)h*Iz9~j)NyS&fCnp5MqCrnyxM)Zn=qnx+Qae~~=lw&U1egc_St$|L zVU+;8n;51UI{B$Iv;FubWwZ)ylFH{j`9@~iqHpd`6&N``j#z7ZZCBLr&ZPdGip}4+ z6@YDL#-gDL&D(!e)ZKt-)&y=`(e=s%q|ok!PykX_cCVT+1dLpyu02%T&Akz zey2{Q>f9(+n{FIXd=si2Cq9dP2cMa$WVB3DV_2+DpW6=+YACWu?AKb5uNoSOAtIjH z-#*?F%gLC|(YaZ~r<{^BmkB;DW0Jd7Yk+E6+AJEZIoX5V{lyqxhm(7pPa9-`11NqKj|NwVy;cWfwToZUCS5>U^1ScU zOW0=Fw=2W10rM<)Me=>>)H@kqFA)vmiJ!eGk<<9P7i&)vr34~)WpX@ZfoxZnMB72s zP$F#f`$)%cOb1-UpKycC%nDkX=&$jXYg@4M<2|3$W(QuWfyWbRei{{%4pmarJhflZ6>idL@?<;xQs$*VYIq0u+eo`X#6QP zdjdcG^OS1tBdT!_E~rF~4kTx$fQd-}(LGeY+=s*5u!!#{kcjcdvedSzIo^!(LO+En|?*X8MT znQq=KaUtR+1CKJ_vT22paDH3?Iz=(sXuqP3M(K*2=)Auzu{N+d%wf_PX?pOsk&$s?O+-`zJy1<9Ef_z8^$|t|QzKMG{HIFTO7T zvC8lnpxsohWqAqYye%(%3fFn+b;`q3(0{85X#c2hc<|-C!wTPbu|Ul@@($ zGX8}bHgSbc?wA6$y25OGs)3bARK#MhLJBO?v$WNN-Pvwv9@N2xxerjxdL}NhD!%5FpLn6azBT9^N&xcG+2Ln z(IAutKp{E*PzQgLj`xs3czwVOWRObk@5I1&5>`t{bok3&UZ~aji^HoYS|l%bc4-Dh z_CyY%dse;U0!>yq$CV=O7*vY}8Kx7AFQ68Xl(e)rSL<`H_yuY2J@tansAL5so>I>G zaeXf-Qeyb5aTLrLp`~Gc6@CnC>uvb6k30!V@nviEG{oBO6zDBn*LA+n-& z+KG)63MSL^T`~!dfv{qv0nL*~7SjJhV&kdE=I`>U#;aM@x|q2jiKvGRdv8h7UWW{2q12$M#96g+83h z7E`L=c^lLua^XhIW7hV4|A9zh3^gcAa(yD}poEFa>w?hvcaU<4G2yAX(P16s7eeZ! zO4D;cH==ti-BAa^(NP<5KYV1!gT#OOO2@w}@gn+_XS_vVP@dhnW+n|bNpx^#Csa?$ zJ{B}}ccJHaLuvmGD1m8bk!zicL{)X*)&40|n{aiDG6?AsQZ%fkZ8*&NB1mmx(yG)( z`RYc^4O?z@YWjp+{d@+GHiN_; z^a!MVpMuL#(2BPIX}+>DQ*PKaAbEFvwby1$jneQ_A#&q;q#<)3f+}8kg@W zh_N3TjvupMZ>L>_FwTA3QA8ctD4)Jq7mDJsDSlh!$~&lIc39mn_*>HY;?~emybLxH zMZ+zfqbaUzmkzd7gLbGc&ho~9uipBM9qWMU0Gs{Y#hq>+2eZq64M6fkoE^ltC{TH7 zeWx-5JYw{dZP_hPT`H#ikJ?Gul+zk_zFbtCr)HIO&Ahe3%=MZ86P2=)T>EF* z*}jojKe4p_@|oPm)wGb;-XqH?DOrFHKwJ28dti3ZWBQ6GN=qv`R_eR*HR5!^1w-U} z(jT5Ju}2xKct3zjN0*t6wK^q2cy|Bw=lRw5&R(ZFP?_f!A zysf0@_^`7q=|_8TcSUec?&XJo>QCW2Zd+D6O>7M~v-|B;pMbwIT#Af)qO#nj2r|=H z_H~(tX#sG_PH$EwvhtOa^QESCu>+2Ri~4P z0~EX`6TD}1niI^2%n$>Xy!80ye`5JCmV zHk8-oH1b~x=hQB3-_R{!dxA@38X}HpoNDyHI@Ff9T#UUw)E@U=rkkIrcgR!}F*S@k z<1B9!l{R1^TL+*0dZyfbceB@aJfu)3%Cy+4iBA@Izm|ZY#tT~+lthz5I}4KB9!()u zjpJaTX` zTei?gmZ0;n(3Y+o{ZWCY3kY&COq4sSz!reP>~zgNg{@?^<_=5XG}IWc)fL9qsut|3 z{IZtiyge#eJui=*u+jr3;n{-&;v!sSsbgB}jl5hnadRR;Nz~36JPOD^U*%n5OV z^LoYaJ5B8AJ9Md_HR(FDnl6<$8`Lw0MWD} z>cOmowgvq3#jLE9l4vTNszXHnZ}Qj4o54_Sb99sC8h+u2+Pg>1L0>CXFH_G+OQT1J;i^?aBen z4Ow7Wo%e|YDJ%DO_Z40O+n;<%Ts!--q1}@ONc@$fBF^U#$Q0{Db?qrVp`a1-3};T` zvFJ&O%$zhHkNrkfw(J`lnPx7L!i__(cCgi`j<{IJ4$%wk)@*ujjp-W~lLPKh_4M;k z$1U=9mYxCUGVHf59(&FV+frYNKAT^|*(K&2=N?u*1d#eD{^Fj~z^vEU82#Ofs(N*c zLstCtm5{@v^XljwXXN+QPMiYZ5ZAW(j6ui>vG&tHJvoCpu@aXGnRhEIr+u5QUME}L zTJtV@TUU&igUaQ_rNzRhspc`_TfHw8lF_>xg{2A8xe4YeR79iGQ&STA0cV-cs5FU- zKIM6@ZM)6dR&`t#ngy!Nh~v|S8I0d42yIROdBXIq>fFC7x`hfL?onE?WHlI0|8f3g zKZJL_1(DafhBk*}`W?T0VXh>7@|?TsHvUCpXVOOsVv0fi`W^3LZ)+!CA5AywyQ6U~ zNvEuGW{|C(S&tX^eMSuHPW=uJylr0eHDi&TNd9bQM(W6kDg!O`W>(%re-8 zS8PU$H@fU5u0qghGtKt1YsPoiQ-sR94m-_~BaQTJehCZ&%XxH*#bcK!Lx1<7DU9C^ z%iZPg3L!zktb;_Q^;KxOwj(#onrq%)=^kd^WJTL|2fPV=`EPZPWjB_BsY+<@LXndn zuirk4L3-UCB_McxUIKlQsgUEYX~c=&?XW<@(@*+Urs)f}eUTiUS;c7x~d*39dy#pbinwC}%4pabF|Z#C4%-8(Q>??C85#Myiwm6zFN8LYu&d`6l2I`m8XA7tbj3-0RM*ln`~6%= z)5KT`ko4qqaKn*f(Bk+@6)+o z6;TGg`0^_}O1Bw&{uZ#5LpVvV;I~e!amcQjGCQJgxgTdqFI}VDi7Pv4inNNxtRn4EMQa;~@{$`8En}X!pnoQ3Y9&of47YvK zo`+}e$bVkRUU%Pe#pmh8=z~{BMA(~wIYrE;8|Amg4u7X55}Mos)%Lf69!7p|^14#HHeFx`D5pMXlIO6YHFCz5UP2Zw|W`o2!!7X-D(ZfJS!!-Sz}@^2wSGyN3Ox@= z2pwKPu-n2}PPI)_0qlVaKy59Nw(K=k=sYf$baXxfTlGrbmxpx#Szt9D>@`IcJw1_+&)=iO;}&Kz|lax3|{Dtbt_ou=GLWS^Nfvr5pGpb!qUVcu6Oe z#@TnfQB`&2%B)+M+3`b_{hO7^tbs^1{7JVRbAiCo)FQ_wF$|AM?C*6n%PJV7{h==- z^vkukOA3pwUG{YYJBiWL46BSb;8ifaLJ=t@|nx?Ah#o68xz7gx%Sf zqn4~@8ZVA?B%m(2y|73qIl;j`NHtsTWv7ILXue|y4r#qtx??yAd8!W}g(mzS%7)JDT4AVMoZ3e`js^Xz>nk{Hct0_ zc5~$o=OUEH5}olK_KGsxSf2@1Gh z=kcH43SthvE9IxsG2X)~eL5%4#g(M;d)x#cSO$K+d3I?x@j>&?rKP!g{%gM*u_1O5 z(c19MUHGDYIcC<)yV)~LQG8`a!`E~1=v9XBT_fqAr{jOftN63ye`zT2d<=lEX`o>J!cnW46mIS3( z0~gC5zRs?@U^x~jZRJ%QoBAK2FkDO$j^8?Ek{|${`?>lX%PY70;R;`d@05~(|KoxI z)zMjiISYkSlj=uf4a-btK&EFpGA|qh+}7 zr61g&5-(t`{}Gt8U@V2tEPKt5!>4k6idZo{o$(vaR!$xc(QPiN4Lb#e+U#Lw$QUQa zALrrFVS2vn8+mIJ&;}yBC9WvfLQWsUZWdDpI5Gn1~W~T2nK=!d^yz1$tf3P|(lI_*Z zX1;hW6G_>;{1-w(M1oDMKP%ZYil#C(@cvs@rL*&=i$r{FBS*X3puwm6P+i&gDAw;d z$Gv6Sa0a>q%j?S}SvjrCUj_ySbg8_hxp@Ch zKMI|dx04~jkurexNP>Ozbagn1(uUnsCylzWk9VWcXn+VCw7Zqs*nlKL4twN#1|CPL z!Y+a=tkhw4rlTr`PHR<1q=q9QXO`%UvCCt4D`WYleNkAm?PBg^>Z&8D5znV$zisq! z62^YMwWVZv=l=oNKqtR5;&VFi4vm@_(^s!s5+1mFMeW+*Wa!b&eXFKr=e7)Y^;P0o zm4`O=^^H{a42Jjh$J(002SlIy_Se6A%IR;NGNt+UdmmXkBUN1!4dv3&!L4EZ+(*1U zGtiT3S`^#$aB9klugl$X)m>L?n$%KNeO!G#-lH+rwfITk`ua);s3Y1qw72NG^jS?b zfbJKta?wOZ*1r!B3RO&-8a?a0kyu6Mr$52(NV%2~R?$czJhFC8`ryMOjZNSg-L-Rg z&9l{+R4x`rODcCHoE!X{cJ7eF<7d3Rdilyw1nOl{p9nKC6>RfVq-DNHe9kSgUOnDmy1Rp3URfI(l&wPP6`ozpR$ zPLexL5Q-sVG-CxOW5!0Z8CF}>b1CZ$&A7gUtyE@PE@i!y)eGZMYnQ39bg>otvJnPp zn4j4cZ#BjB)(i6o$Z|&EFuG$3InRObm|uhqPMPtWMx{C}wHmf!jL6>kCXY#B^K50N z&p8_Jj(My{X4X>PTU#yVta9XRTU;*gmu)!)^;O(EGjnXLXPJFjnHjW{y|YH;(wP~r zGP5a%W+*(2wKt=JdvvuDr7pm)-wNv*9dPe1+4qD70a`~mp< zh&PPqubZ2i-gf%w8#iry?9s<+>uT%k>o=_5fVa!xX%0L?0TSRb3_jw>BXCmh-Lto= zYuCm9aPdPAJ_IH11?oc|`+E27*}bP{U(b{&lZS?emn>QI&_j>>&ELFp{rdHfJo0Ef z76Wcl`=qH;@N~%3STqK!WGeYTKl~rRQd}c-ial0mVCcST)hf;y%WD3Pcf3QVGeCmT z;0HhWLB%jqWt0K~8C5&uj5A=4A2GyQ5A_axcXu~pR@#FOI!H4BJtj@?6N*ZLQV42- zTcnV&m}0|*4f+fSN|BbYfBoxNi-FmP#fuj!8}=!3r9+s;Y(%T&=mX_Qtmd9oIjxCgqH$$lpR{iTs3f;#x?ow%@6{m`ucxXFvJK*xnb^S!vhN$ zbbc2iy@`+U1CnU+U%XOSI8yK+I@#ktxi{pk+VGSTxVV*i0p!5U@2ctU#wk*8yorG` z_C{Qgy+9-vP6=X_#ZEjqH(o8g1LypDM@NHUCnRfZ8mFpNZl`+z=Cls_CVwE(T{%Y#v5-!bA#tcu-NRp^WKds;s5^c|Av+O^O3Q-%EIH$ z=FZN}|N5{0`saWCXA~7U0_K%%Jv}{WrZG>9o(2!y&6zW&wpQNi#~Y@5(cVFWP|@ac zk;-@o&~HG|Eupown3tv5zkmP0z<_UJD~k-Eot->+vXA+b3pkV)TyO!p6GZallTUv7 z)1SWj>Z||cQI!LDtz|%e+Z9pqYeZjekS+C9&2_TebmvIwryIJT8_HG;X7)bU^}Ypd zOPZ@O*r|?1kDb;~In?+0JGaD6dOaR(NN(5?PgGQO%v!zv@aX99VJ&N(UmI!di_L7# zym9HkkM2z#JiB7cuGF-~^sx(4H3|H9^uUt7J0HI6&O1}pOP5p}x^$!(L&11+Hr@~( z?TtmMa?{?F>Uk_x(H!l4Zn$>-xo_P5&0k(Ul&m`L^~dSh>t$6+dl}d7qnDN~>|WdH zdM$6nhNeGSU6ZS?OGU!+(4BN-Zo`zv-tG{7ht>7;$xRzm_udn8t&nv1nBTz@BkAJAn?PUh_GT`{#bVc*;roRmPKP42$&pUly zWfbY*p?zC64sYBr+P8Ofs4tfsk@GDUjmIl0E9>HuTdO8bt7&SD)z(C!v0N+>i6&$= zO~$FOD1cmUXwS~xjq8Uyw~hAi&x{V_vUprS8jDv{)YMnBwboASsA+BuC*nAz!}02H zERGRxk;>xC8{XUX%y+*Mk%zeCPtQ*X%yQvq&CJ%F)czsJ{V`F9OYpCb- zqb&=s*$m)8X$+omS&oM?w57uX5j{QKIM=Z57$ZzPc7dY1@4g!^Yrusqe(bt^QagTI z5Kl~0R8|ZP4q;@4XRcw32?)$|WOI8kQN3o(OE=$qGsdOD>HeOkPM*>+yW@4QJ7N0t z={vi2VnzV?ox65cRaIkLipxCw$ze2$?b+v_OD9tpMpji-mXf!B|M!2#oyg)yJXe14 z#TTbi_=&ENPGwKJ}?jVIU3UrcIkLv4R;H-%5Qc9v%yP>60vesc+>Ac+%w7TW|Hh zLIC)>>#qBq(-~|sv5awmwm+_U1}(zF<{CS2rldn^p?f zmM&;$pqIe>DoA=WSZkM$T)ex(G=|Ab;#*|r7*^syT#xxxZYh^d@Qz%^2qw#DS ze-&E;TtlRO|vx%8}A^UgizEc|*vBq}%e^tiBJ_QiUDWr%|#63VV!yXK=G{YY3$|1>3gA+fwUle8GHV#)yDJbBuS(@-FDw@r;wzUjPjMt$sm$Js^Vr&{t<|#^ zi($VUE$fR%VmVpEGILrzM{ebU81yS<9nb*O;+i8DTD!LUk{3JAJ#6N23#V2jVimba zLo61qtxU(`{kPw9PDd;}ntgoz?kN6nb{?7P>^^l>7p*n#fponWT3y$&yPip&k;LMYYz7y8&}%m=&7*rc zg#${%92O&N8003=UdoqDHrm%4+qDaeHe_jrgGNHJBxWn#_O^e?&atq-@<0_}V>oPFUrjDkZtodIiU%JGs7S&bpDG;ufN5yjZOiF^$? z4=Q^HPC8mx$T$P>LxSwua&#-+Tf`K}464a<7k+Z7TCqo#3oT#i-|xK-$&amS^c-h# zU+Uc-cCewaQIDob%f^@jb2Y)9^}y#c>15yjy)QjGuz4*W25elgsCIfsyrC^DZ={ip zIa$MUo)Z%Y$$`%8yB@iB^G(0#dFE049pH9II+Mn2!kFW8Z}$$x+%MN;M$?f*Rb9vI zmPIQjFJI9xcYeIKDO%SYuWt=u0g>7AUd9Tt89eedyl?%QZ8u!k`Oxj7d$;2!c;rlW ze>%GUNbb_f8HE-hT2a@~vS{h#m50{PTU^=N9qhRM(e$1?V4FLXW|y_XflZb5>`~iV$m5hW?m%iSKzPzUL&8u2M(FhM8)cacUa-~L z-u{l$U!P}!TADTd6JK)eCTlvz{7Ta)|ByYLFn6tf?3!QRA-ZfT5sQBQGZ(eBV&SJ- zGwK2pTDXPo;AQY2BCNY!=0o}uV$& z?H?Rjvvv#CV|WdxiyE~+lO^jobD8?u`l?v;z=MyPGkfN$l`9Tfwc@D5kC-`Qn*0H{ zb`4p;k%ftjN6iK`ik^Mu88pXz`|<3z93kCQ8X6qL0rTfFlmGyb!Ueu!kJ$%2lGoAE zA)a2*20}cvg0%yfLbfD$?*pcFQ5~Mn(AvS!I>dxJnqzI+FcpFcT7NFXf=k!HqsGQg zH5EPy7x~IRp@6lE=tXdJGzNV~)Sob(vg^OF3>2LITDzu1_{=#?-*~oXaL2xZbzLuw z41ce)yZ40Qw;eujps(+lbz9R@X5#G{ZOOhfSLWL4lRvyYx2eBAdcdl#Nb=70zkbsR zwLiXj=S7FdLYbk{55mn{>6p+?g)?~1CVu||@1=+(t0v`UAA~Vqv~Pd7rZ*fsr&E0tFd|7+_`9=N>0Btm>8`{D+o!=beV>>JfhLY z2c1TSe1;!&{i53&%h=vwCS|W{VDj69XiltIHOvpGZVZ?3cLyrzkpM%$n ziZUOY-^i!{V6>qj)G;^JKCPl_2ikH0Vz3m8#0LkW$q{_hO+v*`R(eudQ7V0_M4so( z!|bQT>qjN2*3L!hF8wk#@Hl|AeGO_R=n`IwQRcAf`h}~$zwuXBWt07tO^w-+k!0Uc zX|zymDovu{An(MI_pYX|y6vv&qr@ zTQ_gN`{vCz{w%p;O(LGHsgAX_L@E=wubjhQEFJ*Uqkuj$jo$%E#geJWaH_ZWnbo@< zxII@{*Enb1oa0~Ly5f*nT}#E3*^%0o0(Ys=;eoDgiMpoaKKISani{3Uprfy=bJJ}% z4D25aWkwt4E)L_Z=5-D6nmVkIDqtvoEoY#31~5d#0BOPe`TKf%alQ`?4Pjyet1ZxW zW5|R}TgAf;KWuPt02g6H!y~7ke)`&VYuCKAX6v@CxD){rm#KKU{`J4PUVcPFT1+(S znKf(H;1$#dA9V2Y<;&~p>o#rN1kIK$TPpDWvf4WL3`!7_-$D0=(b->%acz+Zp;_3xus zb^3<7Gy`8TiGynTdKNSxtSKV<{TE{}H-u=HE?tU+!_ebC=qab1!nZC0m8-&_EkBI$M+9!5Afpv|oECQa-I+^x;?}W^ z8Rk9ZUAXuqH6ECpt1V1#e>U^w>#|S2lo`&*#a$+Z$qIS5MI?u>X(S>PO5iI}8A?aK zabe;EC&(B|IuP|Mp?InH;!*E9XmKzF2o)0GN;MGQ5IPA}c%%gG@KrP}VV;fBmarFE zjm-rnm(cF33(uV?st1)g!bjv+k7d62zj)FZuZmAbBiZ8)z$#`f8auB_@LUqa`(+dS zsrztL6>|QNh^!kYRPmA&FWVl**WeIH$8CweV_jH=jL?GI{D-`z3zk)+VK*+ z*4DVYHRCGe#p8Q{BQzywXEDEvR;+X9E=(xMJffS&(&G);-(aXQ`+Ep5`P$dM_MPv1 zN0mNCz1Th17tH8IFqv}WjW=@Mu<8%bm|$@MV#6vjp9Fa2hoE?kO#zS*JlZlWY(p^k ziWMAq9uEQly7wdZIel9Qeii&u-%5+{71QciZOBNK5f8%l@kQ7a!^of**bC>!4-bO! z==q~Ev!ONrr+@k<+)+5{sH6V%U;h7T0sMDhEd88nNQ_RR*pv@c7fh zZK}bo`}#ZgRW{c&ESOxCh^%O<|L_6fFFw5yvkdsbj~)BdSKPj4U}$7%TgAiAZm3;) zNHm+BF{L#=usS!mb=BmRL%ETGY-;l2L${~1Pi_0{(Z@9Y_}=bwRz)kZat`dF&_FW! z%+~0m+j4bH4Tm1JV$sx4s%vAkWkxnt6&X!oRl6)rOZJBPM#2qKqSf{Bo~N>-d!v!e z&fUpb)uZjPZY(`(Zm!3lc+m0Hy2F?BwbH7~Z4v3!wdq7(Da`{~-7H!ew7-C%4GLv4 z19#k%z30wYqEdzhf{>I%GMVZFmnRw<@jbJ@7yn5&k;d=4Iue*>veic%hU-rFfm5-| z|1IlR$t@zmtTC47O`Q^3vLv@{3kCyNkta*>;<)LP&LxM#>6F|~SI-f7CRJ{H$OWa+ zf=o8$S24#gRftyUuQjpA`c-^x;h?y~L%M3y1Kiew5TJdpY zPB|#DjpdUW7^!+bpYx^ggG7!obl!W@lGD#_JMf@b zbrsI5zO7r<-EiGKpZvg4pZjLpVXu>g5@uSVAS%pE4DacB=<^@hxAw`}`0$kaTvZ!h zRTIN~ER7kb0!{?CMhKDPAmWU|PwRxLs&v94vL_xMuBoeSo0LnB!VHsZa@UJ1Ofu>+ zux;~}TYkR&+8?F*cTH-F)J=~j;`rqv-vXyUM&Oz{aTmwDTP##vm8;LL2#*+Xe?BkC=9^??%RwGwi&V$be0gv=P_~3)M{#RS9O)Vx1Hg4RA!6%k%G&MG1 zDjg4-;F=k$FkqNU<5BM%?hhe{t!=Hig_cQYcJ1myvx!kFR$Afk;ZchRA9@hUg-4lr zk429^{@m`K?&qI>9*uurf8YGYbJusiR8<*`R>y~Sr!ilTQ~8M}9)J4jry84^&p-cs zzec_QDr~4YRyyco6jxq(B`)1DFQa+zku2!*CVGG{Ak^s$@G!=o&cIS=tQpZF#rS); zQ12*34A^`I#i(*Zy^q9k#1Q?*KmIXb$Ssnrt4owYVank1Kv3C_1TfztWE`ejZn;G@ zs4Pyr0}nh9i?U&b`u^iT{v$9!=bKvT`J=MN!2IIg-9Sds_8q%-b#{i(?czQ}w9@r6 zP|HFL_nYl_L=^YnRh__EIHXEuIwH8t%&k~1KVUCp+P{oH1nd#VMp{zYFh0oj4-9Nt zw+U@wD4f8LS}s|%u(q}uXAwX+211ypl`zCszB^HNI-b$5(CUE)Ev#h^q>O8L<%57Y z9<@~sWRS?OQ8=gA`}L4W+$KY(Q20|5*uKq*w0c;GJG@gV7VZB6vV@BB_k) zF7OCqO?crHJP;`nfl$NuJ-UR5Km098gA4Rmq(^ z$_xfch(p4$tBX`%!YMZKl@ANDw>gblHX?z81CkK{J)NMlkz5|3c4HmfZ& zvsG`6d$B2}#?S3o-oeJoPE|P-iJRt`rF@Jf7R7jf8$%1;8E@N*0!_$=ACDq_0E|y zrsG)2DeH#FLhA&i5`X`vopu^#CHngZ@qo$qzW2SSo_;!p#@jul?UJVFzZ^@o(!FL4 z%Afq?CwRM$eZxg5Y5kA?_zx_;P$Z^Qz;oVt=kf1jqgTMJgdXzWNr-3ju!c>+SaRk+ z>GwMgO9y=DLm&FgXFkL6sH~@B={$cNBwp<5KcI-C_Q!Er_VOppz?Z)J19-I58oKyF)R!l-vghNNQH(Ye&%pc#jwyO_!FhYC!N3OYV zLsLb#H(Zsdti%fV&0YJGvBqd}TU%p1(HHI?8l66^CAMVM^>=q)l)0m(qwW3%$>5T7SmTle2ragqx;iLe=5S5PJDYyd{#}7uj`R zM4sCZ;~D+Co{BAKu9`D5-8b5`W%G{a=7v%dqCKUDM5l`yN|z_CU$(cq=UZP3#}c_j zB!}ht$#k}BdwlPnSaLLkTQbtZqK}pf$aExBd*Jd&1AcWilpPw$;Qckaq)xj-0cPTD z?UC_| z9M~{(#+{$}#JXSow4!Bl^8ts46U82(PYn(1e(ZqIIlSOc=-!POhCxpd!C_cu32T4u?8AY8i28Tz|6 z<-?y*6!USRY*j-`q6Tw|axXYiIl;n;TK)KUR))2YA-tp@$w)6>iC3 zMq~Eu*<*-TBw^Td-+lLi76V8SXv`Qpq89Ib=R0wO=jF2E2Pv?E0LvD9LvZ535Ij~~ zFrDE8UYUR?lIoi3L^N7aQHjZ0ToPlwhFg}g1y@;^AjhctWys#4&CwLzjehL&{4?1WS2HP^W&gPCh;>anJr&LrVo_ONP7hil4uZjQw5rS>? zgLl_tHf20~5H7=S^_WvFyEd8BI z1dr(qh56GNztgG+;J3g1Em8ph{!C6i^;8JBsKImwghwBJ6gTtm_)ih@Qi{+EYeTf7 zYj4lKQTJLDU5f#0+*$~`7lg~qF&_8C`v~g>fdkioWN)0YsItPW3sQ0vmSPGT5jXye$M#^RLV9pzkohp6u#=7%%!(whcGY_2QbE80rp4v|c{WHWf`q_swtEqL&}fB#dA;Ll5v7mfGfgYtrgfm-&UN>(# zLmM403sg_l=qbV{isD@)USy16E0&o-Yhg92@9f%%S16;mmRU0yVB+azl;DVB>ImmS z`n~Tyr=_J?p5>NzEN8I_15I#EWyO2nb?$lRoV|O`?saR|J^aWccieINx4-?LXwk5S z?Z_jKIOz>5n6nGCs5=%ufw~;@_ z5U^?WAs{AD;hhoq&1(J1_0_uYUJX@)2l?=r31k>z9U5M1ghd4U)i*&OX1g&BKppvb z{Bj50zJagMcpM&NkVuWzC--xfl_HeoG5s2hV7IUzsRp6FvQ=ogV;*y1aG{d>ob z9d&heSajFX(E%m<;s{X$M_8D*Y11Yg5!|@IE1H%pSpq8PeJdKAgop&kvSewjsuo3l4DqVEgD?8fBj5bu z;N~^;a~H)E@_I}5;>)SQ{++9Ddg_~>Y^v^WZ^4K_50dT$<#Ud&4)i!D=QFJeVaBde#@y*_h7mfWUx+Sus<_0fZEnfnvQ2) zuy6^XU@ilHX!b~H9IUtXvZdpqcWF`$AK$S+r4DkwSu5*Htr$O{l{NHXzTkUQFq<$& zJ^H06(o$%!`CkFUSAXwge2wz3UN!z6MO-g*0T|@qhb@Y%SLJ;A%ow*|z=YP{gHA<=;54r3U7V86{ju($|Y zHitKK;wMNd5*1ht(bC!i9md$WTE@+D%yi%!2OjbN@BjWUo?EG_t8H#?S#sp81q)+4%866ya{)CqMZ~dD?|GMfwUnZ=%x~nEt?PCSVJIU+OQ-z`zf; zHvM!dM!&)-HnvhdYkWcoD8*(=p~1+|*CXwK0S|Ju@<~*$NS1v71TFQSG{LXZHa9l| zj3?7E-GfrdSfGK|jw(Ofs%-r#r|#9S?iHUL9-TIIa&t2lu;BUDG!8~MT8$+J+jn;N z4Gd$!K{z#9R~?=+YZ_MFV6HeLuh1I7ByKb|d)d+jpc-RFyZuIMRy^i zkvv`g(^=(LVaQ`I`BO&DK{pKoFFrW$Q9k$e=YDZtx_4M!gyF7U@r*#MDu<^_Fm0Z} z1BocW`!X<}o*czf#L1C#c=5D&OD&=mKL6ZQPIaNqJuiT1jq>LJ9bbeb6BnMXq(y6n z_+?ZlJSybf;ItNhxd5Fijk7Aj-A7ZM{CkMnA_EdhcIQJ z7|lkO%)%l_dF0cph#$ka^WW1d2F!NjB9%2uuXc6D$eKr$$hv$rnjJfK;$%n1fON{i zkt3wZ5=~_yiU-eJmN7b!lfX?b$tN;_%uH%)o!ma@xZ{rb_&b`@i(mXA9&Uty>GKbN_`{fO!SWN#Qu#di{nV?jx(bs86S~A3w(;xY$PA`MxUvBI z43P2K5wzVt!yDf42E5qG|EdD;m^nhTUZD5Ca;R8=g0Qg_go>;px4ub%fR|liMGU%g zMC%vLnKS3hU;c6_Q~th5S$H)Wa4qVqA$WPV=pt|}J^7@CttZSyYaWj4#gaExe(Mo) z(Y#)L*Sd9k2V;mVlNs67n@*+b7EWzH@z}52eDfjAZCeIXGY+h5X{_lSPEVA>x*v&u6 z%n#sbt=%9fGdvVcr?p63h!?l{U!8y%iezYH1Tz|V@2Gf?|IyG#qqr$8b0UgDXt*t; z(YjvOEAU$N{I>}{rRmRaZn{NWV4(+e(k-}R?7T$ek+029IY6(?$nsH*4iEOMeSY>4 zN8*=#-N0)suQSa;60GaS+Oz9*gB5`vF8*aB?xVvTZ7Tn~+S4>jl zU6Qz}$M354ba$hf$HODDI%Y|$o=I=tzM~tzSA{1I@Hn&d1>xqFX1t#4)Hk2HdCTTq zU0pE8Z>+97YT=?Ir#G}!c2%VDLXEu(X0EKQ3s=;{7OtEg&LvhZIIcD` zW5b#a(Ux5!ecJ~2j`Zy5LHph~wdUCkh2uBBQho=4S3+Qo1Co!h@%w`q1Tw~#Rq&rT z!B^iE0|94+6+uruxq*a$J$&xHr*^;}l4H#pBjWL*J zWB9<-KP8Za;*f+S5K2BU)!2Z8F}5+d;BH(cTUKwo+TQoQ-M9OC`Typgw^yTmS|PBL zkbI0*_su{#;E70Jf~VdVG{yNYRejNbReh@YnhLrA^1%xNPHK4x z`vyiQCMI(((~!+yv2>}j@-Wwp6fKx}ay4PkZ>QB69>JN-2M7psG-~{2QWJsL?2q!K z!WK*kRxT=HVnc2L_3Vi69?b0-Bn!J5k7SMdO4~QzWjndug#w#Yw}WwF4!3BDtPw|z zmsJbBez$>7P_ti}EzTUKv$X^=qAT&uiz>xzl1XtCxhh~DTYgR1kz8d(iOpIt$+(!v z>c=dZVymJi4i~e2tc*<@1vBYH7?Te~s$R_J%Qc+HjYrUBU_gvt?)1vGBj*9}QXD0# zNiOVWqkr^o{$vxqMkJ0XinS@P9+w;hqUZLfL3IAYO&~ZNqAO?1iCJK}d^I#Nw;rj{ zb6sP7agjg&K;?Z@4znPFo`~zyWgrd=V3v$I7uCpphyF;B!mZy?ZxDRTtKP$WV9% zlia`k?Qctt96*i!SmGA4aI-S9;1UtIlEW~<;V|U1Yb0TZg0Z5$^96)Cv^wWGN!G1U z2_`@yR@}mo$Q(>To`b{bv#7n<;5R3gE+6k67+BKy&LKgObn2<6ve{LKA5#Xa`!6i1 z&qS8YMoU4!{B%&g)h&o4-?|kPwC_&KulcNv{0zJxG-}+XpXFM{$a>S)Yh zF|i_m^|;PQNOW!jneVD&DxNL}=C{J1PD7^m(kxwgN;^&Y&mTLuXLy?S%gEBYC&!-+ z2byYp8#Zmj`-9&%?e{K9XRE!=1CjJemEM;*xb6)f{D=Gh>Dn_Zw!biv%cU0h245O- z|H856)YUiTGR8f3KD(tn>B24^f1lBbc*b>L-23u?J)VuNuClFNYV00y-@c>dE$gCn zel#6fpAkuS|0d!{1x~vW+xj`pfz*j!BIKvl&CloQ&S>zmcGStzq^eZ2i<7FR^fZ^0 zUGxCGbpX6LLp=ubzl z`Sj#uy#KJcwEyOJ3P>3TCou@$ZV-_=EJ{+A&&+CC7 zejC{l+!DJmU!k=SxYctJxSlF;9Y=j0sTgVm>e;NMd<`p1I$w_d!Mi5!(d4LCh+jtq0g z6B#4u4^$}|5S!Cub9?E^kwG}-WlpDG`PNtaAHAc(9VsnwVBpM?2f>oc%rQL^!*nSV zk7v{AR5qm;sO?@ZGNPb%ugC85WIZk;nMSALbQK330UybnY$7XKlA~PNjFX%gb*Ak_ zH8q&{TUD0~g%hDrQBA{4Uw8S^Rha)XPrx*T%`ymMp}}OnxbW9g1!9#&Zmnw}rVbIR zLWo6fMFeNQTy%ILCaHu%xSp?-5fPBB+xc(=!DO80tYZp?!W=EOg%O>cRN=thmc@$~ zb2=%ILT+aD&5TZ*fXabSA;Oh_zr?$3+K{Ed=TU7jIFn}>AwE;ud`Yv<14GMdf6EjpLyn)a-;~O zP?Sw_^eJpnD6nfr__ORJv684jEQP5zAQ)(aMnuQd<8zV< zZ{~CPy}%`RJwgOIEARthCrB!#pdL8pUoD=nSx&`IGKs^ zL>OKvh*>bx2F)xc5M)+}&>RV-#Ae|XH!}=}IAM>?75F?$+dP>SO-zDFQpJ#$gFzjL zb4w~1vtOB{9?8d)(7M%AU#)e9#q_m}9O^!-x^lDD%+)xv?5Yw}$s6kG_*B$er>gl6 z2dP{Lv7~s@o35y?sd>|zuB5RJheI@l-}uHiP+PROwVZa^X}C2;wP`&JVGUtxYb&~Q zUUch8lO$+JQfLnF2W6remDw-MZ@&3v`Go!;RUrc8*IjoV2Ud|p&x9@(V>W<=nE#Cs zP6(IWDJHx7=v^Lt^igU6IZ-fS8DVWE6g1%PeCInYUB)GsT!O$Vy1|w;2%>|;CQQk|NGxB3|WmqwZz-PZcNPl zhKGkK3#a+g`~LM`|Mh&C{ssQuSOc%gr;=}m=zSxxkw*`vCgb&2ta5r;%e7y!aRFcY z)I)pwr!z(_ZHuRKlcAZH_QV~Ar!MGR(2#0x*l#PVFLt&UJ9mXrSF{xU$`N~I+0~C6 zzGG==xO3dmRBoJ8*|Wc|>&~aLcBf}!Q+h!y8>3vk6UL5#+~F~!veb4;E9a*=AMbU4 zd3R->r_EpAJ~}#bZS`}Dd;<=AStKTH2OgNoBm-xD%sAFjl!f(g_Bnu6$rxHlr)ZwO zW{y7iJk$^f;eieS06+jqL_t(V`G^BS^^sU!1Omy6FE6|5YH7`+tFxCBIf{#tnXEc# zmeyN!s~B5463;#ETD8JaSqYD0tcjZfsYkrI2 zYQw_u-Mj18thd?JT8$NWxKqPpxp=}`T;f3ado0roYJT*Er^mNGjGJ#6KU?H1c!s8I z!=bFRsH}POrmCeU`Rkiq<<(d;phK`b+}Kxg0P5K8t)ttYAKI}kH50CG2o}{g*_~d) zU(9t*j>*W>@UxHY{^3_!>+MAW(PgOD!Fs3I?ld%Ro1Dzr-2UR0mZl{ugY6xz$_9t8 zC>e`#aBTR{-tkwq4t96Ce0H}Jy++pX`rXAvs3GhwkHhOb!fUn|sfn@ZV85@n>?b zl^;+Qm?|Wqk>pgQsJ?l6u%~L>X&xt^Lk=5hhwmsYx#0ihI;RHYgPj9>O>XDo%pwzu z8LW<2t|gD39XoK@xFI`UrD2(m^YQtdVCvA28ksQef*CICZtmS9;!g=RW)FbM$3=M+~0J%7P7Tl?#_Ot~+N{RdujusB7!<+h=An zPrvY}&*ht(jCg#WhV}}dQCwbB?q^rUiBj-g23 zEuXnJ5zUlWm$WXezVwPq=PLy=6C@Qc@2E4l#TV)KzW2SGZn|kcD&UODM4iF19|qUf z%-=H2V1vvtk4-%e&sU!$6aH7j$J%Lf^2sOTBg3j3EHy-(K^82h-FV}Te@ZU1Redxq zQ2JFB$c|RG>u~?r#8l4Z=d2UE-`TN1sqjeU3W7u=&aF8OR(#D477i)}piFNSQ!|)D z1@gg$F13620frcAb6|XC#p1OA&NngnE28|nP}q)wqHembs2T^FvTQ%O`A8gG1voKJp3V|WREG-nml6mWa0V9@Re+h#VyU%amyo{9~wX|d6 zs!Ge}B~qIT7HU8-Gqo-75dpx2Mgj96kxJbhk=Bf=+!X^QkK(WXDJEPLMEy#(QgBn! z**X+ff?~zZh|0(;w<6DTo_(@{Z)Q%mD!U{5a`Jhh$;ovNXCtwslifDyTur5`y*i(@ zDH6Pn&d|Jz?R!C(A0zD#fs{$4MO+vcaviQibmmf9Iz*4pm#V`h7oo!b4Y8CdvwCOD zD&+~yhKllvP$*m;4CWuRS;dg^>2f8&L^w-ahSy@>#pL#V@4G=%38aEXQ;z;}puhKt zC!YB7m%ofL80RN3D_*y59ZT`(2*4qZQ%T_o^Za;SWu@}Mlt3aP{G8k^Dho~o<%orh z0!8A2IfoTVIs_aELc)#}_QFUUKcvXSJ13ogKi7CiC#!8S-REpj{|AEK%wIF1+wU@gjzQ6FN=^g_c}M#kaO($KFNCVx5fp+0T9! zeLt?kV9qWZ6g09{BY9a8$}Lxc$gS|#F$?25rp`@|7vks;iOW^bU56h(Ri~Oy=0a{d zq>#$0ThR1Wq0&=zh|aSxuFI*Xns7c4_44U?34UQ-I-fbIv`?9IHrgE?_~ujC*!h=K zl`XGx`BA^yU4vtuyx36cJHxo-bo;mE_+LtSGCs)-Q?bXg5 zgU;Iy)K8YL+TpIvEOI%AdMm~bKG_wWY+w401+I0)y>?sDNX3lq$EV$;#izZ`z)#m~ zE$Ahhg{f{)Rf?(-Q?+ltoD`77171_rCbL9GlJfhhx+KMoDD8E16>$OPx#1T8+n3w3(-VzIXRQ$FSvk4Arqb~G z$y2SYLt!>kS*2l7Bc!YvDG8chQVSwLu4b@mMaizTEHrv#$le;0EjS8}3{oLS zuj)vuqMXH%xDMCjl4^0YsPtS4ade2po_U>6{Cx3QMHpdn?Kt&OGOt z&%D3&yo*X3{ifds(p8nFlkvTG-d57s)QMwFtC~yk_4P&J+oE?ce&vPt;f2ivw!&FS(e{U&+EB+eUgva;~!Smh(5%uRp!GZIP#} z!C78qZXsb4xtS%~vC`;JdWWBVqWAF!BK?PaB}IX{7I(1P?kzEVc{iwf9mZxNvCw!) zLnCjm1*ev>k?Cl7taQ<;*zVq3GT|)3`-aK{SGZQ?0I`Ba3q%lfG)WbRj%kt4i4=X8j)^`%iJ7!Qi1#+Cq7-8epIz{8mJG+tS_kSpb+{#`9ubdYs4W=c zQn}FXcx4C56!K-1LMn?lSV^ESV;PBdkbe(8_|Tz)2R#8#LtDj$^G>?z+Se!Isfmfm z#CT}?)*T1;_vKvio|h+c*?^wd|zD>(5^}9h=_uV&^?~K5)ZTpFrBedee)*(JG7pS+4@f zsE^}Lh){679t0l<*3;O}$`k+<1{Sy^0k_l{jQO!WvG66rRccarTAZzjo+>1I&0AB1!>q4Q){rh_m0`t0>+F)Zt?aWN#P-kCvcmLE>G;N;S;`6%}b+j&7 z+=A0ZC9oOHnzEC_;FLLJVBnBSR#j%3Dt+wvWzb|cW3xYy$jhY^9cF!+1(=x5?(S#V zDCc(O+G~tpz}^?ib&Y37Lq=#S6Hnv<{@j8F`|4&#soH!q`?i}^Mx8TeVX3EoJbQ2~ z6Q0T?Qx2**ox+wi=kXh@)wU&d&a!|qkTLT&H6?0H2R+F4!^7#`un~#n5=mUWqpr{5 zz1rt9+Nz924MxBXA+@BWRe>jJ4uQs|jXeXIfiYtyhCDW}%jWmz>VrAFf$SejyI_MU zV%At#ZSUs!b~b z4o8JCWxe&*TfxCGB(@LNUw=JK130XOSXr34sn-w^?DUBIQ9)N{w6?bXQM4edy0F=> zR*=@5IMaSaCGwvd@GwfvSd}1RkRuTsGQIb`@12vog@7*O$tRz*=JRy?9JM7)FdZjb zG|?hiw0NM9INhr57Xu)c8k;w7#w?RK?piI9*DxZYTDx8mm7*akWhoRDmmJb(@T%Zk z!_s9S4_;bO)8M<5)eOcryi|WN(0C0W(`(4e>`h)ZKU#8!-Bs*yx0lB{C;XKq*?2bV z%+4gD<6WU>=a|E5)Fh0cqrA@u^%P@rZ*#-|w?Ih06J=>z{Y=r!lRabOGw!Q471fp^ z%wUYeL$Xzy&U)-OT@Bvh%a+azMI3uaji0}ei6^supR>Kn{ib!UeFM(B5BawxT0(*5 zmbS(vO_jFYc+8`jv)Z#G^EHT^%A!*qCRd55rpaorH1_jG*~g+*I)sxL2$-B;BCO0pQQSQL zc^UmB;aD}`Dk`qscux6>4X^y*+sof_UHPJA7`;=O(a=QaZ}01T@ZOCd{p->N%TWp( zi`o|M+Ls*aV#$kf8$n)A_3FiQahtb#<;m@v&vgb$ymgCRHBETL;5o_jn#0ZM^EYfd zPjOHq0Z_0JI_isv^mHBm<&TWySXBdioAcD_9aMeu=VxwwAKhCtgwZ*8l$ zy20?3qFn%W7F|(cKvdM+R@~CQ@Xgmo4tJ*#)BcKbmsfcJb>KTgp&S(wHl|bZ`IUaCG5GO%3&~>5=Hp?FagU;}uoGO|M%Y4v+14 zdH?Tz``t(+RNGW}$u(zH))n{dnkvoK)t8r)oYagk^OX%r2v#Z{^ke{xB`?OXbNsbi zXhax{vEKw_$az3yK|QmI$zBXWY})_e*SBpQnyI@rs-;*icPTG+xVszj|a zs<_!A{`jX4{rxT9Ie55RolxmO%S^*XUmmw**rN@^URP0d!+YNRp_|?k2>7G}CjW_v zP-l0)O|^1Nqa7XSDbVfCp5#C-k3I3i`#$hDom~T1Yhvyf@HoEu#T%C_X*0#ieD`~I zfBjo`boPy=FzZz7;;e;me1v+xG5qwpjFVQc`o=$hZ1t)R@TqMW{`!HZ-}Ro)rW^r| zj4ChoFKDkH92^}S8Ri%@^K4odm>jEyDa!iQi~i~FKfdvlRn#^xv&gxbm@Pb|v@8_L zo{1NETt%4unf{v;p_w!EAoJy^no(;+rbjTdw*T$7>A$;U#)d6l&K@kY75TDb6V6zQ z`e0mwQbhUnvf160McJD#bKH2I)2mk7Rn}1BiQ~>^GXHY#%zfAqH>nty#I+PS>o8oEPKm023!-t|nVMkhdeWELbA31UkOIppkbJ@lU z`~P=U$@S+@e4e#xANUy#8=v?|?B1e{H4>$Z+-LM%S($NdhnsIeCdlH{m=+aRmZ|bmtJ-W%G#>R3S=XQB-(qX z*rSg?ZF$G-KcOMG_r7~ahDX%c*sNi5q@tR3&_sy~E$V7KC(|g{ zs+UnD5r|bMmg=1|J}h%YoyuZG#G)dubK_P7rfJ3&05rY+g@X>!`OKFpaQzCE73cUIN!47H6}2%6Xg0R($me z{QL|*x_9vB+Z-j{@g;%5t}$P6Nvxw}Cg6#CJY}a|nhi~cUg=B4a{b|v(Y}%9sHeU& z*>~5|xwoEoQk7>(D%%}b1=v@*s4KB=c_!Rgnr$dmy>U8gn~EC`c6+|{Xv1JpTc*4Y zBL%;MF3Z^2JK@bnE?+S7CeKU8uC5vPxw-7ZY)Nxw@giGIXKw$4^h_9#B)0v$co)|ba$q4Lbkj-^R{=H6Siy`KaO|&z%kH|5<%gay&oPioVL`& zxBsQ`10Ti65}r~yo~4Vj=bo22uvhizrm+KcPSh0+C4YMRWOy=g@$2lx=y5<-&p~Og zaj-Eo6uIL^#uJab;&C{u6%2I(iPPrFrrmD;DQDPg>rig1H5@aj_Q;&$qKQyC4<)_% z9!N7~sbx%L-hVKxhi+yIAgjC;0B0@3OjE-Rvtyv+QS8n21j$ZHObbUu#1^>Jq+A6q zR~;_(ED)Wl5MSr4$LITgl`tkpkaZbmWmdn1`C8B2N|n5HOyN+-Q-{n)rNeofJi&@{ zzWvjuK6vdjU-?4Wf(600j^s>abmz;d>8Z6hez^Xe%RN=~b9NvJ2V|J(>&s4!`&j@| z1>sMc6GXT5CXD_eJnVJPK^xSo6^Klu8n#B_c(QG03)5FNOIgsF%||6}1gbq!(RwMZ>yNy|sc$R5ec1A;Lk2>#4wHPzRbY zl&g*}9Q0J-AXgo3)h9qJKn^#0e1@ZFBCz=-$8FoTVW5F~a#H8WV?lsp7tGWeR>X0x z35obt5FCIKxFvul5i4H!+`+d}Eo2Kqs*s>mhz9vWe>E)HUkg(qbbq3r} z9e83 zUC^MS^__{v4jt}Otsh$HH2X)FNrS0U&0xOxxewg@58wX5{i@e@`c(I<)D^f0%^0j} zIDP76b+`t?$IGBJ0te{o0KEPtxYc;ThJ zH-G9Ion2Eocaf1x8tKsI|Muh8zw`A-9*InU{L^3m{?8uGs9f>kk#=z= z{R=PdeeVap_^Uhrc|m)Ve3NjPiD$>AlWMsJ6WE-4WYW=dPX|b)&LV zHLL7Ei6R;$-r2<{hK<8+243Tnx$#L(h0C}Zx~WsQ)Ip!hKX%SOHl6#_cTlN#K5#L{ zB`UD9McFiUAUJu5>1905ucM{QP$bx8eHk`vZnLK-Zo~+TQ!k z^j*&}=J92e_fi-reIha`y@ugZWVUSIWbSi6%~TfKFJGsuDo7ih%3b({$d2x;^1q|j z0x*X$tym3)qm;!pnPHU4Q$nYrDzl^3)Hawhsj4ssX!<)3TeuVkfs-Ria+UXAFa;tw zf2P(M@mP{_W;y0ZJ$e-=DQEtfZ$qxGuJ+`UR&6}xl+S$Tvz=XCk3RYsCt%-l%in+a z!yj3`eCcVYZd|v1{p!_gaPX;Dgz&-6;{5Z^yWoQJzxvgG;y@6LHE3T5%3ZTrUlxhn znpx-Ma1DXzSCF|Ci10tbD>{3@lFKbnK^XL3ucjYeQjNp)@U z9DNXunkX9hxJr^5)(g_r)g`yMMLZ^|2|V+ZwW>)qzy0~K{&0%f6q~wsAKs1ZW53>MjLg`&Be9*m z?xa!V@rBCE$}c)Cn#-iJMy!7@=g&k3dTh>sv7lviC|o%lt?zUW|K^2fFWh)pvp+Q) z8%bxq9gD^f)=b4Z{W#WSAtUL2vcKqGb~YxOPfvH0AhY?t%Zgmj?CdEW-t(4| z9gc$s9V^cN?SuB7bmZy_Rsd==%|(GqYGztynuLrmFnIc zy^v4N?(-F$b(Ym zAeZVi%c-`#s(#v*J^2)SU7a194l148)$mAl^GZl{=875?sI{1Z-q_X`4V*Ee=0IXD zYItT+V9!_G;Ks?0ixmXLh9FlJp_g}sWi%OXkNJik!Ti%=%()X|W9ew5q^V_=p&DSsCd0{C%pE97Cn80ab%~h}-<3WsgIPY5OO5#&mhLr(WOAM#x7cP6VlzY;zH!?v(IM1Nk`M;j4hdFvwjW` zZk+JCcI{fWHtRUAd7LROiVPND>g(%9Mn{lgdU}S{yYa1Cr9_NvnD?E0%E|FWd~~FD z&%jKe*k885`{J(;lmttjdj8QDpW3x#NyjzUy`j>#@ZbOG$6x>CUGZ$>;y0bQ?FD4uerpM3dY_kPFF-Y{zD5B;D2vti?@9UChv7oJjTul@BMPt8O_c3>P0(e? zve|sJz+aR1kbyc!-AOL2oMA9>oE(pz#C~mSM>d$2nTbUu0`oM9V7hV~b%sS!7*U<0 z+1+tf9jlIE-b_qP_w-NX&>*w*)XpO1Ldg+yd6Bq&{ad&H_$R+H7aL3!Ej#zpNjr09 zH4tFSDwY@g5i+(^>_RYQFhy}8n1XLI}Z zb=`Q=-@Nj2H~!Ss3GIp47ryYJ58w0_#VC{g`nT`A^(XgbZ7$PU7-I)Ez+f4Q0Oqb( zx)7!205#R!b@)H-e(1wDT|-47?cwg>9DbC*Wb>?@y)bIrkHj%q^4+I3Tg38+xgmb2d=OkZ1Pa9?0JF zQYMw7Xlhw6lTAA_35;F2$)e6cL7|K_l+I*4lD&Uh_Ttqxm$_N_=I^I>bh`{Q5A~9% z(HS#zjBhdy26+blsuVeSV4K1Jjs{iVJXU!zUYjqJ0VFs5!Iimdx^5nur3o zXowWz==eIO9@qIxT=La%gpbF?1K_PtDPSU(;H!sZ*wB2NK;DDgg7((-_UoYHt?29T z|H*B)Km73Tetg^Q=nEDt>UhU>Z^s;im4Dg}OF}ZGHK*f2vx_SSkpUI&Pk!Qil)+$Q2HPFF0I<=d8>Yc=kcsY?fUda-xZ6%cZcgP=&d5t4Ab9AkrFQ zOGfE8C$%;<}P;Fr|pRz6bhM~H#{Bb8*;duWv8vJ+_+}op{En! zbf}`t5ud0FMKh1@aJRMGZL7Vk$@9=qA`>hN+bSkf-g;wZY|4Ijcj=Eu)~13b)ziI0 zqd7<20=qB3mJi+T;lZloWV-pDesO|Eaq7F zk`F<;NAl}1!dvL{9cVlAdvu8#pPFy@7ik&47tpXTWU#- zJQ9h>m;TQ64_|XOwZIBEt$=GmN8q}5X8-ZaER0BZrc@9riCwWt=Yid}!v~^XZ_dG~ z78z$MggMmE<=ILenNEsN?)P$K1w^qSu;;P zntkeV1{bPNRhU3kk3!-2NmQ1nE_*}Z+>30=O@rd1FqA~9nADDqQ{(sV$?n;mWDKF2 z0-Sr%`!EawflDSHC~8fE2DEpzHq<$83`-+xI}`Oda85*N5mD{!4tA&x*E z{NcBCv^jB|zn-s-CVcc9^|&QJF5uE^ou3M)V>4bxlYtV4S2@B^uf7>{`E~vhho>u0 z+;Z`o9QfdLWQ(h+mtOroXH}!B3s@A?n?*(6&tpQZWTJto^m1_XICE1Iqf`C8wJTR; zIn-og)Ls`bbzMh20-dv7Qi;nI1JH@xFHh}ywxJ3oh2)&SMq}B@w5Ozf*`o6v2KT^VNm1$UzV|v=u@A}T; zSD*8Vm%k6#OFN&MSk%6>?X(3Im6f}9?dC{0dXI`W|Dowu)||88 zso!oLy?f}A*IiazQrx&W=rf8|u2_*wpvNvMj#Q@IxmTX=_N)k6^{Ll@6iu$cCBh~V z43im&0l}~ToK&5hBWce#;|vU8z`;8E3t#vGaX~oFnI?Me>D&Y^*;#Qbcfqva+)A7i z?hsYb@H8)u`AF6ISh)bC9tb)gNhOJ?%A8d|D;MFW^T7=&HYxfnLwH);EaW+P(RxY) z3Qsyl`bdpL*>3g@j7&_Z%@J5O+Fa?@rW%S$dCW2?`G<$c?)cdQGx2GgOD%-pz?%_sdjl}qfL#C z_yMI>n1&Ayk6wSn-)wzxr{NEpx{R4Ge&J&u|L`@~FYxN_*m>}_pFV)OhC!a`85-0# zT)buZvIQJNjL;nWdhY|z?cB$<7i8s|jkDWb>Y4BN5b6xIGQ*k_T8a3Q*4oXRPj70h zMghWMSo;t5-S@yV!{d{bO;t%Ydf?!pbY=_2B&wovwvr;FG3eO=3q~RcV`PU1javs8 z$fPo<1h-zVtG(8-p~G?Mdi$2;b{B|`p#29{!nad}YW*nhuBaybdUAaegrXP4Aj=yS#iD*~cPTNU~b7w8GRTiV_a3$i#zER`R z-A3QAk`%>?&ud>;MUByOtAQ3z7XFk|f8R_%{!P^5d64$Q!?Nyl4u7WB8O229E&Y?7#{=EkN^eBu)vNgatqXqta{*Il?^L_I?Ve(&)B>QXh0?t=azzh4H{{jjB-^@bw z4*$tPngTr&{Z*&2Zo2-|uKovehvH1KOLEy&8C!TD-2K2^3qm{V+D~7wq~)H6?;BmX ztl_jZ!`lupao5Mj9nnDQU4TrZ-OP@XcFyDA4tFgH?+ThflJ-sWg%!pvR`rt*PChAvgv zfhy_v&mRjIYGp(HU>yo#mYFZyfU$VO7$P*5dhot!udn!mOHJ9$MH`ES>Tgxa!A7G( z?q(4IWk6vvNp`N#X!@~-BY{BKm47uW$^R3-Grfnq9{gqZ{r@rDcNkZc409URZ*05t zEtRWIaaGh}b$sHr5boT~zT1+vg+{TmHc}7bxWUeGs+bC!DlQP!<2GL~OmKYh#Q4r_ zoo;vY8JleGgN$mOnL@ka4OV2*860OQ z1Ev3IfkFNYL)jniF?NpFZ(5mN9Han$+yna4O00LydYApDahs2dTu=?HU%#GNE5k_8 z*toHM`*v2IbO9`KeA2VI)iTXs?QQw;<%QxELM#q{S_Ferb2uF+rIao0EhKHRdX_R6cAeNcNaq^l|SRK>a=8Ug5w!=%(ocDyq zKL*kyQBq#B>#n;_=Wrr%)eTCsOczLh%6Q(qLaX{*wj&+t$t7ZjyU3P^mRAJoYOC=_ zIlBi=yip~eYz=i(&`ivr34W_U7k^}9d++_QU(qM$94u*IXF{^WZ<^!}@u znIRQzlZG%;B+d70ToyBxbAJ2Y`*0E-GgIfWJeS4%X_!! z>qE;on6K%FK1W`%Gy(z zXzrQ|J>Pg2XK0zvOZ8zHe6N4xht6;P2ER`XiITY#mM6+0W^Z4%b4V&X#}>=Kxz7Ee z3(?M#mGrG7D`AL&-Pf9(QpKUCfl^!O}H{mus`>F>V&9bxe4 z_rL4-m{uwe#)=cMf)1}J3mzm_-UaiUQNQLxHIW_6|B0CS=HD=j3H9W5Im?4({C(mR zAOG0LK1#oWYe_zAcGl7GEm^X7?b?&+UC`8GA|da$Mb4a=hMvIw2$~xXK!p$+Hq`BO zIY>k>vpy0*?AF#+i-R5!WGR=x1wtGhyoHGZ+dA9?pk5C;t%a{guo%4Uw%gc4NXx=f z00){PFnZ`Yd~}k+xFyw`q(V55IY7D9WfmqtByejI<)4&=PpTx%$&_230(u$goM?#u z(F{5oOz>&KE>j&S3%m>$Tg!Az+C3QYb9j^1yW!7`lxfQ|%qSNa5^YNzzCpICw>iGwpcogyQ_LvU}S8h=8mnQ4{yXQ5W79QE#|6prf zOY4mb#!el%+dbn+M3>mu%bBxvR2Q=?Z+rhFRv78#rH?RPWWzrSRxCuawpiBwi>D67 zdV(_4$2RFub}1?Ue}JNzYy@QBX`^CQ*8Gp#6dI;N)ko_cir;GxAATpUa$ z+%psIiUxaex#ieGFPE9gF)m{-y)d(Xdr7HFUIW44g`A9JQm&GU#Y+MW^=L^vl`Z)4 zRO4*(Rn%)w;yUCQ*=Qh3s@TX-W;$Hl(7=a8`YhatjgKa0Vx@Jhv7zY&63ghF~{PAVeQ;*L_E5uRHTIZn4voB=(-mcv9{dQEB9i^Gt0H+72W<|nmd1vpB z{ws{^Zs-0Q<(-c2_$Ij{K)W$6*tz6x)b1N<&_YOq5 z7Bw%QoF4CdVYGW+|9jqj{`8p%raZg%?d?6>6An*KPE8@&p?zH?C4uqKD4$C#lx(PK zFLqQ!Q_)y-iUsyqa_Xx0oR4Ow_i*P+yI2}x|C*|KGeWGW;QCkgEsvlv-O<9w5MyyG1t z%_)~4TX4BO3P;^ga9Sj@OO)qgc8<7**}eOanpERnA(`xGUsP2YZiTjfuxmJ;42|)L+K7IAUEL+@Wc+_~qTuNc^qIQ51!r=(}kJ!b+mJkL4 z3l}u`{od*6=v&|ZiN_y#MwwMID^Dfg|Ng5#a`SZ_{A-wBD(72w_YLKeaZbgy%}k%M zWy5(}PBBY_rK#FOLA<0mfQf-Ei}wtl!&MYC*IlTwLtO(?lY_GUfvR}fl7((}-t#CT zfj|zmHNomie@_qU3RNv={nRW{2`{CwU9-v9emHmgqqYR93zUcos0u zx-b`2{Bj3|)BAhbmg`}2jN#Ay)mbhtr6UW*WAY$!HlHhp{yW8hE|)3yD z8^Lj@-I>WcYO9mioar%3VP?kMTNT#`D(ge!<$tosW9KXva(|_dH7B}1mW$Gh>oeD$ zt+qWXaTUGFI+4ol?2fTh*kL$vC%U51;ZfQv#ZVPVVI3)$x}6PVMwF`ND)HHr>puMO!@?P!Feh$hA?kxdVU7c7ZEdwuIUVEbtFN|h|Jcap&6_zggZ2?ZjHb|Nu<`OZ zdo}-9{@|bZTlGgVx9V^X6+0UV0tCm(UGUjl`Y-?TFI?wPy_#Dqi}{!w69N4NF9W?J zf?+>0$67?B6+%5eUlxCX|Ie)f>-CTaO+kPJPj1SlGNV6!G4{$309VB4-rF|d%nXb^ z7oNPfwAg#&)#rZY-mL?9X+qrM_H6EDwA4u|3|CR^eh z0+H(580fc$C&KNQ@7dvu+8Kn=3^K~q8!=7__5~-AO%H(oj&446RY2!EBx|8Yj z&2Ku{%Zi-c@GW>l(pTfGS!1~8PQq!_;9_{tgs28M-^4QVKvlu$gPut>y6V3wDO#H? z7sRI`nV;XLRwvjqi8w4}V9=Jr?UCwq^Y6c+G3=bM{r6uDfA-Vv&i%^df%O;Q<}eAi z3X^}>O(>7r^wZ=(j>jL=7dBn5wUy-O;LqVgz79G6p4$#clJe zGv)~UXM6j^b6Z8WO&jwL>W(Suj{o~5!=7Az)!S=Vu0io+z7MwARqLMrx33(2RqeN!PD`mYoh&p1vti-)ecaJU&(N!lm|#rkTFoB_-y-OT*?hDGwTc zZ)RZV;B(*n=i1dLx1Mo!$$}-`nkHAU)?QL!;IHlvS!$$arbGL8riQx$^$d;F>m-3T zerh`IXTnRcal*4j^W zWGEGj6xXhl8JDE6`6V72OHPFx0YC0eQ=xHBuqK;G7#@$gyGNt{bB$6jz0{(+M zub+s=NafL zfEPeU(ag=pCnlyQBVM;JK9!mY#k&vp{Pw=zgr~xrUU%}F-*tW>mh0U!*uJu{@}%0a z@I>Fr83C0@MKqN^fU zVh#;#1B2t^lT+;GGcr*a`2C*2p|OvA=IamqZkx^H$FDj0UVF_OKmVB#*+r-7B02{R}?8F1tXqaT^vY}vtweCbhOmk7T2H~1^GGuAD1Rl$KL!lO|NntW|_jiH25_dZ9i)?EDR9?JEP z-~tXaTElJktyqw$H3yQ?7m_g?W}?QSAtUEvNs2GUv7(hqQj1;cy-^r><|_W$g=45Q z)O#k6a5ftqBJtT2dB93C^~hCKn6g^Qvk*xYu#Pz&eBq)+x%+UDK-tw^S-6^%_c_0X*OWd#I8k@=%4%C=V&izp{~FFdKA5AsX>7+S7FOl z>;{PtNK(zcnM#C!69<9krJ4N^K9VGHNgXp@{v2QW@sEF;gSZRxos~=d>Zy8M#}VdpAcep=+yoz<5|oH$;Fct!I>Tee zIq_8BDBV#ciKA9}-^Gg;i-Jj5S+qJ&Jr~K>LL;JM>TpS#!`8x|Lj{^Z=vUb6%1{7I z>L0b2z;!-)T+dP9I%l0$;)M{M!+dd_N^o?D#C2Mcq)?thT7l@856sLLNdnVqBbSO4r?#&N zHqHFtwuj$w&3UC|zUu5uxz`)=mc<9dX{YK(m@L&Vst(<^*LVK1iP65=%vk@7=ZT5( zl9rN1WtEQll8K5{AN}R{cyNi!=}8awjE+V!ZcoZy=`Jg?+me{#Cv1*XHsg;>7~Z0^ zi!}>lI%)gej=@;!xlg}kql?X&u2Odk%U^tt`8V0BLn*Rn!NR25qk2(-+8L6~VD8~5 z;YfFVqD7)TP@G$~BDrLx@8y@6E27WH;R&0GKI`^@Qge-2@qoW%;VdmL|KgV?zyICb z^UpY9Y}>{=mvYU)r)e(MD34X@a4~v%v^4bWj2~RCY-5}4yo<^&xs0^{HDWQ_UKr*d z5DpPbgDI!ojWz`{O;sq>X~GeD6;T4H(kg{GF-68doSb`dWfhJR22XZpusmH};Z8>B za8!}iv=b6@;;Bg0a7|qa=rtHPF%&~zq6pvVsQrh#0xd(ktKg z!q>k%wqt9>iuL}s5`D`_VU9@E`{;vxkKR{%(#dsaT<9pu?*J{#SF*6m6;R z`;)OK2Qiq>#eBQ+LNes?8g2EiNYpWL;Ki-`_7u0aHLgFsdgU5leT%cS##OFdx5|T* zyyoN>?mUnh@8M`UnL=2pu~;VKDh|{&_^Ye!IHM`6%kf3vKIyL8$M^2YC1a+mHQoso zOhumg&Z(RHYr{UNrqj$g;J`+jB;#df| zeZ@wV5erY0d2m6mhR9a6h@93f*;pB4Cmv_yX5yC0+>8 zIm{OqD(rvR6U`iwTV|eJU0rz7;K&k56ZNPv3j z=7S5ZjtPWYI0%pttmGJvGYyZ0CPKggdVOA;)U|fhuUXPuQ(HCEJ37@fzHD8KchK>` zuO90;JkYehs$xN~y}HHWaE(k1yIqd@>gv;qHl!1oNH{tfo|+g5O%6n+!!vdD^>4rM z+NYj+I+M(@Ibbpr?pWW5wNcOB;X8kES4nB0WkK!zzuw77lDU}Y9q)RFRdx|oZmp|c zb}7OQH{8Hh4h%~KUx0IHg^KYc>I{j4fYRao^Ut4y$t{8Bf-8E7&$(iAp+lr4lsdHF zIK{Jav4HGCVw20c=bk$!>6m#*DdyybArFctOf7^bvz60MJI&%R5h0W;u-y=K1~VW| zsAb*MsvL_-7e=Se!_6;L`anoz03b=B&}8S~erZuzCh1t%inSFu7UB3rs1Gk+*fBDt zjVGD)Fr!y`huL?}?3``;4|U^`A?Ksx&s9}aR99Dt7<&(NXJ)c?PAkQ>(w6@CXTSEb z&wn$U_1HWC7I2JQ^ulvDeDU+|D-Wv4G@Z5T-|a>;9zS@X-@x4lr*ah6{P@mC9(nRb zb9{k0CE{fbXLf9SdQ=_8fZBqQPOiPBo*Tl!oVJjT z+R>_@h?^IfFVX1h8;OLcbNGr%$Ngm`9SfS7?9XDEsOIIGGaXl-QHuM_fsk!tIyVq9 z#v`_#vD`BUY&*Mfn8|t%+;TYh7@wwLRF(^eht(M%bS3OGE_K^$%9Xbf)o7aP1?q%K z>g=QK#BV2V&@v4{r&oFKbPk3xI1qJGoA&fbID6Gs69ErO5%rCu0ztZT%D#65PKvob zm0eh))}{V{6DJ#R^$lH4oC> zHIbfMIL!u4JDNxzW9yr zatp?1Kk#n-T{VeR!)2x()g%HDxO(~i48!szi`YDo!8*nqf~u!N)hS&D_D$KI{=R4| z?(-^7U3?qn7h)y2^PZaC4P ztwhm73U+kFUya4b(d_^9yFyV{!%ZS>y^$rytlg#+>{ur?&d4f=}<8 z+A|oTy^F-NPwpN4%H1!1{EGD+dMt#Tzm}ao|I{gGI3Ih+J~FEOc!S>F?7T{Oh@f<| zdKA$71&|A@T8)u^bk|;Q7B3$x$Qf~WAb90fuBHYw0<-g%+00m@DJctn@aE~4wx%BV zjeXB6j_E0~P$m?r3scNhuZM*NEkBzK1yF& z*HBzl7Y~n5_Uv<4)i{dEtt}@OnnZG`)adS=+rIqq`gLm?&%4auv_N@6Ik5`l&8$6r z%T(9i;pgvfX<)aN3T8hnyw#Q2S!t0`T;xl{l7qc_cX#gTdwxm#rgJNnukwUyT(xbE zK#-llbID8wlY6&k!b9jQj<8ew`Fw2tn)=d)7CQ#Rz9J{48K}p1Mw;p7g?oCPSzI)W9K{U%aIfR|r>`5eOv3-dTkz@f&t!iRbz2-O@CaCi;m1R3TC zEO*T}Q$PlgEvYOS=0MP~N?w?X3l8#Ixm#3v1c#&4m?&IOuKlr-0_$T|%7WQ<1O8I{ z?%pBnZgTE`YWF;z_O=EL5b__E`MHmcO^pqYa;k$3lY~@c*^*YFmBI=WATMvcW?#@FCzOYILS4I0-Zj`L2E*VJ389hZuz^L>uM`y5{G!gaAIuo zz@Y=`q*dj^!_hT3-qkl^TH5gnsbPQ(ucbuj0$`IhYHMp+Tk6Ozl}vSZ_NZLc)|gC1 ziLbtncHZQxSgZf!)!8-3EZ+#YY{^7JZDn~7lLi$?CRIH$J|tCf>VHkpQC+44lX<*Z zImTQ5Y33KdPpkPoez2W(mvT*RpTSn5AD;RHG;~8}*E9x908@FfIwZSnv7BM=} zHK{GVcs+G;@ap7V23<PYzF*Yg`bP-h&0o>(kDj8G41 zHb`7PB#G+~!Lsm`;98iEipLC={^401ENY>Z+or|_tS4}_g6~;XRpyt3T?U>^P-&p; zZtv*009VUf&Ru=dN_2ijMSK8AV!k(Kb1Zx;$#KWPY!k}`ROalmlh@8N;<9AJ^bUPF zdUX08^m^cQDExPb5e<@fC=&dL_fk-8iF8LGIOhQ(Bp-e zI+YNDPZ05l2#Iyo@l8aYm3r>L2|B){auvAT3MN+z^V6UHG^Q)q2XZhC&XXjtfsC*-@)9)b)frPSiNl*{Mf00&23sHJFnus?Gs5ahs;>KgxPXDws=zufKK``l3U z;EvC~eeLNh>x%pyr>`jI3-pam4UK2RQ5!T5{MXLjwf z4fh%0>1>(~&`3I68DEL5vB9}+U1047cO$+SWGs>AMApbj$lv27q^UXh!H-Toa9{4g zK^7L6J6Vc|Sscb0N@WBM9<2J2_#=M z?MLUYQ|a&Rd>kmL#%XR;I7dT6&VU~el;+Bk!Z9C$dJvS~Ra)Bf!~?^xY^z&$M)L)) zuU>zq(_bbSCZ>A$@$u2#M}Pa`S3cHs#^wcAUKd=x-tJKrzas5?a_Vpnb4^W7@9gbA zJn%C9=CSkT&>8Dpuxfz!8~>nf^)|K>jrHu>ar@zlWvkmZZSjXjeRXY~s(PER)G{Hl z>T~k&p-f~96AwhP0?&75CSAF#v829^L9oMLhNT1&$3_N{lVeq7nW~Dsm4T3m3>G;r zULu*>8TJ%a1l5FxZB3Y_YrU0V#z%(O_v9|Aa(e^3Yygppz;P#h(k^cx5e^l#E%GFb z8F#x&*rR@w&PDLGOx4pANecUGtClPrAtXBsB5{E${KP=6`#ZU$(v1=no*Is-pi&8e zg+|SJ_M*iWWF%_bv5zyN%`xKa6QW??FDc8_m$-3rrU+-Bex8p8QRO!Su)&D6z-M%U{PG+nUrCj?TyXN_Wf2)se{b3%@y3x4eRRYp~s2>#mzlPDheauifSGd2LLi@bQR@_FN(px4W{w za*w}4O{{DmAwrI*=141HQUS84MTtEAj=i9FSf#DI<87wTY z#^B1VU%MQh5c0(+x1HRuK%L6$-Ftw;Kbh)fbI!)b`qGkO`WHI=ojVW8iZ*_^_@~ON zh?o&7HUop92OoK6`LZ^o6@jI*9vu&jgrdyPP5+xX;KX5=NurL1Wu1h!OBstWl|%-6 zI{mtf&MPifo|{I;CI$xB7p&S|TP{&uQ(jTQ2~&I#h^$l<(nIdq--$tlsTpuCY;Oq! zlp7~i7Krlm9iobaASvb$Ln$=)&QM~sRB=ILz}8j=9YRrnZ!X(CoEeGai{`=9rB2nQ zNv;Go+1p0Ovt7e#mq!wni^smQjj^6RnaUpIY*{oM>}5bNLDi{q>iNgM7B;2O6T()> zLVn4Vb8Uxh^9nW4?4QUEg%cD6NqEZaUNJ+C!426_-yAQR!s$7VQL#-lJm;)*t!hzL z7>9;YT~Zv}HO38JW^vOI-MkW5$!&k77efB93`sXs=4v=ul;(wYg0U704zo2Qr zo?Y^~OK!a3M*MpoQ)Vgx^(U4Wd=D^UhKwGUV_6|Yqd}?!p(Rd4;PgQM`mg`Wtq_uA zMRbz(_V#1u`v-w+31N?kY_LQPffr%|lXm(%Ilxsoh&kLJ#jpu zc%qZ~tH1gy6dWs7tiTA6=ZM2CULkflv1^eB3dAD(OXPod4d^e7!{aVnHdS0S(DU-L z1)DFaJfro{y|=yLb+Mo?`3h%-NAPQbyDPUhNiRS*lNiT4Q`SBa564qY2PWJTwWYyk zn>U&ajZQgzj#PR&lC_<^sBBqFu-NB(X50S9caIEkLe|7gqcPbxmUULv6{kWq<@nii zoxY^W?{@khF&^7BX6KNzs`B6V#cF+{7qwOTloP&V^dd;kqoLI;+H_9fjI)#39ZUGb5kMw7lbvRx2ZxXu zI0e8>0JhF-H&wW*v6+{fIgB9P>0;n^)wQY@R#PpyOBor+FhjAi+5{1Uj1<&dE53#;r2MtwJ{vU;G!3!;TiN=#7!$t zm2sgA*a0)H^p*vk_%FrrsoT%_Eb1FLIz$G0ZP{d?rjmWn>f{_V@D@(7nbB_dln2w% zX}kUZXYV@z<0_8+ulJ%;?^dxS_lmm>#-{gBLQUui=@8QBkS_^=5K`#9_hvfSU@)c_ zY-1aDOIGjgbkg*S}XWI_8d)iIb2 zZBuGlTS@xA6oIuxSy>rIo|r#ls0j!5u)zJ%kAB2u#8NryGElcXV8ED@rcC<^+!*aE z<;%lBlL9rk+!9g*F<1g&zG2TY{p`2rl}*lg>*Y_4X5GQp%nW!Uo?d^i+Y{0bn(f$! zf{z00i=n9AjEN&&e2o$BsS3VO;+Vk%qmdBKqC#7&HQMziv+?si>vpWI*IKl;T=Rsg z3A&i!i-zs7aO~{g9$Z#j>KW|0<%OF!zuR!~MTZ=F}NY3< z?M@hBiU7laL{V{`4GIFa3N9*Y_qM5UycFiZLErLa^Upf{=rwD%J^jj>D3lV~n9m!2 z=GixoJ#tY^by<=rTB3xit)mZKBH&d*9r8Jxrelvh)SltM8iy)|fmW182Ey|DKs954m zIcX^dxmEQ&ai1R_8g`^;%^BK>MHu8Hy8BfP-PjHcB|7NjtSzefmP3)c^%WytOCc_2GNRq&j9cYmS?Pv$BY#9(#jG^8yPHD457n7T)L= zjE|qWgk*p;tv%{6?b89tFgC=L=TgbV;D{JOV8pjH(Ap37sK6U)RgNBGPp(RlF6bny zjN@O;mFhr9NOdg-8P0Jzl|<&smZ4(W4*;j~mD9(fFfupYF{VLhI_JzY?s@28dbZTW zV~{F(McBd!8x@Qpjz0SEM;^JqwWR|`^57ili!c5j=J3VE1u#i~NyQ#KK$^*ciwp?wr0Sy&WaT)Fb~I}n@CMQe13OCaabN)c@7U8CBncHPB;OK08g(I z&MVi(sQw$?q2MnHmK40bjm5*>76i`A5WXs9QGmcfkm=*VDyaNnF8~}&q|ZI~T$Bzt zN|sR(Sss}Vqj5%sFW&?DD{Kgab|`6*6qvv`L}EaK1$nfbr+CP zUc6BVz*o2<4miRnA`Dl6GB3`nlnXfN5XP5r6jBPlj3cLx1!o~Sm9LzOxujE&Au{HH z(3feZ^O7M4!iDkWN|tLzE*Dcv$7hHvl}sTWUoNjKVLBb1DKnyV#ci$Gb2dHxUTt)> z?#wx@4t-qfjkOIq2Es-hQ)&vw>=AubtFr0r_1ik3Q4fRzHhZ={lAYhBEztyW%WRIm zu*s?Gf|II%KPMmNm%8;kI-mHUWnE{~X$sHPx|dDKdG`Iio4poSO-V$DV0gf#0 z#=S$mExA+Y)SP;LOlzup^~tS|+-|npoO2Gv9K3(~mZnuN^>5ppF}`x)H8A{ zkSnni36^%mla*>uE*G)GK@4*iFS+I6jkjFWvGD_MAdqQKn6QTmbLZiwO)>ETkS@+) z)0j;d!1*>lc-M^YT!5zstVlm4jhrEEv|=iwiMM)MyPFy+x%XKkx{0dHqcdSTVHc z7Gm9oX+eJA`1`sd5w#^FH|qB!B4Kk*PRt+1F-ENH6J(MXSpv?>S5D@*Oj*vGo;r#e z_QWNME{u1-Lq=p&TKW=2n;@=YREE)-Sm_|bgAw)C9+;c)P@aB8F8Y*U%v51RlSM%0 zBJ-lmsQ~yslVqSm8JR}CCxCD&;GC2qoP1g`FrB7Mnd35qBonO%fZTd81Xtz@Z8A2W z?%uuo- zS*{;0aC#Y)(lLT^m1|7K;kcYCQ%;8~1QC(tNv9*rFVmDM$Z?q$=aow*QxCmBOz6Xry|qp7aI3P~@EXpY zIq9}P|H5XoK;ccVrdX$;ua%5VpoF6lI|7#ren18W0@T~bV!)m`wVL3_hCfjW$c}SW z_BLbdiXIyW;J>1{D930Nrg@~%L+eMXZKejz3YQT8P!P%s8kL~vE&W&{!o-#~#+r=L z92{MoNYo||Q~?V1-9*as&_@6)g7_9dZATj{T}ayq_ZsDSnjAa2p2U`>=s;ggW2arV zc8BK5)2xS0A~P$n2PSbBCFP=k9Mu9o(zv#@AC>_+m|(#=AU|7k|Mx6LJybLFkBW>y zxZ(yQE6}5WEh>qo9@HZ&R8Xmg!a{w%LsXDR#&buWa3oRJjb2n6g$ajRH9lWs!;W4W zp-~{p6euFy3K;PtuB_`xg2WHVE4Liyyd3B2aJQ|X{#Qg238=9jeC+XXFibUvA`TY@ z-x77DfB54~M;vifRaGVajyd)y^b?rM!n6;bBe6t}8856AaB2yx)S&!Uj>+WGp(3zU zj=dxZ&__UNhf|c`vH=|r&Q)U723MAHMMMsugIlOS;CB;|FI{OkQRT%KUwr)W$1yoa zZ`aV!pgbOs#l{otNx`WZGJR!-o)VE$aRxO`e#M>|WQPM}7!F{Ujg0Um;tHd&N(!0L z9gd=mT#T<|ml3!f5!Od=_D9-IB|v7V9{3hUm@s~Uy&OgWFmM1I4~Wg@z!V!MfuNi$ zqi|G6?K1+YMsZYbm57{C7B#(?aN-4v6&#&&!wolV+qR7zl_NXE}|l1idETafZzp-_`k1 zUlhADoiswAZWY=Nenm$W9R0n|8_;1T8K#UOGR?KwlZHj9LWxdpINR_o$cDn?^iL06 zKe|&=PvDQf1{@>r8T4jmW~7)>^>VMhl%x;f=fX}AYS^5u>XH$(^7 zX&{mc1VR`F&>a{VjLAlc?vqFq<&fTx0|j<-et!`5sLHq}g`Uhb^7iYzJL23+z#cNM zeB|ygC&$PN8%-l~!j$}3i<&=tUvD-S%_mz(uEKD@zx9dxO}32One+5H`9`y2{3+*S zB)b0YTMUms6bpyUIoY|>=2e||M)uT$j5$TJO~hy(2X+se9ZO-+!GC$EW8DXJFFew^ zVYMzin3ZX;ThS@+n*@{z@cQ1`8L4}7MYZl^m(CD|D#C2i*>mXaf#RUEBNiHh%XsnO zDlaLvio}gs4m~;T*FZI+7H7%=-&4C?7xfuLzlSVhQI6g^eh>Lw!fS{x!WU9yPb{Sb zSScG!`!uZZwl~2#!;no|!Ic6;!l6)KpUUhoWo6;S=7`thsHjQA`q6=<sF7O^u>%N<6GUG)*Zzal@-0U(PAg} zL9KwsHb$-}Dl5xuHY+u8)Ou2b8VrWIU^xJF2CR1?{`?%#feJka4HtlcKRU^^F6#5C zj3#*2#89TRB%ceudCRt_FQBr+x?GdtvX+$<0`73weslTR7kuxA1P*qE&LNt3{>9f$ zIPQ>nbEa@E7#-BrH^Q+c4iJZn%gV|MY-yuG2n!7@Msh$x0+0lvt;d&$NK;cgoTewp zZX%&InWjyf2%5-X_|I5w}?kHTj|&BQ-I}CL~ccbdm)NJ&L%h zEK_5`ij_*es{?ansF<-eL|5oaJ;dEA%joZ0n7pc-n^@Xkxm6)6%z;?*p=ehFAs~&prUJyhu7Bn1q2xrjBdaE z_B-ylL!rzOg+7jA!h*o4R7JdWY6?W5Fe)!@F-w<$GNv4790l^*-~JYEoaKy&b1EXR zXke{`0^w8?{};db1?T$V4}S zktvVLqTHtA%hVJ!MAG3heL0ok>6DooW6C%(m-M)dFLO!9WXS&c%DEVmA!Bio<&?RP z%1dpq6l6>ZJmTpvd=wo3(;)z&c_}K){UWF4-M0L3w$AaH&?iJ_Js_@q;4|wN)`En? zX>4hA42BXqy(ijNkZ|EV)ZSo^MQ85v_gXWI84e3}9lHCypuc0IqALiZa-F&vt7IWnr#EYe2XF5gj3wr<#yvGQwj_Vv2|`qu|5%S)Ut$0utyWo2hwdF>B`il2&3ZJ?N@ zq*iFrvf&R(XX^PnJNhoV=xVHe!@~0iKRol$L+6PB1TBWc8U9~xx%=i@uZ7QiQCsT0 zbq!BH^=2d({Pop8#4&6{2Ty}qdqpCpFGobFVM3rupLleXLzI9UXUd;kI#zGrx*O_} zxpQ&2rUs*{A6|U*&9`2Wm7Pf?pu7+pjEIH!xJ8$YC1!C$-LsHp=Z@MZpM2@ZKR&;> z1p8UJu8^AsEc8bZ5)1(pS#FL~WsM!u@s%=<%9IY5>C35W*KOpyKuDKFhR76>S*SKf z1Q}o9GX1AlZutF`n;*KTW6NhHix1OVt^Kuoc0Tiv!D5|y;ZJkt9;!ClfNaUmop9zQ zoom-<)&81qpO-yju`Ux1L{03jLgv0tE|7z9_^NPM;4CQM`SX_K&6?M_@zc7eA81|w zo?#$}(V^YOJG15DQ0V|I$=B63b#!grY_wQ>%jmZk%;Tqt`@CXS2tGa44_VS*(gh z2}_b*KMG}qnj((^NylMo3IZY{Ai7L-*^d&vtJB&{(!WrF`709n15s6@7lVEpjElM< zD5zn?J|$CCU^Cz}XIX4cWp*Q&j{K(NEThmsK~9z9z)VLTh0h3_YoEx#mjh800a@@X zCn<3ZaxgxG14yhe0SNP9?1#lJ*L&`{hjGwkVL1)Hz+}EMH6@P7SU3v1Q4txR;|c;g z5wJ#3R)*aDgF~x*yArLA4D0ff<{iFd(aP7?ez5_oVmRVKYqlE_hQ!7%dVE7+XST^` z(-fCxBWb|fhvwaGabe+Y`lJ~iuO|=)KJnV)8B=T;lfJICwyn3hBhiYUVX)a>`&p9- zN-VpkskS-Cl~-I-=*TnlboXxG*icqeyzcewljnzuYFwYKUw7J(-%&*62-kyL3Z-U% zAeWT!4;(X{zD)VR`KDts<^5mNbIA~n%M@ftda319xm+>@xn>xiPDjSaO9BMy-9uz{5JPiS<#eS?t*wm(oK0VBv% zgP~sW>icLssS}UI^Rlu^ib!pN0P7%XQ;-%*+js7bdV5ti7Y+-_E6N^MQixO$K#=HA5(Ho)YLYv{*lg;`CbN5l-dl!4VA#{K&|y7%6lpK!*_s(^iK# zXlRXMm#?rg1J7hN@h|v;015(H^z8ibKoS>5yHq)0j~8|^SR4~9ASvooquSFIAM%9a zQ1rwTcC)%Pi?;7j@$o+uXjquKV>k<$N(U^NEW`JONjVrN*{!BX{MY=#M~g~z`YESg zyX^+NmsFe7X3+)z*^maq&9~lKdFP#X!;S+2y@!WlP6;r08Vd$MjX@2V~#ln zvrYKagj0iQ)23l99WHB|811ve1L2*=dF``zzw%Qb~7M<4*+2Cmkljyeh*10Y{j zz*0ZVNxmv-{~PJ7TD1ydqk8dl;0h$Y2+$7TU|LpZAeBvN($T&y2wpIRuZ#OPe81ir z;ASCBhQvYhr?$19{HuqaT3!*Do$q&tG;cI#?{>@y8w+C1O}W@Dk}w-`eHR|N_>Bj5 z7wc{AptsbXYld@)>fEB7%)Ws@AJh~BFy0p5hFWcos!QsJbVYHsDPEx;w5a`Unu6yW z3VP$(rRN>hZ84#xgug-b>PeB*sw}hN{0W8V>>9g0J9{HtZvV{EtSYRNrH#eWv|?PE znVnTqlDqzsFMM9#-n~ulzw`0wXPxlki!1l;ZaDYCQ%g$=VZPDX(f!HCo7-DD=P#aq z(3~02-$x?w8l}1JkH2{RjSuSgHaYC}j`q%``j&+U&nd4cm^5|bs+DU4f#{xH^~avD zZ2h`TJGa#>KWu(yXU~&Qyq4*-U3AImD_&X!KUD|Kozl_Xvw7o=oV=_h2hYQ?f6qPr z`ot-fM;y8Mm6zY`XzM90EA+SrVIjA0@oZO??d6wNg@e(HF8TJyA8zdL9qjAvn>lCd zCu=uv+PLk4@1MH-@I@%p-n!OEgq(V!3E#M3`==jo%E`-Fw0K5;pXbfj*5nuEoN?B% zFFyY^c7+!f#J?T@ddFlJ_;2HH$jF>^^&eVSzTEWgD;w^*6+1EX8IHmQ2UmXU99PX$4Gs-q z5nuslwj6ZD@7Mm~Twi-j)(i}pup?TT*nXAr!Rc(ld?+>Mc5mL$@aj{Y>)(e1x`I5N z#e_Lqa!91`10=^~7^vOuD6Oy+7Dk79qP{#`2DRAX!2w+yZemAQAvRp=OeP%W0%Zoy zOJ!98)}6Aa%=YIMU;s(lJuJ4s^+r#duYPACoo1ILg3)Lsia)iZs6dOo%Vw(@rqpR) zT(Gw%;`3PYa}AbZw`#z`{ujN;6(t`90dsaiA_}{wgwEo?2t+B-e=LGCq*ab7ao1c; z|9To`j|x(u@U3}5ssVC+HX03svte~G=1b@zrl$7Bu7S3kaTyM?v#e-> zDXKl?;$z$DdfvWwOHM9%OC1$xn%db(lh0Iuxhn=*|FHg3=->SQOEtjt%rkYoJh1Dd4Kv>cOvyzS&=35(G1zF=}}+ud^U4{o^hCufxu=Ne5I zpCo#E-M8KK%uBDYP3U1$hkV+SavTCdY72zjd%9FQtmeXnL)hiA=jUcI97dlvTzSc| zwV&YVP?g%EHX9y&>YbBMT)uGLRG5DPM=YOUVr#%c(Q~h?o;s!KjMEO!ayg+xp^py# z$bdBJ4+L71x;sA2ZMpcy1kkWd-P!(AA)kJVs=e@ zQYp2N9sR1-K_ZMSu>{;Fhu_^1ol>O9Fk{DtDil-KcEo=7Okh_NJnup~ga;6>&WFMf zMAYyFkJVvKA_|+i+LqYOFZr%G$y8aOH_*YY^g!?jmm$G$+&u_|i>f>yY`9y5SqC<69hh=bciR_qEvo3=Pf)rQ_lF9l99Bc3F*hz-7{CtH-$}Pv4j-0AcRxlYN$9GB^HT;?lNkmFyDf}AT|@^ne|FAu1}nF!Ow!@}c}=_}H%S+fRlW`}2|z!715 zxxDEVI90~kKLt*eDa$nFIAh9`$4Zr{$@J6X=@6Md$H$^3r>4u64w31|Tx7T$mt~Xj z4}`BwCmo+D%M!}?9G7v@A#$orU#647q0d{EpW!mi^jtDT$*V+^R6x+GWM^j{b?m`| zVb#;GY9;Y9Yia3x{E^o#y6DvJUvTALZohWQ)au>4_V)Mmws&-2_3OX9z4{SIh2jnl z23D{Bbkl~d#V0O%_2rejb~ntQKkHYQ-}K}&x4!ev`rKTX)n@j310Q|3p`)$4urLQB ziTm$++F&v^HniM)(|zx){J65JTyHRZzJ4oY!+7BLH~g)=t>;^(9C7y@PZk#C-+9~9 z-#+u0=bnBOO}oKj_~et%4?ptY^`CB=He=GYSN^rDv$wLcq`A2Rx}ScJCp*_wP(ULj zkisH@kWk&y(*DGwE6~@rwRV2?$%bXi7hz@N@kd{sF=OJbw>*6C!Lxdb23ne1VcFs7 zAK1QaPitFOSy}n@H{9{(zg+p?1J5-#x3{!(gt6B=9K#mh`3q(;eOx&xRhj|}S27^V zOQOUXUxv#NnVO8l#79D?%aMCz#!06p=aT8mG-bGq4@VLeCx5r}=u;z}essFfYUy3s zIOPBOM;;UHFG@$4SHNI zhb1rbqnAFE*-~Z`f@xymFc8YkRe}=f}Bg1R^}+wtQhGZ97#51EJdb zcCqgR6X9@nwk0bYigGl2WXQ32;q(j6J^lWBo>x0@-nOd0v-8h?eQ3&kzcrJ+f^O2} ziL2h;4AIf_KyC5s>;8VtjrSL1XTr)2Drb0@07HLR@A3L0Lp`6Yd7`wW0LAR>>+k7? zTR=2fICVrbb;<-$?V{HpQdDOA-@2zsWpt^a*j1?t3bU~J1=1qmwzEHW_uFA#SYyy8 z!n{F)l+K_4HzINDjUf_hE!;qAXbA~^GSwMbv16xWX*^ojljt1`y3{a&T=b|{OqHI-mMzpq_T09Iw10nU#A5vZOOvr8uMRDq~ zPZx~p(Oaqvbe;{0Q+)bDPOk$5%m+`dBIp<)x;TVxr|I*vKHm zIsV^jogumg=2ay(!um3&G^C zIK1$P=?Bdo|H8BD;f<*?(B;%wx&od8Yfd;`lIK^%%E$%-h93#}pBJF9$; zZK$ahdv!Pd48z0Lr)on5_ERb^9WOq?*W?5)?={^Do9%Pq`yxiUK2I^phM(c)RH zP3@DXj0d&d+xFJiHoN;hwY!=vb~2(q?8qfeO|3e;!Q=L1mK9e|DCeSKnH>ef46~zJKxa;F z)`=$_wtCfvU0uC}#radG)xhecxedNqMgPk@zSb*x#4l|Cfig%h*FPu*X7zfj4S(Ws zCy}|29+k87$P?U8lcWS(yE_+;P zC=?;Fl^Z_H;t}6a#Ou>rGh!hxhIc4YJmN>=0b@|bqzQQ_jt7jDCQ@hMi~N$c~x^r_5T^6Gw4G>t;C4NL7L? znpv+ZaGJ57U4C;Eg!ES^9ikwA=?G~4JIIbD51eP=AU5n@9>o-+$At?Q!XS67j0$|8 zb)}?!i3pEuK^A%vVZxN`ykt^vYZ$}&D}{K#AMiK%dz$)HAF0e1vtyjK^ay*X%fGv2 zyVYcin*EuUtmTWB_6K?z_O`Eh{q-rcbKut67a7=GzZ1QK$%rYQ;gk2)$0O>ZNqKg^ z)$8%;E%9(H+OVl}uq#+Tx$wg4kAXao-}7?F8@c?fKgg1h1VUki22H_e5+wh>sgkHm7ha(|x09$U+nxezOrVCtg7*24>!WrkD zeWGASen5M=TWBZ;&zHJHd0Bpj!%mXmPiU2YbNTsiytAPb9va|w&!S)P;@hX3e8icj z9f|BGov`%2M_vxV?*`UyXj`8l9EvrzbyM#K7oHR#A#@h8qT+n)kr5D8caOWft3#!N zIY}J$CQ~L=0?sWMcw&x?03*4byIRPDHClL`sd9X|E5iwqXsW1H@9j#s;YrD$3S(oS zDh`7PGJK%3>c}C6&{N=@1#4`_pf1R)OkMzJ@MwBrV)kkf95dBkDMvsf~EbWC5B4 zgMKRIqL0a}e}L;&hmjzx4YaTbfeD97Q5ZBwUnSPQ`;C#hW3r^c-inoT|iufOht*v(Ea%jW>D* zhd>Z}OyKB+Y6|a(P*;;oqES2j^fPa`{#qOn!p*e8U6CJ?WSq|mxWW+`**hb$AP#p; z@F5BaoNJ(!gi{74qTq90E;6prk6duU1$?Wp8x>K?qQDh-n3@9T2*el#m+|FPhBJMc z2;(qBPL=Uxsbrcm{(;CN^Of_a)8|@}WlNV(kpx>wVHbl_*?|N1C9tC@EG&c@5x9DV z*9_QdvUdz{ff5S-Q}G~VjvSZEn=Yp;$yoS|qeulJjLFo};pzB{Dd&>&a$JTmrcC+E z#bwIr?~g2J`g4&v%5-Ge(q)!YIiAi}&L#6r=g1ToU#25-;aoBu#y|4hA8^wX*bQ^& z(O;8HnehQZN5%2yEfS9_F=*6MWk47VvmE+vK{onr9 zkFgAX-bJU}bI0T1aCGXls+Dhl_|l8-)YO#Q9o7?1K4Qr4?d$KIIJE|Y0UTEkXB?4O zM6cE5l^ZLOGrf_}_sHt*U|``E)ToPQB{Q;x?Y7hd=qbf`xi zbua|5J55=6c9TKd*VkurSQ72AuHL>d4j81fCB*1UeD#o)J3!dZVi*&SglEj2vVQ#+ zZ@jU_pwrKrhb=brrpa$3*Qfjj9(bIdK`PKg9!Z($@KxrR%0c5L42n{>$zNF>2FdcI zo<4mr{5hoam2=5_Wjc(LqDFV1lQVrm-BS;ETI%!r`{8txYAQ9Bkm(5Um~Cu&>V$J< zFaQ8R07*naR9Fy(<63?8#nDK(<<$qgzIK-rV$w^2R{}{;)nSy2Wvf^)L~BOz2(G9# zu&1R$7LkM$_*iKameb;cgPZmk@VI{Dl7cM0(Sf>Md1e}(Be!^WKv)yOQI$pccwuRy zir%ad3K6b1rS{w{b$YYa;K~Hc6j>z3eVy&LN|zxs8-!xvpfM{yI^>1R58$N4Bxb2B z!*Q8@dOZ`~$XtpfoHXhxr(%IITK-Gj)=N~85I#d-CQF2v*z=3;r7sxYI{@)f=_4pX zIXW#1cFS~Nl1wwbJ{9>n0xEJD<-816@JC_FxmXm1pdy(}?MuXwMUeq<1cfq}3tHXs z@^YLmj{yv)^7YU|4`t_QN){!eP~ZsYBgm;72M`E+l_AVG{kby^!Fhg0RPoSAeguU+ zJjrC{C6=A3d+U*KUx!#)lo0$rzi)@HXnGcAWuI>OeAc8%!O+l#p^dqjxmcEsS|j!X zW8Z*xuzzUpmUctPRF+$H>Y~#JLhhb!x5^$5bVac%5HH6-TL6~z*(J{7E}92BxTpTQ z3Nx>9)%i!CyjUqUB8pUqa2z@yz~M`^zP_G)f8ZAV!Ob_{{Oq&OzV_N{aKFkF((3`` zKvMeq#`sL1qn-e z&_~~buiPI9HMDf&TzOU8595QPLZ>UkK{(=DK!n3d$1T~sW$W#CKN&{*3cUpxH9(Jm zhl*29Bc!^8wFNdeM0Ka0c;u}&UuL(F_YIV*p|Ld-A|o}mDrzy?D=Mo*8;nEKkeB)> z94T_y4^R8mFK@v)0-%X@^`^f(al$c&*lgxGGbjA|(zCDspW9Rkc;ixZ$7uO!o-Yc8 z?SWAkN{z3oab?(138>+=&febs9vtFB>IS1}`gHOwiX;d_L1b%e?rd+}g;FP?CTyjw zs3?agT?ot;Xq%d%8sQxT<-|4tF=vO-L-G?vvCw%yuR-dHNGLZy`qy*y-xm==&ow~^3dP^8mDsdAvo5W7~O9syn?jx$!{KR8yS4D6Bta@!J8EgSnH?q0tU?_+G?XkU71hYkii6G51*|6`7x0{z8DHYYe^d#s0Tzz zkt!sCH3dBjDB%n9)#drLYeqbL)h|ynZEyC!{dri}2*Mo_F@;qN6ecJij6@(Q8IGVT zHR^nq#ztdLL6}|()HYQo)oQX{aYun_a%9;5`#CB*vJNjbjHGj*cgnQkKrVS^zJ;Ebb007olHjKlzzQ*l)yNJ79A`tG~$#%U=^MuvZL{nu-t zyu2LNm~iUZmO57z5gghI%F2U4yr!lGeqOOr=wIjf4TW!N;6GCXJXcJ<#))|4_>!Z} znEO_EQ{Oh9Etnq(MKiQ8-;9}cHeWPo)mpL(?PcR~aFj_`i)X++6xGDh<6)H;U57X) zB-|Hsn^fT$2I~xCUOs%s@9J;3=FOb%&dxt}ipHcfYBgt6WF20J#ZYaj6g zyY;VEWVq~lgZ|LN=EJ7fm60)RMm1EuFTHwaOLNDxX*E`x6|LV-etKTVx4XT*V15of zkkc2MUy$>szy3NmH|yp<{nFF56zfrUm~ zotfo0=R3#tcDrGnimfO^L*C}*uHuqhctM}PUs}Cty>wYZB96pLG3TXs1x{i_!Ibr`KsDTQdp#HAJO){+GBP$hVIMyP~23r?Ecq#1jf%7!jO!;)$%z z_?IO3QkikaA|!$EN>IoEVs|B`AEJqc6AMq!e7-W#*R6_>8x(5ZITvaDjM40pjAND@ z-Q(@Dt6*&re&@ZlaUCW`iLUm6s9zOz$IrX?jN;(L`lh{qd+tv-Qp%vg3@p^JwwP>vGy}40!sUptk2>jn+7|&45b}%x0^UXKm zObzWkfY?j^)1Us7{jZJA`OW=TQUjvp@puLo%$+%7$^^{LF@0UKa2n3LA@Zru8m|hP zK(pEK$KPFg)L~0jtax+x-lq0WpFe~q5$n?Eg|QRRP>`QBbxQSVryVzURt;V)4)|X1 z8Xa}SQYiD$^V+PISu;pKMKqHv66p6&IdSQpJ$0R(y;_rnbgaSP=bLsNykw@)sQ=C7 z=Pf&U_VX{T+`4^lOM4%N8PpiT?Sw{Wwdiv*ZO*Jrliqm3ailmyJBlXDq&FP0XfD=B zG5N!IX;RHNB*7pB?=!eUFmNC8`WG*ltJ9fa=YUgbr%$aC*+CiKh1zD+E}sQ6)dY?W z@Fldts5%gY2XthGkp_G~YjW+HT&JcuTQ$E%ckE2Ho%Vdt+#cN9`c;7#TEz$~KngQr zSD&igSf}0Hsp=Te1j3{zuv=9{8L`;)({!KwG2_v-;cX4E zR=3verxQ}i-~y}iiI~f($#rQQX5IJ#T?u(05ktsC99#K+b)xRj$*NaBP3-ANG`iL9 zK@~PYz-|RbxH`Qi)2uCU;%M^3oEi*;h>>`{YP&h{%;lDs)msTtNiX>VrBUBM zCR31cIGzrXF*%i4DVTWR)2B~6{`ljbdFEL>byPvhh%gg!1)7skFS6_VX#SQI#w>7-v} z$z?h?))XqIl`B_53564C@R)Gm3k+wl2NnKVamC@aaKbu@xO~g?lP;Cak>fH%mQbc9 z%f>h|Bps7+(kaWi_RmXBmFaL?rY~cr)0bsCaLja$Oh=}l9+!E^sp)+858+f9Wi<~-&8gA+R6f2j@j+ocgHMpW*;jfl|tqo;`{BHnN)s%a=&Sb4_r$JJ!# z$KG!qSh+jli(v6N(I0KU={;L@p1md)YIz)Mk^}SZRBenU|0oGjRgzD6c3P8uY@X0(uIb+Lc!dhgWJ6u zB6&bt6~g)OSpC+*KY(@S%n6Yww9M-2nhLHx9O;^sNm>#-Q-i^{XfbUo!vh{az6^s5 z=sQpx))0Z-gb5WG`T$T`R)VY20yv!Za5&-}3Six)ykeYS&g50_^+lxN$Ifcb&4d3r zqL-DGiL2RULUHIml>xUCCy&SaN=a!kZ1F;+>o>Ft{u}f3sA+id7k_QP)ofN}4 z&Pj5dA?Y~j)cz4B$s%|r`zD#YOg}w-)sMfQD(gO^mHBdfEC|D8zV^cMU|VgtzYE?5 zHE@DIvNq&0%DH5CI!%ViS2-`obw(2wxX8{1Z%K0a#A*l(r{i#}rGZyYJbsuAKOmu!4Eg&*~ns-}FhG1V$*a!P4m`V9yq@&djJM3g$u~0D3)vgM8^?K|-r6i{$Zk1dzDIkwU zA)St#5plj{;-lyQoj_v0a9)Kn6u9s}|I|}Y3Dqib#2I|DEL_hilip8S883c8;#tauihi z>KhyPlYCCE=A70a>95<;^#1NGHM6q)!C?15*XwIn(R%^^f@=Hb_dXAJ zLV3A)CY>p2gX5aW&Q%?Hi#n^s^6g6wY1!#{;epljj+_Yppj<{pY>u#yxGJfLV?g-* z?|&aAz2}^B4wfo`377w{>ctd*bFrN-;wUNh^>E(w@iVj``1FtpbqNCSG3JDsHdr7R zDhd!72xEUbe9yzs1`b>?5LJ2^=A~R&R193}>gr(n2RiWI2Yk3t0Vzd(-~$t9FM={i zWn@rihyCx!RbF0>wK}GQ0~SB| zQt|Rbm%-mL@}P;(q2a~r0ec4^)8vi}-!R5G_Q-|D9kocn=y8K7*$bf20BIr*UU4`L zz3cB+Ll^|&B?KVxFq7A57cHE*VE%O6V#6#*G7eb7`-UEa3{okBuJ{jn3Dxn(E;-@2 zgNXGArX9ft`uJoiYO7}1sZt?L6xdi^Q(39t8oka z4ON(`N>5f66lk*!>zz}4scuPigKE^MPFtk=_F_HQQ6C2`aHpivGX}&v|6p>M0ZFNv zQi0;B@0UMgVlz4n7mrf~nIsa&<@c5$j5$J`F`Dexch5y65%FKG{`=o<-MMqe=bvwe zT?XDZJRy{Uivh1KV(>xBAb|Ar^kB;g6tuYF=uK{z1P!`yf#5d?M}gw7Puw;%G@NwO zNi8icoT}gu_Q=3_QIBYn?!EV3-l+2R)zJTO&Y0mGEdm|_ikf`ZQlS|x0ltZTQv+YV z28LVEFHhs2%!wCUwD#XR?T~ON@YtPi4+TO-wZ7aj4*e4vGpuH{?Hxi~y#92MmOqAL z@|yB9+rG8Px1hq8dGu+zl4)waC0bKxnOe~K_-C;OH+naXP9JFZ2D-eq@!6(AyH<~- z=JY;e|I#u()r#oA7MFM3t;nDZq&1RqiwQb_h}#|a^sB7co`6?crNKcb*qxtxtm!oVb;w|QbBy%hq&uXMDZLv+H5P|= zPq?ta!9ivWyVW>8R-_u8*-$-1XpeU?Fuu*K?rv}O_4Q}K4@STn9~{t_?flxL5LHo+ z+t;$!Xfs29B8J11f+1xLKH!WdV|EVSGp*Tzb6(Tm6>moe7ORcc4875e;Uv?|S)Nnt39 z88Y5CVW%wmB0&O0$jn!bo2cHoC58?AaCg(ytb%`!K8XQd{%EH*7Sz7`{@MwXi*UT< z>i5@GRu&Z$XJr&Q^Cnq4I^2EjLk(Mc)NcLphn{fS(dWFp>N&Vh9vB?x-qY{HsVBii zJgl;28W)^g-MQQI>>X>W<`h)TE*x?P$|n{}tVTqs5#Cpe0cF8G_uPXyJW7NNzxvg$ zaFij+1$XjLB4C~mAQs7Bpgn?I>INE$va&M%^e}gTH$9|aSrqoaV23>{Kl|Cw5Jymy zBS4rkWy<4^KaOjPO!D``4?l$MHgqdEie*$17FjR7^wJ;y_{XqwRFcj<`|Q_We;r8> z1Doq#c;N*kf$y1Tp842gkAV(WVS9Uf0fa>lwpYUsi;|Zk_uY5jZMWTq101>d48%MI z(;OUDu+JJBMPZ+}sA*~nxjAGiqN^kAg9zeLPy!8PL`eRM1OP;HOJ!jy7#QdhXjpIa0=otu+C?|2aRb))ks&GcF+K4$eWsnP#OeHxG zhzd(ecf^lWE<{=ckw~maI)V@>AP^z|#7jf~LzD{zdC?bRcFmL3RGA2qloLu`BB<{5 z#Jk*BivkrBbOLkA(Lahhg+hV0j|fJhLWmEEfd2wWRYc=Ml1|hEC}X`JXPV#)1>oUA z2?8DW!>o}uB9Q2i*+@zkQ3R0-KztNLn*$Y(QWXHeNTVaaP*PGk0h!0ENJ)y#?1eZ< z=~Kk`cl0%|YyD`@0pF+Gge1A`J0F#mBuSMjg(br|&ME1bsT}m1o|>LF9p|pQ?mYhZ z6QJpg2z?l5p(_?7>Fq<{E5)#ShR+*5BjH670Sy_xZD=u|8ATF4d+c9;D-9??q0WG2 z6zq_TRX&I-fu-eJMgWD|Z@(Q7+$v?^IF&6GfPkoslS-jq87{|V9Ht;cIG)Zc9iE<7 zhK$9L>ByKIXZ*2prNbE_OFkAv<|X5&OEMNjrYuXle`;gplI39>8Impo;Zg&FjB?(zT#3fUwyS<~3sw?*-hea1*4o+| z42Em>)YsLuz|=D@FKfn(Ngu5Ftf!~{l+%vE(XANJnqbOUnA_h!_}JsGeYj@Bl~;d% z%jUh$J&W^7@+VKJ+O@M`&yHH0V`8+J`}+qDJA5Hri=p?}v%6{j{HX;6c~3v_DvljF z@#Mp6YnvMzTIbH2+Sli_S1#3aAAHJLT+wXCUtvZ(c;+a4Gb@t2!I(s%Hr`}JdAYbF*Z~(^c@wg=? zF9ziZ=JYz)TS{|KK=NM->dfD=RZ(qO{)P{`{3#<3k z?^^Z3rvYCi(jPM$OeddqRF!Jp+w0%>;qT7HiuqvoP{w$pAzQ8LCVz#l3Y#hz-}H8a zv)EiTF*6)dl~w142V)!G-mCrjO^MEJ;X5ES!@zhgKRceMKFpj{sAmuz7g(F;$!dwD0%gf6Z zQpf@u91sKqaYj~BF@L%C+H0ZG`rv~Puwxx?ATSOzFMs&MA6S>c_@Dz#$NA@<&$!KnH)z0wW`dIFR^X88fCKlWqH#b*;E*r?3bh3{n<5KM znJEbf#7tyYhzOt<(GeU{Aue(ynGj{cIte5;$$vyJNr&^{O1X#&;K-JwLBg|mlq8w} zsuMv9Q+l3AGA?JNDSj$1u@_Paxu_%P3B(6{1~H((ZsO4R0_>e(9XzG7NW?nDA5&f2 z5ItecB4W4^9H=a$5GN+XIp`L2M7@hTAqq(#mzfgziI8+i1emT&hhhi82(>souNp{5xgHY)m2YF z{nRnX918`c*xf2Bkm>>jL4b$QWZ*2gfr%H|8thiUC(S^vBTz#?99O^*0W!z=cGOkL z3!OT2z%ypdV8}PuZ))J18u%})0nrYBZE~S88LKVcr%aF*zABDBX+ibGqNg5yd)FuJ zI-Jm|Qup|Kn!{Zxi!I^Gz#h}t2aP}I#Mu>jPVJN_;f}3_>~XPyJ({{FRo>b}{$Z+$ z^GxGhWj{Mu9bJOs$hZCIIjjpL0yxQo`mHY!{;48hCEnxl4G#ML_~&2Vap#ji|M|Hv zDv5-n4?p}e6bCpwt-HJb(qEo;{SCJtb@-AW|LD4kaiuuF{rHm({(Qp@k7sCHX(5gu zi$`O@p~yh*z^ymmf7@Nx|NSoy8x4B6x(EhClcrXG_SqLV{*mnIuuq_3eCfP-)B5_| z?q2t;e|o5~p?T8Os{0>!{^vhC@4eOQ3-Yp{q5g2~=fx%YgTA2IWSBE|x-SrVdBr;q z-Txvss_fp?)YI7?jzxb0>X(=P>Cj~dRaBO;I%BM= zB$=mR$d5Y~D?mAQlR3dGx;acV9K-y_9o0(dDfh4jn<&G*fp?s5W$p{V=b=qv}_55{bp0r5(g=G*16fb z_PT9*g29SNI7g=oX|;xe3Z2;r?lj87L1}{?Z*#rb0&T@`^~a(r?4H7&D-D*|;yR7d z1XYeCOHwFL{T_2hmfm88yHBDf#w0N3?XTODx2Q~K!47OJ^5L{hJ3MsgOjda)nI!Rb z{Af$gOmx1Ghz`JJg!qfW7#5qnVO2lxRUau8#_Nk*KARDz0FF%fI`01SUoNy5*eRfF zf)Ev!{x1B7>Vl@8VgSsS;mKBtzo%FA{Ov)^(LB9sJ)Ipq`k4(3xP7%lD^+WzPpv+9 z=7Qb5bvSL+P_CMrHDTi`dksZKwIQ*yq4t%}R$-f@#cVxh?$I@QQ?9@1YBgEs=wn)y zzdK}h=q%YLxKApr$@h5zpS@Na3#!eQ{TZ#uUwTT|6sB@6 z#Gy}ti?`z9V(fFl9uWBWVCxBNCxKGq#v5-`@IeZ*6_|klvUKTE?0kXp1wa^UAPy@Y zXlfpO@Ima=sIRZzym>Pp=ZAL~2mOK?)4>)Ps8?`|&d!}Xak2qBNriC-Y5>+B3aC?b9!2%2D;&>}$WWGX zcDR_siD|K6rX&p29F?6d0YN84M2VykUx#!j z>L2h$c3ja0aRl-7z@!v7ZaL0b_*MZv`skx*>=eZZ=T)xAsKgb?6-au7ab!4CNQcWf zay%V#{}jfOP=+)8vE(0%qns-}F3T+Q8jD&wT&6iz8PjE!A#z-%FcvRaPFa3g5*aSj zl;bi^dR)fbKSw$5fy*qHnkgIzMCNrM63RGoUYVC1XPor%GETajGMsbCdDAJ2)|J|X zeFizNj3C4JMN`J5xGWW?fdYUsol$v4Wsz^8Dy^PeaotUq40r~&eo+T6m0bgY51af= zUR5Ze3C9u}-BI`EzGW4mZ&&76O6H+Yh;^+~hlgO*js2n6*h}padSEeKs`&L`?oa9+ zQwyyXS;MVTn#>SDZp~zQ?^k@44g2#`@;}x&CKM4xKw~;?cP$XZ`Spr%#?I1wOeM+p3>ORUNfn} zWYb>w{nKB4<2@|q=VoWlT{vUF;~5z6&6qLa(MMkbJ8;KQ^)RPIq_5Wv9g5!$UE6Y3o;?pYOozl zWTt(*IgFsxt%A=rEW`H?NsTxa0?FY;dahxN)DVPmWVoD5jx&xd6+;*&j%yz6jxJoNxVSge*zUyVm9Oo<34Ecuowyf6;H1@7* z=~(+F4%xNl=ejCtT;nHYR8N3bLT|IHjdq;Chh;ISzJ?m>T2{Z=`u?j}`z+4Y*sQwb z@*R`Ixq`9yfG0X`-V|F=k;-7wTXUcm;Y5@uZ}xm|?M83+Hj5dPsB;~9e<`on+ddT^Bvl$D>FT+d1kAd(K}R9R+@$8(O+cpFr)F3Lq?FLiHF z-2`9iSf|BmC69E)xCn8W(N^0Ajk-LVD5SWI#f#;XrDYjpNn{9Prc1~;NM!^iOFHH# z1O+oaqR5O02bqgZ2dPNZaW zU!r^8_?aAh{twm$XZ8zzg%nPWcWuxa*Uwb^MH=9%dj0Dz?WWn z3APxBAfu8On_@s8T8}s0c;l2)PC-WYZ+h2VcR`(jrCBH`kOijYus25p^ATnH41CGK zBt$tV2fVOF1y`WK2^~yJfWVS4J4GrWaFA0|FcO{x#2^Fj23i~Zpm7oJiFhs13enU* z`DP+QS&%;k879Rs(M2V720~o{QUnul#jgbDB<&qUE~c(XNbe$%NKWPn4yNKoCUgM? zJ%Wl4P|y%%5o`nyBq>d><#Z|qWkrhkBMLMINR>`}0ixB#IEy3#9YKW>kPSgl7F3fY zlX+PTARrHA5vjOCO#;%|6Z80kFx-o)buO2=x`6u#WD=AGaW;S;3PQRh557{f~F!viorklO&9flhjoT17S+|1$rImvzW&NuSkUJeJl#V>DO^#0sKmM+lZphg{5 zBQ!Wj3?UqeHZ`?>^w9>i*3=7PjT&dI!`u>BnEGI56vJFttN^2M(Kw7g9(r&^M`zcW zXC6~jl!en(tHxJgqjGUcQB_rWPj`QQUS?%=k$)(pHRucTag3~^rLkk_@`G-<<=%q4 zyo&N->|S~OwbdAKV`Iyq%jfUeQ(stE)Ym&Wb?St-UR@&^e$*n>=l`xieWZAEzD}`O z2q+i)-i>$MTC#ANt7c-~wl5+>gTCC{z@WS8SQ@D#9Egx0;%}=j%hsDrnh-o!1U2DE zk9S*7-^MlYDG|X*B*GyDo>V*%!U<=3UCe4W6zA$P?C|i7t-zzXw06d=rRCY>6((n< z&XJ|d%7)iUMUvcumb87m+S9Nt?0{dk;m41!(`JP?LDir)JlM2t$L`O=I28}`BD6K3 zAkH4NSiun!rKGlk35KDDv|4Nhd3c|p%g`FhRmW(7aO7mm-lFMKO>|Tc=fi*|8t8LJ z2EDbp# z7H8NUHC2W0e(>>!>$b$h>X18<#DUYAA0=Zs6T2qX2jxtm@44V1f4-^ zE5h5Vsr{rAx)1oVF4>ppGkwl*6(u6GL)Ig+BP?8Ag zGLl4q0W3k3hy#u|A*SZZDMW@6Vv2cEzW@jYNK7mcp>+>Zi-;U7p}-(OGD4H_RGhL9 zm@r9_6r52-5DylBcH59tA z3rQL)O2tPGP)~i9h(bmoG>Qy(O5W73KsdQXDQsH^@h(z%XG9#uVDiax! zPh_|nMbiOH4n&gcjVf;9x#X)1 z=Ug&nna)_e7|vILPpvNOI>C3|yt!|@{wlt3>>>eK4%~!^#u)n_RARe3{q&Q)N0bN4dN*wXq~m$7gCXO}UKe zaE8e8Fkd;BTozd>ncBY|mnqBgFlG7rZ}@(b-(Wglx$Mg42_VZO%PiNJOlRb?&xo}*jsbDH-aw-EM56H6o>G-2KB8x0Pr*dAV!xYeIsuBsn)Ro@T+!Lp zyL|bAuFh_!({8m|&N}mW%zF^e&d$iq&AR`Q-@)(bw5p2dO zD8zvg7@9=SIrqfQ&ORtRDl3cOUmd5@&zv=_qrL09vrd2{S6uV`U?7N-R)2HF4>oPu zg%SCLiDjsOyuuuuoHDMgc+fqxVDZccA6Nl*^`)hS)#Exl{vUf^0w7marCrtC z)z$mHCf%Lx&c5#?1d_1t0wO4&gN)*?GdjxX_=}E;qX?pGI^c|h%($_SY{R}Ufg~hk z-_z-I()+$|^?&D{s++I-_A5FNRJ0!9^?B!h=R4<~_v+QVx9+>Qe)Gm17hQ5jNogTo zikVnaglh4pz}8yR!h1NyrpaNwW(IrBAu#V!W^#^4nRVa2Z4F=?xmvMz=7}DgmMUIl z*Dv$i>mQFqShaN5L-*&;oRz;|ZtLDXf#jr|8MCpvUG3UecR%tltc1$RiNBeijg4H;TQG6B3f#Iud?*;)(^4L7&> z+PmTwedMh4lCp%%+{E0HxD;7UK#H6IGT!}F`Z5Rubpw#|~h)GGYl3rI= zUwwTrJv}}Vze~dvt`oVO5;Iayz2V;gQxV!)L2=@<-P7O5Jx)lxpcs?oYLgX+V3{Gyn%I=*ACueSyo+#%|F zeAR7nNCVDL_Mr7PR;Y^n8&BK6K}1ARkS zcKY;>Elf|%II#N=-c(%w`+c;e^=4$9K2ig^7@ybe&mI|#-ZG-?2MwUzP_GRf68#*a3B~T%!qH@*VEC~hu6h| zX;@4_mwdo2UOrRSk(D`4E>*p)7dmAy)dc`{z6k&@1I0JK@eO_X0`HrkJHz5piYY3T z1OqOLii%LA{HnwrofjCFftg%!3UH{U7)AkEc@7*n07H(7k<7Mq=~69%Ehhd%cLM~w zoqFo2D_5=rfai$4_S$Qe0o_1~)W!~8Il;1ws=4~=tMxO&^+NXtrq%e_&wd6BMs|SK zJ>wjw4FiJmz%Mpnut9(xvv*@})Hs&T2W&D}(%jSJ0GlCFyycMwuDKOQ%JIB}d#35+ zP^dm#P=>>aHf*+Dpg|mJVM>HnHu*-7AhDSsM5}W;Kw}WHlUBcJ!mFTF5WT@jY~tVu z!x4RiLbzlO5aG9M`uy1Qw3gL}~MB0+29(m)e^Q=+6Lh zhRg;foPd6zWHTXztAoI?X{oS9WX2;B6bNmE;mCCo!ADBuFY~CLjY zL@GCM`EDM$X)t4kG^TQ-_1H&zn*{y(<%Yjx@G%QPrg=O zKJn^rT!psZ@hcX>5U{kgavN{f zz?&uw8#Z*-)=mG=WgG9@WtQxDo7D8RHg#6-!h0lvI9Wyx8%~Oz8vMm|6S3#(KYaEh zMd82w1EJQAxZ0NC*%y5zy{tU3uq3gdGJqukYdjMRMTxSHJjNTHeIo_9mnzAuC$~^`Tqj$QlYnbG@!KqrE(;f^;*UwRHsMk^9#|b`Gw8x4kdf3O@3oc!7 z+Ra>ep_KF>h7fIdVeix>6H{`6hxRqkKViyik8NFc`ux0tv?m^T)j#a-Y?D<&u+3jm zfthEr(*ir*-1+eDp9y3prWGd+_Y9_FCit?FI$L_`x76+GI9!ldFrgx?e_)`avuyw` zmkr^yu!O<>;oidom@Fxfh?P3Ek6b>@m|{lu)ZeGRj%tre0e|G9KY|_UfCW(Z?%fMLtnj7@1b17qWQjY0XnuZpMJ?blL#Jz{8hk-uAbEX#J^COZ zqvOLOHBNg7!Hx~FMdjt?t5>hK80hV{n@$J{4W`V%3>p^00%rQq?hGt|P*qh`QBlFA zZ!3r0jl8YFO2acvf4K)DB5gDK~ zyCn#PneGXJ;j#gdJPd^e2~|U|p2EkXg(z!rm{8P3m{1}gU7fJT@M~WNF-er_$Iwa% z@Sw(CLN)z9>M@Kn&m`u(l@Pih7FNV9sx&X2_~H=~t`zu(t-RDj%vO|M41E2WtfR`x zDM>L1acVXQ8j2qB9!QQqTm%)kK!V7`RH8>wgr5k2iFiS&fy%K25YZDFCD@e4L>UQD zZ3+2q#qi1|N)z!SM=EQAyh@?}{w=@M_`mhVPmQv2*_4MlU1rsIA@SkChFcX#Fu~9B z&p&_ZrI+GOHsDncr|MS)-J?{MDgdecT&gmdZ7KRPusn*cie!;$h&|Ur$yMXZYU1J< zsZJSV9gQilt;z~x9>%bq<6%9=&lu)mUtp2f@46a}H#id?(Hn&zd z1U7v4WbZ69KqFy(metC|vlZ8iy?DuN%XwfP=&`9^Ohqv@D`84zihfyR4?Q-<;}Gs( zVx6*vWeXTnPzJ>-05Ud-CK3-0tu2(0RiiQvA>dwm`Nz)x=-*vvNfd@6Y)WX)BQy$A zTV>-%QnqEVS}CF0szw>GGMqI;jCev&zZZP)^a$)o2P)gF6Ke`;TV=I>RoZwRJ=nmC24Gsr0=Uw)}{JHaz z3QCd+CMRSU`T{ao05M1~kw`ElXT=#84R&LReWmAn8Qc1u7K`&PXj17lS_H_(44#|5TZ$7#wwIDdu?<*=v zO3zLmjO(kZ>;A+)U$A{$l`nnhz~=g@ZHK4Np3>RXRa8*aAL^=aXr5A5(caS4&{bEn zyRoCOzps8MH7huG;rxci!!4by(v9bjH*b-|`38I;yf_t?fVryucwKBrBaj#nQ+xvM z!QnH70$Ux;2@Y-bwNwvbilUB=4jq~SnY$Zsi1$sjI|JPcm~sMxHIz!@b*&RFRgD`DG~2gtSIcs< zR*quW78)#4YuPauDUdj1jK*&%1;evo9h$AOg%@DB6^<#RsP%bNUm|hE%B2H*Hp74W_FiNu?|Dhjj z7y}l}CjGwp>IDwP9)TLyh{;^-(81BZ>N;G2QzwsJ_60>^CpTni_TyhzV?tI+jaegv z1^{TZ2r>>-mR>se56Rv{pqvITY|`H$A4U8@hmYTn>A>n5TBcV{6wYuR01LC}`9hb6 zvBcoM;my0M+uFJp&#O#JHH&)uczxLj!-z!$Hf|hELX~%`lRj*%QKr+lSSmse0WkcyG z!Sb+Rqf$1tvVP6C^9VoozzWto3tFmsxQGsB!@m*PAHc$>@x5&=-K~uYY3cZvQYg=h zXh&DEY~XWen)R*v*F4 zp{{{IR>h2|XJ3#pX-Y7^G_j~OE+t!5d-d2-CYDb9@RbSaXau8KT{bRW`A4X842b>Q#eAGRFLCcZL`F zd!grejtxIL-b@qG-p6qLptE?}Y4X7>Eqkk~D9G6uKwKlh51=}dqZPeYJvaEbBxy1)=L|1;pUh$uFcuRA9Gr@ zzS25|BAivF5sfYF?H}mGYVjE<`Z6F`!gp4mA2)OK#bjrufe@~*z2y((#RcE_>g6^7 zVU~8cxfQEyLLB`yWpCAT{YLPVR;`zYP3o|y{1TY_#3D*uv(n)b@4x$xFFp10#(Qs< z?lv=Y1O*C^z0iu8+57$9yY4sZx9yufWzwXHIq1$nDF^}kpZlKOuw~C3*Z;%7;Ly$g z^+vgSjJ!!D3RQ)IGeN+MQrpt2eqyM1_N6PtZ7Yc#6SeKnA|`kX@!8VKz#nu zMe?<;eJv|XUa7UE9FH3VN3ODBSg%y7VGQeCD@I0-fqA^bb}<~G%g-9umZMeK^~?OO z%o>-$Wn+(CrOc!Jt`+;`dai(6VRJ3j+a~)`*=nqz6+ptW%Q>oa)S(-jS0rkvrD~-b>{YO?Ipzq` z7KR0vXDlmuz+j{rvxZ%>p8Y9COI^Hb7{gv%y{lorTFTy4W_{B1B}VD*-SGw^8RJ^1 zhJ9t0R^-f@zu|>P9{u#C_{xP{Q^EiMKmbWZK~%L=k!KrG;1!JNANp8oVHt*=D}cRN zHvHr^Fc8;#xVN)2AtfaqKLI8sV#>7P0G4_T4|I^+T!s z>=^JTWo1vBUb%Q#?yQ9=MI{N@MG2YtzJvgN!{W?akHTf1WmAeKPW#AJnN#QO_^(^` zZF@B-p+7S{5l@U*FaACp@2_4|7IhP!&P-f&++PEq#k>4kF_W>24+ zR8$$rF7zj*$?RY9Y%S(m-=V(##x0v>Uvg0(HN}kV2s3j$l$o%aGpQ;}G11rFHrUmH zN1WusV)=m+6O`YC; zj9+HwWDjJfVj>U882jaTY!L*G1c%tvN@d+g!MI>+ILf2dvk5b^o@HPZ#!9g1p%%FS zE@eYTsvdhQ*dRvSLqcih_B+SSB??1kG;^dq~i)Z=#gFQXn5B=f!&Fc>KcK4;_ zCJ$j^L$uQd{eykODfs0#A->$(Y^K6ugX&2yLRn*>7|#jLLz#~bczk&^-UA@ zsQkRw7{DfZv$WP8RRNhI&PQMb@`IU?^qbUzY7q9r$uYUK`?yCA5zw>c*gqFFq(f$6O-u~PF=W)EtarNgf!gYu- z3k{7O=bj?7nV4|GKY_~-4~;}>ESW&5Y}iXnU-}ZnB@WH00m2*zQo3WLO=)^RObwtx zuY}QQ^HY6cFQjrm6u3B)&I=5>*;`#XF;BcnO!#Z8Obke%QQ~?KzC*PQZ*1K8-LHT2 zxP>z?XBo=CVyJx8HT9FrC*oZdED!#7A6kVEyMziO+?$0{I^2PT_vW^a-u~W+McFd* zlg0xde&5br2Y+(MLtps#SqtY>iehxX9$3U4v?frP8sC7Ldg-AaNUWw5J1sRK$POES zBdF&4x9A87|21Fw)Dbo<8@X3v#i;E01I0+gzr>?RmLykrf9$*8{qEPl{&ftg`sq)9 ziau9VQZuEYTNzer2t5SAaB49q1M>j=<3Il6j2SbM%}P-g^OsojSWK}D#4<2e239ST zH9?}^1jn+A>0%j(W#HIkAR#5S?92;_SDZZ5)s7WLL5d$;^|=SKPFstw-o)JegkTCD zXQaqu%b&M+)s1&z0QeDmI2p`kv%{CJC5(B!S-IK0ho9xyTd7SCL0Mag^> zvNBG5#^k9}&Rmf{Z(-Vm(qL9`d{$O`a%RXMFCNFY;jNR2D^?dSI<9fkx}CrOkA}Bi zt8H#a*99ID0|~x_INzW=%V9VMo)Hm_{F)XYOv{-$Yv#(;ndMW1`Ne^Z{Dh3mxWqJn z0BfMgg9>l*?9tPW;U|Y)ThqF87Z!Vv(vVEHhiToyNIX(&$Bp&2r{*XU%uInFJ~6p4^q3G7-5L7&2@V%5SfEpTjxvdUh>FPvaO8H= zO*iQkf(&D|k+aV}8!!vmxpSv>XIKpjmdy%Ws0{V!x^ zrPkE7wX}7dv3kLR*%jyo!@DXEKfP|(&T4eNopj>7;}%YZUri43(9zlR`>q z1DL>h$&%@3p0WhrJaGExGwZkSsK&j2#c?xEUo8vn9&TuU`nk9E9jc!>rF7rkLuZ{L zN>fd9d-waATRI+lcEi2{b%8+qiA!gmaNLZ8wT-{K|HXlj|B)xw*4DRt;Ed(C>g3~t zFG2JowR9Y}WClJ)L%!i>U)nr-dTCKX4i=|>?v<@;-`bOxlaA3GZC!mIKJSF)mQFN) zTUxts`>#ikjnh{zTrz*^fttGC-urAreak%$K96gB-Yn_Suos40&gMFhAgxwEv?!C# zqazo*RAb~JN{!C+(2%(me_&qtz=ekH467Wc5S}|j!$mG&rhB#LYOp+#o#T}fWnfO1 zjTxAEJXS70G%&=vNAu^;$12cx-xBMd;O#QpJ~nRLh!^TGGz+7@0K9hj!LprByk z!i7_(PMtDk3dW(~tw>xAhzdNwaCoZ3-qo{)YY|<~b!1tcx;(1!(z7p?mCeN~n<73TEiIUylaiZuk`HgS;5k?Vmv46V#j*)DT&gmx(5X>$+)DiGZEk4VzNKx)mhQT$-ln=?3rYLRt>LzcCaGsaYm1ST$N zVq?rd#es;$E?p6aqe@nC;A*if-i)5wB zaAXCHx3v>MS`#I=hw2E2%nYS!zy=b$lE9_V;CFBQ8V?wz>OeDwYsW~Itw#uqS3Mp? z@uM1Ec~+B@lob7q&5MEcDl;dSDnClKS*YzFi6Oz?kHJNzFCj4)2qeiA26*cp?W3l~ zhO&xMv<8ut_)z@Nz#x93@9P}gU0v)919E%8Hv|k^bn05{Nl?+jMM;B z!@;oLR8y3Bc`Li$e_6@Xj zcFvhIX~X6{SAXxnzxw$PqA%lMUE{ZY{M*TsicURY?(ZLX5sQ?4`|6M4m51Ly{6a}# z_6N^iQCrt=(=Y$`x^I1aLQ&qeKfSlDv-=|-Sk~O!h7K={U&rO)F3CGRJ$VeUTADB8Lzp^wxDJUJps27Gc;3kril+@qdGjqv|@`*)>iGhQM ztJ_*T)~(;Za`~KRU#hwNH;+_ID$LGG|A+tgi=zDO^G;vBefPoZfAQe;*IZGUpCcsZ zYjN|As;~d>H=n-z)N@%%BGpCg!Y)wA;P+HqMKKsu%&z)I$!qQp$s_U=6{r=3%k78a7Os|`jp0r?YW#yz| zJt?NI$=}k})qxf0i)5!xL#47X&9BFYjOBN)-qD*&6RbWFB<#v^8b7;Udc6 ziMVVlzSWz|>lD4s zL6w!Gu=l{6#$lWi91wbF>qRCHfA1#WvCj^96@LgOyfxCH;C{BhLgNsU$SUyn5yD)+ zF#qGX+n+9YB&7FdViOZXeGQFLH?Zv?f(JLSJwf^@ z7G(vYTIgXiOX|gcOkyj)`!2!+v;acuMTEUTR#dj8prYQ})>vPTl8SpKLCg zo_zHXik^9n%=aGBja8FAi!+*K^wkiu>U)UPR7TXA>rO2U%dMxO| z)9$!PSie3ElXo-U?BzA@>LcTQf~yMI9TZ|@TR+7=ra7&@!$_p5fjeZDB$->|Qh;krzl zk$VR?H2jj`_s6B&BOIO_C@XHvb&7}hHag$|OI`zhYe0wRLwV3g)`{$KW% z8Y_Hm!n(T^BA%~{Yf}54$?XNgYKK95>Nt}?+0*3OS?Dr}S-b~K0s8XMhiiUas zB%PH78Uv||5p~$oH)jaayq^2OQ%z*mWS4>c!cCx9E!i$PV@VAY0oUi#)TPycMcTB} z#MeA#2urYrgqw&&Wr)-EtCgoqIg>#>M7c)Gu(k|T3Vj}LnTS<*a*4*r{w(TQAUO0CBh>Eqt&E$f&Sd}sz1?naXk4V{xMXLS zuOXSuGqi7fk2~9e3#3rHeoH9^h_$wKTgeLDPYg(-UVo8Dkx0+Y*$BQx2jd7jD-~_L zc_Fp9K=7EZaVzm?TWyIJ9|RpRh&2lQ34SF(L0K~QDQ4}3Ak)Lq61{m|S1X_6 zWR1}P-$lLMHk1-7e9cKS&|Mii+Fg?PnDgsZVfq0f85UPWBhpcX#j^B8h+o)a;^y@H zKPZ@wzc2Yt<7IirhLrptn~3lt6ZoSy`UZZwe{@^qc*@jhl@)y-<&e3sR8(5}N}GV` z0QZa}eRuD7beLX<*PwzhQd}C}($`-R`W|P}jm$KuzZDYwL-~M&H31a!#(vpu4Do(RW)+%ENjAvF*(y8w4vS*l zijKv>Di8?NnK@$%+Y2CpvWJp*?9{9QmFP6T>ob~alji&Httr~s7++C5(k~3ij2R`m z$IZ>%OhNv9zcnZeKqe7wNPEh9?Ck8Ubhp-cFRZ{Z+qwFQb3XJZaS)B8KP`Sa*=->d zARfZ0 zXeoa}zJ!tY+MS(=TX7E6&y{&PSPPb-POFYfClii0Tgf@Q-mvo|`|C6EAqxZDyb5g^ zk6MkErYVjne2&8_1T5bA$W9Fmyiiq@zzWztW#d*^x0LoiAZc*_g;8MLxa) zGZ!U#dGbX~(vrdStI#}6Q{U zKoF~rk6$`(hTqYz(wf;jAgYzP;JcddZfv~#*^w)Py%5ZNk09Ed$5e<)vaGx}{PV)S zALM)G;XHI8b`x)L2F|5)n1zlE4>=wD^;1?=y0d~KeZEwgraii&tooMsHx)?ogyC+- z*?z?z^Kz}TFN>|qsKU~WFB$RiG2yMPN4!#37;$l{&)Mh3J|^~rsw{kj7jGW;sL1Nc zJFDgvKXt1vC27?Rzw1(TZ+Ia#^eKJ50{*$tCLOy9R<23R$+Koc?By5to#~8>U|m)3 zB=hW?954mZo0l%hNlJ3Q(qD!HwfOEZ(2SX==%5%?SaxJ?N7Z4a_`Pf z8|vl?Rc!5`%?PQBLfl12Mwn?`ukCwbxg3i>?@bp(r}Je>yVjm3t@eMt_0!KWmwz3f zo&agofm)p8ctrwb^na|9KydR#=JT25K&1eLgS*7kN)fviP%67o-1LO}P~*2Ed9g-7 zvVd|d2rmKeIjXr+4Jz9VC3HbCm9)tL=41HX*^tv0p zm-e@OulIIuPFDQtH}R|S?v04>Wf=J$R2sF(-FdZV*4UUU&6bpuoE`wP{@|kEYcB$8 z#%sNs%x?PlG2hi-Kl3y?i*ANi5UR*WDXc=IZAyBN9adv*74{p#315vp6a75I0?tMFc44uRm%he6WN zmmvA$wa3?i#5xV8+ODBsa7<%@K8pRHmNSXq4!oDXQa!{tA zjncyIn*O`!=d~PBk@?xs6#gyDsi9P7YYz6Y)T^k`q8)692T0d*ynDoB0-cUxj*9Hh zT1=iBvE&4U=@V-ad@FPX*wj)Z)aeB2L0S4$t)N1SmOWYVf+eO{ryu8qD+7ST(V9%X zSMqvnL7$-+ud`Wyh(ELOIjgUuHRdiy+7_R5iW&fAyDe{?gz=hd!RtVtt*lsNv zW*4b7LLq$YHubi;{by>swQC3Y9wRQSKme1h+3K}wf=T4PFxABs;=(h|4E6a|@^!Sh zb+XVlJv6J!r?jn*Vs|rR5$Y}vfWW)mlkQH|*@Soh&~#mhZNTgYX|gF^E4ZZ< zxy>B*~3884Ml> zLU(ic=iOMUqllOiFs7ybSHlr++I>m{S?<7?kU@PVI;xoP_TKn=p|>5%87FgojUNTR zvn&CMJ7_HrQ34hP`?|j(hpswoich$s)pcnD%%i>;TL)vfz+muTg#QkWEA++0=;$#P z+|MEZhW{A}%6@^I`uGPr2OV%^BYZ#VlNvH8E6wi5OznX&LSk~S>&3jp?=DrZoywA@ zMz4H6Y%O$*`ki$u@v~;0zyFZx=VI--G{;uz7JoBu#(Mu@o!-pxWqBWF`1gRNY z8;ZF{oBVpO>PAKBP1`ca!FIgPygAMmYl|prD9E{*^Zn>jFS%^|N5RCn}BbAgAAKA=f$7mR-^7l`! z>Wew@uLHxQqF(2$tS_Z?@LMS$RWRe;X}B|AjtW@ZlqN!)Kaj-P!pb zJMsV_-M^rOWX~tn$mopECF0L9FTgEAUw&pr@saS*2I+!dIgy-diam|{G&dY75Jq?| zFz{KWDWGc|^*%z(4{w*zxJe$8tMWyaNC>>hzrnt`E_Xn7#V8mX;@$gJL4hOaDg71B z(X#fvdF5L?EFQU)3KC~mjC#Tu_;qb<&CK_%ZPIV%KZ(W;N_LS3w;cS8N0rkED0?$e z-YeW@$J9*&4Tt7;V1ZQXc?UQFyi1$0QVhVj7nsUVXu130mvPPLKl)%UYkgNEDBm0i z4RzLStgv&B*`4(KcCHe<12(0ErkTd;t;Qb|7B|;dRd+uBG~Rr{a}+=;JJM4(v2})5 zV`s!?1aZODbFLQ!++*RC_G%aJ#8d(kbzD5(Tql-7BtfG1jh9)KG7ehevkH1O-aC!H zf6czFQKXjtD9-j6YtUPNrz643%}a$bFH$u~3T9KxPI(Ufc6CSa9OmrV4Ae-h?Y216 zW6w?!(+#8x#2{~5gv8h>SFYEmv~4GyzthUSB@>&fBTS{DbI z*PHnv$s1%&K^MuDk8Con703Bdo^+XOvj*|HM9pE~d?GEks(xku6OheT6kYvGFdVTF zFcJSgK#smfYz3&{7?17H30=#0wOf&{smd=z&5XcP%jSbDTx>S=RE;VfarIq_e~_FO zGH1u^W0*>6(7WDrEwo2NYH<4jyWdR_v!}~}FS0~zX)Na(&h ziW+$hcf?u$7AWf&ih7s?zpN^a9|GzaMd@dT$@peWa?S;b+bb3jkM`FmTbgu+{W$Q~ zEK?+Xn01th*%Gdu2ISlB7zCw5(bYg{HZ;4TG_6$(@tc@0OXd6Bc;GswnnOzrKhz@1 zUzkr_*kDersycAv;6s1ZpgaFqw}shmINVozl^AnMyVMPZ8f073^wY%)zp6S4=c-Nj zr0K#Uomdt;rjraIYY7w)c;{Nn@1^7?p&4{|Zj%w|GWNTAOiNQI1Lt(fNee|oB?rBf~ptp#5u;ZfpC z4ussl4?AeHX2VB&3!1(VC7hgHKmPJd@fp6*K$-wUi+7gbIKB^y<>c>S2Ek?ygbaTh zbE9{s)XlKcUO>K(;&zREvG%8wNA1;!`l5ghu(h?d@n>!Rh-2kr!Ab{ ztMW&R=!_W$SxOA7e0wbJGeP_xNWl)npT?;%-rxHkymgngTD=ALx>%XkQkk3SZDwrD z6FOl&^0*gyB_GvNdSr!LYy${hy;nmLd5_HB^)F_4IFpw#TCKyki$3;bdTJAbdXi`zUu8;N@TVwvIlY8=Y2Q#dc+Lmr zfod+l#=3^w+k2u*SJCDm;sp0_BoJAet4_Qmp_^zk`at_A>s%zKkW9ae`H@JU7lGNW z_Z*Eoni+yiVrH((;%!Gil==DvXwn{#2p)?j`0Egw(w@4&=7WW|98vA$h!J%AQnYL% zdq;HiQ#0DrBJ3x~z?G&Jk5<7yzx+;SZ0;}eG`k=(!;@IYQZs=gCinRyt!3h33T3c| z9f7f@KS3ufc1(HCRI0;}z#I^O{rsNdB4khg?sd05Hz5ZL&7RCGYmA%zEx{}iKoGj) z1wP1RJbFW7!1nepq5S!WQG11p_%_!M*os)~rh_H$ zM~|E6MVlCuqMa98OSEuN2jONeh0CrcU`g>Q=gDjD{p504ESn{>p%SVg4cIg>tx?F zp{%QulL11^#)RV>9cr8>?YiM_SKWN#DF_KM1Y`#$5J!qnUeWBR3L5)A?XIqFCAT|~ zIW$#paL|H{QcCFMbX(pPot$15FjDpvoONWcWmr)%yNr#RCMBc80huO6{2ES)@PHGOOleM1k;lZrSv01 zBQbSYs4X07lKblLO+TN$T>Of?twS}`H@>-OxCAvCgX3`lqI}P4jgo1_@o*enAbnVo z2@It^0#;waEzO8^wr;dD&q!XF682Gy1q*O(r2}*O-Dg%2@IGh#y+B|Lx z-&AKI*qvn%POr^ulF+zhau7bD)DM)tji7=GJyqRZS^zP!Nec-Vmv~;QP-!7@+=l?n zD}qNlBjnl<^5|KMB|Q#x0fy)ZCvjRC-`(v9+G#Ni1kv8%A({@1kpK++zb*7vnRWf5 z^#$wl)QPMnVzR2o=`f(2;-Y^;<3V`v! zbsoH$|88K7ho~xbYpO)ZZ90Ia$4SWK^?*$E*qJK$0F=jv)-}2-AFC%k&aJo|#HXAa zLla&7F#9R*ShN$-E$p*Jnfcyw1FqHAB>s+;$&}x|v_M&_92NN~Gj5A!frhihuX67_ z=pn}Pe|~rQ=c&>jpP|EiY&so_@noOFRV!P}+t4&xf~JQ*aQ2hOuUHk()dd9Hp}88y zEikec&rb|H55!s`pOt@!j>Or0)Zig_<&aNVaA|B*>D|}8;$lp;mK*SpPUO=aP>OM7 z7SWjio%f_l!8FtBM^S%t3aONF88x7wo_5rN`pch`d2#-48DB-kaV5okH7Y^*o4`p} z!ui2l3a)US>C-#zLC2?dX21VA!T>%P4eW(OtAZTI$x)WzXMf<7Q|kRxzp+!v-(+Ns zEwBFfH}6t!DSJIN@`?d9I>G~f3S%kehkRq4HZA90B!0>~KTT-MCp{qL`$n5|i&93= zjQWdjq=$0E(-NLbvFN$Da=N$Ge{X&FfAcHk&?0Gp$@1$P@Er@CTX$3*(2_lqQOM#Z zp#R9>`K#n%in362QDNcGpFb1~VvcKR?z6rZe`H1HJEc=}_4}0S6mUO(B z4`*F>Odo!YV)-3pqGsM4dTZ!irm+csiO0$<#p{<=-`cvFk4@hOacmDa1!?h7Q~pRR z4GtJ%St04P{9+=i&h#LXjGh+p`%Z@w_OD`Tb}vVt1a`x;Z3}^QVayM z#>C3Pc`cSNT|6rhtssX6jq*o#@61}5aa*iHib8C%$*@bNG`sD)P>SvX#8~)N%dDnW zG?xpfLW^JphCbP+0H_6Yj<1P&i}Xq+uddRn*1(Yc5k<5|y|$R|ePxavGA zP0ht#pSR}l)Y$@Mj=`Wt2c6vxtVhB`_Xzj9?Sr1=Fg_P)P2XMVM(34oQ2u^eAAFko zRm1bfv-WpO^z;u*&GZ@cToNX+b`FXz?W-`)v|07bi?s@A*?M}OTdK_;*!#MZXP{2Sh{ufs`0mz20;;+Pz8so< zcaeqF(lC9Upu;VdDes1{L1RcZ`uJOSof4ZgGBa<{);_(W%pGJEeMbAuMC7#%!`tYv zuxVH8-?gZr-^~{s@@Y?GdL`#77#c^v2_v<2m5ltZj;Oe`KF`HO#-!NSl7Ip@LQeQ7 z=z*A0!Qun=XdPrb(W1)vs5=ns|191U7#eHLM4k0*{ZhFyyJ%_B;Jx4Y($G-;MLFcL5(=oHm98UO&dq%XN;!WpL zp-|dZxd9^WWDe7E)RAjbzPRl*OpSK@hWGxr$0HXpu8~j*Kz@m;z>+mY@%=H#QsH+; z#wS+=G!5L(^kFW#{FOtiF-I(&&VTVzVl{ZWpZQY!yge!X`xR*^%5eL;@~ zQE|6|(8Afor{IKNgNt=FH?Je^vp^^6MOWy=`|O-Tio|bZ1q@D}%ew+wL{hIji*tHF zjSQl5si_Rg2NB6`W$Qw@d%=%N9_G@$V&{IG#)aZQ6_rT2uJ3)#QTk{*H#j>RRge;0 z|7`uhB+96mcVuV?I19W0sc^gLYBfIML$%~b)5Ful;cottk_kMz5G(bGihfA6{%KcU z`eoA&lEg-MQ_!&z6AN+5u7lE z7oEfJxxZ|5xk#Ot4$RnPy((H0%-d7sJwkrNoFz*c{D7Ym+bX{)rQB6vjC3Zd&QhbR zX_&PqHS*)*!HR#qrBHp&;9hz}Ls!X|)uiTcB%{P(!d=8n8)tU&FLHx;{Kjl7YhO>f^g?-HLhBsrI+bcdUm0bQe>2HbLJKx~3^i`irT*5TV! z=WfoD65jQaBpJW!=0GAMBEoaR^Qoyx0e}9T!s`pNvY#)H~tj~Nm$sMW`R0zy4#wZX+~XAp@`#l!6~krQ{nKn$%5j>hx55MjGXM9u!w4P ztte>W*@BoRObXc8r)5-In$p*uPIFQq*HwlC?$8g|DlYD}5u3}>*>*20joo-67>qD} zB4Noj2D|I*yy1~VyTn!p731nlp~l~unY7|O$j8#5`l+|5cG2bT9ms){Qcs`tz3?wb zsgFQX7pBQ}!XY@D=}AuTn!mU>kic&BrItd4FU2*PEMOSwXno1!7D%ynvYqDfkc$S- zzoui*Z87|T-L^eWwTi3m;x(=iEE4zC5X2Cl!u_x1*epSt)AQxaGAOPaTeTe_Pz+~U zg_CWGvje7EtlS}e>;Z^?q#{g&nB)XQmNb&HOu4DM=*HeEow;<+>h?qVnc-GeoR962 zg{+3QCn^R%o-VV?h_4=IY{neycC<28#%(bfe#AlPMh+fIOn$GIiFsF7RaM=S<;u>(XKZP0P5OMcdvpDWm$w%^9|zLV z755x$k-NgbiXv}| zb~81~P5AkrE=3)}C=vYi#-?dnTPjRVJD>#rj{w9jiY#cYZ=iwvD!&e`)rEP$F;vvT z2aQ=OS4Zv)x_@Gd?26^y+620lKCNViM?`!}(f2~sITgKPCa}D?xJc3amZy?-w+4-N zK6+C`Pfs^^%YT8Akul8}NX9%C+Ntl}y=(qmo&%_J?M>RCOkOBh$gPC2m%c_35hCUr zo12@9w{5{%e3Zn5Q16Xkc!tk&8f6BC03&2sM5}Qv5J-!=z}}?^l0ys>A`^sDX?Zmx zO?6rk@ioKk6(CUbU`f5{k|My5fHm4fkZ{#8RL|0+V4!w^>5s%(aTD&Sw)lA66N61b zM>3?jenRp~3++}2QcbF&2EAxVC?`nY!v|IBGy}v*1nIXlSJjVyAUa%vN{g3t+=!p) z*FZ{*v|BF+Cn&orKS#lh$`JM_km&ti3*V71e&U~ZJgPXiFk&FtGMtFi5-v4b)kwao zg=|OQpuo$3)N&BN7Sm8~j0T$-GgVn6)7Edc{68#@wY6Q@=N~#+Z!bn9Mjl~1 z5K*C+IydI%cvAzTK|9)h-x&`vy;TiyiD;W1G9Z!ag6bT1Lpu8#bhMBW8f06VFntg* zMOIF5cEyBj?6=af>AYC-A5s*+Fy;)2feLGqC8DWIW9T!BOhm>cZD=+Sho)K>Z4Cp` z+B-jUD_KAd{@n3+PfWHG{-%PKx!A2>Z#N3(NT>~q*BPyLLLtB%XH48 zIfsg(_|h>IBNxW5ge`WKmg9hK@k?Ln!bojJ{!`PIAd;Gz5B&KIOM4+%>n!M~ftoEfXgKX>Iy><4!QC>_kM^`X=x0*l`E+7OxLi4RVz*b`WCr9f|Gs1Y zBq_ib_b>hckIA}Aii4*P!?GQwR5KQ20jl597Va)^t|CkPGRIR_SJyN#J&oLuv9ad& z?|6C89eLWE9}ez=;K$m`KjGyq0gjedfq_5%$-?)t@RtUj-d9bQxTDb>Mk1*G^}|4f zfBFL;iljwON=lrE)W)J8`rVADMI`V*#SCJS6PA>=?6xIo==R^GNZCy+t`vmPs<`!5 zx>c$~s;rYQJc<8^#jL)J5YsMJ(~SuaQ|*J4Vojot3=bC%$GUX#qU<{ zKX6R!jD#!qJ>ZGKhaD{9OwGK5-dz`_Cu<(JcumxLjPkShbg$+Nsz<%I7q6fRw&G%K<9->E=7AnByh2#Hf)8^66PqF6DVv^0Rum+FDcA=dSR@21s zu~)TfHMXVH)LC@$f)$AhrCI-3(E`NdwV>JWtJ&%{s17rW#%`k{L&YTmdkW~H1}u!p zJLl#5Aoh&Op%$x77J!ozW#WzOYo&?Dc+hkSo7S5qg&s7;@DIgj9aJc+n(K;VgzN!b zE#H}Z1Cg ztSrRLkp<55D+=2xcu;l;xGgm!ql}CU8F*i+VV(PeMQrVPZGAlio90iBiZzZ_B_|tb zx=i!dJ@co<`=^SYXgaety$ib+<~l$jC@q+GLuE!QtT`(f3e~>tOZ_Z$Sah zF9~UJ;HP$r*m{>$P|rQel{IRMY`7~$B?h+~q!o9})b`zxt)|8YKmlL{mwBRl^n9AW79`@~OQ-W^_#-;(s^zpG||gZfYT4EW;5 zLQCupufBR#xR6fO_qpBgX4GhNeblDm&Pkw5*&pj&WBw?mBBRleA;t_rA( zv_8vKNjV3YC8}u>R9h9#Sy_z-M*;p-OUo-{{rbjEhXXo^K!}a4^4Bjob=6FrIo>bb z+B-H`LOmMaTbq~LtSY#B61*MvFBd~PRdtmjqR6IwiMbpbg^-7Ag#v3DlO!W zX1^BSY(yxyfXswwmBIyxgsYCe8ghfky|hKNv#N@p z?*Wp^03A~qeq=F^?A}ha=~Zkm15s}*im#;-$!(?|F_82i-GS~9h1IdGjF|1Q6?2vB z+=T-67Y_%C-&&j;d|}@|Bj^aMhM0rAUo%w5Zst(%H_i5!(c$+lfWm^vidzwj2)nW^s#DR$H#=_TE*te8LVMkN4 z4}S{Lg^cE_aJO&~W}xZ`*Wex-J=zik<0m3EMk2f>2$RJh{=3k!CDq1iWca&^tOyaiuZ@Tyi&{}Lv_#npAAUk3$UyLGrSn9w3 z{2wb$q(&=BdWz{1W4B88jjq>DW}^B|f9dZIxLUf0XrpH%rKO~#L`A84zO}Kp1?J`C zP_=S@+#dI*WN92<(z9}KxY-Mpxoms%?)%d+hdJmvHy2lCrxJ(91VN}yv(uj{(KVjb z>!W@H{|B^{BfR}3K4x22DOx=G;#nSV7Tb)CUMV;ypv>Q@>HByoIq#;t+esFeH1TP6 zp^$RB@wZc(3&-#?N?Y93osre-M{FRtfQ^k}I{!^o8 z=LVMzyebXHOh&MMfp^MC{{K`S{A*g_Um2mW~@e=^P8irAf0VreP5kv|&rq*>}t z;=yu z$;t6*K4pc@&}+5~u{!tf@Xq5xl_?QBU$b|fo^riaqk&WOLoTD|Axk#LqOcse@x3UY zEt{LQ#>Vo7&ENnuc64(G^6QF~9UT}3zvD!p?kuM|%ySiWPB*>mvTUBh5Z1#Z= zpy+CaFMD%+rKRkr1BtAv0i`}I9&WH#n0>qAikgb6l$UvO$@HnXh#b148%Z0DWzzg8 z+>OOm0}h}ndX6fc7+D%)kzG9mLOgj7RhUfWMwphzX>=H}UcA7>;HpFtfF! zp*W6ZMv#FW>@gBx=7_Ub(Ov4^PE*1hZpJ9h0K6kKpeK+p@z!U;TQ6zz=sm8uoSzum z0V!NluQE;E>Jc>4`M6|a#K69l>~8^cSz7a;>;iJb=N27KVr`Qzq^9PKRkfL-<&(vk z#!%KtkphKqa(KvAI}OamLX_!vaUYx>14RI&Wd6Vo?7#P;PS_%nyY4a^H&mp+=VoS1 z-m`5;tJ5l|&-wMB>p@|$2S^oW#x&vrkd+0;5-mE)wmVuI#8gMY_$X-|lK<8M=)tab z^y4_b+G)bD9_Gitfr6)?3lK_=@Y9=3TgDp+l!6i?lQn^B2l2@)(Qf3pXPJ`!*F6|K zXw89M4jyd^Ah5tmu$^YpjEJ9j8I)8<`XB2>A0Fo@mex_H$1s?=ocm zxEKY8($Vr*I5x@u4D_ls)8BWars)T(L&MqgO%Vu02uX72E$ynWp`nqTo68{fK2rA3 ziIouFPAW2Z);h>L(#{vz(-guNX&m5pwXl%=JV%8w7^M!QH?z;U(L(aAd(APy%hsLV7_bO}H$zAp!=g)|9S z2Oeoy6!Zg7Rx90hc7|YEadleusoN2DhRG8;A6eAM!F8wWmj>OC0cPtH_rnYGmdxMB z`>Ho95!qH

oeAaxqLNx=fvo1(2%xqfXtbxb^!kUt`aA^x$k7NP+WdMc--p|iF5liN)zlsVzS~i&?J>kPyJxyw!XMn;OllH2f zSC;`OVS>7|10CUx@i0kJD5)31en&skWr{Xhg{vm2C>fHm7)JlIpjI)Q9Kby+j{6iv z^+u{av|APVG^3$jRi?;I%~nCP>EAWOJ>de_`!m#?UD!dJ?wswOcth2wP){-6vF^xN zpYjrJpas>Qwa^4c&778@l-vp2V=uL2@X~%huf9Nv{wBJB7t9e7)7|P7)fUlw)JF$1 z1~S65m{QRH?|QrYDQj~Az9VK#iDxv(KD-UD^VYt4&jWyjP!>QU!83pN!fU62h_<<& zCW(zQ$AcuNaPtMiS6xrdR6*r@^)G$8-*vDtf7yClf&0y3G8;FX;F1;?jPcUSVxgES zhdeT}f962zs>;jVPIBDf+T-96qf}b!b@f2;Nc+LsI}pvcbn2wbEYy|dcR$DDH|jpV#G^vb`6C>a`K@<-QClpWdRBDvgr4#>UM6ury@U>hr(q20#+%WMSOfFfjjH zdvtW?n4J7#ajCThN**(AkUCjw6R+ zFU&6#ysV#^djD*Rf1K6+$l^e|6oMb(q6fR!O4A8{WYpPUpSYaQEZ3oZfIjj1S3k`28#;o@)1^NIG*;QKj( z@6*fW@t;xm<#jhQH=E72h{>m|3baBoWd_>Xq0*9;juOKKM?dl&vJMoHm{)1~qAd0CQC;TN3g)EswEUqlMpQTUo#eSMB z$@%l^uYTqmPnN#tow>HfKszd_P5q)d0G@7o_6xi|Y4J2mz7oVY(}C>*ZQs>7g=&N- zJr9%m;&IZRaApd67IAu@UxzP`I%Gmj1omG4K=Yp3YMK%1Zn*N+ZxXR(A z6YCOg3(eNAbC(Tzu|Dy@_a4w)mAAD8#MNR$Aucu1G0$Z{6czqI26eQwp7Yk7iXB=Q zBjx#NQi1OL$5In@?z;{67qxYDOP3^>0|WkrvR-3FHL?wD z^CL2K=$hOE|2dh4og6zixtVClo`?J2C+LK@IOieGPLNK$BmeNh>hTywvCa7U)KxdB zem`Uj%RKFLWP>Vx)200S)uXE}smM1LY>yrieSP%sgEq8seP(z!Qgl}A`gDo66_79s zb2I;&c~>Xp;{@8`OF1F0gkTwyou_Aw$Tl?yS`X{}rsg62ddd6x^30NgWgpxRIT%!6 z#sIyve3(OZEJh8a?)x0D zRMPhEX2feP36m1eIYLU|<>!5UswGPeJ{SCqWb*X4K1oOM>Q@AN`%nRB|$-OHJu`eMe(RzhiTv;G|Q2ad6X6C}vJ@z*C@4DkNGr@iim`W0x zvX0KqoNNNnc~iUoalP9#?oY0kYN1W>HBRO#Pe$5)VY7hwVmW*hBTO;ZSdJSA2%{UsM#b!BD}WMSMJ>K^#>&csXb>HXhWMVXrx(kVpB*Moofr|Ri0rkjc04~FOi60UVS60UF#`Ln0`CK}^aQQ^#*;j-&5?%*xu)`Mn~LE%a<{*ucc`+O z^W|w>Q8`GB3E*IYnI)fP0og)KcYg+{s4B>)&|O^9>;zO_b{aOzzJDxQ9@G;eukldDQ!o z@zJAF2ms2J@{UD#)<46wR6sFh?l7ik>?{@Pa_6W^@wf-7(XYs>Nxo~AVbqtBSWFA3 z)bQG9z_Po&qs9~ZQ8w*ZfCA?EFWgaptzRPs`bWBO3W@QdX0ZA#xflT7q;E6@3Wxx^ z6a?8CVex{m;(l+d;ks|G&o93bnFK{o{w53qFO5bB?0$3&K{w`k3H{~gHxUzpXh;QR zRO4Kv=#D)-?^taVJf|K%ge*_NA5nDg%|LHg;JFi z(L6YNZ<5im--#6cZ?d1qLY?rS0CAjn&ITv0-~%^2AwJ>nRHMUEN2FaziQQkulgkBn zY6I_=E}9Cv*$vsJ%cUL7FE}=SaQEw-_sY3dM%s*GrlqUF?xymZe|OwIpb}Y3#ynkb;5hpd3dF|6|6j%%1?{ z{~^Dy>noem08C5DnKZeexOnxqOh~IGHGcS>NZS!G$GTtY1adea>+$q=z*C**LR(wg zX^dp*PbWms!arhvQfqg=Ie0J8cb`GG2P=YaJ)yY~3{N#HlgrxtTs$ce&&d=R? z>@{~Wdv!8rWXMa3n?`JvXhS_$QpH_(*n{-#Ywj?L-k+z0)8gAz3q5ZH34XFjRBaeM znU?V+tSgs}quXT`T3LwKi@cniz3HEI#~XFhwCb-fg>`-9Q7Uzg_Nx68@rSst3c2)l} z%SD4U@e|28@sqwZPq6zACU&Fex6N7qH-F~a$k3KI{=92(XYi7*3(gc{O^)Pb|MTM} zn9gG+>ot+5V|&^Ujj)8XY9;ay)_d6az%ecC(3Px&cF*f?bnAu+8){Fm6jvJt|$6h%Pr( zPuD=FYrXy|!dg~TE+@R}+40rFXL7v@*=xer z5DXTleSMnfcb53hXY<4QOT6oaP-d7rp@4Y;Lf)OQt6%zJTP;ieDRr!F8OaT>p?))N zV@@aS)JAQesC#9lip$gpje!R$Zm>D1zZ-Hi{NW#p1a8KFLqKCWgx=m`H5vYdwBHk; zf?c1j$;$pO03t!%z7lwR-}~N2hL>J?34}r}7}^qp6n$0T%u(Wb=JTKbyjGG&{K8P` zVNhm`1OU4l4-E!rAX?3%-3ccIlo1;Q0?b-PmgG`Kv{?ECqchmegrkXp7>gDyl2Tyd z+pm4?YuE)RhH^j_@5GQNUn*eJuV{!7kI-vbGti5!ADydZB2A5E*0M%YPX;a1uhvl3 z@#tKW)6~$5lq)?nBexHjoV3;aTGr9(c!Yn$J_{WaOLtjkOz*!!ieCt8>#UT=!pIPJ zTt+&>i$NODA3$>&%V zkEESw6LOl~^I^<*Oj{kNmJzVfYQ2s@01z}@%i5NDt;w>vzT8Eilv8MQ{5ng^i~(;5 za+IbQ0yHe^-R02oksKF|H#toW<7BG&O(Zp%U&~~mM%Rz#Cxd3zekp4&n#c5IGE+mO z<}vl!*2HUDEf39;k5lW(rZw8T$R&egK%of_>|%i30#MaxHWTr}R~ zG&PJ$mrF;LFJ`UR47AntqjTw9$G}{TXKsc^8^6tEM68P8j^W2_5p->|ceF(dx=JQm zd+K(qTE4BZz5dvfXD1F6Kk?^fudH8Nc}i&9{1Kxkl?IA^P9Qa+5<@PWj!4_#&cofY zj>@rR$DTR&?2Au1?aJe4T{yLOMC;n-<%#lMbbnNgEB~(>KRvDVgvn*cPn$HodQADw zH4W=u+VSGk%O;#sIQ9hp8>}wGk4;E0+Cqj)P>Ea|DsI$NR#u{W0y327T+oX+2oD4d z^x&I+$XsMW=W1D#YyHqNjsn&(`RGSKx@prU#sfV1L@+>O2t)W9B6rZN#~yo(YlWci z15fq9piVG!0?SVfihr||=nOKC#hXP=9?Sr`>Z+?K<4|Ia65+q`#v3sTiMyC)g0f52 zurPEDELy&L@e2G~Gc-)+9(AMC#zV>vZm^09t-rccvIgHCU7ykGr9dHjz1FI_!A*v> zuj4ZE&r@l(2Yngp^};_-$%jiiTm!>3@PD)h+#BWWtuke%D6ZZWIY*zfn*(F6qyip7 z5C#!2s-H5fw4unzP3L41*)^>O)d-IW0;3?i8lYtW^sbh3vNWNVVZu!-DQ9ysha(4u zU4cOuB3Fk0*2H7lQocjkCUF>pVNR?T0ZWY552lop{?Sdi81So zRyxhcpJNLcG%%zNYm^|cE|PMTj8Eb>IG zaLS)d`jaV7B4NcHtc2i6gvvZcqXWJwD;nxK+;eE}fx52lHq6JKJ!$r|DO0AOG4Z)q zp83%)e)P~I54^bM$<;4zZrK?>;l$&=_P<~F@vpx%XZrCg7H?uO2rLtiDwx(#W{D^x zl>(NQme$tRVi!XmFhBzwl9eNn9`jA8#|0GYN(e@KD2xzBwLrO>uoCWacO;#%U#!E>ew3^B?BV@Z%p63o<~pwk>%oh33o z=0nP{K;3dZ3mOuyQ|it5#nBJ~tpK?7)>}(TN{GaGllJy@$WK50v<#=mU=|K zOIzWMG#r{pY6c*=lTLN6wiVaw4%)$h{lfN4qa{XiBQa~)!TysS!!q!dCXCU*6qRDX z_2;db4^AD4ca0%`jxQ+lX@<7VXGJ}2HG_$l@(voqk>?T#U)n@u9giqQV6IM?WyUWi zLkPOOl+99W45@U48gDX~x%t>k<{`pH7MXQQW|QCK(NSr?CXyPnTpFp_fRrq)r_zj8 z^K02eQlm>pS;wYj)4K!)Vdz}VqphgdacU%GjUm$Xl}|IYZ9XgNP46aNm*tJ*>KZV4 zbUfOlt_@*Yie|jQP!NqYmAP;#G4w;#@u{tCbJo>*613boR42yX)ju@0QjDjNQS|bhD>w1Jg|$6E&$d@$kdR8w3?g%aVh|EBG;sahMdfzI~r1^ zFUn*PIcL}85x)#chYefgQUgJD8HP@gMP>+;wId+)t{|zGEZRcJNXR1?2qFtIM*}Ld z0g#4g65^R^3}u2e#*~TFX+HjZwpy=a(0a;b&WF)@9l6$%hZ?3DY04%tpOubH%fy>y z(UIiKC6BuX-W%EQQ8)g3BP2uHO2whHkOd5hO?Jlo2{(Q3U$6YYIar|J)XQg`dfCh~ z&p)=hvO1QErM#(>-|_gIq!mxvi4;D^>a)Ee+X_H%VpcpF566;GJCF*E^o1&Y-A?<{ zuRM9rUBBDBetW^l!V+Ie&{I_RYE$VH@4}Bv8#Ud#ZtLPNefF!1URZqAf^#L3*C9wj zFj?kfAIm&_-Fxr7@(Is;oOq>=DK^ZNZi^f+p7v2!P%d1!@U8yr1y<#}{`%|T3m!3| z;Y}PjftePeF(&6_umCB*}OPLbb~-J?AgZ_w$uapS)6jcMWQjRYHjMF2`6IR(QqUY!`fJb0)~~tH85NQ!!^Jf z;N2i(`4tci1S%z!tU>X*O7T>&WUcko3wC-1Irs7jTWwBoj)uE%$4wKOGQa}M;Q<&~ z`Anb(o*Q5g1Ps}EfBnQn1hNgM|5t0klhXpEoLWyA?FJhGrflMcN(5~sN(kDPdTK|(U3#VV!-qta?rYck#nm_S` zs;Zh}7EHSC#>>t<^TIK{S(~42?QV;neD>U!9bUd^5kr-frPF1kETsTa%YyvShd#s! z1(;ZYH})h%t}u%RN<8KA`<bO@>NncIE)_Q5cRS;+Ho zf&?3_%oJHX7RbUpCVqS*ZIMaDg!|eHWv$VMTGkkRra{&m{q?VZea$u3;JYmZqb~#N zi~`JEy4F0DHJi@Wddix?luf*5qpYoTuC}F~8m*@-Q|h&>+46ZY^_oA+fYT=ZkLYZg z&2|5pJj~5v-M?C+y=YnUXqmQkEp|tr%hMUPcD1?(k(lqcw5;q4gXhIPsfwUG7K8t2 z#S~Yny)6UUHua`=YP4SSm{Dn@&ZV9jttXq-XgOb0+RF4u2JKPfhvw8s^61per6!+V z$3|Iew5^skM$1QDGx^x^>FGuDo3f6C*Hq0$SzGB`+J5nqANkU!+<(ZTMsw1ZDfL>` zY~tnBLw)ajDth(Pk)Lggu5Jvs_HohM43;F%$M*gz@#;OX4V!xpcE!WmiWEi;!SS{_Ln3Qt<*od%-Otwj3BVIexJV495tNEuA{gF15dWt9{8S88}B1 zfS+D830*(~a)tyAHkc4ku-J+~y=Eo@^yEPnG@>M)saBE@9%)Eh!l1-1Hpzk^Jqi!A zsD~`fw8Aa{Zjn@NQ%{n#hXIM`MbI$_4}H;+UWjK37($h3$v~Je&=$MYP(Z4|Frg+V zSNv+H1Ou6)%+ZrdGng_lc|?;;GSk+A^hH%bG2p7gMkK$v}*m9y!1ELRov! zvWXnvC4GJm(4-B>V5US4IN`PN+ebzz5DZdGYT(+RzCzE5|1#;EYv-?Av+kF7+_~$} zmQkZdjv8B%wBsekq0<)5pFVBE)Cp52j2ROu@Rbx7RacLG#|4+Y=bZPPG3V^*W9Afj zik-NPu8pLV>~0Ph28+)ak^iKK1kkvyPon=qY;f<)=37TDPZu&*Oi1 z{@!2x{^fPgMExBFp^^zTlg9?9k13q;W>pcMTCctKTC!p81}7b8avVIpEAvgz{efqG zodsDYVg!T1E3dpVCpTwTH?N$mBi|iAe!TGP*|X<04S7Ic6UI3gfZ_ABs3Him*aYN! zBU4_Jc2I!C4@d8L&wGf;ITl9}cFvI+)R$J`6921T{ffhFa4E;?s?5b46InJ@#; zYrwry9OX9P)H+JrA>hw!wE~&<&GzE0nYhBK)C0Y}&-S_tqpaRHAXuyhsy}Yh7P0GF zhoJxG%{tms?4P%$joLH2@M45#e}<}$1bw4O8|rymV0jaLF%08PUTpBZ-5rLsSs)^fUqkX3YuTDAXl8y z5UHsVHq&EH7IwLb7n&f==45IEyHpa1T?RmywxV8O`H%!0NlJ3kl1;4WomrxUFsO;d zFM>81S{A~gVZ*s^hZ+cPU&Vi~BbWheXW*#kZ0Y{8%=|8sonUE{0BCa)PYp5oa>=H7 z@W_YfJ#6?mDBhXljeTcx_nJrRANui z`}XbJS-Ioz#jT?XJu_wLviuYV`!KGK6@lKy7IALB@)s)h3 zyl2I>7dLjSh!^y&TC(Q3M;3KJAdY}GY;%&O?V>fS8RUig(b1_ z#^&B86oK`lYe1JvmxcO#$p8AU|3VfpSOfBBKl_;pqXsEz27?qZErL~q1w<(j02vzk zy70ma%gV|`PZ(065`kH?5*P$(v|h7`IabiXG2$+|GbqqXFc5@ID#`eG_P#4|9Y%#~dci07^ZBLelb%^^gVz;c9qY=oJS zXmiG+k*2IQn$3(;Gn+E;I@QdS5f6?%9-jt%g3{hiTkTQTGps=Q+zzyCcR38KD|7q0 z%z`rY@r2joc`g#g5!=&%4^UYA2N*_V;2#A1!-iRLFQkW9=@|6!(&f@egtFE!PHn4Y z&7);nnHp2CJ?eTOlBquK)R+;{ml>P(NFE@eBt9R%)@xt7MB113LRqKcm#!ttWM-DG z2klYU18sE-T0>d$h*J@?ZB~i%HvR9dKYx4n_Z?o>b$E4W`~Dt%EEowiRfRqWKMS^# z^msawzUSNgzuarzzN@#cJBlt1{GbC_rL}( zIp1-S7yxE1X$@ta>U!3Gbq!NbjhX6P&8+n%o6e<`X4X8~R?B2CQ`&01X487@QP+dU zQ?E6&(i(D_u@S>u%|i`ywO(_Y@tBco9xa%M*H-y*%}6v-$FKcrnRse6o5nCs&1}k=LECDX@o0_ap&q`VgbYDzh=c|Lcp%^| z6dS=gHBUpNWXq=3V~ahX_^wspc&=ZVW^k8W$c#_`4`5^_9vbA5ih9UY0wCm~5i3%W zO>%_?R8Yz;1rkz5E;eYeQKomw)HB7>b=U}IuHa!J=1`&_va4N_hZ>|pLJi328ZZeP z803;)IB7_{=}YjeZE-|BQf#DR3UW|EN$()nx!grA1!NHn0UJ$R#_WAQV`DB;=4uUP z*kZ$>wW1NLug~f2wZeT?JdOg3FWQ=%)R-xmO+7VcBpS(hG}4r{Mzfi5YGzX=UZ%5UD-p(@?KdS`G4{RvpmWL1v&RFm8ZY05sNQpL^2Q?+AbOqqhY^ z!PanGzWu@cCR<|~_myRe|-@HChu-W!{ii?UXiYtpk zrT!w@Q;=wB2_M|o+}FCf&{Mcz#@X{`&Hqj7&qkD$_NCBv{axYq-d%0QEk)y})Jz^V zqc9O#c;&3-{cX?RvteY}p)b0_a^IY7=FFKGNx`K(Z@cX_^llUu7J?e(H62t{Rbjva zzIBQ$tS*6XyU52c_U_&L>Z`BPWaw|2n6$53gX4llG~3$Rz|hjt^16AnA!nN4&_k)m zm_YvVkADp1vSrH-9XeD~Q=_fk6#1BAj=^U~@Eu&R{p2SG`*%^=F z87}6e)*g0tby!7#j89nX6>DmqF}_m7&KxlWSrw3@)5n&Xbgi_^jO;ymZ$Bx_>&W9qOKUl4>)KXr?q-o>5p=K#zgX`*35 z0x8TvJ26&J`*x&=9Hc!-TtoG6-InVW?%)45^S}Bbxxo488D(K%`{RR(1&^ zk7g4+1TrASvms?Dg#m&-YBW5G6dgB>hvKVNC6+Bq?bvQLHKn?HLGP~aURmj#GR=R| zN#0XWv1@BBfA%)+DA?#FK*Sytw=`@$`~4Q=~J1*`fZ;XpXr)YRmu zjcwo1^_P2E#!s$!*M~3ihuqJ3==#wiu|!N+CtB9E1S@RF#b-g-oq-;KJMX;n6QB45 zVMvjSOD}-TC0WfUHVhqHHhP%iTfpHR# zlt|{H9#|D}WuYOJLL#FRtLAmQ_(r8NtJ;zmPq{SVnWt>GiI|~+T6rbLn||vw)yl! z`_g{3tmCIXpLa7h)1!$o`H3<0^nT>}eBN~_bV-TVR%R|Wx`v02C!a6PVEWa$nt^(z zW}I3>StE6>E`^R?V{{B!uWhv#9p{nA`S=-+>4h56cgLc2T^$IZy0Eyipir}Eq#2DZvbz=EM^qeyKEJG#3-@BBj7MDU*+V7Fb`K+YgK%%{nQ>PM)AayQfT@N}}d#8Tr<4C-= zBD0P`%j7ii)R<9eThmIHWyqUlUBthiEQ2x-g8T51zC)F8P*iN8g3J{GQ&HO6y7sI) zxb81ISKqPo)d%Vtnh)=5+P`i8rtU~rq99yWT6)1*=bV4m!ZYWcGs0UqbHRj>6AK$! z4*cTj?`~;a7b*+Z?>MyL*;hOEbx*3D91Ii%1O8Yt(YU{T(S4hHn&a=eap8IIp8Ln& zK6c0VACef}6ycOpPQe0-u!4Hd&J&^G#3>JhU>q1Ta}q;*v`)k&l3y=5MYB z&oEf;14jY0IS=955u#BFX1}|? zA`(rjtrgL5nPsM@x#=hLr+`oVH63$Ir6S)27fBxr@5MV9gtVPL3 zKl;&pCg@Scj5F}p*4AS3)a==_M~@zjnQjXgE@YQPKEr39eHQcP=mL}D2!qLKr=5mR zGJymeIC(K-F0V7@Ih)VLKbt;tOa4|Cn}@(EjynZ$%VVYR7~ruT+Y{M8V*ej}bxYmV zso}rC=eeXh>ct=>cS@DpL+5p8!XPVkAZ9Y{ zCF(u;V@r+aUA$NrwK)^Hz2@c-@Hzm_LV zy4cj(^wnjre0}+ns`VB?lJq@zp>*9_nx)3 z%$aixu3(u9C-6BWBo{peDuqyX=?;2s*u};0t~kt zKvvx!fH=)TiuW7AgE z!$QnjuZxgRPYiQS9%`6o7~FfH>?2}Y*34BCoRU&|;WdHVZ;p$10^*5y>;8Df{*k_s zf$ckY&zd}G>yD0|cwg1XiU_{h8nEmU&cOpM-JOwSSKX-Km@{XeJAKlu-}n4xM0s&n zqBFiH6-y+-9dRt#Q8lsjtQ+PeW7Z>gEygX@l=~hFfiB#)gu=(O*mQs0DjE`VCb=nzAelI`OR;BgI&}WzHge- zfkGs5v6=kT3mbL?p93EBSkOENNu`+rhHX$^03(YgpZ)A-ao{MA7%>8`xFO$v|NU4B z6n#?EBh@^j6cP$j76E9Dmc>w*#Y6;+7o`ZyHF>Bp5WS|w&MceQ@=pJDA*ARC~9_zV{)=6{XP03^s6FV&{SRKVv8*^!^ zM`v)Y#8Qv7tH~Oj9={LHN4vWo>FN7Qc?v5H(8T3dKj*r1S{5)`+LylcCGM_SvqtnH zm{z!+$Q&s2fWV!`K*mtA(@#HLKF6cu!HmF{Uw#>D>wf?H-{-fewJnj<&;|Rc;Sv2B z=+ISU2j^}Y}!`)B}UhbMrtsqs20Eo&Zao3BK~94&+CMVC?gGGi87I}x|~+O4gNtytLYBg5B!eLkylf;C~D zRZuE0lfmW3%I}BwSleE)eD0zxZm(MQn(M^kcI`ZS>@ikwz-%c^=$7~J9((U1#_#a z7mk~7;^^uTK@1mgH*HIePR;t)8eO-v(z&n6On}=1iIY3~b2GG+z$8em$>%*Er`B^! z0ob^KU)Z3bI|^dOQ3+5&!<1HTD@a*C`kj5xU#*}I{Wk7+j58-%SOheE95OeVG>3|| zIV7ToAhAKlCV1{jJlKWI7my-QNhAy*FxB2gi7Y7;O@t@M3v+3REOHVf0SH457`x&f zGJlaF7McmxOBRElJ#}=-aS@pyA#JNX9S2F zwY}m0{zB}TrwVWS92Ub%5BMEWX%e#tW(-<`$bqDWCOTK^g-`^YOCHT2upB`nnWZ)P zylY>Y(A1Mnb84Bkw9>YeHIlN4r2ZdZ?F_GnOJ-zNE!Aa1EmV)QN~)Z`aPs6!t)3QZ zOaJYhTSYF|mnS{UMWre-fRU@ys>hhZSjC-E= z?TcHVwPLB(eVuWCxVtk}YE}3`-kSMksYL4eyEpnno=J17PdsIIJYv24%v$SXJPL1G zdhx{*Uf8C zTYNTE^e3HklE{M!Ifrjh%!3a;2tAP>`N&5g>t?-e+csL^c?ly^KK$XMUW$KEr#VV= zCCr~cfBpLP(7)!f$gw^0UA&#fWESEtz4X%We)qe_AAdaGMdKT;8#iv0Wjl_Xa~Sz| zuYp?^+kW?+xqo|Tmvc!ZCX*~f6|tgXr#G7U)$GS92{+u&Z?R6F?5g7UDsvmoMrWDj42ak&V_>K zFNVuu@X`ML`@i+AZwXQaF`IuRk6-2l zx*Y4Gq-SoZc7Nf)jzEV$y?QR1c*+xPEjV}}S~9^uvB)>#jn@F$*lYg!NP`^Ooxn7q zC`AAXWxyb4@X-86o-6!EZX(FHC?w&3^Myp%YTRKhz2EBX#QlNGA1bkqy~r9p)$Gp5 z^&aoD_HT3+-)no)mu8w1gr1JxR=Z$?RWm);D!XrY9(Fc8Z!Pgi{a zxJSZ-lz1}v^mDy+bp>DkviHIZ?Le@fYFPSP)qqbof}FN8L=3b7{u{UbI(>XC{ed^~ z2nIabOFmh9H*D=dz=sdD;(ixdf`q(xtM$o`SPi?KQ8k{{hUCI)>=*ttAn=3}uWQ)9 zYQqX&&|f-gMD4_ys%fW0oZiM=?aLN#D61@rClX6nFR$D2Aa3r**Gw#klsx##pZ%%8 zxQdCfc-V@24z6qW1f0>+Do!|kdQWTg`A1hxnLD<)JS34Y+5ukk>U< zp~UkBhgt|go1g~X`ObIZeJMLH(UF1f44G+i`Q?`v6&2;+g(pV<6*rA1hI+wcm$F9U z+547TZlNB7#xZ7uFpbf&W*$_pXPN+vB|ni8X)mzmrp#1+V!fc;uQqVnPz` z;%)Yz$3oKzUIZZr44d{#jA$@PM+Wd?b%0xMy>-fzDL3431G};C1c--E^rQ$g1Q6!z zrk!Mz!mKfa%H+w1r(R>U-+Z!W%V*1AOr+LpelvJ&YvyKn+#{TO(}^t2JamNlm^DMb zNILthj<_=l1MAgxxYd5KC5qt~UiZ1rvN6P>DrW7^yoggkSHwF%j@@{8Tk0#9yFA^N zgRe&fJf1D_*wg@?_y(L2u1!`$nJFWbl45;XEIGV&>sErXJ`2uVvlL?WKmYST(;HqgsCM>yp;y12`&RNcD8}EqyY!d3EHR%SN1b{-z1zo2}p{f)mFk;$EyEALzrl$$)JI z9LMKLHT&uc9A7Y5;Pd*;$aP84AnWSt@biI&gmqg0W-d}RUHrIXy=qK}LI*KMG9nD- zvP9?^#}K*0hYv#-H*Os8+82;y!>{du0|#UmFmPg46n^+Hst*0^@{x7m#RSu-aO;9A zJf=xvEaAwJBgv2x1Mmz+QbtHUWoA)>6dw2O+lQFh+la^kBT}UPE zbW`GYhkDaif`I@>7>9*QMm=CaC{EJp)2E|o8lxG+FAZVKiNjb73$`#M80JLEu4pu~ zt4ze~jwT1+ZeU%xY04_CvV54RH24*upEJ_uc7kME1(~xE+Ez!C`!>wV+nwRmuea+r zxUciw=Nk8#oV87RE*xP8GVA;3m@$i_EfTq9)v~pX4Q`=qXLk$hRy{|SO-qGIn@%hJHG-=A{g8s$e#h3P*Blnjy8(U7C5}1i( zmevb6M>ulK(8pBg>eA)7%L$A~vDJo<^)3u$7wtT~=ltYe>;A``U;rax+>RQIvp#*i zQ#;1J++a%+WLgn+d27u8ctLPM*+WlqYD6i)OD;6vAq+}ELLh%Sw*F8x2DZfONC+T5 zQVR47nRx7K8A=zI&E+nUP!gPzOT7e;lPm5B0~I4Av*c=X$Rwc&Q!0sP%BUz4FLCAw zVkK0ltMQtdvPNo#e6nWH%=z$IZ`x`Nk#DpQ_?DmE&J}cY zcZLq)jms!DR)51UBBFgc;X4XH`XRY$XE{*t99aratO?kWDOcs&apTW9$-k7uevnJ7r*#LnWuxf zExf0^`R1Gd_{Tr;t@tl~@r!G(y;i>fr#Z;6!I5SgmQe$&5%*^E@3-x4wX1MY|G-!UQo5KxYIlNB7=)?*~8l zLF&;@r@J#Wm@+PCfB3^6(6E2_!yf{y0lKL}21e%&%YT;|@OV9iHJ;+B$+orb`W9|G zM_&fMP=Yq^#RH~iGR7bjPLAdM+dHin&@oqeB|!FSC^EWb%A~U zJARQYdPhag)DrLR3Z1Kc`E*qSoSfv{>in(y%LUi|^c;bmH zaz0yB!p@yLaVdE0vBx;@5Yt7mhBv-S_L_GKxty`M)p5vnQn&+KC66tg8R7?e$rOBP?G*zbY-E;M-c8zqfkj z*0$DoDtY|q>Z_(qyKv&9B0qZS+;~}Ub1R*TYNs6N?S6RI_J?=wXbgvcw00E^$HfyT zVHwUG&cE^Q+hpY$Ige0$%92Wj$ax1LCl|X|iSCx$t!H0$O42pTqv#7+?>NTtl-Zbg zj!idXmFuu5mpIRGV#^6Ec#-9Y9xF=dM#*ViP=*w_x>a0JU?=Rd>cT`av1?_+ z=xJpYXuC-Iqp(tk`1b2VOOzBGoi&ebx_wT5`1#jcsFId%Kjdd))yR1TEH z>s)Q6;~{?NNHqS*JAX!gxYK%*fgW`{W*Mm=GGBz$lb`9@16YY0b1vP@ogM9db=9u$ zSjplfT0CC+VvGj-`YIycs0h#$eE4i{Mp6Ehe`#@K!tkB+$&z`5Zb(E^YrA#D&xvpp8af)?zX4Z5>o0M;q&^w-;(B%pwHgv<`9E89ybc&}P zbje|$8Tj$``-K-?z#|X=QVKD|vl%Wa=)=V{D~)k|WXZvUjqQhe@-P(`%(a}u#$8Dz z5@ev%YcDo&ODLaWvp*P@b?yUdKGhbP^^F^=6cd*&M<=eMl;$AEo#h4v1 zfGSgJ$BEbP?D%s-?~29U@A|;Ab7!A6rhH!UjD!BVq_5BGvGJ4p})0!}308NO+(J)xy;UB|%FysxR67Y?4tZ{qJIp<(u7Z?HwLv)L?BSRZ% ze!2#RO2A-?--xcRE@VZcQMiVZxzJz}D#Qi=#%|y@edWrPue|aKhCZPyfiR}56Pk!$ zWKmBH_0&K{p|C+f{McCbe0VZ2LRvwlRgM=Tbt+cCqabhr4UQ%%5FM9huLNTN1|$d% zuPAUm!4f&tfCK>mgN+)d*cGQ*rVerny(ktkU-VVyA69$V@J%H6RV+{g@BJenHDOnixLw|M}AQ&>!N-!|TCRvbk0Fp&P!1PEwl}rVOU6w+8Axq+f3;;3o2n|b$ zUFdZhiCB02-sLemhgAK?W|ghJ2doVc;+8l@a^yVPR#rc4nQi8+B;-02TnQFd+TkH?|diump`S# zNvq*d#TtN`?P0a3$*Nou^ z0X!CXcXxz%felecU~qSfehR^0nu$JSxYfpsKE4~r)PV1N=Q|j%Bg_=UHb?(YzI*m5 z?oW}l$-hf&Ji0!8OQCytEvs;BJm5=4lKt-nePg1%Nq6q<5L*_C_YjtOvhd(BYt}5RdBJ?l0r|ogzAz{jU6Z2E`GgQo(qI1am(0awdF|S@ zL%b8lqPDfQwFF}@h;(NNxd^zG!CPzT%Mksr@V2c1ug?>zuuA8KTUX(E(u=h~qsi`v zUb*e@)pz3p<@aUYLgONagBvOcVKxLh6mqUzslMLWuC0wf`dZ(DGb0yXRC(q}rGZ!~ zbT}UB3R)?=cK3VZj*SJo_l@%BuHKTbX4oM0bpTGW;dcbj(wjDILKtYeSj86OH9+Ft zUU)owEaxpXY=K7>e|jK50}OAq(`4>X=b$a0NPx*eD)3+zkMa0rfBNaCF*OFKMoy1L zzT&*;dIrWqy$*7kV;NxdXyEkMlg>;wloZouQQ^pfA!@_K9HGX+IxSr-Rx10-3SGhI z$S59ZdwjV!g(hRH%j)aGQ8aU)aiV=zUxyv(cCdoRfDOd9v;Aee?cm_H;43P&{JB%* zTf%*d4(@xlZg0@*``6iXubes!9To$KjhzCYe?iTdZ=8PC$Y5x~h%(HHG9Z5=MgOnV z-^k%mZg?I^vq2LQ4V#msWDz7E2QZ-LJ3qIUt;dI!sBlp@voVd}jA>S)zz&Ua+XP5* zf>1o}xi4NQXTfgsh(%pO%q`H33wXpSU7-}_BN4l-(AOJ% z?2**NkGh8h{qFPTg}(m-@BI1M?DXGwEU|lca>>%fs+GyQz34KxD$2db%<)}ux$oqY zHPSO?Y{3_Aj@X{~gAb;nG4JMW(cj$@svhmBoupwy$Qb*E)ymM@zGKG@Oqs-o;c%;V z!U-o}f-q-C!WPF79pV@^jvpyZUBnF)2FDG-`Hz_Ot)IHx8ED1m&N$lmGiztyz>B4{ z2A4Fgmu4y$jTdE(VDe}H88p9{OARjyK=RD=1Uz_K7>h>}i6j;sBPQLKp$S|WCfG^} zifYG?KKA%yo_uQQT`dp$yq*cQH8U@tQP;Y!wj zU--B7`W}3@sjPUUFJOBE{!poR{LISELw!%*vBvNB1q(wSKWedb_5h)t9tLxWe@pZw$}F#&|Rf}sGU zu2nciKZTgHj6{M5wJKL7bXVafk69Rbi$y>oiy`En{`9AK>&4+V=+3|~#YK&Hq`Gud zr%uJeqOH~ghG9OR|NQ3#ulcnsv^j!iGx5|g)kuLHdwuTv z2=_~u{jo@gEe7yEv%~(_nW;^YSYaAbU^z?s!dDa*n=$Cv`nj?aqXp)!yY8AhcP`av zhghsf&P3rvm*rDWJ;hy2t>@#4FyI$gUaq2Jk<0XmEts{8Cs;njLW#HHc$k#4fH8oU zc*&b~I#;&ueN{K+G+LKI9 z_IL)2CsiD-FS^681Fn`>R2!WI)Zns=E@@nW(F>2?7u*YC+6(4Qpy^`GDHIB$4DfAW zloVT<#%*9YFYpkKAw?ts2E!gvY8IRVemZeBQNv*HLx$gFOh1`4X%a}FghzaP8xt;Y z&Bq84Ue&Pz1%5QqDKAw@FHCj0D3eMs8U|T!MA8a+2uy`RgCRrDEFdXpyzo#dY!Fb5 z7~ys2op<7=$~>uQh{E1;&plXh1XTnE6c#DBwA)NGfChN&o#ssY^(nl4YQ;B7+)FCT z<@exQB<>QP-T|$Gc+KF-;q+t^txeCB$LJqdhYNF0r`6SL7Y?|Qz;{gSroC3r|VvO=~n+*Qv53N1A#2QnX@{Y8E290ZoJA_x9ZEM}!gkxRyF3t+7 zXVD;_m`IVebdj~>rIbHt$79af7uqK+aEk*y`1b6w{&-)ipujy$*r4^|>j_>f7)+Iw zTjM9#vu35r+zX+*xqyMbu#1g)q{QUFpd^g(K$iN4EV*Ju4XIp1_akmDH?Zz468dk_ zj>OQ#C>|+383@xRbOaNP)ZjEYBrH|GwA{zmE}h$ns4dANuN7*_T0boa4!@ zTVg-GE&245sg6FkUeYUTU{NzHS>&BI#m zkAWQ=aPAY&LNUe9(;au*fpM((fq+{87z1+gY=s_;|Nig)#(Id*quO!ngvef_Ly0Uc^v4j8yW@^%r`-IhS;J@ z56}6ss|B0@c%32|Fk@p=-~wjYWiHOBobC+5IP+Ox1mjaCcqo!#6v7Vy9&sC|HIQk} z1^+06oaZod7)#{#oI@*Rtu zT17lU#abm5lu|~#(fYv;en36CIR^cH4Q|IV?uIZF5JNdIt4MxJgcAa~a=2@z`f!`M z!!0+bcN1d{Z|+ax^q1*^kJtSjKY!ge?OPWbT6rHOsfEjcd()7dI{Y`ar$ z#qsG%Zn(u<8I7PjLqU#CMAO3b5G>0FGyIOCvmbMvawwKAT}lnwt$Au=FcBL$(j)n< z5S}XWJVQcULxx!C1}h8ZL}KpJwW%N0{B3F=5sxRjx>JW+QpLsI(&9i-p+Df8aar+- z-|zqSLmzJIZo;4pMB+}e%3ke@0!76|r9sS4kJ~jB<9fo~kytnu%gmqZ58Fuq$Ury0 zb=K8yzx&Pyr%w8RCmuJaY+g(6PAAmj^@ja+Jm78b?Q1V79@UQ@OhWTOC&ipObI=8h z4rPe}cY&BEa`)YLW7Hd3BObG{&ILAnNeatz;uQkg`MKwwt0~c)fetuqxT<3tJ9aE) zID#K{s@N4~2$-lMhPVMmcgFeWpMUD9?x)l?vxxe^T(hgMzIsq9Eam&;lTWe-2Y@%^6ulT|GrbsLk#&Wk8CbLeV>Gh2 zF4>I1{$l-$usbb?n~k|4p<>(TciI}$=C&2-wz>~n)l;*O^j|1fK7@fAX^dk9i>y%- ztpnRKogi+BJkAJr@eXa(9*L}OI<&pLZDvK~4aXi|6zE^{RKwo>^4sCdCa5Je8=f*q zA+z2@iCyMW$lfM7y?xdXZ@2axunKWOLZ6lUZC>{<;vb$DW=(Jc1=fhMu1;u0pcnC# zHq698FOmh5WCC4#hygin+f4s;W6rUXUEB}EQyx6m$70U5SFQ0A(}pm}7E=T_!vf4+ zw;KHv?uS8RNrb@~pdYvF_+Yo&d7V90Q;W4@N9y_Ktm!kXOD=WF$}%CP zY265MDRaGMtJI}X4>pu_E(~Rtg2oGNjuN|WF94jU{by^)g=?fq$XwSv9U%ot>5eIt zgdxQ)^^mb?WCq{c8@u}+tJ%G7+ZCn$>)z)*X!Kj(bQ%sp3HRx&bk`DQ zCQX{^ojtoBtR&CGTK~WOd-CA^)P_wsRPp=o_Z@qjZ_XSA$raJbjuEMtdye~j$&;9I z<-}ZF{FdRxFz&kmNEc|35{a!VL?h7$ABueaYiRcVbIu6e{1v>z9{@PAo#ciMeP6jb zwP}-85G?%hZT_>)e4DQZlIhOi;b$f0lXVRUGTlnkjWpZf1SW?>G!*jbb*_#9eT~>? z#ng+zg+brEOD4u!t-fgQjlX(VXH)OG#d}_Teuq0`16PZ5&m{>v_x7Sew$wG9d*Mmp zw%&U;JeWwtMobJudrGS(`g^*fD^{<1=+4XR4&Nl))|X*EY6RZ!JdagHeF0C?#_o73 z;V-~wMg)Zm0GhMMU5ZMzC>^aX>;Op^Qsm9Iqk^x8@@>%j8qHAeGj{m_0ja%kC%!PFc{3_2b%!0NHxalbbp z!z(g87<@4zfg6WxFGu>G0B?q-L?fFlH320PxQx_rZvqnbL-h%_7ZtZ;>5I<3cszqk z|F(%t!7Pi!CP=(bW_JcOc)mi%8OtI_^xp{>O*DrSqcbEIFPVS)+uzDA1>wOiURI;W zP3j8W8603w4N?j~!YZeyL$a7la*@-fnM2BT7n- z?~EmT@cj|(1$T5+C6%{+=qHtxHJ)H78T7#pILNDN2D$uZ?LQm2i<|`ca^$+=nEPuvqi}Kl}j%6aoxtx7H$gB8u0}a z^lsos6IWMsZD7v4TcBi z9Aff=MEfNtZKVhdpx-qf+M?Z7DvA-ahz7^U9TQPf?k;$dec%xjvC6RF2u6F|-jM8J zwfqHk>1YSt6Zm*A9XmKC)s~MU}1-0I=zjQq~%cp{!GlqwF);2jOHu+{gY2<@K%{Hs&W%C6yJ4nl#jP0tgb)I*C)$h7r7RkNdANax zQe0D@lFZ~JSev_7jKhtI`~T?Pr(q<^?Ad{Tcjq+_k8jSa7! zsw&Wn;VCY18XNt`9_yWPjCS2`i-va2IsV1xMfUEodcxLppZ6QJ8@d^X3k>a-rgEiz8GDZ=?BVa71unj8Mp#uNBx$$Y!Q_EH)mMn=s`b6*( zAJ;X20fSh}j9&pm=X<^`bxgqV&X|#0x7KRyh&}KJ-}I?i`cx`hT$>tZX^*Bx=MF4X z<{TKP>~Aoj%4hozFggQ`64A@%){E_gyW(3HAFP~Ij7cZYKe(Z)rmVLmURhoKk*~aS z)#|m)^*vi&-rLjdPFt9B`s9hnS0odur~kCPsM6O^-(Eew(jN`jo}P|FeT`e&QjMO; zQ>S@H**!7*;3Z<{5J@DkmLI-xj@7+vzt@w%_#d2(Njy(r9U(-6r*Hn8yTQD!4KIb{ znRZYDJW=R}ia{|@VtH`(>Y;w}VW`J&1`ZI&*Z2s_X+!9+c;EZpH#Bp;T-?3sGU^EP z#lZW#oEQe>$3qd`cMY0#v`XBHF-{$!Cc#Jh74snk-2UPt zd;n-p;xRu3PBlOETBGG5ybNbu_<@hH5}v*1SFf)&4kZ(-(i1>nzVl!JcTPE*uOxwc z7pyX;^|+>lDq>v;tH^zsop&qqTX*b?ma0Vq8gPNij`GNBHe?1q9rD+|{*@3sZ=gFv z!!f{KKCp`3Qal9eTr60_W=SPJsHAf#>lVv)D42ZbEKJ_`$~)rapH}$hPk)WF<0kFq zPrCPKuVWj~MhCOEqft*!*iIw@aVyZ!lxXUR?m84cZrZ$K=Y0CpH-7JX-*InSvs*M4 z{@?}wcHG=kI`JhAuO0Q-=)Cj!tf^I%MdK#)#G_j`F6(S>Cfsc{_!YD5Xmj(r&W>QX z5A*0sLz7d|iV& z8aeJ(v3v&TacZDDW9Wq^UNefN&FI9VO#cWsf4~}nrBC|XTds-h!xmpQNn|G2LXA6qXXI!r zRElRC+v~n}#WHDJXH<;b-R(=9cq~18(6Nfk?8=E4)8W7`x-+n`#g>hBHELVi+|<<9 zH>q^QiKDBP`i+Xc@mPIl`_jXQwzRf&M8cpe4;9WRFFkY2gb5|31*-A~_0$`W?`m&Z zez;*vOIs(-V%x4MEyK*#^I~~-csm!^V!7ife z>vGUN1&%Y8MqlNpTYwUCa=9d|HrOM6N)4%EAx&>#vVXvJk^Rw(4`+hc85W1oMi z%K~!oO93fb-;ON^&Z!w=clS7bVU98pZuT*`Wv?9NjIQqQM1bwd3$O&Ty?V9t!t*xX z{bMvUUaQS>SBJ<1E$)Plg6JVo!`MX6U9rl6A(w!(9=zO_+@c;f_c-1Nl(TTtKod2Q=ywj!&0IOMj?Q37dYhYZ)F`51S?|;8!&pvHT(GOSe$3R;M}NPhZC7{O{$6i^Jz{iWpVQ?}d9i3yJcU8vP68`u;QYfofbwq_Et5fOv}|&k8sfE{vW|^%K8*ILktT!Im~7t{P{4NKOeKnLycK8nnBANsr5Q4Q_feGBiD z$fM&S-sIPM&7=7>=3D9R4AdgJGclT?rX1UZ*RN(*O_iX+=Amo7+o`|J7D$3Q;j_>t$nyjj0Z=S`O8K+6(di+>e_~; zeeErc?t&V!VJLVi)oCSnIPpJrwOvp;vc?-UziA@zzz)$eU@5X_kSj{cSR4w^;gr#m z@iIWDNP;ZpQqasm>Rkfzr9umUSYfad9_+EpTrL!4BFRZTZMByiy_f?jV2u%gkmMi% zAcp2lsSy(qn2Q-bm}iOsfDb(I0FD=?@Vxr!t5}y#tZ>JSRlSh~JU*y|yEv>Kg2|`6 zcL$s6a@!PD<@b}PP8$CBaLxWw>?;PX{Ciovd0v5 zb?7IHMrT!yno(MoiB`1%m)Y1@6*_XQL5ITg`}RJxYvj)~Pk){Ml98ba}IU?q4<-Fk_R(Q(uQkLW$}cPpAN= z5=La>(G#0{=+Kw(v)i4vb{h-gyDG=R7lh{5T3hQ=Ep51ecYlw3Xgb}(8#~ssQPn=>M*k|q7oI3e*&xlgQ z;d+4{8w_cSW#}6kuzIL9bFQ;;nLEkJJ)_*mAlEMils}0X@Yx#)H#@xnfCWJ@X&XPS zcie&5-_~Crc4{ZtSZEGUG=NEc(gb$69Cr8ExGgCx1O*7v|C4UuIg30uJFTQScPZn1 zPv^T4G8%YvAEVqo3Yk~J5(CY%t_B=;QZdbhmB!H*O&|*cWgUrNT=U*=^6|%$33qKX z-?)jvOD-i6=E=RgqqqOUX>LcBRZ{GK_ceuI_@dkMgpIZ852U)fVt3sczxN*hb=L(y z{z*INwtu=Nk9W=-|FOqMH*R#gBcA2UoI{Pyq{%MBzzqv0VRQ!HF@N9#AHZOIe7##< zljD~HV>B=@94{~N?pBI6ki%^)qR|+1*ZNLb;Jx&c`0{0Tb8}+JQqRh%_;pfkc(U4f4-onq@X0=El9*;%m2I$U9gFGvVLP*vLI46 zs?f0#b#(_XxbhU7kB|QTmDA6kd&0bVHDfBa?cA|q@wUVJyW{a>L1h3}x0I8NhZA-J zuZ0;zbqxnITm!>3a8xz$ z2G8U*u_&g0Bpj!)&91}hG$ zO*(1PB>hU3KS(f<-zsY4CryKMnDxhfaMGHp?f=p)U$r^@*95nqg%fD%@a;IY3)d*WvodpZ(pUrH^1@ zz~mXnpLF@P(@#0KuxM(qv^4YTdqAu>#!jmff6TUOqmh!J7k7CB@?X~u8zIIE;9--k zNV+zL;G@;iHaS|Ev=}@r<3WJQS;5-Gzil<>2Cc2FJ?5BWux2po6ZgdUMiLgW;#rKM zVf}1=dJOmFSkDVnY|z1h+vt2X^V%hihAn(fwSUGQ=BfIv($Q{5@oO4~JFRefJOz-jlddMLh2i#kv>vqTHd-BrGd&%11sqb#Y1;?Tcj_VN|pu!Rj>6m6I#7ydi$nzkL}vo(A!%P3Y}9u@x-dp)fnyK zSoJ+!D-Ru7d8pz39oza6$?K1CW>&gi>d{*wmAZH9h6lH82e>jAS}>~S>~Z6(3JPM$ z)W)`!r3d%#=;-*ts^vR7THihO80ioJ1J3F-O^tutzG-V)+q_Yu7F3U&F>)mCm=E;! zJhWpw(g(I~>x(6BIPN6vj&e^d_KOW`pV+;t3+<%1_>w78rB#pB|Z=2*{~@wZ?3myqI>aSivkKQp{?Ee`LC?j zRy*i*Dod@&qn+v1R!z|APFNdvTRyM*D?4tyg(Htq4>2spf+1#pmw_a7s(kf7rfPu1 z=6aW(YFCSR`WquncOgXWu~^XHO-9taUIU zBMaw)uDgp6-H+D0_2X`=igZC-8Ox2;?X&hb*pX-ooVDZaG1aIFkoU(Kjv&{4S2Uue zpPy`!_Vn7}m=p8?lkox<^dcCy`a0V&+`z5t;-KfuGx~>S^!3J9tW3SM1XDoJx??J?OT`qQ7qetSEE4sV;E3J3 z0>>YR{sj$Nvt|vxZ-wa|P~sbo7~Pidw}9W?$}NTF$|GY&O!3cu?)151*zLmp*)DnD zgR4=N&`mc5Klv#A5-1<$VR=}(U5JR z+RG4lVJ4nPZEMQ9I|I46A;xNU_@oW=;hx@TBUe7Lc7&Fand+ZvPclsiVwj|KY@m`!4#BO`!88CXyXO9Y{> z1l=PEkL@k=hJpn_{Qi2v;V7=4m}Y`;Abzh8#_ksj+#4)iKaAYe=n^rP8nf)Ctv)Uq zuVq?kThj}9Xr*(tmCn_AjW_+~(`dbB)@9Gy)-k*`BqRqvbJ6L0_6P zA4Zp&vPP0YA75%TKV@yDORDvnM>FU+O@7vo=F}eZ;k91JVDgw6;xz+h>UFLun{1{g zUp<&eYT(_y8Ni#JEJDYV)%E{nF83K>7Z$*dB?>(Do_J#WK0oe|0=PD$H{508Cmgps z!7)+1n<8G@;>(t!W=858@D?dOnj`yt6GS8%ZuCa<`NQ2FkKcWtf?dqgMt8;y_-ezz z>yNH6T?#X1kmDB=JsR|aPY+<^A!Hyi0CLNgEs(*6ud#9=9?gvRv3Q4oVUWb*1?;qG zZm<8)9`$*rkzgh{Jwj$KQ}g_Vo>|%ow&s=t13*k5>s+lj<$Or$b$g?Rk!V?S>NvGt z^JoU`U2{^-d;Yp*&D5l3)AdQcj$zQBavd;a;C;nKR@KO`jdqSDozOeMjs|xUW?Nh9 ze}4MYAN=5lFz8{!hV=*%zp1{m@=kp3Jm^W{X$H${|3?v zp6WVPzp8V(Z};iybLs$?3-S*QbkqdyjIsvns`pCdk3F*-D$~n&g12jCDf3e8lQfgJ zf|xV5hVY%2p)NuY0n1``k2z}K+tm}jJ|%08~Hrn zR$WaCA9=l)C%kR}ha{kxK3;c0j7)@KO>Ic-3=R|JGB`-%ECYSP;ax#(OCHFbYr`!o z#RX9a-K05g$~?)|(RnD~Te$kh{g~VgHI4EibhZdy>4CqEJHJ> z`KxEleE*^|YbsI*XbW63bIztC2fz6AQ_mhcc-NLqW2>sJY#;h=tTOgIs<+yV?AiEK zy!CK+PCc&4c~Ak))2`j&E#H;Tj_022W{eG+Sm&9wLu7U23=aAQa`UX)O3q)jv zk01$27AWI&j`3e?6gN(3TU&nKykO0<`RCX3wLUR+EGi<8mF+u%+ioY25iz6+5+UsB z3LkwexngB_!)x54Q{@^~x7q*tE_df$!IHDW*IW~P|0luL9pWk*RtQX;eTI9@wTY_A zfW)xZy{N+!(uii>PD){0j zg6UHV<2(HI7PXao*QeaJo$f8y2XA^UH!Eq(qS>J{Le|#&`t@Z4nQSsFtEo*cIR_b) z3=bU4-FKg;@Ckd?N#*Z&8*3=G-P7{ybFrxQcrNyN|$3{Y``(Ik`7 zE1Z{WBzeRD&NE1-v@`joT*s#X9dcn1qLgoQ*wRdb)+=xQINleo*g}cbQkLpkMSp%_ zT6s&WmhS%xjLzVRj<1DFfpJQ?ymYt|yej0F{?RzTzPGb4o$uecV`EGG7=GL7fiwe! z_OIMGc1-{p{ay=PWxx1n=8sedR z9UuS4+<y zdA1Fy?uwvx+VqRxaPy+ScvD|4Q^DYl>e|jQ-#ggRpB~_e&ikZZb=oPBw*&bHUbr@q zpOEjLmaQCDKHBd~T)kYSY_>X#iQp3m5jrx-^AP9q=54&KtxcymS;;?!F8E*Uu6QU? z#xOkLCqMZK3smU@6N*@P#pjFJvSrI_Uh^7?q<_LBwdi{_v~JosipIAb65aciL=aG; z-qvUcRQ>=hi%c8R*JL;}^R9_$YP5p9!i<`xVQ;(bHcF`lUEVjn=}mlK#GL6nS)@_Z znSL2PAPToU@$L~hvHpt?zxxM*l`13mQX1(W0&hqk$? z3tZokaQ6$r!EFUhlweMs@461Vp2J}@Uqgx*rCU+ws%pYw3JI!7!ZjuY5IB(;$;Hri z0p}0>vtaCm|J5{xp}|au9(Lkt$}6s$HFx#F0}pK9w!N#9VG*?^C&iAgj<2nHsw16# zMO*ugbLO5lUJ@aB1RhpXYU}^@%yZA1G-YN}OKa^hAEJ9Q=?Rrp7f+k^mU%K2(Qas6 zGHu$yzV4&_{VVqEo!QXj&%!`9dtjjN-fi2C3=F(x&fK@pUszj_l9N^3Qpx1}u@nC0 zv_=2&?3$+!?ElS+n+RtV&q`6q{8ex5bzWW`?CJ{k9?6esF8szR?d=Xf8O>gU|1&2BIqeE&yoAnndQORPo6VuJhAuD$N>zZCYTgQ=}< z`ZPyUkM)~eS9kEx6Yjv#@aF%alQ5#nM(vDRv0eZOP5b=t_V9n-6Ec*hydrVRJjZ~V z9lPCg8^ag2xxf9qd;9CdS6n1Y+|d`#3mKxqnR?_&H+_nCMyfzPz#xsl+8EyYJtp`B4?LWn`LopciNV-oGCC()CkNxkGGm4NN`BiG zmLV6PiW|X#6)RTggMgbk&!AyXtD@jYe~Cf-&ENbDQ3-8HX8&Dy;e{Xj*vGUZhbDoH zuYdjP%wgdZlul+rm_YN_zV!P&!Z)*{ku^g4OEmlfbWo)U{Emf*A_WW9exLwPM`)Mu4ORWzU zC$42VrD%(0Z7JWDT3eP@WYx8FDKv{5*RmFEds;197e$EU#67Vbix&4YE~%WjKi0a% zR635Vd_{}PD1IsHR=$+X$n6lfZZS)Ft1?#diZ}mtbQt!(u+nOc;FV0ejdZMsR^H+$ zZ8?wk^)g}t`DZiz%Xc*rNYyTfFy)RFq{c-Fgp0mfWax^z3}e1{q0@4&%c0V;Pq^}J z4axR2LL73>8g#*-d~Q4m(4x(HZaLOZwX1uTVQ@I_S4y)67@-YlR#jw=7;hjx-?h~O zlNl&JoIze&zu=B4Dxlx1uJyzhsk{aEMYcx6Xb$b>RfZABC_a0oRY1+fe=oLFY1NA3 z*c!!4Ro5a|j@4CuDSxbWi%<%=C{yT(1FHziAI8svo=B^>JoR5cVPgG9G6Nxjuk2vZ*BNw2sjNB% z+>vgV;*sp!gy~_vLe48RF9W&W;OOq4V|ViCUU$i*`C{>LFhWx~ix`9?p3qRN(lXH9aA}p%hNbdSAvt9UJ z8(x1+V##6!7nO6CxOLCxfB3V2xiG)IJD52=95+_@S}kt*%(eWg%t<7TZnQokflqOY3rSwv*$> z*Sz7T-fdgMM;;?>OXjwpCuYp7xalUANECeC80SE0)|ynT@B~CLYFmHy#edxtQ%m(K@_c;|Pz0Yx3}I!EiB@pSMFvG~x%y<$ryi0R)fq1O zh(XL|k3twI6hUURAX(ux#fJ5PksXBQ;b=HU9Ro)Rm9+9+z{^;UN>b!Y<#H(;Avn|y z5%ZY#vASAX=n(6{AF#jshu_2^E$|2Ej#7nNiBv1vr~sD*U8RA+fqa&yBhL=$@G}ps zZ=h$xp`+y~7S#!uIl^@|NVgxFO}-NlPh<=bcNqeiF2gWZF)!l-qa`%iGlPUuxw%WI zoq-*|NU9Egfn4})s& zWeH*#y{J^h`!Jmsot9VsnFvq>sY zlRJa=o~jxdSi$QP88LJe2A2v;WgOV%a)WNmlkUKF>Hs#W)q>gQxTdL&Tm3+H$efoT zN`8q;Sg)6tC%>UkRmPX*B&n48_^w8x+-RVU7x)h%SS6&qJi+3_e9ZGE_V#o?cjzG9 z+DoRlPv#SU@e1?;BM_V{W{6C0Xqai{wpjV*%BrSRDxC`t^$$jU=|W+DKKsy)9Smtc zWBj<2TEX7Z?QrLv&iH!yS^G_8v_`^&>gtwRl8nYDwT)jwHJ8nGb|lhV zxj|4@<0ebOOWowPPqGCQE-CcW5~rV@dc#cv+qSvwyIoh;;P<~5OlvD&c5zWd{sxS7 zB2G~@m>B4lRIX+t;I8onE^LNo$h+yw*{BhlR-bS@6IF;T@X+Cg%(#hfeB&F8ry$U* zkVtxL#?8I+o$n;Xj2I&_9)J9C)cLH%z(AGZ&6;J)y*wt=is$^vh#{w6j>%)f8Mr=7 zSKC7Qwp~;V{`rd+DIJ#_K?@a&R!UcqIobMv)Q>HLl8{wanUYx|a?c_p23(GNYB{T~ za?`Hf?qH}jzp3bqjj z`eUtIS`jSA(zevvvNB5BQl+i#$n}crTKPB~$G4m~ZQCSXsvKJ*<;3mAIi>1u(X5O` z9@!IH8n|V#Z6Y^-_Mu5qR83Ogt+5Hm#p3e>~)&8=sZ8 zc8}{J#tt&HnA-~PZ-*)>GcrTXZsm+3o7-b8+ES&HlX_i8J)(3yvG=@?cBP5+r|u^i?ZoGD<0qd$o*tjN0e)xz8zN} zE7dWNcjfBUJl47Rb3AYz&*+d<(+>r?m6@)~qi}|84W-#DfoYsjSCwP;TH4muwrtt5 z8*jXki4yz-RNkwr?y&vHr7EnISNgc}FpMqS9i z$PGT4*Nvn8st!J;`3P1QE%wndICx;NuP&7uvRg1y_@jJ1i2EGZf0MeZ5;dkJ5WSef)%`$R&!7b>WyJEfsZndv;?g zQhud&gE*b*2Z_9^A)!zA-6eq1J&TEhyJBE5GjQ z?q+C~PIR#z79`E361@n7`*-IXbNM%%oBZ;<`EkvOYZmkRifke3QPh$|D|Gcc2EyI@ z!zxyCmH39+{QG1VGnNWt8A4KeaDJ}(i^R0)=;5B6K}En;5lUY2@!S`QXtcrP=^tGX zI@=?9zhKx!B=6O==>$P89ccv4gZxOMs?C_?NZ_+~Pw>Q};pJC}75I!~dgdAO-i8w< z29sK`j#q_hM%KCC762j;fkm(ARN=>kq}Bq0jZ%q+5Lplt;>(Js$p{g&wgwkpB!`%Y z5M-s>hab;By^5t_S*z5M`s=BugR5U5ai7SX#3qObrlL^=>WL^zy@IPQ3)ej74t0bt zYz|JFFLi+{`=|>1QBQX4k-!A0ST9=|)Ye3?4i$=o+ouNSEDHb6ec{@TiJo59+z3K2 zYkGM0>F%C~-R~cDXPhGDbd(Aj4?QJUERm6Uv!s=9gHk;U-XcWQ0?`-{=xDYPOx?tP zG)Z{LWgxL5xeOF1v3POy9ls2i>O(+?wThxmS;qoAUK?)>_I7EiMr4g`3y^I(0 z9M#f2OwU(Z_wiEYs9h^*Y3r5cTUv3B31`^iKOdR~Wd{`R+7eJAWju&npV=V=U09j~PBPFJ8Qu zG*od(&tdre%{Sl7A_$&ii9g_GI`Z?+f}fyA*#fNXisb0eMXIGSJ#9j|YoKg5>rEW@ zIt){$lxZ1a*)7ikO4D(iB|Ca113PEOe2v8G*9{KuDg!Q zn=YM~AGv)-Dds^iM@NFrBBMfKqRt^d8HR+kwY4$5g&)iO`SYc-aqI@iGYNT6sOLf* z@H7M)AL11JSd7sgDagyMnng#K&X`9a6!nT%yn@^r+Eb^UIhE(1f4)~4Chz&9B_wf1 z7vGT!@^rh2ZPi5IvR@K{6a{_VEM+mITUG5UqJ=0~V6}O&1kZG%dr8pbu^+p>p6GQ4 zE49I_GhE|1$&NuTz+Q%j6osWHnsN=}3NJkof|gCFt!^wYC$Gegj?Nj!4&Nvkdij6$ z_V(PnW80lCZrpgZBg@w|t^v8aBtaZ!6Ffo*(aYu#bbmJ2MIyN%F|n$;vN$twsQ9HQ za&V9pu#{9ZA`82m*ywi}$uT^nCzByIad>dx>%5NBF&}EK0+PPDDj%A~RtC&ck;gS> zO&mEAKL31f-h%SCUJ+)J!DoKO`WnF{^6(k*1e^5S`?|tJdAMa8xn4v0lqs9*?_{X} zc`fJ0z(0o9yyBX0YFqRYTl~d&A5c)!N_biYIE*UCyM6RJBFzx}AKi02$S|#lFR(Jy zLJmSwoy?g_dQSKJy5QUkSxG~z?c1B*w}H&GqGnXex|5Qsfi-wA4Ku?G4 zDYZUFhYYE3X_ATNgDAO*$dCk-8_8)4c{h?nVkpn8bB{b8ZrTKdTv^Oa8Pk+sd}iX@ zB{ESTRK(ciJxnyj2l-=~Zo;^rzCP?d;MQ*lPn$1Ak&Gx7Wrf_aJLnydWq{_*DIkbw zB#h_$J^d7S$Gz@oM|j{+&@3?o#KKoCa}O*J9(y`qDb=w}5@UkD=U}jUeZWO@{+WKX zh(HZ%0b#-9MUTZ+q4@nU-U?Vfi=-N;t0Oga$hxH@0@=Se5|30^4Y+l(pdCJv`Ni!_ zIVu0k8v`;a-uU|5)-9RaZYTLu_`AE)v*)JX_3oghkg1mxq0GaH>f(_K1WzBH2pn*? zWI_kGN`L1cE?WAS)ms^0sQoSm8{Tx&P0SSDwrv|vI+9Bg&cL*8r7>Q*1cc)XmEs>) z_+>Hg{=qk2mb?`%8P15is~2vZuN+$%r>#j#TV3VHb(vYja112bm=mJ+0EBs2kZwQG z2QCY4n*=E&6d}__%9&q6Tq9(X3F84%48tQY2D4E3!}TxmRD!iU2r}{lSapE$i@GYjw#1Tq) zt5z(s)wO&p8TZZN$4f28;#uRahjni=}agMcO5#lWn*Nt;5nx!prT+7x@ zwc@&QE#+8UYbDONH7Z4qY(>3_TN&ABYb%al%Fno@+>n9Uo_!BrWU z{w}9IZT6|HHLj`4gYe~uMO9jguS|;OOReWhYmw!{VVEKVF?f@NX4(jG4HY9Dl(v=C z)gs-KXoIX7#P~VnPJ6}~XME%%ANlmBKW$}{CY#iI-}~PC?z@k7NNZhb_DXm~dye8X z9Lw}taYDr`xaEuz#KXpQ+(1lxr%+db7G1axU zCdbxYe)e#6nw&BUNN7rxK@mjoUUk(~w!RkA@})x27Oy_+ zVb)cWKn9&l=3gvWu)wQRFM=#|Kq;joi=q};1Zx7fK~ou}72L~Hq@~^s74Um4wV1wW z7=g?PVjjneiS1|u%IkjWQ=fX`i6?Z95A`L(95qEs&C&8~dgyhBd14P$(t|U_}*mc0|I^xc65!rWM%GY%FksrGcTk!Qs~j0kQ|zLn)5<=ut2E z%IdEQF(jC5k^BXs~)6Wp!AbNZ9^-alzr-G&@ zR7}V#4?sg3vP70p(W{pNG3qz(11>SYV<0WbDaSOaw*kZB&>cG_AlQ4~L++V1VP_}q zNPq@Yvu1@CpC8Vil_)P`R=$KV0FSCB!xy3vF8DHJPHmB~kZ2)KgU)X48Oe!5NM@!* zEh8u)L5h%RKg|unn8NG7G@5rC!PbfS3I>K`U1zsw@;~Qj?u=mW%=|O!!hgFrc=KyT z9<8UJ%kSD3Hq^Q+FHme@vMCxwWLb_N#E0?YIjanu&9mH6cXxse8f@H_$Ba!jHD?bT z&L26FfA+cH+UteyH%3DJ?so&9`D~aTNNw7fy7}hB zNWJd0vT}r}sR!@`oi4^i@QsI0?+-liz~z@;uF(wy)Kzs}Y0E)?K(@xR3D&ckiI&1= zx+32Lwr<_ZuvrpwNd1c~pmyUN<=efgcuZRa4S!gXVk%Sj7QxcPp#0%5K=>D&C5b=S ze39ZP)9&N=CBqq3?LVC+))4*bhwc$l)u^MxBNBqEGa#>LK>9F(k8<7fSK=3CNs_Cx zD7a+$BG3VSewiI^lKUb#HwA1xR)B2xCZ z_n+QECmcA@0w-EPT7aKr^z$8UYrI@}i+fLBuYP5sQ#H^d6Dy@RSKRn~Ry%o+9##BS zApwhM&Il^y6}e1xHr2z#l;dpLf8lo1yREu$Q5TBZM?i67CVO`D;5?4ns97KW@P`>F zj<}!gPo__nrSQ*^<08??V|2zPmt3NyOcvnnZ+&u)QSwmX+Tn#*=2%E?7rpB&%Vw{U93syz(ckGn>e4RT}j zmu0)VI*x8yzkkK!K}$o+((@|Dj4f-an)=cIx$|q^$*x|@^}%JJ17*MWt#31s=d{M= z<=O5>$OvEIGJI5sDUXQ@oBKl;&+e&|CVqK{6>eOln()6H~` z>+e1Wu&Oq!9nI*B&~+zT{!_ z`WpPpv#VF^-8;6riUmsEGH(HCK`kJjZ0+ix+;3^pDP zx_U?kDcJA2Oj_u3zYJg48T{>auDU{>HQ`Za1ts0#1Jc9ii#w9h50FEp6HGR`_*HJ~ zM7$D?oXzMwvRArjv{0%pX{pL92W9hcy_bN1P1Kl98pOs=FKMF(uooH_Ovaa=9W)T&V_tHL1> zb7c6HaR>IoepXwssb|ly&Pf^nRwl$qmP4d--@F z4{OF)&}Y=Ac1M=A7zYv14^Q5fJlge19*q_w+EF_;JJ?lDpn`jkZai)^l;G2JoKHKj`RjZHyQ+(Ib-+XEAs|>$TFsT>dz@kJS*_8~ z7RTCEzWU?q+$$SPrS<@Jc`>S95iWmo<$W%q!L7}_}uMPpAmg?TN z8}eR7X?v*h%CR8IiJwQ6S4ocpQ#rclx{4Pe1CuSTtC+f18A~ItnpdcezI`exT^$2AhWo&x zr*oTI)*U*yv$ONo=hnPs;b{#r0Ay&h^;P_1>MMQF!j{hEHyk~*rL&{DyzII;bKY|D z!l;W|C{-M)){_||Z3(9i9EC!nKE=R?x(%I2H+OZkH8xd~E9BVbDjFI3;dl;&uefSi zKcU&U)Vw^0(0muHNu?Sqs=5Z#-5Dld@+6e16ibKVDat7zM<~{cE%p3}2xd)jhXxYA zT^ZhRF%0rP1~*^nW={_P?q9b^ePhH**_>XUxxSgsA zFM72bGodi@LTiChL5$*|o>ouv;?dUD>cF9h;DTH!QuS!^h)}8_k9hT5!Bva$D0(92 z7g4!k5{uyZo=BX!$gO|g@m|cM_`v@B{{4xD26x7pqOH#8#Yu0%U-J<@+Tw%!6YKEg zQ?C4p&UL{X$2t_jV^0WDYpbqBBFJPM-nA#}>6ZI8L8!SWxr@$svuBZdQ`#^>2v@8k z32(qCCC&ON%*cDOCiZ0KD@!Q}g2|}MnXjvcAN5}tz$*3B8c21xz-wzIS}~AjOo~hw z5-jz;cn9n3c7rT8lw{lisG=<8E<9~sqNzSS+!;Rbq`Uf}V8fR1xeYQK<=QJEeq1Ls;n5!aO1`&FS=!IqcrWKhaN?R zEsm!t()OesU|POchfu!aSTxJG@=9x|;#-cDv}nrr{&491fj&++!vg&|&}Gc}lS`a8 z1^x?YTK?o1j1fgQkXbHaxcEpVa*hFYNT)EZ*^ z2U17#`OYW_4-wM-{l}SOTu*pnd|IL#BYKey6ZNhA2R2pFQ5j)lCQGhASyrP_4iQ|( zmy3UoZP~JAWX;g00^=BoP>d4w(n~L8Ee2i`*Q{A%A4saMfZSPmw_%+WU9a)D$jETV zwvVObih1VCmG_D#h(EpsWXNOaSnn^)>;2QO+echxK!^$+1CVW>x)^ zikiM$W+0nMl_hJVbcTOG75Hq$SP4z|LI!wdov;808Epx%F_m>>KrYK-cVcA0KZ{aa zwcZXpq%Qn+;*Jgz8-@^Nz}clWZs!a!?XojrFGYHMdVH9xp>`~5q%FB&s$LiJDv z*$=DPYqW-6AWZSvTVNn&9ZJO#>Adj6<|Dh;wcvBaHpRZ@q{aok$pI_ zseI9y@JOdS+9kfsXjfkaNZl4i$(_nRm^w-3Wc2ihEK{;zE}#)sgs#-ujbuR%8ynoX zmSf7=T3MhscG}uVRTqV08Vh20V!g75g3L$j>FsX9Sa;x%>+Q`y z{II*}jcSn)OZnO76c%6I>@i$*<=Gc)!64+L{&4}Sg&i48As zhQp07_J8!F$p`Nb4;|9MDmg-Dzq%u{K%97!Bmki ze({S$G_*0T7GH^2^PA7|mKmqxipXf)k1MS5{z4rlBO?Ihg*hPl&+L`r$>ldSNTMqh zR)JxP|G9_9w2{`(Hi2y;ds!%dH1b-#xTNLBY2~Ozrh-I2o@hh>6!E$!o)I#HOl~KH zH==+w!_X2k#)ke7q7wRJ3SUANTP0RO_KXDCGbF($KdHPcBVR9Fcg?~r0vhq)rzt^P zxueNF30rv*jJkX@b|?Cgh-Xo}gTO~|v$a#&a*9$T{q?sVgls#jj-pxp(cs5rEKVsa z%D4VlA4*}Wmi18SxRp4^dSz{mOlzt7skH7b{>WZg`FMS`RDFwk_;M}A+sz_a+S;=I zTThg4ORcw-A4jt|N{=jG3fc1Gv~nzRoMU~{(m0Mqu=4TqRV>v<8b|UFW%C5IMT-h$ z*S}$<(pswWx<9VGS9~!oX$-jw5I;?Cs^c?}MjE2;5i-sqH71)XW8fcP1!q)O-u5Dn z+@T^?`NRQ6UdY|aZ2D5@qZizeSUe?F*Ow0yFwtGnKR`&Oxj2N_`dq4gtc;q1!O_Vu|!(;EMr=H>!k)>YwuKwtwkFwSYAgs~I@D28)q~c+$c6IN;AsF+& zb?a93mVD0$So~u13~8`L<%=)A$T5<(j+P#hE}s3kT`M!PMU_;9xIY%hN-AyT6~W@z zQmbqE)^p`s9E+w10~?=$i1XtdYcWnM$9m#9L8392m=p%v_|O?-`^&T2ay`H3>RCGF z^0U6Y=i?v!&OhwyVn9CE_fW?=+s}d&vQ<<_{s@Ni^!4@nL@ipxo-^7!dHTFdUtKok zq&;DELw~ldHpLqq|3W^4m-zjPVzhYyj<_D!tpCK+m71}t2iKY7HAV%gU%34?cJ)nH^Fb5F&6}bv{GJ|Lp7lu>Rz=}#v@n(X!yQJ z32>A@Oj2=VPlnD+A7`Q{5V*s;1+7z=527-5@0$XU9lc#YTfg>q+qSOQxA(25EWCE+9QwC| zi~*F3NPbXOSXg`vC#pU$u<0lx-3RF$%WG|Mn^ax9a7;@ok$h_ZzNZf!7++J@5-n$- zQQ1}%qUxnH0dAD&KmB+p6WZ5g*XKwH8d}!zPDRp&I>lzdm ztzpiuJG6g$SNDv@#RoL45Qqsf%&4}0)=k?8N@lHpfV z@KdizbW}`aBFPA+a1Fymj7y3W1T8Spsq)t|Dv>~R{vrucm)90~lYs<)JzBK)l5^bF z?cx3ROUB#knsDx1L5^gQESv~_p$un)S>#KEAOeHQ6qd}6Ef%mDfJ3@ z-fTrJy7nGf9`p=^4fSsR+(PZR38Bci7k-yim-!+nR-U-@$Q;+E9Sf1PZf zDT#8*%TuquHqkm|;7eb0_uh*NEB$6~za!EGIgzZbE`Q@2%a<)9QE0&%5fYc!TMmbe zie0&KB~woD^M^nDA>V{P`N>Z%UAmN~64wkjWWE*g9f(9`?|Rp}KK}8KD*_<8aad}@ zoNfqsVb6g{+(bhsJv}`nJfa&mBU;=9VM>u~ZEa=pAT;B48P!N0DAJeW=l%EJPht)T zWJp1!K^AB=q`H2uXdJ%e=6^3pdLO)&;`>7gYtizD-pMWZ&x)4O8Emwe+ydmk1XF2A z(FpLpeVhE_;Sw5QSxklMKy>=7(^hMl(fdEU*%c+h!<-{8DAA3YRgr za(;Bo*5b^_Z;n5_Ht?{B4k(hlR5Tb|?v8wAD%?@TvlH8i7C6xY|Lt1f(LNa`&vTux z&fWV`8h!Asz?YPo$^Fu8iJsG*uf#Cn%CYH0Z9bHd97CJ*7TDx|RoK#7Yl3zL(H0Jo zUZ2jKTXl?9QTAp0<-ta>jF+Jh69s`W9it!C;Q`2T(AL)G4_}hte$MCL(LW|%67Yn; z%WZ)~#kjDlJ>9dsf|0-si_DUSs<-0c-5tIA7QgwbpEgbS?k|7x)76aEjb0L>$E`P! z&1McBJaFcji}~QQapMMF!V7&D7JaO$ZaZ`7>ZVq zGSc71ROPGMXZ_oj7grtFf7jNnO{vPu+h)WSHXJ$h-RIY>*t7ejmNDO0b{X@yhuZ#F zY21iTO`)8_p1yG3*8ElT!uz%+H~l>L_;tae>2j+1_lEP`(v#gk{@Q){LDyOtE^BZz zD@2hGp1dej*N10Z;WU;3)50dpgD#m!L>Tp0^Ti`C457MF3E&dJ;9umQI167dB!BaZ z4h#N^RwNUZ!W@!kfCkxqj*y5p;igSbOq?j9B3}fQ5do^8+~S@qH%hcAP;8O8Fq2`u zRkvnM_~>$yfQsUr+2J+CgVN347nrR*k;GflxNqu$5*%k zmeO|anky5nF=XEStdUjA>C!?R`}@3;E@&X)VaaZbHr_Akk97AAKejd+au#;E?5#&N4G1nQT&CnO6%-tZu3s} zwI36IaWkg5lM5q3L>)2plD!_X_Z$dPD^*;3{lM+NfX*L0ocYO56QBNcL87X<{L)Kv z?|onTAO1m>IO*z3|Kw+x-`tUyw=gmPWXW;5Z(s7%Q>*^|^O?D~vN&tlH4wc=$|((| zv{roRL#b=832Gz|Ae9+$BUAM)x7@;#;iOB)XKvucnt4c(M}trT$)30^^PmLdbD#Sh zqdBg;@=A**&0$PZm|9M;n3Gs{K$U9_; zx)jO}BeE6k#F$`2I0ZrrB98K{r^nDLqb#eXwDPS?T++%|E#;Tmj|S7~s{F_v#xX4? z?pvIr7EAH9G>&OG)@O@jWt7&^k@;~M%ZcOL(s*60E$e68pV1>*N!vckkNaco+R`|V zl~mfw+tO0?`orX{mZhylrER;FDlJHAPeC4+g)a;iA+Brfma1JG0WJEe4@S!^$tU3c zAnbOzilM9-qLOpfZ39`tq0xJtH;ok?Tj4nnG9ql;V7P0jE9pz44;zJ}?(LN=(L`YS zvvd>aqCKBUpQlgzqJnKJOKZ7ubYIkvzm}tIVoT#3EiJ{@(m1B&l)|?%N~@KT`EeP` zv3{1qjLTRLZH@jgWQ!k1vzT$awlt12vb-&|ZD{!x-|~+uUshh}swNCpEKcTLNT$13 zQkC`4`CM1u_iXP#dcrHO{o9Jhx1IIskKFYyk8E30!CQhsq<72Fjvd=suXOI*lYaf{ zI|wUOR8}=iYF~20+sB`LYE|<%hD7EQx!xeLHy>;`+B>gwn(vXtGfUAySUZQZ(c`SRuT+gLVLB~{&Ofv+Y!Q)PxReQBIHE+%4ezUo?h zOWRWTe)D`>BHZ8|Lx3WG2n|QuUsZf*sUU?Ss3Og!x}NDlJdSd#fYRy%v>qy;r9FaY z4&#KZv^vlgr2BwU8IP$(y}TZZ408#p8mKdzp!={)HsK!2fAaCY+*Q>FWtEf`aO%f} z6$}j%?T8ZwP2(h-(K5-)go8b@QY>MS;+L#|s1WbRk;%9}uvypR+{shEbm2vxA#qG! z{|}#k{@(4|ml9E$);_hqQPO#GMxxmz&mG+NZ`-#$xqH{1{=UjYa%NNGTj!s0>i9`i zq^hj^%!!l7)U18(;K4gLZ>Gz8wQ>%DL1 zwmUX#*wNiRp}P8v@e@+|!O}w?OMs>h6xreTA9Snsx^Mh-**gZ@7k-_8+qd0i^W0l5 z4rjN@(|BT|d-uhO(`v%stam>;;94tk=Qboyu67e9g$pkUSfAIwB@_YWuXxRfpF@N* zhHf?9E`-+C@8PgUU_5Kukx@$QKiS_9P7b*}pnw)= z+VB?4GG%?yw~xq|{3T0+E3Y8AiKs>O)oKQF=eT#iIse@s2HkzZGtaqI&&sbPa$Djw zq+5CYb>W5Q6iz|pv8WS$Vx6C_e2eKSiF3|yJo>Jg#Y$nV)?VIN)s%Z?_PC@ zw2bEeuz$ zA#q6hw|6D4y|!$@0#ymdv{e4po2g;?tN%!}0u}bTC!Wkdxf~i-atHU9oqJy8$37O! zoR$9SmxImQa45O#!paYPD0$|Y;lV>xRZ@dQ4irtw-#CXy4+*MC1D_W6egQy6X1+p5uixjFiiECvt;K*dFSz?il*&Xa!Hu zCtBb?vIWYT+?eyTdyXW+G@0Ls+yz-Oy99NeTl&`hDtY;RDDZOUM}5s#zjW`>8>z*vOlb? z4y%X1&m5;};Im!{-@Eu;1i?yp@i7`2GK zvZwbJ7f=pH`KW~e->;Z6MTSuZgOn2fci~fCj53Xj%4!q?59!J?e2GK$+O22imp6wB zEkF^3j%Q~4Fpa<=d-x1*RuJW|S6zyz8Z*ccK!qVK6!HC#j;}V3H!%Xi_%G)B;4_W^ z^0L~8K!zLaCR#HB{xL}wbwQI*#S2O>lJ3ZJrm}aoX71L*sdGx`ZJ2LrfAEQXj#)rI?%Bs(wzx1om zJpK629ZUdubl1-2ii(EH6l-pE4i0q5OqOtVW8+&+UU1=*X-0Llq`ywA!a1!|vo2&w zm&bPR{LXW0e)9af=1L}xB?<5!;)8E4OJ!%&)n7k*?rUbvwTj23^(rl1mhQ>TZtCPj zeT{3VabNi>_rUYv*Z(cN{>#BRv%{N~y3?ly{R3fJCcL@L?HqK!KN#G$&($W~vgV-X zkefSMK5#0a+A8`Vbady_eLP!-p3l}U!g|OBtcua=x|}24fB-qPU$TM91;=qAWFUeB z1u{7|p)k|}d3mX4L4pY7n2rEH#5r>Vk~vhQf>TZrT{eY6Lr7r&FHsUKM!#{mtdPPv z40rF5Vv&?rGI5O8>V`UZ+G+0mbApy;{iR0|(NDfuaX0Uz;4`0ecix?U^hwv-C)XWt z73IN_(*wqCG)JqtsvGR3l-z~6ux&D_GZ|YEKDC-U2C|@JAZxEqp5#98j*y`l(T2;? z+EV9Z>ME6CZJIBA!hPwRZq3H%bXoFaJa+#5Wl&2(<4f&{Gi8bHW z|LKIJmhetk=ajD;E8~Sd!2^QJZ8VZzWf)=c<2aTRr{f$2x0u7s`9mwCI94~#(Nb$M zj-VVX6W6jFr7g!|TE5cp(m2O@ZfTX&z2#dUtcIxk-LXx>dal&YLG zfPg&Zom)=F&444RHL~?mTE&Ld0{J+{%3GYcq&jA0ysMtDT1s23Qk>EFVC`CYi*IRb z%ew~=*DWO(N3%XyOhvGzaVr*0X=~TYTO4b}BE&7mB};J>ziN)8(WgV%|H4XZ(IP8d zR$ft7KCZ83VR|rMF|ad{R<~>fhSA{-Ns2FYUZ%^M!8leQ!JE)>U`?VAWln z{oMn_&>JAEEkZ}???3z59qHtjOjSp+F)Lr=Wag(N`ler;a0B_UH(9x_KV2V`*HsM7 z2eJNmKY6d_YtqFRUrf(T0}AxIe5uqc`kBC?mqt&HY%kD^*B~O`o5jm`4xdt1ty+a7 zMZAi$x0oshheU3C*CLq0Y-canREaf^CL_}yrm9`Tc)SsHz8%*z5zer_N&btm?QZ-Ym+t3fldCzLy(9p^ zYytVx%B#XLlcU#N3P`Ro*)@(=&sEU+;A=#Dfct#pq=<86?J4R$bXX^NQ*1vzuD95m3Y~d*5!J_8bJ1K4n_|k#+8HSJ+aE zyjwQkT|6&%Y?J%Z1L4iL1kH8fqOtCj&`qxj+N;C2w}$z~{My>!p4H(UE8Up7;LLV+ z%CvCalweYmt1cIdJ^gOei@6=!@@Jn(urIn&v?e@%i@^)^M}0sg;>#6PRTZ4Klpmy& z@E^6K9=z})U$`WhJ&|>jTIGQsX$7-p%X9=*U?pVWYpuvkHo4-;L|q-FBCGY4>(+%e zHE!9m1bGR|jhS6;?7f)7#M%nbo;=IRp{ea~Cm1!ubl9&>y51x((bJkc#V zjRYJLgFp*^bgQfkUUzlS-IJd-MLbs~RRerXqxoW1yh z?EMdinQZ=%M+d+1mCBF&tt7~ZwnSxB#buY3FJ7E}>Zu$9W}aD<+qEm~9}Mbilf;~g zPmScc=hmFQh}&^kUXk6pW$+)rlz(z%o@J4rUmtwo3uPBw{Jae_PfyWjn8ZsrVy zV9LpsEnCPBO3Ziq^yxgRPCDr%ADYL1ZoFi&;3iLqnP;5Vh?>HmI(6zPr<}sQns_=^ zcwljx<*ti5&!)ItlDIR|gc&6a{b0%==I{hS&=7`r{p(-PBor2~V;l!V8hKn|n!w7U zMT>aIBBPRu>^)K45lXGR(w6hc3LdyvcNd&>y5%cvEyg*8^Ib~Kx_DffVjsx$zpw&< zxG~i#wJ(KlajZ<7R%Dge(l{qxYGsrzrCUnUdT0@>Eh|&XbJex`xK_M0?yZ$6Wou+) zMX)k)%(#rT6_>YYamhHxA}DQTl(s&U;;3D#8%HZ;%i>!Zi)LwyY~@SQrF@Iy#3keP zwN~P#Bl|h>*|Er_=u-Hlkga?vZ`Ibwb+*sMxnO3=cKtkHwdFfmB z9hA|((Nlm$AIMc4GL$*jlx9>Z{0yIBe25|%E6O(ky3qRM)F*#-AQ|?sWQ08IBkMt$ zJGyMj5sJQiTYHA=#=?zC8xo0`<>oVE={s<|eFU+HGjeN`l8m>KO4_}ZiThBB6F>W< ze2Cjp&d4>gURh*|R*Ej=TO21Y8LzLk5-%OuPg}>hq(v@8Tdm@w{H4PgYB!S}$nSk7 z+_Su*uS+g-rkrq*Gm<|@9UV8n@$_Zg_rF-Txrd8(OUHp*p1k9U?Q0J79?9f$4V86g zOxBYdf;LbLIS;baV#q1W{arPG~SIIe2Pa`ONB}buO*XwqBNR zX_c`YrLC@|&gM-q0}XaT&EfGO|T0MVCTQ`BHVXS}Iw}LtAQnhR-3RC$@DQ-PYB$f3R;blO{1- zORB1^zP`P_slKw(zuPOwustuAy_xjRuFkIXAlWpUYih<<*07R@^<CI#+lgXy4s>wC=1hbBL0WDs8 ziaD$nddQxaWL{AZce&SmC2Sw#K6_&@E=qL~6-h039&)!o?(SXV_V)#3kepkcpIIMX z@`}W?Nv^Rr*s{+(wB9X$(QV!n9_bNNO;!1pGFM+7PH72Wxg^0NpwX+8JafG#o*4E@ zd|W40n0@5Z(34t)ZTqMsFWwMrUO;J8hekzd_G~_YNRfYOS8plxZmS{Xs1_VA4@Z%` z7>CNS{-_vy#X%Sgc=UtY+`r!G&Rh_D>Vx=G==MbT+G7z^^;2~%0!9ANm%@xppkBBr z)M89PkYTc|+mwIP$6QTi@UwpmCXQ8gDP{Azh$+o}*ttWD=F6^J*?-et=2(jbhbOd@ zzvJDN?|c`LBq=Sne1RrP$har=1Ywzh69q?91_8G0=FJ0t|JmGce;W>F>EZ?Fom2gH zpGh)6W5mt7>ZO#!p$E4z(7GW2;UmIYQ_dYs)zu=R9&jkt&0A3drswGR3vDIkD-?mp zp^N|kKmbWZK~w@tD2~1Sysi=|W_2xoymXklKdhR&SluE?ziRm+2#c?LYvnjyVGB{p z2@k6MZ)Zjl%3B;;Un_6@kNaRH z)p}gUwtp!<<9y{Pn)TE2t#yl_d}}w3qa3@BBaB>X&!(;W$T$|$(iTDamNRm#&x-TGjWt*(_Bjh}J8|TDr z#W``iagNop)|GGN%gR%^aVKRNX8huPE=`_6LJ;MWqdN@icW>+)H@)GKSw||iHay=u zqiNE|m%Z(U16$W0+TNcTm^*I9jHam*>c!QmH zaAFzs^!K>Lq+Gr&o8@BD>&U5k z8xX2x5wz5{Puw4iU`yk*C{-G_TU}Ou@|bb+#*Rm&6i(c30iS#vwqW+1P@|c z!KF<>JZp>{SK1U|r4sisFqw*?a`NyRh@GzmOnt%5C(Px{rqO|2$H26hz#$V*W z{#S|j|0w_JFNJTpFu49K*Cb!5<+jn<5?nGTytFSkGMHc8?KZgN!`pmJWmXCOGzlVA->(k`^Mgzb|O4UGUqL&KtXLTy;}LF;Q<2|k;?E~;bf z1P59@R1@k9I4CNz zA|M9Ao=qk7P%TC>@FnvoNh|c-nZ1^(idvzJ4ZbqjbEx~MM0V;wMA0RYVY4a}!O^|) zqr>nAcLaR{?(*~U<6E#!iRdCGR77*AoFcgVrHFlj&oU7U7gT)e(}N%Wkc_8b9g~0h z8sDZ;?|VO!@l;DCMT;nghA^UnV{nLN5Edbp@yO#iAyXA*&8YmF_a^4eQxLz8`(rtZ zslpcD@>O2XmxqPW_K)+?WsCDw-r7YS^(9?~!OnU29W&X=bhd>DU+Of@!DUN9jsp?h$Be|AT{g2^kAK7+n`!gUA9Bl$evb?Z_?12jhO;R(qTEpVa* z{;RhDcMM)y^u2|;@Vax-)bQ6IcHg-__}OFb`}YT@w!3BXf-~A(OUVYJ;nb$EdV(b2Xl_n6jY*I@1!QjpL;8*c zM;*{ZrQ=O%6@95yj64@8WvaZsB6<^Q^te!-=?M?$%_&Dgu%dh=&{9R+d!`DjBzsD2 zk@8igC@CpIcvuu(I-^EHWtgHpR6qsydylNT{!j@Is5tsM?Ta>Ubq_!7TI$2Amlo>i zJw!%z;n23;4b>gKuc}U6b#=IVSNf}8C87}?=}dp?Uve+3ul&eIlk?^=p=G#yiDR-p zNlb<5lFYK1)>irPPmn8g^y(#0Pq1Hh3mg;9u-AmrBgbR7Zi;gmhkp6EAnao|MYDX1 zue6mZMf)HBvGB)OR4Z|OYu(ZcWjU&A`L@*3aX;gBEzU5vIkfUxYB8<6<*N^tqqLQ` zwDs9q8TtIGWZahJTPxNF)v|W2x0Y}5OZgL*SG#d}D`|bR^43qK?R8~KEng8z;aeP) zv^b@-;`nhKTMLV+v_+1W#@omGW_4dK-}CTwK%1`isQ#|tSyVFv_+1W#@omGW_4dK-}+g!s{svA zhrV9@kt{7k-itDqPt+t6osU11y7bD%lP2}lROVJ6 ztvHk`%gBVOC{Q5;A&qcfI@{7;-dtHxR(5hjVq#VHg`UCZGQrWXJO>&hnadLCFx#CE z>x-QVTWjl^(z+k!C$+Fz+E%v3Y&%=bxTNCf-byNM^- za`}p2{kNEOW%|i^B`bmiNpKRC zspwurPR-&wx@Z({IJP1xK+(wGt5H!&_Ov;BHV+7mfafcIBqDq(=~X-*4x1u)LQB~z zCIm(7*}SA4vKReg^nob}Dqu_D3}f0-i80G)0P27VZMPx2st7(R(2*(emQ6W$sHKXj zo_KukkL9E5aTNI14+c!&ojt|PYAb*#zT!Y*#hX*Y7b)SHL32~(+uzAWkon@5as!#L zH=V!hp6tpMWv_a5>aX5Ze#-G@i>SGRZL(CHc3SHC*QB>>3!57%Kl#5ZmR>k4WyUah zZ{7RpIf`lJEp6p}MIgKna@AD1bGpt@SBTBUTYZ%W7QS{!RpX)AwhDK-n&u$8p_ z#H}9_&NvqC#Lt<{++0b!6$Fl zoVYK$!pHO=Mx`wp4Re=tRW>J9-_KVn8Cj7pbD3Oy=IFqlz15Rjt6Cc}RjKTT&ivN? z@;(V<6o5mQ>~i4|X7uM}C%=>)Q$Q>*{EDTA#|bz zPPD*(krv?T?B8345!9f%(p|kMc;)GQ8h0=34eoz5+_fXvn{>~2xZD3vaoT5;Cz2QH zRFEH2?M@o&W{wYMP6(!s3nq+bVu9X{WN&Tx=252KE1yG8Sg(}tImjrjgqC_42rp>~ zXo}d>1ba*SLlyITHKC=Jd0btXD&w_0tVe(twnb19$cS2W$UUHt)V&hks2YOlIf}`l za&$jz{DsQfT0|V#?LOchSqWNr@e1Q)?>|r{n&_@#8XzWBpexD_P1`od1Vew6>JC{-_n} zRot$XQQG4?`s6ArZ;_Rb`)m=cu9X~(KNjEGwYqUS&at|2OqI8OT6yKiF|AfyKCWds zO21rB;yzfMIGUDPyQO3--`300Dq}INR=jjHb}fD!IgVgu;&j}fIR0k~;f(MT?|G~B z+0v?I(Ms8h^WzphPJP?#{>Cx=%YR>gv|Ey1C)$+0!*>ST6oPwv^C ziXN9_Vb`eNQI-kv{r{J}GXc2ds;c$h-RVpO6%a%~>Bc}15W*Nh1PmC2Km>Ut5m01$ zz$1c+3^LD108zmwQ<#xS=6MboNkYODrr;x@5EMiM1d?>p>G$2c{@(p{?Z0l{9x~xA zn#!(q)?Ry`bL+pi&Z$$UmM-||hP8XHUA1O%<8HS#Ykqm}J+ED{_-4XkC&F~%Fd}nT7dv$*IG7{Tr zwo_`3eLFLV@z~axzfD{(+g)>Z#Ic;MV|{1aIY!L1-Sg@Cof8G?$IDDd>3o;&SVYH) z?R=u|tQ^s{qmSuWIW`?#hrsT!Kz3WRwC-w~e(&d(u5jS0EAP*=3usF}IiF#UP3LV` z#k!-6i0-xfnmzYmO`b_VyhNg3aWSeWG!CoO2w>$FynH!iix=)}IV?hq;1msV0GT>X zyx5{sz7z!sCwTTLLe1oWAc<1$C}U3}EbrnNyowlQ#?F39AfM^n2Zp65+ZKVX4z|Es z+9AqSl1#FXej#w87G(3`{hcM=B{SVBpGlEUp_DG|bH&vM-`#j@`^HX0 zGnWwl}>2KaIvGPNQ9R zvKl0LaUA~hKmYT0zVjX43&q={E)M-mOmnMW^-C=3BG94*c4-Ub+Hz-3=AH4~`$|{Y zc+h>${^DtSo&L2wxTL;l_0kW1T=iSK_a1Amv-YBEUGdzn{BYH`H|%lt(r(jbKc|zH zVQslC&ZWuTyYK#MyDu+%n@g6GS{REKShT?ZkQU(EfIt5gzX9?^K|7@Qyt9_Bc#Wki z-I(vN3`uOd;$D2KOl+7uACbPdrBYP;L6d^B6_`x2PtYog5%Pdg3T3S5JCNeAbX)`& zcMLXb+6BA_7%M7-Uqx}rVPvhZK&!4heUa+E*Pc8tSp14ozExBq$|&WM#RwP!W+CMT z#?aP}&s}w=1D1Y$-z63S9qaIxY;>eBZmwOs+bwQ+g;%_K!^b{)!K;tj_{A@5I)CHp zU%bkyeXl$2vnaW2wZK-x895{SsZeL?jNCgXI$bj&z4WP1ed=HT^Nv~W_LHg+d;Zlk7Zx;OMzNE_qosIt-lP=Krp{J;Lg@|G=pHj@|CZA!V{i=arW6~ z`#r;Cto|hym7UT*?7oBheeKTDbM>HOO_8oU_v9pKys#m?L*P7Co zzx?IFfApgt4O(M%4vyaA+IOt9b4rtJAOHBrMo?-Wm0%zvw zr=R}0&wY;9P4lK}Mz61Rt!wex>bu_ct_L1?V2=_VClA8*d%yR4jJ4hFcDFnF=%Xn; z_uO;;;UE6tf(tG%6PwY=>s;qL>({R*1h2ME>HOJtzI|h7KZsk0olo*9t@&kEs8TrH zciC~OFuSx8C?m>_)fJ44#R}e;L9*|hUa!uvZ}>fD>)5th?Ci%lR(4%_4o14BeaA{W z$C}Q>3g!g&eyr4Lwr4B%47;zR+wPgzH@5lXrIc*Bg15`SUNKt+bFXyMTb_5_U;p7Z z&RqKU|6F~YUnwv6Yu2veKIuNUxax*qJMX;jp0jk;MjkR*d#&B~y250c&t3)lVvpxM zA==$laGsqryR;H0!#M=(3P!@Q*H~^=bDmb~zV>ny#~s7XiET9_j%$vUou~6RXH3(+ z;W%PSJ7;#3vAE8`3dZeg+qre$o?-WO6x&_RzOl_8EA32s+1}o4&F^;4e=L@LJ15AX z7Px@KvD`rl#9+49&tGlz+I17NIHf;9H*eT{O$8~Z2E%f0>0PLnSmz;X-@f-&#~ z2_7SM)WWoH?8vSd0)oB558^#Ifuayj_LWJAgj0Bt$pJfgRyCmo377QDR$Rth6)jWjhhgx2Bn~&F^;4 z-wK`YxYm#D((Y?cFWo#yw&f4jne;kZVXNT`yIsQe$tR!uw}1P$1bxnPo_~Vbi?|tvftO_T5(n%-rz3?}G^EZq; zyzqrDyvIH6;nEfD4}bW>%%H*Rk&k?22>h6#L3ozVc%q`&jsU z?X}m6nwLDt8y2Q>ColQPi-?OB_}OfMJ@?sXkNt17V%_pRu=|zvT>A@qT(IwC+;cac zC)xU{bVbJnxM+bz3;a)R0WQ3~cQ~Bw*EO6Fy^9E<1qnj*UL%Pv+_<&q21E2Vh?eML zjNWHR2qJp#of&0}Akju24AFb%8^8DYp7(j*@Bi=bIgUB6^D6t?d#$zCele@G{Wz*+ zyAHh1CedzC9j`R*A?47?esdG!!Ymu58*QHTsS2^D)Vury(eO6ulO_=;kw{Oy)0VoP z7EJRij4T3?{zl3UutGii<_Qv1oR=_7_5HPw6m`X`{nbihJ33YP+bMxZ9HcRPH)$$g z9nfftfQmK@%jEnLWiqck+Fp<$J(b>gNh1$M?peE?;zUlRpU8$^scnGD{ZJel)d`0dmaNpa_qHmZ1gb@-pZ z5xhMRv^+>k55!s6gz8`HQ!K~;{1htYT>~t3>64v}Din4%i&D8~8wRKdfi zItDM)PFzPgN*%&;Cn&qroMj<(rnjxjtA{_mn1`-%Iz+e5zyD^&I^M`hOO>GFXBZHE z&WGj|DWG5uNL}`(1NzFsk`sj>?V5At-S&U~vH-U-OYC~H>3Dnnut$!3&(5P>(tT2! zVCPMA$>#Up+g7W>t(#Gf?HI`Pk}MzRTw(kty4=SCym2CGlw{O5m~#yc!j@Qi{*> zv^P&mZ$B!~!3~%?KVmz~KvXirm55lx(nLcl?+#t*mu&mds(bxF&3mw(SN@)a3MGUIJl06F4*GK`xq`m7#q-G5Q)&fj+opc)M0z=iciZke6?))mKG&yq-j$s&f_E|)CcRiJq)=SIWG5YTRe2a9qlf@_6fEO0l)5(8DEx5WLrWQ zPPSc^nUP@3z58Mz{2F^Zro@EjMvtHdQV=~0^0;O7dTqn0#1NnE{ddaq-cafV#>9_8w-+lL_m4 z%RllV9D@p%TZfU7txw~VU;SuQJoQpM?d4YRxV|_hSNYmwMZ{*4)=Z4}5OV4pLKFp5 z$uKhTbJ(h(NRnPjcteRZQ;?n8&_=NMGKVqUwQgn>(@5yqayZo2K-H>>Q)3BdU z`*BZI`gV%SLI8IB&z4=`Bj&zyv?OywN2a_hHJ@skDC`d;)iWgQ<+*VeD85@qdO@ii zfb||m`N~f1AlV0A)T77GGIpJVTbnD;PwN54(6WcXN}?=iv=$m3>vJ`OOHctcW>?P zD}Q_!asmti4XCs;m(Ak4VetK9B-ZVz&AsOh3ro!FSE06=%6& z(csEb^FT*#|Lx=L7Xd?!{efSW+$tsmXUHb<2n06hGr(&*zfMLcI|8Gw=(Yb-qvR3L zzpT7V2l(Gx0ZkY4D?+9zHJ$4~Di=>TmhE?QVl)xUJ`h=Ceg58f?u}T3Kil3XA~Our zYx8m zZ~k5)U-?LL<5d!ui%sxA`48yW*iwR|&DBuH)zE3jZ4kXlBv;nCj#QluBUi(F za>u{*7jS{YdUlaevJor4%NV@C%{KqBM~cAhCF9$e=N7Fuk?9#$R|C0MBmxm(jVhyT z>{qLv-61&Jr?s{hVR>Cocf#J2AIXN$hA`EAftrpGF*Q`1II%0&37$$PR`IapnscUu z(B~uh%??cPcv9VV-lX4}uCpQXb((dYxuhchEfA4$QoSAck)QMAi@afla0RQA);CPs4hm5) z2b%Ry>1lvC*|wtXn^TeAYiqvR3GSu#0U(_1!I>N=PpW6SNqM|(Qx`5aTpP;CB)PyC ziO~y~-eE>m%NH_6lskUM+Sg5MZK89^tlkCmv6f6`7{!Yh{@SWq0HTmrN1j zBJ(bya~k$CUWc;WYEp(Z@-5zA$3RwDUFg!*Y~IGqF%KZi{(TosT=p(`k4JHvvo8dI z+W`3Wq}Z5O&5U#5vCD{IWnr65{2wP|38K2&Hf_C@?Q`S4-*K~S?m&ybK;QQDZiphx zn0C?RI)^a+R8*yNh?DnXgr0QW|skw!(Ty>OQmzpBB-aA)sjjr6S&;G zveA{*&v#RQa0SsyJ#c$%*B=&lpA188E%#S9J}--$@z+1T9hY+z{=-i57ZVL@q8$GS zTOMXj?j&qjB=U`to&BsWyJ*bhF~Yir z&vy0-PQXB(2l}0OKNQ!Ad5hamwxgZmmNgY85pe`L`9wDtpTx%*-k*O1k4r2GzR5hb zwfv)AA5FzncGPexc3IMK!S~y9Eg5OPR+x!N(mRn=g?Xu^YI4s!1foXKve3%6Qw!PNuI$o8m|;7xcDwH~6%rwA=4! zRJG%{%*`$%U$0t8N`env_~YP#>fT-t754xSm-<@Lze(OVm}TlIF_vw)djkoq;+v|U z#xrn81MoRnZa43ET)BdiaBqZRT!1G)gV?lq)tZQMkw|zWpwhTX=HJ;YOA#5gPT26dtgqQfiNey0$oPHnHrf2HzL?8Nk9}aKOZ2vhn|2yj}n|59g z3v=;;W{YlIvCcuyKm?psWdH&U>K8KxhvH;Aq6bl;XL1E|POR6cDu@e^cGjeiaz~Gk zD@~JSuF=KIMs@wd^!_v)__)yH2^3P;~OyS}~|T33%GSzj7= ziHzi=qM?rVpx6P59Cxz0H;uyzod)$8*zZ0q;dRCFIXiGz7n2!r(3Evl*}5smA%k{O zy8aQDQuWo}j2U(!i>R(2)V33n);>BPM*~Ua?0UkUe!4x2za^&SwWs)pOyXjmom_M^ zLH>`eh^B)W9UGhZbCnbZVk;)R_}9Ny*BBotAl%t2`*s2@Zozi5%k&M4bjuI1JHk&k zMYta9Qrq$T450Y?hXgsW#?mlGEvc-p@oedr*r*zq7^$q^gfWuS0p#$68f@I>K5z#gTQ4J?|;U( zTp0b=$zF-3sCEy4vrJDNW6a#?Q=)k7CP0Q_XQoo;EU82x7kS41M;yY`Q~6qgknCRB zEh6L!aaNdujx=*uRcr_2{7z*hKNI2lD-uHIMINp(f`6QsnjUhVt{!t3R=`Xe-<8j{KkOH!IeLqO%DoP{(yJalZd)V}jD9>f z)19nCX}-x07F-OaQ}~!kz3aCBnKA7~RKC0|U+E zdQkja{JPh*5%tE@BLFMA5F_Hj2=h=#$kbEPP5ZJ~dGHx-t-wkA)?2}wSf4B>pAnj? z{z>)6x^VYWF$LjPo(c_6p?EEDv6ffP0$Nooz0^OL>!y+He-iPF^VuM-6=wIr{=C-Y zkl%(-#%nVPLUE1n8RS{t+wbNJ@={n7u!AqEK!nGkk^g#d*x4sk1;Y2{PTQuB7KRhQ zyq;u?{d2Bjk46C{L<^P>J9Rsa$FhpirxOiXKA{0@1S;m|HFg$H9ya=Yh$kBKdmT4w zoff}3Ql^@#y5Lb}HJizhYc3dIA`S9}T03u=T9faR;)c#oco?3D%pZCeyYL+~=E~i0 z-30v&PsVi5U*Q?`4%d3NhBjKPdAA%Qyy1I1-mofTn44|=+@?5{A~V)a5Hlg^8nHNi z87PF1f!LuKT(5;u8SNfex7jQ>i)3SFeHk;ZwtBxn3h}t)f zd)Gl4RRnz6kQ*TC|MUO1<$$94Gl~^E;)a5lebVK&H~W4%qVKjW@{^Y0T>Z=AN#Ce8v)Z*mCgN+cYj1tgFE-?z{U zcMGo%NrN3xD61KQAc=$icfng-hQ3KAf}e8EzaHI{OXkGp?Z{@q@Eg{VTd#g+yfy4& zNEa3{b)S%va@LbsDsmv4rZC*f%Lpo|^x!*9?mif3H6Eu$CE>Tdk)qO+Ao9cBn}JPw zIhvDHFzm(EvfvFzRH=&YA|-=*l6y@uGNTWAQTr6>{dW7cc?L&UKTWb7*loOAC+!i3 z1rA;N)wC_}@3*iBA;Uh7YH|b}Ekh#Fz)De8{Nl!pUEcccxx!6CN6_M~~$&u5W~&Cu$9U-DlK-F2t@QRs0W< zbCH#t`l1e6H_(ljBNdhANQ3mXh8d9E$4pL9gD*w%8Et)x{gNcPbVU^TMqGpEfti}V ziN7XZx<{EVaNO2MK34soG2}n^39z2P(5o(s%&^=9;6*Z&#`Gix7F352=7wd3C_O)Q zDVl6ou7oIgHx3|Kg?5iCZ`etF$&&Ssib%?rZ>%3M#!DUjdc#SeC&N$1q%i*KrwVmO z!ALS5`w@wM?b8p5)i)T{Q>l-0KlaHrrQ|3QRh1dk2YTrC2NgVcZMX<-zOoqGh30TW zwO({RyZSP;j-SjfW&4Fl@~=4)TJ1ngPeOgIEKgm;?<_{y^$y}zul95&>y^NJIfATt z{)`2q)W#5Rtjk?hZK<2w%VWn&^nx|J=f3Db4cki3)3A4eJNV25{k1~MD+AZJV_@O@ zId;F;YV>eHSC4v9il+lqRoR#-RWHYQAcdMQ<#CY-p&&&5#F)4%ADL4W@K;`tE)~yd z0={NT=-alOJ@!L*KIM(V5qp8C5d7~GL7UZMo?0PJru);fhObnv8LpOv1tcu`x@QVn z4Wf?UF&c9b5mv5?%6hjpL(t<_A96+X28o2uCFSEw4aLYer^^3Yg)16mWx9xHOfI78bQ1`GBkoW6z6&)(tG^1wtj5bLFOfT|>bOqJ z<_e=TrF0H~4W@L}@##KhOma}$j4?gr%>*ORUHmQ)NMCTkRu9IY`Dw!_YettvWP(Rgs-cvkci><;zUe)zA>V} z+>75!w1;s>=tAD+3^fj&-k$KPhiHbWPWx;$I$0Coyp*=71-+Ufo2wV_A%DomTd&gg zIlgEQPB^R#jc7cdV8pZpnJu8=v(6z-Xpb^Rh-O?bs3lG~EE^Z2T!OIlf}8+x3Rjrz za`XRM&-8z;w+I*wKuvf!?(vwGVR*&4WmKBCLF<7)TFc3y<9Ee}-u#=rtR-%zq&BnB z9;zOf43apUJFp>s6Me2Ly5Qj=TP}_(2@|JW8*s$_VO;P0^koLo^=QxT!KZRpMcx`(yoc;N2)Pk|(eQJebxH4wimCP^Gu0bknMj zpU`x#^;I~x34DG*_>Ft1gNi1k=>eOmTZEo=QRz^^hT_CGi?E+fViF+?EVK8&(+AG2 zIl@W6!9ngZssG-M^r~RUN*48-_-34_Z}qZSnyRU_xy$)+a~;=7zp6rYI@31_Zl_GG zhO12d!3EB+f8@UQHj|^T=t?CzhArpzsN90FL)^*g4@-X>WR3(z5(G)D9?11W3?f7m zte@w68!8GDi6yG?7QP7N{gWpl`@DVceQoFe#C!fL6A^IxJm%S<1O z6cjjS51jM+4FpCt7WKrCRhZOS#q>64#MOZ8UkeTA7)WnffP5TGY`*?9D`-@`-#d6e ztWq%XyRLxWbh+NEl2mgk!V2FdabX-m7Qpz++C_6`dVU|`Yipt}k+$@M+~upj9vqNA z5IH<|$pBd4^=OKH&kUB6v|dO_<6jm7P*Kooe{p`?g!hQH;?cm`RQVFQwkW2GGxe36 z?(#Tt<$xvJ{M~+SlyQrQZpPRu9a)Tb*won!368DRNiojJqT^xKGwlPg8nan{s z&0|ylo0D}v`g_+GuFTl370w*0pR?o#wyCY4YH*j$T7 zxqvpD?E?e6IRVGw2wjT0(wAs64? zLHlNU2m?AAb27$mE*%}@V%p*Ke%PX(rD3u!>rkj{k$zLg{y7`a&Jt7A@}AmhCI{vZ zglxzT8b3$Po@Cg^yV`<(0?>WkREzvA?SmFLyi{ z8T>iInl8MVbW4f=<)FF}iVOua-LxAOsi476K>#nKmM||4F7;TY92l2o8j2eHn#ONE zQZlUi2hl0E1IKS1ev*$htF97z9`vuF#PHjU$Agib5zU#x7n!xzIc5c{5kOFHq7{Lc=Ne7VI^qP%`Z zRBl&GoKxi*LAY5UOF>iT5^EKa)*ADzVR&z>z#c+@5)!%&EqVO?>C^O5C!#}cMa@Y2 zMm6XEs|NW)-=EJaCltIhsWUwyKWSRS$4JC3z3o|JfV{s4*$)XMk@A%5OXBV9cBN z)w*86aeidh^N~X=b0HT+AvURq_@FI-c;4r)%CrRugHOe50aEm8odmqZO87TcO6g?FLBMDIu!~wVdMpDUB1QSQ+I|ssSc)-p`ER=_cHstKDR@|(kK39ZEFnWyM`L{+p_o|?v)g_ zxjb7sUgP69@=ev-EM2BK-*|x;GqKeUK>B66p$p8v0TpD4aFB#e9)obrX7++;t+un5 z^C=H*VUQ7MRv0#IG2vU*k_GUo*hAPv?2ZYGWA0=lfYU&*RCy4~P@tEyERhpa1LPQS zu*Iwj-Ck{kO^enZWK9LhPBywb{QMi?8DSO>qQE{l)nU5#q(MRQ&N@uhhMU@ zZ@>bN>}N& z3n^idAD@1Z7P*_p#~NW;k2`d7lFKc9ans^dhMvJj3ZF%cN@o2l`Fmm4fP;bLhTsGD z=_#(pZPHqHOM6o*DdM2acixFp|M!9Z4_T=Yw5veERI2GM%EBV~($QIe-cZY{R4CQV zK5cmLvHQABL6nJ<)lu05&6#qgTPGytH3bmYTx!X{)5{mI^9T3`yzCV#8)%P;S6&)Z zEFdwu1dIHWmHv~?g2d|INCHCz_BNGosdR+QtbA5Fx)-?uFBJ$KUvaXX%tOOG&`Xtt z8=n2h-*bHo zkF0{HbSi1!A95fRH(;-;jGl`$_pF!OPcJ6@=iV=U_xTahOzi{Q1Qx{r>1L0Th#seR z#@qziCak5VF*Ly=T!AG7RrZ1u%WZAQTvcvp`v}%aPM3Ko8Z)!T)q0nQ#0SiV0qgwU zk(Z49^QBaJLaX;R!vkv(0;f{@2?mqC^hBBob!+8pq5O$c!~B##!WrvbNcP#xtgeTk z()O-jgdA6?!XF0o>RL6Q;6YN~OhZ=3-l`wXyYf=73SMUI;zSpYc^Qr^>4YAUIQ|8< zf!=_0czviQEM{96Wt_5TNNfX}f;m={od;&~hveca_qRoxsx_)ruW?s>ZG3aM9zK4m zP*I98c72F-L^G>yFsGmJl)E_t|7c>Dj-utu^S_G?+DATYZ7|iiT*%*NEjSD~VxO0l zJcND2G^c)~RBH7tE#K7x+5~__>7O+|3N_{i0cVB+G3I5QOTO0*?}szNsgb*FZKZ2k zU{h&m`!3eH6?cn{U9vGeFw7Y1v%uXRs`19{X(($IqAnDW$*8@20@8H64G^2p_YRI{ z?Y&(2MFU)KTq4?#X4ox26{X8OM9Vj@L+rHO`HqYYg~sc}<9zEN;R~WMg@2#}#*Sz6 zIOQm+w=uP6<1&~PK2!xZUZ|IbRCl{a79URW9sOAIFnvGhPH2K^AE&fdXnSPLCvo2h z>;H_Dj}GOhP9(bGpLc&Wdv41<(LRgtnkZ}cQH5V0X;wu^(Q zgMP!C*LP^)Fz1wkaf&ZT6CXmgDVC3w9&v;R2Dgkj#=|u6ebbnh`r1Ang#&edEgoSK zIb4ShCll2Lz!dv_H0{XBsWlDButpy>?o44C%5#-~{x_`M#@3I_o8dJU2ZO*4?}ObW zt*}uJxQK)8%;%^0_N8*0CL=Z(jycR{fIFIII5wBja&a8H86EY`!GQcr#z`u7_F5t- zH(w0LBNlr3;+WUqnuI?tu3p*GntzaNPF=0WrtCnGqN6@0yQl5{Mfv`BA|L>vpQ=*0 z{p&r}I^qzO`2C68Wf@=PYPt3)X?KK3_M^^Iv|ERkmBLyEfM@2tm*>Ciwzxwnk4>8+ z&FvK>8CK$0KlkEa>SmCjUl&=anau1t4jkmMp+Giq&RYB+T^02VWD-ayO5$zQlU=%J0|yZ*MrF(Jk`v|F z^q^r93y%_mCPyvHAq$%Cs;5_2MK%SNT{%V?agEFdT9{GWUxTpq#XGS?&F^Q*-juPQ zbafl!S>)7GN>cLHqk7i=BQY35CoywV_&)7!$>T`TDlZ^5zw z@;e^!fKgHn#X8c9({~pn+?EXjJs-l`Mq+klhO}&aaSk(^*A~H(Z{z=aL{uBAIedWps=Y1 zv|(j6ZWpsx#T`SukOd~t!zESb~}ozpnTG~{kRMu3oU=xTL5t)Ku?gc5_mOhx5D>QP6CcoLKQ4Y(CZym4%xCEgPAGEUgZ6~@cXf6*y-)zxNm@U13sGWJd`ZFd7Y zA5&T#%k8P2;Rp25%k#E1hJIoP#rz_-@ z$7z42c9AMc-pT*4tGl+n|CzMHbq}bhHnuXwpO9#>KKn1Qvh#t=YoyBkViJCXETc5$ zkgI*siJv?NS1K>*KMgHKv7yMBE=B6Q0k?8PovNLxa(*tP$|mIzk0H;u&koe(uQznC zVFJbT&nKHF>Uien|0cXK^qHF<2OCSrz9^86 z$~$!3$xqDiY09ds7A7n7nT#NpiU2rXiJ{-bR9?=bgYVwbIM`YF%HzOu^P@?P~vutsPf zDwx!*j;vizP*XCoh}E!$rHOs|_QdV>q^RhLxrRV-!0gRTdvQm}%jK*C`HSZ!{m6N&Ds@D>E)U7=+6|ksW?l7`Bn5fX13`n~w*x z4ZxTO&+R$89r{Fb_Q^vaQA`9m3o4vBhM%hL-Y|B?gDaMY5_s)(7H&R1+7=5d?Cho!dIcI%pP3;XiQ}p zHR+oqg-s{&__A=ye>}|^_H{j}o7_U`%cRA4ApTy!9JQSho`(D4W0?~T)tPJF;sKe{ z`KMz!Ay}l!$DE1UIUloh3#Q&Wum5ana*HNzignzBe^WMh(XeKFOn7z>vR=CE6oJ}O zOKE{5vbF^bugOYY&NzYAaFrm7Z|S-q70~O?_`#|Pj&*avQNO*MY!_+D0d@7q#*P&RsHfW*y7qrk-~8? zyZ6Z0LmoVHDVf^RA~mRGcDxZxc{LvHYbASE%$C%K$rN_2zvsyjYV@UI)SfdBkWj&uU7Cje_x? zM*|z-qP4i(-}!hlbP~XN$dFTLQpM=M82XKrweBk!BFM3ReVk-vm9iAS+nrENW{>in zkJKA877hO|z0gN4+M|ztmwq;*tGS8BSEhrlg~I?X93dHBS>A*9aKc zebqQrNXDs3zG^EKMy~IKntG%&^AnGgom^OGwsVm{AYt4Piez5||5|<4xGK`q89^Y$ z@O>y+Z(?q7=&8V!^j>A=Yk6AeP5m#9*<4S2XENdojPmHdG)F zClCBTgJ|wH{nUqvH$MlTi4dj_Z2?+HLLYloY%X~IIVwl#bbt=NSWGaUMUKRIm;dxn z1+YpB=guok!psT}r5b`_d-~)LdDnGo43bhsePuUv!$e;QGZz| zHjjD_g*{DtIlO|*Y4q05tbA1gUb=Sx1l$lHYfv(=0XdDpPlcCbJ=`1yoIEmjA}f=C zcsl}5VBLrGU*A>Pn+0a2d}dNWtM{vIlA1QPljOoju8otxM!gRqCEKJ%S>6Zd;PpSE zvalBwrVS9#a(*!iD4Z<)@V;Mv?&l{m+P@Q@QtccS-gZ{@5KF^E-5a zX@ls%nNsUTYQXPewLE>;U1HFy;qxQd0N6M3!xP0#z?~~~KIsUzDfn}%6&cfC+|MY2 zK^AGkACSIwpU{o)s{$61yy1oh2dXe8{m5f@6k*Ho=KK9~V@3^tRRmS^mqP~wktm&% zJMjfWYe!^9>;Zwtb{=^sU5%gA_a1l=b?dIlk5rUgph9)_?8E_*Agv(*L0LUV!%kZ2Sa>O`>^f4FCEC5ABxP(jvv}d z+B(a;K6P#&{=R_99ep`(S*-n5CoJ`3`ktd!&YO+l9jzqm6XfzzCg%H5KhDl6j`%_h{qXe*fqXu zKFDyJ+*OO<$@h}cTM?1r?w4yyRbxD<_9W)rzOV{x%eX z>k-R|mc^_>S%&Utdk|2DZ!>#Zns@t3xUhx^;~fY>O?cPCU1Ux9WL@1w%baIP!4(^;sA zMi-!y9fHEOFg4Efo9f0kfO4cyi1zI%gVJWavTtPl@GcFIQ@&i(J7vm1BR!`}wmBWY zl^8W>#bF^3DasJE$Sd(v{pJrH_?=L#5gCb*?w`bId2-O=kQrDO#|M$)i1{HNZekyx z&@$D{A{9f>`k}1ej0fC0(^DA_9OGWUH_QA~>U+n#sNBz*IlO?Y^2mm(n6tyO7yqK} z?yUVE4`qUqX-K*#$#NsCM~cbI*g;^vrpLLF+HWPH>+94ULHILs*2^ix!9NW7Y7;u% zg$-6hJ2>5Ct9>M~a_OfbK4*r`RY9?mzZq<3AEoaZbiJoDAH#;OnGAYJ+P5&Zr?y&i8|^gO z_R);5w_dwp*w^uNTkWjta=vL;e^Z#?e?Qm{S!2QrIL%v?vEe*pg}>*f^wiL3fFS_6 zxW5hZhHGGBq}H75CULZ_2QprMZI#KV@Q_X4oxj89|7pc3Qeo4CJ|a z=6t=kZ4gLry)o*vq69a5mSo$S*S0)4p> zyw?3)QMUdpxm@}dJ~$~%sr~GQG9M@Me_cu_dvaLD&#=!%63!`^H4hc6`Y5{TxF=4n z?I8_+q?l&$Dk8;g`_dPT&1-->ois+I&mpt*{EoP98BkLujp-))5&o*Xt zroi@gmAZZN$-);sEsVPTwA01qP<8+juXL0UW);U;2gTMBtn4MnO9OJ<+_7#IQ=?#) z_Wc}F4QyB0c)qGN|7{8=0`gWmFd!l`$bE7~3p;VRL3;Ae@z$(#gWLrn8SMi;4MfIo zw|R-;dNl4Xc@AeB526%^gU!k)m?{PaWbR%NQi(*$CgbjV1Wv6<2_N3Y4oqP;8(kl& zdi?h@hhUmPfb$lYZP`M`b{vN?&9w1Bp6w9kzqfUXU50Mkp-0OAt=NYIxQ?)wR)zb* z0XXGBi+tMc7PA?XY?Du5eel_d$iguNt1Y?j^Xpv>(&_9CW^nexNOIN4q{rNI2(A%F zaz033hRT%%;j;}t@I}w~YKh`@)YKqa*3Kl60P4SJHXHRvruW zVz^k^j|z+lJ6vO71y#zvc_cT|Ju9ifc(y1_M;Y6Rk!w}urIPtJpr$cFf``2?OdnxQ zL%}M^cQX>%rOyh^)_0mv*0?lkb5N%?6#qD=pOa)r@p&XdJj~c55;5!dYWLS0D*2s8 z%oW#rUL;^Iv8rAmnc74)vO(m04tdyUSqhDB-?3XS+Ij5sHRA05hWZ(5JEJU5sqk%Y zdRCXfVEl7Ed-poYf3zI2$b1Y>VjHfbv`fp8espEPe}Z?Vx}7hI@p@{FL8n+y9mgjJ zY)VOD%*RR1L$r#VK;KKA8D{XpR2P_?W_AW~U3k_fBa@`{*cg$H8{^sXMZ;>?Y`x^0 z!|lR{B!E(gnh4c!`s-JGqF&w*}nQ82U2GxORY-Go9-ePv0umOLv*L zrA{L3Y1`3*C+yTw6PezKLUOZuIQM!6<;VtwcQ^EfOTmJ;tO)%xEqecN0OW)FIse>S zQi|W3K5WCD6f?nQIv-FP0A4bX!JlV33xw)qA2L&G=8fe9 zg21GsH*1RqQ3FpeKh-PwZN3~*tGN-QqL*J+q8dzMcKlneUyYxyY67Xd0#&NZr_l@_ zk?1Mozhk5HEa%9jUBhvD?zE#u=44|@-$=XLT}a}+^LJByx91{ zr$-+0P33S8UR?k@T0%_+K%VG%^5I=lm5R_);G%E0FAz#!L` z;J~o`=lj5Xx%hVJs60zIa7I-tgwQiZ*G;t9a*sUtxhu52$7miuZV%wXM=T2ALG06 z*`z>nYBq7J{u?7GHCU6%itgSWIm54nD6r`43%;vMrpoCX|9YbZ(uXL|_0)qkb&hBq zi3K{&T90HqauufTVqZIT-TfBRL4|Y#+lu(CeN4mHr=XLHn~oZ2N!&3m<;L!Iwnct zOe@U3R|DSy{%IDC@>koSRVeU%z7(;0_l3KY)ZROl{&0;U2XZ;234`#iXscd{#Q!d{ z?8tfQ`8aiW*^qzbUqykg^PgCM75b;k{%zBI{^=-frr*&dzJb!B#sJ>u3wFJk4fjjW zNzJXYdce2=!K<>PrMyZwOhYi7rGm3CHbwo;oHSKFryB!1P%YGSto3 zZ~5SS@9|(GydZ;$)1&kfZf9-yW+8%G{d-!J=vmED#tbDMgD^Mf1Vqw-lbW=MH3cF5 z^|GiMXc6__zwcG_&0wRm9w+q(<01k&Xpj7g4gz8(YUR_yQq9S*jj@t3Ekqm6H z4y&wl1vM^{zhwZ8ID(=F)*+59&JW6Wv*m9V&0HAzP+w+0n;AZ?8D`5Cd}5C}sWTes zmBAe+h-F;U5!!Ga$zE=bJtB{@jsB`yTj<<=@q8hm<{V%m-7q!p2Kz&cAq(vA2#Xk^ zDg0OHM;BcAsRU+Ie0EW*{iLp9zmpiB-mFTv>=O+ej%{uIw7}K+3)}Gd6ysh2_o!;3 zU3C#}Wy&1VXVsS8s_a>hi|YAK68SGqGVg`@MJP$#=vfe*UL*1_+!tS%@o5bqR6VdM zp&+_en)$cDgZZC5AJV~=d);&kVxP3k?3ea0xHEJin%<#OdD7PQ3m%+&_R2F??Te^T z7{5Yw9wlU&nT#vSPq-+|Md&JX!3z@J|5yKqAt#7hj=;_0GdwX2(PEkr5AoZZv z--||t);&c}lQ-%u@2N;IDu#A1IU4K{x&^lJH$ zdtcD$U$NLJiSb79o{a*fYAVG5QQ+lF3D5?X6Mqiy=qc8V>}ay=e=1lx(gM{8M@KC> zvqOfSvd0G>LRt7oS1YBi?#bf@&0!O8J`?vDg}Y*pW6s)NPBXw3_KWuni-s9!TbI`= zPk2;k)ux4^BhJ0Zs%Uz)aL}j(4yd+}7Pwb=z7ZcQ?qcfBI(r_)W}723Bk2(b#uXoj z0pAQm>&pxuyaLi{3sNKnolR#;qbDuoA9`0S4cS9nG^E3wE1lD41YR??iDh*WiBYn6 z)K_FqKwY@=q?Iz5;6F$wqqKVMq2&{nY%3-tvd!YLpO&r!`D#SfV(ZqY;4jul`M3TH za&GxaV0a@uZSZ%@8iy<$okP~XOMj{vEE)t2a3^17pfy@n2o`%-{QkjSW(E zBsKjigCF!T^4EIcSc7!o_4{#*Af1n6$-j^eWCag279zhr{F_lrWRc$alxotQM4Md& zX+$omV}o?>@=c$m$0(Q8av$n4&JQA*Z4A+5o*uPiI%|*6TtIEw8kXur4c5x~ zFGcQ=AtaTdqQXB@dZ4=QgjZh}S~D1)K7L9Zka2(Lnbi`A7ZS%46)$LNCUh?hJ0b@n zfv-PEI8%`h#b02iX9(lLZ+*@6S*BMlI+{q38lUBh-k?I%({OUV)GXSDpFYwP+>a`@ zI5YkwBN2t*i$pzd@lK!0M#-LrRs>y#Rs{Pr#65Otw(5b`wN$i?QcV7(Se7SsVlnqS z7%f|5J)z}}y-5MP%k#*KEo!cWF@_#@qg?Yvw5HtOeRhP0Vuc67!{gC|612Dfid z#$K~a!!&0sy795-J#I%5%W-XYWyf`%)-k6?wEmb^@G+ezj4Ah=ou@U&qL@EsyB5}T zR<^C<3~f6f>)SWBmFHnv!Py$O-8G%370fipxRqn?7}H+16U#`q6Sp&aK0RA|%`WY= zP9{CN@l{K!rkB-tFJE4F{s1}e>z_4ZN84_P@EtdZ_whA{tAFmaF#RFdSR67xf@{q% znajZ8z>9H*V~#lnKBEJSGFziGfkjS@80lb6G6NYLrq}U`SGQkIe$3Y#Y<@Ulk^uJ%E8%LfG6r58 z78#qjoN(A@4#tZjw%E?Rz!YWAYM=}TF}%h5O_+We&Wg)%K*$d0K!%|7a1LJ&N}v$JVv}QU{V^|L}E%SojRPB4D|t{Q~Z)mtinqtfyxU) zI3O6%au_>g?8@Id#YwF4&0+1qQxs)zVg<=Q2zv|hu9vp0;!0hMcFQ0Fi78zd0ClwOB*lR z`|!id{rsms`=SeWTYZZgf^-GIjc5MLZrA?Rwb$Ek>B^HKj7?{K=lrLTYq+*E_Fob-_?waMUV14Oe^$!TpAnAY#hdLEGtl2?&cUTLK!>8w3qF~GJ?%! zJfV#^L&y+}RH}uCV?6elCZ+sAFjt^l%(U8Abn|WqueiUgW(K9u+H=j2SOnOIy~9CWi&$RbWC`$IO%t5DsgLPTT==*!L-7A0~w4 zP{@D~JRnSr*z9wN!W3H;tkMEw4E7~eG^b%Qc#6tM&KyEZboMa{NofvKR3d<}tCT}F z4r`n0ZTaAghgo|J$h9vfM#T*n&43`d)E2P##fb>CQj65Zw5J+YUiIHU+VsV*8yVP) zcQLVIY47F13)*YVF{dp3#}5WHA71$V|6IH7SB=Ep?RK}@^qvo0_^;pTMMf_C`Wb6) zbL&D@?Xf5SINW{DeJ(oVUpIZ@TUBHu>mP&g+(%9AHet+OzkWRf9Lx=bz#zpR{^1|; zn>9u*oSz#U8608u8Sjuf;)o+a_)K5)q8BkHUbKZVPYW=sg{EU1ipe$e@E4p#3tTQ* zfU!|tA;)(+x6rtsw+nNg+@Po}y7{GvogyQH*vFi7YW~@`dj%Qzu6StQ9@jdy!_0m* zZv7+>BzwNH*{o)c6YjYgaTI6OmB;jhC!eU989a#fJF_F^$F{OFx36PaAspvpW=gwG zN8;7@VfKhY9M??aU9%(M_b6tjv~v=-*U0(AD{5xW&eLAq*Kw_BFX!O8jM=VT*BSFN z?ub?h$9dWg)6v84xb>|WoSUlr^98dxv8_CYIEu4!KK6B8jOX)BDKjAG5;G9MArCAt z4}#}Gxc8L{K4wPn#D{fQ$>Lkz`WAx-tb)OFhH+$lO*o{%J_83?#gm}%&7ON=Y_SLV zoN~%3AX&wY&7-e#qF6A%L(& zJNZN;Amlm>!ePKFiZ6H-#VEqGWO+FZz@$-GfuPJ`@gXGhFmcDQCN?gS4JW>pKkndc z#R;KAIAjZ_@8eaz5W>&icx;lXeC&(O>#pq8eJ4pegNrszT72_wkNN0OuhpA2^05Wq zYQK8*9``ut!Us+k4ot+TI{L0)K`r$cCXP=WI`=Y>tonHSqrDn!X zEJqK{cQ~&d1m~BCd4*NoFs+lKv}a10xW`a9D?L#J&TUu*$K%4@3IUBoElBnqDXA2( zFQkm*6GFh)!Gl=8Gdp5_Y%7OvP9=j8+@(FA@VVu{%t}gmsMAv*DFq~d_EsLVqu(P2 zaXXl4d`#JR_&tg_DeauZ&E9;4t7>U$Gz)EGx@kQwm*04k4|BV z+&1@jdBDSIG-&63?c{Be7lcI%ELvdE0{=5xfRP9$R{Z?W|2)_3OpZ9@kVAMfnX6pB zni(Bn+5!_as%5x#WNih;5h!I+Lst6a*#yQNxLw0HIA80`?BGHtH~GsEgFAAMee7d- zJt|j1%<|2O4d4}Mgb3Bu(juYfHbQesjDOHSZUI0n&KwVu&* zUiHeP45oOXKtDWulHpz)91=#BWj=Tk8!)9zn!VADZp35TjL4M!C23IWgzF2Y&Q600 z;`^4jyoKxXRF-$XFeo$IYtGUG4mbcNjmEO6+jZk~PW_#y@wrC?gOlnbAN}~o3oqb9 zUAuO7dUAZwlrqpoUk!tgoAZkQu3kNvIz;J)^Ece~wzuBzde?I%6tRJ@&lytZNoX7f z39Uek;Owg?NU4FlXhl1Mae+{pg_l85N`qie)RjDT2vc}cDP|!Zcl-(8`~LSQz1QiM zX;2$CUP%9Vk3IMN<*Q$v=bf&0wW~soV8s9X?6W`cfe)=-vwAn4C7!PD8=Q9HQ7xz! zPFtJ77)oh#_UE2^-feDu!1b=XFU*9COOmm8a{k08KlSgd^)T5s-S{_de8U_3`hZ!r zz!wslr=Iq?uYC2Z{83Xx+WX(+#<3?7oBz5edebdYzVel?o^tA`sRic)LzwH=ucN&9 zm1_7DVSMiMU*HM2D@@08IdgvJP6zU4N{X@-b2D)9|Ch-Z7xQ+(vB$32fB#ik^Ws*w zT65MvU-+-zU3=~2Rf-3}fBO2@?eoVE40sXm>D+7YrK?;yPnS%?9$C2j~dE3i& zoyN?IAw9&o$1o$r#a+a26Bi6z46I+jo)xtQGd&%hhJwjGn`)?9+T65m6M_~_2-d0P z8>VwW+?}Z8L+r)KP*Pi7&sk z7%q(+JRI*kvulQ9uh_=BzLmpw-OV?%NAIz$)@^6y(%|#NikE$zd2AzQcHfT7&LpL7 zpB}46G{<(BuC;CFW@gtnGF!nq$t3&WE;Z6Q9SPrf?6G`;cQ3~sE48mVju?E58Qi7L zJh+jlJ>}7iu5xuZ*&NH~Lzs?kX4gzCd)YU3*N<1J@oV~_4}B=tVSMScG#VX#o+ZhT zH#DYZV3!U{nlpI8A%ho(AAWe$SZI^0I)0^6J9@(WTrqc}qL2(l8J=Kmc?LvyMCIUv z4@M^nOj6*7Cx_^8b;z$QH_h&Jr#mql*>&W)lSf4u%YaP6T=TJf2G=wsz@!Y+s1XAk zk>wW@5dAUT=X41*5ZPnx=iMHY;MqH;}f4}`6@vsN| z{yl#$<*0r8+h;xDNl%+RZZ;8}wBDphebi}dqcGK+*7w|>p8LG#JZ-=0T{oG#iGq}P zvVG;NUUSMRr!B3Xr1Gbadc=)xc!Oz#i6H#cv|M7)kA3B9-mn+1RXTse{SLp+4Q}x3 zS6I6?;@P7d@p3%+)5kpfc`v{P({OGa7M*K({_S-pVA;s%*bqoFj=d!q`D48Ra(lcXjpCCtC%q^Jo%)h z|Ni0HL+&1_-JbHO3*Y_Gr9b&2&bbgO#6=r6?7nvQRhtL9&wtN*cfbAZS6yLm2D(a< zaE=&XzW@1phr@G!Db2o=D-cM5<^b{pVv2a-V1<~lrypVzYVW_7(mC?TBRP?H8j(lk zc_6Vw@Ir)rG0QGN&JDhk-}=_KG60c?{K_z*@w>nKyNtF^l9(#^?8TS$NAosr`keK6 zBT+bf<};tkNC7`m73=0Vzd4g1Ssw&t=fpwm9|#Q1Gf2ebN`qyS4+G!a zPN2&C5Esr`jQkEVrvL7Dznhu%;B?6IlCR>FL3;cQ5sSf~EYV6)YKj3q{W{!oiU#Cc z=z$M>;M?BzHV*mG=VzKr2=c7Qd{VpUlp3$=%6K}%RTYK7YaSHhdV`G@mx|S+Q-lKZRqOl-+2>EW#~>U!G;DIN zQpN~~PR@A|@*vh&dvfCO{1LaWQi}?~ON&@vlh5(A(@x{Frd9c4Nhx=d@+2~U5wt3S zr~-cx)ef`(pB*>MxH|pZ=RU{Z6rV_ZAtz7H_PThhD#YNbkH^ZGfyHyUm6cC*?x=yU zc&hWI^xpTrmn%{ppuXcB?^wz_?U9q6$7bC!u@l9A?^PERmWMM+YkQ65!Hn&7Tksbx zuxNor3+(C^c=8bsMY6tFceVVE>m7I8aok+0+e-YH@r}&L0^iYO%f^*WeRuN}&kr_N zCtMiug4Mcpf8vQJR_$&dNOzaBohQ5S;JD`OBOLHXHZ^aLACt~nsw=RWsY zw{9H>*NC^f-R(lVwDw%p5t>G*AM&wG$Uhwb2-UEBG~A-(jV*cMOWMr}ARRPeMZ0bg zCA}3nmn3-XXSj?QZdEUO&~(^Vt>HhbHZiY$HJ$FseEjsg>HYD)W#?(b!C$>I3WPiB=j|Qo3s>TAzM{* zaS+BK`!?zZMz-#YVgk9+d_-~YiE|Md%gVfkk7q*>^t@JG4DZQ4+dzOFzWMI>H)hSRz*A7s*Ql~~#hjz1qd)!a@0@k*U;UMkdxL`?bL{b4cJIB{o}{;F_0n2Sw9Q?5q$<{y09#v=RW{F= zOcR57(T{$_skr(|S6cn^%U?$*Y-H8Ui?$>iZtP@nMhM_}%ZXx!P5h7z@+3;lva7JY;#~Yt#AXtvdRYO}G8c)f=}x ze}o4Bk>Tf~enF%AcvRcR!_+ zUm1SX7#87IjE!}&Sx|XJM{;JYe^5j(k)OhJ+Uff8bH)G--@UA9&BhPq8{Y7Sa^Y9+ zZg;yI9jzezy7AjhK2(_=^??T-$ReryG#9@NmO-U=%_Pbo{2CIA9yL8ohBJ7t89le; zHsiKA^B)-sA=aZF^{7&g0m#N*UdLJYimbp%kHH0oHkkg&oKSAIkF_~x2=hm9;rF@( zS;qCyLl3138ia9t+LNI4gqdH%bWYNvUMa1Hnr1b5K`%eDG!z#nG(s8*wGxj(2qF$b zP(+BQJ?&|P;PQhjhk+N1N7A<3Pyy#m;0lB@iUi*L<~MW6!DUZI2kgb=3S(7#C;>T# z`NQW-Aj7}^`@d(BDI;aPJ=88^cFrF@eF%J}>(;HCasHG@a?T$SInylU%n87hPfi9# z5C%D?JxMGIb>eYIdve9aM?7dxCW#=(ImRCa>wS`EMK2?N5q$d<5`)V>d>K(9A`OKU zIC(OR$mJr*Bv0D5k7&@Ugd?lSa!N3Ngg;L{)|~2GSR&4evEr#-tD$oedN|`Qe>AzR z!nMV@Uz^lhNtI#E*}h%DQM&C6V@JP2WAx~qX^x|W z=~5$OOmkuziI?%-CY|G0ogc(|>)SS`+ldmq=R8jiRxsbV^D%C_=N2R!=VoR~$M#k* zC%83jhdGZhG$*!^cp2|;2b?@P0?3aI^G=yG!4C;GzuEZm!_3uEL;xA>=CTic=3YMF0S`zZ zeml7&;x_{tI{etekh)H1X;8HB zLKd&#f0L4nU~KO35;~`1jGUiA5(|8Dpvh-cX}}B_BpmidC=jE-27vQMsmKxoLq{KT!qcDr zEdG$=I9WD?7llsRcG7&4f3$AKM5bd9)1sw|_z+-iWO~#FK9S@+p-$(dL~0shOM{2A zWj`6zSl-;W1Cfc;m>W>XTJH9U(PVfwZ}-_5sb|%KhEC&#y`H!wa>ZUzCXG1oqzH> zzw_g3t^eH1U;e{0|AkSE$rt70p0Mefzq;yI*RNXtYpd3+U%K}CRqL(;`Iwi!bW38> z^JjCbET4W!cnfE9^ewc_=}wW!1R}Q14`O^;qg9)}{LM{2p7hPwCgWgNy~@((zrtH% zqf>hxI9z^z3T{6*d(-i!!d-5!_)Zg@n#;TRrc*Tk`p5A~tt&b`&xq0`P7!-b1IG3t zNY5mrD| zc^eDAaSUNFa?KB7enVNZ8YDXGIb^z;fbbQ~&lw3k``OQCRE0kQ?uVzVRxTJI zOdH8q2G1mtJTCm~;!AXLX6ghA$rBx1oO7f=s24WZ3ABI-wo|F9KGO!7fB=CB9ZUe> zmxUoDL`anmaFk0QTHIgz+Sg+9lN~YyI@<)zKm#NA-~8q`=_hkF!fSET?dMhpxBBtQ z#!vzcg(Iig7ryWX8VWLfTB^yF12`gFEkI6m;&O%4L(oJ=9d%SfaQ)*D<;#+&T!Apc z#ivGAL}9QXF1Y7pNWGF97b-l`o3V@sJ?KFU+)z7u>}Q;D218LW`J^yNEME4lYyqQ| z^9P)14uo6r{0Y-)4}IuEoik}Ois78AcJWPnqEIJZ+LKu!Ibo_jD;A><*wl-O6144I z?|RpFz3W|kf{f7dcUSpWpsFuMu0W^`OM{`nL}0u~lJl0fjXQrVv~9Vkc6_G&ROinV zDH@8D-v0Kt=TwK!C{nzVbN2j#(ZuoPPfgjh1^Ic45J6#LN%=ZJ~*M+f+x`)qCFTdaL`Euv_gPsh} zA6984$xk&uND#P;XU~8Lzk(@cGDuB-MT83-F1ffFGI)5H>lH3y^7Fu+B|Mp%klc9p z9`_Rv54_OQ(DpGk_4wsvRtX_?{~dVeB^_lJ?YrfPC4##r=D=yDaW61 z+$a9*(U1J;PtK*&&i*#HyyelK{P6L|eEf5tJ>m1GobcHbk2&M3Up)M<<(q!hIHg%g z)hI|#JixKT;Oqwmj(BCC(h$0oBbpO06bA04+;NAS$kJM?S25z45YK(q(@+26XTS2r z(@s12gwsww@stxk{g^*JVvjx6GDI+$M6hb<)5jcl!U>;&{2MpA;YlZa>XRS)AgxF9 zo^-;ezi{e_k9y?8fArr!; z)l{mCBtz3|Hnx}`sqb>pwAoXsLZ(x1GCni;Q1Vp}$8ZvEOH)iVCoREP%PtV=-Z{ObZ zt#2<~_|t1V=Fc{K>JyW3m)&+-{h$Y~Ub|}bkAJl0$N#)TgFBpc^dxy@Y#uSdi&YGGb|AlK+y*F}ywsKu?|ILAcxWOx zzjqA%=P?%6LEwk7_%RqA0LP}w&HPG!UK5B%8|Y3mxsu=gOv@*eKHDWCJS39kx_L6C z{?<}7wkIWOK`GNu7|TddR>H~uy1Od)yQ4Q8Jj=*&SAO?>iaQ%SI!Yhn- zP~Y5U=c6e#Hxy4p1fShd6@q>*7YW-8QL*UQbGgDK8a_2nS}t#R*p|sV>Cf{_L=Er8 zg(;R>FRQnJAZN`o*6#g!ogC7de!sm{~3oa!0#%BjwB zX)B)Ui8Xer=c|cp#kvx9ZaZKvk0|qz*JhP~L1I%?Hgl#E7z1yGn=@uxe~c5n(!*E| zHvNyioS~J?k8P$Ey4`hxSiwlo)(YJ=GCO*hu1ojrrQvvQPHe~Rh)Z3&Zg<}xW0@Q8 zYC)o4Wow4xy_weRI*INm=66lUwQXj%T}Ru_({>bsI7;VOEq3azkYqY zcjrBai|WeIeC@S+$$>=~F>UPAhjIIQBB17Q$>F6z`OKN0zV_PR9((+Wr=EW1y=l`H zEm|zR>_@wdA(V#E)jzqs(*Q-}g&c*D~r899D$RR+YYnLub`BU)SyYSGZ z3&fo}b)x*y01aw&nb`nfketO(RtZJ0EL+$i1%fPRt}uB_Xc9WC41b2mDl)yT2?^Ay zfyb6IcGi6O{)ZDEencw#8<$lR?tftE;^i_yqLTcC9El)hp^%c6g)1w&TzElKzd8<5 zm4ue&^4f+BKWaVqjW?m;tc7K#ogSP_&E@<7%&5Lj#1d{B^aq5-{tF_J32lmsU%3reQk z84%-b7e;pp^cb9Cq;s>`trexasK(|%YNM7|!KjFD7Vgh(KFNnlxsqcIGr zAt*`0hz6LEFn4MlBVHj>_Y^&kaV|Vd_@e&HFX{?@AhN;G{J-4PHRhFG0`^)7yf2+= z4W&weiI5gL7_3-lzlw!JP#mKTgz%xQE}5m9#AtvWdeGBfTJu*rd%G!r@N@!V@ggZ1&^tWBiX`=8DbFt%!{MvCJGk=AVWCP(1KH7WmnR?RjUqb+b0c< z3b1s6M5f@#U=fMw?u8d#$WX6z-W5{v32VB}pkal?B@#@2{^jJglyi($i7ZA&-O7%X zbEs5t>Cuq9cfERI4Vp6SE6H+GbHGDp*VL)6UVr0FoI|;E=@O{I44uPbdarZf0i%x_ zd#pLs;NmFFuZ%tE!;e1ZP~2WUy8q^`JOB3g*RQ|TNhcgf zO_ri0seIP#**D&J^P`VHF>BWB@S!9h3%?Z7cfEDT9CYCE#~q6c07}fGa$+02u-tLy zuO4}1()15L4zu=3KGJYB#b!e`+Hb#o4?pZnI20tD`=_7II^(RbzcX!mO=WfW?p<%c z^(H(f<8Hc@123DJnmPFY&yP-&g%GDMl!8Gc_r3Pq%$+scx;Q-|+n`<|$Fx z$E1$tPg;H0iVn36%Noi!`y=@U*7AXM73=iO%$ifVZfk?k{KAXn#~+^*YFX2>&sNQA zZXUi>DF58&R>&Nkum~gHS0NhVp~2~_gPFkKOQg9YmY$svYM@4r97*dJ3u%2{aavo9 z&2=TPZDO7$CTvW?kdE}ge z6jG0bhcJtyTPIk+6JWD2%+d=r>12oVq!nf9c@1XH%uWFw@nz7(8{PxH*y(U_osi1* z4Z{1W(w>rezNVcVrJ*0v@~JGa^Xe%Ic!lS^E1;6H^^^v)9|TynQOK1oC{$4X@G>wk zu^x~Lk1?1al+=JnNi588syZ;Gr}M3ViEzQYL0OF*J2vlllHwgRW(-j+xa}C~*u>t{ zr2t=lQUR@T{V`tI>`P2Z*eoZMx0$5k<%uFZP^pS%^#M;%>Ld;`V}$^A#tt~R7y_wm z@i^FE(yBT>FRFT+I~b|z)IA1WMO0hUM4Mv)T+p|6QSXk{tN;K&07*naR7IHcC1pML z7{o=nSkZ`B7^Mm$Soka^baX<8omyPk2ltPs_mYM8TEyh#Ty@n|cyaXecuMt_2;=2= z`H~|{%;oVu)m!V~m|mDzySJ*4WG^kwmzU|Kc}g$A!=7IFSej^j`G`h%e~%@5j9}5n zcwxL`!`N#TueZ>{O7(VnEncs<;?x*lT)tAhWDiSzA$ww7o<2^MWG`Q^mmk}MOj=Z* zB>QrRxrFieSfk{4niPELbvIpl-Oc=Y%2-Z}@p6te*rfDd|BA#fEz;OV8s>k{ncdpkWZOM-foc`(M>V_wPkl)~t~=P@VmarDW1 ze&uA31ore&DvZB}^44GO`xPbckqb&5C06*Hihw6QA<4=E0|XZ|_zMm{>MN5cKeH@3 z6hK22jt%|vvpK)G;pQ(L{^d8`4BP3mGBcTfEg)KA(V~Tm8k(E#8$aQ+GtPeJ-FLnG z*>h%JeeEwU`^l9!5u^zAciw&PtH+;w`<=g?J^QmDicf^xPz^Gfk7vxh_PQIsdcrCA zBP=7Zy!zVTjy&eLaW~!m!ACQ)+C!*9s5yUs^Nk;0cIByGJCleKD04Y&nZIz+g82*Q z&0nx+$OJ|NOdF=~d7<9t%iaeKOFJG{i zFKazb))H5GAr(hha%EGKe_~$A5uSY3>_58U$)|)2_aTFt{}$ZVx`;}u89W$R^qJp| zFRNR>V%seZlaoKAY18zWcZ#0PMe>ixv_}A#-Sxu6l8IknVVqPY zarczKCRPOWryyBprVA#TfX+_z@jcdsC?;lD_4@0t;}c=d2@NxUUr(sX5=Ajeg6>{E z-SL)NhS%qjz(&c&MtV=wu@rAK%81c`7lxe|{ALG_`|Y3KnFHwOidPXXdsyD-wC6U|kBC{G5- zlBQJMWv9oWg2z}g127>OUwJLD9F?b4JXFZ#^cv69BnFNVKXsh(G`DC2u}xEv>xh)I8! zC9L$V@`F(zD|JTC=!aCq8U}#TFdhx)86}WM z7Dfp9VIj!Ve9o|~l@*FTOY4klvns7o0$`WH`QN#i1P;Pv;!9=d(6K{uctn^6SJl>h^vS2E zpK&(v6vh=sGG)z7J6XEF@@GG1a8sH1QlRIO4rJYT--MT5ep!!;7caTyXV<;<+TW{d zDxs*XsOr7W+CztKz217gvmE8+FTFhF#+z=YT57I7^V3hi`R$9|edq1UARRH-R#jEm zdN?0H>G9Mv&t7urW%2Sac|qe{cm1KNrY3M^TADM>WzD$oaz!Asa(By`04mb#S+m0t zpGk(gb??R(7vxy(pi68@?v*4ZCMH%UBV$qOp3sI#g;-38lq)rBh~&4m(dgSyCS%FK zl@P;6$OTe4)?0}nowB!rt7c^7QbS-OmE zA%yDm-f#3i%ZO7zV{k(SU9Daf<17kv{_>HPU)-X+Yj#b$vZ1kR_CGrue;g+U!B;ta z=gf!^)YLck8@=Oht&Lbwk;M{K9K|R~NUQF?R^z(EnuR$gL|)~DEFdtOSVTHW(BF_KUPdR1r$KO}^nm0_IkL=I|l z=y?rhgC{`*#5GVR1Q=2*I3rkFWpx1>K4hZ=V4^OiE3`SbO>&qa>j(@CQwXP*FIe9W zFy}0?v>+t$gL#+MNhMSS7N*2(pAoU{>XEU;HkE)U3rbBzuXx71FaJU43ah%w z$qV4}kPpt5c8n~vt}B0KH=Fnm%;#CKPn8NsL;n*xAJORUQnPq*#i^xJdI_ax(_3Cb zUb5B}vZ{~`UUCe3YrSMIEiM_)<>8pp=h0Kfx_oLOF45RZk+F!rIF}I{;B64E#~2xa z$H$bhJ>vCvNhoe%tVQ(2xqNDIdT)b=eNF`{HMU==(&DZ4T)?!F_}!TvC`Qg(=(!5n z;AxB(K=w$Nx5UGq*TW_UV!=MuOAs92iJWAS0k=9`EXN&VjF%~C9wP}}ORP7hk1>fq zIBSmA;w8trgpssZuTq(^;vm5LKwgkWKdP8IV5a(vhghwm55~)Yi-DCH#j$PL z^?R!jFz7@M1dK27`kQw{x(MT>=%^A1;m%;fcQ))*VHY$jb%;d>OAWGFyt>m%;U;Hb z1OeR)ks6GFfEh-N9XpmKAjr%rdR)RT5JIQ!@nASx8%sC@1W{N*_IaD^Phn8#)d3^i zcqKWu%CneukfQW=Pw8QXp=5^wBuA93@fwXOMLw4v0Yu5Kk@4duoUp};7DIf6?3oSB%_k1Y~?Ezq;!;KfdfIO`K7a$vpGi z3m<*-aeaLq#@}@FEw8`!_fB2-IiV0*a>_}^A9dv6-Mg``PsmKk`_4RXM*$ zGI8Rh-G=Xc=po@}m4n{!7mA{xVL6QJuCw--F$Zk<#VvsGa}lA^TJE77_#fKz^~|$> zdEn{TGNfqunsOx{#Hz3Wpa*YB-~}hRY<-4WP_Qex zf{2_D%1Y8mk#+!OfR{lL@hAEOA5(j|kK8x|sKlkE{ z83g&S2eBl)H*NZqDN~`OCERwqhZDv;GB$shtQxdhmwkh)iP>{Kyo zRP(%fz!r&H{5L@ikV=P(H2tJo6oAr50xmZWrUm%ZAU-L zuZ_?O@l&r{oP|LP&z24EYM~$GoCrG{6cw~H5a98lQ`1qMRsvFi%>roBRj<4~$_{re z%R?!QnFqjuk@y_|?Dtt?38aF=@j+;Y@#0G0)C?0SK)m!^NW&+~QUqPFSm=>C76h_U zqekI35hZtqDdZnu3sXS#5^_H?ydHOR@lIip99_VkU zs#ZuQ!oU|-$i3qqs^Wl{ma^qbm!13dGr#rCZ!pp7 z+_}>Rg9l&xvnzMseYYh`moi<$)%d^z6aAuJnEGOJrlo)1FZ}w>+s-}b?A=F=+++8V z2kbwZ!bwUgd#phyk=ztl9~7fNezo+X?_I)n8r&JA_g;IQ9d_6bfUlCfT`Fze6J+xH zuDN8r1yUVBcs05V$s;iL93NpcZC;a7xpP4yKyZ!)ehm=;0!}na6{WNo7m`|%=%D82 z45y!d>EJOu^Y{J-A3p!WOJanD{1UeI0lz|0QP!ch=F3MMMpKb&GSNPvHg<}ulzZ*S z-K+K;T~<|1z!3BZtY|Lpa_2AKfAR?)t2Wtmvxg_*=UuU=Tr|pxDOZ6)g;vqCNAsdZ z64O#!RoTB!!<1Kf%*>fxQ8&=UH&2?}vg;02$*IhS6j0uwnhKOc@|A(n^f;9iKicby zlU54`uK{xCSqSMbt#BJ|ACef2IvsU-SYya4PhV7f^ErRYVfBA$2pTwBkl z+u~8a4AvDmJAgPuJP357EKaaTgXn`;FE&ey8l^(U#0$})cureX zzs#zGcAfZC!Sux(<_$b?;zXP{LNW=bWd`=489dq%Cmf?D%Hnq`C`yC1kqJB&Wn%PEBSmKZ&|*=e2>FotkXW4a8GWa9;N zc=38^9`+Xc+|r3qik-E(=#;4KL}mmUp?aFBMFE@7atO77`JBaBfB znUZ5%Y=SuIsH0e>;J9h}S~YWCg0UOmC3hS$O!%d5W{1u(ug4^CcIfQ!CLF-yV_1}V z@eF|Q%Me?^befI|U-by0opUn^eA42@erTB7N*60meu_(XLapY4446BUG~Dn$@De1R z1%_FybK=N=3?a7DgwYjeRSFDM2V4BuxG?$x^fMlJ>-tPgPfQ z?rd47jx~oKdT{TyM-t)%7&2tzDKAf9&QG)b-h0#P>*}&?OnC*426UWq@(E0dC6ha? z!RUSWdGX~b)gexz>|gWev!f~CYip`2nbEcId!Nrg_Yyl?_T6W1*+A+!XP*I#wcL5+ z&X1!qn(x}ROXKo}4L2Bc%+X&4&!vjN{p)9){>0?T%}qhzb1!nF+{{6P>LI{0ha{@c z@}aHF&Z3q0(g_J9Xk*Lkcf^bd2eH_UK`Z zxFl2xkm)jA$_28qggbZ$LN>BHmw4qtvA6=1%uY=*k6DS~A3|7KD$C23FI%?X{-f9H zv+gG!eY{{%6MNS7++!s387St@U+~bxNsW!mnj6Y@+if^rq$i$uDpM9}msVP#0nVBM zl)&5#pWwnx7(KGfsCM%$nvXc5{I|cAH1HkS#4`4T8P7h|XJDN+UwE;SfDp-1jySJ2 z(I2D=AE4YN8tOZtGC5qS|B6@ClrLAsI_osO`(6t%8tCfSp?vF2D<(Y!zIpa%wJt`l z>Ap$TS6>=vxN_PA<6JXyO+GC!RJb#EvwE&*kwZ1i;gHykNxA zU(h%4NFz*zl%kl+M|w0`T`uD67`NaQz;i)3 zA=@|T*P!7HRGjZ{@mnDr%n5x;0zG>yUv`40v!;jj1i)sMZRjwJDDn8PwFj7J5$@8c zBvGd52@~Kb=it5f~O-$mdE?`bVYK*ViY6 zWsE~oTWgV{q;}ll_4SAB`}!k?6vmi6vCWnwJG~7=e^4R8s60`mRD_7jm#4`QAVo$g zMxO-6Sd?lsaCuA$PRZoa>Iokp14YqGh*!afV4}}=p;i%^|Mo;gNdGVp^ zT9xaOJI6ixt8owCal@(zDTTLCR!iYc(bz`56#n1zD0QpR{(qC5|LbBcpz(h_BmY7B zi??e<45O06+irF*nNFIIg>ek=kRJa(2Q%Sk-mB>W(_qF=_%$pS!t+RGY16rLQGon( zybsbNi7pfW0`v&skx3n1hLk^>Sq+gO8c#sOE>?vZ1G64Q64P&BR4}ZuftMKrfC@%h z^q|-P4c-veTdYGBCA}#p_%e{fr_M--1ui~1J73dF!$G58_ydVRjK`Vc2|j0-)iyLT zxMyLi0J}g$zkRCNg3HOf@G{cHwGj)U&~VvMGO0^QQauZ!Wc*pY4qms%uOeJ*RI(z` z@TS-?4<>!os+>F;kR*|6bE`HAc>5oGFrE5?49=urGstz<-*D}9zqtD6*Yn4r7(c)M zhOqQXayjQtEYB`lA`A>u=w^Pqrz?9*OsI_E*GZ{MC}p}H_9i6bu=={XFZAux)Y!=E zdwJvXGtTSch}aO$1PP9T zmhgjx2aMkP@=Je+WE0}Fw1P}l%UZIP2TCr|q?88XobxUizT4h=k3MMcea9Sp$dO|X zK4PzZ587*=1NR<1X4jE>-+lKVS_Q$CeZgjCYoY||85P+~tsiF+Il~eN+&hsK|tlg_>uf4*lqVDQ7Z_d77WR(F62E21b+vIXrh8uBJQl2lVt!7 zfbdRvF!s|Gg*IqF<1dy6^jM^zJU#Hrsz|XHfAo!5U0t1s|E<$K_O8bDr?`XE8WUFB+DRX3Ryz7Zes6ZbX7RRqZT9YukCMyecoj3s%1AA}Q) zrxzTTk*AC`lFC4k_KPo*!L;A?XQB*%-E4!N_JuE*?uze|7Ao)g$)-`iM(J|_EXm$N ziN%R%VmV7WMJXYk*Q_%6{&Vt_Mvxcqa>P}fSE&-^dA)Ei!P5x#T0EDhR~T|0lmFeKdXV>SYq2)Rp7O%{56HsYZZ&nuXOpJH22zLdZrf4+|=}Qq@p%} zd6vAe*XUu(0$e^95>g!FQ-Ng|S%w$G@GBRwB+gfC3GtI`c3?!_uU|ioxt6+=Oq1h{ zfrdxe>DU;N@H61-u~gUYg_uTP$y4%;9yz>FCi%>DnG9GopwWwxu?5F85I@Q9lk;&wbNzt&l#IO}1;AsQ*-+zCZLHs8Zcs?A=AcBK$ z3d0x!k?OVMW#?1~ei#LZ3lgBA(K8jIWE_SgL&gvdi!TFdXPtEx^L&n2^U`2sB^H@{ zwqz4&h$X;5CP}R5h4FZg@%WfN#>Bja!M{V0Is0+&L~yi+=fxgAG!~j5reL{S#(_mC z3_GguPl03)j?V=LzVPl$X3P!?KUKrsE^T$B-29PmF{8&AZwm011lvPx4)xjWG5WTY z0qy1Qvfb83=>g(hBoN&a4Q(NdYC7C>Mku?7)2%%vs*lFyVm*WmR^F zk<59K!?BvkX>7>Mo0r6mw+=vBI(F<-U6YO9Happ$l3Rhv{TmKQ?{)eqCw%`0KccP$ z+seykeKzZ!@%R7no(Y6)^zGMgr|q}>>aoYHzy2Uu9(GDs2A@cCQ{#H;eSv&|M+L@< z0L;MtefjAV)GuG&z_A@cYH|hXEiGL-5(ANS&%wwAL68$9gvIi3#6^;K3DliL2wR%U zI(Dw*fR8=*7$yIs*iD6*K}qU$=0X;N7^MdB$;ALJGz^3^Bz~nJKa{HCFxun`@<2jz zVlASOFKIz;PBMV(@kvcdj7H#2Y63}CXvQcmWM)DEBrh_#9eBY0H{W{ulEuqDnK^^t z?u|Cw0N(NA?aB!DG44U2}Ea5NvIXPj?LkrQK|r1aw-`89X69| zeIO7R88FN5tkX!2?n-5N%*J5|$ZTp*4N{Zbc- z{yZOPXukX1?-I2_u2=zM*Ej7i15nl#`S6NZLUVb+1sC8eVkbFG(q4P*HDbgFgmC^a z)ilL;`|YN*oG~P zoTkQT0F9K0^~=(RUs-^If>4C&Dw`v(3nnBY906 z%!AXn$PzlQl@PdgzgGOXdaaayiLVvo^;(l+&X%*!5yj4VlbpSWvjCY~cBYnksN|}ma?=+2 zrf^|!$O7T7?ENEacqNf`#@Ck8Mg6!xVaM`WWQ! z)kqgT3NvuHF249;eo=*HjgAMJRGG7@mSGTM2)cjdBN^`lCw-!t*#TDZ?Gu;F$=BEXWILIy9A2_Phvm6`^xq+t>U$MLs53!ek5MM8d7mK{JTb$kvC zt+;H3Ek&S5J}79|Gh!QggvWW5aA$822r__2A93^GgAZzrPAhDAb$$NGNADP?zgTrr z_40fo;HkXHL^?O|$}umaJ}|Zu6Ek}zsm!!nI5H;F+?1j2YyF(M!4?j711wvcN5GGL(HgU*B3q#KE?7y`&H6g%&|1vEY*CrL(T%3dO*oFcLMVnw{}7zu_RpY|L;M-JzuF=TjItNWruVW!Op0m1Ar zqEpZ}Qi>kj`V2OpM2ODaNO95K*pbZYWl{vGN8HE!MirCfS6>b1lB6yi{a|c+g(3*( zPvc+*8WYB}DRw>5ArkKZ%(x}*sF<-IHe0RsLI};EyQD2**yiyi!i*BNqQJY(YaS6F z^!rHDkN|-3K-?pR!Tbs!U_{FRIYzgS6EPSW=#b_4S7{?2#8mavZ3@x zhS?0%p(mOWFH+1%fgg9=ar`tJGnzOxg;D*%Yg0=7N%0nXRN?qfuN~zn;a*4~b*q9; zy~k3jI%E44YNWa%nU9^3l@UL3$2k2NrgWgqzG&avsDN}=PveDV`yST_#+1T{%crLB ztVJ`Jwm>hwI?kPD)oG3JD{DLCGY zo)`f>NRtCQl$JH{P;$o!$0u!l0CO-VYz*8X=EGoF60tg5$0P&TWdez$MHF~2h=rXd zm^&DiGhPNJrPHQOJNxXj5zBXOLICjgF$sZ%kZ=74wJbFG(0Gi8B}bItnW&R-?w-cO z%zhz&3-g56owop65c*IhhK~jKBV)6Sj=cGBN@03Q}iGIiZp<*Pz%!26B z5XeD#T)b8$MP?zYjW<>PI89)rlAxM7fR(@5$@B>$Vdny=a#pG^+*Ke<#%T0yPYNbs z7^5qMGy9?<(M&TB6urrZH}zdGP(UN|J?3&u%RN2XwNwk&`0?ZMTuQakka6vQBO@fK zKG6RWx^IT5@Ve`+!!Vj-=BUtHUZ~6z6h_)bs_aA#(eR<@YVz|J}ma0ftBeOxiO6|)f6&mnZCBw*$-i2gJ8WoRF zf34~>Ew8Vy=W8O&j8m_f5MVxe>KHUQr12yQZ!!TqeH(l-if2k;NHYwK ze~%&{Rg1;##^hqY`K zg{kY_ty=}tQhvQ5)6%0y_w&ASc70tP4m?eanWsaE%Y#a#R;XX!z9h*Nk<8Akg2y1Y zLarGLH7bN6BzQR#FaTpP32D2H*p6d=7pDypV0WHKLxrf_PUfeRhG3>SH3u6 z#4)FyO0%|n{P;x|oZs@qWPWus(B%5xlnU@1JR$>tV_kt$_?#b#Lcn($LY;FjRCao6 z#ie^fNJS1xvY1_mx4b{Syslo4wMQIoNtuaHG`4i8*fShv%$4v2)&$A|O5$z?&@$7F z(t#4et^uTYjm9X0z~mAJFeS|>4G)cf?}n!vV&Fla&LDwx9fpYT;(y1hj;A=5Qv}1b zobd*An4t(A89|F^k!|=C4m<2HTYJM2$mnIv3;{`=!VN7WM~-A23;c;Ep2+Yct85zSWeu7t1y2C|jP%_BJMMC{iPL*o)hnbqmhtebf2v(}#1ed*;*(y)SzW(*ETg8Kj@}%Nf zd5SHTCOM>Pn`KgC^Z=5q zyUH0@<>Fuqd8#H{o(Ob1gaHi5>D znI2|S)$o~7UukF0+}1o2UBU(_{CTZ>#-+bj${DYRFK1<$NwxVhXv*1wfm;Em0I!~|5gJ(VQk7Cl)YzmrTjF&C zWD3a}$DJY7{?1A`kIen-bfVZWFw8=!Br7z`+6z-bg(s?lynw-B#4FGTMIGiiC_X5T zxuPb+$Q6$OK*C6qP07%s5Ch5UrtoROwcw8&Tvw*fS8|ZP@a>yIuY4J_(bOfJSyUb* z>x6k?4w-wHKmnMcL#@=_rBeSV9_gofiA=!gwsGPpv)*#P9VQ`sDbywIIE-l4;{_8+ zjQ%*htz<6}ER-Gt7hVwG1uk#}Qqq_}fGEaZbSE^4goEK}q8ZW1fY*l>E{3@@Yxoy< z9W2^B{`ljxGc<;XqRfd2f(Em>c-I_P$z78T@QAwl>Z`Tqi@e~=W8_421!nZdN6iqE z`bL{3V`#@Pi9um1oWYmF=LTcakACzc)Uo@9ui$gjn^JW1%{Nm}6tDK1q*7P$F?_wN z_2Ts*Pj>@aAH(!1EW#7nZ%E1+$}U|xueDZ>Oj8)nHaCWIaEA>W%CD{t8@3I9+ibHn zf4l6w6By3I95!^D_10S#%480l+`~ibC7G?y(B&rSAW7wt0h?~J$*HHDc-yTvz3|-R zM;;n~)Dd4|woE4QO3au!Lk%OdXazDKdube$Oy7_#0vnl!Osd-m#i%84g%9tX*tyL5W&&rd!4@S}k`xx66q;va#-UY_K# zY}(^Kd+&MLsVAR!!g2haaQxU4PZ)drabM++lS)tj+9{iDwuvai-KNj-`6daW;BxBB z62m}32uni9t+ghIK)Lh8qD(LZ*R;llY_a)fTW+ywV>7+e@?YL@=b!#`FPnfGni~nw z!;R(T0~2pzmziXkB0%%|)0-wgnVI_6;DTwX;NH-&!(TQSGWf)kn9gNpe^&X^pY(iZ z+M*lA$?_ti3MCP!Lwr%8R4QU?($XbmOLBpeMObnS9e3QZ;U8~O8*+&3zi6y{>aWdT z9aHJfT}sgudSnDe5`$!Fcp;iq#N*ngJJ3ZRL}(#OS;b*wOXH7lw(qgUkN#+771h_* z(2Q`DKYj$K@wZ4J&Cb8_Q(UPmiSt^vEa4@EDjW z7}?~-Df{GIQRM6~$1fn_xS;SBQBXuNP*5y3=Pf8yU?Jb2sbCC43Bx57ylYcoM#k-o zB^3fFe}*eGPfPN=CPb-vWGNdKsWSot@cAVoEAc=Zl6F@1Gv_x& zO0`GR>vfp~0Hfp^MjD@lr?gZ6vLkg&EIkd~iSXQ+`7o{WslYVQxRXhxNbxZ%l{>E$ zB>AE#wAX69j46rhBNZB6t8oEfwv|5opm30;ibna7$&2uvnPoTQ*6lZ-ctQ45kOXWpqXO1mmK zeTLU48m7u{(M%hQXqb~T_eLfo0M8|uuoSp(=V{A>w2&krqA`J%TD#%UW-9wF1s*w{I2~Bttg_Rj@N6#4wb? z0zHi+1|x*mO8o~(5~6TbKn6e-aN*H7&~>FO#J56DArL*%08qh&*?h~msy#~zw$d=b z69_3x40}w>Ug&UZn*qSwGbTS#(@_woy+dMzII^mw53rO-opyoY06rN;M|QZ-qX zS*fw)LYPAGdFl4v0rc%djQrpH_IJJe)D7Nfvvs~uckEYB5E*x={>Ha1=rdsO`a?GD zU%%lGFTISa2p6d$0DkG^S0>#5;G>fsf9&xmX3Ywre+w2Y+;RBG{`DKxt-r}u+w3rX z`umm(00^Fb_AhII~oi*#AgAW_H{-)~>*$npI-TiwQm0y5QKAEw{ zz6TB%6zm@`XyZNh*?;lkMc^q?E|UgEFlO}X-FNVi%`oPSGrtZ}(v%8pmsH3SQwJbQ z-H-V4SFn80hWu^Dv8*7CUI5ZWDG9*e|NbQdlF}Ynzv21czEDlEr9y9vX2}iXZtOd7 zFnJ%a{wBZwLw5Al*w{4g##@rgu-V{^H>YrT9o!+1MzYVVA)8p(f6&IyKKmR3K}xJN znhFENt4H7kXP-59(4e_hHS=m~`TMAAm)ADhh%LP|ZlpHca%-kj=avq&pVbeV_14?Q zMN($j(q*qqF|h#4@13yZrdyUy7{Bzs`4Ogu53=}{TbkaT-qNtV@twC@-pwAEjnbx{Ue)-` z%S|bmWluiY!dH+By38iM|NiogMl^gd!zhF5ci!eFzO$H#lbY{-F!M~P!>Nm~JK8!f zIuRv_#l!UV5D@VwESX-)s`aiQ%O9UovdfzG?$Sk zoJYDp9Q_1JORu7LO%5X&5o>fLX}AnNGGDTNTU>z9x zQ!5+*u>-K6Mf^Mj-fTvL%-H#YSY(wn@iNoKFT=c=J~x);O;RdTM8x7bBFWPWX4E95 zj6KxengqTk#_B3AqIJyLln~)r3t{LbJH;ZHC1=7mJqtrhI(^YdwJ0el3OsK?iJh=e zQy~`almn5xsIc4t*tpbB^dv;qvT`QXtc`iw1T${I0U;G;Cjd2pVZX+}k`wa^uRGac zIfsgELd@*A{#Xr^&0eykv7V>=GFS3ie0iFD098DyJw=2o3Ko;e0*NU3DovE73SjL- zW~b&0;Kdu4tr%q|G?c9gH|?tGRv-cx)vKx#U?q&vdkJKmRMzK+5%ze&Df@Yh@g5^M z9b>slu91~wrlrFn%Ijf-rz)EgfEVqYbIxHBo5JgH!h{K|^it3YBJec$4iE^2MR<=d z1x73%KwhzUo!c$8kZK&vtk^TPBc~ivK-mo)^L8#ZuM4P*G?- znHo_|fjt5KROZ_(^Ad1Q^F&Kcs$MGq^IrDaQDr>~uwE+z`V?M|t62W2EJ!A1<1GS4 zwQT@VlJ58vh=O6pPMAp{zqO|)l#WwySlK4CR256Ycwn5Y#Ptz1$1noeQdt$J-{ld9 zy^t72P#N$F4a2pxLc{2>AVV5o1YR3@P-(F{s|36+3k>7^T9OBNt%}U_NTLZ?8{sX} zAhH1gzl={U0@FE>iW^0B$tdXqn2pe>7{&wikv!7bz>X)*Cl!HJecRO0BGmu3B!XE8 zYT$d}ji7|6hq$PDM>P6G%0NRc!rRUKKeh-w?-2q?pu$j$H0}(H=wES&aKXhA6N6U< zf2#j{urwWt?$I~P01Iam3lU7?5rqWq455f1#ql8;YBKRP!1@+Y%E|_VEowVHDI8C+ zc`V2AArR?`+XBht1p|;nln2ch8&h#y#$5kF4BrKxEzTWUEv*H8p2kqxG?mn6TUY!% zwYNi;Du(}ZdAY5RKuL5)Rb}|a4#_v*-~;ym!yoSjSzT5A%R7Iy#b%rDzyH2U<^IHz zlh66axBfMMA>)*_*XcR#x}Wv#y$&G&fuKz<6r^H?iLD~FEy1tv`t40O-`=HL*ToAL zefg-vuDJZirUhO?E!gG|q?Yrm7d=VZaD%~HZMEfJ{_=e1&K(;Xn$I}v+&|v^>pr=P zI_a^;ufF!$E}c8G#J{Xz*&d^Iiw77joW0rB2D1=VY;jU zTMDKpTZ*u%s;YA5op$cn41iiK$ z*i;|5UpQ$Xxv+tMF(cfcc&g=-*=2(VbK0s>v-eJ&YtA`CG#(Fc)dTlu-grCHCka0C zX%Sr6y?fKAGb_7wp}Z`y`LQR<>gp@|WOv6H-T*#~Hc=s#P*z4Ciu+sVALEa9DXk#> zYs_Z&(TXYoV!;MRPpDF2ut$T>AeOtTpQmXD%zRi|S6CIt#Xui~ryh4a{R`5F1i=hG zPu$a9INLhv>gq7Yf4Ph_aQf44;<;eYFOGPE6d)t-ba1I*DoFZM_Czpd(=3tAGIWkC zm6KBzdtlf8U$0j%!+3gs)?%5SFnUoRO8H|mMa{4fGCT_+rpLtN2CQXyA&cA@PvU3E z+ZV%ypMw4x{~boAa>OHxZ5Rd_OAbX;kVHSl-b`u^*~e`}F(Q_^w2qnZ^_(GPs-U1$ zK_RC|L7{?hX{K82+Lj7Rn2}Oy%LYOqtFN!e_u=2Bw;zqF##9WC7mUrQ;9`=+OJUwl z<)jP7NXE22FEOvjP=A=xGw8DVL)0zxhjDCNf3TUP;x-c^iuRPvzC5Q*n+Aa@o(6|Z z4C9K&2oO=camLu*U=Q;}P=={50)kS?R-vpZJ0+Z|&I%15rm8wEE$@QTpnZUV4~M%| zb+PD9z7(B`0i0;#lU4-|^Huo$ON}vJfha|zyHY(qrWeN3gW+xAm;uRRwcL#$MPv4` zyw41c0A`Xv`cyhyN>h^k9Wvf4RUbDhUoR|@rn@AF-b*tHOu{8r^o9pOEOi>Oyh4wU zVXsB0V7RSvOi$8arzw|Ve#R?$XebRc3`c%!7&Kg>=YvwsB(TsnPa)i2yc9z2u^c3f zmV^We0mJS9jM1YZ!LwG6+a7?=O74t85sh>uRsm1QsK?pKPi<4QNnR~$VlVj zF@{B>dl=v4%EaS6m*JPb`H~bF2A~i|5=0r(m;{_zzz{P5z>h!(hP_;INe&$<6%R)t zdE^sId3Hoe_rL%SKZPa?bV$e~2}cj3ReXYaQ)t06LUkg@lkR~r0bRA(BLF%09q_nO zN$AlTDME%=%M@a1j@WWV^=1%BedaCDE34KSnX+cWd8c>-iDJbM%ZCY`j+0)QtRYZb z%8OXM9N|4CrtzsBHUm;Xab0{D>>uVFc>FJnHL1ql1`m6&((6+_EJ|Sn$MoRYL&tE6 zMvM?D8UfGc^A$@LWh^~N@HVlC7y;%%g)(vp2H_s7&M_$d&>B&<0tFWC_D*G92=J&%NVceeEA) z90B)ExPSEMy$AH~?{f+{()1#d7FtqK9t5E-t1C(KjIuOB*82_I{0uC8pEIX7%x3Uwg4M^>e{ z7|@hWo7J+2=)6!XyyT!Vrxbik&mQG-vYnmT(R9O|6-OV-5Ag(zIpS=pFqBgA>?I2Z z%~~Kc8RGu9=)NVvQ<7@Z1o%{s@p!r_7-MN(l-XcNFTs#`0iRReAQNdWC`~2+G;}?& z{gBlsS6<2Nwb^ zv9RW0StxFg=$SloA=AWq8$e>SVNx{AGPs1)-6$p8?xa$-Of|U-319*}5+Rh5FDOg3 zf&vrkqATQ*iX!&ldQd~*C3u)JQE*CKa+7GUTUJNC@Gs63C-T!(ZTC1}RHKZ(oGZ9WX9AOohgz87w*8ΞZ zqEvNeDAaRRb$IRvBMVv2Es02JDg;_dM=Q*uc1m{vEi=lOhKW?xdIrYf&e z+A@m64hqdnV@PHNEU+`s$34kto{1u^EK6-SfLJcX628T(9{*f?;&Ho$!-yvsuN&|6 z;fEilxxtADk}-Gb3*f7wy%b(%K848ig6J$YlY#8t@mf{>yjIE|cSe!ds!uLuEpszr ztQScU@W=#=JD(_%96tV-z+-S&(1v0U`|#z?AeBw3OzIf>(&#WmW{I0vEb5F+U3759 z$dMy$8ozoClQ$GgihNLX=j0=hJULUDLe zEYx#)FU!7{kb;??3xi`QmGoc!$mvr=XXvF#6&&*(U2GigT2A_52>G;vW2SyhE!D^P6BW+d6^g36) zGt8P3c-w7n>T-U#r}E_~Q~6U8O7XftSXonb%uz@3E1}Fcn5b+!Z0H%M zoqEl6*E98I4o-m7Bac0fHsb={bFH<`JnIZwJ7Tg+;~3`fZD*C1@tm=e!g(U$YDFfd zGOOQzztL~L^)^54f}Ho?|6sy|2flXNDP+(n;qs4$2T(caE;3OHHbww(8Nj39q|8G6 zgi0n%<3my)2dZeHEI_pyBRP}{$np>vlbyx`;OD4JJX#gOznbK_O**7G?v|FWUApYL z>+r`Oe_~lfBj~Q3Iulh+ve_I2Qy=g}r$O9iND$@IG7ig`Qlw3b40C~yv0LsxM{aL%1yuAjN-)G4!M zQF&8Cc^^Cb!)qvO3BTmhQr4+6zYbctWNCSGn}hTMY3@7}W2@>vpn3M3vPDbEI%Fdj zn^+>6v7};$t&9eM8Xdw+ni_-8raB2C;cO1QLQj{#I{%nq25T5vYl%qKAXZxmu<}IQ z2N8=*vT6Gq)p=X|=ab}9Q z-H+KdSkJF6L>S+l#CT|)df^pyHX2BI8sQ5`fLA*~AYk5Zj0Qj;39L^7=#o{y0=N@I z#G#?I5lGXTv!`L~;sugqk6wsL695fCR$i=#jTaz=Nl7p{DSWI{GWp2~{8cR9;mZKS zwgR^KnK}t6#OqU2EF{HTDU(u=V83XC$9ce!0GVnU%A5aTkGG|_uYfX$)kX-tKc zKhk)uo);QeJVvl#QmN46Tp$%ZQ6bC^fw?3{+JW)H2qq)-_4S4)$%uhW9WwU1%`KKq0e24-5z6Iv0kr;RCeO9mjS;X@0Bq)Yx0!~iC@j>t*GXd^qrw6<}9`Cgm(iQUxBl%j*daPfD)1DT}Jq67e$6_v#eun+1SL~ zI*@Wh3$_t&u=YB=&Oi5TeqRH;ETk3s+G!{E>GOq4F1`F;^EkXV$kNmvo`&Y;_3H;* ze%X&Wrc^9JA~MNcrkRZf9CQJF$Zf8ju_D49!!J%%GuIDELWiOV?1^Ou9dy7w_l$q@ zt#^>n(9nGEz2o=TV|V;4DlroXeKL3<2jM7C@M0I4y#nEtWV}X<5Q7t%#9;uoB3Kv7 z3BkUUgUKq~>>F&#kQWLk*sm+eN&vxPVZx1UiZJ*=!iWo*!RkPuOEzq`?XZ6R`oH_$ zdpr)>pq^k2(IW~>V8q3Pzzn6Ey@HnH5)F5MLB!*tH{X1H`t*hU`vrZSI+CsB%a%Sh zYWJge-i4pnDQj$;_2P@~uhT0DaHvo{g^TScH4i<+v4_ARF27sXrDvX3xoC074{ie7 zbG$*4fP;%IJd(f4`Hf{m!@>Ji>-KJb>eL9!^3~hyf&2$TIGwE zRy_7{#o!Gpwi=>T<+pr5|HaqeSXEY2(a_ZR>4!Bh{H^SWFI5b#4^onnO#ThGq~ywp zZf5SmScSa?+OL|`mC#A1v7~@FHEV;cO;P6_W3F6E_}5D)ol+9K<-&LgzLb>e@9pC) zF4M~gW=Txr98Sw3de=|$ki(`7VlW}AtE*$HV~UqYpBJV2vhwzOt2~XD>1jM(a4ap> z7<0wCVw+>VF~;liTD&xGp{I#?y?nu**QXYj6HE5_^|pD~YmD)QB*eO6jL(I)+~d7Q zPZLY?7%!m|E>9zkRCRvMmmxh>ooG`=u%1R$9ZH6&te;tu^b!S6ygZFA`%S$4O7$3H z1c+_+luBLI=#0INIa(-_x-ymBaz zxJ-m8^a>=0*%|;dPv%+_5d|oG@z-ig{V8MgTCMysK(Cc0ZeDBq%HQ@@Xu@v=V)+#y ze2Jkqgq$%%ATk+)2_--RJsEd)5RfDCPSggg^z1j0I|E|80I2X$c{*qsB<+H?Ayp=N zG~b6FMeiwX=5L`5KGopVMpMHZzk0rvibey@(po$J2zY^gp6a^QDI}mQi z0jP!S!J|Q?DpAXS2^rr4@q27pqOS6~@yHO%2fT{r;e#WNL&s#q?|l`8uVG(P0)CuV%ufo-ByZc8gAO?8!2KVZ`0z8& zK1Z{^V8Q&JJ=gBnx6dAXjN&I(EIR_$>ao^or=IlBf6k*BtgNi)u~u-w$Tlk$KL4+N z_1N9RX+a!-P(>caE7gK~_ULi(#TUKwQaJXjx|-=NF%GS6BTv8m_QSq+(FJVSY-ng) zfBivT2xO`3fbj(-lfW4A%SRqQe3zXmx5T#4k~cJDPr9MB@kU??Q7SNQvrRS}VDG*5 zV$V)hRdr2G7${-}ceyY$Ra=&XDv%3iXw3FW<&fd;eEYnP9cv-mVv7*vCrh~Z?c3+u z=YNB}KQxTnZab8hXh|5k-FDml;F61|OSpjR>IcFI(i4EmdsS_`Nypo7=Ui)2J0uPc z`i!&s_GQ2p91|54ozFeD_p!%Hlu1Ci%{I%~aWr^?nsdLNne;4MpF)WtClQ4aiQ5N1 z3#=`fZk;PO39;E+RbSjZv)0$SLML|LD7;-<%ge@buDW-b*r)ib}O%LGfJosJ-<0~$dIa$JC)bv4-!E(%O?zIIW~edvymf5777(~R;&TrSpuBP zMj$HTy`0p>#DrpJFhJYLu!rDEMaS^fa{w9bRjNX^Q{@`AH6`$$C;>+AoFu}obI$%_ zvPM;Bpi9$DRcGqW+B`;CRrdzMqSG+(=y9yc>ir6`XJ<&C7aiL4&~=4Y++V^AlcQI| zA8W1DVM;m=-f zQn`%FR3M*ActZk6thRedES-VGk_tU1vLM!25H7r71jHJJM=5}~JPQHjw#W~jt=+VS zFg%Kf^JAjbewy#oLLNdcNV?5p@uu~xJ1w{!Xl^B?}$pv7Jy3}ixG~?21 zHN668`iYj;t}4dLyjCXvl)t#hm&%_tX3tJ@i(^_LxR&@bn&U=H&%M87IZML{kXw zCHV63@`ab3!b@5#({sgoeX4jpjV}j}_ZAkCAiTHJ+on{H5$rKuOX=E6vByh}>Aej; zRb0g}KEHd0m5FdYddE27^{BhFcnmNx1^nerLa*TgdiCl_Gz5U`G5G*SNh&Sakw+X3 zz$Hqh@`yU_9N39ZA!SCOP6*h?%cPw@J&IRygfYE5g4g4|`|Le>-+e@YWTO{>D1kW~ z^pJzcNVO5`(O7V=jNW%|ChFiNT$G+37&ixT9{|ce?1H^^Fq>|&F@F;7lZ4?eKsM+O zMtYT`JE8zE%!GIrU=aX_m+uIG`wlzsLq%b;Jr}9Umw1IqQo<=J;zbKC4F04SqzKH6 zU5D?&A9yg5U;xL0Znb4N3)v(aJMx7TBX?awQs#^bq=NAqNjQ?Ylgb4i@4OuJJc1Xc z;4B9FtEwsne4*m_LzU{~3)a1=sTt=JB6w`+-mQG|uGL>0Vg!<=$8sns&*aPt!&mg} zU3=nTB49PMjSY#+^4i*}jn=OkGT3CsK(Mf~vU;E0TAG_!>VY7Xe)0;8*w+D&oPSIw z%W??&7CEU+dGvQ<1TbrXMI(%0QKsa3_7v>7Qb{~|`93w)sMJDwy&SKxbb9|TRG)o0 z#P&#wTS1J__0C={?-W?`zh4K0sYvofP5DG&okR(+yL2PO`=6Ku-8I zX@dOKfq5F{(*ChT7_qcO5Sv&Pma(&az~I#gu(MI&BYVeg&y>aVmTQA39rW8?K@$rE)S5%51 z@8gB1p9p?s%Pp^{b|6)xxG*^QhA<)y1m(5r6svYfTf?%Z1U}Cac>4KQQtGJB_9>}u zGrUN(8xVLhBp8d5QZjke7zMyY39v^m(~zRKB=AZC!{xQ4FpvpD^Nwm@we!ZJR7n#O z41kCMl;jB{(h19L%a(hM&oB>=|Wrcyz| zrCb9TZ%QeTc0uDQ#Uj;4Bf!W!i&p_GtDu(zEJ+Dj;bZE}u8@AE3EBoqU}!-JqEr^z z`WgzdD4~c<&&jbRm7D>R6w!+0vlivgKeoN7UfvpbwUod{8*Ri=%{}+rga3#b3N9q- zBx3;ZINkWUOE$Y??SyKZ4sS;p$m8#A(jRjrKe6Xqxo-+!m>Ty&u`wtlTbN=g1 zH{C?UabC;!KK4}M+p*yzJ6%-&5%)vt9(=FkEr4D-20aouruXtKl}V=XOldenPOwB7 z46%*^2BS;z`LP@UIG%0N3xU{~e%m<$U<@x7VLa0C63ZiEZLOO}T_zS@?+|@YeBN5nCP~U1_D4wW;kt-V#e;Ldbh$UjwlZmML%Nv}w~I z7O%&fgty;*TW1t|8l^%{D)+kNzy*-JRweOTncf4(Rza`VDoVSPaaB3W--1y|6_Q!{ z^y$+j&ELfiMo1o&3locNf-#k|EtD$a7#5{cO&~~7dU|{=j1qKCEnhV>?4FRRz=VNt zrlz6~$P0cAkhn9v1R^q^M~e(IEnd6mNwU-w0D9?ukxLRpqsIiD6{wOF#@;fIUcO+76pip=7i?mAG>wA&V+v!^poz(_L}IziD%b@BfZ@VU z&qyj4sVZ9tfOUB|hYQj~X&9tQ0vO2$2tpIFq=Mw0;sObQ@dCi5yBJNr;q%&KsiqE$ zU@xaosx(TMhYQh*Qg_+oxqPbE?(s3bD7{9}dyJRnVNtS<$|YFxbuT2rr+R4~AKUCH zJ?trCjh@oW_jpMV#^XirQ@yUZK6rXhSxAEL@q6@kdS2lf2Xe)DS?+C!G2+ra*5XrR z*xTu~c)Tw|Uy@#$=k<8O-l`ZDjo0G2JiV7JeEhyVuP>`uexX!J_HqOlqW4z$a`SjE zA;x=+p3-ad__$0&=_L!}C3r3mi_2rYG>`Z4W4!0`l->qlHrH!^z1J%^mcVKzi>V$j zy@lxG`rzrkRf7MM(HYG57#uUa##M8|2`9wHfnm`*Z*+#v5nxoEck$D_5_x5Uq(FU{ z8d)h~^D>%*l?9ko9DD4s%r^*BV{VLaeoT?yp5P~N_>E6O)|pGJtjrWWM|C2WdcdKQ zLx!}*Mv9D$B$|rzSSV|pM0nhB$EC#Tkr#xQ2N;`$;k7bK(`!9GIch6rgp1Q6V!Y1m zWbV|y*BT2HroO(OVq`kU@va=YnU@S-2}iMV0xQPnC5Vx7XW(Ihk!d*uoVn%6_|EYi z5QCtCWmSYy@*gDb3}R)?u)O78$gr50LZj%IvT{lV+c@L40NaQ^ihq~Hw0&n-lS|jG zTSc0JfT$<|>59@tNN7qC5eP`{h%^NSA%tEw0*2lqC6EwNAciVLq=VE@LobmgHH2P5 z3+-fo@A=;Ie%JYRew=ImKG!odYu2n;bFX{W8ei?%@ol^-2EK}W$%;-P#Oz`NYmto6 zy}?$FyEbz=R#VTW{g-e?Rkc?&?Jrmb)k8<~!@B z70-XU#GG?ilx-PR)9bSU!U-Cmq)Hg&&P%G^D%E2jV3w1v{9EI}nB+MtVa?TQ=3eXC zREJl8sS9|2z4(W{j=XDiDMn{iHJVwGh8Ob0^&8Z=pe-1{9oGhZ{af~C><$eE zx)F>OMiREqCKsAEv3czdR3mupB{ReGuFqfUG8Wzol@)p^)Ypcx14Ieo|K61nbUezR z|Fm^e!e(aYxvwUlWu1Ca19v*&vU4+o*567cDIBllPa*-={fnN@#(Ue`&vXr+>9<_(nzkvp zdoP6FBvw9!`Sqt9eYVmimMo>=gD&~dD1Jv}y=$=|=bqlmx^c{+azfTaTx7C;m^VaY z+HkeR#tXS!z7SUMg_bXJ@cVQ)Qu*w7M~B=9snr&3IOfljyX+i2N+;!hQIox)z`vE# zbo07SrHG86%GJjm)4@k;*UoDiY{~Ga3jWPl6r`W(AjEj*=8UlI&B?gda{};{@aI`y za`rs$UaoqHzk(;;6gXEK{_@-yY|)SCuE7qyw?LotRPW6r-CM2QcX#+y zT%)O-y)k0bBcEo_v?8@Dy%=pQ_{O><)D7=S@YqIHTs!or@{1J=uzoHkNHmO=J2k%J z+X|k%dM@|o+6(BVTFK`N1c9_Se_ef0JOB6A!(Zmw?ELpQ4L&K$@-ec=7ZjI0v3Tt7 zE#)cqYAh&Eu;)iQEp_q>(b|W}pJ$SV{IAtTIx^C#Rdh?{JlL{mp<_nh#I}oYI(!lR zWs`A@t0sJhK}yT&ykdZ;mcvuVJ8y(J{AtOu|41`ly5P$wZ20(($HiaC_fiF=?hmet zXlZ<6ACA{7slHmXz;M1lW?+taE0td*l`K^%o_jr++mciJK4FEd^q_;rp1_Y>mq5r5 zcO{g^Obz7%$*Nto;|C31V)Z7z zG#QnaOSPrxA#bg)2gORXSVkY4(HJ4spDdo0g$T^X<#LjCDLi}bjjme zuhx_`r#{Ypn!#r;6*ZAN-+kmL6-VD_Eb<5|Eka+o)Nmo}?KOaBJKw6@&MkA?q+^;j z<$li}@>IIw?Z1xtF7C)k-f~$<%JtlP{r$7<-X}GCsmE&X;@?nIR`Z8Aqh-yWTNYzMwp{6kQtGQQ8 zSo__9jgYgWMe}oHZF!++#G|RJ2@V~oASuvxSjDgIK?Bo;^869nhRfr>6kTf)34|GB zmLE*&r&MYEf@(P25x*e0SXMg3e=d~st##<*Bi+F6luCKs>-1N0JN1x%Zym*m>!+(eeT~f@-__8{Z?WHPvFqUeE$y|`XV?7kMZ1y4Pg&1JW*&a=TGofU z@>jM#%$VmH&s0KtxbH&FiHn-emN2j|CpNj}{q->~Q}0{?Pi(CfAu0O`{zBIaY3|r2 z;f|r(XULePGc_pp*Ii0XiB51ZUBj}mf5&E<#@oNQKK@S3G0?hrS1T@AXzR79jZNYc zeg4jKFSZ#<*ccZ36F;)m%m|4@T>C2Y31o%7KXI4@wB|AM;8{74{<7BVjFpuCE+5!H$J2&LZ}pxpJ3 zVlR!7e$`y~*T|<|8SMXU{O83&SKl{c4tbs(+_a|v>97&QA zCZ%IYQ=I88)yWyhn#Ok(lOMIZ-46j*_?h2d`Q9Hk42?>;`bF?+jGb^Q`7Xp)c(c^Z zFsT#Zi@op|uoHrvTUDF?n$#Y$$oZi5zN;|3Vgb?jaXUa6U|{_5BHPu2b6la9egpaA zUQrev#2T2xD{R4yS}i>EubA2yrg&4|Zy8E8SgD$cWyX3a#EZ=T_l!GSB*k#_o24EC-tr&u60M&8w1r7 z{X8xe%XSr(P-zXJa!bxpH{+D+;tO^aBgZO~Y}TmW@|D!8-$kmP718&epnCc{?liOx zWP&CupUhTYSM%BW?p$;t7d$6-EcZ@8w2*h7U%Z8PMe-Xht!Em~0j&~Xjn`|&AppN) zy)sB$w#Rn#=aN#x>xYZYmY?Gdg1SskO06TsM>aCUJzACu>S|H$#8 z`)Qs5hyWdQM;-LhV<~hMR2xggP-xNIX42q^iTYpy%LNo*@XhnMb$rONuk#&@yqrLl z5QaOtEy&ck+5LO*NT8^G)sUzpoO|TDZX;4n#k^H~X!PZ7Uu42X@@tyB&fs7cQJA?kPytd&8x}t^^DRS--3PBF)(@ zh*?o4^|97mb$-jjh9^e{CpX(zcZ+(07UuOEjnx;A{Ze_le@Jhsf2&d%mx@xztGB0C z$)K@_y0Kh{2H1sNE654p-2ENu_*aBQelYh+ zYQBF-ac2Ct=kDk+i&D_C{h_~#Z5{WLJ)4uk$OoZ<(S6ig=vMS`U+HfpT2b{+2Abt? zy|Cz>c@rkkJG&3cvY$W6?)G0uVIB>z_Vy@ZbBkN)i8^)3ui^A9b$sgXvwO+L-6D3$ zQUi-s*{DgIYF@fuPr;9g8h!qB74>GjMnFy!Jq0P)A=?s9%2<<#T?^wCKllLBGAOsGff>{}$hDsHe10cu1{hL`g`(EbAC)#p0*F z-b>-wsmcXIN8g+$w-j_6@MuRJOtj1M1>KO75#H)uVO%V*?~nmP zNtHfYjvUB>PA3mcjSAh)$Sy`n$zOBvxjky0pN&F*HtcnS=DdJ%5MQu4zV{wb2ak7}XXMOGwI?Es2=YMZ8fJ%go@Ej# zGGM`7Lm41BJThzB|M&94g8DiQ7+l#X*OG4K_%k?_+)==5O`pSSjmmLV^9thHC8Fd} z92*Do4q}ti+Q`lp1!Z8NqavwqCo5kGJhmk?z4g2K>?cS0F&|z&CGN7`#nj7EDK)-# zv9MK}KUWjgPC~u+?EXEBamN2S3_&&%DPg$0J@sK$J5r^0;6h3iPGT!wkHwX%7f7ka zxFm3R3!>}V?WW_^g4+t0XA7m9)&t~+HV#^_y%w`x$=uQi#~W=Bmf^B!@S^#$WS;8m zQ1{sJL^8tM+wKu+LN}G?n+_5j=@aN_*5yl>iQY3>^Jt2n$K3#~j(fECrW;!$jh~JM zvBIbTwEV8QHWHk?lf2;@cZx9;_#)LM9@|mn^~hQy!~H`4?itncOM^^Xpijkzsb4EROx3b^PT2|T4Xch4gR;!j1D|^<2{XyV zt<0)kToDetz-z5Smb}_njjB{@Ze~*j>xR$X)Go~L4vbsP=a@B`N!H!1@F8ABKz{0> zZ2P9a?cR86jd>}1a_>kacvgfmLsvvVZ!8JFt*;mv#b2SY-L?++!p!fW7B=80e5;Te zJTJh@wgU@Zf>BP*Lr1^des{or?xf1bU2~UUULibO-#2c7y4DxKSr@v6xjfNoJfSJt zKRr|KTy6MUJPsh1-!i97b%hHUKGEKV2|162;^ z;TgCoH@}Ky3QV}rNLP!LT^X2%s z85jQ=Oymg4Q4Oqvj^1Nsf8O&smYg`45c}qjbe`jHGh<@ZCt0p5CL#>#cLUa6mz{nS z2LQyVe^R_l#i9K^tO48At%dcEHAkN>c?B z4w`xM)LJR_B+AS1UV#*&6rB3C(P6H$b=Kff?CcR=O`mfT=T#9jcYabcr_gldZig2# zh}a6bBfZh}wrwPvUf}ogetK~3(%#*M-`ih`8?_vIXY9{EY#!ESBFeRlbSFdI^y4|7M@0O489yAqlN7bE}MvNtndQ?Eg za7OmFRJm!L?KoUC6%wc{HzJ+K(bl1rv(sz5O6&7bv0N24$Z79tKr}B%>Y$y_bK|Tv z-_z{9v*YdEA=Z>cT1WF+i?l7e@SuY7#?*=p5ew9tEAFf(c(SN3i5*F2N*#gLe-=BY zx~yuSEoN%fdzickiSH$~S+sdI$pfRjPOGAoPKj~}&iB3Kt)+enr=ZGjA&{1ypwpAB z;}hz}giZ6o@?~%=WbUGTMZkhr+s|{zi+)lY;Prf5jUmb3D^0hqf-=)pB{a=IdedUu zR)bl14^oSuqy`~+(t`j#mZQ?zvU|B9J3WHa@&(^i@Fa-6*uSI;IXqbAu{2&<| zif!q(J6qOU0tHh-AD|xG2a{v~!h_cB^AoY~<}sIv1kYR!t|lk}oMwAZa$4uVL;%~( zPvp6Tf`Xba^gkj25thbRLX{uo?wEjd@5!dP7N81n(bao+m#OOLqsd66z9@0H?;l_+l=X`GW+)wF8AjfZCOwrq z$Onp3Woc~Ge=nWY3n!vbg4|9NeB=Mi;?-pia;4Q11?lnq5a;(4P=dh?=v5IFpK}4U zw7_lU^YX%3K{fH_+QQJA@VPO|0t6z^2|(g_8IOxWR7C4zu@&Q*?SEyT2`?(W|d@6@li zA1`0JTWwev&oS*_`e8Mf_3x!uT*J$_bE_#lOF$eMA1DKq^VZY#*MCs9zVqdAngsa= z^P!c5EI{Gt3as~R<+o(ey7(c*im5uOY~u^^xG|H*vtiH@W_lx;(^-OF7-Bs{5;aSa zs!`Bp+3nyi98nu};`B&@roh49?tQai@`^x#OUcwi1BgDiQV#vBQ<((LTq7LM2ldP% zcy@_L;h5_Xi1MdqvLwl_{Go=zTI(Br_uaPpfI5yLI7U(`cfcKRMK`;1w*{V%+?>vh zH){Xg{hM>YCJ6`zY%93WSB~iH$u<=heA8mV&Fg3HXsHaXzS{Hb7GIr`;y|zT0j;Z| z|95wpZm1q{D*+%j)lGq=1}C~ph8Y{JbLW+@5yXHdwADZf5z@;Y{?(17%pvgpx(@-< zOpG{Q)d=5_0tX3Ly`T!h%)PT!Nis^##-sO58`r%woMyW%LP1_xU|((n5fb@9#cHzr zk*Bbe(>wo6NV;Rm5JtLSsS1+b`Bc7Q-VEVOGeh$BBgtAU_MY5^jotI4!pdnNn#K9X zHZCnm8zW=(UFH;9(Li^@c=dtB4+Zq!Zt>rO3dg@rRkh^ThKPLb#VuXW)KfX3SWZ(- zCi6b|unvFOCG$*58iy*cWN*Y*(c?eYd|8%Zrmih!rr3Q1C(!Mde zR9wM6D3VRde2#fU!CXR_-)3p8S^s&FpmRsh0m&!!TKDCDv_DzM$JTrvVm_>g$FSaa z^Ho=~=t{*8Re3`2nZyL9!dzvK5s=eQ!!A*NuXHKIYIuCj?t>F$cxhjL1WfqvTs9{mCLG`vGq2lKEVE>E5{ltr(RLu>$PVm8~XtS7bB(% z(n~9L#ih?bt_DgMdKQicF4>6X`;!!=$+wf{3(KMjI|>CI7~Z-3-CQpJb{i{!t$a~) zXSmw@`kag0fx|wfO%(6dR3!u_#>PlIL4EJ*&&|wVQp~8|Quje}e&d}KdG-)bjphhC zc1WP#lb(Jy;*=UmUFLH|?T!#*;s1;|$9eD7jpac#%;R8;y&^*Wo1S9OGFd;|EB#o# zSb^7n&SGrRm*3I&;@;Z(WkYy*3wh1$m8*BB)eRI*)RioP=yuXM4P0{?__s_jye?%D zz#(U3ptl#TrbZdFy$P4xY^7oPQFgHC_f+#)qt6_8vgBdZ`&=#&^b}IJMgAnQqtLbL zssB=bGHq`5`?_e8y+?JWqOLXfUSJ?#N6#|mVd+dBkf5s}ZK*s{L0pgq8EsP|Lb~-$ zxRxV+$a#1|PkJ&UBX;ltXS>$~){&FVyCYo(U$v*+ox?jKy<$O##oC9q6SU0qU&Hq_vq`$nfyvYWq@g{HF0u zP*V1N>0^cGa+N{zNt(yy_1Da|>K=t22ag2w%xvR?zyG_vZvg+nOmAHm)3v4g(laYU zKw<7Fe&Q|51a@Z~`WAq5YDTxWhs!{(;&{>hx>AMRC~4wL(y>ZS$c3NUnoH)s9+lHGgvOn3DZYk)B8_ zR!ey@S*AqD&~$w%z%uw`G~}crc%{NC^KuY8(tElhr@rf;WeUF!Ef@sf!nJzp`A)) zR~kaF7-M68f|kAF)nG^hmh2?{HhYYPEMgC-g;oL0SKiJRu3a)0-vv0O!MgtG&O{*uv5R3Bu7>Jnmx42#ND5;L_`$5P*tzt7MkRv~ z8aQ;27|HX@PwE}(F8)eW?YAtlWZ1EnMvK(R_kcDBCrv8(QppbR9d@5ho91%jx3IrSCY(&w=a%hYUy!EeMW-UnPqlj zpDNqS)WkbUXO;qoa(K4~Ioa@(=5;PO0zk$X3WLjwQv-`*rVK>tUckzUX$%Smd~-uN zFuzpmWJF!8z6x-t;{lsYR3Gcyetpob{s0dXTtd;w(kUZP=B9A3?2LtzFOm&n7W+k!O45MFlP2!WVV!ogy~i zk1VFbU2PkhZwmL@7q^tab|u?5-_Mgaeer{--C@Ew$(~KR3gp})M|coqfHV9G#+PW^ zd#0?EbDrjY(()Z-9ik#zzHQixLVX>d5Q~*hJ4TSa3bSY zw3VTt(>H+e5V#&I_D`XQ(;x=dykn{`d%I!lHm~1&lKat5_pfodtw8I}Wf;$cY<_o9c}Npb$$yPqwKKUja$3--24bs@J(wG?Dt@*(#J84S-`-y!xu1HW+xu>OZ3i9!)DB!i3BY>7fbU zz09r#*=l1+_l2_VznVTd-I4#H=1xQr;vm{8Eeb)RgJRG!d;THk5vdwUS4AKZ#YE|- zZvo&;`B$eLi#7@?6sKZySPm(_Dhv2Ub|EZ_x1Cd!u^sjEhL|~0wLu)i2?r*a>Xq6n zK>Z=i2?_^bA1j)+)E}LYJYK`5$`nwy|KEg64)rkA056lcyZ>uF;Xxf3mcy+SH>-*0 zQCd>7k&AAGi_2DRJ*BnCVp?$z4l}(_l4pjt>;E{i(Z6z8GrN03tU%1$f4q4Fdr|4l z5Rt5Nz1lBeQo~jnD(YsANC0BtP&8jEPj$34!}^Blf;R!vTPT~1RPUxldlFFe%kvMv z%@rO_3_wOBrrLdUSUYq~)wxNhYuw2j=~+=Zy#z7N_EHP}4id6QPsB)I+NJo_yV11c zA-m}+y?-zYDqmARp=fKOVaP)#!ruF+$BP_&&WGZ?c{o&ZtVhLBh__i$3#Nls8o8p+ z!2FuM^z>D>Al9L+=zQ$?<@a^PuZjw)#VJpNOxQ3SZP>ZD>(d0j!+0`+W^1xae(Sgh zCz}(9$6l{m%(n+w5vvR7$3q@PEQ@8jLp;BEb_*_Ys~ zX(9y%^DmA=*3$?VpjKV=Lo*MpS}kba`%)aWkRL1(YltUKG@#VB0fRDZR69l6a>S61 zAP$K;B)3H3{jl zFRLQwc7C*uZ~J!rzO2pH5WcC(UGZku^Rl6!bY}C$j9>rRPLx9LgPk|67;M)?rufWL z7&Ipw zRo(v&(&+Q zvn!0Ia#^nJ!t~;4);55Rw0v{$f<5g5Va#7B3ui&BxtOPL3+#v~2t(FG3$*iw7paTy zIb&-qJeLr`#p5XOH=^HISSB27rrjH>pkk#!lYg4Ly}#tTm|-a8M=F?`!;bSXii3QuJzK7A z(TgeIJ;VT3zS=%!3{a)>mO?*UPJ~mjI^iyDxY5A64oq;H$(;9km3>(>&rRHjQ~|qP z5dw8qQwI`2O)9jfsT?p*Q`>x|6@pXYvZ1n}BuHC|{nHmSe_$1wejC#v5X@%3-dPrBe1Y=w*t4W2*5GMZP0FSj)7j>qP)JSs zFD;+q&6XpT_a+mc_xl}^w_irAu2`(-EJ~6ILwM!g09d1Eb$5yHf1GYFb^jb@{2C3x zsBC&1O`kP77mv>k40HK4m~?G9O0nTjYpEf7wspH>w(|8m8`tWm zwgddn2G6z|9$8yDTSKMb;}vJ)6{mFucRja?{vg~Xob-Y!=4M8UzQUU8O#N1fq;I>AE3&XO7I_fSYz02?wm>U50h{D;9~de&K2`p7S&G}aA{ zlkJ_8m&7S$Dz+!=5OWmuOnoeZH)c%br2{`2wDXE@BFa8nz+kVjqF{EUb!Xw_si425 zdq3ysMesrCUZVe>k#QFKVLrT)Yx3SAu?nzs+mY!)1p9yI0yt8+7HjDY@m>j?$jtn@ zMv{M^bg)u#k~P|}KJ0vU!b#p1oSZQ$ptqUSV<pqq!4c5m$)VkVCfx&sm!xEO1$JJ-*Tq=rZN4vqMG)=1XNAInd5ake1@f zd+Wp6&*uSU(Nc%;sI{LITvh{6i$5M{rzb%|KEtTh7CdK32p9KGsZczPd(Y_rT7OtW zJIvx%_;vX{O?GK5N(|z8v8B?XLnIWL>0n+w5Z&{knE2f#QmIpV={N!tw zrkmClc3brhla^L&kCs72kVbh8UZpw*l^-P@Ze)M1lhTb7+nNU6Xm9_j+LXNLxsLKGRsZf93nJ?Pzuwzd@xWfJ%aH5*xwzAnorFcJMWpQ<7W!&g_Qsd-lk+xm* zVLB!no9}bGC1}k?89?jNn3+d%ZjHoL2JbDlU0TZ-T;L z9JMDEdnL1s?~==Y2eS^pcw)hm>g3_RHuHAa=~TULU^%MtwC3N)hh*ljLA}!^0Tt?W z_s#ni!tWDw|M-H!NHeQ_Bt_r#aDvBEb%}4{HbaL+UF0jV`3WVnA9qGzqc-D& zm8Pn%gSXp@h-~+!EsoeO{y^~&gIPQ)Sohv;sWyTpY%~5o1i!J2eBVmn^CG*QA-+Jq zc5Q1KUNQfsDf2^c_|QesJ?>9oaIptgThI{kJfGqLFcP?qf)Dr1hE)Wz&s64I&|4Xp zps1ahT&<+TZ{4n;{OWs3G_#~XK@I~S92$%_E6SZN3@Y7DR>Ayb)2#v=S1u}e#Nrfx zD)FLXZu!TCxDX5L{K@NquC3<50YQJOukA2Ds=QZ}VP;d5$};+g#QP21PLL*@|I)Re zTLs&`vu}4GRW53U4+McvgBJnrhqp@tuDoeA(pRZ! z$e-WxsXe5(-Cy=^H~thiV5cvX&~Vb@?z^@hi9DV)%4Vao@kj6mQifYgSzP0E2m>yh z?!J_OLj6aD`m8DCsPAe_xxpN9-# zx3zMbm%lu|VRB&iX5^N3_Enj#WDh0oRDFP41f<|wJm)uB1u6-)%Hrj_iiq4t7Z;p= zVf#F+njR=!byg&q!B=j@lKF-^!<0YdALJ&S2=1bK$}cOlK77aFBeDElAw}DP!64Kn zw!@1?zhLM6ZGG6sgo4b{U?ln5(4&~sDuWT>dQG&lg6V3qt6eHjyUtm6-g(quioL(L z7%BC5NH1rn*#>}j=^nV?(FOq*yCyiVWNHQR$MbIgwYd- z@(EIyU|IzWZ#e{?HXJ3D1fI@?oHm3|ja7We5{-&1*wmdIWkkzl8gWQunif3c8*M7I zUzP`-K>+^GWCd2cphD>^P$HQ%ij+_$Uo^pDdIU(UB z9K5=4mOd|*#C;tRz0&D2H^1Y^pVHGWB5T%^*VMDsrHGF=9fK2YM za=41`T}QeH_gL?pmQGKK>yGv&zN#|FeLN8X)B-(Nu;VP2#qGR|uUNhx@uZ4j`_HFD zPcsHiZq|FDdygkT>gu-_f4wC%*8LFEHhItaM5aoP7W==St_N#XaTVZ0ZDgj z@s4+836?L6WXz(pQxh{P;JMUV1sWYF!pE{@bgBcZ=sm3^QOs;ml zoQnMV9fLkT#Hbv^F!2PhTC+Bgm6*i+`s^EieK!RG1Es--IF$vv_u|`ILmSz#KHVNL zix)kiR7)|N&84IbABd+<<1V+C^;GV3Q`$*QC({jOiHF6%GcmX%IT%9-g_ z>$78i45}4XUt_r*pNT72vauZD_Pw|$y4rXpp1K*h#r6kmpPZ64tu5p<2(*S6!M z8PLU=0x5y=;uouz_(nZTnMj0p&@1J^8bik#GY`d;se zzoA>iz9$R(g5468&!Y!n@(sHbv{npjAlEZocQ*IMsx7M(QG8IS2^(XaltjD>f%7ja~|00 zmwXF4&Tg)Ky5Uk6z*z7H-pMQ$6-FEG7bXpXEm_~^B)F7x=;{nNB|wf4LS|ETUa&?Y z&XIMa5ekt~{u*bS5f$8za!YD6wpdH1=bWz2jVWXOFiuO(Qo0!Ex&Rw{Y4hu?*jrzP zC-L?xnYPByJwyL_>Y|Zh%Cz!nMb_aqNj6E9``h#0-sv+0?}!YXiSpPz_Ff8j zHG9tVovhH{%M({Y&;Dzf=1iQ4_I*6Dh9Nv@pFb!Se7@xxHyDO%>+pJuwxZ)~@8x{K zchYyE#?ZOAchH#_Qyx!jjZ_kZuk<{CR?mgf003zvFP*t!B?UA)4YS4Xwb`iVBofst zi}`pIFPkgPi%NId5N_ES^E)_A414R$P2?5Wx$$G}@9Ea>c1ShZARLYHa^O5!LqvCM zhtI^J-3Ta35FA9n6qKJwP$p(pJ*|~pv3kgioiV23-PDXi$5i83z2i9*WVJDj&Gv-` z9iH&q?#OC?S4EG$M->wwWN&GH6hnW_^q{8lj^l7? z5xYvz5#G!CW3M9v^pER9=<*W z%zcsF&|C?mP(;2oH~Vr4Ucdy?L{weZ5!7g`Pb!1i`!q-bjwh%`<6G}dG(RuuOZ))` z;mZ%gy1jxeha)^$JwF1 z0011sP;s!$E=M=9kI6DbV)B)#OQdDn4jCp)jXdD;=Ie=bt~OhIuYBbrFK6n)dD8bz zi}Z^4WkLPKr**3{r7dP8N8$4Ed5Uw+=sRA#6DKA~Q}$v4M1F{em2t(aQ3{E7ACLT})%UWD)AZhZ$UgcA$7tCX>Su~On=h;--7)$ch5vb}bk8ZYle%rZL{!a#zA zHD;#nS5f%-%CUsP`HQqCZ^cG5idKeby#s_Z%^wil$ck6Hm#!7Dme#%Je&Vg}p&~tH zfU!*>t24ncq=VSJ0*)C{?@8kgqh}Uk&d;FtZ@o4y{;aJP?E)ye_G9mGyTmAQ=(D6^&YD&r`SM7( zT+VhC0%HJnwk6!QPu=Q!#!8GKRkRPOFe;}dJ$eTJRFd@^rEZp9hH{?eqG5R5An#Op zl%#fTfh-|K+9{Q3o(2u#F^i`u)!G=M-=Bv|(Z>Wm|F0IJv|upm_Qt1#(9MmIa%B8?7sH^vculyAfKLgB)%?YhC0~{7DLDbVtNq zMnPG2zhuZ7)I+#{d15;66FhKx*dUY|OP=t>=s4|tS>&PH-fvlbt$cE`qu$7A#3RVe z{qrz7B!U%Qdkh;a5`V{#)6`dS?b3oSvr4T~&wKKx5AVaiDMX+ThAnxI(lGRCrv-Yb zK(gc6mN~7Ed0)XDT-o0HN3Bjqch81%ufIaEV?L06SvVa^sm)&N*2;L(W5+kT{g}?X zSKhRdlX$^$kd|pq{l#>5Mvg(QOyql(R<;oRb|XHc&-8eTq(cHz1J$tlMV+Dl3+kj^ zpu^|ksUJd4U15)jo6R{L7Z64-t(p}J9y?)#=|+XK7bPQa4Ep*>g3Ap)!}+c^TuU$e z{rRy+G7MnB(wcp~iz~(i^8LZ6K>!?JZ4-Hr`)fTfrAfclCR#i6;Ilq}Q@*^s`7f;x z_6_c{+_AQwczKSp6>~xau)l1?Z}8L26^mMZz8Z5ay{@?6k4^JW#mkjEsj%lM2-Oy@ zJ&jy%(a1-LoHAhd52?Z?oOE2l-Ed(5+FFC$7uIO>ZTf|dvF5#;cT%wOnpB;iGQlcH z!Z!$)g`{a05x7TP;Z5MCXG_HtA|Oblk+&Fd^;vTZF)5>^#$t|GF_>K;o>wvN1u$PY zSXa^yrCPa@U<3sfRPY}Ez&%LF3xquCs-CX*1PAWx;m`)v0Ksz;t_HxbCr=`D>G)}Y z@ro$zV41$h;iCh-%s z4e)<~n=^zTD@mC;2j@H5ipq4+h$yQI;_B^stlb7;P%gS6goKD?LkqDd+`dwza74%e%s7h$N z-hsF(B~oD`3>KmBusd5w#|~v1icXp`hGHl~xOv;Wypx0vpK7rG%TyOEoWa$wYDY(L zjt29a|1+&(=*WRsNSLcgr-rmJ`pQAj_YTs@v!XxmV3vHlJw9T+j;rQB7VXj@hsnHf2$WlfKwF9usU3RZ_yn90INViF7Ua=gZdjG+?Cr%F zG;jf0W=h(uR<5$O$_yflnl{D*>6dY|o>s?-P{*sT;pK}MG>vdXz=Miq2{!87aIF^O z4n1&%ZDfs${+zRp!yM7ML9}Tv*9pekdVY6jKyI1vS*7F93{P#*P`+zr1#xK!EN+j) zO|7GEA9(|CHx%;F6h!BwBAmQVA;(V{;B`cEIB0y5@*z?%-+DX{J>+3LTP045W;B%} zt!tF8pQQAj#lKc&VQuu5Cu@-JhwQ2CCYFzo{NuW}kQYW5EXoVwqvW-b3?kj~KtIKF zwTZ|IY-%86wa1bG3!HT_KoFh-5htDS;MGubT1ZUh1dhgM;{a|8LvqvQ z-r8t&dEVlfQ+TiFeNuHdA2`Tg7_6*B;{Jx-$yO%4$8DfqkfeVo7AJCan{_+aZ+e`3 zO~yw2XJ}%A!nkSmv{*wTY&oTGuv&WwK7OOJ82w+k_HMGV^~bcrY8(5yyFoUvh8D-s^2wCZkF^{t7 zV0m~v(W84~Vtv|{B4avt%RHrSsmR_SjtKxp4PjKJJ49&I1;=>YJREMG|00w|(6;_I zV$9`S0>(6B|5UChY^cJ3i0-ltWUmDt66_S$t>BH!AE=6zP&%xC2)SPRyK9Q#7BU(o zkKIt6n^1^KB&P=o%u8y^`oH8nY=P?u;{x7`4i?#q3c>@W$)U!J^4iKkMD8t4v*A`^ zgo|~G$KJ9RVOBAX@1DoUJbyFZh%BFMwHOCY$k?lfMd&UuNDm)BS5MIriQ3SVkMq(F z*P&)}(D>y+jk0B_fwQy;5S8Rp$Rnkkf#P8<07GA{Z+fWo!Xi$cXRLaO;EtljIgRF;pP)G32Iv znfM~ZwG|RVB3Kp8wQ|60%VGBKR%y=BmwR2I*YP0b?}}z0)I_0>M8*EW#e?KwyydOy z#VY+#h<_AIv4W#7fEVx(pZm9}%_bwa9_x-(jMCxQf;m^);`jXc zT?+#zj%*f%JbM1-AlH0*jQ&NsDyATZEn}7anLNX4ipxFwu`{aty2zZCkbcd_t8oG-mmo`8k>Jfr67`q+;)>G zQHDJpF3BOz=BDP(dO9M7v(?`Qi}+m?B8+5Im1`UxvF%`KTxqk`_^Rsy^WSek%TkTG z;3{7IRk0YA%Khrt{31u1IjVlz47gta)6k!Q_73On)I9m7r!tarpFbXs<`$UnHl{IA zNl#_*4g~0)b5RFlw>^jt4#2)aS~}Hma>69cUFFq`XO?GgjX~qXAtF} z-8$to@&Wn1jjZWX8D*qOmarZrnMr`V=ZqUPA}NUS$z!;dE}mFgz)y%r5he~5U3-eP zXDN?t*^UhG#_GZuFbLOK9Ro3jYIO5c!R8Eps%Qg*g+0L1NU*|$_3qS9=kI%U(_W!C zSfDea98!yuReJMovEfZVpQ)!WL@zRt<2t(sI6c~kF~r#3?Uf??ZI0{@WzDj;7JE^> zIHQ+7;7ob@b-LMjVp>7^eT*=4EW)?Vq0D|6j!>?;j>oo%=Jz1Hl237|o%4e;wfTna z7o>M{HVcd4VBvas>xssbisW}%@tWmKM9kIZ`Ic#OyFr0~CGkOReZ51b?SOyt! ztJ;8d!++wyG*^Rr?~5>S$gbPss8-9=d1&YTn3NT#X_xO^yUtQDx@7!0$?SUBzprZV z6ZBpl@W5wm1C<`AOWgc8db!&lL9Ohh7Q+Jq6Q+;$mW5Wp19&k7W8kK8*V?Vm_z5~`-8AtQf7_cpQ3W0PbU@t=5$ zRobD+%7}*Y(Yi-6ay1!w7?|8^?3@bd(P0@)ahVr^QwakhcoDZ?0uVY86TFm~L5?=Y zHvjgsrF!{{EQSwi6{mqH9m6j~YWmccWwZfvRuT$M!@|BJNuj%un~yGIqQ$O|GO9i&D!L3$G@0Ro~x z=qM5CMOr8!NDG82T}bG?-|c(OchC9mAHOl~y(5D$_ygA3&y%(Hde(gAoO{O)Cgz}v zk_PIt#@`V=!a2L&$+oW>kYJ|&LxMfMCFWOS*UjD;cIbE{#it-?-@S+7#hBWj z;oJUn>;Lr$@Fw#XoEQ9SN#p9&JHee#1ETOsK)Z?$}HRg|__B zj^)uDsxJ4pPowVOT|%ILSMl6$MN5SDnnd&5NyXmT4(VkOKl8?IDC0&E55~Q$ml`y& zo4laUrEnGRLG@1+A$6U28|DmOsr2NPP4t5W0Ywm>G|uU`S+oi$RaP*`5fkkIb(&nVlDhRjMT!|0P~e|{ zvtixZKm=;?yP7z&IEqq_ECNZvwHpPsuLvY|6%QKuJ&b%_V>9cWOyvw~UBLdYRcl=$-V911D5{XV{JMo@C(JdVRi z3u4sKbmCZ~>`Hy~qp@KG5uTR4gX*{-(93Tf9k`%GB_lm2ZmJj3szTS9G6=cU)7Y#{1$ug2rWCCE^#)aC6H{w=kH+a6G7FpNGU0@W$+I0 zBA&Qp;%%h%EBl<$n=-?!lXAOgp9oO$jrxV;=nOlNb;(vW+aZ2idfeHIuqnOoy}AT< z&J_;CZemP66vGWmP^jxM(8C;)vI996m{CkLqRH|`qB$k^a~i>a9@|-RQIe1#zJ7CW z0wna*#fB9(9!p5#7fJqS?4r(fM1H~i0?0Hde~iM zpHtL!8=22U1L5+H&C8%Ee^W&o5$*;!a=U6_(=U`3YfA(t9wz39=CX?bm`_(!J})A6 z1?0}To7~vt1#BSd5&0FKjF#tBYXO??Ee%|9%cRB!pQ;%oYVLT$p1b3MN25|6N~?4| z%Atsqh9I}4Hv96J5R8C|sGe3>^k^7c3d|X}znZ@AJi`5$droq%NXBQxUi#`5asP@G z?Q!mQ_84nWz5i6+Q=Z)xi*`mI790p^mzoiu+-8B-3~Q`=l)?&n&-Px8dUs|PccM7BB_Y~-fuAxOblJ~iV7AW z3jH)QhrC@&_twK_0NR)*YFWocYP@s|s5j_;r&V1`gc0pJ_iRiyAZbNXij+cCB;Wt< zZ;YgH5jFF=MARdY{L@}QQtfbPEnE425y^nl+UcUvX$!xF-97#=w3-@CrauTp(O zEOc#rAG11eZ@8Y@d}4iw(L3E>Rrg2j^}6q8nU+mKCNounkC2{pw@hh$r$L*~+2UoM z_yz7*X;Gnz)Y=`@?c!0i99Jr&i~YTAXpYs4IidcSo4l1AP0Dn3BTWgmG$0*t{?zS5|J;_wVqzY9hz~;dO0k4 zmE4E$jx3d=8%Yrb#Ek8!pJB5O7Fobo9pLWE#YCJ%x0;c1qz~lI#xAg9j)qGf6={ck zEg6%r-HC$@>H2(ko3wfI_=0fg+eh6oW`M*V%bd}$5Nb9GN^-3 z{V>OKcGd8v*e(61yiz1bD)bH?Y}sHx5xmn>u1@9d0N)A}*iCJ`W9@^PtfWt=uNhi4 zDcq2(2PABJQ4jXDQ#6nHcq-F77#$*-NnqO*~beYwkY8Ws{dU035d;@!Mu1arI1G8a5);P@XR!xvt+ zA<#!t^MzZlyFY0z#pK=e%g0}Nck?${00*q1h2A1j?T^>Qok5SkF@gf;%A2=>(>X7^4(m=NYJ3@~Jn+{e$zYf`Q0V2&3u3A4jK zSizI#gRv?v8|1U!AV5r~I=4&M^GHE2_yo}&OvVOiXkkiMm$2(z(#z-`^WPZb;sMi$ zA$$Y?7r#$%>@tg_Hs;x&!Ez4fZONE$Nu3nB-nOLgsFYLw3?%_ykpbc8 zbds@NDK{LE`RtsA&>7&sWF|dQ(;TUnGaV@k7YHyInN8-TKIXXj#8_h!qwX?w3>Mac zWVok!VH6`w=Z57W_gr|r1@Wo(mS_6fvOyF_lLv7ycSEW$#~*g#0UhX(+v1LM1(-e9>RJY3nG9Fxump zV=h##$|AeFp?C)UH1b!_ehKIAnks!~z;ck&2rrypCsiL-Ri3e9tRyF$VH*;@_d7VT@J0iWo%aLqHR!AqH1r}ZN$ zwbiH2z3FZDaWz9~_Qk@zV=PHuUv&|}aUg_rQ__Eo zF`g$RRUvt$kPtq-IOEPGhLuBBjAWyGK0qB1HO@3oN3vJjRSHQL$So(Hj1v4T9V@wL zI!PVrI6(Q_b8_|~*<3dw2x)%+2?aBvKhrINNgBK1rV&qnC`jAJ+Jub@QDqno@TNmz|G|NlC;_vn2PY8yl|8ruWCkn`aW*jWZ$tiEuRi z{}3VFjC@?^yu0edsKdHg(o62GyGDAUc9mDrr93lV(Mu?W^%;GQaNxv2ntt`(7WgI1 z0V}dDvoY)D9c!+^UGJ-8W`q%z3t`KT2e@-V<-!LQXo>S_MvHy#YNi~y2H&aTj{k*L zrGY+%Oq@}?I*B;nw(sw}z61LaiUxL486#TBfCjo3KRicVR!VPb^8T8SYXlB-OGwBCe0vx}`4BE@I zs{|)z^{vv{;}*Vq+C%r(?#7~+%!MIx%Po@9OoVv>X?PVWy=3gIS5G^k;Y>CdygZyE-F^0+O*bwGz-?B6I^7YajfZ9>tI3zzd~ZuubTwBovIhv(1PQ5~ zc*Bp$@j1TxoRt{n`hG{W^xn!0IEn#g0JlqnfKsHjRz=qhwWyHKdpwd>xJOE#Hq3({AjXZY z&UHZSOM+0HnSyBMOMBu0wLqQ+N-d~!2`1!`o*xX?&dWNXxRr_ypTkADnG=K$9;=Qh z8Zz3^XXJW%fUg2@b(P5H*AAs3Fl{<1eJ)&l>~hv_n+ezs%ZDKiN@)TFbZh##Sc0rO zr{V#iQ)jWShezeqZdiqI*9$2D<$iYkSYY`B1XdRaMpsm)WgPe~wJh*%YR113h0?S7 z6U-Q>B$iaf88IEX%(8U6vMkpCR-9jIsACie{4e72ucoZ0O`CN>X|?yf6+vI(`il$#o7q$C~N-8T~Fl6hFw368JoZ^^>mYhqx2$9sW!ZZ%^w{21c zlWX7l9vLM;?pO;ELGKN`-n;WLY~ceP-fL5(bc5E@kGr{BC6Uhs>amW}%bu);#xtt> z83CTDfYw1j%v%62L%G10n10psMZ>2~zn42YC+MuYs z5I(ys3greZ&_c4}V(s-+pw6H7vaxHvUhHjPrUbW30iFv8iUrynejm-^uL0X}09FnL z4Y3e6RutsK0DVmT^V+tRP=``)%^*%s8MN1+w;i1nBPyAEc~IrKr;V0D4||gSK|D2c3=VC&3HPt*65c0@4@0NK1=Af}#1MlhTSTOrAIM zALt67+4)5$YP-+H>j_8=Yd7uD_Va;@@*i4ZsIQI-q5)Ap?HrLo+JKe zhM}7yl`#$x{Ex0x;ZE;&@TD?HC{XqO$=yE_1@JMe$surXaA*A^Zq(){;&V zXLR-8hx+K9^{ty;uH;Vjm0-y|J?f1uC4m6oZ@e0i=1Qz#pG{ zid62aUjkMavbs^GtF$>gNS+)vk%^+DZYk3^qPO@Fskip)_YzJ_)nv22tq(k}ay|>? zzHbBcEP!pZNR7G(Dwhzk@KbxcY=du=0W`z1U6kd>kz5`d(91Wso=Mn!6;@*l-|$Ct z8{D?`Ip&Cv^d6J-_P{L+3G7An9|7GJYQ)-b*KTnwynNJmn^0Y{R;+u$7IN+oijr4z znc>g%UfkY8*_8L5q+!TWAKbTPWkri@vjRQ4P_!oYBtG!hH`R7``sY)FR+5h=n-_c< ze@BDiO0K(Yd~-|Ez3WAa7tr_G)x#jUWoZpLSxSQXK&@nWW(8?>*|_n?ZrXO_$I)ss zrOnhH5Rd!Fr-Op zK~HK_f!CbzAl0CKN&>wybi*B^I{`Uh6B>}*Ie8vtQ@5uqzg6uKm_~M1(_=iRoF(uB z4VHp!v!dj-q-Kn_Yg4gs2RnarGwIP@1HHJQ`f<`+@vvq8%q;vd%hX?ydxN%a-SdhM zWgl@<7v;i%I`+TXAse9QLX_(#DdktE=V@OEX>Dw=dRdP5`e?(_{CsbTHT8pg&3#E}U>Bct9XXQUmq~6Pi^+|r$#(61q|)N#ullSiVtE2?{ti8^+g_8@+PA9+^dNRj zSuxlJT9}#ERg+{^KpwtprG;nW$O&SErJ&6!XhgdKzb#oGgU)36(V2OW^td*T1E~j` z<=6ROQOnU?Ua6;~Y>FjjREWPV-CmU5kFD(WT|n8XokIvpxDE%7JS)+Tx%?i5-A7(o zBn^8gtEi(E95_6SB)16y^J3%MYc1wU8es;zijdLp)7o=3dBVW4RK%`gP#xz#lU(`# zG70e(IN!;gCS7(@DCk+o6OZCAOj4P?G%SQpJ8W7Z(*Z+&48tNq-gb(FqWX^*I zuC1HL>^rDs@17IjHiuIF}?=u*=csu2;#FziB}oEXA;(tb$(1YY%@ z&_;B0>!`Y@8x6SptscE$+dlTGv#~Ui(g=KM_qX#hxYR~7XkK{?8Hho$^GSY(@FjLz zEO;s{)eAW;_uzEVim`e*P1 zRyUO8`??#S4@(9|PR+djeR*_IB0pz@LnkfA3~SW$8A80`Byf0AXp*i3l#EX!;=55I z0C^c*SCUu-oB_h=B=aN4IkRr0(E8wgFbK8qSMIrHbPq`}5FsYQP0x3QgH_oCqJ`q+uf^hYM9>RA! z>PvKut-q*(QMWeLp0Q)rF>5kpOn7AJhNCFd2Rg(-BRVDF4lN>dAgM;ZfhN6hZR?wu zhV@@{p@RRbbg{^8aa*)+WGMV7Dt{OLExh6%C}=W-mSMVvih84Yubgg5o3_TwTbZr? zs6WKx4nSVm`N1RaeJxMURMH5VYR<5%Zmu+WTDp20Kls+xC`l0sNL%qxaT=nJd8slL zO);Wv9a&IPnMwihGBxetcA8Wpk$O^F`l55r+eJ%~bX_ZVBlSY7+WQVtYmW3aj95`6 z>RII?kS5kU+F^3tnn6JG8~5rtqRM!L1Iatml^ZWr%aijJ;n{v{PCxoyG39*@J4+5G-c6uZ&ajmx!jT!jmwkm{%!uBpM4kuDAeG0(Ba zp~(tuF+&{$OXtE*<}FAv(L?j9&x%(e;V75)C`vC>a}KT$VOO9%atxZ#qA>VWI((|Q z#C^}ND-|*uB{?!{@Zxg)0B-3diKR^`ms7MGdaxr;9aLQp94tu?OOGfDa87icG4#Y#xXY-OC0s zN|7|T&kFPbWk-601E+aXlKXF_=NPa8OgcE7)Ua9;%{HTSz>1eL=qqry|Mw?TI@JHB zK}FWxg_2L1^(~917MU#s{z#%E%Ou`f3J`;3wCp^&eD=qAb`2({?UF{yXt5)s-lFHf zFqgn>{8gZCWG{ypeu6Ur;KvHO->CKZ;t#BPN zD>X(?TG0FBm9UKRzIi^RkDkqeK5kyX`-Odzlote$u!=%0%fok84%s~+EzEFMPwoa(=L?h$zQ)QEUQw5Fn6aw$uv&@G9$-HrA~iYUiOKeJ3cd(c=({w-jUdf<%QS<%OtKSD`$@` z3tjhsdz*beqhIR*^1UNj-DK2rMkBz2kO++kj9n5kn@21PBMCCv%0$v!x&wAQkdm%T$n4|Aw63wnE5|&y z6gx{|A^;Q~=bA*A4pkN97{kBG(d^W(vB~Qb=O8nZ^?7YsZ|lE&*Sin17?H}$kwWb} z4J>~eu@aa-wI?Mp7x2MTUiSy2As+_DmY%(cFo^mR0Im1_Dm_&xG=ha0?2F}Sdaw*p zkASn-bmh@N|BG|UXF@i(5Z{4pa)+x$NL}}_c1AtLH57ZQ%6meVLl4bDsfLc&SXVt= z+7BKq2j{&(`fBuhr5m4or2;g$2Us}K#;4BSvKY}{OmdV%EJzlx7kf*}e$b}F6R-3d zs)r`x$X@~H*<^kXx-EU)Rzc%T&tGJH8+RweGoHOrl8*%tTS%ON&(LBWc(=2b2DR+M zPP6>YnBUbwxO9Ef?L^vU)<8~5u0zPABma3M*_juSb4Js(57avHHCpm$$yejw`65&S zCn#gMCiO02i%oubWA4vnE032STT0VS<|~e+1vA97!8L{NmBnk%l4{T~#gzlz-Y=s( zxz>HlvrdvI%Ja#@7(JEt=O1kWT>m*3k%%o4Y5Ks zDgo#}707L6nsIoMIheg&xq(MF!Z!ttw=zu zUW;yyN|c5JQ@6>!eX209SdQ^|uuiD1D2-WYlh_3tV zaO_Ek0}eH|M7Oy3sPy^b_M*h{bbEpNFFlepo!BM5K@Mr%2;q`g^esdzy2rRqohnJz zTz5zn54fVm7U5@yt9fNb#gj9lYHpPfy0vZ6=s!SYWT!chdx11?=e!89}Y-1T!VDoyzKC=>J5cVf6oq#*C#Y!b{C1 z!UNtfXeMSWdc9~>NoEgG_MT#_1FQd_@{NAxj`76aCxulX3cw4e6F2CL&mR+Z^6g+Inm)H4i@ORTS)ppL-=G}#p zbJDGxM_Shn&r2!}o6_X=lNB9+qf?CsS)g$Uj~@;GN_N}L-$toTEe9*qO|CzOH^5D{ zOu;-EIP^bbw4vG0-Sc z%c9t6vPh~1oELx+liIvY3ZQ`fd+bWC|DR(w+}%|_t_)}6dZ9f41Z75*Jyv! zp1-Ljba6SS_v42>p#2Ti9omwaaDSmh9ZQHgjqS^8i?@|Ar$JqJfYP17DV7omD~W1V z3(}d>@73D30=>H>T1`4&YXb>$p+kGD#bXd*w4y`TlIB=a)f4<5b6OxIJdD26kQXo- z$-O>TswKuXoQKPH!sPPGK86Zyi;53b5p+(|*>*BQtOy`nN$`5x%x<)3 z&X8Oa)7V*sj3x8JI6&Yq&<>Q`0!W7>#M7p6QhYVl%#gi1{z~1=-;dtn#JmVp?C}%+ zFG6pfm2c;}Hb|xo+yu(&Uw+7ztyRk4ibwr^USFwS!d4Py_ zM%vA^qwzOJQ-bs`k&Z=)Ro%ii1=S>o*!;rddK5jKOmrV*lBDpA`W&MhCnV{`6*?vb4(3L}1QHJ+V0Ygaj6 zn-9@Q(;wDXlAPLiId$wq_Xit<*U5(@?R`0{b_p^UFY{#lWPleox7P%qr8_!Lnf32u z+~M4)A#2NHYk|E!<9pktA}=69lM`xI23b_s>&?L zlfk^EeWqN-nJoSvX&t}?Q0T$#C$eQtv6Gc}QxV1?zs@W*TwJ;2U-Pv{PY)_XE6d}R z-99Plh=`e8HiILkzaEOOY5tEFfS`DOq-x?cpFz#-j+tYwjbX9->n@P*U5}E=d%#(W zeuXVyl4|g=cKhkxMuiG^MhNTiKDK){dohd9dy2-aeMvK3j2~vdXsnoHzUyXCSvKGc zmUwxyclAHZEThF61W3e|0AF39JCk#dy7ZB~HRbt0TVeCcF8mAgRciSrJ8!HL>I+bp zp8;KjxNFo6XS#NBr(^X%2d=K@wo2t3J(LVnBD&NJ7A5XH_(pZ9L7kC)uYnN^mot@CE%SNw{;dPI6Cd(XVu4J})fE@hTpkHT*;&%K=-^ zPfBOuOYi4{vi8$;%`zD)Y0dvDJ@~DUZH`Y~Kt#6qZG#fJXQx@1g_UZ+{+~?4TkpED$R&{dGsIMgGUWh@RKKMd%*o5VjKh1mJTbSA z_=PTlXHJsupHT)O2gSFZMVH>+>xPl|wplaV#xwyD23ui^U?&1_+gdutf}v%$@NOs0 zl`wPN|2;RfjK=wvm!>>~ zbr5mPGVl5(i5@8@D`wQp>!Q%j3H(n^-`0EeU{l#cE^agvpDm{3A9C;4Qu=?6P|MXN z#DL3XiDnN)0BR_U2&+%eLh^h~2@l#6sQWodnG~enF^VWRGExc_N4`a}VTD@**r{j% z`#;GA0Py`r?Q)<?RPbrMWivt;B(y%}MsZ!Pq2GU~-6YGzK{eEKqEuny6! z#_brFN9M@tQ`H*tv_JY*g&;Ud^tHOZ4`;bO9opQCm6g)GP#k9Hca`+Q>Z*CFFTE)@ zI*MDA#)z?hkJkq30`wb!T46o5OqqKmT`TxF_d1jjeG71_UoQP;(V(NvV`;4g#JUki zeE{*!s2t$e6OdV@89x2vr1wQU*+mq=`lOl{Kff{`McKH@;mv@)HB$VAOypLnRd!#zAlBf+M>2=Oi3W%A7Lz&; zI%$#K^#!|;h5$S?%`4pi$zD9bJip*YI9XVw6gVVh+**Dj8{@q}*>COqpVhLSdGj~Z zUqM#fX`+6z_qXc=xj^3&>vNtq9%8L-FUP#-BzOM9uz~P?bmjIVWKIVO6nUF>R&`c& z0d8b08^|B2=VGff4X5W8o~sff8hpGt&GALADrx2xe9R;H_q!SBjsqIJn_u1$zdIw^ ze|MBKgAJ^4u_C%rp%J{gaOT>lJvmkOt7%#{N3v0B{Z!O32%gJ+3={JA%(_u;=gb*! zJVH*|Arx2;rKH+O!y)U4y^r+obvJU|dktcoJ7XvoJF zeMh+(5e>W?%C}Qb05la`hIf>CuKgmXk_&lGg?1BE>j+|)c zi!$%RalXPb&yX_K(hy>472mjtGm;L}Zg}?tW5;C{20LDEY+_;tV zIdj%@g=1HC3px{ow+L7Z!qsjxo-}R*xCTv^HOs`vNRs`kxW0hWX-JzrWy}uq!0*!j zO3?+KO|y>*nJ}@qQ!!Faa0m4vSYqTCfk_X&a(rup;VuVbD` zYM)ss>zh)B4J`}vtt@k(*t?%)^9=Llz<`*=dgA?~ME*IV^#7r}G9`gr}- zZ{$N%n$nNgTK@ckEg(OvO3FFw^V~}N8+1)bX45M0W1A(L7A1nLfVCzD+>*A`2H2^t zwD&E=2I0mTPo0%J{Q!P}ybOM(mP#d1iv;ylo~eHGbzn_?M>>jOFlLjAn%TebGGXIN z&<jSCr=-ntNTobB4yxylZTA zrP8Nxf5u5aD0!T6#*+`3B_yJk`zebZr;u|qe_WAjJa8n5*t}&B9wShdU!VKE1D);J z$8p`Os#9n1$2Gpy8+mcn|&nZaNPT9k}T0u#&~AI$F|+lG9un5pnZc# z@Nx)wcNER;r+1%nU6{k-L1$#rXlN$%NpJrt3 zehr+lI?gn=5)bb9<$8&a`L<|3j+eapQ2TRVVDLpZ-7|T=28%hz;ANE7U)dc7QrTH- zbh2`*SuUf8vUtRl_Jj(u&#}nbEb72nwR9et^Ig>UUN!3qWxKBc+v1Kz?;I9bj={|^ zQvaQkU%nOm4x0le`U@!W4e~#8BN|gP)_j*arcSFD8^)C{vMOEAQoif- z*+h=iidVdm71|{%WnfD=wXU-@$z{%VvCAfb@K>Ep`YpmIR$*Lrd7!pDufI{N`RN0z ztN4km(FPmz!KBx6IQ`{I3L(g+=L<6z93>Vz6fUzg|H0+r31@nDifm!uws-Y>Pewa zb_^W*J6kwV!m`l*@H_~6mP2+Y#Nu~_*OR)V-*K|`zkZKLE+pbn(kY`zyI)(WBcm}0 z<%I+`&J9g%@ys$;6H0Pi(spO(?K8VBW!fg|`^@(uUKK*Y@JXKX%jcgN79DJ_PtN;peY__kS*FbRi7RQ9 z{B+m%p8s7k@`q0$u?hH3?KyjhV9Sf)n+%mTIRvhd-PQ(nO8mkO@B$Pv43l?p~U8{>^F=HEYgcsRaQL?HU+K=^CR}8^hqXTEgA$xl_H5vZ0>^{@(dO>&QNYf7Qk4 zZ>wb`f*#`xc|%xivX3N4YsXjoe&|(|TmH&wT9`-pv)}vJrI5!@=(~+8!XJ7$C3UJL zjI~$XX0$bUVxctaq^bDJRn2C`LG@bLfQj+}NO<1d!x>mN>GiwK>pIZLV~n@UFZ0>V zOq8E4{z>)cxH}9o0DkNA zeh!RN33rP>6%a}a)@mInbwY}-S0(sJ_d>8)Moo4l)WKfs9ci;q^ltT!UtdEXrwAcL zr<%omNSXZ~4+x{oNlXS8ioftznKEGIAJ(O;T+>wDnEH4*|U z*zDr466YrJshVXj@e@NrO8G;P9pL7{AB^=sNIoq4Pbm8?pmm=^tClCYr+OtxuQWq# zOY<#Z*cIwA6Y>`+9Sc?c?EH;NG!}9*{;b=9UXDo} zll$W8W#KQsFEdbf$G;56?{FBGVw{OBv0n?|jXZE`P@OWf2Yl7or51 z{s+osAG_u?Mn_d^7mE}Z&AD7u_cyPMBKmiiSUyK6o(0=9+?$&<60|X~LpPhhO;Dad zNHOhgX8aAQc4Xv&VNGMax&4Ms*>%^>rqa`uWWEY^;P*3aqCVWoc6hRnUb6leZ{rrc zZ+Am6>uSXSqfLbEqpj!fi#~m4gI=ofXrLHB!(?1L2%tNoK$DE=R$rguNMPA|PvvG6 zCFax5X8U%zkh!B+CVeCB(CdEl{5OxIFolxIGv(A*o@q0uXX_+a#?HXd_a;On#$R{C z6CDniM&i=VjajtQKZWJ8tC&|*k_19VL-6UFea>dynb@8&WuD#NqCv*6c1^|ckL{m7 z@O$t{znjB$GIigU)(lCSF}TSKUAZet2_GzRLprB!W&Uz=P`O**Ug&!T4Xzl7{hQ(P z@Z}C&>)eQ`i75k><}9BlVce#5Y3{qdx!J+_U6i9O=eKW56Z&cCzRMsX!-5dXTBQ9# zQGAwA*3uNex%1t{YgI+nOuB0(0^6~V`!X;^L?4+Xlen*fa4{(Cqey{F-`k)ins_Ik zZ|TZ(!c{mJ<&E8xgu;cdBkS3}VbkIPv`P&pqODFhP-+f7-h)DklGG~ZH`N+SEW(N0 zBL3|;WjPP}Wq;``FZMHp(L6<$zF1WYzctLD%Tt}W{qv)|=jyPSTfVz0YmB48SC?nu z0{xBl7t`)9>tIP!12t!i+U5acLyyG{-kh0E(7Qvc!~-w4;8**A`}J_->$w5uMSuF1 zUG1F1D5M+y{9ZcJ!SZ(<#naCFu{I;2mdoc#aD1V9vVMK%ApWYJ!$#FLrrE}HZ=`?6 zBU`*aE%{d)UYi>X@m>ng`UG!tGv6UHy_e9I zgaQG4MskU6JqoVX_U``w9@d4D)UYX=B{vMru|TOvLKHK#5)ue6Dr8R3AO9IM-zp}| zSlNi-2a7=@Wx{w3TuTuv!m{!;GQ<47syH`ZjIx@i3bj+szW+_E|K&ySWqg@jnlQHp zwR8W^+fYm)g^x?5Y|>_V=0lirOr@aKUGdMwvT0=K*zLALdm~`Tx`4IXbE+I4$p>9$ zViQ}j9@L*dVa{Bn$t(`)6j8<<`3??cR&01H7Jw-q(7;mZeF@g>kddt*=Urc1r5< zIEYvVl=s*XLWfeaZ}kqUE=@nqvP~4pV33bpZ1grIbf_h;+X}LKFK+4ypglKQ)UG8k z**IFb?bx70r0z9)zcE=KqdpL^h~IqVs$59*{B4EAUNXnCi%<^Enti-dql0hM*WYK?&s;#|X=vIrPfmOTDM0mh>-f zWt%#qm*mC1^fNnN@F7Rj7wO);13#W>etDz*2jPItf9Ul9?8#tbtYS{jT7tpy=;g3s z@CI=R;V6!9bc+219cwDj`cl(*_o`iK8hGfr3in^*f3y6pz56rz4d&?PZE^^@u3b(1 zKCR-IUb*Pf_H6uS=6Nj9`TkdD0l_{N<-DL-a#3_$Rv@2}t+v9Z7 zx47`kYGwjkG+fk|%AIdC+gR_;$4qsBdB0~^UXnQ8xpR~9YN9a;r}$Ldl4|T}KqSs4 zI&>=ip?qdbagvYCy=~9%Lof~brQIRLz|sf3{FkEDW+^!II)x zaN6?LnBVSy#nQ0Q-L+lE{(a0UK)ITZQ>LP>{U@All=jS~EzzL2MNe{{C-;SO!mS+K{Q`9(j$G0s;P9(zzz z$xELP$(Ys2qNt@nYvT`8AEzD4FDr%)y!8&fZse2sM2xI`&8Eey&Qd>`^X+NtTaI1` zWmOeCIqdZ2w9Ge(qUh5evS67@4W2t!KAZz~yW~+irwLWhv@>yh-ZCoQEbm7}TP<>e zU;n6h^IU6=A6p8=_$gnG>o5@kK@7|XYnu?mP_`&i5x?#aE*_SAHp0Sa;6MU}tU9;| zW}Z?0t=weOw|}KQNpyO3nHg2{4l^o#SQ}guQp@J1>9^4?(Ga)joC@(|$e~vjmUdQq zlWX!4&AgzUq4UCs9+HnbQrx*Apebt==!*ec9@F1J2YWoJyd1@Gj5)UQL!HnIY#*t! z63YV!lRx_Tl`}9a=k_y$65r$2xXKj0x(|B2EvB4CK0cq$}O9>f6KRdQ)-7z z&#s&!*Dcp7y;vQyXU>jBbFS8qRR!0SyLly$<|;gQydP!<-=s~)Vi&{C8`^Z_6~2^z z-JD)QKT+8QSM-w+*d7J-wF-^ULKC5fFTYxpML2yp?U@TANA1FDhnEAEh|At5YbX&F z@VLLAMX#K9jJa?}G9+`XLgH8B2UIFsCA0Y27XnFff0}NI>5ID&zf5%IlZp-XtYyL+FG zj4K8RT?I0FZ;N>@s)bsjEyXZ1RbEg@X?u}sY1%A;0C^>%`-lWOTp z(LTaMtAWpB2&O-S5+dJo{xhUOck9ogxB6Z0nAe0dH{bV|k`KVC^{8sENHd3&6RX)c z6 z4y@!f9ke{Y=**{mh)1XN+hz4kmHnCK{yUeHq~y0L?x8W~Pr-g(D1800GipW!LC>%o zZLTi=ZsS7d56p~*5Pv3 z#%HseuQb`v-wnLl>5aL^pw@8H^e#>KVq!^h9_ zh9RF>U=z0Uwr=Tzx1qTz*COgj57QcuXE{?duG3`_|Z}Gb3bE!gvR9*&(=9 z-}YX(-j(C=$k)aJB5E>ai#;v0$y6BN0($=1xM`$-n?Q=XrUli73>oBpm#)i>J2TC) z*<<-UbQTj_L~YwhgdD3d#@o6ZLC=<7(LBr+Tw4ec-Ohe3=t_lHx7@f0BiAp}I>!(mu zD=!;p688%*0R3La5(Bn*a)oOS*_U*7&Z)hB6dlZyT>{lp#9pOlW9gtVN`)FulsITf zp>G4P?l9Fn;c7djrR_1eOR458c|2Jvx6t|)!=NC8MtNjB5acqE$tW{V{r7i49d}4Xz1hc{ zgDO+yOO^a@ACn)r&N7Y3PVk9IzS2GrI{xTl_q|_e?TXdBx=LR?v}a3hl@ZzP!+*AZ zNEK2=Y?3}?uasLo)&_`NrZhvgF5TzP!cg?1^w%|mflxzb|^25K# zI`%I8cMhV&p{p_(2LV6Ml>|DL56bb|&C{Norx{$@{rtB@E)~g*emK;uWjgE)D7N;J zm21X*^z(uZlrJ57uUX#s*%PL)-!3aSarO)5TT0q%-i=#te^E0@-wGP%nSr>}d$r7UbP!JVWRrrKr^H4A+@r&Z3*;nf&LmUCA&~swNdn3>)RPxDx6f2_nzjLM% zYff9^V)AKHS1a-iFc+T?v^E;fRSOY$+!#)a8{H6I5K3Hnss2HeMM|EoZ(rlqj&b(( zaX}6y02ZGoYci;2sTig_NaKnmKKu+j%(h1jY%^2)WxjtO&+nT;{+*a-_mG0|teYZY zhcnlX<8)+B=(0|TmFUQ?UdBIQ!bSNekgDfjEWE}LlTr}~EbVe;&XQxadHwW>JTC2^ zmXF)uQ7d-0_9DX>IMeDZ>VwKz6%Q}Xc4XG0y`c0LS@zqne|q(^@gFfsWPnr6Ea*N8 zpD=m=z6gD*SV}V1%_LIp*Y_@Cgr62W`We>@m9ob?^Byzmu%`;U(aPjY@Jzl+5GsX^ z+Ch|pioa?cMb7VrS(yaXvoD%QhZCh;3<*^0HHMkSm9x69wh6bw<)XsYgiOGlxM^zr zfgDwVq3#jr-1q1uOJ;8_<#uK}8Gj1r21w&md+*QG6+&YD<+I88F!;{>MJVmQF%rk! z>tHOtT=S6g={2AUAM=;Jo0#07F+09x?iE(GDRuI9ljm&PK*pbu)1=X(0n!ZKO{LEp z-5%Al#d_y{Fq=O#))aJXs5FYn7#be<*N_?1poUx&9YP1#Qp^1$7S~_~1jL7Ld+K~p zncK^Qw)>joGqi=l=(@;Wo{k!qXChC8O}xc_ z{tjR7qN*Xwht){2opNXjD_Q5{_D(U&1!%jE?l?bvPO7xHvpg8f?`Ji6P9V!AE+nJ8 zlDUol&0}(%WaVUreo4U94&?BI-;ehD3arRp@}sU`6pQLQyyl}gV$G>`*F*L_X+*%797Qm>@2wQtdt2>SySmoN-HxqQ_#tZNWSOr=3MRbonjBe+*+uQ7svH*GrxFwi2u}-yutsQH7DA+Vh(gD~JDMDw_E;-D^ z{oi+Vd5C9Y(+ZK+i^HH~wf1#O-7w#xoCA<{A4eERLfn-9#$9arY|#L=u+$5_-vyy! zF7Jf|6pcx<6BdIc;FnTm>!(fvE&%jV^w?V66Dum^#Unsmj9;Lx2r*E$J_s!lvV4@Z zrhCybLh5c4WXaFNPx4d|V&&LMK!%6vJTVMM{Hvn7cX!E1LA6e3UiZamqby!mHI@eN zd}N~aR(4s{JnCAzQ%~0?F2e&^@x!#^<`R$h7y{Clbq%!h9oy`0rN+qAmwv1DMVT%942=v%~sZSnp214m3 z*v6yV_u}*1)10E@LcfCf4=RZBHQlqB0NEv-&-jfh$EUSh|36~e*k7Ufm`_T@%OXYByIm!8+$)?&iGQk#D1ps z-V3<75pPDm;}gmV_aV_774`c)f@mN-@xD-kJTR2IwYZ|^UcO9yW%ZwPyIeWGdKTNd z!%Ift`_^WpSH<9A(<_^ zq61r2;}I!2Fd^hrL72=ugc|lS$QBo~Unq;@P|?ix-3K-6t0p@MJO82vdzEelz+*zr zq0?55b^?yoaH3cY+x%V3S!)N`@%^*bjH7T zbKooy(IvX~7S8$h%p$LPUfQsk{HQooTk9ps=j59&?*vO8(ox#X4V(7wzd|PXUOB}= zy*FYN$|7%5itcV~&*53@bG9sJVqp%++L!|D)wvMco&_bY^Y5ub3wcuh{V}hHQ~$)$ zGiUUlFZ;V*50wlL;7dN=h4X8gv)Dj^zNZBDv6r7izHUF9rk&k8CuaD!LD4 zf9iWP0m6#@g7#+>0sLkUIOeOK&OSQbxkg(PJ9;NkaQLg=7Y zIr}_nB+qb#Nc{vwEOOY1GP`z^)^$@_#dTg1lE3SYzlJ;Gbgr$C4~}k+REj;>kDz&u zce$tioTUGmjAm$`#zr~aD)vuXYYxXUa@%{T5W_~H^>Q0$EUu7k1u;8rpEsjQYFm`9 zfEvl~ec^96eqK;u8sPUc!C0$;ZJXs1Z!R{Hywu0PZM`BRZCqI@eBNpxzR9)PE@(ZbvpzF`0E=8&_Y_2R^$arZbeo(~+-i zGJ-h%;o-_kWVgmu&r&DFbO$O6=z2-yx7)dJbny@xM5qYTrTs@0)=4iZ+gEXVw9zy6 zos~8%;{9pLG?+{=tK^W?TH&-_6*;xAa#0TZT4n@x@Jr&s?YkFofE#=YTmGoX$|@dV zL*hZ;df!ZN0{eIeOG2kMF*S@Z4JL3o^eV{3d;SP#qO6^qS*pfx3&rtc9ez%RiwL87 zsaeW|MX*nc=b>_k)jOP81#b3FA8WRJc*mo(p_xfCMiq=hr)g28 zdu9~N(>4s2Lc<58^GQEpBc^}0f=cWoCkEAAtn+M_v)jYIH^Hb$X=MKQpKMzNpulm* zF@4^6W5T=BhJaqb?z=T64Z5=oYioMo4-C-Y!k_tP+ca^}a)(gVd#BWX>iNet4^hZ8 zlesa~=*8G9o=^?&8>|j4G35zU>sLzE!K3=gVL6DfJ~{Uy4O{t|*n~6-t`Qt>El zEX>uE!;-0gRJ4hSZ)+^7e~2uii}zA}GRw?3l00Eh+Z1JJIa$r)u=vAV88F~QtKaWi zgFP;9o+f%akMB#mmwa0=$G;Q{Vai0fc;SLr@JcrR5CNdAgQwm{lHW~3^qa=i;$lLO zbAn8|e?RS>F`vQ&ZEU@nUiEoB6s2W;&b-khG7uUH+SkMJEoYh39Ez~|Zk0nmjVlEZ zv-x2}^u4ZK0yazNWy|Ty?!0xvGI_}4DXYn2DFC02);@S%`k<>Uq-+z6C)KH>{?$8s zz4QRKTJ?_b4%rNu6)j_}F+k_`C?8?&H4Dp{jl-B;{{)ivz%Gx>2u_?oA8TGNy#BgT z!U9vZpso6`^`iPa>(jPznIf`jQkOI!vtBP03XQMucuKxgB-jMPQ>6&9GLuuNw$az0jOfvl=xV2g(Zk5Ux5~F;ziwY1yi*JvE6MdOB-ayw<@W!Cu=Nk5RterJ zKgLs6Im$2kS0HC=L$&wali+W0|M4mv$2-umjV|4{_%!Bxfm(v$OaTnd%6R#Ee$ix& zierU*B`J|A>tw9|Pk4V>E-uNIZ=+08Sh6nNdRk2r2b=XU78;I&`zjZeg!Nwww`%s_ z@+QsykA|HmD3%rWsh7Fo@WV($YO;s7m2c5;Al_JCtq5NPc!7BqgIojb zsXC^!wfq=GC-825&2J|3v0n_|W5jemFQdTVuYJeb<;GaF$JTV$Xc^`_-j@wDd)ir{ zSNlsv1aQ2r58uvc;H=*S3HdZYQCE={Q98b|idjPmcY+h#pc%bsdBe;-_522wxl@>K zV^uLVb%IfIe;&`d4e=NsiQ+d5s!M=qXVG3@O{ODc+<=HSSL8B=_*5C4WY;52-r?Wu zGtKXXdj8n>?f&%KxbXx!^gUa9nJC=xd@H0O3?c>qd-3=s#A%EXUS!r8m9Q zD{|I#J*p~|T!Yg$%>K52Vtbu>?0G@B^0a)G*z`v>{@t)G-eYx?TSNP^Ad!NWPf9NM zf3sP(MF5z37uU^NrV?H$zCe%o1&?%kDOuZcc70Z!Lh~udh@9~@RU*s*?IZO|xx%<& zV#m>`RyBA9O=Wi)4t3WkNf|8pS{sY%M-S|-s{d7HiU_mla#aRO-B_}9&<()BR`r{r zCPESEKuO=dvm>vEDg9orhgcD9kimDe@`;Z6`@cqii0Ubuia#^gEN>bw&aF7o`K{82 zDI0+8lZ+3>d1~&oADY1rj;l>9UXGW6Dk1_67X1)D0j;Lp@>wA+YNCM?u_2)JE-D)| zUL3Yn(9`~`3QGU{>}&#VPg6BvCB;a)L$iea2R$KhPNR&$(xq99torZ*&qg%9sQ?2k zE_oj591^Z(^D^zY^BJYy4yz}pG1SOZ$jUx$F1B6H?o_`L%s2yD#po) zEMWbuq?&z{DyF)A9Ms)FyGJK9ubHO|?9y&?RC;CT+}dbHG;&up{+xo8IaP}2moxd8 zySjdP{Ao`U?O4IFs~F-gAzWVmPQ$R1=pUcDl|S)T`K;6my;X^jI|l|C)d3zbcD%C6 z!{Uw}?|&u>5DiH75r_`&#arrFNeSwyhpmXe$XYFEcly666GmVO3E$Ne^i18wrr8(g zkfa<0md#RLV6__BS?HdNF`O4aq8Dzm06hIuS#Ms8by~jyuZ}hXDu^5do+8dAyAOy* zSWc*vsXqz{wprmW9TBZ5!S@nP1L{-Mb-n_clrp}rm08*xBJ`c>a{ z){ntKmK5XGVvwQ3C(~rsurKnXa5CB@lc^6KZlJs@U)K{K0b=Kk%9KO>6C}>!D6E^N%7k77Hp6sH8-GKnvAc=YOm_Cg?30+qAE_K^| zui#$ikcYgco1FZ9LwB-WHJ$h(Er?J{m%b*O*j*Qtwb~o$4jFQ#YY1{Qk8G3L{?2Xb zt+1MUKMibf+tu*CmZy)Q{lBu?GwN?o7XUdxoQDqvbfwW0%ecEM6@vzMieD z{sgqX|9y5J!&M-$(t|E}s(h0);kvYPLO-r`tYT>kqTUav&59g3w>wMyysCR+2}U{w zwF+ViFq;+pM(zK!CL?(-oEjiKL4qYl))4mz35=8WYUGNmeIfE~V`Sj)riRK8&I~5p zt2tk4gjD>f9W;2ryOKWCB(0*?lKL{zt z-SmlD8vGp>CZcU6y#4QuJn24SjdMLe$Da1@<} zg|Pq^>iUL^b{{j5Yg1sY)1lorJrFs0NAOvsFcGjB6S)8{TAAp_WTtCW zs=Du&Q}y4C7Y}MI{fXkIv0Ba?^CU$gT3n52;*6~i)%RR(ZmtjK5vfGO(Pe;iCe-+5 zMIhq&Li40W4XR*w%++COT|rM65nuoHit^i?;EnR*0qBahoKm4MsT|dYh%gmAM~D)* zPW2)|S+(=I<|86`c`Q1*pFXNh6pDrkWWzZ!VI4XOQ-V`>inbDVpj@NUq9wxfls94 zvqX7hSVd`%8tE*p;lEo?pm^L*+1CD#OjZf607g-+K(7BHq-r~EMd)IImrvEsKuvSp z&>OQ%``=8S0V7T7CFF@WdKYmG#C74yPw4WdfCSpf^Hf^UC=CP2e|Y~9n9%#b6?jaQ zPy-c)B%}=+h1fVNhSI&t(ioh_-KL9*3oh?8PhT`Ku$4%p?go~MeT0jPS76= zZP$S_)Fbft@YUTcu8|jwk+1BRgwl--3A!#n^58+Su92<2;KqC}+X{#3Ybw0hEVHXe z+dE)wU}h@MNwGOEw(%)WU&>2y&V~5%IDZ-~0QP5S zEy^#(Yg{A+#xs*=son9<-9rt>pSO?4iGHxwp%{KXqmn^3Brm)h-6kZ9KS)PG%v|np z)tr4?w^y6NXAy63*Pm*2r0#!p{S-Sj;Xb4d&iN%sX|iD#txlK`epI3`b+`zEjv?pw?X)jtV?UsOsv!z+ zK3*=v8~pGF7G8KkoG~yW*cJ;k(BJ07?hOaDv;7oO{R7{paIWG6 ztO3{lVdDpMa1DQvsryOeU4i5?@j*k$IlLdz^=mn6*}s=YjY6ej57( zN)Kbq%GxzIP;DgnT;MhX13|-ccp&eWnQEu>eA7?l-`bVG(yOUpxSMH8Ws!H+W@lf2 z`2_lu_tf8q#uXdKg#Nl=x?E+lNK35k_Vn2S0 z_kmcK4=?^0I{svG%7n=Y+-80)8c3{i1ccIj%Q;PrEkz4NI0%N`mMH~de79+k>KxZZ;dz%T{FY%n1~0- zAF{_2Qg#nTgg*6CQCa;Bul0PX)B3d5<1`AR{IJVx>9hOsX06iNr7V66mATmTY@c7( z%>+mUYE1S;WW(~W8mM#+Q8Xu217lU){se(8?r583+n7%+Px5QLw=KI&u?h%>RY*2< z)1(9>)9Cd6Be&mUg=%@{KE~y994g7DNl84Yp4T>8C&bstTlYkjjR=$wB!^d25S|@Uw{6rZxn&DQFjhFv}H+|@G=E5fC92orv3bQ_S4ql>fbnTp(+ z!mj4@w?we;G-$m7Qr7}pk0Q7KeE;uQYR_kXrP=ynw;|>feIrx*tpV|@?OC3T z<)qZ3vl&n;H!c z+0!2^h$fwGQa{hdu{BO1tc#kia7KFzpzNQhk=hB-|>-svheSHuBnuI?Oh~G zXJjG6tMqomSqd1-W1HY9mdK*ItJ*0-Bo7p{RTzVtf5-!#4RVmTQOqX!vhvzYBv0I} zCS&>Kuna2H(n>B+O8#u3i=1YFKl)+Z<3tsPrDz8|H5GQz5dQbNU&~pOb((TA3OkH# zk@;XnrCYVhtDmwj9Ms8Xt&+?ybt7bnW}>u2up~(N-O};T!C#Cs7QT4?w2y62K_qmr zJ%UMV2|BbFqm83hx+Y90ADknkn9CwaD~`$e!)Hj9v1Ss-_`VCp2+3sN1< zM#p_p_G0WQDnhyY*zh^{T#f6=t)vKlp~B@lOl1!IT?(@Bb#hzQ3^`lN&Ziu1-J2KP9AEOL{tp_K zp&CR8=bQo8gotS?tGkB%AUsT`J2>jTNx_y5JzS@w=+5}PM|>msSA#p7<9^x7mD`aG zDw|BNshHiEE+{U3cy6)f{@#L$$V&a!qMx1w?eJR7_z@hifm#uStg53ZdJa*=Wjt)w zvoOQLh67Fu(tW$BFFXcK$g~w}>cKt7KDvYow_VyQe#v&HKz&h)-#hG>62i%C|0zvO zL8qlqkxGA`QS?hp_oGz+z^L5G(}d}tbkRyEBhirttzK%|2|6Pr1}yQl0?nFF{HQ(F z+u<_QD@gp4#hWsvn1&VL;IQ^h-~>De5L<+~2=@-Ymc}~JkZJ9C){a#p$%PjYg->d> z;3rOI^EUV6iVoj>4lB(17W1!hx>4|3>euoHar+XKa7W2#U0T>7vIKV@x*03^75s(I zP6)Wwep{0E_t1r>q*4IA^k!@w)$!=Aa#zLXD}(yZls6wi_i ztuH;(8o9PxrK8Yo14Xw?z0QtKDu})^s(Ch#3x^78YJI z4)ynI*c18oMojw^mQof$Mq&PMcAcF4^Sv550VeU=M%@kt8KJT4!gkJNi-^BKYjV;W zbda5eLu@$g$@5>zygSkaR)JX z=&V>sCrRA+s@I#T5#f$e9_?qPO(83(*tWi^0KRx&n)2+j*$S3?R&Th4 zB(BAJAAglt=}uoVha?*F->hFObgrJ6LM)S;-Cr{rbKd>DDHhqv!=be&bVzj>9>km{ zW8z({hwZLk82h2@0qZ?^3f3;NvsQ(rQ_+a~l& z8oqWHS!I{2%}SS!#}w_t(cL*s7gg0nCeuR%10cU^$G)VSV#$s(3KQ#Zr2++03`VXt z5oo~l=24?W%$nUUY6dhJOW&(06L$k1Wpq6Jc*aJ*16E(wjXJz-?rG^UffJ*~?I}B? zU)Y*@r2DNc@~o^xV{#7JNb7|=mbG)`QtGZF`Aw8EKcVuV0J`0&;78r9^yMi2Ry@33 zwQ0mcji?yR{z+nPOK&KARLcH~n)iGt!ggP7HFYBlG<>KwEsg!tqa`Q%ys}*64|}xe;m7 z{lz=?DSXhxNHwt0KyH_GI?nD99=xO%J*?(Uvco9|5&JeyCLcQG zSBIz!L^*-)6Bq%sf9&qH9N3dQWFS9dw!bn}^1^HP&*a{K%9X}pQE^`k&^z_p^m z!&>xG{)C|ih^N_dmgpmTpYY-QTGTuGY}*rmUJ{R3A}xN=>56I!h-t)3**Bj!nAY_9 zeV~N8q08T=R`y$1mw~M)cIUOGSB2uFibfF2$C^TR^bY@oAJ?mQ5wP{%Yh8HXuo#WB6BMjMQu2GtwP~$}GsQ|T+*$K=+I|-m@P2bi*0JAnAP)~4 z90-fpIT~q|9<=&ckTx6b89rfjAjFk4*tMIh4^PW4$WMh4cbq6Drb$!{6PvRaO$$Yl z;9>9Lr7`W-9(z2y#@yJ0X_Waj0F8Aot{GkGnooaXJP2K+cvrP*F|J5e(gZU5OJHhO zlNN6k&DPVxsyRE|=V{TC?+!v`{;pSMn|+?&o!@e;=t9>bFDt^OJlejNCgfq%HY-D9 zTEyZkxNn4tN`#2}{Pos3edL|_30f1FI1*K$$EQ^PQ+bx=|6lpPc)wW=E0K`gtDAB2 z^pUTO8+J7#5g}(EGXur`8owd5KFMLw=QZaX%5Z6!6`6Nw>?k9;?kcO9u!(STL(g4^ zeysEA8rvYC+6sK;+BjZ@(rj)AJM+vOW0{O*V^O=>_?uIA1+|+o34a zxSu{y*0|Zf_ruh$BSy_NQwDNhJMY%Gc8G%XzP)8S-o|%nTojR4v3i)j-uA5p5ciKb zqFN(G()+Zx z)(qjh+Ii^)Jk73u?A{5092P%)7#9Va0}dY#astxdXP!v>hOq)k25UBlePfQao>wsE zk07Mb?|9pj>>8ThP_}N01ZG?Td-ni7%qS-FkzAriv%;B&PPgg4_f8KUOqBuwuMX;% zj(VLxc@%7yD3WJ7$TkKB&CfJ0U;B=Jnr=FpWbP@Sr%kO zd?!B}f|};L-=O~Zb>3t+<90@#`J_Zf()l6$NHP$e`TNw>|6bAY5?St%$%(3efA{KP z2U=fY^#B&FmmKCAD($N^v|37M#swMGX{4=mncr&A@A=vF#{pitKAwdf5ag!#{a}!jmxic0F7Cy6IQF; zV4nNPpfHPzw-q!na9>{!?Bo`5)p$&G5lm*FNQ_~cPyB*+9bL>}jd?;Z47H1tKK+-q zS}9|os-XYZTBsrPqogD$himd+FwG*V0agd6;awY-)kn+ZX{oMZ+(I-r|FyS7XZ_9T z%Omm4wUqli!eAjZAgJfSNmJ%NQ>f({UUxse2e|0O4J3HiQ_M(G3v>8^QXuX~Z#eDm z6v`1Hf@eNIds+2N?~E-s{|6K(nTIt+AgPuB6V(6fuOzU($FC*`B0$k&G&BX_yuk`5hP%j?@AxPoXB@csCn`viuGY;`^x--mWXApD!`(X6G(i~ z2Fp}N{qw==X^m6<6~0n;>!@=2tt~q*3cbz)Hq$oQ+sRUBTkEGTo)g~9Dq(eC%TP(> z9@y5HF-}WEYR*b=p0L#A#cTBbR0eAW#eLcsWyF0-A0ME ztdMiN23O%6{gu8FI7&f-_QP0}TxxMQj|gY3GPjV442B9>u76zcjW@X_`oCBC;e*ZP z=m=yR#N&Xnf7fBOmpJm(Y7!rVfwpQQEc_%EOaD@H7hB~Ga7N3=JM7mnVr$zPatzDH zcp{swC(K(-NB8qc8mtStTJ9~L)+c(z$?Pmub)VgZN%OWD;TrvH@-GBbY3VwI`5)E% zNjfNu7>vkdoPZLjt!pA4JpFS`@vyTY4+fBjxGbRC(vRSa5<jnF}>M_VYy=wFx(ih5hc7gRR*iOXoC5GTh*3szT(R` zGvgI~)Wb%|w0_gkG?o{{T=p6$`*35QI3jdC3rb~(< zzZUu~e6G7au@f5>8f~#Yn2R26KHsu4bw{se{8$9-MVTDqUr)PKxwqA#ke4mOl;~VL>(-`x?Z+~I8o(>QEnxu3^n42{-d24N!#35;&If;>q196o!ovze_k4p z1>JY*+h8U=Ui8ny+nt`7`R*EwtY*C&d9UUweGuXal7zRvX)~KBiE?bi7?jf5DHHV+ z0+#H^pJj*oH0>*q&aYHdyshC-R(kn4nYMDJSBKPw04PNln(|9QF%Fqd&yvBddM1&R zu7+x$n(QY+jg(zYPDb-J>hsB;=x<>+@(eERc1A7WZq(JfdC$pAgqVUHYEamXd!?~9 z+m5N6>Lxj;caMyyxJ78SK0sflX9~~z_44Vh@^qhG;Gf&T8*o&qD`J{L78$5>l$4vb+noG!F9ET# zu$L|%&HE&spwAnM25WJ~NdJygKcdZ}hLPCD!^VL!@WJ^uyNvVF-K9z6n{O{#flGfi z+|pFtu=aVAMpRND=jCxZ;f(*%*Euo|=^kg~`Hw3=k$-6%>zP$b^}^JiXd+LFNVBhp zJUe-;>zaN2(cuqPNZd>4-WS@*i18!HK?!N|b<(T}d2UwY#xuxm;k4B1a=0yhvZ4~N z+62i13D7i zdh#3dLgWDvzphP&z!NN_kkbVGhp8me%(_oE*gRO7=0nyDsl)S9-u=_VrwV_v1Je{}`*mL{^0utr@ZAkekBF@>?o*ZHU`?t#ERim*?Lh9p z2-L#2&nsibFG;slGWiuAC(d{Z5Iwp37UX%%K7+(qPYeSYTm=4iHnE0Jp`;VQ2a_4RmcEljCPM8rSk zz_iHy3EP6V5j&l`$u`6VqZ;WSO z4feE8onnUk(2l<|2byhJB`GWEE5#|vv$g%KSVYaY%#&LyCt}Pm%vZc4Oz&xIj z8E3n;zMuwVRlIpoZqG@j@X=|oY6Yjkv>xlY{dQRSv;TT2hg~0Xw#s;YDDL`0M=25R zyGF?S3Se4=MPunynUU{pI)-{eHbE~$*7T|3YkBh~McX6GAKU_|wPJIEGc}9cT?Ly0 z<4TCl1Ym6B-vHg3;?InV$zawez@j>_4C1~bGRE{%N_ zzO>8n14QUEi-iHsm$7~v#<+@oJ3a!yJyLH9ae-(2tAQVB=dZ`pMu^ZhVejRE5-%g~ zfZjl0s$^O5>MoAhrgz+jhy`V*8XW{1ES&IKDfE8sCR-Gu)}(#CxW#zai#bgLyxlEJ zv|rZ$x@;6ZT<>$T2b@Yy^EQlDwMsu^MsLqr*5Nm84~>B3&KzIu0W=q%Ys=EqQpI6%IV^IX&VCpJ`E{FJeM`AxCpx=QTeZmA;{a7n zJ)xBYffUgWx1v#WLk>es%3S;KmJ z_Q(A~OS@@-%vHYm?34F*J}2Vf14K_ooRlr*;J=W6wmn#EP=+Bj+9C{|s1Nl&78+-t zrfu9Fdr5ip32VO)>9s%FxL<>t83e;;nEy)pp7FLWTY4utztpWI*`UxN*N*{W8q}5p z2^`38xj$lWS>tw_^;rq*#xJ%r8e!uFZqrJGzQ%It^PRP=1!qTbgzW*@LbB$l^5KvC3#Njj*g* z>wa@Xw81JIT24b-6D|onro}~SvDr=8XNTvoTC@v%Q znW-D9(Fok27RG}j*(L8;M|ve)OO0#r_n(Gs}rB81Rt%!#b_hj+)(ZvSh6B z3A;=-WRQI1YCM0zZr-KUbG&9?SQ1Uc4%s=+&JJX0GjT~eoLFOd2kc59#&j)VdVLES z@xvXv648d4QR3FMzwQ$BLQCn$=T)MjJ>$Fpjrb`$ z&$jl=(aX_@Esj>q(3!q&_$?e$gvW~=iyMhu#T>eJHjSgsz^Zd|f3i3hl&BB=g=`&l zS8GU5mDgEt0&9k{v6Wa?yo)z?RA|mP?=H_EIziW{^|*_%nb)j~u2Q}t?-{1C z@M2UETaQVr6{YqeCMlDvfEhX& z_v>r3D@WDU%<$gxcx=q)72n~%r-@zuby#$RL>=R8-I6vpT?bktj(#2f`oDlc7*k@n73v*dM{yy9FbIA7My(|*#jbjJsI zC{=si`0)d_idA|zX6#dO- z+C;N%%oLaQ4gH06#9U=VBI0jQ4f9Ce!G6%~n`=dikMqea?S3=LSL~3)m6dRrGIVUW_0(VTHxVdOOvh_4 zt9KuD$~u{1&b}PD^tTEOqafp2q+`E};Qm3l0_mhAT{pwc?S`n7*7g=eVq-BAf)9l5 z5A*Zy6FCdAYg#>r^hEC_$sdkkBiXxPk|2uyhP8lqc|Ll)3qMK~2YVBKx-+jvC+*=yI;f?v(5`n$e17^blvh7E6hg&N!foc}*g} z_Ul7dGbGSf%XR{J@dCXEgGzak`c=+QEdec-e`(JIH1BG};=E2}7*48h=t)I@TmF4? z$5W~g_rdWEY_(#uR4cxieN+c3>(dCP#DH&1RW%>FNePx6Q+V&kHLq4Mp{wjBP~v;S z>$I5RAHPgi-u=0*(*E@M)ZRBQ7ZNC?RfV$K_geu}L4NT^G9toSJCxr;(tnbG(EBdB zxUzW z3@qfF@Ai)aL;|tbWr*?;N6`86VFPY8sO%Py)c33zc%S>B4E$TuVJ&$&B7*Y{8EZ3t38mx z(tugEyzM&@V9}iTbn{@Au;_I`r<&SkcqWryI{#(%9XpI|s`d#=#AB!~o_p2MP7Y73wb?R=?m)pv~ z6|5hW0+dp_xbao5(o!D>i#e&#_>!sGMrS4kULB@rK;+K2d&V+LDNR|AvCTqIsF=Yn z4B2*>O#~D7C*5A4}8}>`6Hdh(fomdpI(tC2+fK_PD_65bJ2>`+FI; zszn@9zPYY_*{2v#iJZ6-ahmbwhY2XOL{Y3cz%MRwvXp;G8+Kq)a($;b3J9K|8WPZ&G+{etNLw0BcYC??{8q~=ejk2-ywK)i{+SY4+*!|Zq?M@s~t`>_g9sQDIa6TejuB0WFj*p-Iih-oe}dvjfW>c7Xg3NZr19h zb%4Bw=JuD@9XxU8^x5v=?Lp;yOQUnM zw?F--Ex1U+a{1du_o1W(7fVSwY3=Ck3W zz-!`NyRr0`(S&;pI(CSbG28M^*FUN;!T*E9`*OoZE*wqMq1F;R1D#|0fZU!NQoO69 z1z;QFVABE0!&j(lpVOwwR?6$13cMAe%zssw=`1Gr_`*8gK%joUP(5klV+MttsLr1g z+MWGVF7hg}P0#n5J_=N~9quDDf-io#AUR4tS8MoTxqcG01l1jXLEK$c3I>QhlxddeunU=YCSn|q$!wr z>ES`8E8FeL!Pj1e&X99HBI1-kUHKku@}jNF)Hel$K&T6%BuP>b}8 zeZP3HU2!o!Azeh853(Sve;SxqA>(Xj`ou_WGP`$Na3&;Ce4|VE4UIeq>l?oE+-i@T zv((8ff&`lbUn3;fVGr;VG{+45E9vADuiZwfaTmY4!zb}*)Jeaz-Vpl_LXI7_!4p#hdor9@M*M<6h2eS%Pzft_s#J z43JIRUlmibAk^4Y{E*_!@-H)2?siKi_V*J>jV#CsSsANRJ`TSt>&)P$-B4H z?gf|6BK{u$k3ew0U9G7^GzyOm-@ocA`vy)LIyQM)XSY<+Z{XtrOy?Qd9tay-$T1qt z46c8_3Xz{Mx3fUZZ~J?b+;KGHZ7yp-Fpsh-&^;^MC7*gZOa_`l0!fpI?Dr zB75L|y9wsTwTo{0#SbrTXhu9we(vLHUjduaHhY{TUhmJ(x#&Ot>C5;e;xoVh9QafPve)zxVuupYw{tci8#v(xdTCWJG?RQ=HqBkA&q-Q;jok_X* zbjH{DP4&L=5w0b*?3QKe)aH)iH(Ybm?tAY-o{6uYzxAKr{m9>Z8ej@b%9{s0-~5l0 zPWa@>8eO=6hpH3JaC%3V1%eSS-{-j(XxzN2i5~3>{?DYIv}; zUVZ3augsR#%MSbLY3IM^=#SibbGnbu*w4KCDsp zw4d_=htae+y8R#{MyZypu82(Avbl&9hr|;{-Q(R*2#oYhT7MA9J_{v1F;dI|(}14> zhWj1KjPQ_Od2vep>NAXXgN@I;kkSJqt6f<!uRBKMWaQkiXq6bJ0r~^F3k83XY$));T@jSRW?@>ZF$d(c^ zc)dkVIfYcOV8{s=?Ed{Lstq(eX%7`X9*_I$&w9zzT&KsL{N>X;Tn+dA?f}{BfK&NU z(Jz1ukih2EOd&9ORtWG!-t+fwtVo`=1LN0mG(Rf}F!s5{_x>ZKXK_1=F5-#dykz(V z_u1;z?|ccI-1z>|Aw%j^%ZVPmIe+O}UZ7EY-Q9=Z^eOxqT6t)E_j9NI=^F}H3D$D@ z@jvum`alS7aW2@Z@6#N>%xh9h&&riZ{mE0les;Fbzw^0MU-ZW31J4!s>GysWZ=!Ij z2W!9oP77c2-j_XM{Ih1jpeNT|b>peWpN*HKw_Kk-cZLtiE7pykd2!fId+zx3=JROk z_K9CS^@FebcgMiK_-B{nb_NX5Vg_YrV4iaJiuE(w+yOqwlq>52U+ ziUcS86cG7qen&DRd2m!-&~o;bH+njr^m9Dqh2@eiRT?;r)+(yF}T)_ti7;`T?iP3xl-ZJ1#wuHB6~MbYPqNZvChy z9rUp|pHVpdqnY-B#$J2-+W&m_ao_mZw-Js<7;hbTpZui{9{2ff#c8q){jBB|HJo%Uv&|4%MG{qy$m?7{?+2EFTd_RfAQf< z&c9MTG@u+Q@B<|L&AW?#dHKu!{*U8dE+Z0IesuDA?|RjTSFITRZDYiG?YFI5jjI_x z8uJgIeLcT%gAG6VxHJFl??1g_S$l_tA+afK*C{^Mz(Dk>T`U%L$J>IeUw7An_OhCy{$T7EABfU7TAa^?v? zde_k(!7VDsLnnUklD9MHXSaX&-E$q5+3faN-}x!Jz3q1GpOI2mVR(yR^BNkj9;5{x zhR;ZrZlj;Jw=-&!M1O4>YNNg!YZ&@@?5`?jSCotz0{}6c;ISt++fEskhd0lh?VSQo z4LePFvsSsEXh%;rigiF99BFtuD4`PuUKF1C^&jA^j$SIk2{6_QIZyi1>8?EEajG#` zv(~&W50oTk@uf@>R;?LHx9K4XX#WiygOx>c9WPKH-T7<{)t53;XH?-o9VcMFa|_;)^g zDqfCA;erM8$+_PFdmjFhryl(DM=jW9YlPqrZM^@rAHVP?`QH)5&=nV4h1(g7!l?|{ zEM%*#=kYBQhxnH1)b7`A3B%BdjS_V}QK3^^?a699SqEBwvqxzAI;VivkCW~~K|l5I z=pMIylzy;f>s)>e5f4Moz4zY(e^8Jbcs%*IGmd!a)2K5cyD`*@*9SlNc6#F1gP+dj zDac(Lo$~m)!p9x2y6EbwF1-4Ri?3dH$DOfU_6DnX*a|rEL&h?9LzUW84T5{#}%Whh>eAy~|K=(cix7lOgUG{wF?!{LUjMl_; zWN9jW|E^rVa`7eCu4=y3Xsd1J?fKx{xJRe~N6iTS;?t5VuEl3Qu>fwmcImEr?YQUu zd*Ekc_|!3f&fu%d^(!^m+U(thQKJkW!;HTer4m&nLku!Vg^x!a^PZRBLk;KuDE-l2 z;g{XKeEF>_@3U?BgT$vC*Z#pKu|4_7$HOkJ5OE5ekaym>V8K?ee(y_fSbPgor&ctJ z6vySatU&5*x7!BxEjQo#-=F-h2hx7r(|_yecf1&r%j!KZRuR_xQLlXFvtM%PiJv?5 z+|z&Y=%*Y60Q`Emq?Ec*uImgtBQOl{^ZDhe&wIjI|NWzLPCCyqmz;OGP94u&_*=hN znTxNu4l8rTt*g_@yzc^h#C@-a?y>(t>AjObwDuTFg#h4lG5l#|qslpIG)4u4EYu4PB{$9`g1zU1S`%Z)(c>EsRlFOG|@yo^7;8@SQ z?771OAF{izXTUrP{lTMnH2@MnA^+vs7vXAv&*@}W18Ez3{zXjl0Oz917GLz^%P;=< zrMQ~yf6#*-{K)-w-E&8d`fSRh9>SB&FEKoVIRTorSF`)^K1v&Gq>4lEGnZ#{n2*x- zQMZ)>)$xHIY7KY3^1onae=y9>7jchz(t+D-w{`YaxTk#O2Y>Rq=K`<1x^BnM5&zEN zkKJNk@n@^;wVS(Avw!IYSH16;W3T+xV(&u|IesNnXMlXev;OgqUjN+Zz4CC?o|@d% zb=mnXAq&iSV$~pg>pmZW%K!(G#YjU zH3N@TJznfc6-MLGO5AKpSO0cSFS~FN{$vtX%P6d|sDwu!(z<&7UmQ-qQfME+ufR)x ze&uOjKRdgZ(Gspt8rYxn&br{;M<44S3-YA2#hJSO<4gYexVNC&$gaFzm%tR0T_w0% zdDM&p>2{F~w5pfP)RtoH3QVVaXcw;_V%Q`N6qfZmmVMV1#%pC-&%+*A2fDm^#NG!y z@bd1Xu<)aVlfQfhZf6LS7Ep77aJ;JHTOAe-u)oz&2n__n&>Gb!-=bW3!J_m&XYKmz zElN?OVx6<$S$x%niD&)VIryzGykFXV-(7dwJ-yXY2n|(%#f=v?OK)6(QvIzCV0dx! zi?c4oFGg;$U?fw(oQJx^R~V;@xKY zyfMIu2(}H;F_D-Wm3MIu`&Y)oQ^n ztbO`jV}AfeQ~=|r@kjmTkcGc5g!tkSzc2d9|No2M`qar; zEWwX`>LXtG=I41cSFX72+n+o2JD)oR>3}J&!-pKS-ygp2$oRwjq5yZ|CrI!7^ABN; zM2n9QfA-uD&h5Tc6kzlLzdQZe_kIN*IriBch1_TUi~sK*Jnavj)NL=%FtWijal!Xv z`bU)sFe$;;qdwu8kH$R(sU-~ek{)n1^dZ>*G@K+yrx$@o<3@9u% z+_hG z1;3Q!+VBg$(vgBH9954Z&&dWwrJnC;?@CtEb3E;}M5|OZ_-4ZR;;=Uk!VxVQ&Tr56 zwmjcx?uxjs+a?f|26HrF_6c8dC+M|2EvV$TBE=P^_$@m$|9#DGyl*s%-+uP}4|H|f zHB+wGT_yr=KK?fENX=C^*{%-Yy0P;fD4>GNO&Yp+w9QAXA=P|SW zB_YT$aP~QQknb-U0tx-#jZdnpBL_lP=afA@p7{B{ayK`E(w>`}GjFA<)4^96I-LAT zEPq+GdQ8&K$zI1n$llehSXR)_Fth>5H0Hmhkt5s-WE$Dsxa-^Xr=X zu6>OALS~GUT}g`OzK}04cN)?UNpIaeG^1!0%|--F*s@Mp6nYE0Z=v_Dh66#q(OZzr zekTs%A574Ru_3^zQ@7?uTeg4qr4OLJ=iKF}IpEu7KHw*d!H93n@)gSW*|i(Xz8VJs zXl`)TZ-KT}_Hi(L$=jjfn3jrsY@yjqGSa2UIG*-6Mk`iXdH9C!LAKg`%loc=40$Vk zZQR6vo)O0AY71>i=^yL7^x#`5EBV8hmYsTRXdF`)$Fy|FyX%|I82#SM1%o@h8;Gr1 z`RkZV9-8>nq!X_`v{i@ZYB;sBz?UpucH`-HWm_3tv%Y%O8iAZ}^&za~8H$qNY0m%? z`J?@^#mgAn`{+&2=nxR^67RCk+y6?|w%l>wz6eA$g2PkJQK#NH_xpKL&+Zw;%KUIQ zy?R;0CUt0i=D0n7|10>QSCZzljM^bzw+{cu6?-4Qt42y^Va-VhHrj{(=l!_wx|46U zClo?s{&<6hz=xUN&bs>e+rR#Jy61Znyh^%p*j?|v^x^SW9m;9ti?E^}er0m@SHKHH ztL*|&SYVN~dk#56U4n4#jUwl!YUj0?F3vnRd7i);(|m*T`V((U*GgHRl@nYt6sbm( zntqidbxd;zJ@J|MAUqB)=a0DX08A+Nq~&Sj5Cnke(*=3;OW<@DZ zs6-U$q{Ll4B-yxCeRD$uzw+n*BcGWP%kC{R_0n73T5BR-^o$5%iDT>;X*_k}=QqE4 zS)-=m(|Y^aGS49_VsmHYQ`(#E$Wp zbFI2Htm4RLt_DAizT#oK8X(|3XG)ZF_q*cgi~qUfko0O`H^DOc!1Yhucg^GGm?u7Z z^Rtt7IM+^ma}t8hw=7zC=<^|Pa?yiE-Z&5uElyTOd=r*{_EuDsKqf0Df3kdWqS7cM zscjTDUpsv8=dZaW86A2WPB=_XS0=Hx#~CK``nb0?>Cr)ocGgSGdt0w*T^?J{pR?2y z&2+F3LY{7-AG_lPJY}{L;J`8xM23eV%CF} z2Mot<6X5XCCtFlq;h&K5HWK2Q?6YHDW!(+cRN08kwI|=s9eyNi2%iU$j#4M zDQI$RqaGbkzV=Abl6>@*=j~aBj8e)_tnNE(%J&gW^YPJyFTR}ewLyt^0$J55~ z*IU7=h`m3QSDs^H5GwtjADyAe1Et z?#le*4AjSun|R=`Fn%w>dFi2iSBI)aU7ey{$sdLa(?yx|O8Pw&x@^c>JfYKm&eM}l zyy{Sf$46-LWYMCf;HbYOT|s4WSZo{|hRy~n36>C+Em?sI@@x=vb;eu~x;j}Ll#-j9 zEA8eca*4Y-KFMRTm|!B;)({+m#@_Dg=;v_As1V=Jfs@@(kM1TV6HfB1bi$!XRhvCy z!6?_&k$C9pB(oWDh}ewjv#&btR^1mycnS++-fkK;deTeW7amflZY`6E^2+>?X%mC+ z5%7G`&Du7%SXn7o)d-&=A#*NR1z}dKT2Wexu;{vtVn5{rr6v~P1KzZ--y$gWTSOzX z_GhBu|0OLM`aiK(#Vrn7?V3K$Fk9mvf9HVH_S6>##{B-P8BuO*HL$0B8e(Y3tG+fU z6b>)R&)Vx!KTVXuLP*z|11qQQHsZ#YFKN=Ufp1?x&fe=%2JD!mfp`RHwg9s0h?`!y zlwsaco|qN{BU=5Trvc%RA(XcdAKQKVP3hQM)=BKM55B@5uTqp=xndRbr8jKS4fSWn zylh<>6V6sgMUUS6>{h#U-=K3VlN4d{5SD~IcfRu2o1>3fkw4(HJv5>U>jZQ9r$3xA zcqDfZs+|Wkj_|oajU0O2)lUxh))fJ@ujh*vg1Kh`l9#TSqD!?)7C~A{hV?40r$ZBi zs8u`t$d)3&EiBAeCt=&`OA{|+MWqPy-%NYST6KbHF0ixR+R_r(>A-$Gl`~<(G0uNR z{3#9s_KUz{#PQ?OXF6tX<1NGZ6p!=FJ!xhVOg2g_A56>mWM=POx^2)Y%}i@eJ0mO; zP|4z~{CVXUAAZ%KX?H_5d2!S_jJY7C_4$CuQA$hlo6o) zh%&AQ$biGu;I`rS(&6!s`4sKDujDyMz8@t_@3x)zFs(GXBMV0sLls4POh_Q&|JB1N z&XIcArIBzMhsrCX-)G_|QAi`b-R2Od{(Kk4~zJ@ejC7adrk zQUzdz%=e`4JUhwDK=&KCrK%tC{h|4}XyMXvxhLRWfcHa3T(Ezao*j8q$5(`(zWc=; z7d^nYhh7GndtF8Vb?uoSBNhQ#h5&fvh<|kH*{NZ(I$T~pd1o4cHUw2-xXpYPAHDO1 zp4#AKh+djv1(r>|UgJ934cVICw@EKe<%crc?(e-gWxo>!f&!Y_<^9(``hHs45is{a z=dIoWXYAE+i?$UiSDx|Jj2Fhf&K!~A1$=BnIo?_OIE>giVuAzas< zRPep9yqfvlEVUZ8gjP_c!h;2K7hikQt%baB)UsXEbM8F8Qiby7Ig*dxlAf)`q%LRQ zenQ91!%#bzf1LWq{iB|E;JPO~AnY-hK77I5r@*v)fVP@?VbOaXHGnUh=FM1WhTyAq z65w&e9T(nr-*r#ydDKn=58JU~<%&9YtiF34wF}>*@JMLx%td^K;c-A<6#T}$!vi~E zuHUJZjb~9_N#0wmTC*B?)wxH922JbC`F=imb?3+jqr3u;yqfaPH2*X(xRg7w&5a1Z zVe|UDQqo5yeDMcP67A<8Y{Qvab*mHY4qLQKLzhJR{%Z>!2ZKzs&mR+_jTrxmXSU&% zN%;0EQ}AyUmgDs`8?M_%ZP`kpO1^9vuPa!|do9nGyh?@d5yY3g_@>Ki0$ax$rpNIJ zlJ^K6$Mdsv)*p0f8`eqYpBgE(OKUqHxT&H2LkbkGQ*AdpK_ItzZ=qqcr>hh3$3ONq zqtk?0uP@Wf;B@hOlbIs;@o_Pk)IdONuU87(Q#vZl(yK(sBQKj?XMZvnHFbg$W}W zt~`khOLkp#%*~n_3JsHCKs#b;NA3$x+UfjnKKafZkXXP;HWU?~IBhG0g9icf;TmHWcOmOAqItS4z0w?mY}g z90M0&nK}BR2Y8l|jk6i`1V$NRaQEd8MeD%db>aPbmSI-B7EHs34+w920EzP?m(N}o zI>JQV1Q@CG^yruDP%%ONj`Q#HO!!4NWLEtgKAmM-wm5vy%cesa6&>x7;kC)_7u<)8 zFY<>{S6h$1{1LwTNwCu$T1@1=+t4u8k+PKd+F7}AKmqX0 z%zZ!cOyit6f4!g3E0(Wh+0l8k7tf!S{F}3c+Zz32pl{qRi84m{>)TP7mc69J^pC%X ziK{ANUv9m18@Fl@IS&-Q>p4H9gKSlbSo^}M`2N!Q``sr$m{9=n6VpvwH-@RjQy|PS z9N209{@@1_x$94G!^JH$O>Q&f0Ft?WO3Qm&Qa5)6X8Jb<)t|G^Wqd}CtA^jCb;F(m zx&=UOiLIX&Etm5tGx=jGBCK8b;k)C4`=e4cbVg7iK_M?RlQi&OmWQvwsg7+Nrua!$ z3=o#(CIiK1k0i_iL;DXHy1i-r`=a~xECT^;6xFF;gExMR#56Gx%4a~Rd$hqUwRuTo zxHLU;_sb#p?)R6ETc5r&e~CZ^Nh#u$NAQQK$F8lt57L5TwiF-97&su-E-4^^O*U`W z=tQ1Z1o3Nj6NrfLC@9&4>5)BOT9o3g{NV&QD45~jAK{i&d}uo_y#Iy+l&M3=miC^n zBK@p0V!zj}VV*2D$w2Tp?41`T2NOAeCOkEX0SFYCU4-dfCn|Ye=K#K@{)ggq*SY_y z$C&U^CKM3%JR`XBv9q_`vp3H&kZIPs(GL4=J@Wq3Y#B>)35;>KJd2F!-got*21uzS z>o=);)#K;%->WwZm*Og??e;?o0E08R^Fp0-acnOf+pE5O|9H!5m+?^*Us+!9_&Ln% zV*(=ZYSFyKc70ado*DfLrG_k`QH%PQKXx92m0GoH-nd1*&H8P0*6qh#_Q+YRo69rw z&OJjPTR2P~Lutp&H$3{Hf8IIq%7cdO6RpR|s|)5uvbSsaAZKPzkH3sf{*8S=CLNrN_Y-!Q^QWy*w4 z*I=yh`Y8~@hcFI{j2v~zP!1J;2rHEX;`7++o+}$wu~m)< zUn*%{?iXu8tr|*1r@x)lV(Z3B|<25sMC%3h$tl{J~IE6 zNE#0HEoISi&kOV!)I;aE#h<5^U3+yh8D9Rs@#q9Q1cWypZ928EH#QWRu8E2GV8Unb zzxHv&B(DY>)W1#VmMfR7{Cx6P&yO7+4P<@m=}EgEynXx4+o~>QfUMu8)vD#IK70R5 zo(V=EOt?MmgReK-Wc|px4sqBg0B@7pZ??hAuV?F%H?Q*Y18@3s9G6~tWCF`WNX9_w zH?7xi(3VZxG+8ib(c8~Wwr@(S)vhYZYPsnuQE9hv8%7GW-(vj+EgJCRh|&J{Uj8Ug zk<2bKUw`zi9R?!zGeVJ%_Pei4dE&O`3nY`z*^W5>K$v(-&x_o6fBck>-um1I-GIQY z_GvdA+ckN&!^@vr&cBB#Z#~Y~D<0{!!)EjoJhK$-T|$Oy;~`s~0#khKTA+A|I(pJ| zc(#@sRnU3#%ZgQx{44(sTybdGwrRQY>meX9_JV!miHRcQ0CcX?^5UT@)B) z<4p@9F&(B8nm=oyfnpBF*qHE0OgYA5g^wnFwjM8;*wmE4)BLI0utmLg$&B)YltQ&6uw@|K+uCdL`je~+G=5ew%o=8X9E)9+POy>^kW!A%RI?39+|d*k_% zYnsrA z`PDFp(w2Lth`|LHj5$?d2{lSiJ_rjGGW0c=;f@!a1_d!U_dMD&tvX_I2yy@QPki!j z!PNjJtH$yfPBeKZ{7*C0;k#5G^axOUe8bU!hirG{@#!Zgia8H+7tCJFG<;sT3>us* zw(fHF9Val>j2{t`j9O591~puE#0~qMJcz!%>SgkON+?zQnezUtXTC`0^Bih(K#6EdDP9BEX_uPK72#v$t583YX$wAnH z!}ssJ4&h;0R@gZB=4)+vr4~f2l>dexBEpGE*Yc35<>UJrs|*loRNu6cmyGzLvmCGG zOXZm_^}c%iEI-oa%2%#j!QGw=A?AHEoKi+jK!kyCn2k?ha?wxX#yQ7Edj?L@eF|qG zjKUvjO8vU=YJW)veV0Hww7qq8!~uox*}87k*{pDSzwzi>9-(#n=6vtO2NW=a)m-~h zDf1J5K74E1-=S+9 z==#m-F`AK?O|74!(&}}>l=ukP-6V$$9`pU2>C=9^=dwpXeJ}KL%pdM14L_WOxqCAN zjE_8<-4{w%OlIyl;(`P9EF9f;$4{C1R{m}k1fJBMe$z3VB`XW+zL17Mmm)TZdoO>u z=k}YHs}N=|(t6ti~>x6?Bud zXW=X8aOm=FldZedZxlvnu!UB}Xl}%i`9pi-3xR^jSDqB0BzEV{2sPCQ&jt&zL6`MIy%HbNR!+VW zn(*ZNGHOsfWH&`)mLdd5eK~*U{;=@6Q*QUOvdd3QON=iQg{;7`=hY>1Me7g~;5>Z8 zQ%~ISLfH;naPR4x_vxH39WuwEHz7X-$dWymKY9+U=g1!-ZP#jL zW{1t&k(nfq1lVJ-^X40#H~PdQdY?fT1&+X#nRV;c5&~z$-IqT?DVurKwsWiDV@|K1 z``nK+f~ibyJ^$Y2OMY&?elwl630^qS%un)x5k0JfU{JD%SEt>06z#0O5T$pGd-kbqsc4m&3`BIlYF3I8$g@%8EqdfgE4n^dsqxkB!lcyr%5b%v#}6URp(y`z zM@9a2+G0aSN%-f{82a#yPk553?}JzbO|xRbC-{QsyGJh}n^(bTE-@s7pE_Td4uFMM z&9=hABCpIOsw!BKh(3e18zIM7oRU&OK_FATR+WB(dqPL? z!OyJ+_qglg2Y6eJ|59N;Z^i-ux!@!JdvY&f{)=GGkL<8pkE@t_mLWrYZon&XV#1*| z&=`Kr6AdB>2QcpBZSaV})@@X0%bhl}7Eyd8&$tFX%K*yq)>_tDQyg;--3HC->2XF- zOyd93lKiis+|%CN(BVgKe9nj1rX?mfY-ikh{Q4bR8Xtlp+G5TR^9(`E&OJ8t{s@76 z)3)Ji$tE$;e4x>U`6+VujcarM@~q^lCx%~h;%(n2a}gREZYU4EKYVk+km__%o+J)d zYVz_V*{2az`h|$oYJY?&nMqVrgo#leI#^y|IY{R2OLK`D@+v?i8v}KDm{ndgm5ML? z+}UCNoOIoh%p0nEEa}QL(@qoZ6fVhaXg}wg6+u|EjS0y79~Xqx)^dKTD0^N=z<$wu z#Zggys+2>El5ac9OtWr;+Dn+)dyd4!Nq-eg^M8);)$`7)dY)r+8AT@Zr-a&1?I9W! z2;V{ip%o!$EiVQqD=B&0Q>9Q;S4Vv9Cq6gn$cqoGR4LrtBk=y~AJf&b8lkxLp1r)h zkB>R9=J(&|qNvD(eLn{#U6jqY>1zEP6MpFWr!E+MinM%JC&CF=oMZ!Mvwjb(f`gbta>JPQ8Y;b?LW$&d*DK z=4rIVdhfa={T%Kl2Oqh!^>b82KZm=?Bexo6ABEzW-Ay7BwRqnDb^b{13vaiI2r%mE zkOQMHeo#fWKl8TZx^5Z93bN(y3uo!MshGMi)M^6OrYeg35!3I9Kb}kY~zwF>0_wOfW<;oS0y7-`@`kZcEMd2)*x6~e;u=bGe z*-G)4D-PxyH)MS1JFsVu9XFphYku=KO~vOe&FFWo({;cksiSg_RdqJrsuLf_djDsC zH@8mxF#k(7(una^?ds!h;Tsl1gJ8+k^X@*0M>C$eaL!Uz`SF}i-P-dk1CDR6-4EH8 z?w`+mCOVCl=ajsB_O!sg^Pb~PL$Ap_b+TMF0TAQ>Q-Lrg0sN&4md*Hb&h*c}V-h73 z6Tw^bAtw*C7+UG%H$PGL+c<4EYQvJiIxKo#m?VUmUNZlu8DGu$?(^@nrg{C2X=yD< zKGuLX=Nq+dXn>f5p64DkWTPZu7F6PKhQ*|)pHxcJ0Wx!=Bs0ALnaSe}af+T1$6k4eJl?B6KNTUteiCJsIZ_+a(Sv+qv{zq9CO)kH@)-xWD|o3?UFsKy~O_++VUawA&HEc z@}&bU_O}wm0)H9+VfpNpE%WVijM8~A4#VzRH(I6uFbF`1d()>LMU zlJOyg$-5#3clh65+4OZ6NB(;RDjTjC3zl$UU%K$8rHhv{aRWcN(>^F1jhN^F7$JKo z(R!8<@sGRgV6LSWSSw|TbU%ud>LBy-LlfryumBk!G59|%%y(@JVeWTx*6H2Rw~G~( zK`==c;cOVt{TVVC$r@(iiut3C+f4?mVVHXAxmEMg3jGar{(6A^2)5J*yoiv z)?Ui1X&-zAqQDVQUNvnIrohOyqY#kqJFo|LUplls$j~&jkG4u*dhksi37Mn_aENxq z^{e1jvn%y=wK-s zp?R&WWY%IKc9A0wh`JCWrm2bV?MmJR){>xwi*6px(+Fdf4aJ7d>UZ3{z1`!&Z80BBaih=<4GOx;jA7)fqnKWdGB< zaNZI>zz-SS7kW$pcRQH-!u)G4O1Ureiz9k@dG0yBM(~-)W6V|IxliaVd^zXa&!(%l z;1wAW!{Ra_Xlk2+6w71NFJgjHwTP^!j8`t*`ghe0r8tzDfC$XtiMmXS=Pmv5OZqKE zx=eVo|A~We!)J?iO2f~s^UbPYd~0{yu+n@5)`Bv(BUz6I9a#+K$R#&!E7ZU6R5i}l z1G`1-JO0rLdYrL%$+Gufp5kpWzMyfl@P#l;&kq(0IWwmh+zBDjyls=NJ=>cb=_$9) zI@|5Jm4BQ83^Tp!L17l8)rQSB?Hz^{L@go^1IHX746LY-dzA_&+V(kdHx?xo6Gi;t zTyEXI1*4*18q}h>i|D|FWL$S$oAgnoaOf)gr;x^HcuR}*n?)9VK14B*fIXB z@8&{Vjg0SQ%4S-rCwQN!Lmv%aag{!D<1-Ml<$W;v@C*LIch?>nnpnO5#5)nQ-yS`q zY|+gPS>z?HUNUV(;#(F!^>Huit*-Fv=s^I3pOPsQy+G zc`yp%KlZ9PrTIYS{7gzSg_a=)H`ZO3d^UfqsLA^2z$kGLP(G&u94ERQDn48dc$NXv zINZv&W~ixKv7VxZa~9g;jQKMcrD(Z7ER(_2DKu6R9SB~VpKBbKq<9S9S4{zD8!(3zT9qGFE6p$f|h0(RgchHx=)Lr%k|!BD%Rq64bnvr*)_B;e+Q(k+qNzi1^H8$IO?8T$C`CrHk{uZkX2Y1f=K{HJD2fdEB8YWYInibVn4M#u4_z+T5w54vaIy6kVDb1dEIGJr8#QF~E#^VfM z`s$+-^f)7Qb#jZ62%OHa{nDGcERpS^@HPG>z0wf@UY^;Nd_RXG-$lvdv;e1fM~@&H z96ioZTfkg6XHh{{C+VK~2vPB`>)qPsdpp1ZoH%nIRt zCcTv`ZFiGa^mA^;Y-@2V55x)94AX|L6FD#lxksJOH zT?@YNEf}U3KB8Dh#S4hOe%M`nvu&_qvart47ac(A>X$XELZ#h%EOANY*`B zY0{!dQSC;x{wTcFShXVUozi4;z75~^T6#ct?AA7mUj~O8>#slh?#n6D7cW}6bpB5~ zO8aTi&%gh+X8!Dgu2JG)e&lBsFnX4O{F%2O*UXIzKt(w=XL@>9VQxQs!_&{+{fdo~ z61XM|_J$Z5V_6De>EvZf(OtI=-%Y6Rn!Cph+1R9Y1MA_Jt~!Hi%gB&(kiAhN0(SjUCXl7A_-&??= zZ1*xo^Ru#k=QiQ+k3W_yTDpjlgUJV6CLrRqN8jqc&BiQS7V$YGGr#<3`d1$&nYrMn zCG&nNnwgQJ&In@?x-!lPUn5h2Q1GLahR^(hQA)!kpCxYrC!e1g{jvvy&aU+{=ezJ; zl#SLA;rB(a26Y|+;_#Vb}r1H5ntlA=)aMCQ%#lm4A`y}9x|pw)e7;_Ba4hL zjk3rWttCw!OR{LzttH4`74vof`?_m`rWWC`GT(XWlPk*&?hu%+f+-LD>Z*8!K;OdO zGE*@ka(?#r;X+EOnpmav8`p)v$?P2h%!eVZyD?rh<7oZ(Rga9Mv>Q$>E z&RQOpuUs*b^i`v1S>fvDDy`R`c2wW8+lZmf7acn4qY0mxE#df>SP?`izMZut1#&1i zo_5#OPYusb{50p=@MItLvK>IClt8%(6>>%j{we|mtlaOfF~jLxzFfaSJ^A%qy$=e~?!`6BKPji@kKm)*|Aeq9#_fb#^=P zK;|QpD=-cOyuZ}{jq*7M`PcrY%jftCd)k9#Ggy+6QK>kk_#Vy6tPS7GltGt*MTH_? z1?cEdGDX29EO66`S_?iL|7^_Df;A3T-mI&h1Fb{r>ToF2AHMl1LwUI+ee>~xt_}je zt7Cpf^+Ie;?xmkhAladNL6^XETW{FBdhM!O6dgaD2q9za#|VBf!9| zweRW(P&YRgF__SG8r5FMb#=gB{Y(0}{@m%atvW);-qj_)c-*6Ci&T7P z&pY7hRllsvg-&km;wBvF8XuyJB8fF@Uhkk`d-Iyh zYeCBjliO)(Ml0>oyQ4k@GJmL)`W`+ZLb%+>6jK+WQcVBzjJV=#LcA=W-q$=nV0T+U76|YB0!G-heI~`!;}POR0L;)5Ri$A{^}tq-ML$s z5rZvcb8;^5iZxJ$siIX~VP>6AWW#Rds<5aEgpi857cp&d3ge?|`7?i3cm$~;Avb6m z4;KmoWE<1Vx(#7%WdsNTN|n{#gr^1RIi9A}TLsX$5T9tlT<-jp7!OmWB7PaVRwQrj z@GlW%>*53XW`J>Bk~10^-n99bC6a&jvA5>s#x@ zqXd(s03}1InO?PWO-PA#!yLmZ&8nMlRK#7PRLlnmeMxOlUe}mLSe$)4tyF{%@OkyB z)lm_LITx}efdf99owL8L`gO$6Yd(4Vi@+`1e8h|{!})L;xg0hXk18f}!#{e*^ZOh% zz>38GCHI6ki{m+_Wdx-{=$yR0FyVsyFzJK{D!KC)hF!fCpYHuP?Y7M(ECWU-|d!FgDHzy>(Tmbf-}6qFwZXxWux1%PT8du`-`L zkw4xy$%k+`(_$*xzILW)k;$uPa*rbAVJeg8cr@{~$9mQQ6Ht+?YwtOt|K~?%Sa7Y{ zMo?5nbWH3&BXlKFe^_+Xx_=6b7V6S1LztVeEnTYCW_zNh)mSKAp{c_5iD@9v`%MhSQ^+LFm*ngLAEoiV7 z;XVUfE#aw}nf_Y}R*U7B(;Ha5j?jXXJ+)sI+>`DrOo&*3&%6-S(M4 zSZzXlntxQ&jek|Vj*sMAxz)C8r|0Ucxlt{?ji(Mgdj+`d!})^R-KKTJ zr<1q-)IBL$dSV7bJ()w>xEyysvsBXSeJVd1S+8Jf!>q1fd4t_Ie&`1rv-+2&8gA`| zaxBhb$#M&gyE5V6$D=s5W9d~iiH+CfGpiIXI9N*jPu3=_`NIZub3y4K%?R`K4_s2< z$*h-*{vxioYnOYl-kaog`vp#A6Vj_C_7I-Kd<&}!flc=Bf1d6`y$5XRw0QBJevs-i zS78EOX?3ozskOPozP?nza;EjkHD5@YX!7nw9gSVJq=;iWZn$i}6{&J#r;9FBn0T!| z@DA6BzI=$B&meo76xr#%%=f2d!cs&#v)NJ3`v)mKiUnthY&!7;x73$41U=!9VTF|! z-Xx|DD);mG8|z-5K!b^eA~#H^2QF+y+j*%8wD+#4t;R~aU}C`^ zRkrmHVpzn0TIUuPv8qVA=ByUG8FNeG!SB77mYy1c$Az?c0URze+Fz^p6SOH11t@twE;+gd*&9hkwc13Bc7E5ZG$>FaI;p0h zKI`2tY}eP(S8I+;8|IMn;fcXkv@x*ytd}SZsU$GaQ%vuQ6;I|i-hga#^2pqi^rqG_ zFMF)my*aZC93a$KdlwTB-<$u#QidcRi=sM1$8N9y57C9~yDLzt{LP;P-viVf3(-~L z_*9|Hn44?-8+MjyHFVGP3A}!3LX*}b>THo=O5^CMHuzU`-7n>^r5_^K?E5Y{(UL+U zbZ-ul1h6pD91$%w`XpNMSts8xGFg0XLDRp_Qtqc>P37^jxK4oQ0w#Av(M^p3gA)yN zpSh};`<_DMhM-yjNBH4kCO|d_WkMr9QVvdu>kY)H?paeeb^Rx@2+>ps9%yfRNB!R> zNh)i8xG0aOg7UlKev(SuX1n(HbA*z@$<+plU0+m+`{NlGGW-fy6u7R@W6s?^uKhKT z;^)R8{-P>JzLfIBUz`9u zn?1C9JXz52=dfx1axz4SWR8H<(273lS^Owww@|$FebW2RCa7ahZLVepUZJYI_ET=c zBd5hBK1m!bRo^-#Fgp=D1*A^-jvOe+Od1F;(mKHszveMEji#Oa`5EdSG3hMnA#9rU zUsc;iz^L2^1%@Bh?hnXL!XAzaex^!XM{FHF6^Qhq8D@jea~Y7&95YaILTdSIyuy%l z)ZV!|)YafGMZY`6=J%A`GLY<_0%)a9o!T4Q17uDr7KR{7mV7laV}KKuh{uJ{=2$Yp zI#iat5O894Ox!9OuS$r)ry8TO{CByWrY&rYYhaa(DJ37{%~ABJ`?Ar^ zChY9mSGNhkgL9c;Aj z;KNg&mVT+@&R3hL!=rOz?__?#KdT`Z>#flAetsver7_1z(@F{*aEnjL7!i4)z|R6s zukHMF{dQBcbjj#u4m*u=xcXXdmo>+g{)dHjYQOm}iZGwo6|kLwiKTqqqO1gV;^&!s4ht_C;Xq$T1u*!>*=Zfg3+`;tuZEH9qy__FYS-?R zBX8T~Zy~Hgy}T52x0kcix=nAx4j(TtszS{$L7i_S`9`U&>pAHNoplgyjBo@U85zgJ zeW%%?#bB4`erXLAR4@o|HuKL%@<&a>pUg=S~Ogr$@vW=MF}MNZ007CvU*M+9pn7 z%=ZeahVki>X0lGomO@8Vh#DqPmMawy{RET@isGkg@;-1255r`=fX(aThPm)6bN_nN zT2KDTJ#%AP*fB}AQdCXlz-y;D8ubW|&C$YQCzxT;46F02&*&Cu7XO)^7FJd3Z}_oi~5j&cRoZVimXYo|&IsYyI+k9t-wyWCr?d$nIarzachifzj}N7O_tYBVHh<4ta%FZZ9%t zL-!}4OC(S}6Oa$A@VhhmlNmTmpP2+88mLN&DDh6&8WGzsRMtw(BDHQcUne(;%Upg8 zpHl>io~o@SGTdBwfJ{_=sBWkrmG{N?X%Cz-t%x&=LJBrMAje1%vuVIB6UEG?^yHx1`b%r_lwGx|K!^nla%%S|uu0525?;RE5u1 zg_rDH*15f$F40c_VqB?}Z%4eXt_j&pZ%a)}{7RfxhqMhcHrYJl**m$1$smy4i7Rrt z34t(q^aWw$Pc`BU1S9jzwOzH=yUZ_(svpD}HuY=&yv8{Bv9YbnJZldx#9B`ZtyZ4@6o$Lof5 zF#m*_sVvzHcQvl|N%`cm@=$r~58X&QFLftS5;{*L(;dTZSn6IRpz6NV%=#VGWbCn~ zB8zP0ORT8xY~N)tR1{)AOQ#{qiIDsRh4&kNhbC0QT)EDxE?O-{foh2;zIznUnc7jb zWdYaW&FefP1)w0~pEdWh-p5h0fX+-6Q46bq>_W^#hrWd9#t6(K&IFa1HTRIntRbMs zz;_-Y8B83r6MoAhYbUCuYW=1UCDC8&Xm1Y(%ylsp5oMzi=U!T@W9gNLEx^u+mAK0Fw)J91f+8)WN=zG*5tOO?1uqR&cz#Z>9SOj z&rXszn4deeP46RLeHRJwOa(OcLeR1q2yE5fM5i71U)Wj=8}`Hew);78yx<59X&abC zFuv2-#SqHxu&rmCo|O(gPxA0%*OoV1m4)paO`YU&cS*04<%XvX!qqs${%@uEXZE#BaqXEADCHhHV^O4nAm-mXE$%QD`ArKC=8u`^? zG?j6_lu4*#seZaj$%mFV$rTC!ofH8_tD;F810NT<`sGDq;e8=u?75>jFV^zYD3zFo z3jK$mj9+xDd9r#+inmNmR1+JA#v(vdVT$S{{JutnTdBd?c6zFCL$&4WlEQAlM*6TA zkDtzSE+QH9o_dBldOLu)5s~BxDK`H_GEHIkmCc62hBP8}$*d?-TNDL89(`R`E2IUN ze!~C2B5Jo#X%bS4e`}N8cMbqO5NmJM+h31W*XG_4!-AcJu;r-!)aOstWOnv2wcDEB zjV$vk{8S)+Mcsmfo?EI+{_2UUrS?qyXMY^YOTw*ttxZfl6suEz3EtbbKAVtqKf-$^ zbVBQQs*DGo+JW$UW>V-Gk{C&U%P5 z?N?6mTw!4@0V2;Y?cZSBNPu}e_Kv#hW4%4irNue>TSDKG!1#r?!GTk8DrlkfNX zi4cD3*zea;0X^{_8JR?lp&V9Eyu z@;?%H+H*{C=G`qe?;k;*KaLtdFdwQLhuxw%?oPm#Wbr#rC$J}VYS-qmz$p|&i}`k| zRSTF2+*Yb;X9N`dx>5sT@vx$SKa+E}w0d`sRdf%sjJPC3O3454gd4Q51F{Um;(rj7 zeep(Dw(@mEjnx1WxQEG%9i*on=Qz2gQENVl&7C zHPKRg=?i6ZxUP9Z4lU{xKQo(n{Ee13Gnx5EsAf@k6do+cVa%=~TEXmsr+vBqwkKO{ zk0u$r6Ny~;Z{3_n{s$|Q8urPxKq2DS2?R*u=h%i+CFnJ`U_AZx zZ-W;S_w$IzCk(^2VXLy^ZR#J&rEgcK2&ee$5GMR!G>w zGC%U!6J#Kbry_vyzO4OX#_uWZDwhAr{KIydC^9S8w+};f0APs-OabOu%Rs;02uO@67MN85NgBhetV=8dp^5dm0t$#|xz>5+2kjsMnd<#e z@Y+Y2Ns#$H>>b?stovF2G2m3yL&D*=o1YQaYP70R%wHr^?;6QQ)z`*Vw&kdZ zM`m8c;gpSv93YARt&eI>ZnK&gRCD}?0q_J*7_|bpeYPvSjhz2h4|{Fe%`{icHE{8q zBtf@F@~(eM$2~Lo-dgPbr@p;?=0;63wI?^Ve*2(b+DI6#A3+j}4 zUpr+?nT@;Bs>~Po%XMn&E^Mz65Hv~mp~CUPRI{jXk1r&WvS`U^xVDpEy;iFv+El-^ zLUwOFgAO!cLFYJw z09%E}lj`^Gt%U-wL}j_=JTja~xx(yxkZd)D=9+Kn`V_u1m5SmLS}WEUa9T>smY1le*f>kMzh@yyf{hH;*i)t4hvDcbsYo;pj$AE z$@MQIvPGiVY83Z}K<*hC?>Tes?H`Tqt8yxllBs5b-uT}bC=X)^Gg^3iT!z3c=wtVP z-)dGC2xMFUn8e}#HuiUg&Tf*I7u48fW$Z?`e3-|f5yr+bcCv`{Gq8g{L zTbQe!;$jgQ$#F=SiIhN29o-ZktY`xE&Dx%*KsmWv7Ck1Db4Y-eH?uer7W;8H?!NGL zSfl-FgvusJpFi;ZOEQetU^>5z$5NKSj|WYstws)iGj7XlWvM>86%O+Ckhj(==}V+k zEQxc^9>^F9dyf`h&@q^GaOv0fAf}Jn?X%L~=rIwKnS+!SBT65=Bg+*vc^>iIHBAn! zJg;5`rJZ>e1i)${4~QuEts)endP4ljua(@`N%8c?Z2PmnIj6u*nEpJTPvSPzG$1s6 z0@a?o*t-wMJp-kzU%wOm)STySi=!A{kN@UG@Xtfyod65L&rym-H+felUQOsrsM@%z zPZsp2A)?VBN5T54gdk87IZiG0;^`rQoDQ@-NCRo#gGYe7PJBsg5FB}tadNtNg`hCF zW@nK2W>W$bIWfH$&ac5dA+jQYlxlv6)37_)P%8&hqv@+s$NN z?LuoX4plW)3}F!J7Zx|RFQkNkZt*_0?9|WK&{^Vx5T1a`?iN?VQ--*``pomE`Efn+ z&a}}h#OU#JU+&#*8K?>$5M8#Dre9FnmhcNgv;dmgO=||3Dj;6xGzZh{V5PWZ$Kp%Es!}~_TjuEkMm;I*3tKx^eax5`QkrX?VPSXC00II%hOn=iX?%#NJe=07E zmttg=;VGO`1C26nPwzCzTN-@oGADtPxmta4Y{3b=~H1=Zi z9%4C&<-*h-a*@mvNs$7XStu~ldSm7e^vU?9X~SHA^^v=s?+Y2ado&?+DZTRj81MHV zt~{r9(Fk%U1%d8ZrE}0e(t`{S@Q>3}{;R&iht=vcMVk#jX+WX{cAvCPjZ*@ykya?{ z*sM`bxBeCUCApHI;V3r-PYi)9(?0^z z{mq|lsnFjYWMXx=Uly%0G*4p-Nu2H-KK=5&BL`$Nq6GAxJJYIkyKjgH?$1bB56mv< zOFf-l!%nMcefNe45^vL1yucf!$%HTJ6SYwzg~b*@f9(fK)uPTMMq(v*ZYo@YoDGM^ zBaR{-{9%1d-Ir@H4c5T$8QTV9ou16Q8km*EtOK2lzqj8RSNzR>c;c+B@^_G?PwcZ8 z|KWDO#mzeEXtXXsIPi;PJ`TDpM!8OV@X~|TP^IxycSL-;i+Yg_#NZrWMBMXuX5iZED9*a)Wx=x2f9yXzAPD8xbK)d`*r^FXu~%UKI}q%&_*H4dkwAfUfQ7PR;AdM z5?^fifGY_C8)QPHw|D&w)|Z*S*EYTqj+780??H`(*2zA^GqyVJM$&yM?^g#=s#_%k zeq(3xD(zHWD3v~svq}Ih_7IyunLjT31IK>emP`IdcR0;eaoRaenqWt_u;akkYyJ^xR7bl?0Fd@z!R?#go@ z{*UCfR>o#S&-$%YpctvI6m4SlBWj@T6-Au6e3%YKve}*NVdSr2| zuj6l+hU%N6=6&UA|Bc;@6}eoT_2uNtIuCo)0UkbeI~5-BmM@-U`|`MoTl&_CIKa2r zfq_2#ks-A=OsvysdKJt5Q#LhhR;T!&7yO$ph#S}??b`p} zB})HDpu+M>}m^A-k)DkUN966~0q!GK=^*EF5I6J83I({SX4Gq+m?Yluew;BYw zji|%kWWRWh^lgtucm#AF|IOi*{YG-YMhPwtmov?4x+R6Ii{-ogGzfa<>v^!bWOErU zPm?(+31GtcVyvU#5}Fr6YXrd6MLWx#!g>=(&E+%OX{uQY5WK#DE^wrJCJjk3Jx zt`)c4)KKXJcBiJHE(BkpzTl8uLs#L=sQDfez)7gj*C&^0oxd0{1HC}Ld?b=hYr0rp z^Y2F@N*TThmUo&ZnNiU&1`5LW`gfBPx}S1CzA4HN20ZA8ud^;$*OohL@<%A*?k z67$(wS-Bp_2)@Cx*<2d}Nx;YCh{e||j^YsbPi-Pd{l>=oozw#wF_(p{FwyGi@_kwa^c=_aWq-h?LxTF6b;9)AT)<{!TJKw(SqP=a>_WuyUY;I$-_Guw zcWKM%e4)wSVm`Pet-J&-#0B0V^a7vg)A%0+XT^}B->6jT(PYyF987~VYCgVY2;zke zE>@V7S(6!3kEU|QRVzt+RvEu!LXT$r^sUEdy;tKa4xv;~0JG{3raA96TCaHHe6D%W zS>=n+lKCc3`uO_gzR79lodB@er?1i6KX$G8Op)rFUDI*d4p<$vHC_Srd$GSrW9-aM}%lP%{zVK~ZR5O`v}bJZ6B zJ#iU%JB}9C<5=?`pKXG|rLl}W^KF4;WM@6+KlFVMp?q*1@0cDcp3ElLWZ4F#|MgeV zaLNX)oZINr#KHJ>SmL&~_^M(<-=KiAu-Lja(+*${POC&<_FjCnN|6}s`&7jN#g_%` zSv%z{I%s(eHR|5fmty}f3m|5f=ehqWM%Rs`N;E(8P~5eG}Rp$fH%ma8hX@rMpOLiBQ>STGl%O{6M2m( z=^bull-)glUZ68KRn{oeYtl`Pt&^f2zUj>MqFcS;m8){h+W)f6PhKNP zf2k1Qbt1c)jIm((r$lkfvBm$+SdVjx)vm)!7Jjip+}3^Wxfu+M2zQxnL~O{y$nZr~+X{5=Yvfk&6=3<91DYoI-cJZfwpXHkX)OEv$E zvB;jxTk*V>F)o&!&ua2%E;%()bS>}sp36Pr1Y%{;W9^i! zQ>*Dav_*=u$7hmzk8Rq+Gch^9Sx4eDj`vd<&~Zw{>s*VlS%^i6X=Fwn{A`N&kFC zILmRZ00UBcWm0}OGp}&Jm`Sx=szzP*g9Dmb1_(+pPw=!OmJyk*%vjL2q^O6bD1dLS zCg&QQ$@uVcKrEBFiF(r~kgRV>!eK`GBAT|fT(aNLH--D~nWn(pIcH3K7FBzd+DKU= z)K{y@DMC#6;Zj_2yqi98U$s$Vb|GqQm$0Tr^_wT7`6a_)A$l7U)nWip;-$nlA(8zX zqU;sC>!$X{b@6O%XvEFFD_%fxoO|<+%o>%Z>;R(UREw`V`YB-5Jciz_!N)8da zJG|2aNb0|)^#rz4tl^@9as5WV*nl_!Z9I%0X@EM`VT6quW>eU2hP@L=TZ>-Idn`;o zGBsLC(_PE=g!~xzvfhCG&+oOz-&k{{?pBtATvhD_4J;9VYxbaA#LExQAx`Sz#@G=fhbVhge?Ky`t0NDS(a>3Ko#NI&%1!C8MA9T$ilouy^fO7(6 z{N>_)$`|T1G9Ogf^grxd*HE2$Y`xQ388=Xw6Z~Re>i19Gvq3G~<1l@b0c;hc;(M0? zp1V!20G);kdTRKXJYx}N22%0}_UUA%-o3{7#zBZrO}B~pxdDCd>1}pwLfT;oqnzj{ z_m8et$0YGoBTcetaiY0`{g0Y^v|_h-0oKi8R7D><*#7V2L%$#c;M z0}y2De=O(z#QfhsJt>zrpNXhd-p}o_;0?P^)bg`h3zp*yTaM{ho+oHyBuJj%6D$6K z#c_N%?B(9W)5bKp6pb_7Ss^n|!&MkhuQsdKEq2I!c~L^LBP7DXZ_eqI4AN^q;|Ll4 z0#8)Ii;xg;rjt$ANKOj%9aMM{|2q9~|G%S8FT}re`}PtEsKeVVJt^=X^bS!q`P}TU zQ$0kwg$quzXycNWZPq~2(9D6@wf&Vck1O#39L#!)p=OQAQTJ)z`}{U2X}>0ie}yG! zH^Erdh+JXrm;B2aQPOE{-JBFa6a`V#>OXzJu)-rQe#{r4|| zWcHxTM#&etHKuQj1D=_9+UtrfGWd^}5jhN-nH^CxKu%z-+3cg`Iv4@}0jlsLbyB*M zAY(+g{Pd8ErJ=rcP}l8TsPm9}-Ps(qww)h41N3+HK`*K~?RNhXCHD28+C`5^!{^;8 z&cR{%{Fkj1ybei=S%_WY#M}Lrd5DC|(|E#35N{01#1Da&uyGaJg87ikF4pBaV>KAp zux98hzF69$Bd*rvcxuSQY#o95@pZf?x@;aIZ1DG}=?uFdfa&N@M#qWjh5A1LJ>%Td zix0(&@4mz#+JBRrvy}Q>yf;iZxuo05maKA--*yYey077vAlJC?G4&dexVMh*EHBA5 z&40$3ma?| znb>I>pFMZG0iDh3<#AZ4HM-cJJ%tH1aAhr&XlMNJuW~fH?gIrU`QsGEY28m{SwFu{ zwIm#=wkJ|i8A>TEbT($L(J2*i>HQDQxsaE#LS8>g%Pa0I!utE`5xq`Nv7)H3`e$?m zDj@j$FxFwA7Ac}(^0zUdiMw=&!Pqia@_Lp|uwb*2^!>u$ zyB?`o3T}<&xcUxl3l+v5;B9a60aOG0qlr!Ep{?L7e2GLWyNx^v(6ywiqF?@{Z?9=M zDf+)Sf`$#hgID9jgiyxYE)|ZI!ifL4LT*-@+A_GYN7g?rvec2j3T0IN?wF*HQvF7v zQD^*&GyW{7k5u3Opif~uF$HS+rX(r240Sb5QE2oM3Xy7rimHlQ9-Eii zn)~(W3v;=PlYV;4TvtisoLASMfB3R>AbU zm?+12A&O}Gx24sj9E67GnbPoS!!LNWz|-^dQux>0j7(8iJI%ar9v8*z!+7~H{(k7n zwf$+pnlmTq%ZbQohWoJvF3k?ZSqRpcgmOxPxM5&n>hO|z@z7?z(y65h4Tv~?oqQG| zan%*T#4}o-A?^Q4Nf#$UODhv(96`ZKh5iJRDn-c~f_AXz`!z&7fmeSC7c#*)I1o9iQvhzaG+% zOA?16O;wvg;eSq`$uR`ET_&V#{&g|`NmuEEEwM}~1wEwe8EIEI^@9O*{>pu8mgANPLeVNI|PMOK=FV)LtoxeD+@;4a$%=ib- zg?`d1`zI)WB$_&fM@4&=?&#QK;`_6$tjj*`+ksL=+0ZDR8zTN|42|fI_h%2WISJ;r zB^j~%Q|prF6PVUyZ04_J7cNQ)-Ks6Zu!l`Wg=b5yke2oVJ8}?{)?eovb5Z8#TkHI; zD-A>8>ZINCD(+^!FSWDKG}|;yv`Ne}(+N!fE7l#w_8?3LKYJ5F7Hm3m@X0oulHlab z^Pq>5HZiatM6EpXpz0~)QlEaK>R;5QwrLiRhFPs5pd{b&kMONPz_v4XK0$ZW%3?mOj>;YFL?yGHX%zhM)%^ROa~9dhEHgXF9MXW7R(QR$y|LxJM4S5f7rt$@?mB{H1 zZnTpBiD#2_gk36ph0r@oWL}{sF^p2pB~1!&je+gRwo3OsXlQG&1!N5%itUxFSM%Fqg zx7w9rp;xPQP>w@D?F%`;dmDjMw}t!u7q@Ih!IvRTT#M8A@pAvCut8pBwA2PjpSL7%AKORUYE6SSCQN)70eG&SD;4P`M%6{=jNH*A(-ho?=fHV zH|Xv%DY{s;EnII`@}~)Y?Wn?Lnw7~0M}nm^IPji@N%3c-3gb2!8CwP^uTYlS-PrjT zHh3uD6obZ7a)b_wZF8$1;N%%b^ zJX)R_PyCQ;eIzFO0__(4u{_3oFbpHoMfQmQkgtzhHw=A`+0(Nts-=KwPqiYnQ}Rw? zVW8WQ@*3OQbJW4LM?)J>Mndno>72}EQlXNjB^-!^@5Fd{n*MU>vTS`j{m|cFGfh~! zh2^3zV!t5pbk}fPUtaQ#F!F|WZEmsJaKgT2RzpBt_6N6kPUAnH`}@Ohzp^>}7^}8a zo0@&g3->=yRA0V{$I9$@lLP}w3KKFhdYTe6p7N=sXMcs%^~k^kl|kFcl|M34=nZQ` zQ`b2Q%Yo1uv@%!e*V;}(ZZ%KWfvn%C}x%wItSV$Rpq3{p%OmxxpK4~AH;6?u6cUel?3yqY^ ze_x*c#_UPPA9#xz%DFk|XNEnS4A$OJi-Mkxz}?@6lyp|gtsba=?cd7K?ngWH&3_H| zu;F#V3d~PBBW@*#%ou5%)zzvMI$4u0<2=TRDoiAuSG;<)UzZ5Bw^l9vDka!04UmMM zZ;haRf`)JvB%oA{#yR|BP0dk(8kefD z9j(QwKluoS_z1Uy0+HA0Mj)8jtMlK+oQv-RJTLs}aw?O744We@OXO(rUr_?P&=MDL z%BB>=#I3eKG8GwU!dS=rPXDC-@|N!U_|=3a5B`yt0tZcA0+n;?kCJBfoA)ZJc>znZ zd9{2KHeA7J?%yCNJ~) zl+HjGA!SPyKy4iR%CV1n8ng@yUnxls4yUs#Y!6oWPuc|iH?I%%>mba`If=5?V_!lO$S!0#Rhffqv`kU&aM&yw=z939~WwkdHkv%1oBd1 z#h;AAdo|l`it>9SHA!dg+{hGR5^oH=PCEAZ-nxFytkK4N6E7g;2|W)dR}?%AsUVKU zGLI(2cQux@TbW>?S#;3?_BKfl(lxJqbIej94-XiasyPtTvebJkT;U&TKRPgS~(9dl38sHLTdtAPFKis~FzqlYyl#@zVM1e_G4^-A11iEB?&DlRmoHuJgSK z=4o3WjHb8zfUJ<)73hXHcou zdRNBp-Hy{iG;5FTa@-0od)iyDUh}X?>Qr9Kzh)Dq^DwI37F;p&px^b~H`4Alj#=0i zRZGub znwT{Nm(MycZFboZEkM)vW-yOFC3Il;JPAv^2{b);o$E^Stc0Gw3NtD(p z3i9|ISK+f4*`SdS80OU{7FbZ{vn6esLCJdbbscoG?w;SBdp0ZcO{Imw_nG1*?Yd4} z?t$^Mm5TpcPLU#Y=yz%0!%_CIbHL5POr0P>6T{wr{lGw4NvGu=i1T_ItjWidpT537KHFG39p@dSdV%A$DB_>1w2gU90KKTVM906@O~wevIjH)jx^e z&R*Y{yK9GqslWP-?pc?D@eM0bO5-b$jNy*1@k0B4lZVj%LFK-6*aC&%)b)ZR0so!KJbH2T$V$Gs^>}(pB-UH%#!Vm~CY;a^@Vl!@b z8IvJ&iDfZ^lTjiXPfrAw?-PKKY~;c*ILJt|!hR>6CcDLPDb~3-fj|0Z!)7e9*M7mi zw7=4z{X^ut^|z$S7-WJ@deauizu`k%oVpwkqO>o*zt((@HpIShWM6IvpY-_`pU|~mq1z@C)ATI;&9is&|AIrV(xWLHMH0_;L`OX*{V9H(Ql_tnDAB( zt}gyRK&iQ{9r35tI$@yaQFbM=;&gKM{_b=Con=t->=*C zEpRQ(mz%WMF9@@Q9zTq30lg3Nw&}9|1&IHf=)RHl`+n>P4r4+G^JMNa(3w9y{q=7% zxSu^58`tH9j(M5n4xY+Kd3X>>)}ni+5GH^8jV9)agBA5Df}HmK){BC~z|hx{59m=0 zJPdE(k?3xcJf@OF!tc+6rHGnCZ99z*#V=~|29+K4)9S4BA0`bPj%B9KCfa+>sy(Kj z9)WCAmF<7Tb~Cqgj*K0iTUU=~HL4vO5)@w#oTP6L7u`tx|AAWc0FN^oJ#Zx70T?0k zP*ZkgIDYs2&IahCM>kN-Hd38dh`p7f_41mj*oV-f2kpMW2&I&3!l?91uXL&c5(xLG z3wpTT>pfXF%DTf1F0W94NMBz$0D~Ud$u|nplnE-fc$E)5z(^F^(`XqG*%TwR1=&2B zNUqX9(FZ<8LzHCZFNJU_l@H@wx_GCIH*R9nucW|p$);qAyE(oGA$L#b#NhiP0Ah`c zFX*5Oi2UOL0o>P;Z(N%|pG;+hXV3zsB#+)Ncs3$J{T_$B??Y8QIi!vp>H@Y^fzu2B zCOh8lduUvmg=J4A|Lob^zXU%712&DpKT4nv-y$bdZ2}1Ha-eW)6CSWXr1M*P!&BDB zLn5Rqw7YA40-$#!8#kSXXjhW40p7XDXOiY+1Rk_)T+WsRDWzD#k%!ycjA7NrLcU#v zd?Iu>DHY0r@fnEz>j1U7z(+3c9^@Y(7yn%(IkcjUb1)6c4S2tMS0^a%pJ_$aXyVu3)h%gX564Bu}&^<)IX(U3%9 z8hWdR)_Ht^n+tV8NRjok?wu1z7s3hR>*UchdbfL2*Nd_NKJGd|=N-B?1W{XesDPBL zXzVMe_h8g87tIHui&_N4duh*Wj z_qSbZd)ir1jyu;5KGXr1Mh%06uJFqNvYFkWT^CYDF8XK2Y77II6S@E0r+3n++~lH zfgfwn{K5I+eBj~Up6RB&FHd@r@h@vI*4;rj#6bw9pz|`|)5;R`;#g|q(&RZ*uNM(= zX85FH4!AK>UnScLyBiw^PJNi{fUo~NQYn~74K#h+%4lx>7SH|gbh8Hz+`Zz1N;tsp zyS@cow`)9(0v-nGs#c9ya@KD^bH-SAu}4ipr<^y0mv%Jw^!)Y=;?n`*Nje0!0e46I~PGBOD z$_$Kv^_|j2^vkNLPH<3J;DTF?mi|xjLH*u4k%wq_MDNouFu)Tjqtsitf1A70-0KtW zbI{-2UfitK`?yeNtkiqhb$#NJGIjef;=PyNPb=!-$p?quRZUSryDo#BYp&>b;T*6p z>!RI$Nd3IIuEmHQiI#cd$@$*BE;v%Z7kTZ{g`nX>{l4>$*Wj5N$38lf2)c^c>l`IO zHI1(qhacbI071IFd$9e%91mm_J}_M5?XFlavLldYHehq9w-V^Nkq-2%cA;_dq`xZg zdz9H52hLbM&^%+$z#{2&&W2~dLFg%@0bU~Z`0&;KPM}YjDOL6Fg(2O($3)^u9bB!K z)|!UG5*KwbcbG@;o}>O))&~maWLckbpY8JN<&r_~2RB8KbAC<**IR$jvHq|AcV|V4 ztoAwU6xZ+VH*fJT_v;yHvQsD(YCMi}jrC%Kv1TrKvvfwv{qw8U+bDSqP538%VOt2= zcxw&3IlepYRV-2jPaPV&Zz&1W(%eMkmX}y067k23mCl4%1dOir-a{TD?z)%-gC3ZA zw=}f1u^$Y6L@rqT^x0;=7fmQi|1KoyYdg#iLoed?G8=5bd`LL_G%wmIkllGTa7ME` zW3C`6%qA3Vv{$TvWgvh}8yD|As^%9AV|AEwQY#FaC!oHxI+^2<0uNSu5@ z#f~;lyti*VHYI4y$;-+w@&${R^*qUWV3blKc45it)K2-e|8=}n#wvB^ccwN5b_%PR zfyl2kcVe?g|1tn`Vi^|!F0yt+le38?ACT}Z zK(9#X>hi-ho)%nvkxMR1fU9l5`2gSs&BDOx;B~&0{EwEGoP@vxl8wjX)uQd0H{kZ@ zYx1(&z`Zt;D<@qp>XXCKk=5Qyrwh73Ub-#zR?i3Jv7j`g+&qngnW&n&pZwbuAAvyNwG7pYfpGs zrgUM=4lp(<8}%U|YGM!|4h&Z?Vq?}ufS9k!#kAYRxHzk)-8Npl3RnRWjIC-Qgb3W3 zVpTNmp-3fYv?pYZd9n}KIyYwoi*oXMTPIJR9;BY?@~sGwGhkHGj?=57i!wU>Sbn^;M|%2ot{arr!hxJ6;7 z%e9(v30l6dV$Zmv#)jHUV1K%@PjIIcvFTZ8pvbGy#9`AK|Hccf5Y!3Y0u=QbDiq}> z*=6ZBl7za{SF`+(QH^YJ|05yx%Ui51;B+)PyK`plqo4ntCbqcw-ObhPtJlRR2@B3C z#K#bx2^nn3PEn0|#rY{s)nc)!IuubL}F=3#B1y)hBXsz<6xvAJ_LWi6>g#f#0m{w~bg(vBdXmE=kK zZj+?1_=J(3_`ZBP;xIUR+ z!GyQXVq|(g$?|bHzZRjX_;TFGQPLl|AI6_ke4Mqg4Sm9i_%+?M{0ObXgVlmbRQMSn z5zck|vw(-A_xSgcM={6yS`M;VGN;sf=Et1vJvQar)gQ7iYR&L3kg*eIo?);gX7J@u z$V+P;)HPPQE8DJNUeb=?!VY+noL5pTKr1YJ=l`(w-a$=&UBBpKK>?LsM4I%D(jhbj zA#?=k(nLB+3%&kC5fDh|H6cLgU{Hz>y7UqfsR3zHL#Warw3FvK_xkBT8NO9=@pbmJUo^orJFB{q;v`0M(yUMq)iL zTr{g>jPun^aS(_B{=|qQ9%Z0PM5dg12E=jrwlKxu+vn4+nkj49#%5dtqO%2l`s9hQ zPlSl^DC-9KggX4lJC}C`=kh5HTIJULGrnaRF3m=UT)l!gKuVxD1OJJA3y?ET3-5L7 zM;6*E0l64)S7KG2fer(b7G^f!dLSx&W;ngviYNqlAy^{?zDD=QA!XrmM;X_H6FyfJ)R*fn4}zk-0^(rkJzw)$F(6zh zu2%L_Y<1ubNJMvC{tq~!8pF}m5(ObW)}@@6RUc-c z?W+mij3EJfUW(0o|FVk=7jJCXdWzQwlit+5Sk;raNluW8XUygU2iOdSA#e1W+3%!S zkbnUvLFF~P$Uyd9&X_Ftdchrv*YCA{VNV%z(QM|Z{6n7nMKd>vic2Tsx~+J zybnQSamDvXAj?HD);HH|wavuO(&*Y0>Nw4h9R5Ltr%fwlv4$>9TGFdHv3TXw8F2m{f8yx;e&6zPF>Xk7trw|S%~mDdrxjod{vP;| zH+)4{j~2;$VwIduRqXjsWr)PHVU-*@DjiRj2~Tq^otrN24IE#6`*_^)>Nx?*Gojfi z-}AMWKWtEPUixew9zn=m2!cH+S3Rn}CcEV6(+~eRqMa|C8)gRYOQWpi{CS_R@wrXu zJF|p$QJWihdyNIHfMWDi^YavrTYTiNPSYLZnaOr_VvHBJ`b=9N+Iw>9Q{3fiNN{fu z;pO|cV54Nalr_P2!mG3q=3lHFK{-3YoRYQ`%#yXTXMEu`y5+x!wCI;ntmFQJ&OQD7 zjmU@$v%^G3^TC8xR$hX{%c!T(cG^FJjQ%411Q{_A))VnJ){aNgykGhbl?61b8Pmyo zfVgv`*z52xN*z2~xol^TkS9!L@m0oNiT37Uf&QhM^CUlf0xwbAxdnWxPT^@O2cQxU zUy_D{{-Vge%npMq23N|WV{;c2oFB>AEl|FF`P%t=A;pWrma7>x)hk=SgxvDo6Q=#c zG2+zEF&^*Czsvijhg_ZbFXN@0%k$#g#L|xKWDV=1(q%oI#2Aq6-uE{Di*FZg|;vD2T}DPz0$~3FGIRhwDGe^vXs?E9;BI}%KE}JL@jM| zd&6hCNay6Ofpcb>prAQ@(6YWeme3VCZ29-d(w@NTL&|v!nzG7QGopg#x7xn5`fxj+ zxJey|3TH(QT7?!{%#Rw~LRZG0GMB&$~UHQm~n5tA#8Nd`~;Us*w;TnJ4 z_2lYGl~||ZVaaK7@qRaS&*ggG(Dmm6{vgx51zJl{69LBA*%}T`2yG9h{|Z;MjT~dS z({jZsmr~jKbL`%BQ!SJke{QTpfs0LAB?er*eG9&sE#eaVb9FPBBBHfX3uqNQx9E{* zcha+Z*;N=BX%`7Okh^@+XVd5Dr}~Gdsu=N)WgO%n@WAs|_^ZrW_$yYdvWY6{(OBdN zgX+8LcEj#0xUK)z!NXgPmUWG@sWc5TC5jAVS$p7*u=qtGvc z&)3<_l=ujR2EwPoA&QnAW)+1m@j)zevtmN?{$f1`zPV9kX;C|^A0u?7 zs0{(bX(I6Od} z6a+%Tf0YTRs0(zDk%$0&XrCZ*oB4X01NT~z7Hg=afagX%D&?Jo+lI&DL`RTuYv_z$ z=dAKx-nrgf8}ei&^H5U`nW+`%)+aR_;%)!FC(~>j?63UFd#}bi(wL6yT;|3~dQHLyaC;wi%Nn-(i%iJQqq_$@m z-eZV4tE{fF;ov+U9k^HaR+aznq?qi251!_Y9vMEMTORoJS+`Hxk#WI^9eJ7+&%yV` zg9~y{@YPpN7uX%lbzIFqbWhp=))9eS3WPe_$px_=*S(b!YbUfp)2OD=%(?pOK^tMJ z!&h2i!xs0(0k)+We}4&??|%x@Po9XH{*@LPq>y$e%Y16P@yO5Ss&z>IDBcPqqOUIX zLv41g;nhhWYO<8YbUF|ByLI)ohnyGN-c{zIjvj6_Kw*22w+P!=3-YYF50JYu)Vru} zFU!tW@9d zo8z%CuJ{{*PnwS-rwtvq`^Z+tPDwc%P|Vw?ylJ_frJ$0zox-rTf6}r)qc=Getgm@4 z?eG*xE+*@N@>`zl5J!-pN|W1@=_v3uMY(% z{*E{|UDi;Inp|`R4zBxL)o_9R+YSH$Ems>k*_;clY$sSxayxxrq)m&hb(9#`nT14E3S2R#mX zx4*tIas*uJ4DSz#1IH3TlR}1_XU(NvxzI!5_p?P#CLbvJ&MM7{LboVFWgvk_!#D`$ zamadoksa6>7&7@jX||dvdA5z|LVU*)BW)kzdb=HOTk~gRv{>eZ?megvm8)(Zg|J?< z~#p5H|FM#)4Ehy0mYRuf2xVf0qz_RfdprA z(lDn_rm^pQO@2=d^V)rnS*pG5vEe(igKEu?FL8@yG(?ZoFLX_frjSLizr$VQEsPfs zH1a6MLXG^RTu!^%xNdKG4yUj8`CewE2Zv9$aJ)D9#0mI*>L^~mV=r|gU3aV_p@3jH zny;a$m2EBVdUK}$Gca`qyBRHIJ}EluQhfuftEC#yTA0Rsg+e-TBzJ2JTk0-ZCUYcw zZ_rHf=e9=rFIn4mIhisWSia38!xuLCjUwf%Q?u)Dq50;Q=NpP|T@JTlT3Yyq8513& zV-3Q42p}J0=gi>ZKfMW)2l4f421#wfHn1Kx5()VR#xnHH^KZWRC&IO&AD0i@yciZ;yK$a$(11?s)9@jNxzC-2 zPen{+@7Gz(iG(&3$EL*Ag_o(mEU5&(XK`m|=y3W;dVb$2s{%5gho z8ZRE^yEq+vhKf_rwpY22b^2uPKYCLynK`O$XFA2_acsS$d~7W~(YXmeLx`Q*N2|Qr z2u*V@e@&7mnr)5Dx8|QBxrA!>xl%Ei!TZR49YA@{z8ziGp z(7H*@$EQaY-BrNaCsY?{D~B|td-Xdg8y0ZSc(j()igxSM)2 zrS;sQf|j#~RF+7IXiO8S9rJvy2`FIv`B$CXcPJn0_kX63|Khy$J3J+b=$xt~H1ZUF zlO~WKn24?(K^QHK^V+s#*f^c_3H^3AXWIpc3poW+3q%;yxWp+{t`&~ZQi^lKMhLIsd%5P;V!o@9x4eE5UCgf76s?O;i zic6PzEdk2Otr^=B6ZVtiDlM!{qLy<`9_u14FB~L|X>yeOVU<*M11mx)(Y1kg(u132 z*x%$#VY5EIH(HZLROkKz4az?CT#qSrXJ@iI)_<2TS}&O00dD`6V)SMAWpf7lF0(w&9aw>_S_OzKpEj)-t!CgWwhpo!Acltrf=~;GBp-tHC** z?1ZmOyts`2FiF>GmC#I|=uB2JS?C>cX{smC&7w-Ri|G%OeuK#l_wncyRhOHZNPcBz z`5^eiZq$-uG4=Ly<9CyNvD#q+wSUtb2qZ-wEaJrPfF zPY^;yO=otCv-X)a2ZtkJ?@RDG3wxW~JeTqd45AZ5JI*>c{sTWC{J#tTB`Q!(nS8bz zZz`FiHi;_!UGlDDxMS@t4>XznK)u=L>FBP3MWM-)W=F6?Y1v@Q%h7o3a8;)evV~OP zpkJ_gC#2uTI0zDAzM~~Iog3`@AKsM_HLpt)7W@ke|IF!A-;~=6PEROF^7kzNA;jt} z_3dnm+-0Tv;+H2|!eKLghTz1Ics626s6z5d+TjvNs?z(5$nNJc;78t<{(bcDIf}7! zS7&KzUOHIp?QDF8QHdeLi+f{~P7VE-K$|kbpun+@9r|Imt4)Z^dDx;yW%Z91jHF8B z?v8IFKc-P+im7@Td=`EK&7F!W)^Y8muO(yOB|kf~S7JQU5;Dy4vp+=aN{`~F_hZHT z^nmX#Y4b{7uH9>^ZAITbAZcA!_H2A43yACk-jDYJB9N;AOQKYA2r zxW*c>Rxj;viJMisC>jp`bE2XpaAi7oQIDyK;}jwPUW&$ziK{Gs0_@rAgM39ZtUkJY zFnS&4&~}({WO$v0BZhP06(9Z?3InBmsJ3gYX>wT@=NOjG+jb!goSj={u+%Mc!yS&N z-=DRwfRf!SCHgl!6#_CZs(0D@ee4)8yUZ`yoP7;9Z{Xuk?}4^Wp}Fw)eTsB7H{BmK zv@PwZZlXk<;&M!!(X$+>Zg@+q6}3WJ-&yYD>S>9<1{ao0;+!(DDf>%!+sdi$12()D z5%EVtG17k)g0q9O_igUF!KzPmn{Ljgor2J}Ot)(Y*Dep1E*KeHzJC0h=k#%y&)`KX|=J-#k`5@Wk zhx2+27JvVKH4Bi_wWio%h)eCnB0ug}-brb8b5_YoW1vjw=J*7f%Qufx`%LFpc6;xY z_jbl?&CD4!^{9Z1geuF1em@)DvcN`U{YPksJO#joK5lXN#RoF#1$U>w+V1o}2Fdb5 z7h8b!Jg3K~T<0}}wsyf|?XBnXxCbHWf{udEKkLw_lHoiD#F}A|HZO_=CJ;Yj=xaXa zNs_=*e!Kd8Viv&t-u_>KS4)3O{k3Za%KiNjmIKQm=zZpAb162br=_+LnqXY($6Oh* zfe%9myDuEIh=>tdm72P{yFWtB{;zSouN3qdn;XtId}NZ17!p@V0QQ{T?va3tC=LOt z`m-*|&feU2OZzRY?(5g>I7=_nbTRTaoTK8|-G8uWqaRi~*0&#RE>H_`TVvCYmU)^7iO}^J_mXr2+mJ7z!6id2`F{>I4kSIVe-rOLDy}ZxRSB}y z1x@0G%;;;ws1KIHF1hl2k!A0*&yDxWOHGFe*=}@Q4e&#$PlYV_du<^ z@2PtrHRI>2*Er6_QWa6((rjTR4f!2fM^4FVeRfkC164YM>vO=1>5!{I2(Z2wJ0{(G zP1yVp0Ts}0{&L?u8f$0RUgoqaZ#1Ws$({ZWu!8fak!N{Ugz-=7G7i)ZD}>c4MM}-&v^pZ=dThTY4*j()7AE$`xeSsn{-j_voxr`)^a9 z%Ka#eFIJEwHb(miQ7B?DYLh+1$5 zj0%>PdMY?Z1hPe&Jdvl!i%1&(-K{(b=Ho~g9JG0pN`ucu+>nbOOlhj%QoUz? zJ=`9~+#36S9ixramJO`#H4)`Rr<+$hCHQPe-`1F7i3Z>7#OXu2K55pnH3e*rT%-7*f4Xu zO;PNmtbx{Xr>G(K($RB?L_sF!h{l;($CE4QkhHK&^-A@njV&jPtonbd6^19%xV(eWrpk+eIi9`PUV@K62D9^H=S%S${q~xjkr=K^xIL5sc<(66#bD%)4G=A~Tnm@ktU90XyUNcHkg% znalQs=(P`DcTS%9X?FBSc^vZX^2mrM)~UKM2~}HZw)q-U46%CN=)$m?SG*BkqPx^Y zr}XHukV)xzc4l)^*;lXgzmtx=+SI99km?!u@Kvc9H0=o6wDb;aj5-V>^=myeFw(M- zNM$vzY7_YJ+uwgmyNBd;%$k{3$!!w~&0$?g{^HvRMkg!qTZQG24D2tq@B=;va}Iq1 ziO3Dyzd&DizYZj%BJDq=g66JP?tdX?H2w~?XR6HM2|W>%etUNOZS_M)CTb24z^ zy>gaq(|Cd;zS#OxHRq(!xlW`nYp7Z3Qm-7zaX6y!T9~xQhg%kG*M|j<70-FDVzsST ztCR?(^0^2lpyYz@gZrg|MV?{5@A&Y9{+N5SVK6#<_umZLqL?pdr9ZT8um#9O@vZm8 z-xeX?vm(`cRImBZs&3M(D22#l7o3EU=HIGV#EXWrHft}8+Xo^KJCs}3;qu{z_-xSW z9n-q~5VIS_?WHej=qylMC9~wzZz3O;mu-L7al#;1G*d4)Vlq0QTVO}^^&B-FGyJc! zwX&ba8qH(`R3Ni9ilgyYx|IHs!Bux;@^8mpEVODw;mxayov7S9l=5~0d#P2oIfR>G zoGZN_bc%Fx+FvEsil9cIy3rXO7ykyG8h>I5Qzuzzk7fYgzj<6NIc@VpdNtIIE^uGq zbNo2JRK%KNB8DP&iCqt+;>TCxmjBOze+u1z&YuF38+`7`R`#O8x`7pHGmyA8YvD?P z#_njze$u~8nNaX+Q?PpH7--v|J4ZJl=0P1G%-m%j;LQEHNH%)}o+<+nc=sjn=_>J8 zdVk4>n;4w74zI|En8Esdh3Wr#3&0b97v+;rSMEFK=^el)mjilR!1q+OnP-|t_DJ_X z%5eRjJQqAC?yt+c8e+dH;*!%1yNFxlt{+H%D_9~xoh3ncUm>aIG*(7cWfwtIxW_4* z?*`8m2G`vvK`^zFWd+O#gXu4rqo3ppgFpk-Hy95$zZBCv_t&Y)1k?*U4~esZ!eqUt z*t~-lu^$JbvREFe7v_%=oyXT-`78qao^#49)qCI_dJVOAn_&af#RA$Ua!XLuR@Oi; zfK_D=IeRu@XbvkDbICdQ;nC#aHz*og*%T4`Nr|GdNslKboj!VL;Xo%+HY6y|%?w;T z`XLIgRCmHR49Tfk`oOLQ6+UPZ@M1N*sa*3erjHyR|Fi8oTsc!~Y54*Mr9DVb;t&VA zTf&CEqkmg~7gz*RPu2daVE8vh)$xG}?4)y3F+NZ1dLlTJ$2BZsyqlohfRU`@=o<}w zb4lKmQT+*g9+2h~gL2^AlaCzxGHj1{D^F~lIY@tYMIU+BrF5k~Xa@BRoo!CK>8|ly z?<5r8z@OlpK2s1~y@37ZcbeMPBTFROB#mzG^VA4PunFHbd5P>{`<>8(UG*)RrU#xFmX7s`#xD67riIoGZHW5JWt6;S8qaV?1Q~~qb!Y|9FWlaCd==(seAy+r4gz;W1cx{`?tPYvsX6o@c~iHZto} zP+W+Vk`GLkx?BEi&Hr@{*z)D;io_-DBmG-E)B0ibb*cJ$$=ZuN@YFPzCE<0E!wi2d zYv+q;mF5}n%WdcHI|}#)SldS!$kPm9nVlK+4`qls9-%yUu`XpJv1ep$EE}l1!|y_6 z`~kg*{OgB~`fdE4!LY*X(Zd1HrQCejyDtzuO3pKA(H2@~ftztgHhYeSfADyEE}+|; z2U_1L`X?QAerb2&-L0)8NM4p7=&$m7Y#5?4#z@cE!{-EQSn>4rq_zzSyi;8Pj%&TjdqR;$i5#x|lEunSpa1+wkx*-or3~fgs>{p~3ImC%)WpK&7w(EfG zDME1ZgQ7BN#f9+xR7|u~1QWu@jTrpisJ5nKm=i3lD)9QR(|f`Inh09FMl-OPI|%E( z#)>Z>j5e6s@>HnWPdFCjJco=Gm1AAm znS#i9d3>Tm9_;+_N+&f@Xj*^vD!9eKJu5T$)Y5r8Z~so*wWQ+m-fk0Nz;|}H%G|ID zWaifdO>;RGDt%VlAM%p5vnUmXO%RXx7Gd3kV8t(64P2o-xBWjt0v#P~vp=FI`psJ(0}fuDD%hfIv)y2F zOxdeVnJTUZP-a+BH?Njw7ze%@68l3XiZJi&)wAcnCLdXwG9dwfVr!!m8U5tC;VKEv zFCdRusB@rGQouhkkFSn+!A8$KdTrm&5NIkCxYP2z*AQf4Zh?HAs;w1bYntf2IT6uD z9h_!MiUv=DE|sfimal@(!na47BHKjpyMWEr{oc)rx3;TnC-Lv`N-0{X^~||u6~4Kb zJLBkTPQ^3xQDeZ3SH|K$*93nCx0Fk1qr@Y*{pSjXKWn|n;71O=oZjZpGzHZ1n14|$ z{$(PeLC>)Fhp*w6XM3(%hHx|4%KEGYi&%3*!{uMWA_0nZF?z#q3Oe2#UK@JG$RW6v zO`vNn7hQ63T&1K*Ky!>yI%caseaS#b+Ej1kwuVyBfLPtkZb?${pyv$*fSj4;(9ZUI zX0k^j>P{zOoY$9fkr&HkEyJEV0{K?_dG?^RKT7lC*+tfws>w||zns;1xpnEyoyxWX zs1d@j>12@}SWW8WQS>Yx>pHWg*>PJ^7H&dfFITGdf!13Pta*pGnEuzZLB1(AK6!|o zKfe%e)vySuST{`**FV~A$|Br6*HxzD60oAu7nokz7dh(@X8R0-!(T0bM<=z^3es&Cn_UE#~!cLAN#u!}!NaTEI|H0OH@Is2dT z5N3iiJ*8JI6{sq^a|ZaTmswt>>k@qaiFbsE4sG3msrQ-U*VC(rcXGUQQ{x3Q-E#xn zZC5a;&6gU@^xR!Tz=Vy7c(_Vz>qraUpj)ESyAExluGD5yJ2SjXk=Z)7FfbCSjZ%#- znSbwR_QPePQD=7|CAK})WvBB1F$;dOy&qy>Zsq(&^>@klQSG}`)8=y}DhtdV2pSmx zWQNZA@NR8Vly9(#ujxZ<0`GcOn2;!U5IpVehow*tmCGcrSv2@$Lm}X}7VWGYAONc` zMN8&w_w02NX%_(T<7aw%BhyM?qgVELQeYX;t%|WbNy6IzO0L-O)y+m`n3_+M9$Fzi z9h<+{zM3M9^B|oE8>oYe;U<3t;VFp!q3AMuJ8@y?0C$CmyguS2{&Vqb-sftqX}p?u z$Vk!7l&%&#I6U-;byLK+W>5maUjEaag*|>GWX3|Ea;7=p3Rw+78hGucMyH*X=(70c zGk~HzGIyD&+W!8nNIRr?_;)J3Dt>d~_j`%j{pAH9FpB!%jCE#zz_O8ANqpw*q<?YBHl~~Bq~o;DMOtr(rEQ-G*Ig< znKx>ex=l5sr3XbTeLUDJe8zpc<&*QY9ZX_4tmM@VJ=pzHWoNd;HfNhavcNV95s5Y8nYR_B@tN^mLJa0zl$WC zI5s&G&=ef4m*GOn*iiqEEUQ8K6)!XPT#5f7WqXRCqt&B{zWr|nc9BH;rX^1gORMD% zq}Tl+7%E7dcIpt7WGQ`d)0s@~&urn#`Ti24136`&@C^}3*@AHgvn5*Kr!0!;{8y42 z9gnjSS?HZ$Ly~-Lr8WmmZ1wz=)+~Mh@*mt1917y~G)M#1{fQ=>0DJ@QU{3?jFMimK z0x&K&*klaqF}$S=!p&wSR$S=7S2NuRkWcuPc@|ReDTkhHO%je-AMDQ5BLK9NL%Z%!NLD9(JjBPxcO5LS$l_Cs zDvEXQDEHAYxH4@>;r-nhBBS%$YBlwi2o=xFb{Q2arusq#0N9)wGpW8S#6r4mVL+w= z{PO9JUemtdQR|(AnbIGZ>yC`kWA_-l{qgfzYHKCxdHN{P4B=utG~Po)L5eP-K&f%} zuqw3LAV*9DxB&doELIsJeR7HiXd^FIVpqz(R~}^f8-f(7MNI1?8Ka8Kh(cDHshvh2 zSu06VDOTBV;o&>K`?$I`et!ny$bkEa#0KySV{j3XlAuQ#gxuJ-*kc)RYzU_tu&nLg zlt~3?Gv?_lJWV+e()->b=DgG%D}YUiFOW2bVIqAc-6LQE0F zSjqd{bhY>0Uw7Dw5Rzw~VL-GoFl`-*sW->6&Ox$!Y-t9*bJ*e~W%)>uUa*DA*hTsCF6JTZUwIcc(`)v#jP)LIMRf)#d08VWypAe^3~^o?si0+fS-O`M zxNY*YF9;A{Ew8g2<@2@Yr*cSn9L_Z@G=GB#=YVFq;3pYWg<5r>nFU3kk{+YIqtLg6 z{Ai2O{<5#uqBRjb<+LiJsQ4fqFIg@`E+ZbHmwZ~Hq96x;%X6bRWDcBEEe|l4)O83z z&^U5@id_Bz3QyET-L2rbWdpn@Ivg=_Fjmu|N&atPgV#h~ho`9kbo6w{DY)qjl7b&cix;M$FwTQ5zg+!zWjcX? zlLxBd0e7Cj!faFGtL`VijeLd^64G2}N9unQt7EhdElTkGs79eCFl`j!r{2krG zM0z;be3ez10JY#PVQHzkrJsXKdg%63@vBu%2F}vK`v7ApgJF4>FH4;iY7tQ0%(;7I zkM&gE&*hz)VZU~qHKm$qahbzI*GXG>6eaD!MwZz>3u1d^^#Niz9%dw{mzZHmAJ?|i zxv}4$)7B)8iVcoUvFOSn|g0m(m zo9Z-TB3*XkbZYroU`eB<&dFu~d-c`sgwL<^wTJKOU!}paTWuUH)3e6gX|bxND+z+A zs@(Z6>TLo;-4W8tkJfyyj*o=8FW;OrLSs|2kxc_d6az@N-;XH74a_$YOonkaq$VJr z3>MpQ;frLeNsC?TS2|T2BY6KVY2*a&XRq!}B|XS!B%P(l%CQM^pr;Zl zo#@>i$rr=&?Rvpwc9BOHOJCbtm4^QAhi)!hE-b*l#- z$rGwcelfU58i)uFzY}-0bX!C!XoJ#Her(g5_|!7hmepveseWc&qDfM~w8g`GgmCeT zX>C=3Kac{WgxO25xyYp45xXq%Vy4&}V{!*jfks7yHe-~1^nCidS?q{~lX=HlkKJy} zl?XXW%MZwbO1tlenXtvV=GJG?bq|$oes-f(FcFJFOi6D-j1xeU zhf>B=;*wuqe#I4Yx};nEWgN+9GG2$tz*xGIe@YLOYb7z`U6f22|d}s0p zQa_WOcgECq$?OekkczWdh-1eaVb-0is}Kc}k4PO=)gncLPmR=PTf>#goqW!BP2)0+ z9x%eyzz3O^CXwUxC&RF?EbPLJs1z1ib(ey2_m6V{hS%ZpP&TI*C7H@+&_0=6{r`sY z*Hd|t9zwA5%*4p)X&>^LP)A2UmAieXm70UkR>`gGl))F;zLiTvlw|yCZ^-=s`w`X|<7g;w; zAUGM>E86F1?7|=wkLnJ!ks2Tn(My6py@-uaZ_qj^n%i}E45{-f*CXrW*aj6CRI3dK zi~A#4sZ|v}E(C7x&k{j8ZXu4zsSOH}3LA|$C#FrEK4U28_NDSXvBi~(C;=M&dvel- zx{zY^DSRuLta0iyQ^Ptu!Lwah^P;YMg_5rL?BKAAgSwq-D-#<+v70;l~3C zc}WK$e~~}3n;kF?bgGwo3uEV!P!INba41v!wSVjP>ru4No{x#Y2szGhZJg8zxZXYThPKNO z`8t2ywv=TbNCNs(t%b9=*Q<6vKDos!5^)}R;1MDihL4HpKlQn?J$w_MC~q6tI$xJC zo^7;M0<&spiy*AVpJWdSw;X&Xw~a9yLq@HVoxc`<0;-&5Eqhxpe>Ak1tAoz~PW-j+ zV&sq5a}C*rH%N3rIO`7xd=_#juezV}E7v#p3cwLO0Bc5kz`rd?P-IP-#f$$=ky=NQ7x?qqW5;cS zQ*$bP^76(Q`u({F()t)b?g*PyjetBAPBCNpRF0g=#L*M;xQLr>JK6#bZGJlmu?k+&59_5pg9d}>!Y7iEs}$& zlsc%wHsm7CUU6PIu{;#R7>1v!!)+<3f2|l1*E`B?l}5fRT!`)d!!0{Hwx%bNwA{$! z7&2bp@96K_xt0GJGibQ^`j@sh6b~jYKLKJbbva|3Gc=FJ_7eDYr9U!1aoC)LT(y1o zo@{>yCl7)ta^Zn2A^Hy%)G-s2STY%zxf@#gs>Jj8_4D~+MEj;VnslR{?2^3u7Q29D z{yGH0+M%BKVW@gUBh9UAGaq$khDIIj4p1g!er52Mp~HU8t8QjpF|7 z+>b9YzsU~&v1ydAaxUaFY;o=7wI25l6VfZ;uo&S9@ycBgSvniXpg>SPvo}mO$+e~8N1?M5YdI$CR40{3*21r1KmvU}E_83+scg~- z4T`*9eiGZw$5GEf$^Daq_01e_@4plrKPPpdcp|EIUN|3FgG*lC1OqTHZ%)d{S9BUx zC>&qbDpcjFQN}h?de9cAYzBn8Brnhke=GFyY*r`XTP953#gWEQ7+$Z%?4Q#PBpkg^ zZw`UnQBHb>4+p;hi~(%2Ef1{NimA@KrY?|%UgJ=yP>qA^iMKpF0^QB@++e~71do7n zd<0QY=G=Ans{&MpwnC!eKzY!A#)T*!7NFUb^h|jphF70Aajr#j(o6r6`P-^_xro$d z?y=MF5-Y*hq?jm?zq?ap5Ve~UXC7QIuZPJ;%HClcls@BrCc373C z+r6ih8MXnrNL4j1iQESg*U%G0*XmBI6_9wZ0sy;_ck;MZ=j0hI`R>vRsexpO%I7oAKaoKBYKPc3_Kt)3R0YhS;U}VW68-e z;1g@^svT?30l#-wf|bjfWeRE*@nL=8bvz~|Go&wlJaLVNfJC#DXD|4#*FxwB=$#Ly zY$swe3Ocjjf#+4(6aOU>3#lx#%7AK*3*NAI^oi@|3Ukwc%wD^$o zJU6mbb!45eM!)O+0?kVYr;JsUjZB;H;TdP+s?um%R{2dUJhpB9K20MW6zgZcmN1Uq z1sqNTrmThVz@XDzB@F1SXH*h zFYcW4VfnPjh*WlO+&7xqgg|HWb<1%!LXtX}`G3~-O*;5w_Lc&`+wV{E`ak072p5+9 z>Y<*z0~_f$^a5f}%WwJVS(0CVi=@Hin>Q-FSJbQ!pS$~Q2_krl$?Z~=3lW`QQaG`c z9(Q+fz2lP5QH;gdYq$SB*l)U95CsaIj8%tPCJe`kkP#&citaIYa`DW&Yh8az9t01R z<}a^ga4n6C)@>aNo2>^K>-@?FMJKB-iA(IOR`geKjLCW=jspyaN7$NOxD(2sR%pNv z4F$T(2eP0XsS4#5p1+D~`vN@s%F!xQhQM_YMBO(_*Qs~y7bEQH&Y&jgci;Rd#oINo zw|h!gqdCEIV0X`-aUDt(*Gvht3^Yl$R4%#sKS>y6HvKmws{3az^AtQpww_yf{2P48 zDq)y#AX9qIyCb@0JS}l{C{>|5n>i@>?PaELY`^y5OYwqe!kik@x-YvN+!-yQ1|8&? zg(mYe_CpU80Q-hmSeH@AUr~9TqRn4%%Wd117&G3E=;6EGZoY8}MbP70dOWRzWz;y+sZ;C1=mtBU|VLyzL zCWK09it5mX<@|Czz?yJ=cK2YtW8?$k7R2?gXIMG<@)`Wphhke}utm3^(FUF zabwC7{8bNTB(_X4A^&22p9n|;t(i8|chN)8o~}6(KmaY4c^H9I+*R6v4F{n;^LJfH z+3tC(5A&M-!;Tun0u>)&i2Fz!sFe?Xuo1>HJ1XF_k)(dT5-6YvSa2Vg9&=}xQ&V-8 zj7Rwu&u-4=ixi`s4yY3jNpCbBAMK9pWY>r2wJqCbf_CBNXwR34fGQ98oGR$tT7Kip zI24lOrRp3IQLS}N^0|0Vs@BAf9Buvohidj%`T)KKut45m!ehYhyABV-MjIrMY9YoQ zIRT!ZBCInlHPJ;P6>};jl9v|QGi_IvFu>Mphyt*$RUCh1kF~QEUX^{YD7qz!kl3Y` z>&KsErbV8@FJ?$e+YKhTj5W>QEq(ZeFj$jq@&vFJ$~v<|6G(X!$|!tjrP)Xa_@5FR zC(8Tx7ewwi)APVocd0#$QX{N!h{Hd#;FtK`@7;*@&qFb8j;*M&zq6SYznrd!&$qU{ zyg;fP^^I?jxrp5I5#R9D;iF$|e3-(=$#L5FK-m1N=Zi`1x%3#Rog^8l zr-Gk2(`>w4xjwWbC|Vxfd7P7GKgl)l1=#p`P~f9g(m{RN+_9ziIBH81&ip_MCqcV0 z%TJQsTN4aNiMJK zgom$EC6r%Aa$wWT2dsD+x~F4y3UE$hU@-BWzVmm$Ti*+1{&B3U-ks#w=kHVZ$6tAs zoH}ncpj!VTRN148ViwNdQu(AR&QHWXH9!sLpM1m9aV-rx=wg=QGRU?Bxhg&+gj;&@dGA$bNfuAQu$@bUiG0Tn;hR&fs>zi}52$@qwesa;hZC#WoSeteHG zB8#NzItsF`e>n_r>J}IrGKdAQ* zw}5xSRI^yB^_-H(`zazTA>XSOI&CjLS$pqh80~QEj!bP$-Pj@g%MJ{z{-p5Ty43xIfvKS<#yiZ`CT@YTUZ&uqwDmJ0@Pc6YsWU z!k-9Degu2k(4D+dFIkkP?7o((Ewe_q8-{U8(`Z3!G>>oXZJ(JuB#ad(pdIdp=yh9~ z*ei|lVzN9FYnhfY(xa7%mF8S(RO!G|XC&O|DXgxW+rsR1lZ-1bn+qY};$!5pSNrKt z_}==gDW^W+Hlm43@Nr)d&>|j{klmvYFnsCmY@CSdwtIuM15lPPk<31vtj+q*QsFibHXmZoCP(hRcI}T$wNy#{b8(PO9l4%PdT2oJ{WMLl z3$Z71;o3ZM5ykmhMd+innRL1q`uFWHZI9*@{_y8Qk_qpTw<2TpuIN-FG6!+D}& zToVXjnGL`9!}sm9&e#%*7+{`{xgA34Vws^)$~Z*#K&c8W%j*+cg?|fA)uxeIpdQqJ zfVb3z<&D^yWQjrb1TcH%?ZH=d835P(*(9&$Crgvu(0ojBAB*z#h}xQ1P~DnuWB4fG ze-QTG;cPeF|8Vc_YD-nsrl?V~s6En~5-p{+s8L(3B4Uqf5hH|#nu(Iyt3(jH_K2kR zreLTZOX>;v-mt@ele>#j=JS?JdHprd^`@ zmhM-v*Vp;fY3K5@gxVjjsJFIkbq`8su(sRWw@+jrpvN%EN1qIGB3K3J>_uc-fnrYa z63vIp2o*LzUg&l@z^dOjGDSDMB28zQC#HJ-dqlgAy>9}3Z3k3e=sqY$F+%#@v3fCL zPO>@D67m!tx;!VV2KgH2iwMb@bM+5t{|f9xtYo4m32;zLZ1nUav;54Rt_Zl--otU; zSdm>mCt@r!9!UNS*e#YeJ3Cb>EdJDZx|yVub&7vYORFZM7*Tml@fk1{&i_1*c=Xp! zY1i-mj8E>(Acpo=LL9*Wf3@C~l4K16d%7bGD7WOJagJJ3QZU>i{0W6`^o0NmubgJs z?)k2?F44z?_Hv#(fdj?!%%PnMl;rlWqA6&x{W6HJ`WKP!V$R2RX1%=J6eHgxDeE^| zc@(hMem%C8@5dFt@;`Gkqt%N8%mbJ^00O_u82J1zIC**NF`WLbpU?UBa`g_ORezm<%~}Y z7!h|vPmx6bsm-XH)&+g^`(1omMoY#&%c@NIST`WDGfJ;$SZ$}`Id*Z2LNL7yF%0YB zyv&^UWCUb6b3T79_-e!C$6zPl@#3J>Ct;t6XYEk)q^daDtv=&>e+)|$P+>7O5pP7DP zrbR#ZxdTd)*{b- zlk{l!RkuMgAR8T@a#j23SIj$MiH#1{fP@SR}D7&`o5g_rCd+onzjOmSG`H(iq=H2cGow z94g}AQ|K$F^1lHZ5v4UzfsPM^JJPCScM+-g*ZWer}Sk@G`@Km z%i3Z7tJMt4?eB(S*MBEWwo)Ur2fBZmYb5Ro42sDwY5&|wYtMUgVwT+xKJoTcI6y%#^1 zPn)om6@eLCJQpdKfo$+}1n4R*E!pvUev?j``_%M5cB-cTY%8KbWk#G|mh!0F4HsZ? zS^(B;W_m`6s$F>uclr}+iCSLT9iZ5$^=k@6WKR^^Ha`Vk{SQs~9J^Ba9iqS>C!t0$ zht3W{^J3X6K{cHWlxwZkE`abJr_-jyDt6CnDM5_O<_TFrq#stL%Ax0+u3f>sGhnGEP5-`5cOVR675K3e_6x;}APQ|39QDq6Md`o}+rA z>ZxcCphxPB^Ls5i1-)$j#VL~qI$Y>cs;MjSIwI}|&=MckUi-FKpM-9G4Segzu591f z)&3RE92TZydq3hLza$eJ?@3MD@fWYj4fe=WnZ&V;eIB5vH;(Q=o++Lz$wZ$hCei!T zbFI^C#|L$+K+*bX^55S|^t_$DqePgfcnv zk(TY3lYa@cLD_Nb8$LLcVZ4lz`np0Mtv%J)bA7@g2fVSVdZXNGE9@xK_wNV4_crgC z8#56{#8RFOV(`xhqaf}&IHj%TULtTh>*9DuOc$0o zmoZT-@M+xky#U-z&@CZV|EUNuo9Z|g=xC!mma3S9?UY7<^fzVB-#}&P83(Sow+}wf zC_A<}SbgDFLJ!0Ebrh>$V?D2N+j+e{u~ZnzeA|u|02A@6kbI(!V=Sfd+n|%;jgO1g zk~L!wip--~I~F{#=FwDpDw3-*DEJqZ*K=OVRa24XAxPcIz=xa;5pw-b^TR1HB?*-KxhwwwNl9|f zV_)E|4wCNC<1HZ)DLpznU6z^@$lmt>YqAu*V$wOA2p;=yWY?65saUoACZQcB${Ce0OuEKo876{-Y1YOBybvXm(pt-G zZ)tZpCToT{&4u>yJH-zK#tOvN#cT78MA@R(Zam@bDWM7jMWwFj*loRHYVCSFJtT8mqLxTlNu2^H!qh;~>6>7E z_l1}MxiO9b7L0Mc68)+D3n>xmTTwBd3eRZDMEq41o={O1Aox*QbB8EMANoH$l+)I% zp8UT&h#}tu|Az<1SlkrRTXly`97Ch5IMT8<+ElH2%fCpRoPJsg6MrIiS?TSe!X}9}Bf!2lj z2V+Xi!!$|28zrmHH^FLa0Wb@mKo|#`%wL>#d*jlFFVA{YrbTird=GVd}`Vb+9TTIxRrCU@Kr6*FZz}Nl1~Jon695G^~~x$Oe4zj}sHgR4xOD z)-klnD4yKUACO(B^&8n*8&0P*)N^A?oWnMYy(1NIuj__Q$gb#j=U+asA*DNhe7aUo zsdJyyd2ZEAc3Vi*)2svPBa6(UbFbdwk)o8eUQ6{`FQpCy_Ny3)*i}rBhA?u?0(-a{ zRm_rX-fV;_L2o5V61vENmr@dorGTc6_;T@6=TM3A%Mdx}S$vQM&2Yd~;*4HeZSdjG z^K@d;xy^;|)!vmUUT9eg-pZMGP28?zdvk_$^f(a*RLP-&qwgT6P`~;gS+e%m~B-LQZSg*M)zO{^WYo zvUlFG;zHR+=``PulsJjFM>3FV<$GQsW3o8NR1BGm^{1aJYLj6NIpN)MC0=z&;YoGh zO?=44x%WZ&MO~qb&h5>2O5G%TJClqfb*Pet&W5i2v+k&+H(+*e@=Yeo|Ub z9&3f3m?6;m%WS`w6*dj0urhis13BU?cS=^eX-Gd}L4vJ0N0~>&~7pNZGUnY^X zR&}Qw+bd8U#mNN9<$khuMXIvJ66Vd@0_IK=ftqrjnxP#@RIR;LRSQcK9HFJZcjPp3 zyea$Y3?i_8=>j4J#j@?7Ffm!JNq91`CU)$GW50xAa?3`*;K|-fvP`{B3aFKh5+HRS zi$@>vZ=TUw1q?!!ZgW#$l#a+Z7S$uUlu2O#(;{TgGgv+U-cpF3@d$;qEl_h!4{>vZ zk3M366D%uFiMc7J=w2@IZ<3rcAukJ5=pK{J2^RRp@?UD6oBxLzX;0@Haxd?ZGncg% zmZ++!xVn5qK?;0ie){XFP=Ba}N{`m$Gc-5?wkKHy@YaXRO&jt* zj;(~pXoF6ppv`?DzBkGbc$0p*E$MbF_l4XT4Y9BLRB!C>Y#;bUd$?X#Zbr_(9M6ex zY^C4rvPxCgY7)F+Uq+*~ax-PgGjwq|x3G+AC_*m;JIZ9ZZjh|Xn@FNp9EU{g)eNpf zR%Ya;Xz10@d+h=+;Pi3^mT`bZF-_%~++z&*gnvqzG{?d5rODWaAmaYQI2!e&xqA1| z9_R2%Fy6714V+*ALuf-4bC@wT)G76I>SI1e+**=zdo8Pho0Ml-!7kwzFs_FxRfC=i zBP%$4#u#0M0V7;an-Z*!rW2k)t0}c6nR=+*ml;UsEKF@%ozgZVJI6AEu%%8xr=+&% z7|z0KT4LW68}TXImCERD%FyEH`t{&Sw+(?0+q8`8aQ1w0);9X_0mB3^GK2PPt3ke3a`*eDMS1alERI{_QjvQp?2O zS`j7AHM`;J#zPs{@!F#9$W&Nq@&Nh?HN5;Bn@NG9f{^Q-ih@BmIoPV5*kmYMt#)h` z{8HymSRoJNgCI+Jlz2RqHQU4JDI$Q2$4>6zwY9^PHt?kGi}~fH@{0F^D_|krv1~24 zir(|Uc}g!7j#?JXCN9f8ED4p{d&y^W;8Q}38nkilrR`@kK|pE?ONI+QV`;@R>&_7T z^gK7vMxpf&>14Wn#n`uF(#B*d36r3B^ckg*`z;gV!}3l8;!4%gc6vFPfN zEO8(xY&+sZIwZzTgq=b|7gxBUgA(+7&Fwi#UlR7Bfaw0cg?cX;L`&0&a`P;ir&RLf zY&_H!I$)PQ{6ta|?OaFr`t7Bevu||NQ^jvaVP#&WyO%EOEX4|VJVYBik~{yo&rdLB z^;^@DS#1^R5rX?U)d@bhGB$-~_4tGuqLtH}vum2%Oqa7`=A5>rutLfB53R|jP}UCa zzY^Q|UshttZNMBzKlOK`?-}~>U#z*z8VXZX(#3%O&NljeGO~)BSp4=^bWyWU>lQYz zOQ)cWE3s>cb0R>k-|C`S*IC;@}kD zgzgV1A3v2ld{Ie89-+nCT1fxZ72pk@bD^Z+Qyp^Nb|~cIT8tiGn7!7$^?z{9YS$Yg zl#c2QFPKPadzYueiIVwTKx~tbYpSN9(O35N1^+LqxAiJ^Z#XHix#?#+MA@|lbex?T zPTWiqlSKd#6dMb=p8A{AVI0 zssve9Qq!yF=g-(6aGBF3f_JE*y>kx6Po(f^kJvG+YrQ(Y?_{06mt!`Nje<5-%Ri=E zTN=|A@7*3wC#(n5SaIO@zRDn<@-}wZlymI|2E}*Pm?6|h7ADy!SGZytgkOdp-eSfI zx#uz(;rH)hU#DZV{B&V)2ue@sf0!969;Ow%=e=Hx@pCQJxVFQDq4YMmAsJV&Fqm3? z3#5}NW^OZ1y|f2}cuzB(O2<2MO~`F`-$b{0-=lad@%OWRnk?WsXRnspTk(ykn2IX&yZkA+Y}xzgwW! zI4dal!sM#4cY>7`ha_QM`C@m}F+cMVE5w{j0P{&AYiA)(>tkuL$K^qH831^JnA<9Rw{Xr*FyxuA}Un?e5GXw@sU}pq3z`86a&wo4& zyq|m7x&U&qXUz*{LtyyqROM_+5?69ZB$XvSYTmA;4LG^vBOAiZZ1Pw54zE!595=Kf z+MpvU2$@pJ^MnaATF!x6tUPnoyV|zH7kFU)a7w+1phpGizNO{|!$%Z$?aDLkbbkhL zaZ_^nXl!!ZXm^n~k!wIsjkh{n!LOut+pjpYh$!K5TuT14G1)`KkPv@C>&h1E)w;S) zQmyZy1w^o43avSXR}PMB{%s7iT|bhQ`}0k{$v{}tWYl!XoD_)7AST|} zAm;`DPtfiED?~f^D5#LqV*z!X18_-_q_3^FZaIBr8BX?>rcKG0wEd(-{|Vi=(~_k#}C)JfUT` z)O1xJ%@g3xpJ<_SyzqMhzi8$YG@6P^h&m(Lsq+L*iLm>>F4TY82+(C-cJCTV+ ze-5+tZp<&1dBT=~8tbZzQ|OzJAjYkHjX+1{+`n`%M(ZbLKS8aLZz$HnkPlNU*?-K& zi9nagjEhIJ&Z{vZ29RLS1XXJKCG0YydpE@+0qZv@xk7R0Gng9fVJlZSItO}NG&YJ-g29=dbsKiW(l>RNY=@eBfXF60TuR^qujs%hI7JV~rqf zVE7Y>>51`Y>X5dqp=I-i)5&Q4mI;}^8cSFG_Vw>uGrK!gN|YHZTJ2?di(5s_J3aZQ zfXyd54)g(4sqH%WDYwgI4>1CzVxk34*YQVal1vCj^W%qGh>jk())H7TE6G0F3CuZ2`sVvQv8C37%SwlKa^<@$K5_p6-K+mz;(u`{N-z!X=$m`f2- zl@sqceMMD{;ss75B*5<_G~}4FJ;uAc4I<)^$BXuf_!c3qs$)z69>Ju>UT*sq#7&0T z!|`0iyaBu#&AxrOX%hRQxRVVF(?=gqVNX;J93uqi_Um6Ou^>c>$+>Bk8}y^ zVx!?tt$3A2xW1e*<@2B05O6kjhHBl2Dhe&N&h^}Iyo$jKJ9aLNfEqZF^3!AG!V3|A zc@Pq`zywJhTh3g<$F23FP3>Zwx$;wj@|oZFfbhLK+xH}PesGVsMnm6nfx?oVNyj%4 zPP8w{uyscEi?a-$2{0MDDZ%{H%gV~IcGb6uL1F(EQS?xJ=c;&+e!`S3yB#gAw^XRy z7KMMh6DK^;&Ae@YrcUpxhP0FN%L7O()a>ymfw`548{G&8 zD0tW*yTucma=Nd_%Sz8jW?nyDw0uavWXdJcqRz6`J+0@hFKEQXiOHZ4D&P{|z4s)| z8P=cSTMl&|8)m84^6R>tUj36}g+)h<5p9x_NZY8Uc*4>s;jD`7P2Y(NNmc~ed=q9$ zi6OhVDC@F-!3mx5Uncw1GmmgQ;)0xCq!Qyo9J|y2;I1d~+>w?dE_N}iU!d4Lfb|0I zMu41(GRJw$3~giiwgj`8|J3&imSmrWH!`~~l-5`p6_wtko&$;Qmnqt^4R6|~sRYxX zV_ZsSjk^jps@~52q|-Lrrs*;INfR)sH|N}qjYa*{Sc>YhONR^*9nx!Vn^{9thw5Q^ zdTexXx=y;^m%;jLF8jyh$!PcVrhLA8-p{OJjv4rirOV%MgaM$JdHF_i6~>5FpMhQMvGTNLVRTQ3B(+cwd1})jWm0M&` zIpBWfeOu!g!6h6BD$9PyDP8G) z-Snc;0`_6gs3z{`J&yFCg^>#v9WgUhQ5^UYD(LIC*JqWztI9h^;z^x4#gk!-2|XLz zXN%6jO5PlPdkwfVm+da2SW<_yeLScSFSS$H?GPB1@>T#JwECUw@>XF`GoXa)`%-m7J2s8w zGTH(a&Matt-)Hu|#_l#423xU_?3T_T-@KV4e(6Aqe=oc>*ALfpvlF%`_r!{$WP0hY zA2VfB*hCio`45-E$B_>{D@>l8-MusxYB?MsvGy1s+oo0**X)K;UNT+P*;HIRLn?{6 zlcz)j|Bc`%RxzWgg*co=1_g_J-erC5;ZINy@!mKVIeos$Ras#@VTWoxnIt>L>EDHXl3wH;||EO z*l?Ps%762;wH`K~?E&!vlJfa}hk4K4kGX$6qh?ZUTPDQy@^;ze+3kdtl57T~-Yyis3~fW$BEfZ^wNHi}|U&JsyjYODl}Vv%5@LhC}*3vmH} z;urx}lvs!Wv*epB%n*9VJ@#Ro0Q3ZwaY_O_yX!T`9j~t{Z@nCk#P*5GJ9VO9kF^aK zHS*pSP87v-2+Ev)&|J32rm`Py67d+$qn!LggpgG>@G3-gdNEv97Jji+9MzfLW3rHG zk-lg1Sd3!s3iVhpq4@f^*_OtW_EBMuoV<%D**}XsOUJus58J=>GRp;hjZMnge&2m5 z+3@!qZdWUKh)eIzJ>JTw0#~(I%AuEZn|H45@Hi`OC2l6${S8x9ibHy?jk?1WkwAG& zgp2r(;Y_ckOeiED!Dtk4s6qZ&YraFn^31;;RfOW-_E9@^&}{ciD>ddAn6EZlQ2{T1{R24 zQS8r&I!xXoyt~43u^Rd#zL;Rm@_AXSS*N=%IK9a>`T3-)aiA=!l=mTqwqK6n%b|F> z|Fi#?Oqmev@3&e3G`?%#2Ev8g2P}3owQJ4hcc7@X&u$)EK2XL&e-M<&%tLn}?tZX126xK}`nS@}oac-F?)^wm zL5?kHK1k%_23Pd-LRzdVMGB^32a*+sBHgoMBEl658jwz3o_2CKdQ>qb@SGhacoeVe z2=90@evOXvs@k4=tX=t1qt1Ef_VJ(6vv8JavPM+wu}P?ExJnZYaDGnSu`0#n=Gt66=LAraGh^^iE>7K^%+}W_`+Cr3ll}d#}vv( zGqzySCiq}Qszyx>w6LXX=$&t=uaS=9x|JyS_?$bf(-~v&uNlIo@6sAjB&KSe?f8__is z*|ji1QNAuUR1s8ty!#8EcK+FA1EY0*jiBwgn4=%{%qzsyT8B$}-vARKWeCb?2vMtq zWp0*C>KWfq8hggMzOO;zqoogQyT)9A|05EEPPUk*|Mbw`dPpR%zOYNfKEO7Q8kikZ zbRfI)N1|(`$SXU`P0JbN&!9O8X86eo+5g$I%QW#m<9%n_;aoR7Dz=p9{s1oLJ~q3# z%rsW_{OZz?-Mqsa;D!6;j#~lBT8~^`wn9E&B^#wWjRrkL=P8#7$)-HAa&Wq3-ruE= zHB_+t=)opxa{l(n1`|74p6@rk##uYnULxXX-f`hs?17nl$nMGYK_O))qiFe3>yzDE zaoclz9Z^u2R*1K?nm~ra>o0ImvS#EIz=foT=*q5br z-RPK`XN^I-y@$Ap=RqaobNInwOq?qmVhDtDc@>&To0!KXh8DoN+UdEX+8t06K|Aoc zqUvG=ZV4qLn~{n{d8<*>B`^&Z6CQ6T_Ce56B++erhg7pC;ccVWB^7L#xQ#oB2u_O$ z-1xUZ#Hq~Uq)HN=Wz*wDhWZyOI6ig_J}M0R6orhEJro4ge&e$*lN&pbfE`t(B&sT+ zqt;Uoq*@|kK)D!9xFWcR@c@+Yuwqp!jKj{(~$3cwU6 z@>6l$K9@QH@uc(#9WG4$)a`AGq*{c=aPkMrcPGe->V^sR#Rz)J^FL;BSu7FvX}gbD z6L*{_>_ZVcnblGg^J-PRvm z=l<(1VK$uxYV3Ahb_|EVwe9nmg49}i`0X-*u1X4gsL(8>YlCpKhnqiZ%* zDTcrxFYd^h?xp;i&a)3V_B=hWVKo36jf>BqYLn=H$y4yuT#%(!Gm86k62$6UM0JMrrA zmyWQCR*Jb>Nh%hyD(|hS=t2vv#@l~?HIxrkFG0j>&4TF#@b#3JzreAQDK*ZG#Jw_9 zz5Dku?o7yy#TVEFS$1-#9n}#Imo4}4L=l;Pju$yqhFkKSEjzzLqJkHU62|kOeeth( zxYN$oqq`;4ble?#cC~!0%3o`ZFX$MW^EFFobRNOlDQA4q7BrpjjZFdbAIg=sPsz4E z-$9&?XrL73FVHWlaBNG0XxNRL(xgCNmwi2h#V)A@i94@-zi-fW)*J_v+V-XP-}~XW zWY3WP@oXoKbInq}@~KUa+-CsUqgCGG?674Sm1JP@_Dhe73773@6;Oa_Oz3BW$1QPw z0eGdt7XKGe1t>|%w;bhS0D>oD7tg{HGU|V|n2rrn+{Sn0h|5xv{dWX?iUxqh&F)*C z;n(_2_#F4jhmzPf5Ekwn%Sp&sORhTI7-LAm0M_Dxpuv>$b$uU4G!z0<9`Vaa3cS+yJ7y`+WpO4PLX`>cAD6C8Ey;RUVT>VOjWWD6U*K#Q!W; zVz9qGL(lGF;N=n99_z;Z)lCB3FBX^r@i1>gNv57}F+%o#l6$3X-X@~M!seX$^^2I7 z*I5-iZn4URtSl~DmP+iFUKLvPXYypHd|zbvD#gxjLutLr*^K51|FT&o`Ci`jw*O<* zUCUOH4Qm96MfaV18+uetoJ|-^@0K$xYrnk~>67<{k5jQZLc)pt7}lQkQ0(%I9ul{ZR9_s0E7G`#K#=1Jc!>flq~B+; z`&}t~LUutG;Y>m>Y$xxf?Fn@{uXa?CqCscN{Xxw;x$)afPwt%W#APTb#}+(I>cl^P zj2=%*#FZ^H^O!g2ViL&(1(9PK|Ah16Ha*9S9Jj3v4r7juBR0;Cfd*qZG!;nX(ymV7 ztGV359uocB{M$3XgMU1E3Ibe0zQ3*XfZU~h!#R`o?-a+Q*CTCFl5|JP9@2?pMpJO9C#94fi~ClrnF0 zeDts*o-CGmSd_e6h#J0B)~i`#{X&7SG%4WmHf6t{a2G0mJ5Kbx=B>r% z_`Ye~{O$-5Ch2?J6=f(HpZKqbYwSr)hpA}N&~YJ(2`?J#3?M(@s9s#9x&Q4~J4yWN zD2n}-@Pw-Tc^d$gNooms(!cr6KS$!%_#(5elQ70bjIyC6|0*V&u$zaS zY0R_WDTij8xgVc9QB8c{+2r^C%3jtCgzQSb@H5sy)gj^5R11@^$4dpRZsqeQ(id6J zy{Fx0ZT3QCf*0u}b`EZPYejDdz*Y;o-V+6QsV_n^Ed18<7^)brKsI{F@jte;oA+k< zbK2`+%gQGVE;>%D4t;9c2D({8yd>KnJ;IC`-Pq|GvdV*U^t;+wNa@H1%w)C*P#Qch zdNs^*=@!F5&2|0v+35E!ZuP<)dA;;4V^#f{sBEtK^0#-V6t491ljJf)#!@~E(S3Nj z6RQS|lxE>LHVB?~x!5a|53cGF$t(B_sqLXnkqfk8P?~U+Wy+;jPR*Wk@EgjA#{t=B zcQU39Y@g<-A--4YmYMPOKZp5L2eJRzEtTK)3it7HeNwAo$q-#o079u|mP}{mF>VOs z%t8h>S3TIqH1>bglH#%aw>=DgS*#ldb0}^o8A-z5moCJu;)kAzoKS6JG||5MRHkrc z)V-0W)tZ_+Blq&NzWhnQn7h$>aB$(au}><&Itx@_J_Z(}>INhL^Y&Phy;@z?o3=WF zK_n0`V#g4@Ga+iaM}NPZfyb%PJF-#$Z1G~rAuTXRF$0#454igtUf%O;cR`HGKVTnR zbElql2=K{{-#m6*=khQ8|@YmhCpY zoOT|&o?S{i$90@EbCcTOrUD6Hk}aknZ63w9S95(yz?irzi}l-g4Zd*9Ir2S}y*S-^ ztvv~fPOPi9ikzuD4^?WBJOAuURaXQ}#rljFm~OZ%B>4A2N*ks}Td@2UOUq@*MzO=j zyvE?R!p9aT%9IAYUTmg_9URNl!hJcd-eS_}X3F7&(fg1c>JBfD-)MX4FdtM?Mc9*; zg2tF6zWY;mWrIr|tqpvBDraduxiOJ624 z(O))rvT01?R>ioWkYB{E@X+B{BO*7|N$z;g(7y(A8-0H0kle_iK`1TEsd4(!s z&ekK4OUDV^@(qRv3w6JTwW!7dta#O}S3LUFe_+DJO^fN>1BJwo*0GeAbjqW&mP?4A z+NalLK-$3P*BQF^1#*vW64%X0qQh#r-TDf0>zyABT6)NYD$^>@GOE{Fr1q7TTYT?! z>+<<)<2p{b6GnE^!zA2O-q!nxGJ+s!V^KSsXKkTQ=Et}xhWKmD#ML0FqBGMg8VOEn zUbP;f3$#yA*p7p;uR)8x&1}8mFY7fq9;}wz5pLD($;bq&Tul}jJezu1tB>d-$>pKTs8 z$u2$l?U>(CoUyJ!wPIRD^A6uin^o^l9?x7nu?#Pe58j=pIkiCX$XDv~q}4xVeg{~lNxR`0bpGqy4?aVs zTCTbCfV8z#sI*%3+d1?S(i=_R$<+TaF?P_oV|4B8v|)8Wgih}vuy(i3Z)ZVc_Q3!<24ltwDK{10Tv8ez*AArKaqpHd`C`h4w8|tPNbn+rDKtsnd0&d!s7z4eMoVk&;>2(*AImR+qr~=DPl8 z7BiW_;tG{ilXKwj*~vD`f4Ds_!7>$_E#w-B4RrQAAGg0IC{mmGSuf62V)96U*|q2x zWEZrh<_XS^L^JotCvDAib?D2!CbJfRH2H}cPuBAt&iFx);o(5)y&Ok8V7Sg9E= z$&*+uxhaAdJe-HyJffW~iq4|r#v^t=IuV(7+@z74Ih^pJMm==Zf+L0-DeI4i3%eyb5e=KTX=<-3p~xkKoprwHJOY)L$a z1=W6PoKqKZ$Te~1gRqIMYL6Pq{)CjemWs*R8bz|{Hrv1abt$>lprwE|Puu}=>38f! z*hMov1Hr;QZ0o}{a_rF+EQ83OEj~S$+#gR(tKBX-lvIO;c+^m;!9TR+qEQF+`PrhA zVRsd`o$o(3GWWH0PCD&1*?eB`=I5PF>VoKi2U*W;#7xv3r*@6fGe{V(Pr`w?Gn+q~ zkGR{TXH@1x^P#xC5A%=&Ns`O?ZOUz%mE8Jo(TRNdKEwG^zNVq+_NZy7so^I#nQkQ! zTggH0tRRSpL!rX->kP$e1L=P5@0HuRH)c-KOKWG1W*+Er{He}Wn)Nr|R8dQInYuZm zvvj;X&#vxOm1mX~oEhVXup=uvCZg}N-|h`QKZ}{T7=U~v)e=OJa0&3casT#cav-VH zry`sE{nzoyYl`a*DUO;%Z>(gO$9ZHlC>XgXH)eium&Lr#qjEj0V3a}rOQR{~s9N!> z3QlQFZhd}wMZf($VdkJldiVD$b>Z@{=zZwADqjgPQvxNbKuY2Kwa+Sq@rk&-*;OJd(lr;7Wy#Yh& z<5J}{($-!L*Zp05n-n-p^dvtc$mmdRaQ7x((SGS#`@tfZA2l4!losF_47TVzhxNE^G-B8H23R#T3sozRayZm6Am zGcUNm9hOZPCSjNyM!6+Qq0#C+X$;>@R1)Lhh=2Hj zux!*wT7@K)0?6my`NoA#ibahhinXy6@F*LS@`lOQGJ5$xF#Krp0oie1Shd#60bT4UMZRz`kCTZ*X@rp9%6T6^z_H7|w zS`G!Km-0kVxo_eleiZZ|qtRtW>=SnLL}vDQaal)ok-#bOEgply;`xMBB9F~N&IxQR;E!QBtZk$@H0uF$fWS@Mm_ z9k520%vpQ)mq#{Ic7k`zoc9O+9%SUTSl%@c7aOZ)sShH|&5gj_{CJ%{wpVeCeK#vX z%H)__nsTqQP~OZENk#Ng$-CsT8$6>w6c_2UekSJa?lZN&d+3I`B0j0gSj&;Ld*))O zZYZ;|ManVvQsVHvoN{(gJ0SSSW?h0t)=Wbb@mVX|G)d8$PY*2r`B4AaHFL){MzUZ@ z%})9jpm?aS_9f5##t*MExF4AxUh6qmo)QxPkbt;Ex01Rq$*^NRn(=yZSItBR-#57m zzJh<)Z6?uFFSmpA<4oPjR2qx*h0Sp9$DyKJxw1A3MSS4~!P|&T7sEu-y4~kjeGWmWD7B_hzgIQfM)&FSxT#t8&z1Z4zg4r6D%aAZBg1 z$ff3y7wJ7~tXqZ;`PHq&w<;<3OM)tZppeL)j^n2F;Ksf_an9pzkSuZc*DeEH?0bV6 z5N2?k>AKjSeVX9F+T%M74dpW{J#zW>f_(SV7n6B=2aiD)^WN=Zy9`5VvUg^B?y309 z-47@;2do`?l8#xjfyAu`LTOWym2j*@&(IoXX&qv2XKc|&PpAp{dQ3BnYja!jSQ=p& z(z>%+SEV;re@u`YD5+H1EDTzeo_^wM`*k(f z=UhQ9*!r7OV=yREGO1?&=dHJncg^cYh#NIj zjg~#jc-NHh!S6pDo=%&Ua;8bF;QM<3-#w==Ki-oI1BY(^1T8~xdQ*0UD|c;-Eohtu z?1tOQ>c2?y89HtNM3(Ag?so|3@UxqI>entBDMlVtP#G;{jA?q%#?aXlOi&ku-6WiS zg{-}H~0IAtT~ z2z#<}z2}@urN(kO zUodBNR}6I~^n1&NznV->XK?E%-_|Hnzqoo1ZywQJo`TF8nW%k0F)_18bg|tK!C~(e zIVfW3+O@aR6(NRr65?FFWbfZbDQ-~z4ql+qkRxmLw1o9ISWjLLYZJ89jy{8<4yl9x z*{E98;pKK6%A%7L-0J;>kgEot7e1}DUU}KqHV*okEp`Xw^2sM2di21hW)E3fQ1(B6 zdxBa>CYVfFOsk(BKm6)aR<+fJx6`IV4HBTZH~dZ=ZF18ON=N%r z_iviHWobL4Vm&N?J}YLnsEz)zkd^Wn5}V&)X`|YD3GJe-!ML zV(clw;Ilcj#Zw$zBe6@ zPqr~86iCL8 z4ENc|%VHZ$--9zqJVv(dZbKWF=ce^U#zjTOCmiEj(+6`?P&X1+Hp6sPWft#cBTnvx zmd%i$7h=R`HnglA%xan+*f(4oo6rYL}4IHB$Fm{#))FK>OgPusgg+z=xpA-bX|=FN!D-zoy5-u z=h|39YrYiG_J0|kcavuJ)TMC`iO{OkUU612t0&p$JFhP zGwS+8rUFl(;sg2_bje*o^@L z#H|HN?zw5WYHK>@WD<@0^#8OOk;TUJu4+Z{@1Aq&1_nYk+M&(m(yJExE*xfL^t_7S zn1Es@UDFf4+23~3M|+v$^mH}WAp$nmBV(mm1D0t30kwn#Pq~O{lk#p>>;4gr0D_L6 zAL@7`2GA?#N1S4OCHcWNhm(!Q4c6B9$VM?W8JCggO4_q-_y2}XricnxpQL$G*M7;% zGYw%Sd(KI4yAler6xFHXj(MTPAH2Py@@qc^yy5b^GX*6{Rn|pP87ya^MrOTXN}8US zA^0;6sA=*8$|e4By1!}KC9H5TaI^I97;kBBUCw}s0A}$y5&xPj-72yOa~fd~{PV-y ze@>&i4DF}d`C@{n%%PKJ&*tIgnlopwixFy*Uxc`#fd?8w@8;f(r-}zYfV@D)B{aN> z{8`rI1orHb8ICbW8oLe@IV23b$sY83oYWu#DXV3H8Jmjdtl^Dg_!T|V{dAGvoob4U zf~PdbCPvHasn1xGtX$p z>AtMo$y`XvcX{hi70 zC!0}D#GMO6GD8RaaWzDp84A7~puK{|PZgyT)D%TF<{F%@S0DNW zev0)it_JNbL^G)3q#~1{i9iT)vuGUZ;+dCn3ns`-53*SS1YslcSQwF(Q3LSSe7d(# z{gXlRAcA>q(bO}zT+bG32}DCZ8JR4<9wO^w^(z>65UkOc?q2<5^}@l#YKz*S=auBPg7p1}Yn-ihhQ`JL(@u*KT){ z!M+~8E@Gb#KId2EFK#wuh^a106x%8AW)0FXgNGAAM@$wcknY8&>5Fuh;C8A`)&d6I z%ZO|cBMutplA3#BV_y{9r-wT`3}onZ(ToD$md2t)2X?i|$iQ`d-kq(WBC{z%8O|)$ z##_Hbb{h_L>BVm`t*(1VA8d(Np$Z4GX{p=aameWZiBbJOE>_iSqCI!N6W~r@{|742 z{6S0bFZFRx8z#cT7e*B;hjdQ=(g&uvB0K*pu+i z;w5gLrg-O>Tv7;ZyhxXq=$gtpx(-Q4qdWp@hbt}RR6?hSy+P^{r9U_Lnzs`0Q-3Y1c0r5;EYSm49o zx2f4UM~7HGz@tYPj>wjwfR9!zE^9tOLX(9=}W zU89^};n&2z9+GUjY6hJ9&ZsnrcTgVv>O+XLp7xRZ1oV#)Ii@L$sW@M2$E0`dUjV}6 zRu;a+P;P*qY!XY+UG?`gnv0_=53#I!s*q`!kz1F|{!H5!jnkjEEId3Z1Nxy$n@pJm z)%8$#u>{?DW=S7hcIsJYZk+82+ykuU6i|?*(f4J@2LwOC^uuML)9-MA^k|zCh&Y

w-r-&OK*6%lV7DIY(dud@gcnn9@wXuqfOELns=GJc4K*Mgctq zAhsrbz|+2WusQ%C{9HD0NDLmFeseGsCoxa$$fdvt3Dx#dv>}ul86dckm`qH8XWsrN z1QAwz;b?k=h>Y=fpb|(#4i0-gDWzy~ev^{W3_zSbm02~VLJ@vF=26JRR_Q+Ii8A9z znlEX9;;`HQo51jYxsVt4Z3xT;;g#|XimY=0IBnW8VvbynU%oghVvW0uSobE?dd_$w z+c*%mJuj(lH{=vOWcnuq1%aJy86uSKqMpE>C)P#~=Ar<57_z_8>kEQ>YTpQCxT*J_ zKeR~+rta%egiB0F5keWQm>GSZzh04Y_nj*8c-M%3vE2lK4OUjNMfAsutub0)A53H& zEyc~7B&D#Yovp5?zA`cG0FX793vc00*E*fUozvT_I@?ZdFd= ze9^{dL;!a)2;tUGTb#b+Z4HjkU=C+=$6?Nloi!O~_{>JA3j@aVl2lG0ObP3pR&*Qe z9E|-kX`eOecg)K7xl4dYgyn9-=&|^r-IZ_U^_&muv^81}AG0Gq6lD~@6rEtnL=xi0 z%xL~i6s6FfrtfC5+5enIDDW>;6f=c)3Jd6F9x!Bkh5r??2(Y}%6;FMM{?_*eF@!g~ zDO!{JF={3f{^JA1a7=3U#?xEPRX4zWi_I}(a+KM-wki!a%fvJ&JWo`>=WJ>+V-)50 zcV|7%_C4k4;BUxe1*N0Fo-r449ihUH{{ zKyaXDh0WIq!LCW>Z8gh_Q$Xg2!{mnlxq;2awhF&I16}Id@4V zO}7~`aUV}Tm3*4lqW1_PZmxGuJ!ka`_c!38gyHCtUye7%9@Rit8-~^+D#2C@7xT*k zPJYV9zd$wh!k8%)AGv6t22ACfbw<|E&f%AXL7@6)T^of0K`jQTkT#xaC5oY51kHpn zOn!jMfiUmV(Q+}>x;ifwN;~myDP$sE7fzWb*^KD#u8L$z;Z%2HGpbEWhavsNDxM;` zE?T~YpEcO^YJJNL>){Iv!kMXinVM0K9h9$okhNsQ%j&6xML+9y2N-qd zOO2yGirzRLs@Au5*ZB;NTW0vONBz{cIj_vMdFmfSBfKBM;9o+)ShV_RGn65Fg=&4| z*hRA&*>+}a+u=IY#dA(@u&kUk-<#BH{&Ff|LOsO<37|;v5#@z^ue0xczR9iWHg|Q* zUEoMK-va`S$pSk1v(8+(8Vso>k}7omzW!=;nuu1|hQT{!HE-jHz=ogf&Vg@=n136~{K1;P3Mv0g z`6={NKCWknwpP{Mt81gsE*I&M()p&cA((uV@vH0C4-)GRzXrLm?2EKiHn|@iIGBp~ z8-&yM|F9;N2vo1uq;9_R({B8t!)WTrEinW>9O$iZ_?|9kM!4NY{?gEatfoDnl5XFy zyV-={ucy<ETx>V)ZVF${zjP*b!&a6^pB}2Q;ap2f%0m$@o&T?Hfv*e zKMmA9(L#8anGN?B-2G)o+gw#Y?ArG*lciubMVp7QYS)`hCQ*nEn9jWR{+i&!U*R;# z&bd+Pq0u@qYMI8jo}#DZcFY2huu&|+7nK7gOelG##X*%d|2f}sHmfxl|4WE8w^&Pk ze{tQN^C2FObBfAOd1ox~XK7+`5Y_#gBnbVde#4m{nLnSapArBR(M6OX-F}rtd=?!Z zS+iH)7NqcH)hJ!nUDEiaUOsJFSgjaejQK0XzFsYPm$>V^Ss6a`iHoDD(=S-QA}y#RHp8GV-#fn(7qcQGFP<#ARKSU($601JUN^xz+|SPA z(sQ|AReC5K+xpJMq$DyElO*c}@543Vm0U3dJ%>;BzwfeC2s*(guwy?^z+H zAQ^8nOd3s?ldXoY1~|mibtMkn>Zuktra1UJf8^qS-w=@8tSp5O)P+SGne8ZTG+q1U32KCbsyGZ+?9=ibjXM}y zPG8ySYCk5-h+~UVW#**st@f|nB(d3@`Ea#$_KV67^iN)YeL8l7Ww9rzbSHb8GjF8* z30vjmTWbSQ_)y*Z;i2MfHuY2(2RDWTBN z`AY)F$iUhkg*>07G2itmqn&aCsXU>&-mPVGk*VEx`(M~ca^oucfo;Cg3kh4Y5`nvy81XU#P}a4d##5na#}t|^ki^}{mLRbWJ4fC5kP}t~K`ZXGgh#K-ACQp+ zL!W(UJh2{}A@e*grz;LHTt}wHBlcLR3EIcme++Ek6B-&4{U@{_vc&eWA8TY|suU}0 z=!$YWj+tQGBBm%4*7&kEA`Q3wvrL8^*g-E=aGZ#0blsyb_y}~;$&1S+F;Ju}i{o9@ z?{X|#f$-V2S9+=D{=OEYdWqtA>v>IJ8@QAKOB5}aFkv!h-{I?E`M3Y9MQz9u!CHc@ zVBg_gk|6SW`$wH(P^4zLFip*6VCjdn1*1n3_=1GkefRJsS%1!e`3^TVgA8HW)Qb2C zw?~m{?sf4GDVp`KL+-5o#j#_L6dz?hqvXq)gIP!{i{OUghDz;%4C3=4>U8TfVlU}WM{)F=*&Dexn4o4|N8jsC*bexIc|5i0#x=mNlrH|f$<$UN8mgkaq( z4c4_CmRPLKApGCn3$CU6=3FaVwl7E<@ud?aX)iZ#S@Wkn<;^SjYruoQOD$;}rr15z z4lzD&N**q!e7~fTP$Oqj9#J^B+utfy`Pa! zXnuWRCmHw&P(-lu16dak>#I_>nnn!VS_SQ>Z1$kdtzw-mpK6P~KGU0Dt)QF{j-DD& zd$y*6s=_VV2Z5C%TSN-bT_K&gRHhjgxDqG-;42Cf{FCLD5`8t35XK$ z@e(IgB(pvFIYhF0aCUE@W~pOss)UoyaI`-7yBL1zl~&<2#NkPr{N%rrIBW2KLm@b& z@au27KdaP8X7#P#XXJGBIPrGjUX&Yl zdhflfq@!F3djA-xy3r@dQt~>IV-2GG;?ly!hyG&oxu&56xr4dI@bBv((c5PYRtc?a z!sT(_N;mJ`{5>|~-PRV`5*=fLI(O+;q*lG?2khC#zBw<|Lk<@u*4;tBt*HB%3e_I>jt zSeIF>C4kB_V?LdV@rOyV^nqFG=UJM zM0jh#_0FYK@uWY81AW`3N1++a=>yo-kkW&7{orxu!;ayefP{~tzOZlh)+=LS(-nEE%n1qDf);Z+O*l;oXQDIJV>VrUGj1NItb25Rj&#T_iGo zfI2-|G3*9ZeO12btf5`dc&oI)$KK4iV??NisS3n#ZWb6qg~5EUG6T&t{;={X@xK{- zM1G!~z`952XtZi^8TXNSL)~+__QgAp1EMt(J?6!%HDy3d=6G{<6gYC*{iA zVMryDIHR-ukeygvb8t$^k{5?02q%668S!u$0Y z6un8Q@0kS_g?5u4_PE#uGc6@|zju=))h6SWv++pRyRWhs=sqDA@7%xsepBgrLcH36 z_KcA50zPd6Bw5rHgz(jSK{(!_cN%dUSga2kQmt|O9c-a_>h&O_eNn(_Zx8YzRB`Rk zk1YZZ3u}*b2HPa~-;{Yv|DjLdb~UqY)?UretKw!z>2Fq@Y8bR4Uueqt)*DrPGj06` zwstAmG7>CMWy# zt3ZEu#^oYO@CJ#B06d&i26qxnWt}<=7&9`Yd+n=Ey|=QsLWoNnWvD4V8ZYC6<|O$Q z@0-Djs|tNX85Zzis<;xu3K1(WVdbCMPuBoR|L-8p{(lD<;`>X~^lEH}?fz#LIK@WJ zqrQ|XI$HFvZYut}!bLn}&-6Kan|PYgr4%FztVC)mn=jP-P2paTY&O%iZz*s0^3MG( zOE@v37M~1YAQ#a|oP6j{>8c1@2v9`v4l>&;FoH6S&seCdikkwv<5etfXBnaKtbL(7 zMq+>W3+whW|I{+tV0mpgD%4{8BM8?+|E0I>@3WBxA>IpY0QA_pP2xhiW+-uXzcN!)T!fIn4=PdL?Dm z5|SK6s%I9bn;8RBEL*AOf2ZaA^~@cl>c{`YC*?(Fs(dYJ=s~I3oR*zUSuZb5&nVpC zUEpc~JDMVh`Wt0&FQ$i+?i(EYm&n)^XvMy&m*u7|&tQB3_%N%otuvHh!D#(qMDrkj zKS$0#jWYEO{GK2n9}K z_aOB^#&!+M4L93Y^q*QQ*~MrN3`0?I?MC&9fV5%G}qEE4Nk z>s+ySc)e)X-s}@5{OR6YZZDCrLCRrdWAB!s(tE}&A+R&m$ns&x_Gd2L*W#AeBcucQ zFJ5VVRI~1QD8lpBUB;&>G6x>8c2)Ah;VV!q?f_c3zw(9+FKX>dcN0ETkxA4I7hQSg z5%JM?h;U7Oh+TxYtHLCY zg@XDDFiwY-Rhbt2qZm_>$QuebqfGx2vY$iX3@l*~=aO>Pm$;|Wp%GCqCLQY9hmqTN z(Fja;)lG}*e~WC+nZNwAt`aZ-g~wQ%`=}v}65}o8+ihpw_GtT+nUFp=^PyZe*8JV4fvAI-1+b8M>7iZ!Zqy|NK2~Xz1{mE#N{TyNN%|n zsZA}D`V#i_g*qYl)*;S^kP)vonKU{t(P!=UFD>D0wI3e%qm5Tu`*X zwr1(1vP#R}PTZ`o!LU_l8$(m6_tsNrT$Uk$PMW7Z%VXi!i(R5vH<`}Ozx@92M+V)j z>!jn<|L?dsjW;#CWN<8FQgi)>`|4v;!!;p#hMY>6IiscFaM_n0B}1+$=CyTF|a5xn`xXw&tlhNHl(1XL*VhpYpA3 zUpMXrucTbA-DtH|TgRNl>Blb?t2qY>q`4#=f{Dmi`DvsjSP^uxO(B=2_O!QIGwu@YdOyOVH(P8Z!<^VLKfw0 zEefM>l+hPIx2S*$N844|vkR7#@0cX4q8^2fD*(BjiW*v4V#B`|&BediMT!4s0O!|u6EGMPhRY$RLRR#@{)5go&g`*6;AlQ5{>!18 zaPgnLXyG1}%~I2BZB&DW{&%FKM3=FCn$>9SA$T(?@k;5lKBVD^Nn?C^PpuqMMEr7m zCSU+Rd&i^0i2!_YWgShh(*4zEB`~`kUaPz50EF#PbI1n=`!aT(cD4>FG=2l%7QX!5 zmq+dS==#!NkBt@QerSl`#FhK)_fa>_dZfU+bITjdFx$$CBl(Y+7w&!2KflICBid6# zqqX*}OscqgTMa(4YJTF?G7#Z?Z}nFfYy$PdXzRjb6a$$Ho3V=vpiPr(dwPqdSshk>Bs@-jc4F;4 zFl{Kr7J}Xym@9)7I!7sTD95cVOEKI@>`YJ%G6|I>ce2sxy%3*h{fnV0+P?gI`o9iJTW35D4O z)TI!TiL%!W0|sL24z@qv$xO@ikwxNlo)=Cpb}jGhyATqmL~ETtTKOLy=@HB-Rg?mIEZ8QQ0$5`1-)p*xWeU-gRKFIa`{n-~I$+gRff;>k891sirchvjW z4tEra&%A^kI3Y1e1^O;Z?O9w-4Anobc%!0nA*nqBcKZ+O=<$A0)&Zi=&!YPxsMyjwhzRDuBN}BGU)9BX#e1_;H&Z7Fjd#h{ zf(F^M!`E@F6Y-8dATN{&77fgY7F%QZ=XHKna&e;nltM(K{PM44cSAk;Dpd7S*tIv} z-6&o}pRsx)SHI1&4S)EkD2*OVlEqYqQX#vV?I?>yF2ZZMyk;!_q&f={slbcFL70vt zc19+4nX>m)Yi4xshSm>F$(Rd8!Eh%dBfl+kdvAb z8=2?$$*(R2F#Xk4vUu+gZYQcd%S%thI5O>Nc6bjFQlxF;2VY}(s|a~n^F7EW^>$Q} z075qte~SBZhe0B8q~9=fFv`X_1d+_GM40>y34k7?V#=@d?96tj zhz0w8ea};L3h!j&>E>zUHzdS0vJhPGCPuybT6AWO17^(q<#AED>BS|c#I*P<_v))u zs&IN`&#?cfG$W*IEuoWG?0FS)!E?lr;3#cHN*Ud9?D*VjgpMS=bcyYl0~=yll>$>6 zs=DB9**iZRAb7g3rmBqRDpBjojPb7Ephem1|Y^I&mi#| zZ^!`YH^y!KAhc#>iC_GQl(sZZ+0AWgmPe)!e(d@XSmhL4kL9lNcdz%i6c(4gWZhDx z6dm*64$O8W|3`d2uiSUiVU&0ao)ywWF*$i;hDsTip1g7pt}0TiOU++&9*ED-cOn=u zArpo7{?jmRr({5MRP=xA3C$PBu>QyFT7i#hc+16bobyXC`!axvKPxax%pct~PP+s+A{I|}6;F17Lj$56Uk1jIlx?X} zvP1538{-H$oIC_zq#+lfF@QjbP#5^7}?h2S#(lBSew|qNFm4HJS3ph3StQ0&ay!t6O-J z&D*K!A~k?`8JUn(Wy%uZqtZFX5eb`)&+N8j&kW?=FIMKU$b=Dv}rzxOn&6lHnZ ztM*)a&S_&BBX_u89;J@*q^T^a8PCsOIEGAmvn>ywe0VE0YfkjF8wV^avAS%0*vLq? zlnT{%G0U7{HMZnKs-xbdc@3^_uD*F z&XPR>jr;gk?h6Ig-5E1pi{lxw1*G~yHyiU#< zs^Ld+3`%03n1vP$vepe@qI) z)!C_uPvBx43Qz$)p5=EJB)1qYKnl5uzH;{mPlJ3t{m0|L5Xdj>6}9)DuTm0hO(@Ag zxuvtStUN(OotlhIrIA4vb6@&@8K$hT|LrhDZnLqS6}Qcnu^bE5zNa4cH~r+*pVFOZ zcECArQu}y4rH%v8`%K;3MSXLemQMYn(CE;jv-i~kLG@Tygr#HjmVL92g8+G4uQ|#i zV^kyB2F37Vv-(}DCKVY~wB(LuMfvjw9q-^aU_qmg1@HaSU5N8o{>@EzSYil-^ zuCkA0P5D9%TU^hsO4{;dpKCdFFvnSkw%!wIepnO@?rLlMz<8GsH}uA*`k*P}Lyl)U zZOK;-o;sgYpgaX_bkbQ!ODFu)+VCvw=yuxvT;&d@8+*P5LWMv3wTNoqF|F=ppdM#K zGAf$H&og{+D=Zt2F<(=Dg6qf)1*8dj^6Q5|g$cIojD$XxkPR0V#?z$G^kcw%4DJif zE|C!F%_gx7*QfIX%~6VFWoxJ!S(5GJ#&Rmhkj$uFZ>l(nmt#K_$!Nb5=Ptbb(jjG= z5XDIo%+IRzHX{vLij3P)O>Z_9aeW9U(kZi0l+Iu1Tll;wEcLQ~ZK>88h?AJWa?b>G zL7E5WCZyjr65vD{6yD%(AD8^tAHvy}?dseA9@5hj!~c+P;Gj6}SXsRAv{HrkDQ3un zKaEe{@M(OC0o;7^mLSzE_~5Qd^S-q4wqzb2!ab9e`*8_hG-8pi6EUu!Fy`xHv_!&m`xKqv1Iv0ItD&PP1c>r>y%o zf~}c6^)T1E?y*`Nh0-`dYKM09Wg$Vh{R_1mBReO}Czc_$&OeErC(IlRUbI`V?^q=? z_J(Oc@1%a5pbc_HyLkMccYbt&(r)5huYDm&J61oGH;9cpxn{mhZ!yp zmd_LeO$N13MC$jaP4#Cl?@3QQKK>zKZQnEF03-co?x$&l7qEC{j)>Kod5`+1$2;B{ zqOF@xgL@_#{fheKDZW4f^xH)9yZu7J7)FKKF{ITq=BX90%wL0i`Y$-@S18J3izmwE zT)$F)za`u&?{~68mh{@Wm&I)sVaNg><{b|U$$vO34aU>ZK%t(MzfY-~W&hl;&-qET z3*s5Sv0yG}vcYN9@!+vWVEdnU6mKFSJu}CuC2YJ0Oqri|r=un0&yD6}$b)tm@aE>= z+aFA&zP@nMp3?vJVS>&(DSm}Z>vMN}R*hX;Dw1!&Kt`jHrrC-K> z@r@53P^vQ8_*gf3k!AL34{eh2G-`9TG;!-FS5(RJLBzWv8|lCT{l-Bhelzgqh<03w zaS5poA(NqvSwGd6#p|kI?QX@A$-WL=38oLap5zi1HX%j6+$Bx;92UFR>U>G-hw-DL z3-anjP@AN%+GJqQ&%Cjp@1#OGFmLBd`O#Vg^7C}Ktl?HWuBlo>#p0={v0#zZJeh-q zYTamOqP=2Fvu{Z$2KE-ikI|@cBzWVH8*iQ0je^?740z6bhk`_#6MO;BHd1 z;A?wM&#TSRXY}`GsHX&fnbNIjociOMxEo>vXLSCMU`HqDFqOfcGah1!56voTexASj z!h8~KpG zf3}6pep%Nyhg(0-&6<08CR@%z5@dTaifiH9!fR%vN%3N~$3H)?CRkhj?OjNnTJ3<$ zW+ecJ&8*gGBC-Ek& znu(q7mJj!<#sXysdXD%OV)|OGP1ViYVF5KQ2Rc#I^g)v@K^dRutYjNix=F552Fp`E zgK8r&U3zq$IE=Ef4G<``c3@1-^6oBV8t7M69V5DrQyBhTnwU5d+T-LgWcouwsA)9h z@1=wc%h3NPFpmo81Z-yhS^CKQ592gOYqkMzu8WS8Ve2|uh2{52|Kt=@*|$4zW5a{% zZdg-s;MOCTOPO%wrE3J)hCT^Dt2A9f@CU_J^El8S#riR24Ivl;{#5L4agn1uM7!BU zH1X_+m#5dLDOeUL--pd`ZAA+&T@TK{{&~~;+}(h%Rnd!g9|%2;rRdN55ZHxYA5`3t zbA=}!K&!~L@^eR1H%OypGl53~{t+7k@$L*$tnyWplf+;zd@j9@+RxVLVgLN;F~5R= zr$kw|jnVGoNd~yvyJdk3Gn_`Y4%xl1qxydwJy?y*kA*iGB(;;%likcpHvW;9)m!Z_ zgAdHMZ^INmt$8}#tO;sFtd+ENQO(cWx8KfSJ9!q})|`#?+&2;ky=Ni7%}_&PGxYTr zC^Xtv1*Lkm(0F%U?z-f(<}6*N+957c-gX^#=)HUcx?~`1t)^Rcc$z5&Da254V@AZ6 zQJJoop0fHSkL8=Cqwj{N>z>P7ZN3-FFrjZf10EAL-%GdRE%~SQ(#eptd*hFDK9VbW zlioVtQe?UPsbG&Y9kJY1YfU6IQ%Lq=sx%Wri32e2zWKonfvt86FSe@pgq+xEk=E zJ>~BjJKn#4Oy~X;gsonjxva9LMfGBfE-|fp)nwFHt&-wTR)q(K*YiWipPM7qFk$2+ zKp7}U)qyC74&P&tg!&+W0p|L(s1i&eG2B%4=^TA^aeL%_qru^>D)AGB2 zPb(UB2b8>LJLyh;1xs#E&|@DPV%C*zYl4$CrHHg`{J9V-DSiu+cUVE$>Owu(xvE3*;Bqz9N4zoaWRB+xScNWfi+2@Xm~pvK1sDEE`htw&Rb#& zFzkr5(zvow9B0P!VVzXA-+M9#P1~jEVo_BRl6M~OoWaBD5Ee+ozX1(aut){>(DbYF zxYg1nfsyu2jZF1+!@Zk%sFqP=!b{jN_ZQFqy3QQu9l}*4jj=qO;DsCJ|w9W zk`9d+c&l2*^DWr!`@(AYH4Z35H^!7o!O5PxFEpSVC#Ra_(_1?=EE* zu(#tNvg;QvykEm z`12+1CV+XKS(=yNoFzHfAI4ZS07ItJ<*4bklwc1)14vP>osiEb;E78qNsQBBbS=Xm0 z@wdsnp~Vx7!}KYc)eqkUxefRn-reTrj>}O=A4JC-eiAb`d!%GQM}wisR>8YQBr6s- zN0(7qMTl_+)6s53iOogHBo3-{*4}N@+cjSO-l?y@T0>P=H0>H_!Nv=K#~%ormT&2K zi$8-z4_u`^aZ6V(5hWYjwDVsUlhpmMlnS=a8#u0ymc4Gz=pzh0*1TXilEN(?>^Quh z1B|`Ey_?ZK>(}YH`Ck8)?V@qnpYFQS%~M2HO9Dj)T_9^co7Tn1fd7aM%e{7{9*Z2c z8XIwMa*AXo3|^-hW2Gj1ux@;a1{u}hXKXgcDq66jIg__8N-Y0 zeqmvAU9);G%%-qDy7=;l&+sy@9gSz*ZO7IvW6ul)_lpeC zwH}aPM8F*a<_e$w=-}8rNz8{eaYng-SmC>c64*{H==KU{=E&Q}jhmgJzIYOZB5PM;b41iM+sDNmy>J z=fah46pr4knDG~=Bq)LKg^%;=jBfmV@^yv-`Lf6Td+$Mg@68`!y}Rry?|;GGJ@l6y zJnS`U(?cNUuTdRp=ifFkSIgmAzzM4{-M=~j9j@<<*cvx_x808f%$_E!z$tSjFQ+~I z_A_gbZa_OBaH{xE>?CmVh7}kxaxxbFY#m|NWBsOTc#9TlDbFP;>moA%(8_Xg<0Aw~ zF)QxQhTjRekLcxD&gV@xlEvHDCQQ8#mp4^$SRyuj_*iH|=yK@y8voceX6kvU?H@*o z&AXVr3QrV1>>nBES_+k3{v7-~H*Lti^|lPP91c6Czl{OiD!|T~GhyvdeRS%BxW0tu zgq7V&dt#ChvLH$zv#%6#nO6yw=A@C<2beLl$*zOFbisxn*blz1UiQc{T3$T9UHmft6w1VOCUSW7q zrC1|Z)JtMvu9!5@FI~y2V?R9p;UuD>_crjXypUm|b*LScQLHO4{bzNIO6v@CH;*01 zJ@kY3OMRO>Ze=#JF5$49i>cmfe=X*S(dtu@@>%77PD9*Q$TNdz^;@} zNu&_ky;pDUuSyR;*{GM*M4$w^GVB<77TSv_h_Qw-jQdY=`J^~>yMR*UZmYEiFGxBX z9Hbk&g)6Lhi3u+j7LHw>WM{NaS{#|bKCRFb8yT~s()1V=rBAl(dNe>!=Vv8|tKqc(S2xtyBgU$=Qr(cL5_>pUW<7$uHgJGzaZ#sG0_v6|`oLo9m z3n*SW^M3Mp5@aKb+kVYuq1Hb;SR_GmKYkypxv3rQd_ZHkHi#gRLcKnp)4~Rjc$s>UJ8&r91>gt8B zL}B|S?J`+~+oPxjZVnd(R$J6q*-Tq?%PN(`sbr;O(^MIGL5Pq6s!O)MWvFiXsf!(@ z_PWgEb;5@J?^*Dq#x^Ox=a7SFq(-1)PvTokwQQ>GeH#-CSKg!Q#N$)SI;&(5fw|>P z7VHcpal->!d%ASfT%yegS!nT`8uY=;EMFhpWofnZC2l+;_n9H<@sqt4$2@_5q`5rQ z)-8|%TDNm$*5c|Kjm@939rJe03I%|yq^+hM8Kp~H`h>mrvfO@WRs-&~+0yz74~fRw zCw^akMlLxs{B3F030IjIFDl)~17(z(6Ek}x{d-2mGrMSIJv`mW6f{WaiaZZ!Ym6 zQPoYN%bv$g{}2PzDlcY;u5M|NfC7sDn~wGx8mWg;hBUI=8O5#;>fc?UMe*X#PSP_DI0p^{vIC zOY&;DW6}b8tHS+?U!uF4LO67jLZ-O-x9y6hVkdw4e|9=Q{pFSPK@SUiT}st2yQI04 z1VCrJk8fdnG}vvzyP-7tE77dIwy-`iT_oDYHZnn3LnHnxVcnm?J^rMJ<-qd)f#e8!_c=Tjf_kdJ-rBk%j2Yv2E0f9FFV zc%K0>-tim1{=3({^U=TW_iI<2*5E27x-yjF?l7~f{Ft6B+cR!YbjmD3f;yhH3qOZn z0_R>6&)QG^a_gJ{b)NQ5*ku#2VEfF^`fRyw{@BO-l)Zq?HBZ^|@iKEzOk#p3Qi>h_ zIog~Yl~T~U=bz*$Jx+A1Q9Cu2Veez1C?n3W`dIJ)C9uLm*L<6icv-L1Wm<`w3a(+ym>{SHM& zTdIz7ai)LUpZ>#t{ky*R>;Cfp)Kr<9>}fho30KCY_uAc_Xu-5AZQ-eYVZ#5$YhL}b z7yiIUKYHU;SKjOMpZLXp^ZS25OR!2P!0p*OD$XeZo2lZPe*WL;y-d#__~WkVul&+4 zmiIDiccG`LE1Jm_PuyLIM>axcR#My*{cB(Q^zVDoOXH19sA){6nAt|)XwPYNdOqLH zVBYL0+9Ir6<0en>2^-gU zd9=+PWuC1Y`Q%!+<4Z?%LnVx+^g8?=;r)>pd~d#&@zg*3$G+*W|Et^e%*c;55>*}o{rI@^tM)@CD5Y=63^VK&LouJtf#J6AD2hn#iKHM&{FhjDT* zydZYb_%Kd+4*TI3fBuuoSIi#z=&CC{oZYB@@inh{=?j10V>f-|sw=OKH(~zf_x=Cm zy$8G%Me;okW{_w?F-Jf#XEA^vM#QX`b=@_KY1gp99M(0*b;X>OAZ7^yVnWO!m;ePxqOc?z#6pSatXN|INFfxiwX%PE}XW+?h8$;gTef`bc`@tY0B1b7q*B zLlRLg6dHq34wX?7!eo*BP&U%H!F&>)N_;X7%#?Wb)>?iXm^D$m@)BVjIBAF$^@|}5 zdIU#2;nJ;j>#Z$$$ALuAm5-RMAefZG=O$t1O!(Oe=O7LCtQ;#-s#Ml@k+r^%L?<{| z_v$myiVPE~TmvL9Mvf^v2%llN(^L47Kj8`HTQLc<;9IIx_7DZA(xKL}Tz zesZTZ*I#YD4JxD5?;Yk)E5qEk!JLyEEdJSHkG=N%7;a5(jeDijU0o0hNchcN*xev}| z<4(h${_Q@o>A7#t=@{aa<7rW>aI!6C1HIdCMR$HDn2623aS)*MJytfr*Tz0`_qCTB zg{#RvN1bq1zdKgNuZe#E6@r3i{$|bA>9WN+H{5l@uxC516%JvWHETg@U?^v+=)?No zZ9{b5WskfLHWOL*J^IA6ufNm%uGPxKzug1m3oOuFIk=+vDdgtqsQKorFYq*hZ>PDZ z36wIY*?LjKy0KjbL-Ct0KgUzY$>xH_xLmO!L8%uIloPGNt48k-Gva}Pyl#SvE!wnI z1HYU8E&h{%9-OY)wR4DJ!-{KjAT4}x1bNU2r(QL9s-=!3NRGah;&*04jP)nz~gnbx43%6UGkA1RM#BuwuJ-3@O>7Agj%u@lhwr6e0 ztAyk^JvZ#JanFyLG-3Q(<6a(o>-BC!MU*=0ttSUz7Q|6j)}*8-#`0StpC}bBgrXRI z!p2rOrJsUVjZ8u)GRI6jck!!EOo&oDha@f+13$)l?=AhnO0Qe|#BZagm6w|raNNm{ z)|Rq8i~~=Z__nK?Ply1e>Vjrds2KIk#d20Jge)N>cTL!er!I$*D@e-dDAwa~&0 z4*c|QzjO!qCk_459GPsS5W!~lV3MOh_Bg%7hYHxoJ=B|_Y{*qDzPEZ!+Sb7em| z5`5?%=MLYPiOeyHMtUiK8@KRfK9MtK6!M|3lo0YX#qXL{VmluVZGrz-dghDU7a)(tqucbu^m;Uj%B$Jabz4kW^Wpd@9 zp(mVwIj-P9CZuA!@~mLJ{^E0bueto*T{`c$#&Uyi>1XjuLjrlUmGyWG*Ioky8{gi}RAMwI)mR9ZkHEAC=TEb+J72RCooiU4jl zV@3^4@c;RuT0IA;)`~7F)iW1DyJ}JV@_MCYL#h5!)~`>Fmt#=`^LuLdVg1LxHFV+^ zw?6U8e#f4Se^%uP0sAmqj6+Q$^R(`@tZJWi+Lh=*NyhZjwKC@CRz9`#NzQHBF0##@`(5|&lbda`qcy!X?$wu{8BL8clmNZux2=#KLj8$s!YAMc z-BUa^)TpwYR*v@B_#W}VAUl@Xvh6~0##rVvek@aQWtv?vwbHe1Y5VnFM<9-GLT@j3M~DvDTJ#t&|B;< z#iqt~5;X!yz-1clN)5cY*1swi`dVIAFLwz!Kkzz7g>#dbY_TfXNa4>y<#?Bve-C+Z z)aK^#P`)^N#KZ~XnGcvC?0HzZ_z47kFmtKZ(-LY+) zjH^?Re)Iu6v7APjw%V;v!Vl9+Pd_o~jgSd2(Cl@@aa-@whxwz@%*TD>LoPqZO)Ge0 z1~PF^B#rHI{BaT#2L6xFjne7BIHBquW$hJqLvmKewO?j+2ibn<(Xo#15w;E0Kf z)|a$-e~}^R*-|jS|K9i)$AE75W$PVqohsvk>gnN+O#A#Z7bwO87*U~7-HMc= z#2n&Mz&jG(3LGP@w?%ha85i$5v2_5f$kXH4z|c&P_yFRE027|``rgF1o*(@(jv|a#+ZH%e7 zFoZZMPSP+Rt%vaC31G2C5C3i36~crz15c#XiNGW(F?PKbPl)UDpzKUBO0fDEvdWjC z!}=ef?u!h?c2VxH5tPQ@KeJcLWIEY z5+^K|Qt?nlm!mYyN9$ppiCmmu`z3@aGAEMiNpHT6SFlHj``)Nq zPuxb)vQ-6Xxd-u)!UYp*SKRePTFGD99=*YltKYdM(821XhMBnf_{jx|sR9B$ zG^!h&?3_7B!$n8pV>zlQm-rFYPeq=jh8RoY$>8P4IeAc!v#-!ts|Q=mE!53e z8PO=_aQkK3L;j@msX%n=Lly=KX2pax4|t!bln`u&R!=#jM>=lGy{NHx%EWX2pFKN+ z5pbCXo}KZFO=y{4M_hnXD}6+nD&yz)947vOH~xcl@JE zsQZrYohF>n7Say9;o7Kk-^gV5gAcbF$h8_?OD}hmd4t&Cq!Mx2FgS%+ShwKg+ai;c z&{%f@5?F#^2zuepSVVDKl%Kly+68Y+++?fmMm{)*8eFeC;=w^X9q?Nlg{H>O;j5G= zcV1_MCmwW1oFw@Omy}mcNd>AP?xOS?FF6@sOJ`;g# z_3NBOtkv*Zdbw*I)7mIgEy0!un4~_6nCKHPk|%AXuNW#ZP+G5?Hi?_y$=Uchj(GGF zfvdL;IR4c0`u+~?RN{!I>vlWWV#_nBg`}S2z=!YH4G$+%ceu4>{SD=q6sJMG01|BV zq?C}?Q>IEmY)VWT;-#;2{~uA#ZcjBP?W7L%U20XSmR@YiT*%w5xJXtAI5%IHsjCc}#;zcm7P8xjcKc0H*q1T@uqrDGST}<=MKR<3h->UcSTkpAFLWL00a_u3ox*tWFSe!%Uu$zVfB&Z8)t1dG8g+rAR6)-GGVwFI`s?zQp6 zXn|X{U1Z~~TkZOrLzBI>01%RDSQBJ5Fwxgq6dMFi(r7VS+_&Yo^{E6~NBSn3|K-J` z4)s%2^%7Vuy8MpO8E3H8-`o)eQhQKZRG3-8Pk<9P7TFuJhZ z4Cw#)!}oZ=q*ZUb$NsGsTA1t!UI8hXuz}gG5Z3y|)K78c1vTKrYo!-o%0HKZ8G6GH zGiUIkDWABDk4##%b(^T_pcc6VycaXKv4~+E0^T$a$XFG};ihOs6q`TQdM`Awng!)GGrQ&btS^3^A#Y51x)b*xeNm7}~YmDjiqo*cppk?z9^YjOYaYQV#S)nJZo&f!i$Y$Y%k^%WVM0`>*}-YN~>UGPlv*= zVr0*Wq*C%i+UWy@Br7hY`P#oEm~|NOR%$~J_+S3}z`u0{IjgDIbX z{K0~)Td&k%wYlb*$89??D8AXE%Jmtr%I(*OwH)s1tqCtZhxEv^eV_fVtspI^ay-LJb2$9F1l)=McauX)HBk# zI`DVc1S4D0UT2>*YxsS4tAE{IyTfWwck8_?9;gv1n+zRrCr@Wk!}f-_F$y|>Jis6v zC&=m-2x0qe(qzfyR{$Wo#vc&V`#!Y-;K1>Bm^rWqF~8^fD{sE&JnXf_CgIz!J|8mZ zPJHmlgKf%tZHI}!0}2zEAwh4IUw$%+h4R)@>rC>zOYX=5_vgw}I5b z$w2aJ&;1LA1AqAE5A)7D-{#x(KIW_oZBRO~y5Uhk`{I*f$juS?2}5ohbnUt8 z!N2RU#yZ4LeB<>CPdJ<>_&)u_ys4DqY zXTZ=vU}{Y!tVfoyi4`Q4nR!xE3(2@l!Q!d6Z2!U48G4%u-*58EU!!5GS-HAf0AzLm z<_Wm6W8QrEg@2oq?)ZOo&%=+kBqUF$piDF*hf)F^IUjc4T^JIPW}992%wZE=egT&l zX#K&n&6x?kfKMV)U#W!2@>5pyN<5!3%7nAC zlsDL-CoT!&+nb;n_u?2_-MZwm%UL!niy;xzYtY-cZZU466q>bY4S+2O>@;C+H%l90 z$bENTedek8rfV^9e|F^1?fij}<+qqxa^A*Vqc)11nxwt8Lj0O~ri8gtW%SBa6Qx{4 zLf^b=gtX+V#gRSYwGSIR`teInIriJHr=hdkv~7FJCH?sBu7t!^X~>|vZur~TGp2uw zQ@+crwDLuFKm6WX<1aYj2>uRy^6`f@vEOaeg=-O&bIm>1DVOzIZsm^TPui#`xaRq; zzh6B3zW&+-=ACc8l{&4~Y3&UUJpN>z39XYPnE(0x_k+x^t;iu@*mm^|y7bs_*RA*1 z*Xpgz-0-S}>(2cP9?(ADyz})r@>o2o5PCdI6=(5=-FGiuAkEexy%joiQhK!#Bd2Y9 zEhIC6l}9{SWT1Q;7(enj_34gy%Ig(lImfczw$aKq6FUx!xL8|k=GLpeOs_A^5s&?? z2#1mf{kDU@dP@mFfMcGv0QH0ZZ(5Z+Yr;8Fm0oEl8X}>TAY`S`NSJ(qPa@J(sf5Y$ zQx+A4SH_ICqy5dAFNm9%l)u2W;u28FEMEu!;7P+*pLGgHGOZ~v_#GU#nlU4X{HDi7 zCtP^Nrd#b$hgqijZJ+S+^Vn;X8S69d4`sxSw&;n+Tj(OV*RgYdck)upEg$m9Q}i10 zK$MW+krnev0N7<)N?`0i0y42toRi1xwa+*;VG~}$Gb4v`5W-f&@(>AYqND~&>j-eN zs6cZjFsOVRLr#}Gv4l#T{}qABG^ROBFK-U@wHPdaV;U}e0>)yk5J(^ahg8F@sBQ_X z-=d4Rr{2zA8}qDQ;xl&CFhv)}GQaZdXuaL}`O(8=N)p4)`X)2vMk=oV7;+mj;P#;d z@4(4B9t^xNdia~;UeRHdHWLUFCyb*nt9G_`mHP6Xy)y`U)`!%TG3aN0t!c-|vD>=!^$aPH}*3(u`QE6T#6 zw(hLxp8Lr~g6cy}PA;Sbx?rAw6Dr~prCdZ(E3S~0JWA%*5yG=G@?co`FohJ2j8!vz zn+Zm#w$MjomP~luSzv$cr9nF^>+o%cT*G-FFP9 zGv=FbzAd&1m85mZz~YV`YUJez{X}|eMFGWc^*|{@3zy&)%b8bvX9yu-p0;^Zb< z_u-_8xB_DO*V8b<|H0%*cU*OGDJw02Do!_Fd@kM|50`0QeevqpXK|4s&WYd%;O+6R z;e+=VMxgtCXTmt@ImZLQa0CE-Wo@Cyy`TO2U3GBx_3yvsdY#LF9s{itUU_NI&DS2f zLs#6H?~%oCCKi)v@>>%Q>$Sz-&-xQyPm6!9l0xBms4t9u{QMJ-IC=lwFh=CMz)zts z4A7{!NkW7ltJ4qNA8$PidD85uDJFW?3$aB-k9eqVcgGUEdi36d40xY< z*KK#A%sDpRq}9~J3zJ3GmmRoi(|6x~!@{)7rqw5zR>|TibFXcqyS?_l2k+bWq#gD? zXyw(`kV~Dtc8EwQyN7E_t^w3fTsM+Rh6>XwBJH;W*ojnLW7i5-WZz%azOf%`^#yZ_ z?FcnY__0WGn~ud$Z#5QAJ?Et)y_-dn2b!Yo-fZK-Cz-;|WT&b%gj> zHp}2sw>iYPMJCp2Wwsicw_kh3>cdlheJa%FSy?<=9rE-BF=NTmiU~yJ7diZ+9%tKIb za|!#l%Mn6RG0NVdpo&sJhLqhGEmTM*f5GVu4jl0%Bo|B$WaGS)q(@Rir|9Z*xtwck zo|cFD>MLYyr2hq3TMPurwGu;B^uuHE^&Q_C_o^jato^cf(ehBAYiwNWD;fWy*av)e z*}7Nr<}GBEF!A-*D5HlUAQLr5m|8-EIn;8L{y$`3&SI0`TGEKgR#mO0fNSYTKHx6O zWr&Vztfi40C~=O~(m#HmyC~okj?lx*>r|j$dDYHpkp$r@g=dC7gl8|ygz@29 zN4<3X`IjGZ;-63a>*fD=^r?PBp6R;7&e%j3TXM=XoN+iwy4+Ogg+B!6{&XpB# zJehx@uyOUwiml_CYvTpab=Eig15H}_ zCQ-O};mLikI^$&A9wKYP@X3_-j_Tdxjn`fhAQ$ex_4>ni*aDZJ!&ECb>iw|ohEEQC zFeeEb;9{YF`thf~9(VXPXZ{fqk(X)kl&B5z0 zJ%=M6c}Zw6ZY^ZeM_5~c6j1<6nH%k}SJY2PI(jjg*zM#p8ACw_kRl)ihtQ#VMEk zeZUKE-}Bs?Hw+!~(D;w895@UwQ}CN_;J|fE{Vq5IUP(R*k?1YIWbL)7!Co6zP0Gr> zwtf)iY_F|+L<_;~vip}F*kO0s7Fj8SleE}t&z$j{FxYF~HtMAl&bth6&&6K*y7FFI zusIg$mlRe3B-mJqQj02)+GV{G8D7@W`XWkjpA{i{|>AX1xKqKEa952kR)O`JG+w?oyC+rQ$F+bW&c7NNW<1?(kP z?p!oMEtOE|B_+;^+Carv=KS;LW0{q85S4^v$1S}TYMm9e7|FE=byie>#g|&f7513Y zSSGCYtf)FjC9>^O&_sZjb+o<-vhj;~++1VhHlF$CMb4x8ikBL@>c}L)J%55PYW?xw z?#TqGpft6QiRo@5GK$qGgoKa@Pu}oYIm(DxnhX&{XN||!4pz5G)4u%7QsEsXiEm+z z(XB=+P@$CM<02pYOKelvdN=bBP?IM!5z=L1CN(i_T^U2G>O(JbN{+Rx$X!tks~2Ni zQhi}8S=0)rjdfF20UCqVX%UcPM-818AASy6+!gOBLoctr$>s|$w)huQKOrYZ&L_O| z0&fI}^dTC)m=)#}XaFon(lp@4tKS(v&VrZizEI`Qm4+RU31GfUw)1o0)w96{oF>kd0LrpXjsXZfnkvt-HMt+!&GCE!X|=X z&@(HmNx%ul;wjs9p`Kb7Ar`rjv;@uLWE!(g`FfV>;m8CqamkYG!&XFh`w1f?9oAU; zjaOd85}fePuu%`)xBcD+<*89Ka&Uh-a=z6rec;t$^>tk96x|}cY^^J;wg!;Uzybji z;S@&RKR_nvo3&^;>!%+v21_N8I2^z2(!ZV6ueht$S}|<HI<@AkW866aGSd6^kN-k?NS;oZ`}8h2`}#XVcf{>Q zzL8!ESBGMOk_QbLc((=HBc9E->HW~%H_Pj>0e2j9+$nbZwT_-Y3jyamu+QY(x8I~Y z?QR@6%P@B}i969=y+ttInSy7yv|ePF}|EORl&QXo!h?=G(NzXoDycnW(w-@(b}!A1U$n!Ep*lFEcFFv{NLwDXJ zrFz7gAcIP5&cwDfX~Mm>SIoj*`^eM&nq0|(z4qzYYk&2{0&!>xJgiBrSdQNXa}$wH ziW;Yk(Vae<3@{+ACWA*d+b>a0Gc-gHi`+>T7hwt)z;gtWG^q=_Db1Fd8%Gn^Im?d`tP zZjao1D?*SQXGPu4y*!oT8WZAjk`9aNeVi2?dT;5HF`N~3NPxV9))1phy2#;JSUO>@ z!ZIspSXdlPVl2}W7qgIgtd%oF*+!?HS$T-*S!avYq1>3cU6-sJqZb^p8imG4Y9n;E z7p#-j(>2#nU*9Xe$k3$GI1wi`e)124;MtAFIn4noUwcTx9;H=%NfQ>+{*D@pBnJ1x z+Y0I=g5?)21nBWe;u1_)GXL?XpN8FckCgb|d)-a9)S?xpe>07X$OoN=Occn%lGY`d z#A+RlTqj?Hij9TJ)(wmBHxx!Jd4~o1)&V6s18Fsn4J=)7vBmbg5y>0i{kEb&j{jouJ<8(%4mGF+M5&j{6T*vSd$;nDJ>;a*@f;~UlH;v$FaPzp!|d%*PYr$W{fTcc zz2eGAg}?sN4cmf8Auw#)_v{PT+q^q|prFLN8*$m%-T$~8=O$P;fN_EY!?u69@&OK`7xR9kg`J4d4UPlzbO!Uuk9fpi)WQcq zdNGCSZ^RuwFpL8O41QhKo;a`|iBgO9P!c>l?)RG`|8hPaG9p0b5f746=0aM*S|L|9 zrk39F{I_JAKXYIcpjX;z6v36L)-hG$p%lLai&W*QOfZSnI!dO?v@brF2EsBctYQ}_ z19|7)FTL-k(wo;{!WxAM0#+E@5Vys4I~{uJnTb12lihdAKZv1AaW10_oqMjiaK*uA zAAN|&6JOcBa8%uj$I=QHRb*kS|?X{U_*B#sspeF3Kk@LWt{*FLYVy|86bOr%x zuSkO4e}M%DTK8UexpvuP#t&M)%NB#8vn}iwTJ2uD*@7*&*R}w{_VC3CA^m2l98WmR&V)}9F$uvyw(2?$|{Qln`F|u1d~{;qvXUz`dk1KpM3i1 z$A5m~?>5C6ICI9#kKUX3(Yq6$c;rD`nxS7Z^RZ`NOWKsU<&uYOv0&+wgo(6c znef_j=WxRs8P@Ky1#Zy$a_T1thbU7fO~hDc{xbq*vYF!@n?fP0CT<`zKPy@Qi~Xr> zsj-KC*H|WCNnrs-Vl1pXE81*9>n=HDuH`7GZcq>tEv257(&DM4hI7FrG18euWTCamR_1(CI0AzZo~g7jx%Z?Rxmq+n*e?O ziJ_l;@)7pm^uSA)@b3++m_zC4Jm>VU-8=vE#88)sr#7L2pEU4=1LlF<+i9IS^SjXx z5B}+2r6E=z!T7tm^T|g78#FGi zMm7)$Z*cVK$~w0NfHS}|)D^l(J2NcQE_&)%Q-#*|5K4;ygz zTjO3)Tp*jbYJK7bS8?44t?+9X+%bWN3~-SbHSLyK`t=vWQ&B#c^bW@Vm&4=hW1Wd{ zd9t4NeI=++)Q3tUkJ73~Zx$>-6dlM?6tgW5BV=HSU(};fP*C<^gB(%r+v+Wnn4I6@ zDO*x&4N*+aIiBHz9;pT&m$b_*{yx}b0Opj(g=XMQ*Wh1AoAwH;c4i&WzukN9dexaH z>jsL~#ymasqmS@)wh7pmNw2FgTWmZLu8J5SL%fPNgbUd~72o5V)(jy*!^_Dlf!tm;?&p-W` z8eF~g#!D}B$oZyoW^g<{8oQ~129Zw zGB*wkCBV?*i09}t{-*z($PjMt5;AbkJE+8onlKbzoG-gHHYrNKlRvypM3DXwijXmBNCZly6f+k?s@3ZZ5LULUYw77 zus?pRVCCZaTl845Q~By4PeB0Yy$?KO>=VOCi}|=0$D(vDt|4{#``^`)-iNUI@1}qA z?9hkd2Oj9x*l?3Y7I!B}HG!wcPP_HL;cB&O)cpgG{>%A#D-+XD@~Dc{;N0lU8Pic$ z5Pts2CsQW9@xkPY!w25OUADYhv~Kf<3$J!Tr3x9?HaV60B|QFS)-3F`$G|p zTR2Qwp~I@Rwj-OHLV*+b5D;JN2oEJj$O1ztU;m9)32gLWOCnbAGoPH_liF5dOG>v|exLWN6W{)tUdeeU*5p2>`aG#DFSsBnETVf!nG3b@EYw z{|OzFY}ZYje(>%)>@VOKY}rafZjQ%n?>c!0J*Y6Zzk8*I`&Kh3^u$BZ_T|7nrNGbP z$ibQ2q_d*5blbr_3=3@BvXE^zKbDDH$TI|IMb}wR+Z~Xl^}%PI1#wzXQV*VFAQLA1 zcy?HQZ3u)Ci8%$Qw-Zb4>!!1!2$76sIu9QhBrL4jMY0NVu1khL-yTa|tg4T~Hd;KR z2R)jvCgl8)G^?i?;mgNx;_-skpRzhEkytrEOWEpU@Pfc<$X`00Q2^^px*~(`G^r2% z6|HVPh?AA0uHN@QzyHCS@!li5NV(|g+}4ohvHqnYjvl-2m3V-hIsbF!_tsRIi3QY` z@4vO*-_QPYg2B4N?$-+O{1e0R(V|_k@JqJUu6vUKY}cL>*nr^#6Mj%Impj>DRa}`Y zo2(5APd;+LeI$GL_&71^*kSGU+b_Eu{!yV4SH+JRHFT36+gVH^BP%8O?BkDY*I|hS zK&vpY9(V2~$uXIA0TP^;tdjD|omYQ+@V(L>zsL4?t5)arHj*Y0v_2F?EF++|VFXL` z;-NQP2u_e?`v1a`nt<3={<%`A>=#S!FUG-}bc>5|V8uRc<<-~b_JYM0t=o3#xjnA8 zl;iLR?(TEwajLU2A8szZ1h~g;edO{p3VqMIh%Y#_oJex7Ly!6E@7)EfoId{W-6)1{ z4clUwl}XNJs3F+8?XWHVF>HHo)v#?6rSI7nSESdb?OuoV{p*QG2>j6p?@4aFpqj%) zM?7@5k_7Y{wZz; z!&%oH*`Vghha1>yqvqUee=_BLvQ_T2Yptkum#yO&6akaXt?xM(FVe1b8im#}%6d4N z>qD~FK5mcg&+d1}>g#U|K(cix$waU9iUF7z!3mN8@o>?431CSqOyD~FIbIqY+R6g_ zFAJ*k|GCUjRj#qgdW#?VH8Syo^I(OqN095OPi%kw_a6 zc&uMRgY^8ky5o-_p>Y6wyfu-FG%Ts#ep<9@#XM>G?BkEg;IbEg^g_7Mq^QAn$jvs- z%P?UK;hCWig9kEn%XjFATQ2Ep+en6HE!-trO7-)PKc-B)ohuYb14gRYcOSq1K6FVe zfPJ&lld3}6DF6E52iIOd;jqJEEDvNG#loP+A}72L(iN5yllJq}7Z|T++(-Mn$BuBq z`12M!^hWA=_T6^c!?tTkd{_zlE5-W&79X-QUuqO|Z5CSOq`zI+qql!c0@Cvtg2M?~ z4`UGwM-bQFVk=~hCU6Vl#&L^FDQgq5PsWyWFg#?xkOBc z7(`B`wMF$2NrBT^XS4D$!67kYrQSvi{843Fk;}@V+zhw>EAgNgl72BcKm-qR!?@`8 zGrz;03=f8U=J7}I4vr&wb;H>@yvRds#Kkz=hmoHZ+iKUn0Ej8i8rna+MhCQt3&0bP z3|7MUe_+$Cwl8NvG8c*|Id``dCh=oAk%44B`yKQ9;$(sMQwY4q7}r*C5%{mBPK68k zsl>1?Zb=PwYueNe+xpw-V|>4co4~#HKbR)!-gxB&SI0ORphIyH>FsjRp^7Va^jG(t z_Gs3;IlPJd0VD2l{7o z9`WF3e8!CFTwCtxcf}FCy8h$b(@;afTefL?(nWs~brv=+&KFy9 zY4P}R;MZZHMroJaBLgv!rqpXiJtGC%ORy_A>!ZL?aDnjgxTtgAq@ji+Y?sv#Lu)Qu z26}OBd4OT`B7X}-8ne0ob$CCqMWOkg)5ua0?&Y$~kF(C26M zjKwIN+vql*5)VPPo{@spAh?KFB6SO{QTTYAH3mIZmfvX9ih|!G9jPWu>Sfl4YND8= z)mtZGa*gUBff({g34wZQD1(!&M&`;?f{kT;9aY4$VwST~&es!1=%INOXoM$7805xw zF-htf5%Zl7IP}tc9&~Rum5L!^+ii~|zHp&6)ZaVe3I$9qpQgB^KG_?&)K*$;b>#!t zal3E#sy{k)`RB*mKIwpG_-qb{dc?)`p`Fs#+?k?WhsT~3MgDp~4aUt%mXq~l{%^80$sdS>{;+pePTXRArwadUZM05#U%E zy+!NRE|#C@4iSR*>u}Qa<3HW;I{C?#xY)WSrhJiX)K?~)xB!ET`CobV3F+6_v|H=8 z)ejWlxd!F)^w5WKW>=1hz}{}K4glaZ#I6S)aqH-pw%zLh)`3(d*=@_AL9B2R4*x6S z?QV&3%!tSO?!N5--PRj$!!`I%3084xl}KFvA74~`k~-AVTbTL<0!H?>@3;hZVEF8_ z>FTE*xCh?B@%HPbSEIrA*mEw%SS5tewVrN+Ew);C(RLug1jVE`CQO_V9@8k;Yh#}J z^0QAp+|4%IrZ=9qr^Q^TzaAot)&~Y9zRwZINRutBqi=G8Yrg z97p*T992>Z%3OMBY%&R0Cg|lTK@1Qi%cYJ`AD_n}9;;%jHVawW&p!RwE=oS?jKAVd zShtLPG5M^qQ5UYyLM8%k$d{&N+lA(D)?ACV8o0=!#kC9||7gl&xj?XOv3km?!_<#I zPWIYu%6n}_$9yY)Uzn(py>^l$^1&2h)sXQEX?NLTZj1nGvU8QFvDY4w?6n)iJjWPY z0V{)0+14RvLj*_|q;I6qn23#}`VV1tJyaXjy4lUMK0c3CAD=g^79Ix@i!8P{R@dAG zFEa0UQRza_1Ov_nryS#sTKsWK?u(OmxWPo*7l+(VK45ou@-CamT&rDzCka@5Em^%u zl1B&JEjAQt^BsC?(F)!CCENZd!Ns&^MJ;R&=N@HP5{B;&T0#t=lfW{7P$VwDCsWw<}%> z8*`Iyr2s;5+F1J?^M@b4`wkCQgp0M}Ap%#N_JSiNPQ4Tkbo;7D=}scq|Qt8L&MX}x0Et32Fz4)mNuXx(<< z-<^D#8`&$D048)$kDd2qKay(LpnF&7*jc_%KK{S}n&{nQmp#ffAy#@i85C?9oqay( zHtt(df@Rf!6<#=0!pj1{gvDu}e@-D1H_>fbOw0}2)|xy&TD|&)UGOdyq3{k`UeF;; zB7m9XC4KB)9L9lxrmw&FqJ*itP-?qJ@&}fBj00@8{f?CcxO(gM%Px=!M?BRxqcLxv zXA(<`Y!??HzKgVWENg(IGR=bwSwoe1L*&qiBrv-c6l zuI@+667-~z=MjOV~r(A#F%vvqFc}N?sm{ zS@Nu0rndMJ%e*=({IelWvEj@l}y-ce?NV?{k13H{fTc~dHP8=UVQG}M;*WCZ;x?% z?HW}jzX={Asq*x9hWi^X4sbM_XpfIVNz%+-!(cHgunT@`*hwy6$FM@~o(hLK)9l zzwNTWit4E$4`D2G!4|Eme(~!6bcw_LyzF$qp$8m)a*LL3R5+cg8d>OKOVJB>;11r& zddnr}fdK#}JZ8j@=SB`)X2n%d*lqXP4}}$fD1c$0iR6v}%T-%}gVkD3sEnkM$BCRj2?&hL7pL8w*57B7s8Zzh6>Uc7E zb*-!1a(ufbmbA6|^n*rS6Jfa#-PQmy^T9C1%g>A=5*@Jard=!V* z%iUPU&Du4#5fZ_)&8 zdNxF_J_jED=W|GcIXQDS0TLnU9lYSy0`9-%dVEq^gRo6_Y~Vc&9vCq(>{`yf(FNk7 zNb;5{6#G|;m4e00GxyvmGG4ye@BF_^c;zLSe5u~^@$vX8((5|PjN?ptWI zJI& zwb23~TX*)p1>~Wvl`=N)!~T2@_u6l~`ZAO@){Jkz>31IX+7DOmwQcmAS@~4LXstBm zR8}(wF0&g;ncKL(j*QTA(7+@&Ff_gi4MrqYl2v`LWW~eDIQ;(O8NHkK-0_qHcYp8g z2}<~y(@)vBThErQ+9dn}(Xp8HCILxq`yKszjb(}f0?&%xbn&?=Y-7q|86YRI5^KJg3EQ6EF|1(>B*sCPjXtB!C8BHn>UDh6n z2jM=_|2BC~n(*4!)4p&wHCP_wD44OS#wF{>@V` zf|Zk8IBZK7nB1_f#|eP;CK`Y>Y)jaT>2BB-?wIG%y(aG9EL)Uu^DceKrih@+*#gFV z$yYzHG~yoG*IyVzSs3G2i2jIY(6)Wt9b}vkw^p6egUrgUEy+0W@+*h|JSl^#kJBZS zZ6o92h^NjtaOI&SI71s_Rg)-T(F7A-Ia5POsbY4UGQ1XV)FrXRV=N|-(91+xJTQc% zm890NNhA{Unh;}g>e!hkY$S-k;R@SGaw>&@ihmk`=Q@f>?Lo7#2-zXOq zU`V6Ko1rGXF+OBcCUuuHY~?&_#4oAg$jNz&fe_EO?q=QA z-E8#Bey{!XkcY6>23{1Uy>_gFvoaIms)W!+>y^VyFyR%R8bX4V3^k(X40*tsnVU>( zL>t#YKvS$HR-Z>CvKGebC8s@=lL~4~^s;F=BlGl0gz>qDryamaHXHo#rIqUruyug3?A2{So3mkDP@-RTTQ z%(gFuEVl1;p z=nz5;Fv%2R4PV}-8% z%D5KeSD#$H$`dzV=yOvUm@ZY~JFUBca3~NSbupk{@fduX(jN>t)D?#I4;L4p;tMou zzS%at>cX2a6M!)RnP*a&#_ZO?7ooBe)!F6zN<)J6gi?AN#`4sX#RCRhn!!30N_0=v zud@IUeDOlqq4(XX1aiT8HAc2M6R&1A=x(F0weeJ>j1^BNBbC3@kQYYJ_6hhk72K%sKlfUBU(YL1df8p($hL1U` z-|f2}a#Y1t%AO*L+jIAQ#0+8RpnJ49^8P_~4NP8RRGFT|i>Hn-CwN1LVOuSsEcvjl zUEkDj{{V0onZEz}{(G4qNH)w)O!Ssn$V9MgiA<#BOt0R$_4=cC=~?TO)D1^GmEDo# z!?k6oUt!$Z^88z2ZTU^Qw%EihiTll}`qfhFkcJxv{?pkP-En`%HP=q^1Qm~Xj@fma zuRj0GGS`yHEdB(Rx4J$)asRI@<~h$+8fakSr&@IbLuDFu@RdP@U#ooG7H+k;`izrv z7vW`68kOr6tnSma>EcT*s|^-U^RYa&E9TcEE|=ogD|xOw3m~Fs3^W`nb_)ludRH>j2vqB+P1bfXs$0}mn9-=;qzA4 z$HSUTWT?fTfbC*VWv%}sa-DYNr9;kj@Rcq4&+*!9lYB5zf*TWH6=2|v*Xjx>8yB)B ztcU;pceqheN_DX`5G=FRTX~&LyI%d^6PMim(B|9kG*`Tz3jl7vcy;ml#~o(zjhAiy z^h%m)L)Dgnm*z1Vb=Iv$dXlRQvjvYU!XB*-NZEI+~2XpT(~diP-7vL_aO`tFtQn1r_i zv~o)DjxSQK&~dee7HvmI;&D!N`Zv=oDfP@yoDeLJ)kA)B+{=^SbQc#e&&{^pv01YP z$q>Adp2-kM+-HUJbHYGCCf|NDjT-VJ&%(<>Y4&rAhpd*uSkxgl-zoiM2wc+{1^8e! zm9@x3c8-1u40JYgM~hF)s>r8IgqLFvyYKE0+V}6q-M7Wte;&-Xgl!`rWhL2rlrH(XlJ$e1P zF_ru_oW(P0MYUr#if*@;Xmou+J(H!L{B~}mTbW)KNsXP;YP~4id@NqFtwXY&fYTM3 zlN?ZaKLu-*-kHo);xYOl`ZXuy322;fuObD-Q41-by56O(8p?EXa&Qzv|{dtZ8doemqMP=|N9v; zZIpIPEz7LbFE&XqAZtEN9bSovi72KMh+$=F%OL>({DjNs>=6%@wdS!MMeo}q9(aX3 zUEMdcUFPX4^7z#bv#HzlN-xxZRJh(s_+YD zt#fsg>kH|xVsuy|&gXSjLZi4zv~PL(E6~wgJqoBCc|Z zLY7ESB&`Entz#>(=t4U~X-fJ%mZbFUS`?=Q&%9`~KGe`!OCKrSx8<>V%4ATT2@Wp4 zr8YylAWsXkvN#K`-0VRiM3mh+6Q`sOR9ieX$edA>Pu!WfHbt`RyJ^#14*VU5xCoo} z#b>yWyOgHZDDMpu>N9i3jF~fMS~{0!bhG&bqudHS8mM}D8E0D0`1`HNkQ@1mA-CAZ zf~U4r)S-qK$ZZZo{0(caSg~A;&Iq|4*03GL9BQ2vg`HGkz4CY$1s@v9k)*eHr2#>4 zm1>E{vk{Lu&WfIH#xmss6kf46;-P`Ad(4DIn0|@v*HxR74d*q8=&V(GXEIYj8jTf6 z5b6eKd7{QUKyoF#Yxg**F6$83ns{Avd@9HU0OWVhMDYt^zY#-e#-+A`dT&uLST!| zWUdgI`sYz7w>hAD`KeK#ee$s!JFmadxi`9cQsbHy3o8sZKRjO$@2Zdnr<2|u|LQ8M zxpN$OHOR&ef;CXfxh&YWNS2p+SsB8+7NOO*YriD!F4=pN)f5dmi(g6!YJ~)uDh0*@ zxRLlulK-PhTj$CqBsIaJ%0;~G2w7pmh1?r;+vcgE4?;%QfA{sYXGcBOWvgCepBM&N zv64$OEQ2umiAX{_W`Sx8jlm?|h=^fZ-n0*seF;m7G&xNDq_wRSr;9IW4^Wo4sOz?kp#A72L9@K5SopajynmXa?XIr247qvE6=cG|sq>J>n zX|gJ33zmzlxsWpeqY##jzK!AB>*y0s|J{Bz)|IFI;rhp(p4*%)wbt2^msoari+|v* zn~y!~Vyj8y=SB_vVA4b>+b^@cZY-CwzJR^99tsYm-D`6Nw(i`%p#SLo_psO2=H~a> zujzgEbg|o_km?bY+V%ov8!=Gjs;4+$Q;Sdx^@Td_ zb+WC``G4K2+M*$K>Z4`?LlN1xXZ(_o@48{@78`> z@3C*r+^S9K4Fz;McijKCC@h|qqGZO6d3MVkc5Ni*FAZbc>qJQ!l`PhA{3T5Da9^cTB8JwO z%6PCgoy+|-i8p3D*gSLbHuU;D)>=spWo2~KlzFna!X$cfG72#vq8vN|UI~xugITk7 z+~+rUUU?y}F(K8hm!99I%|gBRKg4p<#5wgp{P4rw*IfF@J+~fo{HZ;6+gm;Q<4-@0 z^e;F-eB{6IV(r%H(zS3eXXW(#IcVB+r+t5W*OeC~nY3=*7MX+%kt~J*?*WDmmw(@W z%_WcCee1!K8c$~+TV~=-SYjpITIXa>>V!C1 znL5_FKtSugD)NaW5#ll|IR;mUTVkoD@j}oqr+&)z&g&0kGq@u#`McFvbu0SCxFzA- zqE7)3Yl4ipm~1P`5SOI2y5*`V_aX}9f?EvrHW@}CCPcIjXo3HLO={Le6_8#L!4s&Y z@VuxKTSvbA6uZRE002M$Nkld4XdeK;uzx)pmEJM7%eKgyk%-B~tLPis;#8Rz zqB9d`h3t=byy%WyV8s!SYi8Z)vyVS``}J1^!PQ%PA9kD;D+QM-K5Bdfm73tK_V+<> zE5OBh`KF=t^|#-)i@0%M9ZHG^hmu=vze~)*Z4sf>o;K)(Mii>7RnA6mFz*%c#$>B% zh+|xXbgc_CT$S>&1+?X%7Y$ZVjpYo^{wsJz2d*1#&ME3$eX6$Kw3)0=^a4i!*fq08 zqU!fz{bUZ2(=A0=aT#i}p1qb`se?8~0N;K4jU#v3@|4T2$F&<+1ioTDag0@y&rpq6 zew9v;Rlvsw-gEdL&uL~}*9r>Df6qUz6e%1x-FgT9Zb(VNmLZs8l^!cU1k~=eT_!+R z)d7#Z>9p4R6J8#x%&}pvY#t~MKbW{43gbdoq~g?e3XqbUvs+gfF{vM6#iAvlrpUiZ zllDt5gG*Pw`1BJ}xi_PsX~r$_cZ;6WMKaWof6?t%{0yq#iJlp;Cd5?M4PFTko393m zUmD9PrMxUY=@Ee~CMa`NzFrX;*G#umBY^td_Al{nZW&D6AZBrG5a*3g*ek)}B5<;@ z)5Iu1;B6N?o#E9@-+1N4w_bn69mGZq85Z{MVn-UxRdO79FUxq9e_ek_w-n7{kh8}CdQxALlCbQ2n1{m=zi#KhmXph94%VkQ!(aEspP z+#<&7!tWB~A|-P4LBJzq$Gn2?MdJOW+`k8698w=DoE zOPoRFzY_ky6PsznTA2*FhHSZ>8d^=7r96pg;K z(Wf6yd3#)Olnnd}qlbO**{3ALyvI&^+9v`sC}}a*1U(*Ix$}XCsb3-B(Ni`PfhFL` z{;8o4{buWp@N9f!@>V$$!6f+TAa)a#%10P0z*-*wuw_e%FW*``x&Hf^Ge$o;Se*FL z>AJ&Ch+=!eW@zOeJD0A4!C7APlm$SJ?M~DUEGo~Wf%V;RmG9#Z4SeI37iz(#fBn^0 zpG}38?QOTr(zTGPWE}vL6w+2q`6uoIC;DG{u~t4dXXQ`}oFrfA3leSDB+Pp-;Za;7 z6Dt-s?%ES?=;Ut&z|RhQ1P|_ddg#OKM9kORXmf1KKqkIomOvLA0Z3C{h#?B*`XL;? z%(%p|%UQ)kciwFCOo{^&hiw@bj>)jC^|iFQzE^gh%7m#>d-0!ySGT?@5 zxT03_W2c5IH$|&$6B;$D=Wcx#T5NHxb>B_@_KyecHDiW*h_9WWvJ;p^ZuOLZm^tIgM+Zx_87c6)6ySRPw3tGCf-S6{0Oz=J8b&H;bpNo+%YHJL5V8_Sm3 z_FB1CdX2$Y-^wT!kjRbUx0t0@TzU2NHWUfg`3G*h0gD!c#YGlvM~vGlnl>Fi*gQhs z8;HkY$shLZ&5K5{;mpCd12PV|F&yR`a?>J&7A;F}g8S;Tif242|9tJTRFHY6iIg}i z3f~5&C2>}C?e#4zd9+r8B+O8Gf(ofBTdTKGSbwXbTnjHkz-(c`gTh*CeV#h}vqfSv zhhy#wfYn=BzDVZ3JGt>#OzaHXC01iGOc&$qc%g~?P@j9%Y92WoV{JTAIk@2!XT+TyzNPsdG7AHV+|I`Z>RKYjJtC-1rL^8LH6{pa8G z`C#%SP{C`#Ht}R6LNsmqSpVX=53u&!Z7(rPaE|e8mXr_`t$R_@4pQV$t#JV7S? z{(8p9NACN0%KJElfqOAt9rM&Z{jOjpQ{rW!E!wtq$;Xz4YEV-nqK(eDc5bcpWfvEd zsX6hbd@Qw>g$eFWnv5R$kY2$J<=R~~$CD<~o~_ZwZaeJk&bh`3<0mgXH4;mCwSAgl z269D@uGQZq9`9vm^r43G_!pl$`-p>%=+*6y2luguN5tR>i>DoSAdXFyOvlddS%5Y+ zO=8Y_RtX3b5CYEtGAg@ZoIYeNM)Y(vZs(08P?2!ZH(;|(hH>@TTr^P zM7&$HlhK!u7K3Lu&O7gxy^BL|JdON<6OZOf%Q9}W>z+0TDmiBxledI^Idv+tDloCB zL0?Xw){+ers4^&qZLMzjz`Jn*5gOo`TT+vIuXESFYUV4&U%)t(Y60K7Qb_ z?%ASF@28iZ9(nxk+r2;O?U<(f{&w1zXC8I%Eti}p7_j>seFDc3@8djcYqAr&UM_&`cVj{)&xEIEp zb@+kYYma~Vc?E?Fg5s3WUk*J$!z56yy5_phB@yqxmHCOFV*z2=~~%%S#^+& zm4PZSUX@!*In&Lsv0X-OIwIDSoEAyh>T}s%#KRDn`oVyjQcY;0FeX6x&rM3y+q7{*F6WEAGc zxs7YHrPXQ_kuEN@pR_0;_x<^voq^Y#{E}B^pg*&1u_!`)Yk_M>nty=>{&Mw=C+xK& z-m|DW!U@;^;Dhod1gtFeNpWXP{}wOw5`LvsS6`vi>h5Qvp^0soZ-E8=eDzHy?$t|g ze54;UfvjYdXRblT>cm@M{$_HSuh{|(a?POvQ zED^|z6CZ1L*}@Jq4IgmN`kQTO!RQJ#pZdmR1!Pjv#&}Ji{xw73=o6onR_(mndK+PQ z95-!!FnQwRgZtyP1+wAa};+|j&wi=64^OE7HvINHTcrd8X8`W)H74H`KKG!3@M z_DCgzlrv|}c;b=!w20G}-Tc*C($o@A2aE&bvUUQ{C>-(N?R548Xf5B_(j!o4PEz`; z4!C;j$)OJ}vDC7i)?BwTM8y$L8K+ofbpuQC$`WkA@mDajbP80t=Ge96=W2p#qR@>L zlz=j?%-OQZ1xnmW`rUV5Uw7Ua76AFxvCrZIVg3c0Vb?Ho#&>wxnK0V>QG;?}*2N^c}VFT~RO(-xa!*#lJYtwe&XPrIbwn>fVc)K}H-T{Eucy+>UYIP zTW^n>DX@tB7fuLvUVmd8T_j#=i!y;=$1<_F)+U?Tv!WK6%iVhIg0sLJ9Yk~7SNyNR|L(HM>XDw~A!8 zgd`jmla2!$G1TYQ*$VfwfWzsGWDk*0v!958jN6a=QxKX!z(m>2e+t+03+b$W-(!2) zYA7?DjY3Un?xB{ameq0wjB&HkjoSSO;#xCn)(RavUp{Eq@Auhp>W5Rj!l^AXaArze z*0JM02L&r2))lA}_Yi1>sgV zYt|8`pSw}dt?@kMR;^mCy-7F5xAhSxO~qWKQ3b`3XPme3*4u!sMT?f}Y|=Ga>U~>& zO9?&BF+cAQYly)sBsSZ2#}Zd;g;QGDgTU5v=RNJ|j3*u)eC*7NaYSfHVs{u-rD*dE z0Gn>5Ph9h9Y}u+UNazIN?bly>XZ$$%CHyB{`uD|`T*_G!gB-j7wxUn#B1`qLwX!l8 zOooV{Y$Y>3Ls={JxrX}O%j$#dNoz9D;Gs6UQM+u5YM1~PZ`2Nt$!dzt6IyMmkZqOd zU_5a<>k6G#oiw3%M3(nEy zaCW-WrN9Xcn3ymGC~fl81gKqG3PV8^V4gv3eEs=n&phk^nDQeYC}+$FS8vHT9Pwna zY=))vHm0kHvKEk?gg>iHY6_!O2ouSx6~?hfF2@9NVcf0 zb+vfvbB)#Kas{&uY=aqR3&|r=V_k#Bx_Pq{dF7qGy#0EG1JWyx;#~YhfXF4lD6jdO zHOJf29=!d=;M1hZ;M@E0bOtp*V#uV|euvz0+4*Al`2BYe=(g^1t8{GFerY^{=aUcK z2Uu27IRRZ3^T$#$R4xQIQAmxvLTMpT3 z!<9O(js5KS7oWu``#)T8#kRcUq&lG@JDe32542nEunQLHM7<@jNaWLY(RLckq-ofI zdy>-`zV$__MbV2CjAYtq+Z*HvOfnRbg+;>841d^-Wuma_V=QxdU)Xns-`BrUXGMKW zY^!5f|81;na>;7kXr;oak!(F{=+7 zmS`*+y{MU^t21yF2G3mZa*TKu!)Eid%lMH<%a_G)KO}XhIsj{0difRp@zAK7E;;{! z+i$?*Gb$-04r{f-o?`#l7_$~(I6sukjb?Vj=J&U zbCC%ipMjvYIud$VTefPAO!^$w7h?sLX<{cAOrCpnbF z5zngYZ@9!V%g;S`dMJsNMB~7DEF0HaDI+2To4Lj3x&1F|)S@Iw+q%T0#l#mbi3UjV zs6>QmQEb%En)3+O7y%@kZ@uGcqc^zi@(Uik^X6K|9+nF06&CSYM?52A|Gp>5#H_H$ zlvZ1BgBylCbMdLijd)<7Ftvm|etYyGC!IDg&SgXd1oB7|Ev0YmwXv3B(q6j+s=Wa6 zF4$|IdTgD&HrosG+BD8mnWt4kuH43LGwoix=Jt!X1I|0|!N>nGdD9SMkWl0hrRZg$WT!Xu~Bkvr3e9u@0b)$udZ;mCWq&h;Z6dqa{P|Cs*7% zMw;NZk48`7*%`#6#H=zFQD{t7Q=b#z@yWfYsqagx^-bcUMzHF=yOxtkmo`4w6t$0v z2I_+gtX^*;(u6aJm}mv0^+JpEJ?p|7hK=3xu)fQ#+yNNd1Q!m^!kKTr^|$EJ_pFQV zeD-y`(}d?T$Tsz(k6wH3Ny30!XVY#Aw+lD9NeMkj!}Ag>JM}&vu;FJ2ncO)1`96mq zhjXr8tx5zp>`;-((Pv(G`_tnNI_?x)gTOSDwiA5Yiz=~VB#7t~W3xiuezNt1U&zNG zfdVE{x^EfT-ZXoejo=;<9w*#5*ZY9qkWd+x;+ob;kJBqXcilU3RD93hym|9ht%-+u zu_c!TxhCrnTAsB_Up8I*$yWa2r|L>jXdS~zT-s`%r`?{~^UTeMI_g~-lz3+=( z6h^I|=>Hk^3L{=DD%JOAhJ{h{Cb<^NfTXomI>a|=z+ zjeVC`)OJk8kCZ%EXD_IWl$zEnvXH&PmSZgyvJN%HjBQvg2e%DG&W$f6+Z74ab~%og z3$=EAd2Y>V9j|vV?QU-FUR-UJmqpdL5v(h!Mty!3g{M6& z-p@b21Mm_ei0=U6w+lvPDsS%2`4 z|LOR|rJ~;+L5tr_EuRMZoqy5q3Z!JvKJJ&i@{j+y?}&fuH2VH?nHxK56@kPY;*;3e zSIo_P=Gyh=z;BFl@#}XypC96Wu+)bxTA1+Y=YO%|2;%3!{?T{r)^jK5<*`b&sYZI= zXQ@JAtt(cmGHosv?!W)}YnQw3`mulX!{7dA|A)9o_6>jZKmRM=@@M#Cv7n8=<7so& zgk`(<#OnrI3}UUWtay2lBcnnXH*R^}$CVYII>I$iss(54zmn)J08P6bw)$9(T$%~H zTfbCLPJOGPl5w&sZ;pG@ubJ`dvxFC&#@8`g&k)tO`Ny{k|H&Ww6W{VZKlo)|`$t~+ zDX)ohTOu^CiU*$m(CdEv7kurX`XB$o_y1^oae1)5FQBH;B+IkPPvs$R}~%2s_I8J9)T38 zBab#`JCC9=pWAW$hEAAS+&BL+RSi~?VW7>Yq*|gZs;;FhXS<>XmuDTvR#XLR198+# zTXANH_|(aWYc-!jFjRwr*GI`XS%nr{{5JEN>E|O*#6_p^HHR`8rz?BK*|+uL!X+Nt zji1U4S}2R~aj*Op@jKc}mwMp7|GWRTkE8JTD~vz&m*4o(SA4W-Ir#Wr`pQ4~ZGV&Z zLE?*VJeIjtx1aDyuZ|zNEq*+-q!s_x__DA5Bf5F#IhRqye@{H|+rRP;KJdKF?=*#g zk7nGT^}@S=eg5wEewY6ELWRHbGd^2C%Pixq#@u)D&*B6g7K{^7P)Q*+B;f885AE|-K($aemfUCDL+ znomi!Lj8^*B>a9MzvMxQu)$s&C7E}YSFS$mPpZwZw~9D zygTRiB3pdc0(Fssw(CC{JHVbe=^A1!bwtSp;pR4Z7*GH9LWqeatWV zJzx7@#DDRr6F>bg|Ha#W`d`K8488Zg?|tM2FM9BWFL~jMUlQ+dp^{GF|M<(B-<5jw zi@#*w>MZLu#;KSfM5UlCI*R zKZbqLOJ5$}JViIpigTumbpb?ctd2W`u7XUv?KmG6yIjac!|@_`k&_8KHP3takx%)o z*Z<#dEbp4($7g>27nS#G+@AUNNwxpr4}49$WyagnJ9tUzv0wOwO;s~LTZNQeVm}Uo z;A{l(TLAyl-}=5k`G>yhyT9`(k*!FjQ^&c+()PYDx?shNw>G%hCO@6ept!fv11Kr6l0!+zB)>!Yc5o<_%yU8g!@#8oAPF-wfc4?p8? zPvXD$tMz~W*E*Snu_-*e*+=0D~WesO$ciO!b%b)TBiQL+I( z`h{N-|Go2VKmDWs@E`o#+kWPqKl}E0|EBvNe&peYU-${1^s49HSN?g=Gtho0IEz2) z*S+DBe&AQV`(5vf`x?)C_@RqmwL_fqk*vOIzwf>8i{1O~k3Z9V$nD~nP`>>K-txAe z`q%NBs`owb{`ey6l-FoMFL~G{3Z?cl_>vuW&1SUZe=)0h8&|LXUzw*{ZmKKhi#Uh%P? z@Qr`vyZ^hd{mQ@pe|@*br51mH_65K5%f9$`{l55a_^5tY<&xHLw4?rBi)kt@M40$YI&7YgzENuGo#wIgkG$5VPpjzB(fp ze(kbsII#*fn`OuGsSYFd0(i5pG3zOfs}D>N5ot2Mr8F7;ZTcFI;&@phi& zqfFXkby)q_iCR0$c?KuG;Zv{0iPYLo{PJ6T%t66IGyZFHZao7l!lEd(@4a_< zp>O)u3BV5Td;UY8{F$#0wGg55LT$@I0vrH!u38wq640XE6b$R%#|P zedu0Ju$BsWrDAUQoIew-jb<3z1}tsMO7%fDwTiubKJ;J1>QDB_w5aDs3tnIZ#HIfw z|It@`-*^2Dr$Y8~-teWdUFP4`WuFoFe290t`KIssd+{AWe9*Eh)UJL9@OmEf2Ve98 zbKg_Dvtlximiq0j@BH)MM6x)c6MtwE{@>l+p!bCz`H@9jtCn0$ zss88kC8!djG~!)F?Rn2%zkg~!2QDkGo}XENd+Trh?|+`(@kF^5y6<=bUngUdtt`tZ z`;sV)Mvtyknq`ZJINCxuhS3pAuc#BwEV~FNqEjBc3j%yxX?qcWna%n z6Yz0a@OECsu3l#%5VMF>6Z-yS;cuI9uhAFGb6C~Lu6^USyML@|zF0Z{#TBdS?*7-` zc+<^K{jBy+i%xV|&_*#RtHXBv1|x8_cJ{#+yfFT=ALG4T+uwX&*XA6*W+LG@g7I)1 zMR70n#V>og^_xsP?X=%qIUB(0q-8@SKAI|S8$}BD_uu)}x4xs@ynE>@e$k6x`ts$sGQI5X-(_{Z@lAVtQ2NjR z+&lm2kG=Vy|JaXw@Pm)t_t1kM{qet8Z^%^|rS%p_Cqaqgoie`Y)t~m-_*rIrWsjd_ z#?VT>{9mhE_dK^!Ty=uNpJhJxzWd5gJDNfW{^k0k$2bblxj%mMWBt)%P?~)I_1(BQ z_7lHQ7C#n^A7oaw_&Q&HEUIOS89twdpJm2dqQ+o<_UGR5jC-E(f{!Y9MPsn~xwtf4 z8|CFu8{g6Ew@mWV_s{a;t(K^nEX!>JYR?>l>q>y}#Qpi@11QTag?QAv3e`;c8=}h%(sr(E!+6zjiOQ?_7QuuyQJIG_dI*~P2>1gkig0dbGx1Knx8NC)brz@aujl@ zvHQLM@;$jW>n{jjK1ga3I$cU8Gl7WSO~AD|OJ>CJ=WW0D^IjKCgung`zvXMb{u`Tx z(p-3n;=x1(Hd#;>9Q=#~-9BsSc7xh&0#%bah@S=aJ&12k5m~SFW+98V44=pAYPHTb zU@gtvJ5E-NHg&`y zLV8vpGiT!hU&}8|!|S@)^&7wG5MOoTCwt{PiRYCc`pFwtlw;YOuA(O1^5*~dKYaAl z#0-Ux{OFJQmN)%BUKhLY$S%`p((&%!XCjA|%#^pw zh%%jf`Yx6~RW5?H3>-ISb-u{uviCI65^UKhDO>g5#wyo^bS;-H%ny^@=Y%pKzOOCD9xC3zO25^L*B8 z!C8efYSR}^Tou8-hm8W1slXL&@a>MzoDJO8?rZ~AmK_!dN6?I}R{TTGEwK%<2j#Pq zSbIrb`kurNw|)9O_ch;0Pp$j*qp!b#El6fJp^M5{p_LZVoZYm#G!1vx++OqI^^Mil zMfy2bmsxfnSFKm!SBAy86Ky4Vjr7B_ePM1D)-^W4^?A!&@fd9E=q$nvH+?HR3+#K? zC_tI5xuOle-B}$HZ-$(0z)G^i0-e|II+%7fVPyH-Tj-6mGiNcgLMtu#oL%(wbmCJ?+9|5HNVJhR$=|_t^(Kfy7!m&uVfcxGcx@K*KMq;;@ExQ zQ-9UW?ySnSR+|~3!mMf$!2#GDo+jY~(9N%ursYk|4EjsSQ>55 zsE6Xu;?iVPl|2L!KNtPf&wBkL)h^bh%T~ENI#LVf1$!ZlBN#VX-0MP+RuO+Q9RDre zuo#3*vNw`bIlq=wmpkZflW*($Hc7b}|9(g&}XLj|>%R}It_0W1b#<2C1H ze?M*&ewO}5lTMo2#p|~_nbzrTq-C%qJAMEh)uyN>glzCTeEo()*_B(y!)X*kJWwUK zS{GN>h>|IL%66+0Ew;pGvB~87nqt+$96&XLX)nxu>kDal#jc08LBr?m88x94nTixH&1w$uX}QWEZzs48q21&ddIO+$#Jm{f#D_G_~OwtE%j< zxPX)OG_75S{7PRBe&;6=LN++NETXjmHhFbcSRI1D33(j1@o*YT?Uq?U&5%>AR6D7Z zx^_a9qYdGRz3Twn5)#i7STv(y`y~mMXFf-JllRVe6mbVWl00d@=Q zc4x~O;;NlBIw+T5LfN2(UH};dj!;BY|hZbMUYY#$tHAuj7M-$ z(6zyo`L1LLvV|20ewfDH+PmG`z9`K$%d#)y%oK8jb1ie$YlsgG1lTRG+Z{^HK-JC~ zscM+Dpu0yvCS!A4byYyA6+RQvBr+yWIPxHR|JdZfeJD=4q4k)S}-{8_&al>m}2#nLTpuAwtBSVXhcl9 zZNzejsMc}b^w+=LZ?X`~U-IftEpNzDNgGY=gjAK2_`2O@?P?`N)HnirowlM)Nw>qR zjPX{v6j7~Z)p7Mk4%O6jTh(id!)@amTjt#Gb-kEvOO_(>i8x+s>Diq|fxAn(Y?~a_ zK1{FQ7}jXf18kW?&9njZG)|sz<}=%?Z1>3w9nE|--HxLX^->j;h%+W<5LfKIXH?T& zw=eoUAQnW5fQSYZP*AE6DIo|7A}UC4p-M}TP6DBWp!6o4Kmd^r0W_g^=_Mp8H8iD$ z5=5Nm5{I9v@nrqIL+$)w~@-7&RnDAG?FB(5I zQ~PWyIhp#95{SK;sW;~RW%JwC16JO3X z)pA1}2I5cgJ(nPeib3>T2|?gtA#uA6jMttwm`{f$-HOfAs6;H_7pgnF%glC4MzO1P zz5?M6+qLDZ(p|GPTd}XmVI=HxfBv{J$afq#md;C}4;i?p_U%eW$lpjx#g-QOF# zUOa!k)^M#XDM2K}zX{r{^;{&6l6R}szWqRE7*D4Ck|Xv zQi%)LND8mcVt?WD)^LV}W}tNwJI74$+=X;^KEou`UtrsxcTS&b{L{i`-HjVh9$3f8 z!;P2+hc^amQd4L^y8kWEibI#Av%MCC(?sZ)$)sq=JsWRcKEPw=S0kc;_o`~EG}kfw zP`9P2?S0|di(mh=3W*kxp41J_AZd}mhIR`)uqQ)=SkuN0j2ur&+pxMfWw1H#cm{9D z>-<-P{(Fdo;8ufOud-&pw7GER?WO}sVh>;7c}=azVE zQ2ZSjOulJ=pLw%dyXe7uD`S^nA@}Cyjc867#3u3b?R4MhA>*`g0ftpk#@*}cz?l~# zCwH8VZg`-VkI~1x2Mx3`f?ak4Y;*$+A?OXDA+WT zocWuzZgllDhI>UoBpTW@8{&ileYE1vP>R&yIY)3dK|w8 z-w*;R^k6FA93M$n2(QX<2LG0eJ0CC{v+!}g)1;y~Fwa$g@Q~22U@lP($9mi4(?+;HBrrqyPLEOK5i>M@D4e1e?%* zcGpUz-)-8k-R(5!ssyW3ks4{!-V@m@ocB_9$IZO1h!P>;hN~tMeLBlGvkXm_v}0We zROO%In7{25-?-?0)GUz56mAXF{Q5k7=kAPn!P{XSj5GAwwwrmZN!TRRWhGVgabWQn zbgBHYgPrLu@@|<*bx^ZMPX(aT*P=`NQ2rYGmDi6f24aY7wag{Vk}{GxV2{((dEH8m zmbe0y;W)-@u;y~-AuCG&Pg-FO&(IBgYW%gxpsnzz8R?FkEEBD3Qsy%ZYhEnob1JsA z;1yh1^tqJIh_HBW#XCQ*ec*XBQ}^V2+&kzJo07N(t3j4>e)=GmAm58-UNKoHp1;-o~(6MJg82nkXstcs&p~Z(^A5dTIhj&+z;bQqoj^PU|5+4NS?#0}+{Uv7) z<%QH;=6PyR#Cji!!{C^uic}h|)xYVvwE<1wb3VUOmsJ3*Gm%?+JudHLxj1!_RVpqG z{PsO;?;JCVJLZ{rrHj|NvV6TvQhc$hflO*aHsHOc{eN)MA@1>^i0wZlWBXz}?)vPX zzso^#2dfXd^~tZjT?d4(m!UIT^+pPy=jAiR7GL+`Uq4*+^=yQISXUM%G!#EUznW!V zff174A8<*YFn?rb29Nt=BU=QF)+rVjK~3|Y4!rl|fG5c|hBpBU-cO>Bjn%EtfOX#T zew(s3Ug2u%u+*>{-PlE6hDbuS1i;f)I{Cg(&Vae?wjQ8=aUl;blP?TUj1#XTYP+IjYW}bhoz?>HYFqvqEWZj`EuSXb9=S@b`=jxI67fkF0W9 zmupi}6QRYK@#zCZK54Sdx>R@JXHBj3FK-2>EV({b3D`Suem}$s3*79H71jB>@2u|q zxO*W*?^Q2BW~q(pyfSh}LCfztq5S)pB6mWFTerfWG+$Yqh1hY+AFAtbwf^|C?w=HY zu5;#8%Pptp^+-jG#s&9#lzdZqM>td{{QWMyO5Q%z4)h@~aV! zFArg;9Fdpj&V3kP=(YtJNUtu#N>sdjRB73Q^&ApW6An;f)mz_^HyeR3#f%+F6#dwq z>!H5)?;6%un3p(G@H72FlzkI*=sGg?6J9^^Ciz#JfIS)qE889y{w*oi=)8S4_T=WD z2?+sR)(tsiIb|-UxW9UFPzreqnsK+Yk}8&7@J z7HXSS3A7&A=A(XMH&|bj^Rs0ZoFrl#!Ww@wGpuF?Fi(0`jaNHg5T>L<%ICe#HSiDd zlxqp98mpqesM1Z5ZxJ_uq|?3bY`pfKpZ4*1&Soeqqk&-Yx$H zhCc*}KPfq9WkJpsy{baOxv{wsZ2{r^&=SZ96Y;!@gcKD-`5_igrJQ)-F{4{xI`f2; zxKW`behfchod53)YCH@UI3;_L;@CueJP|{j0CTDBpDs>jzjF~gp1DKVB^+>N6@QSy zCgYBybw_R2@TJhY%bVH{s34~?F(IV$WH2KbYPfd56I{{jp|+Ovg3XW(=9KW){$URe zsI+nO+@AD|&+XU~RMIcJZm$2+V9jj2aZUXUWdBsUAu3vWxs8RegJ$GpkG54T~OAwc!eZ8io_AOCSbhtiB{^TyO()FgLPYx^;O ziwL{8&xN~Gh-zV~+F4X*^EmQnuij@(lt6#X1;7_7^S2{L+0ZYsT9Q?Ch=D;qazpGZ{kk1TuW zIO3>1QZcwyRjZaCVRIH81b=3nB>5S(s-C-C@ zEV3Sauj(pk4^6*imWF-0oGRxTQRxo2jj{VatMl%?(RWj)lbC-Zw%fHUOUh<*@br*t zqDF0M4f5JGF;urlx*z2^imDu$IfwfCza9#NtJUD7;hwYM<`jtHddQ~s~ zo)f;H*laeJoB;VXaS=f5?ab@y5?}C`>oD0Stv&aj`eT3>D1SF3Ry4Ug0XCZ>Z0t*k z9V7c?J&k@`tl5eSLyqtq?Q?DC!X0nxI&k*W~eR_Q~Q7cPe%l(g$WPBB~W-Z!f z%yH8#65RN6po;RRcPgwroM+lTy_pdpv)<;CcH~olUmDl72%#@BUKD-Ob3Co$R=lze z`6YbeSOzMuMMscE3OZ-Y^a=&unaPpitgyzq32meft^2O?h(CdO?FiVJ>;>7nbc}Yy zxje*MFfeB#U)QRiHt9kUPWL6g2J2zqsUPQ0H~8d+Jq|_PQrTSI zI-)yi*id>EC0$I>wjlnf4&->E7AQU}Gn9?EHhu54xv!OHTFVEG1sR9-S?*4xGJ32s z><8hR<>echyN~|F{QFYl-_8-b|+6 zr^d|&`2$IxD^L=|>(D6Epo^Zq>ZWen7otN7V^KoP(|P4?;Z81K%^Kj`T_Y8O$q1%c zy>uhE%Naf*z5g;2QE>9Ib|skZ{f)LTS_iq4{3rS3u}f7lBD$#Qmt(6ndm&Zl*M00% z@61{`bX-Qs^=`2V?dk-A=XITNW{iqVf#AG~GJhrWXHzwF&3^APH4G!#7C~;7-!-&- zD9DR;oGi|2QT@2m7*jp$9WxJQDVvvrW|xEzJumV*tL|RtXfa1FedQ_08D!ehM4u1) zV!Qunpm${q3hFypLc~BH*~>ghs6z0w2lKW~>_x#=qDBzd za~bdVw=|ugy+(%TF;fhNtcn6eH=}j-spN+}Y|BbRioG|;t@!9mzs$jYI~4ch5_EWsnE_M zBm-O!5urESpcA>6lK`tL+q>lQGGqSYPR<)H`b+j=s{eWmfGTD!#O4t_p|jTSmF^|K z$a&@F`Y~XX_m=WlLUUOES1!%Mr^tgRpYL99a;_E!4|()lgrZ}-<{mzXSrlGl>N_9* zyCZUt>9D{5ZAxPsWBb2x7Lu8`Bu94&o9Jruj~;(o$HBXL3kOef;+sr;ZdAbWDHm98Rhm2(71HD}QpYck~l+pF4Z%Sd_EFvpVjF=ytk_ z`n7TI>HC>>Qn8jF4PS>dI3XhP$*dnwhEJflxjET@*)JpRq~YPLk7|ZBCigekHZ%(} zi~LOnMZ^w}oTO**ZLRS%r`x?~P}eC2pZWXo8ScG|O^!Nn^{`;5XyM0krCGnC&Ie>u z0sP?fAk+EPo*|bWe~7sqlVO%ygSlPWSJA&b>nhsT^#G;rBf%Gv{P#`BZ|PpGGzC7A zjUNbJf*_@9kuLZpit^xIQ((aK8!1JW>ayHbjG>c4I+tZn2x59yWOv=OXe?>j!s6Bh zL*cyXa#nevcHQD1xN~Vl*~jbdT|ZdEx7+);q&o6s8gj;5@jYt6?32Hor$V*W0GY)% z&`Ji%I8Izjwq?)&I;X#~b@y8peH~@$Xjg=9rSni5DO5>qkm=forRO-eiK0j!*c9 zMq`i>XvDy?QJ7;nQL|iR1qt3JxQxnP4W7?ccF9Pvnlu?U zG{*G~U5`rW)I))e_Oi!sHo)AB&t|_7S({!46+hVdoqLJ-^zs5g-lR4&t$PC996{^I z_%s#>mXZ+OWQB8ykcEpHN?nXX>CAtWc;20MQJWzR9zElCQ_$~y1(XtPVoY9YNy>fA z!FTOj`oAyRNpdKvNkWcN^r*qAy2 z19snv)E4U*CY}KkQ*g=^)|U5ebDr+aW1hXQZV&wcq&{YzN73!#1k3iw-6nemcuFAJ zF~U+&hC-bL;3A~9f7ic7jSxvn<=S0h7akqPr0OXltwO+!w(L_?v=jWQI*54 z`L4mYeX!b-HwIjytMO9Jp~Fhf3MxUs9Fsds8bw25Jd>Mi<>Y5Sj10(DDH(U)N(8!P zFEcUqe_JJ>^CWO5ichTj6FUUw$WGxw3jqVLiZ6!hgzB|>8M20vTJKK)dX|upt3rt> zeff%IP`Qz1edwv+s~Le-?LlcJ?)k> zK>PjGtBJe&(y*ELI0+jE6CRF& z&2Bux;1Zr6B`zyq8gS=+cd-zX<}HKXtt^UKfrZvB-}hF)c*3mzrq<8&o^LU;EdD_k zsvcwxf0}aNZtZd$6Wi)N0*PUaoUZVSOQ$|MmnhGcGa8}29>wBdFd*b}97WEq1*TFD zduNBg^Ck5}3w>oG_wn%<4$jd&9{F}$2Sy_lK;$USk%knArXW%;qW(KV)`1XDp}Dl5 ze|9ueXOib`a0AP{4F|3*EL4lAD+uThNbaU&`DEXRg`zaOVFZ#$j~N5!`y{P9Ms|Xr zn65kREyI0ZN_)AKU)e=2SU^w)kR0>$M>Cwxj{kXc(-9D0u~4{czCG z+S+g`I@a|D;7&OFeA!(yvAI(yi3cLxQSyv@>~1K~ZZb>aYyM8}dHvs1Pib5bWkcrN zhoKEfHxuwvMoF^p;@jwpHWea3Sbx1yg+UsowGu=XL*X>c11bo6E zd`92tVanACR6M@9_@nqRAH0(u6tjPfwTUYLQu~}X5^*?`ZNSJ67PhV_s2#Ko*nuT07VZXDdBe99>x(ROE&2nPSVNtr1r+Nb*gTOV40i$-cf*23xv)OuYgjBy9G zi;(!!c()g%E?ptsvxCfk>0c4T2qI*l5a&?klHfib*hR+@rFmEmkU=uu@ukpyLEId7 zQZ-44%~!A@T^?s)OOW$f2QIbT;Q4Wg7Xm?04j#`1LHe0n)P~}2(UEbxLJfr;{nuP` z82;?}f9me}Kk07yh2uK$z-$cjQXVG`@mW$@8{(U%L}fsPh*2jf#Hx0%q$kvWZh`0l z6)Dgf(M4T!N9v>5gvlGAOusJzxYDfq3l^n=a-4Nh036uJVUYHK{Pvre`n`*ZS+-B* zH?BAOm_J=cIUF;3fb~AIBRK*Axr8kfNtmfG0exDH+w1MD)9p5koS=t!Rr7Me@ zx5mnNq510fB4pjMDy8l&pu$N31COo=0B$TnI3T0Xou}fcW4;-7f0-gFsRH(yoS$w2u(&j?8(pRPRm)WPp!*jfUd-w=`bi(VWKNsP2yqLx(){{zx~hb;D5%D4y~L@A`g;f2 zvgxt>mN|iGSVp~J#KDdh1AI9gFc*cpaS9xmm=f_L1OVGDAeOy&m5PEE>rtrz3y6?_ znS@b)IuB4$ueVvM-QZrCm^<>0P#Ifpdx9{0&*-5IBQapb62hbd^Q^8lA?Za{3kJ3qtU*LV*@Et(4H6~2fQ-IjkDiD9nI6%=kT=H zEk6o<$~luEvG?P(VgN*@G*?O_|gzPu5?1J&H_2>zF-2c#7HnV;$C`>!o0d6O0a*2 zs%%&&o%bJc0E1cTuXX)9P^o05Q7(lEM#~=pjyMRriG6bWc7}`#Zp@u4ZiTneiuD$D zk@yoF7Y`$G8mL@&q79_3L_t>Z9Ca4(_j+QLMi6*@n}LXW_?&G35{BMIr1GQ;@`MtP zEY^V35uHhrPjB_TrIQuNKg=qXRL>HS1EE=VSM=n@)CFzs53%6d`%r_i%rM6G72y8e zPw5vMzj1C&JrbB3BDT;y(vFn}GE}4ic-3Zonpac6QGFX8_Y~G};CW0Aj#BKjanFC$ z$Q(Z6-r`W5l@)FlZM~ADg%%l<97!K6)BXHk!!xAivEN&~DSqoIM7S0?~&YB9%z?m2>ievbb0wAij3-=U1rE)?8N{A^~BSRa` z6smA=??0R zK3lbJkLL_sxN2=auiQfH(>M<+gHhs4#J8J%?qaAUntX_#t@(mT;dYD2DjBsP>L;m= z=7mm20t=aU>-pW`2G5F9ZB|7FF8zO_{sv~pFZ+~fd!FlIzf1Git%?06CFKfD4c}{6 z9}(>*zzVqt#X<>Il%fNaJiHGau?~Ugz68zAXV<|mBz3O41L{vT4 z5oIT~MKOFnql4C+(vyWqZykGZJUNrCpPqZ15&bPY1?+U zZu()7Vp&COiBg$er6bJqsgmQ>gffa%p+=ztZX!_ZFs1s3b zs|Yak`v(&!pY;1MBZzZrs(-fvBSl%EaXLqX@WXUNToLq?W61Qu=@&t*#-b+rFl1U? zpu?q8vT+72{M$4CA3W;4D){-dE-6|ub&?y`$WuxVNPaxCRVaJKb3Qm zWbZ1ZH->bnTYU5UqNp@#a6c-2P_pscO`AP_r5=oKDtgL1LJnisL`02XB|y4@pX3Cr z6Se-3te)lf33u6I!6=Hl>%u)~Di1d<%6~3u#W@U6V$Ib}JD-^z&1yt!9^>&-B8c(m zBOuhmXd@8>aSk$s1e+Ikr=Rr^CCifU3VCtyGdpr`BB0e2^|<87dxVJeg`ps1I@=rU zOGy+(Q}Fo5O;8ldPOTfmpHpY;o5`nEIv%=(K`x~R%CXNKD*}Q0{eZnsXECU;ltx=k z!^s>2!`ED;l7&Ir2)wXeU*B|SM2AWDF&ZLvoG4(&9|hzWJlx6Qfj-T^=xPWyCiS)K zJd3l2k`g4#dW9;|XP~_a!bmHHvaI@oTZH!qEks&AEfwBetYov~xhRfG8rg%TaE^yM z#H{3;0B|oMaJU!W+~?FSMVR40%ffS1#9U0o3&m4dICp|z8AnpzhAmd2IUz3{g0#ic z-U1aVtmTI1N10zII>203!hw7(n?N9<5^p83N=)}LuUB8Ptv-d=F(7kq*c=J{%MG*( zh!+aR1KzX0ykuRrC%KhQmMbBOTB^PiqsR9uef~(Wg6IEpHUBSZ-oLMR=bW~sW?(+S zPpE^)9!UApvvFy)+3T2SZoRqH!@GuGJie!#PfQOz@i&1NcN_a3=Eqsj{B3Qpcl%Cd z8gKp!T4V(vbG3w7bL#E59CXzHE-8x&E6$^`YnS`v_45$ItVO2XI~px+rC(lpE7CRx zNcC{h4OA3t?XKdQD)|`T%bMihr8{u8g3?%Bph8I8no2RaaMUeAIG|MWn$j&b!iVWd zy`UO#IF@;YBT3-Z{cmJ^>w*5qLhNupGfMxL0krn#Uf=^V(;RhAz?m6=O zJ?V;QgZ2_7Qwfk+=-j{ACeNpc7!ap{oqnSyqqa`xd#hnQrQbn3S81ksX#S^ZD92m` z{5o+|-G5YFzJCL;9o1{3QA@kis6^>~F1vCLEWh$pq=V2l+$l^IV-8d zyT)P;XYoS^nUiGnw8gsNu5xWBrz05L0z?O1hq3r%&$%fsXvcmzIFx%CyJnr@wn|fv z*(g`ei!a=KIY~1{X%lF1GYHAHh+h&P#^WGl zQ0w;iv?&VhX>|^Nm6mDu_EnRBhj_*`x~&*(z8)Y;C#>vtct1&t0L^(?5nG_8Bujmq z6H_9pu!eYvxvNwu0*s`1S*b~tw8gbkP2S(q8DLa9T8hJbQjvlIDqNPx5zvak{$pUS z=?IiKY@0eAIRtWqZQ<#+NW#6jfqO|fl2C%9&B_pQP~n@9jqqjS5&EiQA91Z;ga6ok zJV8VFbKdW!YqM&@*!F~$dV#X#4|wkIvdlx!CzK;@`%GimkfyOT%Fb)fgD?S6pg#U* zijZ@<;I#K*VZ!)m2LWW+=}Yf#$0-(dnru@Wq>_3AxsZ2~7%BG@Dpe2IsxfZ)t{c6= z@^=v3v#Zd+IQyjuOp_9OgMss&#w#c(OD+qCyhc=D%_$E7tiqH)nE_YpWl|MSr||Rv zQia&x7~=gesV7Yp_&eC2p*(&0bYMoJBU|88F%iH@CKwyV+=q6rYe3})WYp7s%7*yf z#US&MUJ?W;q)EnPb?s;?vM;9cuFqTLAor)Yeu}-w>iR!;JD8sRdXraJsW(w2Vsi-7 znxVsPS9==wtYWb_6>~qZVtYCx=xBrGbHChW0fIS22CG)OY}E}LqC`cc@3fgfX-6l6 zT3&JsniEH&Z~h+X#&^s}sjk!-JS`32X`m7BL6ECVp6Rtkr<5d_4`PRHHBM0(2e!Jw zJhY@NV_{)J!2kX?ma<~nL17`?Usx|m9;J1sk)L1i(P!FS9mZ~?yr+KCDfS=2U+ifK z;%LHynj9?E6!%Z>fhU(zeoSJbDtgK1Xmv0JJt${NB+>BIAgKYGaa?`O1B1*BlL1i4 z!>B&mF*-`43)hEtGh8pznT3AYsUIxM_PI^iXb-iugOb!GaZ$h)%ap_6+%o)~Po+(R zBAns<^}-^0;nmgzECh0K7v_L^A&tCKA?KKwD)Q{F26548ot&8r}{;wAw7;j@j%q1$xjxoIb5&QSn|oyxPYT4_(A) zDUL`bIqayIxsTov8snW+4^Fb;+^vs-Xeg$X^iAM}63y9sx)3?M3r!C2y^>Nug0@d=w%~s9>evRkB z1rlSzTxsIh%sgBF8ix`w{~CR?;q)*0j%Cu3cmZvAT0H+~_(xfLF>=r)YP4nN=bg(V z?8Y%Sf(g)(a_{x=4FV{7 zFTPUK3yyGvx=Alx7suqCSkd?2^)_@%%O42ezs_Q{Xk0|0&BmND`P?PU$&U<%;WOka zf5IzL{tQ5UUcXd$kw(RR=|`wnGUiQ4n>w0)pGZ^tO8RvaMf)%Ldh9Ib9NfB4^+SXKCXAEqQ;F-uSV)RPniwI;IPp$@>XvesUjNdYx6N0a%jy|EKZw_gsjd(fCXZS2 zuW4l=SoNZsn zz}oa#b?2VG#ug6smzI=6>kKyNq|~ScJ$?zQyayW5X3u`1kG@>dM$3-rc>-02N)7+; z#~H4t3EMljit2#MB~j)@j|`zVuL&(e>qYs4P_`xMxUb8OyMobK7su}FR2@c&E<05F zAO8G>aSrRJ+Ap=QwZFWV6`M5LoIhmKp#dTGAEB_fjo(t}?^QhH*1CRm~BqH(F*g>Gs|>fEU6S!l8taRJ*JpdFNX-_O`a)8)fph4J;JBce|I#gLF%}HfWLFPW+pYLY3Gd8PHXZx6M0j$NAiV1LYN^oO=7K^@F4r$ibJ z+9I>Y4^sYS-p#1_i^f4kkyZav3$B340`z2BBl1&u9&yGAAT_WC1Z7goOK1i6wf1w# zI?m{anlFla!cbUYETezM|HO5?QwYfTF=?#Ad~5^ypz z2S3;@3eQoDQOxB;imqLvfdlNyFb@|>2MVKo&6yU^mAp_@BfVodjXU*KN%@!tTSHCYct zwxU^t9lOecnoZZSy`SD#yNyq)`FA;{s81V(zpNu`m+sgDU5^**55B%k;x8p~{fwKe-Lgxc+aHC?~c`vziV+KQwx z=0(slWevl0uhEC$omQp#!nZ9z&9h|7@;-wnwJu$_oNq-rpDHUa7n{h_K{8e@)v2uE zj{hrHWiEc})mnk(BBDg0fh~V~UJ3&7sB|k;g5^j2dYI)a*TSc#)HMk*J2|i0G|mu& z&d4NEAE#a*|NPvWJ?6^Pfsm`T%wox0(}7(k=3HCzk6sUpr@Hwx%UAyNjXN|jLYOX1 zh<8(uoQoobuz~f_kzzyieO9)o0bNJF#;~*8yeF3h}h#_}nkFT{!^X%|bnCJs%q338-2Kg^gzT zHhCtSsUo8t-z_KJw0n;=H%tOa-Qq&pZpy!I1mCzt@&17PzUo8=4s|JSV+7xoRfj0m zF0-1-k1wA;$x0;PIhsK!WO1f`!)5fGGy>TiG!*!O|B_*-hDY3S-eTJ@9gKb~;OJs< z`J**xic9)HqSl%O2kP(E7dyl!kYi||@+Y=icC`eDcSXV}_D?c^MC1>^w8Epnc&0~-biELazHodG@vSD#MchmXZZa$H|ImrVA1j$w$&e_ z#>X+Tr1g-8In{RSqz^5Z!G?Z{7LEfHTZnP4(?K6=#M-L$NB#`nYv3&Y*p{zm#7WAK z@~(fx4_1z!%~8MjOFlb)FL*-UH5JUYA`DpHk-t`meQLK(ri}>r8@6tZ) zRbxJ1A1jU399&&Xu=q)9B?sp0Ri8Blx-K4-7g^OuQ0p@id=p({P)Tf@%i}BlGfkY! z&s<^TxPZ7mO`=t)Mp+;Ngm-D-;+7J&&lr~aey*Tl8ZYZC*qewl=>{fxHpJ;j0lv3z z*a_4?^*uwg4jxRFycx^r5dF$M)GQyemGx}y^9RXBa@~-uRaoY)+rA7SKi1hg_RGkx z8jz|}(E{)PP4_{ZnA=()E`D8?G<+(r*NU{#QxnnjcA!%a7Vqs*skyoPxWYa^NrLc(yqGL_%R5*MH<);aD?-|7{v3hjes zcLtfG%Bm8-tT$Kt1&crE>2-@GV8v|Mq~_YM=5J&p`y57&sS-_J^D>JJ&An-r0fdaF zPyphCB+R|R>2KJ4x<^7t)HP{&TTJTH3}~Oze0tbJn?}uNjMl=VsTpE5bdfRj+JxXb z^Fz*3Q=6^A3r8|-s;y*cPh^iP}!RyabDLp;{hDEyaAhXvMNTsT8O6NNC!FTG_3Zg*2M+N8Q5iv|-%c74S zT;mfE7k#q2VPk?d@dUFxLJgi=cKyA))HiQD{w6!Rb1J?ON9w+`B&wMv_R9(ICRSfN z!AM^>wzXUM;~)GR(!Y4}kt#1;Fm+8btrl?+PzLK8MG@&l!&;F)yKkb| zbKIwfYFl=#pd^Y#Z~a3)uWgjppB^?woW%UiX4=ELqAENS%`w~VLbb=Uv+{C3W`(5~ z8r!3Vp0bL=i*LD)eHvi)a@Cd^mb544Dsbe?=KeA%TYMdk%LB+Clcwx1khWM2Iw)=& z{KNzEzLp@U*!X=4#y%CBe38ow9CF1&Zt^>=t)wqTY~w8n;?IfVH=iX$>2to12CS-0 zRWD;Zrf()Qc`{8P?y+>iyzkYGI)KJ!Dfhw&fikiaPml^vM;?Fyp}oLBBkX1lG zERsd5JYp+-T!gDm?G@ z0R1Xk+`Nnx)h5hHIdFe2E@v*5Uv$~8D?*HK2z&tm2Qb()Ub(%+ugQuFG^lA5ed=$Q|?H?8%;#i*g zj)xEjU#7qic|5i~D*9J&9Xb&T^Imt4&rsE`i$3E%LHc$1&8F5b%6yx3QX_YW-vQRt z(L-Td2DVWip4Nuuqe%SrVAP4q$VjOqaZw74KS^D&2hcBbORN(Iy!g0>-U3uE-X*H8^)_Y}g zqXE*%zJm&Q0ynjzulsh^1E^ZJ*~)6AMo0fN;9+q4^h5Esc2TQLp+^_TYGya)ILk%7 zpO@bdS?TR~`!l@&ev{VHnYnaQ{|W;*J$XLAL3m#+EXnnyFsIt+9GN~I{=9;`(C-49 zugOaJ<>pM7^+Kl_81Q3&Br)*sJxQ0 z_}(89K3w`GF;0Q$AQmEAyXA9Q^M;T!*?9!Cq#eTf`Sv_VK3^Irp$TJg>Fi3or6nZxj1JcTsGX(FtfYBDKAfa z=eC{mO_x9aF96Xr+XsD4x1CV5H3|HjRj>$p<@%1P|K<1m7caU5xY(@9-S}b(^453a zUp>FkIqAnu-yVK7e{5>kGuMae=&mW z-&Pq!-Gaa)8N3d%L`E7u4_Hja!bhcQL{LrlIIBhLqlhh{92?TeBaD<#KewLqbxX&| zBNgd)dykVU)*l;9N49CoyU+e@@erslCnxKrv;S`h;Ia!>Jj#~D!4noPt1Y->_say@ zla$sY4G-A&67IpoY#JFE=73*+eYSlD)I8I$tQ#9&!S$oR!6-=FTNx;tdDF=QvOYoo z>RJE6=T*(OxHt<{@&mu?!T@w;<8x#YmtIukg!Zx=>cimXZGBvG?h-y;w7xS0d_stK zFdw&**mwqVEdJ%!pJYL%Z5 z>5rH=T^`h8{QVLZYk@m09}lxHY=#!Q-_~k`9kQov-{f}pUUebGosGZWH?cSGSMKfO zffUHeE&f&IH?Al=Z0i(S?8~SaKeirihXxJKHKeABDVhRC_`7A?f;3$r($8OEWE{bv zv=^053mz485%$E*|1L9e`sC2z>OHGnB-|T6Lu_K5)7i2k>>`|%>wg)j(Iww6vG7Lt ztKS&9*Vu94bSj{B%yhDb2%D+dS2(SwAF7*ydk)=B?`(~TYmpvi5j2WEh z+XDYMtBp8#F1acaKW)9^<_o~q$Vj~SJ}8Rb`yBXJc2T1Joa8VQv8-_Dk*n{Mi?iNm zvt-&4)(TD#6Q`Dt$Cpm}$hmjCo1g!EK&ymSS%^cjG6pnx^fC|Oa;r&|X@e(|Z+J+H zz8B_*u@ZE?()Ogk`1K7)tHBt`#Rs^pe%$F(e~VVT+y2eYNoMit1Lu_mh5Q3g#e_zG ze;rGruDS9gRQA(SYM-jh%$4e4{?F_>OV$nE)KiZZoj5Cj+yPnbPua4^HV=2XiaM^& z`FdQ3rM0+jN!N{{s+1j*TT6w7(2(xL;Ze(=yax!Eof&TIuS_oH>`qJ4Z&?`gC@Pje z0JPHkH{&gXDON-@2AvT72s0`P^H!TAT?w7%??Khimk)8l_pza`gkcHf&N;vq!1)Q_ z1MmJoA6Q(-18NHZiYxKfsTvY`RUu`bc zVMgSam^SUfjk#`S3vxiOrj!Zo#^oVB9vR(H0O_Vw_%CP6I0kBie3I?PIN_ndv4IQu z*RON<%^UU8Y&-}6(+E@P8#QLPpD_dR7tEvQLu*5QHn$OPB380{D1z#?yXxHTxx+bt zX-?My_c5DziPpSazhLcw*8MM)@jvdOw^rP`6W(JRlB!O4gEu;=8{I<OX^7-s8Hoh_wj4A-y;{ertjV!-ZI-r&?W8&KZK+-H z)wlv)yv>Gk&#yMFV92CuQ#C3=dU2B0yK_He*H*=9)!DB>{484EyuSIi$1$lGeBD8b z3~#BnF^YTbmn;&}N@nnPJHtJ(NrnB&%;QGoDjukNTePMNb>jeP{I^}=wtY_FQtDHL zGC3B{)YR9AcG@I_>^$udN~$*0LJvN}_51S!m*Z%oLM#*b82%@Y`i@1yiSR@}hCnD}fEhS8H_NrGiH71;HISIIOMF z@V9b<$vn?cV`|jnk1vaDwlxO%p*O4^2h1M(k4h2bJ#OM}yK!z<|Iu&Ul{-!-G+*Db z>pyY)AAG%Iccx9yHX2Qw$;7tpWMWNh+qP}nwr$(C?Od^Kyt$ueziaIed+k4P_EA-R zR(BVUqLCvu@A5pK#7_UiPu8!92A?KaOLgJwrTKK*iG9Duyq}i)3e6Q2&1h)K_i7V=&u6XZ1A3KOBR&-_mjM?d z>CR0HDHQiLP5e`J*yS0&f`$?09P!s=+Of@iT)#1KKpd$7vTm_hvfK91Kt?GM);BqJV6M$I|G8nV@bu`P&1602J zndh1e1eKS>aZY^K%Bq@JKuT#aw^isJ1w$9_y(Pxj98% zo|RJ||BH#L0v~o*p!>5qeNw6^fNX{Qksd42T&3=PUYLu4pyC?QHr!Ocl)X}O)Yvd3 zA}it(u=4iHj0|F?9-C+P zfBLBw_SUeyVA&8>C^$g8+Da9h`lvkk)5&LA?%y7{LXK^D@g@}jku=9@cU(YLM;2&8 z8x{uxS8r1i$oVvAS!ti-&7ca&&;r&YHYSl*Muu!H2w{M6q|uPqWsM7oG75H#HouTR zW9eDHp`_vXO>9)?&-(>!JKW_=me~LpdC`Cf=9*DSBGfj@+G!=Ft=Hc53@CYmkpY$? zVb1Dc=(f1HKPF;VMn_G3-HGY(32_#j;Jp9pjY|am6$EG3Hc%$pZ~kmnYcHlfPPqz~ z0$vEGs2CSQg{q+|9h8(@cxs=F{S`-VZKK-eqAPwnmJEpp)#5$U+^lNpii!^m57wjm z;NvRSPGff$|2RH5|7xJ+0&A&2b29c~%0@W*+bwz@KFO>fR{?UaskD>@U)hz0k7CcI zX<*$5FfWb8!ho_%KF~JS#`=FmZ8o?OB(`woApXk&mer-zR|x~(cX=k1rM44k637uv zH{VIf6s3Al-piq)FV+N ze8h>}2Fz}UC)AS|XC{2Ng;rEJP^pi`f#1+EqWQNi6156ZMC&==(Y>W>GB;<)H7CRkdDLxT{>POH{~WGxHaykSZ%qxtT{$7dWLnS=Eb`*i{jHGe_?DapE-Tb$a?N z(xGx9u1uzv2>ZO{zU_MTrZgUQFd3Jt9%fP0qUO>GtuYEI=@5m+-gW(~@@>i_&l+(5 zHuS=r?M;0A*r923c|mC)pDAluNl@GS^HKKujXLs$G0-VM zg#vRYuF)BF!tlaX+^-t1JxeY3&)|`U$@zLsC^^d*O_Gn#{R9)l8y=n#M%eOc3_IhJ?G{dPW2Y;M0cb^6|_sb`WioK8v~(D z!~94FH}0{ll7oM;Wp(UPN}L3{D@8TxE8hNyvormjMGrOGFr|> zsM*G8pe4nOIIqod>nwy4!dmZQ3_Cs^A!q$_7i<8+jh-gHM#51m0LuqDz*@+eanQgI zZN)Tb19x*`#;&OS%I)TN_;o1Ptt<7xPh#TrjtSwjXbb|+=clwV-*Xh6s|T;_4Y9@h zW{JBQTD)fjPshm>zHH0t&Q=NWz*SViZY}D&d^-R9-#NK-Vv?ki_S>@Rc%9Z~DW@ay ze89^DH$Leu*->vo?uiP#(Wp`#dU-^gFEd(B@79*5ly5nX<^L*TsGbDM8; zJh6>u&jWhnnK9@BMiWfMI%gKlBAa+eA)*nRX5ZiMnW~*+dR{8uSf6O8k%6 zyiIs?(;lO7XW=XHT+`U3MCiEP`7bhZaBSascW%fDon}K!E{6qR-FGk$I_rx~jgHX& zIbdzl@Q!@6dmaP(F`Tq}I=h$8&7)kukP~Z`s4V{c6tTQMK7YKO!xU(WJg(n9n;jZK z>h!R#z}Qm!#ZsD3Y5(ww$RPN+n>wLizPI}i@>%3hD(v($`K*NYOjjWa*$4&Y7wFh`M`OgK9u@y?Q$Z zrLZ5rgUZh;j|gZGuz6lb9_Eagek%rbOGl$a+Ozin-Z*+$*<0mUsMGe|@9t`6PV1Qq z6ovn583mZYW1J$>B97ukM+Aln))pLYZ8mC%mbiJgoNu$lC;aXtr=9XTj35*Q$)d-N z=cK>yV9%gC*7l%9zIVp)mvx}?ScTeDQR0-O*yveKdS&8b3R6wq^|m}N{&|k-&E7VI zL^^VyK^@?w*+M!uc4lNa!{{wk>{olNh)Q$ZL^{(7g>I*s~pYTOcLv>`M=j?Wh5h%1oMx~;4;piY7)o#81@sa z&fQsCuhRrd)r5CkX%VrX_C~zTZ>n^YGZN220SMYanR+rBUhS`%j{AdN2a$Fjvv!{7 zzQ#26H$GFk_X~L6@4LyrIs4Mz{X8xg65;Bpr#=)04}h9zjZywvmk|d>Y$Me;acs0X zf9L#VO;DI4ECTGVU~5R^|H{?2Y9xHp?hfb%?X^$%rem!?1bscnKgr1U$uy5a;tAvbkRvdj zwH(ilmiD?B;?9zld^JKFy`#IZRmxbDxjM3H6(&CA|1TFn?ynwfCh|}sT|c_hz;#9l zhI`8L(#1-=EkMr%yeraL1v&I%cfF$|b;;>+KwQ*}G_5mDp5jKk0Vpk3NF+zHl~<{a z_VtlNJQlmDVE3(&Q_{6|RXcY{+VN5wtEaG}7pIPG7TJ7u6ex~RVRsy&VUTlYB8|pt znT!8G>XBI|SW^9d9oBc#J^x=7xtFGKtUHxj=Lb8eAalbt=arO4w&Nt|%2i|_OTin? zPnsG$jTiWCY}}wjfsscBRog4v3GQ{+?cZhuG)Z+aOTthO#A0bRTn>W5 z&NSLbvv$w5&y2^~g_*TVjVf1qgh_PNXX zc;Vc6TK35KicInP;RJN~zCU1Z{^}-=F5t4*(9-!1*6QJ)UZ)i)&))az1W4H+BHDo+ zwz%$>dAmG(taMe^4`bOScFg_Uc6r7Rq`$j`I~j)uL{zTet(B zyS*Xv@x{jUqjiS^EtKQK{qrFykFb?{mFO!mCZUfqorGlh{Y7J8^GkE_K?rjXGMQew z7rfdn%-fH*pJ%r|-@m(Uyy8>Si$jQ`>q*aP z?bI&`jMSY`^dP^4&+Cu8B%6A`h{ev>dd_2US>MFM})XizLZ?5Nw}Cwz7~5n z-c@C-6wEWYpSk%^6%SS>_2mn54UjMddXp>2No|SIDcjJBz%7*&!7OkunG=+If#u(a z&K&uk;Zr5MID(+2hFE7I+l&LKU8`n(Eu+T)PZN(R&KvZ~1arBn{vri}zzi!9<+mKN z;UgK+Bsg>-_y{~C)1%>*kG}N-7Yh4B>#rUrb^h!7Gd1o~ot2>2{t;74M2cFey)jxL zmO-X?2c0h14Gbq^6qt7s)SeqDtc3)T5!oDOgnea&>>ItQVJjJEt z_>l{>QN&{ik|DIsRnIctMIdkmXq8ksTtSC^W!mhVd4IJe5V~l&THZc#4wGhpI&9W{ z@D^WPp#ab_8kxmPE4P0SzjOVy?f_!3xbI(YG*MUT2SV>(HH%KF= zChKFZo5u}Wgj-E4f+6dd&0D0P(h6lzFECOiK~RUzf*rv){P#Y8x4uM`({b$V$dR;f zYI+?LAU$&_6(>O)tq`b7)+)*l}qV^EjBgF_`k#$ zq^uswmH#qrWS<=d&WY7ZRU(sB`gboE(`G^n2FY6#Bw`yh3Qy2W5H@0SU_>j%tS07^ z=0<_CnN@-pn_C94E{UEw(OBg{spqj5E$c3lS~kL&D{xmZ#cKS1V~%FpMec;ABspt7 zhdxvID37g|cc(Wr76YWvs^4sl66~MHx|1oEX%C>U&%LifTuqwDA~R>J*LNQUumSOh|*i(mi!*Ud|ll z2h75&{%bu|A|)vl0s3(?;|C>qg+Yz=GdU)HW&|l{`i>P2W9c->Oc}3x=X8I+5J}3H zH`}%;qMGtjLvt+%EV`0^C6^_x<4Z80V{WbX61m-}{`%Nw>%kAKd;SQvtNN|I3ONxe z1u$SWWvl|Dh*KztFbvkF8oKGB82n%L4dSh!uH|cfv_06y@y^)6&K~Fy5iqzJDju%g zu`&I0!V(Jf);y}Csus(e2T%{~`f+#MzPfQ;W_xQ>MddNF6TK@i(OXLQ(s|#*>_!HiaQMqwE{&GheVU}4q(kzh! zerd$ExRNfvmn|o^Z$m({;)0BC-5lJDL>NLUvU*ARx0%wQXk&50A~KP?lr_P8#w~Kv zuzPsmn#KHI9eKYa2oBH;jKR&leTR|Xcn4~eIztDGNzUcdRcvS3*0eI?~@w|0pGOu+-*=DhST=<%C^D4DBeqV-lyAt$NPG zlaVeichYqxo91s#K7qfo^Z+hv4mFwu(vViWbB`B)jal4_w1j2MQDCfbZtG6K%o}h?FTfOFb>hVGvvoYb16akifzZgIK;W1 zB~hj5c<$%vlH3$Fr0=|MW+2q4yuiNHzr$!;&Y3z4jXjl@+JnN8_zjX19jEv>RwX-G zjr#^+TiJ<9AiHAES9olmK0s1Aa(KT|2#q96NK9VG6|Xr*Z2XRzW}2z&p<(%)}F$l1A`ZgHHoKC9r$udk|3oQ`_&bU`=I_ZjtmQy*w&=s}g>a zep%C{jo82ue8%v&wJ(1S1Pa20g93z<*tBCe!lA`uUY{$Z8&cMGKboRk*H`_H03 zuYOeigh*SX3;i3|;h>bD%kL?`^W<{M_+g|OuM{?$yX%&LjHG@4^o>I_h1{}8*GO+T zMam$VmmWiYNzjD?5wX&rP&6)~{tFh|xUY)UJ5V#>NJe|hSp(hMet#Ujt8)J`X=6u) z{Hmv-wXfMNj9}LF<|9IJLru+~Q@NlZ%e!nBDpFs1f|W_Xh0bkBBi|L^Sf91}Asu+E z(hg;pm>W8s_Lq!3o!uR82po5z_m1H_)9tw?i)1L0oyHQ9ehFv^^iGV~(|6ehL zY|#17<|`I;+CW3dAl`rE3mzXp;JR_#pVP?bV@6xK4@wQvb-WCg%vnq%zwAVFaL<>k zH=H{z^;c?_^1@TlDKNqDhg9)>J^n`sU%mU`D0hL)R%b6?GN!808kWSLi(GaExvNwX zOlDckr#a;)e`!7fE8J`|2qc)}o5u`KSqzNh8@tMuY|+^?Q;Fo{!YObJt$7azuD)Yc z^SnK+_HI3p}^mxq;Fgxef}w=vVXeh z59VQVQw4elM)fKlWxp6-MLfwF-8*bKn&lF9wfjG=dW(Mm+^I775(0Bb`?5IfGQ*=( zfOP)$?X!S0+A1Y{#mIi4{LM2w=p9h%FDr)yRtCJz`c%Z?Gbzqws`kD#Ie4OORdlGC znp*Xe*$Q{)td{_+DIH6pECySLz zeo+c$s|)RWP=o!IE;`3$p+5wWxpQvLmOS6XpctMTaN1FBSpGY1I$>zbt{Mt=jGqE~ z*%g+Lk$AU%%ImTh{ZGxVV%-G8<%?`_R?r?*`%!sB!&x!Z~RzSb*aAP=ZD_PVK) z270$4pNGs}F+=`38bAsF26iwut!eKTCU7er0SK)f;&kr%CzWJ!?#XpTx8-Vh<8r`< zZKUOzxK!q0*b&kW=IhIsK)|RFTryil#^OLJ=GNR_FdsNZT)o3rEUU|bXQaChR`x)MVJf#$au6!V>7Hta-jll zqRC$eHtS&{ceQNFMOCpPqogueAKkxcV4)nk$>sJ_6=Z7UyErt`Iifu``f9jI?fCAx z(cR<^t)hk}f!>CPjr6wi9#Y#Sc5WcAf4V-=w#U@~O&vvwIJ}m=?;=rsI zOu&A-{#dMr&BsfEVu9{}aHa>8_pMibpo+lSQ(J<0s*Zu_v@bZ`&QEIS0!7TbwR*uy z??oE0Auv!Y^s{68AxEn?zu&%`J`X=ILMY%Kot@ueaC_#h5q8sPobIzfHFx{N-WE?k zU-+3)P4K#3C#c6g8L zieakkpl$S}tkUtCNxi2i9NF){KXfi%XqqmYOLBblR?s@=pXh$r-gU1s%{f=}CBAF8 zzst3qXrjRJ3Antabl&+BHVgK0vhv!sZr_OHR=(mon0{l>$Ks9f2Tp+n#)r?pig+Rgc7A2I3vpwUN$x@!EjM=|`@-Ww5im!$+4z!La>tmcjeGFtf#*?XfHoBWY5h=jc=7oe)siq~nQVWy~-w1KXTgCk; zqVc&vNK24~AY&ztrjjWkUAI+_y3jF^Cin|L_`cSGI+cXX06CM{&Xk$?hQo#EkOtt7 z-+CA5ly-IXRwdK!VBov0r3nQ__yO7Zq`hE)Lt~)y5mqGtKHbtNb`k;)S6dH*V&y8zO8b6rg*OjvF4T~c znONcy8B8vlH}0&m`dCHMbqlY*kTsNWLzj@JF;EQqGuMh5;%X>D!Ry({psY<20_&nm zZpuSlSNMItLTX^|{x~x3?a_c4A^8jua#V723do|%bp8oHg+2&()BtrYViVe<|5XRa zf9jxQs?`+VLjYngA_s}fXtBgty%&^Iu2%mXMZyxdvk1zS^K<+EPzlU@FJYXUr5p)Y zV4M_Hgr#vD5Iqd3wHY?9LFi)}^U&bAaGW^vf9s0$LA?3W(8e>NITUMq<|!L8BEcJJ zCSj=37TrMlX37`Nwhjs5bw4m0dWWKK*NnKSY$NM_@iCYx_Qv+G0@*0Z{U^Ulpn?xg z0iRH&F>!rYs3!fw{)g?*XA!Qh(u8In(-#>Ns+D4to ztU^;9vP?7Zwq&!#uqZ6gbi8^3B}-`Qf2kc$^PF0Dw&V2z+&7OiJl{Dp-O?)uIt}iJ zh1cEQpK}nl{C%9%s!f)s_s=ao*F!G<3^d=O+8&E8dSFK&j2Jj1)`W#E){Pvd@fBd# zDWL_32qc~|5fq&joQtUwA7H3T7U#^@Fv$@KL@BMW?-0LE;{bfFNPRCh)i&ldjv2Db_`ah(AmILHCVIUl2IfQB_S=;1*8Xnt{B+MQQZ1Ixj*YG{=Xc|gciRbmXe4`tCqo+vkpd9i{F_;3GA=Vor11h zCT6bLR%vZ?zve^tApQK*z@t*s4UPaQsA+w=IBDul>{Kx#aLfa5PpH4TliBxQ6jTE_ z#{yc7m%c}DsTc~~2ai#$Of1VLPrC0#8!<&|Ndn^uqgwt^NDtw&vsvm_yUV`x^?Jua zqHKop)%R$2oFFQeDlmj5^=U{0ou-Qd%j)8C4gM!UuBxC$7g@lO*rAT#0wW+Wn!gyG znel%tgqs;xQ>@)xfGD1YS0MRZ)=U0x+N-O2_R~7!K83>@p-uuyiaZwZ_pQxJ`~Kyso$$}A-~@rhzBR)a|yoZMUJNE9|}kub}|tT%B}0%p`OlTkvA0M1&kz&YuAD5Gih8B6_MEV*AGJb#+QQI5RUrcv=YlUuhEtnPL7wIrrN4B8i*YA~$dv@K?kjLl9}e**Ce zjjNt|tQS5@IPrZ1XCaE2>@Cw31VO%XOXCum+y=!*d}(jH2p_FAuVkzH<;Yk(%dT$# znAxWc#?=`wQLn;mra?!MfH_R7R1C7-BRTIk(b*>-b?J4pGXMNr+=`33bgFe;oH&Bs zG2Y#{e?x6caf403@DT(ZXZ&S|G3mUeN%>FtN$ae=rm5BOkS*iRW1VfrAa`z5DrNG$ zX}nD9Dqp?KR=4a;O1*fh z0prPAeu0r9z3UOm0?R^vhw~7=RrOOd|EIPHcpeu^65d&2L!t zOWSbKeu4;+kA9o{y(ZXb7>Y@knL6Vn#=OXh7&2V;3{*3dei5D8mzQF2GT!em7Hlvs zV0}{b?nWZXb@7A03b_fbYOxtuc)$_G2-8@f`8YQwUV~{?0dCp?MM2I0&2GNjz@z}J2SsAdN&u+ zs4|aHrE$nkotV_Q+<}(S&5?L$QVG|)43X}Cv~C@zY`y_hpw}+WMu{XiT_9tIul(|- z#r5k$vm0_E2g54kRMFEc?FpFx#EfdK9V z0?~$}J0cz5TCF#Z*Z$TJRWfaLt$`7mfbvJNc#FD$yjc@=`!eOG0(WJmMTr@DgJ3cA z+A_nxhPhzOF=1AOCWb6)xpr1;8e9x8-SBumXk0T-_vt{IDh7A!ZSi5g-BG|U4FlrW z#X7PgK1E+hf_#3g1m6DqVWQwpzEi~(Ot6?ncYn05Ogy-iazuQj8d@mXY$2D2PMEyI z@TY+0=BR1{eR`&?0tb=6*lKM^B}*miyCW>aJiYC`l^ApP@aHcS2Hdno3#oD@>VGTq zq&Q0b1;kPV%WdO|{1~csm+4I}RDTvuofj1~2v_%?m<)pU<@GnOz-8kR3`dq&1NyX8k7$GKX$~laXRjXokZU zbW8cZa*kx={R7sX{$N~ke;|sDQvOs7CD9MjKomQ1dC*npD4{}$1Jn#)+vr+on9aQI zK|g}&dn~y+pFXKgL_AcfFMZ0Lu~=fR@XN0-Ftm5+Xuc$}!2hpii}lpa3~!Ps()k}hu2yOt9;z!Yl1?FkN}^8t))aAvy_rKv7F;}% zAU%rDJT&q)oSHoyu_&6$3OgD#CQutx;dZbgErit4Oe%gSly@leUuHVILY}JRIHi&- zviQRuc@m1lsi0=NWQa(XV$TX^jl@)`cMfaE1*XEokA@)U2dE*y&t}E3RgD$qx#}*M zGFRPnUYW(R0&5hNiQzV+*_Th$M_qG?ZD#2`whQ`1B$I8mLszny$sT~IGC2^_LHb~)q8de|xp$DDVQPrw2k{`l6VYs_#6#ZEB4hc`9RH@N68=>^u zDJ+et`)5%|lfgQb^?07|=HOhaYM1i~Pu+IeeCn$+Y<4O2jWxfjw*cakJN9d`F5eHZHwd7Hs~(|IqqljT zFWN|^)he@Bf`#YG=t`ZM+^5n#ij&NE<>0)~W&AdQJQaFFQ=C7w1Il2t>X+3CoK&f7 zb-4^^p294uTj~)Ckfhaa)g6i-deH)9R4WpWnk?BoS2Rsy2n9FGi~{cTbUOly20hX$ z*2e`coh$IlK-|zZJq;5@)F6(DP?n-Lr6Wbf)W?IDh7v0$HgLZ%Q>~`AKOI8tADwwW zA+)X$tos|RPg3k-c+&qcMS7jKKvuI67F{+qtRxTX6e+yPtyWg_i{f} z%EZsF2Q510lNrUu0NJBjm*ytf+)SAeD^kI&qP%~R6{0-y)S)#+tHJEjrZttGFq-|I zNU6q1pW^)MEtfn&xGfyTrChrq-jfr3_YSpemXCt$P-!9su;8cWzrVnrIMYi>Zhyf5bm7DRA5RZd=<^HPq7YgpO0C_NfM;4a zZN&GHQb4yZmWsD;Zsf4Nk??3w3^hT+K7K3D6Sjc#i2Y+VK|GlQPM%OZ``7<)b<&X%!TwyjzLLCEIkc9nMg)?Yzp{Kg%5` zgQ-M1Gl?-30iN-#E;A7b&Vb<~-OT9oG#$3(G3m zZgYok5Yj){fgarNkw>o8qnQi?LAwqMX-?@DmyGz0&F#Y_?xzYOT&HXksqdlA<#k3%KkK5uhB;rCTAYKG@( zFTTrx=kTW3 zmi9cU2r8TPdOLuI7Vg(UmCACJwvhUy=CPI=pvi@Y54VAiTC+lrD&$>*KJ^GY_>Vnc_k%;9);xs$@G zq&YsFM(3YCA2$8bO&yavf&o}qyqFmFqFeS9ovv?!kPM!BJR-c#y?vUDsmRC}nLs-q zi3z6KQ6}>Ho?7qwdkyUcQef4Tie$Ra-Juc@xRZzop33;r-#qo&1rBt*Khwsc)}qL_ zVw`*7Pv%h4Oz!Dg>#7XD0BkzR%|!R25ffZnrQSs%w-y0HO=0Pb&$Fhm`=>UXTOHOb z`{P)CnJixFv2G7U!e&|RkAKC*C_OrLuc5Gw@JAqJ@QhZn?YOG>nLK)J6v_{9{*FIY z(O?StPX(ixAhJZVj}-=WWZC3WprYC%TdDSJ|JE(M6|S752mh&rrecBD+)2;sHk=cX z*qvXx`dC;yjTCp`=zl2i+J)E2n49)wTpFFneZTTq7BVKBf15PjgH2UqD)xD?N0X+0 z`M8Mv^(#|#y6X=7zSMS39=|DxiAgxoaPm`lWoo8uZ3*A$w^`Yf?XPRzsPA+RW`_~V zrKf5y{EY~3^e&T^mngJ%7a!hCUKC+_`R4xUXRhxzX{Cya;?!tx3}(~C9_npGK@a4*7+Ms!GJ z2E&5gfwo**&xRL8l!y4Pv6idOU$3Yjzea4j$1yx7BBDHmOY2SKZ2)KAV-Phwo^(fn zKZOR`PVZh(89KhyG^tD-5O2=__}Y)AWl{%pb&;&a+@U2l9J_vs#wIXhHs`_JX_rQp zk~V+;pM!F1?zIW@H}AZnX^a^L&hsz;XQfI_tA3ER%q z(Ynj8<}y|+Y}7)UcUn?>nUneNqsz$$G%3lL?+sAG`JVs+?c>PP&j;lcOm+E)BAhW( z53Uwab}Xce^BK6axXg8Vq)JcdCW3g?lfE{~Ie)es_mhq|!K2ToiYuec^=O_p1Zk@# z__`R+I|fmmfn=_~69$=)xHoECqb(cW?*TD9)%*?jH<`Q(n7^S>pzeb)CtGT`J?YS3 z|IXIZ_j^zuvCza(qZ8u(^LPED;qxwyv|m@V!~9VcCYVpVN{oC$^WEN5 zk<)hGd@T&G*5XVfEZ3yogzMD;G0$jh+f#DY`Rm$xAhfDc;gFNFNAsPY6VppESal*O zD97jJ>c*O5>GE=Q$J!Bv6&a9AP!}r?*A4bLEZnFIb9>K)g4eiC-!M zYe1WR*UYk4Esjy^k9W28Zk))A`+Og@4@GxPR)~-3S8E{GBg+Jy7N%$#MC$h(F9D4j zOORRL-t647S@7O|cu&4;p>tSwL(}$p_hJ^I87W!YuzJXy$!;t8TwkiF&D>~Pu9oqf zt5%(;tK4bLb1bu(F?9-?jb8xUEZA5jutw24-HHZg>w2M7W0LqW zOZTjEm$7q%aL4+deb$&=#Bc{LSU__SnY2X#gg5EdKO}7Ylj7hh4ZWrDqyU%+Je>Z4 zkKK6dOTn1Thba3w}4@liZ&_$n6L4_{r934-0RizPc~Pm z9MThy8-oDfYw}{L^x4;HXRmFa0;v*O2y8oK`o>S^K{N%@$QA8>M-tO=U}1eJ&`cpkoZ z>av62!BJSO9}k~@Ki=(cj^Vpr_q5;fMuBQg5d8@^NW>BJy>zo7#>#U>Vl)hMu>JN6 zZAPli+U~L~)QQO6=r`)!?ni;U4h31HpKN%x>sl@`_f`bTEjXNcZbl27Xbglid3|_s zdsqcfv??8gT{M{qoWBenc#4v(9pOE*p zt)cfsN`JaBSQ_WnXoNr!KR^jW`uf=$5-53jB?^RppSb)0rib+{aar#N!^D^mMNNRh zFeO1hP{}d%zZi2h^>TQhSl{BOngVwlPQ-J%G1NUMP1+5461t(=tyk2I zCIn=%?m@6&)oA(|vYLDIUqA17p7wDb?&HMf{^IS1`7|H(ts4N}dOK-Qtxt%(OlSH` zaqF34&TJ*DwLI;~_9+LPe-q-bO5kpLpXCKP7}q(!u9Cc1W@=8$vihrVkTH6rx}KS2 z&WhsVJIK$!#)*v~tnun5h(SR|nZ$*x-pst~%9pn)pjcS|HB%SFIdGD$#_V|dAyp0= z_$L|hm9Vv6l|?hc1w=j$Ysfh6m(i5=ms$0WTm$f=txDdn{zMW(VZ4m0K*+Mox2-xg1-WCMyu4iz*CrxK7%>UkLw{A zFnL{H)7~!wj!y|q(WZ7hzQTeWrl`TFu+`$rSvb-(QN8b$C5}w3 zJUW=_d}i(H(~EdkGe{O>j)?t{q&7^)z-ps0^iISE7T4k2M;(l(drjxa;)si$63L?c z@hcS8>)DSwzkmF}fE%x>o!IImZW`H3LhE8UkUBfb)*Qx2#J5%({soW(;e-3swUaTq ze9e7X2aqUh;i#wdmmc;om*q;e=It#Oo#tloz?0E9x`W+OQ1`dHN#DiJiP3m{Q31rG zg=4kb02&((87=CA+qIOo`Cz*9a-fmvJ*FpHHSA`-_vk?SoxYz=TS@X<`99 zfBx$gbI1gSa{}sEBPj+bI#$)J)Q-<5iE@>Tvs+QWwDnTCmISe^z;?wnFt0afu;Tej zWi~e;MeBK8gns@;6C~~+Br?h)YNcst4v9!Pt#39$m)!J@FS-C4Xo%*H6;6ALygJHl zw|Z?=rZHc#>nKl~C~VIC=QTei$(};RFd2jw=2F;j3d}VXWuN$L(f& zqbqT%>Mcn7k)7a;)a7rgLr8=OQ-{jhtcad#H2D=O(Tj`LXf*fZ%E)9uc+c&`>ep#I zjD~%#S*mzFAJx2Q1{%@C1P%5UZDLElUIgDIG62(Jdw#p;Q#yDNuM%B@Q2dFN0(OV> zB;T#MYYxv&y^fS$a(Z5Zey*3V!ulWg~3g0v76!s7X+anwLKkSbI^%kw0~Q^skM zzyEu67g&X$rWgINd4UX4{0{v(1432fj)4M%gNfe6OZa=TvQ$o23-{{X0bB?WDXL}$ zQm<@WF|(Wd@s#_h(o%I=E@#s`%=kLFdWcnHsLkzu2<*diH9I%_r>;)~%Y= zM!Dk6cPNtz-fMJR?jBw5Z;vL~*g6w`==9$oYyDJMJ~m4Ey^5f@s_}%3eifKtl|HxS z>nIpJQYBNHuiQ~bLbRDm>uitW^t{-b4i8w&q!(7e4QXF@76+8b%p-|XUi4sc+>C>d zL}IUwm~ue{PQI|&tUZ`+oM)T*AQda<27yiXCgAlOv}~Be05 zc?>_?&Ul510pE>@7~}N}**MAFQ+}Uq-$8;l4KABv? zi`_j}Mv7mj$7G;rvRDfYN%b$`68t_y%h8#8SseomQ(9ofI9dK?>lrJPB30EfKpgsO z2RBL!+JCK!&i9S0rPKQTJ9|A8LNK$aCK!&8sZtV%bz@jaZ?D8SHm4B6ZPpeFDqu4* z0k|uHl$_O%ABgEUb7pfl2G6SNL8dMOIq7{>6{ERn7gwo?3UeUGg$!p)Q5eKGwXidJ zzK5fP9HVkF>}7wCbc<$1jZyB?q;ksgx)(eoc9j0t<1X;|O3Ah!Jnm6K7mLyUUriYL zk7k{l+sQ2*GXucI=j2w)ohA@^9CB+2-6ryf^l=D-M}v^%gx8Ap=I7&aTY8dfBY~0c z?G-O|nli11c@ zn|_%aWTGVyj_nJBEkD=QDc18L{h~n4Cm}Jj?Pe-Mn4aF=NgrsuJ|?g63mSdOI=2NL zJ_C_%Wf}v}D178}%;+px1!M*ieJgp;3N(G+**`i=HF$2Y2Gc~vRTyl3HejMUF zmER*$Uso1oOI)(^Fi#)9RhbpMv2EOCCo6R=i>#FC(KAdILoPv9_rnlG%s>?4%WHc) zgXWkJ&2{rX(lO6yhOyFm9)zOoh>vk146C*^nqDEpqLjmm4_IBzgS(bOSjMtluZ*~e zmz49f+14SHd`|Tle>yQId5xP`aOGDG!6=%@t$HNH2oghmZ?B7_zX!#%7E7|-tkWJA z?&OAwrLf&vW!ny~z2n%gI&Y-J0SB%xO{8?)8jAE?hq%xlZ4II} zzT-NUN|Oc~l(4Que%q_^`F1e;%p1lT58|cA;x#9ZYyZU0L)6hS&#pfl4-jF71uaQ< zZs+=4^jnbrn+2Ew{=1Uw{{YEAHovAbdgNg#-q+w>J737`8r*(-!s*TSKlsqn+a(Qz z#hMk%*)AWP3yueIx07hQf) zoAymExNP484i=M58#XbL-S(?f(*C+qI!kN?OkqF1x78N9zxwPuPd|dWMliS+{h%Z3 zHNE7@lg?_6Uq6W+0@lPGZ@v;sWUZteWMf$09+LS0jLZj`KTwpu)3$7{&3sqg_RP#< zG0jmb48c(*X%s<_AjeTAMG7CimSmD9I5co0chifD&ZIPlf_NNZ25|qt5sRzsjnMd6 z89P13=nR0+ytmbEKiao12_7>>ElxzIH>+(YBB_Fv9qpY&qXpFFnOMr&C!}cnxRGvR z5hXs*24m0MB7W2{%`U$39uHk#?u@T<5j<+KiR;YZ z;cnT|#SC7MDZ!t!fb9A9tDkj$SJ#%SP!a#Mh8|z)c=w0_y(JsF?XzE(5kE%AKga`oT*PS$ z7a6r)#$j-v35M+&_;GTs$AaPbCd~lI)45_E2+!TqK3HPaKl}W)k33BnK5^}Hz(Hby z50fJYe#T;)CFqCt>LFkV^_#2v$n5MrZuC$MfQ-os6K|2TcYG3Yt695b+P;jrgeS~6 ziM**+v(z(h7}z+8WQMWpUi&f(J@{pHD&!)-xodJ7mjOin$%(dNVHJq2^fZ%tB5!!cD0jSTM=In zf<fW9Kyx!_?e#FINqjxKVJ7BtBSLa+x4HbDh~613WMip z;NMnmsW+Q&05RR%>mj9O?M3UgoQo%B)@-`g$oQ8g)nFs@)UGgO2w^BIj1y^VryG3l zfPvyhn9{?FrorITUW6DJ9C!xLDMo>a5hTM)lnTizd};aQo$ZGAb&DI+W8ypi`Zna3 zRvWO57F4LhzSbUlo^|n+4bN`L8)exj++g|i-M0?y<4!+-^?CozOWWLDrG~j>@$lnL z>M`z@jT<&We(9lq4(as~1B25KzMO;y_@T%Ew=c6D9$f*Ca!`Ht!CUw?*o5H|yx&2G zw0Wq@z6TzPCln$Nzy0!aw^yF+*X>7dRg^5WhM68Q%u5cO)BRGm#KVI$MNO(KCEHGr`!67HK=JbB!iG zRbdNQ>RsHfL$&HVh78>Iki(#cOZtTjr}&j-(@U;m*GLNS^onQhxlOtTo!IwZ zZ(Z8<_R6?_6@jHaX@xKxe*DS!%^`Ay@5hm=hud8zxq|iRLHKq3H1o}gCj;l0L7(x< zD-&di5@%m>)%Ev0f_r};)8j8cO?da2$A|UpS#{@~O?pd|sC{tVkH<~huyzAXy1nwu zCvUzep*`KbUyWV&lp>z)Ng228y!Y%AkUeX`iD#VM;i(t&b5aNP>fWV88w5iPzU}$^ z!*}qre_;~=feDWYsAc9WSF1UB{C8h7^L}`mwy@u|fn}8JiD0qjAi@SiGs1nH|C}|%-_E0qB-ryPgTOFLZ?`S( z%BE9j5Db25`~I^}3Ld(Xnw-<&$rpIJkVwfSiU1^;vWo2+tA`kSW$ZJ08INSlSlPRd`*(luOf~iZL!)FQFc^ZR z;ig!kg#1{K`N6|Mc+}7lO)Rj)-|v6yY&_K*_I{K~?XSN~x$heL$OfyzgkSe>CyG-D zxC8wC=bnTMQQ~U#?Y2>5gMN@bpk#f9VbC83)J0b?5^0M#4Forj)u@G|Y0TgOW*+m- zRxVxguV>8REE_%#wCVW7ktf#2&Gz8mxPJZ56MyLX^3(W4Dj4+6tB>Ay&vS1G52_rc z&w%9FrAz4nCB9$Rx4#+wg^$JL590wR8jJ@%9umhI#{+Hy1dQXs4hR#AdDO@=Pl`D- z4E~4{5WM+03@&p_oA|?ruRNoJ2i0qj-H+hG16vcP9skC?VNEvb0l|lpOK-Ug2Rk5B ze;hx!$4B@*l=j=>jaN@-+!U5z!@twczj)I4Z-uD;CtYztnRw@fe(fK-482L^7%%{iW+`&^ncwT<^?xDTS2R87a$VrL(&}_H8d2%yq#UoPs zo|=?Ub<&y5+duI<5(`IW^P>^dJLAEvCH=d-iyxfZN}Er6d@*ukVmB%B)K=%vBk+l6 z%7k&4U}ye3*`o@u3@{Vy-uoI2JETMjoZ9e_k__0NbJ;aF-1A6j`v6fY@uaZ#o_P!> zT%6imTWn6qfNy%bKKGQkfJrjQtD5r~2RX*c@P3~P%#D-h-!vLV;^U!HsI>r@2O8#E z9B7zt_c`zoD9vU`&FJzxKTrB$`egT;7X*A< zu-^x7$?Q!4u4VIRK}=?Y#RQTr%M|Yt&$mlL%L{5>c=B z-oP(b%t=Hxn>|b#2m?Ci1o%3K9p0`}=Y0-1#Dsw#IQZ3RfxYI-Fl^oo5t0cLZbA&< z$P-Vwy8XTTO&2()L=t;&Q_hKP8YcV2?ZVYcfBySQRXX;0ULr&h2kb9WUMA z!-PQZCO_%pm*d2w9mY0+6t$kV+y|cyUVlP%)b$RBzU?CyfX>d%Wrrs zT(VHJ#zw6;U}|;%@(8*7A<5GWFrtDdOg@CirIl*|=gJpMY}A002M$NklHN3~ z&ZrEk#DvkHC;%;Jn8xTfpp&NkIN{}o?h)m}d2?s~G6nNv@FiIL&M)Eda^g|&6mXS& z`lKK51$)EuF4AA9G9JtY_rCD<+?l^jo%n-eC@lQ-m#LNQn2Y-5kf~vK4tA?u{(e`$i$JVS|K{Ybd>bkb_JQ2Y*Y}#Vaed}C% z`tj1Dx1W9#Keo*Ea)coH_zv)&&%AE3T3Cn+I=C()%+9alRQV6I}O|TBo!zz7-SO93$&GHCz;d{N+#uA7?(+Ve^ZiyX!L^6 zr19z3_*$0z4z6nku+K;(d-)O=yjj5b_8U9h1mI78tU0w=3rs|KKv%A`MGY=e0WR&;cNiv0&Qx5i2$X7(-=`efE#@hF*KJ!_7%ASGrhO`P_yYIR*3T!oh< z9d22|KUCq0h9L1>F{TJS$PgOvoN`w4XYXq-zZsnH)#!zD=T@&(OYjun7r*_+a|&dH z9C@bXmhNJxRe*2#f}T0CqU^DXleq=L4CKK@XXG(NATE*4Y%yzARSFc)OZpHVTM2PR<@ylPR3C8t1ycLKjXk_+g zdyhEXczf1B+aplAq?> zoAZ9VJ9yT^u&f=O_d%{#T(m;)?>ucz;vl5=c=iSo$IBl3)W&iMEu!nUyF7yD^62A5 z2P3j1M>G^9pICx6YkftoPY3p-d?J2nlFDaw^JOPCBp;3goUt*3`yY7ZF~&@Rr~dHWa`ijYV;i2KYdHY$ai&4mDxy;ERRr71V=z}v zaP-~v^lN;R3l@#CTqj=Me}B5 zp>pbSwI!!Pj*6IvxdJ?{5@k>&CUk-Ckj|9ucyL zwRqQ;y2!u5pLWBv5?sr8?7bfK&pcO~8(FpJ)<>Utq4!WcKRh^jn{}3Tg#877zg06V zz(dHQw&S~3;(;^*{;Ay39C~iwjv6{=#&lmQfgi%${lc5>q~(*rTwx9ITK`dxzWebN z?f)q;7>F$h14?Zke9|mpT#dt)tF+Sv*R^2*UgoQai++&B?7EsMPpNG=Lf zv>6bF_)s~d-U)X;^>UbYp(rRo*Sx5@#>FFTwnD3${-G=i7S8(BxVD?glHWu_0l6|~ zR*$3-&yj9*?ah?2hv}SIGf5z=m^I^PRzZ(n^aQ~}X^3X9F%p&)xa}d76H%{$_2kediYJ{YR7}NxFC^X3YU7cSs zrzIP$Aj#wQmWVQ&;95tO3Bcr8swg;Hk6}+3L%$*Omlu_JVCT*GEjudDQ}kavbaVDB zW|Bh!nY`j3cR53Zq}q;5;@%$_IX08S**5J+a!cXRV@|xg%d6olW_?Bse~!&0#CjJ( z05RI1?=|G~mY1Xkl|_&SANGQY2&3I(iEI}1?Z_dsr<(_ZF$z1eykRC5g&|vNV)0&N z>_GuoDr3-CCCxeZC}WSO{Ei&4$wqiU-EyaJiXjB#1P2rZn~xHvQC|)-?wLKGC2a!P9Zk5UPUXj7 ztQGFz@A=Lv+T`K4y6s)Z-|bOEvAmoac;bXkjr_LWGGNVo*Zk4lyxQk;`bLIY#5S=nxZshTLwL zU@vWFzYVq-U@Oe2%`XE8Gs^(Bf@QpL%AVR-icf9Ov(>oC6sI=ZEawqbGRQ@bae@(L zd%)~S3jyoqBF2fzKwypo?VmVpl`?lmVvFi%+|w)?y3sB?s;&k}PQ} z%4kejTUdzu1o2cd4aUdCJD+|f`E{6O2cT+=8aN%W`Pk#us!{U(X+}ks!otD5duYOj z=Uo8D)bMDPT6{ysWg(m3T4$>+EGZ@WusAt1NOYF1@ZI1yWL5;3dU^cIAX zTAdQ;&;F~mRHu;6&9vCD2(-1<)d^hQcK zB{3nH$BTEKj1%&t^eou@t&ctXQlDY`bO5CE6d}fTn{8U%bc?bmSb&qV?-I86*hXh* zKry1bRv$3rK&o1ihPR(p~qNz#gf0(4XTrstCrNZn0FiGK`&{MNpFrAqXI z8o91PO|rLENfDA~3wq5IecORCPVMI1y1yEe#7|BdRM1AQZFFe(y#NQLu&o^ zTk&0GFum5f>#p6qkY_{;rd|A zoLHof@WB9|KQXb07o9N#tTU!_#kO$lNi+4>%WMmg;)})AE8XP`WCr7$H&hE}kpjMu zf928pGz?b+f>}j33SKUWhJiLmqjUo2&6pvzLFCj2N0Wd>7~1D!xq|>o{J;b^a7ir% zovx;rCUC=_lpK>)rkh<0VkRr)K|)J@c00u`2#sKcxjyo+N$|w*IjUJ zJFO8Jmq_rYQWn`M6TTiLwGc48b!X{stOhKaE zRQG+dC@I7(f_`Y9kMyI;Fv$PFI0+mDMILA+OP7|u2mn%Kgx`)Fyky~gQKI?$R#`Mf z3KPbD`QQzgWBdrxT+8V4N_gGP50A#p&leDC^t zoV`>0J{YYGWLYv%FRNLBg<+iAimTJi%*#nc%wqw=@|>&{7aER!$y}XEPQv4*O@|_v zLlOXIZ+*OISzHjG*}4ED%v#X-#F02RT7m{w97YfNJn%Js%!tg#WZ-F?ix;>tj7S*F z*XkiMJ5;Dx_n6~#FnQX-Dv;=aq2vGvLJBy!+6hjq8WVSup4J4({AW&Uf{Alp&fe6_ zSq`@?XU?=@{s)BoZO>0A{gNN{7q8t0IStZ^bYr8Qvfi^jx;B3ci-Xks#FJC><_c4M z4M|@mQei34kQqZ>Ly7~to-zYM2ob>VpCR7xD_>4r03%3KLE|T)X8*fQQnyN?xP6wP~38U z6D)dI73^JZ0rR#2fW?I0HVMkvGn3nPP^^3WDJGMeN9BLlJ@+AxafdULznvtH_cooL zG;Pci6@B@5$RF;FJL8e@>sBPuGA3n-QEFLJqmGeaO(U6y#gYZ{lJP?u)oMkVCHnaB z0yF2%q7FaNd^?9NXgf8v_Vhp5wP0jYQ~MB-zkl!eMCEGLHGj;k8FIz%wbBe_-%y=XLAw7S8k* zjp0-^Yi&F^#{ZiRxo>PQ4oBKt8a82XNQMznvUd{QySwZcoK{Yo;LI-bzabS z9Xg$g|8MqIYU&XqZkxj1IJFx-&Q-?iY<&@L{KfMf9Etyfw+1d9&#sQn~&*60z@6inMtWCTh zbLwgFI5Cd!kC6-13LrD5EetJHo8=+VecrumQYa{BaBkKwO}rQQMPp3T><0DB$&>H8 zqNTn@Q~zv#ErT&=cAK~pnl!^FZOI{CMvM1`$XF$O(s$qRT{bKp+o*|t9@d2P?0r`2 z)Rj%`Q0KunB9*-)lgf;-Rj*k~=?a3?EDV4ACbgu&BN%9CoEB$8Ko%ycxZnnJy)qT2 z=fSYV6NOm4$RZF4{$2On*Att+@$usFra%dav`zx`M;75g1{t76 zPReX|yae6F7gl<9PA`ibNC74y^y=7%lZjybh4 zzEFWoaH=gHBeld}k2N!q)ZnQ%MzAVs;{DC`=o89#baB-xhFCcW*aYAsWd6}jIx5#q zVr&J3W5CvSJM}|;=<#xNhxI-fk1?^ES4T$#tD}C@ZMSJKJMr#(=|8$fT2Gm^mME@t z8qY7GnuMgU9xxKkK#kTL^YlT<`!etZ4S9%0Y^6CVmx@(?O${1r&UrRC-Ehk8IEZT5 zme?xRYtG`-#(rsqJ>2P9$}zv-Ywos3YyQ^2j^nj!*DRbfXTiKV(ekcAZhGG+}TMhCL*y!F4Rr2US{6`PW~5CJ#mQ~9auApFt^5&UgG!&Q!nh-MdgL-{5 zW^h09V2>r^HFrM1q6Wvq;HU4$ij~9<4Q0L8Z}*hg#1;bQF~(yUeEfpp#4E?|AxHg< zRcx1zlN$8e!S#;gy8vi~805R|d9!DI^7;!~?DPK^r#KSf&|{B3<&1O4He=dUf#n|$ zFqc*^Dli~;F$X|`7hiz@7l#~kynY9S4n6?;qFImWJcxJb@x$re_No2bw8;eHg)bX6 zY>=;K@eBM0=d{GrB~gnwk*+U#blclL5`xW#esG_Uy(9Ctk^fjrfcA@86va?mLC(Yz z6P)^~4Ddpm>k!k&m(Dq5*6KqHgeLf{Isax(uo5RuZLdCjpL|QaV%d_L&uv`ql*Sj{ z&=#jQiC+XZlY3cRuon(K>Td!QL8lrwO2{B51ILNvUxNvd1p|sw3v&FEB9YMG66dJ? zy%`LYd)2P9?*WIzNa9n3ew}bz>t=W`Dym3=$9Uns4BRdxCSb#+UE{WWyJ~loBZn_|&>~jXQ~m zNv42FL-d?9q!U&n5m#$1evD#7|vVh5eYJ0|klVv5n858F>T-ARhNdov!GgLH^5 z1NNMZhZ=*oL1@!J7JkFw%o0Bb9Z~NL9_nd+eA>io+unY%qh5`DKW6yIfxVA!cou2Q zo_VnG)b?v@8V!(B)bXHE6m>{59w;gNu`bZyXS zHgB4{zdYW>W8bF@9SMyKyrf!>2j4HI%5KiJA1o$LpZ}y^Js5w9$7i0=pZCEzU=c1? z&7Xry__%)?ez5_=J;5e0+i=5SYfXniB}!9aHnCz+_LZvEkSfWQ7~qG0m$bS4t~!N5ETWbhin)$2GYq8TfTpF|iw@+bR>~Fw&FmQ_rR1-$Q|0aOzHo~n5FKQIB=$X3 zwQ6#M11jgunqeMSVYkn_zlXQvPzD1x6yY5k_1_F1gv%Ky!aE+1PbKyl;{o%V;!iBq|ncQEY+7Ticu$8;@y# zZG~VEK9UJFV5XWNs*~ck zIJlbko4KQqVU(-1qj`ZP0<+$89luqpnOjA~(cItMIMGVgE&x%J;88}c=@+{X*0&@2 z_Tz4_I*g8bjZALK1{6dI*L`&$p^h;xRhUDSlm=RQxp6z9_T5;#z7<>u^^Gco7Ni z$N<`Y#4wXpDK^@5DcP(;?cB}(|2yu^z!0%5W!ufZNDU_gT4B#(VqXkNti+e*ECxYN zh?=RgG4zNLU%%;jM;v?N6%GG}`&|h|cfR}7qwaD>+Iq#3#m_%*$MDa3%8ZX**TX0c z&nRVPDN_mKwQE*sa@?|>!T4`gl`QElm*}G{Jh=2-+$bsI0!Q z!kgNif3RWt<~6@fF7@RS;xip-C|(7Ie34 zLB`0#5va#u=5bf1d=Wnt5ibzfq;0p`UMn}OGhZIc28@nP^c0@I|1P{mTau*`=t!3u;uAc;^TeA z%9XW}A9zHA=S|{X?kN+$XNTeqCKJ9Meelu80sxQWMi0|(0l*tE&56UGhJ@t5CLpBX z5X*F}k2?*>JjC*hWSw(lUQTV9CIDRg_uJO5Ud_8*v|pau-hAqz3varm#Z`aDT3MJO z#N&jvsLIpbm8zt=RS$CQI+vV>87BSxJ57*DYz@wCm5T>@P_a2N2DZ%A;~j2(;N^D# zh#RFf^-gM3VaJ_tS1+yb724883*=R)`0);YEhSz~KC3w&Yf6pG-mryV%-+Jx@bWiV zX!Vi_R6LKEEZ9w8nF1W)^)xWy`U;qx{BOs$Duzt*T*_SMeR`aymP5H1a_NKPr8twP zxul-nkk6;@=^#r`ozFBdDaE-u>~AOIQlR`Ua?H>H>}A+|H{uJi#4k}IQ{T742SZQWy2qY`NA@xvqx38s03-VK#3}+vuTK31voXoP ze)fYuWtpR}wcVrx3%W^rU?w-MTQ5o&u3u+wnfEHRkT9=E{z4zLzeo+KXqAUK@exU4 zCB8Ig1pjv`aCOGKXMO88_xTXXl`TIBVZ>18{KCDEKp+Ma3O3djhdJh?t>Z~kAir7< zq`QFRVNRpMVht4@WrmUFF(mkBL4wgjh}U&W7Pp{cE4U{_YRz>h?-5AI*ccCr_*Um1 zkD1$0u|bdh^eL0vQ`rFacw7;0e^Il+pLUlI%*4BPSwll%H}aP(VV6X{r?g)>!k&V+ zgEzN9+O%N<0o<3`b~}NI;Nj*klqPRP6Jceem-c!O(ymm$cwWJqbYQb?C~~izKojxQ-0kV)_m%s;xfdjRnT?5!6x#S3e6*985-{8-O{2jhwHX{J(W?*L*XG>QRxVwFP>tUlt`l}z z+6>@3fsGs13oV48y|fDS#)};c4bN#I0OQv)<1h78P&Ow2;o}!xbz5`{CM5i0eymKS zj4Kza#jt@Awg}E7sy6Asfws{cXaa_SqVQ+xi3g@p`nhfl#>8wl!1Si6C<~q@0E3ez z24E%g*XJM6;z1byXm!>XJ_aQ1m@mu*~@x%DM@fRT~y{m)`SxMo%Ixt$)v_M-1-& z@%tz-jI!k`-2A{}zB2U}t35P|Du4rlp|Y{kB4D6FoH)|WFeoY&E(o7V;e@bm?P^kC z589HY%on`af_9l&1H!6h z$>Wd!+_YgWy2g35T#e3e4ZV!3fjzpB7L`ML^`JWjQ@?pjV`%eFj35bT-GvO zGCfFs<`WMczy9t{Pkzw5&Y|Yrz`Cov>(1Rc`hvcYM8k~!R?6SPlfX66C2<HdMui}gIH@9!1qntAAzs%hS-DNlza}KRg|+B6%*1E`X2g z;RNF@Vc4sivt~@g_c@FKcw7;0fAQJVewwJG!c%v##Th)SQ1+;=ysuNZfhNOP5DYY;%Reub`-h>k2?k)UGrG`n>2~Y@Bx74l&azg znL$e2SR@^J!tn85j`-y5S4Q>kgS(nF=cMnxdi0j-9(l+7Y#RZH6^fa(VvEVQO3j`1 z)52c*Y`D}?VDikFPz^<_lq*-SiPzBuU@7ME%lt4xE$f(tBvSx$t^oi%t8xLfN0R44 z#;$>+IjVHDlqHjbda>^?=K|kZsc{ix5}2y#Olooo1HR#10uVif5N}T_&`TuRit-vY z+6luRqa8CCMmxT-DUZ=^F1?$HC|6hNi3OOYi{=}rHnK1ztCSrvk}=W0)MqGUohxQs zAHK*QFdTcjT|yEF0R|H+zJMYIpPw}tya3~05Ij~+Po>|Uc>SFZ#3P?pL8FkCm-`N5 zi{iFu@2HhwkO}}4y8YTFk}K?384sT1UB0MX>VFwdV2}#2-k`pWg0lLqdtlEPnqPW- z&E59gY-byW!O^1S1MAi!9`8Yv6Ad&R4-kS4%$boOcs!+o!NW@sJV|KC*eGvYZ>}Lq ziWq$0(b?J#aJ`@qI@^%9{O*41PDqVicUM>=HfYx3rH*&WCGVkqd$f7raXcY!^Tv(C z`^QL4LCXC8N!3xDLRWrL@5H>g;U!Reaxw?=R2>C_`G+XQyUFAPUHu1 zGEU$fgeE`YKnaq=GF)t$7ow+BAbwhvL=k@0l@W5}vsx zJg;=Rr_P~A;;|qC%?zO7IW7CV|E6H@dz0puUx(rr{L-XvPtn)fW6wHuk3f+mhzhee z7ONP>>_@zorI?`tMiRGVnv`12;%& zhD6R5=}?XA`%ZTRk?^9yRAMiA77YZ>-of#r0X}xE!{<74?8xyV?#{jJ`ki;*bCdnH zAZ1|Ub*O_5KZbe0K36{Lo!roUIjZ^#VOfPJg#vQJpe|3F<}3?N5S^{(Zkqw z$VZ%bG8oELsCdAU^(K5Zno*4z+#m0;hJMV@{(?ji&ymGmZ@92%*39VvfK>zrKsCE& z#sEM{^~V`)9?0+xci0$4P^*jVrp5E-iV|xrnZZN}fvW{QvsI~b|x=n`CWSj0?FVb|3)psNHUDrN~{AgJUd zh>D6t$w3qp2}4%oKUJsd_UXD^{k?AnUH|?2G4#8A>)dls-Kq|+zxTR(G8z&W8cOo^ zI!@*mo3bRn;D-#1dNSS&$@ZP+p8o9{fBj1f!hZa)|NQuNd+w*tj$*5RiU``6TJ#fU zMKvRl3mzN)vMa1;_rJkSItZr16exJBe4wgS6|oFY>Dq7B1oFU-f6Ctm$0VVP`f3>9 z&ApD>P=g8>{8NRrnX3V^{CVsYA2C7Ugb5rPaB`JHv%_Aqa4T%I(n&)vX9QBykU?c< zfXOfGS9S#E9A`c;a*wDUUjoEGTx&n|^Cx`wic2@&wqi_X39>@)B?OxQk)F|sKr zBr~2~vEp=I&Y+ALYs^I7m7yw6KnW8);tpwxRZ~(LZd?)|R{%^FQbPjoeH!41IKQkXFDQTPmurDJKoqu5PY^{p@#5unEW#4W?bC(0<-+etLmk+>1(b7 zB{m8m>)^>L>|Lt?e3Q|FrOpd4wy6E00iT@lK^;47+ZIABUsBI6e)I!>xckoRp19to zTkL+oA;z8RpX2b(X<&}fsrkeARz`m{LgF2q)&rowwW+ zqjA+N|9Pg}jnFE&I8RJTnB?(gL7}Jqm3w5{uQ3=F&+9ek0t+s*^)7qZYEwO;bA#rB z0JxtwY0~>n`0{IZ*^0kbh6h&xj(OujU-;qWi%nT7M4M1c9NBQaW7!o}`a^zN=MBHS z>W{zw{nE=Xf6Wz_^8Sv@x&7Yd?_X^1O}^H%y(~q{8`j$M6Z%1GXS1-Ojt7zL_TGPi zc*A3MxmxHrQ-yZX=uTurn@akmQcv-iR9pEwlUzS6%QUdVpVZ{SDK1jwBZaGdnPx z0568%`QsPBnI;yzdKS^{mvqFP+eayxastjAcsYY1t#d7k)}@81vu@+M5+gv4ExLK) za^?x;W*uUB0~=)qnj=<0)`BUZ)bZ?T__L=7?O2s@X|>X?br#J;G6^;Kgx=HQmz5fi zXGlLbk=TDOv)G<&)eJA!ju6)5J=WC2V;D049ur>xEVUWNRv1N~wjws%PDx(RJQM!Z zW1QEtpR#%6@h9QlqW)S&(BGb&U@#KWhXaUU)Rz3o#NpqZ;isp62!BV_m*>~ko-Qj) zZ9jvpwO}h&4c22=XN#IJq_Z#FoO?L0X+N~=h~vJ1p-I;=dcu18*%OtU=AiEg#m|~> z|G6)o|Fc!{6KcqJ-Bmx$#N&8hah2&|Cgr)r2alUDGd!a&zV}#E507EY0C-$@1X}7b z2c(C&bLIr{JbSE4lv?0XN%~_spS4H0Q=TbInDO_Kft|}jNrCZQE_Oti?YN&gXFFpH zwMwXL*s)kmI(IA@Xtps}#n{A(b@ru44`YqfvRH^V;c);4fPr7!7){2n=)HSXQ!<7?_ofODVfUeR>aL2e|o+7x?k5aTE}m_;Wxx!p5v4he^~P& z#_yZd!vhWHfB$?a2q^~QZCeZCuTR1>MG9 zv+Rm1C5AR~=UE3B27q(Fa|)IhzVAcBNXJdgb zRy~#XSjN0xY(YBvQX2w`q>ONAJNy_P+AIX=ibLDf{FNDC@NJHpufN84YKG1eo8)uQ z_k#bXYkzh7O*fcxn_c$QC!w0Z--vT*0=^4#)Nv=_<2}Hv@B{b#;pjIUg6kTc;T&RA zV28c;BY+jV@XS-OdchCES2gX0Kg^hIM(wl^Wf`+F0O&yksd*47u4P!R#&FZsWPk); zZSFifZNkOsoM~uBhC;tb#+nzXK<&2|Nmw;Kzd}3GXmhqZ>)H9~AV0)eObcw)$9; z_gGU8k73LJcuf450AnkSzZQR1F55?0PnR>e5$Y0?uXuSn#nUFn`@Kimw3U9QhCl2= zvI__Eu*Jkz=~udnjkqQ%(WJldBb^Vj?uWd5DFd8QPs5o#AyMy($raMlrVY`?FA4F- zM(SbX3e#q&=4*czzD&p$zSM2kZs}?1cjRyYW|(45>9diW{-#tDS6F$v@}2XolXcZ> zn4AByoj+rsnud z*R*L0yX2hja{hQ^w>AyVDb&~S#2#uBZL2`Tud^c0z+xAjXg!^HJO-Yo?}Ah_4McRQ z4F|R|QwxJp=5dugLup}$kQ^0c;GcRjU2q1W;YD(`jJa&_D>@cJ9oP}sD(-2&?Jj%z zT3NpgnG9iDE`E46fbCewhGkRjOqY5W`-h4J&G~i7Ic3FiOL)%&QbTRdDVvP3wa~EC zvQstauK0|=4(6yFf7<%kyWfoO%38=P{_R~WPG2>rN+W3Dp)Va9)EQwk+b zUl2u5))%56mkH%>!-)!#45spZSe16_~IpQC>Vg4N?yo)TZYbmtRC@5FB64 z>colXeeV>_xXkkDa)u#;_z4p>f6-2OP6VA;UU}X*_@nw2=bmMr4K~|mnW<9&=8I9* zS~CD{knCx$o~kOsAD#U$TKUo@rk*;zH6mdXi%=qAYCSDH(P?YyN-A0ijIWkvi|;Zr zf%H*fIVJSq(DtE|Ph%q!-FD;kVKq|VeQLgNOw8He`U=T?^@2Q^@BIMaEciL4Oe|F)yzqt4!1Mrt7 zUdQW1ozSp!h z?8!o4!=@QsH5d@xVbSzm8G04E@0uyOi+*|}%^_BX#QlGf^=;YA3W zOok$Bw$@WIMiRE$_%c1#l>EJuK^0mq{16vIW9f>au@EHU&{%3e?E*MY7?WzuokqhZ z&8AI!!K9zdsVlC8&!Prs9w=qZy;V@0K^HBG69NQx0t9z=5AG5?gS!n9+(U2=5ZqlS z_}~og?iSqLb%4wNpL_u6}U`rF-m?Y;iCV7d0Ao9xbM>&!Na zT{ArE)js7g0iLDN&9z&_Qr4vz$@!=pgyUti^n!QJhwu+#kdp0SP4zsg>40n^wq?|*LZHTfPpi!3;9>< zI_9oF`)3TD;Hl7`pHK!2{nwU`7*1>|sv7!Rjx(-{*P&09j=p+1)pd!`vxs37L;q#h zD2aGk0hZqR$nq_t(?3HGCKqlCAU-Ruwr%_53It;tPX2-ppR4^Bl^#MT)!!rIA{rjV z^O~B|PH^>JFMrj4vn@F)m1H5v@PBLFPYF``Z;fXjwVrR#<8X;NFFE zU$|J8zW=h#%P)u7Tw~C5q~z^)DcY9IKok?U$bJD>L+4M0(np5+1;8)t!ruTk&-8I$ z;=XuUeBdd{c8X+VtDF{7n4)1#HUAW0I*hM+RxojV%Pp+VU8e?Ni9znY%lnfSQFhn` zlX=&N19@v;3+;q@54|Ew2igRl|4`MlO)Gp`vbuEny83$pM!*`5_qB11(V+#W_rjt) z`FqE0(nbYa>mXrU&9!J%s^cpnHar>RzT{c#C9ohDK@J2DbHDC$Zt{HgW47W>>@}4t zDjKF|dnhrCsxP(a)|rQAG-`8 zCnKG`%rNo&mxX8ihw>7bi~Qy(mMY)6%E(0GA{WMdsG4N&m~0KFIw)w;Juh_9F&(YN zfbq$9=)#5Empq-=Rdd7{Hk-|tq5g?RHwI1ecfX)3iu^Xyr4+`88U@H*{RN+}I&7kv zQXRK{R;d80arj4zh^hk>?_co#U$N_UevDv+(g;JEZY1U$mF=ZzbTY~Ql3ay=PEg53 zk&?LgV7N>)fBBq<-Qw>3EE}Ezc_1v4+7MDo9UDX0Pv$VC3AAh8?@FGx)4fThH54^n9n0cK;I-tE~(>df*@gVRz zsKE5$ha2;E!petyveZmE{&)qv>)SukC=-TOV-;0(WTa9iUe99-iEaA*s3}IzJ7Mth zzJVKA=_VAy>a#?D1q1K_2Tf+9ttI!p|oQs>1FAL>le@m*y;O zqo>0<|bm7ObT60-nD>l=C11lqACb6>{*PbeWhg+p5BFvoBpOPeS0z}CGn<5IH zj*V5XoR?D!Y2B=s!JaU3Va18!Z~IiQiq6^QTMVf{K{VncP8_&Kk!;}azCqvw&! z$m!L&^5EUNZ2MUEb`1JM#ZMK1-dnfdgVZ=I+Jj_26zx$K7DckSZ_gcCr zkXs-kL9YRx@`rItXG{)ZIsRJV4D^aiJfCzSSjW*Q*}scl5y(#1#S$k}sHj=Qoz~m$ z$yz2wDOSt`4@E?ZGi>^c~eS~vE5l8DKKq^_7dUWsk_fP zUgbxDz-uyJo&13<1}<3W)lB1J1GV-wl7fd?{y0i-nOxiSiG!x<#_$0HV|<^%Mom%< z_5nRH8lXMx;%PahK10zlq;u{k?x_mO*4-RkM0EG=XJ-{iL1h9#M4O5>X>0wBjqO zsv8k(mP2-9KJPSD>u^hKviu+958ODjq-^|`s`2K9vM)RDr>pe(H^=XJ{f|i_2@6c; z#M3B;J49UNxs>v%xS>026HvrfLSZxv#ntnbey* z4)EWFKwfkC&MvnK3~#TT)Fll+^geW)yCIvL8*|PZI+_JW+itPNREcn|4lWy(WVv>~ z@8vy2(O#pTPZ4w%jqy~U(i)yecQM2~jjrzr9Y_`I11uC=EkxpH6ZP;b1XAlyjvA>I z!RJ&LrxSih9q*^t)w?(VQP)O*At>Q~%`Vtd__i0PDT9K^tGDaf-BQO3-#YYmP}p+u z4|^O@^mky^UX_<81*N4BSw=ZA|H|8H1JyS||Ko9z!$fkg8nZ4}zlT6EqLZ4Xu~JY-?T!7-R-47p2(=n=(un59I~hlQ+5e{(fTj`021_qm6pz`h8*d?cndiT8wi-Yk zA>He9e3OHzWcB^mDu7+jzM>C1OMEP%;B8C6caqsMx~xtWy(YwE?nU4Z`mmfJx-UEb zLG^t6L5f)O0r2^37(n;kaFu&!ecgBPebf6&lnO+`x;Djw zma&bmyDq$Q0~4i ze`(P%^)UZ6RhZrwaOh9llW*N-yz+TwOe!kW6Nl9vq)k-6Vv|Sn(dRra0J9_n(sA?= zhIVXal|%)DQaGdUt5o8_l6@l0InhsZf~Fa@7uuf1^emZ#OW{-h+(@>bI2(#A3n?oQ zq@`WWMey?vWakf_oH?KHd>qZLIX^BTiHG|OZhT!5{fwFXY1sW7;9ZoRo^rP*bZ){h z+qn?#l%*)**@=0^U`EFDNf>a7-Lb1qkNjqP%uR0PH0}%anHeBTbk2L`H=BVW4APtK z7MD@9MdEny45`z`LHUlBcsBoMI5fiQllAnhZNcGqMh4Te?%Y@Pae+D>ov8xiyiiwD zL9hPkqe9}LA>?!^dRAJVFI$e8cyF;`?@#*D4oWa%%w z{=?|f(Xi4yH41HyJ%mz%SI^8xC{Ryng4Z=td;d~=p<6|MNrcxv029#~0zv|Hpk)*} zyO<*D)9WF@+lG)7fWcT?=TEZN*0Ye9d-wFEYO4tgUsdDbp3YU-IM#K~Ovl}dUDcY` zI()LO`%%#S+vYDWl`aBAGetYBKjuhwx-S1>Ho1tJ?qBWPm+c$edJq?mt2(etl+U4q z&aLBum^DB1gpk`dtI2(Dsr~T#@%D4t_mO5UDO&Kf(_%|%R$ZQ19_;PQ$b25+)u#VO z5y*8BNITnf3!)qIZ)Jgoh$8qqNNJ-?*`7?7Xs^F^#q;auh2iiH4pkANp`YbCkfDt! zLms8c(w|xRw^yXmrv`g%t$39D={(Tgar%Wurk^nPH?-eKN%O0AH>sa4i zMyRSl0rUhWz}G=Qqal4G@>pF0vzv2?F-B>&ZADwA)9-%ub~%gB$ge%7rpSrO=U(=` zqZBthpb|PS1aW;C1GnuBkHo;aSMDs&EDR?3ZoCRPcBCa0%1y#m5gWU(!eX94h21l3 z0*$;si1%Rb8wr$|)=6*rKP+PD&Bm#`33^CHvDK|g2K3rv2vLGVf0ruPPNgJ2LtfW! zKi8uNCGRjb}2_G+m}OryGZ0ZUy-ZnBw3cY~^~T+bopI@V*>Y5?s0we6PeF`X<628Wz^g149TxJ=Z`uo3vyY9=Q*d z4ZeKT$DtD_4@YA}Mo**PN$hV9Br%@+e4gwOA~7qbv|>>nY^PDK_|PgAReZuGl2_@1 zm+(k_-A$&cVa|U5LTYD4DBG3|jo&b-mbwK?6p?5uC8} z@Lqz|NY0218hqpr*PnsLMt7HfR6J^Y?SP_(`M*^diOa{3wi*rUy7whHyMH8tv~Fu> zXMLk{CpPIB&9Ya>+}duWcICCCUN#v{r{S)3lB+#M>_Z@28Trb#?6iEuPYsHh`Qod$ zy(#u@qMc7Q?C0j)yDqcfOvY_+#{hvvDxs085aOISvFOas{bFYu^9RzElctpgu}7>s z%I$6;R>K3v6GVc{wyt8pD|8wa^=(U$x%upxPtr}IOHV{)76oTM;LCVg&%)|Mx4?DG z0+e)rR3jO%Hct6FU9#UMt>*CaM**BB2LnYC<+b}zZG!WOhARj3{qJ^cV!(c`iPzJ3 zz?Q8+|z86RfgmGXaUN)z#=eIkD~@y%v5igqChY7k12B)FZXe$_4lvEtIKzOw*c1HE+R<#-)1v*|F)Mjj5dOwckU)wT4s+) zI$l#`&-xebP6T!x7jycn{ESt~sdPrR7BtS|FUbXB(3&2MK43mSaw-9tF_lh-u*o8B+k3=k#S}fnmFiQ37 zLkaOHlG2-+nLihE)u~Ua`gJ)lFeH!g*e7sGt0y%Nw00Jl$GLg zxX9ISNULd@4&L{vAKSrm^ZzyiQ3I4j>d2wjparA4<U&>{3|LYxHU_>^zbf0EduAzrO)->`}WdipIr1lRz@2GKXt}Awn6p_as-}~k+&!t zEM)P09R}_r4!X)wep-#W@$p{;C*ERZYfBO`Pf-W>2hJK%do&GB$xkH+lP&J*cQUvYfph@ar~ZQqT`9`Ek0HhWlGe1HTeXHEZRa*Cnho@{wytvxP2Q< z?#D3;h+yFr5@>J4`Rn-vbC@LHIi#-g-uI*~mF}T(1V8jb(FJJ#eVckfBe7vluvBGn zbsc|vW8$yk#Os=ds%ZSeh5lOXwMhMs8}xa?-z#fKOH%$)?I%X-;pan!-gh2}0b9Lw z9WP0@d@H6UexH60jP##=zv=8%?@tVE9QEk=Qg@k%P!h80vt9o?qN#WO>ACgn1K|{k zMx%Bnr@_sO6K0J#sg&q*&ZfQl4Z*EbNmrdllr+H!#jn|$o}We*2muZai_P57yI&*t z{J-9K*ABu=?f-J>ILl27F7)EMY+D%I?v%h=Yz&r%@p2V&?(Xr-uM;-9c43u=FPmqa zqD-B7%$$1MrEu}C`0Mz`1ES7j_AtBiAbHhnWBU4Mv&Z_)fI-L7uRl%)Z26C z>66Tkt(Oy)_gI>d@LPHNCx76){&*8y+#K-UGff~$Vk#%=^rR-{U(JPa^#Y5|y+ScM z=qvE7nDH!jbWQCyRG#9wyS1DWNyWiJsl1HnL8)WLtI%lhA`BSR_2v|1)W0u&8EeTtJ-j`jJieP*!Zo{k|P@ zL4_0DGuAnMldwYh|Rne=2D=rwNc>1oAy8%$|M{fT=Z%ijSgT1rMrzkacXL6e<{ zr85nA*k&@kNoWT^YvFQ1U!l(HA^}C76U170>~JHi6aGihW@uk9)8 zaE}AZdOn-Z{M@kwZ#u^FhXYrERUZemPp!5;h=*6v8dZ9t*eHnT$iwzb4c<} zkqn1F1=fA94;f0~#w*JJivy=6ZSsOnnmGYpGl;y%aGq=9&3H2 zse|9&`#HiG^k;`4j;m}Bq!qC61`pU`_M{nSe&i}NsZ2t9DBqUfH?Tq0!&6(tA2g9T zGG#t^oKt1nT>uhxEV&E8xF6nqXsrHyu&q?gQ_VTUbdTc84Jwhmed z)8d70wZEKjT~-h++JOU^VJ_39(`@ejQhN5E$uD)?#?**TWBFJv7Sf3)`B`=~0XEDw z$}BKVD`S;>ZCX{<2NZ$mbgcZ(U8nBv%(GZW@3#QI(`h?V=-iA>dQEz~oy;;*Rm;-? z>+@V-*ymWGm??M_;#Blw$Clgo73WwLWLVdWtq3^Dzf$6N>DINF<8g|u?fcYoh5!?!!$X#mpISs7f`qJ0kt2ip)i3HM1WX-Ah9ykN_;!+{&7> zP-$EpXKf0zcD;CiF9nyki>wiS(l3jt;{k21v);? zKJY=Cg-qwM2-V+x1F#lBI}0i;bcdueB3c2p1je!h@9&?_{OZHryiy-wqS7Rk%e?@7fF&}e5df`Z-x(^j;2NL%jKwavq6{NxKd@Dm3^!S4 zX40fRIv$6r)wjxoVBBz_Yh-MlbfIYc@6f=l3o)`VCfng@D@)Q?)P}t$Xfv(m&imoc zb1GOY=FVOXfgtM}l6AjW#V8r_yMM(H5ZC5hwXR-EtY z4WDboc^b;Y*^|{o%T1@`dXNa0jr<1^=}zTy72)fIG!n6O&O4F^3v?KD&Ebe=VkFg1 zbzR-Yn{EPDR4rnuu=Es5P`}$zGC6&50pa zS@E8&3+h|yLphvG_V+is>HY*nZhJh<reZRz^=Re1~CDC^ytAbvF+TU)poi}^QFsSf|gs#|;B zvPj%ldwd69tez|U(4p!&4a9@5!2}N9{&vw3l^qg0mnM7Z0&qcYc*AK)Q@dI-fCx~O)LJJccaVo&#%ZwJy^-NdyM9D_;djMxX%utsCuaR}bD5lRTfrYyL$3y=3Ql}SA#-W8 z%oJ;_79bM;4F1=?o#3q;@i@l*oe;D{g)aECmCd9fFsD78PS)qTT+#MCXcI;r5>P?< zxOQ&dmr~|j_`Rk^jjyxok0drLpW71#m$XE&(;B_UBHgp^iSj+8#3s5&(gT+b{rEYe1&;d(>VBtL6MaQk38n$y-k#I|AFcE-rB2F#wM+= zrBIjpm#ssLfVMYL-w3~}{0DaI@yN=8VuZSzsinGj;uqz0rsT)uKUVL$+DpsCS@-wV?j$X_U&~kFRJWKaE`NX$6G;hYsk{R09X!nIX#>9UM2V4H2r#Rzs z_VHc2AY>_>zV~;b8;=anm$>rDAk6vqlmz2xm-E^k_!zW;)2kwaTLKF%Eo;K<5Eosr zx1BDw1JgE=n+ZacK5}8)rSZM&9W7x9UtCaOh&pav;~Ur>(xm7k`8*FrF&7IqKq54b z(uRiZJBSe}N5e?|wu+Eh_h(}y#0v+!Zof3uj8Fx5!?cP<8rcq6%r6?7`L@+am4k|` zwt8{m{4pH*YzONcnmqDC2)6Io>H?qY_CZ%1&CeO@r~`5?~#)N25!fxWX|d~UKqusu!#HCglm7CG4$s_Y=&do zb8<$vip?-~pGw&VHyL3axi3uJCw6 zy{)MVVzGGLSJD53fgkq_jGL2m(p#?H39106uR`f`pD$VZEF+51Tgx*1uSaCxpjM+e zP(PeR-?E})^hDvcWS$=Q5KB5P08r9y+-Z?CW%wdSmHfFGgA7;`7I1VebjmaG$NF<{ zE;oz8EtI;rw~p|_EU+ZUpV&g*_R!)cUn!B}2Yf8RQWUz6d7ma z_pC4`{XE=>a%A*{Dmx2_=x&3s(POu!*5^BA7)s3JNmJZ4PPQ|mNB_fGSHa2%WfXIN zP}6C9SsKDxtbou%JO_7kN2gWJr?aTr-nFv2Y+*!Y=%sR92D$1@XL2~X=|HR(vXB}-OA1+jWhL75 z7>TmDt$?6b5?aJydqYrrcBm}v?{bBw!ykkNd}M?nucX8ub0z=k={D|6@hRqx|B_|b z1_qTxt2EE+0}~otWpoZG3|A3-c)`Q9iLyj3`$0D$=4#jq=`Y$=@<}DAvt1c@jF3RU zOXQk(Hp*hUhMpSdH~SMIiFEy%EcHkZ8tW3m91-Q4%#WG!l6S$2sT_q-!pS->$rW`A z9Inxbk3(j7);Fn4)2X3SjMuQr1+#VLrDZN&8h*8qO)T{CaxK-cA?`72?-z5bS{DTP zpLMJ>$$W|w9Um~dXB64zTUV3!f*kVj@uG!S#?B^*3>b2H=y>nvhs^Y#Z9Fu6Xi{+! z^78xO^gr{Bma2Ip3+r(@G~Ak3^@%Dwz`LJaM**2PV<7j{EC#})2e)h~pGE(Xz)o-A zI8(g{AtAFi;hE;Q?Wy#{FW%UO!s&c7uJ({7z0x(%_>b@zqW3aHQ5 za=T>hdHme>kqgOpPAFFJV$#EG&U>I(40@KWW8aYdFWVS|&fAYce!S?<+m8TVbV zz;4lBL)jF>PM=4?2@h~f^sXJBn7i{Cyvta-TsanKdcf{IgtS=%B5orGwHcjevh?1w zx%s+juS9>_%Rd)PnM5em?pf3nAIb&VHU?p79oTr*@0o{Z2NT4R^KcVmL`wvGLPwNy z`T2JhbIpR*+Kf?ZMA}m>)o@z&*uY_gh^bxCt4NEf3d&%VO_x_zED-8;#jTz%z)l2M zA^-OUeKk`IozrPI#c?by zZFv3mde-`B(e$%SXT*B-#<9Hwdje5QkdCHxKC#FiB_q}LcX}gsQ)Y?kg!+ahZ2pcA z*AC*y+;=*T=*laG!FX!n{q1d*QV_w&R}#fl(a())lT5@-5ez1K5g_IB>o=+O<9LMGwY> zu~69v;@K_a9jfm+4%H=*;*sVHq`wzZ28e$xS(qR1J~0_exiHu9bVTk;4wrhtS|Mgb zDd3GlOxR9hHQ3k3ho!!qoY~k`P6B`g@CmP|bUODLm)sE;A7-HDn)v4a{BWX3`~4JO zY7n{F5f*0)cOrCqdA;*x&46tZieyw0LpRz!RJ;s(j z^!t)9p^`=iRio(d!^>Mohu=oP2Ap48RSm~NG$mfZJCE1}S2D((+T|NH_!v)0?HW`I(^2~=3qA3sCd-;WXp*e=~ z#Diw42pL$4AlDV?IUFuaY(WFZD##B>XQF5a*z<_kXaE19oalIsPXUM&LI{ zqyc-Lg%X!3%@1;^`&qhFu4GkH6D<6>w9u{8|8&=HYg~wh2XNh3zfhz_OJ6 zVYF8n@1vixe)_M#QbTuC3x?B2cCga#H4-u`sXCqn4$ zOFG|q@s|&r;-kX$5ayohmtu6dneCr^WZt`o`^K*uA+0sgha1PU`=dsuR=HgEm@VOM zBm4Xq3w_nTS-#)(+JVtxH0*3?fka@Ve}fwTUb9ymxzaDye!XBA`nU1Vhmd`qF{a{d z{V9BXY&2}Q>9_BGm+m1E2 zHoN>C7lpNR`{0;xT-~%Q_b+20BG$zt_b)YY7_@GtLqjkWR&dE^-&C1+U3j@l)&{hV z57a(zzD_dMfYu(Fy=cQq(?7S)2>!LmPGvNA?2{1|fvLsCn%1NW-bmuYcQY$s4WaG- z;+i?I{?1X}cwPSo6s|LT#-|2GnxS*paeZW@ykj~=Mo_)Z@p=Zf5GgpoPal!I2&U98 zae20x?j5WUB~mK@@vbtLo%i~K0t-%A6N0*63twj3e0WF^|dt>45= z7H+KLxOe^n9AC$@bW)M;6?`wr#()=H{R>nWuaC$^ZcA z0H6`1yKG9eo~YvQsmu3>^Kd8~S}oatGs z3!U&V`Tb6GMp(QECH?=0Xb9{pRgRXmjl*4c3<|4K0XB2AJxAkLu-7fw)+61Kp{n;> z9FZ2qeK?8MB7-kAaS!AcOEQ}ijx0O{v=7LZh5fId39hj6-)3@;?nHm}Sl<7sq6?`j z!8QBHZb@;PBVh6$H^uF1qR9_#0B^061EN1?d;Cl>+^jzGcna*@HZhw-SVQ1uzV_he z<~3Q>A~H>lSG+A%Ml!Wt5Y8e9=%yIDm9VHjcIhf3JQQbQJ*BpPO6uWE!vB&!boPGa^wqwKA+vqn1B%w%oY`-9G4TMTs+K zv|(4I;{GlmZd)iTr!*{zAMg=GZW_uzw6RMzc^>1bF;L}*mZs5oX$ZwOw#2dcYfAAa zP??h)^S{%*F*UBgth+8wRlN{s!nJo?Cuh7W`A3*i`UhP3-hu`w(-nx~S7t_zPGFYZc%sY-oAgAak0O?`+iOTo?Y*S#o`p~UbU8)Ki1UZ$S=yx{c0FA5MClFKc4SX?XY{gMpWUfhxeJNkKmJjz6)Y6)*oL)pZw*fLXvnnT_*sSqHjhoMWziGFN zCg!aP&ZUX;*ZM2>)O;g1ZC639L|ZVui?%K-iqTOJU*#Ol=EUqum>}S1a-EuSfi?5} zBO&;!d%x(uH#5)lZ3&WiSJstxX0g6}s=5eIsKg5Wm%^h_z{t02pDf1CMsKo>C{~yP zYTa9yD_nM@`OXU_Ltrh(>rY$(U_b#t1{u28x2&J`VTpx zrRpIDfBNJ;rW)OPiC8LO)Y*M&@T;Ih_KPUmT7oLQupWs-l7o}-pM|H(<&u`cGV_V((Y%l}DZiB_l=`pv5Nqe&MQ9IM8n zf~bywbAL>#TA|#KA7{d2r@eB4D-l=^7;bfS@3-Ce^KjEo=w8lZX1H&1vQWjVIC6pHt}6uXmoE z@@ZHwsA_~)I7E%&R9mKnvhGpHtl_$&wdbxP$c(px#{YzG{OOTqU-YpT2bG1}>-w`t z11+)kb$5<-!=zVX_d!#8awrMiZvp!HV^X!Y#+WbMZexErr8x5S>KggY+U}{BM=hWJ zr#s50wlp!Na|JwVLq=76WRH;m_Lp^vte()Ru>Cad^ZGffVq{yH;J#(vB)Gcq>SYUf zNS7r*>}7U_5KNO|Xc>df+ttWLXqOosq9ROy|5i<0D`)i6o>U)&<733}R1&CWZTELV zZaD;>WofK%8PnviHSg=7Wz4+Lb*@1R$<(QIh1RG1tUHRUbh{UfkUw4 zio7TT-5RMpDoVrXZyAL z)ZhVgcd4)Q5xzeajuL!k5DG*06GHYpI#C|lQ8EGBa;s#{QH&Cr)>il%fz3B-154$? z47~Upk0BzP1z{^0{n)95)CzRJ54$G4UvdA0u~*KYd;lHF3tw4!ZKAHocq*Rj_e`_r zeFYRCtw8>OqN@%y1Egrjb{hXQ6UTg*PG9d+ur8;5HWTFf`fSjx77*;&#;DmUO)Y$+ zYBp@B2>)`Db)+e_IYd>HKHNx~J1?*-4Y8l~jnfT|ukAJGY2yrs-uaGD{LW>=O#966 z%fFKPrJ2ZoTwYK&EFFFN1IE|yVZmX1&q;JGGy3^i$=_C(>0BB6z`4%S7n%~gEJCma zD$CCDZis$Xi+ObXdQ3U=6+ zjz{7WtlLw}rOB@I6+1_u`@bF_Tv3(Uckt|bxD+4g$PqVE)G}a1v42ZWtqj#c0)2cQ zIhohs&x56mqJP55zS-Xs-@2)b_fyXRC|pJT363DUvSSXsk(+!!3JurMN@oGni&|>* zhD&Z|brG?9yi9|;zZ3_5`S^wGysk8~mqxgoY>!&o&&)I&y2GIc_u z87fP09w-*vtG{qs3?y|TkmX1I3?hceRMuQ^5~0Rmq$u+&dWLO{IsIPHZIqXr&y?;J zk)&?S3-BINM7ACjz*}=zp{{l$` zuRR2u(zCiDI_bmTwW!cK(Bt)?v-;)JfYDh|*mhO67~1}Xjcnos9dKd>;#lMNDb7^) z)BZ%M{Nj2)y3)kTZBugjSj=Q(X~Q`9Z1a}Bb20#YZKX9rw)p6Krshre4K>$uqDQ+A zC-DLG?yXV#mfJfvX##y{wf7r4VKns#YQJ{~OJPinQ&SYJ=O4)DlzHw|_{uphpE!*3 zOTs!LMBPUb8351Z@AvZwvDPASzB_EjvzDqu?S8iJC&KFPg|*3Ig*;L&Yl(96_AKwoSL zs+)~&cW8aihW3A&jGo}GzE=ugdG{mUZQhLj2k3Wqs>^~y9(avTX3~v(6ta58PMp}q zqqO#c#CuQXs{a!p&fkp2K(>cy5541{?c+wMCJH{iDDOJ$duhlj0U*S6*`MoNo;&pa z)W~SJZ#9$W{+Qvtvb*Igx|9KRDB%quCBKV0*Pn!tUM?M*d$}A3aa>H)ABaq#y&CWt z*y6L(GkIZD2@n%3<23pUS)>_BYyh0~M@jv&l*5|udU8L9Na;h_I}5?l(^Dpu-r-YE z^Z}?0B0OzJ=c6jGu|)^=77>#wS=!_`9J!jMw3wDHL+6_eRd&`fDRx>a74Vs&idz$~ zp2&I!(l^!>wCa#cyR3#!HJI_N-<(<%Z%x*)lGEe0-fIB=pu4?qnnyPgiJ{kEo&Zzv z5~F>f_+jA!cefJXqloaX8~SPocG3HwD;*m*de{Mz?Ac^}?cQi*i=;9m;NF~|V{@$Q+uRmquKn2ahq5AoeS_w4kx_d} zNLWeR-#i{w{fZw)&`Z7VFOmCb6sYatxXb18vyRZ7yQuVrewx4E{Una7U;4d@+$NiY z;X>~2_Hnax>_cRUC@mH!zu_zmW7lN8vR~g`YqG?!b8-9+$RMw+?@1c_aT5s-E--6f z@x8G`{>P}_tw53&*LV!C9ywzBe|gU<<8X6re)W|=+We z=qu}|ndY6Lv8ygVIq*SVqe!Xf(=cm7CVNz-Bt%>$;AW3gS@5$#o2BE!&yDTBTfRZm?gq>93vfY@zY4ha#2sSRIx@cg*WW z9o(ZT|H5+q?^r;e^G(n$vyazmvbRg)J|_ry#yCvtOi>?b2Wshi zt@myswU(Y7#l6GcDmW`Je1t#KvY@&%_o8oSj!h;yCh1bGi@BGa-PL9bEUqBvdfhI6 zD%yfoQnth7<)9P0$I(=H#6#=4b=0s$peR6r5f-zXhJyRI$i4iy#QMNL5eAo@RDSEI%WFmN-hv)7OMVBEMk~1urYml{sR|^ z(imJ3U(VgHO?xzmxMTtg6=91S)?}}6heTY9sMQ$YzkY-1h4ek8mz(D9V^%J%1lxqc z@P!(@zm@21nVjSFP&!6ZNh$~UNk&^s+E0R*6 z!9~A#>Yk$&i{ap}8V4$I^0|ys$T`H(=)h3xT8OU3+f*pvhhEvggK# z{a3%5+%LrqyVCmOh2sq`Caz;TLfEYK9th2t>1jN>*U%UcYPT0OJ+?~_tTL!N^fOut z4cs;s8BETRZKkuBuNhkL=@?C85iZnDF|U(}xu_%ekS6%(E?+Bwtk(K8(o{-AA+yTV zsq?)>lV3t$lu!+f7folcnO%&>jBoJJHA3{{Patj0aUQc5y1e9lx}}wp3g)=Q)8p8m z8FwESqA2)WrR4=OYo%MC!{%D`wwc>r^Ut+j8kWa*dqk$#H6kqNW+h;Zv5~JvP`;D$ zR?#NxSd>@cmELDd!?0hifXmpDNxA4J&LR?>vRewk)mNl1upRE0i8F(QrgAU`I()|F zLZkih=$jL`wZ@P0I{)y{^854^^Xdafr=*gK4mVCNWPMz2gWBKF31#~shvYY#+u^TR zE7|hkNc--v2yux7IO(hZ)BP3KdwSlNbdv=V{4@h9ZUS+ZRZX%AxGtrED75}Wro>LB zQ~&_0!GRCzfsH_ei5^CgXGh$IUK?au#xdYcB8nr!_g|!va>|*C2L!?3@>MzHinNrb z)?R0o%XKa=uuJ!Qm0in5$Z@Ny7wLoYUq*!nGVky*fpESYO(~R#478Rw zQwRbg+;`X^l}%$9?q{bK7!Z)GgR0hmW>OPs{=IO!7XHk4oF7wG5gXoaN{N|(nPK}X zls5z_pOMEdH;90{4esa_NI@TlZE!hAw(q?#z1LtB_e`!HFkzn!Ui}NuG=KlqZWYL> z)gGa23~Xe<*mz2LzaRDFVJl6iT}+<|goM@TQCKH&*^8_@&%apDEB>xb+eqDeup zKusvm>0U}{OUx53JVM(GG9VbP%IdrmX4>vrQlnR7ZjGNv!2g@YGS4#!Wjo(v(HT?_ z*}mm9|C50s@ZHnRTv$t!G@APaH2PEK!~(xbR7pG(tb|tBn$&7{1p*UrX3IQX+aaJB zaBvyTDajo@)Atj9bDbr)Y*+A4V8}0GZok{8*&qP`6Y=_M-0Wv0&)l>cvIfk^;LLfq zNt;D%mL+LKHG#5dvhtI#V-teJZZ-{5WM zW6>y^dyFTHS~P9Eo7i9pzPB%EOeh}MIGhfPh&QK9HddAbC|mR14fp)N?gEIP6r8EP z?Ny}bLZtbm-3NazCBh)>tRP&Jjw^ zeBHqC-F;9!L}nw@6_#v}gEza^r}inLB@RpppEQGx3ohU_^7Q6!ZAn=Buloj3l13(y z`qZqCW49*Zn3RGET~J+w(wO4-0rz0(z@ssQp78pxp1xT2yJOJ}aN&hu6I82U*t4cy zs9^^6h~51Vev91rA_ElXw=-M+_Fo{0rl5Gm#)Dfw$d%$-&Z=WN5 zhLuX=94ewUjl+DvVdNC>Ci{#A=WNO4TYD*Aj1rC8S4TIw*MSM!=IHuI(n{xcI_+do z#*t|%rJ7INteoh0(-e8Yz z;K9&ZQD7=7A}=mP`>NSN!=+%rJ#IvAE+q+FGqZLWw%H??CtVf6OgS&1g`D4WXNB)` zcO9bf7JLx6xq9dzh8A#x1MLYtwFp*@DYTlig0;8Xm#i2A5#9%jSr0{il5nyk^F^9D zfW;&3eF_IVrpoXy3Ak_7+q^R!El6hi(OdlilYC^0Tqy4x=dg06-9 zkL&usEHs<&4t6@{tiAg)Qq13Cg-m ztlt2C>i&7op*icc1}^KV8p_=|&dLCDBRsk7O~$q{#H#j+ujU#oC|eETh92@hzp+T% z809{;K{dsgw)cp*McCoD71nuQ?Ye7W$uR^DfzNPgS|7I#PLcbcfW+M+3sRM>(@uDD z>6X;K;Fy!C+u*@j-#0-xSPqU6T0fi&#MzrHpi5}*wY*u=PS8FWi5N(7hlh$otTj;c zi&+h-n2p2y{nUp+vWOXSAE5^Y@=7kPIh%9!yn(FA)eH))4#w3=1-5IA9_!Q;l*zg`=*P`Ng;;8 zy~WD8V6QIpmJ=S)q%~?5{!FGR24^-{s;puLgif-x*dyG@W+sPEyk!Hh{ke3XQ|90z zO!wp*z)hOWJc3rKV`17Z2XAwCt-_S;fHV9>diaS{-`y?RWvcJgYkVl;)u4z&?+rpnfM4a#7g88>Ep6_V5>-0&RL9%Ej>fo^rE=EF}2 zLe|dz#cMsJM0?&UvAigdr#!!rW!nobQ0aQeHmncdAbx0sD6k+0`jElh=(=u`J&_25 zh7-J5!P24LuzjNO-5_S9Tv~z~I=HbNDgsBjR1cFc*{}J9x9>z7;t~JXZ8u7+(JdRkVP+U1 z1`5h7^5j6D^&!|!&V37E!LmxnSs(1-a0n$H)8|3(g#%FC({S^_K8$B0%zuX#0O>=5 z8}S-t!M#X5yau_q8R%jXzI_NTv|pvRCI|RA%-P(*`PioOxM1*p5H7bqX0m?gtPK(r zirxU9V-?9~#QHE~@IjvgY)`pv4GW$^^<_=3;%!tW4ohQ73TFb5+%xBa`0Btx!%t?{ zhKs|9%57P+eVu|NN!@HzY&gb**jI5jw~bJ4A7W|><_;#K=+>>%^676C;C|hS48FtG6|8XaKQ9GTc2SqAZEI z1pzg5g%3{`f9~ zFqXrvr)%%jY?)Yt0+A*+s@Xk6+M9w^fzAhKisVaq)}Nq-&0UMkeUB~X6FFjnp^N3g z?uUzzh6RAqmKB%sD{{;}GLsghN8maIZBg7Ot%hA9dAlL)@ZB@T$5R_YYsB#7!=~+Q zSAjgOQIkA+Kub#E>S0aaqBR7*fx%nKjG2Xdk-~#}H;?^ormV(0%^#J6WLn9)B z7k$@&&D>7*gH-eVZ{;6i5Jr`|BenI6!xL7S&meF+o-xtjbNvRKFfyMT(OO&s&6;1u zo6P_y2TQ}i*4UsMGaDhMi=%@0VntYe0#O-dv=#0wmaTQHDAKoH7klOjNqRTB z%`%}(p1xAK-XK!Q?P5j=A#7P?P9(nt%fmcTsmiF=;M}f7YtI(GBkWwQrpJ#sGBW9|putM?AuW~dPSKX8mE`E9S42+#Vw+HqaB@!L zG07&Ic%bQp(FH0oVzU|Z68YtlmT#x{kJ8Aur z3pxh8X>xrM&^3-2=qrY{+9r;DWR#(COEzGE+GmqN6Fj7X*E# zf@vD7m0HWX5-!o%Glo#!$Uz9??R2wuPD(YVz5@PN^!cBk$-m=Sk_Cm0#oJGL&nup} zt@(3dNgT9Zjk~Vw>ksZ9ABr)1vW;WR9WyP7ji=Tu2ozg6x1KGV-EE2ub2)5QA7NiP z!8{xb&?Rmhe&|(8YMM9$xNaW?ogM?64CvXRt*&TSBDwgH`y5{kE}Tajp9v~+Z)c&0 zIP9>D2<~N5Ic!*_w7VVrF1#n?v6$I009BFCMxHj?DwsLv=&e43iqMmTB9Nw?3|vCW z$jPE^Cg>bcK;E#Njkt#~g}L0)M{QMPJb6P!&)@@eL~wW;A{5Td4L|IIxZS|a#DU?@ z|6*2Mc3BK4y=Sqtk`;Z%0WmR5f*ZpR47b9`7wm>(vtq+ZBizyRWb6VoZWtk_Du|0* z!)^0$X+ij^`D3ysq4qe0Wea6dd4&zYJ#+OC#jHw07I_)f5M%E3wF;z=W#GkAPMbT) z|80>-&Zr>aWCSeG4MKuKDg)XnrA69j`ZU^u-Qi!h$?%06Y3Yu>xeSgls5}%=WzX!{ zLswSB(zZ!#jEdU58a6hm<&f>$3xOo?M3Y5+j<@VY%Q@l@`F#~laAV3rLfR5Ba2wh6 zw}g<%w1~hVT1`ILU(U(0_gXHo2R4F!9lp(deqIr?V%#EJ5>dNdP=ciVe?qE~wKO|0 z4x$u4q9@r$q0|1)KP0)wDuY)ptP<@;OKP~g2oTmR^k28reijlcM&72C$@wNwL+sOT zRSgg_>lAvx5u=P6lnE(t-qR+zXP&^CgiX&EyF}nKeaMUOylHkca7=KxIyav4DgvBf z-tIPT23cdrA;`MJs`w<;HNZY=f^ zG$d)*t}_*c4RXa~cyc4j+eO5nSu`r5C2$#Rt$?3@CPN;)0C6nmKxEPo4zhngOwSn< z4e(X4Y5Pn9*wt|jOzTBLgU$;L6v&5kUBS`Ne`%BTa1S-_c1Vle>_aq}sd>SP(=fK( zD!JiJ|MWqAJOaFdnMN6r;I}Z!P?5@$Q!iJy>H}?ras%}KbLJ#1Av-cbYehDr$P12fF>NP*?4;Ap!jd<(LFDYaYD-EbF{y5w+e)#3rx=$^0N zb`sxeo#Ov-TOg;c=2tftqS+Uzw~qp1>|91gc(4L}H@COzHqM!~RK1ofYuHFQ#d5Y8 z95R>oW^a3|lY zmN078B~me8>jO2tfyjkTvs;hr#9W;0&r`gmT{7n`G6yzd;pDv>tV;aYIS)C?-HYkwACn2dU2>MN$?1XKJrVqf&+6u*JoTD@(JEez+3A1zP$AsC** zkp_j?EfbqiyVL%5qbu(w>|ORyIE}L1!1Ir)xE9pwd~!_$b#p7*6`VAw1$R90g7f@9 zVYQ8UIAOk{g^(R{EvolGcr84 zlR6JbHbFqy@HYKJQ@MuH`;%_+7f=%JU@y7G>al9d*EQpwr_8FLWNj<}vm5v+)L1ye zLt2XBuR6x3vt%EyJnjkk9k?%{jnW)lK+VI(e|u!|s4s)>DgfK&F`g*h*o?Q}x2=U% zo)HnSyjEdnadF_>o=N=nErnM5Q}p>*LQv9yIUW_^mq(jhz5;GvA=P_AZfp?!9ncdQ z+%5-A!I?!(ZaYEbnkysuGGOBT8o1M3B2ST-GmX^jAfvQ^p|RZ&y0 zsmyt3wV-}9G8->#mVXSj*ou#C7$%Sd%2f(<8mRH#Bc;62%5s@5A zS6{G>jp;5Dx}=SBeGpmF0XL1LHVQLHVq7w?a1y?--yrO*m<9*pP&A`E^JW;bCw4t@c@Inv^ zO&!F|PcYm4Y2DC5)n|Cn#j7F_zoAP}OWzM1nm=z?+j}lp2!u`4rb<`%=lEPyu<}qyBddJ`Q)JE`2BQw@n|aV<(a96TQKne~T$a{XAb7iO!Bs*MGQaZc`A*ekt1#f7%oG}N}& zzSd6ultTDav1r$R*yH~_fD6;%|NVeGzvz{*UO(jX%-(JoGO2@EEbJ$po_w}&Q}D~)^6b6$Hf z?Q>N7*cywkrT8i^m-C*9li_v$s`NlU_gvbvfYfs29Nl40Yh6}uGyml<6I1(7GzBU% zs@GSY$-{fS?8%m>%@XcU zb_!cWr`91K$HYDXzbX33Z5iQrp8=eN(2b*(6Gws|d@B*9;4f}o+lsx!*yGVaY3^NN zzQzehDRgSv-{?H5oBRZpi4TN?aW1rNC-T`HUa5EDt}iU_Ed2f;&yUr#>d`GqohO?Q4W#j12$tUz@1N4*O6<3)8wMk6==8~f@rw8U0J zl9+yLg7VaqRn1@lJ)x^+IovG2y`*--@;VcBVC|cp*&WeK?Q_qsEk!z)bl>e`d+U>$ z(KDh88lE%aZ|E{tVe|K_dEEBP$f>}1V`N)tk(mhw^LzK`6`M?rdTkjIM{+BOaeAYu zaB`Lm7T3z7dFP)devkN)iY3$`;l>4bc+ba2fYr(m42P{A0p0`s#a}%Z3(#r_N4($7 zKY*T`2@1bIlEXBGF0Li-I(0@tRN;}&E4pd-`hEQ@XiDOyD1y4^Yg<~v&N$pMF>RR{ zBC7k{T2Y{TuD8t#hOTI;at3G44q!U%6cwd!zsCJ)#W3cWrLNw~XIRv3`$ow1a(w{0 zDLE7R&%oOBkKc=@G#^{`!UC9n?whXlm26rB>&ZPf1k%p^IbGepIb1ZTWQi51aTukc zU4%_Bhw1a*A?6uC0eo%08;kJsp2qE8>aWU6f9YU+;7#eiB(ECXbuHd6%0F|}*f^fb z2QYQDw&aovb0lYieaNr$_q7wpLw+qyJ#NxwT_N|dn@4ee&PhDiSJcjhv!Xh=Zp$DO zID3_hp~dN?D^|K!apJrSNv0;fcauI|?z65s4kZYtL2CAvqj}W@EI=gu;xB%$c)?Fe zLl#6BX1Mi#q>eV^#U9E3CC_wkNbOwnNhepv&$<%6YrAwJpOuVsz`kPTm!R1UK3XUt z=bKtTD3--=p>6(|y{L)f-7gF!Ow|bGY-mOtgZV4lpB%zQT8zCVN7!59K?G>Qwtf#R zB=o)$q{ZC0KLxH0>OQJ>pu=H|h2+22Q{OAl+Eu%!Ip|;N^YQMGHr6U4wCs1wE-RnP zyrQl%gb*6^ByNOy0$iwv)p-<$?hyCB!6JyDS~5DDxt`tPj~#kZv4{n#WYi1Ki6C8i zKxO<+XBLvzHq4RCoWG~u>Ohv%=F*HWXE6gNjfIN?s2SbLa@FrY!EqVQsSmDNgW{{9 z1ridpjEWtKeZELJ@+x2}J63vRyP2Tp^*Ui(U78yreEW)qWo^&}&Y4=}{K@>tuZno_o*0*lqI6UT7jZZH;C{gyim z^^KuHQ0dpE{`Z~K$%3Y%c<^K@=dwoZJMev3eu&}V+*Flne)he)(aLong&Q=J=L#1; z979$#ZJr=5RX4n!Q33s?efM7Q{*N!d zjKAu0J$~qn8B}X)M9gG!-SK3(Zzo>0)FV61-YoRYO#9dLFa7sSVj2TC4}AFZ3)s7y zervt7VoJ&X7(+GriOz-Jz+EequP7Jm^<@kHB6hmQgTz@BKUb<%4Y!vOa~g#Nwdm=5 zE`C$?i{@DpE8t5Ims@`OhA~>_{9JSx)V<1Y@0Lo3P3IYJ4(-chNeIqg4=8nsv&<~; z4$$454!l4;H=CVI{=TyC5{q9@t8F@}QkEgT_Si8!so8!|K<~W|*xNlbGfwu^F zX^57r_w6H#;OzFZqQ#Oo+AUV*hhZbFl-^WNvUh+hGd$?WWn8ZJ5)SaFdCp((wuicm z@iJ~R+2AVcwQ83^SD4j*m1?WDZ_#$FMV8_YHK*ctK}@v4%%H8zOtz#lqA&Fq*Hhae zhDA$9hmYS0&o)DR43s+zdoC-y+i zdwqsBordR6Bs5Os0l(Q3QIc2M=^oNVU5&VF_qfO3;gDLmu7NEG@{)ljCFy37mMt~Q z#%{uAM+<^>yF-d<*?j^M5krihm?uM_Bb!X74sKGrx<*u2L0a*T5w)*r{DU+dq^kXC zzTAz$IRpKVBvnRFPWNpq-#Xb-L^FGg{ac*{CBs=3wedPvL#lOjW!0ngIM`GUY|2`{ z(x}P_=mcho`8mlua@S%9w4f=K!4|-}sCc!*ZME^qZN4tt$%+z~Yd4WER+B~oMlfOP zPSj!PTcrJ$(dz#fqfyA-xIXhJHjKoaCCKG-Luvd*h?2&YiTc9&DjJO)0z(y}=n=n9 z7H{WYtFeaOfFY{hL|wH2v1tEyqoqoY2HU*J4WsEidPF#EvHL)%%Mv1RPQv_<#zX*O#?!3oYpl>es!o7jDoao>gc z(IpZ6q#M^$(->x55-ryc5bgdw}uJ6@t zdS{EE5g(~n`ZgsOeIJ)in5)9ZZ?OvBx|FAg z$aQlr>hL(K|86$u!QkhR3skzDL^Uc>_o3b*%l7FQ2H} z=RIG|PWTw|WkB#w{BYo(%^s*-Wg2ZKc^O-?GC84BsO8-(hcn6s^N)Xu@!3}rxKzi) z17gVc$QLqs!Fs1|oI&2O;LhWed4*3>5Jc@Ho1Pyy_U^iBpd&nd`xLNw>UjEIhRfQ*$h^u)yZ`T!ll*IRY zRP8fb{paPgDFtLO_q~S{mtE#ts9+2~eRe4aTL89v3w?|}&*^C3e8#++i&{3Cy!kzs zlcdd6DAn^XJ)c*8w#|X0S`j~hVALc~28`Gg-r<=&bFT)+Jb$pvnu}`JS8uZWunx*c zoNq2NmX0&_d{ZTy)eN7M%CSs`jPzU$pD1h}+$%BE3$a`7U^kpyD4F%y&xrm34q) zJ+X$Uu6yODW*WzrEf_z>V^i&b-$r1 zD&F#{h?ym4T6pEVkXCv~`ULjnh_Wu^Jd?tw+t;u$cXT~Vtgo64)C7a@-pwB80;=lYo$flDy1 zeKD~Img9bi+%z-5{E??_TJA>~GeQpee4% zRpP*snXNh*58Bokd?5iC4<|dS7EVk<&&k?)3wrorSvuP6JsME}Ua~N<2}?z@=vZ2n z)(mjqa=$Hm@ZoOCUJ<9#|lreJtc3?yXBc?!faqKNAhI9H{t06=`@Sgx)`| z-ceVWy&zM^?mj48Hri8mWYNL4psrjK%kEKWnsNADoQ6eJ}HN46x5pjzx>gk8diZYBq_*#Q0lW zS6b0VbrcI)G)24J=)m&*K0iix%uZr5n6;=~@ftqIG_cTab)#xH(zBueq0ErI+ZaEnXAD2Z`?)4M%m6*->{%S3nPsDSmUNi6f7b(=J6ecbuS zZ&bc~->{N~ghes1^PBvEF;s+6E1tfTLuEKsuw``n)skb7fZa=SnMeT0>C4JAsB}j} zh@6J!kM_2CETpp--{v33^8$<$lk{zwUiVfpo$K-W9~QpxC{5m2dMngjREM9f&?TDxrDqivqTIi*W8UokbCWLT~#&!4l`D;h|O(;wI8!+%8wD&QJ`3Lvb zHfd+73{!F(^{w1Ka+n^>Ecd>ZrDt#I$?tl!N7t)MlTX3XPicAfE1vwuQO;BOt1Sfp z_`k$p%Q>-so7y)0T4srb&d##L*8UbUO|j{>xqYULwYlF6#S>&;m|*(a*TTRLsog&3 z&Ux>%(tdO75{$*OF$n&UPZs*sg?tb&xL|SP#vca1|aph@!O%?kp28+OI#lXa9 z6Yq0cR7`2Gy@bwcq^ppduAkwxo(~9xX?L=mJT|C^n>B9fZ`18Iza6GK_g|+wetz{; zPrSl_e#1eXtVzb&&v8Nnza!wU04yI}S@)RGSt_)_DF;?Pd>NLe3m&<(R?8!Ixewad zdw%`L*RYY3mO?>74dEXUIIcx~J35z+;%c9`@=g>bqpR~R=*^r`=j2Tmr!^R>;*^#w zTJl&~@KGESvw*L_&YB|a<%JI?Ur%?amwuN{ysNJ4hKSL!Zn}_5K5@(gwd#tJS|8MF zFm+#+H6|I`C6yCD*;ICyS5v1TE~gxg;6@%U3)2Y3lztPWLK$7*xLQL_y;JR9YMKHc zWyT63X5;U(Ou^2apt|>0uigEMFCg7Sg0Uat_@PbrZW+Pe{GOC#7x+|GiE~D*VW)OPD zWnNw3%e+rbIy7E7tZ`0VT?pzD{cFsg%RP;@vAk-^=H!t)71ucxE3y7ys15#$Lx-JQ zhhl5J1pD&F^!%~-WgHltk}gMeVlwG@v}bQ&+2KXhjX&v?}Ivxl>M^>W1@ zmfq6bK~W$7=D7AEWfq5oX`hTr?Vbicthj%$7Dxy|p<$6I^N=;wk+y@lX8qsgwN1ME2>E0IU7IXHeG8VhuSX)=c`7N|&QQE&Wqllimq z_R54J_jA>fkuSa<0`&qOzumO>rS;@V>1wcI? zo!cF4lMT7VdXEP3+qpAY3~8opA2BcsNTW6dA{7NO-Q^cW!vd^#7?3adTH+=H&y;vSm!r>GJ2d zDn`2aNmKJ>j|EjP7W_&`e=lQeaJl`TtoNUh--Xa+(kJc(xd|LSYbI4h4t5Yve(8~; zV48}}Yad#P^e8pXYR!r+J#1RmE6H?|TXHI6mGiMLgy|0-@;Z%uebGtCsl9ufX}R1E!R{a++g{mrF%90gr@#J|YL+|qe?hi-yze4%`%x9=g_ zH{|ua0B7}}*wbRi?y~spm`NE!ja0pL>5Gdf^{gz(bcM!tA@*lmiMwoK?IyB zPF|}>J$bTM{^Tcf*Da0s{a<|gGi ze%yp+LcdvN>X0Ac#@Qv)xl5JLE34D{))4+s;f-~`{Q|UL8yl+k|F!aesRG2-mHj_f zjYf5Ils$ICnZM*D5_nAgzhW|JL3oxAbgi{N0~l`ybG1Ki{t!rYmvSe>Ju;R-Bn4U% zjclP_dz|DQ3V%?>bnj10l@l;MaX?qt7YUGUBt}<0dyy>MWCZdX92p!GKo1^HH z>wvENR&8a3v#3Q4JX7B9LWhRJ()YRGZz+pL0o4@sDp&CtKfhc5ROYr?Vnc1)(toKi z``K|@oH1an?s<&kFmpA}L*==DXo#Z459Gf#s63BOvJ|@y*H*KJ85B>&a!m0k>Q_>l zM)Y$3P?yvtjM2fMwV68lr%+bI7oS*I`j0B}+XY`ieCB7biM+=bLEEBlg~0|D>8BnC zQ7{R?W8Xfw7ELj!x9rfnTExCecA4xR9Rk%rtfBnph5B3yu7_R-#lnK^0Htb@GHc3G zb0km7@YzRB$-nkk{IGk!ShYUo&!cZ78T0DWnKTXgJxEN}2&pr=ej_T|zv+Y2F`-rlCx7Nay*B*85csnxrEcXEUtp;kBrzR9E=rMZ0N!Z)IEe?=jvXE1W_UOQWi?8v(+OtARs3)ES}Q)MbL6ef6T*K zMvP_ih7iNgzJdDK`5h?UIlMME$Z1;a&WD6fk z$9U0SYv5+3l3#RVWa1l31K(Qt-z#%hM$;|eCfRZuqKkJf3n_L2J?wcH1T9zTsS>vH z)9MWiUk{!n;`FUvG7GdD;&dpz$V@6M*An@^GfHS2sTKTNoxbZFBE5h2n|UqxGneucKvEKC~aFPeim52sY< zh>94($nRbiLkn6;sp^MqCoFX{Df;c|k6)ge9mrRSY!4c*?uv}qLJhD|t&e3Pj;X1i zy;g-&lrQLbedMKn`vQNO$a`o#yI$XN(*avhAD@BfHFbzh6QLG=pS*#^%Coe0$zIir z4h}^#f1ft)*B?}TJj(xZE0_9pfyV|?2b=^L$=UdpbtQrY-+@UmJ{V(qB_w2M^2E8{ zy4!5gv}TbupsdI2;ODd92X~ZLUg1Y6zSL}P1*H`mNn3Jkad_ZI%DeHRnbuiT-;wiU z1HPgke=KMN-*9W0>5L_D7F}oFKVdGAs#TTqd2T8tAtb9+t9<9_1KKw^JwrZ9Ct<7# zcT`+gn$MAV{m6`d6VK&!PMrRoO?{Kc2Jbl5Hf;_a{d+x#=-kZK{jUz)2SUAi2-7;B z9;>9JZ@=!yfee3#dVe%{4tEs^0Z2p#xOueSe-rcx$59}-rCF{^d-G=VmdzlM!C!G@ zw!5R!dn{>=)iH=Y0f^p8GRWuDthl5^YtQ&EL%Es~t@TZeOsjg{>vj~>t^i94D%Ti) z1Pop&<*s;}%gj|BIb=#cJE-ZugD(A-G`a~Ue)@CLO&pa_F1I>jl;-yF#*|84~iHY>ks2ARM_s`)#|2P-PC+OS4QP(wPw#aIb*}@n7d>AqfV2TTVy9W3MEtVM+ zH@|rXVO9_e*Bw#*+`oC_kqD)XS$?V0?wfjivy&K`jTB66`V|^D|5AT-@BMGy>jJ;+ z45)69PF;@NWb=6v=$B(FKHlGQ+pd&Jcy(vU@vDC|-=k+tv6#E@-OL$t^lnI(L@4Z1 zTJHU?G9Q6=%BNIP(i0TE^Nf7m#BE+T$9XSf+xC9c<`j^Uu0W==S7dZ{rjM(ioXWjB zO1n~Z!#Ac?xFW;MR75(N?~!xSGsdVAcP#89pC&f!12mVRD0}v@al+Oj!!?6}Ul;NX zCJhqbNvHA*)gLhovD;yf8mCmS_Q87-h zTEBJ>wA#vLkj^iEBaEt{enh4_yms(1)A3YZS9rxcLXGJ!T^n#`*5t_$c_JlXLguFF z6aJjxyX59lL(JA}?*iy%vD^Ur0qU)x6}YBXHi026tWA?DrW9XG&D&`SMV8Lws+Cnx z6O*3OYYR2O5oL~ahu37KVgUlHHUAb>$1UIHr(g0eS@}df`v+Ej>bG5zZd2&{eq#wx z%?j}AxXnE=bxG!$00S!aA!%MD>v4RgRcFV;?FV^*nYG3LEcvI+5SnK@2To3CoRpZW zse-Ag=wR$z>i;myDU{rO@!DsG^UaS(ns3}5`zPf3TTOOSosjC1)N)F6K)*D}{UWT9 z4i^87=^iM$WY%w$Fuy!%Ba>mXS8RE$cXhcw?dwe4_GSr2J#l=(<7VD+gSevt;ywbM zqb}VVt{#?U&gH~IKEpJZhIUpwsB215NASy(t3AOq zN#OeZ!^OqgC?Cb9I;MOMGw5HGW10Fq=26Pm)9{I_z@x_^m`OG;zwzxG&F39eI-@m; zOqTQ?W+}Y$UT?VT`v7DVa$`-k2mS7*TiiY<0PX|4d}b6492>}syqUE3ULkI}{<~-ez0)p|L8yiiE-bI^z9tD6kJxr(j+a()uqkLZQDQxhe7~Qw&ffdHXPqqR59b~Lq}|=n_NtAIZes>P=0&<*kJA}clDa~k;dcY0XE^S)L6(@GXVxrjy7=p#Auh`<| zcumlydbMSSpStJ@cwkZM?!D>Q@SoJ!dd9M1J+s^$PD&87&ewOvVQ4G*8WL{74~hfR zPX$$-dSt9sT<#HQ6CDd`4C7zNSp* zZTWRoxk%#Ay7*LM9JR^xh6m-t1eNE-ZdWImomcZi?Dd7m@tBPJe}OELxu?tPKdoTSTq?^60pLO@2) z(0Dr2`x#GHnDom5-E_sLwc`Jjb5tf(3Ag*2$t{0MN6DzFvXuk_$szz48{gN{LBH9| zOn4ht;(l~H*)k>kEK(@oiMTV0H@nG-0wPPwS1)MR0Y9O+E^naUyP=+|)8EBegSA7A z&#!g=nXHeLAE~6_3J!PnU(^@YLE0V5B-TpvJ{lENLP5S;@2Eq|h1!qxKnV5hz$uFE zKXoHlVg9sly-T@}d7#kKU}zq{;lp={PF1ZK+-*vO<(KJZ_L)Ab=@9RlUuoGsX){Qh z$?sMv`Z>%(14}_m(~>|C&Glti3o`JM4=B6fm7k3}@#}orW@^RX$Bz1j=&&7GrZon> zK@GgWcNiYw`*X5;Y2BSU*OXJu0&Bsr0j@ z|KLI;K0XipO42g8??4hr;Z{EOjE==AhlXRCzi#BD)ki`RRVIzU!4PKU;{G#oiYS2S zQjm>D{Kj-E4-Fa1^tzj7|aoyiiv_62`&Mo(_k9laAR>ndy zUj9Y>RvVUYS(Ud=RAVkwLUXIP$6~vOYjinRaAv~AcDjr9iyrXu%d5A&XPoJNE;{`6 zeU*JQOS7`;=IcW?`W}}5qXpnl`f|{XW5J}sNQQ6rZF;jJ7ah%e(`;UU2qJLuizz7mGtpW4c}b&6e9=j3r+;lZS&1xOXQfP2+! zc?nZATxlTNrj{npYvBSL59@Y3qql(OzZYw_QZRNn^I*Lk5Ey+W=enrL@=TT@c%a9F z)zJq#HXB-0%;Z0aXbbs_?;S-(B4q3D|E{Wf*V!71h-Cb9R_Rac_r3u3nlfZJIV#8{ zJ@?(q@${6nx-p4`Z^ISUQ^VFZi^HbY4#Q2bvoQZ?xDi{zt#onz>CVA2;g_?{w<~Hw z2;?}h{X?nUyYg>11mi31Yd2n?V5E8?;H27nD-VRdlfRCg+seQTx4{Gcr9gm}$1FNF zQ3KtZFMPKums}S^;CgiI4CJP`^XY)*+XG2!3f?oK14F;BqP!2^h4|dv1>a^%P=lFz zit$2HKIN zgpDf$NM0k2XnAaz2$E#gsQJX{bA$Yea5J?=is6cgP<6767r#XxCM(~=2?0(C3?^;r zZ_3{aJF8X>(d)JS_`985wLUn;e%>b$(@o7$IDY!Rt1C&oM2`NRA~9K7ta$Q7PYwl# z-q0(MuBfqAWvA%B({zykYd!3HqQ*#!U+gAGl3!l{py2!ANU;bKvMxnhy;G8?$Qbi3 zs~!yFm@b(Ye@XC-hR^=%pt$t9Bb_mQFxrP{PP(^a_PIfZ@(1cH0Ke4JTEbNBok{gP;3RQ|js zD8%p!@tJ&VRzmZ^ndY+6hjHfuIH#PIQ2k`7n_?hCC8Dvaecup=vVi=ex4^%T71n+e z=9MrHhRUDc4*e~!s3eT|^_90v$Xg#=p22J6Zm^OyEZ7fyLY{q6cy}{tFB^1h#X5i1 z;x$sb|FvtcRCLsP;wWSo@RFkT=JK$u<2qvL4*di8+(Sr=4B|PffNhw*8z2aGK1>-C%bG`$&#vlBIZ|A zkiGMT!?x>$%->`6gWt49MelG!w`6X**x;1AiHy<}9lZQr0j#C8_wE&3ibi1SZkf`f zZt?DDP@5g9f`6!S$G|*UygI6=)T~|W&JwBi0{HU zc>xhFnjR99{})S4I$}2_1Bk4DGnobtNrbF{D!&Bods~Kq>0vf4qo5x zKVcz%5JywV;H_SLGE?iES@-FPb5DcgraWOmUvZj<_P6LyO&Ux{JyNiA9`SlRL*{HX ziZuWxQ-wuafo#;Zs;Vawb+6=Y%CL z7Z8JFn^Jru-5m~PH^DmTfS78>i$xDvRtTX`7frORQLTs^ z?T@P+nBTKD&rr4(VlPdHJ>%d?mC-ZbohDBqN<6Gl)B;oyT~yp`Y}>|rG^ZyX-^K=h zJe62|*`4H1c9*UI7I!(X*ye`p5F0ovEB!v*}!x{VJ0lw_dqEh;YF|Ri(&l^4Impqz&jeB?Q3Wt6WdDDEAnXZ%f1|0o@ z?&oZN5iN6{e-XLgA3p8~##t3#go{sf|7-62GFLp55BFh>c*@`h zh1{+^o>nWO6Fqcls!agHiQ<&Lx{RXun7Wi;VBigXbgB?{A#*(!$Co0B3`D1pk_3-C8bV74o2Ke82;?Pl-OJp)mR7 zAaLLc)<%7WlsDF!cXl-2i8?`U%CnQl(BW?{3n}x*7X;xO_cUb99-Au&0ew%a9|u9g zYW4HWjpot#U`;Cus_mDm_>bB@UmvyvX2YvAkHScWuHw8ZsZ-Q*lbhqH+3_mFc6ep1 zQt+;werE+tniXFZ+Z)c6pIDw4n)PJq^*(f!g~ATkL2Fq%QlcmEYqHzE75j)+vuf(v z8oWQAS1Hn~37!|gERux(T&ov#3Ny_-%V%oJtWn>69huyLTVgqIZpW0-_qlwAXfksi z9s8h=mTpaKkVNPkF32MQCU;&Ecm%-{QlIr!g4(}`5VLx1{d-3L!&CC}xv`@<`5Ecl zW^>`OcuSoiBD+FINE*&Y4Ok5PnZNr~G)O(yY?f$3Mw)yknA33ODY^4Ey@Z94=_o%g zSCuGsT;U?jr9*DWaD%D5_*p5PKqH0`zFhx9n$_3J>nqQPPOb#T+|q}JukoKYG*cG) z%#0MRv^AUJE*M4oFKyB-d-pULFs3 z?ByTM5CJt+8P3dL5(4Y(A9BAQ7+HskkT8#x|Id;r{$EQ%QM$*D%N!Z~`Mm=U!Pgt- zs^*O}zePdF?6*FT1>T=UuO$;EGP8bM;1~w?C8k8=_%CDBA>FZ6tauUew)KJz%}if- zccnhvy?#61o!Woi@F;PYuRL+RWtTy&%r@YY29`{Z?L(2-dJY@gecM4SqY=dOtO3!! z=MdICRhkI6It~Uc`K+>z$*f}v^P-z_9>gmo&t9P0piXB#_SwPbihFZu!Iyt!4^SpD ztp1n+=rZ_D{$lZKqPe9n_J}JY9&Y77M}%nIx=4^+?~4;m`>wBD|CU>7R_<21mttao*d%1y+B)??do z%L?0#*_C(^)%Ku?BXHmdETnf_I$sigc-H_F_!8&9xEl4Gg(SC%~Q z17B`9y31&*P|!~A=&M>M^*kxto%YQB-{TDZ=QB%3bZ2dd4{A|VAc#%nm4s#S2-?>UZt z8#3+q2Mc6=5bfw>$TF=(rq+QN9EwaL$+`CC%{7@eHVU)@V&cCNFf&~bRyzVV=B0&!)4`@7c}+$#{|0nYtwdyK>>m#pz_&~ezx4plS%XX#nlJo zd=5g$h5F}wVw!wP;x7UB1JafIdAUiHMX0RHGBwC8LOwacVCfd>rnyI1LL0r*y+7CZd?bzeA|spZaW57A>)(q} zqb(A=t|xCFysO-QFu)~H5C3Ci@IUWTXW^3Z!$In>bJA^1&?2s5g2fSI+;BAO9oEY( zE3Bbq$4R??Xxhkl<#U&lvvP&{Z*;EFdO~x^o@b;iR%m`yW^>p1Szy+H`y0tZ_p2Xf zUa-6FRO{Q;2gnZB>a%m%kX_RJMf<`9x7Oq($aSNIuFTDfPU8Xj zSeKmWv=F@U1Y@z-4w~Cp6xFpO3}2zpf!6<%SeDl6uy*_WrQA;S z7Z9d5ubEyW<>x>va!eM6J$vX40G&(c`Q6lT=LgkBS#%dOPeWuqmUjl2&ys~gY}0NG zi?eb$Ev(0gK@?pIesk}{50aq(KkWqn37fc8IH~`47SfPafdDQcciM3cO0U_c8V)~W zrs(yorgl`OS8nFt3})PzzkfWaiag}qon`O<9N730PxLDBKzuOaNxBUjoCIdl`?^Qr z-vG8BrWZ2}5H{+)?a16!+^EZ7ncW~(rK_HPDVZLcnX3&tKZ9Zl6BR1MM>HbOp~U%% z?mxXQ_*nwsE=_sh{?ZFX@Bk3&4eOz}b2+@E%eadm6{ws1TMls@m7WK`iJ7xHy%u$Y zT*cncHK=jU{;o+9;(_{=p7IH~=+=UrSIF~`aeYmOEi11&o3u+at-^XEbCLNTCo16U zJ+FLOr1Olm+iE($An(&$CmxGc4lhA)D>^|7Ca<*;U;!~|jE}m*Hj##0WZ(ZcQRj@V zH~*NMMgiA3XhBT(=Dj0Is^p*c!8IY8(XWArt$R1iIj7?)6uz`O+Vjm6ee~QnCqnmI zn*!SE#E!24Ves^NhOx4JXa1DE`O1DZhibboC?=5Tcx&TtoDgO}p*yh`VuraP?a=gh z>acwI;|t7gO2-H(pG*Y!dhQJB1^#ZKX7AG%jXz z-yG*?Os(8kBjmaEYm0b|1Z9sXK#SgZh!cU9?&0*nzOrbO=y(|_#|DNZnb-BYRlWHE zW&RX5_-?p7%c8C&TRT~qX1fW(I}ZR%QkJ3}>MLcWumRC?FZ(<@1Oc6S;H{3s;a#TP z&ls8OH`={X)v_wFygVPYL#O~Ps~KKLa;|%`45j5S?4u}km^QkH-MOhX56VP!6`om* ztx%){EAZ@Uj2d61=v@+tb+?O(C`YKqnPxDtTXXhEwR4^a{}g^SHg0E)dhz@Ts@fis zU{V|t6hLS0m#lYy?n(UAQTY~bMdjL*{0yD?(%(Y;Tm-&|7O1{}B}m({z33Ip(HoeLL!ABK4^CJKUZig{mxZwwn?Svr7_L;@3YR)8l~H6WE!6O)@U!~R>ZU1U3L%8w}tuFKNvJo zd7y1mecFVW!t`i&SO%QUgjd(hr>gLoZ#Zje(t?tNklG*)t{RWx&Y&1@?H5F)| zN(L0ualma?>hwoC=5r1!GZ}r(!)cMifyW*M%zy3Vi~55r9LEgo1Fb_xy=G7)t0qIP zWo2kgFZF-J$?^`$J)ES^gr{2l$c55#>5@C2da>d@mNXm$$zHG9rizeCT61*2Z~k2_ zJ%IYHBj61QRV~(*KA7bUK`AV^V#|i9c;AY4HXMmd%zE!p976VE`M7sVjiA>NdWfKd zX(j+T&mSDp&kB6$hDr6DwRz^lbCRgRg6)-HT)uY*cE<&}p+O;63(FKEvkROdr%OHH zo8|XXc7I#5+ieqr{}J78`-poso92>t12#)kYNF^$P2Zk1te%83!|wbxl5NSaCJL5s zXMn7jIHIs1OD5=AgY*6|3ijuuReYOFruw(8{7uv~3o7LUqGMd1z)|&4?L1B`pg40^DCH>&2jwA2UECMUvLe}j))V5 z1(7lZjHTAC+pkK~S&q{wNtDY-OMFF4i9KreQ>9^x`FPsNOV=JOlF(#a*pnKVNvl34 z4VB)hJ90E9ftt${py@7vcdBi_&|`PqDzFY*EqYJ$h_X5}pi&Ej~QhG zYqcX#`rQYF_M@R^Hkn$Y|2u*MA15^@D|al+v9gwbH)}>craGk0=6*dqJBd{6V(u^) zf5)GrBR_39J8m61SlmQzXZFz44efx+IU@Nlh#v;>i!-BU6BfMN^sBy-W?q`xV>l#( zxXoD3BelQa#dU!z4LDCC`?B}9ob{dgp?}L@*?X63i=MaBDTArX_F0bQ7s02)3No{G zHz%Ku6izhnO9Ja1_Gtd<&OP{o=3o{1)!ZOMp5J~AkoiRPp(`Xf!0mfdsoszYqp6i& zmy0AqF{nTdc#3@*f3E&6i|_? z9>;w0pbz_K4&C98U%#rq`x{vI72N_so$rLBF>aQXfx zzmH!J_zm%Ah`)c3nKp}K1`_U5yw&|(A>p(=C^jRXmX-{To-Wp6Y3OF*=B*z%F0glwC-5|BJbjcppHVV8Q zB*!eg8d&N#!7MWc_UbbJL4Bz@;0>zKK|H_iy1uKu1MI*=30ge{FFJx@xgg3e^j~KL zBrgyp_KV&1slWo6Ze8WFbiLj%D874d$)a)59G`!!he95MC3~}cG@O6mAo3s+lZ1O% z6R~{Em!xj+Y>!upAyUT0F#f)17fP4s^IQ4d4^fC&v*IYc5LXn-NoH^zD_z7R3&rPz-Nau3-bI!0ikY z@TUG0dwtlBo?%q0G%8;Z(d_Sd-lo;a_{gBESxxfLAu*1hV)4SEc<4F$(87%tcirux zp>l4*r_a9vs??wT(=GU<5W`kYM0@@O!zY0)I_ zPH{_hhujeWrPK;TNfB%leefqL6p;cbeYFCWK1^>;3sub3z%byzc~8iVXXj||d#OJx zB`xvxl@VHUME5`#zWv}A5d)rxdlynBmx~gR&v&dh>S=`G^$)(KRUv3163Z7hz4DGs z5PfE!=*Mb$lrGgg)#E1maKl}I(|e^9DJT9fg*Qzb_3NREZ>t`EK{Iks%%e{^YxzA+ zEGs=uVioEgp&TuVQBd%*r*C8m@5T_UsOwGEOX6%;)7BRNn!k=8()$f==}9&C6mYtN zN3Ckzf#i86+)L=j=ce7`U(q;rRSr4)N_V^Zh-B?BgPYZN(EbPus=_BZwb8RJ1i6A{ z9%}Ru9wb6#Z|{>O@Q|TaAo#J6T=>E1lf-`3I(@IX^s3tK7gZc%^4MAGN)A%zerz}= zg>re1*!>5;q=O*bYR!hOmuU&w+@W|Ap$exfmau$_uphj%zPia*DThM2%*uGm0V@LmIe#&}~4pe=(eoT}jAxvx(To8yv1WDN78( z?lmXA+IAQjJ0j2*EqGvH=_`FK3xu9e#<`1!H#K}AZj{~%z)s%d`7_Tg*hr_U)dGwF zQlD^-8zlV~Se9>x$V@G3FDhkGpfUs7-7H4e^S&e2XbQC>bWGxd)%>FFPL4H;PcuM! zrN7EJ;^1vGaN*q^NK{J`&pM|8UUfP_zsN=_`k1@<+oMtPkQ_6N?Yoz6wG9)f#4E$T zEaywwJiTVR%16sAr^d8NK94Vp{(7db;rFua!C65@Ln~5g`>BXC~&-+Wnet}DF9jo(8O-s~t3=BCshh>ybch|Q^5YMj-z;=5~$+|4| zwve0Y*-7z3bpgi<5lZh$c5#b$dm=bOhDxaqly&Y&?(5R#7^;)=tiU z5-M3i9skkU8V@N1U>l_)jb2gVx;HTAo_gPhab*kg^7;#y3zva% zwrR<4X-!KY`=7cPDAG{wEpH|DpG>r}R)Gp#p9vHSD_}?wVRoerSR^bFOI}+*K)UAip2ZImn=n4c>r^Hlua!4wri1NuVvjTr_Y}VO!X5O2p zOq5%O^uG4GL8o865>+Tl6Rdd#$$$@{B^SZBWv6tCfKAMdy$G^O0k-ked=FP&UMXgS z=w+>coaIqNM)2=S_bA0V z!V7kUeU`+jDtJCc5gbRFg)7&9{!hcy`4g!i!#$G!T#HjP46U2s9a+l zvS+0dH{(}-X1q5+%~pW5VrQ3i@K$RHe}GkrU~ZzCWA7J!rkYI62S!0~Q~4l7VeL_> z{AYiWXivK;fwO0?KLk9*#O8SRvOjI519G}=Mj$Jyo&2D?e%~EE{Wm9#6^%@^w+2+A z?7xlC+eGL;^~XwhnAH!FCd>VcOmq0Y)3t7OB11#tz3*>Q(>X-+uMJ13qUtf*>maa| z=3F`2AotU~&UcSTUol`$yC$>h%1m0+iFO=KKE9QQTzZ)=Q~5jWeKp0DV|%g3WYSUx z3Q6g8g&A)ftDxYf*Z9IpnM*=`j>;D``d3dUP#Yd~dN1!KbhfN;MnEBVHv+t-Y)sh+ zm}b~pTi{TY#g5T&aDX<1 zUttg9{=QFA??CEPs_6E@8@$&VMukxBgFS$DBCpn8Jg8jR+Hq_H#KknEk>lHZL# zcmASCB_uw+Hc;IJbWR1b2K#wbFd*98pl6@&7WXs)!J#$t>vhlAeRrn1-R39yaQ3R~Z_MAb8=FVWuLQ29MN}!w>yscN&w)S}GXMVT_%hkdrT^`Y< z)pCu;Z8?o^Pu6j<77XBwTy#07lxga91%ih966J0{kYcJaYQ@u|Rj7jZ(q_7!OMP4; z9sX(EM@J=|$S0e0t35rdhH*w<2`hT$G8ya_5@I6X)8c?^Y5U^Gg{#TrW2X1_hE&MB(FwzG~lG4TC5-b&d;eMpDXY08;ihOBuq443KJMxr~6un_5q6KG4N&; zmI%EXTNi9Q>vcZ5?22)4IqOJ-yhHRNX~I23Ub7N9zu5$gCL8MZk9J!%3Z=y)U>&gw zOo_r4XHRED^oDfsuI@Pl@BYaY-V2??M~awFK4GGM<*dU!Wexic7kN4xnk05~%sl&3%Zz&V5d;pX@;hLJ*gpP4 z2BGTyrI<9Z0Vg3a+@tH!|T$7iZ&TEG;fnMZ!#o#1;gc

z7m5&9PMK)i%Y@Y@X2{h*ATPOm+sTVQWw=z~XaFXk7YSE$Dx z2h7q6M(sob1Wu`)^2xBu(_!2}lgYc+=7e;;wGW?B_iZBs0l&o{ZV!fqV}u zaA4O}4Sbnl6$pvLi3V>py(uBOxFThKcJggALTS-9bS3>p(#q>sXF7kocL_HOPpnUu zvOyN1Wc6N9_O$7B*jcjQ5ctG?XE9=(Z`t10_@Tir)OGS`1N zFYfZzpU-7r`WJ(JmkBPCNJo4r)LJihhe2~rUl^;%O<&10Ex18eMUUnl!;N~p*9gU3 zF<)}s4zLCYPbT8_Q7dCxko%JcYWv-1%if!ttP3|frUQutUH6TI53=!jj%wCrVRESL zlJ8^5&qV)eFVCUZfkAh^6t`zMPspDS@qYOGU2E*}fPAn0JZrcoDzTXWHA_a`1-YnP z=s4{HtflRZiAlNS!5#h&obU#WI)9I$k-0E<@;l8bA8VsfrqYN)uHUa}P)gQZ zCvQ!+>vCac%^{i$#p6s(0w7U+uCh86ve8w;EzhopI?-KIHs9S##WSx2W|FRQ_$W>@ zBCOB#&6IyOb5+f$PG0prkje%aU!7ew?#BEbVH~X|CFmvhhSeXCX)p7nOKT!^$7BZA zfLmgXMz#g`wI4tx>}n&1e+uJhL~TZQ86!g}B}L|b8M8CLN#yIXM^HcJ<0Uo#v`0?% z`4o)CPrZ`DJ7KZYw+!?dfnQkU&u?}F-aQKrLLB^CzCkomTu)yEY6kXZmfpqK0nf7} zI~@OAbb#|!60T9a4(mS&rxf!prA4D~*CHv*w#5k$q+syYk)A9L%N7?Acs?@!ds6y} zMUy|wldod;79^s)#?h2WX(3dVf7Jorjo}XQT1=*VhS{n>bfvlbp^heRm6S3ZLL)-_ zZZe%+s^@aN#t6nf)s05N6xLF_qi(R1E(+&Ec5j0Ew9*6pGoE9G1F26egC&Ync1Ke2 zJ&m+z*C0|WL;2?A^CKwv`$aHwQ(6)+*;WJ{HFs~ZavJPaim+x#T z2#R$U3WmRW_h$~(iOzbN9(x~!056$5PWrln&^y_( zxyWfld&OVNaH5^zkEOrt!-=RqCPni0Ja|lup)+_){aTKivOu}miw9N2YJ_h;qAWE1 zor{{XX+wO0>SRcJj{w!!QB~+Hb11VBj%!2I>M(y!_}lIKFcbA7hh;eN%~f-udzG0z zug{vXiQbFa^0Fijd!^^I?Z~P=5jxC1UgYd|(w+=CsLGF?UG^XH=--rQKv&tH#tsvA zV*yf|`fn?vc&qi8htG&qEjHQ!@}&?d9&a&_!%UDJajWTTs(n z2HWhv51<{a~F{{ny*_eA)it`VpgMf)^w?b#AL)tb%lHT_NzkngE7;sw4Ypp zyetrJ^=6TqBYB4!Rp3h#AlYM6QUx#U&|#$hR4L8U%E5v#)j8&4XJ#=|ad8^QMn=9k zh#_#8_9(+P0V$kb^fte{ebFfKir};cr`IQ-1#u5Yee ziSj&Ftrev4my3oK_fZ#8LS;`( z?ur*BHEQ2eRI?SFS&R?0Pc$Bdo z1wB3!y2Gr1Lfql$8;QbSE;pX&-K_8hMmi{_XNYWG6tjRvLPv!^tM0k99Jm6n&p`=N zJEF_CsI;To6;dFin&bHW&QY)<1({=b9F?+D(wk6l^ntF>I_>tyDJ-?+rD|>U;~FycAREQHBF>gj$T+dX^2%Z z$)5J9_l}=6+#AB3RG6P+oKj5qy3Dio(^JmN`-Y6A zVxgOTVE#6%iQj#wgZIYQ{TkYJn@)$Vv7=uSZ$|*vtG~Q{P0zS@51e07K>DaoZrcHI ztPJA4=!Fg-=2?`Vx<6tOz26+@bwRfBhZ`{0$<$sNV$guEI3|xahLm-GW;@hvoj2dv zirBCW-XVw$7$5?~GX~P}5es1-po8;g(L~%i(?OT5Hfzr9c?%B%eK{ljR|Oh5>-Vlw z#->V}&6)+jM_mwdM^St`MMc~Lp*JgngL|KD8tF#lg|>8cf*#Az-^#F+I99hi#{MdZ zCphfMX_gjRl>L>*zFw(a|{Vdo!denQfUhRWiZ z8DdVLM}V{4non+zUp{$Z|8n5{!UqM3J0ui7Aa61P(;xmGIjCF4DoGTtIOC7WuTxKi z_fT%z$>KCxa{Qyz4H4#76~ouj-qtk2m+XBe^^L;3i+<+9(&rCE@A{H⋙l?HpjNu zxoG8|+Q;C!hiX0nd_*v>)lEIEWcb6^1#f<5i76fP_tLPCv&qkO3NEglJ+0iQPREI| zV5gvjdNuEzTL=NJ|Kwxgh}GfAH4Rh69{HhktDR;24ChJOV(;Yr7*5*>H9GDmJ1arQ zSOw_EiN4QC(ejdhIG%Su)o6n^8TMALccYm=D>!?QhL%QU@%_8=6+9^5>LrC8g^N5;|Bz6~&^Jp0LLd8qTXDqQoR9($ z6qzaG^Yu(3MaYfm2?Q zi^yH?k=S#&Ia<8TpQY*GiL58yFB@)=jSb*;neg%TUaK^|=dS&$ zRQ+qWGFTHEH^QFVN*KApDG)#JG_ic2tdI$iR7XCT!4dCXVUc5|vG#9s%WJKs8FqM7 z_2Q*Vh0(h)+~>ouNW7cS5>AJ|a(%w>v+$d0)xF10NR)A@ij_Fm;W{9(3e9mb@W4`XRO2gsn+gaXBXKb{Q&L-gGbbQ(KJYph+KH0k{$(yO!I z{PRq=ItuYZvqo?DQdaU1irs4Z`Aa+ari-YtLpNc728{%i|N3n3=nxum?#NwKW}e!w zWZbz~q``VAYd9i|v^--9J(lVkP+$!{eSbq0_+yoXtD zrQG?d9MGxlwR+(you&m;HS2zyUW?&P6!7-O8E&4)s)_P;je6_G+_2U)lJ#%}j%h-; zkLtj!9f{=q0-*Jqpm8*F|OQKBRnAI@se+qkz8hyc`D}6%O*TfMZkxw-icYnA(xJMxa zj=P{^vPY0O!J5T5y1-Y`g+YjiQMeL|Kdye43-t)FAb~4=aYAiO09B?fI&75H{Nk8Q zpE>L?)QmPv`uFWqH~Y{Ox~RYCJUT=7o`s{ms#a>S)>xHr$Z9v~wRwH((lmOlS?4FQ zGqbp~8I04pQiTcSKi$k(oqc%b{Ur$g!7(^>%Gx2|{Josj*!E6O9rZaHDoEAts|=ek z9S|orkECd@K8ae!4G>uzJcdc-LW-+_4O^{k2e%c^pIi8}G7K$lq3(xRfKz*ShCL2s z|I}wmOtS2rec&&RcV-!Mp4qe<^u+awOXzC~g~D6#r+CIk~D)kxYkNpESQQp}OwZ+*Wr;vo)lg*_ZUt^aW->2hpJdtO$e%$*vv#EG6-4 z;g)fEX(jh4nfjy9b(5*0OG)gO~5|poBlx z#}?V%(Q+B+OHv;X03}Fwod4r&w750_!@!!J`=QO|H66soOd*sOMAwk<^Zhp22 zx`{$T6LIJAYek?cG{aJv?cE)w7C4rab@jvbUxX{#s~(ZY5f|CRL@c9JN&}eWq3*0o zPUc8??$;IUPs)Toi1&1F&QpG$A-BRjzWUSem8($-(+(Z}qg*z(&~n1$zCX!hkQbfp zk9i@sS zUyjIVEo@$C`qMUnpSaPEHLf>c?f9S~Nl`$smtBq!RyRe*mTb?Z<7)yJR~eNPG3;B# zzvXmA+j`p<&V0++w4wxlnF~O;2 zc1@-gR_qNN(dvR(dDe7*XeNP9yXN^L6*h8)QAp#te<-N|^j5(2y@K@}BXG;?3oIqi zeH(UcV5V=P(Bl{Gty!Om+2dYpxw)?a&g2f$he0!}oy-WwuG?9O4dZ|BvG^7GPXaDmYX z(}oBHob%rN;qx?rg-2R0Ka7wvBh{$pd4+xfA*f_1dY;Qr@ZDXA@O(&vIZQrL5-ULs za$sS@>;R<(cDG7ZR9|c*b-S;}ln*1q3nu_(*>j+Z= zux7<~`xd~q)m^?b3-yxsLvO4P$nH~kx*N3nbL+D^Z8NdwJGjSn zEW8R;Ab-*CRx4}!khZ)u@=-W-r7{izS1CETHrOxOMusf^Uvb=*-?KR}d!nMFH#u9t zz{gViejDuJ9OBC2Ya7h~j{{U|RyfCW=7W{j#4ZtPDm8f%G%_yXYjwAnC1^Bt~J>+(qr~oNmQK;Uxw|95BMi{;s$|F3`PfJW)PjW}<;KW8^d#IgEA&_U^5GDOSyS^}MuL@ho$%7(A{#i}g<+}ub%yAR z=YF9B4u-e|4uMD97rhPf0R)oKO(IH^PMz?7+ai>in9V#PVXO$x5c0JQpHSxk;C#qi zpyyh`j-(^W?yF?+uy9iQHibj&qi|c3cMjRwEz z-1_jxJ$|CHNmjc4M#EoHPGXd+%5kEJEkoRyV7Dk(u>11ssPs36Of|9`;WLm5IrJ6@c^6CeAG^zZLc z4yFDN+#=$D+D4fdGB}b;61rtjE@SeH;R}ICSvk^ICZkjzXXqw4A`x#$Hk|bmJOj9p zxgN0;P1qGz(ShO!nK$&SCzMUdEQbze-#a-<&*N-F{tePpAMt+qj{J~MRmTs6os|6N zLJT?i6qokBMLqAi2-C;kgvq5w;Op|jQ~U%{F}@CwJKm3fm^Qf55}SIWA3HFCs5|6) zr&gpe9j@jF#mk>!NL8D*`&oOeL4Cg*acruwyYyh| zt69!Ow%9BZ5FYSn?LD?7cc;91Mf`DgeSPE}YfsueN-){XA$WCP!4!Na+bf$647}{K z0tqU>k1Q+gMd&7bFxU4yyZyGNZdg6sIwYpj?#ByD0r_=P=%VATq^0|e8wA@wCXUY> z)V%ZKR^fO?+{0tM%I?8{LyEzI((+~hJ?E1yU(JNfRc!Cu1mXLR9MVL5#ZOYwFFt#4 zt`yW$aU?u519Pk>C43aK?qT$7T=|qir5z2=9P%rA`Y5REcEmRETNIG9`n-P?Nj!PX zWD)Z9^C)i4zXW*>gzVi*0HcP(F5V2slB_~y!&U+h#N)OKcF=my53oWFpl{3xrKBDS z-YjM2FZ$NG?Vqz_-xi;Ibg%H3ehjSZ^Iqhn)R^eH#v(P=&FdC8+}ls5@xdL}hZdDk z-{P+`y^@4Cn|a<^rQ`LOUvNy`358oLIaK&ZT?z(gLlo8AzEvCN5id+xBSi$0S_Xd|Cl}5v;|RxxSkE-tk##xfhE* zblC0BXP>Lb`Yn=w&rUDbvtvD#&T@P=gnoSf8o0*cHT&=zP;?z4Cpf2;tEzzddNd{t zC*3~5jrB`ZQ-9vKxhkCxznNbxAYy&-^ZsQw9#v=&toFqTB(%@_EVr*RzN%+GniL9* zhy*TQ-VU&OJ`n7BTuLBLK=PC3ooh=EQBAs*(@R8kS0uxl&d`_36ov;Vt!J%lwU-I3 zk4MR9fAEQTFZrXtj{G`#O9uEj6*l!5!yeLk1yBjV zkNyv&l9jlT;bM0FO%thUvF?OL_*AsFabl9LCyq$jgMx80m?lIhmaB!1ey^wu0(+XU z`(%TONQ^WcLp?<)))dD>uC%>zx*EO9*JcF#QCq^T%^s8!AgFL;DZjI-tRNV3K0Oyz z&ce{#g{&2F%X@*+(MINkaCGFD)JwWT`tIAI#tL4fH%au5@F_jzX7(@fM%4GomafPQ zLxQ*{F+qNJ!54>(XOWV~qq{fn@~NB>s!j;H&Nj~fHHAz`LiiKk*BK1qtSM~{ab zNS340Jy=a8rrgnI7l-F&{>SfOD(BY;kqx0!S-FAG_5s&sFKGpLhT;9Oc-i!$*$yx1 z>CTYt&}#+APWG5A+IY~gV7yE1>(X)qJ>n%@b2?67Rbt1HMgEc+eCKm`)`AJpYmVy( zcz4!>@Kdp}lW$`%P>*aId|+lE=Jed#^7YMo2yEE>g2Oh~*z8@n?9PV8{Ejm|?b~g& z|5znIb3elBA6QYsqxS-lFs37o^5ZtQwub2LWv<{)b&`8;JS=(nYm`6c`(a=nO^%^r z%`^AiiP7l3Rgdn5EseB|lUf{v=O_)P|1)D{YnY6T`LkiSN1a4>rJvnj{CIuy;#4KO zXg$q(%S5pL)M&xwaG>J!2{>#|aP!sEvY^K`|2XmU2Pkq+JJ4k`9v3p)i@fVY#m77G zKc64*`(ReVljbD8;VqI6?F-zo8zxc1dpj5`BIC1}xf?x)DuCK=yGZr2;4^A3!o(DK zT{@5tIosNE*Tbz(!^kGwos{_zME{jv{OD&YS4GAqVm&kazSOQ_X58;=S@$vzQ>FKOOe#y~<~u|i zYPy&LdkJ9bclSJeG(ci5R#OeCZbV&@F5>A841J+kESBvQz9u|cB+u6;Q#Sg&GPIqY zkDzB&Td;|uSl#-A8D?G^1?Uo*9hb}-s;K@|Qv|o9T5~-OvTW^V$H%8DL#2h~;v^_| z;z*xz@9~ikc4Uyji%(OznwC9#w9+kvy?ZpHRm>MF^RrGA)Aq=pov}iKN@)a7LO&~J z-=u|jk1sfZ!gtqS%0#7>f{vvoR>(p-MRvIZ{WfYm&L6Kg^l@JTsl`CicOx7!w$ zyc3$WO=Kvn>7})eXAnQ94K(HFk*F>-uze+Vj_AaFF0FEXTr!+aI%=*X7h9mCb#C#-{+nTe+I*8&Uxm; zg}96&21oBzrTm??v>_ntfQhuvF(>_U@cJ~P7l-cP{<`LdznAvLI(&Uz5a`ciu+PsDKNKCPxVH0-dw| zR~^4TUzEpvLvW|$Cr#R5^X)D>^>_%-=<8O|J9GTv`K*Ak2z@u*qGQMD+_+$5qLL)w zNjAE`3NTFh=c54D5jiC9GlXHF7-+psxBlkqf5GMRPd>Tf;&UL7f5!t42U5vk&ICF7 zGYa#83)FVpm4hd7M2YmO!_bS*wfET0*kz_w*6X*#s;hVScH-^V zU-68etE>R`A+>Ub58r*;y~Dy4%CO9{P_H4sI&`rgi``^1f&FDc;XFS4Nf+d%vNB@Crw6dyZeMq!TDQ zbXaSEdvpMsc#HsUf1`{gV?TUvjPCjo9>T?^{2t#$UGzzluy+1F$DFYI>g(e3W$gR! zKQi*R8!tH@<2%iC9YssuTQ2`c-sT?w06+jqL_t)`t1mnY8D22WH0x|Q!@h8zK2uFO z70x){dF_?&y!Hx_pL{ZozU&=VuH{&>p7Zru6l2|c>6J9r!>_s+j&Q*>gIL!ePy9(j zlNL%`M#19+n)w%5Y~$^Bn+_A^*=KIL-YtVUWD5` zOKdR&UM)VyP{Y0bes1-Q^&_pFfoGlAgTo~H!^Aep+!Sojmf+FML)gJuc*$iCJl>sG zM#(T9*IjV>#eX^uUtbvWjH3@;V#SrQbXh{%oDm{gymEl>!Dx4uk;4OFhAzb|Bsr(e z6FDr}XSwJ9{y3GxFTX@zS+E4oqxarPcUbKrpi0A1C4t=47V)EZosGk#8OoX9wKo}z z(-|NE5;x{N_3-@-_oOAhNw|h)Qnv=AO!?sA5Bs}syzahRZ^9XU3}{2gMWQNcTx6;g za0{zWXXJo&2X6NF2k*yOVVFpWceFE)I{1R)kL){m>n(nE2zJ^Ne}xtsDsrsQ95KtK6lrxk+h-+{IdR zIl-fuhkz!5B|N!H0%&d~{K`Q=CA5o&;L6+-F|jI8qEF%~L+Nu3Oo{?o%yN-X=__l4 zVEG$h*3{fDi)@h9mmi#Y9l4mB^Arp`IWpJSaLic@5-dittj(OlCxYCiZN|+2VrM;*pSqKr;^k4PlEfx6;}hZ;tcha;Jdzz-{*ZHP%iRn}h^e*%Uvm%0VO) zM&5Ld+_BnJkGbYnlLC0~vAg5154+$GN6Uoe60f`94E%Ys@sQmH@3JSpM#H3t5bUNm zQ1s|;_l-DyzunPwe(j6rE*#u{|97s%T3X3FuP0R$w8RV!*qHvoR^M&uHpwRELDEkj^Pv}Zf3zv0k{zuj&do&Ns3*-pK4gSJhnREvf1CB zbs?5E=V_hA@zfr(ayh$+Z**L_Ty?+?>^EH;mLUff7A!dJ#*uUQwPV1u!4})yb<;H` zAGD8dD08{RKeiRxWfBi?Sn`H&4k72 zw%lz$tmksp?+(SZoO$MMLi({jO3jtm-RQSxT{Jx|+4i#n1Kn%MWftwT%!vo>Lkt#p z+IH^)u__;%8op!}?RdapJN)WU-hJ6&;O3Q23;=`NHE+qod5d4xfq@3|cI2?@FmG{A zBs*@HNbO!Bh(I$R;x^}rT*M}!Ht(LWP+cnpW zTLgx<1Gn4h#iyTy5q$XlAJ=GcLD1n>UZ}s@mtJ)Z{IzTX795bAe$phH=PS?u^Np8Y z2&N7lZn)^I`);`&O`hT78ED9SvI+Eze%H`zmRo(Db-a0Ibib*?5&SbMtkw9xKIT^i0_&eR_`i( z#1@2>2btqTeg4IIpMU&O_7+yWHx0k?;ybUqVyA-+-|83pi_Nk4B*VBkb{4~TG6rFu zQ=98cBu^PbI3BI6q@RaKsG7lP z!v;=lVO+e<1dKh3VE5`diNQYUY0z7I%m-J48+eEAbIj#@JwYIe^)K{Mqq$Tj4y`>7 zKkku{x9itAYGxi>+iWKJewTg4L0A# zb`T^Ag!k&tJ|2JVd8dW<2QW{Xgdgs%JnQ747oN4rkY8-Q&jEP$ignKeA~t?r=7M?5 zL-J(5eXzLL7wUpn4c>Ax!!lFR5|0MX`WCo) zN#n~3Y_!b|IGw?u_>nn$-HLgF2Br}=T=mVrz9yyCUw74v^(2gdJRfR8Tn|zoi`h%( zNP$;ku%`fx=Lz3i)O~so=JH`X?m5_KrFuH}y~BEH1&ZF`b)~g8=(F;g`~@DEJBMB| zXxBaQgl3x4{&Lbpio348%wARDRxV$4O&FOE*ieCF<#JfV%H?rZ-*w#;pT^~cVLkqm zp>xf@0OEp#JnL<;W%nL);${<^(Fk>pBdD2)%dg&g^(FQZUz`p-^_)(frcaXJaH}0| zyY`A_pLi5ez;oNxm+pUDxGjz&B{nyCu$j7%iu~*stS&?QKJR?OX7N^L{+5}Xa8rLR zzs-cyumbtxe*Qi<{AQgk!aBHf%v+pyz{SYRt=>0(j6m$Ddxb9^(g%S(#bw3n6g7VB z{Eeh^J{>w-dDf}8S%WeFCth(g&V8`I=pe=dN~}%0?3#W5KI)z&mtPr3vfX~|;X|bn<7)vQYBIHQ-P5nQ7!$^d&DJS}4>BXAYmnKGEef@zOY_^@i zPd;=%)&UZL$yDYEjMZD2?l|f2&%gau%)ORcW}0m6T#OtCPcR8<4jjy-9|*u<2Gh6y z`qyn&T`J6gY$Yn0tf1WPzM=Sq3IP0K|Ia5L!@p0ztZ6cXSv4Bh7Jd75_>L6`(pUV(s3OcT!_O5c#Z|Q{YXzW7OsN~kTD<9?fxBKe z=c8&^FPA{lp1fw8Y?XOdDw&8(Z8Eo+cmso?v`NxlfMvErkR+p~#WNPX=qQYGu!K_9 z2?9&VW5HKe@+3`5o||%w#n;K~mwG2bY&K6TTf#cM#eh(pR^N(T$KRm;bixG7JR2V_ zUUS6N@rc1@j=xye-E>Qu^v>3Q+a378ROl@mWvdZqU@&dK8a6x&_TATCWkg)qyjK5B z2+qmzEp7M1k3H(V%diqN@yZk5HE{>%wkt0&B3`N95Yd{%V3i>O*s9(|bEMVOE*2g^ zc(=dhvJ1pKa7b}!hLj#lJu9pIqxQv_*r59n(lq`zo-6JXWo9l|ei*<{FtA z1eSs`2di#(ooN<=Q9UjQTFTRxRBU1WbUxJ&rimlArtg!%ap`^ua@PM&$nX>#h(|E0+hHH&9MiF2~Pr`sNA1qxak;lEwNg zj|b0@TuKN_EWh%R=Uf6onb?G(|Le$bX*1-_cHIXNA>`D%08Knb;HCJNl$g18 zzK(@aaqWDC5Yh^dm~mfag8}YbhTqsP^{&U9bK%^CeHhInBk$DcZ}y9QJ9n)ZYq*!j z`t@+ricP>DzAi`i#DI7%!vgI3bU5>ZKhtCde?D(vgU zFF;*wZFGSx8)d5z>?ao7*>`MtRI5*A{FaY1^?uO-INIl%a5#zA&;mo*T8XjD%)xrW zib2bBYeN&MoiE;E@vlZ>c@US)h^>f=o2l?6h5&$Xyx<(kXjVG9^iAiJ5Olfc)}gx9 z5P&6DScRF(&w5D~2G_r>71zrEA%5q_AFlV^eiQEPBzUH7-B&eFz9^T`pyN8`pi zPW&@Dlaux*3zmR*_o%Sc8$gox>OC)C04VMjK*Bf zHR`(y;WuKmcX`^)RG7ZjOL8&_CB0?KWd)9Lp-9+5H8NkiW@Jde(z9Y{hJ-zvdIZ6f6}uve(u$)WIP=`1X5z$f_hnT0*T~I0l}})cW9^z| zv(+mL%yUcm~71rKh`p#WkVtz|v zk$H^`Hoo%Vr*}H=a6T;4M*s4Q&yU}CSFEuw2J*mBo0usGDIvJT7dpmP@mhz+S9UBG zE}UHulu9iY4rbhXu;zxFfXcDOUdy)-V)m-rou+6juCpPFR`A#gTV|`LOdpFWq6R6@ z8hxDxdd@wLRDd<~%X7xgg;&OLRMCn*v{+IqQBpj?fmZHSQZ&Q*>z<5Oh!iX*gJH7L z6!t!2fqg6VC?scS3rFMSW0g4zT)h<8HNJpS$4csmwiLcQ#V0jx4X}?l#`P?ZKX?!E zfyy^>dCXgHigwVhd)u+|A)Jw38*Mejx)+X9O!0xgD3ujk$FFNEFOtI%EGw_G5r4;` zk{B1!Z=>HT&Q0`Lc{O1|cpn?eIp&$)vH49oxK!huDI8`+w6o?T?oH<(e3oa0J`7{x#k;ynzxYG9keAd)=DbuS6O!>oHbT66-Z6rID*tU zHlwkQE>--CzAkjl7O1>6mvfE!?m`$ZA}!up>%4LdHQfK8wKH&=3D0F<$&7f*i6xzU z8U1ECu@5!yq={Tvaw+92jS|Z>GUbATcAsNT-1mitJP~vM?Kkdo%<*++ZF3%I)Gt}_ zS$Hv?MQ1kLW~R)Ix>f`yZX~O$uBOC-j~39ZUV^K<|Bn-awLtu8aq z%I2)yXU7LN?u}yeTFcbpG+O7JZ<)vroCK8tZvG~eG4biK{OrQ2StwLu0W77M@rFJ}^i?*>>9bnW=#2b5T`QtntReClbZRkKBbMH4tL z8ppb>`*dstqY%wPbD(D5+!{W=Fx%rt7%&@cJwzp)@X1GZja67}NjKbVTa+}J?FFYf zSZrv{0c!H@By84EGO{H}{Q3|qU{gmV$cuvb3n5{tNju@e6PNG5?FQZFf9`RIFSFX( zSVur#1`<4)Uw-*HRxX!*!9|NF98rnZ|G5}-8xHlqhz4|N@ z#p+kp%9^p5g8dC_=!NIpb=_4Q3zvW?$oP*xlH-S?-+?3UXplyN&0NYh1x9QWMgS6P z7ru2`Ae)&vD8k%5%su3Y{Ctc+5^LH{)2+N-|9ftBS8!m%GD&+LaUAO=65J?<`{SJ1 zuARTro>;56V(t769mc%-Ho@G_G2cQER1?rLNm>;J%8IeR{%YGJ9giGq9={mtiQj!U zRn=Gnpb4TtjEmC{$rA(cMggH0oB7LMbIdJVk}$byrm1l@?~GBl*u-mWd((h zQ#Jek#MaX33=RS(qNZbILhhL+6Tf$Ndd26Pa0QKG7}0Y-q|H3ETCWh!8%lakka&m5sBeFHoH>QGL=zu0Xo z0xI7{mh5xz06vCIE&FzSwAWf~-^;q*D8txM+?3MFk=N-*jJj=IIeP zT}=g^;Up3|VNcD2cicK~yPW|OAaZoLXK1*z52kKC=Wu!-7Z(%Z?GkH_5_!i}m*E5o zWXI_-*fR z>ZNY>))C3EL8$1GN87oPxg%haVXRw*@`}uAlWpb+sab})3RK4~4G!ZfVmJh<2ApI< zx@K$DNI4nO(^0G|HY2LP_G=33EM_7#S3?solP2+!1c;1lvIz#e z)qM26yJ{Lr195h3b$a+<)zU@!EQgzD00zUYmtDMezd?*;7Ym2gE*3843LzvHEG!xu zUAoS^(wgg6xcJVwF2#O14CNdht164_9c89qWrCd^|13j{$QfSbAoj9Q?wqqB3YFM1 z;yerXT4br^Uiio3bk%~Ouws=it-lUyT}gq&Hw-r(aoD}+vSWx47ca~oLe<8^3Lk?} zwn1lW5@EsJ1dSFs`AmJwfg6)?D9^|);8Oq1ciZRk(@!GT>n}V%^7<>+-(tHkI@J0- z(}35zZxmmCU~%7M5Yc9?jJ@Mb-MVSza(#>bZ2ZSK$DuOL+Px=3DGHi_hRm@ErSu<; z8O?>^sCn>?VI*nZjY!!C9xV&Pf0yCN=q#B8@XLAFD}S+H8kTX1K<| zEoEp&B%{H2*}owA4&2;kE1q>W8LZP8e2f6r-3zICeypAU)C2c%?R*4*BQAb9HCWNl4IougeLGwJNQHIh~2p(kFN?w_j48+h2 zGB(KoSQ(~@pj9fHLuHhNnV90wt}4#fbht(X|Nfm@WeuR@za(;lQalpiTZvCN)$Fs< zYCX)V-M3zG5m(0)fC2_1ACCm{$;ac4-h0Ojjz30v{PDTN0f(xY3x2A&U2eiJ22y_U zsk^0_;1$>2pz<+D0!!c_iz5mT5TX)YSaajeY%nXk5dPn~o4Dtmu)laizsD~cDq0EF zH+_2$$Ox6)gq=YRFo>0UTxP;L0EA~&PutOb(jdA`0GC0cVhY#h$XVY!4abi+vhq8i zTqMwuHJM|<1;wK@q#+^pe?I&G-9v~b`-?puK=tH9_X@xK>U||cSahis*aY)q_m2eF zS|O8X!pGz8y#5NA*4x}YX)noEJXI4=u0iPBwTo1^O7P?I?JzJYf59kFU#Wyim?+lH z2Lb*~LS>%KvFf^9>D69-4VJ1zjhcvJWj3Xedu1|-Z=2u*Z#fZBL4TMY(USx$XKuAo zASb?wm4F1JrUZIbcBGOpS2E`o8Q^aFXxZxY7BiW0Ws*=%)2Wjdvp|Rs-+dGN%6zC4 zkAzA*p7*4Oi2b}v*WaFfF<{*AMt$K4$Hta>wN%o(_pa)9FF`8u#LX-+uQ71Ylv7Ww zCI!UezeKMles$dGz{AeH?LjO`ug@L)Kt1ZN@R2HHE@*10N7SeZm*msICezU00o&{# zUtCK0?wfCrnTv%p6D}5Rvr1ygWP?Jfq5;49fKBU)CU*#kuFg_#IdiM^ZAH!*6i$NT zwnUiCr2*Rv;rL?Dl9Ffm6>j}?%2>m)IETfVQ}w}V`<^6|WJ{~VSMemiwh5Pfb0k$n zmH7bZ;E_cF6y9VN5!9i>)_WbG3q_&5=#)QTv4CWvTF(#5E0^;zA_ZMpf}{17@x%Au zlMG=7|HOJ~=u+urT_EH7j)s0EU^dY&tjLBZNsFJurzT)lW>cEVv91i%qEgHW9{NgC z>w2V`WNUq2eg2>NVHKf=UUK#?2lci8_8Zi9-$8vZJN1t?u@UBe6Y*p1{C$o-0Y9y2 z&OpH1Db~*4dEMo%I`@?C;;{y*FFr38a>8PQ_I8XUTgGM1*xPG)Y)Xlo%m<7$-tB?0 zeNS8}+!mUuvuJ9zN8n_lLvpg%WA!n;Q_VI7r20t+Kqvk%2d;rc(=%fpE!)- z@O}9#=UM8n=fSNR zf4%$mGmbiR!bcz4I_!4n(Y(ZniTSWDxXU3vhg>EnFe{(*dV{w<{|`s$Q6s_+*nUTc z^cW}jO?^iT7H%{>faN^>^?Oi4v*t#FnZLuJ?Yj*T3>uf-%dIr)>^-pT0wMR@cpa`- zaafPXj$XtE9;c&1a`?`haoj=DX}alm+`sm;8hEh5vFILr{6qq$ z7acc#t^^NmAO{%D2X4Q4(2l!nuo>6k{1y`S^(>A@!{+eb+I0LzHv0-@_`=yLy`s=m z6}x~le`M}dT1% z8KTphn$XBSLpZF*yJnt`(}q~qOEP@j(d*YsFZ$~_nPE~lhmYe+Jfo8bvGDQYLMnh4 ziE;WkE{bJ;QH<< z>Vi{G9=!`WCgT-yF#MV)jz3Z>?)-}``TSFlQT^8IuYLXHmwd|75BdD{LGj6mC-BAi z9dW9IOU<2cA+Lh>L8Tb-=&@JN$Zo!j(7ew*D$`hcC8QpL-U2IwgB=FHE8S5$)po zjZ{8@k7M$IlJMx{n%I=j8UqHManx^kLn!oEEPRi{kHz&P!>_zVv|KD4R;CAEm=M{G zV^q56Uhg9M?{A+?_=H6R7~A@rZmqFkbcgrvlaqh^GBLYHM3;$453-sPtc%4f!wO`D zs-S^!je(n;cI3gj(*Y&j{jlGC^YzzwwuM62byX3IaXA3+C-Rz8C+t8e0 z`kyljf3l(s{?9)h-?39CSAY7Vv$SC^jhRBui0jU}bY;JMwWiqe7r*Q{gMZY8_jPH< zE#}?^oJ zk-@B<^>H8%ZSV2R#|S7BTt0)ZmoB0yGIL;PnjZ3x5x}!!tP(OVUdG)eH(YW)PGNlP z@8cs6zRY9o{I5Rye3yfcq-Of*JIfab=JyjP=BHK(0N(Z2j|(&yYwDSm##%`_eEjrElqGlbo(i4Mum;_u)^$ zHo^Z6KymDOc3X8T+t6^4NrDA5%NuUK-6enez5W2c<%)}NI-}0zg}*ax>LcsBk$0IFU~hg;?n{KD(+TR`=DY1%+%9DD03d%)G4@pKKIdGozxX`0Slj%& z@4j82*W!2})r9X3`KsNm`8f z6J^rOeRGX<`Eljm%Pf!68GKv#>hsTTz50?tJMF;`b!pgsB^dk-bJj73D8o7DUvT3g zyP>(~k;gsJ_a1%{A=X((9(39bBLr(vP+SSt@xkFY>5>rq@|OIrV3DoJ3oN?CLQ5?5 z($jw@`29EC7kX`arA`pwU<(f#lm+!=m>mpaItXIbLQD* zFD({sof~Bp4ISw*?|i+MURLm=$jCYGRD#I?U~6w@l|Y+`8@{dO0s)V}2R$+f4ol2^Bi3T}ClWYkFO}s@XS7&R6 za8@S_YT~N!QFjf;gGi;3z_s(Y-}j)u-gEoaXZ-2KXZ}GGijTGPSKDagg?cZA(B0>p zM=lV?js392%hG$!HII^nYReXZ?zIvuzz>_ZoPnBA z5A&&;slN#y~-jD<1d$)mcY}ugG6a z82^X;cgH`6^wF_rIQgN8jaCTKLr{}er!iXR=JB%1i2Gb95S5AQXXC8kP|8f&9?A&(0gLRAm%qTnTf5^ilZX4$>!i2{C)p5l|_(uSF zlnG>+l#`xPnZ}J!yq5C!ddpwQ22`j3vx?d3AhGqdbz@lPUezy>@{0lhc1!Ug zEeByN?2Gy1Ju7-aMsJW;(BZ{!Qb~+)3b8CuJbDL^hK?e(Y%;lVvEyq-nfWAa!b2$f z>p|fQRx%>l@mqv09|F=-0A<1N$E!4NV-1y0pH)`7?t(K!a@~2S4cu-QccHKCQ7i$x z@DE#5$S;h-HRqp>8>CrhU~mzETOV%Hqw1H>PN*EdoyYI|3-x~GpHGvAJyO3Ca*Ihy!?;LHxjC?} zZ7F|GVfiVG4d>ALw>Nm)V&Qk)a??fUGcNWUY_&b=3>QYBFZi*Q;`-sU?@MPGkT7Mm0Tz_3&ZXEsQBI?la)H;5rGYjhi^U2t;IGvG1cRc>9&i6G(af+X! zJa574qXXvB!Aq=wOBD@etR%h01Kkza1jJY8A(_m8D+FZBtKAMe=7?R~y{l+q<z?I8&iE&uDyyp1DieN&5rsup1G6b&0z#VFM3Z^6P(BYM*pITs%-mIG& zWop@v*hgjVlEH=xR69*KYxf=>#bteQT?tNS*nW}5?gNR+;caPLI{3IKziY)Dyyzw) z2?{o*&>j29Y;h_oy5S<|Lwop_r`@qr;(s(<4Kjy=*bmv$Ixj>8)%rlloG z+_S2*Td7a~ZFZiuhg*m%i3^f{ef71*OEsiMdSys~FxxN39RDTYq)FdIIhjq%X=TwS zsX|`~6P^HJAV_Yd+LlK$a}ba8iD3DyY#o^o_jI>1HHoivXC_Hp@jyt}girL947t)~ zl3S&*8A>TZIY=WRH{}|ED~0BM zh}>k8eE*%daf(Ekb^YlKr7FF3kk z_~9bAMV489-h~%IKHRWaI99TZttkd;QeQN_U@Y<8uI_eZ~- zFrbMfaViSG4fsKBHD!e$A*TnD!4DgdWrd)Bd1&UIgaiq<952nPefur3;woz1c)?kc z69IqW*vm2QxXS+!STlcazW7|ZW5tSDX7lUmdKFHK;FaARb=Pfzsi)N7l&CJqqK4h4 zj0-f-HOlwjd_&T4>f7AYD~X$X0VaMcx9ONrCBD{O{D}*igkfKaPxQIF)o2k=6o26* zmioiR*B*D#)eZmF*<=g3KlRXkc#MFW5Gr%V?I(RV*@}Y;%$PUAg>U%DaMszwvaitJ zagCci4V&$ky|Ux}VK>4n4{h<5))vj`D`n9le|14f&RaIyoVPmeG)tLbbDm0oQRJc= zDus6Sv6A7ZQHIkQDh;Ai&dW`W+sDp9z&aZ4pEMq1Q!l)tl=Iu@L@^1=6Vk%_3;sna zyb@y>AUllR0K>mVUwF;bZB{tpk2)vM(k*8Wx#6VaTI^$?# z$O<2L*v~lg%+4fh>@SiTyvv>d6NbBgyT)DwJe*@?W}KxPlgC+)>HW_FFn#^6e=#(o zud-f$=dLI)Xp5CBtdk}U@DJTU^?>^7i_e9Hkn{x_pGbL(=rC#0&u5wmB6&J@os~5S z4CS~u2ahg*3Tp-79!nBLnlR$*@3DWm>X1L3jRii0VFL5@S`?Qt<2`qddFEqKRHIX; zX*tJS?>w$^*O^$WJP^D{g*_fJkGy~@h+ITa3?kWd%4NZK} zJo4O2Prd$5yL^#g5>{TX--RO|TXF3Tp>gDK!{_eC+wXGv&3A{3>tdBG29Fay=A26p zIprLD#^gC5Vf^uM`VAwu-{;riWOF2ElGx8MV;A<$7on3Yj&krNpoa^Je_n(0&40P$ zq1F2pilw7q&3$RLCcZ} z8!8WtdYZEZt07UI?1l5?mi5=>957idLoAx~y)n;0&@>0~(`;q0$$2U=toMa`U|>%z zzb6V=nHS+BGZhOOgR!C~b35l{wpwOlVD=m503$+S2x`UPXzeyS zWs(~Vy<7-Ii7KlpWbYdx#Vh+;;_)hm3VNdnDbUU2Mn=3*Z@lBKG!l8nQ3nn6&qxZ6 zc!^`dDq+>(6A#>bz+!XYA*L`!;wjEP_OO3_{S`HMzuoJoKgbFJf9c_*vrEr!xX%2l zvrY-jR5k9IE%yXED*W@|N8Wffl(gcsGM;7~tcFv0K6>Asm!EOcJFmSGjCSn23dnyx z{>TeYKOyghmh7WgQYHetG>e!?o)_I88_bR&gRtg}UTcX?l0qF@s^_*k5k|e~isY-NLAi6t8za?yF?seqxu+R&? z#uFCyF<~~faygn zRRjO>xSptRkCO1NQ~74R0@7n}!^neo4wK880|%?n}Zb!5@!)=Z)8;T&VYwlJC0lDm>Q)<-#T< z1h4Q)>7%L`JL}-(2nny%NM8cL?YzZh@|?GU(qpiDa@2h%?7zEcX{AweT5Tk3s#NKl zFrX_1fDTkX)*!a)c=W|~g z2zHi1mPz?180njZZTJ55fNgdH=KBtll1ITosh|+>=ih}R{`&FQvC~fL7C*<$T^4|q zz^-}JFB!m*ub=?~v~_w$cG zo@%=3=9qK79&^vlQrj^Fn{u%F+2@?=_)D(+V9Xd?hdE)~*qOU`@3qvjZmya+X6J*d zO}6z0Z-ajzeE;2d-gxQRk3RTd`YtohGRti9E!sP|P1h#CZu)AY2$OJ4tgE)^o$9Qd zrNN;S788Qm@a2J);PmcT}61JBr>0{D6)~Xdw zRkaSBFi%do=$i*h`qB_0_iX@In(F#1ed5(hZVeb!uf3TwpTtW|33;}OhE2-`U)!w2 zPri~Nz@t>95xF!F1f$E~9rwa@!dlH9A3bLubI1#Se{%n0PsCYNF+jjI`B-Y|rVG#E zstw&~8Yd-9?CN8NsNY<1Xh%OSW#TulfFI(XN;hh1@@9)ED{xuJ>O7gmtdM>cg zt1moD>;1!T=(pAOeO6vw;Ja@bdeVV=>KVY&=XFko7)W98oJf2*(ITd}kk&2HUETmz z_sfX@_{?-dgCSG+6|QJkRh9%HS8X&OG*;-4r`s9a`*upIMyo9YMXry#2FJt!6n#7 z?Yh+w=Uk4J$Xpj}HoiyCJofOLFFcPWiht&BD=yoAcq<9iE~3ftWgyL&gi87(p9e<_ z*Py_(R=>>>0Ggn(?u#$KG9M!V69xhocWl1PKEOQr;Jue-Yv+>B!r&e{WYF&~zJ4}5?l*4mz46lXa4rM0VwTx^&M~k16%*Sw`}`VXjngrZ z@wd}ayKQ~=S(jq0l`RBrm?v(mUwpp*wma1=H5!o7rOj6rWWbPJIB!u3WO(QfgU-D9 z?oQLWYZ8gUm5-XYA&U8=Qy_qqu#}T2h)_a8Sf@{5b=OfjXq#v(j#qz+x9 zcfa;KXI~V`L|VPoKxRBVoK+WP5RmNiHwXf(-bS?ggeqyu%T>_3*=%Jkyv!eaL0JKE z7M_?OrZyj6E1c25gS7)2XAdV<9?MR+Dl2-Crz4H}$lRoNp-n?#da{ENm1rDmVjE4yV9bgF+vn1!vSuIn6X(XK@$QP+@x! zA&y4lh;uiy&o#Hc{xXD+UE-xE0Ut*Saq$u50vQc`69!xqgR8|tp6GbQ3k>F|rkt|Z zlFQ)K29!mQJVKc`ldYTlqcB^K9^HEM0LUq96hi@#{2t^zqZO|x0!k4WB5nFw&H?Y; z#ThP{A}4$sa{hVeE{iR@JpLsilp$%zU??Np_K~^@Pz1-J8#6_pFN638;j1R z`gj!%gq7CD$w7CaRz=(?rogrRkKaG?gVFDN{NaaRef2rk>z#Sl*%s)%M4uH`>*#N% ziitP^@#BmB=(kEsD#D7>yB#}r>N;z8-0U{(w9~rYaf@@NN#l_=7UZHnks&h1Ork@9 z%y^Wuv*=)nM72>OGl0U_{Q2%mnk0JUE?!GH6zbRuYD~mBym1YpoR|YnO&;>+^MAQ% zUp$Eb2wJgRG5+Dgv4xlD(_^lAKmOo@ciw#Mm1qC1KgIz2>WeSPh;NyA^o9cArjlI` zJ|fR&+KwG}IpnD0f9Vzn#fC4e=lt%l-(To|#6~--Gk(hc&Gv&mHtd)inWvh1nqU3# zRD1|N_tXyDXkf#*plIxuHx|_cq5A8{X*OY`U{8y5QM{q{>wJ&Bdeu?U*NY8HMgfB~(Ahl)tg}Y{ zO_p4K<&eUL^Ge?NyR06JH4X#}%z;pNY*9hEu*%k2^fs%7}MAO|qM&Om_ zp5bEzp0R7^Z@1&}YxH$%=TG|nrDy(r`ROM<|Bok-2xD>Wd{XsZc6qF_|M&xU(}XV! zd-h*r!!5R(fAJ;oo#gcwp2r1c@4fvdz}TkaZ*o2>+^NK|X8&G#_USz~T6K#(_g{GN zCH+|c{Nu6XM!$)%egMnZW31u6=(5W#SRQN7YIbqpj4BoJhr23@oH1{gSYb8JTR<^y z_ZqnF>iss>ynX7S2cWkuXsYHEWG#Zl%3Do%*C-j&Wa?21kxaV%i+bIx5D3d%}Pw`0+2% z6OVnaZwO&cWEPJ#O^&R5Y*{J^*(u?Lp<|t1upcbc@#hh32xcn{Nz`mbAOoYc;!cdR zf{=S98Z?DRN^BC7yW0caI#MPNeUsi*P?@;7HF2+NDo+3y9YSRSbqvxghh}$1u^7b_ z)qkweVjtU^zbatMQb_^;z`_ed$2$Endk!fsr<5Ycu!;5#$6a*IN`B#kgdaaJ zUU$x^uRia7fs~^@Kkpm)t>w7F`Vk%{de4a%ajq8$F1td{9zMKYB(88R*-S7eP2E3M+o(h{ZDgE4>jhPY>tOg8 zp0eZdNw4Mmxm4C9-0dd*w)8O#0zX+w*Y?%5vg>%<1YB?O*LU2oy3ya8t}GK*=~k~n zWq%A}ZX71##K+^rnH$h^Yi-StT!gN<4)cHhT2d0!JGq-#yJoi-j=c`~-Hy7pg>a&0 zJk?tjPc%mAgf zfp02y04HzbsWXeI-ID6F00a*CO&$;;!$fXBpAbmGKcIVHDonxyLkRRAn3w2-tBFif z?*q2%+A{qbi)K_biB+H^G<};=y=gk{00L(bIQeBA*9VFwWygnMd48a@u9ZuMPf(4u zQ>2~7Ufct;RuMMV_k0ouRn#L6E~_?T*GrCZwEm^{2&Vsji4=z>b(+igcQ_w_eG$K( z3(~;|SyGwvw(CLzoKlznGB$K!;79ZiTK;b>fVpc24WulOn%wvY z@#)M)H@?>hf1dk-*{nU6zp3WAF#31W8BGT*ayOoP-Ld%Ac~t>$#<3^r{|281g3R~Z z`ETR(9gK{zJP2!{Mm(po=?sT@dM}q%fV5x~WR-WLY%NschEAmALGOuu??thOIY0*y-h3sNX z^D?Jm?FmLEls+&g24dVAM+Gi_F>BAWTg5P%n6jRS9>ilC5xkM$V`P@!WX^?Xdh(_c zavoz=(mKZ==>1ByZ=??-XKGkeAd9m|%JZ5MmNk++H|s@F{Bqs%5m_B%wbk$WbrLCE zufu-+;6fZY=&dhWo^W_3^dXbM@Vx7Y$g9=sfN@?*u}kyF z81NceKDyp(FNyE`jMo0ZQs}Pqk0-g0*+%qxJ9D)mo?tAY!&AAC{Q*}n5>NJcoj-)SVpk`v-{-;kS>Tw0l4j(uCG6V>_*12u zaUb@P{fxW((&6hSH)}c$4fFc$chgYV|JPvU;d}QR1Qht&JyMo&q&c2fszligTX@er zcIT#diJnczbeh{9dAP1~^Ru?|RA@h@?AFz<_!zyPhoc_#LWIq$zTabkH zueDAFVQn0P9qOC+iN#Aj;-Qd3G*+(`Pw|onFH0Rg;|Ws7(cBKHn$FW>wiwOzO%>3u zS6cZZO_wv&SsXWb{IK_<^nCsA6XZ-(0}VH;Dcu)BJ`&5yvJ9h}o$tX|;~a2w9DkAl zyB?mA7G{)aQtsaM`DHOPo(n8%;B9Pw_;ACUN(24Jzl zYst7T@N8T6e!ER^TE9 zrL%Uiv9HzNP=ALM@R49juIp#uP_q4@=OJE%dP&Ojy_-OpZggMfNS91b({rWOiQ9X> zX0M44OHvx;o9j225~x$a<-+lNHq}^^jx!>|p&tLdODd$UZ3UjF1TDXip3a=KsCxb0 z^uNe8i@b;+{p=Z_N*I$%oFtseaw1q`u2w`pY~AAPzvesH4Cr*c+pR2j^j9)+J^)MI zG0SS*7$L?7ux;h++2CQg6AoV7YIOXZW+sn1xONY#!;LwaIU3@Ph(>fC=v@GFb7;EX zxzf^@n8RZ>G-kWM1UFyJ7@LG3Nh143!%xEcm#4+yk*?={Wm4sXS3n3a!(s zaUY}5LnRDux6?oWG26K~6(YfUqy&5IF|Z-alOsVY0F+7CbQ;PWtY7zqWyK}Erz_W)2Ah=$ypfcEvbc|U-MoZJL zVv@7ScOJ9NYG?hWcLjcUuF<|;>v8?WQ|HZ^MQbBTu7IoGRsUV3bM80wpuQy%ZJB~TWj3BQ+WsrHM>loG{vmqp1E>F_3y9`5VrXQ9pDFhA~ zWrSoJG0bjhcI!rs&_&pFsx|mqhS1$oqHhqijmtWhZ=ytZhUaQi7Xy;BOsP188n;m9 zaVVMAQZ39vd7UVdN%Gc8Y1~;1#k9x6r+ywrDaTf3p?fQ4*L#tJXcnys8jrhRp0j+-=?j@zR{veC`dgS#6XBDP=^>f z#H>02iX;}tSuo=n$`gfgoJ6q96J6`^vC#eNyU2R=v>meV){+HmfL0XSmhrHJIy~3& zL1I%*{;NDc=I3(>GUYD)XM60EQFBf68U;YLW!9 ziVW;nhl+C5sD5%Zm8Gs9&Rc15_!sF>OA9F~Ett+nFthK4T@z^>!9^T@tz-9h=?mYs z`3h_K&85um*pD7Ax17--0^>**u}Ymz9Ngi11C1hV2+RG5iJpJpU!@{^I<~l3AyP+F zGlz6%HOQ!7Jctjk(&2O+e~}jLRrr(%c+vEKwmQ>sz0NzK>04WLJ9B_=IsvArCmo|E z7oa$_z20HnY5RsI)gmVi&JbvF{hn)I_81*VJkIl zg9Di}MQv%dOL+`FbUC0J?ioQmlKIDKbNMMSedTOwa{Y%n(?2~=b(Q6{bKr=XlLkw3 zPj>jKe%QFzTYn5g>9;f0_1bqb*|#35k>_^33+@H+u}8BSd}?rQOXiV(B{=^P*SH@J zIMwYj+RkHpoHf}Vit*%j_|UzT2(;M_BSJyZQ!t*?Jzj>4TzSn35NyeWF!}+(6CbC442#9UPS4WOAHoRZ;vFKSsHJQA z*ty|xt=z>swE1fk2BOjXmuFtn#$|#5^o2$qeaL>6v%2#Lo+Fu)HHt_8!H1@lmRfB1 z$e5{cf^_$36>O3r2sAS!LftoKYjuf;Tvp3}P}6e{XPN!$i}jP^VAI(4$T}wn4!cEgV>PDgruCqSqU7?Ny4f#Btre{46Hrrav$5~#w6Z1bX3YjKF7Ei&H7t8ZLv3NWq_gOGKzRHhYbzMv0kC-Oy)A#Qi z`KAprFOK%Hdre=|$6|5f92$K*;mco~pqrVUGRO0ooQ9e!>Y1&ux^wI*-0q%aLn8Cx zaeF=b{;~f&*}+pxLO|1Onpnb-zV%X;Dl&qvlhZmv5mJBW_FH0`}VCo-*Cv~`X3lIqxNJ}1YiE$0Uzjh zwmsSG-&Sh0o&e~JKQ+TNz7o?h=#1juM%{|;T|2)TZM*(l_wziHOiI=kz&w>Bh$HYz z^u66fWZJ9spI_Py-gD8P1Mk*|)i4B=Byh$RpC4TI z=^OgcIxUjGBXT5cvYUyB4}1Q;KzanW{-ExYjjCR z8n5_hY=0n;S*k?Yo6Y;)$4ca#H0wL zx;m4F$wLSXoon=&n=8w+dd_pEZFp6xBJtt-zWu6f^!sSBgZ0=Ko(+@t;-NXXQrGta zaAwN@hbe%w=e^HM9ca3tBFKZ!=}m!thtEZ*XtAiT6>PHJ`4-oJjq|(QO<*J0VX_(F z8(&h*OokR`<~I%fll?88;UlAqu(GX?5+%iu!Cc&LMiAV_5dTVjJYci*_=%9XxHe&k z9OXQgQ16(5Idd248Q}?;hM?){IezBnuLUh`eCmFKMq0e)HvL23excyKC3!xtt2gtp zo-eWT+1&1#RpY^J=}zLIImQd?oQ#y{xW>^SCJbwyO9bQCa9yJsN0bPDfdlBcZpZ`v zFk-py^99h$&{)>H?wiqgZ1x!z4tibG1a+`keNBpI5?@s~~9_)Z4yjw{7t9 z8tfQ3*YV-o^v6v4K+o?s`qjuMar&g2zhneD)+c|1mr~^>Zx>FUZeK0prS*%GKkg*MaZ?|hSh7@#9f;P=q%hSG_0u+ zaU;ox>Nu&n8--HRytUTSb>z*mB~OiiTj~)>ypiK3@PXvoi;mpuVZ=VUUgr%Vt{CU6 z*t0p`Yrrx>`a29q}VXEA_gvk$M*4Fn3aj2?ZQ5=q2IM&yM zjT{ym*LK#gD73xi%%#v8%oL|F3>ZF(!KUNNZfubFMl{#4um%BI#U7jV z?9kBS{kqabuVpr9CXw0~t=^cLsgA0NsAxO}2tL6mC01F*nI^o}Vdy?)5t{}TWX^9d zVcxgeq}UEeHL}5>o&DbWu?zp##cN6)8tZX{vr?*VU7K7hOpO@@;r|z2%S>?7UI_ft zAe&|}=TRcis<>rS`3Pv%EK|^yjIgUI5kau+{&}kX5%H9V`aS7o$l|uVn-olF)eKsuX=R zBv0A*YSxDqtZDSzI?XmwVLPfr!%xq_8kxPRY2Kqm#y_qME@+rQ9yb@Q^#6$~u|sq@ zv)J1breVx~(mexeXiJkG`u6>&i8=bF`+Qu*9kQu(*y- zY-V>G?=4aMl6haLmMHksmfA|Ex@`f?dlInJCz)4hdN>kNkIgYD5&md{sbREd$Ls*@ zEe8;ggD#67A>m?|*82VO8Eu#(E3qN%d9omFa17Ws_=8jAjMfp(4sy`KDuWTycZC~X z+q9a!FzeLgq-WN;4Q`FSa}C3sEq*hn&X_{Fd?=r?pyD!vR;8o29V4~HZ^$p9;IiK^ z^^DGIEMqqSmBK>(e#0}KrRjeDpGi_RjZnQ(t-a2>Xd1vRGrLajQ`PuQcB4B~^=4I4 zz8zvrGq~B}+0Ms?+%}Zsz`Z}K*Zdl?AJ>HSL+8Pb*5{SYP4uNAy~?hrAo`GYST0R< zYH+i*OX)?+j0hss%HxS@^wbxY$%!yBt*Wy}#t5wLeU5gwxtxX!bwFIi?A7sIZTi$q zHRN44tJxlOzZ@OqojB4Boh*p;F4Kb*9_BCPMR&K0HFoZk=2YWh$K%!V;jQQ8)Pul2|KD`EzX0SOtfA5}B+Wp#f1jZ%#!Kg(XJKDtcyow16vQgRh%3 zk@gZAFn*xLn5HA$8U@2O{V(EFQ_2_79_9b@78GG{h)w8AJ1i0OOOnt_5;SMzHyhWwUvr zkRMH!Rl5I)aJzF;$cM8M#dwBwwGOS~lQ6XWtr%ccG6|DBJf2VC%m<7lcZQtKOm`sf&88D+_H~A0Pv(x87Xixtl zgIYGPEqz}hwB!qQ;v{)wc7S%Fl#bscpem9ynt2eRoS@!I(UX93eFt5YT2f-%Q0s}n z^xETm_XuvdX(UHIVXD#y(6oQBy@MC?6RGUCnUwUJ2~S;JcbI)cAO5BJ3i%ztVC1%oA_K6A~bE@2!#=0$IKMVLK~9yo5W6N14?yYX?DA zKq6ktDoL*@SxzmB?J;MbTA5*OGUl>UE5*rIY8Z^9uhYNheP?h>t+T#+$^EDhA!?OK zX;WolRZCknHM0y7v2T&gC8z`ksr-)RGvvSh-Im_zDorCWD?6a{ycV^RUe(jxnI5Vj z1vSKq`bYwc)0X}&utZFlp=n`_|1K@P?_hNm;9xP1l80q=wEh#x&*R}6?iO>=S%6^V z4$2#C0;i_zJueavQRiIkYPt=YW%2TvurAGVwN&FjF7FebeN6F5yU!lsif3j2MM)TwS=b|+^`P#8B4*yyI)oV3xr7EKlwf)ZlB3lOcxLy3h(zVu1mf}x zkU6Agd8r%57ak8fGEAxm7c%>ZNROJel-sbPkjr^PfO_t(3-}=5s0c0aJoF-c=QOnU z>g;Fv4%>gJ`O8$(W<&*eW>JCQ!c3)9v4L2R$pgYVwyCHEF&&Zo;B_T}+I_=a&EP*m zxG#sspaRX`yS60ZuO4@H!za2h_F^L9<}X^nPscTIoDC(69*gPEovD$d?syP!8A-;b z#NuLX^2l&-sIi`P>=xGIX1=|H9jofZ;!KT7kUf(R+N^J#<$#lRCdxnC$7ia6OXoi~ z59uRKtdaNr%Xik|t8pDBeeKld-2N>1k9qX5$W#T}c`JkvX)NIQuUFD`v6alu;h|-p z+U=wi6N$A7Vr+{jUi7NRXEcBeT~sF$V-|~CsWJjOi3JX!?z(@ZdG=Y*N>5#O`R*mq zg*1IloHIt{5fVKLq=!dInE|PyI;!^rle;lkCulDJ+g=3EgLbZ3Dl^$D=iIPLLGATl zsluvGFGbTyBxKVfj7VU5>ZE7sVfp0&iR&>X>C+c!N^Cp(V9c{FLiytjni8T)X?5_+ zQ0NS3ocLsQGgoh4UZSQ?PELm^JJO!COw$sRrm64)jZBjjKohUEfnA_Rt|+cuR;S8T zCG`{@a`j~E7+8)+sgreWzEqB02vUGdJG&3|ja-Gl zqY@|S@xJOGLMN*`x!&h0Zqv%z1mL@8zC#7yx$8h9WAP}mddYJJwAa!Gvow6FEz?wy zk&JdGSO3QcP1PiJlv=vEW4^Zi(k@nI(p4_Ab7$*1Lc?4>$Z)cRiPbXt=p61Oz!ltZ zVo3FM6}U9m@#Csv0LH!He3ABdE>6cv8?wIC46{r-v29r zPl=LMlmfq)iH{wfVF6@!wzP!ECfS|U2m2cb3gjc8K|4>??AlK5yZ!d-gxde)=)0f& z&k82bTw=apqq)evGvRdKZ1kK=BfQciCXK%+Z4V|j_8%3m`+UNn)9GWL1l}*wYHav9 za6DNWt<>^I1BVo!`O2(rWZaqb;1jtAnCY+Hvp)s-wve-m5z)gQ!D}BYafJ7=?GN?^N>Is{lq0 z%3b9&fzz$5!#}S{3Et;#`bAOpeeOGZx@H^C4w*#YX5)(!5dO&H^`9E?&8cFhj>wpw z#q7@fD2~-%rEjPIjaItZFOMAQj?09bW72-X&i8uCuJ6xd;`_A%ypI;(7N^NMw7B;3 zdf?8)@2}dDFB^JNEmLXSe|4}H6XCrykha#$67qc+P+89U50PCMYn!Lyt$$8`_W5V! zU~M-1kzqN|?j$4C8NzDXXNeqb)s_iqpUXD9k#NPDM0u;|MS!J=~MsnJH~PO1!{(co0$`_M$97v zAS>ikAl-BVh4pQpu{$zhiiw2I^0QPw>!7@2k2J4@8(jODq>*ff`9a)bV>IK!??nbh zJ4jaGfJ9ZRwoapd&(hI?z9X=rr)(-H5R`(|BqbxsJXc636*tXlo3un4NLUmkB{5zq z?}PmJkR3HbMUq)j%i7r}lGReq?;n50p7@*yYodX(G6!U;TD46YwI`O&zv#KjJnwUR zrCH87AD@CU7FZ!u!GJMwEdCWLz+n6Rb@HgOR=SSyq*MmVi=yB21T7jlpkWbERD_X4 zzWdn&Wc`%_B~n?Di(1*p(lwIK_e*jZHb#+@mTG29vSV?u>M*1*lFEgo8UaI+6)jq( z%{miHyNh~0zf_H^nk`bnfFYHv#;F)dXGp>)Y}zF92IlEqWrdO=A0zESvOmi}5~(c7 zq^xb_Xd208Cgiyd8zM=|O5Tdii{_DRL_(7lEL$hdI+Mx=OM73ztDdu{ph1vImJ?Ks zq_RBWTos*6>MWCU=pbPbQB{P5=>9JbpOxFona;W0W*6}`8zEOMJ-kC}Ik)8uT2ZhM zj~JxPT4&KWv#Fic7~%y$81ghn05>3J@{8kovGaW6*ml$n=hd@%?F++^QBN6oN;OaJ zX3}^vxsH>#L}U)@ahO(u;+fVC);DU<~uuQa=(Af1B`ibmUYxv zTxJIv?EM_XRnb42o&$3;co~fXpjm~+L_rbiE^)UXyn<1Kmu}KO{hsz<;d&F~*}!p@ zHme}=X9gmahnM{rf*j~+Y;S!6+{65u%BJtHpm=!FlhJ@zw;It2>SOuo{dJHQ*DY~K zb#}rWLgb&eR*v3TpD$zrk`XX`9&MKK8P!0@M7hnAyW`(_o zq-{b$YJ;^q;Vlk6V-9Z#@MZMBQV}uI@d~HqGd{jsNhn`NH`jRomvc-Bl1CZZRe_I9 zURL&)z-VGQJh@1479sE#DZ>m#t(RGQQ^ z7KOR|*VmB%rWq5HL=@ISWX6Y|r~~xM?xL)h69%Hb<;%4n+0i zBqgm%cHkP;xYXKT=F_vRbxp5^rUK2wD(ZNLRQtKhH4<4j8IcS0YO;Vv*j=(2xyQ!HVbH7ZKUK1 zQR^b;kxgAE!(v13DN?W}){0Ga*82a20L((!;xsX-kwitSUdQ;Z)5ceOZrw{$?u=4H znFt!v7jI2#qeLo$w(v=|NCWwBILjGnhfF38aytw;tddN^x^xq!L9N6*yQm0B_CF!_ zU{t7hsQp8JEt#z6n&aw~)Pkf4h_{AoQ|8+qccN^j2lMI0d&elD(3~PDw!yT-rP^9fuDc+am6NzZbAa%(=&OFJgpcGd$ zOOOqiVBwZ7LyDipqCmwwu&`NE{k>n#q73$tov5$l2g(js#T8W4Cu-AdxclQbL6}4l z)BQ6YLR?|1?AYmgiop?Ni31d{%}!dCQbq&X)(VXv3vuc*$t`5V%eU$6eYdgC`zvOmwUQM%K~un^J}wmt8CXF?a#SOI9KLzN zQEq|IC6ETpFmel44<+?WudxW1&@rqH*_vvojWF1?z{t)4xmd*!RZhc1$w(}q4)l#i zSre1`Urpj6vD75SZC&lGb2dk*H1}Rw4YEV`OBWaP zeV+!?7Q5trA6FUedY&h(Yj~X}b>!k()|k)S?;&=)UO{7&R}tD=TpHFAQ6dk{aPi4EtueA z7OkFK$$>~gzJZ0#>uZ`*5N_^33>iwJWT~EFP91RA5W3 zd4Zrjb32=$N>Fqsxn}P;dS(uP2QYI;YicK z2xS)HiNfK2gdydTk^N+Z_Ba47H-@NcavTuZS zDnW+8AfhOw)Ep5J5foHiSH^lGgMo%>YoO_%f}jclOi;!BEVYpUDTN710YRtlZ)Mji zx%|4SYfol;f{*&Ezc;I$CwbUiUS@q`*)KGLvNHFhg3mz5Ix7S{1r{%Fx;COVn`>$c z{UHfxvtH$v=#0yt=RTMdy&KQ^#UiHi!F;!gLaKRsqgBRhHK&w%L zT0^O(LJW6!LFaFrWF9N@PPfEbcgtctIcE_UFnqa#0shOG>ZLhYVgZpbYb^U%a8}*~ zs^$=`zwN7eDk{3LsccBHjl?-zSVziZvJwi;ST8D-*CY}pxRjZq1!zkNOg0hxM={5q z_EMQ(%8-?4peYxia->!DJW|0xsg`9Nqy82iDdZk-xo#7FQ93>`kvC6sj_9}lZxKE| z&PtcuHamB$4`&(4Wgu@WTdOIT(KH+9{#$XX^J`7eKktt}vMU_4>u_pv}3yV?tlY$1Qqtu2U^y|8uD)LgO$#-4hPtM0v76rjP&~+^K)GwcAarM8- zhR1zFOI>x{%l~Ec$|QV1dC(P^tQN%`Qb4&hV!D%3=D-BIZU* zI;eX0-p*lf#*XLZ`>NwC0r<;7G+LiJvtC^?hI7@s9o~89VdxApv%WN+NqG&)W^Li& ztS>51e|&mjv=B5Gp4eI6FsZUC>CdtDf7}bP+M6bJzSkm^@%;(HDl#Ia`?H7>xV3`7 z+{s?bmYbE&!P%zVjFq#L(cJ0%p2n_-1sDM8g7tR(P>yslm{~|SKgFtIyL_~%6lInF z*ppXKU1|RG`q*)tAe4{1|DG#G&~h=_kO7V?wiF!+wPh(l2U2&Dww+S!Y3GrAC% z70vDvdEt@Y>VvL&ofu$2_d^K@Lkk~+s$NBYMLPKGmtSp@rVi3MWOEb~#N{&~b5PEhf4#gv$!ODkyFcZ|=T z1^C*Fy1vEU86D!<75=V>ko@4g^5ey+`i9we?B3&@7j2(2Qx&{eu6pE*6r5d#sInXx zp+FuUX|+hl3rk5G8jUpkYAC{TF6)qP>^D7g;#;j^~lgtMPwfwD~D4a#q z0o(+No6bwiDmv4t_#!I1i=3vyj9ched*-ocI~(QG0absSs2Fh23s~%wWlz{=63!Eo z?;0%pi8?)oFo#eym!KIVG#(?S)UU9Z;sgPwQe06uta`v+e&ogNezyi3+C zVjo8^_;*XloIVUwncE}ZddTCuv(eiJ(VS1d(!ax{)%w~=Nt1{(y?=S^ys^lwizj9u zC&VHPdZRr9PYG~g&>T7=j)MEu zl;8Dr5RL{r9hD(d?t$hmtM6PqiRei{E9NY^-o%n~oXt3a4%J!QKKLL}j=DJ;=zH1` zIv)Qy<`FjN7bh|xBj&H@2AZFm47ZhVz!0sG{$qEUk5O^^ z)2b%B;aWyNZT*GvxuYU~@ym1Uu0wCWlvfeqpZM>7Wm}6J!G}Osway@+&a(e}zM8Ch zU-xByU(|QOeAm(c@Yy^F>XRYQ#8@a$*K{nxdG|m*4CZUlb#6ArYu`!6d+$#^&^7=) zA;)Su+Y;L6G#`V96hq**-TdH~gdSRxNN9ox(O=-SC5BFl+EkF>H(-B|gP-yZ1^hn% zm21+eg<^>CW?!QaY$#6qpmj*+H$T0OD9E79qei_-g1IVXT2r=MG3*d(KxPQEEK@2+pmFfT$9qrY)j<1?k1?@|Cp zMa}-a<@fuQvdnQbFf{To;OjLk4OSkb;#Gz<1RQ0?Z%75!rbf^ zT&IrbssT3laxAO;pj%xCrKETCtVB-ZxCgOX-1{n`vHbq&@joEI<9wl~*X;}F$vHa- z(%0nrd{Kh#LZ8eQPPax5GK-lOYy0n z|FwlDC9;V+EW)A8-dnVAt3&Jqh1RIUWJRK;nCu|p=KyQ(tm9d-WL8ind*cy*usso- zL!#cS6_9r%Y0dKmfAR&NsjlNSq3gaqL5>UMOW%Fko@v@@&STxZ^|Bn_eiqj58P*k)31N0! zqQ-)qV;w3tK%62i7BfO+g%m%8Si5B#OBLs`1-JV5827?)o!c`m>BL(7AH%BD!Fw8U z_<2Pca~=iWyp;z_P5)(FVwgHV=?2xSU_&@EqB|(#xA}RQItx~wb%^{RQJS<^>@bxT zQrtLvALEid^PSY9xA`kG&eH=v3<2Npqq*khbH%@Boom&|JuXpk1{aSEfz63KO)d^W zH@-8EZ+eFZW>1okE2KZ;d+Q|JvJj@!#%YPE|XIvmaEMM+}%nxxCf zKOaC(s?E20xiORKJT(?$&Q*le;o=h$EqKXGZqgCsiZx>|!U!>3mx&U~4as(z^1}7ZouaXKioY|M}RW_XC^scm5agZ_<&e_d?V@OXVvsDx8I6-l_ zPh|y4m%8cPaAgTrgnje-oe5%sJCCz26fFp2|4MeIu&(L71nGa}&W}| zTl8C!Hy1m+6)D_sc|ELhiivZL&ys)Tzt}T|Gth8*3C9+L1~J}$2;6l2b!7>3DdjK+ zM9L)=h=~|64|z$7(nW~(Xa0kIv#9faUdS%k)yw@7tTu%6O$N(b5c8x!zV~xIK3D>c8Y3+rH)4MCPwVxy!PSKb8uzm%+S_Eb&6Ik__4ptVCp6A^JR zkB*04^j!`dN-T0gCBW9eC6+L^3y;eCE6@5jy7&`!bEdA$$PmGAKU|{WURlI51hG>cTcN&mrjg+pfH_YRJ__0hSda<7AiwmNRr)shL07lPUvmnRDKK$^BoY zzplGa@XorW*P$IuIE#uwcsAd*p&fAIP-2q_DgnE`uF(WJUHH^qzd6V7RvK@~AD5?=u@79y+IQVT06cYK zjkRBKB7lW}*B#q=*Ai+)t-aG|&7~>$^VC+yf0`|$C zx0TyocRf%qfT~@GO?~WkT~_2)qOV)Fozp5?tlDT>^)NM-N!23c^W?<6f;ihQBCbDa zEHzegVqBCk&s^S<;bp|DE}((qLvu+nv2sM9%p4wW1AN3o8VZY9uj;__fl%MyL#_T; z3YV+n%q?ycqGcnsR?%8nfW^WVpyvhOjdL;2H{z(%=Vq(rVI#3~z@Gk$eWVLhh$F9k zLDoDB$>Mhp9n|+9Q#sjpJZCBQeZ|(g`Nj%K*4K~tzHEh)AK*Woio1V^GaBCC{e1B% z-*nLtsjoPG+Rd!B9E=4yDFhgL+c-XH^1p#KFS&Gz4}3*uX|i=MVT!F;)^!U1jVJ1a zVGLaCA|2j^@3n)hPCM)LHB><}71)0w%cm}{ujbdcCh(OaVI#;6TT6B^UTdk=fY7KV z=4EsqxU903ZGAYdfMZRBs29pkF6OF?@SvMJ^?-`B zO9#|?3Z9i>euFet^GQ~FJT5Y>S3){J-3a|_tWQ1ZA`OTYPF-Dc&Klt)VdQZN)&Ei` z_%((=S0s+r;>AoA!AY2I>t=O0ed8cVz+Wg6{|0>jboz5Pszu{04uDnx<@4mmWvmAK z{0Mb+$Mpe!bptP{uc5no(b(8ekt;KFyYUdHBe8_Xt>P}MD6DO+(W_v~VRQBFd!${z z;OA$Il!??@MPq3R8vi;C)n>S0AwUeQc6uU0yytB@4)N$Av~b*wR{)2r542Lf6*%Na z!+R6sZysW3-+u8G)kDP%fi;)j*3# z*}_Ont!67BNKe4T2pyML2XhARL{XAJm0mDdu1kdCbwKJy%O|l4QPzJkYn62XluWua zY}O8=s<2>alu<_t%B-^QOkqZ$=+i>THzzGK*Y(Jlqre}12dBb(^q_m8L$vH)Psleu z53AOE-y!xmL|TN@IAJ=igjqU5W(FukqU&@T3|p+a-3e4_b!_`i|E4b?bbI=)XFLG_ zv?!qH(uAR8wP0sEstN~)L>qH@w9ck5qEPi1%4KwK8)*i5s6(#ZF3+9*2n$GAX$X;tbY-qAZCjwP3JXlL*D*h}4aiM`9hS ztoLsIN5(KJ-=VleI!H&$k@bKk6wVR56GvrV)e zwbHEC%7hpi7M+qm1667Hq0Fm?U<-y$_eb^Z@9~3GF$!px7u4NqlVaISRa^szaCw$F ze7vDbouk-OO6+oVd3!^|w0Zg7rFh*=<7`_N_Wc`|)A1HbbJ@#=XLa>}OXoCQr*`Zm zr*uK*m0@dcW>sw&OM=+@_&NI4lLdz`?6`LygexbovU$zWrYJgEt%})m8q`;eZmUY< zV2MY!2+BXvBO1bgzbcMTaB}dI=h9IehQX3_uW)X=y|92umnoRt5nZZ^)#YKx76HYo z!I!HP^fjk9G&SkNdZos@NwPpK*`HTjcmIQz`1j)@{~ih@vmO7Rul$c)Z@SkfPy4N( zgJ_yIk*KJ5_8pglyPenb#&Dm#QilXjfdUwU%Pj93hh6WA)1Lo?bfDYRLC$=IE>1aWMbyVe8A%31Uu}&+9F(ah} z*^vn9s>msA;2x;zO>+=#h)pNeI~1(<8XZKJm$bXlE+w=3H`j_GSQ%dA<1L2h5thBo zs6?-q*MFh0)HJW#0hx};La`Wk{&{>7c32gm<8m~6-mP`h1!7Ki^Q~+4R<&De@5}IH&(9&OWwaiQnWx%Z4R-3 zFUZPAy(F*!B%l9gXOEC|)P|U5f1fEXHD{FcJ$$?WoemM|sdxFj_zzR_<_2!DDQi}I z%lE5~J)n$h5x<wE!tHCy&%>Q>a4&RY7rWg<1XHj&jg#GXSub6tXQEy zPaU!cET9@%ou>2_*;U}3LD?6^)W?(d9#RS)HV=?zuV3Sl>>~7IrWf#G4Oe)`ZjqGA z;z$5c&@#)cW(?1b=_+HRe@m+l#RG!s336%P=R}jv*_iJlidH=yh4HPC+lYs;Yd@U z>?uEwS7=#*D&``*=!L`&!*xbAgT<{Vzb z;ejIP%eq?rB%4_|-ZWb*n^oWyCS=_)1+?oB^@CVKsB;42s6lZ60RI%{(90pGxOasu zo1j1qM-vA8>H&||ptC+s0VuXsvVm5~MJ%bgVIZezMap!IiCs}!nuSq0yUG&tDxa%H_OB@lgsU*Sw&2k>7 zK>PK{Ck1s2Aw|$wPlJq?NKNr_0X&4&BhD0kfn;>B=LzhRHGz-cgwwO=d3wF0Aa+Sz zekEl{UwNT%^|SubjvX~;`1-ATRXO+h${x369-nL9dvs%{?07Pr<$8shz6Wh(*yd3x z-lreH<8|Gc-#a+!FhEvEL2x$y66%KXzi2wku(q16ZMPJP7HN^4eSy>_jqA(wTU{96SywP0k(FKd7k;$HIOr^MJgUmUFXnecfL zCJS;e=eNH~v~eay8E`j6=|;Hmru1W(qs%0x@VEk%d~WJq`2kPY&9JE*k5yk*%{AxV zchdbJl?=2=UUpm2@@sOr**cJKYhWR>4C+saku?e#K=A<2(m(1vYX$ z06YKDGxCX^1-J3uGhRL%RX$GsU|91RH+B_!m@F5AFWL&*W-}E^H|~41Kd&O5w%Z<0 z(>>Pi!3~RZN`RIMqg2E}-Y~+AI%Od`Qq&6FyYjsCYLwwDDTKXe9pSYV+z#(uy||GB zwx<;fZ!IdZPFtcsBoh`DoDT`0%?RSpk2GBaEx+|;oLMX+NN%9C-|v`KHfbWS1wNP4 z8Q(E%E{%gGkucd<>E9LhX)gAD11QSHv<6GKydDD{D16%dYMl9(o_d|Kp z9sO@kc$=P=Z2f$9OB>d_hJ~-j+Q%^YJXaK1h>}020s(tCZBU&(__m$5C@iA9;^)R> zG5b?m?<7y?99H`K3hRJBVQ=^V*iUsp#oz#SOhRO~quRw`H?28Ad5xsW?UDqhcP~<6 z?o&%bKos2TCo?0tk%Q2Oe+3_VT{}xt=S^Kl0pkMj+hvDz$2OFC&&i@;F=+Vq=MS%Y zDuo`C%n?7~?ss-k$$EZ=+~DQMuRJ|#z*ToKp)F9u>VpSbWed~_H)+OG?4?H;(bMVk z<5};s->w(n%f(>1r$`wk)d|b$w z*FmjVgq>wlT{2zOf43CbmUp4byj#>KeP^@D>eRd=BK))lvJP^@SV+2!BEn1Ia^SWJ z+sdfVXizM=irmxUnYi}$EAU#o&LWekBhr}p)_C2`B}Teu`PV);_&2U%$H{isg{&G! z+BfSSoJAF|togJ9L-cojYln|OPOv!Tgtdr>ukSP0{#6*=S=djx0ck&NQzRcQ^v{+T zi$r5H)=AP|@df>f5kl1WW@h2=jzx?hYBFMCO(!n0cB1|tccRWrrwy_u<#IQHMMtDu zeSu9q=wDP#wPkz-_iXV~7IlgBsrbL83hv%Myu)_2Wfp4JbN%X(>{{qLdVIpCRMA?p zxUoV@kWbdiI9c_gB%Xr!&Q5;D?R&i&WyIC8oYKmGOwSP2+T146*O6C|$V8p+IFzKS2N$k;FLSFv?g#&^Tu_ALU#b@x5;O~Ah% z&Gm=Dvpqwn&R^S({F7MoC4A$Dj{=5ytsebND4lb!McV;`2Q}zXxXHcNru6;o;YXYdFLoFiHr0{jx#ZH)Mjq} z;f?gjj`67xJ(x`n zPoxKij8k!zcP9^4zx`#snokn1V)mD@$H`R$Dn^s9$Fo=wj21*W8Z_g#*SUhjY*+RMGF@rWH*+yX?673s%z z)2fr$GvEP!pC%>hG1CdQq$Ra~to6Y;Jc0b$>uY3rzJ6TyzEi29j$$ZMgWNrIK*yr= z5JIMczXHOR?^?m_XB|}*{`z2eM+*S5kA&?M-%BX=bbLiNC1St9<=gZ`_@CaMNNl^pD4@6SttSs%A8b_75-n zU}+CaFQxkWk%k#32R{W>?bl)=H=Pm>iAepC^6gGmXX_-iM)7s*(GvjZF5CRdZcy+o z1@A?VT!wQmWfzMX&iRZ3O#a_1YdQ_6JqC908zL|n`zm24^Wg}!C08Fr^L4boaO$!J zQtt1&#Ql~rw00+^A?Jl2o$w{PQww)gb48;A;h_^Ibeef(t19Q<33U&H2o6m@t)F40 zjC|(F_#t|y8P&y6f85D(PcrIZos#N@@_Aj$$F__sd*ayLs}2}t_0&5*M&&Gl>gyZgx_i;C-YH?ePD=9&2bn>%SRu zy)Q25=Rv_40lAF7TUvLr4No3ye&LK{z|(rWw3yEYNqO(Fo0pOrr#zFEwcbltrjM9x zVdUnr|Hc1S@XTcSNt0Bo&uhm-O2a1i=@D_CP7B0bP-}l&&$&3hxiHNk4`myDv<{t5 zy##LfKh_abfdBSo7;NedB1GXyDh$*So?BX<)j z(eNH-Dj1oWME0ceI!aP;kzCJ2M-m09s{0xzBG;=(yF9(3tPhjPU19~~ zyqr(pu7w{gB%=p?`L=vmYgGB?ldgBl)1fQWtio77M6zn%HSh99PWIXO*VLqwN>gQ| zpG?WlA3u1e9f@Le;WCPfl2b-_LI&c4gSgZGR_CRr)Y?UDImdgd6Zo4ma?izZg!p46 zx276YGil4j`h&-WQucd2L75+CuRBfi=a4yn>@VYEJ@q&P>RLA5+-|)cA##9h6@nKi zGs$Wi*1V_n#O~u8eDAbO>XzkvFJ6hJ`ilou)G2ro<`BcjDI-{!6>PM4W$1rIV0d1I z#P2pjYsSdBvbB~?{w?#k(xZuHxyxD>7IQb^+_5ECpI~WknJn;S`|v7Nq(?ed`YMnAkgIKGfG-ex_)%5zNbr@pYoU|xF&^LB>ch6LEs6SIL);>b`fu=@@%K!sQoQ97 z)T7*d6i~X!nC9YoD&jECwMFjNy1H5;X@&=S6zlJ{Fn*WcBa>NQZo6yx@Z>l2pMCl# z6C*P!(&lb++;Px6T;tEtK^Z;^nRb+M!XAYge3^fF2i^gErIK+7liky>5Ga@7h}t1m z@0-|4txW;u3ajxxhFp2@h*J!zvnTh;g1MHHch^YHhafq*um)NEpAuHWv@OgqMMf!U zvfic!)%`G%xBCwX0Z?Fsd)N#{>8nDQlQCT#jMl4|<@UQtpWDY{Jt588XyvGFca!Qt zBuqd6QKhTr+(o}#^T}|)$$Vs6T3Q!B*nNsK$l1g!y}$DQcrPUKV}68Rrn4E)wUdSY z{%T9kL<7v7XSI*GY#(pl#%Z^XSW)=`Re6K_b9HB7!-cnWmKZ;FHo9!_pXGyxiuz3} zFF0v}nUh-}GE_qH>1^A$9_vUC3DE6K&z-eJE&S$SQmD|FXd`Q*m+0y;>wkkw?Tnwb zkZsdDO$OR;a!BE0MkkmuT>F{DuQX~E$WWV1OsJE6W4APJQTdha(^ZqXV-4pj{BWuG z#H)WYXYv)R^0? z+^u==a$zZFu$+5r-7ZmE*Bgf2_5(GZXeD_qt(bW$7sbtVlvSlst4yA%SNkO~qVqyb znAwie4bU}}a3^51A1a$5Gz6JNyzC~G)n zG=sj)4i9X+Ed<#i2} zE^1Z52hI)lE82H0OV@B5hC>1W)-EVfpDz3NHo_srRV~lcn^&|It>63IE!4OS;v-Zr{w)NJb>g{O}jUZ_@6L%T>*us3;h`W?YbwyDl?j6M5-+8+4Z)s#@G& zA6K8=%Xlf}NxJ@fD_ybtmpHwglaXyy%X5#wKS zo(dDMd(Fre3{@0`hm(*}|YvC`A=U6a6fGNjWm$S@2EGz%wEW<)^E0oY5 z_&gutxO$G*zeKtk%qFK59=3##eW1fF%o5LGaB`NMYo#)Uyun%fg3grA$o~<#|F+Lv z6eQk5s81!FE#Z}%K0AxL9~oVmiDl#pnO$`tg=jmJJ2nCw-nq&$0aL?>!J!ZrwhbX@ zH6paJ=yM&Qxk%Hb?p$Y~d3F81h|O0ZTKo(gP3qI4X`}m@e~qT<@8pPLs|#bs4Px3A($m#T2;vu_$h|v zMg3oJ>7|2VA=o-)V@pUGbU?;L%XX|OM!1!w>;7qXag{GfEf1io|LuQMKh;VlmgTAa zq>OcH66{%AC@up0CCN zzM53Fain?IpP;@wDk-sIK!2Rvl3kD$OQf&@>#$3q9N(T1jG6xLEiGw*Xu`4)k>2B| z_huDVdy<U2!8^9H|2=|8(y~s{g|OC_|5VHGF_keOw&OUTZ8;KJ&~zVM++ zT$a-R{hM8OMz%}Nh&CWYiks{=niK&c*_4w5#(NJHDnT2jMk8y+IE*b5OD3#&54~u< z;j6f=zJBkG)m`+>=?~{Wt9#GFP}Wn2+UC4SS=(l~_*Q`;ey2|xL1!Vw+z+UjQjbxB zM+K*a#COsmvt@e0;t}iSe5<~xlw=5dVI?Gy)~nv8;_PbbwI-*stsYwJxkM^bhx#ow zU#Fo_KDc|??+v|RI@_GC*_$vDftz}Y2=dL?f{)JcEG5F4)AaAcF2jBsjXM1jEwKtl z360)Z!KXXTVE#I=BPivs#9nu7TGbu{O7Ix+B8BuM%h??s@lv~P^TyAlYpv3ojdh$? z5~~&nyl(upRUTtgwA&D{#M2~6E`+YRUuTDm0WR=$!PO*7>>;G){t{!MK}1_i3#F~Z zlNike#7TV508BzFl*NqJ}bna!aqY}yP{5mvw z3&Z?0243lacYl>KzqN}X+9rtF8exR9bBKQG9DjTlZZ&EElt4HRd@ESv1$Tx)v0bku z=%2bdZiD?$JW>Z+Vrsn2BIjA~jMD|Z-%@Rs_y-4tDpF#+j@9?ZLBm0ufA*!y$ zdp3d|MD1UcCMa5T-gg6ZPLs{noqYBfR>}@JAvvvTbXZpNRV-f}8rC3Ryu?hVT*I>{ ztARd|ObUy|Gvy4!ZX@46VMBCvmbi5@QEzV*Oi{~a?JKt|WwKr_2h&;E>2+?8{Yb48 zdV%8o#pFWv=lbX{dwA6QJSzNN?=Ic0jb}P=%|N-A(Vs_d7yH397PwT`-q%F8wB%Rl zFvr$2FPLMMI{Re^8p=+e+s-AN^wm0PPZQWUk-tFl)$Dl~z*h|F>s3Y~85!<&Hvs6d4|7yqSJ%F{Dq@B`f&ayh==3yvAFvlZ`!;8~nO zpkR&aP6m_gqXlhyzSieowMfj9?pQbdL(@qW|EP^1q4Ga%K45?4w#Vv5U%c0SRNT1< z`l{vXi>2~m3naMjN}Tq7FR$uXe?A|}r!f=f2XT`f3JvR}@it2Slg4qp4md82d6RPQ zP@>ToHjmi9)sy=5=grp*PjIVWjqPnu@$k9DuOC|dB3v;@75M|O8F=T7Hrl@rpR|Ma ziTFdWo*zdHpMP%Uz1UqLnd6m#=gH6Yi^)*c+%~hulRV~fbQ~u1MemXAKzr3B+-6oHy@^vE=l>ttE>H;x@ zdM?3L6_l|W&MOi~BYWZ1nT-fnj!GL`He#1nOKFK>ls8w{v)Z!lp^t&&1}&$I6yd2<)_J zFFh9A$x0rikuY1ApGEh*dEWlgANl=Jc<1)@IoTX1oBq)Lx?`?&-OkMr$-!fLicB#6 zZ5}9jB0F%{OVbIa-amJBVH6R>Rpsb|AbhF}F(Qyo`&8;&W%Z+RpQri$Bk8~IcfAR6 zxPQI4goI@|TbuUno?qHxDwbE)>{M0>oK^#*UjHp;his=+`CM35tz+^&2*U5qjLwh1 z(-YY%P?Yh|4{VCB2nP?#KJgGkkh1*RU{{;mj19YrVVa zs<8gOIy_s~aouIi%>~S%qq-1`N4$CD#q>)aPU$R@7IVST_Qm`!tM53n%-aKK^HU(U z4oyB|Q2jek83){%WUQKe&Q~yn{BqeTjW!`-LuQ~#AwasJT2;q{IJ>I0M?$sGy(QxN zo2ht>bJ=rhz1(wLwq#>jq4=iJW9QIw3MN32%$NjJ!=Gz^cE|Oy+lB~*iI0W!GGdOd zgoSvs`vpHsn{uGf=88b>o5L{CIr)R9#!!U?BmBodP*UJP^psoZW`RF~=RKgqC@igH znaY_Yk4!8zqHsGMt*%Qa|zfH4$5Mvy10I+ z2K~GGW;;#DpOOl{=z`yR?~ILm4$Y|g?@0ydl4m_Ug9%9HD0qO#03xSpW1{ zGaj?4+g1DG%s>ODN*%cX4@=gQbu~mQMEQEz*w*6J=-uZ*oao&tFL^RGU;+!#cO+;m zucFUZKb4w1)qF?NPi*c1eBn;=m;y?2#z?u6@tBXBJPeLBa#nv!9pmplv62O%P0MY=$hQb?1Lks8aRFG@iE`dy1x&tm$xK-Zi=!l&+T$ zru^nGIeA3m>-w6^D1ETsY)pasvT*Amf>2BTeN+rm50}NiMH&F^?AF?vFx)_Tk`b($ zBf~HL)K@)CfoQ(29hP8b9M5oHC>HzKf6@?=3Zxj_X#ye{A%o;Z?E*n*S{a^=zcZHQWM;aqjD%% zBYQ!)gSxS0@$U`$V{!WuwdBiKr zj-W+dzov)7-;_d@FA|rXg~dD@14e*y*I{w?#Y6`A@n1gctL8xr^GkKn-f(_8dF*eJeH@L_*vT%lZp+O&<9m|*{%h>#jjy1FGHOSN0_ys~M zeXDqP2UgApRJGuzC3alY6q+-k(8=x6UCRPBjLmzrs{#@hB8tvkcw{OUg&7?A~`H@mIR21=rtRAcdFhr(UN zjrcE<w2!1&P*@(zszIs+S9+k+oE+n`Pn2Tn%JhK?8xf@DBv6Kl38W#BM=0><{Oo_Y%5v|)CKeXbg(3q9C>DMu} zOf>-;cx2T^n5Zt zHhV;i^zlApPoIHOj%19~o)R%d_udUDMoQ`F-}PpyJ79wUMo4a>{iDZeN!1S67iRmU z6kccg{wZfUI{;HoeMBUh(Y_kw#-2 zQ}q9h67=6F%hb2&DFeC)G_?c#Rz_llcI?jxC{B|AWWNhZd&lDW`AHpG5?dR4_*Uag zWVfMzoiou^aLp z_fu~3(t_{F3ZjKPz#=C8))&%W9`8v{U+S{T4U$Ztd52IE>5L*ye4!vUMpL^#MFyS3 z8acH>3WJPX7^$2IF_r)QUEJP=P**@AHrm^d$OQD4DJ{osLm6s`*g1I5gT%!Y_rYqm zLiW|Ow`zW44>ThqjAbOIW30h)8taeQ&a!WLyLpt=;MI$x z&Xw_~Jy(s`UzwGZdGPB5p!Z=JpR!(&{T@D1e%F=CeJR<2`&-+vJ?x;~FOnM6#ynfa zm!a2kM+Yab$7fl(h?KR8`CeU%&Url-E&PW1Lh0w76Q{ZicadL?%^xnoae7 zXZOjKwYGC$IqCF%0##&~A?s{|FRs>wn9!b@@}C z#j*J8ApcM0toc-IOpb5S&nr%zDh_#QWaxo@ACY7cNPxl;?+=d>5o{n-LF`Le?w!Tk(QFF$!#cgvKL9^uL!;R+0h=%UuM}lDg zH=XyrLhUxaI=WgDlcs|EAxBe>ffyrFhvBG>Im>;9XJG^=7l15RwZrG-T0IcdalGYp zX!uhLR9C39%wcI!g{Q)#cA6znUiduZ_ntSRN12H-7H$Ci@fQ1>Z~5CE%Ejrc3#tKb zyexM=_+4o6x$%WUM=|&m)?4NCXib#ubz$1(G^L+;_QZ^_8RemNcgQdu+Dnhpr!6#k zqtJQZt|oSHSLFeqcx|0{;^FxyC+6BQS$qm#l=`<0f2YLOlGWU{HD}m%p6k=CyrnUY z8lc`0uT*V38n-VGrF)lcPZHk6aTEy*X?dK#(T}B-LI(-TVw`NpNDklSeHShdaUfY- zmU#yYC`c)@EUOrL_1FtoR`I}ilJhwbUt`*CHrkgA)oHunh4SVSH4F+ckhAD=a9!nj zXQ~r7pl8X*P<*U)){V&^5AH6_)Vw5l{{i>y)fpe+WVd~-Lwp4fbb?T@o6KIXCKAom z@0vIAUDRJ6J`2oW++E9MpvP*J>*!y*eK3XJvHAnqxAH^1BhRwq#ea5I{M1Fm>dZ|Y z@gPOM985C4rud%D^y0=0xDoOB3pbFbr0yY~w8Ug83p_vzbq@a!9E3W5*J;8WTtGbC znBg?0@^EM{pl*PBIhjuvoW<(aL;uL*mTgs~-Nn<1FogVi+{N*gRK~A{oeHhKU>(xu z)Vhw|>vFC-5V_!i-BI4=RQjIsx&$I7*2;0; zWRUY4huF__JS+508~f*Y^Szq3%#KeQZIVMN(bvUD^+EGfjBE}>V?6yKj?G;acV8QbdkyswHV1Q{S{HcA9Xz4!ys)_Dq+~*B# z7A;%WZFfqdEcJjZ21o7Zse3sJFWCxyEJw6M_v%hV$EN$xes}O}!))aNpo9fN@ymnr z?ow_I@EsD(x4!2Zy5WgS13Y@?{ikWJPW1V-;Wxm|V-9^CewDfQ3{X3Grp+~j(Ea*S z%ke&Le*+<}7W}HyknQyGUJ87bH_ER$VBb8dDug&%UWXjbr5bxvOFW@Ifpa`b_!X%G zT~AYL-(qQR?s$F+`(6GDHHpHdF)cGyB*9Q)3lcT{O`Z}l*Wpi8)9!nQ12t7IAn;J+ z6ft?BF&fZeQ^X7ukkQjs`#jbANwLNrv^{&xuogSz0(IEu;{c-P!s+!w<-_=vcKb79 zm=~W34rRw5(}#dmyT>u(hE7wNZzz|}q28~-`3FzbkH6t-p1a}w6OkR(kgxgYMzY>6 z1^bnZF7i&zje;`GRuM?}e$C#5ksiaW4MyC zzqYMJ9~CHO^)OkKawP8|&acjlV{(V`CCvuJFn-`(bOta^lz;_w6^1B|uK%)+IC4UU4sOeYUjZln(0G%?tI1Z4MI;76 ztakrw+geH9b^PF-o1s4t)EQXFDPap?<{SEbP%kjZb-VPbeY^+b`uQI!VQG*^*;mD~ zN;)o(o}vY@6oh-EmWAeHKCnIh3t=EAc^RolSHKbX;2pbS*xdE)5ErxZsZ9#r7~7A{ zP2eZwqu-B3iIjo6gd!Oy2_@MM*gVeF^gV!k@e>qku_eaB4GmjS=@ipA*DJ6;* z-BbEU%Ds%EzxnOqe&603nV-1pncOfj8q#!_Qey1|MV_YXyu2Jd;3^HYL`}+6h_57k9KA1@Ax$g z`yqimmBFX@te(YB!#1|a%t?=L{5k4uFTX+oa=^@vU3*OgDON*x0q>%N^(i<`QqHI5 zpv)^C`zcsUiTId=TKa42;}$A}u1%21GRL;ExZ*f4xd49*E^lr0CyEX#(P-+OS4g=ezBmyyS)- z`ziEsy^(^;BMWx4ub9e?zuz8eFBDgtb0R>RNG zVfYjZeH{@H;3Uo-fzl*ixhEX;a?LnBD>Pv?9AGU|;rY^J)|ZF0isMDRquf78srhro z-RfbDSeIPvs}-T>$~N_#_Xp(w(-igHYPlqd?kcN4TVdRv=8@&{Hp*}U>XVLUG*)tX za^kFtBz|^t_9MMhk%YATacCi!x8RYG2JMJSH5e+;_m*tw;|gdiYq=R%d?G3HYvxx> zu+lRLQC}VzIw~+Aj2uz$!fN)K=y_@JGjmaYMNbG08%VM788^Z7Hx1>Fu8mP1&gnjn zHp%=%m{INNeR!tY!h#X z9n6M>B7Fv}Mtrxqb&KH__ARIu{4`pn*($W9A5!IkFj*38Z-Ssj5J+zI7lWBCLgyi# z@x3V%Q}2Y^^LLU{pA>Gl`(SOZjRWN(@N_SBsjFO%$==ijyME)I9x zvQBG$V4s&CLHpP1wwP#2%tNVLBL~0b6tgh1jlGW2rjo9)5v_kT=TGx5hjP8GKHIhBCGl{excM`v)5bj>CJa$NhY zZy;fFd(38fQMP8vwj-g^R8^pspC$#JP3ZkRSf;f-_-_48eGKhblJAFWMovnr7dip2 z?e%t~5~*SFg2)}IFj=SVS;h;o)7?mLIy;5hm=E4+km!-K`ystn-H1-qO zjPHb6veQDP-yWiai=$~Bhdc5z(cMPETwVsjlqrB#0i za3+EUKfJ;W@45VxFt6vK|8bo|jCn?FRGs_1Sb#K-X6q9jQxX`P`7Udf zVyb~XE5XVv&O;Bg4$FnL#FUgg`m5mX2HDm1lT-BK{0Ey}jwQzq3)6CC+%{Q()f}dZ@2Y(ssxwoQa#4QUn`z4iq zfs4eEy13eoJ1J2!BX88GQZ-XRsaS90&!CE>t6idda3GmP9!A4Z()*zE#c5GP&vJ}7 zgQ;c5vOY;lmUUW~c4oWICFHy4TCklD0v3;ztcq=(2+Yx8WTnt#$AZ;9tJAfWhrnndf}F7s+FwhdS?yx@kFWU!I-D7qwUwH@vHgD{ z4-QbXXBSv0V|+ZPN@D_L85;Y2-|mF?U5So#qs6|^()B_+FNe;HD_7r-UYLOxbr>>z zv^u1L!K{!K7hBOs0al7V?iit=`|68M@rC!>TfF)fHN8Kiz;g0QTCWt=o&zgE%)urm zEq8Bd-sIxqs4&s|y zp#+%X-4fvS5n5|+xXQ0FNALjcoM%7p%_L|XE#$`7tkgByUwZnt~|YLxNG2`k58sNl7-Ob zhznyh#7G8nOG7|VM0LU6g|i}z1V%{AV5B-JqtU6ISEeqD#~Bn^)i}6v(MAc#pXH-~zZC^_(|x->HoSwMvc)3cz=S z<1TP)zshTD?*i6O_NAtTyf)rRjKiP9NgZlL(y}_7L)`(Rzzgh_jrJMnyhLG zPk-xIWPIdSbUBapDlZr(i!2$5@)zpy`yhEL!)djVc_Gfut#;6n#F)01;wb+iz<1ln zwhECax0sETRau1X_~AW!RKuW4tRGT3qxsNs!K_k-{;G^K4hl)=yzlKe$#($9|nq_glwOjb6tK5&A%iW&DKSl_l6=15?LXU zwAc?l@8*q+rH#ENQTqMhEJCeCpm)X+=Y@wbW)QOJU%T1vA4+nvER5eG2;^Quhf%Kv zA6rTTTa@eM+ARd(R`7=f*-|IV9RmFBrnfn26&>+cb342Zo=0JrL$A2b{!INC1B zIQ=;MX&u%|8P>xJy|OcpZ~k4nz^sVu4#ukbx5hBjAb4*}dihpj6aTWU#9ZUg@uUiT zDSMCjuSnz%`W379IeZm&-^2{c+aT4N?NUWi~B^wB+J z9E2*AoVk3u+h;Vxotp>wdt9jEetR9g(XB{6X-lgFtYp4hjqyIHemjP0&64s$yDLaT zseZRLM-S{X1E9H)>e?v!K49ywn!QrG?A(Hm17mfHmawBeSyR^k_M zzhDqR&t4@{*lst@17KjI@i-6D7dX;j(mGuUWylkIGu3;|J(f}5Nw`c_y)N`Z-H8%< z-ZHJPdX6_tw%vKO_myM)4*Vo~((`~kLH;(c(MOM zWNn0-in(>Od2B3HwLFx8b0*=V11Z1{6N`0l!Da(yG2P45^2%1Vs>?K+2Vzi*Hr)kGO`=a!BV1{Rb3D_+y<%_Ujc^TDM;j_ja|!qjXqWt7=t|Bu0#wLITR z%?p8irxAgL7@S2;SwXw#tMEWpuhChL^aL4{Xta`dymgR}D1wm?$q@A+DIY{_$&9Pr zuR&7C7dY3R@z>`Ek%R<3m~XfHa1pCChlcmfnQ5UC%ouCH91U~q^*X_SembQgb!Z&e zFQ-<%rPIPlwl?h!7N1}8xvUEQppZx}goROyi4e5e(rycDA~;T{Imj>lJ;&rL-nl%`KhP-J+Fs-t{V7s=muS5xV9 zM^7yh6pk1pDGdlw+z-Pq9e$OM{@$^faaI9`1zB}8CYlfAq0(ST(z%|MYm>tzBQdaU zE_{`>l!(tgwfc8zUU2kYSt-#%k3HmV00`5$@h~4fU~cY5e!WdNRpJ-x!tfc>e6r^BOPx*Ks>>c{l{a_Plu z=mK!wq2K}LO8hc4>~h=pHE!oG#c{@nF<9FkjgAi!cHG=n8ueL*U@qI&8->prT% zt1%(KyV(`8vzvLNPZuxvn;tj|sk)MzWAE6y*2>V4G>{e}h+_%8?uI_m*bY z5xb4F1+msCSHy-98JgWcbOSWb)lZbma4o_!Bhb?~(-dts7QLy9n+`|FOWw416Qus4 zVkYuO^iJ<1ul>erGL_`*$tiX~-ac$}4{+91` zyU<<8`5Dp3$y#$phL|UQGW^&khfjT=R%OQIKczVavwC zAXZha-3s_}CCd|j5!F-N)~tSbQ+K#itLOzRoW671t)R;xwr%PjC;h(6( zrRun}qa-uWrY5GO9jj1RK6Q?g#`Ymr>rAes*zfJ0yUs6V{nsMIQuw4v@O_j8KhI&H zUbwpfnj4u%+G=JUbdW4fxIu}X&Iz++SV*+nKn&9zW2_WnQpcB=erlZywY##$Jz$@( zMaSc0?!hKgWDM-du_8ZOC7zYR8f#fv)!=q6(Y5BZ=nN)YgxjBqd|mf=B{lP+*t`tt z##wVo)q~Xnb&AISzUbDr)W+{9?q77xe8BFom)ir)m*@8n(f<~6KUB=*S&%qQ^ysa1 zliESLB`BW|L`nbW1pthF#@O0+q-fq-qio)*Y*7{5d?oG8Ffi1}WJ$MLAQ6(G`86j zHg0S-wrwYk?c^klZL_g$v?sQ0o!BpJmiW$|-KWe|cMf03~^xlz3Hr-*g^?abS6 z-}k6r!lV88@iQG(pXvDO#1Gjh^fRIb-l)?7zD4S>5a5I4@ESifQ;NO)ADhcrOR4U5HgNB{oYE=u`$Lafp-Wk4i)pL}EBfd!Q zz29q~%4J)v6@p+n$A;D5 zEXcWVLmK=aGd`z9jhgm-RAM#7&(SHo<5@~KKMxu^aBCTHBs|ZWv=&4;S9x)?|?vjB-hkUstYMkJnOG|cK z8J~qho7{;V`7%#1P(+P+?M7NL3;$37Z^v^zhskH81Bqvxflq@U(i~Qzp!9@Bmq9Zz zxY5#9RS5O3(%3I^Ori8(`G(7Y0vcS8Z@er;G$z(f>1vVG$)qAX>|`3V5X8xoTZ$m0 zhfWME3z%oj5D)zpDkNCj0ZhequPBJmUiT_ahBIAs_jbz8qhkpE;lBKqRs$4rA3=xpbpqSEUsK=f zSipXYZ-_az0b4(|(h%9kh+03GD+080Ni>>EIXhlNU{u_?Kt%XA=ZHTu% z{KBTEdE76)zMGkWid=z*?g#4QtB>nlTQ~1-DfVj@|LFi^;Vl$eu`{Zwx~2!5dv zlRRbEVVAP(5`3x7USpC8oNx5*$u@+{ua%R?-#LM>f9RuDWR#;9C2N|q+Mf>Vn7tVp zPli23&KjNLx9d`(^^e)AQ*ZPJ3wDycUKXrQwtbuKZ#xky1cTdIZbCSmOPG3QP+_=O z3|Z%Hz4;V&va~lkmFf**EsO$6*J_qooF`x?X7`Ej89!^2=Ob-Yy(aBhZ=(7N-Baes z>2YkCaJSr~;VVuL(Z_AiaGkgCA^Gu>FZ`?0TRvVJn)G{w8=_T+<#80#vv!O=DA9fB z#-`&$BVV#oiAh~67loj+vG>|AKr=4i9F?U=US!FReIQ72QB}-06y2(c#5JF{>tZzA zW`>`$DbsWMxtoCZ1KJ~AURw1|5U?@&7XwmU6~`7x$*V{O%h@jTvhLBK4Hkf7dfIw;`2qTEPDMS$yRJbm@ML>MV^0@Gae`6(tDl zx+sR--jMp8N76ZFl_-I{@pmfwuFokM+RkSZ@v%orQ7;^aCOn;+FPFy41(D52+a$xX z@1GxhB@H`O+ChbA8x>iHr^}x)k=SE4d)|jj^v-t3H6yi1)o|3XR#;pO=%rIvl@etG zXq5fW%w(#(zbRoE|530B#I+CnG^Rm3uYcRT)_!xvyWGj}s!aXQ(#>yaWK<`f($AfV z@Q|JRK>SYpE2ckr?iPFD1jS+eLHfX$5HbfPQR5E&$_F5wE3RS?_1~6&BN&%lWP~S z%c|H>D^leM(YG^MJV1r>mYeVlV~1(L(cS(bT|MY3B^Z_p2yLcB%czRc9%Pq7W!%c> z?KP<33RRv(8pI;dp$dSmJ{mGCCkL1V%O z+UyKM_!Nt>;X(Guj~I-Zs4L*Of8ZS1el9L!)FThL0VUbKzl~yXN?Wc&L2z`Y28pU< z%SSQ7zh~Jg%cPwbd;s~^2wHnTGBP0bsmHdJo$(gaJLTV;l&J+j`$FWjpL>lsjddU0 zh=#QaoIWpktbW6AzCRT-xO^FX_t@5=a=k$D{|L|v<=O!u9dXj4r9IGC^ILz?QZPSQ zFFa46LDKdp5t|hw8ZVI+A&aOU%Y!x4B^&*Q32$Y2!++%KD8e2zZcsg+HaJ0fg-woQ zq`8Y|m&&O{sd5(rimZr*p4Ym4ZGgQ$Ig{!fylfGWmS=7<0&LAC@Yt$#5af>W%A(HtR0in{vn> zSc*eA5>zIW55P+r>#BH&-5X$Zd#4MDiWcIAC(K3oKDvPp=!+%A>DW@IBLs#S#EH2YyRf%duK8(A>xEwDx z;V*que4+i07XfLDCynP* zu3TYr9O#4m&CNf49Hg6P-Z{5tG1KhUz8`QbUdND7+LrsAC8Q>vs<`@|cwHaZ3|_!9 zH@7bH7Ufxrc1}U=YhZH3Y{;fX)~&(o75k)Wsg_5CF_Zg0Rs(r31^N^+xhT!G3#!3fTk#{hjFJ!&Kq@q{l~_{7vh;%M2IZ=X_VCwxIsnk*Cv3&*^t4{U{bF8-CVhAC z$II>(Lc#Yg@!s|36I0?me_^J_^Zw)vo53CsB=y9>?!TTM*hi&~(Cu!0JjMwWnjGzE zc&zZ82;tOaiHsOo!B1Y1rmmN0+Wfn*TjoAl0BzF0J9Byso1_{*R?j(6lv(3v_$>)O zj?J<6S`=S%ou%39`<=1pb$Ya-*b0`UO~ZZ{t7gw>t8V>pzCTKRJIdF;Sblf_t$Q3R zg|ea**|qmhltt;PQQiZlhslrM#m3tisIg>TH!q~w@b+^ZSM*Bi{4S=bhTEnyU0AT4 zC1uIxa|Tel?GhFJ_#84XQR_)S zyY-`0pW^;6DFl`J21Ukx*%p~2U1-;NPjHHmqR%oN;w}dPwKI2T^?ew@9b2ECWfr-U zfc7C{J<#UknBNw|6yIA0Wj9`Kdo%2DD9_DdT9`#_l`UC}w~Z~G?I&97Dk-PbJtWJf zd4dS*)H&uIdLj{HeQo*Cb_kvZ$-lMTLekmAzUv z&=k+PoRSe3g0~<$bzG>N^gI0fI7;t@{#JAG;dlBMwi8S2o*xB+VifADDw-&|;+Q3J z8s;BkVJvi6Yi+0qPuKCt^9*%8a6|;W7AAFoC-xZk4>-vQrGT%HPr2nPt}kwC?jX;A z>dlYIcRnw>%}>v+5<3l4R(^hdKi8vs4}%(`f^)z2E@#!ASSkLiNgc?Tj}LbFjyP%M zD4{#6?1Zd=PLpg(%c`cld{8!tVEyw{+FttV^<(rB(D< zk9~1LOXPV5LqoPU>LjD76^L6-SKp0YkS*i!c(iRSj;9HXf9faWY3X_)7m$e{p;=|V z?v+QVEGTL^w$I*JkBkr*Z6R?W4xo|ulZO-z0zTWDiwl{66;Ejtp&i;DF__yU8YPjw z{b**r`-RL*QX?Hj&+s=yDV}s-l3cg<76IF~W4`(2+eS2vN54uIAzZK6^`m%rVdu?p zr16FO(saKKBsM10!Mg;Y0@O;n8=B*gu#_I|pneasAXd)?t)#Nypfy;aQFcbJp0X7hU)L{t<69|xH5fB|uk7zV7|FGEm|)p(?EC$8^% zeMt?dGolcwk&NgC;YF6lhrjBw%OtGBAU)NLO&yQeql}H?gig>q&eT`WtZNFXbCwL_ zuP1)LMa4d8i7O!!nfs;a2&}ERU z5aR_q)$6KHX`f{T{QFn=`}?MpP6PH$t#OpQ^yA2Y9UN-AW$vXXr*^9&Lx_XBpLPX~ zAgj&9^y3FXC8M_IGgFuLN`-786Cw4*IivadpFVGLUhk zlvVvH@}&+IFu0JokO5%p)EoSX0AkB2e~Bb>N#3C+bRGGT9C#xXt@m;Vbid1%!CvrH zv|DtRN$m#TWh$oFj%ca1Q+;g*?<{fyX?l8=MtsY*8`?7kz;&Jpt$zkvhTuZL#vT9dpH|af`9p-q0*K077sw$}6S15G8 zx|WlH1c3NaBr`Cpu$@zVtuyT7DFk9fSn>^t^vcgPhU=V3Z0|`j{+9grABbc(w^ZpH z33hm)>uAA%1TRt5Byz((tzd@HSiKikZ3^F;0w}q1)D2XzeiO)_OiYzTDsj!{IhTrp z3zrZPZ`J%m{l>4&xnudPpR6Ua+d-Kc0J9C6{SJv%aZnEz4Bf<272+nf=XM>C`9nZ2 zX8%w$mN>0n8$reYJB1*B&2cLG8u*XM2EjGo&QB*@IS#9TH~O%WFi1yfdpGv^v<3Xd zxdWEdC!FhlR@46yx6S*kg(lg9-gjM?eDW_z!H8uuT|(Z zom?i@qFpQJjVhV>9uWRjaa@ zJeIc>x&PQMr~48AQ9*!t?xQx47wpGkelt{mLCr4C{U`u0-Yuh~8gn_F7r?K0%?zV!Z3r6i#RGQVBK7@$Lm zU=i?x_CG-SU%w6+`*8Jl8uw>w=zXCgZ918I1Aq66m&?_F7sPTMDbl#(UtA-?LPQyO zsRr$|!{$N+F5e$5u&Ek!a_-FJoUiI|@(dA#qHN@OAHMbl4^QH~;#*JV=+vv7=|Yl zLD5a3l$8u%t&E_!KJl?{#dHRPo z{)d+70D6#EmhqHWS$6D6|15skmne+vSV8!j3e(Q3FgR8ZquT*Ti>!x{Sq0f#w%;f| zZTop316ZA;-h~A9x7LWEh-&X__ak0L)O1d3O3YLS{UG-Su5R>8)`*KxV($e{Y+7{G zDiS>}F(jvDw)KP7Xy_n8@}$8U_P$2X5i=4dV;Ha2g{+l=hT6fpL+(FM>>?`Og?zxp zA3LHc6#LQWKmcV!iQM-zYa+_3H%7VRc&kr#?!a35>=iCo9sHZi`GeA;FE6pwDvAs$ z&xA9;6bkpGgvu@^PFhcH{8!Bl!_$7jgMjbBj}UR+tL61YJ+}?lYFCzx)>7(t+Kkr^ zVz=NB%D+2Hf$z1y9SI`|i@0fu)4C^?kR%~$RG?Trq4Y10#!Ij+FY8%AMEaw_gw{l=+?0zJ^%vd(r{pMl zF30CE!@w&}YB7*3xTI8a($cF6CTC{L zB+m7u==+|ox0;j3V*gdxj4hD!go4M)6lwG~maQS;Jqygq&A?Vh zz#Do1W69j;+M@PwswO5Np;ByvDCv0`a!FBMY5ktHOn{WTfu-|m%oJpAGjrKKM$VtI zQ56IWqemd3gEz+fr)MGtU<_m_or^k-B8XQOWA#RtN|IEQ#X}0$BO)wGGLPXuqSkc1 z;^XbF`H3YH+Kk=T=)JNlW#D_eu}JLeuY+AEyi{wM$NKk&h=RGiJkDCNJ#a25h#h~X z(rOf(s0(xS;I>5f?aoysiqSY|xgX;4;A%zSUBNel!yeL~Uag|$1b(jI8+Hr#&n&Gt zZ+z*zHAn@FfTPqPNXK4pN)Ae!>!)fUVT9m(p)ekhSs(_hW2kF6JZ8k-kWbux;Q9O;qg z*)G$`EqvT?g0eC&dck;7=unaknBe|M$^3Gt47o)o8!x*!?(0MPaU8_y94rvRFAiI} zFu(T3er2HDQX>+`Nr`G{{JCT(B%%CJ#fBNSiVD-`RLJ5fSnhq?;)4APDR6O>(@t!Ea%W#x<+(mCMn9 z)@z-Jo)W%CzHGj8c5j_ftjqHgW(vohx_fKQYWcM(buj@Bt!uy#1>anw+qY-SU|=~= z@Mj%zPx4OzoRwj!DKXD$r@)m3xaj!LWlb@Ph>z01Rkt~9JLj{B5l(_5jKZr4x3uQ? zHqUo^;K?E{`zxPJ=RAp29Q`>p(n!2rO6!UMd;CX>VjIMCnkUoXk-r&Pa7ouxK?RzE zzSxyUSk=!wc=)nGF%)9O*qKtTjf_|g*D7{cA02CPm#+tS0rExW-MPfYHY2u5|8L|>l450u=SjX zht2WbeJ^ocVI9ErVBv-842#z`#;SdD%4@Btoc}ZZ%=BnhTK|m93N3lw&Tf6@yL+^t zWAnXsVm)EiAwp$<0e}bVexlS6<2@KGQf&8D|LGU255;23R;4de5P`60{VS#~jd+uZ z`#Omg+HER+Juvo^?a3>e04#^a*9sOtYd;x2+=~CJRdViy`*t<~5)^;~fk^7v;V05; z+SVsZKFwJML;UnXmGLOUF*qFGkZsnGuQI$}4BS@#+O{mmJe*EJsCJ9tyYt-~!&?}H zF(1dNT26_cb;Hnn|8Jh>8ykc9P~Mz!ek%M92~ZT5Ht`QsXhBLfa5M)Fg!H9DmsaO7yH!^WV*fP6&m6D2 zQAKEup_92P1*(Np{h#?yzqM+F3MxL9K_PE62(((wHsMT3*8XyEMIU?SkR;w55Ro8Z z8%U|POywd7Ek1_u0i>e1L9N?E;0(Eq@z*x9lS~mPLDeYNbU%a8;BbEMSCjJBqYT1|`OQD)b)pc+@@2b$VKznQ+KGMh4;|l zP}%(x&cqx6)1W*T^7CWHR2?Q}%|_jyM=LOaGYXf+4>3L zk&Uq75IuxM0gO)|Ajf}E+Bo1k2w|AJsa!R>?cS#y$WRrgg7L^fY9}>TfX7xh$3Br3 zI@KCsJ{u5lHKE&@&R{P6$YOe_mw;o*zr#j!L`zx74`bAxeZVP%C}jq11>Z$5rkFEn zUfSuD*!QphkzN6Pqi;rL>#^=j1wnTn-srMqHN}tIX$L0MqSthaG}6sPDA9pf#<4VPOk_Xj8oMJv1!z+ ztzB)b*lOyB&)-*)6CV~`{)Hc&e8+ze2P@ar0s%M0s7t5K$G|ChyPAOo6_;IhM=PNTAmGh&?g z@=#EIhaHvwaph)hxk2Nt5dl_At3cc%`q0Uj^Y#};htnRkIw?v%k9yuA{M-%)W~j}O*$^T34@;9q(N~Xt15g~>fw& z@PR{${YpcZWj-bKwwTV}7SGvZd`+J3^ytj$&zPY7GZbIgQcp?g6P{;5HaM_McM@MifRFegzZ zdSLo&rf7VKwdDB@krHky#$=i>Ri-wL`2yCyJ=8s$-ZPeE22ISn{ek=taVFk7E8}YU z!yr6`Ux>8@#hjkWYy=U>i|KKFy9I5=WOa=CSVu!Jb|Tq%wLMw}Hz@4YNzrG|xpjc_ zRo>qQ2k&v4|3P>*U+UPDy8zOR^G@dd^Otf zN3ja##-mV%CAmu zY?2zk@tZ?FTY;vBADU_*_ zE$g8?Vlt-4qQ=d7Cudz`Ads9cTo~Y0_VyQr6rcUxMMT$qCYWhs3RK5e`tpV07(UC3 zb&EpalLb(r%Nt8NX&|e4PEEltUQhPw2&d&@rFPo~q(R5yNhK|#9>Nm=er{w;z5;?+ z(_-7Kbylj!*Um3hrDmTU643Y#65&hSJwtFkUi-Z?RpIKiKzVIIIXY78MG8_><~rm; z%DE$BRvC`;d`>7xWjga>ifkEP@c}%KA^O+bzH1g|zi$?5{7PJ~moL?&a+%h~eUrq@ zc>-CZT{Y+0FH~96BrN;Wf#`s;9_j{Tor8hqnlLy*Yv2VgA(!nwzwDPdFpK1ipVvUl z)W;$nQsgDIzwJQB1P|qU#yQ|xIV2>uSYPnhgd|mXUCO(4u$S)n!6Bwf9c~6q)?hQX2iwNdnsb6eZ*e^FGv+ix zN_N_Y3W9!2Sx5Zwj~gH6S(bTDix=turtoe8UOd3PLA@+7Um$*JZYzue$hd&6*S_KP zbvJ9|lpG;VOl78*eg623?m69jK|i@UFx^m?r~|qr!xD<}RKnJVF6GT}hkDa|2KBoa zDD>^B+&@78&%7y&xRR!RE<~D|E%2}5eZeN^w|tAA*7uqe;+nD}<}DXJW8zc1^+Boy z@NdfDBj)1Ck!>WG`iN--pd3Q+ZL20U+Es7L6>TEV>q_w(@YRW`jlU?yVaT0<_31d0#scD|%qYr~919zI5l=JXGl10}g9FzXd*v;$vsPF;}8(f=a6^?!r2hCRZ z#FV%K;S|b_G&|d&kRYx&8hrC zkY+*Eh~2_gGJ*2CuL-X!dhpl~l7ED$(~y#rylbJg3%^u|7DbZOf5qx+m?8{~h4m*M zIvhH$Rg`9{6>1K%0o$YT5sy-d*`KCKx~#R;=3j|FCTZ!M@QtOeXy7oCg067y7ea;4 zW9HpogRV85Sn5ht@c2fdL>>T=Pj=O|Suvk&J|9KV9((0(D3(iOazBqUlSy4D*Xwwq z%(S1Y#|Al;8@1!Sdf*3XJ4pZFa{T?a6mKA(%ZGlHY2WtuJ&q3$Mj3jYu0p9m#QWK? z0ou&WY@EY#wh!ndrQ&f>Ehdt@fJo-bc7A6M6I+lRMVz5L$1eLi9_dpRi})LWri)e}-siiI$9 zN7-a)Dr77q={!T}e9CzL8b$v@fHJiduPaHv$fYo8A-JzB`ip!}oIXRK`K%#JFJ|gg zEuYS;`rJYm59ej1aCF{y9qhXY3FJ=>*Swhq3jSG=$ra;K`uGD;CI0*`c)|o)ahhlm zPco1N42E9t9In)p3p*xJqWuuXx3op%MU4y;^0rtGWI?x1_-vUliTF9KK_Pp~8O{04 z`*Urcz-XFU@bBH<|M~Xy(jk>X7iF;F*F5891hH=VU<3@BpX7)bh{BHH%3Y4IrBvp0 zm%px zMva5P&iXntz!i1RZ0Scw%$)VQ(Yim%mEt6V2Ku=?pClwewSYSjFt0KlU#I8|gqJGW zi%p?2@@qbkj-xDrqQnHcz&B& zRd5+&(MsJ>^|MgMSobs@0(a2@h+8_pmYrZz8Uh9ugH?Uej?7sn?$EwXD_&RDNwYe&BBw|FNZXGQ#^2d_jTQ`fx84 z`yL5`mCAWWCa~Q!%H7~x)3*P++iVNDT3M!Y0lFGN@6~^Fd~ymh?N*R8od&aua(NIk z5iH~~7e0tNW1fy1t_mb}Di#B}eerdg>{=G{LWy0kwN7*PjTpLSF5gfD_ak_TU+*g> z?M{6O2pIUh-2{wlMB3M!wwEQlQ$32#KXC${=I%v=wEzqYgZUQchr|S1>vT`!gOJr} zD}i2nk`0m7*7|+IwK;o3n4QNjt1v1`UR?U-mO3|*3bGL^4z6Z1E7a;U3h4)w%vEyY zR8H7+&`^^fu)f5?h35%-?pcjz8Cm%fVrz*6e0XI{nQ}$BN-(7VVkJ zHRg(cQ%t`v?GuH0vba?(n1=6YJL1q4zsA4+K`oBScPl2Qc}`!61;%+`0iI3I*p_V9 zAtAf`b4iynB(h+LXhP+m>tM@N;5oA>kC_KzaB~g!hgZ0O;HQBMQqbM=PB^_)vB4?< z{%s%13{$PW7sGT45aveLS=!!K1X-sXp&jqkJ^3`wG56DT)WwZd`PQSwlg;%9RG1jl z8$m3I#P7JrKu1z6h748{uk#+PS z)=Dg8UC={bfK3|_V{G& z{Ot$+aypGqM6b59HSf>}QalA5bS7l14GfdAR8mOT*e!1fAduqgsnEa8IWbGtnxxYR zW0xw`*}rMB7PK;Fm0gTPY!nEAJpGs?k({gz()b=fGlww^Tcu3R!)2)&5QK#+?ToYI zckJ3EHM2^YyQTSarkHPeWi~?K&;3 zLrfJ7y=^aN{VtsMugykXaL%-5i}`$R?jrZcd0&nR(|e9vw{Z@JPd{sExeSkWEvl&J z>l-fL-2>vgBjAwQ!~Lc$XqPR$OY(7{R>%)@m_>|}mT0DVd#ls#G0mCl`*>GE9OdD8 zxYx278_9mV6h25G_b0l87I;J`58!}BCVnydY(?$DVz#X8>UScYEbjLvMzt+R_@8wD z$ewcuw>r?SV;!x6Z;+fHTR|CO6`_C^?l$it&xDLp#V8^rV}0BCD!W7>#`h%>SucsB zL3+4~T6{4eVeED(Jr(;gFi#2#hWme4)i}PNBFy~S)Eqrsi)>aZAwW?vBU96&SdwiD zC+;vmJ&)vDX8NA7<`E#iJiY0gqFIO}Mo_@GzXS)Ara7@_q3JYS*Af10aN zBIG2~Vyq%UJRGc}{k1dvWNNI|ynp_0(<%gWEbjUQ@Md>w==o?B8|)rwn9!`o4Cie3 zEuKWq9n2DDSvU=TDa?9W{mTlHn9)-xct+;mR zJ#3W8`)-J>seMHicqleE=$~q@b*+=}c0cu>{CmR#11htfWuugER?hxd6uh4tq*BO< zu`WbCt>5I}wJUn*N~uT`sL3UAjizSpKm)GKu^333#wOf0$yF)yT;J7?n(M0?c?)CVKXx6;3rP&mBI>nL7@$?RnY=sL3GHMhKCTr*$X?KHu>b6 z-~Ev|8(vZuXYQ+r5BU!QH$zGe(c@ADAKvfG+56);Wy~u%Eax4g@_fhBEdqn^uESpp zuOA&-^nK(V7@#)Ex*r56d_U6Y@=Vb@h5E5FbKyhe1li#_IF+>)XkLvyY;N(G6Mt|{6XT-CyA}?nMKP4`P!ege zmCzz3BAI6gvk7c|2QW&BP13YQEX7~td30)vM0DQ}Vh~TwgTad=1p=n33b8|`j?--` z%}Uo?rj=AZyrx+dP>qPr@bL`6%VYC@mBW(E_a3g3E55Q8k(L;bow9wtOqqN}lw%sJ z6I_Ol9ixXzo!sw_HPtH1yTWTa`aNv0$a2<1RvcID>Fv>Mm8Jdc!TS;Ek9Vo9OyPn5 zhWqVF>4u)R$f<6c9-JSfI)R%YvDwV~+CJAi~kkH$=_u8rnV>l#2woQ`# zr`LiS%GlievVmAYY{(}{f-^$HqjIAKscc^F{+1&jOjl2O$7VtEHJ=Lxu_R~NA)k9j z+#H^T^79d|5PoN~bdQqvwC!T;h*c2q{@l)f;_uo0u)jFb+W2d|+N9prJ zg)#GSy;Q=!16~7xDME%6&;j0`Wdtqs_DaCLY`nV>KxH;sx@*R&Sf|btKis8G^7{}g zsQu*r2Ol6qdVk~+G;!fR>M8?dg2hcK9@ok89hc*EU#hXXJ8rNT^SGENo#emnDy%Cg zpO<};{@#5$PnNLAi5>qP~&tah>rg7XPcGG?L zQ>1A~-O|1_DrN;)E9Sg4SnP0IXL^muLP>M~%baZZzu!&98z;1PZ0n@Q`hn+dUL!sC zbkZfQp*i*2sVF6f&RZ~+)tS?>q?uich8#N^;^1o65oQD`M7i90-=If6dm@gLu(Z$? zCneQvsqx@MdC@G5j$A#R@$c8jiJ82#TtIZV;BuIP;BZMdeUfww;pU{l?JFiDlAl@M zCUN@}>NmIglKi!0@_-)}wIyg$!^7mEe!I8@3|vPE;b2Pd=1bq9`l3-l$H(wuXy?_~ zYV0a2L9+UfK}M&{TrToEtGrXrg>Q`jX^ZAO8 ztAlL!O`$ktbOHwkkyj?lnC!<>lfaH#NBZpPDHZ377{c=>$ER3Ru-E)ziyr61NPSlNP50Q$H}}o=72fJXVEOfHxQACOrPGJV{oiwW zb8dIll%MB=BaA3R9#Bs`U3t$g;+2Ueg|rf(G>|e)G+_wz_b1cn#S(>AwP0FLz@~ed z4bu4jT(PdQAUDoX?(ipJ*u*R6VNu7C5y__d^-98dGjYCx(`p*QerwgXL8L|?^}KWO zcMoF0Q*o91YR&A^I*;#DcgL;Y{Y2(~qV$?1ezWy`;UaYE1K-1rYq=`(zuxxq<*Q49 zLE;d5TSG+u1L98D64oG?3<=-+x^a$*iVN>Xxfk0uFrj^|P-;#I!0A$5{=wI)tXT$7$^%1xFdo0RffsNwaF>vpYx=U=Pk-6Wo>w+AWYzZdWZ)#I+`MX~ zH2{qILe!_U#di)F>dJp#iO6v7xb$ehzG(69)$sAf{vQ%j!z~dSfFQf$j^`poXX#Rn zPJIiXP!M_Px;m3f~(JVe>>~f88k28pDFZ7XOSVDAMCDpxzS&qn+RxKas-bzk`A0mhE#|E0{AI=dJ{5*c ze+KWCOGF){NmEJMimx)KWpyHu%99=$62}O!qkGLcG8+pS;nI)%1CLFgYMchss-Q?l zW~TxHl5Yi0?RV6COxtrQ;`^@Q9^<*AWiLd@QkE)MObyylD0$ZEB+c7r{EU*g>}_-rDYkIYHL`j z-DWiqm7X{t*PZEXBm9~`oN~|olwk3G<8hadex4d)07~K8$wg0voc*$i<@Z-e~w9u-tGE z^=yT-f$LZ-hsV3!H{o}>axzq5*Ri%iz(FBIdTsz9fdT-v?S!sp_qMqD5@H*}XOwi7k=UI2zH}@|v#%8NDS*h_EuzoGM-0JKa_N_kj zjp|73Z5&QSLwIZ}Q`38G;pGLa%4pcPtfPeHI?$e@N-woe7qETybRG2+gDROdx9!Q* z?Y4~9d$_<-;HWi1|7cr$e+$D@kkFepC@>!f3B0;Y`nu`b*hfHKZoR*G@C4QyJUy6T z4n_qLYi+fi8z|U;1~+7>Pz_HwZM_C(d%eyZFoE~MfnJxz*6zy(Btc)LK%bh7I+$y?}T+zt>UkMQ_?WMc*UgmIk&*A9V4ptyVY`g?Q z!>&a9a%w+j)X0IfCCs5-%H&@lI27TGXi}kLm!&#EOjL_v1663grKF@k`5UG<=}!$k;-x7kI={iWV%9M=?#ul#uR}nUxy+L)^3=Hi0l>6^ZXfOVr=#5>w_` zMM@f+Jo+3Rp*#`P@egZhkL|?=l&}M9q;FU{yW8Ts|xLDuv zY4_m~%FjM8jO7)V19ZSo|Jx)HJ1n|zkjXi#)w8np3PdRX*i6P zvGzghFOIQl{?d_DDXXnMiIhk5*5@Esa*tb@5Vso<3gf{dg1v|OOJ@C5YxBO8mJzcE ziJ>j@M@@LY0@Ndn_eGUKem{~B0!R3?76CaUTT>#3mju7zPv`kaAByyW>v6&EZzxOP z&x7d?@$(hRYmtS)`)J!D_!$mjfAZ!1-zuO-hdh9Fajc2-8f?b4LD&`)srd67{mi4MGngW;PscuZ8&(YBuI8~8(;&)jALp;A&DD{Q z6qx(@gb;kVLH=tuT``@wOaJEufU@`sMp)VpeTsnopr*5kCx5p{?5-S=y7%T zmmLIs^0regQCqLUm;o|e+FJZ@+YL>ze>w9QQ7Jk~AMH9($*oJ>YSh0M^oLIT(fC11|2 zprc(Reta>URp*G|j>w4dK!!6tp@^<~+isUM6b0wr5#oc9Mza#tI>78QT zSKZWQUpjJDTL$<|d|uA~KD`5APpe;G{Pu ztpGST(D0Yl|HYzSL8>-D>w2NeH8}4;xfBJNHwh7W}Yj#w(`NE{j<$Pmo*ycORr{n$Rp8sg{haqFN zZWW#@+vqg~R`6@2Gjj1!GObbQg7p-d;H6Ou?cci*#1}UKWp{rbd^5cik^e{7IYmbr zut_+WWMbR4ZJQI$#G2T)ZQHgd>9}Ltwrykc@1DKdi@opDZ{vJ=tLmxTdaTdU zshv&*@DLCEIvb6`K~Frp;kr^nQJ8K*L|@YQn8Zm%ei%WF?KK+!_@r@5TS_Qs-BR_| zc3xq7YF~mm1dTEl=xq1i8~71P z_FP{=H9=evHD{ww&KDr(_I_2=y@ye&;@Psq*UOhdC3$p%f)3wM>)p8C;y3@sPXJNA zEs0KmIjdN~;smB0jt2Dh}Nn zt;BtcH@^S#u`LX}S|!dqopOA%DUDWolrN8Pk5xq@^wIspJMjbMo_;Zx;gdC1FAA!H zEqPg?%GO_^w}hZ{i?y5Zuf}W96w6l8=+#iIdHmH{t=GH5)o|B!`zlw9Z(ylN;T@;@ zJLFx&M}NlD9ZR>^*Z1o-V!uLT^!V!Sd~UHrEcuOEQxDuxGP_&W{%%`SKl1arsUHCN z=W*32Mm1UITypoIcR4MOzgT6^!WEqmPgYL~EGOFS>CBExuY4tNFB*J(#`sqyAO#xJ zPxQRME)eOnfPU8)nax8 zE6FH1{K$8{XjGfk5h(h|f7QTh>=N@z^CIGT(qBMFOWwGq(cS-h*wmosub-lPND$SS zgU9K7t*g@%X%4P{{u%xn?D_W{c@D3nja(d+sqZwcFoM@Hw(XGTI2nUp!~M(9&mJKg zQ(sOPf?@$?um19X`BR;8j5TIv^J%W3%)i;TwROWLR~Ue<4E2BYUGg1f^{|e+%xC{; zB;vcQl-x1n2OkEPV$FJs4kg&2V%xmE^8sUP4l)=(($(DMl z6p87Rg$e?!jBTEhuFlXP&+zf$pGUHFPo)Z2j2e>J4(htxMD{!(QXIao{q_k|kRnks zeSy#=FV37n1{XMO>#N3#R+hRDa3TRQ5b~nc%c7v^K0K_V!?R8IR5JCl@&Q=!I;{`5Ib z2M9||r?jxvhZ?ecIGKb5Xu6Y*-dSe5W2NY`AuBpT2oAMy2gL40{Pz_Z@Xj8w!Tn_^wJFcI-6+7P+e>$2@G+8q(A(X~* zb(*D|sF?7%Y=IQ1JX#}eEgLpQ*PFj=%oj(q`y}4R6C+x6m^)$@iu9UF9R55)lV*5V z50rH_ctcv6oxxi>8FE@Q1WA!69cEzrGhmT7IqK=UimUKwa1J;`2hLu-EI7s(bL18p zLdRHG&DM@Kw(JRSh~IVj`aVm>Zrzb}tp?gEUq`p8i5K;W#@mFc8RNxloyL!VeD&8S zj!SMWG8##y3|Zz97XsbKqI0lT5lz|tz&@?E95A}}nD|l!7yH`+229q@gRTxU4-Ss> z`V`})2Uknl=?@Xy50isoG5Qc~0g3ikij8#NLprg!!k29Z<;s#DRh*hJ{;|$}96Vo@EZOCN~EEz^cN&N`Q#eC49ut z$&Tg05?wYKdcknah>cMr?F&R5i}-~Pvz zm)Sdyi^h*e#Eo{h9P)?>j@r_NMS)e!0xfBuplV60q;p^vU7R0-3}`@pg91WY3KWAUkg0d;uW8 zN{ND}N-~nEkIZkRKnhnE30|{N+L4zF!^R6lJ{Cp1>Dyr$6R-k*pIztNay|4U&~KqIgMaT3`nwLQ z@%MRi^MemESKCRP%T~HhVur`EL*zO8;VJ(vR+a1;4(hTAaE*K=`_TIl>L*z*tS8pD#@WL$xuO*u10YXT-5= z!f>Q(Ptm+n3-V`4#6-+9S8wlcH<=rXCpZ+4Zzb*4g;|wg^LLahj261BFGReXSG+Jp zv0=>O-smIxNtuH#Kcf@<+D%#nSs4=sf-k_akMpzJ(91$TZLK*yA|~Ss_9&uu4EYGC zLhP-Uy+k{Bj)wdMR|%u)c*)H-hpxOm=8OSeZx3ecJF~25Lk*A*o>}he=0~}B-nNAA zm4qz^#On?5&-{qXh4OC`JO9Qrz+GgbVyJso;d!^2570!`MzalI6uGHgXfX8ek)k?n zCQkpT_&V$M=Ts>0LBpEqC@}F&+~;C$U3wnSziuK?Md(GW)b8p2-ty46cx@pNP=lDV z0;NI>jH3w|fY^bmOLlu;|H9@X1+a9-FtvY$+wN)|Brg;y5bg&l( z@5;+RRGx|X#cUiAkN_7Z=TL1To%ng=5rW_$A48Efx$TC?f1BrN>y5@!U(y2dKf8hcY#2BI^aFJy^Ck3>IZF0JJ_Ep#XrrPWWT^V)Yd_3<`=nbT(*-$tui zKUJX3YWd-4j$F%#4OgeI;c**{f@mN6hVOD;YTLL1&Z}FGFW9LJ`brA^TyP_GjTz^5j%mW>FlRC5 zQ=C!H5imDIn{t-Q1rK6ly>>)3GB7;MizpPKYBiK969dO_sv(GM9>MSwqXe;Mn+EZ# zK^Gg_7?)kb2`Unj)1}&m9)(b%M#BvVQtChh|2MfD;~1VV7&9mn06GYA<42E#q&lNC zf>IiWEXo87B6gM~o9a7yk*4Jz!Be$Y4U(?GLZO-Qlgh4Sx6HJvmvjO{k_3kbIo1^Z zw?5I~R4pt8)4v}KmXoKAlq^zDEQK{7%Pe#jb-|U2^}1(PpRh}{3%fDZuek+oO8VdoY=IckL~`nTS%Ekrz~SgF1- z_wpEx_McRr|a|CGTf@;WDsf~YhDgG z(Ijq-9Yls7QXVbXUIzn0n?m1SGbL|Iu%(qdMucT+6Uz5?yW@!;n|u8?g@h@_Z_7w7 zdti$)F<-p$keH=7(wz*B+jYb7NQdW~Dy&{FHs4ypL|Try-RT>QrP*XT>r?U7sEiCE z^tVIL=k~!Dh&NycNL(+*9`D6riivl7O=ZxIqQ%hnY56vpc@^!pr}N$uC#cO%Y8yqo38zONv+hwq?=i9#5a;gc&9#WBTn;CtgLXq3C#Q@NC*)+ZSlypfSi6<^n$ue1-eJw-_lS9jh zB$9KKdQ7Cx8}gZ%aLx99kUv|rll)=OqPZVDM9Aa*m7UY_r0%7nGuuR>GC=1NrJF3F z-Y9rTLcLJAl*ezmik4_XzVnISWO1+w+qeB8<<0GAcQPX_4$JgYt**}O5l0ncf=W&W zLCi(E;y8lml=h^mU|lK*oEG@V#m}uogc=O9Cx?Pu=sYNOQ7?57=i?AnQm{n|Hm0`A zK1`}YCAp@X$D@0HY}J(%2-*?fBfcD8fYFxR_mx_e&M!JE&4OB3Srm3Iu}*uX;8EO= zWioMYjj9ZrWXa#Ga7qO`SfkvD&HOBMdt6a8giFO7d|kRCrO!^T{gOib53VFjXLqLf z9j{umw57b?(Saia#bu^RS?!nSWPA<)J+$WMKPm)ol=C+*$PnLkp<(O&Z<-uorGLj z_&F13a1_@fCY>_rCd`Eu8j}ZZl!>h;5|tXh_fR$q!@=_^YXUX60m2 z#X*cianrQNmdzj2>C!PkNXIgtE$O+FuJa62fE)_mjy+oF1Fm}E8 zp}5?Br<;Dlezqv^90s22J-&{%*KRxXh8r;2vf-B7@`lb+>tN{hpT zf1(^fi_?bZX*T}rax+3@)8}IYpzA65o&~^rGh%#N+d*8D4ucf!)AN05gMGEA>yEi} z7HhX&b#NS<$0qE&gT4@r;X-=Layg^d1Z)`IDShEb%m4FO9AAmfteT3`<1N~fBVZk( zl2^2|G%;4!KpkaQGW&FW95U!;^&E^67;?$sc5X21Q&=%eR5w23G@mZJ;ZdtF^o1wn zfzks(_TzA9KOcks6SsYYmnV6~eY?+!$hDhhO3?K+F!VZ+%`WPo^x+TG?R1*iRblLs z$tuPjOJ!Kl&jM_lWb?FyjL{1+=zZ=r7%y11u;O1^82mWl#aS%ZSt`wt#KsRT;m3KD zU5i5=`@|N9m4XqWQpHPVNUl_gtA8F}-)&=-Zp|5|IWR#7Srj8~7Y>2BN#cc>zzai! zL57a98*G&d&j!7)bySg{EX0_qrN$B$7dmk!FHZV|FMu#gLDqm(=FQ(ev8jsfpx>knV^+Lrc|_j z&V4y1$T7fDej8Ct=8pmpcLkZt?<{z7q`u>rDp0w=kxPi zdjnlX2lM^}tU?!@9uf%1lcn2dc#q@*U#}$XL^iMO4wx&TBRa#DC@VcAV5Fv&eC5a@_E<8j4S$(`T0ZvmX|xM6Q4_MOT93 z75&GzUrXZJp3>F2bHJpIK9j)Af|1tb&72Hij1Ldjet068YucffDgXKMY~6AJ#R2+m zf6`F-wm$65V>CMM|B-X%V*>E=<59MtQOn>rTD(}=xXbEXq9{B6%LS39O6U;tdu^U+ zvC5d0n%QT)zX%+o%=~5%zm)a32VNMkw&LZf$9Q)*iTn6odGB=MGThc=xud{<0|IIh zr2t)HMpR;amhx$;LF+7Vxcz2xCiw*muB)s!ZKpHyZ&m%hCM9N6>SR7s7PB1?KUq#O zHD1n|d;Vm*0aJvkUTR~VGbyxz35k%XiQn#mhAUB8h)CdE&7JLEm9N;jezs1)Q(>*j z>|K9e9&sARqy!wI|Ao50D&~-FI5{B&R!-A40Nt-qpeMPV#8~FwL;;fv(dwRuYQ-e6 z>3$IW9pw@m54s@3xe6qu z44KQ1=HfBGkCymk;^vDJ$Pxk-LF&l359F<}dhGrnQ@6;m;<9rWtFmzc$oA-^!xS51 zc>pan$BMTqeRWs*+!$AgyymPbeC982rirp#7myJbz<@d91C-DVhqqu>ED#4xK z^*YgGT%&7C{dU&e==Cwh6sbMiikKAY?ze8rMIL=u?0b=k=#2_6RwrHz>rabh7HcTQ zqo?rVN9#%q$xuvbl=k$yDwfg;?kc!`e`Dh}NI5n}RDC`rChhJ}RmF}lVpZ=cS5A~D zz3)Ez6Zg}~Q=jLg_5hKtZx>aCq{Z6C2QR2KU}ID8O1pKNwj64p)vW9cB^pJ8Ev6un z6$EIdw;ppU{oZ$B^Qf-O8VMxo9*YA;sb8Svq#R}NmW&($#{?E@U@938Aq9??RH?~7 z(ws(qju#4y9xqu+FHZi0mwLovcWg^E4_L3x+Bw{H_DeO7^ADW1!Wl7ByLh!;Eg4Ib z2Cuh*?)5Eq`PGx1PJCIG4PiL}z9A612Q?EXKaQ!eYj86Q04bXL@>48PyE%TBq&2HC!Z3(R$}%?t+jS>{&|yiCBI z#4N=p!fsJVZUmAz3n;Mj9358L96beIdlo;zJxkunC!qdZuv~08TpZi-JOj@f1L+ZU zeHuD}ThLOBApHE72)dRAmpG)D&FfhMx2PBbkLol~9sLna6VKyw?M_?` zZKn*EW-!(HdIWqdS82{+G~>CuCk?#8B~Nlar+zL8w+Z=4aW~{vK#PNC)##FWpgj{; zEx$a+zb_yi@O_DycWA^IX$DlNR<5@3Z`}$zmRLP~ksiU_a-G@n;!gRmd38v&%fE3{ zbJ(7^M{O(UAa*>g@?zn$T^AoaLDHB+(Q1XxYk>zoL_o{*{AKy^y;O(LU*R-F^82#< zqv;0Ky~Nq|%GBC$7@Ml&{H}pX60C!QLLKTpTu$XIjmt$Y1KAMXhz2hCl=Blm1ZeuY z;rP^)dPB=pcDvGUd*cKt9fjkIA~UC)?AG=2E?R$6obU?25zO}u)d3Myrp3&O`h1ja>L_fF$rfv^N$o86j*@D&|1$ErJ&{bCCPih+l_ zxB3IM$pcE2h(JIL3Vi~`l@a1AJuePsAI zZx`e7GZPT(M7c*)etX|v82q6Py-bies#SX3MiLGpKH$Ti4ElnBxmJ3;?O8IcrsFxx-YJKsf5)uXbZ4J>vt48vT>fy`g4O`_A=W?g=ahaj zkEOD^w_h~fHS^j=zj2YUIj)Y|8&TSH8C#mYo|G=20yFiITvp0nLO3Gt{MiId2zKoC zh$S_7HpF z04K0dxE}Cl7;2QO;6nH{8RDdD|7o9wo0%W)1IwM%EcKklNSbwQ% zR!cnWDR4i!yfL`0CSi~H{?ROqX{}rTRT1d0DCeciD|zpz?YAS5iTTy^Eg#yL{*3AL zb3dLF>c-Yz(QhbN1WrykZYmFMwce&7cj}UL zX}LXhpj3-5`f6zSf65#T~#Za z$x2xs0Q0=paX_1MA2lo0r+>5Ws*@o6+8mA&*u2n#n8VKF^Ks6S<)UQVIVK?E%+GMC zmwFtlLX@}{wl~V> zWnk)yk&Nn1cCXcR!w0|?z)8}8(P#T)b^3h!rb!!bWk(`=2+r(3InP%gp!{yG!HjE3 z?lZY&=Hn2ND5uf@UeHs=oyu(7VGJ@ln&prVUtE;I+WNj{-|}X?Ect00CIclCB^3A^oo`r4DW5fH(+zYCo!3pw zN;$tsOD48?Ug*g@@$&v5_F8OUD&6k-=cBI3c2J^ilMWZ(1|$r^DWfUVWyY;`5= z5^(%vKYHy|-Sr6gY1QcG=)PY2C?#xJiTB#w{G)G2su+gpJk1TvXmfo36HTo`T?Y`- zUjD=2@7wXklJB1uBB}XIe@OfQdL?Esq1Ma^%{{q#t?}+oe7;Q|_p62n+)I0*(-bJz7-q()m<|LozHa# zhs_0t4Ac!|>DhB=Y;82*uv!=6WBSj<6%?w}?H_TgvVx?qaVE~c?QvvtwRuMB{vkdd z^12ZuuBE?@7br46|3=MI_}J7KC^bNk?^2+FlgTu5bruj@4xuy_c?37ekTD`t;5hvR z;}1YqSaIc%EnwCaVIt!!D`2jt*RVH}=bO)iFpZH)%|nqU7C4-t_U~LAYH)!r24|5t z#T4XRs-s#7KoKsK*@F9*oP$NwN(gScSrW#`Oj~QzoA|q|o3XVG+@GKaDwb+S110(f zC4jh97(F`NdL%;BayK86p370J?MJc{%nfs_0`z&4AjW%GxS%`oJdOQa_T)5U*oLK< zyUI?wjJq}F-Luu56yyY)kV1Fb_pDG*BYta| z&B)%Qa03uHoH3zm?nYnJ1d;S>-Jh5KQ=RAp0s(M>m8FI1Metz+!o6|ilIUEZ!d)7F z(Q;t+Q??pF?0T^`7~AU+o1xMKO+NvoK+o$GpR4yH@D5>!?O@Z8OYdP_{0V%#RxS#G zWld?$?V?=pLU@!!^PkX;y~oh^M6XAUHLV}3`^|-rCV~;Utd=?T7c54%PC1iI8Wo*c zaI(ICB1W!HVQXVQul~&HI*lGUTP&R0{ragDw;_e*q0SNJV~EGjZWIkGYDPMc6PGX# zS18;E`R2UpS=K2G$sqB(|1stM_Q%Zaw#R>;+VYRfw!vHdmzyBve$gF2vsO!i!T9hM zRNZg3BDC9x5JkafFe&Ys>y5Ijb98vW%rpFiPRq33V6|g*=*L+&P zGJT)Z-rjM|J0r)1Q8m#|YXwug*3Vg>JL{YM^|xzG&pXCR0xr)$)*#I$1~xkORT6XC zBrn2BxI#86xyhL-IaotH}?{)>#oq~z_^3$!ib!l_G z_^2GK*}qDc-}Ep_)&sAnvD#4|8L%jmz#ibiasoYfH>}hgg|CM6JFNKgmdJM*by5w+a^}bsh-9IDr$RW z0TqAsmY=OFY#3_ncF04y+~WEkHYcbAah-90N`n@M6kbCC@n%spN1p;W?eNKn4errV z|JLkN4mtqwxv{6YK*|W3cK>N_$tif_&}&<>Ekf5=fH9rGL9hYfBG{%adO}fU;_^1X z=i4|`n|yVDhCIWVZJ!B&=wpK`CALptupir=6wqQPLhXj);F}%*nb1!4J#-sw)8kW* z%g5Q)+_Cztk772ktQ84dcENcaFnw{=nBi$EW*h96K$j49^fkDEUCvER8Wu${k>SE+ zhv)=(RjmlzF6v6%dhG{}qYt|kzop)N@mR&;zf}KlqlnEMwerifTA!b6%Y)+h&1pvT z#8t-TD;-|Bw2F?x+$i3Bf!R{!R25A8NAdy$HY+Wffe z`vY#uMZ-{x?{S^VmaOfi@6_ITceZcm@9&e@Yw#xXwtoto9FpO%;-DWMug&7U21)_4 z+<`??!WQ+klmet16ZYh>BMU{}9wJ;f1!dnfVs89MUZa_$(bP1r^)8#=iod+3HD2P6 zoI(YztWwtPA$VbT9;4mm5zYKj1(&&rOKe0?W(YpGyush~sxa$~kURc8dB5mam1Pr5LMQ@vGo0_pg{j>Dx+aUXZ2&aR>0@05^-~23)OYY zMJg86b@R-xWI|J)G5Yhi@$;+YBy&mtEu?vkatkIX(V0AA@9B}2!Odv8Th!yD-lcj~ zZ~J#(C?Buqswy?WH`ONEK2@a)>7V(8rp@fjGwRPL-#D_nqj8?Ew8=NZnWm3sUhDO? ziKG!!3E#()IRDtsI-=Ii)3mus>pGuP8JLq&D*ybas%D6}e=mc3K>U#ruPgZ9Rhmr> zb1qu)So{wCw4w-?i+g^3?a<0_I#h>ZY<3ZnlrUN+5<->ViANlX%x#nbs75T2gb^Yp zY<50oW&K`>SP2WQSP=&ZA_Dj~Y?mtG#2;!h^ez^N*2GI-U|OcqP?O3I5QwCS zrQNY{NmM%3HEHIsr7;%PtBY9636&3FtcY;F zApeNR4VfY)^tpCRi3=yi2CeSR6s%2UP=vZmde9q;;CcAVlw}@>g00dObjrkd6Mrfo zH&gHpPwZ2cCJ{++>_tG3fpR5^m0I#L{yrNa6DxQe$_j&2xH%TP{=sx(2Cr!vm1-s| z#bmgD11p5vO9xrF3>QM`gTr#lS5%|Syp}V}MBSoH8mC4O|7MKZwYf_xVtsbcRlzSd zYAKTq#XD8o=%mw=nX~HW24>O;9o1c-TI;eBjT!T5f!5`Q2)Bv8$NzWAQGqB&P(vis znLT#7LkEJ>*$CHm8cQ(RzcwpxeO_AS1}2kG7{q)_)y&G`Jc*K|IL`Ybo7tC5D*>Wb zK-64=45#i|J|-kGgbae-4w$`@G{ipxN98udeuUuth`wp%+?xh1i}~TFN4C%uw)ART z$5g~-eQ6AKU)oR4VXahJ)5@{wC1Krg4$ZfF-ez1G?Yc_-KwTMJEa9)22ABGHaks_z z89Cx7&x%dpQ*yLo!a_As0{e%L-!DcIj*vyg;U2IxVgl=%DVMmDKxPKF{Z%)`@QjS- zRr@28QSfUO-J7u=UijxdtDwYK!G?kW4u|FL9J~@3DY9BTEe@5|x5}9rlS*8*;$EfE zr>-a=pQZ-EYN;RQKA}*=lWyU#t~&8tg%uu{i)RQuZZQ@37{}VUeD5JQv(k9z^)SK- zsaih!y;PRVoy=?RXeyo)oqp%hFgji$sDFrJXPPxHpyK9SB2k$^i#4bEZorYcdEO0c z@!sc0a5%?4`nbb!tp?rv_r(7N_fCw_9W;$qMHN+!g_a&HfMvirp2VcolQ+%td03k` z6y18XD!8R`GSwU=jgECL8gsGWLi>cyO0Q@Y0DM38eg|cQ1#JO0dvauC*+r+8)3&&| zq-7l8#$O)+Gv)jb`FJ*1qpW$uxz@&X)HNfDcMyszs)1f0-w1$Obk{O+ys@Oq@a(d7 zE|eWI>Z-L?0~|EQhzjr?%m|z#F#J3JgR69C%*4!XVla0SjwU4hU+PGzwo7`r_ zr)1a=v0RyGZx*4RyQZjROD{Y$g^VhPX^8{^OTyW9POAJM0#^LcOJxd#q`E96SE+|C zLAyMrfmtz5JQ%I)Z`+=Tr&GJLI@4fL!Bye3+&*=K=n>uHAGO1W7ML9lDd zRo%*XLm+_&u4HsmloVe~A}Ao9Z@-1V+md6=yzIJkLxA}s&40wM*imM^fnFhgfIzue zXs{UH&{13ZW}ZQAo#*?{=yfb?1DF>j*$)I8UJwA$hz3aDYpQbYWT5ZL;%xBeeSQlE zJF{UK&H8Q;?#)E*DwT+unPdlj7;`-Qv07GB6i$p*oo5a5KxqLq#Au-2eQw3-ZBcXP ze`7p2_z#G2YVM;JGFKNp;W>Q!$;zQ%rw*1QKhGs}Ym$O*eD_Y_py+*F9RAo?Z+}0} zv>t~DY#<(sZxucv9^aJixo;X2Vq6|cO%HOj9{n>1wpQm+))}PnlfGCdMStUeX@ml{ zPWrDjh`hgJm(R%)3Wp3K+HB!ZA0dRyKWZnb6}?UShGt_cZ%7$0Rsr9Uh;>WU{F6CT zK!p}>^p3jp28fi{_GT`NJ!5P0(`NP)7;@!upBQR{%ZLIQSzp6?8e9wbvHjZ<$?nV> zojwj%V(~{j*kJ z^fCn-7xC^`<@0`pw{Oe)f*b*bI*aBHt9Pn5!umg*H_FQ{_TvB>bEMf!zmo!ocZ>06 z-m4N$gGdMF=FJntBW+(;1Cg*Y5Qf*aY%X3RJ!ohsE!+A9=MTu+pJw0(E%UqIc-XJE z;?7Xuy;oc@=Z$!>uJ@2k?AuuGaAJKl{Y`s#o1@j+a( zl31(Ttr(aP)8c^s&Uj&g;P%K^Lfd%4tl=@uE}E$1SkoO_QYIES4Vj3O_>fCG@`egu6Gaa)f0v0tOkJj^=yYuk$GW46&n66Km`MDon*3 z6A-AuHY~a}C$^;Jbp4a9G)?Io&z(B73=dyT}cxaw=-?q z>JxsMS5~jROBB}Qq;58^)ch~;A(C<~32Q;Gt3*eWsbZk@TQfb! zK&L94zd@rhc0-QXH8MZK$C3P#|IZo!SOJ|*mk04JMmF>BrH?0#4Oj+UfajAzf{NWTw(_*z!<7(Skn@E>z-mcr(0%In#J4!*zg;{pn12xu6 z^;g}H;7_3c)8iG8$7yAj{C>kSSzN>$IzAx7xK90kMH*e0R{mOAD8PogP#Gw`* zgk!noK*96M^$^`xoKUX)T_k=xsBqH5cxnA^xN=J1Wxeky`pZQ%57Yv%Qrt(4sna^} z$a`ut#w4u)@WIXiq0FeWnaJo;c&6F_In)!~YQ2~``Z^joVuSi-^zfkt|I>uujw!ko z3hd6x{RtvztuvsJK!7zqJE^L(pmu`z)S~cBILS#Ij9Db8msNWBs(&J|b5&J)vJ>&J zjSGrHf}#f%76KJN;fKziCT3HL-xMyQmhZv5NiOh%qKPQGrzukvE(Qdl%;G{zLqX5p z>K#=%E}nCn9tv6STJ(FmI!#a8TK+iTJocn^l#DeRJJ z%L<%RLA^sx;9d?>#~6p60bt2$fO}Wlh7VgO4cgqrX5C7~eF-l6&MYMAxtbPe0a`=> zSEo#+ImI>vv?BAW?W(42#*XogpoVWfpv&e5KD12S!HDZEjv_W3cN87BzvlP*Xreni zuIjdE?i4%l1{s@-!e3yie^M^DsK@9cBo-A8xwZ&=?i@%={p1KjkzNQT}eY}h3VG&mRuVdo5{ zhtw#G`nUT_w`hA)^YZm0`;9>NHOJPkHLSDR0ZO%vO=iMwVwwnxt+0YL>MrA3m#r~t;nbe_b%Vcfv-gz&0nM}s#OMr-Otbi9{d#}7 z(sc@JhoAGhX4|1+v~G|(TS(bZAe#}>C5_F2tdGpGqI9Ug;}ue z5P0#p?t8}nn9c4yngm@tnZZ*2wM}eU12lLd*m)EpeG<4H)3S}7!^Y=vy__w^^_()n zGNM13?KCy@J?#&Pd*2fjVDi1pLcm{j1(9ExMb~O3le+_a5zSbzF13 z;BgzS6BZ5Kl_YT+Y;+gTQm zo(~tUkLXK`E!P*{imqZdOTKp~LZJE|^QNXno27cAo-=Y5J9!j)8l;CsW=YL6pxPo< zJmp;XoPlb0-(a(IsWd3uK&>ElJ3wnVC2~-G zvNTJPY@kp|B2$CL$31o8(RkWw^8`y^DE!&N(v|C`c4+O~8Xx_aDG@1obmR$f*^i96 z=JYWbvKU*0TzuNHos1#c$oNeTZa18HP~l|CxZKwQW(xm=iN%3xT+ETPs^t3Z+1-MA zkaDDGd}!20h{BPhR|>uS^?kC`Nc5`q1<~eKatdo0BicDXl^AC*?-BLT`d4?`35m#4 zZ*_`?YBsb41jRNUP(X{hBE4s>LlkHvfHmwYV{+WF|neP{KtIwuwdgML=b%Np$Ee;J1@y`8L3CVQ)o#a7ji zG}^c%<~C`^vK{~uV)strr9}st2{_oeh z1lb(!I0PR|X2abEmR0YAEzm-HGl;$|J1H9o%jA!``mY<#zG=^|VuW=Q#~XmF#;Ui( zazYFWr;*Xh+LpJ^_V-5~*DG*3O5ckp>_k&MkM*8)wli{6z1;hay{Vx>p~=?!s7{2H zjp9R0eJ`@AbK#)a&ZCcRdeN09loztVE@A409Rl}Q=QQ&PT8itVb1Bd zpD^aswx3ZW^zy4z!rQBPySNW#tfqu4y@w`21sjF4e&n)S3MFuZb8FX&;?@VaTx=K_ z_N!2CM}!PSuCt$l*tU;xP{kh^$V>S(?VYr1Z$)#iUGMxP41K>02u9rem~L*jxiFI# zv0-Jt5^!2T zH=^rE`R1)!vpPgN^5}JIZSuXA4joOn)s2?G* zNv!8eL_-m{YIiR2b}+?>S+DvCPxlIi2<6~l7}tUB(?suqQmEzo#WUvT41{{*`{1;X z8N##(1Hf%8z4NjYOYCl-Tqr{4Yiq5Y%hv$Kvo*zSM8CDi{dwmiCa1P#V02pM*brfE z4=Fq~7$UVHWhCLl%&`~(sGYtDM2A10bdH#Z=SfavQi?qAKF)i}c!eXX!^w>L zt=|Ge)t`ZoUyrto)H1{Uf|8={U?U!FLC1&gFd8yHo&RD`Q)1ynsp28T`Vz?KjR|#T z>qTq67bT>o1#ZK4zCP+)Kdu{oJ&=;3FQWMY$Yw=PGWfLeOv%mt@m_lt6FJ5*Z&a!N zJiC$EgXEH!qhTUbhkg{dQq|a)_}Z`MY>Ag^;%!QP*>3vYh+TMd>PFu2>IV~Z^53w~ z4uU2>&i&Dfsw{dkBI{BeuIhs(8zdoaOfW21*fka*qHUkZ_$HxTiJ)nBE|*Td1CCbl z(tv5zBcnmBoUc|+0p$W&t|vR0!=s@d8pVK*l6C#$EIXFYx)>*h+=?!Gx3DPB=zICD zFCL#OtyjK2GDC#oejKZQ0$L)UvZ5ZGdM>w7=eMgY4c6&}Dip+xNyM*;(_j5GvQ^-V z@|Qs7rtSp9>=&unzt7$i{98pIEIam;n_5`*ntG#Ooxe?6q+i)E-p76J33%UMaa>Em zqh3lZzwFL6BzHwOZ%hf?_kAc^5dJjm|0A8$WpV|*95@E10K9E?+LzvsF$G_GrmzTo z&G}x(N$<2{xIV7eY}x|#7@46lbG=u#2)!*TXEJM>E;DFr8+OOBZFkZO0b5Q>8-UH& z$U$VNDW(3&%iSMmX<-^Y|)VCzf0}EV==xiHvAaU|*0}C}=;u6fVU^BW zU?ReT_oKu2Mz5;@Eb|ctK%*?pR(B&Hb0)hXYs{cZS@#JcUi<0qA{Z6;5^!{ipD%f@ z4qr!iU|p{0eGk{5+14->Uy}LJ+aAxm4qP$~e+Pj2ZUVm^1!K9*I1y*v0wbF5RizINLJR5CllxWbBkv~qH)LB zP8}`svN|-3L(ia?Yp#P#Pt(*CzE553FC55>Xn&!K?}MPIVo4G>Sftxq45RX59!TkB1G;r{*M za2vGr{MpIpH1}*r9hk*rxt7;KmQzak}Om z61JPr$9Zk0)Ym8-;R1lyG~j{j{h3agYm|eT?SAq!^@?mG4F%^Tk$vtJC@mbnIi>^e8(}p$S?{E*V)H3qwmLBmearQbVBqBd-dkY8CVV6Jk-5UM#owaV)kp|Q=X2Uyl17P&0 zcN>8?=)Qu4{cvaEM|)F}Qd7_1nUM>@cYv14j$_A@Qt!I(&bMI|hZY{2=kF))m7JG% z!=+}!YeuCwC1y%~ad@H2i(4XUf(h5@zuH3V0NJu=*-!uW&j8=~)FBk1Oq-8nRt9eP zrSpCD{sLtRnBh9nBZc;h_C0687vr$yk_n78TONHR0(87rVRsl1P;me66Q1L@>a}bq zK3>PJew&u&ghd+n?ziBwrlgfKNB%8mFN6DLK0Us7eY~QXL0pee_dO4;HmiKU?Rn!~ zHP9Y#+TFW~GT4+f#sH*a`>_p*vGHF3&pp8kJ7N zcY&ym0|UnWnIaa^0bE!YoCGCbZ&jXzYR&x@T*^F9c(0HS<1$<<2lFhvBF6dEjl!f= zk*6J_j~;T)WuqYvlc72qU20sEjxU+V#!5OstQ%l<3C{UT%o72Cmz{%CJ*B5$h911& z%j$_IU_2@3SF=`x@pLZFEPzuOeO*3;Ng&P2Se0m$Fh{h1thHx+&#mIrjWKN!nb@W023umDCc_#HM zrQsRi%j$_IU_9w3-+xzLeNFu1cx`_ zZ?eZ}T=Zp?NRU^oZboczZE$e9xY_=aR$SZVdaByN&K`?tF>^P3uVEb^Qtw=RDuhR; z>U>H$$DVNv-Yq{)+WM4yolo2^ERR=*{TF)tD~|>CkYU`9I4TifQ{js{%fJo#AVJe#DEBvB>DuL+Cx3nGowxFsCGLVH?6KycV`b}J`FC%VNg*nyLggf? zu};h4zx#(AqDuhV^r9)eDmNtDxl8B6PdN1O;}1?SF;=Et8pZiPKJ&b8U2ts+op8%c z+yLe2CyPr$sPEsupJ8?HJZ4j37#FwI4A{hjX4MSXTxJwhmxb>x!u=ikF2{K%C)Qy!`Jh75mZ(*- zc#;s*hf$evE^QFYBVQ`9w*0i_c!A}f3&cR5jKU%tm@KSCS+QC~5>SW7KEfsENm3<4 zqLf!1zW@;6OBmMV$=PHA-f~)*RF*lhCccf4v?dIBVwR*y*^)qJ2r2DmF6SW<#|nV` z>>I;@S&kCq%edNvpiC%OcUZw-T}TLxlt~Ec1rxuG*ogr^c~%TH^L7g+AOKn(QB zC@ivp$--Kc6{|%g0d@EW<4RH`GEp`eJ%NFWJUN?8aACyC{1Zouti&q;9IPZ;A<-uM z7D<`-1O}zmBy!?nH*-0UC^mu{0BnW;%Ta=Sb1G#*!Mei>4AzB&&`6ntpk6RA2&o*J z5iF1JoJyE1Kdm{Q8m`5V2(l)lu&@Ru3u{qUtQL_3)ZwB3zxQ91D5+12(?om%gVJge zIdQR@xtvd{*a)rwd?xR|dE90-yaOoL&H9vZ6;C$dKFlvI`JuV|WyaObBCTxWYze=1XD)E_h3oIS=Ef_3^XS7* zj(>hUo}Xd|GJNgfTaVg&pTqXx7ciRIcj~ayUZeDPZw0jik;@HAZv<#DcBs^|s(wv^ z(zz9j;a|~FHz<4c>4~DL)!TLKFnZ4&Zu?E~y9POS?bZd4nHVy3(B|8Gb=y6*9k@Jx zOYe-n=3$Q$Pt*ryhp}P2+3tA7-q^NHuYSEQ`sw#Z@4nro=UwsoxHoX26o=-5wQ19_ zQ^)ncydJE(>^&Nr8gFS({?Y&#y+>HFDEob5uhYJ97FB8nY+~R$?lm$WI4(En(Ywd8 z%l2D){k7KLc-^gc*t}D>E}*h0`3ks^;$#r2glBfZ+M{Q8QyYW+evKVz_%k_pY3s?mfFP(A-(GnRAz}ok#5%9tA?*Fg*W4F!kjt8fL*> z!kdsrERBoIeiS9W!P_=;bJ(F{#{q)|sPfE*JX%B9uYYkHIj;|?8RpOt8d2AtJ@8K? z6mmBTK-Q1NG4twMz<4VYtq1) z@FWb5zr1_fy{que#$}_2B$c|~f2(~x1}p!{;KuVbj5&jydaU!TirB_E zsb_&<1_qg@ln;mSiVVGT2_JKImMANK8!KmXQSh&Pcq%eCIUCVZ23c0;Tf;_7DRaXV z^gjQOKlCN-6Ffph>~yl5fLp1TL}_ytJ(Z1~MrU!E~} z;=7C9?bB!36;@t#;9!5l@>o^vYqnylIjr~<-lW&ZO?dT{u3ftgTVw71xVenUup~1o zI{f30eZkwJ2%JHXk9+2+sZ-v>Z)1n9vc@XI*K$}>L%LcWjQQ+WdWFnIQ6*G`N^#_| zNXpP-p^SB&mD6xB&Tw(MSgQh)A*rzI;OkW`c8rcYKuON=XaM#(uHNSiBIvlM|2bvy z1lQ-Q4i9CQC%D?QnKx%Pn*6P|roXp%akuW>hO8W$JkLCiE(UV*D>5}X?sJ4DcYT8P zbN~CHXKW<%DY!chsL%WK={scD8lg+0my6;Z7@AE^U+5vcIsT>Tlioy=_wBpf;9;u` zTw!obHss)wa5Q0&+3&w-^2zw!?`z|5qiHmG-5$z_bY&)XPlpvpm1$GnM738Owz{fK zyaI-_VUmIb%xXf02G)ef1@J{vESr*|(^~l2xD08bnZ|{HvON0^<-M@- zuTH_Pc2c&W$p3n85nR-1NEJn%I(Y(*B&&}2oSZlc((=$fckWDdb(gMPap)er@+vs| z!kTC0H2$_1crY1g8H;Qmef;qoFFiYT@EZ zpsP6|LlDHEFXyA^1XW)+#52)^=#{loYEHD})Z_8r>w?c3K|oyoE>&xLQ#e|PcX z&Ye2<>>GacVe`kMM)50#X_Kb4>(pVyTC2tze==+!vv?YkOc3^+x%0=5pMW2P^zPMb z;J{_O^~RZkBGp(Q=N*q8MYe{mBBG+tT(D}4UigN&^@agGi>HXm3l_ugaCkb&+_~hYW$nCp!T5h?1xHSn* zJ$K{3v^237+!`pKo17idSYJoTb|-ZN2}4DwfWkwFNa<_^a(s!W#y3!vYvq*8ux;B-2gUEn zCTGhG`60A)#23D#&f-!XvaPZx|2O_QCt_|K`d|Z~Nsnb7syEPoI9vt-r|@$9(T%+{qQulMvL* zoI3TY3%>KnJ-2~Di9x{a)<*7o&_0JBjVCB8E<14VFRwLo>SSQ>faGtTbpbvi|KaCX z;Q7JRCcbGIMy$8Nk!N1G<`*__N(RFiA^TZVKC`AxyWz5nUVZ-Q3FBY3Kl|^}y~ir6 zuete7yY6=IHvtx2A)wjkD{G_R(6HdIJpVKb{^s~sgxM4Yzw-JA@BG6hCmxAW{Oi5N z{KQDM;&hDv8vm^`&mXnlA$x888|+*$6vU|f3lUDh}GB|Un& z55{>Ns8OG*$EatY?{)amo$$CT)0st+A9BPIzRCZ1-Icfe;-^#JoDd_sS2j6rlMGpN zVF8SSCcpki7mk1KDKxp=(Yp5NIc&8No9(#E?uQ)VE-Ut);-T-}=BR=_oktZ^n^x4* zHThLHJb1_N)#OlO0U0o8a?04m)w{0A_dVEVNI784F!fRx+OYvd#BQhoraX`p&Lb3={EH(?RY$w4&86Q z>W(&cbZ8_Vx_jZhK4#*AQO4VZKWZ`789%=I1K-ujp_^TO?0J{;=#~D$S`|Q3U3vZ) zICQI}nF;Ryy2CzW_xjpVcv_r2g2B4im)0x}-Dh9u58apFa@|j>58deMm@}FEAY1`h z=UG#yqN`th{+S8mU$R}TL$~W{&WQ5?S@1JTt#!7g|VJSn2M8`(6d$kZ%9{NOnsB=em`ihrk@DZ)> zZRdVSzRA1p=?8gqtD#)_v55@~e=Ig6HC*RJ7D?ISt!#a%ukiJ%JRh$i+|R4`Z`Y|K zZesz_&wHu;tOjQm@XuyKnVh)F1AjKhKhIN;YVJ*+jr_TU|@fed7h$AOoTR-%|N`3Vn!6IXG!X0{Cc8)7xxhsguWVK zx>Okm%f)LGI9O3BdWSBZ`geA-qpV=!6%EWvE)v8q4Sc)w?2d=UNE3uCEv8}A)pdir z&#;t2=3C^P#5KMZ{CWPlCLvTlp~RWFl&H6|!qjtZHCE2ZqJjgCTg9slMxw$NB&glU za|Yu}*9roUq26Xr>JqbYmcFRCvDySf3N==XWD}G$b|W?f52G9})+Em9&Ctu0@rEsT z9P{c6PlLYy3PbV07aJ#)J>K$bL=IUBle*^HiLV1hYKI(?Yy~fJ$UQy#y$7X^RBsR=&GwjuL=%@B02KKl$Nf=SQAnZ`S$q#t_~q4<=Rg0!cNff=Q}nqv zpvm!yL}tY%ANTyT&>$=2kIk2#{(ES0-XYZ<%{&iQX!46sI%?s3_k`@I3$Ke8zJm^b z;i)Gcx%X}~Ij=NvHce#tsPfzsk0wnH?4;LTL85GOxr!k99ytER=SIbz7lvN!E66sw z8!t3Ab?fHPVF{2LYec!%79gxr1SvWdQC;xa)I$Hz=nYu;*8*QDvD1Lmp&Ko29of?G zg_2!e$pa<3`mcBWG41M@x?;UGWBR1m#`DmfjE;xy@y|bnL-&fStQK91m}u#XPdoyb zgcPb+Se&p>OQU+$mt75)w`R}4DU8ZPH>~LD@>b7lPfBrchTkKpQguA>Y9;{#*|KhWy zLwC$6@q$`8IEXisCm^v=-oYbN@tSke<{pT_XSL!xE=E}OtCLc!f->x0?&QaR?c!#= zV&RA63PTgBgi@OcC-qnhKyGd3c!6O`taW+OMUHha=>xI}b|CGNZ%QKpcq`i=R&Qyo z42)pb*J4_b4fsiwXipkq;!-0I9gvUHWE{A#G!xggn`- z%r?Q)Ws>_=A}hz9gs9t02(?+_q!w$LNtbNa#b{Db{ya*ZyO5<^tI$cF`KB}yKx1Va z!s@G{IUO*9S>HN*rCtoQ*74X_3|+}4<;j=)dn10wBz?k@>yvy0SSkUd z$On>Zz}YyTW|ARoW1hyzS7zJTMb_N6GUv*%i=?rTe2x)+!U@={X_L-6eE;NJ2K2bX z93g;z`tgTTY#F~pU`9*0gd}qNAPdWD8g$tyS5S^C1`N|8&j@cnOmmwI0DQ~=f z+CjVHVrC|+Y$s260eXoXA9IR5Oz55Navj)f-q3i<5c9y(;(jmG(Nf>I^|3vRzulZOTG(HB<; zhQ-E$%BJ13x4X=<$;H*73Mzf2waTNB4wI$%5-!)kq_~1npuU-8NSoA0Y?b`3>|Z}X zW%teNXn5I+ZSsQ0bQhIu@TG>Ol zNRXIv!=d{-hwaO*7Jxih>(DKe^`#P-psNopAG-5t#W4h3tz(D?ldgt7nGtOwv^{jQ zC6e(v_eh=OKqRe+DSCWHui@$p%>lylh{pTS+AQovhAafVa=c|Dg9?}MTOK0mi?yqc zH|J~->0a%z-OI%5?Vm4TRjEpx3IxIA7V2aAHbi0Fz~yGw3@O&r z(GY4!jX^@S&EV?#YAN<#kz$y9{x<5s%v=#&EtCD1#Rx_~m@U^F zKuFiV_;RBV)H#asAVnuNG{iF>>8&fh9H$Rlk%NyL=Zi@c_2YQq6=ovP6m=kT!Vh>6 z*Z4YET&!An=j9h#L#-IRNQy&=P^6^ymmayv4qkZ}E{2~oeda$NexO6=@QZ`_ z^W2a1xC~uw#CBu$8L`g#%MBj-@rNHwdSl!T|8vo}=fm|5xZ}VtFT42br=J_)p>7E7 z95`g? zk>^~v!B@9x-?8KD88aTg|IS;kzk1@UFSqa9F%;RyzzXBSg3xf}&j*+oo*J;rLEpe* zUtW3YpO4*t7fehK7a?DM&guBuy9>l>d&8rDy7SiSuZEH^B)09Y`{1Jep{oy{F@4Io zfBpj}LGXt=4sFpp3obwBj352(PKSqzKE2ak;f=Gjgv3DL`hF!dBKe0FR$?fXb7gI1uyw z*8&}FT&y0lHYETES)-<2R_Q;T{n!5#-=~HM+N+A=UP+5zOHFz+jCYx3)aNLB=dO4t zSlb2j=VHydLX+>X*S;gx`9i-zLq2@}UsK*3fBg@?KmLW#h|J_89|@1${F4 z4fhLiE`x@ywCh3N=s$Sy%m4hxqxapZ{{QfU53V@(+h5&wM|aZ%GhdI~e+Qa8db_UM zde?nE_r(oYT5XMK(-$9dWPK#Y?H9N&<{O zHu>mds|Ht*6RxJ5vNOj>7ZK6&xOh2@ea*_6H{G#nclT9HgCM0umBth|d& zt(BuIhi+!e1|{AOKe1XhN_TEQ9-4+2O}E==>~W_ZH)gw+pLM^n!h#R1PyXfJjkYMRTgGp; ze|AA}f(Cb#{LbY+*M-gNv}v>L&U@~C#PKKZzmx9B@Z1v*|LxEBerb#C>V}U%DE>$k z{CgL7=+p`FM<0BEKWswZum8YvJ=bwC>fYDQ(+sv(p$f`3d{&q$a=Rnk(?dFPSpi=jWo& z*Itk0?DNt4AM)g%?^%AuVNlW$NmdqW&#xVI`03~3PK`j~_g`a=J@wcYQwr(9LbA{>{OgLN%Y2n-`w1 z@!xz$?KKv+uE3E5VNhfRQ&%5&{3+;a=^+dmwki+Z=xXZgx>{ZgI`;U}4>;N#y0Ns` zY0M_u@64{I6^Cw~&d@a3md2rbjkVX~F!5t09J)vEckn;{df%X-<~DPRX9SYFuKxCS zJ9Ov(1{}H%JoZ!^y7lA1aOmEAJNfC*O?PzlnO9tgt|mZ-?vwW0kzI|F>d-9`2yiEZ zl;~<#@Xno)i92-rxxk@&>~W`J;5e<*psV@L#UcQ#LW#Vw0LP_uX+(&hnebO9jX)K) z_NWNd>O#oW5KBEZTF-yvVfQN+ApMOx>^(?x9;+dz<+rkpSn?`u5S6a|Ai`YuNes0$ zdfX?-{XHS>ZjQlQgT^3}3OFZ+E-yawkQsgX_k0_@?2&Ah@}ZZlhLyc%PypGe9vqSh zWq6b=3+2|P0^vSN<@ylQpl%hFVHguPBy2TSX0ldOC$kQmN8}zy$$;TKyRJA^&oSox zKz=`uWK)P3D!$q=TIOhXuN+04HAhlYu0Lst`dcgUJ_>`V=aDx5*~$axhQoDedXJ0e z>7CVPOQhBKh)}zHylZ6h+gxRMhO)UCbQGDI-%7>;ol6Yse7;Qg!p*mN8NOYfw-Ku#N%(vX}|oUVdCR>+nf5f6S8$mvXr5EaW1h~ zAl!e>UeSP?Zn{Y^Bf`0fjkhkoo3S#a0*|AL%hBilVAq4bZhbp<>2m&cHy^mg7jca} zlf3m8*KVZG#BJN&ck{1s=|?1J^WCd&T>mSZaZ^3B+`xgC-F(m3EjOGy+uipO5wE@I zTz+m=^hDTX`<+hk1t*m)I5ogC9|&ADij^RR+djPGSx^_MP=f68(i!b8WQslqmm%qTb zSywW@-1oa*qRB;!CSPykO=)s<@W&IJQel(hrUpc^$>kOBW_SDQb~_$_(T|d9hng6e z97)AlhLW3Z$iAZvZ0Up z?!~)|J>2nlE%3OH>3P1Zbw49}=mzXhH~$Kket1e3{d?E^Mu%=iTz0?!bv2dOTzJk# zn{P`mA1Ynpy%Gg?P3V)xF~ofwj1De;zm(kA?JRFZ8Yho6m9f5nYmZ;TkYie>X(oE>n2{>8 zMTlk=;hO;qQ&*6>an&Uwo&>;6n;T`v@BUXJCBfoLhMEmUud!FFZ4@gjDG9lMZh|IP zI!79+Z8>ps8V#7j(vmZ!#-WxLQ-@coDus<(=VdWskYKH!L~bT}>-do>vqgwz7U7!# z3sYB+x^dMdBc24nO`Dr!hDh*COc>?)?+|PhD=Q@ld35u6DHu7N90?7;xV`)-KfIhD zdMpNUx!^-51qMynR$Ke?5!mLPx92W;_Z>M>u=yN((s%GRPk0GC?By@P_1o{vpNGe0 z$%MZ;;xD(_{Of(ULr!v#)d~(1zcoQP;OLXwxs39H4DmpguWmCMYGAND{n%dx0mD5v z{ZgC5Kfj*+KukTy*rRvfZFk>C$X8!_elebdTeIL16_o`~;LImcekz;DnQ3Ey|GtGG za(*{2xwb|Lpm-85)W=!P=WaW$JPON>?;#?zS=e(KR!?x=qOUZJgk#O)?zH=INjH)*Q0 zYM8Q8pi3lJ>erP{J7c9&zO8>lPd;7$n&+zhu?Z{15~ykoc*FG?dC4*jNi4smWyGZP zu7ZB%IDMovvI<);8I0k<8V?iqx%b(YcY&oaVkY0RFuw%4_3aB+3Ip$oEu;(wP)z&Z@OK{Kh%n*ks zkP#D7tE)hs4?B=48y+(TP&-4D%xNZh4NY#L!e$mqahaRNm(1CQ!H0?w(vZKjzcj>$ z3YaK){YyES%!`Q@!dpY0tT1bAv$e8u4A(=_T8Xh>$;lKJIb{@ip8(?U9K0Cog^ZGe z3(b`KmH1=flp)!2PHC!h3jCmV7W>>#tmYTPrcqz#bye7uhGz&RG=as zJ0m8RbB{}sk5ik}=@b-^p~&AL17DmS@kFO!FJ9v4h%EGVQpB?wqLq;NAvG+yHP^t= zj>z%YQB*n@3}=DioG%yS1(EzjMns#Fe(=BDvj^D-NdiM7zaVt$-V@jH>IP}=%zvj# z*Y3(3JV;5o?7ZK>qN1LggN6(R1~Qdz&7QUV;30~%>WFooe(W#YyzrRA7tf!Mb0Yk~ zh%gjFtQZ5YQEapOA?_)v9G3&f+CsDO_B-7DyBmna6H*txGat{3CE=7eCx~IU!@fmj zX&r}$kM+F3gjQT~yvp#kgaj%A+njLGWhk08K~@y$DbO+a8W2P~7QL4wdPh?3SnEZr z)A)t-jAcRm$FA(-rxUPJ*yuy9;;L~#{~NB{`c4$jptz*`r9SisWoj4Qw#m(!WL5kjhj&;vuQ3ST;K z9YV&?(G=BjJHAqHqvV>1b+08J;|8(FC(*NE;2g*3hz!PU`UQvXZryt*1R)jqh!9?e z2Zw#Qv#X&lx;n-MmDteLl*PdOm-mP6U~&&4@rUlnz(Y5z;K!kR(Y$%v?{y#!-C+QD ziF=3B*gOk_J9HOY5zk3gSA#z}bfY=xh3LB;dPKqCB~ykr>S~Sxx3&p~ZjK^)!lk0C z<49dK{Gw8DQ-L6`VpvNFjFstvb%LzK2yCs4PL0h(n4xE%MMsMil?z=n3l>v*I>Ul$ zWnPX^9KmXV{|N)ra`mrT>XLSIT7{Pm);dmL8koe;q>yCNIBt`i4PfHu@a;g7OmY(W zTCW_oR2orLMwhiQC&wa;^32C541~s3i~Kjq$}{KBb8B3R9A9u&c@ke>{&P~J%;OxT z*;V#R3i?d8lT0v}s%n2~sfa(XT9Qc~Qof&JhN7yBE^D5Q)o?7*D9?P1!a!(jwa9;y ztUPo6Jh#TB$nlml_a)w9{!?<}%fbng?e5f0QlVvNh>RZJBf7Z z+D)f*rcHXop3bn@*n*3Sb+9nVcA65C-Q-xLsT`xM5^M#e{453KnFq~tQ=xJ&G&$bF zSVmiR;w{F?Z1OoXr{Vd&Y;ru%@RKw-$<^fCWRu&>oA2|=Zm~lEZBj8dIWZ^*?to~U zT)VBB+_FioXz~%Cv&H283%sqi+&{UXaoM_&w)vVQmH8xDiK;TXtc^q10s^VI$j4}T z09&de|7-E$IkRWIJ!|^x*)v>M`IDlG}*%5Je3*4ApQoK(Y_J{!68 zP%#dY2idv-GcEVa7T$dnU5-Ii{`{oj2rPdd(P}7TXA)mL5#|(L4rR2Q0w;o}2~>gE zBjU*{M@bjUX7j1jM-03}FEO_=p4`Pc$I#ZoxGX6%_72gv=qm#Knh6#rC1Ef@XYirI z#8`f-v3OJAP++5pF|Bjd>D5%SVtxauN84E=rSUDzGOaeOmkHO5gEV*IQ3NW}aM<}u}oCa4fTK-t`Vo+8SWyi&U zF|0JIf5A`}IgcBVu~_PFJEE9_R zMx(PSkjzpF?_gPm*TA>|IRkM@3ZIO#|&9{7^7p& z`C@qBwoMU6JWX*I%Q)g6b2uOM$nMY81)_}ze_|sl3jDgIKOL&L! zT-Hi*t1025L0Lsb5wfu+B%jR)y-Gtfn3B~503Hh!ICRSbA+JepEs{18V2c)^tADGm zHpVJ3=ILy3_DhOGmgSwhdyk$Khi;GIq5H0zt``Yg`llD184lgUR*8FyAccVVK3cc} zD@XQ}l&^^U>HI*P?S9_{bOG<;Tp8ST}5wJ|3G?7BG7XZu!gaOd-MKE|3z*u3a z1OtXhYU=cX4d$4J^M6b#*IZ}i)VV!Hp)2e%Ch?T<9pUX%HTHI8LTSPVlfTjTx zS|RC%#(0FPWD|*mx}+?>s3ce~A}!upi@eJSc@eC9$|z@qCSQc7Nf-%2L`4^*emkr*5V@umQSgC-pE)Xh?br#xyG7%;(JAU=^ zgSYzPudetZp9})4olsybPiZ3>fagO2#N4rCXUG80nl>e(+^r3|;zu-wpBMq0;6Gow z%*t_zsneGj${XUzXN|zpV13T{7;{a?C%#D~B;^!Z-U5Gv5b_s>cQxiRkXS~DDx6$bR$V6 zCh+GWkxp4zO0{g_mcsbk=vXp1UB_vV(8NF6K*@P3u z!?B$_Y4|9Ubo4a|TLwa`MhAU)2WvE-WO_DRxl%ZOx_Ww`wSd4~XzKq21fxs=&p<}c z!(f0JJ@Agl--=8m^mc1I3a>52CC;!>5!Os9<_{%5CWZF!RvVF_uyz3VJX|3BMg_li zZvPofQHk{HPzV29T}BD~Ih)Z03<~@UNb0-;gYZaE`>$oEJJxm-RL0B;xY*Hjj&bI01cR*SNN0PplZE^ZyiM8%)8Nh6Gp5pxb3#S8gE-B@9+eJXkorgJdK+5MLh zOFVJB41-LFSN=tKFDYjRapN_0AlULlc6F%A6M7<5f3OvQU5eJyJTd@+DlYkB4o=Va z1xAbs;WhApHj+eR_$yXf9k!Gqx52M^f~PcgGsa4)N@a<`Bcy@A!CA*(jRxfdj2)Ni z0AQlG`4`6U=PVb>91wiu0_5h4TS&vbTln`2%kLfBYS#4WC+t0H=G5@W65z26T6x%V z0|)h8w*NBi+^=XKzWde>%>x4E%Wk0yls=-{3IJwMpq)gp8=9E~0`Uv#ZFb%B=AU16 z=dXS~XVwh*0gs=b{Pest7cY3{uv5+=5^L6tv0|LscXNDQB!>8oUAj={(Y z@4xq6$IkA1Cizh`$l*ToK3kL$j?{O@Z5p5?ksnvU$H!`i6Cyg zDs03713lvg7E{&GKYWAtNK%l;@>3R1q<61HT!LZ_tUmyuKJe?g=}Ua2joCA19KYAd8B-=%Et(t;U+lZwfOhTNr*||tI!%DU%u$>9 z^?+gsssn&_7UAv&uw)63lQIIo+ri)1dgn2JyzZxW-gq6KmBXOe#jd{Koc9*Kd+14L zT0Cg5;B%CeN)m!9fZrN$6JQ8T zGuYJU=K}8~2En?h+h_Iu>w@~sxsIQQIV@KIYqfc@p2Vwr;l}-IsosAX!pCw)64rtI zkX`@8CV~}uEC>WWkxUULDA5gX&6vSMx3l^M!U`)7>)U@IZlZ~<2H!(>-D11i=Fb40 z3_vX5m>l}S$EAU5MzI>n`Jo%MNFM$6)AP<+{LaEdPW+C*c8rCxM#AsHXknf@ba(4o z+)@NBtwXnH!nlyMSD~ws53Sw1xzpBM+&uaPz=(Ag&2PT{f;o@}j`(8yC6XTqxBo_< zeNCurT*1U94F&2sF3OfI_o9_++y6Q8lifRm+2IZv|NaX!l8V%9qN=S}D(9;)PZB8E zZ0ws@^XRz-E>NaXuuW)cNFubbDQ5BI4#LpHusO_XYS}DTbD3KY8=}>6gRl)ujl(rl z6U$InpT<0OUSw`f@(meVB(wl4+mSXew~nE4#hQ=a8e|?j*MJwwd~DDVoST|YMa*K$ zeTAWkL35bZG|ARX=1FuLrFC+HpbbooV>MF~%TQOJ#yoXiWNuyZjX7=3xojg^IY&05 zX&po3idDu%pv%rUA^dIvd)AUAdw=7Yoen&FzzT!y$FH3}zf$GJ#+EbxNK}~*VD=*y z8IT6SJ^J)J__)&!IO?Rw9=PkqA7A{+bI&NmuYPjLrlWUR<8$julL+yPfIB`Uq36t+ zNid@?KX^rMF2B+$mSoObv+W%{XtHd$AA#_LS8^9o!x~^U-_oPqy;{80w^~;8oS}g! z2Y(Wct&inNrofsY*Le}vy5t+ofNgSwK$Gu0_UruztpFJ<;ZkkecBgfQe(=8gF%GWHwE5p1V*9+*FOP8|dE2*k!qFpX=QSG6Y&-N&CE z4&DEI$t%94$%&T!>JB@#*wQ&C+~Y$&g_3 z&BhhW@z5lJlFdq&&qi6gcLu8;&gW(EX+;l}9tq1kXq2BBOIP8=WcevujFnxt`eP7k z9r8(!`f6r39sEo{NIZW7we5J|@%M1Ys(eJ@poZV;iFe%Ye|_kPNEfI%P?_%3q1*ZR)LSs#ES^9x(vzSkysUV=17%4 zLNh8rE)ot&;Pik0NwaY!2M-o!@yve9!3%v*`2du&bvO;pOlYNS`ovc?cv`%w0b58s z3J#ssGt8rBQp(B5EWZc(`Gk~@i)Ks8ITAGhnY96r=d;dmPbe7DugPp?JSaGatRdGd zekePbs69ySeRs<`@@TnL1(`fVLxJphTnA6$Om69rLIHYlMUZ6#u$SUy1Nu}N~ zJ12+asIiF&mfjmZpr-zgC&On+iN=K^f2bQRrv5?CJbik?7qyK699zL1#;$xgBHA6y z^pBic%*XQ6%T|~nV>-^BSfNoqbT8IsE~I46SX)k7$!zu1kUzwYoX({zlM}-sK3HSm zv_7$)VHe(JMIHpT`B<6xV2uTOEI7{aPfXy%W{m7AhUJC7KPJuBPCIMgZ=S&UDi>6c zvfxLGJh6ZoY{6{9c$G{(pJm#ib+bgXw<`buKmbWZK~yS<8c!eGbi1AK@8)YRzx>?O z^j9@JsE?;J$fme*41$8i@xJi%6VNd!T$$Xv-*PO(as!ud-=PC9JON`B3A%A-aV;6x=%2Up!6|>n5x7EHB{E{#aOI z{hfv+qB8h6Di9Spt?FFi=en!1x!QD!%dhvAZM@ z3v=S`Ci6O)oT0HEv{E>o5n}n2h<9+f#~Kzr1~;=PTccx1;`~|zWd-x$prfKhRoZZF z#wpgVy#E3~C+kX?zM&4Wyv&931=&q>%!iw*o41s#(nxSTev!rrrv8+}+6O{zE<`dg z@9&Z^hi*XJq5Fsvp@zxL1-sg9yM?b!q3_KoRg-cX8F3_s0 zA-g)n#YYU8Ko0nFR=~4pAx|I6=B#WTxi?_fWgu#><`R%GPp9;3Mck}V!85*)LF4iS zt~@p$Vu_@lYpV&FL|VMM4)}ByKIP=o;VQAXqE;_%PNStxc4ab)@J82ym@o)!BT|`b z1g;dC$20Bb@p(QNg&hb9E3=|9!3&u_AA^AZUUYGI6w=z{o3saT9B+uag-|%o1!4)>ryZOGxq* zErTi_3Bo5?NiT-RLM2{NtZ$`KfJz=&>x9Ofbzaolh*p-h(%mBK#8L+soiWEu3XM|D={8M2hInT~^9@5Ni z#7R-9G5!fj9O?x|H73gO74dF*IT7ey)M~#th-~bIR1ye7fRD&W%%7JNw7r?NI`J>|fpTUiGy75gQ99W!_92S$kv?<-#$s+3D z##9BTMfp`JbB~s!xObIBVbdZRnQg!}N zY4171clDfGRmj^ia$9DWZCVv?W0=C6pVJTF7}kC-8G;SU>PW* zq9+^)GoV)`H8~(@QtTO^3fX_lKE)sIyO1LGFD{dg*{S_9mWF1y?U`}^kK$x8nt(t{ zl5jm;gKffFkltTIq2KOzUK=WhYRu?;0|r1vcJ5-i!GFuB6nYeSN*Yv!#noM?sQ{}t zNz_`9(@RUX{~gr8 zT4%x`+@ZWwstg+xgQU6qoqoYx+h7X@KU#$`#a%4$QmjLN3K8xrg}A(l#E`G^%tGJ* z<-$0MO%(Mak;Xd1R+Q^FklLzx`EZ~Nwo}^5Nv;s5BB#TCHSIsMP=17?!XqS zwU5tSu$$bdG5yPa2cbgDEEGhcHbL-@3Nb?m!b1cAgDaB@{BKlLg1v-vAluXC7>|Q5 z4*6hWi7n5^Bp>cO-2>D5S>yG#89OzH&8WoJ1fS0D$>kxozxiA1#~VqNF>x!vq%p3i%&ycm+eFzF_fTc($vb+)eXu6l7K&&|$ zNML-~*_~x#n)LB=lzsId8^FIJ)P}Z2Bq%McoGE4Q?hjyDn4(f5@>>foBEqZWr7BaiI;6dfKvWTB~ZH@}_%~_XM-kEVjl*aQ+<;#U(N>iwp0aicg`t(bJx+REKvCyPu9wzrp}Hc%n*;>#A4S` zIIPqfgk|c*rqx9_J`V<0s`Hs>N+|;(h8da64gqi0qFx9nj#w3ka+!=*4M=tcp?Fq9 zQs{wxcg~8O==Z3E$nH$5#T04e5Fq!}x6-!uAn8h;zlI(=sp(M6(2dyLiiD_dJ@i+~ z6qw2)%1B_%O_cmEQD#E(Qk|sNVu6>_np8PPvD6+$3c5p#vgaj>lMFnn-MO(m zf4tTopM{B4!K$$zyOIcgfF`9$VS6H6g``NKl^cSZc9?bsMA!{;UJqOa_iba%AN#!| zOJ-Y3OZlT5lw8-~O1fXUw`qUQW4J$V;@+)g=v&W+&ZRNiL#j@tD%OYsn9l=$57~Y0 z@ABX>7{7biyb)8^FN==e%!1)^+I?^8Yd8(8B}y(0_yp5aIxPx^-9n0e+Nr3zf#qXLi9vQ zuO56@C$PMJ4QIzQzLrn7w>z2kir$m(oUF_ofKpyrER?MMBEn#FOQ{-Mwq$1O@r6i3 zQge^M_wiSp`A&>I29e5JFs7?|`K*6!IJD4OSd?YHGWXhVyxKcZQA;mSTw5ZJ1GX+L zypH<|vq=zwmj;C0Rw(l8X$Cs={Hz+6iXZ$n*Ta)R%S&@@F}Ys8Nex(bc`>OgX{?}Q zP4S<2^|btIUxE}y@t(Jj8h2bT2mq6gZN$H9DxL6r&pxM4gp_9ed|$kZ8*vUSax&LG zB(D1w+T;XH0tlQthc?3*iDX*dnt44nEdMq$x#thxW{uk3kFEDHB~}pQ-Gessy)NCp zJ(lV~&twV2D=oW}AO@SRB(}IV$~Dvk#23OyP-Bq05%K_f2e)EB$Ks)ES!!{h4H`Z3 zB-SiiYHP0}rOyI~3Qa-kupMSO-oI}LmPnAd>SP|xT;$^a%g8-ut6Xv0eJyinwSM}vr~ zsF#Eb&FS`kPtiUIf=g^Og}9A%1;@&J1O;+O?U@;+F1vHXk`RAS@*kFrv*9I_ zkdcYA2{wr{c-2x13u+P-a)%+j5Ekh&%V)`T!C)R!W07}|gb9s>LvhJzA-M-R!O(Z% zi9XNfP(1>21brl%i=%^<(YnxG6I{}~^yJAyB;%X~ovoIcP^YkQ?!uKNIe&46L+ zreu0Pyhor|aho^8@_gN-4dZ&hOs!vkX?=VOeH^;|dxV-`j%WK*ELQt)pXidh!XgrdwZWhuYsH!Hx60F~9tM@)J4r>D$f zl5O`9(7eUyezDx&c0u7SWka8ZyHsPaYbZe)U6wGS&>6!G>wFKv^?CS_AGx1I z=Z)$a-Ki>sIufldAKEofHCYY4bNig8`H&6Wq;r$@Fune~O>dI7 z*GO`>tjZ|Ob7_vU%9`@ zznwtcyTniiy?eNALE;#r>w#;JladYwOmpn_$8i4_S0Ojkeq+UT_f+2PVqjLXzkpje(X%y zNJRtJX6`iWKC6D?_ZZM*-nMgcS2JAh&oLjl@3|b-`?(yjl|;H_4Z;5oD?$SyY}iyC z5UD;t5Mw%T4)!@6Lz1oQ14FJ*v)zByhDxfdvxx$c!fxina7u<;HGfp5q{W<@`&?@u z@0l3hjP(m&d(2d_XbK=WJsvNu1O2FrpWE+`%~FL_Qba`l{K?C#=+!kr)cPC(&;WyU zo}a<;s2UBdu{!3LH)Gf1!fY;SbxEV|vop7o4YEGcmQ6`c@UFP$HaRIv$gqjT&!3f= z<#2qvScFf9lw`;#VR)_8xO1dCQs-4l!*Z}vVH%Z-`3sM|L@nDKf3ylm!wqbM7fl2v z581!shbAPmtJA?N_p9hn)3p}a@&ctM1c~RXxQlV$l^5hP(S?b3Rl+e6!~+z{!73jh zdru}$G$t6r3gM6p8{LinMvV}wssaq7$xQM|aI2y9yr^*MuqwCS^?HBj# z$AZ&`IKx-aiZ%jMhW?nyQ6?o($#W_y*Qs&-hm!S|pEgI6ymL40NqB2ZO@y;sf zAAFpqn>v?ip$fRLjqtX9wisp5KfLlq1`Sg|>fSzXp|p=zS1WM!um;x=`v&*(aades zfwUJ**eHsG;Xwr)Z~G~;54d{Uua5hePxh~{0q>2`edbW`KZo0B!S6`)8$SfbS4nA? zu2q##xp%CFx3zN(FYL>D8SZGG2`ojplOPV?E0>AUNTcQdB6`ohUwK*#IZ~j~c!KGC z+)NK4thecGSytQNM9rj=$N|Hq#)`-AM{nz0gV6Ahew4!f*d0Oj?Vcw}Z-k21s^gb{ zB`-Boe{23UGM0-#UOp*XkB16@vZQL9fEZBCf{OJF{jo0@rUNx|Ag)Gk%bW(W%xCbH zY$d|gaT5XbRyPd-+r9My{Rg}Cd4-9)8K+X6P)R-M-ofV^GaU=ayk&@FYIB7|CHv<+ z{5b1zeGn$mqnYr1r~aV;F2^UEz+tozM>RM0s|02ah}r%cq*j3=yUEIqV`zNZ&V!|1 zDqS;kFL$U8A_4q|_eezTlwo_&$B0MzD%d{%m$J~aQcMxTvXq&LU}&%iX7dEvkqLIQ zqS8utosF5ml@h85XJud7 zkMxYu!)YCN45*-y!N55@N3< z+=mqdpm#L7=Rq70c#QAinF`tlC(}7OW^8c)M=D)aPYcOFCmPOdP5Wl4wf15{PqTZ@ zimok3i#)+I6-IfW(k+I2r}u@`*qrphFEBEkbN~b;!bhVcAi;|+eKlb{!Vs^|fW|on z{Hrg*+C}J>p1Ne^#;CF@oJ^f8*fej{aLO56=?%u2Fir+Fp}JIDI?t#EbJE$-s{oTE zw;rWu?$!{K+Z>DFj;j@G0Ocg*K7!fMl&|72rx^^p4ke97@W{T?VE9>0(+k>cZLPcG zm>U$R10lC!B!vcUb*{ol&IjM&B(TnbT3KWmnxJs4ySe4m5{37&nC1jy4QTJq&<<(R z%RfCcC@lnRZeSZ)z6RpxB_v$BQtrJ?3kf#XNSSenuNR%jM2hI~A{;h&RP)c#KXqqt zJ9gSwq)KdC^-2p=OXygVMDeWweRE5bN9ZAkFa*iNpy6&5IjK&rjW0cj_XRY79my?$ z+7dfu9;k9yyHa{KUti0y`Jf&i>fr8yol0&|GvtoE#+}Z8i;4 zOD`;-?-iacAatG{5?DfnoM9zO0NK@5h_}UlrAYX&KABC!pz}M9UB{iBxB`o@5YfEo z9aP$Q&N&>KvfXot9shy`wXfb`8aKa>+k+#cMlMvReANww0TR4C!{a^vGmo;Vn>vXZ zjbu&}_09~?w%wOmGr@-77Kp^Fo_txmEde6}gjn`Jw$W@CBh{7+kiU{o;I&IvO(U#k z4;1Ypca{l`1VOm+Ag|@aGi1FF{th(Mq&=|3>Jl=b1AT9xvala{q#VeB_rDcHQ14H( zob)wT{Fn}H{OEQ^C>3%J|7bc~?`K@p)HAjMF>mO>OL>n*H9McKY28ny?VTJX1uk_2 znuWGtKXVzj@&QZ>2w$edq)Mm3h#Z4DSB7mp#T>w8n8*`C3sHW^I4D^c6jyF13wm31t#%CFpA2ldKr z|5(EA%y@0)WUn|bR3%6C2L!uTxSf3M+E|}2Rf^z+!HkHwR_`Zkth@CVP5K;cfvr{* z!QTGdJ7(_<;UQOtTa07m)qJHPzFK`nNmIG!Q`C_u|$tWqGp zJ|sd#`I%YHP%3fTS4fuCB2$Mer3hrlXi^ZW7ak^dxw*6{?g5e@Z%h9=87X2QU>t0) z0%(bdl3v;*DSD%lCIjoCa%deHuDISGg#^D4x=eBo@fb#l7I8Y?ZvsfHJy&WSjt9~Y zJ~qpJyv7Do_UDYw9?dLQZs0{S-M&DVHyK?E^`OE5_!$jBq} z)<@m;wogwSX%X)BzFlcCxoe1YR;5SQk2PdU`L(y>{cJnQZrkHo=jl_v1p0#9$NQtW zz%K)mj7g$jDaIYNB3y5eAbOT}FYVc7Xp;{(pEC!=+MciWZGTi>_*J8J zS9ni`@-;YKZ`odlkHy;Vix}rsIoB3WQd8uFq~%eD?9OMA`Qm*#t*fOHDHd05j`ec8 znz^)=_Llyww|UvM`*od$M)ISDbPicgW}AT9R(c*s6_S~jLc|o5cK4tk!p4X5)rQ=a zpcp%Q;K6eUxGYD-|i{ZmSiurvR3XSmZ|{w_z|pW@=%sygq;5cx-a>iUR}`ye&RBNxIYam zNP|>H+F#|y+8&tLMW~BM_5S_JJBU(ro%lmFIl_+h1G029xl~Zs3Dp`5eY?!;UK4$? zKM#UOdcu~QnjI}@u8Kh_QGnUv4CfAQWno_P$RK*-y>Np$)XE5pV@6}u14O2&AbV!M z@N+yEREAsoL5gXjqthO4ak}5rM|WwhJ=^v?4p-mbD<1c{>rLks*G}&7*+d%j=9X%8 zl)MAFP^^xiz?S;Owir;r`nIBSbV*GVN%Wp(XItie zuW-2Hrt#JYurNcty43E{rl4S55jT35xKCg(D)(MCEfS*D9f1RJ%=Y7T#55Xw{HEs6 z7}CY6(aIb+s$0+BYxU?<=Ms|ML3O=_3BScgjPc$fRGV*|LIsUYukNiqg zb=P}Yw&z~)HRrpe=TA_(cI&q^?7ipHX>rcWDXdV{%SPQxbRHRjpZUyJ*h0E3c-Qyv z_Wc#E+Xu+C+j2Sgvs3p?gwLUb&Qlu9+l-7iURQ$8K@g3nWX`3EgIVQ%093PP4A+s8 zMb?rv|M99x4s;x9$4;Dx4pz()u-_c&eVR#**E_7|G}zXAHmT29F^>0bgwFF@&ifjR z=NCX*&3aW5SMJH??X@~`TUX^}+kJeUDfapEC8qoGz3aZ1=f-dP?PB^hNaq8f{S~fK z_Z%zVErh7G0@L{1^{&NpC$Rn8``md=O!GaV(s{?k^Zd7)Ojz#M`Jz_eIqNz~nJac* zNWe{eN>>GRNBrJDvIL159Ikhf?3Wvk?WZO*FHgcxwra2b>GwVw_ul*%=@+i4>1^#p zBxZpPLOmq}n$29^4S&5*#9-&fIFc<3YWzjxIsPi(yRk-m&@4`SV!3{i98Usa3=b0T zN<8n_%Np5z&VU=k&EuG)7-g#Wj>Q=ELK;qQee{eEt!vly_wCOqmF}y@?h~C)B|AT) zTk*FWFdn#O1{`lj=aVN=?*jC}0s5d30~+#kiAgf@Gem?(&zAt6QmHB)=GowX)qfky zOtRfbF7H_es^=ZDD9y+hxi&^Y%Hig=g6S-ca=14!-l9-3}fwY8TTN{j2e_5_+EEZxOkdp33t?%`@v z&#)JC?BZ?;Um&oS?Qcod(T-y)Z;N7{RVH3+Q92s40I`sWqO%3t(D%Z3uWN6Uk0ss< z^G@y3FIcag&)QO*o4l1KnpG)s?SvdR;|@>r^!N(WP)EM|swWAT-31XigZ1HL?`@e5 z4(&Evw~N|ouT4yy_u#g|fEg!27$&D%WHS{oeCJ14FPjd>?(bty>5DU+QY|XrD!Ha4 z;v3gf?_rH@7VS2jAEPc06%%fzefLOwLFew^mId0oM@(~Iar~~E*TVP4+qusOpV53L z6+ph@8oYz)R+~@D_n@N{>9zsG-}nYL*mR&==!p0Q z-HsdrY~ylc;G3Fydh~r7_y_aGv<6x}8=(-hZ3<1OF2H*iHYTz*^(yErL&On~74<{4 zB1&x;K6`*P4a(w$l#G$uU!5Fs5W6d2t$YX79Yzpu<9>wu>K;<E;S-Z0IzGOj; zhsvm};bWXolcY&N2!@T6Y|Enr3OYlr)DAAwl`|e%<}M%))i6azq+%AXA4U6>q?q&2 zJigEt*$&f?s~DmoXSOG$II;LF$zECR+AM^EqMutCqeMv7YYVfCWWm>b8`fCCl&r#* z2A&LqHm!94mLF0E8bgy2k(p@qsdYk+=Hm=1O?!Z@8?EFdnD@HOk<*jw2mV&i;^KFX z(z43evYN-bX{F4N+V(dOs~t*_1}q6q zLlJJxSRMeCJq&+W!s>SAL`P-j<7>(Qek5AFO6DoPOI7`CY~)lslWXKmxfqN^XLb#mcg>)2@u#1khZ+xY z(BIK!29k6ww`{pBhGDNy&+0y$sQNEezJ2LNA&5M0;b`QY%|oX;Yu^>TJC!Xxm6Ub2 z2n#7l&HSR6tIR<}ReZK4J8D!d(jcpyHC`<;tKq z2Ion6Rr!^}%L!Ly)L(10!ze1+QtmE#tnkMMGCoINFvMt^ND3x$d9MSFd+1>$FC7Uj z9&3Zi(`tHRK>_GY;H>L`L7}CT0GPCci)3c81`GroUbBqO`JtgZA!N_73iE8T@{KCC z==S}f#^%;$#Y&46qg812QKDd+cjuI%1!Z)Vtv;PoM@_8d(^ptmwkVP7WzSv#Mc`KU zEXC|xJXH%uGN?EGtK@pVL$TVq)yY=-6QP+YN9StYlK}xD%;?%8Eveo}ZDe2huO@lN z0^QLo-Km5oN9P5|*1Bv{ky6qydbLI^#py-jXH-=oOdc^X+NUY%w<O`Ip+#Dp4V zp--U7$_jHnljBoY;O8b49?r5Qri-&2C?Wk5xy?uW<}H&SkOXaWr!A~i^Y^$n_G>8s znRro!oij34BcuwHnEUY3LUZQqR4j``J2f3KS$;r5_M?Fk!5*-6o=NpmC6 zTiKzkU&mq^avbJCWrA2|4t$V|xMBrk1w}?1BO%h4{gNj-H5T_g) z9(D5(ya?1IB^5v?;HJaTDf69|i%>C)S^;-u?W2-q4^{D75sb}{{{=Dce?i2E$=>|W z91Ce&NRv#+z>f@1kg26Pr2p;7IkWAzNaY95t~<^poIgX-BVVGd6`Ob~_}UBGTPXvW z_H!dSK!1jq_cO%8JeNfGc(Megbs;4^0yVy^$iI4&R_-?MDN@;;^Ix@MXxbKM$C=^6f4?<*E-3utu_h&xhD0X(8cL<7ZRkepMl!TXW(l=Dx};yy~bX3W$}3=6h# zHb&?N?bZA6*kmJ4+?uOkd?D@Gezep2P5V4|3+2TSWW%U&FME1lzA11Wrs=Boz{T`d zU7YYdHtn=QEyMSaF8x?_5)?fd$?zY16omNPO3FKSLkw6ZnZ|6{@eeAUtQva#o$Ep@Q|FfLpmdP#$x07(!V;4Ozlq&T>3ZDF~nUoi~XyR7?EXD+v-1~f|yCz=NaIL|{9=rN=d72(7PP5oTQe*YBa zFPM1RCt9=Yra^7whG@$-*8`|sG>>bWEqy*m+vjKV%f5pFVEfAt6 zC6Vs$lJQctzkV5c3Xl%H*ZAwBfzVtt#zR$#o4gC~7I2qZP^>`qC!m2}r7GJFIoFTk z`G9hJyWh%^j*6*7oqijowTgfpQsm=_$TohEDFeHP#GOl8GY@hj5i|@jCKO5SVq1Vm zd{2>C^iffp55+37V37a3)UBSHq{}{D8tY*xK}TO0^Jvz-be&Y>Tw>vGkgvjzZ32{O zRt-2PvoBXDV3!JKGS}0Fm&3jM%Q;^)+iOM(Jf>7cbI)DSHNi&Ms4h>k-#JVo15+80 z0}*QJRHl`eHvqf3*gJ=C0e-~(^CUw&1rst>@9Tf5e&M#S#@e~rDl{tL84rp8I1(&Y zU=k>#6nQE!WKw-#oifo*d!m>c#?F46{0q`Lvjl^?P_%qq5aE?&*A}g-xKFoN|~VQqj=Vn&QwK9!?BY(8<;M_tK9E zOQ4;yXDo8qp(EAqE7SYfehFQrQf>ZV-0?IAGwZ{1IpDpOBed%FE;gm)&pgUZKf6!n zjn&ROEf*(c74X-ge?Hu|Wjv_K`Q_gcK$@NEOg@?aDpNiK$J8uEXp4dhGL)?Wza8Cq zVp?7|2A!@Ijg5Fyy?Sf5ryHpa`kxL?qP4yz-=rqEzXeheQPM{Kd96P@F<^K@F3G~Y z-S_zOP3Ui2i(|J`fQtD4pz?5Xw~UxGqS%nJui7(K`tJd45wkRgpX`*_k8%BxW{CT1 zwRqm#EnXfkCZd!lq%|hrlBHn2BrvzQwMa>04nh~gY{=3d{kr-#q1}<$v8s2F_;9od z!!0?)=#hCJC?9VF&^W9>-xcaLN=eYFMK;X~U(*MX_NnSssn)`=w zX~b(+*8UwV#;}h-Ed4|s?wW-ko&jdGrjrT%Q)@r$Ichc zc1=-x!+7M|{%G-24nY&HxIrQ4Jb2z^toeSAzG#llrppJ#8NPe3yU=0lEhyir-0tj* zklDanLB|xDYP7mrX_Hr>=Z+ugS)#g*)>M*<@9bfp={moXSuik$j?5x|#k?+^`xsfB z5@<6Vu=+N0d3tS-ZdtDK2(S7UMcivHt}YFq+mXoOrqYOJ@%d_L@qim?!&^Dw#oCs% zfjfd@BlBaOo>7A>rW~w6H=q|77pb72u5hVwoFOi^u@vSt;7&HUV-I%B86wAhu=Vj4 z@7nLE3}N?T^NGT6q^`^)2zD2&IQJjMUg}~XS*BL1SRdoxe8U8kU34P7nk<$dA_{ll z#$oAsw@_*M5+!nAu00_|hPKFN9-#}}fVj?%?4>Nkd2VP@@nNmEaFEqktjj+&pKvf& z+2h(fJhD4oU zBxGDbuqnE=$o-h&E4mG%714$9B?{3gRL1en#rDm(aqqTt3Le2c`&6*fBR_E>=Enco z#3Kc#GH+mi#KP>+ghbf zO!|F(_?ik9Q?u6OV}a9VLIiHMvq#l5TdWorxebsTcka-ZMW0pR9VFEHyMic|2Lgzd zzF$Unxq{%MJZ1)RW2sqoyH`uh8pZGE7vm%L{F%M-q>H6;VJHSvg>s!Y@nsLV4I8;i zf^dqeF`NFgLd%b6Bx5h28dEk1d%kFZ5DPsr0LeqGaY{G9V2XQ@} z?yqFC$K~Bkte|ryzIWOC$!Z?2mbub*_w@|+%_(B}m?;mhZ!E${vzxa=w0Ga}g}`RQ zB_YU!>f+Z=)s*!k9X2N^^YC=yj^*d+Y>3F0i4ic^t}p9zRn$^g%wLO{J2^`_KF=DW zn@2dWZJm3x;VQsZ()Y^US$cjiOi?cvk;j+tI^E;JB{7#CsnIKoewz+Q*Bg&S=Q)vM z@-AV5G$!(Z@v>XwtF_EMf&_db4ZRY>p8Z5Qk_3`I-UtPIf>_aH>=6(hHycMo$aQ9z z0()YxgI*@6UhOAq+DPOatW^N0+w_BwRCy~9p|lA1^9RjIo19blz)C*g%%|4%b#gX zNxk(Uw=7J$5Pa6;PE06+K~j+?alWmLCnF|Q-GNh8kjH);929ihEMxB?BQIgER~7HY z^{(&sJh05 z!&N^l@)4c{yB*pfD@>5I;#LIrn9Icq<<5#1HVNgIy6TUz;qGs$oR<5rpOgQpNum#i z&zIg02tk4&r?J+5u zt=*nwh_N{EU5?pOSZOh-nTVp<`wN|&%}eu({$zew0+8>`m7~-)qixa~W+q`IA+p=$^7*dF}!^Cn4RQKSuP9VQG&tb$0lF7j*IL zc5PPv4aTX(a5}kOXfZ%#eqFUL*U4$vZrQ6UM%z%yviZ-xxc!pKNs>?#ZFG?{oKxNl zEcwIcpp(g2j{NOBT%-`d6IG?tE$gtBDV9xiEKqcF@A+LT1gTy3iHT0X=faphE#ejx z(B(|!8%EBZFE}Q;1B}qzW!oOfvF#gbHWU0Bfd2{1$?8N5LW&^eAm@WDL2g&KV0Tf= z@-PkOpZ^`~u3S4L^JLBaXjXr&8J0Q()`%?awKmM85rvO>iIoKZ6JXhkg;Nd*wl+GS zi(tY7t|Hc26B#%nuG1ngb-+bN-X7eMMvGTv*rPIE^hJ>@zjweKV-VAs>k|xz^~thO zAI{`m=W<^vlc9AiKETLbQfYfO&t0`@4%KRUToTzb+zWnmL#x+gcn-RH5=KEKuAw8%ZU1@Qk#fCDuF$W<00@3h0pqCQ(}T_@nLv< zq>#hhK9v;f_WWcw**u<|Prru)Mwr@fptQ@&;l{4GcLBCbL5`Xn2d8%W*F|)S15Y2L zp|;Fnq&*Mag;>~7FwczotXiL(SUQrys5eX{4^f@1}nkAa4EL z%|h6=n5EGdWEwp2*)I6-s1{k{siB`oRtR|`L1Nm-v2KPg*I-3Hq4HyDv3h1#i2W~F^If7 z$hx6@g;?W!)G7)Bu%P?G5SU%_HhaPs9V8Yp{0u@LCV_@!ju@VYxEOM1>Y*y`LP2MF z6a8w5ei$wD{@yv+k#Eh{`J+bg{BVO*QM8gRR#=2ibX#%+lbhq{UfyQOjLA))m`j~Y zBeuu8wiG20d8eP%uYWA_L>I4ndrL@3qA~IGT)jyOx};Y_tV;&+G7>>r2VSN*p_aI=j%D!qjZe!}$y=nB~<8h!s{d>srZx|2( zm-&8xYI?Xx&eMxh@bbUBHg#x<_+RS$u77jm_jV}#F39bR~Wki@P*B2ydZNfS~8R_xHfzhSO z>5d7eL@q9Q*Wa|m_p%^_4Nu?Kp%9nwFTxR$5Ec2VE4W?Jc2b}e@Zoz@)vxQiVh{a- z=k%4h*Bolkg=%0k(2#ztJ22Fl4g_jtB9LUNIbRm+K5-jpWoSn@!SB6-!uVRimz99u zT+}K`%49E>(5XWt1L0D-lMZ%^?9J=2tVhdf+4{#bmfc7ERzt8gJEE3FUwzVnR)1ex z%GbZiMi1(-#a?D*$gd@DfQZNPFHWG0)&e{};RFXV8$_FEF|EDa?v+K01n4qnVp|`q z=@mOva0u11FRc8r47oS=8H%KeH(fHZ8w}f-wp7m;SgSu_wRr}8i2H;c2XiJJ=6^s= z$PI%#4NQRFOt2Bc7b%V=ZI{#HC1}NV$+SQ7!K!Es&Lw{q6vY#&A9&bUj;M-E5V?~pS zdiHMPnp#K*1vG63tt~{4ueolXHK&ZRU1oRQo7Y_usWDZH8$ggbyD+HV8p_Yg6}l*c z#(O+_re-;}_bNTJFJ^+4Eeg`)mnZHAPPPiXZ3xTDIM2*dqnd}#P>HX4poBQOfh4eV zg+wJ*Xwlp~Y081rU!-O!5I9Fu%vx+|{|&^Fj&C)6&Kr8K?sFN&`H7t)hUQoGu&?>GWQ?g|S}}Wcy&9(3;j4rp z!CTw0kWP`NLC`T3v*~xRrNvl3-=*DcWvK}z8DFFvc-|)_|IfBVuSElm(rniKrl|<# z8kx3XJ|d>6#ZPp&e4FWJqrWK<8v?-rzbFllnIvw)#ynG?c^DEO=}0+4FlIcRc#C$b z-ekLDq}{K{Tw6}{S3oF$ny?R>$@$pPN$T+NXAhcEd}JL^N|X^hBdb>1+9PP4-QYXM z2iXGZ8RP^8tdpKCa$|Gr=d1av%o8RHCYLMB6mN5J6f3~1h{pti@+3a%H;B5mK2iIo>3*x`Z|e*AyocL6l*w#01J@ONp%`(UAqma&|U zEVh4WK&)W#hw9-aKUrELWjaAt9WZTkV3{(bSHNjq=}76>r8t4%8= z2YMKJXfl^@Cer=i3$lr`SDD`g|Mz+&_@y$FECd2-(&2(5e$fSvj>)oJWY+$Xe7*8R zRx3+=N+&>*A@%^VmjsO20j=*^wJ?Jk_RM9WEqOe#K zIs|6g3`=4LkQ_ISwfB)Dzs(~p;DYy;E{u>8cphdcBN%dJj)*>6POG`yQ&E<$;?0VI z!esW2^R5eo-dg@v_=cw5SGeZwIyEyLGWaGuc8mYHmzvn5EGUmeK=9xcx0k>dOGqs!5flz(9R86G9e2=m%D0c znav8@8<2Cr`GRrly-w^dtM5OjkhvM5Jyg-uvb8QO_$n#13V_H8=2W}}dg?@aNLJyP zv~tXP>KGQVVgXxRJ=WwefbB@~4`Vuf8!*o@pYXTf!(t#;x)|HI{1)2M0}Y8p^qR0L z&S^6x&NY5)p%SB|Sg^em2s(U|3x5G3sz#(uhb%q20X($@8;h4^}W zTlD_<@~}p_MYW+~w7#Q`mb#|6PPYLo{bih%$E>cplCz6VFa1i)siULBsH0Y)w#D02 zU5{-3(t7UF(G}}bta;0d)Gn=ZyA7UlpC+A+;~(eavF2 z1QuVQ-cM6m$Jymy9JPt-#SR2Sl0`e`-&Q^x2JXv-kXPrdZpu&RQaHUX;FlSuSa!`W zx*`=CZG{DZmKu-lLKZ@O99Si>D#potP z>p<&&4Yvkzxl~Mju>@KC>-z0aA7KYg<0j$hGmk53An0k;=!=b(`~2Lj*MnB#Rf3Xf z2VmZf${o}TJ(+dq3&&3gZbH1tfO#XtFDE%?EOy@ZrZJI?oS<+6fm4e32)@}N7{P$7?@+3sziJMGSui1xI|d<{40O#hgJA!EX76e_}l zh7R(o*;917sT&3C^8P%gR<{t}Sa`Jvz8g$mXV+Dh=c=9J@)E2e88QUZ*B?fLxxetV zD_f*i)^jBbW_>#z+;X{WLuy3gCxg=e&gy0oRXdl@UFxQ5wY}`EvJWBVmwe|_J>ZnF z_-?%S5UsYIEopRI_%CF&Ty_M9-e2FS-n*4Yyq<1AdH%->U_5?V`w#oEBZIAnf0b8D zY}m|a(E^*in~RkH_4$0`SUom+zV4cW3>`P_ak{*LG?(OBv>R{F@O&P+zxhe{s5^a# zylmhJD*CT-aM#!8%OG-}ua9airgi&hR?X9Uo+TB70@>b}wMNg&P ziB5w7D%?Bu#(^@(%B)UWVNmC2Eb;kufIKYI{svl01uG-mdxt6m?4FbK3I3>_gP>^7L3di;eao^NgZJiQmW?s-XS%Mw9zv1wv*V7tXv`^@6`VYUem&oA9H2%qyT7plP)>AYRI z{bR1FmD;_cS$7 z;^GB?2*DPajg%K3o5@W3EOWg|$4&2uXYZE zfzzVytOn<+D`KWlFe-ILqZixXJ2O@1BSR7#cBK=028m)M4RCM2+}(*DZ8;Mc+=t5s z?HUDXAl-|%6873%kWA58nHv1@87HubPqi&Oie9m>4c0;`q2)kDmhR{*yX&}a9-hwj z;9B1pIb+X>KUd+(GgDr=xJj?cfRGks&1YUM_u?kT{eiMDsOf8h8EyV^wNaw^p=aVVkFgH0HnBdSbnY48ZzK(!~1hgw6fzt~^hz2|;((kR_l5YcZZl zvTeaJ7X0MQ&N_;o;?G71zqU~xYe3yok1k)it__-mJM52#x@LZ71!RKsq{0_A9jX{d zr|e@eHcpuKlUyBJRX(vfs#Te<#5x}3L-PBb#Wx{n% zbU!^j^#F+i8IO#AQIU54|M2yeVQnzoqG(?#P@GbVd(jjx6sNemdnp8p6)SGV9f}hO zUOW(-QrsbEiv)KM4ncz5yx+cOKWE>2p7U>hEt#36vu3TTsrXZRIFGl{OX*@=YWb}h zDQD~#;?p+&g!nT){J4-y)?AjfVBPNYU9H6u+8?gs-en4Ka1*b0o2qF?fd0m(x^XgU z^ZS_1T7tC5I*p#kG# zKRy31xXs)?)UJP4{l1_p0sY~8MRQregv7FWmq8a&A%SzJ#BV?Fo;Ol?%#3(rCMK6i zS8bHqGU0;s+^JHHak|*Zu61&s<6g#HOaG-*mGj07%g?Yp{y0{Gt8czJp?o*HjfPwL zAGp+j9TT}?yGymWtZmqx@mp&4m(K~_npoJ3f;|a^yoEP~<1_WTf7t2{Xf`(xf`fyU zbIB79ZN(~n(C@`DGmc7HD12KLb2bbGi(>vyivN$0t(^Un)M?lJ@#k9^Ys(d>TFZ+>`I2jMPoL7{B^Xt?W$`Yp-eP2<>+b!T)ULef*ac)xf5= zVz#1J-&MUOdZeD$mZNiJ4wEaVbCyj_FIlm?^cKFgxR+ISqpA5bo`Xm~A3?}kec%}@ zPw>xTmtAw*W|(Q={L(@#6`;TDQ7j`OKJ-R|!(TB7&3dl8YM8R-!qMSvqsANl(2+iq zIvsw*S49nZf>|`@I%4<+({wkS)~EpY+5Zo=VJe1Z*DXh|nYy~5FS^!m>+89W%uWiQ zN`|C`ljaD{!@IV(2s-lAGKK9UasQ5o?T7bmf(3uXaPYGhMy?hQN2c)Xj-mS#0N4=9VyShM-`j5S16cSoG~y82AO>tmlj)aSJ9wBS;z zyv}pM2Yc%`y8rAvan_t0=`vt5hV>JAou{R1 zQ0=E#&>^05q`?e_D>`oeNVT7CK#;?@k6vqy{dyTfza1^{GiKH@z$+r+wH^xn>}MFg zTtNZ%5E9W)hcu^o)Ls4S*oQL}{ zkMn8Bo;N!TuBn(0vkx{Lf)vaO9apO>iCs-1DOD|=v;fI{E0&&@-Z)dl&DjihIzGr$ zRJP_AbP3v|6q#Uzw`SUehQkpG0jloarZSWX6S#b?-XD)D{FNq=sJOXL;ua{MciM;N zY#u&e+C~D+s#XSrD5|Sl?0&)4S9ah|I^$-u8&y2nQ_4Tv+8xuCS7Pr^t>_-7vEIU+x;s1J8CgW0pOfzVrvq-?SC32_s)nIGZQP z62jFu<(vP5N~>F-dKqRA1Mfa>HtPUW?i7*xeD2dNi%TFvM--*0Lh$2;ZUiFO4~Sdh zseitVx_4W7@ptI9qfKJYyd?4dp*0}lioh&Sz^0w~;-zm=Tx+Ml*c&6-Or$Mq`VAZb z7cta6Gg3Jff9Q5p!6$9Zn9q%UB|Dg>(Qe4PawOrrgZ}j`4zKSe%GK=NyMKRuh&!^A zY=>_Ta!W7USktD|;oglr$s}A>;nwa?`ShH8Dql?D<^p$MGmE0?usG-{jTWc^pcl{P z7|siJl%LBhww-$E8>&4|0RURuL4 zUP0(WgzGWca&y)){Iq)-mfoK%pB2X1GWC|!0m*f$4_PUv-p2`BK%FP?vCOz^zD-Rz zi?Q5jCqBvkK#X5Lx~b0WT2pPo+_M*_?0-2HC#}`V*ea5AocCkAg|YYiz8li>F1Bd2 z6du@*vny71i$S>hHm+{^-&zn5G*>T*1Y|C-+yLs@| zRT`|zSk=kTI93ra_3wYR5 zcE*Ai*=ICZeTREYa$_XzD8-gjEt`AAva=a>qmt2?OO0Jt&;#F3OBhRjU?k+(lWwJ*=3m z5VBBqIb`M|ZsVL8e>qhA!6;0x&=no|E?jb_QxRYOcJm+hyKPWh*l@(iwgfA}GJ69F zrgO;&;u}q5TSQU1$rf{c|6H$>V{0NDnzfSq@54E&JiE*99BVY+$=N z@xo3M!yv=K5#2vo?z_NupiROOj(Ebk0g@!edxtu9ACTCBmDZYvcNy2#ITj!>8$mA> zP?a`v06eJSp!SgD_bE%RO9nai=QQ0!r&J{21I+1r-8?R(tdD}5^^jK2tw*p&q^zr= zmm$(zmP2nN1RY&M2gR#?sO;Nh>HZl$m}5@HZW>nhXUmm0h&ou9mZolFc)d}n2NsbRl-0=06rkmpGr2MN?;%< zaCsZ@rLLQAj@-Au_~FKmWdt>s`|yD|Y(fI>n^cbEAy?K!U$`5e6t9GQqI6bfFK|-% zHiHdn_bE(gkmQxc0gsyMd<8}62&Vyyi**UJkkBzUeLGBH3!J@dnU$?&6Q&(^M>2*g z^DdU}SIQk^7Xb*-%JR?aFpeqp4w=ylM?)Dw12wU*R1Frm^aDFo@ZxW=lr^@6Cny>4 zwtLnRtoz4o`XkWAfRc!v&_*nlDF;9#Wg?JTr#gBdCO6CGt|vG>Xubx?&9w2cf?PyF zYU`5DHFXMA`PU&b-~*C^!fW9qZ>r%^lb7zUVw)n+z~{W@hAgQi!syILS>iSYx^sY6 zc&-qdWD+Tp%hSZ>UK4tqH!O@5FY8|P(*QP=dd=5HTpjAdv(%6(2QD5S^uEF$j`#WO z4EX>%&0A)A&1A!@XKti`ZFctaW?@5fVMrVd&@eJ&7|)+@6AQ$P)xsmE9~0IWaEq#w zDs3%SLPA_MbE!Ne!%T*vLDP|gW3b-T81gyBXNSgR*e+{(ycNv= zFT1`}wh6&aMeF*6auzm-Ep!Q}R`mnRGQmYp)>rpwYM+IpOlEh_2Fs^vn*LY%OjmnO zHk-`QL3a%JOA{97?9P7|i^(sqN#if{zGwR3bbr9x+-%}KtD4fQnacOq7PmBf>YHdS z7s+=Fv*vWs!%EfHtz;fKh#k$;8BB$+|9l|N;@>2kYQwcJv}?ovlNONYw8 zXh)FzwHBMQfjF?|T4)7LzbeV*H)J6hM9Y0Da3y|-bsi@Bb^(?JYR)m?D2#7ww%1Ln zBerRn8u#Xt+H$Y~ZUZ^{lal$iY53&r0Mw^$uZiZGrR=I|(RC#nxtYx`d_X*Wbsu&} zYNJQ}W)$y7-quEcHai{CDUpiQHoQE{h%Fc`*Au7!I#jXfE00<_9T_#ddaO6+jG)z0 zU3Mx+-;c(v1h6TnWVT%p@4&E5oH+W%?(XlO=C-qbt#nz-@uPmz)6~GK$t-ONcKu5iDAR-X5JXkE0mxz8xL}^v4FP zHz-E9HY!qGMxf4WG!{}&iw_!b|2rI8Vb@pkz<%q^OyVkb0UMhG#b8?{NsA)>duI+} z%+|;)MyFT;Ac|WfI&!XOy{^l#8bS-rT-zkM1Lz~e%xO8N3lP~L9*@f<@G;rB_NCV8 znE&7I-G5)e|G7Bl54xttyH8EddaO)FJNI6+vR4vkI||Tpn|H_UkfSZDY)j_0dN)qC zmDlOkN9oPAyhJ>8YI>iJ0ZtDhZBzQ5!qz4VRdIgv0CB`?bF7n_ghtWnTGQ*Gsz^XKms*lKOVKg_R?*;D650w&O!B&D16F|?HT}T_C0ws{bKIOXJ2kYa3Prl*Mh^Ms zSVnQ<9B#y;*-C)o><=?bj7w7K4Y!xd;UkFTv5=s2;SmLKe*Ir|KlJ6Rl*gn((VTHd z3jp47W|_?krC)#O)4%7{*JEa|o*BZ9{_CT z^I@CwgiQ{^!iW>+k*KX+LGLkbx8X_RTzOt0x3!VfX}v)grZaSKB`Q|M>{x{ZKk{-P z96{R%lh1#ZvsBhDcw?LyqR~%@;90wzOyiuKYtf`35ROe1q*3a1>U4_!H4-&CxncKr z`)#vF_t4LDkFh=j_DDr8M_x4AcimB}mp=EdpDVoWzIywtpDQriaDtm_=dm?)^t8>^ z;D`_E2^ET2$(!>pS(4FbPHI_3hnAtC*J@mKSS$szx0@*c6Jmm18ynGUjk{pk=0wdn zOpJ?=+vK+D!lgr8d9%3x>q-3Qki0=3?^KgfUAey#|7b0q5M+yX`%<>qU3;y*-%Q6k;kY0DX7!78kRtQ|5*Xf7s{6(@ zBj%btjlWccXZ!@MsCtEX=0V97Krx#OL%!*Z7(vJV75sDz-=FxWCIoep!>sdKW13RE zRvzG~xSM8#pnkN+_NJZT zIJy6f3ja2PS{ACQr>K*rOj;D)b8vxH*(q)*ggl^Z0c2TSG(!Yx%EGamFPUl{5Ktv# z&ns}A06@!LZ8mb;nJg8Nh#@2s3H@yNRR07wx4Bdr43HIZb8Ud^#ERP&Q@9b8sNOHE zp??5P(dQs6JbcJyjbTDckr^XstW$=viPHZ2)dFiEMf^h&3HajsjSsxmo; z(nQB{>f6vMGT-pz)NvIR7zEsG&_kXjJm7~0?I!=(S{!6^rp2P~8mnNAJu+OXiw7Dv ztL?5yPvvHGH((K}A1WC=7-7Qy7q$!gV6mFDbwE6WRD%g6(R{4Ybl$rWrsw_ET!nh% zL-`Z{!@is~?Z?=|)&z;0wvk~2H(YT8aqHH=DKFWfvSmzh&Br2yRNbF4>E}#Z5a->4 z$dS~-P%AJD#29KjC_no1J;>4t`Zhw-CRR>Td4s6IK6pSwLq@{dV+K+)U z3Ku+KYh?&M&N#p_HHWS&K?IA$Z)#bofJO}nWQFDywctqQ1j>Htf}}W(yv4sH@BQtK zafI?uB_mGR`{4BZq)h)iyVPk9DQvZY>t}J-`YZ(ECIOS{#HZux>reUX-P!p76TZ0t z9{ma8HzO0wF=;VzTU~f-F6`p&rF0KU5h|{(ug{YdMchImwX6bep;W$nEQsnxK0B>J z7?}HTBUVSG5bg9;>2mez`&nCXRq(%(P*19{aiw-E^|(r{9;lXhEaD>jf!O=^rroJz zcM?2acn%mxM9fp(Uq7ij^LVS)vnuc0XZyX!e!FuGVdgEO@(c37C3^d9Uk|;s@}Gg)HMg(JcZ9?< zueM4S;`2q9y%{7YOl9Ye_x?InwuwL@Y#qZr!e+wsKbvv&ITv9>eUyvp8PYbtl`&g0 zre3Q3@PuvHt(rLahrYRHz8-t4S;sHmPDQ(~xg!YuCcb*%CW(G0i|Yr)(%p34Nbrwk ze~<8MsyISN`8Am@lP!OTsMu_>J^eT5F!Y)PPn(PXP~2?j|0#c^e9)nMpeduXI_1*O z#>tg1lGTF6e4h5v@6W1Odp4P2&Vjb@r5ENr;Rq0W4m{&1(oW`@qjTfE*|m~}k= z3QCFg*Ja7KpLve5f?#$Z{wgtV(J%$1X*rBT%xX69!`5clM3IQYak=pVCM0?yKZJXNWn(YA_@5e!YcI)w~xrHs9!`bfVfgHSv?z5zj|RR)k4( zc;nr#+KUsUNY?Aa7;4~}{ensou}OL75yGm)JkCnXMuEG}D%+@`*;~RP6SvVFPUh{3 zF&J~2UPT%c<$B^C{=REmt0&b`tMR(>TFP#m_eSPO1-^0*&J9oJ4Yn`cxiJfJ4G;Ml zTGFO5rM;I4$$)+Nq~NwKpb*P{tR?WOa%tCqC&IP@ zQ4g&`HzpSrbsL?yRZ1R_KQ=zy*tO-__G=43u1h^woZtY|vaDXJmZBT6e?x_4o!F6!oY%VxH%wDl`) z{ll-Gb6Rgui_HF)^^LOj}zgd=1qmlK(;;e3M!LAx5HuldwZQfF@dp*pvNLo5H zo>Q5Ki*W4ORlgR@UEjOa_Lsc}i1V9T(Cgov;G;HUbeaV(Tt8U5mqYakc7JF8vzkJF zT(iC);g7j{oKl>4$LXR(Xs}#Ax(M*=Z~H*TCwJ4qJ)iiQ&c=#D!MQZKitke6iF7yrl0LwsW=)I_Gu1yJZbV}DgNcKU zuFZtnb~bOsyFU#{u6pl67>hWvsg&an`0TpCdcNOxWUjC+m^AN@C^0gJk^N5JKkn5Z zy8Faz%Sbqb;fRky8SJ;4?lKRO9c!Qe>SjxdfIMMTwK9%MFvA+$I40VnWJ|vDSR)zL z3b9VN0(F{o^(T0R52?v}Ho-JglV5FiD^tB{Z+%NR(x`lZxmP(ha!0j2cEQQU9A5~g zn=;m$OxhNoczVIPR0|+#T-7w>M5cm=6MK^P1*Cc&X+Mww<7UsojROPWre1RB^k}0PRzI@T`o=bwJR~`njzzF zDz}$yqjtM`{CZ=ZeqhT3rs04LA|}7Zk2El5Q8UuKZ(C;4e{dmp9V|GM%dG@!4l?0? z1^8ki+GiIg4NZd0Cs-6$#>3N-q58sA`QYMacT^gHQHxM5>$am$1cbDktvA>H^6Va6 zQAh`TeswFsoU9))xpCb$m3SL^m$qwjiX0BI(*x%po;ALfIOmeEj&GRlK=t}E-=oLf zc_c-E6Iat$Fz~Zs2_{5u9^tm97zCLYI32CW_RYF{eS8I+i>7n>?tKxcvV#{>$a-_; z25Q)rDm`;gGpN`^l-gmO{^SdIS#aw(b+9PjDBHI^SWI2XJEG6f^!uij@5jT-#(2mdhWIDwfhR4q#K2m~)mVicN z|7NG0-#+~A{@sH=24jNWVAii%9oO*ojZf|IIfii%E_2gP_wTDb1Vwb5*)i z*q*EBR*-#gG39)ecDkc<&{Mt&yt|;z^1*L%h-5_~IieEB#3uCjmAXuvg??^D5EsMGqz#ft7dN6uVuR_W4brwYD%aGmRO4nVc%zEz?nFsh;v90mgzlW&xskf=tv zdV^?%F%dxM&-k5HU*WM@_D^9VC*Hh1q!r>Gwo5G&g6?wy2f`T^&MzJ~Rj=%P+PbA& z6wzlt>vYsLV+}(^ULvDVG;^!IfGj5mKJeG5a+(MNXBr_Kml;TM$a~LpDBD zcNrZwx*>@D19DS68w$f0Yf@U-n`_a0xJ^V&21yVa=r-Z>kyoy?Z3KwU=>H~sxq-gH z6vErh-rGO_TpE4J`L<*!A}-m}QR^awnmWW?mR$2ET9fj^WI898EoOh)F%9t*{Y@Hw zD>lQ{^2(zBDZcSCgRH!eQv!8K$by@$b*a`%$$2g5Czg&F0L?gPx`5Ijz9kMEUqeCd^ z--7JDc|kq!BQ>Y6WW}(j#%(EuGtqDd@Xd-
  • q4_J>OsV%E#s|m^`dPJf!(C>JUmq zi5=rdnT=fp*0@)0s7t$1)xm?S6DMv@&7C5f;;EW`NcodTg%jV!wQi7T_jhdFV(*=2cnjx*Ma6zYee#KCy(jF@*7)q%K?#qxNn-wz%o#uOaI*dgU8}C{ z#!Zf0bfLostM}gfdT;r;7KWs}ypvDL`RK=TF1Ucnq2%L_vb{KKak_u_gmdj=dztDFBc<$8$Wk;c0Tj$Gmk#} z2-5L}=L;{qaQd|A#k@S;d5k%qD$9TN`WJ1bu=;lUHj^8B{1hiJ42eWjN6bYm%!8o)eE#EQ-ON?83t(tO?adHny3KTu1F3d1+4AXNc8CF`I2AnYQEd3;{bW$=wn4R=(u`WqJd@Do@2*Dcue)!Rkf%p#G;oJG^ z@E;Xy{8M~njkNrXz|s;%@@L~a38k9}9(_fK78z+L{ViBZg8*Y{0{Zx|P~#*U|INyu zac?o!55JH9Z2ZCUOMe(Q&ZUmdHB`#TA87^tApVuhxX7gGkFQ@MVIV*K>}yHnQI4N9 z|E@gHBR>R6IwC`V|8eqXr&5Zv{1neA$6vMnMzR_Iv=4T*BZ&s{GuXa;`bahLFP(>y z-|^$xAJf{>8Yj-g%B90`CNz!+-<`W6-m1Ui$%y7#JuQu&5u}f_x2jr1rhwjzf09vt zNQ|>(&?Vm*U;R{|s?!W@kuQa|37v`I+zl@Xz?CFii_B`|0-lrd{UH6LJ zAXb~dXU}UZPgyx;^ynv^d;*!cp3c#7f36g5vYug=m#WQ)Kay&?$kz^neWdi~$MG+^ z`uIT}azssyhFUgcm5v$`Xvl$ePCDtN8*aFvxY*udaZLWvPFyLC{_9g3NBj?2e_j2u z#EdSb3>~Y9ss~oIiW?3j4o+dnT-C&L{{BNBy70n_ELYUNG1y`q1RKHOpDJAGSLBWtA8h}g7Pngb4oPqtXu8@>l z@*9o%CZ*vEMF+nOVNm}|W-Cf1l7ndr>8+_N?mI5N;)*M7x#bqvOY|iE_~3&ip&+W& zdFP!!Z}waV%_jp{@{%zlzl+MqpKkwEmXkk2K4~nmba#SBU(&&1I@`Q%-Gf$jR%UMX z_^Xv9{6BjY%;qTkz zwKVWN3(qWYB}9Ybzx^W-|3?mJ;AZm|5a7oOL-z6z1E3(ev56;ZdQjJJvZcoM&WKJi zg^33-&}Uy+D$?8`xx^WM6bLtr>&f0#30zZKHYz67w2gXdP}R1DwO)Y915$yTWa z@pwU6fbL}P?&hX@c5Y>V%Xdte@TKJ|Ak%=YI48fIRu>jsIeQ*r0>~te3~g5gD9_^q z1zwCrB`1y?t#vDAv26J(cy+?~3Ab)oQ`^zb+K}a=(?&J?t&I&Y9IE9KZ+7+Y)5cAj zJG{meL3yXL@nh}hrPF7J-eRWx9@_Sf@JU%HZFe9jnO6Ccp1Kqb<_ zlOtqw?f}77pCKZh0!6^lB%OSQ59LuL?ue6F4z3wy9`8NzNO<^guzg#2{2cIUP;H*W z3wYh^mQC!R>rI*NRaCnBvT{Gb5G;)-4|W!0T&qaUm_8ZIKZ#l`1I;E+6O3os;_J_d?!!`};oxpZ+)#_Yrqc zhkQ6B{F=qB4tm}Lk9vFeQA)7#ZQ(y$F1G@nw(8_%!Q#38zx_D*(z;;V?(nWZhL^ru zS7~%{({3Ha4t1^#rt+5uLZ{94lH##(|xU z_(Ti-QuPhV=B88}OUs!8rV@(u$cE#Um+K@qY)Ex=;MqZSP0pM-toz7}pIP^q+`A{W z<266to2;D@>+H%sAN>IH8%0~ zw2yrBBTROXkz4?D_w+pd)KfR!c%uo+#xc{zjv32jN5p`!@t0lyN%-aX3YxU%<6cv->p~FCb?sviYat%DlpBfe<;K%kX`^7PSH*=EOnE&PAwRqh2_JV zFttb$Tf#biY?LW}P>%=;)W}cjGij*|%1nBN7@wxLB{1HVLgo-ZiXtYBuV2QGt6kan z5`eZ=(Qcrr8fa-1V^AuBAN=Ph6!GcgkCM4)Le0Ku`IVeZLQC7%@FMvINb5saYTyn( zD#VnJMlBA7Xvw0y5=DYUXAWOjQH{2EOKEx3T)Lzdp=4ynU{;o4xif+tt7&|>j1`#S zA4QxDg0N%`-2KeP&+tP6Wyus4ok;!+zJty14{!!Q5+Ev0rMt)szJ42~byMn(O$!2} z`qhtrW!I6xmmGFPp(@~SrT%94G5^G^BmYERNvlJ8WPVx=BFdB_u^TIpDwAS>e>zWw zG5NnTinTO2C!DGVyBg)nOBd?+&(trtG^H4 z`U4HSXbNOw`NJEnzbof?zkinJ7a20?LY*-qGjzI7ryfTYi$F%+dHKoGf{s|&<@>$4 z-sYztKeTg8ZbezVxTvL}?$E1lrh~MqzziXo=D7ABKXLr=v;X5KKjzsMyqdvooORCJ zd;*w#F_Yc*dR=FE<;5z9VvbUO0aQDjX&;2IzE2bI^lwC3zbvr{@utt1am5vvz3bd_ zxkY29Bfa==^$Tt?{d-@XA)Vz#Ulg^JR(-5~4X(ecYT&4ziXWX?HB>0f94>s6n0OQ*W-`ii%9A z0tdDKq<3khrX|Bk7#wg4>BNGJbu89lnkp*Da;8VO+4ulP_vytyS2g-1(2nr(D=uHP z>T%}0glQPCF`}mCipwv{&CiQc0Vk0)KSeXds72Z-K!gbGFYJFi+ju5jXk2JpK~glg zNE?NkMpllBlY}Z)hb44W;OHG*cN>E#ct?K5P6+8$jdO{^(rT};#P4X7xyjbm;Wg}$ zz6Um?cE955*%K;ZEX*(PM@;cXOc8QtbJ*I5a-1`Vp~dle`2nxO6-8u?#wQpjk7tto z7?U%a^I%z%me_Krq>O!$U*=Aae=OPpoMK0B|CI?WGcKY~FlKbW`3d*QiQM@P78yEknHQycOQmE?IZzTojXse(dp%2b+e z0)lQHf|A?GiIJ8f>>jvEKp<;Nl$RK8sY=MGJ(H&yEEh1kf1AUm6M@&!6=QV?OMhBh zy%%0Y7C%4VKlf~3JJpcf^wXw^R^(`>Mac&KomA zB1u^pz+1P;+qKW@N`$8_mE3f-Fyl7;(+is#lWlFNNTx)PlV4F0 z#Mv-N66>`Tjg7ta^_*y;eR#D$Ws>xfm&k9eV^Be?z{Fbe5W?PeQr<^Cn)t#O60zPe zk&}4zv2b`*!NiHd*s)m7Qj?nkd|tN}V}s%DUXNFTvWF3iXG~0SYxwQ$O-GK@!HkoZ zEu}_{8p$~cT!;~1+@|ZV!%v zrV>#vbdI6n8UX7_D#|NsMvmkvn6)A8ZEbp>K^<;!N$D^ufwkpfb#+ZqfhL+&7u75o z>FM?;OMcl`Y@4_wb%!&UtDAqx^{-q&rWLI!0(2SHO5~TgLA6Z#OoY{XeXBC7A{u$F zwpwx<#5j3T%)z+`E6bR*qKs{E?uv2rJAu;rMpldjv5OP4kk|tAP2s!J!M8)-+`!>R z5Y0pmJqb^em@2I2jw{ww@gFpk9q2Zf2@>40^FQfIRa0!C?T0=RT!dnBsX(35JP0fiZ>$q%E*0r4VI4mi`BB`pO`=|BeCxv* z#6rZ6okZqTH1U>JhBp3P-~fE%b!dFr6U!nfLmGmAkdKWoFEc7A#pOLFmjRa0w zr`|E-k96zL&(ZLY&A-VtgncE_o5UBurP7DyKO28Y{>eB1UuG2CK>t?>GNv=&K>t)0^Tq-8wNO_^KYso{tH02o0UHPW z@R@C2BbEe1@I$EM&Q8R?ohZ;DWbA9*s{dC!IH(XBFc z4CH?RzQtp%bk%e=H^*I@3CH9%0Fn-)wXPtTLFK)g6 zfd}6C&U1(xVBf!_{-T$8n3k5Pbo&^9pY1N4|XOE45KOg|Vidd#JlUc#6o>S>fecHX-$m^pKnC6N&bQZ8qoqsR|w+3g$93gOGn z9F*yS5>b?(o&R(;g-IThF=(y_nesqn_UvF|Q70)#MvCF+)XDGg71-U$5Z~}w70aC_ zbKG(^>`)m#lN|-cHb=*ZDfD_e!^T6J^4PZ_IJiyvfzL%~!LV@7JHi+@+jxGX)=#vX zpp4BmRt<9tLO?`?RFXlcYcW4Bp2GtyT<>=D^tAH^Tr1H8Bi3_$Tlu~?muNW}d`tV)xph=>A4VS zMtSCiDK}F#JGbye7tb{uY-D2ucJWB%#Zv5cvHI?uEf43^Zi#iZhWWfbRPRXimM;(A zb$$duX#&wdomjaz2)__S@`!4+&mvH<(II}_(ryHo&T1+Jrgq7p5o(uG{IV--2V#qs zNoegGTVu>$m|}U~j%_>zpkf>dH}ZFE;rbvpYLq{2Ji!#f`V?CdVx_3ObYhjJwpbAg zv+82_gdhv^zzC&Q!^zL(IifS0V6X5B4%BL64i>lZ%9W8LK;-~Y58ABX7=Gu6-t}Md z@4MgYVEbm@=Mw1PVZoZ9oB%71?2!B%N++3QP-Y_X^Zc$Jf88eY63x-bd_-yr0iLbV z;A?MfaPVNrhRdg&;^*e7PZN!z9a}S2^vcVEl}kb%G^?%i_tg^YT;E@|z$-0cAwcSZ zN2RPPPb7c%{5p?EV}_OZXRkyq@nWS@=Q-2Qs0br{i2BRQke*(xy(g}aH*w}kda{v` znUl8m9Ci*$#(duYTT#8X1`8e9hYpE90Iz_tTm32zsbNju7E^drzdKO?QdbSH8Z&k*8;1#iY$@Jbyy~Hc{?y&2r^S@P>`OlEXd+|N)QAx^ zU-`&s|sDxgH%dLf1*REN|reEvUzPj$!-`(-MGtN8%{$*a3t0Lqx z)_(G3C8$*FZbg~m@^?#V(P?=N!eT6?emZ`v{wjlta8*)jI&czMINzGGBQRZPntzAy zWHAiV_v7FBkBXxZ`$B#?8)cCh9i)mzaIr&xCc?}vD$v?rGFt+DYLj7G#Nk9Yu?(0G zY6G>!P|53LcfvV*g(-tcX+>x7^~o3&{5AwD++;WTl$Rw}Ohu5=qJ(J_ zq62FL$xH}$&sa4_VQN(j+nL9n1yvnm&YFTWtd&SAf{f9~N7|bkN(>dPLQze{XzE!! zP&%2ERbnf|v5=A9O*UDAD2k$-j>^}cf8{l;g=obMz*iP)xH9}_<2xxc{EHYvF}a;w zB1L4Hefdnw63IU#zSOk?d?}KK6TYK{K$FTM)A*Lo;S)_>SFWre-e~(|UnhSCUnk^= z{Kw>XY7lBBa%D=+EPQ^NAku3{d{T3`QZn+>4H-QuP^H?J<6k1U?ohIjQEubpcQ~}D z$M%0Arey`q6;58lviZ;0*VI-*&QG*ik;|fIrCuw&50zuhKg9!g`tp19=H2D%U z-98vU(kRl(zTn>}lx<)9hg6aONLkxr-P#}gN8^PwJ|`J`kO>wVp;`W^0u>t@bh!R;+%0G{a_;1d*HU)xo>41g}j!b$;Utb z@tc40(?34?=ucY?KEnuAn^dsrK!=*IK%GNVr6<0#gZnSTllUe$(G+G&;Y{@tp;M9 z4*RoOy&;dX1^&esUwrQ$?p?oegKX{vQzlRN*vGFHv~syG4PJ%MHIJwx)M%ApI*?G3 zm=yNYY-#D7R2G@;Uo4%{n$Xg&`G`=UTVo@$GxxOP`D9SI;oFze7#k*l;Y$hR;fjZh zWTy>>AH++={Bjvfp3AZ#fuldp10{Q(HK28IRsPAy8KBO`XX89WtmJo2S01*L_Jpe2uL6HHDv#* z^3{B~M=)5zA8g#zTbQ3yT zmJfRJ1UmJ0Uw74vA20$(+=Yw7Wv9^o5UKHxvo8v&{dNDP2G?+a2~g*5R}`yz=98qu zzlk9as;jvsj{p=3*(BhPyZ*gg@2Y+aJ343td5p!i`DB{6(0ge$Z`Md{-sD%;kW2-y zUmx+*g*5s+OBNL%&VYwvn+dBie zf=Qeo*_)b}Dd4@#-n7XwB}wZWf~Wt?OIW;yM%7~inHwoB3KuW*PCki^GBv%Ss~4&o zv(ILF6>eMs0Q+`fBw?z`!_v~Qu_MrXvF*@NZ7VwNOjOn1aQftM$^@@@UGV3%{w41WOG{)-q(0o}zwm0R zy(75zjIgTG^6MiK!*#K0q|}Zolz-AypHwc#8N=gbG8cjK^?GlYO0k&B`t+ELaPS!3?=YMQ*_Qca;oqT3g4hIB@iuprN#i%=Sc!z;B~ zjB%=O=k`*F!qZU5pM8BtcqR%6r=#%F{jbnP6Jd-sGH&%I`d{#!eO0He|1#x;%0H^2 z32gF_PQTPNNTAHTFDTxTD$#StKAc2^eKqs?uTBRR|AF*L2l0^uHn~m?R zttL^20O!t~-G?uJB|phjs{N5h4F_qFH$CPc&{CR(lts}<=J2J5u=Q;@(()e#-;@=; z%cY!#4+i{LK_V)H@Az@atdJ~x$~1l`np1K`P!iC74dUmG;Op|6I%zsOeEHH)_!;}A z%Okny7%abnB$ccm;VY$a;E-Pz-*5-mm*Sj#4a```;2%YPwMS@U{wck5nE%-M0DR&y z{*#eE@=sABuXD`w`H$iE@sEh2l6F#lI3zx?#*SePCBI_dJU@a1Ir6kY{2zrM6XtKU z{)+2N`vCQ@Vz&{@^`i>q?AymbEM&D0Xs2uf5-`V>_UC^<{q1G{4VLS%H6ZGQ{^CjAkrr=1cg^S@ zT*oKv&0Ey@1x3Z3@vtkA2;%Xkrn_$J0Oa&!w|xrt;&xdBM0F{<$|Nb@i7DLXW8D=c0Eic4#!emepsOE0BZ+S z;1K;U0?8WI!f0nH8#K87ck2P*7pWR{2w&L;)?XXci8RMgNonc0v18>H<{uQ9uD{ahuT+1nF=ogsDm1ti zfsuDQ|IEmb3`k}+Rg5tr?M!|dSiWj8AQ)Sb{8Gf^R}nV;;lvK|a;>7pz>uTPQfF3| z()d6z*YoL5fBMSzU!^-^p7$?b{t63LIpJ{pIDCF;w5M0uNPda;U)2A4jR&KFEZwLp zmK&686`+D%zy9jNiak5bc_f=LEQO2oJiL@Hs$@}y7~`K@Ch7IGc`x1_w71ZY5}+}A z`AC24Y%e!oCIITtZuamXM(@MGy|{GCE%XXmSxvlXas4lh@uJqzrKMd>jddNJwJj|Z zi-*ZNktYu=zfl+2*AfG&uzs z=7T^cVtA~kGm&WNPIUIReEEfEsRP8x+jXfYX5uz3AQ#hcT0~|O0Qs>PZ#iZ@(&0-( z^0{9+i|{#f{Ll;%%^U<`q3)hKZ`Nc?r0JQ`KeN(`iq-EfN13^*PKy4rg#Fi7Y6Du&BC>^U_D;Z>Q9*# zu71H^zt&r_ELBjzi&1>GEM=+1h#DRb6f9y;0*66q_{}-^EHH-zGcOYNqL}Wg8oo-D zU=myf2|-Y;ZK;+Pa`WO_f)Ix_0%kcTr}C0(*9DJ0>F<7x>v;JAO$6#roE)CC+@Cuu z)!of=b8MO!&l3xEfyBo0H4_EZP!P7wB%t?1d|r#a6Ek4>hFzye2q(1w~8y zmtGBbycU+1rp`J|fG8>c^$udYenDLGAL^0ei(&YopiKQWzl&27^=jWlPb$$ZtK)d` ztyrkHxIz$Uv9P@>Y&r}=-}r>5dMeAA3_p8-Z|&I^UO2OnlNDkxn|6m{`u=a_uzwn zJh1;jZ*Onyf!d85H!WGR#B!KOf;S6V5Lx(57;X?Tso$JRe8l1x5%LTPLpL`s#`!h(n~PtKM<4L%TG9Gd?NI`V%Nmf`2Rt3RwH!i@MaD9AATTH&fnk?;|I zpMBH#k}!==AERIwg8QPG+{F|WFlli%2`0C8&?3WVa0L_S5=vFz#50+c#vQ_sG*ZD9 zE+pmB=TP{Ol7evfu4rpk5Ke`-?t}!^_zFo94wgTSFKLY$DP`ySltB2D3Ft(#S*oG& zO<*TK)lw*?M3m1ktppd~wAZKMOHJv66jEAQojk4$QlQ~G{z(`8Ov_JJ>=Ho>NGZ%R zk!kqJ>J%|8Xp1#Rk&Y_K;;ij8QVl^wI%n3;!T5+tR!JghO%aotY{D}!GWb$6%f6D( z@a030kmWxDRC)H1cCBpr2l$VZ|LFD=5ZR4S_)~Eb+vKAWQASJPN_GC@ek6{`qxiJ{QwVV(DZL!Z(A4+4+>5U%T7|8dHS62-d9GJl}Ghi7d&b* ziDCG8XzeSzpLufl^5yM4EgOISA1pEL?P%Ndr7utT+&`C}a~k8(o-k2-$_YnqyT@7q znkF2T_SQdCd+>&@e)XH*{N{|}l5gaUdL?YXJ-2ChkVvL_ydbwK73Qn$)ZdW(uY_@} zoV66#P~MO4>%W>h+Ru+&bIpSfJ+yP@PQqA)^2(Z5*mwQni!ZT6YFB{3_WH2?VNm;~ zhK6|I(qQR_8`dAKe%Wmt*XTyc^&^ANneZeer#jcCRa>Lwzo7mKl`9!@XfD0eE%zJk zf35!#>AEj13RO2U@;e$F$Bq>zzi6?pjh|d0Gge0kt?Xc=s!NQ%`X#)Sw`}>c-~Q$| z2Wk)U6!*Nj^9J|7k{`n9>Nx~wrI~Jh?Z1p{qGsWn{7yj0pxlG|-==+Bgdi#Xnf5=R z_`Jfvo_dh{V9Fw@;Om%j6J>1l`VU|tC%aB~%^Tt`TX;3`OD=KGshET{% z7IuLAG;|%_<2!xKDCU zRUW(3@F;jOH+dzYc$b9796uuk0^8V3B6g&5w$%DFrm>lY)`)-fbRIyUx4* zd~fn7xh{Xq3-F)}1stkn*A*TgXA(jYvOc78u68J#bAmU28Is|i`mkW+B#nsRQ|+eC zOhz(jNrkl`3RiMn8dB@IjZYG0&fKMMYLy2k6pfsDy z13=Pg#)5evDluFsC_TP%@>!C|TxTw8zE+{kMc8LGKR?E!8%)`;3u{Lg z6L(s)ggEhdsb#vh5moN5a!7%7q7r+)_njVY1B@D-|A7z2&OMjS zQF1Q2DCdP2y6?X?r|w9Y2-(WC>$*?neesLlabp1yzU+(0k@aP1p%E;8_+M61T0LR} zn<OY`~Cy_M~)mJV*{1se9{_rq2i7zvL8O}nCWhE0JT6$zepPWCW#fzFL|<* zz=!0o1QUrA&aM08S2ad#)7>Sqa_rRQvQs_`pVSdQrnF;Bg${ZBd)?K4FII^+dr=T6 z(Z?0dO;GEWj=#0y=@iUtW>BP#6GrI- zqU16;0@+Um>VGzV6yi{o!JQ3_FS7|r#YiMmSm1W1=G+x*ULtu&tcLq_v5imU8^XRB zeB)?<|Hz}%j|u^NvqR)R4j(B7`=|5&D1ku;+wg%ink3Kw`3>F4{8s%ngGE$1)-&x- z#J>paU%!U7Z-0pi=g=KhXOBr2YmWn-R2mo!qrL9CrM{ zZ+iylEsRs{M7nL4tX|z!>HJ*D&GCURi;s5mkwkhRJcibV(DJ;Z&SRK9X zi(if1@O7%3Jf)|XD*FJ@7MG{-m4`!ZsI|}i{NDm8A1R!oN5Ln>;Py?KGV)uk0-G?wz2W{B2+G$x z<;kT*2oh&-od>=yWywNXdANzJYKv5i^Km?xNh|iE3nq%UgP|#Q zp<3IXJ`6JC*??a}N5-NnsCW-_Q%&r{Y_?fq9 zr?+!YIBSZ(W=m@07O$k#KV=y+WujW83ky|5X$GG%2DJZXXI0F2c|~tYF*M;)%qoO+ zhhq~ac)6PC(|i>Zk29yG+8TrQUS_9~>(?^}qqp6d&&c1gO-Flb)8?LAZ%x)63Th8@ z)E&&-x+U-0Pq3$J&ei`*Tb$l|uY9MiJNbt@5+}Yb_w2L1n&HBKz3bnc-1vy$H50~< z-??KaqON}F<;|NmE?T${&B(5Q=4w(?XYvU-FH*^2B7BIqePXRm#9F>r$b%@XqaroG z$iah16_sp0$+Cqb(J_w14KjF*M1Yr0SD*q%Q{;KwwJc@G(GX0C}OERJ; z-$g5hlIn{St8f^Nd z^)Zws;l!;ZlMcp^CJ3DD=*P+5UHK1hzx*Fgzw)D`&8TI0q|yLXBBiQ)JsCGvx>=GwBk zv<_i_{3wu;<@bF z7)_B#94z+Iz^1KmYe5F6^jDT0MFxDo_%E)Tp zvgDzxElO zxTib7zIi-4KdArh?C9+5=pa4qe`kBU!#C5YcogJIKdt=~qS==KQJGG6jQ&@10`d?t z+C@1yCcjv>@k6HlR~c!1XB{g_3A6G0^ACj_XX)CBpn7FJ5BO;p#bFK09udaU)q{Wb5~Jxk5iLu)rHLFRYnF zvIxJc-S23&4%%nBP%mSp)ZBugU>Jb1V^;rL#>$TBku~{+)Q(*(E%)x&+R)x^+JbC{ z@$XIt`KPHA>DjM;Z|LmUe58K$!TnvS)XK3FZ+ZJ!%SVr4`T*W29yFs_Ctb-@La)q# zK?x?dji0i-oRYi(9IvjUyO{?M-dz8y%{6d6a>;?MZ+3-d@F`G-xUnk(JX{S?G=Apl zrz93Hj_WB2N@uYKFR3ld&#%ZUsL0JP&(F`bz4!~*!atV7>_+-utRla#C?_{pQvl`< zgT_A=?tDTTyZVLUY2TP#Icb*1I-W27x_8I^RCjyW(U?kfgi9s|zq-a-KE?mi?^2(* z#oKe3-5YiN+*IfDrs$j?%!`GrKw(iyZjj0iy!--h`B~xQnJUUkQq{%2+6w}G7$RD$ zW4@>ql#Bq75&4g`P5O;H5@ZlE9Is}CQZc3xCQp=LHH*~LhHD?d$3Kb1mxV?)6{8sn z{WAM4@T8!8o|EgW7M8&<}$ zjudN$+ETr{wkKOym1$0b(7f#Dgp+(0h_M`oS3_^zmU!R+6TENy95yuceDmAMwr0(~ zGXb>cK+jKa?)u;dQ@eJ?j~kbF!TC8So}kSiQ@u1F`J(K`5!In)rKKVN^h9U1K4R%$Gi-S{Oe0{L4{5UyO4C zmzGq=#z#nuqeXV$L$*`Pm=nHnZ6&CkLqTbckiu0nA*Su8dSEBX(d3s|2g&bjuZ%~R zU-ziEj!$Kb!8{Kjco zV&^33Uq~pqNIoP#X-0^}zPc5qLzy%FpUsbjXYtSPW8-J>-`_ul@Az^4qay9F!qs-@ z7orTszvD*cqJ!?E%&k~(gm=-?YNgYEJpcXD6S0Cbxng zYX~dj-Z1WNnRa}j9l84ZPai+7{yGxnQjQ-LVqq>qSW(pbE_?5)#~ZvOiELhMVf9U#4lO%bB<-T;n-;n%Ui7vSkX?Qi%_#v_{3GDzJwExonSEFynh`nL& zzklN!H{Em-!wD|TY>Xg*sD&z1Lp;W}8p-kF{U7+i$F9D{C=~e^mw!hN*Pnj+>0(q7 zmQy1Wk>W-v3=~}iZ}0ojkA6IJ#tcKS9LAKo@1u-Qj?w>GRMkt>yUzrMbMQ zQZhDmb?t3$=HV3fu=wW%i}f;;^dSYEUVO1ZX=g`EPeKz7Vo2&6J>iJLqA?{U>+1Y% z%?&f!E5?+T_OD+U@~Goym}KCH0{ks7F1o#CoPFzQs95)J>Exu>c9W{;gyTS zcQ5sGd1s~X4=?rq^-^!;Oz+0~ybHgRI(xc*%4m=G?d0jEyL7ZUGTU?_*Ppw>A2$_L zM}x$5ETAXv{D~Ksrc!M3cl4I?Wuu5keHD!ViZGC#5(rG;BpjgwHR@>4bFiCze6L z;?@q!9i1BCu2MKDfX(6`fx(M1NCunZFnn3uMFH#l$6-p5^x+CMiPH8>dkjei@FOw) z;_$_jQy2@}QH&n#gvRBBehp|P0Kp%ayEYINR)X$QRc$}G)Y}{ez-&Y zSw`x&hK&(Ek;)(r^k*k7p^C}Gx>RB|3qQ(eLD<1%lV&DAf^l$H#Nt(a=A4S}gmw5N z%iy~L96y=TL6F}#%SlR!MlK zB{&{Te$o@@@Kw5zOn{`Ll4apLVI4lnjAh~tIm^D81Ts#fAgSbN2Cpp2E_}&v=_C8P z1bmtR;$NvifpS|?>B23$=y3cfm)yzTP))c=b!xKAOXeeLztQb6+#-{V4NK}6+eni%daz2jFXIgGx!8X z_-UIC;D^kl;mFi4WpG7^g}#Hp4E_LqB!N@SIF&pgIQb2qyczs7X2icM%K(&A*`&x@ z^|zn@z{vpq->SdnYmxKI<^ERdZ&*0bZ+gM^Yq21mQj9l!@#=}rJ*kdg`O_~s^OGMu zb=u;u{ppvtuUplX>|x=6RS>m3_Rd7B?15ibSGRq~_G_>GZ zyd~#dvFPHfd2pH6;;}7hN8Eq5B{;oO57=jp0siCqFJe^zt@`m5)mfdZKgYRjf_ExcXA7^1r3h_G$#7R!a>=k}3=>M3 za{YobuVSQkV2eb-^n5!1$t&;+hw%)HUs@%9B?hKg*xiO{q61d?ApnWTuW<4qo+GaBb+q97TCv62alLF)PL^jCl-$! z{jN!q=U0yazb8y}_j2u>tS%@XRa!=7m6RU82?2Wm==;0cS{u9C%VPOly>MHfFcK0! zqNsGnu=2Z7$rlbDm|R-EY{cj{(fRl+r<7>3HGG_Zqso*bmjr)5niXK0Xl z7@zE|O)aTNZt`P4d?9CRXZVk2_#-RC2^5|&-#=lR_oIis+n@6v-Qb;28ZIsk^0Wq@ zRTlPCiSNxlEgUsb*W^;&F43LGsfd7w416A*$m5|2dkh2drGgc&N~xKP$S_vrsBuyI z0~Ga~Pm@(EHEaw@(N?iRP~Bv!3srWcB{;(a>|21q#G5fzUg6D{;XU=F-`vK26W-$4 zVHpdLY|cSTB>LJnUCEe$x=9|5nbqn_Ff)VHv`Via1Zr&dUV15b{F$(=MN;xU(~Hjc zDl2p)nK9qu1ht>UBdf+dfeNwLyz5%_sUCo90&1b zJ>+LK3lkDuUBT8J9{*?6>|0FI6J(rn;eJ0^hqetgGkUx52!b-Y7llVQwJ$^j;XIJWoF^^cRJb0M52lZ~--1*U~^KbZ*NMww_oXrRFjF*89jK!Nl>gnir5s4$&Bl)BH%UfL+FIu#6<;s^{e5s|iW$)fSzxwsB^9u6+>68CNu1x*Pv~Q?K zcLP>o0gs(gPxPmRM4*hAu_|np+KP&r0O#A1g#Q*ndhjF8$V+3963-m+0ixsyuxz9f zn)X$fodAIa?ADqm(zQ!&f_FXjWv4~wVO_oaTT&q$ugZ#w<}0dGe{IT3_bh zIF{8#P1QJ(#js-S!`#SCR4Suc8qc$6Hc@tm)Q(K$+!S9{c89D}6~}+$DT8m8AWhU7 zI4tpQd9jC`5G*#;%tdckVrdab+PUw!ckT! zH<+KJ*;hcs$p@rl2A^VI7hh?H@*l#@LDbrEp0RHvKU%1qf%pWPEQ)mUD~+sfKO^fq z(xp6!1!#GkeI2f%2H_vef1*+b`G4Af2FNdlUEQ(INDSkhaAPu(-^u3WAAdZUL#XlY_3id|p}4{A{mwbn|+>ptm7h_v>)_1^H*q{bE7!`{teb?Wb?M zf7^3<4xCnv+v9pdTK2MUZp3Zhu;JCOfBnYmue+|Lm9@33gzcSn^4TYT;M$Urz)d?b*83tdXu@6?P~5DsK0oDIfB-nY3;Or_8q2|JVknW9*aBfyYJqOn>Q&< zl4aPtzV*#-ed3eXs&XE^{<>gHr4Dwt1Go~~6N(1sB|4}Zo0Ce~@J2ga|ZNl-aK#ulQf znO71P*XX7)%jgNPc@arfR1wZRB`hpsklp7$9Zft4!zMs6h=1^zX zgS)n~wtZgp2x_8A;z0vH*3{F}(%sF9kHP(KYj^j)))uz*Xzodr7Uih|+X`)G-w}m{ z)2k{+6cjgib-j4#z`2tqSL$61;<8VE;|CsUTU(oZyP1s8bcGqqNgiF@k|%9ng^TOR zf1G7C%cQw9!Dr*ot{lF#v2oMk`g^u*9Z^_3rnr>tPK9qqfpB+XT>9(l4qg7xeb>%k z^7aW65MCbUj%^k)rA{qA}H!3O`C=5TSDx4bMhEC_N7QnTOgmye_Y6ir$HWOB-Z zU0sdIN1BfGRu{5gtWz~|FykkBdxg}I^&R?^a)S?|4XfhHhZ`zdb@=L;Dykbb=^0XAX zfFdz2dXI2cJ>|u{;E89wRnIWnMH+e%>Wp^NC@xBU_;P>rD3eKKSoKlI11aK>cSQBZ#~Z`zs%@T0G%L3;X5|r#(H(+}p-4NFR6kw&8H^k6v3bc$jk?jkdlzq9_h!rr z%WPd06!<~zdoN4vdad`+A7z@h&g2u1bpG$wypMb&IOPX+1yv zDf@UcHIq!nl7|k(ZuyxWFw;~)4wE2s^EoR{$~kd`2iIm$EYd5~ScHOnteq!dV8Tt9_>=Pl-;A~vK;RiqX0S{VUcG+d8 zzWp?(FqJ``=~{kBQmFmJPvxN^PE;$nj~|61IG(6jwHIy?k{h4%{;#UxOgGE5TAP}7 z?b^L>|GwjnI}TifPys~o*5onAjbT+Jitse(uHCzkf6C-3j{n`eckO;{mx{GMwz9ek zb)1~88h{mDzbH{;*=&3ICrH;JNdeO z8on9TinJ``)x3KI;BrOy>2ge}G=5q{L(mVgXFkMtnsEn^#?Q!~mP!;lshqz2a&QCW zNBFelefEv`H}6;WwER?QVOk!CpDjN)k;Hu`#A`%i;bvB{JJ_`RR(_Nr5)o+ASGim= zeMcH7y-`i1Nuoj}RTMaoAD7MXqmD%>l>qlgv%Uks~?Vqxq(QcmR3@R&d$TQ!nzp!Kc;G(W-h#5OL`ERSgY$HXn3{yHcuBP}* z|00==Qh)u@d0z9*u;UfhK%;{0w=#5NeLzoZ?B(Bi^DgvOPMosJ|8Dgs-?MYg*H+!S z>F{g041H}(g4s!HKHxq0;DaZec;aoh-~NefKT+42T>0VaM$cW6WZ=nV5>Hi7;i*md z3Y(C{V$XJj4q3=S62`JPxQaTtA}sfzAZODA4w`ID|2(XX{JsfV_%8g*OS=~1?eDFx|JTP|2?#m zS39aJL<8j3Aen_QNqEuKANe)cK$wbB8m^UCYf)Ghaye)t0akW+-EF!wr6lI+G!dBc zs>jphi6#O`$Br93#sW=3;ZCvPbA4G>>2rb0yINF)eIQHILHZR9vlx+^h9Wkf6$gFe z6D2jIO#NLp>NpnRY&=rm*3o;J&zst4LSNu`zK6x_V)f_SaTUh$x+4Fz1ZR4}G`!9On_dh)C%qiuS#Ab~jq{Rz|_TRR76Ea*q zcj26x5!LwxB2x6}!paW0mWULxiipTU7qu8xpjJ^6Z*l4RJAPRC!s=0v?AhJW*>%s3 ztz^A;`V1kEQ2mkn7TB7)ga7#WBZ*Y%`|DnsTrrHx6yBquRCZ`In?BSaSw6&}eyCi`E@}9oOB^O|`C~4_Ywbw#=AUZmDUTE-<${U-1>eb=YY5th863I<=Ngixr`Kf%AKpeR^@H$;%GlOUNTXv+jZns9m z1Tbc;4*LFa$Azb#>dl^|2siR4o$~Z&qY_HyD< z@BDLo-lpm*2rAGnp@YrG`1!d(MS1wjy70wUoCooUpR4;GoyLbR^Cyp2J7=oP@zkNH zYCHLjRVeX-Q6t08d^ot_hyJ1Z@Tr%=r(O)%OBDg&0a23)Z~7$fx~qedV$jV7nujrk zz~F7I1>Px3y{BFVpAOJJXQeBP1ZXXLqL69r*b$-PLbf4Bhq&R8MEb@5VCs~d1q;F# zRTMp_U4wB+kX4oKRx^~;mc1ramtiQJZ-aY z@7~8ATeW8G8XlZvdz8ke#^YxnKVrlPAcTve{4?aVS6T=K7;8p}IUxbq3C`rTgivLv zFY;joq*rwAQB+*Sk`r8kjNspS=N)_Z@44!Vt0;72&B(H0W$}2N-1KNmmn~&W)FxTT zd-|!TKli!MU3=}d73Jl7_wD=XPk!>iA0MPZU`k7djUO|P^rp0A$nY=6}ojP%oChA2Dl03BIIJ*;& zpXOyG>?f6~3q%Ub{8Z5zy}3gYsz4b_Ar343rGCP~OcV)2a+O>pFoi>%> zmLhb3W|z(OLn0}N^wysO#RMZ2D+^!9QP~!1Fr;yL9gI*unA> zow2X@kODGegpABf3`a$&6mm2-H*VRg4UrZuSjYlnvJg!Ame$to+qO~4g8B1Z z`<-!KZE^K{K!qn;*Q9^8Gcl{K%+EDZDnNe-A~eI`=_!?1jBTnb}ISP_1E!}!6#lB zj#+=(+uAm7*@71H*fmS-rm{Hk44#i6{L@7_9m(-ms=r5$9Nw{GCwJW!EnX1OnPuMr z{76vgYoa+uos>h#PhO*4DV+TJ<~Tb3K>lSwcZ99YEt{y+o~K%d2yedriuVEXN8k!3 zF5On}%i0GCnm#0RBzLy_L}&b8d`JQDllGx>_B1~(K00apA^|z1zb=APFOl$)-01Nc9_q$m*b%;Zzd!%H)ajFgM_)U6 z(xMgP=H0RWv9CY*tH$;gI(Us25y84xuKB<6(JBY4^3!yo?8MHgMfa(WUlI;pCx%C3MiR#yF8 zylC-bt5)^)bTcNzqNQbJ%ye+7R>;}U?3-PGr%jvs@Iw!oHhScEm@V(O!u#5v?D{)n z#>{)}zN@*ljh!7ih7TW3UNe^&$Ii0(M-IxAW0^wP{^Lq<{`02l?}UltS3S0>ySp2` zn6PkUB79z69&KWRR~`E7*3Y{q!XEaH#VfeL^tn`vGc|+Eh;-Ry?>qC%GrGFE_{mKz zgi9n664Z#u;T@Yci@|5yw*6OG0WUzzuPm>C^U|`??|<)mZS8FgVbQO+Shx3Om!tH* zL)PE#fB*ZfEiG`x%oKB28bosZrS+F9haha&t;q{_Xvk#1aKQav{W+89@<~D~Kx1UD zsLYG=76T3*>1qzUI^;7nR;qnu;XJir@q+Mt_A4!^l=V@M%UFlcR+iP1y-H1) zna4#1hE26{LD>QfT^#!Hyi`0-f##o9YUCyv|H+T#%&)Hg(dlP=ef6JTIdrHynf%)G z&;4S}aZg?!bZjckZa~>fkY!!kqZb%8HApOk>8x zg!Zb6il$c%A6Zy*xU2K0%mXHR-#=^K@FH99;QRF*9d~Zuvg);6YwPQm)r@TF?rKZu z-J0+xZN-%ATT@)}-kCGDwA8nC_Z;f%eD4E)xM1pxOQuX4TgEyD*_nBndv)(nnGjx%A3MH!`2SwG z_@aN1}|JG|KY(#5Hdp6=iLgm>@D;jK>w z%O|IfACsDu@Y(KVRDu8AQOR>tsnuP8pn2A$o2wXaTM-^O=*^zX)So630CbskQSOCRl>uAR zR+if&wj^VEPcmcS(Ifo)IG4}1U8qW9Cl0?b&mUG1jvEusn9iF@RT@@s^t%2_u0Q~X zX+?vyU|zUjp0|Csw`-Tz(5Uyej2h|Bn#PV8;>VP89gtIrE(SUE%FPd48rB~6?z}&> zuTHn>c>_{ualm57^UmPyB5Iq=`XUt+UTQn;IIdt*c`T`6UKHO@#e+GYwI{dS=-u^D z_=i<~Z5?lwR((ahu_L_;&+t#zyV?<#l!j!iIr$w?6UK&YQ@ZUn@5~eZX%jTRB4DG} z*rt!w!(j@MBxr&31VrtpSxPY({|v@Yj4fH5`1LRS<_^*Q#H!S?CI0bqVue&I(S~O6 zcl=-W-UGm{s@nfQ_fF4D+NAfHbRdM1KtQCZAU?qYSg1CN@T7=J5d_{-5e2(Q5k&F# zR8Z8XPw8MmQEAfCNKYW7_hgcpd;XvATKnvK=FTJmTzYxc5l0ky>9Mh%Uita+si*N4)#42|-tg=*&tRP9tXj3| zfd?Ptk5Et_(@=LAO>0XYee}^QF2DT7@7yT%v>kgs*LZ`}MJiLRsj(v`t3>6bERNXK z6t7WX&*DRz5?t9?3_dN)+FMjc=_rLFb!$*ujyrI6KrDfE# zY11Bj@WC}}*1-7}KmXY;fBDOXhK9{sw(zSUx5*LeeJ32>r>|x?83`1WliAQ`*2>)Z zE!EU97WL^``cxx;RsZ@9TUsVJNJ_G=3||V0L*bJ^L^oT|bu2TCm@itG?^GcZ^oS8A zDK61gKYD9fR9^ZYSqoXoBT5bfs}Z;A&<^5u%#w`+VB@1 zxTRA7HgZB2}v?$B&5`U{`$ezuEQ3NxpN_cP_rj*sfdG2wQ8bTf=)GXQ>=NZqd8L7>ZAY zf7mt0DgFbBHPzPQcLfO|cex()h-9K!jf7BH7nYl^h_l6s8xcJi-<22UA6p%^| z5cqq1QYuAbMRnf)HVXi3UGTb&->a|r@>jp|6)xvCtY43HCG*d#8ow3s`FDe? z-+IbNO(MPL)*FyFJ$0&#Wc@2dO2UUK>`c&(e;Ga?CSQ(!dc@32+0ewp@dfaAk6&Fr zSrfa#?=pUK{;x_}j!&llps;mSy0txNXWr-45;UqSn-l{hJ-XC`C>jOwD(l8vLb4vd>pC ze)&Qyi%*+~%r_C=oW*rrJpMYmLXI4=JUwpEpn)Ei41~bnYMsAE2fMJfXI{9zx05Z2 z$*_rYVz3ZXW2`u|qcZ*-zrz2VP4aw6be$EL!v8zQFUGmUCE=|Lt9i+*29?ewxva47 zWC=IBGgqBMAHdh9X^i5AYLV#CH>tA)7rJ5ddfvvO^FuC|b(uqhn+pxiG}D@E?X0a& zS=y*;EVOUsjfkM|xY%UqxOCKHox-|DFhjF3xN?z9vaDpQ2XYWMhSYMu12$X~Lq2V? zFMRD{wz_tF-}+1TKk(Oc=X~$EXSP+fFW$WIJ5N7#!_!YTa;FA&J=qgFSMcJTZS8t5 zG)p>3I=HUKvAfST(wLpN~!6`}2?dnXtcg#`GV~o;9kWv7xql z?Ut=8ws753>Q`UKGd@^2cw*^`(^s$Fw6(-tQV^#NN*c4|tInQ0b$CPLbIGB9SV@{EKAoakb8yF8^S| znrvkX1)eR!VMNcR$@~-JXJCDgLxzrAy?N_@E|@)I`HG*vFzfEQb9&X)_T)~Pq_lM7 zhIQLmyTyw|ez)XOAleM*-cQ#k)$ig!Jg{_4@Hn;$CGbfm3~$zDB^2|WseST1-pYY=d{Tn>A@bU9Z zTpb0fb&UYfN=`mmw#^*liWx?M%67uI!lgP_B`f|U1r2kHk(NwMHppRUx2quKqNs^J ztLHXNCSzL)V_M9PH&M6}@K5q;r|LD3X)!~UiR{n=3x^(RDE4orjAjrH zQkDwx(kq;Hq9PE2nkw*ivB=OQtCa}M6w-IUt?=$6(+!&nE7!=*;Qqz>I@+$*YhN=Y zqk-|QA|awb^&)|Q{Q|U?b6FIe*HR-YA zYu<81;g`SScR`RdA_e_NGi%>|bn%m)EKZ!z_SLTxX3pRNxiyCzTz|m@g?*S@#TW*V&lWrQGOq2~o^M(~DQz%k{+ANg?W=+^75zy9t!?_xFzYDk-ycL6(; zvo1e&?ARlZKAJTq_rH7>Txh^9ha&BlpH)Z#jF2KD|67UeSyd1*9em0HTvHRju|4m+ z^PYJ8ac=jdUbl*++~a~%4AF?uaTi{A!P2Em|I9DARDusWSR4uP1sqw^+S+=_#TO44 zFc7=er?^7#Bhm^V&nlLb!_aHG{xwy=JWvINH9M54hFu#v%13-NZy`vjdBh$Mqfw>M zldO^lp%C;ReLD_mjVp>|maH){Z8eS&vNTl9&TRv<#kCu9eDe~93XeVd$Tz(Vc86Lee{k~tXV0}FzUxMOkDp5a*hnzNGJpBXJ1Km~naCKIU_~>j zNSnUbnwz#4pXMLroN_sSf|u14{r=Mp+Ez}$Kn4G<_(n|q68vw209eh^;ujK_NXBix z#@w!MMK#8!B$-L=gxnMj%J`da99_+in2w(_C|~WAKk_D4?%$o_V^OexqvTEk#Q5Wc z7(X7Lh>^+bfCB*F1i^_OH*>Vkw^h zESFT^Q{?o#r=n zqf#E<+G2*yzrwI&VxcWQKU=Zzh4SAxe$`IS+pw_n@tgO5byH#VamjPHr=>MTP6U`9 z@P_V$hVjPavHOw(-jN(RI=z2UVd>_+wY@Go{H)Utc< z`#1JaH`gUkt|{)_d+*+bkquRMtx8vSa61Qg(~1{AA0WexBYcP7-dO#_w)8OFOd#Lk zmFWMS9)H99!-ssY|H`Rrg3sOmRxEr2*Pe&TKil|)nG@Rqe~Ygj9)G)w&x-N2J$wj| z^2gupj^EDlUx5#Vd;WiR{3?D*yjN&_VGBp_X4NKYr^@{*85 zjNMc(Vd*mnfpGFsYPaWOqR#P&(~-2$dvIlZas^e@kn|axY*@)l8`V?1sABk@$C zSnb;UZ(+)}H_Oxux{TtEaV>cidJjw+nlKRKvl5>gJO9DAExr1jvDe^QzRLBaR$B`p{t``DK%zL&W&+*Q3XKC+@*p zRIYjK&okDn!{UOC>q9>6I<`tVZXk-#^j7Q zryn?!i(Xy3(#s8X>9PAK(a^gJ@3GxF&Li7UExd_YMmRxDj2UtLMY`%_dA^q5;A&E1DCE zDJg!X$d1n6gs*R;tXk_0LJJE4(y%{?(;SOu%&F9VA|^fWU%X z{?-6p7*7{`cK|@jn(!){l03@z^VzmRSg}4ys0FG+M4S9 z4`{sc=B+>Z(Y7D{sI+K)+NP5NIikc*8}UMQ74O1fb>pa`3w8Ci$G)d9WJLQNwyk$f;EWSl0KJTVstZehT!=m617%0!JKi#H2}+ z9{lZtH{NjLQ%^s&aly>M-{q_S7+}FmPOPEwo@vqp>P=C-H-^fiULx&7ALpwh|6eCg&-#RGtubQDsv2U2L zKf=)1GUJiJ-!)B!5*rQjKN|XH{;$Bd5hsg2zp^j+IpNs`ZU| zOKxG0N4wh<0=43=uCpO%e8k(vlJM1rY^%YkeMdq`D|zg4XDK)QaN2j!K?j|4?%9Ai zK66u$4LY(C*pV~F?9+bjuw|{o4sS9nw}uatwCiP+{7p`1@;j3P`z#vr;f9>QwGu#C zaxS)&UEf95naSyj3u7m?${3m!VrZk|NB@YJ&5tG~d>kt@DgtF#X$MTo1uQs0n^9GC z#ivB-qALP+BrpJ@qzX;igt^8PWh$|zj2<)kutN{SJWr|dl03eZSPbdAdTR8)d;aoM zIev_)waK)+e^K{VV4GPFndtx4(W4JJMe?`SAgGrD5@m{;W= z$JZ>D^)HGlw2wd~*S~Uz{R}7iC(ob47(aja_=SY`eTV_vlE)`6(zZ4K*r3D{uxFYy zFQ^fQ!Pfo%KQ(^2!;aJYyi+sgM);MDUo|b~f8W)4(rX;QLbFsJzpbsK4>|Y{`i@6N zWo$d)n~Izo{O>1I0H{DX7K`i}*WCEEAT48vHkY>p7Z;6XFJ<#yU)GXeLb(obs;$M<3>XtrfKb6GW9%! z_^SVOP*F!`F|g<^a2#Ztsag^vD? zZ$#^#i_3LZR+G(}!XN26=S@)ln)1~2$Bbv&<{y1Q2VEi->&f7d9$G)w@{DR2wrt2F z7hA2VX1wv6g7&6N>nx<+O(xh5bQ`6_ISE~++L!3wMprbS?4Shh(nk@dpAY**PHkJB zbXN>lOag681Ufg_rGvfEm6bwtIi&18oC9e$DeR~ZyF6z8zhb&unhuRQo+Jwn>=WHY zWh8&h*R!Y^A-u(UvQetPI{4QdI2e2Ek;MMZi;p9No-qEEUR!$*vxhTxyR)ZvA_7d_pWcC@zb!Ent%@5Gk+FV;$wf?Abe(0SGP^)G}3I!fmkIwz5OUu=K zv*y%{Y4^3lzg7?M#WLpZv3 zgkmBjF9WuVDcu@cb}XKEGFHF)M$!E`xr>NAU&59OFqGBpdDC~v{PuJ-hil(@@x2v% zqcemMk`+X%vHGASi;?rJ=cof)1hyj43tqNOmcMCuZ;$QMS2Xv&?;0V$E&-su0_HYG zR(Ugc$V}O3o>nc=6S?Y8vYYt{xF|d#aeGhlGb6)MP{&C~Kwgx$La125MV85BC2+oY zJYFa$MuDg-UZFBT@)N|YpslSkq7@M-E_E#W`oUQW-pbOXo9xv~y4QJBrk(u3w7mKA zgWd~06d~E%%)uXDxQi>1o6UuP-~ZO~In}R$Ch#iwbId+Is~jz2fm9Yd{kZTgAFZ~# zmN*67H$x9%fE9cA`48DkmDQCzDO@@8r0T|tO9uP;GLm}~`>VaeuUWqm&XQBQ%_|VsX;_cQhY{|1 zY}Q{D+Q!{$0c_2GHdKL1#odUBd*^h99&W7AS8@_zCuz2D_wiBypbew!X2LdsLbS~BV+BC zaKg$;>;C&O>F#c~E7EY>9b5N(^#Lz-;dI?n_T$IakzvQz1UX?7fuLRnuCA^Qy1QZ(^CO;GG8Cruck_i%b4CaQyW?_nBFNb*-NSRa zNqs0+>{lKV9d+Lw>nmK&d4TU$%P?0vx#6UQ6ijNd+m{s<##&~j%kLoyY`6WY*BfAX z++JC&Wlys&w7cVVE-JmH^G{tr1+#pnI(O`YaWi{c&8u<8i=LCWEbK+#;+Y$iZk_oz zmG8Mtz1unl4`pQf;&c|4KK{3Nqstl~5xOc>vNbX!QnQ#?Pa~6%%eq50%2f2%sLAzd zT(7Z^JoAVe2SMLyy|RY06Ux;3GJJmu05k>qo=-^Y%0>#;+&T>qq}`p!5sMfonSHs` zcrhn_tvzKN-*U1Xyu*&n0}M@ME8p?J*v}g%RW2Y~a!v!*rlgTFlbK3Yc!1kToxJ`w zPVydR%mLHgUbPo?X|Qg|JJQ3A&K${Sj4M zS1nWzUN&l5vLI8_mLIm5d4_gjK1tYiYb2BkEFI*Nff zC)iprFM4omH@MrSJIoFOjA6~KpGv=@{HJQ>(W{DJA2_+OB?oS?Iqd&dDL8)Z+Xtr+<|fkWXTm8%&y_N#36R z*OXV9O{*~QAH?Ej+}FXqE0o6+RvVr)V#7 z<=$VDJwMm?L5CmUZZ=HN*dQ3Z24Q;dQelaOVS-E$(icxMq{A%+!~T&%zv6?(k#S`I z>=BC)pwHuf3N;T-5gtJ)z>)7ehzzZkSquE1OPeXNGwt~V3L8_W(^Ff%Og-~gKB|Ms z3O6R#vdRWOpP=Yqz$~i3wkS|8II%1a7(xI#8wL z8Ey}P2csWD6^`qtj&F-;d<0FFyx0cboM{Xx&L8~;Fq$#84jisaLe~u38gKUZCwR^RFO|am_iAnZ~MdnhnM2)s)6SqR;On!gz zySt3agP0TxKb17|S24^|B{_cASEYT;)w#}RdOOAKsh%EMMx0Y^&pdJ7;8T2u>Rpx6%H7TuT8o<~Ik1@IQV$aBdmvpN=(#1-YV5DfQhH#NhjjR@a!%jsa1rs7FQi zsO0ybRROSir>>a%Q*J)Se$%&Z4rAa2=|(O40vB>BehZ7=O5)n!0AWC&_H}mRQ5SE= zP)t$@yw1!c8?pv4wj+mD2EJ9 zWbW3AYc^iAFG{3G#+S<=Q7hMMrYo)Hk(bR3tI4>4x0U=b3Snp(>c!Kpv1Ut8-$|wBbc^}LmedZ zc(phN!UjfuPEBo1DemP$asw*cc-1p<)L2Ne+E=p2UUqzg=To)(XrEh{+v2W)P-IHE zofT{6T2ny-Y+D>XPdiUKJMa`Cvx@(V(3jbv%2(?ysjf~hd&k@jGrPdnz3blqqwhA( z^RHc3I+LJXbrF4`y8do;A0i)Ya-Kqv&bg{lejc|h-@cEjVc%3?$lsg+#ykZ_MfL2^ zM7MZp&uQ2dga62prjUT%mudXh-4h_iS#0P{tG(ui)bH<&jO8zJW_|yFsov4#?Dt@m zGV2S1^r$xIJVnzlC;X=BILPnprb9n=ws0rx;WFai*A|+2@%~Y5s`ND@YT)Z+URTh6 zssqI@n?GfAGO4~>FywT^eAY^<%JAn-^iEJ zk6=H3*ECN6JH8o>dE-U9E!9DQ$?l5@FUtpg{MAPg%OLzN+fJ#9Iz23F%V@Ka6P#lj z3O4@0A5}&AC}cT2dG}-!%o-R6*rr3-&>)eaTFh=RX}k}2#hi+%npoY`b*`t(1b5!|b%!Aq=e?NZ|50;TR6;H;%irr*2wL~!_fOXMXhz5%PA>1C)*nnPfuvykLH<^X5;E@t}m-n zd4ot)d!@YquTlD3jjL4i)$Z=p;(PQiGP(s^i99?9h;AZpz#q{c2fBh`zyfEyIT3$r zNDuZqa_r4%k~`tfAnnOfHQ{pU#aIydL~E@MkRq~A|ANoVM7 z-}l|V#E076)?ei!Tkr6{OkCueicI}1V>Dn7ld!brKj^LI0MLt%dF`}xQs5hIKPWSz zQ?5|D?VJ0)`Rajz2W2K6{Znlj>%|;vBS)w>$<`tItAZq)LNP|4Rd=amjbgvlZRm@~ zw{)^M;zejYK^{QY0RA%Nc&zDg(o1=NX z^-xvrSZUg-IF*JiVs}`O!Rf-VFlyOD$RPGZ?3SfXGhdyUa&+q5?@^iPFJ|V8jHMVV z5&Bb;H9_m7wd*%4MT#;9n<^e%Ejs*cfM;#(XeWotdELSbS`S7#24&be*vHtNms89b z{zlT#b=`cg$r@YT*GaH+==XUfy*4PcBxB`?9^Yv>-Fb*_`89>+zhYDGuNlav-ApBM zr0r3iW)?qvR=a$gtZ%|+wg>A3PK_jjX}o?bdo$QPVAT_yr>Qr=`-SH7PKI$M?(*|@ zJwGsFcq>s?1h3XM{p_v@`dH$D*-LRCP?>^&o{v>TkmH9=4L{Dt&g< zkxG1_*BPmgF&)pnacvLUn zZ8c2*+){GBdMdu>v5==?ORVtZXrQMLaN)9i`A|CI`>v*w**?3G=iqQp1GuY;W4a6c zq5dwCZ2+)-=6mR8o;!|>VfXLql`|?X44L^p4dC{{!$zH8_M5@4l`*3DFh{mW8VUyy z56G4t0*=%{d%~jn>z5t_wNYRT6@a1KT{wfpf1c=gwPozXTnOuSB&Mlu1j&0?@#@L< zrJh{tRDt$>@C3pXN+iV~s>BrM0~g|>?$D|uhvLCM zZKOaRc?}U*9a!zKXifjCrbk#ar6YUvLF4F(3)|b!pOY`IJxahfBdANl-CgLim&7*h zF8#q0I&jD%p-gsdm>Dz#*zN!q4(K7E|2>!KcEVT=3G*uEj~g!DlyNBjB&@8ivooEj zIZ0zVJ>D6Cm&7uh$v@~`{W*JLceXtio{_=b9}x}4-tJ_++vluGW<0#{r#znKB${4q z2?$Tk@O}w5WLbU?Ys>L^Xps&!&n@gKpkQL%R9Ey~*ZcFQ=ZrsozI~nHzW$i_T_g;+ z?4O2MbfSYXlz?lHq|gRzmp4{(sP^{TapU`lUMX+Q=55J#w^rY7UB`Wd^i)zQ|ysDj9^b$o+TAl zu(8-k{c(a*$h~2e@$-k$X;MFQS64_RC0VWhO3u;UNfw9R8MjbqMO?ox;oRz*c$tw2 zXnVS?Bm`-hhGpPf zNNBUIB}b*QhY65x56o2R2UY9H;lJ^Z=iic%$V&Cv_H_wtL-4tcKM*Lb#mAh!dt4`W zJH!NPqEIfLqk>}Nd&0~zHyem)B5LU)ImNb@emkcg)fpH)FI7#B${|qrsd;MQN26_N z@>$X6wBXf53XfUx9Gz9Gq9KHIrPb0#BTN`@KF_ek7aQfw+{@})QngT46hUA1s#Mim z;>kNBTJnGUx07wm5)2A|4h28XlOQNvTM#W*p=HIfyDj%ey!$f@{@(8G{$8sLH98D} zKLD4F_O)F~e&l;o`|$jwt0MMN;g^3m*_h?>O<6CSS0F(@M834jt*k*2!UySRu-fBE zC>DuWZ0AZedK0E5jE^Y*heoyM{lHkB@Kpty?g$80FA)4-pSMks{0wnF1bR6Cg}>@k z5#Q_|yjw(efup&@J8Rk-@y^SY*^g>^VzE; zQjl$Qd%TEQ4*QGY2_t@^B@R2(1J{+tn;bZbXA<2@q$u)*7$EAIGe&6et)^7J<>pWM}ff3`apf^{L|9%^{b0B(r{#d`DQEA zDt8}@TX11Db0Ww_pUM35Z^{Voc&d(75`CX3@L{vtPsfSv496zPCZlV3 zZNgo}hZ;Lb^U4FGz#aBB{a{<2QYDk4(;E2xtkWmvg<=I>>C+Iw7ltJ_EKhHx&tSaA zxan=`f7m6$@1lRvP7}G#xM6T~8q$H3h9}N>))OxeHD;GAXjS)VtB*jkscFE;!%{$< znj1ZJ@Is8fIu9j>(bdhEgaTUJ#ugri66!sbYc5?ld8pCA0 z$m|FNxTi0Jda8m;tM$v0^WDgvcVQF^JeI7`4BgNUE6#p`ip|y`gDVeD0$V>#_l2)f z{`a|My;b-3P5%^9Xf_C~iR~@0L%+0ccirgpkLyr=M1(`@^L^#jF(30+%7xH-r2J8&lMfqEpow};4S7j#w0NRknolgHs+Bx z{h?O*ycD3H!6WPa7(xKvqs7!MUHBw;rP1hd5vQc_dN;7sIjCyr;uWT`wCJip>;7-R- zkCN=26VW|0vIvY3MMyU`%3;o&j!`x8GYjU%=Cy4hxV32-#DL9dWzoYPqfjn&4cq6C z&aJ>U6dEoYEYmX|L-?d(vi-Eq8tmE2Rdr%pB?BH}JoCwJbkhwG)aO0PzgQk~m)pE7 z(>ZwT_HUhXrT^=K+ka3hhv2u->H>TuT0`FszF;E06Mmn1ZB}f>sh;;H?eA++uFID+ zhF6DN)P-*%ttW;IrCHSpLfQxDxqZAd!XP}&EGJ&63uA+SA9+viK&v9wNzFrbLTpnZ@dT-(~1H`_psC`{_G%flr^_GE1PEl$Fhz~-;h7AxZh)K5Fa)^jep9;%)+x+td(y8V zfrwRGTAf4*>17%E9tAebqd&|y3x(gRT=sN!U>av*?_NpjI2z|0#f@gZi|(d21UjNR z5_*o=Nzo11Xr44az5dG%c}Gz-iNjy*#C#viEmm4`G-E(Y*`OxejgW-Df7N`q@*Ep3%o@NEcYk}uO@k@ z{=?jzKpRfFU$LN7^Y7*@H6h*qu2voDiy9ry(@BMIBq&{`-Hy^xpQjBp*}E?Ol;{l+ zpZC@2p~^S>7xReNKH1jVbg8Mtv{MJ8PN{iWz*KwU>qp{y! zSy@<>oU~w#H}X52&(n1kgC47|RdKkdTTQnJysc^dTNq13&cfyL+=QH2_UPz*a=me6 z*s`?dF@y9?n3o_F_^GT3%I>~U!&q308ZL)MmiRP!*e z)YoywGALnm*Og;+%aP4vd(`!s7v-zxSQG+c6o>V|9Q)=0=;;FFbH`WvG*ur_GA0L9 zd_j;)*XeTlw~BUHBTvv1d3Wr7A$>UWW^TW3H5@0qFbLZSrn!Rr(ZI{_s?(_?O*ZZM zClCN~p{3gdf6iSd-{!xY;uO;#R?aqMqPMCuDQotRO-UzEYM_5k2m!v#2g0+8xF?uZ zi->;|AR{7YL*Gn+^I@7}YUzT{1|9l`;=zGu+O7VeLgjaE*7wo8xvkm0zxHSq#3MABs1c}4)-qy$>i=X%*96yCx8L&uv`0a#4%;K4)G|;5Nox~V)cM|G) zi?e8#y)=yyS7mjF8I)V*qhsX=9%88_#9Jtz0DCwPRluV1@O6f5xD_h7HF zxjnOw#I}t*_dJg6DE61jGSoYzinsbBdvM<#gRr?E9#@kMRsrMDiqSxp*8+Q#i(Yv* zcO8%d!?uz%yUEd-?14wbhrS+S$Lg5k6n#4?8S?L_yA7e1Q)B+2j63M_J)9_gm~qY` zs5>3w)a&c5?0~EVExguia!1d#Bk$%Dpd`LKoxkt*`?^cyYyWl}gKpCN;)`_fKi2q@O^C9nHm_C2ZWncYF*;O<$=EE>vCw*e%YFxdFX*Is zM*lblJvf2xS6z{Hbp6|UKX@S*s|zooYKR0|rg+*ApkQ@-|5r1)bwzbBip?hecN^lGLlgeJR; ztVaO*#v^&!ss&ZQV8YSe`e%DD?NoCxM|gY^V-U@KI=GPYI~>6B&1z`Pd~o;6pzjUI zLs&EhfIfoV5d?ERG2sB~6oT8gLw7^E)9NIJTx8^KV^PV?w8*B6bDT1|eU{v? zlSP8&LonhO@Hej_dsJ^09-b4|ymXimc-k|3y23}*oMn<-M9gK7D>JZMYdPKhQKQ|> z}d+fA$@=i)&Y{$nkI4w(LzR zFzYDA=YenYO<4!zFBzN&uSqOhL@;8eup;bg+8X~M6r z0Z+NSzKs!qaO~b4J0_-t1S7R&Pm$Qhuv?1t_qe+U^HrOLN0zw%^Q8F7gqp_o&9z0x z2e@X6tMUFO#ZZf8O@n=FN&gE@l#Wa(i>2?L`wUwzfo@o4E5MC^V!T277Gk@2@tB8H z0gg61>mPY)^f$}XT7=_))tWiU-Lo4vkG?) zv$AhHx4WhhgbzeEX?dG74z4;*E`7R!EnK^AfR5WrobuH}Rh+%IIH`0ApBbl&){z0U z?AeL^u*n8lFOU`V-ngtVzRreoGrR3Aa9G1(iR7;{(RgSm>pu5|p15mXkQ-iae& zTZVeH?e-U6V>=Q|&2920?)bKu&4~o>4`G-zfnwwn9<>^OH0ete?IN>L*{ge8)n%=0 z4OHf^wvtK2R;iPAEBPov?s$_)+cdO&j^DFjX0D^m!8=~W6;&r$Wjc73Ilo!b=bu2PpjYy%OCuD`SMv#8|~n@lDQtx#JpqG$9uw!NS;o? zua-UyEuy#BL57UeqpS~64hf3)*{qSVh0-j3&$ymXslFrl(AwQb=vCSreBT~5wLOf> zX+V0uP4yd|40(9Zi5lUBvN#7JT!2LM+RNJqi)YFWl`-=V_pohKLFpU$g&)SdI^j7ID}++@jB?;O$N1mWJbK z@aPCH=hy+Hsijj{!>(io2icH{me&N}CYNj2PKsqqAx`<%TI|h9EEDE02NP!IO@TsH z&G?~(%omlo?=atI#knTxf#Jk`P7V_@P6FFsT)Pg0-*sPC_?cmCKuGoSS9!93e0%O# zr9(15c|$k4n5_-&Oo}w(E!yl#DztcUDuhTd)Nea_`aNtY{p?(xTB`YJz@8j@WcHF1 zGrwr{Pr)1e==ySh71m@1+w$R`osP)=Lnr`YUwfzx(ccDtqc2mjYaF8AGJ{~{!7kP~ zKR6q^xFcD&YCO60Yx@VcAlUTD@6hb=Q2CscWHtq+^U^=F2?>aT#)USXjRi{OVDtDRb7d{XPDh6UV_`S1-vtD^8{&$h1hMvIM+b5#G23^K(}#%^r5VWtzNqL-nrY#bXw7n92@ zS1&S_dMCYL;BZzj`kasK09%f9@yYZ13q+8S5B8i-$vtnT9<0HI8Hh|{6E@!#y4U=E zH=fwWa{~4Jnvw0r0!Pf+vi@q;Q}`YWDABOF{aUl{XR_mw#35PG%3K~>@*MHtYs?)C zIU=XH3S#cq?s#l4ysdd7sOUZaZkPHA{P%Ov!MvfnjSO9*hl@bdp}n#Ir?In3GzX2T z?Z4&^g$+uh4D>&GPfebP*;Ri0NgFb5JCMPo&7srYhxi3~`%L+=MXQx|HIv}i8}bAJ z)s(tF)O!fMn0RtV1F4n{(IUpRce%^>Bl3zR7iz^X5-18w7`PvsF%Pi!(QmDuQyN@+3YQ&?=(j(>1xHw%w6W$MBr8!*fkD6G5Jy+Wmd@Sc{{Y{`7 zRXLX}l|bzLquXj$$shmV*y{I_q~Fp!{fk(Cy4V&xA8m0vHf!)(ZRwe>m(O*~;Zpli zqCypw@}<QU_%lU2c7e_a8DKd~d?{B{RYB-$2DB;u?XTa0ug+ zz%3l=p-i*S1mJ;^?UJ<&`|Nj!71y5@e-f|HzGoRFv@2;?lZ0wE6gHSG7@vx#sI0rS ztpBGAI3I`T6p>>7m|Fe1xedYGN02eTipZh13E1bJGp#ib+6rg5rDHICkpT>rc_zq# zMixZ+)d;fW2VE_{xH~#rWM5<3W)Ynb@jNsOT$F?FXXvk0VShr`>%)Ms;ukHhr#6|o zFT27{PVcj~Cgo0wpF|9b_1 z%atoiZVK1iR`nllIbev`J&Ch?I}v|@_Xnovef zhX^01g+!zo7F3Sawz5ziKeii58*jY72LCOHT&Okz(2TcWcX-DE-FRmn*eegiasRu# z6XLwFAeJBPXJ`wwDm(R1M({1ov4q&33C?on@q6@Ic&l2}%(`pCUH9{==FE`(@4rGd z5?G=iTn~@uI(M%({}To$d9Wtt{FGNTc(;WU>G%z6H!NrorF4Zx%D60i;qj~|Rw23E z*Fxt0F!U2~8{&~Sh55QcC86AuBzXYFNRnrtVPZDSR1L{`{8`R>Q0)Wp6KE7ay?LR? z83d0mOKURo#8l3Qg#TeE)bpnD0!Sd#Xl$4<|AI=!wV^AxC_IJ*0M8 z|6k6k1Y7=bn(%i=J33Y~N?ebXev_BiZ;XE#Vu;(~q@|WW>3GP0_g&;s`Ob;SI$KZq z8U8p0s^b*__LTwDSRLe4JTnep)C{UzGgLg`re<%o+8c59M>n*ert%@Ho%nUEY%E9k zy$2ag;sV@^Pw!os5%9wo35-*!VyVP07b?eZQbu}zNFAFExp3!2|2Y>+GXH`xuU0(A zMB}Pk9>t-Hl+<~Wy+AwKzj)Zq&kNrYC`y!jzFpswbM`^7*ZSfstKYCX77oUJUrV~~ zQN__U@$qja;S&M*Li1mvEZAhS3>mQHbDhr*vz&0(VgMJXO`q698NJ%3NtF||GL*Qf zRaP$hKx9n=Qyj_e%&;46>`e7!PENv z$9UFVpP1>>*lE3c;#WWj59iy5QUl>STC|mGUbiAQWN^Z)k2fdi*J{jZ8%>Q-7{7_~c)WL3IUya?{q73QV!HEPG9dGSyS1xG`NZg*AMhjzJGS=B)358DJ zANZb5X1$c2M~8O4Z7Ftp`-s}$GIbDQE-ABjvOOFYGp=TCl&(^(Tj6{PZ)gh|x*&(K zb!4Bc5?=g)SjsP4ocyc@!s8iie!I8CI}MFNr*$-_1b>gA?D~LjStU&}xk>VT-X?P9 zAc>y+)%i)rnf4M@ATq1IQp{0Rp06C^oi2M`mRk8=x0O~%1f#ESFWiA2%X{+d5)oAYUS|WKwTm1(V0_659xUeWCip36 zL%j2~@9K^5G7^^|qLo`!)e73av;WS)gL#+aaQbY)!7rJ(Hm&AuHR*@VVk>Q(pOrB! zQ)=p_L{Lj1R0VT8V5>zYE07vtLnv<6^sWoO{Cr9i;G%iQNEv#Ul!Q-`{LF~{hVyP* zT%U4gJ85MrmKE>2{x~2Lv1oL=9;R(CU)P@v?pxb&EbBqLnxZHBm?)6cuA_Tk{`-v%FlY}2^qR=%f^9*zhvJzB z*KoM?WR|$oz;M#($zm=1*8A|o(svL?F9zTE%DmlUZ_+z`b5*N0|L!uK#65x7y=>96?tyXw&q&oQdqcqo8tithjRIe6CBJ%)eM>o`~D(%btl3wkJ&^^Xy> z{Q>AWGzwLo8#K9miv&LRKTPsi@AQwY^NQpS7FAP~5Omvej9u@hz7G*}!R5xs6{piz z+jQ}Gwq9@Cy{b5vi0lbx_tzdoBT$Lp(_chgUXqcn>rD6h^BLV2+YS0fZ80a zEB>pHQHnFOs*C&~c-!!@gGF0<@BZ$rA>+O@XCvrNP>x<(EGS>6Q7-+;9=%8cM53NhZa2eja)a3*Ivs;Pvfkh`lInc zG%_^O&VT0_UR&xOb~Tdy?jMlN|5P9B%~Q%krpBUaAM0Ej>tihtS?$UOw8+9Lq+CM~ zb@Aw*Lysf5X4657_opjEwYCkYk@G18kpMk(8mrge^3lU7rpD$M0CJKU12K^i%QL*|P>9M2kgD(jBXXZix!{ZS zu%z4_pwdaIALR)$g{-4hVu*IW~2Tm9%{nk4$CpzP_Z8H6l) zMH9InzXdG3ju41c0|7b`(Cc{F5#N)0(f z^QpS|O$6mWR%8Eaw9A^CG?nybl^WQBUM-$hE8$^^$f_O?DXfmBi-1xlpnc6W9}DTw zxv-4%(4TZVQSqJF>ZO74)sIS>xdwY>d_J%mKkuQk?|pz8mv7tpJP={Ayqg@!!NH8grKfp^JZX@4(4!&2S6pgCUP13y`N#l)0t-FA0)WjW3;fGYj#^Z!j&lNSE z##u!bkL9uS#|kcr+&q2N?{89H;Wpoays=N;>(M~nXuZ{wy$f* z%ehRht&gV)Co&1V=bz*rwFEvJE=VtEql-g^FP=^ktnT%gTPwP4=YQCYw88%=r62qqa6cpY%%x*hyv zPF#F}&UX%rz1)COZHQ4h{i;W3ZSQFg+dzIH47YIg=FGYmS}O%P7*47`+Enmi!5O zqMs}4K+>Y}y--%Y7N0}$n!_6V@6a!9ZWRK$69yxp-rfQ9_5+&92Cgv#^gP|jc#qTa zyeLXWd;jlbq1l2iPK&{Jmi!Wyd(f%XqQ+8xg!h3J+Lq#Ntqdk-5(=RCN^CD{v3#0wG9aT53bO5aglR(Mklg)RRb4u+NqLjxhVs z94l6pa_PFx*@#lX_TvEL9ITEdDLqFOhM8!G64_Y$(gS<-IVzqg5GNp1z zu^zFvy@^&18MK+?emIG|{pZU0UQ-dy}f+}&!XtV~sB zD4EvBC)kc~-hyTuL;V2V&%#?ymNI0yaP8ZNo!@?x4FiWGq)(u%?7;Tz4s7U{7qga&1hx2 z*q26V=!)&92M_}r)hgn>o-Pp{CZ#m`tOm`{qH3#p3v!+0hw`8T((WiTE~Nj4|K8|N z#%@ymEU`hMi^IhXB86S#i^zY9f%;nlhTU92`z72pc3noBO#tJu%k>DN^} zv7n=A%}cf{iTmG?ifXKk`g5MFe#c19=0DY_X6(6IzR1{+A&1H4^#y!_^8T_c0L8~mpo4eSwNXqhp1H6FY@k&9)w!H_6r~*n!>GRppN?h({9d&f^oRH7j^eR#3 zD49s}fFYKccNAf{wzJ^?P{wJYeq z&rLfedoK9Hm$e6ef9#9B9R|8%NB$-C+_B5D15Bq%p9lX1c{F(C_-uZi>&s&3_)q%` z0=Qa=748bd;$SV~&a&MBwOab*A)D7YG>L3t42E6vKSz1fuqhn=`1$5*fgB6IdQiRJ zm{1G)GD$P|b{6_Tqgv*5(k==Y{DfTW!Fcm0$ljJiF=z@legO;f7z8OuqLsX|P5xF zG7@f&RY64q+^aZnAn~BVyz5kSnSbO;4Dv8wZg&q{;BRN_tr#56pQExv7|R8_=0^)J z__S+AKh5m<;rpV_I>=)5HY3keF8Gm>6ZW_HG=OG!*Qw!{ouxco^pj99|4?nY&a?Bt ztL1rsJ1eMNWRlY6FY4(^J~sR#Ch3(E<|sCloAb#on4lmwz+@M@)UzRP+&S_vi;4VK473x3Doc&m~&VQ3Ruilog!9jXHqUo=2|NAXnW$ zZ!d%~rqP4?aZ~Phi+6eaCDsVvMO4!Zmqir$&t^i%)aYs+wj%LO=9C+KP#2_U@OkJ* zNT3I9b8_CEcqEMr@o5e$D3xO41RWhS_0&IUSLk*r9K!{?7{GuDhq;^k=&*}h7Vtxo z0>{1bantNSZk-VoRvGEdlTHTvj_38;{2_nUzS(P%^rkjFYv%XYZ59Ka1@qT=&U?kJa|B531M%17}2`NP^_G5Kz=%HlLGNE3A zn|?|Fo*Cos8{4cN9QTOB8WY$Qve8EjyA5M9;;fME3!n;JA z^}}6Ev6QT0YIx3P+pz(C3Fo|+v=bJrI}^sf{D29Iv7&VB!inHz-+GR|@j-PhWrDEJ zoKSuJbT}nF@AX{QeR>=j-;px@|ZgLU&P29W^ zRa?0z8(iXyVIW+f6-}3nphF_izVSer;oA~R>=9GXf00z{lEJt~nEA%}x6rQ*3Y@2S zR2BIrp+6Z{nldx^hfL*zwVTe-6*Q3cNt9Wtz3;g=lJWHLFt+-=AIP?VAtP(c>N_5E zJxYJUX%mF4RD{6^@PehVnwYO?P2bQph+38BzMvm};W3C4r%6SGxD(W8zbAhSld;!v z2t1_rU%YkUg#q2k6T>{bS-8rIt1oUgd-(sV30#*Dfh@PN?d z!W*qDCmP!Rb_a)~p%{?;kLU&ib-x#ESyduPyW7#xx$oKm12ll02bR*Ego|o#VnCWV zHpRKa&>*c`cgC+?hkf1>7h_1#G;N?q&Q@0E|88+C`m8R+BG;Sw2jrVKxFMWYv+J^= zJIOOc+pyy4RYE8EG3VLDhxOXqN)W~q-Emu%3nr+QmHDR;bsoOx+jX=ie@XcGNO+?G zw<}2DKo%bXMkry`Y{DUb03`Z60PCyvBl8y(&c}zv8J`yyEDl`30i!QfV3OsZct+ye4-I zgBXXhy!E9bGC@}#dD=%U6BacC)VH3fdt)LY`KX9|EgO=$4?-6GKLwxTzx(v)@|MTn z?qrYQdN;>)YkbO8h1s*3H?Gj;>Z7t^BJEF0DffF6+?pQJSNAny>klEdJ0M|cBDiNS zxwyne_tWX(aVW_}lJY^6a<=^LSWe}x316M}ZzJ|C8oLJ9-t*!{Cx$fwlF?PeS)f`x zXr1`qtqa3y6|u19V${DL!MngN8!4K~Q*Y zx6Kl~C>7@%9R)Qlql%^FOB@8-^(3 zpv?wNts(_^MS%#zbe82t`!i;aHC4C&tQZ3gJ#T5*gFgksixO^X-;ObeJs6Ubqs<0t zjra#OvjCI>Kw*RH$r{8FL)r}xUP*@;{nb8DDP!Sp7z(4YA)_m;9UFP9fXFQSkH2RdBM&H^*UQFV?ML3BxaYC1{pD{tBK;;Qb z8xhQfso(d?TqWjfi8}p1*O~B}?=LCxin?K%U*1hmaxVS&!9zIRqMHf*#6**f)b2Dv z52z!hb8zIH7OOVj@#pd(;#EZ3l8ICn=6=j%lLi1O0Mw{Uz)$vAA;+54AMhTTroY||HR(i zOinR5N-2a!$PIv;Ill(}qA+>m_tga9*8VVPVCFM_wFs*N-!`;VyWIQ@B8BYb2CjPy z(*F^_^K-*TSjBPK`?1V5xSp+j*;1;CB4SjfyQRKWqoH|FdaptQcLUrFKMRlAwBBqS z6d*re32HLH_%dOizA%ag|oi7XeILlDf92Kc>Bd zED|?)Yx3z$IzCV%a@Fi5C0tGPv$;AUG6pvj%S7yyGCtg?!~{sxfuQIGMIL<0-o4q zFBXzUjh_>U|AcMyx%V%`5z1l?3II;aElG@ifgf#( z%~<`>tL=B(7d#c^di8f0_?N$;kO)^a>h)>)9Qi{I0Q$d;i0>joU|C^sjA4C%mYkwC zokTMagks7tB5waB#ueK>lOiS1X`FI$)W#5P>!0-b>Bu}}`;N3MIu8Bl=VK7@zxdNf z399r^k#P23#coBIk86Umk4x`q2W!W6kUYv&t=pzwg!K($&>m<{X6C%Tr%vS(EWTh; z>Sx^RDwxl77)uU*)Q#>ylzc0SE8Jf;IN1wb_(S!+3HidWTRe6>O9HDw`N5IlO$|)@ zJPIkpYWZRN*ySbok(I>DR(5ak;BigC;AGn$IiuS$Uz-*Ja>mKHIgV+eCsDEJbR0gt zJ|I8#)JMF_m2ai~-#0v%bgv{owVQ8-brcY=@ZL!jphOS%CxJ_!8D>jDST@HQ007?v znAsqV0RRIHBQp9EU%R^?g$@oWXv_wYoukbYNi5A&X^ zUSx6BDXTb^B$3u2>KZ>|+oHg37i(tlR+~4+exc6J2>Z6lQo%diw#1jA1Xim0b?jP zeV4N4xr35uR^f;T2ryiPn=xCa2K^U`5w4|e*;HLeF6uAtH)c97wT1Y$s}2T)7<;}g zj4N88Rk?Q*LZz@F2fu@uR+mGjZZC?Mf~nny*)pV2WxzGB(;(k9uSH=^nQ6 zHQFp1D4o#-{Brg9wLcCcVQzZKtULzQE%)|)z&n7tdo&F}tE)~X;-9*{CKrKA5I7a;&=hKQ_j2!7< ztt%?@FqTAfJ*Tw@;e`HLjCu}cYca2peb#@ANWUSSUxzLg{xxRhydmxKv~YKh*KrfV z?k3kKOAfCTx@@1SGUxmfPZR(^87tq+KfszH3^1;r^K#sSW%VUP@wR)U1&d=8WB`q~ zy6hI`PjC;WR#eeq?cV1}l|_WIU81ZBqPIrLk!f+7R&lx#Srpc-%)!ZZ+|v|Y{T*;A zsd`FO4fd((t(X~^Z1)Mh#%mDmPN=(AGx|l^IBttbn*ptvx@TND7S6v5m-Zmkbxbyt zGn@J6x;|3STSR}WtOkv;`2XZ1{+NvwY_TxFwgi?@Pu+>&nIG}&IH@X(p7YG`IUY|X zGaNu}7CEBr82f4DWC>5)clQR(^S-qGhGcJwZ1xkssxj(Z8N~5PM_Mvjm*z^#rfj6P zRn>vNRByXQ=Sm;CYEyWi>fr+oPs{u-_KpP|QdUM16;h+pF4coKLw|u2#r=hrqkLaegr=d-JbM77>G$OStdG)1a(3Oki6Il%B=j3aX#2MRSWIW#Eqj ze=DrlJy9EvR~cyk@5ZQ8>wQm3LB$}~&bRQdMaIeBa9EvqMJrP@GtPLIiWi4k?>zn@JyM!Lv;Qa= z=e8^#`+)a=FxP7uao1dRLItf6_Z7$M2xh=-K?XzRx8nUw9Q8pFdH#VUy+Dmf&@K?X zt^E0HU?#(yw$RR{+$GxQDkuS+0HrZe=vb`f7sR(B8#??5$u7LO{qij;|Mc z8`d=-Dlxw1lf>3eiJCt;!axrRAUpx2)!QHQ2W^g2^0#e-&R6PxSaZWf5K^7pt>chL zZzPA2l6?NpliGHS3)JD~p{5r1k(wnY8TlHeKuFyn|31bk8=H2K$UgJM|6*4oyLiw) z*_35F{%J}u$6{N3wpX_&xZzG7^qmCtlL`{s7%lYH_lLJlCVRGgnDXx1%JfG^&u*MH z+_oA*Y`vSSR86fJCWRGApA4BetCZ?s9P{y*lhnMIC|g$G!y4NNVIDbR70F&DoxAQg zqhmmQ=a~wh{0MBet}nvjjc1~EWRhoJt0l?HOiCqz3|6+KW_Ahx&*LrJd~~1W9Y}za zFhl~6Qm&A!@ku~zw=YF)4^RSZft$FcyPHJhqZ{Ium3QWgQ6|^8&^z<713S(=ae_MA z4T4&27`6{rxPOj&Uh?!Lp&u>w;3fAsPhvjBDe(u7Illz?W8Q-e6;O-Q7{(7L05gdk zIxnP%g8BvV|9XflI=0k)4x@{ZmU1PQQ0ASpWx@#0sP~l3s~1qjl4x)kq=f?5K5+UO z&M{yaKl!GsJQj|H_!7{|qQO}O4SvxVIpgvbQ>e(=&kT@pnl*YT))29__qi21W43ow zU<`9!Qg%Fv!H5smx}uZ3DEJ#Xm^Gg|=kapB9WSv6Ba#(?9L|Er3-W4vj$PeNz`_P< z?@vh0UVZF7mi|Ezj0wj10)LoxTG-deXW6hb%dPw=F8Uvs^;4a&?2qf6La)Pc(!{YP z;F-?|r7{i|8a|Nll;#R+TB4m2y&@F$b+JafpbjetlepOZN+TZJapC4bQ=QG4CE@i1 zU~T_Gk9F3f=<`ITAVp)#>#9-ZS!KoM7GoF5pgVOq6aR?yv*%Jul%=Y|;QPb%zg#>$ zKz%N+Qn#EjAa&2m<0&OF8uT{hd~C{xE#7)U zTFlK~t4Yk=xSL-&s~i?!258{2Yya1WZFQK~tv2!MJ_?%XxT~#X*Ws=`nGbE;*yLb5 z0q9W^Hrh@0#zq@aw^1G}uPk6)%)Azd_rFoyKwlxnzDotD9yDZRT*&Dd2|3MiI@Jz+ z7q&C+d&KFP@!JbtwnDye{GYz%S9Wn@wen})+eHYg!TapLSew#g_Cm~#H7PN{J6_0KKlR}|z`dPSG4XuYLr#GH=w$om&lT0F+b{17R9 zlc44=8`+O-dq8Vtb$X@tI@&K|zqsx(W_57{A3L3aKJR-jqEUOOP*EuSAqz)VIOy)E z^q@+;z_PwCw>UV~*uqHKXV?47t4^8YGTJ0si4^xD$K&ZE{2%q3n@WrxlaADg81(rH zl8}nuFB0ZS@`MHJ9X}Bt*|C zs4l*MicXlDRuwaGYck*~ppwtrHe)+r$JX|Slj1W`evXvn?=8)0V%()vX!LV99Rbyr zvKde27KfW-W|h!4GO``Syq1&cONv(haiE~cbR4$frVe?`I%wxXlx%DVW}{V+HzlZSu_l*b$)xT1pJ+bBci6Y+E5Szu=sVI?IbZ#8~dU5E_o_+B%-jjLbgXP zv@Y_GQ+2a#7^Lt1oJXM6!j=qoZp!@+*8Z?i_UGp!^0@A#5AQ$zDUVS4*0Ckn@Z*pR z>7|DGAoWif_BUtBAB>me5OSJ;PuD8&>-imB0WI*-B0n@##F}sXnju|HV1BWbRU>3l z{|tHybql^t&k!_odq90{uhXaNqOFqfck_+q3%Ts-ZGD`?qrxNF8(JuR)I{bQ_iZ{ z(jj$z&KA-7FEkxiqQVq=sNH!e2dL*e_e~I8@U|FiUNmJPJd!7r=ky515_OFJ+h4JZ zEhI-`uOZ%P8Jj=j9>|eBe|H_3=ZEq{7tR`eXrm#GDSSKR^RKIvP^~S9H=FMR=E_BT zdz@;1u@m(Z@5@1YzvLUrV%`4<_-P}B(E28>{wKfowBH2X#Y65}aeTPmjgfcN#iDw3 z>PC^@-X%VueW8I@=?3<@02Q(|>;qX1HORkNf7oQBHFfS&jXTF?q<&uBx!4^md9|Bd zoK)?n5~N@f*KHv*kTz(=8DYTt4Cat^P$^Iq7ct+I9vh}jeUZb(zFGyfa72zW8}&yk zU5=rycg8?4!e z9k_v4dKh`#a%NhmhvwJN3#e?itc(W`wf}`~BgMg#K%|ldL-20?g~5Xqlo`E@O$oh| za>CWI*DHc#AP;|wbmbxXwcg2tL~ko2Cw}m%(!kV<^^E}H2h6e=p`e99^~Wsc?GvD@ zB|X$!Q=g(*f~*g+0Ae^S8sCZPL`V&4#=BkC6yWPMoBe@)fH-QH=-Z$}1Oz1eHNB@9 zf-~;C8DS@v9!Wo=EQ1_S@|X#v%q<=BXPSJO+Z?MlwIFS0shD#VXrk7B-3I)b2GelL z4dpx`hjL)my5g2qqsV{5u;+sZI`l5{yBVHdKW{5F%b3T9y1^0Y7-THm}}9XzrGDvh@YvX`|&KGly`)CNGlYl5ADH(T3QSThbs^X^|N0|s6n zxF5sT`Vn0R;SM4Bbdm4@t9f2RaKPzo`y>`s9v~p4m${J+&K0VU08Z+^(D>eXwy?UF z7s$x-cNH@C5%j|sDF8(gJY@jCre_SWp8;U}2*)qUxUZ*mN_vH5CnWBjNjTlx*Y=#N zqGgiC=tcI2Sb&3#Kt>1(0WyV{aZ=P~z7m}jklu=BA;D7kd}6RremMw3*pYXJZxB&# zKtujH!N12KI_;ldVFzz6lf!=corOGg`?W*0N&wxDi^AuXJ*qmLdj4sEC@BoGPg;z; z*K-vW$%UZ(!=4arwF5~t1KwPI^r5#f;CAb}qEf6R0f{En#zZ_}n3R4<+Yyh87H^$z zdCHZHTvDwy+YgF0p611_eQ_5^p^h zo`#dLD2gEdG!hd7yUq#j?C|Jo?H>x`x&{UA``8%GIvP`B9d14VU-avdXAc=My=3FW z(RZ+SJ)}g0X~1NIG^MU|dDs2m2^SN6T`qtC5jEn%*yaXy-e;@2Rpc^#LNFPj0GIH$#fY9qbxhXG`+pj;L$n?+B zo{z}A1aGD07eBiggXG=;DkwB!*yMMpUFiPDF2XQq#WOB(+U&TYt$x0}+#nKmte1)G z$r4=joPnV;;08Uw1^aJ5K^!K1+Tk^NYZEzxsm5MlI=p)JrpuH74+?l$?v;cLf{6iK z0~yE{5)iMbbk02)cxo)m3jR$$c2pXQ1C8v@g5(_v~>9y$*DCPf6J9{(JWdi8Dfu z8U&ytWJoY5#;6M%+X|b_>T|ZZn{0!cb`J&HD{~*uNWt~#O(}60-n27EZR1J!DqC0fD#&4*?MP>!6*B+e z#t`iK^3J|-7WEt@4x({ z%@oHaDXKx126(9=vpgYEnB<5qMSPWKMAV$E1$wVfb3t@aI>cRn4^{Iw0Oq4JWSN7D zeX^IgIfaF3pl@;wb?;H(vH@2!sn8*<=xHBRTNk!(;HScN4;b@%*msmo@Weg!!Uf87 z9qPRfyxw**%D-{KjDE#^>NukxXm39AL4NQXV;N`6quB{5VtIO4Qrt-BrQ>s7ZSa?sqXEK=3@T3NlG%F{wBe!A&a=e5VdX_+yy zW98?AQt@4vI|6qmIO&a8U*$;%nov4x=5?fteaFM~>R{rU_dq;NW-wNRO7r^Drvcy< zu!~IG7ksF5kGU2XpK7M1%@^1^t^4yim0g2Ck{;-B97gNB&sdbIDe)5o4X&aOM&d%^ zd6`V-Ut8V9YX%*rwyp=zSp&On)@>ekzM3g>6UHboE_*Cc$eN1SCTPHr1CYzZ61xVl zBP7FjRTbvO@OvH%){equfpR-+-%!VJ++K$=z9+u zey}gVHMa42p>+})t0UWLKh3i31iid|oLn z1+5n}a(llc5ahAnAb;zT>o99hgtiH)KqlsIV6JM9iaV>DT~MxY6U{`gK!gfE?C-3>Vr(#M6nRu?dL({ab=3ZicWJ z^Cpa5Kpy2gBGY|F?h)GkjsL>$W_;zwu7Z{bI1d6xlr3E?<`Oy<;?-HFlbZl zPFbX^?o%Jf0gL>y@!`IlM*+F$6f-svH9xT(4bFg4mXoiB7Da+kGa27Zf+d0}-vq~T zH=mpIB^ltAe2>UHqaawd*j(AKkFQlVc`HeT#}--P9{Jv*cTZ0B?34s*U%uO-f;7T( zlO|6f|7Bc0Y;2M6Js&}Z8Q7uacy?RqfoA8EcbOb#qR6e%5_a@5H{32r4k9I&deegG z2Or!;hu@qQ7P0l)?F#HR-_qHp`R<-xlg~KMB0mpg7>OfNFBZibcrFBvQm<wAdl9OjG)-@W3+1Kx1bG<{6aEGUK*7Iv_saR6nUw!?_p1Y`8C zZH8OAXXKK>UOeycunm}NF zhe(1yyK0LZgOi{T0axRIb8$9>h5y^!EdK0?+ik^Goo{FK<;!wv$EecK4;R9PJGUBH z^NkoER-=JCIE9m{#yCpz-*Q_prqKPLBW85CP93fjV~@*^aC%jf7gpABjn7@&oG->P zX5ViY+wdae@XRsJiR~Y_St=_3&_(B87#^oC#Cf*vf*<~72cDlBWA*M`+L?k9KR!o5 zQ8VI^gOId<+~V&qVP(icIKw=5b{rJ_ytw}0u(DQS}qysm4^*sSK@#&OdqKX!+c)};}BIDdlp=*0QuY127r-&^6BKPEF}omL5=ul!F` zID%Xdg$hsU&kh8S5|^F-+0MzO>?t+znhtpZlB6wr5A1Sd$O)-=|H)*!nM{?1x9ENw zu95)#1z(H<&%Y+2CiyUblwd_lN$AVqds4ctL)0xHW59k79Rw5D<^9_JY(p68Z+q!l zebunmVj^?9O>wa&VCvF5k5wXxyKlT@#P}onVKD1djeR@=X-+1u8Kv*q6|@diCGoaK z7`!;+a=O><8+(|CA;PXerZRWb8DHjxX5JmW@87X3;it(iEz7%UI$eoMiF@Hm<5>5A zz=4CZi_S3YSpbP*XaPBmi|z>+x?>)0G;_of@?C#z#qy8^UoM!7Pu%fR0nXwWYiH5C5Pf$T2%V#a+!{xy%RG zR1I+&QOm1kuRRT{s-8mUdlLzRX@&&ylk0#7c*W~Bo}_y5As0`llSArIrf$nfZHo(#Tj1fqwNKz^GMd@W0smE*03UY9F>?fTpZ zFXptrh%ea}6(!1YlaW)=r>$sl!e?`SPxsO}&$B$~(yp9wLl|s9pI2`NSOj}4>VnoYs6p! zn=3Mw-O0R*CAg;aZ6DmTYsUTZqY{JXP(hWCco&oL*;C+_GMtBI=Zm+B@OL{A>T~B^ zT{bC=eiOG&O6ss(fy~-?I-u)Rch4jcd%O#v(WMh_b_gDk7oj<>%COMBBll<)LwiFM5YGow1X) z$Yzt$jP&UgD}5__5jZg#NpN`>9<=? z_;h%Wl5Cg{MzSAjD|yY3=4+Vq&Y>m94zl2=dlG@;Isy8XAt`YAF>hok5t)P_rtX90 zAc!A&tP-oYsPD3pv_|j3NLxM8{6QGHWk)m>@dpL;0Fnyn?64gQ*$0ga3_^?mF-i4H zG5q0vwX?8L?;flcCi7f8LY}D1=5QE9EE#r6t>7z&;e16;rc!;5@Dl7%>^`el=tl@A z9B!3yKrT?rsnBdI5oqa|_F0`@23!=6p)t9_rsRqqJCD-cQx99Z#A0jmJc2X@#c~5B z(U6%G;QOcM$D6yQ=Va_xOkdkyN+6-=AwxA>b+bQypSavHxzG2yq|Sbx>bXVa&jS0r zGaOtv6tyEjpXNS$Ixdb0k$YDx`N$p8=0vwQnKP;4GtgSVJ6I+B1ZRTnF?1z+;20}E zpUp3~+5AA<&Wx`4%E608dS(Be{kiRWTkrlKyO49DVbB#2jYyhi`R0RiYrk5+Ze~$d zpG!e?8*nB8Si6tPN1IQce;D{glV^jzS@VVMPdn|2nSad3=4adGdvL_&PY~ew0UG`r9J0_}{bZ)zjz?7s%c?e#HdZh6Hd#*g)7{oKqA)NI~*=fsO<*?=$0VD^kd=noQ~%4d|YipU$O;n zpm83unc^0Wm&yCKq5#v_Gy4|$KdH;k&%*S7Oas0U^jQ>!eQc@p5WBq<`+*|*ufGiz z(kZu8krNX`@G<>3D0o@K9(waH1>lMlO?W^=yg`JHOZX ziHJ>P5q?SsrZe@Y>Xc5=hy71z&zDj}k@!Y#oR&`Fw7Eot4<4oT&2*&5pyD=*&prYhePJ(>6Ow9E|925}Nx@?={A#AX( z6K@6|%WCWbk7dMg27|tpL|>Mgk87B=n|2iIoftu+xD(Ud>H4wdj9*po6T6aE`M|~# z%FFgIGBs2U>`~|I z$X7_3bya@^coqIU z*=3g&5ItStys+cB(7g668v$Cpu|GUpo+Dp>S|u(C$04`PTui8C3jLf8{g^=K#Ll#o zB8y$n1Qk{=c+)I@wlmLQRphP%;ve)N zkCtZn5nM!QV=l>2)Ul%vSkJBLRf>4T$86Z)d8IJsC5#9U2=96tG$)p2aJz@#gX=u- zBh*gt?}El)4YF168r6oAsjHORn{Bsvw|Hg71IeIbi(^!lwQlhBcC8h!%2#2N*g|bh zw4&C0zh<9Fo@{~PcNo1D7M7Oj8Ywgq=SnTRD8GZ|LVp+k2x}dW$)Ad#to5?CH+o;j5gEgV;nz+``AMqORLvHnPSr*Y@K!){g^a=mTt9bXh-mcXiyj_dRer zT=DJ)%O-ww<2c^MUUlP1z)@{0qk9*D%6>HWKfCdOdri!#egHi%xD^Y_{v1Q?5SB8t zjXKPh%A$sfQnBW8SpAB*MSr^3#*Kp5%BMfXV*85*APK=)ZpfveBF-d#p2$Iu3MsHx z7{ATt)BW*f>x?ST^fuOu>3($0_X=d&tt8v<6gkZ*-vhB2l{-oCi{F_Z+#Szr+#0E@ zYP?>#|9A21M0gS_UkTF45p$U-j(QVu+s6T9nyFthy4?F)8lQjlZ1D|4-sMynsSIj} zB9-I9CimEPDR|YTWmAfzE`T@%LN=Tfd|C9e@jpDVH4g|bjBS07BLDqwAuJh8{WoCs zxD|bxUkVk(kx@#d|J8Fx_pf2@W5U_PHyj(uuhbd+^I_u;a=(LOk`dBwe^c_*=81n!%O;m|XJK-lj^bpZgQQ z+^KA^FRe%D%_sjZRgnGd1U9Sh`QrUuXoZ4m(g5(dIPXi-@B_Z<0dZ{yT16{f7K#xO5nD ze+s;AgP;7m`A$@Pk8 zPS`*rmQ8!o;O)~{O`zJw`9@CX^__i}UHtBe5;Bge`7d}jA=9n>Sj?0Ho3|53CBTdZ z1nCBh<%?6Jv5VI1=qP^i7eW#cAG>c?r0CkNyF(mO>9Zl0q}Tfo_g4&po^x0mwQd_i zsj`Xbz3D-A;KQ~~SR~O3cKF5vkWHfeCkR3OEOS^~V=yNW@g+WzkpB&4J5{sC?>cvR z^b%x?wd}>df8g!?-0Wo4uU*=|RXyS>`er_U8#sEn5gXMNXP`{6V~f(zL5>R+C7znL znLD~FDAtxh|rjPD700^32J;-kt)DQOw%OmiWdiRkaLEw~XTrZOkEowO$NfgvU zW3HB81)+hd0Ne|TJH`+{89QhYaL+9>m4i3B@_`{7m%LOxumsw>nnt!x?T)jb_X=~j z{p@dyspAPeD(Le=3%?>Os)^^AoP3bIROI@`hB#SlVrG>OR(9ZPi(Rh3R&w_m^{Bw;{m8f^tR$2$_D(7pLf=96fYt8M3-eSGOUmmV}k`_ zx{F3wUM>yrTpD00tM+#@OsSS(EvV6(U(q>;zs0#weZxLoI9oC?mUfi6RWs#$&1^r4 zIQXG<_a0T888f6SWn59p0g(|mO!|)UQGUJLHG0c8sEMm1`XcV3y|8KtxQq18vQ?Pu zmbr)VpZjJBnzVY5&zl<|w`ERE(2k^qZ*I`dX9pyj$svnHpkQ8Ld)>(XW(%u#pqpU&-j*JH{fIeDlm|2uHU0oQ`s^ii$pH1BMGNLol zW37$^II{IN#K+n2cOU8Z$JLBao(YYFDSjvrf@JTD3_oq40j=Ru5>_&<)pnA=M7w;D z+;wId$j9aLz|xpcZQkH7jp!YODA(o!BI>PHh`V;AXlw^EzK~s(a4U4Hb}auh|BH7a zk}LI-$R25Py=+UCO}huFi&hsOX2KwQb85`=>nk}QE~vfs0O`ZgC0S5O0lVs3L+etu zkL!*()qr68na~KrU+vvz9*PD-U9#xayQEiWnZRU5|KHuB37BPSbVkF_D6No`5l*>!2Xf=dG^b&I}7s?k?s#D#3_3 zN#G;dtieL{=&c0vHcm5!4*wpsGxc7(KZ2n35y5i z-Iy9+vulk00Y*_*5rH1K@tvHlOGw6#Su50uP2N^?)wgQxMxcP_#u2NkZTus#JyLxN zP6u1~eEVEy2cd-JrivSER6qNZz9_${K5o2`J*pOu#A3}}A4Dwj*gr89CKmVfFCiU= z3lrWea^@keNMviDDF3tUFkMf+#laEiIWNH33j%8uZdRR>29c+{xqm@}bH#(#w0Bkj zwMTRWUD4Twjw5@3FOuoSzU9RcrL;82>YH_0_6nLd$GDd?OL8Mx0Tquy3>gEhe*+mo zXIA#CKWM2n!kNI(Uq7#gk*{aEky>iSM%Fn9ui`?G67ciw_lF9vMe8c7WO6xh?WC4iuUI3msyhI5H$Gh2~3*Hh1*`a3$ zO7(HkNGoWj7CFR2;D7?bb)O>pS-E2b$z(JEEBhXB*mKXg{h6r7yCVmM2Rf|VC$=( z=i3iAzzsT!8B}TJ+-pCsgbzXJfb6m^`h0(|1EyyTVNe zL>B@R^W!es!KmkhQIc9!GF?~FwNk&%Q5avD;L;oHZGaMxAlJU{<8~#I2Cux(0~eqK z2YP-^JO&M8CQ2W_m%c#ma}>>w37NWGzd!6^7mS>zE~EJ0PjypClK9!7 z6<sMwb>n zpU5K+^bC$cwvP%(=*8yqU!BHl@=*Pp>y-UCdNhN`_zul5JrA<<=na2%SP_jGsx8@= z?OP+?GX60#YI0^iMrFePL8zn8Lp8<2F-VRy*YHZVHp z8SAiF^#AuHn5%Q@M~4G9AOXlIOC~k~%M^afx$hb5cXMA$ujYd^8%PTK(24Fm7Z1Fu zNdl-3k4r_)1>Zy90yNs}1FJ?Q22wg5A>!t^=z*=eHDurZL?U(Om|POjgQ z(9wHexd;V|*$0%OO{0!lAh~l9F&l5yk!LLXkAcr8wg`)&k=b$9L*T#qWrxb~hR%Sw z-gfdux5DQ~3K6@aQ{^QeGGw@k15Y37|JQTvEk2+!trC~9#7+jCY3!Ylb~h!G zTStKyQ|Fb=PO^D1SV=8P%zIJW<13+m*O&6-H!lOj(?8sgdPaJgU!bJmQD2xXf^?w6 zcxS2X3*A9w+#Zv6b4Qwu$rG85)5Kk$#<`BH7*E8{q!MbU)cgK6m6UYbiTp~&jsLW z&*KWeTYp0~x`IP&f<8!Hi3GzxU#=7h)+FVVW9q1T@V%-XJbo>z-c32_{wk!+fBp=e z_7wB_^==d=Bxny`R_Jv>8v$k_!xk%asRhk*$;YPwsK2SuzCGX#%0TJ`!{-Hh*g#v~hxh3cF&*Zt>6U%B zdJxoKXJIqK=+`qfaV$@+ijziyqK_WZOa_CajcdddJkJSw0g&sW4YF5T)gRlaE!BAcB ztzvE7B{h%yAkr4yLT#~m@euw-Ifx_3a$6qoG)jv29rNS%`1}vZ<~v;K16ouZl4yF? z)O-*IHw5ILM=G}A=?Cm95@r0x57NhuOuni*4Tks8sp1xUP2u7MO zaBL}66X{W7T5j6Dz__52gPI}qR&arJq!`26Id|f8Ok`2qEtDd-TDC_agtqe~IqRTE zDao!RdZC9sLROCJGUWLaeeJrBc$S;}flHVf<7&r?$}!mcepBQ`EuiacSr`MZq3_#MSQ;un02Guo(d*u| zyfo3OkMe)b9mC+s1yg$<_fZZ6j__&(hRjuw69!I&zg*L*aGSVzmc^lv zl4Pb-5@z>E>rY7bHGuY6W{DLSkq9@UO!2^vh1UTud+{MQgb`i)$Bz3O-EUNXhAFt$ z-~;j^&-c8M@Q=QkDEvs0LW})5(z(Gf-mu*6h0z>?6{lSF ze3kB6z~JOTaqdZ9%7Xp!*MsPC{0G8-;+*4d@i9@oMx#g+XcV-zT*e^J2+M?=Be`u^ zK(R>mkJVE)9{AJ_T74KGGlZ$%hSqgn)j%rL{42EiR-vusBH0BdeuT12lR2b4J?qwq z+#P5-R`)qMGLGMCjV5*_*^YwXzq)jxTMXZ6lx6bAL)Qmt50=9WA`kF2Uj8{BH&|`M(-5HN z;!Pz@Q?}Gt30Rm1%|Puiam+KtOO1iNB5!7twa>X_$j0l(k@gA+*w5s{F1GC2(ObMj zb64XhDf9FvH5kTuYJE&?X)*@@mtU&w)vx~2(?mx`257{Ubz2s3KxXM}-bqW{_UB-? z+~=IeQInU~Od}u$w`9|=8vS4Gtg~*QeuGpJ`y0`w@kLR^aj};9<<_%XMMNuxc;aQ% z6WVFv6QKC>*ahMc4f>aU3O`WCF*!f>c@y=mWCF+n=utnu`G!Q0Z_M%>%joGLB6MDlS#y3|9GGlA zArGfT*~{te=5vgTn3fuJc#i^Uo+E>^Ys6ek1{RBWjuk#;DOUH@Jpn!^_w1kn_D8$O z{EOvGzG8V*Vk%$i+Iij9_gJAVf}&xoJ6iU9pjQ!DJyq-03gq1J1F%bY3u*Sd%%OezOS) z3$J!}l#B1}s_z(8IN;gbdD{1QGv32PobOPq$6Di?<>~Xe2fSm{_ZNr^s@dg4%H&5~ z5^zaB%T|60q^%+-LQ8fgg~F|ijN*26VR~ZO?VH?}x5y>A4CyyJ*IZQ9_EAPoPCS{C z_D;%3+GkhgsSZSjQSWG=zox05sgEt?4!0uIw&nWB*UtyS7-!vdQYUi@epYhivg&{a z0ryRi>2#vk((_AFwMQ!dXkd?i@S&qDJ&cKiFHe^BHv7m8ROX5VJ*r3ms#8SjHziR? z2X_e*^q2HP0h^1VT4s_nl?^7J>SNwHpAlrm3yg}G$mtF|`XFKQnrPSgr|D8b(||EG z*~+S485Q^%RPEw2ZW{B0>%~9z>LuN0L=HuSREYN8!*PP4Zv;N9WhG3LSrbSioDbg+ zOzPmRteOC8^o+*ce|mn!d<%bA&T?pHYq>&BeO^#JbMB7vkM6gvvOf2G8^>J{p|VmQ z8$i#cIvW@u-#i5%dSt;_w(~oXFgbCga(0znW=S@XWJ|27!Snynbk=WC{BIXmP*55v zX#}J@r4|KI0Rib|{eW~z=PunL9Rf=TNP~1P(%mK9T}v*@!ano;;rR#VnwjgJ_kGUm zT%;JfT^=eMAJkqWU-iYkP2t)1gc#fG0UFiPaz%vUFypx^j;alT2H-(#Gi z=jA<4WOM|#zEA(PLPLW?Xk*|q5(!;l0IV3H!jCEDL-4uSWTi}et2fIVxXSHWUyx2N z6{(qT^@p0LUhM&6;u=0U{zJIu5o+r`s!nc|l?r7x7@F7bTtlr+x=~Y6dp570r?RTkE5H**%1m=^Hb@*XQ5mN9}GxS<6YAxxRD7 z3K9m}nA_-x@Y+3pZgY;b$tQzgxi%27(%7=OX)gNoIw=M$Cc+u3(8ur-9U3l17s_{Q zfd|Seh?arXT5iH%=cGo~zZ9hMGNl;-zrR9Hs59yRDpJL_-tVFrjT4|bJC;oP8 z+u7U8S7ltVe5 z9E^#bAJ0$ng1@;~k_MaT_irhVN8^PpJcCmf=bgj*Y-d5GRtj@820B-8jvu)rON#cs z3!fA7)A~55km=_0NhaEx42PeCVOL|Dr7PG?9aKGm(2h_l1uAWHSzKvzZ&8UddUVt9 zGQqkixnPe>rRs119HvRNX;=ic+d;TxdR{uep|m!P{uwwxDY}U-m!z4|cN3Hjzwfns zE}x>H<~*?0l>{d{^$XYkkqD=+UunHSZjj1%BwSf$x@MHrWE|8H5Po4TIX2U?Hh(+7 z{Dk`K#T3Cu$cOOnyE#j^2F0lzlRVSpmbmJVo>`Wo^!OpCD(kQ8+(^ca@F$)=27tQ@ zn1g2jld%3;xG!m37t^mFx$!vur%Y?TUua{rW*egc+IgozW3%MSMA~L~U-vu^ zVMbT2sl0B(>5eNYHFWvymC^doPeR@P3fd=RD`!**&zy+Z>Oln+8#TRS3!-A8W)w0w zjmOkfzl(Ww2*2k?LFY%eSh0}|TALsF4;FEi^WuZ?4tfQJ8;ZoxPEGn$s4v7X>+=5N zE+ZSdOL+oJW7&Pc_-uUf{*WE~pZRMfN zdidbEJhp5z+3G?;uAAcFG1*Vlb+KKmy8J2VnTeOX&EfKm7}U2iHdzEovb)!v!;Xs1yl|uQ^F*LLDFje{{%*MTSDx5Eq7y?YB$Q#lzQz4ox9V$>5S$p!T~LE+mpmk7w0m(rqE4 zIdB&bCxyK$O%tHNP|McvXx=p2l-b7&EH$|RH6?1cTotk9q_IcN=rE~V=-QNIoTLAg zb#W>eWR@M(bLISya#1P_;!@W$i8K*9js=?WG8e; zFd_r2MOm|2%${HdFINB6SO99H#2mZo+Z~mW5#{w$2F)<$vI;NtYB7a{0P@OFGQ1&F zpT(Y8Z8u}^>})-t0(_JbWbZtA+w+*vBy1>{1XnG4unu~8ThbLx=oh$O>F#XG7UAj@LM24% zJ8lN)ja8)m>XuJ0-0hlSw4gbTgY}T`)Z^)I4_d0bKDOmiPMR~Z4}w=yMX?uqf`}pm z^}(e>DXI5@l*fH;xQ}@_^0j>1I^)HYrYV5aX1P8-{EYFzlmO6IDE9Sv?5wX&F)2PskrDeaLxHe8t>)$P zZ%os+2a9Ng9F?Lk+Du04__do|VDm|SN`m4|qn?a`B2OxiLc&dXL_zCtC^S9g*@GC9 z^~4NG0Gf-N(Cl8a`g+h<-X)gF~AUAnaGI~leyG!i!d za^=68n+iR9K(;g%Y=@;jI0-hiHO}V-55;tamVfT5`&ikAb{?W4wB0BgmZjqcY)_Me z%)K4e4p{r5wS?!bCdlQeQPACy$;b#k6tA0zt)YW5)Xnvf7=;QR57iDAL8QA^27+O|P@aRTZ|wD0Y9 zD(tc@@k?NoKG1x{g$tC%R`zzj>E!1P*l{>K6NQ`KM-^H_I3C%Yi*Ke%eX^6c_%)1=sEsXzt7Rz`<`0E@Ci~`!K zKwIjH|Bx)N7n$cL3X*9jmCbc#0LOe&doYL{w=62}V>TK-=KMpEz=%S}aMMd3ySj5R zik=K2w+dD17h9<&ubxuaT-&jLTz^*ZSm!5MCpd z=F<|9hb+wwwmpjZ+8|mh*YDwnc8yESUU8D7yRv%AuSId+X znNjCDT6Qh+-#X3{)!fJHv@EqVzI$!SJg>1Wku#?{;lUWo2adTf*7@Cqc?i!wp|d

    1+XBiO9~ zmO_j6%{i+iNJ{+hIfmcdb=tWyx=~9yVM>88ZtiWeDIQ~(!tb{=17eQi2Mv;{p z$W@z`Om#?9DlHI~>ZOmh&9#3n$>P{{6A31zV`?=|?4_jePhC_gOriv5%o*iF2|R4i z^|&}74W13T209gFVT{TI6?L;)ln*i%mBmurj&cYnj-MsNBIW!T+EQq=e{cLE9y9<+ z{cmcYxn|+{p=BL^M*nEOH2#cs09_eZi6nMlr?aXG2DEr6cf%UWHSd-0Sj>3Y<5^FD{EX>@F^V zbq_V-4+*9Imm}Ra@4(^I|EBs)@jGL}EDMs~A{jq?&3zaz6R|{EQH5fyw-^OD){6dj z$^4~;0J1<$zfQP3@&kuEahaof{9I}uP{JSd5t_Rl+8-Bv^B*7og7#7j_-<;(Dkl4NZOw*jqeCpW`Pt5=J{1|a0}%uUPDb^71bej=z@ z+r~&P+OFXUn?7Y$o6fD2pY^{L(EBT1#{Dt_km5_L)CH{X*|C_bh;x$O4qCk|$vDq#kKdat~COH~p z9mKz^l@_3OdN{bUc@j!yg(ihv#oug)sEmbjrer@c6j4^n@FWN#ct-H=g9&N}AUu#LA3AeR=U0Mjyqu*hgqKzaMD{)g=N<#2A} za6Ejckb8UZ`@Azu@dfjjd^(i*oUIcq<#+S#Hfh`5m+nF_;uQ@FipCBWxI2A9c6yT? zTVYKYJd!^FQwSr>qJDsGKG(;2vO|+O>eL;Dj|br-l3x=j^}FF$1l>P|ICO$m zYKhvl{RsvFk^EDzS^K6cjao}}ux!b48m!2w>8TO79;jbThiW ze9(nWn>8je7HME=PY)_7lh0>dtpmZiSRI_<@PQIZvl^3pZs$fh)lqBU$N8*XoY3(M z3njuoW7`!;p^5{~-uaTp!&KI^!chqVK`d8Fes$$B?Yp!lPf32pPasXgZvV|gUym9u zDn-XC&+O&RPrK?^ByPa}QJ1m>)ifC-JbMS<+#$OT9Aaa|wj?O|mscJW;}R{4+qNch z1RK3bPrg$9LjF4OmF~>JivFDbES+?e$1f@Jk!&GH8knYh#_L+`Td94B<6*moh1l#j z`!OoxI1H=80&XPyktz940xmz+p-yMML6_bjV>9v z1Yo0Pjh6V3Lf}+jVfN&?9ecE!Ho+H$ll){fBV&++E5-qb4SHzTO*=Pf)-YxZLTxB5 zRL!0`zhl=n_RWrxOqPcri#7L?b&)X5ejy@}&;3j&z5b=*{EFo(Y>RXG8PUFFo(a}m zVe=g}<`D7~$sgm>Q0ZiyeB%!-o-bznwxNBpKXty+F~0YzO973{jr8)slfy!|g>YcwG{$9$qZ z%`QKs{);2VmznuX@rwkCOQ65Sg-uB&Z?50|JM4MnE~5RB@r#roGZZP1cQ?9k(J9r; zqPa`v@jze9aFo==+7;9IZ@>FiyY1h{GHbU#DF15YgmJ^iFPyh%$=oIL=PaQrbF0@d z6)SlIzpSe{2~`9S(CN~d{HVQ6ckB`5S+9nqJ8St5BG@OsC|JVJCeIlaMe*x$$$e+F z@6v|Hl;B+eVe*uTvpJOFSH^O=r|*2}rN0efP()7TP#F;a-I8T+i~Ow7_Kx=)lt#5A zRID>g_-Wgj36E6cnUIPU!~-l75=h~wl<#P!Frbp>Oq-YC50{1;?nY87#aFQe-+Qdl zxH(Tv@-&Kw!U%;&&V;0_ebxuc;wzLQN~s1xNWdfkg4!RV2G`1VQY!6$8>E#M)P_nE zgu71rT$1ncOcp;7O02sd<4mrQ**!t~RDZW6Xo2@55;a|Rd#nr#Ojrk^vx`e}@BKeb_Ouj@!CKCBA$%i52 zOMR>o_v|EUqEs-F6))U&;X}8)czM1Sr^#k1$)7nsyId7Yqc2dP0{owJhkv9N|I6!0AzCzKOXn}( z$M7H9|5{C&EB^3LJz0NCf20QO2|w*~Fdz@X6^G4XPzNKXBk#W(Ki4c3MoX$@rG(<3 zB6L_W=wPL7W+R2y1VT5v>bGK|j?=Y;fLN>=L6n>z)yF$|BAmml{wi0(eSrN&7ZxnMXM%nzwmJ-AKZ>c zK|#E5en=nA0Fj@hDm*RV)ObR~Cuevt!R^TA@w|^r^Bbob8U8q!@7%LLxtxX*A%`TIeyHD@q9`5ySd*jn7e54f@Nz~ujZCls@dSg;(kS%56DZ6Wg+*}9WTZiP6$FacnjQb z^izdla>mb$$#eNqK%B7#01QE|jE4sM5=U~P0s;s56Lv6JL}dzi!+(nSNy%|i{gb11 z#lj>XY>6LhspvP)-1g$z&oFzLVwJ5L*Js~!a>K?AB7!8sU@rN8@Phduv zqKG8MT%_@Zg}sj0B_dKnq?qK0ucTtF`dvG|tf8=9LsGFa{OiPTW{ApJ9@kXyef5KXvG zc)?QPhFG-DoHDCZH>Nx=1;@zJL%#CJCS*3M=ZzHHS}|i{(xhymAOkGY#h=(V zn*3Woev5@TC>Z25ZQ1zhzh8tg@#3B>CH%Mx04rkc_*Fg#0^pB`U*2Rq{x^pY*ndYZ zqsS~vaQMUVYi}XI_%D`1n~hptH~3Pw#3sV8Mt=SHjV}y;_w&r6B0qe~hMAjvup_aZ z&n;cHa`5f{c>RgDeAAj{cO)e)fONhhuy5DM1a=;>7=K=3-n6npJ=gP_h)iQP1X5P;seT!RzDXicw`8 z%3L;>K?izK<^BBN5wVS%HB@;BvLqW0%mNTkcFge6ah42pA>Xni@1yqxWm8$q0SOT{cZk5(c?R zb;b1dd|o8@FZGXMjKJhKmQw$y#fV_#Fvw%%U@z=&x77Y)8q-l_nLJ|rQeqWj;E?P1 z-%85(QLIb&73u6(F!g@rl_%XY`a>t>31dS<}r3 z!*;a<1z(l8C;zrUU@MXMWYQ>hzsgg~m5={d|DoYEkq_PKH; zFDQ`NKm8|Bpulzc!5`L--<5n0Mnw{r@I<{KC^!%PWfpng)BpC4^Ema1INBd?VI%~Z zg;*heW!nBg3i82&O8qmWvjcwH-!zaAP@pLKeHOnlfYIX2Qu486cr}5Y|9<@JlN}Vc zg)Q<1`$!Gx5}=B(0EE?7H;Pv})DMR$M~;AylSmB-u1fNYDm)&Gpo0X%5BUiKKU$Io zY$B1cBp);>7pX|e3P>1lK61T-{v;l{vxDVY6t9JaCy^KbchWY;{g`Vr;Ml9!v=-(6 zFn+)mHLis}@nZ`cmX+{BPU6Spa(;9Vj8d9auZyswsV4H{ZxHY03fmeCPjkt#VYBY| z(bP4(O^jU>u=mtuxcIJPDMS(R!wP|JbM_M{d=9GFmVGuG@jmm$b1>9Z72bRCqXT~N zlOa#NW9}%6mKwF3<3pvg6hHB6m1Mh&%I#Gd#$|1F2<0f>STdLOQjK3EoZd8V$SiJw zT140~d(zyFUE6|x`b3}X2>F&}Y0;(ylC89?zrkVvGX6sj_k`1Fk^z8!%$zb4IYFT; zCBtRV(!MNIG)}F=W22@G)E27Hf)LTfF_@n=k%t|^9twy2rfr(ZVw8Y?J=feMh|0#x zkQX+mH&5O1@*9tHdIQ2pic)3)nFbXVvjNrKilK@a>m*r`D+)$!TR0jDD<|3U<%i>X z@6b~sKpsF*!_z1VP9PFgz${sYqyQndw1~p4Q~ZdmT(*Ln_H0T(!bT45iBXd+n1@-J zNxYA2<{!Zn2_?Mi;v(1ND71?>khJz<;be&|8G9`;h>wJHCaO`8Nv#= z6MuK)R{i-@P$aMC|BfWHwlWn9Bnd|_EFNK+)Us2H6Mu6UA5jkQ&c*M%`Oz{PzX^Dt zPs*%cUWxN=7j}C*_%}zwj~#4$U;LudGODO<--NSv`>D)myH#q5M1j97*j?T}XBkZQ zZGa@7aWjsM9B&u`$z=Jf_RkC=2jr1MoPs!}HX+ylz`i4$&Nkzbl((PxsCVDZ#(z0U z`2mSi+wZm|*A}C0BX97Q;UyCkad{=RXNBC{>2nRQFd^w|)VRKfbuJ@?1-FHN8f~r! ziYn?tO!<5*-aaXC;c{N`MYa?Fe8JAgCX-wh1pj~_2bUFvK;MBZjjU0t7CBS<$v7j- zwG8r;71jJGT~rnHB*Bl&M}c5cx8;pu!H_e_ud1rou%50EtuO6!iGPxG4^HEwqld$l z%POxv?k}T1`HJl%pLNNO$m=S}4@xxw-gXv%a4y=x+nw-Fyw-R3c`QO z<9Cy-dtCOn^RGDO#t|QUX~rOf{PSln;;nw>qPBdFZ^5p-`na3ZB#R<9fQV=gr6Tx2 zDq4*n-7+6U&zv+j15%XVpdlY+{15RyYKc+^HJoV zD-p~!13*ia)vr#ncKjl19r^eG9+S*cPgtRyI8vCD_&0t_;!Nbe{$6Wy~r?BR4m#g z*}YguNb-x|Bcq3cDrgHcQo*3z)jI9Z55zCxQs*EjcmPab!L;v--*ovH1Yxg?*_*gy z|40Q4IAA9(c9h32hh7^rWcf;7WT|eS!M@#YTQ+FiApBqUZAz6Je?|GYP&WPy`E7W` zIBLn74yEW^Ja3VMpo|nHW%#50U$%6)u<(D`4U%}WZ==?eWbQN$3e#i~>HwrTt(6{&Iph-RtNUf#ye#tZ5)p(gkrldAh)Q=bX_DFiAGI%Th6{~Ez)S;|uXsQBEIe}2B-Pqu5< zy>03Ig#vBc@g}TCt-aV_#Rq+PiJOfgc^m?xyNNUuVhvAi_uY59&9?55PWyu&oW}DD zfTi&dLN@6CDioajhtZw^GJrVhoP)WM^XQF(_5DOjGA@^!FlK7E-rcI@N3{wo7WpxH zvi6OLPPYS7uHvhyNuut9{DpIQ952i+_Fv`g=dz_fVkrqiwe@AAL$}sh{YL*At&{*f zh|eU)BwDS;4a9Ft&cet??lS*x)TCbHR&kMCtUeuJj_5?>KM-POt=hHv{B+wM+jIxH z`IH};t(yO$|NSq=&!|v~8c`(d{}1D5E_QgS3&FJbKCGhnaD>nnmFm~C*~2y#52HHT zC4k^w?#4PGrHqn7A}Ns(f1=$8iPj?e^h?Dlkm0H)JjKP9oo&ZCAwTS8KeZ4Zp-{Pe z9I;SXwR#QnvU~G|;Pk4KjV5if7s_hpd6dKUnG@5fvLw<@Hlhfmf)R1lagvXwu*f6M zj3R1zw<$Dh_^VJ!N$!^{tj5oNm}%~H9v z^y<@d{OC!*rcto2a=Gny-MUHBMwCH5F#Hg&wU;C*Irztv!rbW#B`~W?%QkM3PH(~x z+(g}BVt%zBQV}2v$+{4;W|9E@*;5yoF&z_szFwo+d~S-@?Hsd8Cf29Qsj6IEnOPVy zN+CY6QL?w#dnik%4F^Gl?0PQ+NNd7J)#iWXv(k&)Ksm!)`*|Lnm=mzI4}QZ0{uwibS=6tY=n)wJWL%|< z9A`>xyR_}Q|91Nwzh~XrwHzSW9>9Ejw<$VN%4%V-%>{b+XRn5z*HWK`S-aGcwdIH~ z@N?Pv2jiEr(OL#y?1Z<3E4pLl^P3jsstnt+X{>fzmvX)SBj4Zny-0!XDKNN{MUHIVZZ5KUw-LsdZ*E5rUHfr9W zC=ldU204QZH*jXwI!ThRY07KUxdmVBvVRHuk!t^7y?PW8(#gfOMf88@DUv(>3zqjksDyqZy`KF^H@N@*rDSpJLn)%estXo5f_ z*KxvH@ykHu=NBFF*+HYTN+2J3s|pjxO>4W6)6p}p;kCcKjjxuZXJmjKoFtE-O0gST zb;wfvYPgM5)NP?quYT<)BJ(a4`Nazs|A_dtTM1sX_=VnG59;&oiysHqhL7E2<(k_r zdg!{rzt`!Ntq(P9-T*M{l`0^$&!L``Y~Up1(20xfL+dWB`yaHE{KN&x)m#xEtF~Yp zrI5&9Rf^wI{l?jlS+Gp}w(4tCaNrD0}6bQLE zezp}*e%w#19)H@K(5X1!j74|!8ah*Z@Q45{nG%ARz$HF-CWRVAU3vQtH^?{CGQipR zS*_$X@Kojaq1gT?GscHnv0nV5#C8DWF}@-v3bFw^?}$bFQ>tGCX>GtmPof~yM$K~o z1%-UKo?Y5^Z#V6mH0uhsn0LuyE)eKxl!_I5RReyuBTZz;2Rg(V*hT-en$#xzOg>Os zcWC22DI5yPv_CVaGPh+$O2SVD(ct%m99P+*nUx{{gxGw$E^TV?nT|q^Z{&rvV6GUy zv!>7YYp{IifHp9@Mz4Hxaw&f%A99K%3tf)nTXkk?Cj&3;Hc&+gKKY}R<>+&>{t;7S zLO~+euNeXl`Yli@q#Qj1iO!HMn9yD1XZF;2w2-02x`#t@m-^o%pKkyWzblumc>kqg zfRJI71RTWl?z_ppC+xn}PQCaTc}>6Hkh}U9fBE+tpCPFfzwjQIQncWc{1qW8EHkcl zo!X~dbHZguU5}7b|I0SL(jC8j=uc1m1{g*7xi%ZMqW#qCL}D^OKRt2J<1RQ9dGtHB z9mVk#`~$(1&_+(jZ`?(|VPoj`AN${6O@91SzTm0ASTEM^hQ`mSWlqGAJ8)|m z%7f5o7CEfQ;-Xx$P=i@=u%yMQ{p4bY#Yqso;xH*mvZ|FsLj1UCHgd;4+x!-tUBy@u zEs9i0S}IvYBM|;am4qnebFf;zMM&X~4VDlSa%4&Qd_GFiW%9K|?SU4Izl8IQ+2^vxF&2mQ%!D+VJ$V4<+vp?y(@d>zEEf$IW_FcUeI z%YV`uH!eK^9=bUhShUR3(#Vw>HXr1)naHi;+bx;@BSzXjzk{FTKVWJc7wMIZDVcl(xnEo` zEhVH1^cWxi;~O`MP5u*ofTu#mf>av~O7fo!TTwn@>(s4v)(t07Ew$^`Y16Sq+fFU( zH)_BzA^cN*@K58n_4 zK@Q=E9X4y%t99;8zaTv8)T`a9Q_FUpTXP5AT#liN{*T?!?b$upUQk-bai}C*X*x@a zKRvZXubrf7?d?1h`2P3}xru0GTVDW7AlZ&OG%>6wZ$Hf}79qRgq5=FToNS*dg@F#3 zGo!2V!%y=6^7bLi70OclBE*Qc`7H3f#x}e5`s}UI)!K5ok#B#&Zpl&VHmtML9$Uq; z^u}+MqlDCdmB4{6H74={?How%MSeCH%~-U4{M!3f6qp3my!;O~*y5uSz++yn#6qcl z7ceVlR83j@l3f}X&RV3Nw1gbG`DpcnN(sysV^($ITqk~Y*GLaq>*Qvbp@{QKjV5-JIE-O}+K zNZQ*PUpBF!&`expTPuF`RY0gsy2#09tvWUMs151H21n_Bf&j;WEb6k#Gxfo+qmy6<)50IoO_N44tJ ze&UuF-X8pcIV@4V;1=L-vr%igj%0CgCN?WB?x)}v4Rbu;GQk~BQ5s+r0PFE(EqASdV?wps>vnT`Tq7ZU?aY4e5+gOeXv+^oMfHy zJt*{mYeB&ALvlJrwWp~be_GDzx_z838-K2jri0>~``D>c#l^9rvNK`p;TIEnS=Jsu zr*oMMIC9dihXYkmibGWQziqm-M1DzfDmd)Tk*il!^2}G%RN{WquhwMzF56dfdNXqCI6f?Y1YKi zlS$G}OL~+!P?;Ho{xOT+7&wTMdM(ywoHAtPisJ|TA$UZXq7qDhLxAy12BtbPTP)d6 zt}yJ)5i3`$tR3H=DX-szE6L}x3}6^LYCNG0h2`>j`sRY$PeUmaSfHCIRa0njivy7m zRA=#<+{05Raq6PQT0lTy+rFFce)!HqpMGDMiRXylFFyY2wZ9MD=eRvh4t|&}jYw2P z{Fiv9LuM)#0wDASW$?`rItyIDj3xe&n#TWn6=FQhsdHdqf4%+}>F|^Jryx?jd8s7E z|8{yz_*J)V?G?-8`@RWSu)rf`{#q!rHTFuwBJ0k{p4LDADpa)V%tVJ-S&#w)8K1sA zdd;fJ+CFm{9F_Htd^d3Xm`O<;3jJ5eaRU;+qIjwQEw;)^PR%V@ZC1V~e%mrlUZ99| zS}_6HOw3RG39A@ihFkCxhF9}1$LShmSb}#@_HiH!-aPXQnFV*IG}VlSf~OWjx36`%Ge^CIyk_R)^24Bd6Jxz zA^es?W!gf1lAwHAfxt?6F@bTNRuFzw05r?Z%TsW^; z<;cO2mJdc|7b@&Ez5Iq73&BNA;9uJ))64KwbnXHG%rndx@Mqq(U6pG~Sx>l;8HiQ)G>)!M#Lba>#*8pa+-= zaT`uW_zn4}XwbajHvM~j`X*EIMul=Lm;3D97|A7P$mf|kpO2>?KL)|f;Lcn_(in?U zC06Y=t*xU)mPa-E!z3Uj{9eq{N+(QVMCUZ}(2FQcA!`K}+S^s|+4Dk<`*>l364KFy zvzPFWha&}zAzeJ-vIV}ImGC6yiCKz=dZmVZjz>c1fgf4m5c~jurx;WDganrKMW33% z*D->g6?KUrSyo9BeP>TX0VmZhMICed-ZzkW9 z1Z9TzMU$g~=n}P)X2N*YMP+dbLyYV(uGT}qsr7M(C-WbiA>#B0;upbcmd5xW z?SoL&80@>1Eu?1cr$&_OrIgR{2(_;h3QhLeX0e~j>Y6^^M^S=~5E5J956k1jfV!|6 zgJ;(qB@%Wyr0?*zzTiltN3q0;NG-rC5{#=j07H!Dm_<;#qJW1n4eyKiRe!$FI$auw z=r+hI3ZJ|+>etqw08{*0utsjg`=fa)6+=P{wiKGRY*e>FeNy<`25f;?(!%cHLr3MV zCZlh@fDGe>SCt$7!IzaQd9k?~Kaca&uV0T#QB)c^6%g?oqO1crA&%FLU#hu$38u9N zi#j;sEb#+)<}+V)<*FKaCl$jDq_*0B>`L7$q|+U#P*mVJ`Ld%wdvg>sfP`$)Fth6K*FDv5uWk6=R*K(N zd>z1Imhw_+o=+4lY0$8NcWp#u(d0W35?)7_85xD#9~HlL0a3pimhtadEAp)x&e6py zILuo=eqjg?42#X408!M+(!umx39Us|}^uK68Qi!dp+HUPC^i;e7iP&a#n3>g|>zesirm;?mk{y+o!@-~61bz($Do zF*Y2(%Ex-!&&&a$;&LZkcH}2RM=s(!@zKu9ULp73wNGuk=N4_-w<0SZ-lj`yp@z0- zD+-I}EvetQp^+xd!M{xt7Ku#r*-Pzz9Hon+@5hc1lPo3lf`MZFYLY2-%Pe*lEU2aZ zgy(2Ko44h&EIJrV;%C^4~53?|;dUoFw03 z{!hyfx1axyz4HLDq$<;P_k^CDVPFCi$P8JKoPz?2L>WK?L6M-S2pGtM$m*}4tl3>L z0E&q<5JVAli7FCYlpF+xG)&H^|L6I>s=C#;y9W?XvK1F3j1PeXAX0!Dyr=}xNzqyw z{qJgHNAhJuo}8V?#)p<(i{>x>{jcuYe&4Ngd@JFg+FPx-`m!6XOjvC!RXk=5R)Hg0 zGsXk3Pp=icImamjSpeoj-8=xd-FGV;KRw0%SH)lbo!`hmnF32_AHBRf8fo}%oX1vD z{@B5XdiRmyX8+@+_FU=(j(+eUC$S3JqFM)&D;G z@lRH1Plxv?#5twf|P_t;`pcnwE>s$IrW=LQ>DSP z{zaLP%!)aPD964r43z~5J zxg*mRc9DT&@+bA~)T>vzqIhP@GGt5NNZ71M+)ntW#EF-I4;TC}aO9RVl!AA&h|aLh z`;WqRF-591K+(oxI47UVpK`uT`88>NGSY>76+fORwgx|NuyXmsk0}=2p9q3DZ-#OS z4F9ryNbV^EJYIuuBe4TE9nwI9NKQg@DzKb_YaVf&nk%HUNaACw+y^V1rTOocwUCrO z`nG#-(XW61#CIXE&}yTAqs5Smlic{4IANH8wcdHd-3XC_Br*_ags;zV!)?@;m9~r+ zHH5E+L?q>|E#Fn{iDGO0!*B17){@GC4W|k}{PV5%En7k#)yT|Cyaow!N$ z8WjNzxcI!*_cK2|^G%2EyvbJUPgrX;?l>4hD2z;%{34oE=+}Rt8=LG7^OM$N&OUU= z2Y$iTP+3*9XN2v0Pb|)j$(-FKe+$E zp(GCKKgg91`F8@);KxxLpfLhUd}OfUaw0$Ar~H&-Kjo*0h8?6QKcL8HM|?6r_3)GD z9C6Vip4cA}TS(mYt@hmP)PH?nUzR#-1vM1*PkqTx_;s1P5J6!que8?0ag+Qhk~DiZ z5l!S|`RhuoC+2sK4`iUZ{Usw*~jEU@! zJN=gfHf|I$gPTp{=Y?t0@45LtB(3;IiQruQtzTz8K6O=I^uW5ON$tF~x=1YK^@R@i z+;SiIfKo61%RlsY&86|OL$hf=)Hh}_uSNU;6mBD}Kht zTCF958SroU`5$J@oGp6}d^+?m)S2>=O~Il}@{>|^_TR`xmV!?UDYIgWAI-1x6Y?9_ zQom%@K2?w+3KLu0A*6o62NU*DcvwzU@XGx+G}^#VIcco&3#tDk{*@R16oE`{9&OQKRhMN9{pND}G33-ImEIJ^; zt_9WkZ)y4cjdy?QxJ!6kH*$iHs7?M$X>8z_GY@Oxf62l{U*YZD_DXgcVOZp4zUw%&r7Ae%JFlv{YbG@P*c7C z@-Q0cPF=sHeOe?(kKOw(v8&Q=5G%p72>YQp^|Gz8_L#{VPKd)uR2DY((|!mPCgSmCo|z{&#YMQx!9Z13ymUBPg|GH^^0%eVjj*LPp@o#y_9JX`It znK3=$Pix;k<9zqF{v|A}3IF~2CoX^b;ir(ZZl3^N_uU()QxhOJ`7P11^`4t{^xrrc z=hV16IwN_a2JNchU*Y>vwSa-me>^9Ow6^!{*KhPVpS+N!lKgP^txsLaYvWVD>AxuA z`YUfbY0on*`^*o2@{MaVzRjOKFX}}GWljG3@K~&DK26bj+POY=&mDgSmaWonbbz(^ z=1Hv9e{8@eG7Y4egGUTvy`z-B;0Jv$)BegC-y(l_5$b6fKVdbsDu#N5lqHK6fBS+T zdT~q{3x4R|$)u+IXNU5g& zk+rAt!Y$C2B0J)K^SsNRdSqHt|9i&`cV7GL-%tzL>a1?{rZ+VCZ}1anX@5d`XuJ3b z`hrAkwo5bZ_9B|05TS(#gmmWVyNC~^4T%d*BfulWSOR$Fmu^NyCxsiiP?MGPh z@JW9~{T!>?JW7Ge*rgu*p{|6@Oz_4=BbC2UURDa9B7pDzvf{8;D~xhpH!+6C#%?df z(Pwy;xbF zc+B3tx85f|`h61Ozh8Xo1~BNdl_MJH>m*|k#%Y701m;Pxcsv+iK_d$2tQNC$ySH%^ z#Urzc)r?uWBp ziX{L09Jw1>%`IRm_-KOYS+^QhHCaeV%@>tIn)Sve*pbA!;tolkT7@nCu+iz|T@v6s<#U;iwR$=|>qJ5+5d9 z%?)OPe`+%}Q<~4XZfdxw`7cgD$4QAx{Q~mHgHI}gW%ylj`R9Ie$FJ_HFhZ(UZmBh_8ys*_1CSZwgB|ht ztobiu6XG;*HXWzfP_tFHwohDld|R8(K@%zeT3bGU%D>-p%l%b+p83qoFMQ;i=fCUX z+phoPJ-6Kl|KQMXM&(qeQL*uDh?GC==a0X=_qy-=hV(cIF-k_ftn5=^%X6GX^?E`} z`>qG=xaoEqhM=BK{5x;D_ol0Er6NjkRljdP1TOd|6R#fl%IV+XQN)-F(UdYk!w+r5 zKA->FbHD!4Ef3%K*vs+X!u~bD20m!$RsHI&w30f-ry3I?{@4B2)7Gx(rTm~#`)ljJ z#WbempDH1xkU#ogq(bl*1a z;Z+405*N&0^p&%|y@>DTpc+Yc$XoTl)1I3C>mT0y@O_VynD&Xkmu>-sI!v%8{YL*{ z1`Ps_iT;$4fog9f9~Y3PJ8s(k=pSicH<~=krvJA5I=MLmN~2OXXp4#LA4+G_^)wN zNwEpb8Ss(ERBn+_Wy|=&iW5u6pMH9$1x{U#KLO!noFcso{HTo&AVaYpYcY~TYA}%^ zr68}g|E=49WuMUwj#lQs_N+LoB&TH5udz5p#7YgM_P;y%tyrx~=5!WKjUs;V5~4A@ z1#H7k{V%+U+X0^>B+4NurF*p=@`1NTrXWOM@n=3a>-2*^eck21ee|zSE}p;eFLyq8 z*+oBL(Rs;2Uo1t2Ldw=&EI96S^zIbS_>5SGoU|Wj6;fP9;QtH&4+8MlPu%|`3y7>) zfA6zbU2yCrtj9$|;K*#0;Y+*w9KA>7zX+e>=craZo9u+d@h$uyjb*7=-tAukD6Cyu z@)!cse+6k@@s?x!8c6yt|C67&GZ&mS_0!)t_d7gp?}i`W^1Uzq{C#gegC{rfmRTU? z&z`U4^lF*7-nhh%#ufhiqc8s4m(+NIEbTY^;O2AR`MJe&7jh_}rT=1|mH(!G^+$^N z8?BG%M2g0|qStN*?Xb!AQ}QHjto-*6H{5;GRX11uyU#JZNj2>kMj6;{`j1;rJ?MhR z9()=`lAlO#pZ@fVzxnaaJooCbjoyF!&Zpmg?Om8IL}ve(`~){l*EKNWm97>r=!Pw!F z+vWoY4$Sc($HbpuLk9LA*gyF(2r7Te!Z{1hKJ0V9`Oz)c{pde0Jn@Scyyx@0Fev3e z|bd9T3TOxcJ>(uUwGYRzj^e5C%K*E{y#l(#YI29@V#I3ojca6U8H__tjxYg zq4!4rV&w6hSwcu46{q4qDzU*(gmlnLiXf?y@Wqt>rk~#Sk%K<<_ybP@l<_(B`^?iX z{`yBZpa1TQ4&CsC<97P!&DZ@NK2?aA*@(ag<-ceAjHEff4YsIOQrKSQ__+^l2Sx9L zE}9Vh$xr-Qfy5AAO!7+-F6tPUjctuw3Pc`01>uwOWV2TcD<-Gv=zvciastIp;yZ{B ze0#MNA}UE?td6Y{KEQ+zZwmyi7hjpP4^a=YgE;EP`)onZKcWlv!l^V>amzm|aO3zT z<d$@P%jaEw+FI*ROl+1*VtR9r#$ZMMvF(1y z_S#m61rCp+V=-8ZN}!gOi{Ag$^RD>F#0|zrc6$P6IM+MFu`*Gtb=RrevB!-WUinEu z&7d%ZY;stO?c>jI=zX3U7lVI3$b-2=5L3Xh>x}Ewx0MIohMfjKE7~tU`QK0fTK`S9 z-;m5m^5V0zzW=$ae);|1Qgo|bHX1i+bsj&F{E#9~e4YO8$EIw#F8WQC{NcuXul>$% ztPO~ScJintvtgUauZ+R77&g*RY+(0;cHn#3f4=4Z!H-`H>Gty_`3bkpv`$VIC%1-O6tns?+57jPJ&zZY|g> zKPe7LrP;Vw+K&~l+<%3RYO46g5kMM;>??opBb0ys(=Tj7_wh3<+wA0|)fTc?q|nRz zBs{Nt1|5k{i;7M2vnpZ>ri34%)k42J?7Q`C*WC$doL4+BL}%E#|F+yHNe$4&v9Xg- z;%;W}Kl;Tsgbw`<89MZ!Kj(e7KKRf*k3=70 zIAWJ;FTH^QB)msH?j>Gu*5MaUUT?ys z&1Wv>EurKgDe%AXiOV$QJ;_}<44IQC6EM3C=X zF!i1J{M-lce&n9t-S^O+A4R^+ciH5;Z+}=WSL(l1XZ%#b{BE-qmZ}K0X9sH{ve{3o>mHb2;on=_l@7u;zBm@NMlt#K! ziP0#CQ;?2HNlTZEO-i~uCnX?V;!k39r*udo9V5mDV|(`eU+g&cV#mJQy=&L*-Pd`3 zP8{QJ&6~HTN>~k@cO1tt4T%8M=5u1x0n#G^C8UhX$Bq=}$&TR9 z7aZ!hYeOvYtyjMR-GBl<8{ON94)loOv$)S&Z*tF`^se00^pF*r*0k7TIyh~*DS!jUl^MXgD1(nN4s;Cr{KLB zJ_`90y{)aRTU^!YB}(-47bsCC5#sH*&=1LtQ9@=N9SQms^FBea5vM48zZe@2(;cLGtp#s9wFPg#p&ZAE-E0Q}InO35=<)$*;m? z-CSROTlnnpca@B-iSi6ew#;<~L%UHj(LR*aD?5?FKUy0fF~-u6ugw2=YRhq&B|7l* z;3YO*`&dMo+AeCkp7?Pb4MH$%FqPcn)wjJh>m;7W=99V!BXpX}1~Z~dOf!Tj5x%3A zDr|&hkujGwB%&ZLuc^T8SX%*ZZ>|zot~JP0)M}5KYdOxr`%{RQyHvy zQC>VQU*!&xw&`L8oC>ZtQ{r~qeQ}$@nNb}+H~u05ubr~fVr{8ke{upG#8uqg{EjJO z>rp=TEPiRn&-363#o<=v5tN&oug62xZ(f4(c_sF+TKfcj;l6U#-8el)cc*Ity8Tl+ z0>O@fTJSBm;jMJ8ZVi53_#MLMCNI%rNXK4|06?*~t&nSw`$DimfuZN=7&P!-G()+J z89i!BWwH*ajIO@S6?p)Pee7fxmQiU)JbjAGv|Vl4-EV7aJArTG`k$`_^@y@ivM9$P zhS-Qt1IAk1!KjJE5G`4u7-sh07aNpEf&ac(-_2)9`*kKkyg!O$p)p6M6G5=dO(vN6 zcYu_EYW3JjYZEMQOW>*U5vuHqNRUsqlWBbblh9y`<%oP)8ti=~ecdLpT><#ICXs!~ z4)(rf_*XJe>GHMofBZvm;*-xPY%7}$U$1lU!IZ9|rs7Au`HLlR@;-hdD0u#q1HaZJU-83@F`PlV=j-7RpzZNxV zv<0=lb5F&61m7zq~xM~&A zeAZjCa>q71hQ*lCb&q|84m|I=^{8d`>1Sm=sodWr5f?I8x0_!Kc?uH}QFamM+C5Z> zZSOlFLO{l+KI@+D|806JPBTh84&RHeC5B$EleH{Z50~i8Sx7rzVeZhgu2wct6Ec_k z9)EssT?gD!NDG9`tQQhcDyy+Xacs5-UCb}*WkJc;n`S4C2mJ`BKR4F9tBmf}i0uXSSclUelNnGjr#pPWG zdk{b4b2C#3&pJ*7oXylP=NB!^s8+}eys_h&n4xSxe#)-=1gybJXLb2ja}4lj=I7R% zl;6pKWA-C)%U%1yesV1X6-DB2Gol?=!|j0W=oz2cs-~_)t=Vn(yBT~(;un(@qHcUn zy_+$7rIH*O#s?sR4d7MI{Abl5IkS&)%Skr&*JI-_%wajLYG zb7SfBS76Y#Hgh)Z+t4*3cI`GwuRGAYEqumCqBRR*d~q#p0j!!<<_MPN z1YcM(U-g}P&n9+iqd@coA>~ARk7mrZ+PY}x`K5UpP`+7@$o9kAxpQRds^4_FoVT|| zznnK=apK*Tdm{GAosCy*{i0uBI8{2ZC%VuW$cGbCL~lk6SaG@XC+1U_bDDp()>)h< zuER;|p79Rc{CEGhj`9S$=m60O&Ax0y{wMVhdRKLthb*NTf=t#2t)1QBws|{LKS;Z3 z-d+HKxSQw3C73T?GTCwhe~I3tVDS_oO(vY}mbgR{ zapDqWmtaHh50D=!D{jxg<|-1LzHx+?=9@p)zbuy(5>s}{`fq7@9@?`Mr1v-5$PT9v z*drsLe`;3Sg(!-;{3*U%E0>?9u)GB zQxJCjA46GtRCvT4%4&BXq;Pv)Rf;NMQl%{?+nvzBdH zA@{xP?MSKm(|3Q?Cx7mYovzF}@CRg2s>+Wjvz0hESZ`*^4!yB6WR%*Ad-!4t%29@h zqTZsRiqF%@-;%4`urSJG2%~}Vs4F?h{N7XN0zLcsK>$pWbYMV(hN<~9o{^zKk6ne9f?BSWFf#J7gL`4sYvr4RW?Uh|ELB{b$p$OW*ME%A05@AZ(f!3pTh*CdHv*_;CIx?w@LvFSqZ+T9Zyo9==OX*Z2N7e%IpN;Xllf5E;k_K(_S& zK6et*0)y(Fvqix@kllEo-#=}ZLB1iL{#UvGa!6b`Bl~mH;}aNzVnbW!*)2bp1M}l-TOI)vqp(qnz8@@!QpodW%SO*RnIH zG6q1tQ)tTQ_aAe6^+S@593~5UhZ!HuYIjAWL%#eI&bH>k2*!A@`u{ZrcG)*QTW^Z* z{3%ZKL3a_wUW_*}<}4^uInWuy4MRtdA$;IBTB0LiF!2W@#fG#V$!YA-ryU!~6#W)e zkotPY{V>tRqX$VqxB5VT>%o(i{!d?5%gHdVIdhDwZ+H2Cz8SEaWLV?Ofy=TB#wP=y zst?ijT~v;76#tWSM#RcuFALI3S-_aiJcJ=@;(_NdcRPhew+=TtNhF zqbX@#Yp?wppT=E=u(GP`nXgBoc6eMJ z`%koOXGuW}cZnY_&APDvUVf4>T(1{1{aJdNMQk}~jm7Z!1bHMM!TqH3L-6r2*{+CP zj*scOVv5kymycxjb45h7Vcz8n&-#4%o$jfTW2<7km5D^(WYHA#e3ndokemAL*jf7w zh>y!{uZFHX=?c_0fc$%xh`ZXyPZ0<3TRFQSo4GoIqW0iu+vyDaY)+>!+Q@WV0#Of= zxBGkD;Xjb*;Z8PROZ&Jj2b0rq7uS8kl2b?Mx@}igg87j#b-bqzXoJ3FEZYh@4Fg{s zjX3cldz_~PZH?<6^D4KtRmS1v+^49c=+=DCgXQt9f@6F-=$JYbqKIG+Gs&BZ`7sT7 zYp_%%89>S8mZo_oGx8C=Q~3(gFd6er7)=%K3Rb+8tKs=@l2(}H)7S$|6zPZTeM?CyQ;j$c$kPhe7+s8v&zqKlMBnWMu7{N_ioTn0)l3kk&I$ z4!7YaO|VpycytsebI>o~J*82`#0Z1Q0#(jiTIuOI#W!*~IFQ-lb^h zxe?srWl;a#xfA0oZO0_qoY@54T5K}JWnx`CA!wmHIh4CSei@{`7afn^7;ey#{jvdU zpX&uExW3(Icwdk^k}5zRyD7JwIaK$HKeL+i)hoQw<66XjL4G_%w>AVOZZe{hKDYBP z(?ULh2T`9z&YD+=uKi2Qk=0jj`1L6U8!WaWHO^{3Zl`K1WVD>RoS0C{IZ2CXv6;6=5wSz20#(F3eX@*k~RN-h|V-x*X=BhA4Tf zUpY~0q8nNQk1_Le9FTnIkNU^WkJp6ti)3<7>m*ueBmul6rBK9!v}U=2s=SXRq?_D} z2W)_^BTID`->i$jcHEWRxyo?SU8bw0-NciIpA2x9OAB0D?DA>u-ct3UHaczBMmK|; zWFsJ^hPY`x=(CQir;|AR2)sUl0+|>ulq&Mddf43a)&5rLsTO9!2}}S$se+HWHy!Ap z{OOpiJ}s?8wySke>i4Sp5!F@5Ak&gbUA~%z^LO!>x^jW{)Y;XeT!y%)2p=S#)eEKaxnwS; zfj0wBDJ{WyhVN0F(1o&S!U856hGTlBDh<;x=oL$2yGn!45PVl%uV1b|lDOTW_R(SD zb(Pjk=dSPm^&J*6xa&Whad>=PAjhEd#y-;UU6jaJhezD&O`83s;J;S82=h#2qPUBD zM}lJ)fO`JQJQZH`ZF~2x1MBZaze6FV_(X*I2IcXwI~3QkLPnF~H(5(uNAEL8PA=deWvlMql_^!_I{u2Vm}NdTfOwXX_SR=M?a*_P=sfCQ{5erVK|U&NRj(@@kBNniJ@eX_ilb+1HqT&Q3-^z=oyhI_ zH%mu<^@vNme)ZTeVon78*ghmG`7#jyQ1M&9p^{wL3okpnGMdE&u2Bs=kW|CRk7g*lj+uYJBOFhBa0EF?)Dk^++#3zxV4F#N4{KOwD2 z=1&(ONq$IcoF}$5a*NuW%yv2q_}ToOi)@JEVoubODJkb7lYG>?#?|Jh_GZ~U+PZhd z+34sASBkO*k?vl4%z#>4@k9OR+yJM%qjHeh_D6YzGo>IsGj~N_v){eAzZ0jZp3{@> zML)4_+Ax{s%{56Qv2@ox25F8%)XK$sa%r7gYu4{+?_?|ry^z7S1$VV8<|{sO%aOr3 zp#t=BpFgx_sR^rQ+{CvHPQ-3%P`oH|`X<1W^?~1_0;H3bQ6x`#8c!8O@R*|Lj-gbN zA^T1?9#RA$j4!eyU2_AXspO7f=l7B=DTQ}{;2a|~Lpm+e$$$W0@?*Y&fG>x84i{I|F`-@cj=h*Rw(XFj_n_FfBfA;KvT;G+*;c{ zUiv+Y&TY&0KT!V__rp+878*oJFgoGZ-2kEf>M)+4*=?BFQ29Z{*cm8^Dq+GBxqJeG zfM0ZJOh0nb0Vj;Q5Ny77IgFa(9!Jj&@fG;n?Z5|N$i`*Ja`3@`LtK8-XI?-b@B1gE zKxsd%I5tRsDfQvCZ7Q^^=7K(CnllX%p3-v3_JbrUMv7-PP8;Lmf> zle-DWfO7a~EX_CB8Y%$dkEcuG-vhne(8T`bn@i;@-P5f*N&fla>NmE+XXyE*-&mIU zE%WEm0*r<&OShut^`rN6o_y5Z=z&VmB~Zi4mRWIP+6JBaQ}TZ0J2$6y4r*?V6D$)Q zm>CI|nvboDyJh-y3`m;NqB)gyT@@p<^R-g^_|9oEYO2R785i2>f!)l((W?pR9d*kF zGoQDA#VUBou#39nHd#VDTYeW!HFR-eG{W7Ols?Tof(6c-WjPbpK$I=K5}ZcWp#XD-)TD zV8RYm$$YH*D>sB?2zo^G_H`~5vZte>EwT`_e5)KR9W+&?2JgkWh~XXwa{83DxO!uE z$`bP1%xL2+tnX3P6TX5tYKTusTDKzzaYi3`(tgwh;`Tv1sNP)EWHy5MzI=q6nHE)W zff+H*dEfWkA@T2F`QM^*o*}O75@D7U)pbo{SWi!5u~VK;?34GGVpI<*s>yJ?4x4qw z#nGox;oxz1W?vLQH z-rPKS!$1k_whJlISX4G=ys4M26S^cLj>Yt;GN#=OGruSuxX^u8}NrCRgy~cKb@#s1iQQ}H5P|JAOVKn2NcWI{M6AzHVz`68lYYg}OyGxCN-#?MYd zx$5e)pcH$v~ixS*C+|9>klUYJs`T3 z!k087CRC6q{NxdL0-lDL@-l8s-t>kca6@Se5RZpW6gFKQL2;9wyX}p@T(aaxOfoU| zl%EUXc91dtkjvxj!QFqYu0AM%TbE<{8IqwIXvUpb5rX2}KhLZ4*>Vl&8no?tz9!6e zTqwUoD!ojFw9Q;9yN<@4I0Kn9ogi ziu7~y)vBXQ2ItXQHoTJ?BYROBd*%XOI4%FX>a#FTJ2z+;<@T>JVZyY5PZ@-MewfP# zKUNCbpfbd@fJMjY3HUg0R zG&YNORtcns5=&x;1KF;f4;AcluV@$MMpkjXDARXUuj4gKIZZrt!+PT0mfj4DSRR%3 zPXy@v7k%pfd~1x;m+=cJf`{c(0X?_sTxJCV820YlsM9}rw>LjMsz?D%c}#KXMSMmT zx1?*0o}^U2dlOYUEi*J8&K>`U8@#EI^83}-6MW0ip3_gYVyYLF{@3K@?P|jTDZqaG zuzadJl(kdmx;d^vlTR;y(fQ5t!rr|cRK*CWM=pu;X`eX%c4BR&_RENIuwQb!SUVMb zRfs5^weUHzfw-^Y$-`aYrJM$Sf`&4Sg3Wu11$7LFCC-t3s zAHJ1jj##MCH+(%d#;9y{@ts^0#wH#E0=|f(AAgf2<$;?nFpU@FhE0T#1CTq{hzGRH zbpMs;UOGA$HFmMgnYt85mu@6cdQJiO1VSZCg(*?e5`jX~H<$Z&5bc(Nt>SLn&JJ9v zwAnYZ#nl(zgj7Blg=|l~6$<5v;ZYJ`k4!C#>&yF~fPQ7-mCg8+usl83%<`pvCOjJI zJoD2~=Vz`?AMlQXSvHIOXY9luh^w|Mesmg^$Tjvbg+Xiw@uBoi+lhKa*qI{VhOYv& zBL{SXi9|pp*2)hvU6~)>?C*g%W0A0$_d{#pYxy|+xKJ!1iD+m_E#V$#SB89KlcEl~u~2ET#f`w{DJaPpw>DN0W&?MK5}l>B%i+bi@xx zZLl1~pI*TCC-Vt`m=CFz-+!TF^guQAA->&2_hc`1ZAg}s?MHmES4$TZ$`F{L5&LW@ z&a35u&A=Zs56Xx4o#Z;@yfdU)bIgc8U)|D$1g>`R9rXyA71wv}YEiUExWuj69=riz zW(k)clH|=?#lDI&cMz_X3;Ao+rktA@jdV8;?&d$H`L(WB?BtZ+c7jUF-C)iprZ|QV zXT3XmkBCRL9d<2)yzck{In;3X-54{U42N%$M^_27?dJ7}{dcos-4jWR>M#zP6H}F|8^M-izXlgm}P#xOLk@_y1(|uAYXpDgWnwnp~`HhtY0#sgfZY`AM+# z!jp{Py2F;Q%Gq2WYgPR)wMV%P;NZk;ODBuPhlXhnFkoBJNP`a42(aTKEi5XdU}vJ? zh7IR+xLoVj__QqVa?v(RQyRezS?Gcaq}*5rulxvNuXh2k{dN9c8Ur9&Iei;lUJJ?fE^ObVWt zU)?QEkMZZCD-VaUhk&l!t6@p@@4ej}$ zEwdXnTAsIC!Vz~*prN!eCxPq8Ow2hAxSmSrL16=XokwVWs!{N@sR4fd@~pYPe#DLR zovFd=UrcPfy>2_S3?sSCYe2s^l}D?bf0;W2xpM5dJFVFb#EYV5`4Z)iRb0PK8kngL zyQj$gb^Se^b`AZPWjn4YeNKz)IksHk{Io&_UU6WpHUE(a)=J8u}a z5ZR1ypwM^2mg8S(!Ybr&n%9S(>+HgPbZ^m8yzPMKSB!*$Q z)Q>%B0Dkg_%D`)~_ZeywpfLaA4S$!zJ2E#OZfAjk2^P4g(u*8|Hw!_7QJi&FoJQx3 z1nYA_pZtcJcaN`Y{_bqKTS^OHF|Qg(Th{RPt`O`SG2ylnnUd~Qyz z2Bx6lyCt(*Z1i+B1lM~yJ3f{OgCkiBbd@+h$-QZ%6U07o$NaVh%Hv2rmp?j8pkrsY z!T9YFQ~p9Z?0#yS`SY1Dt@w1& z_sLn4X1nY_Q-_{c+Tgv;+B;EMZy_y=a?3@W+7YJ9Cz?^R8N8~ z?rAeVS=b$qs~h=zQ}rK~+0HF);U-lj@YMNpl(V32;KM!Xe0u&bLipBAF zK&H!{RZFqGO47Q+CdR{TIn_ia$+4ltJ7Bsk2V>D*uj!`FlIB0|<>$h$OjWlf`I=Ml z{HO22FcQv;XGx_GMP*~cPZYEL-WoaK&vDVD;0$jSz4g&NruRV4l}=djTJ?2-ZRgaE-pOy-Ai1>#m9qqXqn+l ztGt*5$Z;g{)&~ke&()QeGvcL;O$~!?Wu`@epJ~bP*2Cz-js(Jev7|IFKiN(CF>3 z5eB-U((&TN8zBgpYdVc+WE^wbL~<+pP1haVbKJiMi;PDkT<=_x!qAD0czKd!`bRm` z$SLkbh#j65`9-|rVTdS_W=+X$c<#{XW~ZZ62!3{fze$as^E0oWgz~rt&z}r<1+pg9 z!R&Q2Kv!2Tfo@|}`;8^Ca?wqFO?5!?wZ|=1{*l2gi&vd)w^#d>F@g|M-I9DvXZkd} zX<{D;_gK0Q+eE~WL+brV5n5k>R#>XxP#3b5Cg9OPlQG!3_R9Lmn!)|DKpxX`3%vU_@rF3^cYr7N8fLre37Z6$_iMmT=!kOYFG%wwvEH|5 zZw^2+6X{wfIlaZK1(3^ZK6c1Q>=EHgd!M6g_lvDxx?C(zXQeR= z;c4vIJ23VIcPAL!V``dG>F%<4Vl21u8wdwUonr@B=hw$!4)CX8v5X|@oVcm8h**b- zocX!Aem8$bz6N9Vj{b!x*dD|3J!scv-vuadqP@YQ)9z-t5y16}+PB&DltpsIiP++E z5Yl~`9lBSsIK3;3e`^bZJ#&8pB!O&txb&ts$ezZ?0L5H zpYBIC_mq-ll3UvE{m#{6lY;p46F-aMk`>W?z(XJ;-z7|e^@f?^bLEqI0R z-VlBPL@b-^dVlmhb61Q`XP7a%2FVw3T`reFmt#s9&pv=#8gZha2A{b&beoaa(B=-o zS0syc6d_@lMk;O`20z2=$pesx-GJ{~p@FePlz0RtsuPjgFFw0LCTw{l*1n2=1-F!Y zjXp!?*4#wiQ?;L9hPc*%F0OF9_3#B3a00dqo~>Mu8p6};j%AvAa%!? zQjo8=+PFi6_UY|%-r_}I_0W3Z-G9cpQA!`X9gS9lUG4Va<&^ExXk!!2!y~3fhH=wg zQ!UeCEz14BBbi+9CcC2kIu+wA>wWg)JfK9CcSKh#$~GS9dYb;-l&y}z*nI}J+d~7O zB8nEcR8(JCBkO!8S0&|q6eW6>iDl^Fz$&8mq<{cfwfLS5}@hE?A8eWbg=Pv{*bH2vE+!O_!b`iHW9nZi*raVH7lw-Yst4q| zZubxF#KPR2st~Y-Wc{SM3S2O9Xn$eoXxHHnm6Xx$&A`H|O2n|H74M>K=X}+dQ%r38 zcWh-7Y?27OJbbLzd(6k4j~wNmxCkwPgxw^tji@`%ZL`hB;6!f+BC+SFV%$~B^TvN; zvv2=fYYg5S?5fnW@ZA!bTtji(gQp^jHOMK9$ z!AO=JH$m%yy787nnvJq7F8!8i^*NoElhaSGFzr6z$Cu%QOqB$se8fN4GYFr1hd(no z5oVO}t|937qtV*%)(|gMgZMg+gFdz2R<#_)nh1ETN(}ZV`7jAR4GGgt`q+?8of7>U znb$6R%M<%_IVCIhvVd;>;BR7ed3n;H>_@oC$1kNS6qno2v^{vdP{w*FZAZx)-#Ojwe0Jb%8w1ZvnCWGeedH^gq9q} za|UKhcw>#pnVx?G0Yo+F)wf?LqxUT{M9q$7$|-HrtVvwk$-64#8U(Y2SlgzY*_wqx zb>Q{0`^u+bIn|X<3KJ5A+rHckMV9FWZ7&Am!_J@DeC}DMzo9Yuoe3J=6ow5Zp~&ck zxl{n?v&IVtF`KqNIvxl4kkr5@8TDVDnkA*%2ddmvF_GO|pEkNKN?I*BGaS;y_0Bxl zB*rp~f$&;~?w=v;|GDJ((tO$_cxdjloorcVB_1Nteztu%z!t?cp2Dl1wc#r7)I3zY z!N((pGOlE{9~m|EE&zKhH4PM)H#iPWzDtN6NML^)qT=a?ta|FYdx$;j$C;&b zTzzj-cw-&f8PCQ1!9xmUxtFowir;<4O_B3@HmcBOkiw)6H+Qs zc_O!;Va!%kx-(K7Xq7Lt>Q?0$jz-;O)S$YQ(@E>UJidK}-W@&rZ%mg#lwG#oBi)o;?D8tbh)>1!W&Gu1%_D-W(&==6j*nY`F2xDb7RltPXZp zCXSJNx;7gu8HjS!>-o*p>I+nS?-xPA^Kb2U_3U8ekdPQMLKx}zv>*22gGPYRLD|4J zdM)Z-Jd{TtlmfbE44%DF>f11U-a#lPcbD-S#V~N>FMty(=pb*^kMg*T)5Y1|xhLX~ zs8g3sxJhjfd3y;XAd>g`67Iu;i{ILvPsermu2AW1Et5D&)Py}eIyz+rT+G&zFNILO zgOk)JprZ$^JI{~o!VLn`Q}jJwF52)Bm5F~X%!!`iI^Z&9)rc;#<c2^7S9aAX0U~c8XKC z{k~gqlPNDU@RULKQCb7EK0cJ8OmfPVmSD^Wx1Yij@lt~IXMEKD^ArQ6fA2d@9HU1= zA5L4KXS#;6p3tZE7m4GKYeP8tfkw|BS^byrZJl+6OW;*r13x<8K4&U;LNKLS`*9lG z8X7+AVlC1Zyr2?6uRT=kx9zK6(Dn7J#1^Zv4WkQJ`!A{aB4^UY&X{z6-3WeCg8t4B z0{kXS58@_ZIAB&K^bY+%)|5v59ql`GJy(YqX;Sk!L*;V>AtRkJD z;abCRdynB6adnQH%R?498o>&*42YfB$i!Y;^&$Yv52v!CQaCHVva->aj!pf6A)Ou; zJaVSd9%vUK6{*uQIf)GZCqYjBR5m<_8%>zSn#2tVp{!40dTm}StD$Mijx+zSzIx5s zw*8}8k)F{z!J0x)N9@d;|F-iA6G`Zr!CRJ3$&b;~ic981lm`>Bu5(L{Db5clz;;R2 zr{P@Dl8_d4tS31d$YV^T>V3dZ$76iXkm>l^JC;{tr}px&=5exyuM^mVR0te1RDbcF zFLz2OC6>nbcGP(XaKFc*^Bycc;4Cses?{ViR<-CE*#~hp8Q)0umVj`~>gO-MNx52F zO~+>({z)%nA95rw=b+v)Hq*CS-qe{7d8I`DKKp(C zUPVx6h8ZR4+(pYXqF>4?V2c-kh5kA1r`i>8J)yWM?KoQ-N4gW@O({Z{B!L~{?(|#t zhMFC52^EungyM@s(2bPFefZmmaQy<{!NE2>Pw&&_O#wLNZEs6C&z(?Ze=ip{iXB`r zIE<~AG)~~Bh;$-3+V&jo{Tp^-QiwJ0A+Fv8fWLJbvqV7b|2tn3*8`TS-;a20gC^lZuq;-(2Cl4npYu_*+eJK2O&V|oSYv|jOlAaeuMT{D9(9TUX-UdoS6+a6 zOan9U(q)vVc|C7B* zO23E_B^w`p(zQfF_|ddWE}rpUl7=xVt9NdN>l!D2am zRXJmC`mnog%ot`0`2cWpCb|v8IA*)hkJbq6afK)xEGN7Aq<`

    &}Y2{JWuP?Z9|5S=L9y;UepiX+?aU;0Nqf6%{m)_Xe9}q1_Ka$({bI=Ie4pFcc?1vY)@U zH?dwhb0~f7qx8nIR?I`?u&+sT+yWmC`2i$)=PpH$A?2wT`tD>b=jt5A`&sS$(8ufw zXIl57{e{qph5mdHW{lj1@wqX^Ck@6cJ0g2|tB)cSpvVe~s6l1stn-m`^q%qqv}*2% z0UHL7(f^eet(S4*oJ3|mV$VM42h^H0V$uY!$drj*M*oJqQm1kCMU!ERMTRI-9O7ke zRbK+1K~@ZlAwqZ+l)n3>F)|fpqotnuUvKR;rHku&VmX)D@mt3*El+r$Bx!Z*y@~X?}3~! za&fhG7$?4|Y22`v67~{ROSB}Ryc^Xs^^yrziLfT;iCe_+<1|5kCqVd$gvMZ7E>Lp| z-yO8cWqO(UZAI6|M_2(Y^uN5a2#eZil)C}eS?VoOXV1(m)lxYplJko5$+7ADMibN; z_gdZMp77b0buu)51q!0CM#@w3xv=Ixv&ihKJ%=IPks<_}mF(c~heC{(yl*N4$YH!T z!~2FZ-huDd;vcc_7OVd$CCk3Bh!9H+0qT6*prNJii%Cj)3Zwz#$tbc z4XZkvIPcQ`7bbc)Z5`;(tuT`&0_=L{wt{^5SUQ_@)D7`u2}x8HSt4LZHZe{TAGnwj z?10WAe?W>!qT4{Zvq_$1IlA}?{aEGy@oUOCI7bZ%7bmV$U)MHf&29t+)z)hUh4;Hu>J z{D$#}ytlnEAA8p2QBon);3=j)3>s)DwQv*INi+@=N~zwWWNB&$Wl!Cd+bMHAul&zxMXG3M;`<*dm7+)|xPl)fVB4<;)>kswM!l7_X z7J(lB+JUz9basCub2uhPyUsyf4O_y*bZ~Q}0O(4z%RJ|UoCAZl`7X)GkuZ!Ls($0# zprZa0E*+x!7TyKiOa=U90wV=)kJdrXyhbfLVCOI>^`buf@hv%IGlzBRAvH5<+cX-- zm8(%&(ts~paPkH0&z1KPpaG(58+(}phcWDvj7xOqCF5QXQjB+zcf_HisqMq+~7_rM~ z^m5%S@;W$4L`T5p8pUb!GqmAEL#L0jF4>G%?qX~I;X5^j!4-WhI5^IqGv;WMBtAO* zzMbvTjt!eyuGTWjmQH}kkPv6(UZ!Qt-uX<*+`?pCh>`yDEAUshlp7U;{Eqc6=e0Gm*ct52GA*@Vy z4)HXI@OX1XGYpy}bo=U#9=SM5*;fKHRPw_s7G5lr5tT!B%4Xa*&3ARS=dnAH@p&;z zrFT^`@Fcn8vJ`gf?MSQ~s*sfMloalQGi?2|jxs$dBc#W&ly%7fN!a7h2yzoeVhHs; zO&%nF*HAXB9Dv5wO>V)nbQ5t#2%I6>y?1vUQC5bl9>C2T;>>%8qY4c%RYi{ei0QNJ ztot0CWi9jXYJl6C?f6&&Qw2)-V9A7j_jO);#Y)70N<_EYa@#RT7XxCcyjvj)?<1 zkqD~{@k(&7w~CtsVr}3fDA$7Y1la9|*WqF0y-ap@`Xq=YmQ52oA@0CQR$oMy`mcCC z3#pN=pBy2Z46MT6rLq&D*z}z}@+_{vIZ|Iht+v+B2DUeKhm`PtUgEGwR^ARx3p9nR zSFdH*m*fyM1?5%=o7ZoR!Mw~{K~sS>I^m@t)y2Ed#rM)1n2hB0gP{7 zQ>*{NV^^zG7fB)dlP_Jjqa|50^hnWeuLPi*xnsgpGD}llU$W>g(++VzzEDE8yC`{N z{k*5|dE4*GP2MBtb9}brn*4cV~tRDY6#9gKRET|=-Y+e&a%wB+FnWCtw+&y&r(~Xp>O7PYR`t47(!HhfspqqVeGcx zJqC?GfmschLxbTGNV@ivH<)MOF~kJscs3#Q&*=r7M|8zF{Fk&CYCz<{TJ23yZ4B(= zoqp?Uegc_ZNFk``q0xV#trhPX25q#B4h4?*P=%(}fG%h&k~{e6<#GgOLxvZ!0H8CV znI90m+dLdj)Ft^Aj65E*J)`13S}QxJJ+EA>!Hh5#dWEqt$owH=#w-^x1J;eP5r&fi zj^C)E3Y!dcLgF%x)j3aFJ!RgSu+R|w&_62@+{}qr7ojd1Zg@BvH&L<&zy{34h97Y7 z0`<}`|7*j}&y=v{y|~O%xYC__HFPSJ%Y^B`8*HP&?eUxcQ<5Zhc^SifyGBwMgO9KX z;Phqlrfqk?B(h2nab_~vj-XhrXJ0-w*I3eGBGXcQM-f1N8;eN`F`rJp%;9}gVr=PKXkbwCBai;nXU-)*B;ZmXd(N|Br)Wy;KTMB#$$)-juN75f76+Nx09-U#!AUkDr?&JMF?`k{-?r`A3s#Y1vzURYLbaeuai zJZs}QHGb!(5OCBsyQn27L5F<$T1(^wI0Ub1<*lLDs<;3 z29?98>-RvMk&9grj-R@BM&~Zms#>WKOKX8+H~~($X%_#5a5?us1Qotje^u{_r9R(+ zIN_pxN#aVAouzP_Uo4NMM(b_C<(m%MT8?iVtZ=Q%!I&;*neHgcze|bt3-MPu1?eF{ z>-d@zf&5rD10N#Ra0XbGH^%d|6cs|qOSqNGfuu6d7GG)&$i#jV?h5P1C6bhSpcpgX zRwt`{4N0Q}D+m4>KV7XUDH;XA1n#bBQJmuUvjUFZ|FP@L>4*79_MxUv#&#%S4F6<) z(vH8Q40`u>BpKi}HvpKyP~T;u3~>t**lUy+xe5^9oVRuh5EmoyIQ*S?M~v&Xj2fNl ze{8U5vwp*3?m$WHUP)VbDnRiEhAoFrVd9nOEakqxti2BU5591)(U7T7D|}cu?vauZ zzdww6m5xxo%_%$Q_DPycg8- zl{=W8ysEyroig!#iS-Kl9mJjYV{BV)L8*_F*Os~^B<9q^EmC2A}>GD4p# zNuqqEZQgvBv&!y*+{;`Ybcq;(|JdV)V|1Np@6X3tAIMF<_agU6s_N8kKKehJzWbl; z$NRo5YE!GWShY)ST21Ut)YdAsOBJOxDw3l1h*?{-lzLUE+B5bhRn)4+UbPY{A(Btt z-^b(o4?Lg0+~<9sd+xdC^oDN(@J8fz>#B;?X{FbDC**x3k|Ti}KyE$HbT7O5$+W>K zD}B0AN(I`r$lkToepP2R3|?S<{s#JG{Gwa+Z4-}*MbO+S392cLztXJXL}x3%h1d=yML5bEAuK4NdJtLrBdYORvZR<#SC zK6lpQPx>`F_+9pXrkyHO-)9XLACuLOyKeLYMyl9@g%g^{{xou2r0@aM24VXY1Wi?Y z^4=j@Zl#f?{if{%F=EZD*X;vy`;XiAj7}ZPb;q{i-7?i(vWcVE$d%@TyFpx`K_)H_ z)0cG0RMrgUnQ(fI<~Y-94{cTzt13VVS|Dl86Ye2R5NFVBv4>|$lC)oWPackLN8;W% z85@if#J;o2Z}|71%0JAvZd5$Li##n)Yfkk)ckmm|w-T%H-YhGXYH84#I4hjEx2_E+ zbR!zPIz(7J3idKLeE9FAXdtNW0gWCr&l?9}3=730TtjLXJ>+~BXZw$qH2Gvo@yXB1 zM$D8AttGpdZ)Eb*m0mRmRzMg%VS9^3F0qty3t~QBupUx&3w)XJ)e^cj!zYnqvuG~) zs4<@qc^fAQ>xa9)5d!5(g6KREF)RiB&PO^Qc}VGYlLdH|(g9csz@r++g6`qRsLKgF zR;f`|S*d_>eEC^3pC0vxkk|6F2FGKbX7Mj6Nc%>gUbnb6+H`FO2`G~_S;6huB@>4LL_G9`-iOeG&M^PF^twE7^F z@$_{+=tju0!C}s<#aGML>o4vP_va}vlSHEbIr{1SzV^|)Q6h@wdbD~Lt}xw2&v-Om z-x^!t_W9E?V;V^SRP5Z{G2Z(RANwwZ*b?am zsUASEv<&YYWi?UXx0lqA{E@Tvgr08V1FqJ@)c{}Gm`s|qBRBqAWlV1gs$;C9!30w4 zxpP(f6ycK>p=y&DcHtcDRDsj~^r$)+#u8{I@1viU+CkHGb5wQ!72(?OAWh`M2dZ`@ zpM+fA?-%^F8b{+V4#DnQ;l~~LRT{1KFLG1~V7Z0OWVgN0p`}Rk1Q0 zKuluqrwd9eI<{=I|LuFy2Q$1d6~H0NN#gDO-cKDVaCa9n^EKq01 zl?@c4NzsF#cGX3oYqneOO18)QdQ8=k!K zjQME}FAXu>O?fh{t?=!c**jhp!=vE0qPhz{L&aA>3;{~hT-WwGL* z1bDHsSi*Z!$;8_)jj?duYyy}4FzAM1)>?rpkb%T_b>o}t6`dTPIOvE9ho&F zyL-lsst(yz7TjUGhhCzh9c zPX3`^ni5W5@T_ib1ouY?>M7G3t?p{Sg3Dx)d}{RPU%F7iFI$tE@n<(6smgj*Oa5fK zFg>4blG$CBIf!87aw&Pq8FX@jWIDKidKkR_(Qf9z{0uI7o{W%JqIpGd2B7bk$9qQ& z7~SSC@gg{;)Vv))>CgE}yz4suH<$DJ z^nuW-!~=5p`d=thp#yt@`{$}rwiV7#f_}F}KhvTmEtV+^JI6Ll`TFkh(b}zM_PFo^ z>WAE1s6qsKlAT;~|Fd6k`c~-Sz>|+Vt&Eay+9IYlGe_l5<~G#gQPFC?J^mShq3lt( zsBaOza!nlMqC`WzNcYnIqcc;aqkfc*J`lowBxq=P9S@5sVMz*J06aV4(L_tHdqDZH|MT9$0 zXo5C$=nuEEP(EHr`ZF)#IFWI3J&yE4;NshQ4t%<1o`>+xvI$`RMjV8TC!^ns{bJb0 zIk77PA%<7a!xf#<&wC?iif~k<0GkC^G|7^96WCJN&q>bmc(B{l4K5vtC0(7b7e>K?%kOTH-Ow;(E_A>C}9&vI^9O9mE?fdECU8{f8s$a zpUtqU^B~4UZjc*k28ZlMkzz4HTc>g4L-pV}8G0xp{a@W!$y4Q(G*NT1^`VShPiVPU zfVi@XTNqP&P+Yry_=Gyk3^|7yXvER^Vn2PnQ=;wm2hgV{!z-O$YxYKJ@#B1VppOz1 z(lY1S)K2)Fjy>MqXe~<2gtgvIZO-r4Ma&jowwu{WJ2cptZt*@HLEs9%;a&mCwhjJa zD+>QJFj|K5>uqL!lZBP3VA!0+=%ZR*Gf2n(W8vicG(W+$#Lh~sZwI|#XH~6y6>I3} zog6!)qmiNcV?)cBa$7@pu1MsEm6!PwQnlQ|4ySK7>rEt zEQ$zLI^bQ87dit=CyNeHcZxVn8p_O*?bfZ@Xt)R%0&>N3#92YGTz8Z z5L7s`C7)k2bMs)Sp}Tha?7dBd?zHphnRV*>c_nngtJRr(Ie-JkpN4FlB^faAn|e|! zfvR)jYou`aK-_aSwQM9X>S!lqPwqbe1N8_J*1_Xci%-dC4lDIkka9wl5=Jes}kk&GB0|GLTd zQtJB0=B6f|E(Tb#c36mguxCf*{K4>-!=ho|JsaGk1>JQp2 zsYo^?9C(SmF!#Sp49`^ky|R4zW)Jz+N**}dm4w$r;ka+!PeqXqQxRWrRlh~}czH)p zBWZp;PUudr>aXwrQiRFuXcj?b#CnSF%R^Nj5U+g+H&?4~_<@M$)-Ps}R;go&n3r%v zy3%Y;*-)IB!L5WcZ^(aT%)9$sf*JB7urKAeXsuhN#69JtW?2=M#YGU9&g7GlKs(wl7b4YnDE z)gI`Fk8;@#&bBI1LSLLQqePXC3O4-bV1*wqCWtftduB*ildQ)(gcS5YuPKjrch`8s z?hjIfD<0Vk8^1|Lc;0#{_y-+ve2Vk;2I%qjcs)t9kO9A*KW6u z60zKS1<6#R-yiZBjGj4p#TajAY4MjY;V?BW<~>b@xSCT7;n9GlR+MmzQEPQ1Vg6tb zsIC`nNnDRRrgQr|IVl+y9L%L?WXKF3G8g(lQoLbdGc*yTT}iL{+}nZYg+vMU=x^;1 z-ipwoaD9gF)`6?xpm$AFp)>H5q`xu<8Mef?&*ds>*`Ky6>CqQ+tVp;r1o@0IX zyCe1X5WMPUr$)n(1h-6lp)5CLC*iqvs*JYd2Hf@t2fruR?|JOL8lIU3#(IfKgJk06 zn!ZEQYcc?iP<*UxrAdz5RyIIA&y1J}@M=o?a=}s4Lrn{v_LoFvr%NBrSDqB$j^NhA z;1RHk4Tpj_**ao<5ZamvAS42gW}HI^I!w@)CJo@HiT6#QYFnpK-%(j0;cJal{=P9# zEoVPPx=SNy8ldW=t*-|6Xix}*W4v@BPlg7BH0MkIP4p_^jWi6KiRG0^vrhFLv9<#=}msA23l6 zC&+scXEM@(lV5U%F2*Q!uOJ`a4f*fv8i&qNtdaC{_wOiTk zMc|fu{g6?-hg{O79MfuV8aS#r$3f$TaD>h>8_^9kGJW(u( zMAFtD`61Cn0S~=@3Qk!({4^~rh5zO5uOHJt=gXbkjq3gpYmSY(_xYL}n3xFA%B0LM z%FNIl8Y#Qs12N53R8qbUsIBj=O&}XLq{IEvDVvl3=5$VdI!UVUz7xj^Oq1M`f9JH% zk$}qOpE+`idikQRU*QHQ^00v#Uz-~L-fUF(O7j(jv@ zJu6}-a0E=dUg7anbZq-; zhEaE6*@UN91U94_%m_$t%S6vIP=f;aaCb;fSC62qR?s}XMiT08+tuFhqK$|b-$JGJ7%&sT3o27r&IiaEX)!QQ16_nD12{Da>-XV{yoo}P;>R%xW)HKJu* zzr`!nkcg?y042YSvaU>wdTFD#+^};P^3-k2OQYcY>ET7=iwJb#i;m;GAm8d&@PB`k zOE{Lx(%JSJdLjv(@P0NmhkhnYX5se%*M50;fV9_JJw8Czl#y1wa&1?SfkYtnT8)y4NtLYew059WkkJ3@Ep#I7U5a9|s+St1`P~*uWMMe-%G55TX{X3p!5;d5@kK zAotg5q5}s8mtZvoF!BNzg^>SH`Y$K6T_8-JczC?E=m_X|kFR@=cD?G(B`6(x3BXDq ze&uWe3JgkB59Gi&v(ndzQMUzi+--{kZCrH4qTN8@g@Jte9GVK`coW)AQ)M9Uem@KP zvc`MZq~!YQ%aeMYHoA{oStBR#Et3x{z;NpIxY9mLzJlXUdxOWEamD&P`6i^_{; zJEll=KyP^`A>5Eb=Cns7<^Y{TgI}@geJ1qo!T#Uktv?U`Iz4_g>iEMxm4?g4yZ2Ot z?t{ybhr8(w-?)XL|8iM3IYRu5aLMm{!+{tMDUs&YbQk|DG$OA=?WItEY|R4YNo|2p zNHys(tG$2coCyF>!f5?Pjc7{?*uN(%!K_p5A z#~WXxkP^6d&IP2sPLo7{KYeb<+0@Hi(n1;{Ko~zJOq58I9B; zE{H3aX%ME7z8(u;eABTwLBy(+?4Si>(J|^SyHc{)R|S@RD;r?DTd2sWXS=t1Z~nLQ ztucjkPlMjlvi&_G@8IlmFsWal_!;T6W}s<_3@!KvQ4xeY@ejq<}JiP>p^FrOr*dF?qL^+KAD6inoIpiQBx!bcBdc91<%_ozqPUp#=ym0Rj%Rw=F`>-5!N2T-K zv{83gfmw@GJ792^o}l{QIWNaoMIz4BbFFF{{&I;m&qh{dEd|y&U#G*?1`{jmM&vg8 z#I1HV#xwYiTy6BVYd!~Iid|-vZ|u;=*ur$>BX{y=<;5^tT;uZhjC4%71=9t7q(^d5 zLpp2Sf80$1JH0cdgRTeO9coBSK#aaVG}H|ASUmEkd#egLkgCQVxlvK^VWVNPFxefJ zCV$#Tx25!6y`DrokGOW%oF5`pg4^Y+vVs17Q6+`YA=*n=h~rK=u)UfXRs1MOP#UC} z_y8jfdME|5$rLW_8-@=71A5D?m#ql8L|ct5uT12{(h-HnD^41Sh02jQVaqOYmQU27&v~xLR9W4gl?=yqCG2O-drnt>BIZNVFgwVkEqBnAxdy0Upe@m>^1h73Dyb zPYQz26~xE;MU&X@Lft#wdtO5zd?j>BCWV0zAVI6bk8T+PISe7Qxv{;UR+%3D{2ur; z(LeDCY9jL689dQFv z&l&~yW7E2NS({8yMib@*0QOmq-FV>u*}MpZ{oO*PC=D=emFN~Z0a&Ja5VT@qoqvzF z-YW_h$#arkLTnqk5BQ$)9Ls3y2Pbru+83f*H;aXh^Cg>BA#--n{$B`sJppMMfMAJ;^P`@X|=-(<0U7H4_ZItm4H1d#Qwm9feUJ z2+dWg|4lxh`xhZ?nf%E?C{w7|twa{*?J>gFKhZ{UmLWRdUS`W@NX*9DDVi30$CQOO_c5Ww$;kt8KI8Q0j@BQFWAId)?NDN?d!)x1a zT;kImvbPa)6#0A@d{l}~fGLt@YCt!qy{vpdRzwX;W!x;=uYzc3ZKKS_G=J7_J)>+1 z##{TPI{#G@7Mm!52fkb1-&H!-9m8;(E7u5Ew||yvsVB>bi`iI|h{Og!@uL zINE~H#x;m1tLIOawX&AIVPDAK9zSC)#N7|rw=vzyT64{Nm{{b1*A~kBBR?kPXskJu zGp~EVD}ldZWnfXi>g_!rHIk@HpFqCDD)L7KzCNGdEppi}-G$HU7fi8ppB$lqSc6>> z!Vn-{j^aW5@$IM1^Z4WBT*YeD+%xla!xz&p+xZ_geP)<)V=%1W=lZE}#tjl1hyt4; zTOoO=EMm?Qq2CQ&o=wD5hll}%CDN$F}4O!%q( z#tz>fcYU+puaviOA^GlUFrjdWxuj{AAA9qM8E$~0W@Vc}N2saf*glT2%Yq};*BZm5 zm}6OXNuijEc1%bgV!O*>@Vw#mdoZ0dkJ>Ukm2!&*lE{v}-F2STQ552r$qxP_5uWV- z9)&MpE5gtdz9 z>7TV62p%Bafk;;qzyup&H+a25WOmh`@U8`m_?*gi<>f zP6#GtQ?4=ZIUdz4!B)1$cac1OK;i^G`uchv0G8mLf=DRh6Z4eCWM(NtA9v*fdjf!e zJmQ-rft;``(6VA?KxN!(0Kl_v_e$a*<=xJ4k}JBU_uqPGxkw#gcH6;0@vPoeo_g`2 z`tss~b-58CHhksJN%^=-9Jxlon|7!e(r!{7speSvzNOV{$V4@K;FQV}i9&>Nbn&_p z+xt~}idsQQD19qsprW5V?5v32?^_03Hw;yG?I(qZ%=f*q83EcAjkG~;H>FpN14i$* z4l2o?IMw?FckEouahcV?kp@o_3z_4#eJJG-FeJ?+b|Il`s>%`B6)4kC@!u}}A$N5f83b-TP%2unvXM0eWW0svO{D`OcE# zNl3!rz@PU1@m!PbWp9S+UGMG}yod5>ISrJlNR{}P@99hID#iggVVUY1r z3$nal6ZNw8eUXbzP? zNP_l~53qS{*bpBy#pXMXs*PSzx_=PARVr;*_Qe@%zE_9UQF}9-v$mUq-8VRLXe^cW z2zn zuw(vsX(-*>@-{VL8)fUC3V18OO4`%!0d8|E^RsfPd97XI=p!r6Bsxwo9&0@Yg*(#_biVD?Z~#tbG%GV+bk$!TXtZ$3lHx-%^+ShLVG_Zl2yM zP@z$jt^dp}0%h)Q7bu9)A6}KgUEkJ-7|C&=e0?z&%t-J~vZAC$-1bJVEvq^k-BhrN zxQH6%Mf`ns$geI-lX;-3)^4%7yRGH*uIl~z?)2AHvbuQ-CrAQ3fb~%dxAc{=I!v7qVz-T6Rm27y5I2oVmwrP*E9zQXLYiaXu}tT)|bl!(4lNF6XS(i;oo2T9ODirMCI-Os)W z;PE_+W)3$>w(~n2YjD(1-=bAQHVi44w$N{Pk7!Gb1`8)zn9;p7Nc_a|5iMGDL?J5) zZSq+`UOe5{az&UNSE2OG90Vu?94~`BLD_)y$Em+!5yK!n{(dq~CJ9?N0!u(l-g#X1 z0qvG4!0RuqBnLV1=)J-K0{eV2BnE2ze$9T7LKY zQ#R1B4c*VWEj7NjiK*p`D?`c$fpdsBHBa39l`4UWNUPXJat)Ke+Pjy(1oPiV&Y|M{ zFL5T*MoLfCp5S+I*yLoS?hvI>>0YC9Vqm_^npbMXdt3CqTtL5<0DCYrH^s2*c1xEyUXXpgxU8%Ar8F) zO;1E2x94t+2z_+Xx9@b#l{Uzn&p4;VG({1cBrha`imHx63!4GfSjnY5r!K%t2^3YeLT^H^?ze zMC0WLK*PRVt5yLnP@Bz~3N5Q2!5$KpT-FUPyQWqCS(xF6PV*(cn}@z4%BK75^EXW| z^DNhEj}}6DE<}PT&~{Y|2E<8e=;`l?n`l4npWLL{-+EB#pT+>ww=yKhkKun0is#8l z3fFXfT|6dMoYd_$xtr)3q0I1%{lO4`kOJtS<*G*292g)!2pnW5hE2UG?h|_2I7|hP zv#3Yys!z}!?&~~JmD>sJKeoJnD6#nTB#(BPZCX(@@)W!)epUU*#Z7(wD&&6y zKQC-Ga3DHo9eJOtdu-0NxeLkgl8iDYz{Rzg*>^@7qGJC^dfMj&-h&0r$p`17`uYR^ zQ^jOv@P&;OC&1MSCbgRCs=Ak(6mz(?m2f`=3QD1+j{~39Jgvm!?HQP7TThD3FzsQG^eNTh91;~!Oqe>c_F{>e(l`oP-Uv~&U_a4?< z46i+gTPZzG`lH_8LlpxmaW~wU%x~32v13tN+$L$+d-vX zViC*eF%(NDXJp>ta|yo@OQ_?B3oqfh&@2Wy0yv(>Rbu-*sBY{H!wGWOsqtlLqf>ei zCmyun6-WhQx<^lBB=8;AVC{(NdcSOtQm6LFM}O5p)Oh%wm1eAAU|a;y{bgp}YAtjH zHUzHpMnpJB3D*^)8XO! z1}_};w}z0`9cG6_dRora#RH^+I2ZAB40kvK=nN&9z;M3?I4>*7(IanD&CQ*^hf77yw%_s3}_R%vj%8# z={s*58mXjZAxH*r(Z>?N%CJVAOWeQEt&+D&Aljl|ig=;xt-++3i$e5^30{!F+U1J_ zWPG0}HRHkYmg*tFA-vD-Mb1RpT8%pY3PBqp7MJ1IqQ9o-;T6*c@2ErzM{ec+5n2rJ zKjF`AF%MBNe{EPD=pOtc+e7CzjMUiT#7Wkqlqp|-X*BmcVDfFf@0TWfzh{Y%2arS# zP$q+8w6cK*lL94Xta%im5kKKSc#k0fqjPzZ@F6R?Kc)Uj*z=G~q-1|co+AGi`=1pm^Px0>4kHT)JR$S5!bVq%J>0aYi)j;NG#8b6l(~zI7e$i|C>)3Y=3?S z>t<;TFxsnsU+2DRma#mq_au>Uq;ql>8D)RB<74PiHsYqp`z+tH9nt7;*?P@hEe1D+YVm@xUo7LTIAKD9BU_X26vO8(vn|ijcgP>7QmC(jDWic#Fl{{?~7rF<$4+OI}HcB+}ojXgSzm9I1lr zoJPclLl2(#le!}aX73@<$w*TzUhAeVLxPev>-(aQDY(4rh3hP!gsNn#MZ#~a+p+*= zJ|p?Q*YT4?_5SBv=NEbD@r#d=r)07HM1eo3Dyhtu`sBL{e}{W%^J+2ARp}==F)7D7 zP>&xizTABaKaOuF?yva|7>EZ?fbO`D0o)x@&+v7bZ8Z+dkG9+|7Xglt)gU}jB{*!kK_H`V=IF)^D+8%FFm zH6}4&0zWVjtfw!Sp^+Q;S1k2thDQ?q!~Wn7i#XH0_Vm6}rZc(>dTIrAp&W~CwM=Q@ zRKGseb79bayrC5ou5Lsf4yhwJE?CaUsjv6br5A=~QA_Uo*N6-@R7D7q&AwjErf2^( zWSX9HwY}dHO1b=Jl*kHAl>6*da$i4?CWDlUiM!X-+r+$2=Lp7wk60qIK^?!LX^8(c ze&=P=Jc!p#4;VsXN?-oxX!kW&gaMjXI6VXwPKgD)!bt;@HD_Z1t)g{=Dx7VCP2Z0t zqnW<*-3Y8Rq?1^RKp*YXh|^qS9tvkc5(Gh4ubzt7s1T`v)gKz@+@IhoINvhv*PMk3BZE-= z#8Ff%IOuDH@V+Z3*gf01(`79eF;b)LA!)WWhI~a%1f~QLUSj=mV`Sq>JG}HSPm*TI zUE>7>12+?dk^E@Q7n(O#kkUp|S4I}^zO5)m>u+RY-|Z?kB$#L50NF+o13K3J!hD1u zS>C+;9ewc;{uhQ}_B%oMA?|eE#SFenxO8=T0#W039=lmYwt`n_?vLZz01`WS*8Epp zH<@qF6_4W*ge`hhko@x53SV~Afaks`s5r=|_3ylLwkgjtp|m0wbuME z{c02e$BK_0>SU^q2Ghdk;G6C)rtx%ZU81b#7}Zj##}m_(QAMCV;)5Ew`iRn>Q{#w3 zD1LO#ZY%CpX8t}9_Ir+8l9Pnp>|Bs4P?g6Ivo8wdtxbsp%fmK{_y^qm(3Y8Q(G@~< zbwNGe)%#HJmsL6gH8jB{@`8 zd8td8bg>$(ax^$&&MUf<^us-jRuc5%v&LS`m@QOI$2iU=U>HNV(fLb#kwwQCV%^=F z8UvK-8rn&DZh71*_Wfc=k|&^w-kK*hJ^8DJKN7v+6_QQKB$74@Z6ST77vkeR0r$7c!a$3gXJPd6;4U) zQ`QCJA)5c*K04w+KLNj#O|mlt!w5OmK3s)@AZamMYx@!UR}@g$cRTLyQ=zrV7^bXY zFfQlgDwtSl4fRdxWU7W7ma}io+_8Pd@-D?IsYLzt8%&B%)^yd3TvYM0^G|;njm9jM zh-X4%3qkPZ;7u;bGH99#&|`#ictxh$E^zzOc>1Ez0~a9xh9N{rp|%ozjc& z;VjTu5I^ZD&vJu^3UJ-)j+l?FLwu#GkpT4#tK{Nsq2+?l4_^hyAnUa#u*{cmxukfh zBtQrUZVuQ0MDNrdA&+!0U-Z1$h^TL$7b2EmA*7Vk`+-||!``5U!&T>6 z(pEq05VF(ne|cn(EIh#p5k7nXJgEhRqtFoq9Z={b>>Sgyb{RybJ;W1o^~aMJA4`+? zYZRXG)G^LUdRf12ucaF_|51~G;*RgTmV5Ujs5J9cPV2;jcM55i>}@N&Q%b9bCTyKD zW$AOVTZBv=nV4pcK-Br>gRzD%(ON24xMwq!W6Z+W8t92NA&*tqPGz-oJDKxsr&O*l34@tlNl zs=4svuWtjAQH7FPg^}8jf3&@zt&8lB4V0#(UDx;I!XK`FdoS}Lw86l`Q=xc4Kml)V zCT%f?t~{xQBu>6S=Suz7g7SR+#^EgARZ=nySm$KaLH04*WmDrTzHE5)hTz5)Wp*5!Zj zOoBZ`n%il-hA@R1C<(gczDRMVf7vEQeJM%eC8l3FmZGy^pIvwr!3mXwpPehS#9r8_ zLv&|}=z0y5Wm^zd<#E(oLVpWHt+JwH30)GyWC`s{k8-f#GOjv z51Tr08~&F+4RAnSrfi8)&J_#)tDl1c#4AL{aWv?4H@UpeTXCvE^+Ju9%=Z&_{ZmmXW47Rp2{bE|c0P~q-rAe#SJ)B2C|7c$Y<3b%iS*jMN9|S~ zWb>!|=Eekq8|%QLjvz_gT`UM9EM1*%;18d`j++ih1OAEt=s(cq-SRvbv57RbqV1nW z#vDY2nAd`k@=X-oI$8zUVFvkIU1Q+!=^?MUD>p7oOF2a1;Z~JMWP19S{p4PWVI3;( zVIRqoTP?SDXVN%hzpq~`w-Uj>Y&ez*R++%hO53OibmlYF}qLk+J7`#~` zCk0-+3#o&pL+d(luYEHCK9x@Rooso9e&9`b0fyVuf*)rj@h=$B6P2Qz*8)ntSh^{_#ax- zD{6M5!W+?VaFdrIp#VCb81VsNLl)*c>;tql9WkS%7Q!i-C;9!xaE=W$)#DxMjb*qh z;YBZ;vl1QuZSqxDEOg&t4Bm&0*4}$zF{4Lo7^ zc!1ObeL+H%kZr)Cav$GW-yg36r#*1dZ9lmxOa!)N9C5>D$If37_xy z&8d}X12K3x2(A>;(Z3zGiR3~7i9Ic6c^LZC;tdSvqm&N``P=4;&&xF4+pkJ^m)Z|H z@1=$o=q{!I$fd403r|3Oa5$=LSb3txKB=UOpMn9qKX$vkd4Ify7iYcLrmn(f)q4C8 zvU+TxLQLzxfWFzbvj_NoXLu4vHOPrFWcLBUX7mPPmSE0ULSA-i6Jfx=F2Ktk@_K6* zcDZGT#4qN$t|@w!UmdO;l5x;ra)#;b{7Q&8X0Yr1{1G~8ZQ;{w#UieA9Q}H64QHyW z*m+%kZPBCvfzQN zUR!I6txyefOzk>v5u)B*y`2EE*OH%A)M!XQ8NF}omQ?0YaU{>#N&h+uko{N&H*qCg zSL!dM0%7!6&3s;9fP1YI{JbDvM(7n4s{~LYTnZ$_N*H0jHMpHLf9>grhP49s_QxRy zNy9E%C}EoiQZ<*5kmNa??VpwxY#)J*#$wmuE%9=lSXmqvfG?Vpd>g!BKs2&2@1iDh zlOzVtHoP*OgKUwIBi*l7d}jm^iFj}HtMAcJ6ut&pL9jgj0KzGt)o0%w0#{x3KoMkA zaCFv$*q)szoQJz*TU~F?J7Xhe$4ELUs^EK>^X-T>$Y=oOt8*!hm(P;u_noYz!i$dQ zARlenQxsP5gqKjy3$1T-K{US0tHJ{Q*$t-6&Y`kM!kzt}c=pk2Ldf3*RFASfykcQ` z2^q1bw%yRZBCP7a_oU6TXc6cFWu4=H4h41dFi_BH#S=COw{aRz{=RklWAn~mQ*|_C zVuSPt8ccdg&=o<&ll~jwhP@`OI7=+bWhn#qh;1owxk1<=`Im6Gz9_Ahp~Y3kk{Rlk zYzGZWq>k8%17a&kGwARW9T+}=P;mhyszA6Uns6kUWMoR3#EB?0qo;8kox0gXdxhh{ zgeIWArH78Xtl;=wg_{I;78?55_9*Ja9^d|#*&SlvKDfD)3kw``AHa(*LVDb~Rq;Qc zrPf?VpQf@^7OP--U#To zvnb_%G6VII=Gh|Uunz6xCRblx2GmMc3gPG=o{*XrU6~MATcCB>5avfFm zYw36UOg#MS;<_v&wQtL(*%DeG*=f&Kzb;E8~x z?wNFgbmwasLe>A&U!9@AHU3W(XQg;?p$$;R@P-t$Aewpx3@aM`3#g%EBNk z&Z?+R#2nZ|mZ$;z0T}m~l*6$$0^^grH)ppVVb%Rf3E{2h359lrThPe0OLEltB3e{dMI6^qpcYTz` zMQOBLI06v6c((dam&F|;l^S>zWp+IYk_vuzBf)bF`vnIbW(s+V$xcMFkG@&ps{NuZ zf94?Ub4wsPF(^VVqgMF-(YH${>kIyrvupo;Epx?G6Ye}#{e3qYVX`D-nsLdLo;7c; zP&~?P`89sP!m3YI^0!e@@W==o+Us`D)(85;$q%io5==;r>@o%}Mj@J3;Rgtk1dUfG+VLkuHR; zYWp@0EsjUOYdF&pmu{8ZT3Y4CCa;z@!sXhfDh^r*#w$_jpS}MpWq)cA$Z8?3iQPor z)A{=Ov0IK5u3J|kg#ct)=`wYSo3ny$9}U1YtSxpUi~mkN2umW4!r|Fg!-JYPLGgVg zsJyXMbMjWWvF!E`Gwv@HIfK>haVhl(*e~}Zs1ZH{k#idY$L&=uaN|C9J7`~m@D-1_ z9t@Fi%ua+wO=WQ-ZQhT&orgs|?x zY5)8XEz{Dn27@Z+`$_Du1tiw&5@W~JDxrT#Cr-jX+_!>QWjWU*;DunupCaf6fSiy8 zV_&|rn!k^xQ;1&+2GBM^Dr>xo*}g?#nGOBaZHjC9Uwt_^BzQl?ycSncfgjySZ@R3( z>_0=KabNa^#=_KN#7U;;5BBn)ngk(mC{g8AH*&|F`3k zn=_zHn(;=C$nu|#VVopNHp`x&v6|j^nU1m zW-j7JeNu3lv1#=qoX#DpmytgE-|$j~`bKlY^xLD{2QQ%sw|Qe9ZXy2eO!wj~sp>fL z?*#Wsqsr6|mxmIHOOsaUFzsK0DawX;feGBv5_-5Mn>vlw%MO3Ms^lR~82C9Cz1h}t z{fF;GOi!J0id@lAd zS3|8aRL^)`1W(^!^T+%fD}Kc*8E!?4hxDNadTF6C^OP_2(&Ex)mlx~n>V-gqA|5?L z^Y=;O=kH;brsSw_q%?_x;P@SG~cv(xHdS>RlE*&SrSp4EHbbm{lbS!s;M&QKH zi=5T-C9zCT$Atn{#EpYihzL9 zFe#A|>1GN7l9O&mBT~{xY=nSxBRvoS5s(H68Qmcw(u@wNjojGH_S^UWdi|g5(e`Nf zb=~KAobTf}k4hU>of;u^rG-BtP7q^`g4KM93HphWz!*W3i%01BTdeac=QSFKm)=uf zRdLqM8FYX)Y{S`RvYW_{yw>0%Z1ovutqy9*S1 zh~AvEzN424+zliuc~+Rf^Sz1x{)flYSJlRR-j(M|8nAXA~hA zHhA|!e^QnEanI^!D3qFp^rz4B-|2GVdEPIhxQ4{$0&N{7BW2#-SrdKmov)dyWAFL}|6-Y@_!QD$oBga5kjTLjO#v>aml=*=Z2+rhE5P_L_ot-tT!{(lA zwRO$&1W0hDVGdd5ho=ys(zzdS2rc_X30z3fwcR@1acuC2tBha;w`sw!TY@GOhoiv=Wsl`kBr#*ybt@GOF@aX0V3CE8&xtSb zV)$iIm!{?NcY?k}g&ms*5J@D7GkW0zOX14G#Z3`jO)pC`B@Mv)ls%Lq#wDxG98y@|%#8?SLw79ZOfy1@_ zrvmpuYb=wmzO2r@$CG(1vzOhp*O&{@{pu1xp=igTS8BDPmbjJ{_q>n1Qy-m483y`d~unP@}A$=vx?aRq?C)DjjZBjoEvfJVI_8+MS3YEvEFGO=8u;Z-1iQ zX^Yj<2DEWS*2nr+zts)~S|yw2s3|A8%y=JCl{-LBEba)=LkSDKfrv=Sd7p>kzPR3x zoJeX&24WZlk(u2_SE~@F;gK)?x#pDrsqhq3Q5~wx+gBU0>Ar_o4lcB>YwvGtrA*;_ zu9itg8_mL(fCJ0C2BrUufS-%xbu z>tIKfm`R(Xr#|^OtSyA+{1&_?+T@)KaurfIcJiDkw8Cv}-a{VmMLJ zn`W;ge9Xv+ii&Pjxxg1LHQ>uaapz;A46y=e5}F{##n>`nrZd>Uo5Yu}k8K|zx0Sqd z+9(Nkh_@Rs-r>|l7H}itxM#`vcdf~Ay$!em;KjbH*_~;kCwX~NNYn%IA++P=8wp%l zRfGDkL1i2NLtvM928TsvQ2A2)C-|85zVgTD_UfJ~x%fAyQ|H^jDoIlnfbT{o!DgU1 zYJQ>$er{hsuyJ7o@HnSyeZuXb+d(kIRaHCZzQ*fX6Ben?k@%zGiI^+R=~E~#IPYJF z^xi5Twa_Srt}gdZi_M(9-YcogSAm50_cg!M%1em$1>Ts%%0C}XbhOj2Z5{{bJFkWA ze3~CEu91cjk2o?ECR^FM(@FdM=;L{~rLPN0x9Rve#dIOy6k1xSpEsV?8OtY>qip{ zj?X6p%0fO%ueE#&$5n>_pyK2s_zakl)3Vzm6RX6Pu9|3{=V5+RBF$~8x{L5FqtM7Ys^F+Gpi$Z&9h4nYxk!OR{g6Ln?Pj6v#r@ZSI zH3Hl+m0h39{O)-k#)K=0wcl*El}vpiO_3&k>(!3Y_)yXgX!^yb=W!tf;B4T~v|h0`W&Iiqmg;8#9M;UOdu^0Zj_oL{Nt$yQF|Wi_$ps zukd$9A%~&Byu~JHPx$TXBQTrwoxU>0jrOe7fk;s^>29@IZJ52a`6dV;2bI3L$={`h zUh9EAPw{J?mM>S6=Kg&A@wtBEn#*$8>5{nr|Hg2O z?8&UJ7J9rA z?ubvU5`7cW>hR$3th;xhQXhL0WLNf%=O&VT8*3i6up*#uP8YJG7QA_M{g~@bp9{Hr zGWb?9WhA*f>1{5iw)Eaw?dfWBAm1LA=eKnuEo6`noEPI)iSmX~iGCfZLT@Xc65(oIuYsK0v~S+~@gwB}I3M;d_w9@s(T?kDE@ zjb!Ad(4QvI5(1k64q>%u=&S+Ml8~54NC*arH|6GdRY4GZOZVSb+t<06G>IeusMUD7 z%_xYGtK&Qlj8D9{n!fpI8n4`M?mIs0~?uOr!)Z>OQQwRsfJ*5{|A-V7F)h)jb^s(G-lsC%=SH@;W&B zbp3a4IhVL)9seP7@y*>%72;1u#DYe)ZKch(he2|q-~o9JvN#@T;;BINX-?@|gb0ziO|reZ*3?qX6S2$|`>Z5%zE@b&FvC}11{WLT+dS&ZbZeIVX` z{{=M+o?Yd#DF;9}kBzd{n@_QyfQuWjAy0TLxY4f@*`qNqqe!?O@A^|wHWrah3NUy| zPS{xob^am#^d|=AO83kt{dPakndC$cGv9b!NxyR4Fz>dmpMjJL7whA*xA>!eu&CSl zgox-OH!%-N>J9z+Uq*?w$H=MoOfx`!}obM$s{Xkui6()%z_FoW$zSsRoEGh_OtO{)0)UN*#cs|q^A6fX!xP>){=J0J|v>P{2 z#;8oo*7Q6|gkN>&!3T!2(7q(&xav^%Rl&w5D`*Lfx`5Qf9txGeyrn>M7i!~T25x6p z%#{M=G|cv&dtUrHvBq`A`)R3B6dC+Hm#DwDD%zt ze1nW)G%whkGAh4?WsePce1t?#BGElWn=WIRv|o(EWx7*!GeVz zOi{I1HW9@YHc+QQ^+3f{*&aC%#qq4Vp7^gBn+snmM9~EU#+pT%MVVya=v#9XzASNi zcqfeavm##?_JQ2(I*x);<>vz8R5__bB0Uk52u*o!I)s``cYPFFp0b!4b{pgNA8lu3 z{qlVko6~A(1F;8T`{_J9F~w%yUu~bqx@Nk-aet7ZP5+5nFc5{;%GRqi9H>gKZSsyv zD3--c8ZekSze_{JyYvg+OjJrSqPqPE#=dcJnoi`8E8tIIt!dQS%g~^$a%nzz{{pdu zX+EE#8dBGyDhj%u{1k*`T1u4FkR@3_>@z@tA)`s+=!xDK=vC=#B<3O)3jA@h3XocF zGmveib!UQ5wo(y3=o}2O?6upihE37U9v(kAAs=4C0YRuUyCeLDKXRRyEA{QSYvi~T z64Dd1N<2jNz&%~J1ZSz~#HpV#-;49sd7nHI!{`^$CFa}zkFha2L{?MMk<{#!dk0N# zPmrb+j!&dpJCdc!#oH^_N0<>Tpzd7E$rA#}7n+u;Y$B+3r#_cR7=gf6ToCqG-M4Js z9G>ak^-{~TFDRApdWt!rG0X}-CQuc3ion*rE8!qGbZL(S8P)DAj3R3762K=alyWC9 zHBUQ=ZF}?tqrA3X;<@N$Xg?LT5DrG?`aEGiMu7oee!VAV12kTAo)lE)D<2TAeX27dx-J5m|5PH%pbyGSQ;ubI+uc;Fj7j37T(}JjR9I;?>r;y2TTXUNgR56Rl60rm zufe8@I3k>}IbakY)}$N&MmNHy$@8(!#JG)Wst;&&M`%?kJ%nE$#i zvgi14!hesWJw6h{8jU0*Fd5rfpm_uKxW{@H>Btuw9}uQ48OW7dcqSu`*KdQW!|elT z7gA5^04^F4eIyZ6P3G9QenQa+w$ zz+Y~VPSLy9iah{5oZAUIBd~+~7VD5GZ!_(oCr7lo8<F7hf|j##CYMO_ z{hE-fCeD-MxVOzN)Pv0rpbrR{*m}IMk;K5j%h{m`V=yiZOej0rexIPFT5AgOWT{6j zZ*;zkg(%(4vMZ6-P^%J`Fgr`1Ge#vpk|mCA@7-SP$GDHLF$xg(-&_2w}3T z(Z3%@!O?op7ZX5<5OkcME`buNQ0j;$`Q!Rrzjb-81i8yHk;CSt`Rp@Bhp9Uv?Z67J zczCKdjR}P%Uddzmz#mwSyqMeO2hav@iotN%VJB)sYLb>5e`o4K5;k8f$;LgH6|ZVU z$3So!(jIM^JucvN=<}#w($Bgv2+ejAk(llahp$0VKzAAVWZB>Fb1(I$nd_@=NAkHh z*<&|W9~;eG6;D7TXQ>`75hkC>4sUs zgn7`1;a&yx(@yo^GtF30^W=lq4yThJ!hfJnH8ZdsG4!)nQU!Xj4P_(ceG%e6+Y6M* zR!oSkM=F_rZF}yO)Z+nnSH`0Dr|0Ub`UL|RDXNP~tgrwZlDLf6N}=)y_~Fx)6yeFL z-FG2J{8xY-l4sw#)A1$^+d|U7XgO`R>W{932^(>UeuaOxWtEen)eFx*EVTYQyDYh+ zhkuYw^hmH|{4EQw$r?+<^hE`^U)E$U!KGlsMZuA^#(qE~`na0P_VCH-XX5c;wHvT0 zDGjlBtCu*T!^OGOL(ceR!uoB{`vKOwXREDlxe5F09$8P=qI;;;>J0Ho%J-5U!IZqN zM#!2pPnyT!vc#LbB~+BjUZ0_b%bf(gFnFlhco~9LueRJ>xqL$XWLmM-Fv8|ecPU4_ z)Sj5IwObZ$+Zap;Yp?zTyyhjI4RG(u{j<$Rb@ZP&-O^95&ENYJ|27x|FIvk(OcR9A z!Q%+F(^|_`hELn0Q#Q=3HqEtv9A|8$6E_D!&BsDGDV$BT6M~j&-Hj+Z>;YW`?1K9! zEvwjm_<<0WjKm47!VkoSkRX(27 zu>x;>1%5@`0t@J>i7CZyyfx|b7PL0a|8+9eD3UlAV>X&8-w(-7-*{qS*#PHIdhFm{ z4mDGiysJNR#S`2SOvq+f-mS~8@Z(dGBkc1bd-R6z;;t^>NBW>^N6SkSKQE4ImUV-9 zkRLp9@@MYx`-A5^0PkQna4wL!I+4n-*7McMY74dwZdA3@)G1`y=dGJJs_--;@RBMg zMbr96oBl*L?k-4^x{Z&&CV{UB_v!6d_Lt246J3sAO#!}^oeRn%q^Kw^rEX|K1+O*t z=?zixLEm}WRX=#Z@Rzg+7S4q`nn9wSJ772F+Czs5WG}bJHlMxNv9ECt?sx#U5!gWA z^WUO2DqQ?I(Q#3R#NZ%(Z<}j;+=2zFd)hLwWOX*Yxv?f#)ByDMBzRbz@@y9uX2oF@ zi7_s5TT>WYHD9HK{(ga8A+{+;B{pn=f*EwYb)$oJ_hVOu>~UN{Wz9Cm5b4<8 z2y)pDcKP%o+FzG*ueqZyj#IlBxn>Oe%O{|U;aR@fEdI7k`pZZL=-bzYO8SuwW7EId zIMVcH9HIK2Ge%8_%y%mao+Bv(JOIn+Q&m`QA>uRO6gt88EB)djV(m5Ee_S&mOhM7l zmo-4nwbyxs!Rn1ueI6X=UqXA!Xj`AlpPIv3Az?3G#cPUI-T(fSR8VW4ktt^A`SV1t zLAH2}D_6;fImP6J99x!I&={mKk>6 zs}VF}?qe`ImjZ|rhp|V6OEi&2Z;6cvI@s^c2p~74|G#)}8&TD~ji@LK``h}_GeMX*d@vx9*scdhLX_Oxp=o0&)@!um95(ZV*j%#)ULgu3u4N#o}bTT(OLZ8w?nXEdR5gt3B`IsRMUn zF|uW%mDNFU1NX+?sX2<=EdgRJ{(Kd)luv<6=6tE6114ja5z4ghACw*?kP%&WTwT&B#aa3X zi{3B;=JuaV)zvkJnSHjDI{6IToyZ)vcz{5s+56WImcxvLHRk*!4;f1JDsoeck3F){ z%E$c&f?mTDu}I9i7~Aq!pOmM}eNi}Vs>)B%c16GbTD11`Gz1(yz_g8QpA+XUI`Fv& zqEF-~p)RDuih!rZ@)6NNY(&~5NuO_|an!^53Z{gfRG(~+uW{SbRkOUs1kyn0967hF zf=`;6_7(g(z6Py;mG96Ok~mu_oB;M86w(LDl1GzCZ3E9oW0CVnWlPCnt-Hd#h-?Y$ zr`yH;I`$g zlhXQ#xGS+mEq}eW43$z7qlU&TJQ#(px-$=734{PGs z1CzaVq#vz*!3jibj8rN&j)CQb98CNv_kDA~#hEip@C0o1fmiTU@X8-I1ekhY9-z=$ zaY_K&dj>86(Z8XdCF`GRUUg)o?iJf0_RdnIk`oZ`m7i062l{-rCUQL2ww+in2ANCn(k1G2UqLqhrwrei&pvIGDmQ$MQwAJ$4u`>kD-&uYWYrmskDSfQLj5R1cXHL0h% z_vO3y_b%es&gSRE5_=kTPp+k!!`}Rz8uVgQY}wFHy1|RRTfahfOFZYZ5{|FK3xe}z z?EP|cN^4IySH?-RtUGfu!Vre$noLHtK95-`tr}hLf^1e_%|40b>f|Th>99~0$_41A zjP9-}E-4Op#EcQ7uJrQ$@dC!M-xa7~$miWJwn;Nsh88Gs@dz7nBkOy;cRT~Xsh%hp-!_|J9jDT#NL_xcRj?pbCVDD#sZhow=vST%MDY{qWen z#2)y8SR?g0LgRUp-!F4gNtrCem<{wkZy>(-eTW=5FK1fnh70vPPDp_MJ+|*BqVqq# zWs>w(qcm&y*NCxcBtBq0V}^f|MypSAl>*AwZcU!n2i@vNQ4-}95xva~&LCsl!HmU?CsKoI z^%KJoQO-#75;F_@n1s_*pDQo<0%O1w%fl?Tzwff(mkp(KlUYMq;PyG7`CJ(3YNj-# zgJpL8eDLD`X8}ZA!`mG;W2&K+N6?`Jnj$qWEsf@GMR>=0^(X#Ht>QH+uM`yXb^3!M0*Gz zv8YofA5i9=Uh^aI$gzHU!t2$NkRq6ZFmmHVmmJH#C8Zl_Md$T?F404NU|G9E)@$;y z4mZnr+LsRytK0O_g1k{*U7C?`He*S|z@1*p-4%N{O0q0`uTr%yvm#)avn*56w=j|g}Qg5^~1Wx|5x2FRv^AcLCH z#rKA$BykM3$SN4e!Sd@Sdc*%_a2MJVvwT|b(ETylCwdD5_aK3Y6K_tUBRD0^@<$x) z#97-AShf+3EHNQw!RzUXhEjB3a(i(86|^_$S{QBKK{!kbXLoJC0hCmdJeD2@EEb{+ z-pk^+P&E??@UOR7#HYzfMzzuaTrnFkXhAb@2A6nV>|8#=#r?Fm@<1@GkS;C}c_AAhM5mn?C*)TvL#&0?6>|fvB6e zQ-N=M6c%wW1F|l#hf+vvP`eS=6T9GOL4HqC-#=IVuo9CtP@A161*6r>zj^c9N%ZrS z5%7pTGeq$udI0f$wXA~>iBjKvB7zC~%ZZQ`b-BFD1PRgzFYQoU{d%AN$ zI&m&PnIm%%I}&RDjBBegsM%D;k%BjQkbJ7LSIu;n z#`XVEXa>KM9pZ9YcZ#F_>mL+OELe}r`s)TBMc`-9En>mH_m^; zeV*UfjSpAWZ%{mw-bbE$N5F(M`&hOo>U}Y8990`1{oIdHp>fbVDN>S!uCI1Gc*OsW z9D88k5gOEdT9LYKSV=LAJ|mWlkmmtG1IX8Q2PM9IH2t`x7%^(;C=$G5B_0+JIPq(VmZib~O7?JF>lWYFO#w z?^nru(SG|>M_MJoP|PLyfq`v%pwUHqg{pfDnSkhpH$9GKwuzdQMyc%RqDuvPVgvPHFB3aa5JttFfvmsIIk%RIyclc4(`~>X7NW)d>Pdqs+Un2 zVG1zfOxd4YivhOu6nLI|%8?otp6$KRd_nt$fz0+KPkJH#>Ps^j_@_Sd6md12lr>ig zMjt%ipcWDr!`0SuRC{TerZVCdrYKRJ(lW2v6V+w5QRg2VrL@;?L(~6#GFoS7_xmEs zzN4wUb`uJ#bl?}=PAe7^we_t9)sLP{@}iNrNYvF8G{DJdj;=D`Qd#s&r6xc;_r1L2 z1Ts1p9OonzERZ0;9%_&iy~r7RwHKayg=!f|TLxdoA%Yi@0Q>BDz@-sB=Oj0ayi`$o zUlitgY&ZXrIs=@)@_MXkHcGhs`kU5Uj%*w2@j=~vWyFotrrC)nat9+h^AjXU3w*oD zD;Z>(cJq)Eu0jfAxj-2&!hLTNL=5G7o!C^#T%RZ1a=DmwUMEv?eB080jbm~5WCFm( z&64)SEQbJz@YB7)Ywt0D>VI=r{{~RRGhyIJwR@+B24UO&E~0Z}U;gBS&P5A_ZRi+q zy`Pxxe+58;2T+4wIy*PKywDYF=UwJ!ZAr+~8=AV^SmCg*PFCSW(E>Y@nFM5A zal+e>07)@}dE1-YoKwZW)S4D-fcXLFU>J*A+|SjNn^;){Qs(!FaJwW-a&fpSFSd39 zfjp17fS=J3rNxx7*0w|OnV$VjrQDIGm1%pj(1<~0_W3dv@~A;I3pa^Wy=b-P0Y9?l zoYuGM?vS5~Gys|x7#Z@sOe$|cahvsl&} z3oZ@mqK?zaGJASq@bz71b1T+rnw;{qltXdmGNcClbf6v|@XeWuSCYaNK4AfOr2Ov* z@vzS9JyhkJdJ`Fel(AlE;F#7kmw(McXSxlel2n;4vpvx7A{R^r%UX{T*3bE|mL#A+ zAy-4SD*(sX3@*k5=fn7(i!>?z8@(P<$JHgSaLIFvd)TXH7_QJEu@mtt1&N(LlIfW3 zYgI+EfZu@6->*dHH0a)Oa~+08$jw4Fc$?3QU&md!V#D65snq=rNB%#BZzWw_-5f~? z&eyFbxlYY7$S#cm3UH*tcC*omH|@3EecpZQ+69buDjuL7iFsL4=2YLiGzbbQ8rTDhM?iZj^uIifl+ z@lu%^J2Ov*kO_3_hf+GvPiUf{loZT)_<;HdVcqD=9_+~1F@2?W=KiBEsbfF#QxrZ$ zw*0)vAw_QePgq&2y^P~e5S;0Cy8Al*Kqd!#!j=ZUltx)iEixj{KA(WFF+p!FQLy)L=bq9=1AF(PIWlim0*t z<~Cma{RCBc<26;4T=wzAr8xn=y7eQvb%G&)n?WSp3gu!Q^ChwJJGAQpB^uPZW$Z>I*r5j-5 z@h^Pv;Tb`*qetMDP__l6vrm*HJuYZbp+yaQ&QSdERZ~=DA|DH}3>)%;OZf&=4Ch*~ z8}Knqp|v%fCF+scUmu)xL4h*;-xmj;Wrcm3XdnxLf3+@0JJJrWoC<*wfHA=Q_*e@MJ=lH}qeX>p81SC* z^cf;~_knbl9|=N#&Z1zOM(d38Nt7Qv8um5xUHxYF4v~;mWUcx=pO?u|y0k2yXLj=9 zL?}g-o87=9O;~=$k1e*^2k4Nz}%EO9;L@cYHeL2a3DoRit5gvhC61$0Z3YP~Z zm`}7%-@hJzm%F=eiO_QNhKVZIFI!fl22koMxR?c{i4}5k)b<1;ORnzjCGLb&i?#i) zWBPfZ23I=yBWdbYAG{ZGb%I#_o|LEAa&td8j)|!z#LWUm>}vgjo6Irs^<+;td1 zo3rUxgZDT!k5vT_-!CsOt08`cTI{7fr>bERS4FSGy^`Lh=sqI@muk1NlpKU+Xx$ew zW}7FAG4L;v#c6+>^wGF(par*}6%=bPY7_GG){!u1A2#95s*m*!5fDJ%4 zBi}qg^_6A)OYC|_xIZX57RM%9nzI*wbfbV7P>^XkL& zSIA?MFst>zwuW8uf3R6NEjW_}5TILl3};Ph8uj>F3v6Ti%+6f;h9+8o>AtsM=f$$1 zuLCMUH^S~_I!QV3De|!kUMdV2zf+g{1O=uQh1V8l!$H@dV(M3-zeQX*&E3Ex3>p}F z2xj8`T2U0NP~+5OyqF}cpQIt06tEx?&=ZUUNY%Z#k26A%qa;LcDZ?vVP?II^g$vgE zYv9Hd2(}PO)LOdIcvcduD)&Jo!S>F5{EVDQ88Y^2ojjd58U){}{&_UBe9i^F{=X;j zpYhWrdOYtzA+(;JWIjFp8|zuJ1b0DHz+cQi@*~v{D0nmW+5zwR945|3QZ3u03NNGh z9V7Pr-AAy3Yz){Xc=<}IJrVm0D~}Zk4y0Ca2_W3~+O?3p&n${bV+!u(3S1cr$~Lhl zJSvHiBz^ld?uIde(!%h~CfWe5X$yG7c<`P?djq!l>rx#Fm`zo@Iw{9ZfC!NwxL>M- z3D3cc-=-(gx--LT@6Rmcy?#bB%^&1%pAGK4Pm&s$oJsGp06%xI+BwutDMyCb@UtFQ zzxH_N_K}AFHe$`=M4*0+CQ^QteCQl=F$aQYA6?DDC%SLuu!^DoX&hhuk+=q>2>_8I zn3=qOZR5`37|vWLX3ma6)~u^&+D>oL^*hUjH?i@v&b0K6&M1gGj)=30I{b35o#C@& zah87^mQdRd#N)L^{9kIRg)- zA}*iyLj}*3Q)%Ou=YGy7y(!`{%?;VQVRf%wilFkd0A!wzEg#H2KVnEKyIPPVD5;dl z4(ER=N zd%&j+UL=qdo&x{#{i%LJ)9>p%MUd04t^6E)ybkq{L>Hpe-i)oG%d-k79sl*JK%ZW) zy0trYgMaD|sRTjCg!>T_=BN;}Yu;n(X|0oRJFkR^fK2y@KIfP}AhGCvez^WOlBzEwi^vGLsGf5b2>V3oyjk~M- zC()I@U-v=)HVnz#&`6Y8#^PhK)l;^W@I)jliz@K8`GO_yW1Z(f!p4WF_$0F&eM(7x zrl0a8QS#mO;(TfQo7({hupO96fO%X%44%0*&--K9G>*-7VC54rc$d23G$+ObqCc8J zMLE+VkXj?=NVR9VZCpJDFva*jQ<)z?<$h0MEk&iDI3eO@yx&%e7-*T|{hIoYuX#s& zv2B0(j#8F^m)n1e&Wv+=e9z}Uoi%O=g6Yd<#+I&TpSOF4eHUv9dlK7E=Dc>D+Faj0 zA*v?&N+%`5D2=81)_z89d|ObHk)DtxH*KekJ*tu(;9eh(Xu{HuINhcd#j@L104@8* zO?gq-r-!mI_f#aA$)y%WA9XP~L0f*s1GulcKip500Y_A$8-^h+lhwz~n<~gpn3T%f z`(Y)(3bg3T|1&Aob?|_@po=ApX}8KwMP_m1XvBbjD2(WW=88OKNuKl+rcLMieVUSY z3!Z^k=Z7#o-DTJvPP)ap+dXj$FO|wa@?}RIJMB?@X^h+CJuWk>O1X&As8mW+EM?n- zFOcvL76eiJVS2Lkmc`v^>yz<6S7RzVSB*YPspL&%S8~(jk_^Vbt!MVPmZGV%r^_fZ zP~%P%#j!T!dILqiV~;=eZt%Ck)k~fae@5l*qrPtSK&{x$lR zxQ#t~{o=!|Ao>`%g#U?ftjWqCvcMe(iMleT3TRDcT>BI1iC7)P4tcY>^I}fet1-jM z?{s!nhrt>dr0}4=m&=3e!m%(MVPx%>D6t*iSyA08c%oA^e%&h{e$Fz(Y%Zu5CzlWX z8FR(Z(NC*8%b`Pw_#@;LESCdM4X}YJ^^sx{kD=9{gO8=o)mOZGSA^+gvm8qfIUxTj zEAE@cf(7#p<@!Haz$y16aQ&FQ>0GdYp{l+jDa(xod5-h0KDy1&QBNj&uU2FxZ=7a% z#O1M(XQ~jmo_EHGCBOW87@6qac72(cP=}rJi^JtiJp<&8u1AL8A`^_cBE6RS0%_l^ z=2fo*`$uc4^$-`iaI(u+{NBtxxhG68`W%GDnmaS<-Ks8i-Ql*B{a3Ei1aI9};7^M8 zn5InT`vVqV?G$vsT<(G}=7gtsa}#4dsvfyw3p3z_Zk_tMbw-3CIF{w1cQa~a#@;+D z_94G-8RH9PIr8{8?6>^9Tc|Cr&RA;j%lnKV zJ#vNU?)#BVza(#|eb_bic+%e-)cYHFGIN?qvbUqOVcxW#WS)S@WNs_DUpD&M7+#Nj z4A#GvCd0nVr%D6ig*iBf&A-Fx{@GH2g+A#q!h5K_kv=*|ez~kWJY)D|>su6^cbqlq z>5&cbeUAFykkY7^kI($Q+hBtpySL%tN%s!YZq=Rr`ESV zTvp9j=vV*_zTQH6 z?i`pY6BnkBW7tNVF``Khgp9v@FpN!9FpSpaYzqM*v!6ZHcJJxm#$w}>kU3#*ecI;W zVyd8ilE|`lxZW&3Tut0G4*Ows%2E

    ~9h|^)&ce%Q=$=dmCD8Ev;z)IzNLV_g7zQ zfB$J8$#a2WaDPNmk@ho$5IhO{(O^5>{ir;|aTJp?e1z*_N4CD7kSVK1p76;C|EPHt z&Lcr9?z?Z3O^13FC=V;Yad-NMOEdF3@%kj>@uO(he1!Kkg*IY`9${gmo%ZFoId?ysye%1hBKb(IsL*w=>MplLz4OJS3nz{K$wL$^9jb|6 zHC%2WPv4^YLFYrNL?)%WogK@7;KrnYccZsB20CcEj#=x`G&KC6Yjfgtw?OUQI0*Lr zXMoO}Dh`z{F5D60Dk==_<28omQ;f#)d9#pEvMQ-$GR7;bUbq=yg!-5PjCe3~% z88jZK71d+g|9t|6QOg#8szYbip&L#$G^)bKcAz1lNeF3iPs{8!KPe&;$b8UJ^&z)x z)#;Tqj?eFP!5PMXGuo=gQmVsFFCXp)jCGRc7@|sI#8Y$quivZo$kk5^yq zLOzrpCB?F1TxvpoUfre}E7)nqHgq};Jd`;%k8ydF0Ob>lU2-Sr*LrXfdyW3Yf^$MIHo{rJ9?yvmn6bXNF~t*ZE!zj4w0LgjK=?G{Ja(w)1ab&+ z`KJf3K_nowk=3bJ5B_d(cKVVQ!QhW#t}N1ty|9K@CNP(Z7G_k@820>^9!Di9)9C7A z`T`}W&{kU1@>$aM7JZMCyYJVejUug6np?I0tQfhu;vgstp$CmiQ*kyCC&I*?{3iV3PoKqu8B-M0g&2Y60qA@^<8R#B(rC>z> z-n#vK#_Yk}&$pXcgx+Iy;@L48c=sA@fII% zbB`sCQ{#9AI8E5+Gn)E@S8^w158iEp8mX|e5S&amgkqCdJ0Fc`;Dg_+OOV$HcNesv z;bJ|$5F@qQ&YTW$6N;m6ER~_$DR=H$Y!_k;_c|G5a_T=47tI6^cAqvLLYGRuXs*`+ zwHLncdkE(J8FKL0rnti)FSn&Wxie*{;5y}1(D=hZ`Sd#(w%UZ~SK_#M+)+PdI;?%y zHz3}l5?F|10`1GQMiJ?VpS2`jc=8P!0ZVWWvbEu%Lu4Z&>vzQ_^0*K)eZLNE7%dx2}CefMZ8q{SI@J=Ascf-ay!ra|I#1=eR$u; z0Z|$Uen0-+qaU8cjI$o-lZ}DVU(ZrWt|Xf4^J|I(_w<+K5m`8q2J7H=mzbz5@QGDW zV9%K@`~1ozHyPKI4p&jrmbUFD?v|=diV`&|8BDs!w8hzM)4bDpOF`ZD;GZ2Q+plVp zkVV&h+fBD8FeZXtvlSqdpA71*BCDQ-LrM34u?oMYTxi_w@l`BQoenLTr%~h+=-9+j zbcC(fIDAXNx=51s<#Ex2BVO2Z56gwv(9H%d;dI0ju^%(223Rf9$;n&t*X_PN7HlJn#&dx5|q2OMfrgElk=nYs9>hH8u(=a$t z1UENY>@Yk*w+Y3NI}8A!ew^@%y!DDob@4{3bD2l|s;=F#CsD<_5W7LYr_RNi43Xz;{Hw39@4PW}U5xe8G?!n$( zgE%0}PUv~U8|KS?%&at5ZP{Y0E*(!$$KD!+@h>1B1%f2z*0Ikt{?vRzFJSBEiBU*v zs>PywoF_Z{pR}+i&W?BociF9nga{xnYbCcKZ$w zaf&wgkvVd-n=csn5L9@}x}lPfv6d}&V8>QuAS$Um8#MJw*{1)|wSZ{oM z`Aw)9TQJWkOq=flV!)*Ing2LvMFV3pA4G>* zY0{Ka!%KS)RJJ(sBB{Z@X*MBdE?FR^oB9Kiqt_|r49ze?3>ya53_HJMy$ct<6}vV? zXLMq49fEdS%HfdR1gvb$QbLs)lC9Gf(5S@58P3XM2*LTO*T+Ap8kSc&R$IJ78eZ<; zCiy!}BQkV6A&LXWFjFI>JD)nUQWmpPax;Cd)?IbwyMrz}Z*c!EUJ*#$nri>w(G9gu zLAz=Ibh;?1EF9r9`VS;f2ZE%6{Vs)R`h}x6sS|(yf0OzKHIsr<#hM82A@pn5ToAa;fJ2I;vu`T3JmG#g_Wr5gOHBG+!J4xd;r7(WE1j zNJK;ujN^56Kw1K_`2uPo-Q2c~bLdX(O0u73i|@+Q5afpW;Y>xMeXCayVo z@!1sZ4*)`(%%L;5g*{Pt=Jyc5@;j*Ps=Z}vV*c}Z+ItPb#ygE>`LUI1KxNadyaJkv zt6by2@*SQZ1#g%vC-q!kD0X=q|Gock_LmZdtl-Q0lK-RWI^)^=-|nx~(xTL^z4vNS zsz$1+RyFo4q4uh(RYX!0tx^*6?{~d0jqfJrr($>>BvJ6a)TV=0}6?&mrH4jQ{7_vJ37kHWhoU>ErRJ z4^^sdTQv7bb-Q>*yK9tK4iRI7l3X2GQV~q*YxjJkob{* z!dRAwVdS;dSU4tZs|xRPcf+l#j^T>JsYGO*s=*MAyI8jB=N?Qduq$D^I6_Xs?&jsy7;NTo&1EwMM_t)yFuzaJ7)c^_g;fe%B3v|{ zQbh3`I>#bmyhCxuLLbct{*C=r#SeffRYh;XFPSd=G9zREeY@?n&Gl*T`=F7uLNDqR zx1dPuQJnBv0M8`C57(u0Izh@WmcC$1f9hvH4qDnm9a?(5d+RfL{0bLRuGs*t04{OD zYG(tgC|n?68%45t<~z$KjSSrVFCLvoUq*YeuuZoQt93r?79uVU-&tq=RoD^`=Y`cW zK7O&;9s7!V^FH<6^s@lsJBOt0+p}{jwcTu8s5bklCXQ6eH0f?Vo&@oYNO0yIP!q&_ z3_BAZby2nSF7;|TM<)D%K+Dc<7R8rf>3m8fj$kLCG8HqZVS46T-39J%;8vPGGVfqwOamRh-!WqxW30y)#ML}d z8<-)|4Z|?Q)r{Jurg7669a>IbKLMcN<3An?-nrX{A?M^Gip>B0cxCWP@Y=WXwX(Y` zd+K6*YFR0T#;^}k|KWuVnXkd4wbr_g3y1hpt8q7(y)M6&X{|9O7;5)2-fp39;k#z6 zYW|X%!(3i_C8zyV`FQ7O+P!Lc=_*Hk?Y~#2Iqfq!Dy^FFuq)1d&5O2*qLzlxtw#_+ zvrCHaE8MjvKi=lMNKqOPM%`zR^6u<`NY`H1{h3z+LLO6oYV$1;t+dt zi`N&ilfRkY(cPW5Tslz|q9-{&C{uE0gO+Yv2-!>2J9l%J7A`WcX6}ZiIr$lM7n(=;ii} z|0ay<*Xsv;``4X&jxnGpv_doTV%@o~AdB0(oUX2T4CP*|iI_Rv9M^+X^AqtZRL z%C(OZKo;fNENl7rm3+h2O$a@ZS3DsV z3f=|HShyk!CygQ(*5im=V8!g)5+?ADU@0zoO_!B&X^dO zQ8;J(WANgcXD&!^nc5o2p>36DO@ap!MeLH0R0xM;qXYfv&f-F0113J5M89s-LtK}q zl4^d7AmZIYjWUFRA z>t8>}{`9ltmE~~4|Ni+1tN?#qkd01U@Ros%Cwkjv_}g9G4_t=tOVPZP<@_C7&I!3# zUB`vllOgngcqqMi$OBb>DFwXrFj+J{hz|VdZgoj*UBLy$m4D%kVs6 zSm(^ND8YoEeb;IeNSX6;*~8zuPYxPJGJ!675H}XDnZCb3Nhf(a1AK)Sy{beO_{co2 zdagNWLTVYABHAuJ$i4pw--Y^xWnUToK5?~=eKi4kMGx;f4lxZezzh5}O0QX-w0!CB z`10B>x|ZR|C_%YyUsDI8s+KHpnfh>1J(A=r3$f|^HZ78BEclkm@^;a7Dl^CPtVmPQ zuO?U5$A6C0sKatQ{C)TSmc8;uHDqp}{@z`h^j@FB@m9^7(Dt$2$HdJ&Qb1_wTD#JT zjb5g|jsWkzFr9fX-u^G%6ziMiYSj95nlJUu%|V4-5d9!%J^rj}xZ3ch*~Ufshxgyt z*I{CR%8)_+4-e|fSK@=zP0F=2pi{>dj5Isa5@PFokC==fXR^a%pZ7+3^dcI$5@j5m z#JdXj5|CZo5c!TT@qa(p<9#sT<8%>+W(Kl)4|6My7>UZI#1S(&;gEzZORtpUnfX!m-oFKabE|YisPBia~ zgYYlbm7f-d6PVdx{?Iao@?kf?7HKPs1)Ub#`4t~F(fQ>f>O$@yKbjVjIl!7?kpY#2 zy=!OD=u1ugON(%=ZL=ZS1fQm`Eh}K_Kro?uDi6nAC#(t4cnOWH(LU>#$9@JJ z_#E-K!!y;<2lWi%#}+ybl0+0NhISswzP3HQaD1{Sw|k#RbLGZiU-p9;vA??m>1&V& z4Q04cDiJ*q5b8b}924L%a$GPW1n0w=_ASfv`liNKidIJ-IoWIOK1Znxp7Mvwn#=Hn zRwt5i>K5y;Vf?Oa#xJZ{`G1+>W_C;&T)$v~;taa7@XzE_E4!&ahLnbeQXRt$xsQ%{ zobvnxE@P#FW``UrYbF!&fP(gWv}Y32=+A6fmjjFbLeQ|8fw!!L*^T3VVN2^fa34*0 z-Zo%^PlQ~O+!;QiB#@VG4BV@)t~GZ9Wq4MM?)puT!Wl9aKY^Sw^5{;Bz-s_Dx49p$ zm0^uqFD$k<4`=ZPTfS?o;+*&AQjAC|U%GavhZN{tXoRW|`%jJ7;JHUnsUavAIeXA|_PyyfmgILRe6D1Rd z*`$I@S_KSAr9(J1>q5m2Ss#AWRygI+^R3Lk@T|@lfv)c)YSbl$JLCJFMnI1+ntW}t z?;%STuh47%^XeAb_2V~CCs_9(+nE0(XwVD%7CODR zz9K`TumM|FB}Jd!4dydBJe0u$BaL~~U zd`tFWz&YsJ>U|>RuN{d0Qv3O^lG2J4bo6H6Lhr{xxKL1;e{OA^-N)v}kJknbxJlRT{l0SdGBimNm+5ek9ap%~onkrZy)C!v;p^}a7#@;^AN zXfw^oy&8Wa-?&$g3edvTvsUNxfDlV(YOhDe9`DFit8!ID4$FDtB z5-T`&0&PE~!@Ya6L#t|$?RBZyPM=5IaejHTY(s+k;oejcbolLKakmvCOgM8o#Vg~u z=rxF2I-d8+7oOw|2L!3i{bR;q79wz_ZP9@d|yRVw!xtstWN9vMR{R2_K1U<()Q%uT` zCEi2NTTQTkykhQ$$~L=oY2*5y`kjYT4a+AC59?AV+|QYQ2EJS6eoTx*?7H1aPvbgZ zX|B&712Pe*(3<5j1UHh<2mEI8z$g*bkOC&!w$7XI`(6y#80UebE|?jy4u5(NLMf=n zmHzV{oOO+L<1D?+W{`W}(c_?nwa8;jiq@9CFF-*QHM!PNP8krmd7vd6aSFn+i}%g%>L|*lU)sD3a)N__sKrtrPsEIBA;#P zgsvRIhLYy>f2F#TJt83lEpL3dW}+7ib+VirfoC}4i2j{zs^2P(WByTNb>kjS8u1H9 z+Y3&kO(}5-`dLLe4K90LT7xea1u`f1Hf!t^vOcFwj2Sy(K+%rQg3He{#y#FBTDQuo z?jZcRMlV@PwEjczE{CLxWf7w(RRw(pj#=5tsBmQhBv)ZL%yIbT8p*Ac5X?%@5GS;L z1}XqkoP3`AONzb-q7{z+$2go9F0LB-R+uP=P>uj%QP%;H=59otTWucvTTOhKMFa>F zI`zI_9a8ok{?o+63DtWv%y*Ob~dFbnbNnVm#*;H?2gWD3&~Mvtk8uP ze~Tqvel4m`VW>RXf|+=Ym`tr|b_Uub0q@zX^n4nxMGPbI4nEYg3l&gG2pW{2@`=GE z-k}EJq<}$nsm~zANrLkoD+c)a3K{Ci>1sM*{x#&FZj{jakoe$KNd++3z75YErvdb_ zUSsi=`L5cO7i5uQ*GqIVF)C84)q}0(>qH3m%{(?J8^aJ9WUZ0_ zgQ&VRpSeQO&VB>R%{okGvpRfw(3B9|ElFEysn)(%Mu)q_Yw>>fKjVi{C1M=~8L4`L zd-gf3goSLUjv|v;gt|`jT!NSQLCRMnTm%q-(U@%gnjd=v|1@-)Hs4WBqrJ?mfzQ%h zat>0E`akHAlTlhTRbDPI>&;`_jAP2TPB zL+d;m&(wq6a6;8O*{k$ba8PN89FH*2Rt%2;jsvf5vOP%%jT=gOi;W76C!a&ZS>llD zz>n$-19r$9oZ@hHSx@fytYDZp@P3r*o-D4dFL@h{=%L2y%nu`hh z?I?7>7Wnz02GsE7yV&DFzT=N4cR^-6KD}s`lji%TRTH*<=5~adeD1wc05UIzZ#d+Z z1wik?lg2Y|uowTURO$Gcz2=-UslwR@(d!+{>X&XkD&~ic`HG`ezl1pnA4yb2brq~; z{dF_tvBYa}MX|1vF^-2V<8f+1xqG({jR&iZY72#yt@MoKEJe6&(lT_M;b~ry4TX%a zb9It4>>|0>96%QV0wxU4L8CLi`_V@abZp$SnDjQ?Z{OPOc1Pq7m^tmzsveA7Qkj0V zc@BTxp4ZyThy1o6jmW9Pv%(`pMAz^x)^$1qHiO0fG9G&njAD$)rGkv1L;^K#2ZHg` zD5H<9AIi5mlRyv!QDLBL3=39zBmJZj(}eLcB6b!tLI%7l=evL@DWbzdJUINtt~j)8 z&+VyTIGFi!;a^8Ag5=nq%A z$0pE;FbrrX{MmazbfV1k(}Ucw$M1d^;f^l zI){QxIQy2nmr>21eg~X+vqGgdwyl+jKhB^3q5yra9TOfmx=IG~zRco})!^^~PrbazGcej_864;&_Rx4RwNvp}NPIGN!)CYDe%S1Dhna%yJdcYOm>VTuyK>CeZ9B<258fI{L z=v!zRI(^ldhI)j>PJaWBD7yWSRN&K2qr5F$m{9TbtR6Djn-x3tBthZV3WJA zs1gk5Bk`=`Ag;8m>B0o8&-__l|Pmi?wcLbJVC2D_q&5TUKb{`AqmImJ#le_G*ZcXEX<0OXKe9$&H=DmwYRs<_8~sFd3W` zn)9rXwl|2&1V|_QNDUo|d^*7oOiW_YmTN-D-x`lO?0nKRa`Hr$>63whx^ujZ(G;d~y~1xO)=7OarJFKT-{33i%*U zGY>Q!un&jLT9Q1{?n@U3$_kg8wB>|rycUA}mMS?f`mF9e@}3 zvJToPz7r<EslZO-+>!fc4In)&sQ8I7+h|snK(Kz^HrAON)ianM5ilDN;&7M z8(R$V_+FS%v?*NuQ49@+Ro`*0jpdeTy#xRGn@c$s$&R+G4cc<<%d2V8YL5f%Xm=c~ z{cnZ3uhBqaFJnP1qxVYM1Sic>XW9&?gwIGwy!6%6>V8BNnnkM*YmWVoj&X?z$prYJ zy)kDdJzg(E*a3Sk+&3sudV}BY98ibRADiP}1whz$_b3Cei#Jj43)fW{?DO<1$ghJF z?1(1VZ@}S->?zW=HqM^vQSb(Bf@r8}+N5rA+aY~OWks$}@nyP^kUub4PI_Fcle`#d z5OEByhU(Si0H)yf_g7o&}N1e_;1w&+5 z#8M3uoOapy=C z&2{@O>Oe|Bobw`bU8!gL9J~C!fIqigG?qf6POJ|C;H1yjxwm-!8af3Ba-Ny7I z<#OHZSfG(K%sZG#2B8{@n;~y}7{>ewx_zC9CA}@i+5{a-v0Z52erAf5aK&dID#B;I zfp0pE#kZ2lXC7?NwLF32$Tm~FV4HJD%7tzq55#JyAlTD?^%~XoXfQ-q|CLan!hzn{ z+eDG|yhu z<`C+~xMIXrwWdJ0Wp%Gz8dh>t0;SVvF_yt0o~e>XhcD>%yBX+f8S3~~M*}bKa2Gt? z2Cq(L7&!|eF|wkx(gaM+a+K2AB9^VxV)ScO?FCVx}oww>DQo`s55Rt zvA-iya{6UCCF0pC-%qxRL9>@df?DN;lzL#o-pqK?SVH1diXbR+8M-N0)%}fG%z_MF zm2gGa5|{3m<5c8>C=A4zzp6lQn>X0ew4|*MkzcR3)XELtU(T=K3X7E9kd^-n>}YyE zdD!*JNuBez1XK=lhd{>XSXRiUoUK#dG4S^LiMEv%?X%X+(N}9T`Mw|+r+=zg$A-o9Ne@@j60Sp*aeeC;?oA-x!NiBcJ`MkY|uje z2c@Bp7UaMkyzo3wP3R+X{i&S*MNzu7*Nh>$o%4`)XjQY%V-k1~rhKu#bR=+r7kiY5>yt*J zW&|sY=kq<$zX?X95^@-k7kGHkC|w}AxqknkugLB&MMvi)VEWp~MkU~q;zv5v&(Q3T zvCAjPcS0mI5U>fSv_l7`FGycT3Gez|nBpO|a_2YIWh9b(HsR6syf5uLmR4<%EmP<} z>9C9Ckc-WC!q;TwBUsKl4G`S>Gk5yn*Tr^}+C9uX);w;%(HXeU z_bDnUs4=C-as34bG`WH<=ferSSfR%M_X3c29xx-%o=3J{^pW0tXVE8g;jJ}r zu4>Qg5mWkb4iJ;{_@mC~+`Jz>R&z zkDGSgYn48yQ|CQShk}Oa(=)@nsieO*+SDJlbxRV4LD#7)VYQuQLGNTt@FR<#{olt> z(7s5X#4wNpq164y74 zyjpW+R)R`sceuIi`d)nn1cqL7Hux|8wc318yiL=>`+c)IdV3G~EnG?kUN|`R!uscx?3USL6>1GHM|g?eEFS_^NdG zUDMI_kIL6OJy?3NZzFOSo>c5jOT&lfkM()!Wr)HH%`)MQnwl^*_~a&1avKTKnBv z{9qHxhAV@YlW&+Ynd~nj(=^I0>FG4CzKl$di{r!&UB#kHszAMybv-IuC0p|t!=Jm5 z(IU}Le?^g4dyOsaa7K%#y72h3nj)cLLse(P83{K!SQ#}OJsuHX?D6TH4(tzq2lMGa z9k_0>bqrDOi%^)vhn;&XT-g}^L9>>!2CV(K*f#6G3uwv{cvSic;R=ddmmM~C_seJeHs#W2UI)uECXXt2^)qhz>|MX|)GO7Q z^qyhBg(Fl14fhWONdD?M9`b3qMJY-q@N@_kb!ld$U5a7cy%r?MdC>F#O@hNqum_2L z06%yZxBNo-^A8xQws^;2OipZ(e_`Cvy?KR>EB0AP;B8x(4l*X@O0%{;FzhEf2LZOB zC9PyaS5W#W>SUu00f&;V=|C6!_SrGv_Shg*2(;E_?Mkl&i$+r*g2;>WhrcQPgl~QG zA7)Lux=-w0jpgL2oU}fHudJ80!tsB9!_x^e{3C_+^2B!(-fZ9sHIbc3d`+bH!rWch z<1krYo$kAzRL-v*+$TnN^THxk`Xqyhj!-JEmDnY1sh33Olc?OB1iOj7d%_koE)^;N znaHnIyy3D2PSf*Bhz!T2#!|kgd#!Cp`WK)0m1SjmU9;ZrjiQ&NkN-AT!!N*3UxU-F z^;u1J4C2R*;T!hqaFl#ltHNcvhY0<7(uIMz3%oG$lmKfyG)uI;t<;7(SuWR!=WY!Ax4&CHa17app#wa!BIgLCp0y4;nThwaEL7XKBLV+ zBl|rwk(`G*74(1a+nWiE4t4s3@7%m;Rc;XdTEe6JQ~v$0o2$jDXDi7Fncl!doN#UXTzR{bPIdvhj~=#EK!<*cKs!I;`` z$13~0kPx6Gh;H$%~ zh`-&9C{Tg$?lcBOaYb;gg#@k&{1%TLp$!xeqbt;9Pk1gKWuDWkB}SCfhC{eI`J4QJ z1jeK{{IKWWe>4q)Vv&5ZCxO*pNDqO65Q#`E6q-mqaxf^~`6AXlu#=#8b{iD9?UG@r z=9A5RO&5oHHBafPafl@)3$`D_KRY@?SUSJg!bn#z=h5n56(1ls8zBTH8`9GC|RQkW@9cA<& z6L@?Dd09s?txpr^7v4rmiZ06?cE{6utmjcAmgYsqa74ts~4&rKKv!e(8o@SZ{QhhN9NGU)eKl zx1D_(6)pV~huvBgM&$7spDFlzeu;Cp^@VC-II^>}uD_dhNvkh$Y`I?6Fuq=(ANx}W zR^<_X{-|&u2_Z0SszL6;%6RtRP35iNjnc}EVtTU2RL|c{H+L@cslf-F`y1=|4jKjX zj2-#N{H0egXI8~@gtnY-_G+fTS}$C~$`Kb_2)H#3yxII(0k6w=^ckDX82B1eb+EOrdSD_*r>ENot%Fzlb-Sm=Vy;uzZM-zM6} z08o63G^x@va5bP+sF)+sOF40<8)NTiNE_6%lExZ@5pgnO}SS6I8ynf0vq{qXD-O zFUhJRw0%O$lzCEGvhWFEq|Sez{6T!YU4u;)r%7R%h<)NOtb5cPe>OJRF!9x6b7I|k z^GtM2D>Fhw#Omyr1EBIfDISOyg%MYhQL6ZVb&DCRl}FCPtgr?At&nI$-*J`IT_p?+ zzhO4O;q8le{@c(cJMC0N5-KzpzQSt~!;xpAN8#mIyA?9q>vy3~A6R&PSNxN9Lt=NV z7e!*h#58VWA*y;NB*s>-526U4?MI!BeZhQ)`_S9xTetTM+e7(4!zj5$N!`^_d*b22 znfYE;4&kdH1+$mZQ7XQs6=U?G+rpHU3P-DQCi)UTovupDU7fr9+o=sbG?bKl%+C0) z`{nL_kYL(q5GgJ(QHi%CtxO@VY)7*Rv!1;OIC)o;>} zZd(b}5#{#N`(}eL4F`pfi--cIeu5A!6ZcZkHx1ZE(a6(|UVa3JvGW;cngvk`dOjfi z*3)VhscSmf@?AtyQT_~^KZ6z>qTm)tACo1P-qd!-HXGUvd#pf+G0P#z*|%A39_hCm zC*i+>zur(*q)3hv5&wu(bD4VRg)in?l0Mr*V%rruE|O*v63KS>mTdx8#^Y778?vI) ze>huUHB72y3rt)!@tjugG_E8=)nRnfsTH>Kq-RW?Zj>X=-JisJggm8GoY#`V9SCQO z+83ujo;zM5s?2!}@LMs*^Sv+z;M8uNvCEe>iN(aXe%F(9lm0_lw z?4B}w`^Aee?%>toWx|LTx+?DHjqYi`{s3!6m1MLv_btO%x3y^CU(T&6eDTHGN$sE) zi9#$BOC*@i4M&SBzQHO$WaHw00SV!kb9a@mfbmNE{>MUqD_yTH`S~Nws4{E8zzUb7 zl%p;?eD2BZR}r)WA4s&{j9Rz&_csTBU9UzxpJ0SBEkn;Gf@IO#rPfaP+xa;oU$GmE z=wc~mSkaSzV>V}A+v*?v8fO0?uJQCXRFVbv>7X3^6(Mwm<*9k{WQ}bd?tl?=~yZzu>C)DcLIDxBXf!Mvth89;?mdE#~E_->BVJ<6jw|&Pne+}HlGsZ zf;tf&kb~Tvs&N4lgQMCSRh+k{|4oZ7ipB8rug02yLRiZuv;tg&K#nV1g^XrB-!c>* z<{l`!gMa9MOcANHG@aLPkV5(6jcz+W1i7sT zlqDiY#^r^l3ccJm9$?Odo)k6v;o8ee*dDZltxk1RNgWR3KVVkfV8o?=|8p7wjpjhr zUTOe&UoEU9s}~K#Z=tZCvUlX%0-uF@a19=8sn8zGF)Uz39X&M{DE;eLfJy0Sox`)i z`Mu>GjjpA?1G|@Io8w(vRYXa_ZK9;SKI_6SFo<;L7&ls>S9168A`7f_qbX#XHCvnm zdavYag#T>Z5dC%(MMb4qv9FDB*qSNKC!us1)~TE@%GIg)5bnzzY5bN~b~aobM)~}B zpC$k?F3)*EPg0QzYGzUX&0Bq(Nn;sWCuEWX~Yz$S}QC}ma%*oOTsCe?AU_3pPYge{(0 ze(IJ^lOr+;-PCpOz965}i$A^hV?J8$v_63OH+;{cQ*zV$?s_m@p!1?k^61>@1y3zC z|J?QGhHfZqs0vZy8?@uE9+z&#u$d)Gx1y}&(o_V#Ea6ItRPUSe3XK%fq)I|TYb<6g z*1K=by$KP3qvy!&!IAH`GIv$^d6}6r^qeY$6qlY_+I3Mu?pw};a^sM530+(rQE(Zb42@?jOEOFK z$`mt;(7o~_D9UblVT3pF&H4b({E6L9H50wflw7nX^R_=)_R_zwG8OsdCYB1c(lpW;5g0KgnZ{yQGbjN2;?!7F~e(y`w@l5 z+{8J#ga1WTmzfwfHMrNmWdL;dAb$mTBBpJK)c9c0@gtt&jcp>al022D#w6szV$$ol zAzWPVZ8b|$>qY3CRhLw)+v~*IC4)@#pIJc`UjwF^Z-$mPoP%oo?A;2t)WREx?cwJ! zZ1supmG~A;=nyCcs{c%Ng64eTy&(FJRFm!E#Q+2K{&{2i@A;p=h=T=UpM`2*_Jt?coP_pP}S3{Xy zIW+YEn*y~;k-tOGRw$%<&x%YvN$U^Kz3iv29Hm_HTt7Zs@Or9zPvdQi4?aSa2_7~v z0;Z$-osRZeQF?4rKa0z(D|&OdycQqm@%BeTO4YF#>AxDoe>T;cm71tHDLOn(LFBj- z=bao#n!<>1PC>JS(@H|?AVO@w`Rpd`F6lJ@)g4PKfn;E#Z zo%79AEi2O$iWe+_$N+6YbtUstFUD9i151`4ecUfUX>8XV8h$OEcRjO|$4K`GV??HV z3T9HFW0vdxYh$df5m%=8W$!DMz@wIJhlpn>1_$+el~!<{q-P9s@7UVsE5WFO_qqA= z%KWbrhqA&LF+Xk>_c2@a3d;&FBrJPTTmpPli{`VsQ z6G#{5qnf!xlTRg-a7H7p0v?LK!!bqR?mH%RJXFX3gbIP|-3vdz5mNpZ3KANOC1bIC1uC2XScFizKD*3>>4X9lR?@c{hsVIB<3L6OY$XnsutvXS#-NpF5$2 zci%(_$*waLjbnk%eD150?G8))u0+{PRWLx08^Ij+S> z3@M1_4$Mk`GO!Pr^w176J|%hjMLp$5R`FI>W%8yvXxS@@t6UyQQotK~cU1A+ zGv>pl*dlL5$r6*FRl~3g;;LGm&83=Mh_6@f5(O(DFbN?s1O{mYxKKEp9+2v7 zE!NH;_3LQn2U4K9PtfcaBxz@6qpVW9p`P{W~t%~2gHasR<}Sc(qCN}F=y9nIoky(yr~LE4bYHc z{;OefTZ$Q zV$$)yuvXXr2U8;49f?|7{Z>mXH}R~=(7O513cda*C6_4^)x zs+!GEnT?&HTX^;DpV@^njAm2bBRG{r(HFyXgyd#XZr4G!dlXKbe2#rjMN~GBm%Ym& zhcU?g(aqGj1ryg8>GdbZZ}8h0;&Kw3(`=8!>%D1*;VM!7V=9~(BJ(`n<%RX4 zCiayFG*Hsumrvz)wr3(4^c+Z~-pq)rn{UXe)WdW_RO?UOMqvLLxNMHB(jy+nyls?u zDkC1GPJ9Z_dU)aGTK^kCEuLu&7qi@1{{icbQ@s|EC>TJRxS0)!#Ick17gOfe_zS^@tH^lsqHBxG4r3Ht zxH;*NBifWEGQh3!l3M6bY^h1UAUiSi#b6dkWt(JKWz%@R9xg@BTN+LY^xaj8i#Em3 z4R;B_J@&d_lN2jt12eM-oJNq?4Me%0|x0gQ>M{OB)%GkG0C z)2_)kA%fUz6>Hd17(~q*u*WR5ghqz5E(mZ0_D2F>KKB!|M)o_H%hTr(+X*(jZArB=}RxC!t%K2f~yeTcY2Wla%2L zC!B=rHkrC~>p|{IXE)ny5=hYSM&ik96AEZe9S`Hm+|!n;pA~*MZ2JcH^x7^4$5sqW zn?&K|YzSuAmgX#0qLY6R98`E`oWH{X7Nttau0~=!&s(r$OmYrkeooj$WKZ0@P5wWc zgS+gJ?ESP!zW|1cWJasiA1Y6=T|Qi@(zwFuo#lYvFweusfcEY8u!+S=!h+7>32jb$ z4V(gAf-I}Mo7ylm?bg?iH+Z;Aaiv#neI2;2XFE&y)k&&M9TW%Yktm`XR}^Sg z=n8Jg#;eiw*Z&@jq$K|DVnsImLK?n3yZn9=jQ-e`$bV!23}R~21vw)cNn)u*QYnvD z1qAO|5jCqO#Tkf-mirKyZ!E0)bwBTC^%(|1O_F}YV~>4+qUdStBRp-NEY`@42M%Ce zh*j{e>9Q&j8DkDSiO%q<$B-Vw4!l&0X{VWC%{%36+v`$V6A$6=swK+fI_82b43buuF5wRm^QjvZc7H}Lq+ zaXAubh2wgqV}Cy&EKB`Z74{5~Vyc@{!y?{MJrdm9IR-1|Nfi4Udw%mb5 zE|vyDKZczo>;3%EARVBnc!jcnVQf5*pE3yBq`x~*c15|?U3{`#cT!Ego3lq3u#gw> z29;p=_hDakAk?7w@9a^zEqA6Y(e%MtHJ^5wVNw^pq9v|r4W&PQFwS>bv4Mb|*CD1W zFTvKS5o|p-D#KLYtG}aO3M&Z{Xksk0e)(QXdf;DVAWCeg=W;Oxp)8IYuNagbgue7E zRjohr(kQGayMFxW8~s;jQuNakJ_g6OJull|?qMN{toyq{C;7U<63FDZQ~U{s&+{e9 zT~D(@_w;HPQvNlYt?N*oNgow@kS)%yK;J>+?lfPiU^#r%2tx%3n9}Jcn=_nsFBepe z?JdaVcE`eZLBmZVCjG$~OHF^I@!I)^I#oduhKj<3Wal(6*UjqF;vV7g(`-z8ho16kJCfI6aiEfaln z@%s2Sa)idpw>)UNtT6J3Tr1;X!%~5&oxU;`w;V+Uelc)!o3y<*Z3ao+rwk2ac$s~Z+o%5{? zNGc}rbcGYJcU`jUATS_>pU>S4?hDd=cdlrD0ce~N2v&dHzFvGM5b@Q@I1&X zVShtU{vN?6>G-|~xdyt!G(*__U#D6}*c&w4Q4vY9898~}ZAgs#SL#*C#IBex|UD+`wk*CkWE4f%~6n572V6pBT~lgc_x7+#{vsRcH2_ zkQZeg1HU!jZ;`IYk*}|E&xuYYLcWHgu&ia@1i$``MQF+@Tq$&$5#i?`J)#|+=}(W= zU--L6>!7hL4gm7I5ON??xxvW^KMvl=FL|5Ga=n7I3XJKZw0{Ef}jh#SQ+e6_i< z)yx0b9cc?m~Zf@IEC&d=S@>1u1G>f7pDOf?^%p^jd-6gPY;;qID4U z`sQJCj-Jkww*1f@nP)rn(rg_<#s6HhQ+gst=Z}=UA?uyPY5dy<63FVER}#yH-ZKXq zN4`RDEa1K-C4<|>g8(sI6(AWoa&tb_dH{X!)a-#*-|O1&QFe3ab`Y1f_Cy1%DsF*R z^^jeg<#NH-+Gr^C1%*GqK(_nOGkvkB@`}<%7TNSnwdQ}-Nuu>Bkk6(`ZkD;sPZGyZ zrq~LLkA55Z&PRU}Di z2oil3KD(d|t4-W^fa)oGHFC|XKj_J<#-g6r{J`;1hKR=8yQ@gQ!X}5upN09Lee73U z)e1>l1ugEa?guQ66ht(;IGzQYvUW#2o9jkFxo&Na=l5sjO+QZICqCmye)rJXXfzW^ zZO%ckQZ&*bs4YvA-8ge5Bisba2P!}oJ{P&EsU=+iiDt+}WMg!Jq?QEkLYi}Nu{)=8 zH=~}0(80@CCe{Ag>qOGm0Tw68mSt4?tp@F;vH@H8*-*Zrng>91aT?DHvk;Qc2>EE_=Oyw~4lH4$*l()CXjcTQC&%DR5G z1=B3XA7A~}gzX{$w>e&t`OVFbB!lY{qbYK;Ug=13hJ&4Mr78~&4dMu5;|D4RM1}j+ zF@XNEa15Y#mu(+P;^!5JWv*+eMb*Pese@be>5ZHdcl7o(Rbmh{o7>K`%ESjY<}?|N z_3`nf?NabP_*IW{Mv{;zK1wqiOAS;r32sNAWu283 zT7iGNsl{J2`}?PGQZ>t=s2b`1ufMRtBu$YXOefWo$o-Oq*vnlK?A#BTACBY5q6p@C z@*had~u}icXNadx`XxRJ>@r+iM3w>YO{88@D;uZ3p&Dt zLFc+GzW<}?%cG(C|Nl#iBuQjVQ>kpBl6{#<2-y;nWwM7Dk)63oNY+9^OnBSZ>`RO# z`!e>O!Pxh)&U){6KEHE*f6qPloVoXPpRd>R@q8@Wbh@DAIJ4$-G;7Vgu#_Ee_|+rmmp-&=B99z|3^3ulLD?pbrcic&%^T)Q5~#yd(bQ5*-Yca`y5ls#$Di;Mz3ylg45TmqA0U@iJ_AjKvpv{X?{T2( zzvEY(h$pJ6PfS>VvDTtMjAz2sjzsl8GGup+T?MQ^p&X~blqB%;)ed{06K^?REUjXIhKz9*bbh;e-F zU|ZeqgTph4$6u30dBA0}EF*-9J-pSbr}y9blcGAMGSPF|4O?EK z{(G{|9j?3Clt!DSE(r0-Wr!u72fFWMdY0;NG-`WfOCK^O`iqmjIs^q1FO6C5X7P&1 z%bm22Gg+{1Xb2pRLH5o+pEpQ(mQK`D<&9NSQ$F@p0~;B?1))V^BWu&SLOFXaTe2Uy z?B%NuV!o-Qut0RSY{4#UZ!3QkBdL7*v*tJK@@_u~vx<r2+^lv% zTXXlMX>1I%k0EpHC{mhme?>yEE~O~0T=7wv2j`^5#59hY7$|MWNqe7n2TsAr9T`~Q z{V3XfglpL<`M?YAIGh*=Pvl+wUT{9K?9r<^-OSri-qF-9&t5J{6rL+%CL;K#TljI( z@K;c!MH}_i^Oa!b$5Dzg6Fs^^Iqx6zKW&M734(}Uz%Olbd3M^eWRR-T#Y4ws6gg(L zA-Ub^pBLC98?J70Y1!1-QV!Ej0q7;j(&QAByAg4!g#9p!IKnp#9%)G|FZFuHWoLuV z$}ECIX4{rj79?}#C+1mBTd1Hxo;na(6W))EPoq6^~Bq(h$%-JTSIDrc5WT<#9vx=EDW- z9{PE)H_`>w?3RSNK>DB?UA#;X?b6F|n?NxTBr7_S%6*UEmE?P%4rl@_!ettLc3QDK z+bI>&+hr}6ma@0nl*(ZHAMZW&{{2YvvaF53h=UsATV>TG+n&uT)AK+4SH9`!`mU?) z#Tqb;uybzgC31}1*x;h`gm`XAK9*Bxcx-{2w+v4|-SQ=TRR%@}xnW&HqN&~IQg(z4 zczvwGsV0!tLJ-HxbBgqQZ(6X1>37Et$r5vc4!v*k>NfOU&I&Gt8dCa4>4Kt>+z zWcj0>EYYI<@7`7SI(sg$Cspl34^}t~{>H4Xt^2JTK9UMQR^pc+re2(yBDu>)n~)xZ_%KeB z9yY{Zg<}P_S-1T*1s*q^NUjcbT5SO?uezUlR^hUW$p@I*E`Dh=ggBcw$8rPVw5iMV zrmqPVjy7I`#3_@(kT_JC4|sLrY%$UGySBtKIdE`F037C|r<$4|7k0O*C!eVn#*|o$ zuH?&eXYB{ID?YoOdmbG`V>mNa1g4Nw=u$uzGCJKruBchPl(drQ-yvnEJDodEQR9Or zx_V>o@%RAhM2(_lxXBu^H9Ix!AOfBpN;L{5c;<}JO}&++|(OY3W&sIl8R;jmoA)O55$wLkX4o0O3boNb*B?dR|ll*kRn z|3;ET_JjA5jqHp`T|1w%cc77khTSm)b;S%rXs_?9sS)rx@7UdQs#uA5kPS8&Ks|XG z7+k&Bri_C*l-J!;u1-NXtLW0Db-zRd*6HOeak%0;r%xuJ@$$nM=EbEKgPcUyN}X>F zd%X2x+DYj>9f6{6`_xub%k<7Njj;ni& zKn&K1YMI?R9f-02#*V6;+L6V=b~{OlbLy;7X57soA3AT*MR_=!RB+L0UR4q<0u1xnTXuLnW2^JDUm7YW^jT@UukpiPcn z37y!wU~&xa9C1s^rL0`ZxcQ&##AVhaJ{5`ZQ?ZSLp3WOb-MVJxz>?`oJE+O$psdrl zg^X4v2uY~Rjw2fxS{lDF<~x~d%*?};yh?E6J6`bKoib`CH&6qYl5u)dZ}zF<)Jd}( ztMpYGH2Yy~#FGPD zou@}1iE2sD6Mr6mP^9xi1_hq`8vnzl!>9v3u~5X7_}3F6g6IrDPGQnXaJxj}>(x!; z3*GiLV@Khn_EhTyHbM05(7E@&UjMT3~d4&B=d$eDakH6vX>%SyJbElsc1Jx!f z((ER8zA{TU;f(_LzfJH9M!Pt;DcW7sQbv#BC+<9E`(|KkM@oI4n`YT`!_;M2D9orW zHSB#at76wqnVBCRi+FHj?_1A0Gv)6?BAwxMyY5V0?L(Um^=Ey*Cz8PUnnh+*Zu0OI z<PeqAqrGF4~`w&f18Nc!A z_F0SV)^;;yuwVn1G(igb^l8a>4A2H&K;JQ=m|LoK7${#C0W4+0s8eII-+N0Y4PFZw zRyTN~7N=&TeR^lvqcI&y{9AX$G-;_12-}s2iX+(7uu1=Tux?ZI%x1+q>6k)1f_f`3 zcBG6hCpgPUVYw|$v$xh?!Sa_QEOj$q=nwV&=v!rMrR+%%zU?#<5-O)Fjwt)e*uA|e z=&0CaP6F!GY{yIZ`y654M&ola2jFVf|7?1KzFC}p_O*_LLPldKTM60Pbo_146k0Cs zYwqbjADf>`he8cNm`t zs#%BopZ5~$YqmEqpHt^{h$W%J*$Vfrmd-fT8Zif55?6@aV3}j6;4*)m>o#%m%+H^24!?sY z9BK0uW=}(hwL)uIw&1m_|5i9+j-`d|^NCbh1u!@pO}c)Ew>obap>*7Qh(yhlDo3^5 z^czCET#r=j%#XN(v2>30kz_FGb6a`l_W4?r*+^LY@(PVk7h{DVCssyjb>0|`but}! zQn5KKn4QdNxJzB7--lMQf)Mi8aN?_V4||ggYVW81X%PO|>YLP0i_`eP2a4&cmUvPU zvD3$4x_bIXuC+gN?w>3ZuY$4A)cY{L+a#F=g2z31<=(4G1)aWsynokoDdtem{+0&y zJRgnZp?4){GNk=^I2obItQoG$htH@AWq~fgl8UP|_ripNI8p_IpE^v<04P4_a{~L< zXj(sf(tVs%&vVje2NZCi9Y*ePKbHsm=rST7n>Sr=q4)F2W0>Oy@OP*`k= zEV%|j3>7TCr;%RhM4LL0K&hpD-?49pG><5A_DT0&*B2b)94q2cSqWn#NnY9C+#P?( z)bKG3K}z)uS``xM1bQ1@=XUkf zCTws~1#M(x>lb7Y<9^E$F-`d=>XLbt&z2(ts_93;sp`N>dE+aa(J6V3|NYneRwZ6_ zo5^6%0-7AN@S6{g50LW8_qNgT(eH1Q4&;X>;5${gP++C(!GFW&R5Ur_Uz*CA73)

    2MP7y7_XBRT{(z{)1lD3sJ(+;KK_V?QiaT8f+B3 z>@W3rrN$nzTw%Q>-0=}yuHv@U3@6l=7h`A&#q|ML+>0S48Pw4ZNFT#h1^cRQ?)tj?&Y+$f0uTG_q8H5Z&qg_cEfcIsU!Mp>t}qFpf`_?)KElxUm_ka0}`VV z0R&ZdkI=`7i@)=C&s&}pOWRc??4u}!`dN-J`Inhz-sQ?X)IZD;Oto>qL_9#+#c32d z+T}VXcDG|W5E0Sk)h)}N?eqmlTYuv+f^iUc6%(9r^nZTJm<&w-y?@n+8VR+9fw{kd z>vl)1-@K(L0lQwxw8mnpWxD=$R$gWo=ZGm*hNcgZ{`+)+X$uM6J+Iep_sUdlj38eDiJCJ$?hM=Zw<<O&t zihF5^i)#{KI1#(N>v8!{F6CLU14{b-S37ADva#Dnp@^PI#-=!Zi3T^ESKH&mk<}a> z_ULlb4{p|Nz$j)R(*$_j7xIWxc9w`Qo`J%2cU{E#D}9v%icPUk`?}W|YkChA(BR?!s_5HqQa?iK-G=U-+U-4tlLrvg3g}1y8Uo5DFt=rrwDvL# zZWd3K;t*dLzFetTl^;@}YFMo3u9b|SaKyH;pE(yO>oidp)hpzbg-lGsB?Zurh#5iU zn*Y7LF}@UBTq{!duv|YD+-p-?hD1Uq{}Qx$7Aj{kk|06k>E8hyt~k5JG()ILoxU(A z_u6`x3LlWMVA?!d`I%F??rDi#B#R=veTRR>j8rv<8;jxB3hQi;2RA^{_ancpA%_Qz zl^S0T?;gkm8#ymwYiSlxTU8O%S@kh0EsfgJ*pB9}8AyT5{^34(@M=9+SnM2yv2<&Jgz$A1j7$gnWTLkQWlIOy+Ykb3SM;IfuM(?_ZdFlF zuG2T!USxp5^zmdu5gz0Jadd`?vk1U1Bbe_`Wo}hnpTWa4f!#Au$orPb;z3|s;%JwQ zMzQ-o`}%Gb0d9R6iyBydnB5Ma*YTl=PVUg?&z<6)mNViOMtC0xzn-s4Wqn-_ZXw@A zVIHGcARqr!Rmk$-MyDQZWui zH6tRD7u;*rhJKQYziJPe0D~`CZRqyz*9u=!{t2cEF}16YA;_bn1lxQ-G*=RMxp4W& z4>zVvXcPo)OT&2T4vnvqo#%={Ykx&}16YBa`9_rM**T)1p=r8(oXNW2TXiO7XjUB+ zP%-$p$OTciu+og6wbl+yMY%fd7f--#E}ee#yqz?*%Syqo@Id@WPLL5b;-+Q6wAkB> zaE_c7ZSd|vduadnR-ytuz4#^Fx`ekMX$jVOfSGx3c_^hfxFRJ)DO>x+Ib&@Jav*xk zX%}#!MldYF=^A2tI+2l0GP*VvES{08mf({eWs6s1IyMbG}S`td9` zV!nU=bpOwjZt?O8Jk6pC60a4D1^Lc=UV9uXp*uu;hR?WcCeUq!n7!XP&xj^-Koj!= z@dY2|T6zu1RkSpS64kduC=f}xOuU+svbSk?u$;4ow3f5J6>50@*P17ohiXG{dYM0xo^ad>_IdD0;IaQdAQ_xP`rOJn zz5ip#X-uc-GEHNXjHo;E^!Oz&McaC$U|xRP%ZTgy%h1a7*spQ3Q4g7HK0yV*;%S(B z@fy~a)#4TENb=w5OtqEhvT48G`ep=37f~Sh)F~pjN2b$yKo1tcMHmw;3RL<(LL~?@h=rV9@&JI#e<*ks`uTt&8+msaxO`g zmk+|%uBE>oNYJDc%k%%1q>%XKzaz9OQZgto+IT2}dJ11KWGzCE*xR@HLQi2dK&3uw zDR-J*p>qNsIgY5ae9(i@B~p?=2xxZ>Y=t<=z23Go+V2j{ML0YnhK&{ix_`sMgdBUJ zcX;R`Wuze|0gPXemv_*6C`=o;=U9{!gmM^`rbJf#qxl@x15exgP~n4Stf=dzZ1Gklqgt&{@^xW{hlxhD5NH@ zhcuAHKdE=lAyx5DYdbhR?@(UI@;UK|*A-JwP0+arHCHIkLRa40{M!%9f8kbcolRu7 z)xH^w@yBQAzj5}z^;GPYqkkaVA@0TJSpv4o3gi7By{qjPcu4!ExWS6e=hFmA-Cx}c zH~LJJw3}+f2?>Mba(2xJ_FB>JBBB>Lpzuwuw{e{?w`rD-{7`fM@L+Xr)ews9j~pmX z#lxrqD-3q+&@y;3t!RK=v?xX+A<(~Nyp*s92{%Mhoq&Uj?+WoO?7bjEQC zJNkseM@gR}_b6R)ON_TTheWUz^dy;M=4^*YwZlcG!F2pACZEp=(-SJQP}O(0J%J61 z1IA(K%|D2^pESw(Y?pq8peza>Iom@N45IYI?g`gkr8B7&GCNzYX)e|6xQt`bif@*Ia zD8fhWhO7Hw?M&yf^Spp!K=#5)z>(?ojQIlkY4j~KxJXd3O`fIsWrH^L1vF3pWs*6| zU#3a(V}5>YilMt|Swsi@XNKSjj)=lD#Qe^CTusJ^ZR{ zaLGQt>d;Gv_0{3-50}Y+15TIdN6~3#6ALC&`Dq`$Ki54m`zVWuMU$AmkF@HJq#Vd7 zakZxt2@=YOcXzr_yH%;ew6YL9Ph3HWY@RAO5ZzDx?6lJ^qZdB)>}iwe_nd68iHDc5 ziI4ioS~>VZ^fSumHDA^7XZt_ypf6j-7<8+3#Djx1$Op1hPoOj4Do$SAc|@HqC`bt5 ze(mxD6s3-QnvV>=ic~%caJD+lGJ`s=?|$$_b=|Vhgul&sKlP-Wg?^~|x<$_Z9GS!a zbiAcUB}Z~ItoW(QMs>OhcK&Xy@s`&2T%1M z1ys%(YR_$BR2oTPb%*4huFv zN35`hsn8t}OM>3ko|7()M)2XSNh|jo{=w~C+}%)DD4B=3{e&~n9H+C^N*Zu+j5;+3 zbZL{3{J;v~o2e8}@z|j~u_1}p7W`Y}>NT=`Hfe>K03#2XSx5B}yqx|2!O5ojs~BTq!}~Z*+_js;GRYKEQB)UgKj&h;L2gqTzodXw6^QAa2ZB z;}3GI>`#MwJ>KlZ3ayX@*0$>2hQF32(Gtmo&K~q2g!&-8{{OuIE*~juZajhh_MB?G%Bhi@rR69`1mFoU6ET&xzvfHfrVpCX-0AqGV>( zA4zRQPl{}f9>Az+rc^j}iUXJ>k#;gy5R%|gpRfOup?)9qOCs;&cH)^pVwypxfO_axt<99W`T*Edox(hvL^X<1?Kr zi@yrxt?5RIqbuols&CbIB+v4Y*;`U(+)G2@4`<(Q1gy;MOio6!cfBav%FC?Gd7lQB zcZ@CxGx=J!0Ue=e6=)qt_l6H%x0`#Py$k!w0h|YHJ?V$4R#*u8ITUKBLNOd&s<-~S zVYMbWbEH$1<4?IhsZv6OM8MLTXwiy+=hElsUEn`!Y~3ImmGo59BZ-n=6oQ_0Jq=bN zF&~_3lUjwrFgkvUnLlG zr{h8g46I&RUh$+HHm}mJ?R9CW=SKWPypV?OZ2)Psu1^D)?^eBVEq1J*)hafMH2vZ? z9!_DTPAMwiOrxG6%hJ`+f>b0CF@So^$}=*i0cF!%ZBifou}iAUuGeCiAjB7~-CYPM!YH7D7BIuGPfHbcdJEtFCU z$0ZJrK*Q`Jp#MkxDJTtaDDvbx>R$Sh0$M6H@Pf{(Fwkribw5uVb&)orcgk$yO`NvV zU}`H}nbu@}fs)jQV-KZ2eI7z*Kfr_7szAi?(uc9y;Nx1q==jJ-`u!mnP4VoKw;FDu zrXGMr?daO3uez=upXQ~&=|z7(quqkqMW3&lh4>a~InH_1)EIcwXl{$CQKhNun1g0x zdvF+>)8&q@liAUu`Ulb`UGG_hqvhh}9zL>~ItBWeU1WUgH6`(@mPV|>vjUY2ThNg{_u3#$Axdo*fob{G?z#m z5t^QV?2f+AA^)t7duH2eh|t8cyxy7qDe3j6i;+ai!{DEO)x1R(<-0y~`>s#P#mM&i z)sIEvn6iGnTHg^nzr>X@s^szVUi_<3J5$wSY*FGpTJSReRx zFzT@sH4a|ofO(yEneX-XdD+$6WXjvi)Lu^TpfbmLkQ|De_4rq(9#zx%>Y|lzZ-f8E z#&{JO-Y>UQ=|<%5Fh|8?7SxZqB4Ay^^qr31E2i_#OHevs3adQ#xC31s4 zDUfj~?yS_|YZGOv(SADa3D7IP^bhMeVGMM>m#Y z6{m;s^hS+9lovE2BYIJRCQ3%sm%>rq)tAxUfA2SxUl9Iv;hC&7IL!@sk!6b1Z-IYH zX|HhubkK-Yhoz13;SP`AuK5?W#jNh1&|eE|SKcG;gH}nto?sbLV`>U2#oxbq$<^yY_HFm;X?ePDyXtTLQ zdXMRGPqcT_|G1r4Ert@O#YRp$V4ncuP&r}gXkn%Bh_bAGlLuYdO?8?_Cy?9eMd_XU z!)XW1!>KHsC)NKC?R@dW6OaD5%lzG9yoT@vwe0SCPGG!z!F+Zj=M{;}JA{=~qR%QF z+hpgLZ8JO}uQu9oZ)acZ{`B&ow60#p&-tDo zSY`@3BuuEF1H>ojOvf$*D=s{8`O@msl^dsZd&E*ta+yBV{;-aApRu|SwrDRlkUFP7 zUga%Mm`oI3`{&pAWiPRN#&7W;CTriRchUZ2>iwP2r%%qDwOpSdU{euw?|o6lL+bV; z5npCayt-T)4+gsJj$I3t_e^H>KqWH<8w0VcqFPH)C(>MQ-~UZ^5_icO8dc^L!Ih=R zFYYeh7v;}{W8aq?cx+`2>9uKdC|kEZ%cwCA7M9VY=i*g|ooXH!N_3gg(lt}C3pnZS z!Ils*7&lblJNLH!-{K5sjY7%GuHbV{%dRSPwBleccMX>Od}7*{D=BS?0k)V6DIH(M z(4^__TiN2LTiaKMoc5RR(}1ixd-YYcR;9_we7`QNj?S~CkrL$%tKvfzqu+z#w4s~j zy9h$|LGk{{a;eS6r?bRA+hJQ7cj2Bb=Ho4U3nNAu(R?T*lZW1P8b!ydK<32oFEQ_9jcfnRvm;Mft<}W?b4~m| zZu{74>|*k0FWAUAuH)v&Kewka$|=auH-ECmeHTZQ^ly0pTp4~JDOckx5A|ETzcd_zWuv{ z+5G@nU~x8=1){HOyHCNq`p9XgS zEBIUBjDS?b6-w_H^^w(#)IyXzQznDz5d59PMCFtO=Q+6U&j)`bN%j@mRcj^w7c~Do zki#=)7dc{9I|2DpVXZN3JU#+?J0>M4bwf=rE7!K~)}(GZ%AG3JsG;4{*G`;m<1#X) z7i>{icYeFd;z0x6661&CbHE2Cz9)1k`2u6Hd_fzI2heUl!`eT)fR=E;$UD~D9o5%9 zQy_=V`>z&N{_npSok=p$>Q1o{9{oqLwV(~LJYjc@aK-DCH66NfkDj-}>65>eR6|!!A4>2Tn1Q_t8yORj z64&nk6&Y~0VL>i4AVTN#IL}$&?V+=$TCDsg8*Uw%+bcCYb$p3lQJmEvg90n*74F*D9MDHNz4A(Ea@B7NT)2V5K5{1zHxRCSl@ zGx%U1F`bl>mi&vlpOOkKjK!m!S21u@mbStc7Z^d@q8CW!WlXbh<;QOIEmqnJA%Ix5 zeYOHO28MEd@JmFvw`54_|Ll`>+uv-`1uCh4huNX+yn%2yxWsXoG(Ph7xJC1a))Ak;H#Y0r00tp7aUbmew(y2(~KRnbC;+4F-*K< z1!RLy4`sd}D!){u#dDLPSNLQY8U6A3f&H09iwC>u%pr)$me}hTww94(_5|*aLylR@ zM!_NG9d(U7E2XGKaZqS<%OePRmnG1h$7C5IV4{Ae9 z&>#=PINn8+hCB%@x$8ci$A@1SrErC~EMc?2nvHA%f`)9~4#b>ZqiJ2S%;$_PJmYFS z7adbJnC4$-mj+L0AnE^X!_tRtW)Utlg$<Rtn~tpZ5d zGIsn>#fi#W`3$TfHFe^GX2SrI%qPEX2Ge6!QHV34Dj1?SMa~%7*%xtYk%jbr9zo?H zC!kH*A^#;9>Zj~gsFC)v!07^Xm5RriRjLn(kr6hSb7H&(ek-5Bc}-&RL3=RS@v+N3 z;eECgWG4mw8r?*IQbWr*Y~LKK!mr6z81^N#L>kng zd(1}dxLJ<^f$&B9Oyoa7W=aJp1)1@38ZxXWfg`md2IOesR|6AxG?zIHh=kceMg-kf z!7nf>rkv*XK4M+ZV_A?wtm@uCw$Pc?2SiI=puWOUs^K5z8?XSZU^BO5? zk!6EB`uX9#e>(IJ;~ar;!u!!xJrJ@Ljc(g5HDhho(Rs8sob1ZPj5)KA??}XXFj{F4a zpXOH8!>e&6XZ#LfV9l5EVPFa8B8swzl(-;6X1sZ%jCliQ{&**f65nzPqQ{Im<1@(a zTh7|b1vMi?9w?T;2BC@f{%E72Wep!^{WedUbKBA~rkhx}07YP0(Ptoge|amzEfG}C zq`t;BL$IX2XV0?lC#Lo6$3ME>w8%Ys!|B>m>-{Qa`Q*?{fH3Z@ZdeL zTrPg{#GiWKU+A;9Ts)_3_C7m3rqEy(VtSQ|oMA>QFkiroS2$z*kHotN)y4~10i8kAhE?O6akQ*Hh0 zq^M^K!{~~JjG<#50rc1CS3b$7RJ;#X$gH{{u~e^vz-{g)^jr|kVi;I(lV^R`h{Hmf zlU5_&p7d}}3*pGTIQ%Ot@)TNVw8bH@x$5|xp6MR6B`ZVHoj48`wZv*HfvgizO7VP-1 z&feeqcJZ$ZU?ygmpU4H|6ccqvns64eEqib>jM2U`q-fx!Q9tI5>TGlAfdo|x@|-*x z&v|H6Fh;`X350)8R%A^aJEZ>rq&2WmYzH*Y)CT zwqbq1$B>`-)RQ(+DMY}K5-gD8*g9+k)q;DW+?AA-CUlo}kv65GbEY8#7@81)N5gce zkHIW3p&aqhU~Al%#CW;?E&cE%$ey$_uZst$Q}nqmD@dO8Z`tWq&CwKFKOMu8J0*;s^&7>bUWFMYQ9Fjm?yD-BK$m)6Qb z3ZU95dI(~J%cNqjbDz=4prrxSN_<7m7crfXcySq+4J7*PP{$H=U_OxA>yrnN(VqvdqK?GrHx9kAm*aNa>H~!w`sK|nwza3X0)%|P zUcbGv-_Pg03qI;Hwd=t~sK&MVIKFHfg~p8D z>9bwF6m#!Z&pD<`vFGmSKT_kd`gr~Gjn8=(Zb?m>9s{Yzrhe#v5I=W=T&)-0o+C>3 zI~8NTV|BC0S57;U8Z2%jJjlmy5KhF)VQoy_sR2#rZW?k1EKcuZM#2P_hZlo-UPu=L zh%4ZxQ(Y8P>E1VTE-FFN$NDDjIMVy-3zJM&2RNUm*eq4#zHTsl$?UW0Ltd2!`~zPr zvMe7nZ|noM^FIVrR^9X2k8_B+O#UAlE-|xx3jqyS{6MPFU*cia*MM-$anAz$<~gNh zctrhyu81R*=LXkXB34YNS%p}18FQ@N`ffI)@y5qPp|=^CU<$!M?NEJO=^+!EG`ju4ev89d zfinHISkvF|q}6!sp2~*TKXi!B>zfh#E6Y>71iyTCU4{oiqoegeSMz|*x^Bvd^jV#Y z-)>BKqjDb5J+>Xe7|5l|a z5UFr{bx$(8nj!nn18K-i33I=}HDSS#R6{%__?59=b0~f{FTH4B8+Smx!M>B22C&S3 zb?M^3VgMImIPb3U?x0bxCEdQ|>_MrdPE58ARv2f5#~8a!{VyDtH4;+fx_ zIKGK834Q+0*kM|-n4kLx)U$~2$Z>6zvyWa*Xrh|h?)y;LO(ooxDI_(hnejUZ2TKy_62jaVI#+kOl%U^}`ALP;$8$(z9? zFXq~jx3ur|;@Bj_U5p(sw;!6ri}^8CcLF2!1G!<#9kd0+mR-#UrKWF87Lf5n~Rt z2vt$2Cjys@zp)lpG>@o1j!x1oc2Dj^59l|QO;n=YJ<%}PKS=@ks2#|U4O68|_-~E4 zP6|T&v)e;QA%@j52D8yHRo7Q!ogMhw&)w-ju;$q<(&4}27heAyvllH)0mr@wTrKGK zam_}a8bb{5q0d_}nGsa0hsiK!Yr4WsGFLn?z>al5Q$i=AY=OVI+8*c`btMeM8y^AOHI&xcHCHzo?V}^Op(AN7w0|L%OAm zn6AxK$?=c%vKAtvD5(!VVrKcgjP98Ex-o)%m^HBUhSM|hlWz1{dS18CZ5?d>{8+AD zN-yrdJln~+QM{Lez;{jDNM}Q~npmAd%)-W_{_IPR{lVzVZ~JRK;;!-UorV&LnV28^ ztGKvR#e)5di`8pGc)vS3%}o3ZLfd9G*vS;=0*Q+r0py$WwO;QOn|x=%x0CVEL)+}# zpdVik6zAZDx?$6XneC^wsrESl78daQ_lszfJmuoy*w9UN-O{R^=iu)@Vf`d~ma0ut zYW@>~1T}&nPXy)>PE<9CO)lm83!kA2hv;@>jWrVSG?fRzoqqr$0~21?FlSX)piEu= zI_54P>cxY^b|e}w>c-N@mda4y3NDQLE-&#lT$bWV${vO9ld2H4zv;2(By!C!n`Np+ z_VR{3Lw0oW$lks`Z5n~@8^f}bd@2t2YeKJnwieqqvrv9(JLDo>?7x4V)!&#*@W)#g z7Mb5+OW7h1l+|?Jdry17j|t3oG7FrVigbhB%=|EnNbMejHw;bxTOGT!o2GrqpOtZ^ zXU;r)c68@lCT%tCyF(G~^E=Ti+VBCdmM-bqBw{g%Ch?h> zw_=CmduQ{Q8fo&l5Z-Km7f*F&q(ehbOx3SylhymW({{u~KJod}zY<}o zLe?u^F)T@?_t@D3l8)PzNB?s<(7%AdMk~ME68l!=&|iXdN`9tC)+(P8+O9~CP=U3; zXsyWYfe{*L2s7U8DW0&E(XK=IYHyGSrk^4`BOK9nWDm<^kq)ZV1 z+K@n*bqg^YYj}TlX2XE^(ZGEb8unO>42fy|dyjHW=`Q6-Og(X>5l|zHZ0wlGYjqEX znpfUHr4(DV5uDX~vL0mcwd@=wHcHrBo7bZRqg_@AQ*k?nBBs0JCY?gR+IJ>F$!Odj z+AZ0KdikYI8;PprYm2%?C>@!09&Lp?qB^NA8q!}^_9fnbR$hWDdrsYJD2U~WZ6}c4 zmp3q2TcPgKR`z$^(B>Fy8c@q*%Lde)ED?2&lPi)u4h^WH9s}|)s(0u1%e9P`pT8Rm z$Qv9DBgy0_0~(oH2h!6Tn;GY12-Y|J>n1oGkb%^{_67m>HRF;-)eYSPyb8xKHDY>W zv1WevS<5$UPyBRu3coE&&uA-JlqC7)PTxuJ-BbIry?w^W)A5eDwTA6ggvwNjOM>@c zgf)k z5{h0*ryPGlpTS_nqU4E~U1eL{G~EX~#A9-I!%P15s{0ia^8F84rETMufoRNY|1ekc zUB7_QFxIrgIRpD^m{W*HGd@(_YrZRrs92$MN{avBPZSykH&o`If+k|{SFz5HC^Ip5 z4_<~e@-)S{5{&nxzeBo*A1GId{R}L8-zRyk!r95!uWuVi?-NH-M$MpujVX^J;Ph1l z8oPp!O2uN#pBJS-VZ9`(U&?YjJ@8DNq{K~d$CShde?w&bTll)eAtUh9X1t^7aG3N& zvwa_XKwavUORfprV&we$#_8wDrStbr8igrcDceTU%DWyJTo1jN?sPjkX8$xmd~4{i z^nWyc^Y7t$c0bjef%C5NO^Q&JiPX*Q5<1qrFqNVgz0x?wa@j&5Wl zw=uTQ-rvXL`w#4gJ@5NE_jRsw&ZTfi%CY4z*>iBo1oh>>#sc%uM}6i0_OgS#v?v5Vq`bGF@9 zc7a{6jDv<_Wr)c%_%iC`{LfO;>8(Q8s2j)6y}wTp8$Ms5nlL}SvG1>%oB=m|PP7aR zDjLrz@WWT&1Oz(@Al5?g12%??%Z3C}ZT%mtfyd?%6yGur%CWc~hfmCjh6LdtQh^nx z$&70?LVmnHJ|W4~mHq+IS^n<-6nFlEWJ`ffO5ZN;tPTXpNpK$?|H*cj0Z#a~&moSo zxmuh%3UkFov*p|M^V>RFgY92yicFWzFRjdF$a`$B4?F1|7@w9ka&*{#z5y(K&bf2= z4=i)`ZR&Ap-gdTf$87yQwQ=LK8127743Z5)_QnF3B`w*^OC_GwvET$KPm}nrWWa^i zZKP7NmPfP@(=BD-Do9~AMn6}wQx`_w^n!*5W2y0JCAMSXzDEr7{8`U_(++S3K5rj| zAZP~$gs7e~ot^*j2^-C!L>JYFTu%yajdiHvcbiy9Z*r#gJ{`i~T0>AI!P@60!D*LQ zuezXyy|P(pgfQLcOUyA+Ep3 zW3Hn)mmn6_*%#v@e;w4i$)xGGq({R%$2xX-+y8+Vw?d};@CjFRBDFH_6oa^NvvoG}RxN9Zgjgy|gKbHfLN1$yxBath=fH}NW-kVj;ZSSUCAV}81()G(1d z0ReW@YB&AQ!_k~r+%~q2j`d8~GsssX71jRfZFHu*Mi{@1<2zg5Epi)XBG2$L>AQLa z>CqeQPiA+Wj|5(iTHPy|8d6MgguIl<7CS{om=n1>uV2P>u?BA|DK#^xydSEjJkRF( zQ9N>`TDffb!hirg#ibn|u8yg6l*KigZBTy=%E8tzC*EvvGg+C3zFPN53@UZ?e z+=6jk-IG!(6-nBiEw)e{&iqwsCx=xiWZD*|`0Q|7=<~Cy%_i@A0x$XK6$O72KFcxf z2gdp@&ZqoBIvyJ}-TLG<#}b!n*05*HfsdhvN2hWGK8*qPbkXo1OL+7r-kdF^g+08< zi%DPMeDfZwBjCt~?%vUc7}udn8Q36&X;ANft6u=3WiLBA*fP2#j_|h(HozDZ8PZJE z?^;UI8_)IC4OeS-!7m7KtPFRH*M;4O2Hd&J_bAA2ZR07^hMBRB;K~hleu&Z(rHlTn zAUwx{N04Ur-wxI5fS~RtaE8L*{JeHm8#e)B*4oY&|EmkkKErh^G`?XCvT44>nkdD%_} z(J8%$4OFMn5ic66HE!=MGA`=xVM}yIK#yD~w_(F{R^M4uLPuZ%!c-7ym0@7kv&e2% zmxhJvF$_pmCs=ON)xz&n8uA`EY0ohZI#RRF8ul?>XX$RK(W(7S8sg@kDEb!Jif?70 zf@Y2m)y`whXUSBTU;Ms%F@3c}H|fG)1r&l)dAvd=vu>YV-yN6-EsD3RiJSmi`V2-W*|Z5rEv}WL=_-Tk z(0*~=(~B#_n20$3j(+Jp=fZos!;l>25l8bDh&-MM#o;Gv(?>}`efuje{N9r4Cp&?J zc(3h|x(~v|J+x;It<6Um{EICt-f7W5OeS-pB>W14Pmz8fOF}T-7-V*URV$`x-o+Am zv=%Cczj*Y^s=UPYI@Ow&g1GaYSZCgIS0q8Yol4>QOZC&+^9Hw#e4Nb5?6<6hXF2XQ z;xckdM{btuEv2~r_jI)4+rXQWz1)GO=Z&|LR#3u$ZzI4x_Rytq^OWdS-R8CYyxo|^ zE`M?B3wlaj_Slh@d;^NK4Z{tJ9tP`sN`BQ9hZIS*gJ6q4KaX4^_;5Fnu{nL=MY!%a zEw&t`91SRrzizKA@3ooJ|o`**g7m$HmL{H&8>Q2L2h8h2m4@r0dVUk5pW4Q zW8VjiKjU2?o2+`U)b&z5vhSA<+DR*PaA5IJNkuXQY^TNcr*igleuMg=IPtyGuMY%= z*gvau*vrpl$h(A@tY|mTRP4yEw*Gx3jGrsMTo;{Z{tV#hkh5PZw~6rk6-3Jv06VOf@3Zm=Vb7eEw6gcZvy{IsqxUC&}XB- zC7(yOgsBHJAOZ{lf0ICeyzP@ff&&$jzwHu4BtSLWI6I{ij7N~D48IAs$kXGYB!=X*^fxU74Z-v|1KkqfW5{kG& zCp!#qd{A0cK`_P(xLW;IJ~?3(XFmuZhn*ac2dE?EC8hR;n-7~pm; z%t@rQrs=^BYUkmfhZRg+N(4sC3hr0W4!;=<)py)(1NY<1DiUX;Nt^jY#%)5@I8I>Hy2v$FI^#b ztvsj{Cq^HSveaM%BU|pP&ej=!{GHfcre^QZ6wYXTl)xTkG+DBQA@HBxv&y_Ej1 z@zA>bQzjar+hL#P&5YeJ>}EZn#D0T#plJO^^>c?nO=$=BX=c?8Xh(&SHM`^*#e)}U zE0t?IZ2@IjH^0qfNtRTY=!42S-*6tvx;J8ZzNX6!Z8fj{;~VA8{z#r1_$2GUuPo9Z z`Bl#M!lt;v%@J-Uyn#|eoS4P0@2LrP;fvo}IoqNmPbw-tN7d9NM6JrjQ)2$f`aP%) zWrP-C>cOEIotG+PVjoIBo;jGTD^FjbU(3`##sn|=bq+3zuiN&%}@N={j@xn?6 zJM$GLh6)`aP|13niqL1-j!rF$1%Jx7J$~W^V_du(k4{5?0(9}Wp5Y@!Ozp#}Q(bI3 z-L%AFmo}(gTch_VojrNXvQimWm&zaYA*~|rm#;~9j_4BE!8ARX*JBxKJ!7x<;NOJ4O2ERT>(xJlH7bq{plexYTSpz1gu z{0Q`p@3oZF;4PBQ;M<>xj+4*m5Dt-*Q#t}SIqS&#yLF2q1ce5-V_5RV5$Y9z-B3I@ zE@~Voc&-sRww*{!A2cdEgCjpsDt(`nWr1(4Dc7yR+;c2}k7xfQXZE`7u~fD7|VCqVHP<(NFFBxvXMCxt_>$M85(o-3r7u;?PG7YJ^Zc&I2;T@egyTtcLdt zdzfwy-1k@M`@M$HjenES;^Qm_h zaQ8@nwbQFY8XvNsx|w#qwr!eC3C4H?RtY<2mu5dM+yu1n_SpH%nP65Pza@0oR zRfUsM2;Uc9>zcyWk?i7K+uzbK4eAUgL9BlYXO)PZHx z@l%4Q$7Q(hgeimN2;@NZi_e%zLCmvq(+vcx>27B-?$lolhggMmVEUw)5-5v2(!z#t2Z&cM!Tc6w;*?212YRp3W zXUxj$ba*OX zIQFDdb(a^vwHnNZ(Tq24kOCV^aOXb`y!DeU)Ddxtyc9az@!l4CzHh=B_d3KAlXOoU zxD?#^4;C%w5)6qPJw9Cm+&nHRulEa5?ALl?Odmxi>%7gK<$V0JI{ZI<$o{#M*#Lkx zSflN$wW~-;$rFOK&1gj$71SdZj>$Ama>7hT>iv2;bNaX+nSfdn#-F*iaP20$=hBgN zh5|Ek;al5hc`BNenRaBlcC;E`%V5Q%LD@|hixleB#q7+_#REpBA!!1A!sI&ahkaYl zAlvMKh7&cwWbpQUp^EA?S6OJm_%o1+ z>MlK4pGIYxIrZ0&)!J1=%koyfYgWzJ0?$L`dT7cf4y8p5I@UK2e z?f>mRIaCkO;rVqy;CTKQ{L#~OTWRJWs)QC}|6X~2Q`AP!q&*0;w zFTP3qrgfH=WjgZL#^GX%<|F$KJ+u(sIQ#C4W!?bvndgeDUw!p*!*MgwFw^5|`xC$0 z;7nVu6)Qbyi({)7^;2S=N|tcY>am{rowy}d_1Gdhxi6VP8Vptf*^@72ZaJc5I_0$e z1I;kxH0nN!>p$1Ea$?A#RBC2m2t98~d2}!;k(W}t;k|6&xPEfH6|Zii^6F#;efA(C zhkFYoI<|xS7#_ZTTK%KQ+$3!Sfqx2;_O|4cq2Y8=mGRKsL) z$73Jv&Ran9-c`t(L&x93pz{^5%T`SGFkK4t&X*^|YN54Wm7>5R{F!=(F8;>uaC-Bya$H`Ht19krZprgb7NzQp^i(+M=O0hc2Z zc0b=JI?E+X1^)NATdpw}R%@3qG}ucAH9N;-mGCR)@qdqUG%cFqUX#kra>m{0=LD?- zvhs%5IvA|^Y92)6BXNb#mvf65I7QiT27nKd7e$@@hHHgS$5sK|XuYL!*q2?K^Dq8V zgL=?hHar_N#)eCn40R1o#@%YVH7mZ#+O!V9Ke^soave^@luV1j&H&WUVU|O$3hrK& zf$+((5<0ra%I*fUQ`^L%L~<{}MzHixoRAl)XiU}>*)7=pF*wNKbz=}GTt8g-iQz3i z9!BR>AYfB=21*Y;#fY^_k3S-kXl`Z56{ZrOrPS5SIP@6%h?P5K4~Lf>rL&~rlQSep zzi-8xV?B>Bv%u-U?}jG4a~^LWO&NTiE8J8!m1JkTP^jtp4_Npq!|QY;kWh*BP$8}h zAy-b1Df-;7h+f<@W)!!N8->sUJ4%#OHW zN+q&7;?ls(qIQol{?E;ZH{Tp3pPLt{#X7OGRmz{ikEHfBE2YU@gO9JtFg}<=y~*M^ za*r^Hv2Hn@9dnB|AI#jZwhcrc#FibJPF}pQhb|rMz>nPY{pHx7tVtE{b}?~-l}PV| z$paX?0?>ctB&-)Cj&gm+XRB_yaS`ay3WDv9L&@5soCcV#Pq|!>0VKtjvKMR9*`ike z9VT~^sYF6BUfNJU5ZWX$0Zz{ay;nlQ1gD`4Cs1!ywCjJgAvLD8n0*hpL=3un(_P{4 z3d5Y*J1dkIEFh+Ns3b0T#|vl1>-b&8Uz@fxYSNzwI{{sP?uJ^aEZqjDJ6u~M=d2G! z%aeNSNQ5B{K^mE1)k=DOJgRmL#1&J48{WQ2IoDj#$PbX8SdVll4xe>Ld&Y=xygtIC zyb168Q1Ci-|+`RwoCRM>cU~ z=V0e!p1&@zHTRp^3cRiuF(x=1>|BdO zln4d*n&UadT|$`b7<31S&1L z{Q8jsc<}65++6}4s$3WrUn_VJ{+4S_K}q6I!z0;eHP}5{aLeTz?E&79xB(GvKihm&LlF9J9 z@f+s(8e=k*pgF**4}R@lA4vvs-3IdK`S0K9v__jqC^H(jc!(Ue`VR?(!^C@)v3&H`>nQY0ocy^bEmaE#iFD;9> zU%8jHp5?RnXgtd1W#*!CxqyQUuRm-C$QC$SjBe6Lk(~&3qw<*Vu2T8cMm7Ta9(($Cpal*yzwF zCsMvuA<2w!{ftNp{L>s^m!kS0x{Dct6)ZV}_9gD;EgjIQ(D0AEBij1bXbo8oxCx=^ zv?e|!_s12BdsFDZw$Vz7NYdD^XI=uLpJ2p!Y`@YGaLq5YKNhx6 z-LNH9Z;}mu$Nig~qr;h<60biIR&p^%^FT#|v!VSUKouD&NjjK(yu?=(SR9tZoFu?QI}!LMR^e_{ z(Cl6(7oZYE^e^K-yXZ|y{TpdZ0mi=$-z~R0d>CPIUG^B;nEJpE6RI}srnFrg&Rfeo zS#v4=d9GTgaiZbzTH+Qf?Zcu5G=e zcP~gZ0oQ1lVi+Fbwnfi8A8-F?EhV3F6lZA9+*=lv)+F4}A-I3VGxdFu$^B`f-kja6 zn&^{WT9$!l3x>D;f&Vrhby+*>T!GS{`r&03 zkkn>i1st(n@t(R)iRLyBafhqEu3E?W{m#vcQcjZ zr8$P0weQu$p#05+|DOfm-kt7~Y%1>X=X6*6 zi=EK2(Q%=M+Y+A^G$&)s*AP2MOH!NvuR5OyZ9bQxWt8LCFCbRtBjO)IJZ+a!*QBt5 z5Cf-dQX1evn1mvmYBD}q@4uB=TkxH$OSPgbEfeo& z;mh4c(xtsg-0<;$7>}r}o0!idf)i)sBWqUrKU#t00Ri)@QAc#jVm}J2FKyG1GlvbVpH*)yV zS>CiIu}&9V^00CQ>QCrY_)%6dI)L%PMaB|9Vh8M6hH=Ei1JyTJIG*2hoJVDu3fc=S zb}{A93^_@k}QMWEtL(oY;mq7O!uQUP(@2U5h76qE~uB&Vw)ZRs|~UZL~di`AD}(x3Odd;AhJeOC3rz zZTolICK*kzblKAzQN{@?`rhr#BM=YUj1v8-!B11MUXa{a$;>Ou+m|j%@rY8dJlF7J zUkXg8H-<-CfcG?6vJSCBORhh5B=o(i?9NsDT$I9%Btm_Il|)2WdmoiVG5(!crm)UHDJRRRiYt zrPfVM=;>8>18%@ z;RQH9N|)(uQKm9x{XEx(0ba8}E`JI4)JvgFvf1ii)8?W*WnCxV~ z31BpMLrn-Is39adr$}nKTZvDS?mi}Xwr?Y8qLB_naKN>}#nBvHG${0a1El;M^zh>> zMkepSIPxFl42K2yXQnrjy603*bao470bR+Qkkm&_#1x0!aNyX z%vG>pOilljp1`rtw~v)*@y62)&qHK*@5woDO@gG_4WaQ z|1{%K(^`J$TZB~0yODoT)7Cwg%HO|KbN^<0RoJ~_xFO*&;7}s@&1Zv7RUrX2fT091 z>W5yL^9Cx82nGqW{sA+@5dLt8JW~FQ8i651()|DtTB!~4T$!RA%idFjF?ypnI;1^z* zayPj3G|lTmAG&;jtGnTEPZ4|GWN>v%4cy};ph4hwdE*@k;8#gvrqRcDUitJsnAg6N zB?K*rcND@}Cr&Zh1IJ-RPA_8K#{4`d4j(OV7h=Q1NVPB-ipJW01<2bbxjSJa(=3t{ zTJtv}_b~jqr9AGap`MWcbX^8h+dC%oL({*1FAW)3EX7ZwLhgC;v|?`)f(Rvxq;vKo zPo;h*4U2C@j>0(^;Jvo_=|l7M{O%XCxZmBfYq5YYPi5f-n)_k$7V+b~ESAS7)O{C85-GtBS#79YXk zG3<`%I;&E_w6GBSUSm!apy!g7MDE6B0VnM50Yyy)L{)Bn0R~Vj+&oFjYBig*rv}qb zrEgXf#DloJ&h}8u+SZ^|gT|hcf-9Ow>EM0)-BY52FGNvObGqp5v_x4j3on@ zlBN*i^tm!baKnkzq++5t?qXTG(sLvNM~U6rp@q;ei*Q}xs2&8M<|&;M|-w*S?;F{5RCudP>;v@J!DO3gZp zt0N`sv*)+rcJK8u@V=JC#nWlja}60OR(i{y2D9aXjPKmr!tLw|WcHI#xf_mS;(=sT zdEKzn0lmLQz6Fs z;}|GLj(8Wc2khyPz5`LeaWV+BRdxlRk&S8=B{rE!=hrerN%90r)hnE^On>oflKnky zI^v)9fpy>;DS|NMY;vGpT!>ggewaZFW(@qSN^x}wzqY+Y9X^(8=7Zn%){%bwm4aGy zXh=6WlNp@#m_m_&tL+sa3C&6U%D~Af^bB)U7rG{u0}vr{y7pJs+WvJXtZZlrGqJ@M z;2m^-E_DW_S8>BfmIF(I^wm%yngnh2$mqMhgKlpR@1He7a^pIb9ip1>>R(7oKB+mn zQ>@$p?06CDW4)gC+dFg)5phI1^N;Muy;(@2p_e3Xu(&{zMuWV=T=m~$_Vi+P1=SDA z>k31B2cnNh#SF*RNlMzBYM9&V`6+<~-fy|?gR~@CI^i!4h%BQWQ7@!Vlk3}VwKluU z^aV5zb@7X{f0)*o0;wQCWPnRiA<86Hszq|^<5K82l2-8YGkC*jZyHu-U~havQm1# z1kDQ&$LA%VT@`OvPjd`O~l`H2K^vi4zs1F+ZRPadH&@eH;JnlA!P0#K~*cQrUv zr|AZUHe_lS@mK(*1^wZTyCftVXR&^Z+}Gyga_a>$3!t?CEwQPFu9%sd(C!Y|1eF(* zP;=;3?eKRZE5sgY$hn$E@LqArR(VjKY^HwO%sXEEm{NnMa2#}`h2_d^N+1M zP@Q7CvmMoP*)iC2#75KEk0=Sg%z@)Uk_2}X5fgHP_&}Ka0MFc*r!RBYzndI}U3jrx zA6vbEJHsX)#5cT#aE@Zi`>MG-#s{3n=*Eq&jWu#J4CK9wN8Cd33!mP|L5GrnEzZGT&8p)tL{i_If%gg~1*c|u+^<7}OY1IA-4vJq0r?8SAW0{E8un6j9ffp&$w@ zO#0Y(jwf-0rhxH{rW0UVx1z?j&+=BpTi`)KJ5=D#MIXL34_$uwTIriyAJlCJtlF3q z9I(*qYPLoBB`6&b1qGc(B#ezvjo@*QO5{$P-E-y8`#7s~fSww$dQ&xCI8f6+@VH=? zvDDb%r)h?d?}vf3U0c#mAUO4Dbb^5M3uI3=z ziGCQ#OIh@?)ITrrYdGB`E!fh+%zNFtl=XU#_xRY|S!gEEwG*)Q(IuD}I#3=Y!=ByC zx30APIJhMyjvzZ7s>VwwGi|@Qmi|Q_uY{+z`9;q|* z8(=BD+8h)j=@T_H7etLbJ!a+m;?JGu55OdavqX?TvG7I17Fi)r zN2&UjXUEgRUp*cJE)wgkh>1}FDzTA%_X&}Xq-@15S88LHm&<;5t5#tmsR;f@Ox!{E z;WLWk>e>dXMi9~0g?+=>h3cWO&o8$Jo)b#3{2@ZrYvP96%n2rmU;~CBo;(-8IvuI0 z*zI~}CsroNpKg$JcqGj!(QOF%sqwxM59^9vKb#s%?7=I*pd@=h1-`VrYMZD^{QY5@ zgyhJ0GazR}d2jWLOex>Uxj(qP?ktp?1s}xXgKdGVy_}mjJ4s)?(Dr7@v?w@?1l@>v z#PsRBGK3~KY=x#3BD+lzYGoi97sppK+Z;FY-+P2Kt)J#x@Jl$b(4zXlDx-Cu=)uPR z&tQO`b32roU%5sZm47(5I3ChfODIbSAW~jyi{PCS%}tDGPZeg{#%pDN2&9;Tn2Z)R zLEJ;RTg5yoCBl zBapN~^W@hDYulF|mS_}C#wTg8iFQH@SCi8zS4(~)cl>~Ew zh~t`B(}tVL+8bQ%8$*rw!!2%#{CAaqUoZs~W76PZ(C9lvyQFU9#w%~*X=nzpVGB3| z9fYP5gAuPHBJd^Gk&0s-bupq|1gYZ0=i4&9`vmrtb=W?z6!C~C>~e0ZKug-9{+;mp zmk99*k$qvFY;Qv6@MSm~wL!A|X*1B>M-s>QV>x& zQ$e)$@w(?KKB;xJ-<2G>AqzJ%tSR4$yi4%X!yol+iazJwK9%SPAc*^+sxJYz?O((Y zmRVo?+Ep}#*{}?2ag<8Kpn_+m@Ao&HdPvX2DDY#cS*cVWXxpZLHLNJ063bJ|jc|JR zC5MKPDxD-GP$e-pOm~Zj!9hD9{|*muo{xy6Tt{e$lsm;5yVCGhR?+vH2{Nbm78r&*RN!=8f*(+A{s@(5tu7R7DqlT*W{B zD;$LUTPL&QM~-hJUq%U+$B($G{O%Ep$KCNr#pD>Ko11<5Dxpva)7}#BQnl<0T6ncB zo6C>fcf-!_eB06+l{c#Do%b8r346?qI-J(B>^qjvGXa@I@k>u=MqJCAH2BOYZCTa z8iFz2%9|VY$Xzk0dAgXe>{B4y$Y=bzLF^gm%|7fR z9D3H3o6!~Wsum?g7=ym=VhcsQ`rd_@cZ~Y4=4hDJpm zp(ydQV9C!*Zk<2?E9?ZIK8ZGgIooNqSMEtbiyVlW^n<%TU5V~FMNX{8pQ#$~p36{8 z0IK#8Z709QG4NqgnzF44XjJ!%!YFd~zl<(7&U_sNEa*pnd&ZPy;{ANgLt@Or1)m^w&G|>Tu-v+5S>aI(d<3r%LiDxt2w<| zefR_=0njtH!5aUS#BnpCwF39`hdTW=Ur<*{9t;0?eo$%6@NNXyQic2>eS*m7U5{*e zcxqA*afL z@^^N0L|bLWPCld-typ%=`*?d{dFGcV6)@)NvqAc?6w0p~lD?WWC4XF$VRFZkQH%N@ zeP$#}ZSNMY z-2u~o_~ac-N3%ay8FQCVdL(&L;nR-24J3LAN3OUewvDtW1m%Vfi=`w$C$mV-B741r zn?YTu4>ZQKpZ`Fkip+2IHdlKeR-A`#O3#IzHXBRBz$%!g0~|$ zp%I-X(yfhd`rv3<8}Va04Vh5EBi7eY|kz=1S}!bAy89Ja{zwW-dUfovkO?vM8MY(<6252W-U zspUVLEU}3Dxyx~StloG09wJNx{W;7$CuJoc@EdBepF6WnD`E4%#DbWj7QMbI(wb%_ zxa==|f&s}m6GH;^OndhT&RLuvG!85Pu{o?h7U>WHHc-RpvzrMo0?w-iRj1EYfyCl; z>u)6+X*cLTseRV#WEf2w^aO?41+b)Hx@mKKv6;8~J`d7`gNjSPzwvqVL`OE@lgy8s zCL-MXuU!>1$Xc;f!WTz>zwzD2{(CLQx%ut3x^D5~z6Wy}CNH|}#=dzJ@+f@WVoUtB z@q5B|Gotl0>^fZq>&E4Pt$pBQdl>bz8#pgUJAfkb^ayb5!QJ^(=v*Aq_xHHvP*0X@4CY^9Y4};zp)Gf=W=tF% z7^V}#e~5Zd*gGr57M<9SNtDVAA{-1q7-Qqf$#g>h*3ro@o;u=2H%x|hPTk~yIBn;^ zfgR*5K6wxA-{)plO4?Ll#^o~n zggn5gRa03T^7Fj@Jnq;OIlcovY0W?VvxC?KuDr@7jknue^Z_%Atp86a;RHeM%)<|& z!GcH57{1JRHq$JBZ-SjdblMVwVu7ly%}deIQ6$gPi3@8rkHC`_*;`_uUuC_?&+;ihls)IEh(@>#F+nOFJ`US|?nm)-cRvbB z5<1=bYW`v)+r6fI-QL9InQ;WwLj`G~#}&vo5^*OeTJ)Zj*DkReR-i5 zVd@$TZl?seU!BZrI6OoC7mJj57{<8{#?4j-VQwBbu`jK62Lb%?m2~j-S2yui_Xu{D zSL@&LN(-Ds;lLKd4CRl=FHKaY27eEDhkhbA%8kzM#!i*2h0Y%U^j_ULYf#}B=_*%G zJl68bR>_H0X(w#G>SC=QfYrF6=cjCuUJ^UPSHm!3Z8gs{U-32<$a zm-w5v4AUoV09F4{CV9)gza9gH^b~-`Kk(40X-0@hrrY}sI$U}ReO{1@8{{{$2VWgw zeJ;eVH%I|za8wt0JvWt6((U8uA-ww$T5C!D4N0-hbO=W;;ejW+wY6@35vj4^4Q}Es z+i0yBQmcy~lBnwqto<%b-Mt!mqH5*;9Tq_3gR|H6;7P#}v?n)*3f?c2Qmc7^Mvtct zB8^UNGQBJ~rD&<%LFTEPV$}9X1M2C&U;ifnJC3M?p?|RCM_k8YSEinmM%~pPU9ov{ zbTSeR@;k|072$O$Al=PKuatB1&nhrzeTP`3?>sdweJrUP|&8 zeqw|3jwLbUKa31Z&ZprW#S`kIQXZ00Oa9~CSVaEx6H&gUS*P^d178Hr`*xJWh0$Lf z_v{(qcdjfdvK%bJeuz+6tICOPkH^VlE2(pOvZ~R8IYoH&=yDOlZky`nmCC1pTMLr6 z{>}K7=T1pF!O7RNkav@)pb~-(IR`AqqK5hw?=ZuCV9fF>VB67DwT1s)QESz2+;$|l z|G^SBq@>`VKb-QZ^K@z@8I4P>=er>e5Z;??Gc7D0wI>5>Q8+lNeg%r>FX;ndCIN`2 zPEtu5UmP)!d?~xVWohR4K12|nHw|)>T#e-_Y{xfP3%~DLKir=EoK%A zy)t};ZVG*?fSdTXtri+_oGD+IvGp#{Nv#L9^Svh+e*Ex_dYt~7&c;UAgpIgC`&0oVW1$ye>r*&Wie#ssa=rTQ@J&RMzs7oBKJ+ERU@)6tY!GzZL&tGap*J{fp=Rvfw9!1N6W zC!5*Z$lX9um*A71JNP$^Los!My&W-4q)8;mwR_eY$(PYPF9hrb${bZ-uDf?sJo_E0uP&I= zrjOoSwp}v9F6zCiRKsqCYL32k>;}3yqs?%q!iqno-d1)ADH5m!ani&~si6!Pb-bdb zBtLQuj7uxXi^?1~!epeb$ZW9wRDn|Ds#Oqv1AE=JC|91G#$xlf?C0;rQUU$~os z(un^B?t!1vGsi>eMw*PH!Q*ug7TSs+s$CBZ45+RttIqoI6$6k|otMlo$PE*T~3tjuzTGQ|AwM&J6 zADWcUWHri=dsw;qT3;=jFe{A#h!T}g1m>lV-wgqN2Xi`+E8Db}@)sfVG7t&uIx9SA zHl^hUq;AxF>wu1m%B;z`eUGQzrp<3>3Lt5czj6m=V<>fO}siln=5NFjmZT9e#O9I!Ps~6Iz!SUzk|JWLhDu z!h&788u~7|%{4Z+EdS;U%QR078j*jH?T(x@GI54Z4?#WtDIrft*Pdnhj#$U+^r?=k z|1_n_dZD1owLQQWUl8_4{C6*(L~QjvYm8~ag6)U3_7ZsB;E6Bkt^wCQKzh?mwWG0< z?3E_vtCoI!TP4}}DNp42g@h%h+NF7B#&l}!I5?D6<=_28|8*(Izb?T*2`&HM-#NbM zt4+%UeH<7|pNjN<;d`a&#QLGl#T?*zW3ZdXa*k02!*Yq?g2sB8I+F84iScW^J&n7( z_Dvmf?&>)F)+yxl%%sCv%BS~m#UDg$B%T`}2sFNH__o>2;D~@#>E~duJ}{WKX+4JN8Ih#yLrQ1 z3kXONf{!TD=Pb@yf8O0u$bBU(yjf9i)!7vVx|Hh{6e8#(&5OQa+FWxEpH33ZGqB3C zBx&<9@yy%jXXu}x#L0A7du^Pv|6_zG+@rMZHTC1I#$nL;hlY zki;>^Z-DEzIHC3X{`db0&eiu}(bJFrY=Y`uqtb~*4SLizOk9O3Z5`-zuGTnoHz_CN zz{tG`)C*l#KJ>nL#p$nZD_qcE{Y7#aAM)L-YlLD=2XlnZkb8Xd#Ig9@oon!Wu(5?Z zXcCC1dtn+MvVwYqwty`S36q^431#;nM>y>W0q`(NM@cS|EldWI9hE+Z|)LGMqYd?Fa49e3_>hLXzlD3CSEJ;XUP$2XbH%Vbz5v&zvSPz!vgSIMdutIL3J}?7#R=OSxw_o`@1Y=!d=P<{vjPY z&^QC0bV0ybY93R^b8mUMa09k}E|RaUM^@MGKGRhH#u6j_W1&OrTU|5dN4Y@15_)HT zM!D68{@W7|<#?-!lM?;o9QxH#*5NOiG{|r3Lx?YVal_kyKOx~kh;L^q%O#b~@75wg zn?vq?gF?>KRYWV}W!A-h-E>ifZvXxQc}LK)cG;->VNSo@5`*+y$*)leJS(M*^v9TNShsUI^tUcv2$d1 zu65^UUAi5@Gr59 zpy?5E=QeJBdD)KI`bI91$bIp(?{8MD>|%Ah#ZzKt65FAk6%x;3rgo?3g93-{>Uu8uEQj+SD0>ZXBGhI7x~ zX5W^(X5C7AvmZs$(oPQgubX|bJ&<1rUl+gde93nkm30_7f@<5lH#>Ik&cpYeSi@|g zeZqS-S}&mW=&3)FpZ2n5So6hCHju8FRLGTOel1G0a|i<$x3k)LqDeOxp$(eecogub z6^Ci-W}YN7lNbkqX;4b4CLuH9L*KyS2d>maPuw?$1WF16JCe}UCnsn3LS55GwqUem zGs)D)RR=I>8`=`2kM()#vSu}hu6q4gY4Zs4QM+U7uT?gEX~!h+?epoXinx;i`Qlw` zvT5|8=HM2P_)&El=b&tNhEEInHgMe2A9*-1$X;<%l^Ug<6SO2@xGG0?LdC-q;Oo5e zaAKQ(Z|ut+b!ZUfndo7x;Lm;I9Xh8Ly`H|iUfJHqyw9yAztFM!j2P8X9Qs_sFXvX&h_*JIRL`{XH~mw#&@T8JHB}LXHjQ zjPk%T#7QeIkS1b`R9)3zr%Sp8>x6RdZ$dkZyH`@CNYH$jCG_Tt$;djUKt~)aUg-mw z-V;F1NgPks9ZRK+BM$}{w^uK}@VUW=!n&iGNIeLZ4$bx(FwFmfSt3-VUHe;-n1ZS~ z&7(Gk1}P5{CuPUpJ2NI*k(=C^ilGNhx$ql^lxFI(h|vC} z+e-C|LG2$Dtie<}FPtufy5Q`G-Ee~_H&3$wy#9zPbEb3;YsEl_crZO3fQ}Q^67ZFO zidmKdsQ|3R$fh}%C3pmBZ$Gn1mWJ&__nmCOuM|p#2UQ86wiPgeo~HF zLhV2I$7p2r%7&4(2*m+JLcu8{WhCKM$vVR;LBY(2KEOD!r`O^)QIAT2Fn{a1&iPXgD5h2b)(HvD}H=oGJtJ@goUw6||YK{oLzE_0@+_eHwJu!mPQLx(WU8 z`CSQdUMyZmaKDRGe^!RB756_)3IHc+3*I$qsi+n!5 z9@ANIYm&+WX$>k$*z13e`bDb8gBzFS|K4m`2%rj9=1f3;Vo<1%ZSD~UMcxfya%A9F zJ@@B3VYAEKzc~`l;1pqO_8s1vXO`V-<$!VHF%sp{|IAv0@7+>J7S1^_s83)1MdA(1 z5WZ(@JBYS!w2J4JAO;CbpUZ?0rVBEz9X`gG!g0!4&HxaM9$1=Zw%Ztv7cV$^xQ*;O3Rt~}2*>U?zx*J%-#FWet@Yx)DDU1$)VvX!SuZV(t$Uuj%NI0yz&a^pHKSBe z+C6AYkJsP+*<{)2C1@O{e3yG1X3sY3)c48?_dRFf@v=Rr#P*U~VnO_#A*Z0?L-?nO zTgl(7ubxLFf8aiI0eLz1cZ>6u?g$CsjHLN!F7O>WKC z9A2z+{DN^qI=s<_KXl!cahU5&J8*ca#eK^^0*P`iVZlM8K%~yYc z*p3-g-Bf&avIC)=B-kE&pn#6JxT%?;eS|{#_^@p?myYK8LmQoiT;))**HflQ4G7ub z%qDiGA?e@HB6;=-z4vExm6Uj;dbDm6A0gsFgul^t|L(c&uo^hu;k$76U*yCPqmUTJ z`2{Qdt)_xlkYBWwe~=G9*h|;;qra`~h1`kmQ!A%lP7UTpZbcFgYvbySmP?q5B&u3> zqL3tU)-rJAcaGp>7~a&5~H}(UV{5#L!FR}?g|k;2KLD0Y2O?s za#KDPkkK0}p#=ikGlz@cVekZp`@?X8_onK@i+@1?$cF2S zVY6uDrX?e8`iqR9>lMCXXRUd!wK4y7O=cLx;Y55BhURmQ#FjnLQ9+~~r=)@Qy4fDI zRt9j8{2}Cl4G={Z7EIsk>>9R$E%WdPX2YMk!j?m}5clsF3Q!4Pt?^ntYF=7QP|wQf zg#P*G=WnW6d{_+Zva6jJhhn^%-Qr=bL$!U4U(`B8I z@0^-Y8|u?CaI%rwL0`QrwYx8eA`hYyDq!I^l)yc|S_F*C`_8%S$OGHRubb3+o(7^ z$b%87`OKU1Py!~(xbhv1Oah3NW)+@0)|-xyZ%cNnkKu46&9E|veBy!i423#M#rFb`LX9xJYIC%O3D=WIiJTfyLA}?^ zv45>#HtVxiWq=|fkW0bcdD?(R*slICBLrgD8B&!P=doJ^e4QK0)nBc&9ilY%kna-D z>sKq85DHn{F7$975_-6|A5e&CJ2LOnSng1cAZ8!HA63XPm~vQ|`-0%}nUt||ImW8g zOl>)kOo}D0d#^gTkT{wquES8IISZ|Iez=iYa!cL+9_f7J1lJZfW$5z0yqGG`i}5ay z6w7Z*>=}U&>nxDeHqO8qY|Gl(yhxxUkw2I;O->JPkb3eeJ9fW?oDr(QaCf8*x6l=P z^lU_q?dg-%?!VFe>_!mC@_E^d0S3{{`lYOX?z!#iEZ1Z^ZUl7vbD0}Eg$Q0PGu8Z# z1J@3K+Q%+fr5KkQYJ2N&q&vFj-a6)12$2=ckxr@`2O;4*dtZ6&0E>-*PEr-eK$*i7 zF0)j__Orwl3`^nZl8N?b|H=a9Vl_xS?=j*JIiQILCL(E%#s?jH9?Y)YH%ASh5>;xh zaq!~We{62S#%ix~&r7d6jJLZm!e9LK6~xbQr1x|@iU9y?F{Itwo|&7uaYsV~uVO_S z#H4;#z}5{zT+VJ9^r_ZYSE2``Kt9J5_-0{Fj=%b7cc#O|G~uQ4TbCH-DWfyAw)&RD zubrTygL%!h?mkBk-4}G9rWywfmc9$N^_U92Ru^_IM!T+4-wztxrw+AUpiUaL1#F2! zSoBZFU{^cip%7B8_Yy+~9C%h{ys~DchWzNXpDftvW=?sXjYLpr%b8l?zmxhsI$byw zuZ6P?dJG@%qzYTRD4Kj7{acpfTU8N4*Kvze#Bb-Q&~jz)g?E4rQGib(caOz+iSpC5 z45*nCGOGQYZz*_AC?Rx4O!L)r9DLToqizYlaQ<#t*VQTgh4nX$K~ELb zVu`hjrKPTv4Q2RNA7C)<2XmwtY;eCqar#Qhy)S&U$oqZByJD{!b6y|tNqSx%`N>I% zA6;os?u63QKSVq-?I^w=E-%*JtFb;>^6DVA?qy@mD5h2Lb4xay5AI&@%;1c1L67^E z1FTY-^HN$y8E~gu82LFoz$CLB_eK4k==DdP3cDSS&2-%0OMy?W%~a#AhIf=>rhl=< z#~$=pOiHiNQ7d4m(hG+SJzkB~qHIjJ?6xu6>PEQVp4$iZLXd9UX09y&#A4 zPA#d#p-p{~ev&^KyHLCHxjsvfUC4kJ2|uZ?Ape382?N%t&z?xwQdb!n_jPffG+T1f zH0X#FNIV29ruMXUgSBWq7T9vI$__f2vS1KGMUD;_gs;f9A>sWqQ8^0TkeQZX%%#Gs z*4v)f#+l&Vd7=#6Gbw7N>{BY>OUkg>lUJU3sQ}4m@PY3VJH>3~-F+Yf)9EY7kU;}4 z&U2lP%foX2&1M(44h}Ai9R&T}@b}Z`f}{b8#B@Yw3`(kq+r#qLGvn5kV-64v@to6Y zjHUOu68_DVen9+;Yus~QB*UXu9YWo8zDo&+us_Qe7oc%i?%&*H8{f2^8wtVhm9)`Q zVv)FYAjEHL`gY-2!i|ki42Y6$*hUOjM-fCaM7;v+^!1ZNRT=xX)EwN#wEVNZ`DR@r z@?;3wz|aA_4g5Tf0he4NO7TS|*GfbNberg_I!VjgcoQbmpv|s<{HSG61adDkjXsb7 z+F$c$pcDODo8w$IM9;j4mb~^t2DG`Cabs~4VGAQZ9b(3Lo>*zx@8%O18wkIn9VdxS zlAq6LQz+ja3y1BLW%aQ0sH>*{ygnV~V6?@vR*3(iB2J7S`ngw<{P&3A)nT#{I~x@% zydPim}+$ zw5pEKpwRs{_p&(KPOwYFH2C3ON%>4o;2?88O;iPvO(S1j?@0E$CN7mCc7$8M`ux@g zq--x>o*-=d5D=LY{BvHj;++f(NZYp1f-Qj-u(=XNj5BmIsx0pIrx^o_uGgd9znE|z%@b>cA?C%5o`e~8w zwC`h*CmmXIGceV-N&8{pZqa zO~eCz1_nVSVGhcY_*e!A2a40w9et+4nZ!)GqYR%yotBZc@tx#F6aV95hyZYmbpT_< z8sc2=m1t{2Tml4LIrK{4j&3D5S{*F(!>Rx6`|v1q^<42e9PQKC?+2X>p1>}f@Ov}M zUfb(FXDzP|Auwz!5E~4>_GFBBwqm?*Me0%8r&>Sxy9d5`PXSKZ<80|qz+><~2>+7_ zRm6W$l|~25{L^f@g0|$m%dI`f!jer+-eJ?Xooy`iTH=ca`}sQdVJ&XSVzH>yp;Lh> zltn11ho6SiW)nI0?e~MHyo?!rs&pd(p!p}r7|qDR2L46Umy?yFl7FROfT1gREQscd zg`r*ANQ8Tu8Zni1>$O} z6S;8(W6eauR5=f+f?sB5?IPR6o-)olx|USv9h1LB%JjG+TOk7rrb(@!86lO`hnPU; zR?5I&g`}Yr=Xk{~iOlR!aO_$h$jIzbG$%Uk-Pv2Z=@7rYL))13+Xe8?ml@Hf=I_r9 zp+s8O)4?iRtU!2Is|fL>@vT*U6^GozFb&?W zh5r^6SJUoLyI{QF^|*_van#XzUs`+j2DyV|o9}X!IY%?nek{2DpNBLot#R)`Zi}mG zFLDs730HeZ3Y?;<+kh@^r1g{hnM1~%HRqI%pv5u)BBDrX&~(6&mDd5NypvKQny!2Rw*yFvKL<(x9Q}U~urFE`n~+6YuOc8}z=;&}A|~ z?81*w#jyZ7=gumGK1%{gkcrz={gNWMwDM2nSvO|Z9dZ3>H;C3MRdBC5WK*mU)3zf# z0;2<F6J*gAq#MYJ9)Uw^8<``S^8Gx&(JZ8>>*hxfKN0%+?q7K}P3`y`Wy{hNkU=kf(JQ>XAX;-OD^pa@1;+beuUcOHZJu52JmX3~BK zYcif!Ixr>h^ye2Fsa``AI=O=@0*(i1)ez7Pt9V`k?O`}s!;1s));6HMlfp-hti-?MM ze`r~OPDr^3I4}Setc6Qtuax2iQvXTh&^|KqrMg!&lC`?1UyQq4ips%9bWjbp2A&g0 z78;n??S1b zx?Blc%W}#JL?3zuuLPmeDF)$YRtwTVKBY7iGXUnN*kJuj1+KfO~Hg8{!-5QmYx_#)jhSy7StJ z-Sytt)56_nw6-|^el)r0zU6q$&!AE4umZ*c3@Jj5XPuFM%@<`>cs{3xz>6qy_KqK2 zbGLl3{hivkcg>MHa6wKCXc71{GKB;4mbFdig(Zmyf*%+GDe|ld*Qc1vXb7n_;ECF?_ABI{G*&O6c3IxYpS8)H0PIB9$kMRJ zp4q|M)n4ZpupJB_7 z-~8@fAV}+Q_BIY&77A-4_rwBGFv}kWZiPsB027YR5?b1Rl^Ly+UI|TrP{%HX4hLbj z1{X3W&(&$HrPfP_N;Z-Qq-0fguskwM6NPz}wR{!OOw|5-{n=BGu<@s$(zjwJ^+F5t zzZIg-?}M1VP1F2xQOY;4@#e)g z3Ywe@Z_)z;b0@wTtDZ!&@Y1hF;l5^XrHAGar2Wz?yW(~Y$0r3ARc`Bqbdn0E3(=5c z-?$r(y=5j3M;XEV$nB~7H%*8~+Aj~&xmv$n$YzY1V;K(QPhCN)%6GbNL}*Xw7uuhD zB9?LLn@>|^yHtU{%J!!E!n$nzFh9}zLWExGlf}AFybzKK5jk0pm7R>3N&0^lK+Vc^ zSV|rqUc^g(NF9GjxiC=iHi3#90mTP&iF-iq7DOt-nz^vLSA&6zxXB+h-qFT`$a+*nu>6jxMLF7(TnkaEl8+7gdt=i>MgY(LFP?7YB^|(C9ivb=G9M zpguz=^>`)Z{Y+*;y%il(DtjK&4cv{ySBQQYgW$>x5htaEcy0%6W6ip*gch_R7=+(K zlGXh+$mL9gI(9!~TpqTv?wobPpzm4lTcNrQJ}wUN&-MjE9dW_O>ZqogO#d_aDS_B8 z){9?H-@3uq7CL#19|;<{3-jezELaEd)=8R{UZ{^qHZ4`0HH2v$ZssHChDaI)8i;a> zS9uUsR~B;c*FTip#cv#_!XUe)cUJTMQ@j658Q!@6SVKhK~4fVbbG3-m(5Vng@Hqs`~=r(;ZS4z$LUn4i8P2E7jo)sz)Y&rNT8K!o7 zAn*=d;6d`*T6{@`#>wY;eijj|S3>X-`Wt0Jlz8o!$~dae=~AE2`L@amGuo@Ck3clh z$}r%L>A7+H&meye>>*+w4@VugK}gIW-W_8I+>(6V=W2N?)iOBbTkC3q9aZc^eOn?P ziRIq+$J8S?c>hgvxZPzz9#vWYvZ_CjiFygg_W|iE4hiIb{RK=U*8iV1gcf6mu!D!k z(JS9kLmPo?-=ECZAM8~ipA<~3c2dBY=ZUTN+QjyK(h!VR(1E1T3#>ZL%&(Au%L)3$ zMK87pduudrT9IIe21PE?qd?^O`o*=^%9xqQKF!sKBbYY$19+d1XQ`?B81iVVc?VZW z$n6`~KO=gTqc%h9ulem0ZT-J5CB2l%e~(A3rd70Gk9&5SvG(pWPPdT*WJh-=uWG!T zU+TqPeYRpjjhT0SS;0+Bv_V zKrrLgS}2FS)_0;qXPFH<696~dUy%c1kQ z9;0(CK*xUtEEWz1K{>xyw@WOj)(F69YGmhwff4v!INO4E3AwMX^PMf+cY~En*RuaU zY~@N_3dv-$m{^nD|MFMFqsEpr5+}?t61i?+gIl!*yqT!8jS$I(2=*r1Ccu-lA5>f}iF#<@KKW z*HnDH_^aa-=j99Q{bR4pLe3gUSv7kujn_pK3RSEYu5^XfOeVmEHqb21>qHVKra~~E ztjw%~nc*+4Mz`@Ys!?@1eVUgvnD#2flKXV4;;jU*Y;wTf>?_Z!(q~i|{Zv{i9$Gz8 z)SeW#1`XjEtDT7vO`#SKqqcFXI0uJjp0gky1X7m&G*?gEp1P!zQrlJt>>wX?3(oZ< zVSt!~;V-xid%i!n=ZsXP#ZV%Aw_9!yZ%YQEU55Sb0ux_Lq8U%%&!@Kr+keDH=HA_Z z97HBVz9Adwf=NTbT#r?V(pP#Q%<@oX1N9K<<8~8Xpa+-x-aB0$ZAT>iB!tD^4SDY` zg?9E>k0~nYNcgoF^#fj)Se&cRqnoGO`rpi}D{iauCd9R&9qKb-@Jq{fC-EYiZ$tG0 zPjsbIs)r`_s>1E_J}w(y_oH^)yz#(H2Ili5uhHbm5#cbUJz48jwZ&2^G!pv3T%;P~ zZHQvUOa2H8_290M{QA@_zeF-*gAbXRT@FifzO#$Pg+8+ z{48H^i<7BC-B#SFyL2B+T(&*6@*nIo%Nmj9(~$0WdF$VPH8;wWBl-ucH0$nrb`XAJ zSOj#~YLO$Tw#w9*=pGM@LblfrDvHnERrfg=wKTUb%}s%rH2wlT;BNc6_CC#(6f1jC z@>THt3ueNMJL%Al6t>Ng-uUfLy!(fRiF|sT9HpN*O&*@NJ`QCcSvYM%|1C2)cZxO5 zn0_{cdIr4Glm#gMgEuSKyk44xd^21(@XqDnTKC|-&|H-~|KZDQoW3gP2OM9JwmArQ zmYzL_v;w~9q8gFJ&l$D)n1m#33Z{X(L~O1ac-^^fJwC7HcQ4`&G2*$+@2&DB8wRkzJlE(OiAg-9Mn;_Mh`wQ{H zJv^lw18h;;9b-cUZL!Wh$3P_?AbC#rwsWQ;bK!v@Q7XRF^Xs2e??&Jv8Iw4JRue0E ztu;Kmn>S1$X{aGVM)riPywu{yf;rQJuc-_M|zb@ zyy#n&@{H%0X=Xk79$T7YPBIS%6Y57$862RC`u`C z!mIeH%RjeC{n~q~ z`Oc zaTjuHx@Goln(f5TRtr!;oFDfR9Nhcg=}UZ2T1x;aHYv_K)GZ|MWrG;UYjb?X6j8sG zw567Y{^mT+cB}0menkKFSw1;N1Xf@=w89R=x$7VKDav{+V*x@ja+a+%Ufu`E)nD;p zA3}A*y9;1lvah-2JRi~s-i5qPHX%E-viTa_$je3F`6DjHOcAYbTxA4$<$UVC^|j4} zomXlH5XT=(r{x|hAC2rjB4hAr&mWai|CK_Kfpl9(`s1I=E3;4D^omp`3&QWlglxYR z$SAT5q}@0134cTT5>@xgb=fr2j{AYFn$ky>@Yhd7EYZe2-_5@pKB^C>O^dm%qXE=g zNx$XGp7GbbZ1cOwmWGi}Y&&QBBe!?*!EefEIdC#O^Km32gK~S)AG>4L9Uh&ewm;)a zXAa8OtH0@Bh>y?HI1GPP@jI=-SQ%|Q6_1r}Wn=&+tFa^T60V6?*2lySCRoppU*_eS zjzePxMrC&;lg}=C2Y2clzMjuMR;&Dj@r+{yO&}S*7M@wZtRxj&#_W*@+PxOzG04|1 z(e&u-YY@F|FY&^s(l6u{Ss?bUm-PeN9EGOWMkCwHSM+JzTC^fvhoFEUFkuw|0g8Q< z3hmS*EXS8xm3&yW=9c4IHajoiS}jBztNS>j)>}>>U?OjYvxZSnzcOndYzm*uLbnR$ zT^C(WBt+T(>mN2G;=4qepop%7r$hmeQ;7dPS{PuGU9t9awn}Oxs#s~v6S3VhaiKnV3X`qw@n$9qBw_;xwOGW5amQ1!vTN2`2**0?MfHqP1Tb= zCuNi;Mk^Aq7}{lYyR;DbESWnXZxuB&LUY-9lmBc5H7bRJM1%L(>(8{Uo8u~O^j~|2 z)+liu{qX&}c;l6$hf)dQtGt?dHmv4yyv1=sGM!y@!#GpVYM}%16r>_rM-!&uo6R#2 z@Od-gl=sc^i)BFH?QzoTya>@3A>6nV_ga547JTUi{I^Wab&W?m()8JqexSM|Ddn~? z05tPafn>t;+pi6>*yz=VZ&Yx;y%}ouxA&E4XH5D_9A3G1q+MnQ{8wB%w}iOzE*4Mx zcS=|Q2N41ANPXBc{Qq8zdss`xYtIOxHT&^0<>Yd?NOfATr+U+_FFIblXtBKN)(`96GBKC z%UvEf>^gfzW=w5^KSbELAGxAZnHoY*?E~`+9)ux^agnN~!-M3V=xyTBi#vnVKk$GT zsd>=tuZ^0#XTbuAt6S$Kc+m1s(%Ab?e;8_WTO z{4DXUI^o<%(ZdHXyhs$y4V1C@C3jk1cs2L7_7w+gRUyJO9r97Qu+R&Z!n`C@xQa?H zcVgK4ezO(;Tqsb5z-h}v(;76Q7_{~98bFN~+r0&eF_hV>$_Kn zY=!6?JT?L^w3Y5y^np&sMkk_T=7cG`s3zY`H@~^P=tyin=et8n_A5>T)|)soDWUac z@-On}2k2xyI^e?1e1%_$9nUd9az*{t23X@CxaH;B@uEaM?d3vZ0!z$$F0DH(_Xdgy zeH*`$K&>lZ@piBv$%Ooki3XK-!&uMM#dnc*(j!w#9^9g*9#IP$*x%j3{dyE8hclID zq|##B4DZ1Sl@Fw-52+yxg17N~6@{-DvLn3N3AiEo>H*V#<&@?wUl0xeg zSNnQoUgE!})m}8Zj+3$ZM%I8>mmL2$J~MlFak@M$?5vvE>hDre8BVwI z3J}q78wIbvl^g zYzr&h%buSfcjo6Iqd<@7+eyU)csQdPZc?;FRtiX0V9 zL|qeBcXcJbZ|;b3K6~|Jlbyz=@c5V)Lpss zymw23bdfjX#@c^dnLT@a?VloM{#xyW8zwJxw@*2bp|UK`WF!^{V~T;e)LKl7`H8VP#&*QPK*S zbMOrR1tCTNWmw~ko$#c_7K2clKEko`-0tP=WcOB1t*pWMuk;ylnT#&yV7F1Cg4&O% z&pG4va@I*|3jMgfBKD#@?X04LX9B6>(U+bPjx2&*!Bm$|)uwETwegHMJ)m5kM8Pv4 zo=<6#lW@77{}S;4n@w5Jy2;f`$3A*LktgGo($qn*U~IG4RzmmWlufbxp6^^a+S}!lL zKJ&1t;n!Q?c#o+1M}w-#n{UL(KKW(RpzJ< z>6vub;dE1~8NIk%y7*bZ1HxAuluGqu?m-+_G!E%PSGor1ymxgJxdwDJ^nUh8MD>jZ z+``gfXvlA2y+4wSd-Zll`~D)aLzrxp?S;12qZf07_TLHK#Z|$1detdOvm6cbwhH#Pe^x z4RN5Z9QT`QA^z%QTh+Qu!_uA72(r8~qn98ue)QpV3So^CL?z~*FPpF3sq;(t>6-

    4io;td%r!SCl_(W1jGlQouL7<=W zelgy{A|+|j@4F+~dKSrn_g~s=ZO!AN_yXe6D9I2YMx>EHG6h=na z^g{H}B46F80l9H{a*?zlx|Z_>8S`LT9eBew9;xKN}@A#VK<3 zeASXi&Dz}+%caYphcloNlvl`{)K{lhq>|BOH!j8`9{Y6!MMld%P9=dAl4(1{TSC#G zD8;0!h(N&|?u3JMo9Z9M2Cq&K*BZCyG)}A}HShQ`DPu{7$rCT>!jz?`-!R|qnG;mX z3vSsE0a!;X^;~SMbtFh}Et(C>*wanJg=tx{G6HJ#7@G_?y#GO{|6#$^JEG89#p(T~ zW`}&dY7yA)nEU>#f|^$Hu;M@rv&@+QfY458W7V? zY_Xr7O!}~T-l=kW1R~xrdwuT4V+DHO8{z$%Vr1Lk**JcN@5bl1zXC}q`^@K}Rml*R zHl$-Z;nh#`Jzuoji{XK{O|l^?Mo+Gw>Rczu)|H>|^&&81VYLXZw`*}frjh3O>H2iI z=wszgak{dW)=al{Eb_am3xJk)E?!RXV)|vd`8RLg8lQe8xHdxz!i#+kx%i*k9U*Dnn`TP+BXULh9N|XmZ`*5~U+16{{5X?z) zq&5(<-d||Sv+aADiLJ!LK=8pY3ss_S!} zIjH|`W--0{&yZ0&kg+rK9;rKwTqnRE$z6^b>9A=KYoRChP;fqcO789T@*2|Jwe7uWZ=&tpH0L=m*!>5(0Gy4V4W-YmDC_rNb0 zecJ?igo!Jz6t+C~){(wmK2BYN6Xn4n3b0a4hWO<4v9Ow36;>`V2l`kkld~mt9 zuk=(}{U^h?@BhYM+{Teug;kyqb9Nu~nSHc0VNcAu9@S>)bwk|P#${}@e4#Qtc8L1n zQ~iPDzgf^Q;#4t=R3cMyfc9n{OVx&5MsM|jZOkrrKFScFy+bQ#*YvJmu>Vau_^;39 z%HvqojT=ADDY|injw=-2Fj8r}AQdvoCXpfWQ?G@5gR_LQB%P-!xNhPKhlZCsW1N`M@x&l17v)H}{S?u#Gq_Pi2zD9HPW z_yM<^SJ-_2t0|^{H;wOgW0&01N72{o$I5&&{dRBC-hTfjavOvOJ)SUB)dFDQm=x#i z920k!>v_>8Ji!CfZ5lqtSQgI0ybMIZz8)VV$4wf~QlSdE-@R}ZS3#E)?>G<{)If7n z9W4bw3Zuwe-A}reF*)Q2qpDwDt5|BVUy!QjS|6siRc{E2!qO4gBqTck@EPG1@_LPJ zUN%~h9sLgP-)f`;IDUz|3OyR^9uqUAw;ldlHhcg9)YI8vO{vdD0Yie}u#I<2+_Sp2riJ*9T4<>)xF4|cFmLTctnMt%i@GR|pblKC(6?bp?YrK4Wn zsDEKGee{pf96`{IfyZu&LAl&M`F)N_>IFC82Mi+?;3J>6AbO#>mcKV#N}KIY;_5XH zlR+Li$s*B!CT_%SdIy&jF%sauKgsAVLExoOmO+vOy#XXxLoyu}2~F8=-rP6)W8;Ib4#sGZ(w>*v$Y2@ynf zsmv4QZYF>odx>?MRHJR2;Plx7+;4@Oi`Dvabi#rQs=$(yuuD6$!(^{mZ*YbZ+q5D@ zhD2@BH{1n4Z(*WHo1_hEBPP14Ot z2TY2cZ6Tv|4uJxfEV{j!>o?Ny{FX#qN7cgBI&RJN46)?LH_!4~rKrf2@2p4+CB3~3 zpJmXQxIN@g<6h3!c@8lM$xE;SQVE?zY;?px(9YU80)I|AHer;TH#}9wibhq2yEbT% zdZ|X#haq`)AqS51T8#}9n|sqS0+XfWoO)9FDh!&Pw3PqQqH5v?ah01Tekrho(Rfe| zz&7!_ErW#MKB=wU)0ycLeA!5B+wnkTzCLSQ%&aG(?JbiUNXc$2>pgl4DIocGy8yvg z)q@Hqn{I`$01vjF2_PEtf667h={@oL)&sAF>zG@=ds|iMGhe4j+PNwu-SF_P)LkG1 zsjCVsl3SyzR}crca^X41G5Onxr`q8w(I0}Zg*2p}dY%J8y``tX%{)S5J2QSUIpVIN zv45~%k?aTn|3?7+Q}4e7#x~)1smx8djwgqge$ql-m7Wv8_eO+*p!Jt-WQ|(?t3jeq z4KyUjC+=GX{3C)fLb4v>T&dn9csFTEl+`Gq4uJS8`sD!W1nQNvkvr}Oopr7l-p?cC z9ZY==obC;lYiK=KyWSvbIY@mdH)=`01Yg;@tyM9%mr}^KGjr9)r?q!Sl<(*k#$`=3`Bsth7juz>~P?QS~}$%xakmd9hy7EMju*E_GId z6!GjKd+6;E;3Uox0)H~F3ct^2eh_18VeOJ#zStV)G7@?MAYbN>3eV~3{1WVrI;qUB zD*c>rPocyP)K{M@aD~Hxdh$h!1D{r*`WyFw?~*dEqfrpCdRJwjb1Hldn>}p&xg|*GIa6hmSV+(JCL?^#UV(A$RVb82mU| zlb`VaJ_@664D z*zi?sGm~TXXajRgZ9JxXYqG=mXmIN)$KN(_mBxaQ~7!f=o!1l zyx55E|HR3FWk{5jb_-(j>ke~3s@tPUbmx(Z2T2DqW>}S&7?q#W zPYTqCz&_;m<+J@rks|nvFjdU*isdUkD)U74rcD~SLUlU$db#$|zqbvmjye9q&$$_u zZC6Q|M}1yy>@%NpvH%(}4aVZ8eZB1csFWr@b5)(b%9qtfBz4cfn+bh!*~gRo6F;Al zJpLBV1CcyZf)fbr^~AhAWt)(F3GO~4#=Gq_Qwbz#)!Q306o~t!Ertq;vyP7o!N_&E)(Rj<2z9p_*1O=Q1U=k z%nn}5h-0X^SM2q*Q0j`4X@l-XsniZBocu;BSKrAJz1W+65|@^iY1;Mq$u%?|lB z<767!O)j$ahY5gILla2+PIdxH4}}VhD0Pdq{*mkIg#;bRV>b+{`}EnfWg!y(i3r9% zmR`VB7y@bN+gV%RuVN+tW_MQGHu>0=s6&dnJ>OhD#(k@kPf&MVZK$d_ukT0-&wH?e z<8DiVCqYF?lo`$X^X?L?CkvU*Bc8@@rmB1Vnz3lvo)ydoySnt1=}iRj?yZa7(`yZmzdRoSa1^f1^#EfezbNb)Zcyu+lzabx`V2R!=$|3}kRg+1Gt9ySqCCh9PEP;-Bx}Kix;qzV@}Bb+5JV z(9(*pQmATPC*Zk+&y|D(%wFD0><^19u04rN|7}xi2aRYyJ{h~u2F<7wXPKaIi>OPl zNBHNsIt;I=K&Xj1{igo#(E2!f=iU5NDP#%7Q~wH4ld;3Cef^WNYA0mj)kys|OLf0M zEF)w9fzAUbl9*lu$K8Iel_29P>UyYvfUQuCq(5la4jN6iCq21jG)rQ%m!=?UI-KSY z*4x*1C0Bl4!sLQcat^y1Rn2~1)gH1iJtNZ30+KanH<4G+p0Ff4#h;)>0Hl`C!TQOG z%swnY*Icm-%JZBF9nCJuj?WV%y%Pq7))as@LH`a&I&Ae?Hhh;KpWHn8LOAukHQ|?D zw4Y_L<7|6FY*d6c^FQZ7yujkj4MEh*`+>gBQ-oo5LS2dLS!{7`+77?fl-W<90!$&FsbO86|rwoY4D5gnrs=$n)EfqfqGyY%5xq9-RP z*$mR8NUvEw=&Ge}@*|ZK-~JRtH_g?Xvc16^4*nfKnXpp=#d8PCNaTKAA(_)teXQ@e!kl!R3$tI!i zvW1C#zh)+|rDhZl-E+4BAd(|CCkfbQK+-)d^|$FNp-Xiu2PI8pIuLjz>9dm=Ej)U)(xP&)<%EPkG{jH+W%4&W;>kaS@+r2j zQqE`Yfya}2DDo->NJoJEto;YX$FxB=+KYw<#@sMB_!>Td*61*LMl;~4S@18AF1&^@ zZSxpt@r~{g51s4Q6P_Qs3#_{7`-PP^;GltyDa0we-wX(zg?SDW)b5(mxvg7Ql_8#; z+L?}y{70Svi){^cCnJjCl3e6poa3|_m^~<-)9&4^o$L4}k`ZeoI-uJfB4vfBzdO0b z)D1htaxS2S@90uN3hk^lxZKfB-6|)Q73u zw(T(e#K)x-^w^ZXP(nmp=P#n0Bmh17EZEl{Bj%iiW#$ygQ5K4_Cp6;6_MR`D)-N>t z_vOB2$09sU_M_WSkF^G;CDe+&D0!QVh%?$zNfGU_~44oG}c6aCLIhQV9=K=$v6$ zBk91EZN`R_qk4-7x-po0&x5r|eot-l;;Kh7T+Gb5I^zM$i1xJba7gj{kY%L;38%optkG=H zg6zq>!6A@)1_0@If4~j1V~ScC=sOJoSb^hlHHgsVK5!dr&5w^gfTH?k+y>ma+WTs9~&VFj7ZaSJEf&tQB_3NQ5H_A?&>_B zdELS>dgL!8s~~{1?s}$X+P};&kkM;iTN{C7h^B8jlsF?KRv37?<(&!PvIh3l5g`p{ z1h25-wqs!?wTMV8o!^G?ePYeruy)=g?O9Plu_SGYj3bqRmw@yL?xOQIP_bCzz?8i4 znl{X=lH8eJ#-W>tX#^sNw0eVO%kLep_pN5I@IOQ5`s;|yhLS-Ti;g03npC(o9b2Rt z^|-x4!API}PF*aN;k;svCg!gXLkM4bGTi`4#(?e1crtNPiq7bw20|TfMaVM6G&x6# z)X2#ww}xoca0352TcWUStHkiG;E{+@tNd@?F*1hF96w^N@DuZp(z zBXpvfQlrJRDrlYNaU3I_WHGbku$*#br8xz~^^0Sc*9Y%>zETiMGJ3qMBa

    f?(#e zk)j7JT!4i{Q5uwkX&s@?A&#IJy&fg!&i-&>MTnM?Q;97b z0bBU6&wL7LzqlCrqhFz+1ruJcn!*azrh6b4TMu|l%+Ew4=8_J2O?J)iU-cIP?b8^s zCP855iK`iP!gkr|Cc20BU7YHL+;9jju^}$@!nm6s5HwKPwngxSpxL9vXOXb}n^H&7 zOW$8S@LF=NaxT>$fA1|X!EoiTu^WyZlVAyG{lsYwR>m&byvU;IWjv2tp6CIz%)G7q zzdaStKOo7LJnXR^7FNIfmrVy1AuyP9!q5l+=*8P zL;u++Szxw=*J5N(+l_X%+TZ<<0Ev8`sI;cqZ1|2&IOGEh7pXNHAs3boo?;~#TLe$| zh%UqOm&Ebsiewd;~o+5(l|=oz6N>BAb|IEStg#wLWwCvoxsPY)g)jP&&B-2Zt3YMq-J%c$^ST9bNS^U(%J3daci>oqSYhv zj6@K3qUdRm83a6o5GmVX(om6+6WQKe3o8?j$_ls>=WnoBY8*)N+c=t$bT=qXeV?=y zc$UHu=olaldOZPSjGu`(`cgLdipQXmN&qv*8`s#w*4d}hwwERG9$2_B|J-xyd~Gbg z$Yt4ObFI2(51T|FVI{Fy<&kz*+JTfg=I7HjAEG~1gxsxkx%UcQ{}aRh!SC)S;ct)& zetPsEglZ3AGn41JvSPnsd87B8g-p5;^2Z@kyCt1GMG&=N_{&_dR05Q?(Cq{uw&pC3 zj;4HO_=dn;T#}he$D{bKogv*%RuGn1ABIE8-RzC17XwEC5;zW3?l#h1p&pt4L`rsD zE2KbTJ4JEF2dnhR!a$rTQK)+#`V=|^y?W_$@Y9)-lXG|4?^F6-g(#?>^KvG4FghqW zY?1=K8MLA#)H`}9rpwLS)<EYLQaduMja>4qfY-uZ~#zz!6GXVPUWJc0$+7wsK z^9uX^M*1Q`WN8gwT4|-RHz)6Ey5hB>SLnzI|s@sOQlxop7 zbt)>$0{YjtfR~i=R4WfkXTPm(o=QN@$$|%D1o3ZcrKVL8h z+|ho~dDRAz;cLu0fxY1Ric$&tfST?gNFB6@CsPeb6OlWdxKRn`K@sXf0SD|W`i?e_USv(&mha8(}qf& z_scQnN!W4aF>8)ny-AzFpUZu#_?yxBD*4yZ>0o}+SyPb?OY6Pq6@KDfIfhPwcz!xZ zZ6xAga3pE#b6~cXu^H9k@nmf56CVR3keKqD&`+$sPSE=fxuo%&k>H2a(K6%<^nfQw z2J8=|^tMv$7s0&5<02UCy3B&iI%`aY<*57X4E+pz4tjp%9$83kFMwjINx2nD&A;x2 zL+c|VYGx4(U3N<8pZ8^0{MR3UZ{v@c#{98Ne^v*FeQi1?|3OiXjdz)qLe|?WQUIwA zd5>A>ykr8^Z8p5gnUk?%(_C31DfKP-`Cq+bQR1+xWu9age_JfT#&MM>nVcpfkb)`y zSUtP^RxXBhQvu02Ro1S~>CE-?b@s13V8;9Ohk>V~IGvvm1J2KQv!9EQm%SQ&IfOW8 z*Bl?{EdNFJFI1kmHk}wZ#!(-foRe);dD6JcZ6zli)NJML72=aTNd0y@nVEA3x+h8g ztT2oq@Rq{!{jp=Y>5u2Tz17Rn9njI@)&RJw+VxIr(;Lec)8TV!wqha;IS6b zAY|mxlhY_uLiOCsY3I6Iu|$QvA&;-G#fEgYrhCRowqMkikqm{%`p<%uaz!U{M8kv4 zSbFFI>nzQhDi^<=DDDE{F#Jt-of2mVetBF))kOrzmF%GcX3(+;cviPJ^zy)0QzV*H8X9dYOVWX}t6 z?$<_xc(r%0nh2w0UM{^+;`dEXdt?xj0 z@a-y4TRfz1`2I#jl!{=ADCoFE+Kc#ygziNnPf_ zDVHSek5Ig_qn(*>oVhR@$ZIbpkZN+mN47(%Rl`B;gC~}pQj*6z_4c+)-X6j}0)Klfmg=$MwfP56GnqN@ORmm495BYQS@4(}vx6qH_5uvG(Zz(c=!uYEKWQ8>l7WrZ4`wGDy8ZC#Cc z3{CxgexBn85%NGUU5zaRFg#k*7h|xs*htAR|RzkB)Kher2Kpi<_{UK zECsyHa$3j{dVL5AhqwI`e68PXuUojQztG$hfRe{UIyWp}s2wEf;8@05i|aG{s%BmEpj5q;6E9|6~dnI%%)FCSTzCSbB_!{WeY)Y>)G`w6)Tq|urExKJj` zwL0`LNW|xxoSuuI^Wr$kP-2d-+l_vb%#A!;_xZ=vr#?PI=o5NhT2p|zyZp0(=@VGt zU>kKY>)SFdTcYU3Va61y_lYjZyz8vOi9ru*ju4Iev!-`0yMir(Xsw&iAa5ZrOT=om@CrNRLd0KJ-Qv*1ECA$$;3BfzIjg70*z*}X#? zhksWA|I=be`mJBe+GfY50O{OXK13f#Xz%1sW`jE9U>KrP_oTAB%wLpUqT zmZs0|*N5SsR8HhNcr4Cl=o@6q5`uQAoe2<=L?U&jhU zk;>UszT{pG#W`ctY9`BrBQdiu6SZF@F$puaGR83;vl}a;M83WEZUys-?C)dK;s~C< z&z17>zRCi6HHIHrk=^T?e4;R47_t|ffBxi4HP^?yAfaO6Q28VaC&f$E(qSBian#BX zSqG#ZK~5-rFP+@=WO|T+_H62s#rVG!jA*2xD8R9U!gz7>Z_@La<9)R|ZWxi82^A1f z{WR{0kAvQJ;06an2UvJ1@W;#&tC5gnfHUkpb-^;7i1ptRzYSic)|V^NPjcIiJC;!E zswY$z3Erek6?9Xnv4D%&haESavy8FK9r)g6`;AU;db^BAHlJaTwVWtDT+pOBpp270+!}vY=!5@7u8b>7~ z+xdvZy6J_y)`#znK74*XfJ6GljZ<4c0Gk08gUd0;NP8P=IHLV2s1ycxpk`=pa^%J4 zGK|U159g(tk>!a?G!5yvjQS{`7ZWX$`OXc+{$3A;>om(YbZ>?;&4mbo=kBKc_m^Fx z9}LSZxrY7TE@h<1+~R^xT1W`6?zMf?B2In?XA( zQ|y{%KgS3T5E=8-jGO)^N(BjFMargCzgB#uvrMCeOjFm26gVFA{)CIV$QF4@g2xDW zSQ4Ki_Bonk7^S!Xi4u8=rLMvx8rlIA1S)x6w0;jnUR3iUuA?P!d|mdx>+iQdb#sv8 zQw`PPU3S8k=TO!NJOKM21dbRmWG3DbIKwoU{sy1|Ko53x)%CbNBeBtL7|?sWv=$U6 zB-V3+){;Ibl5}IjzcH`khlO@MGWgA|p50I5Uo1tCrhwzc<#N|+m@>*{UxCGM?JE$v zT+LNHKbv;Sjc*vQ#(Ml{$Dp~$OoKCqE@&n&4@I?`R$5x4t(}UD$?!rfO>I%reoTnc zdl;R@0&0!8f7RPRSKgvV>wlvZ+YIPR9P>f#rX{KRDu>0|DEHj2zTXgAHGmB7VaEb2 zeS&^qA65W{M4TyPSu=(}skzsxc{ul&l*7t)GP*j0s^*U^Ek!sVNw5ToHC^723**VZ zSoSF$8N#dH3`E}YFO6R;cR*w9KP)#p3IPT~$>CiA_dZkI*HL^G7JbjTs_J&5u@zrJ zuoxnQV?gV5aUS@^#9YYSLzTd?XFj+yjlwea2)qI(HKPt*#i%q#HxgxMzT<(&8PQcj z3UlxMVg4iH4R{4d-^jUY6e*`7!CNP&F%mq4Fu!x}6GPR&RT6?whTsQipaX?6Tcj?( zb*{0`BA8Wf56UMKD@tvL8s&ZJg)r3+2Tl=zak$E6M?ZAYuq$efCuQdJm)YY=At2Fv zjkgNFSc{8T!6PDh=~Bn-WUGD_?*{CzFO#DmJmd1NqO?>&%?|(X1+W_QXj${wbhqzl ztA_+VfMLZKu>_=oF?@5q_Ca%=%v%*PJR{Dk91zQYrw5ngjzAfqW2+DKD}uwc&!LhKyh zw^z*m4!qcHN|cLmegiUZaYjE9*1Kebb>GGgkBXM0g^eQCJ`H535P_K0UIVU?o{Zg1 zr?gU>nEDA7%fr_z0J6{e<$^$!2&Dv7qpJlooP#ZmpE-y+2vcJtz^5!!$%*B7Hnt6I)n)+hm-Y z4c_Mduo5A`x;V9^W+QXe%hC|yw$SO*RGtSt0`{UXsZ?(M7Z2RmV#QnrpA7bZs|M?c zgTi=Q`2(d9L-?mpCT0wUbBLN(&E7I8d05EqC9Cg+ukR(k%E}5D^oJ-9ybgqF9%WS9 z5>R1OsY9J~?ynGE)6NRN5J!)YE{Af4BjC!Wnc!camkhJekEu;88bVF2>%J8&f!p|z zAJb?eye++qptQ+n6F{kcwjG5fok%lPqDRA5*aO5e#vmi|1TYTJA>`zC2867%Za)z;!!B@|`VNsPLCdCPCp=Ew6 zoVDS-=bSnE>HA7utp7dy%5^W&MwqB&4QCivdwg?3A=uYD`-hTLoa;tb1qWy7JX*gt z*ih%57+7i>IkBK{zQIIS4JcHRPZM3Ts}?I><0z){hjv5|N)JbJKNCqKWoHt3jyzW8 zGv%B>2KXlakdkS!3n7tv4w%>C${U}@X?}kflT_#FTwVF%_GuyQ;r8!}x<Gbqz1wX9H zz068*>8|aWvHM_T`mf*9#>HS;FQ1E#h_2bEelzODZ$m_7@0i-?%qVt*3}Am|qsvbV zrpsKO8Svw$%isHvqYtC{OeGPtaP8gMAt^kw)A*;PGn<1yhzXr*w`_*7{VL3UXp4I! z3DtMB4B45pM?_p!rjeQup}1k+QGZ+Q8|E381w&w(KCy|vb%?PXPUFxkZ)9SLiIm00 z_$_WzbBkm7BzQWaTW0RZfPWMSz6@KxStuvyoySd4+ZdSKib6cejZue9CZY&MZ6m%; z%Pqhq-7gTBc@YA`Wv`o^A>9z$N+k3b-6afqNCnod@ik3MN^R_6ER&(_%a4-^iIPg} zpHU*!I@H(O(pNdGvtP21Bt@1aRVIBEWViInO54!bFc{xa{RLs4i@gw;<`?u6?t-96 zwm`;^X1q?`e)pDfLA=*~yIUdmJZljHg<8}k2DdPN&skWYdwY($0@HPt(CZpMAebJ2 z-;~KFpH>cruK~VzjK}rkRx71r{+G{_V41;Y9X>~jRT@ck;t`JgV2F&$73Y7q_?q+p zILvd}ljFnJ0AYfk?3(oUVmO1ac>Dosp55u6awxmKaKOae=un5-yR@pje?3+zSnlE?0Fq6pxc>0+)-p0 zxVIwTIlKQVDn{y~S($|ti1k0XqO|DyGk~iYaQJqk`sd{H9FRKU9|yVd)|2TJn5QmP zH;vCJCZM!UCqfUd7rOlXpstF6aTK+{kkGtIX>D}HWkjGER5!5|wTJD6HOmIcJiIS` zOKnh!qu%_Bm<}e}LiRBFqKV>{UPz1axEo5g0K{m!!q8 z>?8ed^=^|VVxDEU0^=hwxxNn!S5*EsVfL;pAF(t@RLBCTEM;%B71pck*~v@?e;v;j zE!k-FeKAn_STa*0H$xY?be|1eGd``p>L(%B>HReIA=A_i&L;z>#_!a~M{tRyI4*R$ zI;HM{YcY*zF4LcBu+_MJlGJb`cg(C~4`yhb0I1e>8bC}AIPlaqo^miIW-;`Bey|?J zd`R~VXECN{eV9VNptj~oqCfp?3bJ6888p@qPtn>yeV<1@b4~YGXCnihW z{*dc7TA*X-A;Qh&8J4cKTs%ph6NwO24tOV+g-K-3&k`(tRBX6tS<#D%vguoT+WJMV zv{kG%q+CvT#^^`biww+6bdsm`T6qUPpYHs2sQVtn09tY%CNau_wq;LY7Qi&%0~nDz zcN+puIP)z9&H8SSqYzZH1Q4nK4NS=j>s!X1x%O&S97sBQTT8tRxLC?cd;Kk@--=}O z@l1$syLU_kCNn5>=tp-Qtxq)LkGWfS1lxXX$7ypU(cLnDH($@9f8Nhckg>4Kxz4pB z-T4q48%~ERBlrWQ6#pe;Fm|T<8JRdP_*w33YIu$GpA2^2`?j?#KDrMBVJZ6LajrvHudt=v5rXz_fW5_J#&qQ0kT077- zq#bAX8JBqZi#~^_kKWtXt1QyDlbbq2Nn2KU#%zHcoqaCvpg%f}o}aj)mw)fOs0O`G zxqb12KxR)~OVtio%psz{&O98)-U*o~2P7v`#{J{&^wTZE8Le)&7kF+?R+!08%W5{| zNE^aQE?A;jUB zEpc;1CfZ(|dBQcmSAD|D=RRCM<05GMEu{xC3ozoozqCPly!+T; z*Xy|PqUT;tMvx4BSL)x;zSoOal)y~tf&H+do5*-`PnL8)V<@maM?9B?swOq+2ep=n zBB(MKy`9#I<}Udj?D;@G^sd*(vx-J;ucEm^!V2R-%sSn}NkHEs&D}!OI%q!~+gBP{ zi46onXtoy$jZ-5*9fEh>fc~L%C!~H*RN1dw@x|l#nLH>onYwsJmc-O5BCbNuO7S~m z+`m@~ki*UU$*t0YVqs+pa$o{D;#zFLes_pfQ3nZO#t)k+CA5(Q&twY(F5Y}^veTlM zJcKhRZTU0S>ah%`XT>j@3--vCs^Bm1WUdN{7(=x zh5+0yBe}S^+!w0fl0?{ZK$1us{6|xW)#L2~7Z2Uv@bAp&sqifR4e#jJLcF~n#~y_} zFU&-C?QI60cLV93BzzcA7w)Qp#OX=dmaP6sBnU4T?WfxYP1(V z7zO6M3FUS8QqVrx2R;b$=J}zV5 zUq`q{7CY?owU{DQiwCA_gOrzX2;0ZMERH)0*=wR5=w}4CC!Nur>z0xhpHYQJleu&W z8MEcV(R1&5i>*)+wC{ys@--P2&+`#+=2<{13eR+-Y<{Qx;rnj2m&7y4|I82gen@ce zPC^n=U(&TM#ZCQg^@&-CCfH>n(k{Cy^e*Kdty)Tx6|*ykQ7*j#;5|9`r<`HO6iQt43v+?1OVcQrzC{hq{7W7MsYZC9^c%PJzc8+iBGkNXRyNR#g^!fB_YusrZ4 zRHr3W?5{Xx&=*Jr1b=K?Y(UF9RM@9>I`4ArW}4FnMsp5wNs!kaTj3 zI-N{EVx80yfEOrHEDBO!3d+4+NvcnK(OAznmpg8Ekg3#QX$=)w+vD7s6R0CWzrSu1 zMZHd$mJlML8-wQ#GP3t=Kg;ETADpiIPW*{m<(kcRFL|Af?mGK}DG|md(PT8`R2r6tC?Mz>*6$534OKvagfQZPPzvL1TNm zP?Fz^3)y!?bErf};UDZaZPVp***&!S;m#y5QlV{a`-X+2`}!Pk9ak_O^D$2MX#X?c zZM3+R`QNIja`>w|A2V3G{}hGFyj9iAR_Vy;nfq6_JAys!ldhv0J{kRiCj&+Ll+Ol} zh_TUl6I-jW_$ieL-TvFA)%H(&94eDZR^cuW+7IC|d8XnGgrEiZdjWGn0pAbL=9Ftz zj<-`lj!fDE3`gqwlA`ezo_VnhgCK<?`VZfsk;PbjQA` z_HrLKw&X4=TY6%+hs`BJ%SMqA z(_X=N+xJXZ71nbnZIVGMlVJGz-?L)1Tu~|2=Yzl7aHi^-J&)_paWN!ttMfnT07SFn zkPW`nJy}3?mZW&EVG;M6^`xa~Kw`1&pQRwym{s-y(neisV@E69-*z8IE;;!QL|~u0 z4dsj+=D8YnF)TW+7gb6j>{ky>+H4y-wrQ@u4r>ki1K>Bdh?~=c5RFZfJ}X3D$Rr_B-o0%7BV7W3>(I_ z0T`fn4e*IN8w=-G?wKH>dZ%GPPJls3yNUT-rLv1$4|OAUYyZ|8(%NQKeG@|N^99_Y zUzT`5WN3F8^!ia@NHs22%Y<7XoSxG0=Madt=#>+XjWVLtPducSGl_&N1&WZ~&#Ev; z0xZDZnzf$K`w!R>FBgIybb~|G(*_gE^E9K>Be9c^6jo!Fnuv5r+j)oc&_UMNDit9{ zk0TMtt{pnlCkPm67rewnpOR;|{{;^mzRkR74tg*$swxDD#`+!uV?l;HQx>CpX-`o{ zA)1$==y4X3@$Ux;8>t|_)uZ20kgec7O=A;%8vsm#tGmweU`o=Qm7@&7Z6`;m&_o;}U zY!DfMcu!^W4BnOpmSDcohVhb^^EGL3+A{_~J`>&R(er%2JN(Nx)xCYl5w!*T%d<`g zD`)O^XLz{QY+&SIcPQVy8@uHjLYI1SO(?l`X*^QCHOqiNFhcus*pBhRn`^3SEm}b9 zp9g|05gBkgG|Tj{<*3n=t}>x*v%z)flcBy}Tz(JLw{_sJVo=rZ52SFfO5LD0U3p+x zYO9eZT86>s)p4-nXfOQ4*be~1?=SNWu0)wGoxfegK7FOzM>v>8fbc!PO4Gtw>lZfR z_amb5(A(NbI_@~xK=+0*E?28=SFnrh+4Dcw`wRM!4$q2y*fZbV@-jP=`UH#11(W8x zkSC)FX&8t32FhWjnioKk<`U|~Lxf_Lh=ibvdi=Yfpcp$QtWy6SsYeOhVK*0D=N|La zl%43L6eA!=(r_Zl8$ojDzHGUp5^ykNZ7GlD1sMtvb5uz+=x}-;HzNe|Gw(V6UFY2j z@ewNh?CQQK{#;C7s=UiF1#B1~oJxn!yfwnl*qy^RBm#tUuwoL_cibD0T0WYm=~|xa z@p<}z6%3x!bbL~ELeIuG&ke)1V?W*Oy-lXuUMuFt{N?&INKyc72`-idV&ItCcp&91 zl*|=g`FX#&ZM-`tK>>|&$RV#B==;%i`9t`P(f6nOh=?6Z1JR^n)_)NwQ&mdBe-hsN zcFNL^^k37XP!oecj#b0Gs}(X*b;g2a-W_4B6ZV+~=5?blFz49L%@dlqCD9fR(`$ z-~g}Qvp2l;mtulLRD8e5QvsYe>u>O@AuXBZYB`b!nV8&`2exj!>Q~uJbMx4=p?!Rd zuY;v0S-5Nk)*)xk+z7b{no_g_hRu`cGBLFWC!I+uqpckl{kO`)V^DT7M8^Kh{7;42 za;UgY#?EJDJ;hI9NqVTcj5`JOs42eL%MM}FrArw61W^L}# zyf~}fu#t8MO)oJrk(}lmp@bp#P1fi5S83}yR4;!0S_;8S+Djq$ya#zIQ0PgbN9%=r z;zxk+o-O?qH>S4g(tgk?uO4-z*<(@F{9{G$ZQDstQfq{K*LmO-jO2@KxP8lS5+i^F}1-oYVH6)NmPeO9#_KIlBE@cv+^TLZDoFVeum4v#;; zh10b)KVR6x`ShD4I6*^&(*mz_zsPxJ>pbjC#E+Tl?F!2`PLaG+ohrA+bFVQ?dNGOb z0FezmfjCf+485d;Z8gL3nfnD__2|P9gvFc4OVPkHoY7S>ee2M+l!7CX5tHG%-r6t9 zyz-wR_S)4Xzi-7&;*z}!5XpwxEWB59_yG)={(J5&)W8>{fWc!nii zqoGKYUO6SANuL|?`dXl(pkZ$l@4stFnfWQ;gQA$>4M27BRpQ<^iZ5~h0zcd@fj!pX z5rBVjQ(96Vv)=lyBB%GX&mkp@bIGEl)a0T$;M9yE?G-%F394kZcnA%P-TJ8jBO@!9 z+5Gm9Y${QLlaE6(*eXnp#{3(0tvo@L1U-b(Dz?X}bJ1&~3^=fzdi(=<@OF^Vq(U|^ zzNj**`dM}o$tfe``CD;7!X#L}oxS11XnoB1;L%n(*c36xhhXWI-10|ZP_L^hq&_%X z-@@+u3t;et{-923aU0OT*V&KZHge)hp8>sKGqAAqD`}AWxtGcI47XH8_c_bc&DaJh z6i7O7(msuSmFM9txNo#rR1m}lj>Sd^j`2DtU12d~3%HumbF~(O|7(F~(AOodcfdp3 zHJyfzyZfAKdXd+k&Da#Zy`_i*uMv$lMSuHCzYGRlD!e%Q#SXgleH#Dq30SR3BBbqW zwPvoE^PKPZMc1|3S)V!vbGIf#z6uv_VddizkjzmsEcNf!G7 zma+@NC|h`o2_~R2XI^xCOR;*-CAp?ByvKw@RdPjJE`n?W8an;q3V(YPmh6yqy^H~b zH3~Rv!=ypfLs*K3XN8P1Ae$i7!TCw>0-WwuC6>naE7n1FaP)zI?XN#`D?>@BkFYrf zbTGt!y_|L~epY@b6J*i6nzJG9$7qYwdt*=skZyMM;NmE+q>lnUKgIWY?Pgn!XrbAW z6Yv(AfhQxx z-dNH%6>8r=(V7%48ld0`;#fvWc4R+;19HD4wqE+anb<*(*R9*N{*wgndm-qOC|9D? z@xsw>b99K6)5zIeIK$38-+RcTqymA$43DagHhhQaS@h45W&Y7`-SLl27lD5TR1k;F zMnTxFkIIL@XhzX4_~bomcSz=wBkJFw_#9x};{>QHPWV+G%Lo``{kwX@XbXA;JolWo zdf(z?7+!b?`~kaH*mtKatS&gI-*I?hRfOK10B3_^5NvHN1Db_Q;);%}ul?2fy{>D% zr^4nr1T1Q*X#TRvd%vw`Jg*7=N0R*Oh8T-{Ue`4*vMeP81fXy>mc&JCK*2w z^-k1%cK^&j$Qq}ggD~UpN}}-Bn47Yi3T2R}25vgeoVnE4*-oHrRk^YhvzW}(T#D2V zyShc`Fo~To$pWuo#}`6m^B;wO@oWa7zTa3pwk1{B2X+lXxcW@V&o#6DHKtNmD;M^0 zqzyv2(F|A`#i#v(GF|F%?9U4iL)-{I?LyU~(BDrU((B}$`qWK*`4QvLFmw<}4@DuWtjS#&47??2C+v?m$ItG-c4^OY)dtKmNSE}l9 zEu&JcXv%-_vXvS>HWao`8=8tz!;k(0uIWt9c{*t41`j;1qGqxAFo*Gt3!Hb?+S~ki zqq(O*2`HjLoOhy`^Ui-X1$p&PK3+ZyINIwuE>BxstaC8DdZ2g>-(7GAgspt}r z&@MFGnB2fT<{~ajl@X87E_nLVa8C|oB+ z9nX)KsUqux%-^*L4-_l$3deDA2UIFjF-m&szr%Y-BI{M z=z`!oKk4){^v&<+c7QWoi0E+a67`lI1w`TOD;b-ww6}so(C2Id-Rv`=d-2m{7A=_` zpEcic$R91y-Wm3jDd!)GWztw85W^)2XciCn1>oc!-Ft(Gmx!dWq(^X~jZaz!Emi2} z9)I+kx0|)-MG?cWfvwL3Za(`xL7HOvDnM?m%_1DiilB|M&QCtENQ|~{X~;y zdG!*>DVNoW-{WqygAEHZk0b3h%;B{@HXcI3x>ybJQ8>Qv6h<@{k5)d~!f*#2DR-E7U2RLhYzXptY)1gjkd z1`dZj@O==t82O?YN;H}ChmG`5Z~i*&yDI)P%QJ%lFfR16*3_ad8P>M7zS87FpDS)A zA|p0je~Nfbs;#X*pxTD9Xt)T?{pp9Rnqw#AzVDyT$>lu3^7CSOFPVk&=Lit40)4_- z(z+Qr_07-A({qGxuq5o|ivd@153JmwB*XJ)2uhD6$>Vh4fO$lr(BG6dV0Wzm=myc{HV_+0t>RzIoJ#T*Vol8M$gy? z!K)(B8l&!KLyisNk;|{4T~Kdyu`&6ce;>qiL9tk-m7{!sk}=NoL0BQtm5SvAHM+xU zv9;@8@ae0^q{pw4aTRsm13;}c>!A;3Z3hqazo{Bi!C|_SbY-$$$(DU$v&;YHFf$ zCf0TOip>w>fpLZ2F&zHMwQ=Jx!>1YCn zT<@T|UVAIvJ}}o6#@aUQC0z`w1K5OR%!S?8%0Ha>{xiDEJ26_jIqk7ta#I7Xigku4 ze}7V9$$_80D-=Vhu#c|bK)Ef()A(6_{(ctCTEGxFyR6HIfR1BI(o6?He5WlqNxbMM zm;L+G2{6C{mAN-JwY1e7lF2EtzX#|tss5Kk)0F}DOMyH8XIsd}%JEcKCPq%#3wJIn z8Nr3&>n|7aawfvsSU%kdUd+Kha;y#4`<#g6pTB%J3eOpEfPQwh)qrVEIYw<*5B+*+ zpE#p=_TOCS+1Chc0z{;|#ApiFlQ1VowVomh*&liEYt6@3g&NwVB8wNCWZ@J2`V7hMRfbZ+!k-%+G;epo*}Y9{u0Q=T(VIdY)Ox4iPTd znPm!J9H7m-Z5$TTZ9@vP=%-&n`s#QVJ+N1iU;=c=NFJ}I>R?3CN({)K=!k+u8a^nb z?bWt~(U|^ilHXx_G~O)|<+He21LlLUwYk`@uj%OS!@YuEAxf+{JxZ{ZbYuT;d@Ip)cdU-F7UhS7_^I0+v z3{Th!gf5T+f`T6gp;NBy3!=M}1obkK=QUh~MNb|*Ci3Bk9X}g5Av#KRS5^EKKj9Qm zONv9t<7N743pd#5TI^lodjit+WiuK^6M`!b&L^_6Hoey(S2aW0iM0FmBq=kF%&j_k z0c6>xBd| zL`HT}zV{4YSgc#CpLM;(p!1#ut*-ic*MygP?g(MZc*5KWg$MiRk zN?N~zuXiFaUiUNzUQ|ZQG+b=v`;GnpJSEZ09@PpgSAhN>O=tbr^!L7TQ4vKcNeQPQ z(hS}p4HHmM0TC$?VJIphF_Er~5RjH`CS3wj(i@%PC<%$t4P&D=_G0_)^TYRt^9Q^h zuXE1*xUc&<&x=dSi&w7$sp9>a$QbMvA8hIkaEoR@*FM!8_d)NBeIg~9(JDdBk*Q3Y zyyQAAm8~+?2X;|w`Z{vjd8KNwLB8RIlYb#XUq$rfUnIs)!DW;n;W6T=j2fy=to7bE zEa*h$7+APdUd}ckG_UAaX8u;_yFO!Y;uNpKzfOvz!A|Ris8Ii7u{|PXrHx_vJ@Btk zJY=-F6N=hbM5|B>n;)ljBN2#HK)B~CV!D_Yf^QYZ!p^Q4$8a<vKyXJ-LjZWt12!$JGdDX4x91>QmB%oFmGkSYEs5ktFC&I}ohr zh3(Jymn{RkAj<^47+g)TaxnsE{6OWSxW53BOBYXXk9b>0y z4^=mlKGSK&$b>puCD@X*oSU;MnihV0LP5-*ml6a#iV&pLx^*P!FhKWD^RWj!NymsF zs?B{$ek?Qfox>61Mr*CpZwn>f6D^&@a49CEgVpVLuQJa+DPV9$B~!0JM?Mx@-vnu% z=es}i>zQ8*R-jqH??GbXB#d`wK&(gcbj)qg+BXw%wo6(Uf`sV^$uq(`toDObi=SEl zCm4-%8y~pC9@6D0cr8-0&>Es>|9OX+T|=T(@q&Xvxtk4xB>E5com~Y8Mc_5 zpMO2Q%2bqz1W6-|YX2PHd^GEpSs~b^8#&)o0-0Hb97fYT#DW@+; z*7!pIIMZlR$M{uq0z&P)1@X{48uU{1xoPpSu!o>}hj_C~sE*L^!`=9?Gf^pV=kTZ2 z4UfPgZwxUt_$95T0WX4vNnG{YA89xH+t%R=YQ8#`pNIE%FP&xpkG|I?F}l7V+Vb4x zObo8vXj{K0U(I&9PUN?L)R7`BRl0D%&lMdrG5a3TtbY9G$Q&6;+;;vYHhhCVWNBx* zW)F25@ojPS#uX6hAddAk>~O(Oqs7alJq-EXa%i~*rmXah^n@V&K4t}OMfBl|Bi9&Oa3lnbbTZRhe=@4AnkCtM3s{ir)6=)5hRn!>5db z{{3!cc<{G&!vXAqatuCfv=jaLB@1K{Vq+{2R7UWesI81Lw#q#xZCHE*;!2(9M9MmZ z{&)7spnkMPxrV8N2a?30CowM^^4VH>d9RHAF`^gwOkv_(?wz^0go6ggTY`#nJZgaf z-$>m42SA-TFzC}~6?=%gQ2)wv6t0T=N&O?TQ@?$Myz9|HL1i~kokc!3P%HkP3i&=m z&(z6`@RnZJjbhQ%uQfgEFjGCbspc`zCJSYiiEW{7s?+|!**ES#O~=1by2zhz>YZ=% z({sf^tK&$_N*Ckqb(jx9yJ%_>8o(Y%x0F*=_#B@LK&~K8$4h82v%lGyr`~O?#L@~e z@fk!zlN-V~-HvX_y--;i(9~|xlA80Fs?$^da1FeFI!4h+=#SxkbLr7X-P>=b9UsQj zh)E_Ig|`*`BNSIloIhSL=ONjh7jW)XQ{S0^>b1s z18%$iDx+P6&Gt?Gmm_%2vFYCDco1~!k<59t&ACdIJX>B)yNVcFew;w3*WVku^kbrr zWX_4-(9~h&e0cG~La}ILIFlPKyrl(dn$>Ff;mTa7uVL0+YnhqX^McK#6ia8Ke z=$J`BTs0oK8rUuo;juu@eiy%jaLywVRp-yorUG(YP~+?W$#X%+QPArSEZ#73hi-9h^3wW+k zrsz3Wsww}N&d~q3xyg36SMGtK;D)DuL#^dKj{y84a7YeSJk&bCt%`D5li)inN$)R z;go>Uwt#GVFZWhfYX=|OtBs9FmImc!+U99k*74CiQAfdr6NUeL-};m7Skd7LUmg%3 zT7npbA>>?J~vz4Rp1B2IMY$AC^I)Qw<%CK)G`@T}V2VAHwRR5FqBd_iC zpWKpJaZcADi6dWMCrUA$dj}S5u*7qwWf12UOAM(So9l@EFR+~|YP;H-u|2KG&f~E` zX>aVdb&fg%sou2AhCMy8mM8&*M1tIQW>2ZZ_spR#Lw_OV>5t|YQbf#Mg(F`+mD*9Y zp*>kz#=+Nqc|UoF-}Onrm)W%=Qm~FE3rqDhn!-*!;i~*cioal2wnsYy z+U@iv1Ha0V(H0VpOBoq=#Cd~;g%IKyeI3MaA#vKfFq9RhB51yQzH%wB#-8DOD{ zkdZ=}{@@d7s{}K%-Uo``oC#hJV0#I}tAOq?vm5Z}8r@**dp8)G@`I-diPMqf%T{*s zI_1cI+Tt(sxUA7g+(TpcpnJutN3K-(x;xCuSr*z3&OVjrHU|#vU54Ye5ibV}Bf0s`uICVT;vy;rzS@tFzk)K> zO6TyG(m1Kwb{Z-K25R3B`Qb(nh3P(RWFhgttxorDME~rgL>d;;z4FY%zlUNt z?F&WaH4o0yRHLXfQN+sfFWs9QQO>P$hNrpY#BUgrBqoYWXed%tGa*!CCrm>aEd?}8 z&daymGI1xQ%uj)&`DJaZ`K-T!eNRZS%~4jg$9u}2^OwMoV`L2EB;KI@U*YdB0~Z6_ADxKrCr@@maw7Q>2FWFwbhs z7HYwI_tYt`011w2$%e+^TAbJ$xNZdo- z;Y7EFdYk2;^N#QjODP`Wy0C7Aze0CIP;VEAkXcnj4#wSoL@mm|T!c|d2}etmj0*f{ zeSWyI;b7a&eDYI^!i%ViTaJfylD;bm91n=&*Qoo=+;ha{ZCWH(ntDK?zMs6y5F^u- zcQ)kZNvQI1nArMMBsSC$4Nsgl7tKHkN7E#lju9e00tG&L*;{iXah#q6mLTF4=zHh zs=nXl2%alteJ~bl4H_%q|WhGO$fV{|^V?XO04p znB`y9B+oA(f{4_ScE~u$Y!he>BGoIlkc4io!Z^dug}z>&!JeMJeS$nmzV|jzrSQcGb_T!du!N zV`4ec-+YftM4q^Pc{6L3T?MWme21F%v^&Y_N~=a{<*Z_sK5h3phXHtNEXg2wYd*G8 ziWoUt*E!M$M7H!L-tEDG(h)9BzPF&nk)jz=Ty8s&Gf~UGX);3{Ha{5a-&^#j<;%L* zr;sxI#{94$+LJcBVqM4ni|eEx`2&|0)3O=}NF$))EhYNBfnPu5>EGcTp+(O> z65>wryGR7mbzIg@Qv6B@YWY{jpH;r4B*Z@;3K6OSwYdy#Ihm(1>`$zj#fF`IPTGLM z^{HCsEH){jlV0;~utLDwHLQTgTr|rH%5U>;7@AO5GqEz5?IG>zchVwT1U{n9^@=Y+Yk z|32_1zVL!-#3Kodk4yE+h}yrwuON2$YrxYX?f5=+40;AvJu?HLq$NnVp2~d9s{1t{ zmOy}ho=;oP#&bTPeVMs2pv6dy0Bu@so{|DF=qkp61%%7}?+nY6{dX=n%*qtH|C`IK zNcjF~bGG1v>D+~!A4#D5J$Mzra+W*d8?04+eLMT!8x^0JuctY3kRIKcQf-ut@46lp z$UY+fW+a)DOVN{+YtVNVcCSXU3v=(Z`JVH+kLv&KwsWCE1W@fCI%|J9Egpmp(?;1{ zcmHT@&K>sCJ+^51`qtgPcw>0OXPhm_klIvP&zmPZsbpM|*01>VJn(%p@r(t5=cXK; z9i9)&ZN{ON6PF<#;|_jYo)C@SpGtmqDjL)g8zN;_Os{ITc}~=$!7aN7ncT^*7IjHA zT#mv80Vrc1j0!Df*fjqAa{8b-gnR{MFc};BTZnL#)lxOFIz$fd5F=au5n2|9_hWXTt(&u52^2@gQrFDGa1%c=i>ZGxm1}yBXVlj4 z((vJ`B(${AhJNWejz0J`CVS~b?O&E3G-irVzJySMf11S- zK!7hrC!-ZtBLYH6K!9)#ara$JWe}DPKF&tsCF$Uv+th^=iJ+%TBZ@SM=OVIGJ5nwaeafff2J+x$vTubJ^Yt1z3%PY}GxJfT*fT zSFa`>N|}u9vZ8@jo$sPxj^b;eso`r_X^D7}f8F0HqSTcq*H5+)LewAwMW>rw3$4|F zjMG=6ds+31BTwYlM0SCCz-d0=ZJ42-<)5@ZO z#yiBoZ!cGeG|%UL>J7(I0Ehylk433%wO|Uiqv`@U*AF_+;96#KXPnhn6F|yGa>y1K zUcERh=5lxN5OJyVCK)6s+0ZUd{e3^`8D0Wn&nEojHJVFN31N^4%0+cFiz%{BJMG6k zoO@u%v|LrW-sPL?Hd~?*CIeyljq8DJByC=520=+=LXp`;uUcROf(M%8iAh!XH7tr` zlolVQLx6Nn($4fFO58d3@7ng*DQ?e!aZb+~`oFe#7V@?tYMB!}s;PAY2J$9bYLLot z`dJ!Q2(Ger!(jerp}_r6Me9SRvPJkUIJYw@WVz|~!oEG#dlri#Utn|RC)ppiUj0Ma zb)Pizl~MfXcva({RQD4&;|(D$v&|~t6&z1rKi@Z`%h>sSX1DgB8R9Rfisn|(@su_>1j4H&@L0QGwW-O>`*W--%yJho<-SXrsnRq0_k~zRe7Jt;KiKM>G zG%>Xzt*4UI@v47-=Eb`iEo35C)uFSHa3{4-+-Dw zJB^F^tYOaR)wO$yb85GkARdcoaP5eG%v+>5t!as_Mv**^TbN?ugR0CLTd9hz8_y<2 zvjDXx-m!K9;|NMeqkUK{$n}vm_JCXXv@mG!9}m% z_KLIqoKxnGkD?WI2D&d{-_MfzWwrfvrQZej+8Cw3@+L~IFW2< zw!Gew4(e&9+;{K^ddzqK(!P4s%8jP|zk;al*J`|2m%Hpi24CY@8b4_Ay}pnYC+S9# z39(Cj!)?&yxRtmmzXF21TsbNiWnKQ43e!|!oe>2%#FmtZ%=6cxw$|uY=?0$?86Waj z;|iYBGd}q^y{zu{IDnQ2QsSTi54U;+n|j~tqRy=0yK%D92@Psn@a*!ZXj8YxG4Ocp z&KIK4omU>MZ?qc7e;z`ZzfdvO*a;0CmJcISGNa+sA{Zk3M2BK zYe2p$|LA9>y6$dGgH=MqEbiv#`Ba`vv!HNgNNVEw!^J!=5d_|qdUgUj$vJ8Hl+v5& zy~O=t8=T$nE)WFg(e?%opGen!Dh{1!at+aLFoNq8*0Ibdj&(3Zs>(>_GCB9sUl22b ztBAgeduuAb)Ua0-a+%2DO|Z?-^kZDXGE$N511SRa_<;GFl&L^Y1sm`UiyOH%{jry? zD$abK5kNjTd1OznF(kba{a#b`E)?Is||3R}}rFOGs)=XfY8L9DAWCjL35v7+`u zyvZHF_`Ed+oGW%{Kre-|R0N5?&0v1%9cP%pEGlm8Um5&)y{^%JPCZd1J|{I5 zxSW^I6&cgbbl}x3nblhq^S2B&ZFEh<)pI--f7K4LT%dpx4z-avH7)6mVI0?X0XK+t zMFtu;^g^C_E_2C1?jP&OXlSo}99M|NwpYmVT6xC{o{vPP3_yfUTVKX~B!#y=EBmsy zR8MpHKt4%ZL}ut|-JY2W|ZFzjFjxr}^|;*#-= z&i;FK>A_dkeP?dXC6=8UI%|@M?%y`dWS|8Jh%}2x1%5y6afi$5nDL^m;2#Jgwtww( z^9bu*cuQjV;t#`*@GJBmWEJ3xU`Zva8T_u6$IMB<&@Z3-u96=x>e*nKwQHX>nzV{h zp@uaY;WLQ0j|KD(rVWjC?e#i4_mgM#C5&zw^<$NxB*GaSkborr!kZE~GTVe~4;dwr zoIvoY%Xbu@JmC=@zdz@X9N_AoE^%}-|GK%SzTCF{S;-mj04}tSa9PseK?zX_AGC3> ztgeHO)!`gls1@WOPv`hSkp)ubcQH{)MFqIBhJ8Emtou`UZv$TMZvgQ8JDd;pYju{M zFRfmgf2n^$uIB_4&Jzx9CHyl>VnFAcYqVo4&D-E2^R`)2}z0%HaRHQ_MVapWE5|KMSC($72ug zwxWv5{|SMM?Zkq8Yem>?ms1=H>*u@@?rW}2E)nw&=8zhT>b?G?x!$b(3Pb(U!cqaA^1!E zbSLA3=9OD+5{&{iDAQ3C$;u$OQbVO@U;@)YpJ4qxHWGH>WAvr|N`;o{nQpL>>1x^D zxTD+aSo3}1AQd-`+%W+8C zpDQ0sat-K$Fags=aZ-XU_?Zuz(G0sHzm@_Fhgw4T1PDth3X<1fX#c%jOW5=4THeL=>$frM z57I_#{u&8EdOva4Pcu2RE;1>+{IsuUoiioR;qYO4w19eg0`sD7L;JHn3ZMiFDhVu= z#6Qz^Eq>T62sx1UEKIJ#IjBFJ>3c&gP!GJlHt{N)>(uUra&?x=XjGcxBZd8+vDBv& zW^`Omb7%n94}{63jhxjabGM*Z5vG$H^0M&s!@X8dd_5Y^X$2j9A@Fp1*@A)F_)}qv zpglB942VZ~(Z_y?dWjm2;LJ8-|I))mO@DMrsBl-Xn&#$ig~FL%QgZ?oZX|-{M%yXO zaW0XFm=sBi&-3<0#I_W)Ame+ob6ba<5Afk#el6u@)%y%RaGE5d|cjpUCd3MeJN z1-wLjs*zZD*)p=K!92R@`Ow|3IiY!Y+CDTI8z^1?lXL}Vh;y5pzW9tho#x?IT()aZ zZiaKy{*SSbcXQ9!vs=is23|_eee^>~{~p25K{ZBG#+eCA&A4IjAVZQ8e*4imW3juJ zA9#^|6Y*uQNEj1kQEJdo{l*D(484Ctwg9yX39|yMJt))Y*5j0WT;KU%O(Av;!6dqH z4i;7{iQ{z_DteC3J_>3n8u;+z^+*l~hdgCE7DQ&8Cr~?andpr|K2Eu3GwSx{8rty5 zA{4P=nYIk8T+gE`4`U1w;hfjEb374SgE-1c$~>tA+yWrl9kdkU){|FCt;QbzEjk88LTos830cToA7sa;Zm9fjW)VaFC-wuC z0kD&laukpo{M+2y@QZzqC{(#{XlK+e-4Eo^&xR)?P~ygBv^irZIszMA44E_jH%8 z2iIYs2!pRnUpGW*7`I(nJ2B-&Gw}dXYt+Tx=-aY;oRPxpQC;Ak%Ii$>Dv*4M@YI-p z&f}+n+6TQ}M0tfvSAKf^^e17L$jW*RP6v zNKd?Ye>|jT>(Ac?-VR2_k=tl*lcM%lf6d{gU-n&#Pp^SQky%nV^*TBrYg3;4mgNIn zfnDqDq3jUqB*WgH&d*?CPYIelgve_zSDbb0_5`cpjX~b{n}%L9gVw1t8k{cW(=|yK z{lIB#@8P)dXxUI-jN0L+AtfDn?a-Z~y5iN?780Qz5Q^`6r}F4u;!V$(maTvy|1Bie zg`!Bq!fL!Z&2R?JI8*%AzZk+-`sO?VBv^H4a?(l&tgY>UNi#4{Oe$zH?O586thn}- zK<~t$z`UaSnBTa2*NG9?fobM|oDqUUy%g5}3-hqAV6}K~X%w|J_m->F{vy|IC*}e1 zB_=1VF=AzChCg5h{A3TM5vdU7xWXd*QJ2r^Rl@D^IQ)sWWTsR-+j`u%L%s5#@2lm?VM5DyHddQls>aa^6sfu*3;!U^$UdN>jABcu_Ca=!5};CJZ6;SA2gBcUo(+o0dt#WiK77 zW_sLM#NKZ!^@&&xHL0)C>?D=1x9t)3BDkcJY z>6~L11mppH#10^trw4E{7MJdkn|# zKcr~=zsfraLLWsA`!MlIrK()BSjUdwGoJ08AayDUV_mR@(RaoVwLBtrW1-ID)FN|% zqX9dd%gzWG*knP++Wf0m(RZ1k^qQkmD}ga%!LUL#pD>ZFgS<15(NT6vbn6d9dj35b zYO=q%Kk(~7l_(7PZTe=LXfECwSg%1X;jgYfk}Gu)@=^I18?f@#zwM~W@r&OkJ)BYG zH_PF`e?sAt9D#kOZ>siQ2*2ObqZpB#u2B(DL;(cl;kW|on_`gSmyKqrbrbi3GXUq= zW#nQ1HS6=lA{M^e@*qNH=V$kq44rpZ2!} zNT-kWO7<)vGa7Br*|;whO;07XPylf1@xw!xWyM?FXy=zN{10#hG7h~^YYacl1Ic{3 z)<#|cd?$A!ozWz-=dpq(K7ON{0gJsIe|gY(7(|xWU0Oz28D$?J-(TXscN>}V&GNv? z6&{!V^RjV!_3QHXmYm!H-s*+N;bU#)zdE{QB{6gm^+A~l{F}!Im%8OYGMCCnLPYD0 z+!tSQIV1&xGydzU=WjH8D;gsBFWSENMhOdA|0~8M?ZfR!>Ar=_&f@nUy5@=s{C@Xh zPN@WSYT{ce&Gm6{kg+sU6e5*qHuy~Y@xFb=$S$m=kT@b`}jTc5H;$dDCO;5=t zWu=MgvlE);zWtFgbA*uX-*YYIGv4mG&LwoX8)nYas>?7@wjutS)7ez-f!;rbpX%?9 z!X-_!fXlaof{sEbU)WB}XuxN;t%*C$&dqfLKX#AcXOT#}#O#fSbW=agO@3=8wTJLW z=nwDn!oPKSD$!lJIg#3U$)QfnUdcAuoPAThjpf|ekJ8B?I`T?=9eqsDsN7`grjnz! z_V=o=DaPKI5Y2UUz$Up^IRG+%TF>lCntPu&nT^q?mDzkj2*#N=MJ`^`_Ql=XV#3hq zz`aE~@ad3sSq~rYW-?UW2_rMTZ@8>e_^6vrY7p@ed+E7?AVhdk1eIcZ9tgYC@3_Zt z-KPfG8H)7XLuvoa$%(|MWFah_$PK+TiMM0u7f-sv>ZwD;;-<0Z-%8v3$=?<=9)4Td zclHApj;@;exk=_ z400O6lp~&4Apy0hU9Fp|Nzuo|sXdHJ>ZP3O`=&x0%x1(zME87{2Cj4vZ8s?oIT;6| z2~(avJ#RJhI7Y)hT5evAhDF69NyDi7D+kgXuEn9CZlm}$`lz3SRgUp(DYGZLHGf|G z>`#uJcMdZmSk7(FZDv6&x^+52GMu05jT!-$A2Ea-G^Pu{6R0D%uMsLf^&?5~_b(>p z7!m4C^!d>@SVW7|Lbst1C9`~%gdGpd--1OG+*(!IkxERscTJ(7mHL{m(bVsJ%`6bO zUY8N@)@@b%Hiv&3LT3dBg>cOEfvN0WFt%|F1iWA6b3w_wliESq8 zrdUU@|Gbl`;V^L)$vtT($C2}kjn3W);(zMaZML6`w1!c+`Mo6d)HcsVP5J|NHg5*6Y>L)AgLch+Y zj#c)bS@-7PV;@>>@(tSBw1lkTV`nz|uPR{U zg->wsqsSW6=TUXf|H>8lrs2JF+1AgbKJ5#)-YA||_2Z?oEu|Q#P|OD~!B9xqDG}cs ze{_NS&w|C%2#@2b*lwvRwf=lC#BG~_x2t(W?mavfG7`YmE%#z-@F6NA!~b+=nq~=ZqJJRY3d1jjk&EQQ!gXvX@lms$S~`YOV3Aha>F*9H!XCay1ad$ulf>V6hcy(~|}O5T36#&sF`yGEBxe()4J!|tqGRH>#@)U zjRSd&O%ACUb`SrpQ&+|B?-0Ofbq$OP41)Xoc6MQF;`uE{q0Y_t&_S>YHN0896%6^Z z)n`qNLv&Fu{J3)%TRpv~grn(VODmw44$jf@id4!xDwoF#d#;X|Ds1oe0FEwAGO1pMthE2Vd+ndL@E^?sTzGO5>HpydZ?`2C;L%>!q>h2^lI@_=&6CB-%%egf=BCY4X#LXP z270bDetiqpB=htt{EH(y5I%kte&Ki2r^OpkD<4pObDwBAEw+wy3Do=fDTq7mrVc#P zQKYjRF#hFpxD4^xvOsutwJQQ^=tw4;!1rq3|NHoTQ;f@ITx{sQrTSJtC$xv{;nxHK z?J!Yy{^zL-t>$OTz8=$d*zOZaYuKE-=hV?=;#e~hblPdCe_M%4n(W8!8-}PCwVdu! zTej!P__>6-N$aqyOo1A=?@3vFLq24?H8UOO_YjS3|}&+F}6LBQIk2wLP-Wo&nWa z&?YYKmjtWmeO{$c9v$#Fqi9!l7@XMj+)?1Su3BrVDd(CoLU{k>oZKe{?~L7IDu1IL zrij?bdVgWqm2renhtzEOczx3Un9&SQm{o_KpEBU~GVN0(9zrJ?qF!dwczq+_kTrvN z#CTQrNomBM&U-PVWd;HC=Oz8zJ^_@Cz+Pz#R>ztcJ{prCI`p_MG4c5$)=QB*^k3L6 zZZDa`BdBX9O9c<#YuR$6WBq{R*r{U|wcqEXmn5Y$(3(F_Y9?O#TKRS(2a|&Wtg%DB;??4a~bAxE(^aML=f!>mh9-z>pa5QO^E;}`mivP*iU7G z_v2WNrPXK$R+~%XH?CoDkHjWH-{yMcaQh*M5ADmppBdY_{31nroS6yRlLf^pBZ`Xl z#30FE*?#ed4}re1Kfj`_#I$VVPh?c%fkp!wT{A@S{eLCzf}Y{2pqiUL-H~*z>WAry zYVX5Do-entEzfs*QiZM#=}W|YuOB)F)JPk|5DEDr%j9vd`gFyE$Ur7Bi`jowaoF5X ztb4md+|HTkrPoPVvzuru8s6%zgyzGqj?4+xwSrS7tW5VL{@#8*qtYdQjq0eN0>3>L z_R$lV>|W5MY!YpjJ%tZoMB`scUEoY|F{_(x0`(8#jd`oNst10CHG`DGc$g2BT>YTi zvc;6)c+_`-I!z_-S*gWmt>~MjT9SI$MCFO9q&PKVd&Ee4632L#5Pv zQjX#j=k|RWA~hDU?lVp)nFzBl-`zv)S;Ij1DS>%P)p8*w(_WJxlJ35p0H@lons+^{ zpZolH=pS1g^z-y?8fL94h^}1`Qv5lbDb$w$Gw(W&@Iq=M*SnefKdoHFC>?3o?cV!v zPXG4Q{C=JU-R9LZv3#@p)*-cxVOtI>`I)bEmdrDq&$ui;j6on3HS_!t8Nl@R8KG$C zhe2O^S^Fv@f8ONs+r@{z8ZLvn8a|x{3ac0X@xjRsjGD#7Ckcf$A^Xdp zwD9zbA6>d4F&e?-?v~nYJf0%Ig-45vXeks@)pbQ1D;6)uzaGd zn&0!BJ=#pMa*N5#pG*@$ghvYWyf6`w9}Ih0BXeb-UE30}9>{uX@y0@gj}y%rdG{h2 zk>%ghO^g2dn+F;^(B*PIzI>B_pK*Y@{F!{}T#+W&@ddjYlZ0Ya3q1Xy}O;09+p+ zD00k7VP1{iU>8I8DiY|aG|dj2`kvlxtX$leQ*Y6MOXO;jqBHS^T~7Tk30JMZD@%Q< z?Y0*JyL`T)QxZ`5@#Y<)#t`tg|AF7c#epbn|5F7WIsV5yj5w|6>y;+YBr-S{3lAN? z+FZHJby2T&LFmP0y06e!p~|V`csCn+hG$wb=&g|%w@6*TrNgT+v{(!DafqQ5v}b?Q z;mtEv)xdur`(8MdH}AQ|QnouOy$q^D>2-y=e`dFuja zWtMUzsDbGsN1=rYCsxkPp3mBK*Bx1X5}4>)U!)WMy2J20EqW?Aw&*RqTC$L`*Rvs zSqDqZyU6B#Eksuw_-}28X}1X*IEFn<7d`=BfojjXohImk;iG#PQ9rAa$8dtN5?fxk zdC^Z!U_$~02`WcD%lqEVQW=o{Liq=mafebqFfya=a-a(%OQ7rxJ1-3q*hHS6&ev{^ zK~J0N*GBFvR&>`75-A|W=LK1cHu71}i*CfA;l<}FD`PYcTC(@KJ6#*>Kl)jO?fNAw zY>K8qG1y19rJFdOFbh_JyJLN6QXE}3szu3boIbnqdPB+U8?QK{LS1$pl#jH$Lf_T1 zQ9+z&vIdSecN)X?PLlDyWPU&mAAL?PruoAS>ftToj2hgRtHnz&XenM1N_Dt-uJ#@} zYjzvYIv}jKhZv($L_xpBOSFoEap?hy6NsQ>>P%<{is=58Ck<0%Nw?m( zl>}4!N+thxsQ0?&h#@;A@W zA7a?i<>~j++4v0OCUr2E12%4+c5KuNLPQ^%I0L%dcrc)s;UuIXFg7;ttf2q6)hws> z?Uw+qm}}c@0x0~(YzMK;@k|K2iGAkMw^_{-Smg)8m7R@|5fg#d~i=hue!v@M%Dbn5Rcebn zYtWT097j7{I&q2AE*IlJ7f;6J1-#jtKOHIHR`)*fS9-=V9kU>ezP<1`G*~%n`-l^* zkXxOFQ1#P{1pQ56U|4!6onH%q*H15Aq0v-^D&ZdYI2)|E(Yb%ub>4VMZ5C8g$?6S5 z^LZX;g+J*0Sr2eq;n=^e@MHzf6#cmZ(WAf5hlY1LoueMfz8~XALdaN`lk6Y&eEs|3 zk?!ux8`bSBYsDnRcc&PnHgJqVU(UZY1aNxxE}HW9ez1@blq`_pIPS*5!G4dWJvfZC zCn<*!NGK&(7lsZVf1E=M0}#4MLVv7WaNlRib4rl+ZgZH*Ld}tD^3W=*D>@iW>0T~l z>vZ7jb(~;vD6iKz7r~`m(ReE2cUL4|m{B1JyWP5~4Vw9sHze;ULQS0S7^$xG%;(>kQs;qGqG3kZt~OTp~@JA76E z&B;cnVm(ww`pm1j4Vgk(*GeVCa@04G8V~b=HFwZ{ev5_hQRV-r6?;ShSBfP;D3D`A zblUCx!4W7EVWf*m7lr3}X%f=k$9y6SW7i&kr){dv|8!iE_q8imjm`A+-iXwMfj*kW zgDzDG z(n?)JQce)oZmV&XWZw$6u5Z7hmSz7t90`1*{(?u4smM#&En8m{w}gaZyQCiT zBr_;6USnn{(OCGJ;zyZ*r~rRqzfp>w^cYOv8w945O1AUQKCRIa0aZxa0~Ny#5krQ0goL-Q`Nc3+&%ifq3hUJP20t}Jw9WbFi!@n4uai3<4&__FZ!`&0SX znKX|c0fNUFSSn)e_l(SQz4&V8qrH}_SU+@sC#iz^x2}rUD%TaZ5f?kRcLpV@u!!&} z!Uw50SnqE}qaU7ghMlv&1y8A3`07P4ef}>u9E-=#5LV%C5Dr+Z|z znTkU@6YD1d;me-5;Ova{Tqukf1cN8VjxnrpCnd)_I>9R%x#;VXNR5OGom-F*#doG- zG#;bmea#_Wv=ii-ui#c9J0Aq(*?m0$}z&@b@%~jD72wvVar$`ha31~+_FC`-hhR-go zLm%YmlXWOxRg@5>aGn~C64J<_QGOk1y8n)3+MAw8Y$Km@R}__W74CMJL3JUEK%BNr z%*@N%mnNBxT3=_!{8m1;)ByU~(bR)U)WL;q3=;E#eF(k%C!72$R)Ct!az;*`QPYWQ zLcADk0KI4a2TxI_kX(ye2u`>(6@MKebk8`i?J_sD_t=V(GjeF}r~L9}doKZa^=`!j zK4Q`jdGn5P)TEW>Px?Ih(E>g6(jlvr+5fds^0S3*N68RvpKi`u?msqU=G z%Y))X3nK?7OE4-s#TcvoW`uN!q=1Y@K)NJk!06lN@1Hwo zf9&ks&vW1R^<0qxIvjZM6gS)(_sLt@zQ1eS;`PSoL}BscRts6vddnxAWI%9oA+D>X zUPwK_QT%fBQm(|UZf?Um9h$;oM#(`;1H0CF$iO@49-qu2m4?^UV~JfqeP4*g_Ma4C z>Iyl#f<(7C2c$ynzoD7+q8PG#e>#sivpGY(`#pYHL&7TS8Rw$W>Az$2Bt%s^K=;8I$KTD!P%IY7zbt7l{ybt4AAj4Y3Lz$o-R1UeM`9iB^Qq?qpP`0b)Sgw|b-#Lf`29u_65{kU znLYdU&lx#yVrfwVvDDTZ_wNgZ-B4)jl_}>uwAOe2Clj%n!>E#nn9!LjeMdcHVE*j* z6wA7&M;%kr{uk{ZX8r`tKHAFpb9SP-UWgUgGcf_->mIF2XlA?QH>H98TT1fMXpNOD z^~&m}VfzecaG||mhl0Q>M{hfCs~`SY{1^1E&lp{!;(f?n9~^(*5%Q;KDGY~33H+Wu zhTte1H$=xevuP35$*UA9=&ZltyLDw0(P-r+8V#{qygU0bvcde@wGC` zp~E9VOMw!Q9*6JqF7zt@UQ$qitt+;=rq`Bbzrroq!9>#h5&61K0)@NOkxOP^&HixG z=dv@tFjsW9_Uw(JlB-=jkkGgmY7Y$-#qdbB=g&qjuZ4%WrzZ*7UC#a3-|=+Zogz-u zf1lQ6=i)Ra{10s$$fd_E|1wLyHF-r~T$?8y!U&d^e*%nVBDzN3?T2Ahk!-ql6K5LR zs#7I6wn39JSQbd04VM^44p!6v3g^FPOH?uI}9=ILTK4skaMgnnkB6+pwStTav$N&i_ z=h5Gs_KEx$2H>c5*>CA8l&A~iU}aAWrJ?Ng^Gm~~Z@lmvPB-BO#MP|}j=bi$qbIlI zgP`}E$0&clJ#+ep+&>hU%=pEbr#?! zn>rT{zW%$vXrLcl%J|WR5Wz8ElzFd;k##4*;@DfP&2gV{7i*T`*iBzY-oV|($}rxE zfwcC(nl(*p0-+4DWfR3~Di&9b$h^Y{dlm$}%KQl$0quEF8*&e_OBzu^w4j=7DGwlw zH`15F#{M+U<^3Y%P=5ViRUT=&NW1BptqUFKf4RY=e;0`y$JEA9(baS-()O9H_iASN zdg?sSYgyMrK$l`_67)Gf8T5mSa$UCJe`Ujhz$@b^u$H}hqYG0M(|(okBjk&|S+L`N z5?)1#1w<-;eN5&-p0B2N9R9udx2B(z2YmjJ`Hl91WF+w6m4tk-k8JK zwO@VSb0X}B!HB!OM4jOTnHivEv+51S)UZ*bJ#>e7j%`z@-x?jhyAZ3)jtC9k%oj?4 zo+arb-!6_=5w|0QmO`+(%`z{m8&^xRJsa>sxcT2)(5Zp5I-B~AkXP8+Q3t$H(j&t` z@hBve%)2Tw!3K4&q6L#{B%w*U1%*2Apv>lFDfq6}iN6z~B~O2wyyyu%aVeGJC_VFE zRB2;7!D3xG5Jj_*jnKTGvyz+TZMURiE;~RZiVKZ?R!|c$@BUUgT;y5#PkiTGSw5JL zPOUPIi6~eWYFehW$-rgWWddPXTJ1bf-Akro_f2?2Ht+G8W1jzsM+x3D74qIOZ|@XW zL%RpJfm;D|Wk7`keo-yZpu9GGny|E~_fB)FPGXXFZmcgs)%Vpa?)0F4`v!q(xIP*4 zbj&eU8+&BqQH%Yv#Y<|G;pf1gRQ&g<{RyBnXP3fs=eE97yZM_J81ne+m`ByGS+%{I zt)S4vpkQmoM<9&xtuyjjBldp;c{rwTjI=fcVi$6g)Ox+d-GszK0&pHAuP&XNJjT)y zBkd!E`fNxd94&{q2jJMXQ%P9*V?_A*ASA@u^1f|?)0;Is$Dg!bZ>mpMEe2(qK2uwT ztz8)%k(>%~2!C-;wmPlEo7lN>eDeYGtEE#t#{Z7TzjHoJ2ZCQq-z$l~duk6pW1n`P z4FEZGr^UMgm+94^@>G-K4A7D^g@Yl7oWG*_An36*8(U)C5-oV-Bl2t6{qipUgs0sG z{<^iI=7GWe}s3K=M$4> zN^!~sNp{moW?P7AA}UwMDVNvxhv5tR=G&mO)iSrh*H?rqs&&q+eAed}o-K=I0qY6k z4SG|pCZgmdh$g27z{YJ=g|})Q5X84<1z4UrXWT+SpvjJP#qkoKw?-gkfP3vncYbQ3 z#^mCzi%-D=FZ@sM#_t|@Chq`kJIapO&&KSjKf{bqN!lK{o=_@h>isch!s78`rtdPO zp2Y?K&BKN*%^TDqx5_ubQR!g!#0b+=1ZJ!yQsAV})-}r7ER_3Xeoi%?Z;_1!-*|rt zDdv<)?nG2XCjga_&@QJdGXwh)A$7af#W5+oKCv0wh{_6n)RM88;snv)$6G@0m^0k@v) z)XNal9m$((3J9)qr%x`ZtHE3M>_7pSHc~rd8|P{yOUan^6_D9+OEXDmuoXlaOjlBC z&aJ;4n|=4E)XDd_0@o^)pk*@f*ENtd1QiJdw3Zyp8PYb_wBqi)8EQBPT0Z!0MUtAk z+>qZdM0eU_icwD&PW?qKgX!fwAxlHI1X+xP=t@1=Aw_E{(jFu(c7&aeTicUdwXmTs z-44SdE(<0_=$xHvE@e%Pi{ShG{%W**J=)cPO!Rc?T2#lE@Kz-JO61lwLdlwTvOj?98Ji7h5RFfW|;nj$u@me z$;)W=~knHHO%(S%X=BrxzNp4{I=zS(+>UcYxH$Hjn#%XLE7 zv9)VoFIOr;FNmyO>msZVw$tE|8e~B)&-Z6rwAX#jdsq+m8b02VdnKGA4R)GgC@|_T zix0M8)ujoRQvm=#Ts z1Uwh*ujlFa&VGeW2+kUZT87%UK3?0#2j$NDFOw!vz99Zd-a zGmj!G@oL)%8vZH?@xRy6tTJcIo(!92N5JJ3{l7e0+U$?awc8@>ByqH-W?MIK^n`Vmd>hiuoa^FbhxjeNCci?aM=NuW+qN{`s=b(!xb^&S{-69*}YV- zr~X#FV*K|)&GYNUF)|daU6LW}qhu`Oq3!y0hB;!$aQa~&7}6FkDa6MeS|sG5KQMZ8 zC5Y?0$BhUh&k~F`8Co~OU}=BbthoA>pq^2$G~maXoid@27sulhFTJ%LzS7KisHMOF|as6mCTBfE2V5i;x;1l<#CyYsTUkXIh9<1Fq z6}s$miC5$+R1#%oKXagH1JyrRQ9%@?WJXKxm)_0HpIEw1HttM#+@$UG$g}+E4l{87 zLqZ8#O2x*2Badp;$GIEEH`jH9(qavVAUHnqnMqz1-6b&+dtH}G(3)#CP=r>$5PXaQ zRqO8axDnP}^XX`zHciJI9X-ltBlj_&UtCN&PCrZ~m)DhGI}i|En@p&OrDKow$9mohV47+_TFe!;FZQHQi=p;NHLH`4L=f=~q}n+pOdCZV6TG_?$2H~Z-Qu&* zP>wdX+&j4fn2SOI^K+ive}Q{O=l^`q(G6ntTYo5S(v9lmQoZ|Gz^xA=aS*7%82mav z_gW7ZG^Y>*!%r4BFZ=X_4S%f~Dedsz6vOmHA=wSSE(c684!0k~iWL6Y!yVyv))A=q zM-*JaJ**)V?#tMV`Oi_)z1TVUu3MaFD%aZ7W(&xJ9T2>iWEGRGGH z2xNz`{r7(TwdP@_=!}d}(Wq}NF|NNIRGlk*#lb$02R?3mcK0%TTz!3e$o#W3@NZ53 zInxVL9~kr+kl$SgBu;&)A+*HE3SU&Sk_UwJwiA;*ac2vegy%@M(gm`n=*qL$?)+vV zAZ~#Mnf?)|lFSrF33xZ2i?$t9e=T&qrDM-fEvCHtgCAZfOL>b!(VegS8F*cmj~3(h zcJ(y&V3Dx3PIx;FWPi_!LsDX@(^esRFZv(8b>0IzZb4e6e(?hYuI*qYt%Uz8;23HT zSyL$+Av8q;XvdSrs~Vs1=WaaJjmbDAYDHx_)5C0#bWRud^DLDBSrJL+#_mU_?7uP{ zNPlxHuN!IB`F2Xt4VR>Og$b?+K&y>+sfq(UXSrskSPQFm3T63kE7vwU4ZiUHXCFlK zkN5XbgiP)C&$TZ9Sy&meSiXwO8NSWi@5KBqkRG!2n+BSk@5(-Jciw8Z@zW%J6~!p{ zSk)8#tMzx7b&c80I?S@2jVbCs*JUAI6HS*qp$EN>>1nrY1}(r%*@yHf${ z4cc|d?p<50ajlbi_CDJ#JI_i(#U}3_Oud9}@sG+1BZNGjUEgI4wVDwATBPvf#b!0g z&U|pYU7im2<*CLG%e@~O;jr31Q81S7d(wxu^w*SV7P^epbm;4-upwpx#^pj|6$i2B zv-0h-+p{YY`kywf+0tf(VqworXP-hPbaB4FHA#H5vTWTQ)c~W^+IL$;H^`fq@$304 zfsOy+`1^Tm_j-3d6eK_1x97-z!+x;a|DzcCx6|B_W)d!FNVQs8Iu zEY}9QKy@-&VxhlN;Jlyr*C6TFI<$rddFiriJ>hOcw`xXu74TfpxQ1f_8zhl{ntmzYOJsFT;6Vu)Un};~W+PwwxFcD)FaZghOz1Rl+SIad&m-h#b z8t4=5M>3jEHzKe!pU@F^`y`;IjIDA?SQtyq>&5yj^f*uad>weV#uU^r0*_9-2dGL2A z;o>eL%HsO>^Uh7A4vCf(!JM2f=;TLIqRZz&phjMe+%e$PuQwV+w4S$M7B?bk)ybC! zAbaFBD-tXN(?*ybyW7)?@lfPKI&gVkzN;#=Gd=XJS)XlNHH2u)!N;h1%q~S7>ltDy z7lq!L8&d-BDzB;+sBL-BsH zy;#{q*eUF(P!hMUdK#}H!Ol1y|Aiq zXGT)d`qR9qyGZ`?!s+VTu; zd^(nR@Z@Fjr%}!TUZ@(Wd><&?fP3A@$I}9l9yi>j$l^0Ypeyr^8e}R#+=9vt7sSM| z3il3jh(FY%esPGYBe1jyh~%j-ydwQrSokb#g%l1jgcl^%;7mWfHs$}<5bu6Wl3lYZ z=MyTE(o{=qzmBw7Pap%Ty+^OeZqB+Ns7M!U>06qi*maqHx)9}ngE!mscTnlx*LEJH zV6uVmBPQIKdlZriP$pZEx)?GM$>0>&iLw1mw5-eI6W;R+1f!_E8xWrYT6pgI*=ZQq zMu~e6S1O7I;S9r&;(8ah0eJ&XPa>9MfI?L|NC#Ct31q&&xME1A_7eif2GwZ0(;7!r$;r z7~@`!I%mD(no2|($m}31+%wq5OWwg>-&^lSI#YGV;o!OjqBAa4XbwcEWqQw)>RPMw z(1CB^QN;YW5%8(x@Cu9kw+CJ~p8Ibz)MPZ8VcL}w#FIJ5nTV9R9A3j-i)nv=f))5q3x&&0h?d!H@v$cOWZBdO$ z^cz46-ZzDP_$u>R^xjv_>~&w^YGC@( zJf`OhTmF;;8Q9IeKK|IFzw&;G=!eQzML4P$@DXvM`PJgkQQDm*Nz;Qz)iQU9eE;7I zpj!gM{3LqrZ+E$%oFvNnXay=1!`Hu2hL+>w!0|qEud2Jq53`bnRaMXo34w)hePEK1 zrV`gx{_oxxV~(1TFF0dSolBZm(|$Z4x&28ie*z7%xtMHOmw||riqdeuYiTEPTri*m z3E!_L!wLBf`+1{f$^4(Gk72zeeM4J0y5SUO8oaw@3R9pNn9 zeOpV71yldZYe|iCKRPh|Ti5R4K z;(ZH^0u{V5b6AFh9&;-{fy$of-BuO-)|n}B9Wlrq_h=2>5c6^`OXOPJjpH%1Wxo&q zalj-WD5r4X+@9AzPAe)AwG^VZbl<{Y%vs;^jcY_ohU54##*H2_m7;x*Idn}d{5(<% zoFx!QjDSMO%hZirS|DNR?F+nR5qpQ^wqhykE4=Apl%;QS7K0e|mzlEddt#|J_2sK~m3i#l;Z+ZS^HwjCU8t zgT${R9=l@(>W)P59}WkY{4goE*&t&5U0d!x+G}aQ%s%B^A%#bA3)Oq0x8<^6N*?Gi z@BQH??1fKznA8{2>9t3!igz5tv=DpOhBP39r)CbxFFIb5cq22?J z9|r0E2wdqCq8XWKh=+@ALrBzJHY&h&m{2Rmqlm=Gn%ewVGyS#e3Levc zyqBsOKv)elp2KnX_QToOJKpDV!T!(}N{nVoUeKrxUpQt|MqK0r{lgEbux^SuM8C}R zlNz{v*VzapBy2QcO$8Qr>YpNI#SQsKsD1|0e2?cq9ecm&^(GVir}4Q|M3XJ#9dV;; zn-rHU+iQ*iQ0rZ#8=!)2;TskXvplT{GEKM(_B7D*u9Q`IS9$$atKGqfscEGirgac} zeqw3lg|W)lp_2zKh8Fte_fwq`Vgjvs%xBtPT(1>tX(;~?O#O4Qw!c9JP%QN<8IN0+ zg+^V+2Rfnt#0cKh2anSA>9!N1l_If=qraV5C;dBLnImT|#X`)U+i3#vQ;`lbJo!#u zzfDE>qF^})Wn3!mo1)HBuE(2viQLgOq26%Xy6b!d|9BCImuZxPM|J&jTl@tJ{FNAZ zqn)bMY1>F*XT!c;tz(vcN4(pfdQB*PCr2l*t&E#zpuy&PkSu0>y$f$o8 zSC9o6e_H;VX+mGhk5fF9&{Y*@^F_1>rxzRrt^xYM`lr|{KAc}V&i;J7+kkbZImq|Z z@0#_IE7Mhw=sR{M(bd09 zyQyaSy8r(2Uz9!j@7d!q%S*q_E#Xstt(QKy!9594hgA)`#fAg%O;#7iJjphsAh~Y zugDVxE(GiorOgb&|Ks{GUWQLvdihO+ooibOk&%^a&ETDq;_r|GGrYD5+uiNk#jG{U zfA*wv-{SBWRgH~oH+k4zN@YJ1J80XlIrgNqH>0lmfqba{BPHtHWKAY7QrLG)<=Jn} z7nIF;5Pxobmt0eVo9VF{BV^vqm4&jJC~hOOfcvd_g0c!N1b@|!anDSDq6LI4px-l! znI@hYKoTTC6SIB!^}|kpe$lgZPkE>aYR-o|l}lD5H(}fORqsH>gw{QXW0jr@p7VYs zFZn0ys+u|6=XUry*(MisavLiH7D$EoiRUd^Q#Z~1GIs4ppmv%6sL4gf|+RWvMk?JVGm~P94vrnb(f^Ffi}R2LaORT z>$LuongSEa;_BMR0*bifsntjj?RLF)e=OPL26bY zumdO5lTtU=7bxAlS<@R}KunnHV#6I7YsQL?3ToiwwY9mHD!j8SxO1o{3pViQh={S3 zbJ*fwIDAn9W{}#_xw3Pvd*x>}C7!~#9V>MUsWZ#O@8p?0xz4pUpbT~@H66aii{+7- z7d^FQiE}TXa&s08%D$rCQu2{7WGI$>;MQqd@R)mI4wzx&wopD*-B|xveVH1;HFRad zj;9fK;QBzQut@Ix@`4%YjjQ$Co2Zn8HX$~+Hs+EkSRh85E6e=JX`x7fpg8$G2IW!9 z{3u*!W2N2(l~`hzzj7w%R2c5lMb`ljQuT^?{4;0c{n3^agW0-sj$QbWlAGJmU-5;8 z%Wz*d(_efM5_Wb!-8VQO6(7f*JdJL6X{E$BrEb#w-dXt!X0c^S8bC#Q@CME)Wc^z! zMHgv#r86hOWSLbzg4yY1A2&4IFV;92>0u@nc^}dh{BkhAtkbdNAW&Wnn6N0B#9+j% zR4*AorN)nYB3Y*>ox|~iyr(1b$E)&zj4m0x;}04{)qiB+yO(J|Gut+R%5fz5D)ACI zz(jNAsY8=yXul_247nH8f86t2^w#ZE-NinzsRrqq@&X(Ab{`er7b&zbSnEoWfE4*r z>jt-kUH&8mf9sy~W4>xUCeL&>qIH#7gp|^zaowiEfAKPkt(f71q^gN%m7*yi&Rv!I zEE~i2vu58$B;9Wu<6U56aw;PLMZ8;X7I>muIOH()_Cb!Ig;D>J6VR_fx{L7$Ec?&B zMG0#FBqxgW^eNcAPXk+rx3z&Tm*9!=XIlhd_?%Cb$}agi+u-Oh2TT{9lm+mYH`rT7rJ4qR;gzYB5g z8Ir~IY650sK{)>YI;ZglCli!t=Y#6^#_oxCpKnCoG5E%2%9}BAcg(NisX`hkneN9D zRcFw`WbHF}5Z=>|Xvb`CL748+j7jhaR0Z{Q#wBKZoo*; zi{Vg{aYTJqxjB9Pzet*ZW(!{OQs4KpfB{Fel{fQ zs^5#7PJpz|bUI@8rg>EL_TxSzM%k8j$MHPULH}%XjRPh?sTPVj`Ee0g1{?CBtHf_V zdT2&5yeRlQF>C;hDpabER#Dhha4U9n`}+VeJ+{Jjr;~4i5K%WzFJDLEZ~jEVWz2pz z(aRD))o%Q4pa6HE(QT_Uz6AJ+2jMq33Mm7dw^G)@u)Yzh1OkAdW8JS*q@?6C;<1qS z01Th-sdtG8uxUa&yX^(X|G*AA6GEQ)#!`lbA{;ylp3{1@tq-&c;W-`lj(tL!AWa=> zmHw#Ju_?wFi0)ld%SzI?j(8bb6-NHJc7Cl*!;{pXJ%S9Tq{IKk)AqpVAw5j6fWppP~(TW=}I#ZJK zjjx?gVvUsg%&A!JMoqEriQ-w}2myoDLVuhYfrQ}^>+l?y*4jF-xNf`z$b^yX`7pM7 zl{0lv2R*=Lvi8K$VZapc{heu(cF2Qh%&t^(bE;*sy9F1vhaK|1vZs>rCA!$2Eb_~+ z<$qv9_Q@VTDKN_pJOCril35fA$Z^lWWamC_mqSY!sZye^w*L!ct=`?9Gx65KM%TDv zt^c%M^49avcg_pR%ksv>;vX!aD{t6321d+!vTh(J86GfOJ-G|MwM-H-${k1&csp33 zOy$hG#X2yY@c<>6OeH>eQ@Xs0ong-Kqjz!4hWL%6KvvFIhv(=beh#g)1O2?sZUsg_ zQhcjb(;@tWhVKvXy}M`cp(A&TKV25ksZlriP`i+&#uM^Oz;Q9fKVT@!W64fu{NaK3 z+3~8innK{`N$ne%a-CL^>fu>?+!dVFSrE?R&)zy(v+sd5gG-m{RC%Y-GaTO^&#g!( zI?{lD;f%N~^(1s?z6{6;nU`5`7-(C)TCbJS6GR|$#IM_g;`PjFOm69{K^i8d>*6TZ zL+e-DRw0ESYXy?NOJBc5=F;;@ZP=H1Zu1#7>`zkv3Uf-`)Wq9{)?HxI*dE_|KHkCQ zUImn3e$Vyx`VDL)FFnpWyl9Ha}y8`IrbusU*k#lHvcGrmN6 zjJaq%hnjRT+y7Jw9)yt0XD5owzJ_sp}A`ce*VTB$pI({HQPN{Acr(Oj;u($n@KL6mFIMj$b z1?znbK2fH>>caJE6o@Lgvju72CjY>5T-3zloNsHPe=b$xs;#R}uL)G+rXD8oe=@SjpYVM`ZQe5exshI7i%9v&XK@V~$$AZUAc@H+I zPva`0YVFZGLIH2po(+$QTZ74ceMjy#V4hftIEOb`n-+7;xBk5jr_In$)r-7T^ol!>VrS_w_Pfu2YaFW2 znZ-mg7(7@ntuZFiL5|J2UAO^FJl3-0285aSYOyWl!-s9O?jZcp-YdME*^(y zS0Mx`#cA@`G4l0FWTPb;BBGXut^Jq-dLqSof~gH0y32*g4I3up!p^qfgSdlZ|AB3{_vrhYIx7wscypTN zv|=^zNB_=O7ouzi%&$NN)kyrw??@2)_ej~`j-2p&(Z}I&J6_t`nleA~uO7d<`z(X5 zfcs0%j6HF3w#BI`pptcD0(UYJxp;k)hVzTi^i9Zy9OOs$sjg0Hs>ltK70K7~V|lm+ zl%wNOVx1Cre$RD(_IiFyj!3p~wqwvw&Qxq?lsWFqSW8;YdW-fc^!X^*=WRW|y3`g( z$>O4xIV(vG-&k1U-Vu{Jyf7p~lYGSb6ud*j6ZH;)U4(JJcMPcY zQ#swJbY#WMd^(S}Piue2bi-M({qF)S9rv!u87_vo7X&9gC&$ZQOM8KxKfD} z{?c!5O~M>NtTP4biD}w$MLm6O`TaUdMLNq={OZC#Ah78yCk}~55=4OBeG2AZmYhum zy@Lw!A-Ut_gpxz&>B!Ou#?8p}wm|6qvA;90xtBL;v-!xMk{^}74$8l_mM%U?leQf1 z!l=afN)|7vDx4Q$&Z95(ysrT7%mJ*F#s&rY-PsO-_}58s|9n#Pos zh2uW*y(x*$T_{(W-2t5@EO3<0nm*W%QhQ3RNEQ20h$j5Mxeh zB0Y#LhrRm6X@2GDJt;lI-5h~;*-Ee{#<@3MIv~na2P zD7^rSxk2%elv*ELBc%n?9K-_GP-TVBMZ;DdUs&UMZiOi>4A2%~yw?kj2ar>)=z&!; zAQOCB9mv&8`gL+dNQ7B~R=s0ejX>o`xC_a9O5dSvn#J4vnrbCbS7HJhenM5h8-z>v z`Bk&Lj*;|~?4R?Vjb_WAnpk?GT7}dixl9vvi`B@c2#sBN?tUs&1L9Y*74hS9s%)|V zqhUOzH0ZHj6VL|p0z^iyChK0GbdfDKghdd2dRO+QpWZA7QuKHQIobLBk`)a(s?P&; z@9CxeS6fNc>9Z&2OZpQ_l0U=z9+Afh%`;Hnsu`#!jA<7+dH>&TE&=|NY-mtYpfhp; zZXzsuU0l={fYH5wa1Ymnzv6Nt;z3&C2?z-%o({Vsf|G^D+!0v7$8Zt+Y7gR9 z)qmuX4Ys&=!!>@CC{Xc+zBkvdF7jHLjM+$!#}B2q(X(%Ga-o#&{L)281K(%!IO# ztT%XE#At?KE5q;eS*k`?zxPr8(~R*3%!>2|*ZEjwprxeP*BWtHeeNF~f-g&< z75^DXYH1;+KFJ)tXC27j*Mgy|qxndm5hkkRy7IR)3p1HB_Ej)1mBb<%2f&BsQT!9S zF`Dl*D;%*a2u3S81^AAhU^4VXfQhAg+NONqSR$v#FSHpR2_Cn`7_) zEkP@>R57Y2X_u}UxptZV!8Eb@E8Xtc1 z@GV*8?Y|*R)Y@bJ6vHRNxV(gWy;!*tESMbWar>jc*Y;gu?a+dph~C2FV2HkYWzBDk zzwXEP0mB#@ul(jX^^Uqt-89ai4Wa38tFh-if1`(N!aGOGO+!Z}`Z70}R ztc^6nr3U-C0Z!Q}FcwZ}nPq;BPSBK%pb~p~4VqaF6AUC+1ZJqJ9-Z>=L|=J|%jW{m z9q~gm`34dSZ~{?5W#CQ>=%rBXsV%U|cWX_BEcY@2w%VqaiHkKC6Ph$74G_sRz;;U3 zb#hqfwy(ubU|$h~n2wp-_{mACkZ1XmJvHj73Vmx+%su0OwT2cA0<|+xtBby88a^2_ z&_)xyy>Xp`i+3wrGzi8~b24uzCrB<`6J;0?zWMLlQl-jSb+B*woFGM>t`f+wv! zY~(0cDEU4E#-+78qAs{$Q;kh{E%8?I3c}V?g#AIke8j@JBzA zX$N-^+6dv9!^*<~Gsb-XLQ#J4M`LBMOS1pEy+|f}JycLE>|Kc?R!g1xh5`h;70{{V zu}1{A{vlXMu{87}doaFJu)<_by;!r}oIdqyA(YDMg0GFoyA!+dv@N7B9({YM`djYa zPcHEy(~M5{L5ibEx@^aT&4}JkUEX+;{Q$eX9L*i8Gu5?*Go}*3*~A!Ma)ER|K(*A} zoK;gxd3ZV(6XkEKzK2>LvwW^M*^{EDHJ1_JD~UB@5Rdrz>Q1}OYwrNnM%Bzy*CcUULzsAIo}#|>wYIx>49t*tsB z-#BPv*19B#5+UiBc1e2mAvDAO<6-|K-uH?8TMC!|w&%e>Y2tsiMseC7Bs!cQx!iC3 zw^~X2Um`sH3W)%kT%`uik|0IN2(nx|dh z6<8D~`52BFY{j*)Q|C$EE+9p~Vo3y^`BJD}ARDNm0Jf^tS(^D8jTiWQRBuR9#o6{B zg1>Q-Ag2b&jO1=O(fmgB2QL7gF)iv{p11Xlct*Mn+0nJQxQa<(--YMo))&OGKOl&# zp0+8x<}H7_<1PGCRBPE^Ylni2H}d74D*2Z2Crx!k87}R7@@|;_9&=o8miUL+q>b-S zEEQZk?Z&Jo=h-I{AgbA7nTY=IL5Cm644@ax}e}t4?jej9QZ5-lVR??Pp zZPm=Ah$up&!ya-4o)j}Wbh0edT$|w5gS<5oZMCa_$~G#KFyCQ1I)Nu$k9vxj`S+rR zWY#n%M!IwhCK1vd8SiYdG^G7-)_W&h;v1AT6RE@ne2J1=1*0Qk)pt0}tUi<#1}TzS z<9>OcU<66wF!60Yz&y4)fy5K)a~-yAtRqGZNRiO+Oz?+WPBoCs(=$sFSB?B0$@`KH z^U(T))h15ehQA4NJ^a~!N{Ih@9X7O{l$HriaLjmh!|fbj3g^J#DT3w9dnYId4BgEg zy<$IGoG`{9rNBSvu!NPjAf+TU4ir- zz?*e9;+9va9qK672YBP<(*snGfnFIuF+7RX>~Mv8`1^4bjDR$)v97NC%?*&nwp(&_ zDvz^+{D$x|V$H!hkenQc1yxzo_LV@U5iy&2OR_8;Ottf9KQ5=L@Zo&wuf8pFZV|!E z5+{BwCc)>aZi8=!UlZZb78t~OXf|H@;=!e<|k_L6!RHlpr5@U{e zSSjesKvHHu?`|$R4h(?H<8cY`EyxZR((1!WwYcbSvoPT3g&P8VoM*6k{AbMC*#ECp zM8zjmRfvSEqMDljsqvnR@K(-vxZekp;vqXJ1u;h-h4!Jp+33M58dAjc@^9azPjf>y z@2gw!JgaAi{&Lr$UW#F|#~*`nYBAp#kv4B{lJ~p5vlP?HDfM)jhS8A5XzAj=4D((kG!R1w%Y#^C5ghH4!CkppHhbxK=&l9U&`w%F}GE;6&2=WjuHer@?S z^p)tu?T(+H$W`F^64u*%Y$qIt(S}OOq0xL(gHiTe-cL}rD&LFLS)cng78l!rWiBSQH;3PA^yZapb`L1~2*T?>l;3-SLuN6-c zL%v1W#KpuG>!l$iqq&K;5!ebonw+J$+Gyvgt3CQNQ`9fMAkHiaFgDM$?hS4j>bW|Z zQk?X4#15=0+yQk&8GXhbxKT2WOB48OMax&Vnw4Na0?)@SWG>6A8Dmw@38SFHkFWp| zfT>;4iQ&#)%ozPn>O4KlKD_7W*VXdo4S8oO<(=!CvFY6suJo%No4<^m?=EJ8Z<^5p zT}KL3TO3&6gMZ;)nP|FE8I56e*C$({!@!{E?Fe@TQC7KC>9ZO`wT3`mJ=e2i*2YyU z$rtz>2@rFdC(DOqaX;7`M=(p#>mN6N zqBF1R6052x9-E~f;1DA~iiLA+O?F_4MCNRh6!l|$j|9cp?A6t;#m%V6H~)qdMK z&9r|1&v#v-=6%rCiFC-es~T+BFA9RW{hT6b9TUy{*d+Wpdfet&rGWnNWe)@b?VGog zJz4o~=%twCxCMgX5V!juA%HC7sb?`T4Yi!S-*ulgO$E$updhvSaw7SqnQIaBP5ey- z&-+7UIr`$OO|K2zl{2^mVPu9Hq1!v5w3;My8uhrmlEWs2sa{OCYElGk~uwW$`3z6Iv*Z3co0(P zalJmVY?A?AyXNKbyO90Qnj?OT(1;B5uYG(h1OP=b1=lllH0<8*cO1e2OD8+XW5 z?dZ2s&_^sWx)?{pJGv(vx1HB}{n2J`FPE~dm0pS-eG5XkhprBN%Em~jM6;2H1-Zykh&JZhP4Lwa^3nx- zjkD7FGqtM+@Tks+C_Zk7xFA>&ldoT!!&N%&=2e&;&|~O$NX9|0m&d?I~ z+H=I+k9uPbsxfo@%6sOXf1ZZC4o#5b=Knu5U3oZ^@B2+YCD{^@b&@T62njQZvXecD zQMN1*iprSDo~&83PWIhoPnNOE*4TH(zAs}RV`kpp^u4a%KmHxp^*rx$pL6bW--iQn zeqrRE6iIq^-u#QclB#bo>ZJwUlKgrLCTO)Gz zo^>2=sMJ7;lygil4S$Jm_J+TK4H-^4p@(G%yzbQWsH+i=Mf`MQT2FnXK0KXK^lDoE$O2?WUnza+ z09jH{?D}GVWCmzjv7Q`(2p6u*HWyKLR)vl{-ti&C_r$+kX95F8?JC3;bWF#$ubUp) zh3_xNdI*#Y7deO%vYCEq6?3tI`BL`A-6eg9Oj|zlH0?>PQY1&REI9o0L^NUj9F=WJ2eb0FU~izN=XV4LSTSFlk41)_oO zh_Bdx9fjX`8!Js#JTlBKxVHrvm-=0sGUQf_PGFlIzFKmdG47OHs$A*(@`S=39Rcx# z5U|;wf=sTXu3Lql7qBC@n2z$F$m1(N#?thL_Y7Lb|mH9IgVF~luA9R zp4uxreg5yw95$M&`VlR~C{O52fhe`!3U;Gl{J`U5wfp3zGu~S03;Y)&x8uL40Eelq zl{LtjNo2JC*FwWJYm6&-J5++cjN_5akVCHev&5)hnhrA(YQ)Pr)aNtO8SSj-XE!@Q zpDSK=o5piD*z&35A1(Ibf7`On4JlZ|3_Rbq))@2bLdwbS<+{4IIb*NEsD_WnwVQvw zxNOj;2D?CuDkv{Cr7XN+Iu;>5mAE;{NS3&Bmhv7AUd#TPdKMm&V7Fn*7O9l)rb|z} zjS*Y8qli=4AowQ20<*Y-qrHRObDpm@E~S18p*Gsk23-s#mrstOE|oS!jk>>T@UMAF zhgJ4W=ayTjIQmv(ZVB6|N2K)VocRXCakse}HY7xj^&=c$m$YbmAR>Qpm{wt>ie}vm zhnuLTK>Cq#`v!Xx&wLa@~~FBE4S-;x;8{(iyGBeUF21?bp=aYiLHv ze$wA3H2YsL1J&F820h>2ralM(O3n@?$i^h5M@2$}Ib{UDkZ}WjKt}HEAwbwE7PWsm zL1c5nX&pUyfgZtYx|=V~WdSJ=vH5T(l5eXY_bCb`S$DWco+QNy?;&4UALJ%JkDp>Y zJ+I9sdl{eik6*xR`?^%hzl~aOB)g@PoLu@gq~A2pWd7i3v6f>TMb_AG!KqhHv;IF@0@>i6pV7O-xp=Q=!B_>;96h zo~Gbrgp>~mVenCz<~|I)IOw8}g1M?tz4^s~$GDOEe5@ssl$yA$j4vyrb>`n(hMj*# zR{wx1WXL?bo$F3{UO~Lv^Y5ff9s=E4I$oKi|e#k(yauRaHIC|-#E@efD8U*hCkNW46OW%hvX(FSCx-fvS#Gp02M zxA5-$d&MIVKFC%(wg{U|_>pz&9Y~(fezkcirxQ6PMk;Qij|N!DPmo1WMJ^9uvxs+K z*qH|QC>&k%xOR~#mjn6Y3nhfYa}EXN@6T*%7$)Z9>VAa-No0pA6uM4-$Hf@EY=Nib zBO>-|Ux*3;>T=}J<8T#+2@$EiVp7>j{gk^qG?dgvaWaHdPPy-e7jtneCAAK`EZtWi za33SDwCl`LQwYbKpV=D!gNja1<(gz6lX#2+ykZqo#cnx#5AYGH#sYmiVwK1--)vcp zes|ha_L~G^<&k4b)gXG+ebups@>^$z&jI-BG4!e5*Mk1jZQ2_4U8J?AaTkdtu|R%t z3<2Ic=@kmJm3~-u!KzNCaewIi32ak&Z9}W#myh0K&G!=PIi*{vVmOsmZX@ zEAaHW0Fbu5AY9PRLidWo0z1O|WOwvJj3uuSnHt(lvkxL@z8+d*BSwN7aUz4#WXVrn zGlrD?fr$;Wy87`Y8%#~og;4zvu*O!>ONVv-zkx@z$=2;!pz3IHkx-Pp&pyRT{NWY? z2U2~SA8#|T2L9_2)e zbBuw84WJizk1m+iUyR8G9AF>5^a&vEiMU11gNLKL72RG#S$jP7%8N<2 z|GYeZ)M^2wVda|we|BQ;XFQ*?Cue&^WdeDArrQ!^#i^u3(=eyfI%SzASKJzbp*aPz z*UA1L2Vv%3lj{-U2LKvH5=zO#;9lon#lS@L%1yFtbaXpCc^-A5)XDx{2Y?c0hMI~V zX)n`6L8WVUm3QTvT#5PDYn{`t{rmrsmSg$U*jTXZl9bEBVP#x+@m3dca`JavKx22D z*Yx}#nZ++Az&86XpSD6j(^h%9^tiY0Et-Q*TxtSqu~usic0#jou;T~Y7T3}dYb{q! zc1&+_PAs)r&C3>4jLb+#jB>TOllRAJ6CU*T&eKnPx~X3J5$}ij%j%KUg4YEYnhxDx zr=u+>@mY_`VzfW7!*-Ow`ONtTPKTZhqRo|!ZMW4Croa8sv?k%o-Wo;L#EbHh74&qi zVJkNETmFPUeE4Abad)~fGId&9;$A!2Ug0cSZul7V-pV1AEQ@FNTnXNPg8YVpg()=) zQ}VVJaAVa)G5*H*+_K4)0ihHo;)ns}@1kw_Fq%00V=<0Jnh~GGfa*ZIc77COV)zkN zD>Zy#JI_QeCMC+}3ZTZ8N=bny#iY(;kNio5**VWnxb-824K>WR>}7AjVu#&cJ7~xb zjJjj30hB~2YNKOoABpl&dNQ~F+=Oa#Tbnu{V5%xnqj2Vh0(ZL0nC$l^>5fMKn#d7ei15pE zr~&bp*i>UUJvKHQBEx!#3mF~cY7e{Jf+zo@@(veZs9Pvd9+sz~s|)4}M`#{4gb zLr;Hjuc1geknh3n_#gutzSHqMM16d(2T^@{vo?8F6(LYTKd?^RTZ9x_UnJcGIgn41F^UcPAz^<-!4*%##Xy@)0^v{1~`=^p53TWX1 zL=c=cUOd)tbty_t@P(>5dhHmh{uc>6sK z;ht=b<;H>Q8NVEc$2^}4YrNF(zI4s8Ac%{2@+STpCcfsas43N4fV;>8s`1ZnPFvoE zt?yZ}P}@R<8={8Z;1zcLMYzJpd!kzBnA<91VB*i|u$dh-v_?d+9DDQioWHY@rJrf= z*}U?vUyKcYnbiX_nc5WyKFV0fniuk|v{)-|vY=OR61W_6{CgOb>QDn^zwy__>HZOO z-KeV0eSBBFdkNOD1uesacSs8`tj+!`+xS3gRSju)&EV>f-lvALn|!t;Yd@U&~kOa*Z2Iv2wKdklQuPx4Kr-xP{RU!}>)%GDrS$KkKV-)#~0j~R1HP$Fkz=`o=F(Og#%+aXy%*&-E zlms}VKWSFajKZPwhieKdC(n4tk;1QED1Y)h7GM%v36~f*7e)fhEF%Xr733YC)@f9H zFWx~41ng(&JJZHrvZd==Q*2y&%@WCkKnw+|P7mkf0w*W@zc-sj_#4bu&Kq36{+GBHAmv}bF9pLV zT^Dj~M^Yh%Vkj{+FFR?-P~xSv`_0&3(`A1O%vsdE)Q}GFk*P_Jwr^R$n~?;+A3(h7 zhAcjspi+cb=$aVvw3z#;1t!W`Y@a#iep%H!RU>h22cad%&S=r5VMMmU2EvyVHBwBH zXL_-t0A@ye!k|kiBb<4ZbY0gv_RGu>lD5#Ul&fCn{P|9;fSO9Dl=b<<+ZTw%UP$)I z9A4%K@&G;?1Uw`uTpoDv)gS@2&;HFl1)2c9X+o-D3pged8>W){c{MhDG7xNIDG+mo z;2dY~K*V)fqH%ErcmVk6_eOL*k03!@#)OBuev%XVVQqG98lQ>IQqL`lIRHZ4iO1#+ zXSRy(cZ{_@cD9D0ReRgCA-EGuL8-kvPK(O7`01@p^mFX$jy5>eJ@$9^i2CC`bwVj? zZn(vhw(jmw^r~E4!C$`Hd-`E5IW}BPpLc(-n0`2PWE7IBKbhKcb`5%6?(_QB0`h}w zc~p0y%vMR%;$g8&MPbP7{^?6GtLUXB25mOVQ^cQoajzuqeAn|r6^pHCc2C;j{I#ZG zXn8BLAVB7~cp#MfYK6Ej`0>}`8tCq%KbbWo@o63xOtN2vVDj9$i`mHccsjc;=g8?w z>5FMX^-)ft(NX)b?;zig^mwo|c62WU-SfRQo6jI?q|lI;I`L3c^HI9(>(}1TKdQ$5 zuCg)h%Cp6ZCK5K1MCB7dyL&Pl#6vm|y+jKcYcI1%6YsP!2hq)J{D z_I&zN>8s8$+-VC^x@I7iUp8&qToczCcE*VqRJXYQsotb%6y6(lV$VT|B6cB{FcyI@#AP+l^tjIw;d_7M#g)@|Y!~OT>5$D9Zv?t>!jYU05X+p)yh_XU~ z;n7_v`m8)WE}ZTO#@J0-5d!f$SSBnImk+cqg7>=fb=nu+C@d&Q^-W)fK+o?uUJ_SA zMa?G`uX8+TnAG^nX@%mi&wGSpZas4I#s}6nvzp|(P?KkIErmlQrzOqb)!emiHlGQ87;^<84ipk7fue}FI$|$t2Zb~St|H(G-tLSs zocJC-^RTS%`mE#)mJ_1!I%9=uS;5z1e!tp%5MbDZk%2Mu7fl=-rpcwIN#|Lkq zNKsa@5%q;YoenVed-a`ZDT?dSUi3xdpQU5HqwxI1dKU^|=rj*OFxvM7o1H%n9m=sb zKTlr^l{;ZDFGo^Zqbb_n$=YdrBYHcxAfK$ zuxS4=wX6E2HrNj|Cz_pTp4H2LdTI14?xAxp=P zV?*P)9$J5#>1;+7F*rQi-WtAJ+%A`9?!be_3$7%hqsJr2>2KRmI;9%c-FXKA-?+KJD$WvyFU> zzZPZAcdHNSBCZ3(Kci@KeqQ06X?XT)9FYypQe=luvSgfrE?#)D3}^;@e<(<^UDZg7 zw&Nn(#!}o%ENpUp6~!@AU*B=TVzUkG%_W}_mYF5Btha^9>1#Fc$C7%+=*QW(uq!Md zZ&2RyP(p;mw-jR?^0l9Y|H*76WR0X$BBC04^lYa#MEyMF7Y)hs7_t%LLxRlm(~xCX znX(c3j-v#F>6Z{rhX-b@VrLOh(xEeYxVf?u+L+%^(5S!U8yQWJbF-XZq?Mc~t%kOF zt+K?6hElSakcNDJ)s`N(LQ~JqQXmTDpS+Z&0B#<%GpS;0Vmj7A>3#rktp>Yy77dSV zAP~fE-)*e9hujZQz)dOVcM^`&OO?KwCZznyUQ&V_4cWik*n~7`O*b_S=CdD(1-p9^I z6C~0O1n}~I4`w!)*l}SO54UqjPd(&O-o!}3rrmc>L3^}(qmjp!D5TA70hpH*I-Uue<=rS^za%gL?f`lx2_I z-MxuaUKX|$HBxn9uqE5*j6o}`QXRZUN$Jy&et0c9>m8pV$9rkz@Z>df+qvLLx?EBXC9x!avej^GTT%wItD>otNi9>u z_-1xvzA2_yOi2XJdKxQWp6|#GV=HA|-S{kOxNxZQr4>~Y;x9~tS5e3v5kc)?#7496 zfs|)Vxa@adBctT6qFx?2Mq0l1z=ubM($EV-Xjg_JKg4ZE+2V2e-v`8rGHx7tcn7oh zxgE0e5S-ZA0Oam8VaF8|A(grd=|Id!vxw|%L2%)nl?B*L49^*M79mdnb<}pU3_e*T zet7yw6#oSPZ6etHGJ*kj;^FIaPU2n}1^)Y#U?TzUQ3M_+!15l@6KRGMH>uY_Z(B#< zhVvHd=qdLwd7`6i6E+L8j@aD=vVqAXclqz#0Hx1h2NGVd?8M5E9bR|$G8|$?;i>8h z>uMTZ_q@kT#_*PNBWu74B=sy>$QalTZXBevA3hxTbe%2Cj!U&cy24`jP-q7d-`k`! z44ogbRAW4Tm07)O#CEn2=e{=(#`32()!G{B|MBSPV3RO6+8wuc>i2rHd!wVf7;z%P zot)qcFzS>oE^)>@gP@eXldm(vC3qr7=boL!%^lC><30aVi1}IG`aWyIg?UUWD-A!v zEZ!fT9w1=P9?Ei}b>I<+HO(x#doePi?#6sirvu-ak@~E1r?JLWbiy~*bNq`y15cxo zlL)4K*3KC^q~o9Gcfa1&YbABqygWH$F(>Z%0?(H)T+qv+L|E9xjKw%=JiPvoUScQ6 zR_7-MX|U$tH((nG!D9%lfM#1X=N_J&+=ty$Y`z748xIMSczRT)g`n3)s=<2lA_j-0 z+Hf7oWlZd>92CiWFu5cJ+yU|qhK!?9tdJJ<3Wp$kC;>wds8<0v2=fp%#DF~E+Qr;# z0^K;tab#~E9qO()bm6G<7em+*B$Ohhwjh=uHqR_g+Wf&de_$IzjlH@-b~nxg8$rQO zb81nRn?5H7bc1JcOU~#v1mm%A4iQcCs>ZgR$Zg4Om@>{ImNTbZ--dYcB7E(8Y5QfDN_NNaD!+D?D znbe)2_Q%VHB78kRw(#H4|BGyDvf20gYtWGf-7gp#9?nnpV*9L^>~hm}-*^AkXHs(# zjn0e2la5W%m9#yzJm*~&G~`)hZZ<6#?`MwU6w!;d zt|xe8T4-bML2LBwdGr1oh(#mbfK$9$KrG|G(?%~rHuzl%b>1wZdd3h}Sb<1zD3K17 zb0Dx?%&G>svL}GKi2TP&xI)%V_Lmd-f8eUcP`oxKK~ZDhyy*?;`Sm!Lk01;9s&_`rYuHz+y8eFlcoP=Ds5?;UHIIfP6poBFsG=0vhA@C5F zAY-xiU;8t_jed3YW7FgF2-Or4g)AOm3gJ|Ff7%dHl)Dm5>46kgPoesKtDz6>5yYZL z;XAi{fx83+-@_v*IrA)_mVfEllKa;9^q*&E9*&_}TM#qs%qy?Q6l!{PSi{+Q($Y?5!zSqnL_-6L4 ziqriE$#pi}#l@a!Z|BDEoNIFwlx)+Nwy$K%f+24gq$WQJSTyrUs15dA(brGb`P2Je z(7#X9VAtyKQsP#z4)1h}dj)Q|auWZO$0ly{w^N7A9f0L=3+%oJ6@VZ^eYT6k40b};)%q)`Y|tCYB3|Pjh592OtR*s zZ;afnACUW6ydUrF5iNo*b288-B4ieQ)kQQ*p9sV53aWY7F_Uu*o+$4>p4nep3!(vQ z(I5)$70lCa##e# zsk$GFnUdmevy%0QVV4lq*iDM+YxS=%Zc^U;`cAAgymUXwe<%|ciSD~211UT+5Q0+t z?5m-25EQxch*B735FkPVDpA%!XbaHm2nq8)#oi?3k;liRd9g&?(@`JxgKZ6@yiqY!JEX5*1Hop_Ri5es6^PhF;U_b zE-vP7f1h{fTCwco4>)rWlRz>~aOR}^`gD*RadxKlH>HoZd*DsZjV6lK&ohzYOxN7w z38_l1Ufo=4g}EwZV%ACxyhCAwff#5xK!e8#tGSbW1k4t)6Q0jMLU zy9dzYdnHW4S@1J4vPj-~DDhGXQPzYDKAgWqH*`#q^CK56x=6xCsFPx^4X=G*5|Ro~ zz%u1Q=_jm1ib-ZrF2A9b-AE~0;?yy6nCM^T9Eo-zrk^8q>(U@4`?VCJyLA2?^!G3h zjGjt`%*a5x)xZ9ArMwzwMSSF0LO$iHs81eYT*yRKOJE4M6m}By$FhfoJ@&EH7=+Xc08!sN^uC*ZZmY$(QaW82%z9tLs zUfv%eCi#;ude^G{=rdz;x?$#G<|`k+y!Ly_6kub>An-8uIpd%j>=h^oQw#0WWm|)( zE*D931^#h7pXS|j_q-I?n(4S+Es!JX7nam^!K2MUf597*SHjd+8XZ<5x;~VYNBpSLmc$dS_|4k>1 zYkd08>q*|nVPYJ|-xSC+SN)We-!2`Dyj>L2(qaEuf%^V6`^b6C`0Q7cg6mXGUD_P) zqRPn~Dpx$ux|he@f~$@cp>oDiXZ% zdhEdBAriUoB2J{2NWGxkpvLynxGp3@Y>@%Xv2x*lWn7LnF!Aib>|^I6;hzPbx{PjD zM--i{YaL&G&-$?1PA@9AL#MnV$-DR&`SW>!M+=-tjVmn^c$F~e9a!|MXW2n#VNH@A zM;b}>wCDTP5*ng(DKYm@3Qj{*a}IZ(q(j0aa$_yc9-SWhvk`tmzRedzy$RY+h?l{( z(Wo~6MFe6zM`qjxL05C5y`^gqhq^C-b9e!U7s1{r(BgMvUwLDRc@wl)_BZBdU;;nmFE*5(;q=QH1SH1;d~- z_-BJ3z;AHYI~Y;6=N^d;IxFYCrvwDOWfpC3a3fVd^M<*@3!QHg;))K}{LIn&6%61) z^K(-mn;bS`1>&OgimrI}0mpXa|5b}kq&p{cb)N?{dE$N)F4r_o82ZM0g7Jf4k1_xB zR0x5uq;%*cmEyTiD=LCEAr6!&eWyb}yX4=(9gSl$NH&mSMXr zux;d(#s*~-M0}`Ev(a1~RZdTwx#F_z8^^thVl@L@nxtqBdNG65oGa_oY!4NV`bame zf3cRg*0l)OO>!?&-Ce%!yC-;0R{l-{CUA$%>*D3Whc^d3nKjLFmXXO5JG%?i?T$<2 zZ!;I|iUX(`H~7I?xfUHtN>XO69`9iKblAH(3AF;VI8;e1=bw37y3?pV8O`~b;g&b9 zbxJ|oS0pF_rB|XEx*4pLZA{4t9t)w|F)^ZKctw=zXUohY78|K8u(S7u0wIR?VegCF z;KJvG_GfcJl9N+A%yutJA2ZPR4Lmp!H^Cu2!?u?>g!ei@vbFe6e1TG?cTH$C7!Z<1 zpO*;V88oJ%1(&+3ha$lTlf~HGcS1k^a~N$XEBD^j{rI9;o-`RI{zQqC3aDFp4s^$N zCJD*q=MIPYysdip$G@r&+ATcuYJ1CeW1-NFKGi`6->T*5$B*Ah>)SzNbl=$AKr&&Nfk}q!Tz1F694_(XzA3%DegAF66Ycd=|q8-A;ZjcDDUUg+s4P zABDD{4aLNgRi|(9-QqnDBp3XW$3)+mX_ei7fv{IG@)vTPFFH0%6bNCzEPSj>ujU+5 zq7%MT9*T5{IahW4e9wO0h>n-vs()GLvW+0hxT$*g`Io<);J72EoH z>~)wCF~=PL^H3b0UVddlwX4Id=?RtM-xnp!X=>ubnD840+*P06mTH2AxSyaZdnFzR2ay?P{R3n zgQ;RV1fLDR!I|@52K;mJ4BAUB!nR;%K9P!|L%b=Cz^K7pf@=uEA0~`~+UTK(AdFXEOj@Su#s<7oA;!6OnNojw*(pU;+vp>PKAl8kKbw1f~ z<5z&H+P`Sr&90T#6kz>9y_auEH7fY~f^ z^m_t8I!`I~RSYLWR&xk7f^X`CcHt~n-TxwFnoiFQ^aiJhEIbh@CHp|1cBeQj5 zmRuFTqt<^lV=ZZbXc3LNq&A)ZgUMt62g}HVfC=vJ>4a{#L@NvETWh8k8~HyAv)A9R zei<86u5Oz*#aZp>;j`Yu_$mZjPG6+Cd5sR+g|~dr3`)fgsFXLq4RIyPT^6LuKRIT(u#h=(Txf&PybZuF&bTK zt#z14pHIuZNYc@Ic1gVAN%un-!z%&Mo%(BIqV=lc`}hm(o)Zc64?3Ig9`jH4G`+iR zy7Dq=JGH&p*ck6_#}4znRrl%7T(m`O^v8Z8Ka~(a(+ZVS7icIQxbu~$XE(PxUySD> z-_~NXQ)NX|BOjoQ`Peoi>Vs8wy^5$O4k?A8F&3NCZ1?x`b(8tk{YZ{&$e0O@ zzjkaswVOxkli%UlXvxQ~kILB!Bx|G=k<`fcZ|ymCv_Y&^*#vQ6~F zBqPGywE2cHft{QPP{Duo=7?r(#2I~l(sCu41aaU3MqJl`+D%)_gxVb&3emtC8ROeV zn%oA^o957Bv`Gx3B7vq+75i*Q;m!SiNtF(Yi*HDdoQJIno-isptTaeDcl@L#`NlSM zXX_b&o^TP7p#2X-}|&fa%#4FTyyM!0n{gj^U3H24r+6dF82zR($RS)-GPfj-3V z^UrDRvcD|bLsm*2{PJ1E4E(MbPq}%A5vE`9^sd2_qjLU&NCyF8t|>z*?Z$!#sIb6)`B?g%Vok4|*nTwf_kvUM+_o z^@K&OaNkefq_tU28JWf&=acjm$Vhu1D!oq7l1&BxDct|Qk@uj*7Pfvsp1=*Ar~z9 z^`#HExCd0Tkmv;r(S0Al#q^XIXxC3kRZJrOl97aplea;e6kVQ&Vxrx1MvuDZv8d6X zEBtk*Ow%I3V`$3kzb2rDHqNF&xe#afko#3~ycHJ${Q z6n%Y`Y`#ONeO}qW#lBb)B!`&m#~9{(`jD@uuo>tqm3y+4V`-W;c5ZEcJeM>(30klJ zZ?2!Y{`n=79_`{TMae4uS6C_tIimbTk}9Y5B(PXg#R}{FXdw8_`t%R&(v`|QYQ9h4!Zg8?Bj>=4zWFC4H!!u77d z8axsqtAV^25t@OmQW5lijQhv@5nmyWs(QZ}5IVX7G)76}NCX>3SMRS5k^v!&8tCka zFQ|T*7o-7{MeCbnhUCDxnwSJz_)9Mq%ENAy5rQX~$s%D7M=Xy}MHjkVpO8Ma1*I~s zK&TPsXixdQ`?dlw7Uw+Az_s{Him8Ph`^H;KEw)9;bkdi14%`li-Hn#rmg0@&BnzV5DL4 zT^)^ay@zph8Kh=Eo>PM}qwrK7czJe|=t2avz`}k9$8zT+0_ke6d%stvho-EZWtCGp zI&J=|;spokpc}bCl^|EH9**u=;vdhO;zQ6hmK)+<_n!a6m0zLCybS-N0JCMM7+CzM+`1^jO3yI?LEqj)c zeT)(d@Kl0Y@j5UV?%mU7fA|6yV4*M)Tb=skzvE-~$1Xf@i|z|=xuNUaTtvG07i_U* zPnEUc@F8x|u=(7f@f*zfYwW^Ft_+6}MhCDInnc2|q~m`CI@T5zk4~u|oV=QWDAU70 zqz}tC(J-+ec}?V$D~@%eUq`K{giY!q12Q5WzdX3jhrSj^VsL-F4_QLE$o`iyxSI>DShaurrk6j zHKcvMRd~8qyi9vLv5Y$>bh?!fbS@lcCjjTLNe=z; zYw24msLznxcZ+uL_kMl`?E<9SS^kh6x*ua<8V-T+HLtQ$)iDV{CvJhRhKPDr&SZ3K z_z(*C@XL>c)DYnyzXP$;baZkN(~ovvoAKj8mH_DI^)_C6dtqQb?!hMUv>lN zrwv66gK8@>v_YGKV0*>8R~4^#1155JX|DwUEmGt-;1=|QJRnZ4L9MeutFKsryhenQ zTj3{w6~%MBwOaMQV#>XNCe(>E{qK@Qk&#jD)~=k`ar_p3QFFM9(z})Ey9=>ca@P|Y z!|QA{^N$fHIp$IzwY0veSl*K8G$Mr{#Qj`*P1<_Znrl}oN=Sj={b*b1h0O@Um-~qC zOp<1sW&qpNlD}`h`S$?xu3|=NF z4;1P)6w5wXha&G*{-^`jV*++Izan?-7M%8D9bbo3`3qe&#|FkE_2i|0E#~unX!mI; zIM?VRK)jor_3Oa1ApgJi8@Uy+JUW`yEM&Fdz?IE@cMDT%K z52p={g0}9CQVM(}m|54)tr%OBxe0pbBGxThwfTI;8C?pDoSUEPQCEpI>kNq&BP@3m zNL&*6Rg;@)F88vCp-(#BA15B-E5^~3zHp<1AN{~i%3A5ZIs4|nhoa`B`X2~T^Uaw1 zH(ZkmrihL>6k_AJIC!fsg33>GcALq>HDn498KE9XYo}rBNya#;){x5m?yXU`gZ`8*iBjmV|{Bl$AJ3uD_}BUuP|7WeqEv96ZgCy&1VsEFIT`Vx`&nz^ud5YZ z5Q6$h6a;TccQCo(p4%RQn~C~KNkRmT=3h7Z%>ze4vy2oS*hLk_+=CW(neUWOv-uTp zhOhs%9MLBw0`JIG4E)ts=ynEQnWLGtk#~qK5a`A7ud9UF4T|vt(PO31tZc0pIJ0r+ zdgM;n9(BL^b46-lM9Km~M)GF85YLVF*}*Fitw@Z1M^{HB>f%>~V6BQ#TVC8qR)w*> z5MUedJ*GgwrXq9}OtiF!uNv(FrEmkeq9?}`+FwiJf;UW|Br7_h(_@?nU#y#B3m^?H z+fRyml|d1j!A~9t%lR&u_Zvh-M>FPAX5yxuPuhvHphk+}Rgp6PhW+@jP9*a^C|t0! z22CbSH)Rwb@2w%WdzHX}mji<*R~p%9+WX+W~=Y4g$% z=48ZEf7m;jCtsOj-U*L0zo5GPW4M)qvp2-2 z#gv-_U)*Kl8{|)?LY|rkc4KszB?XQ_Iq*(f|H1anoO*FQ3Do~ zArlA=n#Mdkb|>`VZ+xIfnl!=%4TUxltjLGkZ?r{j5Z)Tj+{VvSINa~2 zDrT48x0O%rH6a$oK!mNu-+AyE&z2mA0mrOWOARPLQh_l=%d8B%qG$Cuk<6tWmV< zp%L)Fu{&pp9-9D<^kK0X9ev6BYj0vso#Hcj?59CCxKfh;_bH;-`zQSR4AO;zUHew_ zLels`Pjktc_o~0O#Z9;`-joQH!6JM((>_j$RGdc##Gs9ItcaNyFpx-Vw*DOdUnUbq#*kwuly*hP z_V3Cb50!Ow`-qfCVa`ZuvKi^q(609PBZX0lagS4w@>hyBzAlAbaNdA(F?A`(0Jgst z#vKefH`q`s%0$1(=linv5noK%?^GWXnDXEqEYk>`iz%V3SJ+Gn$_1Jg@|~~K5X*V+ zrs(fqKHGLd7QT_&ki0S`L7nKM$usnz$Sb@8DG)>1q?HRD4kS>_o7LFDe;&?Vh)1Lx zdFYbTtAP!D(%KB=j*J_=i$wb_)$_K^8!d&~$^b&Zeox))I4%jre|}AGB_ZLM5B}dC|Vk^qA~Rw@RJirF5NB z&?}qo(P|~|A}YDJRL-!H_wT19X)7+$YClkT^k?v?4@b}im13o$@ak}j5$%NO2W1$^ zv%BK;^hx5rW%^R6U8q%>vXhJIuGpn^IPWPWEPOSxSerLQ9hP6bg;I*%)=}@ z9wO`6HPCGq_;1L3(A_-Tqyc@A*RNLdd+u?`XT5 zASm7FI)wF-6whS|TUkma@KFF1^C-%p4pJKZ{BVt;|D1UdhSvN@N@IJ;Ta*7;xZZ7% zJ}rSN_ddolGj|g>2AI9csn&N7IZ^qxRg^{ z(`u47#-z1zf39)3w|9#U{vna+dD%yYj&3b*c4GsPm|ylnJodr%-MM+$Xg(o&nL0X% zq$BL`%9T`cck;Nwgv@U#d@vdzDeoHq#bedNs#R&~Zam_eTDgiZtxL z)JO5#gkOHwtwmY`qo`atp-& z96yN0souH4g{UqXa}gpK%-lQuQ|I_CtGgtA-}I4nO5Gz+AvACa;37E!@%KZA66(y) zyCU%wU&sOsT;IhOgWOdD^T0Q606l}v{KR^&(M@9R>ibiJa?!?1dxrMArXwX0VJfX4 z>LH#E8jnRNM=!zvSryLe+6p;BW+JMyx8g?OYbTFKOHO=8Aiepa2+9~@zU~r*FP}t!~>+CZor-sdTTj@^~#)zDXYalJ^Sp+@V#7Nqb>s4e|E& zTL=u$KdYP6IeqY}u1(S@(jI@g{=QNz=Qb7SE&s*x zWzSEWOd*~fH3*5MsduX6Fn_h7*SCai&SEc%lzFNJ;n$fYt{dH;!Qb%yKbpS7pUU_D z|7|6eky%+OGa;MAA$zY7;z;&LLe?2k*`pAdCwpXtY>qv%l08oLc5G+g-}CuB9>0I! zoO7T1dR^D`daf5WD7ItZS0p2r=_U%%Km%Xg@2L9805N+<`pCVywS1#wSUgtt4JKNz z&FSB;&Cp3=N*HQ8nNo(J<8kfn;!bTOXzvyA5R=*Qs!Qmfo_#hFGsmq%Op+l4PjI-B zvO)tpb5%-1_Bg`72Lda9aoI<{QtT_M=2e|lxbrJQ@NBmq@F|H~tnQdZLU$5Dj`V@l z!=J=~#cDltz1l)}1o?Jus?s>A~w zK7>pa3#xO?uXM<%Dk%MDg0ZqdB0U`cUHN}S?Sah174AmDSCy=O5R3jFX#ft%jnna$ zI}2(98kEkfa9hFTJR={Rt35d)?}{hk*M?VLnq2SPD4b!7ajJb2yYfF{;Z_D}6i03b zGC)X~tubqEh3C>nllrp<7+)W;GOpI>m4CMU20ng6G~F#P+2qlT?~}XT=F=j7Fdm=! z26xM6oM#hLTnY3L-bo!f$lu?_GjDnR^Bwp*lBDNR;yU<2J3uFmf5vzT1WgjyAtr}% zT=FVzo$T)5FC|`is92vFVZQ!SHP91gXJO-!dlNvn1Zk;R(eJMM9pq%EH)uF?`mPF- zXAq-(Q(fX%Htv-_LR-nfhHh#$&vzttSVTVUeNP-E_zvtXPkLK~*^N^go?qgS>o4`F zSN9}Vtt#Fugt^}Ox#RcVhlVLMWJub8FoQLJ$PO?3FIw};`2Gj9IQy^lFK6s1#`at3 zkfaY?9&j41?n~I6&Z#)pk8fNyXw@K=S38Anv#U)BHHO2l%7g^h{|LYPp^^joo)V#I zuh8mKd}Paz4SLsibnS;SL^g?j@%M|8Xi(PJi$3r3==3;UD}pQZ<8@$t{%q{sQ(*;r zQM$2Yib$LoQlHvl% z3+S|lZ$=8-nV_FjpchvTpRojg?FeZ`#D1z7>*SA zzI61Z|NZ_Ft+WRFIDKZbrIhp88Za=YCMAXrb|RY4aDHKZJ^oqH%5QA)DRm{+f`q zzEi;d8Jb`h-c1g-`+jibSZ%#YJf5N1B@R2C{osxMO}y{w;% zteYbRF||wSS-&-tRL?y%%Z!*a?^G4I1EgA?A>)zs5&}YeLfr5@P%?0d6Z@&-z~iE_ zbARd;jst#r11{hG8>uJcOqSA`66Fu}s>!=Y;~!UYT;aRj27b_A*FOjy59eQ~P0$Y_ zPdyn?Jcyq(A`^<2WA3W&+!jca3{~eo{rw!x-amCs@lQf3)x?zy@4}B%()jO8vwjZ)8pCYPBCxTeXdM%qz2nD^czjURlTP)#oaKH={YhyiXI48? zH$BBS1>zSr_&@XOSgss8`qzUFhXsOj*Oz~{A~WL6gp?yP)NANoKQMs2Bon+xamw;j zCPCtje`WeRtoY<2%2Fv`-OJ-%)M?o8@fa8Wavd(SnhirS{rNwUz! zB-52h(2P2Kx1l7ddUwXFw6lQjrKzC`gWqSHDG68&nWGwW{+V@bJEt~(B$Nh)=&Qr1 zPqr-iIm1Iq<~Y2~IVau}$?tIam-4eYcmI)?!_kyWp9Hk4W}f4XJl3%YpdqUo>j-f}0ibUOm(fBfppqBY zDU=h0j#MW3!3QPqb&FZ9R_;pw0)C-5`y^T$akR~30`ib7Zu)r4`*@rzgMa3>wzds` zAFrVUA;T!#8vIC<+fce1Hi8Q7E{06IqGVK<$y|2d2M-ma``ga1x8YoJS=<#^X>s#A z&RqkuWZz#i!&9f6$Jj&)jm1ZB*vtY7Ck?`~xF$Y1i6txkz{fF+W3BuW?rq*MTXPQk zY+lrRW1i+B30Y7-!>iWjN4#ZZf)lzw>~Z^60&}^(YpL7D3`5P+o(f^}Q8>=f=ggPQ zqVMCW*NxYMcaOG6kyV}Tsy#J}!SWSbuLjLAL(HM^l-3O^wuUb}Sz}+sWNv7#=-p+c zd?_<=h+ExHQD%6Z(A@ZMfs%CX25Kt^dz%sc$cm0N{JcIAB_2;7{cx=m*T zS?QA^Lu$Ieb%nHp5H_3*@6VRqdcyydp7HGGw_Xq?g_@J_oji6@VtMg9>C2_AoU*9q|mtJaqN8>E#f@K#&muzs5DarrzzZ9v~t}ZQvu6Q4D2HA*<2vD_)WFsQZxAk_Va0?OJHdN8h&ZOuJymUj}21k;D1O5WX^qC#8wM5dFDhxdeNQIXHh!;REh*S()L*t-~-uMxW zTWq57W0*J5PoAs;o}mc+F?PM0cz6yXr*Ho+4g$)vd3?D*+5~s+WYzd4Ud#<@!}t3Y z8(kDgD-AHz9vFWP!q=jk6qK-hk*#YEaIv$=`RZN6^VSThYvV|UELg8z)pD}tskrKA6&Ko$DF`X8qi-~u$P=j zz*Z4(XdFqKCz!Qg1lW=0pDvcDKTg|Jn_2l`7lpr5nbi2W2IL_86NUF||GYu#9V155 zF7g&SDA=%ZJAw$-%sQ3LTD$sW)!tx~gY+mC#Z8 z7W(s7_n6;V+wG~<5~rg3j17)zQl2Df&ST#zd_#&Sy$eV1v=0&cN4IGguN`lJP6{;o z8}s;RBn`XD3V2tgH_YXctm<{fEbG}RzwFqS3bv!c$pf3L>7h-XjH=UrK2ULo^uoynNO5C+l>ekKGDh z?`Gm^RI+fAHaA^^`G`I*jlPY2j}V(|UUp}JCqCF!2#yYbpK}7M2F1-ETe; z6AosDyIUJ_ZTk>Up5KsvQODk_!TJz``;^P&a?8MAw)hsNFA$m`6bRWrH^e5l?-_|8q*&R`XO}^^S2|=$)YaR5xo;}Ae;}5HFui)zo_V}4s zI@1=DLa5^g@Z|{n+!MLNOGJIsCIiV8sV5&eLhw)^6m@C=-%o@sc&2;zZ_N%wc|!Ud zV3YE|*D4|oAcnR`wEHih8KGk!puK@GIk0BPRYhn_T-)mA>ALU?R9&)DT>poVRVe}= z7hzl_v`qCtkT@GZY6u7UGu?vHr1aSR875pq)#<=*q zkPzZr6m)st9$Pj61C~*ky()Yx*iKzO;8GKLJ`5(+^plaNxp-t1d}*fb!V}rt{B*vj z3NMBvE=MZo1m}pYz5E8Mx@fTmKY2!OUfXxqxO8_|E34oqLNEd^DSHM=wr+m-W zav-sMEKj8!lKT#!F~a;KviyoB%iFBM%wqDm;&xC#*ZXdx{y~9ZQ7c4Rnm(o;d6(T=yWZ0bFp zc1KXIb!kXXQ`Cv^j@kTz6K98cflmlBm_S%c*`TPbu-k-1os8{o{Z~_k6mOdZkv^ob zb}hbrdYE8#tGxpC2YFBOGmrxh=U@JCBv}o$HoNK%h>+S~A6j~8@w+D{jIbS! z)>Ci5ct|JL={^TGhtQKqh!sKvtxojwgsRn{c%fTLjosTPr9vwE$5xQtDoj?|)W=P= zgN7GXyA7cO>j<|9)^;*YL&@_D$sa1bfSycC1Lo3Tol46%Dhu+!8(Gvm+L*B2k^@_jLMmznYuaIMh zfc-EITPT)4_Gr!g#L)kmWSLO8wzs_job69Nb^+j|O|tIzUQkHpcFr+z#T{9N`)Ewq zK<=I*@h!_Q+SWXQE-QGa59z=+?ktU*uAe60BPUYAWoy8h%=u59%vFA3~|KGZra z?GRSF!2uZ>_3=DRTF5mWvj*d5#~u9X%NrCvM-f+57jPvHO{Hez!{*o7>fURJ;eET3lmgk#fYGd#- zMh`u{_3R4%K2MI1SXt`|kBK2VH4CnGokBTC6m8he@$?X<-F#%>AA`Fy^`Sj%m~N8sI+ zr>z8hE0ea_nxUgXOn+oET|QJ1;UsgLklH51yQjx~Dkfb&+iRgnBQl)c=Nab1hcuQk8R_-LH}Z6r!7c*tb7V?k$)^OAojdMZ$QGhg zDk7_#wt`v9qy0@0+>BMZoFJ{roCaU(x4wj1xD%|yY@Oe>krK#@RSC6gq^&fdX~6oZ zLHkfy3(MIq;$?f(FSxO_Pfl?LE?i-_z&4!pcb`w$S*@G;?fXdVE;6fTaT<0GBn_XW zkgPS+s-q%Zcr5e{;d%&CWMDvtKm^>eOlty3*y8g-0!Kc`LvA0unzG_~7K@uc7lp*3 zazqTPtbj4nz&9uVBGv&XqEU|{Goo^ z{=vZvZiyF8zU-f#7~AJ3(qjv!VLm&ENR^?}Tu&s56c6_KVJNWFayh$2#7TIc{-^_M zxeX{T>m-db7rmIzTLG97kcFnE3RQb{ zuQJw&ck{9d-CM_H_OAJkNt#=Ejv4!!0^j2_i^JXb{Mj!aJn9a==5d=1b3R?ZwpbCp zeaKQ+zA1RH{Shklkrf-{b>OnjSKID@0fwW4LPe-1`y@ zOjY5MQ{V6Bjk!WDw#Y)=O9X6P8wd|1{kw#1H9$`BD4>=jQF(WhmHk>mM%cnm`=rpV zZ+8brLHP-69mqNOCQjfE1oJU6*!L`?%@YsZ6?v^+%eAh<_T;?%vf)L*M{bqgTL)=2 z=75F`Qsli=Db;0k`Ebg=E5wn?6{N$w1k!{Q0456EPtnY8MO=u2wuWJ7UQ9_M?sa1a^MUg`j*hNH=7*^fEr(|!YJF-~DUc1Q~ zFfiqfxWIRVFhn09tU2?K2X;|{kd&rOx`vgiCdZbNo|JId%Y-!cqavG#S~^g^ka^~_ zxHyRl;CJ1Qx&pjDiL9iX=3unN6I&$+^t_~-yyQaMEcSpN$HR9#RZdZG`mYz{>*@}P zn_@q8QL>sNPkpE}eJz&0)CiC5Weepd{T&p@q)T7C3H-$pLtC3bOsW$#R-bQPxpAj( zSkNoH%3FQZUB7qcdC*0~$T#kIX*m3&h5^U^Zl^bQdxLV+FvX3k==PGA`zgf%>jTo< ztxC^-te5?|o~aChm^pR|F$~xG9ZKAs;75L!w(5!fXWUiJ)l$WP;1 zX+k3il3xC&l)Jvd@UuDY@Ske35;L1D!Y|SQ6LN)GRHND$K%w!=z-}OX4vtT2U-3g8 z#hOo+UX}Xw3^E*T>+R6mZA|J#`mU~JxlvgyRh9P1%e3-py8lBwtRgp>q(e`DfFRCY zw&GV;SIx=)C0>FQ38E(zg1y6^>?@iNR3aw>hpLkH!|>6qTN7-7ko|-{g?t5una; zYtE|PX1Lpf&zs4GXiT2(s%O+w7DMB!dy|;|QP>cXU>yD@Qm`vn)YPF5!ltYzRS!SOYo^KoPc;TMCWwYzU$Na_!QLL9z=)d2H}G@)t4BbwF=6 zqk-WWDfJ?jatTm0<;DM8K@7c?e}>bhepyj}i12|CQP3seq&PTkaSq#C4Bf24r~_*V z$?jzkKj1nCvn8G^J{}UFA_9u!^A@KKTetIpbCM!yO}Y|+0x(Vpd1%Yi6qu;8Yp#O# zYsUnVgf>J{WN5s}_}>%S1Nr!P5(~XP0D;%czd1Dyhy{vy44Gt8czEjypEu1vzd8L2 z_gsF=6yFBHmHJ>KkdEv-|G}rKNawugLspUo%3cRn;l6&h0>u?@6W-9L&T7mS_fpfy zRaK;QfI8B@PP4IvBtEduqJ`!x9A9ERk?u4L#~h^k__Hx!8x7w>VNT(jP=7rR4&*+H zxDT8l0rL`Uk_YmbcReVmA2kD?lpp8uJtl13b_E}5IKD>Ch1RgGm*l+p1rDuoC$zo7 z|I&|6T2_c%-{8GSP)ry>fIgC=t#L@?ngVaV(eLqPa<*;a3&V@U_9HObRw)S|y{&0L$;ZJF~re&w}vDh<5N5~Cc znf+Fg+((255!7a_<^OI5M}i51(J}>|r#ihd=JsU#l_+yJNMU#~%@|lm;(lJXZ!KAV z`WRDf314Ld!jJ?U9Cr%Zv=AxvC8#YXT@lbEz~dk zkp08!pPX!(PYbD8Sy@x+uCka){+BFHAz=7aQ%!(BCF$^RR9!x@7KH{X(iQPss%R0B`>(YC^nH7Iedd?C;LmzAvr)(4_ zqmDS}N$ZL4Uc^1Y317drzx-yw4oNu6R+R%Jhgdz80#2yK@7}BL?d|F7EetqZ-_~{9 z{&??U;oIP!%wf=t$5&siroS*bIs%P>vV)==IVtURNWwsvY7NwU#YbU4WA!(J?rzvv zx6pUKjUJ_1GRHhe3ftcG_a9NnM zC=F%(`3YlA>JXd~J5v4jj=&5k21RUX$iHg(Z?RHu3HRIoyFyFpz9`DX>v-@Ylti9P zzI$}paxD5(Y8-pK(OBA4?h=~kf+g#Gk|_ZW3hx{!6%a@({uDKc?!kY+*gf)_KfWIs z_erMAtcpTrte-s0+_+ETRN^9r%QjJ`QE&b(=&CN?CuP!gaz-g-l?<(_gC8^BJ|6xL zQL=_ZbpXoxr4t}_B#%m;ZTbrXn%sQ!#Y3qVeCvQ0@M8db*;D1%x$gzy|IFD!jrpDN zej9M?VHZTjlCp!K^#kyxCIRS^k>#VPhcc)EI2p+QJ?6?6vJ@q z!QTF{t`iU|l6eBB9k)T+MI6A-IrR6^x2aO z#sTkuVJ!3MhsFQI;}6OWl*@^hg2!fwG^rjBiuT#)?toWPa%*!eSL7xC!kd51zm5vCr-@O+zX8)zE0la506_^;`*Xd-L{C&rSDby%Ks= z{~2M^WA);|oPKFGyZ&7+__nO!jSYKF^o~T91e5S1l%Jg}Rg<+IuhxBy zn0|)Sx1DHU=A8YJ5+`hyv^07!(IxeyFvTQYYb!}rO{lv-GH^ID8SM7reLMHZZ&&D= z8%0=qP$S$&`l}1v)H3zQR>BOp*F6Jj_pjg7kTw(6bkB(S#S5v}a*$T+^$0oHUSW&f zWQ3sO$uGxrT}BzU{RSz&&^7`8<~wCgyXL%}c5k<^hw0{D*=%Vn&9~~Q-_-KXJ*w%< z(WUNeF!Q_D7v_*Jb<*WnjM<&_e)AV*>~G)Eze{!!O30eTD|s&}CLgJJ58P$H^7kS4 zUoQ)ewg2?_!(*CX)qXm-n^3fH*X2b0BRzhnl}69zx@H`Wn3?w6%tUd5aE577i#(e?=JD1`x*i_=+{(VQ`p|48AMYL!F4P z%-P6W4+qMiyHxxE0kQc;URa^Wx+W`1CLVRIytm{h*W z{Y+!jFGd1Jh+nfBUH81EQm#bMe~F|X>AY8gPiP#O0evLg$*H#zZ#ZAASBrF@U5;qHrsIW zCseE>C13)`4O?ffCnv^dNVqa=CFMc;!`z`0_yVTQ0pR3DW^l1(z`~_KB&%~Iz;+ya zP=LHbYl;PYSdgd1#=5h${kNkfsn+^r?jVFkCXtO8i;$vBt?MZ!@QMTUB0$Ps;~4&s z^O!Y2;HjCi9L3vQ!p?4K!sB!}6u?Iwi>{F1m;Wm|vFCZuNR(*@%S~U$`JWe)&g`-` z#|XS=xVKc;bCY}b0Jp5%5u6*P2A$sXAhw5FNmU9~dg~C72-ldddilItwQKNiUi59Y z_@{05Ljw==mNp$_W7Geb7$bE*xWrNL{}%`&`e$YtO^hM4ivIrBe7i zIw>Oc=t!pZKj=KonP%!B->8xTRqpV{m_`vtcy?5WPI3ve^;7*vDr2{yqd;8zSAC@^ zXM+O0@bq}4ws;jGj(1PXCf#EK%2wBrOwc|V3aGUx4fJPZ_(wLrm2N7j)$QT2msM*M zzB?YAXt99239~b4r$N{nHybVC5wU*g5+5MbY@-4f6x-gabAREA*=DeRX3!^O15>v7 zxXDw^1I_M!9Aq|RES5Wj*xu`20k(FEyLcb0{#&U+JCWNEL-E)zgJbf4ih58F2)g*9 zBDf?F-KXuhP&Zj*9GuZ!(Nn(nwCj{+a2L;5hPg1@Kx|EN(3UdT&B>>?zZ7fAmMAVj z?1NlTu~?xym}apdQlb+h7`TrC3G7u(U^OUFA|x3VMCz2sp;6o z7jK9q&Gbktnr+d3Be>&`EJ(VRz`OS|v0lRF(_Vw5oh|t$Ec_lUOyUl(j!BZgg309m z4WbVz-`LoK>k;^AzApz((@c!#}G>#G_q-ub; zcf3c~LQMc9{3-_|r8l1d3LuoNnb0>-2#FC0o4rO)Nee4C%vwWi0s=@r_T(M|7D$(9F60c#LRY1Ln$!BH!|Sd?Wd-)v5E!AEP@c1o zbRv{LQ*Zxy%2A-DT?JUoPq+`De=`c}&JjLTa#i1WnrF|- zO5B_h`97?hWDJ^`9f{I;dVZtM;IZqBV|hZ$Y=kMlyz5VKr2683g5_TG(SYxWTj`6dnQGLrzP{An##F{%JC`TqHG@qwWNZR= zN5dC*@H$SYoHlKSeOGEn0ghBeMCrOVYd)P)sV9b0<-xdGWMVQRC3iXJUO{fV1TvCQ zfYKu7Wl)RPSlWv5eZ?F5y^%hViTBz6-DhgbRO^>XEbQBSNU193t>JQJD6=`SepAcb z(0tY^&flXfUxifm0aP|{#K*sAmu0EThW+p}VYq!0m=AckPJLp`%bTF1gHsT^WhnYC zix37i|6{e7@k3xzTkcqz&TLiUsFyNgYeF7kU@vCG@(;bOy=Kf^%$1jJ@$YLKS?u|i zNrnmw#f^ff01mN8MjOx68^7a)6uv>zE0qGygr}aF&VH138Dvo5h?3X>CNKozsysXQ zdc4b3k&iiWCS=)zH!{dquU}hAfsT)aK&q4bOqj&Y0%~!KMK<~SFB8IKlaB$4;%6atl>!;5Oj>g`%tDPa4kub<$1b-D^LJ-eXCJuCd zs3~6Yd-Z9Eqh|}vgdD_)G`Z&rTzd77@|649eiWG98PZc;l|uba!|r^`qX`ZO+cRmdGerq1B>KE zp6GEFo;Y(eDPw?N$ESb|L5kT#^{RGkkUVS5A^4#Vt0W0?0J?=71p%lFBzz0gc3epg zKaDczMOT5KuJ>h{EXptf%9%5!6TC>Gd|#ELR-n^1;R~rR@!rApt8ZCMgw=h0GF<*+ zeK}3cGFts4bce6jOysf*@k;E8pRoxI=mGS=Hq4OJkw$RALX|DzQ`97j=V8nmJaFWQKfVyhY3x= zI)3TAXeO!MoTsnUZ<_E$jjsRQ`y8!-+6hcxVSG@M^h;*P-A#4Z{|dFDTbk)h)t};O zcm6$=RO6AI8LWbx#Ih;oR&CxZ(KHBV&}R27u6&&T^?gG7R8rBX_DgEr2dYeEfAhm( z=>$SGWw(`BZl@B^s5rWvNq8#6I`l=}XCTt*3(#TZu-R4fx9oS_ICS~` zt=u2~79N`lVxIfuJlwvtKwgxoqTqI~IX`{J2{st1frDBsD2o^5C2MJ4?0= zHD;gTQ@hV= zE{0tqSx}te9()^-=kq?_e>GjYrT+DNi=Q^`clz6{CVo_5nsz_OX9v{tvP_`%i0-WO z8u857^J*|gqdzSl@c_oiJr92~cGyivC+Yih?(N1%x{yy_oHf%wL;^}$D8YNFadW(r z%38{`tvqNaE?t$x8wF11yh@RwmEpIMoaclel2A5O20B6RZ$?k32GDcO0jSZ6omy8h zv(=&5pwDUDns~+v^R@d)Z_VW>+{8J^Bq4qk#pUfJnUF*qrF5w_QyDQ2^(HOu;OibZU$cE-P+^V#h}zApK!1O zELhcLCSin5DbQ&sNsKdEKwI41?L8iAy>yi||CpAN0~yK8eAK2+hhJNrbQ%6Cs-4q% zAx=f1$;rpY1}6We|KOg5g!kJT+h=rZ7y*mlmGYJ){wgXHXLcl+tz@G4y0OTS>)Pc& z7!-eb_@ogIhd?0HT)}?!PgR{YOHk3M{}4_DB1RcT=t-dek;wIgx+{%CG)MeZ(X->s zGC;(GO(`Q{GBao-2Qy}0SUYV|2dK4ZF`9}(@vHKAC}LZYFtWi z?7=@^l%Xg5rI>Nj$K?J*r-2Ls?Sw_elMUnH6Lk@yHfd*Vt=}pIuC?WQ3Kze@J=^qZ zD`CWyIr;6@NTxiha7W7J2?$07bbk6knkklWE$BidqTqkvd23b_5}sz)#URDM-&@Jav0Z59q= zCAuO9HkB=W@eC4A^jS@>Rcd(cB$LFazR4AQx+VI?+5t=eMWOBj!cp-sCZwxTw!v)M z-kYxWwf@9Oq^r=}<{(FM>TzQpHx`Kg-*e8U#(SpojAU73dh(-kT$Q8a5ZFt6I-I8Kw-)2s88u;CibCMsPGJjd^87!$>KQ-BIJn6+fptODyqUvm-Bf9h8w=1-~HJ+lp7=|CoEO8vnDCfmJmXJ{1=4TP`faWJ~(KsOTF}!xAv( zMB3T)GLGLqKq-cJx`hm?VwM4)l>B!eZ}CA{Hl-p{%}yxPWYvi<)i@XRc*oeeQK=-A!inIT0KVm^6$1Z%+Z^iM!_Azg5Hqad%d1AULoKQeLvl0 z<|OWw-LyuAY($PVthe!3&(?Zoa=+nH7elv=v`)Mn7M3i6CMH%1TQaYF{5+mp^g!Kp zp(y@7?b%{=$Xx2w52Gvv@n8#{<^q#1wxY|n4Re#h;!S^7)Rl^>BAyWN$>~ir+mvsMv_v67J^M9P{RkoVIH2>T!y|$(6*jl@yzJ!?FB8?1T1Y5TT!+O^bX)XAwrCn#_eH(m`uyKi96Lr)Wid!(c>X zSYK6q-lga1ukX7Nkwbvor%%lxj95`EA9VbuEGI`=DPDLoyKi0g8dd#n{d>E4U)*CD zp(Gg**v>9$TE9LeFB8(9-N(`nzoXKTvyXVBPzOhizebB^DF*aUi{<)pCFR}yEaqCt ziLNv&HXUBIvZEhOK`+-doleFxkL9eri=b*bU$1_%2Il`;zYI;6c9kpX5HEF%4>`A~ zPvwaLj{!S{S{Hkgul_W8b54ha*eU(}X!!fyNVHTkLcaM(;nC^Usxz(*rPuzyd-Cs1sBuT! zh-4HxYUekqXz;di-S`{MpetTwQ6#j#Lcp#DB5=&GM`lILJOFY_svO3@l|u5fXpY8t5Xf{yiwCXox5wJpK#vFZIKW5_E2HxSV zE`OIi#nl1bn5FjMH7lx+*Y-{RYZ(?Fc{FDKy|vEGYk1b)aIt3W9_&|9NIn1O#f}2` z*1~Im+0EyXhLe1Un`QNKT->zt7wP3u=J*vn@9Xl*-mB`rZk(w56UwZT*&sNAT!R{{ zNng0;?J)#gX>#$RDK_QdCk??*^TC)%b<_=<8)0DV@6*2+vv$Yi*{(>%(A>AF{>W|p39sHKD~jv5JMDm_q{hUc!0 z@h4x}gPhvT-B09k^bK6a>(!SpJe8{|i$Bmlf zw}jYpOsby6-V>7lvc810G`JB^4nFC*?4}*f!F1OI`+3obNBhX#_&NS|F7x$|7e8Kq zjpSjY;oI!3S}~3><69Jj5I2O-Z4~c!{eeIg_sw5b!>UgiY$gvaMGO4`a_Eg zG{1;%Pv>n9%X3Q?p)VgP;{nIX);|^S&b&hRU>Fce7!Q-cUuG!!E>6dLI@7^F9Dkib z4PBeQR9Soum5b8o7Z-p(CN_@-D8 z|0yO_+^Ju_Fj--1NNqcd?J$=sITyJOFFnh~n=%@3@*S(@^7Cdf? zc}`6~m?WaWu3#U+T}A6A!g^pgC|!9Xi_7y)mr-rq&L8{WSJ+UG=?a1a)hE$zKEADU zd#9VV?MDE1PXdp;6l{~L)NH^wOcsyk7;-JI*#+39`rHhgte-c`39cd9zI}QWeFyaz z!UXx_w%mP>%_&Z2`_Sh6HcS!wRgP)4PiO$m?~izjZ#|Y?avqzpe=g&w02rw*U%UU6-`~lP0S@Y87%rIADZ6NsEVq@wN#`LZtP0>f;xLG)%#F z^;X^sm}c%W%Fy(lz9LLeEuL#uzq?EJhw*&1$vZ2ufVSxmGyrio6fD?iIJJFDXC&+U zyQIk+X8+hnke-|5rm@$EX&jD8Et&g1NW78$=IwtvJHS&d2Z4U{MSPr5sJBfmWOCQS z~wa(Cz=PeW3i9AKr?zfO^ zV%E*;vhtFT1!CL@O2>QKyvh&<4P=hAAYmZj&i#ki^_EQm9KdoIaTRWreBqCp;pd3zzL#3O;_D^y8H)4WLLk=me`~gRU zzK4nh->}86tvzXQte{&n=AwMR1_2k&Iv z#(Z<7^{D>#odq*dB+KJ&fOsYMmQD*hM0y{#f-9lO27`Qe`P1a@)O*>r%t`CIjxy|A z9Ljg&5KrFy7S@bvkGv(*j;k-IsqMc@-uhHzYJ4tlgA!4F!?ZVTbMIPaSxR|Syl7b_ zCO!$f9qQ?e>LJ4 zC|p~u)rgOtnE!YNwnR{d4KGHX`^`&&7zHJTX9SgmUn{(R6}&RwY;*C~EUTomc#4LZ zSQJD3m5Xb#c(-%9>UYD`pU5{`3vyp-Y!W1$H?QoT^y)|EZ)d)lIJ4-?-`M^kE09rD zN0s8y8q=-o$<`!MFy%A#OkD${ePi2zF(S!izH_wcTgpw*1e>tpjB6IJW%!D&DMZ)| ziV07rX@hQ-te4~&=H~m*YW7hG&>6y%CR5)(-mIxTP(@6ty}Bm59g3+vO38;?8jH_G zXR*;_iB~XZ}{R?*=c%*VQP` zs3w1;gFPvCJi(At3;Jhk?eX1HzIsBwG$x=IpQkackaBQz<`8>y1`AUfXrO5!{IMv-EP z+hzil72T*6bvfUQjiU^K*vo@?_fdB1pKfYX`P$1~%ML5Q-6J9gss)bENCmzl;De~} z{ge86K0BpEqW>d~C@$@5y~A8znd@wk9>Iwh-hI#*_vR!m^C6#bAB_RcbRi6=5k@34o%M!92_kr4pIQJE{9WOhuIZX1ZyJ99%;fh z)woC}AWRXR#J6M7OIqD4Wf&~&&9vXyb4?cJPfVWbtNyA+NUbC_)uxSRD?V zM(3KJCbU7MJ8}q#Ewewhe5Mqzy8XQp9kl$~uDFmdSJaN_)L=EjVmLG@N`-UrBKvh= zRoijkDu+Fa5%2b|#V6-Ut|EivpR}uC3l}4tS`7Qd^Y4jw>O+2-W?DVVl7dr?{ZXua zTp!{Hh+pSonSF20;ILlfbm*hMMUrDd?MXh0HA2)i)d@Q$WG_-pi=5QXAa9)obu`sR z7wE~Ra(ue(7(y4jtJimL?ZlX0>b%92(^5hkX5QtsC9?k2pSs;uI%1YOo z{}2ovp?`AQTjTq7j7F*iCVs}_cZLG#>mhG0tH1D90@q}kLb+XY5FlRkuyCzR^E6?Y z{?5Y;$&od0!_jR0dxk<#)4dvjaPQjM>p}R@r0fbBBV{&7ccFj%j1W|%d+GXH<7`k0eoW_)zDG&r}n4Gw= ze|r9%L~n~+Y^3L=w`8WjdWDGIloeeL*^D0fVHJYurn%!N@?tbgvF#0xl|+6Dh{@XmKGV--r_o_GHKZ;ggQu)Sp7eii9rCnm!+*{!uR`BPu zxMGRsdg|%0nxv`gThdhL2mQn=?ZCxcZ825Mxc-yScjrl(KiDNQl!GB#eD|+^b>O-; z5tkYqPR&I8Lf#eT3u~3~rKO|l^5H=MDLNm7kingDv_SlQ+9FuhCBP&<5{3oa zc1)*C!|e7UX~hRQOSgKtm_Es!A@}^ztC(U7VgGFh%D{4`;cQoVdEJ9$5hj1{&5o$V z5Kl!&7~lT^)<7x0wJ2vx6~Dv#blJCSJ5}(Xji*jg0l17eWuo)V(EbBEcQ|wUlw`6$ zik%+YHf`EAu+KMx`!{Iy!sBh)Q-S7GVIzo!ec0*SFNU1aoANxZMNXUQoIP`P)5_)E zXtq<2GS}aj%KwA$vy=bZp#!)^MbhwO(~!YUJ4AMF**JPopRb3g6M3w4JCk1)gs(Ia zLug1F#2Y!d-;fX9)`YvsYQ-AAty;Q>`>%PiXy+kc7G}g{YS3IQLC^4a@_c#8{82>}N?gP2mOcX5tNDTcsJnYk>Z_-H~Z%Z9tNs$w$jt}YD zY22rS8J|0)Xdg^s(TwXx^k3cEw;B7{C#^d7yz;smDRlDVhseKn$wG}Y!{HN04u3tg z-=l3_PM@9^X4q!%1o;2>#n9c`w*n*i`S5{*%oJ3v^RVZi$S=XsL0?A>9~%3`5R*e? z+cs~|G4uc9$Bz-e-+wyz^qG@@7(k~Vw#!&+{mKgZQe^p}c~y=cd82v5MY9!3Na(0>7rch+uXPj!^HIu)-^FugcWTqbr8R-3 zgQ@~~ey*7J+p>AT{W@{{Yu!J}nlqOkrQm>RC;UZU(x{2qr%1`NtLD#>-jNV_Y3Ifb zmlP@P#)r}I3)V(MAN*8X-+zpl2&AU(<1gCsG_?)#4+noIemM#?u{8O`)U=T35R(8Y z`d^A%|K;+(lOOudz7Btg*d{TAu)ZBx zDlItfR;CvPZ619o;C8%nS=K5@H%DJ{Cv3E2b44~@f-Z^6bmgbSQZ4<^a7{a!>V>h1 z$N~MPcT!#ZZRNplHL#5mkyRb5hXO`l&7ciUZ%pqbG6f`=hBS82?lDIaQIieeBLt|T z#UDgf9kVkWbFxZW1|lk(+*DH0IS_*;mDNOV{xJGnQNQcH_;a)|p5tb%@M zCQqCp2yxH+6zD>V+=fPc(u{d0N#w}|N0E}aH1@R6q@bATph!g1XNbDNzVWfBTfmaw%h_ zFXtjUZYmiFh@yUzLJ^g~;lqa#o`0lzpAIjbJ`HzCO1U65BvjG(M-mf<^muplz}^s} zf+z@oQH4GZz}CH8o4#+oa`L$5-JxKUjOvs4q3Yq#-tF6-sZnwD@`Vn6Pb7EH?Carw zA^SofL8zN7Kh><%DFl5<8f9M-PLZqww6soL?j!xwxYN^*FPJe6f)Y=geI`Y?ty+Fz z^gA?vZ1g96)!z$Qv!o>$KP1&w#6zL=tCqd^Slt5&`ymtUpW30R>PQlxIhT0u-8L=z zcY2jxN+PSZ`!oer65hRI^D{Loty#9j$uE&<&80sRbK;m0ZfNfCH{k_?NEtakD4a9( zXKs|@w?bS6#tAyIvUa>8uf1zdTwpBe$5S56r?jbi-`*YDq2MSd)2F@Yh~{roCN{aR z{F=oJ)#@VgOEvC3kvArt>-fwQ?%YhWng2+V2-m~2Ih6R66Y>9d<5xCelOU4Ar5YO1 zGC6*A(4^GMp%dS4)AFNEubn=9#sMNrSs+LkQ)c(}El<^|xOVvh9TDn5b5DNwAU5== z5Xg@wx|&Fhn;0o6fc#YBPR_wyI=6kO8jl*m_nhoumgJamv2RhEJ8eR<`^v9cGG8R9 zm8=v$=|~Y3;6UlwvuB5PduP-qy`^4M{$$Q^=!c!!JaivfG+BQ0pH%qE_wFsfnp-oFS^7!x zAb#m{*`R>Roayl+@K2Rfku$^9%g+&s4(g;x!J@^_k6$CATRZ~!RU+7BH8%Sc7%+k= zbeW-MKRSMs&Ym08MM^Q@<>hC?Us9X@i--~8gs@4~+g2`K=;D_^jqbl-EV-$RgC!<) zdg^iJ4aM3-;-Enp%94e(hrOV1_Wk~|K~LU)$ELNbU1g&q;n^thT3ddb`^&_pm2aIj z>Bk^{qU>Mke-rlad#+yf-fz&+Xx3O71e-C2JwBMomGf?$M}FKpIez7=a)IxM4{lMd z0%x%7n%#ui`SA<(zfGIer1H(&s-Swt2QAe1`QrySzJn%`;DPV9Y0>qiXSmHoI{NV? z{{&v4qWt;6i%*a2-wT(e-#y2`BPsEk+wSC`jvID!rcF40_>gh)@@J~}UAJt}&Mlh* z`o*uhCVl!0dZ-YtNaFh^Ka?J9At0xx4hk{=(gA*mp=>XwaQRdAH8E6!BNhb?A9sL} zsAf8%`bTC(hjx9ZZM|xCSBMl54cfp)N*NOsM)1a_Rc=|ccs@lG>wu&Ij3naAnn->M zG2c6FTJ)uBJ9}DkDBGCvOkS7&cWL-j5`$xg?A#~P@2yw3vsJKLmgI+M zh){C>J9pYd@iRX_5FNJh8;E@O*(WWk-MIxnfvE}Xiy#pNjmq_V?zBmdSGjq{#Ifjr z>dHT8r{tNbo^$@oYm>drwJv!9ZtmW&ttyQQ4gn3r8@OB zz6E6J+Tcyhk&3uOEXPlx7+2x1s>+~|Q0M2KWTwO9mul3mR=?3DR}_SHxm$17KAXkw zoT(E?sZFLHwI6ba<#xRf-5 zFctV|_xHIN=Hd&ukn)WdkB%7B_v}eKGZ%PHC5bDUt+ClHvUbI?7aP}RzD5X}2~i{% zpHuh9frJ<8*X;Yo%V%xE#rcoJKiWSTM{z}J^e6pbC7@{uFNL2MzsTIBT`S&)sJlWg z(*M%$KU1U9n&nFo*^Pfv#V?)Y;O_4zd#e){N+`y+)N^q7{V(rd%FZ1L#mkNciF4W3?SW)RAs*d&JZ)wXu|vX>uw zfRzE2wQxa5oBx!$_9kdLT(}`f^95=Y6~9IZ1sBbn?!*%ak;vT1lLG%yxt~9wKkt`G zw1>Wl5Gq)*6!ax=B=lyBMx#FI$4tlX#4q&O)~{UFzEPdS3H#0XI{y+P2 zg^LUhk~ajKe%Y49yawGWLrfl+=UGzzvFTdBEUiRjZ=7T#%&ynaxL`|-KD~8CRoNZo z)BSLnmU8?8D?m?irhSa7|0W|pbp7{p@~P>ez;r9Xr{uH^yPUe=^I0_ zS7oQX5OG}Hx8*S%<4>P-x?+lCECmRv5>Xl21l9Wn?U#q6j`I`x`nE2WX$UKG_q%Xh zq2lO^gyyHIDU|6;6z^}4j6sYW)*!&1j{(h7rVxN(W88XMEO-oC<9Zvu*9iKe$ zS@#c(nL$l+8B4N3hkqXZ03Xq!Fv`A$g<~rJ;Z2qveJO#mfqjjO#w>!^(G~K>CKCDO z$1!uJPIPt5FLcDoe?qr73P0$tm^;hScZaxr>(W9+Zn*QFJMOK)8e`nuD)}6G;n0aA zhlX|O1cPMza#VJ4rVxF8vBVJrAV(0j*?2TMZMt+i%74gNDIN1?J>CoABfy7iI6Yr! zBl*~@P2K{RRe$91M_RwgE&atxUrCwCud5a=z6uvD6T_|kZZnSFZ~F@5hg!)iufS~b;8VZMFkKPI!QWy~YffBbIFl<`hvPk#3m z2^~9n2!XCBQ5tupNq~>TA4;@x@=Kmjq!h0o7SYIx`LiX7HsJ$kJX~<&9hL8>TC?Pp zSEo%!B2okUh#Wn9XmFQK7mQ!}n9i+RES_!YLTx;JaB^H)Jlfw}gOVxAn%K|#=MaM5Q+Mo82 zQdWyQic|8)KDN1b1rG3f_g#iCL;ba~O@rLMYGBw)grk5&6F-xA*P(UvjnBNo-Yj^25S551&B( zl4W?LQ5A#yJo=M9!+O4_5HtB{uID>$xTSgnB_*^|b^kv%IK1`j6U>iL`QcQMN5}87$C2!*~SHn3Lg_x6h?g1z56R|&Ybja!&H6Sw76D@l3FRiu@Y$7 zjvh#Ov-x8?w`>s!2pB#Fr;#*kI((-6`0bo2KM8Pa>U)7OQlOdq5JZMxd`qgQ|K-^w z2P8*%j^NJwYC2S%Y@YmHe0fZb4vR|}s&ZNh-Mhn!(~^f+65^g354L%?r>T;rP>C`{ zOO+8#6OTt7dGwDx`4=VU#qX5wzTv2ogzrkV>-i>=LEdfEY_aY}@$$2)sO(*Y?5(us zSEfp2(X1JsUs`-EpEt{wLiFhniSe5%RKES*>LsqcDqZ>vCXnZ!Ck`FrE&y54sqXNn zz0` zD-)KT^dnTAet;1H8%oV#&+2~Wt>4fS= z-?E>CHvxS5)RGhqInE9A6AdIP2PCKjp^243$Q`zQ%Tv4d{qfPIw^Ae?{9~K?E1_3wJ2Su<$JH%tuTM14!mZAvvA&l21A0bQw)# zq=ad01`CCXmo3MweYx^q%Jt!SQ-3a2j+N5I2>SS@QHyZaSN*Z3m1;edJ$F8b1f*Db ztIr!RayKmXp+)_1Uk-h&?JGI+=2d$qKQj3Kqgoaq9{W(}y7Cp8w0pZmxvMeQ8n9~9 zV$1qq0BQ|JvK{YX0(DC9b}j^$tM(!?sUuUcW{d$XJmX6Mo=v zWkzXow@CQf+bWD%u<`U6o;eDC^v>(k#*Q*igIehG*(>5WgFr!CGASu0eY$MiziL*t z_vdKAm(Tijk;VnEvhnlnbL72b^`iOnrcErWbMlxH<*psQX#JT}r)f26a38J?88||x z=jfmE7PvyNh#gBQ=QkVM)VYX;3S3dR#$!$IsrgWjJh`a?GtLtB@88qw)fayG@jIy^ zdxl6v4cok&Gj~2~l$@V``f@OdMP}dtXLM=Olqh>utM_^_9Oni%EV6Us#;-o^{nLoy z3#0<=?uiCzxMAdr-aVXd&c1ivSL>y&AEnEXLDH)z>F4i9cY6un(_>r!!^$Az zM}JdL^Z@U8_*R5E>nm(fj_Et^h zF@%qB-+uWjHLR-_sVemzUAaV$a9fqogl|XO_sHXx2xL#MobibkKMMU~-Gqx@&HyXc zsH@+yygbwSr?W3|DstrJl0ntN74CiDg|2;=cyjRtQ<74*ms_9Y z0e&vpg(58cG=Ze_$m7xdF9gVPh7F7z5bPpCnrq76L4qxL%{7UMNo$ua`Jhd+O*}lW zO>AdQ8n<=BI?h_DDhy2D*^_@_9>f`%S`RdC{`4E~#>S=PAo+)=W2u+VdUcsL?%Uk? zE)@g{KruJY;Uk9+e%`m6qYpvoq?MC@c+(yC!h88BZ?I5>QkbC4WSg=Ne zCY5T}&z38<9N)&z2NL$uDeDy>NG2MgabFB;)TSK^m-1UK9S+@IrFy@&+8HfLK6~=S z)}3T1*FVCVgvNg!i<54~&%7nVvns0>&nj zXsUXUhr>j7jYprjr&fI)vvToElY{&A_I&MmMiDXz%os&{Hu%w2FXzmYFUZf>&~z8D z2rPu8umKT?6fJvIqqgl7eM=awPbgaMTAhDceqhx4d7o}(H>8y&%q=o zkNvuPyH>2ZBZA>DciT-I^|@C5^5r*W<;AduD%QNbWGOs}_;r+5Em;WG?))M;esN&h zq|X0lU!A|;p?rxfW&z-2`e^s}IsfGz4C!eR=L}?4GWgBj4h1^|fYc0~i%m+yox}z> zSxg<;nYfVBHuf`Rr7-4IW>A^Y=NCWB2s+ib%BrV}f>w=Be9bPxIBD2Z8u6OZ#H%Vl zQDuFJy#7oWBWi`liGxC4e&$@&JFdnilloUSCzhj+(3W78O(@}*QP-z2^BRbq{NJ2R zM%G&)6@R9QRpKaKD*5e{*N8FjRYP3M=2qs3B-$hqKqJ4^aI))j^@?Q!-|67+Wq0qM-P_i! z+PYz_e*ujn2pM^)TmK?layLCRvn5AYdP7C%-&O0ujVqVdXwbOQgAHTCyxKzo*v79k zIr8M`(0^F-svFj?Sn5j@+Oly?&U`#jP4k$nxpGqvoVZNwnn4^5^RuL=SB*tXiWGhBy*{-Xx8RA7EgRNcdPQN;HR@_6)lzQZ z&=YN6V;mrt&?+@CC9l4**O-Z|9;mc+y;fvo!`UW`_?$(OZ0rEB)b`Z~&+rlYBIMiG z@VD1!h_4uEBTFdWoKVsi`?4?l3N6{!a3=#Mipw*Wi>C>rv#r;S6%bK zqs<-uj{aRW>TO!R^u7jCn_S~zMF6nq;N?&zdpKm8UTkD?98^B6J(&ncxKWoy3xYQhxYt{Cp;yk;lw(KJeI8AQSn=`#)B+Q(c-?_(9Y?{{R`Zt%e_cJ zdM9mP1n& z5Xg`zi(-l-XdA1_##L%R!njuDxRlN^WcHl7^1SiU zuom}~=V3ZHBME)Y*0t+2*#TkSKY54CSN(gM-ypd#x#7-AZwwfo!KNx){1zxu^!fLC z)@s!3t*4*ZwtiKCf`y!0NxMkW@SY%c(4tfJ6=Ln*CQryeC5rL`qw~T=O7h5$LJKaT z5Vt{9t@B8HT)OD^HUIbIM^~hZj$aH%w%E6Gr%-`6f2_S%sF=h+Zk4JD@$2wskX^87 zN%a!ed5Etvj#k0!@^WiYlOz)6;ip??&z6H9tXD`eWkY+t&zj95X;veko^0RInNlJt zh9i|~Jy^JSX;znXZa~eUJwB*Zw;^{_`TZAw=tfX+F|948#PP1`bzbf9ahf>Je!`{7 zU44JUrX*Gr;uzFE>(^D>rI@5)wdr?ctcUFF5z|eH*Gl7`$b6Bg@KKDVdnvGj_e5MgA z(O!B*A(V6R3#6a+`j7<$sfYYX!5l7&J`^ccmP5`z)1^;e`uZE+y6)3%?VRv%611GI zKiIN0bJ_fsqSn8YZ3GGyRTYylUwf)J^0P*Xw^P)p)3Iv69WImM6?bJQCqf5HbVHrE5<` zeA<231|Yk#k!_DZ->zXRonJwWDa2CcuIW97uC2=E4Xf2ijl&a0e9Cgv{B+|t{GT}y zWpO)9-^$z~Yad#d|4GXgVP4%WdzjS+9MM0XF44iEtkldFE3Q=x%a!*e7s4= zr=MW7!wcyosvlo>*L-l(s-?{ORjKQBB0O=4BH)kzGwal$-!N8!TgPZtn-Vgww#lgp zcR}U+lY84o_V1zNQcmu2^W9b7>^C%R#*CT>wIJZ`;}<^YRqOE!Q2l%8$I8?QR{Xf2wUc|xFFZ-o5`&nsHlG5agRjCkNC9tD{s*-I(~=seE-;y z!$JJkZ_%d3YaM|)x__)xvo0?OX>spu>SqmHI`MnzG>q3<-h|Kj@rwt`IsN=^^vO~t zx5P)%G@Ko5ShsD$Fbs=l#Uj$Gnv~W7}(KK-zmJsZ}O`25p+M8t;xB50k zNl7dM!oxD6&Tcq-bET?o{NTo)chzJRvAlZ2M)znGq4=empMS#4)V$GeSj%dRnHfdE zL}a$CTg}TWz4-%|f1pdccKs~1><7}{U9E1rULVJ@fND6*HNu*YHFxx3f$$y*kO16cu)1Zg^QJFcJCePw~g%Zp6{_nbvn|Tji00Nr=9R&n4K|8 zR&GLZlv(Kp;pAE}pLhNud~AG3k=XnGBi+ruF1`$i0A!Co`_cWoS;>i(Og}n)k<+6u zJ^k||aj|5XY()7Fb& z^-sh<=4^;nOI2GHXe!T>tz}D(zRxI{zB}r;SaaHw6VwAiMVioLSymSOF;Pv1Xcc**=Gm`bs-Dy3nNE@}X0T9coR zt_eULDjI1CLai7Y@MTirc*yn{h(Lidcff=W&}M4mtNuM0+;drpe^bVcAc-^6K`S=~=v=Z+W^3#sVZ_$%EM0wA)Es|0?YNl139DT=_ z0!m~GH6YEJBvQ-}miRpUNteSf!$GHTuk{_m=`rjr|40JS4<(T(KbbiD{bAoc@zQIe zWB5pvG)fDizy^m%q}AJ9SzgXWko*Q!67Tn-Z;;S_=N`e zOCUbVV+bYS0(=YxK2TeHoD{~x#1a6IF#AeM^D*q}gi}3{y85QuSf)NcCf@c;F-hqi zr4GGm-|v5^RZpAs1fV=JA7zFJqn%X ztZv9{$)a!ZB^oHkpFKP0lZO$Q=!jxOjv|A}l7{5hImAR)z zJ(D51m#ltU@AE^GP;#W)!9Jwk?l-vP!E?U0Hga78IG63;)d7ve&bfS*^C*WY_ks&VL*}LHwG3Qp?C$?dbS|XmtGU z*|zDsVIL!#88pl_kjBry>hh1;N5wA-_f~q~p}_75yLT?0^D7P6qUlRA{`~Elo6B>G z8yCY&2b&W3`SI)Fo{T?yT}D&vK#2wXUGwMP@73qKsrT1!#JfC#_=RHNpPWH8P$H5P z_8H#0eRHb#T{v?Ji%o)$b9ELbc%$!NKIK?eNMzB2Z_StKRXxA{?!m*-<2lY*U9VnC;O-6 zpCgA8M1hvpC~Hm@mOC$gbtpN$OgrT$7r$eN4_dWgu87#iBLEGbd65>fFA*7v?A*2) zH!vAg1EsiwfRHwWDeR23=yN-I#ai{KDfPqQgk3wA%=%Tjn*i{07hVL7x5=*M$Yb3H z#KgzC_?7&sp!`F_y*qY%2c#6R$7eY7&d^bs>CnE}+3$aOm(U9z^trJ@Wrj%F#3C}J z+dE?K=bwMH|6MSB>bCXk<;S+s#k|&Ua0X^N1fGYFFCE_d@YDy zUXn6q*g)zE0h9loVIxR85cA|Gwv_W2nZ)402rm^)bUuc4LhlY-h< z$+boMPHx$P==g2fzGJ!TZv`X+L)K>LkRX0DWX*O*mHRzgSbRT>2FT5Jq*Yca!|KcPHF;&$EA+~V#uR;9!`0Lw9StWICT^?&q*XG-o z;xLcaXRr5^Uep2|sTtx$)LW4fu>=3-{n;{%9k;HH-T{S@<#+L$IY%yTO&5KOUwwxe z5#cAg_dzO&OQaLidBscPC#WxQ3^*vn#H4xQgPzQ|sj>voKh};_1r>4D0u47<)iH^v zO_j^tT>h0F{nKy@p$ab$za}m<#Sfq6jS|v_(ny4B*Vo;CH}R`sld4cj{x7=x<6Cyp zG20jyDSl^87|Ya*bfVOtxx3aw?R$Kr*-=NDWsew9lZl8SOFf zl;h_jB})&Q@Y@HUebeNH*AWq=oPR3s@*DONPQbSA)Qy=A%7XX>e_n&mJeT1hfvC0{ zw`5$W9WQ=yt?xho1O$wFn0R^oMN~G`RFoXQJV{&%)k%z>C!yL_eUwr7fkLX4UF zjh$Ro0$(_Oxs~pbXI~P1Z6eO=grV=wj~GrMmU=`c4@?+yJ^dg!Xh;}y$jxu?3rLlZ-6yrYsdtaCC?lnO)J(#jNQt&S zX{B=8wK{Ik4pBvw@Ey@tT@%C67Yep(jn&_(rOB_x+N>y`Pl1e2v2q@`sW!jGLMjmZ z29F%qM04?qqpPxbX{r_oZAXOE-oj9K+Ng=qcb!BueQQbo?{W8RBKbuF5y<=skPFc_ zEi|D8eNRXAmX5W&n_nE19G=1wj7_xlUtenM_t9@X`B>kM?OE;-FbzSzQ^-$sqY6n6 zpY!D}RDz|{lj-jyJ4gae+X+K_5RI5=_XJ043Z(rR#!Gk=G7ghJ~*$w^y zeU(cDM}{9LbOC&#?Q6!P9&PMHO|>JJP@=qu|7b{PZ0s|0)@-lz85$oSe?I;J{Xg(e zM2K+CR?3)r-m;UEWZgc%5nym;aaft5D2{|0KK(-NM$AcjN2C4!E@5R4In$hH2X?cT zp@HS&ldnKQbA74GqZXUjuDXEy@@(OwMm9JQGC};hgk%9l@7T&)Yzv}KBUt{iJuHVV= zYgL?l!oR^Y&v7kI_B}8E++Cv%3qw#xkVQgxX^zVst6&vhx0wVNjbE1T=V5*| zR;~&?*5+ld(!2a)NT-P3+RdKyDXadDO`8M%VP7)H@bVu;UpSm8Yqr<=42eq{uL4O! zp+2MOQ-*!jOgW3A&|puWYJyT$=I9#*5la=n(ew)z=ZcUCqQ>D}+qQ}Vzw8SM7R~q- zT~%fqiG@WFIGDnJEPdi9+%0u|g?rL@%L7S%mQRZ$GH|A`#xRXqzpP6xM&EciJ9zn_ zcC)8EHqt50?u{F1Bz1lIk|XecAD?9ZwD>)CHEu8U-Yb*lckUE-7SufWAs-wBHa)vC~oqvMM z)N$XOJ$nXp1g7pnuc$XBh@neyl>6fc>)&fU_JoR5Ut+OpviZM6;J@pC^@97Tt#;#< z54UU^;2-!uEqH>XQr&t^e&sOTzA+;@zAAB|C}^Y{(oXt%1g)I^sL=91>uj=6k&~ZV z88T!nU%47i$Uilcu!gr>z{uG6LKb)15SXCisVtQHLr!5 z^($}0ka)(wQJa_J(x$VYs*$SywWOq`PPiRgHl~g*A4}6E)bLrlwiY;ph$BK&|8jTD zhnS?0Af^lxd8y(n-=$YLPAEs$wq@Na(GKkEi4fJl`}k04>bS4ZsLwK(LgDN0xJQ2C z_rH`%HGK#SB&*S=iK*-Xt2ave^eN&>f+xR6QDie@%l3M&!K@h(#2182Yra#^rvk?1 zeM|@T?(rmIiH(|dnM9^ys{A921iOPpUydg;x=ZKJ?)dcMc6#6m$9)sr(?NQ=>Fw(KjNI&{WF(jlst4_f>8+ZPKs7UCZcG==D1ONP;_>Gp|Q7l}NnQP@A zp+%>>b(_ndf&cUT6r?#U7Lm~GDJ)l~{1e32!rx}_th%cb_z(DV7w3UJ03v75e?CIt zIg^>}@HTFFxaO8yIbZ?f$&Vii3Im6RmJc7b% zr5w?J+hi9M59pi6x;-5a=u<}w%su6zB2E?rLVlTb@i4`=qy!TmXS!%_=E7sDMt1Xx zsEJPW#t4EnA5B0ZB>Lj%?q-n(8*y3~eTg1LA7#bb5K$`)M}(J3RgRppF*81*Z3r>w z!9l1gPFPvHI)d8L-0lzL9zoIMxF#zceNJ;g!qj#un-a3F(H9k)hfYL(qYnX2p;bYm zFgI%F07hTOr5OLDjZG9q(RWAqRE!y5_8}^E$Q;~?OgeW^f~W>nLdTCB<%5~!Jn^%so3c|qvd;X|INnz|G%SRsiz3~dmTBT3mP`jVKt zo+Nf;$()%&lnVMEsot=Op@P^tDtu@UHLZaHbp~uQnD94CIQhj&thLpDvaLZO`yAT2 zAtCuCm-8PXVf0OYQMT%BI(N&S5pe?LO4Qsu7^QH^LFI3s!ZmipO>XIve{(nXMLe@Z)n8>QxtiV5g z`r^!n2XL_dQ_rW)KR!Nb(`C$)_Y&%upy7jib|(7|zj#&wYxBdM2peNnH7g)TO8fzX zz|oK$L>h3}t0 zoH~Bh6N;*bzqz)V*5pS(lTohd`;XWJ@f-Lj6nJb$ghWl#_Jf&|L)+GsJSX z)V2n2$4DvSSNT6GzNoi-J-(Mr|C-zHmJ7p4QkwjttNegeJpLe;w2i}>0{+M2_s9V* z)Hsg1k_mcl7d}D!s>tFCrlbWY2NDrqEYHLmIs(K>S#7j5@1M5(DM@WQYsaRII_~4o zst`_oD!BX*&_}KOC`Eg_OABFGDeM}VKAwu#YWXgHjjHf*{qK&= zo5bDn3H9r?e2RMDiK_{J%Rl&a(Er|8p%M=fIQvrnz`lJaPaaFgUr4C`^5aWdB0pPT zUjgU$ziz=o?|3A{60>l}`IYi4KfC;6s3SA9Wa)su2OY&0zYjHg>SFQh`uAHZRB=lt zNrXs*XWEbR=HsaNHDM*M_N_d9hEK^Pi=2W**qP8~KG?By`)oHF;nB{xoD9{wn5Q%J{wMuBzOfE@e3u3LV(L_tf!|rn2NlLZG0z$?6H*wqsXy&4GV9 zZ?*nCpzrbB~__ZsrDoAl1M4@b8`PHDxUn*-?Dne>rd1>!i^T7x8VTj}bmb zKk4N0BZ3-=(e$I_2mhe|g)*C1QFi|LM^^hAo&N$(Fh3wDO>zOHO>TuFv*|R;Z09sp z=X@?!tCKR#!HxOM)nQJ%!s2osG(pdZs5z}f&i11|G}Ykf3vJ_~`4hjg15v~a}(FF+qS?VMMw@Jf@$Oc4rQa(Tf9&przPyuCAD^2lR;3982S;BV1N;S{q;PfB3u)gzefq}_ z9XxqLFB11I6Pm`=1ovVZ@xwJYCz_q~ri-I|+ho&QL4UtZHftdJCa+{;i+ z9kG{_7=2z1<=HoqW*fb`>We;2RxFyka`AjTNaD&3A<1D&GE|~e2kb8#+M-i*;}jLY zQqaeTUpL=#FHQb>{Kkm4zsc-u$mjc~ek3U?$t3hLk`NGaFKg$%tPxEOpPae#u*@nd z)8tS>LTdjp{4Gjc6{-FE@w;QQrj3M;A+%!g+?Bjm7C8`03BzQD7#yh}N7Qd?64|zS z6Al4=+m$YM#NjUyZ>d;SQd7!7!`}qDQ2bIs{*B3uq}fvO<@{f*)f8j%LEZD?_t>F> zCyt-wFpexQkH!C;Dowr$zy zbg_e>6$|mbxrUJHp|<`z@^06~uk0JefwT9_Tb{Mnv(|dwnb}hiBQB}Z z=!w>vf2&lgyxaZ<@%{pl=RaOo(VJ)37G^0HnmVK z{Z5FW`gYnk;aNU;ic_o3T_j1SA@+yo-+43fsgy*25O`h4cJf!At5SK-{SO}T>Wemv zE*zhK+hLckl!yKrA8?7|6U`lZva`(ipU%HnR>s zjt9PgQB?5>v_8tkKk*=2sPyvHClh(CthQ9lXz3D;!^_OS`y75m<-2du>j}*kZyvqkO9FMDxwDCrF4_JMri;aF z;n@VeNi?`{B55z1TFoYkPw%mjKiT}dcJ+#JZ@sGCQm?=*d+oc%&+i)aJrb7Lf77cO z!&}iOU%2MXlh&Ir=Ke(()b_49aZL6{wb6kHDFNak1c4$^X34Kv54WjO=n)ZNpl5uj1LaF zeAF=qFYK4^MtlZ8@u~60{>^_kO|QqsN#FGF{(2>}?lzG&Nwn>8z(Ku^9iRr1s*iH{OBsoS&w1DN z+Er_aSQ%6r6Ktwyx&7xJNA-K{Z+{jlqjt%bkH!q+F;6no%zQj{jE`9;^+~nW{hYhyy_*5{~z9eUdcW)&$HL@OJt|1v9_6iWf`{k z>4K`kSj+3tyKj;;w$=QEf^sB^&i+<+dbX{G0MaoFw>GQ|JkPn!lL+zkmEI zdQD?YBB&_?N|%UJnz)&3c+R8_v@j!As8*=NLoTbpCEKbsmKw7G0brHlYdgVL1IqzF zXgx!=fw)(PC82>}Bwg_(Mb*wr;L}tqu@Tj7 z5Cw>}>hu%_D17k8e})d9zEVbOpItyPwy?oDAz;q4`3-Wu}MJ-2hZkzNKWOqnot+`DfabYy=yK?;}ApDSib zq4(c@ZNli`i@%?}c+mo$fWUcu>&3`(prKKtswyr^N$S`_@TAtNBLJHyDXuI{Id0lX(01Y4XB`u zOWn1FM6pe0yNy)@SzE~?8i_vHqG!%ab(tsCrIV{^X4%V zY}&K|YeRW`OPUx)2mSd_30)~Aek7E2RkP4eJW_&DDo3v3PLj}=U!1gQuy_xS`UpuH zb=;-f*_U7Y^dIjOwlL_#6Sm*v?#fo}cRBv7^ZE`xvsTR=FvSm$p<@0@n)KH~Dq6S< zN5+3Sf3Z>O2-pE>8Rhdgb_Emw3TTLMh-`g)sN0}1O>hI4JXQ_coCFf$;FJE62@JgO z&+OuCB$tgJbs!99`A1X-iTJtw=jRGv0ysJ~8^X}SXhc@fEz93!Fo}%f{Kd{B#sDx5 zJwez~q96a$_<{d&`5WD|+u*qrij=?5mc_qLrg=bg{_;!9xOZRwVC1lc-_Pdp={!Pz z18;S)y?i22j?>P$7QppIk@m}TE0--5U20uu(5N|V+rL}SUpG@|)7M;b!eW}!ntp=5 z{o-?6Wmy&^?i_Xc@)F|@&&*%G*iTK{TK*>S4`SKzVdJ{B99+o+V0nE5&-6uk=^}IN zW#up4S3)XJqc9Lkh$Qsqa$#>Cd>s`vtfv34Z8Rw|)b?JW@lm z@)slpXNCElJYMNekgyDs{B&8fTI8`m%r55Q=(Zh z#gtarP9{@SM1SfvzrfGznNtDF^7oF5&ojPwl^iuvYkhe!RpOtml}i>S<7di`f(O^C zwRF(fqY-o~^Dn4%8Z^n~UpOxve>{G+@4VYtm;dsw_ueTbB4Rk!8u2#&TX*Qh4N6B3 zI;+->I~LFH(imJW-S?pmYS4AZdiAPTuOY6a`Zn{o5m%e0*$c+^v!=K2ysPpn#!u`J znF$RDvPMLS>`V-XMmq*)mf64EQo^gjc}b9T6R*76^;CfoQOaMAyljPe5VW&G zDaB%vGab~+Aa&fr<&b)~lE>%mlHPq{crC-vxoWqJFib1HSGJj-z{0NV8xP!8#m}@ zU-OZS;2`r4`v@1sr}LM6o>j}1D+JNFkc&a1IDg@gF8UDv{NKv_t07w~C?GM`Cpb+l zS6vRe|Zt9bj-&8d?o*4A&-ceeC}4|i?LX6Y)yk#g-+p7_=wb8c&R)D= z!SW>w^aAFDO?%sdlwS^+B7QX39CG9_JXTZK}d!R%KHD!{Kiu!ZsGT<#YeTdT(q0nO_mc5db6JU7)>) zMN1dW<8DfbiT$6L-%X7POpfnFtL(rN%0F=w7AY|_Pbn5-{zW&`;1i4C|NEH?(X`iy zhn}}xa;{moQ&j9l&{~o&nJ==muVohjsDl8GUlc!XCN~%KG(#4e=6Rk=rD|2Un1wBu zW%(<=m+CJeo_}MoFEc*;(EM9*{~27O1&u0+<}Z#BJCM(p zSm6p6@yr^8Yj@hAtpefWmXb{rv(0sia6>qh5EF3zoydRb(Ahn5tundsySZK}U3>?Gx)v5>)<8)%Oz+Q>b4T?sFHSPT;~ z6Bxc6Rnb@NRiRJ8(NrSh%lCq5NYXQx2vBkb;WSlTxv7*PN{KKRP$=yJK>}EaGFoI2 zmf=%j7=FXG{(_~Y@dYjvH3kNJrI64pF!bL(z62NJ+ZP}@C*ezV`bRV4JH~V*bNF!s z_dcNK`F(nDC9PbC%zXZ-KmG0?Zl9B}Y)d{!;ICY^_`%;?KlaU+;|21hGx*8~8YJYa z3m{To+z3N)??TbzabJpfkvRR6I_gMELn(ptg*EF|b6Fsr1HWNo?wg>8QjDhVm_P?{ zf>iSedWp9+5*EO!r&O6GOdnUu2I*sVI8>Gzv{Mlu)<5jmHg&0gk>XZ?O^ZNRbqxO!pLCP@>?zYsCzAkOv25}Ezs5B~81ZEx zU2zP|YIKny0?|rhmsF8LZ1XR{cUu!{Q*{cNe#Un@b;|f>w_BSO$QSxnHM!_VQ?Ysa zW!J7-v*x+S?~fW0=Vn~3^T4mKed?Y&248$757^fLBo(N&8qP0yNQ>~*&ALk))KPwk zbFSNTBlQvj>Kb3$=8anEBhw>3?M}C;u|plhiUYcWDJU9C4b~~hGm)#SYpE}jUxNJ6 z`D?{&hVXGCpw_$yAB7WsBpMwMp2m-imX<1~mNN#N@&m%^0whEM>c9B+sa~z+Znc-4pKrm$)D`zk1Y$=;AkfVwp#A6W8Dh*O9$hGj%q6+L;oy( zY*55cqCfX8Ni32?fV`n@+O*+W{Byow$)bmDxqc+kzHw8GKR$?cGxcH#*`dF;R7jm@ zUcYv=4&7O-RlO#V}5jJ;j1+MTVYv< z^@_?we@ZwfyO^iPOChyy*sM9a|KR*vr%{8zcY@^CuI8o4qM1Q_36CMLP~%B{%0Kl; zP-V4RJatH2dG1~8MNq@PdQ3FZjjl5E2j|}n8`m$IKR3hY{TlQx8vlh8$WkMne>rrq z7IGwMGT|C{qKowNAF z770(|2WF|mX6v7pYyS$<8I*z_%Anyf}Y#m{HchHEP!>8b1?!87}_m9F&5V z_!oRC^d+dg#_DkPu#CQ(kNND;^3iQU-#F0?mX?{D+gV0 z8BfxsA2uSDh8s6*;1>urg@!_|NwXGmd^!9I`CGScgA|_c!rIkqVn}vJ;jrIsOe!Ts z8d;oLOF1_z*5CWL3ad(mw_o~CH9|pl=JNU8C-T(cWdGKfLX`fB4B&(fL!rO%wGnb+ z{=NI!OMmm&-I_|`(RjcY69wvo+$%eISl59nA3BQ5qc3Nqf*!^SopSjcF;a^=h`puU&Z{qqd z-<|-D{T&>Fs7YQiCRW4JEPqK@)X=;m8}jwJN6-4i{Hy$xv2@b-0a9uFutyB4=ihXa zlN3qNwhtlAHyyM51)ROp()riTB{=M6O8V~G&;*naEvd?k@NeKtCD~$o^d+SWjNVCs z_n((7UMO9XR_aJQ8Q~HkaCKqLTCx9`o5MT-v6oNU#yaQ%o6-yzW5TCaeS=` zImHdaxjqneiH@68rF=@IGvi`}56k?Y& z^Fl2rAJT0qGK6eQ^LO(#bhb!fK;reY||Bc6aK*$rsG$Z ziRAcGD`^0Ti3tAX1~ria=&3?!uZ;*iesAH;XTN{;m0kAeu6pO2{i}#rwQ}W?ciwWv z$w%=UJ8qHKpw{_CV__N}Q>CQ_Wvhe^9!oZFSa0zb%SZUv`V*3pXg8@ijZd_*nw5?i zFtB3&CJZTs#S-Uox8+}H2j-8<-#FKS2_jG3ASe{)Q}H7wQ(}Hl)g(&yh@Ygb4ANjg zXE#Xmiy{Up6^k!LxKL3|3uTtcs_H<9@y!vT@eNc2ZQ8h=dfKRqJ}klcYXB9c6=E~X zUpgJZ%UR0huYTbdto{_TiAAgmM?jKQF@K#(F`8Ikk9@M$_*wj~TDIhx(~ceX^7EYO zpt9J%rE0oD<4>vy%9tT3v1z>`hbHoJp`f=lmmUA9Db8PPPR4sG0OatPS6RjQ7Wk=P zP~{N&O!%Tti>OHB^RVLwMEqp=oAycghbVb1#s}&E%I9^Rtlk3#DL(C8 zJT8kFYSmGuyMLMdl}TEpKWwaBz0$E813`{Q*(-K_X#U->W(^P^NL)#7qumgkr zOo&BFfR^&_4Sm$C&dV}Xvr+M{HU98#Rh~&~d$aN1bul7^krc$D{9UtprNYJhU^`DD z#0**O@!zpvY`UUQI{&U)zn1t!AT250yiPfo<*&H`MAeCIG~QbNB2Y=6FL3oWI%WC0 zVZ%D~FP?vo8gzywB>6jh=<~rp`8E30p^_pErQ?w&bJqYV>eN+6vPe7Iyv_WTrWNO3 zb=Cat{FS%Bmt621Ae4vxVJGNt$lfAO$?{jMC8M)^gtdGE3>1sCD)lemN1S+a`TR|q zg%H?E^9ysqQL!x4!fQ$EC;$LJ07*naR0hXafu9V`<>y~K{Ii>Xd(X44@f!FPQ9bxW z|1~RDK5^$Qmk;X8FAz*bj!z88tzEMUY!sFVehjRlT{UWp_}R+%ynfwU!K$DBHf_)| zJCGIQCjcJB6UR$yT6w2IGFVrUKfu!8E+HhS|CFR8i^k^-8`fJgHpoKWJ}#5Phbkzuf2)*>*7iA?Du}r&aQU)pPdRSb%g^z<9fByy z@>gxV{}42)QvA_Pa@c|{)_EXtasF1Tq(?6o=dX032Ok;#74w(9)tK?5B!o%1@Gtt5 z%U`Rg=9JrXkdye~G9vOl%U_jSwrsI?XquQFhKK`q$Dn+r!;U|hZIw8Gd8xMQik8g3 zS^n1GO+!Wd&pd$R>eG%N{>t;4^z=;$BObP&zwB@EE%E8#BP zMbmiGv*+Ir$BdFj)@$!W4~L)qdL7PN{$wdjA8#;}cE2+xf2`N-1U$s)@>g0Jz z3AfIfZFx$RXo(eVc#RJwo;KuPaH9>MlvBgckCa8{q*o%?=`1HC`dVS=jvP+Bn_ZIh&z`d5|F5JQEO@FN8@_7>-_ zhYht?r@-P{RM|%Kb$OzUa6^Lx-?gGM<3}Kh1@)k~Tz(}yMT{iKsecnv6>aM0_-CkY z*sR6k@8?P(hu9;pj|m=;p%XsZuhy`q-C2~2qgW^{$v0@yw0;x)ux4HOR{m-npjTqA zH1K>X{T({ubF2A_kjN%^Lb|2=)6?!itF^0F{ooA4tQ%zWZsDQ;UEHgB}!o#nRwVghaQ1qdc0*>}N<(cCZxm!GCRBF_uN&P0xTrU60kzy*1ALo~CqAqMk zVa4YTJMF|lb+D?HFhxK``AebjlJRZ*`n7+!?h0DKh7`#aetN^L4fIe?&4tN9lEe!i z^%|uKrPle}qhS;&ldr;=zc{m}8ODo~TgTY!v+}yi290@mUNXOg{=UT4O|7cOWw9TA z5Uol;ThEOH^&0X5piLV#P|C7&(PGe?21Wd*@srKJ8`f{MfW*3HEnBDfxdG?>ta$^H=W^PJZn68~7E|oeHDh7<&1wzpq@a8ZU7e`_60f4F7V4wmi6Dm)(sodibwJ zt3Qyx%C8FZuk@wdYHDQArqvg_;S1Hp_^wMY$cWSa9}3PN_qmN8UT{{cdtWw zK6J>~DIdJ|@-vUU_tqP_ltv1uInBQM_@le8y!dy|@K#sl7pl}zoK)pjJ5iA5X`>M* z+sxnauiC3Oi$0sSXf3d!{IzaLt!#8=%51L?z_QC`YgA_?9Un%#_8+!d!v)Qc95^`m z7Zbv#F|Fm(&|m&Vhkz9h_4x~b@vHB?{%Xpk_f?>PEIjhtU+vO;-){T%NJv9#+6m3B zg$~b;y-zsx^=F^9QL>BUWOA17+@&(=P2LKm8v3XL} zi%VpuEYpxK{4kU-(Sta$Z<*Cs_RZaVcHi{XcTs{|GeW) z4SF1QB#&XP*R<)5Ov)1JNjX%s;?iGy${ejLgLyP^|1ksrJbC$G#5rgtWJT&DjT{gro?VuDLVjUAnSWUi zdHQr{azx5qX}Y`%HQfD?H%1%CJa86G4oLPN;?$RRSv~(YXwpoVKFO{OAH~E!MUh`P zBIRFt!4EYRU;{mCwRgDXrqzq3`TZ5axbNLOqD%M|Ps{1AV zfs8eIFAw@ueD;v!3bjK8vR9?35 zlSO^B8FdN1>jU34#FY>;{OI8Z;nWcIosC?sYW3>8 z51bc_)#QQgk{U{~me&zAC?WwmQn(`hlX26or-=qvQEmR|k79g)2Wq9t&0DmJr%WT4 zeNYM&stzjjSNv4q$ExVwMo!<+JT8_KxwJJ=_Ajj@RJULXGU0~4id8)x6n2*3=G#-Gq952fn-LLQeQ`Usb7)14pGE@u_lqlr5EGmZW*amwHC?rtqn; z3Z?M_bs!BJ@}J*K{)CQ}lINjx+&6pp38&PkQJqeXPiWQMUVpeYnYlsp-anGq@vv+} zYkswgi5;13hJu$ccnSOfaD1r?5WZA4K4(-{ZhT_c-X#6^j=>|M)|^ zW}I_pjxIrcIQpH@Zw@`I--*Z$d<_*Emx2I&LyG-i=ZGcYH*eW?;oLb+Te|XawXVG5 zon}ELDsiiVEEL({MAoD`g%nlHUpl5Vng`pWvan8FWjM{Gj%X_#A0$A*j`?&blfTIr zDoAx#KRFkpGHG;{5d916WSdCAQ)=%TeHmXs`Y)+sOJlA|O2-EYQy;rkG;4g2TB-gN z;6A#+PqWr-)6+)a7ut2+rE<0EJWN24=x8h}EQ`Gbx5_WHa}cVR@MRna`EC-@wQ8XO zv)%mVkOk{(shw_5+DI8yRW{;XiAI# z6&p0r=u(+u20a(uuuFJ-FX$EmNX7h3SV;YgSDLqKyLj##;nS^9;Aso`fzdRJD1`wT zChv)p(8~O)13BUNF?Hb&df2mSDUbRS9!uvJcE1+QOHQwFT+MyORjbu7|3)w7Up5Nj zFZbNBXwEE2v}SwkcToS+&PxpS_>XQL|3xNhw`i4xfrx3hOV_H^bcKZmWD_N|KoiA@ zQL<2_5hG|$fe$3HLb>lMdsV+;n@9C_{%Y4wZEP(^aV=ZTnl=?x;f8D4O`6maz603O z%gn!v=JE1Esl-CZmhE@u(}7y4TD9suk6}m=zO|1;P%H+;;A?y^KE0Z>Y&G}W8DZSg zh4XODkDq^0xoOMRvt~>cE4kc?MY;&w3~fDs<=;YqCtitVnJ*h!w(nG&zrIl4@ub!X zbu`QLm#S(0D&m+Q{qjX|UMSS4#qDJ~DG#uXZfeYr2laWo$M|<&R{_s0TE2Y3=y!S? z*`M3*`5{!go6J4~PEjWTkqhEq_yM!s{0;rxuyp|hntx@t zofo0y-4!W^-YO{*Kbn8{=yC8KJq}s1WYP1FJ^1o7PjLB$8K7keQ`^WlnA%Q|v363E z7H#x!O)q-3VoZ<{Su}p4q=cyvBP>ClkgtM#^A4S@iT2;nNAqMd-m01v|Q zA+G|{6oZi;{^IkZf~tJJX0081^*ydk{z~PTQ6MDfQ_F-&>C95#z1%lH{M3bg4~qMs z)u5#0mW$4M;+^;F)^8Z`Lo2uCqcnea-*5jWEn3c>Ll$WH`jd|q&7ISzMaxe=9J_Q; zyb4zqGPcu8E%-nviK(~#wbWf3#$Ceu$jq%=qxiv7W#-@LOq(pfh7Eo0n@>Lp{{qYV zcP_u}_q_+4!YjQcM~o;07$1Ir?WHy|^cQJtM*ecE6R#JUGjpcHBDaL>3%1lmyIV0o zDwwRgMtI^rjUUNje@W=+U$k>$rq#0ZZ{loQDrMY}5ADB2{I}ASqzQs*5Ic055z`wuj%UbEIAeUH!d-_H22J%C)vu%Z9?)fDZ5TCYO&YSoyO zxgG~%9Wh9@u=Heyl~gp^CDLClLT?^r-Sa3e3f{umb^DhGy`6pa^)tTtlBlqM3)QMs zduRSzWJKn7~Y{XVbr_wlz7c`-m^QW3FJ~4EInZDogvSCGx<}I zsV0SLHEOWWeAw}W9RS`xlrf^@u4_2iku9>&`wtx7aSlS(YX}gA{cMXOzg(*PjpGYt zpU1amZ8|QVr{gTc=h}AaTAk;Nc#sn#$v~mAx7*@B%U>YuU+pIZh3k3Y8XQTjLd}{r zx$Gr5z9`CHQxW4cNSYtz2F!ANVz=>y7}{)S{zdNp*6}X}#$aUwFj`CaNvV;p%3uEP zO6_AYJTkLu@+hn?RH;(fQZ>IBSHc-S5tD(#<4`J|f1m}6y(9tTo`ZxU#smdS(p!y0 zsnl?}dpX+J^i3{wM;V$)EWaJ|sZLt-g?awwBcPl|+rXERv`g_*D2-o&D}#@gI!Mzi z8=ZH~0GhZZ%KE>AI)M}n=7VioDAY})MJalWFEV9(m}`9D0Zplh+B&|JmY9fFVX3WK zBZ^^7`m2pI&iJ$q{fqH!x-=1KT0|`W833feVaw_7`7ET#pb~Cm5@MKN^2Njnk|y2s zeZ&1v_8WX=^%^{0s|-If3*{`5Pq7f|Do*s*zJ7w`hCmeauYe44RNJIQTLVZjcJVoN zo#_t@`c=d?Kl;@s^rxd$w(7A_#J^KTCImm|g>S8vR?G1Zk}_Y5`PbYf*oyqiK(S{$ znj~tn&EP|{Ydh6nB261*P$a8BWWlc};~k&v#iv{&5%E#D@qs6gKII$^0;Rv=ycplj zXsD}J_>kC4f8t*dGzsx^w32R66hGohi@L_QzpT9DlPb;P3IBl)D5lq`-{gWDe)E?R z6Wey`9QhUT`Nc;QgMY=R@qrF0@vT~=4g8o@xPwds(;>f5Iwz<2&i zrueZZQ6(9l%kdxj%Q}Q6OJvN%@zFCaF@hwn-R(&$3kKR^+{6{xg0;|E|8lkhaS{I; zHf&frkMkW4JC&N~GVxzNf19;x4^e8$^G7ysn)W4+#!&ohg`YJ3i+@<=!jMMIS|;cyf7X}Q9?L!L+N(R%Ym zQbx2MemwWEpnS9|Ol~g3LAe8x62h5rNrSYHTuXRl4U&lZ3x&8d*2`m@faLZqYeb|N< z##t2oFiNv?nqS~IYuy$t#3{CH-Zb@#DXhf&QaXMr+!FrL+HvC0;zy>cRh#NbA6lrQ zQ5g?4L8g`6?S94(@ww^;)&$-!y+q@lgeV3#B>H0i z=ZF(d34TmTqJPttZB3D4RJ(F_HIbW|1 zT*Qw!6v;uh)MKkS{*l+DWt&WYE{k3?Z+;wKP*{=wa`BVoZ@{OHgT8Y5lL#z{{^BJB z%3LM8NezCQv>{)_k0@8ZaFvHh5gkSO1vhl1Me1KemgUEm;$Gq_x@)DFe``dcKmh-2 z;)g=fqFs*bAJC)>pXwH+#!t6GK{YPcKf~XlUOnDwB;EbTV}_P2TJZ6>kwT(TKELZ; z`|-q((EotLj#7Cn`AQ#*e0%AF`P09cOhLrY!F`V_lV6Vi&nJG6#Sd~goasCGtm-wY zC;m<67kgi7BnyK;Xr8jZN4#4o0(qK$32$-ZDpfg>o0p{@KWPWP zRCQm0#Q6v7uX_IUG+3Ea24CbDU)qFVwPoG0578`a(vI6q+>*2GI-wYaQ4*iwB6-SO zifnhZT2c|?1*qb!;0GqnRI8RaB$24EB*>%}INH-DPLxB$f`Sbd(O-;tFGKBg_RH)D zk@I$etWSd}hP;IBG^USO51g2+*#I9X?FL8EWcbKpP_)L>=}idQkS^#*rOsx_Divs) zQOiVs7b_VPe7BM^w07Zk^k2ifL+5`FHcExS)*MN)o#Ida=zSxJz$Q&w9Ku8Fx1m2> zq^uOw)!X_KIjA8j&^{QUP|W&D^DUG9==Agvt#bf?gZl;T{d0Bo2;257#w5+}p2gi+|L{zuSp_x;yZ-f5RevtnU9% z{$jfpM1U)2VZ8CJytXk+C72@a?Ge8GznFiLLO&@0qf()%u?Y}3=dg%LEest1c@%Or z58R3Pkz=8Wrno2_f6^?oKMI90ZwyW1GaLjQOX^=oP2yi7x52bleTU6%<$hM_ELT#{;3r;_NeoVAq|U*#%Gfw zyR3i2e=$GwFV0`C2QJRv+0%K+Xkuh({>J#@dxP)3?h+45Zj&pVarrM>wA~qDS^lC; z75)Ol{%z5wbyyVfKkDsQga2~*i>kzr^p^V}PyC1e#rVbi(2jHG{!OS_8!_mGke!F~eq0=Sq-dY1XQZp2$>`zp)XRQYe@E z3%MCze=dA263(qvVmY1P-~7kZwwpT7ziIvk|5QyH`|e!5QWW`Gnze3gg|rU8k-vl~ zi)c|&^2t($|E=clJ42r{`GLo~T?aGH(!Y3oDBi!-7$3(-)*m^F3@?m}yB*Z?+-q*6 z%>Gm){&T|Ezy9HNR%it*{?#~M6hE}=bK)t15TB!7eVzx>k9q60a0o+x=iR$=8w(JD zlZ~Iv+O(C4vQi&!{pZsOKC*tq{_|uV>n8r?4i28#?4eTegI3`TbmqSVNIB%2^0NAK zDO_kCpYOczOg8_jpW~CB*v;ow=U;q>{&Wv2N$)cH)6a|5P$kW*O7tlc|1H|K&G-k7 zDG6vwXe#NBvts_yuiX4gV>J`DnB7YL7V#q}NRftxDHAm*v%pcqCQW)CeS(OqFn)@1 z(a7Tbl4e`a--G%dBSs9$$D~oiK6!5p+Y(k%vd=AT95X!2 zFUk)(@|cL9;{2V$;f&{B8Z>O$D znx(+Uf6ZF8q^TTfmfw4e8+Hs@H2(aU`~@KqhT@)omoHkpV);_|3H$~Pn{73JLw}hQ z$49nXBd`WYzPEDxf~+LJ%Eo^&J~>)Eze@+Dykfs%{+8SSEXBvDACbTEhZjXsteH#% zMqto@6&5Ix00mG)o6)R0)Qtx)0K~X!aD0#XPGtoe2NnE9_{^NYfkHH7Wi{u2pV$ep0 z{xXyzJk5e@!$)pGnV5)ynqxdk)Z%EPhJI>C3F!lll(dhxXaduaK&|mLU#MphKHEC` zy;XcMkm8#GR0J{by)I^o^w02h-=;Jck0#eapg;}%T?WIPCJp@`zw1^hMzx)|12=m$ zZRR6O;cr^cQ^eIpx(4%uzA%|cWmLPh!uero%vIWAD$@`YT? z87B+v61UCpo40LmBn?CedF81mHgD$9*;*6|LMg?^p!bHo`Sp~K^IP<2xOnl6O%ICY z1obWApHK|`!4Fpnl?8S*RRliW6@j5Yj0R@r-&FsM|FZF;aF^qSXGGf)<6Rki?B{|i z&=p6t)(X_t8A7gBt!g=Z z1#qH5sedzk@(Z0>ci1`V)HI)a{?Yq4_$_*@XYqsXKR$nH8X>Mg#O73g(6`1v1W5TG z%wK$J=OiSu&l38V@{hAcaR>*K`9_;w+Zz9Q9WzKPHBoc~eiA>9&zg03Ns1yz16wY) zZ1Ey}ZFT&~<=b@JIe{?H^N&BkvmO&y3171K3;vVwtu()oQ9ggAp8TxO$R`Z)SYA4Q zZY6(dF6#NG$mQYb5ubDo{&_5Da=VR@8|fC-GWjbjoa1x2rvSr)dA`|8N%fEXZPvbh z@IyJv(7!#&10yQ(0|KfrXEU(ybA2{`wj%b5@Sl71K|Qk6`MZ_zIn}=`zVbKIpViXw z*(#OG-;Q0n(LKxGnNzQAV|;cq%H?mV{>E>=OBXearNI44|G9xj zGj2P7pL_hF3?HQq?0rm`_$(TKVB`n$w|TqvAw51{edD~c7$Tg%@<;rYiOmR2%A{Bu{i*F63^efoQhuyck$=n1zp4%Xci>L=Simggc+n%=70ipFQqOUF;2 zYZpIg%s=g)IDgN)rT*&?7p?lsR{^oY(-!?n%(y`m_pg0k7-%OeO`ahmo zxs<6Yv8T_;r&-DJ{5NmYPED{^=f(=JJo5zi^_A0Km1c!eb3CK>=VYrQX5G5A+#Z$r zw{eqZ72?NtVipQ>r%eSv+`)^r^=;Y}5l*6hD$BW>AtAirbQs z_uqNro6kRGJM*u82(s_>semazKe{*46hAEZmj!{`B$Mvn5;~fHdF^pBR;XR>xySC` zv`+bwn?)?fJP{>ec{925Sfj7IOh{p#co$7lJw*P*>AHWk9(Ul(_%H{B<_KXwdf zd8l;$wrtzJ-OjrP{~105JGVdUxYFkEh##t0Ef+t;)vW2$g{*4z=1E{NKIr%^^RH^8 zq&_2BiueJx-2QXxojbXu`r!NL9(zzvmbLz6_}|+2mhC@h@q^rK{{3#+R{|Bf>}l7f zn^`X9Kf|xM|7rg!zSXx#fAF`Hzoqy^`X~E`Mf%Iqh@T4h;>Rcz`OoqCj5NGJCfl4X#@w$XZf$cS`M1MR6v87}jFOnlsG}74= z9H!!ss*&1VZGll}7We_IdzkbhX|-239qQ7{8H4wD@wW~@*c858=kP~YWNJ)M+_GqB zV>WU>Ju!k>+Rzyn1-@Dd1oQcnf((Ls!73{}fJ?rCA4IFVL~oCidqkM}RpL6Hz-N=| zCPD$eGyM(Dhx{7&TEQ&gr~2by^AA2uzVzfHuRZ%`k&DR|!$oMq#A-0O-ls;5`VAUO zP5WTZ%o%IetfZw=-elR%@1{+=`jlf37ml!8)Zj-wkdY^S@LZ#2oQO>v0r)@PA1zrn zCD=CtP350{15f8ZI_aTPV0qq&^~kL^F+cU6J^%Gfahe*lzSMVqr z)CVEj5SSKW=@YPMzBk@0Q6_)ot2Do?iepYws?1LLPx#@bZL~L|I;j-n14_pDAYu7joYzSfan@7daT~ z9Gj*%$zL>T-)(o(5#gPm`Sq9C_?+c0+S7gZjA?(r?=Du%9|q-r@%x8(!$`UORS+=$ zqJ26(69z3>x2GWZU%qJJpYOUQhM;o!Tg=avO&gggkdW zNniPw@$HBc2G?!am~A!Zm}K?p8_zv;<@;c6^(b34|nJNV+O z0{_1@|MorU^cuBlBiESurHdCldfSb>lTzB3oqxIG>8ioU+FrB?z{oWF{B{Vc<` zLX;RAHuL$aHEQ(icY-!GJQ9FXn_=SFhwnMEUH!hDn;*S%qo4LWc-@MXTg~6V zr>c1VMTBf|{_;raRFw4m_`T8KSIplS9}el=k6-nrndmEU1`%)c5@RFpsZ#KAnxH~5E_RV$ZYcix~?t5?PR8~KZpw1t_c?!EJ$?~U}P zGNN|9hNoYAH8{olhpZbG`_G65$9lm^rTcEY z!X4F_iSn>xPs-w_c>d~p@~O-Pp+84}ix(|;^wyggeJk!iFPiuLRqPi2gwgw^$gi3; zYt?6$DfKT;>RG*dRknX6>cwfv?CI057}Os%GX6{FuXO+Ur~#+cs97`g$J@)6EP3p< zo8)(o3EB9JPk5Tfat9o7wD56!y8i|~k)wTP6OJ0O&rD28P#-rcVO@S>}yfxG}^Ls(flXx z{0$R9dH%isAd78E$;J(vHnH29Sx&Aj=U-{@gLmXE^uPP^+gB9-OySA=Ld zDmVZB-#Y%qQje+PM{^{MFz(|JFB-QM(+WcVv%o|_uS~IHHdHQfWW9;^aBQYgrOj$U zZ?)1sq80$Pe=I&@=uYsdqR{iQ5--CvU1v~JUDs836crQ|DS{AD5D}H$A+b=TcaRzt zktV&jTu~IIN)e<+5C|Oskxryb4Mjj|fB>NkOz4cbvR|OmMs8F7sm8GG8sk!}wylt9sU$g4=bpl{!x6(cY!F zsn?vaGEc|-kFd`%m`KOV-+JDu)xEeI?XtGnu01e$uGKg;qR83q?Ro2y@Th$9>Dqhg zsqZoHRgMU=4w1iO4_6Z%{_`n3f9L79_*?JqWyqDm8-r)l$C}W=TMc8G#b03|gN{ki zzzqu8b_oBV3!D6E!3)0nn>8>o-(T?&AmKk()_AgYpu(_l9HPlgZCS;ycAppvo@V*(b{W;V@nRjodALJ(787p~tKGdUk5;^&fI*j{dvT zBH1B==l{*C$9dpDg zqDp^j0&YM4p>uU?iX$~H%WYzZkgMitr%LWDqu3Z~yqsMVAQay{)}BzzCL`q8Q~2~W zoYb!POUQRD<-Mw~))gUZY9H~^TTq$sOS52S-As&->MD0rFdAv(ae-pVXir_>EfJ8g z897dFV?e2CE3J(Vv~Q)a^)^`zz-5>^i&_uE&!sH2&;#4bV6ekQT=1+XSY+d5QVQEA zA5OrOlM=MP((!`Ejav?(WjYF@H(ipsBt4Q_QJjcH!3gd--$vShy4#&1kQQ%4K)_!Q zOY}&A^U0S20NGVOTY2^C{ecQtBzKu&fKaInVDDS4VV?I7eU`cS%Jc$U0-+1uYEP^w z2uXY`WfZ|xSrL1KPOQ**3z(ag-5x{xCh$5VPR>VB^C$d!)v9z2Xwp#jo0&5HnO=24ZKp(ig|^m;t4MbLjtRKPq8(dGBbl^s8`n3wQ3 zJQj)5dA-#4=EhywR`A|3q^Jn|^6PWVYx@Ve8YJ@78Q=AB+lU~`8W#%Ey=5vqc}cvp z$$rtX8`7nRyPS3}4zoxvoA}oA#s??}uFc&VLP2n6{5umnEfJFX(EaXdOl-J1LBSL{ znoX+r{xb6KJKGm{@J>$2>3!>ttdX_9%$IT)U49#N7v~FhPaj}wDCZs7dx`G=ulB@# zaW(ZvPg2lC=Ax{IZBeU7*iiQ_dAK84{gRNqK>b!%VA-erJDGul?;AJkD|$R%CilP} zI?^r5ZI)5;So02G`#oAEbZj0GOo%Z}_Jl6=gl1bb$3RHbSbWwDnpNYx@4fS;Tb1Eg z=lUDi5Hd0OIA#Y0g-?vHko}JxR*rUl@6?6(VL`xNJ~Iu`&CDOYg8d?vGK=U@1e)=K zhJT3lqv9)RDAym;*6=%02h&acyE|1M4Zns@B1%CCicgK$DKGi{6QtO~BGL=k%l zhe{XbEKOq}owj!AVM82EGBb7}m3baHtJCkoo8~K(-Gi6YL~ISxF5|yS4;I;?T7>J4 zOJkB~-_3>)nyp{hNgdHh{bJ^Ll>&daIrH1>EQlD73dT`|jp5bwqmA93Bp5G&hw}tr zN&gSJmeb}74jceZ(+|)-XKe3z*W;0OKx-ypDBNMccNr2JlzPN*hgkw%v;JOwj_aPR z(2}<}NXZe?U#k@gVogl$?_#mjq_+N;@KEv~N;b+hk}@BjD307Q+pPdK$wtqC zLIyGHQL+ENaD9DR+i;d4faqIgzd&w}R1w$p_~E^zg|ce6UKmH$Bj?oa$CFL*3jdw) zQ)hLhk6Ij);(((+*dX1yaPrV(rZpn?^xWh)xO=JFb@dG6!%A*zUrTe{a`8hhnBZsL z2(Mlm+2?))zW;&U$9IRR_ZF-KN$*BV%*#Tl@n~flu^C4Dm|@P-?AK;~eRIHn`li~q zz1}Z$GlWIHDJ&jH(i z-7rIE){Z})JX%pcTv7P?DTnUW{i)A7Uz52jS+$KCh-cW?47wTpXi3~?=M-pzG)Q%+Co?kw%DNp?57_okog-mPCPJarS+qih_Y9H zbKLHa{w%Ka3nh@eAxx?`#1#2kE@GWN^X7YyFhfFV3g+FK$-Fk}J$h3U+FdN4ua1{1 z)gzHs>6@_8yIltQq#qT2xO(PE*XFfr0-iU&8iRdt9$3Aj#JJ11-udv?w9m&@QLH!N zH6_0R(Uup6j>Z&p8QqM?wV(qFulgF%kAAI_O4eD_yp~5U_rn}V{9=_%XMC-v=e9kyVB_xRUa0bjn=pD z`#2;0=Fiz+zc#I#Z~rs?^C*PtNb235VU}@K7q8)9irf~wC*^3G^8Azkl5FZauMd{8TJT;)^VJX;Vq0zFVeHQ z-NQ@%FX|<6T|?f)&MVGD^P^%`zXG;9wRxTc5P_A@hys(!H`lbbe=$G~jh1`nFI^V4 zQnRzDlOifxr4N%gy_^&cb(RCvk+3_K{DPLX{r9{DAcprs0P{d|cyJ z_z3*C%ux`VQC?4LL7=jeo@Q2rPN`coAJn2d+%I~PU$lL!jl)*kj+(m#>dwAaHn@sY{zU+Xf79|fNt_;@`%1zQHJ z!F-brgLU|Vk5AZiGgm*|7FpZeoOA2J!5g z=f8(v@;R=J90Gb^9iazBg3jl&a>WJiQ;L#ArZwb;cyFtv>iwr0jd}Uh#J z3G7iiNakE_LFVIR=?k-EV}p&p|LR>!F7VfVwC_ngRb3F)uD^Z@fl}{#6t24Iwp{T> zwLJIg$?LIF2(X007ZVMo{nafJPT+Iz)vry+0 z*P|AzrD>3!)tnX9m3}2gA+z#UrN{&!48*X@xlQ?Q&9d(+FW-GOQ|tK*iSviee-q{C ziavRday(>t{ShcTh89~)CdS>ScS~d&6R~<3)J{#IwbQ4x1ZzD9YbBxIk}xrSLE{jl zmZj7sjvsy2a$@w2?Vo3k_`NAi>~*Loxa!cXj*wp1t;Ice+I9st*P-Qw2RWKS8b~&v zW`*(+SZ!JY7l=6+gQHE8@UNvGA(k8@%MNx*NH69-gxb$>?LoT5MX}?qhX(VV6cf;W9;Exb5VT90eLQr?UdwCY&cy zI_9{f<@Vtczr04fo;dPwEJ)ml??G4Do^WAFgW%{iu9KXV6kfS|oE=p)NE(?7Lbof+T${t4 z(o9$$#;eKd#ZH9RA|_ZbTLLW24*5A%@6)J1YZc&x84KI29!%Hgxt+si`BdU?dWTXn zSat5n6Lk2%yG%oRuMCMm(wVm;a0x+Fyvd;Q3v|}{7}hU7Ou;adw47Wo zjf%YVoC&FyQL9ACL~2`xXfs{f^r~fp!III;;tiRNY2!Xk_wEO3>wxVEgbS;4<(7TF z?oI#1YYa|e^{|O5_58c%ng-Ou(+AQ0mg02ELp#zdOh3VYgBIP?L{xc>%4!tZ(2M_wg&>?F@nhl%34x^T% zQ3x;%*X=v)@TJ7`YG|v&x|m{A)<4*#nBC6=C^III{A*+OLHcRv%}pcb(^bNnmjtp8 z?HP{DhxLUyw{N-E32E1VT})D(dO}JfS(5Oid$9&5l>HWBf6Yfj=zZVzlD=Qqtg{Zd zze?T2osAfk4N(1UmanC<k^e4$=0(Hz#iXYTCikdy_bUO?MGK-wM4|>;pum>O|OIF z9X=X6_3V+&vd{lz@HcLb`yrXrYu%$CwAmkvut_F{5vnAUX6l6yff?4T*=2@$QpX&- z*ZQiMBlizoTpJSmy9SEyy3SVEHWEy#3l?8)2S_g%lnv{*>F~;W^jPYYp>{nS{%VX7 ze^^4{T6C{-Y2FOP-v1iQ(6uU-CNLx04V3KdiI6qf^hHAhHThnz!Ci>U_YvBen zfx%i*b$>)H4z}qr^)`OQoc*5muKZ&6?z^+{MxDFjCL`H22z@OQ_qt|}LT_guTpw#b zhWLL@22G^!D@zG`H46}qQII4Bl(@ran__i`s_UZUEIRRMc{f$!SCtY%P%5v$2RQUK zP8n?4SbC2($lAd`m6V*i_<}Ui^cPJ%mqI&@*5T>qd*XLG=})#{Ay~qsng)?$_E4$L zf=GDr@NaPqE}O=$@=4^YvCtH~v#3^SKVTaF=Ggv=JbPhP(o2Tc@69SqXdxeZ6zD3s z%wN6T^kU)drHZBpC!VO9ez|$~9QX4dSCbmA+~@PSc-E>LJzRV zk4=X$9z!$8+0BLIOqB%H$XV-vseD^hdD&DAK9PBDTPFIs+X$`h7)r%tJ$~^nmvq&@ zxl6-Ws>Mh_)h_?8k(HAF+8w#gZYhfOPlk;GNbOs z$yh0e*##)ZuO2$R=G>{LRHHLsAt{d{8RKsg-tXHU&ZU+aA94OR*L1QnVS(!EkyHB) zOqXLkT@`O!)eSYey_R#^PnsuB>a@%i)VcR%0svdX)d?_Je*jbSn4ML3VoY<#pWi; zu^6>)zV(a0Lflmx=bP$nPF$w0$hprb994@RQxhLS){xf}4aKi0t!t{3q{Mmr5Pn?~ zAT}k+XiB}w80mi0oSR|p=_-eCIh2&@^c5|2V)=>_yUqgl?D;xjmtGuwaK7_7LmsZ` zi`TpT`$gb$7d<16wMXigkyl<(`ULlcYiIlj`>8&WIlL&kbTJVg_T;SU$^Z8P7{b^( zm>qXjry)K)3DQx$-@KL0H>etPGPVCHUj)(RO%Yso1#NxKbVX^N842+>EI-z~6FEEn zVuP*!*u8#_Y2K7m&b*n&P^n7)KOZnT?4SzhwDJ@~UY+L{5c_6{n}Yk0%TksD0NM#*m7}N51ccI-mD6DPjkayIEHv zW%KOWbOa~zX>%tuLvNI8kKLJ=-CcEq!9sGd^Vw_Y;K`9s_L0w+E@M+a{L5l7WtR0j-$~!i)}LuHp?af*0e(J(%cSRCU{5X(WRP?FiZRwXZ9*uOJYEU5lvC9+1$z45TY$H0z9{PwBO?pDKZT^Si@SCEl-#)hHATQko3c9eeJl0X`ivz0qt{H0If z#O|-p%=dR{aoZs?z6_JG;++q&tQQcge7MJL`7g3vYQNQX+e|spb@j{oGHu-H))gcf zmLg|m1=KUmk`SJhr3zldT#DpM>JY?7#x(F*<+=kor@aSnOQ+92XmkaM92hLw=T`>5Yqbb$0d>@Ttw^32>*uHzN`O2@?Gf%2K7r*F*3OWS~ zxll^(us3|sxC&diEm+^e}^>##E^6~21lB|oeXTj!# zc?NLosw5;qiH9EOGr`6UjKu1^N$LP`iKo>?~Jn#vT>N<+HLy<37d<& zwEpyQj?niH(z~rBp<<#fR7vS7!@^T{t5l(?r_c5&vupk71rj@HTViF`4(F=8>{DCh z-8;#ZVz*)}72WAkTDa7B{@TTNULXFl=&bXw#JoI%G>(k8w!Ci~x?yxD>Dmdk%D&e3 z+j$nM>QDP#U_YFhDS1*Lmv=+tV8JY$v%!&f>|V;y=T}Ntd%BSPuiN<%x2AO2a+~g` zD*lojeTy=ZgxX4lUJF;LQ9E!bu&r1y!=)1vjd@gW_$jv& z&MUV=J6u1Oq(H6(bOm)tW$ZlwIwT`>+SkY2TNt9V65{VyyK{qc3Ts+Q%u5iH{D=ND!o*2$AGTY(;a@!QTl>?Ma8izjOO4mA#`R_PXr_9`e6_$esV3(>Ihlk9 z;ODQQ!sEl?djw#ro*TPo{a@s3<@WiJPwtB_p%1{9o>p{YUaX-ew99cXT3T})ZrBQY zM(1vsehHoj^M#hZYj6^i);~*R?id#;cDNo?+?%-75>Ni#L6y=k=_j4FQV+GWCU3AH z%PXd59ipkE)c?G`W}pU^YZ)XFdHU4n@`)Zh7Y@E*$U@;=yXFYn(yD={U1#T>$~-ql z{YVn>_%k!lVqS>#!-P%wuB7Os$aFVn^#9s3R@4cH2m(8O@O>3(U?=*NWhM@e(JLtW z^qo2JhYC68QLq@a$fYI@8e;@1b?RqGfJyi#15umpRY8v;nD0x#MWdNS{l4M}7l)Na$lwba*(!b*!Jx~2Z zXM%)oUvStH%{zm2JE_WJV`H0E9r?T^5XR(G2UwojldFodDYwO!0|JsHa<#Z;^O_8E z4$tf(AUrScK*|4vV7otwjC)cg#ibLHu15b6m&b$6LrDqth|5Ih+$B0N@yzk(sVOI# zC*oOV_l8XNd1cPEtd>jY`)51IvdS;oA~u@2?G9b1hJFhPlRlMqQu=D{9kPejI`^5c zsD*tp2y-5eDwObJy9+~8j>V>H29mYRDzfx;9A)Hn7|{ixtGEzd2y9?QS#Q_ z3r22j`*8lklT#7eq8NI}yiwSg$vJ1rvpr9WrAp=FyW^x+HR-a#8o|X`YC-J|=Tz62 zHP*=Aq+OF8Dbvp<+yQ<`(!c8a5|^g?;JzoK<|6lDMK)m9dr@_|&y+r`lowlidH&PW z1al>Z|72}<5+Q~V)c!tUZJ({_Bz4IE9sgE<5U;2&m1ABzgPX&2(S&`~%Z7&`uEnC%YM|CHU+(!J2;B`9?`uaOc#0rNH2?r-gt25qEak$WQrcsuraB z(HuNb*%q>h(s(aytFm|YpV13N!II|iqNRV@$ykmcTG3j7=r zwOu@ZtuF`6gq%!z;&O)`Xm)H;pK!CAO=$xZyqW&%H6NYzy?<-`xYp`^ zrNu|@qaL-C2!H||5dTs@!*t@}sjO=B-Gl+On{q&ycSod1rjH;`a-`7f%f9L8voKM; zW*YnFw$=HI2^^)I??Y4j;Fz{(AAa7CZ12xx#qb;ty12ue-J5cEOShI`V<%*3ibuNndAa@sfMSE6=g$r zExZ`@TZ6NpQfQl(BSURJZO`|-O0SKL((xLnu;_>Zf@(~&3S9FSWI=)s?Kg%#Wd6J>ijeh=`7}0nWC@Y6P z-7NX>yZ1PI*8EP~y$+r3x0Ja5dwAFlk#vpAw=epa>4DfE&QMVIt>L}%pB?}v-_)7Q zU*6IdhCgsJI`v@i5sT)7y04xS`L}dYbER#4EZ%&+)#pzQk1=ulei7>OSUrX{R5sVI z_{IFqv~`8l$G88a@>Z+dhnHF*TVgHk)Ua??l~U`^VLiU;O3UYy0^~7UoQs`yW*HbNlFQBEOoRxYg^is)|`G zit07hI%C`8n#dDq8Z9IVIIQKiuGxsNab9^?{1hMtnKPQyf& ztcIsXW_hj{tdG0${C#6ui)f!mB^w1-)&6~sM`?rx&L@PGB@0=V7Ef(A>BP5FZ=W*f`?BG1%Ajd_-r7pom2^)W4nJ(d=>7S&wkXC*(~HO2Zc9t#8q zEuD2_NI-7(_M6tI zkbMliMx&Kt@dp;C=vMaMk{X~|cmKg&AN`95At3}NH*53Mfb5XL z`a`b$ytx+-IX`}!e;292cFyT<$LbTA+0!-WmxhC;w!}WE*E^ z)<4Gke)tt1CuGHzq>p5fBwh^N%y6_Ax*hH4Z|XbUYg7hJp=z=VcA}V`01Y zh^tGOu+BcP6AkPmX*(5i?_q6QXN;XI8cDY#q{m%<*T2tOpp9>7X0U_vz~kU+Icg-H z`aWmhW5paysbSyQfRSTyGzBLYqnWL1Ecvqm`Yv?25!@1k5ipEp*LlpmTY_?^i|VFy z&-roR0DtJpB(Ow79HheVtafcj7%tlzolc*c7mG_6|M1OZ8UlnJl>uafuL-H(l~F&a zlFyhKoHBqt0CNBB!hf4&sxJe;et&HCEDEvLjp#{5Z^A(uoU#i)sOlK(O5GgG%NlJl z3AGSBPg*r$kn!GUZ|}S~@mltHvgT4cekk%9^wj6tE2X&R%}uYP zRN^B0sf*<;+tdoiHn6`#AUi@xY}q2>Lak!p*A5VhodL%!=u2=$gU2MZoEKn-BS>fO z&k&C4;ov4WI^t6eXT{s)Z%1&%*cg%$Bo2ZMd@4hZ`8?*(|p=hi`gG?EF*pNq^r8J6eN>cKN`xSfw~7oi--Pi9-cT+($Ly92q}DRW>J!FiWlH z-v&K1mAtY*S(%y*z*?IA;BfD!6XOGmRUSWo2Y84Y_5U0Pu2O+FISgsruBR6=siY;8 z>QTH?LoASRl2IjALGSed{Yy!c3i%rzx&smVk4kzKSNd+v>Wp;Sy_K~R`Ed&WUPykX z|KCJ!o9I@SqRE+2idLl{eTwfVMafrt7J61Cat?UOy|juqc3wIRmYsv!kr?iRPUOSC5X zdI@6_PUq`HN_?gaP9Q1KfSVHI8d%oLzeS6|(-R0sX%=CF(Qs;n$m}_$dRF-A_bjoB z2I?q`(aE6W8KYP_%ief(2g;PSWvi(F6=PZl-8)(ORW4dRV`X(!5kvb+=zX1H>({2T zm>Pq7a_4f0Kv-SRMhF0!A*ZuW_q;D==C!apN&~2@SJgauH>(Rnf~@au-HMEH*%vFX zUy-S`_7BZ)nge#7tP@VwEv&FdLofM3K`Co$FPgE%riB5&i&vc;$zHHW-V5bgf{z?ovBIOqxnJeZ4+0+46<@#EwtuR%Vt$y(W3?HCx!ouGyBSn3I;JqoYFVZ zaUyr&lD$}hXuufSy^i$i@d%Jrj$acrbs-zLa0mKTCC>rk=*8j+a)Ae}8wUDdUVW}f zmP6lRzX-i=M7f}FVGG{0GbT={s*v-{1@c9Yj)R7sR_3N6S$Ek7hJ0ExwTkW$P8vqb z(L7w?-^8i=6$O`~T?$N?0-tX+7pH!2pKjSE%p3$T=E}qw4c6%G4G&r)JYc#H4Qh!q zrbNO9`u_L8wQ5AzQbMfU1$f2*8y9P-SJ)XG_?-TLO*tpU;EAE$qE@_K_0Zg;H8p?K zaYs>=M1OD#E32DvYVQ;~^yR(8dOT;7neKlm-5kSH%e{dVPN>CAg80#P$kFE;^Fp0h z9`|8bedW}gYV`&;x&Py0{uIU`_ESq?7kUrUjYLtW&%6$){Mm;(6RT;~sMU9Z~_OWO}m!Z-!Cz97f@~m?LHIOz|r%yDwM03 z;DhbCwN_wW6PZQZ4xI=)oCSgzdw%jWbpZ#Z5uOL$M0AS8@bSzoyUpxS&y~%q0-DD0 z-*ocQJ8#MD2UuMACM2f7`Uhy^1rYx&Z3na@5qNwccTFOSz&ue}PM8j)MV>r=bhvv` z)*yCHHN-zU>;kj?@tFv*JLF?5mj^UQNA6ZO0@6Oq*2rW3yw}3t9G``Kfop$zkdK=T324ECA=}f_ z)6EB685lc%^CK;VO30QP&4Zd0{SNF4!-YoE;^bSl=R&rdsOzp3E!z#iA zIn~lD%W0Rjw*G-36O9N$D+@McrGIk6y+x5d!gWh0TaU5(yE&kFZ6CcqE)U9KZ1HEA z=sbLAiU_p$@E{gzeF&pF4O(M~)5rT{f~3ln@-!X|1}881pGGi$+#G9JNBVicPiEB9 z1_NM$gs@PDcMk9Tw8WoQBiRa@*YY3fT$}Sd3=ybH)xc{0*ODAfTE6oY%G1thE~o37kh9sLcp8ya7Y7&4*2EerQSomUd@#15H>q(G=!3Zs!a_pY>tqXmQVk#y z5E#Plf5O=U-8k4G#>PMVNPgreDOG%`X?udv*Gve)s287E?%@qp3H$X52!_Ml}=?B0tg5dgN_Ap3i$P$79{ zbBb~LVk%S?+*!@6Z3g{3f^ZuGbu@P=yP(vDI1m$sw$XsyUXFM~bI2`&)Vp8A3OADJXS8~VY+d6G@Q4Z%YXAs`Mnz-eGv@XSx~ zzQ5MlzS9G>&@`F>rEa+nR33ATK?gL`_$z2Nj9`Loz+4DEOIJz&Xpfzk_1x8jZPbTG zJgyXpL0{JPBLrg_1?uonXy$x2|8|%d?Qj08(9A&?Kt>{_4;k+~p7@zY&8EOW$Cl5= zd*XaY++;fsTqN}HRiftzUC1Z@-7Y(tmPJ@PY0!U>c*1dzD4RLnl7SP)os#}9CcM$G zqM4+zca5q`X?OhTL6I*x*_f5ZLTElbU$JbhUa@d+w z0`dTqG1TsX#+~;F+2zCTrzrblHg`+9Ap4a_Ph2GIAuZ<>N}n{UNN6S?pe-+=#>fyD zu%ZZMI~gjORw1y<7S~tNa_-|r|Il!>wjX#XwGlWU?i0z9npMBqC88oY9b>%``DLJV z7#aD3zHL3X_e-8}&^Wgv(Bt&H_! zG#_TQRu0jsqN#`&^x38Utq=;Y1o~s24fPrv^#Pkj1)qIj*jrf=rWlQh%NsjKb3 zEs5$-z@$BM(5Qc5rSPJX(v1$C`(e?Pj~qJ8991&ndwENu{$-Q8tS!%$?#3e0jS)pVG>!b}->yKlw0`zL78jR6+6ammjm|84kU4pZXb`*}?>_wWeyVG%RBQ{Y}^6w*> z$0|Dp4zkvhw4-H+I1b(x*;yB4@#s|TCVG$MKlS!&4K7c~?TFH7elJ$Wi;TUVNm44P z?1s%}q=Z3l(FpO3--n?S@}cMXAL46$_Vlofo5;2;0X{J3xWsyqa|CUhiz5VtjZV=2 z9fqaGSwMo;9f1wE_{JTVbi@Fof?|;MQm=7uvPbz4vkXJ;5}G$f0AfgDhYYah!Dvnm z?X*a0B6gE$gSA+auz2A3L*QeNG;-etn&XD+kpk`B;t23#sZnU(2E6;XfevhYM-14p zz|KQUr6c74>n`jtdWt?k32rck+vwvxNYd4mF)0xGq5Ih6|Htfp9(N}_O;bCJvRV2R zG6^Yoi+*DxV!n~J0D|S$+evLArJ0K*3K?suslNmln&~he`uy$etgwhYaCu%!Yys&3 zZ#x?$4bD}Yk}Y+j2O#8}+yCT1%2)l+U)E8DS`{CN-(emC? z&F7FV#H{vkgTxU?=!i+SCUN+Uw3ClkJ3LzCRqAsrQ_z;K1aK^`KoU(pN8ghg6YtFfXs)q zd;I0LF6|*KnjILs|FY0C{lX_tO01_VG6K5fyjr0oM$7zJ;d0WTSz%#-W1tY%2e*B4 zWtV45nE;nd;608no0@Oj=oZvJ_t9SE4eRZdd+@!Y1NHa?pg7D8m(R$>XfPULPg~yJFCkQP3Ws_TuYrzjc^j5`jPt*$!i8 z(9x9P(*pEkSq$J$2DJHb6Y-&kwGaU^U5=?B18E^(w zMqw*x{XKFQMY%IB1+G6xp;X^IS1mgGTM8J*PO9f>AhF~0-cPIQasXHlVLVs%Q<4*V z!wvSpYI+!wa=@!1_gLpdwrEpTR!%0{R8K?rqV3g7oy8e1^%+mpGlGA34*#_kb3M4W zG@yOQP|Muu`kP+BU`Kl}+(6?1l4&2^ZT;)AO%|(w(}db*Qj=8=N2J&2Ziqw-?tV~U z`7a^%0L(%Xhww*D?YXxG3WoQuDasMg>oT3Y<)h6V%@<-L(i^^CZQmTCF-Gb!{C4qN zF|QAk8K4j)ZTHmamI3*ucsQ47YLEVd%e%Yhvegq)8ZK3~ot^xR8nel2cYZ6*NFJa& zCIJhP!FEeUU{U&J#EaOB(N=UXoW$Oud@kn-VrId9^%jEnbr5kZ-{GIjE3o z;o}SD_EQY@A=Ibu@7$rLc8yoBzq&N&qe*jLAKfEZbG|ZN9dBXkr$8KGW%bI8C#_*x zx%?9?l{`bT`DQe$YIF%AJm{44jc zMMud{*8>w!v%T>IjehX}%jZ$QoCLdxv9AC>tuF18fAG?&sGM zh#t#c;)5h^WPv^Y8&`wcGNUJ?s*Q*F_*vp-fJ z{lGkP?Lg`sLN=Rt+^~>BvBy8=w|Q`^oXH%RcV>9R#m{SxJk7~L2O*qKyjlHog#HV+ z{MluHvHkb+pO;GQ70K7V0|5Ny_FxcfSygFZ@dIHr=(E*-Ur$9^u(CmsihuLH^w|rI z#@p_hUs$O!9})dbe{es1&N|#_IM&QvBC-EL?|u@kIKf+K)Hms4#>v9wY3?S^#=BJ! zafhuxTJR|>+eTVBvT!4e4cC^9L;RD|&OC2yByX4zi_R9_k?u}q`y3f*0v2{TZY^3^ zdly{()9%JUK6LpvJL4}>D)%S?cMCigqc!x z^8&pcZ%5&PRqbKBm&h9^^+4MhmkcYDX6TUumjYNkc9961>y<2WzW}nIN)C+1UN69xF@ttF9-DIi2$-X*U zF8kI~^NZz!fx`Ftrow zMcGRfmd_JoXqIxyL+x)kZP>a+ued&+tE+g#L!sH1YMZmtgme=M{Dd9PIs3c}8;nP2 z-6fA-CyRDlvSHx+A?%_K^DGSu?`{iwIg(F$ABiLI>@A&l7U`PSg2(Hn0*11i`5WJJAaH@HlLu~bq=znil^k|U!YAa?v02s z4g@#R7;4^e;Fv^yYoEss4;vi)VV}C00_ymPGC_+d^87}z@u{n1S6kz>V3ap-5hrdS z7CKPEu)kI*PEYJZg>B>684`5-Y``o9V3a^^E91jI!jIz7e3H+9#-NX1rKL=OE3<^4 zTCOk8kF}=+R?Ei4i4>9rBaPDTi}?7&gn{-$IPfa>O9AH4^Q`J!^phr!`58GC+3iL$ zpDvEuO96af_xu25x>E+U${+?nC03A-2GSu4Aj386mzP?*Hf?T_?Ga#1)-xM6t@}o{ zWP=MCoo~)%ZnHmozgBwBDPz<*>CqFS3!HR>ol77XP=*cL5bgC#SfDw;xpJycL!dzqn(+e#M;2!lB*}LeAUu z@n!$-Tn6zhTQYq|%z&cnSeBjEapT+fP9tZ1V@kimpR^lg`Ys)|*D_6S*T=kXk_a{2 zy7`2+rS*FL`2ABy7oy7FobbGoDyn{e{jtl&#nT+29)pG*rL$Wha$3ALPYGWOxr?pH zMeo!N&-lJINQydX+#R6j^w=jBEm(f+g!C>qeBFfpMT%_xhbJQcj#WHY0h4s7N2WsO zto&l*{r!hb1>a(W;G^(Fen`g%QswK5O2g;Ttl1)0Ym7YFV3lX2hhfw!z~2vOM9zK+ z(`$MsJL9s$ytUH=YKoiA3GgJ|uG_T>8l>a^BhT!Q!hRp;f*}1J;=We0jkw#0Q*tFtt)3-lgB17-Km)kY0k}HldjU`Xhe{mz;W69(y4>~M+M+PBiCe=rd29MLk5F7_` zuL(ouFbM56@t_=|_k>aC1_t~hZrg=TXWZR&%+f^yO3S>v*7UbNI0rB^VGv0X{K_D+ z!Z;ka`xwMjkh(US>hLqL6oZu1jUDBDG_GTL@#-I?*3@67&n|yHIw|W#NxJJMY`?9o zVY3Chhzqo?#BlKbW4bl-VFl^)EepPsnHI`SaHMt%2n06vOo^ zunldgH^$_jH*5Z=`i-CXF&}wB{KrqFC=tN*nd-oxD^XGQ|7iN|pr+pEYXy`hAYBAR z1x32_p4brSNEHwwO$0=`fDj@G(iD(h4ZZg&HPWR+=rwc*MM5>C{_f{H@B2q)GLxA* zlX>p5d-j~OyQ$XsBi|4r+nstnhKUt1iFF_mXaV+F)vVg1qYDH(T;jnaCvKSZFYJrN z*D_&~Hz($;a(DOw68LZIQcy_FH=f>MxL5PZMA{$((@)TV;{|6!O3&Hh z0UI15VT4=+);#I_>HDrmXyYElvTWaB$I#{)DWM7pr)z>vW8&XVwVOnC@k*ecRDJ`v z=^k*1Antvg#HHP*Bw2h!u{_aGXX{$S;cNJgE2WNmf~zzTBbw5S;AarS2Pbj+2sq9Rwj0r?dgkI9_{HbV-~HAtQxeJ{a81x< z9)?>(!nR3parcOCh`QGdCXj7@f+v*f z>vZ~DfE9`UIH`aPd3oQQTv=a}~n?Tsqp*4|w*sMp!?o zqh^9j=JCxmh{a1QP=tJPx@fm>iqG}W)RAHEtWiw)XO-8+!t&G+exW+-@!rl2N+~r_ zt~k)g{?DI{s<9UAX}SecKVpiD9lIJmI{uCGGk>!Sg_x~4MPnSIt_9TBGoGgeNic@F zxVW+1HFy$Q`QdHNoX{+mnlG?(j!%Ft;fkJ~Ji!lmpZc?=M4eho|) zVqRYCGDr7@tA*$If~o3+N5f$VSb?fYXQt1{UD%T(O(dE?^MnGnVyC}m%(1h8aF5~{ z+9yr%sSTAqE)%fX!UOtn*h4s)33RPI6nEW{z}N-8=Jx852($#EcW_+SPn7G*i*W>fl6vuY*!xc*xRJS~Jk@d$sEKWfnYG@d-`Fp*=3 zhc~!>Dn4jAMQJ$`pwlNm$-5AO57RP?yu6GSgiT6Iw5(~WE>kbU;KY8o2Pr$ODtYfi z>jj#RytobP-R?HO;W$sQ`-U)m?Kf4NTZ&=AMPb8=$uctoWM$BI#mHX`l+q=^ zt&`iMa7-&_>;3{6{pBCZ=M#CTFA#ZB(YiHXR>6lq*-o30e)q6l!QY!@ugEa0)T$g- z24m25kW6Wf*bw)jV6~mT8hb@`uJ$W>J-Mg+2~R~UFKMgE` zp-0BYxq|l(mIJp=M@GAR)H6=RKIi&dfl0f> z(ikV3t>+f}#~**###H~yL$C;`#>rARuSr=N@Cd%TfQh}Y@QI=@fMZcWu=$z5Gkw9F z5_`=JdRV;R-2Ha+H~XbN$A;&>K7+IDVg`Pjy~l9Fo~lqdUvw0U6d$3YIR4IEgWO^n zXrv2qyzEH|g@+g~-wZteQlb{U3X!QB4`&;F)u2?daE2GHtumbG76(wpX}8?fYsABK@+{#i8S0 zCm8R(pN&x!y@+5)OFe<$X))JeV?_qtJ-fq{bTW1D+j7ur-`njn(0ur^{0}1c*=Zx( zG{pW)VB^Cns~gea{XVvROD|}N@`v5ZR3SU#%n2Hr_Tl0wizIxw;zBPBjmVGJ6eG%? ze_{ErRFNP#VD&O1h(LwGtNoS)ANK^lj7XU)&1lZ_V`cXX37XtLop}0w3YSMrdU3(R z?$Xo`!Mzo5SyKeWsiaDLoq8QylFV~LPT`UmB=|vz^`{VC(^NK&^OT=p#0qUpZse^` zOJ})0;{1i%c_PrE->DMq=wExHc&6tY4_$j)5)7*-VTgCy{aIjVy_))5+y&*~Vm z{_OWPk*?AFbH^x)Qtx~HYInarzuKDjXrAQNBc93_S;=}5f{Q=>KBW9jys+g`O zj&yE1>12yHzp|)~T8|2C?y_a>zdUtU^oC-8 zOyPR)KT29Rb3V+!1lE8tLRusr#9!pe2fdqP&kNi@3fZUpKMg5V7)OWiEeXYFT!$ZW zvcr;)d8R5h8tNr4szwkmso_)`Ss#0bciBk^;1+ajUW38X?`?m?+y_%WbDFWmCPV4s zUPSR;&FcUW3&#nWsSX))h=Z6gi8$0Fl{Zvh!4?-jrv z#D`q=jXu1^3FdlI{=T^Yj11Gp{22Yl^YabK5{Ukp0j7P(Fh~nV!^-|y-;473$K47WyKltX z3g@=QHsDo#bDg7fP_^CxT(pa%W=LKeO~+oE|88<_cFq*m{J7*y!pyN~ExsSs^EJyj z$BUF_!7_SbiWetxLhJan?a&B?@A+hLq7N4=`DkXm>#@KIwL0_HM0>)Nyy*I&6zr|= zruLNh+TE;CAXiEonr_37;DZD%)+U-6M1@nWom=Ylr+@b#GYR7N-gW)x?Rm!d?X{uL z@Z9vFfK)2&YiEq-4@F_cmuW$+g|9zSH(_+Z9j#s<_gYs3wYfUt=?rxu7+^F}lbXJQma<_hwcb|R8CYD=T zY)Z+W;!J?^&I7ml0Evr}OklxXdXxuccjNve#SM?%ruZ7Jz-K(ze&>Ff;+0Eq*T}7_up5uo zeACDgOuHoNZ8=_uiu}NvW%pOm06r2!WAyPLR+$@7u#VS+8NKd>I7paJRE+|#XlvA6 z%7uiGM0ljJ9j@1DIOO(VU;!n4YA4`La(#t8X>kU!fHM_1IMMUB{YM54ystaE-y150 zE1!n2;su^V&ch~POEGlGuiBnh9ywp!YMmiHs@P{I^#8I1rx7m;@9nNLV=j(n15eF9 zhn#L4gli?gD`2=UObc}r1bNK^t=aq&5u092yx7QW1=4T1C_}=(X?mHYZf+R zlQ|Yi_jEic$sJ>HWb8iE&>OU}ja0iQKZSqn!-$Cx{K2POTS<|I{;n0Y)>84c<-%;Y zUl=9luJ@q3OxlMjrI&)C!sO-I3SLt2pHv=)#-p9m`J5DGUcjOyQ-;|DLa7d)miD9mBl-@$n@mv2RPe1(kv|D$A>;s zZJHmCemot}OlcLq0~2|O(XcvLz*vN)5w#Cc{(g~lm)LlrWp6P2p=VMJtNtF}Kw9MT z=+I|7F}_;l9+QUoQ6wyx)bb56h_zTZ`7SYt{qH*=HG(qig!=yLLBd=F<~^xt2yO;_ z!^uZDt1MNC`bg~TNb9<%Ar|@ir$SnNPwx27mS-I)CutSK$h+c@g%1&#eA98A)NE8m z>!x?Cm{sOr4+T(}n2;~8tS)BPIbKja z_qmQaG!DAL`ihorYAIGWfPT<=*fQg2z~y1n;CTAl0`7O3r+!r_sIJ&PJ@M!#%%kL5 z&yk8DtA|>>*3s4b9;>yununtNd#@*EIw&eJsU9U)9Y1diEK2l2 zBl^Bc(A0Vd!11GV*qSM@#JYs7P;~Jq3}A>jWqSYF6U4|WTi}vfNm$CPUco!(lTnwG zE+9hWbDAeQj_w3-k`V*X`-SAIShoBXX zE6d{aLZ6?cx^VF3OW8;9U;J4bSdV+>dK$~%taEO|lQ8tT@x}YtfX`1V^Y2wv4S93a z3+P!tk|Eqgcg3g7zZR}~nBcc6U9V|rg1!6px<1={r#kFUG)q@ui5NM-Nn?L3w5Or;FV&beaL@szf z;dyIspr}YOJ)Z!wY+rr)Ybg0G{_BIP2*&f!^<<*&zsWJd1A2g|Ht`E}wHNy$aNnYF zv_fyZp`^e4EtwLgg--%S8Zh+AvED<9H!QtFB|vuWu(o{jwsCy!6sJWt9?B;>mE6p` zn)ufx8QHFIZS`&v;uMw5Bsl(n&gB~<7X|>z+~?dVd7>v8kUa*m3(zsA07x%H1>g%u zyauuzQ-OEpP>OU;o9ZcDz_qtW|N1$E4>}N*>8&xf?b=@y$io$5Io|ZC)RoqVDn^9t(^pNT(x^fNDSd(P+lO6Na18039&T$yFMWBhx0 zdy+xI|fwvKA25&cp@%WAyrj&)!>s=gWp3_w<9E1 zb-ZeSb?*SYI?&7ZDi;5og>Un*2LgUqMB%-jpVyRjANvIyQ3yzA`if@c+i8E0Cb-&VouOxNSrRKz$V_qq=Tv1(W`~X5Z2xHc z-3C1Uog6%T!J5|V^m^%W;fu3X?x(E887!;pX?8||9;R_&n}d)@GK3xv59fg%!(Gk~ zO$VDnDOJV_(d4gC-81-}ExF`l^q-=}8orWJhCHaBp)J`2=Q(~VAt$&~dkQ&=lKeeI=5=ZagVTxxj2M*ss}cn;U=rtnov166_I3HgE`{Uxk|=TK(L zbdYet^oCek6kV!4J82mT?O2tBMJ-sM_#QwMmiXS>*06F2F#i;7o@ejc@zJ77&~C+Z z;N_J=i?@oV)52aU`^78P{VRW?CTO^(6=|QwA?LOf3VBqtgZRjH{9msH1cjcQml@R_ zyG;i0_j+0C{`chU9KyA=O_7C`Xmw_aX>J=!bQ?2I(0 zn5!^x8avy*n7l&lG3upu9A@CK>QV=wG{EPV^i=<>#9Xah6_I(D@Z(7bA0cWw0pa)s zLXe<>pZ363;!CkUgtlzGph~p#CIqJqq=3fVr7_2M8eXE$uKLNYbxD&FnRHGRExY+0yyKk z0vmAK$F2ZWJR)24CfgIfgBbl44kZx-51Nk2IB$}~x@$VPY_w9`E`W#Rf(8&lQ-*Zg z`&95cY|V6fMxAfy%r%K(UpOxYDM$QFS!-_9OS&W}j;#T&Oh~`NZ$@w@e$h$_TM^JG zm)QAR=efW)R@vKlwaNY__SxIU4kGQ~jmimD{v57Hp>ZKdFURh~p}(2dh3h+~BjXos zOSzuP+rH-L)sbb#M+SAJl>-WIJm)w+taO>UN)$L^;hsr;37I(V^Sn~snSu!C{G9ZJ z#TfwmSZ@Dexuy|(80j9DpI=?Z-fm!WanEm?DKK&3M=8x`9ZT$WF+z#Qy8YF|*v-sb zOpmVklTT~$*ixO-#ww%YAkIIM*!OXE1TZ4L%J&O3fnr(#_3$-rs~vx2ha7# zKKLiw8sgaqPuQcUHJ^M&e)+mn+fd(-7&Uv7^cLR`Ec!N|%m&+e%3|okV~9`Pg-{h> zyFfGhVF^5!A@zq399H02Fgps{`^g@?q{nh2xpNLd|9115>mMst!!->n@4Ya8dL&AoP(lV^>|d5`F#!)G_EgmDLTm$spy4YoT+MNZ1A$-3jyDTnYWGH)Z8CsJ@?Hdo zWf@gDHS2{?3CVsvdX{%CRS!oKLPSgLwq&xdCy`nV?mOs;HG^KroHnt|AdJGlIzTN? z^aSOhB*i7Z7bVrCaJu<$?#mmZ%EiScbC<1yH{L$#^xeZ!nb~|R=h#XNc?kBpQq{AG zF6c){b^nmw72LffBeX-})i& zD;n_<_y~XI7aq+mOvgH?%t?`wD#gqSNPfeCH!6~je0GC!rb=KsXXCPB>y@nE3P!gG zZbLAnsx-CYTN03RR5^&AHGxM?ATVIhkb>QsxMOL^^=!HjNDMaV;O)H|=q@g`=eB4; z_cYm|Zoao?ubx&D;oLe{mv^eA%De%*A^y)88L?{}#U}*K`*4Pa)Q66P>KUJ1&4fyj zmxMt#dX;zJks;iD%OJuhEf}&^wq{v{{69{ zUK$f_c6Ps!hhg>4$G9Jgi=nh+S?CLz((BTu$dOY;Mp!)A2)+^A?_@xK?2diPhh&2} zdO;%>>Y(g0!#|qS8luawLiG`kq%cbz>pR#)pA~x2J4_Fq)hZh-_5F(L6vAU_Z-uY* z>L(tMw_c<%{#?@u{>)wRlzeG{^OJ{GP|P%F!eF{mAtOug2vR$0P+%1XvSA!AZBSiu z2LJ=taOUpoa6r@3V+St5U%uytd5L)AMnGEls8%;_4Ty0rmLJiG)7W3rm|Bsv2B5&4?wv#mb{Mb%T z)^RhF2y6Ska)?dvfjzb{pKOIC08d7@FV6i~m9A`mqiwIpZq?;{bu0%ZKsYXXMs?2KXd6?A8mI+am8fyV9jKDT~g_)^kteymHtBzwdNg6I%ze zWlfQ9wkR4KgE$Pz6>=@wc9Z-&L#3+S6w!;?%6$&3*_36Mr~_Ax+CF9d3ido8vy*PZ z`0)p~^rfCgzri7aRq<3Fb1amW_pUU-cjd~8--z|Rcy;fy{@HsKod=`BMugXLFBP^G zbjip|F%_vl%r%Qx9{} z{K*c@n=e*TGmXf^!vAL`bsILaG{@-8?^y3jZqjfN@pNy}1rIl_FOO>{7h?-0A3aJze0A)43eY%At zuR-DZx)iCm_hIu8zb$xa*5tq1uSmfg`@F>Mu8AL>AP{@geHi9j+0Bt+Fu2l(mC{yH z{4;~0FXS`|+JFzEga;#^!V#&B|Gqeq@BZ3dsujp)H(NDr6UfvTFFd!s=fv1>iZamA zaF!~w1oQxFusvCDR&Thr;C$};hZi?njJEBG&Ze8cr`;|esaib@xocjF@Xb6bPx+7_ z$~$O)ZrbnZ|UC_V<6lf4u@Z=VrYirORBG;d-dHr3Z<9L4~a zRy?D1em4lN+9R<3a)RW%%#Di^3X3d9qeOG}_)9gpp?BG4#HJDu<-7sR+gN8@RAUFM zBnI?=hG^k1-+Pn%P;xvCG<;Q=9NG9=eGW#C5*|MAWI&v1*qn_<> z9AVPT>yRQ0<5js2p=821DlJdWKOKYf|)xFQ7;_iF}CTx8?a2GIndtY*TS zZH+pgbfqB6U@kZZf17Cubb@`r*F0N~rZkwxQOZ6d;8q27&d3GL@Awkow(qq8l6N0# zuKg~^EdC}DM%`Ln?EYuQG#Y!4`se)yILIku;ZkdQstQkry`qyYf^q_*V;hV8o3`Q1 zTyG>mfBQF8mrb*Ls_pC#4NcVRL`0*FmJrPp>hls?s zOJ6~cE(1x#H2XQ{sC=%bet~!N?2|}&W zh$#fp%#@UktElwEo8erg(-pKbhq}otvSH+6JAmyv|DwN1{-O58Xn)5ft0vwno>D#3_A{Am(Y?{qVes(yfB==BOR)kp7XHik zVKXhM4|R@&-mh5D%)MTFX~xPKc*KUGI5k^`U{&2h;U3+UPpFoEHyVFiag&9}C?%}d zNsbp%d*rh@;(aX1y5FF3|8mY_$Xjp=Kwo!bxK;lzNV?CX#)4LYvME+QqPZoaS6%I4Q zN!(}7od*TF{6|BE5yub!D487vf@UQU&*p%tq(%;GmvPt4Gw_#U;A*xyd0@71aQPGr z=vPW$)MOv~+ky$di5H15TN2_Fl5fq$4!t!Fa)wrWlVVfV%9-YIS7a#Y5&JMha``#z z2xv7s6=fHRGGHso7^tZo$N9M^OwOj_Lc3+$W@WrvHzJHuTYEZJ7&|*$pzHgDy#J- zN5y(Kkk>V@-Y1kUuDuOcBzj~$e4#v3s3{Xe_S_3fjD zQCCom^Sz+l0vaVsg#Y}8qgh|&7#LyWlbSiSL=87SWgh9}U^`S2>r;M#Oxp)|kDqdg z)KusVP95d{aQbYB4T@8Im5JE#{_ETaZ;)`T=R0}j9zFRa!3iZBz;Z1hTf|j!gH2F& zUS8#im}9flL(#Q%%&I8{0&SLAXrM!_Ce&8i9D= z#*6mASW%ey+p(1vVhrH?o1CO>vpW?EcFdaGxY*<_gEqw{P=!d?X)4g;mkE@BJ@ETp z_4f_j*X>G1dC0FwyQ9IaG{XlDk#Kgr9Kq_^;m^+cU3>QO2wX_rI5##IN{^_~K0{kP zWFBpk{wc)&7VX;7{d3i|LkH$6z3x;F4I|BEbAEjm3U-FLe=Gf-CksWk3$Didr4vgf zsNSACUQFlj!>OEcdEPS%$wrhY!o4e%>oh%Ye1M}7C(iIr&7P%^@5sD0+5({cvzx?0 zR$SRSa6np@RR|x+hPGovh_fSSCDp4PADNhr^Ak^6lpTDmFu(CMBB+;B){@$PObFT^_8+XQf59P*jOd@& zzxUK1k#S;x4S*wGU@?E}#jq_c{$I#HVun5#x+)BE&9~h zyJ#1q>xQTINhYN9b59;vbuR3cS;8H&ZMM{GvVP1bo-nf-~-N7_v~>-HWNo^b)rq%k8f)INFdybL|Nio_j*O} zLg_%xqm8%9Jz2d$@*E~#BZCRsQlwiBV2i%Kn^ejF#n}|Tqd`1S_tcf*c_eJ%_YT2#iQIx2!X=yyhW8+W3TG-%4)E6( zk(Wuo246)f5cu{F_A`~e>-@bU2bVIxWO8PL!(dhmRKLk%Q3XV2*@eZ|e9XycU!_|d z2^k4J4#%P|)PxoWE1qk5bTlcQ`^4zh-cWvHYv%w**(th(-3imrNKrT}t7%v@rAxl* zqHG(Da{c>GM?vV*3edHVN~6VZps@ik+g@ON5j&fn5rI#P^vaRm_%4->arCO*fWUGN zxgYm?jJdgnFUD2-bUV+yZa#TX2WaE|K_}nCDM&3SxD)SB*RTU0cQ5F?N!B6Mz+?R; zaj}o%*2Aw%PxTW#^#-mSfw{mU*8Y`eX$xc%y!N4smrF4Nsk9`ch#Y#3ibmPN>$ROx z`Q(PJ6PG`HhT1fw%V%tE%k7(BdHf6;)w)?N`Q@e0=>SIrC@s${&Lki5PXqZ5kmBmo zr>g)ThH0TR`!qq1RX8${3|zg{_`(YR)5`U&FishmRWAxDV{7@5jxh}C6v>$dq{q7n z=IE{FTnn7*a6&V&PF&?Az+%hvkHUZa%~A$T_iU~?&>Bj-e1=jv2+9n}ZTpf*`B7>1 z2Bvm7v#r^pb1AZzHPGn9fzRSfLVd~i>p>P_#U_mkWu0sXMSP4wUp%!bL&q2)}mAjyP z{UZI`oi}nfSqVRTLkN{$Xdk0%b5v8F@j3taLdCPSo8oOn$1)Y4{ac&w*l66?CVwM9 zhi|;}sV>%{h26|ulg}(~peOVjf8L{OL8*FlE-y=Ecp)G~ru~lqf3A&U(h_tfU;TLJ z`dp`ww^cHSJ-54(GlXKuKEa*mpk@zphkyFh%UVqyQvc)!gWJA3uodCQZ6?oYsZr*6 z&pJhR6#{BfM6ek$+7*3%!Iv~TAArZh{*+_K$BR0(>*t%g2`G^TX7d4BazbmTM7ku` z1^;g3)Rmpu2)#eRq5ZKqd%VihBva|xElw1X;3|bqlsGQbF-54q7%Z|BBR0;#icKq@ ziX`x^n}*!G-%v62PkiD4u^~o0Vwwu3D5+(Tj7Fjo>K2i(ar<--RWivQmG7V-Ur$z& zKUzdE)EBM~UeZV%23u${URM}Wa5S`--}|cfB(CfNcFb1e=dY>PFc%~AMb9iJRe0{$ z0Q>eJr6)h!SV%1QWBnw z7>RCB&jHpOX>#ld6}S!+U}+Otqap~n9w=ibv|pQLP9q+uhPQ(>33lHPLJa2gO0kky z4g*~QK|xHV$)#R%fS-JYbCm^RFi@HX81Qr=VJIeY0jjHQ$2#d~=Eo*8pytUEYEQnt z3lBFWHl;)#-Up%xP5n1Sn&y_PEST_UOK|BFh}S^|T7O z5{hr3vtqtfc$co~xB3gq3ahA#5U>z+d))N4ug0Uvgp$ES63d`vm#FKaK7Z=W6ZTSm%c}SR2me8Y7P5 z>}#Pjr-Jt5B7|VxxxNVYYXC0Ls%CL2fzd8lwWKBz1>Buy#u1tePed}5^wuKGre!Oy zJd|YWlw0swskCM+F(feW0BJjHl4z#`--V=;fDBG2*Ql3gaRLUknX^%Q8^GDAEQELT z4C^OcPw!-Z``#(ql?zs%!!tnw?0qu?y0!Sm1(*($MKxtAN4BczBZ zdte24hCm|B&Eoudx&eR%s5)suqUC)uoEXlzj!#ln-rjQ-4e&kh^kizZB9KPl_{`>E zuLbVhGWGdhpA7qpvA-Iuo`1xt(;xnXqg1_rDl514z?K2pbHYC{2;=2#Ht6ZH5EW*cs&7 zpcdH^qujYb_s^pNRb7~Y&IKg|SH}~|Si6!7TXQ&vXo&Oh4ZiDEQoVjPa4e0;(+3>G zjk&rY@ha%B!Smxk_X>OG;9wztpu+uESxnlUR;F3) zL+h2~lb;KBN1`@t(#iYo3kN4uWW!rO-dCUgmJvk_kSuDqkYgI;l6}FIbVBF+%>Vn# zNk@(J9sb@Kt-5Ctm%snSj%w-?oheoy8mnJ4Zf!o_Z6cSQXRf6;@?UR1g>ARkuX%k+UH0oX`mfq}bR*109WN)De0JDPcHK232_^;l)**a+onktss0k;|l>{v9 zXx|%K>pF@H)mvxSg`acdO2%R>@5kqxa}PJ~wiq_JHjHa%Q|a9;PP~zPgBmUDEzS6Z zu*B+rs8EuyVsyi;blZfjUCyq&s5Dg8qQkjRUnt?|3T)Quo=Cv;wCdFP;OD-#+cJ&? zFDIvC#Vk@jhLkW?8Xrf(G}qL)v{hCiJ&B+mM1oMl6Wu%OJt-7CIG!+xi*B6@`OU8E zM}_$bAX-!>cyWsQb?S_>@av$jMuWPKOHcm~=@`#R$*Y=vCGxG7HNC~LzKO|NtDdN1@41tkJld%ER^#q) zj(em6Qtt;~WGK1BfR1bjy$am*@iS(;sjPJ|MlxMSLnQ;+`dYs@6>3tW`4siQ z8c_xU=%~D%=$zDvNw6qUz5uz!G_@qx$Jr@ai2N~=PmU&qvUT||03&6J^z8$b>g)+i zQsCFUD3Zp|`Gtmb8Uvo|10cTw1=f$=!5zQ#yyo1I_h>{{pi_=T2?H6t)wqdbXd{GytBo##*4ZFUn?eVc<<^Au|TU_Ml zd_y_O-ya2f2;1_Z3L-+AdFh` zXBUVeh5RZ!)*ITemTO$dIhZlaBuvM>AsZkb1MHd`Gyl^8L(L3J7=zW|WEF7A(2WhVU_xPRCvdt!V+Gg1+UN@zlwbcIKv+H$?BYS@p44Sgz62@$X=1^bE z?Y+Mb->0>rBDA%sn$jmcy#)(zjvQ*vtS(uPPv#H(WFY^oPOGRo?%Mb3BR7|&nRP4= zc@Zx{9DY-VYJd6_M|kXSs}5IxZ-&hnut_{Iu*taRI%kzOaiO zCQ8aukRgHM)cZlGDixWESV-uQAO)2~4$mi_-z`xS7&i&P*2Z6sLH4=cQQKeN7kp8g z+=(iI9YtOC;yAknoVQ*2eXDFwj(44YGh3R5hG~nB;=To379%}{zZp5L411|}P&hEY zdi?6&G_SrXL48tC!MM7YKS892*^Oec;1V~fUZ)5_nc!~f@BDkv-i3@zA}GT1_= zC_SP6OagKAX~R?F;ykd|4mSz{_jp8B`nO`BOx!Z6X?v3I_6Q%bd^5EwW_9HWFGi3O z{B2sthRPa!;#ZBVQq3mKYh#~l7TT(#tb35PaYCf>V;TECOTXS?P$>0N&_+U$L#Ubda$=-q(rNm5`O zx7&;PFZ_HmajloWX1~y%>t+hOiFw zP&97;@2wA;;2sI$+yMMQoFT9(8(kYk2aIcecX%KcyGsbEz8j`DO(u z--5ruTs&@7(r>48NKpAey1x|y##)>e{$rV-p@M*B#e*CnQsxsCdPlOuH$jf`(uJq+ z#M}DiBUd917ZP9;lj<8yq?diz9BOh+|D^L0C=A4L>E8%!$M9Tn6Wj$qN!Aj$S}^}W z<-`E1*@^Y59)bg|Tt~1W9)fd4OV^W&L9+lwj&db`s|51@KU*RmWlP>H)V(o%I-{Nb z@uT>dQ*nh?W-h0XjbHGvzWak!a5=4s3^*mJC{u_k$3%eS!uc|pB?%s=|7S=H<~`qMy~QU%od|@zgAA$M82g2JQQo=D8wbud2LoJo zXiAZ@?1x8fGrdZhv<7#lTnF z8!i*uPnTe>ExXzm@@ZLUW+CY6?V+UnlrE@Q{05As77cG|u)_Lk< zlh_Gsu6*`8*K{+AR0e4&rFtrx*~mA8E@fZ}e^EGNIh>nrg+g$jr7>FZfPDLOxO)}$ z%X8&QQ~75~f%v-;T`)uV8W2}V-|#yUFw3OaOav~BcXihgJJsHU($Jp(PD9}l&@FmG z?fjIy+`b1KMc>xu>g_FpW~A_-z%&M_PU3|PATZ2aq}jsr8~A{=O_yNvuP#%bZpiuK ziXI=5BB9l!pfh1h5_d&*5L{%Y+hameObr`y?aIA(N0XsPHrwpkukfu=t3^2xE}Q0fy4fQU)uB>Dn#5y~023YaR?{($sPeM?xks!9 z0QG+fa>1vZx@IUBS~gcWpaQ=(Bh7=mWsB3`0iff;P zZOWdtLM_r*#C&^D!Sv7(1STRV;!G_N@4ApCN?3z0_V|6~YADbcopP`Vj z`Vy@+vdyIX!ji6RzJnJPF^3*>6wSdJuGOGcOMc~fB(-Kbxr z1sOaZ+%;#SvXcuj*-oG{bBuTUh_X)2vNemEGib zHAoTAN0pFOO8-`~vgPYrPtkAIJd)zd)1{*ovahuXMF#rbmMnRH%b71t?UMX+u#!R# zRZqq*1O4B6mAUI`9Ma}ru~Hh@d5og6m-*Bi2BX1E{m&>)%vKYou@+%L5^OHq4^W*0 zw=4-~OBCMF>%v4ZK`4Kn6L5f>6xyXLM~Zm;@*#2K5=hbkzCMJkZkZTg%cIdtV`pgG z0-JaYu}~>N1?xjx%?DciJ%%sTfrV0zRkA5NwMe+vv!Q*#V>812c6){D`_tzPf%zv1 z6v)?x_+k$StY+(S`-4wbrQsTYMA@aRK@FpRtEFk5qleu?usJOSBHiw9$MC%~?_!F@ zzF`|xg!OHwu)Uu?Dt|X4c;01_jIBLS6mFD7r^UiW8Bl5SH)SILBI@NCr1zycgJp@a zBNNvG^X?z^!mr(|^LaRV^&B!wjwhW>TB1=%`mX@N?=E5E-Zx&IBg~&%S9^nDwOrOC+q%(hmMt0t z#NXKb&FJ;#lc;)kXR?(<`@co+fSPnvyTZDj8I^P9#mt4YnAs|Kya;u{)&qi zx>Km2tnAtpQ@H3@Iz2p%@e~j%Xw2*4I7loB?o6Wl7l;gzqqwMZI*1>+PM}kQ#7}hTSFBRn1snD{ly>k^W#B0rs^zKXZ{*m>#(8OdR#APW6R9n6ik(u5gmO_|Fwm#U64w zV=oq&mNl&i@<%_e6Mwucqa`B|7)MkGTm~@ha}kz~<6|TU_9p8WX`pHjoSk9Oyw3_J zY**tsMmC7`BED=v50KhJtq%No0OOeq{?GN)dWOo4Ds$NQn9t&9m}JavzJVU_`NSz7 zj%NmF{we@{0dL8=0JnFimYqzo;Nwh2t|XJR@as3co^V={+s;a)?ct|?Z!BIpV5*}0 zt3S%gT=mZrk8K5XOg_u4m^LLg6TUG*$A_ws4cT{xcenF+Rf8hDaPxzK-aidhXt4=hqwE%;6UQzDW2EzSs6;9e<%@%=($2Yv#2S9C0{+yi9>AWYwZU zFDCcvGZUT^Ry^Fucq@riRCA+w68H40_R^=`E1?3+9pZv76$s@NwXB-7m#>`A31=@J zmA5lKBgYNA%Iw0$MLC&?0kCqfrtdX1&5;46Sz7w%Uc~2>^oA=zZ-IrP9tAPNv6H-W zYha+);<>5ccW}+TlHZ+erFKRLsUS9QDj9~E>##tBryP*&tMNllAGl)|{Aj~hg8{hJ z`p2Ue9_55Ws+X{mG}4?81g>fZ-@X#|F}o)ckv;uoV?IWEKTAg9(*uM(Q$T_4tl+BM z!{}EL<@3o}QV-Ja_kqt+R~M}1j+mppWR9*u9Aix+UL4YBqxmOhGc!YHdtr5z7bU7) zpa2VUT!ZnMY$1o#X(Q64|6B~uiE0^p@WlX%D8f~m9Ln@`?dBQ3>pgZochWEvjCm#x znD}D_mBfki|8X;?|3}kVhBe`SZ(K@BK^m!nC`e1En@UPZODUsKKn4;bGNw|}N=oia zgLF%6G?JsFb#x<)9Aoj{@5TSsUTkl6UC(u%^W5jYKS$=OUwanTi1A-{9suXdv=DjG zSi}HF52TVkF$Pt?q7&ot zG_97&#IJ6FTsG%og%*qJW_hn18?dkA|d@cJfTbudII?N)LGhw&nL> z9nebJ43YEOaaXENN)hdn-+^G?|};iaCZBTOvv*$ zG8O2esT|d-Rc=sq$o#WmMsjk&w)KR~D9Jm@4}@Cd!#|-eUU$M}D}QH2U(oJH7AFt} zdY2{}FMR3l2CIE-%Q>eL5d6p`6!CWsi{G$d;+LTpkI;0#_xb6Wc2|VVmG@EyXJ^u{ zD&FOv*Fr$*skhNLZ=p9h=COc(H`6?4#k7_aj%1cf#X?#WFHF88LM#|kVV}N7TRi_; zBp@HYY&XUZ&MiyfeG~`#7j7HPYpdb+< zpPXQ}0U7nD%eLuS-V(>IKw`$wIQ{#6{3e|;&?S8c8A%i4Qj4fEKBB)HxXniNgEAmb z(v8>9r^X-N4;ZZ2Wzmzmt>BDPWY7A|r4Ua2?Z)%Ic73=a-iU~^zRyM4p1Q1FNdHv@ zDv3ub@m;k^zVj14P(MwKEP|1Pt)lr}bIc=?>!ZL~%)!oY#9gm*m?Yw025WpK zBn+!%{ycjCD`qw;x5zdMq`8l1mRVWEM01~m`jiO`umEC{X|wUNGuPfh7B!5HFuPC$ zl_g^DOZ*CT#50Sw0!Eqmrx(?>ke6x14;241-^r7Xgm#fQ>6b9Wmu-D9ey;|u_qVK6 zi4PV{!P;O`AQ!xCKl&g1%nC~y*v0Otq%f=eKy1K0e+;U{gEWU)n^DFM!BKn1q(7nT zqyLfpCmFx7-lW|Vco}dC(?M3$oKGcg13F%eLeAuC^#4QSAATzU`jPu;Iz69@|V=u>pT)2i`9?6b`?kVdBP#b(AGXoSgY zx@+OI=#B^R(nAUno5J63)Q;?i+SBcr9%d(_vds+s=7x_9dOpxA7J|jxWTxu#6Jf(U zBz<9ymAc2<7;mPed=iqa`m6V?Eq3r=pgjS4 zoX)cq0kPEMU7((Ar(FC-Guw4Bb*(H`h9Wq_P;Z%0GTv@wIOAjYMT+{ZAAZ-7(fs&U zTK6H4_N}c?)$BuD7}bnjs)tVN*_G2KLeU22`@sWjDq5qbLRCD!PI>WNWIxNV%0jWw zD%ii9I^l+GZA6;|?_K$YcSXzwuNw3s^Z`3vYBqJVE<5s%(tv`=6{i_ZkjZr5v}wUj7;5k(PJvij#&q2D zg!f$did|#jT{-kKvM+QK=e3Fi5UlEa5@bvK=B>-D%fr+*Wa@Oory)PcPw^6hXC<_; z@)%-3B8;D*8F^p_#B^&CS(ch0r>|py@u6Qza>if+o@>?4H;D7}xh2?*4z0*g=U3C2FrvwiUPSTaKCs5M+EDaAZQDP_cVLz~)s^itHhXX^#AZWmi6 zPDXxEO+MZ@L>}N@{hR(8IjL&0y>PE^NL0{ADvhHagxVHwN%D41Fg1 ztHn(s)4;RjkKySjq_)v1;C$-r+tZbFV1&#P=D;&!1y1&e%R>!|30c5{jOJ8=X9qc0 z>r!UC`~x<+55BsMOOA_GCB@d0(o9S&B@>&#ZXY+V`%Q&@q^f(X zNU90mtF!elWkqU(D2noC*&%JB^H@kMX6f+L`!v~X@Agb?^_t@zEn)drcGVD($ zCpKyQNU-R5WCrXRq24|tu=!xG=+b*Y`PneUle=GftvX4obO&1$SdM`z!MIO)fFQD! zcvl{e8sB0LAjX{}aLs1U&VZ*b2g%I0|1R-<8Dw>228odmUaoLi#M|gPf2~#>)pg4K zyP|Mn;RL$-o18T3Q%EG}c_$V&ijWf4ac& z0j_xi{425rLj7*Wfx@pcs>CuM&CGsc1zd= zKws$vzWDJ&f;gj5<~V^V`UNVs2<1)E{I-6mdhLCv_KxvVJDE1Ay&0QZc%*A6g0n80 zL0p=-W(Ib{gTsHgFrtfV1Ae(rb>Fd^croRpu^3=H%rsbqJU6F^hfz;vR@5p!Rukuj zD)=+VL)1QFq7&4cGQt3-MvYpW8IWXavYc`Z}75lDDW$!$>F#V4^Ju0V=#tbF(6#{(meMjUrvz&`ORjPXcUdtqY;DOaz zCj{*q8r|k2Gh}>j*9>W@7fP=k_NkJ~QS&H3;Wy(hvNrIh{m$lI^)9`0=<0(0BFf*F z91T8kpS{JSRVmXD$2^3y8um{1i19kvvPC-g3#YN2tA8(Q=(V9fJdX9OK%=r3)?|x5 zEG>4)eE%dAcg{>wAsL0_#|@j-w!gKkZmBPbq-|xa`x$0!oqN~xrs%mq;>Q+1PZ#BX zH4?5yhwYI3JBx-Af{l_st_nwkYJFSXUTh&(O#v=lDL7XOop#A&Ig@O6%Y|G7b<8cf z)k1s2Hv*(D3aaID(m(#VW2wEn+>O{mHaWVJ!Y$oq(q3@+Iqlml9u@qz5pk;sawHQH zR#rCs9IvUGp^oTNCZ^=-59S}aQ)N*rKIsL&Ty$Z-t4$O5Q!zKb za{!XCeFcfJ+DwX;tGq|?+0P57OgXhZa73oD)kzU9?(886QQU8Mn>QLxMls4wZ%_qX zX{7m8z$Ajpfb<((iFZF+G9qGoj+wQJ?8>i+ZhT>J-hic%&5Ah*GF*km7n$q1V}Mn0 z{;~!9n|x9TUf6362&~Vi0uR3|@^XK+UFW2&(4@dIAq=wPz;&XFuCTYCy z3jFZVXBpcq&~IIQBgXeOto(ic?UFshaNrZuET12styhL4WP>XC2p3ox%8S3w9zYov zpSNhkj9TVlMj=0?Eo7y%GzV(RB^*hxWx_jkq8#k68%cs7({(1p8Nsi(H?;UbeDneB z3@3p1FlRK%eCxer{QR%lVc|HV-7pCB(QC&((ky-nH8sVbNtJO0lZ%#1DBuqrVBAlU z_5ordQ-*0ZER)nUYpCXEew16nm-GKESU-+&_IOM%x&2Iw6S4NWOXR^z2E`%4zc1h8 zLc;EPd`ObEK11qR8&bYr$HhEF?QoGuJ@rXwW0ow&DH#`kz=9 z1z#1$_pXbyE(oF-c2I0$d5ufs!uaQ_ifX3V<*=naA>hX*aD2Qe%4{U_{F(@CUl7AQ zxxAH@@FS){Z!hlS@yPAT@NZulTtvptk3D7jXK%5Dm}ewcBnb)_oq5WcanZ01Zwn^!(BmN%pSA=>iRln6+CK|$Zj;?rfXUfq<8ph;Xks7c<)}~xRzv^ZX3c-@(00>~3 z_(5omOq1rW*$fdQ!>+zRoU1B{N`Bum(vR#<-!0oI{r!n-hM01X1MB;j#UV#m($vqBQfNuv2#eAYtyA_&MnubF}Qqu0n*KObxBcFcK>aJ zO|@+8TerG6q$2k^WA@3}gd8k4;m`)@@Ch5<6k2x1pKb$y3htq?;F_#=n{oy%hH03A zxmQCuBMpk{nt}T3TQC zP(oB?t`B}=56?nrG(Ot!=uY*i?v+awl^idPF%ULPIAMP#s$f^gn%g>aZn)BOMsh#Y zxVBL`bdFSRq3;M6`7#;H{`m1@$xlc()=UA8#nGXGYoFU+!9yVNq?|j4Gh5l!y7<%@l-z}s)1$Xb+$p6`?Er=`nr(51}6W@8|k*o>bVn6a2 z{xjGu^%U^ew?B23K3AAn-$cnj_-&gN&>LKMCU=7? z_)P5Vuumq_i30g$JQjT_JB=0Xg#@|Xc$AQ&)=}4tLe}%b!DX1N0?PzQ$EC7vo)h#K z@PO1hLgzjf!vbN`YJvf{Ye~|a;JEgtkA(9_Ki5hI|UVa7Ao!K2lHaZp-C2H!#J!m}3qDZ5F@M;G>H*|2pv28$Tcs)@6|(#y=VLGZPBw} zyzt`(Q8|xN*(hdPi4O6fOB>4sWMA>yy|eui|3{L?cRLDhkE8fC9TY83of%6~(@nv! ze{*`#D=T(Q>=u{eomR?EoZ|RbljYZPx#^Z7Q8pCrB@?`;?)$wDcop7q*FNA7)0A)< zr{g+i?W1rD`e^p$?n(GENEuJ%0)M#9>oel-yRgY|?|6gxh8BU&He@)-QGKHJYwF{X zZ}n+#GgU{%RNz!NEZ0D5E^5pgB z$3R|X!~kfi)oB7wk(dgd5IYk)5@VWgfG88%R!e|`V5_fG=MnYqZ0?56S&OoxonPZ8wwp~wYA@>n3_>1WxW=I$_>X^!L9C@gCXkOGa(E9(kTPNCt5&El7@-5+ z&u6R3_^nhnq)L^eXGjZIL8PWmP>VXg!((mMRBcmcfb<;zRkn!5)Mp}X*Zkt(?7Mx{ zNgeuYvn;BS``=X-`rUIfDw@uWA;f^etD!8|2MlkI57txCF~B{_!>k+W2i^Fh!xLuf zg-N??0Y;lXOgHbioW%fce%N#O$Ij-G)K)9F&3paLaqBmXwA00aK4m>}PxwcUreg5qyrZRX9(Rg_Xyjz_&uT%rBU#mV!HQaFR^gL$1r*;;-h@1u)SYdY|H1?Dgy#pM2 zJL>@4hyXGSwpwB&ex<|Jx?+XvjhBCo-nLDmj@zRQeE%FDg*D#pnU)iUJ_+w+uEPGO z*u5V>U_%zevY=1h=W}o-mN!+$v#9h~aiTd-LvC&k<;0O+ndBspol%I#4GijIAq$Tr z9=XaqHje#)vw>9|CUGSQ<87!-D^&v`=dyiMf_qcdI(g0cF8w`!g^le zv_#>3xVxP0IYcx+(iVQY3FZnpI};t~IFqM(y?7b0g$tZHIh341CHz$da+SF9@3F~X z`!qJGl;&1;3Vm~&v0s=-%CoJw#`2(9s=JN-q~7L~El#&|Z5!f1U)nFhfkgk|>Wq}F z$M$Bwjs-=jKSm1LHCgVfqXj1)zzUiqpD+Zp^GlRIp2{h;+26V*$R<5Xlk{J>0`!S5 z>Oyaf)ZW!4pI@r2F8{p&pFkPw7IQ>rrT&qwu2>-&7)GUq@^5eH8D^ldE_H3(MSi&S zqCBQPztG8HG8H_s{K#MN?P&j|M67JrtBQWmv_!Dm+v5@;NxtT+abq}YRf^{m zB;epz8`8bG`0MG} zuJ;`J#PGx97~1Qq6pDd2ROeKj%ABz%?c42`UjTm0&HL<314>FytM~1&C7F({fVbkr6-6BzSM(G3Bq*u` zreiVa_63;FCcZ;z;=v&&S7-mxLW6${4_KGEL_zl+Yz?mL+y+tt_OW#BLTu!;!)X3vkYOF+f_uS{H)^TEq6Cu%E{fc1$PIX&5um)|14qrs^d z-Y_4Pzyy@6z(8ltfUFbwM)XjU-vz%ufyjjN$J^~StM*qv6X2BM6p~!gd%({3TX+hp zB=@3$Iut?}@lUHOH6?#y0ehtM6}+;`VO^-mLyegty`WlpJjWi}zrE!YZ#Nhq@m|hY zC9R>2MxXQM^7LxlL;XM-I3r6gT;lvy_)x|xEuWdJ5}lU+l!b#-csD5yYiBu4Q%n_D zV*Jthjlsz-1MjIbXud;~O!NEgs|I@SYQ<900RNTU@`b2xRlh*q=Z*Gzf#{-8o%N6+ z+6a%?uW885P^3_P`D^y)-;J&HbY5rBXgE2@l0~RwUA!i2r&Zu zr)Wv0Vp7c2$#6^xzls*}m8#Eib{YIad3 ziH(QXc&FrjeosXUT|c&K9J zTR{_FsEPTF+pw5Bflo&d?Ak~p{UoVHZ*oQ2We;*&NwpXCjICb4E|GgGI2SJx_eKSB znHhCMWdC~?_DDPq?T~ZQ=D3&zlaR2&Y<#p#0C8WT-H7qIt+sxWV?r9qhJ%>-i@*3- z&a+Dbha%~Gwb>TgLLL3v_QvnZ%Ua>kmp{Iid72|)Nl`6Rq)G14U-XuoFJZg$ z!#Xa9b@qf=@mPmhz`Fag=G~*3NA=A6EiQY!*^ilj?ymWJ+l6)>CiA@h#d<2gH*N`9 zPB72>xuE+pi;dOd+)&1ziRcPxc z>>!|1AGkN#Th~KTFo;#iFnI83BsSbBHqVeaKhRmSYG`=-X8?Yh*1>aXwv$%;EH8Q` z7bY4d_@>t3{!`0GA~9UpF6|IdD>Kxs3m~bV_dWuJ5Mc-bM{YB_7{CLsp5rA5?qhLV zV&ht6#=DTVxmk{yn{x@`@2@zjbaf`$dP^$MZokD`IdKKr*s^+MKVI$p|r zHyIm|Hp9fFm;9?&?N{l{QmK46%!93O`CF=@8g>W1T~Pg-P=89&hcm!&?5Q^lKPS^3 z;LYL7#xfc=i@b`A9nmRVM8PziNpS@D1UA&E@{c~h(kcZ1!*}kFN@JTyIK@%{m56ZS zKP{FdTVff@fwN2~OUW@R4TwYH{Nq3!Sm^t;S&T5hU1EZ-RLBxEl9oXF&5OiZsl(Jb zRu+{)&o|M0B+?sl@geNc74$*2IOlnopZ1{^))wnm-_ciQW zjV-(l5awQmCem*qgT0`H{mTd)#}g$+_r346M)4FMd_E5I5QslCwh}^MePrpHsHp4m zE#%D!sn;F9mEv(flz_VqH>#U6{T%KH3Vg9;r6vB1J+{)N)WXZA5BOk@n#X;A%0i0_wAUKa1!`Ut3*A3C^5#d+Y zzDv@0tR!1y!x@Gf*)Xpq@6*IeV}X;o@p8MG(9D~7JwxKZ>$F4t5o9iND97s>lWBQ*pj$6=ar|?z(Q!Bam&O2F%7?mJvF`%lxf+e z`7-|!UX!!yk4@YxKkDI}n|<3oW(F_Z_1s<))1@oXl7faa5jFAhrMjDv*f6ebe=wp06P5?Zn2A-KDvpUgx&8GidZ~@V5-+pMbLu*10JMLjrxQr`XKhu zONDZD5H3|M8YNyBNo!ijh0H@q_8Xs;;oYSV7+( zHg@@qXXSMCwDc8;>f^EtEkC$Gwhay-barh!?|==E`Mp$3v`xPNxRE~FTFE+Evi{|) zPQciq`1iYZX3=$x5nLO-`K+BKdUN)L{Y68cBeOFyeU5-9fN`wJe#7;c^geZHIDwcUbVBIVnr$wC@Gy+)n$Wi+(mAQH*}9@Vuez$ zA=%f2OUK_G4Mr8O2LEIHNT6rgfMYY!HQ!*op{K2#x-wsmQMVY!;Y2mHQ9_V%;*AB1N=SSzW7pGTcdC=9-l@mGya2kMWb-UnUhL%oA6dp*ARHeik~7> z%vUip=iVpKLasfuW@otzBj-!#$0iiGiadx@WAMluv9I7`HyUN;gN5%+!o=~***6wX zKfT2!BK`wLkftc#_Oz@UyQyhV&c9^pisp~~!TmbVxDTv4x|SG6iTno-fMV&w&hx)5 zXX4-J#dVXJHlnRHGU3N%O%tEv%RsIha=kF3B4*g?{ZM(FzgDb!VfdFog;e^y&q#ZI z7OuY&vE;jSW;6Zt>(QZ8$OqTnY8HXqk#a|FCcf8~pR?pcoQIr6x5b5ASzuadDgH4NQnjBA{pBO* z!(m-Ta_-zJY7Qmq6glYfGh`_19$QBd0ng|zJDZF=KbbnJu*rp`?$JJ<$X)Bbgk;-{ ze=U#`e*1>7kF5~C@&;xp9QP$&(q8fypgU`HU@*VU8;1p}KL(D7lGz(?(X&Ueg@DVk zCz0`p1x8~fy&quo7xRKbcK8401I_4JubIhfeKYcvdhDWXp5$)Rk1*}~SL^v_-;Pf^ z|4i=&by*s=JJz4$jf>6>h=YdYp0K{Md~wtc&S&mL!lf#nYxOra%cy zC5q0`UeFR-*GOImlkwq(c=3R8^mntyin|&E_omX-LQSfUqK>QAnz6&Wf0AI?)5jR!ER!haWY@*W}_i z(w7)W)hEi;#fiP$h|Icc-YewSBC-qgF>|U$RZbS!G z3J+4`%Qr4F-v*fN-B&APZNiuQBRPJlIYNHGZ@SF#08VR{d}QC9zI!R48$gspJjuaf zkA~vVfAM=i^C;F{mJ@VGknc0MIO31cUmx9m8?yP}c`HIl?JAIil%U>vl-GqZ7@IXTHE?$eV}JG9_ni{x6Wbb2Vh>1puID=lfJQC~HAqRRY6xJLb{v&%vO zHTMw3PN%05!IZQ5wP-6>$hG?0i~Ksw1?Kbb)K&-ZoD_Glj5HrLC_z}!(52<>vLtt5 zd;J*B&%xP=zHU5NIVQuPg~yB@0hqy>&-kZ)GIW7QQ5MIdAdI%ZGEA|uSLW>X$|4(Y z1V~Do>J4tZ=n3F>1T{7=_&GgpzpXm*^Mg8OB3}`)&_v{{wlNdvV;F*M-eJRul6qxa z>9$R-eWwIz?RA#i?ky19KLQ&SlzQC;ooC#6(2t;ju^(0?61;b&#=ty|U;sUpNknxc z<-GsGSp@R_y1^fqo-wK6)tr9GXonC^SKZ#YLGRUGIUk{(c|vb>(OHb+ywcW~ly@C0 zanOx?NqSWJBvY8T^h7jTDP?K%M4NEndlW@^9-m9oa@5iaEwi*>qFJ|DxBzn_S8V9G zlcG|DBGy&zq+21b3#jsBjS_q&p8#Sq$mK6ZsE~XUFU_AaM5DXk-2C~a>kE2rz$HO^ z3|T#fJs3sGDiET^kgCjbmFrl(`fqD~D(y1AaZ&?YPB;1MEg(eby%8k2J49fz_L37$ zs!w4$$8)~W{EPHGiCNiNpdqQc8Xdg1H4Qo&Ons3c-f`wvi_p$gp&dDqVRhy=%?6{i z@qu}IOti25hUE>$fecm13(kvO-NN3?=nltRQ8x*`4EK}AcUOlJXI(bS zzIDXquj-c1A5x-mG#0r}lO8Sl|JCVQU&=7q+#C8^in0>Ft4m|t^YEED8u04KK7qkkRT!+f0P zwPrjVv-vdS*f@kq^P#bK2gBA(cjDt`Wa(Z4pCYQ##%N-4tflH`hYRwRIjv2>rV?MC z4Gs%ZFnH&|j&*B#564ToOCp4@idUTdjU_KJzwZ|^hHRAXXxerm>khq6$={a~{Y$zN zCB%w=`rrROF8D(z@dk%z;(dl2WNOM_5nysYq-aCvVf{5Wk z7*8U)7-ko850Upxf43`@)V#Rn)EtlkvP#xX&+R zFjBfQta~^sjAgS7&xk%;==`9sS`&;dNh{q+wzt}zQX(chpJL5+(6MMb)B@hRznUUC zOV}j1hIp`V0f%Xld_s(lJfguR0Y4|}*TE<7zj$81HRKLHwxl(5%8DP(ui{UwB+9RQ z^gYDMu;k0Vy*0nFMA>u>BAbei-WxnUo$fJR?)R(6Du9=4hE~u9{yUc3&{ZK<`(bYi z3|-Dwl{K19;YTZ~6>#+#(dp_L(fzgfI85b<@EX< zG=(P2F~g@pe#zf&mJL>N_){-ATXy}&G-55pRNYBtJtEf2C#szK+~+G6bG-e zHQij9!q)xhA?bI9(cMSa?lJKfRoqZW2k|wi_$J!+QHs^onRKdtzir3qG62+`w!}`0 zNR@Imyb)BId@r+o9SGo|ddzxz87tB%JU0J=5yWwAbEA_O( z(GjGKLJYWmDRr{(>x#N|uhUB4KPq^WUbvuonekY~@|KQ-$Bwd{ld=V6G$QFc-g#wb z@V|UNyUzpuGJ@}26W;>)g<*%9&!!r8!uumtw4C(O z48(h(Ma)GlePYVmGN-Snz@N$>=o-nZH#ignxMRBU6j1O*>P+THbtdWWqx?TX2jjY8 z`HaV?gtpIko4Mc}mB@4;$1=_chQ3FzzPSn(rQyKT*y z9c7F7(0im)R0BD^JwN;T<$*ykH<2vRpSJP9Zc+Qry$xaflRA`>Qt{v>{fcA}umh|8 z)gF(?UsI-pou3?*%Z7e9ff%|yyGJ66VK=%}9}I*KrmWF=^>IpAY~@Xtkp0Ui>U+sn zWB+U1xNafO&YtcZDmEwn`HRbl;9KY?Z-6&|S=Er0SPvg23p*0mR$}dBe8h+MMArJD z*oUO}O}^4by}2HCl2b^a@4*>GlLxdLC|Z{7tLQf3ueaoQ)1P1_ zwf0I{lwSNW#t|T-A?E_36j70V>qb zC^{0ryXH9GSeWUD`-s`G^`sjMn5RZpP82$aJuq~r7- zSTLrXwioP(<_0ykUa#(l+v8V3Wz)kf$lG2C0NKXZLZh1aCli@ObeT}e_$B|7;S z>3Uw0y>gQO2E|Z^n;Il0e)t*6I)Gg?LWw|?MkTWT=)d>e+9!_xsQD>V=M%Hee?6@Ja6}sv&4qi(>VJ6`fHKQR|6Wt z@lv4+Gg}Td^d&|=Y`=IJ?!_oZ7i!_6?dqpcZ z9_E5^Pb(}UF^dv+^r-pA-6A7a28tMid;}AhpRFNi5yJ;7FVL^vUeKba{&V~IVD-c6 zPjbIj9`3m*{%hZ9)g4HT6^cN0^lmw^p%MjD+hLzf4J471`iwG7;j{F(pH`pIXC_Y} zw6~wlQ=(&~IYMvKRp`cZ~V*V;aO`H;fP;M)03 zUZ4|JCkry{rz;6R@3WB;n1Y9av?=nF&=13IGu;7Z5yg^tpPDiedK`PBWr){dkas3v z+XN>{x~>>~5_?7Wh2ayrhq@OAN2Rfdb+aX$%Y^8|;V>rrXC%txIk3}=iS@VLow>pu-1O7A{Ynqr?~;(rAQnv zUg$JrjVlCR9R@6dO(I$0rylrfyl8vMb76#@ww?cWAA~wgdjp3rde6x$i6*Bln%Z zEu9If%_oqcciB0;-Etayw`P-m1}l<7vB^RR+Ko%DwwBPnQDh&33BdsBM8P@K-f$xG zm6Wiz)e;ScsC2emg6%*9wlZAV`>k_>zriXM1@G!nfEH(|r3lD$v~+B5G4CN&5s}l)2JXT?H}u0eaFYpI1SpMNMBBjJW&gAEDMv13*_!g!xqS?p%+%#ea z8)1&+GCXKp|2+z$Fl1rYNED0wJ}%FY5w7+ZZoJ|Mb%*<$UNGYeZ`N+ab`+XVpIfj< zRJ9PV^S){M69~YL-!YJO6cu~E!u*3-5v97beaL@AKUl7rUBRQ5!j@T#(@n4rZB%@2 zKuzSCX}-av-(MxQ*vF zmQtTVM#+wqtw4JV@f5eM^SvrHhvtJ4i#_2bZ+)R(k0Q{cFa{!N1Z(Nt zXAWl~-pCsQ*eMd(2l}iz4Aya2&O5R9&c3`nlf*AXw$58wYLdt><7+>IX3Jo7SM94- z4E#j4HLAPntBfLEmiRjJ>t7%06a7WSZZsb9tu?cu6Q+p-`6xvtGJM|u#R=I z+SwT78LA8Is~SbJ%hqzk!S^tB7cjNgF2^#P7eZ^Gc`1@HDdaIm-Pz%&EILt^N~(3; ztmk@^`~%83D&1Rs4DpXeR~a8iCBO?=z_l5d09GWGyt~FF);Z`RKUS(wRN?-u97sHT zVgGUjnQ8m2v6VM>1ka&q!>5y9{~SX6t=N``A{}2g|0Bs*;p;*}&&H5(3fSOdh&oAJ z#Mk{~=6?$PkF)|R68VMKbUD%YDEpv}I|1XkjRW;g6}Yv8=4N3>zy16J>JjYQmby6r z8RD3cOK*GylHVKI{IUWyzvSn99vzyM%Re+aNh**Mjc|%Q``IZo3*tNX+hx+5x3w_i zoz)bIh@sZr3JkbCcKBC+mP5`;`TEEkL*ZLe+nVNA7--R`m#6Dh2;!IP7sIu9o{ZN8&%dO8^$dQU zArdu$aq+(O<{?`6O^xs28)a8xG1Zae=fmHUtW7)KFE(2J2_8)qC9TYYFE0;gpo{%g z^*^gMwNUrp(~$n!c{k<0ZuhhN#Ov?U6RY$_f%bZFBI0cG1EgaodyN><%Z#`rCWgFC zM1^?gUT!t3+z{j5sH;&^poRQAb&@fjOZww*^kF}b9;vY&3&O)hVI3IKBWW4VrK@%X&sM3OjCyj%MB}JZo6C~Y&wNP2NC2~$d*CZ| zH>Rh~99B=z0|uKd%u0y*7?bFsAH`bAUh3Tm-38=)(l(8B^HcSl?tF@t#EWKr%HK?K z*gm{nky3wHY(5fh2H&>wT^Pu$?lu+}1V`1iYf;^)!ai`@p=ZC_vj4#;Y?c46y>Ylv z6tG99*Eyl6^u~fR9&}mq!tFW&H#mRA{A4H%IG5f)p3vrd6(l6GpAuGlodsY)lvYaV zHt&l}3(~xH080|2LbW#yr-=z1{MSVTMAWtD1Rk&|uffov5K%9cHSeE}0oCkl6@$hS`Z29;yj zD(vxYAuF1{vCl8yxz_d^!3ukp`u4VT6& zv&WFjEyfu1YZS2gf)nx{pGR+E_c~sgNZcQepR~uT@vS*akt+5`ODTg_Ugv28B5?!g z=Pys{%y3fY$ZGw9;BiR|Z)(r0cB<_&qn(DLF3ri4*)j3EpYwwd2tKz>O z&~()i+Uo@}($~Amljtm%bI1A}$+H#eRPD8LR|5V=4var*j~H!Nx6<>?UR}_1?Pn$Q z@B8enlDt#MNVw(zgPO!l6^OjIk%JU&IH&T~$dgH?m+{2kx_#c5MewBs4RhWaU!-~U zwJ)|^eVM-0UiPT3Gb6WwvH?HeMW2tpQrZM?qTtu$h%^z;@UOfRuu5C({qM7^_||Wi zJ6pd`$avb7TmOpTB$ahoJoAm+yjkDLY|y`_F9h$*GsUp~X<&-?>1W5jKfmV4#3phj z`RW?u>n+1v>~;_BKjL3a*t`A3riDN^@RYo`*_)YxiZ4R`?9X)g`g}|8-?XcJSC8WT7IfAY45azqk+xl}UVKBzrMM~K#{W93 zMHdADi!5+_k=!WQXI*QC8%u>pUEXyZJ}Hpq*fD zdqrvY2Nv_Mm^3ajX{ICMsPXqnS`4h!GTT0I`ReRQA12;+S@8O=S9ej=5GZO{Q|!P4 z#_$Befj5U?o7W7?a)Tbs*`T<6r_ld(3cwH?#MjN!pSRLiVG&R<2irdMp__G!g)n9V zTx@}5xtW92d)YTzs4KFMNeGy9GUy1Lcdl-yk_#Zb-ny3vK9V4qkEH?mdaVBN-Rfvh zsp1+OlDXpM5gh6la($Y(r^_->b*JHaYLAwNHzz&~O2r?NwIfFidIu)cdT*tF8+6+{ zFDkyd7-$4wG9Xp#GjiH(fJ_+dX7=r3!E*DqxZ$vJxF&Yd8w*n3RUF3VaFFy#>u54_ z9iBrh825PgZd+%Cv`6zV)v0H}OA`F8y{9`UZpzma4gg(iTBq{NYJ&H zpnE>;CYsb?E!RG;oSYbi|MyrWE`oOt>6gq52=cM;dX}T+x~j=m)wVhr`wZkLlGu6+ zRo@NgKaMy~BsJVWp~Hh3^jlo^5E-2=FOww7BPndQA4}3bd}&8F;tg zj>-LH*ni1a&}Q`U;DIvX_YRjdp>@OhKs|)wky6GW`1HmA>kE9>RHEv!IT)veKUqk< zmw%=YU|pShIC=GqxVGXN`a=7-FulKrsev2*{JoXVU9(B1@cYa_AN0ZO%eC58ooDSY zf{rnekYSxFZ6|{d3kbI!1m{H)ry2L7plyBm+)sB}Hie={mt9wf5tTPTFzE>eUR5Da z^UX>AT1xUHO2$x(C^EXjKe%_9Lb50F;sR2$GsA&MO$Lq5|5W}N8TLL)^3-j)(ubb3 zk~))`eijt;q{w)BCFS$TcF6}RB4xC(D7%yrA*xIPPaCP=KA^n;%DNtBY$Y5mIZ3cu z9scZ|K)u6?#ziMe);z@@FSy)dQ)z#IHD8!6`9 z$MAa&8PKSls$71UFbbXLI(tg^Q{a>4rBw2k$}}a94Q2E`VQAT$@@uCwcItVzHuEVr z?znr^ia0?Ju7CJwet-;9T86NY_y8jbYKS_aqUK zd!r;m?eJ$NOBIm-{A=M;sig=Jy^nY5Z)o4VtO?iv4Hl5y1^PH~kB&~P_M3t8R@VtQ z4DtL#=9%g+_9kf-Mz@gePK>aj&Zsy`agbO!$2M|}-+3VX90O=u>14rdNVj;K)^LOT zLWZ&Vr7ZIAFdw?;2RbP@JDB3D&rb>+} zs?;VQ)Tmi|WmH>LGgYg#QdCj3_e$(d>`m<05;2m9LQSZLv{JxM40cFzRivWOABs zJjRQu)iD-LU9|yDTs)V{U?3}%#A(rKjJYvDQ?y++l(mr2FJld(e1OM&{J~oStxR!{{HF!RyjxuLWe9JqbKQJSJBWPL=g~O8cAEiNOLl(pMPk&{KzrCBXeKZ%8rgMw@e6vry$I8>}6zuosj?5P>c&PF@F+Y7WcL&}%|+Iu!Y{SzJ;g z*ZFN%%uZ+;RSInRioe&dMdl3iubF)tUsYKejpwsTa=h<-*l!6IuZ_8mh;|G{I%-l{CO_&n7D5vK7KYrb=|s08hcWfAPDkFO zt!9~!xpm-^STYAFQR|LDujJ_$(jPwX zSyl0Zc6IiZlyj{R{w=3OIqWx`lJ#V61$l0jlCY3%WqB*;_&EQ6tw2E9gQdUGgV2D3 zQ9^%BGb5iDX=csroLDE#9vW@bsElewW!zRDZ{$wcntxAy5@eB`M7KZ967gsdci2 z-d_1+$8WdCSMW*JiF($bfCxGi4EopD$%moPGf*H~#1$=!FU_V#nJUN2BJ;?#=t$wV zqKvU@+JgeaNMHjIvW4&3uiZ>7FwP5WA8notAy{#Pw&cjhipT*qC;*BJvnnP>43x|9 zS?vP2>EZ4)7*j5*06tbR6##xa7I z9e>yg9$Ix|XoJY2C{Lg-Uc87Qj;SfwK=HB(2wS?=1PflaV7dS3?;*gI8y~Iau;0Mc zqkVW%Qa_QF4Bjo@lAR?st!?#nVt~AlB#Wz#dA+RJr{fj$Z|%)+fo>f$ba^Q;(@l?Z zuqs>~tt0)OnaUMgU|yh6=f7l#zf83tAt#f2NBK-F%XW^ktz*$%%osbcP+4Tyx+bSj zq)EP~Hm$7l?KLj_eP{eNZ#|7}xO`w%x1ve>0dolk@5mLvH#?r>NMHA0I8NQbGVxwAFnm=^m8px-Mpm6%IwF zlmPlTM!YSh$*?SPsGzv>t9?v!q#GR``4b=imTljnNfH`noyc+D`v%5qCE@vGP^?YP z4|$=COQVDbPo|Vve5aj0c)>{>VwaYD(JjHw^3IC|{YWNY~!pf-dHS9Tg%YcUo zH-M#Uw|*|a7u^s~iFO5o9BmhY6n6&K7E3Ajc-jB9EMszhoZi2b_<_U~CxDUsIw)QU z4>{6G>Sn%nU!P>%Ciw#*t~RsoN%_R4yo?H*dFFB9glgrv+%yOPZ_jqkc{`=Rgzfc= zPsv=i;DUYXCh|KT~cWuMoSn}*nBbJmW zpy2xfVf;=-@!fJRHWN7=ib1O_Yc^zhxE2nHrEB#V&4b4)Q7mz!?3*ud`d$S=BCfeb90aWT!L*T>Bt>{5Wm+Ah}x7OjmuJ)fF ze7icO2Qd-GHC^B%7Qp;W-La52(Y!{j$Ck&Aza@cA+`#*zJzdWeDd#atYs7ucrj zf9ZGW)!olci{aFtI;m#QB*3xGMu%_m?-{69fzq?~mBm4Q%tF$Q?vR+lFE zGp67Li8`eKuF3~i9iStX_CFFXrgz>>m4iCGqEQ@oxK<3w=po`w9ljdM4YqJpY2 z7osutU>yq+l4B%jc6eh8Gjr?p2-PwZ5@m8Mk|%&NKMNl~VG-oX6{!DIQ_U$vS6(H` zEF;RSF<`ZOpgI^+j9Bohs+uTT#~=U);bnmso(?`1DWxq#Zkw9!d=bAbdUT52oPF(9 z+QD1}Xo{c;*T?lEJPI>To)Gg;6EyMR$-udN`Ypi=rmm!IwO2c<4g?BG_xcZ_e0fmZ znX4ZB&8@K5BL(w-@T9*ahLDTKc^Q6VUhuPVI3mRUyGd_sYaXp*uef5VL5&P#YI>Iw z8%>m0Ir)sCQ9j>=KKFjHfLz%FkvhDb0VoxQomm9wD-Hr7^vj~BPgK+%Lrdy$=`xVF zlkG7N_E)a0NAUZSx6N#KgFE(jC#{d_R09{Ut=mb5mf`I61OnHJ2m=Z<>p>KEI{nDT zR@Yd!JxX_?B!Bmzx=9KlfEC%3whrPyn?y~Ny~azngPW&HmfD_G&`=~{e;fwHF0$`) z#dCU(xZFq5WpuT5uM$x3o30rg`ZmHr%m0s_*%dD4Z^k5#NozbwH6UH>NHJJTRalkU zJG-6{h=g@g^!NhiX!%CPKm?6UI(92*+({qZyf!=4;DsUsxIK|)(jMCgEer;&vc`v) zVY&FKlgb8w4jHmz?3pN~=@?zZWNFCyH#@-LK zqYlinjm9^C+?2jf00RKxVa(;>6I$tn! zLoE=qz(nzyCCd{>uVIBDn<8H^@{~l8XCK3t2~ikW51A*)NtPCJzNq@mwxMc!RNaR$ zV5>sZJ7kOu=LGXOMek<$y3^V1vdD&KrXGTTf;m= zpDP~){l39}Yxi$#Bc_Am_*717kz-w6(~Jg4QKym0^9p#Yw^Pp4e@j+EHa0#KY-JKp zl#j`w3^Z6|S^bA5i}fiFNqDG&H#XlFTO?3Nd06vTbSLAc&g(S)!Mh*OQ zpW<6+T)4m@eS0)N2;6?@I#Go|N~sIyey@yycIP`qqiGeO-@SbXpzSEYaTd3plbi^v zK#J3hFWSOlk7_Y!T=3*^Zr#Rmv@b+GDsjs#6ORR!rcfloaJdqZ{B|=Z1LbwN;LoNx zJDGj0hj--ztglBzvyNlq+)4ys7zpR@?vawvvk7Gwj&-|H++RJNV4za`^ok6t$C`a! z&pi^iVPuXVlgeu=a>Fq@b?RO++rbOz3tz9B+@_!}athZF9L!V-4hIRugtTA`8p+ue0KSu79$g?wq>BtO@3^_*^-prYELcac zrCrQtzm`h3#Huc3qcU3U?|GO`r|c{JfteP+8R6@Vi;S&Eyg=7VSq=}Aw$OK?#Gy>tMt`22r3jlbM2+$pEW(I#+ewU zA{KUd{jiwQ;fZI*l;B2TiQRu#PDe_n+xNc_KNi?zUsFzbR%%87{A7UU(gO>s^N@ZX# zq=s8*J)&}*h6nePR%DZ=6Tc^3xNveW>O%%mxHip7pT>+bhkT)G9I3ya{t9TBc2U7^ zw5i+_09&T^5k)@24wBF}&6M6s3JPaD@bt5x?daV^&6IAu3TViMuSRSb!!eDOi4v9% zCd~Z70=J>FRVkQ|qc?^IAu2~#;2+KJnJ)tmn}3*?T1?kX_(7 z9IZxOtW8#vI~#?8m)~rnDej5^|GK=w*b`%{R)8CiGs&O7j8zQR9V0e0dO&|ecmgNB zL@?;SFd<4&o$aJP{(lz0upI9TT8u$8rD6~G_WWoToNcNO$)G#IK<9&Ih%yzg&b9wI z@AvASNc7^w0qn`8WPAthzkJwC+X$^_E1vY5{Rm+VH2*t~94B4@Nz_mhM6s5zi3ke4 z*V5!~rO#j_>1X8VL@7qx@86}iRd@ea`y6o%B1pG2}TP+>qqQm&s@?k-H z3Imb3c2d&?Xq#)__?Tqzqeac@{^B~DMl1_p^?ZWkJCnPYe6PF`qCQ93I7;GV$-kDL z-{kuaKPDJNWM795^X;+cTMk%s-HRT!zq5S)OS%0!4bc?)+w)DwcZcs`hhDt7*ui|R zGzeF2`ueemG}~}vN`OPo9bk6$gKBJ&uo3aCQxMI~%zCpptwb!N-&qvscs_ikPvszw zH@D+!&b_rCt`EDUNTWU<|O%xa!f~{WolXqPj|}{DX^N zmlb~j0p;XB@-U1iea^Zz$4zKD1F9WoDn~2Dc8bpK`gJd93tlDzDb2N`)We28H1ITR zIMA`;zmHdyP7J)tM)zn(C=g%O>UY6Uzq@tL?CMUHC+r{})U1$RH8N+Env0{1ac2XD z24^r-8nL^La|}%;bMaP9G44<8RA5Lw&!X>tKh{5MJ7+S!ih~`tT+s&8wo@`8TXZ)f zr{ZCkTadgICShiG1(D%JWmqo$ibFd&z+)#u+*>~_?w9Z9sS(Ea+4@~F$q}H8!@RaG zI8V;}rQpXfvGf1*}^+u*Ip$k~6#y!SH?vODwGNeiI?v{$#0 zZ}HJTh_F}MsfVyo4O$_Qb+q?I^e@zci=`ZO(M%WzcW+F&@2lI~YgsA_y`pvg@P%x} zrwPF-Dd?NTRUza(?SRWR^oF&F$swX#Oe<`0?Q;6fmdS|ezXD0b12g8DW8%l3?gRdm zFjzMS@nh2zF;|_a%~yo4bE9PE^L7z$Xo|R-gv7(G7&gPi8{3IL6<+Zeu5updrqps@ z8A}bPMtXIy!>y<;h3(Hp9kAov9~$pS^~vyv2je=e4vA?FnAl zN1gaf7SGTv-B^s9Or49LN~^W51JwD+Z|>r!hc7gWQel!bK{@#=7|1N^%3i}{o0CJk zj^-?bsf;e=nTGkyWc5`}X?=~i>}5~?dPcf}zb-xaEhjEicnjDPwGl;RK8xrqVTm~N)TI!@F+BcI z?L>}+7IC+H(3KM5yYMH&7C9+d!gW?Q-8k+KU+Jrz23;z@33i z<+7U?Ex3K=#Ao*`f}|r=^`t~QBGTuBPhjg*S)g^`K9yC6`Nrh0b6?p+FxJ+UJ#XCZ{H+Ln z@Oxv_?<0Y`1h@)p$xrmZpAl{3S|^ekfBAF7xl{rCwpag8fukG1Kk5OH2<;uCcJ%~K zxxOL$%Xh-7KY~`79a8yF%FzN7b;re<-=T|SAoN$ zra9MRvAmi|_qxzbL{BZhma|`1EOM70pnnu>*ECQ4duiIcU)E+BGaOHUJ$~wME2e7P z5$O3^m?sh)W;UtwD@F+EG`%JXz7f^?)8k#}Z^DeDF<%QP@FwV+mIH=%_O~cy)}K2A z@YbBVPSKw&J}=TzNMphr6b~d@I!)(G6;#lC-o|w7bXGm>Qo-;3B&7@b=C}`c#)U#m zE>p8dhySIp|IOJba5W0EdM2%OCVc5zzF56-7L98b1ypjqW-ue8w*9tL5_0d4nlxti z&=AtC3yu;ubHro^9~nL7!;Z&I9+{PoAQJ?jpH-M&g8f;&A!2Un%!vBw}~W3{EgDsn&7ATt(S{6|^(3+<}Bt;3gs<){hEeIuOEoomqC zlT5f5ZSXGaa97uQ^Xtg5%lmI`x>&^Vq#?>HfoZ>iZ;tJ3Jntd#WYl2Jxk3RH)a!k5 zl-S`!Gg6ROMeT^j8)boKu2M}^4~2#T}fQgc!-?-Yqf0ry3H?I zrgyB1F4}_LQU)1O0;qA~ol%MyMBZR$9LT;n!wclWeCQ_jK8*B9g7hi=y$h9uy#B=o zvHzWOxxCz;2*lLN53?dx zorqIo^?aUw zD37N6+3I?IKS~?A?>3repnEQLWNMiZaCEVq`cY|ghz5dBxz|CQl^WSYm-_a+3W{4|ZlI!f>DVl?l zIE7pzT0902q9NH;)kIcP-Y@zP*KPD~-Hfr@FE1x_$DJid7D><&&on=}FS z!L2ZmjjLch{PA+7YVQY)%-pQz)t$qa(cW=I&wh9GesVAUbGmXgaBa`ruf5cXb~!*r zpVe11&YZ#cb&%+LRXbg)S;Pdgq0NXFxw zIVw+WUI!I@yy(gH?ogY_o}pJf#sA-C4{XEjO|AX8Vq}kS(W=L_4kz#60)gC|&tHOF zc76=wua=g+cBp$}>R*}_V;GslW|e>0u>o8Dr@hAf0Q>q;w+Kuv(~_2vmN#DzIx1&2 zq-vDyfqNrSsc)Cne^j~~fU%l?Yye;fA`~Ay_2x0|{vzyxVN4604@d~)V_YF|SqD|{ z1J6O##w~n~<4a!LB)>N|VXB>OT>8$E2_@^gGcGb0{zGo1QhvBN42vXNo_O%h@+i%_ zqz@3b&d%&SxPR}49$-s{P$Ln{W1qQq7@0`t0Z)GnCJr9{VvQl+*?O=1yqwGDZdJLA zqes~^_^`!00&MSv_?yuLpIJS~Fy9GnyGeV3*FlaIt&b-UaEg03`>wN{V$Q@2GyC(u z!vnh2f_kOzctITY8jn7(H-`3SP^137_k%xkVrjt;PqhG)r4wJ@KPCfav><-)`~EW^ zdD%Hnf9*@0j%?L1&|cG|3U0nwhPQReKQw!zes4yB9A|Zm2`*Ijs&8@Ss(CiYn8ZN5 zKpHFAX;3~n<+w7HP;W~P7g*zk0BKuTBnc164Bpb+T7UJOG72-1JNB$U7=hG4XXk6I z0kNDE4{6Job~|?9V`>^=!C3-S{awOJ*w?G?Ba6doxR%sbW9t|9C9K^Wyj)rNh8T-X zz4;m~U_JLM=ze@`vobI9(o{)D38ZE6_LbKP@M#vY$3{_!hR$(N&7N0Jr@CF?pzTYv zdpt(@R{5%nJgBj*>x&jzx8sJjX@Q`l^YO+mybR0v2Y`2ynQ-qOd>FBeRmnlw?%9wV zu`(jmKeFT&FgjU}RtX_h;jy@Ix}t^uX)vuGIv)jE!H7#M4RUCQ_d$2s6w`s4U*2OTs+OpwU7Ap0^vy>&3HzeA(~E4;-y3t z@i_zrS}Ki`m<2y*%eJDhRmrFg9Uy)H1VK2ou0H73xBRUIN9r5cqup8!sT&ot8TX$d z>JTs&t7owhI5l|hY{0sFT}Roi%1RV~zS(gjjtLIgi#W`D5BzA4aM z!2XusIHh4TG$4d9D{q=Rj%YL4OL_KTX!)_b~2j=R^avQ~;%53oTgSCF~37-47=6aC=cQGq`@h)L(1G z-L5-G^`i6(V$S*;750y&(S(;1YnamNyTS9YgB_EfE1b+{&U~l^5XdOYZGL%DI%1b| zAWO*gF)vJcZ=+@NV$o5g$KQn^Qs6T2ika==-Tttj5d{?p!@eIn&(1dqpQC^H)5VL} z3?DENUR6Ekcw%rS$Sv$&Kees~9Zu%`eyndSGS|v z8>zp54@7G(oRH7)-TWBtAo%7t0&GvI`5bvD6c}D*?VkAZecZE-uTFWJ_xNR>4>rYZ z#mGptU1nh?Iz5+o|1@2B$LB_I(}CE-mSs6nShS{8=wdSQo|{&rj60_eCEMjt$TE7p z75a%@Oq!y-^NQ<}o2j$K#T%&RUXk!9rv!w>TY`wuKfwAYg@)@=AFKhN*rgH+4q3Kw z)7c%tx?mC^O%|Ry z^C=gl&$Kx&Nelo}32j>b>orB21yV<7XV4QkEcwr)c+pOH9gV)!5QW+I=S|;krL3-5Qg7(a2}qi9I}izTd1N1LK`W6-4^BUuBSze_@CK)@6A#4JZwr3 z2pwy3`lJS3-^QG-d(jSB3~v+3Nv36&3#^ke9fsA%*w~ z0aO^K07b;-Q5BI@T{0_HEQoAQ>7z(|Rx0R&3~_~pAFr2th?t7x<6sy8I?DpJ+!CXV z0qpH)l8yFB?L2HO_YW-{CrPVYrd06Y^Lw;v?>7>c=?mS?hQL|L>#pRXC9MR&lG4tG zF}vE%WSS?6pyV}Mt45=q)IfFM*O~G$;FVuLN11x@fO*DX>yPmlPLOsiQso=kD4g8D zm-m~rEa27U80q@1*Cp~OGit(o32}0nrBMi4Qb%me3(5{V-nIHsa?i7xHPfVH-$ucv z5d3&#a%vZ)R*{dQO!A;uq-wjhfw1l+XWR1r^1dYMm80f7KHl4fe=S-N{sm<2?SyWr zb?pf=va>s##6K{zJ;HkQ*&N?)arBdOF@P@P^#)ao)(~2n7vd-6X_;g!W=WucMmHgr z<65I{#)fEJ`c55{r_H1%>QM&@rPbhK%CA_mxUBugw9-%wg;bgQP?nLAdrBW_rnG^| zGVS=%YHDyiK3e@RDKIVLWCO4yb8F;@;au(N&gi-*)DTEMAxddw34NEY#R=^s{v8yD zR$0iYX|^@D6aTYsE9Cd9~Ai!3(FEF7?w4tMGcFey~x zmpX0>*^vxpl^cDcU;~+6!)7QZqVNp%bx_mjUf3fjy#|Afo@+08xue5JlYoA5#>vE9 z%PlfGK}5l{DgVPIK7I)tbz6BM`y5?a#Is2|z8mGw4~PkteEqu`x`zWajXhf`x@sab zKnY#w>CiZ9R^4z$g*je_gpB+ylu-lFvN~XB(n)xyR~9XO9rHrBB$X>J>^z7_@`|86 z%{icb&2EKV#T?LEzxb}BmcA&7u3VJn{d6&6jnSRxvG{}KgY0GH@z)}~Tu*A>M&`pT zTU>d;H@?OPK+hA|2brJEgW zRN_C^(h?Bw`>;B5Pmb$Aw%bC@&W9-mmHLt9n8z(}em1G?|GFpBigce`J(1vMyvbwg z^hzl8j-QQ!rtz?D;F}0hhgYW0{|-6hbqTMXh9vRt!c2pDpT?X{|9LD%T%L{QI^chdH3giF&vtg!s8TVEb$ByC4X2JXSDVp#-c#~>0syI z^&9&gaqV7@6{e3Nz_SC|d*w)uHnOzJ5m6OL01msk&N7wM4rS7I8Ofn?Lt?i(#70Hm zX^q)M2-w+6&^ui$ZoVb#g#7sS<>PI+B7P|%SA?mDeD%CP;tWM-yG9t)m}x%bk)L;w zu37$Tt!WL(I8vhknLJ8+CWf6y#1~G;Z%n7(_j$0-`OjJvBZYW(8>k1-7vmSU8P7kp zkgy1oz>g5S!$9OAD${h1-GCIPfqvGrc$Uz;Cu`F^iNA2-+SHJWlO4{pVPL|tje@P% zm*&_K0Ca3?yK(y0CSUc~qmp^=5_l8#697rg_61c?dIMq+YkHq@UI@+H2-SxHaXo10 z!7ne|&aEMN=t<}abxxfL`k$LXmc2Uas^S+AcyZQ3uHW%rRDbsQg83oh8j1m_=8zMl z*(r1F6ZYFOmyj#N*>iOEJ*zmN(U+X==k=7;FOr`)FPkO6l#1_ah5-h3M7-?_vT`?` zJok{E5d!-?a63=WcW$J29w#JyDIn%i%h6mzHqKvfHUW6Q>Ikf`pKX7|RLY!qjIZcX z9#?31^Z`ADwC1>ZP}A7OLbV>?oY}=vhwUuG&J+pp?{|w<@-jZ$rQ6A;y!e2lA9g8k zrjBnb2_fBMe@j61Pab}Y1Z@3G%oXqM;_H>r(TDXnf3@l;xh<}#-n+ze4fGC_5M0dZ zZ*!?~r|>)ULMkzgn{u&t;>Tl2i105C2Ix(c;04xB$}K{Q!yo2O2emJYZXlLb`36#o zfTly@+XFqwZ;tVf{bht5-=i&jXzTu&-L*G@HPdr6KA^dCTi|bMi)_D98q9mTC~)4l zkR9ATQYwYT+%$!?g#s)0#kJ2&+q}nvcFx48c(Vp57&fK8{n6O4mY}u&Fie~-h4KkW zzb5nQNfAQKIk#_A>79+ouNLF=-bu{#3>lfFLMS7w+FZ&k_G9g++2s_1!PGiY#dXa7 ze@L*1bLREuX?bTP_OwQ4>@*rR`o~cDtQKgLDgDg7s|grzGjKxFI=?pN2>t4f<0w)f zwRkX5XBFx-zY_k&B4e?m)9W1Ld*3CI z(vu z+x#B51cz8`ek3LIMe=nlBZhH}@0eg2v+RMFbgC77-BDi#IlJ9@>CSG2t3O0Ua91aG zPI5=DOx#$Moo(=k^Fnwnn(k$v?#vOO*0n913WO46GDFulV=!$f!Z@qKtZNW3@I%cy zDAm47+hLRTJ~DtG=nd)u!yobsa2rHeDMYaM-(F!5%}^ zx{Z9J603Jrj>({~Bag#3DIKu-L6nO1LA>|Me#0pj)YYLZH`&sATn-C*FL-zbKUACx zPxQHtm1q$7%H4PgOB<_aoeh7Fn87W$+4@@ehYC82uB2Jer8;gj>9etui(D zXMKt{{Bpiv>sIG?{oq#zfh9LlOVS}jx6!b$`}i9s)IaoTudBK+uUTZ)0}kS4H!Lhc zu-J?SO8qzYs&48^N;O|%bY^?nmFdw#*nktEwtfZGw5LRkwikUBe3pf{oFz8K`u_Rj zn$I;Fk4lXj58l1!ShC}%JbupMcmYWZ|%#9232VigESm~{jYr_8EUZbc7 z@Ia6Dku7qc%+jqSMCy}kh*;8H!tSKiLe&~HLZDMzsZF*Z46yBg3l}HyZ0JK}TH;nO zW@Z5Cm|~^AZuj(sVdg5lRpP(&yLh&m><>1g7LbJ|tgZD9XK~1NOS;Q&In^y!am47I(J8t0syOVTQOX57+_G$NoG5$XovgAg%kBGJ z@TqWWZQIA_`^U-pHT4ILvb*LL`*Szqrq58cE^WDG-sW$~6Ghu?>Dk6rxOs*Lsa>GT z&?$D$tlUeP|+7qX3pfX8!^GMF%ln~G7iPX=fDT+~o zcjoU8tf7C92Qd5%3$qH5o;-{_2#g7p1VWR#xx6wCpBR?qx~`mHXzPsD4~K=e4E>k& zt|l5Ubau`*?tcJKR7C$v2<~Gw@tnB0CBnTF+L4Xk7C|E_J{^)T|Ue$3@YNf0=atj71Rm%eq?y?>cL?R zZ5OY%`x@~bwsFx#3TjlOfk>rl9L8UyxZ(N8Pk^(?F!LBMMUI56pmBL*QfKP4iHLeyE?f)K!;t&_=+3AT^3mCXkg1)Mw3Zt5UaWzM;e8h`jF}-$n+&u zEd*gDg>1w;!eW@rdu>9z&xMUzJB8m=m|oyJXH1Xe`RUh@_vFLk{wb*hr(p@+$g_`K zHv*n6nKppmkLBn7(f@gnwhz;P)howW_^)i@I28JrG6>@W?$+`6LLk)LUC&UGh3Zxg*AmEXHFLoVy%UF5XyTWO zDG68N{#?qw_bLoQjh*4AUis*xLyJ5;+|kxz23g+8X#+N0zWzn{;9H^!AhOQIL%T26ZTYsQW%bI(z_v66GXPPPP zBgl%PCQH&)6tU_#aXT~88(RG{MkpWlA&AG%!aA>OD^;ndV>Ti+{alobB;`t zV@o*KwX7HrX0+z&>)V6U9Hu~OtMmJUccYn~3_B-R#Sn7=Rg*59{251tQbT~x#&Iav z#HP$Er|d*@`#=x;L8^XZSsUa}025@!uAydHvPO=d#Llf4Gg0>_}o>7-p|5ddfkkNNP!Z4nly2%Z!Jz3}5N1X-;2#D$Vrn1~G#YGhsmEqZx zcMpaQ`QFvPbEB9%9{D)B7!#HQ!?*Pfs`?)C*IbveEFP}u&Wy^s{mBNk$`a2utr+unM{*b>Nj{WNH~ICN#w0r6x#zLDOnAs* zDB*-o8_ja#R=wa31WKQG3_<1$!CD{3v>uWDkm}MKQ(_Mm9VKQ5#duXjV{X$?sQkn4 z{U~R|hT(e+Rt+!YMYM#cd;N-M;GCbv6GuW63p(TrK*9V$W6dAKELcAZZWx67x(_}3 zUSYuVO*m_*pFDK z8`M~N0>Q1c$e`aWGLNcPQ10}#JM7f-YEoi^KTa9sPE_6o=w zvpvk2lo|P&b39)`@sUtGgzNR6uFcGwEl2gD(-UmP649Yd&|5000o{4kS8he>T}b4@ zB&X*qu1fN)Gu#}%6l)e;pZl&r_6dGCLQM7eGWYSp->}$nR(Y(IOG^_QaXAa7;@QH$ zZ6aVMqW&~=C{Ei`FvrN?^jO&CbeVzE_B%|zZpc3R;wo1RnH2K2-B?qn*~$j$_p~<) zty~j=7K*;ls1$x}IR-^Jf9ww~W=ankyEbfGi1aFf2M-JPikWRNz6r)Q>CPRyO)+~c^%BJHe0U?4}T!Oy?Lr~(wb!8_D{^$XGnU!^7+1=z0mydNM*RlES z|6k6%k2jyUFWd;-xyi|4^z>ERMqzC6Ka@j{YcPX1gL*;C;jP8sRUXu~G7dKc#C7~NnQ9F z{HAR%RX1$KfX#6GrdSNm`8d}5p5<#3R{M=(Xi9Tkq5#_Ft%8=31-M`9M)Y|2s?4+|eJ8`?;`;x{(YTQ>GyB69xi zLZlL-?B6_cZh252DY5ui5OqM5BPdYxBzeAV0myeu)YKf2YBFP2K+oSt(e55h>hutk zu^>Ha2>9F6hng?Gl&c*?*6Mf=OYXpinSw>Wle&l;>(yh0^686bROe& z^IMMsc!;f)Oy*a>hahjNE6f096sz;{2uR1BciA>kgRu zQOv^MOuwOCzK*0?P~MVDTZuTdn5c!-&Jax_PPyZu_|jA9l?p@gXMuEL=JI&@ql-k_%wFqA$4A0J-zm@3@Y{_9!Db0`TTcp}?^Hc!>`dXjXjP8Qr zUL2U|7p3HYUX>0zs%HY^-Y1^**+S$@6nk0)LWN@i$F8MS_N|w6H7xDYY#OTGVHVVX znX#SH10B&^n;{%RpYszeU2ERrvpe*-q-`6&b@Cndn9o#4$$cMR3nZx^TloZFca7)OVvyE$9>zXnlmm9Bv;ooQw~~qAyGmU!|)Wi^&mxlGYwCaGd?!_iUs3 zlriTPg4?$hDv;3S^O+7qy979J49{2X@?w;F3(*5=P2;1nqLyE0$`r`>Y&|vVM4vjb zp_NpSC!R9Evs-M;Ps6iE6^DVbF!Jd&Ldi{wVZ3h;_a|bJKh;s*bg$1k{M$zGMZ!J% z7S1*65U~B|)fBSAPOe~jQGvXU|4X&~s^S`+1_$!!VgQvwC2iLJR#bw}f~S z`xThSN{$Z>Er21(75w1-(nf(3u{BAlf6=y;Wl&U*-DeG|InYggj{R#;zew@cEbgHN&StQG&po-rv zk&8P{@oes{;Ta@{3=Y-?_-t>Rd>!Ni#0MXe9{&D~;V|&Rc`xSyLbI_^+zd&bKUMYaBBX2mFxu>) zJt}9U`s-3YoA;M!+!8-y`*H7(GpLr15q@0n8M+R?XwjlQ z)7=pgE(d!X_`6ZE&cy$jWQ@Rb0gHcG^J|4_76xT5`$GeDYVIA%{1GO%Bgq^I1dZ*KtqNa*g`H5wQ-Zy_O@g_LGd<0jOI8;6MRS5iaEklFu>aOnTj=ILpJ$ zuJc(lhc)Ie6*39te1C}V&CGW(9KMlDo9@=o66O5YT~lH2APs5H1cLb%#?<&VP1$^G*m z05n^`aAgJ}y$`Z|8Y;k|@Ln*r?S?L8qksrz2PP6gs&S7faLs>!>=*F#X;S4x0RRBu zuk*9xzOV97F;D zeurbtAWRxQpRnEPp8C3}1NPOF-<lFtGUaFusSRVXzL)?+*+5H3f^C|r1GKrzFY{|Wv4N?j2Crs~wqH8Gm+&sa z#2u4;jZo?+fH6wFCWHYu$%#>%bCCWNj(8X8JjtU^ZC_DQ_kJth1N^Dq=tSn;dOpC$ zgemneBbMqlH}n2r(z1L;DQ&P{_PhILi=>D6ym@_2++TxWuIi`RC4uL)lUx_n2DR#_ z^_8{D(|e8y0hgSEUN6Job$obflu?@4>>|)wfM$<#?6Q-r*Yd^&cyVu zZ92n!*lM@1@*mq6$7UkTl7X6G3X$A;!o{QNpDFd##;Yb2ppU)ee6f_qTOtO3mDE%v zp0>rpRo>O+tMks-8O0SIq=sE6JsYDfeYc61pfL12B=sTcEqK7_ zevPlFubObbWpuMTefUB2pA-Ar2aUNuPhzgBrQw&g%`2rdk0_A>Go{f1R!!i#$2=DU z^t6v5H+aNNkBPDSx87v>0?|ZfD+|g4A>EgLCE) z1P@i&PVPr|UmaK_Fw0@C2;7@=p1c(M>k-2mSQ@I_Zxh#?DHqwJAM`Sf`}5!n)b$7- zKmcRrG}O;-e6t$JlK8EZnqdy10jMU@m2!*;{vJ+*r}gk~GiRfIs;0Jj<-ddjz0_Cr zwtYYT+W>K1F^IFCm32lPz_NVL$^5_iPVg-7W!=QVZ#(sg^;y0hJL0O;Ro>7QW>@Mm zM5Z9bmcDX{=)l`19d0itTFjr`@_jx#{`%{CZoqd()A*?>NI0~4>p-J%!3@$b^lfxA7%4|!z-h?%2I>o)*TL&OjfpV z=AumBOzkI?a;a`0G-*iIcXZ(CC(xfH|NVFX&p(ZQb!bC38Ep+DVFpYsGFA+NW)qMv zgC*XHMM_$|2K}ftiLrcwH)}C9MH=SU6d%34#tpGV=ehl@J^#TPIE+Z3?5DH{ zKMkxy>kQssqD4AHK1kbu${~9N@3%Eg9i9S~P9xjK%hOk5Yo>q6Z#8#*xMAPXgm zD}Wp45TKvZm=g?K`skzl+9vqd2%o1twp=YCXYQ(`V~Lf}DA2coa{{#f1ag%)5CJ+` zC;rO_yg(J%zJ1IBnEO6#jg78JEoz|$_M0HkTwUr?Q8~1?Y8JPBAW!(|KPZ~()x*Xw z10QcCGw)5@(a?3i;ryR*z-v0T=)2_couE4MNW;!?>GS^>4XVf)YvvkI&_M5}xZb{( z%<)$$7Ym|KGc;x0(b*nl-pj$+CwYDtx9>M`X}_jwY>WKqT_Vr8F1%rHl*DXjl^+Q* zz2siF8A>>6?ZJ^3?f+rGba;YlxYKvq)HI*o{VH0+zCy~*Vz=KLsDdH-DA0?Xphix? zys;ljw{OE*>pT2*lL%&)R&|d2LCBj`b3eg$TIy&pTZvsVRprI5oVkL&`M0!K=Z!Uc zl$7OxjdRE4=RNsj8PD+2EL=jVn4gON_m4nn$QDOn9k7v@x`sE}E~-J}YfAOgj_6*H z2*d^R7IS6GcyKfasgbO^=t!42OphXEMY<66OrsgR4}pb^X=Vh^zAiU#Aeba71qTNe zc912U8x|e?3;6#YL*lkfE|kDy;!gu^04k~ zvCpWhQ5y2@TU*RIW9m=X&tVCUz&zwpA>zgr=$T~5hPm1E#>Jmt?uTnB#yifxAfZkD z87WUNXkT||$2zG9d`+PFGjNARMud;5`rED+svu?|VXn3n6 zzkKx~xgY3tYFgJ5zu&Ua4>yC{W%G52VM1sYq|ZWsMGP|X5yQtAFmPhl{s|fEH2!Bq zUa0EYADp+KF|?G7^dJ2tu8tAkt2ZkKDF_~5|hV2SS-BEI8#kndsps-Rv`|vw#E<(hjH)a`N`xTuD|_IKHmjxR0S5Y{PY=5eoUBU9qMBv?0HWJ@2D zKrvG);=n-s|MF`1h2i_1AP=5Yw}0hNupjBXR~rsRr3WSdt^qf{oD?f+&MQL@U=a$Q_$sKV8x~dX+hK9hXuxp@5raF*CR4IUywC*W1qztJCbV)^&+1 zU+}pm&<3Z`3l}1}s_M$ir%5-@qlRPm!{UC|$T*%W9Y~3c+EgCQ=44bfnOZKDiG-8cYppAso$r?~o3m89>&RoGY|ZE*wqPj^^kLzQW$2N^X@fPHijK zE7ynE-g|E~e5#O8#)w}Y#&jQlD@Dawi);#=$BZ?Y>?iH>sRVw)m~})b{(lyL+nlgW ze}tE5o6~~Ral9}1eWc6@rFXBL&5F-Hs)mA>{s$kEdyhV8JdTWbqvXyNH19Z1sF9*2 za&CB0WqbfHVt@PDZCc<&j$lUm@h^skMpwz-Dt*50Eves+r!#4*cD96uKyD_)%E(`m zeN7DWCv78)pDUo#cx!G9{R_MLT?MK?Ko{_Uo(l`R_3z(dJ^5%h=k=gSyk1*v?bEMi zOzo;50}jey53P2M=2h3R#szb=jlCsNC4A9Ls%fp;J*BOL5sLpZX47H$EPyX;RK;(o zEDD~7n1lgIOwhcFDkYrLI+ z$$I`zsrqe9i&-2Ojh<~+{-{ow!4+ag^w!5viYLn=c$W+-Jb=Gil8u~dRIR9iBmCeTInk8|$U0*7yxGl&v|I%(u4Sbx7s1XxkKI{> z=LE=OUk@x;g`0Ze0ySC<5Y{oMxBvawdO#>-r?%$t0*xxX`cgvq?P&A+Klf_JrentT z78a>gQLa?KqczB+Ye^Qs2R6j?hx+9jcw;|}F{kB2b9)_fn^@}R8SKqmNsZ3o9F$(H z_CtwhZAMV@O;?1BpwsM*O;#t#zPw4c8sP6DS>0n26yaun%tVs;ronQV|0(cufNt_G zSFOgqM$_L6K78kUxK zv=xBURg**AP#mZC$rOCu@}r;aW!1tj2$gy3fqfgS(aV+NzHUd%6#R>JTDG>u@u+45srvTJ>L>A(i=WBN#=7y?G9EN&paO0Y^X`%A#IKk|0@>YL(${`{jaT zsp>~mH_WU~OgPbRL=zZv7^}W~D!d}{!ekoI-WzFd#&(@7qOx|8aiDFbHk!B#0SeMgVt<+K@?YY2M9>d$JTPh0 z$tO$btX3N1)q3t*!E8&h`x!f8(Eo(EFjpS4NK& zhy(POp2b$u_AgWfc21coGMgEB(1saahr5ECSM{$p^OSbAGf$5#7^XNe;O+y}mo8FF z8})D5m*x)MD`nwPcXaB5@}6V_tRFAOw774555Ia2z=+v^0+K!D6Uj^Ft0hjq($e($ z90_%Blaa=*&x!r~W;+a2wa)$T2q?*Cfar=eI@7VAZ9&i;OFei^YSApOt%-&&p!b7` zAF#a(I2zNHwxn)m_CK-vrLFqL%-?0(#cV(3%jhI7R(GNOAd<*$GyO}c!D3?bH+^a* z*v)hIsJ}n={$@fYP@QJ%H~%hrANl>NfoN7!NacYv=bi8VG1)mV^K)qY$+UcnYO915 zHen7vIFHmA2%$)j|&>@<62n1bZ}dG!=)kCQ?vclwab@N?XWd zxNrWIx(Z!)W7RtLs{PQE8Nzl;`MzBETy(5;j2{aXY@#S^6SHleylKS^P4H;>Z%=g_gjS}VBN=go2t@+yTyHl!|S3SGbJj2|diSn5pjWFPqe|5FqJ*FKW!k;h)8 zo5D-Uzb{rjxt=8>*i+z*#+WC7VNQArv}^Im$C4#lxA0A+;wVi8uqqi9!%*sg=1&&1TLSj{fvDaHNW;N)eVlWEsa7lC)DuF+ zFx3dQN z&-Ls-buG8YfxHmk0g?5VwV#ZX!asvXBLZ2fTP@bdGfmwQix*!Y8eGRQ2!a<^2WSN< zgt@mXZsediWw_sK(M~>Tk4Z>kYQGs*I%SYifqH-ZV>8=Z%Yq6(Afj}ph+o(bLs1pt zL!f~go`pXuVdcy>XSysRJnh%ZGrB;`;u8O1C%!$%C! z=0B+7v=@;NXj$@8#&%8%T?<%)uPOZ*z=X)3Z}Wj53=!8=Zs+lrs25UJML!i~Q|hXW z_tAGbaH#x`XFsWg~)AqteqKXMF^FFQuH@dv0-p zpM>5PMR#J~w84y0zW>vLmpYVNC4jk^Pp^qh7Z4waQ;jmXA^MTW&oJknS7^`V{9n~S zH;MD-w``X1wpXo!25cT&ARf_`y-cmd#;@Eg*~3@9Mu+-Y9Wmfmwby!2GV0yd#0Cdj zcd}kL1bHI$_9BkI@lmeFhDJftVTYk8Mtr1K`3>uHnC)Xw3X{2Ga9bR_Yi*VK#$7!S$vRYry4jF&&O9HASTceBt=v(C zwQm|%TUU!>uNi%Mi%p#QW+XI>pqL&O=Tv2_ge|UTcl#Gmxla*@x&}pbfi``Pt#|A* zJBIrA+Jkzdr!`gSu$RNrIk>K&XelJO8K)NS?~^)|@nJPP z;{6TaZ>5k4TVhJD1SamiaRm@Nx_rOuM!ckSzi3o2(z@SmC#+Ef={yDHpqeu?4q}eG zMZ`2-1Q3v{SY&3`3}#a@(Ml{cq$X$v`*yc#tyg;~p(jocMAmuK_n`1sNA7z`=LLgQ zfPm8v5dbio9m3^5YbIt+l*&L;wcheEHgg+3_Ha=3Fw;jmNt8SU z>+*@W`m#Sk!UIzI2G3LL-orNSvW58A@@VH z?@k1${(V6d!UxK8SDHDPVZ1K82=N*psjjuEn+|CnzTw4-19wgE zrRc<63SCRG2!Bw{pOBaS2X5+J&#&-0e$V>P~x*ck`J+@HrdccAgsFg|y4)N8(|qGnLwepMO>Y?z>X`O3x*@-pdg5A0BzX6 zc7E^^=tRKrb?0~2w+IXYqwlgQU%SwD%36QGAM*POqLClG?TGIe;UHISGp4P~q!8lc zNV1{=aoeG@G;X8E&41;`i#0OCF+9c%9x`+A!?Lx2X7MwcugkA=>Y}a&`2dCwMK*Kz zoGTp5Ayhdj+n26r!(D31;afG}(;G-4%r3C&)63puSPEp1Nu^&aQkBg586PbRcQ9=$ ziOL2!TZDnZPZ%OgM%t zX9JYYpp2+T)=Jw2wwf$*o?0XLBrx@8Pp~8=tWU=fV-Ifn)7B;Z`EYOz49+J-Wb9KQ zt?y-ZFY$gL253j8kU29Y@#v~N&0b9aM79ivqZzJS)&~n*Lig9yvCq~Jx&h#k%b{Ip z9prEjH06$dS3Iqty$jajm+IKk%6%Kyh!BcmVisgz;@qki_dfZYcy;}aF*)XuNOS=?8znxS_A(kF+sqGY^<3BWMVFeR zPzs;b!tMebboTprMkDBHE{N+S!{$QnD;^5%fCM_UG2;$U@O2Wv!hF~1Z?5)D2SG2A z#rqn{>+rXrG&f{n4a_Ij@1 z!zNsfu;DQ>{q9EzQiT9_95?`djRmktFL8#t4@^93ND5&sYVAdPE}!xl=nT^1-;q#b ze|_mLsaokZLzE`LD3lWTA$?onCLYIa7bn`aO9=N0ed8O+v&7Nqgw^|-?+V0%7(Qvy zuBE8+ViU)D1l7MTV~&iGHJ;{ws4NF(jnM0lf8X;VdoEj}D0DDI7?^aB4PhaY^5Qf| z$-P1?Q;#zMvp+C|NR>|}=0~uP!+vn2gPw5~d~o-Rqx{>Cpu{)A%tz)(UfjS0ux=d_ zN!;Q7M_Unexr*nQ{Mi6l$Rt7dPyU^d8vyulO#(Yi8s{^Q;H>%pvAVqJlRh>%v>yO2 zqVKws6XcaAuXz1W>X$N(DXR_mA>ck0RKODw$G@f8846dvm8^l$V<8DLu0;k_7U)o! zs(SQ+_bB3NU>Sd^Nkb$=0z!#TOaWjV8-tk~)m->4EYl8N_C-U16sf`h_pOTr+-;;!C@(n7B{=5*1V%;-@{HGFqbf>1=LpCq`}Mxx zD<(n@LHY?^f^T#A&puGK_j}JZg8V(b6zBegKqR>%zr=Pn@up|>B@3CkC4RMH8fC2w zq>Ru}e^}Y`R@mYfS4)@QPmZh}*Wshvm5Ek!9oo|F6$sLkHC1bkz zx!6mY+JD9_57=)E-JN8hJ2bi4hN%9XDt|L+v>hpX{+vcV+G%#RrtKlW$%yYYk}+^8 zhtQ>NtD0<$$F#dXWA`ALjfiNO!WRC^KbdIhh4y}krN+qDR3WuR51P(&F?hnJ3{Tz(v%U(5X%2enuOkarNhPvhbW7F8 z{XpIYW`6KsI;F3()@rlKX`_z4b7_#v6S`=Al^%8pKmBI=S8u-M6hgiYD~37R%~K0) z2Yc9!tmwqmeTne={DWR`;+KSwR!wkII>zJGE#$t>!{jx(ao#9R(&M9u$|gTmMsTy) z`af|G-aD~Ncq&^x@fL>f+)zd}XAqhwg>=xA1Sqd;ms3d%JW3h;Zh^OsAAzXgahnLo z+3j}L=sRwq=`hZh5s{EPu_4fJ-P?CWFXk~tZX%e*J{ZFuLdmNHRq-Pp0&m_-4xj;K z%cDyJS7AT@i;(ymXpbS{h)4i}2!k8&Hj6vBi$EL32MP*%Vg%M8H2YX$Tpp*)ewwhG$Y&@+0V$d!%Kx~>>hlLn~O z+l9*9d;~JYgaZUWJkNc!1D$8py#$dzwtPtM} zN<)r+ME@+%Dwe04ewL0xkfG;N>~p|#Vn;o0$rd@kPz(_%K|JXF3sKK!Z|=+9*FW`6 zpz6>flJYoQskG)=g2Qj@QNK-20~7&wyGbS4W2V(%`|);a6EW0a|Dh-)?D5f>xE0q{ zj^^x6SdFauGP&Uk5qpw1_c0_@wkl@=Ds*UVQ;f!d4cS8c`xi97V5_C@ppLNC1NX1| z-z^{Als5^3QRyl(Zz0ToEL@A`tGrKV_;x|GT%qlRR9Bn{Ocv-8qFZ%(Ax>p&#{ouv zQBc_&gae{5F#oUYFYgLD_ANOk3&q85B<3Gdy;oq?qZsE;FRtZf0*^RtD>(SghC&CR z(g#XDVrG}4c?_M}O-}D$njn{c)gu3NW{_rSeHzGlSBVl67;CGj`}((6CA_?>yT+(=)!@C>S&H#P4Cc$tO}p_|4Z{&=F8mE)Be7D z8gANjH0P`(2mQ2-B7;g_wH!?8UD`pxGn&?zkD2GSW z)%SfiIZptIpz}zoD+TrK6KAR0Yvcyh^V*^D3ZXNh!s55dI*Sx5G zH(9cAU1qI@$zMLVP=`}8w)(*xaO;| z%pPBgtF5BbHxcgWtiFk8IH}!mrb^7W6ki4x8hRC8 zq%|Q`E)1=7Fc|PlI4ZhQ=Dna8hW>-&tSN&TQQC@Os=k@W3smzjL%_HJrg^g z>}*j1@+S3kSC^hF-{?)mzdAXBG*0~;v}m1EH;#nZ<{KUmhKqcs@8Or*voB=$BPiTP zbNHgkgSi}#|Mro4&-8iG9`sO(KOT{>&>w~;rmq`QO4Ds!C4Tz@kzX1Zhp(v}O{Jny zA9?RZOu3<1_UB(J?vOGzCW|9AD@ z&-RF4=ztpqa1p(d==_^NO=*vW!#V$hE>|#@Ih&JOnLDUk)1OGkFDx}?F4Iq&U|QLs zA8Op@@fXA^_^A94%XoK~R6U6>3^2(jBcB_&$#ULTNep(JZY_G+d=JvG~Yf&{D#Y^t33;fPbVZf zi;9&~9kDPl$KlGS;<_%shGq)u-0(3m%#glNu_-6L=dN!ABP4p1GGh>5wa$LR<=H#! zF{bGo6Zl#qP0^5K8K1TYVwd--!8|cXmzJW*1|%&NZn)+idLU8D*s3Xnfar zAZ9Ngi@#3b+}^aia z%5@4}YwWGMv=0i+b7&bDJiG%Xkl#wXSBt9ytV{j1&DwVU2vG`+)glysri0zx)O}d> z(IJ7UG6Z;g2AT>A1&Tse{>toZ&Y&59zL;2L3r4Mz%15}JvUP9ECV)yXluX1Ckg-0~g^XV{O%=rVC?qp#+UP_bd2DkDA z)sC#;0vZS!O^2L^x(uxGW=c33kAhD0kjX&sO~*gX)M@F~2JVzKg4#K%@B8v}!UH_M zHS%2;_B9%9V%&5zL~x?`*=WF#OhV&zQ4|k<`98Gn6_&2qRsgvzL@IvXa(DVl^t$-a zW^B7QS7)4I)_*QGGNS0QSeQV=NLXYao#7e^7 z0S125i?<{Zr##+4fX4a~X~&-%SfI-ck7V}F1jI}&GoxL<)(or?JS;?Wrku3z6z@eZ ztP(KH=W1^W12__og=g~z>)wf+r?G#R_Qi^CsFmtWggoIep4+yW0Ffmn{pSzYCvdw<=K44-4!8w2iWt}+*L1d< zWv#B$LzhP7Yh%<-^(_nuhb==#3EvoTF#&6ur7csDMNLpN5tN1U=nYWkSYt~@8)@| z>)E8F^iSJTMzgLZdcL;%Et`eruwp`h030^oG1I^dnS#xu#Fp7==Qz}MKRbptExkco zp*8*TEqir&;@-tm+3&B~ns-%(&H}3k_iU?cK0Q z$q2)qfbMZVkF+<2#TTc52zta4@kFV4S*g@5_PS7e;l65Z($b&a$7LkdK>yW-fqKKh z*=cN<^jo8BEvA@-gN8h5g7up**mD0}%mlkq_w+d~5))rXC*heHu#Z=|ik=(9 z6GZHC#kZYl&?T>x>a8bqeWs$e7Yjm=o*5;c2ILK+PYZzyX~*YKmI{v^^QYE_d(8bZ zbJd(K4m$2+8#vAIK(l8c)Q2}Tb#zP6j|uy(oDX|Ew53GS{28os*_)LvWCCrH)F6Bi zRgJ8LRFOx;{iRYs)=l32duZ&fr3`4R#rUy#?me9c>+>L!X{tF-31#6-5=kV2#0n`~ zz5wjrJ!);#OC$9Jg5+Cy28gsfT^b=Ow^HetS9yjHIZNHLC|A9z_z4YTMCK1MR~c=y zENq!{=O@7bfI{jt1*XcbLTt-Pz3cDJ;>3`CUhA6oNJ|1(+Z=Nbh5xVYv)3STk0Un3 zqYg<<*LW3P-7B3u$KTJXbcqlTx<&p$MI6sd49YIgO8KC+fws0qj)3j-kPm|_gi){J!=no^?wajCoM>%8K*NF29x7D4QB%;VdiDO~Xj zCaJ257y|*cCW!`mplzyp-r(Zf|F~j8$pIC45J3R?8@e%CSpDpw$|W3T9`?BmXvGp( zzW)e^Nv#v#$T7uB3W5vj@NKWqsoqJ7o3>xQ{5WU-BYgh((*-1bsF+JHcH#g;sHeP@ z99+udV!(Bt2G<6Ty$QDV71#@8vDN^9W8-X4;3CI){{na2o-cZcIwm4T7S z=FGO1fEZQ~_MSp+N^99b7`uCMv&hbrE)@XF!8mrzDqIlYvZv$6n6B0N7{w%W_FfmG-z{s zRxLp@v5u&udo_Y#M}J)ez(w_5Tf;y&L_ZV4pof1#f*KQRgK(y{XkK5#{t5~Abe`qb zG`^KVx|gDVqV*45Dif{Vu++CzeHJ$QvelzG0&?PebErb!{x(k;_f&0oT=UXx`-0ii zfD_o(uG8HmOAbG3q-+L*dL{Nr02nX!oaUfbcnBnriuY03sH?nTBSCuBwZKyIckQTD zS4vUo7x1Q%hP*O__HovMy<7ksdbYSlf{{8df)UGFr&50&kr7W9DBV}dYud7TXHBC0 zS!}-`hr96^>t~v;-G6=(AzJ z;(@=e_mxxyk_^n6>|h4}^(44$ykdGT5a{+V-fnH&(5{Ss>td*`@DujmVm%e#=l?>A z)(;$)LW%+3O4DKX71_*%kA7I8u@+0)J0l(pvavu-Ao`o1`M>V4*3|}@Rs6^9J9T|( zv|=+n-Yl4^rYueuLtN1ZNIQDVMhIVt)k=Gs%gQ1O?+S^P!2(1L5>BWpbo3Jf?nVM+ z7q>FuX*xIecdm6TDOvYdX*k~!F5Ja);?ib)EnqFg%dLBTKA6%!~Fq`Rct_B^P8OU;nhDt#p=25QVylnE;wxGcc{rJbE7n ze=Gf>GQZWoIQLH}xl;H@nrlVDZPu{*#e#^iH{lvXPx%*&8zrjwfB(M@-NP^Mr~b6o z`|kZ*VJ|R-u0DD1Ir`(LU7mD&j%V*+afs|dsa_=yL_Y#_Sgv#5 zjZ;jB7r$WeK2D($)W8E8sRZS-bLv(v(0ADR)Y!a`2fy!p$=!JAM6P@6);WSseR1R# zRlK}wYn@R09e?G$Wf0wahItSymM@bNVH!aMk-D3;VG79o zf`(jA-9e@Ae7a$u{+>|F2!eZ;tSXKv&g3?VO5SQY#sqILkn-n8 z51-5gh|)K|DODbfzFjX21)cTN&uhItFB*Y^B|aRW4KKucpi&TxfsUu`4fg((N!eWABy(KFt=Bz=B;u7#Th_lI!=zYYtXb}D<=0VyO~Rk_Eq`r}mrYC7HzHD1C0jUeb(EsW^v9>2YN8!%QhNgP zR<8wFZ%f0JDkGVIX~x#?Rmy2m4rO5z(GA~%P%*X7!PIg;Zy@K7pmzh}O@(qpbbU3Uev zs7wYRA={NR2DuF!TT56)-ITQTRi;_EF4s8t=62yIpkKWXSQ$;}ngtc<<=nMjxLg&@ z>$Ib;%P25J%h+K1G(8RE-i@CoP-^joysi)LrnLpKea7Pd6}Z_p()x}ibB)?%X++0D zxqg=i)kfo!1d?1LW3Zz_sXbMY<9XTfOCWBcx4ybWhp#%-iB&FQxzFpcdP2;V^V~2} z1#>%-wsc`gucJ&~hUh}wUB%MSKn5E>`BF~4#O7`~2jU_Hh_HB@2=?7uw$%07$t^ImH=VzF&4?)U*MZq)>HCsJ zk*C9M+WL;b*zNbr#HLIz||thomwX! z^KT;b1D-r)if` zqaC^d94RmMWK_#crI&STE`PABWDaGGSdzXNd=x|F7MMgeZJE%1hY#6jLZimzWX4~} zs_^Z>7QLk0fUVGhB-yG}P=`h( zFOuv20zHbBh(zv7ZTRwc6F~BX;TQwc03yZOt=)L0M6LI(HNbCVkXP%)15|oSFwAfDZkmLeKw!B#I8Q zCOL4kEX4(K`Po&}-N2ctH}k99ncJJz?WW&i%N}YwgzTy2wYXxe8O?FY^IQdDSdk@q zw!!G4M^l1p6O=A@YT@9iOwI6`tARdeTInv2Hg4vaipS7?wGtJKPdv#K!kFW-CL6)T z_1n>`A8RuT>WflHEZEa|XtT8R@xoAjF@hXXAj-`QWkcjL%PI;ycA+wSTN}b`#*FTa zM6lC{4pHSJn_GpQBe$ep%i<&LS>-|vRwCH^{Nh0MOR4MuHf=~`pM{b195pmWT zMpq!C4l7Fy3^8sR1UDjjTt6(#CmxaF)b`)wPv}6Rn;ixI$9tcWjhNc6PQjQ1txqrW z3Z%dPUg1Iam@Zv@Oqa*V8C#T6E?6C;{@6#;wQ*EZ<_Kn4((v1np&IX2?|u3ulo*Jj z{Q3|qP7){H)NZB+NbTmt`Ih+2fN=VvCzZrXNPNJx;ZM8Bsr>u{l$7)85uC|CGP~Gx zu43&0ey|d2z4~ri4jV)RPjBM_{xN%ems&;n^e;Uc;)i)FW6U(fO3jKI85lRjY{d4( zcuIMJ`$_NveXsGCAE5J+nj15&b92MbxVLAIlE-jUo1h}*^szj(K-u7gHt#6$Uwl)mH4glTVi3~BwA79sa59C&<`UvS-rYms!Vl5_VD z%1Hk{{1RTjA4cW(*M^(zBN?wzOCo`0W(U``lbaVAbur3FmzB@di-Z-Fv+m7>cNp#`t^>gHoibkfGH;;r?t>ZI4tv3qGOeNYq zrv5mi=#z|w#8I5fMPKs(M|1S)n3fYn1c@#xfDJNX7})NbinlufQ{OL-?L!4!z(z+^ zjYeqQbd!Q8t1(rKCh)w|ju_@RT`6Ipqu|=#;TQHO0u!x{X7Rwcq;@z$hh`=LPGv16Na8f&|Cx4TBQ+`U+*x-Rz=beC5Tc2VQ_ zCbz)L#N*!a?Mjh^4p-~kq5U1_LT9&E#8`Z0KvjtGFaNhXvB$hK^NAB7TUqz?6;BcV z#jfa;5LId-Rl_Yhrnb)HB_*}}ke31mqjUxsTE-7^AC!2Jv#WfdD--zpI}S=n5&j%6v|0{UjFjZbgVu^W zeRF5}k{i9bku)*-K)d81Re}Gj0-+s4q2I_MPtk|_g8&~vZ=Dg>%BEU*J_#^o$X+FX z9ns!Oj;e~flFYq8uDm`23uoC5dQVU%cx5W#l|^QRR#;N6p1%W8es@98nZiPUc~QEy ziuTvnUS+8XdmPjc{w*o^w#(d+G6td)+*n|2)x-=PSG+5KC&^7E?Hlyp{kgauNy~Bl zoC!sfb{!(K{%_x2rL^3}yAZf6paCFXmkN zM8sO&G%wY5`$M*Shcz?~_TRI9XQh&ggIB5&+NiZXHG~Xytw3 zMFzgMb0J{GK@CNS1LDBac7o>Sz&f9(iQ}9jv?oT3T#dxuJ$>d`cy4$qVT^yH3*P;g z_!HvEs6^}C^_IHVWXt+peeWMQzYVvkd4KWDZ)2xm)+Ur#LeH1}es;fNH*HLcc;%kd zC%wRoxK2GvX&ykvYLXg$;d>8r3KF~#DNj97hUddte~sq7s9$Wgq)`|;@U8MTJlOid zmGFI_)~&lSR(!j8FS8dKF0fsnxt0m1Ql=0G&F)lwgpAw5dK?-W=1XqVH;j7xyF!#_~|1;mitaZHwM^hmK~nvVO?u>hZ=xC(STYhaz(6L5YNBSPOY9eUWWs zLo`Y1d!)Bh`GON8`kJ~_sn$e3lo`kX- zht~t9=aE^SxwFY?3X*2B9R(~52&zS-yaWZN)HeO<&!h^^-j-XQ&?Zgt*4(p1_Iv4ysbQ$5$576xOkFOT0%)7s7Jn}%1|afFGPyw z2lsDCc$Wtu&z|MzJ-ph}*%h>`dVGuooUb-CRy+5r7hcl0Tr4yN>g2XXCR|hu<4m-3 zbe0OePbkaiQ|_D~AV)k`s#sGDDMugHG9(QQYErv%ylYpl6efw1|!WC|Teym1?`)H5nxtPd$stWW>Ye|zlPpuJjYy~Sc6hpHG3o>Lo| zCF}ScOwq>8eu=zCr1%acJgYfSeLq67>QKeM+^X7SJDz>`GV^m8aZivA)EI zIdtD{R%d{QuXlLSbH|Eu101j~mav6AdBs0v@!gY9=<(R^G6@J~cs41ur?amNHkW^P zaW`oGQevHZ`M$w5C~F(V&1o?(l#@YH%`~)s19tVMV^%zD0p;FT5=}Yf^v5gi`ZX)6 zE8BM709edA!!V99_?Bp&(=Qwmh_{@{^ckI?ptF?!0*~9VHo^J;?XaO+<$%HKXpH??Z{T;`KNW!I*u~Ciks1E8z(xtHwqtID6NHb ze<%X8j^2d2NER@Za7yMnx&6tQn0H^C8ma67Kut{kbk~0n8Eds&`noq}E~MuAXPn^u zg=g?;Dz|3XU@D&ffxS(QkMa(VPbm@7U28XI6RUS?o|~bp*4mHc6{a%U6d|Te*WuKW4ITRuwA(sRJj!GvAQbE*MzVxd$F}Azb+EONJ=p@K=J&5+!C34_C1>A z%^&m-B^u+OE2yQ%9<~?s-|-P~7iHXvd?LQfoVWOMihephY9@+H$?p#7HV`u%rRT+;+s3rFQ04D+WC%m zjVk^zKK^thDWq_|Kc68+p-$+xIAAR2MXdez{fsH<$uy0~dO~bG=aJn|dJc85?cQJFO6cKOnUGZeSoT78<0bAX6c5YSp z`ma}JkgLDD;SIMceR6vt;z^g7P1YswcXvi35?@C@`P5au7;WPee zf)(yWLVF=jA4U8+^0izk(RhE2w?tD^96sj0oD@I$NIu#Rm38y|50hD8c0jyj3E%DP zj6eCcx1vpZ(yq3T38A2?v$4=Iq;uIFisgRGVmcR0Tg9XS_XUpe+ZtD{Inf+~Z%J?V zmKEz0-ZDi!&wH;NR6f~fZsFV&$HB$Nt?;_AP=Hw#8celTIos$nyRm+PkVC+SX7R7y zl_WWA1ke$bZ1-GTH{8Nbx$u*?dej$d)nFXsVqVtIZ)t*ayfZFZ-j8C1EP%F@Qk&RO zi!K=j_yYbNNGYily#RjcBCPhaQXk;=P-~q$Ko1n@1AA8wR4e60;$>m!E zA(RvW&e2DV5*suZH)#FfyPN?zK>YRP1L3v#m&()ZVbbTG=Ls|h1~<--(U(o4cP#zB ztZCQf?}Dnda?AHVySY$W$yqdc8T~i!4r1vxWkakVB%^YW%5_cVy1~PJh4)XAs&2DR z<)H5H#D24T`9d~4^XM70TSGQW$@ECaR0GJ#{eEQKl{Ztg#Zp${ zK=Xp3suDO=bxz=-Ph7%7(_r$=x?c%X&i+`-3$G2`*@n3kA0Lu0x>~EQ(ZF%gex&_q z(`pdn<}a0#O|bNC&OfSwPE=G)Vt=Ol6Ni&PcUhOY_k@W;&e512W@JeD{=zP8`8QXE z!14aRuA=o8iNJTWZIf3!qG`6J<$BtL_l6WD2CHN^)V!_7!% zK(b^!+ZH(YokoA&J|G>%j4D>tzfpjsFjF;g60zzw#TVAyOt_P=oL396(B5K3{a*UP z*p9yY=N$uR*hzv5K4cSE;Ro%Rm7Kb*V@vXCJx7si@S9-5>^aZq3xyMA#z!|?)j(cyZ<@p#gV5k$`I~{`+Kh( zn&i#>cAaDsgAw=TtL;Ge!#L9!WP`M#$tZ9a<*GZz3w$$$V18E8#YdpX11Qdf-6cb? zI^~mh*#zEH(bhK!UQoIM7cJK9IZi@3e~TGsdOi~}n5sn06$xO#{EJL&Kh*cE(beGr zh$UTuLiT)8WGJ$fGJe3L5YWla^N0Ug%$OS*3xAw4RN(+-Y#P(O%=@TuUC1h=92~)t zuZX0S!7#arbqI`INEfUY#%HDK?o7xSDQ>lEvAGl9wW**_Wzk);D`C{E*A zMZ=53x6NzqKidg?=BgXlDEGy$vEgN;?w~#;b+O-_GJjh*n%fokhjTDZB0O;I{731R zO!lnzlDu5~&l_uvVXXKcaxP23w9{qN{x{PzzCWWdDUd}wGU@Mg$++Q#bQ_)3saN4T z(2C}R@uF<6?YrcVOFGDlFP1)SOKrf2(bv81ID)i_gf^*b1yQNHYXYo>jtQ#gGWu1X zpl=$EuDfh^OGR~jx^(N;0@W;%uNdpAfPh;Pp3^yo+kEHO9u&h#Y>*RL8XvvZb$d0d zx7l!V){*I$^qWW?vZ*`nmR?D8v%~(95w!9zKZuIz5er>zBiTf82f6+DT?s4x1mh18 z--Rjum=v;E)xHjxx}8Xsg7*A?)ex+M_M^}(} z5Y+d^bD`c;b4DxutuRm)k*zF$l8W(3o5t`BERbQq}(ZPQvF7$nM(NiuWz6i`bhOQKhvX5njLhp@ab~` z7ZH~^%eVB(TrFIp?TbZXFgq{<7=XIGtr6YU&i?Ub+A$zKgd&UG82t=4@g)>~VR<8& z*p-0STGKf!bqX*Z#jbMyvE(bAEb(U!f2jb|_o6HZp4nJwZ>Ae*3=`L4u7j4&FK;3p z2UBg`JCS(lVu6K-nn}d!nWdzvK1+d~qMTnc)AG}#`&5xTd14o%g-KNM4!(p!BLvyMeN99`KcnTxA(Iu0fpadXxfnx;3T{0J z2zjs;g<5O79My}zYFv4u5zaUMA(!us3D~vE&ZoW3*L``XXXZy+Iwhn0SGX7X(T^N3 zex>jy(}g#bTsbNWD2BIvA;P&(R>bgC_jYtdpmi>SYUk5I4oUT*!@(9x8+UeP58w9j zp4@^(vpQy_-zPc{F*WEFORxHBgl#RVU#QA*IyMP4%Jsc&f0oJRkdMKw@ zEeUIg%f<%Zrt1^NNGv7&qLo$YSx>DUaB0Ke+T0aXM>k-7m0JY~A=>_FA2tN?xd1T8 z7l|gSDNQ!lAX{SZnE%+)TFPTcT|S}7Wv(ygOI(u!-lZJrkB;op0Lp;w@3JG6o6z7U zBsA6V-!z&Rdd_-Q{@Wh(>ICq9%ZElX6zO>o@-b^cJMQ^6znC)EH{^zvf6fykllJl! z35^WgR}$LSjubxNis~ew(Cy48JOHgz3#uf#>-=}nmK9qFamuyP^@(HWIcwN<_hu_=-c52HPX~*WG756!r(vwpYe>h#x zOQ1hwHLRi65!RS!d+w6>H0HL)uo_RZfbn#kKGerB-%UZVSoT%hVkbDejKwOqiVO7Z zUd+Ag0tpgw9$e<7<7v;**IfH9WnQ^%rc1tkxqtCe0Ian%iqv6v>u~yEoquodwO+}p zP%Jy{tvUY7rl5S=(O<;?uII}DssIy=DHK5TJE`2 z6S8Ll?4hYXh<+3Nzhj!ITQNpP;-==rT$@&kCQky(&&kf;HuStrJ3D5!gmSpv zXp~{$#KW@s_&X6lB|TT{9t3n2D(KuJZV1P&HOwWIg}ekTbN`JQ{|2`iMjGD*&~#OFm$#WU>s=bPZJ@(Ja~livIa_r= z>KEOt$oTte4!njz3%_Z|eI#*Kl9{wy^h_B4&NV4|RfH{Ye;&Q530x{6+1!nMA5Sz~ zteu^w?GUE%S#uinH+^HajBBq^1%io=nFpveA0&A8Y|9-{J&$C_mNQR9rq2A-o~PrGBZv%4Z9oV9|lJD@#fB z2j#lelH-1-YD-lEd6$C@Wf}7U6;XbCWS`i|F!* z!%7}8w_&6M=$4*3`2$cKLTA5Sl5wct6fL2r`TCk>PYR0W;{2*#t%pb8p`*Y6!w5j) z^pWY#^r81J`1J`<6m#?~-W)*VAIolOkT-3_{*;|t?c)tjQ?uiT;Vhs#H_zRNfyT!j z8td%X7sU@h`82c`@)_4lm_DISoJ8a5Ga-)phxN9%sj+7yMwLwKm9nRe#HC5Z63DX_ zUo$=+`Xs~*1=S;+v}FaZHV6Drx*`L%f&81-VcVrd?iAXb1EGGdV2bCkqdNjx+Fj#W z?ev-}V0fe8c1)o`iDgqazZ+u>Ao{*BBlSxRZ!<>E$B2?DhYX~z_-dm@Q_FRCFTz@4 z=>vo*#;@-4Khu(x$~! zKBq3>|I>+lbsI{J(Nw1#wZGk7b^NECW1Htve9E+ZYLS0#aT}VKuu3P`wWd-kkf%XW z?Y%5yoot=u-E6eTVV>69olm4L%w5C6uJ31_TsPhy*=IeUVOf(d6a~iY4po7 zstjb~zbuDmAFJ1CfW3=Ri{IATgZ8#(K#4W8neMg1nN?UL@$g)6DO#2WMWDF+C7)`l zL6k)_Xe=8F$zO4oQ2swJl&w7W&i+)r;4L%JUj%tJLK?rNok73aKzvFgHm{`w;I;e2 zQeV5S*qE8ZiWq$*;!7_2$df$)FWpQ3y5jn$EeSu`>+A^1J1p~~^w}u(tn|=ksMefR z#Xq08VI2@u#bRmq0SRrC@S3t^?b-P1v#SdgEKS;_Ii0%dL|H<4lwY+n@YQtEEQJqi zdo1g9P@;LySFiolrlA@Jr1i9jm7SMw1tI_7wrSkA%CBzX-;WSLICYszLItuT40XUsX!ewfXh+hl3t-88TspFBLqhf^unlYE ziG%(3IhnhDb_n@(bPR!Y1*l)ehM?n{ZBrCQ@7jPju@c4MIY;{b1>}=rUJ}YXK873s zRDI-Epg|^Yox!6st;qJ;V2Wc}Y!J8u@Qu2fHVC|bB}yi3F?+pclC!+^T4zk}Ee?10 zm`M5NAV{la{cLjpq*}9HzA1>AhS+)yIp zmZ8M{hprLFey?9X*fD=0`}YoO_O*dRo0GQ5>voJ4gMZ^=6nDgd+8=u#<%!I&%Ky2l zdQIR({#S)MW3jvAx!kASwXlci8%n2>7H|-}K=tEr=uKwME}pk?QL@B|XSt7M8+_mL zYt2s>l&j5@T`J(r0_ibwsaIi3%F5K46e!1SQ8#5xjF)t?c{ZV4vuZVZ0xM6kJqprr zkGTR#!FPkk4PA`kz8?IboJ(P7223a5s<8EgUiobQFopP(la6Hjc88wUBsf<6F@kR` z>gS=Q*WMNDXBK}ZIO{5Y)mh!>IT>F{`(jN`Ir^+Z;u|QmG9FBYEjck_(Fo$ZuT4`; zln%(jA6Uadjd?X&TtMrGQ=G%hau^04j9u%ESqDTC1CYy==| z4Z1?h$1A#otm`Koe;MXC%zo%tJlk6SYZLKBS0OLY$W#=v&wJjfghLPUKh3K3n6DA# zxeh=;o1&p*2o=?^nb&p^QFq^KW!_8+=z!N?*{nw!-R7z`230|%I2wU*BuhGrD~L8< zJFwuHHO~+}iGFg=*Y1-<{eW*qVe8#l_eG!V7;g?~zlAAgBshFz7U)y>;0*u%&w#PO zKE59Tzu@xlbEpc5ac3HHYw2(UZBzt0JTR4ZmhrAVaw{uu^P+k2YRyE~j82lt6D`R- zqg(7NnE(=(7jmL%L;Ff>{KN$L2I%fTCXZi#Hnk{q7LK(z7~FBCMGQb%J?Ahgo^qdh zr|WUBRQM*pS4Q4709(E8&B9?@5urog_$URI%Pl#(7V2W@YM!TcgL33u@U8vsC1uyx z4g*3NqdHWX<>>7pj0OphJ^A)g?3=!t@g__$D8bOwPs1FNsdSIUZts_quPxN$+c~Fi z6;}KZ@{UrflB>GSO;GpC#JH$bo%NlKrGRVlubMg<=j{=un1v{zT8+yi>CP;bI5D2A5F&pQE!+5i1W zI~U8CR$wjkvnFsc!e`+#$k?8dcC`zpUQs% zbCo@+=gijiIH$cQkk4=4@%c}|FkF%zn>C}3+8<#Z$JYu=q^>9;`1O&|DIc5Uj0Rh&tAADxpn~GS~ zB>9D|RBv%Ud+;jh_oZK-Gqy7*RPd;!as9!5j;%pwyZ^~ht8wv2$cSIgS-Adh%X+U* zaV?k6P1)c;g;VX^wf@H)K1t6C_XKzl z=lJk3TYV4)0QYj^80|H3P;%MeMY+OW7m~2X*ukiq;<~TD{`r{pLYe~Jm`k|)QhaVp zEy~YQleZ)pB(G1NcuoC1g1KM(FC^vC+YF2M7F}mVXGiVO);A3(ATix;aPOgu4}`3n zIT!fq^x@32k{d7dxc%o)NZcqa+p719nYhA5PLTxgku;GQ1Dgj`q{|HSOMZ`h$~gq9 zo8NwnCLj498LVXqgEE!;Fo77-7kDzso0|%tzPd`EF}^}7(xGS_gq$ewr7@$i`x!gX}HMTgT7b%k88#%^|khV|5YP`rw; zXN$t%96o*ZxFdb3>gzowQM7>38COI;`2TUzC=oqg8_ z_d2T{uiTH-e{6j;P|QmiLABK#H0=grNgGs!O;1>-H{moj%itjwqEFSuWf}@d|HS$H2w6W*_GKVkdD{zdZOojXI&+E|I1K{n?n zh^Z;#V$kIDzfTI}z3cED7O3}yPu_vr7Tvw8owjal*(HZ6PZGR4XWdmCr+QM()LaCj zq+u_w2|SU=Y*Lf?p!MH3&!j=cf2@aVLX=YSjt;&dinuV%(CF%nO}M{PP5Hvr1yC0kZi^9s?v$Dx-sC2;dsjMk z0j1nD8spAG+0 zyz5z~30=GBUpy+5^{Cg(>aBe9-P$^?JgJ}qjTjt*o5|aNL|*4r;q0>7yXlCo%DPL* z>4iT-N&v+7%`dn05#edAr>%q5XS$Ow>!q?*!`hB^z7LC1eG-1G3nbLWd)5uhfm8~T zzFoPtirw-i@37dN5Ta_<8m#Io6f8aKyG(+X-Z=MGG`-|BF*-sGl^Si0unfbXnoSJaOnW9%F${ z);5zbWACYV4yxYEcuJwPi~i=xTI%t?myO9bJCTS+id=fu9!%$2gcqm$6cfmXq*A-h zCSGhEGauCV{!%k{G>Z~yt;8-8m@lBSeR}E8IUF%^vg=Y`D|(#h7$L%Xq0ycE>3@Vb z^_OrTnaeP*-M+eF!?q*Y$*b!y8_FU>1%7#B78o@AB};t8zj`=i=-y~de_`)Ee*CSi zwKI7*c{>uFomFZC43_-lTR-I#Q@?JTY7^m3az3w)14j3m#&1;E{bxXcu*30RnT9O) z+;xh2VCU6RyG|W2IrKeEHz9=%;l*PLfMO2JgpJ9!i3HoP)vrgrh+&Qj1XUXMdS|#! zqhQh}8n@by1GBuV?X`e$`nh{lLCv1oKKvjou5D+ZO8A7}u5su;d7k74+S`Smugj~F z`e_gy%HyzQ&)xmN26B~-gerBf8bX;|QvTgHhh6|!M_iLZ(i2-Kj_M!$aI@BD9I#;P z?Jip)A1a9QD{?e=o)*a4Gdh!MWS^{)b|fnC*@IQ^OXWXo3;kBsh4y5v~2L6g`Tn*AZcu>cew^^dDJjU@QAK#Nw%a z^!R!-_-w%wdS18SMLLn6JDiA-AQOVxaIMMu_bf~*v|R*c{^E?46F=0&EvfFX$quC}r!uCQ&KFf}Fom->Qhs0yCjlDB z1zeCaDt#l}eaupWsFXZ~S~dyxA8&+9(*F`Sd%r5kJbhh3q*DBQqHf|a>k&9S)KIy9<;Chps?pipU8AbYOyG6-w)ZWv z&JkAd8D(@t@gpR@C-mC}=*JIb=y1_;*p| zpkH4H^beju=;#M(VUHa*Hg-IheU1dQ%1x|JH}3vQ=*F-udX8%H{1jLut>7R2@=jYm zT|+-N|HDh36XqQXYWN%p1s|}vD#Y6RB%{nt`>X6$b?B}PQ(vYM4QR*m! zAZq&TLg-$#&ENS=O98==0?0^)A0j)jbx{R#Hi?0^)Rsu=2G$Y^WOT*w)t z+|pq*fPB#DH>$Wl<(+d>c;|8JA&9dY(H4YsO=u1vl4Vg6hVs}fUgL66?Gz_LKQb#EX(?gA zrlBMx`G_4zI7SeDN_Z)@y}u0~Pp0^Jcwkd4c?f&#ej)1S2m~aYs82n$Jqz!|4bC)| z*N%?whk$N(Vm$|nhOenvA(wLSJpg&jrR%V0Dt{%aYtO?|v^!VLKMOwA|Uss9bD6a)OE_fbAW40WT~MOY}Hh z%}j2KYKCxqZJvA{;Br{aOBmdwTM{@U195A9ysgm zrEEt)8)hSTJf<9w?>8AzgMwLKFL6Y|AA@OC156%rEDZw@Q83cvv|ELBct5Ujba#(yRxLjE zj-RlZlaWFlfYOCgq0)>J*B|wvJ(l(~Dz?u@JEBx`Fl6XSHIy^U-X5vC9aEfrbP~?| z^qK)eQ#HcT-*rK`XB2w=&v!Oz{g2wf0L$imyA&wT>D*;LpE*+i4(Tw~gXldt&yDhP zoDrzHue-#3vlm%jKR+W~MC>gfvjd1Z@fCMwIaZcC-e5v8B%hBOmxlP>ZyW~){`jZ= z_$p75sR~7|F@o7ksqObOc}lctTr!>PF&Z$8KRFcU>FJ4|08)%lQOesTenOj47ojWE zFv2yjCly6?sQmDgxw&r$ABop)oY1COHD>g4=D(*@c{E<9*$w<^IP=lD-{#Bdh-W{t z^`P)T@&nKhmMNL(LOyX`UI^c5cYYh82Q^72AW~Dr`>|jj?a6j<1~8OMUTH_xV1r;` zOS_N5Ft|bLdF4h!Zk@+GVg*-Md3L3j{SKjMziGDuartTf1wYBju{JJ69($8g zI>>=#p7jcA^AdQx67ojXGc1I=SLi{~L$FV7am$utF?a(9qF0OSEvg{lpdsl*b_yxt z)^2c1g+SsU{EXgI%;$O04qIB}$&u|oV@RLkU%1ZODWAB2Q{%TRllH_HMF+w`i# zKj$T0V8go8<&@$-&7*Q~RP554j>J=#wiRSwmJc+@m1X5Qm?6~%_3W8Sa!mgEd-Cb5 z=ZwT>$L->_4=a6y>JQki(!b_>;}#G3`*Z7)Vc5uscmTRzT8gp)!l3~+*yc$+Lgw?{ zHhNy)hKC|OjInTL)%8C8R!sI>>(4!R1Z0f^LRAE-zNprX%MKbb$D_mW;&8n1*J#f& zPbGkD<7Ce8^n>^e002e%GsUzjj)kx1SC&uZ-W^RiBTo|rxa-m>`gr`)rAEb*&uBPa z4B_f7`;)Wzpvheo52OA_^xm2l8^HBV8KWKW^u2S42ydqVW+ToLOn34c+dK9x4iyek)?;vKLuws3Eg}Y-Fsbg z>JIvEnxQvk@9mY$?=U4~7W9^+MM^VEh+g~p+?1kUZv({f;4{1G+icywJVHpaiL+SD zJ*U@BSi?N|(_o&*LpQ(QtBfreOPy{Qo__f=9ry>j{tL8hEGq|ASze4TX{6kZ(>>bYpe~L(;5Qib@*{0rG&xF<&o;o;I)lwswK}@A&oV} z#GBx4!H9J+%IYCDpSAeU4WqVc;TMAZPSWUg=foIDF?`}TXaIjrFYJc;rB26d#mfC? z>G1QO1XU}}$wzYawch)dEKYaVWSV6Hm@3_yyP$HhbEcH3mpy`~x>zP))oVB&Fa|WZ zj)44%kTk~&o<@#vTYnRHU45;SpCb2JC;O=|HIyPK8LTUCMfVwP9FSCYyp4-u*pAd! zCz_xE*X1Q&M+Fqfgc0l5;4na{2=!ZRK3QjFLio!CrL?G5JKIPCdVq}d5AFqQ!1knj zW;xm5GHmv1P~Ta!%jf26+Mg#L*u5FOC)9g~Q>rz0YJm+7S&z9|DIw6a-E#N0Y9Xfp zf4-CH16dEUn(lmU+P!$B&f}KDo+uvj7c}5}Pr60lvR)b{!3y*j6id|uqx_)Y%`T>3 z44}1$=&Vq_PK6&u4tsmcT@GC%t-->dN?+%>K z#;X{KhcE$_+Lt?lfPdAZ{Ppxar}zqLY}n`%iYiuJ0x4U+xt%#}QEKE-zb7QryQ#ca zNdrj*K^l1XruiV}4I*@|U+o#Po74%p8ps?~if=|iU1@@0D$Qq_5kdE3XlUwJmLN0A zeyt1ZhzaA}n(S_00|*OmDwDCAmAi<`Ehj_>0vY?_H1=oJotjAQ_5_3??d~CHT^L&1 zh>@=Dm?~gycOU%C@X1p+9kKQ{n{ z`q%`LEDF_0_2W z8d8w>ge#ski4Fm9mwKGMoy@d(p0iuo@;LrHi=BMVM*i{WF>q#$-vx)(g#x^*2&q&; z3Wg8=7h!fbfC{p+TamV*G}VLm{216h-`Vd!=A;&=SjmO%A^?N-%NCT!?g5=2j+&O4 zl=#9s*k(=9M3R+V8?zKSh zd-YFx@*KF?WDwiA_=?Pw06)hU8q_q2qDPB!jvqL8;sf_c?V8-t6Tbr@P#C7>gSO$* zpoxhmy?e{&_XU)uKx6)Usw3iJ5t&678l+@?IV|}W{IT!J%e1=sUE+;5#77Ij^ zL?vrBvhZ;x#!1VXydhKJd`I^b<0rltb~p%zE{EbT>0kd0&$htZhaSz= z4aY0-(C)+m7*5aXiTWNoyu<78Wi2g{+nldZt9_XUv0YP;g4BtQM^*dW?74p8AnbYo z(*M3wKbe}KJCUpP_$(gief5TYA_p2BC)37%HIY%(TVd?k=j!bER~|Fm&Bblq&ke(G ze`#ZeX??#6^6)xH*=W*HWxt0mHWS+Xo~$c^#3K#1P?IY6E-5A}XdLQRrVCa7RWa+! zkS9zQtRAg>@VG?IOwJW^l5)V;n?LUMd=&DO-!}#*T41a`bMZ$HPD-#%JeLXgi5*mS z#sqEf;iy#Y;`oP2l==)Tck}oWv8MlaD3Ec+nB%)Gq&I&FF`#}aE=UdG2lUxNCY`^RueS2h`#t4Z$++Qx2h@aeB+`2;< zcxTkYd8@kbH(uT;(n0nFLkb&ORZ35`J~@?a{C9Sp1HdWn)RJdBO#MN9;0!uN-@eCi?@2ChofY z>bh5D&`8ENI_FSYG?iC}faIYTX*_%n4`Jx>9;L=KURn~sT9%tu_PG^~dv;UHnj5s% zf=JeZVI*fg5GqR5hZ@C~NfNYUdp*`LG$j-x59vnb9KopJ%D8ds(}hI`@b!~uu?9_f zbAm|t2m|Hcu=1r0Be7k`X#={I*T3)k&kBr|b?A8G98#``3R=x)c2>bunt0U(j-Hv( zto@C5vb@8i_kO8(jTxs#V2P(5R1%pSCa#>pK3P1Lyeh3waQ zBEKrP7wU4>o8_fvUKxs|Q%!gx9H${Eyso9COZD)W%a&@(V+QaJ2o8y7LeG`(Uxl&n zU~*pC9He?qh`iO1^f;qrsZG#jFl&mh9CJG+99O37_Yh&v`%%l8)LX{>sn&khr~wi|v-s=EWYP6F(xQBBnc{06tyA z@}~I)>ygVN0C|g+k}uC7_pKr4YE(96QF3|xGQ}DJ+0T8Euzu0@?N%x{9`Qlt=$k&N zPafQG7LGwz_pSkOo1;MI?2xJP1mJ|4l3|9gy;eb2ObVl;dV?y&+qTo~ zF*T%vTaUj~h=&osVippdTfK?Iv-8=(mD3d^Oe?cKRAQwEnzR&Ca~947LO8dPhWyvBw4TN&+=(2sq>`f)sB==&671+SBU~4?Z|`n*7>}L z61G3nJrP9dXb-Of=-#GJF;V)+9QTtL`~Sd)j4WT((N`TwP|UGPQ-B%@A4utQ5k!aj zEDx>Br@$#+#2j{Q@@^<(W`k^w2W$1o@OEW?RL&~|+cKSh+hm)Sk zKy;UGnspJ=LULch{i_crb`x;uw8rw1Y=B?ZKT_NKf&839nTjo~l3=`Il*_+yJVx=L z03J4|B)o~`7#Jfuj$h?!I|d>NsR=yrMO6}xyZzyyI~p(-de*#r?hQOo;8WRv$S*ix zUy;53Vx6$PKFX<43cac?D8RQfNC(sr1kCm!z7>Cl3YrZ+pkSb?*b36JlB!80$hyHr z1>B5VruhJyd4en;$#G*J`SLb}f6nApxSmeC{@wW-K3$3RIF}9?h}l2ZDdwj_L%`q{ zyMiD55g8=skO1TQae>Ny<*%QSqCz*)f(gr(BAZ8dnI)J4C`i{!Atx9CO=g!mCgcN% zTzDc6!`((9XIzoChR0RieICaT9DLOI%gk#Qsip;U6A9OBdrMUGiEf!C`)*>OQ^&DS z*Ys2)Ml7B#0@R81k6Y!~Uy_$DOMmqiPWSk6l5r_DuWR(zc7*^P$6z zZ2M$eQ3=1mA2HD1;ph&JVh@E8I=QnK-Sd&X3K8$(3QO-C-TY(7iOzvj9XU}&FgSUiC?%XeG%_Sl=!sgStT)=TB#dG`Bi6vUv5-_cQw#7~v~ zCcf?NOQvkj!twjwBazPG;_W8e=YC+&PCxX#kCfkjHyHq4|0-P=+ne`d?dmuIcLr&H zA$`#VS_Z?PPx66aVql25SPFkHO5wDDq{1;*&X#wC-B9twJdK;s{3qM1F`HLce3NU`s-~ zQ=M@v9NF|0(^E$g6(_tko>@4hhEvw1JWTD$8X(B!Rnhi(bSvPeMcU$y-3=33ElXm-L%L5W);!mhcEcg@5~h1E$ll{iv{|duRn|)x19=cL&Cq3xOw{kZU4R~yO|bxp48wsZnL~MHUErYTncMnDU%ZqOJa9Xx`Y#T#bSR9Dx$lXU)>SW6H4j^YVqDw+K zEk;7w%^_|1EEB;R=at2YlEgyVjtMA|l6pU#DKgzlP<4Nq5AvA^lL}mwqS!!!ll!~o}x@42Z zv8)fgw+T4fPmhecVy{7R-m}+l(#2HFkTY~vYJB^X9JxGd@>&_oqctTg+xC7=LWs>e zkBKN_Q{UURS^EbKCp%w8N|f1h;KKzXN|5rRA^U6)X!ZVI$j%g_wS3m z0TB@;rJSgUv~+JMND3$jNI8*`mJZpbpmaz#qen`ENQ`cnG)N2>Atj6)WAWYRK92i8 z*z1Q~*L6PQaT3C!sBCh9vE zFnhtk{^!6pNSmiHZyWtcAoVR|Bl-2vJ5BHYWICL=08!u^X?c2yM!`W}&c)xPx{M>t zor~Yd{IWV*I7SgqIZa9bj<5mBv*g&8y@pxEs2R0XE{aAi_FGRR4fzCFqUu#EXsIjl7#>zFPceIiXwC?#YNmx>{MM zljqcTjNq#Ia)Wz>Q(2lCPM*Et2|_|ZiI2CU@L{y`1$2jhma0;%*rzx8MZLlMm8~7- zztKDwjrsajF0!#ad-VaW5A=cz<{{Uwy47C!@1Aa;t$n^d{knmNX$0Ugm95t!-@Z#kpHVX9e$bE-rCfB%E1)XvbHD*0*QnH0DiB?8ll&$2%IZhxR{&?I-p%sm*32j zLXc!Hws@Yyp+Tji9`Br6$fq?7o-W2%$ZSYgCGCH-{qq;)wgnRI7AH04N!f@~5EX8+PBezwi(jK9MYAAay)e7z4OEV|U& zl%|I2JD8f{0Q!s|+fObw;7ZGOeo%(yS=bm4vI=sRsDE)pUlLX=&uw&)N;pP>PCN|8 z1^xm8v6q>XFw&S@`;Gtt%0ftO+puC%IeN^iK)U!rS)@`!#=22|Kj7jy`f1vR0hNRADS+IE1)X5(|$X`+%kcZGt`TvJor&!#BN{tj|PtC@PH>nalUl}av zh&$V(BRHxy$Un(~91~33u7did<>|Pe~+|E5nTmVihx6_mW)3}3BHUxFg~-h2D`(|Y|#0vc*e_1Mm!u_ zO8u|HC*7Aj2Y$;tYsHz%pCp(%Wpvkm3l)3dOOp;ln_~|C@F33@oK{aRyaGP%CA^Ks zC{`QwnHV8PvOz$&GmkQ&`v9z`cl2g7Lb<@Z_si^hG1GAon2EhNn!QVa1ea#=`qEL* zWBB&-i)&`qO}8E9&!-!Z{qiwNtr-oYPct&>`YO1V44^_@DA6~Jz!*L&`ah#}Ut|=m zj%9HH@1ky{Gn#q+R7&84Alo8K3$fj7O_ln+ODH4QJUaY_k3s;&T zRx-cOYw(O2f6U-&T1ga(-1?Fvfs!2L(Rixc5xh%JZ8<;_;e0zw0O~ zb4Kc~Z3EI`%7_mQ;18o8ak~-Q|g4Dm8X(w9US~D3(^rOWmen+Mt`zF-> z(E7_4GIdZBqhNcA^&R0VeyzQD5)%%lc zoMqaXQ&=CBG2*fMsQvdd8?$8&l3PDk_txAC+1~NnCcH@WB|ZWHy*M_$;mNo_fH2nNWF(dIZg!<14ro=iWBBui7Voju0kWnO)Pc*m4+Gm@P*$vrRoE{+sLB0ABE=At4qskO%?!0k4K-( z)S73df7%P`n|%nl`H|d5Hnuedah6SUWTAtbDbe*a zSFs0}Sp9y6H)pFjL^fBgpGJ7?T^8DJ8r8b@4vyM;RxDL(IB9bxADo5%tz} zIUhMX(r;Z?#)wTu*2Ol35+TE%_cHFw+?xjo*lg%G7QU`wF9>j7HsOX>*D;FF%v12j zzEU~a$46-yJlFSV)+#E!5Rc{ZnhU*0F!}OoGR>m5pGuMf{%aqMjiC;TEHPYFN_#Et zlt9`4tRRwBz!j@(mo0f6$QNqHMOx=stD+g`eZXvYn||Ko=?mF5bs`wl0evP>skZ8cld)U-EZ$Aa*xfSH=)(;5M^2ObI-uL%_GuAoi zOryw>_c}E9Uhnm~Rd?O6J!cHRa-l&t#mP<=s$nKwNg_ROa!1F)R?9!JTJi=7i$eVS zJcq%mINvjVgTD=UY21|2tK#pQarvw8uV2WmYv2p%OxhLQKdE|z^u5f$cx@RjQowPC zqB_T5H{a!>SI2KobqI)Y0rr0wZg9ZgT^{@au zdlJX@caWB8;2z5AX$QChbvyqlOot8FF9#+lLiPHQ zG;2-M2#Dz;=}spimMF`COmlexwf?lVDeLi*%y|O;YY>_ogA{;fz*#*WpeS&@LFZGr ze>l;W;N?r$Un}1f5V(7Z)i|oDq2nR>GyaYPqn0{wYXO_?9s^LqatF?$_{irLGWd6t z!_aItlD;2pAqoxKa&OA~pZfu}%9|yp!ggYs3UGQ!4^8$qotz0Ey0M8_rr=|)f1U8k}@~4sSEE{eBp)Ziy3Cl`=O~K3EhnW zE2ST%j#`;Z24Ak*!EjP=;8~Ob94_@B#%p`xokEMV$8KBO+$))kelzt)OzV|~^0wjr zJ_AR0gNOataU$1Vx>pqxrmO5*L?#HgtYg0_SShd$7s}{)uO**|)c$wlHp7)Iogpi( z_mY>rYCZNDiE$2{qc_4)ha;{}k8~xTJNrHajp+ySg|jrq(@Lx#(M(gdKd%^GyW;T| z@m6r~lp2&?9{TB#p2>)`RbSS|dH4}wfoG=M!g+aJ&Qc`lXip)+>E)cYBmX=tC@CTS zVUYw~lNl?yOTh6Lu#rK=HtCr$AK-hf%|3gcOuQN7-FK4=`}t6q&De^C$C12B{KvO+ z_)p4T4OHH04srb~oB^&H;U2QzWK-OTzdh!&LH>5o|lkct?y_d zA`HEuE#{SBHZ5v?N5;Zr-UQaLl64aliQYj|G*C4i$uO90LmJuQ@{B;oBt*o(1D|bE zq%U()t8;rO?1V}}l)a_Ik)y0u?4;8Bb_H^}fAf09k01$0ds0^1qs@9%-X2Q4_HUez zIY{zHMX5Vjtk19o+0P%@&r)2agQxgXFysWPivq_0TXEl%ZR_P<;=6kC(rRI;9{%MgdiP&?$)VJPvM^Lr& z0uHCf1bLKzBF~>rlTqEA3IYg z`?~MV5o_F(*h7=B&E@KafO3mt$aHmYDD!(iUxHO)A3F01l>5?Y_X~`jv@ZmuIK2K+ zAihW;S=WHpHU$+WnA;}V|7mDQ1{pO5kGReA?cbAQIM>OfBU)q^yps-u75mDD#s3~C zQV43oJNeg$z=x%RkJeqz1P~@8yl~6*V7RAj;`4~Bl{#efnpav~M6m6ei0kR3x51P) z+AQuDm66h*Ju1HPxi!Rw_UG&8H%0am1!&Ark24qtUWc%(m(M=?hf%_~Hc~;$+z%B7 z>4%M?H53}g>VFnXjy$WFuN$1KSJsXmV;#A+f7E)U@NgJ-SX{GpZ{N+c@;Y3{(0HXA zAtqZe^#54^-Kjj+a?X;q>k`!IVZ1Y*uk6x5>-m;3=l363c-9Hns^6^pj7VkJ?K%%y z4U21Xe;G-mc|s4&c$E0vrzJ3=V(k*lTJIKC`M@n`L)oKJ`HEqg_m8>i5w*tg))3$JFaaHF!fji0F0Vmw1pFP6*jo}_;%79jmem@}DM^!fU z-oml{`SZLM05KPqIZli;Jv!N9lInAxH)ZJuU1Em?*TM6Ix-lnR`M<|!hI>u_4sid- zv1XnpdrCd%U_$05s19?mM=;y!kqRVDj#ep>Fqbe{_77XhlHrb;kLrbO0aH@ zPrKq*yOm3=1U7>7;#qi!IA%1@plImc4Hjyj$L((ExMZza*fSBKKPmGUUP_?;XU;f6 zGl2yaT9#9KK25le`DH)`&(g`4#$UiM>MjG}Vja#+KF5irSUc#KZ>aE#x1`8sMZs`g zfoplJ6Z@hW7Yap&)dy>9Ebo)tuNS_ly^vvVHe1GFrx&iZvbud~4a^UXN}_+cd1+jG z=TfO}-*3q%zTs$ct3E*u1{hm3wzy*SLgWvVc^7ApLUj?dzpbUtXE88PK_aX_THuJY z_OTtX%A)>C%7$EyaE_?i%ba3QrT?`4&_W?GH#?mngBUYtcR?yc-?=$S>=r9| z(NaxeI56yvDLgA&_;;e9pec*Ri(%5;M$iX2k1PxQ7(N%4|_BDrs>UO%6)BI;&7x%#hX~* z#65(ETIU@jay%bRuz{kW?K&z_iHN`+TpAQ0F>AM=XxP0`1)|(-s>bE{Sx{3-`b`$0 zlrUCWLP!;<5{NO1AanuzwqYlK!j+p$2z~lJ z5%wk`Z}K&+X)n>v-{1f5IWytZ|E4|0G@an%s>pbNeJaJ9p*de64K*OIo)5Ab_j8Dv zAFMpcxuPDdvDFc80)mzr-xFO`iPa;sq(KR9j}f4L>d{931=pCF21%%?10jab%?tY} zF$S6G+Scm;^&G3+tvcVj+#H=Qnh;Nc{b3l1Me57lxO9WoZnIlZI;C|fBF#B~#+7~( z=O_2NMW2vN+(lR%EVr*QlVr$+;@YO&<=xABxR&Brq?{SyYzvePTjOY|!%Z)F;3~jr z(d*}pR^Y$?ZWf=J$v_5gJYmVmI2#*-JLjA2re2$d!_SEhzN9^r|C0JHI4s2R*g=C~ zQ9j2he63@9VBUxJy=J>@kPDSpLyZSCBn6z6Z7Crm9u;1Z!TjGZMTL;6Ekv5becACcFAUA1D?egI)o~mp z{i4vZ&;qleoK-MR%#r~RP6frD=U>6GuVvwJsg)cN4~hN7|uR(CrEuGbF4ki=TkE4M#X0_i(e zj2bb32wJ>t!$trb=cYSnf(6lr+v)G2E7_aqo&+>eN+f&Z`{#6)u;M{>qp}5LIa0U7 z?UIBmBj4zUqYpW#uNU52Xp%m+MNyK}tBAriRZ56QR>j!EkyTJ?=c?@J%w+Ifs*3Lf zaw(UrIo-eyeI>YD2rLgrPr{CQ zaVUkSTBuif3Sy6FmyV&QFf~9a+Y?ifRwxn?6Jwm{v%lzp9f~_wp`g}!Qk$K6d z9HrMx${q*p=g>jz_mM z-tye(k~6CH2ptp_z8+^0B+CaqkO)$-pT!iT5%wXd>~HV2x-Rz?1@ zt62{4%g0! z$+MNgI_5JIj_FW=rN)(RzAL77(WE-qLb>~=YJCpDEI9-f@lktH{Z1D`?DQIsE|pfu zCiV)vu^VuQzMjy`u%y=TL)Nomm35%Vs{mwR7?XE(i?hnu2S-Z}97K^1pKA-RY#9_9 z;*%75+)p{o;_Jm>z`XguHIby)>7jPp1ozeME7ax_4L>Uc+1lU7$9z&-`!=Iz_yhet z7}CsyIHcB`2}K%@Lj^4wPx3@M3j1U7zsBl*lDCGiBQ!=3Tm z0y@kgp(h6?+ZRklTb;-Q@@qiM!|${Mj&I1yW;SIgFwuhig|$+M-0!p_#S3oxklU34 z+^{Z!Z5R9*G9s1ETMvU*kN;etxY&^9GoMD^_rDkMV=Sw$Pt|a4vd4V?VStL1h~Z$Y zKI;>A706k*m63u z@!7s~JLYZ+qL{J)(|9GcAN+5!Ii^?Ke!owg;6!;5f9gXwHucplm4({nh^05dWjvf=XA?9<9swO+{oOIMUpf@$QLgGa{T z9@iZR`}q5J*LJ>sIk7hDZG1}RL5xn_*lJ)%O>1#kj_9wMR8LK?!7!v;GWdGG z|B*`o1rqamn?4iUKJQIAKi2^u)0bK%Y`r$G*Q#CnRygtRWB!BWcS1TQuYR+>ZgG3? z$v7+NCRj&=7(SO`R42uV=9+(_&a&Dd=}ixx7aM$=R;@Vol*uIe$><;RAa;D-nr_}} z-4Wqyzeh8YQCZwUTyJrL+DYCxWroH>wxxVtptDlk!}}Jb;k=t~9Dex_EC|duN&Kd8 zSXdmLFJa?$P$f$cx=wZ%qb8gx&jnE=8B@R(ZyHva+NZ2Cf9drjuJ*p$#rffq7k8+E zs5sskx>hQvZS^=Kc>*wJ0>}t^6$ujHSKFinCQ;F6{9f4EAKhn+w;V|&u3_L&$rM8) z0J4fnB<<<*Dq6r-eb|}k4`5g3~TNzn@$@TS9`RArj~3Pw0bqX68PX4=@l z!z%xk)zT)CkSY^o-YydiJ_J0$C++_s2@{rC5N||VtVv(VWg5u|wq%kXPZs=k+&`s5 z1yZO}l}UJgma*|Y-AhgL(5lX?q&Csu4_nQ?ovw4;Tl3Dcm^0L>`&C2I-t))o?JZEx z-icpVfo!)3(;Neze??ZD;w)*rvGiu%wECA7YWIVpL7ic%me(Fy+)kp#B``wGg{*Ww z!ci6pR;_IboynPaYYK`{H%eoeYS36W<@s>BmJT%7?81P)9pCO zi{#rK!rENCaa#ciUaM!5#(&PGyPE91+DsLE9pV_N#$)i|w5FciTJV(7rYjcqxRGE% zj6$CN=xKL)>N7KrM@|>K651vpr=GHn_0bAh5{A=M$Z_(nw#UnTraaHD?M&62+$>p* z<(byJ4t@QC-W(BG2@55EC&fUXS{|>N&d4mqvaMDcd7L}p<$UE0d8IX|RU#WdInc$0 zZ6>9U=G?gZ*dj~9kt?CDX&2YfAPwM@oSv|-k41y5PBptivx+b&7Ve05?8fsyYqm_J zplu~j1)q5a?csNSH1DmU4Sg?ai#Xu0F&d=8Ws5*I4r7G880X(6cfAm^OqZKyZBuk>y*oerF(cJ#EK_1Mcm>2GdYE~(?2lAN3$m?gb z%{)*0NykO}E|XyVO#l;|8fKAj@#0Ym&u^`6^&sEP#s^=8IkcsZfA<(Ma;t>X=S-UY zxtFkG=fe3pL$#I-=SDN*{^P3K@wv{Kp#ApxuP#l~t1I`r(rf*{?UxPHyH^vVKhZF# z(t^&IH;4N##;!)mF_oq|dR&u6J@tqmf?TUT+Px)IFikxWq%zXUq4O_ajHz9FwL^f* zl0Ent{OUSTmg{P*QkI9U9vT~v%P+3&8ZL_e(=@- zZA~0}@>_sF5hmvSa$)nS_W!{;;N4>Fc)Zc2C@a#51(JjcO?96{3R=#R?=h_%A~JW# zNzI)k2iI^g{AC6=eUcw7&M>ITA9F z+U_-x`kt2{FT;Y6zwcu%1M5^Zc4epXVOp5IJhJLcI_ z$yGbBf_0z}7=cvrnRx9_s~l0kXdXJ0JTZIsy)@od;i;#|)pXhP-w!h)m`9PhX0`K= zcW))t{;T0TV#pT~rTDIqQv=Z8CBxq)Nr#$_?gvoE$lun*3|4ZOq11&X0u&&lW z36&pUPeQV-e`SkAIjlchFp6I@G^yUJv1z*-VTNO+ua`N;3JzG^xBQaNE;s$I24g6{ z&%46V5dccK)rf*KXtCB9mK48rzj5!WnN#tIyM!+tVIUSxz#t?4PSEyi+g-d%6W}#rq4S;RGK;dR=(?+_azcmV6bN~ z^!Lt9(V?43JfjCg&AbLZ;+MX-UI3QQ47ULy)fS0;wI@BWSWN}i&$A=)C+Nn$G#X3& zEg%(WEe$Ct% z_jYx*BKZYh?{}K{HBb@qq$JzW(sWARPOB~nm`%DWbH+OnMqwZjyXoq4$o=i#P-d0Y z_B~ZQhnktW!yA}ug}2J`upeHn--{!&PVCiTqpt5ry^<(>C{$=?%8cBtL|Qk$P2y=G zbTJ5GIr^*OH`&#;6K4);GTA5S`}^2MEf3790*SlJ$G7$8ko@iMJ3?U>OfeT@+6eo~fr5iRK4kol_lbHyhZa{VmvE_Ji{-O?|`Uw4Cx5XDKsRYY0gP$;k z&yJw51I0?j;aTL>zv-lMl0sW8Z+s4Fb0nsrZudN^Y}<0gk9eoa+FxZz8XB20dTM$_ zCBFq83r@3V1p2(pKjc%*8s(>R&$oLpE~%eT!RGI2%$q!nr#IYsDfr}?{H#y1$yO~~ zUzY!Ak?OxL=BW!Ii&djMLEC|zXy1*W<|sF2trHV&VeNk2E6(+IsuYZz*-HP5WO5S7 z{%0;ifAmTYXVE6UVj;}{0!_Q~N#!=uO&+FK88se0P}yh`dejb^#8$>kK2Padzw>%# zO<;OcCQ6J#mY#mlryHz4pgnR8ZB1&7Ei_c{33pdpcqEf0vc>=Pu)jVVc!$!V`HQ*y zpuyb3L@r+ILNFhh*Cp3hx#Ve5%`QJ6Y)tJ?-+hxL7WC4bsB-(BZ`$a8AVP?NOyT+A z{288Hchx(93?GA!L%n2IX-oO z8vM0nY6tGBvVqHsDD$o`$w;k$se@|zO7JltR8KARM4W`zKZy?<1`t&iszq=b@V>M{ z$7v0~Sy`CVUM5b6?0(3tZ_-7{p*ZUkQpmdv;V-2zU5~#0aQb;+(yScLHofR8lOUXP zl$J1+e(Fv<_GLyo8CC9IY@Oh&U?Gg=jv^oDpL3KXCy@EE!%NYB@@2pDI3CK3&aGAc zt0qI30{;wy>#%F0@_Xl?4Kn39{r;X3HU1va2vQt}y!fmehji^d*uy!K3&;y(0ahn8 zRevEg=ya1QrCBcy>3?*@=JiwERjGWp0amOh-CXR$ul2R$)eSorxh3PIT38M*jE!H- zB%(w-FZUS!l&q`-d#kpIdQ(Q=w5fa#m4pzovq3D!zkD~FD5vP<3yn*QcSL8?;mvaW z%!0;YM}r%Nde#`fH*>h&uL{FyEm-RM#Wji91NaDrdXTHd{*1gs&J;58G5$+HSWBh= zW_)Y7o!I71TdQv3= zAz9iqAQb->5J;Aq$44GXXY%Fse`TV;uvJvct*p<)9_E1%_xMV#Vex{N4;(&fo}YW5a<7Vhfhl0q>{eZ z0pz%<(t~^+EleA^IF}lAeQFs`mL|u`$PvDpR$K|25Xh5=LEaEz{vtim69=4Y!hO>H z>T3FRCx%s;HeUoWGlH8=ak}B$(iO-*$g7_Vdtp`e%lHSIm(bet#0DkkxQ^LcZ6J;B zj2WBw#G~I+UcIe8?0O<^o-}(q)_$rfI7KzW+FxOv)gRXO6*n%sg+0hQZ7mC!6m9xr z->>A_wjayDAtn-J$)uPO;2MjbX=BR|HC3w4!_ETtD7C#hj62>w-=&fpCbjlwusT_{ zW$R>!uPzf->%3>m^nLk*FTuEa{^Hoxixe%lyW+bvN>HbBcM#0+Qr@iE!g9j0V@Ai| z#-Hc46oaqXIXpesHt#R}c}dZ>d`W%kVu>XcQ{F-bi*u*$TkxHwF?c3P;H~-c?)*=W z{py3bZtMocAHgn&! z>?b3vO3kg7e7f$o;@(F}m+9+D_XMM;j<2R?q?XkKzJEijc)#KtK2Mz9l!ieE+$%+& z928dbo2{E)qhA@n)&4DzVDWIKK@{2zSG!}@j)LD|8&ADb<6KNdz6&gWSucF;@YVCT zBOhsAQIS$_%_LYER4;7Bg@uprsL6V&F4UUwCBoG>+S-q2Ck1F6R+u!5?OdrL6&zrf z4;BWu0;Z|6BdFgH_<5qRIU1}KbH6lG%ACXq6*2~VTwDQFFgZ5=TK^OU_e<9VBoxh_ z12B&UwVUF{LwX#~_74Y{$?*|RpSj4<44Oy{axnMmL34fnI7kx@3j=5@JJ zu-eZOHbF5*@L2Q_BJ$2enMtuhmd^m0*W$7NZl8FDT34z!r$o-^6Jx-5A!#Ht%--k_ zM;6*E&#Yj>MhRW#a1FAuBmAqqeCt5Kg%$0&qZItMeP%`9{B+|0?*MXJrLoTemVCw} zg+o}jHY#sy>Abz)T2m!Vyvn06nak+aRfF{bX|!jovhXo#!GjT1m*vZ(KGnY<38n`Vt{vKiLGCPHZ7HXTo-VQxqtjzH^!P{~MaP*^u(hHr9+j zp&cdU{?+XI%m`5Tdtb^V)cub1{htj*DHWLJdjA1hbp97K4XR$K{C8Ny-A2lGBrTqz z(nIPKrInnWikN50y&gTv^0oPiwKIAPUunFV!#cM#;e6|%!vfht{!(6r;jNhK2REbM z^J=r8xvWRif^o;{UvLID#lrk=n!Q*R=V|47{|jqRt0iMG>Oc5~W8wHIY zROmK|mr-rJ2@{FS%zbj9+z)y5&upl~K=HEOD655kf6lsDx$3% z0grNvY`q@2Ty<#6v1Nz~`0i|aQR%|F z;&5+aSL*k6c^|qgXR1_-Ypmb8v%vq#`Jrh!Q^}eI+i8Do)|t+Au}W4o90fFhokyA9 z(AI$&mfRq3A2a2BA<4dOHZ&Zj1_YDe|UROrI&TiCyJI{)f`Gm6< zqSOYAhhQS;quGy@Kocs<`KB~sfNK^#=z}cs6qdyG7xVxq5{&HYsj75t^u81LO#V+r zeKv29w)?olYwrGx7;wXz)g=dcRolE_<7(rcgB(Akz|QmD8s{I)cm@DD{NEu*Zxuwh zrHCm#294VqTJvk@0|%c|@->R!S#mW)_;~Zu#FpHqDER>h>(r;ie5QE02$I(7Rt1IQ|&vZpDmlI6P=gU6iUA!g-1Pi*7w8 z^KkM`lUr1;LlY+_mof0)5~hr7lGdmF^P7gxMY%imXq$u6pRGU4qKmuh);iZ!>U{9C)KrSuq+*l{M$50-Bjx8D{Pfe^g;hC^u!Jriuh$4ZHs6?o@beHk9@8}PULV~5oVbvvP z`zPmb1(*8D!!s-_7OP=5x52fiCv=rKR0;BIALmfj4`+6JfO~nQ`;e;ui`phAU5@<) zRRRle61gNW?g94sKYchKakd)wTtZ1$b}tiH|4fH;QGNq|_u>!9wGVgwx*Zo8UPIo# zMD5>$?iM#{sn?I^8uuP7{qs`Ycu^k%IfA%@+r0<1$ZDa}k>CQ+%#C?_%&eQ29o9m{ zfxb97pJU90lg5BaUKzM^=}K;V)L%72S!yOFnF47c(M`_aWb#q_HiT&}-cK&lIg71D zu+4!S#T*F;R9+HTyz-9Jy}E~ss`ej?#+V}F3hWj@6y5w)!6s-47wfRhbL@aP-|pCl zA0wz8q_{HZ%XU*KH?cR-VGHFybaF90%8T4c*-kRO#CKS-OGY~_XOCBi38cj6DZZkJ zYG`uY2&%NEh4d~WIrUu=7c&}ZZIJXhaVSwdJPlRDBSG9=ugEoFQv^DzUH=Z0Y< zyvpkr`>jE^@-yjPBPxfHsWqdNBf)E)p11zovTHLzMQ(M4E?uuv475>S;feuNF34T- zDf0ZK+G!KY6={BSRp*gY0gDz?=VLbrm!~ZuLAbH-!W)w}-u#6|Wqj2lnJsf5QE)vi z@rV^nQl*8f90tsM)y|ThuSy?)%alCn#$~MnGF6Y7whRsB4U+A|;q8!`tBXCVCa3A0piTqhihKTm!!T$!I4bz}q4=f#EE0<{AwQXt86pQX@J^o4 z)wePS?P{F7BD?G@^qU5GuZ!YGM!@kmG>^=Vy~%Nu#Uz}k;$(3WV`S&I)ZR@0{cSMe z!L;5jWLZD#!(jAf-YXe&=F2k>HiIs!eoG5jdiARn>?^8G~Wm5iw=F$4Tti%K*Z0i$5`2hwCD zvA;@I$H3-1zkAjs)EdmaM2ktyky@5fBCT!uzGE|=yd;wd>%LMXuH|hwy%Q-Q{Q5R* z!g|t%X)_EDQfLkCzud!>BTr7Bs^U?_96yUF(htt(AA-+Kosjat;cYQ_;Zy2rAPI!Q zDMN!oAj@ZGpY8D_Sw6~!gkLWi{yIs5kr3dFBg|EcUS#vudEe%bIeWP(zW ziSJUETt*3ni5M+`niut8T>;Ox44VFe0QTMqq&}l%yJ^7v*`lRd3;7eL(@B9 z*g4wUE(n8Pr*=13O&M?dJ9SW%pIDK-zj<1&t)|*d8sj8yW}qtaz1m z`_&}2_pZv|?2A+BfviEVyPiPxBdY#FcQ)H&um+??O4GgODsLYhkUxm`a4WXwE-csj z3J?C|LJpTxI|ghIQX=KBAt_8BeZa?ZV_qmO>K+*geb7!*ym~n}=nc89bdNO>_<@y( zz@$Q49S@>cV^+_ZRvP_LLaG zu}`wbL5N+X6J+g~3Nh?7f?4VUNm5t``JTmiJIOA=xQMP0Tvbo|A+iSVhoy0Qj8i-$ zXG%jg@lAcc^e#@xt(SYMLi>gD9>skiS7HfJu7iBWhLk`KJ!bYbwi1 z{V1&kip?0WI~)VPl!DsIo-B|F;A7>oSl z6G+~}m2gYz1KVIZGoU++tZBS3%8_@IQ#c*A_CN+CPPXoJ*tEa)i-vLz`qIh@`!Qs% zUT7y|56_)mti6+`%+{y)l1z~IanpcueS2CSS%m2&xxs_fs)(rqD(=A_5WR};hi5}u z6x~wKgmDv*QReVRkqoFz^D#C0zP>FK01JzZTS0slcfpap2_mOb;w9$CuB^42fjXhA z;ve3bM>T%=_anT5bMB#TLx4=fd}|Fn_KWA}F@xb(%cqh^rKz2zd-DdL6P^VcC9-)5 zYH$8ah?!lvjB$6b_~;)3Oi!!#$WY(4NnYoB@*+ZTIEQKL{ARj0t*Ime% zJ8qbpCr@i{-tRxK^gzW3Xk35;nfr<~zqUZTXMuZ-F*tsMC=(c0iD1^-InXt;^0~&( zq)F5)3v6@gkh@vnwQYopfjw%gf?X;7+PM4+%RR*P%vUHWySBOMyw2f%zCc{_VkC&t z#c_5eFn^B#ua^#jPb99;kwU_}v7i1<=G5)$xA==?9O)VzK#9U+Gz0tsAq+fPd4Ed7>bhEv4$FXQB7d&Sa1hB8Q;7pD9&ll(M6$Y*2{xVLKzw)4;_s~HXZ%%$teG!1!@*^Q3w5( z6;&E3bsO=~?FUlXw$o^PZ|h0-`n&%LKqyMyC}?Dlp2JaKUv2>}7h4u53-N5*NH%)t z1;2)rz@;w4JVTIH^8gn1AKV z@@h+L@-5YhGdjXPCKRU$kks#P^6wnByob1+U(5Qm<0hJdBiT5I7H=1w;?+2*Tl^ntUV~FPrXnp^M+|hmj zukU)@EZFN!dSvk_nssa(Ap9v5_U*ut0Lvi%UbI|z31F*B@bnxq{6Gu24pMgI$0tGP z@LLj!(%!i+EMU&mK0;%+j;az?#^kfCj4kJJ!KcTpmt>eeT#Q5V`AzQe6nB)69-JkE zBHz(13ob@gEf^8*zpY^7ZQF}frL~xtt{@F}?f89w|Ee{exS1#9=FSWc`}Usc-~*D@^$bF{?Le&@EtYu<0V zvmm1&S?B|8?Wxu>ry_f%ZPuovY05JDII7$LvAp6ryRr--@`)DH_kRZ|1d4+r?N#Ytx z=T9M{evP)eXPjj08hghpoT_FQ)rlP6^K4^ zII}wEH!Q{^Qb_svy%h_K^Md(ef$}Hr)zgy6cE$6FBLiLSAgTe2CC>K&u}58=S8J=? zC5KDn&@!sbddjmtL$Bf6ccN6?pa(^S5vmsu|pma-i5SW$kv{k?uBzn|Hg*wPo}Ovav9Z~hn=1uZXn zyn_ERs*_XopSHU=+9*w|Ujoi=Bk6?8)6bVfRy$RKNNj9 zKWEv2;RhY*DKM*m6-4q=jH!*wozaut%~wW;j5ihliB5g>*+b?*UpIp0n@&ofpH&0b zo6a-z?hBw+N5o{CUhM>0%m@r9u!9*-Dz|MlK z>dhi~=7%_X$6%+g#K&`I`vIHoZr?>5h=-|k9Hl*}Kc-)t1Bht1{Au)vUJyTT@SS1| zniA?Z2%2Afb=k$qQ0BG;cMWR5y}`^xfYbP{wRNldn2*}OpEu(jeyBI)beZ+2Evl+P z8p^^ID3z(P>D$O1joTU`7+9}bO=p&__|KII29#-l3Qe`LaRvh+ zD^sclO+^-B0uDqVndK-4}53NnffPbb+D}P>>;JSx{+~TeRx@&h`Q$$)~XX z_xK}gNA1e6$!Iu=|Tmi$tW5Vq3S5jZb+!jr|u)up@hJjC;rUz3EK1$)|7K@BT z*xm@#DCT5InSiaeJGW;6Yo|8DUbBj~uJFGxWDwu>s>oPSFDbn$_%b<;<;U|$1!+4% zu1sBtP@e)uG;4wXi`_y>^&0^eZb>tG&k&F}VM0R1HAqWlReSX6Sg72`T!D}AKMAPh_q z2dcXnKIeZ##0}M+Vuka@5qwP0c~*gj2%4oyfzhw974FXb440lJ?I*e8tEfB*@=8HZ zQ9J&dvsBVcSi(c$&qb%f^(vv&%CK)p9f)f7Yqh9Q&T5b zV{*q}*12f9BFV9*8btr0a<_}ltoYddBwp(REFW{|PC!nh z>og3y!9n~2G!dplXsO=P8@A4MMMO~ubJz)Af8|}I^lQT-S9FHe#iU5=Axbt($>oEg zHZOUs{rdJurKGu{TPe5ZWS_C}Qe4$vaF;U`%3N;?&P&Q*N4|{|0gI2Rk301oET8{E zwZvZTEWE|$!-){|V6SyhoWJeFvKi2!sUw*->~}%kc8$Kj=A; z5p&eNQ)vs&Q6f6*3e*P>W)bA|Eea(0h+lWBs&0&uGg!B`aLx9`O8Sr#kU;VXIMXpG zh{z~6Z-XXoiod~e^c(=HS_u?L_u%>j{|BIigx_q-GGtP;@jtlYOg%0JQX>hyZI@g# z4#&))b1QK84KfLL6b0qxz7_aJa`sJ{<<7$r_r1VjOqRms;yG0m`wvRT@&+;Uy*F_I zd<)5UgSIi)3$IJxVScRLtCzypH>^rdI^3%#ab8=}Vtrdi=SY3!n)7w(^pY)mE*MH_ zH#%wzxbQJsX=|)y$yI{M$K)XSXln!o#brm05gD_5TZi404jSV2`ud9&mP$v*InTRLb5ukq@+JhYI5=8>%{0-hT)DR*WSUy8+*@*MrhXl{6*q1iDUK8s_XKeO;)jRd zAJ1Q$!{HqE=YHS!eO<3heBML!LMAV*^92{2W+7q~bT~m=lbqUXW5U{?1Z*km<+AHC z5m;6@T}(v7|0~Ko+DWKDuA*o6Jl1@n6wxi`dZQAZ7Wv^X%)?+431FkzrHh)j63-jl zb@lfZvTq{RJHUqG$t70>iU^?|4)?Xj5H!cv2(2Pe)B0F@3pAurbHHy#<7 zp&|A5v=dPCp^tCoM{G~D;dw1#5fKAm%&wxI`cm-74K;WCZ_SEnYS)Si?2hTQ(xN44 z8?k2eZ@1%AkhD-%JA13@7uujTXkQ6QA@BETQQ_DV{{K$(o08!#)m$FHkZ=_`)m>j@ z#fg4GPD@pB#hz#VXR|i9U`SjWkZ2`oVDG#yPe%?n9tX5NtDT3Pj3J2zLK)(<4lHX@ zYp(qSF6i6B$O<)@;b_QTyD-wui9tf?I_pv?M`33;(v!8z#QjZgdGtFb&Pyiu{Nz!2 zM0QCxF%Tj%HC;izHfmPMvUs0xka-t|__}mE;?*;BF|F>;U=+Acapl!bpDIY(SA@5j zPQY*|TUoL5k0guR)OAeH2z(~0ze}WJM^)hFs)vJFq zHQ(gVkuwzjA=#+=4u#8XN=Njh1FYoZVG^V}{eUn9+vq^c2g>8Ssk~g^w zl#h(Tlu5G+G7;g0yZ-ji+NY{+&J!^H78~@y#uvO0T;~55L~HkypMU4+v!^Q_!$ot|>qHvPf z5$KK#!i`7Nr^PuFoyHb8>B&BY1Cx5RDgribUo1Z#cBx%Sc0DCsnw{Smeqa2-+51ib zy1xet0Z2@eK^?a+k`1OyoT)SR<3nnMEE!HQ%Zs&wx{y@DX{{WPeEZ$|B2yT;eb~31o&C*Dk8qO*AU3F_}?MixRu&~0iX1( zglTGfS^;}|@R3^@lT>->*chZvdCTexjYo-XC}lbr=lC+Rv3}{8h)`Y?ay=Ch2>A4= zUHNkCY2S3@ zmbl!%R8m$UI$aJP^FmwGRy0fgU6EbxM!iyAUT*xCDI!%Da!zpamyHYKKN#SK&-jw9i#0>gPmHRq09&>PXL=eH zk6zhxez3Nx78Ow1h1l7r?~uv|E89< z(%Q{jE?0yiH&!nuHCL*90WZ3WL3UR&Js%TI+NfT=P`tp$?0g=j!du>E*t)M+P2@j^B&P;74M`Hs)2Y7TAFw2pk zBa$MHwXg18zf0RUIJ7;XO$W01>HK=&O#4@u!JmKag4E}84|2KP855(<{JB#iCtMVVftt!{5v%{^L%be&BFn)>i5(5HpXb92lz7FWa0UDq8@EXdz zV-6_z(YNJ&Si14ohhJmsuBbkC{H!p~5cFd>|VRCwIDf7CjU1-70A>WP?9Xp)aCl(r!<#3d}6Z zAzL<8HKceeae^(f?lvw8;kQwin zDhG$t!EkbU*X5ssvd3dYJlXn|ARxaNvcrWhdO}E;0B3;rJHt=!48o2a7><|MJ-i|9INRy*?o5Q03QK z`g1q;q|oBg?nE&tVrZC@>PfCEu`_1}ci)6g(0&a_R_c*IC{jEBU@m>#>w5-Wp-KbkqbbQGZO6b*lce>7XQ3R!!BSxZ>c@T3e6VHv@Mkdx@$ zD6U&-wx@%lh29-My=($iw$_3h4CDHX_#$4#Jhz_|@$|b?+mMhn+VOK^-|0jZ0VB{< zAkra=AeZLFPS?NzFp{uixP0~VuMK??Y%^%Dp7KjHW6|#6p+eImEv_0e#|%&0y{~e? zf)l7E5WjrWn&slGH-xr?d_)ecYXl}FuJl!J%HgVes#dx=vg3M!n8g;I1tO_($-O(% z1VBK@3L_qrh6p%s-+KFH?{&%F67Hlj8*^xK>{vu* znZ4=P#VdAM-g9Yo^L|H43fbPtCw5sw!_jM*LAYtV%gOE~O&yG$mk&Oy7bd%*={dgAsS@?q9V9A!_rEOBWb6Qhd-=TG)*&&%VW^ zy;wbv!}I>b$s#%=TP+B97_WnmL_Tg;=z-3e1PHVWPgyBCYI(thHI^S?D>fqXdsnXmy&0rYfmx$rU zjy&(286Z2k)ZW2glylv{v$_BlkVG5W-ijb21;@Pc1>O)b78s1h6p)T=A%tHQ zKtpU*^4?_Y-Hg?9x>5ew2U<_uSE!5GxGkuMshbNJV4B|#P%b!`-me=%; z9Om3$4L?h($C&$o5-r<*MvE8pk9^#C+$mV6Cxc`M8$H$9luT+R{*i%sj#i|A;YiwH zwS*$A=qt$4(r!8n!O2GB@B=<5|4oilbN@KlnG=)T7|-<7Y1lGsL~%_&1I~lUNOeKk z1%y7iP%F_}+cAV({?f&se+bRmv zWR7*-983Ufoa3x%joK>SQ~DXOl~An@zGbI3e7M)ro*X+{XaUBd^=($HJ_0YR_njg& zXjfWHsGp&%;cs9f#oV~2$S+BaUmOmMUCsM6KQ5wO!nZ|h(gFJOqC@2YLeXH_ zWum-qcQ2*?dWx(ll`{Q05?n3 zQ9E09J;9rNHJytk2+~M57xi~jt0uGT)*2S#ze10K(C-XGmb5!sgBSAvG3pD|tfL6Y*!~jdh9SfrZg+wIH7E#f7z#K@g%$Q zqbJKNqoI+r0+lsKywpAwhdIlP_)mLwHwKw`0<6I7Z?L$1Q3+S%oK>dAASoX{+@!5{-hW6u zf5%fB9v#{6LE&Z@z&Y^x2Pbwz>8`vZmG87<+HYXI&=vA8wU}ZwgrJO` z+85T4Llgp!I&(bpwVp-}DLrZpRVeeyj}>%0k|9+8dq#SN#%8Q*)z9kvXAJ#SW0;yo z!eRy4!sXNnpu~60$VS>SpOnYqcLkIIZ@Ksl zF#S*+N8I~Ie+$6(fq%(z7NcETf0!(gG{LCMnN*%5j@}nk2frCU)ehJ&^I)kQoI6{t#?;>nn&; z1s~z32u+zAO^3HQis5GE2wUChY=hhE_T!V`>E=L^qKq=ZX>lY6kT|u8^^SACa%?Oor@ug(c)F~710&5Rk5T|01^4Hs8auGGUg8jFRzx-G_HNa^^ zxOT8qqWk<~ZRa0ZVf@&PeuMjppQW{zZ-*(@AkwSvaaVr1ZEpYi_Vy;^2|M{=H&F+7qErXXSOVa*lLH%c$Y%=|w zG}u?M2-?Cdy#}uc#J)YAu0SATR+$&?4NeG@O0_92H$AYz|H2GqG=0Z;x>9v9a%v-k zPZ;p7P_l{9KF6g<&iU1u`&6=(|H)yy){pWY44bKA4_xWONdFa>zjHqNtu*bOZPeos zJrE^AM9se7^+Y>T z_*JmSJ)QI9>2Y!IpjTUc@{16^xwMf{so)+)-{}#a4}OqYV(nmA(^D z@g0s0bEKXHn6}-#N06(zGUir4Fwb!1xw1_G$s6M;g69EF82cn~!F%ONv|#G|IYmI)lfZC(Mbb@K|0bXYxnx?eK8U1gytOZEoX zke)r?5U*U9oF4xG`_c^w{ag+&6UmqTYJdMM*yw<5)KKM|_m;^iInQ;(!YTX9 z6*q)FT=$)!t~^xfsN1{;56NE7lv11ZJ_y#6ZSz&u3>RD6{88>Ef}O7P+LUj88lZfO zGmryqgLCIi-WuTE?EbT=iGNgn>sI+vZ~-)sk<*2qaf^JB{xs^WDgMeGl50{!=c|U1 zj7|YQB>AJuoI}a(c0LfgxyZzfbefUIm#7n=l`;WgFO6=!^!#i8<^KG})fj^({X^6|JT;ijr)9Jle}Va&{c?!jc;mH%lwJ!e50#JV zr78wIJ~Oq3T|j9bqrqC;kXg+;aswoJImwpCP z7N!2j;GYlYLsc!qOTnZ9w!K%ITj!4$pwJtQ>umkpu;2N_Y5X;R7BG;N!$xV>r}>cX zKO}9~6apOJXtfrS7%(-xi6zy=?)hDN(-TY0N*^a`g19c?2sjfL=MR~JhypcgAOVZ# zf}f*}(c{2?S<*LPA90O#=IC%^4Z}_2jbFm1Q9_aOx;=Y@Jsb|ZL9u>ZibiB2f3)AT z+=g<#+?BWRBJ#iGV^EGH$WJB#@F7SI*$KGLYfC^pZVX&;y`ml?KOF1A`svc@jVJWa{U2h$-?)uT%lV7Mf$@KPYJ&KoOG7$RV?aL(;W41&v|pd&Wn_k;3uB0 zCJp1H@euc>i{SIH(~L%}FxyNNrtj5H9)vliIKnPmWiN1{+28;2ftz2aHwL!)5fRP%(kKbw|ouVL^qZcqV8v!T%}sVRJMNdu0`y`o%~Nu16mWrmSJV z;~GWeJfRNM2E{-@{aNEXn>W4ZgqQi9X$TNTAV(@iW>lpzr3%y-z_6M^bzJ9PyeOfdqQ@odLhMQi^7OV1$BXx zgi!|66J#b5-a#=}k?W(%9egCZeo-Jw4#*~sl~|<=zXoG&-YT{>$!LYarsHUfFQiUi z>>jre@Gl}cy~y1f-1CPO6&?cLrHnh}Fy_a5R{mY~r`T09yprqckL;C((Zjf@x@xr| zBF8bs_Z>7nwb2+;9P8wtFGbUBX}4~Aak>E4!U6RD^_Rsx`b2k^zkxU}Kd11oVeEjX<*I(6_C7iG) zDrI+az^X1_<9nZrE@qTstAweZRQ9bI3K*mVgdc> zE~oh4__m;$oGNA?^XyC~vK!)%?KRNcT%?1MrA7*|jA~@O?+DJa%k9`rBh`t$_H7Hd zPG&vKPFD~dY~4A$5W4fxF*>Otwp9oqB)uurd7y-L6Y4wWea;KvF$+_tDG@W%iU`L; zzBiwPoWC(V>N4JhP=o$_lBe+^dqQv9MfN-hA}$?zRbQKI)@~0rDjFW`3;MqLTWRl2 z2IZNw&f2iK7U1y>o@L^evvoM53}xNA@pDGfRn$1?UIl4mYcGgxsges3L^FkcpZ0m| zlEY2-$Xi45G9)A@9?C`a-xaN)R+cQ;i2-Ki#rF7a8ABHh_`3ajNtiNF1B#?A3<&7D zgzX1fFM_|r{|g{YW9-ZUXI#+%8Q0-0so7K`2G>3<8^M-RrN z^A9H<9!6zEI|1UCbrAAue9+bd=E$gvmIzXKK1*Kl-L_CSOVnAy__6D!91%sfBiJqx zeAw!HN+{%$ibj}F5DG23l`7lHDX!+eer;94@VgG)Y1d)TnNe?n;k2gxn7~J;BsQ9X z@Nxn8(Q&E5+rYz_t+Lqrs|Ewur#T8OE?WXVY3WT>`tHlp^_CnTG&mGL zg^hiLr78IC>eD}z`q<`JV`~7k;eJpOG=vf^vPxv#a5IZ`VL zN|#lHI!fA&`dgbw@5967;Bus7S>j%Yg^N}5`%y=y-#hkcq#EZQy9lfY5yZV>Y7sOf z0Z$RL^6Qg?2Tah@9g-@^or2yKC-a!m@B%KPD{0HU?jvwm6+HibH;)l7t45P(_HNJZ zzHIi!K@e=vX}wTCwhxh7#XcNvj+$%PlC+FJJfG|oCu3jx8WV+9!hMTkJ0+NB?H&+XeLu2bKuj5^ zXRQK^p{mT$yZ%#^y{a za+qm;g<~c~-2IIA5$~xb!lV$IGu1fH^v6#*Qn1r4CQk>>Mlt4b3!^T!o&Aw5_N(Jc zZ}Eqp63xXNcc!t=znCF|lAJ_gbk^e<`3f)>ZC} z;?u^ecomZ`%V+Dy*whM4`B)|`cV;rrWid;PuK$?+TzbVgV`7M3jlIh(@}9{Q_h|aX z^WeQ#W}33a(wp^RWh<>M7K0e-_uuPY_X|vH1W>0DTBJ$D%UxE41ltyW_} zjcok&sXytfk4oiX`q2r$*E&G+{bM*uDLWreHR`9mzX_bLym^74%X{qsMq1au+;)1s z0lI28UYO@?+K}9$28Uf`ZT$*4YnEVKY9Oq?fA-ucJPIzG*#{mA?M?-Z-e3$4Duz2m z%L6+>AyLaEB^j=kY+|Glqo{Fi-zd|~LOSil(&^Pw5usx5%}Wa$6Sqo%HXC@WNZs5q zp5^0zrDqSn0XJ2&=qGwr3AMfRIJ)Vi#wQZ1N@8B9I(liztR;;)ua=_bS(lF z1xW^k0IB=0PAHK$o&z47!3&bv)D1Ye*EYEyVr1{|Ic9?$s3!Z0!D;nwtpMKVtb*eQ zBS1X-el{tr22GA6ZgA$xJ+io)e(k&r3N-t8c>8&mSAN$@!oM8r8l3(Wv)rGG+DT7+ zvPge@bBe3^pc;1EqqhkRnYRzf1K+B;ir6^L%@|Vu1l(4$OcsKR;1T2V@1juo^l;G2 z>^vhj9yXUig$K2~T>Rp+2)^hK>+ANSfF1^nlM#so_dR7P68!QV>`6U)hZ0LxP;)v= zi$ey~tyF09zaCpifHCu3+)9garn-sQ0sSuI0WIw7iQb(9qw7pxW8f|V57{X>@)nxE z#q>H&`_)?OIly#w*xb6UT>+BHlf%!jEZL$?bBFSxe<2gep4G9Rqk9nQ5qCmG6Vs>; zIg3HSZ33Geh18_{R4DOl+YFXoC}SKMg) zczhm_l_5~2xW@-+T3U+SZ6XL4(7LFq#Dj!#JNeDzKM#;u#wVD6Y+=yXy z1JOJNhYOIDiBVUT{S+_-*drzCH4SAb?44zsf0vP!?S&t(QuUqhkJs;Pg)g$7vZpvx zUyCM*){4UV9n9!sT@T)E8q@Jl!`^s5{y-G6k`OhK^-P^W=Mix@B>?N}J$$Lt(3c(Fy|vV=HUU zqa_W6nv8~ZS?PXk76SHoZu_wW?0D0uC_xxKj@0^ZU8Wxkp^{FJO-s@X%pZzcj&0jE zV$oqwLVcCDnD9)ri#t(!~ir-7*gfqU?&rdX?H`%(I+x;8-#_V>_?COG4x)k$wLPP+tv#c0>Jk*;ffr7!`Q46Hva z-K+eJY;aB($<+<_2g4712^ppVb=$rV6&LobD7hFxhe!@bVCZ3Pu=nX#>C|7UQi_2} z-vNFf0*Iq>a)uRG@``eBYC+Wbm`)%Qb|2EJ%EjiR8`8~yEJfPKtofMh1?U=bq)uoQ z`jO%E;cOK7_=g!kY*h=I$Nve+hU zubsDh&I3~tdU9A^`)8nikm^jbh@At|s4C7ucU#i7!auC;y{10*>8mo{>P-J8ykBRi zN7-6ak6(>yp#M>K2IrJ5D)l1LTeP`(G8)Hs*=k7Y3jLTVPA_!6-tdixAkw~(quKeu zl?)w!!rc&!uxrCjKIezAN|N@d%h;h?%S%bCgVkT^nCrK_e@H)mCWEeVXet&ku!&?k zV=q@WWt|`@>s^2rJA|eMNPvnDcka|wr(f-{THzPLDf>da> zjE9HjSFtsF>#nN5Lt9Jy9y)&023`W{zf%b4`-c_4)HOiUiis*%kqV=M?30GG> z>)(cL7|;$%fr%G5EPupXO7FEhlBQlfRHZwL+Za!wvkzcVYU2~5iPc`yc>B{?S(%-` zbPrbP`$=;g;e7ltClvBxxBTpmGppc%Yh0aVnM>5Q4Dhf;l&`;vt|Aw0EF4ocb)$P= zmlI5o!PG&FN|&lPa_Nv0)u7(#K9U@p`dRa2;2#%g4a+;SZ`KxC9=j>SNz)$hvFk>{ zhRd$hSMj+Co>1&1PS?I|E_K2usRie$x<86?0gTpx6tg+9)1pjIM0{W}+V$J@{sb5f z$5wVjH98L4TuK_drrCKPjb79bZsJkh7QDN}cEVCMF9$D?x63A%iV3FC6EBA==_3My z;>DUUCC+XYA?OMBA~u-X{o8>mq%{U{`p}P`WOa){Qy%Arq)|CwcWSi_*=n=2HY>hI z17G(l(+S{dj-at8jK=R*Zo>Qjbrxlvv!h~^*oi5=#b1%CZEDmc!}IE-`EKr5#9u>W zoW^M-@5N(!A*JID~#v2>k!LsYH2CD`q zp8BdS1a;=Zc?)BLeiA~lm0((TQ?P-Bd zevV!ZJbpy--(FQWyQ7EXK*$h}6K!K)i-HD^)@s;W#-9vb{3;xoBG$T41g19|JeB_>LA&p9h7cz6AdsNg6y)etJqBr<&hPuRG24y?tK`#tqtDc>juWbje z>#Sl==F9etK|v@a)`tjkq+)vXX)in7KL!Bt!h9~xS`YXTYndKP(vxWVO}u7=lAU99 zxU)Bs$z|B6BYTPJ8KVzXv5yS(j%bw=$sgBp+?ZE^xDzfcg|AIfSElUwrz!@V z2qiDFENFh?G*v$&5snwK&pX{(In}ii($c2#hO^>Q{ zsZ~>ww{@K=RG$-8qPY?u@~rQyjV{kEO^*KB+VnQ^h+a&I1Z*lGy7TOxGyh#AB%H#7hGxRDKdGm7+v6Da*t2STF9o%OHD_j`=Y%^i%QWp!kc#BDG zWTfxAFeyfF|9GrNiG>g9STu{E{S3$hYR?g!_yXps}9fFrI&ll3HuL zk7*kclo9U(GmAiRsHFAX)rKVqg#p}BjdIv8oq$hM|Jp1AaJ?2WCcY)#}hlzFnZZMJILR2og)HyqpDIWQ#t<#AlEigs2&$zYFq66G(I( z8(P^`7rs>OVseP9UIFVNE|Jz{b%3X5qZ8O)BMb{@4!HAxT*2RnB726|f6-N;RZ&aZ zx_odCR<$gP3<3OfUec}56ch?ka~<(OIf1D)Ed4cs;luvpocC-+&t|GBNi`x__tvh_ z92vf!?%VxWOOjoPumu!I+q#2TZrLnkbK4jFa7fg7s0#J2D12*}8?jiS(Q^pGL4ZH^ zLoEQQR?;)SbhV=!XSESEHIJ31r*N)}wDmlTzsODnA7`m(_yTnO#McthgEZNf`Pk3w zzh{*MS2^9V5vEDN9I5V>adHNKvj!0_amC$sMQd0oAX=*+^rs6Y1uoF4T6tlF{}+ki ztYS-XD~yhQCohhqP@=Y6&5~mJ73D0Q59L@-?O?-sl?!2|aM}jVA5}yBBQsod-GB<} zMt)7NBF;BI$!oYtA=I_PPWRw@Dab>zN>KVE`o-Xj?XN#YK66`VMDMDN79Fgu$TDw1 zy0WTgWms#k&;DJgu?08Sx1Srl}ml$3Tttzavtf#uBkN z1(=(KoL0T4n{$*j(VLZwgix0?B%%Ga$Ob&-$(SpTSx6@1TWTV}PCU*EM^Yh> zgKZ0!=KJTrl^=TtFU-0p-2R{57sDE~kF~@nfO--u50hL!dvEKq^ola~e?j_C$&f+p z_xU%VU$413{@5CMp{`#PwmKpBD-1haCIV-u!taU&_9!a=E}Feey%@MON^i7hT; zt(qj$VDGTLuqY3|MDiO5C$@SIUqPO{J>VuvUNu=Ov}h1yRoSaXj`j737?L4`&a;Gs zfkI)VH1Kbjs#7H0cI(kGPDJ7Xz(xOw(7f7=C4m)k15pzzcdlR$zmmrguIh{J2ch2F z`$Lp8X<}IlXUW%1=`4VvI~LLK$qJZHc9QLYZT5fT?Qj~ij}w%!kO#a+Z~k$Y1b#aX z9mbs~;XJF^LHB)ST0c;|7%bh_AXw{P1GY)A_eQ!zSl@%5L|vkOqOv1ih3sd_E3iCm z9!8poU1hCTk)NeDojgGYjdTjm4?>JPjnz&UU4srUBmB|uuMvtW+k^wUp`~oF3!z5q z?{*pc^Y6Ee4BS}|Xxp5#0>)rB5Zn{{6=b@g0}joI+M~WuP^J37Sb&}7y+N{>4k_|# z0@=53m^?k0dJM7S1r$xj0U0haC zkJtR}A`inzyrHO6{Bi5u(2r3^_==NP|M-OTzG$3vD+nQ3b~HRZa8&Z>0DXNW8g>CW z?|FUJd8^6y%-@nnhS#dUfUF~5zRLVY8U;Ju;mvRCthVmP-gmxhRUx?H>T3_y0@Y86 zSI84D5{8ko_6JgQE&hQt^n>MJ(GT{0cg;TAcP88a%zo#0=Bav*pJKfKN$}ai71uhJ z)Nd<{h~VqD_^pdMMI6qnm^dFCa|7IGTJP*+T#@(_<`vt&+4&J6dymdn!M%K3^)shj z?@0v9DXB!WSjKeU!z!db^4h)ivzDkO5mKW2%&V6&cXgK_1fI^P9diGXy5OzpAG(mE zOu8yuDkf39bGmVC(Ffh}yE>p!4aDpaWP4+Yy;q7>OTJtWd`QHca1CJsq#9!@E_hir6!tMRUSqDt(|u+I zs#lVlpV-B##f|&ae#s9bqeUz5b)W092n>dIYHhv0*|!$~sqiHJo=r)$6I*hr+K*q# z=YA9)WUib$x}o+eG2N^}FS3$U>GAAIW%mX>uOPaRm6ww?CL*zYfMK|WivV$Q8_wK3mST5w&atV2zMwU{HdJH$Grx2 zYcR0CDP=n3Zivchz6;PH(8qcW2H=S0%Q9kK1zG3c3=Kle&Rt^&*l^zLV|p|@%<*xv zaH|}cG1LD5&jk=Cz2WaJC@KEgYDHUqh?WzwQa4obI`8lX7DS91r~7r#y$}^9K38A_ zgWStQkbBSuw28g>!2M~1pjFRb9I{J%m%#}Y1va7B z3h4UG9sh*G22|$V|80b~oD>Hkj+e-MpI%eX7E~NJ6Yy!2n|6nqDX^CB9k27@d*7)A zo}W^fbiFe9*Y08jFb364`undg8vdSXwkADU^ z3ab+UvhxsxrIaBH?^hP*l({Ya$pv=q#$k+{mo9x>_?jCj3Nd{E9j2iMti0wGhI7ix z!b4Dx+}DI+DTDo`uZg8S~XfH+rjyl;#ZF z75{CVVU?{!`(-1Kv#TX@rO}_8E=l?hN>tnaVfQb9Z%^L?HvqBnBW2){KMW$3qgNj- zJ;{^kdH`RP9LZ9m&A7(rO`3gV4T=}CdAfcF_VZp}<>9OSUTT&sEmx4Hotlh{_y{mS zqfG1&xn{qQg3Idv%QO7zZs_s%qM;vWg7DyYm~!TBH~3I7zhUPSL)u>!nAm^+Bt4e< z#F%+v);YA@7?J7q#(|GaE1rs%4L~k)qSKaC9L+8FPrtprkU_SaTUC0|F48gMskG4; zC(z$0ZWJGL`khJ5eBo<+9bTJDU|AI@D+?{&v~GiA?Uw%R{s?PKFmv5J87Xsa2qGIl zw)DW=Vm-LV6_9x#$(Hu=RNnH(M%;r2)g=xdjjxECeHZyO&xTHhn&Y2MrOHTOd>D|U z+06%iJG$sOj5ic}pfJu-bV3pw8U`5@-glO={4_(64m^mHEa=hFf6nLFby7P3O|`uG za}=w?Kqjsom)DXkB?!h{9(yFN9TVM)FF?cW!>!ohd8r}MANsTn%7WVg=Dj2;b9E|X z9n$+nH5c9;`%>zm938w1^$ec@&Pm8I z4DE!Dp`_GPU>MPA6MN@7v}>u(Jq%v0A~Wb zNV)-E_SN3t)UkDJw*f(t`VnR#Emo#}_a^GLc_xXjpnKLa?qEck2;6{&uR_{nMF{2? zP4R09cDvRETmnE(XLb|P#7}13xlZtKQV4Qdd;GX;9K6a)R7Ps4JI3xGpUqJ78xL1q zqPv^iue)zXCB)92G)OZ1M{wDGjHt-+c=W-gv)$CfH4uG8UYhHx3Gml(xAK*^t1H*0U`}a-*e5Si3sR)KE9IeGeYE9KP`IuC=~ALiaqse z@FK+D1^JC%!^iDd*AERhwHzU1-#8D+4V2MG3wd=5v^>yBtyzl+F1%C50QeV7xASH= z6Wn;c=;GMA(s>Gs(n4CioC8jk40<@i*g0VaAx0d--fv}PXZ9+<$;qL1t;;JFkNnU& zbQ9`ldupC*_DNZF>p`808a-KhJteeX+b z&!sXy|N6d;BImms1>Q+kIn!l!vVw-#y6}wod#22rO1(7LX?qX3z;&VIQ0o9#=cJGNcjY zIRC5EPfZ*}o^`oR(=1XJYOCoZwU{O+xbvzW;n7K zx!^Y#A1;6RP7%WU%TD(D$4|Lm9nQ6VSvOuaE@5pjIOicaE=#gX{7#R)YJH#IzW=Km zrO9P1f=~P%=nsZAkofI zN`a1-Lghb$_xQ{jizjFpfGo!D*inyCpqm<=9?n{^=C(I(ojUI{mp-7hHA!fdOl;bQ zUP7@uWRDY^v_~>yhkHejOEOooGWeh^J?4)22$nDV3P}@&hc8h92iMxCCLkXe?G=A` z=4d~6SC5Yq5m9_ay&t75x;VZLaZ|S8_OJGX7dReL>(CSN-xnFs8-3EehCS#| z(s)^>EGJ&F$)s|N5gT*x{9yUh#p%!4&YCNYnJn!dnEh4Bw>RMLoe05kZx0-;gp-gA z4>V1}I>+oPxvYNT(H$<(dqFRE%{zsxu65h!1I@<IKnHj?Y2IL-;#?xaY0B zMvE5*5|F0VZ#VW>`MZrk%8X4U)VVtkk#DlSN3Y4htD!8Lqa897;Jj44!oeFTIu>Vq z74Xvqwk@7j+rJMbFUzP1ofeVlGO9Q6#Pw&P-|jpo@m))UN0UDO)c^5P^ibWvmRP~? z9WJGv!AsltT-^;PuR%}AF;XhZo4`K~yK1D^3_AjYW@mqEdq`!YtVpTpOxtib8 zK?1t@|3&RB=A zt5k<5toagztOsS`(V{~Pvu9QY|GLG7Ac!Fq@QJa-g=W|6pp_R~E;WSn6D27~i1Kbi z-JFlp25|)S>9j4}B?tJ{Ae~MjsW5o^4ZVbr4p5RU3n{N`qW@q5=&_~)8iDHK<@t9Kv(2uw^>&?FgP*QE?}ctCK35zmib$(8 zOn`KYbPnAqAQCb((wzcBcMUl+9A^IWd)9j1pBLxsb=JN2zVGY$TqyVK>>0#iE8W^# z?Yy<83sip=4G)l_6_NT&qfk;?+7E--55&DgvGe%6qSkKADM}P^;(x}u9zFlDK%V!s zr@o}Lp9X#-ot%|0Garp?zjDl@P$}Cb9?KcGiKISu=2}-gPDpi9T){yY1*WI)b?TKJ zzH+be5V{Xb!5tQ31{pig#Va8|Uip3XhX!6`Ng*wR>XAKE5&z5lI7ZEU`uxv68Y^Q7 zKx$S00opZ6^DKrcKf0uR|D9IrRA2dhypAb_>~`-niAZm)vF+Y9{rEqJOZ3rHtiwoC zI(pL==$uX7v1u4H3t6`1O}%nPD4ACX?JMls`mJx$)hX5ouu#xd-Q}$i_eOWtRqcjl z4F&n$@cIy(5cJ&eM=ymgVlpAN(mBm0>E8Ofj0atiVR-(!6vx~*7q1(!eOyeCB;{)p z*%ML)9Yd3gIR^P9l$c+P>OulT{5`Jg{5F4IZUVHw(_|Aav-!qo(XOsf^ymhc_0}Et z`+xIxI5~yjb_GBSW2l-kUD<{GTr!0I6Yvw^A^rO0*G9U18ughRKJiSM09k}k_-tsc{N|LZ4-6%q=7(*nmP5p{ z#T{ZCIKdqX?Wlu|+Nb5=#~CA z?SLJ7cPDDUuHT`96ydaje>O`yjrGt7wj*f4IIr+R!XUk6n(s5(BfGx<)@uiMiS7JX zQa1HYqNzj|g+k{?6Jxm>A4L#(kVoO9$HLMzx1;5-v7?Fy5pw>>C36Y1g)rZZq+RF} z0hw@csTB>B$%jw}n8QE-#=4t0N= zz63$|0R>bJ(T?W9FgJ2wzVT^4X6JZ`w}z&$!;JKR4(YI+NVy00E5}g-XaY2gMeNqs zH%yODyn5a}FBb4?y(?`>Aj`ss>nF14?{NQ zedsY*NF#4xAW2qO$>tekh8CRG(G`PyRL+36(Y=vEIjR~Cr2CQ zgrLAwh&Yzb9X}d6SQh=MZP^yX;GGU^Al6*_YCmrKvNgF5`1IFVmLP+N&J9x@ZOb(6 zE{9uG$YQ)cAC^wV`bvvoD2lmydbe1)rCgR zC4MuwW{&!G=iyTJ{Uyw*Np9wM{SBuSbXQi_JMAr~-^ub+v)Sw#X6X>_>`DDQliLRJ+WqK-880od=|2T*nJcEZrus`1@_iDs&_%q#rG~KPxHNlGxBB zLDH<5u+bn&`CKY5Gr({|{m>3F8!1S?;5VmA3Xfw!2195-g!3gpMv+42RnX*K2JV4n zxo5j+^l)(Z8HEyy^mu4K`0GU3^@0k%6l*BI>pLM!_gDY>#T4onD7ki<&SmxrTd6#M zpZg72c*PsN9qEL~AZK6?(ag`bqd)y3>^w}GsYNQ2550WS(yngy;iQ5=bRgu!(>bfY zIdE1>} z29k%bY&P`;eZoDy7+vaS$!8;HKL`9zvdRBvf%^+rzXK;W);rxoQKN-f{LPt|$~MEa zPW3^C;P4{@WXsi)NG8_sq(f{c*Ym=Nk43@{yPMzIJ|oFUTd`f$_V%>-FVGj?70LdD z?)M9Swe#$}^YZK%UyAZJyt)c!#FAQXqMgm+vH3fE0uw(MjQ>tr)zh)PK+@H9%}=$* zO2PGW4!Am83Wpg?_LktBmCS?x=p+v8&rbxd*6{u+y;{hEHe`-XA1N#on0VLRrz`J9 z_*B^R^Az_lLdyS*33@zJ!02T?-Z)W@Z^J_#wNa>A*~@-Oi>Y=xVlW(%x{mR5>j)u> zNf$CNJ1tF<9E>m-kD!>!EZwpzT(j`b`aHuLi()WE5id_1S(h7o%x_5&PC65&& zH61zWxf3Occ}P{nUD4wEJg{5p83@>5N4Ep!*&U+m_S8iH`X=+W>j4{yAf#W8S~r#E zaq8;ks#t2ECE;`(qJ{s37TU zgd252FkEU&nvQCXG{STHNL$HvFG6k^g?pu?Be1><6+cSX^duCnW4Lm8!xfll`;Pmj}mJrDbYvT&E9G!)+;$| zgHJSeCUGvoM`WcL+XZ3A0v|eig-$4#jhhJ+#IAX}M+K9&xvG;rsu!==%y-RAqpeNy!vDJ84slH z25EdTyc_YgC;QoQ9pSJ!S^`v`dpOn5-SF?=7sGoh+?S*ZFaOP;D=YYm)Np*;$h4d+ zY*h|YiW9qhj%#0n%X?*VSdsioq#8SXi-Bz{iUTDWthy(K#yw@&va@U9vT1pc?77>? zF!6cFX-GN1@|A&{zl78u&wI#$(6pLw8KpVX!;9gxKfPsf9+t%Ir57tM`lXu_+7hpc z+3D|@ARXC1QqiVW_ylwCjLwV4y1MX#vx{7}^xl#~WP`l6|2dbRG9zOnSrWWyb5gr)+qMSY0rh4M^E@L1Q-%sLcGrRB)^ zHF=~TeUiYN?jDOyo4+&*J~^&yM-Rc}^*R5Pr%wUIx)ppMu=C&=#USExJQnk1y6@JH zyT?hQ`?#)X*w={(6Z54FXn*@-*pwwQD^R>)i&u`G^ptD@(Y**w;4D(sqK0tPDGvfa zUQ=@8{oe?ONs@2+*dKJ2oRtn1o-*rXkf#%Ve+HVZ_%@%NC4nA&QV#)H21qFAp*uX4 zh=$e4NP#SK*j<=MDP}cMboX>H4Y`$l#76nz5$`*84fp+%w7_8pypq@DtsOa}S?ssT zQlyUZZYumTZNZkC{>|IhKI@lMy+R7Buyvx^A2uaDEwd{hvV?i1Z|25dtZ7ZuUJK^< z7KC69d}{x90r@8Fr+QV@QZD18r6)#E`kuu9&jMHx3pq~E!`j~~b=r7TJ`5WaH2#Kv zqvOr^;P?J?71-E?`fsM6L$>Tj=GyH8k!OTKt7Q13Go|zO7cg9Lw1Po@zR~CvOVXVX z)2y=6)+JFegXw$o=qBr@E92Od@5Ph7Oe<99;)UO-n1+XB=&T&HxYllUCM2xmRm? zHU-hSW)Y+6Dh@2Lt(TcE&-Om#eG$x!N@^$7NEAzgb6u-Sw(9 zkPM@`O_N(pbCnF>q%trZ#uYXc`#QuP<0S>ZseI_kG(Mg7(z3k}B<6I0wRtCW{*Kg$j!t707_ zn;q{;`~5!N@v*oTH9)FW zdpI`O=5S>qnhxnjP9YoZy2>)V49Z=(UT-z)`CTAXu4vW2a0E#f$zj?HDw1(QUQ7RO zq*KK&n8SBdUsM$g?X5a{Q#^i{zx)MG1C6ygjBd(Wq%yh=gqE;F)-mf@Zz>(D1NqD4;+P2yw4Nt zf8LA_VQ{FEL2pLUjQd+nhoRgBNcp|RONtJ9tl(hKi*<+LlUG^&D;3j3ABbKUBRzNKir@WW- z{7wZrza35Ht(ejiVtv||$O5wAP`<34PakJ@Zf;SzOiUP)&LL@)WSiQ8H(9mRbY@3) z?0lFh`Rm9w0r-Q-Pc3WiN~v4;>LP|%vu0B{6fl1Uum@ME`{n23mZS0Kuc7<&CgNkO zt?VwZhS?fO|snFER>qQ3ISy6^NmHC%CWi`Fx_GL;KkW~nXJi9yPnmb3#5(BC*TWFyNvy7UxKiD@atz91rM|TulEwQXO9Mp&E)M??dF+F1I&C z5y0gI@6nH$F8LMFcNvtQnN@>6B&r#{J0g@d=74GH zOMys^w>=72nFMs}2D9gD@*pov^~>cwiLBj3ng}UZXE$2gWHD=3QmcHAY2uh%ZqM}P zRR)c!Cmp+N0f()sb#8A&Y)nmXDyv`JywhroG6{9 z#4hD3aNpF;ZtF+=Tz%QWvcH@;XHc=Epjv8Ll#i)^o#K+~p~`S~#&NuRM&c+n>Y89I zi{;N#Dp4BRx_ZuBiX4`|B?OHE;OAYQ7#1DI<|K=bxHdbOcbXN!YVC30CLP83p)XA( z>eb>EYKyP-Fw4tdE^@`pbgG^0-9<&(a*u+B-|Qru9WuR7{@yY*BiE~_jyFcSq#j%oN$S@hvp#)X1rQvyay+c zO6d!Ym&Mgi&lMyyv$~UdjG;oP4mGIDh|SJhZ8UAU-Yg4HIaVmW+y*M)ij0l0gY8L-B^Qd>>iyvtYHYic z;@Z~Ts#CTP$z<##yv^)!}2^RWFD&>#^LR?nL?sR+PcVHU~41i z??MT}zn!)Y8!#5NgDR7k1Hx>w-@RUwz+4=?w?)t*YX{zBq|sX9q26sJ(tKWGZ$h~} zAoiM@J_t6`Ymf1Hg`c`6Q7??NP-s2V8^U}Zdy|&5kf*QyRF*2ktm-InO3K0~@uc*- zH!y`c1Yf9<_@cZH%!N)D+n~U%b~l_rh7Tm23|l)M5qSp9hP& zRbwl-US_ohZlJmkDsLZ${Oa8 zqn8KKK{hMU{f%;u10SVDcGQ}S+epQ1MeWX|yPDo3*b?-_H0AV#Gs4wqRYV-yp56J& zOMN2lqL2FRn=d1Q-H1brqQu~^rd`CUqw|dQfla9l@{b(is~^WTU&fWwJ;U)loeIX3 zm#@1Bkx`_%W9LM4sN13S55S+fgY0a|F+2$s7&~SgwcFq`KA$w{vT02IS-Yx9eW6@E z`mvyKXp!UQSK0ThE^SLH)@Qe2G>+2_S4Sd?qS>l*U)=b&W%c&wn^AAZM?KXX2QPMT zpvE|tH!Qo;Id&fdy;VZwzi8W+AG$P+-A_DkHyzq_Nbwn8=E{*;+BR;Xfc>MvtUgq_ zf`tb&8*8%By`|wxu6G~Y{r)AhLo_W?>PDr&<)=p%ZWFeAI`@P{6du++t&S*o@O)YA zo7*kO8zfB}N!bg}x!)7vn)RU7W8O(_3rRG03E;cDcW|iI(0bmh>=;7v`UxLw_%$)} zMsIn&uLI7!?=crPTGmy^4aBo4NbtET)-0S3UW2Aqo4COz(I*CL+Ym-XP&l13=t;vB zMG=J)SD;{P>G9QV&-YitVHfm9|=pa^APZaiI0d0y9%w#!rG44#&`W(&cw zavjd08erQgcDgrjX>ae|QFX;A2bm+xfomq@nX9Bt*O= zw3-j`FD9Jror8(v2?KaKc0Q^Ikf9Y{)~2o zFO(&9V4?M(qCxp%)~&pn{GGq?P7wxXcC>Md#$1j^{5ZpCVANJjMl!puS@erKqAxi##S`;_XW+`YHqgD-hXro%G|cDXvk z)=z$JV5GphrHdX|!%5@B)zD_nXb8_9q1bingrl=y1@hp24b}018`(wz?I*Jh5xpllCMdl60`#;7;PW{6+Nj~`OIfxqQ$uKBzsZPh)nnRcIj1lBFy4|RTP$X)!mx25^$&JM~6{c4XMl{Yz5Bv9q#6wf#1ZkM9*qC)#UWc`IhhSnP&uHC*4y2VH!Wx4{1tS1NORL-M2yPYmcTfYAHEfAth#S`>>6mVt1JiB)j30dR&#$odeLQ~O z+V4=UpS-^fa+)ce)Z}_MD*oyA{bi? zE?~+S=ZjV5*Dyduo7LciNZGnu6@oR!8PekviaC#omDSfF#j7`=W?VfXsCr6fMXSb0 zuc@ain5HLG*S9ipJJ)Az|ML%}8|QmF`$CO+4YO=$c)4e;`pnTYh;HaTabhm>u%%uD z`L6c+x&PZQ3)#b)IfX!KIodIFXf&8BVFjDRFGP$~{O-~uDtHTYnTnGKhcaVwWd`}= zC|+}+5S3qY88jl6t3NP4?|Z0iAu~gp+5pXsNTv z#FNDU{5H?hXT2)mvA;Q3;sXZ=x>qVupPz4krTp55l%D%?4qrGXn9RE$OL|*NE1w$j zT{cbu|FZwFrhV=2w*K@y7ZE=_DIp6H0K3@jgH-eAoh;Gu~oPYo!+FqDagj#I;Hf-m@V^Tj|qA z%$xsF**~9OXRuZtKgGs>k=7>eHw`so6jx~Z8LyxTGhQZdzG5}FoZCibh;lCcc(G;p zS$OcVbT8&3!d50_5C^@td~E|XzV4iWH)$Hb^%+l=xN#0iAgd^#l!{N(6E`0h(Q>$m zJs#Km>W;Uc#2ty_wN;%xF``g>S^?%Ln5#^Mz4_wrl=7;6wAZ0=biCZgmM2-O;Iho@f4|YA4th?s34up66{%4le-0q!>OWn}hnbfuh{oQEer`C@ap)P>mQ{FTecwZ4sulW{l!B6{OC@#cj+NMle1U{5b*&Hq89t~d+nYh)yM{=ZUUppxTserGgP zEhh9QZc1g49!1b_nfzQt*{nc$bG-8J+fp8`EVphb$kYnz+7sqt{Tc z_PT_$?2DJda!EbHyJ&96;wq^(JNmlmBylG=Ge&)1RVg{rQA0F;sG! z>CG6>*zGr@f^8?c%sfL=a&Yn?&;Pm47a%^~FcMP}xXcZP*&WrjW5^Kp^|D`tZ{|*C zbHkVqUT3P!WD)XZk8^`FgT*;-^gqPLr&R7}Q3BRh`5|GLjmXnUBG=OTG2V-G_ucN@ zbUvc;S<(in&wfaG4@VZEricuuBZquc-%ZUwVtX|ZA2ju`izTq({xPl5B6{Rfp0V&A z*HL5yxvpL3i-x^m*4QHEA3l>@;4-uS5*)Q!-g(1mK0L#i3VZ_3EF zJB2nnyLU5r<&K~C#e=f4``K|V$dSpe*$WGESQamlm}bz1$@1bm4byVgjo{+=yz z+lROopjUfM_9LNg-FRhyrZ!EE();)z+nlX=17j)KY!t*W?Rxwt<5Ex&jwM@F(_NZJ z=a5|Vm`j35GxF0AK1Cy7{VqhHHn)(p~e^HKR|afq&wEt97Iu9>h&rFzIJ@3l&Dcx1hCGC&jP-d}uXccZn2 zt^jKl+I5tAPp4XO3T0(xTMo|^l(ec6(Yxui&|AhnVY+Hs`O!(dGwCSv?2k5WTn$f46Y%A82O_923OUln8I?MUK8GI=` z&FbP=TTO%Gd8q1bpH@`b9Xj4*vF}mIW)wf_yoc&OwEi&gUjSA8aQ7`;@vT}+>WXorY4q@HBYu>9-L%Z2<9ve?Vac$cMET`Z$%s7O$x+!Zf@L`EqqI zLj=~4C<}0z7$~=*@91S3BW9Qaau^zunv@A+>7;hl6veJ+`cILiyWX;ZHt7h2YYurs zbQ|G-e~|9N{FdtU1}+EzB=%;(?cB9NOawz%7c}K-nbpM=+i2>g*$~}cY%K4RI`@-$ zteLUP%Vymy9d*eZ@vJ|r+lMMQ5M8Ko8EXYPN^DPT`F(C`MRy0Az@{#G(bjh{QiWYJ zuKWILTU-juW6IaAeA224vq38K5&-AJgO|inB~mtI2{IK7h{H4(Bm1pL|6%c9w}gMBJqK~LAlrG}Cr z{AiCT{6vkDr9Q)Z(Pe#bZDBSEneN<<0uL-$63d%r?aoa}3tL)Z<;_Q07u(A+0#1I& zzBHsW2swp$oAXFwi0+Hfc^fKS{yyS6+mXo|*VZ<3XZz)McH3Ys=ajn-gT6Cn{iCxU z4_2c&C_sE;7RXa- zK76Ln7a66Lf9r`W5D?PD+>vhI!#54!fsq40^Y)VqxG^CQ*>@|u7bqgQxrN=l$vQ;v z1cw_}NI!Hk;qil!g+Gamm+((3!5_JQZ;1?s%&b>*(jlvshp{*t_N=V_Yl2riQ;L zfQJvIua|t#@c*jk^zuxH#y1LL2V?*g6uVPG1hknSPAOA9z9t0GPfl^Hk-EuhX?$h zO|aqSFoio%@eSJNj$;m|Us!J0ebaV#oAtEH@wKj)$96~4eRg+hu6kdYBtHkiTEA|R z;!7;>4$8Zx$>3HT%@DkOlu`%k~Taaq(y{)iu0) z!d5n(!P^`z7rK7m{`LCljbniBrCsk40E+?1tVrkh&Zvi=Xh#+$Z_F{F@@>1$s-g= zgXFUbcyS;*+NA!3%VRQk7)x%T1nL8EH<^urIAk~yMD)SmKwAPOtRx?DfK?b$IE5G0 z7lS5Hy_D`vY1(-*ucA-1ez*ms@(i7IqdtEBo?i498tE+9fW5h+?80{Eybf3NwnT(1 z&E?{yMO{n6mV-`u8v5V)t-2*KXo;5?bfaS+|Ka_ox(oQn8gTKYHT_(VJ;(2F`P3iD zzcjEZSQ$ceqgyJ|#ENhpxbjACKpSa3*{IiMB?rU3H^SCo)>_ZGJh@0x5TZNIPT>X0 zmi&a-`=>iP(YQV}C~ETn2QDZ5McZwwnnO-NpO(AOXOrPW%-imVYfo!A2nvLYSdVw2 zI!vl^mAb9+xN9$;mbq&W%i$89p%Abr-18%P=;0#Un?hq?I!|R)oeeeV*c&^0?H%hx z{!aB??73$LM%m9W?xNxV4$FjIPw(@Pn@gq3O*srkYkKDH!7`g4F z`83+vJjPz=&!49iTlkO>4hOI84=mh%Tu|8S*kEJY<4#fuo zQO7A&v^OX>5)p{2Mu`g6ra6${gNtwTL~P{2v{L~U7dgG*k47)q>ByP~J~vn~UWYnIJ!DKQ-0c+Qzci&t znYONG`}RAMqz(GB8O{fOU~?;*O4%VXc)!^r62+~$VdT8-Q8cOpK$`~1g`U`rmFj0&*j-s8YYeu!Rax8 zn$KSAeX50k{HCs=YkWE!TXH(8MKSt$Hp8K5) zD68V^tE5pufR4M;oah&WEJ3bdt`fO_KSL@16GfSiKu3ao^4C%E?3ou>YDCAwD~JES zBSpJKCmKur(27RD_4#d-M^`2b;|cM^suF@Z-)=k8YTUZ6`WrSMS(Wl4^pR>{p6XpJ zKV?c2ob>JS@rl!8W7t>I6$YCeFLE^PGrrVus(&d@Lau*W zr^IAAY!Arq39Vxm2im{(w*5JSFzFH=FJMSGP`bbvTwzBWfhU<(3 zynV@CUnC2G{yVySh)!*095)j=bRB2}+^W_g;-@ByFs9mq{mk_^Ij!Gr3Kp8gcWNQONDdGX& z?XsD0=riK~(%0LkIQg~Rqj;5})(2ZQ#xm-jQ4Z+B!|oO)WXQiQwXNNC#c>-9B=)8_ zxi-?hLC>_LI|vLEamtO>mEHYv@=kVnEl>To=1>Lufv7G&B#n&yQdpx0minEt88ce%qJ;7MdBX#CH|p8Cux1YhF*fg zHm7~vSz}z1`noK#`H60&rA*=i$f5PD2~e=s)jPiJ1jB`;=kFhx_H{>e(lwLsx2sKm z4C+s|wxCZE2@jp|71Zd@t&~H zx)K6(Z(OtCBDGbwqe4)dGA-yx)C~2^QV_ z-ik%G2ksi=UMEhgX26CgRxY^%1Q z3mEvhC*DVrOeb#DR@DTc4ER_+VFSOi~~B4Pt;05iw`eoz8pF;a_YS zK2wuzE+DeTcA`y6uKozNLtU1TCGBfiObPFndmS9rPh$)XLYid?5DnFiOoZunyjMTy z*8d8r&xwW~`2m^K>@{~crayf6fKkhkDTOe%S^50Ux7+uTDLy)1WJ3o*A*IzT?=eUCqCP<*}sKy7aNXnI<}?3MZ9P{9wjA0la@&fNfAbMtVG zLPxsBC;XO3c+hEm&PK(9t~oC7*OyMz70eo*H|6`v?8&D7F<_-?I79$VQpY{M4MRu z{iB5I2Iwu-a`3kl=&W?LoCLS-WZxc8!dRRC*~r*4)uF1>9DEcSGOqchLmTBY)5%sV ztXp`i)HxO%r~_fb{Q(@8q4~&pyh&=caJoG_d&1f=qtNPUkXJj26p63vq4XCb8eO~Q zqI(RYHSKq9?=7CxVD*k({Bx@F_YTwVT!DD) zze{y!w!VGRhbH}dF4cJWZM*fBE&K+AmYUW0M=gTM`a9CxYY(r~f=(&z18AWaE;}ov zXLxhMtup6E+`-{Qa{2D9>eHE5J2r!2TH;%*QQK9MqBWpjudG7e8$RY$(p_nxd>@7z zqKlJV9J=||KiVilp{=`Q%QhRNbc=~@**Dl$MkD{?>^d#(L+MHxXscy(CY{T~Qu6nR z5ASl`p;JSL$V-l|fK6SD07J@-eB8z_^W@HXVdTWO6lHIJVP)a>RA>7W@j?$w=3%N* zN==o5=DoSC)O=!NI!A&2zw>Svs>L^HNC}$1j-gGlH*cnA>#4k-^h$5~A@1d;QB8i` zR_TC4-&zmwaiEMsrbXCP$dG~*X^ytQ(RhFKiA2cTj~25&SCEJCiirhjZYl;*4%njQ zy}OnPHpI6bhhO!Sgc@I+=9|>KoY*`wgUsG5YNIv@4sF|fW^^%#B4nCvse!B}+~E2D zeJ*u~S5Exzl%<&U*0#TZF>^ySL=qo-UDPwM04Nc8X*{0)WW@!HV!eDa*0=B{FEpu2 z%Zo+(@x8NvpZYU6JKXWMV^c~=<5!V7A7&kzbmDcQ1s4+COCb?vy8*?kX%!^hvVaAt z!usea|1@c7!sGAS6&617Xb6FH8Ch&fyq$e_u{reW>eu5gy3IQ5P*0R<|7L)<275)# zX3+zq!MVJQZ^!~BADLqd`HWIeRJpab7tG> z7azU*Qa_3HLr?U_++4G|Z@$LB9%8MVQpnaw3@o_z{SWJOtZa+?*U5lryfFl}ee?1q zb;#W$l(gE_6h#qi!>|ewKD>WwkF34WMoB4O>vJ~s}IEn9;;Kb2how( z*~J5s8}bzI?j)dv75BCTfOrvH-aTpR8*8tSk``3Naf&i^ZiVbzsa$0%keE#4Y;ih$^6aT>htj32ddV%?N09z#mCOHuho@|bJi{2* zIOIC>UA;xom`Pi*v+Wv(y!8FTrIiylF#;iu_!pZt^=HopLyC@^aUJUG;s9sD|Vd{6YX+OCgP^scF z{vEu+$|yJYDE$S~tsbnKV;oWb43}pcV6h@wp(#t3i4ywRjr$k8o|tsp+rd9l&8vWn z9vIsQB?1F|<+R_v@trIuf|ewny?NcrXG(keP7ZIF0h-qKB{?2V6}~(WQJxFkv4a`A zC+}c_kR^~CjQt0d_&(2&clRr=jh=?;wfqo4aC>E(6Y5C7^n2rS)0o>~r#01DlW(T$ zgMfKQ^!4ToJVfb39viQ1fD@3*n4zs%{-Nr8D=xV{AS z+i6-Ecvq>(pAG+T`C0Y#7fVv;q*>7;iJ;ajE8fj_|Hd+H&iDPJfx!ExZw7<3KuC#IGPBz)fypO%Dz=ZDP(j10-iB@~*U}U52 z0gTux{@oHBb6_Klg;f8|-nyGmEQ^>t&3TtX_kT2BC5S`yt?a++J4gRN;x6*;w$uRq84U?SejtNv>$?rkS4^m<8iD9`-$h%}_Q&M}Nbt#`O z$m9abh;ZzCL~kk)*um+YSV6XbMHjHZ7kXr*BfQ+`XzCqOqU!AE#*WdlS3h3T{)j5% z?4N0kW&XKi{Jnwx10~|mh_Q3j;b2^l{F&^v4BxyqBf--(TEo`N;n?cx#8At<#+QZF zIZdw%O`<=A`0S+ddbEufPz&;1+CF? z-o|Uh)iw1E-KiUf_5j0zC(bdI`wOyl-wZCQw#r3+FFao}&Ac1o=kiqLleP-;>m88Q zV5U}Bkon#Ajd;MphKVV&K=w<-wx>mB#A}3Q5k;*qL6?wkJA}%Xqxe@%?Ptv_&m{GU zjuD=!0BFe5+S#78Rg!G-Q;Xz1cc}+VRRu! z8ImH!EFo|6bZ@Q2CA@lK{$R~z;i%}{9J0ieg*W7}!L^%|t_E-#G*3V%DiL8m!`zKj zHV`FF1eN2B38#GV#Rs@+bSGtbpQ3pH*Bz#LnuRST^e`T!g`3JgHggmb5+aDn|Z$XL@6FA4nw#w59lORmgO{o{_ zN(8A&%*SP3o+RNvTmq?+pM?l_>@RV&840YCSc-O4g^QaZ^2A@6KLjp>~Qr>!22#|Bnbg2Yl+r)z!PS0(I7hH8x-!C!-|fL6zZgx2l=k>zQM) zA?($3BiEP94%baELFn3FErGLteK?E@BQDFcY5Jo~z+X<_`v*UNSj}!Lt`y^#H)g|$ zYUBo*DxGG!UFVN}|Lsz=z;IrvOm!3o z@on9!KW`QhD7UjPB+lTTay8b~?I@U9`JIAfyvMHdC~B^AgZGK+l>ApXkvw>Bn(JE) zwVeLSxd+eW04u*$mwo~?5WbmoISDCtL21gvIP@Lc^XW=lY8|>3fB7?`jhKoeU37zq zFD`p^nmW%X;6dmiRNz}UaXQAD7^>I^xCtXefS}$)yIVGdjHbxSy^BEo6SM)5EWE`4 z7T8#G@mv2HjcNRpU@7beTQp9aE355xkD?^uWh`qn}|bNmG2jDw`&a+L{3o_@_ARXdSq%HP7tF$GVr#Vb7ucm!v$+mUlO9-Ei z{4-dMq;lZGDAI$l?kX9&$OGdw%yXWIPN|`^34YS|b(K;V10IJ^GlpD3#QV=rR z4K))kn`My-L-)L%+<1vzrjZFvH2>ZKX!`rns4S3P4eHu>s2kxB%eUIo&iP*}DBLc0 zBRb2yBf`(<*RM|W73A_G6-j6eK0=+!Tb;%;!^q9^$KT`;kEz9?AYk>wO;Or7u-&aa z6;B-R_G1^;-g=y-uK=(p@EQ83&q6Xdv~s~+^2zObgJuONZi-`irTC@$s!S-sJA=DT zPk@~sUg~Jzd}qInDK$JK1 z8z6V!pN;q+k`wq?5ktwzRChO~P-W&LZ@m-@uT=f;uV2@XIrrITF6W?a$Fk`N85RbpY52wQ*Qip zxGc06nLDmXWViBvqnYUKYp%iCy`lsrE9Y@j2}s|t#w^HSn=I`S zyku;u9n#`;5>8d1S)JE!$Q~Ja`S(6F=SP|E2>9YLqN}fJvX6sef+KE5)v^48g)yO5 zuEJcTEHFe0Edt^7xMJPnD zQ}@)K7FDn81%JKSBw}McPR} zTKHS+M>Sw(u-C%#46E`E=f0VegNjj_{d!&_^E$fYNQh>LX58VjJxC z4tJ;Dg;vJz?ziGklu#E3LqM!{)*JFhZ$C<&|{ ziI~rM1b4BfEi~K%vyg7%EzsWAIRrodAphe`(es#I+ELAMGn%b zuI8HG)Tn6tfKKHbGw-Fh)0o#dq>pCI0nO^KjC}SrCXlhA9D7GW98TfozWRzHEdAT> z^^f?54#Q5AF8jdQ3}24yRXOQ=ir2HI$)R3=G~-12w@wDL14+VmKlB&`)!W`EP?Y!F zQMxGQW)}H>7Jw2*C>ZVi!@ZZZhAhNauhFdd4uEk_b>5XU?U{h8$11}<8@s-Vx-_mp zlxT8`6u*clo*kAUueWl(pcHgTju)1Dgjh8+GWol#fkYMtSqXpPsrjAW^Y_Vbd|Ad+ zzs2sLj+S3K>fLWn02Yu~<3a8+0|hPXu^YxmZGmB%=&FV`IS#E4DMee9o$~g~D5rid=azrMm3nfXu$<(lM$uSXj+e0ChM zY)E$biMRS{2S;K8T}9%?s)U!P{`)W^kX&#Rxo^1bV7#jFo zyEvTBdZ_~8b4_1Jkq8uw;qc{Fe8xoxkj7m<@e_cXM#ROTJ6$oIpOL=ZOt4!IoTV?Q zj%4rRglPZB)l^c#`hbw9Vo+SmKAE$HJI7S>G4c|FgrSU`+f$gn8#0}P;A4Bc-Mz}x zS6`wgqgIWs>|5^fggn%}QX@o?J^&_wOQ+atD5j@(2{~M|pTHMRChmlVDVlFzCNDU- zhX3_2v-)7zwrroWS;fPn@K1*uyfm-5bB_xoS1I;d(u>=GN%D_|o|BE~dxBZJ`W!$} z^VZIa+0W_I44 zSVFtywtkkuL%s<`kt;6;NRH5F91E{{5?v$3q?!rG*>{h@jc+NFTJ#5~$c7c=?}bL* z>vSr3o#3F`y}}yMceA<3Vx0;`v{-D>gbL)b|35@m3_v zeq$ymB#=&7?kZmi;nljqS6;8MbcI%+quF&_+N3T~D@<8QS{9PhG;BW+WysIG3$EBX zs%+n%ECNEwn1R5=4D}taF_aVM{3)94H~s$-czLfn8_w8?<*zfDcR(9-(6+e9n34g* z5oEVh6hvt6@(-Dl+fHP|X^{Ld%+`A-j1Vto#HO zo((?VYH$A+QeU*kPM!v?+_NJ1;CxGO?qX3V-=1!cSd3#34FMQ_P*70M*ju7(JPq1= zf||E06x>ljX8K8%V324qUhPrHMN00a-9^qSgL?j5nO^D42+}QL7o*}IYw@vcSqSmT zRaU18;Z*`jkm0suz{n-=m?*>VjiS-;IS68q2Y%;-&%ex4NP%#LbY>;sy68y1oH5k$ z_mb2Z@}yxi_u!n;^D7<#my3JEjpE>g#;I(TJzrcg;|!Ktj{oS$3W$qGGXwS&pn#&I zz&K(&fPDD=m~vW;eCP#rK1~`;MO#Ez?$|rbFKXkKURxn7p}zpweDYQXbf!R`$j zTA03VYf!LXlhm<1zxdT9;T*avw>fHYi?rKK#lWd?_Tg#dW@Rp1WV3SV3}}NMsgLUg z6#MP|TH1(Pvg?XtR791ckv0>_F1peOn2~HXCsE{W>E+KiXx`2&o;r@@s>uRqfk|NK zv@_jspW@aCIMpy7?$U09VXaWN?EtBbWh+QX4r<~nkS4HJo4(pU z(V1BJdhh*;QGlrKKW`q_3&>k+_^IOw8~g3>552dn*yG~VchhAN0cf2zUo{+7F7G<4 zGlh_ArwJ2PG`h>_FkH{`kIK=rBSI>sojb%p%1`3sJe!GMZ=zan$4^>r zzoK^xn(g%U;AAKR$B6zYd;xvQpx5Eg$Xs)Lm+kN7d2f(xx?z)t5c1XwZ@=GKHT9_+ z$9}bf`{j)s(Nph zpI-<9H^n?SP*#B7D)yVe{dyX`^{%L*k^ncn-V^|Ss(71fo&9Id_21xEPl~G%A>xmc zXc}T2Gvqek7o%dZbJINh*YRlyHwun4*KH~ z=YDu~d%*%O5w5b(SV(Tb5qU#ke+sR*xVFTLieWsfhIs34+7!G*Pw-h+_x0mwZo#Hvvqtwo7;^nhvwIHQ* zO@WrW_s<#9bq0Srx#0Z=@uv4Z2veP>reH0#c3zizrjj4C$BTMotmRMs^wXlkk?-Am zewukZj^v~SxMBqcE879`78;6v(`lwhu3K0axkheT@6EVTHpYLCG+8MzaPm8ZJu`O; zkrg(rbxoCcg==-$dEK5AQ<(s+d7D33bZn|D>p=~7m2A5ZuNZW!Y_Qo}{QDqT#AbZ^ za64-`6N9K^Pb#{OFqoVPZ*Dy(~$ZsB-!2=Mf!sezEYTd+sZ%SVJ5Xh&VK+Z1Z=dG-1r6L z`~SgnkuC1+xw6inXH&F);@6)Tsy;f7)$?pFKQV}RDoe+Q#~FXxH+RdNMzaBKql+qgBl#w9B9BG&?nQyAceN~hwd=1p?hL%o zoePIFIp8PDD!(fDE7syesbW)~xUp&P_@65%DKw6v3DzU%EOky5E>Q%N(o>wO($e6d@v-ZBKox%I}9A`nyU?+oV_Ir>j z7Jpsgt-~6(%FzqI>9ro-(2RiB6TInCR`uAR84^RfWUiW%C!TXdjkJ5ziF+F=m~HZt z%K|=NxBM;XosE9mOZ-9)Gk_{wXDYuZQDnyeTBLWmh?mMFxg7Et%>?fCjQPtL2epJ;EksR zH3=qIb5k9w;UIx8L-5jxAHQdPL_sHQ0;z7s%m^!-VNaP|g`5TPo+#YUU6Mq=3hn7i z#DJdFTj%EgMSa?gOFQQUNTFX>y)g>e&#o|+&I zj9k&#CbAbvt7q$ds{9S;u={jc@@I#@I6~WSMbFj%akn^{Tqx>hT=g44bJQsad$E3W za?@h12siX`fo>HDd^fMaQKla!WwGl+17HiGzb3ogcg?9 znJbk#3cr2qe#(>adUoyTwg36lGB0A5E(%s09A}+J4$&3mq?& ze@R=WunviZoV}O?YyZPVirkRPlAkt9vVQdd4OHPLy>@tv7%$GGj%jl-IS^Ln(G-x& zog2ix5aKjEb~@WoA5re3hVg>c`&v!l7Af<>DmSQh2^JW^Co)81H45sIH=WEufgbuN zmEkL6{t$mOkjO~-kKmgN-%+bqra5|~0Ll4Okb*`gf`)zYdxvR1O(9QdRmSV`rbTd> zv%6tv?BBSrgxiuAhJS0rF7Dd(6kXihJX~Wmdt8>`F@fwu3IO*MGQMHHrr{wUj0j(l zOhWDP4gB zkU|N3+)M6dEsv5PzLChit;bbrMbIOI_@&qR4SH_*brg?OBF%{36Mxqe()30Fzoltz ze^zL)CB8S5BTgr(9-j@MBJ`l%eUFLrIeF_-ZB_E_m9%y&0?C)-GLpgSAP=Iq zF#YhCrq05wMstY}>J$H&pWKU0Y?V=Con0a!Rg_jy{+3M#cX&WjX|I2RQ8P#qQWq;cE$oy??3a9;i;LCn_oOB4Y@d55wH%; zPn=8`1~)n{ho0&wP%rp?C2Qn5mVDnQg;i_3ZD>NS=9CLrDuf|$q~iBJ@fOoT_+L$$`e#(u>MU`#cYzves=XYmT&Ufu z7zNeYgcfu8wa zE9VVntFD-_p@%=@G8!|TWqIpX`wgVYlU17Fk~d)!U><00j>-M39%KKRk|VH zw#&x93$csyH#Yv(lYXI0ouLDt4D;7uk6)pDKIKACCOrQ(cgG}+0m|b?C{p|U@~8+5 z<4`fWOG#r&xV7eQ(7P>n<6G9ME`euGbB3Fd1c1rCb45rl`Uv(JZbP7bYFIOZ2-^Ex zxRkrV3$dhV(WPvPeKxn9V$>O~E#NxYLg4NWzN)X<3Ta#AJWq|*VNB`*E zwXZriwQpHIw~2_oV8)w1S{2~@ul9b2;|k67Ds4m^mGbwA7)6R%xXg_oclj)0GBZ=P z$vEnbVq%1hT#rqCPhD36G38W@;KR98*Pxgc1yUj@s<8RZM8k*t-m>LS_YqKA0; z3`p$9bTTTu@uv$^^g^Gy+?V(d2kIRQD7uev7dKvk!$^UpF*a@W?=R!+V^1 zn3wYsktW2NnTKHm(u`%8JS7;#;WZ}@Zgz#IOCXzu`DOG!p zZ(7#+FTNftc6(%x%r}u%KJ^E@(UYvq@VIYK8$C0(czfgt4&>2|Fuk=ej!mrWgn5}gsEO4!4Bpv7fkr-B2TdK~# z6+$q|{~}@D-=ymQm>$5~?yCt^$hf-{AMJQ<&HlQ;c64XAUoHjDd*kCR1mIWKb!oQ1 zIIW08j@Rz)&Un4jBMaiAdM!mf(T?@T&p!hMW7t4y@!@IQMu%LVLRsHS4 z*lWYI$%Z?>2??@ByyHlpy7c{-Z%9+IFygfMQNHQvq5q>WkXPJLpo;i6Qfgs50=j&| z_4FXCu7MnyWEbgU_MO=U&(tflo!{$d2W zAg36>8yWH#GifKD?!dPG!p0W}m=^MyTt(~LiLxQ|9)5%8`y25M%D6V*+HiDmzHi(b zS37sb^}izNLayc#K}H=?=>|+{X~mt-qB7tml#)4}Ru`W}zWV)^OP_ltQV_kxdKb`# zOaL1F$DAuiD+H6I&qX%d$NNlUhw13kf+88%x!-X2H#coqA4P*KZQbZC_jrUth;h#& z61j(OmYFX4usK6}zB$vCPJxKO9kvjCvo{xgvpgD_&Y7~~tCGLdS>nG^!&ujL%VaEI zQzLsLS;*<7!|I-;gHai`!pw%MV)0xe zKLS8QE+`twXX)LuqY9QPgy5Z=`Pi55jbH=Px1!fVe< zIPWW)l8?J5eClwKUMDi5h_#!W@201_Tc1+S-&ptvdd#Rr4t}_OQu(!5&aK~dE#;Tq z!K#zZxslyH6^qW`u2z-y<;2{&AdkoU)vbhreap7 zPrM$Ur6icKnlQxM)NJ)Wtq{ML2edpqNpFCbawdw_sX7G?w%Af(Ym~bN9+4~igeddw z`L+~Ezy2@~!O!`uLMM>QOqmQl{FQpVav8|j9QR?x$?+ZcDpmRnMW3UazRxZCkzGro zLH(GBULpjk$U+5~9KN9m0el)v+ z3Xj4y-_NtT<%9XXVW`VX;<@(y(u1}abpMHC4so9$Oh}YH?*fxzp=VI=dIz|ITC0p9 zn$o|K;J`Vv_D$9KKnP_t@2IDYFQDgZa8**l|DCaZjt`21b!U z6KG~Z^K+80Un_&?zb-At?@~CZP~Mf$Xj_G+(xrBC08P$)M#ec=L8DOv{scth&5d_j zN4DPtC07+AMXO^^U}o>0-|!e4FS+A>cmyV_mhcl_DJAv*-!cI0=eFhen29x1m8RO#S}t*d6fIcJLO$x0&Tm-R|3& z?7xXi=Otr-Xp0Q@M&07|deezixo&Odd3-xLI(@JCAGy2{LYxI*7*4<{yWj-;;>4`= zY9h;^H`He$nNa}u$qGz75aj%c{DEbr7IedcGEBiiJ0i%jq3#OdnU(S zhk$eVQ7r0g^q&Bs3JTA@P*C8!F-}M)_4iNt|6Q$Nkfu13y%fOBFvE|BNdSyhWx#5nQ#OzAS@Tet3g>Lb&al(bjb;#V2$NpI3CKwre?hif?tyvR#S@tcY}M;|#|yw-2_ zqv#*|`A`j4FE=mO(j6Er`82wh{XtAKKB>kc2R>DD@xkCi`M+JAQLfNCw~_%r7YJnq zHy^-&*@rzzaIF6`Tljrq!SX7}0k0M=W5pp&9Pi`$wo`tz!~lOum_Fv@X%@9NT)PG1 zAki<3fvLz-!lKRcQREVi67ai8|J3SA3Za1iKI*!za88Hae z>m9SH!!R!z5sI*t+P~VAFe>E!C*1Rh?D!RAAwl1$_wW5Z)EC6dw=Pb9lH#1aaO`;A zknC@BUhR;RSeNd&-|mv~&UJsL_Ukdfy^D}IXC1_uBX%_OZp9^;@i~A0YDBgT z5sI{58%iAvgLB4ijZGHz3TU3#F=OU2ClF;4Q?JPC-*SZ8&&j@UcX&wGo0lJlHdUqy zQK4et$NhC*E&pP=avNA^51UCHFur=sSk3j{DR=cPRitAxUX(Z?yN4vGKE`$UrF{B| z&UfHQb)ZOO6N%=>bH#rCjW7OPJ3kja-V@*WQ}O9ZtzPWWKc99(f&&dunc zlK)tlDyH#8!WV}1jafym2OUOJ?D?tpY}BOdRP=&}H6beVnQj`nE(|PNinBMY@e5Dg z9d9$hni&9$Y4pj~qGuSv+vPk`InGyeF+zN>AW4QUfjT-hyV};t7Go6}!`TPb(gxqm zN~cXQacT*kWbQY+O7Mx~E5@Rin$-@tDIQR50yP4b{n*3Cf%v_0^Huh%D91-^W`pvz zmp7{w^XBdc*9Fz*0$}4JJ?cjYnofC;=JT2F$Ty5INkN9x@Yn9V>!^oh5rF+25<4eo ztd^#k4;rR}KPL;ie}jahN=vUKBU?^lM5T`MkD5evvO5QFN?4(f#|+%=bz(G(+5a+) zH-*mMK>qwp-Vl5yK9~KG7;}DfYF7rteC)M$!P(jT2Ga`J@%{mdzwDj0nLkGUpem0Y z`!a!sC}T0YX?eJ~k}rBAzWv~fy5J0c=AcsIi?fog;mn3zKVyJKR7jNWLBCT!2PsY_whPM_&oEknhTHv5mY{ zP{^z-c1tPc@AfU(0XQ=KN$`0Z^fr<}oS*~HU-5Zva3fL>yR zc>W!8Uw6C|>hgSyxIPDI-=P#(2uuty`*Rs-pcYA^AhfH`sQ;sKNLu+at^v97*&9Ui zZjQOs;zJ%}O27&KDswPJDa-hhO7~wQ8r0ZmGoZ$%+hO2otm#jiCwhjtsa^G^me5tF zWq5Z@StVfq-(mGW&a*4C$F*5Osk1NZ5_DZV#6{iGh;d6%J*7tb3gkH81w)nLOe(X63@cX$je@4=y$*( zpc^sXcy_Fpr#Vz6gCwk@2)m49u*nf6)SngSK3s%TE^HH?13ej0iiE0&@Wmy{AU6tW z&Ej+nTn-&g)3R`R)GSLEoVlZ7m{3bOqQO+&&v-ammoa3cICbYPus!j5|3?MWB_u)@`NzCRn&g;!sAr|4E1K;8Y+HF z3XUkrL~^+;+CYfv*BC#kX-6NzW)~=6<_c)19;aWFz(~PQcq$v$wRlMj*5~(2i(VDT zhY?)Jt=!Q9pOzHy3}@en9Eg;o{AWAN0>sYwdX~LW!gMfU7QWA;SPt0L!?2$~2217? zyH2^MMM-;Gl|Kt|aOjeITDt!NyC~hber%d5aK|8!fka1W1@7mPwEc*NB9#+a6q4Fc zlwQ7g%#)-oMl*uj_u^3fESuDQ!u`QC!7CEdYr`U|>ekyo$c>hGuh%I9)AuPd;UtNj z=B)`pJ6?~elq!E1gGu#KCex+gsj+^Q2Om2b*F^L=rlQ-80+%|5+)TU6oLJkMP$PYI zHNGC=mxOQe<7+K0<9~2|qqm{+k3S~Y57p6wJ!yWVi~fixE7*-b3|g;YT&_EAdDAw( zwL}%D!D|ow4}ddxr)Ht3wfu$SvRufo8lX2g9L^*CU_RAAb6b+d1i@4MCHxT_tsG zLf0H${~t=@CfSBV3?Zzc$UA3;>Gj#>4Vjz=tmdm;&jnu%{}D{N%6cbsei{%D+5QHt zF?7$4uFHxUb#30boT zK@NP3%r2O13C$VBiG zvcHXv<1F!I(_i6xXEA<}@&(&qn>aZ(oF8^}6!qmNz`@Rp2GqT`1;Hz&rRhTiZFfpv z&E@&h=QIOcIIq9-yGR)jYqI&~6I*_j_mt6gj!A?;4NTst@7*TJ{k zw{zliLh9K_`OlhLRkrXp-XBDIG?h@v5Nsx>iFwaTQf+&r0z%T-X;Aesl z${YEd-WMxdC=^W@bUgIRM~<;CP&&k@D-$Wd*K1rsQV4vL0$V*{K8jjoLbn3tOgXWt zSq{!77!Owq8AR&;(Ojhe*iKGZ^4*1dx!@aGi1%+x5L>>~X+xmTDBauaH)J}Uc}{x; zA|U0wGe|!8!7wTSi2hHF=tC3&WMOC^DBDf4ljP4Zife&imXbrSG755B$7&&3i&|-6sDCOGscBsYU7Q!wQFMC@kHSB@=z#M4w(-1YU zlZ>mbY^RP-P$mRm*;z_$1o!guSw!S*l*!?J7%g^jMwU=_jEj}r$5VQwMiwlaOj`x# zSs6z#))mTyIF_@C|Gd_<@REM83}*6OKL0lAz>~3gqfAJY9u98THuro(_QdBG<@)Sa z!Y+7CexI%>LJq!MYvGpLlzIc%^EkZdh}wVXQCRlwT3&Z>c{!n<5JEaf@=lec?@bpSpQc|Ijag7$KlTKb|Vvu=CT1=Nlx z!H$0WT|`+4)Ywdf-bTdCDP0t?kq8S2B8;QK`A<}ZalcT!ux_djKY#6P#lqsH=fOOu z0v>c;L3xB@5U zgIkY6WATL-8VKARAAh5jNsekM7Jo8ObafztED*$=TPLTQj%BNO6GHdwg zSWX|Dz5(|KLtI9>l=qD&HWo@*;s*#$N$>QF%he4r;gd6(^&S{jj1=^>ZU%Qx=xLmM zeM>vs?TtKztDaP{j3yRxLA`U5(Oe&#d-U$JoOVWH6-m-^`TL$nILoyK>wFtHf_T z()XjLA_VCK4U<&_PCb; zPaB@Dl~%M}2BQr>{B*pNIWetQX>s;9zdh(`TFLZ5yzzFd8lr(5XLm`YiS&&E}2PmNnXhfN!- zzA~0&x^uuNC)Vx#%x-X zJbq%<64rAF!m)to6I&{gXXZR_Y<&pQ3(Ia}luXN>eKa1e(^e)@0&3^`Q-I6P^K9S2 z0hd7wxr&Q5odjrS7BM$Htow22^W~B z9M|GOUl;yUP`bxwe23LdkSBiZPvx!^R1uf$dMfZ~O!(xhXknN*Q3AHU z-l4@I(PvS2psd@DVrYzRjmBFnpmO-u*L%?T zCshhK&N;~rRwoZ=I(*iPZ00|@@f~-L7IEy2zE~Rm+@2+UdMp#AnD9X?T&AzL=ocC_ zfP2FA*@~cq_WnBN(;F;vc@mrQ-)X@eDl*RYHBK}lk=TcCxlJNRWr>W7w&Y1>Wm0K_ z-1SUp_y8d|Y7m^g07k4!7hx*(M|zpS=l!m;J*hPR`h&<{t@$|qa7sXkkV`Ew6Iz$AA1#=gzSIcJ%~=%3 zp9G%~KZG6T6lWof(z<-;EiN3wXcd+z6scgpBnZNRQxB~sCM9mpH<{EY&fWBop-5vj zX-e16oz(?9PooHKj#)7u9*@{e(%-7&R(C^k9=8mB2(K;>GiwUhPosD;)P=!E37M*5g(#{Tki zL$-~;q+7O+8$wp%RklL7d1`vo087g6-Y4bkw643YSTep9;-R!F4-bimC=AF98pSMc z+aevxD6R)b=mi7cYT0PQv8!QMhp>|*V?6mP_OTuRcC3O+XbY;1$Nrv2kZ+*KFaI2Y zj`ZJ%fWL8Jx%lp_w#`6ZNPV8^xK#UA@m;v<-eYXu6xrAoe^Q?Q3+JsHMOiZ#)yQa? zM>$eZuvNU$P*02J;?$$^K&N*;pVy47i4opEmC>i60JZS(Wh63x=-e z2p=;)Xq~;I>NI*sw_D?$-E?z~X3^NvVMGQPX8fV2X^&hWh86*@ zC<-vLBHsIlC4)4R{4J_jNW`5VQ%uWyDiZ0HN2yN|;^NB#=DnGJG;BkDK6u8ZMW;@6 zwcpo%3`bphC6#v8?Z^b2=H1!mq@d02E)toQg?q5iP8j?Z#w!KBV%jJ0E{Vo&0tTai zp6V9no8~Kp`~9n86s7NZfsMaX|6|*7J-r9J!vdAldFQ2iE%h#a?(fG7_r_G7k548@ zXDW8vs2aMK&V@h6-_GI(v|te}b@Q*+ldj!618kegz_(}|M>Bh(t2u_QeM65lEb}}) zXhTyXPD_qwV`kr{>?|FXCKYw# zZ?Ru1p=dcUCUU3QW_{uk;4s^E7ChZ~c@p?}5C!4C!dRWd?(>xZqicf*VG1#BV#~!3DzzSe&;VUlvKiWjL-elxf#A4ttMhlyVzS?kil8pfCTSS9hOOS=bbwdm zOJ_b+&j%0d4?luJCM2vc<3h&N(DEL4_*5|d$eaCWI~9Gt<;*)Es^3 zryEt_RY#HaBIegvwA}`gx3^rq?O-2k#z$p3!CQnhR((zPV~Mi>ccPHCR)Wv&)G(O) znDiXKcX?X(cLe7I?YB(*=?(qhwqtk!%43$Y2yw0;8VwQ{Y*sDe22_g&WO};S19%B} zgaY#=rYi6L9f4Ci1RZpA`a=BS>x1JA$|Ebb(A={SQlwWE<3m5J%spG^OMXO|o&8%m zk~X3j$%YYp{iH`YQaZ9W?B7p^<~%iGyz8dv>pt;6f_AK(ScPJc65x%LtOI!?+%4jv zMy&pXS2kb@H2HyC5QE1f+?#p(yg1xyc2UbXn^(ra3L2 zaAVv*|33teUc`gs#epBg;6SIJPZI?7v%rqaIrz(S6_O*?f7n03eRfO%z?MI9C)L2O z_2q+gNr>Xg&^DfLc_+WNS0|%#GfC#`$dtYOtvfHCX)uB0vIqUlqPDe|)6t+mGtESB z8Nkzvmri8UdKdI0cyWd^rP^h_+klen6I)P;@YX8~a1Ij%gmy3|pfEJRb`&$L5l#NQ z+K6^>5tO*dP&yN zsxeRlPkXl1W5t^Dnbpx@=g&m@kc{&C&d&;f2%q_DE$Ua7O;gvN1SXp^hGCL6h9LMZ zk@kyjE2j3{(z006ZgraVgN0X`o&`yd&APnjEALoPML{)Yix2C?Omgk0G=4HWNpM*z z4peIP-cC@xaw#G{F+dQldbj^UD?VNqd8-(vrjz?hPJSV@cP6yUijb zmv}!RoO{12bb}_lQFjIy8IE?+wbHc}$n*Y8O-7Q5z>UW$hedO5PX~>cE9g_mZ1ZQ5 zHJ0#yHTLb;e^>nEaN0{;2!>OD8B?y@5O*&_!DWZ!&L6DVU@y`8l5{8s3|be-Z9JKR zoy3yj0E{H(LY)<$-^uZi3^MtDXgbTdCf~P>qk>(?Ya6k_*q;4gJg@Kjes$fS^E&JJ9wzRy3Hgvq4E!1y5E|i< z3?$Wz^O3`$ueeTSU5@Bs#jTr=zCP3S3otknQela}7t}&PfI5VP9PYR_7HSZ|Choq9F`FT!hrve2I<=Rz@yEW zI>hN0qzz+IvO@4g2nf?jJ(eq3b^b-x=kmvwhVq6n|113ZUsL>{G^AIa&p8zxK=hsT z4AVdF)Nws#K;yBMMn2)>6bkF`S`}%&KT*t83&zJ#hivDyYUOZZ;I0Ft$B;oE=3fIE z=zQ|n0$f$U$NDi32mgxINeJ{IZpvxr{5^HZk&=ab5Wuc(7tUKf@%1HMSbpPHIC!KT zO@F*Wjm!V#1IO&wj)7Vy$q@fqfC0Yy#d5`5K=cxWung*aSoqQE#b;q9-xfv2=r#o9 zUZT|U5f=#Pfrw@{RJQ8ZP`W5=VZifu!FH6q{PNA)kAA_c;8*KkI+>$rH@O-db$00< z)@mi_c_1QvSk!M5p!rOUkZ*Tc={ldzpga-D-5YrXC#el->)_(iV~;c={l4*y3XAyuTtY>~N!OOnB@oePK>4z^$1T74LMb z#C$m>%oONIhE=26ZqdLI2;3JL1eMe<(}`lpd8&sPSPr-N7A0I(4$OcNEoLs9G_@uu zscuG?QS2n)(=X~JryX?54#Uvet)Z{GCDAlN?BF=axwOXj4NtDX_lQ~ak`xQ@7F~Ne zcKs;i$iX(4O2Sw!gykO*H(2Hwe#_I5`;h8PuOJR;4b~}w3C5t|x|pg2N{ITR#V*7N zCxz&V>v8`Ncd%^=3FZQeDN#+p0?M(~>$;Wqf7+Zne&u;L__x1J{u55jMjoe~3#;qV zPMT=+;)M}rQgqG(PNW@@)_5rn8)u3n&-4u#5Vo>|I(-{_Fy%3VKJv-}a^|cz=!(lH zj{bKcF4wJ?lo>o8?lok*K(VQi$1905#RjS+Ii7#fzYW;JEs_~@ISQ5(L&$t76IY>Fo zml9YU|BbRqk5%_my`r+jB?-Oq(y`L{$uV{-Wj-Fdmm5u1qdt?vrIfnyjm5)v&oj$s zXzxc+qh-ys>MirrzaM8P0rfZIzJ4Ra^`#3|6YjAB!tIaQaiEaxrrJ+Che@|RE%Ts8 zr9lS*J$?=Kj;w|_(2s`Xu#kNJNUSPy@*etrO_D}uqfO8Y006TpofNVePX&uw=x-oi zKi}w!_y0I}+uXTqRnxkKY|sT>{g|1E`@)qC3dR5+fzxa^rzgm#Uov$Ut6P4JBz0IrK0c_4W5r&;8u8)P5r4Z_@3s2mX$lxED)|h-*cs9I7(R_n1Up5fvmy% z4!Z>*>#|Py8V(fgnC}+WH%HRo?J{Uy&xAy8*otcf`cD%=SWe>(AtC_>E-BxEBY&+! zYl_lv8m6^B&;wyP$|F0-iVI=Skm0!3_#{+^EOO_TcZYi7j<=xv8tC9_85!dDV`zJ8 z!j0=(Dbv9ZcCI|Pw)OGkzh5?ut4xHf+Kw`SSWMhV&3Sv&fJP0+8CBr1H$=hl4s;{T zNui3lQu4H~^~S0hzTeqNNRrWnk{9HMnRCKoUU;2}R6>L1f5j-_Fr-R1Z(<;8V~@TE z&Y$;}Y5aQI8@u>uFmg`TWd37=06!9{@722=Fym@z^6 zL&ALe(Nj~N&7(Q3F8|<}-6ZxbraqG~H<5*YeHDTei*Fiv`P|&7Y+xh8K~rONZc>#M4jkZ(nYzf>853 zEayL_OZ%PrNwt>`zp8d35!6H8!S?%-3qQ;{gD;Q%W`bs%CafCmBT5%`xI~J%U~;(j zZ@3&UFdT<@ob9MOW&7s87d zm8p1z^?^@@+ymEkuuIvyT8nd=%g5(mSpgX5(erA{#k;=ml-#eeeauIQj55gfCg6#E z@YXpVSDWYgI}Gq$FwC#H&rJu1jLuj9uN5Z%#}#57JgZWa-6Kx*h&3RJL+`mn8uwC7 zMVAl*nG_dGQ%j<*zQ((R#)0{b5S0uwFiIH0h+&I~6#B)ZC{-S&%X=|R)_G|+6z3Y$NXiLHXc>m{9yW{H^VofX~F|l|`5_PAxR0{0=e~!O$ zbJ%=JZ`R`1pxYCDTtXCB(@Ns%aFjY}?s+p!t|g1oja zq2+^!Q^v0$Mz^geXpDP%|DO~SU`=hq;{iRx6*2qx zU8TKqBjdl>vDSCR&pnxjy!PYgIqRpAd~}6$VJEv8tiSx-)U8HV%HIr{uu>c5kjkGa zkaB!!@N%$5X55pdYcr%&Dy+;jv5|==wPO$#I^WZcW2^JsVi~c`dX%Y{+yDOkV7~dd zOQ%PV+mNeUnrIaAUjpJDUFFW{+3){n0pPd*H-lVCNcxf{fUkMebvs<)L{Rv)Y+D-j z{0wne({WPHl?=X>2idserc<#O@n184oiq%yb;WS|xphOXw+h&N+m4nT@Yvjd@#brQ3@Gx)VH&6jn6jx8|!%p?G5t3KpC_k-^hj&Wc zpDS{4CY&nH_}XHoY}p`gj`QrG@RLT$aH?d#GL|su*)Ev?&@qr>VkFqc`-od z`a5UrJXIy4C8KxLm7IS1~V*14XnU?BEhHdcHhz7)}_Cu|2 zKgDW}A!91p^3=df)s1OV`XlgF)c}&wtWYol--5qgyirOiNy65ZeM5yWWm8Z0AX zlwat(FHQ~_;WBjso;qvFbk3P^Lkd=y;aeOQY=4{vIwBqXV`i~Fo zC`cTYRx~weqMmL5J}c_mn9A)g4%^TKlS)XsX*y`+r7rxwQbo;!2t2Fg83!{Uq#eC5 zyo&zy-)$6O?=@g1QF;-BK48vXts|%~$YK4YL+p+J!WFM0BdA~H4x#fro0`=AD#(Hf z5hu?0fi7Dv*EIQhw}D?)?HuzjOz?O)%dJbNC5p$o3o#h%|TFO=274Eh$4wI^86 zj;DdDuGPr};^ul!L_S|Zi4mxH%ZzW{%&VUsfrEpku4RZ8U9$!}byyUut!)pMk+q|cmE+mAPhqlc!X7^hiO&)~y z`s{sS8V!;uA<7UP5DF27@OiSkFS26&kPMxPw8cPeCLHZ`3r^?9Kf*5k`RNw6`->_8 z%R%GB>{enbp(A7bya^%x1lr<*b4=laX@pEs>Hf_vk8zjEd8QJpr<(7sgSluKg&TwS zrbqX87uK-L)o|ZeiJpWQEc6anxC4)5r^^2j{5>Bnp}l-VEo-m4zV!K!th3tA-WQYL z4c?gZ&Yk@^eLNBNsDR7 zV91eQ^-J;2D*tk4dev}Vvrq2z4V8khSd4Hnsl_}WyCmI*et&CGBta>AOtGrnogHak zNxUQw2=ekp9W!5M9~RK3A6Py&=2)2j^~kC7Md@$31mk*+#N+#BI-`m8`^9=PUZ)EH z1iw*E;etK>EDz|#YU-$;zOX85Obar3Y|XZ4go^@9)tj>Oh_l8hTg=bK{OZ4V)4C{lmgy4M23hmb>sm%OTy)~EL1=(yO| z5B`KomR>zU0{(TO=8`#ZED>`r%oodC%TR6#&zJw_a-`$*uWb;jq!k>*VxHuNdC5~a-VGhJ!-F@8oewCxz@oO3;}$1 ztiPZis06h|^hQFq_k#kvVKY4Ea<>&+HREwB?!g8MA@idl)JwZfc&z4ed@9fFPBJT) zqM4)Ux$qfgZ1??dB@JWIT|smFP|Okj0vZxH5OcIv@_Gu5j#<4}cdc!cdj8#1{Mqe& z09E^GM;<@PMwJGnfHl!Ub}*3~hpO${eG*`kgqarY`iV#1yV^+Ac@{X_R#?_;noS{9 zWK|)Z{9NHE}Z1P?8cc8esj4L@U6Abo16*$Vx`V= zI{X@Z@IUNpQHw;+Mcsn_kxw;9Q`lQ;$q^U}V@vaVV6bXq8Y4kA5Oq2v@hz9<;c&Ni zO!MQfDJ~l~TTnl4R}1Fv;ROBQ%Ma~R2EN!3&w_g?W~*>*`{@t%TPIf?00$4j&aV-e zyd(_e?DqfU<`oe1*u#Mkf@v`XnFp0U$b({}H=51ujvAl(PVU(yy}y!)TD*H$c|W5! zXj6RdJ7TUop5BJ4C7qGm)4Dk$+RO;kG%-_1BCYw-OjgQ&>ktGU^pFoZ6vz8-`YpV* z{l@t^o3JduZ_+U<>C8I>aXvwh_w*8}OHpep5Wb>)EZnfT*@pTj{1P z_zTnZ*mR186>Aqbvg~29S7!}0+8AgSi+`e#o|^;8cXB@yge3x>T=U8cZM5UP?u4cL@-o`LZ$iuif-*Hy!9Ve}`cgT6duU*^`uDGI(nv!`c$ zCR^WIcCAs&f!}y;zhz<9tI@GWt7?U#SE{DSjRPFaVad00Q3N?fW3khKw8efw622EJ z&xk)rM;9HM0~f0irzR|o;VM)Ax=s(%4Qa1?{u6^Riz?_1gVc9~z(?GQvuI#1UTOmL zQL`c*>I<-t@KeDb8GOYM=4YZJYoKD3Fck7WV)eCL5iGT8Pyw5XVxU)VI{o|__Z;V7 z@PzeJdyaUNRUSgPq3fLpeCWsUFR-WZpWv>(j*B_?7&dQ(L#Z9{1+Cq+qtZD4g~Fr1 z+EexUZ$%5>?l5)X*)nB=RW*h`qea2eZkMcCBv);VD7Z;e)Y&cU@h`Hmf_y)wW~)y> z4G7w#Pfar^b&J)x^64d&F|S0$Lto04uvoaj&Z-uaz=I~6=fG!5to(5eoHDJJd1AHK z+_GUsJ?X^?7CucXr`DzVV@mAz)TL?#IzGHqsx4I=C9T?y3%}(jamyZ1Oyf7xOZ1s& z>smehk?Cy|F_F!Xe_bY?j5)aKJpWvayXJwxc92yOp7N`o_>5V*GDoxCat$^T6P99@mQN zBj!cPGtw8QhL0vQE4UI3mdeq~N-b(CU;5IF=Oh6{>e{(F8}R+)^>+kz;q@{{M~0Fg z&DS7K9zVI;h=~^J6F0~{>IV=isHq3g>xCKL3HG{1o9qoxr*f*rS+e^4I_9ygB+~q! z=#U&X1^;yURfF_4s7G=Ezrz5-cMrgjx3z$v@%>%o{c2r6WBo!75%|OJhxfH^5T7rq zG}sU_z{T({jWW9w_iEz-ui_Y{!5IOcsyEzGB6qWwRFd8N7RF#+M?3^Rqknip!m8Il zoztM@7%h_vKEKDXzCG(@_mKig#cosP;6sg(n)dN=o$skbXu>uXbxphf7qP@77T-NX z#vecO6^eiKFeL83K=^bggZnIuoL{CW;3{s8FunguCB;5n7%vo_&A?U47=TEfsLdiU)gDvK=g=1RoaH@S3mHpjALy>}4opG=SL zH$tS12!?=Q+bS&CuZK7UFIGOdDcmfDz4wNmaWkChev~J}LDJ3vG%@61Ul1*NX?5!q zndgCB%?92clpWuogFOAuxK8oYJg1l>21QhlCwMM~8Zs8MdVOHw(Z^SKg zZt)<9Lv=uhYxU{PWuaowhvL8IL+R_?RuzTY-`@;iV73&e7094mv`q2~ViT*i<_`8-E>qv^_BT^>83l8AVJ;gDCD28OC{+I`M2VyFNKS4q< zuJm)&dI$Y6ZD`9v*!nGA50x$QpMWmIQ0*|gb-R)%j?2eofyWo?yzTkpUY3N?!Um`i zmLT7Rx`C`LSdTlU)33z=h?i=y>g#8VcjGjm*osYE#b#F128sKtM+F`oM;fqo@)s4h?f;rB z@OG}7Z}Q`BRUQU~h7}v9zd$5C(ycExc@u09xv>8fsmy59$MfPHXAB9KOyz%ftBo&l*wqBw}I3kQGpj>W%e41)+$_w@=}xlKE!(tnVv1?8O$WXpbg&H$a7i6?H$ zWlRf}YdtHKKAE}u#|YC>VstQ>sug^&R$HVh-^YGO%)sZ&@WPX*zL_6T#KKV_1LTYElOgHQH6{*;o_id%u>gb*cCv#SvO5Y$W>!qf@)#CCXz|KZ|6BhJ zDERBw=Rri36#l}6QY=@6QR(v1GQ#xDQW7B{yH1H-`S}_VjtP#^u->O}Wwm3cc+8fn z75S#{ZXWt{vPLnWH`pIWOJ~ae@6+)y59ESaC_M6DBdv|j?f#VQ32ESoC%)wP_b-&k zFWgD7?#2+Sx#jx(?judGB?LohV9M`ahq$jQWll;-M~LfS<}Q;WYHH@thLB>AYe&#adGUpd9YU{Gkt&DqAxxXhrN zgY**JCQo6p&KP<}M;XyUMiOIW-@*y{^QyZ-EMo)`oG|Bt(`A;%jl>+wI&8?$_1+I- z%=z*}j6bc>3QZ$|Fn-CMUS;sst-q#yqb!bGZq!YEDB>KT#6OI3XaRs9L%vGh( za{=Q8(gFUG;GKzrfL%Y+f2bSRWX0njJ$iDI_eYJMAph!+8oPN&LYB|GQk*NECT>$< z=|8kOEfLo+Y6`qvZaJN7(lX68e(`<-76*LJ#W$&d_V&^eiTj+hzKHd6E77^m{@zHw z`jdf`OWiv&mLjyA+L0zIg^01T`&nGen*>k(Wc1c*;cM{$C%0H-bE9D=$-_r)dmy}T zTIE{q2mYa?*B~kv{nFjQ{}`D0+h|jR8NP`n$F!SES)f=y!7Xbr~nKc3Fh8!?pBAnK6bA zKW)8$`gZtAIrPvml4z23%aCfZ3zRVN8UEPaw7q#dwLR|qpZTTX9dntaC*_-Psi~nu z_M|MWG0>gjsi@B=-wi!>bKISsS|NUlbbz0E@2%^P?JZF;clKzx3dPJ2oeN-4T@AarGySnzK<=V3c*rbKa2nyzdAx*go?l*yWLf zv!ZpoQ-y^#BlY?jm~74cE*P1JJ*lP{i{d`V^-Foe5RQdjci(rdfgh-GX0GTqc&&c} zS%2IY4Jtr)i}&o9vRQnM-IGUMd;xZ>Z4_TA_+wii4u8Nba^w4)V&2im>S2-f;2OdI z#5;|#h1imR;Nq8opDi1(LM}d-&p`Thg=+YYR@y1?xZ%uKu03xW{Zx1wq);c$_Ch(M zb}t6<=_R``nd@g*PP98<#ZeEm*=8u@WouPz*7L)pvjI&$@!J{oHjyD0)4~Zm^BhDa z)arsGm4BGrg-uVYR@Ip}A62w&z&XKb&;-EYOmH?+6jm0sQ;45C4Z)om4kT}A#x$Zq zpzNLTF98!{kgaTA>yMYE<~V)sp>^LrFXLw$Gq-WUY#qu8e3eEAz;c+DSgHZ5+});( znWxbuuuSw{+nBND6%X;YSJ3Hl8!p#P&;;Kqr9#j#%wl)E;Gb}&$Mx9QPd?x)>}&1b zU4mqr;FKh+0Yw%;6=R>VACoKJiS41_c;qch^Bw(uH|7KrKSrN@so*T_LB8aiPI*mn zr0O$_^W9Z#fbS%<&|%f-zXd)7N6vCWWS^@J&MxaahJnnjBLc16@qvj;mfPxWa9r8#D{ymG5N~*|vwFfP=&Xty z=C^&-$*Mw*X?wgn%|85y>)U_g*HM<+6M5nmuaw1Nj^%v{^s!d$)n7k^N?mI$lJAUH zXTm99A3NfHxbN*xkHu%sq4g8zWy+B!ePQ7MGItF=cLVVC0_UsO!A9?@8w5A-(;Ga9 zd!;VkS{PZN2}UFPAXmBFH* z;ac{`kt)TPpb@wP`*v}qG3H5e=^U2qSO*&;p!}*{*X1(^_H5%S4fZPd`P{I(l@c1I z5uOfc5JKO95Pr1-ki8d+nadgJ$u~nSEFIYGZ3GZg%jh_4v{}XvlLH&v)F3S`Jcp%c zT{kssUIn|@{gHQxfFyCf%)^)x4e9#6!Nz$TcHRWqD{}zJRXjL=tLK{Iq5TK#+nz}3 z5OtM3Lsk3F(@KNb%rqu&*NTEt9#-J{nGw4y4NwO)g2ZM{KTD$CU)O9V@JnIp3D!2k zcKfP>AUkNa84dT#CgvTFS6D-P8+w}Tuz9Y;uQJH5Y%n!7tf^RkGlw#U_blq)l>mfG zC2uHuFc(3ZID%qm;imrGS%!i;BHmn;qOT$fA+EL5k+PjnEV$`!?ib#;{SM?G#K*?= zl}5Q^ytDci0VC5(=n1S<9QJk8=xhSstWtk%&a)*rewR(naV~TC+b)JLG{i-)vcy2B z2stU0`aHKQYl`(A$(X*K#=;AK;YO1!jUc%vAu@jus-T_sWprj|P`-l$Y0dYQ56U;9 zT3Tg77r(xxrO2DqA(#AQDq9-}RZF5S6#dBndmLlmVz55|?=%ysr~LadWcLAa-zN9? z%|U?BHLkSI#$jc+{&?KN9d_8PXnJ51MyWNSbUTqwefvR)jB}DdQ^-y|kReQviSle3nuhQX)Mb`b)C+EmI_Ai6H7ER`x?0LL|YXWs4#M zjG|<88%3x4AoI5|0k^So8$9R%-`pDxmks>tQ{Yy+)@PVc%Xs_A&CmptK;jr+74teH zqRC#?W|T(6LvfYTNMhi+(G3MAd^Ntg#=dXnv0C+WkV2SzVqD)zT*g2hNHo1FM)k zx8IcWb1v4xj_${6m5%>wqfm5apz(Qx4B6#12w$fo0FhVFWy@^2K+mwrkTD!{?7Y3c6Q45PoM)@hR__+|DuhPZuUrg>mP zfCJx|=D(bpm9dFOw^$Tl&s;g+KYns4S90e$4!#2O z^eCgFQxNy#cTd^j?e@+Bfyy@jI8cI?lVKqQ;D$vq1l_`js6qGPb@ye zPZxlqN54nF?c>qRWz)eOI~#yR>@JGG`b>3{R}Dp{763BeIz2QjdmZ4hO(;Tdh#0U{ z7l0?-M*~uR{Sr4C`gA~6L!~D=P=z;7!7dgD@?K+&(GnxcVX2=}9-P%xM+5lFtr%Z? zHiNunw|huYx_{S+c;+7^ArrE7LeQvOGqAA&=^oRkM60=2RqJ%&iaUcQ$@7<|^_;4&SXL#HC>u|^0(({14EqQFB;)!Vy0 z!_P50o?}=Coi%ODpT=IuM+~{znbw3|kFtnZ9MOzIA*){r|d za^Z~YR&w*+5@&0Q{x&?P>9-s(THbsFB-xSz#r5?N+Gc}I$R-&q!36gnqyb&g3)A@n z=&9ijWJIW%1x`wUpM#nDtAC^_KJcEbf6Dx7OHZx7!~y&izk1hUQgQw9oVQ#$z5I{7dXh&;aN8J*?OljGzI#{WIfQZ|VPX&z z0yoFb@a}>vAlxYFjTsF2heWICYBAgc*_{N!>n&hcDX-6$28V+i|{6^hrOPf`ued+SCEZ< zyV)bn)4`(X-G)W~$#`b+>g7soJMDr#9Qw8JrE>mCu!@~5L)u*PAgya@-E$_;f{t6x*Zmo zB;Hm9n}-2IE&8ZP)r*`2yi8*+3wdw;5|5`o6?ha^lJ_71`X%-k2$NZD0)C+JLM*o3 z)~@>V{pQyHO6>OTl1-4L7#3o#nUyqW9=PNy*c_ZOs9Pd$M}AqSDB)(kV}AVl!({7V z*ol6>Uxm9kWG6XeB4j$O0K{g&TI*0@O$sOYQR%*$4x-1q0J{Y=2*ddbbjSstC&D@; zDFi94UC)GTZl<~YFuyc(V1I6Xll@AkYLhq! z<@R1}>{($n=f-3)_3!Pa?^D zoUmD!xFYwkSiU_GW4;I&n! z)EBgs`0x&8dg%8b4fKw3t1+Ha%aNripNKTw9g9Y+Nc=n86m(~`2mx5tQc81aPEjzC*(Jc>QZ6#6{PtN8XyX^uJ`r?FV z_ny6{e?NlV-v27BgdujPGZMO8*oP}1eL(tQkE3d9Qbwv0S?IsT_7F-jXwp|nIyplc z8D(WrCAHtYV-LJ~ilVSxYm%FcE9Mfft7!2z(v09v%I*1vd$Uj{ntoJgJ{J z!H>&`L1zeu(SctWC2jf)W{;eCa=^mjue1lU}D(P+MFBh90x+qu+}! z+7><hv%=pVng5k^a5_ z9r!J~G&b3cf>V}%vHUas#k(x5CVuBySy@wf3r0YEK`d(YIAOecX*PW_c9!}lAb)gMmm@S<8{{Y(2XjV_ygR5hj(?(MA+At~+k9@>+_}%|fa_h6XD|kCN>05vg4Au%cO4zMkDR>)Gz9~dNvzn9ZWu5l( zbIKFi-+uVMS-#ly7{F?TG^Nv+YJh(>8qZU7>fRd?{|<|OO_!l{ywoI*&)z*h91uT# z(i?LFyq=$6K+?K`)#i7)wI=>uO?IR}XDHGFp0;iFN#CBghUig6l|rpP?kmMmy2G2p zsCksd{I_HncsYXSz*PeJZ|~52E0tE7E#FumY&LacK?lH znGGj|J`WqFiClNvUJctH(h8_2+sMrNy*R@xdj3(2vFd-NqB7*(bamEjn9C0u)>441r~QkKpfK9={tp4S@>#tQ4{ z<@IWeZC44m&BwEb+oa#Cn@asojo+^wIEFm#-W~fQZ8O^Y-tc?zlm3_RW~*fs#9tSV z3z2*!(T#xoR`W~=3a_|OkHT7bS_GW~W(S&cMZ2KT7@p1qD4@x1%HG zpDKJp^KQWdmhU-*>9HmmNQZ=c>YuF;s9s^V>oQ>mo)*Gh zQ{1aW{M~|2I$$-3$S7KgTeY@J>KP$%dzAeOpvDSr6X%RKA^q)PV_7t1f?q)WhOd+Y zBh=Anl$pdfmK9k!n&7C?R3>9=9TSZXAJ;GBH-;c*67p&1JaiuoB0ckNOT^;dsCcyb z&gminXmWe;UaV{0Z1r@HR)pthOS<|FqXZ(Io2ddaCN^qFYn?@Fj{-3;zvm_P;1)in z%S8ur3POxX7(szc#8d78O{Dnyx@sRcq@y)U?XUMvlI|#q}noDQb*0g&!ze zkwAU+3ALTg$(eXCTVYpgY)T0sf~ROs9Hcv!#FjhaQcGHY9-`N*T$7;A+0NHlBMqzI z5h(+nn6)3{p!3aL{uF?%Guzg3Bx}dvUyrlX#DGl1J9#@*0MONbCIR5L@j&kP2v(qv z(+Hf}9=1IaV|s%4Xs^*)KMq^GYsC88&8xL1Yf~InPp9#O>D?>r z6I^Ee77t?Px%MpMo}~fKv96M5B?A+My!+bW7Nfj%nubpP05nPY9*e&V(e2W=@&5TJ z;l%{#b=8ik71jw%hafC%qE0pVAq9gh$_YN!2ziY;g{dCz*RRH5fWKJ3;734-Afat^ z3ZLaud=;ii0fH|^NpFhxJH_m=KyP1$zm-g3ZL?y3_ePGX<>POOWYXyu9+rd2bua#l z_UtKsBY#G2Dc`>DdvExZ*=m9&zR>z?%L)c{|m421@&f@#>pBb zC5xRB)0Mz!_8ztyC&N|C@EhS0Z2~!QD5mfuvWN#7asaIz`zW|$u222f`F6y!JrY_p z#+U6ADh+}h{%L26d{RPT8ML-adTM1BU5>V!s++sCaBJ;*lqi(>UGIz3b!NR#(4V?i z@6kO3ua4&Rxs}cZ&X7e+ap1xHV@OVUvfEom;d?kOE*Y{z!cMeceXT^mJj*=ADApJ= zrHV0%h)QS1G@AAj6;zZEW!$Mr@oXU~6qsKFaA#CxF@|Zcs>CfATZ%8j;@~8ZJsK_N z(YsNL0u*}5B1xv^{2Tw0Kf--%L?;E{o3l>W?OOnhnC*UeK&w+hU1GgUuWx2FJnoR z&nChgKXx-AtI($Ar2(Jl9yU0R8Th8@%GW;R%Xv)V>bF@0UCrjaMBCq7GM|)k-)eF2 zeKf@K3*O|pcTAbPQpd$>a;mzxJvPe}epuSy?$M@!aoa{(lb+tmX=I_hVG+x)ZDYqI z<>7e2Bw;bb4rA~xs$tvUGY@HF%uH9TR2yUD6Q-_i*in-lE_7A!OLI#-Q5LUXaD!>x z>EGw-Wx;brewB9a(LNS@b79Z zj*e{XHVt136|WIsdyh;{(jUPnWL+_jcI6EnvUt^d_^1MeyU?HcZwv z{{5jQSBXoqruJK3qV=uOhSEVHZRP)H>=(_RT%edsCIqeIUVLR zNIMMJrlI(X2H%7&!xAy}lORZNaae~@P<7=NgBcn4E)Z!PS4xLdR0}vpRIx>5>P z|71CPw{|l^D>Qiv+dhl<_29o%isg?Z4j(L(9(cym-Z7j7eJfbVqE3Go=+w}d86xQU z6rW-6+WA#$4$b%1ZozwLH_a~-;!&1zGHPOwwzV_v$3(;2J-X%$O$`X5d`zicKc6wTpo>vD zi^pS;O&cng3tc^rOzA0^lz;O(>FpBUYn(YV*@||cpFtGO{>wdZ9gF=(pLv!s^YTy3 zTNXNGbyu<^x=9>?ctyz`Yf;}}`tE&$yY$~9+B~=VvF~}F?fZ_)Zf$?(nLRr%o?E!V zFy^ZBpN9eZV|XD;wr=zRdtm=@0_L=D8yoN)-%-o?UjNusNivVgfrjkS3z5b zf<%Br^1~jtEand*snS)k+)O@&wZynUMklgC8{s9h-@}z1K`776Oe_t({6jM3T9_0{ zWB2wbfp+{&(P#MmFjbORWv3|o+kw0%b_46p5Tk@Ok2^6pCc0s~z%#X%MD<-lyav z6v|7ISyxK7rHNW3v?4$aTJy(ii-q?~*(DiJ6j6<-x<$Ddc=)0Jxrzt7y zp6%^E$2duK zL6E$KM9E|fM3T5naw!GYP1Q8Vw+zZPGsdw1F-D`4?+TmkDan>iB>>kgji+$AvnkDW z3R+QAixjMzdqDc`Gl->SGPe;ixqRu&yD=8_F_A3unrHMOzx4T+9;V@=+k*V#(l{Zc zwbGHf6bpUi47V4_0gl+2=SO_-z@9mf{Plm3z+pkN*~Qo=5?pkFV5#x0CyFXyI+#L` zJNHZCtrAf=Sqb9YY4K#G;9-uSm7p2F6=D0)r3b3sAzw7ol`e@E7G@uYOU}bZE(WP5 z^_jUSDPQ8oXp`D5TsrU5*%GKSG9d@E;zE~1|Tujho>TA5+6_(#o21#*Cp;e|dnbgBTiG`ZTt>vGl3jJX6T zXTy)#QmvNpf_5X8MRazM2f1*jV*Nq`hXxgP@hK);3$rvA2~zZWdj{VE?P~N9w~>k2 zZ(*H#O+<<-wy}rR(&egkC1I^ge>~88pf33qRt50q7z6wJQTq0Uy!cpE8X_3O4 zlZA;wrkrqsTVr3tjcmO%oIo%~-UNV+^O-j4lomL6k!1q-7YUSxn# zd-%BdFTK;m#Q~qtZ#ICZOaUr5sa@D}uo$eDP@;SLgkaYumk=?U1^uvGY+R^w9K>EC z4+1pDFryPuvF_>5)rUD5h)-qgeic290SsZE%{&%=lOVg}GKO{S-l(h-#UzJsW0+O* zt6EqT$no9w9-{Z96FsTk;!iLPz0uBY-BfEI^Q+H1DKBCW*(kY?|ZcMG@#x^5+I2O6+f8iUNVoIX=UQOgz;-HS#IBN zBx8o;JdgUz8vg)u{ZoquG6>LUv7rZ6V`u3jY$qgz-ly2sN{kiUseKIeZW8|e{O8_G zFED;D0m}X+Fba|?<*_BxD^OXFNSJ`2Vs>$(s+Jlh<~5ZIWgTo$VzPlN8BOyR;&N{z zJ}ti-l9KoIJKs&pza3{`BmabFrKSK}YMOt#h7>fZ?2a!V|I!Hq6;jAbaEED{Jj{tl z^I@5>1G29IH(EbWQEY81yHc)6Vnmtly*rq`5H0*MTeUDAbnPZ+3hy0>dJHIUEc3v!|Pn&qz1n_v^m#Vt4yqlq+zVf-lU;V)WE z$*(`7jHnt#olU1~Ysb}Nzcw%d2YDjtnPlm9onxW_rwW$AYFnb=v`9T;teqN`MaJPoFkUD3db? ze#F&lu(jPZ16O?o*?*d(GOj=L(!%b=YGTB$^+~`)V)~^d^aBTm3LL5W3}7_AdnWl7 zRWShic#VKtVT(JG75qn7{mq*uUgj|4R)95_{VQwijUivAuBy~(dXc86F>nM+e)|T06D`Yr^&BH8f-oT zlG$aMo}E&gjjC5Gn9zs?xN|(twO0K7Q4Y-nZnKaXax$%9_76)mv-}fZ^<7Q0dz0eWeWf9nMpPYvC51V;Zga0d5thN`zOW&PPY!>0!2*FKg`FkTVq@4QNl>ch_UY%=!n?o zn0oDpk6pI3jQ%9c5DZ{NY--G>{{y80Yw-+8@18Y-H1W z24pXwKDgq#;6BQkUj42yb$>(5n{(o!^1W;rXFCc9sEK4k{%k zIKOB>thdp25&!?ubk%Q7wr?L)K>`DC=tN%zV7SnDnr!-hv;-5(Q2mTz8jR2us6!}*;>Bj^M{pV-fjuvKqg0yl zNs32xMKqMYkVop&5hhqEpA zeo0_iMr7*d%=g2Tu-u0hyZqi8EU{Gd^{lk@Q zn83#RscaXAh#FuLE+_Ixq-U=qNiNVc79jhW-0scT$|q?Ws`L@LXTsE1MlL!3T1qTi z9!vS73=)5xR{_@6Yn|)xO@b!M*Shu&OvHQ+-kc5;d(>*``9yy+AU~`L9DeV}0ZAzSz11m>gS(!jA9G<9noN4`GbYmcGhA$o z`MrHqY<}LCPMmE{!_WLo%sq_k53FyI6uj@rQ&QcfskSRW2%js_x&E;iq<@v-8?_+e zNrG-xf?69KK13w&ZW5rMJ`bVyoER}?Lg1yow9ADvOe?V&u>LyVBqjGEyv>njXnVcc z5m0&3g6hA?F>}8og9I*~^{czzOC!#EirJ;5srE` zJq&yZ-eTQU@(-Z7_fLA#O@jxV{)qgGV2})`8n=2Dby1#T$10u1DxjVM9QYF#OPc#j z=S8ccJwC52vL{-q(xv6d=lImz5moyEhtw%XE-UTe^@V{g0X04q`o6;d5s_v1o+5bu zX7U#O!tZsP?rOEke4UJr@2cNz#^l4dQ{}6$uw`Rz~U=ohRwPhJx{he{25v1Pc8kNEvn``wjG* zvH9uFr2aRjg|CyOQPV2^5o|ltQU!BZ0w(`^}fPMrC_pZ#JIfSbCh#J)?l~Ns`)fRDKdp(o_thc7$#! za2RCGFD;6AtK%$^&~KdHXVXqoBTuy2kD8tisb5W z8{0SDwnNRHhl8>&=3gfM#>d@AM!RIeUY0b35Y;N@m*)g(Aaoqq9StUytW}3=W1wT8 z(>{w-!*rwphc^9|oeQc+IFe2;yU%zG)Awrs^Q;_9LK)6d zV`$gCO|hIJ!tRYH{ubl}bw53_W|*uunZ!;ywx%q3%)JR6GS;y{D!qrKK%%HGM^Kjb z-!SwT?=;*~I96m8k6A|wzvEXdkDnYHe~B4HsF2I|UySuvagwMNBjcN-+-Cl=wLE>Z zPVt}W=n)wME3+bso^ROhEp^O)Sr&Hv+T>}r5%a=~nQWh5M$a?j#0s(1JYtLBd(V_5 zrJezo#6vnZ4OUT26|LwJ=?~j7#T9;F19Ex(-piM_QGfVRIj@m=rcVgRW8<&HGahW9 z?+jwLS{cyT8_qz;%W2>}xxV+LxQKKRyqv+kp16&UEoU$lOwm6~Hsxw@nhz}%AcdUn z*yRP0>V4qUfT2fnur$Fm>&B0)sP{i!>v^b-M12+sJ<=Y249ob)$E;u)*I@Pq_i{e$ z??;m_Yd%JGpQPfNRkljb^FK_~5<4oH>^RksxU5n_^wg4%uK5Ic7yA*PgnxXB=*tDZ z**Z`nkK=o-iPQ&|8Oy89mW`JKsQ+t-gpY(b%G#$|IE9yq#N?-e2h4S}&BFopq-})X zFwqjr;YK)uWa#GPc9|X%!%EgOggZqc_Y>v|*yq)Z(}%u?KcyN3W0?knW7+T9)#iDI zOSfdrr(I3jwm`rpxRtHbKMnAeiha*C`|Z_l`shGMy$n=$BwwT3uok$fQwnU@svwy# z`go2WE0?y*rf-?n^`;z(dr8FWE6zz~7e$8NR*gqMd3|_PP!7{@@t{Fu!i;l4Dt;Az9V14J z24+Nn_KQM(h5bu3xjtxNG*Bsnoy`$UbeuQ2O15E6V{@9Gn7NEZu>|Hf$aUcxY;#C# zg3xfct*|e?|GFD(iI96g{(Y|d|11C~`eByy%^f~4_H|Q+D-6E+0ftnFu2a%S>t4>Y z{sHm0`tjpSY&QRf#j(7s<~wb5?z`>NBcGuDPMiOBD=JXf6xUz~5cqQN5xLu4qat`v zt25}(!8T)iNFntz%1r{Xr=g#Qnu(nKzzsbBI;ckKGcGQDa>IM!s|>H2RAgJVBzArQ zAw$;kO5E@FC1DO$*Xity&9vEG?7a0|@}9brtHJCfh4a0jMSSq3fkpAJdN?x2b7I3a zIf0SSX9xYhP1H-Hn4AaRelCZylQYfKTeVl+E$o^E!#e=G4J7~Z>mk{u(-VqogU8{n0H*N4d>GWozFpd>$u-i=7m;PVGUv7J0mJ-N`12C} zpmbCSnoHju&+Ut1_={4ILfZ(j6T8rr`s$;Ml_BwARKa0cK>JE7SKYP7*-HkxiQDM0^fgSz0e#(y@JLVQoYx`}97 zM}qXymV}#@KkpT|EL((Zc~&Av_rIaA8(0!E4v*UC7zVZ-4nefBAsx(9W2D|#0T=J4 z?<8m!`2Lh%qZT=+rJx=TJ-O^tZn5t>^_ihblm+4 zKa0qMJTb45h();B+czRPZ?{Y7)4##IIEdocOOVsd_6(Q-1Tu8zOm}ea)(QP3AJdL> z@|ztf9ZGw9!*XgFlYeXF{N~c>&G+dvglVQ-YQx(r_*>KGZp(ROzvCf8Q}mb^jx83M zf~X~9Bt^}tgoP91VBJc0b?&Pl(k{yHPA|HnNhngwIE>5V0``77AGDKj$;!%qWP63b zF8EZ(`IzovoMuPqqOOlT@8tcv9VMyvze_19op^BAe!8hZ013i3N6Q-owwUuI;6+(6r<`3cSrw)Fn6)aQX%f4$>o<|qlhLP7W@pJ-omC%gO3Gz)xn;=OWn=)QIMX~iYgL=+;4;r zYV%)la(l5s>MZA24j|=nXU^OW=OQV|om=51v_&=)AkQKwjmPlV8oOkeh+MN6=SvJ7k&Hbzt zUGSN1z<93s?N6pNDMa*x>#v)i`zA|{QE%^v4Rh53h(>qmUfQ#ou>_z$4kK&-Jq+zGyI^sFU0$jfFZ*wMs~vP(kH6(0GaOT=*q18T zOnc-Zbjq?EPeQ|q9MZSQ#BWkwU-Xfabbs9ke0v8jd$%7(V3zNU8JapS=w$Y>QS*-# zfQ@lFo-h{l;HFWYfad>`+#e--INw3i1k`M=r>@&;NnSM42@h8-SnW2I2PtIOsW+W1 z7=@}0uU+!MHqbdWotsg6lbI3>8(GiSnLADc0-1egWrl*$c}z7W;>g*Bnw9zwoCSm< zb$vLbANT8LumH$Zi(Hf@3Cl9wfX8W>#V)T`YMbwwXt6}1t*OZVlD9$(F&fx4@DtYY z(6a`*8CCkP)v4!L)Ip+IMs6#=d0z5xw)U)b9=CxUGo@w|A94B|V{(dZ6P zT^@Cs_HP}RqVg8ddXNZ}Kvofnc^^KKJ5NE_H1|R>Xaa+eW-8BM$R!i7=i>eb?|80& z($3QEpttLPAzot!Fn(()sBP@0@aKiE*=$i&b+SqGwCV#ysD0hNi52VDVf6e5RPa7T z(FOVP!f=ux$mf;=D0(ytTpK4#cQRgD{uOw8a=XC!V3?L9`&=NBo=0|lrnd9}wGb42 zH4w^V+AGK0u7H|Pbi2XJd}QB~oHJ2C{b0fPLK8Y11#ow$Mw$$l>MMZ#yczb^E`D)> zZCA=DY;g%?z3V5Fk4PpH3N@9zP2`u52Bw%8v&gVghah~#eys~o0_2zM*La#sgSX&X zYKK%o?MajVy{J@}H-g0KpPjq3Y%uyI=F9?}ru%jTZs7V}=l1~OgL=Cn9!eTV!W#W^ zo4~qXOJ3(fF8YUWexlhH@W=jGMbJEFfi98)ST9@u43i5vP*01@f%VhTJ8d<-0Q#sy z>a~qz(Tc(22-ApUoo?vJ*U^&Z->TfmoZ-(wb9(Wnj_4)bMboFrD*A7Ef-iq)X)!-= zDczJj7k+x-(znC-5=(Y-of?aFbH-^jwm$}gufYf{f~o0$g;m-vJn|zgeP@r zqWTz8I81?#A^7ffuLr}$?Mg}#1Mq@I{!7Hu-X%H?QN20p!osH$x(6t3NXs}6%Cb-Q zf-Z?ljvwMbL5)bpRyXY7fe8~ZWI~V+VD2MG5&R(wl7hh$A#-rI*CVZR<|di@P1VQ9 zZ6va5TfN8uB3oCPQoHH2zNRZap_8hP%RX#W#DLtTJkpX-mIS7jpd=A4bH{A~%J%lLGmo~ahDNVl-rXT!r%XTDvPRvz*J5O1W$g?O>D5HLK8M z9ley#re)&|tyhNKKk7aAiwFxqRmI8M7(BtpN%%Z}O7U1;3|RK2iu~ylx6jIGIVfF- zrGu$)YX~>kt8oW~x3bXB(DD;2g{WA_eCUbM+or0DR6|uJ0I)Jm%124*4$O=X#-s+% zWuj#q3%|(7nIQn62Gy& z?N?r3R&)>Mn9gLqN7leCE=v~Xvn9+UHf{BSR!i}F7<8f3t2Ebkv?h-a^+g671XrC` z^h&|T)Zv08hwvZ2V?BVpD1q&D+O+VMu?p3`HI=drs=&hQ?})Q2$i$yIU|~9j_RH=Y z5(5T|^_pyov>~FWDn7w>niI5LFkH)sTs{@MN~Gg_v&%L}L|e5GxYM24`NPzspSa%G z^}byR6lAx{6!9U$_207Z;S)6B%bg;^>o2I!S#gZ0-r*+fPZa;3s7-I{-@6+SWNW;= zYKnze84!T5`-$Gp+lh=!WD)Z^Uip%TmD5I0BOtf?L~gL24{vFG59d0qhBPx1g4*WR zl^nJv-F{)l=Rw#z*(gm4X>Oab-iBRgl3TzHZ|!=cMUn7@aLrlv*YTa(dk9yeZn zvPNVGs%ZpML%{NDSw^YQZUo^WAs!x;<6}}_QoCANRgWHO^Eu*y6o^oC3q-{L)6){> zCE=S@SpsnVc~@>huVd3zy|dU{j46+<>^cD@!r=Jt(TZ9v*mIX@OEErE1S|-&>vDwQ zI@Q1|?r%;~O}PPWJ9-#OAReZvJPm-dCN@CK2UqM?Q(|!V@*l;zGh09X*@n0+!l`}o zexOv4V|KUeM-cAYWt3WmusM{XOSPT+QS?Z9{p&YB`x!&zpbLCqPu?7^zJTQHWC*e5 zi=q*EMVjQw?Byev^$THVfwSHhGBQ1mm#FFsGO*Po#v3KP<90_!I^+wGx^O&mfmTr%v=&nO*9 z*BFh-Y&01KeSH%<1a**-h*lBP`|=l_u2etiA*71Rf~_0K*#v2-cq6bO0`>W2ziV4; z@1$$iwI2p(+ao`H;R|{_LC7~}%#|cllZPps6~1H5^6hsj8hqbJwwSty03`q0*sB`uXPpT}UgVi<7$rSC;+eXUk5F+(su*)A8XBr+!f8r^1# z`hdU!`V^!LhvV~{twozjPv3{(eg8xvY_m=crz(y#7)7XH8yAAx{;@mhMs7*H)7w<* zz$2QDj&J;7%vha~4r7(iCh`&j4fhXMjG!n-8re-)P4q3s?lWN&Ey^`pO31S{6nC=j zJRv$AwY5yf3E@m)x?R$9edn^ARM2i$OU4CxM(i@=t+CbUx+604>c-_mRY#*^s2g~q zBV=V$=s_qUsBGj`Yf1E9aT)DAAQhVR>p3zs0_wA&p6F&l4b%EBTS9+ODbZcj|Ecmm zwPKxkP4U7j3KoD`-IMBly?CY*Pc`JPN%s&#fmfFKe$PS$-RXbJKgF$b*BN7A;|iV> zOqU{SMNOUo-M3#o-Zy;iZ>~yXP}ulLl0IxMJcg{t53U>z9DD(0ugN z7ODSkIi0=YhBxe&e^kM8fn#%MUsYSRhvVJeTz1+btfiI&{Cl5c9 zC#emSo%Xd(2A}i;R|7D;Ux+L$5NaG}=m)^9m^GQdcJi&4;D(*{s|<60bXwQrJ8$zI zUe#zI?trG?9CZ}oHS#pZekDQW8I+)x2)=!wJT-bNmHGP{z~HZf-75d932l)_L&GS}AisUC^HjU3hUXG{@5AStnBX}##H=m&_ z=6Xf$jha49M`tJ%lgks+cYnc`!zJ%3)hjFrQFb`rw&CFMJE(eUw@8qDvnakqtz=4j z!A|rEVNPiv#-2IDw^)}#UauD$y6A>8XR0C$ zQE{VvprD-ws99R$)6NO>>OA8TpDp#p7stSZIrml2uvJ=bShs>W-IxS@AGdkfM*%iQDWe0$4-8!iH z9p3no;{9%Iw>@)?eHf=>5vRr$I(3-0z8=rgyxq%0rt$MtQO0W;sHi(;ik-)=C_QZ8 zyXftHXY5z|5t9FIGF*tA-b0W-N$r*@ubGr{tO*1v5$(69BzUTYw1~kh?eOb?6UL=$ z!$odhbg^9dyD%0&1!7sK>8SalqN#vaj%06PI7%vujG)aRL<8)q~GS$)LS_DS9-=3e1}I1KT?v|xpricSFv>71c}qI2M~l`R6Gz=twN+X zc3-Lcv3&@4+{^G{7ZMU|0u|WtKF{ndOU0Gc8E~6CZ_EPqY0cAoWNtvF%L%PUPHreQPx4B{&JzWqY$`l^Ov~R*ggt9INj4~YZxQu%nDuj;^FAaKW2#W=vuyl zfHXzC)x^vN9cKi`JNzqr6B-z`HToO=3@8JpfnrsFF16dBM>J8Qo#fDKIDnv#q%qR=qD0`dd)(KY((v)BM*cC+ehV{A1{Q6XdHI|*` z+CH|_o;&AtRt*D0oZA6;(B}`T(Q$AR)Cy45f_p1xMsPB(;huN57UjpO^ogH~=!Oqmsj6du0HvkJL?$Tk*(;eQVGN5G++t;zd zsfTvX$Y5z?4%a(XhRx3pW2{)lx_jMZyV^X35Vs>JP*F=oqRO zwln)C!S6MVMCtizTwd7;h*()g+oCu50NRe_R#wi7OQ*B#i#j&Dk@dOkRm0@PT-&xL zPABV%&$9iR)mzi$DsE{Ba7fWtE+@kTuG3^LoRrAiJLY_va zf(Nf*jK@b^%cpNO)jEM`psAVZXjZ;{uM66-Ubz;yrvaP;M?hDZ#Rr-_FnFFh+aN9JN9L79mI8`Q?~2G5e~&PpLGePafURc&9AwiX#Q)9ZsueE5fH|kdvlfG z3p4I-AN;Xc!SOf3E!lY%3xAMSS3lXvq^XC?v>L`0k_rt=k7CyEL)$SDZXbhW^6Nd4 z9{XITKJT9$IX#J^x|Jp`X4|2jLL-FrJ z--~QCi#h4)UDqZT$zB{p!uzZbtq(s+`BZgM9BC;eVcg{lm7V|7Ol+bPcNw1O{W>lU`Lh2FF2U`M-+N z>q2*F|0E3FydB(ovB@dO&a?ktfi^cVqAUmx4Z^Sy?wgh@LNGrFr8d14GHz{h_=gZ$kHFcT0oX{VUyxRuI(!wlA z#&?)KyXL|>`)KBtpixtu^kQ%LCGPf_KkV)VG0m~n(nfNrU9k$0xU&nF2&+(QsVT*5 zTgGfc)8a|Dovhj;Nli|+d28(i>5#=LnRI(;KdYGl%)&GcSjf34;`$j*%=Cd|6|_jn zmUyQB$5V4cql)g>R45V;jlC{Mn(lBL!o4qw&ey)fBviy?SF)T=S41(pNIJUVW-EgT z7+mXzNc!O?b8p>#tz*0*wGj~a(Vmj&S7hL7Z4DDJ;7P&IH$Kb1H?a7!7C1$QB=H~T=+6{HC(V~!WQtoBUQBqJD02ADq>>*OTvY$1_1P5JF{EfP z3-8bK&tWMG{fy!I{I`t4!x(UMYZ^PS4>~chP^b1#_;XrMnut>3(jw7!F4JpQS{m{W z6*04OQ2-UdBVA1S3kknWC#9c=`7G>xm8`wLP7uuheOBAG1Diyf%H{kl1z%o*L^ek~ zT)Mn=_a*}$np?<#yCD<*#Mm4s#65X22W~ufJ+*)Jc(CTu^8vS@JhSgqoOz$*bPqgB z0g!tV--#?qI@=IL$|V??Gz4Ar-%mV(1WAG$eGXF(ia`JMB^j+i_8zae6|w>wgjm;E z`yL@ZcPF6yC(R-WA&(q8%Gs;JYq22#Z7pZrEd+1W8lQ3g7nS&217Go+3eXTIgOs)U zO5_uk!mQA<|IPT{j7`~$Vpma7r{1yPshQH4?x&xxTwko4r{Uy3TJ~Jma{w@((T|p3 z#al-?!k&Zbz=r4~k5a8HDO*J5nrYC9rxqr~cDe+Pz@V!>;~d>uNsRj9;%rk}e08^a zy31;w6l;3voQ{fqKZ|X0sHaM21PaQx-m%4IM?+J!YNK5k3PzKQa&f`;hvT-Trd?ND4$^)`-`d+5y5)|A43t%&~uMN^}-$NDZxoQd0f zmfw6qa$CudV?hJi`X6$@sM5yABO(bt?#x$`=^$ZbV21|_!Uu>5JI_DOrd;J8IiSB^ z)Ikc`Ze4bUuuyEPwVnUF6h;+C@I5~!LtYt339*tO!c;Izv^a}Bp93!wx2%F{HcM5` z^#=f!gs9hR@X(K82|LthsKPjeifCFkUI7!hNlg1NhZtNQX=wA3Sgf3?E(VBDL5?F^ zoUEHWb;_~AaCnS0TzEi>d5Y7RlL7pw8d=B&3J266L%Z{-RH!hK5bZBx6%Ypsd>uQK z-UCycO-TT!;S2CZ?gG&dHcAD3)aj2PVpa6`ko`JqNP@$UvIF{$OPu12*ZOE~I78x7 z@C*#Qng?Lvb7qirv7-YJ8x2B+|Gv5fuSiL)^5zKkp#9%OZz!sS$r9VY4a-lE63Z1@ z$bhvJPojcMgvNP4^C~!Gh=pT!#VgQ%_OArWGDg;clE1s6>aZebl9YAML4f@;47TZ% z8F0=^{VshmF%m7E^+0n<9h}=x-1Yb6if@rt+m0k9{tz9iXIZd@|RIJ3ud{7 zqI{JKTSnzPlG;R8>X>rGU6b+0RAUB_bi(UTz89{=q(*amlldvIQXQZFZZo0K6e#bv z)FuSk0EtGVbX756Mt6FSNR6y=+5TRO=8Yk5bF&1-*R?wXV7SX?^{{VNTauG7@|e$? zA)hpn9m1n)7!#}!kQZW>*N-}C-HJ-{<7H;J+`7+!AsAm!Bn{kUk#kuJt5zDY$8%Uk z5#TnUowwOeb|{lxJNLOwrPKc8_Fm@*eDr>BI5iV@jvbvGf}(F`M|0nL$(Ih0@EcZg ze^YGdP(Ps&pU!wp`dUpxXymcD;$~}EVW-(<6{~WSNmDeP!Jz9;HYPJ@YghvAOgc_} z&DZaKmm@Fg4s2fR4ralO_t&A1K2)uxM3NeOB`oe@2Ob{$=cq98?d9F3_qdO%M;UEw z(lfcp5M}~Z0zfBl?|ZD-ir-~(3wYy*GnH}av|&Dx$qfm&g9b=Vqu0L!;BE=tbNPFf zeM-+L%`8srwC=7#LWBEhsOax3tGKE=jL*)@e0s8=n7L(-<);^buCwW=Ge{~+H|KiE z{NPONzjpp8d((wHc)+}d?^^Jme;Xpvo+*PXZ!>Nd?B)Sx54mp?UFa1}Ds@77GPnKc z==chYE;a`LTJbTt92_$0oVI?$ZK3FCpygE1K*uUcvac9~3P;do-#ycqiup!TZ z=aRzOF<`Ax7%W1qMFF(8=UJodTxdmd3a$F@Ym%*8+qD|;xuKDz!MCMrUG~-9__s_` zd)2(}sWw;?vPCycu zKky-)3(Gu$!iWw0Sb{Tuk>QX*J}#24;KfyNhRXzc5h{Rp@mVj-?bme^PYY=w@iVo3ZGQNheer#0s zO`NBE+h!}jw^rGRiiwVtPv=RR=@nz6n%z!Ol(760`K?EtX?aT#u8;r5Ik97V%Rr64 zP|T1pZP2>HYx_Cdi2h=R*{>jv+Ky-Y3<@8>e-qsv(Lh4`5HWC)X`E=~O+`I;AZrKK zqWA~~dMRsIufGM5P{~0w%b`X%m6;dm0b_4m97AF2Al?NbJUk)3kArp^x^5ms*;{Xh ziNI6)2&xM>xx^nTj)Sp>gqvB-k{tTVN024$v`LHeBGh_mA|%jyn~iwdHQVX)$3bBq zh78Lf@^p=N{NEr4bs1D7X*4ML4_6d0xSYn72FgnYe1hncX2^E=y-OF`%Z$tb1k4X4 zU3m6a41u^|Jm_3Aq7AO87p~pp@vCvP>jq?p@mfD59HJc31_n(rEit3*X0`xV20i0(cfuE%~lj2tId? z-K`j3BvVJ^-uURc=vGvj!+VH_-dhYaj}6*P6tMWx15dx}ab~sD$g7&(t>HFMpWHnkyVosHqEsGcoYo1PBA$b+h^Fh~i6o2;2i@ z)H2}tv38O_{>K1;`+%R1w$bu2Eq~IZ^SXS=gKr#m=zW(Rr{=IfqnqDpH^dGn;g`+zwJE>ujnr_8+DBMd+7hxIaw?MA#=sY=*AB{{@P8zt1oyJCFUsapFId@i*~efY9dCb-&&=YIj@G3@dNXcK-KVc(39oPO&pidpd&ze2I z_e28`6hWZ?cN$`R*{kXp%;4jsB}{BwP4b_wnDQ-BcoXSA72ON6W}1`8r2FDPH{=Hi z3dN7NBJRHA{)(%u-l)&>2E_*Qhr0>QOutC>W`huk#Rcx zT9=4;nP;`JE3GritHgS;ik{RYM1?kVZpL3x%HJfzbf_lggl@Q^vLmdIVuLn0`7)m; z0%nU_y8&7K<|Dq&mI4A20DKuD6x&*Kvzi~rW(up@OMo`k2}RRT^jJO4cN{dG&m!`4 z=dIC{ECVhOPe*l%DEcNu(ZZdv%3o?RV(mX2J5LqQpmf5)t#ze#r)?tXRSoqlX}k)@ za}!6V{{6Kw-0zmDzwu;x^>LNoyMF+G(h|F}N&}ow1d?DZRFAeyG*GX8poN2%W!~@FwSeYFujeA6|j(P`- z{q&-4v|YLJy@ushCZ*5T0%(NQo|wOChmy1}B~+%^Y_Yli;+3|?-TY=)?$?OIOvs4} zo(4z!iF5*5A6z&MLSY@bp-`blyFfDKy_$awk4d6{_qQrmL#a$lq=ud>QsCLUXuviB z<C`VLjEqyYl)by0ocN=1QA`@jtjT@j`(k5On3(Uu$8peU z2{Ur$QlcY$(I=NJ=gPb@X%|OfWrMruqi%Kra6e1Tl9#mk}K|u4M^giqm;5B9=?b3 zPXrtp?AX!bA9PKzQqaH|wzNi3e}Af?LiRwDYwn@P+!2*#&bXoULTIy@{IY0>*9c2O z4x&B;O*@F#?P||Vrk4{sc_=#U+^sDf{}d`4xm~9}BuuETUwlavdTBRlsWM-#mT|>P zOAOgP_`Nl>z7*srczGmu}!y@zq81F5zybUL*M!WG{C4l$^k&%3vOumMdB5{rc`1lw*#~ah~ z$^7@qDbY}Ri9QzN?}Os;f|F}_Dt6#+L*)jO=%pNb%N85$e9xvU z*SfL)>l*Krll*(^F+?ViaftE>7!gC8QX;*irC&yk{PoaB^WgX2lf2rTpc@}}^^_If-w%xQgq8~96elA}xu|jhEiq0XwZ_#f1uLGk ztK+|0oby4@q!q|z+>s(oEHW#6a1nM}(HZI7u<+T~| zupJ!yuUJ?DJ^Y~$3J~?^l{a2WkKCDQW2(CTT_6tdIhN`Cc#zP^0B~7u*&`xt`R88x z);VWz56lKUG4sDpi}>;DXQ&d`^`j-~`0+_Wgz=L;GE9dLN$DLlgMRTKeKwiE_!}xp z`Ze|2ahkG(mEj(I>Q$r22nq=&TaRA_ESyae$)Sa(lI`K5NMFa-8OzOXjgtj|@BxDD z0o$*yv!9%<)X%GSB9J*OKL2c!9{Za3H>ZXF$N3-a_DE9CKk~1H+K;E5)>slIRS_lWw(i;uVsY6b8=FZ<|j@96xe(=O@E{N|*SY~f*u&Kz|(TKkTG zPKGO8rh z-CoygniXboH_~05zlk^^L1)Xgus;`waB!P|l+Vd@wNlc5fHmiEjTRJQ;{JF{pFW=v zV-6^{q!Yauby=SIbW`=~>DgSu0Z^3uzh-5uFi|?M1ik=;K6;+q@oxgEjA8)(hAaz_ zV0UZwz?SGpu`9Qi3L2CnB-#4b_jtwPEIg7{Bsr{PxG)}47bX14)pKh49s~agELOsT zR=cbn#Mqa5q%cZwfb-s1sercp`4E8$F@`oJCHN7U#^I3g*Ta{L^e9)caXy_#ZeN0K z^sE)8YuG_2Vnn^D3b!e;>M`zUyu_IzyhleDqqvo2>=?ZcJ>(-_Luoq9sv26%FY@KN z$UBPX_9=Y%xO@$O-x`SJ1|cof*C}F{AR+^$htN_B13c3Kf0<_`wTZ`RM&O0{{D~_u zra=w2IQQcV3MK#i3Pr|2KBy1ZUKA~lrK|(6QApK7U_G-0z>Nb>JEd+9=sY|kM0)@< zEKGZd{V#r) za1Mk8Zr#?Kb>iak`irWbGKvNfhe+ZF!b?4SVXsKq`rYi{20Nu?tP?)y=5JW@VBG~C^) z4|5{!h1y0({Rg=d4!klq`bweHjJ!WM5!w`vaUbWr6+b=#!s72oUE7XJTR@jb41Y@4 zf#J>r=AFX-T=zb7Jv_K_zw>X-gZ+7_)p6YGCh_^~glKni{@q)B9I~l@N5hl~b)|}xk^!89bnQ@rG7v=$EYvu9gF zx-Hll_(xzHprS?A}|`-a9eeXY%N&oPa`$W4NJ zb|8a{u4DMalo#_0Ou)pS;%C0Yi?G%p`5CFr9dhZGi^Raw+i|s;7>PGp{{c^bI)NV+ z9(+Ats5Ly}%dvNECuQ+n#MDt~0!xQRIva_z>gD8&?047vFL&46e&85xK;Nm6)|Q_nxpXEoR;aE(T^o7RUC1YH7;z%L}Y)pW@zz zH4d|=J~mB+Qm7X@q;Q&Er<{>O&;TnCLu8p_kk84I%ewK=N!=uxAn3+=0*(fF?BTvr z(RVosA6uXiH10Jc=b$)%}_;AI!zRFAz#WeZ^Lo|Bd2B%*L}v zVU&@45iw)La+#qjx0o*Pii*nn)zA$|x$%|L!wSzi^T3nN$?}TRnwp`88l-QnFzh=~ z@;1y7aw6`#=FzSKCOfkn0Wh?zpv^cTM;&XX7ma_CBqAO!*g^eY5_`kTjT(Qh`8Atv z^Fv#O0;Xc1Ya7A$n9*I3?!R!^uxB|~D~~bKZMFTkSlQLydr7p+rQ5?CwGyH7m$Y|+ zJ{ugjd9f%y8{|*lGxR|%__j;j6|-Uiuzt>OB*5-m|1eSd{aUUp1DR2U{_DSHwGVdh zm4qvEzB?q2|4+cSGS}yS!Rnx<87{dASfE?|;;fRc)Y)oy2#??H;w9{12lPE~BECGu zQWG;&U3cO1({I@V`Y_Wl`u6OTP7-vYWI77IQUCkLRjFI+X2vfyqJ+ZjI`FA#`$OJ+ z=iXz$mfDW3{}CdzYIm=!+wM$B>u#kNj|k#3VFVM|GfCl;vOgvK^wI5Z!~hV4D(AJ? zC$+wX$ABqWY`3vr_2Im-RMUMQz1(3Z+9>QyZW54XGw60T({k2y$wRyhltsYi=NZTCX)e|5q34k6h_FPkyJ2{_}=m<_j5sjf*~yUp9QEq?e%K(CCcY zOr1EV6qqw2quXYOt;|a^H_g5vkG&~Y>!*miC3v^#(A-l>!rUNv@>YpLcFJ3Zuya<} zegH2j%MzzlLV5FXswv|4r%aYlk6zlibsWBDn`|;u6<*)p{U#LLr$h>K=>3}nL~jo# z_ho=Q6ofuA4nwb6{SL}o-*^4_PrNI5`E6dn;X$$gmKAd6Dk6zGz<#Oa(p>zc;kF&& zKYY(AA-I$Lx=$%#1yRgvZt;bN?^$xDY6Er7rx#>?q)i!M3d_l)<6pNufiv6XLw0L{ zw~yV;Eq?8MGz+$I4UHKUvC37rBc^*?n&&@4HzD?=y2n73jJ}{o-9rN{ONZX(zW0b< z%QPBREKWDQf_EmHR`79cTE%nFLiEho9Hk8;4Zc2gGFPxaf1kVP`vl-dx8{qF>-n0 zH4JR}l=p9cb9Ync3Y!T=`@3mq;>oY=q`Uq0aw~JAqckbkDcu`Sp2<#dCLT2fu8zwgW{uR~us}`pWjV7T!A%#>-oIY@z4i zeAx?#daN@cQ?}4njtP}Hu4-+(bJs}W<2LDua|`n!0K4MBi~pnPyyL0--#=a-A-l2? zIz`CdWSvt9A(XP?C_Ag{ahyYv>}**lJ6mSfIkphT-W(izAM>2!jGylxzdz3(_j#Oq zocsO0uIu%BJsnqy5G|u6DDphBWnHYl;!gyf@l8kET-VZ^6i%)f7X|0NQis)$p~W#{_^ALfbLX;@_2f8M{=!!eA^p@22YGH&V;b)b0eDL?t>qO zL{alM@(%kPL8F4rBU*y@lX0VKt}6RJ2U)QD1twIUGB6cN8d~tAv1wpP|y}2GIW$L@4k49H7u6g%APuS2=>rUhnzo&QyN~(Od3- zu`Gx!Ye-`4q~!dM<^;Z^rtFn4Tp6|t-WcXAVcnUCGSJSP+*;>Xc}6p^Co%KQbEOCx zR(9ghpeENg%8vdF8Yfzu+-zJ1U;FmuX;i^yRe5|Ri#+AW7_Lz(g}gOZIDgj59T!Bo z>$#yCzYeT=(UNMhxpgIPUJ>NEtASRjiH7e+4l>Fr&-Rk0B`AF??tgYy@{q*KQBt!_ zeT21jB1pDKFN87jGRJFcneIFrG7LONpDLbnXk$)xxyCL_4u*XMX&nVU5ZZC88d)Nc z_9P{U9r3j9N*~cO;PbFm2&crmN*A@Lq3qLXtvFC}%pMT}A=GyGtS4y+s^QukfCO^0 zJ0)t>G}?7#eCqt%g1o^Rxc684fSYW?{UY^BqDW6^1=_h1*PR-xYCDuzvu93F5;c8# zc}l0}8I)^(nM~gL?3y6z?zk^C91BG|-2WU*yn$PtBY(t5AHYzukd?>r^ZjF?towpB z`>m%6`%bBl@0N!JvhG8#YYq3s&KEy%!wn*^A_zHq)GZV;KlfB%b7kaR~xAN#6xe60PH&33IoC+e< zXn{`_YF;p4+J_Q&cRdIGOC-`xS^A8OIAk`i)vGX7pnV{Krh`RRVD))dGub}UiN@n6 z8c?aoHFdq^BA*0~y6?-q+UPGT{s>0r{N5^Hi znG|i^*X}{}I^bRxnx-jxAFW2Pf&vc_zNFL1=Z0LIAFP9kL|pcPD%N5HfpryZEXa!z zo50b{dz%oCbhUzZi)YB|ijrF<_u62DVxfaoUsiWK=Afd9m?n$pX4mB9x4Rt7{o8td zm{^-B?R~U|L~c!!e6nm3Y{Ejube1iHz8|#iO)43HgrOQ8@+Uu3e3=9Nmv<0kx(4(l z#VRRLG?|7?D_OBNNxuinu2{BJ*AS4RIVsseeGS_C-#E#v@Ntad;8Y+y{;O935G z3PmMV)R#Iy)o4al#2l6rKGrT7yRnPxDB{moRmru9 zG4L$(U3@f4WU3UqGb(*g0Rxvj&f&f^^yM@O41cACjB+c6?X4XYh5TedgeJ)hTWPes z%sPIkocqB2$)ol*cR&5S7HH<^2sr^XI!@2-R5YG7L3rejg3X@X(OwkB3Pb4F6uV9R2PaC1 z%gWc2-L3Vi1QF8~_IOV58sZ~jcw{Fe$B}49efwKa!zOrZY!y^)Skj)VWR-sJ*_%p# zR{IJ0)b|XW?|-ozI(53uUrUoKN;9^U=PdHrym?K03U9DU_!`I6c=VyP7UwBic~i8k zI)S>VU5DZWIhDp8_S%Ilvq}tz>n0cYhas<>&dCCA2i^03d_$%q`V=nHSk%?f@}AeH z9IL0s6qj^l!k*j8Cb4irmq)*t466Tyr8CwM|CsW?b*nS(S}I{1)6=FAY|GtTE_sTY zeCdjFRw}2(1m50E{>Yjk{i$!qWV_lB{uz8Sa#fuV_7uFjx$K^34>=4d{5OCbjz+l# zsXB@iy&pVH>q+-u6w&F1D9JjgfQ_Da=V=NdCc?ay|33>r3y!_q66-;lPo`q5NTNiS z@L!lv+pj&6gqns3Ng3-A$8W*?F5hp`V+{ho-6d%|D43XVI9l~`x-+?)JlXSNLyHF| z<8?Wseni6B9h_OwpHZ75eE_XzrVoPI5t5(nUYN0iM&zH>0SD>DW3rAp?|S2?o1tvTAn5P9`>e-SL8|MD#gULLZyKOwXNcct&TXA> z|Gcc)pOJks_YNQfI64MsmfY(A^s_h38LifX-cmx!(TBf{|smau6 zb4npWo?eqK$a->o>c;5~?&ol#kXk@T_*sA;K>89H9QbBox*&-(33}^0;)|-y^83sI zasBzJzQ%tj(Ex%^>jWRKH%PSZMg{CN#2!$=nlXn9^>BHc5_P&Sn_0wvHI>ipD;hJk z5@Z$l-AbWCi@pGjXW5D|TdL7u2a87l!Moxo=f?$UFL(c463nBw0uK)^ZG^|hvUf4I zk2H0MYH^9Hy|AUAUf6MFg9~NfC#ZP!F-V(S>WMU%iu~;sjuf4ggjNNJY9j`BE}3Qt zG1?x^KV>P^4PtL6RI&Vlo>;y&{#U*ssBC%jao3(I8iP$iE`0XR)_!Q^^xm*Y+uyk0^u61J~L6e=w{3i<_7k{F4 zw$`O$XJ-4lwV+4Ba{c@gMN4+DLVDVaps(t>C&b$wKSpO3W#js#!!P%;?q&fqpET!} zpF{TS6pA{jV8~-kvwLp?rv^jZSG@<^emF91ER-??Y9cEffQQpzLJOdfq-<1pp^X`> zpRRk?bzpJ3&US0mDNHF6)zLS`{16zA-kG}2)(0Sr}%PylVw|(cM90gOO;ZbT(w-ZSW zRn~-1ugT}QkI*g_X1=fe2C_e>kJoSyDopC@>iIWa-k2Rl5uCmI>N|Z56W@c$&x40re6@BawlUy-35w2V{&;?+bGRNeD2Dy=eV`1oW7%l@O0+>1#5BM6x|0ZD5KNQq=pM)QS)B zV%5+=ss(M*Q%l$(-oYTLI>RSkxNrq2Qao10QLALm`!#wTeaV29pqkez-& z`j0!tG%8L^~Q|;u`P23bK|*JiZ~ZKyYx+VKI(y%m9IIeR((}d9!0% z2FW{`TZ8-l3eFd<<))GTL46tYg&cbF>NFje){An3BpMWnkaWpu-)Urd45_hMA023| zegoLqa!12WwG#Cq5p_Ugt!(K;^YLbTvQCc);Q@@%shq`AAoWTlP%!*GExbz>lnc{z zS(FgP)P%;|QXXg^_7>`s7vxhZN4xHNEeHEI^GLxNsxK}|pq@DqXtTs4~v|MJvtxYSsUg$ry5z`4dJ*VvVgD>;L z&Zq4~i-z7=95ihdR{AsfD*RBk8(nB1;+qCw_%!&A!kppc zYC3_M$+x}>Caw6rmLS3gn&fkmRxpDY910wQUoh5Zr^#bF4zh%El`gfRy>T3i2`>}( ze9+B8FqiAi$~p9~4u@^VOMqZd8Lx`*DSJeMhW&G&zGURKfT*3KoY*?-HOWw0X8HPy zh98WNI%I-wA!)XjSU>}IA3^@04s%9g{mbA>bLA4MWOp81wx))9|rY?JHWAKZogk``%ZXIs;#b$ zSv7ReX+?7lHe7-Fp4ve!dB0x*H6`SGn9K3LBmWsrWC|sU4g`^$qrvA37eTfYb(2jy z>man>njMQ;+Z4}uw2DubfBz>7w`2X;MDFcx%26B(UK0gK)$U~lrTqa=;|p?aF|2 zAbNE;NLF2uQzdtfmE00V9uCN2Qk-|2YE_&ePKZ`E4t3yBN6UGDOz3titeG8vO^?TG zf%!`fa$Ez+Ob8x(R5qRHPrs%5dnyOV?RJ;=78T#OUT0>gBi%~|PLv=B*XEsSNlfwG zj|N(&;CC(@Ap2ZE-DI4MGmpm+C%9&9PAle)Y|v4jqLqK228cMxf3A01*1pW>r)yZn zI=X+pt@9kqrOvtSd+H4)?z8aJ67D!lZ!dnFSYt^AfHzDbBf0KDXV)Zsg4=vUb};Y^ zXwa4<>cZ;U4d(Lt#ju=2*Z^zpLH+wzY=Av z-#TEU>MZ@8-2{({Ooh`FZfNtk;Xp#0#oiIZ-xDrSir<~+c7U(h!rd!8>v3I5oLIFUa9JP z93Rt0VSX7S_x{`rd4Km8jt4yq*SG%}`a0P4O(TZxMLIRg>K7+-dy%J$NzdqLI5k6K zvRcrOv}QtATFbs)ukr2;`ozpEi9^;CqRq^1I(W)h$MyM7TxaB7GuE@k#IL4TfbpUN zE~jNaL|fi^+YV=>(t5Z;TPbtS2{Hlu{3mz>Y~1i?RNv)!lDND6b9ncD%`YRyF*%mQ z|3-gD-;|*bLH*!E>sf~jwI-F*NLMgKsIP~(Xciu6#?apQem39V^D;DJC&1EP%{(BW z4HhuCA!t!Z!*8GTJJhv+L(8e(SwEE{6pS_b^#qsA^T_7;Gs+c1H70Q>D~NS_oX?mq zA(7h{Q`cLtj+n})|MlXACduoI??vyYsTU@yA_ih|*#fwF5n>+fFw=cUGmOfo^|t8_ zAc$rQ^ZB5A?1Ov2neo5BEoOjRgJ^mi$_T|?Gr$(8jQd`k})d! z{(7hy4-$7e?9)@U>+2IqTo=Lrbn1M&tUMGw0N5GP0orECW8a>t_@B|-7m%Zv8h*6R z=gPpwGBGnTuT|?mq@k%dwQ;dB=aAU#sN5C9WX!l9mjpR=@bllw;K%?;Kasr=@oHP~ zdb|4+=XoL(vf3*Dx5;!Zc1Fe#!f*`Tx1(<7c`+T+o)<%zKb>1PFd0*-*7~n_$FCdx zNw^kLZI`kNstHy4AwVc~izA=i?m{kcl4DETr@jX`6k|uUsJLK1lfMGJa059F&^cV_ z=_hvvNRm;Eg3af2?mmB;F zH)ID35})eI)}M)dGvI>jxn5{i~X-rGU@o z-j}KW{`UCJF`29gb^329v|spK4EHjt5^O`WaX=M^{hQ6M7w2F4aIC9x_ zP7W8vA%SaCa8L6?E%aU8w?itLT7xot zNN&x`BF7tQ7OTt|mvh*>MI(*4b;=>M5yUA*B4qE)3cbnMzrm zm%MvA(xRnMwX6$1Yb~Wb?7Oj+V~LzAGssR1Y+C6?lS6pVrVkELS<>(249`8=V}Co~ zHsdc>U=079>+il&1sG+yx8+Ry;Q7T6uFdG6(`RmZh=R9_IOvg-?Ymr=5w~@|-AaZg zLQv{)>ksRqL9pgEl-5BVD@?1kUROIOxwIlY1a-HSHvw+s@@}9#`z7bGMSIVBW2|xL_g8Z{_i56E;wAt+{x9 zwaQCFXo6=3%h0cno-=F4xb8icl%%K`*eS7jZJX!!vmMnXw#vU})wv!c=S^Dho>WcP zsu;H0S(z=kf@rpaKR;p^x>$Kd8zC*#W>tQH_Uuoo3H%~`FiikJT&ec zK!6v;aqV%RM6IJ7MEMfo=cKGM=y6Xa?>iN zCXy{BYLr`GH)_jK++3<IwVK;zs(awcngN^%0=r^MiFr zxk&EYf3cQhLVC&`2U$Xk$BN3r29!2V=*Q(9bd#P6V$P+dzoijcUS?L04nlW=d#AF-u{42r2#en6q&l~C zv6C%&-10{5o2=Hb1d;!qi9n0_^1U_c>lNNdIk$&SZhcT*<~;ve7^Gj@eU=Yg4y|@b zkUPE3X1wTf^`0V!4o$XHl!55&8w}YuAIh(?0U*3bw~|^#AkiPi!w}+-fKRra$hK~$ zpjTgLKvO?O$po5A&o`kgR>j63^c7$)xuxO7QCPC_77q}Y9~!JK;O>1<0M0tZFuaQY z+U0vqarlelNos_726LK3t`?w(VV(*4zFXaIlDKsccqF~d`q{<+{eU&YPMn|xxH3mL z^tb^l4w25h=MllC@s~zI?tF>6NP@&#d+WO~`I%^Ne!g&@JPx-r;}e5oUYT15{JDR^ z`M1H-GpJ;wH+p{%UJs-O$M4{B|KumYd9LVw`hzwOe(-X4rLw>v8N>mOkW>`@hQq}I zOHv-22VT?9sgcKHt}0YD+JfXLZIP@GPK1QEiLWT1!CSJ=VLOFGC_}I-xlcy6?JeZ~ zX(`7Qtk!{tQM{==M5<&x^@p?kqsZi|Gk<>U(=R0kZcjzKH+KGbZ*Nk?0G%BzZTDUH zTzpJRvz0&0mkSl}csYHIKk)G{HrF!iY-%4LnvlSf)RK^T>U^#ye{FTl*%P5l%nGNu z*J0{Us3^I_aD|smJCx&o2S8Rsa;bf)VY;(|Rxv>|YEm}{FZK%~TT(UKiS)9QR1EJa z$~7p-Oxj<%$c+YZ{kTKSHE8b*4Di}Qudo1ZXK^iFe+(-kVZ+nIrJ#kf%5jb5%Kmbq zw%2KGIZM^=;RwmfE#q7pjt}UoT~6wQZ(UD*n;p?sPG{g|%S`L#U)BowUzdI<@|rrz za>W5j<4nGntnf5l@d@ki%f_D7764LSHC$@}Yf{9|j3C$Hz%V#e(?@OX4XBaTeTIcM zcc1qfFPx#_@<*XK6w<)gX1u)%BJ3 zmJDFfu3xVPi7Elb1wh|3gG-r&ug2O*bV}zRPZ69R$;x{I+$|?-sRn+#JDlL-^ac;5 z%BRu5Se9qYp<28S%v%sj9t$Yn@NxZ);>9iA7pJhJ+RFdY9e;$CU^faX0|sLQ<5&BF z$z)o=dFr5ag;N#x#g;x6wdd2iB)7K8<2Nt`}A(C znN7IICT&HAKJd?M;~VOZhhd3P82Z1tu$KNj_ME^7L|}iECM;_Y2>q$mgjm8)VH8&i zRJ-%J2BUH|(9tZ8^=oG;y#BLOx3wJyJGdTUxJAED9|}k|e=b98PC=L}HgMs!kN2r! zRgK+k+>iKK1YX!7ZU1&bq5Sw$iE^zr)j1BL(#UvwZZvcPt5? zI_C4n7Qvi4Us zS(4eg>qBuP@cQ;jnOtO?^tJXz8|gcxi{F;{wPL%Tb;4W9*U`}v_flKwTf2Hu$>d@% zZ!ECPYY>@E%Rcb?Dgwp9hC1?91l>xQt-iVOg4oZLiSOup3QGwS)taYC0Ix2KQ&s9~ za#f4enI?ZW?gu*%C2=!6Q3d=;W-O5*^yzHYY$cge{Vcw3zlgFe3~*t;*$Qv;MBiU) zfnAND*9*82SIt(`{)vz2&RigN>)?Z6PjT$z2^i^uYuOCA$zV*m;>%FK-xNzWrocU+I zTK4?1tTnJ?jqdoO-`Bm=`+*a<#|_aNC#qSznYw={N)BW5Ps~v!4&)g6AxMfZ^fa*V9NbyE6th_T9=c9&9PEg zXq>$AK+S0H6I(}OJE1dfk(X>I=C~6_6}!@geIbi%=BQpc5Zk6<*VVe8*4^H_-;%?ZEN&icP{;XbQkhQ=Z*~&rrTq!$yrbJ{oUTA`VYSkdjc$01q_K4!>K<95FrNvj40u;qhAWl*<61n)?AHVke8|2tD) zK(}~3w|j+~dP9A4@A5Cxws@!Cu7xsU&iz*2kt>K~;Ag zyc&lLfDSn{zb{rXvKu%FaJj+u)O0>I3*gZaZV!D`jOE}^Z9-Fv_F!?h{$F3fw*RDWj9dns}rVT zxIk>)VIPdHf>!=nny)%~hcR(O;3iR7~{WYACORhefa;ZJ^boi}bzwVjn$_n5;~F)vV^ z8N_jJ){u>(*iV=9fNbZmSXwF)HzL#hNGlEzBIGcbc`?2#2>b8d_kOqRbI|tdx&7ag z*Yb?RP>Z~a&6U(n`GjmJ5sUZScv{MZ){MQ|E!nCez-n@8U-+93NB?(JL13_G*x-#!g1pj}{6>1=-ai_HX@wVAu^2Ev4XL3WtZ zo%2ZK#Wq&x#%>=zO0tA$D`QIj@@Pc1!kAaG{1k9o*Wr`=Gf3dsil;x_brqi-@X3th zn91ZB&1G}?c&jLTdoz*v%5ULK0x6Dga(1x5nLTM3Me%9Qm~g?M6KfpsBiRAORr>uu zYpJOcEw`I0oAPPVMDPm|4T?4;h*{}FDhZgMo6I&r>^NOhbZ@0%q9=phnh>FA#g8-i zjqL{WbN3VaYJzy^X^03s1Hd)NMS_VS))SSc1f;Q}-r zgJb3OF2tUYG%IjsM@9c6M?gfb%FJ|L#b6FNwPpc>s2KL2!fbww-oMteOlMpbBerLSJ#CN6sRz~@#T`l#v}`KNf^WV2!~SBnz1v&i6Zaip z)l2)zyCjB#4$xa=+{1{9M-xjsPo&l=_hQM?Z?NFEWVWjx$J~*L&@X5nI_Qzs+=tvE zYjy};#axKi>3i~B(y~p%CM!t=>uvv%io6*)_|XmPesZ%TMbNYTyxL$zM=P5FXz=!y z^A^}Xm>xj>Q<%d+(I;O4ZwSSt6X{@7)2j{)$U!u@Dw@3h$hrO`z~lc?3`hyPKR&%L0ll3r|ZS4FMWOwKz&&BsCf zNzYeYOSCKGVFm**#w%A1IGuznL44--+#ZN2sIe>&oc<4PZ#bB`p!y%ptBo{7)yoRE zW3p{`u#DB6xG9mhtbbjGWug)DgfUT4y8P&)&CRD|*wEmkP_rVR^54@xqq-wy#L@EMp z^ekEl&NXro{623*Ky-}mx*w|30FR7G+sdLgacV(-%d|i zwZ89AT3d|mRB>x43%#FoR|%)T)onM|F!cCy5_X2 zgvM>fSA_$wih^wBWh7VFvvefbAtx!a}Ncx zz1VaC3q?ID`*pBM=^jr9DA;??TRpS<^S?}BYkO}$jikSsLi5gXIBc^T03X!yOs#_2 zm^2+IE!|_c_kKx2IL+

    {lhCAz8;?K#hDbHzE8|=&-)t$sQpjv@>9nu;3OKwEJ`H zj{j;7t*PsFs=H>6y0#&3?Du#zhMs%_Uu--C%FzKxDPz>zCXiW_i0HXmGOPlPZR0{Ux3H<{z^h)ETjYl#zscf%Rdm^H;_-X zZ)afri|tY{His)$rqGmCRs(htwK4X?bDb*$pIJ|kc}nM%v(dtT;L-|NgjcE|7;Q6F zbTWQGli_0gqUxL>9N^c72Q6`M^DPzmu$sGsX(obPB=K~=&w2c$c)pgQ#T$YgI^mM_ zs`zUS!o};o+AJPXOayIM0}D^IcvwuIAu3gqTlY^L69L5V(4Dhbv#bl>Atbiq{eRN22K_1jB$HXy2&p(SdIRj4RIsV| zhWtmC>Ne`-lNx>lQl(w5_ysZ>)_Xa!GaQJfs$WHo8K~x*8k1zsq{ypZYT;!PK2Gm0 z2)K8r*c|~-lt|Eo-p@o(tvN*?V*yEOe zM{y7O{hV)g-XSan0QC2$k1<3h3N6M-3a4r`LznVKSgSi3bGrZv;dIg3~|R~E+7tWGa1epIsA#a|J%DE z#vK5x1B)?GedkIBP@Kq>zN43)l;33~>c2hjO8glaH+V`r$xU3jX<-@}CiCp*mWu*= zFRARO`s3N}j}(S}Q^`y}734WGKhB&hEyGl>exN)=Jg*79+4oItAQ^PVZ1D)r6qe{| zH7-ZK6Vdm%QaeuFnHb>C4L`g4`%C${s_mT~lSUci0Ng5_z2A;L`ISi>dt%nP;7YN> zSAlZt%{A(t>nemODxO{c#>p>UHv7L&>pbK;R6wIVX4M|*L%xLWt_2JO(BbIb!#{*} z(;QPN_E~$ULt4Ud6eCaRtGrsDU)33I!-DsCiHEdYNi=BAyVh)EYB=W-%f|J5chhz- z56rbhIpxKtgz48_waZ{fKidy&?2Rr(bj^`Uhia;WrF>p-KVUQc;LQEC{3e|>E5?EZ2MWB{p8UVYEuG}hB# z_o*xDt1U6NXE~|!Rr3gm%+cb2YO6%URi=4+q5}`nWNcBtgrevA%-lF(# zb2iW{hniuW*|=+;Hhr$t^67u}lBq_w2lU`K$54t!L0p?MdTFNuW9vAqilK^2D?CV; za%cN=`?E7A;`4u{7NMubCl0j_vsp%kx;#H}kuJM&?GIaioqqCqjfgJIX=A|qu4L4T z*nQ^3zP%@bz^7x2M&Kb$?Y{rE*Cp#=Z*$6zg>XJ1jjpQlPYmE$`6a|~#rwj+oOFE1;s?7;LUMz|h}{Sw1Hsd|TnW?=^!3I>ZMy#) zTKnkJQ>z`Pmqoz+>-}@{rqtbr;2Zz0OZ|b)lz!a|-Et#)@Tz@Sy|gb;Rxyo@{EE>K z;_rz`^SQ$x`cd_Bo*<~%&m|PhXE8NaiAZ#Zd2f(F)u+wA;IAwj4L$HCgj~`6EbRSw zj{+{hEWUAMKq2+ZadSwTg8yj1fOs!WY!+T#FaIIGp!=6d?L?9HmZ!}_G~AzRs9-dP z293u%RFaFfV?(&tJE1vDV1iEiGt}n&QYk&?(_7mulfRmY2kbS|q||ZuczvBL><`Xc zujqF;$@?sn6mkLLK?baxAcJOv_H0mWNCfipJ|Ah_Anl5!jJYgAvXjrL>z^%qoudoYyQU9`|@OEU~!!IM*7>E4< zT<%&bTrKKBL+|SvhjPT3-y~n8#ibV$eJP4FhPY8nxi6;10arh}L;Ww4oMrOhQXHJfi*;u}$4omOB@z-ecZj+cdzi%cJYUyv z`JG^uHtg-8OWi$?IP<*ZsP@^VIFkRHwb%KNA$h3JgP9pV*rw}AwTaL2z>lFOc~khM zB`=_-c8hp&TITe+5#>hXdSJ?(&;-^W@V|v12;s#v)um}C4*tgT<8VE!HTKDugUj_Y zviwx?=XuV(KN|>GiI&vscChBFeTPJgLf%gP_JS{@@@pDQ z@$!LmoLQyR_Qf4#J#!iSaq$Mfm(~Hx(^C``)i=1-6iBuVB*pZI)t%H3EA-0rpqn9_ zLZud`+)4x@!s5%X|JWpd8+SL17Maq+Y{m3#!%QbI1}d;^V)Sz(^5cZ(_1I>-i33v> zcIH}sM5>TSM*WHovuyM)ic1bf)$Vmr(9eLZ)_d3f?&f`>1+{V$o0x(QZ*U&}-N3A7 z{dLgw`KXkhTV|DL!}t4*IQv7vjb-siY*qfhf)2{on7X`5hb4hYS2eF-br0Sev6r+L zl~Ua(iZ7^CSnD(6y{vY>n~RuO&i6AQ#LdS)A%%@%-30#9QuA!P_Gp5F`DjW1xqm5w z+>y>OTb_K`85H_ikjFV{HB}?{HCJ5*Ersp&y^-CHKAed)f^CJK5j^q zE|aSnM#!S;ut{Cw^}7*GUUEx+qX{h|=!;8J?4%ddooV;w>ZAvakB zL`kQm-?2&Dt9Qw}gHI4_+~kOya7=MxEZH;#^zi%hd)5iK|1z4*BjGjShTY=ih$ORD z1?IkFzk*=fm$s){hb9cvMljidhK3s<^y@8|UqnX~w9N6Zn;BujS%&X+3>dvmKDoTs z=yfi}u|`KcYv1so4?}lMy*t&G?%K&^fwY(xacxBYQm+?zN_)9@a8OoZvy7u?H?b^S zhm`QhxCsMDBX}kkS?R?VlnZM{h{=Uo9tr6%vxD<4GEW)gwf;NyG!2AQ_45LvQ|U@+ zHOFIe>Q3!Kfkr2y`?4q7Q93`Xg0wW!>JVoWCA8A zOO#%h&lMzUBtGpKzY_^M9f*}6ulP=-4#^SN$lA0wFa=-bUw>thuY$3o+tk3Mo(xSm z2drMewA$b?A$ZB~2qZjKik$SmC+u|b;7z^!v19{>1aXD-Z6a@4mCES*cf-h?bW{=O{kyZWTT?sl3W;3q z6M(veR|KvZpp42Ns}b5r=ts+m4ng}kS08`$K0w*4{B*K+B4=AsQzJ+8;wo#71DUts zKX9V(v<$>$J8#GJ$nXx3xD;ggO;xqBr6(O_i4?79+RkGEzC1Z6Gju5tS>YlXBFqu) zT(G3_M0nxv``DJ!p_1TnL``cS`~{_O9#jpr`!y%AE}NWu&JsT=ehE@#9Oa^%O3yhR zRu+SIjX!+Rdkt=pp;(*_DBrC6)PZ`9!fjoN57T^rcb7mcDI!}oC(&Ss#bMEJuVyse z+Q)c&jIzGOl9wDLEC2oO@;$^JPuumV0X+-vb^M4T%Aa1GWi1{T?La-(!ZxUbIXZEh zT~qq1>8h~aii0_fI>5rSm1zLmI1y1g0{!9m)^Y&}=B|Tv0XvF;<(2=Cm(1**PL>x{PGEp`EaxCG&9&))VvO$*AVK3+ z%s56?y=4U(*PYqFNAxEv4kBCD-iA#-EYa?R8E<0}dEO=1>=wrXP8?s7e0mxLvSINJ zrmOLXGBk|-#Av7Set@!0Z4mK_I7he_=hiV*n+b(TWL%iS3&L85`JSKduswGt6{>C% z8O90n4wW%+YhWA?ku)WVRR) za#>)q{9M-|QVJxr1r-cjj-{50lrr%q-{rpCaMtH9uDmy0R*W{a@x4i3b&IR1!Zd9`={qh;MCgKBkM10xui+^JNid81}>KtPYbY9H78MdTT4w8?5T4mE! za_IE(WpH9VH6Ji$y4(EM?+Tr@=Tl@i@yAac-bE>`6>o3ejM5AcY!l)jGGBMV6NmHi zRQyIIVWHS4uIGO9<1yf-q|x_T0Ev(nIBg?z`FsZC zHq3opQnV`m*r#mgPxx^r&*2r64)w~brG8ne@c&d^C&#(_U_47IpQTa5{}ZcH!zsDQ zX8O6dG-Dgjf~l1G4Z}f%!`x5yZfA;Sy@TK#y>fx>_XK_!_y8;7lkhS0FD2QN9HM@?wM_ztdb;cd^pqnYcPQjGY zf!?8AjwLVPvT*hW=3y$%-sFeACge>&X-o(jCWJrbVexB=rvt~v-&erFIR@m!$hQ$;fs?B1%w5$))&9Bzqq z*a-T?S$Z&$0dYz#<(;3Wdk4J<-hed#uhY%`B#N0y#OXtXxn2mpyD5k@P!mGrxV+j7 zj>>+6F=(4k>U8k^1v|jy-4K8sHZ09)+>TR`kuh>RjwIwE{su9R-k({Hdw93>8d@JR zMhi`G3PsUJP$|mteF?;Ruj6-2?B$`_C$T^#XhbFblHVY*2k;NAwpQu4ID)>JZwcN# z7@_t0qU&e2vpE)fu!7aDb0ZoeM}#6~RsbZYxPWzQUCL+5JT@7>a`H+@W*$3%d;gd9 z>4v11olkl<7rgmcixh|}Wnn+~kD^1Nfs40S;$Oqs6=4Z2uB5H*_X;U3_N}vntbDac zLI~{34TDZ(N1U%_Ne0N}>k9bC)?w;XW_?+)+aF1eYp|1>Am(2GhhNuz5?qgHiP|in z7RnqL*7G}j!Gk}1U_3E*Stcz}csr^7-S%OFo-CQjux+A|tNR!GlFwo*PG667B;F-guVg~= z^%po?_}$$g+%gpz<6Q*soGw?iEXAxDkY>=QU9hk zMPq+|5a70c^qUna1^H))%LmW|1p_P^lT^{$OdD|juOZj(`*1?SpUyPBq*6QP$8U*E zPk|x>O*ss%Y@XAq=#UMvPJR_uNAq+VU?t`(kkZrtkcZk^oS~M-?jr@O$mq+YiBHp> zheU2+lD0|m)!NtKoVh!o8DRrk|3!>LtAEI`d%P5OM1B$PshdR^QN^PWD*@%854q11 zDgP^@MOv<%ihf+5OsMlFTjzv=_2d(Dw>h-i6b*8Auq=BsD0Gq*U=~(71znr&Y}Acn zW~csb$YFH>(iJr3uH67jtT@JXi%Ldiz(_ra5!TBb}7kG4*4x)h0&eHFkI=IH-9yN{Kir~Es~iz=7d0HDB7=bO!D4iLo>rV? zfSLHUOA%%TUU5O*JlM~ycS>CNYW-#LZBIvWGC5T8d+r-5t?Is+f(f4_me$=t2Kyet ztA5ucEEtNPT6CvMg=FBi<#q}_6}(ffR%@P51=WwTrAQ=)-|(Tr3$xEU)r(ay{-9JP zm>ym>JL7_0{y8qL^5!o#O4E;w`m^C__fS*&`J!5zOnK&klfJ}9vfBIUwr}sqJUdUJ z{3dcf@0gZ8vxN40&9D~TcK;KoSD|G3!K)a1{Pcu;)9ZyQ$w{KqOhWs%aAMeBack}~ zM|-}RpMN6sNlTsZ^ebphhIjcKJoh+W`R}b%;{Aso($_P%s#=fbQa8@)3!N)C=wYSM z5W1L57SHor586>7VVN<%Vn0f}oMPb}7Ut~oW;+rOtQ=le!bOz(XunXCzude4nwkEg zw*A&^Vs%TTJ4)B= z9n0LCNnRJuvB~{9eq4`<`yF!n{#qQmCw{;#C6f^PxH*>W5|LcET}Fjn0|ah&rp0o% zvphYKe5=Z=aKp;(dglt*m)w;f@9|KAm_c;f0X;KeHP?%JFLVJ9BVK73UW zm-^Q`(9{IF4I2bulYROC@A6n|7(_?lK_2@PLU>fg(iGvr*IU8gNn-Q2l$bsgasp1L z6aSBswjhu3v zvmEDqKFy46wtK&Q|M>mA$79>$-hIFKeqXQadR~vW%M&Ju97uV1@P^;WDKQv)g>6|N z+=PVy@p%w#^sN-lJ+@W~_5Mgr$Pxjurd)0Oo=eQew+als0aZ^>B>RSqO?iWEoLlJj z;~Beq%KlBOiKh-UFuY>Fj#WW5)s1g!p#NRRc+-{d6`+;Xazjv7i3vE})$Hycj;pOj z?4{hsW--+uAr7r;mqkZML6LrZK+33J-uLC@sJHQURa5gW$yl>AN=LsN{Gn54y)U4H zXYzJ3&ewN>j#2If>dR1iF(pv)bV|k;%jX1$I@sM{(F+oj`YZYI0*F;0IFWwGXzRLT z2UOrwMW^cThlTqfS}ER?xu2v%Xs@?txa?9sg^&A{Cre`PMenPD?f$e+aJDltwtrZ{ zH}RJ<$EFLvqAF+a#kW#UZC+UEA@#;<0F-Wi{mGV+s#+fosJr5InfI)-A0$(fZ3Zy= z$FEH40VS&>`WV&Tp?FacxGhb#q)J^@h;VZKOOcCv-}a{S)2^S!-3qJ|o7)DltETJN!R(7tM$*ii+JVpm<=Qvtv>sFde z(HGtK?30&owtRPZx}X^M^a$dy1=|4#0cc{WXpXka)Rk-&kGUo?j!ZQQK)qgM?)p4q zk6C`WzW=v~+#>i2$T{v)Semuj`jsK@)6Pz_06&2EngOzo~NI?3FA7OwO z-#L7f1Xbljtne}A5FSs>wbO|W_fQEaZ6b$X!;J?P5$Ut!B_lDP64Lc`NEw1No5*45 zDd4BSABk$JU?|iv*Bjq|-!8?Ip%$3f@bY9MNx`o@q`?a4X{O5UqmY-htjo+a#Uc|* zdl>A~!QZw}C#Gqae*6}}B8SJuu^-pIAs9?s;R9WrpUARkuQH5OD%;BF7pai3+N@qi z9(pFD76fcJdVJi4_z!qq68i9$sX50G(vId10^fi`9~!JNW)3`h9l8jq<3^|&IkPAi zJfA9gHLM{EGU{h&5ABOV6aQ-#!Dd3#JP-rfdMVS9SE3dlxn#)M{z}N$_kJL0rb~|N z`F@8we6JLdin5a-8R4DZ3f})9zFi$KH)HKL9`JE`0C`V2tqPZm^mK=RFb%j+TSD2( zL~0BFRnG-R$(7aAzJ+Ir;@y@gyapm|;Han+WB2hj2QC?ped9Qe-u%ytg4 z$UK)CdyEd4y1_M~L(@j=LS34PHcx_-eV=GLU&X1fHy%~TuVJ&HhpMkKz1X82;Pp}s z;+zN3j|D`u=mPr2Z)cTc7iyE6A3+)8lzE@I=IJO1X`pZof%4PMlH0Z)?W*)gt_4Vf z38R8h| z`e-GE3-+15}X}R^i?q zl~>+>W`~}>J7-$Jt*GT8^3D-{^`Zl)l?TpmWGouLa5KiN(E>$d^Nr(Zes%b?DW8QW z22`u?arj!*C({qEUw-@CM{96PY0LSXVIFxU*E~Ahd+Ghw#%DxKIxS+)<@Ad>|FQfi ztg!B(K;d&cIipi-;S8JR&knZVpOYez?YLvKFBvu|OMjrMOLwcEV=M7Kj}Wg&oFW}9 zK>5Ixvs3$iTFU5oLe`}c(ya}{C(ja{)NWK-uX5kqhjY599M0R@ELh>$0xr^j$%q4d zz%|~NYTEUH9MCzjo4dF5CQ_flxh`YFbc^bLiv>@REq|&FehG#FgAZu@rp@eUvBETC zcpFOK_?6xc@fvNc$V#sjMEOytiuAY-FZ|bolK$@j!VX5ASbBggMb95ELtG474cL0j z0g3Ray%#(Utz=QSf%U;(j3e+u@Z69jsT&(1O53nm=DSHGHc<}uyVe&s`S!1(Cp8K3w!)+(2gj6|qmZ1ETewVA$-sRH>6=rO3%g z2MnM0KFo^$E9P5EazUR?X88?68z8VBbo&-nCj3#~tsBvz=26yH`lL!5Ner%)x}deb zveMHnKkh4q{<%sXgi2Npe?nolISU)?|5*Sj5FW6m_9+l~N#&t5&?-4KX>(V}1Bg1^ zpG8XNs_WtHi+)~iu5d;WTgVj_MmGLSz2`ntSd-foHXc&~(Hx|Y zEkPErY;LXFVUSnrW+y_wHtoD*K%aLY`;bafKSN4A8yytBv!4+~3D8M-A-Y993$rKk4YA-UlTK+0TUT0IuS&p9bxX-?Xw{c`$y_${3lEuJ7G0- zw^)*YIAbTiF@eg7x`e&jVh0U8w7S->u5SDmrLt-_eneTCNbC>wrU=X!eU0+FP`3%2 z`f3!)O`C8mIvmqg`RJ$*>L#y2njB1McPH>gytY*g<0I-RMjc343X!^07kqegW)}RO zj;TPTSw2SeK4x0TU!`u9-MiEr29g)EZ*4UHcF8>LW%agc9oQPS6V{Y&3Sg?okTDT zrp)E~QjLVi7M=YTbnfG{+Ci1>6m)%+Hh4Qh<8BwS|mQ3QaAGo z(T>-{;*I>j>)24m*D)OvejvskgAA%bbjTW8+}#+>FUQEuq!;;y&o55EH3@#vyKJ5t zo#?m6B7y%JarFO`#~mdln@V~*AwEZ2lQnK7T*LcoA4Hf6AlW=0Ou^ETr!Qn7C(qv< z(qO8y!BRCDD1;-(;;(Qnb;d=JKfzC!VHnPTW10e*w!Q^L5P}=Me{J+m5$zK?t5a?@ z;-hW^|H~{}-wX4e`nX@NWb3k1=W~0oo=716k5R=7-d~B_-x$DtLnw=Cd&^m@zNe7H&iVke z?0*LGc)L}%o%fuX=Vb?&?EE7hlLpn*A82>sezl9MKi-W@>cn=&mvyn9oRLcAM4O3U z9!UXP3;wjZ@s3JreTEeK74e*{yD;8)btinR(&R-dZe@kSCiyO(oBjT|UmjiN%J@XC z_0!QBp+@oC2GS{dGJw`M$rILJ35p4&32rvgy%!%!I*!QRY~@9nKu3s z2?8iSbmlVGr&KLchyQ0$^A94FA>*`&Uf#wYROdUh0w1{G;#pHI!6-8wFZj_PR3$Ix zv7mv@)#X!D)$6Ig+hycPSHqes;SV^>p9M9aYGKXu=ygTjyl;BGAlpWCBm(HsVGGTy zbq>PYAQdkHD!T4dYfHC{ES}vy5TeAJ!#N5i&{dtx%s)xuXkltDwMy*rS-xs2r`YrR z*HATA;40PSLr@dX)wS0z;MpIkveZ^1INSe1WDK9;`2p0-#{y$iFg(v{&D|ck9K&E0 zwAZ)VJYJFND&#>NK<;j`(enKCL=r0Ny=!+bHYAmwO$2sG!;MjUrOy=&t>LjDsdUD0 z6R*ht)QkyVnIw)@78J&~i9>i=6t43%NBf4q>}W^=HP@lUTW!i}5BN)^3{@^d9kUr< zFgf%}KdYz^^0cnYe)*&Z()oI2)K5RFAb30V$A1j7Zd$IuDYNPp-biKiTc%%%=u6xu zRr_;S3i2`*I~JhIz%w8AO&j;O|2dEeG*2z(`BY!EiIqZjxi83gj4KCn6U6MU2xh+4 zwoY)M5LEtL1G2sakhWB$$dS&GqJQeZ+fN)A4!lR68z6>tYWMuZJ`cSGz6&48`T$xa z6tR)OrHp2ZD0KQu!usnimd5&qpSJI+AHe={!7XQna6F}lE>6$a$Sk?qBS*d_(2o>f zM8mVcw@IDKT{dq0pdW_>4WaqziRcbsS4iS%*aKs19z-$(K zw{j_knU=TEZneVs2+mZWUDAl@zflYt;0ixz=Mp~79P6r#Bvp^gkTYm0X-vs!;N4r~ zw)4eO%c`Fh?saTkfA-L!u%TZTDq)R}DT5-!1LJ*Q)XvI^m3Xg5fjnwDB&{_DZEyqzbp?tX%hQFa6pD~znN zUJyMS=(~!G*~PfeHMysCzYv&z@nt-V>Tj@_(aKg6UyM^hBh;-X=hn{%JYO7L7tMV= z7+d+Lt0G5fnd40)%Q;{2BPf4cf!C~0h|V8nlcjpZ~v0Lcg=_@GH6y;5}wOtA|};xWnNB|_EQ)t zmhwRTDE4N2CUTF`i>itW`pv+chBl-`Orc;J0a1T%v5}V`w zUc=5M4N!@9P-Ej+{mltbBx#tl4`ch<;%E1xT$b{1xbJ1&P<57$$Hmd}8Bety6O_?> z**vm|+dMDN#JyB7Os_g!bMVOE{(e;$rt=T=eds_UHDdkhQLj!&jqC-~%y8W=n|*Pc zUm7D1BC7Jy&b&TaS1*}!@7}7EJufV8z@GbX|N1YDnEN+n>3!xWi|H$C4G>-}Qv)l~ zRL3RL^*GH$3Q0Ua3hr$$ZDbcI29~_vx*FA$&yUEdo+NNQ36wC|yH7ui@@U0g zyVk#7bZa8`Rm1o3@^xnLay;rciab(au8^mSP6P(| z1u#i|Hi*S9)aKx!b-w`#Ze!l=s;iKjejdJ+K zgX<59grOhsr$n9eE1=IBmjo}0s_nL2+hy3qQI5gg4od1L9qjX6nofw9=B=FUG z-=3OSM^UpOSF(wYL}9v4%f@FyKQ?quM5Tid*@0((QQREqfdcT6>e-39fG98@VjLFW z5*kmboU0EheBZXYmDEidCD8YOQyDC8f@G5&-n9CBG33Wn>K0{K;jveE3Ds)+H)Wc_ zI6l#x6FqdfnmWez+KzGBC-}^08N2OT`rD@z&3l3!DWKIv4rLCBy!p!9Dya&db zBUgKx4$taAU(v+>gU?ZBgXodK(rnmZ={!UMkN9mIQ~2YSR*LV$(;y?+X9X~zdDWdy zN&B(wG^|OTbxl-q1Rfogp-=C{LR7#1P!825hApVOD^t4?_#6McDAljYH>Yw9z51|E zB^tY=@1qE#mRDu*QGJkRgvH;@#MI4{t9dzi{H?xG1@XnB3aV`3!$X5_8W3UJ1DqS& zcf~!$*=VbN88%du`Pz;&OO(;IJOhco3Spy2%t03Lwj3Lct?0ec2rSQT%Z@ih;@~MG#MycdR0%@{+%9 z=rwjEprjc$%C5W{&_Pat)LN7?+(yG-#DIm~bHe!4Ig|K4s71=e0ETN0cDy}Ft`#;t zp}-LI7=05dkMVY!xFOMRdi}|P)nWUbjpsP6F-kv*RkjCHQITD2=zrB?1AFDK54o}f z>9l5|pq5BWBT&+N*XcdE{<(5zcSG8QgXIO%_cv`MZO|`DekJ`tG)EGhTP?Q~NwTIn z1`gDZ^sE;A-5h)DcNl9BZCBp|-EWg$UcdRs9ezY|P%|a0Jb*}L0Zoj56OnLUReH+l z`!Tz*m0q)yuB2%1>&crI=KdAlMg_wYa-O*cfrESMs(y)9 z3?9Tq3*Nnk^}glbTCyq>YHN6)4-X4_wm@a_v(#vTIg{SA@deG=>N>Yv@a=Y<*A0*n zykTv31wLa)*fy7M?;qv2Su%WgZ}4^IRmVV{zseYgtM{0i{u*%vl+{|~o>8m&vg711 zo$&cQ?Bakj_n@|Rfq6g9>EY$?14U30bf-ngjh22{GX=csywIix=7IZ@6`yOky&5H@ z#c*2<&j;j7oz9MUNUF!z%|#{KJA6rYb%=h=29eTw=lZUy@T~m3mZ7-1b2$5i!6}2E zH(xkP%6qOl-T22re6aZKx#59FR+SkPo32T1J{1jY=h;xM&}4uQI^*aSsDag{$?okALUZTJ`BHQ9k|Zj zovu4QbBj`{v*^WvSL$V^rY5}JD)P>UmLs_hF4XRHqxQQbNZsbCH!Ut!&-dPU*&Vj( zxJQxo371uSX{A(6`VV!1ehzh-wX_gIy<C?FNO0`=Gmtf2NULkVo1FF&jM{1z!ZAI&9r$F0~^fL2P`+&UQvvcZNN_-j9p z?bEV~JpFs#{r;Vq=d}l=7gR>WQ=Pi~T6&|( zk_9^ESMkM)iV5Amr0N*HF;6}Q!uEcXqrC)xO57{22;AtNu_Mv0#N;0UFz^OMy~{vF z)V|-uUaF#K<7R}YYcjPSfj!A0-8FeJZV%Wf{s15bH2E6pE6<7*7LGM+mPSK23K>e< zj1K*9uMj=+tTx}5YJ{k=6@PMUI)wD6Ioqq#4E0_`;uWKmz9Wo*%U7f{gn}nP({85O zi}Fk%HoX5-JAXh#FZ)+!F{nqF@bGiRsL1G=XJp<{-7-723b%B&6%v>hr)yj$gnB%_0VZegCri z6N(c2;!((n*=KF1vVSRwIAA#Gbd94Qj3`>+X`L!-%w@@$a6;n56~Lo)@af^i(GMU4 zeL!?wjy|M2z{d1o^x1zc=TSn9Jjrs3ZYG$;hyW)M5QP*{`3EXWhSr+8INm{S@N*^YNyAdU^AD8dJ%hGnmg9}5-}vVx0zN!eNtzIgdR z2pW`3CgMZFs9g>tU*^AhN?S{1)Xq12S!TCYyf;2wU2_vQEAuo2^bTMDb?Ef2nMN5B z-DW)bjN^{X45cBI9c}!qB<)X;bK0Ex`)>SuWscMJ&2{(nO@DJ6mq^~$x>6-KaCOP~CRUS~`h2E9jaC{+he_*WR(P75R4qRva5-tiSh&yLq zBM1L}H)=>Og^Z@F*4G&aG$DvLW4DSCHi&e*ZsDH^wt|PbIVt%+ z`u=_7yUq5X{e=;GXrA;5=Vjw}l>$%B`@S^iL%(-+u9^22S5x9Fw;wBhV@(wQNZrII z<{athmoJ(K;?TN-u*wHcrn7W_^BIwAqNi+ z{W{mqA@5MdvE0<*>Y*6#xH0aM3+Q@3heb-Frrx}2oU=fqIC_U%$d7w&T`zxrn)0h~ zLG;)Q_-n<;whmSgF~~5Bc($!n(s16Di>qPe0!Vt}L5zx%lNWE~Ovq*?(hE59JS=)3 zzz`a;Nq-)ye55^Gl5FPXgQ`cx{w>t7Baf_`BOAUsr=F>(->RRlY{VOTd{YO9H@s|R zu(iL&&{x(_0o#bifUWkDnkR9O(({uMqnm&2={1A3z6s#vNX<#^7fhEn(*-NF;S)O* zl2v%n|0V+DvwOzvz z^VRp&M`83y3|EJT|65;%eyb!}KVWtVQgx!aw;BNC?cE26$Bz2w z2+j}78)D=6EF5a()!`C9Qj9YI(sse|bRrc5j)msv2W|4r#2FZscZ~YotF2WQyT$+G z3$|YsIQnty9Y&5?TeUaVU)^5W+9s^g@g==0S?{J^j$|0W5vN98z9J~Av7r}6XC%~> z87YyjZzk$5P*1~;)u!*CztPPVI~zww$B5?mjlSn%geTD9=)OlA1+!>$%r0tw=;<83 z?{g6$PPL_LH>|zN2Oy0O6@Amg9g%Q_NF-gMS6+2In~9GjoWXRzGtcAJkGZzsQ8)?l z0p#Lre#>03$~SlYH^|c+BwhOem>#a`rA1-xj#7<#HRzt6nu8;?ir|kFp7$Ni+}F^N zT}TtFqX5&NiGCAyD830jRkayy3e3jgF2;>oG*P%lpZ^yQyRooQSoLCK409zt_VEQ> z8KAp~;aYTOmcM*KMFx1eP46XQw5i(D4*snn2wI~(icCPQAe8X>))V0yy&=y%CqS>A z9|^k=!5GQ{Y?`d-(z9G}JToa+_71|{G#$fLeprt^75UP)_u`rA&}5j|uccsv7(1kZ z_CX6RX6HB2p<07+X0oNw41@#?9)!CBoW1RquR}+k`QurTnT1_)%E6vf?I`qC9O0%6zPikRXxl~BM$bT1@XR`R_n){sq#%KP&a+T54^?v&d`Zd&TChdH%jW?)HT_ zRi8zs;%0Z+YjkTbA<^l&XLwkKc+c^C$Zv7}eyO-GJ9Cdtf4r^cADx}XqWqlKeXT#> zD3(DKsYlm6zPBBIq#onWQrvsDm!uRaB6TDJ_jR&eto)MSM;ei%^X}hTp#ruF{qBfY8D9Uwha zxKC(zX**_b49M#=bG@e1U3_=cqgnFQ`&BE&XJ_&sELbLM=jd4Jh8-~Dp(K$-wU3Wt z-5q5fHd4T=d_nt_ILS7r6I1QYMrGFSNFmtsI6<}afk*w9?pq|wA)Qz;|88$AdCh(v zQybdxFvcyIa=A8A9wx3*P2wHh3+xiP_)*MWt}t({eVYdq<<)bLI1Qb#hnB&X%w`F1 zh?@M1TuYF4KP{w_hk%&jS=o65ZKT<)p$P2);OZ#8AZ4LU{YRhy5aj=9!r%7#?Qhcb zFUizm5#ZS{imcl=Qhf*tztZEMYH-V(bOSjZVkKD$oj|sj|FAFy7A(HnpR$yH_q^j$ z2j3MYrwozvU?$T^MnVVGDLT*vrU!@LHKJxB`+4f$f8JI9eI2Q1j(B+1Z;hkOk1W1& z!(c#bt4#BA*jx;FA1FD__XDPq9g#Gb+(W;g7epzgs_5OI%GYF3>Ud>wdfJAB3TV@1 zbkjy{=(5$>AD-0ACDAD>>(s;?Z6xl94|z1MO53oXJ1oK=Hj=eBZ0WmV$)Ve1KF9W# zuP5ypQFFwiKJwTN9s6Ygx7JGUx&mQwjtR07tEL^o;7cg#t{-GW9f~+8E4dYBdtsOdbss^j>Cx#Sb6);&i6VkLKbEm!^>e~hs z^#bVA8+_=!X$E}Z^>G+-vr#_mM9`7egAJq9h!Uc(O;HE+4X5mHPXEbn|BhF`zrOR{ ztZ@1o(z@c(?T!Z%5AFJ(=hQcuqnERIK&u2;S0+rYktO6MayuKd8@*MDJWNIX40^pK zhdi2^po-`Y?40)*c~RPc5F{}4@-?W9@mT`{icRGRq<>0yNBQbLOQ8#u&gswa9}teS zK>%J7d=kQi7t+Q0rDvs8{GdO`37$m0pW@w4S~zb1;b|B{@{AI|(KFt~f3kVI{V z!P`V>yMEx62B?3}t)iwCb10qOeBesuc4v@!z-t`nPkuKysClrp_`M@C7QVYjo5;8V z#6UFD-JwUIpS}NX zPa%Mn-iP!JVYv4o8AK-qR)t5&Cqtw#_J!G4EIZ|r(WePukFRcjq`f06c z;Er>mKvQm8#{BBW+Gp4i8AhEysWhY|ijr6Ps9PAsVLRrLN$#v~+Q2eOJ&;E@5RGxIw?g=T z@0oz$1JS8z7~*iUMqsIEE|OxSjmm2o^pV;QVpjqaC|{B#yf(f=;*SV>@C+pU z1ULE2Wr+_>hX|?v2*QW&p5(#+nirH9mdvwx5e=!mYQa1AC3x0yre!hIp2inX5$rx_`}D+?J3p4!Gf5UR~!gPq)8F zF?~2^$$9eAnB>Jm|7z2^_UF^Xqq0|QmXNA;KIR?gmQ_bL8J>&Sy589J(+Eemgli$B zq|)UlHZkyOkSCr`Qo>*1olKN`2D3U&w%YRi$jP?!$LbCC{z9Vs?r1@%@Lxk+sbyBr zYk^nq7+Rany_rA!WcDxh?Y%QA5Za$BElD8w=vpFQQqk!+Cs5FniMAyV62F+zFeQK# z2SV~fk8otZqTBQHTHwMQ$mu+*^qIHm$i5bl(5%C^$e?yMbM70XHI~w)vO!~FeSy-! zWBiv4(hu34S_Kg%PlgS=@-fjYQ>>2&&}i`zof7=&=wHLu2PCvXx>>hYGC!VJx=CQV-wYi90Awi=c@u-8nE2tFtML8ho z9E~4*uz|nzopuS!jP7L`Gl9R7<;_)DS@NuquW>>(M9sww-y~I^# zrDYv`<`aJ0#Sx(^9clMEHK0?mBp%<|{D#Uy-F?f8;!S{)J2XZ+JVOqhAyRd)OtKVdbJ+ zF865zHR`lH0ACVH543n|XiqJOjEs?+b!Np^<|>9u@u9M}%Emg8nw6#Ak6=lk@Oi*Ve4XT;5Saev`{RWx3hP#FM4W7j;TV*3% zd7^k1C8;%UKB)IR`mcls68*M3!Q#(psylpa;weQNCo$gHNt|$?-C)#SK&YHh&o=K) zf20211TK}I2eQcC(x~}|>I*d!q$TO_DC?E8KQ0hSlQYew(f~md)OZ+%rVaP|RoW>z z^`bS+Nhxcy1T>e@txjrwP2L64U}!H21IYP>YWF3E#~mawjSXlKK7+!6`iF(58`ky$ z9QS=H!^UJFhfIhwqe6`U#mf;G&S+Z10Ixdrj0R)Wz`pRQHW6Y%C4#B20O`4|BbEP0 z5pxssjihwOTIZ|^Suh4g?*pAo?$I6xORcSmA&P$0CEN?+SAA?j~jCyf#Q6tEzw8=ANBjPo12gSX1<8`xxQta`?c zq%>Kf_JPil*BKC&bM1`)URaD7W%$(k^*wrGza)4`{kVx9Sv;|%xZGnuG@B*pcM`AK zv}5XbQ9f^SDS)2o4mn}LJ$E8a=aFJi1Ww$6JCfAH1;k?lja$1aFh@L~@b10lL-8i7 zkIpH=n~-%yk&a4*q3R3#?o58aEjf+&x6|hj(N<=Udp{* z53VpWBMJF3z7HcHp+Rq6?z=x>Oz^YJd9a_=&|`43l5Oa2@yHE3N52z-D$t)j_fVi)Q?@v$({@ek zgp6|_;{D3aUoT50Iv$tc=A&~;v{dd|O@`}cZKPn6t8-`cu1 zvlx?~j&U-D3Jqsvf)lSR-|$$Yld+^>K4h|mDM(f5*Md75=TJdyKBkF$1?vF=9YsDQ z_k^S+cS>M*Y1_Hui;y4r+>;l(Nu$;)TDd$1nl+lH!Flb>5>>k9N=GTsiyUsD^LcHN zllV#dC1(YBs(OUX%b!Pguw`t9jCaj5#45&#w)b)cXCMOv{))}~^(V66U1;>;y-rbnw3HgP9xsrIA-1 zc9nTALk>j&UK9Lf+Q!xNb`-BEUXfH!x&Z~Zlp`CCKT<4$SD`LV)NE4Oda}{0@+#)= zFAIs~R9C{)ERPU{KP1)aO{u4W;SxYxwjsjV;&H3pygWU;sVl z^GJyL)ak+=lKO_rmM*=5>-P0O(3S)hB`*3fn@_72CCEkF?m~9kgcq1ef}e7!^(BzAWu}ihI>RQXS1PG_(%|3iS<8FOq6aTE zHc8(MUwq4lzk#>ti2y;kSHlUBjn9%BL`+A71+w`le2qPB<;T$I zvc5e=4x}XC`J(i;Pc}vm2pofiuA?4C~Bcpf5ABys>+tTH2>+_y3cufM?c^ROk5#PIMwukyQ+xb_8&*# zU|JsZI1+Myl1j~fbAFl+_-l@;O*V(T^%KeNiAADRRTy=gCr>G1$X9EPbF&JOB@Do6 z*mwhs%3!Z9)PU)?RveRx+;z)i$0tc1Wp8sf?8PIOr34)@BIbuV!YYYd{23K+8J;Qv!zBwTcdnAPjtsuSYTwN7EZpL9T z2g-c2H14ECf@aU`!VCdPPM#yZmjo*(J`>iyG#K2M{F2dbc$(0}hggdTre(~)J4$Cq zS{(qljs)9}ii2eiDA|4FVusf742S$Sfnza{MsT^$LC)4a6ym{Rub}x-{p)>#LOE;_IIhx6eV0thzDvDq65kX6H={7LA4^92ty6Y(-w~ipXK?Z zKHA;Z_~yte99-?!sPHTjBu=ErbToedqj_2|TU7J!<&!5qZLG}#r~=i9N~hqg|3FWw zlCQJLXs8U_dLf|{e#^k(bZ=_O!)KEhKV^axT=!E}x2HI{8{f4)AJqL}&Y7l5l6Y?M%-#Qe`0;mZWKLn;G(Qr|37st~14g~5*4DHKfr9D4(Vb26 zkTvt!Z(KPF)xFF@WKN)v>%uw3d4uIL(alqr3ibi!7E;L)-N;jplBxG&TmoV5Q*@C_;#!oT7Ka>x1#n#Cac>D z;IGZ3lAl@h(_oKLK{5^a$&0q>I0X$4r!4f;Bcs7~wCeACs#@?m86<>R(B&jeIr;>g}h zR>-MD&#>d1TG*b^##6l=q@(B}{RFX~ie7<0LyyR7d|Ao+volS-&AUWp9GMw%D1-4Y zYMd25WA=_@@vk`yhCMW*qJGmaIG#NyVq(0JJMJ_h8AWqHX!aHRWA2js8`T_#WjwG8 zAVh~KgmMd7d;B$3>}q1b?X!u_EwqZ99iBxWPnjiu-6nP>Ud{B8lgaUHeA^&cAr~lL;{64ic7}a7QnS^Aa!Uv z!&6N9F;>9_rUoE3VUEN?EKu12 z?AD+Tqtfa&Mi>P@^o``#)KsO`-6A~GqAEi#5g05fFOQY)I#emK{7Zc`y|T9fsN9CW z?Jg9)PTtOiGWwrrMoqL6?8{3SqwSp9wdK?+_WdLr!A7R)C=7|mFdBg!;k)FZ#qxhw zh7T%^e`D#i9LQlL=0K3|Dcok09Jb#G+nubpvCT&Yt8{C+$K5bclbLQl{3MPtS@bu*i?W9S)_OuX#!!}JxrRtI z`abs>NC!U?UZRFkY~6=ODsm84p3!aHed-b59TWX;1K`sg3$^9Qy8$W~<1oVCW7=XR zvNs-a^qGTu?1^$z`|NJeUL`phLej%f?fK@2zE{1i_NMCAmYY{&EE2Q{s zX6(ZEgWfwHaj0$I++Ya{AI0}Mp*^1XJjx6`4@F$epyS;kWX}^bjfRu@S{AzR@vmbT zhQvs&x>nBBY<(44aq5Iz1M93fp-$dGDQ&7+AHN^DrvJ=)X;47vMXtge!{?%4`7aXb zLa>ZG)UE8I-VasiYaYZ8M_Z1-*K_*-goOqx_wVIEa2^B>I)RuRy2$q~lKtyjxKaPc31rLPe4BJ9=7w0j zycBq2FK^i<896yh8Fx{s66dU50&audFQuGQxR@o^^DJj8wV(`p<4u3kC0smH-9qMP zLT$z+1*!8kRovVW5|5?(uJV32?M7`+FJ-yzCEZ~Ar;;CQbMmd3-A4-xVximQ zZfBmu)9?6N8ZbpZkm7A7snU3^&tmrpo{y_OCJo%D6{I~5qV-}-MH}?)(H;GF#5*8x z@BcjR3>33C!BzPniI74IpZ}1f=jeZIkV`ZVYA@;recyxR>O{8r!V*4;({%j}rrhW~ zcdI06eAri$CWtshEwbHLF{8FW{Hjwe2G(Xz*g3p-|1zRU059Fg-Fo;R$(yXa20atu zvaoh4<3T?t)TLB@MHFTBc(dr^$Kw#8gJb3ihto1=`BCrOX8$>T*X<62C)l)ZOpMCY zU(J#%_zomW(n|w^429QMY9$RmK8i0ex7PbN0mWB-IjK-0@goJ$QC6lPUTa{JC%kPs zKuCtp)BKo$oIu|hcvyvI&BsrCW|RiTBfFi1>C$gPg69|OIwu1DGwZ$!d<(#_`;mph zmy?|}Lr$(=C{*kzfDqed0d`Q8i~kywV&BKK;!7&d{dWo_qKUrQ#uK)0DtiabpjQ{n zE<=;sQI`>;QiQizD}E!ARIhP|Rr3&5`C@Je;&Q6yAXu7Cs^7Q_{7^yUV?P6_;M+^h zUo7aE;8^pth>m&O&Y>Tv0IcyHUy0A0Yey{-{)1^m2}J9}=J)JooRm{r0+#PR~68e0_wQHhW}Xdy~yEu0iN|xvFc3jexeiau_gc z4k=pGl*rw zPb@C6yxgk-9cOQ3t}cAZEN-SN3u6PN{deqmbaea_*v(<|;b>og(MM16Yy6*oxbz|w z>GzFFD&svS8d6r?0I63E8zanc)~*XjIgJu-7Axkz{yn>>a(!vSjAV}{U{ zM~=GS86gu6A>t74s?>OENmTU*%nFn>J#`8CY{N5z248O(w-yDiU1O}quI6UVeRe9$ z`75B{X2KqD)FT6svy$Og2lyNU@hptlL0JuU))$~PnX9{RMGNX&s{wErQjFq&Hk*Kr zqdyB&Ayiox0k9myTOtR3wW%*kf_J*bdUQ+HD%oyr6HZn;**+}n(O26;#Zts!06zGm&{nmTvZDG;ZScn+N=J_tU81>Ty z>w8zT>~d`VKK0~1hgk`^asZ>XkhO>zsM(kLZ-w7-NGwGGgYBS8s?Z%_J9XzH8JCfv zg9bd!G$E~0l;|QFfgf6~Isf8(YyA3*-5<#|;%y)S-w^b2OK&d>7d96(d4fB^u04I( z+AMnb!P0!%PvQ(+&j#rIKbo#Ro~i%;E0x@nb?v*mwPNbQpscb*N;bY}ATAfofjQ-j@W8FEyy;#ObZ-YvQQDi4<7X|>^{-LQ z0P0;A7`1?ICnXoEL#Syqb$ZQ5eXd5iOeUIZ{Oe|C((ZD_y20>fSa@~^C1Rhrqy~;C z(^cahr%tu?8b-bNWJFD+aE%suB_1V~o+3_l_QU_7BM%t&L7QyAe}SvFPrLRd|D6IN zs5<@!YsYpG7iz4`Kn2D#I#l69mOe_D3$l{l#7?FQt4Lw_eMk;@T!V+=DOw_6DDO^sqoBBFq5l7 zz_!VbB52mk^D{=5pNR!A*FgGo$J$WlYgj=;U+;>&@#s-aW38<`<3`_}{rlHB`&YnI z4eR1hez@e2FU-5hztU6G@`%MhI(O7dVh1p)OI;aiThqi;tZaFsPc6O0?sf}Eh6-&D zNGG^4Nki-VlvuO_S3UPX>5(_{9ld+uJhJbvUE>pf?G$C`ko`i^cXIdImXPeDZ>4IQ zcQ8En^*CLO5;eOzq-LK8+pl+AHjZ%GLmnscD~w$4Jm*La&|@9B=^LcfSpDe`GfosU zD*bkoofZ8h#*OIsi`JoHMO>E$>d0;fA!oAY8-)TFV#*sHcS;^*lN;kYL$!|;Yp;S% zRH)FJsFA1Tdh5FIBy=}V-$g34pY=*!EG#t_$iPN1fFAD?%V2W`Q=iP^co!_nF6i={ zBv$W_nR)tw6Yl(Z&PTV+v#+M8jlmB08xGW#uclvl5)Wvzx}1>!n{w|3U@MNeN`3su zcF6Ru%nHT^ztg#?e$D$9K*7#d!i#RT0D;2^mXRJ(T2zfxNAF^$Kd^`I{Y zj{;M?R_`@!RlEK|r^?erXsa@1c5c_8^NM_Trm8Aj3Sd5p3^rDF?cSo-yg~Jvkvn3v zREYZR?C9I-dGFr=MH6MJ_##?*0D~}t^I`F?u#^$R-~hm0`ZI7Hz&H&)6iVz-7gK!* zXQoeU8|IAJi>1gkG6QHAWt{Px3(cX}m{stcjqI4mXppLn>7`iQ&fOYeqFZv*hoKF! z@R@b@?8s`w6Awgg!xvi0fVk<;N4s3K2kB;M3NR#m4l$T!*YNybG%$!Pk4Kmuo)YBf zha3a$?cv!U1ciP}0No1^tXNzi6?mAs{Q5D;=25!!m-sLxX=Uihz`cSi-q@uLG`9?F zxLyFjfe$H~qD<1IUWmSTZPt&!uxh2k2hFL#zaN8$Ev#X_J{`YDY2BF#nOIyxHllKQ z5Jq&x$YvqZez?|=6c04UP#8q7h_81AJ@qd`ikazjmaxA}1e3okegZ2+r20}LmMii{ zR6aRa!-G;AMe%FU&zN8Lg=zhxV8#SeMC2J%hOjRtuYMc*@jHX%9?mFeB*mi0JBU9a zybTx?9TM*~E8Z9w5tUmTb2A84I|y4h_c;F}Ca0y>{r+4)4Y=GdKQBmz0)IsA#UZp$7a@?Un_YuylyWFr2~U9i}qY+o-E!J zrSyDan|-0Naug>)X-oE^DN*=o*;nnl`%f7B@nYmDxUI3E)JRovtMDo||{lUsLXA$cQt48I5LP&SNa$-;I?)o{j4jeP6xJEZk zkUxYEqK?IOGvb@xwUfX34uaA{aJKVWSaG0QU(5#2{&tC_FL?J9<5AyCn9{S%+*H*&*Phy<>@q`e*QF@{r zrAn$saXoOhaE_1ymOK&}FQZgHKL7Q69>wQ+#i!usGelj56wFl9WuO=z>P}YIbyl`K zCODs$-Hf0c?>9VU8(!kzRzaxDQQH`T1~Lg>v*JnY4tT zqFtkv3rpBP_mN~8mh)ugC0cFlpCnU!JAES`#uj@p;RR+jdV{G_V?4KiVNTNQg@<2~ zg)V~5r5O2yNu1$GX+$K5E`k19q1a_J?p1fU5r02-wgEE)_p+9WxKcu(PcM{A;x+VQ zfS~(H?l7E+Bw9eoNtbBWO=zd1mQ)!drP!;6a(K?feipdYe9IYS>1a-w#2fkhmjxOd zbW|UQ!yt}0aYwite0uDxeD$yY60`R#3>OA75XvuWgpL#HFoT}kU%!e0u{8W=g#WGH zaR%{ysd`hgGW8|86m^#-0Oto>Ya39@KO0U*#9AI++HcH~Amm*B<-g;i>ed;t0lArj zXNnWozq({aK@Fg=YA}~S_LtrO!Ul>HK&XM&16TJ7EJ)SP>uK(b>Y*c>^?o=?@tqdK zq2E`W3BM(oL_bLCC5X20(NZwW6qO`^sRiwr;I6(}xu4^=N&8LD?5dQ?4 z<49+EW8fd&DGuQohgoF0CpA}j2WugLH&N!r?^oT=K>?$;RNH}Te3p~56FcN?K3Zxl ze*Ao7*##9B6Y*ec;g7=C26UgeFzx90{5CcJzSBE9F#YA~=nrfF_EJ|?a0aT@pIgIh zU~_}nR~52$(@-$0*=YM7JF&{$#b%obRj)p9c5izYn83f9S>-wS&OQ0)zMEMphuefH zPbCWdn zj7EBHr2X+rs-L!fny?@C?J+>Z3eU#UMq;#U--VWnRp*Kv+F!gwMS z1zz&lQ?d?r3dDAG;S}cUfNeI1FTdge24S0`bXVu5_cd7+xzsN22r-J4Gj(mMxibc~ zZeUpPPc?V1xtvxEOr0e=(w-aT-_A>Z6t(CR4?vCYq%SV+2ufs7m8d~v)VixRX7E5U z67ybzO3Wh4(6?A0CFqdirAJ8I5MsSrACoa`Qz)>XdDqDK*qkgLDYHlJM_56l6#+C` zCk!GPjR8l5L;b2@NpNVljOP4B zVjZzyGc_Q|Apbo=ggNplKS+!^aTwaG78y>-0t%xy0{Bkek{6ZZK!@fR0n^^o?vtQb z1~5FAUjIGRQ97vHhI-YD!v7n;qJXP6HG+@9Mf&L7M4#6d@7xDGc(=2L4vY9=f$tj&$VO;dr}ek; zfEZ~Z)E0^W#hYWAY=!s_(vhw+Mf7>}@KklLPm)mcDjz5R>WJj63VhbBOs(Mqaf9f$ zsle)w{-SK1IXf)TzZfUaWsh~d=nMdPL)$e7cK9t6;(MXOBKN*fN|SaS4m`Q|@b&ct zRxE8)iKZFKy%5{954h7(am=`xL}(Y$yiaIN18p06(2?TLojO3?L3e}q9ik&|^}dB` z+P_fxB$lhxVFl&M0N?hjMXfgqcLkkY1!fsyBD?c1K$YfK_rT$6$%)l+!7Cgw{9wS= zictGyDcg6B8d!Ju)!FdHdsfsbFE4%zsmh$MB>5Bd;C5HRcJ?lw1^3%uhHbxzx|SWw zRD}F_1~W^Zn7#8ka8L{7AlayDAkx(!kuMXXvRUb3ke;s3o?-*qh)JmxF|0Ud>6V$e zy>T&}y-QMeC!U;&5{1P)5d8fJx+*jipDl{uj~o+w-u`n^8sUW<@WVkE$G_c$ya@P4 zNDJ@qp%DHo2q`mHqq)FNU9;Wd_)eDy&5X;yPQ%x+`{7^}$#wf3_~&N~U!31V2+GH1 z{Hc|NyuiyzF%Nb?SkP|9-%sTo3G>M!QFf&Bz$#d zH_n}cJHTNxwxAbG-n}HFjwGS_thzPSVuTa!Oxhr4yK@XKyn9T9A1&O075M(XO||5d z>nR0@^n2c4C2~(X-2M!o(pp(p$X)V%YA8;=d?VKRnMd2R{!`!2fmR1OJVzOwUgBGW z0TMQ5jgQ%6q0zI8lIi!56{qP7^#NKKX=AB9ht!Nej9rGKLdlmShvsi}hDO+~iBnkt zK4-Ca^rCesI4~?g~cmFv?_H_)+F%Cb79u6YBy?%3i48 zGhIaX__4+S#fJ74w!_Vs3ls*>?s#p_Q%O1;zx321u26r;v!DWkk&mT3mpWk0AWDeY zkpX|u>vBXbW&hnKUaVkS^LFdEqqr;NWa?;E$WQU&1jH}!z;c)}UY-&-stn`z5wPnq z85CtCSi%!w?r;8^2&|I*H#c8q!1R`fupbR{09rU!UWX?Loq0d%i(AeHubGp7cyAUW zUS{_Uzoho24TLESRG2yu65;<1b#8+n-c`JuMJJ<TfS3+>UU%ja7E^U?LaBh5zvi&05 zY74wMM084!AG-@|)EpQegI?Et*mHwGd&xStgIDm6E; zdJg-=f*0DOAUe`p2xt7`!ey4ex_g#IPYsR4bj~jxGmWiz%mmJCZg<}p|8M$|+`WUO z)XTzkzkf6=Hk8)JL9IfRz`N&stV;@NkKUGBDb;>H(5U$(RO$-1Kl^E#wY~lX5;ia# zq8K1?Y%%B_H`=Z^?{q46h^qm)*9diYNlFBcS=>HCM8T)HWCBwi22QOyDVUU z4)GJ~)d*W+mhSWso7*a4K>wcaAN}hSWp2vD>CgOZDW&!?(3!*vv4I3VvB9hKJ)rD= zz+gefrvHxWS2qS=?>vG{__*jB!46I$wsFw{U;hLzdOc)k zw}1Gg|9-BghBE<84a*Umb;tTI__52s^1Oy$6Qznz_7?Eo&i3$0BEI_ahr8?ekJq8_ zxZ^g?2q_mJ(eiz>?uGPH0CDd>4B&#rYf0SNQmeJ&&iP0i?Gs8013SpUqExnO!X8S^ zyPr?n-{^4J;LI36zEW0L|H&Ky-HIA|)c^H&jU{mrt@)(8V!k1Rro06oyMhRxqr9G) z{6lCxs-J}_n^BM0V|E$834oT%Ne`os9Laf#Cw4FUH~t(oSgFd?7Ws}38Uw0{Sr0>Q z3FXW7n>%W%z``DX(*dP<3 zWdvNnB38aS?u3Kve5vM!Z#UkAC1leTH3GL?4)3wgG-|^O`b`3nuHw(A0)_EKa-*|G zOF-b=FK$q+NP=B!Ou=e`qwM_4NR5U=UR|$9)D0xjZ+^FeO-=s`-s9fmwdIl-FRpXe z?|6Ov;d+aH)Rfon6&>MC_fuPfGww$S(EjLd980F;37nA@5ba zwgu#mlf2nnOp?s8Ie)Sb=PZ-h_tN#7q3Lsf-QY}eCP#N$KK%6QBa8E}b{kL$ZrmLa zOKWteR$6E9?Ji!ed1AWY(u^4w5F;X@f4sYjzu1p=BTgaB8Jb&7mOfd#zpLZhi z$LchER0V3GW{YuLPYv0bo$CRLud;TruX%`wzd^=y1 zRh0y3-s~dF?_u5UaP%Wo%c^@2cYR}+&Ww{MLBC1b5|DqHp_xBnGc{ut#i|x_%l<95 z&J>@Ik+7W@&bJpSr476cYY4^2YsvcG*1Eo22RXyH!0&C~zF4JHjsIXImxt4G5&iYt z+R5jhv^}tnd-D8)9?T8QPs>mZ_g+IEwff6N+G6ZL()f0mG;C_gi89`|4K}r}H7DA| z&oOJ<^oEf19QSTu6oKE;cK#H;PxfNrQeo44x@~BDp1awn{z8<#ay2sJd-;1&=A?4~ zgW!mpjhq9B8DwkqV|M}cua31n0WWX4_E~S@Dz7q~hUf-8xCK=GnkJzR`h)xiIs%WO zCo89)K60@sZ{QY0B0jJXDv0HyjNCBF_rjNR0Cs;xJPmVh?B86;&! z>Ofsv)(y)be0_B}?q%q({M?UE#WHd8`EBzv1L_}0ZsPCC-2--Na1h~Gdwkpt--}g) zz4Gy*)T)Vt3I}`6XcyDm=Vezc-`fdvMWN}dsf?dNx}vgtYkJ}w$=%9o*X*+QQ&{1hYRt{mfdW>PaK4=Z1!6nc!=CwY@=4;9WL#W<_Lp8svyhf&~DwPuBj(xRgoeh+b zE2#h#j+vl&?t}Vrk zKDs#F6vY4J(>uQK$dZ>H_8}tR=~VEF(Wp*g5<|N&vAcglJLq|&HPlO)%1U%7I)BgL z*(jd3r9&#R{jt){yTHp<ag)YTnFH=4R>?<{sF)zGi z9c%*594JaPrj0i&-h^Fbj{KB^;o4eelU1fMf5%AiRWq*FwrsXxE?m z5C7<^CC%gMDIxFt2gZ~IvWpIFmXhr-d~V);?B82$4dDj2U!#<~R+di23B#xPh#Fgs zbwW6`lV=u{7Ws2(mDGZGX{$}^ZO;!}rRz7A!c9ZC7P2T9@7LG&FuD+34E+w9&o31Z5VP`7yj>Wzf;#bpWr@ zxEQC%7mgZV;8H3x8d&PmH|K;`sDC-=Q}nt*hj(Xt&JOtBRf$4TX_y?M^jehaf+RP# z95>;~zwX168dBx)2Fg$O(B+1Y#LQHitPi(+gIe0A3(zJbdU2Hl<`f^6+$>H(#5*3ST~J=doDbuoJYzCcF`^ z^z^hV&9ZS1E+F)zh*Q5z+o*F zBQ%w;Erywo-JzG4h+)vJ{cx%;_k)2yY$_aJERnxraOU;lD`gt+FgnALIJ+h&+SJIo z_N!3PfVy#*HcK9S>z)htqBx)stt3kO7&dY^{Ij_CPYY;W(9Kd*+ zu!m~8%kCAK1E$sE&@E;`2Th!-&+rzmIqYV@++;wfO<2!-snm8~5 zXn|GARNtljo*`>DMx*!WQQ|UsvZ! zExY&T)Z2*5KxWo!ypn(&)z-Po9Y&$~Y?hMD9%uePGe(mv2hGEta1b+O3q^5Fj9Oo| z#7-rr9d`qZXo^p-{dXkBLP2tC^jCH1o_igm_*sqGK@+h@St<|e8}yi?YG_b^b0Z6Q z<{dnYD{{?(_HC1s?3y63yB`uZ# z)cF3zeD?eDr0ilTXX_AiLqGicr29D-1~Mq*MgBx>w)G>AfYqjX9WKCf5a$LQ{%iG@ zI2;1_?sTI*kBCu)iPdSXU#1e$D|G4AaE5(|b$taC3T77KzR)_^x=p+m7h(+H>SW$jfM5oNTLlQwD{ zGT7;UGsz9A1wU2=Zkv1epe4g$FyV%fT#r>?ncv8V2u8HA zzInG&htg&Gs}NZ8)Os)3t26SI#I10@S^~J;)fjrUTPApAMj?d!$;Md7t-0)&ImGI( z*!s$%4>xW07jd86qNlamN8=V%j^n^n>`n0X^!Tt$(k*VX*B&Q)QTnk(ux{?Jj6HymGp`2w$-$SBWfJ?W~pm@kpN z2ijisuaz;lF*kR>pS_`spI*o@S}k~HHzRTdQGdXTt;#JPrbU-eM?S>4kS00sbA`+bh|zk|-3oF8&57 zSNrQYMT5%amkztztgxI#db2-ZLc17oAg|L{bKhhu=x;&kE21k*8!l7)ngcv^STEx4 zO>W0+ULd(X^x;MF3Y20uA>7CHM6S303Dnq5NGN+SaL^H9Rb%TEBH2X*bhH6EnVg6M zw0L&5S(Reo#3<1Vl*6l%-Vc4YI_q@Fm#BCMUQK8-PqRlY^|!k4bH9G&Z7X}c1`;%n zA>6auvH45~DtkR&rYN{jy$m)oG~UN=y}V%A*Ex)s_Q|XrN}pFBsnz^%44m=w{dmc9 z(}fbnb=4Hm9>tW{gzt$czDqhz*pg>KZ&SQK3U-RyQt$kmCCEoA)dLf@(4~XQ?dsg8 z_=VD{^`&&}l3(Feou@c-2zB-@tcJ9AA@@4llwZfQiY?nLK=S!(N5=_!nqJdnm=5jh z>M~h-9T)D`8QJxI4A%J9(FtV7#+{Y0dEvXl@7%l9dImvsQ*13;EC?2kIy8EhpG}mi z^_jo`UZAd45Wqz>k3X~rzb<*k>G`V!{HS;25`_EU_xDV^le+>iu)H&4pTF_&Y1i^w zae7Cx~Eo}APc2uqC@RyBVb6cc_GJJjU8jnV%00^7$S-|xRX z`;2%+`m0*}w~<`?_-vI^P<2m`o`sL0Q}il=ikWbe+($D!8M!z}<(iqIdfY<0RfLKbycMJj91j`Wkye><{NFNi7HeaELQcJB#l z(0vDyY_4Upd@SR!I~hWn>SS_t}LVXVIQhri(HuMOPVZiLFODNwmWwIxg-D zO_H@wil-*stTmD58u}rZ(K>~|D4DspBHgV*3lNn#_^<-R5)vhwho}UX{8%^9);f2q z)Sb|t@h}CRiW1kSnqimdCk~b-j-uq^;-1Fc4)V$p@d;{~M zB{Bs$>77D#bvuIxd=g*S6biu)HHq-&t4`Oss6&rvtypI%OU{)V=Vnkz3-SOe(6y5VpADp zSh6`+RVJ|ybk;(=fc!dIoFwhbbDmTxwII!07iUYS5W&1m-sQZzxFW4f6cmwt z(fjiv`21>en#!zhYY0_<>(Yn6&ba-6EJRzRi-$XO+?@G1fVycIP1XB}_Jk@~DtT%~ z$)#`3e-O(pq2u!pXbcmqYI6!8jO?0;#yQ+{aK+ypP@kmTdmg* z^JE0o4$$01i{<=7W%|c~$;5RtRJl0KUhhGFP}|+jDwDE)tj6dCcuD@-u<1sGUX%Ox z!s6FS_utenrq6!Op24e3WvHog>0I16SicC>s*oHkFxD~O)Nr3Yzo5LdC5Je3;efBc zg9Hr9Ctae7BqN_FU`D_cWt<6&P82e*>5dbn8(_FG=Qd>>RM}BHt;iN-Skw{xNRoQ5 zDvh!^eUS1g0WsL)M{mfckt{8Lj&ew?M>wO;v+rTy-tP=3Jb#!n;qTJoXlB?12=7Jv>(M1Q4C_Y-D|u8x1ywxnE;p+kbD-w*sTBS#2aH=z4kPRh_9Vj zkwHMn{YT*m;+@7Zv_Ne+5sZVRKn5+Twnrb$htYcfS)v&WzPzG<@P~6X&L{*t zhEQK}bb@Ux%CAN3TR%l1_UO1qSLB`K6iV{i5)!mY*vmL3^S8P;cRU55@8A%|d$@c+ z^Zi`=x;;j{+9Jn>kt$|H*!Q`6pznJh5V~UtSy|gM{8%`7WRK9>>f4eEaYV0IA-gPQ z*91)N)*8=rFgiJx#`-=;>dU|5i%-vd2L-;2gS#w*yqFtEk64!Kt#2H^b)5ZKed=dR zvV2|9wtl9z8s*w#_5(9$2c?h1r2A@U^A}oeLSm(-mHoh0t6)B6w{GP_&r_Y$_u8&y z1Fx3@(d3ulW{IOSa)&b~OGQnSO-UwExoG(|HC;FEn=>1N<$URG*M|Is{fP8?L_+Ol zibZ;s5}jtW`t*z9CC9mn?wXBs4~-p2-x{Hlv&~7|g=$2b&lgxU!+Oje$GxT) z=S_;6HI1#;_DRHHG51LCIAzFv=^X9U+pXt*@zyV@wX_n89CpBTer2o~lJi(OvErTp zKT?+L&(`<$njD@Rx+bZAC%T$6m*$WBmu*EJ4D!tUn~2~j@-^9|u%Zq1*-0b4?r+jz zNb#}+PL`R9gPxPyDcqoxOf^leTTfFZsYO@Lu;@BYs1YsLr>^o{1PV771twcuq#l+q z@if_-?z2F{MAS&$Engjd?XQ(IIimEm!4e{PeX7XwdJK1-&b_@5>c@aR%Iy!oH03)) z%t@?SiQbhOEGdNcbl3wNj8LKa=e4pmzaUHtH@ZnhU&h8 zjDN)6y~PVZ3Sj}k{SNi;xZ?zTI??4X@a-(hA><+|*x*0422djqWRTv9VoL3}8c zUW6&Ld$=VGB{c82kzlU+F^yP5ll>N^3H?yb@eqGT@Zc`N= zy)@OxhAR^{a9FlZ$-R*OM-giyYG?90U3N-8mHbn)AU@|)$6>sqAJ5w4rPHF{FBLa! z>G3@~Ycbt|$F16|Wg$=ShUa4)n|=PNrJ75BPEo$;`e!9})Nrg`(%%JpA8vZ2>0@V2 zFtTv?bicbkA0oJM!%!G?P$QNs9>X=jab}vBNj%Ixc-V-F?+H4HkbN%qlflnTGL@+O z+`l-vn)yOesKL05;We)iVuhQLYzo(nkhf1TIwa}D~eWogklVkBCvsk6Cl_;Su_C(qP$}OY2RckNFcK0cQlCrH(z{FEYTd@1P-78Jn95(Q^TMnVt z5!A3t2_Fy_4pQh@kjh-B8_$~lg-+w&YTK&-vsGK4P!)mMA5L;!Y=-I3H1Cmk0b>I& zr#EkMVcrk=*VzVCfOgmTE`NIV^{RTa%xKzl_LRG5B%QOT2}`5LTg;l^8?~v+m^~a@ zh{ZFpwLxN*t?EtmIEvhFa$|mO4|hz~z|v}ARBuDMdbsps=Zssi!c}oYJ@Z14cKyR8nC$3R{x%eJHG4R#PINHNEad5Ve zu=(@D3IfFsmp3N{UQpNr`@~Z#Hj}08tQ2T24H^J%FVKO>8kI@e_yi$j-MYub}~>2n^JiZcV?4a z7=1a=sT<(;Vzeep4hb3TyON}jg4S1NXtI%CSac!T`0tl4zZ!{Dkf1%jF8xb>=01Fx zy|ELvt{%tvA4!Ey%L@|O_P?yWwbQxl&X4};-*$YMC1k+pJiyC=HDQ-7s7C}pj3;oOJixR z`=q&lf0FH){tCc_OG(>a1MVyZ)wR6^Uw=vLk$9Q&(P@n&`edLc3oFobs=GDe9N$EQ zl!UT8HqnruCDiZHj>&Dcs30naoa++hR+E987~A(NC!ke^aaq`MD; zKSb7DW@o8(`dX_$rl>}|%GKiB?lh@5@_4HL_rcRhx6#i^lZ%@NH@n>FVO|ax#x7s& zi6hC8hp44xhL_?vtSDr}$HeF4%9a0+Ph!Jv_*h6Yd@>BRZLkrU40#Goqa@E?LHVq+SGi59$)0o!GP4PPQ24*iSl?x7xLw0Jn zKxyC~xs1>Hm=dIg9yfo%L&)}}14~%m*KY0@f5G zw!kS?WBjmP`UBT1e#4Xy<#7@6)_v8h87<@&tvU|ZI;%1wo^Bj#!YnI4J}W>Bj~0TN zOFs^bcbO9A^l~@tl7b#-i2Nw{K&q}hezZzox6%hd_lSt-DW%9&>J~b@{#oK`A^5jC zP;i33N4R`%t(23A4p)u5cJJWXTbg9zhwS{XaCCr<`IP3{ogVyv%% zf<1L^u8nimjjK?DJvYWRlvyfv=s95o^ZfG~SoygR84)+L>))$IE*g!#vyWc-!TMfc z`9{+kSYnM4_>S&dG?9Kj=3>NG*Q0&BgMKrSjZUy{%>eH1XXX9cvBE21(|{X4$l@E^ z@HY8Ab!F;1&WPRq%2@M=eUg7qg!UE$HK%YZ`a_J9b@PvDH!v%ARP7x&n*j^BkcE*-G$(|0b24%s!6K*6a@%^u@kTfnFtxe zB3Wq9?nbrRh0?1nD5dm0M$tk%dSYy?yGg!GUi*cWZwN z3Zf28j#zzO+M;AhI5$Mz3EtD|32IqX`|j|p-6A+^Qaw$L+)9*{H4`PSc01=qJ}k$B z=h(d}D{LYYhpcXs-%g2gQFAax7$j zIlL_><=_mtG;UwoX;5iNY$_YSbWR4k^(}mu^tevP0xR~j4ox&kc42GbFF@DAYKQFJuMpT@<=GT=zbDnyW zmRkSswgoZUf|7;D5vxB@c#nqF61`INa`m)F4_2fjFf9qN?ZgDrBuW)u_-;9`S%nW5Bb-i`6&d6iU*9k<0XM4o-;Aa13cTL zIUZ}18vlX8f_9tw)c0rNe@;%s3|QJ6Mdve~{SDr+*Ve+xkMdduI)BqL_8nCk#VO!ZHy^ap_qH=50*jv1@%=#IcbD4sa?^DB(at# zK}YKWWXkt3a5FaT{+nDxDwf0x8Jmr(Q<;B>RYGG@WI1M65Dult?ibzR)LPPrPY>41 z{qRf=CMJ-ubxgAcjfSdHFuUnLjEDNyb5VKbzWfP4g-;@)wi zt92e4GDz4@JXYdb9>67oQm1`X81O!qd`Z`LXg9$Y5u(ATk;aN&Qa^AEB&~8Lm4wA zLb)Wt*|RkAJC)dW$M|*yhB?mN7mw4`_D(P1TYU^~3?Tj!&g|Lf%eR)L9=?}?7slWUTlm7I=qOy4ye{PiB$c=Nc4FhzJ+`$OZw8j|$JACID*J?w;49#R7NezGE7=*C#d&ag8xnKh}b&Nn}on?jT=rik+rM9x++4 zry^<;_!IFSM3C3l1Mme^k~b{}*{SS*qvW@)((c2!lJ(7JyXC3c1ST%lXh=n1h0tnK zMawy{m0`q!_>avr;9>=&oDKvqQ{CvPOkHjz+WqU+9y?Ua-;D42`McObB`_fF!O4QQ$WG;B`G}{u{cUtVCl|eN z`nF$>p!@$Q{~bo*eFb2LhyB0M8=i+jxquAB9?8zdZ;nup_1~%o0voNcN?K5EZpM#R z8B@Mn(iQMy%A0)pWk#tb-tu?oHO23&o{;u7J+nj`iKiEJ>GJr@<43k3vQ*(etgz0= zr}c2i3<1rv=Ps6Av{0#bp<3&rK)C~w6EGmHs&=OJMe8pBNWUr$tMVUQF~V=CI1wn>+hj<=}yE4rVoep5!S|aDz_s)`4z%g zWS;h?!EnSu#MV<8A3U>Niw&Nh3#Ll+bZvh~6Ubyo2a9*P6V(F_<$DItVH*!aClIgl zfXS#y5FTF1`>{lp$^>O%;7BF6Qm* z=D>uXB%`)r(Hr%WIWkTt#O3>4HGD9WJN1rO^e++q#-Vy{D76;fHVcn{j%C;O`1sD# z!KKFIv^}Qibz%a;sRMw_Y=Of1yR6U}%ora3ioEe@lr^b<% zQP0n^?+XCS!rXDvwaZjnTJe zK%#3xv{n#!gA~O*SXQ&v10Z=vJ@Z;V0uY1X!thNgbaxd!8%=$}eM4S_qW{ib{g4S1 znz;=|=1`KjA}@`cuXnjlo3A~ii2VA3D1OL)%ukA=Cez|MyM^fbcd7Aaafu(gzA+m} zC(6CVWd)h#Elkif)0bLZQ@CX0b}{yy6WhCerll<`T3bcpDxLi>j*sM6I3~{3as$6N z6HAArrA{mchSbU24$76pw=_Ol?h>Af_TX533~ZYG@jqQzqt@->3tXc3l(7HhLdNMU z(V|_)P07J$l3*Mi6o6DyMv0F;O~N=2jsDyU!JyMIF~7j}d5Fc&{jO|ASL<=$xbI7Q zl%zEGJ>Ah{lv0+93RcOjIap{ASEEbE6~fOfzoI z^zI2&0KSWOQ|EegbPNg-Uk@HBp!u~&n%(Lvt>b*L@@rB11+IgAFy}`HQ)K7&DG^~jZ6`xWJdP0BJr3yBMG<=>t$=2;S z9JrVo6YVDSO2tn4zv=p{SqumCt7;O`;G_Mu?OwR%R8zh-3vi8f0bdpKC~T*#3``!f zntJvYA)#7GXRUuKPxF%O5?O=IfLOXH6=u#^&HyK$Sg z1^!Q7S7L{PMf=$}bjeb`5_`=4Nlv^&Ny|?^%oPD$H=$I4rUrS9DOuQ{?t15gnzH{g zfVp|5bTWdyQ0R{-ELaWQL`lq_A!*Z65l;Y29E^I8YFMy!w5u?DJJrhF6tEOZoo1!w zz`~q~R_@C5NWSipw5Ld^NrjmV4Wy0y6S=*HGoj<|G`s@d^dH{0?wS0!naVF`2B8S+zewK zxU=gC3gHm1DWy*b4M^94G++D*wlNL0$$+I>Oo%2drqsUEbQcHMAL#P3>2;-eJ%VNu z0r&mIu;#$^$vK>@dx=$mb8Ty*WTeVGVtt%#7ybYA-;R<0UbCj)_%B%t$*6lW zEVgzIG6UTV%Z}KhMrm5#>b)=G=g>4i5(6;$3~qw)!e#NLrhM`i)2XIW<&6E$S`~s( z=OH!EDtv6@^so<7*OMzhHcRmO;>K%Rb#d% zD~O>HKM_430a_h>!ORZ0szmR;R`{MX0_XH+DkOxBDMvqlN2wkf(70g9sNVWJB``O2 zB+ioJUHWDL-kvoL@Z`n7MBswq!6c5Z+&(!H#OWGPzEWnW11Jo0iwuD>XDz>j8Bj`a zS%BStF$BBeLwNq(`Q$<)E#YcALxw+FneseheZ$UXxZa?7$0j;+g}#^jNK>H-K}dNw z#A7-^D2Z2>JVc%G@W(;R?ct4g*4leIL_dyKv=A;auL-p!0scORgv=YH;U&V3F!RC{k>_pR@k9~#h{qiveOtRlgc z|C%WW%^CN>wy$PJZ+pV#cX`8`=E2C-Lo)ozaSpbI0kcZB^x>P?6oprxw6Wr)PDdYA z-}sc^yw7x3G*ht&faU{#N8=~7RjgSSAd>fZ_D51}wUJa_=1rS>FDX4UA=20C8O0k$ z;ylpOs;#j^9x6q7B3 zKTCo5O2=`=(RLT$oD542+vo3AcZK3d{m)(k=JI*6^kV>pNowSt7?!-Uaj7yO zQf<5cJsA6Dw(Rt7NC2_ao&;Kbw7#S_Sgf(yQxE=_hauyyS0ylmm-U9fKRMh!Td%nF zo1^t~&?b5(P)hE(Ct6*aFL{odVXo_8&~uk34o zx4!8f_wSkS3;Ay|XT8w0dh>A2%#{KrqcVP{WvmUj+v|D1quDDDgRVjj8+fgJfGd}A z76X$CA_DEE&$!Qyu%{Sq)UehSV~5_iJTA~MF^}Qie4qa@dd1#KFkBBXa_fwnD3!ob zj!mCqTGjAVVdE0)vB?6v^4bnM847*V38w_7Rb!Xyd~!RHEW4h~tQ7(aU%1wDWGbl5(d`PbjLKDSdiM z$M|e!?kq|~_Y+G*qMb*UrelJAa?XQ30Zcgce6p=d#Mn}yx4Qk}k^+b`$=v{AV>(mi zEcJycPA_+b8kp4hS+YoZni5vn1nG8D$V-3LPDAmsZ$&IzxcPRsEe|7I_^aS4Hwye0 zAmc-6e8lrlVcR`m97e#x@hd$J;tR~G&JW2+7%`0>eb}i7ReqAU0apW<`_Dc)bFjqC z7e*3aeDaO&kqT+v~sBB9&>TsGbLR#CEqPfRK% zk=Ro^syEdDh%~Qh9?SEPr+g*DnynwFdlPYyF^BJs2f;z-3RA-TD zFQB_)(O?!%7!d)?Rr$SN;J<#F92z3wSDx)V4*oZzoA~ffA|~)XOVHlrGctk{;`X#) z|5Qof3QEb9swCpD+#@%|kNvIZs+VcYTlm^%#^!I+mz?IPtdNkq9ICFW(6GEy;plgV zEo?UI4}qCC3@F)oQLGSlxy`uVIXP^bu=Kha0A~WrKrL1!Dc0?I%#>dR|4n&B?6*-b zh->$*?^fk+;vf>I+l%erqljR;;E9|(x)plTLs;KzO}IVyn1Z03GM4&D+E{P7Povwx zG5S%dV)7gOKqaSorl1|8vug%;O=fMnyYA+|g;D7O8dMi4&!y0*DC*`g+N~DL_70xT zx^sxSq`a@Qv>Sb7kD0bZZTn0}wv@Wn0TVIVqM6modmp^R@TKDqEVoEeD~dpUT|oAM ze!bSS7z?a}z}u!T%M{_5<&Q1g-yz&zIZq{UV}NyCPn)74DIy;pbiMmBl4$TarOnxA z_S@)kzOW3li>;o4<=!V`U=f=#5^|PkAbgrEFfrIV8+TQ~ zzp(hS==pw}bXy3&S=4WoLIh}UL}!!iz+>UnM)S~9-xNS@5fci26SPT`@dXhF@-E27 z5eaV)SvT_xRg+qI>P5~&SBy};(LQhsta&1Tgl#zK^urhAE!&)Sv)FIc=II+x1XZ3b z9>pXSaMs7$Gw)!AFWVPzefF^LXx~PCCVhLIw0n)yeV1j>3v`8cyh7#-{vFilJsL58 zu-sqDSS}@YaizrQ?JfB?E_B_Rm97(+@(F`FY>&uht5)H!&w%vFa-t*JuUls{I$xDPk8*P*?5%0|eGHg^&Ip#Z z#2LDV)=bsZRq%Y%41Z8sf!yCT@5+DJJ2UJY_Uk3>@yz4-h-OS=MEm>ByL@e*$EnSu zr{=9z*s~`>&N1fYD&_8B__s{<-=u8!nH|%Cv)U+$Zgx?~$RXB!gr%9ZdSmaUX6c+a z1vJ?^4sBElkAgAqk35Hq*v*IE{_}F%Ia*hO{k6 zc0U+0op4Neug`^Yc!m8q#)!`2`SI1$G`+h66RUT%;`%mJpEKEusg)hW_6!i?TiR5l z+8;+!Dm$^|&?yj{Nh_SYKCh?a*L3WXJzH-VeKTcO-#O*A4~T-zai8vo)WpuKxqie&tE8X17zhg(LxNXu!%p z__&=g_P35oC>cttR&~u(Bj?Am!Nf)F74Dk5RO;JiOW%{(WBv0|8K}3)k@NG$%l~BH z5}}zt?O;H-d}B2_>#po_15ox^OOJ&Ii+2I>h2VD}cJEF1Ufmg#?}>8yAn`Qk=MIuc zt>q7Tlo_P}N~K;z%Dco{PU{Gh(Gsz4|*x!MLCGWRc`Pk ze62L7Y99NCYlse}{`0~P#pm65eY;B=*Y)li(-0fP1Wn@jUCLa z<+joIeW?Az9kMt5fHM=#A0sX1Xcxx6EgFOG$BU)}sdx{kv`=3d^4GY}v5(PrjO4Ey z75IK-!aNBKuVlpYUB-3Ng9JuUt-^qY4+=$`yv2kdjO2s)#uD%AJ^s2ki~sao%1^<` z<0nUxsIQ!R0k}%e(NW@jMkZnEL7Kvj*#*=S!z37yo3e&|l8H6=r zHLQ-G_5{f`A=p#HcuQzjv6vHuI#0Xino=H@YBu){e?f#FU-pV^UacbI=&~$l?l%k0gC-8rlZ15p z!v%`u+b3OyG1WG;)TO(k$xa_Ii&gLYTLN*$EvX;XZNo4A(Zg*djQoP zW^_O51XoSu$qz~#5FNP?m%-QSfrfw%9m>C$-ekcp%YCwlhfjNjnfW=W!Oz!t`L|CA zgvBsc_u<<1pJ)bI_d>D3kFtxIE{hEeTv7EX{kX2j3kgL<3<4fyYySjfq6Hol#0tEi zP^(L(DVg_FG*A?M>B`gNNd3645B=7AFmqG2Ue>y?`lGICW#7f#+0n)I#p1=vdd84H z_rDN}i^9h7RRsS00&ou8E(}H-bbcA9CrrW7fMw3>-Oq_Lmz)BB*a=f{TP_d2-W6~C zVp_K>$8Nr|5qe>9?Ru>G>@Tu&Je8!W1KO$p&n1UBT@n9YOQ^q$d##wig*aSBdR55y zMc4#DR=|G=1<=cJsfe(M?D)EqaTnZJ|9p6aeV^|*Xc!}FqLmzTS$(SD&R(r3ue}kJ5$%~^gtufC%PiAPwzjmgd1ogMLZc|_wCcPX{K2mF~`+rV1 zEqsFBh=+$c4qJLz4-kq}A**1VaO;xan)=v>aCRTe^4=sSUC|9Qd=Cd1cXGI0JlKi2 zt~S>IN7vD`%p(NuLx$|xg2LPRP!f>pJ4vazeXw;XJ#I?7f|(!*RN;g<9cOht#s-83 zCb*x<_E{c!_Y)bRkb+q#B^=j?bkqORq{95~@sya4tBlYz_t!k2xsY!{r5no~L2pbzqSTj?v}}qqgXR3nF-! z_$G$+k<;3OjipIT(fRQxwr$gQ2lEf{ zGkiqjI=Mx5N1Sh~L?S(Q2$reJm1ShXJNB47S073AmMQdH;He-3#UMoIi|Dm5fioxt z=Y4|(sww(0Lw9qUMS#KA)+&7k%*QT`-Cs7=FFk$+?I7mYJL#3|hskk-*n2B36xde-=N~47D=~=f0udoOzpV zJp1_Ce?TB8T0DOPRJMfbk12)90+4dDTc_M;fM{k~66NjKQ~7h)o}9BZtsxcQ%Emtz zN_I;N6a~`x_lFf%jxNv7y`^Y%o~G2TUMK&|BqW|$_YwD3 zujp~|&^Quj#-C9yx*R~LRI}YI&$`%hy=y43jUZpP0+{FAVssM@$g>5XUDl5o@ywx` z@V{PL>~m~BTA-V2{q=M)Sb-2k^^y^u51n5M)({yO0JQ^!o~;rSI-w7@5mSQq#3$VH zL1B*~LVAys3>UjxSpdLlv!df zMK@0*aQ0mV3V5e*k+gEP$X#g!cZ`7JfoN`5jdYJ=XzOgw2e5ExOxZT z9_R$r^kEY(0Ze!+LSM`3)^R^>_k;^CJq~v-&&Xv*cCl0fPXOb6@AiN2n*>ZW41En* z4AvspcL|UYWXcIMqw!L#)G$tnBHjviN}N=gwH6Xb%I#%DvtEtuOG_cvWTPMorS&0! z{m<(Y!4!O_cZEbiiP!00ht)7yg`n?cvTb3Y!v^4ta^9d$gfm``+S>NFIu&0;% zT>CvPxO;yjPvb8PP6)&uI&Yalh?^aAOGjj)+7l5W1a4gsa?WZ*vLO7LF9l3EsD{H| z2Ly6vjke=%yz%l6gd`k^9`2`)D@IuV{UPsaaVrB-|GQFAbkjcf^+#`Uh_Y9FhCe8N zd~dl`lf(EXr2d`e?Bor(Zlf4aXBG*Zu_s0uwo$%%3HvuC+aGbL_AodK05nfXzx>zP z@Zs!pOETiL>WH?!(0GOEGmk^dYe9(?{UpU@Ek>-#{_KmQ?Vi7qP(LcjicvF|cb7h0nhR$>hZ_J#(>`^!%}qtQ4`)>n?;FnVCQz;Y+-BT?IXtllByH`Is` z98NjJxg{cs`miNx42lWaj!2*8tyMed(-=_QLK{vxTz>kdyo*FJrorIxklTSRQIDb| zB!eQ<6Fvcjl;=Pnu2`f=jzcD(-0)#cZ+a$5X>2^BO=N<;v83M*ZxnMrgzXP8gDlS_uo%<8V7HJHdCDl@$Jo7#MO*e<5$c= z*?PcFjLgcET}o1fMP=oq2s0hRo?{>%a!32D{p}D4G3Ef$KiCclh)Ay!R%vSpqYA_E zbKqITgP+)c^ZT|PCxsco1$yIi3y<8pK&NlJIO!j|eQybupa#yG@LA#x+iL$cnLkh7 zZ$!xD?pTlkkC50|6AO13&!m5?y6`Pzu+(d=+HtRPWSGIm9f8`T7u4U(Fa@AuO7Mb) zCiBB5ba6>rQePg>RP6an_12gUw(t=qH79N;hDwBXF8zE<2`r!_E1_VoSh-Z%6LW1V zoGFSA$Lb8TE_(JMdPKDC6+1FqbG^s5;Il=MI&yvMc6v17@E`?l?9yrcC+fz3TDrJ ze6MpXkub8aUO<}fVUH)f*&Q~NM!j#_vLqUwBwYZl!W##l|Jor@I)?$2dG^U?81e}FhKAXANY9Ur;NFRPN zusbbLr^L4JNGc+bEP_~kQ6_`=1)I18w&Baxc8y^ApiHzc3~b!V z1htK5h1EOlKY|K5ySPMwyf!4}wTGf(3*WZSU0isIW|(~Gt!dRP$A*(dfMZrCO-7VeF)v9;Ah15PTTgKGoJEe zbgr-2b^zmh$<^uVz=ASvpHLF-B7ERrkyfRnf>5)3CN*)_!xaW{EuC>v3 zqU_ZAJuGeUM*%r}fBJCP!%*=&68_w|oeQp_EQN8~*>+h&FjC^}U=_x*U$qGfH=7-& z_as;CN)>{YW z{yiR^h;YG0ppX}Rkoik_ekfo5LUn*^YJ2s9{>GiNn`?{Pe-!FGv41qN|8$1~a10GK zvj+HAx8h@^2;TXuAHl^2?cKqZroZIj?4*^A$mxc{*=QP-+;8%N)OX4Fq4TxtSC0gq zX42wGMQ|D~d_I{R)7=lS!|c13&Pqbhzq+pXfb=yV3$w=0fK}QanzflECHvNiJnE8w zd=|9xynR-;HM%wo!X9(e{kjqM3DEbr2l^!i%Si}<#7YVjiM}h*%^8EG^aG137K;*L;Ewbr=A2%gv^c`$td#8C{J( zfIdWCSm7HJ5r^A|;c<*^IjbA1o6h!rE?8=~CE-2W*?yMpujbR)7naXfgZBPqBo!-< z`kg*_`6|`rQo0G3Gc!ATUmFH>FRo+i$28r80>-g9b!4b+CBDmUV8AGcp+Zi<%H8yt zv+wc2Ug({r(4%?JF`6+RhKd_LTk+?miO(X#I^13)N=#CJy6n#H|>Ji2*Q^dmFZJt#X8+O-ahr{p}_~?=GW@pp1|haXra9LNusFQN7107&;ZK5eM8zi zWxRawRS7Bk_Mt77{IxD&s{GhP*Z-gW9j#qt`n^sqm}JB*_}p%BMZWEy=b%Axdl|D@ z@O?~xQvdp{^_6Z=AJo)m1fbD&f5aOA{)f?3ll*vMVAwH!QeHyL?_xB(HE$)22jjM{ z1)X3g1g>1aL8@=Vh6s;?!BKVlJIH}Xh!!gEncAfMZw4>M)}Ff8Hj;hk4)6X*y_X>+ zYh6G1f-}RIi>1osMLc!g$Nd0q1l%!6e#x?(8wElgB3x;w)xJHtwdc|eGd&T^R&yz~ zMoCPOF6-dF`_t<4IN=6&A5XkHv1qD=`bH7WI&li5LXd9ox6^;Sho_6l#46nP=9HTl zwk;5D17d9E%&ghI9N&>11y{AXpO(8G&%j3{`^JyIj5~!|I?s4f-eqy1^H=c?iDHmKj$FeV9mnwj2J)j9 zssB9j41wVM61D!bzM&yTR*=P+od_rJSJ>AFpc%ha$&kG6uV!Wsb6!LnnWdp^P@Axt z#thLa$ncg4M|<;OVSX)}I|m{KN~W_{@^T0nGdO4IWS zXGOhhZoC6;%D`}7BRl85nZG~7%DUJFs2Vy$WzTxGfvaaUciI~9s?Jhogti&*qL(y6 z>h>#*G8%}{7cU>p!5wxA3Qk~C&|g1A|H-BhpO7aPYJB}$Rb`?HiorbaLV-Unt}UIWXg9+!hk^D@hWWy3 zXnt`HIL=q`+s9p?l>?&}%~W+GS=_^S{t^YbwQ8ND1+3PokD6zAk{dmyI7F`}DD`Ps z9Lw#$CW>^MZv_8aSZX_5zyGt$bk=%vkfZ-b`iR6Tg@fH15WbSWuYw8t9u(joB`0Ac z{YJ2)Z=UoSbAX{}NprckDPzYkCWx^FBpwzPu`w;Iw`D;2_53A|f`(iU{+-d$9VLYS zQoW6s2@@5*P+R5Y?V5CdIuQ2NVUER%<7u7KL~}u|n+=6R;q$@V?MNS4S7NIv0r~Qd zfhR2RJ+SCQj_axL72~fcxn6`tF08`+Q#??uYW$l`VskF($wfLV)w5xP3w9AmCt^t~ z&?;E-uzP+ce5xle(5v%cVJ_%^ts~Wd;DpS=yBJW;UP`N7vbEL|GyvE%(6zJ{c=|^; zB|hPl`^pUoTI*4QXRD2O^sHq`?Yj`g6ax>URxgdJ9u8Mo=4G$;$4F~BDy zkFP$v*tPs)0kuMj1~tDM@i~sz1DlVGx5poulmW?$V~5q&>hJG4%+HF?p~HV_Jswn? z+3L~iTAD28b=QN~cxXOuKZj#kLTtd-q7nU=d@|5JSCL{z5}8K1?!M3pEuyOe-W!8J z$}#Y(ZAuatl&5>4!!R=7?-I(^rOwuxnY=d#xmw*YOP;CXwfu>3UII6SGMwk6bzoN8 z(3>n`zF%@P_9ziJGy5hAFO5qOZ{gX$-#Sn>Ict;1jrUmiJ@fEB-YR!>D47pj{D|KW z-Fhtp#H$xf#rfA<-ov4AvRmK}+YHvT=I5U^E}BRQTi0{ zND#6V^qtn)Pk>2e3SNBiuUnw*4f)53j&&z6N0AdT)2Npf%r^wRGm`9Dl69~C2Amk5 zGfAbE$hZ7}>W*SkqfmEWE#-$sY$u4JUi#Ne{7$*gdJ112q#cNLL(OM!X|GSap$Ttz z?e<@N9sr$-6Ygf}0hoR@3BY$je4x*%^*8Vb3-K)Sx1Z%GP*p%*-w->lC!di`PAlDh zr61odqGaeR9>n)LvD8=oxx5!RbRVbSiKU83{$HCM9e?@azc$XGExP~=EWgb9q`rE0 z*@e~*HewMn7_O0dUy$FR%$N2%y7S-GmTL{7LLXSn$Qf4BP^@H-bztmvrt`5q%k{y0 z$r1_J;njC$aDr#@U{Q`I?(ZF#hE3e8PDUAL%3}VEb50=npEu@gZvr1E&$mG#%lX}MYQRD z-xQ8mNth`f8qbDF7=39$!c#jWBkkg=-l@v`c%j^uM)n zd?lFLn4#~Zv7ahk(6CpHPgFVg>i%)RnU1pwa64PuZsO-Kz$5^!Hl!4td6e4jntg*R zqr!&O!}E+$0Q%ylb3@=pfu=awhjFXSi02mV;Cu&my)fwnEX!Jf)(2iOl%My~lErEL zH8toB>qxQ@Tq7gx_XiW^m3w6!d4Y%twGkLXOg>KC}{&a>zXTG)iLDnI6FTg1fr zHJ#ef^SK|Za;hh3or{ORWQk33``WGmQG^OmTjKj!8x zo`$#Ila9bYpD0;|Ydjd%6n?k(h?ftz9uVVJGTHvaMrn|D`__#eSN)X3X@sgDz8+oS zDbK~ZZj{;Zkm$Y^{Ur0P$KnW9X2-oS(mw!J4vgdgJy2g@$IAcUs6&jhU%%Naa~63k?<2!5H91Lt>DiznnkW(n5C=>BDmgBuy1f5au7rd zYg`)K_Q}N;JzgI#Y9;>y#fqpu`W)bLNCLRmwM}k5+21a0PuMSu2{MHR0&$<#Qt<0=YOoBu1wdq%*0b~;8!x>`7pUKH@$LA)X?s0HwBzQA zkRIzd8QQZMx1o+MPCL6@_-PSqt}%ogE2 zul|MSy^9|ahA&n}qMXYgG#q^UxosFHE36lGd&g>7llE`VYgYNj+bMz7-t-A{Zqawh zVZW@yUjqxa8Yo8EB524yV|H8?rov)hh75Plr9b9R1f>h|{e57?Ad}J6oa?N65B(uf zEzZ(X!;mZ|H9F~*td7k>HOUe;==|UTD`Vv$|9iUS~x=m*5xjrQcz3B zg6C}x)586@D8Mnh#~Ns=KUt}xuRD1i<}&_Dd~NLofJ`nSGGeFr2Fh=XrWoZPND@6_$lP(#d3@LZv3b?LG*ZWCBRGds>$>}7i_auUE&+jES zyyBfB|DA@!QzM;+$ocH&eG=xhdecn|zzeNt zwy|nmz6Mx2czs$oe9AAB<2P^0-)&_GNlQtN`Cl!*p`66M`4>A=zXjSp@t4&xzuO;d zz1hN!QF`t&m=tPgn@bG}8PPBtE_cdBfpVvHZXN8!mzj`5n%fsIpIhMFqMOAtp7ex1n;GaI#l8HK6DCz8U zqzJKbu>lChZDm7$M#curT=(Z>b)auhe9`e6fuC|C8~YrbAVNSKPJm&7_a@Gtc@n1` zlfe7;c@F(~U<3m?68gTys>SibC@3K6B@v%_hz$O(YF~dGgFD-_zTmkl_w2E>j{x%0 zo>ijmN5=q!tbwmT-&Um!xuY2J(yacH^CY!>FlBy8@NX z@ekCCk9DJN$Pf1N#@XTj0`KQt=r}(#337i~DUUy`$+b$a3UG4T*B$PD;h*e_k+rxi zs?dDql-DnpF`XgEm6cABvLo%Dt3+;_0DgYrO|C3H_E+u1sh}^C^*!cCuaR6W*oL>u z^*G}JW^8@2rFVR9D36W%{d9hDQT;G~YpM_C%Mg?ie$ic!AdGkQ?E|hk$xCYE^Z;0W zr7;vuCplCJ@_DO`qfGnyn{K!esq%5ZaWB|InKhh^K4KmKNzG`Sambs3Row`}+5Dh{ zjzEiuD(E%14gfXjKOuO2h!tC_vot)75yblqWuulCii3*SlLd^N~EhH!8Roxbm>$U4UN(Er9 zUbv>lL&vbti*X86b0Xp{ArZaZjVZ_j>yN!GODu88qANSgg^Kci7xC*r?j=`S{$Mss zBNUM8X8@(tBF*@&O&wmY$Nb*X@0`){7bF*84|Pnz8M)l%D(grph_=JuNL}%A{jx!g z*go*oKFxVahN!)rR*V)3blipHaYCBSB%VOT+*wD`>SbakO1#`X0BW z&moSB=wQ_gAVp&zq{>2iQXmlKoC{*Wv0FSdWvNlPt06RRxih68fj|w0ygM3G2C_ws z4@Rmd&9y7k-ng7w@CZKxtWU3_u9z4uH-AoTFJRp=Bo~4w1nKp@YM%4q(*2zX<0nsC z`cM|wo)^_bGTj!>eXxAelaK%3W0i^kfziI=kM!3w21b3#JP{&p(z733;;zUrx7})t zZEK8zzByQdRl4gga_}p2Dc*dmYqomnFkv6ivq#J2tq0$f`sxfd2ekY<8ly%IRUG}j z+s1`2yYG(_i6+;1zcBu-gPp5i;U_D{3H3&X=(SfU)=PG-Nq>JrUKTXlJr@I3S)j>6 zs*(0>jIOwBkRtdqF>@upRhy`@dzv6~eZh~ycnfxN4Z}9HNSdtL&Xj)y9rec_102w` zBg0TFf!v1Sjp$@rgo&zR^tY%4MNiADTa*Zk1$t=2539);K@l1JQ1M~7XtL6Bb(0u!XD$M?@@ay^@Kor+zW`E>!3f zBoBXLS(jL|ug~E~)n{{}92I6i%;L+-zI5&Sh>1s`1VL1m&okBV*?q6pL82 z{B&$a=<;gE+uvV$Zo_Dqkfk@lwDQ)%|Jp2k@U7|Ai-}NO+M$kT?;%d>@sLU#rv1J9 z1dQ1?nIlwxUDQL!J#y#Sqy`t9r;5+_M6m3Hi3r>!ep=l9F$jT+y1GLS@ux+S*xS5>stnuNY06{G|$ z%sYN*yRJ8#>3Tsf!2PIE!_fH~OS-hq$Fn;)(Op{+%J(Pb)`fGAc+oy$g&%DMOs3m^ zYhm72lAh9*xtx2->>OE*78t;$V9&iUv(Kdt% z+t(k(ilME(+e}VR-09{Ue0!)?T_~Z`8X@fvJs2k0$sKnFGfu)+WdC?{zz0sciz7)I z_!JMof6s0@%WNYl_f-=K-0&Y=V_fXo)KkA_{8RH#HVUhclzw&Aze#yD^idqEm(*|H zlQ1TgZ_12sDpJbW$2J14D2sNHSufxF+9BIdCnKfQnLaN~ZRT0i3B2Y+h!MXNX^&N5g;boyNw+$MQYeGi6*>4AT42!lT%9MSi!XRlf z?mWnLg(Yk@q*n9suT%LU&Y)CIuLjx6oUIVy@F0;mj);QC>q{nvo1I!2xam&M!|B4@ zs;@KQW84^82mV&J#d4(y9ln&2?ErOafqHHn+cP?osi~IAeLr3Qh0gHj-FEEAKM@V` zgI{`*F{!KLyVBwDFLx4EGf1!eiIVpN(9*0FI~)6UU=cA4-2ij^^GuuDeQ=_kMO zwdFWcjp#Kc%QBa3BNP`j7Otng&k3Esd7IcmU@U0y%1|C9Q$cG6Gn^hQ7IEnt1$yL= z8e0CTWwIQjejhNM;1bERru3>T%>kJ;ed6fy2G?2{R zG-?e{nIlHB_n|7VOtSA_OE))fP<$W)d}^Ps+-jCN)~=4l=y;O;TuvKb3u5wFpd;{o z39orGEL|o4o&FP;D!~=eqMHEZ`u*PAgEgJeE?@rPWgfe7>}$_P1g4)|eo6$52NjPu zL1NP3HIO_Mj3_Z$;EZX$Xd+U7PR%YjU@j2#INU1b%9a^!cDX*ek@W_7J8nMeFV7dF zBT15*p@!V-anDoyD17*&n&4dB!bf+wu9{886IRZ`f#XqbWZzes%(Y2t$6J$3*#0C6WmmrKoPQ+nPZSP~wE8-MX5=rfw*YFeg&*HCFZfY*;_|vDXtEmckwPRn=w~RrA!r9A$(We=_}H;o)0@-%ixv($mOp-eD;Uf zcB*bRlCZUTbD=N3?IehEeVGyJiO(`&=MK^o=RB#Vk9HQNy8bk4Ef_n%O79~AEWnt| zOC?;$zl53E&`a zqir9EHH7~aAf?qXx=Eyd*R(6wlu8o8Kr-m7-OxlZ&TfERF4yDf`${A74?>@;o_$f8 z>OR8CT=S~#Oqy&Tg^wwkYEt0?rcnHUQw6c*g)t~LxJUK6-f)WTP&WGfOn~o#uIAAc z!agQt38RaKt?i5L)D{0``kjKmU2at;%ksDQ#_OAUbzh{JWOY6$cR~9Lle9j`Ty3cI z!oGAte;5C9TYC<{?tTvHHjN0^D(7!uUl;<3f3bu*@2m$RP5wYcN?Ng{XCNmnU?fG*N$69dkcs{@91t(4I{1D_bT(zkXj6RU_gYjEe|6q{u|+jj_?uj~WA;wvqxn+ny#?r!Ue~bY z{U`}8v?WcBi?k?@xcQL8_qlYU1vo8G*26$*q0vTiAEoL+b0DH)1jDZ1h3b{z>0+1J z?R+a|#1vrnP!t<{b>eKi>l7St6TfMHC?};B*TI2d`lSt(J#BUG=|6q{bwLoP!1R#l z82X!W112`=2>w#CIPa^@i=(1z^Zbg-lgz{~eNSA6-7^)#ejvP0NxN5-#BGjQei!=6 zNz&)>_M=8Q%9)Te=exsH6WaQEKHX(NEHs|++xeypHU(9GWOziiPVeTcia7>B+j?$~!*FZ^@8Xr--nb*$~fv%ji5W){$P1=g%&vkh{y z2+qU9bJ`J<&4F(M&;E{PFr3_lE*m^q>mqedJi^UADnAxR{5={8?ea)jvrgICdM0cw zV#Ad%QrK&88y_>AWAHB^b#>kfiSc^;^O$<*E-fUpNUN86I`kbt$fV}cS|j+6rO$bR zesymia^qg=a{({>;JdMfz^Ch%=5@9CB=Q+7ss)>Tq{x7mBltZ=U@LV;ZI+^@ET%X+$7hu(s-#$|XhR9#ue+q$cf!-!{J;)Lwncy6Q1ze;wT z&N!|CAkSB6CD*iJhsTw2U3Ib=xq30aYOLwX+nxJ8-Z z=`MEKCxdub2V*27*oM2+^b*ey$sl~xQnE1a450VPPV_b_>@l^$^p{SApsR$H8MfF+ zR}^`hG6ra840md))1qXVMMs-7S*1;SgCe%W{cFWT?MO;no_i7ukTLlEU9 zc8+t!0CjPS%dhm7Ml_zFp?}(kwtV#RH@>$YeHto~z7g^?eC!HRI(CtnDq58)aQ^RI z=cx`mJ~(_oAbdT_2oILd%SrWFgC`$Zlmpz1r*AOB(nsR#r>7mm@O-fIJJcfFkKJdo z;=h4<5yOLPyiu%p>uM71u(S2W%n>g#!djk%Hl`PjEPm^BgjomwXK6~ltQcPJvJ1dJ zM<<_2KyFBcueJe+5zb)j#P6%;e>(W!%fS+8m3%plj8gVRYyc!!0E8S2p&Dr8 zb^!WAXgz~r;Ck;pAUCE!N*#<=a*~1=r|bW!)c@<}f$%h|CEQ6&4s@{1*%B;mmVU$Q z+E~-ob-I-slt2xR#j3L}NyVSU4PVlUV~E4nk|$HaYr#8n zEnjhVoKLKqtMih0-)2Y5I6bv!eZ6cd?rlv=6ioy_#^r_LWY3#3%)X2EWN?Z`J{AgH zppEzgsZ`}q>#4IZ@AMh1{=M^w>|un>*wi4E$XYruKdJcnF?T!X=hPMxAk6@kMQU*J z1y$wS4GaA=DYMYjLiaWgVL2b1yIMu{YBu_@l-bqj?yNn_E7E$7oB9#J+q<4AW|J8n zLV9%A36cBwHM1wOvEzBLH98Gwi!i;GWknDxMV@^kHdGQ zQNTojU{d40p{8Sn4Pm^w8@bG1nQzaSUebtBJpJUGSH;H)T6)+8ELZ6UPfzPVJoRBX4k0! zs}&Q6bcInwPsT);QW68$U6nh?Lvj)fd2Bb8boligCH<$4bXjs*_v)qN&KQV=nx$U^ zpS{ATt9>qxcL7)3N6a-rGlBh6u)cL8FzN3OdCE|~o2^s2k}bv8-W#o#(8i+$LKceG z?+kD%cc<)-$o$@O;fBNXaL*!;5>>esZ>-jcxhh){tmgazrs>)_n)(mRNCclxd2E5r zGFbs|)|XSV#kbGYnE-1&Igk!!bPi$!eqgW#PK`ziGY~pr--lom*_;}R-grel!^b?D zz|g}i;o~%Kq+iSl9+QC5Kc_#$1SpQcpH|<;JNtynbS5{*aAWk+BNgBSrlC1WA{@fZ zS|lgoR9t7RIy*#k*=b-Z`z}XEzBG%ym4*!Dglb%+oICR)Mt9I_gf6J59Pi3i}1`T$Jq*gcZanlDjV8$b3vr})#dG#{rzdG%VH;qJ}9 z5(2f}4Z(mZm6beb5vJIBqDi4UF}FE&4---|&lRh2cJS{eauBxJouf!A)1 z!djJxpoC1feK`j)*O+cxbKbVzq+fw$8|P@@NO#O4ifj_aCv*q1;XY#WB>Kjid&mY6 zGKb?4@JvyD7MshBIo-P+h|5@2>pRwjSxSXk)!@JBUp|rddj^>WE?$VplO`Yc=;#{$ z=rB5-o}7rEW|H<$x^?U!4$Om6*^8n<{ye`(EQ@bpY=_en)Je^8OLwH4l&-rvvFrRl zxo!nPI?Hu(yMntDA)F_{8coT-YP+tBFAGw6Ariff_r6QfK~u`l(h|Pr$h$iULJ7^B ztGWPtI+#}GU-CLK;7db_XX;VFWNF-4mMW~lTB9K$5-xgDWVS3DK3-15DUfFAw7Oh) zG=|K)x%I?~P|GBYZC z25{y7Xgc?QrvLwqSCop#Il_t}6gi)_N>ao`Qcg?AnQ}gDOU^mHopLrgCa0VUoAdds zoSE~PZDwpU+jpNIzJI{>!}fYTZ;!|IysrDD`TjqNe}sC5r`qts^`H5`0#xa^&xQa> z{)q13KI?QNhxgP#|II{RrOZDuEMd_MtxUh_1^oufx~;n{*KN{+MXw0GDsLV@?r{1f ziA?Q9u%CKU%E34^D@0O@)9M-7qw}6`RxPF#g>@*Xt@htL-lr7w7qin2Kt{*(9zmrD zY=->*>{30{3EV1eYvga(yLI%w86Zf3)~6Bk5q%y&ov!bhVgx)GKxY5r72K|Yrr=6w zoPFdaS=bP0u4@XtPSkK;>(#EH5;X&kT?7`*(Jia{ubDAgvSB}HN zB<)((x@t#;*m>*&zYpEUvx_)J#w^>8>@xezn0h=+D{nbdo2RC>q1)sHUl$oUj(RRt zw?{t2X~kOE4ZSyOHOUYA_-4(icX@~G`KN%CctvN%!xqmiBfvanH; z$cZ_YU9bMg7rKCcfAJ!+^72H?zJ>5TEswglx8;so0ooRJ;_A~=&J04>92IKKGpmlA zZO$kbqLe^qJ^tbHeJLaIm!;odth!2>L^FaaQpRsl1oc)SWM0^IBFQ4hS7DYd$d4BoD=LhRmSZ)a#(97(^PUzC36sE z=fY6${7N;|o;ayQP)Rln&3vL+D8U>0Aj4){R0DS| zVnotF#OsCCweXU7^uA{O^)@!=Hy{R}MKyQg3gX>=ly$ALwCv@O`xE@! zgC|t|5UsuXM5{Z+K_CZrCWeo;BF$fa#o^h!xFQN4HE_K8u=zGj_=2h4_Uh%hI?S6~ zsnv`X^pOzV1MHo9%@>kS%V{8m^8G&xAOqZPle7qar2=xf+L4^KYff~MQQ~=W-BT-Z z`?ZzWv6eVOGxGBMvy}r@k#GECpp&o4YmjZW%d0}$nqu>ueAqKVaA+rlE7pau6@aEJ zg9EXNrze8*^V90v`3t-n)Jkc}@#Y?I{m#A_QI(zTWNo5KUYv?+3}drhGG}lI5Q`OVO7$rDyo%5xggS7zf3Vp-aotY-EjAT z{yoO+pf&MZ>VKeOR8GoDoYu!Z=7-(3sSBWF)q^XX+V}5v@y(rte1qKg+z$YH&t0ou z@b0kinfFu@@)~Zmv-Big9HDmGX>9X>N0HoC14NDZ6`zi{ZclY1o$QUNwvCnD0aY0M z%#Jnzp$^ZjLerG$-oOt{V8olqUD7jrBtHs-&qDdmi5+}Ocv$PJi$POKsK9n3DQdzX z>i2sQnzBpk@It$uzy4>Hm8^l*U#X=KZogXbYkCVEWJV>qvgJS(Oz|zX#|0_PSJO5; zSLL%**Q{?=a^J$Jd0L(pUI>(RnBz0v=Iy)y&Z}|kz^X%L+B8!Jt7~&sc1wu;vW1Ti ze3aFH74cmU&BXDHe?g4Z=6?A1OxFmTh3xGAC1P~%luPyfl;mM2r^j9iZ#R`EMkEX* z2if2(mL0tQRpd0&EBi9?Zfv)o=F3wJ9^fNEwB?;%cpsP3i+~}vBw4LXPxV1>vaxl2 zIXD(L!LDWfj>E*|SHz3yae3{iEH?O37Vhv3x#8+n5F~Hpe*#>vv?L)%>rVmvpIJWW zo00bExb4x@U-nQAjSBL||4g5#JVlJ7j*1l(o}@colHuJFfvf$PD~$8fCK?@kakp={ z3$Gepx;V7*J{nDGGIejhzg;V5%#7|okhnT<8$7rza#=#U9YZ%zGX?ft`FqA$xv&zq z6i3p%>-B>}e}Cb&-xHc$t!fu&Q@!Hn^2p+4h*!>T8cXuY-?P0HpLl2&-E!(EFjlfl zS7Ty|;oUaG*7x*{N0H5v3-V{?0*$720~O|p2XM>0;=rY$J-T^4-WIxVK2m&B`B5tP z;mqxcOH;2ck&zc!+0<;xbVJP-07qI8{7jM(xefi^=sO;*HzI*~97ZCxncn z^F%b2q3c)Zu`PO>9**?J0&bPg5!I-Z9Cs`(1@zTW#E3&QyZxJv8pzhExz3DsA8$7b z6Xh{ricenkO(69A>Pb+55G5H-xd{k7C47o)er|YF55r7Rk5Z^uICAG5IQGu~vF!EO z;Yi9I80cs*4tWM%^q~o!0Dv{0yBmu%&Kkj_ZVYUBI4iKvZD53%{-P?v0L^f;84auP zxnH>u*IZ?NqAy%5Rir;iT!8kwj>F#IKY9sWpd`|Wh47;R&_g+p>K)mqhaL z11puMkV1Z$%`Z&^ux%}Du4^e&66K2ZE2;MylOMbq{G{o4U)EE~!-Wau+Ou2V$nKT( z(SSgGRUyEWLCj^lYl&ksEYazNYOn_O>`B=5wTD@Bj~&1$6rQLktU?Qkq!YwH*!GOD z5iLQQUBPspscu{GzgitHR=8@COc@iEHKe262arXn>3?!vnIVG67jc$OlV6i;tTQ?C66>4>6^zG47Ao44s!i_!w^-flCe$765E=u3-2R ziV@qrK;v&bw#~5sVGoKOC0wWd3+7N)%Vlz3_tgX{O_8j!*UXRd$Z5hm|p3tId;NTlUXi0}{mDKg~amHy>FN&?9EFukFc+UpOrJ1N_tl zBFZ()!*B`pdZM2D?@8mVYKF)^t$O~1Uxw7BC>zuKbAC}El!>q*J#9dj{f#f?rhK2< zMIU`uzlO%C#7Ksrb2%8WH}p2$FcZfGjyNA@ik+_9EDQ182Fs9_Z@u&?*5^@kkOnpm zh$r&(@5D2NZ%^SWAKy~T;BtRWzg34h@)QNh9X3$x$?sEY3gbe_b~f3AY@8Al!E&GL zhuW&oC<7ohpnmMqbi^g|@jbXbZJ~aFNiY3`5QnN==PJTzy(hhXykVd%_Oj6U*DDOp z#Jr}?T6;UrGrHrkip?uwrKYH}aQZwH4j%vZs28&o~QSNPHrcL1WI29$RyKq3$v>$>Z!@2srQ`->jtn zNKq8{R_n5wTT93Ezi^g=*CRIVx2e+b3jKa~t198hNGJf;n8_G=Njk7=YQcA5$gAbu z8v2;*a~2FILlC4?x#*wUq!W*5NVj4(oW7E2gNr}keB2`QeGP7V`|;y&j=BgQ2Z8dg zoy7Xfw{#}hyT@MHkA><8>HU!oEL6w6SauZ-W72wepYbi}Yy{$wr{k2F-t83Ndgoe{ zKvRgT>bjP0>4B-4+91N}ha@sDNtQOhx3R;D%#a+NRqy4bJet|R55q-S=Jt%g@dm!S z_i2ZugLEAVIIkb%y^`TnC3-z=dfE5aBj-VO>hqX;vqu~umS5eh!AAAKzRjOYS8izW zfYi42?W;NDps~zo;9VP9Da;TOI2|I0DZkE7Ye#GVhM)5VGTm00)n69Ntpe6D;bfTg z@_SBOAs=1EN0y8%J{(S*TapVKXS%6K6@=%r71?V{BJO@8RA=Hf&Ww84{`&@}hk&`2 z$(QhkPDg}Mvsd!qmHDh@75+v@QCXG;zSm{{*{hL z$W=cd@wY$e@B}W{j1pOy7zCekimu3>{9Dn9n4n*iZfg?8i#tOrk%fHyNb56k1|`kU z@jKWRnliujpcNj#bf2^Bfi_kZz_|Y#N}2uOjE<*T(<;BUV0F-X@MGl5)f2Zx>UZ!0 zRt6S8tK@|ow!8U-l4)g8=o8fHgZjXg)xeU2iz(*huWlqobKzIb(U6X=90mL6XkS7K z{V*%zw$#^jjpIRyDrijZKDU_KJLoJIlN0(d8)bGp8QT!Z8{ZdwYvUJo#T;Hy%4(zhcqlP~JgNvJioA8sW6xx!KaVORbQp(?GV=Mdc z#hUOprJ}&(g6V=41oEH)c1oY$Mc58fbQE_#()#Tzam=VQqb9IQnwR4y=IQEu=T!cS z>l#<#gAfYKz`y1x7hBy|vW68QiLw@E=i9i%kbEquzq;%7+R)H#dd0DKSF6WwZ$yEN z0X4e_al>7Kb(9N;#Y$JCPeCkt$kQSWNhrzoKLpt!i0U$^uA+LO6>>JFUGiWY;N(04 z`ks6Cf%YQ_@lxL~pSP>l&O~GZevf~5#Oeg_i(Wg?kY7esseqfix`Ib6 zyuF^{Aw`>VhWc9Lr7cgU{#u#)Eqf7mEy<-uVEoA*k<D{d=F981r2;C=xD*s&5<~rL@5JVfs$sgpBJebU7N)$?@mX?g3fVEXL(soIf-kVR%|Fs z=GNcF-n4U_alaMjVY(buSQ)rH*Kod8ADfp~&tq<~7A!gd&2Rtl?#08PAFqfBE3dI0 zu4YsNZ_y$n(+x{2D;XH^G~vOw?}~Y;|~1xu=blxx!Ord4xUyaGEC*SWf72Yfy8;jriz z*+6_tAn#%5&gwhnHSoIn?NTa_QTTXZBprt1Z%T8){^0K^PS&{fSd-kI0`EW^!i_y2 zKw7r$>(R>Nf+hY;CY)f=B0~@NLEaYq$uxV?FFuG-{L=GLc0jy3fTypgU zPWpjSagTVR>*IS?tCZz+;I|GcpryD`c;Kr}UIaTh(Kwt{$R^?jC9a}nJKAH|i>*X@ zwSJSy{H!w)eH*@$+!GC{NUCunRq%yrCDaY6hi~^q4%X=Z(HG~Yz!sC-+~pDO80B3F zL!7D)XC{L9;d#q50g%#ZmQUb{rqaw6FFigfoxwawvy+9hRuqkdUOC)P1`%V99WE{B z6C1+@dYT)}??fgd+TbTQE2t>=0gd}U>-3u-?+xz`CJ?R;o5t{t=Fm2422VyEQWMW%N9ZW9cp$ zqVZyE)kUieV+uy=CtLU7uaQsAcvzCMjw(#BYyn5(Z&0ZZS>0)l~YsAP)Vys z(i?B}%A1}Myya7hvnpNR(u8(P(Y99fZfD40_-DZoMY9iDyj4v$ie?y_Q&TiL3?U?9=nXf|Wk2I+J$>m-f`Tvv|N2H+Gj}?#v z1Z~C3Z3acV^`Mk*sUfbXs-BUSHjWUkYAsWmcRvUnxlgsrhEJNbEIDA{1L5ymXTFm0 zQ!VRi5d63?;C5v%MLJy?;YCTuhLcj}us?Of`w-XAl*uPwT({rqiK>0?6V0?&{TpOl zquF_ny*hqxqg?u&0M_{DF*CwQ(dU^?kEPqsA0`d)p?!7OG%nL(MvdPW@V6AcNt09u z8x6Ff4u(N$@*fT?ZTTT;Rdw#Z9q1b~GsxN`{&(W91g}J}W;lnVxy1u_#v^ybj=DAr zfVa5xkA?1`Q!i(^a3ulXnx>fw)V)is^Ya=_H;vl%bniOXaAL=P@^j{sd z;?KlSGh1TEO%4_t2bkV|h>8X4u4Qy<1Na!)j%AzpDU4Wg?m@U=e<{YG z+QU$Y^3u{0KI{ovE#|X1DD3S91jN@WBCj zl~+xu$>=X!Lr-mbZU=1ub96~Lv`E8L=t_sp)4>nR-^!J=7gni8P zXIWGa^6kv#wddC<*R`7)pO2IEpLM4I$`W;D3XU%(6mvgR%?*Le3u7A;x+Tv$KS=5N zYu0k?0*JITFeSde^hcJn*9mnZ2|ubbLD}?_XMIms6eB*0|F4ILI(4HeKl)lgPXd~( zWZ!DsH9{y}ids^noH==mP;ZK=mVbd}f7OlmRzPEy`wb`ugIhm7&y*R* z;xxa38XoPd(vClo(_Nb^ixq*e55^QvH{54Otalboz5rKO+wT zkc&gJ#!VuBh)&>5ze>~=q(EyZ8)wzgbQZv)a&2h*$A$vrhQ zrO7`}aB&N{_L9|@y|AaRmgBdLd}P+Jao;tN-4sqSJ@}>d!yKT#WX6ofkC%sxMMv+p z5VmM(b4o0Zo{L^XL`C$5z{((UvzWF;6e+i_}xVgyuHqT zU6nh1DuZ=TfeLysZQ8`SHxld$I~oZFOIPWO3VRFlhaeSQBgw2#=O+D`f2$|ib07=O zH~(~WZ!UwnDgVL8t(QV>GdT;+!=HOPhS8LuBg_pmuZYd=8$kg8U$j}d->?$A%q$K)0XVYa<=V)Lf4L~q>=j19x#TLfKG4QRwM7p_dgwV(gc(Bz~CSj}earMc7(>M9D+po;d z4gT0>$NonDZK}!uV`_P;Cw30**D?kprW|DNzci~4Zik%k1QKSkF5NomMI2jSUxu@c zZydfOHAJ@mn0n{Gi> zu?|}SOXwpd##(mGONg*o_OP({T^ZJQ#o7s;f@fvj^Wy^F|4zSrBI55zHrucWimiZ& zP|FSEA5rAUb8-B;2gY7DkUvNMzO6obtftH0qxj+*v0B0J2Ig{e{?y?Gv@GDv0)w4M z2IN4u3rTvsCw1r&^3}AfR$3SUtogaWxvPM8^iv#zC-{U<2lx zb-16xX56wL&+trevK@^YN8?=E$R`rxs!dna3hR&GB0gdZ!AoYabB&~t&FiS4mRq+) zT3$>pz^7Ls#AIj_?P2L^BpQy(qiZ!AH_vw`ElGx}7#^5TR(^lBcRvCRp~RtR^Qe>Y zaWTe>pjSUn^(E;aK&j8k_UjhSJ}+1SiguaUpNy_kpnH6l-_Po5Jf&N^7*{vrdjG2 z^YV+U05J1ow=_yyFWp7J<>{Kx!pRcV2er%~?ce30e0PqqAkj+5cuA<^Sr=h)G8|2d z1VdL?!`B+l*(6%BWBsM{g- zXZ_yWm1}%|m8=t3rx(okn>g{gEM+Fz$T$BxxyluacHAGNAGQ>G1 zrA{?bJ&jf+sT;5J{+;i+usy>rHHyNX`A*OQC(te#V=mf}aqRh7VQ4?{dY<5TIH)oIxs7f?6aN#Mfra*K5CUny0U{=}v4r7$>^Z@?> z^{oW-S(0qefxWDvw`_ZI83P-ZAYYD-?E;bhRXRdAr3@tA3U(rgb%SZ)qe<`$%%sDn zv^|AmfHN@cIqc_d_}0jO9T1w$jec2P)CtUyZnCc0=B=P^)88Ke{O-#BToa}+ZXWip z8u1s#AhgOStNQN=jgrt)p}$0cAmE%IBK142u}7m*OB%XOl&hUK^}UVH^?iV-!$jex zZC(_*$RZ7E55+2-SH66I`M6V3y6x>KUU&-Zmi(-V5NV_1DcR!FL} zzzEwL%tWTFto^4Y*MI9IL24-H(>{5s*`i~pq0}fk zzSlRYXHBzON%!X#g=op{ba z_kPvmA5Ar?-%lSgX}FdP9D+I4)pRT#{624cb5|d$5rJ?HACZ8CRDe`j0-~?yUb$H~ zgVQPy{4Y<3Y1Ff-y3FpLuIORSNKf$*gf_FgtIxDK7P=n;C#sd@ZLz-PkAkdDv!W+I zOpkqmEc7FeFPkLgJ(D+o*K!&=R~|qM0QwIF0Omd1mbFbvj2^fmBA#cgR3I7`m&%qk z7iCz953yntKIcChy(OHlGJu?c8dm{JS4!ew_gvHNEWdrCDdF8I!?t6zc@2)uK#{9K zXTLUqXEeWkX(3hnF9nId{!?&ULiwDlUU+6LI{bK#4`1)Rgfc)J=}m}JUr? zR!!&V=QWIQgYnYKvjfom&l$iY6Y^_?iQydDlfaoIXj}oj6Yp;|h$ys$5ZDkG={H%C zwD7n8Fce){7(FC$k)NjBW3nF)?EhZ2JtF?%O?*W}+ip9tps$lwRs!UtW^(IQ4`X|3 z?^j;Irr%={&v92`@vKKBG>kHP4qm8=x~rmu&{vrirg` z#Xnv^qttv`X<>-G%o0a(WVi;bZ0JXDulpBLN+il6EbUzIh|0SH*`$kh+e_R944cX7 z>4dN0mbfn%#Ik%+!4uv&l2{S02t^I{F%;u=aK1CNv>WAQ=0-iMA3!g^%c4PbKa)_D z+6eio(qv2o?J|f2p)O5n);-)z)fvOH3DKePKV`_4x$Rv z3XiU6%LDE4eXLGDb!G9tggT|Gy_bL;pzaIJ%XuT-cL86kHS(59s2XgM(L+opzXkgH&k*BnaDOp7xomO^xa|G>s!*GYy6Wayc@ua;tj;4i2 z57c18Z&E!78Nk*p0Xwx4X^mAaa0vy1Biuk9&3@9k8#B?|603{IMqmB-+BO03bz+6r ze?#bZn9Cd~{_6V4p0kUE@3x;c?H#(7dbFvoVtjRg=ub1Nn44b0s^v(_axswb zK`e>!k?rQj$#jZkYF$wU+RC+w2W8c#>oPXvG){coFZ(ii@P6V<*aClZ=u+OJ^tS*; zs@gw0-B(u^JbKDo+LCRReYi^mS|8QvH1EGX(bS$x6(872ep7l+8A5{gr=KYQtZmcg zLM9f!Q6%wH4@$kXvu%CCB=tq!E<8MH_Q!O3OaD|u@xRh%O4%NlG_E3!RkTb!@KKOg z7t!$#-3+DIRde$d!gHSGY|fXd*-$>%`F;q4fR12T1Eo=V@@f|($s|$(wWk9 z=d58atf@$012dYNbWdP#d5-7B-`wowM>mh~@oxvN(t$&pvbju5FEK%eA_(c9y7_&& zvt4n5_0w_cN29y2^^`?IA?UOKO+3R3`LIKyaSo?7cEG0S67IS>sg{+#&*nE7I?Yxg zqThgWbSr8C=`o9DGCbdT=e*pc*q+7P=lXutDyI9cX+ByS>)P6MxF4ep_a+xQ+SbU- z_qm`dV4ID#E=ltuo0pCfPFy=>rD9OzY`ZfEOKN#jbM_=qIf>>O2vT`fh-Lb#eN#ah zrnDSa$CpwT<0Sburk);@xvoW3XdOq#^(ouUlmd?ic(Uom0uq_%M{Z8o0a~0SOfaR> zq3E?lS+8?z{>qPFhb+^y=PriMG z<1{@fo`6Ej(v&#J^$WB(uFTASgdEOHFNXcr_2GKOXUErPanC+NvtUjq7(&;ALf$lKg03YTPD&WX(&~ z+;Y}sugHbUbI-$M7#(zBN6-lL>vWg8y)?Y35EW@Um004kJN z5Ng(Fg7iZR62l6*%UxP+zVwzVj3Zt`jD2PjSzc}>TFLbO;%yMtV^YkSCQQV7iG_B9 z$k#*9dg=6=B~}Jg_}^9Mwvpb^a85l$)y54JQO@rALI&ukGhfUv+k2-60bA)luscU05?v*d zO+FYhJ<-j~qTZ?!HtW#p#)DeXMyb4Z7!uL1Ca;F=6iD?U^t-C)g!HGTVWs>!`4>5k z|G}SJf95;Ds|;hJ`4A_l#ZYlUiX8rS@}1HWdoealVhrrUvEuKGz#bEv+F?f|4{GQ>4#25H znb7|9I~>s21&Bgr*ORlAEH$~htyJs|mPdAht4`VGq}#PqNE!k<^$+f~aHo~vhVRfk z0eSA5Rn*ViL+t|{Ha$@5PJ>#OyvDL}AZ)Dx`dAZv@9t&_qhc0choTfKngnWAS9<#V zYIzIpj{fRX`CQFM!^)e31Le)BQRvT9+OJIuF9HM_q4P-Pzp~`;6X4*9df@bet!O`|F)tGEkEhkcAi&~pF>pBC3Trr z`C4l+h-C-Xq!CxC22ZoixxxR4H^(W5FL24Jwd-7B|Lm^fQ%OhhfB7NuvXL){l z@iwHv^H-iMH~!=P<)t5*tH(d6w`Hg<3$R_iDaa|Tzhvyuhjl&pk=p#;&vTUn_H7lT zj|RQi|9i3Ntq}#@17r0@j%*+d@NUMg)L>qK#Mn0uD6_$@3)(%CqvDdF%O$oe%VoaN6sX}Gz*oFdPM>N4K!o` z=Ff2!hKT)#QFUK?=n}OZ18AfIkQ6Oi4}wYv<+X^GEE)L%$E|~@#qcw?3cnMzd!%oh z>NLI)m{5r*;fI`RSjm~@U8%Yls9>6L>u6m8ofY3Fco8Q?b`Ez-F5SCC;iGE3wptT- zR1r4Q^;qM`$6gIC@wcEpuWGv!WWBP1ut0oup>hVERg*?q5wQbLjgD)3-?_J=!ki>3 zKMli2MTFEQ5&rq0Uz@gA@TZt^7GiY_glXb(@hl5fW+kOfr|)rvI<60Vb_ybl48?q8 z`1F_Sb&37xKRXSKmJwBM9L>%`4t%_tLX@DDF851)B;34EP;#B)#GZit(V-R10CTDr zFr}#=iDwX61-kic1N`;kjmI{YJl?yrKB#@Lt)Nh&t#*1J{m+eAcs^ob)ck zP||tfazY|`)@vi%#A^XYm({2p%RiiG#5#J=OM3N$F$&!e8Uk5^1Q|G`)8PvDCwvQ8 z#m+Y;P~QLP&Lg}(d(hYxB@H&AjxnXdES_zYWEE$68zW*~ zjlM_86o*t-TtpR?-n;tb4l-Xd`1pw);fIohS=%RU$F+8)my(-}t-X58uB_APLLZ6R zrG;_#4&xeriU66G)Z)02CdaRJ^=-TR8U`i%Xe;2OIACbSJ%gd~pUbDrVz5Mk;{vgq z-y)Q4{X&)x@lZR$=T<)0-tOfh!uf(zFdb-qk5?b-D5@#UUjjq$?{!gztjkRI7QhXh@|CfBFDvP!{ z6v8MZ5Du(e_MAg#n?{O({9#1p-#oROsXsy-(*HFL|26CnD3d)P`d)RFJ2|Z=^^T`- zIM)MF>d4EHMhB>V4$-BiAXe>ma|f}|q=i_Sl8v~P)hWHX^DQWSMu{EP*oj${QlsVe zAi$#?jj|xv1Ngq~R|isBFsB&IsZd7X2r#&!-kCjzWt_Qd(O2? zeZ8b&x;8*5I9EoFUs>9hr~6S>F8xqfN>(`}a@PdvL@YAawiVn?oGAahA`^NW1h$9myYH`h7 zb>%MonAH|HVylS;&4Dk(0w(pp75iTA9ZCU8Za?-A5{lS=oEPqewv%#j1YD&Cf z+u^`Y2j;!w3nfY11@=tGZr_jayX{)>ARRA3t z!BkMmA>+={9ikoQ*x``$iq-pKgk-&t!+&k-^_f2_q`V3oSK8L;1m43Q)YNn`%St5W zJIL=W?u;T`aWHS(QRDP&Lem4eqgRFtie}cZGCVZ;3K`H%z*GF5o?n}{d(BnSX!DJl zZRI45Cnn#ILXq>6g|9Q-3sbyr-+Zfn_ncJHf@z1~<_)MDG_h>DuI~7t@r2hBzXLxZqs-@OZ$j-2DOLm* z^ogc4Z4pp)OM&#ycBmEpfcej0j;Ed1@Z9*lsh(~h>4sJ=e4sx z%PIY@f+C7i4H8yGt9cJN{nxM_bV?6|1n(1&aPz z(~1mw4WBjh`(Flmo+5YKlKh^<+2Ge7UfP%mFoXCny_>gUTElXpS=Ox@=TzZ;w{#`{ zAaSihtd)sx@UZ-SKcbjtrvWx8rx+EP*g{cw^N4s{Wu;-G_+NBJumZmp+O%=&N?Z#q zyM^ial=qq^t9WTMiGU$Z0*Iol_G0eD33j2qx3T3dBp;G-O3miA-aJVYApY=XIiV>a zd?@VGY%B`k{@8Hbj15HWrGIcbTTRH<9;3;o@f$N(t>3hD$SmzY9-3kaTRt!X>$G&E zQCs?qB3z4ZY~}C$7{j_s#P0c+1^8l+yNhR)8a}|^CqYak{;v}wz_SD~OJ7`~kPnU% z&~#d3bm-?+c)u!ifRX90S&y(N{Cs>J6Pw^uZ1u24@Wy*yAPN__uLgqc(k=czn^3xA zY5NKowRc!#;k{h2IWaz*s5aJPCgRu?S`?B5B;)0)4aVR!B(y$|ay;kOa5ijqV1aJI z@AYI-JBuui|JrFpLN5AT;XHi?kk!H?{MD-9tE+%!-0)4zOaLvIZIL!Q**%ukuca` zX1U=O*W7y3El@&i3=liKH_eHux(Ls8wWnI~`0 zxi$KXWS!u6jYpJEYnY`}Xy5FBxX&p>Cw>iB+@Xogl>`wIV~)W42*`nnAn;R2rC}PA z)uhkwFA}GY>1o84?JzUf2J-jQlRc>Yi00ECzRk>GR0;C9FAm<{r8=1dPZF!Dkq)Vy z*yvvH*xHv`w15#*!_|0>OVZpskb z>*x~IrnU%LKZ9BIBRQ0u-4N=}tIT5T0O*=3RSfj>`=6V3juYmy0Z|o+q$3M9fu$zU z)Rfs@He8dR4L%ryq;AB~%RlJdYTIjxQUSB86~42!yy{eS%B$xHi#&MuRZTvx(_^A- zIdb>-6!Ql+tmf8y(n<|3%Je?m-ps~z7Fh-^7Hqyb7C+;=#CC}PZ9p+H#M;H8Jw_IT zPgz$UH0dw;QA0P&_ttoia@j-*I6xUU5%gL1hc@e>AlNQDKvlfvkS(N6QL(~rvj^za zv465qYj+~vlui6QAQcz4PQHhaB25rwUu_?nH@)13&}p99Ag>N4SO51Ti%chb9p^^^ zSqee>#gKq5PDG$Mqp;rqB8~&aQj4vHs2ugzVwz!iEPH1uAtSz~iX6SE!A<54aolxT zc|pxDCy!R5HG0cMzdY_GrV|S)?ydV#NiWnBw$pF`ClZ_Pg))M{qW+5a$^gN46EeQ= zZZn(O3RS0+c-dbCWD|UcP|kSXyx*P?=rGWlU3L15?;0qfwp3VFab12t>*?W(^Y5g# z$C6&O?rbr|WK*E{74b_% z3Ww7l0K2`|h124oMCO6-IzKI35_XD{ETO@R(GSZ zz68=h0T@Aman>1a#6WYatH zbGygyxM3|OnKL1E?A8=)+tnN2mU*2-AsjhTq~iNV-p^CLY5e!M_Yb4+*1$UQ7XTq% zHt{PXkBiO)1<~j7j>#?#xen~G6i9(-880YR)|QeAZy|C8$V9SeDVCeckoKos8&25Q zI7vGXw&DG8t8ocm-@oSjhoi>yNZ%KoqUh?+5U=+t^;|{qH}>Sxjk<=1PY?g4O6B~P z%Qh&xpIcj~h&!ZhL3-jwZ-AzE!oNE1;T(us>6fl=yVF!blT#ON#lo-t8_n&Bc#r^& z!2=SdgK)&W!w1wWj^yq>USJd;3fu!&hu}Du>_WILF7Ym7MMdwvh_bGdo;C7sr^IyG zH3$0hc1XwHGnnpLuehI}zppxKg~NQAM;(1zmp)vEgu93xx_y~iH?vo!C6uU|t*p@i zhrGlC2uXRVS6+OVfm*2*)W^*4a^1SiKeCesKAaNP(8(YAX;Yc33L1A^1@f4H!Tp+Q(H4zr`sQOxX zI@eem#`@$AFGsQttT7~HLVD{ru>``d#i!-DSS2BD>twphLr4<8F!aR9-TYMw(i%Mr znF=U2-pcFYUQjHXmWcm&_wKp83zTTK=vPZ7{yRU-I2G2u`Vpje#gJH1_xhYo{+&yA z#*FI1JO1q&@B00F*Q`1z(H!(@o;8^ z=%B`Lv4P`PzueQ-2U41FmFIb{9TeZM5o|JlUeH&6_BM(3DI!tYsArgR5QUJNAl<%# zv!*>|J^o%|aNJ(f1o?bgV7&V}44_t+c)k<>Ogia^Mr^{VtA)vPSDB9n+JpT6RrdyL z?Myb|bC%C;)|ErJKy95Vj<+{-``ij6xvC%k8EjgD9IujWfA0Yxc!)}qZH|tj1VAV+T=)V?0G12HKDY3HncI#T{Cgf3=0W`B7$J^#m#9HxFIrGq+bi(nYJ$h;O%m@qJFL zZuAoL(BJ@y44i=!z0ngpl-9gdJqE#`JY zY)bfLN$YL(IQ2@FHXgW8^c}K{&Hj6Ka^z1xiZb6K%(8F%rPW1!e_*Mxvv?vaSCs`P zzZBvVQrJ!VD5mh^?)mpA;&*i0-s7y96oLX{BFA~owK1lQTzw%3py6=VuS*KL)Y1GP zId9u}ud&4NYLCpjvuI*xE4BVe#q^UWnb;`TJv20%aW`=faW3=;w}j|CaZ90v>EvAV z+Cca>^%SBe*dC}X zL9L10)^XSN6t>ha`rk)Am$XO-4*mkzgdc}9g8*__oZh`B7X$Lj**<#{y6KM-=1^{3 zI5N@a7Q=gS2?piCd@J` zZ`|gCEod>UiS3q1zE8j+V?z|BAWxc0n2p%J8Hj-$D`%H%!iRedF7W(adgYOK$4nO; z?VwbmTw<5<$@gIfr%51|^YA!WWL{G88ih|4%Jl$QeqUkK@u_W1%Ig4-ELyZREb;n zGty^;+LCR?Pdjf`=A3M0o3WoB+!zxu%b zef*$rla{S%Z^p`|{t;0iHS{H zw~NL9ANg^I5-P}KaU{Tq;FYJSf=8I*_q-?P_iZYuuuZg1i)$j?Iv0{>J!G za|}OLke8FoY=z$y_WVWQhULrBcO(-Ejasx|paX#TF-5FHXXf7BY029&_U+C)d^jPo zd{YLHW=%_$$t9XnDst>um4jnx1EKmbWZK~&Y{RcqBEE6Z2ZaD-CDr}0B8 z)(s4uP@uUz7p6C_Ta~##qfzr#O@ELA0##rh(f*W7pScjosqezEwLeHyhwp@m=mIOP7TY&T_l$EbWnE&Sp~+hd^7 zWpTrBV&@+1O)MXVIH(B~AW~-(IPTG&NSx?9bo5BUhc8e0^1a!1b^r^G>t%kC^o(8m zKOHq=$?T=~ymDKc4$ZmFhRJ}p050MOsRen*CXM`J>U* z&RRC?m0Me$+Qj&QC5}p*pV0n2nYr1yf@lk+DVx)FZr}atoo{bhn=F}j7RO}DL$u1t zIXvdkiCfkt{r&0dOK>}Flz*7yv~@iH%N8sjIrP25CZE1?qyse8fSsG2KYha7d7m$S z?xP1=oYGWhNKMnXU;gy3rzYiO6#+B$JY28(S^Mb-LMhw_!`w3-@@k5oS>aD28X6D{(9*f_l!MSpgSGp0n1NT(^E2< zw`+pzg;Rcf@9D{LgsT;OU5@oq2`zv8dMSkW9@OKm=WqOD{_fj*Aorl&A0Oi;W9-&7Nm2C3aTF>b{6;_W!PeC&w?BJb35Fo*j?dqu4av{k_y(y#W5>jVAtvAX z=Suz?4~(x|r&{Op+OPU~Lo`sN1_}cFfi+PI_%4jBTC%?DMI8wd#Y>3&VZ?h$%7agx zIX(Q=BmdsAI{CI|uj8#sX$pgu5*q9%wer92e;c-{7n&)V z=3mJ~NYeV0>UwR5rphYP%$_TnTKpUb!Rprr0POrA0{@kL%D^#o0EyOxl)Lb9_w(t zd{t3GeSq&o^4)J@*g$LFDO(tS6h0bc00o1F0AUi!qDoUMe>#!+lAp9kQrWyIllgW? zJubVF@VlV@)f-o8RHgct1+#OrvzVufO_T2^bIfgddh1L+)X|(HK!jw!H|Rd_xEX42h%7J z&!O<$Pruk_;7ul2;^BaDB5ZH!&i6;XxN-H0lua8rsJ1$Nl&es&dCN9EF1_lCTmKF} z%2H%Aky2L4LkbK~f>b1mBi2Nr6*7LPT%>4#=1Vt{1RQBoBE%<6#Q#}6dv!SToE1xc z@L0}1kU9JFkNRJKtKpGk+6_6UsGs#sC|{vzt2RCQ_P^q$yL1z_at+6b zpU=mQdFPo&9rpQ`4|sCS1Pay3<%^ntABcPHp?kmn##Pr}Tf#UmwJU zLh*B)e2YkzFNoB~=P&xCG{&bj)TR}R(761iFkU`C{llv(mdyPodFy7HNf7_d^0H;i z_Pl)HEss21t$NKs1NKJ8hrGkNS9d+>P^K;qR!wh{_PC@T@Z z^U3Eg8FaIVN4@4M%G&=be~WCDRg6D;YKTXw_y>4^U+N0E(v|w*s27R<D_zA_|Y`OD2A4l&1>Hu`NF1^%hOU*tdJ}Xl`K=b zd8>9^&%fxpdmk=crnJldAM^LP`~n_}0s#x4TAx~R(##}<9*X=F3s1A@trlh-&3LIN z_R_(|Zivt{KIkoRoM)CjdKaHq5>2#>sUVDPQ)ecvA5~J#^r&<9lgn35%*|#$XnZM9 z!i&E6>4y8SvL;BBd|gK+xC$fRe*S~{3PP}7>vr+!Ix(bXlm_x8g=v=0beOtD@l&{c z{m$2hzLT|J;xfS6+4ZWhdlk>-rQ9oRT>p zmneElh~YtjuPbs4o1C&^Q|kR!{Np+V-gT+cg$jEtG6162dE$oLV9t{`viyF%Q1aL|47Eqo>NliY5-%I&jM4 z;Nh<)%p38>?G$q}PTKbLQTM%f%pO4~&W{KO65=J!@Ji~O4;SzQqCNU{)8j(M2>i4E z1sfm7@uMsfH=LO%EGT&8uCaD(1|H)dq1YRVoPGJ3!mMF2QaW5><&t%;-2JxQl_2me zMt6Y!IQfy#_tTfGTe{)JsZZ9aUmXPE$9Pd|W6}AsnYy$d7ysr4LAJKeruQ~`#lI0L zI}NPl1DOqve`S!3ebgGT(Dv2I+_G_`fakqMn?;3KqF(1;B9XLhdq$0T55o~6tkS2Z z)>84Krp^z_SU(n-_2I&HXSD9lJuJoX(@EhF<0pCb&PT6!Y5(2>G5jvy$LE-TXMVh} zbYhuXpBgM(7C)E+qMYoUmu?@Ek(v?3a~yseo3SoCl)5Fg^SSNvbMvBh7%yrh{IUFX zBdt&Pa1Mvfq@B#8$?6~1y?W0(aFz|X+L?d~>y zT1EW$li-FU zzkj!8>|-Br6-QnEYX#awc)jzKvIrQ!QyAR4{ z6Co6~(_}lAlBPVeXvWX$e#7;TSFc~~uk+XWK~xcb;$dx)>5zPO&EQR}l31l$C2&iT z46`Sy(90V}ElG~sKVrI2nuY9@#FReu*qvJ~hv;g7$4_*8KsZbW%t%1gzi8m1!T(|Z z+n`xJ#IT?bIO0DNPF}mC<5}%Jf5|#db6}(p;zYwu`&jX$yA<;U%RWezRO%^nWY z6*oR;6}9qYz=qsy3&x2V5|v8U6cVhv-CUAp$E23VuGPv8U!+PBwcAqSTPIa{MZA1j zEs=p7ZTJ8f_Zf_#YGuQo|7V7B1AYrR$aQQ~JaU$0k&9nn7!wCHqvFc;|ajH%`a+Hv(#E~I& zKD4Sp72VR$1|!{~ufL?+%0fLaAHZslORq$z|3)Ln-iyAOE{YJQvr|ZLv9Ktp#U%W|`5_?}-8Y{Fsl$n*EbC_E^3POcjR;|d*<+7sAbXX;g)hRzO zhtnT12*>j>FcQhk+;=4JFx@0ve2Aa#zMgtRkIoCe znsPM%h*VK3JESKqb;r1upL_4cr=5RJpV8GM$-()Da^84!*n`*hTeWn_p+okRni5;4 z6&yXb^!xAbA9Tgb!|z~FC9Grzezy4eVa|-3F7EdAryp!uv*JkpQQB*f{rL3Iy5E<+ z_4K3n!QKwe^$Kc~AN3YuFn{&MzTN4>ydqtMk*JyoxctM1vS&~GSaBq#1LZ(CE`L3Kgv$7T>#0Wub!-3e zJFjz}j_0osqfx*@`T6;?KA(6)_YRy>^Y|ypRnJE_Lb`ELUEYmEI1z^A`22--=rQM) z*tct!2DnJ!!2_AO+|^Zl{DBQq2qE+_`K0aUHLH&Bq-_3S3t$}@P=}PmIV5{dzS8}0 zUT*G|HN4PUL^8fyRX_hQ>?(qMgIOlq zpmzJVjT(*%3*UVDk-_Jj`pLU*aHazs7KVnFjR;rokIejJ;-Irn`F{3v;pgl-@o8Yt zNPO-~-{rLJOHWVPyyf`C*S`6@v{j$a1JvjJ^m+re8w*Wwog+%vXUrmCjbHAT8V|UsP z@JD5ad4w1TO?>m^J1_5%nzWrNAD^v-DEp83D;uRv@ngOcGolD4?2~%rNHPn6Z8{bV zqEr+x5@;VF_c7vfUIM4cU$oRzQSvqMkmr>MJe+O&U)zmf9{sF64HYazy@p6DjK+`D zMmUwzwfE`Nh)#mm@@wy|{i}c80Dl@cxYtY*5KSu9{JdfJj(td>rBIu^^ZK1BLHZxa z*O!ciW*O`F0m>Hl=RI}vt4W(lbNSayM+}`&&$e|n$u6H&trSKky*@Q{>n`}wCCxZ~ z^7D^Ad-I!|I-#NA5!ptSZ1zQVXg_N{;R9s_g_A~3*|8E{A&oyqk@KSp;}rRlhZ{h#%C5VYc z5-OGzS=2_t%!OaS_g@G0a=BH`&HHR&ffS zTuv5I_*eKTWJECeW)@?U>5`oSylqV~FJBNsE1|Jw4dRZ)z?xa4?!1o|XJ_S>C(udWB2ct>rFGgjvGJCwXRJX z(X}sq|KwMu!lI0h#{Y@(S0kzaf>Z)G zIdwdc~zW-4$#O;5@xWJh_!%Mxzqr4iGj~%o?I3ty`Xv1Lfq45DM z9#meeWHqwc&BsPIi~MNE=q}3kRccE#*!c%zQE+s~Z>>CFT)xpM9r6r2Kjy<=_F0j@ z$>;QyP1=$@av>#hi+WovD?xB`CYg6+YtKPC<7>K?;6+5DiZV;Bz%m)H<SMIak+^0aGA{#FT^5rv$u`y55Q6WUAipqyCEd&p1S=yei=oL6{?O^AQ2#N>K<9S>biOD?;1iR zI>0YAF@7WzZ}{yS8g63zs6tRpxkOqfN^pQ5aZ(g6Y~T5`)4E?^I`y&g*Pj8UT4zwR z2NNdaBu!sYes*l%HuAxt`T2InCyKvBsZs(PZM=JDGMh2{^anpL3c34=3ua830Q#7- z)J3ZJQhPpUem-gW;)S$e)0uhm@%x^-eMoNBftY-HI6&#-zb&wLbFVI2*6_=70YBY( z^`(nflQ!Rd^MyEdnqu;8^T%&z7UXd$Lkx1g@wD#e#p8eWo?BmkblB0uy29^^RItW> zq?-iZesWvyZaiRBTz=3&Wb95&dGhw_^9~=1$4`Ai`NSgsy5OjPkBd(-`j7a5z)<9q zFB^XT@A)fRiuiHnC=A2O4W%CF~h>9?QP6K6662875-iE)1s;4-wOOA zPsgIqDRdd(L)iykduUirPL_PFyf%V_LTF8Ljjb@ljnbK?2e_$KrW@K!0c>KJex4!v~uEw4R%uieHH zq8`3PHm_R#w_e?LZr>WiFaH0V{FMM66OrqUPp6O#Nw?G%aTw$Kce|xkeJpg9 zLiEJbOz>`rts70F;vg+ZYRke{`1A+YRm!2WE zyt6u~*TQK_D3p9nH3>G!pJ%sq2nrke2-U1t_2e^Jqmykf@fu;2EmA}sN(DF)nfBhC z{W~&@10k_L(IAqT1D8aL3(z+UrNTwug2?;NeF9{Jh%)1c-*%by-mHD8`y9Wp(^$+| zCk}CVnV$>}A3pAAzTKN4xk3J#&>%k5-({;bq9Z$@7XO7um?H|IwoNY$c`Bs&NsT+4 z)3$k=hNVmLbBYFBxP_XH1N*WTe7S@T{OkPF)%h{wOpxK1?V-rB@77Iu`x`Hn3}3>L z_FY;KyUB`|&$`OgD{!QmslEN@%hiKSms|2U z3@bdsZP7Gf3FNa@zGC_2Z5y9*PTQt!8k8#KSC`d--X=f9NrCUaSVBpB{<5h+MH+ub ziR5aNFRg(~7HM0bQMIb$wZXF{ejAl9et1{QviZx^+?u)=UMqTT-!6Z9W-yoNFS)U2 zoyN748Is5Q{h9qmYB#K*_9kCFSxmb_K$zr7xr*!6uK@^5V2qpkuUWhk#9OPGO-D=c!*=x}F=edDLqY0U&j6>4SG($6A7JF_r8L>>iCc{M*)XUDMD=o-L73-IjHN_=%3snk6E_z!Z~DnPU)6 zaU|u!Wc;vy#K2&DF!Ax@gYa8hHF0Z0+^99*<7fK#ImFM5akE(=erAlD^UcHsJco#V zHXIh;|JHBHlK}xvQzc?j*6a`}>p-C;m-5S2itpc@A5{Xz@yG3dJ;nq4!mH>IepaLM zrC&s-)YnCbSkb~62@MLicu(3;A_-=`r@2iHqI5*hKe#-XiDBMeGR3x(s)2>$U&u$P z|MaXiW-^0=?dQ9tUo(whVN1^BxZf_U1B>B7hno5+=B-^vR-JlGTibfzV9 z1UE=q^vQblNV->snMv!kK2OOU7+ScFU}FdpPMe;SvLvHO>KbW z6E$qmRd-N7{IZvQ?_R5PkYjK)vR3)nOGq_{A0g+ z41SCjxy#B`Ep>d_f3wG`%v1DIiB7(9Qm-)7>&ig|`A0ZM$Z3nI?~j#af*@34$gqc{ z0VyIYbn$@eZGz457}iIAoHujY#PPx``uGppYfnC{3s1uDPTRTR_uqb;J!8-AH0vD6 zIhgg<(~rG5 zz1nu@wCsm(S*2M{)`9fYopl>DMx%8N?uRiZ1YE9y5lju0_+aF7Jbq%WSKI1cBTEiaMtvH|BHLq%akjJ<&vf_5Mb74{Cn)y^>!It?krKgR_%+gxxUq@ z9ZG~t@z#$ypG?}ZZ8Q8Jne$m9OS^%{;yGVW|KL4?t`gqLbAJD8+n;uN!)C4brSDw3 z{I|ukrteMXx-K^97-`Hu9)07p*&$7TBlR0JI{EajT(+f-e>|9Q)lGNGVCP5Srx+tX z7k>4nJZC;Q>->u=SFY;u30q6%&s;F=Gb$CO;ZWn2t#7#Rv9_JMRH;&V+s2JwPo6OA z(~05;8V_YpeD(Rq-+EtyqWHC@O?6I7Nn+hitJk<`r?YzO-Lree4+|0!Dtdg{JGyD< z&x+d35AvjrU#}sdoJ)+((eI zb>sW5y|i`B3XjhN8T&qZ^VPpS@(=$F1^NOD=|ahdtRpft!J)*c#*qp+0r4B$l=%85 zC)$epmpCM%cssM>*Z81%$-m&rYx0lebN@w7X4V(~eur|SFMPwWhovY`8|pJ)2&1re z85Zv*Tp#8R#kqI~>>u*is$8mFL^b@n6faKXbz$e}?zvxo_2GCL3l@V+I86MXa@y&Q zTD09u{Qthx;y+FJLy<+&3aB6F%~~-1b0CO}P^fXMwm07Q$VnZ#RIFAxX+1QqhLv1k2H8^%5N5wF~%%0L>-Q0UihSFHSLV~4J7xyPU+ z!&5kvweR5PZ_ktvvq#YEe`3Y5fB(mjle)I8U9Z~CD*)Z;zk9bnuAX{P>{$J;> zT=}a6!Ug=8yWv34>`8XB(P2t#s3z|>z2+Q@;q4n+RwjeZjG)d<4*fEF#nnUmia${% z{t>!t(VFD-J7g+E80+*RfBDgeLHGA-)45ebxw0u+ck&{@rSnz@e<++#A)ypW3CM7y zN{z~b1h{axYON~gUww9u%g=1wrlII%e-SC%oxJDu`^IrohJkTCB0W86FDI(}IZ_+{ z^??+GDqdueba9a`WAB02hK*HifKdKrY1p#C6BF($l~4vKyl$1wyaMCPcV^Kp;K%$^ zwML~!#^2MXV@t&+ad%(j(^1ntd3~C(U^*1Wp%4E#<>?RZ$0C$ti^r#tPBjng&{gpd z@|b_6*MYLQ%jk9 zbenMfru?u@iK;cLdnnUKgkO2~c~^98*s7txfh`Y>LB!=cQy%rD2zR1+;W5=BDQYgTz^+}*7^wDj@k z*oerdZ%+H@)oE&B^JmG7geJc5*;AA5kHx3X4HG-@i}UX#kx;D$HI5!Va$rw3oYZMr zQzM>@ZwwwrgdO(!9-TOdj~{@^y;!5gzo)-Dn|{V))H|w^BZ!6EoFiA>aS8iQ(Ps>@ zLr|r=%w0b4p1#;f{1cyuWpF^3 zi$D~8Prs<+gKynYwrm2$;oHOWr+z=`zVWKDiHdZFLQhW`QK@>RM$PIfyKH=u1e0Rm z^nndJo{)MG!3V>PR0bw`T*$v#B7}GKi>^6`$EV%ktl)pdEVaV#|bol-y zV|Nzkxg-GGCWnuT)hj;!-aTzQw?ZRnHRORSC%^IK$FEH{blS&B9jA|9obvR)NBl+p z(vUyID#%Pezig$xhMy9{7pdQ@4#uHKZymx6B1l7sb|o_0#(#BJuJfgGtb%hINs_tW zN*QK+|E1?xpp1XCR3DLs6UB!8S5Fqm{DPg7BubKVt1Ws_Qi4ax4qtWG#e!k|;`hI$ z%9pBIyGrJ6I~Odu60@8qxE!sZhOz0+E8qhT`G|H?b; z_M_vQ_N^+>92Pav{g>8FB(P0SEXiRUbk;ACw zKe-4hZ=e8+=lmo@(s|j+fsllb#jJI-GNR=5MXrS6+je125jLVZ2 z26R*jcM_Q|6e)7N1cW)5Jow|c!DR%>di1@@#LCm>_3k_B(fc@uO+)(6`|^{)cZ^_7 zG>aLDdu&r;*2VbI3OTJ_at{30Yt+QV z(M_NaWMtH7*jU7vd|@+}O1|7?A!Xgyt@zdYD3+gkXJYI29fhEV|IcS-iVWqIgq5Cn`~9vxE|P_Yujl1g559f){nuZfym_OSaee#O zg|oW!=uM2%<%0eL*ZlUg%nn7?{`PBH+Ri%l>a#9>WPt)KD?4lHqIm?R_Plht@@kJy zsXTUU+zU^NTKa?{-7e_;_^1ixc(tushTERnvCXNYPd@GRm+v2{Hlfghucuyn_lV|g z+r{&5;YFpX6u*%^18%tY`B&}tz-40rcfV;|Q|i)+u@~v~@Sx?1;gk>%D@z30Mr0c6 z{G9@uLwcR7q>U|V;T^Q8e3S1q{>XG8(N^G9Yauq`<$Fm{n1iu=7?|MkCV@q`&Uz)Xfp|8Jv#mvV_u|kYlT8FTD?yF>p8Cx;)f)XgOeTF zc~`q}#3K~O@|OW0G-z@|bz>a=EUXdtSNUthJ^avIhR5+Qz=EvU828e%hF{GR%Pao7 zoOhwjHhn$&4j}$V4Czb!n`z-yKmYj4f|*^;@8i71_?L0O)HzQJGS^_YARa9|_TCGR zTcuoeX3t(vjDDZjg-RelTz_ik)*U;w?R>__2k$nSp+#R$8Fcr3E!wunP6uKLK?$KeB8>IW~^zJ_sSkvqV(uQ%}2L*n?-dA=pk-hf36I+Vac`dX2dHvem!+ zgew8M{Ktin&@&JtAmW2oFe?QD$_iFQBG-?2^!j@q)M+bv^&N2b#eE+f)Q@uwbb+-m z$B*rPalc~Y=fQ(noJ}(klG3~(VN57w#SdqO8{%X~sW1EhnPwIU7$cKhAx5?~cd=?!!sA1c9*e5H~R_2FyNIAbEM?sN&#vBWcz?r+kn0eCp# zt=q6x-GMb5w`=g!pjQqZ)X!MZTxLsOX!^UePCc_tjGxJ`x4T;m^;9;nl>rsgN% zNUes|Pdl%}y5Ba99`W9;5 z_UbKnod1ZDIA%eKP zFCC=7Mz+W&qh1^Ju8`-)(sZ{B*yc3c6Punw^op$c2yv3$_|1&Nacy2;M8N}JV1tU;j zqIRR2!(PAn&K{3*oa1m@{_dpRhQ26&L2RZ6ei43=OM`INUL8k_zTLbT;&zykcfYe_ z<}xEoAHHyE3H-7AP2R90S|dGfOY2jc85K-)gZX*8)0>}s`n|L?KSnB6O1x%7e;qCs zaI#~TxRIA6iI8v9s@@F`5472xu_pGZ3r~J?THaxQpPErkX(aOZf8NL`EofC!IVp7Q zJ^hw1l%tD?jWF7;J9g|S2PVvGEu@0|d*C!_W?U0K&Ls_h{nn0WpNzci>r;58RDs8E#+IIA7iuzT3bv$U7T`O;SnJ;YJBc_)j4w~f4|}P%|SSuL$dZ~cR0H}G6c}Ze~sN1GLB!Ub-c@;n<*5* z;ES5$^B0WWF77;H)ZclR5YD^$u1jCJcPyu{(Qf#u*NPpPlV_TdT{cPEE~> z8noT>&zn!`+=}rz;6EH5bZ@^Ei`K0FeJk>eIKCq*f6&vWIwpNbvCu0ouXpr_P^br4<^hCH#f@&QpQD71zPD0}4ix5jx>Y~l` zU=QooiBI_8lOag!%0lTg51E@4LaMiujgo@zDReA*Oh z-oAaq7A*lv6)h`NX0Lyt8YuuHv#09LRS9(Y!Y_VtI3EfHc}qpcMKpTxsL0bTti*_nod&39we;x7Co2PU=%lVlz z_2V5|HaIYp{$t!nyd}}JBgO13QKClux}&GSUM)8p`Qu*thba=mo|j!wx-|2=fdB8O zO`%k_C=|pmG%ud>74P|{5SkLJROxnpZ?jnOGxO6)$y);VC04BT@W`>{c=xTmRurFo zuD+ptr_Qagh0OWk9@IZU}?kJIlcQ1|K}*Zzfge~E?Jt}ViK$kA8IFrLQM23 zSJhX4(686b+wJGHvk95|SJR-9?q3;7+y7JkYHWo9ti$rCeE08P=dZ}cG|b_}az@(s!QbOo7`}*y7|<=WN;k)|c~6>wbQGd}1Qh z^E)*+-S??_Czd}OV zJ^r;R{|4h5@jqtjT;gBw_zRcd{9t^1ia)_N3M2;sY*_qjBni#>@%$@UqSVVDep#!2 zL*%2#*%gJG~G<7Z){Mf;`$@3~AZyq$9p;U$i{uujcnGa!_$LgxIKO_X%) z-TC3S?kb&71_3b7JK%45dQd5Tzg9NTNP_vw+Ly&0R`{5ynDZYwTEH*t8h#5SnwtC@ z9vV=;QEfVle+iI(Ima<};mhwYeY5}HFXpT{GVm{2p7~(JE7P91^roJP3A|J|@UK~- zUf{3WtWLw`b-^VR{t*f7PRR)5D+o0kh*i6Pz42$Gr?2~EquF86VeraTDEGv~;kD{k zS1cLT6eD2mPyY63`~qyrYuvVemkT@MU+5E&(fGXNRy|#-CbX1^VXg)2PtQ_BVC{+G zQzEUY`|icQqv~-t!4P;Y)mO^e{tHQm)v+!nQ~>RRfv&Fl8a1L{z}y9nLw~SPeIC)3|=~Ixe66 zZ?XLK@qy7HK7Qi)M~Pt-fuh*3NOzQf8*jn|`CLcU;Hf=CY1^ zjlQEy84lewezI4)<;iPH*pqSb@(*NW@p=x6Ezb0n*paj+KQEtQ)1@J=X1!{4>enKE z;>YLeb*rJ)I0^XKld?~HakAdVH$hP7xgym`c8hT z*7KJa@bkXgl3c(K_y?ZWGBYw!7oWdtezD7q)=W(zktVGhcD<-G_>1x1_hznEBFFIy zYeB)WJ!u(#k-zXG1MR3C;Y8%2tZW6mq_E$^Sgj}WlZ(=AIfr9)rvbSKe^dtJKMsO( z_|YkTT8b?eeb7U@Ku+}AP1BenW&cP^@PINlzR~^!`D-_Wh#wI&gl38$651EwEPgS8!{f>$_6C|

    oAq0M8(~e4P=| zw9MtxKrsqlMTZ6u*2P{Neydly7(|C_P=$jl&ISq3YDFRX&1aL4qBw|fBS7a<&pFp* z+Abvyf5j@*dS7`hEs>93x2;>vLw$e5UyL8*2l$0l{xW_v-_b71_!lm@lnvP&;17o? zR;ezmRPdj3T_wvHek+LaFFN#K0u5zPD9pT#pQk~LxG)R zap??fFDs`ao7b<-<;N+t8Y*9<+Ue)^G_-;-|LZCAEymAxGxW+^bq|Hk z@6(TKHO7?i|1|F#WwS(u2i$gdVzp{f&0Z+M#83a*?vib0;LdGZi^(@)q=5#t>(+nh zwRia8t|0ymzpA7r7XS1J&R+m{{Kw;u`BxJE zB7YTQ;((X$F8guO?>~OW-?AURhBOG#XJ) zPZp}9@+pqxm*=lkS^pS*HRg~11_Y_vC);(Sn0&^!KloQkZUeqsC#43mIRA4#on%9x z3{?E<%|tRc9zXKSSE=gpZ$uFPYj}04&_>5k<4Om5)*#BijSr~)Y2F;^H!26-dUxf@ z)y0?b6U$$eUUlnTbSHo5vV*r##PLIk@A4f$&3X+V8u^YUX~s>Fe~sW$B}VGY=igCcjLq#lP7PIJJm`Zhr9b-q+lq>L~xt&n4GfCyE6tlAV=x zl;6{d=9g<;vS2PctwHGQUYFLYSC1{_7aoYkXQV{EhK(*BbdwDTLbPPwtf-#QfZOh2 z2=(!&n1B1d`AOS1gI|2Q{Qs1{+CS3VQELB140`+o4nhzGGL99!kdy|pAK*+# zjVRl}3a>R14E=HZM&8&?S30~{__^juw#agQAyug(d5Rwff1Q+&vZ4>28U@!bArv*P#}dkn4wo!74m*M^;OV@jPe()MEsS*q|Wt_|w|Jyd~$UDT9JqIY=$zauadKDlda z$Bze03eWF}pGM8=81g7TjLP9~YI2%F<#_yTBA+RKEdJ$R3TYO{kISc}$B*+STf`na zT=7FyG5O5CWf@hnP?6&>?|tLeM$PMad^$gpZ&P*mFQ1Y|As;56XCz%dg5aOG)tF4l z1qAB{c|ENF7v(RIRH1gjR6{;aJ-hA0W9}+bx)koTYRRuww^FN4Ewvrp|I!6QGWXjL zOIiLlXjxA(geUl*<4@l#ySLX9b3gv>a2|&NnsEpCQ5Ywm{E8lr$8~2_c*TGG_`^#8 zaWwI7J2%6~J^Ymg7@)o&>({xS5}vGk-pf@R|a-zaH+W^48=vG}K*k3aQW)m2NOatM;Z zLtcN8zsQUFmn0YdL)m#T{E{UZp|TZq)sVQbzoPhL&q$HvYyYchhJsK=RsgjC06+jq zL_t*ZNbzIw$CnnGxtEMRjSn&W)W*hVg(%B^5kCRPxGAx0Fg_ojU%vlMOaytrF?b}L zk>Nlt``i^YE|zaT6=e^D1ZewPnrVIbeaM|K3TEFU%I5z6{Atm+aa zQccx1+Q3`*Gf)B@JwIUW)zCRN71j|?#mh$y!b{mGz>0tb%u5NwJ>hCSOqK&t9|Nqp zGB1?WPSE=0Yq!T}7eU@praz2U-kVy@_|?h&OoD{jST6abp4A~*onrE7X_F*}rjw4; za>7hQG0jx05}mk$Fl~!nd?wkN9@PmPyZtFHUq;#}VwRY8HUqPCr3-xOCHzVN4wY=3O)WHa{3Hpt z52;FOO}A)MHe=SV2Wv@-}bJ#+A3+A!X753D!ym(oS#U=rwT2NAHeeQ^>e> zl)Qx-#T~y+RGK=;r@g!jNb?a*MZ6Pee4ryHU-1@WQk<%QZ+@vRH%Drs%Q4*?0&W69 zqY3hnDs3pyi8#q&oz5-Wp0a$&ceJJR3-{idbMQd#fj4L-udz&h)P-8<#X8w}C@V8% z>xQVR(4_;1;9=pHI~jH4Uqyjh>lv^Z{vF%5NUw3y|K>Ym{JVVAxO}AbzIyN{@4ghGJrM#k3^@SBGRV-J&!eM(s49)pvP#z$n#*kUU z(n{T2vXmZ&-;UZu2TW;zlT>X&HM;c#hsGa|Luy2`HE{Vc|N3cZ$TU=PQIUU*cF$iL z*!WglKDtkszsLzlG9$nxu0RndKQ(D9&Cts~;$I>B2mJ2eJ_D~K{v|}z5&zp;wBa!m z=+^jY(9J-UkZ>}ThJk-~Zr|kmAiu+zXFEUA6U{G^65+8UiKb&=gdlg~k?;A7VvjoL ze&FR%iIw;TIPwxeNR?ap{)_ze?!+N6uh*c7Q>7lE$iBV1>o#a41R^<{lbOlS_d0$!zx28r)mK9@jgR*)!VN#; zGqownTWBHXBB4`t_ZhX8CltlyZ==R7Xd#Zn;q;X4;8uPrI{Eq!`D-nq(DX3KBmS@R zSMuOM<2Ed7T{cJngQ}!hSi66@6G4xM5U1>8O}MbJZ65@@IZU-&2GrEY!cl%TYRqm~ z!BtoUJIb>zJAKA`bIlv3i>YKm!H?5^z4)e{fd6*tPr{(Zz*v*+{359adCNoL^-Af9 z$1i7vLc5c9b1uLdMa>IEI-J$6bOPr|bw$*qD9~k8=vdY2m0O?EV*T<>0!lX%$(6%> zX_>X^S0h+;;7w$zHh{$UDQCAWQ?@kxaD%JHc0`F$#T=&?T6b)|ei=IgM?p`1!$tyY zHmK>um@46S%CU^bEju;Q6;XwPMmo`J^M|UnE1RgGtqaXQm|LS(Rq*>&#gwhPOqxZF zVMRs9^V)?=m6H49sHifvv;DS>|MD-o6n(TH5>zVy{QTUGjQtr|SsB^cdkvT>#ph3CsAJsU_ z&)#M7ZgT5P7rBKV-+uPv?&Q5DQXW!%SjC}zdO@SwNtLUOq$TY+^(^_9pIT(X1zL`= zqw_9$D^ST2&AyMrZCO+)p!ku4M69vFih-~Wj_16~yLCRV0~L%B zbkU?44aUEKYCN^ayAT1?P~qNP`*{E8fqmIoMms<1$LroU@yj)mOm|t2Rs*GI!wg$T zWDxKI3pSe}x9rfoL|MHCUI60qw|X?3Or)6{iDVx;KjFEL$;V!U&TG-GF_G}c@j;*rg@5@f*#Ul&C3pI~ac^oyfn6(=Z9%Wl zyw85voy_YMv`tIoyN~!O4Q~X0CYj=J+P1wNyGpFiVYvu4q(TGh}7pEVm)D_cI{NKT%_n!N}1A6m9x#Th+1 z5uA$8!tn0hnZL~aop{i6JD`u7+?$|*#2|jy6;kf;qxvX7*5KXe=j`=;e6T{%P8~{F zY2Bd(zqyWt`20m&)mjxX(fwd8B{}l@l%i7$}en?NzOP~Rz zQo`7>EGr`?%D*|n_`&rZx&^Tj9e)D!MgFz;G1v3*FP>F&MW#~>g*l-hJQ^Pgb%H}! zDX^hLc50B2`z(Lq+=9`HX{CmS6BM3b))Uqc;>c~{uRzj2>tOjSi=|54`NMbti!IU< zinQy}<}dcY2^Gr3=dbwTH^>g&6G{bN4&;A97 z?q6Y*X^KzqnrzB!rlkTF|1^qdY35OirE(_T^g&BS*4le%=0}tTqKp;=goWAh@~u5V z>lC?scS=k?Rgz>sjAUvIk_k1HqRo{X3atW6`Vhe$PQh0Id)(0BkwZzLuEX(5wDgS` zjTq3FB*~Q&m8gxEO0Tb)d^wpok|MT)HdvwpQNu{*a0Q+s8oRaj?R_jOskKR`uJw_7 z^7Q-}`ejfjzg_1pwd?b2E&W}Nge4>xCfPfb7CO!iBfhw~d`T04A_V2ZU-URaSwyZNK zbg{;a!FsNW`o=9&XS}gH*eG4Lebh5{7N4e!>XMu(|yV54HHlVly z$xuh*S@_LFqfAvHei_8R1LzeiS5}j#eD3~Y^rQr3_Ewo9elEH8`V~tSna!bsBS%L* zJZ$niqXyi3`vq5AQ?XK2M7UT&OlBF|yNYy4W}W6de~Q&K_UOYA6mJg&GiR+9&iH)D zy$>SC;xDvl&P-fX&bD<+k%D9CDan`vRDy8I_;+}$K+Mn+B)aw;B*+1t z(D5b0 zmm(YA0#PoX9zk1+;aGl&bTdl>uF1CDrxAuWqvwzDPjw(Y*dZZB#vy|EFK45N#BrS@ zC8;H&hvC=7dy51i5q|0olnLV?KQ$%UEK%i@@nh#s`2;P(ZQ&%7m9F8;{Y+eBGyV4L zAjprE*856=@tM?8FPw{)PdY^cogd|y)bYooRt8o8Jb&{KAKt%bcgCJw{48`9KO{_d z{q7@9mllL*V#a{Lx6;+TCH$(D5Jo?}lUp@3iH#4X%9N>Gts3tka`Eanq)hnIp>ZS7VFy0OM$lR2#uZET2t?Ih?< zJX;}sg1IYfqg9{0N0KBgwR24&2IG+DL*9kdOucL@iwF(rZ;aLWl~I zD4J?7uAae>M&1G4nI$>qU)NZ*ZdGaFniRxX=8MKS|1#6!(j|3{pU#BjvhP=a|Mf4+ z7OcwDZ)5~&k?ae)EfZ-glQp<8{1|?=O+JI7nIte$FHEvbjqI2!5Y@D8L+4*JX)$cj zIS|^uOTE<1#|QO2oRhb7!Dq=6uPspr6Z}a&I};?pgAMlTh3UL7MJ&Mq3826zo?p&C zuO#?>`meu#zec}4Bg*7oSBRXz!k|=E;d;)}iVD*bWllZ3(EGA+!u+jLQwN+{j4b;A zpe&L+gzz$y2d^Ic*uSj-g?u}0tVD7T<t=tadTc zMO9)$5{NEs&{pp{W>1=$%>6rx4cOP1ZTRzZk1SoFAIOtXDc9otY{g1}XiIjcUIF9& z4Vt-65h+tbwdz-gvOoGaHol!OfAJ*!H9lB=S;$Bn_=Ej#C^Tl?v$g8i)JbhjpgA}| zpdi+b?+FQ|vB~q-`7tXEv?K-iJ-omlKYp@DgP$Plje;2e3O|cg@FS3$hBmL-7Rcf& z5*jmN!mZC9IlIs4)NWY0?ac?q=jr`N#H9oo!V)|v6pJ4Y;1#&CkOf%shkx+x@rhWf z9ddmBntNn-QGN-64jQY};%;t473D8V053x?gRJ%HuYt)JVn4=dhDf9#X8h*;-$A1DIaUTD)jKzO3`Bnwh z!TyCH;a`0I%4AmyMOeyqEX0f)|DeLk%{sB-{-aWh*P89JUgpSLwUPX|^ABv>q8iz1 zny5PzsD5RA!MV+vbxj5LA=xYV4}4irA#uoxTD-%Nb_PZ$*M;F@r zq6-wq%ZF7YnIg9WafS@(29EY%kT&M83L%==2&(`U)s*=sVXPOeE3^bnk;IOKr&~-g zASEdhnm_fkSi`woE@xspM@CXC7yl-1*IDof~ z=#fXcD;_1%>*8XEneb%tDF!?$qzpa6rmj{gQE;mV^`XP8is5Jcr&aRyZLUx{^b8)e z0prJPcON02KV7*DF(B5;-M-E7bGR;by3;|4#Q?8Kx9C7H^D|Ca(L>=O*#;K_%@~!kP(!Pje65!VRj*x}yEaTs820c8I*E<42?^c%^qU1> z;H1o%zyg z73^fGD>Zv{{1(eE@hhtc8liC&^j0=dqTJM$`!~oh_R{KO=IWC@;h*NO-2fcL?-W_4 z2mMbJ|3DF$QTbv-ECl1{@$tW%!Ht*^MwIv>8}#!VhMo=azc(ehkXP=b)odilef$Yb zwy{)A?Vnb#Fv0_|$fu}W`HGzEpb!a!fF{V0g@`aW znq>(OHOk*ewOVxqREsfddjII(wB(cm5yc`3j41Oqo&N;CMb5D!`FZ6M693EmHHj8K z#<&^AROwImzt}IToPWhX6YTTqCSDiEbqDW;j)tA-oKmopAw~ zVJ9`5f9hBamR7htYLYm}3AH(;DZf&=o8N_1ja-5JEvu6_r;xsDfBFv2RueMPZsNH% z+l38`qn(8_Yu3{`3pZCO!7mL=@CpL0p{lpydYmQC>U zd+uKo966fDEy-}fGZA}Tp@ptMuvw`_WnJje43R|_^TZr?j)<2C4{zFNpgOgNx=181 zFHeKI&c&%ji|tgX+Xb&5M(dfZ3NOyTQKDa~2+&})6ub(6bWwiNMo(~nKZkGqO#Q3oz z1aOE0(`OcG|7h)sf&(mqxjT~sO^*gNky ztIDhU-#akO3^4RwMXG=W1*|C6C{`?p#@>x4_HGhOq9%E*(b!_|1vQEV3pVUV#fB8= zy?2-b_xD|EpL^yGpfT^~_vd>C?mg!zd#}Cr-p_N+xu>k>vC}%H^Um@9$L#+3>;L}u zyWbd={9yTFJ(d5(yWf8O(GREm@u>Ze*{jz0-~i8S$Qy<;#g7zXqVVT3vnS6M=EP91 z7Ra}^DP&Q0`A_@LY>9ArfyX-4jq!)H2BgR{KOr(mC)qdUU&ZWC<*#%(erSSxHp4GO zkep{yN=r-2%IYai)7@Q%zVesm)s~OTX2 zJE(?##UFwcXngH9bO%m$&`2yQ7n9NNFSQPJu`G^J2|pX;FWtF$3V|EvuPy*Iom?eUm|ux3 z1qCcvmtacN3`k4>y`^P^pY$qYN-rzW4jf`t6CQl9EwYMm4S z0Flb1kSSqy?FRW9@+)=in}h^)@#uh6tJct0h~*wkWtAoUP0i!K?F{>VO1}0&wO1oo z#rz_9DL>9%%ZY*Nr1uRL;<#QHHQt6KF%X? z^siAI{YhKvL#Db2l}t|M%RVhCrZ$z&HtnhKC+ql1<01|8r-G2B5l&8}kk>EKJ~XI} zxs6Xl`SH82r;KOLR(B9Q_rM)*zxb>#JeGLv2?CXU-zuGb3;6(e{kf;M8?=XdX*XBW zV}#O*;8|QAIEWL=MG+T4aoDLv{bZ&DC~(DvjYuk$6=Z>{frpeD{0rtx4;{2EE-B%f zd~I63nkg)h&#L_d5;2wFM@P(&QVV(70i>>$d}+($7aVT_-rm@ReC!r}0rZs}GDXPQ z{nDFyWg&TI^8q_}>A24YH4 ztA%E%eA@7%hpZDmudyGeZ?S{)y@np4lO5(~%7pKxO#H4(k3KNL>n*?d;O(HF=KCBr z9Q~>M#dBszfB=P5zRaCHCL*z~fGEXRZhgmY-8j9Xc7^^;=(MGlha}B2$savwj`$k> z6a2D3i}_as1RR+a^3}_RsYnAvY|@2dHr*#WspMz!S7xHj&_lkA!^HobzjO+inmj{w z^66qE=lz?==L`JvX3aS7i2e9<*sEh2{|RjKZ}x$LtQvkoQDRtLe&ELoHT(#gti}$x zOdVt>pXg#})|zrcfBLmfBJvahLcIlyMb2hjoOlnp(7zVwwuyr*-yROV8|5!Z?9C9( z2x5o@{+%~#rj<%-#DC~c0z&aqGk#)AVf@o2ji02K9SL%bRiH3+8UNfl(vl#d21Bqe zZ?eqtl4Jo5sn2FQZr-h|l-ISWIlbjy{S$c@PLfK%Ci%&&pE+~-1$KHO*~;iCPgpS#_oQJgdil^sl0uc&CAN_BmWoXZz7$Z_OBu{G0s{x%HMFM5UWeT z0UlInSKT%z1EqG&Oh&rPLhIyqIgb+RFSWXd#T@GvFLnr=Ba#%+0fpYvFg{mSRMH4|6?zy}AQvPdBUxm+t=32@EgH&o zB17_XRU8BbyI8{jtG~*%TvZ9{$~=@I%KXV9;8DiD4f@7f6^@+zDf#s5~9%{5>tMV zCHclLd92b@X8;oZ)j&($LbatVS>!HJr=F9(oA&(OqlW!v2>8Rl^n$FlYy4ZZ8j0kT zpO6yrUG5gO<;zm81{%;Ery2-~bf_k!^wgOi*3^g22^4h^evNNxP80bkS?U##e?TP_P6Io+~v0!3YVMb zznx!Ps9Sya6^}G*)L_?xcaSWP4-zKZ3*@Ux^NanfW;B5D)vGIQl!grBkc!Z=LH;JX zRVn}SCoMc%dO%N3yT70v@MM$7uI4#ZJgHG7{ z%_rWQI$>s%TZsU2ZD6KKEAF9S3jAda>YaDXDKuu7GN!~85kEAhgSDtRgM#|$haa_# zcorTjqY8jb5b(c2{#r&}IsNkE8=}+kP4C;1HVF&K?jZg4Zz@BLXc_nm$LHSLcmJ9E zP5qnVuir$E-8FWL6urFWyfW}3UnCUrFTFIE3%rK!MJaw!scR*$;E}D`YSj>V0oibY1OV;;@=24tH3zJ zX2VkuVt-10%#UGz%r6DwvrLg!WlJRR_(Lln~|7JJD4uoRi3KT*i+924lk2Rfb7FDzHzp5vz(XB?;?Z6Bad;U0!jD z=`0~Js%Vfn{9>%x%%~Th5JDS;#u<|*^UuAJ9pp%(LWI6_HGmIZd#U2On>pJiCE6oG zoX@vfBOQe_i4gim*z5ck)Yyo;=`zDj13zX7jr7Vy|1<06-}tHLaJxZ6?#`8gfU9P2 ztD=JQ$H)h=L9#3*H1-%Cc3xx-KY9mg%~78BEM z=5|u8J!2_emnkkQ*=6q`segliHIWDdwKxe4K2$~IVZJIj%WL@npd{j7_}SL^LW$9Y ztq5Y#Gm4de(V8!Yu>5|nP$4Ap)MzE&!6;3e&R=y>{3`=cq2T^S`74CxDgCtD1HU4a za+Bo`{Zu0P3+K(d@Tdbg(}7iC1mN3kwt2JGZFu)^ojQ7U>3grdfa9GTA~pC?WBysL zxTm)^Hp*WSC{?L^Tp%ehE-Ed8Eu?D^qlj`JwTk>AgjFFa`VOYAAhz zfG3o*BGp9UZ%%2;A`e7r6e`0T1}^w4>MoXeR4QN@|M_EQh~Wr?kzZ?9uBoc7=2cLN zA5S}l<8zX~#^1O_3&}GqR*pFBw`C0)z{bY>5OY#2p6U+T=fE2LU?paLHh(?Zg%Tqd zZwf1WbrbpP2~$x{t|D2~hnp!w8Cx-3N%B~O>fH|7VFV{TtOGXaOS6#=j;^k#jQK@bW9GKoW6Sc!<+L#(?SQBG8}g-B z#%TT(gSm`vFp;}MzWkZxj;a-WQl zMIil=5%UYT$8-6Na!5g-SW}&-&|tNOr{cQ|f!IPT1{o zY626r|1=_zkscD~%Mry}c_a~2BZP>UW54&8laAg#UPMxIS`UrjoO!j{?u1ln#VxfUZ`$4Dt z>a#b$oa0M2l3Hb9=82o1-)Z0eZM(D*5Sr65lWp0aC!5$qy4}`B7G>)v}pwpyE@zbmB){Saal9f&>V!@5!P>%}Kpb zBs7UpGAszeQyv0z7{JeFMr72<{44(=k^L)=ssiW!tlyx%jheCtrSXC3LR2Hm zI>oeMr~ail_;1lpHeFb`S_eCKhB%>($+U$bcT(bje#F z8~CqHdrLk81Y5w8)5Je5bjgMNOW^)g{=x<=Q3R{L+C`#tIk`+q<>N#`u*Sd81F0M5 zugxTdV7qYI{r+Lj@u_l@13|7eKWeX1S)|smuw=oq#S50XJ0CTRh0&7| z?A9j|F#oha9oL~tJLKAjq=+5HrY3^~P>gSCt>M}Mp@AJFY9Lu+kA)SCbwX)XI`|el z>a2etPxDtORH}m!+O=rUL(Qd98uCkn#<;!-+lC@N>ZJd&*sIE3e#qH=-)(zs-35jq zR3Tyi%7lXaP5cWkknu1(r#Qyf6N~gr8AY)k;gP9J7mE}e%gLKbM>$hV$WI$sZIbgL z-xgXKzqfG3A5O7#bmTml^g#f^xI0^*R7aJL)4i$5LxM2f;{f2+t$*uX$wNNR!@yOjbh-Em1@q_V zS_UorcDfl4*_Quf8|W!NPCCg~H;h6SKoaK*tJ9qb6aTW#(+T7Vu=#)IrDvv3{2?(f zli6y(4z=*-%?^M_vyT#fe0E6a8#ifs;(3=JefGtly!px#cicE`>=z*rz>nW~)8GRR z?aDJsWPhejht3jZ5`oKI`fYg8W8z;%63bVvDW64$XP9;!yWm$5Y!((SSkSh82aR-e zz%EbLCP(Dr-vx7Lp&|(D*oDVO$E+%T=(NWH!=Ae5Ho+;%zVqUUlP|mi;qUxwM98P( zUWXo;_*Y&QmbRU7pO%oh|L|iwb?vEOkRc8>p`#Myd{JtR@Wf9E+L9c_vxU$~E|AZX z__wvEU&W6W#b-o8=mM3?qw+uJui6#JPw^w) zKKRM}C9K4PM@Vc4e$xC!runIfA00qvGq+uN0e2f|{BiycKjWN(k3X$-hfeY@BJ@i{ z?x3ySs^(D+=7J zg5UWY`6X>MlmG^8$^WVRjrak7$VXaW1Mre*VRQ?9z=Yn~&MUABr|B5x%^YJ4_EG{{_OPw*b>9~1Pq3YbeM;*6WSC2gAKTnR# zw)JWR56XYXviAhUq4ov5&`=zs*--dUeA)>3x%Kzw&z&*N!ldrVGtN2mgfsLO>H_(n zZT;qorAyJG1*w*!_*YD@+s-S0gp2NMhJ1SC;==sZK0{S<6=e?`etd^+JwgMjQ+_a2P7pBf8VZ+s{lfe; zl_EpY&tJj(p2Wbgj%h8h>wVg3(PFO0GJd|O%A0Vey`6vo^J8Vb9 z(H$6)ukK<>wW)lft4*=32Zj)(9&&9JBQ^=a0sKNi*_?OOIY+gbu@MhM(M$IXrX;g?eR+j)82WgrK-6w`4Z&S#*Tj7Rp)Lc4~nPdr3hW z_`^Giz522bVP5>}SVQ~^kfJQve@LMCM{DyBKdn2r$(t+uT5{RNvlfBh)B*M<{OcWd zepvbJwJ+2ufntucnXPv0wcqi3@eUncp-K$H^q)?><>OIbx+OMA5BZhwFTD~V52`}S z^{qDx)I00Clde4KdNx@oj&_`pz3r0wu7CPc$jXloIfzoN{A)?XX)b%fiF+M$-eHL3 ziWYNY9k49?YO7J)e-WQ@3F#mF&^D3J3V#wTCku%peB=0cZshR;Mz+go=2J|@UGZ|b zE>jm!12W*S&N$^l$rnOm1F=rNY-MG|C$D{}hUzFhieUDvcI-Xm#KBu^*R#B7Lp}O2 zli@y>PhS5r;hKil;1Q!Hf1O|SMxJD;mT$%9Ch}JjWVj}#Sm9Ip)BKHn4y)?at=$E; zpLyc^;cq=P=C#M(SvY$k@rZm4e0l#aTfE|{G2i;F(u6AUpSQ}2q=QA6Bv<^$_>*CO zsikTp&j5^n{}cBITe>(Nng{fbRQ!Qt)A@^1q44<3t_J$c8#Tx?Lyb&8>6+DR zB7Zft=uSOiOZcr@2x{kV(oOt0yrf_4>>*u=`emdadM@PQ$wF+LMcY@V%k7o7)@uVlQ;*P^1U3GN10QGZRuIQTmi_eQ8DOmaN!NK*fJIh+z>N9lL#HAPX~@3EzFywqqyB*GwxakgwB&(wgPXBUP1^nkr-q!^h@{d^S9i1wWQ#8kNbG zqxSx1NgM*JRPv#jsMc;rA^+>Ds{i@7KLvi-m&xpL$YCk|L_Ss_5=k)yB@>${My|2z z6CvErH)!7jcKOwTBkuplKd!zks;JiQ^+jZ`m z%9k0uzGvv+Pu+W~dCJb3GHLubU$<`C=8KQtmjkKKWcNJq2qpq}fH7pUl+~}_yjAN( z^Lav^AS|3UtMleOl$-YQCh`dj`B79FzxdZFJ%y57s+!n&Q+w+H9=ZU{r^L|SU*=!j z2mh)Jek`943WO8-!@rhKZ{#~_qXnz>c%qIifs@Q&%+h6Uh|-$6fS*-2(XU$Imj$Bv z7w0dM>8~@#GD0FHLHWyS4L>M0euDhl&pt5oYI^FWe?00p7bu)XwG?YHIIn4&a6zIi zVE-ytR?d%~jFL&?Tj}kL4#y@MtprASm+7NTR!_ z5~G{WUyRjt4IAS3_9pWeafT$W6vK)MF&Oca$+l?MHjRI3e*E^!4xBW!JY|&ts<~`T z;=h8;fXb1%Sg;?}L;v%EM)PpB`Ke!@pI->_xUW9DaQ4hD-Fhl$=Wt+td=NfEo?(CTzy7$Nf5q@J=cuB3GdVE8yYJ^>8|EK(wT6L6Q`&S%F zrkRlWTCBq(G4YIT60O;sBGx@CNL_4bg+E#l|u}Yaj$-Tp(E`yo{+Bg5j%Ez zEP{lym`J;W%wSu3l?qZn=)mnCzv+3|$F_!iWLT?vYBiH-*s%T%d-M}EY+F9~P_0!h z$~_t?wADK~qHNQ#HDJPUf6A6mM~_`wQCVCHYT5|`hzNG-7R*@q6 zGQA8E!EHPE@yQ93=9Z^;j{ch45SXk@pN5wJg~qv47ykIeG^s{<9tp2wEGa7!N|h5y ztS0=>6AR@<#T+fvkG~rJ*)qRoB1Ot|AD|0oXLt-sAV|hH;mPsLg^G%*dZqO^8Y3`+ z|G=dKGqJ$ow9bJ4Fm_7iI>r%!iUpj54j6dR?PqF6N0ywxIe7z5nn)oGxmK$>b!cxT zGF%`2Wp(SFdhKz$$Ti z2%qS1LQ?Yh3;7BEDzNFNsM9~zkS{wV3^A0r1wi=c%nN?>8-kRxMTLOSAd;0>lT!CZ z7Pcm%l-mVn*h-RxPKlY|WL5u>s4GFTxd~rQ=3axaQ5vKqv-1H1F8TYJf@rqsfPC7p zL=h{k_n;I^z=yNr$erMQqE)_H-;5x+g~urv1?{rrdb zUiAc5x1rwFNCO!-dEAU$58P3O6q_Wkc{@}Y6g9pOSn{E--DWw z)%nE@3oKWS0l#aXq87DPNToh9I893RxA0oNdj2B!RPwS z#P6m}96Lout2rN;PCeWA*s9C;ucpkKIe%T%dLGV7Dz)v}dh1>K?)aRxm1;r_vVwoM5f=HY`ryUbQv4dbmo=!b?$U^Co5+aAEp~J@U zlZ2t?-!)u~=>9fG!b-~O@wU~N6d+Z3pgOxEKmG_6Z7GeN$6EdsCKlMGuhXDrd;=#Q zD9qo4wdjLD#Y^HSWFPkslN5eZbJ_xrYxOF5Vhsge{3SP7 zJ`%i0K**0q-qNa|%Wf+5%nll}?KX;c4J9g3Gg@FOZPnD`5+QEygoZ2LX2ppZJjf?I z4?H)|($$Y{!sUqM3+s~snH@(NCFBVYwd6+)y>YQA6B;#^@9~S!Xicy=lOw1sa)^+z z8MO)(?xA@7`KQyeo?G?1?t!PGzf>BY`bu_!llOzX{P_Jwmf?vEOBc*<-cqA5nMp7- zZQgR}q6MLS-s~A2xCFp55o_wU= z&bz9XtJEZjVjb&+{NM)$xg?lNLS|0+Q7AK+SD$=jw;_k*Z6pE>0)NT4+`sO>$I#f~ zu%uM%$so%&gFSk0-K9sbDHF!apIJSZv~@>*&b5?GW}96Gw`kK=sx>%DVAQnj)MfDk zo-r-}qhES{+d+G6MPlL0^6h!~SBo1EpA6!fY@#R3y@A|TNaRbUsL`uwl*m)R@9pTrM1ep&v?1WZYAaU2CeCDk@KivOcw`ucMOBS)yEX8wXPUC4QweeG4 z^xkX#O3yaR1rkE@)bsEu@LIw>eCP8w@Fzo*d#qWZHzOL6O|};UfP{E_lf@`u8`wXS zzmlWT)!pUubo>eVh4D|*SD$=%xBU)ANg;ksKwU78R-TAvHd1 z5CUP#3sfh28u=~THsL36J*YTYG{&u7zWU=gzu@s7;0HE)%_;s@9(gM=#M)&NZQ5+c zbb!_3rzP)#saKchHyUIh&a7Oq`lC0$+;zwR@k3CWABoLmUwv$}qFEXQQz&w|Haf18 zWVaWQYAT8TP_(n6e>5oM-|nDXQfl*4z(4Py?5H~e)W{f24bS;r_1?ce+2_bXh5U#7 zxic4!dFi9hJv(l{=Qa$vwc!beyW{396*HSv#{#1qN!3cnZz{@~L#CNY?R&c}a3 z2K|vqbm^36!VkTC$fb^{e41gx_g_vf>P07~eD zuA-I8RtD|#E!h8bZ6296#E-`u>*sQzvhwTZ+x+6?vmzho2y7R{-l4= z2wK^q0VY8tdOSuY@tHtWT)L${)W|CN-Y9?Rfohbe^0i4nK9Z3th^vVoQ6Ncz!`BA+ zOJP1f6(lqPxL1&B+{Ujp`B(f$ag?A~trLDUfFPfPX6}KKak5O+{2M%Ur^A19fQk@6 zBjQ_tq)w11SJc<(g6_ zP$QpbA*;m3@kvb@pU@KHgFra`Ws91a1a%4E7>fi&i~m$#n7?!YKm3ajt+HbOvObHEJkRQznM&vmb@QxL;ot5k#GDoMJ4T3V16)=zry(|$$gdQ<6j1u z0vS+{zp}%&P;Rl#4sFOp?Iwhw((fR^d@+zTP(x9#`yjaoxo%NbuXZTPas7nuj1;t= zlWz*GLvm92tfmeU*WO()AlJz9n47gGHsC@1q0qk3@J+aAW?HXnTdFy5lLB2P;>CQ@ zfCFjs2+A-xuQ~QmJq=Kl$QMBVt0e&RA>tKQXhyfzR4ZY$Q?aTU*nH-V8ICBH$?7Mp z5j2$*t3P=4U(t=4!3Q0#dKs8#vor-oDu1to4mAj=0py$idKQ=K2m!0JMNL|OVd0dn-Hd%h%G@)5pt@xoW1dDN}=SijGqM*$VsYM19{WBJqqUidX7 zi@6Y(@FT5?c3ce@S}a+-;Hux8@Xs5rk|&HGFCTyId} z5uE7g(@#&c({!87e){ekQz!lqkp&dYLcScI$u3^J@a3l;4f&Mqeb}$l_z^$ogr6eb zBXcCHG@#|%BcC1p(z9AK|GyeKER7$Rh&JlF_uZNvtnlyk5sxpJGuzbFQJoSm&p&9v zuR&P+&{{^xR#?wxt0{|A6FiNl*rE0`sue#PmSq_=5uXwk@+ni8TE0~TeyIv$q)(*$ zFY?#B7}Cnsf@yq;i2s5;@7u-& z6;wbP6oiR@bP7_!e$pW=(j_3>T^mSucT54LyCo*wAl==~7_q_PzvspCc6)Pu?rXcR z^E|KP_#P0dhp*)fSQfJxfJYqS#>D%NT2~BGpeU-TA|*ZcnPFgP2vp|VBkD~#lN zM`q?T>ON=wWk)zHP?P59k#dPBo`F#Yn>VLmgGGKncK^Jy)efnw_#>_;Y$OlLDN_+M zmJms)K`289oCz@3y+*}5)xm;Ha5i3m1 zbezAZ;8K29m>(E0ev%arFjt1rUc#Y+>wl@XbLJZLegOs2<4}owoIzVCEyT+ZaWY?l z9Ev56&D^$ru9mm9EYgTp?>s@e4fRJb`iMP68g%&Y$)_of!bde)!txZ~KlAjNY(TX$ znpW>T;#I?R|Hk|IEgOcAE-DQZ?F?iQP{;j03!uRYnOr$oP|ZAc#QNLG)ir`wT!(zE z;n^niyo}BIQ#Oz46+h?%>2NjA4DeG?jp7t0ai1q$X@e{&)`2c;)=PCL6bb0~&)e>z z4k;Ably+?*I{FjV?=E)<*YR^k=fIx-VQ%iPRFLE(NAhV>?Qb7RwFqKsSf>kg^yIk+ zOJ4}A|9kq4(^F2qsmMgB+)JGNhf}}BeLblOcE%e*9=By$QTG1!X4(RDTh0Z*jMQWQ zSbu)e9-h$}8o-=vXBv_<$VO|MF%pN~Sgr;#0#N1V(GW`Ti?3KgpMDfq+Iv$ZmtQ?e zw$O(|QiW-a-7md_BiYo{-o|$t!=#tXUjP2%LC$?VE)%A{#qM%=O$7B;SED{ZH4CO- zQVMW;IWlSsf8nqz@s=7MqDMOY{PT%7>ex`wk}BrEnp-+l(C31%1!@PSPQ<2|)C0~) z0LQJZ@FtmFkDGs)X!-9~u+DV(hVKZ|Mu%UrZ<v6(>HX^l@Nn0kQXYg)pr=4JO%> zkRG;YM3?hw8A8H}dwiQ!eWacnx^peYqtK&AA9IgK_s`2%x$>d=vw+S!a%8P6>JerQ z?Vm}4>%_|&?fYWrD~#LTeKhWrP(9_8Cr%5d9r1oR(Jqdo0Az^#q}-teB_#216e6-j?F$jFRxGZ7-s*mYW>6k8vmxkQRm# zDF7Za@ZQ#zKAyR#R#dr+;qA4qL{S4?9MltZv(?zC@zE23>D+Z)f=%3dkxj}N7$uCo!ixmhIJNf3er$t zUkrQ~knj+g0TqyX6w#eBi?@ua8O%=qE5UT2d`h=F08wj(wSsSUB47-W)3rEkmk(yW8-lTOU}chWv%Fi-HKEP*5J}{!?H7)rx8fbjEms(4~&gJS)S+NCmImZpbN@B zaPYmJ{AF8ACdxGc9ZiHR6iKXDHsQ8_0KeWspRVtMGbb$YtKf(&uc@swM*i zFB7auSIfW7a*7P6&s2j^$PE0cdVd~}7P27}@-MXR!nr^;w`^@QX~VSDZ83*AU)1#$ zm$r2Esxd0W%}5F4dG6>b_bP9um<;-?!Hft+_CpAXLN`ymY+5g|W}H1uDg|fRC+h1U zH;=1Bio?~8K=XadnOh*v=os8#5(Z4fr zd?z=M>-`dV+*OM27ASubcrUCI`uvf7_%`Pi2PN41V2#?|Or89%IUKfc{A?x?(4Kqe zeg3n@DnT_*!u?JJ67a$%MrFk7rJGr-HYZ2qMB&7rAE~z%ZK)!|CcZ}(`qRH{PHF@A z-+V}$@E0=mIbiG{*5tH<@jX4Fih@zUuI#;Toze4~6%*f527 zr}`}1xn=3xcb}+Ozf%DI^juxSu73u=3=N7)6_OI;eR2QvBZ$_yoCJ%H3t~=kqVP1Y%Tz1B&_Yps?-ZuAkJmsU*CKTAZ#z-a@b0m0N?DvKad z%mpu&tqL5p9++nZxjrfG5Jy$v2T%L-7Ry__>FQ3b>OEkhn#Bc~n#NS>ZJelcir?A(p-c%KNcSsB*oR%5l~K}F;RlD#5u zvx-6D?vP>B3P#u^3ukoz&~Yp~I?7blZ|Cb3QjXImqrP^X!~^G^&wi|g!YS5wu1t^WNgiR&*W*6gRjV@8cfE0;l( zH$ecljEV8r;>PXq0jDms7tW|Ue0>@=(9BpHsNqxYc(m;94e(HxsV)&ht?~tfnFqJ- zXAj5&MfOhjr2_WGGbYwKI zv1Y=g2YQ}Y5e06cMaP5Vm3T}w+84I>BwyU>*!UvJv4UOcMTs3>2RNW)yw7<6CW-5< z3il*C7Q1jT*jG{=OoqhfA;2_fgH`QdJ3TRQ zY+ZmE73j>zy=y2?c0FIErCA`&At(`6U#4x)1EUxB%idm>H$w7Y1ET#J;399J-7C_4-k8o z3nGV9{F4W3(fw&G3o|ummO2gEfT*96a8}KRc2?}d%1ih`1xYzM5;@oZNYqzBhk?z; zmw(s!{4eHI7+bEB?=hgEhs~yUe;qS0aNI{ba0oac)pSD}`E%sA;73-J1V(cG3D8yA zy$|aAjhvt7F^Yc){~N>ogq)FSDdqR>Hy0H(1&c zo4M;JzKgp(&Mn6Yq;UZ@4U8$N830s_9>Nt>bgz5yNV=o3I0#@RMm@#$wum2p=plK@ zGuXV_e=e9}Z=*C|{aBV4x7DP4k|*l8)H==`2#R%}HSB09#aXi8_OSTO zZL+xVDv0UP04j_GRqt=!gUtD``ce`1>7cE~3= z1t~C-G1?O97h$QSzCEyOi-|T&Al-C$=xYDj04)Rd-D^@2c?pazkl^jau9Y*E>h<;U zb(@M}#}Teh8_m)eieCGj3?#~L;VBf5scxtcMh=QB4ncN}aR4v0xG)UV!9Ze;D`!E_ zUS-rpPg<0@mV^80#uan_BhD~pfgnATjPeoE)pkiMhT;!yeHDmia|?xss$h935@{)} z(tz8ksA$XYNl#8+-2E&@X)l-C~74a>ks`mz(y zHVR+Pt^_3l0zI2lA26ZvCzI{jvzH`c#SDIMqbzT3IC-gIh@huxNBH!muf?44& zRoW|MW3LnQ2#D8BI-1a}@}4UJc!`tSav4#~EAb2<74gIofr>q0SK@B=)3wv7FyLU{ z;)?OSiKem6V&i_gQ46fAME6GWvPAMZ9gTk^gShu9_JGnP;HuN?d-;zE$tzv|=aO|3 zO%Fwm=j?APY$-cY0Xx$PRVGyt{Qg4>zIW$1xhB^L+1MeQ$MFMA;>?mzz|Q>&?IKrh z+5GLDt7$;MwwyK5mv~x}3+gX7r6J0Z?e=!>B?C&`Ng$~)4va^na!QM2v^KE}>daDU*C2kVgN zZ*}OYOA`N6#i|LfWzq%D>WMY~X+3xC{y^`W;I>o$UIF8Kr!iID6fgpFZOsUI7nsef zF$ObZjAy5!XLx^HhrQS~4Y&d^Tw2!0(2fC}7eM#Kk_Fz-;JmimTw(XSmI+|sZkECq zclY3iXFE086Si}8taN=qSopEWi^{vX(HebGPiZkC!WDwyb8-b0OW zZiNR|-*^m3CD6GMPS>9Kt)sowP;tj~<8X+lT0-Zt!&<$1hSYFrsvmFwcAH9Bb|?Ze~9ZmcNeJ;+n#DM``GAhQ|GWn|gV%q(0p! z=Z$sMo$9NC^eQv5=>Ia~2<9<3Fnn-tt7Kbs^u)!97{B8m#6`|*55;i znxO)cOQT=qhWd(jyjL641g`{aiY#QeruPnFp;!OL>lrMgZLET1pAceqOqJB})5X;s zaak%4o4^-$#j_dMrzUn$1a|Kv03XOY>IgDZFJHg?@hnLu@%*6R3Z9SuXydb$ztgQu z6BtJ{1)-emF32vCMt^K}^D=-ej#1)K??8D$8kY9>!;Ij*nu4#O@sAH-LlY8mZO<7H zTOEaO14V>Cj!?118kx3k#g717Dqfo`pi?Ejie-Hb6HXC>U7A0N3Z9nFmDrli|2Nh8 z$XJByDR;>XE%13s2#%`4{=>khflb{|@-p+i=oB!ud*TkAupGF9uAYGOIhxX(us>i+ z_)X`F%}G9FozWQof_|Huk_K*bnB*ueDi_8(?VtrBw^p6+&O1Fj2>MJDkmhGsNJ04{ zna(*SZIF1srV*W)zt9L7^PNZL>ra=davvP{yoQN$=i1IN4o=ZN=Jp3+xU-JE1v$^ z-8I~rH^?%}8WgEKMEZ!LKySzJS3ym7@n}ugPbWIyXu%~wkVO|}%WP8t%O&B(3HAnc z-}P}>Fl$&b7x0*_6-dZybL2$_X#7Ws-{z*T9UO2tSy=}Okn%eT>S*G%)wsFNJm34Y z9j}5EM+i_SD+m1Lmu8to+J`pCMe{$ACi`)+zz+_apQLvzT1c!Bzw}<5O8cHU5+J#^ zTs;u5+90=GW#}|lX2F#Tf^W7gro4=sRX{YBINnyiSFn#IO)pU7NCaeL*d;#W4NW9y zJz?ze0-bJ7Qg&w16(ERFlo-jwlaA^gHNvW??#U6FYMiKlC>hCB@os3{_Z_W$K#y$A zk+rT~!y!v8Su}{w25b5$NvS~pp_CNdlg442dJ9OxK z9uB4`W-$2{ibT&NRv;53N#M2YIEc%0(wC#Kzno{ziLL4ZJk#JJfr_h_T(~b)odR(5*0E z*|x$hn+x-{_N$A7Ayf6VEq=Y-V<`4EwEm{E;A0^H!=DCTd%Zfl!)!Ss8ooTJQP3=L zR90&}ZAnUylQ>w3c#rSFB{|}0biAZQEY_j@k&kKkk&cQkpMHCpZA|!+H*XgnUn73R z^jK<2L3b!IgQgRpABrFF@vFB@>x8xlnOPpQejf;Tp>$?s3x#|YNF0jwJ=6}aJ#`2l z?Im>5A)R^pY^UC)rzHO!2f*_>M>3U*mf|z83agD~lJD`2r=*^pA4tD0LiaSsotJs!~FWJvH z{gdTKVCOr$9yIp*{&!CDR~)+CZD&NzCzn>mb^ob-a<&lHhKl0DoX#5UWDYl#xhBvt zkJ3b8aWw1AufMMplQtJ`p^44{Q@>uyy!aQbUwcMOv-S5AQC})q>A%y02<_Nsj8(96 z0q3VJiQjHS^=^wJd0&n5bD`cs)R#vH<%lEvU%g_JAmiy=3%H5W=0z8}SPB|GU#Ojb zkA~jgS_}P3PN7m?bDh@i=HT6`cIxo^MM77hH%hsrb0$5MP9d)YuV25|K5q}-cYOow zk+8zLUhkG&P1zrVvO*ro34L=P$oD^tsE#vgG0V@b$)9Jge1%+i$P57gP0kif>=!RPG60UnJ8c?Y_<1C1N8nD{-f@Vt=*ux-d@_M zuev*C8ptciqd!zalDXTxJ!B^ljD?mGD_qt}Y#LY8$;!VN=_HtKa>K_4xxcL4lQA-uwkhMLij zWob5(#F*p-jE~#{csc5B3V^Xh@Y;=?Zk2!ea4`>j{IqT)?2D`^E3x{xj7Kl@KjoSu z82~dsY)G*+80^^TLKph`#ddMCM-jq;sON}99n^Z$~ zslz3rM`VzK>O1=WSU?yN0w$+d`G|Qgv7IY1_?l;tdL(1jWz7~Wc7D}cWE0EjaP{}_ z%hDtKIe*JxM==++_;=tJ_s(74j4G@Aw#IWHE!vJDj$a0?Ooab93>C2B`Qz-dVYXFj zMkQ)X$ib2QgPw2093y&PwLAXNZFNmrNuRxc$t6Z zKo0wOmJXTX`iS(7Dou$mA=ngN3>sJXm6w#%phlC!;Ed4Yw_ZP<+6DhYrM$4%upFMuE>C2dp#E<#k9KXlJM3tVF$&|yrZ+}YszBCWn zrj7>|LN}FDY$(J=zZn!pN;h&3=U+r@>M{;o08=C%kRNiFup-lp znBFD;b9SS=Qw4Ad=ltr1KZWp;)zjuZUz2=*_xQ&obZ#UX^0law;9Hv|y8J2EAACBK z`kC9>DGe3Bx+LVJ+3Bzu-HrMM8@4K62VrD9>&ZMbGI$g}<`g}XDJ(j^aqdYK7q>yV z0v{KSUUlj%t5qh5pC?8*`^c|0MvA0c5$UeMBL;X|u89Y%CvE33cgZr?inp4Mj&_yH zCUc(tb7t=GyO$y?2vub~BS6=v2;A&RjmL>Qp|L@=Kdfk@d7Zliw!&B@Ri7e@^vJf^0gsHuAP-Sm}*ImJN35iC%+& z!otyJs#1W{j_K7;rda6*EerfHnV%)VcuGVQd;Cd;@10*hYv-#xTRI+>Ns%0q9hzmKxKw`cAE&`p0v zbGw`B@|eMQsjYP;7G!c#!(-7RmTl7Gr#FB3yf_3D2e1UlG+-y0t1s6MC^QJr{Xm|q zzSIknOYq6b8s@OCVz#8bP=r@2ciGBq4lQMS^LL4BAD zKIz+|lAxPy-AD$dXmN8EtxS4q$m0hMOvX3PC9^co|4o$VtE3>hJX~3yci0@!5O*$2 zeA_pCNH6hHj#|9j$$xRny5k;yha-0sttJe-=`o(4kMAMNwi->B4vah@vM1771jP_jGV zor9n+#&kV|#@~!BuHFJaR})neg{kpLS%o1Ii7C{G^2?{)M=B+}kH`IdHqn}VQ_-%m zs-bAi4dI{bGx~wkqq1=rt6O%HO<+&R#7WD2XR?)l7vIvhLz>gU{DI#_wYtM(Q{bzo zL0^a@YSOj!bu=SJs6FNksd)^V?)D8KXMX^s^E0c*h)<5I$<#wPTFoVAN*Sol>IA4B z6eOk{hR~#OixF^abk4nGu03Kbl$5XH+xwL>Rhhm7kLSouZVwnPs%O_Ws=nH=Q1`XD z+K+hUE$DE8!QYvV3Xck?9fMo_U=#bu?X3S?xp@U-JyaVYTr?chNLW(9bARq~(+IXm zRYTkRAzICHpiurl7V}Ov)Ox#k-+3gG_*e*E1x`8LGbF$Iy6ZM((A;@GrcMYD6}sS} zSfL$ER(~dn5!mjH0fVIF2=(-1JHMcM(!Y4CrXBrlRR&N`L zZE4xaA^73(1$B2MXAIYycs&Wt23>PDJ0yMObJV~$%o&yIB?1J)t@uac_!F{tLrz7d z{V~1JJ!m>uu3R$rPNcwyGlZKXMkew>@?M6J@;MGvhC{nAb}79dYWdvx>9PFPMkiFD%~_c1CxxE#)3A@s65` zpMpn+c$Q(^&*cUQkcOIlZA}l3A|%HvUE_KM z*YT@#g^>>+7;`jx9OCKNlQ%4z%_TMmL)NG5w`7kr3u-3zt84Ud0`n99D}7O8`Ypnz zb6xMzVRUQkLZEnJOa0&g6lB6Q4Gxf_LnVIgWThF)a+u(`U%lzhp^GnilBKC>k1Dbo z)Eqb!x3oE0pGv;F?Y!(tG#|iFfqH!w$T4-7mp8`mRgem{(o_VZY%@8J+$SH+rnmENcBhk4e&Q3oFh zJdfORUXa|M*7TWM6DM7v-Y}&B_ha~$GVGidQ!2Z+mm)$J&66U4#& zL_Z^$ooyw9CSFZ<|Nz)5CR834MZd+9K>Df&SjU=VYlLO>wWU{kuZ&X&z?+H&W>}+@6jimWm(XL#16%o z2wG?BcVh-3*zLgm&5w{XPmL*|UW2F6Hp4yjL+dZxx}k@N`o5-BQc+(%&tzq!I88mr zTYV#|g~!N|Yd$*1&&~O;pH=yfe@^zYo_#%+#~f+!Or!QV;yZU13ehIjN5>rA2I%oF= z*h?EMcVtvXO{3Mlp2bARBUV55#zwkg9kCLk*UhTnQBlwvGGBU*BDfK6W)}8yyz==QIhu4`w@; zSyB15V@L@9xaCF-UpKi?78!NpFt-Y1AyU#-A<<_VF_A)pime-NNX$H>gw`@$*?LQ~HD+InN?7Q|PfRA=`sU zN(Ju8c@N$er&P>_p#=?=)7>8=`+gYZK9%Rohta_8C~D!+4`ceBs14&03*IjQPQK_ zr|fX~OUGW*8c}0hqD_?XFXnM+9cNcDqoy+bkCM_&rA$27Cck?}*(&>|q2^5qPU(!L z&;UZ|k3R`Zv9E!zc*|@mxu@`HhGsJG-d>h!IQc%>oK{5y*J?3G`@=xfzbk#R>VL^q zacFeKY}SbkUtY%CV6TUSu6qZFw0$Tmcl{@lZ=o18QskCSy!~U(+o!BFeT>-PfmhfK zL%{X2?d_#}968DTwX(9ZiYHS%vsh3_7VYm;G(7EA%!lTKkmk1-Iw2=P8wffMVjeX4 zY;aLtHQU7VR)%)n!+OH8iHD0z+aE*7Bx}7Yj`pGpSdP>j$PY}CH@JRMU9XUPk-9+& z3WyEX+PPbHM>|eymsl^?u?`I)55^vn*Gz%VJRuj7~^Jfg)s|SpCob5H`I5N6X z0{@i;0aAsWrczsqYD|>Se!o~BL3+Qr89GAY41Aqph`&85S1w%#dm1x#zuj+5pip?1 zp?qwa?yxL_>Brd!6xn(9JhiMkC-`C2JLSKGIWPYlwAg~a>%w?)CLl`>kOy)vzodp@8h&oA!kO6>49=!HwYc0 zvDy9r=t;kX;o(yWlM`*lg$mvCcVO_UC#lqItqLutS%PAf>6;@VMivx1bjfSwMkc%SQYr#^ve)(oQz&>*3Z6w_fO{ZuDgY(%WGSX5Uc_BUQdv=;~}C$NasFg7+z0&QrZ132R}WwW?|M;E^Z`mJ|4w+mY$;P$f#QmIw*`mxffc0F3KfI%{g5~iTE zU1?kNIJhlZMJ24NDWGeJM%Zp@MWJ-D;7{L&uTceY@N&a*$1^(QTH+>Kg9B}k<``nA z&r0G{a_44$z2$ugKm=8zO)DU zdpk_OEeeN4fbqEG?GdCXJIdf+c)?%+-od6soYv}?Ntr@yhZ1u1otql2@W9Pr_U?1} zH}Sb-U_si2XtBlyB><=6)+lY931Bx645Rq|L@%E2G_-bp9OUnmfxh?0ez;S9fYm2p zXqTK*>U0j8Q+odPA&ZNlYo?=A@(mTtI!LRKOES^K17+2Nb8FW%8MmN7GRPA49mx1E zt7|pvbgeGXP2l|AECrehkc-%bxb!87d~km$qJc9NsR5qs=t1`m*fY3zd8D?FEUJA^ z-k6A&KeTRs3SwE8NTm{gOu?;1f$)@#1+~eTRCS#P6wAtmp%hci=8)*X{XExGfj+A~ zyPbF?@1|G$ba?IGnFfpkGEpN@(FLEN9}O8iJNwIjsB{QB+W9=tePX^2FM*SuwxMa5 zI7{Oe2JdwlNaXESG~v!JgXGzdT~WR_>uTvTeL)ud!P)4e5Z3~{3~DR2JG3U)@3J7? zm}zA;yaV^20g$j)=xQ}jJntC22uCSK>ZOSds-upHnE&;wfqRREB`&4QV-sJx^0HmX z3=_bjWaDB6c_Y>BU6%s_l-uA$SU39a3-?9(A~q8JB0i=NouMfK_~bt6s+$=fsZBax z1l2|{c?W*M!RN}(UdW_FDwHPCIqgnlCI{k3aLZ$>p?<{kJcXOw^@tVh8+N@{nPcpV zbwq`%ZNW7id`6Z{ix+NY+ zcvU2@CyUEZenC_I@w>*I(X+ZB1;RLIaZ=W`ovKJMfpC?3jT$u@hT1=i*U*D1H4dfn zvy|+=pY>Xsx|a4m)t)LlDqn-a@VS2}lwDP2V+li_%R?V`%79`attag-V|c|*CT7Cj zbPdbUzd$%Fp(8P~gw;wzHSmscPjX1pAjFiRiv>tlE=X1|g%>;ZNV{Y<{U(qy(HE=y z{oh;16yb6uq;Af2YYNuos!`o&;U^coRJhEr)7qp$XYRt(s9FsYEQ!&^yv6LwsvoU8 z-Y(e5pt4pn&cVoLWS|@|}-u*_>s+BN zAxotytzt1jR+RdV{kvaszhF@LPCBv>lbO#=PacpuZ+>TXW^YKA6F&Mc>WintCeK|< z(vtR251q&NFGqHFG|@HBqb8$6nZ{ybzn*Keb+Z==sm->i(aB&W>c8AQ61eMP=9>(T zq6l77eD+fnO7<0^S;E6(#cJQ$N3;h+)YjEa{pK-LsfZ;lsI@nK17QEoguE-r5`bmP z6j&QudHQbm`{A1o77t%iH)gGjmTLEd?{#L&bSjs;PiCvZXaYZj(33Gs+VMQ&(WrPq zuhrxGdCshpD3kt$Z>7ey4u@B;^@(iZj+Pda9Iw1T*Q?1gWmG`7gYY+gk3htk5*9}@ zPq%4CA%<%uwlOq3?~gmI3guO;@GdqL!YO-xKPX|*!vngeKdpMC`DXP^$poBP?(Wo) z66FvJ+y#W0Xyogg>a;o>QuMk%`P|&lX*0G>3VPD6w4Pwh?|&1`N)&S}Vl(dwf1W2U z>@=i(=J+iiVBFKoUU`Omk#av{v$!RvJrMZL{VugmzMZL9^|lk;Vw5ZYoVsE5vRW;4 zn&{Q>9By>!xD;8Q5N-=t)MEVC=H)1d z4&NoAcWuG{E!&wmrBLx~s2^e5Znn0fpVB{;FO}9)i93=hvp#@+wa7kogvg=hFHH6I z8^116czqn97uhr~xr|4We}dF-@337IV;;wVKm8ExrCXK9sKU;dnm~+UAt9JE_B-kj z6RQ54L0sf(H377;iUHVdsSFYaXYlBcCIDVcwQwkSD_zJ3$fMX0wLXO?oR=d1HfTIo z&>VA;Rk`GO&^Q)!dD(x5>_%}LHvG3Qu6*p;M|t2Fx)T8=SCE$ljb#h^PBoiW8P|uG z^EWMcnb`(TrNdOKt@Q=Lpbo#j>m&D~YVEi#@Qc3s_ZhtUp$+mR^a9Enhrm>LWkNT~ zxHaJL>M7;7v0N>qYLjCxoJspochsKSvys;SNx>m>cwL7pL+9PSPNn@?olc7qog@Zj ziJ6PX((o8kU_%8&iQw~MIBD^KW0l^YN5vvGpG~ypUiTB`UCr|fUD)@5d)fF3^WGmu zEKakzZzb1zsv0&OQhdyxSPun{&hascjq25g-ecTwoFUqr{%#ms+=K_peAJZRe=c^2 za&ID=5%ZN_$RUthpFcsHAvcS2*IfL2MnCUu*cU#-KQHoIoTL+oR-b?o&34Aub#g>v z=2~AM?_ycxx)X9`r*;Q$UaXM}?(8dcbA2&d52~5+teYnYEGs2S#G#6DB2=I`;buQ%E4DN_XBhBZW?dlyGw%=xw#Vx2ttL^%?W0 ztPD=t{_UtG>*)AbLj}Sc>psj&=mgm+9zK2OR%Pc&o$xu}c<45r$N2HiLFQK(0%ynl z@A2AIujanhH^pK%w0s6kibTIZGK*C2b!(RCTM|xKLvX(*k@lsJ{zSzAAj8i0rTrrp(Y8u1fDSp5hZhOMh0qe8>iNH3_oj z#j9Smt7X5bpc`{D<^(#;zLY0PB2{rqD)5N4?mY$`+=X{SMkrO7EZ()Jwcnpa~4GB9^QlSoAQ*hEhev=h_a~z4RZU?0y;lwNdfe<1?)ehmyLX3aTY#B>`beM=Q zGtTi>S~PoE8-u0SgC(7zmp?U06TUPKzazuUTg&-egal-^UToMis5up+NU`p(x*d(* zJ2$&xK!_fB{2mes#)h5PJ&4&qiva()XSH65Jw|8xP0d-%WR;x9C zO{Hfd+kmR>&qYAQEGK%r_r`3x629|)i!?f0L08|wmP_MBe)5*#rXld>n;g(~)8yr) zLH1Jt!?wY(eS$Vgz7_P6eL}1We?lJMm?)&u^t;f`fh!yOcExXk*&7EaWnG=5fI)56 zredD|vDcp+^b2XIqX;0TnLMcq`%!Q z@~;lq8E`IEDl;T%@>%prvWts3erI)kyox#!KD{2d#rqyEN0e^j^G`VD8(&@Z?EDk; zhnob*m;zFHklS?Y?Z(?i_*9)m%(6*qQ@d;6Yf7LGDx&)BZ`f1D8-ejwSTDovDek3h ztf}AVq&gc~{nlM;JC9aMzwN6q$s@J?;hVy7pd~^q!Y0modCb(U>!ZLcR>Y!2*q^*` zhn$O)A#Q^j;y9vjJj6_|@ln3b`JM1Yyn^q(*=?(yI&_?M$ zT`}PS4DP-*A7y*@#qP1L!_C^2?==51+^GkE34tv^z8P`ikI{m($x%J;ii3 z8ZAd|vrm4s4zPvHh$@qxr6yXY@r`xim%={j;QF@hTP2e$HRc@o5JY;wsnmhBm(YfQ zNMss5GMN`I`BaMT?80Y*6j?O{w?Jmo_5ObL5EzKGw)$B5MKVMYP#4EWEbfXGf8g)u@a*B8MjQKR=DaM#qnnG z1F(4j&9FLRXh2=BQ3+XR$Bc}|LgFSM$$ZMaF`u&l2bEpUg7z2Zb85OPj~WHvaeKgV z9>Mc56M7WYR>sj1OnXrMjVZ%F)_S6CEcVjd0)*iMWBI_~x8St;$J^MdQ+ipaB>kLG z+#v=$QkhwZn*Cdfn(YSV4?LLCq;0<>erQ!v)m$8Q6N^}sTP%|+Hd*xiksquH%(Z3v z`}k_+3$J!%U&ac@KjV3jWtvzD(88;4s!J}GtMMO?|EScH z{CkYfQ0Wa&Oo||i490BaG+U=rj&tlbcV=tHsb&jru%|}Tp%^g>h300PL$vwa9{mnS zx6^OI5{`X!vm+T54jLv9O5eC_VVmVv9SB~>Q92bNDCjZBf>N3+CnM9v+gYH#=4PwybS+Je3000sjtBA{QNa{Y@Vd&*(e8mPc5sb(2G-7_r2q6 zg*Ud!*w5<;a>KOKr;m<7iaITA-g%~>^X9cxBI+Ky;1AbZu0vIwuib~2InB>JbZkBx zs$*ay*d|+abOpcs`(CQT(M z8cp#s*4}1>ES%*9E6bBV<%eD?PJfJNwP)$iI*`}J3Rr~s)t|*A;=Rp5M~^G@7?Kkn|JPa z-yC?`+j1>o!`|F}E zt#mXbJKPhN$DMt5HV4))dPs1BsZrMppagbi{hqklq7#ASgI=cTY9NjoeUIm%SYsnk z=zie&4oO(_`WX;mIlsTl2ihH{g-SnIYxjaDncfV_h&W#%uIrc(B?A4rhaq&{Caw32 z>S&a;6l@WUh?+=_Lu3~z00MU=z1y!dz#H^yw~pt5$A-So7Cf}?Ai2BuqRHTzbKo6n z&*laAR$y!*}%JEk)*lNo0nei65s01%FLy|xDrKX|ybjy%L3C!m{v z`jIuWh6(YYb5cw^73W&)yS60mUqv?`0T^P;Rc47-hIU2XDHY{sG_emTvMS zNTGU{krf@gec(;(#yqNA7GKS3`{QCXuuG z-tIJ4CwMD`!IpOhUC(CP;qDud6DrT1*(N&o#^F$t=`ASr@LVjHUE%G=sp3i-6ZZF~ zvqnlT1?vJYVI4iQP6%}HPKsf8>y4^#l`}n zD&@`#gh|?rqWNk94;xK%QYi21ejnnmzhU3+R~)c5XR%f{jx|MDBjP!|jXs+k5d}ml ziQK(l9#*2iqi^nqRvi|1X>|2(Ob z3M#{m?~COzNhJA{xF4NVBgI&06{$S6L-6|b{L^Q^8lSG$tswcPWcr)0 z$JI}V(TQDbJ!*Qz?GPIRRufRdX&+&R$)7O_WQ;R!`gErF`E}vv*vl+z_okP*{y0ls zHl^SMvW?&qjmZNICE^Rc(tUTft04b5YQcF3s0}Z0v)R+)!HTlcT1V3_YoZuQI8#R^ zsO}O!avJA^-0q@;lJg*bG}{vYAiBWi#C<#(dn&{cP-Xfl(^ILi6dl(QZ;C={G#Vv0 z{5&&=$aeG?RXmGB>2J_TrIDHcA{~BpFZDuilT;k>0KbkNya7BQ1;Y;cI*Gym{q;`! z+}v}lr1s0J4wer43$CdSd3EjjGwoV-zL+>Zix%O0)*zM) zs=00d0fa3FUejWaf!7wUV4j3&qc+#eb<`?$=WgscboUJii&;nZ0#KL)3aA>Pc;<37 z`cxgg47_vxM}>u;;P$w)8i~d9fcNh)K2aLgEJf>=&glMi^f{)fi0T(NRR-2JOJtja0dr_>|hhjG3dB;ghZZo9aGx<=>KTCuBaxyuWJPb1Qh|L zn4cm|5L6HZBoQpARH@R8*pQA?fsjFwBGRi!i3mssDN>{asRBaiMM@y_8Ul$VVKVdO z|M0!!A!}ta>y~rR-Fu&XPW-udgSsm9QuM;ERtK>;Js~ z^6L?(fWxNB`zj~)yPwN_kNU!m;Z+By{fhIhuMN}w(}vs@v#*^;Kd;{8K}r9d2thqr z#q*(ym-SddI4`a3XB|rkV9@zH8E&tbVW{PbS;B+?^QLv*-ZzoGcfyG+9wgUDXZk*G zI0E_lCCCj9*bU*=bu{RmP6JXu%VV7W*(|rrTz}y#arx0*40Cs?&>7^!_@8EJpR4>d zVaBRM2BXG&K0XvcUYD*OgTYQ|gpThr6C!ne|Sxb~u-DZAU zp74q7{is6m3M1YEAD_m51e3l@3E?Sn7e9Tt?`E;$m-0!}LGnp?mN9U=zy8IC5AOwi zcjApbZ3Ecdyd<-|)-1FQ74PvjA3P%OnBOJ(3q{~Sn0)EXcEk+wCU<=C!l=!t9$$+2 zZ@t~%pCjlIdgo~yI%HOXc>o}}a%LpJd4zZPc)yZ@^rN~fQ32ngeT;;GyXW?sL?MM{ z6fJyJ)mL1Zlgp)1kK5k^Tf?8`_T$%3+o8Gt-W_>Wc?7#M$at19pv~Hel1`7L+px<7 zzWoz?#4s55X;s*B2%z7=u}cY4*X#N^6tdH)xygsIy9kbk86qvW^En!b@SMe2(a6kG z*lO1#oDOT3PFRL>P;+N#D_|;oy9-GguzOo7+uFE{5OJXp+KJa<5>=oeK3>T9-!Q;S zv4VDVRI#Yt3I+RnLc;>(NRS3M(8r4@dnkAbWeZls<(AWYHJJa%%&I$>hBKtygYc05 zG!2OjX^%AjaUBWeBla9Wl=OhKkVqn)F-gFrXs#t^8V-x{od{lwI zS}FS%>>Z@aNx1t>rA2ytCXP`gZwRookKKEDsPa0k!T@9P5%IRh-@85iXVu00@S*a~ zH}Eh1B!NKfmknmGLjZR_oWbUso`rT%34Rr3Zmgd5!#*=PLe`A^6usj8hp5QAy9Vc) zn87dWTV@V_=G5Kc#}3QtYi3$Co_yD%7_4vzG1N%-D=9)0lR3Lr&zd1uAH26+qI|qU zVE;souTJPLqM+FL4n5TP&*qIagF8D`&&|UBVlVGw{%}PdZ0wV=1E;SA8Ho^>lIGt6 zOx~g{D`G5m6x|t5^j-P`5#Br_91;3+R>kiZQa9UNbpmHRl1i)>EA3W2-}_yCR4xl{ z4cwd5sK^l_DqT)_{pGyw8}DVWMn+GG%=6HNvr;cUB`)4YaI6G~jTm{Q{xVIb7hI-r z-hE8U?odUNI|F0OlrXE_5HCWob@~B ztGB4I!PD-2P)1e!@r0Lex=PpP{2ye5MaY5!p@TZ`pCjC27-B~_^5SNbVS2UHlps?G zxab8d{oA(i`+KGR2lQ7c^bE-Vj|UZSZSUQ$Ponmi{oYx~HiX;7N_w+s={am$t~jtp zpsfi81gZoM?Sod)%-=i5@mJc;VfiTP3rn0~#3bUy*EUZfnhnZBeV1v#$l#gQpg7k5 zHjs2ByB%+DrB3bbrZ-5_^!EGwL7B|GJxmUCzsFOte*yU;?pM8-ZpKM=r;wDbo_&%| zOQ+%LBwuOZuNRd^KN{>7@)>h~7e^XSs?Quez|;A!H!a+Z?)kCliP6`S z{hSE{oC~7UO8Cp;Nsxu+m3GYg!_t7C8K#{FYjoq@ssYk2FR6#Z;(d4FZxm{Kv~jg| zh*L!IhyvERYABaJF_?`KGmeE=0-Qnx^2VAk_8kxPg?7ve);B#U~LncFhg zaN?ykj|v@S;)jW38uG&1xbV_}T+pr+P0x*Mg}YJO;4QFGb&j3Y`|;!=l+#FI3Wg8` z1&g99jGz{&Xe>MZM4`Ysc*p<@7W$of=sGPwiTWDcAhIsG_R?7pk&?OmGOvzH0|l?= zqF^6JK>TyHZNh1^YbBi}%f_V-Na>{94Tr6!nSG(yZq5#HC)y6WBawXgTnpd5vK34H zu#-;JQVtnj-e0aZU(K!B<4d?6zNWIXOJ6SRMYUc#^R=*CRtnuFo zJzSn;sK)(W5^o7yLq%_idr`clz4zrGHa|Wtc;WH4w6c?*<;@SdtyT-9`Uw?C$x(+y zaqha!GSZhtMzId&Qz5P&(v!pvmjTdV5mC$BKk#zM&KO*JDD~zDM}H()-Y&S;O74yS-)p%LuOS~crCay4$v)o@T4jbHD7S z%_60i==tL5Y+y+NdF%5{$-S$`TQtc|xnD9Ns9m?(RsRlLVqh(>D@xBjeDn5{x{)Wa?O{j2Prr!b`w4jxv+%WY}o8>3z z6c7i_08a4%?2X!HK7Unu5h7s7>SGT#dQRaFr8LHIpTBlyiG7k_pyAv!n{@QnT|v%y zRXbfnhgdmb5#YCPMsJXt-0e>XtZek3?pxZq8osZjdWdMuaf5!rW5>Wndh9>ek#*R~ z*O|UP*d^6UpA#{{Ivsr^bsncTBPHAr;5-rKs*rLcZu_*9cHlOeVwWiHW32EzHgDFy zC4}wHeFjSxs(8PmjA0zQ0tR<}H>X=~{S;1!%`ATNf!d36^gy~~HevrDu14Z%rzQ2S z8Tth|thmy&EYVRg$jLvrfo6}n1Kd9;&4of zzlH7Ox7IB_B#(`jGrhZJjUO5+uX*|nf9+DCrC(>HjN!E0{g$r(dZ;O-3n zz6ZHAX6%PR2Oo@dgQrQ^2IDI1>?P!;<856G#k}4W)5EB1wWlk>o^K|(`e9DiFI20- z%K(q|Qk3@7^8g#sq7E&Mw{T$wchI0N7|vvz)2Gm@4Zd(UN0JY z>)J~6gM4>d$-68^16DA+47^$B_TdjmQH_emlJIaonnD1q+L9Tw$Ijr<;6O6WL_@@9 zZjp~bxQ)o1AJu@u%#RskTM910@Y7!W@!yAyUW%_g9~|^lZRq`TJy8X1Q*5wFoa^L8 zwYzt9JZCX^8=rg_702_wQg0O3Q_v!}hX$%VIdX9M6nEU6u!XTNm#Q%(_nt5PtN=Lr=J>24Uj&da#Wes{@rv`C{1;(^ef1fu` z8?X2(dSFm|`k%ni^sOZ&Jmu*As-ckV+1>$hpysk5PH6Eue@NBMtmSxr#7tpximZYf zp#h+ze;hpi-{-pfuRGCdhi9&3&P2T>-v3cXe(;B)>TUv7H&v%(C8GY=fnxg}k-E_= z*0}0TC(YP^>q-yG!=IYV8#OidOWhDgCI&oiUQBbfdw_5V)OECCVmVA1#DXZ}8PBw{ ztxMm{4(GrRaLu`V%s4z%1yn~$`MUP_ke7cItQy-00e>H^9)G53?#2&)?{;oG6$us} zI}d{*hJ0EfXK&vSrpfGC?(raFyv#8Ox3PO>J}1wqZdYfNyKd$qFAshs@uUmT*xxbcIo<=5XYa9iX0Q5NBD z<1bI@kNhd-b8~MwXCKynf#=(qhl8&|4B}$5KVCgzn38HdGx*Ue<6BLy_`&HF<J8G;hojFbsb?r3 zCL$z3Mms9$&Wg-{5=Jn!UdAgOZTcc$LAU#I&u)pRLt;)`nju2}Uf>;VDk1VBp@44~ zV-zS3t;l2{bMVHcL2<)c0Hc&T{({)7eW-fO=YK+2ycz%ST?CzirY!wbKdygX|4D>6 zSjNrShD*X2dCj8>-WU|Vp%(-QL@ge~uFS=FAz7T`h8GXy;YH>aZ^YfzN)-_ZE0!33 z9ffADg<;2~jDN;CT-gdZ$+?fVWUz|>@hkD$19n;&a~-HUq$A6#>H|6Y*)RvlzI`1d$V4K-kZSnQsd}R?@=@q%L~i%Djt}Qq3jK!{;7Z` zBx-8y{jp2SXvQcIbpO{gg-slzy5t{M$r628+p6Vp0o_E3;*O? z#s|Fl+#%pfs+Gcg3RdVvBJt-Y-y*O=IbqZjg)vkUe38}lOzmY+=>MHId;nV{NE%AXptYcB2I zH%;vqt3xJT$#>bZV=m3zu+PoA8dDQF#aH_f5q?f#h6iZJ&kZ&r8N&bO!&6BrS(^-J02F?LJTTYZ zRQg&3EifEM@1Ef9Rb5{F+PTlJ)TBy)Ps2U*o$PHTPi)vH-Fw2fUTM&@Fb)ipYd?Z! z6n+2l5q2{l==W2EiM54`fr^Vhsn4+y_CF{Q_!)k7uockw zq$D~z{XGzwJ{m%>yLm=SK2?ahujNh+7>@Tp!&9v#$r1uj*nV)$T{VDCq40KDE{>2# zfFL;Ax1lp}44!Gh%|_At`;B&BYg|7Njv8)S3b{iWz3UO648^OQcZ|?sNkPHUDQTTj zdm*>dp4$?a6_B_m4V6ftRX+BhWCf&TS(#kdG=R`dU66u{Libi(YrLF)aB>3`jNU2b zLXcGc1t@kXpgbTc2*mGhx)un}C^1f5c$E<2%glTwe-pz2swKoV1wn_g)- zGxwYBea_#zxiszsesjYjvH0__tlsj^+u64{oM;zo`kNnaa(pKjI}}QU6{XDT0dL*8 zZ>~vP%OhKjy4-8!5KR>dF*T#h>*S~4;7?t~=}@lx7m;>Rop|OGwVyw`JGXV$Eu(6M zXKpW{9Yt~6r`|8xAd~l>%&tC!zxAcn>f6jq_Z;(!HDJ~ak5l_*JX#8*>s*ZXJo$wVqD*YE>}a)jYlTBTPFtBpuP7bLmG#VZ2eFH$B8;U zUB>tCM8u>N?0akb^xgN+QDGqP$?SVl{(cnm`|YOjdBPMaB2;`emT#v%_!#Tyv0%dw zByv`S_2jqr1I_6rOIf$ezZ&N?aH3GQE&JC(4zmxX%c6Cs0!}Rn6N2Ada+0A6hk=Y& ztUS~wwE&o(smqh%bsbKNk>7T$xRoGZ z!-yTa0sERcN*VKP!-EKqrYfq;p35(SgXbdZ(?qV#i7{Hmd6UyQFbU0G0BjOPg=Oy~ z=R{gNj$eRXo9PR7*#H&K+J%RSaB2pfSv`B>y;!X6RvEcW{=RPZB(1#~+o1ygLb3Ls zbpy=$1{yvTMSmGZ^l4lYBkqYb7E(2@`t%am-4&7Q-8k2JI_r-VR?yMxo421j3hrUk zI$-=~AboLX9eMkAf~1nso7UUy(p_hK(vpmtk=JCtD~TAyFh_A5oSWx1k0HdScCt*t zbvfmu$YE#kdke1#bOV{9_Iz*CrxM&4TeBY=A__^BsNRUd{Q`0VwLkJD^{*sc$xd_C zGtRl_Hge{RpXIFL?-A~v7up)D#-+}`d^!&ccMIx2m)r^;4N;28s@P|h_4^F-V6a}5 z^C>8nv%yA3ZE_n8+rCV`FoqxvPw**rlD^{2)uJyA_@!z{tHIu2CXXV!#oyx~5F_oaQo0QUifkwyB{Xnq+ zlRMv;84c!-v+=rTpCSW9fZ?=ppE@LuUA{rZ!8rwX=&dN(n2=k3B^B^gj#p=YXCCEl z>0w~V&B8e`1SeUU8zH-2H6g&_6mvH1J{*FoE?v!T>2G9DI5CQZ6GI(7>BQ`N#!vVMMmAMY0Ud0(U$hNxC6Cr4}bU;#zGuBxE;iS*!Z}i zD-_&v#?t?0#Vs(2qtlQL5=hwhE^61m(}TysOf4Lc9e7N8+c}Y>5J6A+9 zJw>YKtB0+eWpg^=-sut1oUjqBU*MjAS}}jWCj${_jfus&>v>E(7 zcFyIeF8#y4BeC!JWn((CDz=halrhhOU)u7e+JLhQb#i9TphZMX_^+jdWjx2+jMLVR zHl0?H7}$30Ku~8dFaxiximUNl1r4v9dnL|Bbf%r}aZ=kZHf5{Ll+u>Ul6&#%jy?wg z+NgCrG)ovLo+DS2!*H*Il;=>h;^re#2LYunpjYWZ$E!Kir;}P!jh3OXB%DTBQa-gQ zs4CJ3?#RWJHOUJo|GUKUTAoo@7xDjAA;x_5Z7W)G>@G_?fDRU{*pY~o1yMFLCIk03 zKH$;pRbmNI7BrM8Dqvf+-ht%)6-+T)60P=+uGUIUmBbREl4@24ifILmBAAzeonZV< zH3|ZNMFhBRhhhb*qXTH0vOZCz#5bcVjp>C$Ug{u|Sz~18BANjau*@a$lq#N)0?~@$KR$r$P9G|tp>m&)9~KD)J#BnpXq9vOWOA-iN5UeCUz9O^ zCh``4)&1y$F1EyH95afQSjvy#kVTTe$8~<&0jY-SM$719MDe{Vu8Q49hA#dVxARL( zJBIS85zUjsGa1$wCjwh-dW!A`D}C~LtYJb^8&q>Yz2fs0rr`VqMAhcv`fG1J5vh84 zP18`k!C;~9mCvK|`h`S_MbKN)Xwj%oh{yoarJz~j&9Ms&Z!Ma6JVlU8>c~WQX$#NV zeF*=*Vn3)sy6$JMa%qnBC^fW~?xdp%e=1ckm<70>ox1Jq@y6068F}Sz2z(YJG`MdB zgIvaKpTSD*Dp$6n#C?i|S#4+N-Gc`X=L^yJJci3&Nnz<8elLkpz@?AmFH4NmvmS-Y zV4>)m?MbuQw2Dm5jM=;i^V2naVCE3bVB;-5;gEJd4v|dE`Z2;1jL@Xcd__!#?eC3) zjo-y1Y~vqWzdB$%TD@PCqBt6c`_oS!i~}oX@J-40+E*0Mzm#=3^n1dJtG1l_9r=RP zr%b=L-HSh2y31SdTXqK_m_Cc!Q!ur{QY&q6JkMJIK4iexVjZ8hNTxLfMv3B;s7* znlW>yVE1-K2x`>u?0c4HF=woTduxTxy&9;Mz2d(kXB=Xlcde0H zsBE;QAtYvSy!+ALH>jzDy*q0m%Q6S{`TB?C2wjrBdi2+NXq)U(D~@;z**`&ozPAF731(Eq?TjWj4`)p6+KFA zbSFV3=-7{%R|!#GZ(?rmcT$~AojWBDW&tB%@sJ>A_c*dE2`M874&Y`js;NGIxS=a{ zR2k?6=GgH!U!Sz*#$PwCaHZz1J|~_*1}&l#O-FI>9wIWSnv98$+=?{mJr`;U{gyqI z5Q0mU&H_S6At>huKwwZK1#!`yi{@JH86e3@W-f=88F?KyKTTEWmxTYk;S3 zJepsxmabnwTzFx|=_1CyRl3bdfVd=DBT=(k=^T*E)i z>QSH?jK2JS0>%B|^NHG*xk2YGY*ZG9yTc4VjO91Xpq*?4-UGAIV4-vV!{2j`wDNYT zS}#bmV#kF0V|@hk)XPj$!Q!*(jjv|WYqBG#s8@YX)w&c~OzlFNQiJiJJw zSw(57{}ZF6CJZ7bzIoXS!WU<~T6e!JCO3fm^~&jLFVNm~TaKRlM^rw2eSizOyoF#@ z^m1u9-}&K}S@dMwej(vIew*+kAiv@t&?nesQhnG(L48Ye4N~1k?-8mP=G#sMDl?_5 zTo4e#=3dnAD|n7MjGyqK!m;rGn3}q7-QY=^}c1EQsRx}@;12V(T|Y( zU+sOJx=>3hy-{R_quf&N=(g8_#j(i-??v-2r%e>qK78?udhH=1i)w4Y|8n7SS2TIY zO5y99mYYw|3#2>fiGr}vg}brID*r}Z!nS(mRGliFcdz|oSy(oyd3wh52Rf&u$qSAS0H|IW!bTtUj5 zUFCT@wO1Bhtv;`FB^a!3_mNTkFi5FY?5c3Tfo$bCfD$+4ZLz@VaL4}`LZ|IR6Adayw!+z(#DMiqzp++RHr zJ9|_ml1CJoiTkaSy@vl@`fR4sM1>hm=iV!!S!|K%^c^kS00RHf>mP-@cUN|3RAocS8m$WFqKQ7l=k+$=td z4xOz9@PD}x0TF5A{Adl}EXl_ay732pO_s*s<`gMa*upKjPTnBBOe5XMyDB6W)&`bUdtTjuDx~GmwsGJ$&(C@y&C$&8O$9~$gZH9N^`1pO z(=nr0!}G%zWx)01W}cCpa}rNFCrL^LGK$Z3$NJsLzx25a1pZ!!%M>Ss8_|H-gY6Vk z`GOzvxvNFd3Vp)5wwhc>8sowK$Aww30QqfZXr7wM>^q=naO>b0y>z2)1SpQSNw_+( zu1->q2{V{oMxB4u97w~F2!;6iPtLM1())?HDNn;OyxInI(EUdCc=FjM1-QRU59q>} zh+{q@J)bWuqg#JQUYCRFs3*>x{#89zO;-TfmWqLa#aPQ@x4Qb;58@XP z@B53`8mqZzi0CnD7QjoHc!mh5YA?N#%)kCUU7!EY!9REbyzp0Bww|wq+YkFaS9k6s z?bdc`MSlW7f>T7_=6upMS3b6MC^)#8;wsztaIP~0rLxAUlUwT==^EdaGw9WGl6a+( zW-z;!;#`!7vZLs%mNNROsSgFQH9A+6c1W7Xx1)IO$cIQj$u6Yv-JALTuz1TkUDH^) z?1S;F%C&Da(_&)L$Oxb9OS?1JXRBBWf`16KW>%yJQ&K?tF_jw<-;pbY5;puq(pxw!Bq(qc> z(NI7B0nzK5@JqK1hl-cPi=|juSEj?XOM{U2waMd#S$FHV?oSRI+Bd&=OLmij_9$zA z-G2d*XdY%wlN}xr1#Z+__c}1ty>66H6mv1>mdi-x3Gb#}+~;)Q?|7-;N=V^~-9YZ> z3e%)>KJ;uct%IZY>$vx9a)5t&6z*4N`HlDPTffS+Cht!$6#f)yEPZP%upG5(MbO4y zpnJ4a6M-b;oxmM2{8-iY@3ETqwlw5+X;qGaRaBn?35~rzq0zQk9glfOOz^M4!z`?Ne8_&C>Ti{ot(cD9F zQ|!gb;*{Z~h^Tbd2PR(_kyibKZ|${(ifN`VqCex)&UD|G3>ev&``oeHk;X6o^yM|l)7uvkUEh&d^p&2fWk-d4?}Ad`addBa8qW@AIH}oFv{@7 zoIvIff+=@`A5N)(pCgDu(|+mQlJUoSTU;9{>BiFwGgV#vgQe4sjPBr;AZxg)aKifj z!uF}qg-sVVIlb-|J&s(DMFMD67X~AFaIEB~6BnKZIZRW=%R)U(&5wu?qjKBDk;8#s zXW!)LO+^`G9fr&x|1EA?A@~F_FoP;no(`-Wlx^48R><(D8)xnigYX~WV0Lntk#pi@ zA=HLm%&0cvs3{{KTvOnE_4Agqc?*K3nO6cFtIynQ$M*qu!2tyJ#Yp2I!`Qd7^ku_u z)b5OU$2<4{0{Jf&t3FrQTVQ{&geKE&Y>?mB(^v-r^Ju2IIr}oR6Uj_~O`ut)(m)n3 zo0t}?|Lo!s96wnLwA9bl~^^!3+^t^W-qQLx@`{v39K z`8{d>?Y?u^_}Jzx?!!X-J)puN28m6}}l%=XEp5o~~m%3xdHVF5L)RcW>W4pYO^7}oI7 zt$J^z_bIKuiO|9I^&I-_OtRL4!VBD*xWWj@^8#Azy1FyJ6}D2}`N@%!TxW42OLvYx z_sLUTEgk*K_=YJam_c`xB9iF=N2VHwpN%>XNr~HRCmVWB2crYUfWc?~)UYeYStb3+ zkT`a;I|RGGp*kWqi5Fr8a(-Ry#tWIMpPL|89mEZaKJEd!B0}kHZ~vzOdiQ+a zPSYP(ZMc=Qx)&YFsNS~Qo;`r<1I|a zaXkH&fy;H>Xays|1=l9VZgBOD`7BZBeAgND{S9YcJjd4ixZF)*0}p%$s~=*(VOPJq zrr5Agl2L7)ajzhmB6xEl028K#b9g4Zu6rlPsn5#)&^1}p{`aT?LP}IA^N&2{RSfan zLdk>0ue#SZ1)j<&d8O#9H+qBwJ$d4ylpm$m7%YnkSHn3B?0rR&Mj{^d{c@)H|2P2@ z6R=P);;AA_T|)Kt&7{C6mOEIe$}>v0L3Er@ZAuAsA-^#~LgVdd~)MRfvyD7hKwsP@r?eE5!v`9is<=VAmhy-vFvn**(UJN~V~; zO5gg8*o$Jit?s9yrNw_&_IoWwy9!~Kj1%&j^hcMDT;9}EqDzmcsbyd=~cg}I*4@*M3kT1C5W>pdR} zpJl&z*RP-d+v`QfOzS-MUkO1Kk8pAz8j&F{I^->YwqoJT=u(0!kZW!s<7dq7+`+Oj zKa7Qo5;MBFc&)|3>a3BN7^%9W+l&pl{UrZE(+_+h%wrGwOj0$|M(ON2?Y3UNX8E#O>%l|%lkDRWW z!5!og&06*1!o{ZpmUcrP0bKAlGrQx$6jhBB4c0-yXyOqgfe|(rGdp?vG-EXRLtoYB zHEK|R_?`8X6Y}dKwaUV5zYfRz%i@-$SUAWBcyF!+-?5ZS-g_H;VQbN(c=9Z1*-kc>TWN{> z>9oE)TSWecFDK^$SpT5$NL%lh$Dl`PCf(z2CJ&f;fg6%H%3f2TlzCu4dU5Vqe87Z$ zmYtn+wbQSSC$^_7*38P>95OF8(VHvS@}1fx)ol^ghRJCCSzut;!*->I?{U{n?*+oq=rTPX?QyeEjQ(i9=B*jqGb{5VckI~n zyoxB$fzBj6-}eDNb@X|gSspLMUmeS|Pwd*=XQ7%ZR-w2|)SoBFLud@dP(bck&vyKF z-$n2&F?08oQk){r-kXrtchTsp9+VaVBlkhyd_ow@#kn|o--g4j1`K{iwNobc)@=@{{{I zxQ6jhr!H;1(3odfU@YucaMvD_d4yvkQB!Cnmbe!i-4uT8!`N7+i1(Wpu8Jr5)`q;H zDh)N}kT!5x@AA)>bbO$e0wWnpQ%nwW#S3+i#j9~xkrg$o6Y{+uS-otEBaMS4i^XrP zSD>+*heIrPxOwsf)I^}+Wcy4lWyVGUMdz|hWGROWK4JDXpa)P=q7N7`Y6a&5fz4KyA+kUsi!nz7q*4Nk@dY)VK{J5fK3C!Z8Tr1foF zBAlF|flcX)3}nhJMKQ_-*WTyEpBV0QLL>-LVoVC?tBlMw_N!#UQuf@wo%8@A-*d5n zb%f#oxfSu{u9sfGHiM+wlMHN+C!_y6oxDT-ga}79MDEP}Mz4L+R|cgHq|yUOBO4tV z*q_vB+uTm1i%wb-pn)fcqi?*AnA)EB4i#ZDA2@V#mu9B~tl2pvKvr`;);7Cz%Pn(P z(J%N*f+v|FVk%Y%g+IDcrDhRc5Upr^6yF+8$9? zg1<0l$Wf5);k{afDG1sEC@FTL4{eAm6{7umsl@y)%{d$>h*Q3w{5o;dowC=+2M7Ln zO8oY`|I6+Nk1Iu_zonX{o!op=&pFL!9`fqwFZlMdqSmyl@$Tk4Ultj}sJt`k%FR_y zR`#Zxqw2YTXGK>&Vc*>sd;OKS<#5c4M?!IG4o&~KQS1^!fPUuC4g73%eqNSNh_XXi z1};MktzJqSV9?_EQK25*-Yr(w*|Iz9KNqm~J8ywq@+XAT_I53v*Y8Gq2x(Fq`+sm3 zX=>|hUKW+kDN!&vV(K$G>(c>3j{JZ^ZXvqeziV3&W#Cs)ysHN9 zgc`gA#k$8=G_c&G+}l~Hpl44%yKEnSd(^Y&OG^2S;8mmTE2_cQ3yEF|$W^kmv$6eW z_lrBs6}t=uXn^E~Iw@XMO;avyNL!_p(Q;wbmuESFD6Py(R%N@4sj#{Q2%FOy_|E8h zn>iIMSI4vz3$+!5pctNPy5FCE-$JK6@%y@9gV|{op_e{sn(F=-0-M7CqK2 zeKQfV?!b~z;G@|+#d}Uc0otsOFhwr|uclV#(zOZU`*38zc4=_{Tr!_(JoffTUVk^L z?e_>n1-3UHS(o#=E)U@_izx~iaE?h>VeN!|1_XiP2(H}gk9Tla8c9*+VZ>cvzGX~^ zvM!Vy4!AGRT^?2DkaiJ)EmqK!J@%lVX4n=Yo%8M~Zk5stjI&^KXm)*s-Zp%;F2H~L zsY$nz@;E|?9fgOjjkEj z%#9N2=;eT~prWA`MhV;u+@ufe5CXQfv5)vVycqp-bvrvoLGhfUF*XX?s;;xg7J--? z_OB7}GxtS&e3JYA=dufq_>dOt9L_O`U15@sV9e4$D~4?%&WqLyP!TLoBubvNOW~Ws z68l4Sn)I;B9@iFh#*pC>Fg>~?j{c$+Ch@^X(&ZP?&s9(gRP;h$IgRboerEtT!dPfU6bW1*b2W9oFjHT?978# zE8xeiQ7{{ztb2`6_DCH_vwvbsXzj~A3U%8%iKN}KiR$TiFtA&TLq}H9!hojgnblHNp0K=n^lKps&=?j33yI{-Y`lI=!&j?#yuWLQu;%jP;zu#*H!eIq4kzEhn zAKf6ow#1_BrB|a)UDbQ^Ps%T|#uAnMdjVdfvxc-$O!n-Z!4HD;Cu??l(_;8&4KK$v zn$YA3BXf^WO3V@jBG4`u1GWvYMIB zjf7n3dP}^RGOMHI5~aI`ce*C$=`22prpWLx05|KQzhzi?i1&kL8z7pdx%$yJTH^*# z9ncN@E5!+ol0Pz7AJ9AV{h{fC>m!@8_H<$Se$;4{#I@2lyw`*%YfQZ`OO$^GV1ZFf z)1c@Wl5v2|?=FthkGcjIK8DsBEu7MZ6t%7NBWrHQ-E;Tjnv*H2e3XB5kX>bR?8Lj; z^BPP1DiL!?^MWF&~1b&(MUZ?$3Ws>Ac5h6I8EfIFOZNLUs(5 zXqUm2I=YZe%Ai9VQOhuGUJ^4a3|AB(e|gGH((Z+MLfglH5|dloNX1v1azj^XAgCrZ zZ1L>1#>t+ao}#vvej!Fz$fXz4;5x|UoA4wnxiUQ@n83VCJT{%eJO$9^9}w=ub0@z= z5uL4=dwoaiYka*G&d;BB%0xZ+&oI`(g&{u1Hgh;<=3iezSB!DWk1b6*$F1%{qny>8)4F}@5Js5C1>gLy8ygyRXQTis*N4{JoNda>P>+8ar(W#qenW5AFx04`5dk;In^xt|~4NCsjg#{ryiVD0+? zx>i^x%+AsO-RyAbL3Q8HJ{`M8_zV1mysx(_{4bm(G{e0CEXQAP0p%mC%g~$Ki%w2L zd^YD{^r&8MS-CZr*HPMkg`P_>1+!2ss0`3JzhUS4VrV41uLgy!<$iA_fKEpp&HfJ;fb zn3H9CooQ^Fgs57mPQraCT_!J|M4LyTrDhh7zK%wQ4*)&*HKOG)&|+1*im_i>-(%;B zdMYT&n(eGgL5IVpr0VR!Gg9y=w{V4iRb8y%y4o~9-f9Uw`zBt(Py=O$C2q0v)Gm7> zIFLw&f-XqUG`A$rlhu-?eOjYx|Q z@_h{h_HpoSwUwyP31w>R0KGyxemUE{j2wEp)i{M@2uaCIa#hHxEGj3rFo`O318xXf;Z)*pg(pb zM|o61s0g1fOgrmgwTf!rRPxb_w>);zJtf7Nn)VY^yOJYpN>QP5l+S+ap&3wfC zv=L^yHF97I^`gdy_2+t*$LfzX;+pvGWYXG_nJ}N+v9Hm7t&Mvuev|1e;smzz2-qmLu z5U@($X_rzYMb1^bAvj|upFXJsX=SGx^+#fbBz{Zh0Aoktl}qDkUxjt%#{SbYk`XZe z^C=we^rGkt%*C&1kLnYadHY00hvC0ot^s9_`G8Be6sM=2t^n>Zcy=vRQQkFRJLDYY zO_-3XN|eicMjo(Y{5J$~AX^uT>Hj6kda?4-&W7D|`?%zZ>EZWTu2U76isU{AVlTpv z=Pl=faQNsg8TblAndQ;%MVjjKe7jgI+={_qt$@@Ne6w=-OoZ6y=37rHFfHp>#h8Yov_P zI?H5RUDqnhk}j8j#aBkwUkwaB`%CV{*-f@OJG3?+mmY_rmdxJo>uEWnVroACms37= zJmBqJ?HlWoitDZ^doDHS2oW#tNCo{?ZR2p>HYh;j0jyuM)oyE39H^GoewQ+ zQ}|W*UYQKtzwcoa()vVx0N4gDa>`O(VE?n3sh|hqB8O1?=tX`o)Og9K^aqGZr2k0v z#m1w;uW+Q_e5;tZ?bq9H@o>&0qP71N{w2ei;mnV2J+t}R&;2tPQvdHYjh=)1JaxWG z*B{&TEs^+Hf&=b+a(Xn0u2K7%wVnaxCvFl^wcd*S+BGTPq*!Ox^%(IuA;<%hLJzLC zx*hHJ__IDk)o*RylO%He&BGqaQ%WPN#JOF}8kh>+Z2!~~-OXTeQPwa?{^<)m{TAfD ze~A*35O2hGy~&HUW6aSZRQu><_$sQ|XfhajYOR6wJ@xPJ(b&MeMY|t*NuNkMP=;b* zCGEcMt=;nJllUnu0{ z+QyNRX;!Dwd)QUjMLGH<2;%jNFc3)3K{>aYB!NZdzmIv#AgJcnthcMGYghIJ|ItnN z)@}&?Oam*Us6gd)o!!M$gbae1D`T_jC|ARFu)E$RNrs(91K4aCRuqT+p4bo&u ze4U9+Q`Z;uJ%eM{GA z7AUkEYvO8<(He>8+>l1)O(+0c7Q-dnVOuK!KU91-DN;>|)Vw}@H2ApM+$F!# zRIS4K|Mvp$setCFKm4_KuMIJ#L_i*$%~PUc!!5&dtqh}+J$Z-DOWcwOEXPp<;O+hI zJtb5qO9@j4Pvj~Jt}n3C;r6Yx^{-5xz6$bqv_n^nf(8_U#WD$w}cQ&ky9bZ z99O9%Ig=ch6gihea@ej;h#V4fik0&rCPGeIIn8+vIUg3ooS7NhT-$Z;_qgx--(6#S ze0E*$^XvKk41e%tHb>#Rf!bM{%lO^(L|MnHeqn3V49Asc@^kcy)~0dOof&!WN=+5+ zhz|6O=BdO=Y76*mk4Re}HGl_)A%XQ=rFWJ53OGv0;6YK)Fn1M0w+v8(&o%Sb1hjt* zdl@N2R3wX_^5CuIC%nh}PZv!kvw7L-9?VMno+|NO`zUaLJYz4+F0 zQdf&cI!c$WY1>9$hjQu*@B+B>#k-V}(5ur|uua6(S1;mj%QaXK_H{>ytk4hT&vU;x z%s!?v7ug1tPHva(Avs}-rR5Kcg}dRD!PI(5MHclKu?@&z@0(wkZiCkz*^eXFWWn<3 zB3sSLvo^^jaYH-jyu8H1w6^WE&%_riglh-Cy#esxfBP(^b+CQ~mu5P+#4p1gq);84 z6+}-xaknFxCm6!xdC+;d&FRwzj5j4ZGHTj|VxcQ+ofe^>V-IAb2o3)6MAzN6pA*)z zLb6@{62^lSsKyuUJZKUA_@SI{6hh9J66nM0#TWyBx74S?hKRECRqKW_a*a`|Vp2$$ zphbM;WXS48!0Ki(V-LJ@)Ib-0f{JlA>YP%Bo3JgZ^*iC$NxmuCvomo+M~#cyx31zeGB+-HlL$*U1b{amNsnx?zkF{DD{#m@}e6+ z7mK#!Uu0T8w0y_@XZSn*TO* z^|gNZ%c>`l#~kIUNA-=r@~%cTFO_@@NkPC>aH~;>vX~}^1QDs-ImB=Q zzLzNSY&?k7#o$C*4ZBPYId9OCx^54*th{#~$P8YG)|j4&XxOv=T}a|`MI9hBaS(H4C99WnKlXm5U zxxg$_{_=)zsZ}z*b<=P?YKcOa3L5IU)*Gj%*i%1yrfK&?QFp+ir|zjg^_>O13is}q z2nMTng!A*PSf3vJe(JAv+6iap2oX9-(CV?jG~n-d96G8=Z-x8S_Yvyf1XsSV+?U1N z-cR6X-f>R;Zz}rXqgnIgFSkCvw@G(vSgDv%v`>90Z!@9)UOmCoL!NEve7MUtz9)9k zG+`)VG07xH`gM^+6Z+iaQbAS_1wJDx;1GB<_kU_h{AXfqg2hi54eSl-EziUNdGAo7Zb@qcXs2o)LVRwO{(! zixywaJ0it?FSiC}k3ULKR^$Ray*?nmTQI(H=C^*qIw9V&B~wKX>lEUCj}R8+QgWus z^tYjO-X<=0!A(yv<@e-J-#)v44*27CR35&!&oLe|>kH%yP4i}MHCxZ_>X@Dk_x^yH z<#{^qRuq8)jD;#W>LB9W$Ckl1>@}ZRqu&EW2?MAEuXs1Yt(z}<;u75Iq~i~jUd1O| zg!_AvcMi`#`70ZrJrxutSuzBkf#ur{gl;u2!apBCnuFO7=WCALx$_Y_H=ECYTKC0O zahgr&8AZ@-1+3S`c0y}D?F1n_=D92C#XF|&5hK?$P6_;GW#Y2=*bmKXq?CK*NOs_x zug7t^Usw1{yJn#;br#@7g)az^rA?b#OZxI}da3qgMO#9_qwl05aRg^<$lR8CoSkE2 zO5NjXQDol7o5x}ER)nsaXMy=-qcXZT82Ne#98Jfssm!D=CjBJf^ik{73k4Lw;3Pz$ zZBa^0D(vY6t^OBMb!R_z5XfAz9jFBF3q(8pGS9oy=@)}=OBkYc zy~Ut|5YT7jZfib(o?sN!th402^)Q1DI`hOj*- zd}mhYZgyCqLf0EpOxVu(F8tl2A1|qY5oXBy8;z~;xhP3!CkaE$rw5x;fX&8^rfoqR z1-AO!&Y^BrvH5f%ny&rQ=7;%fX+2l)?NZ9q1=7(^3KlbcPd*zL_R_gh!^4>xo;?h= z6Kh5X2m2#{`?rEEeFlKzfb05$z-BnppU9#w^q52EMZA{|=#@GzZTyyIulmr?3o_L@ z9y{}T{qBO^Cn{G4S@`EoGJL04`OH1WY`ilR`9)IZ!`U>nkrpOK~_pWSBnaPR}j02%rTsoPQB z2vK04SaErxUJ3o!+W#a+oU+b~`<EDgoOJ)yYw-u4SU?=RJ=3HblI}^GA zah!VHOPA9L)p(m#ai=@)ZD--Dm%vDoIKkrMd-qYIG`w{P>S0l*uGRYmBM%{Px5d{Y z^j%`@&Kjr|IbHw>)G!!JVF_kZ+&lkJ4PLm!>V&S zZ1HrqUGXGA=CQIwMYDT7Sj0y@>+c5kDfu*cLPEtLu$W`+criI7G4KX2?H ztbpa#kRxS9+WN&}j^GDcM;RDaM2{uh#=Z>Hj)_oV)og9BEt!e`vC5){z=>gai|ecz z#Bu8*HWwhHHecoYpu~%_TcUUuRQtE|SWt*kk7SE&M*`Szp}lI_5LcVx zBfzwWA$^Q~AMy(%s~pCm#c7CEUwd5RFs(A^+Y7wnQZR~DArEi4Ah}+s`Vn&h2$2|V zTS+lSVV|NJ{J?>sIW*lQi%$7aKwMOV0%^*SUrs3ZxzpBGdk`FS=pB%?o;d2;u)0J zWxxl*BAV0Y)rx2PZmZ9bnS3K7NLKj+;1kSgg@%cV7SI-jE?Km2)`v1{3VXpwsxHd* zr5(}%&PP{kFh`%{kY(w5qNK{A40;Oh?m2q1>~8RxQo$EVRe6Ql!sMY}p|QR&N~QSpvsry9n)i{0qpp3*_0KzrdPntS z-)HblfAAD+;oCnKPROu-&^p-BmybC@*5qzZ$mu*3Enw2{K>{-4(Q~;BR275ntwM+{ z$a+0w^Mp~OcLrXFVSH8#rc7UK=?BseJqvZ{14rfx(v&Tc$^C(iIJ z@A#14X|AZ|9)%<@#?ED#&6@jfJnE*WK%>IyuJ3#j*=1?J7_Hp0IRG6!MG}=sXETn4 z%pPDLxf|98q^|#iJ(Ir-7Tmqs#Y@Tm`M5*dtj=af0J5X(if+Vh%vUeXp~H51u)!Cl zMAgsZf4FKjo=lmgUWyS(jKDQV0w64mRs85OTUhFfzy56bZ7ZgYtW=9;hS2N8En~09 zd}J5gdh1<$s_(4lU)=1li?=^!KZxy;!R8);tP39&lgzyTB0hb=^SB@sZQ4M=z58J_ z(2Py65zxH@uD@4oeUt&Zl>+xvspy!|UAuZfS!ruR3myP1v4JYO+TaR0_K~b$30}9` zHyXa_cCLOl<(CD-0Nzl!1K4TvP1EuE5!$2ZYz$e*xcvsMK7Hi( zku_`TybCOxxmv^U{1;F3o2g1e{1RLE9;CBCE`y%TYRGL3RMV9d=LSE+ZcJ2;^+Td= zAI8C76xix4sGI?KH`xC#_HoS+M>sq4Rb4-2SHG3vlgRkNX+nnFS-=c|T~^sgudQ-C zX#w4XQWC7#OE}J^MBkM?b$v@6N7@Rr*$rXGvL7eMLVf8$2Pk(STnKo(D%v2>GhxZc zT`#y&Du=BpPZs>12CE_9(qM^+nOanz(Q>{0k7}T+@-sjhOoJl2ae1%P{{b$6TG3Iq z)0fyo^16@4E2>XuL9=yvzfL<#atJ@!* zeO7pD9dH~bWSfG*LMszj0OjTWlX9vQ>Ze9d^fvr!(?YmWRr@@($Rom< z((q@a4ExKKT0l^%FETk(2FcPag+{2PLo4KUNEhOxhz^TF4bNCH>`Z{iBL?vb`8o3s zwv9#&r+7;H{J5%^v4W8+vaI9TNKQeEM)oRpa;AvXQu1<-U|@#5m?x~T>)>){6eqkP`7gfmy#D)oL z%*HZAwF6Yg=BqI(MRU*_LeQ!tO89_AX2FUlr%Kc6Tl4H{_b}g`NE;EUJNPAwYTm%1 z)#v#K%LW@$X_-1@@VQK1++gHN&7M=q8e6?cUy@jDq=1Qq(q3eC<3_o~OQi4km!cH(wUkKW?64L%!xTg8rBkK`h zR86N2Hxe=yE6e9a^;5HuPN&%oOcz{(mXgTJ#QkiY^+i6YV&NZtx_XPe_mTSAnr>Nt zMfPrH$ct#Gr9m!YNiu*^upR>^CRor<>84R&oX&v;bAk(v*PJSalNk+1XU!6QGJNDT zDHwjWxGMISs`W3pomiOd2|b$@E+BV7Med?r87CCD@*nZe`5CGW4IQ$Q-fRw_dMK%T z=VnVz4M{yyW9P8l6#yF%yEJD!TpKXCG3{F@1`c8*W1$!(wO1&ok`(x1cEP!+ls? zcmTTx?yj^@)6eNTrF=kO_X_&UW+H)izrM3C0HXI%@@$3Gd~mf^0)CGPzXooJ{W_vH zs&Tr7(EAlwpL6&J58A?Sc&glHc^qQ7~7eYfM4d)@_6Hr;$*|28Z+FB~26QXz! zrOG(X6&uvQA_eTAhaPSb{-$>u_ZZxv7e5|gkYvYBg=jv!8seAv5by+jG5}AM#>kd9 zU4auB>>9QD`aPg0nf^T&cA34p4XVeSzkC?_wx32T#-h$F5RFl(+q+K7?Laz_Gq;i} zapuyNeOQPzaAe?d3G`85VFk8|W(29=?y>d1D2wt^U3Q_a7T-X48uB=F+jky%yEPVD zs=k7DX3zhm>!oP)#2?cwH7ygz&3#;B2Wzq@(N|Do@!|P2u2DqJv9Jl}_vJ;_S z*eMvUurGYN5tTJLTs*I}tUW^Xcj7l;1V>}0R%*IGsj$lkj>IP7Zq17}U(IXa!BfsR z`=`Orwbk!2KUOcG zIkBRRQSz)DPQo$gGRm*Rny%70@`U9z0#2O7^_*N_Yc7y0nxq)@Ixjg1$sA{f&i-fE z6kS>}Q#pe1B!zv@A$`e4INbmQNY*f6uMD6p zcX-!yD>_13U_YdBv2#&J^4gh&Gl(C}Pd2-LqM?@U8L+G$IzfzIjk>m%SmtM5`pVR2a}`s*#y1_%HroLL zKzgF-^McL^wOFo68Mij_5bnRX4S~6950_q9O7*pT!Flz*v1f!D8@uK50lzXs6((8Q z)XvEouC)uu16^ynK6jL5qH8wY0{l{hpzjA`_MAS~hhxaOGWI1FSz^6e=!(T}%92+T zVlCt0f!fL0YBZzDDz?m33LLkgq587Jn{TOKkMT>PhMCGGR#uZInQ-jV$)>`?C9ekU z0l$gOgxt&707Zz?pS!X=kyR(~G!%c$iF;oBBhzfZIpaUEuoru#6z&GQOWSUY{im;< zyf4U59KE|N`h4IgOk13`8>4X=@ z+5NaSsr$RIowdx0d|*>EAfvrfRGdwNFH0xRIHB3;qJu|B=Rb}JLn61AaPlY^u>86ATD?d@{3Qle2d@)x7z@8x;T!Cr zjDjJzUf3xuKM6~YOyacCFKUmP(yZ>~=IR;5>oR_-%E2p(Tf#dAeo;OW|6czlND04i zX!{h!bMdCf>k4wJQTypW{rSkj!8$I1lAm?%tJs@QcUwyB72&(z`?cpI9obPh8`5@L zS^p8$3)p+$zzmyyOe}2(e0ujZ5DH?~La`|(Pfn@xMou*{UV~oq0hUYA!+WIN9|b-( zTOVU^g115qg_T66a<(ytTjBGv_N|KlnS$xk-p%R+zE<2qN)_awiOLglWr*GtW$WSs z*;iuTe#`%x&-k&hE1iWBxof|G*vLu|>Qjck9e6S?ZHrz^9+$dhi!jVDCa@yyhU!25 z^{sm`GSp#m+0E|UN|LnJoxbX}4H7KO0!ST(t3lCr_KeP(Vcou-tlgFk7g(6`1tC=} zU#+hg_uMaEsJ_Yi;XiW4E8rn}!z6<4O1~WZJ2@o{DlxD}J1C%@JBw8(#l&R>*Iv(S zd{(*aiX31?itFl6?@bpxc=<}Lj5y2vT4S~5JKPOyt-T3~1f)2^rTyGExzyh1?qo5; z69{u`K-~{x_{Ji-_w#pSCVSTHQZL$%5Ue9BLk{IXHMRrw{8l>fdfNd3C{FRBk<#U- zW^!i0uAD+JleZH4csQ;vY1dl}XR5=HOh)W6HPds^-W2;0gn*9WMeNsEEx1=EOo;NK zgmcrcjkvSUUX~~`t_);v<-B`#VPpPbDk6WfPqO#x&zmnUvbxe9$-EM;XKnKna35Ro zo{sPh96LF(g^F<6A#D7;9PWuKc*5Y@7sHC$=;@YuRLubI^zDM>@;Ju@<}O9rm&W@k z!w&UenXg|kSOmYA3F0EmKA|(S^mgVFNzB^cgTv}^zhapj*(Tpw0jtHIDIfpw+rDp& zxG&-ZhCN01t1RF395(UJt9qQjYFJx^?>V1t+hR0hqxQ#dr1VAc6Ur*Ryrc}jb=aoy z1pWCz{Gi78u=(pzbLv*AsIOAbox0B8JnsU$-Ulvz?*+qN|B17 z$G7Jt?8x(j5_z)cIBa|xm9|;(WudB+Qlc|-9F>oYaCfY#%rosjL;LnF;m?6)>knn7 zo*knq(*p(b{r`d>#nCe8DKW}z5M5T}wX$j@RKJ;F;JT?Q4`NlAgr|s)$IGdh%oqhl z|2=g*W~8XvZDcfbpJj=B`C~-=l3|qp*|793A2?p;ZR_A{hT;=IoY$0(kN ziT!iWo+ojCsJn;r@SR59aKUrNjt?!r9UWtDr+<06k$)w-%>fAa*}C) z)7^4Ql2dI45?G-N_<71R&(aO7Tk`q$t-KZ=5u>-@T%mMUbZ zJ!7@xvQak^0WTP#3%}Jr+)!CM#jBS&9!Io&QXn&0StZSus3_uW7$@YnD`@PL`q|kl zPTX{rW?Owzo5Glu{QUZ4$t0}TTFAWP^^d^T;*V`uz|6&zYqNNKJ6iHe(^o$Hg#_MP z{3b6I#FC#XU2-wU_C-d#?(KR7SO1axiFx2shS<-`Zf_8w&T^2Jsq+kMw&tjE`sMUE zW?_cUjj4-niMDS!GTk&-UB<1v&rqoI>Fq;Hw;|Y6jYt{kIIHVK_@Is>P&`T_Ag+vGbsZcJFaO>O_5sq#gyY!|9hyVDy#{{_nrm~dd+XN%Ae zy$yN*ht1k-C~@+>rb>&is2mtCTgAF?`G0@K5z2JvzF_j9ECc0mLkH85T_xorL02+% zlkrklu+JBWI?qSWOPk95T)KSag3aO9cO%>jmRG!-nw9S+HslyN56u+*YZ&NtjrFpU zSB+~p){D-hyI1g|=gsBrT@!zg2>&N7??qN5w;6&Wc-7ks?YW(&q{A@(IQnM6c%iAn zYKGn(MPiKMRulCx%kzk`!#{Zu*3c1dw(zfJ!kwon{LBdviCSlr8qaAB9Ae4#UHC>iVwgU^ZSf+50ddWd!}TH=H8(GN*Nl%?+SK^&qMPFL4#X?ElX}4R zqIGm&FK;C(bl=_^WGY~vqH$qB1+-ZJ5uhI7Drr*Hjc zeTjxWCj)dn`Zs#hl0B9P``I(Y46-{GcG&?Mc8OlV$X@+<1=_PUBS&_6IVsUry6MLw z!=4{POU2axdXVLPE%(UW@H0ge@_x33N7Y#h%gnq_eJe$|$aohOiw7B!<6`6Z$+!T8*nzi}kDv#J>L>h;CE=L%nr!cTLU_qYdv zVe|W)XCDfkW+Z$1@cqWwWoBhMQT-K|p(AKZ&oHCA)F{qK&9PzTX?8o#@S<6@H`7XN zyz^Z29S?@FO!Z#40gd`76H5NvQ0IsYtQ(`4;?efG)g+Bx@T8b1{ zhAY}nZ(@FE$e7@pO+Yk0!1v-GN?2rlxr@6n++__F1uBf1pih@ajQ z*X&=}Q?!q~omhsuchz2OYsZn|L9WUuOa(?goujg>uU3?DI>((ma~g3F*R9pl1t8kk3$oX{Mn`+ej+VM)Q{X*$jTr z6DlM26?Y6FE4D^Ia;gRPQLO)P0s#@yqDJo|Fb5QYFyFB%f#}m;o(`T1X4T{o@LbOQsH7{oD-FAhFLR9Ljn;E> zkCkRBR~JE=vHaimcO9|Pc0L@SAT{Bb=AaVS2J4P|SGqjKm-pO` zX>Gr0nz@7OQUJL((T+?5SEjjG)RKWmOSLr+tqon|DD9uywPMwx4P{I%HFxQ6jDm)q zYc*r-X~ewv$$Ywp5*G^Bk@TECdu?{jYNuK4^N{C-hDc6txc^QSM|6z76}=ACgZ_Sd zng3U!6)ThwC6_#NcY=($g6q$H;$b;@5e{1*=2DLj!|z?A)aiqt`f_fuxBOCp*rsNt zW(l_Q&ds}}5XU#&6{#PFe+fj+`E8Y>+Jf#~qAfdAk(i77hv6BEUp8x;&=IeLjd?9m zE(Hj<#pH-@s#kVMDqt;BZNEG?4Vp%`5QYTyhSt1KoJ~s(QPrusddslXZ}g8{#zhC? zeFM+b)i;L+U&Vi;os+MwsgU-x?NTXnpoZWHveV}18T-N0xIeo}^i6KGw3Wr%8^nkw zXq(;e1m|NDN}9fw@mEO~htC53d%!QHXT$qCafbZdt8>}_;$LmROZm#R89YyrBP ziGp358%*s3Y|S=a>o?C<0k7jMZIo}ay^@8w_QpX&k};n+@TmaeJNi2dg>DF zopK>4L|+Q|z%JIEGd0b{nR7fAZ&G*;>{c`pq zR%V460DVtysIWf$6?pyZ<$J{Am`|>Mj&Od$9yEuO0RK+i|8=JtF=G0wE@MyS z2i`5bh0<=>NCgC`p_~ybMX7wlRV7FrpT5L-7#$VnIu@_;QC_@93?@nV|6*s7Ip7wW zFV?oX=(>Du@OAunFR{2i>z3w8qqfgyw|HaR9N*Zl&xhLi6$^L<#MAMnq2DPb`he<1 z`p*H&i{MwnCI@3EAV^nFqZSxkQTeKdigp7_#nW4zUvm;f<};5La^L=&rL%m;BZCq+ zx{p)s$KGeEp`JVb3u0G1;d1T+Vlk=y$fu{b{s;GPw-C6E^eF!sz`RG0P000nx)peh*sdP0p>EI>PMx(ntiB8V z%N7Q~<%djnT!K}$Fzn6S+#&;KN>EQWUpXAmt1X!~V6tPh?SnJ6?pmabP1 zCTbmravztlJ;4^SR`ygeAY8~XbRT$=(LX1Mnse06#H9i5Hii#lp&; zpdw-}^HP1k2$yj3g?=|1wj!jsSM0JKR8Y-e&L}~8ses6Jk-`|Oqfy8zaTad&gU8{s z1$@*EInE=Bn3Az)5F}6uZ|?|`g(RczVtq1+Joq8zxA9BEkQ&a6v*RXo+uW@(jlom& z(p$W40OC1?lP(YUs{a@ml-C@amU4xbuI>2d>6H6(er_<7zcM$~>Fy)Umf+Ee{Uv6`&e4?c(WQ4#$L8 zVpiuG!oVp@^}Fv=-^hU#(P~xV|KYUc$Z4bPRk=n4lp+nn(gXJV-js4!P!!B7>Tn4C+ zFB*Qlu};GfGCK~re2RCQH7XThyx*GC z@btC_isid%lbXHpJMzT!NkWAscpScW>pMmIWSr!;Kw(hb!7VES!Eww8aAmz)iY82R z!#RqZGQTqrg>a@r6i|aHHem+5tLA?@9=O+M8O@DUa3^;C8rWv#Ih262lRiBu$){;fVXxG({Rh6wR`oo_I-1NE0D&vHM^ib0U+>kgx}jh>?sI0 z!1VZw$mHHzPsf$L<8yth+l`~rg8zr^Y2r8@H=(0rR{0Fq7nP{5bH$7T3~{@onOUnk zg=AGcAj;XPzwltGFTi@wE)Y3k;R^Ac@1(0QI6PvpAZ}uKm0`OpH#<~QOR4wzW%sta z8<}Z}*$xoP=iSrOGSA@ET^m7oR39*Z^B-JcGj5Dhg7L7XySSz^g$Amfm1ni1y}Ii& zwxV!uDow+gJ*{^P)&-#HdUC!kZhI!V$qAh~kjqihTN)$3-?zW+23S7Gc{Z1=R;VOO z@t$L-FMCWggc?Xo$h<{mFmzb3KH~%L_I`%OrXh@1qr4yQ91e`QSxTpRn`v(&NH$cb zb(76tZ!_Uwd7>-A{vG^rC4cjiu(RVz8QmophYmw$Sp}*;tl2q+RkMQ5OM?^x$X)E< z7bMb!8+$=sn6+Fr$wTRSKBr*rD*97PB2y)AqhvnXK8R{F_(`?dNTs;Dw;%eJ>Rj65$mK;et7_^9CaNdeLoz?3F7F=(2${IS5!6=y+QAvazOFs{J zk9hKQVUg`_UTqG^jXk)YY&3Q1vq~i0yRK*;*R*(xiz+a+;?ygpJ64X@44WSA6&;2L@_3gQowqD2=;MCKN!7yx z_A5MbjSq^cb^R`;z0;d0>qR}85WExjz_aWMUc|pF{yk^6WLi!e_)3;J^j(T#{ZPK^ zP8G&6OJ2z8vVRI$LqLJC%5WpXFTAJys9RJaKzr=801?I*qcR@xm_iv8!!gE2LQeg; z>XysD54eK0ZpWi_PId)NmVCWeWS)s=({d9P65y<&%f1lXR^l@~kP3@a>I}c#Bn;kA z;y2!m_-oTzj71D`?B`wj5BYwhRUjsH(WDfY*hNy3H zt?PZpobjH8tyDH?#O*)K?uKLEVD#F58D3A#a9jUo;Z?{wrtJ$!B; zM*lTBAWfHNdt^rxYy)5Qeg)vQu}NoJZEoY4U9!-ArBHcG6`CM)jy!~H@CplulYz+| z=G$rk#HCw&DcqprN0Cb+uqC9|GUjk`?pmKT`Sxf#kZ^e7whC~~WQD)CS{8u^b6$ns zryq#9u!J6u95AqoCw+LjN3ZICHv- z2-nT79MQ9Q8xux6pIdywE~Q?Why=ySd>3Y}GnRA$j@36R*f2~DPlX_W_~8^JsHMic z<(5VCTfa2<0Jp#hhT-<;rTudPYAYlLSlbPL1`m37`}(8SNgB+}x;)dL-H~MJ!!7|EbWk7nXs3Z-%{YwkZ7dR8CyjJGs0@*|nkVaoS*3tK&F4)|Xf+S& zjlE~3np;j|lDHqvxv2$YXu{nN#J-3@e;WJxUaU-U1{Z~{-r5*{Bz}LYzO7&q+@**{ zWy6aD&U-IaVsM{imVS0Zi=V-txt~qjEDsng&$;#P1tuV6vnia<@sEMM?cMz8qncSb zH+o@S;#WMvNU+te%mvx`>P-3I`tQ{}axhfAfmYcUEJpfZt`mm@??$D^eJ5bq7uA#- z7aL4J!R3*tv0);`(58)Nc5~i0dMO#CxE#J))x8|16ICuu$dDhhI{B{S;Va?9jUWB( z81}!n6o12|cZhH{>U-AB(L;3k1Fn{@xKg_|M~?p3ezF;;N61~yYaiZa@-n(kxv;pp z6}Dz!Qdh%?+XD{XHgx69{C#}_dbaHeBi_iunuagGy?TIfUnOmef5%d57SPI$<4+St zHp5K|uHJ<<$?3 z)g-9)1ho8v$ASF{gkwyf7VMQ0o}+r1-JmtQoIW%=07jk}p1T8m5;GAXwguVF6ZBB_ zu*Ydz01PMjx$-%O2Xe2+1NAlR)H4{aV7nuRbD&Eqn2q$WJG7t^WwDeY{+DFV*<~Na zk6w>u7|ic6>qQX#9e(pxG?$EjwN6X9#{{0ZVaL%yOrCd0$l>l|WK6#P*+$F=UXb%I z3rTwyRA)x<80%D&=)>)_5&U9HNj!-)MA)P2(jOb}oO)qscv3e|+*E%N4(zW7LVSWnTBJNiNx;RIB&5vu>XDx~2qc zotIh2aygDT@+ZZ8(%f1758rLN#yY1$g6=P%NF)*0L_8KuOHrZi=d?ajv9TOosw`Nj z%2epqtEQArWJ9iD33+L-rTBoe|0a9|PVm}IT^&lf)cY?oxYBEy^q7m7_4VRE`g^u} z_F*N{(-G{g0cPY(xd$Oo+2fB}+%wpHdIzlG9z7F&{cE)T36<~T2}WaF2St%@&=>3G zf_Cf7q&mK+ich|I6=fIaJQU~95KnA>ritubM0bzjHz**Hfu8AOAeu~>+P|N4dEBwv+c%sZh%^jBisaY97j>lkJGT}ZW z5d`PY;%M5+cqJEySnVrT#dF2Dt=#J(X^g)d3)sn{+d^wZL|oK&j7qb)-hPd&F8bpE zR=1|j#wgzNEcr1%Oi+t6izgpulQm~*Pbx_kZ9<5icj5*_MA3;+aPJiEYo6>%y&1k- zC0bx^e6IWB!Qb1;^iiGnaCLsF(QJuvzh3mlU342C*(EIH%OLq4TL$|(9g+=8F41G8 z;kt@{-301TrsI#CMS8oN>71c2;hiLO97c%+#KxEK)6bu4aX~&)r^{=Aj&V=Ibd|8G z>J#VSm)LqJ!U~b$QiRZ@Yh(G;-fK*#p;0Wa`ta04U4L`KJ_kJorAweEt;Xt6;5WEQS;3K5g6aM* z|6tb|7v8Z4Y(g9&MV9YZ%TN?xVI`UTSU!xH{&xRey2 zWL@lymu)ptigI751*89*|L13=KJ?BNxb2>iB$fAqwebkP6(ktqbYFzg4mMzL!k71$ z8L^2&eRphkd@mrt-nuFbZt;Ri({X#fjgzw6!qlrqHTL{*+Z$sWcQ=DRN` zKu-GsahFAX&i?D;PJ|~a;8OS8dYaNz8~hj(`|2#SC3D_}-mf@sgSr(t>Um#ZQut|OP zAB>;wM`hr{!vjVCl0&V;N(nKhtzR3P5S;mKswtxzN2yZH8sLUUf^RM7^-C7uBx#u* zSerg9Jp^sJF+QU{n^F65v!&XTwxS(R&q; z<1;|5uPDAL=fSWh?xK<4ZI}~b-FKHV!BmQmrR0GeE2e`Tm5UKI%-o^RhSkA~i&mdO zMK|J}L%ocCmQZj=Av~&Uk)8JdU@Vm9v*%!1EB1d4ubXHxKgH+M4Q$MAhayD_)ep;2 zc6;4V{Y)w``%l)nQJj9|aKRMM#&x1T?`@l7Ny z^$;TE%6q*s%tG-Ilvmw1vt{k0{7^u3hQB+lv%cX3@=eZGTf8yVG^_BL0Fi!Kd3J#r z*=nj$GSq@2RRs9|ZTlq}QE+VVN#@P91>Asij}1d#w8k{ZFE8S^ol|4A-jAr92M0OO z-EP>&Z@33b$_#&{eLU%(1OS12N!_HGa~~$KllToilm?k|n#z+p*aZ%J)lGcC!9-L!q9zY>5jf3ls7Ag=4VAx@=hXpZKgxiw5r7{E9Zdx|Hc|b4m%7 zBB5W^cv0jZAN&1*y1D{O*1KNE8DiLM$Wx8|wSAbqfqO;yH3$exU*W#^`{dGxy%B{n zH2^x-np}_-w$Y7VU^{pOY+v8a+fANd6d1NwFJPD7|7Qnnj`(Y2HZaFCmS=U3 zMrqM9e5}d>`sHPdNS)yaqVK)%V5?yR*vpF2Ti2Et$egg?{Fr%2UL5Q8fjD;l}mbPskvH}Baz9LGqVd@~xz zPD+b@aN;hk$Zv%e|KTKKrdcA}zLTKs8y2Jne+PMlUHSF(ntUVmkVM4R+Q_sOyZ&J* zrv$S_-%i|U#q>=DsKI}I0l8nqVCD+!l(6eWU^xm;a#Y_k;72~b*EahbG-IGSfLOv} zj#sVbf0L1K!3Jt32ME!lLIrnNyGzo^LnjUf?laKWnri?c zYu8m8`1Sco3y?N^sF)1Qz-e=PuIyp~uiTNwI-zJ9z;tF~pgn<+0A z!9Ey$Yeiy&2o>MLr^C0HT>+)#d#_$nMkbz&Ao4xAE4RoqBg6?zqq*guoHAV6`6k5m z;CD0)>RaX&g0FvAUgaj?J1!j-0)3|Ck)fvNVL+F*@RNTO<_g2v7Ln_=;HKmL5N6unSnTOn)y**{ z_NYVPiGODhA6J0>b^BqiwE*5@NTL~t%AEJtC(i9=RcMk z9C7BRE_Q(xJmXc0t5VD)N4kfJVQ_Qhl&Dksw!A9ol#-H7I2QOR)eH++?{v`{`9QV} zykRYQbV>1owG!#&vHK*Z>o3p0PCi!D{&?i4qo-{Di@bHyXwsUM^GfuB(#zq^99Y9^ zH{I+Reyv)bGS+bB2ySn-3-QM;==jjhob^uK1WNN2N$juP9t_Lfr{kLH*e1?ze!RpN zwb7b_JysnZra~5rWIo(UGNZ*pdjh52j3HR@R};kdHzPg(IhM#6Rp~+RKNxuPD`GwK z8kR!COm6e@rwmFw@-0oe0`0h`P4Y;2Xyp=sq1 zSQI>`)Hj2sdn90~@_s$575U$+~&dEa5Td;y3*k$v;q z*28ZBP$dC5%K@Ez)7V`!{c>6LdznMnHo!`E;zB5(|>(HJe zbjV*=`=;~>nxWCw;J6J0n7vtRKzqgHk>$Et833~R?#oEkBupmao~_6m7#~PUvoj3q1bDlMC8mo z{5ZBnCzZIIbs(1mz0+!1aC13#!Dio4hl$h1$&}))lShg-jXmXXj1&a6)y3Aqec=Q^ zF`rr+Nt5dldDft~9S9$NRLO{ zx9G^$dc=Mj07a_{!<=6#FkI3*oy<>t=PhInjbOLzO+Bb~ptVI{NIN@I1J3+mI%Oct z)LYp&_2npp!R-{-A*OuWZ0PuFyEkC0Q5UW$!~f3$Q1P$@g;*p>Dr9XM0NWk)+lmEG z*HwGo_;}&M$Nl$nuD%F^Fl6>7!xYFSYbZ~A{&cHAzoQcth&Z=AqzX7atwXUPI(>mD z?*?PAOs~nYhx|NQtW|$tyVev+nDi;b!5E{`5$vWPk}jx4+AfqU1S?Pg2ao<2cCuw3D~gbn%u~rqGBUz(PF9j}6tbPHY(ix__8!?04vu{a2PfwoXZ+ru@8j|N z19))mbKm#tzOLu%b;%jL8XT7(+phUOrOXVSPJmwrWr6;nEQOy9jH5`1`-G>#t<>GI z*_0|?75Lc1``b8Oy{yr;YBzpRSwlppUZKCF%QS;>Wd*aKJB) zy|FwCto1UoWp1smjr!and+^isd%3kR-R-rtZq)dmegB!&fTWrQkE}^+CO9H~hJ+P8 zkHe!gACUNLWD{yikgp*!Xl4)fb9_#hAge_oeZw@V55Hg3#-yq!ZNf$AIlyd^;W#N<~e$UA$^I zvd?x89(YQ&pttQ1*pIKU-<)=uYX3=ELw};9-8s)=hOObnUh3pQUe1@T@`dgjv1LTL zcn0r94S7Eh;CSUr7=1T(>rn4x*^IDK;hC4o^3lgCvq9_v`fr2wRS|Vq)P85FP#JVz zU*rMe&$NW=L9hrbYsa~HvOZPt47qXC$*(wS?@@b&ed37S=c4>GQ%P!~!}pK4cBEMw zUIzIu94*S4M_4VT?RYEh#*%l^J9$^$pva}h==F>6YSoK`DN*$e zPaSRrUa}42W^ask1?JPIc&j^rm(^>$XjAas^UAkxb_!VqAU6Dm4OVAxYs-AmOr?0) z$99hcgec+k@X4efR3ECm<&z@>^^t0yx&_t4^P5O!4Y5JHo;pW(G5it9MDG+N29(w> z2}-UKq!>RgDfFLtx!D$R9r#SDN5G?a8*XFHPQS6k zKy&oaSt)YhEa2YR0>} z?3v4X4%5y7brh($9ITV1UvieN=BbE)%VH)z<}#4;U(@VMU%@)Sr?D8anlpT->Gem> zZS%*)^u-d6?k=>Sq^Z)iiXOgm?3CQcoRH+3slo!(rj(n0Y)WTSsenpaE?xjvd)Z4A zy|0CHWA`Z`G!AN4Uvbzuw%Bg11$+38^1~{yOh!bn;5lh(1>qWX0J&*C#Xpqn36(r> zJJ&MFpm_LX7A^=mXzU@itf8`K7}m2}6($I$0yku$p-<5zjTzpPrWQa~$JQ zrcVgdJQB3U7c}miSp{h7| z(Hqoc=h_W^5wK04e_6wK<^)A7JZn$Mc(%+ z$R>aiYwY1#-FM-X5+l(!TktUCRxB`a$Gk)Yut`DI4RIc?{VBcXgI~9igrpx;;Pz(* z2qV}K{b5-qQAGA><)IN7F^Ap+QJ5<2?b#Ev)HPbDBx=B_thxZC9!nP2TMKNZYX6u- zlQ|&hhu5b*f_Gf@)6}f8YM>C6@frrjZ z@6x^g&=%bp9(YPIhMzM9KQGXDEMCz3Uh7`Dg|5o=F9(-Ji^cNq*oij0Y~5Eu*G|;w z$Z%Yvsji{NJS#uGd54&`2q)06eETyezv6#bnvJvF^_$Vp@pQOyjz?-o{BdLQEs_V3 z!V-?%Nk)y{J^yA=)YHdGOsdd-U;MtZ?Pyy`UZWX#6f!{Hz)FZ2P!3-)M6Be9t`(A5 z;5`u|oZ1amWW0nmo!{u@1F@n5t$(?$K9Sq;oU+e_FXT#!eMkz9+OB7KvH4Erp5%P- zd%P_58rP?Wyq^rb6T~+tYMt4dN!dfE0zdDA+acCEY?2|Ewx;xCX}jSmC$jqO&+nV2 zxFrj~bj{pm_XY79!mx{%;4VriUTG;hdNko^xkD3i@VejsU&1jrUV#YMWkLb}01uwSEAqI|z zizZddyd+e(pE<7gVNyoU9=rntQXW4~jZE#dy1i@^JOUxjk3=6lA~8Blio)HfZUHpO z0ADKquIw2Bxwgg{a)Rc?o9MSTrU+S~>#1_q9u&K~tLF8S{U*`T{?quEz0+N(KJ7Fd zjbs=;d^PcJBT>5jM7S~$w}wj_TYHGEZOY&O1w{;YNM*DT74NK7u7#omkt z$Lt#i=Q(B6_oGmC&ffVv&D|ijvS?9-vjr6)PYJ*lvyQ)f7`8Mh2nLZ|}`1S3yEFcfdc^j18!5D>e?N6As(AL_$8 zm+C8QeJHYk&u**+_0mw>(BLCZZzifD@7bHQ56r+})R+iZPd!L#&)@+YXR$S3_N}lr zq~bOl6J@7XBCh!*tnf{dHDr^ho8s}Q>Hz$m8k)l#54|B6QhWwP(gvTpYZ$m_Y%V>Y zAyw${S%rQB8dfkECCx$L68BM*mPM&P@l4$b(uHE_<3-bfN$C`@GZpMqT1tMtT1$PN zbx5^Kf7$GytsMKOYfJc8fhSeC{jfDTGg%gGZPl|e3u!e}Ke4f`f!18^6^7bvS9lW{l z;FQXBkrYCQMj9GojQnqZ=GeUxcZmWB^t;fy5fc0~@%Hyjdj1c`cee1n6AzDmKc^zl zJdjbHvY02wx!*qDK4X=8;;3VJCpveF9z#eA2CrI$OC7SQ`$XU1d=PCA0`+C2KWr_C zY4!>GdB=iaTQgzcjej0h=krSn$Qif!sMmIG-DIqhHOXuv-t&dE)v)ec0Xg&9)ovL& zF&F!JRWHU&=hX&nMHmnxy<;o%C91GwBlrLu1EO%{bfV+~C%<3HI0(GeTtYAcmaVWP zi@xgAGCHy5DE0gmgb|U0tOnPSgPXnOO$2S0?JjWA@>PB)QqL(Ivzp1Z{OD{6o?){& z6I@2N>+3FpCU1EBD?I`R{cg%;^%L3OT_Q96&U_{3WJd_UMLuX6A=pTVs@-`w;CEpL z=vL^aX@{Oj(uG&9RUFO&Ai)swnz8tV@tq^`6cK24&bot82_m4(_>Xt0#&f`J?I~W% z$b^J~%cco4s@ZBhCcHqObcA-d76B>yT7wdN6#7Ks@99=I-YXAYFy0QYrpWmN7zc%# z9eVkn zOUk9dLn~3a&o5~l{x&@vurE!^n46id!_Z3g9K{AB?0Ruxd8u4n(pw^q3ofp zIY-{N8uy4z>jb(U;c+5+_T^lFcfxXP6kB#JMEo4&hbY8VCS>G@!tH-sD{D8o&({WY zjS!@!`id`A>n;&P0_ED@k}^?3Lq*Il${hBHJMK+^-2>$%Ev=agQ|@=d$QkP#Ohlq zoae?)WD|WgX{F^}d|=T`bSQl0)WOt+dtLm#n6}hnkGam#`$Ql2fGgDIkdSN*?w3MS zrasu^32I2lW|8T)G)ZLSrY@uHi<_X+ zg)TsRc6a65St^)SHnLNorE<>as^>VVlGGq(xY-|)Ce4a;_1qfqpe@+%oAxLeAlS-d z-aL{TxY+p~j%m==8=#7ikgLe?X{C0F@`{p!Z16M9_mLoKYAG(Cs@td2{{*`8Fwi>} zEU~iq2#4;GHNn=ep}Ks+R}@yUtrc!m1&wZ+Dvi&8tq|3<^0T-mRO_cZV1{{ROydEK zsgG@rOG5Dzrm`BaS~L1R4435Hyy0!x>=^>UlWDdWA^L2c?+w%ft)6{Ma30(q=oGq% zW*r%Al^pNjYv*JWWaZ&QZXi^#^QYQwyb0+K5WzQs#SB#4%;u@gK@OSE$c>axg!)R3 ztQ_J;$&kM_p>WH~*4dYPtOym9dxxk(C^OLC4>op6acK7q4WjuPBlR+lbyufI3F=Bf zd$N+|2R*@Ido+KcgQyZN_m=!}hQ_Uk<}FneqBhKeAb@~`PubQ^u!q^aH9yMk0JdiO zXldaW+XKR3=DEzWXtjhmx*9Wfv6l(ZQnD@F3EWpuPyr^cdRhN+u4%5gCccq!F-#Kb z9Td1};MX)UwqhV1MtL8y4X`8cjqCJxi z1z^b%+-GCKt<7JHtS{JDKf8bLK}Yjlanf0te&fHd7e&`Em5SxRq~NrVYp$0+W}#(( zzZwyqzX*mQ4yhJmJ5z+$5_QRZ4N5GuQS#*%N8$?2sy()#itKa@!tc@!a8S)%m*b~O zfxA)C??*mRBbnh*%=&M)&ASg?-8n*V&3viDxv;XZCw{)H)Iax0<>WoNzsHFMj)lZ83fkVg@ga_CCEkt$74V(PURUz6bVQCIhVqNU`CC?M~ z;94LoFAJ1(gjQ>IMk|Lu7*n#$tf?Y0LpXE))Gf?G^ninT!{CctiTz=$ov)ouOZy?P z=N*7=-$#y9(y)R5AvJhpxHZhGw6+&DGR_NCPqw(zj>6(>HmazTZ#csGi-*MbmB}|2**JmkyX4jTFw$Pg;X1bta~;*?Y~Xo@eZ_p@}`Y zFInK$;}gx4LkC$mqSxexT+-vzIryHyfb;@Y*O{b6bKW$PK1yE|gX>I2TL)TRB=0#L zxKk%e9_99#cXPt9QC zo`7H%_&IJm*GaG(Wkz1f76-eBtKgADzTGo$)tGbr+A#L5+cy9j;;z~~;A4Cn=f+Df zR1zo$Y@?PM6azNBKfZq5g9coC*{2m4eC^w`{`7uctnEJBx-JVD*i?g(4!18D_)WU2 z)nj1bEdP6fX60&v;opW#obBrW2LF|?FAw{!(Str2ufP}M(Q}Y1hV$=*2Z10hn6MnIm7kKNOrxunt!DUF9Y6#Y z27TSL9VXa9!#$B3G2&NmCjQvQd`bk>)G4()OSN~JLuMZS%NM6Cv9};g+}|0?Dl^+ z?1;_Y2bbArfCNg(1O3Bg6HB(>--OIHGmiZm+Xr4l;3xR>7ZxuG%HpgN#TQMs+o_?z z$aZ^9mfqw^E*CE}*nCRXL?!Oq5J-AE-IF?hG7jM1lCiZ0W7{h)&|n7!nURXqlWnUL zaTnOwGQ+%Zo_pCJb|>o-qiaOVQ+HE$XlfMLA_aBz;gQ3^7NvWmzlOS7)z^k$og%r4 zOGQ|CJTJy2UVoUWC0AtrG7OiwMSb^_7FV<@HBh?$mk4TE>^#=N4*hcVJb?+q zmAU+PGOdrj*9F=h=OjbEl{?kTK}O&>@5+PwY%`ohg`Yh8#A&2_scOyqdo_Pb*IbJ5 zdkKbUE|W?wE=&1igtpWy99x=l9;&JPQH}4qz@w|}X!Zk_0)N-)Fu%|6{@ZPz<`ZgI zN@qDZq4VG2A4YZv54WZ*WsYw^YDnN?ripw+*${n0>$XR@A+$m`H%(iS3T$;Tiud_y zYA1XfL{5rm5rZ5*@_D`V-sgE_jo3h#aC;Oux6Wy;ZN;aLt0M;WJNMmwLXBA5{s=&@ zZTdXD`qQpmwFn?zIUr?zV(lNFgHX=^byGt02BF@4_yADHH&yPJs{_O<1fI+2{Gx;s zXE@wW{YO3Ur7q4vMz6mxa6sO#c7bEj4!@6}CibM74L@ zvbPRLY!#V9H@@TJid&Va#o0 zt@97)A4M?}LxCb{!fBPq0~x}(n&5GaG$|(&K5g9JiW0beDHZY8?`48_69NsqDGXE_ zQOhI9`;h*VlELvsr1k<~W8&`1&wbvhvY8T2tH1w^T)k>t3v<(Pml6zyD7k$ZwXR^^ zZB?+?How3o`-qwI{KQGUv7K)yFY?}dk~i6U!>mu0rTuPM6*Ra1b|@}gbRD{F*1>T6 z{!J!#pjwJ|dAG%u`*HqkY^VtN*tkZ`Mn?R+1lh@CLP9$hbkHh@NiRGb5EA}gP?PG8zu9rd~xS>r+iKU^q)s|!*s8-?&9XA8MUuC~6nEGdYnN$Hy-K9v~euGn5Z+PVsS z(n*3>;A6Bt+mj@st&!jrbZ!mV2S+zUoFX}ly8@!!R!TDeI8Y4kXmZ!^(bxtYi^k>R zh9}N6Lt)pnUhiQ}a;TQ7LO%xqLl#b2_fy$C$IcgR?)fAk+`-Y76jo?a-p0vJ@TjZ8Ko6vA?TG45@@~!7-4g(|=Ch z0@Y3}ZZ$6~r3Bk8rd@}Q%B8d}2Ds$brA?gAa2$Fgmt9gGut_4dv~>1AUODB{wIyj( zcxpRGFtztX9_0T7Ah!9;;o?d9JCDGg21ahsa!M0@=RxP&1ASEFrG_h<_2;4GBa-5t z=OPgnzp9KPB5;MEvP#rX>DeIS_3#3n(?pTB>SLd91H-!z(^b1iGYFJK@F+gx!|+qJ!p@skse&-Thcv`67dvfJlrdrKQG zx*HunOB&T9ri4}VxC(rGoOyiyxFl5}+2+s)^dj^#wS+KE=!?!x>N~mf&io2H^`#l@ z3ObzpDL@}-11e2EA>>5a5ajjJ5}P0r%02vY@+$!(yq0i<<)Pgp2c&?EeTXZ3GTKSFf3WVt#JC< z=W{Q+euQij=k*TU`z8D_}> zqF``{VQPGqD*ip5&HeWgt(}@A32g%dPcA9=!y~@1@1C?T_Nj}2Z6d6c2x-eeF`+RIeqWq2XD0+(7`SH-!?2#bx$XqE@BEmFQQSErXK znBB7avgDN;|o&_fh+LDC8nnC`0+O1a|%A{uft$Q>E zC+cNK3e2PpR*XEr$ati(3=&!su`5_SH(15Bl&2G`0i16dMQXEP(lL47QiB<|?+{YU zTjZK;Hm>p?>oCA6@nK1y_?Ha+4m1$jfVx%Z)ThE%GU(Bu>ZH0Xr2xBTe5%2Rx{uvQ z@5TA+>{3Qn`v<*U>I;c7DU+^;D1h^&<%sG==1qc+5MPe-TMHlMTNfH64u!z^FyS=* za0Rg9UtqTtU;db^HHO?i0<3>SwweRE{*GLF zHwPj|_$M=gz7wELtvSo^i-R;yNQ!YTde(FGmn)vD-6dY zA@zN0MJNY6Hans44a~Hrp-z#1w{Rshek^>4g*Jd3*ztV%WzhxNNe}VSKIVwjgr4oW zKW!k|%zNRhLNjb9a&*y#bLU7ho6u#IGU51Op_UWA+8FgbkzVI^BX+Hn=z>A@eH;oq zeo=+E#0%NVCiR&3bs2@k6oGDU=iY6c+wOF|s2o#e@|vxE=;@{PI;|=MLn-G&m?_EN z{3?dNYy~w5USV|svm6zxSzMqg_)<%6kVR9movGS6zb)P4ULW;$h2BNx^amx?WDSQL zgJ<2XXu4-c3RDjjfomZxq<|*YzEFKoX#SMDtJ8-XSMzwLKjDH-A0Z2m*q(sgkDetN zv7C0rT~dc9u(XmICn2JP6d~EcZ)-kEavjI3Q@oH4U2i?fMia{fN&q`S>+GAjSEa}?-A>F2J{3(P26cd;}Qqumh*u8NJw@#0_ zUsgovX+;TlexYQK7#@YwZhCWvdx*^dB}2~$v3Ynw;(y9o(HD?(!O~-KK$O+O&MDV> z9vj-#Q+HUxo%6S|eVU%Rwc55Y$4H%h6#`qKD_8jU=(a98t@d(+OoRK}vDPbd#yTZ` zXT>MY?n})E4}>UqRXa@VBaD{;P+NKagk9`el{N7HDLO}2hB$ro;X> z4#<@X1+fnn4;A5;-g4>qauqNR7xTG9ft-&;{mVU|B9;OYtFrU*1Qu(f13wRzG=2+M((c>vo|-sF}w>?7usfKS-f z1hjP>{MQ_MgqH9@o`!t}fh%WzHQ{acRoL7Lr#MwXGGA-J$M)ej*E^}#yu&O1u>%GI z#uRUHrtQB^ulNCLp(!&^*N$9(7p!Z;OsYlU$uScZwJjq+NokbqW@F6tR*=^(7)>H9 z8^s&E%!Qg1KTrFjXD*$aFJ%?E*`yOhC^rhxp%9|t`ST2X;9I6kNHi*Rf0kp8OTxC; zm1{Ou>5n10_VVZXm5PJDz6KoU3xVcrHqo?-#CEI*saZtX-;Kv_9mAN$wTX~_8yYDG z%YQ?}sPtdospe#JnWhw1`m&$pA0w@dlVw&rvJYn7i_XKxNLOinh^bBU?YG*;9i)uA zOTS9;!%7yS-J(!gJ%@+nqm{>f!6)Xss5tOZI&~G2QR*fXtIc@lUSP_P$=OxwlnLXR zcxqi)P5?AYHu71n z9Grdc;8mGQ=AS^VwI$V_3l8Wc$An1HF!nVU0zjyT zDpsAwwEy{OQ*^!zWeF;9E574?@n6Xf%`iHCQwWgV0O=GhX8$ji$HGNI2IB>R7b507 zf{e|8E2Z@hT65=N*Inc??K9_m@yV-G(D!cD=;f**W$U5Wmpd>B}_k2KP3 zJ&$h>nZE+jL-saavpKkNnB2Sqh>_FZi*SdJ+n5&t_LjpXeP(}YGj~lFjqPup>`PG( z_QS9A%HC=x;$`P#Vr76QFXO1_9$Z&$BCo?gJ3Zg3hT)~e)T4B*Ra7GZ3B$xy3IGLO ztQ&Z5=#hkKdScSAlMGh9gC!}&$Ud1LWyOnZh0(Vp)$B{3i9{xXsEV)>NjDA{!Rqnt z>Y23Vsv{vd;IM^r?}Ra?stP{;+P>?1>RLhQA{i6> zO{;@p^u<7#Jl31lT9N&9&q?_P1qt%83oQdyHT$wIMvbf5hfU(?)7bB3J|Z{ z2SJPcJ1MXjINu2AHVn}`i^BQ+?d!DdkGf8krnp?ft6-$huVS|ehM-mUS~X8{YRUd6 zs=&qlQpT}`oow_ddT%lGS&c*LqjPgYK%$9Sd&0Q2IlKA^&g9Qkv+hO- zEAM1Qsf@MD)6g_9PkbB<3#d37z%>q>n!K7v30a5=%Ow%HE7mOvHO7FMt#cU3}E>CVd(P*ZH|JnN;+=M*x zel^|{@cXvc;#b#0{6%e-`-A>?UhmZVFOGl++##`xVC&=EFaBu0#g;I0!t@{=g1>e$ z9h_K3VFvLWAlR(wU@mv~4k+=W9vtz0r-RpwwpM+n7CDBP=SYP9HWn?*=1(idZMxT> z!1x2BD>;*O+`Mba@b4Qh8K|12jqsC&@BNbk9;XQoLRvdti&oA1&;HworKC59Z^m)) zTAm9R=++=?;hQ%3Y~|y$)$>6yD($AL=3&b%4!B;naEWm8)FsuA4qfm8IB)?*(|*Q+ zzJH$2r#_AVHy2(sp-FF}ajppwHY6S1lnVJr|_C#xn z{t1TIQMg3Y7PF7Us95RehTWrGV{hB#a(HHxo*SnUOwEa7fGKHzxjyT zS|eUDyQXQLZ^~1|hrsmBV=?NCG%F1A+tIXhE(CkNe1BvoABQiXjY24sO#$qpil48& zyb!Y`DaKjuctNhFd%p`F11jAvjMY~95iD4D@J)DCQnCmA=RLSjOhzH-MUXyndX!)e zrZ^ni-6A48m0|vr{h$XTMc9g!XSph%3x|={IYINVR7ZvWc&_Ax%i!tE9dt znv>u~sWJU+2HZ(v|Ks{B@c4+2On20QA{ero|3p$?>_rcOw>Xx}=PV_Av?Y2hBmrbs zKx2tOQRZxBv-cBD)VuFn=T7H;&f##%jAQ#U*kj`-wBj%)XAI=iB#UZK;M|!c z+7X6+hfq&Eg3S>4jYSeH4(}QR0zzdq_G{cIvD#UtS0@t5JA^uy`Dx^K;tSs!?&K6) zAf0SxPc)2~<_j0C4}t$CgdQ~SBbA$NNfOsS{!+Jvjwi}?J-H)-?Iaz>tH6#*C&o2? z9DDKzx=}`JWS%J*82X2f1SefMTOKP|sHyTiCA$FAx;Q4d%Qn#;>M?9)s5{L6;0S<47SO+Mli)AI6YQFmHS0@^(m zHwpg0fTJw%p0&>9RC}b7b-vlb3Uk}S(Ce%L)^~{ui9ZzK8EDk&(`6@{>ZFm-)#|xR z?=)_TCK%_^|J6$W5ob)7Nse#}jd&YqE5^U3bikGUK@qnBn=uNpd0G^6C6Z z@%vSMT))vu`~oT9-enHNJ=$@_T{9wpg}LcTy?q>6 zAN_z)ie*7e+w|?G0``buSCOX@?}?D6*2|8?XDqW5;^pu6eYoZ88h%yRO#FNr(A>2P zf>sQkV|eyo!bJ|_6w$7>UZ-r7ur?FRI^?!pv!({02INK6Jul^PpK?>!(@!oS)cU2V zFeRWnl}VH#rr<5!%dfRYWYSGHjamZ_+Ia1cqDBgA_V@x>QBT`HqnsqLVNrv!Rep>6jdm6j1({kBsx%LAn=dh8XIXT4a?J% z5A`l1zcAFhVe|+SK84Y_59A4M(NX_jd-6`6+Ex`$w!fXv`RyZGr1_w?^7bBQ8D=a! z%HAvxwI}VYTfNqrE`Q_MA2{sY1A?zY_7~VY1VgY6asT)Ju7J75%+Ifwhx~m(LsFX; z34?lRL#?TWWlM7`b5XBImH#?z*$~SyU$nbYzdT@TSN{8&(60PQ{NU_{%hebJ=PyyV zUm^y5>BSllTplqS`KOiz{?_2>Lpp+m2~0xFL;RSD#(YWdO$oL9Lz&*kx`KcXNQ{c2 zsX=`4?aJNa&P9!QvDOdZZzd2nhlzer)v>wwKOWxms5(ZnT%9C8vo2wI@v64*5s>rR z!UDU?q8qrQ8wd_heS97%{;)`}t@YJu(w@ynBso>#-O9$;1A6r{j6M?`^i7UK*@EnT z7M zwR%ljt>wZ&Z`78InCr^3+^I* z4w2UU>cRuk0bqi=4&IGf4z=mXpJaYS^Vd2Shdl0Kv2iR;zrfQhr7vfBDmPJs%hB^d`q@F zuA2-Mmo9`_#~SZ^rbg^<|Ezg*ljw3;uD%0?JicA8PhO3fBs&~3%qca&x00^V4|n76 z2yNKZ2@uq08@apK^PkQZ4&g`V!KcgnYA!_-8o{; z$%=ZP>>h3p_hY8*R{R--pGOKC((i}HKWI(*B7UsaON+qR-ZI+(QIkflLgYEJ^MQA` zDp1=I`!kA9hkp1Ii(BfixnLm$^d7=V4KMFmE|5k|&M~J{R8XSsXSi@fro}F~K#5va z{Rg&T##*xFRbz^-35>gkuW5yPCx!>wXS)Rh?JA}^PHu3JpZX;r5*Z2@!S86YG?`vY!3O)}31c?z#P5Vt)IYuu`d1tM zq-U`raZSH>i{x9rnz)hmsyky@7#kD+R_G!B_%8Xn812Q1?5h`FHhqvE6ycIz#LCp6 zr8%uGXlb+aD9GxlcR*&6RY<<-D)f5$Pxcqlh>yi-&%{T%UR1~IU-@pjZoev(bmI$a zvw10VYNO4AGaAM0KL|)(zRq7k(vNMjP#C?>F@%3hE$7h@zlV@d={^j-Umxd38|aEI zwfvi`T9ep#yzz$f$=&gawDOxbIj!fy4)`K&Jwjf`d!=L7|26o)aWS&X?cwf8OOwVs zn@cjV%6xO?vZMYv6KL9Jv1uPY3+(Zk-#n<$Gwo(v*%^0k{RXY^RWSwp(aA(y@>J(w zu(LTje1AE~*DfAv$@ zAF8$A7;?0UBo88cZaufn-l-aXve#2;yMDER>ET|}t4}_P?Hp1ix@LU30+`{q{@Dw* zi@!zlIE>B8)k=De#igOUtQ|G)LcdatzeEvNOB7)GpFh17q|bl8o4o#*nm_D7EUV*e z_@*`nBE3;I!pTlZ4U$!mBFKA!$6AcB1U}M>t*sUNmmLK%ZL$h<=Cw4n;$#1gv)J|Q z)0!_O$4pd_N|ZE|V?$IEHrppl^8Ck}^-cTXt3hOj+%>uJ20|IvsNC$7;r7JS zs*{d1-K>7qs^jvvlC>?D^|&)R4a&csh;!T`=ytq(B&fp{DqlL+3D)9@9Gdtnwz}E2 z(%&bR_mkNpL}PodqamvoOz30MZS4PC%|N4@cGpcZ4g7Zr@-P+aUAcd->EDXzEzYvC zt^Dd=(N{9YX8sqhoD`L^5; z+#0?TG$8E~5dc%6N`~`F{Rt#`LF@EOPhf}Lx69ZR!j^@S1!EuNEly!Wp>$F>Q^eO5 zR;R!NF^_!TgafPRT=n9c2Hpk))67akvUnSFr;K%(Vs~# z#9?K8+WuyvDmxbF$CrK%V@_&xOQ8`Rqk_Qe3}lRgOaMF z5plq6Rgpepz4bX!+Wu9Nu2k~(!xEgxcUE~G0&kpb>p>R-{8YaIMS(1w`Eypgzrk1l zE%#Nlw~Jp_bpZJ|4^i`CGDTgU3WRqN2m)I8vRlRxx;Z9iRcJ~L9~_3`+j!k}O|j<_ zKM9@XCAEO7!%}f9y949b%V_ye48QcVD4O{Rv|$P5n;ED--OZSapod)7ZRp{R(mpW0 zt={u`Tez4Y80_WWpYmS=;w}d6A%c9Xd#zk zYXs=}N>=(1x8SpVKF2tMZiJ=RcGUIRr^SfV87#|T;bX&9plju*F+lUA+^P70988e^ zta5C$TSdN6!Vk>x*hoMBS<@&QW{Vi#h#N#8!ORTh8?)XGh`4_khf9 zXps0l)hPUZf5amA^whn!*jAhncB#caFo;&zD0MA>*V6+TuE*@x__POzqAUfR_U+Lk;r#O8X$%r^?68(H=TS zb`5f-87+f{^TU(AuNY8Ah@i>*X);GVomW})MDacOf*}F%kNV!$Wn52nEO(=XiAPPL z^heR2OSB0GpX=-?5;`-F{wA&j&n5AO#fwN)J^89txF!$7TE5v2y$`1pDXy$RC>1BA zHt7=df9}YYSr)B7mcocYXh*9oP3$j=(fFx0dC zo_asg42MDM@9ga~SnQsGNaiXovp&f!oxKdDeBhdE5}9YnM1|F^_#Cpg`$o&cdj?q} zLE&b&^kRDtRn24+D;15Rg}E?0*>c5>H(`($@3?$8$aG$GcWSO1EY@xNa=G(O*b7UQ zv4$f>0HCZ7pg(hWgpk@8&}XX)y7=6>h1M&(TfIwnw&(i`>rMv9@n-w{7}Hkzo9L=0 zmy#}vf}mUSwK+lFjzwn$GgO&S|F)WMJikutP)hEtkc}RXZ5_Hf)^C$@ajFv{$oc}F z;yG{Qy0DThb5N2`>xCh$E+gcdH(nZN`8q~?JSPdf5QQOEibcVr`!$20y9HkSXDKnVrQ<4iMWYL^kcP^F5yq>z(|+-X$vUR`%WdG)4b$LG3E~I3hu{HMpspHyAFAvLFle z$x${S@K#pibK#rIOb zsY+Gz8p%f7TDoG%Gc1hbw1)sgO{h5Hp`I@@+#vHDvf}SJmP)jxZE!q51p_!m`xE); z(RaG;J^c3h|DOe5OCmfB)@m2Fl|~m#n`;K$vFv3SX>*WtD??^duJNMHCZ~ zvTn^NtcW2r%w;uSB>#Bxdd$M4^%J!h$x1&-dTew;>^b93R)$nh_h4=~M}n~&)HG2J zY6vxv_8Udfy^G-y4E-e^9P1BKZ2uAWqn39QseKjBWhAY>ssC^14$YwYa`Fv&&#U2) z`-7#&Ot2lFDZ=h%B1pD#+KctLo*ww&psUs2l9qCWd$xR~yk3i;dZ3C#MJiEJ&@^EOW(``4 zwiQsC^SK!(p1yM@CV6oFGW>-nr|Lm3ZFAItxAly=$(C(`)0l&*EllB5w0+kl5#3GH z#qX(_MtLTX*jCkS)n=}(Tf^ufV)U1-&$dWk{nGU0wI@W<^ZcIxzkMo~Ghr>=8LInd z8^v*5MZ{zP*^~ZTk)c-3#N0AoOtfyf;s)Je3SqY@*-GJQdtZss*Z4NBUdNB%i*~M) z(d+p~bfs%E05!sJtPge5XGHCX zgpK-KSbB9x)<+1yj2RVQmi_JS~&gcH<~z-XRmkh_Uf- zvc2@NZn1&guHW0B3oozKFe~%Al>J0d5GLHIrO2qC34M~8$L=QL4*roV)~55AtZOjf zQF+8yc4G2_+T8!KXc@>|BZds(!cM$lzG2JYb*jX@Y`kb(siVH@olepcCJBTUwD!bc8Q`$5VWov7U*cCp`0kjVB&Ix9U|>IhrPo#4X|#;%{)y__ou$ zq0)mU$y@(N(^-Z!^}l~yq+38bMuUWO<3>rUQ$QL=DCHLc5fB(mK)SmfAtGHOpvdU% z2Bo_hxsAo2--G|-JvrAo=Q`Iu_x*mo-gBm9pP57;7q{vhQ1}YP2y7ItrTEtb?GGe` z-8)UMX`4>0(iDhoE9$o70Km1aL`YyN@U%LpJ?+t?r|*O` zCU^W^3mkz9dn&X&j3e?%npe6ph=hLa9{u{|AugLfGn7{^9DK6+&OT<4D$tVGHK;hR856NG1}ieXzD{< z@nd6IvEQ{#KSl{A`-G0Nf`OK>Knmy`Jyt>TfwbC=1f3G1;^t*B1wyXVZbKEf?L2rpZa4p0q(bCUioeOsOAs~)A$=AUQh zY%8Jayr9H$5v0(AauFkFVKO$;aa#!~Fg1LBoYaB}6yHMeb*8}O@Okk0?`8PS)Z&U; zl(&GSmd^b6fP6&W0z|;>B&SPjzMz=C5nrJ+z`qVo6-pPr2W1y)tVj`O(RN zRRP&6o~4@D_Df*9yF{{;*8hu>UUv@SWn>Y@-p+0!#kl0nEBpwUny&tbHG8bTi#Uo+ zVJq0i*OZVxZ6tWE+Z3BhP=E#KPGTw?qzhZOPq5MQ#~p=0md0=S?1T_Z@$PN5 z8wc;uqZctdm_q%5CXera}@lX_i|X%ft6;FBHa9q@+IE#DNV zE5iIall7K%=|iKRT?|Nw;oqAvjA3G><7&27Zbh!2Bj)@pKfA?C6@KjgI{t%x_L_Wq zOnZv}3>H7?PY**^745lqWg}UDU<<>Eo?Z4Z4tr9Zh6!>q>lgz`bACl4V^@`LLNAEc*DA%e7em5(jAQXLWH0=fq#V2o5+t(uuLnhzIeMk@$ey>T%L>6utS0A>=+x1GZo2So^jMoX?60F@c=PDDqw;rO zkXsQcXM^bwG~8pi`ebqD?f+KPUJ{Hb-&PqPjnBi~0^@Oy`rLZWCI4``&*TwhA`&%7 zBY_Kra~f14D_p+uuXK#DjanUGT$s{tp!7!OPt(=@blzz7r$K2sEXZp76;Z_eS6S2= zAd0%o9vYaK?aZ?AHo(LhXKA(;W~}Tx;=cf{E*Z1+z}S9^-lKrn^gzf(nA$`Zyd1ZA ztjM!fCs)&fGXg%^q`h;N2TWYn@Gin>Bv25_-6(k)iTNy%w+o2$hUVKk{Kp@Vy>8^! z?aZ~L=hFO-EFH9$JZO%#4B$ZA*>dpzMixLDmrN|z=GpxRV*INd8pqEH-Pcq zz9^s3J8QJ>DAYhMRTuQ}=BUf)!RrQ1^j`piq?2|t@` zzNwoIHYIyQh%M-ZdS`n3UL^&$3PG>zLd{xHpQeZitLLXLmbDWCc#6sdk&VRPDM9L- zkU&abGHYxqoFl$72)&gF3D_&S7cZzahnkVfmB8rkVw0b0#(WNc@oL~bJHoO&UqzPg zJen|W_!LTESt=N>NBhi&UxqCBp9SZQcW`+fs8;o$fJK3f{JeET9nA!M&y4L`Zp5D2 zvH_9ZOk+Wq1rWT8yBQ=ElGW z0TWhLQXSxg87r2Ssy6mntZu0;KC5*yFFrw!?m?eQrhmX})*p+hl$F zOe)kKx9fYx35_`r#hzw{j2|m%@vUee4_G&cy}>L6G!q zC~Ccfn(v)`K~avj`>&zB6hG!TAFZi> z?FXzW`}`C5Wl&YwFP^`i(X_pSAwg{X#SEjTP@M&Op5rgDJ6Kn&KTMy3d9s6`0f-d2 z75?t)V|bysVhB?4SHZQo`33FIFr4#2!#3(#KKeYR_~hNQx1@e?4FSL)A@&ZB<99x(KyPw?PtF6GS3 z&7GRIn_tJnmr!gU=;I@lH$L(-j-Y{?Y3chx{(|#5ve72$KaXjD=JNJPsY`dhpC+mW zX;HNC)#zNU(43|Z^`_cE6C?}+MmSnUA-zFbvK$$&XkS+7oS(dxK)k_u$Fre0V@;7> zEdCndVZ&@v7@zlkB9#1l@yPaH{<4N`U-$&#-%ff8}bkb?v?zgF`qahwKL28(4vDgf+sz?dF zlS?_aOt_i;j=g|okVmT3tD{#Uc@=dg6h;k{;z~Nb^SgqnvB3f1iWXGtf1Q0iFlEkm zz3&o?dDD5^im`=b{D+bX`Iq)V%1RO!?v}{#s3e_d!s;%(Rf(q?l=)i+_g1LXKueZc z;sna1q^I`|p?k)~nOwcZ!F3FTUI;St)*s=*@-S()M&-=_*r)K6h?-DwQF6LP6Lh@4 zVQUT#I<^eo+{z>nw}|b$E2sCvjx@EsdElQ6EGlzX|8r;3DNa@)r1H*tpszoeFHl!>o6s;mA6RtTCaw7ceqM?S}?Nl7eL}=-Al}8R0Pu%mD_$7`m9{{La!woWtv@*cE zR&YU*{Zzw5-hjT)#N<+uk8OGU3mNI>UbAn6A3dQiJVjJJe*Wt*-{Lq^-c2&qE2)Re z0SA&=1(%O&6+q*U{kp-Trq8sA%e2N`Dm*su!#_{%Cs97C<0kOj+H z@3vIOj@(h=M`l^d6G#9SA@tA|0jfs)de%F3_z#8Pt;9&}(~+s5=%ZEm=pALAC*&aP z(&`-uX7yAbu%R=r@em^qOEM7owGlm(?XLx(*=u~jn@J)G#r*1lJz*8O)SM|KP!aAE zEa9sro%sG&(A&SahQ*UiJya_j(?#XD?v*X(6-)S293TX?;Ro%4$-$hko6ZLw++5T? zhdRNdSnythN?RHsTI#(OKz0G`fzYoSCxz5t8_t5~vi~OW+@KzlZq@^5p9q!E8?}S3 zQMgP5lG9=9lh&}z$o@B@Oy_rI8-`5dEYG|tpVk9 zP{g91bcY#_4DCzp%-0W+Wfrc+e!-q0f~)5s(W9YEox zz8F~=^wJXz__ClW6g>~0w-7&|snYt-Lgj~e0I~#8u<$1kezO7YZm)_-{m=9}Kyoc7 zwi7bGsr9xB{)z1r|9koJq37%GFqO}1Q$%Z+n-Zuf26<3GzkmAoUMy*1-QJ$9 z>5vd>h%YtgbMdK=6RKCHE{wPOp+7E8d1yzbS9dC{!`qvhrF=?{Z?kV{cT`+h&4bVA zK|cYuvBYYH)#0P|(4zuOwRaV%tQmh@DGhmb$ANZZd<{z{5 zz2z9H2@$lT4Z!nqKR1a*|0{(DZkvqRjeOm%YFIU*f|XkN3fA6xvjrT+s?25j4gXsB zXsx-~77j%A=-S{)gs|rmmV&IyyjS;77&GP=RTu~CFv}}c{n-og|BAw3XDcmryHdh< z4Lzg>c2dux%`qC-(TB?b*6ikJIcwGr?)_$Pp(CM% zntJ)O%Q8M+`@NwRUN4b=Tsv8en4wWTenN#f*PAQdR>FNI6(Eg78A}0ieemFI6S(Ru zImi2$?BA-WjVEb7@O_b&Ibp?!eP`So&ELuJivdCmtT0CQ{4W}{-M}i29R=Lk81(xj zjy=b6&#r?Z%u&gN%BlT{u-m9}4HTV@sl112H>V%{(&i(F*2d;3w!a$zl{H2UgG$$P zyHm5D>r<6g-kepTBMsBM1p6N(xP6QQw;&@Po!mw^PcW<}DFXk>CZ*kkJhw?!9%tS^ zB2uI6W`I;HKhe64k3WeKO#`)kyL?+#%&8htp4Ufx(vu%x;w?n%DQ?nUIT^Q*mg|^FEEaAcZp!+Ld03V+5?4JH6nc?#nSI96NWnB5(F8S?A zZKNDhGE+9UVU$l$<}i2wP>d@@V8Peb|8D-Q;2uGVMxNiA=TLyhpDw-+(2mx656&C! zoa1LdR+|a<1aqC^*LH5{L>?n60|tenIVddhBD$tA1_*k4P}482Ofq@R>U<1y#(P8M zL`a+yJT6qc3F2cECVE@o3-)mRe#DlBahs?vfw^spabgE6ucTB|3W~kinT3eFn0)li zV-g*PY&11(GUQcsr%|RWYp1M>j|!MMk5l%XF3#6Vpv-kE2?d`(pvNsB%&h|Ty9jsZ z96?j_WCT2UkPBn~_r?4^o3{Pooh-6Mbr<0i)%&tc)Q^n^lV4XDh~tKcm@-fIwf@+N ztuJS~Q8}tv`*ew+#3U*EN$PcGKeGBSZi*)Us~nz7 zWsDSy;8T@L255c`Cu$Jv+&S~t?>H;6m2aC#>0(NJY%R}2_MZ@SveC=KH_Zr*4zMk|ApKu|5BlW<2&7pgN;?IQ^Q+1 zQ2TkO_DT^7W3Np4=Ss$MU=!=KuG-Lk(lp&cO-_sf2e9+zaYYh?DCcn}9R!Z%ye9+*% z)hF5~32)X8_LV_h5ZK#{gal+D4&ws^{746N2Mz*&h(}vs!R(EpL#>O-xN{MV2&ixm z<%_R8RRdLFHKE~AI|5L&F*N@u8r=`mxzCQ016f$E`Qa&~ux0FMICc+9;>7_G$H#*b zYn3B{hRV_mfoI~0mk8J_nuc&moEySx-(7Z;t)#GU|Co+NM`S#ebY|H+K94 z#Jx@tXjWTb#{_UJY=1uLHj@|r#FP1wXJxjoP9lE!#i;!M|%{D)W zR!cSd5|1;oS_K9V`D|2_4Rb@5xiVsEDC4l?^5noMz}%JP)~_=Ro0_r6hMy@XuXOHB z>M%jj{cPe054&0>IsREGanjmEM)KKsTXV`iQl3pFge~HS2D3Y}Ab(C26Uro8zL&R~ zBIo;aea(VK>v)zh?bsYUemV{2-YAM`$+!sz)1;11$(te#RxWYRr;Z2&|Jg-nxOdyh z@1}dnMQ>LE#})#s{2ra015bY6{(@?Y&8#N;*~dAnvxOXssFx!9`CO=fJ-z9(@1_BNTgi6n*)oET2&JVg)&gQ7V&EDsLTgnl~t;Qsl)8zz;w{N0K?nBhn~YIbI_} za^?9jzF+hs^W1TXZ_b-~=S=^RrD_n*H;jHcrEzz4%fmXCt>^J@$nvTpUBAZBh|+sg zj6H6o7uJQ|SNd)q)f0S&x6Xf9L+|BW z1|jLEsb4}_yjA6|Ep{>?hxHJUj16u7=2}1*KqKW@u}j=g(yRzs72uR<94WAGlJuF% zEmE5k%2N`Eo!`oYpZ>6itNZSZVTV^**Oe-vR;Zm^vbF@SaL4__HP*O}cmZ0xbY# zVPwR&zn)RM4`P=wJU#J--*7w*23JF{XMhn&0cMKDx<1gcS0Z$?cw&8AvNz#{ixK;>*(aZX?mdK$uvE>-O24c&wG?&ByIrL}no`-J`qV?bg!76w zlASjm-x1DqW`fDRyEjgulW_Lllpym}K#hw-e!65<=@_xwxq=wS|IUR_$wl)%&67Zt zA6>Xi>1SpfkZ1}o)cJq)oCb^cURi=n)6EXnyeh+{iNMsr1{c#+G61>kXM_ZIk&4ft ziFD_mt);&rH+t|f?7gu=G+N5{XnNkx)^)z^76N4!vN0*0=sx%-zWb+2*hiJ zcggO<9Xl4MbJw8JKn;3tnX7B>aq8le zVoaJpjpFsrAKV|lLj-4#as_^Q1;2&B3wiw^n0CK#O7T+CJNohQ&o7YIh}WN?_4`MA zy;)O1IQlMDZyuGEK@a3WP3;GIhjniW%yW}*l*JNA+d2LO8`>vxiYqe?aPAj>qa^`_ zo! zg=v<2NT1VgXYuPiu`AB+#CYf=ZPV^31F)e$g??VPO}ig=8l#ofj8&_qOHZ&{`K9m;@~{S+>@~$^thgoE2+b+ zF;_9>_EX4FENYFK9U?&ve^jrh+Egm(QAo;!1fw5=5iZz4wnF6u$XN<2VpB+p$7;cj z^r2sB&Z*&a%>5Rmh}k`&0AHkM6WnGt?KwI8034 zoj?@v>kg%|u#QRLt;bQP9uXg8s|b6yPQv?%Lx}=k8+lu?+sx7s<7Bu0dK8oVkJ#A+ zlt;HQ8<+yugJJ%mGG+Ogat0H(;E@_?Va2)!R=`N0h zbaeWmHd`9ihXTTTK}Y6vMfG>R?^zOc=glNtQau;x+M6Jb0x^=Tw9ZIN_Q(wfo<9+-_xub4>AuE-PK?PCgE&*^;({j1e|7pza2(Bm|lu(Fpll}Ts-3{3N zr0vgrsh*r=1>hszruVfShLuHLjRAgB&%|methio+hKK1XW`5$Qhojsage~Q<#DwwUi?~-k2Hj_zw zh&b@D(pkL1FDGka&GC*R>cw>LGxRJ!FU^ma(mJu=Io2~Wa3Zh0p}e>ly2U$ZI*4z>eaV|Rob?SiL%oQs&}kV zGYQuYV_64letw~>O4Hr?FaVM2YjzP%I-CF2I2VbqkqyjvdrVqp#JOnjJYCQ%wgQZV z9nIlWy*9VVRT1myHQ({4w+qJe-!!?SXq@P?+_#$R)o29@?Fy}2Oxc$Y3*5UuP2pN^ z3hrJBM(nGt*ZHf8GY2>YJ#?|41$x_f88#=N-eiw`?3MekP9rAw@Io_X?cUe+I|B_e z=Vz5QPsbkehRZ z-@nWhhuY0v0(j#>f&XOhnFZbNPw%H6V8w=mPW(<)XQC?T716DA8nDzAR64PUgzlV8 zAgzcyQAlQ^F*}PrbA87kJ3O&^-?HoKO@C*Icv$}nEc5<-J_v}fVNX_hTk5vPYz4@p zXn4FD8Pzx_H8f06sL13F($TLa%S-Rdtkv66%8?2f#;`MFCu>fzG%#;1a@%I0jp&7O z#om6cE`khtRnjeI1uvf;De7^3bxnln(&Q;Oz02(Q*g;BDx~f%kS+``*GebV*v0mEW zhDIJ>3g<$y^3;8=$Iy8SZ3saKH)FX|sOwILqtlzS--3Q0ruj()1{!$_pAuws3KqvO z1~v2tnCR%@5Ad&j|5Z92xevV`H9$8|E-GB2v10^mtA+6wW-c7r(7hAON0v$N2iNm^ z#iUX6yplHx+2zJTzE?G*Fb`N+zcq@Hu6P9h-7(K%qsN18I8l-lA9!;@gxVU*B8BhE z==}7jucN;u{ms{Qw~3t-s5MXm$k4>;937+rK}J-`euf|&6oYtGGl;mnhn2*?Ut1An z)w>RCy}`OFIfRBd%xaJNUU&rmp*+*itH2CgymTht*8Qdx#fG2`zFe4tRgDD)u^_mH zU;-bA|INbUSMWjj-c?Bh9mA_Z1PU#fNanM#mxi?lRe{Of=fLxHIQk)vojGHWR^|S= zOHd31uQa+cyCc$x#a%)OKBY#nyYqFh8xKf3c977!{@+mka$QADn-yn8W$u%Qs(wq8 zhle4W!pp5rN#_+W_x*>yWPCYqo@WiHUS?C4u>R(f8^YfDgQR^Km&%qQUz(iP*YPvK z59zveIv_Zpzwmzb&5^$Tu#ES;cCAcU-kZhcdsizD3>@-V)i9+sWWBL;xf%P?s!GuX znls-vi$4ehN8Y)RFTu>f5mIifhL_Goe{T#t39YsZCKBf7FF;imcST;)8A)FDo?Fr0 zOinVLe7CCqvprE$Al-RHB4MIT+iD}5oEBgARQTkMS53|gC>-c59r?EMcdPEC&=IlU-Q{|ck+qo~P4vpi zz7SL_ixwO9NQ4p!SZQiL>?YgB?z{8Lul}wc*>J&jh+#4bvd@@5g-pjFp14v&qr7R=V2W|SO%4nFDy)}cf?>e|At^Kx z<@DMb-UZi|VXwWo9QM7oi!&Vgr1yPrqMCFvVKZ1?3h1s51& z>zHhL8&Ol{(5g8>agGcdOaWVn9+IfumSN-~N^1Xwr{qroaW2P?H#~SKS`yQ(w z7xq^r@~MLlW5Bth#h`b_{9k!hT*$e!j6OSqc2sy_=Sr}KdO?}KF?FU;KFTZ)E!mJy zhMoq`9+_kcaz|px$ay1B+JJ{4tJe;H`TPKvY7Z;bIsR&J>W%sTqp_S38Hu+3Zr{== z70Ua;8Rv>S+>q)jD>(^XoO*~jB*D@a%1}rFm8oRuwjAkZhg1dFww0Ex7l7up8K&&pEm7C-Qmlcz-UCpv95HzaAlj5KSE|!kdPT-1r*;hc-)YS@O{qcrE~cc3JTW!?lZs*f6Q2?hw=$Jx}b7bJX)H=dIBo8TfE%~Z*ar=W;tayhrgy5$Y zj|>)g%Ron9=Bne7`P)e?fl@W>R`D&*ae(s!z(xSq24Ospu252_xA=U-%ww}RdLs`@j44@+c)lh=3L(%+))E`^}O4$Rw>s)dJ7;@{gxmT*O^AV+{-1lLk z;**o?*N7#^FW6H#aoU&Y4}`bw$)#yOPZ+G7GX4LdfCPJUJ4Hd-UrUfdzx@eMB?ZY%!g*Q7A*w;@FO zi0OSk^tUCQ$2z?FiL4GYMr2;VcN%|Nwp>G{Mjuh8XH}gJ@kVv56{hw0-T%q+Alb&X zcZrDm^WUV}*CMmw;8AsSvB~j&?mLs~=PP>~=fvmKK%aqfc`PY_P}-e&qlv}Mr2NII z{hdWs!~V#5gwn4=ppNTsJpllw9iCF?Q#Iy5ie)o{s`cV{T<*|L+MB3b zoDBJ*h}lrW&RrJEuq^KV8+KA3lOGk&Oj++!apF*O`wet`(+>=8RsLMfjir9r{0X4s zvN@fINxI+`Ykp9OSfG$`(JdcI~|2MtnX0hg_vaK@J5K`Ba zI@;HJCsnV6{g!PXE{di%dC4fRhzK+NJ0%iYD9<5Q9K3rjLD6WEn|Al; z=RGZ2RlD5Bl7|aDfH+yR$KRaC)Q2@#i+>C|$%Xqv-B|n^o_}~gpJ0+VidL{@Tk;I3 zPj*_J3ObwjhF`bpJvvLKoYGMH%yhgdY&7rOT^`&VG5;*M?ylV*pghqTrOvsL0C|}I z4{D{|{=Krl}e1b9sz;+JfQZmkE zjj2IaAd0s))ua)TdrUJ*s80nOq2R`&&1-3#HT(c$c>zu!CvZbo>8E$1u*U}tVG!1; zv2a4sSP{scrJM-R3)6#h?wnY`oEJ1N(dKHf2Y2YQTKcnX0o!f{aOS6=kj?(XxR*IW z%3JuN7d98&eIM6kFkj`!Mvd@Kp$E=^g!hN5d`Qza_q_4Ed+$}QYxa+dWGP&J-j;We zuTj@60E2D?$ug_5wZyUERZ$@CUB?FT3==T&F|pyBP#%3$_h7j&z-30aAT}A z@h(h{KniO?SfL1{NLrWY=~AfRhT_PR9%o&+|2KsB&xc26Bf`wm{ZDul43th37eMiQ z;OMXP6F58X);B^4*Db+!p7{Y<{+-@x5sOFKo5lHLox3!=PFLOEUPTXe`|m zTiten z5qNauKE)=H1IvZbcI->~wW4P7?&$xDc*Q?94*L-5LDlm2(t1cMf-9`KO8DQu%&7b9 zhcmD2Gwo8y16`Vcdt0UE99nT)0lhN&giZgQd{&YDdH9WAsAq!h4_fDB)Ekw&NJg$s zl{pLAZU0WjWBglw_saZBMkgdQ^ifa26ISRYm~A`W9 z!i9ZekY^RXwIy&*yS1>yUX@)Jq1+|u{<5;6yt3^g%nWRS-s zqK(9!aAZxZBx=T6U|1HPcF!7S-{s|3u0-xt{=n!5sajCq29^HQ8aAC0tDZVart7aY zD+FU>-OqUlmsr{9)LTUah9sleF%mg4u+dljapWxu919?ffWO-mIm0Fl%NZE{Yco&A z8EG;yQdP~7b99)8#8Fzux@o#xr9V6fyLvqTJeqD^$Ll&AI!Uo05^@Ht&oGhaGPWt zRSbsSl(4dS#$mY7C3~-TGF7|pKoNT#_$-|G*GXpRJ+b8+koA8+KsuO)Cd|8JliYh7 zv+|lf6(ZT10e;q?x%Ux{P>#A66tl8sN?fo2)0V{c;nf(c9I@~o4|k8&C0HLWj3Y`? zt;rSKxq`E83piyp5y`L-W+nUbBFXr)kl;kfM=ihvN|?o+2B-JUv8R2FC}PjB8uz^R z6|8C&%emSKn9DHt$Vo84Q|_Ge1x*!-sX;$4|FR7AcKOlB}B&c-R^)^xlrZnTwLK=Da_O;}05pOY@BpdFB&qt4!vT@rq z%=X6+8bZY%m|_}3_yOlINabopPg2YK07 z{FX;HEv`jAX$F^`8SlA2)-WO2hiqL4;r|(6^qqgGHm{&6nV!_A+L|ZUpo@z?N;_-k z-hY;NxT)B8*qOVyBLXF;brCrogWEy3A=CF_gS}>n$t3WXy4|hr3htesXvb_NUiX^q zF2`2+3Cj~RJ9zqC*E#+33g4OraV@K-!wl6FeZBRfht_vSiLJ?;2Bus+{47F>Xv~+d z7^3A~uTr0kEOa6YKNohG*uI65okPT>uqw$KUJtg-+M#1w#}Q!_v_P2%Z8u)DpMNPIOYBhCQwZQnT5S0yMZ-)LiUuMlOke_ zV2^y+q08FcJZ zv%YKh{9idhCC)y@AedZyJ?fh_Jb6V1CdBJ>0p|6_e+z2f_b*GZ+smd@{fL`L)QzJG zMmg)}>=tC3)Vx=ZHxB5XqU^L6=uZXzIV}UXoDQgAnMB>`4o+~35*%uCwUJ2PGalf{CHV^+yLpAD-GQJh zeq*Fe1$GX^%)@#hS2b*m3~m~KY01k;HUg2UAjS2_Fm`_T9&oUL6KuKm<|G2rody!`0P3>&;Ka01C#&lf3kQPC$Sn&!e!awbizc%9UG;^bBIgwe@2N|cLG<|Q z3hf_Z$f4Xd+tzY~ff&Bej*w4s?@P-PRxAWVBV|*4VJ_^)Wv1rwRDFy=TsMocX~g6Y z-uB`|Lzg*5Zz)Wr{k>K3htua9Erf6T#>zAfo|gLLbrW13pXt93=>@xE??Mv@RY>R8P34_dIa7n zoknlxKk3FnZ#xWIsFI$TjvhZhE8`1?Ef!Nr^KE)Q{b1@PPVD|eIe`tsyo3dkd*MzPs4!GtY2eIm~*3X11+q;~9 zJovj@{{;d+Uw|=56V>mY{cg*t;UCikzU_28v&O!60i)*x6(s1%zXnzjbYvsA@i~zO zNbW;>=vyo|E)!EJl<1uAO{Z|I&QnWL8H{bsiBM}yE zvxn$~To@p-sZuQC(=Y`L__nJv(MeC`NI1*w+%p?81DhbvRe+r$n~=uuBZpVa|Lmuu z0q(CJWknoYFJoAyz1Fld7@!O=ZlRNtoU?MBJ+J*GVg!3Tzs{U z)J;7$k-VBel}L@`l=} z&2Z?Dvgfl5mEAtmdM}r*IF>KA!<+M0$6UrGdtd7x8RU6}mh9i-@f6VOv#Y+)lUw@yjUEvddn%h?h$vwHBs=>`R z*HYa0*Bm{A!021N^QQw5J3XiPI{^~Me;X!B;TV%cn00m6eA}wYey2&fnT*PLg74X3 z_*4=+4kwYo?xn?4sw+dz-saESRU`#s?yvAwVEzl9yA0F3oYTZ!J`f9E;?EqF=Vx${ zY4z^K=z8ya10WpdGWv?O7}r;`3@mD$EM$5iEXU1)l|< zR~%BFX#|njtceh!k9yI+1%ITB4RvjwaK9TnIc%RU*O-Fcgt8VK%t1%yj77`HS@Ygj z4@ikB2z(Qqv9lj3n?TQHGuBj~V@GHm2um#>*ACS;+c@w2@v`Ua9Qd^UMH5|Le({FT z|1MtCbDqG>F}xI^FTcGon-DoFuZ}?%TRxe2Bmhn5c3p#Vu?Vv$eZJX=fXCFge?cAs zio;#suWcagA0cLwkEWe7bnQtVX(a(M?+iCbjoFPd=kzT@N*m-`O1Tkx z&MQ0Het*WE61Ia5%S8@2+0^%cpt!SiF=u65!41ZT8;Kdbs-F)IMXDd*y}qvgiW)-M zZG^%s`tJ~9-otynfwrEeY99|FO>a(q&~-S+o;8fSLx*g=$l3w9QAq&YeH$HUc3 zvLnivzBg35U6>5-HzxNq>DCi7gYeXNk!ftuDluV%GVdN_lb-;ToH4l5oJ~9j>=5c*SY)h@i5a@uMgd7;M0me@8 zhkX6yd@QE>jvja6Sm?>YeAn$z;sGD}clCo}l-_Q!i+E*Pe>U2ODPhgP`4uHL>%XqFru;hL$EMC20JLsin@ z;UhTW4>Pg!^z{~oKM>pivtUOS5)5u(rC;;S#r6}$x7g3WI!dRwpD|q$O$I(DpBRqh zRNtk5{{6uMuuHl7|1JPTYvIZzuK*+W+gf+1o=|dlCmpaYRQ}XHh^_5CQtDUKaUrmr z?G}oR>lj=3_&eZqE@g%3`CA2nU-7_j!&a5}*LfXQi80BWqST!~T{kno$aa3)QGan? zz8n_ES=iM+DDa0X)C$;=+-Oe?>RmPqNd5C8R8%3`M__o2p zR5f7uA|_ER45S#xymk*iB_JHYommFE^sVfgJFh5tal?((oa|Q~CjI2FZPm<>oFI|F zd!3wdh6p}ailL?5c{%q@A!`xP18-O?*;>&{d4B*IW|$ca*X={h0QD z76E(<9j#%-7ts^@oyNzfeObnC(H5VNs+q^Ilg;+9LTG~UtDWIY_-uqlbK6DffEi}q zEiUpbk)tju(+h3>2u3MpV95R4f-+Ijq_4%p?6iA@;>F}cM%`O}2e$;yTgYk&Q;CpS zy_RdSB_iQ{e(Z?XGacMR``|NWF*&HBC(xcel%1USLdeoV7`)zDYGL&TNB>>wM(SNB z1RPfNZVsi94nvlG93Vqr(GenlU$!Of;on}@6CyzI)C9SkKd-tB^xG74O7hBjrk%+F z_+?{MxIB{bL>j7*|7kL`2sIOV%dFk*@Evw>TsKmPcwJm!1Tf;~A;JV;?fnUIjA4tJ zNV|y)L)}zuxAe_D4any`zy&eo-EHrj;O_idM@DN974GA%eo;QS1dwU7BwPuInW^>%(mSb zz|<$);?IMxd)?9~QnDxspQ)8hc$NujX-I`E#;#!E*k==LPybqAQ!>=axzGC*PG?0T zr?2)Q%X76-Z2XS-CToDe%eDEt=^w3**2V`CfWg5XS$Nc7d3 z?vCbfeiUUMJI|fm`XPP(`YtnM^Fve1XO4t#wfcWLc*%aw_h&+-Wi#m_L(Tn}I&&W= z-Pd2xmyy2vC+5V1$NlxK#VuEG<2>9H$^C%?_0e`?qY9Pgk9@r5aXRC?-lQTK266-s zzyK>QVJC@ul_*v~kVLw`rB@Qa%Ky=HmT^tLUmr*L0V*gWO2b41M5J3_Ls3#i36Y$D zlt?2X3nW!iDalDIozhIY86^#)TNsS7U~%vN;C{R(yI$XO#X0A5-XHsR^tQ>5g}QW+ zr^GFef?MTBv)}(K&J$;p&KIan21|X1-xXNwUB2l#jS3%IYXIL6%;RBQu{eTGJ|7yF z1pbTo?HK_Nr74M!^w!t)!6vZ|+z2N*_IB>~7vYRKsPd2#1?$x{RUX6$BT?Ojw`~=< znn)+{=Up^~UaG7{j~5X{8>d*#j%BbBG+9T`5(P z#`P!h_OCo6N#q8A#=G08%0(>-t1z(dx3dAcpZ}YfOtqFGRJ4)*>U4jN@`Hslks?r8(Cq5@9_5X>)ioGYw_G_0`(EdM&bXquaey=vd#`I9N^X%$C?iG7W_Wp z&AX(3lj84hVs~pg0a-3NsVa7|d^or&9flqO0-aQCqJnT~XEJ!*<-1uZNvM}c<*!;M z043mfFWxD_ov{?Ci#aS#*svmR2E-Z-)yk0BC!LF6Jyd=w9lLp%EQ;JC-F_|b4`Za$ zpuVM&`E%Y9?r6PZLf);Jqz}}qURAGCEV|Gz>j z%msIb{llQ?mTGhXh0#yq-d^N;sO2jODNeL}%{?FGl=yb|AyrrziW&QNH)8Jbx$7T$ z9k_CtT(08|D9P%akDt9CvHS7m+RQ3e)o!2o9SxGHC{n(D(s}N4!dH9LdgE$MFgyas zAfa`8wU*@7jp56uy@oV7wKNl&Jki3W8E}|XiQr-0EMwBq2t2p7iiD+*>JxV|jOKEKi z$zrWyRgjb^%2E_-8o0hJ$o)*paUqUpXDEXyXfMuon))Cfzu20_Aad9iUYz{zxmL5S zDzNto?C{*`%kx)+k9da)N-o8&IL%&kky#>+y*nfRdZRI@61FDoyMR~maS%4Tyxk<` zLi$nfJ%90L?pdZfL%kxb@Wc}RZEF(${_J9{a-;Y2osVGtQ{@z-OKv3OIm3@q3rSve zqOV~Dhcyp!8GB#?RO4IGmIS4RrcizE-DByFIsTH^52&BXJn5ah5rvD{Vggi$#1?w- z2y|%eK-F=^kOT_N;-84xuTkdTN|}o{WC6eM3X@2C_K~}Fq%=Tb~$P>p;a0bVGYGOemwawwnV&B{>pnK`VuqI zLMAUab-<-^1=&-P&vTD(tL#d7B|G~&T)c=)>8^7WT({TL=HC=DWJ58Kkzb|vS@u~7 zOGs&SCB2PNzMO_iC@><1BoRwK<>}>j3DXR&;#EqCZtm-NjYi0ia(kYSCH{5h0QtOw zOP%WT^#0;{pR@dI#_>Z1K#E)5KmDb_xMv4~=hh0|pURN^&LIfb3q6aX(FkTHh6)cm zU)o0f+cRDbk<$M#WlB%Jfv>$5Vg9B3_LkG=`kSeEg45k;esVvEs03$O^~4%|`r$a- z&h*^`Kr{#FyPtyFw(mSBs>!KJyLPODxH*9ATK_n^ULhYdOx<(wLQ@H;?z~t`>f@7t-wbmbIC;_*-D;(X|vl6Od zT;goMkA|aPEg4rF`R;Z9%lE__V8vG#q$A>Cs%i3NC$n1>*`K{*k(=Nv%siAJO~?I5 zo>7v7n*y0bWvX$$1tzBSx|4X2O$x>S=PSoe2QSApz-wOUeLTw9E>`UQOZWqUA(if1 zydTvpg(I_liZv;vYpsHv6#F$>2p^#Y@;H?Fn#-UVTJJ?&i$G`en^NNf!GQLS9NPt6 z>}6-=G@Ptmys~5i*0SSwex#7b^mHglS5^obPp?J^f%o^R3xssdDP1bthlKE5hK|cSi_M z+ktk+3(d|+-!UE?_2H5JpqbSd!(S?t9BL;;;(xC{L%BVfZz~53@2`%LIOKH+^NCC8 zPssYNA@=dT4SO?6uv?b~ep8FOWLRHs0E~dkccY|!1I%a+ldp@cBUg}8(ESTE0}Hxx znnwwDQOsT9Sfj)yq}N2xzDLe}x{2#6(Loi?A3Y-!bPKrPPmMc2`eo{=hf<1_nazw8 z$;-GbqkhuPns?V}{{dztZURn|tH3-57?!Uo2LIhOXN5m> zKl=dttCTqa&+#81_q^UsC3R)RDu}-JhdTXTf%B&prN^!%5vml@jZTQHV=h7$dwZNS zc7_&Bt_g95irtzPXwqi^&&6NS+mwX-W|Gf;eq!;l%{K|8&33<+`i(ijd{^RhLHDMq zj#C|iV3&J2z{lf90UyK`dMprgnK}dSL`+}x8K5#%o4#QTcYd@;o46eWCKMfdkr@dA z@FcI?o+7`pMJ4JwT=V4IcyTsiWk>ovwS%sxjHCV)w|IB@WG9^#%5HX#xouSY*LxWI zn&(VROr=I1QcCT3Wa=DUiLA&B=p-m1S3uC_NP%Q9bNQZgCy2yg34z#6k!Jt)p4DHU zRHo%fN1Lk+lP(%XUk@9Jdk@3!q5pc;1AE=(l7XA%9J#L@*Q=i{&u@II-S+)LvX}!2XE5f6=u*TI*Rf!-Qh}g1T62im@aPq@aUsjFMw{-g}mmw`ecodyh$i%k)PSLa9)G0NwbZ9u4%k|>L(iZNH zkuy~H6W^eOwb5g-oU796Z6j(EDpzX(F8^*o`*MnAw~3_t|J=xaS0wE|i^vpKTG(F{ zRMNi+9eqs1A{}%Uv^3bhc@FiDdjF1;65R02`5oc>{DaVB9q=4yu42c^YjUKZ?nRAu z2l%kkwCq#~@h}OG5}wzJiwAI}h2_L{Bu18g<2zc@_HZLNF-fMZQDSRi=U~}nJ&XxLQg(c{PlG-d?KZVMzQ2F zLZi*B%>U$%kI2U46->sqquR7U6L_pdrZ;P!Ok66DVd_jKAv^p>T6K1HxFhF=$a#4B z^v7^h7gt^vOu>IZAu-^|$0S6J1b2_qy@w3$%luR~h#?m%Nsd9s_*#FjzK`s$7apr? z0=$`VavO>F{f%!wk(*c%_b1mX2iz9O+5AjkUb#u+2BcVD(O}%8E`SRDDO@|M91XoONPs zk%b&XH5H8dzUg{hJJl<}Mb{gdlP;8>j6I*Rp_CY%k1dmPQ!X^a^pf%_zZg?|)3J2l zR@a683aMT8LZgD!SJZ;9y3UfiVx|hFhw^sA)_%FrU$DkY3$Oe3y!l{#PKw zf{#=PjmMd!{M)H38>oU3EzH)M_yJkaFsZ7278i>n-kTPc**J74|3pc&rz)@O3^jFi zB#ryHVQ=Dh2F><525KkCpdQrvw2Nv-Je`Zf(OvK=r$-<7aT!9T1g&_b+FB&d@Bkje^%KdFv0k@v3%YPRrl*VrpcLWw|vfz^RX5uw3zrj;;%YBWdXsYy|^s{or`J?!n zYrZkG;{?tl-a;6#R>Y9ynDoJ?%W>>{e5g%UP+w!T1xQO-2+nxPN==5J! z?|^smevP%F-6`<u48U=T+nkZH*@E1bF`K4ut#%1<4Npe?BG+Nlibi~99On`*CwS@d=BUn=fE7-^a2vcf z5t|xUKG*I1T6UmaGH&X5?x}nfK|sy(Q;b`*LjcA>Ut#~gQ<#xChE<*5D-x&(n-%H< z#WKQWt#&x4zV{)eV`Yj3+mUvB3@M*)In$kU;>7@e$ywAvtAbMVBeN2t*SZ}VTMwYB z{YTuZ$ZhltCNa?ls%W(_{i?A%IN)?!9?_|uE*8Bs{hH+gyntH~FCq7F+dgpe3xzb;k|)Jw-M{f_Crjx8x)> z7tFIzkn58o;(Keq<00AD?9Dr0$5&F`8{E1C55TBp*q63|0y1^ud%mjTpo8X zAYAMs%NG+a`b2}E`$$%}lsTF4df#W6IiWV~)grq%_IUPjXBM#e&kK(1it1YMc|1nj zIS*u^X-T6SW;L&XDUC`Cz1&TZ-`L>)XuI@wB0(36o(4=~)S6(EhO8|X+J-PQIh+M` zB7Exw;?==&1ulMqkzV_URs!(mCNwDY;rsP~_W0F}HZz)qzyF@Xd%ruhRTU)5RD6({ z8RZ_yB@LOAZfX8PIG6AfMVDCSqMBe=MnO#l>-VPne*XEz`Bp`?r4SdlXb|_Ye}7is z>q(*Q6CL^c?>wKMRbby|faFBIR!9EyU=jOPBbtjD0YwWmj)EEGqM zwc}CAkj+wU@QfX6y6%un*Eo!Db$w9-URpWcWRNfbm@_p|JgL_ zY^3V5#E<9MEZDBYCPLEf_X+7%Womr#2x8617hYNIZmvg5$m;4~#mXSjmwJ`g)7;zn z&x=D|C5YUa#r59zxY{DhTzyKuYaG$>8Fox%{SeeG0`glEF^9aYUgpMMfPl<*?tsje z6%W_nU|y;`f&Jg$h8l`Lz$3xL;>X3$}cbX1g7i{ASYOhMq`j zG@76tk{^w#YQVHSO$15#=$-n;fvs9>*zF_n+AsEM?uVfyv^TgQLcMahDhrvqN`;V+ z`DzsCQTlTjX5P|f551ysAK>%J{)IJ)IIkEPzQCsZRNHfnbwly3+=&WP+L*nIfefe_`j~2t|`$0-8SbMddh9-(D&r`&wV=UNgFo z*1!Y;e3%an7(G4uH&8VRuZkbl#RhiEcRmobmm!uZrnxD6IkH`;q&z|(bqagJU0O=UNJsYy4!}NXd!5zrx-o{y zsFHmNYf=hJ6~=cBR_Rex&pHbb?2KCdaXg#PEi#&U+i4TRR}#NnKGm;%nsnuXgI5*P z`yZu8-iOWzA>22u`$#c(I@?{s!*F=^L|ZKT%Tus}1pKwEblbVrJ5N$5)3Vk1r{!81 za_}`1H@$S%=9eVQV?_du7*#>41E1BUBFW}xOr%Oua?ke&_+Am=FOs08t2w*Bm-luI zWY|B7_$UxJw+X8ChO5HNzgUazn)NXF0-6iV9~z- z(UIE~%^pbD0DMgR((QZ?*E4LYMcugW^cd(~gN%p~*$g2{w^&*(l#_o#z38<^mHitM zYbG*s7l{e*86xTT*{8hol0z+VkSE(7j}vcHwj*(Q0W=xz(yIF*@aP@lFCX*9|fMl4AsN7`Ktqv>nF0km*&$clCwW)MjpMB~no$Wl-iQl0oX zu>eRvi6ty{6@rXg{SqdXzY9mrcRC!Rb-<6nhcXC_Gntjmn;wI6`U+u%;;~EXL%SPn z5mnpwe!AYU-NVYw!Ep=Zr1j0Q2+{C=kzz(0$*c-Z{W5tiDRS+OkG*8Qo!&WodN3TL z4vmnY<}zd?{X61>wv?FGD;}@Mxu^`j8(MetbnUw5H?f41&{kUZ;+$*?+mY~zvO-*w zz4Xi`#<-g4RioGQtJXZ4JB%ns`2D5-__i2t5kAHNbXgw#2s`xTbq5U9@Ac|X^=F-V z0*X9*uf-VQk-z`2{G5vaD6gp79t{Sc%8EH|#aEfe&J{$dY3Qx-HJ`7l*N+OzxIFk1 z^fpI-&awYw1aGKm^h&thwTI~osoy3qyY?>7Inzr`V(vBm0>I7od)CgOP2RB7U1lg3 z^tn7=!ClQ$xqp05F}(V4WbaqN)=5vi+^R$1aNRLV;GAxaDj&f<0u>ssyga7qn!$$b zxGmcDU?BEQjRAv{z)wuF3qT*Y!C6vH?-Kfz{ zF8^7?stkGl0_?=D!Jtx;GpYXmjr(s$$1sjUzfwp!y*)g&u*zEFiG%3}g5Sp4`koMr z)dZlQ#AN3>@EJwlNp*ahGKAJzM}G7LO+Urm8iMPjQIn_;b&r(P`}(KIAErw{@y+Q@ zoeZMu)+AgyCSBQKeiLwOW?Xvx3h-2&(1d%n!rI@EG<%LP?MdZ=#>$lO-i6%z9nMo+ zl#NV;KM0LDKLUnlhhiFj(A_~ljG7aofwe1NMcgt~rV~xYHm%{ex2wi`S#Mmz+RzK| zI{&8QMWi$$&nPl^eO2b5_UYiKPY9bslHmL49@g-daxBqN`Dwc~+W#muEAjOOxf+UPwfHAZUubW{PYQ&{kmtO-TPtjS2opH zHlK_i#BcNAtq3+WJRbe|Frz37>@D9{b_ymQE?dR=c)7pAq9?`knQ1nV$djI0u8x`N zQuYK3EP!x!<{7DgNryuJ!26@YC*k56IRZj{XdjYBOLYu@rr zL6pnGSdLiI-|t{ZUA^M9vET#w2UQ2{7op@7*;C84O2 zK+=DHCZGp`Bt@hPvm~f}s23z{@Js$yo_OWcKgJ-%@yy^PMhsSj;o<%?N&+`qhWgr) zG--)g2;Jsiddrg4O5()*7ES_=*r{N(eJVuA76ohGK*P`B}Py$ip4)E~lbkrhIX;h!u+}JKj#NTx<^SWRW|5aUt%A~Q+ER~t(#qO zE!!8{wv4^_P+D$Pl;aNN>DLjNij$R^jK>J9{lsF1p!Y3@dV%wTpAiwA)5ujOXzbzh z3>Cr#d*SehbyF&v%}LcE zYxRSV!T%21sOb;y?{AYd4EN+R^7PQ3KgqNC*Ndnbzh*id9PF~!=Hyhm$0g!77nNKs zSrYrLH_Oo@WYd;1dx>V zQBE@aQMc3f<9fk{f(N(!Q?s?4ceRGQ@$UHvn4&jWt5(rGL%1$CNF}?Pw;-2ABxSuz zMLwb2fwey7n|h4KWjHSBlPy{4RMeJw!bT0>V)45i8nid8M~nu9jtJW2qLkZqzjLDp z(@0(pYlqU@UCZtuOu$GW_+yI17?;0{L zY%6?wJvkjrmG7yIj1+!!pD!=0aX?opQn%D!opqAsS{720YADZ5jd6SQ&}r1CpT_%U zUlRK8{CQ=*EN}0XcV4DH)7Grb!;E~BQkKQPZ94zlE) z_f~8f~h|*~I zsZlpx?6$* zp6#DOsg#4Dc;qm(1Jz`ea-qY7$v~-f{hJ~BIHf)_Xfq9bEDH9iZ(e-rDG5<&mNPP; zHI}IKlZ)s~i(%?~>QF-Roy8NYvUzg%OE2mG`b~j!xF5BN_61o_+w{HO+YaKnBEqL- zdROSErQPWF0OHnlW1R!!iYG66dMrJNEqOSqSu1%^H zG;m=YynBqQxt%2}1Kp9K4#JQA9n$`=ctZEAT91<}$T6q}Um`Z{$+VH*a^VdmPZ)CG zPzZQ(;hhj>%kLLr7gZ-b|Fw~kUqR*74R{&5pcn_GYOY5H{@CX4mK}XUwG;lq|*fc5Rhp7owV2OPSLCP?1pq<(z%%N@&`$kKWa2q9Sn{7;-$Ie zxyyO_=fdwSx{96d(1Jp#3-*o>YpsCmtN{ByUufXG75aF9t`?nhD+liFXJ{-_@iZ&= zo^O^W13(e}KU#?=bl&R4fgyDTMBYbH_jlplZnoI*)V3tFYT$_4mIfV%)Z8uQ!QrS2 zqn~G<)PE|J1sSq{wnxyQ6>JYgo63L)=ex*oh!?vvYCNQOCN+PK2WIw0obI$y>7PE^A*Bixn+* zrVvW>RmenTUV5vX|1KnQV;!KSCPNAVJNe8{^xlMY_5sB1z+(bV8ka*^uSLp4cYjk5 z4+NH-IcJHfw1U>5BJjoQ`uzH)dDn$?EYFuT3uL{_6ncqhg`W;z=}~^J8*jq!@5nrV zXkG8IRrj)Bk~L4V))J^eJL(Y($^2cp*va`lrX}&h;75Sp2h%rUo{YAx=ZzB+_&lvo zq#qfRj^L`gSCfl(suDc|*F0NEU5>EC(bLM_e`&pkfl=zvgRLcsM*szlQr&yEKzxZ{ zwOiT^lXzAb{<`%j*0W{5nzN0C>v5%XtNPh~MyBte6&cq|jztEb&oaH0?bxAgk*qy~ zQ{U@~fDKn@AZi|nGzK`U0+H509lOf)t-~s?HIUj?rq!1$+ia_cFSO}GtKhSF_=clQ zGd(yOY0aI22ZRI}Qis*js=!Tsal*Yi)9iW?N$T=UM&r>`ecKHxpmDeJ$0A9&o;XlJ z9-v29(RDO&dHT#aY1Awyyh9)JdCii1Ew*)~v&Q+qWUI$>K7wO)KdH5X-_SGhEa$91 z($VgbSH|^7k0nWcnIBS%bwx$RFJH*iyau>f&QB-{1N?*{y2M ziO_K_rBu!Lda}qR(f+8kSs=at1CsdPdYv9Qybl4g!Y@A@+ZvLAB!lABN#8dDL*Y23 zgDfJy1Q+=+A|6J)LY9M@so(9>dcS2Skc24*&fQISG66{O4nN?(`WIoy$WGe@{lMdd zi~h}z9LCW(R@eIOjol z_rrWAH24%8|M34Ur+u=6otYVc^YUW%6!r%D6WN1jQjhQRLNme6nMV^K$^>^Njp<)SvZ`urEogQgKd4lXZEx^$r5eXUHFWc<%v?;^vg!*pAM*JBWalXbD@?pz7@G~OtZ?7L# zhV*Ih-z6ETLg0@l)ZGE(J~Hh_BJzU~c`l#9&h7SJG&hMJ5?lUPpZ>-S0<5yy5$GQh zRW{I>*0i%oW@_Cur2|DF?BIWzmB1>SJ!F`XbPD;_ zdrrFpLZB;#8{Y2;H39MPWt$8mbONaT2$FuOFzqipoKUo|UkK~auc~}$9WdP4c47x0cd{1;o&>eC+Imr$ zxSvo0whjYeTS)RB7Ft3RX@W*{HiFMR{!0UqqrjvU0MQP7AOP||n_oPA5f9y$*g8}| z5G!D~8x=Q$lE$au6qQV9;{1=j&bIx*CHxXTF1To|_s}P`xnQnR^WwQk9MqC51=DGS#e*i2#7zMu!iF(yy*w`dpdB|do=K6&q!P^#tjl4B3?>bwDLbvdOw_X z#@!ZkZ_g?!3i};j*Y5Ku^G$cI>?QvPiud=Pz@&cv zn;6RQ_52#ttop)kexz;pnf<#7;KJfl2|&+w=Tr<20ror53DDEu@8(mp+L|38`Yi`* ziL`w%ckB^em_O2-B16%oZEFKa&kKxRk(Mt+thLyA*VAFrhqN#xtxFtuRH)MGYQQKK zL{wD7ybMwKUD0wJ|BKajb+t4%Eac)>7^w_M$pN2qa|3bxEtl=cir_)@&{pSNo2M7_ z`_th|>@*A>c8GtNBYcFcS{A(R%pqMm-wu|*c^w5IJi3 zXXY$0`1nDHiTBT&ouf*uzq}X~q~u&+!(S&R6Y73PTl((g*!@CqR(!aTwUr(05zcjv zYNvFZ!PWlw^YrPry&{{F#fDZ)M-dMlDTnz*c-#4Nl9eJLd)07NSv}I({oPVKnBW{f z6^db>xv2_^07)G`Y2LQCCzl6Ms0FFi)eP8KJsMBN!?(5RbCtjADQ*ZNGBbFIf?Q@D zp((Z%lpe~ZC#oQK=xwOv)FYV5L#|I$qsG}`kARGM$mt^RObFk?f@mK%%Mx|y=AFgE z(RfOGGa8zhkxBWatFfAmZxCM}T6=)S@0OI4J^~0I@vVnYmFAtN0Lu5?1zdT6lF~eh z{CM-Q36>kA_eWkW^KQ?U@ug(9yYn~}r2){d{@cULmP{kW_`r>dnA08CGWIZyRcI^K z6d?N2@j-sgK!w{Cc;mT3=iNln2Y{)!$NYY0>hQ*)-5w45&@jvC?4!{0>K={qZ%jZN zfZCi3ObH!&S`6QBnh`x||7=oiKkQ_nPt*^CX~EPJBU-P!XK)6J5QA!=Q4Zi#x@TXs zRd$U6a_N|E@6-#XI{qllxeMv>Uc$HF$~=cz0`M(cn#5Y9{TvAQah|?>c5-_&?d++? zu|#}eSLb063YjQPPyG|KL3G;{B<&1IqW}nhhCTgjbV0jg0j={k{Fc*OkMey9zQE9x zS?r^vaXoWD{)>xjX)ji||NFq>6c@B^O);Uk{*aUMMlKg|e7fR2b05hFMnpfpH_R*L ze`Qd8LD){)gx+{-Vs0yZgVxS<)^SAM;*`!+ zROtFPUW@A#xn}K)`IRMUhI6n)RaW^bnMy$lX z{7~L}Q0!LD1tKlSyc9ioO*#5}qy``bhB=BB!{4$0;r@tB5_`yyO}3247jf7m-vCWa zrw=Y(LMgJ5-^1EDr_8&vU06QsPWXGGBB>e^HdLM-4HX!sQb%?P6A$Nn$P%;+SdnOM zqhFt3H4T%{ZN^*x%-e#}>R|Ji7FuB?cXnAsX9sm$3@m=GakNFg;noh`Eh49q){jrN zJaxXN=F1UnPSm`pwm$uaR7@EBg844it?hJ=i}HDIF5a4QF6De@?i-3x1cHz+PU-O^ z_Y8aZ*Jy9bcp3{oSTZ(qc-VY!j=h2An~i-pw;ebZEfaK-2Z%1$9YV9)1Z_BqKT~yn zCWJiOf*t(x2v$EK_L-R@>6^_=%U4@4i#QU{hIIxLIwnC2y`+$6#;rpdt`!~pSGPRH zDt@gOja|YIk&1SjtQL1)g2|&Y)Clg@Pkx#-aoFlnJk9yGPa1NNY)@mNUB^y8^6f*o zawNFj=p4B1^ELsh@%J|Gy&z0Hqio)75iaNw=)yXQ4xLj~F*OdB;r(W4#iY;Gx@^Q2 z*QKJ#2`+lsh)s1EK~F+&U27c)Jhg`C3=R$g38vKlbOL(jyFV&OM+?KJ0rRPaPzbYQ z_=C5DNZx8jbHcPO&5GL_o9t}6R?}j70scpi89|{#k}_b;9pegykExRr-(7l0Y027|r7n?-ZCGv; zKh^f`+It5mLpTrp6NB)GK#>sn4g4Mbo8)e>o=F8^PMAYBL~p7IS7MhQIPm$}^V_Y}j zQ?D0qU06nivE{E0-Z8v*^*iQnT)_Rv4B>`TfnjcRXfS7m7@Ay2xA*R*k-#*%lLH*o z|G5lBzy*X{v}gGN3($9rDFzUs-1~J==Hh_8RcU`nAsH0&JG+G{ARcWEjg~b7Rc?G7gD zzY28-)gs%eqN0CdOWp@G5r;jgGrby{>d3ujkAUWRE?SdY1#~nFaMI2W38{*ePUJMW zcX(@QB`)9teUIVXjYz+(7LK`m4880vg0x%qT02d_3)pP>_M@Ga2WS%ejXY)$ zusthUv?tA<@Tf^5qVE`dQtyzA7TK&Q368V(*zH{r2w(AiDM-jg2+uVXA?s$RI&oWV zUvlVo`_fmk;T?|kS~Bv?6&;IWMUW;%ks&{@Vqpn}u!5onUhB&3L?^m~@pm_lrxxfb zQZz!dbkBJR`#hlweiKNN*{i8Ld5Z2TI%(MExX@1>Wv47Z_54snU#6T9wfwM&B6#o| zG6y(WW3-@@8Wf2rv2R4P#`lZXG{pNU+&q-!ChNteHeY2xhmpHXD!S^rVfFMumsefHf? z)+yxQ_2g**x*Vfr7VaI-ca!l}n`D@=*Q!;%*$n>#S<|1qcnHH|v+ZaTS;}vwt_44#vwyT{+clkt6?*1*b@GAQW4QMk6oaD7M$+l|9o}#ifCfo6v0)QT z_WR1g6J-8&ZJn0OjocnLly9Xitql109_X{zgTTtDMGyYxnOTrdfpL(2xfER7;XU+sFq zS;lMQrF>HFcFZnJHxgfh7XqhoGQI1mN9u$<@}+J$IzBDmaphr|a$VR|6IJEXqlB3D z`9h$pM4OEXz_J`q<uptFv)6owRR;(mBz@vmgXMfZAhoVDGd=Hj7PqX7TlR0P;)| z6}*JaCX}HKWfFu0@dD8spB$~ zYoI^5*Lsa$v#Fp`oJiaK=U>V#)ikM`pd-%sVf>>MoZTeql)O=s3x*O0>FFM8CrEk# z!J$p7AQe}TiC{v|9Ga3H06vt(f(ihg_THpyf8r>z{LAR2hzb*aZ!_ekbGzWfF<-n9WfE(LKZkUEw zVue!;%*l$x<^9fSF4`S6+z!uEwP&6*P488ocUe>@=j;z3Z4;*l4@LS~zM_Q}@5ze{ zB@=Vn@kJu#b>DDNwoTB9H=8Yl(6sA`2$G#@?!_LC>szVHd|%2tmqWWPWg6F+AiyV< z7*}gTF!E8rosU(^++JGIC2N;yleYw9v1O_Kpv%gUc?Z+v4R`m>5ejr{i-DT2Eh4T> z@9!4BfmP;rzGC*u2zp zeE(4{8H; zx^w-U!!&F=9(dA*NB%E|4j_Mi5F3YWJEkY@+5qDF_;nEvh29Ge8~wFAo%JmA>EE~j z$S4#0s-QPe`YaJdOq@m@T|Nwsml2X(8T=UkVKUDaKQ|2}xdRs~7W0?FZnvd331lCU zrdv0W)Ft@S9MvyzXv!k(xM>=BfTbR6)m0O|F%jCI{B!6+3nin>C7j9YMpQbNX-(zj zmT;_Rzg!%@0yyc`fArZAJh{Cu0>lt9AU-B1Kyh6ceA46UG|{h6u5Oa_sLWZ6`PE83 zxe-;_B64r>Yl65@Xoc66+uQVsxOeG$BFqar$I2{+CiGh;jnHPOay-AHv8j+KANi~D z$ez|P1X=#}Gq5*YSK|+@gH@&!|DE)jJ?h9)`rKze*wDdKcfxE|p0=!~OOQcB@k$n{ zvgUGT?dFnwZPc3uLFWzV|B4nP0WAtPkL+`@R%9~d?v7IbR}zkrZX&yH|PJ)ZJUd5S;XxPq;mPV3~&B?oxDJwbeU{+i0?~*tDx&edE z8f!ZZ!z#p)z<~&(9LEya)r9Vd7;X2yTwo3GEG|IxOYCFrYcluIYUdn|0q5q1+mk4ZIiP&( zO3gOo>pg{*XTj8taL_M++nwGNYI|>Lk`fztR^oH~JM5R2A7ALYUY>h(v8BC8v!_S} zE`uch$0!uI#3)jYOZ(m}qn))jsx(giU`$@?MV!_a*-Gq483IUdUMb;SGNZ%117y@78&?7jc?rv)puUGwgEoPzO9j zLX>Ux85Zt9ldY|vKEq>mVmIb{m1XU5 z_W}#eEBrM4^;eToCP2ngnJN%_pW$xJvI*@*@!8=O8sSSr3KAb&xkm|$W2RvkIaFFJ z(#eZZn}$W1_q_OB5SNnpgbod3N$4E-dJRC>1&D61zB;iuM6RczTQCVSANtGyjbG4B zp-)c&)Dt=~k-zvG%W&xRdeftg-Nrg(JSmN+WUD7}IXe<~w|}^eoCdV1`?Bvs%n z2;%7u@Z%g%(BF+z4lXSI0sDsp(u|$$Pa_w_FIB*ztyvW8SxV?UKou_Namh65-ql45 zX9TW*wrvNZk^$_;a&6{)?Dpb1!R`y$$W&XB@mv!UKHDjzd~&gP!ljX>=6{SNr=lI} zo2KayrRS7)QwRivYV=!ZL<4&dCL1-RVDZG3Iz$4U!XeZxG}*+Hx<$jHTxs2@Z6`o{ zTL952YZCccAzLFI-MQ%*vq|ayp0hcTb{pL(XZw5L5NeiKvVOB0w6%&z9~yb#7Am^fhp#mi`#= z_<`IXYO7p~SITT$BgbT`f^MDUvqr}Vex-iYXnpN1+a_{NoLzWIO3D5G?ZM(5+#LBEi6_9l8eg&k@IB8M8t;DsXCe%7Y)a(`g#P7gwi}=yG1Z^F9JRJ zX#f!fV&)M(8rx>KA%~$F39A6&7=VbUQPIear${V~IupON@1)Wj1A}FsvcQ(X8L?07 zm@dij^bsll&jRo&WhWq74xHhqTkm?7{)wNX_q=@RxzU&JS-vYW?#{=rQxz6(&x@n! zdIhj|_l@il+N-_B(GZKV7rpUEPC@x<{-+ks6Nyf#n;*F#tbG9lo=4-N=iqwAfbJ-G zcY^d8Z5^P@Ea%mj^6t^r?XcP^fOVCUUVaFs=_C{@^LnFczpG@NLtx~!#yidS>V(ci zWmxOU-*oN8>Nw4RJ1sj$eg~{FALbqh+D;dpN|M<)z#I(;+g!BERAri+EaMd6VJU{2 zYC1?c2%rVDtO+vG2y(Wez!PD1$`<+<)wU|xc1-^%QE%5if6}&FJ!q`E zh2IW2K}aAJGW@oeq$4Ngg>|fZy~jNmV2jIT$8mQ(&vqN zKH-G-#CbH?Y7(*ndY`pnoMD?(c0H-s`l>jr*^emL5jUmAlbBLLIId_p+gcF3|5Uc_ zWOop(NvJqMPGlXDzthv4DlqNU+nsjyrazVjX3IyzNbDvH*y|ixtYotGJCCZOnoF|Z z__%tGCRP8-FTPv*SebD>QMFanP=&BKyGFcvnL8!J`>k6*%WqN+Dfcyp>Awm7&YyZZpC2+ZKYyTQ`aJZ)w|`=W z36txSTTnFe?P2i2KW0Jn{z34wf5mlFT!mT5+o&4Gqa6ZP3RW{FI>ZB`_b`v$<$<`e z!+V!URM%prTT|3_>x9!FcOma0gSxD!+lq#(4p2;gZtJPN&}x3KkfN1E{{~9m!!-Zn_Fts#=4S9cmZ$N47^iJk^;0a>Ay&M7x z*fo$Xq(gH4%ty5)Rxe`kp9~S<-l|dTU#aSRzpw{zeDM<-8I3J^5-17$9WLPi<~}d& zBEf<1`K+Fy_lTQ^1_Uy6G<5s>Q1wdJKtg~$PPpZuHBLsyTh>wvi0y@E!Y>aU5rz;T zC>iJSfDl*C6jQ;pRdn)dU6TX8iUk3XoP=(;0ptS&Hk*#`b4W&11P!}w0>BjNys@G5 z#;73MK{Gy^flNDPrGJXm9eRBS??+vnr_)spqr`H^foJ2$M3t->E z8OMf$4||#p8_8!!ZZ_4?6M?Sgo1@z7R)29@!O0M7|paalN1Sv=cuyx;xAhz`6i*YWnt#WbxD6 z$KYwuwf)IAHWMGF{IYq#dbh?`Oh)bVTxp>SaTT{|_o=S&6GoL3!Om&RC>g&K1P_v+ z(!G6k+|Oofl?R4Ggc`=9->IsbA$e|dCkfUkT@?ovOb1Flq|TedU+7mDqeIdX%=yZ?vxIQI2<_S}1R$9Y|6TunZ2xrQ~R6k1Nd zWQ*%3SJNdHKbyUP^dr+4fHi-4a1Re+4j4KcTRBZRUWB?Ll`Q^z_uQw!Zm4*=b3f@z zoGNI<()0AO_CAYqDO#u_q`*pmAFpb~eYBkMSDxio-D2b4qt53)6Kp=ux~uRB(GHmh z^ovWf@FYI}mdpBx)*SD0Wl%T8?AG+$cY)xUQ&)rgeztcGdq_7uRLXs{LIQk5qLt`- zI@zE5=mSY_%(GNB{@S6&6(|3N)fUX~JIS|)-up-clQW~KC#o0O_7V`y-|=@v6YUUvmjxFokd*o3gjNw3otwmc?mo0&rv@C$fbX9pbG-K?I+z@{&C zyRFhP$wm$4w!LMR^xN9%Yi%^pf2FUYMej0NQVk0lt{><1z8q~$konJnh?pI5U9^jP zx9T^)$!{$*evq1{QsIBvGAh`1f*n;Lh*#HmG1|k{lsx0$QLzx8xRPb1kt6Cjt1G<{t_ua@0|sM~gnH>MNvcX$8RwLq8D?O(zDyyKFB?=2HG zBzM10OBW+9G6rK|cQ zMv2jWbEffminuFAFySAeOd*YYY=e=n<&(U{o;zcSRNhOz^n#Rci=F`oe!s!na4R^G zd?m~U8oB6puB+#dxvE0GaZ*59c@9DcxqZ)CuLRu>yTjk6eKEc*E{M&@P}|RHyHoJL zdLjL`d%V@|gE9dXo};eYq3lM@M<<2I%gha=&Du!c5!Rv%F#t|iYYhwj_)F##*Up;K zGRDoA#{))Lho5Nq+A$%T?lbe33I`7X5u5KQ zv_1Q_!ogn$Sr&{P1PGpseipP7|68j*h_Cp%>#|pMtBY-gD2Lf~Zr9N5+sV5BIsQ-E zP9&E4tv3EMqwKT_=auk5H(yozeY^_mc96)ZK-+gm@Oh`hne!xK9O!(1?4CQ?ft>We zOi=6DfMvrR0Jb32w3+lbYBIFR-H$00y4nY~02Ht8Q)zRZ@~6r@WxF-rG(zvz+|O?o zfRIIA?7jKt-zd*A6o&Ye4;Mhb`0gfuIUIid*D?-9-Sgb4Hq%iPMoOs?a zpt#%o?xlL20;Yxed^<|8^S0H&xh-^sLIr!L%VmcudCLJZtEUTMHY$(L1UhzNWwmqs zC%;I$o3bsy*1XE#Y>g+_VW9u(F9nyR$=gn#?$!o(JacL;MZfON)7$Lf#R$G4Z7cVN z6|rj7_%E2cx0t4?B)hvWVf!(w;tR&^@uBpN2+?%KOZiUC!5xOUC4ED);oj%2BdWpj z%x>`~Rg*FfXYzNIo%f|!RI8S1YLk8<)d1hws{*5rrQP>j@=ukH%iop$wBF7|hX(ey z^cca?{TJ3!zXZSvQ%!GYX$^cPu5y~@nH>1*UqL1(hXww_Is#g*T8KNZ&l$!WL*!75 zlg}HTtKC#It&R+Jc%OBYRdg<^0dJ?Rb84gC3BhnMTPm#54<~iDlQ|vN(^^Kh+4Pky z^Tu;fvgenp2WsliD?bTW6Q8}KP>eq38V|2qYZ zR(SLBw=x(nb-L9QK@{CM?tRn1IZqLKEQ{gEK{kEGm4$4M%U>uXR1>fSczygBgT|(9 z(d%}H)B`|$Tjv`vlw299>!*IFhso%HONys|_??Xjw+n^*hIz}%-B34@PxD9x+kdo` zeHK9L4xE=1<&3;%o0H>efnd|rPN(G9eUB@ekX!9n!l3OBb+_yOTzA^B5a7=mlbtA( zty39|iO{Qid$$; zGhkrA8RtG7`J&y}MVQ7cD8qvNv3PhF{MOX`#or$iMq-s&M_4Vt1t$kGU_~W!*&+|U zH#o@gq79+KSu=>$+&yPcLf3!U#Eji~zj>J4wf6hH9}T9Op^Nye*8lD;x(t;&A_K2` zzuqjz3hpM+dwmry?${jvh7#`>WjS3b^Uc4h;iKjI1K@sA1X^kAskTWV_Y>HYgT>pX zE|F|YwgKHjc01{T*^d{xJkQ=(wqGQB%5G*jS`G@a7*&_a*NzMm9@6!H59=`_-=^1# zwBX&)=Or|b4o{o6u5yZ)vN`jD)0EgVCw*Fg^S?xUu)Gv7u!L)0TD{H9-J z*`|$z(tfSRHkfiR>vA1y54l!@f*B|_7abSYpr~=@8~C48HP}mnP-l5wX8XRUfZnpo z)=1Ox=JkuVX2cP530J3Dz7~5C)w4m&0a@f!UyeW3#&qhGa8%l$LO+ z=>d7!i3I8kk1^XEYt)CPb<)=36=PvqX3jjD7wI6GG71;sW&pTEC{lnx1 z8JhLAaFbq(0G*d;Iu=r6TE&Wc3x@JxRy`edW4*L+jd`1ShZ?YDDhY^Zw>GM7LQ2<% zZi)}AHuEuAxX(s+r{AFcpU?9s-2&xbk-pWuj-m*8$o!ve!y5QzWhOs{G zjBay;$lorW2b{MF`N5*<0l8NdFQq4wrmpr=(tA9Mi8NNdB|5)H(E%h(d>`{wunQ2iya~3{~S&e&H5% zW2*`^}{s1h~n{WQ5<;o>Uy0a*Z% zYky$P+Kv}B{0X~=-PFhclWZuIo$(NRv?7bxg{SmjLWD-X+pq`4Ul6=ZzGaAzxMDxg zF_v1Co(lnBsLl`}qaQw-$-4{~+2XLo1$;h{YCOLckFBZ;w2~p{urcN9+jq)(y%N|& z92xiY#1+HY&5U0`?P;C7d;QP$)#r!J6sAIRwXIO zo31VD89I62#|0!J@E^=BT{z_yuyP!=!@sEDyK%bmAOVy07XK})U2H%RDxvJe=-2NVyO*1yPbs0>73hn}~vh zmGBkA1K{rfUVkp9SW_>OR3=3p#-=77>C8=CYm*1?p^38mMtHtINb%{FpUzJGu+?$9 zMUFn4Pm;bDmy!UD`azr%vm<6L2T=zq0Q5z8cOF1%r<=GaTfx1mLn;Uto2NNoStJ#S zUbR(tcGEqj!rCvM(zNEsH*U=6^HlTRu@=U4byxL}7P7@tl+!vYSe1thuSajg*KD1Mohax>6L{vA-`ON9^V-SoYmzL0@^tsGT-*hL{~l( zzPC!ZPDSlWcEavsB>(7ViW~hneWa7mIh-rB%l|=aB$cfS2fc~?FTk_{vL1Ni1krYn z+>kKtfPcM!eD&W_o4GA_>;EL46xY>Y<9C1`RKRy_8=*kfog_{5(i)3ac~3+UXfp%= z9{{81Pgi}Q3pt_?sqIfPKl-2Rr52=zeoGi&d&ic!RnfS7ge{t3OisBcGH$jY0cqqF zrv>tS+UOWfg~R{0t1sS0eMj#}maqmWSSJ_55VzPu69sxzD&Qb5JXUV+fZ_~}ldNX( zJs@Z@jAt{%bKX~Yl86XLarsNZR0I;}+;Q_3u*udGiH-bnpa<{Kysdd15I8vxg+F$D zGeP|jKH`DAs))ZVro{%HOIUYjtm<$nkgh|Mi;>iw37)uO|8%h||E^F41yG4EGxF(q zi-f9ea%UxpNPoUlN*xLrr~hDh{x%O!XU6sW#_2H(P7m@G#=byw-30F`T)z<)lmM%wpADZRLb-BvZ!!D5&IF5SKPqdHPWcsFbG#{Q@|>=PukVqK z4dN|(2|^~ui*4T|we##uMDqTtnctx|gQ=945R|=hf)BqQ3EZR(KA`VB8o>8~A=tUZ zxqsOfe^X8hZU2U3dXo}evOVQt!8`TY1v6ma(JL$MgqMJ+AkhG(WHgFU^9p<->c23>@zy@h7I57Kv^{t^t z@iLxxjV5O@^)q2|B729#Op9NCzMF}s0y`yW2v6R9+G!0nrm8G2stVy2B%?F?UX$&; zZXG4i`0M(DP!YamtMHIMXJC@8kNW-`&iuJdGn~OVnB%z863%$WAjnfzN*&C*5iuNg6p--2nnhC#RA}ipJ2+s*PZj43X~{?iz(5wpBoQn zcUhS5E4D6&1}QMC8;rSQ1$auR4h&W@u=s2S&1o@Tw13T}SN*KpP@OPCtUPvY(~q3^ z_`;*d?77ROBzlgQOMuw-;3(+!tu{plE%f!HY+QE#ldUyQl3E$f!s`M<*qhd|h^I8+ z*|(;~nB^{=XB&D3F-|Fm*Y)3b4OJ?btuKIP@@uMm?2`l}m6;E&0hvF?8*`i~ugNv8 zves*p_0zqPl&xp?$G(2Sm)-lm_0ltW-#ud{+iGyv95}rfXQ-Z5q>M4vd2oi;QR$6IpEw+h$Jux zv~d2GFv7s?RQ@4m?r5X5=nt+LEH42_dH8W6gQw(e4~vFD33XlKzz515hyFLj(MJ1( zn35)uKNa6Z#rx~|gtJ-}=zVL)Do?`PJICS=`-Zol)I^gDWDu4(z>`ElE29ZA%6pJ< zk(_&Ex74HQf1)n~aEqRog(@!Sv` z*CCH`9=8HCOGOmHmopgKs&t^$ApPqc^Xr2GNV;0`mfo=P#WvuZf%ZEA_`rL0-leyO zO`kMB3!}XUo#`?7kScP;2?^w76LK0O@s)$!oc=^>I2V7%7U?$IT_SK6`7y4~e)HbG z<}LH*Coiji!wUbb<#~QY!DH;N+#%_!oV$mHbpBzOANQ0R^*WSE5+FYGEL^QJ#~XfN z4NB49rucds6%h|F^h4Dy}03SvS zZ=E8q^bY13>I&iwfIKh6sHqJ7eKh9e+gSx)D z9B()Y8JJ_UUwx-frJr95uIh9{ALEV2Zc7;S0iJrAdzA94P zQhPYZB%`3)A3wkU>ViStflv8>SU4%aN&v5c1a3E5N-H$AIb{$k&G+dLkl=`_$-5|5uwMQK-8W z2(>DCalLNDCF)MGD`fF3M>f^rks>StQ~dC&EaLZ5<@2JOV+IQa+Srj!If&HD=xLOibR7E78QWQ*M z3kO(+K>64U5o`~ibmL}X9;*DjluLD67d~r)SwgzXae`m=JtmUWr3Sh}mrWuEJBCPq z(1r?6IYK|vR<0=O=@`Q1q{K`FjzAG{X0KHnUYPgw6`{_z9mpdw{wo_l-;re8jgrR& z9++2jVw!Gq`4n6`qaYHz%&{9_)B@PELx*RJ?D39nTlyXz-`0nzdXa~3QiiM&^&dI2 zss19ui%9S^h_XG1=}n#C(L&s-*D710KX?^7Jgoh<2;V1-IoH6N6XnqdWJDOQ)$ zbxP=|Bb9_VS|h_0CpMlGMtVhK^-wy#xY;Wz50%#fg;{#e;sJ$} z$d~4Qo@nM|O5+Q(=-Qr6u9-nPtysA$l}9{%Zo(FLI?mWs+mKI{@8F|2-$BCUnU#u% zjz%5oGkf7=+tGoCx(zF_4DD@YlndS?7seGPk3o07@HfD-F9|(TofIO&xHhyNj~VXt zhu&P``EGFGdyY?8Y~p)~+f;m0YZx5Arigqh_H|QE!nq(uB4Dn}?B=)*Eb!31iKT{_ z1~%A}5#uhg3u$Ne$Z6JYpUC{Hl`)$cWE!6-(yB6|LCV!uY`sf#PzL#pOh!)6n|u%L zQUl&69M1ePbw3sVeMwa4f8JtSQkX)n-V0It^tarM)5FB6W_Mo!gLpV^qxWzBlr1T; zsl1Hx;J=c0_E|^l13jjc3hz7bIv}b)!JJuhg0T{nBahSlI+@2Wio8*&;~M?k2pji- z3{zTT0rWKW&*#6g#O%0u!EPCrS{WZTXZYif9~L6FzCt>@04GY2udhcx2kFbq3hqj( z3Tk=M)#^~CDOH{XzmB-Mr5t?8acp~mO3&$y&c=%A^~r2X4A%Ppv?03tgQ6s^HhO1^EU~A_U3E3CbrFL2Dh6v=0kN*Hf?qazws}f=KC-`=A26 z^-i21h6OytD#jmZkpo`UP2rUAP<`7%L}py-%vl5 zP6Tw_URD;HdPUh?s_09nCU@Z!@WQ;s8VdWNmxO~42wkI)O?F*zvZzM^o*`K-kZPBn$r&)7D zUzPw#kg}=?8zbX>IC--q>^)pAcy{o`{;it|m`ht&$$o-iSXhVz{1G%_?}pSqd@&9d zMvy7kf)M6eDqC*k8vklE%mC@+P-z|z`dO}*+1x38)|*#2RDOP42x;eM8J`$kJEkYWF!x5>N~-1AOB|ekbrVAOS=H6Mb-5U3L*=jB(py3+?K2sK zbY?zV0AEYJM+E!1U^KQFUnx3?<+3-v{^8I$b>6g~Vpd_TubCv(52CDL9a`dUFMupiaNs;PlgRi(Xkw z1Dp0v8lxpL!En-kJQ2S(Ro6&O~b zn3r;}3z7{+sd2&7n`QEc;y(8KO_}@Wp~JeTUHg%IRgYqRl-F;6#2(>~ut*iL?teD7 zZRfIi2L-R6gTYg|Xd30-!x=*$2tnISeB4dQ%8%=RHjElwxw!O%(EB*WG>1&4U4$$5 z$8L(PVka~%)dOT7=(BQ~RaSf1L#hh*cC!_H+|A1Co&OH~E)p}K!5uE=8JZ~ag{`)q z&gASmH2X*=o;p_EX(8J?KO4NWle!ZBpn_po53T}CO$GbNP zx?R?cR_b^LwJWgrnVy$TlGEe=*=r?GQr*(>&J@c|PSOoD=umH%&DrKr6|?;KtNTEj z-=S(8U$-802m$~5n01)tO9n1XP%O&2tW#kHy+BA0+26rneB$v8`5{x-c5g@jp2;O2 zhtn6={^Wf$`Yy|>1g7@$0dd+DLZM^*>a*W)VbgN(e%NcBG`jV^NxS-AWT`%vRM>js zvgZOYd{YT5_RLY8jOY5wHU_w}JyeDaZ@}5Cf)m!`(5SN|yN>7`DO@D(gAou%z)XbB z{Of|qZ_@>z(&pT4xE<(p$Y1fGew6$YaX2RZ(y5<#g8#GSlTm{4g=kV;DDjm*@E%0x=$^$Z`D>5-g;8!NWghphY+aKx>CQ zLGkw8G9sp#b|!_%9em@8{D@+%Cf|U1_r1zN=$|#W(?Z1!v2&TkXs*OF#wPAfiiFfk z4`gpgf6+yhB&$FZ(rw%-iwu^cs9e>rPu`%6kB{Dp_uvEn=Vasc-<)oT0Cgx`X&TPI z+(oe9P>=1G7QV3vUY?qZvxGs`n|RfR{~|+tIA|rrX?A#Ot3C7es_yZszLeMS2!NhH zvZR}kl^+oLbu;58Y6K_q0a>c!seh484mBGQ`^cxXicjG6OB?o*a;$Bv9=Mh-N9rmT z(5#ouup!qMLeHOo`%bX1*MSA_q*;3AdUXB2HZP--7%4n5%&Mfd!>jX}HSdjHq$^{` zLI24s`(HBvcxWgq=D$R(J^Q0j=x&M$9avDmUqNIw%QqiVAwg4n4sYX9?N?JtG4m^_ zXk49km1k}>m%3X$c~z~&BjjakEy=`!omnW>>eRZid1O}E#x}?EzN#2Kmu{OK)c%P7SJ6Yz(Y6pMYmY$Vr0zTc|%DFX{M-*n&4_zX5*6!bvy(%u`PgV@(&MuD9duecuEv1pZ zHSED|*ZWC7LIZU>zRfPPTl8{g=w1Sz(hgpfwZpfy!q~H4Gte2$I(v$-sq=1pk`C(HeHeMMpD&fhx3p5^c( zOW^FL{uB30(0^6o$lyT^PT0VQyC>11#U_X_4VKA}$fzPkD%7*% zEtNw_`@062gx?{aqqzCF`XR^P0P~hxo5{LemLe&9da6G%QzU+;d6vwAvZW`1-Aw!N zPi0E&YT)$GF2p{DEmHYMW*XISv`jxQbK+&)MU6^<%o_a3GjR*v_C>de%%*=jTDXd? z*Ch`Zk_<<@#kk=(>6i!Pf1MYeVZ<$&0ISEg4IYrU{VIs}10(q37+Qd_(GQxS*F9h3 z`oeE0>HztGDKVx$w%(pw(%+kt> zRE!KnmbgZ+_!}DN4mvXhWp))2?XmiwD4|bl23Z5qR!F+fqhSBvc@NnZu)`h8r$ah_ z+(|SCN~L(Tr^QIg>XH{h4|f%fN8ef{oISSi>6yKmA@LRh)l0)T&#&Awr>CK$Sjm`v zIE&xs7$8@w&{^9gt)ZFk?34zsF>6@+btkrQI{02;R%Zi?QLI zVFTR%RcVf~o-e1yzy}r2uy7g$m-)B(4#8tq^1WJwDQl*5C6SzpkjOwz%)IxBq{|T5 z-gu7ug9W*D9qMhz26`f+pR!=byT3>=u=k0?SlIU=>D1m>DdwLkcO)Ug?6CKq{qvVq)# zLho44>K<;f9OSY+<;+i7)Pnh($Q2HL zV~}H*B`^jxA-5BK+UW1#2zG9V4fK*Ls(aTb!ekktw_xTHE2{8dLE>i|7V8ICiMdve zME~RG=3}Oh12O1Bt<+5+(bEMe`eq~hsx23NJ-dD2zd+I6J5ARlt85pHbc-?GnkeQ5b%DkYQpE(q)GHPr2C2Fn$Mzn_>qEe zTm(x26q(0=)orVIwWNsLqTt5-8)QaaeHvqe)}s%%qn4V2rm?ts&K=}^rUky$bs92u zs1R4n-2cXc;UIi_irExcz+>@R+#=7|iaYJaQUKhs!Yn}GtT1z;`*=TFILnB;vY`7RZvW*iW>)?vDaNT*@1hh_`v$t4Qzyy!|A1Xn|N1J^{-2LP! z38}C7`yNYaCj7G-lGn|2YW)BcLPi5^-3+?JF|zA0@>81!|HZuUZ9SM_P7>o2jMu|y zXWmkBx7UheQ(uBsDN>u;{A>pZi=S`r#x^#}Kc}mt+Vk%is!-K30Y7pn{Afcljbi%P zx4-ETz0@`Z@A<#gRQl?|e+51M_B19Etg)cmbp9$>S78lJcLYZ|tM`4#AwBd5SHG^B zMYFXY0{w?hoo&W`e>1wi$p5rLzkB#W-lcsEjKnuS$BAQS0E=VgFgV0~qZ0Q0OdLRK zi_iL7&XjE>8Lz#?LDm>N4L;sT7KLN|3btiq!Z^fFCnN*| zKVTySyg+}gUfhmC?H4JUz+aNehk?hmRcI9^4~+Mfj%mP}@5CNLwuDsx;uAo~6`?Zf zj?cdmD>rM)w_Ze*XxPHJrAPYqZ)KfwJ&`cl-KNe{0X4zl2MfDdkiwHey?`URKv7Bn zXFrxNN{)|idm7yr4*9P5ssi>i53l_OwPE(a1ZvkS8{=coV0apbei3lG$kq8(qW06$ zg<3BB7iE_~;NCHi2hyMG;M(k-v(XhZ z@YBRp(L}NYJo!&=$ta6%hg&}qd%U2709Awhw6d}p3$)AN>Jxy!|Lj!pQ;=535ddr8 zXi0(s_^I01i2S?^@;BpACHO&tFQn@GWf4hANZ}(ji5Frxfphv`9!8KJ5VL!Hhp@K} zuTq|Bd;vBw0eQX2T!ju(!uH2n8h;uZA2P~GR}B_LV9>rcZ;%1O3dBHD^yh;&FIo}M z3+{efeoP5{I}g8iwPoNlN1r17I;Za8~crSKQ4P^xjFI zLSj>DX+ZnY%D1cTT9fH>CSc_4EMsmKXI>6Q8oVbhR%=0dnLMaiyT>SIZj82+kstx0 zGalzs3;O#CP7ZF{q%#BN%l{_+$&a&U3O+ESVRVQgZw)-zIfwbuD~Hkp17N0JYc22X zH~&V-vOnwGSM8Z{Rzu{!ay}QD`0w55vLqB$!Q#Exc?_(2SoD~YWJdG?Qh5Cyz`lwT zCqmD`Js9j_L&n{G9H_H-#qevv5!t#uVVz(I^h};%WHInRTS5nyd$*?H`we)k&zm^! zoTN7AygJ6aoN=mp9v#r|_hhNg6cF3E8sJV&&5H)E+B$n(jW#)hj-l9KQWg?uBlL^t zO$dN>%2#1??^ig$PnG(F@YXhV`g{GeT;J7wuWtN(_w7w3p}p?7xXXI#QQZ`yZ@GcD zXWNR+x62N5M3^JJFQpieCo);s;0g4S8};(r1~ifrd}7Yig4ubid|E>JK*$ zuMiGYhCutMqTH#1PD4M1bYqLTj`xOHi_C*fC-8*d6%k;2^O%iS3O6NMLX8LcV*GX2 zx}7&08zW&vZ|Iz>HVZw`VK2^o%!z+MWz7K#)0P?GJme$Y&hD!cX!xlk8+&{*I(og=0-IOXx z_RP!9P?Qf6>4WSg@?8tOlilcFlh-oZ|I}V}`K7aWI z_EW}~d2EF)jI}6-^mb$48NRy-HKwm_7rZ>v#q-}v%lU^oRfQc!ttuS`ow@r3Gcvnz zimdx3Psy|2IEL_a(Yw9I-#yMS3h)#9Gi-Q6gll!ZJF*~smdyGDanl-ni>V}skZKXjb)po1Q%mVcJO zL_xQhFMT%kOC#2|($B7)#~LA^fE!p17wmknYTX0zPEBBhMw#?Lu^Ba<68n((Ig!BYI1}!L3!%6g7Uj{{6zVJLP*xnFld_GFLj5f&4QroADir&|X&?>T1!w5Gp6O+_~LW{@Jj6zM6ju$i_C}tS8mb5eYl3mLM#~v{}*|GzF z{ZSUiI9NpHxDA8{Srn8Ujxh%`A+v&=%`8e~(SCAh(!Sgt=q_dvO##aec2<4G%_7eT z(UPK`oVS)nXZI=M;{1mfS@VR-4h5gK4tfT z;0G}>o6{+|k2V#RdxkpDEEx!@mk0}v_YIlSog*mMA1Q^2yNDSJzW$-BV=i)?K6?1{ zgA3@1`=u@AFQvk*YevzNX^)j;L(`?5{r9%+F%P0GFGE(GB-gE|CyeB<1|s8pn*Sy# z@-~nIl7=SPCVbt!)gW4hG|5#L4(Kh7g3z;~*Bj=D7C8n~%Vct|%h0<~=g!>}SG3-U z&TGu5Xrxf!QY%t5KcqJlz8i@pii!pPx5oFITo&n4nf(68%advpd9gty!;fp{_NF3| z+}{Pq6DmG2gDOEU?G>l;9ZvX(t*5xuZj?66LqtJ0eB}Idb${3p4bSeFu%D^rtd+2qyYRgYgkjq%BgRqcn~8)lbkJO->Xhv6B$iAwqwq27(g zK!D|&%m2aD^uOz4bYcZK#zrGrj{Nt=gpOjLXy|NHmUX&|Yg|j>uUx7G2A4kjHt=yS zN$cJ@aYqs!C|8WXqP<#5_mhHXvUTSzBupLT6-ELT))Yk)2&CDpa~ZxXqtvaLsnBK& zaT5eg-LGt*_X<8FJ}T474C_fkzVZ18`c_33-LN!Nk}Q6~`9s?yc-USyC+Q1ZXw#-_ z3^PUEl<<9JNCHaRdBg(FDo3ydz6#R%0F?oVv!eA3%( z?zL^P5BuFV$Gz4wSeqU=x2s0bnY91w7n*ntSohh{AY}IWok`Nz*#P;43TunN4(!gy$+KJLRq3%+*+9kwsM@m!s)EOlMVX%*Y>_WQ7(!j~*2a-V~wo?`Uk{Y$%Y)zGP55=;z}cB}~DZ@#Sn z4cLoaB0r}uLx&jt;6=set=?4a)iGoo0WW;FCU}an*x3o^RysVFqZ#`F0j^=B0yS`| z6=!J2+Tbn~c7mRF-05l|ON_*>-h~4TdeNNAO0q;DLILP20AfgGx}Ws%M|64XWvN_5 z`QpWJfyp|StOa8XnWzcrzOM@CkHpmGJM>~<-Tb($@xK9bC*1su-HH`QNyi)eKNXSU z|H83EXSjb87J9~?2Hq*YUJ}bq1bTs#IO75YyvGP>bYbBA(fg_1g|rorIu zQ{$0*3LV>E>ZFe@%}*!k*nZIFiqEIj%o+H?DLW8pD||MX zO1$I<&V5+j2fOA5ehY}XJ9j{zioY-TmF+P-yee&%l&JU!d;EAGKQm_8zNua_SS0!j z;!h+ffcAw4W&y~2ah~j(Jiv}0DaljL@*Gks44z|6muzxuB9*Ft2{b*oFN;#Vsn^5g zukBNCu6xw2w;n5f9TFG%@>qYTY#TgMqBvOa;>6|IpD4s&Rf*lfZ*BPbSE^sV6!O=v zAO#l4w={UuX^EQQ%znk1(FCiQn{r$ED0qQg7Zs*2kKeZT6n;gLNhTpM-r+B}O};u- zne>1VZIKWL2Y(3A6U);17o}uMw^Ok7$F$nHZ3G!pDuK&fpdHY*TF z%vJAASSg`W&VJFnvcU=+9>j~p%0=jNd#dhJF!h7(@+msrT$568LsJL0m2IBa@DDcu`&Pu+B~s6YCwk>0;~o+f?Xg6Z#3b;>GyO zuSx7vu1>bJcNPj(Lt|Diw?%5-eGGLI7@}+8dWs(U%ZqCaEBR>)x_a=(+yw;~XTwsb z@=yfyy!r$Nar4h9R0-4t6s6Pel>a3%#t?WM#C7LgUq@~^||#^ zlRLH+c*)7>O#9KE8QNM(SnvscxVnxPNjV&!ChwT~mqP<0UoRZMJ0)@TT)CsuBqfNX z@%cGGhnB82SaP41QRk@B{~W7df*Lz@AZ$RQHdS|b*us1xbj`qL)4gyYf;$J9;@Ct> z>z+&~=3$PRsu_%kYhV4ASwX2vr)rz&*qo?@PLn=k(8Thf!A;C`PrKy}zdw?OpI>g1 z*X7!JvjkoXU)bt@RzCUa@%Ig(!A1oXecw(~4S|TLKPiW7xrM-_D4DMoV51fz+)U z#uZsra(wqTPJwz%NvOCkW0x~tW7=F^Np52Z-zPYM!kYHS7&RTzbOP}(txrB~g9Q-Z z&hH-s#At?V8@k|0v>C1NviB{gC90tBACOg8EUvfI1M8yRr?{0xMS|ybQr~P1s zV|$1cWiCmMmuXnm_y9ARXho?af6}(p+c$h_vPQ<6`Yl z)Ui=%4ia;}0QJI3!A1cbG0?#I9iXUx?9)RPkilGS4ChIlZoFZ%k}cTp@LVKf6P=kS$Na9han zIOX1D+_=Pw*x%k~&dot83?u(VKMCi~39vBQ)eh}E=sUf1xfOShW)X}IJNLV5qs6j( zm3jFTRyryWZv!fC^tl|*Uy9aP8GU7~!-HCOshE4XHS2zS?hZBcT`2W$bcZ@>wACemsKm>L5A+iP{Gtnho( z@jbn4mlS$g1cAII|7_=sRVeb*qZ`bou(X7wz1mArP~6!1uOsiL=8!re8Uj>%$q+%*0HS zcHgT`@B_T6cI4yR7=m_j<>J1Lg>EZMQ`=Buqj#AXQDB$uG21LuWn*qT{sn zDV9^A?Hk)s0t0VeS{Zrf#jIS?8(Eb(Hu4UhxA<4yNqX+2`qOA$2C%v1+m|aaup?@Xb`R?^)^C6<8IdDI#1q z4*giTVRtc_iwUL{u4Rf@zv{f(etqNoIvRt-#(<~^HHamAW$%RqyFD_%zZ%q|sRaPf z6tx(koMvHq-pj9oP^p~p-%nj%u*@_NDEgMFUQ~R02-m3C&-FCkN`4D9Yz)w&BBL*5S z#93!Y1$sjV%CDHz`T_B3W{l@S<~{;CH^?P}`w83Nw(hm(=i84To5fTG)8zd9^)9}V zit^PlmV~5oSv2rRm?_2v^zfsj)3s}T1IaoP3K*3j=2|h%vBNM%P?f=U#hci6?2E(} z`@`fO+6mYf@xN20iglFfD+KL?xM?Qb9E52Eu|mmIh~LjAO#C+QQRNwyxBn101g~BC zI`o6g#zqWq=q#RP*8789dTjO_2i*J(OEe-)g)da9e`k!KVd?oWO9{rlqf%wX3k+7ImjV9Ki7uO>2eC2(&F41%?*$OH&>4A$ z+*e37(MiAdLx1^o+{+ORz7|#iW39o=C!1k}yzSH=Lcz~6^33h0j?Oy^OEq1-B_m=J#WUClI@5KM{!bRYEh--b-A@n&+16lEvHdpFk z3m|bK_%;syEBl}x&cMfgeHI)#^>txQieWXm!#iL$gFjds5Nynju95j?ANcJ1HLqXw zfrU_w3T4k^cIZ@b?BD{F&peGBvrP4xj!@?kh{jNY9L z%l4Kxjf;=O3bnm6O7NHvcNc8`%RR9U+TW;b-#)4XyGsapMTBWOUt-;T&I8Y9XsG_w zM-9MTAR{J!ClsNuE9cBOm~u?qc{je4!T;;ovV7-Bp_Jb8zREr*H)x;~ZWU2YNu~h!gQM{$ zl%sgsEP5WgB%Ce+a@QD0rzJ`*o7J-{{WUzS?0cDd@E;BG(>(W#P`${U9w^LOVc{H=0)*mld_A!`ket{_%9D(Nl_?bu!5Z*Ax@Qp{3KI@qFn*U z*H=HCnqIk?H%+BC$?>>^Foq-d5OY9B2K;eSlT-PRiNY6L_O2vW-*TIwc^hw00rGy(sjE< zf9=K_=3iCGpbf6>ML{F1G0g`YQhP^POLd~dn|Nnw#USc$wxi8Mb!0cQ@h&q!XXMfy^mfB*rGi2p{uI72mTvj^bJ!JpZX)U%P^J-}OZR9u2EuV|Qmu}+yD37xOYS-K}mmUjQGA;kV4LJgvUK0dybn^h>Mfv+;-a7 zY-ceQD&N`NsP*r(GXw|vkkfKIWY=Z=UP1(XSx8>-wQF{iR6_s^+g7nkNI@GNSRYqB z{E>9>x`cpn7gt-n!KY^Gtcjg++`=PU+iS| zaWS(M-4>STYlp>TQbM-1RzGPOh(ubgLV!^mW^(N>!uT{{hLGy;|5oS=;%{nT4q6%+ zilK9{1TPD$zfcn{;Be0>FW5PF1z5J!XG^! zy%l2gb_e$6ocK0Zi62e;6`5@LmPsnAoqr5&rP|D~+0#Ia6;S?*!22M#-4ie$3z&<0 zjuK)O3dNC_i17~6lp=Qy5Myd2xH z-QWBE+}CxU`?b=8nrQVb9IloGLYfo%wRm;6=Q?LNX z48wP~aZ4Ph-)xEk7#(iz<45+)P#d(g8bbg=v>f&N%cgZ=JM8P4gctvrT740f0-$&C+yk0nWNJ~F>a>bd(FD9 z$xJn%BFil3FvIM*tO%vSRdqrkC)NG-6KbH>!EiUmKtGQp??kOvwRZR3wa86ux8&FT zjhzoNZPg<1U>EE2K(PSMK9{81X6yVb{TyDYUbgz3Q-+<- zzf%x&Jy?W3;CJ{C`Lv{y+2Zz?x8D1NB-8SnI1sh|MmykUou6L;tl7GvZp?8gr05EV zM9jamS0t`$>Zenj#C;6!)k9zSaI}J?`>VHV;ziuO}D`Z z`16vAaZBG~4e`aRZEr*c>3_GzqzZ;9u{&k$mIhuZo#n{@QeXf=J+w&?aObo8C&Hdd zJUD>_lB9@oy-esWPC9$cac64SrFE`5IP~to=mha{RR9A<60@#jj1G&-YqTOA3)9R& zgSZ=j$LcmsPv5B}lXs)E%inQ_DmOQ=LH$*cOjfQm1d4H0)8*g5XL!uutN6iaL044K zf!f?w3s$f@^kvPx|Bv$7vDV=|BT3+Ed5l01J3B499ewr74G3a*&+U+rAhr18<5!xP zRhPIFB<^B@)M_1q;IW|N_&)T01qyF~PR{T2(S?bM6?)fgX$1A#Uxd5_DwSu8kuayI zSWdml064-Y(rah@U|8U@j3T)*#TV$-=2R52sbPS}RuBh*^qoe%o=LeTD7!yz^aCG+ zz+{Itol^6Y%&daaPFdb7{gGgMZljT~BAu#|hEc#|W7Ln@1?#qG6^3=kA-&#IQaK-h zTs_N(2X-2rj4xK`BicnHeayy$+_(WE_Z*o=(%Gw&5?~L zvEJ{IgT^eif){`WI|l+Ox6QH^++H(!p8shU;;Ear~Q~2J2Q~` z*9zmWtRYH20V*xBZf>YF)aJw75w(peEj>2~PoU@)=_C1k8@}|ZvvIk9Z|{!G2$K8b zU6t9Rlz!8JC%3vMvCM)gm`8gb)L-wIThFQrXT&>jqwD6gKCO2XaCs&_U9Rp?gPyG0 zjl#|Er7v69ifqw%y%Y~emCAI|G4%WK*K)tVQ+&)aC3B2)1@{gr4GSa&SQMk? zq8ak33LA!@&=xl;X(aqbL+z9gJIT%8T4~?JnI4b{VXbiGo@%p^s@hi^mtUE;D(7!o_utN@p$^lsqvGZ7aSy z;m6c$^XY-psgquC%H}ZY$FVG!^)xnT2 z<3Sf(k}c=@o+yAIX&knX`>9E6^dFWIExP$LdZa*;6@I`G`!xf7R8v7P$jk3}R-!@P z8nWW%Vmokt5yiIkLM|LHk4UkVQKUl`;waKM7f&##8QS9-QC9tnI!qsMZ;s=;vW!KP zXB`>f?BvY{g%SP@uWr6n5Wl!63kdrvNIQuNT9I4tan!ZM-8t(cvpd~A5)Z6FkLF@K z=#L(l_;$^4#tCYaBLe|*RTp1(#z!(;8Sf>0X zTBA1sn+**6+Dr!F*S&zQpr>slS96bE9|LA$?_C&wNnni6bk>f+jGiZ4?8jU#pvt{J zxIp&A>xZCr{O0X&6GUjXOeS#&D-`Sg-3gXclvJc@)_b6?{U-)yEIaa84)k0vg^^p2 zFzEy^kpw|YYEh$#r4l#b3z*oXMvNFLANqsi@Q*%a6tZIVht!e_$Y(K>&Esp{GIZHY zp>tQQN2FLKq}M6K+VL$)Vv{Vl9#y#hV+;c5(WIh!6xo5_MO7H-=Nj>Y3s#^|b4HZC zXIhcC{n2Osso`AXm*8&k7rl7`H=Fc8=E)~`Qx9;uE7eA3q=bTJ9%` zN*^vm>2|9qc}q%GBe6VM~`VkI5LWk#|;HUf>wNMRr*5@huuxXYaDc_I{)cya@iYSJijk8ELT^s zbt;*Rd`;M^-2aH_okfs(s$omORsE1Rs_$148BhH2;7j$vpl|I;YKeNdwP!sC@He92 zIVIFIinU$;7&T>TVyJY+V1bj$lKBP(gF#K(#_*4(hZbK)sNzJ>Q&>aBe*yB-2ceNL zK%sGoT|27K;WI1)onXxW&@?5JqSrvmTH@XpM(-e4lq6D5^K_0@=oQtF&qoCP9o&Bh zU*$Dte}~IWwwogN15!!k^Un&5@9KIUDO0$GK|@bk8$aLaN&ZxD^{;vpZQ3&{grm6` z+9mt=%>Ka_RuRQ(4;=yAk(ZM*lAfq$|XS~0C<}kg2K)UptI9Dh9x->r=^8dBX*CT zg#QTTQaaOic^3Z1Wl~3hd6^LL{!bZ=!RljL5%Jqb0?? zvq7&S=iA07a>sReNmv{qwimw?un_{q`YoK%h8?ocwgTfe9(X%c_Q2GKIY&CjtmUVb{J6ypw)cG@cg(hV_9|rb>%h1% za{0|WiPhlIvCjPCw4+*m96kAieuSu1Kvzc1&qMxeqvRkJHHK_}SND&Wh`HmQxFl)~ zp{Bz+cO1UfyEjdi``ZX3f6-3V`DkD+w$QM)_IpKiXDbiY{0fde{qPqProao0NbXd9 zXFCup@pn98gi6g&t-9U6=bB53=qbLs?NbvKZ^c0Y0N2R=rQ1w~l{4R7pjjgmdw}9h zJkRs_O!WB`l28HmSYs)gcbAQ*`i9}ADWhCh_mxc#nB*FlxdgE=tnM z%?R4J?}9VWkMq2AVpBO^jFR_FX7A$XD-(PNkD4aBQ?BaArl`3DYZ}CcRG89ROx+my z_I?Hs=Z)o-6nC!Rtbu9f&lpT!5+ZlpfFIrd4zZfx|I`}3Rf1>3UGVqh^4620h*6p< zNT_c9Q3G}g6WeKl_4o;wMo^P>&hDLfgXKKesfr|$@Pp=Nf->YYYf?^@thmUiTz$B1 zgL`ABNJ*mJ%1k`5Sv}&52e>5n@~susm_FcEG-4qw)rX+FPd4t>PCpd3p0aG*=-llr z)s#xl596qkL6Y|k9H*cV zxcD>Zm3agqI1x0y-XMZ}6z&bKaM0;>xP^A#3jzBc6xQEghDP34!F_hh zX^IZ`33D8X*TIQ@XAzG*_X()zG=@ecFBh~uwh?-LX3k$$G>tf2sp4E^7lV?(?|sGd zy-`JD2MMeI4F*bv2i7QA#S|(5;WVk2_PmR;hDn4MGwt6_gl4mrT35v{Rvu#1c<>O3e_C0E4qDUjr4+-vhsZbM0tK0`a7S++1) z6CkoRlAc4-LW@oQ)UC5j$Hc99a-4$M+#Q|0Cpv8gZAR~~HOXzVMWz~PzM1fbleu4; zQ-;FdO;xqgjqB48?{^iC3oGdPP9pUE^gx5Tp2SDZd!Ev4XT?|L%xhJPtiRHkIEl)i zu~F;T$A_;t(5*z;!s-P8v!~;Di=#A%@uXN*r%>Ea1ci!*jyw&QGyZhI_==XQe6vKnGV3M+J+n zGRoI+5|(iQN2|NG*!b^91+lg!6F209e|LR644G|Y4o{r-(jHD?g?wOHxPsFUCGy^I z$9C5L)bpju{PdJ%H4vCRCn4FOMj|6$-KdQ=xwOJeFYirE8?^0DjL0ai30Xa?9g|~W z1owaUv}VJwLJ+VIkMC??8O*u$gCwY(0p{E!*q|&KM_}7yC$*uwj1MYBHjS)7i26n8 z*=(nQvGIqx^A_>)7^>gSO#V^|QwSmS!JnZ4`mP70c+LHAxxL1DY&`S63$D}hA3={h z)jZd2wkBaHO$DwY+Dyr`_fg~2J!|v1m-_q9@G|v@LJd?NAci+$@rh?0N0~#w)?3ZV zc+@x4ckUv>mQU$T&hK%*ryh4J%~^v#(J^1g?eCgSq*g$f&{L-MspO$=-46HWuLHM`(yM=U z(Sob{ROy5R?|;t;q#y533AL<`Q1^!G+W=#EV_PAc`H4oQYHA#M%wUR7H#+zR?@ zUVrK!a({7I(e>sN2W81r9B#b6IM>7HfQQf41MYw5si;!FyVAX)G(YC&e3yM;=|Uy{`ExQ0Ll=sBN#v5b23Z!gkan z_J7kqe7{sb&rltYKVT6VelX)=fcJLjl?C1U?aWYF$Gpk@6bH_0MBUK8j*De=vn0$E z+#5V32fpjrSPecB+MH=0YhEFGgO=;&*D9ZMfCpd_z~MbV^(`Ou#HVhB+^jVqwgnhA zwc+~5xe)3qVRXbqoL{2i;opw^@rY}mA(k5~e;^*KM~*Kx2eQD(YO}*-(K+urhg0e7 zwm%cFeZT)g?Hux^EN_22@6+IuGJ1@(cfGol(LrKt*0J?l9L+zl!83vsf_tc6M+*o! zYNxT?6BWp~HgyWTlS|xIm={LBm-M`ie+R}tz8GqzXw>ICG~f0;Rk-<;{Z!j3p8O88 zCsr+lmIP~PKMsw5nE{B%2`+p}xpbTSKD1(Bt{I&9uDN{B`)@s`QdQ;YNTBNZ=ZB{q zC2fDr1v8wDiNO@pP?)IHWKTqGgqfbOTxc+`-FXB}i%RYKD^Ce(W6~pqYViJcVmzYK z65SgAxP5B@WF&q_*AML{ZrCTc-O7plszjf~qX>$bijDZp?k(E73Ddj_;8eWzjwowk z!OeCdHFr-g3(Gv5eFmH%Fia>R^>GxtLePZx9Vq?zh`5K-&-_?K)oWAhJ?4OfVt_-? z!sNKI!85i~M9UCZ9=)6%+SsRnO68q3Fkhk9atR&qtotQN^22iHlx+h%@6#b)0W$dW zgz&SHlpqF+(m4cv0ogObeGm+9cr22uh<7d-=?>{^cI7Q<-+O8v9m03Djx|T23 zx^$=YJ}G9>;|tp{lKksB?ZzWRutto$T{lV;v+pzSKU5<5ZoTrZcjn8 zc&x_3G30T2f{9~(bu6B#)EX>(szvVV5VUhi4#EU`WQkj)NDkRnp=zFqjGpt0cpneNq5 z6V1CV^RAGG0rAdwdbm&Y@O);5ut8(u@sHoi#7W2^=x)`T3aa{y&9~!9n*aW6Pk;^z z{(S7JL_N%hu+rwmHnesJ=V(Abh)@#cQiG0V^?yG5>j>A+TtReg4Kfv% zZ0)97mL`!GDE!pE(ofh>R1CdBCN1nFg)9VRsJZ1CXcYDv-CM)O0%O>qBRTSNC-FGF z5ys-l(y>0>m74urfgpos1!8eNOPz94$H>Sa^lNZ%dV2vLX?U`X35HH9gid^SagwO( zAi=Mu1woJb{dmB&9NEzLK=vMn;?d*tOHyk{I|f$p1w_3zt9J z6ZPP*;sgvv{Tu8{V*M0<1f>!7;`kmc@2QlKnQxK03*Sx>bV#(*{J8auHaY>9vGdSQ zdb3-irnmAr%Pu_x-?q}?Z!@$fIn+RVy!l=nEE7%2RqZCRF@CJ^`~rIS@Syo&mX*k| zpQUq~695^bZ^da&P>fypaunz?Jd0fAN|=C=przi!?2U z$cz9Bjv$ak>f4M$)IRcKx@&{cAMzXTC6SvVe*;n<>hn zX&ODKnG@)^Gl`cGPIHP^6R7WFp&cVsKqT>EqUoa<=U266&Z$g@Jmz~c2q(_DsOisp zemxtYI7X@9L3PB>@;R?NR?n{$;HxhZmjK5X=j}>=#X;doTJYL1p9F$4mp!rXf_)K- z7{h`+z8ig+{Na2s!upM`s)UtLBZH)H3_0^3NURbPl`-F6rE&v`dFG=DBF4>K4A&#z(!yfzEuIEV03oXJRIU&6?|%~odB05<0B5x7k1W3Fyy>C^r*YrZ!X=_f zt4=poyr4GZVGu+kYN`)(H<~Q&T0d-jwu<>~np&UZ8LLJ~y+JbJLd+)+X<&rPh`1yC z^${k$X)>b1f-fI3A_0PAsD;rXN zrK&ouuSR*8zx<^Gez70mi77R9o1K;cn-Svz-#>moOl!n(Q679RkPm#x$g)3C`n}X_ zz(vQUxXneR8Jx-b&M?a*(RSe52qFlp1Gc=c2Gc^T9{TPS9P6z5u_i7DPuT%!38Nfr*cD9)j)Goao*B zEYk5FI*aXfyy{1Uojo1udQ?`AI&UPEUjCt^#!Et3<)KUh;x4E-x|Qa<&BA1r=a7{q zP=E(!+)E%hY+KlYm?k*zKd3~&YJEl0fD4hL5Dek}_SM-}l7-i@zk>Ncp-tDo*y{nR z%BoJH`#6#SDM}n7F$F@1)o(j&?W=MA{J3d>HA$adZot4h-yc;t$!Db$DuNl@eyVb= z2|RpL^hN#Jq1d)w@A;f1s|a`y-u7Y4QRR6$dn>z4f=z1im@_YX@C7Wm6 zR3vL`K8P`(e-nsw7qxtyS;Ry z`2oH8*ER$XD;q|s+8bf!5hw5|e4)Py7#Yy)Oh|M>g5AGP*iyTDiVnE#WxepL0jLiN zrm|-5N>%XXh3yF|B&^Bi+aTO&ZXk-&0fyi-A!$cE7NMtCrE#{=jWY{#S@{Ncdd&mF zGeUHnvUgIKURsgSUk!})KrudR%*$fy)4N(aU|0xz+yPjbM5HeyefIi=Q2u! zlq1F1oZg#4Gf?2p*9VDPWl=OchD)*a?~kQJG*-~(wCqQG!(LQi3AxC>ejJY#!hZxa?GHQF@%TXq3mylQt$7qXo`!7PeIWu5``wX`9h}_| z2@s#}pC?W-1rB#LMR>z3-8d<$9jLvK5Hv@|QGB$n=Zo6Z<$uG2BKxchyuVRv{e*MV__{$L` zp&8XD*%{j~0J)b=EgRq=A(nYa={QEz79Uum1CQFOA7Tl>h9h2A9y2RONSEm)Lfuhp zOi;$TuAY|y)h3|@w)Yv(OGF524BYu)>Z2k~&R@II2mVY{p*{NG+ZpKkrFjP(T)H~&uz)wK4U$rGa z!|ywCd|`2-STpB+#zS5>)hMIgz%VaQMleSH16pah%{tbbFB~Zfplgk?|iO* zku+v@#DXaaA~Au{FEQE~Pb3wAh(H9BXRzOi4rd2N!6WbX%h1B2-RgAc#|ubXWZBR7 zntA=*)4iv^T+rngYf*!z_s>%?uQ1>C^EP_TZDq&Ewad%TA)b{UJ^RJ7AZ~zwI^LEH z)ri1>LN_h`I@`u{_Q=R{rYqp#H_x#pZtHiI^N9R7{`D`;c5$A^-yK;ucy`ee6qQpG zyacZcRZ8h1WQuIdqg}JLq)C65uo2j7R1IP20>(Yz-E=jx=jQ3Y31?%*6q0jjkaB;ZW^v_>Z>ZV>T#e~{`9Y*;p3QsL z=>?IQ?Y!^jgD`GyV9&#Ls2q{y5Q*hLSefru_@_ud7f|*GZYIdy?R1BnVw3zXIIr((9{dLM@hMRy7F(<{Cq`t3DfJ-pC0Bz z1(4U{0u$V6V`jKJ);n)T1%UF8jVmql0Qc<5#4+Z^I*~sfR(YuB53{v12B-^3wfKCF ze48TK8OjbQ0L*T*WSTNC7}DV^WjXN^Kcl7HKv8l0Mz!bgXy#)ip?vsV6Kv%f?irKr zCiruvK>Y8-u~0VC-)}C;;040p29a)#J*Dcn1ks=W#a8kn^TwoI@NsuGu-ebr-yf-n z6tDVFr}xH)Ub^o5=gx1rYj7%-W|3tk?B4XW`d)u74kKVlTg-=}YTE}5p3U&PqzrgB zo~~29kDh_YT8ZH9nLyIUIUqL_*G@r6gWk`}f+}V)O_#s4U;3JV>EaajUuE^e9z1UN zU%lj44zlQPlu_m!UD&t}sa;0D<1QO>^T#Fe{AXaw{UV)=X)9F3i1d0Frl*V;No~dO z%X3}3I5JX?B?{wt^imdg*jBilDO-Y0F;D^2z?C?PllEV?_n2E=EO_G6Myd$u5DFI; zKnEX;^7{SucyTnhcuIxt5-|X&@r1mg?i~Y2f`*AgUNG8(%ryZgoK;8$YuyFfi(^K+ z?`&Qd?WU&cZy*lbh1STWLBwyJfyxx2+dmWRR0TMWkaoh8bu|W>Gqs|itZ;;OZ%-Rc z(AK~(jZ7}+>i3X0laKxTp{03xn~zuuhK-uLWR5ImQodgVb91*rFE(Wq>|j}DAqKr4z!pz~rg9QOM|7P};!U&z_Qr)_F%?@3!C zspdALQt>VWyo6;XobSXxAOEj5tm@ZzR3l>NaniJixveq~t-sRKZ%{$ue&Etx=;!u%7j8~fJ?`xTo*DfSn1N{KeNH&E*43p zUaRn|zMnHS_c;P_EFjYM`v1m1*YDU{_UL`HF5tiBr)w*|%|VR1EUx*(Too42*+b`=KF8YZXb6h|lg7qD%v-_F^%-BYKx`^L69Iglen zAq>3T8CAGXnPwYg5q1%T#$_r;iq zr)#eci2DYk``{h#ay-poJ`4wbw@+NooKg?Dc>+QPYQ&A>*aY#Y2edT~tFBbhU0~ae z{d^Lx_C#wmLu==&s#h%mo=4%O4D4L!qdDYn&kKQ@G>%P?@WX*NggSJd%GZShQ?Ll3w}=A#zIKswUb>yhy*vYF!d#x|FkpARt9s7A@Kj^{ zTLk8WXVoHXcOWAKPWcL=Ha`z zn9ik3{a$640rI-fnlEx-!`7`i@gXQ^SD_}S;)3n4B9_nHGGk8QMO6kJ~}QZ z^t!d~8~6lOi`p3ikWs^-mjfbeJ_{_$Oz=x_7Fx6dXaaoFH18$-6meZ^hc#TA_q_(P z;Fvv;-W2)0<60YX%@3T<%1*d(b*NRi*_m1$u;ZvG8fN7~EqW4l8=HbcVgEL?p7@(o z*y70U;)QZ85?AKFwlN(r>(UIsVp(SuP-xv!t&vF z&qW4AAI&~o`>55E=e|g|cK~`b;gn&6g~GJtaDP2Q@xSI2#|I2gy-iEXpvlP`8X&vV zi~rYl_YYfHSyNCH?}L9D^IJHpdh#hzw}#xl^|`G~B2|o9`C_^h_hjWmaa1pON=$ zWZs2Pm3RE>zW7_5Sb^Y|+MJm8Td@6fSYGy`AzmZt(<%%o9aWj+in>NFW&TM4DB;fQ zzU8mg+A*6zh*EJmdAw+VfG?xn=FR~hV&O^Cm*-+HhVQ_wZqYDw`xIVy`}dc}Ce?3t z=d;k7`aGfH_^DRweEmcSv$1Gze(Gjng;?-bF;JD0W_Pp!)U)IQUj<2X*ZzE?L4ne>>5LeA8k|*XkYA)$?a^0yQPh|ei33r{?+ut z?4CqS+)#tBUBLdoa0y;II5eV&M%hr~;WRa{$z!%YDW4OBcW*f0K@>PqVK39#tGf+x zaT{NpBzyj=h3!X4xcDU|_^2k8uGTE~YzsCD0HDQAS?^oc0bCRmd8OvEn8mlQTUt&6 zn-}&^-Tj7K8?oZ}{0}2`_I31>vKY*<3*heeFk8@Yq6l(_4?;oaqBYI&04)kemFw+e z6pc!Lo=16+D+YqM?(m5RqSO;%a{Z?kjCO|9c95P~+9lAou>#0I11so)hFtC>9oNC) z06rsp4}6KAbD>XfhuM}T)YSXTdG9hDYSYLbWY~Hwixi-nDGbAyO>Qp@SmUeF)hFfB2Y5U6$hGp!78J0jM0P z6cdQ5!EQo2?V3fs?{+;qqtwmyUmc^)ZCHa*4_FoaI=^1Xi+Crrt_8ujo2(Ue)}Guo z)gtj)zhR6s5X-!C`C!volv3md_6NTG>eajU1MZn~yWt)3s|O!KN1V^0Ls3v{HPgIZ zC2^%pho$1~p8lv0CNS_yC+-z?;N$abCHK(TyF9ZUlNSTHKonl#3CI32OS=S*-ECwc zRM9rG%=ydn2_h)fnWw9mLArZm6#gL*L#7#e;=MNqV?LC9)N71Yi^8VwWx@qG*XBLR zwES)uKe?qW3%3pvG;(x!@}DZ|z3T-N!e)b4JhL%sXLY}f{1bYYTsJ4UI&%>72HtV8 zae(T8$7#>NVO7cAHJLMjlwqV~SRq}wT-mm37ECHb`vD+@(CV%Bz&8Zr;qQR1K_oRM zRi@L3MdFU%IIExTWte%L7wOp@d@fjBjJW8_gJ%cX%*3uAb!3Sw`0ivsOVKZmhP%m$ zi=EY8#<_%LEV&;(yC5tb8iSjX6;u~g8obFhrDwtY+RCa=XIZR@+^ETPZ_e`=HWhL? z>we#b;=zo}5TZ|(J!*E3nvyUB5Gbr5pwqsRp4o55aJ4I*VU14sH7vSs!xTYrseIZq z#A|@3fDhr96}G6OI#nr?ix@Ko`0@Xo&XtC@L2sEMDt8wH8GSjVHPU^M>5GMLF)`-d zA8sZMP$Gak%i*{oXC2D+$254vko1hszg!;GZ4AncoCWJ$H_J@Q2;(JW)c7^c6#EOU zPr!;+ZVg|_K}_r7XC|f-t>LI=Unrp1CvZ+xLpix!jA{6ln|W)T_mRaSE}KJhKT-(M)erW9CW9nDKB-Bul^M9_ zA;Kf0WED663qs=~B*b>%6+;46KXvMuq`+OziIZ;hho~k09L7zy)Q0<`HIT;0Z0cC@ zk`K{^+1=odD>^w_-5QYV=O@ey+>Xhx4R$+a>nFw}%AE!tVFCpvyhI#t)?K`>DCWc! zTKUu8(!xu=COrnMA`&nM@$coUtOYPEjMaJ(yON;|Bh$ith~;Dr#W({$Lgj`;o;y}_ zO<%KZyXjU2*!lbvZfJ-oi96!b$qfEk8AIGdA=0qZD>(n_W1q{a! zLZP~1#rt-M>&K5!pYfoo%tDqwx3C<(;JRYBeo+864b5mQqEx2tNB-eX_{RT!ya61w zG9>;SbE9lc}p5(jmWdPRdxP29m8V-gsYQQ!Ss(#S~jlJG4S+Hr|>J# z$5rekejB$1)CFtFjns%sxA!)ge>Yo^Lw+a3%i%wFW2>O@){K30Uitd#5f(miX7FADITBfM6O#j_hE}p<8+<3 z?LS@njcLJm<@zh`0^`4%|Klk8AN!n&RNA+1fbJBt2(p*GZxN>WSPN$axpMGTaP`Oi zzJqU+X;6|s3XMD&@9-fu2llA(70;WsLni1sU01+z5D#e59F8A;FA^;nRN+t)-~ajP zyBFJTOeiVb-rO=QBk76pZ5;IedgXLj-xl)sw=$jCH6pv$Jo7@M0J|p(?&oexa7Zol z{yBv0_)gf~iQ4O<&SE}l1L$h%qWQ)E)?Ua?9L9lEeZ`=QF{%Bi`U!O2SeF_Va3^f_ z_5+@|;=nQ}F5C z^$EPoYw9K2#<~8Aw{qU2kG#_3Z1bCJy`(s@++}2`0I1lyL!8i$ujl>!l$xwPB1SFi zhxilVo(stJY}wam{)73$SgqaKX-od4bssAJz*WFuyCWww*|mv}4KEWuDrcGP$~Rig zIeIa@$_UjfWR<1E#CpSPQ5nBGcLRZg#uqu3JKdOja3UWJ9tXO3)bIMB3T2FzzU(PW zBajj(s)`JVxU+o7aPG3gw7l|_t0$eDtNxRJwy6qu+D%$q()E^zNT#1z$}<2RYe z_XHa21QN~FdS=OB_fC5})~dDFJ=${MCGCT$P}~umXF86SRho3uSr{i~!+6@)`#Vob z6Tf|I>0tzTbe-)^5z=&%k%6|Z5$Rcya#T$>=+;V^#$C2^~}x zY!r-%&Z+tG%;CJOpQ%Qz{%KpkEYOYD(QaRCmRo^=ael@qJo=*c&|*K}(YnaEfJKB` z08iApH*CYe=mkFHHNV-%bf3Nb%zlk)DD8s;O2F2Ip^TEYfX<)@glykdiRqFEae0gc z#9nORqGeuX&tMfj$f%c6HnK~uhG2Buxu`43F=BsV%Z%H#2KD^M-^$ZRFzGRq78s;= z0rinc*Jn|K7vc0l4pb1C4`i}S0H!bSEAEgh+@UCE*Rz5iA(wn8*eDmBdU@mjxtGx3 zNAo}Pz+&++#e*8RKYz2}4H|AAyiMi)@pOB(InrNJf?G}Sj*}_Dck9kjvT8R<%Il`# zi<=eV-{#4Wy~Uj2IK{af2)m-=jLW;9!=)3TabMf|BQGH7)jn28LJspmq5S9Lk_11Yvv~e}~^HI6ybEHR! z5cvc+dP0bxvg+s0kZ|O9_e}SP`9f3XBq=!Ww{Lq4(4DcK3(o-tEJs*uM)n zkbOfk92EHgqtEg`uR_!etYP|&;PX0Q#j8aK@JR3IZ};6PtV;-d-VA6v9I52bmfrYy zk3@A5^l_{*HnpaA`vrb+rCNFIg7SYnqrZ!3q#L*}OLtd|tg zV5zz6`b@@+XeOm2C#ih!E#eUy?mpWNQrTFrLI^h)Q@0E&j+=&Q3d;9mEyT!_SZ}sL zOa!^on}@mT#>Oa@J@Rc)KD2tpKHJGVTGO>(PY@NIf2QXs1K2uY1>yd|OFEeCZ%Mw4 z@Ap{(s5Y=~G{g3upmvMrnS3EV2YK(vOYPyfUf!ExPxuS$gt^$_-=G8PL$M@Mn;I{X zjnuQ{o_M&VvbH-`x8JKBxra@B=;U}dBh9;E(y#W1wc{y`%H@a`I2mZsVFz9;7c%o4jZBwE7; zBlWb~OHtCm5}`G5Pe*ipImr>SSd4p06a5jJPS4oC1$l3{-mKZVH4MVY@4br}6v(Z| zC+YI6l&8{1>$%=;)!Ucg4>9|aKogJPK9&B57D167Fz78KvA)b!-z?#{;{aPLW*gfH z7%`uK#d(?3c3oRxi*-Wn4;25|p_jH_On}g4A{3%SzDuf20Zv>t)8yp(&sM}0WVdrel?w-1kMYdl|GUf*Q8`g2n||-usNZu zwxCh602S_XE&T>H*#m5fA#gJaVKwBBxQnP_VT%Nx$Pq)wCd8kE_j;4GjS(&IN{876 zI~bt97S420w&{yz0wg$o_L>;_aZ>0hc(ggd%!#%K)?@c~qwb=91SRL1b0-rM2!6_j zpCiRGpnq^CS{(LgF67@L$aj?-ldH#ZB#EEVa}5=eN37r&|1eXW1U=YQD^0#~Yc#DV z+wkL-Z<^{!Zuxh~o=&Hr?fdLl=S>bc*^>$+gC%2J>`#cm8OdIr1KhsWJU9n;`OVA0 zE@xVKa-(WimfUd|yZL{-$q~pt<6${^yizj&UVknfvgbUpBV}Ld)rsDxw|&pS>jJrG zA(#I}yyMicQd3(X0H(-{^sZH^{<-Y_AG7jmD7s#czK;5gwSCI>^WAKFSD35oEyaT-WQKWS1PV1Y3A%QKo<9XjXrcJXww5nYRh(n$eRBxl-@W zRgtAG_MMRPhKAh0j;gS+YDet&JqS78fx5E0rww3;kw>zhrV~6s?Zn6JfK~AfJmQL2 z`g0H1`msjFK#*3|8?l2g)QbJy*7FvgbAI!dQBl86J1<7DD+gmck5?*B-pB1&ff@>V zN@6>kbj&2;1QA|brbN@IPk8?|eS0becWh=KVxrBv!^+?NKJfgJi1feL@5m zA;_2#6VCdf2`g}%G)=Ia0{p=I@?o>+46HZ9k$!>Pq6T!n9UiQ##?o$DldL^X%sYV@ z*c~I8AI+=#-@`n$jngklt*VXB%GtI7%W#3$-tk9|Na^QLB64SQlwyQ;d3FsgE6{$D z;3pYcajNCnRn6?Y5@OyN_SXbQek$IOnf%YSCVLfblF5OJd^^uMQ{;Ezjr$bHt5&iwgeL1arkBO{&t0pq zs>>^jI{Th!rr#DKmev|xuWPT%9k#)$PeZu@rM!3MUbROopL$^};H@|;_q^68 zz-a5D1n8du`qX(_@!Gxdy)NT;umXB({jOm`(c@xFx^jNI8=EM{+S*2iyZ>&p1OWMB zd-fG4moe#>EaS}ya&7;cm&Tl~KveY=c8Lu+=m%65nux^9e7CIIj;#>o;SOBJr`jBc zKvCzj88al@Hf_d!95oLnMfcmz1sRUU?UQIoa`{I zRuj}uuF}S=sLo>a+vi0-eIlm?N&uWGQ%?{4`_-)v;f$%6;Vkff7>q5bScYu%rs~Na z{CTd}HT+yE!djEZBv6v@T}#RdF8Ew3Cj~_hDZn8SXhkfy)uc<1J; zms^y1KOA^h(yDGp+DY5r(p$BuBv>Y7CtnNflWTW-Dg5PjD zD>1zed7H&;+%@z+n$E(l$^LEQASfcGwA7>}1f_dYqDV5`BM64D|l zB}`g!bW5ky#)u7WcK`Og$MOCR_jY}+IM2^n5)z#VmesDbF(}4=aCF%wLn%yGz;2t? zts*t{u`Hw^4;H%9uUb?URend{F;Rs&h?W&Fgi4~q>wZ}085 ztyUkCKcq90xz19YjRKoiG)WPI?-+Q{+!eq-E_G6j@tXH)0N_8Qm;;N3;94C>xSGSY z681$P3bEvpGN0MQbcUBODKcgu#q{EIL_rwx#@=krh^1YDGzIh z-qfZg5sC(To0e=DSzB^;^i=tYKFNG5KuZ zaKqYbW&cX;oQXOIpvAFQx6xB{B|iJ;<{k-8^5#SmDj!L68y4^P^S3_N{F|nATwaei zmHB$yX(74m01waXG_s3wan#3uX|Mk8(1taLuOtNoQJTt7-x*LK6vhZAw!T7L^i_=+ zs0NmN3Kp6oytw)-@6tu3cWA21FKT&)G80S`@Jgb9N_815q2j?0g)LH(06Ng$aa*L< z$e_+gavg#?PW(|%pv0*zy(CB|!$P*Fwn=GGk6W?vs2->)LWU2q_~x{l#2+L@RVrt2 zhj!ioafx69-u=$?Fg#yH^VxMuh%%d`UEIIjJ~!mrn@fCJA9AkhDHQay*;kF<&Z5oA zg02XYzld=(V(kCi1!was&mNjKcFClNLW}$*c5x<$w(jP~T9-fUGYdG)ESrRR(d#nH z<)m%$MtGH~wygSaJtdT`eHQR!iTjnG6_cn?owpv$hseBp55M(URITa#VY>O#?W8b< zSIVf+#N}U4Ua}-gvV?SSE?`CyL(^DfxBs#VP0l}TWB4&D2@;T~^Nf0cq{m+6e4}3z z(2yp}#Kh_GjohVVzQfaG6Ul>2STcc$i9eN$fBNbpHco|pIiR3hlF&iO)Y&kU>(w|w*HrXl|4tIY0f`JA*hM^k}*u&J=D$x-v=ed|PRJNulY zsGd@g;Us&EHVz*xOk__!+`GlD)F&H|n(YT($K>3<9rH4NmJo&7c2ay_wMg)MQq1q8Xt0 zh11j5{W&0!b1|C9D0%lz&ibtd(o?kLM&F;|<%vU(r0EIgc_~+mh*29UI4G1{0b9t# zSvt~bZtdNbsKZDVvu$?!X7aX^Id}@JhKtzoSXb;}0A75ivpbx1|BW>vN^&QY(t4;k+0&-x<^P&%MGxD_JzY<05lK z>G@$^^FbrndTws>ZSZG+-%Xw9^{kncO1^jHrpHxEE6AuKch?(W!F$_$f6F|0(bh}r! zoKz6WA;(6|$0Q=8g=!|FkH~mn!`(WC>i;fTsQuaUSFIf+^=$oeir}jNt|*;U6+~L2 z_nz$3nyz{xbRJv?+L5eo_}BLzLox2J3ggQs3pR7Bi)a0FBokVdENj)!dddP+VKc+p zyK!eoS#f6d@Oc7a#en3R5Y+=toK!eXjg365q>B@}SenA9`td1uow)SO*uN5g-n^_?sY)j?z$G zVp&wOpNNQ&Jgul{UK^R!$_uV1wV|_yZ)V5LKce6Sr0xmKIpVJlN&AcY8~pdt?8_MS zxIr)v>W6B;M!r61Np$}qCSh5F85?WW^=I0rlgHGYWV_#~SIK-sDFGZl!wTQgy)(!19u0FMzw{CixIwR*biPu!qG99;>! ztB6eq=4t?uyDNn18%6hb?gxnYEf3M9N4FUQqL10J`GX7%Q$%YAXd?K_qy3 zwXQoG`;PEc=d-oK*m}V5%%+`cR9?uHLty>m_oOVf(1X^^JlOnET)Kr-%oc@)#tE+D zKc(cL+#;>azeVCnJ=jY;XCSO1EPFk=@s)sjgQe;|HqoRax2yb>0hF8Lb?eEGpTR-N zRklPO*oU8pSp6Ew1fPsb+?A~D>|NULTtnbS-sFc<$z{pVf90~`oT!f`dE&vu&0Rzs zR6N((=&$}wCl8PJLGxb%D_p;{Y#C#>v~|9a0^l5ToaXB6CJlM~{yfv+3Z2$ms+H&B zfi9A8$Uf{VndUXQ6c-w7p#Pi7Un$ssEPQdwvXZP!M;-}svpyX=-VHUFM!9Aqt1$!&iKEYkFFja#w-+A&AR5T5~26yEfdCfNa~eTe?nVz6VbvovgY$ zAW(j3X+%KkzWb{Ctt3ksrp5HWX zDk{_wMgmXh-)C*IWRkpwF(s4L zKVIsWr+r(yN_I^l>@;exrXvmpC;ba6>!ij``5y9+usaBR6yi_OuZTESoc2>f2Ad0; z+7Pu`&cBCPWY+DqA~HitCfa`=5~WwJ!TS0k#mO@4BO^OT zC%2x=hJ+guezPXv1uTmcat(A%7AJD!3?%2a^}Xj$v4$^mUy5+iA8N07+PLO?w5;=a z@-JIaHT2^aWE;(nQVGvxyO$SwcC+WrQbT!F!(ra*SMG0D$)lQM=sh5sZAYwfhWXYUSH>%)}&#zhGm3H#(Hfn;(GFY*a9t7)KjK zQ2z>D>S*93N<6&RbIa*S&JtI0y&$(ZLm+Hv1;4eUb`IqKjHbUh*jEV^U#OaeeUF|Q ze_?8C)kh(X>#@7M2i%u`6b55%JnPL;frTF~eC(Gv?}1!w^|+~L3VHp^YOFhGJ@-qo zx9fY(QzA?HDt(e~GW%RdIO3KBLQ5Q-iG??kvG~7^WBL+}_>?iU!X5<#Pj(9U6Yula z3T&-0q8tzt<*?%u3UJ)psej`gIM@Ii!wXVs>05N)#8&6|UUpk2A^AT=8NL?R^Uz7`DiN(y)YCX97!1U$&O4bv(4V3sy z=A-GD;4T(cjFoP=TliIV3%=&ZBC2Gx;MeDC7+m~_F3tCRf#t+55~FtpRQvtvuOFw+ z=EVcf5#5lVfG=2Xl{|cz#i9$=`Nq?|;+?aT*$e*3wj`Dp=2qVr7q2O_a&ZJoMwE4- zwY`0}^XN@SH`~r3;gNfKJU4f^SjBDXe4f{wFOI%@c{kwc0R(gWj00{o=hyE)-P9Jz zDfQscg!V$ZlS&h=HeD$pZ{;x3Zq^vlHX{8O`lkXH-cyak_he=*E!{sMrS)~w^;Elf zzSKVEs(N`<0&$8W<&u)f(q0eJX5*lLQ=6D*8biNzbF4#Rg0Z$*$NzF0oDSUOIVwkADd2c_|HIU!s+P%YM^4?@x~dB$t+-~OaMffI(RXHOw^(cKevD!)WQ*ci zzrWUjJeO>U(T2Y%Q-UfJjcon(^MW36cfUwQCj518 zB&YZcramJ12J)urNA9_VL} zZK5p7Abx;^<+Gx_pNspuwP>Mel2Y~t8?*V!S;9|OD38rE!g{ZvRSg;Z_c^}I_cw&o z;v@R9p^(4RIUs!L;fuC$?BgyD%U>MzR%DOifpTSp9nvyZ5=9o$zM6N}AIL`D;S`FF z>!>QYH-}#j_BXT%R>jdbeU$zP8M-)Z#NIj+Cdp`YG%wdmwEl_8-qw$c!&7BtKy~LF zp-dgCOs7g-CSrDH*evX0ea50P_tHf!gJ|J@u`AClr{}OmZ}+-j{7HuH91|i3-$d7g z(h_ z7CyRjGhBWV&aY{<)H@=Sqvk-Og*v7!lKLUL?ceY2j;Q8R*d1pXA_4?R*RY&Y-o*$QgHB2LalcuYc5BHwcoHA;i0s_btqpO{4=^fTttMS36G_ zd2!!o%0$5({I8S_60D&hTw;q`?Gb$kU(V-n*CbK!Nn1wt)9J5M%}=K3DrXXq>N$xr zxU_MTcK-V{g`%=|_a3y5l$99$78bL$xVW94YqsM4dilpq{D`2P@k6=-7I4d9j4{k| zg)wQQU1395dfMD$tl#R>-_&oyV>A*ist3_^S?It))g+_64+FHDkfNJdDS(qR7frGMBsm#vU0Weesv9qqTqVHNeDEJt~v_G{TTsYuPoW`r)K?&&6;WUntfvP~~m* z_2L}Gs~(#ON7|-1{EVt9b0pzU3TMQxAGFw3Gp324dFzTy=6Qwq`VWNq0 zN{!k0XN%|Qy#jOT{i#r19bk+UFVJXf7wJtXrW?oK+ki}SqjX%hoy^C57e3%wF?*$QC9v&Ktm3@$IPY-0q~hBzly!~w19GuDLU%6$<6#QyL!<{>WAOEkUy^Uq#qF(!G6ymNwnfx-8@Q!Z~Vl(Pstu=V2R=6$=Z|FkBkt z;C2&^0+8n60f4I9D&djg>sgamVxP-K+tk)wZfNl9pSg&S_6|fJXeKV+ zl5QT=$4I4`DiI1(NkgKTG`>F}sF|p{8Y~35X5VQo`+h`}=ldj9+5<`V?&75(q7dVRfM!@nEIyXNRIpsf79R1_)N%~fdh^`k@U2ZQH~gF1P2RqiI!ubI_$ zT~n)YUkdv58k&aLaz1^(-(`y&x4Fw=`u^qI; ztv2Wjy_>kHn_d1P@QEobk_90O?&Dnspd~Ac%u|NTiGCsj(UJoz4mnJjPgj&-$ijMq zI7!CEJm`sH;)s%Lt2C~5We%H|Z-QgWjGJ6XR{$kzXj7sIR*+^%6A!skGk+0{&00pf z5uj|LI|lyUQ;7(G-)oG5F%$E9R%JgR7n=A2*stj+A5z@$a6LSpZX``O{X@-5ssP2r z0*J}DzlqpmUqd6}vFF=p_$*nx5V8wrBscqzM#L804^o~#>+#Ml+>tsZ8`C#ze}bvv zf5*b~MeX}Po9^PBM=boqmPTOt9-;@Y5d*Q~K@_N*DS3i!=P<Y3S*u}L^qU75j8gE}q+Mq{69A3yHIHtL>w(_t9ZvqT_xkhQV3VZA8e-It% zZNd@YyoR?|O&g8g^XG!nBp0Oeb4Hts*qbZgt3O*s1egjdLXWWizhIy5H?B}AUQegf zR#9!A8j6f#yr=eL+xs~u3))Vo`v54aJxi}iY_ZnNt~F<+`EV7-LRQ|gbk z$+O2M5$`<@sdJSht|Qw^21j_m(uW}kD4C2mqFUb0)VpQHQuifaiZsqRxL8~;*tEZI zupPgZgR#ihz_!YEi*xf^d@su9y~LMb+gZ}E{6_H(w6v(*iJ1vyJiD&kSc$Z9x=5%ojOWet6Y>-d8Y$-G$lfm&*x&+os}eLF5P!+ zz#^j^5;IX|&-YA0F=tO3mONS16U_vD95nr3yhqAB;hpg;huGQm>NKNtM2)t18u>lR z8>K+LpEEpaHSR#}kpR)N?nYys5?uf=uJ$xpu(<7O%51sA}8U6P!&QKRlKY;MJbCkPNSOb%R`NKa*2Je!n< z`W-&<iaPIUH-vX|*Iuzooim^e8vR`%C_VCMn$`wOrQz^c=C{YQ zD?nTDQe)*hoRz}u>ZnpW!o!eG!*3stj)1Xif)B}eKC=+l5iHx5vHa{Yhls?*Dsrti zRq((!E5YP&GHUJ<+Nup~eJfady>*GS7l!M^&E+qp0oxZ4j%x-TQ}>n^&r2|9l?z(^M|)zY(+*n@{3!uOy!XOCanZB>>CcgF zL#lY}Zwm8sd@NLl`1SoZaz$(~ocjM(csdIVQr-TjzS0|5iAx z+J5nL{NOwL;p^Y>BZWw`cooyNvYzvv$z%YCR|=Kj%%W4WuF+(`=NsNOSWP=LbRW%R zEg7j8D`{HNCUX0(J?TfomZSOUC4qk|7IU$YpDie9!&>(@5#?RQRq?PM@j+8%D5q#J zOG0rF(Nb^SNO$+Njp0oaa-%Wc^+8@TQ~kv?$)Bay#6BVkmw$`d)zAR#&MUjML%u+_ z2X6&yQKf-iE>3=P*kTM`aUjMaQeR0M4b&XQR^9%vap>&jmwCKu2-nyti28 zZ!*d&JF5g_shEnrX3l%m3ZYtjowFNL-7gxC$bKHxhe3DnK!YDVjtP{X;~RO9`0ktI zLk=j%8pM0KTkG^$0`lTcosQN?MbYi;9qJckUw+-Mt$gcSv|uLNJyz8E^qQ5}4d0g9 zj=%=}HbdAmnfs=)%*>p5&l+sqcFNwDUe~>(%hcHCd&6i6i!^qFEb0_<(3gDp8Rifz znQ)u=bp6>AwP?BO%a)HJ-cGt3%LoRQi>B@Z39SG2jpgWmRLDlEHa61{Kb5rfyS&Gu z83_DRd`+8W$0@f%INcma7vCgOA>T6Dv}tI4%()UbO$ravzH~{j#XjBY z{O3ef=@%Dy>+Ll4g)`AfPkvEHdw1>Tx~kIgN|D*jxurO`s-JZ#kdK_xyHZX=%3ms~Q9+#PjaNAT`>McVD5Er&7N7=EEp;`>- z7EBpUO#Fl`ze5yb+t&YjD63*PK%&m=GnT#KD<|$W} zT=Ann!lCWM+$%Jr0!;6IXshT!X9cX=St^qGn9XG0@u@DtC7y*RpS?w?Ty}bbSg}?jUePUzd6jW! z{zs9#r3*Gpk2GDvhO8rbP&kqg!>Ue<$R_;nsVn4Ranxm-iCf(Tc*C}Xl^naio{7-1 z82W(X)m@+m|Ke)sc)n`}kfyh0d4eCqTiA%TMklV$Ew3UNBD(QgMS8suv5h6kUUlL_ z5#lxh-O1p_YC_V;ovLC;ZPo*0wa<>)d3Wh>3gs&L$K${O9=(o~M>#?Bb|W($OE#|Q zK%4d-p^n#Lt0RB=mVy1yCu*M%%b-vKERPCy12LMKSC=Yj>j%S^$1hU8zb-JD`UPU= zt2mRp{6H{2;MN+G;7u`8Yaz*2JmXMLl1!{~9kr|G+KcU=oxkQWtCePN?cZImY;eMwm?Wwk-d^}H6j!67HLymjA-O#GWCo$QL-T&M zdlPm^A#@$~VqmjX{z0km1yFIN#=`cM8ISLuKJNOz#D&ven-o2W>u}E7S?H~j3G&(; zyt)16e)Ap?n}{sPi;0oUJ-(b5z#3mRdO5iYr_AFN*xSYCq>=Y>-@hJCs#eUYH4m3R z8|;7UNPwLkX7)iz5P?!_;(msk+fByC1ZZL_iKj1(RvAn9@_(+0|90*^x_JBSi)<`- zoC~t<;mz1$Aq+pXBr_yT%9X#horC`@JL0G8n!i-5VHzIni}7XRpai2`}DSZhIe8z{l_sidmw7qgZP_x#8baju|LPDs=_j!OI-spCv~ywppYc404?i-g(WC4SE5m zczp}F!ev-tH|&$&09GfNWf4Lr3r|!0=e%4^t=K4Yi^Tvnyn|n*s4axw{R>^n@8*-M zx+`jyjy`7zF&Q9AMB+6wDFMdzmuy zuGq1Z-N`VBz4)WtGVDZK59D9p&RY%6g_=Pqab<_X><-}8?dv)9S`!Y0!q#OMG!uMQ zSMH*TK*wSNRHt3Oy>^!S+5O0}l=X2x#GGWb;|$g2&vJqy4EjNgYpy`mP?86$8>QV1 zs0t|~zSfQr$j$FZ@*wn56h|Iu&+~>JMKhQtLXQv=7D71vZSWJ8Qz<{~vW1dQQN?#; z!T$AyH?u8xxwdBYoF*hh#L%-_1`E_6G zx=8#{yYzZk?S$dt`R$c>sJp6~TqFu#4M4I0$rWIMK{ibf3hJS6$R|3xt^c$ zEH;Qxy1d*9@Yuo}+qVx=X!1>CqKNervoTKzxDoq@(L4Bn+HRcS>G46*}CA0uUuE}Pv>;w2n zZ*?bMWcUs!`%BA)^dA*3U3&k;PyxM%TuJ0X^x|HIEy^w^4qTu_Dz6a4% z7W$a`JEb_zgx71UY?)aCLSZ_=2a5k(P0IMv(BU~sWAR0x=coFMpt&3x6J4opZ4_g4 zxY|Y8v+0sd)=7Zzdt2nJWrOUoYC)&LHksf_n60gG8+}VaM>R*S;7s-RrIYtJ&6kIt z>Cb_zja9EB5x1`El;g9=YMpFZQfjw-;abk3YMCl8Z6AA0d(2;(z*$i2x{FDLx~r9k zQ(4<*tm60;Xo2dYIu|$#^iCdo3*nBp+HyG4%frj@C7*)+ah|>z>*n;_!fU?*SogG_ z(nvv@PdmD-RZfE&`%YGo0DIv9d{Y_PxUXY)sEs_*WV0Pbs-=FZ&V9Ym0z?WLgii3Z zdU7NW^bAV7BF=;iQN+ldwtP-W;6EHg^UIOv9uKviJLAV<(J0Im61C2XoW2I^*qt_^ z;TyGInI>?si#!6PcY8Bfw5ss4v|JE==?Ev>LhFL>Ur516LSd&n5cPIZF22NtQFlT$#0ml66hTXQz!l15?&mX z#gz3*sl#i%^{}&fuS|ooZ*;i({a20~FZ*6;J<;p(elX0B4`2{ft7W4TIngx@(KdW9 zA@yM|$u6iS>{kG#oFLtLQNwoZ-N0X66wI+uwxUO#ReC-3E&@^bppTr;Dn@A2p}*-3 z4z+X6JnL?4R?&d?F{2x)`dB0ljM{U|-#Ogr12XP651#&F49C{o!RGKA_8Tw!R=gN5 z+)_}U%6mCL_2u;+(`8i04Z~iqc01hi{Ha;W;z(zH*^_TcQ1SkjAb0~%V;F+&tZ!>C z;m?<&L2}4N;e|ADzp{)xB7vvKZb(fd{uKY)oEUYLX2m*hL{$@T1H?c`FW!$bQUc;1 zFKL4CB8~z46W^f?s0^%iYbE79>Xj9%WKKFEZdez@D&9j6ht?xr4ZnSdfn_ltks_K< z?S}s_)QC9;VyriE7gwZNyX!0Zb>=h<^x%~5dpDVhra!r2NGnR1!OpP$TOL^5hw640 ze<4z;f-GgC0`&6nn)vScqSOXBi|-0A)Qg{(1$BFQlbV8Cc5IVANSENF5?T-&E07Xu ziX{gZAR~_HpHxkS^x+S1R3dz~^$Lg=WUIdV^@2v^BO(7JpzbcM-_p)f|M1%IJHb4zD+RnX z)8q>9nv{T0s0<$AeYP&7J;6B~k_L7E8dPmeavjlYvI6EKLWmP~MC1m96nMQrT@^%f zl0OvzkAQnEn~>-K4p!q3^Z5b9XTv8%a%hDza_7-f$*Q$Gg>LbtAr`X2KrqBHP$~UR z+leGl%VHogh%?}c)zMQtx-$y2A!el`))C9AU=~)pYXCp=55f&~fTuHsz`G^AnP5#G z+Xvvn!CvcJ71bf%PkIy@-3UO%)PTk}7bN`qyl)S-U|}_Q+Ba$=wG>q>GzJc8UKGGcf&%plD5G_be}nC{YKzu3?*y>{HkmqaUd){WZi zKI${*rlf4BTR_GLh4{O?(9rQjlBLOcP?qhWx!QvIUY5mE?pOHb2v}VRyz#c?ZkL5; zl^V%SQAN7sgf8#ah5*m=zRvCB`VZwRe|mYl(?I!rHIr3Kv}hDS)gJ`@+}>|KAdoaJ zJAiZ#p*lFtQb=wwcXJ&2SgsfHHiSscw`DSP5E_Al9ozfin`aN9zkDB{7q6?uuDWcs z$??dARG?MBf*%;MR*&&xD$V2t%&V={&0Np2F8-O@7*myyxR<_0iIw;{Q@7B(EJRuQANj5^m+4YBS`>HYgy-(FyIHNZh6^+s{unEN>+B{^ zR>)PZF5k@f`q-pFC^}I9sWn>j;`=!})T&6ivnQ4;4lTW~d)+;sGFwj>zPQHSY~r~;Qv{4SrfG;wP7B{p9EPS2^y?_>nLy; zFQO%m6!$4^A`dgWx-xCv8xbJHL0+prWkrXu&$kP^ zC(?Kz;76lm@*f|M3Es!FMDD?tlJ{9|hxb#G`o}IP-FFE%@TDV;_lb#>72m)T#KA$@ zeSI_vhd{#!Ix?Yg@51<=UP?ZEYA*E3z1!H%a13{VWhnP|+>UU^(9ia3cM$ETBBSX}sK#XE%HG$4?Zu56f9xqsA?Hyr1LKa)UOpAS5 zLU|3x0k%s{1UN<*V+YqJXq2i18OhTXo(~#|YVfyJ=?jwI?8(#@h@ktexw+{O1iawA z;z|KxqmWHb3zVC*Y&JgXgr1*kh6Lf2YvzPO4%C(Uo7P7ybU%nqcn+t-se2F7{-TA3 z7Y+-8KN<}@^zXcI&QQ4_B)cDKEqb^fbj|8(jf==WT%RZd=bgA6E>*jW!mFuWaCy@q ztCv#3>3(5Kc@5(&awNVqbVw2d!micP$-Eo%)KW!k#kLt16TDLru_c6_mS?m(7Y_RW zDf?KIPJk#pHCane?Cmtf(GKFVh7r+MVuK;bWA)4nqzQc`U;6{dS2tVUxtU)}Rpj{= zq8+KQ$~KiiU|8Mpd~5%W)DSsZ9)e<#Ptcgd!2Ln zLpo`|p^|FrUrr2k3>s0b9Wrc3y8XVoN{ld!Y6SRK8$yK;fMM{}44kccCDsG`sdw># z*&(v@Ehht+DQQ@=FU*An4rr<|&FU~1MkuFcK6trz(9b&cvb zdyronvIq?69>pG4+{30U28{yV#k+J9M?#(}{()=L;fAF3K>8xi$m9+{+Z{B_r zDRkem5(PIl>)f@vwiH{JR9_*fdXFtFfk`VnfL7lz*ufxRggIGM~);n zZOa3F&Fyu9?TdeKS;@g~C7bzqN_wgYQ*+%O8@%Y_SaOvi zlntgOm3?F!RQ<)mr>29iw-)7N{%MhXD2Os(_m2ExNLvJ3_>e2f;5W~_-HFWXauwEm*g|O=)jH%wE^S&3 zbr!E0r!#kphMK~Vx3>7OdcuX;nO0q~^e0n&eTzgNvF(vR$?4YwB_^z2y(t!nJ!aF<02%mH9CGZbf+-!DDM3$Y+9u0SZ%$#}HK zo2!aDAF6|#$K;{KPts?13zo;^1y8vHZvND;JO3YjtB>^!`C5ivj7^1An6Ai)3M?kxMTFl9R75q6CZVqG4yba z`rQ3pQ5s?V_fRM7!@@4XVJIZpT)i^%kcL;RWSPJ8Tap8uE-Tjiv=7bI{DMWB!emKf zz8#t1kL>}sopjF0-H};SulI`dkX-IR|zEmK-3C$knR`u0ldL{w#OP3?0 zSR(ZBu6}Pr07~1nL&P#vgLpr%>JoP$7M~pnVLH-2fS8-k45F)ov6Wcu!8*dLXs8A8 z6u%DgAhwY0nWLseS1lKUOLSskdyG>ex^F~>`;)|WAwMH;nUGEtNwjx)6L!Ch_5ufU z1iI%fOPfCw)H`4I0ar5w_?RkSmNj{4Ckk_x!{k)TsCyZI{}4w^KYyAZi}3y8vw_0f z^qEu1a<`6pg08QL+b2B8^ep_l7x>Ql&Ua@ln9Y>~?c-9?_X3#_9&3E^dZgW){ZykQl7X*>Fh21&C%RStnOkthEl+z`x19pd%PZ@LR7_HD zjk0q5KG;OwcieBA-3LfI3=8!yZGT;$q4yXW75`b+xyD~GD3EbHll8^qM{Tt`>tbdq zKY!dNF{v=Vdx)9Zfw_P&efd`*c}yxCwkLo4t6KlP7|eg21EHNK&9sDt^L?@E-Jf$_ z;{kSWf1EGIN`5l84otV${V6u*+Preuk2)UoKCYV5%;KZYZWyNFQ{Vds)PHoM{ zsx-~t60hqVbcwy&rX2Sa7Q0BKoHHHBj7mrfcj&nP;66J&cNr68q5H#?>Q7S(R(k%C z4(p?$Kd+izeI_Ox|3sKi`@joEqyysS;TbSgv%UAeA z3pET8bY+WcB0_`Tk53DU?J;M==G4Sz4Mva%?E-9+@LMILd!MHVE(9pxEp(xqh(KBR z%^2@-{FUdTp$bmdWkF`TJegelxwL{!!Po_C{I_fx$Mf%w+J9w#KG#?~E~FY`?`M$^ zevgNxQW5wCMOHqL{T%Lil>}88A>&%`8srB%`17um29E#cRh#&B1bzUt7&nf@X=LBJ zJm3o25}dOwP(ArLA2heOk2+{QItkHP-{c}xw>(P*Ml96ylSKJxLO9}3j)&_sqK*|+ z3($Dz9T~Dqy>hLT_Y7jpFsdKCfyaRoKZJVhaqOHZVB9!m`tiMDO9k)k$;S-JM|UOs zElUS|GyGWHzwU7LZ`n2lxaZ-u$DZiKT-h)WaGoNewc$uZx*{|(_Fx$Ntq30WKzeMEv5jPz^jORB1iXjm<8A{NVB2e#&E6zAGNJdp}PXgQ2{8J(97a ztz($IkwlnlH^nM}>0AmbPcdr6Gy3Ib#|2*Fitxv*q=!`O^th42A^V~`%?$X}mu%pk9c0Bh}s<|RI zSgJp1OjmXe)3H8zAX<dQn~aPh5?rmGZB2Q&h+2b}H4fd61Qu z1XAqFtV8ymNyXe_pzQpV%Cp2Ua6SHcaG!@hj^TXqWdoq;Ui|c5q4WaicRQ_Lcsv2y zpSOz$%mg%>*vn&^@J*+v(DAt0ne(2fmrPcWcs}|G@@Nsg`F?Ppml0OhjHP)KJNva@ zqJzn47EAqJCafBf0>yB3Tj=!u1vNpH=@Zm0%HF&Lw~Eg|kc#O1XqN>jjA-u##_xoD z!TA5S#maIzO|4plo?!R7CFi>dl86C4tLr0N=jkviLfFi*q1g$lDKZe_7K2bwd-ZwK zup=xqVmw|_A;KI94an(+f&DNS`U`K=8UyxII^qDbPJ9ilM9@a{4C4jEjdb7NfFxLd zf0ph1jd?lAK&i%A$n=?b>Q357v;)hO3G6&`wj;y-$6bTeyASXAefsdnkn#mkT&u4T zbFaG?$5Xl`o~~gw{lK*kdxxs#;ma?UAyitVi{&G2AyplF`c%rHh0h%p$Q|78Ui_nS z{x4$&n%&i3j8aI`wzV7F>G#zyEqIvUwjfCmd#$)5b&%)jNOCg|sC>srP8G)+(UTvGLDL0?LW`-)f zI#EP$%Ij7AIus?vm}&y976QM^z)|`oyVv`HXKp3JE<%QcHDTm=GpgL8PK^D_h@=p*r1LFDP*Ejf1sC9%|Vf`fSomQtP&K%l+cm_sj7$E3wnZbR2 zk%?txsTfqJ@RG^7)mP?$-_yPqRrLRBh6281H>sjpj}z}xuv||acALLY*7mqKsEM>y z4!@^nGao2-8CT0H7+|mwsddt|Bx-(3tHU>}yN_rzdMxGz*i<~j<%zY@X_VIsj-*=C zi6*aHbMi?Do{@vsMsBpTk#nw4%+_2#Nej?X2Xg4nT@(B%>3+ohXtqA1 z@sZESGzfVSNArAyXZStfbIaBtJ25jj9kXrYt4{-y2l0*{p<9ENe!m1;D}&kV0cu?2 zSjt@vnc65eDCc;Fv<``fut#9CNg^}sHxzJ2BSQXxcB`gvtHEA!#KsvF&+iC5+SYf< zWqV9O254}F+dk2d&QH~>$><5bS9Z=V@Fw_CxfgkewgJ@13oY3+;46GN;*ev!>c6co z%~(OHsBkvBm%O;y(aQS$w#`lm#swOT?V3BXjb=*BMhPSyUYVpaPI#bFk&o|?(|wKA z=|Y6SwI7rJ>Z*Ss^uXvd{9?2k2Jh{P_UZeyZTE^;|JL9`hG+9t-IdUpIAy4wFG z%G~eL$#nvS#@wf2-oi-i9XfAH)1yV`o@ zSubwpS4h}4r8&LaL2hukh~kjNrRSM7V96dT1RU;?%K5Jqn(Fp`T<^>C+$aY+gX%n# zg1(FTd44}uhifo_b@(u``-XeU%!wCh_L{w%5y*ySAwJG!eV{A@^hCUI2-~G-QVcJjRm=NZ~KT55|1mSJ<)YU2O+N26ST%xI}jeDgk zI15I$=&P!&W?p!oyfH3v=rE)oT7z~Py36Pt8y5Wy(=RH|QA8Gx1=Hlnzv-V?4r2mu%r5SWtdr9;r`$SyD)l*YMy@TO;u{$@{R ziNBRiP3jbV+re7wPWT@PNEYwmYG`-4iQ)w(k3t zRJV&1Ke@n^`PFM~;D=R=oR&{lD(5ixq%!0A42|rE;*Ha7k0G>B)zEG(ZzbR;U|BEu~e z@3qAjkx~{6#45hQuf{^2H5Z!X-g0tLT;Si)AGD7bB2kqjeNwOLp5ewwY@WUMG((xV z&dxZy!vZslr7-$f&Jgfs2M$!ER&tS5>)sEQrTGmd#7K<+aE{inz|M! zVf56uPx*T5dNx@)torZod!(0W4)v_A%2>IqLj=3LI9lbEkMSqYshfH4MZd+XhUSKM z4MJ;BT1SRhi>S7+&-(k_BA9sZfY=df=X1z6aPDLgG-Q}YfENr#-Enm{#&Qh^w^~=? z@(>)^e8vIc(Fy_U)3dTa%r5`H`oGZr8szl&LE3wnSW~fWs$=d@bDA4hAImm!euVpw z9Df@RD9Ei>1rqm`f|+38+0+e+=JLrWMS9_-0PoG$Xef=7ys8PlH8R*3;f0Y{#zy1m?69$5{w(+6cm|Une^0?ICEOyxI!qi#{Qy z00k$#UgSQ)eIt$r1-%N zf0l7?nm`5eJD33`Kwt`J1t%{^D(V|(gD`)V68Iq5G6y_E`@*B}6vQW#0T{DM0TG~>@VEn>Qv#x#9v@6 zMTnRs20h2qZ`MB}nsN*sAucbv>h&P-*c`l4p^Eo60e>kYTKUFL3N-WUhHUuJjbCo4 z#kC8`4Exf{(tRlQtV$w2T!>0@6KR=?cMhIQysY}v$%z0k#`r8{UckC3_}n7ks{_!S9NP zzAI}SeI;sGk&v32|19!O$$3!KQb~9(z=1}rBB(O;#EpA_Tqau=O=k^{_8>G)Wg|6j7*v&lH2K0yXmJm#};0FZ@ZQH*WrY8 zWlV80_Tev2x?d<|kv=N&FRiYLn!0P;{OV6roa+&aIJH>&h&*5Ke>9zESd#zu$20wA zS!s^kij`$*rshtCmMb+Yx29;W+_^OoDl>EMZIW40eAy{eh>c7 zfJfZdb$>qRocB5B{VIrhx9zNtPmA{@EQwv;_~E7UneaK|tt^x)V8*ZMk0{4w`{+F- zwa$@Q<$8UV5}9+dlrK_+$)Wn-NS}I^r(RCG9~?K=pGt_VrmEtc>DCF-p=JfLZ?ivx z+LY@*_Ha=T6Ed2r(>Iaft9vOtqLJQz^U=?(x$X@rlqp-9?@sbn#mG6=9X@Lj<9c3W z8sVLtAN=E1^TmW$lGHo%*N3)UE7X{8|A{WWi(u(b|D2${an*N4d~_ejT~u7N@1P)i zf7>eD%D1N@JI&=YO)LAgJEF%wT=*R?G5a64+#eA)9-M7tDvd!8=3HGdZX$?YxfS7c z=(-s3W72nV2ZaUEwE5*Fn) zE{_LhF_Xb~ZdJ61HR7QVy!GA-gTuwUXW*GnqWI~gSJ*jJA2ODHkj{&+oYfbq|{=A0%=B;XQsOYjHX7v($XRltIR*z;A zd}P~t!id@U^@VlL!;pp8W1wzA{OI7De4U|-8ZalGdHucn&v-xQZ#t7V|GDrzPz}?$ zXzJq$5zJ)wr#7N!=ZwaNI?|ph?Z_>lx{G0a3%AQCX}{%YQ=5MuUVQuSxr&Hm>He8d zT5)Lf8TlKVEKqA9u)-nxBO!mD8^nhnb7S5hBWQ0$~?@v8ivCzPe#a>u-;{7Y91ScOXrhx=G0=b(sq(`0Oiz)H6+>_(`ir(WFL? z{j0aB?8rgTg&)EnX;(lm08WS|kENnk1bZ}f5aP2EhZCV9dr)*jT7l-b{{)yx5zMb= z+LqUbXpezh#DgMAdH*@!A|fBrOZ&pYn-<-x z+=d_DLxfW^`e^FZe!TJjB*kD_mT1IYTMQ;zJ=Cn@N|j^uy7g#BKM4_eCUg4{$fqc? z>SflW(3LMZ`ir>mU+F5_N@WNjm~Tg=%L`a&_FBAz4>lT3vn>5$RP#(of4>-Xymv&I z_)Xd4y%)@(=)=r1w0-Jb&0HN9!2NAd9245EJX||uYpMAkSA?6QO5E4;jQ9FKXS^eA z97aJ85{>j>i}h@O{n6??(yy5QP8$Cfe=PXs%S3`{hDs*=a+^mH+~L@7MbfaPk~$P_ zN9|T0$#&t>p6vx1-B+qTz|mI1R-5P$EZ&Eg3f|eS7qg~sz^Ya5HlObPw{M69#sn}# z&E+bebmhG{9|VwuOrC+fqioHVm1YfG%iwAi;O{_Rd7q|2-zEH+ja~1(cjU?!rTyu- z<4GrK2i5-kiS6%pPAuJvyUB9MLB}HJ#5CZjUt~z;(n}`o+8fah|Gu^)KD?b?mRTr# zL6qs-IbW7Lj|wV6a^oxR{`j(6w!6v41^y!9ANv4rQ-g%bNl|J=Rjc zt@?J8tKs7>kEJUkCUF-lwz=KsE594Z&|a;&*gq^v>am^l{8c3Rk}plqN-z1h^CGlh z>$kqdE1@+L6Bf1F5Mf_cgMRaT32~pU`{O?cKg!GZcBP7rPgZ=>EAI%fWF6HE#{dH; zr?pJQ?p*InlGD=HqD357%-`4C!7p`ti|DCBtDWd3EAbDiu6AStclTRA3uj! ze$llrnBDEBHKr3rL zdW3p)qoz_xpm)Vm&+QUlf-3@5TBnVax1cv?#7N2+I;#hbfF#nxSh z<2c_frrFZ50-0j?xqnhQg>7Bk71H&An!FW?m1DaqV@~nIZO@Lq|HU-4{CI0Nc;g=1 zVcdu6!j!ukn4-s|Wan~3)EOrM0jfgo;Ar2oYbTi{v%GwK$?IiI2Wr~)HG>`}m-XGVNoSjrZQfN2?t*#PsUP{WM?!JV zmbW_BVh#VWtw5>au*+RSk@fDfuigj9zzexw8mJ%rIDf7GVu4FDDXco#T;k4u&EAh# zmmK^bmA|v7DSXdqB1uiGcy*AUs;Ou{ipr~HVS^QUSP33TpUN}zeSqZEx<{SL>bc1L zb@H+THl3+9GR3=})7)E1?A!a(Er63wQ@7y)cPOwkvv~3>RiCLsf2yC&?|1Ffhfm#{ zvlQY~?V$wbg4^~#l15lA&hgB@n}7LZ?YxHgG;32J`higD#D#lp^gXT%8(~Pt-F=PH zw0+QE;IP2W`Km4c=ucD(6$zX5Rm`*o&+xc&4C#GYIwN->9QO9>Xvz0`c5q9E)Xe09 zfB1m+o~M4lm&=uO?hy2mt&enjpWJy-uLqOMbYUpuUa9zbU|^-_9OkeBiW&HuiK=1i zEFa)?sr%y;$zg<7f!ldUd+tmbI{fH?>@dACplL|g(1CI(iqj7?jAL?nD=c}$h_15P zY_M5Ty^Ddi5EZ-kY#(_XZoepN4UR(B%6n?H>0oTrN&QTUtS+VeGSmZam-yj(G^} zweyxH=Pm?Jz|R`YuPTO1x%Wys{*_=v*o_zX0)*oaoo?Pi^Lt#atHZ+-4Zi}sSM9h3B>GFmF;v=p(n2e=1Pv!K+KJz6l?QX|jyZOs%^_EY!I$8NT8L#dr2!&&a(aRwrNK#M5P_}$wkE0+hAg!Q zpKRQ@`=(}K!9F4CIk=#Q@s8O70&|L?H6l)wD5aJIhPWEUm zZ2RLs6ptrEHqzjIb#Pa+Sxs}}kh{J31Ru~N?YPW2KZ6iKn=aJ49Fa{9!}_nAMdqJ& z`zCH@#=)ky>9Y z+mDEHub0+c{{$=Bjsgp-gU& zYl(Ww+d8+FOLDQ?o&21#FXRU<{s1wb+rQXwm0_%esO;fTM?m@Zb`1*rC%#fa{J?>* zV5QDLKeEM-m)f!vdnNN+i^KQCtgvE;x$vs$8<~DeM0LHWm$Ur$@i+O>@N6di`QRJG z&9R|7BY_NS_OznMf*0P5lxE~>%Xd9UkJos?`+M2?2u04KQ) z(dJ^k{N#%F7f{l5tHvP0YCP^dXxU6$RDYY;6k{r!UlIK>;m@8T@$VhLKC{n|caHFf zI&~xlUUo82jJqoOM2YBPt_`{tk(bfNnLbA~OP^YrW1X2q&e_Wg%Rh*0`O_22I}vjh zuSic(lEx}Ng)#z;Wm2kFCblE-jcuh$_%&;CfM9gi%*|V?$!Q@$zi+lpX-?N?RH34f zFO-h*d!A)F3sHgyyj!TbbDRi{zRSCoo+^O!~PanO0FzV%HC~~?4qiTHWmOV24ZU3Zg9*90j8Mr&O=?Q0SCdYZ* z_X^T~mlXreB7Gy4T+&6JODQeCA_wojX4io#jRDt;*7Cy^kO;}y5vO&7-7g6QqHz|} z%OBd=rNrqhNcd~q$}%8rvZ^HO)8Zctmo5KqbYnAO`UaNsSPcKQ&grN{H$ZQdONL4C zfbm+{?UUzOS3neX)4{gB>_!7DOH^*`Y)GbZl5FYKGn zGRJ-oO{nIdWKEaPKR8;D6lqI6{TcWeaUJAL7H$#=JDlTUvej4Vzr)FybGmBR_wDQn zBVvOYPF+I?b0NP?BrZGb4Kd%opc87z2I~FJ^9_MV(@2n0C)y=PA}JWLw}(3DF>qsq+_GSO3_ zd0C0v0?lbN3X$0lY(o)ON3+Y?6?ju#yQH*CrG}wne+t01hs5q)cbAos(3i4n3fAYA zntXHv!t(M4v|zW;db*Z)zsTGzv?<&mSpU(*7nYuD?ek{!@vfE7!US4s&rkiIT#b`3 z@8@rJTqMjGLR^2kdl-B2nCF}2k8R#M$Ned#u72b!W32j9*9V2?WCc#~s(0@s8b`my zD!;d#id#X#s22A(~+-NannVAR%t%8fPu2J4lF0z@ekI$wi{>N z@or|ob|&H_2rsvO>+;sW0v(soXuh*12BkjohgH9Ypr_|GcnFoOU^-r%`5;cGx7?_n z^bhutioXh3nxE~7O&R75iHfwe-mKvA6{LOPDq6kL%i6hNmO&QTQa7-5Viu z_izwIoeL=+UamQ}xSl5%*nVI}!{M_$wS|b1VCMtZaz0(WNMg?y#TA>y;O?F?s+52Q z@?k87Y;Z>p?C+|RVAmJa_9MF6J$&|x0Ho-L5YCGG8#hhyQRB3hMHyc5=~`RvRO_VP zb^`R%0Dwzdq7mtKN`9vseVyg!^J=BL2QCsLlrLVJ29nmKqwMdmBCkX>3DB}SU7^R= zq}DKl(F=Q1b{V&{Vruu^BQy_NV%(}KJQkjP=o|s$AaEfd!Qsf$(atz!Qcq!vog}{x zqn6%((c@Kat@HVZP;`Z`q=lHW|2z5bVr#wmHxJ>6&npcl8%?hB#P=%-o|XMZ_y`_GtnO?;q;Bls0ovew8r`n& z(finE?fuHG#Plq)o_WSOAxcZ92ultT&VR>61*7@`K zADppiitm@%bL8TkMQI!fjytP(UAe|1##mEnowRuE?|Oi{cn^<0k%v?XNP^z9LRA)qInIM$20CS!7Vx zE;C}ES=N;ofj)t;)hc5AEtyEl~HQ!zafuBKPX$onvXu4}9YtU*mpP z%;Im-&iCm;*9KTw`$E%t-V1qG(sjgZGty}~stHcYPvu(G_>MuU9v{UQSXD@AI79KH zWkrdRbKNI$r#%mn1it6aLlp{Jj+lyD4bHd3JQs zQYO#Te=t&fXz@KtL-cJLcu+3@d7@dOI={Lf#$(cGcmd+@_CCBK!CUVo`ygwWybBMc znmt$6PYPZersEvW{T!EH7oNJ^GlTA)srvmfZM zn9qH*Tg32F!1*HQ)ugtCNP}kQFfFMPGyD0B?kOy@WAfO?DcsRSGia@?QrZ(l2ECyH0Q=T&F z>tuSf-Zp#-?pv6nO=kpq7sW+7K{7jD)T@&Y4PkuXI^z!MmTf4jDf%v@#~FAdDK)A* z;IYd5*;=P|4!(!=;Y8Z!`>usO9(r7zaBv2eC2=161hjOFrAMbu)4Gm;tat31kJWI& z#c}ql0H=fmm-+9w(RpC#Sa=K7ACqjdWI2nMM7{?|z|1wC{q&rr;9s zjoZ2kf)ju^4^0U^#pqV=Qt-=wWch2M1~b5BRJiq_eEJ#4RSMuh67#T{w9QcJ8ot;l z&u&dOPGIDa<67l}n79SmFx~9&ddm~qhdK<(2#*xXvj3EWMm>&v`n{-EMPBIs*RJwM z_KSNRabAbA;bJp-9acMBhXm_ACvpEdsj3^PsiXrJz)Q**=@*13)*b#`j+3fXbua>aam&%2NmB zeaTt3_`3wTq`#=Y^Hk<?R1YlPawtsn1zzI5R`H|R;Ic~%~;9i*E}9g&3WCH?C|Fr z@gE?E*TN0nzshF=P7Y<&@SNZF!#!Dt(9`t*{>(RK1wELB{bx#_W(0f7?u9o})iKle zC-xKO_MC4L>;Bs!#~ulsreB2wV~cyvK^j-NXn%}e{YQ57Nw0QDxo#|e`OcHpcN5Xs z+(nQKk}pRoeyIFXB{%Jr=G!Du%y6db(EHA;&a?4TD~g8uSqwke%CgvO?6N|*{V3z! z#-niya|1g~@|}!(&n{6ppQXQwto`@fH}w**Ac-}c1MbW1K#2=lf9mV0w-*_ao-p)C z4;m6gD0IepatZvmt|J$R&-?lAF4ZI*xONGi{9>Vlb`Q9pcS_hA+4PC_oW0qEi~E@O zT}i#`7q3e78SUtzEamD;YC_lqXMM4ELU6e(h;CF_aH=!= z3L~b+2SP1X#Vd6O6HN)8fTisLvrMFzXDCuDMA&KH#r6B={(I4{-2$SC&XOP8CXErQ zmhQ=2EXO=&%r2zzg{@&L6w)f3FOk|g{#aK`t zT$v|ITG;u%3y|0zl2OFi6EoGFnLBC=6A!d(lS2h<|4~@?bFj|rtr5bD4G9%@)tf97 z@S~yV7NQk_PL&d2&KfTq0j~r3>2mPv+pxRIu%j1WYKs8@S_hk`x^`2q8W#EP{mjo& z)ofOW$j!E8r!8>n&TEvQivTYa-+0Uu9ARY7wX*R=uuk`;7_4LhDD2SJu&A#8eh|o!IX$`mOOEuWSb>HCd`mEb5hG^!j&sWDk7T7(*7OvUjyhV(od8qaNctG`_EM$W&85A_17^nxIng%Y zN}_SO)hP~vjjOU_5a70?-sSB%cYu9&4)!e{+(eB8Q;uVS?at*moNxGnd7Gb2p7Oxq z>(4?2%8P#^9TR2Gf7g#oZ(qB4qdADl$0uH5_BW@GM$}D##0iJ5N46qi#;&?{$Ul#= za+|I?-H!oDbgVgcRXhXX5^OdT<4i8hyw|~nx)GjiDd5{Q_K<2#piV1c6jH ztu7|S_AEPxIJkUK6umo*RrMIT>_0meP6T(xePlElP55<-cDv~BegJ=px<;x?w<8Ez zAQwY9eBq4dAcbEW;0MycwbzLzm_up5bu zkf}Fz|A?4QGADfB|m2=Mih0GlkKJ5*C$7u~4V`e5a4859g)?v5ks&T0OOu&SxyJr2res zG|g)>*^|C{zmJ=q+}$(QR(~lNGWXaXLqjYZTlQw{W<2q}{N;T1zXF?wE>?~IZN12E z)DkA4sqF}AC}KmJ)`uWTd*SUM$97T^$i5f)^N?2B;2v-uq>_6vh;c=WwiO%2fIGorm8hMVK-^;xY$=TA7jcw6CyCfJp!M8&P!=+s z7Bam@w!(;7Rgy$w0KS6`bXL*aaOJ4DSOT&>Dt_1re=-2kcgX{n2b=fX@~C1j-agCf z)&!qwE1`xKJuG6lD$Me9kP!!6rGoI{o`T-S4tp=NFMUXOrQUNWYQa?X#?>KYdFE(r_W1H_KLZruaj+-x5Pe&Lu*`SvP)U@{ck1(MTlrytgpQ$HO#<4>?|t z+dYvys2m5qdA&b&HGFgQ`;6!^*cGQ#xl^Kjy;rLxAR`b=b39N+A|`>k{dFLt+pE+5 zKK(1iat%Wu>g(m0LFB>WJ4AAwexmT!4HtoN}Qg@4Ozb97(==(VMzb-7TQTSm z8IZ-TCQ^JqgPE8oUYS4G>Unj*V99gHeIlzxy6KnSM;aqOe$)eumE^jg5EU7p=Vb$0NCDK|!E^dcnTFWui-yM8QPpO3p*lN5FAL6I%4jxKm zTSk}zJO@=o#3ubn1+g3aEr= z>i0-7E;`3qlxmV>M7E4$33Jg8w`i$8vIR676}OR_#Ysr(1^^tYd1L z)xik9yH?2hvUQjb?Js;#hJQ=)o??h}N$}F5l)&&0?y*wv5{S}uSc@}=;rlHRVt+Y$lL>>1fb<%9_orH@Feb*X zqMz5a`)&eN>NM<_!Bx7K@_!baILU?WA7^tJ&2R6wY(XD~_RdLM{Gvr6;vY&}BF>Rgo4Y5Qs!nKS&5WWf_@vL{JkEl=Taq6{HMUWaO^L?RF3W-H>q zuy+^hU#JWvI@3&$2T;mIc=7v}A@5lN-xh#H&RD~?yXs@k@|IJj5W#_+26*@Gu>xQg z)rgi&LojvU$b~d4J4h_CkG=4|4%?I51 zaMsq>&qWH>JA)cwv&YZcT;T?Xt>IlUE1pMtK>&Z>8evZjbY{gp7!VTvp0bgD~Urb^+bMmcnU{1YD^C(^Wq6h7XOHwniCgcML?K14^m(z%u6 zP)Zi94b%UzeZ|iu1)|{6&HYA^_0FnYb@)mnhVmb3?}P?dpE#cBP@A*PPJoZxs@5l?zIvs*qk#Sb!#S2ONKUAc%%2`Int7F!dA?uot{7~Zt? zGLiI?SBM^3eh+Z7nV0xXllg2st_D&xq(=VG;%oMWscj|5(N^C~qsGtxhos7T%gAJ| zoM0Bk2<>Wv(NhrdPdTY&k~$#(J+aKwP%RqJ*L%YKHWHLB1P-@^32QN9OmrLg;JltMnplj3ZR6DEGcRQayJgoLu1BzlG?EMsP$8u zKHuncE0GS}#E{A*&z!{CeF@47t~_^|3FzKRKZ>c7fhvGydvtmtO2Zc1V8yM`H#};_ zoi=DPL|x=F39%!UFQEO=cUtoI6{y8*scg|SJ1R3V?WjCYjF@2MKEfW3H{g`X``z)^ zItEJR%Xkfrr?s`SflX<`Ta4E>tXgRkp3|$3v?6ab+7yTXi^6M&=r!Jc>c9eoHby!YThD zYaLlI8-p=lgJ$|!LuMO(L?`WfoTg=eI-|RO>23jyHM0%x=M4{DBwCOm{bX)dIY1Kz zzJeG)vW3zzUO^=NOdVNzZpy^;F;ZNS$;X>a~ajyG5NYx$QQtBmEK zUlaYRYsY_l4UlA7SR74Tq{Wug$a_F{z>>t9>GOQ7 zU~615{Qa z3F67uNi7Qtz2SPeN&}o4j5wkX!=lI-S}5v(rw+GeL!HC2^}nxH;e|Tp6@u845*e>C z)>u_@lZ@;xKeT$suW1M)9rrOMLb&Z zpIG&Ahrp3xhQ77Jf3qh4VcrSAjzCMKf6@OvwC zMCuBp#~CSc5xHFo?kJg0MNBz8V-F68>bwi?c=&jxYwgXFDd?ZhJC@w%^H|lan^#0_ zPFr@K>uK=DndUim7`)o+SZgg<*?~4}{C3yA6SR){87CY6iBn&RoD2!$nw~aRFq%cN zh}e>(D1C^fg6E7J$I8w$;Vv|mZ44VXdVq<=$OsAWRSR%jq`BXNBVn68Z|xr6*d@qn z!%J_VcmPq0xZ~^XHLfSSO}-SeANp@&9l#o zLm3E$bX(#Xxz6#{eF=33kX@dC+>Y6qdy85{v7F8^KCFUVbdC6uWe8ZQbI*Yw@O5wB zJk)6rA_T2fPxvlk<_ABt@dWOB3YCJ+stoQ$+_M|J_D&czJo}+QI0uAY!}MV}L->(J z?X-*u5?-nd$?TGg>7@UBNVn1lu0K*8(uqt~k@QK{Epi8x7govH-*o&!;U2Oa@HAPy6KhDNU}zK&c__Sy7TmmOrS?$I@8B!MC%=+# zkgfvX?x9bp5VV@Ln5er@7@n>wH#Tpns?a-)3x~~-3X4d8qE~@5&?-F!V6jn{Q;p`q z?^wi&Df}kHUt63AJxYNdRQ3C-9P)0m(Q`Mo;rS>)ULq@ zc@nCnTbVp_b28CC_1t4p)V#0oPK}WjjJLlZ53T_W4wQ98Az@|g>+5M5U5Hbs zbe!A1Wtn^w^ZlIWkl+o%_E%LQtP;lZ9Al^Cph&(cR|g zwDC1o0Jwfm>EUfg)8KbG0UQS}8du&mRzFrA0m&Uqm5|)aum88vCDgk5*0qNk1l)Zn z!4tk+vGYJVj{b1^5`8A1l>-FV))Pg|ODc`Tl5|Fw8N!Tg)zjJkEMX^3_+x;hPHckh zPdd(FLauy66#S$PR~K`eiYF$@2%k0m9T7tOLr@@=A-Oydyly{FX` zxpozjFeo$jh6Y%xcU}46n}62Z{*1vsHw(R>H2ifG`SfuydW0Bq_yRsZ^2BpQ%eBy^ zOAOH^6K>&XF@0Qq%7&$pYT@uha$ec%8~k!K6qVT373d{CxTb!P?AQ6}r70vPFOP-5 z&C(zwFK)w?Xt&TI2F4eo_URSP#}nAU)8^$6f35q@(b7xpCm$!-19mWU&)RZeQw+5@ z3HxUkr)~oLO?f9ym8Q|POa$sca;}@0coK2}<0m0z6^HK&c`0ts2-c$>(N9J2!jVUF z&g0!A6#G&s-z1W@38hG$r;?$xE#W4(l?I2@ERi&B0Hgou|1t?VboVAogApG&ndU&| zf^2^VH+1=bLD!W%(!H{QcRRA0c{$)Q|3^U+`rJC`$oRwS!jR(|?g2i3QO z#^%Je&6DjdYluh9xOnHJHiriAInL*eqs+ly#612Gt!J~rEDUT&-=}^m<{sDESPr$z z!D4C}J^|qdCMFVhKAi(AE>>%IP+eeWlmCQSmAV|QtdCDxOCs9dB>w#f(NNS55jwM^ z*y1*$S+d$%1w=uYGo9l`mCDdXu~BxEHji~&Q^lNf(RnwWm2zB9Gs!Ka;>JPM%G)SJ zB5J7yAr=KUrv#VKQ*cIL9JOqx>Fu;>2+q6BARBuLU+_#(|s!V8g z{kBrZ7eBYEn8!{!9~mf{c@4qqiFXjM6s}gBcG!Y$z1`O=l67j;OT4^Yh}eq#-h+tF z{vx*>+TtiA+-2}^DzQu$%7zAMiXEZ&Y0B`Ypo40mMyfk3k!`0oc8@g#+XjM@YZ3Nk z*Chyo%+)(jZmf#ene0ZYHMLBu@S+UVWIet;XJY+kv^)vjZGM|3c)X`}dPm9dpj2>y zA!w*6@9FBwq4^sh{Y! zY^X4pipJHodf-r}u5hq0s(lR)#5Y$V*DDC56E!$JIKJ(eo8gKxWL*(;yuW~Q|3(D-zXrQ}Askd|zXKk*&d z+xPP7fYSn&mu*;Yr&`&KC$o=JkE&Ywh~r;1eAJJBfmXh0cIMA+Z$oMR*UYmX1C=AV z&W`r*3)@5CEX>5JK^_NgL>Y3xL+a~ERWNazY)&07P%FJs)J31sr2(pyzgCYj$woA3 zQlhkMIdhPbYUcx&WoCJ4a__o8||V56IS(w+v82z z{l)&F!wqvnR;Lrvu2qa9RfVMIgL_8U17mqSeySl(NRGp*1{(*?E!_=xB2Mf`mav2+ zsKEswru}R#K_tLm7-hO;cbmU4KZ$^L@&Z1Owhf}0fk{$`DtxqsKr*$OB&~eF>Eb%O zQ`V&2?uSK!@fb2`MGek=97&29ArM3I$hT)^mw+s^jla;k%p3^1c2rH!&KLXBR~QUM z7EhuFX~oSlW!j5t>qD`Myr`d(Wp^ zDW5*Cyz@!_;(k#lWE$11dn{A4TYvy!qilm_T+7z+=vrMCLQ)6oCwF}dr9X0MoTc|H z?blAqZ3`MoJD;}kSAW$_bz}>b+z*wrd0<^~Myzy1a=#C1=WJh*V#X3_v-Gi|Nif}H zBYy~XdFThD+n6A+H0)vV@|ED(;C^U{5eAF02r+DT0R zQAQKHG`Arv+p$l{xt4XmusfR+s?gWZ-C{|DgcxYz0E~m1;8jd~F~P@)m;;SyF6Zn#}2GCN_r)Fg{NaKX#Elcv``}4b`I>D*d>vr=;0yc9>-ACeY)O0J`PVz)K z#yM+VDevO!GP%t?BkB)piEj!FTY^OQMZ4(W+C?-0Re)sWk%@sj} zf=Ge!h)nTNKoR>VF7Nc@P0cQGm{fVY^rvJ|rG4i8A)~ZZtB;V)Pmn>A`S@qLm%}-M zQF+0=ePdfF@_>i%knBb>MS3`cYwCcmot;7m)9?0bO|KN$Aqg%%UV#v|r-}I!%FBpo zXk%@Q#J8iH&n;o(B4To1FHf6Y`~}pDkTy&<$th*x<_Q<|_WWWNr-|4R=6KHpdF=VC zhr~kRK|G2MiSgSwR;J!;8iw|@qh;Zdx95)bvT&$z(6!Bw_gk=V8c7X~L@e#b2*l;F zDUs5Y=OzPJr&S}9i9sY`)RgzuLKBZ&ylq}RoOC-bM*flb%<*27^gb!2aar)h^U>BP zY8Osd_qz_qy9KP+LccjN?{{b)zmWhS0O=w!bc~>|acPFQVj}My>h>$e<}Zlz+Md*i zxj)M}tswfqsL#yiKIg!lIz`CkIV_LKi;v|Sn#c)@*&V<~7&Y#Fc!ys+i9jF$jh$(U z_HQqz&{ats{^!kMdR4TB4+hixUwS=Bf47`dP$-*Lj=y^GVp%$^@x4k8O)DkusPv6z z8-7-^Y9PboZ}xqN#5fqiDm}UeqbjsN58o?&Yxj4+H2^|MXe&7}LsleftF4-7guZaD zJfN?2fEu!|-W4xDkp|%B+A_xBKAlYr_)$4hLn8B191+EYkvujeE_t1WCbH(gT*_>E1^9<{=N^B6yuh1#j5j!d5_1j5ha@C)i=az?3!KzeS&Sxkzd=;#R2ylf z@5%+75RMPXhjcPPmA7D{Z@Z2BP3b$AWDJ*>K3z*Jw}_tMBC!H`^6~kxyAsQ-U3zsE zQRRC~Gk>2|_NzT&GF`l_m;pD42!C0_@JNqpoz^Mv^Q@%?z<)_tJm@QQCj+p1ocqN{ zN8_ZH7`UwbEP#9mFgn!ETF)E3j5ugIlLgeseO*PGFn3589fTtCBhb5*RB*@D8-k$^ zMtbEPnHc)VdcW?@mJFjTDN%8Jq92?-g4oZuOnQobSC>L?SiGQs6sGr;RIq-KPkOvZlQV!ehTQ=Z4qr!(w z+Ga&&2Oid(T2dJUIcejNR6Ce4vIBYDjTH;I>yLLX*zkx$FcGtbPyu_*`(sH|Q7NzR zCpssjqG6|;pwE;vC_F>@evN4n+*eB;EZpYVt2kH`cP5Lt)cA}Ii|S@32T*m4vb)MZ zXx)#)(5q-%KtGOtouU8BdEZGECB)bLNH=~*-2{*s%P{J^Dx&m=k;52?JF>myq-aN7 zGPb=sH@SOqDgKXLh+hff4n5IoXb~V6&e2;(^CqObv$cw z8bV#EN@q*zL6KAvdl=y6H+j;(g4@$~4n6lViCrDzfa~*HZc*jD12b#o$jC+s-vaw; zL<_A2TzUmgPn*KKA!p9%w=8;omANvDvJAd^$4>5Wm?sp@iCn=*T$EyU3^qL1UaSQ|4OD+WJXqkbwb(0Kc}d!8o;h5sO6 z8B(ImS!xn1k`V9>J@x~Be8{snaJ-kyTc_)fpP}DJjacZ5#O+%PsJB1m+HphH3)#@^ zZo@)KqqrK3(*9PbCF-3=)-gV=CVJ_oMhqpn1k5Qsty3Hwu>2H#iOf$9sQ z=Pfkw|Y5Ds)tvbj{*yi?&6D%~MIC+bD@D?iew4dsYH?t>@re@V0; zusq)u;4K3QbZ*?h?2vZ|J`{i#`uvY&wH1KvLR>?#+%lj9^>N*7P{eM6(bAVF8|Rmw zk~WzeMXVBn|MS-=FPyQn7QF{y+;sRHvY0hG@EyU>SFR9fde$x80E@G(%GJyMeX4<%EXL;p=^tpJ!$mQ;6MVWBLD7)jNs1TOY66y0n?1pz4ez)pP z%)CO0T8@c-W9t5!!-Kc%%$ZDDu*^(Keg;@Py(M16wz>p%O>qAAG}rYUw4@ik&rNcH zZQS~ET~D(vIl5afRuy<3y5m~rqg5Pf@Zq_X5Kre=%op%OdxC7YJ@%o?&!4T)qMZ8W zD^a#J5&Sa++OY3UJPW(}M>Trkd|7gq-wl!X&58R4JBc%GY<0eB0keOT*2+PJsO@mD zpJhltqyt3`&m~Ut0Ueb(H>wq5U&ku~{|vO#CzH9&cYaW172!o}#PJ^hgV=`g8GkJr z>-v1ud1kU5XrmQE%-q9y9!+Pq73TDH_)6oG5yXP3jS0Tpx z4Cx7mcfP7p^iJk1CoS_TZ)Oxv))kSiaC2Z1BhIfNb*g9SW$pDSQ)59I>O9l^L1z6jePuquK7MB?bev`w7hNvd;&U!iUv7Gl# zqs{S;e_zOQIgqtS&MOTA-u}9H^1$UEPG?GdTck$J9dDxMydYa8xY`+CERSGlG@)4I z2g|t5c*(YnBI~Qf!Fg|)Vm)wjFv#!R>aith802buhe=YvLl``|v&tmdKcU>O3Co1n z?H5PxP6+T0JYWxlh3)TvZLTBBK2xeFB;E&CWlkl8*jMhYuO%3mj+>V;RrD8a$btKm zkFEAS+W!7|`c~!b<}s23|9ZmhpOxzj!oiGb`fP<0wd=v;{ExhRhTK)meLVEEa-^x^ ziOajhQYI2z*W0=idB?sOZ%7CQn^CZE8Cfge{njfLkwP0rcBFTf)~6ChsNuj(o~g*- z_Adyk@GhA7uT-)`ocn%2s36N;E?UOBwH>Q8y`-^ZxvxhdUAR#dy8vr^5qz!{N zZyvAW8BX3%7(p<8aH%=zCJ5my1XU6xK9oa0FP=6L=U$EuPrWL;n*-jmfoMOEvu`2h z4zuKbTW(vI9kz9exsqYP)a9q!5>1~nx(XkLI-;i16Gy>Fz0w9uOD6jSAT^Pl`?OTJ z5JrynrHcA^k{FjowH_xtBx1Ly{qG(vA&ISiiSI(Zt!s85yEk!6_x}C4fmPBxG(8LJ zcgMm?Wvh9U7sBHYqloAfmZ#sT(`sh6F#olVbW+z_@4tjb(?{v-z0KM9UMCrbEze$l^p8x-we| zQxP>X!Ta-IwZG!L&}W+5gJ`P9?BKon!pW7>M^94xGbbbnsuVU3fB5M}*S@Eh@@5?9 zD-%8V{exuKe;;t`;;N%)0}e8JoIY$1Az2_^haAdN5|#7BN{fJ9fSzfgOTL$*Whykl z8dSaig-hd25L|z;Mm*iDd76W6l2o(^a7dtvmb-!Da&NifMv>kW@_h|T(fA!y+B9)| zvd2g|m2r$BlL~(dUt>;6{f2i-Vl&>qpJg}eNXoy%EUS5AWGYH5?|UP8 zre#wq$eTQ$Ug&P^3g+T;pLO|amy#mTpV1^UB+XlPeRyVujrHhA9#5&S0*?|qi=SFf z%)ehY9KOUVs#qePiJ$%4fM)%;TyXeI2}yYEl- z^QhVwzS_qt>c4kSFtV|2&CDZx3Bm>|-?C=dg2pA>JWNi$M3Im*1*GR}K2S8CJX?JL z#F7Aje;%(sZoq6tpnkj-Z=XXZzpgs0!_ne%wVcCzZXXZrbwwEAbYPz=pK8#Cve@D@ z#7&#gVi3aqZotcUAF_bT{C6&Pat^r*qtx|Dr7&NC8_%JGd#0+3*Stm-s0U?`4uQb6 z5Vy;}^(}AJc>U9s^pS1k!qBCmXG)Ee*21l|zimIT|13XLKmVw$@TXD5g$ORFc?x;j zc9?2+#IaX>B3d)#z=8YSfh+}7GGpEk+r`h`yJZ0T-SM}7GISt_8;#Q;OyR@U|L$r) zqmHm?HRW(*%XM$G!~&eMi=&lV`rd4l!T~ZZzE{Mu9^GZb^yzDpE`I2o6H<(z$7i7t z=)QVZ4Wv8H+d2 zZK1;=(f&R-!lY5IaokzErlA2{h(F9nOBg}>AjYueGqgN2C`PMC>10xSr9m{WYAvM!ZH=xM>b+lLZho8f+^BM>uiP8@8$T*3UI`Y3(X z+$t@iHsbmt`Y;fQ&N;oro+|kBrdI81K&5V3B#kXKPi;r=n@&4BVe16&iQmT;s|zD5 zdl;Li+5!QCFdWh4VH1S;KK4cg^NmmNR*YFr!*a(-2J9k-d%bETWFvIcdy&Ba{lXfn z{I((c&K6>+P2QL=hWYZ~04j{+j%Py_Pnz!R#4e2_R(8|baQ4@cG-%I0WpNgrBmG?B zD*No%`jz`3418(vS<(Tyn9ybbXtqMX4#XEo$-LtH_cFu(Cne32YyQ)ut$RRkng>Vz z(t+?TkiS}%%5i}s8ZPbc+?BqGSHNnVtLeOYZCCq2`CDA4kNAWML3$N^)PJ9hZ-IBW z6Xz2=3t!au==$@-D(TTs<#pl$v7a3{dh0?9I}Gf2`c{NdWIH-IaCLA5X8qJdb};8y zO~G5{c|3mQsA}e`$O&O--h6X_+w8ET#E>$E9j`A0#vZyd;Qph1=TjSZKdz~(Vrj#q zO3?gWhQjg5PWF92#k>i2oTZ`YRK9mzJMaG3z&n`0Nnfy@og!+nV9$dZXl>{HZTxr& z&qvie`Dn{)7~+#YB~5tDQc}7vTsI8xGwQkW*n*+nQ}Ke2d!PZyFIn77vZTrsr%i{e zm(%*Ncr-P@{!$WLhlP@r%B0DyhZV&+X;5O4hE)OIKxn5^#evum10<*eUxiYpYwvPe zo3%SriQql1-$jpT4X-FM6)YSNAI2KV$2?v78!OSVPMya?EM?F4Z3SF>&!b`C!D$LU3L`b_H;CPeQ` z;Dv7|-2_Gtc@AGCeeyV}fvmLctV+HT)H;FWLs$K-OuuhL@Y-_8EB#?Py4Dg#?bQY< zW4|;9+!W@++MLQqP@c38{gB1CG9>Z0i|I~%!TM&voxbCXeJH1;^@~(2X7#obE=%vO zsgq%Ox+dBBHPBZBLK3UN?O89};GJEI#f?8ZI&NXK%mrHngKJ z<_mECIaN0Y)JzphEXMR*i`Sh-i5Vvsx#W%B>>^luo>`Vzf6vqZ>V0P;zhkc&-MpGg ztHN>ocs*%pU{&^Qe9s2n4}NV@B6Z!>QWG3^*|Fq61nAvh_%qY)JCqnSDre*Q=CqUj z1z7^8?Dl3^P<=K41;=yuekG`q)mb$BGJJumz?*M7%NuNI1L^6YGd++E8EQ8bhUb7606IgvYAP!G>cl+QFPVtd-~C^^xpvgIA6l_xWRrTnwK>1NxDwLOlC>!G9)M6>m*CoL0Gyw-_)_T-27@+i+(JDs>g8bTX4VkuXV6oXfGJN@GH~Q$4_c=ryW)H z-f&BV-LPo^Z_g)IO9~PHPXH%~-%sNxw!?;5MqM(9>d(5oz(-^=sObPf%I*55U{ z`mJw3$4G1vsBMRKtLH$~B5~4j)Ag|aWZ+q|N1ijG>w2a<$C(sYAU06egyVzvzAA5- zH5U@tQ;pfx+?$GQYHa+i$nnWFBVbzt;O<))5uOb5Hs1FXkj3 zrR;v2DJM6n^iY1vi`sV>sz{*!%AbJvTWS0S*Wo3WP6sTro+&Y8yHqV*u&|jO*1|VipJ>o3d4K<+g_4bw_5uAANdk`FrK!R zBT@EU-J6Rzm-v>)J$-&Hvk3|ss7PMjernsbFqa~gOgC5@KYh1;pcndGkay!NSrR25 zr_`D0Oe<rt$NlObBoDT2Ip9)h8)iTK0I^oN=u7s}={)>KydJg?DpgaY zRagY2PkEq3*8Y1tfCyD;<#vGp7fx_Bky{Wx7}%bjQgRwUo_(Yb>w%X*uNUe4uvCz9 z7vrWa%K8vtqrH)4#$Acjb=wcJd{g=~!e<7Pj#K)C&0~)^esw889)lt_69~YOyJlo7 zJ%436Vnb`*p&Wk62g{P1ShbhE9oKOQicxOH4t?$6?fb(kHr#2syNZ7Cj!ps6BT@KB z8D}GA`T#eThY2xz|9f#7zoQ@<@}XwaQ6^ss>|w>E-6E@~_OL4wD_1T_*RnU`l@H%WM)2o&5J zp*i_CnAz-rJ3o-xu~A8#NEe*HWX*d^GMrtMcrrtZgtm4aC_C4+yC@Ut?~4$&hiAxv(xWU*|dA=8-yVAux<$_ z;{CAsdqt;=h#<9jeY}w9>>|CXwpBoXfhaL2nl{SOU{k^{;@2i6^5Y|LL%)QP3L4eq z@x<7W!oh9+3--mO*42QwkH++V&}Nue$jqeB?Bn>kOu)%(bQQG*jbaYV6Q1|p!&2T_ zuyO2g z^vi=q&4Kq(m;KG~)B2F%06?$e8M^l)nQEKW%MbOqTqf2F`E_=$G=a#bsQwOv^14?s zM}enA?SV_&%rqXVzGFQHrnSAjSkBhf%7H^lf)LrwmdLopGnxn~2x>uc@R3uV?02%f zb0cn=mZVdf80pL(OvH23x=@CdPre`?OG)Wb;TY50bD=X9L*?FyknXL+dQd6{s2^~H zFVrSA_dzEjZX?Y-9CX@7)YaAX1cZ3mqGg(Q&$!SIWuu~IJ66Rft;jHWE-Y)%t@XDe z{op<;M!^-u(PUCLnNPT4s~*#B%X&~v6ttqFbH_VJY0V{P$xL39$ok!y+)ic(htfZ) z=v&2fylf*+rC+e(>w%`x!5HBleaB+nnd?%9mn>xuypI8p`1$LLC=(8F&4z_ZR=pkKfItP4~135tPZmi+??>8#QE9xK>iVI#n2&K7% z#w1tsMe3lL+16O6=JL9#US6Q;prZtmY3~{=yNU-Yd*u z5dl>Jpv3TK?zWWcV9F1#*+G5#Hsy-X!~DNRIZ*``430fAq`f~j;pOo`{{5@FrHH62 z>yL0gJh_)i~D&~etBNPk9S-pjP}QYsYl3U0gr{16luDzI?rDI&|-mEC*$ciL$< z`o}%2ZUVbqo#hD#t!N0FWy3G+R?N`Dl=8SuV0A1)?2`0>tL%g_*tEwvP>?mZL)bMp#GhQ{>l9aT(im~UgDgQfm#Zc83 z0)mT2KPVkvN4qs_gz<-JZ_ro-xS&ZsP;{IAu16ol6NaeLkA7n7F)XicP`hp>3=j*k zo1|xElj+4My&rAr{?14ab0>;dk?|FLPn8euL=o-g;LiWkuR9)z27MiZ$FA%mujZF_ z!#}DwPWj`6yk(sEr@3ZL|FyllekW=wEJQ$vJrMHVB;+i>{y0Y|5{&pi5yC^(ZD&|@ zIQqYvTlkEzL&4FKnFHBB|3b&+!mVyW(;>E+tVSANetnI`Cj4FO>Rax(&b=S>KVb6G zA14okkp3J&8}F10%($>s&&WboOUh?3BTPIRM1Xr~1H%{hPi00@s;r07_zcQdNB4Ki z%E!hsKKNe_2Le9LSn;dgeME%7O?`qQs@3DQoW z_FhaB6*Qg)z$+l@t9#ny%Tu)YtF7WWFr>?J{mRFn{DK8&`lJ767|*$)`tDq*K{7H_ zBl6@Ynqp8OBn4KaBvtsHx$ptNZhe;YPsj!Y_i~(}ZC**yE9B0)jvp?d8y2pHI7Ege zMXQ?aKVWJ))-2PSy(KxFBrXzpz-?5$H#S{WlYQ+(W-?YVtBtKZ%N&ts^l16|IDzQ+ zH=Rb95DE}})m7H+VuwPhJpMQ31MDVY1DasJuWp3Vfup{fm<+Oe(y0~^kwTc0AN&|L zL77rBz=jWu)$#40sx9JWO{{<`|l3};|-b#Vqr2KOQS9lQdBO^E#f8PD_o_z%-43thI3c+VcTW6-0vC&Vne{`H~Fp%KvXusLxDn>knNX0U60D{ zLj%G&L`Qa}$N$WT3(QE@Rq&)9_3XUCe$>b9vLWM(rh${eD)hu8+2WD@Zefz0K1S*{gjaO}b~q$HwVhpi zyATZ9J5RxKbk;aG3iaXAGg;{zXEfJKt!nYzzNm^pWvbdE2Qy z^2`SZgszH?Tiuo$CPjnVtlLn#MpF{Y#K5b;x@iycIL~63nwzF2h3oG@iKr{?&y!VR z`o2aT+DY{`5dme9(j55}CLdl-`>)ko164P^JjA5!)2Yjbwg_|_Pht8S%tTT1Lq)GZ zddxLGOdpm{fxUgGPJ)Z`#O$t~k_P4@Bc`H)kFBl`LkrqqV1Ry~wbtTr2p`xV%SS5+ zVfW6E2p|Z~ShCMm#8yH{IRnfp2#tYY1^vAWHI=q^l!XWfp+6y%`!Q@!B51UNU6`NB zB)@{us2v}8Tu!nhLtos0h^OsGD*wsXmW|8YdV0J9f}D>J*gS>X5Z=dOVOOk8MMByv zfjBattwMjVBT@x(V4pa@$y&CII}kOwe|1S`IC$@;?=RF*aj!Ae(gvRx{XD=cVsF%~$Aq>fiJ?FJCX_{6M z=G4a(r&O&Ruaf*vDJP@hRhPGkZMqBZbj*!E;Q>$N)NtMsS{nZ^yf_BDuBeV$fPsBE z3G^c0(*##c5QeOfSDY7G6$9V64U&!5*XE≀uSJy;;&A;b@?G<#7HC#=!$|9B<{jP*^Nie`1z#>{JfiO1QVNl$U zL@7U!C!BhHJFAEkRP%PdgwMPgW-*h4vR63ReFQ;I&>;g?HpiA@K-D{B1fhs8hjE9{ zHZh7eqmAAu{v+JuefwdDXqj>tlcFG5CiZ{BAVaM+gR~(WEjT zYDp4X_PuRM!xvNDtF{TV_1wVxLt}l|;m6&mo6K_T!uwaV|GOId%@VTS7_=o^h|M0P zA2SB23B(x=`mIJLbTj9CKE384n|;(QMr5?7o$>G8oMKH7EeK_{9+rV!QuD!*EUqAI1sZRf* z8cd+|1Oxb1;_h?YK6fL(##IO^E z*Wvhvh+_4tnZ9x-apG?sX^m(X4UT?D52axrmP9vQ2)iyp%N`D=V@eGA{CNp8_7V{G zYVqsW-`7)PUPklC@w9TAH z@H#IyU!4jzs>9CE$|GUNs{*~VzF2CPq!G#(NDdEE;=J5@e^XJszkcuWG(~(5AMy3+LBc8nc~P zV4I}dDYrhu1qT$4K9Icc0>&=66n`7q9TEQ9PwZc&QTWs8wkkn$(>9}`56=!sd5`S@ zfjWfo?A$K?)LB9$sr8~BHjjCgV1^qi$9Zq0XXHY}3-(fIfl=QYqj?PJ?vd6(BQ5jp zmt@pj_NSvuS&CP{b&+4YV01|Ui?KcHZfK404#<_N4r8+Q1{EJlhLh!?_{qH>aeZ%v zM<~E@NuX2`UjU)Adzjc%NE)8}=a~wR|=Clck zbi2h1nSJF?>SG8`!-pjvU8yshJiTY9L_47AVXTr5-Lp4*+qn7;9q>mlQr zl_)$hI2t4+VTdMNRlH<6JGpMU0aV_yz?^dPk5A^y0QbIT!i^2Ohw(uT$HTRI83?%Q zCwVI3o@BoyK))WjFyY%8-HHPK=dv!T*tzg9-}k|uewvRqp5FWZi!MC-R1Tbxjy<-d zyO@(MhjB%C5WJ|fP)*|(N7yr8pIeP~M+6iWcj2*ebrg(=)v)SQ*}SrJaE$HP-=d#j zDenq;B?j5K!ZOUXK=2#{y8EA|j%LTfwkPYyt2+civ-3D*U}5FmrF~%c%;2?9z;fwZ z<<&{>9f1ByN*{m1{GTp|mY6iF=QS^=ZEDMegm%i`;mn9GIq}7)eN{>t__Mq$UlA|} z!Xka)g6zn5{;taSYD@I`nA^T|UB!l*V~PCAN2_(s!%DR=AHVj|_|{0WaEK5dBrmk} z)T+7%gyB2d!dl6qh%PGb`!k^jmYI~jpA_RO0dZ;Adb9#9%-t*9Ucfeb&vCbU3%-$$ zp~df^=N0PMvPDg>ZF<@Pmeen}>vmVy94RzKkcba-u46feQ3bSYLp zAh%uqZrV-^e0*}4gIMWkBNt>2S_;5jvq-tP`|Q&rzAa@Z_pk2*Ga)#ko4jw@7cO|` z3t%i^WMC{yHedNg%{*=)jvVsH0Gp#%6O9?AB-B9Dt3v05Sl3OD{}m;O8MH(q8;>(Z zudo|D1H#JofvJA?O_M8Yjk90Ws+xD9!!qrsv2(TnVo4EUncW zfOp>JK=fh1fWfQ1@I8qU^zi;`TgTwKZNd`9`pYTUawdq7zJSM$7IM{vAswRN89)M~ z5V*5yLLgp`bCV8_a%N5aoG5rfV#IabfGU^XVv=lVbUKdIeCgqi!*HjM-i^v(I#FN9R}B7$Ej@D9k4G z_8nzn?8M?)uskMdS`F(4x>wYEQQhLp5tO8a6S^%CF1u3KTpz06hedz_F3XR-u*5Z^ znE(SoAK5PXHQ6m^#yQ>3?zw)g;h>1Y%L^RWHrP^{OdkVrAsq>oxI^*U z1y$IjPB5#sexHjK&KH|_*G}vitu=b8U66~Ab;AN~hQ<8rA9Tv!<{->!l*Lb;0}Lh* zX|LrMo1M>g6Tq)S0)`WrO*)_#>xqv*>*MhV_YR8HG1P$vV#N=&(rL4E&2d%#<&k^z z>Yl)Aml}faLtOH;-`S2xanDks7nDLyGoB56Q(sn(UUYLEyL<`wu{aS(phab~yfL~` zH=>xo1lU*;a_b#(EF@=%_kt16!YbKRC@yeBqUS^dYJAhDw zN5p@@fIn1=q;e>+K`*T5oNkv{Y=Z6bNzw10X~5{X#(bzmVjb&Ggg|)d|DB?$t|MYIN&lj%6F%Uf~z;_l=CS)5*2;egUh7z_n@+zgRl>_5;RZiIzF=2?yNbdd+JpAtHqIGud6L1ueDYEs%*L+2oj$(}nYyw!I zOdwegb8exSweFtQ!?2!d)y}Y`9=(kpS%YP;0qEHAccf0f6^`dYc29q&1t~N~eDuR` z9X9Am$4uBY+SrAN3kpxkhr=XuDxR}O8}YIhG#v%kq(y?3LRNp-D#T*Ge)z~gy8OM` z(z5saQ>nNSyIyukksvNpR$lw4xTT!{Ua{&AzW6seRWD zBWn%>cKUP!m8G1QeAbU1^P(2vsSh~czIweWjwx&A-GR~8E}xMRy3USc`dnC{CfSgA2_|}rjzsPM`tLkr8%BJAS!!0$ z4i9aF-Hj|LTkb{xrn?cuKOVdpo0|$gid>G~`Utg3HcSkxGyfxea}Q`Um3@5}hZY`b zpv&41`!NXIuctxN>cPeae*#zO^TKKR?vU{&L#^+FaL=K?(^aw-8k*Ra-Tp$h7W7yt zCak}1qrdDIn=$z>29h84??)n9FFc}+9scf`A7iyc!gktPP-o?TWyh~h%o@-P?8-;W zew8i7h%zQlxdrHjjq5G8b@iqdc%cJS7CU9zv86Y%`{Ot##aEA-zuj}8WD%!~MfB5l z>!4|qx6yG*H4Sb^H~;LDH`ssG5_RiRe&1MtmCOM@w;gV)9h-CP6nbs!KL@SO1i=|+ zW<~08u5BmNM~1MdsJ@2z@mC8YnsvdpFOs=Ll*b%Qd#$cU7;B9sxp#>?3(BmV>J|;f zL25qT|AoIcLRTw$%EcQ0F6}Q=>Yz`#RrB0%*~mv1O6gWP?+gDUm-Y&QI+c3uFV+l6 zKg9^*NL2|E{ya-F>*Fv!dh>cHez3TFW^)t$1(j`*akyRI^K0*9-E~b@k@9joOg`I8 zz4nu5bU8Vmj^(lc!7z{OzgAYti-Zhx-B;_nkwnZ3bf8gXp0HUwa`Y(=#g_FMEO=vQ zx@toz&*ZsDjq=k^A5+O|Rn|(Dvhd)r>42j~U24zS;uj%S?C-f;b2gy=YJvq*4<~vb zwHb!_+a-;K8mOD76*pDCvQw~ue6@l&i|EJxjjuoPNOEO#LbEEe{__}y%j|Q$FMxgF z+pnW`dPs>ipqb28n^BW5ZRS!xA6wzs7LVpRHyx{r8Qva#VIkfL->WAz-afy(IofK|=cPd4Vep7F@{l|EXhhum9D{1Co7`FBaQU_@}pVMlGr zSjXGguVEabW7fShhkgT+aY1V&mHWdNuIVv zCIImdD(%HaO{!-#8>*DmBQYtHpBEb@NjmFhXNr~1jb)OvcqC?RcOlByk6RBxlzSGe zR$3xDsJ`p*?^`tNFEd8DPG8UUXdQT??6~gFYsIR#YtNI*d$)5iA9kOR7SK)n%D(XE zaA-7)iF_ddR@5rTedzy`q;$yxfNkE*pXy7@P7J-I_g5@UjAe5S>Uifti8}3_MJR=L zmN@zD%l>Sk@i`2-dKL$hle%_MTC(fny8Ra?X#KK1ii z7a3TIRkhrt_j5mFeAzfZ&QDk|Q15P*hO=6#i}j7?_KaUnuUTYnroHz?jvK7m3(bL& z{o2r|hM3>L{I3)4^eCo$N%}zf;q87sIY#P9 z=t7(cb|0!PdmxGVXBP&NvP)dsVioq>YUW5jOS;Oi3|!p9+eX%*w?I9Ec--ymwgUUw z7RaEeM&W=eMR^N7se@hb%D}-9eV2{^pT=vWs0rd{Y&G>xPfR9j9iZ~x<(~n+Vpq`Z zgb>eGH{`t;Qb&80Z_=U8Pn($ztDZ1mv31{#D0u+yyX)iIz8yb*Y^OfK40@KAi|s{| z0MQ*Gd2SO5M_?x#Y#&b*<2||b3B*-u^+rB(89F7|Fo%xiHlew(!v()^AqF5C;AHgw zro$Vf6-)(7`=hitj~khJXD{(_OeL)D%cr@yF$;eoUO^aIhdixqW9N}SXRr%b5UBj0 z2XdR-gz7;EvF@mgB{o*_C*D>`PV&IuagwA7_S-7In~f|Oih!ObWs<)DUnj{;Gkk3= zIXr#xat0kBwzKM6-R1D+c2x6y%WgiCn|wlSPVjpNx62yhplDnelWieqI5`b?y>AS; z=5T%OgQXOnn6 zmyuVCLy5Qc9&_*gsfvI}fJ39TdLvmgmrx3#X@K0+ozLTkItKl+`geAtuCNnAV$DfiY_FGuO~ z$~vBcPLIBU(cj8p`^DV`VvBRNxixWhp6uLU8TbXZVGCyca;?Uy$R<91ulPowFT>Cg ziD!g=E24yY00`JI_)S+k>#dc4&<@^v4I_j(d2I)q#00-W8(TA)$bO?3AxP24|u!8Nhh!J**_v2F|BMc=Tis}HyOB5K4mI| zdz7!&bi&~`YXKwZsMD+4#Xl#NXTyp}q)D1K$WmzSU2wW;7%6hAU@9WJeeW|wC&?+> zcih&BGqX+9q^az(*Yk^IfUyDjV(Ec*pMR~~6JU(iBMa7Ha-3oTR~;AF6ml!^y>o+T zIQ`H@0Nc7TVL8`xhh<`;KAYNIGPkS8SX{NUDifOLItFBtNmt45wLdh8`pH%AWPd5| zv$m2NjLbjs53 zLG|*7k(ra{PKy#Bc~h*3$!ac^Q>v6D1pb-!@ICq)+|u9sjj__|GXkfjLk*nbT>CqP zu1q!T%DqXcXQmsVmwPa2zfqh>{wg=7m-og5ORM?LK7_DiJ7UmkKK_F3jz0`4{b;n>&gNC_B)q5Zr`rO0 zwhADLI_uB=q{lSQ?Wi{5RVdJL<=UUj+7GI>$D&t;~zQLpDE6l91 z^0Wps_fVd7#QQ40ctYAy=dpYSQUjWHO{Bs8XbVh&m6Z>bc0OeI>}jW(Vzm4BVE=c0 zPIzw8HR&R)7w-BX7L>{*HW!bSe#+%TIJMP9^$Iwd+s%8B><+Y}qq= zU1n8k{<3ugS9|aFEnP$`Zw-y-bCrosK7S}&_a47hsX=t zJ9AShgbnV^!d|*GFB&$ba9X{H{5|Z=+=a>{YvZ&NhKEIV)2#0}64I5`geB}h^Q*`; z@ZI(v-E~&$r&d%jZT@p2G2-6#hO@_q}CI^Nr+XyPJWZko;szIF#NV z28{kTRv=^A*$1rR+Gd|_8rGv4#7D>rx`jwf<42ITDZYqdygLvF8nn`1+bD91Y7 zda+DS^QL`g!y8L+;-@}lBUjE{lD|r z6FH>!6x8~&{pnOnP6XpSW{3Kl^l7 z0?++^AV%iW#Rw?2`Bf7ok%-K$aAaBA0pe2s47)t;f2s03qC#x=rt|^Md;TtyW98&9xVBO?!gHubD z=5^xE#nns6nq={4Rr6*G-;gt{;@=Q`CIjPiQNhFEi|khiBA&i=>nU&HwS7D;y5(c6 zj1nD`SvwuGPoB#^ETAKqXE_rKuM=}|n(E87!IF2#L7ctXR~sb=dp1kQta0~%za6o^ z$@aJ3O>6b!_s+Ioz23T)({{73Pw_zp_Lp1ofcueo&;okM+75Zt3|Ck!(4q`78KE4O zSEWhiSi>5LiCPC-U7HQVcGXo*9B|`F7vzB5KbyPBdU_5GiGE%iQAXLEE9a?}_2R_= zR$=$@>K?~^sjcp+o5LY(yNQ#~$sp^Z)OCRnpJ}<<0*`>XxZgYK-D5@mf;3U)#?S8R z$`@$_&|dX&ZbbSVh-m$LP@K9#|YYnN3fh0}0))CM0_(VCwS0gWSKEJEmoD-VOLqYvPdJ0trJl{04 zh33Sf&#r45c(2*el2@4BlAr3&6lx8IV~-sS_?RpR$xi~{5Ns|E6F~+mF+S5 z(5loftz^4(;HkU1Mk$T6Q+7&_Ni^iCRe?77GjXUHwy2BRq_!|%L|oAP>DO03s;jR( zO#Ewl{$U+tXsYQ0cLn?f{;oS9K0Sp@+qlA$#LPe#maV7bQ2m{~9K(FdA4u(&sW_ke;Yb<@1*?{0tU+~v&N?a+_O z*y$~==;r#vvpKiY1|FkMOzu~!|K;>FuRr{`Jw}6)I^mg9MF9`M{2E`6Mek$viIolI zNF@G4KyZ_30~Rs=KMSC9ZZLaCD53yqQTE|s#hWrcZOQF=!k-<-*ymFIySHUyvPdWZ zGxb@cl(~k#MRIq%u^pJJx@8o6kj_oXBX0Gz=g5_9#r?1F3&kx$Otk2R$mg zgI}V~>QTgv+xOEo-y`<=M*jq|#S`MxQ7|lx+IOt+cG8VP7GRzo{f|w4@LKY2HruzA zm)jN$_R``%*05b>oIcssVSh4NI9_a~NkBJiQuc7L( zlaN*?yj}{o^@;9mtz7C|`L``-Z6t!Z9z?_8mll86PaJQ=^)AU5`O~jf{HlW*=T_6R za~E(=$W`A;G&$-_>8`mTvR3HoIxO)mrS6fmS-8*-G>nn?ws(-~=0Sz^9@c&xZD{m2 zDWqU0QTwox{$p6BywIV&RNg_xOL_3aKfDViUzKh1UVr-v@{T+&B}H^k$o_Vkas8`o zZxMWa@I$d^+!Bstb0O(l2qa_k$64e?wrg4UCdAHuPbgkeRaPz^Adj4A%K7S~HvUU+ ztxx$475kG7AvBY|yUxeuX+7rXQV|krF*~?*L zK4nivQ4#x6;8WMsedxKZH8ZSJrJp}-jbtzk>m=#cQiJwif9%bGrviPS$xqnKOAw@E z_MbV-VOhyGc+zp+eE%2IbA}((DGuoyNfrHUaCDMT%st2yA}D~HrP3ME%;NM+!n~nV z_CydBy{oB;d(t$akP%ROVvdpW(0t>E(eQ4nrPEwz;P-ZU3}OhSXVfPhO$D*3oy^UI zkpj}jP1+d>6<=++INbNN5b!5XRZVhK({XytLY7F1SM%>ijC`&|Pmq`C4h)4u(!*Ic zu7Zq=i^5K#-j(JSeSOw6O!85<^*m0a4{HI-T#)Ta*-+sY9iE}$+UmG;iz_Z~Yg({V z12ch`MBA(d+}3)@+x~oN-`s9UyOyy1l_zkv#YG1*F9`S}AqkYRye@`K_``ar*b za(|3RSFG;S^egcFQ;j`*m$gm8`@~h;Ca`kYr^yL99CsTGp`y8%*wv81O4_O=X?i}cOZ{2&tr z?0+LHj=FajY$VRz{X@d0S%S)IjWkTdLgfyv4iDNPb?bb-a+98INMp76N?`=C?diLF zvThTwYj?ND=L++rHorVVjgRZgC&#Q^?TZ1^gGEK_fCyCy{U~mA6&>e^(TQ(ifMb&F z8|>Yzf4eS#eOpZdKX8P=q2`u@KosiDkh-CKfJsWBcn>(E(b#LC^4xd1e z3>V&z$$sgQ2qVNaD7HKqKaEl>=EyAJbXyZ#L(FQq=hGqh_$41-XY7;@D}s^M({OAt z_{5WYV4v)d7ObP9bNFuXJ~&7*3%#zrkOr}s!>Vb*1ch43Ps0%zB-hef*;I2ul4nvYKFG%6 z;up4hhT@+&A7sr!Q>T|_C&ddBm`Y?&1wR8P9^l2m&HRSqjZ@g(-n-xzqY`^HV%$Z9 z`cw8m0C#hFI7yuNgrl5xP&IXNEp2>ED%6?hEA?O(CK#fV>|>zt$Po(LFDarSdIXP4 z%or>FxVXpZVJ1DCDc{tIgF&wh7SzDKkk2%QVpavYuJZMVQ>DW&V` z7Yo!)+WNh&a7cfq*HyGT_jp&%l<2<3=9fygu7++~>JHIWH^M<821eG?fyY_UTb}F3 zOC8yKU7_$tOcV>kIIpH%5ZWg`aOBxLnr!8=7qpkQ-Z8N(=T8El`j4?BZPSU;QYCXy zuC9*;Y1OEBu~H|ZM(B0k%$YyR$rs`9FFimW#gf&wAp3O&h_A6>ihUQzYmXtkT+nTh z#eIP3*70Z7^((+81G69-d_1-;&g$EVin|N=ROk8|2l(>i`o4}Vv*&RTD8Z)SVpeaL z9-_bx-*vK<>Lbad!=F}i-Y=#l*;e#d*hGyd&^f>7dA;C(f{K=Q0n4Z}x+i7X48C}7 z(eZb`@Lcd}UDykU8Xv#UzO8<~9_#E<-8s**^0dP2her>jC@-lSY3ER?aZskdo1O}WbDEwWhKZA|dZOB0~G*m9# zgq;jrf?T){l1PSPdQm)DaG)(fYTV#LN70zfbP{D13M@sO506m#>yd!wh@OlM?Qei6 z?6n26qaK1v5fMQz3pr8+$4(fsisUa4To@|o2D`O0o^X+<^%jl!f?_$ z|E!GDdOrGigeGRPbaS+AzeoIddSG>tS;TY=j^vvr14b|a{5IO}Dbp0W!^hN6gVYM_WAPekuv=s8K<(>U*5ai;5&0qb)S!e+@-|RKo5@U;!x@tp7 zEbeZd1I9c<%Nt+)Im6GRyaoL4=c7lNaAxyi zC%MRo@6I-k8^AJQ$+KA`J4}|6{k6~9=zbYI@$Pe0(&$?Ul@<~ya~zj6#CiVPzq(H6 zmv8wmQkEr4Z-4cz4kS^fSd0$#r-IYYoXVl8@RtBvL-E89a4kRwVIqHh(JF163e}&%FZ{73Au6Mb^d&7{uABxwWQLWE_*} zXJvPihkU}pOA4j}RC_)EV?Pr9a~i6Um=+fB5#YO9v!Tt-eumHZOptoyC)NM}gnr22 zjED+*wywoa3;sKt6ODk5YQ}klH+@!MNJy2a3Nk$@j12dG_cNZeSWsZD8|^r{Ge9q z$FQV+Mt+4!eNOQU9Lf3PSF)ygVOo8%DMT*PJl3!&ReR03Ncvdp(v1kMq3!V{@{9A^ zz!y)D#5#s;i2E8AmM-5N^Xh>YpA%I$GAk=Q*~_;ib8^jT$`X$?khT@R-C^Cw0m+LK z4sW6_Z*U{Si#boi9d|gw5y>-T4o*Dx0kBn_3&r9S+v0N|5`WoSCwyEQeCJ7TTv#vM zTJQl3ZQPXjN00ln;Oe)0!mr!LpL4!l%FD@wNew~pSd~fcsPb@k#R#nPwbDoiA+b;D$efWjo2a3m`N8PZA)NDDk}pSFqg(lf zhYyXoBrUw4kmDm_70;v$qt(Er?_~bLu8Mtrh{i(wXp*Q&&%YHe;K(U=cC^&<%Vv*H z-Am%BYt(3kv{763?ote=#;D5q{1P8etPHyGSzpQ(QZ9HMj7kM-}Z4L~OO#SYwv zPihr>AO9}?6`!DYF+W8^aRX+)c*)K%$i+}U;522wE zaRXZQxW*XI{X@YMs=!BUaVPYMl zTl}F`J&4eFm5vjF!Y}n|Z8UsS zu;HU}hJ=EpxY)K%{2Tc76xn8cPTR0E@SzTW{(+zP6#o_<{Nj{j_wKWZ+%?x;gRsMr zKx{1UJ%8`MoAzy)--^czJ%0wAC4oq&jP(zD{N+=?A!CZfw*Ytr88`cOO62al?{X@~ zxvI3x(bpL37#ToNp4D~C?MAEkE&?A~N8uIm8frVon4a#gKL?#Vi^u#&TM+X(x zwfB7Rx1+iI$s2x8naR?uh5?#We7B{PBa%gxYJ>GWewc}fFe-izO~S0;#_~=3_IVD? zqCoyA`#9>_hS+HQlWlVf|C~!124#?RE8A0iw)w;t@X)g?sQ8>BAX)fCUu%tj5s;X^ zmSrTfY==Mi5B%b(W_-dcC<_8bH2CbzZWWjA($eldXI*^JMZ3>BbCTZ}L;x+3 zEM`2FDMQ`q(fXlZ_Z{_wmzVYr+AeXBvojOIuIKvZm--O}u;8m_(i(o%Kk}7dfvt#? zg`v`lV3Kh;MtsSLGUE@v`W?Uc7k-+LiT{Cv2k$&^Fvr)1U;Mf6tHnwDMNR?q4-jdb zrC7nkK+4F5rst2*UV8aumzQF#l?V!+PxSYBdRglD{?8A+ARza zkc_7BQLM|C3hNR8A7Nv-&GBmhPVsC0p{t323y7s}h#qKTUw<+dORGbff`c;%} z#K^`4AOGu|))2NjQpy-?O#_b93O5}5(l)ecq>D3Y;}-<3S9#6>5=Z%T1D}b64N7t| zqR6c0TCdBOqV@$crSy1im10fO{n&`tz|M5$e-0l7*o20JO@|+|LDEzF7W=LEYcaHl zY+Yk-i9h8(@GCyLS-D-rmp_UuMzF_+?LU3*>vhvuTL?$jI7@O+%HQZGgIO7oM7aBq z3Ch;3^Ggp4Flg1fb~hitjDgei0`uJ;`x^YQuw8!1L*3!ZX)z(KdLX}?L=x() z^Q+6o#~4^N^Y}jRK|;M<0f<+BN_yH2qWP^5CDHLZB}(ZW-DvR-B*KgJNv_xUxKYH8 z@$_33`hZ5bZ-Wo%DL!CZ z@{ieAur^|H9zO%N*<9eyRJ`Y1yUk>e55$VUe#X=5uD$NsYp+e@^Z>~l_ua(11`GKe z^`?w5o0SKa`hqZ9KA`1AYwrQ|R3> zbOnytcneB`-Q73T3Xf&j?mae?Z*mi_QEigJ&vuL&g}8)d{*_sD`I3Hc^b|oJ`G5Cb zySkZrys>B?hJ3AQsg1~r-T=EhC%pMAh9Ae;r~l%Pj_)X_`lP5VWKmo#EQz-G3&-D` zvh>tawbV~SU?d&Dk_&eyB|Ttq#ry57Q5%E9~fA#zW;$j3o-~n z1)dfP2G?k-)CZqUOvm!HKARUZP>Fsg!$|6@Xg$B`x}XvQ2pj*ZT8*ZQuUD{ww1j&rqgQ1ZwbqlM3PUhMwAaetUT}T{JFLW|KN+vu|4?4WS)P; zUuK#FFZ_A`mQU|*&wq-K3hyTDIg=-1Hm<$yx`4txNnI9RMxSF+<5cTE?q z#U7Fh60r)*#;r_NL{^l>P_)Qb)74jBX;=JB`|`!SGxgz%7V4+s4|K|{>{(4DQ7z+Q zk!d_nF%+6u)vW}xr9ZRdV@fd&ww3y6 zEPM@hW}sFVVAJ^xO~WIdvNgf){C(v#+E_)X^Dn@12N|vbMd6x{CosHH6`REDpd5)h zHz7Q07m2CM+XStu?By>dV!8VnmAv~+i=P4-&u+vWdS1y?uU(Kli}!MJwJUvd~jM+|bXn z5@DsQBfqGG6O{myMSSG_vJGgeWPS=CRz!D=Zwu-5d6Z{5piAg-Ujw!+{yt=Z-*^tK zS$(eaT~E0+MtL#p zwVLt8uJJL|?WLk&o@Vny^$M*go5!!-*#&XTz{=Fq@*L;S;Lql_l9!ZrWgvziL8Vo+ zMlzjRVsvN}P+4ils!5HB=`?DnOvR84MV&A(`)OoV7Pa4(X2xC4xz0Lk_eB?7T#y9A zd63%7@%O@uFFbhQ0B;@?J~CHadBwT=_F)R$Q9rTP@=K4F;FNF?+cCdo(wex_>k~RZ z!sD;8E()1ZD=;2BWQBEJ5Fw=PaheLIAu0K|pR6~SxD(}qaGiyY9)D$63d=3b5Rlov zofG`9ve(chSP+myKm=eB-Y-#VEtsgM>dG#sDlj-F;) zU)L#1>?Pj{D8C2_lcH*GPX?vpfs3F=yEzm6X43Y#`5|@;Nbr&!QJT0`Vx1|KEqi#S zNh2{lsZThjH!5u<4vmP8`UD)8y!l~A_ls>>kj;VDEJjV^+A~ND7blCMb==SG;rPLn z|IA+;tYHx0D2z!YP1Va#jn6jTd=kN)?3ftySKV3 zK3tt%c+rK)hsDO9zQ`*+3Coo0pcwHf^19Q5Tdt`z+p0My30JPLx5Xd%1wdj@Nb$*H zikf++;$tO==MCv@!BZ+`fx$ngubxdW@K-hXk(78_j)A{U%W5t@Fu1n8pAe*U-h9(d z=kR_)uTbJv^N*7N`URwaEo>#3cHwo<9anq}2+PRkP^nx9(H8@q?dPaZ&b0WD3ctd` zCshg`oldhXH@i3P+qb&9+Og$nclK|+`Ihr9xS)2@zQoDLMIX9w8h`2%jfy`T&B1s! zjX&*$3bMzQ;Ct7=EVca7Y_c451}A^24g9K4pezsDXZ1Pw%b@c&yp#Ib5`T03Y>7WZ z-M&7z#2@@e{DHdu`s?e)8BSJxa^cmU2N-@f78 zt6;^9>y(Of@QjiN2rufn6F~rtTm_w9Jiwr;F0=U0=hu5`o9k!9f8bYsbL+tH@?8Rc zV5H;YP0w$Of92P2Z<+BozfrMT27e31yHv0S)4WKRI!?p1{;40opJhJ33-!NHpGJ04 z|06!1^vMpRUa?-|M7P;`xrV(Rdbnq&1_W!2NpD{Mh52eg-kU%k#6-)Tr63S^=6tS^sCAExSc5^`A z8vjEVUU>1v7mG|JS^Dw!;K73*{J;kWh5Gt4!;|{dO5`hqXlbpltJkiGjSY5LY^|>N zF`a)bm5x8OhQ9;9Q4LzagyfvCB;Co9I1IPCgNct5Y3g$17aAeel@sl%gjo|?Ru6Y7 z6D#*(B+YbInWZG%BZW*LNKE73;wDtTeq`G{i8r{kWP1z#ovHZS@(+n#$s;5o${eSi zvNeZ4$AKQ5m03w|8z12mL46jnNZ)wlO*QLicxjmn!*18tJjmu)Lz5MYHhU>HGB`3> zY+PPG@n`uu9u(IP{pFppjD+}ge4PHaU7YA56C4%h4S7W|ZDQLyb^>cN2ATl-baEcS zZx9bHb^Z1p4;kt}C-!KsCGXzrVVZ?P#>yD@$6ox>XNCw~W5W3CU42>xA)=#S+)v}1` zT{r_Qd|&A7GzK5yLlGJ>xtky%A|{i~_;l$KeF->1)xbfWX6m!6H>)@KV3AO)^mmyH z)Qd_*U2@P*pXl=F6`VYbF^do2sDKCx5;{gU6S16K5dlvui&xx6OshiV>^ZCghvI2TI7G>MHd}7;D=hIlkt1=Ew^&BqcIH}l&-qsO4TRzPY*K0As^%-T_CCBL+M^R2gpGV*Qwp0#_=ou9o^szy}8l!9oIZjHqRe9mKbB)>9mmkM?|JJ6kB zU>8d~7cd(GYT#SP1YHX(@f0tFiV|ua9~OD1pQptgX0=QOqtU1mOnn0}Ed%@Ty^1RrihtOa6ui3d0c36xEiU^%6j)IUS=_S;JSb!SSJ3Nhk5 zw&k%##(sYWQNqPk56AnReBC7Etj~4k4J(~%Q7CF zSkRV~M?bz)TO}Gvgru6PVNW2)urAl=@JyjC5r>2lA<*($-A%o@Ee+ZW-#SvZQ7U|P z2{6as){0O-`7Qt+hM+aK3-Czy-6gGH2+Yr@jQp+{9|DQzb<4wwaribOoAQgkhY+5V zhzt=l)u|s2V0m5xoAdeg4ICm)9(lEXjG2cG>mzfYaT@vM%ep4~&99vJP>PYln)rOt zN8D=l&uLa3`dF+_)an#t%lgO1X?+rl!5@523i-$n4kkX-Y)b(V=Rehy`1eQ|StN~0 z&7X31p}m$L;QYDfH`J@-K$}*EyP0#ai%k9ASuYPww<$MOImkyqm?{5aNEaPdpOnqu z4-^7*epNr)t22i>9!#?7{vSx8JR-6g75mU7HvAlSG z4rYORWOCkWR;qqBpFdka`W0|INfuqZ=-3Z_{B00$a9C+VkIG+JVZ%`L_{f50iOgL; z(%7Qr2?m`?sg!@LnDV`XtRqiF4J5%t$4-o}kyqw1UK9TiWCcFww}y*tG*u}acN+in zxT>Z2Yp2jP6l?6x^CwfmC&A*wz3?lS&bfa2zN3UJ91Kh($lkS z_VGJ>3GcR7=U3zWQ0>^l91HUgYg%+8OPxP^q{mBt7S4b4W*Xotl=%6sCaKq@TQ6qL zV{6;vCG$_gBD|CRrV}UVD5OL<6x`S8*3V`ZwV0+e)7NdV*zy7l$IuzbNNNxLknT?9 z)m0l0$z06amPjh!?50g7sX%4JC!nEh3DFIhFNs|-cHad)Xm{w~S4RY*WtiBABZ;C$ zYT~igC4_yYVc=)O2uWA!q$oJSve{@w$!(c5I5(2d)*$hqE5<-9OKLi=H6Ig#hTnn4 zZZ>B46eUpN1gwa&RxB-XR)LC-^4sDkMHM3<60TsY9=5SnmdcSbZWg0#{s0hze%GN9 zS>qNO=oe}!OFsA=E4vAHS|YwZegmnzFdIHcea6sj`}Y@^@mbz@-}~OjEpVeV*vK-X z!OZV<j&NNA5rz7~w4M_% z_7hcBYC1W&_T&li&L7@`|G=*?V4w;?`kDxZIsP$rGXqV#^pRpeG zX{KDtx9;M6{iIu`O7`-Pd;~du^~0wSJ_9b&~Y8nYp%Jb zk^@$a-w%B70}JCz7?uQ=SJYbsRUl#mGGgx1?+}d`!panYx!E*+n_6+jqlO=#Uew*V z_8r!&N0aCcpJvSlV&HHPqSJ_(d=t_X7Kz)zhgOiM*GpFx&R4jKT-wN^0z&)fzqiil z5ucPz@oDoD1e}S1Zs68M@Pn{epF)NQkc`%8e8?jo z&mmmg(Wr+Vh$=@k;!hOOn&y|==7fj(>X44e! z(J79L^*N3I=KL=3r^xC9o#1neM7L)3&yKZoWO<#c_Hh-eCG}JMp~Am~`d_He;_t{x zeZrVWe#r2s{}yQEfHQ?`rEqW~=g+uY`K=PzG=5VF#>M|sMJtKYai>Z}^gwNcuOH#! zhN4wtwN|mJGKG=eNHued_0zOcpM#M2y*?N4b-14&PUq+NgX|2SUvuE#3O^kStMJ7L zd(w7`rbgEddP*`t@eL-1*XAf`ppL&bPyhyF22oOxu6AM7HAp|g^JfubSMYKqbZu`2 zX~t+Gsh2EfO~>^*odr?+=^CRY*cRUW@mCB!^~pz8Zn)tF}ee z7-E5*L{Q+5MeACAuKH9?0(tc%x{LKm?-XC*QVG7crB}_B>+Ce59eru(*M9A6Wt}D* zaW%S}-BC_{1Hmd_fvp{9{)(rTQw%8yM3{;4&VAKL>{?#RRQ<%OIegB4kgG5L1VQAu z?L@s5zLSs+>!wS`Fa8Bh>4r4DN+=fo0JBs43*YnkidWzWVZH-vj{kP?5z_pIkcN|& zjvs>@aT8lQk=6S0ZJ)WVICTeFeu3&9=@UHV74Ge2l3y*Q#>RTg52c~k1lIXD)@qEG z;o?b0>XSC{%a_z=?GEI`%;V!u9jNIeLjZoc?I%~-1=2}Y<|r=Em4CEGk#}|!GsUcJ z#SEh3o?pWe--_|brg)WvdPuW4NQ#L;0>Cci66KI^zs9zP%#jNc`VfCn1sHf92O0v1NXZ>m&g`4< z%iIulBW|*et*i_oXOKNda!YUgXeO``X6C{z*W9Y(rDB$*=#{65VFWR06Mxo~AOku2r-0vwW%97^u_wK#z)?3*_w3`0-kNu2j#iv^B=#KMh z{H35m#!j~45n{2ge3Mj0PdYC)!xq}sXSEB(GPG1KK%+45C8$LkCX27fmHgpulX=zv+< zQa^bJE9=uc$${E9hvn>Z_%2jp$!AQgGkQ+)ptLxpJ9xMP;pw#VPW zKjN&K*LD0gL|Fu3-uJ75{CGlK+u@CEKXt-g4Nli|I$s-!SUbl@31LV<0wUfikS3VS zS47;q5^J77qXH&``2h;=GI)s5koere06Nc;nQR^#>G-(FDPi!9El+MCkCJt*p>0Np^ra z_Tz7~fa!Kxe4OcQH2!>k#78&yjEWO5N#UD8>GZud?tSED5%IAsNPWe&#ozqq(F(PD zep5d@NL-GZNgoL!znE_IldA)t<1krB=CY02^ahi+4+y+ZH>cq*$;m?dPCV{{=#;?)0 zQ3cYIVPk2eWdUC`W5S|J^mu2PIW@T^BVJ zeHKVA@`}@Dz+JLI9VG(EZImd%y0BEX;cLHp6^J=}AS?)>4XKzSH~hc|l$fw*X+@58 z!yo(NW{I<|Fl!CX;}?Ht)C#aR7x61TUOULduR955SR@di{T#nIdmWF~GDq>uLHx7V zkT(8OQH!=>|H99oh3&!9`HHr$z-q1VTU3qV3p@To5OP?+-#_`2Pslf;lh-;Z6@Dm? ze&?-#zu=ulHCG=0N=Odtc-GBF|#4F?flT@p9}(xq#iMo?>TDxE2Ig$kuEwd z%;90YwbJq{4sTI3HFrp<6^TT>O;@7oqDijA#;6gi(a;2P?!c)*0#H&}M@=(XXbN%h z17xZ^6*ETd3RVcMDcFrI{o2{*I9b`DE-zjw({1V4$P#+CxirCSXFpYiu2N=En|dG< zO?f^zpwQOMdHg~x{P-Ro%>uXp;0{W`C6F8|k<$ri+U+x&S+?X)k9NaG;JBw02LYjFn8(Yy8qc2>65wpHktN zdeMjY4o*HM>IaU%;!jJQcW$ijK8xR5H5A2xqcY$LPDF#>T2vAE+;h(5%Q#TtVUED4 zJBYC%bhJB;=R<6#(XO=5>JwR1>QLcakg*^qkov#ui6Hr2pKfHf0G|gvAXkrj_^wE= z|0}Mz!aE?6Dpo-ZN4>$otN?RRF+R-&L{($iD*hIlWFjDVZFHqxpQ*XnUw+QS#TplN9LAP+?A&$yt`mRy zRj0iEExS%RwN5oT28jynGf#i#_CGuN_+w0FhO7Q9uk1ShxZ_WM?TN2=<%?eWs+}jC zQ2542qTYKD{OHnsci;NNx4s5zo!+r>!Yg0%({K6JUB{o`vD43s#-aQg1mQ(pI` zI@}F>3Wd=4Auc}q5-Ol@KZv^>BR=`s2qWn~GH1?+F|29JV#Ex4?Xj+u7?f6@ZYZGA z=JOwvn`)^9D%o^vmfwMYEcH{22ePb0;_F&HOCh*k1^Im|p26$#M04C{ z^m-C+>%&gQT`A%t!CpT_c9LHbAwD-VT=TuQB%LFIryEs2q2QEA!ocTiR&r@?78mfR z@dws>58Lr))>*)x)Q|bi*#(qfkx@8$?tff2FOjv|z{G|dWz zDJ!5_akMDi7g{o9(Uy%tPkhCdeI9&WC>V6+@q;1&OX2gzfwBn|jW^%2pX27}WdksN zALJ6=+wjNTjSma+r}d9o2t_{I11*a!>SBy~CDsP_tKp}DO| zDY?vDA2lU{X?H^|>ENe1BR%RBpDBI`w4g+00l$Qw7aw98O#FG_TifgX7}{xL8t24p z&slrU{G~IAAOa3m7h)1ciz|Co7b%7o_Cl?gSDaa=vA{KLORLYpUqA!%BRc*zf}*ml zuk#tl++I%6>7~9fXXH-1{28D$hFKEmkgelyhNx9wf4t@nrq6j9j1iyoa%#^jZlIHE zPVWv$1$>$e13L+qhS>{*G823sD=Ykx?fjj$I)6_L!r!rDQaq|8v@Io#KSDJ{#jRt(Yt> zvDxCNX(5@uQq*AH6JsNaa5hp_MQ9p_G{g(q6!b4_Ih9HHK;0f+iSjBYP_;5Ue(eS_ zr8tk@4%op*4bn(1@Soxjx>mKfP8$2|@QZ(r$R1=C;!~+tH<2A#Uf{2H(pHN2oO%Vql@FQqi+rQ+@G@ z&f1b2Xp#&Q`%5knh-;$MfuU*ZuK(+mt>#4wMWl*l5$#YV*YF#bFSIw#*(=#yg>LHVwe3(X_ZCKwrCj;^Zx9ttC_lkq2% z>Ws%vsho4p>UrmJx}#sMpd94Nw?wgNpN@`c0BO&mTm97hBXBHM*d>gcv-%Og{BChg z{dCZVpE_k*?SKVgV;!Ek3uD^_7hJ&Yo5;9=9{Ke;GkX;#3AR|LneFN}R*nXTR^!oV z(=XeGQ(CSIpU&1~5~Gs1RAaTFUh)+!R4`IrS62zo^!gV36|7NEb~MpAqcz2McaFbN zEqvEF6(6Il)9e<0tmkfsGKo)lg~g4HYp%Wa_S^6H=nWrrI?t(w)OB=zd`V_`8vn){ zQN`h+PT2~)>a{WEhlUw)7Td_3$}Z(jZ3E`nG&*4-?}_KC+U#Il#a4>)kvQ^%JVzwu|^^dJAj<4$`yXKq{&^2w10?!NuLTRy=%A0p98 z6NZUN6?LNhig%p-x_7^S=ZPInvx zK?cebJO93R%g4TZ+rKG%x;(jCUq9~T(_Z$rv*H@SZDL^Kp}TH<=(D$SD$H2-GZY(3 zPd@amU;OQlcow(`zv6SGzG>Ar(5U3O`1l?J4Kx};_&aE^Z-O#@LzOU0&VM36dnE_i zB-Nm3&Ing>6FxFAe|)4rStgh&N=JFjcDhvI11P*q%2gocGWfIm1a+8OkF;4 z&bfU3j1HZAQ46M4jy&_scMp8>@MDh?Su$c&GF9@ckh@NL$tgeccTRfEo5(pG#^3uV z=I!CHeBpb4@j1@E>sSC6uL*Xp9DnkuC%ybNC%o*{C%ojP%K5~Ej8TiLD}3JR&_nk< z@YzqVA9@N)uvF6dgAZNtHmS#)C7B+q%S}7$o==-^_injJsB1o%R7%d z{T;vZ;#d8QWQ2%p>gUMP`s06b*Ms*SSU-B0VDH$qvg;);f5Un2Tjr?;rCa^*LU~&; z_@qs4*QCaMiiu=GZ@if{Kvm|5-=|l(kHht zLLv@Ycd=CXjv}-*sJl!t&R~cF@8cY9*ln*;#nQ%r|2FtCgBD+U6_-4P(qx9~X8bw+7$6vifpzdD<%lo-x{*n4 zf&XTFBheQVkv)@UQp7asxZTA@{lv$#D#3hOeUuWLp=$B3Q1IR25th z%)C+@Fz><-*4C9aQZN7+*EkrRbMC6f7{6vawJ{_TE_CC8@@|Xgg@K!HH{-;y#;-I@ zx?PH(Hji)Sia~>H_zg~E=xZug^gBCxRiGY5)m%DFq%}nY6@xX!A5eu{fUQy5T3N*RGT?~gG`;aK z4j`NQN5w7E(&|4A${3F#5o|1^E0iE+nmr}LpEcXi7l-$!5aXft;CtDdH}zMJJoSXB z7m{xZ34Yo_vhu4)0PF}% zQttpxeyg$y-ugFV4r4omHSmjn+E}%bg)sQ}B2q5{cwR0 z&dr~I>k)(#8A9Se9LP1s%5g7x&0aFQNr3DL^FMvd2Y%<2*Z&+} zBv@f z1vI0|Mj8x+kxUDiG_#IOqNI@d9p}$!5C@@B(})3yrJi5grb}hZMz@5@7 z;Id0E#g8lbV)bMZulV{omOZ=Aq-;GQ$@z(D{8L6xD~wb8ie}54mW?KjMq2q6Pj*Ip zs?$uErx-|6h-?x6$cS6Z*7(9nkzs>;R&&>1sn*yOMF~>u`Df}b`|9WTl?b|moa0Pn zLXen9x*hb5ieW;;u4bIAFrZm{gBKq~FZ_&1{8T_1GOz0|#xKgzDUNV5^pNOWWKp_qKp_&Yvy1JLH2#6I zBlqj0DQ6(dp05NyhCN~7l-)@dk-}CKSvg6(io#2ezD4}p z5iIc;K2HbnK}|#hihg&!%^o(Bc=5&aIcB^`gGz$wxf{~2 zE#c*##aBx$BhWD5xHajgp8Oad11%qlHta1vxjT*+pbPO4*a8qAsInls_NtJKM6)lg zghYC%WJlqs7e%+}QP555%Ib<@sRYCo^^;F(HOYvmL%gUTvFsNwU)-)XbAd#4D`6%cOyH1HM^i|ay&dZMR9{!C&aG;@?<)h9394CHheasr6! z9ObkqWZ~1!juNh=S7GXCi}(Z+;G9l1{ECnIMo6p0zj%Q-@CnSokGb&BLSq<}Mf~Ew z8Nb5H$x33QN5Da!!|JuyT*K#CiU2-&pF7=gS&}+g8Yzy|)zykm*Ooe&VtTjER`96|ZY z1ajbacEa6DySTt0_3;k~Fn{FRUY!CQXo zzeSQu(*#lfSaNFQU9FsC@TH?hLsX7CPe1+5fAh~d*1UGlyYs9qq__$O&QKxb!y^eL zhxrFS|LK!o^Txu*oi#p2!6A2f>5+R6t{;Ae7VfQ}0}l9Jwemi^52tW(&hsY38|3vv z4?pnacfMxc#rHWBE_bSM@FLjLKeR%r$|E8lk`(Hs}tCEZZMQNJnHU)iflU_5YxiQeRw5RhE5>B$iHu!tb< z%Pbv!=9~ZapYy?@H@){CNl&{gYfo{fgRtYg3P@5Cd?6@pwy!Te{pdqq{6GHbX>WSl z8~)xuIR2$C$0(8&|0nPNCiqw(SsFHbOW)o9Z%=*w&x21i9j8_@=u*A^^ZO4!{r%)u z&)HxQ9QyvF3~Klj7T(mQP(5(^Fws3no_UgQiUC!8Pk@#lzUTIr{oI+9Z=L>C{b0re zxXznSjNu3Z#DBymW$7oF{lNf2X8jmXQ&g|)=mfsmTAR;3e#PISemwud8DOAeNeOH~ zZOE$CMRrS4izVk>Ep&-bm}G^LaBWdr@y45uqD%J-KaK7-;0b;|a!6gbrc2?o{z)Fc zgMZbJl|Az1wr(;16USR_*?;S;H|t z(MrS`e&pxkk6g_^3-K3F(tw#DcPIE*(Li64(-!~OL^$=C{JJR!giSG;%II%Ub;hwu z8zv-L5aHb{q(&US$fQgVkpXj{6NT4D>^U-W{pD2C>PP;JdX4%GLt@me;>Z_V~ji;nh1fys9V+!={$uc0T}+q`GP!mZlVPxC%?7*>*P}1cW^=B+a+e0-QoG0!+_dto#>a6Mv>Tx z&|BjPsn!;9I=qh-R?leVe{Kfg!D_}X0TF~c8xam`zXofxQLja4VJ{o$fs*g-UVpoJ zSeT6i@J>F>iD`rIbq!p z!i2vxBV#RD#PoM<7LTKrz`muvr5=+Fy1SI9?n=^Pk^=8q+yb$`j$mi<4{_-k`Isw{ z{Lu9&N}Vf_r1vQQPFb{SbQ-90^SYhI`WDMJAsdLJybisyfT05zqsiuOcO-9)7=ivb z6WD11AYaAhO6J>-9f4QohE)I3aqD51a2)Jtl#2)>p!Kg?LPry9h)0f{bElYhb}*XY z4pa)4Vd%1NX~L}1o}B(@ErLZ;{VLm<3L)!Q!(4m6B(yK+u0`!?=$!lBuQU%)M0*P! z-tC#W-6!Dv;pj?x5ot))^!x~ou*CU1l2TOVE`1XbL>Th(cg3H+X?HQOx5?aaprQDN z8cRDL552;RKIKxeTZY1i*Prqhy`sNmE*%JND}N4oQS_BtN%U+4UuLj#m%X7Kd$^Y8 z>e~0nXXSDgfgcchQ|kFQbeSIhi&?Y)tP?}l6PTk&@V+;X6g(&k-OOiHRX&}nT{-Uq z+>xDNCg(H2oXPZngwvLr{}7w9F>x?^F1H^H#YOZn~Xq);Fx0 zMU9K%m#2(jc|g8RT5n_FAl3~N4dxd)1J8e*9x+FP{?aKZYi8T`i$iU)@1)(SI2Hbp zxD-7Zg+9w@AW_{({+=I}WO%x3vwVu_pyzjCMhv^6JnFc*I2|vCbXT7l5gqyS4t}t( z*35DzU*SIQQRW*+6Yy>EA0RAv=j2hvtJ++53phnyvt@vS|Dh!)2U&zM{$IT4_-V@T zAn|YOTJLqNId6;XWOMu@U!PtHvphTbfN7>A9asTAapiVt2!4W9gyguKRXAhkkP_5G z?Ac}oNMxh^KAVB#SUa}IDti-AleEeW&8Ok3s@Y?Ij*WO(teCdM{`bJq5@G*>HM|%7 z(v?l<^nh371|5OrR7m}r&WlhdQr@_ne+BTIVR9S2=_hr%pp2+PM zeez=SMY(k3&lzPg50|JDik#@noqTi^xt#krF{f4adJmrdSUYu8`yDNX{tkd;OjI832=-ll11Lnu#t$>|P6R!oZ^?Wn%%qf9*4FSG@TZ49 z(hSV=O!aLb?`VN?!%N27V2d~9h#c>K#m%fd6x#21wc=s~Eqd4@>Yx;^UjPYTb=E;C zS$o?8Y5s~FeTN?_ZtQGhCu-_)D+^M(gbC$CI{f%d2sGH*PXN8-V0DJ>%-is0bDFH0;q;V|y|Z~{Dc z{-@gO0NX~??!_mde&Cypj8^go9Sz3;b!l5aTx-xu|J3VZWuu;Z<*7eI?ZY(r0w%tI z%)iAG;ypCL+QqFj<(dIWKiJNTi0=+FkcSLk!;*AH z1Vz&`+WLF{a4FIUJ5_@qXf;er%Gu2J0K5TUchNNRhMZ0^T(HN2n^eEKj=B$_ju39|dxrodWFI_8~TznlJ)IYv9 zF!Qr7F8u1w1x~9}gZ2wPA$o2pYlR5v@%98yc(R0UPqGCZT?d#Z`pnqv?L~KmVB&Nm zq_B1_&9RyN`a92XR^|C!=l2dl=?(sS`H>1_(Ww41gc65u8u3+zfNc-^}AM@GEBV z&TXBD7@Js0>}&=g_|PW2&ewKhsaF?xQ~DEha}M9!EC0ZE5Df^~y+zzmEPT^x;9`QG zml!0;`oN!#UojywPqF}fjE((V9>@3a6eT*nfwuE1K^Jm<0O5a(I63W%ZX@!Yt{PvB z&9L-!&bQlQ4#8Id`+G$mmP&cEStsgE`1NsVv(E@RtV+l?edwz;qA_VH5u?Uqrj>!N zY)CK_ynT@Bn`qECABbK*S!!0ZO%EW~bCeWVr3AeD&?ll<&3A+zT%b#1K1hPzX@amQ z08{Yhn{b_XzfX!mm=$No4(c}9q77}H{YMeCe{cW6q;4fp4U6S~o2wZz#PuRBbmtm- zL!>;ka`yV$AmDDh99z49yMyZ0M-%xyrw=beK@(GICO}94?y5oOQybQ8S)6vDTTf z)8B%9UO}SkO1mGw+*~6QWFJ22T|x;b7T|f{nakyeOa8Oy`>sV>adA$`MfIuS-I`Pq z+`PG~FWTwuO6orr0n=wkIw3>i|A|TpkICLC4=O%UMDxkPpFJUby8h|)bU>Smpf)9D z^hd%Efi1mafd_XOkqJnTxsXEPn%8r?rD8{#jm9QJE{cyxvXG`%rk2E{L_cUg$VF^J zw6VY2cE4$HCCc0@S5M zr-o_mu34^}^B-?T8Jcq|Uuz6JpzQ^czl-`i3w+4;)%T_C?CQILPTzGkKxSE=ZX2K8 zPJ%V=_Y*+b-HpJSvfAx`9HK8vl|QaS*FIsSw%m8M41lXqm?`{O;0>}uOo-=SY+Y^~!- zP`P?f!V0p|R^7SK_6DCKu%xBivp&|4i7B=Muj!utAna~dOO>XG<0inLXz^3?-22oV za89MyCv=t+OQj!s(X<3}QLOASM@i-HR%DX^O8HMCDZZa}Ru?oF%orr_)qMTxtv5jpTVIw_ zM$O{P%~t2g9*`hvxN+owgUx70c+N&62X4UaHC6;m{Y1y0&7v=$oMaGlpx0!a+M?exG#PO9a_((PA0$pHiL zOgVQWf9&1> zOB(tukI~)<2+dH!YnqAFQcq>{cH-Zk6L!&NgLK*hp6JD%#3pIYo!pSbcQI@a2UhN) z1{{hgcekbP#LtO^zVo8HPc&5RX|=uSa4&3N_$W5YlO^&x@^pa%IhENn6`T2c3NO90qj9}1`1srDrI@R*F-Ru5|#n=OJ ziI#!l*d`~*Ea^+f0Kg9%qnZ$&-z!E}=7$xJ@&iwEbmhPR_YNstRor+8^uTw)@(CY> zsBGW@GZ*|UJ};_=$oJ?HtCaT9dY=Q(fyWuiK4QP_9_PaqN^?Jbx7Ixi**}0B zj{#7g^0z=7>BIbsYCa(HcPmM%FL*pI)Q{HcSmcyQhO6@?feU^UZ*u?D1>Yv-z|rNn zo?73OnX>@WPLtdqi4VeF>Xe*}ViN)vhWOCwfW2OYdkO0hx zaNgH(_V=MT+tbjzw#W6YFWQ=XxgJNRcNV&_#R-mYKPmAKqDQLde3;gUi3uy? z36&&Lzis|PSi=1H7?De7n8u#W=U4Z#nk%Q**jYsdKK68H%n~yy%XgEe!c5mdbE51Z z!xF&}L|m{ETF~Qa>Qi1|1sbgZG;dRoCSIZs1Ua9V)jXdMaxJcb3(FYUQ`AR+4Aa6G z6EK(dG8kUs^I7?NHL`?Q6vjyd> zmQV4}Yre#9+cBSap^_u9iR}wGdGY9PaZ<(=&yRPqD;+yMEMdc$RAC92bkdwOslHNH2qKVP$F=Cv>sT|y3_ z#5V#M0HF(H=>RaMrl!UR9&sSSMB!w+0kifU1N=RWepiFcStqnv@tL~SQ|Vda zT)&SK!SZ3F$Pn~aMjjOVSPV!%JZgQf(%4uSZ7nC~71420-WXu?>=TPHvk`sxWT@<; z^5{J025;U`sa?@QWmVg2203}1Q_6EXr>b4|o>~-bm6c3@v*_iI|K%tJd7rlDBm)@D=i3E2bUl_ILaV7wx*9h!uW=onK6)D$F3_+jAXk?UM20tQX>|3{}5-+FDLBTsD|QwxX~ zdTSQkFPOVEMoK23nFK}Gu;~IjH~J_^vPx2$CP*O_t(m740(pqyVZmN(Cc9$*ndz$F;oPMeM9^9 zje)LjeYD!zgo@ZUJ@4t26bn?X|J(OGx-|((7;FH`q|XW zFP17+PU!#_Op)Ns;s=#^3Mg;b%tP4W>Hv*EJ-mM1{l#sJe_~^k*|(cvZH^*5S7{ya z&=vT5b84t zlts^lEWm=nN98(5|D+(7@(?+BOQNkiiwz~bGCMluq0-AjsaNEWEp;=uPKuB7eRf+K zqs>|-X9EcJqpkIKf$8Tvzl&(?@rsT#-@3k#F|uXwL=l!fXflj^ySg4NjOR4AZ5_@> znDP3fG;ZI6GpOD)+a@82^vyo(Tx2z}w$-1S1f?oxUUXlG*VR>xZAusP@F%=Js zFAX0WP0142o6ZW794-ARgIx4x&C4sXfUAG_|7sdWuMu2a)TzlD8Ht8zi5h7|wu4oR z$3}RLd^1PiX8$V^Sx7YLmmW9D4rTgdnp!}GxA$3;n}=2Ix#A5L3Wqy>551C7r#pzp zP3+Wc9(K;(%AcH|<+k`;??(|j_54E{n>2HYvk%nDJx4)oNcYM0#vjLPBVwqzgB~DN zFC0@Lm3AS(v)Rav(R(=p{DBa$|qE*rMJmZ#;*d&ahr4Y5Aa)=zXb^ z3mDY`zBv@@vB_Ta$2ur!OuZGuj+U_Hd^R-*asT-{oHd<N-Juymv{;d2RTJf&^|XTxD;0rDSvV7 z{h4`+5pLL=z|)0%R=tcxS)iawxTh2*4@!m?7kV?EL{6=oL)FiC?4PMt4jSYR;Iyqq z-=wVP=^#jc2ya&ivYm!0wVY}~XVQz-m)9j;^93{)Id`RCPA&y)p1Lq1AlNCs2hAOq z=d}6ZD6S95@2YojED-f~Hs2P-j{qp5>C8f>A?&}q>u<%0QSS!?F0DgJ;1zgF@vz<- zPv`khA3rtlswEsTw6NmHvCf}GmhBk3L5GVtwG+6Pi1@gqoA%dL#snS}-z410G}N8r z!zoc-&2AC3(;zeo5_<ZCwcWD64ug~j9iY^QH#oNUmf6u_|08A3r^@bj+Rtl z7>JIX^|1CshYN9%6o@Yh6B5+hxSOn77R3Zj_@|Sp$7FRi%86S=RFY^rmgA4*$6X-D zQ96qcs2jg?&z}t7$rqiNe9)jcr6H38=z@;Zs>LsCsUWv%m zjL!|aU#xM$h1!=FAmm?21~%uPE8B&{54_K6Dx_TuuVx)J%t4+sds5hoZ;>hg1**Sd z=nX!%&Bmy2BkFu+V^~weYSp}4=`5a34Xz?)NnlXhxVkr#ZO?;IJrv9GIA!4$@h-Y; zz{2d9_$W{*61fGnz_~X`VAT3(XKptB{ALKUWKNn%G@z{o&n1`6sG+#hjSbko)@{~N zI8xp#k8t`GRCWZn+j7_Gj?N0)! zuFo(XGoInt@^PFzueI$L^QHtG6OoQjqqqgav{d9fY+T($+0ioqQ0y;_p=7GE5@m#V z*^=qkuLNB~ehDo+-mk!9Y%bo@kxCMZAfF-yj?NdfL2-F!;@{^?1E<(CLKwrosr7wV z$KQTCJu<59qGC|#yOjMTg-DI@aiLt%i^MQeN-a&Dr!M>F8G0Y2%apPCq!yd!y~ASG z6{1of;zLsPpWLYj{Cw_ifw`~!Gx3z>X?jnyoZ36?5zC&Kn6vvhKIO%mQ~P@{iRy&% zL(DqpV=UCknYqZisW9!hUtGq?az#|t6T{fs0|a#SU)AleWk8EGA0&`WTE+M36uX|)LD-{yH-+y)Y6RU%))$lx)avf&eW)vR5!`SLeT1= zWQaBC$+=f!e(W8K5*#bZNz_!WD9N-x8#5+sQ+qd^5vYz+v$WFo)5;e2^u|yO5y;Jm za6H}WtZTTssz^apG}WNz>RRx0NmPdq)R7{eOqG^)hib0%kRnA7`kd<)Krjzg>Q2dt z@D{xEmv)$xt9GxC;A+gFU(KhXK|67L35@1sRdf8cy=a4F0D?)iDOm5fyg&B8g>xX! z1r?6=y0dLcQ>0%0UQA)^WomoyOAshzOF3dP{!6GK1yrvA~ zgvrhWjs%*m0|eUs96{t~eU``HLvZo8o|L@}QRBC!cL6lOcQc8eJGs6B^FRl%m8+5Q zDxsiK;5+fmiM+QTaIG7kuxrI`jwVFs+E20_D%$REMNcdocUv~mak~%2M$2^hbencv zxbdMF7txEfn`HzR@#_0c=YrdMnLiQCv2G8rZQL{UNunRTpFN$pRmdu)xKy{1UUNzu z)X8kPg=iVs${`K~I*z$k&!uF`o8etp!fnCoZ8gEN6JMd;SN^PfgbD^`?QthNC;0ee zxc!>Fu1oxvd|#VVjt6k{zACf-OxtHwwI}DDB_0x?Y#^a3xj_;NRvc1+Zz8Fzy`i0@3^Cl@FZtp?E$G%?Mt%UR)9QqHg~VSr zipR{aGAdk_+7CoyFYy~H+?`f;u99gyNK>~sSVsx}yICL;gHu7uG~l}u;n28tBsydH zDV^P|?J46NzpHy{)kQF>zIypxhC2#Vg-5lq-Eoa7E~#kHXBEwsJVt~5R7v9*h@*}t zc|6}@=r2Lh`>eRWk^yyASB%TScin8C@2vXHx-LQgBJ{t0b$_3JJA!Ld5i=G~Oi)OU zx+UN;!Cqi+TxLKu5vCIR1yckRq6Ud_HSRmiZr z<*!vo@M2sxfZjAP?VcM67{2UCj|SD4)J0ch+e+b9V!J#Yz@+C68ze}ha$Yd?_kPbp zzWSodxd`FJ&z~pKDnjZ^IoIs}(m&s#{C?^~SlBt^vi-1clIDfrBN~;T@!hFDYhSkH zFrV}>qI}F-Pu{iTpjmIY!SV~MH%(sCMY2!!-~~BkAS49ctRS#GnLzGpDi`5ebUJUOwZI#A+ASIPtDo z&rD!7*eTVCR!e7MnqB;?s7jj-BoVG_v236cN+k9bW@hHB-0U#12Qb*Iw(4lDOOJ{b zJEX-yFzcS`GwBb=Uy9>JGQ}hnU>@^5$^PLD{__+0@6@pt)2NT#)xy#?oU)->IeKW| zBzK-j>lfop;oaAm!L*&2NcB)7w$D)c`MC6VZu6;#M)12 z1*lq5Fb)4G_@9U3^-dB8lI+A?6h9wB|5|?WU&6HwQd@uK$Neu0Z5xvOV%CMK1!aVV zg^B}N#WY!mf8sOc`puh(6XHm2Jq-%~6B^ACbNKEJv1^s5FYdU*GtaNj{Ouz>1m{Z^ za(+LTN|5o-??CDLU-L~$bo(6aUPY!48FcLWxJ{WvHJYGvj3E^2%zIgmFfG2S9t`}} zy*tL`y~}mm4LaXtYtjli$ts9_LrJY{JHo{@8hvi2Nx}!m9UEzGdpnFY>StguE;{|~ z9zChfQRU6q+k>br=y-RB#T!M_`rU2byvAzA%#IQHhz+jSK;+ltA(=y(aw&6uG`d-y zGn2I#Z|^HK7HD&HowTz*=KNmLsq{{dg!R|&4^Mu`58dy01l>{-y>p#zh>%f>pJ0U} z(fWuC;pGMnvj=#w83uwd+@DeST;aahkH9sP{a34f`A*OEX@p+AD4(vC zP_`<6I;fLX5oJ0s(OqMWe5zA6Y|hhUHdGPyM=urV3NUZM9K+O<952^{Mvgm+&!T01 zmmm8+6=;wNJ2Bq4;4IePMC2p)m@c0P2| zQwE)Mk)%^xc~;)}$ieQNi+2C`%iCeSgom4$gJU&I`-|rR~+)Z}C9K{N zQjM+v0={o6GU6ID@bt3Wy4*i|&JEmnuEmM30NGA0O5g1DE{yzG?l?vRG1X}I8^N9= zW&{NMeX{;0Vqsc6!$WIAM)u~;05zwF{(Ui#B!@Gg*rn-wtI zdOcUlIk4z(^;v(2&L|O7xGIHO&DfpMOIHuK)K zR!>^30dT*71JcS2kC#x#0%wd@f`k&;IBtpArlg5rJvbESN2+?{Q zN}1=Z_+|;Ee(pitR@o=EOp@~=tyICt*`AJmHF*gB&!t@;mFlyWExE?(3YRIKxX$Zxue7$3P2LSyWjBCX-j8>Id) z4g!{=!Oiw076;n?2FkBWW06L#a^GA?eDP=iC8^tx0{sg zEmG|?iTrmPUOsAGpO`)$H~iHwsNl>OANx5CDp~^m>mz_IEkFQOUzS24ceDnliXUR* zC_iH{v%iK}@i$NgNFyIg?9?;D{&jiW#fl=og(`+-?osv=ft>36>ySq}wH;jU&DPFt%35>COO~ zw0CnfQHV3f(#5zfE&{PsM%6MqOc0(Bnw|tfCPtZ+mF}%17s;1h!)b98j*R|NB);SC z)%%x12NZ=&ThZ>1bGv3~MZIfBSf-Y}yhPU}C#pvBY(Exq-2G92XkBvXb_#T*UFP z)q>nG8|;{gccfjny9r6PbsE)z&hN&Iwvs{$@^1p9Ux@gGb}i+p2b#i4z~s9 z8F4?(2Y4+IlQg{Q>83k;8SfC~`5tZ6>eX)39}c24!v9R!R&JZad@YA|kC2Vxj8T4V z9h)croJcmnf zkIJ?|`h`;xz{tBs)OE!e>4r&KVKyZhl<11=rfr-kE5t~1&LEJC%Xfmb(6Z6gf{ zd8jf93+s2zrr8CynxhpykE=>(LFLv;_O2-}*HCCKE1)LVl;+_y^=is=$6~_Gcv6#e zIqyeCEk`EfK+2t@t)y)6!@ESB%kJ7BYd)~$i@8hqL_WSloUTRm)20LVTfv8n{pF&? zYKJ_DIw@P=kf^)vc!nq(A_SY$q>D?~k+M&8WY_kp#C@pV&}%8|#9d*D00TKK6Fv_l z@aHz`Kiz5Cgg>WbGT@JW#XT^Z{H;&A)cuhb)@eQokd?4{Aku2hb>Z>|Om+F8T`rALd!s^kVsDNaD2l++7*ST~}eo-A0L5 zS1!FLHtlTE@0OEb;SRZTYTX%%jqL_#b!~%v3nirrmM+&uy2+}4%Gf_V7UYNbIsY5c znb{boo{7ycqdQDtWj)?Aoqz3x8sklQU%bj6c>FJ@6)ugj#NH%VsR`KS z2pij*kXp`Z3M=PMMNj3(Ipnb^yN9Bk<#~2EyQ=h`pB5i1uNnyPAk2ORU)-|gy+7xM zldTc6)_5jmwe|Ng2LNOs;R;vo(ELr>3@ogfOmM`Am{ULy_=mbUzCGzDI&Zzt&4cC@ zhI)i1qWfHXD)De}jO(m420UCT%u=vAoh-)j<#=~$=1TTGzkwWAY2I;!gk|^a-RcA+ z-t@1c@P@kX8%yjOx0uOKerUd1@QJ%?I*=Ip{J%U=+n-`_ESB|w|JXEH7QrKfZ^A?@ zzMV1tAoA?ez#?g7b6EJlq+p^kU!-ykKAyG@gd>|06OiGM(+OlD(;j4pAD9&|vpyG! z&5$N7w-HZ_O7NV=3f$4YIz{;RgBe>qYye)?ajDT>#s6W}7`#zjyr_q9$1^l5Hn@GC ztK$TI^kV8C;iu1W-sYBhf}!k0&JoQ9{!6Z~h5XkJ>$7j7#g(+|e9Cnl4x@^T7b$ zsq=pzFMRZX2N`glRXGdeJaanu#)IWL2}nf_x;bgz$&b)BZoSKJ&AF607H@u7^3=&% zb{fSi9bxU$i7qXEbrbFLLqw~nCL0v27rw*Je3%FxY4`7J z*xPPdcxJ%$YciX*hJYR$GQ$1tGgjp%P6D^@^w*JM-AfK;SkkucBs>^Ig@OxmHhsf1 zOF32l3V;l088z54`DPUzD~a~ur}Yp@gGhh2nsrSbZpSQxxRfnCZHzf9%^ z#!n!c*t?zb3j_<~|LUg^;2q}j#n^DCsWy|iH`djz3bb5R@A(_pLSvqJX1O&1Q^(oA z%>dhS3Pii-rmcz@CM0q?e>TZs@D&}~8%M$KS`4Hb+UW z-0I+p4_TTfmarOfr+Qm!-g0s53F;V(sr4Q2tw*KMv!2P1vFM3aJZ`g;?;Z%F zK0kgVu>+HYjphjXai;N&j4Hm&7P@((N;?)IawY$9>M{d=bfwi{4HG0`+5Xj&H<0Wh zS5D4j{_g+J0yuEIDShJhfpC*d+us~(aWLBGF-2nXpg;eKAAv{Pdz!4H2o9$wCFAS2 zL=+&*`N4dRrIe&cF2oaJw~(X4^*Er+491l@|at8p32MU=s`%Jkn zBxXrE_fM-7d-G~$)Nhm>wF77+Kct@(_sTqb$ZC>L5PDIrYW!ccRK7spHf6|c5!}}Z>0Bm(u3)3&B|$GoWreZXfLlF#Sp!(4l%S(p zo(Ir)2rWf_i^Ynb3)Yd2B`zmUl&Q(!t}yuUb->TVk-8L@d6W9OHJ}?dfp-|fj0aWA zxWA-1m$K1S7}aWR$}<>g)t?KR4;{n_Wc1Tok(hoQV>JwlDh5oSIjaG`Lm{H3jJd!{ z{E{woz@N09^CI)*|0t> zCg>6|wFS2PHh3B8Pb@I?t+;Y}EU_>q!(q-?-Gdv-%yYs!J&lU*-lnaqxVWj1s4cbrvbp?HteJ#fZq!bD`l zh9WeH8xY_2^teuQ*s@fy&5RtUo}pQ>9dvhLVImXq2h}3ueazBtJc)ZfUw`|EY`SmH&RImh z|8i!|aF0!u5g;&vSbMf`QSq+nq&3D<32!D(Mr?xVjY-Zn;~Jc`tA?z%Zsd3XGh?TD ziu-vZ=&_HC{B&4iW-Z3 zuVjT~N$lo?5U!P?8omdw7IvNfF1ouWvD&-ASWfHjAH61#=D(<1InZhixjG$+;`|y( zm8|-A07XD7(~xkgD|Lrt2&SG?;sVY~jWcqv2#wW!z03hEkA5z1ieY73KH{ z0@R!cb?F5&^HBZtSq+T9{)tkEuDipP;9OXv-pEUdxMAgT2>fz=b#f%_eS+A(Yq8?; zd&>Yd#SwGiQ<yOh7w3!N4B32xl=H6p;WeYH%5S8kB*H@@KQjY;WiFACip zuUT!lFODDMalQT6^z}&iJ{1&H^cLBv5ssS{Jw}z16!ktylRK;Z6&;p)GRz zp8|_$3MQQ~GZUad>dZdtD-Ez(>ob#P?e{jb62R%GosWUcU+Nh1Q;(oWJ|P6&Qs?oF zAMszlDByl$Qx0l6D+d5NIH8uTToQktDB8r^?HA!cK}{gv@VxJ-8y$j z9MXf@n{hns+_BbOK2a}L+i4g54viY!^~YV`{<}T^-MM&O>}J3*_5@5`?<>tO)-WdD zt?$CFq3^K~WQmMGtWj<7981VyAx_(KbA-G1)^oxc9tj;TeQfCXyRCHH!i_L~f+l1` z2{=NsFoC&Zx@NjNz1N^($)E~3t6V}uFqc0YFt?Y)R~szypoI@Id5A~5A5r!XhI4*B zF0-eGEkoTkZA3Y2-X8|NKP$h~m8OXb5;o=679;e*2KHV`jAw~3e(L$aWfDlx^2h&Y z==I+nWD*p!J7sH8URlw83W!Th`SiE3`k?l9DYxCX{V?|Ri-?FHuWeOzb0wXx{|3lh ztSX&db}Ek}xk4iYMn_oV^okaT))bj4@HQSR2vLhIjpD-hRtN85ULN&qzNbRdZ@o5+N$$AM%$4i{C^ zJpH0R_ilms_A%zd46Ffn9)joZG*h%#NkzJ}aa;U93-+Pi(~^QkPB7znRIx|gyUh5Y zct(5bt(c=%2PY{efx*TOGjZ`pol>XBtQkNsHeEEhe$`WRnd zu*tJ?Z2tD%@w;&yD63l}?4@B#Ymg!Ly7DDUZ_Q8akM@kQ=3;ikIi-PkdJ*Td603lv z%VZ@|dJ%uE@%h>PI^~;~-=fiT7)mmNf-GI3w+FhR`(NVH`O1dw&s*OvFKA~swO8QP zpv-O-`Nfu?e@UJ1{+>Vr{z?mC-P^~~CxW(pId!sML26ttPV-WeuWaLNualT zj&s9CqFJv)2jZ;3FP*JW+F<(U!arw-v6$jpR;Ei5=uts|C5)$P!=eAFmS@{*0F?tT zIGz_5QAj03?a?bP<2om;UH7LT`858)q~LTQMKKV>#tIQhd9Wc2}ZIsbq%-&-(ZU}cWVxhIpOB9bxo_=kLBq$qMiyGJfgoG57BZ$B{7c64uK}N61&~W zpz~G;Oh;j>iT{ojy4NcIR%1FlUCRM;ze}#c$(1q(LlBhSbX9fT?M|yuWqRS`=Zr1R z6LARqLm9&4`QO_v?uNa#OlCUtXy553!?^j=kDB&m`K;-NP8*<(|4UjBx>k=A-7F(T zzncw0)#S{7H{VkN2=gm&h*)cHQ$<3zubZ=sX;)A)d%Cj3S8}+(3}(!IDThOKW~_`9 zy2|O77{f|)g8Ew&X+H5SEEIeNSWjp zAVS~5MDKtk^JxW_HR6mX->^=-)9IO=l{=tg@KF+Zuk_WHyNsmMRt z1)(c)`Xb%%V~<7T*qLe_8q?qU{*4Wqgs4Hg&Y?0*Q+wqoVR;!_W8Ao)CTYyqk||Sn+E1s0YYo#|uha!z2)sW%g+s_WAUeh|ft4ls>FQTJ{e|;-l12 z-|#slHvR7AHv`kR>etKWVjLj+4h|X(;)LgDiFh3kY{T%cWC$i70zOZqrFTwv|5|Mz z`VxRnBg*#3_TA!vXHFr##BumR!iiT8 z-yIWvu`89$hu6GhAfI=g=zz(#(YhPkJiKnyol~m?jRSc7^e*`rZzyAlou9S7;0|=; zc3RpeYxG~IEjUO2A{VHNrM^f0eYpiaY;d?)b4YQK-=Xskn`0PGVGs(q&{uqFaf*7z zbcqs@wp*TcN~Q}(IUpDgwb$)=X<2mv_enLadAKyl4|p#QjNi4&kT)Q)l(d%T8wfDh zXfZ|g##O0QOv({;VEBaSu;|Mdjzx&B1As6LkBn^^yB@fHuigX{51!p+R%~Vxf1fvw zZEimJ+!QhIzj%KlgxMA{t}|l-Sv0s^S3<(BZAP7=E}`Z&6|-+Yq<)t``x|?QKR&2e zN(f%jqip^!LhOW_6YzorScj2Alr1_@oiYcH;0)Y)PN^vTg=w_CY}YyA?jv9IBA-t% zBAU2`wjC2ot;iqiXLEYD?)kN_2-qbHA)biA%O`B@co9AC&2dc-A@-V4#jWMElaUoa z$KPL?DK~OZqrN<@Gw!`@Ikx%GN2HZ4&=mYSuk8habj5^I*A57NCDjd2rsUNAAde3r z2*_2PzV+vl$3{Dm1EL99s9saJ0TTsEEz1=Yuo-N4}C*)2Pb#!NgcGKX8SUt=0 zcmx>H-8=hGEncc?E18jO$EKUl`BQKo(a-BHgRj2WU<6_kwMEXQljnhX{5uYM-qzf3 z9eZ3CdOHVd2&qXK`;qCnSD6JwnE?NdSbKdgZIbtVp}#vlEH3W-P(4k>@J-OcgM0%e zN=J_}Uh-&@S($##-wB?qZ!@g#mF4b*EvJh5A5F)+WLuj3gP4$gb3|>v2YRCXq^eqO zzWhZl9`jV7?z8WQ6LZtWT;(1?5xi!e@oI@F+C0UApT3AAD;q!VpgCN6($Zh000Tmh z&299_+d~2Z5%5Jadcs~~`rU<1cODY5xUN!)E=7#p3yY>*U>}S3K*dg{j zPF2R_qy8xZNcTK6_Cg(^CRy#S2QXzk9;2|9nqVmo!JuYNmaCuTh77i0e^{zt-$DF{ zv4x@nfA)<6yON8o9;I%;QR(&C?trY;wE&%+er0jC5 zmnh9#1TgSKMY%tJTg;*Mb(*FHWr%6`IatPnl+cMi^!^L%T@J$Du3r-JDWKNGu@@vS zm&FaF+(Lu3IG5}r7cSLbdGL<3>dM^P?4;aazrvZeu;nUxZ&MaFX{QNnI3b4EYXfCg z9YUQq5k>wt)xOhviUE`cjvfY;?5Ph*wa1s^ z!>}sw&X*}{ZK44Ar@n?2yJ}A0xW1|Kv!rk``o3vWdX4jXLo?Ugp1%Bu1i^io1^b=0 zoYwoa>AjGIu5G)M9}f3Njo#JPzd!hSSOzRk9y13AeQ1g|!0CliCF!xYqclPH4lo}b zlWq!|c5%&!F}rf?#c37;O`Y@P#1m>e^O{bdXJqs0N`@*3u^aI~*RiUN11lSzG#>LK zlpARlu>*NUT4XlIt*s(t*713QJ7r-n18wa^$gp8sHM&jg=b~%myJgdgBU9LbW|-Aa z%B%(K{EQ?b?5>!&P`&1xI6US&>Hce%_mq?|p!sL5JiMe{j^(b;$3okXo2lfjR?a(D zsxAWOFm-rS3px6bVRZcBwae{B92F2Ic@*&FpIcpstH?{9+y(?50(ih4r2m9tMON~Y zjL+Yd(3imZ8@LN(0i=TAL7cmNd|J;N z@lVSpWU(-iPB5xPL@aw|BHP@wEu8S^P%Kp!+M308g9#CT>upfh01#OPwbdV4T*0^8{ z`}huovcZVFeAx^ZvG#m4|N2Eo#NjYVAyhK)blKNAn{b!>k;TgJ1Obb@+bP{I6}E0; z^c=s0P7C>x;H#%HJ4s^PX9&F<$>RgJRp}Hd8L0!I$-`G#q5s{6#X}Xa=1pFI8=c5U z`PQypH{TsCSapV6lS)BHZnA`cdnzoqn_F=#Zx7T7NM-j}z>aMJlPX#)m7IHRFPNu^ z=gb;CMrpz4F|cq%C$2og<*W3?6G#15vXrZwK)0lnTP=v6NOH(MB3(-`;xZpveQc`^ zisfG^1fOg+n|6P^#wS-yctv-qKF#6@Z#6fsslVU?J>@)>4(Oylx93TS#?sxRmNhRg%{6E`(}byFK^ z6Mwr31%lvXi?a2#u z-jbT@Wv!8uC*y3FsgA2dnBj}CXFi|PlE0-V+2pAO#e<__yYi$wZYN!13xyksNK7pM zrT$z?P@B~b6HhX4asI0#1@*%Lg^1HCoBJL@K+wQe+$#9dGmUcNI;+l;t$kirS#kAd z3ftF5wKk1@MA>l=L^lk$F9uS)7~!aJ18MejxpUDPaQ&Q7I{2Zb<4*Jl`vdp&I2O+v zvrZK-;=MoAO=?~~%jcgq*vSrDLk$X9Bl933p0&lK!zN}sEY5b#!kBHsO%?MtVeb6D zTlk7c8n-{IwIudT2QJ7E)zAEd6_dg;N3DvbBeMg1O}q`~O8uEaWZsrZAjKVvcdIIV z1@oDT)rxNZJ3>7DFmJFu8Lv()totM*G~OEB8* zj7lt<>_BkMp<}4d15M72^1Ymt3XRHX+ZaaZ7UJ&=rQ|Dm$|A=H>L+pIfDbH5%~8y*nHzLS62s^|2OW(wusYpzc_{(kb8kL&FkVk$*$_l|!L^hNW*j3ckrM26bgr@gS845=xEv`ooL zCl&G)vA5(^p0dFF2s8Mq?*KP zYZOgWHuhVbPX#Id2#-=@Oi8=@(%U`kjF)=>5rGiV` zj`h+dd2;>dafNwh(Zvt#i9b6x_p5N-E7(rA!H)^=O`s$drnk<`#Cyy zjm&h!WcvqUweXIRO-M$qnd{lqwpI`@SY_bg+sRcMj(qfVAhH;@BWQq*& z&(L=8U-QQwtt+SYg?m_pBsYIJT4k@*W8L0IX=hFZ|CLq|q5mL*N&1{oQ*ri?K*ZuN zKgjFz)A$7T;j-fRUzB+ZzrL>!>(zX;mgXZ8djwAc>GZo5DN?7m5;JdHSgjtl|MAaU zyOms=PP=k?0ak8rxecFa7$`hviG`YeSCi)E%-AkgGtKIHo+QA(h!X|XS+}^gIK19< znJNfA%m8(*;=ta%6r=-n(6$ybC!mpT3%UJU7lF4l7&-g_Ckn zrxQ++PG5KuQmfE)+eqEK&a>agy>X1%lxKN1jgCaLOF|k{w+Z;z{Pn2zQ_(SCyj3Im#yO>`O2qL^_*a(DZJOv^{qN;l|DbXbK6T=pMzv%l zoSyVQrEw*S95zMv=7@gsglCl#j6J4_NAls-cxsF;&GLi<0c2x*U5y zKbC@i)XFY<{}@%&pSUbq z0a6$Ql|8ol9&;6D78EIxPOeZ4#iRh`N9kU)-yT)Ly6>1roqqLCF||Q`?ynbNh@bcH z$RrY7+D$>v5vLB@!Gf26dt3l_%ZuQfmLSJR5qVOWNURVHd>0D6D~btv2#a;=T8XoX z(UEc&Y)#pOY4V|!lT_gjm@z%;c@k;Xg2(D=F9?fFN{mi_KN zp5NRnQ2oXo%(7#j^n5aS?PA}bT^w58>~vU0j44B&(3TdJpNKYws0xop<4&b8H}zfM zF1zUKKQ}(=I4a?~nvzl(#~gyZt^?ky(ptTax`-E0*;HY{obTs*M~v~5b?r06}BaSLWlBf@V-y>G*4uFC0 z&1=c$aorh#E3RSqPbSDXE-aw8#xqtPB>GqHZbA{z_dH>Nc-fCsvS~YHs`({()85sn zNqT1i(qrE&Q6ZX~3-1M)ttrf`M`>SX@Y>&OljsxkFnj}$nSD4+^e*X2Ih1jkei(dK zAY++x;9*GSwLM<{HP4wb){?2Tk?{#1LtU|A7nYK{bWJMX&x{OEmgOFtUG z4jvLDo+lPjXnhuToqO9bw=5mE9XC&KZxXDUM@#yp__5@fCn+{szqK3f`kA-J*+iO7 z(vxAoh3ZRfX(hGNlw*nNnM9MwkaK|6qtA1zU8|j|rvP?@~HU}9sH18ME%@NKjbCyL}@@x{Tp22J_uP=R#*UcvZi~bE?VAlp? z*-WP1W=p4D{+n+tPev3)&IG%*<5rn2o=(flc;3AJHWz3i2*!N0?)aFNGD*6TW2(^) zUxhW!!G-KkV{{6*f6Q3e?J#)__lz)}Pd?LysOzjFiV5q`Y-&NwMb|6MT#7Mk+l-Rv znToyoYhxqKWg-KlJ;HX9cS@^c>;bmsMta2el%L*%iws{Q@OFdn-uVpDF#V?|n~Y{7 zXm8<;Wsp@~+7x7tM9zkh$X8jy`2Sd+|FbGs^sUlJ0X!WmpeQ%3IjEqs{P$smjrN2L z?ei~#7S8kNL!r&{HwArro9a+0)snz|62_hl;d zjWeA`*^ppfZu8)@n3KdDsUc(K=rgs(h%nQ+Ddf@q0`?qwtEs+*jjBUsc5BE7t)DQZ zL||@bo`!{4sk*+{u2yZc?#I{ZbPsMkMB_x2rU!ErAM#7sSPMmV0wQiS6|kRDfX3uc zll~(f%fcGTJAhP-*L>qr3~Z_0KPCV?Azb+x1fRZ^MXI%)cStxC-)#z*r1M$UiKG#z z!#-Rk4r#xb9&Cv=GPQ3G*&BPjIAPrukbFb#$K177CdY`iuRJFw+E=aeU_Dks{DNNM ztX7^>E28XQ#7Fl1znV2kCykBWr=g=rltE^qSprmPS+}) z#TH6bnYi4?!bSz8W>!&FZ>bOS2}-CwI=TF7Tm`}tC%JUjg3Ahneq~vd|A}srR&Lt+ z7EC-*dLY!>Zw2jKt%ZJatDfnbHA6oC*i`fpp|(A~gzcWZVJq`9l2A4M3&09m|K5dR zFfgPk!>jW$?ZS|m7}|^FPYR{$wZAVa1H13mv6vd~?QasOpZd@kEqL&LQ_*#N3b1qrQX2SwhCt3M$qv5D?dYI2E7uAcW_BTkzMn<(Y2SD$8PKMjvc zK_4qU^KE}}b*}KuJ|F0bnJ$v+1m{nb=>#tY98YZ?bDOs0PAghJ9F1?Kl71j%X}e`0 zb}2OYgbJv*f!T~x{MlYA3?}6deZMRPzDdTl@2*_LG4q1Z;}A57o_x@?4aUiww2&pT z7WQC=gj5fJ>1Vn9a10b2tGzqI-3$mIHjs}coGQ`mRruyTi~K=Bli!!$v%Dpb>1BQ+ z)3wZ3l^+eRpABYPcVRJl2FBCk=tN$x$rzfd9wELhpHkT#wO(u0;m1964P>&tLl@Q=0HO{rS&53d9Z<8Y5I%=KLqYn2iKWk8Eb<=n{l*Z4@e}tWP#3F_@(~bnFo|Nh zv?8PfQ?%&VF&G;&8)=6MF1LI4b;2z{qIhWwZpZQ-K^}9V-Ln2*AnAwaa}!_VNTc}# zw*NfjVrhW9T1+9XE64bI1iLMXkS%0$ALh8Wb89n-eE)>%(d$o=rXq%g_=VJb#sxqO zijXCU)nq$i{^)O2x34H(LRcR1bC=3m9q=+gu={vDjwxVi9kE9F?$cjgkSbyC_DC6B zd)@5GA5%zgM>7||KffH#G0Uh{y@d!x*oaH&3xHu^Z>7R`$C7!kUEDX0YjXQTFvBmK z-e0{3|2Ej}##FWRqY&E`Xo;nFuJi4XS78}||NeAM70G3H@|kg5IMU@EN;auBEm42g znILG(ds~}!DkF?`FX5JZSYJ}l^phUt*1v#Nz-%(~uc(oJ2)^oxBJV){NPOi^Y($-- zq;Pv!!lHmhtB}h$ZuP}b>GxK*e(3#djTN9#!0xm|<R4+;a3TX9Xsb7{#k@{q{6 zs#0J@m7SZJP+SN~uW?3rBQ@^ZehUC-4xrvC#oi1{k^Yi=2$|6iZCXYXGRw4S_tq$N zBm4!4%{RKraLW+);Mo2*#@$s5)mu!@>?T+hn1o`LdTw_TS600tg+_znh>bYR!!x)| zw;MGJx#I01F%tj%@c`|-Z1e|!VqA_H=2dCiOy#Xm@Y$zj0szXamhWqlfJ{=nW=BZF z0^nH(WXHoRBU^joYS33EmPj$#CF?-v#l(X#qV!qEZHWWY6~3Mj z2bW(IEg&cBfN?3v*jKq zW#GZ3QJr5ZR^G>RG(52Spe!dQ{h{E)LQdCTlyJ!Uh%IXM3AN2s6S|M2wxrNbhc~b@ zEIC_DgWOQ|2lk@tR3>aZ@QiVAu9ANLkF5yxLiHnWAJRHGP0~Q5hKIZfSZX!wp%5kK z`QT~ATq#TV#0+iNMMkwCMb85!vm%j3e%^>vUlGKbdsVK5DATznk5+JAxzsh5qX=>tF_+ zpxW@t+rD2%{q9^x6a_MawrFp?I!^-zsZAKi53+w(E-xuujmBMgQh7btDrvmG0rWV3 za$<^!ThH`7H*?o1kcOX@fs;EuqKW2lkDJ`S*WVt}4U5P6k@vH_yBuEGl=>8+2QB}^ zgeog~*fyJ|#&MuN_=AkCE4qFpSO7~{9QV#%UGDM+hy+JcKL3_1bmbQ0G~AClgXCv+Ym#qYXb$$y7ZmVs^3p% zXXYJcHM-7Uu_y#wCF+JfYtH&Ls&sI{CqxJ-XLt8y=m9D$vO~~&LbNbLT^XnSB)b1@ zerZ8=4e1Mm)V2{IEIazd1z-`{QhC`i9WIF4VM8{OSWW}@OsbLFv=Htx-Gc#2z1@}& zyJ_q;QuFatqVH@M+TksFzML=XSFv^69Y81U{O@1e1k1Rt$C)L}uUo46cgdyjR=nT; zlo=!ZB&B-F`gg!20Io!Rg{X608~#|5GcA*K=Dp(Z$N8-arPAta!b_gRmWpw{;ED#= zW0@_tpK<4NVAdX2WF(9F)wzG@{0k{gngg~{Ly5yD>I8o)G6P$U3ub z3?(!;x9Uo54g+nB%&TjQ3eWh{s#^c|%9*>h){as{ukTh8q)w|n)l!E5N4`4}=oJF8 z>=HV`Y2&hY{*QoZEktpTXJ9Pup&ak>L8xS{6-kuqDi{sh8b0}sh9LVA$p9x6fg8bP zNuNj6&!!{Jny_#dEPQ|#)IP0vM>iZscST!gJBsEnee}-e7#g3%;8srH66_MovIb-- z2|;eyDiQmTDm=os&RVIIxPhu!L*48>FI?=Tq@`DU0SV(%3xFpQs0zNC7FLRPM2-wL zckp;`kOru63uQ>^Qh5CH_`o$B!{hk9>2ZxrqgG36aYuC}x&f%#bTP_ep8ZO4LtHV7 z*2)iV7pHJtD+dz?Gm`r`Pc2=@;P4S36<2Q7KD}|LzrS$0F>Tcjz`!XYR)KA+Ar_+l zG7WVxAR(X(%bO2cY4USG?=8W#9nePDxe+E?-=B;w9dN(nn*uTN(4I`S*=1(zn!{O|6hX?iP;34B@GDSzf8 z*R@yUor~y+;uav)le=}5JE_Ols~K{hbY@UgG;M7__DSwmcp*)&lZZ>E$p+8+5+hJd{*Em?);>tUw0o6a{J8n4fObB%=#o_;+M!3XK`O^{WgL_3IX8q32 zu4QQP3vqcIn~z@fOBY5-c(wvB#0t(*2_hG>_$rE;hrwZc(M_-4o@Nr{Hw_c5o(Oxw zqFRARbAhs3S$emtm#xcuVD=^?o*s+wR-4O^ckAOZAH|6wowd(*Ebkd;$+b5EvPV-^ z>E6}$>9<~E;*#k)&5*wQV>zRS?=|dufL8;W=LdXK1vVK{Il{hJj`AMIroYJ~eAoT# z`+q0!Q1>)W%3zum{r1vcB!k{q7dI%8ir~wDv#B zx@>?7X6fW_;oTv*kjX%EY@!~knQ07!xtSW}jL|Q>?pP9SzwXr=f8HTC1l#jFxXka3 z7eYKoDR5*=B7fKFvLD;mat;8*ZsgECkWj?cJyL>^vN{EFlGM16?~Jjyp9cSkJ$T6_)Vy2aR=ZF!(VhFiuZtE8v>6N$hEb4Q4nOS zJE|y~c8VUGkF5bJzvl+uKEiaPzmpF5bYQKhI&#+f%H{;8r}5s+kRMxaHwwF#jgXZggp>ampw2db5w`J&y9h5uT!^RT zx1W*E4rsUt==!?P^9%I7D^)a&!L?T`95O1!V}w~A)}GlI&KLv^;G%szcDJg{hM9dm zB)v4om|<&ThIQyJog>`tPlSIk*K_zQJnR1F8{wbP94A{UJ1&+w5@uh8!+5!PTy_7w zRB9Rmd^v6*ow9lV7)Yo``86AtOsgpiiJdo4lwO= znpOhex1pP>6PBNWX2$55#e{s(%6Fn;B`p_}A`CPs;u_Ng%MMI1)c z03zTU-|0E19u9QOlA<5v<$N+5)lYNE`P*aqr8<=RCEi7i27Rz=)a&zpa(!GQtHumR zyJa@pnq<#e^PDaebrx83J>g;%o;fe*qa zM23}r5_>}~1N`JVp(hGE6XJgk9_7;?8eTVurP*&lD>S!Yr}QDKk~x>^ zZ!X++76)S1TSf>Q8m+#@S&k4RF4y!}yg|30<3g3^fP_DV&7$CuU3y_Y6Js^g(Gojk z43k;ZB6pHiFkWuu17(S8QpM>ad7K4qT6*WIX-Yov+xuckp6JC%!BXBVv&`xR`k8ry zA4|HHE6GXvrtEIY^(T!*QS$~Zj-%U<`%cIn?chYRpbd3yUmU>%o~uJB8izhieD0A@ z$Tf_6*US3if#cEZF_J)C>+^WM_VJ+okOZ~-GlFG>#vYO5>;iy36ENmF1Y6x?0XH;emQjI zq3u%Oy?Q;Sw52+02DJaS_gUY}U}mpr`DB|yP!3ow3zSxN4K5S;ZQ(K?uf~buWq;FX z03qYXL&jVo%J<3WGTT@7&#w8Nw7?;%gplu|`zENvC=LAqdi^ojRVq?NlNb zw;+o{L!W$bho}Dn-n9WUU<`=>&y{&0>!2CcmNHv;616L;(~Kr$BlVb@R> z#miil&P~emNFujvAsb9uO4wl`OuqXTrhq_Spi>H_OGc`&vQ&=fSC19cnv>zfq?$|) zWpwC4Mu~xE3{2QLh&|B!Fm{1tFnN&~=yph(^yTS5kYxsw=9nucY(yVxBqMJj9JsUH zip6ZxwgI}rf=7QU@7io!kCU9C=MNO`gh1$l$92;GjJh$24^qysRf}Ewd^Q+^37$!+ zzypQz2Z807;{$BgL9UOwluT2|HLrA?X+wI7(}-iQf6k+i?=8r#LNEU*I|wsb_kQF4 z%Xlr)Ocr0pM)@V=s02c~3y}5G~h1mj`)CA@DF)J*PsDv@Z@C?{Cy4BXq2dF$ z&|MrZeB@8KdNRF&w3k@Y{sjT1pYbMVi*3kNgq&;ntlq`b_k8B~3HuL+A=0N`&Dy45 z<)%14)KhRm&mgIn^k~9bnhN%i&0%jom#CmGr;LG+O!lDf7^m5Oc;mBC)De59gJuqv zAHf%;*m$DIUQ$tsNwYS>xS}W-L#dPh5w6jSN|)n{st{2A8(8f91JgfbLGAznq9*}f zruDa;fMALX3GA3@-+-e+ij#;J**m(Jg01^B<{AyfC>iJN-Cc&#Ku+yEVtWO**O^m2 z^M!~Fk7Fj|37igT{8yxZRj_FSeElP+j32Y?l=?FK{AIGwAac|Ym;9s{yzXTd zT)?`_5Z3}w#&z7@O87+bBOwh+@1)-MDtfZ?q&Ki2AI+sV2DlPpEj?+0w!?I+JM3kS z9cl+t&C7+c-*=y9s74*I=!yn)Ujw`@MM)I55bn=yN@82gPh@E*6!F|X60ZMNJ_|{q z%@0DpxX<3i6=>uZWd-aPA$bP>elpYvE7QyO`Ir5o^TS5tcm#T9I{au(i#g)(g0B`i zjFZ7$g0PEM-VfLAaT2qS+wnqaN@czi4Z~gMkA=c!BlIkrgNVrgb!~noF%6#T@X$@h za@W|sN*LA4Q|V~KY!$bG4b%Xu>|sHd#mG)Y7qli*WpaA7y?TzV`8<&Ua-}+TflZU| ztJ-Fqi(2~)`r#Vz_5e|9J_u8o19nbF+%(;+BZw}Du;ShYWLu1!m&+uizJno|^lDo1 zb^W#p9j$Uc+eM+U;BlQc{qB!@M}!B{2M9Aqo`|82YL;0pTrAL^+Q;!G`|3Ao%}?;; zShBSjYA)5*mo$wEqxIQ*C8Tg{hhRlGAAPNDDHW}UixF6f<_FJ9N?J4@yD-NLB_w7A zBv^qE3*`5gLN7#q-1W0e?&C6M%C-OlY8hy*GGeQnKE1E4+tk>@uS=${M&CYi)S%10 zS&~*SDcvJaQfHhc8m}q>@1kR7O{-nE`nB$goS#1SVR*Gycvj8O!+_Hq(Uld#(Ue6j zCxU@0=cG`kz?JWf^a#rAc3qs-z{&h!&B7F!+}ViEo+Wqm%2*cPqG&`&jWj0CI}|f= z-_d&F7qqhSDvog=2iu?*Hc>ROh}(4Sw(oqcyA%qGq6*n6>(`MtZ_w*~&}MJtJJ~W{ ze7Kz`6msewR8(>RnIg8%7MT-wFU}AB`VNA{TWcCLNguatNHyli6 zV!RVdv2UdOkYijhnMwp(VANA!)tFCuTXL5^y@2Wi?x>n>YnDHBjBavm@owX>DA2w^ zjX>OxGN~v@XNVn0Jzw1?x`f{9rkG0Ks=ANKzg0*sD4QKi^iJ_paT-mtsB=(;;pf46 z`fuT?#+u7B1cN7Ug`+|A3hM=aiwF_Pi2M30F&3tdnLR**rb>&~n$J{JoH)5bI^_bMaL~Q`Xc`2-)lz`hHk__;Y~}=uf1c7U(Eq2bZ8UP?@?~Sj# zaL>Um`XEpNCF@!lyE#Ck@L}qYCUyGYR;QoRn|ZhX&X)@B4Ag${i|)JZCY*(CTo&yM zT1Ir+x2?DT*m5ysZ9jKACs(*i#ZB8)yR(l$ziqMi2=8-McO$6-jyy6$WlZY{k%qBSN88%^%qGS}u=|I}8}l zHxXA7`@^Y{U%g3xm zg|GPxFZ&d~-SL&nC#h=skeVzHcey7opwzwPiT39L_42(r*mOegXWVZoM$3%**=h1T z$^oK=#>!iE)X2XU9)%d4VhH#_hkTV@GYgITM_U~AMWAP@N;Ff8wM28U7AT?TS2~4& z`$L}&(LRZX=YPIfE<~RbD30_1UTg@hp8owv)&lJ z;OAWZD#RT`#G%JTis2SaODgO+scw+5;}1iH?)0jm`>P*QCSN+5U><2Z;}=Fbhez6^ zj>^=dt@PZ=ZA?TO+SlyltgrMsEI4*V@(u!s6rL)gAD3C7Gkz{Q4FS7+gFtp{#Fw#Whm=wBb;~gHb9ywflgMlyh{JPl%x;YL!=3?sD1IN`^+>fwI zd8Mo-m9>4tou$w~S-r4RvtwiVZ2^AE7P=HH>73wn_}?86P*hYS%l`slRugg>KwG<82s)4N7L0rRj&5+sP6> z6bbgnRelfxlb98X_JiW@0&Pc*)VxQw>8OO@~$hlRK7W=TsCaad6C zs?2S9$^QAcpAM<@ck$7JrF;QTtin+_$H7F0Hce@IE}OBEfjvh5`2&C5B2mtKKI6M3SZESDu|^y&Vi=6}r$D#|0o3?>B(-iJwc2eb*cCXt~^zb~A<%#ANY zlR^I+KNFZ9G{)>pU%6Lbal7%(y}K>ZF;|=POvi1GjzIR zM^d(@uHgFWC)NQY4S%Jh&Hec_&Aid-4E3JVhVtf_-<<%!;N#$<|LWgjjF=UYwaJ0y zc*+&gsSOWn-~jetZAG*kFI1CP&KHR?2OU=Wzy3rw$x(?&XtGVJ7^d;!>Ns9mP^tr0 zOmi7SzL%|-JCk6E8CJVj8}{lx-N)zm#7@D+05h&SX2rF(Z$+bF=$M5bE|R50bWfT( zQ*XesQ?_Q;YB$a^d)D5s+jvzjrT-Xe#|oMqWTS_k3R?YsNt=}gb;`E-QPP;hIaP$V zXuV>K0J2gPV&Q$*-Us6f^m3=|J3_FIxIWlctQp{_zGUXWltR3 zOJe=usG>~q;z?8u)G>JdIIIX4iIm1kO#b!vZDm#6xF|tRhMuBXS<|o$|L(8z+(Y~S zsON9rDyz=vGY~e|2}OjJS!~fXRhP5!9@2!#yN@S~!&ZR34)uGDI?ANX!aoRR1IbjU zk*p=(YK`{a@lx~i#4qSgOGXDdBp5uPeg7+I)9|8?nq1kN>KC!0}IP9WdxtA%IAiC;*vh;n$!I$;VI*~b!p^PwXoA0 z2mJWi1Sa($^7%aI{4;IW>8Ur+5yf$Si#||<4U!G>w6#b71M#0&hhMi;Pi^U`%;xF+ z7>mBhc}iz#S`c09E}NVw)H?I!uZ3DKFXg~~lm(+59Z#0TV^j!)h~Mp?u1) z-7KoXn&|l*>r_6{bBR{{%(LgBY)q={Yk>^swW+4_TH~{&FGKAZbXH#L%;JYxa$w&^ zlq<_D%B>7(x-=yayx9HyvX-LmBE#U%kGCA5D7_8=WdgfPLt~?TnA7=7vbT*^AK`=k z0mRSKl8dRXZ+afsxCX!CUlEyPk8#k#_09-5t!%c6yjkI#f*mrFu}C z84y;~qDAa_-o=KdO}(Nvd(9UxL5ZRUn@xaG6g0}}6QXjdKRl>tMrEth4MxC6AFLEh zI}zZ!pyzS+b~nF-tj4a%++$i^wEocu>l6xK)5EIrwVCD zBkbVB%;SaND+R8JN-+#9v2L9U{`6~Oh=s}fU#QCgY)Z@JgZb*psT%}oKUG2|UEX}| z_jvJF|AbYBm8mI7RC|~1qvD@Y$J!I=lt478^eK{NskGlI*%JTvT;qq@v&RI{c(Ia7 zKck-dVh@5BWey?`Q^h%eyQurz_=0$-77d z?*oI-NsX^1Y2T7EP4^IK$TpEP>YMTEKFGvle&F+#c+rQ{TFT-Jsw?S z416d=wl_tfH{q*%c8Xse{b|ek)qb$JMTaVT7x(+^IbUzfgXD)r0S9VPWGoneJr0b| zBh{rH0ieP_ytl_vC54U`gWg28Gl9&Wm_CBa3G9G=!-_p^%96}6GY_Z~RV{8H z=I#tpAO*UP9O$tK<#)C4ZyXJS-Kxyiq{&R3qM2}-OAff6a7*th`!};98L}74bkAu$ z(G+~!tDF9jNfrsu0Xlcv*tL3x@jrymL5EEAot8z=r=zm@(E_4FJvEC8k$9eA6ql7yzxJ*p# zR39r#HEL*g%P}EeQHSnHvFh@_b z%Lg#V_`q(;ogxwNB3k6ypUA~?b1A7yFOH6G*YmGu>vCy-nd|JIuP?G~B4lWD$68g$ z6kp$kI&6d-u7Z&~6F=V4GodEDlhjW>0U6=-y*74tkM|`sxk5TisoYxI2}? z-a)((x2GT8-qPZRJttJMFWCAwvKUYAmkhywGvFS3YD#p=!vHEb9`}NS05lsuQO;+* z)7~8yO}CH!GOZtY$Ta>B6hZ60D@GMnD|PW7{njGCf;Iv2sy+p^Mp0rTK_j$i?)DOnJp;p!be^135! z)(N;0{K_eSVhlW)X<^xsSu!+bt%M#6JX`^%2i?tDpmnrc^+V|hC4b9$hP7D~jOpER z$L)n5?Fv*9M#4mO@WbIVpE*gCxP)(fz8pT+^Iyno@W4Oh;Dceq9{~pF@u2u8S8gWi=m<6!^MC;J9dd5`4?xR6f-Rk^4M!72W1 zG?}aoS5{3YS?k8|fPc`hKvLS*HF~T+4gS!~$3j0g4*hXfxOe*DqLQrl6tU?iW2==e7UVMnPp4)~|TZ`doX$YAx3KLv18#nSv! zRWwF{(ibp{GN;W{B3jx-htEA49{e>y(mHg2+35C_ru`QYK)U#8f$P`8@=sV(BVI6w z_&1_blb|cJIr)+{8>SA#*FE_&#Y!J<;j2G z-P5$@w)`J`bRh>fT$TFx$N&0U=bz7MM!}BqkC|zO#D)Dge(7`JxA8;bM=4fD`lpnp z;Tmk{Dc;m1Gi%iI;=Cut9=5oT@thwgl7m8SxJ5BH{N=m*I_sf1morKIj{kliLnyop3+L}trw5ws(j%)D=I6GfQRut&GdechQCuKz8@4- zP|9eby*_eM{JF+n`1+GG>fkeVF~}d@-A$jP@Z+Bj9|wEUerf)Jv=b|Qw}aa#0HllG zdC8ab$Y}pykugzs$2f&C?0;7ou)z=i#y{R?5&xHed==<0e_e=wlB#Qn-}zrR`y>St zw(lT+wmpG27TTylnpuYt_{HBtc}7Zt0VbXQvu}Za=O0SdJm`NmJO6j_Gr+&WmxfV> zn&I{ZKkq%o5A0Z%Y~Q8F5B|s8!S@z?)d{4>YX@$cFfeBQU3 z-Gcn#-Nlc7ZvAi5wCxLjeP>r{lEX*R?lX3uJ$p8VGiJ=#joUVM+m&wHrQLR&PClEJ zM=2eD)|}bo(42{QP}(kDkub1-p)9*Zesdc;bMA4%}gf9X|Mh_wy{|@y8#>jU=0Iw)yqfU#~zE085>$ zqL$QblmOc)a?!MweWjB8Q(^e{Dsu8?We`7g?1Sj4eD1&V23woFzrmCZT1E1=>jtPY zVYuY}F#;?CW|XpSV|M#rKK#k6<;@!EZ?!cqwXGz6dX3`ZJ9oEq{NhO|k|aq17XF=q zu27KnWj$L8McFIzPoN{-ILyw1U5)nA&4oao(rtK;K7NjfZY&h(u-X8yrVo-7==NXP z|8|uuAB~SJAoN?SkHA-&S^nLo&zd!x=g@63B%-D8-Q!l%`L7l@Ut=2mBNMM;vj4 zbz$XBX#57+4wYR&`~zQ@z|I(m zZnkxK(}eOSr#pZ7pY>n%Cz>cF(Mp*sCfnDXtXJkA?O%ojq9Ej&LLtRhF(X*2VkKf6 ztz_)3K!=A#{6{MVPSUVelJpe{B0y~%%28$QoyGT0!a)eWN3QI2d01A2gpo&KLa>+8 z*O^_z<$4`w8_VTZG^LTUzZKcGL8p2{13^jIm_BzxzOm0Pf6CM;;ZIKQ&O3ig3E+A+ z1pPHdTQ2pG^WcI9Kl0I!o=Z$e$JlYp64SB+5vE+^)Hz zzxUR+Wf}rh-#d7bQ6Y0Lc!`&ROBIclBIMDnr6{+mx9 z>6CttdBKS*EC;N#6cRrpz6(Z#jK0=FSo6YPq>=s!zC4p`-%t!}h_ywwFEg*aEZ{_+ z6KT6b%O6=0A{|m$BhTNM)fOAV$s{-Cx`A3_rJ9=xh6i@C5^u#xEdaU|P+k^Rv)nhCE%|d^?rhMWP4-`Jad|{jVh486QeG1%g zt{e;eQ&)Ksq74CWfpe7w=tKYwQr2@n>@fa>2@?V#aW2I}hEl3olRv6gXf#<8F?2Zc zYWuII5(?>G#UF-){z=D)or3(y{Wl@j^xyJB^tWcUltC-@5rlhd4OyO{RLdKk`)06?!#82T@9CDO7d(Iq=2J~YM zCUKmqQjx~89kW1HS}x;GMz*TuL6Xwxf{leXVtF{01*>6XhHsmT1Z4QMu-fbzHI`Ct zhI;*N+Bd6YjP_3#pO)bt|Me$KoY3KOix73Snqn~<{}(vR8lREL@h2HdHSG{*RpV-- zUK3v-;4-9Vqv0R?ltrQ>yYWBubfwo%P@#<_?2s;^LwF5a(ly6FJyHK;RpUdj+`m*q zHm*X<>IggCW9oDJXq5D>y6UR?AGjY}ZvER~N3PP3;2#f-?7u-Gzw-||!N2gL3pd(m zlYt$5V7I@n<~~- zSQz4}>Ay^@aD6Eu(mTxghU5VM!Z&=Vi9LvTR{T=&Cv*}6JaOT3zd)gr zk-GTvT~+XFe?`^DZoTLvCth*fI%Ri`#f8fyi z$GdR!UHtEq5CzcEE?r*ui3iqO{|xeP{iA!W$HGRjrZY7I_`uOwz~?p;)?pIS#{b11 zHncudQX;s-fZ_Nl|Fa9jljR@XDgV*_FaIpzf0PgS2Y(xH`oHtfB4ckbew=Lszu3nA z86;Z%HNl6$?)LN#t4;i{S^ls1FTa3a{85_tEPVX0H(sWF%m2jB0Ds|hIBH!QV+;;M zyUo~DtH>DHf3-FKFO#VPW%lnLf%JOZ>J^Om8yGvE5fHHv$Im)vB>p2gEvL%Vsnb|8 ze$Jc6T@StW)>~)BjNK7VEP7z=pFC-NawU)Um`vAZi(HnjJ@d?S+rMEuUU5q18@Ai- z{U7|mTMpQt%Rdi1{NTR(?9E$^B25KBT+adz6n?gJ$v~OH9eyM!G{PIra?<}qQ&o(k zf-mLT5j6YXd3W8ZQStwjDO3Ij``=J*=oNX`)&7^OGN1V5vGD|pp@ZLg$cW>^3?x*y zY+!uWQth!oLv*set#OUMw`9rV2WoZum%;sS_ItE-GyWqMi-V$al77)KF~+vY*haql zYlA4>>-Ua7?l|u5D|?~-Q1ZuT7RPnyjgg{1t>bj=3+`L+lN)a={s$j?5ST@Mv&}Yp z?dHB_9=$i*@RNrhe#o7qrS4YC!VN*Ny4f1Pv7}A zkE$78{r*u$y(j#mgbHcjQTW8eAikole`h6qGXaXvVm=@V#BxN~)kN5EW_CqUmt}dhvmXzN(7qM!HdvxptjhHKQ%)lqI$u zRX|g@PFy^+!Fp3<5XpIW++qAnIznbE&*RzqA9xTe&prD*ukm~2(MKYrDDQ0h`>*`{ z+Uu?ho^=gQ;)*RQk6Cc^y+T`}zZ-Y1w=l|^~r>wHlN)p5cR_K5GpZ zcL50P|GK2F5(FxK&_WenufS=lBAM8eXOgj+_HF#r^4AO!NIJtbD*mIe&n?O$;~$?i z@ej#6@4PD&`DDiA?RVJ0a0NF08OCod0l(@}h4^1I=EYJ0V~p2i(=a|8^XK2o5^+w>k&HrRtUL+!41!W8pFkiC&lAo9;h|*&{{#F0Y zD&{5L+~=hK4&c*j{NvRmeE&6Z+Qbjq5FLyk8E}{5%$d7S-yH?zNs#rOq2_Sh@;AgP z7-W{j3l1uZ;8yWD8oxKhWjFAN(xy&+GF~~j>mMzneusg6w*3#EGONZn_o?B-LvUUE zShuU-s%=4KhuOu8@0d3a0|1(|%H*APW{7nn{>OlcA#cJ3iZy53N>DKKHPs7JorE{+HmUB z;(y2Oo&%ugh{KPl>R}Y1!2mIQ9!Jqc!jn{v9%wLkt`=s~Yd+(KAwG!{$ zn0M!$=klh&$A5RpB^OuwHb$kozQyw>j(L*FmnKnq{dL#7r6Us9G-m9oJZhKiL@8e{ za{O$%RR~v;XueYnWOB}0m zozXnUR%Veuh;J{nVnrRg)4iA-Xz{t1Azih@mmW7YyvVmi2kxbYP-Rgu$jk7P0);{3!j?HUt_U;NI)7v+Cne?kb;8=XGJ z>iCE7M-6`+e~fJuIjRsP7M`}lYS*;;UpT}-dh;+oZdzm*aLaz;i-D;1{~-TDPWv&M zWwt%|?6nM%=3K^Ci+J+i;A?WVZ~1@FKZE{nCYT2J!@i3jLT!})TmPUKQ;b8JsQBAf z;XjC9@l*bxU!;pP@neIeec2xEe;c&HulOJ2Pk~c}KGFh&T}*t#)LN<9F8X6QL_(6? zTtuMk7;Aq@N^rZGMzetd;{8gzH7ctmWu^j#jUw&%Lp#UcG#{7|nSamxi!btu@wl;L zIsQa?!2=IHpZibOpE6ar(kx6+OJ_@H)cDB|Rpb8`zxaju_uUJF(@#5%S3X&}*W>QY zefHg(SFUp`c-cRHKW>$U<^*5HQD+XTtyj#o0}y)+e%b{{t5sDMH1-bfe{nT`t+B)6 zch0-B^iX%b_16DW_rD1M7y>_Z&u{1d{AWLBAo<t83 z53%m}L2G(ep{Y6&uAj_Iwzvq)~c#j69d|S`o6} zaq#io?-AA}N(^htJ#f*ILl1rHttbDA5*`%4?9xj=^5KtA)G$evf;1~WjC+O-JM1u~ z27uwh;3XGd`2P2QApJ}JJl=fnx#!|WHg+PcH{F=JY3V{$hVX;CYpi?ll0CAG*YL+? z6qSFxFu_*SBGPDp)Inm1u(&dn%Q~AR)=xU&Bqn`7$6(3#i+@OX&mU5v z(O#19(PNlwu*EtS$PZSD|G8I)^qK5+bwlwCWvi0PZ6uj6tTnplc-S@Ii9?1;hB)Oj014_EmCZ{6u(E{G<|f+xRbh zbUps~V^8EYw%AT?;UA9AYM+>c53^?qE^^u@5nWOGKx}&@S->k0hp0qZw-a51lF`w$Q6NNCwD6;x z)%;SqR-zG<5^S?e7R3XV+%b5hg?gJ$!Z~gU{$T$-?X**aKD5==TU&o<=+3+574fOl zrVR8irAU}Y>0tj2eo8B_d%Y*0_+37F-P?52O?Vp_$JwZV?)m5b{2lMO@<%_q?6OO_ zVQ1Ud^BQ)HO?;wPp4x7LhYr(k@Z}t*wk(PVvR9UkIRojs>#q+)dBQdtD0H3kX|#Aq z%lO}%pn8LKT`=c?5}Fxjagl~y>mp|o1W0&^1w|amefr$bkB*%B+IPrixbaFwd^`dl zF*Q<~qbE;eC}sc1(85PmgRhUPNog%4nSamcQap9U(e_R8bHwTVFXLP0B#J*+%ZwST zty~0Z_!mC((1e$A-m{Ucrbp0eW_TTYr#V~-Wv%_G(ds6-Xh^o#>lyI?3ZzN5mm z)XFxCpcyuc@4x^4pZw%T-p$E#)5v`M<9{=L{5Tqsmsu3R2J5X~fGnlzkDg56NlG~R z!H1QSwZCz#p2-fjciw5|k9~}7tDhJoC#se-san{~=UX*8V~iFd$IP zBp<%3IA_)ywwt9P@qn;`Y4F->ul2|oK@1`CN4x&{6zFGul_X5rHiR;gZI z;y@C69>1~eE~2y|W2$ZaW35`M+5Ey+s6xkFRo7-1$iv9?*+jcq>d7kAH6nRK%y%#o z_Ze<2^gR^$vp|gm1FLGIyBYJWSWl@+tR;T5RPWhATdCmN(FRhBzuSx%C(WG61j-=) zu4YxRKFU0;PWM0&tlY7*1xAz{9pyqPk|{TUSuwWd=su<6FAimoJ@%ND(z+|J%*%E@Y%0WDTzuh0Rco`&H``?6jfF1$Lqpsr za`MS1XCHx2dBTyxI`=>4{c@V6xljL9cwoAh0f6#x8 zP5l@Af%t(ss1~3)_>Iy~Do%+%aaIM3Aej1*bfbV00wvd2`cPw$k|GauDf>oS}CIuiOU)-ig>7o$9TiY+qsv7Qu59}?|cyg z3DF;O-}U|XS!ZbIsH2azz@4_W{of{Q%0K79_K27DPaC=Nj}hW@{Glpj7C6auOI@k` zR<$p;cyQC&%;Qdwmul6|4{`(T+inHMjU)k;wZ@Fk=pY6mpoEhg0B$Am8&k_AsL{e-Rz%PA;&$dCXz95=fDn?i*L{eR-jlZc|@+lMygM3oWzvt_-; zU&TL>nc;Zk_=82@XM7G0U5oKSV*?uhb5GE0(;Oa}29@%J*a-YmXi2MD2E|VACwIU< zUHpt1KO1XW5PcFC!GGSoJTe+o9wh~yA~X*A2UFyrTh+~fUyX>~xK`Al)sR{tU--ll zA7qJ_5F(ZiX4&;$fC87n!t!DK(edxDs62x|-2)Xqs^wK>`DO7+;%BM;0Zlt%syOtD zPE!kB8JbKAe`~eNB;ta97{HvT#L?62noI{mNw->(RwLcSI<@$bIy1=iuo>zrbZRfqA}U~H+& zWyAh&Y$3zE_;fJ-oBKS4MKZ;xi$+e6$Qw-6G?_TanY?(-Ob4gd?APU!!xrwj@KPd^ z1AZ@qdYnSK-^`mnC~{VShPUmlRj0T(LtB}}XwD=Xd`}uX{KSv-8+@P6r7?%`!@!#X zI(!2sPDCjd!k_cV*c;Uj$WA>B@((^|OU#$ZKOBiq`4@iZ`!WuXiun`>|20#yU-V2q zJ!8N>w6oXytBqnmo52S`{;1G>;+VEteYrDsSgriSW~Ko24-&y2@PEsn$BH`q_ISi zI@y8+_hZm5JMYYsnBw%fy!T{JhjqvvPXa4+qx%7=2_Iu+SU*w%n z{QEsi(^gw;$*_j`_uPH=@c4!n_(^dDzA;a>3xC3)s73GV| zR>|xLj9kc%{6YV_5upyh^MB=o)ZrsKh+iIUG&SyZi+L0~PH|}CCkBo74|mx5Snb{)rDeqht))?lY#(ojYgFtXaIPCU$kKhw;sish$7R z5$kF?D3(i>PYqZ z#OJ!(ZMU8B{r;`D{OUKq`As0{h_sZ}6u#e;WKmd~hAnSfSFe|0QfrYNmaH@A)Z)QXoqBCnCkK zf3N8HIren@ch-4l&idBZu0H>3OUBQi)PE~JC5A=y!;(Y+$s^^80=$;MBok2vU2CSYrhUmU1xpLvAMD+-+1Y zc_A4QLxCC@B$C06ou2KHbiWM|Bo;C-yuAHcGWXtluYy>$c=5CiH&QzJNKvZkz1|*s z?$yThCwcw^UAbyL^xmV7zUrE*JNyVY_zWb!{N*h~o*2EMw{jN2!({02ZW58#Uw?z` zx8Gi4`u$(t@~dC}`qyMer?eN*MU0MEgnT)5s5fcS!c%9Q5&@mvIG9{1ofI(b7#-l zeUIIvibA~&j`O}K^3Oy*srYwsV@{>uOF>cpSyvn>89(hVi%g1w6<`+q{#|$7#g9mf zht^$pT}nHRC2zCTVfoVf7cNUx0q?ozo}c~v=l}KdpU<5=`>S95>S6CXY>z$n*8Uc04PxMFC((tSQhHfJRSN{nlLM2Hgke!^UZ35MZmOIp)k^nqzv^?c&FP8EDM{?C=2%7S}DkwY>Nb#t)_#wULxSlo;KL%&~Lw?-gwO+<&5F$%n)3|rvGylJS{$GR>Q#fDy z+L?zRe)t}H>_IG5|J|3naF05w(MuIun}sH2)KB*bP2#iSAKE0eRPZEhT6c}KJmnp;_!loWqBcvt;Vue$R6}R%&Oh73MJ4i z|J-`3FCtxTxrsaNyt5|&FTB7Q$Va2sTYr5n7J#J8ys@xy;;<~* zm;xCySn30?%D1+7@>~!FL%Z#|+cno*zwpaHxE@CuJn+CzhC0EUTuX-WTmPF0 z11Fc_f9#6e<&S9=D?m%dKXPI?VrSR;!j5ZfK50V5XP9D}#ndSN<)5_67y^LM#)QIW zlMBHPwy*OSyEV~tttr+0KIwh?x_9mai9M5 z#O0P>j`trfzdTdAtbF1V$LdL5a`(`~RlTYt(j7GmRPf-liGZLw$@Q$dP7vA~UBemX zE1A^E)Cx=a(sE!eUc>+0Q$`rmQy78tPzxmB?`PebX z{NvZY`paM4^7+6099D;?x4O{uwf`-~ED6cQi}LjiK;p@cyhbHOPdwnv;E3aAJr?t; zeXIY1Flaj}Oog5Tw~|tj1vRSS9AFq}TQ%v?6`#}?^iOFa^Te4mnZ)8==~5b)=J=T$ z&{-nf=)%d;t6Y_qK$<1dXt6#hncb&PpTp}tX3ycJyw$!PCKZc=(#k{AT}Bf>?sw2a zTc;qN!aoiNW|8FJkPrqTq+;_Z%lPJqUThq7O?NXXCayhi%CxDY@q>$}xt)inQx81w zz)u}}>~(YJ{>A&=SNOQ=V;}pgr}-jmBHQEJ(81-LNXkidW0spZk@H9YUCpKHtFF3| z|5vZ{s#k@y_Fb%S$KNny3~Ccp+0Isf%n?T(p{{H*`NH`lt9Qhfd$ z%ft^5#829{!LN1lNt~C?JbvbGGp5T;f+p!1imF$Q%VB)B2l-R@$Kz87K}dzqPXQ0& zb8bU<1p7>iE1lHg2Ovy~gxj`7<4QCATNReTCp-U2W4MX{EIEx4mJ}Q7ZivBz3e=s#Z9LhhEU^C+nE8 zDeEX5*?$9Ei$d-k;N>-_{{PN4~yYGH{ zK55F7DQ&Hx4W?}Hi(l{!@BaPw-N$QjghRO9e+j{6ysB$z+l-{NFK_-MoBPIBSYZW? zWFuTV@rNFINDc%3L0!{95Dyd3`C0#Wl|Y zCMRkSAQSa#6=#*4eG*>W*v!#71w7f&oHx^e(Z~rMcjkMoJ1ayv&!+GaM^-pmB#(1i zkD+t!h%TZC`(9eS=%Js^eeR(Jv8_L0l}W3=Zr8CZt_Y3PLYMLR+#~lr@*i_v{QdL& zvE$d?cDLo$oeC_vo_+YwC+L)vLKl1UhiWH0%{WhoJ;9k4)g(Onz{u zfO9hg_)GB*u57KwME9%k07A;3>ghyR+IP#z6VBLS^-uiULj0rq6_7vv;U){6U>9)| z^Hz@L*8L_FEK!<*G(Y8Ra(v)rSu)*Jgz740!{sfHJ?jriVqV1M{~0?>+I;c^q;W-1 zsh~Is3vuLN`(}2^xfd~*?JAvVU-F!eXz(cqn-)<{5^@8z^$&%z81{d~Pi{N`gXg52 z?a}|#i+{*nYJ8%#!w1qQkBmBvX9Z_ULtjLvuBU>ebN+_Ya(NLT-figy2>gSTyO!KYpk&* zUn}>e=1KGyFp!$&ix@G1n=kI6!@v}8j*D2lnunHNwB*P4@`_uQoWq*SwnsgYw$Bke1_b>1swBFtK+{5`cpUy)O zNyRb0TgNykshVaCAJNK^Uhgm7bM%>Ko=HBZdX<|}imP>p{U7+DiQ~st8U5>Co+cm- z;1?(>6awPg^WI%QWa#nT~RNyBctPmK#l4Q=E&Ih z0#RX&bR#i{k54#INHho?@WS!Xg{a~chb)w3Q@p2B2@ zZER41BXYUFV=-oE+5WPpoc6c3-0};)X^g;v1rMBe{`nvN;D_yi3fHjTf9b`SzIW-R zWefNTYmpIxn-)X|nZoiGO@!v;GNP)e+bp4vDW*Yd@$XRT*}&Exu`5@S3tFw*&tjcX>GHF*A<=e#RQ5W}tmhq@(z;m5nZlHyS`iGRQ131S#LDF!RLS zrtca_DjA9*NYnprHf>zQV}u|!12Qa!@pIIIzIdtC+r}q{FJh-|!z$d064}}6Pnj~c zV9_vt{`>-0i`+eCtNx{pO_^W_8>F&!Lmyb$Aa$~s#2hUzKK#@tKg9=$%Y=eS{74TC znjAJ33Cdzb{{?&)pY88$FmdmkC!|VOHpg>7BJv^T71|) zvWGAkjOleuR#2){JHPS2Rg>EIsqu}PG7N2+DL_pSQdwC3_(}hrHf zj3EM~NXg`<|C$#+#V_207fh8@>Qu8y?)dTJk2&U;Pk;K;UYi=(f9V3^f^J~eYC3rM zlS?Z8$aM<9@A&`W{#*P-woexT9=QMjKmbWZK~!n5rkiQH&Vl zXZ{?>05!AqKh3Cj`u&f7_@i&x|1IOjkEg4yv-VoLTx%R)1H;F&HewB+yYIf6Pb#`D zmJV5Z|E0TOEdyT?HS^&E(N3oM(rP|ft;ez?n)a!++LLVj_Vzt zIj3@-K^5#DIn{AxsJGW%dq`cQ148Y-lfPUkW5t3}EipDdx%7O@d|Hz?V|XydIU!5# z*CdzwBN(xLTL-rfq#Fd^&WVFyA3e+r3^9+(X(XHY+P33XLdzEOmOC%KnbUvlmYeUp z;v!YT5P+fH!e89D_4_|Fv@Cs`lia)mtv~eGZ9jeRXEz{(_GKqb++f{}I1~k8;T=DJ z^p`gRioPdq`>%DT@1K#)fJ7a$9xpt;@Q%yQetN;(9X>7b;?qw)^YDVlZvBtDXJ5M6 zpMP+V-xxdGYM`LzP)Xve{c$J;=C9BY47Zaw5-}s zB2kfX-k|DlvHEhS?zH+RZ{iZS4e6d({NCw1t--J2X{l-;U7nmUO$$~*7a>)eWphqg zqNzsq{y)0ki*L4+H^@Nf^qp7po{sE^qnSK83TX!-x#2ikzqEwsobdFcw6<>?UhPW+ z*~)tQ9;bidj7pR;oU^oa-NFZ|_=i=Tf%UymKT(uP}3-e}AC zUp*C{5WDY&%b$Pb0ZvfIO`1Gq`XSs`LvYc8_h0veCm)(m11>XuxwUuLYq>So_v}YX z-m^=hO`m=2!Qb6;=W`F<{oEr9o_X}aWtSbd%*v~-wEiZmZ?(h3No!+p-p7G|a+fkH zttNk0hnAs7uHQ9-hO2jfR)dO)Cma0|R~LfW^BAB-lxU{v?nH2oJnv#*)G$6kB# zY>G@r_+RIQN{mQ+VOF(MMFCW}_wbrov%-q{h@*wS_?>MM%`-H#(#k9EwfEkvD8y1_ z{!)WP4erBOYpt~&eDJ|L=H0;qeOvS8RM;2<0Lnr!QURm^<7DxxXkA|UnH>$coxbmV z?~Ay|^*3t&nEi{m8G!^QfkQj0%nj%nHRNRANhW+=!TvX9${H`!@l^vkX3W$LHvG-) zzX6)Zb}qg2dvAN&+w@tAYyYLyBmJYBvj2sQd~_bMAW!{M#}7q5lF1*v|K%FzI_s`? z|NZv@Ue86Ae;VC!ao2zIS)5`ql>5g~=+L3!f#YWlw4rGxew+R8%kj4zTM)fl9qb8RIw%zdI;MST76`cy20b;U76-M zfoYEE)TA18SQLI~c6iS5RYii7fr&?G;R2+F9CCig%!T`)icE^bnByP0)r>rbwKWZ@&4>mRJ7qn@t+YJiPyk6@Hru;9jjyJ^>5cV?^Msj$wB zrzAk42v+ZU=etfj{Zuwz`2F)tsl4m(!!Nk-!ouHu#vW6qPSxa86)H7ZNIC-^-jb(o zGum|?#g5yG%2H$Tu|o@l%SI*jlH*%?u876Is!+>%{~!HVfHr=NsPd<7C}<it# ze_6qCB3V7aRaKe$M?oWR5lU;V*O_Qjia_ukv>4fP68Qci#Qwi(5ru5 zNQg8I;C*S_8JY$7DwX(&F=*9ATRH4(=Wh%ayIDQ<%?#3H$ z%uMs1d+jxK$_6QH{G+ID(U+C@+!QN~>S6!Kso`Jasb=u!zhV9X4HtC4Do$WoS;SBI zhwX;{pnufb{IS~7iCr?|c(D4HWsH%^wpyxvrIPj~o0Z1@648ustjoS#XF6yiCCvFel)NQ>Z|Fka| zON*cIUZVeICs7<=2RH8GZzncXYf?k>6?Jl&mWO!x-&0xi_qKh*wtMcer>$H`$Pw{B zZR!*UgT=k?e)qe4#=B~zTUr6(IJD&tdJr<5SS{aM=hbV^zvrGcR$rZuZKZI;_~ZB% z^$l`pa8j2M$c;jwB9j_>?zsm~?X#XnNa>iqZ>mQx$%S;vYBIS{#!bN%brs?q5X5IM z?In7QV;{^pse?u>8TN3hY#AFlWz%nzNY!0CJ)iqNayG&P0n}SI?_ddvp z8XJrSN}$3{?9rcJd(V{@Eq?KZ^t6;Ex-b(NbiMTCT)Eok|T~gOqoHRkk{T-+7xaP4pEfg=` zo5ji2SCXYio&KOdKl!zk2tkabfbxm*EZMC1DVeVSdR<+$Y2RM|)SV`C3yCt}B(fSS zMP1q#P57+ZxAQ+OM&_V@td`@4auV~0#z;BWr=uX%`UkEypvp%1f8ND^|NN7WbB@K$ zL=->x@3S{L;JvH8ZWlFCDe(!+g4vf9O9Em29ro*wS)t-{{&oM-=eFy(cH&B_Fxi1? zTK+r_)qm)wtL~e1S?|R~gk!>!2cLS2pLp!4d*|JM!;jY3X8N?Z90n;GS;qc(zQ^}} z%GpebE#o6zav9OS#y?q2{BSD=cX}w6_}43;G%9{h@^$s;QpPqQzijGelf6PKrcRwg4x`0ye*K#WkQ)}st?D9FVw`AK z$xt--+3iFEXGgumRPdXM$R+-*6=Z(U%DS`$@C!cq$TkAoY`gVuZu2>DBI=Y=PC4|g zhj#o6fhCFl75owV-xeG;9$%HRSHAy!&8C~)&!=j7{k!hG^NA;(;K4~)=*uioTKFuv zDFBs&%f3`4L$jH$3M2PLMNSYVi`+y-|-pRSaiCKl~=&?GhLfKNi1i{HN|x`!6Np^B4=elH!f;B%KCO zrIpFy&mTKLa^fIXQywvEc>2_4f|81=K+eOsDvDvjcbk=92&I=bD;0uNoY=M!B{@{e zk%>$)`9*%iDZpu|I8g2|D{@i@f_9by&W(EsYYsjTTufMRNLaf8B0Yqr=C4Rg4w(=! zn~xjw#tmPYk+*)Z@8L%1)U-A_FS_s|f1--8US=6CNdRG8VYr}W2kKQ=BZVIb>s@x) z1?yC1vZb?%8TbpZLV@{@V)u{?SJt&80G`f&)H>Wu=v( zn1Tnz&xWwxAX`jZK)a&A(=bpxo8^<&H)SrLvd>>thQ&h_~D)_}bk6&HW zKP<{>9qg%u^xykl@0=vxt;CPHl-an-kQzJsJ(MaHr*TVL(@eont*WmGe+w2UAVQx{ zWAeu7VgBlB43C08z5j|zh&Pt95;bM25ooj_!hfx@iCN0r+$1Vw3xUuPK-#954L;J? z&cqoj*IjpQ=YJ}cC(AzrNa4}2YTA$a5_6-*hH%xt@DV?Bn&wk=N!oD)rqmxdlB86b zZG6Z#wpIT{N`3k#wGgLr!B~`w2o@LqI_s>>oAY_$17e&!8yHCBuuPEQ_yjHex5aPW zg(VHa_a1fhS!bUm{}8D75n-@@#X0EW&2}l01uyfDON@WFNAf4*G@VskQ;*+(e}sex zh|-->QqmyZ$NlNZF3IP8^N&yYcq2 z-Z1-Yz9k99z`h_SiS^MmFw%)4SRe#9RFw*|*uuwFe zbD>4C#3e|M8g;oqWQh%+dD~=$5NF#zd$SfTCxrbgaw(?4Bcj^;(|POn5LB+;F;pY+ zx^fO|y&#_?y6$`G#(jw1a-rQW&}CfWGf?jMz$ak@>L)4)&*-Aa@!q}zNuQ+q?L(uL zn^Hyp>v=Qr=h@Q+3(9BR&iL%6+(Yyn<{tP>o6;L`nMeVgWd@zipjpB&MSTIb;QydN zB-0xyuO3Z8)E2DnjJ2}CmE|McGRj)#e%puq{GwE(*&v{^64zW87AMc9@1HwYxm@M# zOLwYuUcV9z5dt$#Ah|~|rtk?f9d5;}_J)(52q57PYt{figzJU~gKbZa&;IiN6b^Ax zh$+#@_rwQcQmtx>{ZsiH=*wTwcBVW|`eDiE0F=b7Uh&yT*q`ZJ+*=DC9>G(oZ27rG z8BQic70VEWBf&>h$dhx>&TcGdivfgyQUJf>=T_uhrU2J_vG)G~!$GZ~lrKJnZaxn1 zfdm{!=l1kXGzl!Rf@*CipzYMdtk-?H?dX3*&9cL+{WsH3%MED6MH~8q5%{--Juj13 z0-;GI|C5ks%KJ6V>+LKvXyWTq9-y=yoX5a6HeT@+0jX=K6a#N^tSZfDGr1O7Q-tF_ zw{&3f3c9ny?>`4$pFsq3(}Di-rRqJ3h>R9ecU9K<9U4^4)b*j6=Fq&;W!Ys&>zc)1 zo6pNzAKxY}{QEbuRWUWEPU9|0E@LIS6Kv1bdAk6-{^0sm>36Q{AjM#G(FF}`JINUK z$%I1}W~2zYxZSuuFK}vy7ayMI`fap5cH07fkYCH32&SSg!}m~VoU<@`!9@8*U0YDd ze|Ue~1$S5oNH*CN`H?-u$h~O6&$;~O%qzDH&y^CJn(qwd&e03y=Q)0-Jw+tVPT;3I zFk;N|_tEVFx)$&xCO$6~P|`SWZ~m=^-=7R*+GE5^epr8)Xl9Zpa?AECw<+akU(N6B z-y9?Drc0)FVqfu12616R&nwB91?r5-H`)H>(y(kF5Z?fsnqk$cEI)~f!$k#>A}SHb zoQZ8yXysg)!_&7E4@XPUP+NbDmY1yheK=4W8d}kUgp4vtYOs*PF9NR!WbuUC)K}r- z`j56+g`Jno6b+vuD94BSrG68VuRUa)BLAGZD=scKA^q3x=tyRL@b(t4@IZwEflnikh~4G}vYlO`WJKyeuO& z3wJJqyd!4 zNbP+syM7ZC;x85#nM(vlSV9O=wy|JNA)mu*%rK(l9ED+J%H~|V6qmi~F~4i|9*Ztp zR9Mors+=x6VAh3c?>vl@-$OgtyYJUkJ=A9j!k%WK4E+qv$~wr$8l{2oB1cYSeh@`) zRl~W&^lHag&1m`urzI2}N)oYwT}PyodqUiP0%}4jG>k@J0(d~HKD8k8>@%kwZh##^ zd5k9%ReHBa>l_bi)YG;wU?TWu)O5ieH@OX-gW>B0*I-DrIQ+Ps*+Gxg4S_NJNCSs* zMbh6{m3Ka69KBBUuenS3P4U;|F%m1asNZ@K#K~h{>VO&*37NfQdF11R0wwD{9(78d ztF7{Mp?;3v2*`3MK<5LJky=dIET z*RpStI{2?ztW~JU$;kC1!nt)?;o|)RM*{`%6Fe83eZiefrljY;;qv6+ch+zpBf3#K zLwgY-7g{@P+;#53i1oO~2P#qum9tGcID}TI^e#UT_AoDI3R`=26-e*#n3sPs4YuE9 z>Z8HX@}UIQUe+6xW9F#dFKd4D-g@{wi|s3`_}{=07bit^3woDkcD*2@28O2%WlvL8ir`yW-< zS$dEBn~WWJ1yeRaG?}_I2(-kCRdv4T2GZyZits9M682#x{X+m;p-s-9ZJFRgdB+Ra z2SC-|5Pp|_arGk&mP>`P%1r$J7l0D*z!~?W_C`E|2*9UX0ki#O^Xn=q*T=W#lfDOX z2q2ERHE0>99zlqhGSW0XccTd(KD| zb?c$^&adHVW%2ye=K+1~NY$GL0xir4(L}d*1!Wn!JM1T(;@B>hAr(J)oa~2+hxK15 zv1X-GmG2ooK;G(8K3}5?b#j z1P$b75G~5S>y7f>G0S^iE6iy8%;dj1E<}-?aq$zX3&i zC>*gcJaItkrcb4BVoCR`Vi_dZJrHAY4nH|QX;S+))>6}~CMIe!Ut@3*>TI*?8Zvh2 z_>Tc`z(x7tr1rFVuuJ4(B9~(%?&|$sEW`*&U-a8kBl4NeS8*!)1mq;8t+@iRUg>V9 zqg*8BOvDcEQ!uGlN#M~1(rNHqFvtH&`C}>kU*AU4)7|P1dkSahtlf5j*kc?g`m0L zc4RtrE-_Xzk54RdVt)`nmF&15&sk!>s~lRv6#1?5ctE&wsXY|08HSSSq~CN9?BOoj zd%_x#39_vkWn0dlWaD4_zADtKe8>3+5f5CD$AV=T<^R?C=kai+i_I*}DMiNR-mCjk zU+m6O{p)l1E>8Cp0_RUzh$&Z@RhX5wjHQ(+h=eM+^aE*H<(e2vWW7y&!hazy)>p)) zA&!z&l+}9kuO2awdmpgwP;W5 zu24q0W>|rV+yi{O4kBdNkR z!?cCHK=W5gUW_18rhqdup4d9WppG2O#l0~|%)J?0ik19q9X77ws_uk^DPw0;>HW6C z$f0F-m=KltvzIXt>da}_G9$9u$$K;Ba%@|&ojf-qPL@lCa$x~q+J_R7UIU`yBUL${ z)vBf_mcvNhQ!=~0HIND{yNVtR&b_00F{6<1zqP(x6ea_%%VCqhD8b*vPvn`Ae!H1# zc!_dch;6=N!=Lq^)pSrzvEw)XJIqCuS+s%$)o$Y6YBAXNrO+XjU#zS_hp^QzZ@51w z>s5=ialUJC|@OLk2oM{S1pTWzagx_b4YL`r-3 z>mRz&nt+LKd$5i`zeVj|Y;L$u49ZaNja7-VyNpGLedm+h)}iRynSCNKEFR0DfgYh0 zdk)^y3r*S_&XsfJbb6XixgHZx5$3jjCLF&)_{w(8etwhq0W6Roy1DnkUpuJ1Wj*lm zZuRVPkpX4@rV`}8mwb@r&g(vkZZIA+x$h(IcjEfU)2VYEsSJPJbl-P7ca4*0`yO-V zKFpQCGE6h7hVFO0sg6HOo_$};_G0=4l9{^x6>$puzzR)bCK$=^m=Km~9a6wv4K;mh zHu227nv5bKK1yYPojG(ePUV(5bwpg7MMDb9sztWcRox}>5GTsV_Q%Uk7ACHB*}Rc7 zSFIpsUQ`|p;32L(0-6?XDMB%EDg8)c{h{?nf$ebGT&hl$(xx$|JA{+nxp&CFX05=e)KmD*3eVlE2Be@*|=Sto<3kNLgizo93h9)lt zuG6pj!vs%GeKyf>vk#w#00;EvbNF;G4zg$uM%O(rJGfY-iC(ArL9zuUT`WQAE!xcZ z7{9O5q$b@)iIQAnr#PhA&J<7kY(i_nQQX2Jqr%j7$CNK0FZq4rUaDYyikjL z_pA4ergs3PgShYk`(E+V8w^*-^&Ama)i0mmMT`1BjGx6xx;GR6x>iy;E6&Huvb0LRNd^X1Qg@4 zR&^@kL;dW_@ZLFT=C_>OtXFUGNM-*=o9IlHo=^=lpZ$2!?{TccRVeGfK*b=9f=`0B zonz<8of^=RCo}#b+&9S6$&J84+Y&(gR7tfM3VjUY=}F#ypz}pNLR!#TT;{)v40jvW z&N*|8t~G~fq{sN)2@v8*Lq^4*^+~VLB|>^s$yeZc{WP9%W#(IZ!zN|)R$zXy9C;pR zld)B)U6GHC#hctn^Vd`gAe`MNNIjtD8w{N_j@C)=&7p6QV-VYt4Re504}f~QnTuMKEyO96%XhVZ{&Rffl6hh|8<7mJQ#N zia>v|B}8U@4{6Rs3Q&F*%STMF%jU;C@JYPkJay35lNOreb&;T?ydF|)m2}I*I)W5QFMLM5+ z&`>4#i)$Ba zl|zr**;LM6Qygs7Q&vvTBr(BAC4{;C|AFq8y#Uez#t5=or%?e#dLQiCf!yRqAEgiw z9@`+>jBp-J>9mSu2JH7%1Pt=@d25U%S!zkeo9720I?^RoCU1lL0$i$-AfhR(B*6Tz z$M^1)W8|NGm^hnmwfk)jW#9Mzh$sywGB9P5BmM{&VwQz{*Hz48RdtpZm;ag~;~hOn zAwEDDS}D4})O)JR!L$&=B&0NIFczr8t7i>sQDKa{Wr=&=zKMK>o`Oc1f}hTs+z)a* z7M63R?hJBfP-_PO)b<|=0#H%1zBpWveQ^md(5^zjUWDm{1v)*S$?qy9V2T%iz{$z= zPN9}n!d37i$9uGnj7{h-_%aI6cfRT!q@mgO(2;7QMn=3T{4t8ge+cjBI}$W&ywT78 zOo6PV4T!e z`!_R%s+R4?N3N}Z)TDbPI`!jcmknA_xr9;@9fMah9R67>*`%p>C-TlO!DZ5z5V@wo z?mwGaZ-QZ}dtg>1BWfoW;FNB4vdngog*a!Uz+8oT6|ot8>Oa~RR0mX%8O>|hzEuN{ z`SO?yTIeg>@}0B(6hBHo$bF)99#_z|{N?EZyT0hU&pgPbXhqsp)buzeRZ_e!OCCI= z4CND@MbrH*NzS)RnyTG|gm*da8I)}jxtpBKn1w6f+F_e%K6>6EF%W<9c7s4B`mdK? zX_t&q3cUo(L;8mQY%gf3i8NVkbNBGQZx@N2&%R?zX-%%b1i{CEX8t4cDHQ4CkeL*~ zyGf}>hg&|y*LWj;+TF`z`_)a!3k@({t~_*oYJ;%1p(`r1Ll}GE_0zY_s15f?#C0@V zf&J}B1$5r)N1_27atyxHQX(%YAPcYrS_V*kshjkhZo(7iUE=9TBUvgL!;3lbY}%VLF!XU(&4c9|)0hqQDqd(deL{;Rit?lfYY#HN(~B|3*MIr>(bM z;w~|c_mm?HWP~81X@3*h1--XOKA)f3?1xe$vFiL7>W2)09#k_`APyds#@lj)GPJc; z^%uycIZX=b>}B++s9kHF_3GBKJ`7% zx3Fltz7999q*8_a6-92{Wi|2<{R(t+*bX=5PpjfY4ZlP7{5OTOGqn8to6HTnhwX%? zLR2X~OUz=kGidta0(>40%f*B~MuBQLoB1%x%Xp-n-VOiv-3*Ha3iLf1@CzK5gqK!d z;|1wo)ZTv!5A@!<72Vz|9E+v{&q&OQNbh!KcHPe=wXFDTOQ3JlUlGKOeCEV*3|&6} z4A!!$6sXMt0Vre~`5hW}HV@te1m)_vcGxALTyHO=Sc+58jv_ z{7a{~k6L9kTq{&+b0TmZOn&`{|C&80pE_|Y!kYE1e@gA9YGW>%%wlaS7{#|-)BJEu z7WdZ2>J|f+?)%xAy}AT^_do9wo8-?Xffl#amu+pqd?q>20lw(ebH=7V8|1adIKCTD>Jrrx4oR3AAZbRxW@Y!#|bC9wBp03a

    k>*qQp5M z$}nx47>QONU!iAB|9Y;T{k1(Su31Lg*NOIT)%d#7JWpc-5O?tplg;+WT!8;p@LG!4u=W#?aSm$geL>UkT;}rMGJy(@xLaIE zELh%$V*H#}{;O`7U{ZsTr&5q8xAl)@)C-V+`BY<~r8QE4655V8hZ&L6N6!kHR5ysd(F`t9N ztOm~Yz2YS2ou%%B%7wYbe9^0Ss#+>I5aeoJuHlf@{D>%R9H32srxYXwE-u9*!Pu`!J+Fb zN*TG2RDaOBi-ZN3_a~kA{p3vDaw?|(_+;2%4M4(n8mlSVhwIX{d!c%#7YIP$ zvK0g#>(wt${Rv8QznXfXNDguZeJ=FA*z~leiARyKXmD$7lZEM*hMi2- z=!RghKP8p$^Ppme{vqk3fuijW3-Pj#hKcGQa^@V^+yTfj8-4Qb=?%VhQtJP=BY8mdd_a9@%*i_5-!|h`}_PV zZ3Z+hBD}zlH-i7Zy#C>x=6}oRd%q42xTb-q`I~t|Nv}h!0uw+X&c21)^d=UrH0qsAkRl zi)1A#$@e6mEcMnZ8tEzT;>?st-eu(8Hk(#u(qec};`6c2e_L5_EN>|6?6VGC`v+L~ zl=w-3(w=mP*<`;_qp{l!p{f|+Kw1$VmRyx(gcmX*F#9{-;}@(hZy+vz+qrXVa3hq) z^lhBf_VBDa>ceR~2wsxr9%_eH$_A{9)O5gl34|znP2fL)w~)cxZ0_Ixf$@HR3H^cY z5()1<0-sj&YrtE++#eNU%TM1w&CN0(FT>6J^Wh6u=sl!mIB;__74#gErSlf)B-E{Y zD^uP@>T99c`JwEy$kBVFjFsRgedAv6Ht;k^)FUiR%m59jQP0BbWjF*+&m81rpkR}^ zS8dRVmbIVij9XxtJ29j8@ZBV*w!M_w1=E)8@cSRrOXy+NlNE~Tr+dJrHQJ|_A|}({&ceFirrcycf)`wh72D99y7`7UpsK$ z$8>7B#4ccR8Ha0(XM9(tHKEB%}{xJUWf<=6jA3Vhfg#hZzIP zE323t-XQ2zR&qu}R&1IXCH)Ks`L16u7#l!GlDP20>lq!;#=@Gt{hvh~d#I9wr(^gK zgM#j7mB$^uEFr6l7U-y{wTo5)Om?VZA)_7IqZnuG8}$8=8Z;lB_V+uT znFYROUv(7dX?757>O2MT{Lw3#H9cNEB&j@u?dsu4)Eeghz*j@N(8-#q^v&VU`c%Yn z;Z%ISn{<%m%3R)=BR5xq!nyzgMy|BQp$F|i$OComz)~HHg_y_>o+AJ%_){@=b?4O& zT>4hj#-nrl(LMxu$IBZ8^gJr+IBh2w{e5_Qx8^=MBo6gGt!tt4wJ8woA()P%a~}`5 z-?&HP*z)S%`NT?+)cR1gP3C!3%?PiefD9>@q%T++P#rX3nZQOe|VD*%qR_Gotz zRCE+mp$q5J`@5Rfe|0kZJpCBB^sw1n>Gz4$Z;``^v*fMMa24piwK#fdg}+Pu}n_6FIK(YKOM z(l&@XW&~0fJjpp7pguEt45+K~6s*l^8mH3&qBzp3v|v1~2(~PXuTQDSP=f71MYcgS zM14?!<})H}7iD?zDetU(RC4k@Cx?|dk!Pf#M`Z8|dYyVkIZ}t(m_FzQ8BQNp*cUl% zHRWL=)BMD9)|9;;MiqLxa$Lz@5_uDkF=x;U>&W!-zD?z3IX?cF^jCN|7T+eZngwip zpB|sUUFhh&o89nBW~rWA&i4y(KhPuU32aV$fIda@cp=n|UQ)5V(to`dL4k|8Ps_R2 zcrS)4BWha3wtfV>{QB)nVduAIq?N}~Df0~Tc8!bQ zFtXze)wT5y2ik6@aXKA+1AXQ)fWYwiTca_?6{{%NWYVd?=<|jtK<7eFq}r##5?*ik zr{M_!Iu}3a+Oh`1$0h@c0F9P`CHWl_CQ$7|RY=Wna&3{RF1Bhnu6|q}e|Ipi0j$x; zyKJ!R>rb#V&}jxikfL@n=Rs!NmxmGW&dAOrl;V| z%_hd!O98Wq{Fl-nu*j_zbly*3(^$|?Z=&M_ye1))%(=y=K;8P#F5B(kl$C--4Z|1H z^8${m_a&F}Y&GV_RFG!3K~(gwc2+m6PnpE99{W5Mqgijg4yDu;y=ocvn^$0HProgJ zbj(`abxWK36$J=Nci|l_>gsZ^`w58r&+BvdX_0>@Yh3Ns*yMlICL;Nc2FRnK=pyB3 z1K*ysWB!RBWTm2U4RzLmL$TQhSpj#MflrG*g_u#_BhdNd`wP_H$2P~SV1{jX%^n8O zjI!}nkUB9ZCDcol^lldw@1UlC1IprdXw^ktPUn0}mfs#4Ot9=@GSOPR3>d5a=Sm<- z9bOnNOM9Tssj1&|V-Qho&jl(Sp3zH>c>fU@65|re09I~yE}Iq+u7{_r3v`ohF{IHR z-K6B6dm?y@d8N{7bX|1{{GsMpvh}J;2syb zirZR?j#+#Nc-au;1aTb|gJai%VhJbc3DkB7W$LCIiohg)u8PX%CJvS9Y^(~^?ju0e z%iHA??V5UZ7y$0&`J9Bg;b0@uf?#5a_!3zNY=p)q1zF;RAoZGXA-Y5t(ZV_eiOLjD z6d#V!9qXr*)mF~-ZTk|EasI_2?VPBkh zWf~wGJRDQ(y!v)vwXtZB;j$DL?Bk#tuP8;ux}8&#YkqZPt4*{eJ%Mc zD|K&&a-aL3{Cl>E_Y4md**dO?C#ljf7kbwkRk0=kJI=G38Zlz+Lo(`EQ+@BL!T%OE z5yLzU|LP_!2!|f9a0++MYs@Al9Z98|e#4@127COgWK+!IFTc}puJ#Zh&hFQh6`J8T zPPTnJpR<7`9T_Jyb;Na4$apH4WpbR@t$&B?eqjepJ5Af#elJ*>)u(4y{vWs)D!%9{ zu(fch2w(na`yt=;$QW{fjKvz`va>MB*nH9QTjOP-9E_{m%~%BK=vr4Wy16kOL=+F(qL`h#r972m?nlDme<34YH&*P<)#EhS#4EVzDS% z?d!b7y+uy^*ssI;p$>AD#LZ-qFH^`{{sUd_YKhmYgm4VV4k5x`gZQc;MD> zSJ8+#QC>p*_Y0c41D;NRzzP1t4#?vv_Z;Y3~s}|Hse{SoGZ%`%6@;nfGf_DERX$kjOyJ@J6X=KTCQHD4D>t6JxhMb5P7^CgdpYvs&v3V^fO3ezIaiHS2^DDYoJ z;9l1C#%0#E*`f?)hym+;3P9Gqk$#6@#BD&Z)Zr`t1Bq2Zo>|3Il7_;5;whO8&-95j zv`nGHx$s#4>Kf1z+lisf{nJ^4%AT$Jo9o1b z2kteMSTYTfM|70+t2UeyOlT?yu8&4~i>8IU@7~)Hc zZ87KNeBDx7UhDM%f8X$f^kK(s&WM6v7(pAH-=#$|ib`cUNt_0zn#}-Qr(qYG(GGuD z{OuDf6&V8Na=RZ*YXWcW(Jtfa2=_-?(Wxf@6hz`&KK7cXwTr{J)le6i8g_yMSG3+z zl(|k)?Hn0K(=A&%Z_%qmjO-rRVLjwtV#9Uh0z*x&0*$mTl&{m@@zqgKSXH;S?ykO zESDDit-^Y!mT9IYSJ=rc@d^cAaXhEKu+>*f1l+vaIE!4)kWy3noui5K`3FZ4sr``| zsV5dYEI%5Oz}^y$sH|2VsT-obu14qc^EH8F0QTZuI2?>%RENPCH*KcU+x6f1bHrGR zI;|Ou&xH?efJk;+{+0+&EJcC}R*sn93oS@qi21&!VD+M4sS5!wvBy608NL2T-u#~- zuQkrxQ%&!hu_lV*efuA4xIT~r(*+BA&gHuUYHY1Owi!ux#lJtQC49P%zYYKWeYvvs zcCZC;(W7!rA?gP}tBLihgNw~MvyB^kp|vbTlWgC1Xd>-65TWx?h>%NV_V?WuLNHK7 z4Jj)VftAC4$_L5Yal}}_4qgM3I~k-5AJiCG9}PZUE2MT<+j{$x!snnd7APg#t9{R^ zBJKA^uod-l%xG;##>~yeedTgJd5xn7o&%md>r4T5ar|Sm)zu5A%$l<(*$j4bvp}XX z0ItzvB|UWa8=pvCdIchr?gOs5PZ+Mfe>0CfBLUO&{X=j`a|+r`W1PKxCimUx=!c}! zj&y3U6LqJm2LvVohWbGmkNLHz}t}V9BkCp~T9Lt0X4BJtl&(ZC3X?p8u3`iWt)R z8akRCWS?i41K0!EYysEHPsCOx(ph1O*>~#BPfPQw7ah2n=#7B`Y0tSyDX;bEIJO-B z^pre_c;++3GzIMP#C=I25t&Sbi^)4s$TXvNFo7=Bw)DVNOxzqPIC3>M!I zSh>(bBiRZQ|FkMHNJa}c+(x6i_1E8yvV+I}Hu>2slxH-4l^1KaPHT|Mzd-JG3Wlvz zRQ6Y#EK%dF%Y9Gy2TL(Kq|<%l0VOI~3R8-XFNA}`;Vo2n(wVle8I~wRV_C1G2%@A2 zZ~j-x!n{XEyAXfm_4koanIiO$X*)#kLj?bZ)ik@9G_`u4@U*xEWX(qQ5kqcV9xmWg zwik%&ww8l2(ST4IYrPsnDRk9OT?$$#56G_i1hivxw(>iu(Jy{JCD${$#|H~I9otk`Yv!+5iHiK|Nah)DQMV?c~!sH%5Vlif6b zg!HW1B&}@t{O$EJK*=-wW;5+)tTdKY$fE;A64oAL)aZlHRvhy13{3^Lcj2J zqx}A2;JPl1U|Ebh+;of5NTY%dGN{Gt^nU2^Qu0`cDR#9k5^s?0{F?11|rg z5Q~+%if9eNwi6ijFOS&qWuL#aC@ABVKar5b zZEtEff{ok*x6<^a@8#*AR+Vc3tpY3kd&BKC|4t(n?|E${SR#_UmX%|NUZO%azkLK+ z{at)s3-8n(CBMY0x=LyIl)C>)Sc9;7eDT?cx+S$GwW7DUd}-P%oa-Lrw%FNPS}Xx0 zxn1Mub2D0<%jU|b`Rwr$iKRRF?xT@1&-*KGm%m;5<$q2kA3{z#=>+9YsM+*=KY#I| zT+EaFTJ?XzMAYS3>Jc+cQVwSzwYG`hmF+8z{w$uTZ>Apkr>KhVo4uM#CPoSixFb(F zO{N-hSkmq7MAwZKpWnGV=)4CP?6>)Yl>yMvbc5#j8rKA@+rwBvJ7F>Y)-zK>(FXrU z--atw6Ow(sUp~khApX4pf?!qX^U=F{caM&d?G5OfSazS}bSzej%V53>W1{D$&!`hZ zQK0RQk8~}r0BW!vTX*EHZH?>fIbXn0G1_Pm%ojx0{ENM%2t^aQRB0au6Y(cyJM6Z8 z_p0=rOg2;qlgvV;|8&)Cdv#U8W4zXMJVA}KR7S(T&2x8sy&LemaAQi-dWQ-zSh1Dq zeVQ_@LQx$>e%ApJa)(?CdW!llFSx?3|ANqG6g8qQ9jtubep1bs?>%Ty=+6o8!L|<&R$wciJ{}Rm~L7$TXdD^ez;UT7OW$^_T>qg3s-xi-t)Iw z>-$*Serl_bRKdU-=IF~NyqGelCOXb=YEu=*JvbSksN*tnq%${kvZd5SDv%!a`pkwEe0BPs=&>=^}gk}rym5U#LaeM#F(pZl6gBE zSI7CAAqk)$7|9u!*COL`TC6-XYOjvdIxL5}Md)iyQ))l8q?S@yFH<;>r3<}I$uRf7 z7F*Rp{+HP-&t~F4?xhkXDK-iqbsd!P-^qnV2_z2t{)6m9EeeqRO#uBjo65bfj<)=~ zn?>DbBnuxrCs1@oS_A!19onN@)<+fkNJz=k3`*_!E3f*(-UPe_?4L$nT%eUMG4~G7 zwJk3LExGxb47s?VH`GH=;rz2V@3LcSj*bViju4tp=g(l46*n*jWd2tqJGp=RM{5q^ z$crYXukh`t)L-@E{0bs^(iav3#2CeR7*wXq7cERzRIlQvGYt%fS?`$HF4$VIpBlX6h{O)0sPTu6GH`D&d;!_pqW{w|4RFzyl z?jAk?1AZ{K7hUkq` zppmsr@7@jq3{ZRUR`%She(k{1U4v!XtG*jx#we#+CdMmwQIlJBsvtFrqUt~6tE(HH z%M2!WBAP;q7oN@~xt}jHQ5aM^6bCjDHLr@v%~eYVb0dm0jADT7y2;0x1I`oQE`@yz z0E;%vH}%&nzG8JMar@c4M|w#<{(X}s%wN)-AO23W$Z=*bL#M6_qETh)B{`WpANmA_ zXbE$sdg4H5lF=&LLE@8^KoO^rbSYZS) zf>>-b$TIdlwP{4PH!>=vAXb8l^5ZIUeqmC0WxFQS=iFU<|Il(iO;f!^Jr@-g?yj=L zdmo1dE>3~S=rGW4&KJLyj2a?d1oce5tf_l=qe*=W$pP5Booq&=AA#fXHBCDWI{HTx zvU%s9H+jTp?xvaNMN^onYyH~;;fAIi5`=cPli}XK-5&m_Tx7LvY_Y4QT~${;cB{u~ z^o+aHp{!%)*@`vUER5iwlI>kBLi$&+yihH6Xz7&1*(o=vby=v;k@19>qd~*nl4Tqr zU#iv;-^iaap5Gg7C;jc{h=q*4+)o9CSVPa85(VnaWc!Du+WwMv;R=bQb@?-5iPeHO z0?UiXhZ2Ni5ld~RVa2&_>bl$5I`6164I|Vv`%0>q%3I5na>pmDO#S}W4M~SRXJ%7@ zy?)ou)#3X~^|G9>`;4_j@y}cVsfAdS3}2!={#y=yZX*@SgxW$0LT1H;Rt)7(5*O0w zeD9nZeg21nMkSq9->NLUa@!rEDjKOumryaRZfxD@0s>de~`2gdt={!D5HT4`w)(?CT%-vF?`qSl6 zS|WXB%ZHB1%ur!y(`E!q5buA1F=?6d7abSU{NBEL{7iq&@zh|Jh)sqh6AZfB-z8Fe z<@U`_M;#mRi;b%VKFE!H4(Zgjdn|FO&)5IJ#c&!hl;(v`7wU~TwnpggvPzL3xvLJ0 zyw%wdj{Aexxat-JXHWdpnV}Jg?rCa}s&&4fbS4&kJ5eym)1${SE%#}wBqMlN9Rbb) z2_Fb&g%r4c4+R`0^1$DeFAGe7$iWA3G7)+VUx99+Z%+8{`7z&6`o9Kl{v^eG`&ob~lrX+IL1)$kR)aqVTfM63)Qw@lGl^#~;Y}yVI3u?~#5I0KE^{ z=aFvj&m7SCy7dD8e1Mi7u6!Zm!w1RLsI-}M=tQrG??;J`%#kf+H{h4c3L6??Dq(bN zZkv)kGv78Xw>D9u>ggO3FLavo_pT5OcuMT8zhATm?N=siE)#I>>kuZB77Tfi{1FeY zAg+M_{u+z__8=v8#(%08J03!NEhoS(d>zM4x1T9jS297DJDs4FVobH3usKM$5T_`; zXN@lPgwwa~tX>%#wI7Uu+#!qIt&N8U`)$P>pVcZ-Q?4wl)eH&jV^pN$?eW`dlh|3x z*C69dZNX!$uaPe;R}Sc(nWZnme^J3Q`KqaerZXiv+x|8@2!LXS$zCd5Z!v%wcX+VK zKal$3?R>`)*gUZOuJQfnz={*{_oVSJ{IVKt2dXAA;Jpn~W%XBfTW#6=ECw(8>jKK< z-=is{EeW*QkiE7})bZ!XkayGL`8(GAJX$d+S~Z}lY)Ri6)MuLMjD~=XH&A(X#&HJB zNVi49ua=Vx;GV5l-$qeJzeH%$Dk!6n?J?ZkQ}mC~g3hSJaE8{g%z@IbdsR@txM^gbPHD+5t-wYt+7o17MpY2+=IA(e+{(CUfph4lZsw5 zP4ydf!gOJ}&P8 z-pDMq7Mn5bG|2V<-@miWvXM209k#`{_p_+o_eY}eP#BYWlS z#MA+Ji*ez74%i*8VC65RrVQVlRz3mq){?OMls7U|uLyg3@aAX>=qeY->+z)t zgy7Z#tVUdp6rU{pr}`t4GctL?J=mGP8c=@#Miho1Xu4I{>M4+8&>@7SgLLfQz4Afq z=p)z|d1Y&?HaiqrW31f^n6cv^Ep9%^&Njc1-pY18CEv1BKZpHKl zCME%6RV`~i;a+5PflFJutEHZ(L+bR4X@H426j<6*ZCVAM}D?PZ02_W2)%BmTp(u$`a4_E=&Un0wU7s$J-<8Z?fFVg%Q zc2syJJJl|S;DWo>zf(@Jfsjg}z{KmdkTC{KsUM*xm`Qw9mtT0gOY~rNF9_rHYm5R; zcp&@vZWW7_d&;(@a12D!c;(BkE5x$72A=6f zkft@K4CMUADU5MW3TuvSRGaB-!8tt7xx_ujcWNT46QUO`oI=>X>F2mTbw$Q#neBq5 zx^*;4hvcG-1-UB3b=_mt0=4>V(}DK8Q#pI%x)I_nI&!`%T#KL=D^-a#uUwA)THASd zv1m|f-&h1Z%M-_Ui|(=-E!T~E|G38YM1h0Bf?JbW<1_j2`ty$$uM5>eU4Cw@WxjUK za5IaNYhBx(>*5DZg|U?F_=dB&Uqiq{7x?6;tX%j07V|u%@vgnXHx1jl%dh+`g5>T7 zb5(g*TaG=n+!lm!+1p@kJ168E%*hQRrcxse8CN9-C%VlJcc&j&5?zgAO*^puirhd( z5Pwcbp-qlM;pM}R{3uN84ufy!cQA|R;r*#uN63UrvFlN84$x2(W@#K_w`E?2Aq)p?nyg`boHaxc!?n0cL#F>n0|02=d_6m-G*F>jA-0;(+(lm!2&^W8K|Q5j$O}Om2Ow;CK9I?wlFT-sh3=odGGj;E<2yZ? zQ*6u~h^>iDLCsZL?n3845w(=76*hL*m3k13+fZDguwB&P57Z0kaBbL(j}3wo>Dul2 z=H-NQdtcTv1=11~BJMc0#1`1h$96_J!b4$3sh=0_eKz}$eq=F-!WSZ?voG1dhuNKi z)}ei{T&1P@D?z!=95$16Kr|1m1SjQvE(6K`0g6C%zkC^FATSd+07Ph*I?2@eBNXxU zG`{T~5`X#mm+0d24|U7&Wni8D2Or^jHw#QP{_QX4zgjVLYWc-!^WfV0i3>4Qi_Uy3B*xC zd%-6DM(JMqueVK}{6sdnYPq(|@TX+P{~@J!5M_g1+?1$Mhm$n6A9flX2S5JN?P;3p z-`Vr!vbS;Pq=^0(e4eL*&BS+%AF7eIyEgo~(jg8g*&MiX7|4nL2mZ0VPg)%jQ3;8_C*{x7JC?>K+bK0{TBR{vev z&6ToeKJ%GCIW@*4ov6H6}!g zvIJHmRvI5x?7twe_4q!l{)=^TgI2L~x#4IcEuF8KTtDIw8+v8W@#Xt0tjQ3r_-You z@qa={2nPQok6Zr4Biz9we%3#%x^$OLUON2AKY8Hw|84uN-w4*+{a^X~{a?SySbK#yYa~Ha|@e*OS`u3`2PL(EUf1Vg2vZjkyH}N$Aoydt$r}I zVB-JG-7Nad+wf$0gaG>(<@kyw?^W?8CioM2s}?5tACF>*axH(rEqo}6Hre!lv`~a@ zUVD5|W|^c)qK;b@Ndxi9ALzla#vdn}VM8skG;3(Abb`)cZrf@W|64#quk!75W!0z+IWsT;tGwlJA1c2Cxbcv&H0hpyH|X3 z$84)X!*L|}PUpzARSu-#h9p`UfE=uB;b1c0I}0Yh$>N-Ul_cWywbA*h_%uToPj5fqPk>i*#e%Q3MoDiZhwN)caKJ}6@z7}XhW}aAK4w^uI z(BR0fS8sU&arG7D|eR?~yZW0t|^kuGc{C09H zd}a-_qo)n(!Phn?wM;6(x478F3p4oMN{}3{bRY1&d=mWPkF8_=TI2%6GSRiR@rNG) zhCkD{{P9Z1*DLPJsNpZ*f!cdA1mMk`7ohPov2%R-KWfe@+iZQk@y zn5c<{ly{(@)tnbesA1}gNaL@Q*d6)jhv~nfs7B=4LNSF_Xp3*z_M!hG8~*e|8YO?0 zcHeNrC-`YnkDCtnF1+vpj>h~ezQg#lV*b!%?Ek?(1w8m4{PMs2@lsVcQ}nw|k9pJ# z{x3I-@tq*`_g>?hNX9f@!rz@S|E517D}0$NJi#v}sFO27=8sRvwxC?s{a^Ns@xc)) zjmP{cO#Ux z`F-yfKglS-jMn`H7xD285PTm2IAjq9fAygod8!uAdz1a#++AP(!h?6-cHm)0i;fV< z&bYaI3I5{q1lD|?0&ZN|0B_*$a>^#*@68LGmG?nf|Hz*ef={62W`SG-K?N=9s)tun zAZPz{0pu+Ux}hZ`a{F`t)xCH(<1_ErLdtP*hjk7QUU6X({~NbG#JBuIwU^!bic-%X zCc3)nQT`$NS)cfhaN+A56U6M~x>fa;lg@eZ=l=BHFOshS&duF?-TO(rT_zPpQ1X{z zh%btEFk$d~;yL3><`(|iO-Ijt{Y%w-Q1sy4x2->T!#KX4Vn70W$SSp%O-SMeQ9KOX zbpE5#^`{Ooe?Y)Lz|xR6b4NQ2e10{E4^!%@oTdRNKJR){tx;)yd3O zwyn2R_wk>KGwN5F0F7h@4;8|P+|#OBKNy?bXo3dE0;Qv@SefW-lwhpQ4L)R*={zU= zaiI3bYYBSJ&Gw8>OWl=zQTVYti5}y|E+3e~A$dv-!)v1W2U?D&kxyu`Jx~W0j&IER%I*Yl|L}C+9F=fANlZ_(mp;+`Y+2H z{oS^RaiU9mhxmlbGx2sSyEzJz=bJ*knv@hZMFx zkU>oak>=`vOOz;7w8IABweWN-40UI!W7jkKl2Y8ds6$bojSN_;u4TzvwpRo-|LIJe{fAEcC=a2Hv zKOH!vDm?Hn{2&y3jG{TtWEHPE+3N|6RVlsM{Li6lFhC?kRCB2^@P;3{r8+3^Z}=2y zO3Br0@W=cS0N*wkrvLK4`=6u2r!tO7?KD38;R^X`i-y`2Is6N?@a3-x{ie>6)~ieB zop&C7Px^222O;=JXPAPNO`rDsEdSuQ z72~UT?_U+)q-q-<8viga1%;mh-vG_zPro!-QWO7|>k7X(VlN)sy^9|SaN)`(npP7~ zGH^0J8yo(Gq#kof8_ZlGe~Q1}i{rX%!yocz@K52F|Nmb4FHA7H@r1U+GMheQyJ+iH zZtK7&@!s6&&w5t;Wxds7cxzdm#eM(o7yzNu|fvoI)4#@aePx{r9}C!*$o3Wwj1R?hgGY^_8L{j z1+>YX4xFVs3(_mNTY}H>QC%x3STHDgX;zC&I-abcuvTPM0Hn)FPCni9t0cfDJ)Kf$ z%;)V4V|>uhuYF+(zEcgHw4V9Py9RL3VUIg?{S!{R=cdmv{PE#DBm_hTIO-QrO3_XT zLNS+M5=x!yXnN3*k9pu*U)8CYy0rW2SHJhcoCvW zgwc*!iG;?<->)ucDf6Vm95W(um3e*6#}T&ULT~dx22XK!01{jW(Vz95)xy_LAY~)m zY6NJ%?7#Y<(}k$lrd`(%2)}KlxmdAF{}@0=nq~3zt1T27)~WpQmRdb%Fv)616En8* zhf*#Z*;5Q&l4$qb>UivsCp>k-aZkASYd7IHxP9lQzs9R3j!sSfLMZWFR4aE_aCwPe z@IgJ|OKUgsMAFEiZf@z@AHVA0$DBBhFHMQBc42Vx(<~Na*KT79HRD_P8UC8)#}piB zq<8ZujT-GSPe=#ws9wQ8$$w_TmJ~4sEL;8{{a3OQw_PIEcF}po6(6ym#%QV#t3gJG zQ7vmLDYUhL9EJ&NYMe*d9us)&z&oS$zd!{?0IBgQId&qjg008xuuW)q6Th$DR6O8| zSDSuulru>v&H9-h0~ttMckQ)jZr+Sw{FQnX0q=0u*N3(KpI%=L_(X@s{?`E#(=_6d z9$mg^_{Ii)GKYhqxi~Rf?zf`(qOFnRFwXWQd z8}KV~;4^Lx`BSM?=SL8z^)29uPRpwrHE1&9|EwSAiY*VUmQ}+KLd)FpKP8j(bNHhG zBYc3h18iGAJ9~`J?;JPGX7DkF;QDSB5NmruWrTtERKT-=7{63R_ zh&lFyARNGS@#@;E-H@VU(e`5jL;8$e8*FVuw?vs7|uYm4Rt<$XT z{n4NQ$^pH#B?`I@b8-R`*XtV$);u>&pY*R4*3Za;m^QtIV?&!fD7O+PKCJ%hm#(2J zop4I{-G#ij0}c9_tJEN1rt3HY zZ5u{fXU)otLA%cPAyR1+hVeEq-qgg^1--|n06VJkjX&Of8RL^^twQ0}&aN2$q!Fb? zv+!%&nZYmY;$MoF^T*6lE6{DkpC0pNgfCloT3x&D5wbn_2TI7rqneHE;2&}d>eXvw zpYe?!BB>r&g;#V&9RfpP99FWDps5?{+N=f?_c-E}Kai;$pqOgoiw+>Ap=@o@OMFLM zM);MVk&l3$tKoQT=LEkQKfZA6fgzz`t{x;K#!&^PlNoW&4bOf}c6S zVXqAlYDtpo9JZDG6SqCA{#*VRpKauy^1uF>9v_PTXnd%K&Hppw19)ChiwJMY0*)Nt zw{G1^8*_ddzhqbPzy5Lj%j&Acnt$5C3jAmC2SGa$-2@J!@on(`p70I9O8)QsWuRd( zO)VN*mdE;)^1qnqIR5PwKa-+zg$VxOpUn3pUh{uQMx%_xwPLUNBWo#4;m_n}MQ-qa zc`6KARa%0n_Xmp~*qGXLdZcE)Cq@~cAyW)x~v{#qut)e>Qn64h>54D`>IsQ08 z+I-lk;+U$5e^OT`16Bo}GRZg*TkUYt`>!;5V=I$*3iXXb(R==vJSPd%kNTG&LwslY z7rGy!|BkEzh?#Gv)8#5!t@2}a>GA@MkK@27JAn2?qXCzlOMzHrHZeMRLd)4>rYLs& zvVau5dd^|+d6py}^6R6VP=cc-7pD&QOG_u7{i09a^x0_fN^K{2cH`{yj#-LqW8OsEqu*Oe#W1m#aS8O z_P73vna!<)BUa_&4&H9gDF5T1gt%leXg0n*U@d>hcm0D`ZDy)qJ!c5Z^2aNVXu?}~ znS}BB*8JQF&-v${e#e_B7A_$9n1B%`XQ+e!xw|Kfvh0JR`21Qe2u3KuyZv)FJoZ_? zu;vJVM`OrO0z!&^_wL&l58Z?)`LC8F1yz2; zJv1hNR>=SEX6l91@AX!BWzs!F|J9;o8!a5qpr-Mo@gY!pyRA>6rr}fC@EgZg&f7RY zg|;Yv5gF!}s(Q%zF^w8b?GO0IxfS_l#$mNRJKZI!8N;tS^J+N8^YfQpatY_*Y_QZY z{!!P0(!o;I=ikoX(5a4Ous2&=RrX8=o~$35OqlEo{{Gf~i@#>}`0m9&mt1mjxeP~` zvG^v5xaEQiw_dbW5FM)I=TIG=M$mR7SD*Y~<@Gj-{6IDM&ZQ(h64plz<7V)~OaC}&mKWn2S3n34?FzpT zP-K>K%_aDe^CK?_l%X=}K2En}+2#5wb6)UWPYaWD7V5B8p7#Pb}^ELpcY7 ze^M1x+K&BS6~PBY;1Pe1g0Hshryico1!dR%(|@nM<{F|e)w08Pqd>lJOFVP69gyOK5q5KBX{0+&ja81yB!bS2lZVCug4UZO=JF$0Dp0QwU@a- zU>0Nk$o!XaffH~FyoBN12lYfw??}X4ba~3`YM2ysA<9%vS^S{cNm8@bqLz~yWKj-> z{8W~$)tS<4w9@-2hh#e}hX9hwh#}zt`z6bM)$U9G&HAAhH+C&f`x#a-!&Og=lGZA} zvf(i4*Ll2RMU)(WH9(?Vlk<}%hf?dkqt(k>Fv^fGdshfd0yFvJXR=)7;3g~Jq(Ykb zRhokzd*Y^3pK;e0J{_m%;W&`|!L%e5n#i+SSLp>mQE@_8T6Op-Pdn(C;~xCh*VC4g zU7vpEzkSjRUbo>1Pm#Y=m+rgmn_s{BFYfxnr%rv@uN`*sGr+I>C(QVe3oC@tbx^#{ z8(GQ~5U<`OzPyqDvD;+l3JsU@B8B5v%wZHw@Z`up!#$IVQO(Vc#qEfn>ig_ zwLU?6GG<7#WJ}La4#G|u=_i-8hO|u_@qdUuTJtaCb8hJ%XQF3r{^E-+zUu0$WWs-W zP53XLfAPFx*~+~h#wI&4f36Qjs7)C?l9GfaYV?UmpwTQoiTr-oe+T{q3&w_t4Z_)q zKUM5pa>*surUq;2?R2-zo4KK5%jV6S6}OZ>gPw*Bq9=Cq_UQ^DxR z8_r`6;Ql^FFb91dQTeYDX^Mc{aDGHN8(zfi=2ZHEQ}naq`lp>gYFuRMJ)ikFbBq5-^k09#9+L56l!Mb&RB znebUlx`5G*_aK#35OUy}H!h$&?;T`vr#}%-T@UF1!cyZr+ zDv5xE9WQpvJ>!cO{`mxpXhyd47f4OwqZ|IZjxGGPGy6`*pvXi6Dc36=MR@EV(FQZ( zc3<>QRw*&90%z+1KX?d69kK*B(C#wFq_~uoR$WOIDV?e)(heI^;@0yg`yw>PT>wek z21aD9q_z7&j(CiTL8)jGO~s1%)$>DGF|c)*n{Eqe!&liX|6(t(aH#xG;{>d@rB?nlvz_+$TzK+^)s_(N_~LTF}O=linwMqlZi${*Ze zLoz&_XU6|J%gIrNd&jq$h)a)QO=*?IKY>jwUo1V5Bw_%eJ2 zR9CeDO?)Y8Dti8lwX}5Ywb#i9`Eb?dXuUAcRTHK4eoEt4(f_gkmcR6Ayz8guOqQJ1 zn_Q0XT=!t$_atrg(gNp`%yKMF7M+K%(^}R+(`pp?_T6< z#^5s#`!FfjVBYpmt91S~OY`4Hm*QLpLpgA@BZsgaFELi!Ud8n$aKd656j4} zz;XjB4`_fscci5G!ihiovd{j>zuviHHw!(kxol@O_r34lbMxP>`n$g+4RUg75-jmu z7(Wp5-sc(qo~g>;6Lt)*dxG&#`3VbbHS$lWv;K4m0Y4lTTS!OnRi#zQYb0Z!`JA$< zQYN`+evb41N8f*?9?r!#XH1;Bd%c4&`2i$f?a+Vv2ZOPY7;zR^I7v1}}pCwv;G-alEA%sj*dlrJ6 zb?^b5>tLJ+M23CBN8Ua0*)RX}JO2&HXfqo3-~YLHzxBW)j^(Bg&Ii7G$G7s;r+mkr zFE=GWE52T@@+8n>wZ^dK7pa8W%=}`jiZAcW z@MrmhCM!Ht=1k4M&pP9DzKVFsC6_WVrQy=HYqyc9_&FWc(Kk%ncCbikP z;fKFk-9e)Kfcn0Eu*J0hsids@Kj5RBKT11uYiMa}N_pKc=ihDDUV}ezBMc=O%ICl0 z{PSOV{^m1qDf}yVnh}>Z7jZaV-|D56{2!<21uDz`WB=gg@=y1$jxyg8{W1O!m#S48 zvca`vgfEkLyn+@J?1R!}RMJ9UQDG9niWIBZv=PA~W#AOyG^L4i7ropZb6gB%eG`n1 zkCsL(#WVOVr{d(oM~8G&m-EMtPeaSSN)H$L%h~x?d^{u-9K%sgtrO=3VdhH73(wi!qN>tg-Fj&dvZUqZ@=jCop)k!i&a+Yj4+ zc|0~0L35Ktf0c{VUfvPlpSEe0K93kmQC<@6wNXqcE#a?idcm@6?1;eSiDX<|{t13_ zL2s3J>2VRO5g;K%g%(WQoQ6^4K_(^nvFG&46lPoY@`C>e{nwbnNQQV*sHB#DtjT@ z29KH%voeWnRR#!cAu1CZK*JnJh_!)z^#m%fg`C{cLu@qf>LNj%$J)`bQ(XA80n=72 zM{>Oup~6R)YukJRPi+V+oqg8X{J3TZM{1Ivf>h>D=YQcBk7_vZmrRh$4)*J?cBq}o ze|=vtu_7D+Ow`KWb{%lcznYkns&Z=_S~vvUm;OsDS^zTR|3)dm<;F+~JIg;ssBiS< z=eP~kc5D?~8ZFJ8zWI!;7hK4r$iON8P@hP}cJ?bq)h_;*mh1}I*E*^O<)0w*tmyI8 zopFor?)cxfYGgEN8;I3^2mcqoV}~T&qO0L6mgDxCT4Z^|7b`P_b>rO9C6`{p#SU)w zX!uri(O83A;p=l*RdvuyUngu`c$8#>DM?Q$ZcfwrC>mvqA%r|_kgWby{+Y>tWGbfP z{0YtduK!+q@kPxq-A~+sMdxkzVNK_M`6p5P$Np<35oJP5CXme>`!QCshl>Nr03TRD zJ=Y8B;)I`Kn^*P`W`m@&jeS}q7c;yu!!u9P2LazWk4(49C+^M9NY%6*9i~5&ZndQ-2Z?ekR}e^Egq^;Fkr2TD-(V zWBHfK!B@nRv41RqKlZseR+x4u!y#OANtsP3lCyhB$NjMX!mYZ$Ge z@}?yFO2yYpJ@SV~{?Q5W50+(ot^9#f?lXdTz0Tk#Y<55Up>(u&$e#h9 zaTWfs;f4;LOR5h-MlUS+gpNP{lRvJu%_}i2u8C-@K(=z4qZr2P3hU=0E6=Fl5abRH zi$`Vt9VlTx9S~yX1EpAK$eF#gl@geA|5#OjB}S03E0Ip`EhnvX{w1`tP9Ao}LHGZ1 zd}X2@O?LRw>AR}KM6XsRap(xB-(0XImdM#MSKy83aQ=wa{p$P|)<)T=$2E}Ffc%Fj z)Dr#mBic-ZRUL+EtH^H$7t^Wbuzh)N7SNTABZz@%a?1`6rLmiLSyI z5PH1jpJXf{keNcTdEy`XScxU3?7!(>z9}gXTDCI=;op5~jX8h{R=^%8meSfQ#3~fu z{0SY9I)|8yh+G_9(z0Q-H}ErX1Y8Q5+qnllH7S#iAl0;Ja4dKJt2P8}@a%KWzT(QO z*eU$7kLET7ON-cT8{Ax_iNQ97pBC~d)2%POhcEL!WYQ6CM<=&MzjJ=UUuDD1t{*+e!*QI`^N`V<&S+jAAp{)O z9Syjb(wyG&NN&H=p_BEd^Os9yCLmrwJ9q<5|Ng@Knz^|H)^8~M5+X{Bo+mu##oNF8 zox4B(@#vXbIOyObpY#)%l36UzAGmG=)d@1&bZqBVzOI3~qk+ zDjAiQHmor)ZBU9BLMOlYb?ZKQ{LNQg?t@S>XmHWOXpzjez2<#1sBdr6M2VE%~Ui3kF1QRrktB@0lqXp}$G{s9_6|FEC^80E__ezuah z)NYi4I1-hfX+m?~eM83atq#PIM*}Wdb20cC-zpgLAI*<8kQCva#i#Ae_&I|wH#ew0 z!t?KiTeqBZ_BCazHpDuX6YN~NbO{z11OB%vc*VFJ@&X^*PB~dw+7LNJ%TFoNM;un6 z03ucY9YCwz%GxSTO66EfDX#LCN;xP#v@N~Pv5Bq(R7g8t1}g^nY7(iuq#^{HtVHAg zf>K`FCB+Wmt7ZwZKB;YM9^=h^#kbQ)I~yiZyD)euD{q)BLh~=1ntJ|dz-awI{Y?JY z6fDFhTqgNhcw_wLVU?BE_Fl@1HgMPf)%E(1t@mH|C!Uvo6L!pDXcmzFYz9@lEw)LRFhYS( z4-Bg*{saDU{{*=_Yc!hOKy>~y`0yU%JBXAYhO8|oxcONh}SN zz~OoK45S~9V29ljY>IfpeG%tHfYqy48_wi0uD$E7yQLhFV$&4aPJM+?=*jP9VVXpw zSjFMFa6+EzcpKWRmNi7X4nv40v1W`PenfMEZ$DGgEpw2(C65REVp3Fc4bS>n{N#Vv zr~nAq9}7|(ANM_~3M}{@OydtO6eo7cL%6^;UszE5)ul@*(O%V;;0S-?#*L|<2{QJK zcs#Osu{>_9hw9~sveNI7>DagOPYPkt@&^XW_3^6Mrf8Y6j#Ky*`xt6PaD*v*_J;hV zYQR^{Fvpb(3(TXJTzv5acy4Yh^R3T21YRyP__|B(aYzn8tHY8W@Htd&3_aN&VC6Ch zw*2Y%1Ahb@h{DE^v>H)TA4yVPDAV{+J}6SAjDA%6?={z4`{qk8xn|q8@-U1bd;y=r z^DFKnZ)$X+F!*QMRRir|X9fNg4Xp~Wq_s5T&%gm~CD|P{z=E*H{4D(Vt%Z9AzYYfu z7hlZvjZ27LdaS`XWAo`e1B{PTkQw}s;qln?&#tUliaXQ5O0DEZI}9&c*&Z3p)WCO2A^^gM}2O9<(4q9MrPkD~uB zvcyX6s(x<%oXfwG(?Rz25AK?YuR+Za#W@2BfPeHx@~6(b<6&bD8vCAJPUlZuw6_B; zEq&s5|JC;I+)e&0u08O_|K-11z4pM4&;8`?-ucDPedPoHdB^?tAk4iOM?U$dp77$= zQ?AbW`Ty_te)Il&?<7OmPWri*ANxbH1rQHRS)` zpAWt8q)Gsgk>0+2yE{}rYU}42e6&}dwQP$=jD8V{4WnI-$AV=e;@GZxvA*!*>t1W4 zt_7ioG5=nA$;DpaMu)?`{ej*IAy=)uWh=IW!hR?TXWvGOmPDnu5yuMt2d&n#6MT*Z zoQOA75wAufw3J5<$yTgnfv<>9rKquI5VK97@f$l~=1YJ(iEa$UXc&9D&IF@148Hyf zj2OWbt&6kAyy?{Grev7RG~tlm7hSyd`~=O4C=i;XN`;DY5T{@Eh#x!DjRa6-R2?>7 zcKmWkJTk#%E-U}HqEUPcit_A~-70}L2M!G*|7$`dn_Sow=70~*;JlH%w5u^Y13r6u z$-NgZ@E^*1ljb(rW*hi>)G7zck?H)cWaNr}R^$(WOv+sM zZLu7`JYTO9ogUMKS-mcIi_D;yIV#E07*naRDcU|T5x)q>kX0(MK0fQTtAxV5(P9` z(KJ1P13eRbr$B1kl@OsbS}GZ_rftAi7MX#+@)LYXGVcd%2EVQ-YI{TGO^f7JtekyS zKHwLBArT4pF}|0sjVnt;|EMZw&&zGT4M^snaZWP3%7D|zHr7Fr;qVj+jNps{s&ql6LaE!N@7~mg{6pD zLNX9lfy~bkSEKnF)$XQ$nFXaV__;@iJ@b>l05)*cS9}Y<0w_#qj{M`Xf3}z4p71OG zK^q)3=1<#LL8TJ1jv4>@&6jSwX4|9%;_d}g6eAKH%mXrJ(@Y%6W6KDo7y0av$Mx&U55i=+JEB|#OT3dW+CN}?hI$e5Y9ZHb1 ztkcife8m+Xo)odyJwF~F3bOoGc>Mfl_*16SMDQj4>GADu$><%fdZS2(i#iMYZ^^;=!73W%%4V@3|Qi+?IPH zbAtzMG@45re1h$56uEX`S~8{3l1fClsY4+jS>YEsbe^|)Ai{ragx%E%TxT%4d$a40@% z3_j2D{NTBq*6Jqni!Vfsz7$#U9j8#^rYsPk;VE}Ln9k42pDJ$qk*k-Aa=i}41@2*G zWYQFLjw*k8Rz%g((zD<56`tF|0ZynD2Xa&K)mP&2YzrcV?W78?%i~-AsYk>V8Q*B^`D3>q z_$NU$+%MITa1L@PGF2es)e)Y_nk!%N&xC)zIHi9+@`4kAw=oTumUisip}Bh)A1eA$ zfhYcH%No&TvQ znXm%f5BLyCXNBrNP!pPJ9NJ}<0?h=qA*hqHTd7{ylvJKJRbsuC@ic6H~bz0i1_ z$j>RqOl7~e z;cgB<)TkqZm`hu|)lg=zA~p13pebrBz|_CGl0^ijL4AQcJA#<;AzUqokCdEl%E%~W zZsZVgaHOL3UP$0E?i|Df-(0xZPaX0=N`kmnvPf}nXJqp7`ahrc`(?Sw@Tu5BE2d}#JU`6Dt(IILI+zxa>w<)=$R83Z_B zZT#ct7{Yeft99{?QOASqwf4I|_u7sHtSU+Gu4F!WG|83N1=1a>+PAe))eUKV=jzNwn>?{D;4uq$1v~`a+5`%N$-! z(^6IhqEf8c{I|lOB&`f0ZSj5j$51BV%J|YKhzJKCe#`TRYVPXrainY<7~|}+KZ~IGu*oRAT%ve(KIt zQIq^fROxtSG`>xb&!hYa^ng#xqy?V;$W>QuKK=AS6`uVUH`n-(Ft%W3B(h~TeUJ9q z_A!6LSA5TXJ>e~6U-y)s_yrZw*j`l3zc@>*QU0W?D3{b^ew3^R3}|%2AAbH_vzl+J zhQ6rt&-t%kaT16{DTkG(Z7}D=k`V#_)bD+2u`L)D>_R-$V5b)=I5%Y==XwXbnw#-0 zd4p&2MP$Z3+ogl)bH4Dx2_dZv!xTOf5%@d2%;JR#e(9p{{gGG3lXjm7ynJ*$ubfqS zsX?{6l2&#>x|$azIB!jR?<%kY{+Zc%#3?1>bD~aResqTOXNh4t4`cwDCV;|!z{d}I z0P8)K^%~G{GQ21J55L%p9Xci%%iWIbwiyw0NQ2Kg`tF_V&Wt}{ONzMuDe{)UvD-4_ zkE~Ro>+-H&IUexw@?2Hyu>$ZniJ#8UnLp%FMZ_P6N!^q3yS{ps!450kCv;1 z51HQUTM9&=q$cA7{*_lCw9rhD*|2_Yjm?P4j1onz9nxt3pMSZ-gFoF88|g2}3j0ya z@{OIzFqtEV8J+Tv^Dj=o74sKg%>5h};Ts19l16_%lI!Om<@~$2#{1F!L9L$&oUvmF zb0A{G-tW$TckbL#Q^~k1)(}L`HRJ5Rq@DxmFuqZYOa%HURqvzxq+xKZ==&Y5=J#p* z@%dxn2eLIM^PThJPlgA8> zr5irG{F6b$rKaF0R1#4pcy?8oIV{DzG$xkIPN6Zr5^kXJDt^~c#d!@rZGF%3C>DTB zXxg?%d@I-*zI&~tit{?=uPMzU=)^A`r%^&li`W3sGfqFF$H*%87nU+%M@DZ`K4`IB&rf%&l}w|s5sCVdJ|M>Vsch$9 z-wXcfr}HzY7xD!J^@d;~-Dtv{!mel9KZ!4H2rNvrS=e@*eHCg~+klG6Iwfib{0P*M zyAmnr-%wK@fwEi3TPq&LFXgoy7OzB=M~Jrn;_kI;0L>aF6x<##UN zN=I>12X#?lyHR}EPc!n3hQ7|kHxZy|Oj6cOSW2ezXC$MX8hyzIwto$uq9H#EKjN*8 zwaf4Us_{)r0~?-#YtGzm9!Yv(JyJfJv}=FrznG*$^auWzb#s>%WUKWMyDgO68(M-R zTT-R8!Z|g*<$Sh)z{jcncYKaBtw;bXTXdnARXD*b<`%lo+)R}s`01rr9O%;a(W0u7N6fDo6qIFd4u+x=6RO|wf>9-wfS6| zOBqi4ahT_oBB_dx)@ilF%OBbfB9&wOFaagwI_(xXEp7O$g3!*drTE}X8u=X)eD1?7 z^Eip%qh&_2wME6!g-eu7ftTj8T$)%!I6 zCLQrb8juP2hl?Z_TmF%Ck1sLzD?fR-=-?mXivfOmj4JB5RJ-YeFm_P)P)*4DnD_$=~w*0P$c|us41mKtY(it5|WL=aeN=a^|KBS zKD6CQgyc{uVstD|93ZT@RA^d-CcLx^__d~wOw*y!^RDpHf^j+tIiFveZX&PfFPHv zm#?2oNiL@vg;FW&@J~U`@u?8~q4{q<-5c=HFqhDfKab-4i^FlF zCfS+tmTV{z6ebBc!gG~xai!IkBm^EL|I5!+2COD1WSuWS7)vX}E%a#{3(8hUG`#{5LdoQ*%ppdBLT+ z%5esJ86V)V_w{p#ng3XubirlZ1U6<(QuQcHbthPG+t?rD`qhCg%Y41QAsen=O!ZlXMUMes#- z%LI|mjbYoj?@-QC-$^WY_&2x#N$-Hta7-!T(&u3>s^+n^Ui>%16+Q*_jBLZxz2r^bL3h6uBK74J30ul0+gQk zMirZ?Tv;PusWerg@2Zg%8@zpApZE_u6vcgJYtKz^J^^fQkDCX+O0lwjj8Ix$X-s-iX?aj6xsR=K#MS{8hzW^K2#$WHkFo~3H@Y{eJ4oy zECKc=Mt6ocKxAkFV=YBkFLA*AOS+Iqu{(0S9m85f z+F@L{I;fnr2G?gBF?fyQH8_vFDlFv0Yytv8+Jy*-46@L-1SyNS6P<^#;_OIsq(RPZ zeES4eW0jjPc!8y)%@-rI1x|ZE#BkNQ*9`NhlTCa^eevWeg~dV!SZcfm8{wYB7fPVk z#=DPcGjV5w_g=6qRo<29;Do2~QM2&{okdt$-GdFnhe2!P zf*pZ$xQMl}Q-{)1yUqy8|72sPF>9D|oCJBvmFoBtZwmgIpXIm|U~iu4W`w1G!3OmZ zZeOMv@r^O>?Yp<`G2Kfx!x28Un}yHZxo6Aena1KLo|+z{w%B53+|@)jOHQYz2OmJQ zPnJhxtK>S){}YCB`l zBq9}&y~IM@tErh0G6c^>=h&w$OEdkQgYAsKYb^8L>X_!Q_{Vu1T1BZ3xrUdx`%Wtw zpK5iR2^l=X&>|TYk zq*kK>;Jgm_vyb1SF#xgih-7#3;Yj-J=%FVlC-c92<=9o1zbTCRh z=J=&bK1PLUc2hDj!r-Cau9Xmk%KxQe={RzHv&&%r8~IB zSbr1lXut&Eg7t??h~^j#N0^z0ntO%(1A9?nBuR-h>YTO!nNCokril#(T!eA7hJ_l6;p@ z!MG2l#tjn$Rz2?MLw|S+Hk8iodb@c#J z_#Mdz{o(1RHZrz}*-1xTMo7-l5!hiIvRMRL2y^xGTg+88u%pmviuYQ&52is>IEGj=sB}4do*>Ln-+Es@1L64B@wALEC8(K1PmSa>~ zlCEc~)+w8?C3SVOGlrTRc|c9tx%{2?RT5JR^|cu4h#dCV-+nsP*qoF#?_GRm@?lgw zgcu-m{br1`Ua*(}chd7>5jKD$AspETWw~lR$jlif$b3KkvU+kJ;P*Rsy9!g+`v5|B zEFlf^!FdLqs1~FDaKKoT2=vAl&v4!)zyEv)F{GY(tO&L>oI8PmPTj()Fig|lM$X(- zrmYn40?c=GQ{s!9Z3Y%Cu4yhl5ZNp;*9|zk{JXcmnYA(OtV)kDP#x+1PBJ zkc>Oux*n>%)8M#6!^Sg`Bv4_)_r4KkIm9<+4|`-CFaV7Iwp-&oBGTBvs#&{h5?gOn z&*4N*s=y)l-wgte=QHI;+4_c;9k&l_ z#`30Dmt{6kVq*5Ii^=YIqt*<9uq-z;5#*xOl$7Lx8}~K&G)Xq0N9BhYzY@HFPR!hxxFr)ylAiJp}ByQ2yQU zh+>=Rv}e^y1irIa`+eOX!+uVsLfS}pB^HdbPsC7vcGV0@rsl}5wb)u)0pi3w6Tf4r zBBnj#>kmGGkuelS*6XdVNfgX3x@js))o}&SqvJ8PeCY}8su1ERd~nYj#9Wz}zizYO zF$Z)m-Tlm>Vyf2LCc6i=A^2O#-ZT0Y{uz#|aVa7nU7>se?_B=ju`W-dRla8Sa_2wA zh7r!=Q26l}9rg3VzK$F$QZsZPfPWT8gQ<_$E$C}z_$fnZI~8m@R*?ItRLkge)^(7j zR;2tngsmXA75zt&uk?+ovdS=7ccU(qv!ybgLzPWC?>i2 z;xgl&sd1=lD43;?tYWv~IX&Kj*MxafEE-&sm(m|0z$Bra>-l&oy@j4Ydd-QKF)ztx zKQDc;LN0JU_F)|~14QEWV+H?FWRB3zZ2i3N{*Z8cRrdH5j)9Q)(i6IfNoZ4Ov*+f+ zTdxm^3n!>eamvlDGk;jCSN{mEc_RtUBB(XX8Ek%`a$ac_D(4Nolij7sXGowDa!5Tt zimF=VsA$ z_22dwg12%ri<4g?OjS(ZOj)a}aOs)8rs0}SXRNTjySf_q7{bT7S#EDTk36|V zZV=!MsV!1hSbYIm&i~(n=Yh@|*O}xn37>vW>&S)Sn<>*HbovN0ePlrYPurNPeX9!W z=}U3-bGnheTtj`gBB%ELwe2L$SGtYY?GjjI=xF$NZYH0%6)?{xAM5)>o|ZUQ7B z7~Vu%jDw3a0J4T0OSZ#nKu)=Ay@aqVJdc7>JA`}uL66%yIF)GPPB3VRmvGnCS!t%_ z{-ky);Z8E-H;S*LkAqrAmSTw(Dh9ww`uNrL2Nk2uFd!Mv9>bd)K_lsPa-#|lJ?S~b zT0J^h>O5*oaMop2Rs7|P2Hon26GD5;GWH+zA5n_oj9o*Xui7uKdnXT6PI@)FY?G?9 zZOmU7M&8Wn6+t=pHyc)yL!*;B?ZAbZ136Gv7C)Jd!bV>GrQvE@I3&inkE?Vrg2IYg zL2-qAR(&QGE;4O?CQwF-LkL}4pof%F5pR?319jG`5kO&Of_-t6;8F`(pCh8PluPc# zV--F_BEW_?bR2kqbbRB}XABDCTT~yRY9b0e}Gks|Q zau`Ofk`&Zv5R)?wUTP}1Ll>Ozi~@xuG_Cq!xo}GRdP|X?hIXe>v4kC>=xI2=X|KRd z`71?JK7tGGKXWPWrCLJ5ydj}TAD>6IH{p!|&bweX=%90gn9mN&+RzY~m<`i?S+i&M zk0B>7!d4aL2T5<{scv3A{^VO)KzgQ{M+q)EJVK9XRvj0gZe>#m9E&icJmY_(^QVBu z%YT|a?MCU#N;<8b4Gj%gYU5N=Ym46mn1&G$u;)g^$DrMdlXc@G4-hFaxLZdpdGo7h zVj?Mpr*624;V^VM+1K%#({p^T*?l9W+5@f{gTC67shYA}xq>ugiwZ56sGo|P46>^= zjF`=j8^{j{Zrj^n?5Ks5P2DPIB$^wnh(Zgjtnzns|R5$aaH!pQ2mUBKY zeTOrgtf%fXU~6;k%`e40S|e+D?q>Gthd;iL!9wrA49iC&44gnqDkZUCNF!eBYo$Mq za4i#nB5|#aVekAq)3&-{a>Q!N^!P$;G+?#NA7%;U$~++nw_iua!k>@#!)mHrSUDX7 zmBT@Js+y`GTzGswDWX;x*2o%Wvy33nxoomtyW!U6-%5O01>f}j#JV{ioDCNfusKBq zgI+SuS5~|qyT;uDp2)GTWTu8GSkxWV*&7=^jduS?RJb=p@sX6}KWYV0UWv(%xXY$Q z^=4v)yDbbiiXB=oCHbV7aUva!okiYDiQ{73-+(W7c(5eE7EiaEnYW4|#!y3ul+hW< ztK%8N8vq%A|D(9Fg05`=>HH8;qRYz#3@K!&RihMNVs2q`RoPfn#>A2Y$W-tsYF*QZ zPj;j3%(xWuLMH$Y61=_Rs1+n-b7&A-kOuI0_w%u!d(Vp)%T?fDlEs0OoW()YHx92?$uv|CA zI*!(S>d7$MQjD~e$2}k{NU4FdFsW~>-yI9ZadGc%=oa1|wr@*RPt8V#Cb;K{=N`y- zDEX)qiE%UR+Kf^MP?fLWw;)oj?O<-QzUh64YXS4IJ`ch{u160@jxG=^^FL(ybb~$_6LhI8| z$Ge}iVGpe5#fL%W_P(K*`zHdd=jFgGY^Y-jMbC@ zl$;NJ*e98JfMIBXTGrK53FKKVjUvA;)pmF`*-{A4SfA??XqA7PsW{BQfZt#~ssjaK zA5(U||19-TACZEd+yTfBo|7}6b>df4e{nR@02kI#`Um<_ZwXJJO7Fq|7)t0b# z{uj?#4?i!w6vXLYJnqsWP~h+8H)v zKR4cJJ#tPJR$}w+i~sqUAH!)G+iWn8>jJ$sTO8k#f9_Zm1<&KfrDyYYIEG2Ln00~Z z;YQQ;eI3d(_P@et*ux$@&(8{q#?P>a<>~6l zcMmj8>L>d_D$zt&c5_W1zPvpmkI@!1t^aBVBjdW}l4fVdxmRDJHTw!chBLgMCOCYj zznL9ulwx4^A$h)0kVloB9vAL7@f#5e({Hx{ry4dTK>;Pn-)4d}ZB=X!lyRQX36Sr) zX5KgFxDQ45|B&2Y-a6&>?i@JEv<=!%tL#Nz`JF^Syj1rwRo}{D#u{VAhhLEdkYuWl z^M!*OFf2ob+zz_Iy!7^86N9tu6NA7G=?}~8lLm}o{VILCpaPCZ>_PF8`Dx^Dqi-FpLNY=8tObuTl|#PTQKzsR^`P z*}ooA+b+R(S7Y-W^7O%9#==ckuC=EOQUUxrOjl*ki_9&=6nXt=?tC6L>bLB5qaD_& z|L;0)WvazKeW5!Z9dNdz>o|ywhRJ!S;E4pJxWp%47i*k(30)&%VE-KG0V^v5FYm5z zwI$RS3F}Mi8AI0^bux4KCnHfM9_4LX<8VqNf#9c11iNoWFWlT z!9RrTzsPRHEcC@^JpjWv4P%n$5@?8!oD-{+D$Cu>X^xe^a{7l0JSRCi^aOcc$0fY6 z=2W~RV?zcYSH>aS^`F|s-H+}!_qw{brdC|AS$1Z|3I8ni0FL74Zox@5C&80~+hXfT zW=9e>*}1EL?{J^uP;k|TWlwYN=Z}zIJ*gD{uMwu{?^n$mNznl|<;p)p&q+t4@$%$i zeuoQv`xZKxhG=;bmfl;?L&vR_RokBDIvDS^%Y^Vy$$-6qt*B7D9=O3M|M zc}%h>aqtV2tG)sHCgiclu1}pG#gt^I63eV^mV+Z7mRV}{;+nZ{SXDSV9ZD7VkD^BT zn|8a#{x$;`;<5j`!cm60!h*_0rckw2&?@u0{OlOwme@I># zkm|lH>IRhk{rUYv+Rr)aubSXu3_a;Y;H~@N7;fWOGmu9b`M%{tpy?VPk6UdLgilt; z_9ot)HY${iDa^$;!6E8Or?z3?N5=M@&W8UEz7N zI2*a3R6huj3((QA2>~}${k-k?|8enOP%bIJmhPvZhvMn5byfX2gVex{ol(E%98q(o za877q6?XgY={=DZ5}@KWm)1J)?c13LdMvrTo;T-SI;q4!=b`$PI$m6a2yq8vGEZ`{ z6RL+0vEH;thPAvG&J_wj8T+ga*%|QGf|sR^CT4BvB6DeDF*(2 zW#*Fv9V!@)#xOcV?^eVRUUqcGoH-5%j>(d~jKuw5xf?4+^XSa!l}%HUAQxf9 zYZszqh+0OzIc2Bj@1w|jMP~3mX6(Sx;6fCnt=9i`zyI<-!Dr#B%U=dVWEUnmHmIlE zJGaT%O$eRokl%1ngLY8;PxSt~O`o+0eAn1GCX+)nj_i9>e9#+;oSqKx4u2l$Oz;LZ zr__VhSqMI-XUhG1)5F>V!)p~d0;k4%4VvjNB#`0_jR%vq^@lKRCO?uKA`Q4BkKxr- zU%uT@Whn3?4_ll8#z zACoXGBe1Ho3be^d*!R==x?XfOO!h?y-_T~i;pM6+hHJ)BJTHpp?3k{cLjj4@m2)A4 z+S##|LhDr@$NS)P7RiB+kf}oTD>+h6@6;asd(TdWbFn_N0!1IXLhk3=&hZ4sLF|!H zvHktAS+b`Ruy&{IS2NDj{jp4!6Y)Xb3lme5m^`V5G02w4Vv`FOhqe^DHy(Os?f3Wd zm;?%~rMqPRE?abW41Wf?zk_;6<_iwoEd~+#O0RFGe@M>~8gNHX41V z>t(xB(91#q6A=2=QRgKl{``~{bj!mFLOllJSoZR+J0+eUL1$Yv$Mfmci?tp9L^7j6 z09fVz_BI@buaZ5TrLJ&r4Dnewi|K55`WMO|;^1==Z?x0Xhyg_kUaDsqf5wN=n|?Io z4s2L&3WRc?QXV+ooYSLPh^x|*oWg@U3kYMLc%($a{4-|rA&%9q6R?>Y(Q#NIUjXv1 zb_N|a&Q#e!Yi(CmS*oV~d;4gs>Dw~#5TXlL2}qPKPdynfp@GxU-1hf#8;FgRf(QdW zq@ncpca&Gdei70BawJGVm>eoM=yD*ce)D)zev4~8y~PmHlyr8~Yiwph;02~R4+vtN z+9*pDKW1@b1f`S&y`M5PH$$_fR^nZ8DJ}V^-6!naWq&=hpH7rMda*eskB+5oe|IZI ztHj8Kzf1bQ0}%TCobFi)JiP)oH@kEf9yd5OZ8-(&2Bs=^t>BZfQXIfccd6ry~F#Hi?2+le3X!dQ$@2~6bpsV&9Xs9oIG~Bt^PK(`J>wcPLpQ6cid(ZN{t3? zKa4~T=jn8bEe`tf^{=rW(#4`-SNX4zP>^0STxQ-F-FJC67hvleFpFeGML)3s1cn8P zvCV-Ta&S)pn0FVWykV|18}eBH1Fr5_MZk%kC`M%pkqL^%(w3e5 zdvG2k1ey6LL!`_8T-V9|TyJp#(P+fAzHL`GP$XB@a$e#dw`>F?nuJV1}e&zRNfV9Km(7s%mPsy$+bq0f%3)S8OS00i@vg?U21I+*Ok^SkV0N4 zKN8iO>6j1{Rt{~LRC5?zuRbnB>;KWrr_5Hz#z;ehQrGhvdIR@(F|?#fhn>ciSG!Sy zS7r0BVLodw_ye2`Ou*&3AdeaJn}54^OimD8aerm$XEt{x)8Z1{PODYpT352TFgzsJ+s@o99=LfThXdF)5kjAfw5;_Xh31!6*hde-dhRNJ;jjvr=S+iCX4 zdmb6mu81;bNw}Cd1fHMmPEj(<-)2^-1k4j3zI^@n^2cO8CyrzV(IFf4$2l6mr_C{hP}N(UfyV7`m%^c@N1-%hYv9kK zeZi)R_y1ZifdQ#x-hW0r5T}`ADEsF=6DJK(%PZht%E%$s!wX92hgT+=R!=0dbj3XE zIhXDZ$D}i=VKxx1eX!~IGQ%zgg+5r+yE^(e>3ZMc&0q2&hunhfdrCDB4W*c3D>Wi= zg2N}h7m^{B=7Zwq5WR5ab|rbe+1E3?LqWge9^Djj!aY^<+s1{RpGtB(;aQ)87(=Gz zb%rza425NvNrmoBcr(ck7()I$?Vf<|>KL146Wh~sD+~;7h}T zERUnAtelRNA z%9{chjr)WW1uq(R-59U-3Zx0sl=$1gUG!Hiu<*QwKN!@VHL^vDjL)q7(pMIUYrCp%36<;cD;&9WiyzK z@I3u}o|^vJriLPpHnbn>vwzfl$+{`&07+o4pRlSl%7UJqp=mj*3z4oeGE}l>I%zW+ z!dq1&tp03vzd^toZ$2bn)=s?7y@b%kqc+-WvZh+7WotOhiM)ep;)MVb4>&E%5c_0_c@>ZFZ!?>Ip-r$$==X*TUFf*S;rso zj+XM37i`X+de$eCIzu`}k-r}^(JZq#PQdpNFV5s*eo=%~mE>@2ys8-*PM8P*;J)C0 zOPbOUy|v+;n{^TFif8i_d|Y;7t!#!#w6WJ+<@ykoo1bz|*4j!Z0bPMQTLUU8mIlST zS&?iH9DY@v_?GCiVG|f#GpIMf(=hK7rhr}SFj7E#`A-%ioiQ>BQik1at_T5OL%$I2 zXiAqQACI=Y>PE)K!2RrBz8=%&1M+%;%KD*nrj|F=WN7yP3V>P(p7gSHK@}*EqVQnl zXQ_47AEa(jyYS4_OstogYTnCK$A&(MUB0U~UfcC6#%m+P<6mX3$e~5CbCakLxZ!YI z=1m;}wZIa$Yz5xgUjvS!W4pz#ha+q$yvAI^U_WMbC9jYGl=(q|;)!5v-5>6v-XMg( zg71UtQFq)YzADRwCf9jh>MgWLtwa^%okn~oaQ#EYfob?A6X5ZRV z(?_qzQ$3T4`U%JH!afc7DWJ=tpATMVx(Cc-I|m5y@lEK$maW@`hdbi*7>OX#C8_CO zoMvw@s&#gwQOH$BL4zFSAiDht);ewAdYYM=JUtOUn1mHq?ANBq_mR}`rtMZUJplJN zKcYIx6Z?GO9Cg;$s1=TvYM;Wqsdnh;-{i}Edoi~@i7B_7{`X`5QRJi*>bQ?mh92nH zrTX{NWQf$|h(%Llzj4@8_)BZHI~*Hm{bJc+j%{@FBKDcZc&KQ=a0(Ul_@0r9%VhLx z>;5>#;)utQb7atPEu4PtFO+b{%AnXQi`$x0J#wXcTzD~~;P~>mP$MBPd-c5TDQR4{ zT=iOH$uTJZ4b`O8qBvog|Agk*C}%An$*b0n&OKyHGz+A7zU}y7^^1yKN}CfX=TOLA ztvivmrw4;%Ggm~S)q_5t!IaPT^h71{l55gt3u=r)?Yu_CaIXTFNR>NZqB*(OYjigj z#57~d!|vP_00&SP1(t=x94RW%cAQ)t(XMdVAq>kIdsr}@LHjI8a>SfwNoKlC!SFO- zzUsG0j2cl=OI0h^+Yk3=`l;oR_+Likh%MAO*et1D4M(w~A{KWAf5#qI)L6x@HcrMH zK{Pw9!56OFQ+}nB`8nK>uPwzH;b=dI8-YE=D3I*Cxl0`4LqMttT;(6dT!0N4$Az|* zJhiv{)9`~gDL=6!WKyCx12c*+USjA3_Ae*r$8VBuGFvkO@aT5?6g_En{jUTZv^U&YYB)F}gE=q^i6&RJ2$mL91E@0(Zxgi68x z*nrL%5^lK`xADj)OAh7+crUhrsvdZsbCi=-H^tW5=JlNY#My?*{Rp%UV|J|KI2emyJ0gfV}5HL?uZj@a&qEKwIVTk~5YXWGWi zZ#P{WO-dn+KKEcIGzn|8`u(wKfX_yWjG3!+2lHoUN-ODZH4+0+8MvPPvjMkM0YI8O_Ts{qWc(Y3ZE9Gb=x1G zi#SW4F`CzUPrJ-jqko&Z$sDzvKIgVJe2}DahrA6PTjyQVGdBjjJP;Xs8-)3&s#64u z?N0{1y`^=4v55llQYV8v`X&Bf6VrZI{3;G+k@oAyxL;cM0Y3lqu*EMdp{{dVk4PDw ziuj5GKk0_z%JnV3@9Ef);UEW%i*6RpR(%UT`%UdrbKsL@U@`3)Uc@9sruEZ0{601FxKGr^6 z`@|hz+HO+ZpQz&sPg;29+MAXXsT8ciabn{mbuCeTj1B#b%Nqx^*(DN0WMKhHYL9Ld zWe!0+00s+ktUGl&&A{v-dxx?3CxwwEcquh?S&NxvsllS6E==Tex4!sEHXQMU3npod z*mnkgqJ8aq4)|~3tZHHQQ0Em>fnrTnZHUS4q0Nn$@mF0>J>OFN=g=x_1!m7x0X$y?s{7$6a(34u#z%&Fw5CA0#OP6jKv0%= z)gMuBoJx+cth*Xw&G?Y%RI}cyW)qM?S8Bt;k5t;z>HQ1+RINW; zwJ=m#c+M)%ntwhlSgc`uO@Cp5UGcp6_db|=cK4|Ke3nVjbvfWN8QIacksFU&G6tS3 zqp;Ih%0AfFBKgo_7v=l7eAn}W639L)zQNfoDs zXZ1lOQ^3BIjWzEG*Wb6k>@o=;@I7M^W9)q0@>85hfqh4l%rD6sQCF(`r+=>h zUGKhwfNoh z%ViiLzRC=mG8|&zYZzN+u0v4W`V+BD4q71LR|5o z?Q}(gB2eoYW&vo(@z!H-PhiiuT2_GVLQ^}dRu zH+yV9y6}@e1In?ao;W0BDz)9Ydw6Wkgu#EZ6wotHT9``7Wfbqoet!zJ@2>Ps!k24o z*fU|2VTLZ7Bsa8N4_l8Eci`?7Fz0N=7rCJ6p)UZfLPvV|FD0=3U~GBlm5r)@^N51h zH9eH$ou3zKQnU+;B4-yKj^sND7L9D5$l0*aO*}^(_)y5!p&|X-qAj_Kj`mkeS(wg zfkojf`&9a^m~i$HPexHI2LziZ5QFh4DUPG*`Y7$zuPw;RW)=D+NxlD2Ntj~BA-RdT z`0#ax-&q%0oDZeHdA%pQ?yy|DjN+@RPMi_UGRqY(tIKu~t^9GqtMR5Nx(7{8>w-pZ zb9!zeXf0gk2c`F7OWjOz`U_mqe8N)+FK0>d?OK=`w~SSc*4Z4X4~bRA@**Y}Cu%pO z&Dh#oan;1}h`$W}88`W$36V_4;lmvA>48chrcc0^iozVlr*F|d+8VQxp=PIX#{}hI?8)X|KScOij zq&|P>J<2RbdWVc)jvUzgCGOMgOW^!Id^oI?4REzml>X~t0#m7!!;P#32|#7u5AK3j zvGe;9Wo-X_1%E3j{8NwIe;h|fnC_OhEkNMB`;BAVICZI(!;q)TzLusau|Uf|*tMj- z=rBk8{o!PdXX;_Ukr=pLNV`>)5GA+NI|Dv(MR^IuWL%6F5xY^^0> zUh)3e9HGO@sm&eAA=;ST;b zos%GY*-ai^WaQE$O|?#=Q%WoANHvBkQT|B`yzaUZ)aPi@Svgt-dM3fIaK;SM-7u@H zM~Aw+?*Em&WY~75`kD!lkrXbZH70xeCQrgs7biV}w0Q_gUofB%Es5(KhYm-9SjzsZ zzr;KYEv!H~p+W@t3m8-_k|g|DpstCvWU|v0KQJO5~meI+T!)F7Dt} z<;X7t7?@j3cF`J5KFTHy9ue><8JpBw*UT)}8b!b&luSs&40m zR5-YPv7(>kkk9OauQwvxZjd0zO)}u>%+1{M@3R4lvpp56$vAv4u;b#hN0&jGo%Xu? z8E@Cr`=i~cp^Eq_%d0;Rwwu>4;BHH}sfEhBjGOaJ2vy}zWTWR=s-%Pft3$8b;%CaG zk;aY>Zoffyh>f8w+JmCvj|HynjBdk?6$C>Z-#-73|A-w_)2OH0{r5$Wqiz@CbKz1a z%;G0%!sV3j+!_-7s#@Sw`Z}_6Zb-@CDv<(~0Z2^0^4Mf64}y>Eb_+wCzgQ_P_b?t z=sSTLMg^wp*SPIeJ%`&}s{%UIqxSDhMOr(Az%>e7V@LP;3NE*9R9ziVI6V{-H zUH06s7?UM1^C?NeHgr|-T}QAreZr=`*DoN5ZEHIMh|gbS^oHaCoBsOl-@<5QC6+6A zJNxsm>*dRi)4zCMx$GXS*QE~-TVv3;rA*Mu694-H)6eAkL%u4p)dNi_@%IPh%5x^_ zmwHDS?1WuN4uMOCANILE`I^&f7>!R#{LoDzbiia{2<;9EpB)Q>%bdgyB?bS4`B!O= zU06Zmh4^qI*f;do4?4K4uBvH1StEZWu)lbc6l^;oRM-1_FF{e5Xi8Jg_)*QEcmc1T zLKMsOJE>H?s%*H^yzS=U@I##aBc8`=dIg5F{yN7m{}~-u^FrlPCdcLJct>RS^~YLTb5_ewbBI7$%mR8>i&?fBY!~e4C-L!^_oXuqIQe9!b3a(-+4JWM8Rx zr++*lb6jky3=``m*w41;8Vj4iSUB`GP7D6?r>wz<0`p-jOsZ>ayN(sfqP6gHP>bt+ zE7h>=?7K&`lbVJ!gomYyCqLsoiip0KQ~lNg48`+IzGR%y(4i?%7DJ>Quh$({$=Y(y zYy{RlT&c@=?2&j@nh_HJ1wk%{k=0=@U(tUVmAW_z8x$k@JyrYEu1Oj1?rCNyBbgHtbbBZ#t^L0cG8zG0IveP1X;V9KZ0)14P)VW=YanjD-Ej$ z_sUsh1J=&UNpd8-`j5=rFO2cWp)h=K^t7TmLKo&fDnj^iEDk`kxZQ*F*}@J}3Lhno zxZf{+eVk*)s>IfBcPQ!zZr<(4k(`iQuYVT7H;}G&W*w+(#|ZB+LHU%r?#sddsvM4T zz1#0A+z*uPY;uDQS~dX(+HD-> z^*%jYs*Jtt(%H;qUuI~y&!ZX4-+e>0P0TfWk)ZS+%#9wK>jhME9HxtWVixh}Rw5oM z*oge!bHx1G>Pt^Whoj+g zrA6zsI+C}qK+}^KIc3jp;+gC{B{z%ufDb!@EnkmGT_5<}3|Ff8GA@%Q?wI!E#r*pru~�dJYNn z`3|C|hsOyZAvV4r9?Tm);3dp-cn=E7s)(^YXRQm}1}sFfKy#Uhb7@EhSj= zjUgI|qjUFdIRUvICog^AhRVqiWY)+k6;~pB5;kfQ zyJEdRjqNB42uj;|7QhFP2_FeTyGM0xu=(d#={AdNeC9_QuWUrs z4I;|fWsXg*(rDu9ZH8!$hQTdZV6h9PJ4)&TKz{J1Fb=e|)X)%*o$bGS?JKSjA9=lK~A0 z9|>YnVChdl zwtbORn`I;1<>rr4uKVilKs=LllttPV7-e#lL}AxWI084PX>z=f>^UfJKdK09DAIm} zQRFI4!`xg5P+|ODH(19@Vv>(?^p8{QFAI?^nz6#!XL&HQ5d3|%53!}=$SRj#4TZlRLF>ha7^OF<(9BP;Itc*p9 zJ6qP&@DaY*qi%B&USHL?!D~U-$9`K=Rf?YF%(*e*gd(*!DxHKE{B-_t^P-R$rwHc2 z!RGB;t(#8NW}2C}+u!d|!AEM8kh$EO{`s$@wU)4zE|PnMe3Itc-2;_a;}&9?L#2cF ziXET!S!6!_91^kHOW4o7xp%n~B-mc{xhD z%y%H?^6%$c>PLl=QdHde59Xk5%=~(cfV&!l;l1Dl9E0rTmEbWl^%g%fEn^QfGC6DWG-~MC&{z-4Gu$zh^@&$1<{Jr7*N!cf;6Cb`ESDTj=yE>=Q_a|{q zWSL^;d-Ly-D`O)=I;XUMr2Xz0RqALH2w;_N5!wa18w6{5AdpDG6ym%ld}CL}#(qfg zw9b0>*i|{vyb3L`Fpn|71w-0x z1X(x2v}W3WO|@pTqfS|>e)Z5>3*t1i>1LQP5Ju75{|-kCtxjiD7CJ5DEjO7>`o$Qj zp!ag=xcI$eEME~LI44gU@U!LXf#OiJTZ(=@8wxLqDfmDg>L0`wHUaDH*y z15YUx-oGOH^yiSCx?EA`-Umr&pBdT9i+9um+Ru*~vS2l`b~G~QmN8GkZ=|3Mz_yT8 zhR}5~kw-FWkE~`_DDDSnRhYNS>9;Cnzz9>VTaeS=B$(hklkrtBS&j0Mw<42k)~d0P zZ6-VXV#OtG9Z2T%&l0Fo_7qbnMn1gbu; zDzSj^Az_%VX}?6vMN1ZJM*8wU*~!ZrS?DbPVIhiR*@;(r$_*R0ILy^Pd5jsmdDr06 z^gY*`;waCDTJ}GJalZs4&*kQhv#;}&?iB1#0Xet_Q!K+|EEPtD`hI1sY3ZYD*rWPo zTV_tUNJGi=pAR{34?6w02cfzw6Ihvu64$bXec|=sR^O%TWdfd3uuZ3bEVPvMt?wU6KLtw(tKLDy64sW$ zQAc>mw1n%NWWc1&oqNl?qMTZpKjQ-XKbRvya9?lln7vxnTg!aJ!6kaJzKY_*#isGW zSJB4o_hm(%c&mTCR5aJ0aBDtNn;Pii;GXcAnu|Z%MFdMYVQ;1`Gd921KZE~`P#Ib6 zw1%TR(Sl3kuIJTjGBv4<*xg|{lJG`EW#bo+71Y;cT34N>Y%RGsj+4&_otkq#yTU*E z{Ea9Z-@2KEE(e!fQDi(TqpSVAumF}x6Q`tWpCLR-g*hooNvKWsnCmr;5BbY0oA;!E znEqZt7<};GE#>93mZfUju(KRq^?x*-gl~ain)W1?+%nho5dP2Q0@}wC$?iTr1^6!H6s+o0J6BH)*eumeWVg@T$YJ2i zlM8nycBINkb*7Z!Re*aXb=?qML?WVHtA>Z=DYAW98_P+LZgZ>io;ENlazL(AF$U#O zp;7_5;^h+!=T&OU)yKGeCiBCCr)%OEqnS^0^ag66+STlT$iDdbCzL)%B(YFrwxM%z z5EMxWK=ea;h>)MpbRZ*_dck|wez=iznXA9&G>eMCH_FbFOxJ7T1KarML4h0nj}?5> zGq3PajMioM$U9*SgW085i*|9b-_;G;yt&4Os3$e$HV(g=9b|=})uX(|qyvo&soXfL zD+^sZXADxg)i)?GHdT9s%6ww3DajRN&vRzkp`O0p?1y{MAIcrxDGmBySSSPI98G+| zNzmG7gx)*_@{Nx3X|IX72X9EA{W;QahZpq7^3Te`offe>5)i_OGX2UG!>97EtWnsI zomQQ$Fu88hKqf8G?4+c>+oJEp9C%p1A%DHG^mJYfQg)hew7j59jRtpGJ@aw8=WX8F zzRZq*CH6j?_>4}{T+|forJ*Lnc)V7 zXRhW6-dDyV%ID{}(-ain|K?wbIj7luf ztXyIPek_WFK5umCIoY@7KfGl|6+e1$rv3NhVMST{*R3WRSIXF67%}DMxVI?zZGq;x z>Q3eXHs1+U{pX*XDjp-93fY6Z>_2^DcCd*H;K4gw^z!C{tMZ)rE^+xQk2+$0M2kH) z0FCQ=gaUg)r2!ODCr&QSs2(nV$fA?i;X*AZ)%t6+NOVfHP(66;k@_hKoq$%~fn?dV zCgA2lb^10CX$^1&Oji8*aFn2@#ro#U`SEf~j{U!Km#r8Gj*}*7R2++tG!0~Md;cuv z;PXCf2yT@N`L04Ko%jx=J4jEKv5H#Xm-vZ1@?ImiQ0$~2dZbeMv$dTz`*gU64|UAS zkCtlness(lHIAzzN$`=3B66rO!k<~HM(u8Y7uiB6<|-q4xe1x5JM>vMM zKo41NNjSX5^FvUR6Z-8fxRFu37+XajcVU^>EA*OzqU$k?d2B1JAup&Un8RV;5%?mE zHLBJpMXjS=&cH+rWPQVr`*4L@O0`r@H+gudX@WYg&-o6ANZ@ zcg_if`4bbOTtkIt_o!Ug-en$+8%lr@#PeM6n#Y4iq;c*dRgQ-k!27rO z$KZw%j3$otV0Bi!Jw_eM_&bC4#y%l(RVzI!;HF;{o^(VEyF8*6M$Y5bGsQOb; znF(s|rYASqJkM~+Cy7#;`a6i`W)WS8&HvdvPdvC6<1tO42ugv5pI>cp(n?;cQie3l z{Sa3=OmwuR{)5@lX~9<9eHq+two{HuAHNhGIuitVopQeSU?ml_8xdSnRMupJgRbv_ zjXEOqXJ8fH3+;N}lM4}Z9b{t84^q6a6y#CNH%phSAi=9;88+B{XL97>QS0%Fa=oKp zSg~V^@TsheTw*f#+q`_!o|h=L1UZSml3|yZMaIa0_6sMeIspj3<3eD|9$VK3MAJp4 z7R)w!tDB?W%vZB$2JT^oLdWeXS=MsoD(F+b#hHP@O`%6=<3-4|FOQ_5KSkDO)u^mYR zK*|t(oIdiE3RJR+&&c2C>_3P~-6S4paw3F|S(P?j5HNFWVJ)^REPm+Kk$d0W7rZ$! ztDTkxWdc*bvl=_#TU;7dJlq6Q*B|&=_LKux;c;9<3<7Z7rTm7$BD~egp4rry!4sl{ z8sv1K;ki={ErhEt+GpsA)X?(C=jyZ!?i45%I(uttaCuj268@ZTg(d8|{WB|Jh5JbS zO4H(SekpAYFak5n<=&@yI9yYtv082VSRNwQUuvd05lHMMJ)4T&e-97GJG0})fQfu) zR%|MJh{mcBW-l5sphXM3Zw=lx-(^o0Y2FunF|4M5qQ)shi_m!-4-(N&lI&>w57E{A zF^f9B4*EG zEq|<6TD+fVyO}7NGO_EIc!aX}0HP6MJ|?}YbXSqu$WBnNIB;t<|MfYL%e7O=KYIT{ z!%xuhcu~|IySb`tjng~&kku#+32UE0DH{ZD%u;?saraVJu7r2efxqCA^#`$v(GDYt zz-_PDJiA>~V>pj^Jn7iyw+xW1OD>zC8D1HF`vemos(I!*XE+|8 z54OJy0~-6Vzdmt(pw1+=A19)f*UAR2aklfLU_Xn%G;V!cnXF(Goe+e&hl>_ZueeRr zDD=em;=7YS{ww{ljkz`~N%a_tJ?%Cmun^P4v^bvZ|D2L96e6kq)+hW$Yk*B|;79-| z%)g_938G#oO^a4olsJR$VCb(upl^jDJO;6c()OUv2+`e2rr5ysJCYzK1c<5s-~ZCZ zu~%9eX>S@CfmJ^b7yAivv~atZTHad+JCZ5%MqMl)wYdOK3gXOLrSON-SSbg_!3X@O zJdx~xlK@Kac{aPpDkl%pf_5RXMYmeQh?Deq!dol|;=y&w|`@X)n(}=y08rhwy2x6P0okB>A#XG^!!tlNxgM7#90*@%K96B3T>JF&5lxtjG%=vNRG>_KbkXN2N^>3`D|OLFa8@95!b`tbYY{FN@G6Cp)3kE{ z7eP|$>m1LSS1JkIONP_LmyHtn*ig!3LvF9|^dm`Cb;4>Xz!Hqk0#=3bfv? zi;=?7<;CLXGs-Mvx#l*eRp?}Bb{==|kDe=+KC8PBNiul-4p-Lboro=XqOv_{&o?XX z%8>-Bd%1k+A^XS`C=mE8&;HBNYd?K`bH-MwZ#3W`3wJ;Mcep(d7`SIwvLR3FEc08c zZ{gCkYm94r5#d=w2+{c!p}Ve}Y1%`mZ#_Dq=2o}sv_)2?DEj7P=cCk;cy)V&g$R99 zwxfTSpTH@AfnB$ZXi3G+a#E;!qJ}?%?r#$MW!}-9yQ)y2n*7!Z^ZiHo z3M4Sia=$csUs_k>auHL0jUu610?yR8htWDlQ@tHHFW`)!<46#90A4Q`&6XG|B#IRq zK`y=$R3EVf0L~KP#}#*g+sY}vmEf||a&GyfDoz*M$R+mm27z^a61vwxHq6B5- zLF=5qzaZfTU;Na4S?%j>gfK;8q28z)C2r9?hyHM*ZTsZKona!0OZd$iuehiHQs&-@ zS5j9(;aXhL>Eh$C)_ZponV}!`rLN;RBx~Oq1?w9mm}sJF28q7LN_*Qn@()1)?0FB( zP0sV1i23NJ_atjyI~lw%PcylYkmeFC%8g9@?Pvfo5LLfrCv#6uclU(dFvmt1#`F?B z^#~+69?slS0&i34knp-a83gTu7!ea8LCHi@`~ZZ3UuP$&6XtgJ!WiefM`yk8=m?+^ zJ@UnQ{`qxGFu=%G`Z28rZi@PONN7XZDQ9{$;nuqgt4AXFK7ot%lMs1AKzUC>E>#6T z&NXM|4!g)aL`wJC;3g1SAxQf(Ma1Q}Lu&RcDUoO#VInn$B+lfuIX~n0Fk*e{dMY;= zRh`^1N{sd-E#-E=haTs!en!dIuB0is)1iJ>7kj&rXU6#faJ_Q;8H$PZ$w1G&w8pcR zyeAgqHn_4+GxhjwId4_;V=q#>@sqacj~(@Qq@P_JP*8FYAUa))YxlZg8XNgqWzhr< z5qY`&dj`E$o+H2^n@&{oNsr)HcSiG3F?4TC>-D*1^#2T3U|cm~X?d63_ylvjOhvdZ zrNZr-YVuz`6trl~l8{({o}$Z*584wC>c9GTotpr@oc==jko%`)14o%Q$b}y|tXc02 zzum5&u!Jr*IT^K9?ymI4$e#bhvvHHb@=97yhVu*Gw}yC)?eD<;%GF4RW30UhZC)oJ z2A%X*Q%vv+FW}C8@ThHJX{vG-Cjr4q1w{`PPg>q>7+yB<<-EVFMX4P!q z`f~a|5m6nGj>RH8?`=~hSP7u*GX4zlNX<$wCQg<&AAU88GIs z>laFC4P1J&E8nLLby#&2V}(JTD<$Yxin8k!q^;e5hRaUd92DgF%g^!k%w>G^A}zsl}Pala-)oYbQORLP_o zy$T_jaLKrU4rNs=v!W@F&!?^4;TH{U@K1~W{x5~CZ zPal(VeNZrx;5`uTl#r&qA4m6vL8Xsv!})SP&W2?SC{4xDJ_2adzFh? zVius`mdZn3<@?v3Sgi!ZBW66WdBHZ@9i=-mhY>~JH412PWjLxlcwZcJ5HJp(OExoj z<3Rx>$#1LIlr~hJ_M!2~bKD~542Xe@$U+3Z&<~~dX49c%Bh;Dw>P~NOeQF}EBl!j3 zZ>{4pJztXk$4AxcvIeBF|E@|W_usnL3dS87e72JYx7yQ9>vz6Q&HJc@6ERRW-*9-! zudl9a=#D$Lm?U(v$Li|;00qz7aD!>3-k8~8pKVGV&YsZk7WdG8)Spe!J)hm3%{9J> z{%M3pnIsvAH2r)`;b%XLKQ}2#Z1%JgG z&kD4p`z1+1&RTQf>y(dK29#Z&vRQ zsc{2JZ#M9inl1FSoB5b++!O(8CY*r3Q)1N7dOXpozWbrk^sq`Sh)wXBe*DH3v3 zpia6B?twm?$-NQ`AhnO`rWT?3L@SQTWFX`jTOg%i^=< z@XyB{>~KG|q+aiGZ12F|c-7at-&1wb{sQ^^DlGw1_QgFCbZ0(C;^or~-qR`~+s6(mHSnbb}U{v)E17cvhG4e z;^VC@Cap8|+=L3!<+2V{2KKs2a?hj7* z_fBRpI>rD~-Y~RcloCi9@K@G1X+=^E3Q`@UGJQD#fI5x4U!t`pkE8!e?sdhFyge&v zK7pkY)1*I+=WPq1G(Z2YJRw7TrwG_TwlV=ge@v*@rhCtrd#x0{)3Ic`aVH1uk54XM zmwB}wj5P%Pa@@z7P%+2(#9mN=ga)vTYb4LXA)^q~OK*b`z|}g=NMdbWHTr3qMs|Af zwTvuu=&9`Hs#@Qwf@J#p`n07ukea~mj{&YzRaJ`8&11J8wtR-Nnk{Cae_K1ew8W!i zqkz(UhLzmFJNx1=>Q}vJiHyuT# znh>{<7w@M4v0m_uuE~Qq>>jtl532^ZN1-k(zn63*3jN~7n3K&t_gry&#~fQ)L_^>- z>N$GW&~twsAifS}fTJe@$j+86B;8K7{>u?gm=YKrZ5YOh-c&o`9Qbm)h{BfaJgWTnUBXb zjNMiTDGGC4cA8y2lQbIix%(Y4Ru0WK z;<01NPN3WuhC8e>O3wJMxGfOeIrHgaYpmL*!52%DKZ>(#=*^usG#{2(J`?zX(QUmv zsh6u6bb0h;Pu%8(>z!SLCysj88T;n@J(FOXy-g~A{wP(V+*oWZBzM3=JcudIhzj6fxqSN`U z4x?UwZ=12t{+{`xHhK>p60Ot2U18Mr zef)p6v5M}l7l4pJ5h&Md0fVb2atjXF|4Il z*W4%bz95rf+*gT3iSuMtfznk8jQuRI&Q1FF2*jgEJ%7uf@Ps!+P$)obU_IwQI*IP8 zues4(9kU&`RMg87AM>I8|Lt?tu1`Ai^-Gh1r-y=-_%IKBpR2*#6l8@rv^PkdayrY< zxdfeAL}UM8kmK)LtW1Dk@9zB$C^Tslq_KW%Z-R7hT%d>`CD5s2iL%Z=b;m%hfWXiR z?ckq>4u}B3UrX~jIUeCR;@5OI-Jl-33OA-3*tsAgx#F=IqT&cwJVmMcj|AZM*Yu#=aH&1-5iPjrlJ+mGnyhpg^7qe*+RqvI zL*Q1=&KKpDB(e(zNMd1E>a8W_?`8*$3E21`5E2a}9~U;pAvb-}uNg}ZYm3`s*^XLH6E(a%kPlu@B>Yyce`sp&u$fX}0O4Alg|cc%+Rr8dyzxO^(Isn;=AH!e_o+{jzzC;;dr4S-3M$|OiZPOtbS zkOextoq+8Cb~7yeY@1%y>xAvkB?`VO-+OFgi7(k8T*Sydn7UkU9!1{}>O6Kd%LXG^ zWKPH7HfF;-QD0Gv zQS5O~mfep@W6m!i#gl#!{Hv7J^? zY;8-u-KLd=Gb%MDZ0o+(AO%))gsT+i^G9XWKIgUQUzXhIr|t8)l2=?!;tYhi@9YN4 zN0kN$8C-8SpV}8%+=KNZ#uv#EZNvw&AOc2itVBRg`e2mhrVO((E$49;I$;qJ0k2~N z@K|GGx4Xs)K7;0TW-Y7h5dw53#ePI7%=xNpS;q5-MNGHwS#`W_cHP&{$u1n5y7Fe? zmGKRu1y9h6b*)n;g=w}Cr+*X6P=l>(GM>i_<1%+r3O-Ozl&;KOu8SMq$ySc>T;`Ue zTZH*B3tRR^)h-C=UYXY#E>`+oyN}A&TTF7F?*#N^hOpiT8!kgHwQ6-MVVC?^7ew97 zye)M8bbY4K<&X5m@4{W~Jk#0Hn`W<*)eBKhpWa9|YBa?dwth|RA6jwce53mSxk8!-5SPcfGT8m z&J2I^lVhC-&KPJ~-3iWNON^9ZfT_j)r+*cU(SBISQD^{(;DYO)@S&3oct|pLPdYt8 z&yX*yzSk%D6llgtk-~m)iN7Rd!h?;9S7-s!WN~u^yDqu8ymgh*x?R17k?%b+lUX8u zb-e%AD1fV9n0@e~B?lft@tLN@p}k_a^wJb(dk<7R_XX&lg4SVgk9sXvutF1DOO5-T zN~fh0exJ{ACq$j|13)E|V#8~f>fikiZ?k+dXjL@kxD;R{2{u{daNAquQBgjbHG77jZL&_4vA3W2KWWr13i7xd08%Cul+@>0V zEaKFLe{`2PZ&YCHXyyq9Jfb1fko3>yg{vw#B^Ps`8vRbZ0<3uLE>vyv674*y3bB18R=+*EAmMb4B>XzdyKyjA?Og|*|?J#r&Z;s>6 z-w8TebpElb;#QK;rQYEA2#KM`TVxg!TFN%OZYJc8QRC>me`BMx7;vz{Ou3u(g|Ty2 zG|y)(@gI3<#*JtI=V}DM1TnA}t4!mu6{4PKB?|NfcPo`zJ zhA*D4TQ(MUDMa<;5$Ab+cePNYs{J-kD1(^IgR4Xq?}U|ljkpyyND-E#K)~SpSpkQI zYoVCUA;Ym&>}3)K>m9_6wo_ThKS;aFB%Bra&;}$lJ%YrgYK%4Esj59r^|lbl39tAG zzYVM!{N93fv`w}z+|BsZXDPQFNX5JdR()?_La+QLvQ5gEGdBO5a4pRFHc+RvS)%L#9IDD1zc~P-hlv@HS2QCCs zf6^+RbMh+jM|jfbY<`&4+boqHVq`0Ny6HuYP8A;Hc^dEgE-bM^>D(aZVph4Qe4i>| zUR$c4Pg%y{Rh*S0Z?cKSv8HPT4~?TJ6u&K48C=8Ut$buqam9p#k5D zeIyFSg!vfUB&-8y*&6)$vyFETkj$SQK65a zu(HouB4(C%gr2WE`h9pKQsuIi?|@i`7U?k}tR${G4y3`};iYWkB(&?CaUF4-asu$f z(?cn8XPl3C=bWrr%g^|VW{zds3?D@x+CK><+X`DGiQ#zb{esm4^YCpRoG;k7${R4T!T&}V9%KzZWP^bm)9O}DYsmv;!!vDnV{#wfpFnNXk z()vM<$32NW-G{wyf}WeV9DSK(;b44;H3ZKd^H*Bs;|r*no8K>H0m<$#?yS{T7paFJ zfuJ!`sT1$w41#hTq2Uc1;oa*WFG!AHUOU+>&r@t9O|KK(HSgnuVQ&bUOcFNjCGogw zuwOkYu$i4!x8Sz31mOjmtmVcYzHasfZAct7 zn5FgnE+k2MaY-t0%GYUoVWy>atY2LIX-3)nL-;#kTu4(t|0lxP2^Wzehf<<%jEJW2 zZXt=*-kJrBW!r`BmtZIuBE#IAlpiD;v zBV>f1xj>lsb&)1N_@&}6=?~L+`%O+xDy2JuvB_e|UgL$2PhrTQ$Hx{Y5#rVG!+I2R z&%f#y4Bu4KHQC>NV-cE);F+ZQ#?>u!4{k)b0hpi7Ltxw0gq*C351&N7yfEM~K=#{r z8f7l$gbR|$;G@QB;>Xe)t9~tpL77LGer>a2C#y;6R@%xpS)vI%Ge{i`FA#^QSY;tY8$$a8DzgK_OIsMFsNC zbU1x_-1&dh3G#2o0MXkzOII7vh7w;3YR_I6r44(G%jyfshisiv>j|iAEt&05uMF>r zlh%z=KoiaqaxHTr=Qu#x7}=C{l=CLryZmJ?TnK#l2;A(Fr931PpcF1 z73syaf~&6W&C2X+kD}>6(iWmp=+Vj4Pn)@~(S;e7($`&OsB8cFQMcl^p*gVaMvct> zHtqOp4~K`fY_0Lx>P*ibKdAtWz`tuQN31L`<@TCm-ubY{5O=hO4j}j5hLNDXxF^#p zo~awmtg|U8AmdZVFCa^xTP@QGbfN0$sLB~7bofQjA(Q^YOWeEeRUZ8h8p@Pqn$Zq0 z+(qira^V2Pu!D>h%>~oexHg7d`aL_MGy6UsfT5Jnk1W5a0pBLZXAJfIXxvv{t!6wj zXJ`BbAUCb^d7QeJsPojR@!8uPj>jXq{@S^cB@0G$9QEsfLRE6YfCRWu=az`87wFpW zGBhCLX3}oHQwEt`$|on~TTUQy3fN@N%Ng8S4jw!iYBw6Ud@T(a=yC!h02Vb2O}ayd*-xPQ>{Dyg{hxZdrglG04d zHd!ssEaAE{uF6B)=R7-oLs2BfuKe25;P>>MUoK0OV1D7M9C?n-TY%r>oTql<&I~yE?+_ zog1C$^q2OSYp2wrodEc47aF8qyB&ul#i&XlI_tO4Z;vR(ZfkpKXi%m=f zxCyt59s&}Iv061x`_=%JTGOs-U$fuZH_)S+{~{^{%~f6_q^C(O$c8n#zD04)=pVYr zwaM1{q~ErzUA^$g?yGPJ){w(U*eE5p@3TPFNQ{q8ja65zM$ZJN?}L%-Ycwwn zY~)k;0Pl!wRu6<_S|st`_qQFP(#dM8RM2TrZ0wH^_%IabB;MFgFa$q_yI=LKWQ1+> zw0&ZldL|l__@;(p&@O+@h}_$+su=S5EIQrN*QY0DIx#}mzjav9$+jd_$VRl321NQ} zdVM?D6FC$WW~odr=f&hS4`ZE86ZI55_{|3!3jaOJv*>laQ0Fe89p4i*Va@2VeZd5O zS1i4{Vu*=Vi?B#+3BV*fT$GL%Xh z9#nBnr1wiJsZb-nbU?Zfc7k$<1o3$ys<)d{PDjb8hKkmVj6?M6(p|UlZu;y+whQtc z4MI*?^Fc?FKoaWRYeVFQcGtMdGh!nAlMK!l5{}R$P#G0rwIo5a7%rB(Mqwr&jsp*N z#`Dv%kxX`uzMIh{`DWL&y@JbA)mCk%oGLUv9P;m|Ro*EHlnZ{LS)b`;u$ro)r1_W= zDokoHInJ(QWz2I#0DJ4hrO4B4ht$ z<3B9~tD*!y*0~542R%+}n?-OhwDFBwQ~oP#gwVdTb|o_b*Rc`;BtO z=I5;^>&C1>E~$9qd0Dj9nC_p0nsS4|=tSw2&5#F-yb|J}ct*L!dJ$L$WS5bmWl#`) z$l+K2?8}hbksfs2Go}lY>eoNQi$a94}} z-q&v%9gY~msJ+ih9P;+SjYe@2UkkR-Dv9tXZ<2(EV|qGg-SFu`&fn|Grz(nbFkT(I zI@IsaLf;kJ4s!e_zfsj?J8AaVk{G?GGGfqxJ@*;QEK!w+L`b&k*SVc46zSE9! zTif`D+PGwZ*NEBBMEqA;{-OK;EykFUFD$|ZUK_}u#%E_pw?if&MF*zq^-edBH+UY9 zjSxpx9#q1^W@lyq^>v=(r`CCZ_45K(5h(_mcRU2{kw5FO#rzr+G<5od;g<5F{iW5S znk~os@g&SyL}L&Pd^yZidq8h4cvJgX6&PA{hWDVty5|OxdyEY~-$qRNl{>283~%(T z=ng(&Z>2y&HY9pjl%l?<`P=qZbmq3pMFF^AuYQljxiTv+v2D|bF#?(y(bs0U(+AI^ z-e&%tg?1&&D*8pDq|TKuMlKA(UU`U*!dql~CE~rN=&Q=KE+_A51-=_m=_1W%sp=H} z8$~H^sR^!L{MvHf$9l_{o0K*O+))qDI1Md)6RXqFM4GUVwW(;!u~G4dnkY(TUkmly zB)djcKod3S|HaptBS{`xP(1zrEC7+|P&R8&MVdbms?o1@hLnF4a5*-mp=uW9qNA!T zPsl2E%HlHsJ==r1XC_v06aF>&=fKyt4q{(21+f2otmn79kk9m;8)>cXk^4(A9=ep3 znkZ28&Ybn&y_sxrhp=wKlntWognjOzoEkY_qjMWlNb>80v@_`F*HpNhnbmt% zO(#6C2TG*FFtZO8NhG3_g}rexm|`c;`)3nKoD9)A)PwVVMub3^8kU+@bwuzA7JM9k ztUt6^i(PYp${n>Z{<3~iH9y{0@J9POh<&U1H|Xx*M}k_ODb6kCc_*(#xZOuF{iUET z^Z}D-MjX>bEvWpR3GzwbfvueP=eSYemz%-?Op?s^XZHIV2NLNTL1n?Kcf|PtrRA@< zRO&l7=ei(=vAr2TPEko07Cit|SG?w16lv|BIAk&QI_c0(Z|;SUY``@MKiUfA{1usc zV7tM|$cz3)Ndc$`fQsJ~ZZ*6A#ZET!!CXa4Q6G|IS0m_35pj5qg#)c~BhD9n!USu- z0(ISDyq%gV*>;G+LfO8cOV-O$f_*_2q|PM`s#`fH;Tp-3nt5n?ex-Fc2;enhCXtG> zJsE{$%T!sLtlI*hd*e#h?-Z>d64*i}VA1!hgT32f0 zVT944_?>T*LFDW&r^=eH57Qe@fduUAz|!@h?{O>rK2TmK`es7;bWgGo@*%tVzks~? zzd)%{Ev`|{*vLMWzvnt(mXUG!S3BJ-oYAOuRTA!#w~%DZ(rQ2LSp@t-1gfgCjFEtx z+hFLSea&lF0c;JJ#<>|XqHv=vPQKlF)w3b%q-5ExC!g33q?CUgHwX#XNdDZY__2ej zB84&P>^QGVhKVy-9Fp3QWS3H3P6Ahu@$B8vc$H6;%jcW6^>OY$YMBs@-Vo?ZH4=-Y zqYJeBi-Ir+D+FZL1=t3BNgmFl`lEY}AoCjl_l8h6G4563w`>kkoQ~&!*5ac8tL#$)ef*gk#nP-(Bzuj;F_xH>j`!(sm{R1yi zE6d0G^u7H$gtOrtexL{)DIfWMC@rD2eFDCb-zGo1%90Z`i70BFsV@r%1+8K6t61L} zIlv!yqY(~l1_3-hOZJLu*}7`Yog4$Gjg&8L4ucKhe$Eru{zE{3NL8v;2|4Gj=~i_lr^&zq47$&FoReJ<0a}@SH3|UfkmK7C-+m3mxK7-I zJhSh&d8q+~ZP@Gq@t;_k>Zye> zvcad3%yn?)RI`gVcIB#q_8Vd?AEEmsn-L^pwZc`^=hr&qD0g1Z2}SM6y}kFDwba zp=Fha_;6dHXogfrA;vNBXs?((=x1Igg^PC&bso8D?zFY@u)4J1ecn$}Xrbt%mAbu_ zPLt%U%y*;f{w)0I7rLuE1hRS~S(78J7UuUpBzT0Pn=6Y{Z-m!*WUyvDeIt9njlSaF zDFQWV(X<9_7$dX+R&xLNHotlF3^d*m@*~$tTN&LfC*0MvmGF&{9l0K_i1%)BArlQxlFODG*%Q#K)i}x-^ z_V`0;sPD$b>@%#ynZzr7Xg3^tg0A{2TvjL_p~-Y6{Gpg{sFWm`akyD7*LI@|o{aI0 z2Bw)4?i^z!hSrnQt+ehB<#pmp=O=j53uE-rE1$fwF6Is2AEQMd_^_1fM$UKi&yVF_ zG_DtYZ5+lr4nFnVqEBUochN-)_&l{eK$61qxeRO8)3k4vb~4Zlv!6J3PnNf107I|; zDQW*E9kdrDXWznH&p@sp!U^43d)UL1+>O5x&M(hn2z$1{btj?_qX9mzvx)bQ9f$e@ zF*1BYXJy!$+=HDm=3d!aGl1$@@|Qj7>ZDi$z`b=;Af{PB-F+MkCH}gy7Hqz*)uQl! z;Dp6l!ucrdhG!^y>Bknn`h=(48P{ctzk#C95m$Kk9F6HMsT!?50wL-uv#I&|>XRf-eqhSTV_HcH8bn$o*$p#i0l z@WK-(roQAUiR{UjyRgT9^WS8saPd1ZMc^;%AJT2L#4zLcp(5VfPN{M^`$^ZfVbMup zf{mnAuPn1(MoRZ2Lr}$|9ps|hD8=H?%B6G0ib!@)Z2|s4vCIwQTNhTLNvZlbe3pV`bH$whV|r>RyieCc{tRwii_<(hxL4gFar5;QHTwC3zYng7}kGJdcSLzv3$ zpHd)f$G{W>J68xdXPh=nC7?OZu==djlk<;J&el+cEW1o*t^2=-% zt?yFcF9{;UL5>1#qWWH}Zk@d&R)#Zdl6LMhH5j$?#uilfyTm#bJLcXHkiWHd$EpVNRW zOaa-5GZL`xJC&#*EKMzMDmpEH>tYrd580)#T15_+{xd7Oh_j)Z`w`v^&6z66bhDG_ zO7m(Df3-*d$|*>UjD%nW#y~+g7qb8YhA;#aYTTGrMzY!)5krURnd!6s8I(Q02mdk9 z;lJDBo)AQB@Tz37Jr@GDE0QCOjcnn866IfW@kBf2P*KhTgN0H_=o{`e6CxE}ikcxkhUloAAE_>FRIC6$4U zq37>emsn>;XFvIBoK&d}xa9`*syb+y8EKy*ob>x1-mwK6L&%1l?iZK) z;fcx~>m8H#>|0NM1`jdasg%1uif9sIi+)6033C&IAYFt4e0xmdhFiXUjlA24XB!m9 z*?fUfgP+gOdH&pxIvrdq)M{)!!$sA<`MqIyhy8d9`?P|?8NGzM7J|0Yw(ZDBS~d2I zBDWd1$2Pds?~%$`N}^jc1z{Z?sdr@vkvi#K9yg_rUfv=!Z;WC=C$k9OcEX2YgaKd| zy3eqeiyu9x#wfAx{8vQwmGap|^-PEkRDjYk&@tOLNCY)t+=Pchqg!%x6wW)U6)82LYdw-%Unwb!_*| z`3}vV5b*VO*HfbqCJWF6>KZX~U(|1gp2m&fn5yY$6Xtn8pW<0;ku8&Qb(D7k_y^?q zmmd!7pht9&X9@QgJ0@})&K_`K(C!W98k>dx%93WyGf*Iei& z@}cjowjr*c2G6^;(SKPtio-UA#(b-{rjS!~wc&eyWOS=1V*69R$OHT-Z==P3vv2Q@ z3Uw|#$$j)voT?1&;7fH@zre+=@;)IZiIb8W1S3ed%4y<4;Rv|Y@`rGmzrz~`oMd$X z$j1{wO;lwn#W7aTEOSFR(SF)jJkRprXD~>_LM|dm>)vwXSv?rL(|FtJIkp&b!7YBr zVvm%i!h^}T`FAkevT3bYeR`=7Iy@HknGY zZ|XF?Q>(l8;+y#Lc-*BU{WI5AQ3oMMK^=o$ zIxU9-WAbm$Z2y5HjaT1f6yFcc+ZV-2H{?(~&Z~x2rJx(NZhc_99%UgXOZ~a0!GPq@;f~S3 zpW+Xc{rjDZwVpUHt!nI*n%4gqXhSM+*)!j*+onW+Y>1JQesxj9?&*kIf0| zzQu@s5MUD2UVx&k>g9MiSL&v5*h4t4))LXl z@V-pxm~<9+8Z0iwz~SxCX~NDO5sR3se^C+J>w$R&GXor_xzK zN9E=lg5CQ~Gd+$q1~F8XVmCid6k5q}_P^FTw9@C8<(^Z~ zWsMy8E*{Y1W(;0hSoU=TW&wOwcY+D)ysK0m_+W}x@26xSO<0;0Sjx<{#HGl+CTbFg z!=q`4s!tJBpimk9HuCP1f7?b~2}yF|eukVCJQ$}LL%%ZI-PJZi4mcb#0$Ka|`7j?> zn$jiSyal}*`Eav>bykb~$p?&O-K?z@I0Uo!!3)<75hJ!xuVBrv47#S&D>uY6IWCoe zC7`#VPq}ri)^q=w&y}OCdg0d*fH3@UH@O&u{c`k`FCqRri)YTlpuYpp^kr{wQj5u( zLPyN`YU>I?65I*DIw%JWxozxU!7q-e@h5Bec~@~f1+!+tWz5n=J+}G8XS|UCFCgRQ znVk0Ll}l>VKq<`HIsl@74En~Mu z)(xEP{S#w8DXmL@RQ{_vv}cL^alxGLH;s;ARzk&<(mgFYSf{(gazhKIQ*3pz zjIwyx+)3KIYHEI`C^C1JYqUpzlH78=Ke<%UFhs(!{gHQi#XG6imh~t9N7Gk$HT6FJ zqxhynK{^JCfJ%36AP6cYg3=5GLAs?kq&ox&iGfH-w^E}7Y3XKkjT$f*WAWv4e&=`2 z{Ri%IpZmls%@?Ce+OmfqCp0`4A0_W#s&Il2OnN93g8IG&tRu<7&!zv8Y6eQs1ZntS zqG?sk80y04F+Nf#eBSY@=t2sRvcGEw%ZXy~SVhX@OhMyQFa8PWXrd?o6Pe5#jk|Ke zQ*db3`Q+K_g}32<6+<>*q6g(;iFz69KdaGMI>YNmXcBFlzOD34oiZzW->JGCvrg^S zcF~udVfRRXBGscu1pyS62&M9ACzL~0`{WAZk|^X^R9H7Y;*C@C9)G>}M;NF%ryV0hy4cuLnU~A za2_2;J4o0f*DGMsDZ*#z?W&20|%kuXNUyc%`ONRL7+Z zG+{Nwn(xcY?H{jyli;lGh>B}e_i0U{T2XB~ZcA9N>apOJo(FX*SQ-uvC3-KuE+eq{CVEyXIR zE(=7QrDz{?i0bxT1oocI;@;er2L;kz%W3darVa8)wUuIFL5jm-J2{E9jh(048jK-xC2rxQ#Wv$y-nq_^cgt zhZ6j5LEc~sw4y97#V&;FwF+9(1gxCpb&oyeL>&tqX-dW&GqJV*ipaufFcV5sIh1s1 z(+?jIyI;lCQAUo99muw6K^*ig@+JTz}4UeBU4M| zZ70`S*ciFU9&|DA-tjHHb0~NgEID9?QPMzX?)$8(f_|5TR!G29@2b2PvZvt~$OT+S z!EQX-ur9tIp_@;(=ENmDM1mZXsI)V^{!P$J^=Sw7G2Qu+6m(!?87nZAHkwlBrA?6M zKNBba7PAwnDepIw=aAmlxIwz;Ju9=+VDf@ZR7mU;R^Ko7OLr>F@CR?fef6tE8%;WX z@)GflsXUq~RsQ#eVz-ulAm7Q5<0A5d*T0DdCiSC|dzk&_R5+rKg;co{oO zb)XI=^)6G$nWFM`kYNC=Y^=Rs>Zj7sJFRoiyG=k$p@H5mIL`y?9}xv#&KrSoh?@Z2 zdUPuNU-4&OR6P@)s;E3e{`)Vjcy9vZjK`gxHCwj%#NC>!l!^#Zd?Eg+qhi~mlSf-= zyqyS|yE2pIO>U?g!sdf!AWgr4op%YRCQOT6%SmG2gYRUGWCF-_>0(vROc6C5#e}HT z)gj)+n~)#;k{6hcQGI-G0!(;2-0+XhL-L7TU;U2neahh8Z&>jtY%QeY8WfZ(KcJ`& z?Hp9WOq<|!AVx4uHK_2$oSqXyKWDptx55-;Jx+!s51L*4+lYb9j`_?Y@@a8$;#Qpf zvz>rfy%9OvI4ZdO$AO>ot7E*FxJ{{c4gy{Qs(GuN?8B#@V|j>uKqVbkb3cVWzS<1x zn66$a!G193M>EKMf%^&iom%MneK=2%XDB_f zmLUkSEsDz^c|0zJZC(cw2v@R%n`{47e^SjT%83iFQzY^#`p@OzJ69bzj?svnO@+M%v@<0%QZ<@+;C z=kiX#v(+ICS2+2BTe;%1tBWPy2gS9&{nmj+i1z5fPLm3F(EO&%pHIOUwQq4}v*j(8 zPlqQoQ@-5)6bOm>E44gO{fd^@WeV-%O<=o+qd$}78aejC-OQR8K_k6k#3k5vrJH1w z-$iKj2yt`2Pt`%2%(B#f3G}-_J{fmDxUzV@g06ocP|iH{BG}P&kGMhHO~oVYp=eFS zOaD+={^{bLx43Ww`=> zFm?hq-9=7uQ70UfA<{&5nk=4U8~eme+_`$1=XVIhbwBiC11-_{rmSI8aW*4u$1F5kUf4#=py)6(ro78Ncbv3aTvomw-DPx}4x-tFheBbxd4&Fr z?{(pcdU6H%5oS@F3frlPEMUK`fqdS@wY#zoFPmDtYWNvnR)}}khEev&t$RQqr=b4d zFC+#F)R`j4b?06){wbI(ulVTP@g#b0p=ouIVMy#%#DD6oQ*)z;_ktk2HZJ+`;t-~A zij7ChnbVXIKnZ_A0+d>POjPuB<=vZ}zR?h~JnzHarMmbO{DOU=`k2n!JF)$!7_zQ^ z&=w_R9`dbsTI`9a#SZ%S{!Rr@k_V%Z=l)-bn@s$}9zDF5%0Ny9tl4rEQ7QN}Sti(A z;kPo?cN=6Aot@01Q*v{*THrDERd+?!AHS-(s2K9?>8WR!x2r1 zCxjO?=)3sxqy*t7UtA zT?9LB^TPp$S8;MUG}8=gmBz-)D7~#0D%4BL3c(?>V1w)!D{iGxq&cF`fuCXz1wD3D zU=xcJFVsfNeOwiy|M7gt=igAAzWbM}z&gliKjW>#v?1%)qN`!$G&s`9|4f1k)DT+! zQ|vq9T4BuMYdmwaCp!!eiGnHj*jR$NB%WX^qW8q#L~7&4BZ?@8{(B;m_x?UI7vy4M zP4f&l09R@c+FW$o!LSCrornTb51X9t`0V%SgeS>}MUzOv6ZQWZ5PFVxJ368L$CAb7 zp0s{D;E+>tMnWa5p|uCIN!aN^vzW->Q+qeA$$cd2KY5%7w?OTd_Xd03C?j9l?_9do zeAU~ndFo%741V?0h)$Q=8NExFsjz5aj$2iu>)Q#g9dpMt*A+-VAxkYMBq zbgvPGETX$vo*%LVm(Emul`>(tpgo134#c1+)q_5U}y zf=X5WRN}2|Zyb#VFRK|0$-R0OTlpENxrXV9#?8&txEj1Xs}aJ zTHRG+_Arby^9v6%`xXkrbMerxh`zUEf_;d))sG%_`6^j1(fg3;w)vt>fFIQW{Z4t? zr4xlUT+PojYnTACvul^W|L!B!XkG>biCdzGfbr%x9|UAowKvA_OJg4Uq;cw1Ro5@{ zknQPk=}5_mA&;bim==6f^B z5|&y0N@1(b&DS&Z>3^EVF*6R)oeD7e0M(rHnMh@OnBV#Mwk$&iTh(E#JUN|bwxe%w zGP(#Wj@2Q`d7S%09Dev#kF`TCaJu|&P#!KPdi#K8oq*jzIA+Ex^~LxP(;K0L#9|^y z%cGZ`VCL3d#OR>-(44~6Tp5at=zpU{GWBpdMGE^=^Juj4L4W?dDO90%cl+;FTBPfl zui=#+DM=)zT%(5Q~w+}+Cn-7xz5w^ipZ0y_vYdx}=kUt7852hm#{-{6`kiSiwEoYmfg z{m17obhbX8?VFRU0ZOP{_$zrf9jA7R(85DqEg%o=_AzY5n3XgfaEh!xds_c|`3>x5 zWuOc8Dt8N=ZY~WLf&bAhOCmHN^l}|tEItJEMx8_G&D)sn5@j{c!#zWn-4%)cl^ljW&olrQmL2oD{7uti_!zqTeu-yKPR|H0 zC3xZNt@Mjv7fk3pf)y&sZzxAmJrpZ5aI8DEtcG9NM#T)9GWPP%^#uH-eyR6r-@pkmN-lsJ2_DucXP;?G=sr`=Fx4%<^mK zbH_DB1J2zbk7(u%OaHYfW-mV3GTwXhtUM5l;twZd-fz@;s*h=*^9J__?_8vN-ZmwG z{t~tv02HM=iQwQ(8*Z*vWEg%7k~20QZVc`vQ_J)*R?#~f<`q#&_pET&R&q8Db-Oo* zXji!yU)ORrb!gfox@*A0_DAvJ5S}6hDY>5Nhw_{olW!JTEd6dJGaTPK)`DTzS)ha+ zsHIWa%Rc_|-@*Z$^jpmxu?5#X%vUaNNBy5c|^u#uY%xA-QQCeedo8+>~*c(og5X-BKy2{3q3 zgHnXm`0W*CAcD?DpjV=Bjx(CyEj{)WdrWRFl)qEzs_CudrnHIk65lJt>NL3>T>RKf zmbl;sU(y#6#q)C3Njo->(l*1BrU<$uD+$iDdNl2ZTb9%gYk)<=6r|W-=|6DPDR8Q`_NtqQNjPj0Yajohn z|1>QSg@TEzASH1dOFhmW3Tcs0<@cwAYPiZC!*CY}8y@k;Hz=5X`d~2N-h-~By2v0LvjE>By5R%G>BuFq|jiKsvz zlceI&T)Y-7G5zJbB)RpT-W37Q*|@>t39c*Ep&qet$GEE@4{EyBp6iUkLvHSgb3vq6 zIF-CpU3gGR-Hv>(wb60~D4||BGshg@FeZk|rp{#9^$xDGcvA`M!EVIl#zJBdsz*_< zJ3U#mp>&)OO=$AyRa|m~)~~Jg-8ZzjssA_ZKniEd^mdyeI0Ieq${zN5*ns#UqB6UP zs9Z&ytgL=A5J@mC-$>_S6To@2nG;Z*9#ks)!{w9}xJ;ph+-MAWrks)l3qu@N665i@-F~bY@XJ0qPqBs1s z?=pV>Abj0rzv`RyFgXG|L{(b&?@;Ap^^xLhCWjK%cTIMRtCwad+JC&VRUMekRB+J# zfMGj5zq8h76Hp#)c#&TF0j5kU+dq4%gh`iN6h%YTBVKO%7fx}z=A`XTY@rhSM*NQ* z7qj7o@_}ypNBvBQYZ-*5mKlRRjvT1>hIY~8>67H@PZyZ7aqd6Y$YX)6bCb-x_P!w3}^?%BC}Z9>u{~ z+|y{>?d``&^W~2b#A6334#{GIBNr(NXW$NWcXfL{IsPMNiUrT&E~bf3>--7Z_WfLP zqej*n7O_38b^Ys>$MTv!=lSD=U7uU#n79H?3cbp)ir`ySu}e*#FIxU}E>kxl>i>Ws&DIiQY) zXY!>~fMyW+?K#*iG^!aW&D7iGZcM2xM)fx9dw}+}#OVBKNk;q|qBA>HGH-j)SW_qGy6MUVj#+=X2ML6 zPVy@B(W2DcolhpS?Dgk(l4DrGswQ`uh63fM*J&Rz!(D}b%)jn7M5@(%YT-KH)6Xcr zD1y=y`1SyXZt~L6?^m^+{`$|PFrWqF^^GJlp6!b-l~eG)&iwNSXKD~O!E9%Nn_lZm z)(@E9Tg*00@_$56LlFB>ig$q1Y*wi!^8$IDdMF3n$|Zs9XH8Ci;gD^(?vJ!jltS-q z{+j34K;7{!DG$g0t_%Kr*OdY^t1FFNLTG%{j_N+PBo@2FK`D`lXJ*;5L*ib#N0St- zd_c0zFVM1T@riW`Ns|9Q)KE72GO6n(o;a+rC6O^3FZm2dkK9`~MyXa%kG2l*0M~Py zIs{58nY-^~5hQHc6ARXTI4}@N$;xmc;kJMj>|_>%xoRNk z3wrWdc0S)`UuVocHpSF$kmcx!`IYH@*e4n%SiT~OXB~=M;WYAmR33=<`WOOw$PlRUy~WzA z1#_)I`*#50c^8Oz3UAvIcH_l_c0Y!!c+=xZ=z>o*YD|<*b{xhtI?juBAF*dv!~4MC z;D-W7>4dvZ_;B9zFBvNpfGgI|A`{|t-)@L^RyjvkMS?%R0oa(ao^OKFcs45!sg26` zm&#tzon&sJCA#wD*^bbdeBOCkmP;hB@Ca$$NHr2|&x#FCLPL|TXy5J_A^X`ABbzjT zyoHrRzJcN6-tRyda;mT|u-7HH5(xzp%&$MzSM{Pq)^PGKoUyuh9zmCc%f)tCD|I_88&R9S&7_7*NG>KOBY@ zA+*ox&5k*Wz#3i9l4pUv&}lf8Pf*`&eS*6Q4sj1w@bU3y9RNyb07v???moQg6Yiq8 zeIvf8?-lF2j-bpPw=0hDXCvst_I>*3s9b|8BXu>@U3SeH>{&-5)k*2p!pA8V*q1}&g z>O_%aX(*4>q@DL4lu15TNYF5mM|KPij+0?B0{7_ZN6<0z*aP2Vw={z2g;~6g7|VpCg~V z4>2AVlVrvlTn5}_{nSVIRFCQbQGI_5xcXY6tG{n;yQ`+Ehvma50J|66ws5wYss}k8 zhj}H^7%t?O5j?)#`+W?d$SFa@6;+qYkq2DAloJduh*w~LQ)0J8M%%0 z{7mD@R=SlbM-3vir@j9^paRGM_t=njdU^RgEEl%Nnz#4bZdnuHYzkr(>lW9r;cs{e z@&9jz&LqL%eF;3_!N`G5H!qyw}CJaGv_ z(yuVrs+i;tL_ukDf1TMR2dpisuZR}`KM$%z+6B$Xmhs z6?q-dpyFPC6GE=1>Fo!dMeve{qNqtj7tY0PvzK^20mi^*ns$k3S?3+{lt5ySs;#?5 zjye2O@RtmlT=W1HK)+Ji{Q^6GW`^58w)C|LUPMN%y(8mHJg)9`T`g(VV2@HVSQUh4 z=1uJkx@ol7sB8E%Vs5|#MFNFU*3@5N(P?r*gAFn9%t$MOl>z;>AYOwYfW5?ipNt{u zXTr@NVaDidFuG!nKEd_c50!2oAr~z{)AnGf)F-C}AxWQ4EfmWxTv_B~z8(*Y_HGdx(KyIyz8E(^Zdh>Lgy zCV7>#4+&dFXm3l=64t{_=A5K;z8Q!lmc(;CF%0$0%sw`9K1g@(sgi1A&IXgFeOr|8 zAhrA3*!6@~K`M=X%5j|nY2jV9L!%Yw@ya3o6?wn))hgC$^ygTb>Dte+;gicPg8Uw~ zvkIORe#U#(?ZA`iAfZNF-|wBp{@g*cd)q=_ZjGNQ!@`djF11#<&C}udIf*Am^WGE3 zr@oQZo(>7ELTQY=lOf2|S^1=StDM=-8_MO!L%ZlY;dWIayJxM@NxpvaBPX?tjS1W+ zdJN65N&Er###Gm9j2kCcb`ps5Y)t+BC~-f%P|7bXx7iUL5oHim^dTI>HMDxO<11(L z9Xi?kQk^T4a$zBXKgNPXAha>^jHUDKgSEY?oHHSSm5c@0NE{i8@C9zCCsJ( z;J>fWUJ!=D?WiA87bS-P(M1CAlL*<$wPA$W7>L+d-L4S!W)OMmwDd+u=W&z_#&t&T zWRml)FO{0+W+__&%l;(#S=#4n_6OS)COjK3E5*_{6dkjbwDQY^|N650c@gZ?pmHyy>)j0I`_rZvoUjxAomNVk&$RHCiOk3YM=QZH&`jm|KQlj)hlK(E^>QjmTgT z<=W>Zajr(mVZeJCkqsYwF9;tVk(HMYyY9QN_pY2MXG9r|%AMc8?iM=UqoQZ*EvDse zlSIL7s5PSa?ze7Ee%1wNm|}rU0iCbN0P5QL{eG=eJu%ln-m#ecIVc# zP-^bu!^?Fx$~xnEj^wq>YcK095Q>z=lha?KLu&Q4*Ynln*UV6;!@oZ08dCBuH5_-A zXKzj2w&DBYN{dt09uwY#WVE!-P>~B=pM|3mBq3hjJD9um%3Ol##PL-S4C<^zY+7krdn@ zo$+|^!-Y%f>N&8<{ZA53%&l!H&D=b)c(%~Jwrb1m``()KZj!L@sw|ui4SvV)JoyE! zGwiD9zwRyjY95W2)9mTUPaNbOvaAJQ9EzXO@Y|#O+2pT1aurcW?avE<0B+eQVW8kj zC>0JpYgza$a_HAhA?UgtR&%kzK@lk5j+np;?ED8nYr5{AQn z>|sI*@ci}c4x8gE=Y1$(80_-`fA{)V2kU$Ff_GEVHW%Z46-f!{O{w6w4O0iq&ig&$ znOE+rD-Y*%OlBqiJ5NwAO{FPD+i@uz>FRq>233D2^8WQ2|k}hPMLapb>&Z z-hZt*a3?(A^~;4A0{yn=!grAGcf52_FrPi8$5q1WW9z4-Z>7seUj^Pso$75ylFK<@ zSLY?OhY?QCTjth=EAy1tz}k=KJH^IPZwS1-ev zn>rPNO_mIvL@0{DhQpl!wR;A%1Wvqa5K#hbYi_G0eScCIv>L{`?jy+s$j;f z#u~Nn7D@>@ng{9(9Fl9V-s{zU50r^wFtlo`OJdkHvv5RWFX2q*U1;AuH~*!#P_*ua za`d_W;(|Hj6!UAl%S@M*o&xXUnM``&D9T=^dRAkvAQG2XN#=^f!(+5T?7zaPpOwi! zPioQxUWmV{`J>iPJR=ei?mtY$_#On*vEGG;e^_`KW>C@f<!S8-(zb&e0~SE?G=g(8n(qqteC{;kS`cc3AXz|vCa@WmvCRVr3AX09E( zUoP5#S$C?2%k+)6xWtc(WRtTMG0LDDp$mNQoXgu1Xib4`&U^d;Q5uZ&Z(fmX$w-;r z`DXDm<=SG_;9kuK5kt5K^Pp-0FAD>eRR*$!bsJR_>(y-8tgz7UVF!tmE> z{<4%^@1R@5#1AZuWkX`c(5tGBc%AH;q_>^B5nd-Y2}P?u7aTq4qB3j)rbw-T(o zuvbaMY_K3CO0c2Wm zBb@H52rS~N+RArxPjnobxg+{j!#q+vE-;eWy06aTALy3m7ab_T*0xZs!IqCa3OUgO~xN|U$pkx>^I_Nac`ZkP<*n%zFV%?32a&d+z>+Fe06dYG!7}q+XFz^2mT!Gh=4=6$R^$3;{xHMI$r?mniD_>MZ-=@N|Y>-o*k&W2`~Z$%uA zTEsUCXzQ${t08un0c@WhIFb;&218a8-AjSc%VUCK;_2w*wfzbl<7y;Ab@E#vuaw~) zCKg`1YP^3Eg(t|K=YKy4n+Tfhn1L0CudvJAXtz%n(ZaFM+Z_OL)EHBK#4C$}+lw1A zN0&xf@W;hrzDcnlhAfknOW!{Tmy8Ui@fbodu{CmPWg;O}bvTHO2lpuCz5Og$&dM!u z-o{#5PC%7ZH$JWJ1=9!vrJIPmKSkt2bbL+66Vq_3=xS`>+y?O#j*jXx0>8Z%^|DdpCnRQ-{J3rBg0yx z%0aI8ef}@s&|8R1wR--Y+?dQ}h*&RIU_>?! zc^h{l09~tx;U58Kczq7`NqepKeFX>0Pe!U&*9ucT)<2d%7pYQn)wd9m9cwJhxvx&Z zHE#C3qYvm38_3uCQ2?^*$FNeeTt7}^r#)tPd4gq%C2Hc(93A3QIbAWiGl|KX+tmK7 z3MGU`6grVf(M1D9FdoN(K|Woj+1~;@BfZ8x%~xunpDlaou=v)M!aW_VI0W5p z7Jh$XKa#S0xVS_XAWjPj5Egz;S+cGVLSWq)gRTipY&!XBf;b~^aEu{x44Bk$%)7l)T8nQC@yNGwAkAX z>m*i$;T;9{8epY*_J#n6@GyF+PT zB=+OQh#oxg#P{Z)z#b9Fj`a3n*S!+OCG+`jmk!TMl2O$*ObKt%?_$l~3t%)du?4w0 z7rdjRRb^w!W8nUr6EMSYl|oB0qErEXmHp`MR>{@5~tCR^Z0rlNZV+~cSEAzCEG1bUlH8AqOLph8o7HnM~23N z@P!J_DAc$bR@jrAFOv>bBr@VK8KAowY=mEO<3!8Tr@~FU@Uz)JQzb7&Aw{o-Zj2od z1~{w&c)dM|oEAFPcRFRhgn(z~XEoTADrJ6#nzKotYJm-Y^uH#rG((|@tJpIa==W>0U1@mwt8rLJSmaY; zk$O&i*3@3|QRjU}Zot~4f(ebUI9GqoNWV(M=}Vg`<5Qn$ul)lXu0jotCh-T z8eQ@p8D3eN;z$%lyQ=dq6Wxr}hnXW{$AU>pv`-_z1Z^5><$J9;zdUW7Z|zb*7x{^d zK_pqvsR6#u^MLbtRFXi&FW>Pf*AtWo(9D+<`oa;4DK88vbeyvscB{C}_uYE_f!)J2 z&>=8Z3;(wU(>0o3+sX_|?cQvOISL60e?NMsDCv-bcxC1+ng+&lfgYr}bo-bm{zyZS zfod4z)+!>6kww^#@-pAV6juXdCC}{7R2-Zlg=DGWN@P^xDK)A;s%O_Em!H$DTB~rC z1g{=oF1tYy+*8rUK?L9~3j|GVxufH=-ZOe!%11d;%}c8w5oX93Wkj!$Au6x!DSuno zOvPPnlvP6aEg%P`}J^ku{_Ai7I zF_NY0>wk5i4Yp@ep*QY_%#a-YgC6Z5x?i-Meuka#&xeB8JacLKANIb=j@{3^@#+=+ z_rl0<_24#f-e(Z8TIZ0AKEqo=MfXURXLbSzZEb{im03{VFcV8{BF%r7i zMn1FhI3z{ZE~AuzP3s3DSM5W%+y{U?_(}FdiepSQCF~&=A zUXF3)TuyHg-eq`Scis*nJ00{2`=NlD8O^tcX{8}fQ{A$1*REZ&#M?mtDK_yF{mnbD zEb-ebC$*LsiG%C76LLgy=vdctg3ME4Q>*S~C61jB1mn}ckLQk<2{O>6^S#uF2x21E z>Pjs3XH`S2=x;TAIlYVvLhL{4H143rMZfu9{b$9sbIC1kuH7PE%2E`DazzwVQLxoDidEGxgvT zT|6++C63Hl3&w@muz5P#pnXQ!fdxZ}Z6{%}ssfO|L(cvEZ+{;m%{0|-zl*Z&6cv0M zYJB}FYPPM%ZsS+KL!sEHl7bkWb|3BS-+!8IN;maS#4QkPw;$8@_$3(h={_rX5Ohmy zgE`I>>Lni>)V(j$G<--%oANWXigTNg)a?`b_nGdc8TSP#wf`?h1p~3|s~#1`?+ByK zui)H_8k|GWsH*l7XMG;m+Ty6!pF4jS9AX}76ADuPW2)r-hPenmrXK?iGd;?UeLUO; zeiX#c%hQjwRlt&ekAul1Uc1P=oa5^HQw>}%r%)U}^%(@QY46kmAHPAdZ5Oh6lYpKV zaDWbrrNUF5{ZnhY6CZIaUS~VH{K|RY=?-dzGm9W~4x#?2#~9!>Ud!YdR%gA=7?7vA z*fiO#e2zb5a25Fe*M9YEgG_X#<#~qxiz6B#2mgcGekqRsmR_+k7n4<}JH|)8c)r!n z4)Xj^8-U(b?-Hxh4A%AE__Q=(kML#J{2IRKi@!6Gj!Ro%Q+uBs$~ zO9Q>-w(erb6{+TVQPw*OKyLt+cA(GEKPrh6@Qq`q*9^nT{=%&tJ@ZtweOUBM{AP@H z){{=IorprMUS118PSdFh2nN%}P0 zd150WiKyg2^_^->@9qap2^OpC_RyW7pwf*97ME{816!=$0V=y_iCyxTO{h1E5DzMY zdErsRDxB6i{cFs^36(G|*hUKomE39#Yx+OpBq29b7liEWI zul>)2i9=qs(qQ;O@qdjM=z@=IUVj8y9zV+fZPn3AV>cDz$O=cIG6Q1ERYzu|IB~Y@ zj_%dJES7LYEmZ&$5?GwdurYkOQjbig%)@=4gW~m#kssR$pZ$u z7scK^FejON9j$xT@bl2Z92((`Cs$cb^Y0LK(>jJZY=W`py*GID8^RLal=}d%as;#4 zj>vjb@UD%Nd+(fJA~kL@=|$x9zAjKP)AssOxN0-h&*h2#nob`eMBkC=w@#jZ=7Hq7 zCFgQ1`YDKa2A7MW2c4fRnrq+BN34)$2zL*`KA}Eu)l$#0huW7~G9;*4F%YsOFC;nz zZT17nkLgvlyK9)G9R0{2CyHZ@ZB}<=?%yTj=jdT<3SfpkG*T_B0{-dGKt(~(W4N!x zZr=l5_!}k}++^4xFI>WyUB~N6#Zb0)DKm|IT4`V&El%ty&i@W2QJ4P?Hn%& zEpdrmV}Iq@J)?wQ@rB;8;8=V6m6j0P$g30A2?=)iHsngd31CaJRjcZFho7=qZ>C~5 zytpx!QyJp@M?eOj3DZ$gT${xPRoDD7K3^X^Klz;{O-}uiJ-A@&4*~;oJR8fyjx}n( zbtt`;jrn5*M#$Lx=8$@C)G#qv^Xsk}BtKxe9E_Lm@2&ke6oXEd|8r%%Y$GFSuaX&O z1kuTm>k#!Z-i`?KHY~EeO2n|gj_@B_e^C&gOzb_LQ0KtZQh<9e8&CcGMiIE02Jgnu zIHXVp%m9}QMWs5tR1xK+#nsm3>`9_o5UPK>2%TVV#akaxc=!fE%5T#v*1)RNJN(bK>u>y4 zb2hT76l?PwH6jrlgk5P78xN;=dtCkKx5-%NA0`Ba=ysCT>+jsBh}nJ`&Y#jv@qBNP zMkVbF^DXD&?yk4#^BCd6(lv@^R~&6O10x6-{#!Nsl_qI=diqA>9Q6*dN*>BNQO#jg z-G&bHw+a3^jz!D(}LBs;~z4DWe=6;is z%DAp_ZZ*GYAPfl$*K&?tV(B#winBDvPwpey-rn#hGmaag(Ee}q!h4RvGvtRzjkOS4 zR;u;%nSXLW$Gn1G?Il#hQA)5B>zbUY^{osrM5?j3cBHMMx--gQcO23{P{$tVo+|c5 zZ2S{2r_bpD-MT&$J))g|_V@aq(u6mOO{zGc_T_Ohc$6LS4%C!2>DT}EOtDUhJ?W}h zTuO8n6debbe{&Pu@&3!j3q%%hcCDm+PjPioZuX8E;A;Bd4pB`Cx&`Tj4V5ZSHM&qauy2{sR6hi5kRuEbnp3*)PcGY}60wtlFtNHX^kU(|Y+Nm|arOlG{Yyk9#XfWMPW(v07qZ-^n?JhUpcg+U50Yx*^cGUrS8drUaxP+G_v-=>7Y$o!Ag&0xo zcoS%(hxaAiV|6XoxH7cgw;|IExv;~>t2&SHrI+(26l6pje&gqPt8_wE# zN`vy=1P$deC458oBurXti^TEU95SGXcwz~K;!1#QF>W|CJRmiM{XV$>5*iAchGPZe zIZoq|qR@m#^UXT$D%n4!Hr?B_^!xJ@W4m%-_jY)-n3ruR1HxYBtP$nZ55jpVLN~ zkqn#4)60^u?5v>Js7W|zelY0bhHv$^!yoro2W>q~0ukhL#Zy6tdk^<56^~b%+y#gN0uX$vf<;Pabpx^449Bpt6JOYqV z()nM}RZVP6Qyzkf|fj}=225W8MDrzY=a!GmQjJ>nH8)~wCU6HHfsP?sH^ zQunk4`WFA_>T|6GBn|bZK}Ut~ zq!J4Vamt+Z!QS_CjUWbY-R$l%kLJ!eOpktiCv5p2pE;R_v+_^ILjY%&sO%&Jt|)u+ zGQvzWk}>2>z}8j<=qejdiEAPxT?&w~BzZtq^{%gB@Jfvni`%EGSu`F48?lVPr?lo% z5e>#g*#u2=Vb`$e$N!`0EW?_7|2D3Iq=eEr1pz@C>6nOuG>C#oOhmdwkQ_`vKtw=5 zI!200jFjH!fz%J_9NjQRZe#K6|KfSOd&jZ+xb9uo{jKwJzE#Y<{;>T9fMDDk0o3Hh z--0lrN*tyS<8hM-#kv9SaigCf_?Hyqs-X#iJwOLju09D^8x2+~6G4-UUzyDH7kw~d z4Ar$C+&i_($O0)28Wg?O!@GpFnr=}FI=8P$)HYn>Yy9vXug!mkgAb$HwNm*B344s* zE>dDUSs*~)o=n8Ot;=i#7?PW#a|rExGih|78(-nqzKuZcp$)>QX-fjb)CNkkIXSKF zKic~AMwYc;OnJoXz4REQo)k(RNP84N!SY=0LU(CC=Ej+~-OQ9A?T05XDDer9aA61@ zs-TA@VbTD(zTn?hY?=dK0?(fDVR`zqoRXaAoE#;5_2`YlXN4ISG$6+sj-R(vEr+IWSJA55H-xa!2&IDR9oem%-kXAs% z!T_7TTfv!^5E!Um#H(L{(=vy|b!v_M!Z&}6ZhS@qA4A=P2Y)-VwRy)*nk{^sd(Oar zvmoM$d}Ya){LsT-%EV~mQ+sC%c+QJE7Fs)AH?#EiBtiPW$MtNaW`CZBA|NI|1USvm z6+xq1^$+6oWR_PO|1BMyDaU7Gvo>|R+}-f=UFAEY;s{Wnsj%R$`^ilaH;N1c&eaJFOo$N>Wi42i* z@xfkX+mU+O*7P9P_zVNiB4TTfZiNMqdQoTk6GEf*;Ez#OH zk|klPOn4M)D4(ECo+6I`K=ixU;18UM(6yw9(pSx9A}5UC>%I_eNTiTu%eJyRvI?Jd z61Z!MQia$7_9%;RMmxRi_<}U5aUw}pNn^b%SEnQ}i;YX*@)M7lADNHDlX%47R1aB5 zRluEswIVH^Tc#rsgJ*t4ELiI^n$BL7>&R*He?O8+Q^wo4+qzKAt5zRQV`aLKLHUL0 zf)Lw=q%-=Dto6_dV1h|$h0>!@g2ZQU&o_3> zT9R9jC?i*XqenH|9{QiC&o*-?GP}({O}QX7*EE&BOT&1$X~ZR z2_H5O@cVnCi(6l||!1!Ktj)M40)NG(zSbzIA_>g`^*Woii&?v}RW{(dr zw~z0eTZ7W{2be=opU-(rLl~WxqWhLYjtAeqlbs4^sfAZw`d!+Jw(jCg@>7DJ@Rq3H}7+f3q}NKDF^LZC0#oG>z^7RaXmN^CyC z2eqkbu60X&ip+>rhji!RCBcZgoJy=)2AMv*MD{0Tm6(CWG~~c&K-matmnz)bjrT{P z?F3mGpL+yE0;k2Yt(Cuiz}$Mn{{T@UeZjI*ohmiC&HB&V<=!l_iK*(nx0sz)@33OV zoC%g#40G;(tFd3P`N%}+Kk+B`b^bod2ITF@px-uc6s4HGp4RKda>-PqS?|xIbKj#!F_2TfU6*Zu`*{(;dLFoHeoi$Ao zwQnk)U!U}x9U$z{wmN?3j?KO-cy3nA;I?%q-i1CAnk^AN_407#5#Jq-Qg>cijd+%# zJ-P=ffc}Fs-x@r>R(W!GT#J*#zSzl;JNJmZMLOJ&a3JGU;1DN~iU5{uW5pW<-$51o z9TAmueCzKXV?A=AjSqD5f!F!z=$=DYks2Y_0gaxIXdpX|I{BS*IVC;9RUf}^VSP62 zssK2iwj3Xj-_v5J?rqP2hA6Rh@cpb$bo{6F<<|tid^3zL6VF@?eRS}GWsi$eZkhIP zJc=%ntLss5(w z*dp!8^mVc8=w}b60grUZSuK`^?=^2}J#ZsdqdTK;q4vN^h{5(YKGYIuiM?eCJwDN@X%b*cby>P_ zJgz-(2%FFNJADWkQ$6seV$>FI>t8+POHyE#;ZDHq9dy5x2n1uWfY>EkxC%758Z(;wzwSeX`i<6^e+DP@!9U?F)<4TZ`Oy z>?1vXnz!LhVA^$JeBtu)7a(X@=8tUcvw)XJgQ!#j^yH!g7_$o^*pYjI<`oxu@3;x& z2)8^uR$@n(=+^o($x{q6b9IeqlgbyC@;*&Gm_>*VJLd7XY}6hl#iU9-+it_ikhPw6 z%wQOZGf*kk)bzjiT9lSsDstUw{9V&LF?dhR*-_H$O!15b--Fi27+3QjZtclHrB9w2 zwdh4YZ>3aYrFUUczg_=M?-1**C!0DM88Pb1eAXw`9)0fc-KffI__}IRXvwN`ZSv56 zY`QKUdoYi2T2(4fv)XcZx3ZGDQ6bOnV*+hE*^2u`BXCcy$|eBI`r|qOaMKJQJ<6fF zE$cHW#{72UPy=wpL#u_~mc$+}VHIqxmB0@caK%80?}&TP zxKP%azUmO5p)n}@*E7jB*+l^$4Hfwi&wmPt`_9mflV?hQ${+&sZ{FzIew2FWCVuxkskqld)b%TBU9lLcHl(7t(o~g^7vSqJr4&c#oWh z@(H(9eP$!Ny}}lU9>XwC2)l^2!5(8m88nz?e=6y~gw{fLtGeLD7hdSHj_2ekJ$#C% zZaV4X6s>~6yR~eBN86g?3qp|hl+*rC&ke+$V_hD7MLGp<9Hxk{V7vdd7GVZ`Hz>Na znk2l^h!=;H8YWFq_h#tvkAqm07FxB6-rgtOp6hrim4wM40U1s%nq8m@hXjQ>?;%i-KjH6_2Njj3yTn^8jHKLR(WV*^P`rKALI z5#0iY>QUI#)^DgCgtkTX?Icj_%XxP`B9zz|u6E}f4}<|8Q%K2q+0zq1&0+K_RE*PZ zY@a>q7XV=Q4|yJzjhM%un>Uf*jX1;FDg2bDiq)`@;QjiY_7Rj&Fe6{j?NFleX6rY6 zm@)m^SH6Y9ZVS+0@eezBR#&FLCMttO;M&e0(eEpkifRdnjI3Qa3Z97BTXpPibkWYb z7`@@3TBdyQD6m&=K6(@%&;^K9vBRDaqY2GFKs{wHTP$$Y>;I59A5f=3aq6DPp(?AQR|bLd`Gr%|Yh2-i()Fxv#Ywc(x<_;f zXC37l{!v?|uU|(*U_p_+H7}ID#AH;6-CKqYBX(i+_xKAjcWN~jPxThRewUHWyPCJO z{3*U(&9h}Au zhv?dUUL|6R&9LZ4ZaV$be4rfX5$DOsMDM?5ZGj7`;ORXZZkhIP@3uBuuca4M6g?P2Ke?Y_q8~yomTL^>7efrK&R0!Z z{zvJ54+>R9-i8zH9}^4?n$5Oo-rJfnPopBx3@c|D_dC6U0Vtr~<<=V8)i9Z4k0`F# z@OcXmzfOKeuuoy2>V3xicr@`tMbGTvC>MnFFQ{V~jh|7aKF6zUa(VKW6F4C#fcWdf z$mLP9ozBTib=0$8H^11hL_vi3@uN6K6`DuIz`uU;Rz%gD3$Y*p>?%D^$5B`EGZlMw z`?lm`1ipg6d)On^dArzD;%ZB>CQgwW^+ngJTbz03Ygx?N{Kp0V+@o4q*8-4Fzviar zaaq@6R@3;eE8KP0o*MVV`Kyw?3f+!PT#5#eB={d2Rrq(IFF~tsKJOER3MeQ?<#NyaqcT%Jj8j|fynQv41VRQu{K)bO{Du^OI|yda+Kah?6HXoRWJo^o$2_T z1vj2S1hfcjPPhDr$o5mTG1|3zEVPAa>8JbQ?@p}BSQarly>vasuLaFq+|$3m3UMeD zXR=`R2}IVg&aa3ND5FigXc-Mt^oO2*;Oc=Z_#IyH` zLJIM%Gxocs=J|cPq))B&Befh_aJ=tx+l!<5(6|{YzH@zx)=)xK!wZP$@W30 zA(~eLC-iooBB!3fO<%O>q~wgBI)8UQKG(|Nf=l9qN4b)$r)8bRG$tOk*{^hY+>6SU-5L4*TE35Ilnvo#< zs0B-O;MRgT?SP_WBmly}mkuiX*keTpmK|Ae53Q(4N8qvU$0KOojs8oo-g(?*&O><@ zR~ASO__QCDvr;A*+kC{V@KQjb%O?srges;2W4B?c>#qfuQDn$H+|fy(S%{6A+QPaCPDGhu5@2Io1U%! z^Zp9tQiil1N}dLgXH0>K0>i>l9bRX}nxI7Wuh+$%ERR?ksg#^?BU0Yn6toVIWr9Z? z+7izI?WBGbj+TX-2j#zam1*cIXr2D9;g>HcrhG4@cmQ>5V%2h;f;A ztlx4QlNi^{fIGDBwm>jLfVM;)?HqFt#jK+7Sn{rPN?ymq+i<5_Cb`5bn=ps2Niud? zHS~04KI*@F-O0PwWL@4dJ($j|Lq!5#Q*o_Sm}{pCqeSp(s+V-4t(>htyIi6uiKqZc z=r4h8W%QMrCce5;oUQi7`nQ^=<>*fkWx{X^)~p})gH=~0G+bnckd3Udwp0R~h+SI? zm4yJS_h1ZSJf9cwuGi8@fMvoW8ps8=-h_G+tkfuW1cQ0EefCbW;=v~Kzd9GfwgUu% zl+@9|q{R4PTecRhpqO*)GCIWPV$?WBnQ5uIwZG0YsBVhy%+}2AX)Sf?<3qoV?5~2y z>R}Ft2udjumgPT#gLi|>-prX|PRh)Fnn~}3ODp=6S7b^u9ZtAUur!vp1JRf99p=xb z8Ujj(wH+ReNVTHxGh;WOxV+g;N})aV-_!$~>i{0dNxJFR9^y#WkImQ>Uq{Ol z)8RPbtN8fN_d3VAHqSf_6f8VV{u;kdMBM}?(vmEZ0xlxfwCU8(kCQk-Va$(&y>aW; z(&P;WdDz_dt(xVS#)4dJh9C@w>Uhl92F$5q zzsPrcy)K41;I_a;A4NOvgN`2gvl8`uh7GK;k5}2hMP@=r{*N{zcr@6Imn0dOA0FjV zZb=h~&w$MH^l$tMemv>6{iL?I!v?>d44FldfMhJidVVyEs!I>@uth6(8&+%#75z;b(ahSoC>X=Z}NL*)GqS;Q# zfnpHol^+_fpn~q*mf-S-0Aa+RX_DqJ$TH|E;fd+T5W_Bmiu=q(e0nDGV)A@V|s79Hc?#U8<1=T;3Vrq(s~>v6-+W9FHFovEaBO*r5MF=O&+6b5aLk@i^XAf>U2*<}B{{AsldG zW%~y^f8sGe1nYToGM0<@+}*NPu{fn{V)U;KQe_)BMjMp|+A^VjF$EtSZuaV)&Ge^C zed7DuNdrU$sVQ(0QD@wQ4k#gucii)Be1j`(zK zeZg_-AXt-$5L1hVt~KTkLXa2uw3BGyo3Bb6rm_Z4w>LY;y=8InFLgcvSMt~94V!S+ zX(_+&lgROS_KGMDvR|0=X5bFuG-4(KvFJQn6K*YN z`NpG@CZS>Z_A|eS{*)mJj=R~vSXy2{f8Ej?@iu7qMs80W`_LxpIwwPG4 zSW=X4&5gS8C^o6{WPvPt=sWYR!*r7&mbIMVw#pHGyc^Q=TG#qG=R9~w=hbmaAd_24 zU+^!J&A`#WP8hFcM=tzI>)2+d-jOgd5|E)NF{}sa{0+#3MH7R1}uwNPfTjpK8 z4Q~+$lTdBSG)W`&8g@gdwFrxQGXOVsvL|)=zjOpV=0wkCx2D*EHRg0|<6RgpI7BGK zS#fa#^zhV0e~Hn;FiY=cxucvsxGo#}!k}ZMSWeMu)Zrx&&ym+DC<9RLCZT%VL1zofM=9)` zmP;$)ivMgDd zjKyf?7_nxRB)sa8mr4a}Q$Csz5gTs1P|DU5lWXwlgWbcLFFGH-x6I{$JIZ=7BCs~k ztP5F4w%L!l;}OaKLS_!z`gZ^?PT3MHexKxT zh<4mC{TrW0uNnTIkK5j72F5lSNn}pqFgbWlSw4M92VX*Fcu4lxK^5r#T7&`9`z_Wy z9)oYzO#m#&Tfp$E=pzU99E=Bt#nl>P7Il*FW-J#Hy^WcC+qI8GA7Ngobc>^0Y z+KLoua#ZFBvN!v9{l*3RotM}&sOs-;QjnQ#1wy9`>AOF+mMX{Ylc`SO%*aP!>x*>~ z?Aa@h@R@jun$X-0H!pFCG66W)kOIWj*dYNw@c+uWD+2NCkiU9IkaVo$F&c^9Vu_Z9 zSKlXTiOAwlPm&&V%sd+Gr-EF55C1^!FUIu5JqIpM^Dg~y#_4}t z>5*@*2l&2O;b*O8W?!67H!hm-dTRbydt5)K=HY@oZA#y8N<&vkdfmr93o4CIEOE}o zZOlk?n=)_T)Z+%oG!xntB#SGtGjhAUVdL-aH(-YjX@LH+KaTQ>683o*8={rP_UBMc$vUUz` z7-LW2H(98TqJ?s%sNmnxNbGp z8=daktjq_V^b}e-?E83=hLqrWB!)fg^_LE5A8aQ`(Ri;!_2jKAYT|BgPcIz zFz9%#^vNX}fcjO^*o|Z97Hgk9&wQYrRMdlN{#iTl{oh+9!hv&ElWyIXQaB$H|4_&= zR)LQ0dNsQE$b}D;UU86HL(WM+nd5|Yggjn3Ur7=K8K;cb=8+9K&T0ZrGbg>{it^sw zM*b)lOaK{-iUP~M#`dr-<2VZvk$NH1j`Y7xJ7xJoXQu6J56F(1O#<vEZb^U*+mM@qa5>D*Y^?)7RxpW=OX6|C#qzNAml9T}EvSt8NUdAK=8U25;@%{SZFZblyL>MpLRw{U>JcBx9Z& zviV)SapC=7LgNBqraMWpwqFSL%C}d44~7`ii(mLxPNLY%t2XwIm4Ydp8HW+i~3 zPU4xu$^2vL8y{=+3mqMT70A!q6*0*YciiMQNzoEV&`ZobVv59do#Q4`GGK>^QAFHN z5kGp4{XLHA*uc)Z1W=C@D1EcN{`AhTH<2Qrqd{;*g{1X|&7h0gZ0ftX9(^wpk>P2{ zl~+&GM6&4%uO7a6tlw{TbNerSZXB9xMqj4YBWYXVxc$_=Np<^0>KtFQtkB6cE7GiyGt`{Qx1DZ zIH`;tvWc01CmCR`Li46ahwGMfk59MLG-wfRDl%IBopL zG|y)oNpWoD!opbE;hXQ*n3VG&B%Fe?$(F+N&FDbS^TaqduBXs(ZxyZEm0EveWT))| z7ey%lhlz1h;YFc%NkaZbn@d&mCIi>ZmyY|+1`ovy0AmMo^O3JzHJE<(J#LJbIp1~g zvG9?3($Lg3oR(;ilTs$}3u!N71fdlGAC+=}JlWzEuzr{tY=*Z(mb-oSMU&eJn{1Q7 z(DkIz-~q>OCgscrZaSeEiDP`yu5%`h?>v6U3I?sjEFu+XRsaf>_cmw;Y)CJrH^dV5 z6_h4lI%agqIJR5wd2H^<$7ONiD=|*3ede>D7n5KO3sLBhZKeJlFu12U{SFN0$BU>J z`nU0Sgcjrya0%r>l$v(xs+}NB#XHtbK@Ho#ti6_KotKCiVh~^Xz|C*}=!<_XRzIV` z&KTceZ*n_uUbPW0t(+X#n0ne_z3K826G<9j!dtR8=Ame40OSpEwx*?;?SIC!>s5B$ zw|%gspDcw>eY`5B_2NfEyLVRFrbNl+IsvK=>2Eb*>W+g_ zjbQ9btaXwv_KCJG-7@Ybo z<+Cw+g5^v9Etwx2Z{I)6IJ{RdaZ|=)M{^{V6vC@YRz8e&do1WKITC+~GPonM57c_T z!fTN*cQZpsQTo8fE$1S{g1lbYX;R_&G$))$jw7P^Uvo>r)B9N6R~O}P*H5h8Z58M4 zzhj7!JuR^QMnL|NRPaEFIJXU>R1>Qe;2eeiD3bmX{2<&|ihzHipw4m%CocL3gk!nn2Xhz} zo}b5Q0v#&!Ftl2ByqRsi>*}QNGMllzv_aUr6%j7qhNiVq^s@l0e_!rr$J3SL;l3PC z!i0tGPQSeD(doD;x@C{{fPpbJ;QW0^(L<+72aYiIpF@#82_g^6P32`0yo!|spbVU) z*1V%`?5#?W%t&Un?Sj#;!{sS?*5>)rB2njPh1tx{e9M1FKWmRGL0*yC{05?&$ixfU zm5)@+)Wajy4r1LrByrb=wc5^&MiqNUtzCCV`Uyx$&jthJ3!1FEf~4ruz2kqGXRY^G z9Je5`EY}#h1>j)=|Kv~eBDWO6qgdW0Cy=hsZ3#c142P2xKe^8JH6B!POj7SeJ7W}X zaGq#4++Y;3{6Gj6+HpjvWF;46aVfcFbw@5wFJ`jWKIn=o)C$?&LP@Zg=@e(pW>1}2 z%06P(I*HQy0K!oj>ApBQUhgd%QgCa<#LL*|)_2$MyRqN;RL1jguKt1EUUvi#M}I9o z??v*pcvAGjDiwR(h*h=T3ROyO(p)!2Z&z5+@yPSE#?cGRgXk@pbomSvQ7z_MHMJ`n zomp#~L_+sRc%Hib+W`rfy5{`K|A76vwbyB{ue(da@J%Q8a&bkCjUs^X0|(j4nA7R} z-XHqM?ge6;7yhTorr#y27}0Xur{X88pcB7!-mVM?CuS}>k{q+f^-ofOtdv>VABOiQ zv@(CVzf03YuEWIptmR;}$G>6{oTdnx!Uc_TtHV=XBr6M#m?Rh_JESCrPs3QN>2;oa zpCF_RKR!E&Oh5&YeC$3m$H}e58#^l~WJDJvEd%*`stBXpi2jwrx0{U^MT8SWCBSA# z?%mE5!}JJCB5dpuMy+}+fcC}>pi2~g9G_DxkE4{sIUzFQMWu6)C=8z;ym_O0bVnqU zKImw`CBHc-p6d-3dH8x%6X-kZlwq0~crEnS%{^>q|4P@xR+V@sr@kR3rOeXWGYA(-bJ{=B9~zHJ0paNEiYKiM0UjN435!Sv$u0+*W6_Mtwcs=$A`J5j*3 zEFN`q(}|DtWkY1}{J&s!Z9D}bn}bMu-+cv$-Z{1jJ++!85Of-S_he~(;D9W?_b5uPubw3U)uZG) z;st3De;jNaDb00uSCxD5U;n!Hr$4&1yB>THqeFEfd6ckB2W~LBnW;$L9w>Fm2n zXHPl}|DOf0Lb|8}Ea}Yt9xXm4Z=EB+38g={kN}B7#?i~iPatSz?e|Svx4E>(65`!i z-zg0M{livU24Bn=n`S0l)&BXM?yijem%+Td!HNb2udOY;RU8zt9MaEf^E(}NUG%#m zl4yQt4CzmTX>!k_s8T5p?+ga9Lq)X%prm4!!$0M1O|08jC72!$c#M3aMJ|S-!no{$ z#*A9#`D}@bFgt%t@5|uboiai2fAPt@^EcJm)AngRShTRq;$qPEq~I-znxEce+%rg_3(!k?1i7ydf@Vx~GK%+%GTid{l3vSo%nMaW1w$|3VqjuC0b??(1m_3%xLG#m% zD@0tJ-q3$?5sX~=EGr(Y8NG<(aZc`8qh(L`o;TV>ppOq?fhlWKb4p~sFoURio$wv8 zD@5$olGk_1{<2b01oM8`S(W~2KF&F=|EPEvop>%t-uQJ=NhbNmOR-Tsi^lF& zD@F8~#cCMl&zi<5o zXFc)t80@<64!=_Ss_n{}RfvBWf1N~FMh-)4_Yqd~m<_RU@ zJ1=mkmH0B>RA7WM@X1q>8LeE(@87q+%3(dc1xt(JQvu_21gzKQ78lY-8U89FzZa`$ z7US+8{=+$wB%mdg5?;UoQ^OreSl#>bxX|Epb3t-wKN3S# zCa#~_i69db(cp_#N`sOTW*bu*rSPeG_qz7o*ot@%_}Hhq(}vU9f9w+7%91%4?l&s{ zAz!8#wriZ2uB3V`=y%*v=;`3q*xL8|aq@(tZ8TRk_?R;qFeMmjATj>66%`S|W~$-P zxTtCCl6S&L7krh5q>9y-;}2f)^{r>yjwe47G!2(7r&(_8kO%1E$(lRyX84@7a1b=TSLq_1>Ez?e!Z`-NNsc znPVti*2j~)Achd^3Y0v&f%xUCr|a3cs04!$U-=DduKWR_z~{4D_7LLPe?hAQVx2C8 zYepkK8SWPCydRW7Ouh0j@4B~1y}LvSpwH*_oQIy9rsw#y1jkD?29cT7!8`GK-{_y6 z)kl$a@E!vWiI*TXC3*PmR8h%ub_+k1F)pA;?_Kqz$G=BkGN??lLdnL7PzOiyMmm?c z8szV!3GMJMMFJ@vj+;%y1{K? zx1993bKfrBaa(nN9qhw*S1)4yOF3KQyBqtt^~j#c%*($7ZE|)W^Yd}8=Q%1*+rtN* z{-Unt80=V%83M&if>B`?bsum}8H<=k1Y)1VFnC9l=TV{b@(be5u*OAhor@9F{>a9s z)mVjtsqJ%9&n?5Y11}%P1dnelny1SEa?qq*pl^3HW8J#6`k{X>8q)aRH5v0eHF0*sNX~5OLfNQ-L#TqMIwFhsG`6rVD}sy3)FUs9&P^Tn5c@60<0h2XjTuU z+YZi!|E9{a@NV8nXOwQb{qeTGsDd3INY`|=oT}U9o_r}6#C*+m$5Gtlx}%M$WVRVQ zm&i*4$(|g_VUm@|l&s#}U4*)JHPcb{e}kJ_>qnCnD>(JAS-HjHYzms!JglSavYyuhQZrAmN?eZbaA?n03%i$U;>3t8 zEEE2+7in1OF^)tnT}>TYZva62!&9RF^ucAwF2zZj(gr*T{LUaGIR-nZFaxJ_D@+q% zY9uEs+%HALw{#Fu10{jvF>He|)63~k zH)H)UJU3MHuB*3t(0t`P%pBm|3fm<&?z6KN26ZW?$*x{&%av-*(> zF8-+O@oOJ@VW}$`mG4G;a}bw{NI}0F{2#MJrm_HdR)AxmPqsT zZ0_Zp$r=*`_S?&U5xZhDZJHC%>mx%sfMHudUDV{CrE7AQCseL^EH^nOHGCOK0HB3| zwI=%gV$@*O=r?SdOnG6pV;$aUC}+`Ap(|At&RLSCB2!*vn&T>EkBwRNj>~!o7QJZH!4^Kqv@&3w}+xV(n)o3qAc`Tvm z%Rvz7_yh{vAJYTCa1Ng#bE}sLKDKB>5WeiNSNj1b{ZA4%5pMN(D+`cSl+v? z-}hnlgVL>kf%usgo{j1<#NDD;=STkut55V;>ClY@KL7bj$}!!u){Je5XQD6n$qmNZ z$1^uiat`i$FaUJ;ny+lQZ|X0B7RX8lw+3A!wU{gDO0{qu#ix2Y!b{xs5m*?CJmRpr zV)>)O_ls=4K)!#SlvUBeLk!(y7=+WZ+`-GTWc)7ux3Bm3hq9HFu0)rSvOv7F!#9l6 z69HS}ht>vk39?ak;XBb`><1M-*<6mnDM!3pTse(1Z{pEm=n5Wj!EB(=eci)nkLSHq+wr)t>j7?x)tXf1&}It_Pz*@O1IKmGle3*iz0IgJy` ze|xe-dr023g8#jc=hrN7mVohad~!dI0_69pW$xUqPW(0={*G6yJZL;v*AIhadD6Bj zlkyySw3wrG5OZG|vC6d9ey7G2yH`4iOzVtr%xrbjknNZq95)40!TYTjyheYrIH76pJob|c%mRD>7+dnAnv#{y%z0d#iLF>_u)hNU1*LCy*UwG9eeT$vF{|a=!?wLI{Rg6Z2LOG z{|TucvYxDWKI;OQ_S+rpWe(v*iSj{uE3xDaN?5W_y1YqugVR1}#q^*dV}oI352zV# zIR7`XFR-cZq9LjD1EJb5W@jkWdyT$e!q-}vF|H=#{AOLSq~ziU4yWu zH;s0It%rDz_IkGJJ~f|MKsdIIZF4a=BM+EnTRSwh@<0W9ZmiUNxfHtlvIcL!q!N8} zoB{VQqg4La%d}K|;c>*Zi_Ta8Wy^+1`0R}Emc!@^h}sO7DduZ`avF(eGM5K|9~o|( z+aIT7>{H~GPqcYA`dn)iAB6vUx+iy9Q#pOp)^;oDDA}kJsl34i>19ZLe!d1IV>}=_ zS`lJawWG|!gth7)AB?uGA>a+q_beY|m4vrqQ)&*J9}s=2$%=*Z1W97?)i4UK`nF!Q zS=B3vJpDO(Ge=|sl~P!sR#C|nM%1BWdSo&P=8k?k-OtrK! z?oJKyd>PZ{lLL(=M<#(ntntg^a<7pNWhws^xHOQNOgStvR`5>;7})Zs|GE6MI0iJ6 znQFgfHiwR89-P*wq=gv#&gs^Fwb-+y)G)S7bs_&lsFIIVe1c%Ir4otdMDWKi7R^+y zoOCm)gpYHbp*I`9$ZW_C?8UE5ugePUY7c2L-jnofAU@_eu@2eFRyCE>>Hc`B+a$v zAQV+MGUst}HCgma1u1o@jf8iB$ry^=z$C9zGWDh{`@|}zp z11_@MGdikQyIQE{Pq-Azu^}p=FBo`9JK?*`Hv~ z>PD9cv(VbU{@04b;ycJd?7JyB4}$2o4H=aIOKEm<>fhD&KLxtVs9Bz&-1jQi`?z_w zjkDg1_TF(Ix#r`j99acJmOJC6TYl>PZP}}A{eDqBd(l^^^RIUk{!v!eI}u~!Q>W-M z7M(QUgrE#&(1y|hn7{fT(y9`Xz#p1a;ahnDufUjI2BIDif76tPw-X@jA}ye=kVw1K zJnNi5wnu+rOAPE#Cv0r0#}j|^cpnXj32t2~it{amlzU~r|I<5c#lU}bo=jicld42> zmAf#B{bxK^Udu`6OTUomJnH=c$qVxlQ8`jU6!X60 z*y(*^x9NbtOX*trIakbGO14uP8fH2lCZ(C>77rLBf)1AT z)D8>?JstA_ukm?QH1`cFFO~>R#tMR*Lh$xg{;r2b3e(%RYoBtID1iLY6RF=oR7g2T zC8>gHl_xglFgQqtBwqNjbOm}m;mmdRC;|1Xd>r7S{}5f|%HZ@7^lPXs=f1#QuUoOV zdNA&KRY6C7(DR7GyN>(GGvx1&V>1>0(Ky%gO$vpEIehrlE;#d#^`&qL$HqptH&?h* zXzZxyS|cFd+$h!+OuZORLxS<-ibYH*B-I=NX&>xp8Q`t@8jb|#`nIj^Ei*o@XT!C z<4;^$kN+W-Ysr=hH>IP{J_ftdG=)so&swl8msVFFkbK+p$42)L@`<=DgU+;W(V$P> zvkHFF*C~(wT_z&n2#+1>NRB?#kT3)$Y5l`;urGdNhReS=C0AU5@P*#_t5ues53FwZ z&<~)FxD+)=Q(P0tcN{g9WC_vXqQ64Wf$`YHlL0H^58+y+sV|#$k>Q^oH8+srjqwsg zhvwssIym>EVS|07eJ=sF!`89`Iq=#Y+EB4;`05RU!j1{?6D4$QGoz&uhmo$ zUgQ1@Zc~sMy|T5Vdux2HvtXQKG%4ISGEcVU%D_QoK9y}hWFo2XGL)r1jzi~<*g$2w z?&aLAve%e+W3;S{f7gA21ssy-*6DQ2edh~NhaK{JZP?h&0T7Qd5~_b#@wN?*U54NY z)8?ao9iB@*ET_!Xl9&HomEH4(g=Bpy)V%WchcV|b@W~-T`0+(mZSm~QV5YrC^gW~1 zxIyR*522sxOy)ahePd-cT1j2EP=RLTp!(GH?mPgw774y62R`=scU7DNCY6r%vTwoM zc4niKmQ=1yE)Geu$s~K zIpth;cw>fH8`D5R(l;70&Q_`;`1QChw|43gT6_|G9P5TsI{@+(LHglSB^SoatQ5dG zS^UjZZPakv_)(#+sT8BicdkSA1H|Yb|ne-4j(P0>=^kh>XxwcB5QupKSfKi zezqhjGJVRmty4SA{KbgYGjm33iPNom=q=X%3W5JIN*cK6as_MN&f6I{0cTjE%y%h8 zVV^e|U3eo>q_JKY!iLgK^NBwP>y9-?0yKD_B9sICcT1MBTJc2T`{@sXgZmB9`pl<3$;kA6XiGKXA&>z01L61+6UAz zci+ZklMF!3-&zS){Jz^yYWX)IpvrF}URry_GuGkIh&6;ADOlBVdH}q_7*#baDANME z7-JB49H6Qdhgrw364RpI955Oh=XYPd^pT6;1tD>lBP*jr8#?kz+t_Vs__jDv_yIN2 z${hqlo2;5wv~~_rm_5rGml5actz1%uCxu+ zBw7U4G&~hhN~;eGC{A44o{Kf&)>Ep>p%rVp=oWcN{QapW^S@Vt|DXrAJ)s~!@avj8 za%SYTns<}mPC4CK43|{&Bh|td-r=N-VZuMK6PEDU4HFU+ZH4YJ$v)0cX7+Sd0 zt33|hY%s$U%sTk)3-*M$#SruE_OIrwT}kAzd4vmRM*Hni_K&UyItl@ZWk3A>Ec}0F z#PvV@X{lOLE9k89syruGTK1{j&mWz>CV5B$N~C0UDtKtZ+l$Icp^h4=#1sMzyx%fZ zFGV#aU;dY4{BQSQ7@w*dMm|`K&F-Y;Rx`e%*Ar+*Y{--QSL<0yF|X4p=P(U4tFFfJ z99=|b*b7{^V@Qlxr?+N!jOhCD-5cFwl=RM`nf8dYJ38S7Sbt&_-?)Ic!@i@6r`~7A zGImkC5|f9eOI{O)Qn1Ae-D2j1V(IQt0R{HX1wfiVAzsSQiZ?$(K(TN3fM;wg3+{t( zq^#n%H_yF=)qT0^yebAQufHFx!gV(qL4n>%YI9#`@Es?^1UwjL;sH*gh@dZ;DnUm+ zeKGAb{5<D*%aqnq1JF1B1(f1~+ zzN!jJPTXHx)asNx139?STFl;*{k-V}{qnjI-Iuw+NivLzmU)4C;j|VGIwlJkX&9!KJ8LUd@ zHD}v{66d>QCc3#iA%CXjKsmk*A?ae6_*ML~x5OKb#jfL+mY);asNZp)6P$0)5<_`P z6Uj9P81(DoJD$7p{)Gs?{t&P)Im&Gns$%IdZm^j&>l4lVXNZ)7_v>si^8DAlgCb3A|U@5*+9ziy<7ml`U+B)wciyIoBpX6$qHZ)t1Lzm#u-jVg@0XF;Y zGx-r7FD|H#Z~k~Mp;0N)h`Yr69jiOJ?}*e^iR;cMt5&s%Oly$&P5`h$M)`*Gc83E3 z5bx8FVe0tHXV%1iv7)g!dgHe=k^mV>eepZOJe75{Y8=$%~Tfz}0J_({}` z^It-xDHx68qQB!bA_?rD{S*W5xcoUm_9VQzyiMI(0zcU=J1=)#Yn4l?Z8_q2PG9(& zG$^Oq?c8lC7&W5-_E6u`v6#)#zBA)ZM~A-8r-&!1;!__j6W3&nbcA?a5!KhB?LbOq zDV1bQk}+R62$@0`1=Ghr9TWf#SmvUOp^xme^D}(=16pa&uTHTrmA&P0GVxKe>!a+0 zs#Xz;daD}_zs;G>a-92gbw7LS4nJT};b}kA9Xma1-ko9YI|n9vDs^9Nn+Acs4=Iyc zKIBjXUp)ga9TXDMo|7f70EQ7RGENF-6#fi~$3kE@>K4Px!|ci6eLkjIE*wvPm4JP# zsbwi@abN%RDQwcI&&|`~qnoAAb?qgBaeIko&`2YRlFR>TsAt`){nV+4KP5t}5?I@I zm8hCP|B3l9GgGkE_#3BxL6f8wvcH-pg(mJ4(4zMO(`eU-KgRy0{qFhpC8?sKDVASE zb}O^hO?>gcG1j1Tt2ZDi%vs5K|J*r|JayVrh_4PW7rM4qoXl_DhpEat%JKU0)sZ{l zA#E8joz~ADB!mFV?RJDE$R0Nlc+3x;4r+H|UvKftD``Z3X_`OeIFZ?Z>$TUjp(AtE zM%b6q2>1H@`L3~o`$0t`_|^O@y#aQU=>G z0_W?|91uboOEj3uu=ISRkyNW+?)K&kj+QQr%U)7Dz&jI;?*RW!c1qy7P%xId&zNu) zzdB2lidW?6Kk@ok(#>fDN_%R9IysN@sjWkCsSCN0b zr2kaD8$Sc@0yf+dqR&KGnCfa`CN#~0P-&p|_rUt1Z1KB~lVY)hkZoy2&(&Y7^KPe0 zaogw;0RDjiu^DD85bLPlS|S3I^zn4gTHF2|XA`DA}wE(>uxA#Pp%s*z*cVV)mqa7yI$y6OJ&>se%1@XpYDVgWB~b& zvoN_GC0fOtDyaV{Dj0CPchrdZ#^KFlg>=C6JV=BpZpAgWT~ruZ2H4g@@`w5Cy}+A~ z1|%OFncEhjr85plU=madCA>JMLHb#j9tP8cgL%a381;>o|qb^Ml&>e z`)_DpFJgop=z~cO37!h2O^#UGC_4{0%jT~b(@~f+vVjoh-Tk8dZi-G<|@Da{47=8eB8FhvvU~5l3!WD>rsLOm*H{^?Z#NYRIX*#B&WY9dZYEE4Ko)~|0 zkmY~mLC~*8=sMz@UTjBd8{-Up%I42Z$&n^xQ?e%$`7Ag#kTvWzmxRn{q04shK$=PN za~S>-4#&Vb>J(suFSh`SII+JDGsVBPIl;RjP$1gN(0Bzi(};Yv*wv+>Jmj2v?0=#= z4CgxUAg0xX*tP8Ng(-mbvK!9_eD5b^y~tHOAMf7D3G%I8{&@Q6cbjWPeY%NjGkhsf z@Zf)dxV)t`fhPVpm5x0*AEsdR7&uu4W2SBtF9!-bPN};UaI=xCWH6V>{>dEzts#8I z!T98;?M(ZY$lX{M+}UCyQlRp2RDL*#54oicU_W1 z(+(l4A#B=N2H)rOjeT3J#5MBgiSvaj<^4hnR4CNj8mue4)nB?}J^F7kv+y$PTI4v4 z0ED~XLyo4N8LLqC6NfFS4ANpi*avuz)4>wWtU54S3LWkCUbG}Z`qz2j*{C#P<>v6> zFM$ZWY95;xhH(V`t%ed}&4mya7IUj?50#3OT4uP#J1bx?lL!?(?=?~<0Wl-%UZi7a z?4}&8801pVbuY&b8P)pcNu!H@XDG2y88)5&j4T(0IAL`setq+y`mc5>7eN zM)RZCXg+kRdqMwI0&b;W@p2+XN7nZCf^ah{d$A@Nx2LF3;=*K~X+z7@g|DC3;wXSm33Qcb2KS^EaL*!3HR}&)?}R~I-B3l} z$1<=?ETSn^&N;nZuyfYI?upf8VT|>-v!h!RTPfja>5{^C>v9SNf8q}E@Rj>5dA8HR zPE4GL!2MBB5j7rvL6u_7G0PPkF~d51o|S*}f^H@@xUbWp>0NT+yKXT~^x*42M}h!1 z*!40onKvc5U-q3w$1=Vlg}4#;O5#fuw$mmOh!y_fD*?OPH0x1Vl(BE~pDq4adknui z_D$g}OoCJi*W0#rLSIFGs>60e5#{Ed#|TSh!>$vyib*0JXzWVhF>zhAPUGzm+U?K) zDc)kP&Z^Lp#Z!-62RoXqP^Rjp>H1iGCEGa-(I zl9-!p8d{butpB~xBT`UlW%K;7Z|@Z8CZn`~XwzyayV^A|$b@F@My{13<1w7WArn$hsob!w zL-F|yTwCTC&8l=Z20#fD@P?r>K|<2gzN8RLfQ|JC5jL)7BjJ@LxOdBIM4T3#sU9Hrss;uu4lHmXeuo<@aJqPp+XVJqKC!L&NtN>V}Ml)Xd zp0oerTWyhW1P45R9C8n``gw9~O2&W{+zbxXSt`5Mo&n0Y2Vn9 zt@BM%?Xts6ciVD`Z5Vc-PQ@|voX^KxJ$q(-CqB)Nt;jQu|C{Hs40pHJZ^S7rDrDNw z!e{G)Q8Qs$-~c9cMpnExDg=N3k+aQ z>hYcTFU5Y>_hmh9e{qlFd~I0pm0KP-a61!wPssF7b#J;E#UPIgA^jc(Ucan>`maaV zj$IE1H3TRAX6>8=#_dLkN9ZA?@gPd~ip|0l?Vq59I)@Ak-+8tf6DE-3FHuk_r`O*J zUpvBQF8V>hTp9Lvtq!Xl{kZI)C}n_+G4fU@NAsTuiJWwlFgPNW<^Aa7AXjjXrp^3@ z-d##KxU1p;wom9umlD^qVD?Yzh`Ih#rlprL7fD6g6ZQFs%g&D#Xyx`Ln!}bPr5_a9 zozW#kU_0lq+pEPyBa4IaS>{>FzPbh#nh4ve-QtHn{yO(bAI|zvi?QbTuH?(YC%W^L z-Q&mt`SbiI2@1~ylMsE=gH^wcgZe7VTS+LyrR+&QEp&=y!o6Z@>BsbYk&+F zzPoXIV82v^&kj!S@yEO5Yr0|R9X_Y&w5PEZj{R=P5giWxxH@6b@1FZfLgs!O^Kpp4 z(Ne8D1hmq!$2!8vqU|8bW^}==q^*WmP@O!aE|Q z_uty>ZK%i6SgJXrTnv%d{?GUF!io+LPXGMY?coi5vHIoD*gX3K_$3#2m*-Lkx;H)3 zZ}KMA{w&5w?xzyEHuHn-kqHx`=U0m9q=D1E-qz+MLCng={$5*tt!IQkTZ3nN42K0B zIRS67W?_anYSj+USpWRR-iCx^XX@kYK?Pdbb2`1KGBa10?Nh{bmQBJ_;`(ZVq+U2Z z%i~_j)Qv>e3ZEptiz5v)(+v&Z5XsF6H9hA~O#eGEXYhyoDV@j@xo3K1^(??#ehjr} zmG0KMKH#&tx-2Xb)SOW3ina6`g6d`baN04P%GBk*`a6<(*TFTlcttj~uFy;!{k{AN zzU8(+Eiu10=2MjP%;1NaUqqZxO+9~M5Gl#l*Ek-#$|s2pj zSwdB6kYGVG|HTA)+L}78q%!)`@C6rcpYHSRfSCML>->hsd?guBmMxh11j#}Xsn{)p z`BO&IDH<$KrF(byyY!t8oVK=-F0X?BOXR+JaPO6#)McnA+E|~kPh+wHIi;jN71O|h zula?(Jc}f;+x`%XKuUk%&6CHVI4XOAVkXpV{fzp#cs(7|3mj`ND}Po4N;sI?_|Lh+ zVGh6Y@9ut8mN43enm3$T&Dz(-*qe(Cr&!J$>uVM&6F;mVtcVudiakMZ)EShcW0px7 zZ?qwPBI)|MO@;J1HlqB4wRKpRY_u=BVLFcK6Pi;7e5*7`mAQaV=I%3yUQ)`mAIQA5 zVe>xH|Jy1SYgbub7f8baDA8p*3N~_w5LejwOphTcVSUvTHkv9TZ_B9|%nsi!?_c;X z+s+mcdU%AcU%_^w(XGc2i+;mq>0-W751V)v@a=Jo{v<`1mWYq0~eOcB4~V zc{NcZVN-u{BeG8__@>wsIS`$%f1eniRBdJmJ83_8wY8bDyi3gv!QMaE^vL_-gKZx3 z+yD2=wrx3RMH~S-Q@H&*l~wzXzDjmxSNy0iTSpF6_~XLR)S-xQnY1PVdxF+^f&0IL zFIR3p<$1~th@Rl$BQZw}i^sP2b?@fOY29}_g!jeYtkbJGqh}fwFE&#iJ{kF@20rNr zCd7}7f zNh$p23Kv5!ce;y)h~YazayLalAZRMn?z4}7+_F1&``d+hK?1*oGXk@$ZC<}EQN*bV z_@usP5q=(C#HECqj>OmV?$VsN;wy8vt%x2BSDyl;U~hao-#luhpXwvW35`P3*zx4_ zSK;$(CFRQKO2RVPDr6P*zQy$hmV^bqjQUVQ%YC40I$4)eJKb-|d><_l@P6|BN9%H( zXvHi5I#)6G;VtHl-WKKy?b5=TpM1;6e_9oP?B4yNt>aqx=lNs9!&kK{DSO{Ot6MWw zh}yu>sXgnDwjJ3dF~1pev?G5CgV^GS&b$Qt_-8#iKl8=e3E?Md??w8m9I#geShkxmrJv}3oDwpf?RDI& zi29`?mpeqA-9s0Q4IJsXf*`{tkskq$+ zT=!S_NqZ(Ix`S-)VcqO2+}s8QAn^H9TJGkZ%2yDO+_FJ7;hlC}3D^mXE8P=8GrvOe zeQI3SeS?n#xINnO?WN&_untUGP18mYf4mZ&&3~?Tjikrd-kbmkHI7QvgZKS4AMb}p zD1VR8azhF?GoWe-a~ONTt!$MLV8fapca0GTN98c3QwCeh;!BuxU)c1WQ?Jxmx0LEL z36zn+^ePXirIwKPOy(VsDsT5shV^1P13%KR`D|Q{TBitDY`0xna|G4xwT?<*{y6&` z2%eax=>&G;Jatfk-@a&`4I=V8%bAQ zMWC99I*&mx>n3j^=)0FVX>aP8L*66vDRRY{AGBY>$3&-^M{V`>h?M)PsyP!9m5?s6 z{i}(-Lu#S02SonamP~#D=_@v?S_d8c;3;ltOoNUte`O6ZpSf*v$!3l({M7 zcF;_-n7wuqj&xXqSTw6jy%WWR0Y&Vz)yAGVYGgcbG+BRj(sou^GxXeg$IThxg=i;D zgu_s=Xb!w)!DaaN9m)Yy`gdGx0-r5?=H+a|u_Y!9_iDCv=2-5EG{tS}JkMM4biHtpmkC;Ad%ai8+@Zro$}T=YPZMvR1OimUDW! zR+&?Zgq8N1br?y|Aa|!!oPh zdZe@ccn9(Qn9PVM4>H|n4=}@0#A4Ja&^*#J4*Vrnp6+54qDh>_4@-v2TYHK=vFi&(B9I53xD*IV0sPt7-BnnEccsn?KPf`De!@s>Le&tu;2|8jP@@&Do zuDsvietSsheKn#8v^3@+epj}KlgXwx;~=a^rvz?zDz>}&;Js*T1pWmmW|ojH3vp4Iu5Ivc z#B1HEd_rZKCpE8Aia|u@L#co_dAH%mDl85rmyt3!NEqyRAA7NzwBt~0fY+kRWRm>l%EU=d=KOMn!7sk_x zwN{7<>CwHHGL|_UZin;D7SG}7>5|>abb{YriVF|oGZiX<^4I73RNwxFv3|+4n+Hgl z-PHzPwGMK7kOn(PB0EKCXaT3YDonsnQz4H5qXXgh6URE)dJ)$A(<^cZm9^_Jf?_kD zd%}~+=A&C7e%PTf9=tw$Y?`_x&(bZD!|yo`^YuQp+V#a(?KdsQI%C?b6 zBdBDIFc=6%)2j(x)MQN1l)43o0%QUb`yW)2eEYAZOnXT|aGtoc@ zZA?n2+&}Bku1wT)fv8qa%dOjbY_-^5)Y0Uy5Rbjkt>#t4x>m@UUS}P49j~2lhL@hwfK}P=4qvyW{{b%#ALz zwd!3)EOR5x!&**6OuLE|zzs{+!i+OE(>oca=?T%KYXyZ340TA#uYJrvw#jh<&gjHG zQB4<9+;?Bh+yjS}sF8zb1V2N{=|c$%LPSyhjMA(~2PJx|hCE4dbI4bCV#5LO8?x$_ zql^@Bk*0Fu)4Hzrs*VmxGhxW~OyrEC!7aRx1f2y%f;XPAWj{!JbNVBoeX?d!L0y&# z%KDcfofIBw{i$Zxh|-cRFdi0I(qwA~nphO^(3U492A0Bl2G~2mmokFm(VZjl!@udCme@X!adSN!h!Y=cz%vaig z7SIg=N?>zk9PV=A;J`f>JR6w!5C-rcEXqJ4CyM#fh4IBDYdU0|K>J)DLsO5Uul7r+ zzx2_Msim@h{b`p+6{h4lR8gO_42u*+$)_Myn!2SoCnnqMiyEei&qLq?V*1kWrstK;cls?LA72-U)#noeG6 zfw}R}HrGF$nBR{7(K~fgI7gnugc6Mis#;23UypTsG`qWo*y1;QS@WX4`&uy0N@x!} zOReIv{p3Tp6jgKzsMqt+K-ebQgW{Fn)fc+#sC>s+v5|WfnO`v2`Iw zSO7tO+(q2orZS)^;P9ZtI?XccYgW{1(9Rtmu>uIN`f2SD#IbO;g+-r}>P@_8D_!L@ zQ?`)`56vCw639pFWDOgzDDEGsGjo{={-p!LkW%oJ{98G1;*xELPh?@~?BS0tB0{GB zpTKdi7Fchk)Ul{BGKyMzjX&T{LmellILW(r`YUT*#vROLfW2r9!`>aBDm7WI%_!I!1e#wpR%80XI0QRh*KXXG8Javnv?IjfB%|J#PP)O8?F(v*bRa&rbqIj=jF)5V3!Fil92`hNPo}LDNbX1B}*~?P4VORV|MvX;Sv6JaEt|%*uR?GwE2eih0=h;q1 zyBcb^9|$S_aIc<=J7~YLII+C(amv^_g4`@ls0ELyx>DL2zKC}?2wW3q5xIyla{prC zTx?3!z|P!ZlY?hxw~8oAc_Zt+uay z+x9naj^4x5kt&TBfi;SKSt#Qv|~pAlX2xnL6U|7(^vx5etT*JJH^xkA(yiD zR|h;*t8S|0XQM$QbIPiMNh){K^@EVoN{s>CfH!ZDq2^Nr^ra~qFFibdx zSkaYC0EQVc-`M*bt?!WG@~h6e1dXl2I-r>-eh%3&Mj{?*B4HA30Gu`$~avFF-FaaF-+iEI8o zw|;<<1BTtY01{h;tL^2i(76b@hYv8c^y?aO(lHJXktGFB9oXmV5RYc_Gnoi+Z_CEn zOvehEaHs4~FjZb{iF5v*RI_aB{N*&QMK_B&!&y@xC#dCnDXD_4ClGn-dtPu+&N2?&Ei7LtdH$vFYpwJp(F_6sC@$~xN6`4n zyKPxUYlT$xn;hbWYemVKBTkjg7y@u#bJWq1<@9^-aozSqC84;x-Dl8`RnplRhqc+* ze%dE?&xPl{v?tV5?hl4#hRJ6={1j(-d72?wv&&tf*nG$lJ!Njt9w!NoH<6g>s6=V2 z1@BxC4p@A^O<~O-2K9hnjJMv`;#Y-NBhBcSIcHo%WJ~BG_?r0W17=#guBcmbiGXnd z{%4~G?8egHY!#qc{>S#5pb9sG$$w+cL<5qvqicVrpVlVWyXAyEgqhlj}ckPK-j-wCX-JSJ<&|H8iP(! zp98r^)`8bhO9|^S4EMjs&%q1T3lcdpQm`uPqI*SJtW#!cl3h}tswM@p2hPC3&< z_RqZ67&NnH?=;g5e83(MdJr?hCgjG_p@1j(@lY!HSKUE>2@(46PXD<5Ll}lH>e__I zs5NusBW_b64#4Lfvt#Fj`Md@_{9!;y5bVEtD(N%hc75$etpu(25}zAvPtz~2@%&d< z5p>O?u^vlv{s#bCTD`N)jGFccqf!7APV?>aA09%ei_uiENfMA}tSkstk7(I=k5oIJ zjo3#885B;b50|Ve)9W|?e)G9UK&dI2xh>cVEz5^uBpTooIqVue=T#;FUfN#kx%r*0S?M=eYM%>NOLt0)neOQyyA5FTS|rrxa_*$tu*k@~<_9~=HEQ=%88_ZXXX=FdEqIiOg7-A!TrN6c=Rlvd4-Oa0 zC*Z3Of^8I?>r+f%WxpK@U(93%eA8_=Rp=4f><o~H< zcd8aJv4qXqKj~y}e6U?07UlB&i3#QnLx=PCTTMS~4sQ{A;k)I(Ka=7pEwf#pcm-kU zP}wwdQR#%kbazhQv^)G0iRko%Jp~(3~^&# zSk@1I$|z>|#pIejr)_eZjKSi479R` z`QATP*Vjalz+n|3<4G|?Sv0rYU0+X6>F3>D(s?aPofC^6FtU|vG>#kAJv31h40`#n zu&eV6qf^9!-BkLAo`)OF(N=UNZK>~UkCjV$9PWaO*#ugt@^BlR{gZ+h{soq9Wa#%LW{Zcn+av;{;@l4s z*0E=z1BVVlV!aT=R zH>24vz(JbhTmKH+jTiq47^?84t0(Kq)VnZ4^|jG|+An5n?;Ho&t$;xtdeBQ_S{mo3 zwFuC4xM-U@aTG}iV<|2CoCrQp@=~S%`6~tp+)5(iUc(mrv0{5iOl)L%!C(I3F}t}# zT3VUmS5sBOflC*f8<<={i=dJ6PqW&K`AC&ayrgQ}A^)p0jTJ5{Q+lcjjSHnnYA09B zRqt1~XfdpX)A?QpBqH{eWF)W;;G{q%5#?**@&GA+JX$3~G6+PHK@SSw)9(W14}bay>HL!Oy|eP)qzai2wlmiF@}wvl%%qv9BpLL9B4UHI5{A&?Xs zC@y_fmg6MWjv5?*ijca#fpY*KbFDHHryCqJPxU9%?0SC!;k`$o&jCcn0k-fegOQt^ zor&v$);078y$vR#91HJjptcwx1>8Wh8lZr|7qg}`F&lm@BPw4Xb13C~Lr<0{p{3I( zo|sV7+l#+vRsgewJ^i}>Kw`1qo|4l4Et%low~w_Ih^yYKNMy%<|5fgJ?k0mRKvBkN zCFAF6R$gS#2=u%;Op^{}RfvApZ!-6t*ZW)T&}L$=^!9l-`ar?Kj_dngT;w-%ed0(* zGZKIc2&UPhQ9-Ra_k1;b+qf&~-mu5#NF6a<|1(NAnGFefG~-*-_l*4s2?=;|tO-ig zr+gT!)NA#p>P#tdS0B6O9R4P4oG!3W7puW}V2UeB48AW{b)MTr!-)&0HH)+?r^TX3 z63`Wjgk{7)lW9+K&f!&kJ#5kUuWTbwSuqdJ7*QG}wyvEoFVVTUXY!$d8xwQ}P2JIX zr>g`Qr8C~uJOW+kRA4LBhSFoM`WvV0-+|{yGv}wGaZRQovb^JJ0HJ)alnI)981Feo1|-h zhsR+9Jg}qBAuZUP3pC%I!sh%kNTt0Ph%{=vNv~jux#Um%$0=r$l}9K(QqTSXK@?x* zjIN$dicdt&jNE9#Sb5|sKg3Z=`o!^9jL(e8P3}pc{m3FM081y$A>$Hh?KN{x{Q(Od zisXP=6MVJp;}G!LP~0CY#K5PWRqP3ZFp+GJ(iC5q(P+t?Va+VGZ`r zpV*x=ZQhL|s+(f(W*O72mlT{YF5jgKI{ZscF_u_M#5~`gR{HB_9Pr_#X4e}*zwvs7 zjtz#(l_0m#Y6o!Np8a9#iQ)|HIog+Hr<-|dcR3sGG}_!ms6e{UISOU+@zOUky9@Xw zF8e_aQT&9f9tWU4I2X=eR!a+Zm(Gi6L-l2^id?QYuLDnXjBpMMbq5v))UJC))W~i` zC=s~XH@AKkhK&)NYrzmAf!5=N4~8a;RT9FuTtzV*){#S0kP+bq_o_HgmQmSmOt7*k zZs+k}qqP6|RYK^4ibp1&=v_A-P(TE_pii*jd0X{GjrSzLnVqb+^Ko~+PL#9Lcg=8= zJQ96b2v?#7%T-CnqGdzusx1T{mo)q`Ho>vDmg#>u`k zB$qmWw^j6U^6&M|txk)yjJn}{vlWd?nCU7o< zw+(dLf2)M&Ck3q{7L8ca<-QOBk*9?&2j6=*ZXJVB|{MF_09(4`FSoJwsd7B==$yQ>F5OQfwOhL+K&$j?}U0u z4JhdwdaH^bN_)4@Y`hg{bdOS2LVc44-Z`&m5qbs<@l)4nR_TRHIcILyhed6s-r=R9 z#2)?E`VjnCei21!8Y-#DDMJUG)bPBf6EHC_Z&C}X)js1F8p>W9aHW$ImVK21Bjs}1 zH&R&wQpqIevsa@3v}9Ee+K6`eF?=<7RC1{Zic!~Kai?*@NbK2ahu$?BypGX3merT~ zI_5(C&y#fNnLPn8#~E3uam5Bp$K=UGb>ic6wttO(1BkR~a0h%eGx~(6qSZa!-QX0O zP|j6tnbqg@c`~}0$r^*$EAS#;@LY;RHa4H4$&&slP)#Hm)l3{a{Y37n2B)=k5jvu*`2n<{5&8U0i*)9iJJGcVXQg*Y#*MX?6~_P&i8gs zS#rm}fJUgwtJ zLUJ3?$2)vteMtB9DKW@Gf#$!@6bz}EL!Zm|>j2VboE&h$BxB{L2~ z+%735N;JbVWk0(wy^;zh1MciNV4&BB_IUxAOm8pb6cl$Z3fW7Gl`Bz};H`V!6$0NQ zyza)%ZKt0MaF#iCT3V&IH+2JEfClCR7}5eFo}3&agllp*Z+#@V!>ols$Ljf%U9{(h z{#z8=nh|ck@RbVzyWDd=_JC;Ir=-Bi)V5B?XON}!s?x%zG*s4E2IXbrf9Wgr2dhiv z-MFadvm)4-7`r-pbzKcU06%B$iIU6W==mOYe!33=D5hnOb(HDuHOs;~s)U6G1f7D> z538+`AlycNXU`?s_FD+va7h?L>YH#?$zhwr+x@Yd_viwbh*| z|2D<_ivq+!)?UdUb-nW)G-nDJ7nBa@qo-T*6j-QRbf)Y0Qg%O((`g6@L zIgxe06Fyj5V$RDQOCUFSjP&cttmFH3n6?F& zSt6@k!hr>U5?I(t1gkv#pX|b(b_s&N zm<8eqh0UxSb?)ensZYKHZ6I2&kwJH%$cn;P^frvsR2lQxsnsrdI(dX1)7TE4*_h4w z+*jopk$56d&hFY3ZcA0ViA(5=)d|GpEn*E+EX~c0?JCn|X(3WmOEGr8(rY4LTrZ{X z%SA;?sXV& zRUc%o(GxCXESfpru@@@)&I+E1PRSBqdfpL?(} z`2$qB7@n7;ptQ`aU$g|3dZxsyCI)s2Q#urOI43`rOeqk7WU6) zauyyb-Jq;3(RW(}h-c*4#(5(d-h7l%lx4ReA!F6Y-6^?`WHT#L^Mpt7I7@}MpjU)! zS8aD{&3;x<73WTJPvrP4gch!;wnB%HSsZRo3w z6u`(;HfBA0ml#oB>g7lfSfzd*5@NHbh_2;bJ!$0kER6D1vf=064|EjYLf#kmFf@==9R#G8b;2?i)_GX_%)N3H*A& zBt`$%0~9YxN=vP-MXZXWbo=MS^>eZKZN2r|m@(+t8^`C60gB5+0)M(h@JC^n3le^& zL(5+sB1J)GW{7pWs|e*U>jL9A@=4b}&Pgpn2;$jc6 zhsL4SVnJ6)DrI#H+h~=!>ov)v9K2RlJ8r&HE#h2Ogrl~rtQ+a;4Nl|Dypt9|49CJ3!aUp zZ?*1Cx58(%ONTW6em>TAQjyslqb{J?rK`yh$xCxzmEzgM@$X9P1z)1B!4oq*zosV0 zDR>?KohM15rs}MmB$B0QSQg$06uR#kip4C`WAtCY|M*GhTYe1d@f99$vT$?gb=>}e z(xH0@wfX7As65D#6AGmxzC{~hwOnuU- zC~=Ro^QO@`m(?6Q zc{irVLw&T6U53O4kza(J8`|>LTb;CP6D1OOBpCqPHw(9hZrzv@R2Jd1C$;-*U$i)h z9EJgg`;f>nQ0Fho!++G233hRk&`@R|Qjcm&BE6dA=hleKr+N*8#>w0$b+7&EUm;qy z2zq+4oJ`ro$N>Mr4J@T-lkSw&nJFco?h~^CPX319XQs#hN7GrxHT`yRSOr8vLZnL! zkXC5~W+74{Qj!Ca1_9~Xq@|JWNq3hLn{+c8$pNEtBeyZOXa5(^i@n;L@nO4v_c`Z# zu8Rh8%oGc}eap+uyjr|^+1B*t`-rb0k(y)-l%c#H6Hm?7dO(lX+9v1USW$szhxp%7 z|EgVV;ny*WW#6Eq%alXn_ zrMzDhdE^9Af7psu3HVGM{kDm=uQZerCtRQRkNzZaRtcK2_8`b+#6jF@5lK&K2Dn7& zDM&gP2e-Kod^A-x$DJ?C(vXzr?6h9SO_WettD9f6#W)8Kj|Hvl971h(5+1Ej-aq7waRQBD%QhE-h8E9;GPiQ{0=|k(1$) z&hk1lG2%BkcyEK*5X%nNP8Sx7IqNJRsl$n&XvEFy54ctW6NkSNYKb23&~S&yAt0UL zXKoG+z230b?_xvp#SvwO|D!>7ZP{!hETYh;j2J{NXoRj+S)xqcn* zwVR(L=QWNm`SeIHRupabs)jG}hHFCGw1l4V5b&`87L%X0GLd%hSD-^@%7&jb2#_WE!Fxdk&`7c z1TkuSUmglFtn4DW5x;fG=+(N5SJ>^)zj_K;fm^R`VRH2*)I|GztDSlT-Z^rXY7ozZ zDYO;?f>m)&tlx+?{asSUa=}LIh>7C0=iTtd6zmP1%^owbo9$m-YF%TkZszzx%$6bZ z7Y%d`2{`G=b*nJK#N2QqDFHN1R{KZGiMBIYk% za=-z*Akk`~zl3XBJpDmuqi-nT-CNs~$e5&c2)_L-+CNv5?&dSQpECG0{rz;-gngdX z?MfCKe$Lfj%rEcRy{39NtXlqy@maMu7IqbN0sOXg0ae)nUQJViE}P9h=e&y}($WO% zl)RK>SKMisOc66~S})g2J*)Ybk5Aln+b64ye%Qm znNyGC-0@ ze7((SZqRAvlYs#owt^SP`9>Y4Oa{|`L-ci%sG1D9hE^8qWj+5U9&joOUGj=Cl>T}8 z9!oJXMtshg&O*>iF!s3YBgwo=i^M1%xy0{6J!l9@L z^WyMn97qJS5qJlyo^)mRSeR*<{H}*@##Wh`|G4rL^M9Xfko;9}FEHiU3#~7D{&0Wh zhdh5uOSMg7Htb%ItmjzI>>2_&w9K9>Bxxa~RutIw!UL$wV|?+*q3jJ6pJNC@NV~u> zc(+BI3$O?~0bWcc#OsOXd8x0~OZ}yn>3^!SiAlNE+WE@D)4V|8OW`N(wrb>T&U|^N zP5bFj8&a!ZiIH#qYcP0ch8O6OLTz^XPMc2TAL}XRHK9fkGc4R_YgV}$+q_~>Ef4R_h(*e$ZZ1$3(*tZRpaSYnZ3J@g^N5Y zYtDEOBKkJYDK83|(*vHpE7I~iP9z;73t=&YU#BWcc#HSZJ}WgBFGD)270+f_L`=RE zD^9tnEWK6n?&tA$5pH#QT_lbVo+Vz7!lt3Oqrr=%{Di_<}Uh)+~c z*5BydE#fNqV|V&0IW@hMlZfplqSLjm){wlw2&aTeJ!WBY)yACnT7U~VxDS#O@uNqP zQ@V|qZ~<}cO+N}i0KZcbs06B%_Tyi_uvxleYQ?aHNs0OTwx6#o@t%&VhLA(6f8ns< zaN@@&uhN%Z-$;8$#=9*$Aeq5cobF%RP#n1nNFN}#ni!m}8#munfX*Jz&zeFHkIjpI zehgf>Vbp6UdE6hM)M5iT7V1!uPd0WOt(x8mJwF~$%=st-eX2ZpEsZACt~De&q^_@{=PmcVvQ z2Z_s5Kf4P&qxoG^ciIfzYX)CW?*_xIB7LOUs_@8QaPBLc*n^nceVNan8-UZytLr$31S-F=apmz?fL0>LAz+;MIoA@_Fw$XS`}JClnpi z3`4?eTvZ9W;>hvqs(KK>TLldVyqIwvrWtfuSPhyk$0%AJw4#TkkbkigTgA7_S$7(` z4gI+{1llIktMqIp&Sv9V#>z6~kzNH_*FDc~J&gcfD~i6FFn%gBrB6dpKy&XRezWMC zu=tq0HWBl?T34bH$R3`0m+rw&O@HIb2u3Zmk7aD7Re-0x45qN3j`Px}9dCWcX~Dn1 zm3Q;Ko6q_Pl>M33qUeCg;_4(2?ch#I35Y{jnI3T*r&fr#ghcK4+`DOwk3qF7biP@| zy1L$yu%s2Gd-=(&S1TbJCF%kia4o}ROtKPtyn_HN+#pXd^B*6S!>dT8ZF(~}xCFfG zV~0O5;#M64CMQ!;2Jbp2{gZruS{vD+%PVSwmO;p0(ExNJF#}*>!47uX3Zy&6U@_d?Eyw3)> zBc4kuuQ+u&!T(D5**(s*{Hgr8SLYa;z2W14)c=5r$a4$D(P7d}(s@O-Shb|aXiDsG zRY;Km9H)38dlpfQswcY(LWVi}qt$`<`Pr&=udZHs0(gE$E1&mV24nrm$&{YmFV+|s zjt5`K!2?c=35vyz(feMottI@ta2#>U)0c#Ih0GS2Nfh zyLrd`Sa_z=E=PbZ@Ok7!-%V_P3Ny;CAbhQw0dGL$YL7YHqD=>8wH?y{Yi4r1eNAwE zaVDGHTjL=_?+w;6Y6cIF&DosL+xnN7P9U|O<6EbIg#|=T>d^b7FBNDXG3RxiUH%hcQ!e?vFk_WL}aV?yBHhbIXYm6gs3Zcn`qsFceX zl7!w{1V&hUtjBbxkUzVsoe!MsYB7p(x)b<~S!HTlU9=Z9si4$ZaBbUTKOmh6h6#SS zc_9M|$Glp7ihtnbQFF!$?B^s@6g5MoM{JedSl-cN>aA40GZr0ZnzKzHJ>Vv#N31Dz zbv+N2dZO;DoN#N2K4E*^BP|uRZ6@U797=*$t(!;I)5d)I@S|cQol>19S;Mu#-k-=-sZHi=;Ij1>%YKSGs>K1Q*StkLjkCau zP;vj&d57wa`EVr27k4@d=@rp+V^*o>6ytD8WAaS{-az=c&erBn2~lvnZhuRT^!ywV}vE`%ToTu}RIZIVW(o3Z@8 z{LcDodc}wTB8T%7gIs8y9jyGim-+ewN8^XIWFhJNnm2*&FOs0)-xrDK=Ls6{6(l|1 zoC}d7z`e()W!tF#gqqTK|18$yD7$4N?9FfV(V1O#jp&nq&8mr_t)}t?!Yy0QT!4>H zWpy*lbk)8W&K4VWqlfS3O#bwfr*#hpHm!MKON+cM>z6q}{hNiyOxKzU${LB0p^B^G zt3_c5-QB*U>4WPp?l$AMsFbR`dxUR1(%}Fdlk9F_1i=$`6u!5F5%r({OR{E52K>tn zKZbr8c*aB4wEN>hoc!94L=v*WBAn+Y-S<@r{v!}ozZB=_VyjQ$VdfjDeGRw>`wm@Q z*k79_L#efPB`aTIuei=_lZ=n)V79OB)DTkht~~|hFLYY0@K@5!1PAf7J@j|lt&b8f$MmQK1myo!C;m!y3e6V^5LH=W zF`sR3nwxKUEQv4Ddg4jW8-Lo%!wy8WHlwsc<&pzbCIwJ-<6-9TLgr&>B~Wl??PP1d zrccn8G|G~s4t;y%Zx--`r47l;{iMl~30oSY-o@L(#tUmtmVJ$<<>=<^X@-1$Wg!Bl zhGnHFC7Y75bPPBa4M{#GBm?S^oRUGee%9gzfe)4|=7u_qi|LiyNjJJi?ql(V&#{U( zyYSAWuB`$k-ewQ^c%S8(Oh300Psg0HY7vayN^V5R80pxlxa`rHjehNfyyXFe z+&2|ZvAfqebpp-bf_bMX|8C?7vVt7Et}~ravD+NZ^5P!<(RB4W;-=nMyTj|BdO7k% zcMcngF>hHy&)Gl;1xJhBgjtr{!qz%+;QPQ%z3mR&EMm9#=NJ_NogXPZ#|ANgr9W`5|q<@e+f6Or=q7p>iY0GmN9vxyU!Kvn>CY zPko#5*gWIAjmezz1{m%a!g#fJg+5^W^{grUo&IGPJs^wM{UPIkkJ@l19V{fDiei~e zAvO-N1t9K|4ry@N9<2eFn9_!Bk*I)zIcfLo5O%cyyWLl6$>U2a1N`uVy^_oDg&c+V z5;Dd3>sgoa=Fc5L(EjcS7_({BD%vI3$fgVqw`b`Ya+kbf0$`a0vfV!_76kiInvt+< zEbw4DBcENnVvaM7GwlVs9itfPVsMWDn#91E ztvFM5-Qm+d+tPV!ZSA52sLWAT<|F8MCw?dMEw5AXtcONQ##mONvoS61FcC6!bI^dW zKmA*&9#t5#fl8yIRnNO4tf*wb$0BOfV0B`xMz|mnfc_|`EuRekFoej>-nwV@EgCFe zaAmX+QbLoyj~>ZXN!PQOX<_*qDjs9l;65i)`oc|RZe(#M> z5b?5=U0?C4lIJ3U#?1Y-_*t#{YjbO>2)4h7iO&um!`3|74t*37nosm+^3dO!@W^l`a^|5l1~K82loX6jFUz*T%bR@jSGRA<3=$@+hCoN8ki|F+@C(i+&Nfk_NfFvd)EJ6& zN6%Im*29-C7du8+Cnt~m5PnB(dtz@HFA=1M3b1@t7lZ1PC+%jmtmt%3#6Yv6zGnN3 zvcoq%hgVd1DjfHAonf+PTew!}S}{5sUxRS|&pP-avxkkWfWP0S^$fIef8YVJ#Ssvf zPO5(;HM|OfL}$qZ*_5Jl~%+vjPLGe`tUALG-Nb z=f5}-d)QSN=Ft}H=99=trOt&9>Il_8M-D`q!jzyFu8*G_3;iaze{cQt$KsD-ZC2A9 z`dPqKC*iIAfNE~N?gK}X^7DtkyS1;?&$Kj+!1qqp(lSDFHwyGTl%)f*xwJX_kc+!|ihL(={)4h*jA`N(H|J^5^aQR+Nq9@=4dQaVS+ z1)OBcgd}B>igjYP^WZyc-`!K-kk;_LNo;xu`>7do&?VvW$ZH}(;PKw@U4$q;JT5>( z;VIrRKF}!jp2H^=P$Wx^&y1no#`muc);WH?>`^S(r5zq#NwCbq0@o{D+xBGo)7;n@ zvxA6|FQ0cmnDXGX`g|xK@{|^T>{yLE>U5TfOcn)qBBaq)w+;%bOoQf~r`1}G zT;oUquln#OQPCHMg z!E<$13M@1|cId`xQ{(r3SAI7etg?$2j1o5fS;pnYkju0C`tOFe(j#Y=&ydZiOSD6f z0L+_=dA$=-2|w%ucy$l}FN-DDkJ~NQgHdWzESh{m1*BOlwe&=WG(3RvuXVf9 zT{{LsU^W*FuQ1j@FZcT~=%8h9(BtV3FPgghmDQj#;+UT@BtvON<)uL~m;Ai6RlQ5g za>0ktnCdTs%TSKz6#|$Axw^o;4_Ejiz2F0M{YaFoY2MDCEkHN#D1D(89H@qjc5wI+ zSA^;FK==LPnvLYz_YVJ7NHNJFs44^#*{5R#1aqEV4OAr;@f5ThVi^t$0wDcq7AXePPDEgW3{MZfR zCo4YBh&(Ck%t8+BI#YkX8+V^H`})s(ETv9<->#`rW@FPj^~?i#qT-2P9Pf{RR@~%@ zkN#*hMt(c_DcPLLTaQa|YA-y-rv&!x_d4{J;^Sxeqbpy^HyxlgE1Q_+Af}*P&t2MF#we3pJ{h^q*~*Vjmaw#{`E@`fe=gcgR?CYYnwUo5kz@ z>t+4hln6-JYjn(tH0^jN^x{SGTOl3!FBJ8d{E}G-9W{^sHSVrH^`2koIK-NG;B_VM zoiErWC4vs=oBP?uXK8N~wvl3d=T5WDo{o-_w%An^pc;v@@k;we(a)OrAnJ=K+bRh! z^pt9Om_&g0x;#)4@Mu!n9GIF)t-^igu&ENN|7#a%tQ0N}f-swL5Q47g1qpgfNG}fcPcwv=W0m-CHkvI$zC-ZeUy-i; z;55jz?P?%k&in8hsN|hMh``xTZR?X=`yJudd&Mr%fIMBX(7(eZGYULzJcV#xHEHt~ z7K5-jxCZnTL8nMf^?$Uwdxk4$K?88xmcj+V*7NecSp;2r4mkY`U){1hl=6n+t9>xc z7$*VEHk`#lNl?0~(e7&-fHbvvA6Kh${MeqQ$J8NP!M#Tjw3!dCV$6%-`*_u%0Zf?& zEAaWk-iU9$TxTkE=<{D-I4o&O{g1P%X2&@?6O3>-{LtgViLojo`@QR)0RDlR2<5}% zcRn6NE?Mfu)YK~>dRfuoWO;Y6HnX*^;&%qTUtAboj;X2J#nWvxE$;Z zEBK8S_%urASRgRb++XU7bP0^SRsCE6`eb@B>{b?1Ym`%{;H^nxyUP)Ecx0NO91X?S z6Y)^0mrCVK{Q#ig8Qa{s&vmbkAd<4|6UkfS`o)LEY314Ny~ zDITh7`F>1yTKv80MO=a*tKdN2WAS#BvbgUdJKSdx$#d~TMd8?^TJeYmi0K341_6SJ zqy9{+dVAToSFDRjD03Nam%hiW;yG7)Fj7b9r^>f*?6}H_$IO2+d(ztmwd>*D|1qw| zo8+OhIe!1_t@jXw+m$&n_xy=x+xc;FyCn}u9#a|Yi&GKOoCRGdeswSb+sXIyAZ{aw z7m(Gt;yC|qOWu<-8wDhLweNKKGTH`wo?H%l_1q?KM^MZk#YG6XSgOXr3*n#q1x^io zY_4Z9%G%JE8c(21hdR6CIeq_)iK{F7ww}DE^*R7k+3LtnJo0`!_ll5b%Uzt@kW|WL zZ&OKObxv4vd8i-Oy}gl?QNr<1+vf;fsv$>lzi)>P+I26g!v1!u|3AUEX&E-sEJ9>O zRdl=QJ&vdk1H#mw>fUetUJQD~YW_IJPncrIG(;`Vxv{JFXX2)v@{oJPY*;mXGldyC zt1sCqhG~oyp}tNZP-mJwnkx5rHG>8Cz#Av=yJ%&LZ(2%&2ivd0P?%wic8|Oz~44D8}Q<{y~yzVb;Zx;@(s>5Jo5z516xd>nc1IJQ89CVkkcTzIJUQ zMtq0irmgmb#|L;%O>b%qxe^-m=`X^y3Pb-es%kxP<-N&49 z8a1KM$1T5S#IcKEq6-wpoL&2VJMavH_$Bw3ytA#?7!O#NJiSu8S~A(2v4*lc@g*AD zHH@4&A}#dLJqv0O1n|_>)FohV_|I03AF5;XPLqi(=76L=lAyO>R3U^SwgjpKt0_3` z4M~$^J1c^GIcmcHK3Iffh4*qSeq{`Bc|0`Ze3pQZvfB6R}ZfK|Hp% znngQ7y75^D(D7a`TFS%q8UT4m>y!xVZzq z``ABoT(0!KvF(dTJ#CAeLf7(AOkOL7zs=7hPr7N=)ZSg*fdze(%+a+~PufmU3GZSJ*7ox{~60xF^czddJk28rX)*%#bHV znYi7@Ug|z#3fMYlKxq<(arZtiojuVx`jN8k=clBvDWBDoPW z`sqihYULIco`~WlPEJD3-SLz0^Jjao%Rg}NAsLDH)JIrF;$KvRN@rNZhi;k26!sOj zVe;v%#eK)4;_u!)u$8SpBZ(th&t?|F@3>tLip;wC*iefKJF zxL*Pb@{=zgfZq0wZ~a>j9_NTv#Jbn|{p>k?NvVj*ixLMBFU957<6qm8vl8EppAIPS z5x?~rSy@jao?jX6Vk$S{Mhb11%%%bp_~SB+b7qqou_O0GN^ypL;rIR)xSI#a5=%5& z?A>0}foBuelFS$V5RF7-@cL(M=X~|5m{W@w%w|yYD%&U^wtMs#dI+wcd<3)ixhRw7j_~{yy>YAMXXtFp!?*DUC-lAZDmp1e7h>RlFJ;Z(}Pv zUBr|mgGbF;NqL71JrL}U78j{##WPia*B|imed!^VE@D~m_r`vV4#H6BNvp6~Gd~hg zP0;*CXX?yvTOq;cg`COmUFzJPvkkwj;iUK|8cn`Ek1EPHssupzxDNUsqHCx%Bs7 z1}@;IxEoD}C*f7hs=J1&T8g77&(=Aw^7x6J%_(3pC{Bh3rZCvV^b(Wi>9n;1yFB!3 z+ZX{)ZP9z-xJZuiS(5;#me=M<048jN$;wHjW^Lx*S<500j85{u4aG#AvwmBC z@IS6K%BEGX+Z%C&(3__`e)|L!XCH}BeGSd4jYnXgMOT>1h?GISqA?_GN_~igpnxm? z0O3>SBgTJ#`&h^wNEn>b(#f3Db?=LfO6P#RjnwLTl}}#;ul_?~u@W;`5v{JlRuVe! z=X?8@^^NPCu&`fQyN01zOx&Aai!}x-%T4_S%Q`FAYzxzRfG=40!mzO01oCQKm5 z4(ak02Mfl|Az!miG_s#pr4Srb2(I#!V81uUoj-hIoQ7|nY>^Xlm2d|qeRLYTlSqTN z9U`<=eYm(;O!inY`BvjBW$>juXav3uhMcnZdNq-zCh@9t4VZJxvDEf1;>`9CE0E&0 z52J?2QeXXzLA>2p8il{UI{0LJhWk)=fJfXU&f)GVttcujd&VGE@h1UCU`4I|HUIUk zZ==och2L+{@UDcJZI(}JR~B! za#7*VzLQh}6=HqrDI$vKZmAW4)U+iIsF-g_&FeD-6z-j1rfDXxX&!mkcHokTP+{tb{A|s{jYUS)V*e(u?E7pINpjmluV*r`Nyh$N8_r zN#PoGrQ=oDvKR2H9BbSIVO+x(^ja|hnJ!rzuS{iHgk8-HrdLX{`KfSJQIDy*_)q0U zk}VE=JRB8o>L9UK;p&y0&Tqn}0}1jp#5T55d9a}WpMpHQyD2H;0LJjv#w4M{?ip!w1{Nh0`H)U*c$~U~uS$@AtDF4aI~{p> zxzan-M8aD0cny!yYq2!i7wIemF$G$yQM<}=V+v*6jb@f5PRFG!x8HmdF_S1Tob*G< zDPK%b!<0o$S--Oh4;M9cmNea~^@yzY{Ldvu3bZRbWYyyfux)Lwp~4J6>TNqgm;#f-~0de0x$-& zwP43cw`|ZEdT+)w5UZ`i>$Vml(9@gf&mg3~Pj|lP09SbN#hwHu=GdxMdZf#5`ubV- zg~hk=@XjLV=2yRrlNh2*lN2Rysg=HC1};a071_nz40P2l`acU`O*qK+gV2c4V~gA} zX`i;6Rd<-**=fmLeH>&2xl>F0lEpd&G+E(O_U=XU;CFwp(^+AqStW%XEI;s zy_Y;@K`Uh7p6Z_+BPn*VwV5JK5*`l=-TR06P0QPF@d%2pwUx3uNscTGXvt#d;J%wc ze9vX+LWA9;pq77U6!6%Xlrf+ok<6Z$|3R#i{xy;pk4ZxCSsLOPDf2<{S6sKDN~eQ4 z1uvq`rgoSZk<-#*s|u0MXmP#LO0=HiMx--gLLdL(dSKbzHPyeT)XlzC_;KyYVhb<5 z2LXrmuuUELFzt?pt`b%jgU4{|yd`CsP8&15=+f=^%Be%VEr@@@jsdTu{> z{*HBl`gfibzQ&JK)j>D2iB<=N4#5g1Jz)G=KIm{<99x-+$6NAV)`SDr#=7X{8eAty zJ1v)iFx(2szkN6X*<26nH(WtfVwN$i9!))=d)=72;X_TW6Nv>Fp~9y9NPHPC&-w*| z=@wYYE{6V84+*9ZRXW>&yKIq^Lxi(qZxXBxaxp(!ay!6hrPo>V8JppK#m`SkQJ4bW z(!-N_i~)O7r4??&cfic3uYq=|Z=e<(mphs~KG5)xu`m-`Z@N>c?jbE!d_Lg&~=Hm&){uB)?pV$3Lp&b1TvC#`vQ%oBdb$U z2d9FBlu-$2p8a-XuHr;rQz%QEr^|O@FdM8euAK(_W%sOj8$<#I z8q~Bems_-A@>vvap9A*uT|f)t{8rqvjrXe>cK}mpb(3h#&ULkI#+R}CZR1P}|3&pqKQ9PvV>F;JI9>Xwpu=`TPm&s+rhi z(WQ8?+!`T};cC+$Hk=sd{PE9*;I*S_?~S2RY>e9>VL6*7Rb-9zv?iRn>$U%31c{a@ z$ztU@kQYi)X^%KHFOevqt)%lKtA&wRmoqnL$|e)A1>9b|PRX@$dqc8t$o(eR(G_-{ z7is`>J3DAz9&C&bxL9B%SU3|}ha!dq6js+sFDiEeY2RYS&f$q!o%eVky)TMn1Z+uD zi6DE!xuM5EBJs=#e5X&WYK=Gz6<>)s$Pk&P(2X+erbBb>B9f-Q?J+VS>{PVFUD z60w!|3F_``o9kO1AzHJ~N-8(Oy+=P_8jrV9RWMghIm!uJZ3VUkFvTb(Zl_%b=_s~C zQC(TJA91JDYLrX@*bv?*bIJTWN8G8@jyH~^V9Pdyd;q{xFUs35q zt)M>jUG1W9<6P*9xpmml6x#nCy0{49ay%flWGo@oXm_&dVx#vig8WY&msuTIC9TnE z{q>;I9YNc&)`;(~)`X*%sKKTqf#Q$vS#9c0;H-RXQ7M8&g1WQ;^O{AoySKK!?WlE$ zhgD)l@Y=8-=W`hxuI=jljFbT%B07u$1bfpyvas1xXjMOnW05Cr9R5;Ny=}$0GpVK|7XYMV^?H!Kn-!i2>+Ev zbHcYD#QfI{ zOl5mfJEvT)1UqOJNkj*PMhA1$bE{T?}*o8seaMefp#r-b4CxsR(&-2VhR(9LCD- z$foIdYLx2gYCq9?-M#mh_d(Y{UY_(6jAX{kAB$O=!;lQAJLo7tu`5KvYJ|w=_T-?_SEA^ zGrbE^sq%JVLXM*x!vOSp_&vCnON>Ty8@%21na^|}VNoXVXddHplbXkPfJ zv*PQq#5W6tp>uGlH`RF*!MO)-1g-_&;ly6bD>nKcZBTNb{tCE2fLHS#-Ah<|HXRKp zE*LJk4WaqFSly#l^<4+EYS?u%u4ay!K;l1mpazo{g+AZKZ!DEw(JP=jD7o7%Czhyr zbKd)mM!R>E4lH}%)gkTYi;{~td<|#l zUS(^U)A^4D^SG}BXI+&ZU>49>o;7ve1D}0sKVzY4q9G6c0$9AIUUP@6!}gnA)S=3g zK3_<+cnHq*#QBP1*TgWMjd@3zd$(b0LrvIFz0+d3sPVeCSI#%f;wms|ou3F}cWzAO z(W}SBRmwhgazN+u^?Z*fulg0o+;WJvyb(R!l*rECV*RvqckNOZIx@%k^Gd^SRjX@| z$P-9Uj$v6t;71THq#LZd(6qs#T}se5@_to;Z0|v)_qjUjF!DPHqre)R7v3*yLwC?E z2606%i>iwQ_BQjJD$4}bSs<4b^!a^a!G0c7KUGX|wQ+8qM)-|M!6$);Hyx(Eg1XI5 zL1=5U9?I`jP9D(^txNwid5e|LeN?Xr?Ou)vGfw!H^xs}5U0?jEVBM`4eOvzH;ukNT z`6U2GX&H;l%O3vp-O%buogdWFkIYr%glm;+u!+RnQs2p^v$f7@cz1^b(b;Tb;b?>p z$Mgo+`(l-b5Xxd#X9%vgV?H+d`;A*t!V%8$9UPMr!>SJNmCmS&3*Nt`^a(dzk)|Ui z)z8$(Csf)2pmeh4#33%w z7>X}!hzzhVd080lp^8002%}9XUig-)`fTV&9w!D@*pLZ!oO@dFiYyVEOQ9Eka0Isd zi0OTFc!>=xd(6jUKE(<`AGL?GSy>{ZUN%q0BB%mB90ii5)1Qyu^yVrBfIOeNneA>7 zdj*iQHL-AF(dSbcmIY22TB9PQ6km$D(Rut}k*H5dQmQ{GCN)U8M(C$EoR0pB{bYZ` zXE;aS4*R%{{?Dw0JRHw+L+GJ3O$KEzPI+r)-1b`3!$K`OP{-exi{HaD z5cwXSZF*woEYGRombm1Y-knER!w){>Nz4Y-ZSITb0QzMM1u)_1V8`T1u)$8*h|=}N-S=&;jx zLrXoHM~d8WU!dI*Yj|7WxEQWwUBf7);UHuAgiP_Tcor6Ofd%5Xx{k0i4cC?6W6<3p z;D0vfGn9%##bfO`Tqg(M`?u4lq*_4uzg_`&-;&oD8szL(riBJJxbPuf2@P)ux52EX z!D@G1bI_M#5YYApP-Pz3v5> z9!g_#F#v30b4pI~XX{Z}uzXUM-AR`TR)QUG!w}*TVA*>4| z4JnN-xNM(HW=pDn*26;hv%{%BH-~bJX!pTTTF(y}#KG7h*u}2FiNeXKX9c2<%k5Oy zj6jmz;_(3dm5RZ7ScP@irBW)!Y?=D7{=i{#6jH_$i?Ad$I*vUczqj+E3|D6iZ?~Y; z#kjA=EX}+KSOZfJ<_rD5FrXTd@(W&i_@U+od~2i-pz6b9vUGDdfQW3y9GQbgWdKun zGC0eOAx>juPjdboufD@W46h|}$1i>*XrYRWnef79!FYsE-w|mzyu$Mxo%u#DVvZyr z!C{^e;c|d|Z$eV_DyH$v6XL@!PUqXm%Ps^_D73pJ;PZ^JOe?YrMBv?hoe0{&HYdEC z&cdXPq(e?tNePu?NHG2n5h>F#;65@b^9?^-8(umf{AZ>1#kojZIyN{IiV+<8>+hr27#Aky;;OZm{Ic^It z{2?#7$UBp?q6fPjKn>dh@)%<}?S54AuL28jn6I3`JdvLIndx3ARbOj|&B7hl6PsB= zb3+NHY#?h32>5~Ant&|=YWm=zbD`2*T`m_p)-1Y4F4KQpZO31BNJbBIcuV6J?f_cpipd9Aa)jUg~t^;M1`MdQt?p~41c~9jj4P)8Y z5Z>%3f&$cfct3yN8S5ONv z!b|RZ9nODJ>-?vJxXi#+|E;a7Hyu_W^U#e zl5f;pq48FL+7#FjiJJ^4CCbJd2`0Ig0fH-?5VUGVeCa@IN5})Os?(rMH@6 zG!AiIcYVFs6M8n{e2m{cT1GD^dLk@IiqMTMxr@cFA@+8q5$^2{m>06WdC_xd;*q+~ zC*3{p(L8fz(Sxd|JC`iMq)W~TLZKepQf+q8=m6{0i-X`xAqx9rU0k2XDS`ADjOeWY z+21t1yVgZjgh7K?x%1peeh=ZPEJ)eFL{C}MyaSxH3N5KMD}U7}oq4B%TCb^RVOmvO!=?$vTn*=#Sfs{J}1(dX3<+k~gBiS`2W(8j93kjdw^p6$VN zWb&X)k$TiLA5B%`&f0Sm@!%(4^uj`=r@-B9_op{(I`j`1bR_e2|3S|&6kk=dV&?1Q zMqzQ{y_IFismn4UX+wLEFtE+j$>e}__!Fy7JRrp#w~rxs^@HF0pHwdfF0bzS)hK-v zrEH#HoF#Sl4`LBG4w{6`$-n8DSown|(1Djp?hS$!q%dpto%*_ufx+`!zbOZ?i4v}n zpS3T6G=?wS7;EF}O1VEhz}Izf`Pck$%oU*f>J|cg5n5e{PfV#tsb8nHKA}uH`O#Tj zaUHRr9MNotDLk$z-)tN#r_#qB`9&5=_re@~39s>a^T_f61-ye+8JQ+3kh@?gRZ-YK zlxutPgmi{kh<0J#ucQ(ddLW6!EDXaQ{AdM;p;XxZzMLjz>N_|8mFdxMO3xTRLB(SH zZf*Nf7YACM{`rze8STE&=3Qx0OhTPkN;$<=b}HQC!5s=p>CIHVN8Zb2KFm$uR-X8M zKZiwqnH}vlq-n@E^Br?BK`t)3zK;l%mg2RY85fN-((gsRm%6%WkI{Adr>1EtHh3>` zl8qHw85I1W0?lc$zt-V9%N1SmX|ToHH^P62 zpH`U8g)cnp2mB)k9^_EYSuK2?Y%Z9Glj3{`?6{%xD?Vya%wu$430H2`yK((1RQZX*JL+4D)W}WKuNj zT1Bk2uC^&`{K2dIw`v@}E*@NpW*m|UNM9}_fiEimfb8OH zRz!tDDE&wHX)u8=D-lc~@cFlu5ge*s0kbp~R=V`~L;ijmld7^R?|Oq-aC`g^PK>6e zxl*>@zWdQs2G}~9%%hs0b(Z`Y@?W}a>4OhCNZ2erewgWeh+krN{P7X1H0EXD^g>eU z@KvkCPa$tIN&ZaWTONIe_!O4@sjbGN`08x-_zg~B@%p^8&MXimwrb|J=OvfM=O3US zsPQ}G-|N%kLpfTgNAaIMKS+3xpVb#j>_B&=ygF0iFnb&S!vCS|f8mx0Burgc`~0_) zFiTv+ckcZ8u^YdH2c0X>>0KgY=Ufb4B1o5eeeU1*ZC<+h0a%A`cYLL@S>qQL!==|> z<5}aY&(AOgJyvt^x3hhGyA@jX_=PYrKWHGKrjOLhr3%H{?C~2Ffeqg|@FRHBzvtiQ zzp%I*;LjdkAWz_f6KZy{!m5q)S~h2mFPay^eEL@{n5=}bEAg>w{BD2$>k&9&<$DOE zyCJRPXXj`?HzWLNJ9+%<*yA7fwp%4ct`fv7(ex+S{OH0Q;DH_B`ymH8 zz>?!D^o3r)ACOY0(_fJ2 z)$1=L`hXaHXr-v(@6l1Rwxl7mPyY|0{n_*NsL>t$KA~aU4x%OWaR;|rxnGCcZ*^1YEQX1~5(u~ErqCLP*x%TBx z$f8fDJXaVkL7@~&GqpVzhEyIc;oHw<^{sms&jg27D? zwzPf563)bT(Jxi(T=_?@3!e@7;M`Iycp^?ZLvi5)7JP|e6J|qvIk=tI1h2PpP^C^p zj)NVVc_5N2%t}tTu`y9(sZC?rZUCWq(U+%SG1Ff~tH&2QKi}Y~YpUH{*e^C+qC@!W zD>B-q*9W^U1-lgkl47fH82)K2fL?r1bY|1wq@eNY4Kz&`)I2LT3x^@TkxdYsmg30T zs25|K#J+Q!cS;(1e7KpV$7icBfV2;I>GD(lJakIMEu)U+rI)_)uW;NI>=l20VWVIB1jDMlXDlS@hvOXVvlcbXsRau7k59IiD^tJwmw*;@e@=D7>0CCTK z&AI2DcV5@KN;A7fefo>73cFPAyZE{p{Nk^HI!xw|W?5{lPX5_v7DsM)IgUiLB(ZEV z5AaK5=5N~-Od+S0n8BB}OUvGcD-fw z4BxneV;A4zAI#RLDsh9-^yYI{wkN;THdi`GrWUE-hm{z+#;=+wntv_6Ewc0Pf4lr= zOt84G;bf%lgi2h&>Ej#BAN%pEKy9YYXkxpT`#XL&t>3tHV>cs|`kAIPP7FvCoL;!c z!bN*8o;`l2@7lWI#=AHC3qR+)b@7TNyRF(|(QZp%)Q{f{w{Do)B)I=TGznL04oU46nQm_@|nKbw|Ce1Q8!2582bI*GD<1-T{10Q~>bAC;Als^i+H?l+F1; zAJN|;Yl3fo6u1iF6RUcZEOy4%>hbfV;;%Sx7az#<+BJa-pJS)CrBRBr-Qy4SQ4p>= z=KQPh!K6+8Z2rZozEt{Gxts%vLBnqL?#_yb)Z`CH95-3jpT+j_7vKRueg<~e{@3(~ z{zfQqsyyqDKa(lOp^k42Qv%gqEBf7rH{ql~6}On7GH@A4m0`((tR_XW);Xw+Vl$}K zZ96%}qPTWV3YjvJh9*^M_9S(<$3Fd%BUOHl#X1{%)=^+EA|ZM5@#r9)B9G-`+uCXj zJbLBA5=1EJynAIA{P>qEx0=ny&*aoxdX2?7NWwC2QcA>)T2dR5^WtZ+028KMY)vz< z=HL%Gs|WmxZoD_Q#ox)%C@D(g68s_m(g!@fb+2v?`1j@Lz3+;B(Mkg8@1<+k;;RhS ztXU&tkaRk=>E2Bu2H0NZ5ZRV#Uha%bhh|?J3yiXqQLV;o&isi?^WrzC_BKGo83|C= z$U37%i#W=Zge+&KavNJ~0)O(qHmIwW+5D+UM*yCAik8q7W{sEf!6C^%nBdl*4BEPPQ-UM|D>u(So#-inx=DO;#fQ%enl#ZH7o~Ue_BPZ9+s>$KIB0Ke^eOyCHwLB8w)Njh#;@{9 z-4RrysCrQ2ms}d=4^_kX(ysI&YJK+;!5}6lRB6}vjXndFl&vO)3U348 z0p}J^FW!IYVZV25I#->Z`ra3ByXmjLwe|MRe5&ES-{{BZu?wd6dgQ+Q|H?xa?c+Ri z(+9t`{)(HT59e|V0m_|v(+l@pu*V};?05Ww7ah2?=FgpE{2IVn`(OC?^V_cRJKy*k z=07UdbX}QA+H7P0TVrPD8NZ7bFRGlZ;eRtfCfNGsobHY;j4NZEH@eQY>B9YcJ3h~p z;PHQd{H%@AaLs{%DErp+K)NQ%8D)`yy#HPc?pvSj;+VuRZ zWCSvf-<@A1j=E~+FP?>U`1h56DBuRly5)~g)&nN~Z2sHErwxlhm5~j-#&3Uq)cAXh z6Ag#_CBTneE?ijpD(^h{Currc;m^lEoLYPdMvuSk`0@h|j+EMMC<`zZ-|7_=l^<@_ z@jtpjm{vc+^WREHBj7&MgBH6GHM=SaXV4_jn6@~yCh$*&Cyr@@%B2GhJCFpi-z1RZ z=SPK~TqrKdFGgyoPZ-H$v@(*@Vv}$#{Kmv)J(uz`#=mB{9=}>ugRD2VtwcSr9W22g z^5-Z0Vv-lAre&1!Osv6CS>-u9i}b{Ry_d=2W;nLsP{L*^M*h(;WeulzrM2V?jElYF zRm*}7Q;Z2^U7u}i10Scs#jL$!1(|kNA+c*utIWVHd^?3Z40D~KI`8#L9%4{^;jOsh zjJvigex;A$Zul|IizKUe^4~VT!2G3O4m)`R(w&E!Zfr{@y~8(vwqdjQ(*Rn2(40a_ z)rZ<#zs4BLmL-z)*~_{UrqCLH3VVHq{1vjp@3KuYxcTCjz9dG$74M>FiN*B>&jQpW zrNj7&l!Sq(V#Nfr#!r^5!YNXXvfM}fKsWqy7ionmeDL({Xp25>woqeIgtbk_*bIqA zcEG>UlE0y=UNrt3rFD3*0x%Hu0sFiIn^8(#a{1J~|yP2N0 z?kO_amab0u6QO(#PY;IrI1XG!`eY-$8J{2MdEwXmlUC~JcVcO85cQ@$hON7rQ@~|C zE4~8SqVj!P_%ZA9vv?%31uOwau2~C<`xf8My64ZaooMMzF`Q4IA^&Kv6qDmC_&t9Z z4)t%s=WQn(zjmBRB1KJlogBxnKH=PRU!&B?pGajFca7i5Kk}zCv)3o-)aIo=b4#wu zf1{rqVl+6WqA3GIn{IqLy0II~Z$I|qx6>&D#Qm2^b)V+v&YFf>Hg2BU?9a0?2Qkjr z1M~fVkuxp2OJ=k1H}Q#QCLE5o@80yC58QCq7uO@R5%L~RBxlF&55V2|rQ0`e*z%D7 z=Wy)rzW&Y}_!ArVkvTf@v*pcqZn@)Ax8Cu&+YUP8(7hkN!XPF5!}tP7eCL=SsL0+9 z{EF`ndjFd&?B=&!u>UH3{?&}%$~s4uSxvH$X68QY8GLNG|5XM=VC-z!;yh&k z9SC02OFAc|sUeS9CADeAQ1cv>*4|f<8BF8MShi2g!Z*=!e9KXCP%_j28#{i+FU@HR zEY~aH?gmGSb$)?6VRgm(IR9A38|@wBNMHFGZ4%4m6r~kY%2}&OrHB#D;Hsf-a52koUk9rXIZX<}m9)=K z73s3nH!Zp&AuNCYt$$|QnW@*j$#7E-n$APKU&d`#}hVT3^9&q=C38OEq5WT znZRenS={ig&K9lb#qaqu5qQt4+(-C!NA7y%sF9gw0>AVr{#MajujENG?kDwe1|7%W zaeapPQWDC;Uyi?C`HEL&!XZv{T3M`&z`k|yr7Fy%sZ404Z#8_ z@?#$yH!4zj=bDViQ`6JP%I^TXwzxV$V3f7sVPclGMWuU)&gsw%$CEWK#uQm$nz ze`fQ?ND;v2A7Yr6(I@;<)S!C)G?}gcZ2puum^=JFKh3v_LzM6J;Wu>Gt-IWi*=B0p z73)?XzZzF;g0kKC_V`2ojk<(el)=t2s+pbty6+s*>vY`EP)slz|Ae*lkHH8(og3!Q zkAD1K@yT@vsg#+I(|L7ew60xy>74vgfbvd;mY>NV%k86;YMITEKKdwew86-Ql79Zl z=D+^&TltSTy*`wT##dB8pQcrlag>&-=JI1Re*5^QN5lL#P44UXUATk`6skcEbvrJ$ zS~_tElTV$!l2m3j|9t0?Z{GFg^^DLusfvBFo{wj9C3nlMn<_u~vI=V~?vHZ1VnJ$B5@w3$^nKCk_69iN*&p6Aow z!SQn?NKNOCtt7i=+c|#bpxC0g@?t>FUr;Z@Fvrj4m~G;`@Hwzn#?1W@H}#HX{#(^{ zLABLfItD2>KevZJlhM%wGsj-O}o&+Y1Vtdhfn_slfS z(_Hw)0Y^jqd;A5_WLx?NzsVTem7b3^@`)(H#DH{AB^3Uy8*6KdzZ=NS>#WDhk>3N>E%>!DRl>O3iGaNF}~Fj_7mZ*CpqMZ~o|APl+!V zVM+#z3MTxsY8U!m!q0qYSz5VjQuUq-wOFN{1vIGqZ~Ew#%Ic}(C}CC~YQSf~67do( zZP}#Ic6ORxH+@X_!T8cZWtu+9Ly@ImY-jwMkY(HA#Rjb{oYX&FwmW`lc*Q5LKp8m; z)6hP?_R8jkv^Z}^a`r7ixj216dtVGY|aW=lH=-d`q zVnB16dXU)ZpMZ|x7d;xAr3af~^wCK_*9@A!! z6GQzQzN62aR2i4NhH#fZDUnpmfAH8KKNWK49>41@zrx(0SCP~%&P;QeJXy#1L zpOrr~KO;*P%eFm!`8~Oxdc{MxZrns3a-hEXdt1K!uItTU8g89g`QpRK*xLUt+>@`R zIRD)B)$eleh6b5?rWY^V@3fV>ui9(j;stko<9qAZ-Ms0>duF^1*YUfBgPnF%GQINb zLv}lK@6GEs-uj7~@4D*tlyBv6A-DgJoAy6tW#+@3bpKoVvuphRAojmIA4+0I}$lCb-HHq=Hi8xejMptP7FRKd?s^uUpcgN{%07m9~d1!Tb7y2 z3ltuzY2r&ieqE^ph5^1ij~sVU=2%?d9p7nMFk5gucKi(g!SQoEwEb_xcfaE^Eck;@ zN zqT#DcbDSBb@F~&&K0xV4KCBw*9YDSvo$%@6YcLO=Swd z=|j^_AJwbly)M4lzYmX}u|{TvFqCX%uCY5wExYAuO)>VoG$Qh9*fDoe^dG_T^UgSa z$BwT|48s?VzW?LrN`s;P27@5rlo9X%9Y3?Ll`zhTyeaMV%NFShKbe_qr)&01^Q_oa zijvi|bj{pI9sDwwP{eN35}5K*N{M5eS@@lYkvDufmE~B(Qt!Xm>d#gJQKKr%fl3I9 z3PsUNE)B9nWSc_f8R$vgldkmIih1#oK2YhB-Sma2^&hBYz*=1!hupuO`m)2x!lcB`9u3PC$HhxXSd0}@x+(~Sl!Nr z&qNG<@rO*z!(EMvzs)E3Q=nyMUd<_}auTUuN6+fxeEJl|4vyc#CsMTPIn2%9KpKV8 zKa$gCR(z+XFSy_WDq3%*r_MOz%!@9%h|SV}s84lx7+;g#T$|LVNh3>;G_aeU{zH9K zztew+AAxD*P;dCj&uDIexTRf(U-@Ye{Wk}HGj4i{hpqY)n*gwV@|rdEAc@NV!UtLx z4M_qOK301C0f%&MNGLLIL;psdcAY*wJ}`A7eXoBRCyx4X0l&-7(!l)O3C8aQ7m`cq zQv!76nPF=`i$~tF#ili^?7euw-Q@Qv0ieg5B%@w@CHyG>2;-(}PKdyP0L!9|LNGrJ$Q zH||)>X-hr)xaC9N;wZvn5tQA1@lU>R)slzo=J@5~+yfTx^|LGPx^g`Yn*N+$k%BQ_ zxcd_B;8?WJ;+1Fae#5`txZ#_3Qcm5y-`?2sciJ`|!$wFaA{HTjbo*b}&_Q*O@viZ! z5Cp{_5uyJqkvgT^yoeTGL94&Obd*u>n?1vx`W5hNo+F*FSY+AitG)! ztk~mQtKj(;{74>0Xde~>4)O?>j?nbXqGW{RNI=*xSg^1f&@&N;Ws*EmHKGkJ4XjNM z7uM;1_FiITp;iOdF^ZXZ3l=h~Re7bEQ;1BR@Y)ziivppJkznepZ}yIk5mrSk9L@rb z12*vIP7Ehagp$9P^*pKYVL1)oN*URhQ1OquL>+w@g4-fOLtud0>c^YjYLl0%vrg(197Wr@qrqf5~9A{YzV8eGkOoh(WAMWv~;c&4gq-BtUr}A6TR1MQ0SLu>J zRRVrBq%Er~@jnEPiNsta=SR>jKi$rZ30nR+6ex+Q1YuGQDJguUWTkgN(EWHVMQJd# z=A<=iFI!vog)drK!QY`ZYOOw0m5aisi8fKA`bfDr&87TNz5reGIlPU1+Io$owk6e3 zE$|^!&x1lUKa0rscl@?9v5tQSsAtjDz;`+A!1(1#N7>ud9?jI14*S?$vN2{#*t!|} zIpV8h$1evlqx_+vZF9U&E$b8fZ8MIqee^Lave2y0A4q^-1OBZm!(_9={CvFFoY><+&$`kvA@&BZUF znv(%OzAU65^WYeP@qulqaW`nl5Yy9r$_)Q`xWv!H+ zu6OxE$;9}w0Qgz+vzA@{*Z!AHpjdXQ@hhKh{tP=i+x8|M|9*Vk*YPWOMs~(;rESKw z0^?|A^R1gUetUy9%yV0&mjB{@OMhbb(%-f)TeM)GU)Vnq+oaIgzh~!ciKQ!yC$D{N)*aY@#A#_-=IFu(n%<_?A^GtW|{VF@30naZH*e1_5jQhsaWpqWY9p z6ZlrPj>cOk&xapI`e!LGL;Nb2M5f5NonCv{WvxNi3Ik6HdL4F-!ETPb^K(E+I^#(ya<3;IuzI%Tm`PlgOS0k?_rwnaR|M%sqn0jdm|0_H>NB7D z47+Vk>5;>ktu9%1J)fdUFjP5;}&_trI! zL`XX>HaNRVv4&)aPxr0%;~Oz^h9f>(Y#<87{2I}7>G~X5S9onR^HjQ|7|ozpx(Ik(9Bs-1Pv|Ht>x` z_|J|nLYIJ8jPf}LI+ojoneFoL@x}6JDFfNoMstAQR;AN--=tG^7_#EXs--u`=JCfr zo}WL>RUWpzBznaa>n^z9LLQ$vFFuA6Xz?w5Mu|0!-|Le}A+%xN@ppw8&B0mT!aw*S zGvXhzd6@^l&z~Rp_`PBs0$#9g-8$$daB5F3wJyMahsRgxW4xo7!5jo~-to1=_{K5& z$00uJGVkJB`P0Z-AsybkfbA6HH-y_>t%yOi^!e94ertTy{E)p}i%uUP0!a~@Sw`cl zkMFj|FB3j{f6~c4u!EHLM4!bK?!vfbgS*ij9c*1Vwfy)4syrU&-aneZ?6rCIZV=Oq zHSS9^v2?OEUyxz5C0O{onI-&Ncjm?)%g-Oo{&&~-B^Cr>yZbL9+UMuC_rCxpwenh_ z-f70KX)iCk%(08lH(MK0LCok*k_5; zvdk+J18(5YoIwFi3jpJ$k7@-n?Hlje_%Um3$MM~gf7rA)v4;BClWL`bYd}?tk^VMH zVJzKZ8^I{_Z-lS29^WeY5AoUPPHH;+tIb^ay*_^07dJzEW7P63v{W6D1`CbSpbGo( z)gS+e&>I+wdC!0UFB!i?RRw=A$Ilb`6Hrnin> zv%Q)jL-baRad!}0sqHMlqeMtkX0W@j`D{xN8>>sf_W0f0(qVV(`|`r)H$h{eQ(!Zp zU;;xA5oDS%qHp_az3x&zda<>I8B(L-u{8{8!qP0rX~L0*4XeF2ag-|p7IqVsH3UTu zW2jEcs#n~?Pl(vq+)8V!eP7T|Y?Nz}1v94~mI)`>v^e->mafi+U*$c~#>R$oLXY?p z$7r*FAJnXFgpGlLyCqp+W3)5{tQam6`Gc$h4Evl(7CteJLV2;hD^!i3Up9iT@p)cpz-gp+*1e=9kJ6vq z)1P--wJ=!~QSr+Bh`IGW_W+r(6pPb<+gstMU*fAz?XbOZr$5sk!w&|=bwXS8($dz+ zKrA{p>W|}4vMd~B*Wnl2;?FHZ1uN6;I#JI*3SizwA)9q-+6d-qus7~xk~fxCbCqyAlf4)JrylH5vK zHn-4hL`VFMh1I@f1xYQr@WKn$U2z2~#?X5EaQy1y*REYlH8!*Jr<&V@r2w&O_-@-O z`IE>7ah~{Ui9b;a!IelC@ZbwP03YJVC=AB)v*K&_La^{z4@?!OuBr{nc;O&_s)vnL z*b!?vvjijV@8|xvC`F~o{O|F%p^p^o_>bTCRruuMocTlTaQwzNS;Xke{4+bgHGyoo zW7Bo7`(HNa&d`O^NBsMrt=Jm3%x|}?`%Zn0gg;teTJ$82mTA5_%r^@H4?ng;5m|)peKK5hDJW155 zwyL_UZWS4{OOh}p3{qkVScrlcgd{+sNp5oQ_kI0)-*>Hb*4byc-^~q}kiGBy_St(4 z?|Rq!|Nqxod!K#Ik%Y3Q@6K2I@~Xs@mCyd`Pu%r|yZABz-gkNUOCEc|V@?3+(9H)w z^RND7m9H^lrr6v(^9R5A)bBovM*FStkN=dos{9+iSb-=Nnl~-EYk6ZwLC!+Bf>7$D-Y~%D*M# zV4y$M)w+5wTYqE6FF!r&g(RIV7yM{K=N~-lX9tRAZ(Ce`#1@5nYsD^ z!RV^>s+`B6Kn#E7kzU(*bZTTo*f!4m)d z`ni;Jqy6;@)xe)LC$OpOJFyKricaHCn5BfHPW@Lm3+v!>I;#%_xqb;edoj;WU1-T1 zc#~ehD5KvV%<+p3Hb^Y4vqkz)4CTBZt{0rqj7Db|8{mqI7*1QW@Son&fGPm*Ay@D1kJ^pQ1see)#J^slxXH(a05FiEc>=T_+|T zOmz_0LdlF(-1_LCgj+V|Rj=$Ld3Xb4q&wNmj# zCDwHLsG)4zgi^i#_$sTctNbpS>rzPC7XOSf?QdF~U4F$Cyn76uE(=vqg1z#pR~9kd z3U$RtOHa=+)kDNy8K%qaj%q2+Z8ufc0fi*9&az{lnnnIDugn>4%g_Zsh-peF@(be^ zq9hf1N^nz?>J`87XOs~cZ{i@liKl5fGI2%~vuKkfl~)^Oi$3mK5l{_EwAEL<;tCb) z^pD}1{vknW{?SS_%n9fq_v25-hq5ko`O3mRX=8;GZDu5u7%_QznO=)~;oswDe51I& znbieSwdYSt`WyV-|Me{}{3eTuk>?A^sOA(&dzcTIYn?lma$W?#Y z&YP39tH3^X7cAzKedpi{@aX&p%cSX0yt&}&#WCq{e}PlKmbWZK~#OIi1~QnLeGWzKEP&fMS^dqO@Vv9sL!z2U$%Cfx@+K{ z9ukY^z)tn_6bPOztlK|!>%qUi>+sD7Z@=NoEB9<|u5YdJtpYn%kAK`r>B=Ji8ThA= zUBSF^tVR`jYH`ZG%C`TDg$kAR+r5~7gRiE?`1QyTVI|1DT|eJnu2r=@L}2DSVAc zYAx}r#0ag@V9GP=CUTdY?PVWbSx5WD;-WukxTnTcD8 zK1#J4=@)Eb-w@zfDsf`}p_u&Gm)2!)2U{#k-zUUGCVbQ#K!UI9hd>)ufWZeAmI@WO zX#a69wIi|lr_l<@2oZc5#6O>Y>7|#y{Bk~gL9J{B#5dlhh$PQgoQBBn^ig-@;VR>z zy!~gjs5vEUEx%G$FBMUZCAqV2bnw#$%A|Lc7nk6t0%@6!$Oyif z2du|erq3m_sg*L;+0e=le_o>x-r!W9lWpI<{EEwYqJefbuG#!5Z%q0j`&!#%fg_G- zk)t~P=qn&gH8EFv`YX&rGWe?QY(NTr9*m@EMwzMu)q&xgdfm9jH>zn=v@uf7iE1_N z4EXo>z5nxLsNCwE2bdCaa>oecg%-BLvf*#aCuDaMMy_#y{=gloHNz|5QU6KlR2|3k`Rw z>f7+eUzjaQ3Vs{kBkQiHy>8`e4u_)O$Aw!*^?X&9P5jl_zW)iJPPAs zh?BYcD+GfNl)h{V4&*_>EV5-RECE1dV-U zJq**$zw&&N?fzuyL-801pO#OMlg;+!jYxZXQYqv?PSG@pI*q?dY4Ix%TttS?P)csW zw>uU55*~SQ$Z7gus3sYqrK*xE|K^_q{_;^V(+tt`FC-V>3pU$4e0_Z1_e@o)!|(YA z-!lQtYYTsdZnE@GRXsa#D0c9ZDor0fiRVI%fAC$#HR+jVf`YGJR)C&=^zZdi!_~fO zr0O4i62Br?@fi=)fmAcW9$zK-uQ#Kjno2kP-hX=j>T-HwXm%QP#`)LQ$owvaWCRI? zQQG*g&>(b**p{!x7sWgjQ$^DYMN+OGpKZvt$T50{->;u3nWhaQF7gODok__bz(4Bw zH=zwtetK4efZagd0FPtEK{`i^K9pzMAC=&KiGVhyT^%W;RJ0)En6uz>nyumIjnh&| zif00If~w^}y=8y{mNfz|1hASBL8ppUE9rxlVgy9jXD|d`tNAeau@v}_tP-ZAO0y~5 z10G-z1mTiGV_;G%H&{6?l)0^_n#zTudRBJ7(i(vx`2wMt@SIPpqKSv?KtC^!~$O&h1i{C~P3< zs-+Zl_z@~jz$iH897xwG=^UGjeO;uw@-oH*KT<(ed^-HJ%U&6{0{bMnv4(%;E3drt z>6ex-2(Y|rt?Hi->j@S%)~@LvPu!jFHN4fwB2CdJo~QtB z`1Yy;e7DXOS8!8O>B=ldEY)n^)-3)*0WhZgN7WJphM%6l$WnPjmx5|UGmc@0;~R;X zbZY)`gR}nATt$5{KA1X8V-9hF9ozg1;K2V)I}J|7SDK!~U;d~OfIU=Od{I8UrB59{ zA-@Qp^apzUPW-2i4+N=IDWjB3S^ejT_H{6L{zpJ&MQbSaRMQS&&)9L=PAjb4p)cRP zS?9;5f3t7=M^>RehTxYD%7iYfaN}fk=h~^4o_Xr`Jhb88|L5P{|0nu)`uLC@OCJeV z^{@Jj@%sRXUp|3odub;lIyT~^8Tsq5?PseU9Id?E9ryOglYO!XB~JSkj(dDsQmz|x zPV*lDq;Oxpr6|!GDScwnr)z?)4w<8m2+6vnA)QT*>w_Lj<3LU9kL&u`dEHQ9MpO6= zt>FhS`<{RBXUpqypz0Hv;#B#kyy~`^eFGt68GHT|nea&$i#3%T{33_ZOrTGY^5iIe z`k*C}T}DC`q&dH&S$5wiER*q>Uo~P!@I?^tgVh%ZqkjWyF zcktxDInr&c5|heKoz%N9EC_us$*HPm{OkkHUifM{Lz<;4#VD_@fXdG@ zQ!5C?F2-;C-GX*mo#S$BS6+D~-&YzN0A=k<&j53S^h8%vbyFMM>P8D#(amhOzOKn> zPJGn8DC(oQh(y7rwT@2c1}2-qr^~7y8KVhh+?;esn*z--jV5@MA~UIqj@V7%D(yVDT~sdabs26aE!f@I@6uP?k1zJNk>JHYM7B+A#+K zr&^7|;$SFGmi=d;eeH;nTj<8mzO^ERdk(Jc+7th=I0YZlQ}`96A*7>Uc+dZR@JpYi zffK*wi4WQLJ;}}JO!b`7uZs>G;CEPFdF7Qqd-YZBB{?#ao;#JCQ})%!2_gBRqoOFo z`0gLMlb()S+0I7fz1yt~4K54=e68}te^f5|3ywCW^t64YaFj3pW7?@CgQg{hbH2hw z=GH~Hx>fnZMM@<}Ey#Zas0Phn2ihp-K#fyVL$#Dfi!K4O<4Y$_V8?$qSj#AjPnlhQ zxxVBTWYOb~WS^?l8o%yHVFFAWAKWsa6aVSwui#KuC3GSQsG9`F4U+TqW2C56Z^0J@ z(e;ND+^_iOTzoO3ZX_-+mLkdCr%{)h@TU&x8o|U7m^t&WdgpRIz7kuvDt=n-^#M`E zrsv-_K8uVub^d-3zu|wc<5vTC4Q;xW-p$5u(TA?fM3`2F(%ID)JC&#u$<$b&RpuWa41xpn8}+Frd2)o?^XT#EnWr2GJz zpBPzP+p}|R@9NIIyLX(vYxfzuPu_p}%JG?T#GiTw4S(74{yG0z=;CZee_6}&+l{S1 z^ylbL))>DQQ0CU^u&%Fq{~6==-p8*av3Z#u(dVR#U;Ofj`uFjxko1&MD=Ivb$X?@{ zvNh_A@jK4HauWPuW`1c7Iq;&4pLmYvm&yEo?Ca;a=+|fdbz5}7SHPMHj$!_-@hgb* zA=9E1^u&sSq&o(o{>Ux~WaU zYtzL{-Haoe%4M}yyBz%3x7VL~g5o7u{l4OFZ1TG~+LGzlDSgDj+@epzrwZV~CyOm0 z!cvcFV+K|+Z_45DNcJrtHf*mC7y@TA9>@IJC27}P`yLXUrB`yXFXiMvoaG`@Qcy-7 zCBk}3IC=Kb;Cbo zV&CxA^`Y?8SnyqU3g0P(f9tRNVe71|zQUh12rCL7{OivizWXi)n4Np~?m6w@hIhTt z+Ss`B^MAT=@Nd_TKjp+nJ{C#`Z~6Q^H{Z0ib$D&}-o4-c1er;jds6MYx^eh#KKsWT zci+9XzT>2CeZtzF;|?F*xa%+f^w1q&<$mbiM|{WH347HMDs}#I%MI`T^SAxX-Cz9N z`UxjJ{-=KB#`Qn^aGpA+&icuxUGSFw8hpCq-V~SMxR2kf_!ZvO zt?RD6eu;hYe_`sPNL$mt_aE6f+43HZ*eMT(UjOJN2p68UZ@BjUqb4a_t8nCBhVu2c z^4J#oCn8fZ{?qt0{^sTgqLjv#pe^(tMo;QWOq1cspi6l&sE*HKZhsIg2X%Y{H0iay?gg=QTDZT$t<>El|GLAl!){}$89Rm%sQa+#)x+PxE&+_Y|j91i9Mb{~7c+XivGM*ah=9e*ko6d9fLAIvXj9kZ#U+;u4FNDV@r~JKRLza z>{*(_qpcTd!fSNDb)Ty}<5OL44jP|znT}jkJ2)609H$nysKJysHp_LpYI$8J2wxY3 zKOk~4ooFH9Wv0WvA+pBPGb0oVzRJ5j-@EmAQFM?#0&}<(sZuXrRVF)RM2fB5@anD=CaFIcreef@C?|MaFXGeufpSs4yAq0RVY@}7{J zFO`<-Pdy&R=ZjC%uO?Ma@8IfMUeA%}NU6T$=D_@8-{`|$K%G8L zSEYtuvQ=A6hCl7azABi1u}-zGaVV|UowEwh=AX)lKKy*f&-@IfR<^DvO^c1%p_WQ1 zH53W~!3np(n!8$o#q@ljBqOV|#E5(N1blAY8b<7Cv9w;0fM-il@!?;KujC~+$PIr| zNzh6bY_eTV9Wxou$kAcYJNz(-&=fAWMVfoNe)K6wq^zv`_qSYe)7#&|0gt$_KlADT zVM=E2RQi&PyMb}{-<;V zeauQ-{^GU2aq#B9U_jZt^R`=WeD~h(yde6Ui5St$^7yqnv>;cu$8IZr`E6sBe?^$Y zO{4(LRaMD0UsI`8imfO=xjCDph`~OCV$;Vx1wx%wXZYt6_NgLx;nj2ZV&I<qBbB=Y~LkV&6sa3v=+oWkS$Yg5Gbk~-sq45X+aVUGUFl9JMYj(zPpN1lH% zfbNkBpKtEaysYLEVe+3{4?TYMgdHn)9p?A1*4Ni=yY6Q0Uz~9EN!XWa%)fkG<;&OJ zym|1@>EHWs!7vu{Zh@Yqj==1Lwzw)^;e%k`WBeLTK*Z;eSUs_eej$h5Z z!}x_024h$kJ2ugDRMz+{p{_Jp)|r#>nRE5dvk;kM{7SxY{>A?v#Mi%$BBnomY~)zy zU;C5hUb&SwK{{9oNY;gC%>_z)1a3>@MkOa>c~n88<4sG0u&Km4`VXp<4gXp{rA46IZ5Li5G}Tkum@ z6sGx8kOFL!Yx)E>Ra9YGADydnWk1lL^j@DHANCm!`t2s=KSEf*zZRX04~z8a@o)d*_wx;UJiE>EfS>BP-=+!U;HHMRdq1-WPDrXpH2TUe$lP>&vE|!n#Qm0_m&U2YK|Dh@7k`lGrsTP z`Us{*76z-Wo8Iv0FaG<_e);`3-~Pc{Zh7Yy{^F`XzV)~M^0p6unYtMtWG{wm?q_+F zqf$RV#{X5A&GWlFf1H0MleRYgYed$xrgvWkj4Gr*B)tkyx{DJ<;sQ68PuQ**L%M#=3oT6%pgE)Ix8M=#P z%8=>oT>cH89uN#ho#-QYCpLX_F7fKt%`d?RL3!rCX#7;Z6*7t%ZTRYn@v~2tQh1Et zetz%u8Tb!bWBl5+rpEud{+|WX10{YB?B91_|C0}#e{SLnJ0E-e@?-~Z59nQG#lOS5 z_?9DC(aaym&XlYPe}^p@6n^(LZ{S-gRgBFQbTLv3`b`Lmm6`!hQ62L%U3Xkk|Mza0 zSve|4PSP@0YHE&LWNN0SICD#-=E`tp4y4@Ma^{LM_aZHKLhf?mHYX|{CoV)3kn!XD z$M3)Ug3If5-}k<`=bYzx&U3&d>bQ8aX}jO!{1Bzwz^oPZm=%u5r406m`(41vaXp=_ z{jSF*eu1~BN3UYk_e{+V3oDy9!~RxKexB(Fb}DI!d%nkE&#d3gVwAgW(jN20p7+QZ zF?B2@`sF5QpQT6o0RQ`HoK zFY=Q+KtnP?njPczfB+fhyVojqongEhS=1P9Lt+)E7h4iWKKIxOWt^_{t)N@(>zcfp z5X(=*OVlH~pT>@bG0g3*>bi|-@l}z;k=u20J0jNfHwq`H4-4;>M}X9!2WKShbW}^w z3cZL{OR{SC@gt|^WBet|WLKlG)5`gswe<+v`tbZQ)M9`xU)66P$GM)3>+k313uo*w zi0y$HJk%Z7!D32O@U~h6LedWA!E7Z0Ef9Oq>f9>RrfT?132oWise{#28t-YkjeyAL*29C;fIeWQ-J!v z?X|4W^}v=h?)D-kKY;2)D~cPNU8o}CRBv&t%VJn~{B(NI9+ofX*iz==*d|^HZ`^pN zJnr&ESV$Yw-*~l$4qrFwHt4vk95LJj7zA&AR$smSR$QEpCC!>ry{HpFRr#yJu&>fg zA0Cm9wXtcU5nubU-j_wHpO09ixMe>HJ78pdelu&c7)ZFwr|7&Exn zT>ZM%U8|>Q#g)!}h9x&Sq%xoYmPF2jYNhw(wZrVoH{`qgT`OvyO#vkt>`IJOytg2R z6m;J<&tyr^~5V}QmVJ=l!P zd&c03d>v$5f;q3ynw9RUYPrXUhxerA4&M4rph>8(EsK--(=#g3H_(){ps!ZR!4s4? z)E2tSrpw?O>fv+EO=?_b3srVo5&Gb2MIgx7Z@1*YBuq~W9MbZp&jjW-4kI@WTT{m> zdGo21hT2nBxWlptxX9KLRE2uDuLUM=?B&ns!5x5yZK27TlRaDgy2B@7IE`R~hea&m zpSQn%N!MO{rTyJm3+stSZZIf=F^t{ubt;^J`Rl$n7ZKj0P{dhd%ZI_4-MqBZ2Bof? z8K?L{ z#2{6kz>f{~ikMGj7+`q8srAd$e=F0x8BUH#h5o!fIX{x1f)UcGctPPc3eaw}@5;Bx zV^dghn(Di#l=Ij!tXneXtzy`Jxk{)Hcg}kJ*Z)ruh}`8PAjspFpqr}E_leHJU2sjZ z*E4G%5u__GPq$#qXdykGw>}VHP&`uRd6iqu$@e_v8jw zK>M2T(|NHXs~I+(zOZEYVGG4<5Nu|)fB(JFviw;gVHl2$)gvik&`B{^-e12Sk%^#&$i9fwb@#ARS2bht z?ARkpB*@+vf_a(y&&O_-1SwnMT`)g2mgJwD<}>QeSx^8{HK=3vCSxGOozuGaS~Ih4 zn!#ETXz*lzX;dl7jxDohQ{MXT2=Ln386Q1Zt7SS$1>e@oN8GZ}I$SK?a~PlBTBl>> zwGY!_zmV%{0S;JqV7F%yfOmx8)@G_r5KV*4$^u{*3h!QY?fk)e%zbPpU<-SkEx4#J z`D&j`9-54ECT-8ko|_0*F%cx%7k=*EvjGyn<}dmU^>?&Qr2+Bz5gXzX+ADS3m97B!=kk^|cNXHQ5@=g=&FRRUT@Mo$INE5dS4DZj7&(imzo-ux4tQ_Iq zx4FTH38we&!2Gdmy$jx9rj#idH6u)f@_1GlYcsWf;eIcK0J#5%FuDbA*Z!}50kkrG zj?L0<_;JZKNU`q|)GR6E@dUv;i&+|qSsyNE?@bLi)U>I-sjl8p3n`m_U^(SWU+~+i zjiYscgTOFDP3?@SHdLS)YOg1{u|pnFy{qXny~rG@_rZ8Lk!|XDcccpxlvL15Xp)C9)x7|=mqD83%T)h2fGm{W|NHxl$eqfM z>36=lK{E~uyx#3+%5?FRNf=zZd6W?S6#UGm2NF~%zN@C6my#-VGWXgkoD5wY8( zq{f4D<3}`|E>6bmkO;uhy0pca@qv#U#)BzwoKBPFrVE%&eqEY4Ab$pSd>9_MgS;9f z@i1TTxj_YgI8G@EI3iKDGPilwpIX-FE05Q=;x^7Vn@AWBrOp~1SnC)RLMz4&{Y0NX zP?Xn-_-(1pQk`!1Wbk>NIs+7`*Za@vqme}HbjIKt902LQI`Xx~sjY~y&=j!@(WV(tUZ+mh5U+vMSO4_EGbp3lM2&klWtOdcl5t-x9D1N^NyOz%&5HEjuXa z{)kT#eR2;u6zm=i+b;#4jGaUf=9n|2A(SwnT&ap|yT3#;+I$~%$jVq&g0vJ^$!Wcn zIr_>>Pp~3K<-dI9`fNz!q2s`JF~I8Ny}~DBr0IiOx9nKA3K9<>{YV$8Sjqy?whv@C zuWJMVH;%P;@MYJicw^-`-F0tx27wVX?l%rS(6$~7EXO@Vs0)F`5DlBQKzut}T{M2<1 znSRD_vLF1Def_)y>ArQUv}e!5$=dugO79QnUNJ+Xmo>UHtszlXgHgw}D(gb+br~%J zFjy%|FuB`3+i{!|q>{A#w2Toxeb?I2tBKut>Ay;y9wfXl+PrcGBEb0o zi^}xNKCb>_ZGJYl)*}*8H1)Caj8rp`C!Rk%>H^Tw(YWSqk34`QqF%RRF6q6%p$sPZ zi`U^hAF-H3%Ue|4G_BS7Lm_%3cR7yy0rb!RjMS(12$h1J!uqup2g5pZHr?yZ@7faX zQ=PmMQLQe}3A2SvNn;-B#$I=QfX-a{^O^RJ%O8#HCXu{4loNE;ud^j&5O$AlS}I9& zh#9Ct^6RCvMf8Azx2=s3GnSML0chkNMau3%Sz#$kz|efB;gQf%My28utBa2R7nqx3 z41mG5OyX2C0m%561-*@zPbf5fs-g_00V%3OByR=!iN2gR(A-$MPtXKUeOS zJz24l8;)8zrVh!%Ve1af<8^EB*wbL~#wq&BG>pD4++@IbVL<8PNqQKtGmN6{!6OJf zR539gwq(m=C|U~i%U0nv604N;tKtkZN=aXSe9ZMnXQe3_Qu3&Z*VJvl*A;e5kWa4$Feeo5OaS`6fIn1vt9sd`HR9 z##$PIr3ymxjNaM+BaY}@4*)hfp+_Kcxt7w)4Ps6{ucRlW$GUkFthC;plaztL$0b$_ zMiu#7T)yMvdQ@E~nwWEmH{){P-H9nK#OKp_fRQeoL!fI##Uzq-zDVsQ zn;ZNkeL2tXIQe+TW6%`?M1?c|A}=u>eU6GDIWueoyb9CrjH=yD2J6|g^}WCZ-Xj@P z%cc!{rhTr!D(qB`9S>Pa?R0}0PF~KS=?Na$IZZ~{!VZa(63N{kiWimL_)dWxf-H+w z8O49D5c!vklm4n;m{YbyeF5EwSYA2nw_Mo$wI=!>&{o;`z-!Ng2*Y#0lYG>F&eDm2 z;M1bs62}GMYZYmz4@fVMS^E}ZEJjq76296SjL(pE-17~M)$X!5T(^@4?uy<}Z+Q%Z zQk^jC23H;wzuVZbV^4j%!vA6N8sMb+nh3MW+Ug84n6Wbi2oEbKBjYPAllummcJGrU zQ2UWJQNahfp}wY}hu2V839!NVE3ubUuJv=j6(VGlsyIIK?}$1C?D zCPt5l@nAbIdsvCk9Es~FiPVJm(ZH6`O4L%E`=8r?LV)&wTZu*K(3j!tyB2TW!&31M znND9o2xc8Y;-HmH1D5mYI?kz_*=69F_&h8%sdH#kHt|rf8qiBhv78y-{-E6l%T&a| zjqnsqr$`Ab73c0P!`{HYe$2e60oMqA?6f&Ks1Ozu^T95f=TUF}3!ZilO}FvR-`dp~ zuDHil=hBHUqarStNV!s;{!~2djr5<0ZnC%`(JMom7;P5Xp70+mR|lG-kfrc?5R3D~ zh5IaE18o~qH-n2jJpQdb!-5)}=tn+@t4~ZegKI8nZ+>To)igQ27vsscX)xjU=>=@) z)*5Y;aIf8<{P!qMz_ol5*1phv-omR`nkBh*PE6`NtT38gf3kMC-1s@igIyy3#-5m` z&XC^(Hm-c1h%5ZP&vL0Vc+bjMLaY<)mlnHsXVT(|4+~J@eiD64>FD64O>Sozy(Y9P zPPg%lEjyrK5!oT%)2gms(1@^|iZ}emmNzxCudhBx_WP#!(Ki0?)3qH_K-Jaow+FhP z0~N%jZbC1dLfRQ9R+%2^*w$M-;@4htQJN3W{bzIh!XrNaMt;7=n*v$}rR1e~%J+WN--IvH#wRrmPS0%{E$|2MafNNA2|%;D%|QaxH{ zQ}HJ^TIp`$#YswO$<==)IF>5lL~~xNmS^K$30LivvaBtg>s3s0>yyWniWjZsJ4)YA z%h-M0umDD=uwwXYEvMWGcqyILScDR`t@L{O90C2xD%6%-C0{9d7m(D9K7^K=N>T2G zOK_6Q%uK<&P-k6Oa z7l1}`oqNQ%ZRQ2<#R^4ice6c}(M{8Ga2N zXVuKf{l*DwSm8_^i8Mp*z^M?Kw!@Ghr`bj5egK2qP&)v@QH+c=C2#*6mm_H}+|WyC zsL=(}wtV2!yLL_C{~b5uIt(5)EBMXQ-D_$$2VyIG>cmy=G&gmwf4%eLcP!pGfa+20 zoe(qlPzuR5iO$yCGN6-l`Dmwvu`Bz@NpKS;WwQiB+dxzlLP}n_(2v7cnRX&HjBn47 zu*5EHScbkZY&pC^vwYOlX#!}HSw4RhL{U5$TVitPvf8oAMKjk?8(h<*ylx2L9|`qB z|Jz6=J(X50eG{LbXm+!SjB>=-M4e@*u5mN`=vnW?*GlrZnT-22n(|Yl7Acx+1V857 zkGJ7({zd}h<)8VqOEFE(21qkl)t@iHKMK+$|ib_jNhlkSZMkjnhk z85Lr3Z4u@?-`MvHwIz_h)E^i6(F+COePkPLyS{Z2+L4jz9*hleU<{Puh&}Cs@X2x- zs!zhR09KRyCIFiw?&3JJ_G8rbqLyTN(^o>{i0i%4gBXKf6~!f&SzJM0z)|z10r1_5Ns>pMvtj zm&xK>tS?JIXX-;GZ@)5v=cCsoY1U}f zt%}sPnwyYfBIS_8ZFS{1A<(7V@S3F42ws!kIao>|kn=APCzDEHn+^g>T_S*~cDmQ| z^WaQbBP_^>l+@-=%tknTxOfQ6uM2$KDMoLo_G4s!%Jm(^XbLi|j0;Ee(AA)kd&;kL$5r z8k35HCtjB0Q^=u;i90y`Yn&X;7%qMe)wtj? zy7s)puX?k`(M^=reR*zbYuL#hI>5%sXtX=&x51HAdgUv>^96#mNVvVi($kR$v7WWbYHMQs;>bn`N*r-dFYBtdcv4W7NsA`znc|HXH zI_5S~nSUMJVn~21LF<%S* z&&tBwq1Rn_DKG@!)c4=OaxV_Ep?m0=+;8OZ2;M=JwA-_$QYa};|G8d2?7J8Wvdp$j zyn6oz@c5dV6&u7cVn1|&9niAdo8}0^8mRHQxKs4ASbnjGPfjXdK^&d6pKNOm3Cq?~ zStXJ-uD#fO>kwbzHr54ZnlZKZs_N({hxhxo3Wryge^=^!W;&Ce zCp{oi77HTKDT_&xX3UEQqfP33(DU-)(%9k zWAL}Nm3mV+_1-x@ZXV52k_&CrhMZa3+R7o_8u%K9q<%d1J<$yYgEVQGdn?9TuE#~3 z5)ZpKs2>^;nh)CvFo7>>J|!({KQXc8n$?A$cY2|UNUR7Jx9e;UWO+0L#}LB ziN4Oa*%=K;XLQHubzr1_^T@uvSRF!I)lQYPynU=@1S{`lCB45=bz?-vutMhFQdeB! z@&mC9nFlYn`1r;&Oe1aN&y}<`#@+2sIajh#eXp2XR4re^G+n=F{~PEOG5=`iqKQ`d zp9dYT2loocSirSD6Ge1A@P@Wc16F74LD}87)%hp|Qu;oda#Uk$5{6RPmKS2=CORi%5r2mgKy2UDHj@NDX=ImF3d= zSGy`vic7^P!`$MHc~33a@+VY1y9WLy!ko63;hYCkV;ykiM-HI+9jA>?xRZ)XxMXVUh(Uo{RQ}#dqAIInbyi4 ziWx@fe8ie!u2l_%3BXDWDLo{xcVMf3a~-x<=6o!FVVPKyr3@KF4%{^i2-$F7X`v%D`g5%G|;ER>wP3 zCf_^$TonFX!yTla!^%6*wExJ%OH3n@%#Y|;)M=^ewbXZe5&D49G?q;b+pTDORl*9K znTb8OG7vEC;~?I8wk?DD$nVNA=kOFbMl~Kq7*ctg;0J(D3Bc-Y5p^*+ZsnI-IcoZQ z3@KjNH>9J*gUAuP6nP7w3;o(f3(dRt;Gzlj3;D?U2)d@19UROCp<%bR#2+K z=DziqTG;Pl6cb6=68Vt_bvg7}IPhxnyCVL|n_7qZV?9f%&cjk#p4eLA0ut2Y5qS^8 zuLN%FjmTgb*HM44=HAT_Vf%xqJ=$*r@<_`H4w(dY`p@@DLQ?j!lXIB|DDvrZA&1bT zS5kc%gK}j>Z_6^ zKE|ZowH;z^62dU*cF?LL6|_Qa)7#;giw*)TqYXWpY-r-s+tl(cRds3~C z2dxn~*`OAZX_Q8BtskQk?DktcFZPj$ya{REDtH z&yipx;y`40L^E*6*z;2P_Fb91fJa)7-ys^lYd03jW{G}1@RkX`6n{Eq=O#F`wpx=C zf|##!hmauiA$<~H1}dj?QsN@-$uPxT(k=W;rotr?%j4RErnEtR_|U?yiAs8R@hhXf zK2T8Q)*eUY;dPXvWf^Qm#)zp?Kg_5-KrbROzwx+;{;Y-|`nO#h#+_6REqe1qJi`uJ zLSim~sdT~!qVu*2;A3a68oj&^JDI=Ma|B=PZFF><)hQrp0KdsS( zYR4=q=BSt`i-!+Xw)~=W<~$(h9E)nbu9Lwz z2n#z<{fDIA`FoEKTwgSN=@KyxHR?T&5-)GiHfTK6#j+Zrn$Q@;@~F~iv#`rn%kEY> zARN!*J$cuqR{XFsBfV19g#Vkhmoe)5?^0>6kB966=H_X$9TMj30m9|#i@)ymi;_nC zTBWng^fM*6nq*!vQNF#>m`8a6W6`4F4Qe3xQq;GrXRLXIgGlP95eGJ>4PrR{SWH#5p>OGyE70D6b~6Q^t8A9|3pHVuK&0*6(bh?V(a>1K6Lif^*p zsZD$a&RBO7v;TFGbWCBzR1bYw(LKjIaiv-Pb%&Rw3E;p(;Yj&!0<>q62Mdg>qvHWx zGvB0Pgloc$A}AFf+Kp$-;fpOK^BYu4+p9BSo9B*LxWZ?gC5x_al%mG7PHT%dL%0}$ zb(L|ZiHv`oIz@^5GpS+`8r1jNW7a0zqsGP3%YX3s^aGrfSsJ~#dhh=7N8kba)RI?N z0O7+gBNTZ{!`&hh6$o9eE~9b%x_8dkAad`D{`ZCyqUEsKCL62d)Cp%H3(UnkY8k&P z4_gjE(R$G+nhA`o0in)A+II;k;+g}H!g}L?NATi`?8b(lo(c|8N)AjGFdv`6|9&Bb zh~2L5o#q@~d0_FzEnr>B9f||eJm34~ySfpp01?N0m=i`fOhEhX{wq2svg5z#l8nQr zz4F-n3+2(d8(H6^823;XS*0by*1e<})A0#iox=pWg`~azEzdqu)}p*j6 zf{)Ksh)@ocvF#-vpVnkF67#X4j=+JEK5~<3mLPDm9NW)Nwp_aHgCf$i(`w>6Y>I4Yt>X~VN{Jef#(Y)%B(S~#E)b4zm z*~6JXM-6ALEKSHIagD8{HK0xB^do!g~hGA$RTGko07& zmpJnuPTu$;bOpRUFGRh>-fQyJ`@L=Gc(pqXAX$Y zHLZgMr^s{vAp-Q*SO=<)8nYHAGZ8 zhM`{9y(~9OeQn|kiVPD?C+x~Y=}HjV4Eh(bAN3ZO1rNb7ZJcc67=$KgQE2=btzBdb zvQ@OM2-u{kS^XY^Lp!lQF29fuA&53il(KGhr)v5he0O(xvodqiys$n8kb#E~cm#vP z?W}*L;AMI~6r}Xl7#T1-H|eSwdq>`o1>})(;8i= z%(8&$elm>$j}r+|Z|H7pyTOuIo;62obp4t(sA_qA?m7ur*6LVx`TTg2s$AHgrJ2OB zi}J01u%rB$uTmzV;6ZE^T*=Daep>gWQR)p~h0T;nX;_;^D# zk2g>VnpwsBAPZ{U%j4LjQgT^!Z8pi+%hVE@C2gmEm+hui_}TGP`=_$OP6PJZl;?WE2sjK zK(F_JyhfC`B-1it!XJIGQB2fqjl{(LMpEIVL3|b}ElJy}a_d@ufyG_Al*t!so{++j(;Q0=vr_Ue- z^q+uoEl!V98%Kc6ajT%(ewQ%@lGpG1U@BaBqNsPL6L}H%OGH5&1*LUniXVPQI-~vNtH6apQ z9dlN?>Xd=N`Z1ch=QJYx>?5>JT~ZjDx52r?6d%P04g5U+OQdiA%Rs80EGHDCdoEn- z<4m$6H2T4?-uo@X(3{pQ0nYfK+pcFT)eYpj+zdQB8s2)Lc*O*5U-m@0<{2pB<*TjSeRhb+`Ww}(@(-*Cs0<`I8d6zI81gGx=>DQ2|^ zwNhNaL^W_;Qanp97@h>P=l<{rq`bQVoya5B(f(r`nhEie(w0hTO5srmN1eA4#-`N1 znkOsXSCD>db9{To4koaMt~J7Gbcgl~y7@3^Z z;yh`z6k+^4tZUl)>kf1~-gN&`nukTk?K${XgMSjIVZQ^`6F3bofA|@&S0V6#+&R%M z5w>&+<^$dPd&BtR=XtdVdCY1DgP1ksB=Pww^xbSw=igRVvB3;1un%}pmqs0ho%UI{ zXgAlo-smu-`SjDqH`A8&4ujhLVDQ>p(=n+05iCdE`Ccz1e7rLiFN2W0!mZkP>(=n3 z1inEtA_;Ap203~Wmw=xvD!j_LQxl9A3_oZBBF(Q0H7+tv$1|!_U(4f-)k_xWRFdNG zEu>nCNC4ir7SBrY&2zO54DKamnKSECImxtmSTD~H2sXga-fs2Xr>L@-Y5MPwEW_G@UUvRI3aryF!G_cDH%M?vt>)!N}uGt{iXU&=NbFs6yK{9 z#S0SzS*LKj|MvnY`+W+tkRSNHas5R@jXueURU4zZ)NDB3`baLwj-OOxGk$^bA#gA8 z;CYYsq~(|G$w+zy-(tDiZ%%eLPq^f2@>J@5j}~cwusZ$qw@m(}OT|||b$wTv*tbUd zRM=KYwDj!}IXk(vY(D98uxAxa0ZCiVlc>Jndc_$^qkaFm#x13Nv_;azagv{se-zh) zQCy?WYbm82c+!3tt>V)Qb`5rAD*dNbH3hxGmnvuGU&#G)N^f9`bHc8raQ-Ge;~;ML zS4uGc&1c_>Ulo4ysnQ`EO1@PZ!6o50HScSMqrXN4Z;(^73ULyjCt4Bq9 z%Pj^~W}9)TLt(p)3(%ERt-gxlCru3@$C%ebt?tluKkv3w95$G+8R>-&sd zu9-Bh=$$mX#P|McUWimilvVX)#v+za$MMD=c3D=d+&QrX>ASs< zGGHXP-5vWw!|;L|);`*~E__BMXS!rvIXN#2*A-v#C;Rq7zBVWA{P{cK_S;1BM491L43_hv9$ZXUR67pH5-ESXwCwEY2d1H{g$7go#}_H> zF?XB(j#15cAoXsvto-LY;LyXTpJr3-8DdF+pZ4Ge-DPu)#cq=J8!XI07{iqHEBJiv zEsSP(Ojf?-?UI?bKE#hU(#pJ{Rj{>hGC-D`Rp+#~ z2!HAJ96@#XI?9rw$B>?9!n(h{rD0}BUi5@UHD{_N>#IvEO>V=fkRK#z$D4oR3b>W7 zn#W2H{ls@9yw?2Ka1Uwq_B`+cv)Pt&v!qTa8873)%<`z>bWAUd>8}a$q!WTQ7{k5O z*HKJzU3G|3Ty0UQ%G7cJ{az0Lr}X!aNAnK;+JvGjVfLxEK@^&l#L4$bkrx+}**QO$ zc?O+}YjgQ;&ot@vW_%w5u}&fDh@k#u{=JR9H?VcwLv?ul<`3e(0>4Ml8}p>lxj5Of za&Uz2)NW6mP$cd-)3_NMI()XcT_N!I^YBs9GlGJtwEt;t>Q67!)f)-DRexWnj``s; zpMe4Y**uKz%ItcZFn&?@-C)tFbExhj=`cG>`8eybRU(ge2m8~>vgIb3K09b9Ja!@& zQ>43mwWya>k>pi79_Jzep(GU;HcxzDCj9+>t)_q%8URLHi!c_ z-yyXHTT+}q@d*hqoF zAzxOk#G#I{ci(@P8l5|bku7ibE?*XEE|OdIi1w2EP;jrTfyIm+5Wf#0xo!q)SXb}c zJnaJ)uw1dfMt^O3|At(L&7>#Ps)t&!ny5>4G_JB-%*IB2k4G(+w48D$L9PL*VYFqX z5qG(WLqFEC!?NSe95=J4SO~!oeHtnKWG+XsLn5y0XFB#YvQlmYY02jTZEKfsn!0QJij(i7}bL!EBA($dv(mM}zimBlp zl8U4MZy_vnU0E9;Y*R8EkG5lYQdCbYUVfwp&{%ad7PmF_0bjmeg%3^PZF&+p6$Q~Ls`RZ-#^qeL;SJCEReMrj$UP9} zGxFSNs^WAzeHrE|3A%BLeW)Mibeuf+;t` znsk=v@SE#aZe&3IQ|N@8mC1~!bcv$W(DB_@BDaC6;1s1kq*jEz9^9EJ=~+h9a#voX z=2@uaFNV`=q30NZpv_4k>Sx9SK)?l!Wfx4dDrE_2!pQV>Vl$|DMRwG^8%DP(@Ww19 zZMCOh*5IRd3FOOl`5rH) z(Hj>zFJqmy7rwnJndEV!BVMeoUT z0llXx(@&@8>_Kr<*GP{lC8<&+xcAhLfX#7b$0arMkk?iqEB*IeK7u zkRYRiHOG;QYMB z_?t2yq%jugmg#b>^_QBxyL0d)h$4SAYt;YxkH-n$?5z@T(I7O=yRsoGc+}`z}_LkP$xnojM9Xmi5>Z2-! z2s|6PY`!)X?Lxh+*-H{$$1tA_$!uL7Bx;F^YatA%F-i<)uOQQUC*c?y;zQHf^8MF@1L^BRwKtXT%p;=#SJMpZPhlNSzV= zWjPfEdj`uA`}URG_hfpJ^l<|sKaB`UPZeF_WMlg!_RTSze*dNHhSlZM<<2J^X{LD0 zf5=aVWim@&I2qsf^w+h-TZxk;y;J=`hc9A*$L`IHu|u7;ByV}gisoEfpBhKT*E9}l zUs`z*u-$|5%*>E`x&heSxE)4ghpCN#gV|nTf)=XHe~$~RbpywtU#%JEI)YQfjbl%V zL4y{YB-$# zeJ@IGM)$@(x~U(G0P+k9Y}8yjmiq0$BQTU5(Y_v=wgNn?Hk{s;VcUl1>|!J!gY78$ zc3*qDmZP(PM7s2;ut=w4qN~CEXsnEL4Ss+UcbUZf;}hT;u@rSR=F2SAJ^U2oRHJW> zrTEE|6hF7+jh54+KhosIE*Yr#YPXvOObBx=9ky~pxzAkDVGqLU5}5XQD%zaAtd)2F z{E1e=XW>QZ2?j%}oTtYSUE&^1g77P%SV5`{Z!+iDOxi%la;qS)`odjUXksKk;JYgX zV+;@d+&@yhI`-KzZE-qc>E@vmP5#)TpEPwM0d@b9#4uBQbMDp=G>vvz>T#3HA0Nn-3zR&X`z4R>8q5@FAoL{Ok#uoniYUI+}v)FDUQLDqaVLGJfWBpEw^( z5Ud2aAvEfxNU~DhijjYLFL?HX4Y4Sy_jIGW_Bj)Ev+7~_cKW^!z~3|cFV#43e5!5` zbver?;z_tZZ}}0v!Ij>=s|$!a)39~`p&P!H0|2-S(#>(K9Ti|p0^%W){l7lMKSHDk z0$=Wpy!$SR{0;P_x7doW{73_f`dw{OV)W**e0_SEBYSq3n#6GgdieF{oX1k~TrN!h zKI#eoNIWY@y`i1V#{unmcuMEh!OkEmnR67ZhQI!AV#ZyAAYYf8$Zk+Mi!6q?)7#%N zS}p=0BaeMFAF$5l!{$h4TcP1AYIAjH)HH0ri%TP$oO23IN_KmRO^Q%7<3bs)VXDz5 z_#KaROb12i_C|meLB8Fr#^n=ij-n~{$D|eiT!Vlpnj>I~@bBy5PB|(vB0mrV^1+(i z!adY{{Bk@&-ur8G8Gdz;*YVKSDGo(Jy`{HrP$jAo&;CeJDjv~)^r;s}|5g>5%cR!B zS+lx~mBbwc{h2HG`prpD*Y>Cn%6Z3J{9LVNQB>%ES7OgLGS}ELTXg9-ls-fe>a5oh z#;85%f3t>3(tMJjFgHltV;)zdo!T71*JY-4s*Te$8FKu9cm0%bGBqqg{u$_M>tp;q+t_iRDAkx-kQxUweR?|MYd7prA` zTtr0HktRpOdGYsik|Ddlwf_dOB_!Vj(#F=ocxI~fsOx8owfFJNjg4o*HutzC>=`^< z&P}y%|6YPoN`Tobh`6j!R(B_M3A4Bd= zWrR2NZ%M0b$CkG@I(!)9;XHcA`V{W+PVbfSEIoMA5u8x#?F@eQ_by++hoH~I`5edK zuDC+MbfM0)OM1fl261xQdyZiL|iaw|4LD)~k#&@cXd5|A$`Z=4L|sMfAQL_2iIl`bx~JVv0!nWx)f`x*3h zvG<%b!WKAU+!_%P<=-EBVepmi$s*BE?yGEOgPB2bjzw(*Q4mZthHTN|x=HRg7AmsZ zfZ`k#lE^iXDjsXozGh<6el*n1XnsMevUuG#PplA>(h>we8%wGJ6CO=>YN}<~7Fn8% zp0zKaEh-}>)4iRk^!X}`|72AicmCmeoOO_obdDt6ZCGXxxkhyUnSeSmTe)=dBHrEU zsHhTVFHh+M>fAV9nEurf>|YptQ%#lE-j*vcM5xB|Hq0A51IBm3XQ+j}eoDZl2U`qH&lynoa8`%ekI#+odKC`rZ^qj=9ym~oH z9{M>Jpit@cHvG$PwloZ;H%=WnolzCmJxq#%pKb1MW#2%(#>dr zsi*R=O#%r(v#d+u6cQ?X&%rRX=}t%W0*_YnKIq)WMsX!iuJLh_4$W!?uFm3`kI|RV zu#2yyJbv$7%|*SyxKqdZHmANj@VaLeX*(@{P_8p$Hb4)z!HC3Khk3>KFLBj(|2Ph4 zHIFpR7l}syv%xanUm6ZOR!00t7k~@GEbjhOl6NWp@8t`7zR~?A5f2%Q4K5C$IMk~z zYuD$Rn46Cee4@iUk_C+UX?H!szI^y*Fj*9&_;MZN?s)Vt`*ZEhGf8V;E-nROh_ULSZFNKZ%%ZUbAe47<3MTU)nKaTgp3O|G~jc*H9;gWABt1#ij zD$G3%ZO%0?ZM)s@5-8RFg75!63!jun8FqKOEK>*WmQ)?~gJW>ilger2YUT7VDLUs3 z#`-X7@7G$8qi7XW#mG>_xW2wKMdI?7Rf&|YgR1tDsJG1C+c`n%a7wkPY(o|qH_I9E zi5z?dmi41kKpL&VxA}4%9J#`u6Qes0Q`Ei(U8Ge(MtUxvJ5}wl24*LrX>4PEY!w-b zkJ_mv^5xc-82@sJT|eZ)o(lskq9ZugF*44SGrg@v?;1SlhQhJ&$!MWInA-J0)D1^^ z{d(1d*6U#x%xgpEg9}vm0Y(CMx>8q?>>5xAQ;&>Edur*XF9E9uWsy3O)zWSF)A?`1EkQ{ze_wpWdG(q%_6ASh1=!(p z(IKKy$waUkLyCN^G z?l+UP*awB%%ijbIbHy=oVaor+=6{ioE5Q+1bo-Hs0n>t}QTcfvhTW%qOBvF>l{2u_ z-eZFXV$nKNy}ReAmJ0RHFwFOZEMn|Ia5VXNNkV&~msRv%b}p(mV#lGDuKIT1${Hs4 zd!?(^+R@E#k^M(d=KhH1UPBLipZ_8BFULFD-}q;*4fPuCuAV0LzP$&(9`!%?uYTRz zguS+i9h1vh^)H)sg3akc*R}H9?>hcivsY&zae32M z!K1LdFa;|`Mq;G;%I(BB2^Q6KCDcj@+FdKw|Bdl|Hv)(~3mx(mYB9+-gtEEt-n!(_ z26|E!J~8+C2dz}zP8P{c_MuL**Agb)q=u463nAJ3^1SD~|7JIHc6Vm(%)RrypPQKb znz7brAVmILTYu(xUUDe=yQ_49*x8}p%-=5qC`wr(glsPcRDRgcgbzFrP}T`^bkfyG zq}wC*3g?hA@&pZ(H-qGo z@^Jgp-Dvpkhy=&I0%V)MPSxP6+hSjeSaw_eZw!Sj{W`U)WP&$s?-d{e?##8&e6XQP z!A)fD{u6vbj@}rJEL4PG5@B|}|8R9tzao^4#r%R3!!)cIdQo&@H4edS$laTDSS=bw zs+ZDvt&>YZsZ~Xn!wOw1B|engyd>%5m2c9^ZvK5dsQ^CJ{iwi#Xhi$*Yl*b4+Nq7v zRo9pTVo49=7r62!)H@?VyN|aQ(vzfNbGW4# zzH1m(=7kO_2=0eJh!XlW+%2KHCM!rYv>4rsG2MERtUl30EF({5>M? zkgMQ$$zwel4Xz0xfUm;ID4oCfv{@}?X#PHuV}hpPEjy3 zh`WP*b>%KFr$JA%M@3)I+cAl5B6CQ0j4f_{eH8s%iF6bHPem#T*c5vZah=vas8C13 z$I*9Pz*4T~fj<#bIY7`lEBYXAE6&5Ai{;n*qqm-Yk`7OHe@iYf z{^MDqRi7MQ<*sb!S6?z)$b^cMr5a6g19C=DuHm0-lLPpp|LwkaEDqFtsX#K!a^`sU ziceCe!}e~(lx@)attlNsd0%?b#J66Ev_1{h8-|w;DEo!isme5|e*;2y1>GF~ObctL zY#T@MX5O+JCup%REiIix{NB?@lz_iXdSoM)G@V5M8&S_--$9%dr4(V|XQAg1n@_}& zez8H%YaqSWUh&;pAiI(?BMaWJRs(pJOTtsnA(1t2`o=5AaTR8GI}OonZGnT`msq{a z8yZYEQ#R)Z@n~%_k5Zq?;@vqvkz9H%QB1sK77-6tqV+poWucU&@0%iM&DI4+!Y|iw zdNMXx%{H~1Q2P9WfZ;SVM~6M|T^#p~uVMQlvI!57ySZ8P`8@0#951tykDC)nyS2)5 zV%hN2ibhq#_8d9VzrUNc=d<+W499&?!BpQ?Fpvsqdi5Bu%@&|nVe$QxhhkPsEeX; zYcouSg*Wd1TtG9LvLAq{%l}NqTnIK^rp+ys6p~3qw(N+g=d!GIWIeP6l9Ba-uHK(t zYmKUoeMw6f*04yCrAcr58|uYsck_4I-X5jx?f3q9jC+e)DzAD(;Kod@ARUj6mGs8> z>35M5W{I8zqu1XdVrxf~+_<){WpQ32Al9TGQ~B)!h(l)n^EEI30jqf>c$xh!_31dB zyh@*KRaGT24HEebK}QtIxVE2WXUXsedN)N~fUsPEJ94nEYTK(~HZ;+)qC!$Nk@Cuy zG*gB6K2@1~+sOii-2vgcUWbTW&nh=98<~e~0U2(7_mdU#-H-GJUz{^tBh8)t^xyh( za~N)^@0UD$$8PO?p=PqK{c7&_)@_29*hH8`f$$;rBY(ayc|#UMI4VvjTXV6MGZqjV z#kVM*C0+eh_q*YjLF+{{EW`_3yB|uP9ld_{*i7XSugb(LP9}M=kkaq-ma9g#2L(5r ztdi$wUbKgoL{tEp#*95*&-8SZ%I;`N7G#Z6>{nUWBSt%P2qm{9t5^gBZt#%}dayN3 zjLXu|>L&gJ+~TneY^wX!NB?~0#adufBLoLFPqEG@8Yl~L&dR!MnEZO`#@`K_M*(>? zyNPQzhCEZSoq!5BYCZcyTvT$_tAc~6tMXDfSraXakv-tdWU#}bJ=6{+p4*?xeQ*0-31DVK23hoF8Tr|n?{D~c^Dt1&! zPbr=0*(PG)s4x1q-CAY*XbWKcoF(Jpk!lg!<+XvK^oz z*=j0$hmMw9nmdYeavZy}p6Hrv==OLxmXdp9euSVUlj69{cf4;Rlqfk+12Gm$ai=o;gzZ3me z2!qZ8!Rc;xLEkM_qtp%OF#H=wThJKGfE9TTgo#Hdp(4MFFs?}XpyLfn>l86rjhZkx zDyjaWTv9wAB-+t4ANl)NvFa?}nKt-w^VB!UCO?2t(gg^I;1eNFg2W z4sEcvF(17N{rV~RPa7=#3u#pnscO_BEk$ZUv;bY&fRQc%v&G7LeBte2;LGd4vRd zpPnLe!KqTDfA1(5(xIeX!%lLR(7@TzY%j1~AM@W7n@#wUUpZ)WsaM#%ieKSNJdp zzeu(msyA+0eNNZzd;duCuA%w0qR0Pm;XRnMR&6!Ju_R%cZ&YZLEl8!P?)~fa=gvMU z_&&X@Y|z)%g7Q*$z9W$xTk>Sc;arPRqmzMiqGXFV`IW-Zw4 zHhu~vMPxme%uV6==x{>L6lRNi~ZgJs7iy(am|^@_cTXK$@jC%LV2%;>^wzxdNNsx3@w`U=pVdo ze)8>6aNnPzmD3kOwdUs&@!pZlywZTz0Qv#k>3(hY*(;N7Y@ z`2?Pv^Jf}0>A@iEg-Ep$N%Lwj!QQYI9ogocH(a@bzZ@+ei%?}f!0_pGLzJ>|XLw7S z@zRY~Cf7>8TL{liKe$Qg&kJ`VMi^btrRcWPI4pGBFZmgsc(Husjx*9XuFQZ1aK6z~ zVfH{94J^Wc`ic(VhzKXZR;aL>(VR*zwu**F`lhpKs>VZ3Z@gT%hQGk@S`*6$a%lxO z$F9gbF5LHPxaHv4r-%@kkY!%uYgCGDb)ztF3uy>9hilko&J8Dk2RwreV!K`LlMB;+ z7O1XqFIOcQ2FIav`?h69kgn1n@3faBN%pzhDpSk$yk5_nb&f-oj!W-bPsA$hE_&vG{Nj83svPUxC5K17u@N&EduVs`Sf5c0=YN$tR|6ApT z&%7gCB#odCeTNM7Oc$H#=FYpnp_dEvyv^Q}`Sz}U{gV4ikjsZ-$;ZvW-%;R@K>eAB zCu8A0-qYB_mWfhIXk>-OV!1+TRG|&Ykurt1^v}*G{3LOg3&BzY`?s(qe$khCLdvKM zi7;yJ3MV*I4R&_&&h!vHSlck8C)C}f3w$}RM@SEU1?*jFaXkL(y#S=Q?56GMuFY~l z(2q}+baRHs<-1b7sL|}7$o4?gwUX2?p4>s(9(sjaho}DCHU;2_dHsyYebi}uf^hTe zTv))XOBIfysYMHx;YL9Pdb@1l96=Lx21!((yz+X6E~~-0YK{$pF6org-FrQ#rjH2d7-$3{+IAt zN2^O@yo@H6Hy&I=mf!Sl3}|YBmM=zcA;THTdQ?tZGB-xW*8)sdXGF!4D`;+^;y#OTm{d5D{-K$ND zE4XnFd*w~gtIm>vOW?mv-G~^nx~U;jEZ~bSnvb?vA{{AD+ht;uMkEve^naUL#Nrq+ zKFkic;ddYQAvW~hxyAZr7lIwv>;d&g%6~=Cut0!Y4mfd(#XtBI&`y3&OX7Y7StajFfpWMy+^I`;w8J(v*Yu)1U;~UsPz5!A z{BS50^fQ2*NWMHOL<*;hu9Bl7vhOexnos$)R$leoIl~Q;v9FQfLEaC1YtOr91}Xu` z1bXj!{N(Y^j$Z4pc+v;bC9-mEd-9lE*C(KljQUyo>2)GxFdq*V-XW`>G%`UQFsN`Du5`dd^?*B?(W3>XjLX-o8U~w^KmC*u%w0-I;qTk#_m4v8A{53I!bn zWw!c<`%~3{BfoL+5?hvD)$OOCt(A^z8cMV64*SiUHbWA>2qco%w}vy2z-gEQ$o^D{ z04eVY_FSol-vU}o_39Aw_WXturDTQB<6T0}M>w)Y2o~=q|az=!OqVb8|e-k&zcWn@c0rwf>W2cD@D2XdH7*|ajzuYBu&z3 zZALb2CBO9Q zq^JFHhi?eck~u%S;S!IyZk9{KQl=M6gs+oQLEUf%Y)Q{k%AD)=l}MWIFHO~nGGFzu zCB38?U?9Vsz=5vr!6*lf|bU(+j8EvSH(4wlI$8yyWxDr;m&wlk}RRg<&w+2 zcH%L-p(Ikdz}PqJEqGmxsVA79xQDvSel9Jhi4%Fd9J0*dvIxvpJ3|I&K%xbx~9 ze1V$Fy=GFw{iBn<+b#hl9zRHwO1*Z^t@O9MTldGoMPCcd@&k z+puC?4s2;SmAO-uqq(?~GlIpw{Y?{6AN;0z{97;6&|PRdtdaJ=7w2;AMFs>-@Q}2| z9C+YXypiMc@KNv)UH{b!hYtdcqZm}USjGxKR}_OfhHfR2BkINBMScVbFmr&S7+3M?j^2&A;j%f8vzItvx zzlDh<>PJ>cjGo1r{U;4xD_H_&DS;Oj0O%mvx}7Awf(Fsw4KD@>vJ|O2p|whPFT*DZ zr;?Z5Bi-{M$4)r59Ch#6ox(?ffC=mIVPYn*cs{zx&bPhi6);q&~tP=_7RwGDM73NTp-P=U1lj4+?Dw!HhSG~o6(_62e)KLoI878f!?0f z`-FU(k2ez#x=$U>6Gs%_II)GEZ$Y<#*z2Mlzk_cgY#_arZ5P%Pw)Zi2{406EJ!KaCXrhkv#64h9ErZ3NDnoFW?^Ft%%*#!-nzQUNl zLPQpAJS+!RXc_t~@*| zB4;^B>$yd_G0?pCFU2M66Hk!!bSs*oBH!^_bXKs-x!oHDl>@;mWf$?uMAkPl<(QO-IBTE4jZbyHzxlDIE9WyBq%4$fdN$xi@X;i{jo7 zjp1=>n={AmPP0ptTkEx!N10E)dt+wre%WlMK8rst5@2B>1BItH6;yqsnwos|%MCL2 z`k)y_|Aaf<#(W6WZF1&&v#FnTt7;3GP+c4mjn&A8lMAwncyU&%y^kU&r9>_r&DxAa zirG6=u+y9KV6ni2z6Ky1&%oFlz;R8X4Gg7CJ-`;ian$W(OVN#T9lUrYJe|)y`d1p+ zGV=X?)^EXj?rL}D>A$LPZ&4EJR2GlCq|i<@iO^0O-H%Gq_Z?k@__>*$!h#L z9@6<;kVoJPY8z2jJdc5DjLpNV^-`xaAqb|RY>Xl?>blx^@?t|{7oWXDUj0LR zkO8n1sTpWcxrKM@{eW&N4?o?WR`j=X9{DX87N}BH=O>>rqMGt(g_RJhBr3AC}$9bN{l_wYt znLAw%cYoliWnfu2of!2)qGU-bphmKz_GgM5huI@XsHPmSA zK!Wptx>=9MzI5@(#zaXI@0@9XQb(=a4mcaCzPl;|C?kIn1QAwyWFxKueU%MMV6;& znZ%gkJ#mL@ut<{ZJ6o^~scjo-sQzmXJi5kc4pTlvoTn)_H+bDoy)g$G2rIp5)gH2W zIG;wP^Dv&)<5BxuaDRMQS?G|)l?y`TdsK-Dp>1R61DppHp9YV7tW5W*Cw^A?fdiCMR2c}1E6v1dM`S|ac z-e)LnBfF-oU7?blLJ3)J{!j#U2)(J*&~eFZd7{U({&PPvA@f{EIOlpfIT^+A*{jH2{ZDqd>d?f5jG>og%aeVggf6VS z*C#Q9suZ&!rc%mSR_-89@*CaO8+A6PbhKo?Fj?560nonk;OYEvB3$Oe9lvPN zKhyK&J}{Qv8spnswBk>lD$YM%o58Mao)sMXigm-INPrBk z8sz&^4CWChh}bN}K2ugU2>mQ)FwIe{7-Rppyx@B}Snk=Ki3f`G7W4w)5B5SoTl?X= zEVl8;bl*N&)|Lll2Rk0DP)HZ#GY7D@yyxBW5&YJXcFd+e75T`b;l&L=v~D#j4y~`2 zW#`@SfTe)({J8L@Z-b$i8lqn2)+gt}tJjie+)!v{%P=-Vo42;0_Git?v3`OqQLXCq z_LZe?|3)ev9Nfn#kzPfDji*+qYy@*a{)&Rh#TrWNz?G6um#cSFl3FYK8{B%g0(^qI zdrl}#9<)nJ%cD`p<0hgxivs`f?-ZB)vJ@YP045hYGI8hOIM!R2h$)?XowOh-wR^f` zFCjvwXsFtuE#~j5HHBYujLzxNMJ>VjbNb-Uchs`#+Jy!(Qoy;L5X zl+gY?Gfq?M0kK(Fgf>cYdpWc0^NXaJBDVoni-<#0=_zni$*(`?hSic3=i?*XPMOfHZl4^mO39C*YDCJ8vwv{dXkpsK+m1?>DH^yvBDb6MVrNT^ zc!rk?!qUStAQ#3hr@D#ky->$`v-Rpq9bD`Y7qDr%_e=Vfh@egjAg`l4ot6-5*bR3D z6A26?tHja&yg?~IhTv-y^uFHU({1$OOSlA;nEJynKJ*hJZ*ec-2Kl=-8k-lNf2X#A zch3X_&cx1?!BWI1k}Q$^3}QoCI_|WY-6Q9*RF7hyzx)e`zYoRk=c!mW!{=|KEHgbg zf~t6RNU<1H&}T$i$Gq^ZTYf*pY3~(P{}6;QP4nuEBWT|V#)JO?1FF)OVbSEm!waH* zTt%yNk8wLKtLT4!Ft5m`0P-8d|DvfHJRz33XW3P-n;DmOE4Qen0JF!XU){|hfK*Th zXO}iQPj;Kn5br%ugBM=`N^MNp1$SCw2rZYV4f9uuA-{rD?M--3=Y2e7u_cHq{56j5 zGQ9Y+g})!nUY_9fmb8Dos0~WR5Q7^&SkF7N|x{YRN|Zex3p*%qG*5devon>?p=Re}N~v0ga$MWkzt`+p|p^n%(wDW<0WIxeaN#AM-y8KyGd1)u!henNS+6@`x}3cG z3V^a&0CvgzgzSJ(_?8ra$w8*W0}M1G1_Y|dJt4?pOLn@8)^&N|H&MoU zHkHVSS~kSt0`h`gM7$fkX4Uf&&Z<6hCNN*H-By~orWZSkzCZ+%<_}T3%QVWG-{of$ z*iIos`b|jAi>+Xoaga(&`Dif?g+?8c3P~E9zLiLc?iv7?J{PNb8pUtF{0*z--w}0R z1sPu(RPs~C75MSgC-kP#f&jzMF`l|L7Qyn3_#1vk%*zcShpi1W9<W!>IJ>}5r*kMN~MAC$g5Im7n( zm)xjO$`rTCv)&HT>&d$Y`^O|W9qQl?C#lHCwAN@4%R373Uh z@Q6SL!~?}mT?p#)8H9qJJT~u%t-*LXy|(pBlWor5~&tS7)OO?XyzMtaDHb zL##nE8VxZ zeWbxp^g+_*9#Xi=x5@`lC4;r!0>)mP*k3UH_ zajQg5=mUN2g3m~@aub~2LxmDJVkb|k{Nn)p=%LMF8oCn`sh7)bH?EEi^p2y!^#G>n3g zw7<4U9{yQA7f#scc(N6_z^`p<+qz=SqCp_V_tUpD`A-?!VF@R`Fc|r~1%22;4j4v} z0t`MEKBk-|R+%=N5rleeNDtfV=}XUiZAjb~*njo4!FKSK3wED-R#r%R>~sBJD~N;( zsBzk-j-8qU+b0^NWu+&Nj|;uSqJe0ZIZ zJ!-v(bGW`%@_=J<`3&}gSh^dB{P&Okoce7sS3FZ3Vm;9$h+OsVwX9Mm&Hfmb23dsW+iA`U-u-+!9$Ix%O$G*eMH zBbB<>Xft(=)E460E>b4yd8=G*dn36dHd(~70z2p~e{F!FdVNO72FW3AP;OSx(2+ir z_AZ|(%`GzK<`s@qmyfSuFq>T8psuANXt=i@xmSuQ8s7VAQTLB8mTaD-#a!y()NdU+ z%t3lvpYj3&OST9x5X>sO=YvPtuA-vvN+A3g`WLl%RxBm9d-5Ey3-v`H;i4dIV0!PU zxURQ30JuvPRZLnDB2sWeH8}_(_HV4*iIyab zltZ-ZE_Fd^&JuOdP+s0F;zv@4RA*S@g!J#zGojEjnW6zeR{(WE_>g)9CT|NaWjxX661~+xjwwst@G@uDieY7 zWj3mH6g>8eT0P}XKE~o0<&IrVo<*_oPRSf|&|&X>(X9nW*1GMW@WZyl5S}ja z@hLb9itoUk%yNs*Xj#J#JI*N1#e%)c(C)mv_7_nr$eY^%q^`ENLH$6*t--XC>xa7Co6xND&y`48?>gLAHB`V1UPN^evZ1XUsa5eV`Q$8Z z(&d>wmVPjG>A52L;sw!vl=odN^$J)jht93IP14e$_L&t?k!_eFA(JB z%!7Ixd$$^2+7cCh<13n(o~GPJ$VDW-jGx(wW+3WpLhYRXQ^L^TQ7c)3B@7$S7JBj=?R(uYL zt}x?YuB6FRM#@>z3DKXRrDGoSeQ@4H0Vro9G=iZ>Qu|TI(LUfdh!3=R#*h7=FNBC$ zvgF;bB-swU+AuN1gP|6%V_H#ZIMA2YXme6S&B9rSf? zqt5H%*`h8E;O8PpUn;oC7+e*1jtI&g(j}a{vNu8G1db?S_NxPUh|O>L_0wIMReKN{ z#)Y`wypxkL7+?BVuj^Y2T?e$F=L*o6Zy5gf_p7_mnY4v|fgBNP9yXW4@9;xUbDG9c zt0Bi=esb<fkQM z@Ni8;ARjV?A~r%CAC)SuLpLD}jEQb-t9V5>zo?2z%qXGbIJH{G%a_6!Co+EAjK!l@ z;G3?i2q*P;?$%cDae}Wx!x z*61-EBZsRCZgtkb7r|jlfbrG+L%%cfYN2obRQqJ})PHgZ4Z;|;G;gq@4f&rpj4q;p zZ-FD=TNltR_ow3I{tsGHId%t%e)}C+pmYSMB-a)~)0g7mJFZ*XNl_!Sxk?kx2+on^ zE#4sV4P_;pSs;$E+(`UywejwwXFX2W{&-`RA@?wwzHNgndqz|GN8D@BQl-)M!XA?B$2#+ zC`Graw2Cb#5&bQYmnCW?_?6(<_~loK?LssZx8u7q)MqfNIAzFhL`YZeh2( z1CtR<5keA7F*uSkHQ@wDZh94oZJZS}z8X@Oh)a(=Nrv24WdVeY88@p%uAWf2;|}CH zfIxdf+0MC1sdq&m3Q~bSX`e+L8MZ6|}i;Crp167&;tDU_+H72TgGmp#ZP&7_p71 z91sQBI{6JQqWlwcxnG0*LI5uD8DoML+J4358hrL=c64O;)8gD6wvte7 zkga0y<)2jX9SY1IwQymF_@JUC}jG`S4Cud<> z;IARvt$;O7ROv(R@!>3rynQ0y4jeFzHKV}CzzN4=@!feqIn#HAl+Wv*%_JD>r&(2v z1d~34gC{dds;pUX)~D2jMT?*5xm+_S#o%2Yxg7?x-9@Cg8y1?VWsM(LXm!^ zo2y1I;_bmFB?e~Ml&!Nc@)n`9@t+=j&emy}0p{grI*s+)q4g2QV2CD3yt*_P*)B=veDco%<#xg#H`2|By z2WEFoa27MrUgx_%XwIHTyAOjdh|LewsIWEU1tREF&G!rO-Pw=tK*x1{nX5Q=b9h5x zAo3tJBR}HgEQ&>VREfM`Z_?^u)gKzRYDNel=P$vDvp3LER_B{yE6+bU5%`Rez`$1& zyf4sZ2iXX$0H=r_7QQ<^&*;Ux40#Xwut$fm!4o+I>$4rJCkyzZle1xWc+>6Z+pPfT zGfQH#`EU$jq>4|N2W8? zJo_#t+9hw%)u+$Egalu5JI)ZZxuWV3RjRa>+ zuYpCP0~4#(wBtzv$oEEIASA>4wR8Q4W1RF+!Nb(QsR?%8qRUsx6a_}v{%JsKT|^27 zFDvj6hUzgV)RvMC<=U)DqDMs8*PH}rj|##xkp^&$Xl`5C@Fy#{sD~StJCAU!zaXmZSf=JQXf1V0lY0G3fns@#q~d!|nGEYdn9q zVLn|gbkGf%*aF|t$4!=)(uuMIrON6Bnl0g%6BqS|K`M<;W1iNXo0O!4X-%Bw0gqEg zoz7Tf?BBe-e|gHq{0w>CWb<(TjEFXc$5tXU)K?GBq+?*^b2ukm zsg*WU`;x@U3A_=l>ZW14CVyYmZ9De1M7=A;M)rM#1yHA+q9*J`)C%0h%175pQra-S zcrHxRC+WsJ-qPY71$?^I!w*?3eLDjk_QFYOE^iR4PgU5o^;B2wJToSGpf*l;i_vW{ z?Sx{a#XYa{5`bpNSeYYWmhtMh%aebjXPCMWPrywk&-U{U3wx<^okBi`Y}e#D=IKpB zduO1h>sYTUAcj~yS0bFoaO-_Fc<+i!?Z8>y_BEgvDX10X6&gsZ-H6G9U>yP;Rtm&e z_aM*Hhm1Z*(w~K^-)SlTo&`&;PWxMN)^0>TIleRbzW6dP@pZs{P)o50QKXaYxuF+4 zL-z{jM}k%~<_vSK#{AuM%1s6nFWwHpql&~Z`HcIZPi4xd1kp^kM!dm&Du2~Xs83Kt zC~X}t;htMI)ORGA1GpW%>-j;gW|sXh)%f>=P{~?jw)mD{>{AV$a^hF{%CD*R#XU`H z&`lfSbO`Nq69#=eUc|T>Vx#S=V<}n}VygW#?!jr#V00K&riS;_&3RazpCpL6~|c*O+?o7X0dV;PNjzZSfA+0Roc3?GaPLFF;++!N$Pz`e5yJGLQ)8!Av zexvAo&<#JrbDhgq_{TE-*;C>BZ)oTQw#Yi^m`$`uQpnRhKN7PtEV6*tS!ev~;gt;l zzr6LrV+dd2`eIK&;bYL%M+{4B72xxknG;A(IM@LjDfzk6@}-8=DON`&fMkMn?$r=Y z9k|OWsuOP0tf`ZpPCOEldtIJVVW6Vq5GAe`a{rR{u7s;6`A>SXkS?Wo;>Sr159@-M!mG*)CQKQ^)Vp zvk$lUJY*@;Zz=L61EmjN3@Zid#~agY9R_SXUT9Y1?sw>a96nZ~si5X1{I1rLlbVGi zv8-Tpe59(R+TmmQ#--u!H`Mki?w0Z97%a@^op&A9yg0GI5#;IPs+dn60PlobyhVKX zvp#AMcjshx^dgpKHiSvXkmLMB>0*&<>s+`*?e)Vi?WRpPW{{DWcG-dQ3K3_?D(~CL ze~mAiH-;K2`L9ztE>)Lx=06+L6cZmUX?z1_Dx|$eHo}V-h#Y48HF|18 z6b}mhrv0?aL;@cLW-pI^#|nfLaOz~pwF^{Avu9eZMT^Rtt^#gPa(-=@7Opc692~@s zT308X5|&>?Sij}(A0N?~JreQ_(0|SYKay`T!)-n-1C&P#4Pex4t6O2|&M|HBzCa-Q zq}F-=f~oCH%yPlMi-j7aMEB+ec+*|y_#_j=z4N`*hRq?U6$+KpV>wM9fV zVsVX&C-ruP#wj5)Cucc(v~BwlXqnBW;gv_+*tkLZt|hKz4Wr-PMfH5wV#GRd_8c8^ zHTQbV<(IpKj5dRUziq$o-n}+qIvBn3IELs=m^3?;xmWOX;#f}ONX};SCzKJJ8u04Y zT)aYvhPGKG`QVUBZC$` zD+El>Kw(yb%vKM|e&!ik!0k{@Tq3*sDuEf+hQ?>_V*K@6t>VjY{ zqM#_`aZ*+j(*UNW_rH+EMvf~*V8W6X4co4M`(8gR{El|yHLmI3B7J^Gndo7}d2iab z#jy7cHT@%d*tA)kbsN2W$Qbom0jb*wYrM@4&pNeb&ZoXBVeN%6FBR2bdaz7cq3NwPF^VM zD@*>46SNrj{h7!Uh$26$Bu_+XN>(-UyD8~*?1`Xl26+@(xWAl37!_K;+ovVMOEmQ} z=f9kr@?EcLT&IeW=c6(zdvA1`Kdh*yusDkWmh}?a5$Ts93+z-@A}<$6SB2&KqN}8E z-~Tm~8+=$U70Adid6xOqlKF@Z{^KVmKxgsvyWz#Riw&sOksf3Qh@ zo^M@lPDEBvqvsNS@rW8lNG(%Mt6HB$aq|9w5Jk|3@g>!Y<9IEt1$(xDi?)px^r1o_ z%IeZV;|ZinsXHEty|pA-(jZ{l64CkcjfF6VaIbk5%KjRRMjz_TLIaaee77z+#$9<7 zDOTOO_l?Y!92GY753*Q4X;Znf(G0fQV!TtM+ger&o_+rNmhMjE{X5Y=0p6vpzZrP8jX!$ykgCEDw(zh|}lcuNw6TOWxg zABgQN_bVxx!~(UKdn)V<6!$TlnKPg8J6&Fd6va#Be<`B1{3qd)HH#8&vx;*=Y+ za|ooEuV3ouHo%qtn32SvR@oD!@nsch`XKaAh7EDQ^2(vvw7@g9BEzhH)7f7vCs=qT z5=IcCt7!0Qd3e2!Xi_FSgKF_OrTfwo3+kmjSyRdEh8L*@HD>A?IGEIzdE?wm=4wuj zyz2&Z8A&GF>7g`7b6K|Isg<8%$;nvjNJ#KH$7jWWJwAM{O*3hCCz{lIfFOq73lp{J zqfxH8^jD2WX?IdoOm1oeGkGt!!ScLo64H)|>u;caXLI?O@jT5~r{BDb-Sh2AfWEt} z=LhkMSF)MqeGQhmUr5(c1vBEtmEA=4SJqw7t-n~|6ngv+hVPY=`OoXjmHj;`61+R| z4YS8i_|`VTrFCcu$Y~BNH;$M3@cF;vTB4|WgxTctDoyK~Z}EZ_#p?pf<$*T{ z({}FQh|a47o2}Wxx0dh64b^J={bEL7}`acrG%EEO>jsH6S%Kbp=xp6NII|D6>@Ipl1WQ_kg(9NJKlQ_hq_ zD{>Aw7q*2Eikxza<$M?xLWt#Dia8%+=6p76%#Ppp=lA=5|7Q=|+r9g~uj~1GUJh!X z#`lIcI0jr8O0-${Eqn~7f9<_Z-gzb|)wk&vdEqiZOw*kpy2}g8!6|gvsglsbC?KXJ z+#0gErI*fKeh|A=yI}v)iMb|EK5hM2a8plQi5gQXEJns7iR!KEW3BoxYOuO6bt4WK z86B_#w5q@kA<1DCm>$on!HpMf{B zJN_<&IwGE<@ztnxtCW)A*FMrYe`===6ULlhvm;7Z__lQzz)K_Yo-Nd>d zg%Wr=#d0=KM%U)2^q18ZH;~r3ci4YP1%IM(ui{a)?jk&?7vTxHkfT>o88VOAlX;*> z=L@YdUX?S=sLNtA{EfQd15p|gU)%>+-jUA2JuHTg+Vnbq8xEhM=hzX3b2KsuSDopn zFT*VD)S4_Kp71S&!Do*4X#Io>-2{^ez?2|Y@G=YRo@*$Z&v$I1jraCv(gOC`(0Dnc z7k}M0B;bjz*7l}F`c1CA_!D31|3o80h&9C=AC-}0Hf_CPQOdwTno$a+{O8MZl@f0D zk8%gYo5QbYwx>7)*{sj>6iv%cBJRuF6E-b$689Y5%SvB5+9HMD*fXM3EO0RECUgnL z`|x3f6x ztHz356n(A#S?YZ#_>T(S_*z@fw1rt?!Z!%>jWfL&bp{?t+!LJ_#?rqphh z#M#XIPeZ%WQo(_K@s^Qq`n)qgnNo4Qq~Zl((L=mkXq zqhw3`ak+u2;hg4zn|nWXQL6uZH-ulSyR;a2qM7v`%&j`FH)_@y!P9;slBm~E57(S zEoHy6_idlwu48INa-`~87zg{IXpzWmirFa$GS!JZRCyN=-4w;|yj6M0|g(x12p4MJX*)yi=Tyk#;jLvnD<6RQPr4 zaozhi)7td~>(3E5GMnoDG$CHTOU!0dZ!7!;oP0=b0Xy19Sk+MW`Lp7NETV z#L;kM;dJ5ea=;p+xeFdb*L~{344B<@)f6vrv*ISvyytOLUjIL`ROSC*8UmD>ezHS{ zrXR)!JX7cipN7^~Wq%*Kx3=DXpwQ{6&_XMGD+!w$CAgkKCoAY!>^Y4iBx?o__jUhkm~5Se-$zFqa?vcrL5vl zG&U{UtDVE8!$yaN9?i2hg!VnL94fZ%yTi+yg%>>YSq?y3FCcOtZibYv{Di!{)nN*c zZHGyY2+l;46|n^62D^=w;Psf%JFFMu!TqkG_ux4=g8kyovJ8a{dAvPEN7xIV30nC} zNGp+$9EXt48$lHL)x&Bid?%=nS3m_G6jOhFMb?9@&w*CO$lp;^Tb8yU1frTb4;w{ESjT+G`I)H!ZACobY!@A1W&jTw6*RBzhnx%BP#O2d-p zt*#zjbepM@S9(%l)Ztsc0#W%kXPgQKSpztw>1!S+|A%tJICi;|%bt`|nppB^D==9r z?RpsTM0-Fg4Lr;$i5X|PXoV^|trK+Vu6dI43vxEO2X2dT_kI%ov#Ga;Qc5m^-u}Kb z*8{)ibX|(Vc6BWmLQI`rv+z6B;sHs>J^mD5QV?LRei3CWUF%yjz?NTWGr2+TT`N(% zzW=xU>1n6!2y`+Y(?YMnD)`&G>g7AnKX|VGxqRst?yXhvso=|JVes<75evmyle(K8 z?z79PZf)+F5dsWBdI7cxae?^(e6xJQ#HYI&ab5+e3ys>*_%?Im)JqOkcOhPXb_p{-lR1iZDmj8^nM#2@E3^`$NlF6$IoROAkV}Xs6$64X29UpJa z4`=U&pP8Y4{hEFP_+IlY4+gaJ^qg|7%cPqZSll`uo_xH3eOG~aIYJO4$J2O8jqk}1 zMt6~YJG#5$H!{OLkaB|yhZGsuIr6`i=eB`Q>OgsYWauUhsAXu6JRX#!rJ!5oS)pFz zyB*&T7W?H#!J%RMxtquY*o&c(_sVJN)N!$H%;GmX?G{a!TH!WK8xv~SAOs52@R+33 z{baBlqlaP9SgKANuP$yd`R74a=vSeopH&*e-FyG&E%)vcH_{~Egyva+nO>;9Yj(-* zX6e?0WEEgT6MZa^*F-`!n7F6i0Mmkf5!Wl_4S+@8CD*WxmZ&8%o{`ai?`J({DkQ!K zr_UhT(zFfzC3}sI>br)Q==OeA+VK}vMc&%f-=?@#^kWAaJc%LSslW94lh)?g@}JEF z9S1#)6pV2w;Jni+=rR$4jaC>ZeEgeRlGVm>yn~w|@2Yh|^!=#(`|i7O-I$)0vSP#) zi=z#v*7NxTvgcuINo9$wPt-{zo5|MaOvie5(jycY$}Bf__L?lp#j_>ufV&dLdyT-^ zZ$^q^*N2~*fJ|>a=_q-ZaNS=-IIHz+pX#=Nn)$JpYM144yfk zp^^`VJRwx*cuAi^x#C9&wpf0!i<^Zqdt@%Y8EPjyrbY|ihFZTM?>x8JRrt~Ism>cI z*V+Y+VfN3j5O9+em0?#GNUYB-CE0$)^>e$!6$?TZI;1iH z5c)vMth)P=@a5R!wl6u2UHxhfg*JR9#|)D2n4W*ozX0QmrlD|1pP?RhHzWYrj#7sw z8BhTSOB!<%jnULLU0f_o=d=@67t-f{v;4ivpBhT_xp{RdouNl{9fo7GQ0MrXu zZ=U^QV}0l5a=;!6^uu?>OJt@_8iG)|_)YRsGcxz?&7gZD{ds<(y!ntgNbuwRvH z1+y~Y?Z}lAo|LzkW1T>9!i;20#_yC2HKka}!ClB75ZTZRc|yxc0cb41$_g?w!Cvr& zIV`#1Xlttkj|E`F7hBH{lomi#Zbmib%4rFk#mVQ8Hk+a+2)y1pKl2H54ow|5LW&?` z0&jEe3rf`kAAXNz;x4Y3S$uS-k>Ld|Fd4?|x=Msi9E)LoI=V9kR>JgwGi!>-nzc+k z#*P+$+lwi{K#5;@qDE6y@5FRd>}FgmFwV@i?FP z2_fyrBf22t5khz>`;9ozfv(13(q@(>ApS8mVNAbG#@nw_AL9Rpq?tVJ9(`j&Y_KSO zQGxLXf=!MO)Hzuc4(U5mGuqDz_;Q=u41N##FduxH{n=0{11>#dWrfYGf4Y+r#BtQ} zMQB-)i{&0fix7Ox?fHng<*t#@r7ZL(2G;KI}`RX=LsDlI9;w*_%0Af3AB6p z&ybK-ru$0;$jP{5u(P$cY8AA1Badel^$YmkpCG%Yw&7ONHN-N{oQDfT8>y6R1#kT~ zZarsU6)YjTh}}f3c*?6c&bLVIt9Nc=Qjd7v#5{N(dF3Tnt4H#1Y3N1g{2y^Lkz~I0 z)IGxQ^0<6;S<}lpk z9BD!wdM*k)-R$LHGP>ndtpGyU3ax=`l^v&oucP-~ThV(cRX?_rEAxmm*r z&^hi*Ko-t9F*mc1`Aqzw5AsgX(GVBhHufyS{v{~ox6M7GT!z7S5K;6DebUaxNGXT( zPa9!4);z?`cklGxHJx)l`Xzox)Z%Qynk_0dH*_WUgNh6p-lrS2zFW5Hl!Jk}2O!XPSsF+3w*TI2t41kOeMk=%ZlC$N{(uB_5&9iGVELXBRIpsRi9K zu_R8|Y^CVMfIW5rqM6(WBpNNRQcmF<&c4(}d|7o<_g!*PKLki;sh=Vcx%xR(Dmv03 z11Ey4GI@&J=3T!4QByA|$YoxJ{qxX9e!0M$afC=`8TFK1n6Y zdBoKz{v@{zKHx6pMj`H_@{Nplpmt-tHc!St*F&-%+RNep``hps^RjWwcuYPXgQ5@* zCO2I}X>Au#?%fQhgXiAwe2(SQQ-&}-AgXIrbUnt4g6>(7bYfnB_O3ia;%4END`LBn zrqEZ^70t8P4TM)Ygf<5DOrld$m2Zc$$Ci4wrW6FXky;yJ#SgEZ5ZUv(Oz(K+(TWTh z_~_KT7-O zCPbeDenxRq1X4RKk00m3%GvkKyl%)jA&~BT>3o}Uz%@K3GauMViqa69(8uvKLxp-I zq38B!)ULj}M>$N|LM-a8dsj?I$q(t<1)qWFI;cSLadalp=XTq;8s%G zZk_uf3eLZWVe)VISh)PR&Uz?s?ug9cnZ&9rS64LOi(%dVQz7F?j&Sizh)$ zyMvSU59S&HNS7sFso7z~s&q19&?3&+VN|64@^>0a=TyIp_C!c1KnFyLGSk;7DH+}m>`h9`8lnPlQL z^EY-$rCRCdZ{2;;fow_ufBN#vVy*$NFqLQ|V_df3a)k!>?mZLu^l8MT=>E^|-M-on;Xl#VZ6^LV!jy2wxxq>_P2O|IU)?yN z&BM5%+S(a=r$`M7lZ`CU&QfK3xgi^JzEy9|_mgF*g>)F~XXn=<*?T~}M$+T$6T;}i zd(7(s2#@-^%_9#M?p4YCX>vMcAizIIni&Wi+tonGkagIjEk+?x zU<(^i&RoOi@6PV7^~NAY8Rp!^;G6qn(uK+tRhR%gV@0zA%R4cb>1ZB>d6Mc)K>-hY zm4iQ^q7Qa8WQonM22}^>h}R@PY(YODen$M4yipK&qg`VF)PSiy|%o~ zh3Dr{(&^-@wv(TaVZn*V}NBs?6=Aw$DO#?7tIMcbIu$+iME1 z7_(9sUhn}He)9YG#LZ6Xz6+n?xg(QJ$qk#$5-7;Q`5#k)cyW+vyN&n*WfPqko%BET zd|T*vro_zR{ye|o&%f-C`1>uo_9R5SNcFWyPfK7@cHhnOV{(BR)@Oo#s#t5+%et3& z)JYist(6HXT14G8=B!V3XD?E8ZcRy*Gw_s6K4uf0dCHjJmGx$=VC3c}#q)+X24rhQ zJNAYpXIca_8>qOCo^9sH9sHM$4lFBJOZj2R&WzXu3r1wWN>GQTx>KAT4gFoU+i0%j z=_6jPsR6orE6IVzgnuDE;*F8S6vap28fnrVrQk zud9>i(Qgy@nb9Da0Ake0PBoK=rOD6RC+*;=$;~@PUuH=P=`V(jduy0>QxnVyH5g@kn}l9M*<><_T&5YE5gug+*JF#B6A;n96%4cWNDtRF zBC`EsjSBTbK7)N5Uz7ItHo2Bq(184~A&Px;p6lN6Wz$g3AWxR>pTa+`8Dq?kVU1VV zBhS6>76mzs(Z ziRxICe;YHV;vOXXqB^VyJ;O+j_J}+{W;NP4thd zg_PL$>M7i*R5|kFxZ-F7vmE#HE%q^xsN;&vy$653mnjTHqpNH$mb{MIAo(s0ppx8_ z81FHJA|su&m&%y)FZ2;^)axg?J|4xG-F`mO0tO0Pn9&z~ayy+!eJ4-(GAi`t-SlcE z9G=aHC#MN+UPX62P6wtjp&vY#okvO5u4@!ByP@ZRs=Ip4y<*4udf-PFJ`Oc@0oIg_ zFvV)MUgcf$0^8`d6*jRxAVy6WQ>w!%)aZQ`w0XP;qw^KDBm0b`$Gg6UaGO*2C_St8V5M_};tbuJKvv>`@yJ>L6v{sy0E_0^yO zFwq0jKL1;KGU!-cVPo@QN%Pwdp4aMmRww z%$Y>y{eyG(t|8L%M&TP_#qT7-+xVhfWDMl@L*%buPR0i1Uc=erge-OH^=;}OhR zQxdO$Z2o(!9ic34U6(0Ge)9++?m}*1+(C^0?Q9Y}{N@c&Vw822C~eSKU59~aU7h_j z@mamWF}nngA+_OJDR60gG4OJ|AjUM&CSvZpN~_ndg39b7w3?|KX0lDd{bsmQ^pGyh zSZu-vnT-NlU4%(^i*p6$3IB^gH!fK61Den7WS4o?ZPDYugJDb9P5%_e^g^PREiDr- z!zhIQYQj`AZqbY0cg4!5fbBLheLIg3i8C0$+Z>*R8~~TK)?6gaOHLeusbM-9wo zCO#$j2Q=QXCYU`KOu{k~dmslfMb-_>M20eG11-m?(!=0S$@*)+I|)r{4nQoLszEVR z6vyj~&%d0BP`KCIG4wLi<%~*S&RR%yr2J#b9ih62Cn<%M>>idJ^FyHnA%+h0o<>V0 zc-fAI+tB+zrk%>1#aiV-R)qnVy1Q_tDhP|bPwZdrNuP7rP`V0-=lvu23O-b>M5Kw* zQZ_`^#K`>C{ql;>TXx@gQWTj6S5Ce$VNVPf`z}Pl6>?zB-*bR2xLklxZcj_xCnAN3 zgT!6a;}$Mgs7J<4?Sr6UB*M+01?J+Sj6tn4AX^qMdb#2w%#=4sX~THHOPpiAp3wrzT^{oIgr9U z{bJWRi!6oXtzg185l2wwJe0X1$qobYQYUe*H&<7=4*zkXGA2#t0r$(F8=J;smHX(F z&XxH@)M27d%3KE`_<0;}0J7(D#in=@bz?Q<%cqY~9x^P$iu+KJCnFgT96bB{W9(nX zc>GdlDN5LB$*5RFJ=9v{EYFJ&dfa80#7me<@3KG5NJ}aIA&m6aaX$%O8h*J#o96nU zE(?>qs?5fW@7g(XJ+6Ov0jFvqVBO|JQBFuba=#K$Q_sy1Hv`h#9);gsAX1F8R^jp& zK;44m>~-A*Eb-E-gw3b-zW=&Ld}O(hwh7-MR^2sOC;p)e&K%?r4g}})louorjDcBS(5*zs z1ly4CVRE^Pq6aGGhKJ)tE5(F|>>j|tEbp1mAe!u$N7bJzV^v)`Eb`t@xDUuLC?nBM zu24kZy^N|759wGgN*}Wq*C0xBb_cnCcssH(<6y2agJzmuxZC(Mv`Czmi-{E|@>n(B z;3JkNhD2CTHa4AmtP48)9BXPuzx}4H+*(7~@479r<0Ru-T1+HEJpu$H=hpYC(^hLS z7*KSn+1qF%(PLnhpa_M}lQi?sd24kY<7sPa=LEQqePdXnWA@`25A;TV10D-Gn(8VP zoUqsf4+${CE3rm*?VtDE*Gm#W_#2eC6@b=X{ct(4Rc!Jdb>jfddfkQ2_A~~qBS7sk z@Y4r%2gm9Ta-Cwl+-#>6Crn>v#(~T2tz?S}vf8&-8tMXd9Bin(gT2sSXI#JZi!e6S z9Q(h%Db7pD?d5x2XEMsqkVz~2L^P)fF=VdZ5C*l-@8#h!u}r`}6Mp>w2<`$_KOQH# z^T~x7Zf(B`Oae#E1qC#6l)m4Cqvs~lf7yM@sdwJq-rjFs{iZ!)!Cs#moxey&UYUa@UCGU5&xU7X_D6fibFBO%u+Ai%#oQ9s| zMVSyz=GVS+JLd+JO;?j@pF{9Jq7Y~dN{(tnFn;qunpr837 zli@HumF^->8B6E`cQBbTQMrJzyJ?g2 zHH8s98*PFT4-1@LR^709(7zna#*m=q^%Rj`%YtE}dEXTeBH1GrLMj0D(2F4eKbp8C z0bFOUo93|gd~-Vb_8ShUh6+}P=OxqmM!#W_({u;_)G&Q^w>W8sT%aHOhu%!Qq`c79 zdOSX)yM61*RX)(|q6XG5+P5cViu{&^cI4?m+8Dleg0#m{7+kLH*QM5e65^lXX;T6b(j2SUUD#;HNZFZhm40H^N+jVStVQei%c&5B_6suT1#hfl(5>W=eYCK0I5&zc!G4+8MuvX0dy;(y zbwX|`47qm!59WX(R=NNGMT~+SUz5aUD>|{dOat0g%};0R1iF9bYw7Lx%}rah9Xh+u ztXE8z1RF(UWtHZ2?5!+OLW!%;Z7^*x=Oir4LF2}lh_&4lt|9tWZAT1PzFas9r|JK; zHrKK?f5@SdbP#Fl!i7%spZaK~cW(5qyZ2Y~lZ{1H5Rw!hp{!8gsoUh9vsdZlVDmt( z_=0~~Z$(CPGnU$2K$melllfskTdfGt-dzHQG+Yt$U+A<@6Ekrm=bHm*v4<8eOj;Rro0^88C2pAwMXQ#p_`d>i zbuJ`YgoY}R+f?ryTH)NeuJ4!gFF-SI7u$cRl;rW_FUx^$gPgdeFZ4DsHV#hmU#}~e zYp9q1H5x|;^bX0+E}FWd7ko?drH6ML*D*G&go@e9v8>%ypU)iv3Oy7QCf~l`v7_za zxopXZuD!~~mT@i1vC;ksRU!DtX@BfQUax}F)*tdM<_)a=p9|n5cVqtAt+2X1qe7d8 zKPu6ce=`(gGg{};eAvDE|M4CZPF_dvmHcxH3rcw2p{MtGx{4<(T_f?9=Ki;$Hzudq zsm&z(<2O}&vL)NKuPy!IUmPD1f~n6XGEF*Hjgk_17;QFKgBw9rYzV6{p0GH)El4?w zc&KDIDn&CcJCEw~QNI9o5BaG0@C?Ej#A|K^+c@p7Y!AS3t&kDoO{R-U6f++za#vY_ zyzLpqT&G2|J#~&q7-s3e)W2_cw4u@y$B^^C1ZihAojE)ZeV8i?Q7UnBhC5liU!8bY z+?fSCk&JI;v+F%Y99*v%EmVYOrxCQ6;^}UJoSLT$Li# zlJ)(SeZLE5vMq=%W7!xD3ZSpw-xc*yY_p$ahevAo$9h zs}GJ|fLOsL1b*{hA2|(+B)@^u>ub>p;l2NeU$n0p9X6>S1e0miA&0eBXKrk5{n1Wj zS6JQO)e8*~sjh9tpa)*R3iue%>eJ-^WyuU{aS(K1audeFa;nSWnLD2W#o)N5n;$%F zuk_v$kj)^!qlDN+JEr7K4bBcSwzqj~wrD9#&y{io;r}jy6Q4%NvG#wj_~pB+#nGQ5 zh7?tlJgHo_Ze51OVaqvMUA;DuwMLaYpz|!+lpS8Y{ymcFNI)(n8^dHB-^P-I?Df*nx;ORGYhk7bBQ%F0v z894_z0yKn|HwCjgn=+a&Ks{6QuS~yMni5{)Yc2^zae^y4C%=9k_*i)PlwZ@>B(v$yJ(={C)xWVUBgAkFGyYiZzl{z8 z>oYUy&#bM_M$MN*SjbmLh|fxSv|@n~<-k(EezxC(Tn#$p_(;q{=R;!WhTVz2z&S!0 z#@f2lFgMJ4bj^?-ajfzC@OVRZ};ty;dKDV<_^*5`-u0rc9E3dumaHmzP zo;B|fnve@gJWje6N~u;>?*lf|a#`}uKlrLMd8ZspFjF%-Jc-{Pr2pK*onZf7zb5BZ_X^-p4J zqk+~A6xin53AmCe#!(WEN%m=-Z%$z6AO?J7i3+7^j=i(k{WUmS*MREDx~&Em5AIM3 zOr@8^)v)x_qx=E;7${5UAbdK*^ z`ZaG+uf3+^M%O6QxY@~t=+4|R>Ah4#PiGf#i}p+3z?nB`=OK5>bB|dTo-EZ3QgV^2Rb}i^leWvWV)j8T?IxoMX;13Yh1iSEjcluU%yh(W(|yVHsCJ zU6|fcduGal&mbnqLth=}2g^8TF^uvh0db=JYcjgx8CfB?)OjljA%toDmG>jHq9QTD zKO(aSSpl^pcqR^0ZpQ;MAS%hsZ`2gw%V<8-g~9{+aMQ3hL|7dE7>iPXQb6f9d9n;) zR(mZk3LKDkO+xq3B+EtQ{pz#GkUIk!E(WVRzv2^fp{qYo*dW4B9hufFH)^m!W4Wm4 zfW8_f?5x%c&_v!^!z#+G4F@UeCzf4kffj&TuAUZRca>^}(wsGL6o;GU zdDACs4j|<*Io~(_>%QD+Y)vP*Sf&3J%TIm}UR|E_1zHO<%CpSj6$;~MH<3x@z=dq! zof3P^wj}o~y;lhwx&g0K<(F1N;_f1dQ{ z#LUf1qKN|SYr-u?6*uHDUZEeZ`XqCEvhU8ttNeo(e*`~mEkBL;kX}W0$vmC82F#ut z^dxk>6UxAEofkb>{mDM`BDaM_%R^d`30brbqB%ddDryw}@P4v)YlO);fziiY_0maZ zO=~1c>{1rYqpaFfqVmt&Fi4kjlf(|_5KcStUBK)qT#1O#%Q!2Y|LC&n?mazZLL@sd(;tihuE#j&Vbq@R{0@ew%*PS{z~1RnwnoZTCHKac1vPAp(BZTig(2uZB<4}G=`W@Gu zr3T>^>aCuLmKmScz!kw(=RPOnH#MSM0VcX)!+MRG_>?fK9|DAXIQpwye z&DoMlsVolab}HSJcrXxOIA-lmUB6}a$e>Ql2+jBK?|SVfCIR`8NbeeSN~G@v(_nJ? zO(T8ROaycSwpK_MvSxegz)mLpO^3cy47`W}StL^(m5ccq+_2R8# zhxZjyU;SSD(r21RXP2BUnA&dJ)+zh)tvs_lF{I(N50hXXx<&lV?VaPI?5Z ze2*uK*RL0Fw&uOGUa4+q_``zxx?uxlIb^J|=L|{BYl}X+806 zRaX7S-H+3>XiS;n-*!;{GBUyn_7DJORcYOzS;qo-zH4YDSZW7iRhepe(#23OOPS$~ zdCaUNw=0^2OeH8}5Mol;4WGWC*zog9Wi6+F)(x}TCJlaZgd9H$A6!Y^gpzyKEeP4* z%MO`YJRIKtF|+w?7XJ_+#C$ALGUU5PAyah=YSU2!hduk(5PA=AIIgF;c>w z`$`96t0lkxz645o?`lt~aD`AGL-#gUAcwE9jTAL3Y<9nO`Lx2TWnP%?<4Ws=NC&UW zxR)U;syVUp9SAOy4a%eWS}PsYYJqw+fNi+YF7*HQR2H=MbN(g!#z3l1 zxv{#!&ggT1?}6H5L^{FB;(B`nRA`IVozt8MDQA)W`nn%?Tb(MLT1Bz3!>o#goy^E~ zcz?S8=%pB-Eyl)lX}by=BNbENzH-QqFE^2sVuqzb%xmsn#c>!~-6oPWv6~J(*pp-s zP4e;8#;0nH zalXK01YID~MzTq4&b%g}P4R4(?*Krf>Epw(S0kXs)`t@{{2%oDW8HsE5y=ju&)wm$ z(hT)Q3jf=U;s==(cHi)C6pF5GZKq;mF14-vQEM6*el@1)b{!fsd%a*CjN`Go_O&4KelUGU`5``c=E zSv29{3L*Q!u_h2IWAoJ2-@Wk*((1KX{Rhj#Ci;ErCDdR^{2VBY*bUScW%<~{R z9%>{{SygR14L$!CWMMhpR#Fplw?vqkLPbZ*mKJjifPbPX*9(As2sd%|prbSO_c~Fp z9FI|=wiT;WX+I8d5OO4hoCal#vOs8g&D%}XL-f}rh63UMkw{P%z@ zyFa8y;kfpS%tHs?uHmXzQ^UCj|5m_<#I<1s_-;HJvrSY*?5md9xSGoI))F}wF_YZSl z4@{xzo~Ki0bJ%P@lV#Ix$t2$kwMxaOEQb6meIVtE8+NU;K8zXRqpo;p^}&J2iAdL! zZziBinLT_sqtPoeO?UcwbHtDYZ9P_&ZLiWpg*kG4(VMCX@Hz7t)hsjmY+M5OGT73_ zxc@!+G)GZyUww{PVA)S+-9jD`v7iwCb6jD}MKE71knM}@GW$oNxLfmVzudl>FA++H^{epSt@u6>j z4*Je2(FGCGHeEUQnZ?*@ifB|qnX8L_CVN=OD9@`*GpoNe?{?M8l6A7d4DQXOT_68I(7iai=HZpv);Xsn_2!r^ufmW)mjI{VMdi?*oDY*CcHV| z=T&Py4X^wd*^;vB{bVJoJ4LqNX9@kO17Wk-gZ;7p4Ddn=Ztd@PA(nuv(cZ)kjEI8~ z`ZW;CuJGk>77%$v5+a_MWMPkA=tbqiNwrn*+5OzZG#e=WGYSpwPP4hQ)myT0GbT4! z^MJg7WDrZp|9U_`xdXz`L+t)Gfh{Wm$q!YaIS6Hv4qFdfN*?AkEobq<4^dD$5>7=N zM64i4!?_INy|nm8z<-q@mpD9uG``WDVvFE_91&(g3<{9^yW|t(ZuQHtGuvTroXtLG zDEqne9t$W{6f%Gw`qguXBxpzy=B~4x7|xd9%^gUQ3_ADX)+>l%SCgG%_ba|vZERU^ z*BHfs^dhW|u$#%(&~^8K@-vqE@PtB2XhaMz#XQ5ihtr-E{OlBC6O~xi=zPdPye4X_ z1dG7Vm(neok&Mp4JCI7N5170%+(w?seZ-{A7M!F)ag6^`U$Nsvw|X@F4U@A4QQ4ow zxL0M=)X`&Cus$qR+Z?KRAc<83kz#)6sCkz1hs@ALW9T?$Vj|K2HA|=t>Vun2Qc4jL;|M4~^wE>R6fAV%% zz<6*l(~`4kh7@Ib(vklt!bdZ-5~t89@?0`~iP7-Sda02+(v{qG*aMOU4 zz-4;0hI^G|t)T$~3FLKeIr2EGo#+d^3ey1$ATea9mx;U{Vbt~5@0;N_d|Fc9mLEyj zr>k4)#c+CvqR-m8D}3ZmGS!z8*T0`NT>pGEt+l{p{8RK|F?6GPR&_wcwff?+PEFzfJ{YM#u9bNhlYNrj#Mhy3#N6U#Dl9iJX>6puw%lYj_`-UnYlWxQ_arSkra-cr@T|tH?Iz@i8 zRmrmjJcyY(ZX-SV`WD7I9RT>coa0{BZ(*M?TL_(87?K@^UduehkfSuV$#qa_9fZ0M z8Mb^9IBEsmSAp)g^R^Nw8)#-Ub5$^xse)zpu7oY6hpnY+?q_00N^bT38lxQ!_K47l zk`OupxjrqApfzKtOnNVxvECSprT6bJ`jN1C>>>=xSc1YkE~wHvm=sVe%~^pNhbG5K zBsOgdxp^dLPDA>&o@3;lX`cJucR7D8DDF*-wM_usix zwkUW6cWu-?#Mjk9F}*O-^}RTmgH{THB8f$8dD_v=t7abubBCIiciZ$RB!un#srz{V5|FKMi5lwzDmtz27M#^E>#33WG zOQGPRq!sDVO;gNa?V%`H^DrXg$f9x%l`6AuQ=E{CsCuSjk;vg_0own zbHxw1jRkKZjknF6NO%RppaT~cC%bqz+({VJj-q(7|2AulIp0beeB7uD3N|nsjT!1A zr@o%mfUT-{1tc&o;7C#lifS_ZVnDm|@8=tCDJe$70e~fFyJE3ffl!pBgi{JRQFlQ(U1L?g)Af)0lRJe_|oY~ z$gLm#WF_VbmbOTprQ%scUXk%JQ7=81BMq3%Y0Q>5E@+vUiKaJUc7e0@Mmee#@LG>q zt`yb+y_{D8OT*|N>+2|@hDKH6au$YKi3`EcwM#PRQ$W-X$nD@A1^OnI5zp~&HylZA zL{g(P;P?#;W21GWxs^)t`EHCg-qbt>a8gx5D)6x*-5?e_&e*4Nj8gTY*JVZaW%6Tp zV_mWO*w(*=A(Op~yfx&I0)cesZi1uQHHg4zc`Z;nB6n{1vI)VZtW$l?ENY&dAR@GC zb{bxYlPK>wOW8=H41X)nD-kXsUubEv8rr`^A2Q8-u!`<@LO_3VYIJ&Lc}J>r5zolW zbr%9lgpJx1uNYbd?2z6PmWnw)TN}M^#iDQ?<#HPfy>EMFxoBgJXiE2tPsr~<3_Ug{ zk>1_k=2ZRz1s^rV1iWFnsn^Vs=-|6P(l+kUz^ubnND5Cn%oqFx|G5_XIE;|&p<%|; z`w!F^2-LyKL=RFTJk-wAKK1@=0E{=0U*kyw+AF&~V>YDNo0q9cjt4TIw>b~79845~ zw1e9h0pBfR)`T=xT-maIa)z`k1Jg9{Hqr~JdWZcT`Cvmqrz9AgrS=hhYEx<`QNA5B zfjRUEgtIXl5jjukUS}^rX9iKg)FAJ12<1K9-v{}`@J4>rNmTiPaDlw*`HX9V)WI|b zP0wuHYhs>%nUPWFb{*!CV*V}~wPq5(u<%&CKS z+?aAmd8?(x%7Hm62_t11pci^*$v7-JBVrEW{>#}PUO-~BhvvKcAEz&H8a>)dO4V?*_g#0K9L_WdFBi}`CPt7Yq{ zjOs|K=!&h5hzAnX%APqzB0+=Si5)^BYfV8MO~i|N-Q8cqbWI(?XA1rO+qt|AK1?ZH zm$kkbbc_vSIx3v*wYi}*iB5XqJ9PeXHg(_a2rm?+CmyS8~bX|Nn#i z+SgcAXDM3pgBEfZh53c4pd&>(CAEz|-FVb^nQD5oMXc1+?8fQFw3G@Rf1J*D&%#Ie8+{`u{@bM=6&ckd z`f>Wvjics_WwR&zRf4f-_~`WxnIe3MlaKT&bwEE#Ohuf_?A<70{aGJ<@9>C<%5gHJ zZ&!a*0_BK79SJ_9cWqQ`b_GQ<;yXew{z>pLI+6Z}sAV^XThLd8dvhgf#wX%iL7q4I zfxdIa~iN+^CPTSTxU`ku5E{U~>gzuii)u&=M|S4i5Ctv@bb0Uu+p zB!++Ft|B=^KdwI#D!O9o!euM}me9A-q(AM_kBV%MeiY`6IAhuDePzFq96y8oR#a3x z^Yqidx$!p>Cr$$S!Gi|uy4&vG`qsDh*nN+p;vz#@ME@1`yOWL`XV0E<=bd-n|G)!p zz5VvV6AnJ*dlUlPllb4?!xo7H-3!4h^bL`bH z4@3NCe8Y!0*bq+7SbR)YJvV()miEQALDWlX`KPc^!O*mw;7pgT3DTAv{1ZKiG>3&W zWQp$;s38<`k}%9G4*zy_xbkC0I>N49bM`wkv%?YW|CU}dJGkq6P1O}_*pY$HL8F6$ zPOcmO3i}nKretf^uAM$(#$&zwGtGwnqg8omgk_mHj$? zK$~s9rfaF{4`8lX7>%QqsykcH*Vf`87Af{1Y^`D#d z({1aEuwU2oME_g$4=DjDeD(a>5k5)#?ur|E7$8dyxn2D!z$V;Omf|e3r#+^(>4U`>LqMW-9t+M1tx$WK``Bc8fja7y- z;?$yBO@P!8014z_^#cVWjkM8GfDRieF#1Xv1l9GcUs`MI11ma7PiYH|RBRL{Lh>Fwa*UM-{pR%kJ;U5(wEs5y+ zuSYn22qyB46RB`xK=pS0QNa|c&W{OQY)HOjv8lpbxx_u8C=r+Fk8TV+a<$99s3i|4 z5H>Q-zm?(vCO3=ncvnav+A)2LG~SGV5+?fDj88zE&dw} z3I7Q``ox1`-M%6Q46t_STZyWu(c$E@oALJ`c(RrO?R3-5*z;uw# z@e#@~{^~({w0@fXgX81o;Pj#61YQ53#;pJ`!ja)v!AfU;DmB(4U1!l3Cl)F8p%~*Y za{sjbX2Pb)%RZuxP3C}+v977Ib^sL9^m4Ei`r2IZQk7RVVvaokNolF{C;}^yLm)>J zS#jq+)`~?oX*R2DlSI(N<>$(IXbnA|x?tZw{5aIuHn1cwZ}VP*ihO55A@kp7zi&*O zc^UpR(UgH{J9n zKe^_llTP~SPk(yX-FN-r4}W<1<(F@|>85@5*+-&_^#{Fj|LH-JVU~aX%6=33a(up= z_KQFfn<=ABWLd7?8=y#EQiIi0gjM~z73PDq2YV9XY?>v&KuVkl= z^!8s5;JK(-5w~7i2 z_t|g1Pgbn>$HNcvS?;~}-gE5OZHqc}GQk^LS?!}MsJE)F|Bep?UH_f`hDPA5tFEpo zD=VY)#Y>i~D=j_f;0fKjbSW)e+t|pWD!7zzsgWAB$gJY$l60Eglm-nPkl>%tS4Kf) zxpk!@BBf?e)#vzc>_0bFR!;rK(X{(Aqp1IHqT+GIZfP+jX~Hk+3o? ztzwL<@;QCZp68!ze_PRa{fzk(PsY3W|Em3!y-5|K$I12B@0X$Un>Pn~2Om!5r!@R|pNB4;#7mQ_j;gJUu)GGj8kQ);J%8XI{s zx(3QXxN<&S%@dkV--;qx3;v!V z1YuB~6XY?9fxk4HrL(Uz#(s4e=F{g-h52EsJ{2+`(%2L<;t;eUme{V-R#-e@1necw zs7V(6uz=W1Q*K>-WvaxgpDSplZi| zKR$(jh=@)WK8B5Dca&m6maLg<9K1gI)}DGo1JCeJ@PTRK;5kZR3KWN^b{5`(ep~(b zypGR$-fHa0bc>dV)=S}UPc4F zMbc(g=ZDoy1E;qrj06Q+nne%#@S0B7GiWQ-P36lQ5R{_a9Gl*?NpX+A9dzEv?tRh5 zNFj-it>8;;zf9=3=BjIMy7}hD#)ka7yi-m&`QnRzICSVx=RX#c#A+e`TKmn-%^NXd z1WQ_2IP-(^&!0DUE>AKy-E#A3r=Hreen$F($i(57to{AUe%;%|`opguE&M;tR4e~R zA~H&*|DN`@zMAbVFnzfSJR+*pV_-OH-<((O)(#x6-5rcQFzh=#m7A9uwoSNjdg_yz zO{*5=zItysaC6eCBcz8Z4Y|TKt`s8PA(20#9lz+^^2z>v`$7v6dcp(_S*SK|RVBh;=bd-L;dt$>^bNJ&#zv-f zJXTe$S9co^8QP;~Pu0nIPWYz)4h2$S1C<|7Bm^vjB@io7W(BtHKPUK{uT@ z)_?G1&Gemr;Eq^|OG)UPSj=*iwFUjJ`p=H$dhEAV`vR&4EXEW0+X#O_Ybwzae@l!6 z9AQeo_5rddQ+o4dl__2;HVGau?`D1PeTQ(+C^J z9(X9dBJiGMz&7n6MG>v|hkcUiP$ruJp*fTca?-J9V1nc{R3%um$T2TNt4bT$wmT=t zVV!9}=bzoMQguyXL2kO14u6Vil$jzP9IQIE+6FvT1s!b$u2-Avm_><&Y@H<@A)8HK z^Te2PDff%#fn6F!m70@d6*p%?-K}~i0kwn?{=%dgf2)#MRw2S)_@I0iZUmx+ia>=K zNvA9Pg}Uw>FmVDE>8mvYP9!pw7W7qy*pB7v{DZ%UIXlL`N|gph{6o;WAen_nB`6$D zG*;yNNAyXi7t1Z>YY8oZsCq5p{4srjWwccr>qIzx(kCzyr8w_mNCA|=^@m{HTXZY_ zv+)syn7%l1a&7d}(>Q%g;&fZ#A2H_iNnxdnr%0N1shlx=2hgRHtv_Cnl2I8?^cn4! zY^uHS9Rn#moGDXy-8g;IBG@vq@N20$;{OY?U-LiWFvdSyzG4CPI3#ZGsKJ+7ebZPX z)sDltl1b!x^AjnCXH|r-Hu}2BrZ2)4r6i_5+4ws?!povf0{K{+~ z8}lsm*dWex=u|eSUO`|XobNAR|&qV)Yxk_`S6_T>F(DmnEYQJCm+SfMQY_kh4Jpb9JpFQu~ zbIZ%hF1X+VX&Ow(V59c;EBi$pCYS#*`<1t`zb{Gqn|bBV^m|XFD%P?cMneDQbbUjB$(o9%aPa#d|Jj$XBIq=3hJz}vyhBN}ldV2b>3R@*IMUAl- z_B6$fWr9t{1`{p{%U7kQJk2mInKN}&sYid7dFmzxU{m!~Va2L2$Yb~>ocCV%n{UaB zv$h2aq%H0ijNBzS{L;^1zbsC9f9i}$lUe4bwKCMgyR>je9}zdohj9f3&y7t@Jc{Ek zwB@6zfbZd@LC%A6q}uL4kyNlq5o{{Z+(5k@)av)&6&%~!~XsI zkJw~HL1Ce@OOc2?ryx(008^>*@~Tl=jH<7%d-l2KOH0?h^4e=uZns@_MQuJ)`#bZ) zS+BqT+KN@Hps%((=?0*YIlsd3e_nIGXsD^fMF@)@zTfpkD4o#?j2b zLE!X-usIv~38fPU;>N0IvC3=zTH*t|9!kq5PBmiC0-p$f2?}!3^oYK??``P2^w%ff zX8ye(RltVf68=?EYx!EL$jm>Rd{sdnSKH{1xxZ%qtpEzvFBLQh+&jy?aGQ-!l)!+p zj4kjNg@k_*GVIJvP@d3pe4-huf}s_C@+hC3TH!C$ITHI4aRqqts6MUu7X%Ui+4>U| z!cUg|f8q99US3Y$C0C0S=qhQJ4ez7izk%ghnIJcpu4<-Bm#&Q884iY*<72uM*GrYO z>vr_V`7bXoQ&kwp)L!~wPJSL8*(S{=$aL%0op6)5Xh2HE7=2;W5`XTZ1TKUY_(%mC z>gy}YE9kn1jc$Tsja*n9M3&OlxBRn@(ljCeAZJ3`| zfd11dgddhJBh^O6J;PL99((eoc)#S!s*O|QUqu?9ZTN>@Rn7}mE#(;uJZdfeVbAsF zyCe5#$Yd^i{eetQ9;2{B+jR=95AJgPD_SG;Y?|(l_rvy6i#0~Og%~%SIU;<-<-Y#^yRbP zOk?Bq*I)lXzxa7oW!1oe1FyXDO19@P8AY(Do-T-%`2XANm+!Xx`GLP)a_J9$|NGxx zdg-NgwRJzb;zvT=phk6Ef1FLp$y6lK9ff4`-^PC`Qf4GkCVAWHXB+w*+OLtL(b9o_ zM)Y+rrYYi!vR|>=+JCChcIjJ$cya%qt)GsMC6LgQ_A8?Q|Em4n`RMSihZ@UPhmAFA zL983W>nx#)^|G$Fg3U@BgB%{t)$!rb<0YSRU>{H(@b$%6!`L2urBb*oeU?@(PQwD~ zFlOxilAh{$VkHFs$eA?OgUV~t#8{Vu6>RUb7;@25EX0)-W^ywXum*z-0uF}H-Thcc7T*cWjCr%RbzP>_G>$tM>S z_)vwSl_IPE;6bhIcjd|z)2B^mxFRPnZ|}YLF6dOi+7g;Sl?U9pIR&}7^lures;;S| zqT7xcTUk~8=A=ms53O0Vrn0J{q@*)kv}(Vy@OL&OWe%Xor<1`jnL101kLfiA9gMMK z#||1iXp^Bsd-v-7(kri!om6aGvc={jHQu53kwJX}9s7WGeBsh13+BzwWKdQS6}-|P z)sO4Me@YK5G`oLD8pA;+5_Hu5ihrX=s))YnOg4QeIDIA1b(ANe?}3WYbA&XFqZxlO zA7?eQP;3_eN^M*aNC)k`gAcBmGe>$Uf_7j)5Tltpv|l&)2-znvnhP|T(;lKk1J75>yr`CI#cV_0;o zJZG5&tneRu1vvKc?@c5}2^;tQ)Mc{(RN`+Vd&$cNqbwttdnq*#M zm=DP1y#SLrKJPD}&tnnvg$VVcvc_}5 zKewuf5LOh2@$bkG z?1{vuNy3YF{b!tVwt$d*mWerfq0JP8_<3G_KRSL3M zSb2yrG<>WsI;mVEsq~hJIyZ;M+zKP%4-1igI{z*nWwaxq@3-0bkeAnNv-duK|Ht1o3}LvpN?|o^;eW{D zfib9NcXk8)vDO9DCfcx7~JIUcT0^B?pCGbny>wx%noc?Vwwn zw1o@UwFpHTKFv~zNt6_qu*qfLK7DuEd8Y#oIAHt^JM`(@JC5`!eNOv5Z}i^7OL`ss z^eqkSC8SY{Xl6%|dfl3M#u^s7pG;sYWA$Cg6pu6`G<6v*QDGW1XWU#JlF z$EXL6vU8APVGHkMc_#5?FZqn;+Kf07WYK312o5Kz&3}dct}QJ+_ndPcc<=!vx7VI~ z-f`!h0|yOi+kQ!$5--Pp#(qmW7vFsI%`}C-yXCi6UUg;9o;}a}?wJ6+;r1K5ssC8} z3%732SfE{-lW1w-ZmQY%tB$dM=2|t5{aX}nO-<}=tZ!j`4U)AsjGyWaWoMn)(0{N})$2rHJ5e!gbK&lC!~E zLD8PQEEfFPrxbU_ey3qDbWE`Ap~0eQnFVh*rI>R9XEfZjI51%JA{1e83zWt8Fkn)Qv7k#3!hk_)^rV==kH0&o3w-6Rqs* zd2`mSUDvC3&wl;-vva4qj3o8xE$Q?JGiO#+R1O%(zX5UAMGBfksx-jUV{vqK?rzlmkqy95d zb@dG_-I?*h3?_{9@7HguEw?1YKOTChps--;t+w81(4ZV9sN~UMmXoQgVKy|QD(M}^ zkLNW>O?CBshuPsJx!J0m{{8+Yp`TNtRGZKB+A%_%p zqHAr^F~wmu&9I?E_ugx7*AuwqP<_Ji&B>FOE?H7pQDKjn!r3Bqa{rm!!Z2B+vGK9~ z{u_xCZ)O<7P&0i~6H17l{|iG(P=_iD9dD`p>h^ zIICgq{A9+a>=$)F{IWL5NA<*pr5czRV}$lV12R0dup^hgoTRrjC__Oy)oH}Ah@UKc zoCnUTm$7Y;0-VyC!((pYpJYk@dGh2pfB)NCic30QbM3X=x^;84+N|vY7ps#oZ5rK~b<&uMlsH&JqvmgnMP?eZW>7c;OL->mh zC0ADaV}zZ~aP&L@eU;%*8htM)qNw~G&{umV&C`)hUm1*l>6&QkFXq-5u=fk8`X+`m zn30|0iPN^9A2ARIzB!oN*Fp|NO1Sg(?7&&II@lH&g zWE`0(mT6+rT3rKu+(wvbrq60dsWR_Y({)C-WB$c-7XLB+!iQH1;#jqDiTM@RpN@Zw zk3a{gn2^IB(P=AR7oVc_=ki@Y{t%Y_x4|F3faSN=pU+O;R4T^D^e1A@_=~VnIKot2 ztwct-#1tCXp?=2rD7}^hNB){z9MEYGe+yLnXQ1B#f67YaYxu-s5zR!+6Z-4HpVX25 z$NKZd+V2a`y|8BO8dFU3hU!z|nPJWFQVS>8ZoBQ)^M3GsZAJD^*spH}$tx<6r?-tL z&DQ|d^DjJq?RD2(b=8$jn#h)~Dy<^bZI(d%XwN?w!qT6^4?k??%$cMY)3x{U#42=LcDoL8XJn-NONih|TXBvi17z)w0rGQ3L!D~~# zcmVwpRlWuNT=DPF+qpgbg_Nn0WMKT0RKnvhDf=afvgP})u-^p>7M_0E=`Xza0xQYS zJ@-5Y=?V+z+C{t=KgMj#p4(Ro{7D?!FRxI`%gT8OsjR9hD_d7vQ^V{YDvI2h)Kgg4 zsibpBaS@imUr}*!5sw9mwc;ppL+tlwKmBRxnl*f-<*IA0+G^`9ciL%Z=he6{r@J2g z(X__@X!{$>EW_i^ggdz;oS%fYK{ zzH55Nqs4Gb{eiaoZ{tw$c4X;~t*o&}2>AoL6TN!&)R;JL(sEKuUs*PF>eS6P8_~bt zfF3=%L7a(xb+xw1eL?=h1qzXv&S4pgJu!U-`=z~FzWkF%AAghu zEO=mCupYV800}RUGOrARrnY?j*BJFajn$Vbs%;>cTwLf+6<%-uIS3XnTJ*sOGa8zj zx_0Y&;DHBVeM^=sUAS=J%2g{DeY|LwU3Xywgt?!HoJo#6N6nizpAUGmpJ-)eRbxZL z)M-;28XB2`!HSzvn~&N+``viRkd23I1kR=FO6w<0qLqesZw?awY3%cy0;KLz)wbx* z?3aoGt{|e0(w?}wX4hWz|4aSnh1dUV-Lo&S?o{~jPv)o&zvfQ4;3F8+XVaul9$93` zSOT{ZLaboT+i2t7haR%g`R55X1MLg!Kc92fcc)FA2HEYm-~PB0j^FV1T}q9jx}lnU zBmey6?mq+Xm#zI}W8=Mc+a33f*$*8$bk^LtU$*vFkKGy)M{i9%t;g_B+OLKNO>Lq% z6w+*&g*>O!JZTT?!Uuy1mY217o#bU`67)q9(nsz@dKuXXSN@zy7NvFiBB1m}fRoBf z=v%PDNRJ;ALyVD&pgb36)BA{i!oL;W%s&+C8>`k`rh4~Z4*l0Raq-d76gf)zMkXT#( zX(0@451s`Xv1o<;{;D)LocZ7(^WoXWk7gjBIql*&<{yZ;Q!btC0h?c{Du2X<`5>#C zqHV;KB?MY2VE8L-7Cs%}Z^c2bnZC+2DJZ|umsAKt-)*qp!nviQ@n}~pUq$g}_**YT z?X>PRhutLVoKpjDPEC+xN%%L{7GWUadaGmfmDOX6K5-EPh_v}RHj*#&6&>T_`QV=m z^67EZ4$Dp~lrd{qgfkxgGIVM8$yeR29Pw%^*?x+k7|thR==5vH)J7~q)w!HTt~#zMGc z+0uLOx#x^C&QJ=Owb+Hx&t$)nF1ci5pr?+raJL>D(f8xB002M$Nkl>Uw z#~qhES10($G{uL>NtyYR0$V^N2*R2gn=ZfN$4@=^v^bN5)E`haxHYMAOD@wNRW*w~ zJqX+a$;1(TC>wb-e&vy6ww(D-sTjJh3N8<&j#kjH>ZCDb8xF0d zVW^e0EgU?xH1U!&)V@TJWuz8n3RHP!yD=uB7d))&DF`x5lCDi_84G?-%lCN?$%v1Z z^YQv8!|Hs9B=mXXC!w{#U;M}V>4u;b`iFh}=j?aJjG0Fs`Hgw==lAa0=holdddR^C ztA<(nBbdZ!q&Ds1Pas1Z&pi9g#Mj@LJ8%Ay#f!@-%4(`>n%Lu%2NAyXW!ZrxACoS~ z&Ce_BRMe?^_wEA+4j3_f#11>{xYzEx^L8u7=Tq9>!oouKC|&sR$CD;byzs({pL^~Z zeb7`HfUsJ8*hK|nu>d-bF{24ZhmF{fJr*=M^eh`zDyV9 zFoiyIVkHPaxRNtP`VckQK*aUM<*UG|HNjgCg-fPs(14MKuU{Sv*(Sqa#PpYf+OiZ~ zIu@%8*d(?8SwVp7RUR`^$?l)k%v!8fMR# z^V-BoAt(i@wd+b*!Li_@g)3M5mBkl~ZXP^g!XuA9!qPM9&PPLV3U5!IJp02BDG_`a z={WGfgN62K=b8!jzWoH1|q)qhf zL6+VT<(Ll>d|_=({qz~r`P3AW=)@3j5uZnq!{z`^9K0Nn-H{0r=2a8`6;}Dpnmv2& zoH?Be3PubaHgdC(Nay8OUs_kTj`ox711C+ISh{9iRZTU+zLYy-=1g(Kej?5NA6}41 z0cd}ne$u}pP(_=dTiLG*g@!`PKnrN?uPY7Luqb{dMe?!zOPyhWc&HhJp_=#rKT#U{ z`M=$NUVQT{EB<&}Mw9i;v>5R7ceOOvGAYT#mZ+F;fr)JkX!v-70emwDPKCXDjr#MS zJ8e8zY9Z0s-xt?^t|%`@m1IKc;Ow)$d+hPYThT4_A9)+MavCN#?wb9xszHE+twWbY zxAe~hJB`--TTK)C;K4OvuKz}fn@In~gAo#LcH#QTUEB6$1ML@nRGdsrTsZ$+P&C&G z{kU54QlF{+44H&~m1Iyk|6C{Z*^tS4d_<>;3Rsb>H-!iV9+XGH{QW1}Uq%yXY1G@c zzCtVc#{Gw8d^*D4Dg?aJALcBW_kgS^4@e1m)mVRATY{*Iz-Tlk0xr09>_>zg8&ewj zSr?{f20omSNzrJLR;`$@bo!(=`UH3v#NP!FO*VV#P#Km*ls%2Ix1ukpMud6qP+qh2 zqqVOst$J(W>NnTce^gOd-lKDmZw~+aus%DgaI<S2zhRXpx%rmhry;wW#dn5Gd`4R2MPG~x(okEFWbilf?~HNK6Wd@F_;#eML)rI zL*y$W(ix~pfrKarWk_pU%C}?vQLT-iM1LZl`LV<55bKYsBrR2poxC#KIRjiI`s4Zu z{Z{%DXXRH1WMtkg^)rg_=$JD|aWvzvaj-}~jY(x9BuNw98-3-}g|#JdeHCWKTco;9 z+$HpdBX82Q*4S`W864R~HRB)gZ}_-;Nf^<$EL^vu|5@#K{zvmCO@2$P&|_l7wcfhP zI3*A&0y(j#J{Fa@_nv!CJn1_H`Gx<4{Zf6oIOs?CsPYkg9N@{Pp4_unul)}=pp|?{ z3=dZK7^^n&CH8^8JoxKh|B7uuBovM;{^i@LkTtLH@B8Vg-h>*gWZ*x5+E^ODjQ?mj zMuHUvN(nw7GCR&(HsddC~6NMKI=H zm*PwcVaymG&OZdHRji+vj5#nbo%+$_zc=yr(>P$XmFXK~V74?GOWmu&wg;0%8%QCnL(ani&; zKltaz9(!!XiWOu>E>sJ>055bx8luX2&5XP>g3bhZO9oPnR5)nh;6o-%IO>~6j~_pt zcb*A-k9P>UPVBd9m#(+}@%CMJ-{q~hCjaKfn=k$04@J%JB(d{vc@<=Tt^J!yb*?-K zsf|AA&3-w{u*oeQcN($&lhV^zeOl5t`9{@sevCd@NSyF*rF;r~Wf7Jw=^MNe2$*rr)d^9C9 z?dkNkhiFh^?aJ_uGcuKHSP>EKa!hKcZw1}^XG&KFlm8OFau?6CY*3kV(#_$J(Twi_ zS}XmDG3QqYGX7`U-*EnW;mh~p!J6{&Go$uM58hInI`CvW^SR7|sYonm;HJUur-b<> zVPivT^}=BGn_*3@Rsw8yP->e4y!f{IiF{rE*VF#0UTU$?mox(zt5$rX$#lHBWm{MF zW^1TtmKUbNMMFaaUuJ1wszalF|0+l^&7r2IIv1WZA@dXKYUmuw82|>%pkrdGWBo5% zSN6fI59iLEqc?21c`W(de6x`}(vzROb#?VCKKW!#>FUD5q8>ea_4757?r(zN?YG}q z^zp~oMfa{L>FDl$@hW5&c|EJ2wMzP$Fv>+iky9zLO0&t7A;9kXcBqBW)K zKAN|nuBMJ$Be|y-LdRZ-hF77HI56 z_!BNMTQp;$j9>BOOacO$@&6~Df0+@erO2kdVnKS~MQij4DKn+n3YWTZh;IdIIhHQD zu`D72qe^2ZfdY#d9aU1-$%z>OBSS3+;?NZGD4UgR-kva_AWjsAT{#Hc#3i<*&&QTB zb+cDL{_4VCE~}Z$z(hlat@J20FE>Z~1!~mH#2QtKzGr?>VZn%AorjO^cPRANR4kmh z{Kj`axrgnx8d-_LkVlHOD)nmp^f>~aS!$rBX9P(EGVsqN4yJh52RZqSZ#QO`7{XBi za_JNwF<|3TW`8J1pR?7&n;Zym5duT13nL(2HNsU@B7Jl%)zH*HYFaYtz?(va>zxT> z_t0Q|hmCZSUj?Xj$;RJ_IDHiq%U7KHp=Z1Ze-n}Vpr!=Ct4g?hExod2<8M5QNy8I} zO@G?Rm+G2nHR}&lV9W$aDTacW$)b;MHW_8}AJI2_l$z?d)KA68Y#4MhivS+khS)Un z?a)`a_rgsDjez(f3-oCt3E`isib%QA+1dGTrmxJx$3Ye=g0clZN&?T~E8(gc|5o

    DtG?B|c^#)HEtu^sNf5@M%Tg7~D|%h3gldfBxfz3q=#Prk(~q+_TB0&1IHUyu*XL}tFPz&HL8kxC zf1;m`4|SnNqy>ASCKQzL-($ZE7JPKV2`A2-J9qeqP42$y9!41irEqT6A3@fE{qk1o z>1Ut4?bcggeDOv2MnbBBIel_m9mMmHPyWjurwm;l<_}30U_q=m=+;K-?VMQxG#wXi;M{GLcCqKFF%rnlo z?z(FZJ!Ha`TW%S1EB?u(%0wF5LjOBze;K%8_g|_{f7iOmfNFHVDouVWxn8*M~g+;|s7Gkk~mL`aTQME=&g|)}O^CDaD!VMp(mbz#G z>-pc{iw~%)9%lol&2PEY=uumYhFDCkeS9dtZTpM7ct-X1E8>uJ8dtVEI~m)~JSAlD zFHMT^5kqj0g-_IID!0qmYQ&6$oB{p%@wl$k|MB*hsSslhOs$-kS@3SSVs_5EkEAx= zBP{A1_8l5*v~}kFXTqw|)a1v49)nUl9+m3eH{5)W^u+tZwafM4hqwNgD(MmQ*|@oV zjHDUqk9@eKqQm^T{F`f%l!;>l+TU>X;$Z$8>D3=Ic`e|HHTbo|*S< zx}mlyCoes4R5*H1NTxG&nfopYtJbnKFC4fE9!TxjQlqri&IhCPwj6lyPdT#@-G&Izd#|79%5A+bDK_EoTJ7G=tz$@ze z3%SFG55MZ_&O7ZOuuNh8MfS^(;1!qu*timwaCyh=e>`}?!H&Nn%t<_ezxfJ#h=?R1 z zq>uI^M$A3hJw3tth*3KVEIh|d`kA z|9V;FG*a1Y4n{r<9fm`B6-sth34Apo%Injnuxl#APNlSuJWxPeGeh+0M!tc-mZ*A% z%u@zi2in%zZcC(#`ER~5+?Fj{w%49}7P5xlR%96ToTQe9fLs7&ut64u@O=lkA*7_F zq))G2z5Dd;-+#auJJ<)yUJ*+cfGK=MLG1I9#S1RFh+Uny1|@8aMvDXu4NPfI-*U?> zn~vDj@wZ%LYJtBK0fZZFxZ$OjUV=~2&Jqku*3UWbyrYjg3i@cq*=L_Kea1A}UG~<7 zeqB9JRRo8F2M@mX+G_{EXH5Tw8?L8Yke_S}DU}!Yl)!5nu57r)Uf&T%P;@zXFr1~Pap~v&T1fPYHH3u z|NJA4I3h~gQa`DoX|eE6=wEo@g>Sz3rdHq!NjWQ$S)l|3RD|Qg3oh7ox7~DahbqhA zRU<^4>Bi_Qs{&&EG*aYf4}WD-E)k36t9ui1EBa2=^`8_QZof>(XJv}YNe+19#i`m< zaiykHjgGm>RLJApKmPHLz4q9%uoJVKJWp%>6H3|q6KIt+bu>{$^ig)E6HrUl*G_D% zUsiU?$)~Ud{m`L96&$0;1vid>Un?7OH+<&LpTGCsd)HO#Lj#zTHoK##!!zh*5DBF( zA^;!MQMEJ%WBT0bKapM}XX~bdDk`B!lPvnd1&~Di%kuyfUV#)kNnZRK6)sgM>ZfNm zWqkR?7kArz_eUOiWZ1A_R=lN;#t!`|%^v%C9Ji(xb zwiGf)MoK0JR%m4kxirzk6WDTmw)ueVut)`HFOMSdv4gp41Y#r_8BzC$ZB6CWYz|0J z#8+wYbm#-j>G@nhz#v)M@~=RzLKgp6pSkB2{(n98>lDPAD@9zALQqJ1z7#&s=oy(2B7AFI-Rjk=Sa1J=9c&pRfTZ3{ zfBXC2*gfRpi!Q$Kq6^#N13TIFd)Q%z-F3&EOzvPd&>#N5CSDQ$5!}XnTl~ea9nJI+ zcP4Z9op;^;z=JutvIVcNlsPsA5rajWA?*Im1e?$@;3!NgY;44GPi#b{rSEk#TyWt9 zhaPeW@e)Yv-<%`P3iRTQe+yTD(QEDBRG@WIl}xOr-+kxZW4?WCq4s)|4Im&stTTGl zmPW@I^we=0-w%_jl`|lUj#(#nj35_q6WjZJxT!JdwD2teZuRDt+BpV+kR8kYA zJwmL%e+B)fkvpf^_;dNJbX7&TWLD~(hl1UX7m5AO%2_-$tX`KYogdDABQ6gO&5q#q_2-q<8st$Y^6jU24&c^nxi(p}q0i>8N0{ zo!|+4B6)HX<+jD<^_jvwHQqwWWAN zJiv&dn{2$%klXLNn+<);4M+o6iPl)(aL|DVY`WP9ue0Q3T=+@FWd(5U@_qgFH(q`1 z)u#FeUSD_b+2f?+zoQj5QBLK+Zn;zcL~T&gK^OBACr$d`gXu6npl{!O_TJme^$I2Q zMMBlIx@FH>LTtQupS~MyG=wjqjM#J}ZNi7MX5Deu-K1f) z%dlZX*_|k%sRtLMtP6@k%xXTR8R~sm(a2Y|AS~(sd(G98R{Ka&Qrc8&F4Tt62khIY zPgPN|6kbp93Z&-jEkX+1umbs7a$Zhx=gw{D8vu!ZIvgh@R%1%+_`pbJ%s{~GeJ z#}rZwyYZ%;2OmJ`iT_FZw^nPxph|ra_Iv+>51Q4BqC^#>o_OqW&eA70Zv{q5*o?nv zj>RadE&i&K2#DX$hriXdx#)J~>-r-ljOA-A#rQb8#=KK&=HK!6)QNpj28Y}D+M*Sh?u^@>}W+lDm0;Qpg4U;#rb!9C=UEh9b_jF7sd+3CG<%qxySm?O@jXh z*{=wRT1WIf);V>`Ubae4Niup${wx|C$1)d#fRn#8jmD3pl{L z;8O^RU6r(nXxB?JdS3nb1_z`!21rI0d@&bE8ZUxdO)*t*#OZ9gRf=k2yNLN@0P^}l zl&E65l8}1UgggC3Yu|fj{txF@yoAKWBuxrE_H;Tg%{MYLy^2O|R5X4_uO0KklBedK zUpZ?9{INd=`*#fOw!^?KJB{qMS8h(R$U_lUG`Fy1%z$t6f6B5yKQ;fn%BD4$2DTrk zrPSu&RyjQ(*wW@)#CTL&Pgp!kh*Pby$cl zyoywTNA7ta&6n#I{5t8MwJz2V5Q*fVj@(9S7ts(+yxz!>n_Y0h1xFrvq%G}^<1FX! zO-D?5ZwhnCBuAr<_J{?O$}2Cw!oYzS0asAtfT-h7DxwHq{>di~{o^6tBtSu2DpYU3 zJoZoHwpZm$f2O?u-sH)Xl|V_9R0M-y_=pizRaInB!>72^hcjm}c3}0DXB9gV4RO&Q z_2tIWx4_5wA2W8$amOBe`srt|HlAc0&cCiqKZRC0wQ}Xk7hZUgF=p5>*qtfL2$w8b zqO{0NRv`WLg}CcVuS#)WwC-{`lRQe1pRbi zzmwj4^SvqWD?$9u1Udj`)FU&7hRa=e2f=esKeKr0V#doN?yYKKu08wp(jy+k1=t{C zK*rcb-%5_?Q?vr49|4=&pLgDQ_uY3N>aD673FPbeST3CiCYCc}h!YMw6yrj9qEv+9 z#FbS89wc@>F_&@ZB6NHU#`ANT+nLFT?=qys6FrBUn#%=qA3 zsIgw7BX(_33||>!Z-0HT796OKbWe3NmRk9L4^;V)r;&MBYJp6+Fv;@&q_;p(bN!T< zB9%~BbFA)kh&XB6vtOn#FhDl-{rCAE!hQGO$4G~#_oBtKfa%b_9)9@YlTJOQVjbT_ zHbjC{-#)!hJM)ZVzWJ?9HTBM5 z`z=1?3`xGNaM_P9=hFsfo_VJF=t7Btq<7@bl5g0%XRkBPJcG6Ok3I1O+l-AKZSxm8 zuwNJu!*+HqxJ}w$8p=^yj{fk+GkHQY@RY>hCKU&z>z$ytFP=RjXeI=c99{ycNeQ>g z#*x*25yG68aodg~CE2Z;Ej;`nM9GaV@)dk>6u-|u|NPXc(@sC*w9fiyWpucFRT>;B zXUJSFHV&w*L7j~q^xSriP+V-k21F#^n35N(JMj?PQI~Gr_`t;Pe)n5bPwMq~+TXfsO0|Go!G+_C;RSE_&k*Q)&$)2gji%g$7xh5r{6k^WGIRgl}} z_Dgi_aKA?iV@5<%IeD2;d#9Jr4W_@CDP0jx{#$z62|;m>rk;a?(R+uLYtyBxgE?oAc&hit!H}(Bk3uOyjGBJ_GxqBOf?8_v%wXQ6 zpll^brF#wt_BmNmL_GJ~@Z*_bV`HXMNqXaLf?;D7UR9d;%T<}CdL}S8jToOAvUR$! ztLu+dSL`{Hb`}{_s$Kgfo$xX3#KM&*6Ruvf=GBKDp7G%a^>y{BT$a%^_3GBGOP4MS z7c87HV@6?d5gRcwG|wWAu&RnlUKoviF@|$dE^mC+CZFCz|gh!olDT>%s{J|5!K;fl$Hf7^^HwkN=k-o%%-11@FyG{ z`DX~o|A?0Kr+4q3ef##s#Hd%N!a`mVO`ATAhMKX({r1_XU*EnYrj4@C#22vBHjOMR zcyKaQoV6_#6_xAOajsysX8F1@RP?Yz4`Br#ZyQ8kkn5mcLKT1s)od3<^y=AT*znKb11Z8~CvJZw6}jFOEv-gwuYcVW^Y_mBdASRtb=XHa7X4%|Y^z7TIl zmqni_4wbCB3ba_hkz#<-=V*Ujf2e{HRuRI|`aiNV3+K$CsQ66c&tbn!wKY?A+e@u) zEHd>-@ILx>ksg*pUi<2KL@rsWh~zzNCRLIOI-hjnMrWOs)2%DjbPN!;yuI~rVa)j# zqhFBy{_)mZg#g7E;P^j=qOQFB@~f`7#=;{PNx3RCM#g32A)Nn7`-OBz^jqLBbC>>; z*zqBsI3YBv6XFv5RQrYPq|l`Og%VX%joQN>%6^bhaBNurnY69!mym4!qxM(Da;U+^ zU{o>PaAl6eBMpBMBsi7`F^<0?0YoaSg9L5u*Q31utO7~hhQ1f&^dlT#DPlnIt9KqT zAh?8o>>r%IN;Y-+RQuIn2M5^%yFuXfvg{I(3gB#vk`hTPu?tU31O=P%WQ#0PH6OmE z)E;dY#)c8))*cH8F;8j)A`NIk-@%3kDH%kF6w7&1ZKN4cTvj&i`GuFyD|?RRkPaZz zX35eyf6(T=4<6n38v~2CE-LIvF_c_hyL{rZ|5;bHD9q5$Z>lI?zNqZ2bneZpwAiTg zwp;f*cB_8hD9r8ZgoJoHJ!ZhsoA)~K)rG%$ed(_n8tc_%GFtSbs3>6YepLQdvpE{o zgV1h$96U|f_hvk&vzQfs_0?DZ;upVQZ6^hZz8&4Wc01sJ{U3SwVW&@+R8Tikq^F;G z`qa}-CD5{n8;eo-gs5chQ>l+W`iR{eA{L!4&wkcc?7sW%G^rxsR3hE68fX+>({^CS z8#GH4rB#)<1U9`I=OP;Nck<%U=vykK6=QZd|Gcw(|NW^yx%OJ#BL3jK@AFMKQY8E< zp(%&xO-}?6F5Lz2p*> zwpy7{0{z7JK+NeY0EV;d%k|%Q@Ej_9{rVHr6*#Jg{v!GYUOVIdZC=)D(nZCXIp!qP9m(O%Lc4A|TO1AdPtFEbDvu5p* z#Y z{72wgtmjpAOt4X&m_D~oBmVXBoNPPxOT^7L-SqeqPw+jI>uj6%z9=B3byuTSxYrUy#55{`>Q&bI-l^{^Tb= z`S|0H$(X5c_|auQ+HTu%JNP7XRD(+DuLV8;z5fCG4;(a*?;||(^wXn8Z6SeCm{mck zIh?Jw_=_F{V(@}shw&fKAATfOWmGYG2&blQA5bHQC>vR;uw>Pte>U<;A7uzn zJn=-Hm!dn1Z3ll+TPcZuW9a#;v(92d)?<%8%AA*d_u1D!)#UsuxtLHnJVsh+7G!j6 zW>5?J6*n#EQxVW`v~-3)V-flX9dyt=ci+Ws1xoX|+Fyz?ipHSfW`##Z5I`xE$+5|> zVTT`nxQcFRzY=Jc{ySo-ttz5wb0qw8&2W$NAH{jeq)<{wAy;KleT)bZ+5XL=#Iy8o0wa%I+!~t*lgEe=r&1fqZ>7 z)qQ|+*Hxz8f2yfsHF;B8?-P#I!sMW)Je>Jrxby=jpR?-;!GO)FzdV%=2Zo!7z5+JD zeoX-Q`%Fb;#qD?8!7jx-Jnp>1j!Ty=UA%a4pI&`-*<~m8iFoYs$0xlpk?o2H^zX;m z#M4hdGk5;H;UhL>fywJ{y#DrEEUA2(Z=X*%WWwlCqfzWwPBH$mj9eN_5#4o%o!M@B z-n{uc?65uGC?VaNHLH2xcU6~s$)alcg}ty*S2NYSckjLS+6yPcCdb>48;5;4(om*Q zr%%3){4*bmJz)R+8IxuA4R$M^`N51Cvt|YjjSM<49M9;(x^?Rq%iy304YOo2d~xii zn{UP{>l)Z0lzCl^e4bEy+%gxVsF-;!!9dtI0Z0h2WBN(7iN{%jH$HI7$7=}V^Da+3 z@dQ<5KJw8=9?8UB30fb-t=9$+kdp2A%O3T`y*ct~8jyjHzc5%sO6YT_?uL(1^0LJu zQE;YM>pxh8onms?`e}8D_2=JizfHBZ@9n-%EDOr}pr%1Q9+h8?P>3vXHl<_Lt{7=R2XVN z@H%Ddr)ojGW02K9&&Ee-DAb^HP8~y0wnO%1TxIKz<%;lWi+{WR`7_xsotEnADh5~! z^7#;-@e!Xv-tl*~Bp1U!X@B9|`_In5gTuA+pN)^|ZTP8zS@^WzzYTpQj{Q$t{6!O@ zXtEe?O+Ui+Gup2PJE9`70~oc+aCS_Hbz87|X`z)IHQe0BvXtL2L5oA*2gJ#mIE{?3 zWm1_icluGe0$b%t0j=M!XrskDF4an7TAoPf#>VQ$=3MaZihH>+7piF&>D)}uf+4#N zzH;nD*WhN6a^hzynLD{5*Rc|~#|Dt^cU$M*3?~+ET zReVsG-(}wsSB&a=_&?^Hw4!EqQ++iv-Wf$^mNI(FMJmlwK-@#vjRAK_Srp>eCI2iFC$q6>lmSS0HB#<8h0yP)rx|B?;xLf8zv5tdRj{rW= zX+RlCF-1A4FMJ3-#6!H$B zkrT&6izAIdGX;ZZNH56gh&|)u+s7OeF{oN0BZZ4v%s*x?%4k{+V?*kx$7N|;S)(CF zOYX%glF&C6WL0rLT=BrAs)<__N7H-RrI$VX>~nwq^PhY7>Ls>`FcS8QqLN>SQo~*p zWkr!tASg&od-WwLl_~PhYLMbpgUulqZ3rr*cI_AbkyoSx2ELIJ$bv%cw5o4`qOb29YFNeT zD_rlJrEI~?x7<=xTy*Vq*BgJ5h0xNkIDu{IgRZh#ip4c}5hSG&7+%uG?kN%0!XhisV%LLAFm82-q~@L@={>R^vj zE&la|WPE*rf_Q(92dp3pqeT(OBVBPEbqNm&F#cV>&cDe)iHU-1eqq@Bhxm_b#D!51 z^5VyTRbR&gCz1G1kuYrHNZ~`E#lLOv)j)jG)NI_$T@UMJtXXn(4Fr?K|K3 z4vP*?Jn=-t4Ux*~s*5kV_?4Glia0Z#93Nv!k|vA?szVPu?4}!UoH*&tv+C;j%8TP; zg{Tr6-u}YFH;+2zkw+dATUKv4lq~I#q574mS{1BnsEKZjGyO)1L0gHU>MKz7?%k{J zx#yk7rJOX`ErBENMUu%&fdMEuBkY4fv!(I$AmHKf{cpd&|Ni@4cG=}yZnc%S?@HVZT(NAT!LM+1UHz!XEQnuPJHSzvTO=ar!2iTFcsY{=**TB@4`-UqP zWme4#7tTmc{%d;BmZ|*0aQxvpZ{D9-I6dIgB<$q}Y9?YoB1~wyFLjlwtLv%+t4! zk0oW{HV3B$j}G%YMRW`$$47)=Q2zvipb#tkHtUb$ZEp{_jTy?6klVFiW?1a8$L{0Cj~mo~0IArO^Nu_3;w8%X?Y190WH6%u zSOAR$QnuF;bWll^;%m6CzcF#U?Y3jUf`JLv*Fk?h?Js%UzO0f$gBSgbl8C)j$Y{>9 z&pp>v*HF-@aG!nl>DI0L%9Sggd*MYs)KXJh%dR4@05a@R#Q459vN&{L&6RmFy&}J` zxR4gv%qN211*;IfW^RE$@2g;7U0K=c)vGJ3H4_=))it%fii?@Mv26LW7hig5!GZ-+ zV>~roYvy&^#07P@sfcxk=}^$)?_5;0#i-2>*zW)cLPY-%V9Wj$960_Gw3(d}o7AHS zDTZ-sXNpj`8>mSByX|-8@!x4&vl5ozK?g5ybLc3+*i`q&x=lU5@%2q^ zxWVhCTvkP@Az!aXI;y)aV*f)u?JlaX*ZZ^D?^ zLRUeG6aPv7Rv}Ui501_X0(5Z8y zF$-bRmc9~3euNrPjwyyR;=P586-3N?OYVDm!9`V#tMm=Wkgp=vvv*OS;!XD5`1;NJ zOpr5-=^HZ!8vN6)YxV^ldp+-H<8GUb!DAC;H8%2@#l~kBT{Lm|ul65u!r{=tj1MnctrhbBsSELNZaE;4#`pn|s;e{cU zT+qp7T1TFhPFgkAUMUz>-Cg+LW-?GU6ds?g9+t@?V8SKt2~@iK5%UI78X6wIz4+pb zd3A91_r4e59|YTNJC;{%d`f^~RasAG!sIdj>1Url;_FA4v?KXO_$#%L;q5dJ@L3P` zMuiiNN|(-i?zNW*Ux*lgux1I8m+Dcy_z``Qad3f}!HCF^O%J6J|0MO78yU#vYJFh^Ue*DmgNJuSRMlYhAMx10S*ixhas*IOpiYY#=46>5% zf%_j|R3+ln3`fszN~8Q-Ku9InEvxGp>~QK{g3(u;Qa00<5;=YGs5>VWU2tohOGgs@ zv9b(|&uhPs_+z5#5#7~vzve*LRo_wPUO?6c2uQexGaFy;un z?z-!K`O9CzhIaN9qoOEJB-%0lPE7-kR-%kt=|s)wc5tXUG$_U|yky&h@^kX;{_quAJ3JAVOO*=v3@wamyE6QsYRlA0V8tym z%)u)-_|OoRhGM0v^UBuHU&OPqISsSg7$qqrfp2_<2jqjbvA-8fRNl`Lk>PFZNAfkc zAxaXuGO|ihHa#J|YGprO3!?x{j>LW)E%6vjhf=tX3R7COUu0Dv4A)k(qbIxfO_*>f z!xqqw>|Sh2z9jXFD2qZwjX<_CWuFdSS`tRfF@A^fk3RZH_io)qKFJ0DDw`jxcpLgj zO5?}*jOde#!H(N*yN#&?>`+U=6DPj$m%sk?o8LUz@gV{AXxsjtaNO|>cDz4j3Lj`F zEbNrUf3y8kuCgabOZ`dcA93W@fBuWHd~G3yQ-bpG4uL3v2oNvAs$gPLm&YU*(>DpI zNJHJrR`R2cK4Q?of${@NX>wK0$P@`u5mq*5U6}p8@X`x^y8B*Lo-BM4qN?iee}Cxt z7hb&LiYrb(?Q}k!Nob-!j3gRuW7a6<#miV60g)9H@Mid|r~Nf%*nQ>n(@#e(?DeQ| zJQG^PBM%eO*@%CLV)y1S#Jt#0NuGffe;VnR^J!}0?KHbu5bvoN3ud_}v+KWc#Nm9H zgG^*5u_L5P8|c~m8~sQcl8G0^k00`%#(kv!LRB$F*zu3(XW_4M?XZG$?b22)`i}@l z@0FSLdWx+;E7s*abz|`LOIQXHZhb)R!l_Mb7Nu%RgOA<|=DZqgv8U+ocW&yzOMv}LNeM?#+j#@%}Gw<;Mk+4k%Fiw>20`;iQLaDaXP&2qN=T3FPbo?B|| z@|^dc%G6hDulwh zrW>kubaoA_AYcK@6Hh+5wyeCSx|-3DZO4w))1%p!D~n6V&#nBk^d{CH@a9;yd}7Ee8HVbM%!#N_QD@t#QwXj+IQuR z~h=N>ow>}T%!BCRxLTGLmZQzPUVsr_d*eEsQl*RyQR6xg_hn@qrr;@{rZ5dKu%y+Ue)MoJ*055&nO8J7@z6u$h%9|GeFISC ziH1mmyy&Yy&eAUWmkiOWzJ!%lEqZj;+4F0jtE+2dQnnS;z2C+AZhDnxQXV_F z@LE;Wt^A~N#)_)Syk)~<9A@%q}{1t(nO{%LY_!$`JbJoW<;q=X@}KNq+mVaKjP5LNMvIQ{6dvO zrCYT{T=)dMEM+b99e3PO#8&`}z9^$vDvf)i!4=t*+$j13eMB$f-wAp;k70i1efQm~ zYoj9Ks+|}SRS`4d+#6y1>Fck*w)4)rD8Q0gL`2`1AwZHMNLiFlBwhb;waS!*zoPI# z{8&VUhH9V?3*>Tx?bmYX@dg(Y+b^qaAAS5WVI%4Q9`*5521_1_irC4ONeZkYhB;Fc zii>dwgP;HWXD6R>a<=_ioN6e7rWjsP_@t}i6@;=N&`oquw!P*eH+?BZa~MWwreNVE z(iK-+zU7vqcHeCe!^!g+d_i#MU3Xq})m26?aYrT~M^3(;^}Wt_#J~7th!TU=^KImR zgud`L_Ecf+^|c`j7d{qcyh%A7G0q~U8Y#x)%WXQv)ABv{-o5XB`<{8`8B7c|L=k5S93UGH@e)EY88?*7Z}|tM#=fXKRL?f2FpX$DGG?2?8bv(CHsM zc+d?uTqokjR)UYAXh=|8TqwnZ50eYn9b@gu954&%o8Q!cyXs6R0iaUa2J6MoWvN|bN1D}>WdXXYuDmAl*Kn59k5?;DwsZtp9TWqn#OE105f+l~B zZ~mK0 zgf(RnNcX|%jmL#O`onl;$;|N8d+bGjZqrW3qo)TxBead+u#qm5C882&&C%(*HlDq?^M zqKw^B<06KpinWy+Qd8CJB+-bfom4i3JA5thAvr1erv!HB%C4+G@Pnk=~MDX~k`GnKMfxO4ZOjT<(6J8RZUFFpUun{RpKvB%1lesEiX5%J={^h|3b8>TG%&Io`J@5c?C)qODRG0sI{lKs5Df_Ux>c=1Pg_@Uf z-OEwTi}8kdV*bK~^B2y4@}K{F^zp}kdDAZ$7Btk@%o6JaxtN1z*wAZ7kN!~hu|LZ+ z8JB)I|5vVD@&0@7-h1DDci(savC1#=@%x{3*4%mXthInGV02&~HC4LBu))lkGdg!X z{=oKjR+PXiQdsy+<4;n!D%{Qy|3rM@=HZ{^-m(ASHf?4u z)>&ajL4(SjFKPF3!)o1K8>KP+z-J%Z$_dxBc8#hE$mtsTe|1N=jJZGqT@Ras&oQr5`Wme@ijFow|?il z%ktwpgF{6nv{{mfE5`N(eBQ1ZAI$8gATt3ki&-Q3ww-z6@hsMZVtyY@vT0{Z z9K$}YJQg&A+u#x8W-|}dGn%g4n4pNv>q&Gc9r-cannOT-=N)%U{c0LvSc&PBC!caM ziy-nXkKEk(>qqhIvBEPX8>J`hZ~P5{D1u=8mlO8eT3Hgpp&a#GOt32fYess}QRc!> z8b|m=>_XJ#P5n17FwTk>YHrf-v*Iihh7p3JGf2-gNPe8eoDBdbDY6-9-9Dp6jlSd( z#%(lk;}FDHtX-JfN98Bkb&dQ#D*7Td8oCbrKAUNCdcjHN2oUF9@16Zv4jw{z$9CA}N*@3fVAOzQffyIip5@c=69Ka#VsR8@gJE zgVM|?-lYyKIi?92!aGn}RCWIFP^d|7{8=DECSx>0c`RtWw7>S+ArmJ}YSX%PDmP$1 zapI&q{_qC@GhsTB!VJZwJ!X~iFU1cFQKkx)G!Q6Bv$KkeJVv)3VYrF6UkM<8@d1VC z#%&8z?)B$n%<^E>w_pGI_M#%*I9Y$#rRZ$@!|>Z#-%gr1dGJ+(%_=SsXmBMU;&1kG z(9YkElqGj92K8hd;b;aB9o6U?qg8zaFGy{F~kDEx!Oh!Ml5O`Ya+Aim}1n>!xg@i6{|_3->v5(jH- zjSOu8W$;Hxga<8#4IB2-i!Uu&qUQuQk6e?R)3kXD=nvzcvSenttPUNIXRu?=oVoq_ zosERU%I{(MD*z@*SpyeoEZ%Pa0Rsn~eCo+~bXImIm8-`1l%78yjNnPO94~eI_Umg& zz$g~2Teo60jqI%Q$_S~Wd?aJx&w7xgJ8P|?djsmy=>`#-G7&E=h_6h<|*#;UL6wQfigrz9@elA{;TynXtdc~i2F zi`&p{*;B97xOqu@2R*upQu6VDfZ z*2UPeF0poAxM5kacyie0q@ZR)P)VJ``2V=_D;4->?Qi(?yCvJ!$WdDMjCVaR%&HV) z!M07S6N@MFq^FNn`dlAx-bG!qO)KKxjnCM>1F^9RRf2)H1r;jGtmL8fZ?Tg*oK!yC zJ_<{>kD)Kc&lhbxfs>4vqmc5wMzMXD0Y<&%;u9(znGAc_&R;QJ6tgc0%Xwy(%PL=i zmC7rW&(5k@vxa0?{&FELllI{kiHGH{5Q$zDC%=YA9l;$nwPh5@gfEz2Aq3t#9snLWf(k7#NT{LY{UYNZ0vgAz?`{rzn=az zAF5<$Rj5&;`bT5O?%BC}$IcxL+@EpA>6Z=~#FTY5oLaS-rLk5p7<6ENes*>i4vww9 zojsc}d*DC;S=X^+M<#ce#s*Q4U3d8>K@om8LKs`KW?gP>?m;%r6D$f1r&#`hg7M?O zxZr2!Up{CM^afQw%=>ur&A0e=8>0-m?6PK!8cPFNlObItmoPDkig~*IiR7O#Gy9n( z1xK;-%7F#8s&7Pq|7TT zdtu?m(qb?ys_G=Nh+XK~7M^wf(K*E*TGbL9)(e+Vx=NAhd2`}@6M};;bN)1aGgSszK<8vLt|cKY+K37=jQafvg63(>I}dK#94`g=G9O7 z#R*ehUodcYVnd=>(^*PvV@JMG(dr=qBQaWXgCQi|3x;97@+QuZ0kb*O4dE>^87Jv7 zvOpirJ~AWlNFaxoyCYm4%Uq^M8MD*}iV0>*vYX+-L#&({@YU$rHETwW8pWq2A{JJq zMPti3ZWXwA$eOzncki>FH zd^p~C<8{mFs34HaQP!+clWAarZzUovf2Fi&?9T{XXk-my#L$@X3tc*YUK+gp_M2y% z(I?I{;4sF^guT1>@~y;YW5#|kVmR|tBHtHWr8Nf5KV*s4L4yWKVwpqy1VEn#u3ks|@1buwmD;1?iJdJ}GM95L2^oY58Sx$7Ppa!C0Rm5mn@$ z`UY3eqNF0ZP{G?+a7`$dk{ZDYyEcU_Y1Wa0{O+q*M-_iKXU_decTsZ#2{L^JQiQ|_lW_0OT_Avao z2%jd!>Henpd;UiDVV#kK2VZ;b)vbA3SxfJ_q1Wjv9cO9wIh?-?5FPMmvcNmsLP*RezUX74;ea?{j(6k2kMh1OuqicmZ|)8{t_+u zCLo?@)uK7XD_5@MEQyqu|n%Q~hxMl0?bOywC%`XU|^4 zhmUyo{de!W<4*SLB})EfNeQ)7ixlcGSvv1$8nE#(R8~5Y z(H1VBLEmHBMa@j>DZi&JW?6Kj}B#!{WwBsNMEsHN1)VmN6js2gjp5pZv$pVEEdMaa)WG;4T z{KI<96YM+t{_lBTp0QwD(EWmVm71~aDj980EzVsSTQ)VmXKQTYGjUe;poVGDJ=V5I zeAl{gZ(bsIZhX$@@N9ixn8sU|#{b`!U%)A?r7oWGMr`9sR;1*c;Gj*9Seuj3Aiir; z;*&>X?8eHc3UykAEl*(92|9&2v%}dRsq`xT?I_*EOUnR&UH!)_7V=|M7rzXDDpqs}^F2#pRR;*mk6eL!IyyTKWyc=M( z49byOh0UMW28fh z%cwFm{5iPwYXV#sdl+ZO?t{ygFJtgRw!^CN5@fNw^sH~cWs($Yk(_mQ-^TSDkjn8Z zn^;g!kBXVuj0a~i|2pzLi#ojiPp{wP0}duiqNGRK9}-vz|Bq{b4O^-Im;@;wg(vNg z`mTjV9bP}Xe|q`N8!>9#qmLOeC#y~{8qG22$^2v^07A9UT5zRe$2Z@o(!HDWPvQSR zXn#>jMht_`$BxyPhFl9oph1m9P`6I)h7B8R+O%1A4@1T>{`~O6huO#NPxQ8JTdx^> zm3RG&M)fDD=-KI3Juo@5-6b|}-h9vR?|9@Nk4FO2%I}@O`<-v0s6?d4T&cXEbLk!c zAN;(a@U)XpzU!WQ{`BC3ZhvJB)aJKCue$oZ_un%H)!OsTn*|XaWCD#1G%btF{U+I{vgoFVTH~#wY!}r{G zpW8N7e)(dK#ehBY<^Mt0yLa#Zy73pyS~h25`>vfk*}m_Af5SvjkF4na_cDG!Xu2n9%w4}#C7^3@M2y?xJfp7kqQ5h#9u~ZLI51Mt1og^=`bN?Guvis0`NX?Gwl6 z+_Rxzo;F*ugQ*(j>;aW=f6v|&en~NmaA;by_f5UNotXRQ85^ErJrK+WDfYM$>JqL zh74J-@H_dQOENvZCCT5ncj~lhXZ7XnoEY=%R=tJ|_O8@mM=EC}F3Lz*ZQ*@f5g$d#}9q#_LR4p)D78Ka9U64VErj_Qm)w z`u9KA8gGHrAz>&K@%N|IC`!g~BD*p@32>ZTFpuEVv13eKmq48C#Ei|}J$v#Q_T6{i z!%~#e)%a`FE|K8FpOGU+uy~5`QKSyxbQ6ld6><-b#$d$P(dcaAEa*u8ab|XQO7ThY z2T2OQ)1;T*h%${o>t<^bkV8`)ZVihvO)9a&$gmr(C)E{K4SxQG7wG*s%A`3=ZV<3F z&Ntu8;>#&0iE{hPCf4>yr~*MNn5nbp5UxN4pcYK7pcc*Pq#L@}leJ$;sH_;{tEDS0 zyZrOdKd(@+V*dX8>u(sgV#RW4orWJNX75NbW_qgfLYDOtBmAU+zvM}j+sza-{{+84 z*$?yQ6(Jew}V6{#|?d5m=CNCaO^|7yvi z1zoyz*NvFV28!Ykk5d0pfW?W%$Y3^OYwlwU#=Us@wGEhOPQ3U_iUgIhf5i4M{xE`$ z{v)6msb%UK+XgU<3?FK@tbi?S+1`caOIh++Z#A92`zNf@^0ySWp!t!+k0DI1@N;kt zC&@_KpWD*%djx)@ro&G0-?(}6m`}&R@YlEBZr|34zc@poKiS_BdXP*a03i!r9%ARw zY15`c?d7+5v*uq-o67basH9{{XCPDdR~-m%I}8E*l^6oWC>>H|8I2igu*{ht5=M08 z>5*8N2Crw#nt9c=SJPWpa)EXQ4W1>#J?hr03;p)(JDihi=JqG#u>3VOBX}wMnDTDw zfW0A}d;a-1-+Gf3F`j+)Sz$FEZXcyIoW_DA-hNq$5S}Ci{RV}F1%J5n4@|RX*mv0V z*R#^+k@(@0&fm-_?ELXjK*fq$g@mrv)q}4(?bK5r{L=#?Mvi3N0hqhJbq>R&E0%BH zww*-mh#@1HbO8>-^4fj8)PK_XbA4p}c~tu#!}hR@I18SvC50`kb#J#4gg7D%nAP~K z+N*ImCzt%FelVbr=>Vz{h>HQxg_xQOhCd4D$75upBm7^HQBtQjsfN=sO`PEAC_UJq z6Hvw~Vb=ix`>kYU zhX?m1)+|V@{VpEUcP$y`-yYOsW?z_C{cXvji5V;>kMdRP#!kP=a|w4cle;CAFA@Kw z{ma-#fo1HIvVT!w5q$vb)u?2zShbQJK`^8nR^hzqDlTF2xq8L%#I&!z;)@mLIM7Dm z5->%l%9X41Kf9l0ixLWn<-FUU08ypY&*mNq*|K%>H{Z;pNoG6I28|jrxbN$mTK+1P z*jfAKmLr@AS^EzZEMKu~>z1wTz1{w}_SThx?r80=V^L&eAC1w_qs_?R-I@~6a|5la z!4lTts8p#kyR?9xFfyl71?~I57A$-A?&1IBDU%m|w~%jks#UArt49ysv^iJfZv0g> z%66oWz}q!O#Qw`xtX#K#117CouU?%xwU;be%BV-bzWp|C+04#2Oi$joYxnmU-*%iu zc{XP`P((j4aqv(PPrOV|W;&JGHmFjmQoZ{1(M6sQR{fQ&|CQhFDUtC3nQFg8t_-qt zb|DqSg_f_`mkAf9V;) zB-W}?qh0&dEtPe}Q{Sw6V(vdit8VzV|o%4La zKMwpEv)fdq60>CSA{HVU_xa}snQdWZnNI0BXPvcR=@NX{(?a>dJMYezF~i*q`>?7p zb3CuT{`y+AYB8@cKY#!9ucp8F+%pRn@a0a_+R_>hx$Zi)goBAEy!7W6EnBvnYgIi# zKv8+~op&i~TuW7SNhHqd*YEdt+;RQTq3^!)4st1W4g&m9@iYHV;ZNCL@m^e#HgR)F z1+IRu{HE7$&TH7EV2J#`PJggXzRKqIbsL-zcH057=Yng zw{}RD!G=g}xO(~uQ4(rKCo&Q@hb|2w1PEe~+xRS=Aw&s7j9u| zxgadiNCAD1taz{bHxF#{S2LPnXZBjjo==zkX8w-1N(zBOi^Bd=dktJ14my{lT( z=vN%yKXv_si}IQuS9?I4+Gk0ciDkDo@B=x!T;=oHKGL94*O9BQj~_Y+K0NQ>!QxRX zZ>e0aPP;k-QrNhtTeFCkGyp-2MF$1GZ+EK}6N5!1dyhfl_g>Dr|W z?*^48gsQi#DGM=1)jO6<_<{#=y7SbxaMWJRc z7MnbI5;L!iD@o0kMnKY2Pd$~@r##I4M`Ghzr;T5zEK2+%N&cH`FMQ-sI+BPLX!WGv z{K2F{T9^8dY{_OEH{Wzqj~*uu7%+gf`Xc@inL%ie;qcn4|2nrn-}f2^K@d;LguV0< zGE-Xt+|n;q>ZoW5jG<6+jwMT$%$+@3S4t;Jnf=hbRa#5DQQvPuH zl>%w`3;mSIlh>|U3quJ}y(mct@bY{9#TT||)0%4*JnP-N*OaN#?3pgSgZKf@YGU?_1C}s{qNZ> z_u`9w{>{v9=-H4g9X~`#BortmQV8G0$h)O&bi{e6$E2zRy?)@9SXEv!y4n;TS~c6E z)IP>RsZ_1;ivybg-MIae+({!cq#+;ve}3>mKDOdnSp7TWFZMPM#maB|qfb9&u*29Y zq0-qedW%JpuSLO8vBH}O=51Ls;M2dKP`fD;I=S`o>xsFQD0n$P*+HCOlI}msc0OZC z#P`NavUzI;L^`z$bXX=+fl;$jTF5i)YzBw4{1vw(f5*m|BU%5Kwl5eKVgNV>r4!o7 z{&3}=k;p!v^pnU`&2lES2bSVT(7W%xvv%#8Zrx6}?)vLQkvw1_93(NB@R0=lm-2j| z$0{(-J@*2=4Fv)t3?@yQG;P{661iv%^(eJ`QsQMq(CW;gG^#@c&{8AImEcxF((_kz zRCNAzo-+;Ei|0EkYI-~-gN|-c=kE{&HifH;RAmgtAU;1!x&wqhv}*lm#$eN@x~i( zee%gCg#7*Qk1?Zo*svR~9eRzgPA`eFp`ei<)&9zpbtLxg&%foSoB1B?rkii6RjVd6 z&6+fO;iVTZyy&8P@4a`~vSqHR(%a1Ev-+OJ1uxryXkp?)J;nF@hQGA@>Q;EwpmD+F zQQKc4EQ?is;Rr4$z5V6U?zGcRqd)4+iUh$=t*p2)EW#Ku%SMt5+*@jo2C~yC3q)#J z!!-beMv*!!N;KlD+sHlyxYnusg_2ldgO~HO{B;Q67fL=mp|0Ny|$zb0elte|$|c*R;_vuZ3axi^eyH*Uc*_536l7z9h12`o@XaQt>5{|pWmBqlr?-?5%& z*swy?M7!RxMjd2;jjO^5&&3YuwNkv<39;tg6Xp4)sW@2mb;jy%l)daqd(O>4`Dzku z2O?7ZBb&I?Bmut`U|3~f9WuKhTHeKC4DM6tFa-zCl=EiK*^|G|wKH;_`SlT#GRXSE zix({^p@O88Tb)38f-lQ3ozt%$P9_I9B7cpKxdCx$_Ti1rnw&K}`QU-{-!8xWvPzX~ zC#ED)2f&w6N{kZx#q{vgrgf`rTeq^>1yl5y|9H`b7YbgQ|6mr<%BqLg57N9&y}At= zH{xA)jq26u;qZ|NpCQz&S+i>8DokdnRIx(j#mxAtSb?wG*aBwP&fV|5_Z~0CiVq&F zQmx9#y?dW}$|)LaFhLYi15n2Q9lV`YnweNSclMmM>(*AQQki8G=(X}P572Zc&O4|7 ziWMt6b?!80?z}nQep?*pn^#h(pUZd$13`QD?%@l#7+=#d3XiBT(Yj^JYp=crqd_Lc z4F`tz>~?c0kbC`bSW(ce^A;&-6(LzhK!|yUVfH8|b7TH5m*0Zi+~pSzl0-S4zLhtJ zvS(mE<6tHMjWLkue3@%jIle>ZPd~=yN!X@!Y5uDHJ-Bh>n#cZ;_t_Y2D3AR#7H{nY z{Nqee4;ugWH;r$*37MX6|3Ud>zWm3ZequILWAa}&-ISfhs?Xy0&|m-d_s1XCGmgzK zxax{4<}X=-am~~^nH__D`}a+sK3zt(2y%J$yt!@L9jCic0|bMwzWS2OFKbw*E{&$_ zqaJHW=aX)g>8|qo$>@*nVB(~hs#2j;4IDUN_~;LrDev}q<@MLOU{dpq*Ihq+#0QeC z3=bZ-|4#$XJFiox&L%wAvU&66gDxRl3fS|--~ajeEw}xOSXC+3{~@+R$4eckm# z|Mu6v{pr5@P)g5%iNs6KKljH6A9DNfE!h`ej3XE~2SCe~EmvsJN7q1dB~;kVc(1+j z1_k)J=bjV2g8-PBix9DPC3USUB{S@nT?$QJ{QL4NGpF^Rb<3ZXcrLj7x9zVG7@i;M zKg(Fw_#6KpF2DFQq?W7F#hL5Z!rWF0P31jt!cECw7O`e~`ysPNb(xEop=C#Cy_VPI zG58aq^vMIlCEGtAz3jRJg4wupjrdr!iPaGQckVHLeNo)C>?Bm9(Cdc^r=skYqCa;90i&n9l(4}L5{1`ZbImhP zJuNYiij(saKc-#fuE}lNrmaGg_VN53HF6{gQvBgVe}!8JS)AN2B2eP%T^He3)hxwa zl3hn`rL;OMh8t`W@#luxoAQ=ScO-wJPCT*4y-YZ}?_OTXx=gW>EG3^bX$s%j($0w| z;bI0Xmlly3d=VePU&LZ#thj!fuO$^GVrRuIGrG^|e@?ZkRpHNe8?U_b3h`Wv8-mO} z3l=O`x@;+*`9z6H?)XytEy80XX=Wdjq~Hj@v~d4%eS|ud^a%VGC2dn|q&Rn_<+p79 zBIb3g_azZR6*5k8Hb**S$hC?w{A_7FW$IKEO|EDs5^fb6CO-b;5rqUzg4|eE2{`iwW-v7rGe`3YU4P{Jq&tK`GyBS+^F8u7D z`%8)#AvR*=xYEjNewzQ#52|QZ=2LZbPN~+;GF}Idiy$x40oBo}dEZa1o+3AMlF^oRBCY#Z68uLc?f!MCSha`*=yun5At@4d&y(r5MQ110MmTyVh!)SSQh&2Jcs(E@}) zFzC`tZ@l3K6QlCTpRQ$Z&a52C45TvU7xg3ifa}=pukyoKxP|;i>gsKL<$pf$&kpT7 z5RWjGUxN;f+xE)|Uk>1FaB1h>TgE=lS*WaD5?qqo!OxN9SM=%jPs`sVen;R)os;s! zt|HfqEF$>Pr95{sx|FNH>KlpU&kPn%Wzs@0_v5f(+eF7R6ZKog zx?jNbjo7Yj8T6?7XIY+0Gu{y}2@j$p;~V9mmyFXnn*61M8i-d6E_+hVJ~O>9_RlELiVa>1bc5>@$DB@7l$ zj4%BvD9Y!1yI9?(v2Fu|KdGQ&IMO~xx4)=mw#aZv{0zra74RHgsd8l&rRaX*3A`5K zb3up&K!=`E^0sf!$;~0CUO3|&=r_;iZeSD?uTWSLHy@58}rf;F$Yy46Du3ZiNo zH=A^H6CPNxYVFg{?43CyW5-TrWui3?Et;YkB{2SaqzU5L8m2V6E3I_b&5tg>L2&(b zL-iD-Vy?xUSS=#m{+C>O>F5z7e7s}T_p3K=+GKMUoWH>d61>GEy&T;RmW(;BeFxR~ z%0^QS?3r0N-E`C69({y2S2V5&)N%Dic zy5^b-&;OaT$2fh@=##f&r-=$)efi}m1%OI|=U#eY*svSis;T@<*#})V@7UJ4L&v=jfS;M z0JWr?j6*4P78h7+Ss+tRlJqCX<(GFvijHCx!=Xnzg;*I&awA*@S4mMq-A1Pr10odW z{1>i&KOV!y%W05s4P&b7tS_##0-=KLoW=^lv5E6 z3loV-JPvYYL+<1XgB~b>J(NQt5nJc4Tj^$hMxb~V6df4J5QtV_I4eeh(CYW{%dg-O zl*wHRf{EE&k^(JcO~WiUVia`i*1g?v?f52{e768`#}Rp^qtaQmVkMKrDC#x>2Om{0 zyLRuIK79rao=Gpmzge>;%a$x%@cg`iW;nJg6q!GId6!l>{l*7-}W@XHXU znV(H%hbtw-4gVZ=(8plsK^3=Tz!`-`g@dn_e-aQSMT4=qIIL_BHT19i%)aSCSm*b?YngD!t716zaUyR@up(m zC!c=OyZ1@VWrY>MOrER?5)p~k^wAH&3R95T$72*Adh;Km?1+$(L;q3vHI{ZVZnM6b z_03GGU`jU+wz9B;Krk8Ex@GH=PU=mpfCV92igfJQVd>IkYHyWS%0atC!PwfhYrh;n ze%P?<&4ki49luzMNn}XN4$^9%0q!`6qLhbvyT!7Y+}(Q1ZS&^On>c}Oj};d^GMa1R zEWFa;wcE;-t5vS;O9A&ERaVkL{ZXpf+itssb=1a=`HVoJHC@1O(&Yb!Qoy`^lU~rc zzlxg4)W%=>>Q7l|Y1{#n(s>mKW>WEKr=Om?CPz)Qc`Xwig&ha9 z+{$kh;P=U!H2Y(RQvRv-#Sz2Mq?Gah`Sa%EmUZjaIqy84Kgm;B!DN;y7J-2`Uj^rF zeeKoPAOGj$J9q4o_nF(vV*w+~B-IVODYCAl+%n8y1wQ^)U>{U1KUkr9^=f<{J7MAk z7FSm-qy!Gk1(RC;FfEOmVJ7n;YnaAKAtCRw^1+g@8Cnd^LUg%P+me242{|wEnj^>vBf; z&8uZq&ro0#78Tuo`)}Ao*VGhq{Fh|~uDJX%ADWS9Ltt^y=z~KC9r<gf3z@M zwQAMC^9BqVa^3HK_uDpY+K{Mr?OLzC_9|<6uyYM>^B(&1L-4}RD8<8Nq@G$8rmkd? zC5TG%AH_$)Q}w6eKBE0Kx@3N+t7pZ>907$?lZb+}=5^x1y#j4kn&KGyQ~QX0%-&8; z3e`VW-^eWH3c7~@=fldc(ir|y`=|N;Pc6Tx`UjXM&8n<4{K5$a%QoaVf5+hJ2ANsm z1$V~QFAn)abHkEg#n)kj+j3@5)4b~r0&UX}2$-6=uN{LIa( z;=8vbCcY51dgZ5`iNEvJ9#lLz7gkeU7c$dDh zMof&4B@Pw_OD6|wm(mafjoO4~-k8YJ>VvYctZaG3y@jaK##hX$YGM>>l#vybAR@JV zR32%XfJQnf`>YAU{>1AyZ0yw0C*DIWD_ZNJ?pH>UJBq- zvOn9M^p}x>d`@O&V|BveB@Rg#I311yS3_n>204YB&y( z_MJge=Bcsk2laBx7Og5X?IoIgm#9;xo|kml$MB;jmr|B{1|@yR&t0TI6WJQrHvP>{=^9KW$J;<3_c%+wQQ7D|D^m=nPoPV-JnS8f4X5< zU+?p=d`T)o^6AP;E_3^f+U$SzHP?(BKHLCm`?JqJ8+*gB8=+AW{S$fh>eaadud2;N zxMSC@sZ*!0gx_KQbIR$b58Jj~4+CM0)n3>PNDABEh~XpcBNwwz5PSIHzp!tC3DwE7 zklz&kbI%(vdD>SaKX~7YRzR6&N7&9?X^w-Zo_Y$dv@?vVwCmXMh8u1)*NOZ`+{%_; zKHwTYYE++7Pore=$icweZ*KqfORxM3U`&wbT?G(nVd~VY?>pcsPo=1i^!E3`hyKi} zCG_R^w`t>MG6kA~!lKVV*R&2-^r>f`Y0MVXR-BCkCz(*yn>Axb7(V{QKhN&lSFUe0 zg-W5)+bIn{0R$bZeLS-M6t&B9H2>rF*VsrKuiwO-!(2^yX8xmSv{azP`|q*Juc}~c zXjKu*%8~C9tjga4OomRzSp}gqMZdC@c@I+y=U#B^T4(1^U|If7TgtzGyFV_Q z^=X)~D^bkQ2G2Qsj#jRy_`sn<2N-^k&+~Ti(7p+|cONV%NcKiDD`aL4Z2d&f#@D6u z$1Ma(&+|;dU?q)sJhSNag%{k?^P9@qwE<_Sg>iM>)Zr^G-CMj@GcwelFUia}uTqVIygf@^`EJ0$aDTkGBH%Dbz_bqSx?-Lc6tNyW@|aZ@#OZZqv)CEyPGBS1 zPImR`!^~CIK0?kBy{D4F9G+Q_C=~;l$@ti%UkSh4Kf8QZpELV>{PD-0(9QzS-+_D# zcu@S4_P_k{!4Ev}C*2BJ>PYb2I3EQJ7&w4-#rYVL#O!Zp@7lf71s%r!$}0y;v*;+z zUy3tn!5_s*j`@~?;8YgB#X8!gTaefP8WiX}yqPa703CkBEn#Q`e)8ce^B3a0SOaGcCQB8^J! ze^`EvjRdi)IAcPS^DI>0Rv{v{6^+e_8EcO z%S_@(!R*{HC(lEUA95ajAAR((%P(VoqwJRCPqnW)O_tVS+UdeWJWIqcvWaBKjR=+^bEH*s(c7)Yr!{%B2~l4n1@7>eM)i7T^z zia%r$tEAzayfv49@%-+BJv=pL#d+`|#_ArY2Bpb}lrj6m@u=l1N}-WME!crr&J_`3 z)7UhRE*@4<7Es?=v=}A%bFD+5_&F;7RQZi44ZqX?vl5w!?ZiFdAa@&!(9(!T&c~q; zsYdc3@%d*O%sS&=Hs)EB#9w>$RaT#2_OGrDx? zJb3V6`s?%pSt%f^9J9D9vdIGbu+mq$_14>Xcba12P)TSk4R5521*KgZjZp~_g2FGx zjT<_2h-)0hT9g!oFrLWMVq*c*4WzD&;5gCMn42{$A@Qzr8T+LCr&;sn&ph+&&+J8( z*!@KXgmtFVFo-hJ_ z3Lh5=cK*EeWw+ODdD|q>B3R&%?*S)Io_O1@f6ccoq~$a3TW-0T>FYbT<+X0r3Rd=N znU4giLoAt+(nk!*7elFA3AHoAB;Ec2+67BglEBDBnk>JVG=0D~!D1pw&~Pr%#KbxxF5NyBW+-I7$kdWxJWKKa#PVw#q@;6jewGmgI9lPi zYr!N+_K6OG(xY&r_8G_bk1rTkvUhuI!MLzq>+qB-!WxYeT`!2w9UI%ZAuQgPm^C7- zS}RefWq90aiM*V|XHNzN`9aRYVDXfo!|A5ck1fB_K3*&)^TslkmBA$2e#3%B?HFeW z^47$c&5Z3_M>j2_YRz!qA2j`!F5Wl8gH?0FlA@Aw*|Da_2OZ8Z4GcfgQ7b7m%%`| z!@UV@)4B~)E!M4Hf3UcC%a+a4XUt@YF{Zn^e|mb(%6cU*eM}2I>%v#{)+^bw ze#3^fYt~_hZ9BFd-=0n1PcL6To0%-%Em|~t&g^YW=umndp<{i{IK4y1_Ph!@Safjj z-hJCPZyod5SXlQu@dP!=_O{9NOyalp(Sfs0N1r~YZ{N0~QKLqBfR$gGF7BH=r^Uja zC!Wx&#|bH0u`=4fUVlxCp;oEV@2s<$HEU`Lk;;sr-)NXm_#{&yb|oHmt-HwoEmf-g z9<}}YzbU^9&%5x@o?S8&FflG3XDzZEjxTt$Vn+#GG{VZo!!~ceTJsFbTIxUzsd{k7 zj>Us6EnL5ziG}Vs_^En2vIQjepA&J6OW4Q6jDIk#&dq>bun(3oW=i0Bmc)bvg&gKT zvuDrUvq#GUT4@M}4!b@|rSx1T`k#BQ0gdqf=2yRF_(1d$Zau>xu59vhPXGSK*7|Y^i?a0YKm6Xi2z2pIIsLTX-Fb&`iulW> zQmst?nZ@SVuu`@YA3qXo?#9eVZuD*^48~8Iz;$GwH2+axX}Uzu-o3B7`s%mn;fYP+ zjn`g(k?A3cFef(`PLRT?ef#4atqf@C&CV&93FiMQHDje=z=EdU=V8x81zImEyL|aU zgEX_=bUwWNMuD<=*sHhRRtg}Zkf8s<8yU+{rrAHKOPVD}VBL0af2D;EF;Gd>rH2W> zNz$2Uq(f_hy)YA=vT-SYamPtTq7-IfqdZ#w*(8<0Pa5t$S$g?putPGa%LLCc>sGW1 zkmaolZa1cn`px{~VB8=ztIhPY1P0C0Ax;Nuy|Pd77klHMd~M?VJW%>ouh6JV{R`%8 zeT$h-3~Mm-pqW1933gOcT$*^R4ox5!fdLO|0( zE|7>~9A|@`jQrS!kCxpuxXTANa0Nj)Yu!s@a&G6flGh_@JTt=DnQbrc{04JLyhZbj z%*$W##`y43@AxghM5vTOK=S}A+1%JzsQ9o(j$XSNQ=NkF zqKkg^zyp7@at;rIIJw+_Z`9c}oi#WJNb%>bzrKrezWgNllaTVPN^QN%dOUu?unQuL=4Lr(L-4v|=iGATN-?2y8e2iHg1 zN0F8zN?@!xX;3@q|5$_1P@#gr4=c1>CT_Vc+NpsX77Xv(x9`7RdU@AgrY4CPS#aS$ zA9eF7DGy6#YnjFUA5C2w<1LB$=lN?{pR&J%Sp{IYyn0e1I4D^Xn$cT88h%3#UF1Ko zN|s-H%s2bV~Nsiz{r1SO67I1Hq_8+edqxu(SzD+TX!bXddI>-z0IZ{h4)#mSUR=zpY_@4Qgv!BtQbL{)v+Qe z{#X(b#z3(t|9Ah9YD()eBl}1EQ0tE7Z2oFEdd>~9u!>(D(#uyYXrW2FGg z-xPl&FWk2N!g0^!9oSQ1@3r~N1bU!yK&=FJh=r8rfS3Q8U{f=lKLk`;?8YJHA%|ar>y@gg&odcY#qO+ zPL^L5x#S%Y8Qr_ri4`iAHyWo<3L!H#_yk+G{`%LqvAnT^F$eL6RCKBGOBS;z z!}s6koHy{iOD?|nAVNo_f zVY$OE43{ro#`|GDZ}wXl7-tA0`)t^-0amqY)y5=NpQLj*-2O0Bdcn|PnF)PV|D4nR z+*@zC)!tb-(vbZo?z`{4(I1Xx@`~HoD@W`SL3KtV7gqwe@7Vs>-ydW73_{F|iuuj$ zw|42;8A<4?a2>+$P(bDNKX^X)U^upw$qa(q2cbDRxwqf;tH~25F$mkVNmIn0e%fg; zG*(vbal(Q^RPqDPj2^Nx1{dTPFOZ@2`ZJ=iv@(Ht3w&h2GVQO7^rUrt`IQ5hlk=l+P!S^gm?e)azft}gB~P~2zl!ja z){;wg$L~fkP68zXW(UU~NfWI_Nw_HdhpQv&f9Efpmd5xCs*>?{&VL7M=g0DQh3vPo za(d9}gpkEY>og3{92S1~U`cUCY}QAKwkHME8wQnX#M<;KY271SJ}20}J@M(IVe6h@ zR{50ZA1=Q#7Fun~jZJwaUbvTGu6X6T86Em+^$>t8pBgS0qcU2lW^ls!LH%Y@{$OG3 zyDt;l*C^ZD^a@WJ9AxmGh67##FZ^aHeyzO74}WU=OZ&KT#R^zeu28X6t5%aHPvPx% zdHd|&KcOwJ;R6YxM|zNjGsAdRcKMXad56R!I%9u$K5ZG5ywwE%zWh%6Y8q1-5y1+e z&6+kn;lv(NEQ#L)%FDHg6`?3>zLv^gY`};EGo(&D@dVyi6%-YH@%gweojTbX7Rp{X zla~lMr2Gf_94I)jHE;Vm7V%oUcFUG6+qZACrq1gDEmyyO!}_Jmm(80ypC~@iV~j%x zN?5BQ&bo<>8a8Cz`uzn5_Uzk7jxnjNe%*S!Tfi&P09wwDA5&N^*&P~$VapaRTeWCS z8QHOOmsfGZYH0BvYrj--DaU9^0yq&*0aN>~PwZIt4fdin7jQ+D@A~KUgFK+Le2Kdq z=AT|)K?@`UY0~XltjyAm)^Rz#@wd4DmGZmvhW{$e&6Q>#1-9xku5^H#)o@gfiHQf5 zf>`s{U#WijX+#-*nV17`-`{)xmiOP`0bQLGjbYdjma0dy505$+(99y+VNjmBXy~D3 zN|&`SGOQgYIpQx2z|qN~lkKkzZV7H+zgrjqK_rPAue&bUe#utQP+QZ+6&}(=cnFG$ z3P1knqYEy)5bIk0LIq7Q;=|F+8a1N$m~^#|V_18<(WkX*)ok6S?eU$v@Di@?*=I|r z<%?8F*8e+q?%0acN^fDf^;YYvE8ntOSAGBhKmbWZK~#i6KR>$vJZi)U10X4k9@eU@ zPv_{iu{X2h2Xpx>HonV3>AY=uIp60%$*5+JUOgG?kSLSLiZgB7wzUsq^@s>Ija>rT z5Sr~IC%yc#60m|qA{MGLVj)ZRTEogcbC;M$RYx@9;;dA^K@NR-`^%s_AHgkNyci=v zEd9-JDY*)a45Q>krniqN{_y?b^7}t*e?jph`_C#9QbOwZ;rOMyqGT}sC@N-~QaaT= z6a`Nst<|cCkrdXiIj@B?fQb*YQ4f$ouIWS%kHSu)HZb0{Gp*k3drBfhEyXlk0bXdZ{N*J^DB}o*0`;ep?$Iutze3FpZoRHuyLyPT zf|7);jIF&Vp5lPfT#l%Xzqs({v=W}OPsCr5;iyrgc(KE!r*i%%+Ok!v^9BsGz{vhR zd-ZJGtm&>ri&B9?h~yx_cH)Dt94!9n{%`!v6cEBh&(Sg_{|uq&C5T6sFIPbPqvM$P z)5}-Nf70y_|F_cdN)^k~aZJZ=Mv`gZ!IHyL2T1xk z!avP=(BoDiTwrpD#kLgJp&NTH(>rnq-(N%km-d&FwFG&b%e&hO!-A9{dcD2r``Vd3#31%*m$2&=Cs-MHdGqFvKKdvl9hL+^@(wJi{IaS=80477Pa&?U-P?^Pv{Z&O z4A-t(w|V2n<}F$n6X8!Uzc5&31Gv3=MPKJR)E~CLNs+w6>K|(2-~RTu6DLk&^=0)5 z1&^hUCcg378*K80XaQ5#*X<&mB4wzl6exJlJ^MThU~{23JD+~)skh$tD{gH>!#YeN zl=`rMr?+3?9R$O!A6lf}+9Yl-=>M;4h&k?=W2|4Q^a#NP1>I=6;jypv*$6@-VEUrG0TF(jmP zFh`cJANHSk&Bn3LXUF&M3iobGte6?A-#Tb;dr+}vtjC3kuigst)-V%2G552ucJo+^ z?m_eJv0j%Z)-H+{?F}|A&G>E{8>z(0mH#p2SN?gZDA=?zzGOO;SUEaYgYHh$Zo~_y zVBO;QiWwPucd#U5tY*XTtYOLu3_bp-#D=BCCB>N)s)sGQ#+!AvX@Ro6a?vd;Qzz|{ zG>D95EG&y^cNm1;%g2j zXYnP;rI%jTpk6(jLjjG34%sUNSnMc4K9tX9(o<>v9?Sl`VJa%ptFo3YTU>Dd1$uNz z;>T^k%+4cxo5`yMo`20%6rXI8r!- z$MOVbs>{KuDR^HIfkg9(&DS$#R;;i<lmQt$KNMgkIm{pf)r>r3jg)u%M5m;^CyPy zwuX-yJ^1oVaY*F@nPRZE@7%S0)+|QJUwP?85MO%P<-fb*j^jHY??&+c^WHssc_z)y z!U00oA3gcxQ=?2V`$SUB4gx8?j>un9TKij}Vx=nvUxkI>-?e*puRkY#IpO?)^#7yd znP;B6;lF>u&FE1he`VX%Cu5|edo)`W(I(pZ`s$o1oJ=iS{}bx`<$xm5*PYhZ$Xafn z?OV4fxs{-<-A{yxXw%wvj6~p(?JqA2mn~Ur1192ahcYVN|0UIHhGW+c5r0GI_$|>f zpI;*WVk~56BJF;VKeTQE6fx&fO3&Y)T7FdpQ1Mys!5|}8qd~H&0g>^n-Nys$2}F|3 z4zjEY_PsaXU{Nx=iinjpB4~0ApZG@_!HbfHv#2|0>?K9{L0WM$eVl)*8fP`Bbn43D z@rjc1EWATi#%RKpq6KUAOmAKLZ1HE1xN4{N)h}4N|IRd)IOOR7cT(@2;l_;YtB?Pv zarLf1;&A+1_bhvD{($`0Hon~C=}F6Q(0&EX2GI&JT2?>{{S+8}-pHX8%a>DiXW`fk zjxZ-ofo>mDUz`;LrSM`s8^1OCnr#(q(eRfD8J<`(9al1&4w>C<6dxv-V5xhfQ#7n~ z;-3s}Ee%)V!V2@}oZXMX4oUWdR7`-;K3<2u`f6H>mMsW2{u-)e>oN!A{G%iZeV8ru zB(*Qr(*4J4I^vKhP_&bn1PX?iO(;;@wd_^4X!z%#BUJFUwp4|dsIvZPoRz@7l|Z`) zKXlU8Y?+_WV`OACJVCk(ib5nwM%E$8N&gg$qKz<&F{u3EhaVo)HmkyqyeR9!*sx(Y zXxVRLBX(wgR;pXHn86Oo*FV_V<-&#E&6qjkZwDR>oDZdDE`|sMl`@_+rO%Z+jpf@JMWd~ql%u4N3w@$so$}h_n@Vxi_`|q2P zfu%24RDD=a0+gsc{tI^hJqJdPUs)l=RT5-h{gdAE2L%vh8OJ5r0aY3f@eebYW6v^* zzq;+#gNG=y;zK$_8aH+S48I&#tb+t!9gi71rg7sYp1%^qL9AZNmhzYU$S(v#oP|vG zp36QVPQ@PILL0n;J>-HvG8!6UB@WblJL3p zOHXLj`gVzr{+l;%At2^bUP$2pB355ovSb-6$#?D6J!Kyf9%g^n`p2}tOa{E?p1ZHP z=2{V(%N{yZ^7PYBu@HUZ#*GA9up<02iQPnnHAqaIJo%xAA4Z|{H=;6N{Dk>6lIk+TqxpCu$p+kmVb=B2>d*p95 zYt~SpvJ5+y@WNT#BJqiG%5fXwTk((+3j)9;CK-6ev< zd(gVF_Qew=Y5Z5rd9q?HPx8a0P>y*Hmk>LM)d*}7ut7h(f8~^}zOvOJ^7GTnuVXL{ z^u$W5b%jg4Qu`d{KawW7Mx_EB3r-#!tots$I4|6{GrsEESkBzAW%opO<=|((OT7J? zps*-hHYL`kXTUf}g{onbPQh`f2lGD{i!qOe&AP_wG(*pSTYgne*Dj1tc|FF~cz~?m zE;#PgFuSt)Z{K~!iXjjrYBq`WydhlzTI(D*_l=8=1Dnke|a4TuKfoO zY|qPM9Z>oiG!r$eS7YC6rr9|qsJMu($;%(KzffEKJaE9djT$#*^s{NR=4`p@^%$&J z$$08lQ}ecMFFtgT5BNFIoA(V>SOL+T&-uGhqM9{ra{l?}<*ZqIUjK8e*QA#yFwj^Y z%VaW0ntj-hPfOA8otV07JTYeRrlCngury}9jA~V?%$+yy>lt734GUeM+O=x;IPpY= zUwJ0netO=N8Plm(CrzEg7$`osC2w0+R(7qLEUJo7%a2lVQZucPg4fDs(*C*?PNmZ$ zeh@BRuyDni9NvvHkp!C)LB)@&c(LMJ!nt$jLZ@*6d%AaaFkrC5^OtMms9!5zVsCyR z8M2J!lw;M|ZDs9?aEM6eGA>G&UO#i|hin`1H|*T}=<>Vuz4x~;mDnFisWQ=r>kO2_ z%Y_F^a!;Yi%#3yUa$JQLEhN(WiQpA|tf5Ly2S2g|TxAkep;;5T+5)+s zghQN2d@+8UEUN%%V(CQCf55qGH*7qAz9{zrZ?gTcwlvA%llR@njqdx%FwF+ zyeiJ0z47H1HP6Uoiva;l{H)z0kInpoWjHh*k|^0xxMKd6_s?#5r|enEU!|969U)s$ zhm1{BN}hvNG*cwziOzgQ`OjLGF3-vZsEjM`~ z6_u}G;)-kK4|fWd;+FON$lIXMpXCc_JyP}&ag<`K#9ffM#Y>j3aA}e;&;*&ktn%g0?AuQ_ z+*;8}_aD`yrijs+>l%LvfS6FUF#j>B3NeVK{JC{#Ccslf{+aGS#^1D!@RNuXY8mK- zsL=airj`g30}1A_OFKsvJ>36U8;lw6BqCmVi;41}e8ut?U3igFqK28kIwHZ4q1Q5_ z!5AyUBgtmg=bwL`odsMbMMbTQFj&5ejqs$(7kP&^W*_5jlrByfjQD@6{Ib8-y0vQ~ zEs&;8g$%5gDF`a?G%|>t&!mp!uL9lvC!KV1+jec|&Y7nmxD24r19oZm;+n*L9nM{eHN!dq(XF_R*@zi}WYqFXYb3`D^79Rfg0o7D=4Pi#t5+M*NrW+I+>AFXipq!9?Lg z-hT%1N^zDB)xaK#SoJ3cl1MpsDbhg<(*5Vi@@snVm5K6)5~VB8`tyel2wb$?XHViH+(9OK&H!OL~RLJ&Zpu66v8=u2h*#6A|eN z;m?KACCe9C+N^oAUOjtFoHS8UI2(d>#h=-TUzu4y9Lfx{VH9Sh`||hi-Ipu9tZtS9 z`V*x!gz82bFT;#~y5oqQz_=Xa%%w>-h+I#ZtYz5D{t}pOpRu2Pwrl5(6MCH3w7I%C z|10e;D_ei~;YTA!j8KhZc4Bt=eSf^~^;cgLGN1{%l`n2w4Kq#t{{8H|$9pOvltSu! z{Q2RBnCNSIDU8G0cTtFq>>%WJHq8~$ph1Is?z!ijbNb)(%U{l$Ki{M(Qu0}kt*D5h zkXhf(dh3nXJ9V}7M8beU`HgyiNXfN21=5TRbu_`2R(_*SNYXz|KUhjW*G6kH43Pyg zGMmx4YZr#hc&sH4qxwO^d{Tmnh#=DI*~8mTIH`Tp281k@*rrt*8Ipq;ZHa*I@A4~E zvfVIayeF?viAh)T5HsuH`J3+lVj`HZ`j5-6qcAYWE1myQ{YL@??FjqSs26rVH{8BH zv3W&&-FLw^!()tSl*>+ZJ1_S2yYba?Lw3hlI9@AyXuFn{!6{e7R?Z0a?~d)-8vkxw ztoJ2Bl{#oEbxekt`f&S)g?mfZ%n5Vn@hL>Ca+UCs`xBLFs0Cj(C0sQtUa*%e$ZXIi zIB5_DREsd`;l$3(^tNI(>L*&97&6qt0e_Q$ti$rx+Al?0a+$@;7Z<{ntRKqyPq=*L z%Ju8kQFGO)U8`5`p73_!RQqTepBtcYydp# z|2U1;&&E2!5C6=p>=U|mi+Afv&2D5?qX$9brcLtm_uE$sD5TqqUtqj^7&YUhV@%CYr(i@wW?LJD`>G& zWP!Y6r;f})V-kKBRx|3_r5+ReY4qcX-Y56YUAuP0sujgW2RCls3~0brsa(0!@!HHt zsm&S9`?*4=(u(Cs`_lyK30xWim9kDwYb8$cFdaDS6{_B8)8r{sJw`o7Y7&ZQBv6f; zXW6mpALRu}1+c7tn!aM3RzC=IQWGvEk5n?bJ->*HZqn@k6Uy(_QKL5AbDyaMQ=}Rv zUSR+o3){2{dJMQKYw))JXmrJvEgggV57hAkg_~aamms3PSwRkKSq-ET^B^L@$722P zp}Z>+F2-L=>0+-e=A{4R_SehTyKlW^HsB=umgUkTs#lO?`DZ-`b*6;Go)+%E_nrq? ze_G)9Bb|W`f~wW3O`S1)>5`>mKKUVdS%gmCtJG8@X$xS&)hJxm!p5vGAT zIjKJ%$v?%MLj>vdv-^)=U>+3&kx89IP@|?4G(!a&)~!`)Or>m&%C8{FLG>j=VjCUC!q{4bJ^x10W*?}j{=%ntRYWP}V6BCd@cW@o!S?gkE)Snz!eob#en&rVrH6^Slz7s@(X#7#7PNjBNb^MUcWz+F5*f`>&)xRt#V#AjT8pYRQK+K*i z0PVR=|9VQZU#;2w%_l2=xpn{QM4|$`RpHYM(o#SN6WaQbHAnPm3<#yH84t4xmqI+B z$pikWUDSOtoLI9zfmSXslrla_Vs0Yi$(&KDm?rUByjZ15xjX?PTu6uV44uOmyJ<9S z+GP3iVc`LyFs}oQ z=#?nJ;yrLQxa1wb&Pf;8RE}-}z^~dw?qEWb{>g*rlu1)IY+Q$UksAO`=+)=6(;GHw zD43RwW&yEp+qT_FC!aK9#thG^h`&Cm4W~?*l9Q9uvPDa^LFxucSs3WU$;khs8ciNT z0sy6zFEfNx6mAnLO<^DfO9Ost`Q^3HxX(XlXJW)TL+peQxKcRj6tdHJ&mKL^KC)YS z`K?{M*1!P+=g*s`d^R-zf-cH;&KbeXZ@!tgV8IF9yQ8aN^ej+j5mM@*vLmG<#hc_X zL`kYk%X!(Uw0>YmhWx$zo_^*T^K63zH7Rt`+42APFaC>XC+uUER{qM~08zHFq8c5@)S8{xhs;m`r ze(X{pO^=W#C0&^CJ){z<&hoJYpXbrbVt|7|TWHvyg>+1VR<%Oe7Mrxsk>!{8RQn?Cb_fbdlK$@&FyV$DGbH^7018t5li0I&A6de3lI#YJRL@`4 z_ka|nsQu!NC4$bs;C!}Fk=d18ClZK8?G$=RngTgn`AwA%1>meROYLp&A2il|)$?Jo?yUbLPxpp9m!n0(O-;>x@2J6A6DyF1{vFfzF?} zE3dfx_1E9b&CRj$5uDYx@8HV^LuzbA@BCqs-oA@5Yz;q`LV(mT?9rpgq{)--z4zWX z-+YsG4GBlKOo=ZBAx#y_D|YVEjl{4mC^(=MZEyq1uL8pI<+9iv2bt;R_elFF*6i%& zlA7wJkrUL?C>P^H1KHV11yLAD_;D|*Cq$#5^bo`FR*vlr^pP%R!rVt&`{sKW}fz15>SoU}R z#toTbdaUvrbU7!nW`4MJO^o>zx$_dsr-hw()e#2g-kf;zj>PUwVa}Xj*|eZ(r?5)x zpg~(LtMTEOpw6pc-vr(N|{pS7vv_MP0gGrRH>fX?CGlQG| zr0g%IB9c>cr0VB$&N-V=Kc;ArO{w}h<>WQ0RvU0`f9QGc@<#)uwIQDKS@U=dJAkg@ zbq8CuWM@}w*0fpoXtKS^yqDjU{oQ}Pee^1f!XYfs;tE`=LE}zOuRTue(XwSLDg^7; z%U4C`#P+m9IGudS@LXrRAU0$qJQhuET(&dP@yMPURYLSiB%b2wyM^>2YF=i@evwIB0o#SX_MMGKl` zWWcoTb}zrY;+pH&Ye6GGlrmv9hNz)6eKjjqoMP5e^9C6AV3IQSQD)$9+~F+J)%m*x z+y(@4oAMu07=J6jVffG=A5e(F;dR#^?!2>69Qh*9#ob?Q*hD??;JwO=xP$e3d_Idkb;Kio(}7$m)ZPA|X8 zdXgB8nN^0WU%!DeK}TZlyafco-*YEbeu=WMGW=)Gm|=-5DXM!)u@MLFA;=)!KQ{@o znEg}z%ppp105s#8C@X14Ex%a6{Rc7`;QulA9spWYSO5Rav%74egH-7qMZ|^$REnaA zpvDpvdu+j&#Aq~X5|bEXj6Ie#6C(;5V=pM8VgV~Eh$2V_r7nG8VaqfB&-a{}*=Ltk zF>ikF|DE0Ed1mfC_j|tQ+;i{TX?F}zt(!HaIE`4l6a~T2XN{BcY`A^yJ3Hg2MF>=t z-?jG3L^N)q=}WAVEpc6wPAZUME1HfjK4^&x`-){5j(rJ4LL^g1Y^lwNmo1Jj@5Kz% zZGkWQlHnwRNB6v8;rp*vHJDRX!2lFL^8|kI??3fxeLU6Ex#_@52Y&d;s>dcR zxUwu-Q(b|#u!I8uo|My#W0VuyoW=1I7g~nEO)7B{GM*AUo(=Pl2L#_SND3KJcz?AVF3@{Az8VqA=~(jnThvJ~Hi#NT2&#WzlfTdOfIsIdE71WnNJi9PiYhR>HDsMjTQ#l_mVJqpaNS9M-@f@re85lRfBwcb z$31`N+HLEqndK^sO0vWpycdAO~~3~}l*Jc&<{lv@B)|12rXHF2H3@%W?; zNMXFh%^iFf@Bm=QU2wWq27VE@kTVyN7fujVRYI-@pY%HXMNHD-BF9r!-d$c zwm3eVciz7+sUG*$6%`iy41CflzBZ}9<0l~KIef?vX7=JeAX2ZIly+$rBjn5&t?JL{ zi)fOp6h6q?NXA60tRVrIVi`nz_$uS}sB!=2{*Y46E{*XDR+vt8*>wgka zbfCwZh*Ut2jie{ipVU7|dMC-HT4KL$PHth&2icN=@ zF+Cx*Nk>5$aCnVlP0-MG$gb3^z7Ph4L*gs0xUz5GKKMt$lq_g*|Er3PKPA4VeHoq# z=eA$uzbgLGg$jYQ>;k@|N}7dx4b85cm*LKfwaW^oPY8M+%0r|n`B9>#)C0`{f>uV8Q#*syP}r zHEWX@b-BDx{4g8H(%I3rP23=^2V@3txhn{^Zm4?v*O~2_G`+fI*Su~A1kKvjt(w~7 zPn?13)IS)_0=_ENoUEGnRHeuMZ%|Oc>)u+mY`MC$w12;T6mkCMIae4kM_`}cy?EVb zqsEOmo@v**O_#1+m=}ZWTzpCjH*q_wx&Bv%Ennpy&-L~TFb~X@mzVQ0*W%)a%%0n% z5p$=;uZcwx@VNc5ZuRTM`L2Vn^f(G?{6k8K-N)f|v~&BVEwpIaa_9C7SnJlUPdfSJ zPiD`aH*YTY*>&&MW8ZxS@ZN4If{fzWqZO?%tLdNX_#fKECB?mZ_AH{op=gQOX-TL9jNHlD7q+S!y$qhEQN1Sl*#Kg@xhWk`^Nfq!f$ri;Lg9g zW-)h3=s=m17XZNsuhCoTR-h=KkNP{+w+A2?7xmD@Bv zvHG;AN!l+jL>qnFamS7s0}d|<;t3bd)DvDr5;#0ORkmddH-l{4wCU&(NAB3JC(ucM z>%I2~9;4=6AFJjQC5a9(B3(PhC&@<~am4gD-Z0X^oKHVR4EcTb+i%L$se-VAg0SVg zr=51vcfYGj2VZ8$w`LJ1dv(HA_v#jU6U$@bKPD8aSz0+&S@wAd+yHuNyk?^x8Ggd|JL$9k*^^M z+Ei`u|C;f$)tm9ES(AToja89;$<`@$t?933iR&8|Sv%0M*VLM<1XazkldYiElWkEW zTqMPs1%$OED`lGoWBF1mC1OJ;U08t@X0j=w#;>rY&}^WRdw@?W#hURM52bp;P+(mRyo@g|3C zReoMo^TO_@_Io`!913{(UL;(NOQq<&Jj9j%V zQCIGWqeeXb*yGIg?WGLaDMd_FaneaAk@CU&?`!^l$qRfl6?OvRoTMSz7|zLjA`^kZGqv}tcD8^>^* zY~8l6BhMX}9xCMPI)ipicBHC~-tADt!qmYX-l&L*t}qx^De8{VM~!F9l$5`ml=Kgx?YVqQ zpEPp8aNcM0oQtj@nEz=Ca*&2Qln>jqZ70HslUP^d2%=?W>rZ+4&r8cUl<*qEY?j+P zFse{p6C zxPuL%>SmJ#I6=moy@?qgb2#|ceo@Lo8$`^?##r9IjJ{HkF#&zCp`^rRlu|(! zlF4FK-nX`;{@)gp5eWIVv}@b8TbFL@)|RS}jqq@^vgKmYtkZrCJjSvT%2y1KCbJbS zuGn1Vgv5*Shh?Y-1D`Tp)y|R9%BA_JXA=I#O&ggepiAd2b=oiQOXArXPD~6NHdH#j zq?b`Spw*ixr?&syUK6oWu2|ZnnNlnxM;&v|J@Ysl5I8 z-{iV??{@p`w~swz?8mb{9(2ecD`aMXZgv`cO$?h>+l=eNd6_ojk7`~reegjCT{7;H zM<4q)#!xxD4v@FThylcm+Tz7axPrkxhC~$(gJ#W|aVsCn82NVpY$!Wl&0WtUufTnlt0@Nc0Lz>2!?D{p81K zY)Yx|kM~DfceVdX`%U{FV7+}wz14f{1hFvhDMJhDCU@3bWHEWJ{N;KkI!+bikM}za*&K zo>?)sdhVORaD6x$dT!8dU-OjWKbOC1%-F}iRWr9Xu9aQg|8fRx@S%t5%ookUn!EU; z$?Mqg`}OO$=bn4Anks?_I}7WBVg{a@`dh?m`(K_^YF+<>$zmvG4$wi^i+A3~rYJsL zfE6Q&l_sCyvq{$1?SG9qDpLP|^Pja}kT~bZB>P9DTmLT`E&iQ< zQG=phJ$mr(#mxh

    tuz+b$0vNm_ZmxOzkZ;OAWpcP$Sp}uAQr&~|sGaWxj-egk$ zpjjT`+ZM6v$+Z`F7@HzzgG2KtifneA$2;1L&bK?UK zG(t+#O0rN68t`v_-g#vo%q)F=k_vlwWR`Me2&VKV0=A>w|=F{GJ=ZK+0r%j(Gl@<_8;c@ZB z-{)1$jAN14+$BNq%o9)SyYIf(q(#PYFKyF?m+Nqo?&v)|GAiMMqmY z^)-Ch5e9{dh-uw8So_}j=X()K3W7z&^`TW!c+_rz9`iUPzA(JHeaYPwJTR@XdryWaTIlGJn--i zKbpGqmMG8wo`uPn7HoRbN0zTAr3}uPP*qU_{st1xu(@)%MX+5bOl?7G^@|$PHN}RSFin#m} z`yVuQ3k-0N9$8RaSh+>htm^pI;+b_AyghyT)~(yPV({=IkFcp`6E_T*Ktw)?*i#ELRzxefp`#9wVG0 zO^Fxjls1aExs;>UipmN)<<)D~7={%&}v+ThDH*3#)Wej8>qV!YV4N zv?JkUL4N)fS6u$sWB;bYu8 zo+xFVh2Pro-@SYH2@@xb963_2EU_Vi_}fkX%t264$g#hC9xv1`nFi&qqJ8PcF6~Rr zR{AQ70fm8xng<+_e8jkuUKq+FL1omxVU?vO{uh7c3PD2lEBY+8`Bzj_GUt{ULqM)d zkgOb_JnG!NTQ|Yv)GzppwyYlY{5>l+u4yO^|`qjaGmqG*rBSphpfKj=kOUILU*5d^T>eQLs%NP=NH zt~_FeHN$1a?Af0jIs8b&i21Kvv2y$a<9Xd3=TbHOs+nd?EFxVWjHE~E?NfLE%iR#m zmM@jD6mm%ojwH)Ik$H&MtzXAkS`Hxfh)$ScQn!VR76}VrF_dWew(^-XXBHM0F-oT5 z=%YE=RZ>(kc+jAaKKuy#kfTadeBJsI1bgn$Pj9A3D~u?u&c5=>%e)g^X)2;G6!;%c zq#SzK;HgtzWimFh_uFT`Ns}gRFW;_%0nWDMH<&VcieKk6Q?-l5x~HFcnu)9AGOEP- zAA>b6;$*gRdqq|C;KL5>)UiXX7@@D(|H?f{6?&D$m1TU%HP=AH{bL50EouEyJ}y!# zen_s;`5F%NlK8hUQ;Z2y>|}>oBM!CemrVXV47IAVSgO;nR@j2Tpla}q5>S#hKnr_$ zYiIm=+HZbw(DR^h>CEusm$U2Fg)jZBdf(BRyn?Xv-kF|*g0){pTh;~(XNK=T6^ytf zC~h8(xF~x6iD>rIu5Q;S(`DbFQ40`v_D_nR`bQO+ zjDO?@%CPMVzBB*VyZ_bk2LmjRPIqm>KlXg$f524R=F+=N@TFqv@gjxPiunT zgzp}RDl0!d`Sk3zZLuzuLWy0iUlzZ%}LF>ycC&jAZl@6Z6kk^1aH0!3)n%T31@C`xJs;=FE>?n>tlvETetn?{3uN8}+r{ z!w);`t!dNkAWNv9&iWWrd1&SIv8OX_9Fv1EhGCYs3opKScjl@WoIWZ4-1@DmuKehu z4^@rEr%k)IJcPB+zWcUqrR$fdpr^j@-1ANtJ*JRHjDh&}?KjKYnYA<0J;Kx~WYYHK z{>kls)FNQ4_%qhF)v!@R-mCKZ>#x%(;h%9AU(97c%~K|;Tuzm(s4!PM{00+VPsb+n&aeSte5uRTf6!Ho1`_oPww-ThP7f;X}Gz)4e&b zBT_9|lmum!YoGn>Cv!Ian|D3%MqA$gz+-bXE(Tcn1&xMvyy~#dKj8>ZMC=p$2g1f5 zlorJuPu+Wb$!BfeUvoE;b;RdcEM6(0CGiKpsI75NVzpB3(1*J9!I=`j42?jNsR z4+qwj<9P`piE0Uy@~WW^O3$4O39CYGI6QpNp**ERf)B@~MN7DG^X3mem~q%)hw}12 zVJn+5?kMS?f8oJYM4bs434U6?q*2Hi*TyGk6}Om-jqFF|3kk+6f|nw$ci(+ik`_VgYK3Jr zXyqoq^wMF69>(qB!0}#mU@9wkeOv9Ltg@Z-9e3S% z+G(dri-KHcvS4Y-pH0#|BuqPbI`}|Bk1O+|Z`G5P5(ITb@;>_vxc~n9PdM>Al{=V; zR>wZ=+P1sxw%a(#MXupXoosUd2ntmkFku9ZN_vPS+X=@S6_AdvQmh&Qt9tj3fW=qH z$w%8Ns6^@Vk1X&6$tjg|bAtGD^E@-^3y3;Kxg8!lRb)BX?9!zR^ZBb>%3mnK#CbCo zu3J)F*pQj`nWSes^{$6XbdL(BAu5O~^P5_iRVbU%tYq=jrXiXIV&dsLwO=K3GixRL ziYVjwsAK19^MjDdgEK*Uo;x3=VOe;vv!GE7FuUum5bI_vX923Bk7v!| zhC97y%CVwg@|4NE=Bc>2_@aw0@-%tjtNo+AM3>=<$BQpbzWS=GD%nk{iK{iq8f=1@ z-PY`q3P@XH|CO!ONj|4x1_uikEI9szY4t~FQ=1w@1Op3*WGuKC`_?KN)FVPSwa86?J(av?>tg0 z&3}!#3^bn^J{h=?w>H4qDKh5d6`tU@{9~Ie)F=sP`X_=D){!(4=j! z$FQ(XR|G;@!kZ6OZ{5V5%;5ne!@Y(x4hwf|$u9jMyJQweY;;eVV}BB~>q-ENpH+cC zYVgfRwfy6`|0njp@Pe}lng5-c``OaSv|MGx{^!@~PTSuwepaDmuT;B!$*{Il1Mb0T z-?kScJZya8fv4u9`tVXBD-MP)9XatMF>|@)#v5)0Y$Yj)_#<#7)Qq2D#S-INjJs^E zf27t@0ZZNE@&6V0F(tI2JH;!*<_0zwMc0cfI`%nEd?vefx7q51o;?>WjOj z=s~%G+U6qSf?a|yE3|1lF6L%725x(7ASoUY`mNP|$(cWQ-n6%-lgi;gnVhg-LfqJ` z)|F5ucmJ!3h_alLxa`WG-g?s?K=6?I{f#&N;g(x~=+>>pXp!o|I`P+z1GA@x@zXuN3Z;l?H z#0qublU{g{lO1Te_|i*$^^0H1x*%X4AD*Lm>E)M2*+bB12s7~yt6R2|J^9#UXT`fD z%P|RhHFjW}=>VVct^AO-FU6YTpT)>o`xSqs$fTHZFyWC$x^(Qw?BLonMbXg15C3%D z+^=cBKl|C0%m7# zz+D;_X;I^m76rYn+KN(qN&RCkRy5@J-{@%+tMY_?TE;QGe&brOpg-r_b6$Ay1#}`^ zQ#)b6EDXGumgmDuSFeGZ1XR_AXN~2IGtLk(Z>5PDH3%-~rbvGtuRNUMHHPLG@=WdF`JR^X|zvh>I6H_0<5DSlEsjUN*3{^Q-2PuF5c z{w70pbEZN^2VuB~3%iRK>pm;qbKv*1D6_19lNbjZ;9 z@YN*=-zclRh7eMtbXH8aZPxTZCN1So+I$5c{3ED@trHhMOqqX!^jK2mjkR!N`ls*H(BQz^fQdLh{J442{Sz0l{u7t;oeN7chMh2gICbVr*j1+cL#Q z#9T7Cjz#AYosDhA4ltud4XR(=ei62(T|wbSn@cqfLt+H!}T} zx}<-Qnc6n|n*LcyEOqcH#wp0vt5CGQJWRPI!}VnLF3aIRUol4Lf4Vq%3Ojy#fyR8^b_9gB+Fwr%U=9B`-j zH#WvT)+lLlWcm8`YeKPJnGr8?TkifvIOft^d^6YeP%L(dc{P*m+&|oVR51Nt0f996 z?r^}!sAY$+RoAfB;PA_NLFwXv?`h+slYhfyqRgN(!w+AmTJk}rvOM_o&8U69uuslr8|QSD>K}+Thb)2=fTnLI@HOo_&j*&&%E}A zN9v-khv}s-4H`Rta^+-gz?7|&cn$wJw#hMTtxn9tF0b_ez5TBg(>ng2jh_u)KB-f` zU$uY4Yw7+=?uoqwzF%MOTAX!Kx zZxDjUd#V(@?Y(ZvsMbk;I6ywO5;ablSRU0q;r^XR`G{o%NA8#Zp_%+8!MLS`Q)zkj5p8Ea&xBWs;9OW8YG5^R*q}r0UuU&7yHZ83TPr?z0(yR3C|~WDH9QTI_&Rzf`)m({%7t9jR3k zmgfw3;oc7iZ_BKz&gyjIj!KShm)-lp!50j8wOMgTlMmZB+3N@Uzw_ef*S)=pO5KsI z;u8Sww^wieVClryzr1m4#Y!$>X(rTw+d7!*9)IT- z7gQe7{_-Qb|3a_4)fHiNdDx^c|BxzkWXk8IQ=a^M+~)1e8DZ*7Y-CR#acttQGj+NC z5d&M)^oe)T!wqtp{*YH}8a^6a2WpfSFRW716Lk8Pc43~?;gkDCKY}$tJCh|}P$K5f zgbO^Y!V@cwC_9Vpo#G%LI0RRszti~ zr=NLNt}$~|7Y@g{4LgmYvN9mvdn=A&Gx3Y*wP(aA`=nN6vzkR=@q~bbo zTNW=`#BG^CGwN^}EFP<4H+*PX?!2a3+uW(L3;n zpx~rNo`b)~8i~nbzO_*fDcI;9b5^mHk>Tlbo^ukkXp*k)fDzS*SEs&u(S;ZC)(jzOi5C&Z4gx0a6p28#l*T4ETL%QC* zdtZFXB}S?q|Ln^C7iCy20$QZlHjOO0#gpj!A?5%tOBVPzN?NpGtnE8hh9p-79)9>? z30Z3U_U#{f@PU#NO{ALO*si++X}{Jg zjiO4Y+5w9zG{sjog7$?fi=w!G>)rplcpd);L2<&j43s&hTph3%P-@-&*=WTAUcVAj zapr~iy76<0k0cAKQ71~S+BQjbe3y}&a=o?cf{jTb|_3_d8Rn zY9UVBwQk*a&pnO1Ja^~4cmMmbe{U-<_ubeR%a-lESHDYt^y9P6I+L@^Ma9Kj09e0n z9kc0OdBx?9gOfOx$}cEjE)2sF|H9(pSEf!KHgu@90TF-aop)QeXgTf&^>CKxp zdv)!)Z25A_3%NSN3zB%h^c7eAv|G0xoKNRQa>(59n_sV7 zwc4s%{vmX51)Ex>R=B1dpLQ{@IoGt2;Z=m~+GE zGVr)N-G>dDR@_F|*1MT!_#&umDelYjlOj|;_kE`#a+Z<+a%+T+Eqyc1K~da(>_>h# z5%EExbzDZn;CQQWNr!X$KlSh@C$8WArSc(HnO$G?#os>~aoV2$Zr?PXBD)~J;pjbX zIkd}9-dK9mM{E976>UMe`7Vb-GSUKn`KB3fFS~c{hNpH^Ra94Vm9#+bAhY(yqy)S= zsD0tU(S82du8HQIvZcllHxssso*>+?ed*J4f3R@V6edaKIxFvO(X&|LHr`XK$I-X} z&6_(YHQo@UnJHM?Td6uhR%+@jm0-dfnV?USVImf~PlMNfQ*2AA! z{ZBmcqzA{3kLA_;ke6L=y)!-UefYad62x5@L3Hv-Cvxf(g%V7@$!AkKt$tZy%V?`* zBj>_q9;+lcaXC<>VtiH+;aE+RGp|^&a^b>-yb;Fi2za#QqYq}j^6JZPPMfxMOPOiY zsR5BTTq|qbtb$w99#M2g%$-)TfcmAw!#&qS16|saw)WsMS6{ND- zW2u-r466Y1i^-6i<9|eCEG<|OVOnmz&_Rf9Qd2+se3)jwXshRtu;#lE&ASc$VTbJ~IF%MX5DYT?K`|Y>`{MVQaqCYP6{bv!nWz1O@PWd1JY`tm>y{ZV4$DPA8l9Z>i(0v#TN?@r zuD$k}Ywf`z`7OsY=Ehi;AOs?pN|*4&A2KAXep#xs{+R{E&Y5%kamOK8r+rDW)sYcnRl&e15+RM_ zjqBUBSf&b^P}=`k3I7thbHr41ntyU}g;#5llyTEH=a?!gcnRx~M~(pJiN_y%;Gg&N zu-KJXUb$QI77BS=VCMYI8CJjMM5|+g=laK#NryNaN%x?MS^sPjGRm^Oe?!`M#r(|| zzMlBUOrvNlr*K+~nHrmzkai^dIo~y!iqUFH;`-+!73*R8fWu9RY&2 zMYg+G=Eo(q;)|NIB-F6&$Vd33P_%2;uEU29vphECH(wW=1aN-kl~-Lw|4-|rm^z=7 z84!3x-$)8t_}aAo(?-dM9CFAdNO-s9sK4#vL&h5)F7KeB_eF#3aR1(|*JJ zs9n!!;7QR7cUM+t^WXSqUjHM5;wC}!HeruJ8D2p9#hj`QUuItUbJ+6$h-U^I6Y(|{ z&Qt8z94(v?&YBzyJi!31{my+OTJ&MKV^b!tFzR_=xc5=03O1KUlkW-2x9Ui%Nz1&x zhexeCsK~+(qv;d5xu1vl!hMepJM`w!X5%M6AYk@EwU%cRD`G?34XWv!{@IG|TK|I@ zTT(5_=C|Abn*VFpZ^AzTtm)reVUqss#?S5;*s=&ACuZvOpOz^XDvT2U8iY4@;_p^@ozux^_G*REeDpZI^-&9|(5^hu`o_BPDuN7#BO)W+Nd zr?%;T!s6XK?(yLLtp8{2x6_X=-Tdx5o8Em_fa)1F0XpdPMYw?aRktctyuyu(7^;Op zb9Tr-WaQSbHYjOHcqxsa2Ojum>ALmuB!!}=W2a8dn>YXZ<3}(YpT`x?IQy($UUMyU zY{(Iu@!jvvnlqvi8DcRfC;$cUwQrY7Aoc#9l!0<>4ZHYg)n;aeMc&R(!EEed*-WTivrpb!wIWN1XB&mV zrghsF|6}$MC-oWMx7AT8ezTJHWBT5CbdQ@BZFqgj#;Hp;yvw!bh7Fn*S0*_nK4?=w!!(4@xb#$lC`U zOP6rU&?3Pokr5ln3rEj7D>r^p|A6TDl(6#yl;D?k=jIEV)(1Pk%`9e%Q>B8gQx1e4 z>8xzpv`LRAD92Jojdgv28vo&S;?MxBkR-_Bs(9~SeMkX~V(Qe?HB(r*5)vfetiy&4 z<(Op1{cO)a_uQ7PWkxC~V%j;bE-7hPzJu3p^3pxcgjcQkUBbcwo^;eG7C=2|kFM&< zty|00kGej!7Aa%-_~Vb?nm+vyrgwF_CiWTKFgKeasuS)6#1l z*+OrMuQ0P3FfYls?mX5w9VsqytgMkw)n-`_9pCwrfh3uN;nIuUh(S3W<>LYpRonq+ z?23PFsYKj=vTmCahIIM1?LYY8r5}Fqp}b~F-Ty+uRKo}7`{OPGM?NwBBFGseZ@}K&Bd zzEl1Vgk*00ruft(6Dl1$dK4!+n0S2K_H7OA`neI4rI0W!$^J`bSt`>lH*0?5?|*;b zL5Ey;;rUF`i6XAXaQSHK)~#h_TUD64F~6yR7VFtmDRKWJ8nL0~2W1&PDlFDzhUs^^ z=yK@p%sAC5Yr>b8oN6%nr5A6y=_aNJ`o%A%D z9By3If@6w=i(tGu81dG$U)}}ak4KT|{WD^j{Cqnlo48N4MLrO^v&*5VZr&lMWa`{U z1gzWsGi|^5dHJJ`Ip*=Fp8^iMd0qh^wOK_hFfqREL}*R0<+%toedk`qI;mBKNAqlM7sX&EF-2W$SN+$h>?H3mbTZ>SPqBE6~ zh+J%{_O!n2d)L_q{paut>6s=tvEt(m8`FS+!g>3IZnWn8!CmJvRHQ2mLT6m)hvvo9T z9#vKZYZpczPhuaC&wG1E{2<%DH-QzcTo}%KBiy*ADx1k`)FK>tc^E2^jiADXT^L(n zru_rCI(>-|N=o`q6w4jr<^J)i^b~Ub7xup%ND2wl?e+VO?bqvv5=L9&&$NCOSWu7S zr|$UK_m9fQsPH(PPBZB03%i7AKa*j;8VN(TDpol{x|M_m(ZGk@sr^E`3 zN5hAgEn4Cu9H9$aP#WJPo~L^?ThPXx>~e1%{~%6c>L1z~5R5l|b?q;xBeg$42EV`g zW>y?cNzUBL0vjJcdNf;spI&*@&#%75+OKS%zhJ=^%a?ZO*!j@I4!!x7+b;jfWxoFf ziiPmt+bjT4b||W)Hn{gMe;$3@@pam7`;NQ6_t8h+89ip}QcR$6t^_a@Gb>X5 z+|x06%9H~J9^gH&1hgpfuKD@3d+f2t$tRwu4K*cY&>9&XF>`{M51Kq@fvoH?$DQz( zzyGbMkTV^4K_z1T;%uaa!D11uTe}A8vKxpR|1&w=iq&fl8$5^!P0%O;NmJJ8 z8~F(TP*hAXkeA9o_|QXiiEL}{ZwrO~@3dcBBpYZ0N&F}RM8y``o}G%r9Xb+X0Uu7QH-A2L+4Y~4KE}f`*{TLXMiIn2DXsl*J3rpR z$>8&b@d||}ezTqg1jBK#*gxhafqSD>9LgKiEkBA_^MV9~V>!A0N&HW{SDuOS756e= zT9@DzX8y;Nq<;JN-;4XDx$)ZznquWSl(Z8PN62!B;B*5D@^SsD zLAj;Q-i3)`KX^>IS)6)kIldQioq$PAtX~K<{;|H=bO_%iB^XA{n>PE!&woyew%|n5 z(MbntjWV7b+oUwIy~`v5EPKtW)!f-`J{P>1=5&Iques)uAB=0j5G)M0Z7&DDQHkD7 zv2EK{B#MfQD4x%1|NO}k|;Eg%;BW1z@+lu1|{6h%hf zVT)D0CWmSiVqBIeaUn&TN-5|_!Q+n4+|hIAeb%>+p1Za>)P7_j$|M{~@`!V2)CGjQ(h8S?MjaCKu?ahOw^>H=!zv_1;`|gbY*EX-irW zq%#E=z>e{huKV|C(GLEdnzj9s*DK^jMNzmZjJBEK%tB;^c++jj?Ih6CC{IewB9oC4 zTd`lV;#!u)Qe~O=$Dq6@;<3N*l}f9A^_#R`TbB&!_2PT{k3RO;U;pwt1>2l?`|Zz@ zEUwHoijIndoZ?%Jdixr4+_7D{Oy(ZizJ2>Lc{5MMFuldXg$u@yf9T3JT< z?a-kEcb&jTS#|j5_y;Dg)!4+8oSp~T3CACQ(7=Hj*)XVONKLhodBf z#JSzsvuB^k*iOtukN-Y>`f~Fy(>Zyfk=4~m<&zDZf+^(mx*0}DD?mDQ)RlRbT>i-O zo+e(5LNdkKkzkDvF%d&pfP^Q}N(}2nM+QpczCZ5i=bo_=ZlfgcxbqIq+H-F+)9JIn zmRA((;$x(QZ+}7aX3dLier2&?!3p~0v|r4}Kd#^HaY^jKOJXE5Kx0Kx?mi_HA|AH^I0+c%6vTZR-TD0Zvc=Wv}V`sjsd*~h{53Z}UZ`wb0y4HT0#i=$Wn z5p3J8gOFysN4*D!jd$asX}~=oGba-XK}pkK|1nGyVs?nAw_m85w`%yyfqJ*5{E+sa z@%W_1KftJ;u}YG^QvVyiD*V5`|IPJ(ioY{|V;1}00~&fY{8d+$Yw=Sre|&+Iyeebj z53M6kP#2)c_x?fo2zz%_8ri}t%V=+H=oz%{}jJo|BEY2 z{qczzF<(T)ITv50)Yh!Hyiy0hUjDV>CujUD2Iha*vce+g+WJ@V-~86wD}H>r&U9!T z0zdt6>PlyM*@crp|H+k)Y>?NuxZlJFIk{}LD89nyC*d!RRmRS^MmB{}HMG-!Y##CB!>yJW$lwWYj#P(3b?&6+elX4EJ} zqfzspTKp6L2wy&u|4$z~_93phs?suSaw+cOapTCm^hcNSwBtEvov~~wM=0?k7u4SW z_U+qe{Dg^n_3xkS{}jJhuil@`n|I*_=Rfz%Gn+PSlnczwF`NfH6Dk!I4jDT1u_vC$ zW6F{o{&Izg$)4kw-DzFv>8GCh=9_P5NB~81lwoj>Ki_>1!%L~=bdon{ z)hP=@y^#Iyy?^;D@057(p$E-)^SfkamDe>i-Nt(3zs~Wm#f!l8snfm^{?cKHb=sG) zvM&*kqtpFw>=25@T&36bKcDd}+F+xxh|WG(4Y?b<{a}Xv{6tm4%+zoZ z#WaZyB7#!Hx?Zy!D%wn(C|_vAgP79cq7@Rv7sc9i>dMt(d1P7^->flFcVU}z2fXmi z+@E~7`k&R=0!2!eCq1*DmOj1sy{UscTybEB3yKQj31f^H!I#G_RgW)JHPqC`QF&!p z>~Bwdt^1AE#@4dHgqJ;tiGf&3F%@VDm#K{um zyhh_j#~d@Vd2^jpBa5l#L{gC)l$4Yhq`rtBPdw|)GdbCzXpcP&L-FTI>6>r7xn${* z&Ye5U*=B?5q9BxV9r8>xVd(cxeCeLqYnNv8GK~#i zJ5kjLvvsg!8wD9AVorunWQf1Vzq!U^#tL|+V16^7dGvrpXJlWBG+d{8uvIx-n_?2L zh?5{C<~+zHGdKSG?6Jony|VN*cIx}?x39#Iw;oSW@?KoOTDOItv~%8PpMCek*SiPD3GZ?BcG_1TMPk7inFMHh!`-MDch;R^(%6no5~RaDfcPamFt1Xl8{SDLkmjB%j2wxvn+t945!CAWRC-l=nE z&o<39{bvGcb|o9Xz2SxzCcog3;+WD9Mfcow&*DXkk2`M6A%g}Z>)JgF9@cnWJkC-( zjcxV*r>6aqp?*NN=&wORgW*REZ_&I36Uzx01ISC-Z+?S<#fz6tpY~=#FvfR3NG0x5 zAFl)o)An>QWQ)a&`n80olp^hZP+414GN*pi{fqMlzTW+>1)ks(6|lAH5Dx#Q9qdFe z`C25m%k4KPX%-!DZ1nbnc`FwO*~;j{7sA%vn2#gqyl*)0r0n7u+|HL-F*lg`Vzkew zut9M!V05%{K`?uAwDyZ=#?wLnBf=&vgGqO0xd$Yu&g?ZJ>OCkdY|Ol*!IyKwxv#UA z)ETGauMS&u;AW&~@rS|uw}LIJxWg>Jb?0E%_v9>FSV`M23Az3;tLZ-#mSg}LlV%6<8z9cd(?Brv6VY^6f|g{l=}9+yiJMe3AyjHNyCOsn{g*mvLx~E z8g2N)W7B1s(v{f;MVS`uA|A|1`Zr{_48D02H)3w$lnr%F1aaGcVPR3@#*Ovbdk&$j z9(Qc7;FdIAO-;kZ)*Y1E@t~XqWt5lC&Vh^Vx;RZhbnMsx zd>*u1vqtmvHgD0Qq_7B&RaI7f$%DNi^S*arFo|{RO4n6Z(ET&lgpND$6smbJE!Y3s zwlN3yR&Ji=6eND6DRDxRci6XR-m;{`2g!DTv3k#52U>_RCd$rcd5c!Rrh(?LvzdEsMAVw|4)JE^@s-aJ51rr+tx| z`rl;g?tdLkV%Dm^ZTsE6w)B&+XLH~}K^?K1VFZO3(KRU5xUo`HG;sQx1YK27pE3=?zGP{&&(P%nm`V?l@6cqUc*!YShM5L~N ze)WrA*xd@M2$s{&INeH_G$6>=_P^J!TjcmOB5pe1A`r^GYA(O*GER1wOgL!Jpm_@x zd@}o!-(LUQr=NU+BG#LtEu;y2_q(V5>iXZbYu6qv;%WV@SenQD2YgEX?Y_Trd;9ax zJ%7iow@#lvofIMIFY4Q;&!s=U?3{DX)sz92;p{PplCk-g&PFpmUo)8*Lmr+m;pUtF zux!aP)tta|9ii_Ydz?9T?70_Qz(gJhTeZgfUvL1dWO5?YN*--TW=PC@Ck!p3?J4TN z8O|v#?6DoCyO~xMl#<(j@yqLecJ(#Wr%nIGHCNAg|NYcIWFC3Mk-z%w_51IC0PF9* z^ESdd{tflCrYL{A@w3JN=3_L7u(+Gu&Q{+^|K{E@nOgm`^Z%;-FRFEGx5rU!Utb43B!MAioFwv4Iq7`2XG9@SO`RAT3hCem`lB$T^4U^^NbEd3p$C@ci zu6b?AjeT1mH?Zx6U78;Nz8PhxUrZbH*3j!x( zhY%@p*_)9?V#=Sg<}?%2mll&TTS?KceG>l=V)6t5m^jvytKc2C-G0%tqgN|zjb($e_+R_%Agh7C_Xp?!v&=Plh#RbZ&^ z%^q~nLCo}NLm72S@MmYiE|Lnh+K~HILF~iSd^H%9SRy!?H!To0HrUc0Tjc0Kur4*WTlD_HqeU|CCtgjxwme>itNnW#pRoWD_)YuY%1T}@ ztzC?u1;{^#C6(OS!FG(FTfgYCg`eE|rIc9w585xc315B9&+fYWZo4rPr8H4nOvqM%8W#y8(`OOGDi4y_eH&3Q)f>f&0Fn*EHxI+#kWHw*_)Pel zLrekk;JE);)1MZ@N2#r+``@p^N4lQ+H46cdoei61M_(QNV_bD*W$@W6+|!ZQwpUov zINPybxX1ADqsg@H%u~M)y9|h0bqhM}5%xN$dfBYZ#+6a&lHj%bgE2p=-mQJOZh4Sb z5cC@!wC^K+%VufD(+VC4%?~>C;0y@wuazycU&?;|7SoajyLAW;Iyq{*o5iFe&>Mnm zVyVwRc)7t54Da8spX#?ZelYWe?DUD@=kE~*oW=-t z>zWz*y=dU*Fry(B)~#Q^0jO7Bdksd!4`<;jE34`AyLIh)<{78=?B1PyGg-^NTz=O* z_pV#NK{w~qz4M-UUagX!f6DR4@uV0~@j{Sivs~k4qI8S3NE1MYVS@sW2*QH=%pJGh z(zsER&6~G8IALN%RprQ|M(n?@P8qM;xc)ac-pEOYU;X^rmMvS9t}UJT=wt8Acpoj? zY(-fGRg~?~qvv^NoyjQ=kFL3&&wcpO$CfNxUR21Xa&D>C)KY~F3NJqYdxso!puB7z zQPCjc#n}7qvrpUBZSo-#`#31ywvAVT{_*a6IS)ejh3#0^v`LeGefy3$;)ve8dr(*A zQ?=i@^XEPC$iJ5?U(QJ%I|CJRvg7z;M-3fvxS6l`60nw<@u6BY8NQ?yWSepPtIiZV zb>rvTx8KUO>pnX4P~EobCx~=l5VuFgW;`O$6yaR)4%`T|N1vKN(W_I z%F4EE=89Uw#*GNbgeM$h|Cnaf=CoguY~jN8=rLpXXA7JyWt)qOikmcRraoBymk_n> z{I#-k{~VhWa1Qapi!VO^!VBm>H*ea^DLl@=71^0VQ>8C@5AfuHoc-^yV~(G{Z~=6| z=lo#Z`gH~>B>wDY7c5%Hp9=N@V<5pQKVutXI@h|^Nv~K*9mbPE5yZ`x>Lj73Mx7+_xN+#+opkV6p zKcHH^Mf+7xYJ_Bk7^_NHNj6ZUjQErc{bNjDy_5DXl%OO7D0FHFpaTfmreh^*047gn zdYmSFg{|ojv{!(Y;wf1RS@q@$SOodvLxH5F37s^-M0>YCrAN!*lNVh5!J3J@kwBE9 zLe7^}2b(`y@xVu`$2SNXb!m1;Nv3tn;$HcgMwOM@t1GwjlJ+JITDEG|vq@f$c1`wb zUEI4-QL7YRie0zGf%8Y(>aw}BS3mygn#WddpU(^pcFe<53|!;H0r-Jv=0FUe5RN5b z9uLcKj+9E`W%BGrD)KTV)ma|8k&7JPJT7b8|I*374qhx#4pLWP6=2rNXDW_>A6Iw@ zlTQ2tmZ4KNt~sktBpLhaz>%AZg=tv8Tm4ePtM!nB4{p({`Pxz*Rwv1H=}1%FHqBCK zctOXmo%{CbEh=J#N*St4#O9=fttre)Kd&+#*Fu; zO?#7k08@OnI7f5XVMBJ?t)&VnORlPg4=7$02zi3!dioiskG=D*Kgu2zQlcq|xYY6V zQ%`ZiollCQ-$(WeG@bOQ~-|%(FVgiCPC9K364LaWV+V=bU8?P_o=}pO?OZz(h zbwN>KsQSO|y6Z-a7~xEWpDu_~#2qG0+Gf$9)gln3)~s36 z-kv`4sH4>Aa4v4OaX$$1H`;aUyYMm6>(@fNW$Ll2G@ALCE$D4j(n$y_N9 zvO6*%&)_lcQw|r0Y(kA4L8gLNGDbX-hggPZTkt=PjcayHq`(Od-Z)BQD`aj_ZgkY{ z9cy$&iPNz~+3@5X#JX>mD=V4Uh#ru8*mNKQKJ|-8 z9yY_tn~{6TTpZUgnUXSp%MZ3;2_kQ1GFgNAfAbGWN)#ghR=;WdQwl76XOh}-|A;fm zARGSR!GnelA2#)sSFgDIr%yil1jtEz#`_n<4IgPirTEe-5JLk|K8TMqTejS-N3ULA zu3TYCWRgPLl}aDp4tOGcqWl=_^5+!Qxl8l#8KWmvaIWc}-UcpZ;yX@?9&rz>w zjx*4B|7%P%l5qa+e>JrK?*3O5tG414`G&7mkSt8wuc}I_F-cUdh%EGO!TUEH^q;j~ z9wy@5G*|re%GX|-Do}T%RZ~nXRd8P6j-Yg~tsvm4I3{B{8U9Cw;2f(Tbz>4%IqUT3 z0Mv`frIv1pY5$)){;?vp`hQAFpH#`i()ZQ-U+0SdDfBh%t4{p51Hh*g8a)2RB&lEH zt8Mb4OhL%hq`eNx&V4tpaz`|CO3=1<(0{mIxia{S?C0-WEqB~E{lHPehP*50@t;_HZmnE}gA(ObCe0^Bdh(??J z_~qd9cbTu0;bgRX_vnDpas;1UZoiVZ%7MR`CjYbK#-~h@sZyTeyMMUN;q~d$*00;x zr%!JK2+GU1fBebp;lqYfABM$`iZaR1<4-;P;>#}=H7M-evnOvgUb$xVXP}i zh7CVq^f6ZZREmkeaGT!XTTvc$>L2X3duHp#2=gag5w2Sl47(sGZj5$Nx%S_rQDa`l zR8+wAGd1sR+qQl7>72)&c=E*KkK<-Z>KDe9%q|&3Lxvnm@Uq*|HMqQB_YNK1QmaG- zGwruqmo5z(G3{NncI`U;8Z~U(u|o&kf+EE$W@G}+ba1AIVGm9LpTloXqi{oqMI85= za2H?&!=Wx6Iu$o+Xg9S>yJ^ggswr%hdaNjh3 z79p$qn)c;9m9C1GbHwdGYQNPxDn1x_47W0A8Xyvruh~C}jKW-r5|)Hk zcnaP&l^`SJD}dBX%8wWqj}O0Y|I5jXE?v7)*7slh0%e3O&AuF;JFK~2=C$B@)Be+n zNWUtQ&)1Hh8`G%l!kef{l^LXB#^0c8_YY=Wa{O!J@&30_lg7Ts(7yPa_P@OTvSGt6 zz{h&I{`A$yzw`hdI(3TKSpoOInGc_u<7*V_&-TBZzG=)YoaPS7{tx%R=rRup-`r~c zk)^)P@WjXeY5P(EuRZvibIrR3^Y#1Rm&gB!0koJK`P%+Z@uk`E2}R)}ES(~>bNltj zSR`Vobk|170L&mAAhYFFsB=HM2FSb=I^$^IIsj0p+s)Xd_+s)3ZfSLA8?Y532`PP8 zwA0s!L7R4;wo2sU8zr@ww9b%3TINMjQ9<+L_xQ`;&Oe?s|C-M?O(G@Wq=$ApdS+!u zQ1)5r%gr0LI-}21om%X#ATa(i%3K-SDcMvxq`G93xmW(@4+AYU zv=+-6KaoKh)3)h1?suHO68KL)TGcTni=o6h5)m{x#=(oA-UyKC{@;JU{oC!{Zf$9) zidh6x1RMlZm>*V*>E3(o)vbFEuV2eGqPAv2=^C>ZQBL)%Pyj>cTogh&!qQ+*;HFB) z0xBz~QnTvoO{u3MWSVtIkTQamZa$8kHP%)Cl(R#J|Af5;gd)QOWEcJpKbs2z2Kk z?WKlf7=PIylEKr@KD%b^>JGc>!Q-@j!9Q(ZMnQPa-|<;TL5dF+RQQkintIWX11zrJ z6yMv|-???g&8MjOUf>$C6Rp`kefJnTbg1IfRhmF-Z-S2s1XzquP5V9fyz_6l`PQx5 zwhGUMF&2_iB>Qi>_10sK9?7ck@f0$KZ>L2RBwl?aS}glrOBAMo2t=hoQ34cLatK*1 zvrUZE>3?ecRjP zzDeWXi{TtN#4fUm@c!T2YgjTjC;r2b3mO-kf5EhA(_emN%EXBi&ph)?9hX{#eB1Wx z{8c-=0gQPQwav$?QZ;AXpr^xKrNc;W)4KnvtgTvnk?BIdB7dpFLp(0dnl_ak1miCG z{`J@YMmtAs*4_UiWuXWOg+HTXg$Hg= z+i#4T;8Vl|oDbje_(6$s(zj^8)2F@l*T4Q{(&S0&OMSL~GsOIpxXpZLU*>-q{MK93 z-+AZlzI*f?J!#AW79o+o!PhVyZ_{ z}Y zTL*2sNBbX_dGi4-MFmsu3px!58@C8rb`1L*5^>sM<>Kt-)zOqc=bd|3u>Wy+2b_>? z)fw#Q^S61zg{NAo@*4)d4vxAWAphX-na@Ql=iuS$4tr(xKQSz6fvod&4t%?_>*X(p zyS(#9`m5tJ^^X-19vl`dTrg|q$Bj!GF*9!CMvXWR@bJGL-LRQE*|*aF7ZmV%Zti3;7qq}tOj5=nT|6=(UUwpYDPq)dyN5}2hA!2d_43dvl;%6WHLG`6J;Et}Nuw>bVadjA{y-@5gi;~xxi+tvTr_(ZJI_GJcR zqq{*{u^SXf&VT3nJ@ml+$wkrx>Z#uvd^s}tXtgFj>x`dy>y92ZL^uv3q*>S?;3=nF zINN;GMOo*-4xXbL5Uq$2vJ)PXQy{}nN3)Re3pg4vK-TyuJ`t0gf>A{@PO?FR_1;oS zqT|~tqUxUH6^e;MyqFhwo>N(^8|`tX+Bi-}> zJ<;9F^E?Zpf+8}hsG!LMXEQjPL`m|0CNW}SUd($@llY#27$b-xI7OKph$2lh4>AcT zGBwRS^nCC6|GsZk)!Apbw=aVUnqA#@pS`PAeYNVVwW?~@o*r6z)5d{yt&LnFRiDgh zp!C*R1Iv2r1{&Ow$Ahx#Shi-hc_Z&mlg2!P$o5GtvWBh*4>ov4k>_R9;6M<9AC6cu z7e7S13HThqpZ@8eGScXQF1mOiF(Z-w|Z$wCm)?tAR^(f|3; zC}jM2-d)&WB|fZe+0tc^3B)%4$|NnQ%xS0n37-fVH=c>LB2#-&f89$KH(;n9uX@$1 z*!lupDmqS(O8bWWP}b{Tcj5sD>_36WJM~za?x@sa2k$kTxy6jlS(g>3(Y~)Z>ZqBs zXHOhAPH*AXv=RkmuIL%ykj4|d^vcu-d~Q7B;m@VZmhiCmR`cd@?SNGa$uV9G}{p_*l9y{-}(?p(zHI&-kvsvy3<5Qr( zWoE>GvLERs&KwidT+CbTyX&sIe)`jY&wGQI<1%a`w{8Q^Qt9TjwQJY#xtRs?nJ9#D zNYASNZG2Mf$)P00RQZ>8lwf|u+sOXQm$xsv_#%T-LrRpeJk5_M0LFV&nY~dY`ae_u zU9e!w*Pi&=FMQz(ro`Gn3$-#nzd!ii4=%gxCkGvLK7Lyk3oAD(>kOCB} zSz=SKN^i8D9&An=(nn{RpK=EW3mi;UXh)t^p$JwbxNB$)a9ADo(Bji7EuM4ndpC49 z$K>IL-ZM13@m;&W^3V3wjh)stb;aOIe-l1`%8aiZ^?Pr+;2*Dl_!r|FgKN|z!sV;C zXv2TiCo9Q5nuh(OW#9fm)xo3V91^;OW^zLX)ztMBlH#L<{2Og2;NJEPen{fOB|MkM z8*_MZ7tiQzx9zr~Sz1X=0b1o`KUx34kM=71Qxdi1o?7lNz<%7nMP{Q2E&!uhGrSUo zD|uBy^IJ(u)~cx4qv9ewftUsxRP)29{n4NQ`Cn+F=Gvn>IjiHZU^73?r%sbd_S^l# za~^-g{wc@JfbRAPUICNm579nUTOzhk)z?wXX433b`fK_e5R5Gvs5BKlbJ~zj^AkX^dYkI|>R)umazc|0cy;XqJ>b z{JVY3n&8!lD)kxp@3MUdb~tGG+9#S%KR$f(_nY&#Zyj=SYxjl}Z4YW4@QcQ`Kh<3O6i=o!coWAl?{AIgjaSXaL-#aqyL|ZmTj8V0xCwZ&~9(l_bvUvJC+`uFu;SJw8aoAbo3X@l8r z`*ikXr^P-WeE5<3?z@i*vKL)^$qC0F$6XGWuUNs&4nMv5W=8ONzE1(n2ku+>#k0@i zv5wRJ=nv-3oo%0O@p6inyyS3(|IdB?j4yxr?Emji{y3U+_*?&LaMhiSlRwcq@I>7O zN=HgtF5YAKcfT<3*GCV3=kEqy`{#|Vw!xO50ZnvXfdT4=`Jvy_Vtzz~2yp+vi8D6j z8WN@UbzsjHB~dCkVdD6S&52Z9Pp=GUL4g~WxD6f{k1rC3pXi^1&v>jxy+KJ^Q-Hek$YF8aeEAWf`hYUKB5dI|*VwFch>gWEAoX^yMZ+yea zYi_;8M9|JAflCc~lOB({>sAkabhPpN;== z*Tz1V{@|*Y9L~)^@)ix?xt+MO%AACo1+gO6KQ&f1pWsIg8Pp-wj~~ReDkV+Xm^lvv zZT-!^Dj2nC*XUQ`zo_6gR3lPgwg0p+ivR`LuBZMvNQjxzqrr0id=bU3K7EBI4nu0?zA$vFy)ZvGk)4JayY`tG<29L*Ke@_4OP08hmSm zM+Z3^dFWQvH9Hf7+{!2KT>11*A6a|TU5{M$;F=qs+W7D=cVzICg->tjf>^wUBj)71 zkB_$&QY25>X{E!JALb^G#(J%HFo6*_>gySXxc!A&Fe!|aWB>>IuT7PI zI(MEO5P)lhx+t3Y@1ws3#eZZV7xH{WfEAMfC|kEhqE7A0A3^x&s{(_6Ub+9;SHFgl zvShRrk}8BCeJZu6@?TP`_Vd2?zQ0sP`KY_w7vaFsj*EX2m)_p^M(|^@PJ9Gn@~(c# zg**+YeV{7*0E16e+m{&rN3qCrz)W|~z4i*sP@uZ60uUQFc-lu!(~eay{1YLcrH#px zl+s=mznx$tihx;OHu8oC;( znGl1lREZ`*=P6UB@bON!bKAdEmRq0p-IV_hoOV22F8<&v}9x%uV)J zfBlK`&pSW(afP&z6x`?(Q&z#&7dzaM=F(%`Ez=-2#3#d3rUraR5Z%!~5b1fFM9HHJ z;Oj~`f+(ov12KS(p=B;ZKt>17LW1`UL|((JK*tU!3^sj6Tmnn_+wFlSlj#17HV6g+`;)V-hxdIYLKGxsR1dLacGyc+wq)uSUwqj+ zj=kW2|Kjlnhnj;dn(!u6UQ@)RNH4e(QL3XdD8K8Q`)3_H6}e(R>VHCBylmQkh3Wu) z)jl(bimmGZH=q35tFFB2jyvx7{oniDv(7qu^O-Y8_pf9;o%$E>r=NQIu}2@3ZR(%u zD6&G4P%G-V z3QiDMa4wSB&&bcte>-SUF*F$Z>pVY|q#==NQlZZFmF}`{i>W5qfDBHj9j2v=f64Rb zOhX$tKKbO6+?;>+-M_fu#v3oW_>v#~;0N4k7}}IpkbLHU+0t##`O254&zS!A|L6al zfBx6Gf=izxk3vQoiwN^R{N$5Qob|;oo_o$Y3^Z?j>s$8Nb5EAbB{jnQhpb{2e?@OB zbK}OVuDS|&WwFjl!1$`IV~JF7vpXtn9(dpZ$7iK7KV9Rm3gl*TpQ##1z{1q8Hl<?^7t!mWUC238Zq!4ANik||F#Z0wRQjPLw8&~@ZcSdn=fr{e@J7_ z;>L`5jhDP_;NSm7t9)0S-&(v!gIA4Co84Tt=g_PL4bJn|t{eW^Uo_tSPt6J356dUn zFB-b(dm?qp%*M`#HWuuvnqzs(&mMl{Zk^0-f9T-8$27T$qW~Wr_Fwo3!bC=A`wFaL zpMCi!OPF~lerxTTHC!}%{P8EQy5`!Wk3Qu=;;5&-^W`4zkEx(iQ92)xx=vwqz@ zzr2sfF6M4ATc`EO7+YK?-+ac5xpU@nFNoAO|AQNz8rph~!P$#g+aWtaWj6;0w^}+h zacc9{%bRPxil?Ef#cQp<_@%SQ>1+3DM=q1x`SZKC*>>5yE$7(`RXTm>+SO~$|JJvG z=eivC?(M$oZc7#~^yh&xKEEzK_)%<9()N(~5wnfK`|rE=I~QKG;_9nWZOs}gw>~}y zS*8{lfk%xJTWY^txojiF;8-KX9>fwjK{qNr)MiZz;V~;&{ zTR4A#20NA6T)%D&&U@!wcO81*fqYkD#WmM*ha+#zPn!9KrcXcd#CqNf6rZ>3>|yfHV1M zC}`uCO+`uxDfRz}G|N@{ zpAa;FVEF6eUyk79z>G$WeaSzc{u_qonrK)^J6-()C3fjw{5SOP;lD$FX>Me-stD>H z9|OP4e~gJ(3{!T(+`dGT%Ab<0Y-pS)a(hWxEH8&hs`hWfAM^OTN&IC0Qpd8PnRXM4 zs&GEPE=}22f9J2{1R{*5lm9Y5tom2Ae^v!6`~YIwD};rnyr?)5f-BZURUYulpdN`W zF>&&lN+s6h#-*N|ov}z03tNelj(E0ZU&sz0_0{mt;omaGN)QScB zUHWf1ZO1LA?eL+Y~9b7-~Y+w4@{Ube!!$~J5cey`nUNpslksL)o#d>bO06pA=W0NmO}{w zVx%&*6?|eMi)7ut*a9sn!eiqmLyXf*z3Q-wf63{uv@b`B=xpC!`-yxiL^`S@8K&*0 z^Rbi$>#%uOP|FF-^XzBIiP9+H2G#gX}SfD1FiXRq(1`3C_||7Un0ei$DgqOh$mOz8$g?0 z{zvlP`|rE|yz|dz2C36y87+GFtX)ln&knr!#V=mGa54Inve=1&f}g~{qY6KQ$^N_j z4m<3#&)%0WzkHnD6d@paZ^q~7(jQ*R)f-+#7OTHbKshr##L`3^ae;Y3`lC)lcNZRG&^GsYE zq5PVi+b)n)om{#iQ~;U1L=we0b?<%lIpd$t*m>7o5D^Ju@LzB?axb#_DdA4zMH;Y! zHrxP30aoFuR%7abUPtOL*-2l*X3Zu`A(Jda&kL>&EKic=GUi-1S+L$_(LJ7+@ zpYz>Ue(x$hNevcvI6*#}ueCGHl&S!9atvi~CouR1=C6wWD?vOin&XQDh~-`9m;4 zsiStM6e!M*0QZcu3vbZ8#jKfs`H9ot`sTNwQK%99kBYTj_0>NtyXZ`DWuG0FBXu#^ zP5Li+Bgp=>7nK!bVgGSrQ(ytF+7Eo-4U|H>vrxJzLVu565t))WL3-$+fB5V_tX%cr z_19guYSpUy@4ugC0}Reangf=1I1&__0}niiMcdtW+l{2Z{o7By@4fFk_uQ|2>8vkZ zef8C%o@p27YJ7|$*PWc)9&*MRXMXjoU*+xeAN$LXZNL5YL`JCUFMfEtjUHLj>N$lu zh&_U^wI>_m!8OOI4F0GQSs90TRS#?2B=IyK%WW&JzAD<+?Ki+DB)G`0dPG%nhg0qU z1y%YVmr>f6BSmz!Z@2ydExE8>m4w~>3di3CI}Y$Il}GPwJo@199apuk{95Cv_p~NW zYaD*c@a11=t-80l=85Kw7Z1U}Q01=}=d-eJ+ z@y~kvz4zYxA6&I+*^;e!7wJtm-+b-W*PVUtxtPXoyX|t@?LT9Qj+a~bwto!?^uoN+ z)?=Cae;beHGgBr{<%>?dBctuVtZlLSv)Oc-skE$1MAz3rAwE-=D6dw*GHP%zOud6>NRW6KIbdghwQ7_+-$;xg$uUgCi9p- zn4ZLsdK(#t{=<6|MFMQRayEQo-$)9kNI#X31EovD#goxvYe{D<~1kG zzkJ07mt1_uop&slzvcAl(*+^dlJOcYVWKVikKb&HDvmKi~Cde{ugA|DrFRz`6Q=O>|fbqX&Ug>IeTs zb5=2J&waN2t4~l_H;Xo=k*YlZ{yXQ@$E~^Np1XN!n^}$SG|`>w(QQA4y-*~r02WYMJmQ0Icw388p8OsxsB9KSF$?VBX9nlh#al z^zOUwdFlo}zBtfYtLMf1$`Pfrw9%MgU6|HTjJ~Tf)Z}wqD94P2VtB2fj`XKM-_F_x zq>s}qN7;E-6A`T6r&KA5#Y#i=d~U^SqBs-MhZ;>tR6FzIqyGGdpthi^KJtV<{3Aff z{0HM*mLvoxglzEdiYNZHZ;e5s2nubtZ%(T6uL6z5Rw|8PKOsWV?nk90L5)bRYWqq% zi3my}+Ud5R9{sIAwx8gj!cX7}&Pw#fxtX|FEAgpY!E4Pe1MA13v1h%1OzAfg_GMV#W+U z-u&$Sm$!8ud(1H_R$QSv>io?el}Jhr&B@H!XPs3RJCglSZ}W^Atis47$jU8K$}V=Q z_TOPYN;J5o3P?1XwkC()=&2>(7)B5+Q)0il0YMQmYlWqMQ)7TD`y_j@Wnqv{HRRzR z_K9fgA3cJmHI{=_tEi3SJZI#^v2$n0m)=Xty%=lxqMUj+^gv+fojbqekN1A1 zrqZ4GQj?1Nhoq|iE|@ywhbR2uQQ!Jo-QU40#ndd)U5Fw|f!`O1|MJ|~XFv0qH^23* z-}&wZC%^GzPKIZ0K9h=7+gBQQ+E1_kWLfp#L#uHd4TEY|SaB{_J`Nm{HYOt25FaYT z&8C86MUF`6vTjoh){Z;v^e<r?QEiZ=tM)&d{(bugw`b{pD9EilB7W7sR6_^v zNM|&v+GlV7U-~yK3G~yyiobr{y1)GRUotMpZlWMpAzOPi4hfX6bbj9c_IG^Z^wYVg zLq{fW(c;DLd;j}Reaor5f#ASzD*yH7``8zkcL*dBZ89x7X-bgxp!}nXcVgYV9h;=3 zOSa}YV%ciNekiE;uZ)8EsgxzRaYV84TCJ_N+Va4C_F1;=wtPi}4_BP!Z()!q|J6G? zcuIGd7awxS+_`gCuU&J+6)S%5qaUAe{PE<9TNH(&drSrwFIez~*PY0>$tL)PAPW~R zV3C3}D$5sbS3daQ6<1xg%Pu={gUrDPAGrL=70XwwSiETA?z`QV#e)35cOdIz;uyV!nD{lPh&6ob@$1i*7OUOb1vctbX zG(Sp{#&Wd6kB&_azl9&sW4)vusoIX{zYm`Eu^zY(4rr`q}$$6rTD|d-Nvx>DB+g zYWxk;f|k<%2S4Uio8JDv^lxJse+#}EG>BH~QD6?K{o4F1t=hhp9VO?4ykW^W3w;a8BSQ5XszXLZhgI+ z6BDml>e=?jdOa#fA$n~eui-#Uhd7$AzHRi==)&SsO^a)&E#BorZ=f#^aND6CvTIFg z>BB3WV(?9GKq#JX9GH&AloFSd<3v5S4i5XIO(LQ;3u+# z{?a<}BfXKg{>q-~n%X5n$^Von5Jzc8^_A*UD*M;!r&oXb-~XLAlVQLTK{&Ca(|&AY z{#Nr2J@nATUZAtJ?$5}7k3H`A&wTbX53GEU!qs<36}5Zl&!B(t#TVaw_uY#ZFP5J6 zU%eemC52L|jZUd>9oNKP@G=85ePvja?f>_!f>IJvk`w7xkdT_v-HkXYX%K0FO+t`v zkerk>NT*C%N$F;E$LO*A?{_@M^L~4=>)dsI<5S-V4#cm;oHFAr#5(<$nkp~o4^N%A z7m3p}G8uHfInln_6TlG28@ZM%ZJduFvVc5bEGiYFdp8D#=p2cAeJtwmS){F6F+kw+JknIai*wQL{4 zW(t414g_@NAF0mdQ0=9B1z6$-tSRx+9K5fIh=H7VYdQDW+Rlv|y)J~fl-${qUjk;* zGnu5Sa*DI$gAjv*CfUi`}+j5F6;<`PF}>7nGzkmC}{882PZkH)%04u3l=OSX-YX?3MfT4C4@*pkbz zaOz!oT;(}(P{S_{HanGV4szz6Pm^aXc^^o*S$n^~%w?86IG&PHARz55Q^=6NEy3sX zj35ly@uCd8GMTeia8n9B5l$EDsQ#Pd-h4rTHMe>JJ05H$e_Z)NF9?142$MMIrhSgh zd*nF?HEBV}Rw})v6BgvZ<4Uf_jrg-w?=VdCZh7@AGw3P=ZS_6Igghollj4&}joC+& z<`??CTF~{`7(p`xzu>C9hEOOm8LKiiN=hzdfI|1y_{GTf+4E;B%{{0lR*2&mU>=vW z9^~m>-R_FSEj<7uzYJ&q1`Vxz#O8v3T{To%w5`2PpB2ueMAXq&oH~=vK?mf01`*Yn zO@1$fm$5*1`OD#f9&6?Yzh2)xoNesF%~cPYQBu_I#utwsQ+kXoDFk0xAZvqv=dVd7 zC;o;wH}=3H)MMD6SXOtxyudDZbnhMn%m(CeYXwe+l^noj_KQ`>8MSjHvDdN?!acYo zWL+N-7aadF@z+f|fv^qvG+Zs~0lNpE{*M|BIoO{OG^=@Yae0vEzf4nyfUjqfGq+&v z*SeunQdQ3JQ-h*>N$9Xv;>eqO4_O0jz1tNm3eOzAzw{d7>MD#sd#6qiZSJ^MIp%qv zTr%aHHuhe-<82dUZ8larzx}Y*=tEz6Is_XFaEiaxINiZjZ*8yYu|Kl8^*#M{{avE< z@St)RtRq=V=JbO1v~hnblOXMOS`R*<%aaWnyTNa&Ip$rY-@wo}yJLB&j-MtWLygY= zQ%-;TAo*YMceagWHNdaiS*vHrqFYs~%9s9QN7jA|x4b|L+;Gt8it}#Xp!a%cy#(Y* zY(q5n6>!JF?>IynsLpJ4e~lEB%vEJhdIym&$}L!x#+%5N}PEJ$`B`h^Kzy)H%$&a z*7-}c^A)BnzXv~_1w1`#@~#D+t+SjE%ps}yvIm$6{!1HF!TME252okAgx^=wVK(md(Pek8YhNX?Q=x zURs~TN7F>_?4)u0^>rLcD|q4#`dMm1!{awo^53Tg{%o0;UP^F=LIVs|zL4tDL%*$Y z`Aivz<&}M^@iSgUPN-Q7PFHt4SvSIbp8d{y@J7oV^*)R{&@Bpnwi#)W`%d9SK9g*z zRr_g&hUm*bGJn<(5s?5brX8#J?3#avVmycxN(NI4iCtaenxrMHm0Vfj(yNHI9EJO> zc>VLWM9&1P@%pGU6{0JRgY=3l$TY{Ql!ys3lf&HW1QGXRVj?Ucr0e6YfidQc?FXo%zu_ zUgR#F&-OjR2e(@beHuYro*&)Hd}SxpS;n1x9jmM;1Ej)BFfD3I~$HH5n_H+ z_+(0GP~UE6u}iQu;rI}Po32!%-iN(fVX0UG3&Nb^TzE8z@82Jb#61oi|s9a@tM_10}Pr>8c!;p{Df(a;ungS-HrSr?-p)K4P-Qo`X zP$<*Jp!tz^WKQ0`iL!<`s9D9%3Hn{LqTZeJ3yq@zMJT6khp)K4rO?;bJpa{wDm;P; zjzCFi=Sts6?g0{1Xw-Lm;w%U(`3gDTXjsKrCXMbk8ajMF0~?{zC`y1On?(xW?MuYh zvHXln-W>F7S-Cc|?%(9yNstMmh;exunK}k~29+Iq8#paU$QCA9-Evn$l%#^OR9f`^ zLxkRj^qhh415)_y5pjagH=uxXv-X9%B%{z9#B57vy5i=uMMiA#;>8Qx*f)iHYQysv(cH;qFM8`N> zX|Ly$88E@c%89JovAAPpN1_fnXqXdxa007tI$(@1g}N z>HaTHyG-wQ-#soK&Gv5b9J6{6u8I<*TsHGOiL|};b;p~L*%0P4nW-{*5NeK zbL=+{?#5IT+zj_}JUik|{$uZZ5)H;h=6I|U2JTE@i!j9UORLUnY zF)#{zj9c87ySpNdnck@qp15~m&>N^2UGP)f;rzYYFU;3k$i-62Di|GBy)!x=`~x}I z{`jF}BhrupH8n~dDiN@$KJrUXRSOt_e+N}gqFObjwyOJvw9DsqFTEn-muKNckXrtiQsTlHSH*u z9}gb@gjkA-(<@D}Fmb~F!yR=t$l5?F%Do-mLM1;^>S$q7^s5*q7<29oIsm8Wjpg+Yv}DnjuMSpQQkNC2D(6tEU2 zo=wXugv|3FlfQ1gPusf(c%#%L)sAg}0&kiCIqV`AmCTSLj-$wC$`&oRrU zQ-J=vsf^DT`sYIygUn>l`JT<2500{hSCa;3YO7@dC<*1EICWlKk+q!pc}sLBS-Xl(C;+A8zNIH_mdQacqrRNr$rHS^y_IlnV4~+iv zXVt(QN^pHeO$?oJmVh~sE^A7uz&=3HaE5F1x_+#U%pGf;MTs`^MK#sNS)?D^;s12B zZh_LIQ}8G4!!-mV62~*Xcrh17`u9qRk*|E=O;@E10*ffM^j{`feL3fpSaLl6vJH}= zJ}}PIemTdjhj%?+?$R9XM%KqqcJ2M>17nCH8hEfK$;-e-3HKKYSLHBr?96?8>V-B4 zd9a}7YFX8W9Hg;=k;}2HE|O2)1-^+;z)^?bjOuU6Rq-XMMjE z+EvZ&pZH^q@p{Ikys1pHAM$dk?JSaIMbTM_w(@w|2gMOlTdEzZ=BCk1!rf}d^LCKP z^k(%e!5_TE39$J2?)5iT3$=65Qo45YFnD=dpv?>-6(`7@sK4E5;_EEtIsu2oiam<| z+-R5nTI-`T?)#5%F0O9nBE3`cA_E7?pi2VbKJPD|HfDvA6Kokzu(>{yuPq{sbb@cs zVhE0Q4T^X2%D%hL0tqH;bZy({PU!ZukzU|&bgjbfQEf2hGSM55_1vm9Xywp<9V^|l zpAS^FDycvyF`25-r-5tU0I}&b8h@RFw?<+n+a8_p<$SIXB3*qlhOWT6sGduk<$mU` z?TOWEcE?lbGpbdm&7j4wp+1XHhu+EcnrG zD*%6o8GFIzzuZ4Kogr+|el;75>{+R@uo}sc-6gmu=RAAu5Bq4r>=1lOyf|hB>@+7H@3}pF63sZt=)*7=9ZvQYY@{o?u7{87_&>gcSv@?^?p~ ziid+BcXvtO&!?9PbrsK@KV3}YSH3m$HJJLs8{4Oi)vxg}nFKcm!tOa>z5n99r`MnE z2bD4bpC_j@ zx7WYKiM!nD1@_B+wj|=vX9ax-+BQ(bt_@~d2mIw;@@R2GHI|cbL?7k(V>hT2g0n4^ zste8bbA2WcZ&rd%c?oJgLMoRf3_J=(djZSootiU*t=oOhHe5Yh*RN$y2`Rnq#$O*h zfj6z-8z*wX|Gip`?3HI5aZG^BOnAQ^r|iF_Jjs3O?b)_HJC2#!<4URWtl!EJ#H68uFJ& zms3xbuyRwQG5&oa(FIf-k`FoXynxH(6v^RVha+k~;o?FnLQxBeRZxg#)6%N~OAt2E z(lF4TVxntxNa1UXhKDWY_yFFHkwVa>4Es(L444msTtUm6X`YFR?mBsot}QP0u0!>& zL4Mm;>&GAcWuGyXHQs74x8n)4xdvWuLuGc`>^?mKi@)Oo5DV?g@`QJEd$-H%;Oqai z3qPB@JoDn55s`Wvv}Hna)0p5@HtSpD{dh<)UmszzNf|sN@X_MmyMd@$1*90q-zZ!O zh3zjW85MS4i$^b4+<)jg4Q3N?FCHk}%G8ojy@VvX+DNscpswmXDCUUgZjV#wbC?)F zAn&}z%q>5|NSmn+K0Mf-WU;Kw3AVFMGV!<0MYyvX=C|a2{2Q7mQU10yLgDl82cR6< z?9RKN0_;*-%srLGwZ08zkZ{~zgIKP7VD=~hI|32NJspv z(%*V1+2r{EZxVdIuaA^`z8Mz*pXnO1=x)IkL~-B`~>YG?7B7{o7Y{_B+5A!WebU2jzDQV74QW4c->dkJeEtbJ@pM%0cEi1TEK)~H8 zow*kl5_kJyGDk+seh9sq4HyAuNB^<|Uj6`JE>Ilr-37LZ4`Ifc9UXxI2P>z6H@CYX z*v^YF*jiYuuIw=E6_t9~Dwsb<&Sq?hEI`0TjHNI~j0QZMs)rB#ze;BE;_tM__Y-K!ij#-fMAs(g3 z;&+w$$EXBR&MWB9g4@y0UP_E^mfV*_o{`*_;VOI{4v>}TTDVOqTiY{{IWk!H(*X1#VV+0?Ul!p}wfG@xJ3s&)UFC8BoEe~GLlJ;Q|JVi0(#G|jE`)S6 z8ey^ct#zSz)X@!0c!#ln0np}c3~>bwB>k+;+;ZOP68NYtd?S1Y$ICZ{g-pWc$LFz9 z2q^rlrD>{Q)Wn|(eqq$rIN6HcSw9qYJ^>e_lJ0@Te;y>PL#_X&93tLN4Y+scg)?TS zeE7!_DJomq@69?q&XNmL+Wb6+7%L8qXJM8j_a0SYxs`cn21~^Jn#HhfPL-N~o$5#Q zs5QT!(EktlJ`F|R_B_{f(O!t7GH-hCu7E`nm`@+zAy;t@mA-qg9}uWZ9tUyG!GiW1 zL>|M;J4NRY*aFsizQlH+SI9j&E_7G(>ONFfUoQuZhu;8`{0+@Ff66;AEgpUw2(xB3 z3;E?WMa8VAFqth=S%)5G77cW?JF(59)rwe39Ms8E2-!Jk`lS722?%PDKQ9qQ;|Ndq znWruo6U`(}$e;$=Tr)HT0OB&9A=SQg?;exLSZQMPKkdVNz>+yHvK;#M#Y+V>x~K>G zhw}&ApL@?P#-^~4WyNtv+ry1D_j^v?dLHD`xk>?p0D^;e z+5FG}xv`Z){Oes3vbf%0WD}w$*2U!z ze*EtuJ-O}>CMx+$621HD%?-`-xfkv{D`IoquTAqQ=l$Wot`~C#-#%qQH9ST&8{r09 zM35(=Qt+Yk7vX?|GzLw$DN$x$;F{B4kFnUi-$A`*oddX$RimmoPjw z5fjM^J?}R#e}aly z|MAy|V`4b(u5VEUBWdYVCgy>Rllquba4+uNRYhBhslDAes^}G6Eakc)aoZjMxtAl6 z{52vT@GV(zh|K7qhYCxathA6RuR(eh(EaEsx2$aPz1#WIo^i`)XqZMu62ZnnNeo%O zk@-diuXAoxudmODH|YO8D5qGN1TV(*KPnu$yH1zjui)y$tNWi_cZhHXEK$F%L>Q(H0J8oDj3Ot6r=2`05^=!uHgST50 z{I=U@d9)`iWPUu8>x&;weK2sw$0izTH=4e^;MH4Q%F-)0)6T-OhT6R z@{5w54FgsVb&4g383KFyvS_EIN_${$f8vJK!QI4gY$Xg=3Bc|RKBi#22l5?x?un|F zI__FJ$xX8b+GzE|dLRU=DcDZxDj2C%n&%@y@@Lh}JGsP%_+XxM=&pu{TCz;w&~zqY z0)~g|b`VgG{!|{FfKaq3NLT*ws6l$_yY4{@OZ9^HP8Y6tPVe!c_p5`f1%<)0%$5M~ z@p7#@A=G|(sGAmKHIlFNJjpG}ui$BW5#FsO&5`7%;G!&A&%W>`VYc;hQtS2!x#tvh z8dvI$0r^v{Yd<4}rL!AXxHLW=3YYg>HEkf=CJtoq{C8M8N&nzJj2bb$)-b9Psf+E{ z;pHZ;P)Nj~I}>4}9T`-R8}WD+@ShHI7Vpd#zyPpaL;0uXZ9#zh3NnPf+jZp4y!6pl zXwhnQ?YcFte>M>6Ruz)syp8A@ObA*ym1H=55j-wjeSs&WqCpT}I^w-E9(Hbe#|JL$ zT><1V)U@#|ONs`Wc)tb5zh63^EvR1HteL4nl)1{Ah$QT?APF0=PO(U;u((|wYE*{X zJM8XwFxGLC`R(i^Gce0A2KS8=DcAM7JeB8s$k3YowV>t0(YfmF)UbyP3YSS31R}dh zrni)p-zloUjOXY5P0xEK(E`^EUYn7=!E#Q^PsqI;z>k{Vc9;U5J9633+|kd_X;mRM zXD_@1rK&~zCv<7hVXNpgCRKc=uY;%lMC{L~sFJJ2^`lMcP3oKHd-*h7DGJe-XL4Q* zpGKMcc=WQ~AfF>uKPHo|82slRSY_h-R6jm6l$3TbF;)ID3m9H=N_6P0T}At_@K+c6 zhS(}5E%X-_3crdM0bk{*S**#7W5WLMSjas|u z)4#{2~d;aZqoqqnBLIV4WEFObPn$=MWWroJ)6PeT0`) z`w=6)r{O4ClfkX%rPmxzc$b^nImRe;fRr zlk_ajY>@dUlAEZDjEy&MltXnsJhA7h$Hz|EcBI;rQW8cJV!DeiV>`M(P%Q(jHPS; z$Ax@dMcz4GhES?;0?D)zV*2<#f2JNNgp?~FoA3v_!@MgE709nj{RM+iUBIL-i1s-bRreHTfpef<9=kou+ z0)S5Gg;DIE_9AF?k$<;=Ua+cm)sQPY&LY$7;?|D`59r*27XC+?o_~OZwd>1Xf#vdN>e_kDU)^dmu6RZs< zsCE009U@ib`sOBYy#Ws!$Q}$#vR&H(x}NxNAc*lDzPEFhi7SJR(a(Nfv5y)7XTHgK zBRvy5<$M7C#pV6HdY)aE$D_UY!GBkbRG1}amdM!){*llYtetBqqtv-&#b}{B&{j-H z>)LtgsE(CF)UABJ6!UOci`<2kFCv8Dm9s@6EUUdoj}hO5kIbV3w;$~$18{?*o(k#X z#m>L(8C~&H6+ge-xijDZK*t7p%DtqZ%C&{sHsu3{`RQHo@w$|AE0C`Lu>KpA%X16>7v<>Af)4>QBD=pf}}zcnL9krUx5>UsD)4 zDC?f}a)c63X+Zs?u>}vNKn( z-m~WHUF6!l#5Kl4bE8j z-GdX7={*Jb|7t3P(_uMx_b^rx^t|g;Z$QBxlFpu{X_~Chw#5J}lQazxAVj)-aAv

    🧩 Components

    +

    🔥 Show Cases

    -**Mooncake Core Component: Transfer Engine (TE)** -The core of Mooncake is the Transfer Engine (TE), which provides a unified interface for batched data transfer across various storage devices and network links. Supporting multiple protocols including TCP, RDMA, CXL/shared-memory, and NVMe over Fabric (NVMe-of), TE is designed to enable fast and reliable data transfer for AI workloads. Compared to Gloo (used by Distributed PyTorch) and traditional TCP, TE achieves significantly lower I/O latency, making it a superior solution for efficient data transmission. +### Transfer Engine (TE) -**P2P Store and Mooncake Store** -Both P2P Store and Mooncake Store are built on the Transfer Engine and provide key/value caching for different scenarios. P2P Store focuses on sharing temporary objects (e.g., checkpoint files) across nodes in a cluster, preventing bandwidth saturation on a single machine. Mooncake Store, on the other hand, supports distributed pooled KVCache, specifically designed for XpYd disaggregation to enhance resource utilization and system performance. +The core of Mooncake is the Transfer Engine (TE), a high-performance data transfer framework. It provides a unified interface for batched data transfer across various storage devices and network links. Compared to Gloo (used by Distributed PyTorch) and traditional TCP, TE achieves significantly lower I/O latency, making it a superior solution for efficient data transmission. See the [Transfer Engine guide](https://kvcache-ai.github.io/Mooncake/design/transfer-engine/index.html) for details. -**Mooncake Integration with Leading LLM Inference Systems** -Mooncake has been seamlessly integrated with several popular large language model (LLM) inference systems. Through collaboration with the vLLM and SGLang teams, Mooncake now officially supports prefill-decode disaggregation. By leveraging the high-efficiency communication capabilities of RDMA devices, Mooncake significantly improves inference efficiency in prefill-decode disaggregation scenarios, providing robust technical support for large-scale distributed inference tasks. -In addition, Mooncake has been successfully integrated with SGLang's Hierarchical KV Caching, vLLM's prefill serving, and LMCache, augmenting KV cache management capabilities across large-scale inference scenarios. +Transfer Engine supports multiple communication protocols including TCP, RDMA (InfiniBand/RoCEv2/eRDMA/NVIDIA GPUDirect), AWS EFA, NVMe over Fabrics (NVMe-oF), NVLink, HIP, Barex, CXL, and Ascend-family transports. When built with the corresponding runtime, Transfer Engine can also detect and route accelerator memory on CUDA, MUSA, HIP, MACA, Cambricon MLU, and Ascend-enabled environments. For a complete list of supported protocols and configuration guide, see the [Supported Protocols Documentation](https://kvcache-ai.github.io/Mooncake/getting_started/supported-protocols.html). + +#### Highlights +- **Efficient use of multiple RDMA NIC devices.** Transfer Engine supports the use of multiple RDMA NIC devices to achieve the *aggregation of transfer bandwidth*. + +- **Topology-aware path selection.** Transfer Engine can *select optimal devices* based on the location (NUMA affinity, etc.) of both source and destination. + +- **More robust against temporary network errors.** Once transmission fails, Transfer Engine will try to use alternative paths for data delivery automatically. + +#### Performance +With 40 GB of data (equivalent to the size of the KVCache generated by 128k tokens in the LLaMA3-70B model), Mooncake Transfer Engine delivers up to **87 GB/s** and **190 GB/s** of bandwidth in 4×200 Gbps and 8×400 Gbps RoCE networks respectively, which are about **2.4x and 4.6x faster** than the TCP protocol. + + + + +### Mooncake Store + +Mooncake Store is built on the Transfer Engine and provides distributed key/value caching. It is a distributed KVCache storage engine specialized for LLM inference. The goal of Mooncake Store is to store the reusable KV caches across various locations in an inference cluster. Mooncake Store has been supported in [SGLang's Hierarchical KV Caching](https://lmsys.org/blog/2025-09-10-sglang-hicache/), [vLLM's prefill serving](https://docs.vllm.ai/en/latest/features/disagg_prefill.html) and is now integrated with [LMCache](https://kvcache-ai.github.io/Mooncake/getting_started/examples/lmcache-integration.html) to provide enhanced KVCache management capabilities. See the [Mooncake Store guide](https://kvcache-ai.github.io/Mooncake/design/mooncake-store.html). + +#### Highlights + +- **High bandwidth utilization**: Mooncake Store supports large-object striping, parallel I/O, and end-to-end zero-copy data transfer, fully utilizing aggregated bandwidth across multiple NICs. + +- **Multi-tier cache hierarchy**. Mooncake Store supports a multi-level cache design across DRAM and SSD/NVMe, enabling larger cache capacity. + +- **Elastic and disaggregated storage**: Mooncake Store decouples KVCache storage from inference engines, allowing storage nodes to be dynamically added or removed while keeping cached data independent from engine restarts, upgrades, and scheduling decisions. + +### Elastic Expert Parallelism Support -**Elastic Expert Parallelism Support** Mooncake adds elasticity and fault tolerance support for MoE model inference, enabling inference systems to remain responsive and recoverable in the event of GPU failures or changes in resource configuration. This functionality includes automatic faulty rank detection and can work with the EPLB module to dynamically route tokens to healthy ranks during inference. -**Tensor-Centric Ecosystem** -Mooncake establishes a full-stack, Tensor-oriented AI infrastructure where Tensors serve as the fundamental data carrier. The ecosystem spans from the Transfer Engine, which accelerates Tensor data movement across heterogeneous storage (DRAM/VRAM/NVMe), to the P2P Store and Mooncake Store for distributed management of Tensor objects (e.g., Checkpoints and KVCache), up to the Mooncake Backend enabling Tensor-based elastic distributed computing. This architecture is designed to maximize Tensor processing efficiency for large-scale model inference and training. +### Tensor-Centric Ecosystem + +Mooncake establishes a full-stack, Tensor-oriented AI infrastructure where Tensors serve as the fundamental data carrier. The ecosystem spans from the Transfer Engine, which accelerates Tensor data movement across heterogeneous storage (DRAM/VRAM/NVMe), to Mooncake Store for distributed management of Tensor objects (e.g., KVCache and model weight), up to the Mooncake Backend enabling Tensor-based elastic distributed computing. This architecture is designed to maximize Tensor processing efficiency for large-scale model inference and training. + +### SGLang Integration ([Guide](https://kvcache-ai.github.io/Mooncake/getting_started/examples/sglang-integration/hicache-integration-v1.html)) + +SGLang officially supports Mooncake Store as a [HiCache storage backend](https://lmsys.org/blog/2025-09-10-sglang-hicache/). This integration enables scalable KV cache retention and high-performance access for large-scale LLM serving scenarios. + +#### Highlights +- **Hierarchical KV Caching**: Mooncake Store serves as an external storage backend in SGLang's HiCache system, extending RadixAttention with multi-level KV cache storage across device, host, and remote storage layers. +- **Flexible Cache Management**: Supports multiple cache policies including write-through, write-through-selective, and write-back modes, with intelligent prefetching strategies for optimal performance. +- **Comprehensive Optimizations**: Features advanced data plane optimizations including page-first memory layout for improved I/O efficiency, zero-copy mechanisms for reduced memory overhead, GPU-assisted I/O kernels delivering fast CPU-GPU transfers, and layer-wise overlapping for concurrent KV cache loading while computation executes. +- **Elastic Expert Parallel**: Mooncake's collective communication backend and expert parallel kernels are integrated into SGLang to enable fault-tolerant expert parallel inference ([sglang#11657](https://github.com/sgl-project/sglang/pull/11657)). +- **Significant Performance Gains**: The multi-turn benchmark demonstrates substantial performance improvements over the non-HiCache setting. See our [benchmark report](https://kvcache-ai.github.io/Mooncake/performance/sglang-hicache-benchmark-results-v1.html) for more details. +- **Community Feedback**: Effective KV caching significantly reduces TTFT by eliminating redundant and costly re-computation. Integrating SGLang HiCache with the Mooncake service enables scalable KV cache retention and high-performance access. In our evaluation, we tested the DeepSeek-R1-671B model under PD-disaggregated deployment using in-house online requests sampled from a general QA scenario. On average, cache hits achieved an 84% reduction in TTFT compared to full re-computation. – Ant Group + +### vLLM Integration ([Guide v0.2](https://kvcache-ai.github.io/Mooncake/getting_started/examples/vllm-integration/vllm-integration-v0.2.html)) + +To optimize LLM inference, the vLLM community is working on supporting [disaggregated prefilling (PR 10502)](https://github.com/vllm-project/vllm/pull/10502). This feature allows separating the **prefill** phase from the **decode** phase in different processes. vLLM uses `nccl` and `gloo` as the transport layer by default, but currently it cannot efficiently decouple both phases in different machines. + +We have implemented vLLM integration, which uses Transfer Engine as the network layer instead of `nccl` and `gloo`, to support **inter-node KVCache transfer** [(PR 10884)](https://github.com/vllm-project/vllm/pull/10884). Transfer Engine provides simpler interfaces and more efficient use of RDMA devices. + +We will soon release the new vLLM integration based on Mooncake Store, which supports xPyD prefill/decode disaggregation. + +**_Update[Dec 16, 2024]: Here is the latest vLLM Integration ([Guide v0.2](https://kvcache-ai.github.io/Mooncake/getting_started/examples/vllm-integration/vllm-integration-v0.2.html)) that is based on vLLM's main branch._** + +#### Performance +By supporting Topology-Aware Path Selection and multi-card bandwidth aggregation, the mean TTFT of vLLM with Transfer Engine is up to 25% lower than traditional TCP-based transports. +In the future, we will further improve TTFT through GPUDirect RDMA and zero-copy. + +| Backend/Setting | Output Token Throughput (tok/s) | Total Token Throughput (tok/s) | Mean TTFT (ms) | Median TTFT (ms) | P99 TTFT (ms)| +|---------------------------------------------------------|---------------------------------|--------------------------------|----------------|------------------|---------------| +| Transfer Engine (RDMA) | 12.06 | 2042.74 | 1056.76 | 635.00 | 4006.59 | +| TCP | 12.05 | 2041.13 | 1414.05 | 766.23 | 6035.36 | + +- Click [here](https://kvcache-ai.github.io/Mooncake/performance/vllm-benchmark-results-v0.2.html) to access detailed benchmark results. + +**More advanced features are coming soon, so stay tuned!**

    🖥️ Supported Hardware

    @@ -152,81 +210,6 @@ Mooncake supports heterogeneous accelerators, NICs, and specialized transport pa | Sunrise Transport | Supported | Included here as an additional specialized transport path to reflect current hardware support positioning | | T-Head PPU / Barex | Supported | Barex-based transport coverage is available for T-Head PPU deployments | -

    🔥 Show Cases

    - -### Use Transfer Engine Standalone ([Guide](https://kvcache-ai.github.io/Mooncake/design/transfer-engine/index.html)) - -Transfer Engine is a high-performance data transfer framework. Transfer Engine provides a unified interface to transfer data from DRAM, VRAM or NVMe, while the technical details related to hardware are hidden. Transfer Engine supports multiple communication protocols including TCP, RDMA (InfiniBand/RoCEv2/eRDMA/NVIDIA GPUDirect), AWS EFA, NVMe over Fabric (NVMe-of), NVLink, HIP, Barex, CXL, and Ascend-family transports. When built with the corresponding runtime, Transfer Engine can also detect and route accelerator memory on CUDA, MUSA, HIP, MACA, Cambricon MLU, and Ascend-enabled environments. For a complete list of supported protocols and configuration guide, see the [Supported Protocols Documentation](https://kvcache-ai.github.io/Mooncake/getting_started/supported-protocols.html). - -#### Highlights -- **Efficient use of multiple RDMA NIC devices.** Transfer Engine supports the use of multiple RDMA NIC devices to achieve the *aggregation of transfer bandwidth*. - -- **Topology aware path selection.** Transfer Engine can *select optimal devices* based on the location (NUMA affinity, etc.) of both source and destination. - -- **More robust against temporary network errors.** Once transmission fails, Transfer Engine will try to use alternative paths for data delivery automatically. - -#### Performance -With 40 GB of data (equivalent to the size of the KVCache generated by 128k tokens in the LLaMA3-70B model), Mooncake Transfer Engine delivers up to **87 GB/s** and **190 GB/s** of bandwidth in 4×200 Gbps and 8×400 Gbps RoCE networks respectively, which are about **2.4x and 4.6x faster** than the TCP protocol. - - - - -### P2P Store ([Guide](https://kvcache-ai.github.io/Mooncake/design/p2p-store.html)) -P2P Store is built on the Transfer Engine and supports sharing temporary objects between peer nodes in a cluster. P2P Store is ideal for scenarios like checkpoint transfer, where data needs to be rapidly and efficiently shared across a cluster. -**P2P Store has been used in the checkpoint transfer service of Moonshot AI.** - -#### Highlights -- **Decentralized architecture.** P2P Store leverages a pure client-side architecture with global metadata managed by the etcd service. - -- **Efficient data distribution.** Designed to enhance the efficiency of large-scale data distribution, P2P Store *avoids bandwidth saturation* issues by allowing replicated nodes to share data directly. This reduces the CPU/RDMA NIC pressures of data providers (e.g., trainers). - - - - - -### Mooncake Store ([Guide](https://kvcache-ai.github.io/Mooncake/design/mooncake-store.html)) -Mooncake Store is a distributed KVCache storage engine specialized for LLM inference based on Transfer Engine. It is the central component of the KVCache-centric disaggregated architecture. The goal of Mooncake Store is to store the reusable KV caches across various locations in an inference cluster. Mooncake Store has been supported in [SGLang's Hierarchical KV Caching](https://lmsys.org/blog/2025-09-10-sglang-hicache/), [vLLM's prefill serving](https://docs.vllm.ai/en/latest/features/disagg_prefill.html) and is now integrated with [LMCache](https://kvcache-ai.github.io/Mooncake/getting_started/examples/lmcache-integration.html) to provide enhanced KVCache management capabilities. - -#### Highlights -- **Multi-replica support**: Mooncake Store supports storing multiple data replicas for the same object, effectively alleviating hotspots in access pressure. - -- **High bandwidth utilization**: Mooncake Store supports striping and parallel I/O transfer of large objects, fully utilizing multi-NIC aggregated bandwidth for high-speed data reads and writes. - -### SGLang Integration ([Guide](https://kvcache-ai.github.io/Mooncake/getting_started/examples/sglang-integration/hicache-integration-v1.html)) - -SGLang officially supports Mooncake Store as a [HiCache storage backend](https://lmsys.org/blog/2025-09-10-sglang-hicache/). This integration enables scalable KV cache retention and high-performance access for large-scale LLM serving scenarios. - -#### Highlights -- **Hierarchical KV Caching**: Mooncake Store serves as an external storage backend in SGLang's HiCache system, extending RadixAttention with multi-level KV cache storage across device, host, and remote storage layers. -- **Flexible Cache Management**: Supports multiple cache policies including write-through, write-through-selective, and write-back modes, with intelligent prefetching strategies for optimal performance. -- **Comprehensive Optimizations**: Features advanced data plane optimizations including page-first memory layout for improved I/O efficiency, zero-copy mechanisms for reduced memory overhead, GPU-assisted I/O kernels delivering fast CPU-GPU transfers, and layer-wise overlapping for concurrent KV cache loading while computation executes. -- **Elastic Expert Parallel**: Mooncake's collective communication backend and expert parallel kernels are integrated into SGLang to enable fault-tolerant expert parallel inference ([sglang#11657](https://github.com/sgl-project/sglang/pull/11657)). -- **Significant Performance Gains**: The multi-turn benchmark demonstrates substantial performance improvements over the non-HiCache setting. See our [benchmark report](https://kvcache-ai.github.io/Mooncake/performance/sglang-hicache-benchmark-results-v1.html) for more details. -- **Community Feedback**: Effective KV caching significantly reduces TTFT by eliminating redundant and costly re-computation. Integrating SGLang HiCache with the Mooncake service enables scalable KV cache retention and high-performance access. In our evaluation, we tested the DeepSeek-R1-671B model under PD-disaggregated deployment using in-house online requests sampled from a general QA scenario. On average, cache hits achieved an 84% reduction in TTFT compared to full re-computation. – Ant Group - -### vLLM Integration ([Guide v0.2](https://kvcache-ai.github.io/Mooncake/getting_started/examples/vllm-integration/vllm-integration-v0.2.html)) -To optimize LLM inference, the vLLM community is working on supporting [disaggregated prefilling (PR 10502)](https://github.com/vllm-project/vllm/pull/10502). This feature allows separating the **prefill** phase from the **decode** phase in different processes. The vLLM uses `nccl` and `gloo` as the transport layer by default, but currently it cannot efficiently decouple both phases in different machines. - -We have implemented vLLM integration, which uses Transfer Engine as the network layer instead of `nccl` and `gloo`, to support **inter-node KVCache transfer** [(PR 10884)](https://github.com/vllm-project/vllm/pull/10884). Transfer Engine provides simpler interfaces and more efficient use of RDMA devices. - -We will soon release the new vLLM integration based on Mooncake Store, which supports xPyD prefill/decode disaggregation. - -**_Update[Dec 16, 2024]: Here is the latest vLLM Integration ([Guide v0.2](https://kvcache-ai.github.io/Mooncake/getting_started/examples/vllm-integration/vllm-integration-v0.2.html)) that is based on vLLM's main branch._** - -#### Performance -By supporting Topology Aware Path Selection and multi-card bandwidth aggregation, Mean TTFT of vLLM with Transfer Engine is up to 25% lower than traditional TCP-based transports. -In the future, we will further improve TTFT through GPUDirect RDMA and zero-copy. - -| Backend/Setting | Output Token Throughput (tok/s) | Total Token Throughput (tok/s) | Mean TTFT (ms) | Median TTFT (ms) | P99 TTFT (ms)| -|---------------------------------------------------------|---------------------------------|--------------------------------|----------------|------------------|---------------| -| Transfer Engine (RDMA) | 12.06 | 2042.74 | 1056.76 | 635.00 | 4006.59 | -| TCP | 12.05 | 2041.13 | 1414.05 | 766.23 | 6035.36 | - -- Click [here](https://kvcache-ai.github.io/Mooncake/performance/vllm-benchmark-results-v0.2.html) to access detailed benchmark results. - -**More advanced features are coming soon, so stay tuned!** -

    🚀 Quick Start

    ### Before using Mooncake -- 2.34.1 From 268622fa706680b133749a1c145f53cf5c51b536 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E9=B9=A4=E7=94=B7?= Date: Thu, 4 Jun 2026 11:17:17 +0800 Subject: [PATCH 180/382] [TE] fix(efa): short-circuit same-process GPU loopback to avoid libfabric SHM segfault (#2298) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix: route EFA same-process loopback transfers through local copy The EFA provider's SHM intra-node path performs a host memcpy into FI_HMEM_CUDA device buffers and segfaults on the first same-host transfer (loopback self-transfer), e.g. checkpoint-engine P2P weight update on a single TP=8 node. See ofiwg/libfabric#12328. Detect same-process self-loopback in EfaContext::submitPostSend (peer NIC path equals our own nicPath(), whose server_name embeds the per-process RPC port, so the match guarantees the peer is this very process on this device) and satisfy the copy locally with a GPU-aware cudaMemcpy (cudaMemcpyDefault), bypassing EFA entirely. Same-host cross-process peers carry a different port, never match, and still go through EFA. The copy direction honors the slice opcode, mirroring fi_read/fi_write: WRITE copies source_addr -> dest_addr, READ copies dest_addr -> source_addr (the two are distinct local buffers, so it is not a symmetric self-copy). This mirrors how the RDMA transport already treats loopback as a special case (rdma_endpoint.cpp self-connected QP); RDMA relies on NIC hardware loopback and is unaffected by the libfabric SHM bug. Co-Authored-By: Claude Opus 4.8 * test: add EFA GPU loopback test for FI_HMEM_CUDA same-host transfers Add efa_gpu_loopback_test.cpp, the CUDA-device-memory counterpart of efa_transport_test.cpp (which only covers host/numa loopback). It reproduces the EFA SHM intra-node segfault on FI_HMEM_CUDA buffers (ofiwg/libfabric#12328) and validates EfaContext::tryLoopbackCopy: * GpuLoopbackWrite — same-host GPU WRITE must not crash. * GpuLoopbackWriteThenRead — WRITE then READ with byte-accurate verification, exercising both copy directions. * GpuLoopbackMultiWrite — batched GPU writes through the per-slice loopback short-circuit. Self-skips when no EFA device or no CUDA GPU is present. Registered under `USE_EFA AND USE_CUDA`. Co-Authored-By: Claude Opus 4.8 * efa: log fabric name and document same-host GPU loopback segfault Add fabric_attr->name to the EFA device init log so the active fabric (efa-rdm vs efa-direct) is visible at runtime — both share the same provider name, FI_EP_RDM type, and -rdm domain name, so the fabric name is the only field that distinguishes them. Revise the "Single-host loopback" doc section: - Correct the same-host fast-path attribution: the memcpy fast path is supplied by the SHM provider (FI_EFA_ENABLE_SHM_TRANSFER, default on), not by FI_EFA_USE_DEVICE_RDMA. Confirmed at runtime: a default (device-RDMA-enabled) config still reports "Opened fabric: shm". - Add a warning that the default SHM path host-memcpy's into FI_HMEM_CUDA destinations and segfaults on GPU buffers (ofiwg/libfabric#12328); document the same-process tryLoopbackCopy short-circuit and the FI_EFA_ENABLE_SHM_TRANSFER=0 workaround for cross-process GPU peers. Co-Authored-By: Claude Opus 4.8 --------- Co-authored-by: Claude Opus 4.8 --- .../design/transfer-engine/efa_transport.md | 58 +++- .../transport/efa_transport/efa_context.h | 13 + .../transport/efa_transport/efa_context.cpp | 71 ++++ mooncake-transfer-engine/tests/CMakeLists.txt | 26 ++ .../tests/efa_gpu_loopback_test.cpp | 324 ++++++++++++++++++ 5 files changed, 484 insertions(+), 8 deletions(-) create mode 100644 mooncake-transfer-engine/tests/efa_gpu_loopback_test.cpp diff --git a/docs/source/design/transfer-engine/efa_transport.md b/docs/source/design/transfer-engine/efa_transport.md index 1f811a9e..dcb0637e 100644 --- a/docs/source/design/transfer-engine/efa_transport.md +++ b/docs/source/design/transfer-engine/efa_transport.md @@ -120,7 +120,7 @@ result = te.initialize('127.0.0.1', 'P2PHANDSHAKE', 'efa', '') print(f'Initialize result: {result}') # Should be 0 # You should see logs like: -# EFA device (libfabric): rdmap79s0, domain: rdmap79s0-rdm, provider: efa +# EFA device (libfabric): rdmap79s0, domain: rdmap79s0-rdm, fabric: efa, provider: efa ``` ## Unit Tests @@ -374,18 +374,60 @@ Tested on two p5.48xlarge instances (AMD EPYC 7R13, 8× H100 80GB, 32 EFA device ### Single-host loopback -EFA NICs have no hardware loopback short-circuit: even when both endpoints resolve to the same host, `fi_write`/`fi_read` drive a real DMA round-trip through the EFA device. For deployments where the producer and consumer run as **separate processes on the same host** (single-machine development, benchmarks, co-located workers), this is strictly slower than libfabric's emulated RDMA path, which resolves the same-host case to a memcpy and skips the NIC entirely. +EFA NICs have no hardware loopback short-circuit: when a transfer's source and +destination resolve to the same host, the data does not go out on the wire as +GPUDirect/device RDMA. libfabric handles the same-host case in software, and +there are **two distinct provider knobs** that select how: -Set `FI_EFA_USE_DEVICE_RDMA=0` (a libfabric provider env, [documented by the EFA installer](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa-runtime-tuning.html)) on processes that only do same-host transfers; leave it unset (default `1`) on cross-host or mixed processes. Mooncake does not wrap this knob — it is a provider-level flag resolved at `fi_getinfo` time, and a single `EfaTransport` instance may serve a mix of loopback and cross-host peers, so flipping it disables device RDMA for **every** transfer in the process. Apply it per-process based on that process's traffic pattern. +- **`FI_EFA_ENABLE_SHM_TRANSFER`** (default `1`, on): when on, the EFA + provider routes same-host peers through the **`shm` provider** — verifiable + at runtime, where libfabric reports `Opened fabric: shm` alongside + `Opened fabric: efa` even on a default (device-RDMA-enabled) configuration. + This SHM path is the one that supplies the same-host memcpy fast path; it is + active **by default**, independent of `FI_EFA_USE_DEVICE_RDMA`. +- **`FI_EFA_USE_DEVICE_RDMA`** (default `1` after #2041): controls whether the + EFA RDM data path uses device RDMA vs libfabric's emulated RDM path. It is a + provider-level flag resolved at `fi_getinfo` time; Mooncake does not wrap it. -Measured on p5.48xlarge (1 NIC, ~1.2 GiB per `put_from` call, same-host producer/consumer): +```{warning} +**GPU (FI_HMEM_CUDA) buffers — known segfault.** The default same-host **SHM** +path (`FI_EFA_ENABLE_SHM_TRANSFER=1`) performs a **host `memcpy` into the +destination buffer** during SHM SAR reassembly, *without* honoring an +`FI_HMEM_CUDA` destination's iface. On a GPU buffer this writes host memory +straight into a device pointer and **segfaults** on the first same-host +transfer — `__memcpy_avx_unaligned` ← `ofi_copy_to_mr_iov` ← `smr_copy_from_sar` +← `efa_rdm_cq_readfrom` ← `fi_cq_read`. Reported upstream as +[ofiwg/libfabric#12328](https://github.com/ofiwg/libfabric/issues/12328). -| `FI_EFA_USE_DEVICE_RDMA` | per-write latency | +- **Same-process self-loopback** (e.g. a TP-colocated rank reading its own + registered GPU weights — checkpoint-engine p2p weight update) is handled + inside Mooncake: `EfaContext::tryLoopbackCopy` detects a same-process peer + (matched on `local_server_name`, which embeds this process's unique RPC + port) and satisfies the transfer with a local `cudaMemcpy` instead of routing + it over EFA, so it never reaches the broken SHM path. +- **Same-host cross-process GPU transfers** are *not* short-circuited (the + peer is a different process / address space). Until libfabric#12328 is fixed, + set `FI_EFA_ENABLE_SHM_TRANSFER=0` on such processes — same-host transfers + then fall back to device RDMA, which is GPU-aware and correct. +``` + +For **host (DRAM) buffers** the SHM memcpy path is safe (host→host copy) and is +the same-host fast path the measurements below exercise. + +Measured on p5.48xlarge (1 NIC, ~1.2 GiB per `put_from` call, host DRAM buffer, +same-host producer/consumer in **separate processes**): + +| same-host path | per-write latency | |---|---:| -| `1` (default after #2041, device RDMA) | ~830 ms | -| `0` (emulated, same-host memcpy fast path) | ~390 ms | +| device RDMA (NIC round-trip, no fast-path for loopback) | ~830 ms | +| SHM memcpy fast path (default) | ~390 ms | -For reference, a cross-host `put_from` of the same payload (device RDMA, 1 NIC) is ~340 ms — i.e., device RDMA on a same-host loopback is *slower* than going over the wire to another host, because the NIC has no fast-path for loopback. Cross-host benchmarks are unaffected by this env: leave `FI_EFA_USE_DEVICE_RDMA` at its default on any process that also talks to remote peers. +For reference, a cross-host `put_from` of the same payload (device RDMA, 1 NIC) +is ~340 ms — i.e., driving a same-host loopback through the NIC is *slower* than +going over the wire to another host, because the NIC has no fast-path for +loopback. Cross-host transfers always use device RDMA and are unaffected by +`FI_EFA_ENABLE_SHM_TRANSFER`: leave it at its default on any process that also +talks to remote peers. ### Tuning Tips diff --git a/mooncake-transfer-engine/include/transport/efa_transport/efa_context.h b/mooncake-transfer-engine/include/transport/efa_transport/efa_context.h index c43c7e62..067c46d1 100644 --- a/mooncake-transfer-engine/include/transport/efa_transport/efa_context.h +++ b/mooncake-transfer-engine/include/transport/efa_transport/efa_context.h @@ -125,6 +125,19 @@ class EfaContext { // Submit slices for transfer int submitPostSend(const std::vector& slice_list); + // Same-process self-loopback fast path. When a slice's resolved peer + // NIC path equals our own (same server_name — which embeds the + // per-process RPC port — AND same device), source and destination are + // both valid pointers in THIS process's address space. We satisfy the + // copy with a local memcpy / cudaMemcpy instead of issuing it over EFA. + // + // This avoids libfabric's EFA SHM intra-node path, which performs a + // host memcpy into FI_HMEM_CUDA device buffers and segfaults + // (ofiwg/libfabric#12328). Returns true if the slice was handled here + // (and marked success/failed); false if it should fall through to the + // normal EFA submit path. + bool tryLoopbackCopy(Transport::Slice* slice); + // Hot-path submit: post a batch of slices to `peer_fi_addr` via the // shared endpoint. Handles WR / CQ reservation, MR descriptor prep, // and the fi_write / fi_read burst under post_lock_. Called by diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp index 54807e77..50f5e62c 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp @@ -193,6 +193,7 @@ int EfaContext::construct(size_t num_cq_list, size_t max_cqe, LOG(INFO) << "EFA device (libfabric): " << device_name_ << ", domain: " << fi_info_->domain_attr->name + << ", fabric: " << fi_info_->fabric_attr->name << ", provider: " << fi_info_->fabric_attr->prov_name << " (shared endpoint, max_wr=" << max_wr_depth_ << ")"; @@ -626,6 +627,72 @@ void EfaContext::removePeerAddr(fi_addr_t fi_addr) { } } +bool EfaContext::tryLoopbackCopy(Transport::Slice* slice) { + // Only intra-process self-transfers qualify. We compare the SERVER name + // (host:rpc_port), NOT the full nic path: local_server_name embeds this + // process's unique RPC port, so a server-name match guarantees the peer + // is *this very process* — meaning both source_addr and dest_addr are + // pointers we can dereference directly. We deliberately ignore the + // device suffix: with multiple NICs the source slice is routed by its + // source buffer's device while peer_nic_path names the destination + // buffer's device, so the two device names often differ even for a pure + // self-loopback. Same-host *cross-process* peers carry a different RPC + // port and never match, so we never memcpy across address spaces. + if (getServerNameFromNicPath(slice->peer_nic_path) != + engine_.local_server_name()) { + return false; + } + + // Direction depends on opcode, mirroring fi_read / fi_write below: + // WRITE: fi_write(buf=source_addr -> addr=dest_addr) data src->dst + // READ : fi_read (buf=source_addr <- addr=dest_addr) data dst->src + // source_addr and rdma.dest_addr are two distinct local buffers here, so + // the copy is NOT symmetric — we must honor the opcode's direction. + void* local_buf = slice->source_addr; + void* remote_buf = reinterpret_cast(slice->rdma.dest_addr); + void* dst; + void* src; + if (slice->opcode == Transport::TransferRequest::READ) { + dst = local_buf; // read INTO local + src = remote_buf; // FROM remote (== local) buffer + } else { + dst = remote_buf; // write INTO remote (== local) buffer + src = local_buf; // FROM local + } + size_t len = slice->length; + // GPU-aware copy. Guard on the SAME backends that register GPU memory + // as FI_HMEM (see the FI_MR_HMEM hint and the registration path: only + // USE_CUDA / USE_HIP tag MRs with a device iface). Every other build — + // including non-EFA GPU backends such as MUSA/MLU/MACA, which never run + // on AWS EFA hardware — registers loopback buffers as host memory, so a + // plain memcpy is both correct and the only portable option (those + // backends do not expose the cuda* symbols). *MemcpyDefault picks + // H2H/H2D/D2H/D2D from the pointer attributes. +#if defined(USE_CUDA) + auto rc = cudaMemcpy(dst, src, len, cudaMemcpyDefault); + if (rc != cudaSuccess) { + LOG(ERROR) << "EFA loopback cudaMemcpy failed: " + << cudaGetErrorString(rc) << " (dst=" << dst + << ", src=" << src << ", len=" << len << ")"; + slice->markFailed(); + return true; + } +#elif defined(USE_HIP) + auto rc = hipMemcpy(dst, src, len, hipMemcpyDefault); + if (rc != hipSuccess) { + LOG(ERROR) << "EFA loopback hipMemcpy failed: " << hipGetErrorString(rc) + << " (dst=" << dst << ", src=" << src << ", len=" << len + << ")"; + slice->markFailed(); + return true; + } +#else + memcpy(dst, src, len); +#endif + slice->markSuccess(); + return true; +} + int EfaContext::submitPostSend( const std::vector& slice_list) { // Route slices to appropriate peer handles. Group by peer NIC path. @@ -638,6 +705,8 @@ int EfaContext::submitPostSend( // Fast path: peer info already filled in by the caller // (dest_rkey and peer_nic_path set on the slice before dispatch). if (!slice->peer_nic_path.empty()) { + // Self-loopback: satisfy locally, skip EFA entirely. + if (tryLoopbackCopy(slice)) continue; slices_by_peer[slice->peer_nic_path].push_back(slice); continue; } @@ -669,6 +738,8 @@ int EfaContext::submitPostSend( "@" + peer_segment_desc->devices[device_id].name; slice->peer_nic_path = peer_nic_path; + // Self-loopback: satisfy locally, skip EFA entirely. + if (tryLoopbackCopy(slice)) continue; slices_by_peer[peer_nic_path].push_back(slice); } diff --git a/mooncake-transfer-engine/tests/CMakeLists.txt b/mooncake-transfer-engine/tests/CMakeLists.txt index 7fc5af5d..70f5be2a 100644 --- a/mooncake-transfer-engine/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tests/CMakeLists.txt @@ -91,6 +91,32 @@ if (USE_EFA) add_executable(efa_transfer_test ${WORKSPACE}/efa_transfer_test.cpp) target_link_libraries(efa_transfer_test PUBLIC transfer_engine gflags::gflags glog::glog) + + # GPU (CUDA device memory) loopback test — reproduces the EFA SHM + # intra-node segfault on FI_HMEM_CUDA buffers (ofiwg/libfabric#12328) + # and validates EfaContext::tryLoopbackCopy. Needs CUDA headers/libs. + if (USE_CUDA) + add_executable(efa_gpu_loopback_test ${WORKSPACE}/efa_gpu_loopback_test.cpp) + # Resolve CUDA include dirs / runtime via CUDAToolkit instead of a + # hardcoded /usr/local/cuda/include, so the test builds wherever CUDA + # lives (e.g. the DLAMI pip venv layout). find_package is idempotent + # and may not have run yet in this scope (top-level only calls it under + # WITH_EP), so request it here; fall back to the legacy path if the + # module variant is unavailable. + find_package(CUDAToolkit QUIET) + if (CUDAToolkit_FOUND) + target_include_directories(efa_gpu_loopback_test + PRIVATE ${CUDAToolkit_INCLUDE_DIRS}) + target_link_libraries(efa_gpu_loopback_test + PUBLIC transfer_engine gtest gtest_main CUDA::cudart) + else() + target_include_directories(efa_gpu_loopback_test + PRIVATE /usr/local/cuda/include) + target_link_libraries(efa_gpu_loopback_test + PUBLIC transfer_engine gtest gtest_main cudart) + endif() + add_test(NAME efa_gpu_loopback_test COMMAND efa_gpu_loopback_test) + endif() endif() # UB transport test with URMA endpoint and mock support diff --git a/mooncake-transfer-engine/tests/efa_gpu_loopback_test.cpp b/mooncake-transfer-engine/tests/efa_gpu_loopback_test.cpp new file mode 100644 index 00000000..a7735d77 --- /dev/null +++ b/mooncake-transfer-engine/tests/efa_gpu_loopback_test.cpp @@ -0,0 +1,324 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// EFA GPU (CUDA device memory) loopback test. +// +// This is the GPU-memory counterpart of efa_transport_test.cpp, which only +// exercises loopback on host (numa) buffers. The distinction matters: the +// libfabric EFA provider's SHM intra-node path does a host memcpy into +// FI_HMEM_CUDA device buffers and segfaults on the first same-host transfer +// (ofiwg/libfabric#12328). Host-memory loopback never hits that bug, so it +// went unnoticed by the existing tests. +// +// Mooncake's fix (EfaContext::tryLoopbackCopy) detects same-process +// self-loopback and satisfies it with a GPU-aware cudaMemcpy instead of +// routing it over EFA. This test reproduces the crash WITHOUT the fix and +// verifies correct, byte-accurate data movement WITH it — for both WRITE and +// READ opcodes (the two copy in opposite directions). +// +// Requires EFA hardware (fi_info -p efa) AND at least one CUDA GPU. Self- +// skips cleanly when either is absent. + +#include +#include +#include +#include + +#include +#include +#include + +#include "transfer_engine.h" +#include "transport/efa_transport/efa_transport.h" +#include "transport/transport.h" + +using namespace mooncake; + +namespace mooncake { + +namespace { +bool cudaAvailable() { + int n = 0; + return cudaGetDeviceCount(&n) == cudaSuccess && n > 0; +} +} // namespace + +class EFAGpuLoopbackTest : public ::testing::Test { + protected: + void SetUp() override { + google::InitGoogleLogging("EFAGpuLoopbackTest"); + FLAGS_logtostderr = 1; + + const char *env = std::getenv("MC_METADATA_SERVER"); + metadata_server_ = env ? env : "P2PHANDSHAKE"; + + env = std::getenv("MC_LOCAL_SERVER_NAME"); + local_server_name_ = env ? env : "127.0.0.1:12345"; + } + + void TearDown() override { google::ShutdownGoogleLogging(); } + + struct EngineSetup { + std::unique_ptr engine; + Transport *xport = nullptr; + void *dev_buf = nullptr; // CUDA device buffer + size_t buffer_size = 0; + SegmentID segment_id = 0; + bool ok = false; + }; + + // Create engine, install EFA transport, allocate + register a CUDA + // device buffer, and open our own segment for loopback. + EngineSetup createEngine(size_t buffer_size = 1ull << 26 /* 64 MB */) { + EngineSetup s; + s.buffer_size = buffer_size; + + if (cudaSetDevice(0) != cudaSuccess) { + LOG(WARNING) << "cudaSetDevice(0) failed"; + return s; + } + + s.engine = std::make_unique(false); + s.engine->getLocalTopology()->discover({}); + auto hp = parseHostNameWithPort(local_server_name_); + int rc = s.engine->init(metadata_server_, local_server_name_, + hp.first.c_str(), hp.second); + EXPECT_EQ(rc, 0) << "engine->init failed"; + if (rc != 0) return s; + + s.xport = s.engine->installTransport("efa", nullptr); + EXPECT_NE(s.xport, nullptr) << "installTransport(\"efa\") failed"; + if (!s.xport) return s; + + cudaError_t cerr = cudaMalloc(&s.dev_buf, buffer_size); + EXPECT_EQ(cerr, cudaSuccess) + << "cudaMalloc failed: " << cudaGetErrorString(cerr); + if (cerr != cudaSuccess) return s; + + // Register as GPU memory so the EFA transport tags the MR with + // FI_HMEM_CUDA (the registration path under test). + rc = s.engine->registerLocalMemory(s.dev_buf, buffer_size, "cuda:0"); + EXPECT_EQ(rc, 0) << "registerLocalMemory(cuda:0) failed"; + if (rc != 0) return s; + + auto actual_addr = s.engine->getLocalIpAndPort(); + s.segment_id = s.engine->openSegment(actual_addr); + s.ok = true; + return s; + } + + void destroyEngine(EngineSetup &s) { + if (s.engine && s.dev_buf) { + s.engine->unregisterLocalMemory(s.dev_buf); + } + if (s.dev_buf) { + cudaFree(s.dev_buf); + s.dev_buf = nullptr; + } + } + + bool submitAndWait(TransferEngine *engine, SegmentID segment_id, + void *source, uint64_t target_offset, size_t length, + TransferRequest::OpCode opcode) { + auto batch_id = engine->allocateBatchID(1); + TransferRequest entry; + entry.opcode = opcode; + entry.length = length; + entry.source = (uint8_t *)source; + entry.target_id = segment_id; + entry.target_offset = target_offset; + + Status s = engine->submitTransfer(batch_id, {entry}); + if (!s.ok()) { + LOG(ERROR) << "submitTransfer failed: " << s.ToString(); + engine->freeBatchID(batch_id); + return false; + } + + TransferStatus status; + const int kMaxPollIterations = 2000000; + for (int i = 0; i < kMaxPollIterations; ++i) { + s = engine->getTransferStatus(batch_id, 0, status); + if (!s.ok()) { + engine->freeBatchID(batch_id); + return false; + } + if (status.s == TransferStatusEnum::COMPLETED) { + engine->freeBatchID(batch_id); + return true; + } + if (status.s == TransferStatusEnum::FAILED) { + LOG(ERROR) << "Transfer FAILED"; + engine->freeBatchID(batch_id); + return false; + } + } + LOG(ERROR) << "Transfer timed out"; + engine->freeBatchID(batch_id); + return false; + } + + std::string metadata_server_; + std::string local_server_name_; +}; + +// Test 1: GPU loopback WRITE must not crash. +// +// Without the fix this segfaults inside the EFA provider's SHM path +// (host memcpy into a device pointer). With the fix it completes via a +// local cudaMemcpy. +TEST_F(EFAGpuLoopbackTest, GpuLoopbackWrite) { + if (!cudaAvailable()) GTEST_SKIP() << "No CUDA GPU present"; + auto setup = createEngine(); + if (!setup.ok) GTEST_SKIP() << "EFA/CUDA setup unavailable"; + + auto segment_desc = + setup.engine->getMetadata()->getSegmentDescByID(setup.segment_id); + ASSERT_NE(segment_desc, nullptr); + uint64_t remote_base = (uint64_t)segment_desc->buffers[0].addr; + + const size_t kDataLength = 4096; + ASSERT_EQ(cudaMemset(setup.dev_buf, 0xAB, kDataLength), cudaSuccess); + ASSERT_EQ(cudaDeviceSynchronize(), cudaSuccess); + + // Write src(=dev_buf) -> remote(=dev_buf + offset), both on the GPU. + bool ok = submitAndWait(setup.engine.get(), setup.segment_id, setup.dev_buf, + remote_base + (setup.buffer_size / 2), kDataLength, + TransferRequest::WRITE); + EXPECT_TRUE(ok) << "GPU loopback write should succeed"; + + destroyEngine(setup); +} + +// Test 2: GPU loopback WRITE then READ, with byte-accurate verification. +// +// The first half holds the source pattern; we WRITE it into the second +// half, scribble the first half, then READ the second half back into the +// first half and compare. This exercises BOTH copy directions of +// tryLoopbackCopy (WRITE: src->dst, READ: dst->src) and proves the data is +// actually moved correctly, not merely "did not crash". +TEST_F(EFAGpuLoopbackTest, GpuLoopbackWriteThenRead) { + if (!cudaAvailable()) GTEST_SKIP() << "No CUDA GPU present"; + auto setup = createEngine(); + if (!setup.ok) GTEST_SKIP() << "EFA/CUDA setup unavailable"; + + auto segment_desc = + setup.engine->getMetadata()->getSegmentDescByID(setup.segment_id); + ASSERT_NE(segment_desc, nullptr); + uint64_t remote_base = (uint64_t)segment_desc->buffers[0].addr; + + const size_t kDataLength = 4ull << 20; // 4 MB + ASSERT_LE(2 * kDataLength, setup.buffer_size); + + uint8_t *dev = (uint8_t *)setup.dev_buf; + const uint64_t second_half = setup.buffer_size / 2; + + // Build a known pattern on the host and copy it into the first half. + std::vector pattern(kDataLength); + for (size_t i = 0; i < kDataLength; ++i) + pattern[i] = (uint8_t)(i * 131 + 7); + ASSERT_EQ( + cudaMemcpy(dev, pattern.data(), kDataLength, cudaMemcpyHostToDevice), + cudaSuccess); + + // WRITE: first half -> second half (loopback). + ASSERT_TRUE(submitAndWait(setup.engine.get(), setup.segment_id, dev, + remote_base + second_half, kDataLength, + TransferRequest::WRITE)) + << "WRITE should succeed"; + + // Corrupt the first half so the READ-back has to actually move bytes. + ASSERT_EQ(cudaMemset(dev, 0x00, kDataLength), cudaSuccess); + ASSERT_EQ(cudaDeviceSynchronize(), cudaSuccess); + + // READ: second half -> first half (loopback, opposite direction). + ASSERT_TRUE(submitAndWait(setup.engine.get(), setup.segment_id, dev, + remote_base + second_half, kDataLength, + TransferRequest::READ)) + << "READ should succeed"; + + // Verify the first half now matches the original pattern. + std::vector readback(kDataLength, 0xFF); + ASSERT_EQ( + cudaMemcpy(readback.data(), dev, kDataLength, cudaMemcpyDeviceToHost), + cudaSuccess); + EXPECT_EQ(0, memcmp(readback.data(), pattern.data(), kDataLength)) + << "READ-back GPU data should match the written pattern"; + + destroyEngine(setup); +} + +// Test 3: batch of GPU loopback writes (multiple slices), to exercise the +// per-slice loopback short-circuit inside the grouping loop. +TEST_F(EFAGpuLoopbackTest, GpuLoopbackMultiWrite) { + if (!cudaAvailable()) GTEST_SKIP() << "No CUDA GPU present"; + auto setup = createEngine(); + if (!setup.ok) GTEST_SKIP() << "EFA/CUDA setup unavailable"; + + auto segment_desc = + setup.engine->getMetadata()->getSegmentDescByID(setup.segment_id); + ASSERT_NE(segment_desc, nullptr); + uint64_t remote_base = (uint64_t)segment_desc->buffers[0].addr; + + const size_t kSliceLen = 65536; + const int kBatchSize = 16; + const uint64_t dst_region = setup.buffer_size / 2; + ASSERT_LE(dst_region + (uint64_t)kBatchSize * kSliceLen, setup.buffer_size); + + ASSERT_EQ(cudaMemset(setup.dev_buf, 0x5A, kBatchSize * kSliceLen), + cudaSuccess); + ASSERT_EQ(cudaDeviceSynchronize(), cudaSuccess); + + auto batch_id = setup.engine->allocateBatchID(kBatchSize); + std::vector requests; + for (int i = 0; i < kBatchSize; ++i) { + TransferRequest entry; + entry.opcode = TransferRequest::WRITE; + entry.length = kSliceLen; + entry.source = (uint8_t *)setup.dev_buf + i * kSliceLen; + entry.target_id = setup.segment_id; + entry.target_offset = remote_base + dst_region + i * kSliceLen; + requests.push_back(entry); + } + + Status s = setup.engine->submitTransfer(batch_id, requests); + ASSERT_TRUE(s.ok()) << "submitTransfer failed: " << s.ToString(); + + for (int task_id = 0; task_id < kBatchSize; ++task_id) { + TransferStatus status; + const int kMaxPollIterations = 2000000; + for (int i = 0; i < kMaxPollIterations; ++i) { + s = setup.engine->getTransferStatus(batch_id, task_id, status); + ASSERT_TRUE(s.ok()); + if (status.s == TransferStatusEnum::COMPLETED) break; + ASSERT_NE(status.s, TransferStatusEnum::FAILED) + << "task " << task_id << " failed"; + } + ASSERT_EQ(status.s, TransferStatusEnum::COMPLETED) + << "task " << task_id << " did not complete"; + } + + s = setup.engine->freeBatchID(batch_id); + ASSERT_TRUE(s.ok()); + + destroyEngine(setup); +} + +} // namespace mooncake + +int main(int argc, char **argv) { + gflags::ParseCommandLineFlags(&argc, &argv, false); + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} -- 2.34.1 From 57da883d30f8cbe32e6aaa7f9b8006f8def00974 Mon Sep 17 00:00:00 2001 From: Shangming Cai Date: Thu, 4 Jun 2026 12:35:14 +0800 Subject: [PATCH 181/382] [Doc] Update Documentation URL in pyproject.toml (#2326) --- mooncake-wheel/pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mooncake-wheel/pyproject.toml b/mooncake-wheel/pyproject.toml index a7168499..3426c72c 100644 --- a/mooncake-wheel/pyproject.toml +++ b/mooncake-wheel/pyproject.toml @@ -21,7 +21,7 @@ readme = "" [project.urls] Homepage = "https://github.com/kvcache-ai/Mooncake" -Documentation = "https://github.com/kvcache-ai/Mooncake/tree/main/doc" +Documentation = "https://kvcache-ai.github.io/Mooncake" Source = "https://github.com/kvcache-ai/Mooncake" Issues = "https://github.com/kvcache-ai/Mooncake/issues" -- 2.34.1 From bab96597645c1baccbee20d3d603792b84520d83 Mon Sep 17 00:00:00 2001 From: 1180300720 <48373350+VNightMare@users.noreply.github.com> Date: Thu, 4 Jun 2026 16:21:54 +0800 Subject: [PATCH 182/382] [Docs] Add build guidance for npu platform (#2325) Co-authored-by: ZhaoBaiwei --- .gitignore | 3 +++ docs/source/getting_started/build.md | 23 +++++++++++++++++++---- docs/source/zh_archive/build.md | 26 ++++++++++++++++++++------ 3 files changed, 42 insertions(+), 10 deletions(-) diff --git a/.gitignore b/.gitignore index dafcb87c..88dc0c75 100644 --- a/.gitignore +++ b/.gitignore @@ -206,3 +206,6 @@ _codeql_detected_source_root # CodeBuddy Memory .codebuddy/ + +# MacOS +.DS_Store diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index af50b0a4..ec0cd121 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -77,7 +77,7 @@ pip install mooncake-transfer-engine-non-cuda libhiredis-dev \ pkg-config \ patchelf - + # For centos/alibaba linux os yum install cmake \ gflags-devel \ @@ -152,7 +152,20 @@ pip install mooncake-transfer-engine-non-cuda - `-DMACA_LIB_DIR=/path/to/maca/lib64` - `-DMACA_RUNTIME_LIBS="mcruntime;mxc-runtime64;rt"` (semicolon-separated CMake list) -6. Install yalantinglibs +6. If you want to compile Huawei Ascend NPU support, first install the Ascend CANN Toolkit following the instructions at https://www.hiascend.com/document. After that: + 1) Source `set_env.sh` in the CANN installation directory to configure the build environment (no need to manually set `ASCEND_HOME_PATH` or other related environment variables). + 2) Mooncake provides two Ascend NPU transport paths, choose one as needed: + - `-DUSE_ASCEND_DIRECT=ON` (**recommended**): Ascend Direct transport based on the ADXL engine. (refer to [Version Compatibility Guide](https://gitcode.com/cann/hixl/wiki/Mooncake%20+%20HIXL%20%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B%E6%8C%87%E5%8D%97.md) for details). + - `-DUSE_UBSHMEM=ON`: Shared memory transport based on CANN VMM APIs (requires CANN >= 9.0.0, driver >= 26.0.0, Lingqu >= 1.5). + + Example for building with Ascend NPU: + ```bash + source /usr/local/Ascend/cann/set_env.sh + cmake .. -DUSE_ASCEND_DIRECT=ON + make -j + ``` + +7. Install yalantinglibs ```bash git clone https://github.com/alibaba/yalantinglibs.git cd yalantinglibs @@ -162,7 +175,7 @@ pip install mooncake-transfer-engine-non-cuda make install ``` -7. In the root directory of this project, run the following commands: +8. In the root directory of this project, run the following commands: ```bash mkdir build cd build @@ -170,7 +183,7 @@ pip install mooncake-transfer-engine-non-cuda make -j ``` -8. Install Mooncake python package and mooncake_master executable +9. Install Mooncake python package and mooncake_master executable ```bash make install ``` @@ -259,3 +272,5 @@ The following options can be used during `cmake ..` to specify whether to compil - `-DBUILD_SHARED_LIBS=[ON|OFF]`: Build Transfer Engine as shared library, default is OFF - `-DBUILD_UNIT_TESTS=[ON|OFF]`: Build unit tests, default is ON - `-DBUILD_EXAMPLES=[ON|OFF]`: Build examples, default is ON +- `-DUSE_ASCEND_DIRECT=[ON|OFF]`: Enable Ascend Direct transport and HCCS support via the ADXL engine (**recommended**). +- `-DUSE_UBSHMEM=[ON|OFF]`: Enable Huawei Ascend NPU shared memory transport via CANN VMM APIs. diff --git a/docs/source/zh_archive/build.md b/docs/source/zh_archive/build.md index 2de375e4..62ff9d2e 100644 --- a/docs/source/zh_archive/build.md +++ b/docs/source/zh_archive/build.md @@ -16,7 +16,7 @@ 请切换安装如下 PyPI 源的 wheel 包: ```bash pip install mooncake-transfer-engine-non-cuda - ``` + ``` 或者请附带 `-DUSE_CUDA=OFF` 使用源码编译安装。 ## 自动安装 @@ -71,7 +71,7 @@ libnuma-dev \ libcurl4-openssl-dev \ libhiredis-dev - + # For centos/alibaba linux os yum install cmake \ gflags-devel \ @@ -144,7 +144,20 @@ - `-DMACA_LIB_DIR=/path/to/maca/lib64` - `-DMACA_RUNTIME_LIBS="mcruntime;mxc-runtime64;rt"`(分号分隔的 CMake 列表) -6. 安装 yalantinglibs +6. 若需编译华为 Ascend NPU 支持,请先按照 https://www.hiascend.com/document 的指引安装 Ascend CANN Toolkit。之后: + 1) 在 CANN 安装目录下执行 `source set_env.sh` 配置编译环境(无需手动设置 `ASCEND_HOME_PATH` 等环境变量)。 + 2) Mooncake 提供两种 Ascend NPU 传输路径,按需选择其中一个: + - `-DUSE_ASCEND_DIRECT=ON`(**推荐**):基于 ADXL 引擎的 Ascend Direct 传输([版本配套关系参考](https://gitcode.com/cann/hixl/wiki/Mooncake%20+%20HIXL%20%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B%E6%8C%87%E5%8D%97.md))。 + - `-DUSE_UBSHMEM=ON`:基于 CANN VMM接口传输(CANN版本 >= 9.0.0 、驱动版本 >= 26.0.0、灵衢版本 >= 1.5) + + 启用 Ascend NPU 编译示例: + ```bash + source /usr/local/Ascend/cann/set_env.sh + cmake .. -DUSE_ASCEND_DIRECT=ON + make -j + ``` + +7. 安装 yalantinglibs ```bash git clone https://github.com/alibaba/yalantinglibs.git cd yalantinglibs @@ -154,7 +167,7 @@ make install ``` -7. 进入项目根目录,运行下列命令进行编译 +8. 进入项目根目录,运行下列命令进行编译 ```bash mkdir build cd build @@ -162,7 +175,7 @@ make -j ``` -8. 安装 Mooncake python 包和 mooncake_master 可执行文件 +9. 安装 Mooncake python 包和 mooncake_master 可执行文件 ```bash make install ``` @@ -198,7 +211,8 @@ - `-DBUILD_SHARED_LIBS=[ON|OFF]`: 将 Transfer Engine 编译为共享库,默认为 OFF - `-DBUILD_UNIT_TESTS=[ON|OFF]`: 编译单元测试,默认为 ON - `-DBUILD_EXAMPLES=[ON|OFF]`: 编译示例程序,默认为 ON -- `-DUSE_ASCEND_DIRECT=[ON|OFF]`: 启用 Ascend Direct RDMA 及 HCCS 支持 +- `-DUSE_ASCEND_DIRECT=[ON|OFF]`: 通过 ADXL 引擎启用 Ascend Direct 传输及 HCCS 支持(**NPU 推荐方式**) +- `-DUSE_UBSHMEM=[ON|OFF]`: 通过 CANN VMM API 启用华为 Ascend NPU 共享内存传输 - `-DUSE_MUSA=[ON|OFF]`: 启用Moore Threads GPUDirect RDMA ## 在 Docker 容器中使用 Mooncake -- 2.34.1 From b0b601bfb9bebfbe3ad900fc5240d0f2b7a93be4 Mon Sep 17 00:00:00 2001 From: Yufeng He <40085740+he-yufeng@users.noreply.github.com> Date: Fri, 5 Jun 2026 09:54:28 +0800 Subject: [PATCH 183/382] [Store] fix: make rpc_meta republish idempotent (#2311) --- mooncake-store/src/http_metadata_server.cpp | 22 +++-- mooncake-store/tests/CMakeLists.txt | 1 + .../tests/http_metadata_server_test.cpp | 99 +++++++++++++++++++ 3 files changed, 115 insertions(+), 7 deletions(-) create mode 100644 mooncake-store/tests/http_metadata_server_test.cpp diff --git a/mooncake-store/src/http_metadata_server.cpp b/mooncake-store/src/http_metadata_server.cpp index 350ecea6..72e152fc 100644 --- a/mooncake-store/src/http_metadata_server.cpp +++ b/mooncake-store/src/http_metadata_server.cpp @@ -57,14 +57,22 @@ void HttpMetadataServer::init_server() { std::string body(req.get_body()); { std::lock_guard lock(store_mutex_); - if (key.find("rpc_meta") != std::string::npos && - store_.find(std::string(key)) != store_.end()) { - resp.set_status_and_content( - status_type::bad_request, - "Duplicate rpc_meta key not allowed"); - return; + std::string key_str(key); + if (key_str.find("rpc_meta") != std::string::npos) { + auto it = store_.find(key_str); + if (it != store_.end()) { + if (it->second == body) { + resp.set_status_and_content(status_type::ok, + "metadata unchanged"); + return; + } + resp.set_status_and_content( + status_type::bad_request, + "Duplicate rpc_meta key not allowed"); + return; + } } - store_[std::string(key)] = body; + store_[std::move(key_str)] = body; } resp.set_status_and_content(status_type::ok, "metadata updated"); diff --git a/mooncake-store/tests/CMakeLists.txt b/mooncake-store/tests/CMakeLists.txt index c1048073..366f1a88 100644 --- a/mooncake-store/tests/CMakeLists.txt +++ b/mooncake-store/tests/CMakeLists.txt @@ -90,6 +90,7 @@ add_store_test(task_executor_test task_executor_test.cpp) add_store_test(task_integration_test task_integration_test.cpp) add_store_test(dummy_client_get_buffer_test dummy_client_get_buffer_test.cpp) add_store_test(health_check_test health_check_test.cpp) +add_store_test(http_metadata_server_test http_metadata_server_test.cpp) add_store_test(mmap_arena_test mmap_arena_test.cpp) add_store_test(mmap_arena_fallback_test mmap_arena_fallback_test.cpp) add_store_test(object_data_type_test object_data_type_test.cpp) diff --git a/mooncake-store/tests/http_metadata_server_test.cpp b/mooncake-store/tests/http_metadata_server_test.cpp new file mode 100644 index 00000000..5ff426a5 --- /dev/null +++ b/mooncake-store/tests/http_metadata_server_test.cpp @@ -0,0 +1,99 @@ +#include + +#include +#include +#include + +#include +#include + +#include "http_metadata_server.h" +#include "utils.h" + +namespace mooncake::testing { + +class HttpMetadataServerTest : public ::testing::Test { + protected: + struct HttpResponse { + int status; + std::string body; + }; + + HttpResponse Get(int port, const std::string& path) { + coro_http::coro_http_client client; + auto response = async_simple::coro::syncAwait(client.async_get( + "http://127.0.0.1:" + std::to_string(port) + path)); + return {response.status, std::string(response.resp_body)}; + } + + HttpResponse Put(int port, const std::string& path, + const std::string& body) { + coro_http::coro_http_client client; + auto response = async_simple::coro::syncAwait( + client.async_put("http://127.0.0.1:" + std::to_string(port) + path, + body, coro_http::req_content_type::json)); + return {response.status, std::string(response.resp_body)}; + } + + void WaitUntilReady(int port) { + for (int i = 0; i < 50; ++i) { + if (Get(port, "/health").status == 200) { + return; + } + std::this_thread::sleep_for(std::chrono::milliseconds(20)); + } + FAIL() << "HTTP metadata server did not become ready"; + } +}; + +TEST_F(HttpMetadataServerTest, AllowsIdempotentRpcMetaRepublish) { + int port = getFreeTcpPort(); + HttpMetadataServer server(static_cast(port), "127.0.0.1"); + ASSERT_TRUE(server.start()); + WaitUntilReady(port); + + const std::string path = + "/metadata?key=mooncake%2Frpc_meta%2F10.0.0.1%3A12384"; + const std::string body = + R"({"ip_or_host_name":"10.0.0.1","rpc_port":15228})"; + + EXPECT_EQ(Put(port, path, body).status, 200); + + auto second = Put(port, path, body); + EXPECT_EQ(second.status, 200); + EXPECT_EQ(second.body, "metadata unchanged"); + + auto stored = Get(port, path); + EXPECT_EQ(stored.status, 200); + EXPECT_EQ(stored.body, body); + + server.stop(); +} + +TEST_F(HttpMetadataServerTest, RejectsChangedRpcMetaRepublish) { + int port = getFreeTcpPort(); + HttpMetadataServer server(static_cast(port), "127.0.0.1"); + ASSERT_TRUE(server.start()); + WaitUntilReady(port); + + const std::string path = + "/metadata?key=mooncake%2Frpc_meta%2F10.0.0.1%3A12384"; + const std::string original = + R"({"ip_or_host_name":"10.0.0.1","rpc_port":15228})"; + const std::string changed = + R"({"ip_or_host_name":"10.0.0.1","rpc_port":16000})"; + + EXPECT_EQ(Put(port, path, original).status, 200); + + auto second = Put(port, path, changed); + EXPECT_EQ(second.status, 400); + EXPECT_EQ(second.body, "Duplicate rpc_meta key not allowed"); + + auto stored = Get(port, path); + EXPECT_EQ(stored.status, 200); + EXPECT_EQ(stored.body, original); + + server.stop(); +} + +} // namespace mooncake::testing -- 2.34.1 From a1495cce80c8611a89fd6786cb4a64a7379ea16c Mon Sep 17 00:00:00 2001 From: Dao007forever Date: Thu, 4 Jun 2026 19:01:38 -0700 Subject: [PATCH 184/382] [Bugfix][Store] Fix HA snapshot restore rejecting newer metadata formats (#2257) * [Bugfix][Store] Fix HA snapshot restore rejecting newer metadata formats The master snapshot writer (MasterService::MetadataSerializer) packs each object's metadata as `9 + replica_count` fields: a data_type after replicas_count and a trailing hard_pinned flag. The HA standby restore reader in catalog_backed_snapshot_provider.cpp only accepted `7 + replica_count` / `8 + replica_count` and assumed replicas began immediately after replicas_count, so it never skipped data_type and rejected current-format entries with: Snapshot metadata entry replica count mismatch, replicas=1, total_fields=11 Failed to deserialize snapshot metadata payload ... DESERIALIZE_FAIL Failed to load snapshot baseline, falling back to OpLog-only bootstrap Port the master-side format detector (v1/v2/v3) into the standby reader so it skips data_type when present and tolerates the trailing hard_pinned, matching every shape the writer emits. Add round-trip tests covering the data_type-only (8+rc), hard_pinned-only (8+rc), and current (9+rc) formats; the 9+rc case is a regression test for the live failure. Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: fatSheep --- .../catalog_backed_snapshot_provider.cpp | 39 ++++- mooncake-store/src/master_service.cpp | 22 ++- .../catalog_backed_snapshot_provider_test.cpp | 89 +++++++++- .../tests/ha/snapshot/snapshot_test_utils.h | 155 ++++++++++++++---- 4 files changed, 263 insertions(+), 42 deletions(-) diff --git a/mooncake-store/src/ha/snapshot/catalog_backed_snapshot_provider.cpp b/mooncake-store/src/ha/snapshot/catalog_backed_snapshot_provider.cpp index 21fffc2a..18f5ed37 100644 --- a/mooncake-store/src/ha/snapshot/catalog_backed_snapshot_provider.cpp +++ b/mooncake-store/src/ha/snapshot/catalog_backed_snapshot_provider.cpp @@ -131,14 +131,39 @@ DeserializeStandbyObjectMetadata( const auto soft_pin_timestamp_ms = array[index++].as(); const auto replica_count = array[index++].as(); - if (object.via.array.size != 7 + replica_count && - object.via.array.size != 8 + replica_count) { + // Optional fields are decoded by type for backward/forward + // compatibility with MasterService::MetadataSerializer, which appends + // them over time: + // data_type (positive int) appears before the replicas; + // hard_pinned (bool) and group_id (str) trail them. + // v1: 7 + replica_count, no optional fields + // v2: 8 + replica_count, either data_type or hard_pinned + // v3: 9 + replica_count, data_type + hard_pinned or + // hard_pinned + group_id + // v4: 10 + replica_count, data_type + hard_pinned + group_id + // 64-bit arithmetic keeps an attacker-controlled near-UINT32_MAX + // replica_count from wrapping the bounds and slipping an out-of-bounds + // index through. + constexpr uint64_t kBaseFieldCount = 7; + constexpr uint64_t kMaxOptionalFieldCount = 3; + const uint64_t total_elements = object.via.array.size; + const uint64_t min_elements = kBaseFieldCount + replica_count; + if (total_elements < min_elements || + total_elements > min_elements + kMaxOptionalFieldCount) { LOG(ERROR) << "Snapshot metadata entry replica count mismatch, " << "replicas=" << replica_count - << ", total_fields=" << object.via.array.size; + << ", total_fields=" << total_elements; return tl::make_unexpected(ErrorCode::DESERIALIZE_FAIL); } + // Skip the optional data_type; the standby restore path does not use + // it. A leading positive integer is data_type, whereas a replica is + // serialized as an array. + if (index < total_elements && + array[index].type == msgpack::type::POSITIVE_INTEGER) { + ++index; // data_type + } + const auto lease_timeout = std::chrono::system_clock::time_point( std::chrono::milliseconds(lease_timestamp_ms)); std::optional soft_pin_timeout; @@ -156,6 +181,14 @@ DeserializeStandbyObjectMetadata( std::vector replicas; replicas.reserve(replica_count); for (uint32_t i = 0; i < replica_count; ++i) { + // Defensive bound: a corrupt entry whose first post-count field + // looks like a data_type could otherwise read past the array. + if (index >= total_elements) { + LOG(ERROR) << "Snapshot metadata entry truncated, " + << "replicas=" << replica_count + << ", total_fields=" << total_elements; + return tl::make_unexpected(ErrorCode::DESERIALIZE_FAIL); + } auto replica_result = Serializer::deserialize(array[index++], segment_view); if (!replica_result) { diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 20635b52..d918bcfd 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -6472,15 +6472,18 @@ MasterService::MetadataSerializer::DeserializeMetadata( // Deserialize replicas count uint32_t replicas_count = array[index++].as(); - // Format detection: + // Format detection (decode optional fields by type for back-compat): // v1: 7 + replicas_count, no optional fields // v2: 8 + replicas_count, either data_type or hard_pinned // v3: 9 + replicas_count, data_type + hard_pinned or hard_pinned + // group_id v4: 10 + replicas_count, data_type + hard_pinned + group_id - constexpr uint32_t kBaseFieldCount = 7; - constexpr uint32_t kMaxOptionalFieldCount = 3; - const uint32_t total_elements = obj.via.array.size; - const uint32_t min_elements = kBaseFieldCount + replicas_count; + // 64-bit arithmetic keeps an attacker-controlled near-UINT32_MAX + // replicas_count from wrapping the bounds and slipping an out-of-bounds + // index past the size check. + constexpr uint64_t kBaseFieldCount = 7; + constexpr uint64_t kMaxOptionalFieldCount = 3; + const uint64_t total_elements = obj.via.array.size; + const uint64_t min_elements = kBaseFieldCount + replicas_count; if (total_elements < min_elements || total_elements > min_elements + kMaxOptionalFieldCount) { return tl::unexpected(SerializationError( @@ -6499,6 +6502,15 @@ MasterService::MetadataSerializer::DeserializeMetadata( replicas.reserve(replicas_count); for (uint32_t i = 0; i < replicas_count; i++) { + // Defensive bound: the data_type skip above can consume a slot the + // size check counted on, so a crafted entry whose first post-count + // field looks like a data_type could otherwise read past the array. + // Mirrors the standby reader in catalog_backed_snapshot_provider.cpp. + if (index >= total_elements) { + return tl::unexpected( + SerializationError(ErrorCode::DESERIALIZE_FAIL, + "deserialize ObjectMetadata truncated")); + } auto result = Serializer::deserialize( array[index++], service_->segment_manager_.getView()); if (!result) { diff --git a/mooncake-store/tests/ha/snapshot/catalog_backed_snapshot_provider_test.cpp b/mooncake-store/tests/ha/snapshot/catalog_backed_snapshot_provider_test.cpp index 7068bdbf..6407566f 100644 --- a/mooncake-store/tests/ha/snapshot/catalog_backed_snapshot_provider_test.cpp +++ b/mooncake-store/tests/ha/snapshot/catalog_backed_snapshot_provider_test.cpp @@ -2,7 +2,9 @@ #include #include +#include #include +#include #include #include #include @@ -57,13 +59,42 @@ class CatalogBackedSnapshotProviderTest } } - void PublishSnapshotPayload() { + void PublishSnapshotPayload( + SnapshotMetadataFormat format = SnapshotMetadataFormat::kLegacy) { auto result = mooncake::test::PublishSnapshotPayload( - *object_store_, *catalog_store_, descriptor_); + *object_store_, *catalog_store_, descriptor_, UUID{1, 2}, + kDefaultTestObjectKey, kDefaultTestDiskFilePath, + kDefaultTestObjectSize, format); ASSERT_TRUE(result.has_value()) << result.error(); snapshot_published_ = true; } + // Loads the published snapshot and asserts the single default object + // round-trips intact, regardless of the metadata on-wire format. + void ExpectLoadsDefaultObject() { + auto provider = CreateProvider(); + ASSERT_TRUE(provider.has_value()) << toString(provider.error()); + + auto snapshot = provider.value()->LoadLatestSnapshot(cluster_id_); + ASSERT_TRUE(snapshot.has_value()) << toString(snapshot.error()); + ASSERT_TRUE(snapshot->has_value()); + ASSERT_EQ(snapshot->value().metadata.size(), 1u); + + const auto& [key, metadata] = snapshot->value().metadata.front(); + EXPECT_EQ(key, kDefaultTestObjectKey); + EXPECT_EQ(metadata.client_id, (UUID{1, 2})); + EXPECT_EQ(metadata.size, kDefaultTestObjectSize); + ASSERT_EQ(metadata.replicas.size(), 1u); + + const auto& replica = metadata.replicas.front(); + EXPECT_EQ(replica.status, ReplicaStatus::COMPLETE); + ASSERT_TRUE(replica.is_disk_replica()); + EXPECT_EQ(replica.get_disk_descriptor().file_path, + kDefaultTestDiskFilePath); + EXPECT_EQ(replica.get_disk_descriptor().object_size, + kDefaultTestObjectSize); + } + tl::expected, ErrorCode> CreateProvider() const { return CreateCatalogBackedSnapshotProvider(MakeSnapshotProviderConfig( @@ -119,6 +150,60 @@ TEST_P(CatalogBackedSnapshotProviderTest, LoadLatestSnapshotRoundTrip) { kDefaultTestObjectSize); } +// The master snapshot writer evolved its per-object metadata layout over time +// (data_type field, trailing hard_pinned flag, trailing group_id). The standby +// restore reader must accept every shape; otherwise it rejects the snapshot and +// falls back to OpLog-only bootstrap. These tests pin each on-wire format. + +TEST_P(CatalogBackedSnapshotProviderTest, LoadLatestSnapshotWithDataTypeField) { + // 8 + replica_count: data_type packed right after replica_count. + PublishSnapshotPayload(SnapshotMetadataFormat::kDataTypeOnly); + ExpectLoadsDefaultObject(); +} + +TEST_P(CatalogBackedSnapshotProviderTest, + LoadLatestSnapshotWithHardPinnedField) { + // 8 + replica_count: trailing hard_pinned flag, no data_type. Exercises the + // type-based disambiguation (first replica is not a positive integer). + PublishSnapshotPayload(SnapshotMetadataFormat::kHardPinnedOnly); + ExpectLoadsDefaultObject(); +} + +TEST_P(CatalogBackedSnapshotProviderTest, + LoadLatestSnapshotWithDataTypeAndHardPinned) { + // 9 + replica_count: data_type + trailing hard_pinned. + PublishSnapshotPayload(SnapshotMetadataFormat::kDataTypeAndHardPinned); + ExpectLoadsDefaultObject(); +} + +TEST_P(CatalogBackedSnapshotProviderTest, LoadLatestSnapshotWithGroupId) { + // 10 + replica_count: current writer format (data_type + hard_pinned + + // trailing group_id). Regression test for the live snapshot restore + // failure against the latest metadata layout. + PublishSnapshotPayload(SnapshotMetadataFormat::kWithGroupId); + ExpectLoadsDefaultObject(); +} + +TEST_P(CatalogBackedSnapshotProviderTest, RejectsOverflowingReplicaCount) { + // A near-UINT32_MAX replica_count must not wrap the format-detection + // arithmetic into a valid-looking total and slip an out-of-bounds index + // past the size check. The entry packs zero replicas (array size 7) but + // declares UINT32_MAX, so every format must be rejected, not parsed. + auto published = mooncake::test::PublishSnapshotPayloadBytes( + *object_store_, *catalog_store_, descriptor_, + BuildMetadataPayloadWithDeclaredReplicaCount( + std::numeric_limits::max())); + ASSERT_TRUE(published.has_value()) << published.error(); + snapshot_published_ = true; + + auto provider = CreateProvider(); + ASSERT_TRUE(provider.has_value()) << toString(provider.error()); + + auto snapshot = provider.value()->LoadLatestSnapshot(cluster_id_); + ASSERT_FALSE(snapshot.has_value()); + EXPECT_EQ(snapshot.error(), ErrorCode::DESERIALIZE_FAIL); +} + TEST_P(CatalogBackedSnapshotProviderTest, RejectsClusterMismatch) { PublishSnapshotPayload(); diff --git a/mooncake-store/tests/ha/snapshot/snapshot_test_utils.h b/mooncake-store/tests/ha/snapshot/snapshot_test_utils.h index 8ff7af27..45ac1e8a 100644 --- a/mooncake-store/tests/ha/snapshot/snapshot_test_utils.h +++ b/mooncake-store/tests/ha/snapshot/snapshot_test_utils.h @@ -44,6 +44,24 @@ struct CatalogBackendParam { bool requires_redis{false}; }; +// On-wire shapes the master snapshot writer has emitted over time. The standby +// restore reader must tolerate all of them. See +// MasterService::MetadataSerializer in master_service.cpp. +// kLegacy: 7 + replica_count, no data_type, no trailing hard_pinned +// kDataTypeOnly: 8 + replica_count, data_type after replica_count +// kHardPinnedOnly: 8 + replica_count, trailing hard_pinned, no data_type +// kDataTypeAndHardPinned: +// 9 + replica_count, data_type plus trailing hard_pinned +// kWithGroupId: 10 + replica_count, data_type + hard_pinned + group_id +// (the current writer format) +enum class SnapshotMetadataFormat { + kLegacy, + kDataTypeOnly, + kHardPinnedOnly, + kDataTypeAndHardPinned, + kWithGroupId, +}; + class ScopedEnvVar { public: ScopedEnvVar(std::string name, std::string value) : name_(std::move(name)) { @@ -128,30 +146,10 @@ inline std::vector BuildSegmentsPayload() { return serialized.value(); } -inline std::vector BuildMetadataPayload( - const UUID& client_id, std::string_view object_key = kDefaultTestObjectKey, - std::string_view disk_file_path = kDefaultTestDiskFilePath, - uint64_t object_size = kDefaultTestObjectSize, - uint64_t put_start_time_ms = kDefaultTestPutStartTimeMs, - uint64_t lease_timeout_ms = kDefaultTestLeaseTimeoutMs) { - msgpack::sbuffer shard_buffer; - MsgpackPacker shard_packer(&shard_buffer); - shard_packer.pack_map(1); - shard_packer.pack(std::string("metadata")); - shard_packer.pack_array(1); - shard_packer.pack_array(2); - shard_packer.pack(std::string(object_key)); - - shard_packer.pack_array(8); - shard_packer.pack(UuidToString(client_id)); - shard_packer.pack(put_start_time_ms); - shard_packer.pack(object_size); - shard_packer.pack(lease_timeout_ms); - shard_packer.pack(false); - shard_packer.pack(uint64_t{0}); - shard_packer.pack(uint32_t{1}); - PackDiskReplica(shard_packer, disk_file_path, object_size); - +// Compresses a packed shard buffer and wraps it in the {"shards": {"0": ...}} +// root map that the snapshot metadata payload expects. +inline std::vector WrapShardIntoMetadataRoot( + const msgpack::sbuffer& shard_buffer) { auto compressed_shard = zstd_compress(reinterpret_cast(shard_buffer.data()), shard_buffer.size(), 3); @@ -169,6 +167,87 @@ inline std::vector BuildMetadataPayload( return ToByteVector(root_buffer); } +inline std::vector BuildMetadataPayload( + const UUID& client_id, std::string_view object_key = kDefaultTestObjectKey, + std::string_view disk_file_path = kDefaultTestDiskFilePath, + uint64_t object_size = kDefaultTestObjectSize, + uint64_t put_start_time_ms = kDefaultTestPutStartTimeMs, + uint64_t lease_timeout_ms = kDefaultTestLeaseTimeoutMs, + SnapshotMetadataFormat format = SnapshotMetadataFormat::kLegacy) { + const bool include_data_type = + format == SnapshotMetadataFormat::kDataTypeOnly || + format == SnapshotMetadataFormat::kDataTypeAndHardPinned || + format == SnapshotMetadataFormat::kWithGroupId; + const bool include_hard_pinned = + format == SnapshotMetadataFormat::kHardPinnedOnly || + format == SnapshotMetadataFormat::kDataTypeAndHardPinned || + format == SnapshotMetadataFormat::kWithGroupId; + const bool include_group_id = + format == SnapshotMetadataFormat::kWithGroupId; + constexpr uint32_t kReplicaCount = 1; + // 7 leading fields + replicas + optional data_type/hard_pinned/group_id. + const size_t array_size = 7 + kReplicaCount + (include_data_type ? 1 : 0) + + (include_hard_pinned ? 1 : 0) + + (include_group_id ? 1 : 0); + + msgpack::sbuffer shard_buffer; + MsgpackPacker shard_packer(&shard_buffer); + shard_packer.pack_map(1); + shard_packer.pack(std::string("metadata")); + shard_packer.pack_array(1); + shard_packer.pack_array(2); + shard_packer.pack(std::string(object_key)); + + shard_packer.pack_array(array_size); + shard_packer.pack(UuidToString(client_id)); + shard_packer.pack(put_start_time_ms); + shard_packer.pack(object_size); + shard_packer.pack(lease_timeout_ms); + shard_packer.pack(false); + shard_packer.pack(uint64_t{0}); + shard_packer.pack(kReplicaCount); + if (include_data_type) { + shard_packer.pack(static_cast(ObjectDataType::TENSOR)); + } + PackDiskReplica(shard_packer, disk_file_path, object_size); + if (include_hard_pinned) { + shard_packer.pack(true); + } + if (include_group_id) { + shard_packer.pack(std::string("test-group")); + } + + return WrapShardIntoMetadataRoot(shard_buffer); +} + +// Builds a metadata payload whose declared replica_count field is set to +// `declared_replica_count` while no replicas are actually packed (the entry +// array stays at the 7 leading fields). Used to verify the deserializer +// rejects a hostile count instead of overflowing into an out-of-bounds read. +inline std::vector BuildMetadataPayloadWithDeclaredReplicaCount( + uint32_t declared_replica_count, + std::string_view object_key = kDefaultTestObjectKey) { + msgpack::sbuffer shard_buffer; + MsgpackPacker shard_packer(&shard_buffer); + shard_packer.pack_map(1); + shard_packer.pack(std::string("metadata")); + shard_packer.pack_array(1); + shard_packer.pack_array(2); + shard_packer.pack(std::string(object_key)); + + // Exactly the 7 leading fields, zero trailing replicas. + shard_packer.pack_array(7); + shard_packer.pack(UuidToString(UUID{1, 2})); + shard_packer.pack(kDefaultTestPutStartTimeMs); + shard_packer.pack(kDefaultTestObjectSize); + shard_packer.pack(kDefaultTestLeaseTimeoutMs); + shard_packer.pack(false); + shard_packer.pack(uint64_t{0}); + shard_packer.pack(declared_replica_count); + + return WrapShardIntoMetadataRoot(shard_buffer); +} + inline ha::SnapshotDescriptor MakeTestSnapshotDescriptor( std::string_view snapshot_id = kDefaultTestSnapshotId, uint64_t last_included_seq = kDefaultTestSnapshotSeq, @@ -216,13 +295,12 @@ inline std::unique_ptr CreateCatalogStoreForTest( return nullptr; } -inline tl::expected PublishSnapshotPayload( +// Uploads the manifest/segments/metadata objects (using the caller-provided +// metadata payload) and publishes the descriptor. +inline tl::expected PublishSnapshotPayloadBytes( SnapshotObjectStore& object_store, ha::SnapshotCatalogStore& catalog_store, const ha::SnapshotDescriptor& descriptor, - const UUID& client_id = UUID{1, 2}, - std::string_view object_key = kDefaultTestObjectKey, - std::string_view disk_file_path = kDefaultTestDiskFilePath, - uint64_t object_size = kDefaultTestObjectSize) { + const std::vector& metadata_payload) { auto manifest = object_store.UploadString(descriptor.manifest_key, "messagepack|1.0.0|standby-test"); if (!manifest) { @@ -236,9 +314,7 @@ inline tl::expected PublishSnapshotPayload( } auto metadata = object_store.UploadBuffer( - descriptor.object_prefix + "metadata", - BuildMetadataPayload(client_id, object_key, disk_file_path, - object_size)); + descriptor.object_prefix + "metadata", metadata_payload); if (!metadata) { return tl::make_unexpected(metadata.error()); } @@ -251,4 +327,19 @@ inline tl::expected PublishSnapshotPayload( return {}; } +inline tl::expected PublishSnapshotPayload( + SnapshotObjectStore& object_store, ha::SnapshotCatalogStore& catalog_store, + const ha::SnapshotDescriptor& descriptor, + const UUID& client_id = UUID{1, 2}, + std::string_view object_key = kDefaultTestObjectKey, + std::string_view disk_file_path = kDefaultTestDiskFilePath, + uint64_t object_size = kDefaultTestObjectSize, + SnapshotMetadataFormat format = SnapshotMetadataFormat::kLegacy) { + return PublishSnapshotPayloadBytes( + object_store, catalog_store, descriptor, + BuildMetadataPayload(client_id, object_key, disk_file_path, object_size, + kDefaultTestPutStartTimeMs, + kDefaultTestLeaseTimeoutMs, format)); +} + } // namespace mooncake::test -- 2.34.1 From 5889787dbeb5fffd883e419f1e40df5f5a06d350 Mon Sep 17 00:00:00 2001 From: Xiao You Date: Fri, 5 Jun 2026 10:02:40 +0800 Subject: [PATCH 185/382] Optimize ascend_direct async query and route failure propagation. (#2323) Cache engine and target in QueryBatch to avoid per-poll metadata lookups, propagate route failures only to batches already in pending_batches with a single disconnect, and treat NOT_CONNECTED as success on auto_connect disconnect. Co-authored-by: Developer user --- .../async_transfer_executor.h | 36 ++- .../transfer_executor_base.h | 1 + .../async_transfer_executor.cpp | 228 ++++++++++-------- .../sync_transfer_executor.cpp | 2 + .../transfer_executor_base.cpp | 50 +++- 5 files changed, 197 insertions(+), 120 deletions(-) diff --git a/mooncake-transfer-engine/include/transport/ascend_transport/ascend_direct_transport/async_transfer_executor.h b/mooncake-transfer-engine/include/transport/ascend_transport/ascend_direct_transport/async_transfer_executor.h index f33d6879..01d30af9 100644 --- a/mooncake-transfer-engine/include/transport/ascend_transport/ascend_direct_transport/async_transfer_executor.h +++ b/mooncake-transfer-engine/include/transport/ascend_transport/ascend_direct_transport/async_transfer_executor.h @@ -18,10 +18,11 @@ #include #include +#include #include #include +#include #include -#include #include "transfer_executor_base.h" #include "transport/transport.h" @@ -45,15 +46,34 @@ class AsyncTransferExecutor : public TransferExecutorBase { const std::vector& slice_list) override; private: + struct QueryBatch { + std::vector slices; + size_t engine_idx; + std::string target_adxl_engine_name; + }; + + struct BatchPollResult { + bool done = false; + // When true, disconnect once and fail every batch already in + // pending_batches for this route. Batches enqueued after disconnect + // are not affected. + bool fail_entire_route = false; + std::string fail_reason; + }; + void queryThreadLoop(); - void fetchPendingBatches( - std::vector>& out_batches); - bool processOneBatch(std::vector& slice_list); - void handleTaskFinished(adxl::TransferReq handle); + void fetchPendingBatches(std::vector& out_batches); + void processOneBatch(QueryBatch& batch, BatchPollResult& result); + void failAllPendingOnRoute(size_t engine_idx, const std::string& target, + std::vector& pending, + const std::string& reason); + void markBatchFailed(QueryBatch& batch, const std::string& reason, + bool log_error = true); + void handleTaskFinished(); std::atomic running_{false}; std::thread query_thread_; - std::queue> query_slice_queue_; + std::queue query_slice_queue_; std::mutex query_mutex_; std::condition_variable query_cv_; @@ -61,9 +81,9 @@ class AsyncTransferExecutor : public TransferExecutorBase { size_t active_async_tasks_{0}; std::mutex async_task_mutex_; std::condition_variable async_task_cv_; - std::mutex async_handle_map_mutex_; - std::unordered_map async_handle_to_engine_idx_; bool finalized_{false}; + + size_t last_context_engine_idx_{SIZE_MAX}; }; } // namespace mooncake diff --git a/mooncake-transfer-engine/include/transport/ascend_transport/ascend_direct_transport/transfer_executor_base.h b/mooncake-transfer-engine/include/transport/ascend_transport/ascend_direct_transport/transfer_executor_base.h index 687bfece..d08164f4 100644 --- a/mooncake-transfer-engine/include/transport/ascend_transport/ascend_direct_transport/transfer_executor_base.h +++ b/mooncake-transfer-engine/include/transport/ascend_transport/ascend_direct_transport/transfer_executor_base.h @@ -93,6 +93,7 @@ class TransferExecutorBase { void cleanupConnections(); void finalizeEngines(); void disconnectAllForEngine(size_t engine_idx); + void recordConnectedSegment(size_t engine_idx, const std::string& remote); int checkAndConnect(size_t engine_idx, const std::string& target_adxl_engine_name); diff --git a/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/async_transfer_executor.cpp b/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/async_transfer_executor.cpp index b0d44574..535023eb 100644 --- a/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/async_transfer_executor.cpp +++ b/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/async_transfer_executor.cpp @@ -14,21 +14,20 @@ // limitations under the License. #include "transport/ascend_transport/ascend_direct_transport/async_transfer_executor.h" -#include "transport/ascend_transport/ascend_direct_transport/utils.h" #include #include +#include #include "common.h" -#include "transfer_metadata.h" namespace mooncake { namespace { constexpr int64_t kMillisToNano = 1000000; -constexpr size_t kAsyncTaskLimit = 100U; +constexpr size_t kAsyncTaskLimit = 100; constexpr int32_t kDefaultDisconnectTime = 1000; -constexpr int64_t kQueryPollIntervalMicros = 10; +constexpr int64_t kQueryPollIntervalMicros = 1; } // namespace AsyncTransferExecutor::AsyncTransferExecutor(const InitParams& params) @@ -115,21 +114,24 @@ TransferExecutorBase::ExecuteResult AsyncTransferExecutor::execute( adxl::TransferArgs(), req_handle); if (status == adxl::SUCCESS) { - { - std::lock_guard lock(async_handle_map_mutex_); - async_handle_to_engine_idx_[reinterpret_cast( - req_handle)] = local_engine_idx; - } + recordConnectedSegment(local_engine_idx, target_adxl_engine_name); + QueryBatch batch; + batch.slices = slice_list; + batch.engine_idx = local_engine_idx; + batch.target_adxl_engine_name = target_adxl_engine_name; + for (auto* slice : slice_list) { slice->ascend_direct.handle = req_handle; } + { std::unique_lock lock(query_mutex_); - query_slice_queue_.push(slice_list); + query_slice_queue_.push(std::move(batch)); } query_cv_.notify_one(); return {.ret = 0, .status = status, .retryable = false}; } + { std::lock_guard lock(async_task_mutex_); if (active_async_tasks_ > 0) { @@ -143,21 +145,42 @@ TransferExecutorBase::ExecuteResult AsyncTransferExecutor::execute( } void AsyncTransferExecutor::queryThreadLoop() { - std::vector> pending_batches; + std::vector pending_batches; + pending_batches.reserve(64); + while (running_) { fetchPendingBatches(pending_batches); if (pending_batches.empty()) { continue; } - auto it = pending_batches.begin(); - while (it != pending_batches.end()) { - auto& slice_list = *it; - if (processOneBatch(slice_list)) { - it = pending_batches.erase(it); - } else { - ++it; + size_t i = 0; + while (i < pending_batches.size()) { + auto& batch = pending_batches[i]; + BatchPollResult poll_result; + processOneBatch(batch, poll_result); + if (!poll_result.done) { + ++i; + continue; } + + if (poll_result.fail_entire_route) { + const std::string& reason = + poll_result.fail_reason.empty() + ? "Route failed for target: " + + batch.target_adxl_engine_name + : poll_result.fail_reason; + failAllPendingOnRoute(batch.engine_idx, + batch.target_adxl_engine_name, + pending_batches, reason); + // failAllPendingOnRoute swap-pops may move unvisited batches to + // indices < i; restart the scan so none are skipped this cycle. + i = 0; + continue; + } + + std::swap(pending_batches[i], pending_batches.back()); + pending_batches.pop_back(); } if (!pending_batches.empty()) { @@ -168,7 +191,7 @@ void AsyncTransferExecutor::queryThreadLoop() { } void AsyncTransferExecutor::fetchPendingBatches( - std::vector>& out_batches) { + std::vector& out_batches) { std::unique_lock lock(query_mutex_); if (out_batches.empty()) { query_cv_.wait( @@ -183,110 +206,115 @@ void AsyncTransferExecutor::fetchPendingBatches( } } -bool AsyncTransferExecutor::processOneBatch( - std::vector& slice_list) { - if (slice_list.empty()) { - return true; +void AsyncTransferExecutor::markBatchFailed(QueryBatch& batch, + const std::string& reason, + bool log_error) { + if (log_error) { + LOG(ERROR) << reason; + } + for (auto* slice : batch.slices) { + slice->markFailed(); + } + handleTaskFinished(); +} + +void AsyncTransferExecutor::failAllPendingOnRoute( + size_t engine_idx, const std::string& target, + std::vector& pending, const std::string& reason) { + LOG(ERROR) << reason; + bool disconnected = false; + for (size_t j = 0; j < pending.size();) { + auto& batch = pending[j]; + if (batch.engine_idx != engine_idx || + batch.target_adxl_engine_name != target) { + ++j; + continue; + } + + markBatchFailed(batch, reason, false); + if (!disconnected && engine_idx < adxl_engines_.size() && + !target.empty()) { + disconnect(engine_idx, target, params_.connect_timeout); + disconnected = true; + } + + std::swap(pending[j], pending.back()); + pending.pop_back(); + } +} + +void AsyncTransferExecutor::processOneBatch(QueryBatch& batch, + BatchPollResult& result) { + result = {}; + + if (batch.slices.empty()) { + result.done = true; + return; + } + + if (batch.engine_idx >= adxl_engines_.size() || + batch.engine_idx >= local_engine_contexts_.size()) { + markBatchFailed( + batch, "Invalid engine_idx: " + std::to_string(batch.engine_idx)); + result.done = true; + return; + } + + if (last_context_engine_idx_ != batch.engine_idx) { + auto context_ret = + aclrtSetCurrentContext(local_engine_contexts_[batch.engine_idx]); + if (context_ret != ACL_ERROR_NONE) { + markBatchFailed( + batch, "aclrtSetCurrentContext failed, ret: " + + std::to_string(context_ret) + + ", engine_idx: " + std::to_string(batch.engine_idx)); + result.done = true; + return; + } + last_context_engine_idx_ = batch.engine_idx; } auto handle = - static_cast(slice_list[0]->ascend_direct.handle); - size_t engine_idx = 0; - bool found_engine_idx = false; - { - std::lock_guard lock(async_handle_map_mutex_); - auto it_handle = async_handle_to_engine_idx_.find( - reinterpret_cast(handle)); - if (it_handle != async_handle_to_engine_idx_.end()) { - engine_idx = it_handle->second; - found_engine_idx = true; - } - } - - auto fail_batch = [&](const std::string& message, - const std::string& target_adxl_engine_name) { - LOG(ERROR) << message; - for (auto* slice : slice_list) { - slice->markFailed(); - } - handleTaskFinished(handle); - if (engine_idx < adxl_engines_.size() && - !target_adxl_engine_name.empty()) { - disconnect(engine_idx, target_adxl_engine_name, - params_.connect_timeout); - } - return true; - }; - - if (!found_engine_idx) { - return fail_batch("Cannot resolve async handle to engine index", ""); - } - - if (engine_idx >= adxl_engines_.size() || - engine_idx >= local_engine_contexts_.size()) { - return fail_batch( - "Invalid async engine index: " + std::to_string(engine_idx), ""); - } - - auto target_segment_desc = - metadata_->getSegmentDescByID(slice_list[0]->target_id); - if (!target_segment_desc) { - return fail_batch( - "Cannot find target segment descriptor for target_id: " + - std::to_string(slice_list[0]->target_id), - ""); - } - - std::string target_adxl_engine_name = - resolveTargetAdxlEngineName(target_segment_desc, engine_idx); - if (target_adxl_engine_name.empty()) { - return fail_batch("Cannot resolve target adxl engine name", ""); - } - - auto context_ret = - aclrtSetCurrentContext(local_engine_contexts_[engine_idx]); - if (context_ret != ACL_ERROR_NONE) { - return fail_batch("Call aclrtSetCurrentContext failed, ret: " + - std::to_string(context_ret) + - ", engine_idx: " + std::to_string(engine_idx), - target_adxl_engine_name); - } + static_cast(batch.slices[0]->ascend_direct.handle); adxl::TransferStatus task_status; auto ret = - adxl_engines_[engine_idx]->GetTransferStatus(handle, task_status); + adxl_engines_[batch.engine_idx]->GetTransferStatus(handle, task_status); if (ret != adxl::SUCCESS || task_status == adxl::TransferStatus::FAILED) { - return fail_batch("Get transfer status failed, ret: " + - std::to_string(static_cast(ret)) + - ", errmsg: " + aclGetRecentErrMsg(), - target_adxl_engine_name); + result.done = true; + result.fail_entire_route = true; + result.fail_reason = "Get transfer status failed, ret: " + + std::to_string(static_cast(ret)) + + ", errmsg: " + aclGetRecentErrMsg(); + return; } if (task_status == adxl::TransferStatus::COMPLETED) { - for (auto* slice : slice_list) { + for (auto* slice : batch.slices) { slice->markSuccess(); } - handleTaskFinished(handle); + handleTaskFinished(); if (params_.use_short_connection) { - disconnect(engine_idx, target_adxl_engine_name, + disconnect(batch.engine_idx, batch.target_adxl_engine_name, params_.connect_timeout); } - return true; + result.done = true; + return; } auto now = getCurrentTimeInNano(); - if (now - slice_list[0]->ascend_direct.start_time > + if (now - batch.slices[0]->ascend_direct.start_time > transfer_timeout_in_nano_) { - return fail_batch("Transfer timeout", target_adxl_engine_name); + result.done = true; + result.fail_entire_route = true; + result.fail_reason = + "Transfer timeout to: " + batch.target_adxl_engine_name; + return; } - - return false; } -void AsyncTransferExecutor::handleTaskFinished(adxl::TransferReq handle) { - std::lock_guard handle_lock(async_handle_map_mutex_); - async_handle_to_engine_idx_.erase(reinterpret_cast(handle)); +void AsyncTransferExecutor::handleTaskFinished() { std::lock_guard lock(async_task_mutex_); if (active_async_tasks_ > 0) { active_async_tasks_--; diff --git a/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/sync_transfer_executor.cpp b/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/sync_transfer_executor.cpp index 8e71dfc3..8e19431c 100644 --- a/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/sync_transfer_executor.cpp +++ b/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/sync_transfer_executor.cpp @@ -81,6 +81,8 @@ TransferExecutorBase::ExecuteResult SyncTransferExecutor::execute( return {.ret = -1, .status = status, .retryable = true}; } + recordConnectedSegment(local_engine_idx, target_adxl_engine_name); + if (params_.use_short_connection) { disconnect(local_engine_idx, target_adxl_engine_name, params_.connect_timeout); diff --git a/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/transfer_executor_base.cpp b/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/transfer_executor_base.cpp index b3ab63aa..0a8f05f6 100644 --- a/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/transfer_executor_base.cpp +++ b/mooncake-transfer-engine/src/transport/ascend_transport/ascend_direct_transport/transfer_executor_base.cpp @@ -14,6 +14,7 @@ // limitations under the License. #include "transport/ascend_transport/ascend_direct_transport/transfer_executor_base.h" +#include "adxl/adxl_types.h" #include "transport/ascend_transport/ascend_direct_transport/async_transfer_executor.h" #include "transport/ascend_transport/ascend_direct_transport/local_copy_engine.h" #include "transport/ascend_transport/ascend_direct_transport/sync_transfer_executor.h" @@ -270,18 +271,35 @@ int TransferExecutorBase::checkAndConnect( return 0; } +void TransferExecutorBase::recordConnectedSegment(size_t engine_idx, + const std::string& remote) { + if (!params_.auto_connect) { + return; + } + std::lock_guard lock(connection_mutex_); + connected_segments_[engine_idx].insert(remote); +} + int TransferExecutorBase::disconnect(size_t engine_idx, const std::string& target_adxl_engine_name, int32_t timeout_in_millis) { if (params_.auto_connect) { auto status = adxl_engines_[engine_idx]->Disconnect( target_adxl_engine_name.c_str(), timeout_in_millis); - if (status != adxl::SUCCESS) { + if (status != adxl::SUCCESS && status != adxl::NOT_CONNECTED) { LOG(ERROR) << "Failed to disconnect to: " << target_adxl_engine_name << ", status: " << status << ", errmsg: " << aclGetRecentErrMsg(); return -1; } + std::lock_guard lock(connection_mutex_); + auto it = connected_segments_.find(engine_idx); + if (it != connected_segments_.end()) { + it->second.erase(target_adxl_engine_name); + if (it->second.empty()) { + connected_segments_.erase(it); + } + } return 0; } std::lock_guard lock(connection_mutex_); @@ -319,7 +337,7 @@ void TransferExecutorBase::disconnectAllForEngine(size_t engine_idx) { for (const auto& connected_segment : it->second) { auto status = adxl_engines_[engine_idx]->Disconnect( connected_segment.c_str(), params_.connect_timeout); - if (status != adxl::SUCCESS) { + if (status != adxl::SUCCESS && status != adxl::NOT_CONNECTED) { LOG(ERROR) << "Failed to disconnect AdxlEngine: " << connected_segment << ", errmsg: " << aclGetRecentErrMsg(); @@ -475,23 +493,31 @@ int TransferExecutorBase::deregisterMem(void* addr) { return 0; } - if (params_.dummy_real_mode) { - std::set engine_indices; - for (const auto& [engine_idx, mem_handle] : it->second) { - (void)mem_handle; - engine_indices.insert(engine_idx); - } - for (size_t engine_idx : engine_indices) { - disconnectAllForEngine(engine_idx); - } + std::set engine_indices; + for (const auto& [engine_idx, mem_handle] : it->second) { + (void)mem_handle; + engine_indices.insert(engine_idx); + } + for (size_t engine_idx : engine_indices) { + disconnectAllForEngine(engine_idx); } + bool deregister_failed = false; for (const auto& [engine_idx, mem_handle] : it->second) { if (engine_idx >= local_engine_contexts_.size()) { continue; } CHECK_ACL(aclrtSetCurrentContext(local_engine_contexts_[engine_idx])); - (void)adxl_engines_[engine_idx]->DeregisterMem(mem_handle); + auto status = adxl_engines_[engine_idx]->DeregisterMem(mem_handle); + if (status != adxl::SUCCESS) { + LOG(ERROR) << "DeregisterMem failed, engine_idx: " << engine_idx + << ", status: " << status + << ", errmsg: " << aclGetRecentErrMsg(); + deregister_failed = true; + } + } + if (deregister_failed) { + return -1; } addr_to_mem_handles_.erase(it); return 0; -- 2.34.1 From 5b9a395cc52bd58119cc30395804daac52ed7efc Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Fri, 5 Jun 2026 10:04:02 +0800 Subject: [PATCH 186/382] [TransferEngine] Fix resource leaks in error paths (#2332) 1. multi_transport.cpp: Delete Transport object when install() fails. The raw pointer was leaked on the error return path. 2. transfer_engine_c.cpp: Add null check after malloc and early return when size is 0 in getNotifsFromEngine(). Prevents null dereference in memset when malloc fails, and avoids implementation-defined behavior of malloc(0). Co-authored-by: Claude Opus 4.6 --- mooncake-transfer-engine/src/multi_transport.cpp | 1 + mooncake-transfer-engine/src/transfer_engine_c.cpp | 2 ++ 2 files changed, 3 insertions(+) diff --git a/mooncake-transfer-engine/src/multi_transport.cpp b/mooncake-transfer-engine/src/multi_transport.cpp index 69d0ed2d..dbe81ff7 100644 --- a/mooncake-transfer-engine/src/multi_transport.cpp +++ b/mooncake-transfer-engine/src/multi_transport.cpp @@ -422,6 +422,7 @@ Transport* MultiTransport::installTransport(const std::string& proto, } #endif if (transport->install(local_server_name_, metadata_, topo)) { + delete transport; return nullptr; } diff --git a/mooncake-transfer-engine/src/transfer_engine_c.cpp b/mooncake-transfer-engine/src/transfer_engine_c.cpp index 246f1ab3..b643a153 100644 --- a/mooncake-transfer-engine/src/transfer_engine_c.cpp +++ b/mooncake-transfer-engine/src/transfer_engine_c.cpp @@ -203,8 +203,10 @@ notify_msg_t *getNotifsFromEngine(transfer_engine_t engine, int *size) { std::vector notifies_desc; native->getNotifies(notifies_desc); *size = notifies_desc.size(); + if (*size == 0) return nullptr; notify_msg_t *notifies = (notify_msg_t *)malloc(*size * sizeof(notify_msg_t)); + if (!notifies) return nullptr; memset(notifies, 0, *size * sizeof(notify_msg_t)); for (int i = 0; i < *size; i++) { notifies[i].name = (char *)malloc(notifies_desc[i].name.size() + 1); -- 2.34.1 From 7de0cca2a1298b9d3ea9d70d0f28ac5fca88cdfa Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Fri, 5 Jun 2026 11:09:31 +0800 Subject: [PATCH 187/382] [Common] Harden Environ parsing, fix opendir leak, support .yml config (#2316) * [Common] Harden Environ parsing, fix opendir leak, support .yml config Fixes #2315 1. Fix: Replace opendir() with stat()+S_ISDIR() in config.cpp to fix DIR* handle leak. 2. Robustness: Replace atoi() with strtol() in Environ::GetInt with endptr/errno validation. Invalid values now fall back to default with a warning instead of silently returning 0. 3. Robustness: Add leading '-' rejection in Environ::GetSizeT to prevent strtoull negative wrapping (e.g. MC_SLICE_SIZE=-1 yielding ULLONG_MAX). 4. Feature: Support .yml extension in DefaultConfig::Load(). 5. Tests: Add 26 unit tests for Environ::GetInt/GetSizeT/GetBool/ GetString covering valid, invalid, missing, overflow, negative, and trailing garbage inputs. Tests call the real production code. Co-Authored-By: Claude Opus 4.6 * [Common] Use strtoll instead of strtoull in GetSizeT for robust negative handling Address Gemini review: val[0]=='-' check missed leading-whitespace cases like " -1". Using strtoll catches negatives regardless of whitespace, and also guards against 32-bit SIZE_MAX truncation. Co-Authored-By: Claude Opus 4.6 * [Common] Address review: restore comments, use std::filesystem - Restore helper method comments stripped when moving to public - Switch from stat()+S_ISDIR() to std::filesystem::is_directory() per reviewer suggestion (C++20 project, already used elsewhere) Co-Authored-By: Claude Opus 4.6 * [Common] Fix clang-format violation in environ.cpp Co-Authored-By: Claude Opus 4.6 * [Common] Address Copilot review: non-throwing is_directory, fix stray PUBLIC - Use std::error_code overload of std::filesystem::is_directory to avoid throwing on permission errors (EACCES). Matches original opendir() non-throwing behavior. - Remove pre-existing stray PUBLIC keyword in tests/CMakeLists.txt. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 --- mooncake-common/include/environ.h | 6 +- mooncake-common/src/default_config.cpp | 4 +- mooncake-common/src/environ.cpp | 27 +++- mooncake-common/tests/CMakeLists.txt | 8 +- mooncake-common/tests/environ_test.cpp | 185 ++++++++++++++++++++++++ mooncake-transfer-engine/src/config.cpp | 5 +- 6 files changed, 224 insertions(+), 11 deletions(-) create mode 100644 mooncake-common/tests/environ_test.cpp diff --git a/mooncake-common/include/environ.h b/mooncake-common/include/environ.h index 474af488..78f8f450 100644 --- a/mooncake-common/include/environ.h +++ b/mooncake-common/include/environ.h @@ -52,9 +52,6 @@ class Environ { bool GetWithNvidiaPeermem() const { return with_nvidia_peermem_; } int GetEfaCqThreads() const { return efa_cq_threads_; } - private: - Environ(); - // Helper method to get int from env static int GetInt(const char* name, int default_value); // Helper method to get size_t from env @@ -65,6 +62,9 @@ class Environ { static std::string GetString(const char* name, const std::string& default_value); + private: + Environ(); + // Member variables int num_cq_per_ctx_; int num_comp_channels_per_ctx_; diff --git a/mooncake-common/src/default_config.cpp b/mooncake-common/src/default_config.cpp index 65d7eeea..266e1430 100644 --- a/mooncake-common/src/default_config.cpp +++ b/mooncake-common/src/default_config.cpp @@ -28,9 +28,9 @@ void DefaultConfig::Load() { return path.size() >= ext.size() && path.compare(path.size() - ext.size(), ext.size(), ext) == 0; }; - if (check_extension(path_, ".yaml")) { + if (check_extension(path_, ".yaml") || check_extension(path_, ".yml")) { loadFromYAML(); - type_ = ConfigType::YAML; // YAML type + type_ = ConfigType::YAML; } else if (check_extension(path_, ".json")) { loadFromJSON(); type_ = ConfigType::JSON; // JSON type diff --git a/mooncake-common/src/environ.cpp b/mooncake-common/src/environ.cpp index e45277a0..29a03e8a 100644 --- a/mooncake-common/src/environ.cpp +++ b/mooncake-common/src/environ.cpp @@ -1,6 +1,9 @@ #include "environ.h" +#include +#include #include #include +#include namespace mooncake { @@ -12,7 +15,17 @@ Environ& Environ::Get() { int Environ::GetInt(const char* name, int default_value) { const char* val = std::getenv(name); if (val) { - return std::atoi(val); + char* endptr = nullptr; + errno = 0; + long result = std::strtol(val, &endptr, 10); + if (endptr == val || *endptr != '\0' || errno == ERANGE || + result < INT_MIN || result > INT_MAX) { + std::cerr << "[Mooncake] Warning: invalid value '" << val + << "' for env " << name << ", using default " + << default_value << std::endl; + return default_value; + } + return static_cast(result); } return default_value; } @@ -20,7 +33,17 @@ int Environ::GetInt(const char* name, int default_value) { size_t Environ::GetSizeT(const char* name, size_t default_value) { const char* val = std::getenv(name); if (val) { - return static_cast(std::strtoull(val, nullptr, 10)); + char* endptr = nullptr; + errno = 0; + long long result = std::strtoll(val, &endptr, 10); + if (endptr == val || *endptr != '\0' || errno == ERANGE || result < 0 || + static_cast(result) > SIZE_MAX) { + std::cerr << "[Mooncake] Warning: invalid value '" << val + << "' for env " << name << ", using default " + << default_value << std::endl; + return default_value; + } + return static_cast(result); } return default_value; } diff --git a/mooncake-common/tests/CMakeLists.txt b/mooncake-common/tests/CMakeLists.txt index 34f38ff1..ec43cfa2 100644 --- a/mooncake-common/tests/CMakeLists.txt +++ b/mooncake-common/tests/CMakeLists.txt @@ -1,7 +1,11 @@ add_executable(default_config_test default_config_test.cpp) target_link_libraries(default_config_test PUBLIC - mooncake_common + mooncake_common gtest - pthread PUBLIC + pthread ) add_test(NAME default_config_test COMMAND default_config_test) + +add_executable(environ_test environ_test.cpp) +target_link_libraries(environ_test PUBLIC mooncake_common gtest pthread) +add_test(NAME environ_test COMMAND environ_test) diff --git a/mooncake-common/tests/environ_test.cpp b/mooncake-common/tests/environ_test.cpp new file mode 100644 index 00000000..9a0b5340 --- /dev/null +++ b/mooncake-common/tests/environ_test.cpp @@ -0,0 +1,185 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "environ.h" + +#include + +#include +#include + +using mooncake::Environ; + +class EnvironTest : public ::testing::Test { + protected: + void SetUp() override { clearTestEnvVars(); } + void TearDown() override { clearTestEnvVars(); } + + void clearTestEnvVars() { + unsetenv("MC_TEST_INT"); + unsetenv("MC_TEST_SIZET"); + unsetenv("MC_TEST_BOOL"); + unsetenv("MC_TEST_STRING"); + } +}; + +// --- GetInt --- + +TEST_F(EnvironTest, GetIntValidValue) { + setenv("MC_TEST_INT", "42", 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 0), 42); +} + +TEST_F(EnvironTest, GetIntNegativeValue) { + setenv("MC_TEST_INT", "-100", 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 0), -100); +} + +TEST_F(EnvironTest, GetIntZero) { + setenv("MC_TEST_INT", "0", 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 99), 0); +} + +TEST_F(EnvironTest, GetIntMissing) { + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 77), 77); +} + +TEST_F(EnvironTest, GetIntEmpty) { + setenv("MC_TEST_INT", "", 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 55), 55); +} + +TEST_F(EnvironTest, GetIntNonNumeric) { + setenv("MC_TEST_INT", "abc", 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 55), 55); +} + +TEST_F(EnvironTest, GetIntTrailingGarbage) { + setenv("MC_TEST_INT", "123abc", 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 55), 55); +} + +TEST_F(EnvironTest, GetIntOverflow) { + setenv("MC_TEST_INT", "99999999999999999999", 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 55), 55); +} + +TEST_F(EnvironTest, GetIntMaxValue) { + setenv("MC_TEST_INT", std::to_string(INT_MAX).c_str(), 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 0), INT_MAX); +} + +TEST_F(EnvironTest, GetIntMinValue) { + setenv("MC_TEST_INT", std::to_string(INT_MIN).c_str(), 1); + EXPECT_EQ(Environ::GetInt("MC_TEST_INT", 0), INT_MIN); +} + +// --- GetSizeT --- + +TEST_F(EnvironTest, GetSizeTValidValue) { + setenv("MC_TEST_SIZET", "65536", 1); + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 0), 65536u); +} + +TEST_F(EnvironTest, GetSizeTZero) { + setenv("MC_TEST_SIZET", "0", 1); + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 99), 0u); +} + +TEST_F(EnvironTest, GetSizeTMissing) { + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 4096), 4096u); +} + +TEST_F(EnvironTest, GetSizeTNonNumeric) { + setenv("MC_TEST_SIZET", "bogus", 1); + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 4096), 4096u); +} + +TEST_F(EnvironTest, GetSizeTTrailingGarbage) { + setenv("MC_TEST_SIZET", "100MB", 1); + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 4096), 4096u); +} + +TEST_F(EnvironTest, GetSizeTLargeValue) { + setenv("MC_TEST_SIZET", "1099511627776", 1); // 1 TiB + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 0), 1099511627776ull); +} + +TEST_F(EnvironTest, GetSizeTNegativeValue) { + setenv("MC_TEST_SIZET", "-1", 1); + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 4096), 4096u); +} + +TEST_F(EnvironTest, GetSizeTNegativeWithLeadingSpace) { + setenv("MC_TEST_SIZET", " -1", 1); + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 4096), 4096u); +} + +TEST_F(EnvironTest, GetSizeTOverflow) { + setenv("MC_TEST_SIZET", "99999999999999999999999999", 1); + EXPECT_EQ(Environ::GetSizeT("MC_TEST_SIZET", 4096), 4096u); +} + +// --- GetBool --- + +TEST_F(EnvironTest, GetBoolTrue) { + for (const char* v : + {"1", "true", "TRUE", "True", "on", "ON", "yes", "YES"}) { + setenv("MC_TEST_BOOL", v, 1); + EXPECT_TRUE(Environ::GetBool("MC_TEST_BOOL", false)) << "for: " << v; + } +} + +TEST_F(EnvironTest, GetBoolFalse) { + for (const char* v : {"0", "false", "FALSE", "off", "no", "whatever"}) { + setenv("MC_TEST_BOOL", v, 1); + EXPECT_FALSE(Environ::GetBool("MC_TEST_BOOL", false)) << "for: " << v; + } +} + +TEST_F(EnvironTest, GetBoolMissing) { + EXPECT_TRUE(Environ::GetBool("MC_TEST_BOOL", true)); + EXPECT_FALSE(Environ::GetBool("MC_TEST_BOOL", false)); +} + +TEST_F(EnvironTest, GetBoolEmpty) { + setenv("MC_TEST_BOOL", "", 1); + EXPECT_FALSE(Environ::GetBool("MC_TEST_BOOL", true)); +} + +// --- GetString --- + +TEST_F(EnvironTest, GetStringValidValue) { + setenv("MC_TEST_STRING", "hello", 1); + EXPECT_EQ(Environ::GetString("MC_TEST_STRING", "default"), "hello"); +} + +TEST_F(EnvironTest, GetStringMissing) { + EXPECT_EQ(Environ::GetString("MC_TEST_STRING", "default"), "default"); +} + +TEST_F(EnvironTest, GetStringEmpty) { + setenv("MC_TEST_STRING", "", 1); + EXPECT_EQ(Environ::GetString("MC_TEST_STRING", "default"), ""); +} + +TEST_F(EnvironTest, GetStringWithSpaces) { + setenv("MC_TEST_STRING", "hello world", 1); + EXPECT_EQ(Environ::GetString("MC_TEST_STRING", ""), "hello world"); +} + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} diff --git a/mooncake-transfer-engine/src/config.cpp b/mooncake-transfer-engine/src/config.cpp index a7f84fc1..44fd0393 100644 --- a/mooncake-transfer-engine/src/config.cpp +++ b/mooncake-transfer-engine/src/config.cpp @@ -17,7 +17,7 @@ #include #include #include -#include +#include #include #include @@ -282,7 +282,8 @@ void loadGlobalConfig(GlobalConfig& config) { const char* log_dir_path = std::getenv("MC_LOG_DIR"); if (log_dir_path) { google::InitGoogleLogging("mooncake-transfer-engine"); - if (opendir(log_dir_path) == NULL) { + std::error_code ec; + if (!std::filesystem::is_directory(log_dir_path, ec)) { LOG(WARNING) << "Path [" << log_dir_path << "] is not a valid directory path. Still logging to stderr."; -- 2.34.1 From 92e7770f0766f4de7b81c61b99c5527ed0b287f6 Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Fri, 5 Jun 2026 11:19:41 +0800 Subject: [PATCH 188/382] [PG] Add MUSA build support (#2329) --- CMakeLists.txt | 7 +- mooncake-pg/BuildPgExt.cmake | 6 + mooncake-pg/include/mooncake_backend.h | 30 + mooncake-pg/include/mooncake_worker.cuh | 60 +- .../include/mooncake_worker_kernels.cuh | 74 ++ mooncake-pg/setup.py | 166 +++-- mooncake-pg/src/connection_poller.cpp | 8 +- mooncake-pg/src/mooncake_backend.cpp | 111 ++- mooncake-pg/src/mooncake_worker.cu | 666 +++++------------- mooncake-pg/src/mooncake_worker.mu | 1 + mooncake-pg/src/mooncake_worker_host.cpp | 469 ++++++++++++ mooncake-pg/src/mooncake_worker_thread.cpp | 23 +- mooncake-pg/src/p2p_proxy.cpp | 3 +- mooncake-pg/src/pg_py.cpp | 7 + mooncake-pg/tests/pg_test_utils.py | 30 + mooncake-pg/tests/test_pg_collectives.py | 5 + .../include/gpu_vendor/musa.h | 3 + 17 files changed, 1028 insertions(+), 641 deletions(-) create mode 100644 mooncake-pg/include/mooncake_worker_kernels.cuh create mode 120000 mooncake-pg/src/mooncake_worker.mu create mode 100644 mooncake-pg/src/mooncake_worker_host.cpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 56e4c2de..30822b81 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -101,8 +101,10 @@ if (WITH_EP) include_directories(mooncake-pg/include) else () message(STATUS "WITH_EP enabled: building Mooncake EP and PG Python extensions") - find_package(CUDAToolkit REQUIRED) - message(STATUS "Detected CUDA version: ${CUDAToolkit_VERSION}") + if(USE_CUDA) + find_package(CUDAToolkit REQUIRED) + message(STATUS "Detected CUDA version: ${CUDAToolkit_VERSION}") + endif() # EP_TORCH_VERSIONS: semicolon-separated list of PyTorch versions to build for. # Can be set via -DEP_TORCH_VERSIONS="2.9.1;2.8.0" or the EP_TORCH_VERSIONS env var. @@ -159,6 +161,7 @@ if (WITH_EP) "-DTORCH_CUDA_ARCH_LIST=${_torch_cuda_arch_list_pipe}" "-DSTAGING_DIR=${EP_PG_STAGING_DIR}" "-DENGINE_SO_PATH=$" + "-DEP_USE_MUSA=$,1,0>" -P "${CMAKE_CURRENT_SOURCE_DIR}/mooncake-pg/BuildPgExt.cmake" COMMENT "Building Mooncake PG Python extension(s)" DEPENDS engine mooncake_ep_ext diff --git a/mooncake-pg/BuildPgExt.cmake b/mooncake-pg/BuildPgExt.cmake index 38bfadff..ead67c3e 100644 --- a/mooncake-pg/BuildPgExt.cmake +++ b/mooncake-pg/BuildPgExt.cmake @@ -11,6 +11,7 @@ # TORCH_CUDA_ARCH_LIST - pipe-separated CUDA arch list forwarded to torch # STAGING_DIR - destination directory for the built .so files # ENGINE_SO_PATH - absolute path to the built engine.cpython-XYZ.so +# EP_USE_MUSA - set to "1" when building for MUSA (MTLink path) cmake_minimum_required(VERSION 3.16) @@ -35,6 +36,11 @@ endif() set(ENV{MAKEFLAGS} "") set(ENV{MFLAGS} "") set(ENV{TORCH_CUDA_ARCH_LIST} "${TORCH_CUDA_ARCH_LIST}") +if(EP_USE_MUSA) + set(ENV{MOONCAKE_EP_USE_MUSA} "1") +else() + unset(ENV{MOONCAKE_EP_USE_MUSA}) +endif() # --------------------------------------------------------------------------- # 2. Ensure engine.so exists in mooncake-wheel/mooncake/ for setup.py linking. diff --git a/mooncake-pg/include/mooncake_backend.h b/mooncake-pg/include/mooncake_backend.h index 7e6ee010..1b748999 100644 --- a/mooncake-pg/include/mooncake_backend.h +++ b/mooncake-pg/include/mooncake_backend.h @@ -14,8 +14,38 @@ #include #include +#ifdef MOONCAKE_EP_USE_MUSA +#include +#else +#include +#endif + namespace mooncake { +#ifdef MOONCAKE_EP_USE_MUSA +static inline auto getCurrentGPUStream(int device_index = -1) { + return at::musa::getCurrentMUSAStream(device_index); +} +static inline auto getGPUStreamFromPool(bool non_blocking = false, + int device_index = -1) { + return at::musa::getStreamFromPool(non_blocking, device_index); +} +static inline int currentGPUDevice() { return at::musa::current_device(); } +static inline constexpr auto kGPUDevice = c10::DeviceType::PrivateUse1; +static inline constexpr auto kGPUDeviceType = at::musa::kMUSA; +#else +static inline auto getCurrentGPUStream(int device_index = -1) { + return at::cuda::getCurrentCUDAStream(device_index); +} +static inline auto getGPUStreamFromPool(bool non_blocking = false, + int device_index = -1) { + return at::cuda::getStreamFromPool(non_blocking, device_index); +} +static inline int currentGPUDevice() { return at::cuda::current_device(); } +static inline constexpr auto kGPUDevice = torch::kCUDA; +static inline constexpr auto kGPUDeviceType = c10::DeviceType::CUDA; +#endif + // Forward declaration – MooncakeP2PShim holds a non-owning pointer to // MooncakeBackend, which is defined below. class MooncakeBackend; diff --git a/mooncake-pg/include/mooncake_worker.cuh b/mooncake-pg/include/mooncake_worker.cuh index b924e2eb..c075c753 100644 --- a/mooncake-pg/include/mooncake_worker.cuh +++ b/mooncake-pg/include/mooncake_worker.cuh @@ -1,13 +1,24 @@ #ifndef MOONCAKE_WORKER_CUH #define MOONCAKE_WORKER_CUH +#if !defined(__MUSA__) +#ifdef MOONCAKE_EP_USE_MUSA +#include +#else #include -#include -#include -#include +#endif +#include +#include #include #include #include +#else +// MUSA device compilation: minimal includes to avoid mcc compiler crash +#include +#include +#endif + +#include #include #include @@ -17,8 +28,31 @@ #include #include +#ifdef MOONCAKE_EP_USE_MUSA +#define cudaDeviceSynchronize musaDeviceSynchronize +#define cudaError cudaError_t +#define cudaErrorNotReady musaErrorNotReady +#define cudaEventCreateWithFlags musaEventCreateWithFlags +#define cudaEventDestroy musaEventDestroy +#define cudaEventDisableTiming musaEventDisableTiming +#define cudaEventQuery musaEventQuery +#define cudaEventRecord musaEventRecord +#define cudaFuncAttributes musaFuncAttributes +#define cudaFuncGetAttributes musaFuncGetAttributes +#define cudaHostAlloc musaHostAlloc +#define cudaHostAllocMapped musaHostAllocMapped +#endif + namespace mooncake { +#if !defined(__MUSA__) +#ifdef MOONCAKE_EP_USE_MUSA +using GPUStream = at::musa::MUSAStream; +#else +using GPUStream = at::cuda::CUDAStream; +#endif +#endif + static constexpr size_t kBufferSize = 1u << 24; static constexpr size_t kMaxNumRanks = 64; @@ -36,19 +70,27 @@ struct TransferGroupMeta { int taskCount; bool* activeRanks; bool* activeRanksDevice; +#if !defined(__MUSA__) at::Tensor activeRanksTensor; +#endif bool peerConnected[kMaxNumRanks]{}; TransferEngine* engine; +#if !defined(__MUSA__) c10::intrusive_ptr<::c10d::Store> store; +#endif int bufferBaseIndex; int backendIndex; TransferMetadata::SegmentID segmentIDs[kMaxNumRanks]; SegmentInfo segmentInfos[kMaxNumRanks]; }; -__global__ struct Task { +#if defined(__CUDACC__) || defined(__MUSA__) +__global__ +#endif + struct Task { volatile bool active = false; - c10d::OpType opType = c10d::OpType::UNKNOWN; + int opType = + 0; // c10d::OpType as int, for ABI compatibility with kernel code size_t tensorSize; // In bytes int64_t broadcastRoot; int bufferOffset; @@ -57,6 +99,7 @@ __global__ struct Task { void* transferGroupMeta; }; +#if !defined(__MUSA__) void launchReduceKernel(at::Tensor dst, size_t pos, size_t realSize, void* src, size_t numRanks, c10d::ReduceOp op, bool* activeRanks, cudaStream_t stream); @@ -90,11 +133,11 @@ class MooncakeWorker { c10d::OpType opType, size_t tensorSize, int64_t broadcastRoot, const std::shared_ptr& meta, const std::shared_ptr& connection_ctx, - const at::cuda::CUDAStream& issue_stream, + const GPUStream& issue_stream, const std::function& tensorToBuffer, + const GPUStream&)>& tensorToBuffer, const std::function& bufferToTensor); + const GPUStream&)>& bufferToTensor); void Start(); @@ -159,6 +202,7 @@ class MooncakeWorkerManager { // detached threads must not outlive the MooncakeWorker object. std::unordered_map> workers_; }; +#endif // !defined(__MUSA__) } // namespace mooncake diff --git a/mooncake-pg/include/mooncake_worker_kernels.cuh b/mooncake-pg/include/mooncake_worker_kernels.cuh new file mode 100644 index 00000000..67b56b9e --- /dev/null +++ b/mooncake-pg/include/mooncake_worker_kernels.cuh @@ -0,0 +1,74 @@ +#ifndef MOONCAKE_WORKER_KERNELS_CUH +#define MOONCAKE_WORKER_KERNELS_CUH + +// Include the main worker header for struct definitions (Task, SegmentInfo, +// TransferGroupMeta). When compiled by mcc (__MUSA__ defined), the torch- +// dependent parts are guarded out, making this safe for the MUSA compiler. +#include + +namespace mooncake { + +// Kernel function declarations — guarded so g++ doesn't see __global__ +// which it can't parse. Parameters use plain C++ types (int instead of +// c10d::OpType / c10d::ReduceOp::RedOpType) so that mcc can compile them +// without torch headers. +#if defined(__CUDACC__) || defined(__MUSA__) +__global__ void enqueueTaskKernel(int opType, size_t tensorSize, + int64_t broadcastRoot, int bufferOffset, + uint64_t submitSequence, void* meta, + Task* tasks, int numRanks, + const bool* activeRanks, + int* activeRanksTensor, size_t taskId); + +template +__global__ void reduceKernel(scalar_t* dst, const scalar_t* src, + size_t numElements, size_t numRanks, int op, + bool* activeRanks); +#endif + +// Host-callable kernel launch wrappers (compiled by mcc/nvcc, callable from +// g++) g++ cannot compile <<<>>> syntax, so these wrappers are compiled by the +// GPU compiler and provide plain C++ functions that the host code can call. +// Both CUDA and MUSA use cudaStream_t in the declaration: on MUSA, +// cuda_alike.h typedefs cudaStream_t to musaStream_t. + +void launchEnqueueTaskKernel(int opType, size_t tensorSize, + int64_t broadcastRoot, int bufferOffset, + uint64_t submitSequence, void* meta, Task* tasks, + int numRanks, const bool* activeRanks, + int* activeRanksTensor, size_t taskId, + cudaStream_t stream); + +void launchReduceKernel_uint8(uint8_t* dst, const uint8_t* src, + size_t numElements, size_t numRanks, int op, + bool* activeRanks, cudaStream_t stream); +void launchReduceKernel_int8(int8_t* dst, const int8_t* src, size_t numElements, + size_t numRanks, int op, bool* activeRanks, + cudaStream_t stream); +void launchReduceKernel_int16(int16_t* dst, const int16_t* src, + size_t numElements, size_t numRanks, int op, + bool* activeRanks, cudaStream_t stream); +void launchReduceKernel_int32(int* dst, const int* src, size_t numElements, + size_t numRanks, int op, bool* activeRanks, + cudaStream_t stream); +void launchReduceKernel_int64(int64_t* dst, const int64_t* src, + size_t numElements, size_t numRanks, int op, + bool* activeRanks, cudaStream_t stream); +void launchReduceKernel_float(float* dst, const float* src, size_t numElements, + size_t numRanks, int op, bool* activeRanks, + cudaStream_t stream); +void launchReduceKernel_double(double* dst, const double* src, + size_t numElements, size_t numRanks, int op, + bool* activeRanks, cudaStream_t stream); +void launchReduceKernel_bool(bool* dst, const bool* src, size_t numElements, + size_t numRanks, int op, bool* activeRanks, + cudaStream_t stream); +void launchReduceKernel_bf16(void* dst, const void* src, size_t numElements, + size_t numRanks, int op, bool* activeRanks, + cudaStream_t stream); + +void preloadReduceKernels(); + +} // namespace mooncake + +#endif // MOONCAKE_WORKER_KERNELS_CUH diff --git a/mooncake-pg/setup.py b/mooncake-pg/setup.py index 595b11a7..845960f7 100644 --- a/mooncake-pg/setup.py +++ b/mooncake-pg/setup.py @@ -13,62 +13,118 @@ module_name = "mooncake.pg" + version_suffix abi_flag = int(torch._C._GLIBCXX_USE_CXX11_ABI) current_dir = os.path.abspath(os.path.dirname(__file__)) -# Link against the CUDA driver stub library if available. -# Same approach as mooncake-ep/setup.py. -cuda_libraries = ["ibverbs", "mlx5"] -cuda_library_dirs = [] +use_musa = os.getenv("MOONCAKE_EP_USE_MUSA", "").upper() in {"1", "ON", "TRUE", "YES"} -if CUDA_HOME is not None: - cuda_stub_dir = os.path.join(CUDA_HOME, "lib64", "stubs") - cuda_stub_lib = os.path.join(cuda_stub_dir, "libcuda.so") - if os.path.exists(cuda_stub_lib): - cuda_libraries.insert(0, "cuda") - cuda_library_dirs.append(cuda_stub_dir) +if use_musa: + from torch_musa.utils.musa_extension import MUSAExtension + from torch_musa.utils.musa_extension import BuildExtension as MUSABuildExtension - -setup( - name=module_name, - ext_modules=[ - CUDAExtension( - name=module_name, - include_dirs=[ - os.path.join(current_dir, "include"), - os.path.join(current_dir, "../mooncake-transfer-engine/include"), - ], - sources=[ - "src/pg_py.cpp", - "src/mooncake_backend.cpp", - "src/p2p_proxy.cpp", - "src/mooncake_worker.cu", - "src/mooncake_worker_thread.cpp", - "src/connection_poller.cpp", - ], - extra_compile_args={ - "cxx": [ - f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", - "-std=c++20", - "-O3", - "-g0", + setup( + name=module_name, + ext_modules=[ + MUSAExtension( + name=module_name, + include_dirs=[ + os.path.join(current_dir, "include"), + os.path.join(current_dir, "../mooncake-transfer-engine/include"), ], - "nvcc": [ - f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", - "-std=c++20", - "-Xcompiler", - "-O3", - "-Xcompiler", - "-g0", + sources=[ + "src/pg_py.cpp", + "src/mooncake_backend.cpp", + "src/p2p_proxy.cpp", + "src/mooncake_worker.mu", + "src/mooncake_worker_host.cpp", + "src/mooncake_worker_thread.cpp", + "src/connection_poller.cpp", ], - }, - libraries=cuda_libraries, - library_dirs=cuda_library_dirs, - extra_link_args=[ - "-Wl,-rpath,$ORIGIN", - "-L" + os.path.join(current_dir, "../mooncake-wheel/mooncake"), - "-Wl,--push-state,--no-as-needed", - "-l:engine.so", - "-Wl,--pop-state", - ], - ) - ], - cmdclass={"build_ext": BuildExtension}, -) + extra_compile_args={ + "cxx": [ + f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", + "-DUSE_MUSA", + "-DMOONCAKE_EP_USE_MUSA=1", + "-std=c++20", + "-O3", + "-g0", + ], + "mcc": [ + f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", + "-DUSE_MUSA", + "-DMOONCAKE_EP_USE_MUSA=1", + "-std=c++20", + "--cuda-gpu-arch=mp_21", + "--cuda-gpu-arch=mp_31", + "-O3", + ], + }, + libraries=["ibverbs", "mlx5"], + extra_link_args=[ + "-Wl,-rpath,$ORIGIN", + "-L" + os.path.join(current_dir, "../mooncake-wheel/mooncake"), + "-Wl,--push-state,--no-as-needed", + "-l:engine.so", + "-Wl,--pop-state", + ], + ) + ], + cmdclass={"build_ext": MUSABuildExtension}, + ) +else: + # Link against the CUDA driver stub library if available. + # Same approach as mooncake-ep/setup.py. + cuda_libraries = ["ibverbs", "mlx5"] + cuda_library_dirs = [] + + if CUDA_HOME is not None: + cuda_stub_dir = os.path.join(CUDA_HOME, "lib64", "stubs") + cuda_stub_lib = os.path.join(cuda_stub_dir, "libcuda.so") + if os.path.exists(cuda_stub_lib): + cuda_libraries.insert(0, "cuda") + cuda_library_dirs.append(cuda_stub_dir) + + setup( + name=module_name, + ext_modules=[ + CUDAExtension( + name=module_name, + include_dirs=[ + os.path.join(current_dir, "include"), + os.path.join(current_dir, "../mooncake-transfer-engine/include"), + ], + sources=[ + "src/pg_py.cpp", + "src/mooncake_backend.cpp", + "src/p2p_proxy.cpp", + "src/mooncake_worker.cu", + "src/mooncake_worker_host.cpp", + "src/mooncake_worker_thread.cpp", + "src/connection_poller.cpp", + ], + extra_compile_args={ + "cxx": [ + f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", + "-std=c++20", + "-O3", + "-g0", + ], + "nvcc": [ + f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", + "-std=c++20", + "-Xcompiler", + "-O3", + "-Xcompiler", + "-g0", + ], + }, + libraries=cuda_libraries, + library_dirs=cuda_library_dirs, + extra_link_args=[ + "-Wl,-rpath,$ORIGIN", + "-L" + os.path.join(current_dir, "../mooncake-wheel/mooncake"), + "-Wl,--push-state,--no-as-needed", + "-l:engine.so", + "-Wl,--pop-state", + ], + ) + ], + cmdclass={"build_ext": BuildExtension}, + ) diff --git a/mooncake-pg/src/connection_poller.cpp b/mooncake-pg/src/connection_poller.cpp index 4b29d73b..9f74f62b 100644 --- a/mooncake-pg/src/connection_poller.cpp +++ b/mooncake-pg/src/connection_poller.cpp @@ -1,9 +1,6 @@ #include #include -#include -#include #include -#include #include #include #include @@ -23,6 +20,7 @@ namespace mooncake { // On MNNVL clusters all GPUs support fabric mem handles, meaning // NVLink transport can only access cuMemCreate(FABRIC) memory // cross-node -- CPU heap buffers are invisible to remote peers. +#ifndef MOONCAKE_EP_USE_MUSA static bool supportFabricMem() { const char* nvlink_ipc = getenv("MC_USE_NVLINK_IPC"); @@ -41,6 +39,10 @@ static bool supportFabricMem() { } return true; } +#else +// MUSA does not support NVLink fabric memory handles. +static bool supportFabricMem() { return false; } +#endif ConnectionContext::ConnectionContext(int backendIndex, int rank, int size, bool isDummy, uint64_t* local2global_rank_map, diff --git a/mooncake-pg/src/mooncake_backend.cpp b/mooncake-pg/src/mooncake_backend.cpp index ebac10fb..40ce0e60 100644 --- a/mooncake-pg/src/mooncake_backend.cpp +++ b/mooncake-pg/src/mooncake_backend.cpp @@ -1,8 +1,11 @@ +#ifdef MOONCAKE_EP_USE_MUSA +#include +#else #include -#include +#endif +#include #include #include -#include #include #include #include @@ -294,7 +297,7 @@ MooncakeBackend::MooncakeBackend( } auto& dev_worker_mgr = P2PDeviceWorkerManager::getInstance(); - int cuda_device_index = isCpu_ ? -1 : at::cuda::current_device(); + int cuda_device_index = isCpu_ ? -1 : currentGPUDevice(); if (isCpu_) p2p_device_worker_ = dev_worker_mgr.getCPUWorker(engine_); @@ -385,8 +388,9 @@ MooncakeBackend::MooncakeBackend( TORCH_CHECK(options_->activeRanks_.device().is_cpu(), "activeRanks must be on CPU."); } else { - TORCH_CHECK(options_->activeRanks_.device().is_cuda(), - "activeRanks must be on CUDA."); + TORCH_CHECK( + options_->activeRanks_.device().type() == kGPUDeviceType, + "activeRanks must be on GPU."); } if (max_size != size) { TORCH_CHECK(options_->activeRanks_.numel() == max_size, @@ -395,9 +399,9 @@ MooncakeBackend::MooncakeBackend( } meta_->activeRanksTensor = options_->activeRanks_; } else { - meta_->activeRanksTensor = at::ones( - {max_size}, torch::dtype(torch::kInt32) - .device(isCpu ? torch::kCPU : torch::kCUDA)); + meta_->activeRanksTensor = + at::ones({max_size}, torch::dtype(torch::kInt32) + .device(isCpu ? torch::kCPU : kGPUDevice)); if (max_size != size) { meta_->activeRanksTensor.slice(0, size, max_size).fill_(0); } @@ -421,10 +425,12 @@ MooncakeBackend::MooncakeBackend( // Register a lightweight Backend shim so that PyTorch's P2P dispatch path // (batch_isend_irecv → _get_backend → getBackend) can find a registered // Backend for this ProcessGroup. The shim delegates send/recv back to us. - auto deviceType = isCpu ? c10::DeviceType::CPU : c10::DeviceType::CUDA; + auto deviceType = isCpu ? c10::DeviceType::CPU : kGPUDeviceType; auto shim = c10::make_intrusive(this); setBackend(deviceType, BackendType::CUSTOM, shim); +#ifndef MOONCAKE_EP_USE_MUSA setDefaultBackend(BackendType::CUSTOM); +#endif // Increment backend index ++backendIndex_; @@ -480,8 +486,7 @@ c10::intrusive_ptr MooncakeBackend::send( P2PProxy::OpStatus::kPending); cudaStream_t stream = nullptr; if (!isCpu_) { - auto current_stream = - at::cuda::getCurrentCUDAStream(contiguous.device().index()); + auto current_stream = getCurrentGPUStream(contiguous.device().index()); stream = current_stream.stream(); } @@ -513,8 +518,7 @@ c10::intrusive_ptr MooncakeBackend::recv( P2PProxy::OpStatus::kPending); cudaStream_t stream = nullptr; if (!isCpu_) { - auto current_stream = - at::cuda::getCurrentCUDAStream(target.device().index()); + auto current_stream = getCurrentGPUStream(target.device().index()); stream = current_stream.stream(); } @@ -549,13 +553,12 @@ c10::intrusive_ptr MooncakeBackend::broadcast( memcpy((char*)tensor.data_ptr() + pos, src, realSize); }); } else { - at::cuda::CUDAStream stream = - at::cuda::getCurrentCUDAStream(tensor.device().index()); + GPUStream stream = getCurrentGPUStream(tensor.device().index()); return worker_->putTaskCuda( c10d::OpType::BROADCAST, tensorSize, root, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { if (isRoot) { cudaMemcpyAsync(dst, (char*)tensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, @@ -563,7 +566,7 @@ c10::intrusive_ptr MooncakeBackend::broadcast( } }, [=](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemcpyAsync((char*)tensor.data_ptr() + pos, src, realSize, cudaMemcpyDeviceToDevice, enq_stream); }); @@ -589,17 +592,17 @@ c10::intrusive_ptr MooncakeBackend::allreduce( opts.reduceOp, meta_->activeRanks); }); } else { - auto stream = at::cuda::getCurrentCUDAStream(tensor.device().index()); + auto stream = getCurrentGPUStream(tensor.device().index()); return worker_->putTaskCuda( c10d::OpType::ALLREDUCE, tensorSize, 0, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemcpyAsync(dst, (char*)tensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=, this](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemsetAsync((char*)tensor.data_ptr() + pos, 0, realSize, enq_stream); launchReduceKernel(tensor, pos, realSize, src, meta_->size, @@ -630,18 +633,17 @@ c10::intrusive_ptr MooncakeBackend::allgather( } }); } else { - auto stream = - at::cuda::getCurrentCUDAStream(inputTensor.device().index()); + auto stream = getCurrentGPUStream(inputTensor.device().index()); return worker_->putTaskCuda( c10d::OpType::ALLGATHER, tensorSize, 0, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemcpyAsync(dst, (char*)inputTensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { for (const auto j : c10::irange(outputTensors_.size())) { cudaMemcpyAsync((char*)outputTensors_[j].data_ptr() + pos, (char*)src + j * realSize, realSize, @@ -671,18 +673,17 @@ c10::intrusive_ptr MooncakeBackend::_allgather_base( } }); } else { - auto stream = - at::cuda::getCurrentCUDAStream(inputBuffer.device().index()); + auto stream = getCurrentGPUStream(inputBuffer.device().index()); return worker_->putTaskCuda( c10d::OpType::_ALLGATHER_BASE, tensorSize, 0, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemcpyAsync(dst, (char*)inputBuffer.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=, this](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { for (int j = 0; j < meta_->size; ++j) { if (!meta_->activeRanks[j]) continue; cudaMemcpyAsync( @@ -717,13 +718,12 @@ c10::intrusive_ptr MooncakeBackend::_reduce_scatter_base( opts.reduceOp, meta_->activeRanks); }); } else { - auto stream = - at::cuda::getCurrentCUDAStream(inputBuffer.device().index()); + auto stream = getCurrentGPUStream(inputBuffer.device().index()); return worker_->putTaskCuda( c10d::OpType::_REDUCE_SCATTER_BASE, tensorSize, 0, meta_, connection_ctx_, stream, [=, this](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { for (int j = 0; j < meta_->size; ++j) { if (!meta_->activeRanks[j]) continue; cudaMemcpyAsync( @@ -733,7 +733,7 @@ c10::intrusive_ptr MooncakeBackend::_reduce_scatter_base( } }, [=, this](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemsetAsync((char*)outputBuffer.data_ptr() + pos, 0, realSize, enq_stream); launchReduceKernel(outputBuffer, pos, realSize, src, @@ -764,13 +764,12 @@ c10::intrusive_ptr MooncakeBackend::alltoall( } }); } else { - auto stream = - at::cuda::getCurrentCUDAStream(inputTensors[0].device().index()); + auto stream = getCurrentGPUStream(inputTensors[0].device().index()); return worker_->putTaskCuda( c10d::OpType::ALLTOALL, tensorSize, 0, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { for (const auto j : c10::irange(inputTensors.size())) { cudaMemcpyAsync((char*)dst + j * realSize, (char*)inputTensors[j].data_ptr() + pos, @@ -779,7 +778,7 @@ c10::intrusive_ptr MooncakeBackend::alltoall( } }, [=](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { for (const auto j : c10::irange(outputTensors.size())) { cudaMemcpyAsync((char*)outputTensors[j].data_ptr() + pos, (char*)src + j * realSize, realSize, @@ -798,13 +797,13 @@ c10::intrusive_ptr MooncakeBackend::barrier( connection_ctx_, [=](void*, size_t, size_t) {}, [=](void*, size_t, size_t) {}); } else { - auto device_index = at::cuda::current_device(); - auto stream = at::cuda::getCurrentCUDAStream(device_index); + auto device_index = currentGPUDevice(); + auto stream = getCurrentGPUStream(device_index); return worker_->putTaskCuda( c10d::OpType::BARRIER, kBarrierDummyTensorSize, 0, meta_, connection_ctx_, stream, - [=](void*, size_t, size_t, const at::cuda::CUDAStream&) {}, - [=](void*, size_t, size_t, const at::cuda::CUDAStream&) {}); + [=](void*, size_t, size_t, const GPUStream&) {}, + [=](void*, size_t, size_t, const GPUStream&) {}); } } @@ -830,17 +829,17 @@ c10::intrusive_ptr MooncakeBackend::reduce( } }); } else { - auto stream = at::cuda::getCurrentCUDAStream(tensor.device().index()); + auto stream = getCurrentGPUStream(tensor.device().index()); return worker_->putTaskCuda( c10d::OpType::REDUCE, tensorSize, root, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemcpyAsync(dst, (char*)tensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=, this](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { if (isRoot) { cudaMemsetAsync((char*)tensor.data_ptr() + pos, 0, realSize, enq_stream); @@ -879,18 +878,17 @@ c10::intrusive_ptr MooncakeBackend::gather( } }); } else { - auto stream = - at::cuda::getCurrentCUDAStream(inputTensor.device().index()); + auto stream = getCurrentGPUStream(inputTensor.device().index()); return worker_->putTaskCuda( c10d::OpType::GATHER, tensorSize, root, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemcpyAsync(dst, (char*)inputTensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { if (isRoot) { auto outputTensors_ = outputTensors.back(); for (const auto j : c10::irange(outputTensors_.size())) { @@ -933,13 +931,12 @@ c10::intrusive_ptr MooncakeBackend::scatter( memcpy((char*)outputTensor.data_ptr() + pos, src, realSize); }); } else { - auto stream = - at::cuda::getCurrentCUDAStream(outputTensor.device().index()); + auto stream = getCurrentGPUStream(outputTensor.device().index()); return worker_->putTaskCuda( c10d::OpType::SCATTER, tensorSize, root, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { if (isRoot) { auto inputTensors_ = inputTensors.back(); for (const auto j : c10::irange(inputTensors_.size())) { @@ -951,7 +948,7 @@ c10::intrusive_ptr MooncakeBackend::scatter( } }, [=](void* src, size_t pos, size_t realSize, - const at::cuda::CUDAStream& enq_stream) { + const GPUStream& enq_stream) { cudaMemcpyAsync((char*)outputTensor.data_ptr() + pos, src, realSize, cudaMemcpyDeviceToDevice, enq_stream); }); @@ -1024,7 +1021,7 @@ void MooncakeBackend::syncActiveRanksTensor() { if (!meta_->activeRanksTensor.defined() || meta_->activeRanksTensor.size(0) != meta_->size) { meta_->activeRanksTensor = - cpu_tensor.to(isCpu_ ? torch::kCPU : torch::kCUDA); + cpu_tensor.to(isCpu_ ? torch::kCPU : kGPUDevice); return; } @@ -1105,15 +1102,14 @@ int MooncakeBackend::getNumSyncedRanks() { std::vector tensors; tensors.emplace_back(torch::tensor( connection_ctx_->getTotalConnectedPeers(), - torch::dtype(torch::kInt).device(isCpu_ ? torch::kCPU : torch::kCUDA))); + torch::dtype(torch::kInt).device(isCpu_ ? torch::kCPU : kGPUDevice))); c10d::AllreduceOptions opts{ .reduceOp = c10d::ReduceOp::MIN, }; auto work = allreduce(tensors, opts); work->wait(); if (!isCpu_) { - auto stream = - at::cuda::getCurrentCUDAStream(tensors[0].device().index()); + auto stream = getCurrentGPUStream(tensors[0].device().index()); cudaStreamSynchronize(stream); } return tensors[0].cpu().item(); @@ -1176,15 +1172,14 @@ std::vector MooncakeBackend::getPeerState(const std::vector& ranks) { std::vector tensors; tensors.emplace_back(torch::tensor( input, torch::dtype(torch::kInt) - .device(isCpu_ ? torch::kCPU : torch::kCUDA))); + .device(isCpu_ ? torch::kCPU : kGPUDevice))); c10d::AllreduceOptions opts{ .reduceOp = c10d::ReduceOp::MIN, }; auto work = allreduce(tensors, opts); work->wait(); if (!isCpu_) { - auto stream = - at::cuda::getCurrentCUDAStream(tensors[0].device().index()); + auto stream = getCurrentGPUStream(tensors[0].device().index()); cudaStreamSynchronize(stream); } bool activeRanksChanged = false; diff --git a/mooncake-pg/src/mooncake_worker.cu b/mooncake-pg/src/mooncake_worker.cu index 10e7aa56..127ecbcb 100644 --- a/mooncake-pg/src/mooncake_worker.cu +++ b/mooncake-pg/src/mooncake_worker.cu @@ -1,163 +1,21 @@ -#include -#include -#include -#include -#include -#include +// mooncake_worker.cu — GPU kernel functions and launch wrappers. +// Compiled by nvcc (CUDA) and mcc (MUSA, via mooncake_worker.mu symlink). +// The __MUSA__ branch avoids torch headers to stay compatible with mcc. -#include "pg_utils.h" +#include + +#ifdef __MUSA__ +#include +#endif namespace mooncake { -class MooncakeWorkCpu : public ::c10d::Work { - public: - MooncakeWorkCpu(c10d::OpType opType, - c10::intrusive_ptr future, - std::shared_ptr meta) - : Work(-1, opType), - future_(std::move(future)), - meta_(std::move(meta)) {} +// ── Kernel functions ────────────────────────────────────────────── +// Both CUDA and MUSA share the same kernel bodies. Parameters use plain +// C++ types (int instead of c10d::OpType / c10d::ReduceOp::RedOpType) +// so that mcc can compile them without torch headers. - bool isCompleted() override { return future_->completed(); } - - bool wait(std::chrono::milliseconds timeout) override { - future_->wait(); - return future_->completed() && !future_->hasError(); - } - - private: - c10::intrusive_ptr future_; - std::shared_ptr meta_; -}; - -class MooncakeWorkCuda : public ::c10d::Work { - public: - MooncakeWorkCuda(c10d::OpType opType, std::shared_ptr event, - std::shared_ptr meta, - const MooncakeWorker* worker, - std::vector submitted_tasks) - : Work(-1, opType), - event_(std::move(event)), - meta_(std::move(meta)), - worker_(worker), - submitted_tasks_(std::move(submitted_tasks)) {} - - bool isCompleted() override { return event_->query(); } - - bool wait(std::chrono::milliseconds timeout) override { - // Wait until the task has been submitted to TransferEngine: - // This tries to ensure that the CUDA kernels required for the transfer - // have been launched by the time `waitUntilTasksSubmitted` returns. - // - // Why is this needed? PyTorch documentation implies that collective - // operations should be enqueued when `wait()` returns. In practice, we - // found that violating this causes hangs. - // - // Our current hypothesis for the hang is: PyTorch assumes the kernels - // needed for the transfer are already launched when `wait` returns - // true. It may then launch subsequent operations after the collective - // (e.g., `.cpu()`). Such operations may acquire a process-wide lock in - // the CUDA runtime. Also, they may rely on the data produced by the - // collective, thus causing a synchronization on enq_stream. However, - // holding that runtime lock prevents cudaMemcpy(Async) in TE/TENT from - // launching. This means the transfer can't finish, and enq_stream won't - // complete. Thus, a deadlock occurs. - // (In practice, we found that replacing all cudaMemcpyAsync in TENT - // with cuMemcpyAsync actually alleviates this, which further suggests a - // deadlock in the CUDA runtime. However, that change is too invasive - // for TE/TENT, so we do not adopt it here.) - // - // Strictly speaking, the wait is needed for another reason: The current - // stream will be blocked on the event below. Any subsequent work on - // `current_stream` will wait on that event, which effectively waits for - // the task to be done. Therefore, we must ensure all kernels needed for - // the transfer task are launched BEFORE blocking the current stream, in - // case TE/TENT use `current_stream` to launch those kernels (though it - // is rare). - // - // Please note that this logic relies on the assumption that TE/TENT - // will launch all CUDA operations in `submitTransfer`. - // Unfortunately, TcpTransport in TE and TENT currently violates this - // assumption (cudaMemcpy(Async) may be called later from a callback), - // which can cause hangs in PG when a CUDA operation such as - // `x.cpu().item()` follows the collective. For TE's TcpTransport, the - // use of cudaMemcpy on the default stream may also contribute to the - // hang. - // - // Besides, for CPU-only transports (like RdmaTransport), - // waitUntilTasksSubmitted is totally unnecessary, but we keep it for - // uniform behavior to avoid invasive changes to TE/TENT. - bool submitted = true; - if (at::cuda::currentStreamCaptureStatus() == - c10::cuda::CaptureStatus::None) { - // Normal execution: block until tasks are submitted. - submitted = - worker_->waitUntilTasksSubmitted(submitted_tasks_, timeout); - } else { - // During CUDA graph capture, kernels are recorded but not actually - // executed. The enqueueTaskKernel would never run, so - // waitUntilTasksSubmitted would hang because the CPU worker thread - // never sees task.active == true. - // - // Note that this also means NvlinkTransport (and TcpTransport too, - // of course) won't work with CUDA Graphs: Kernels launched inside - // TE/TENT can't be captured by the graph, and during replay they - // are not ordered with the graph execution. This may trigger the - // same deadlock described above. - } - if (!submitted) return false; - - // Once all tasks have been submitted, use the event to synchronize - // the current stream and the enqueue stream, but do not wait on this - // event. - // - // See PyTorch docs for more details: - // https://docs.pytorch.org/docs/stable/distributed.html#synchronous-and-asynchronous-collective-operations - // "wait() - in the case of CPU collectives, will block the process - // until the operation is completed. In the case of CUDA collectives, - // will block the currently active CUDA stream until the operation - // is completed (but will not block the CPU)." - auto current_stream = at::cuda::getCurrentCUDAStream(); - event_->block(current_stream); - return true; - } - - protected: - std::shared_ptr event_; - std::shared_ptr meta_; - const MooncakeWorker* worker_; - std::vector submitted_tasks_; -}; - -class MooncakeBarrierWorkCuda : public MooncakeWorkCuda { - public: - using MooncakeWorkCuda::MooncakeWorkCuda; - - bool wait(std::chrono::milliseconds timeout) override { - // Skip host-side synchronization during CUDA graph capture. - // cudaEventSynchronize is not permitted while a stream is capturing. - if (at::cuda::currentStreamCaptureStatus() != - c10::cuda::CaptureStatus::None) { - // We still need stream-level synchronization so that subsequent - // operations on the capture stream are ordered after the barrier - // task on the enqueue stream. - auto current_stream = at::cuda::getCurrentCUDAStream(); - event_->block(current_stream); - return true; - } - - if (timeout == kNoTimeout) { - event_->synchronize(); - return true; - } - - BackoffWaiter waiter( - BackoffWaiterConfig::constantSleep(std::chrono::microseconds(10))); - return waiter.wait_for(timeout, [this] { return event_->query(); }); - } -}; - -__global__ void enqueueTaskKernel(c10d::OpType opType, size_t tensorSize, +__global__ void enqueueTaskKernel(int opType, size_t tensorSize, int64_t broadcastRoot, int bufferOffset, uint64_t submitSequence, void* meta, Task* tasks, int numRanks, @@ -186,371 +44,173 @@ __global__ void enqueueTaskKernel(c10d::OpType opType, size_t tensorSize, template __global__ void reduceKernel(scalar_t* dst, const scalar_t* src, - size_t numElements, size_t numRanks, - c10d::ReduceOp::RedOpType op, bool* activeRanks) { + size_t numElements, size_t numRanks, int op, + bool* activeRanks) { size_t thread_idx = blockIdx.x * blockDim.x + threadIdx.x; size_t stride = blockDim.x * gridDim.x; + +#ifdef __MUSA__ + // mt_bfloat16 lacks arithmetic and comparison operators; use float + // accumulator and fminf/fmaxf for that type only. Other types + // (float, double, int64_t, etc.) use their native accumulator to + // preserve full precision. + constexpr bool kIsBf16 = std::is_same_v; +#else + constexpr bool kIsBf16 = false; +#endif + using acc_t = std::conditional_t; + for (size_t elem_idx = thread_idx; elem_idx < numElements; elem_idx += stride) { bool valid = false; - scalar_t acc = 0; + acc_t acc = 0; for (size_t rank = 0; rank < numRanks; ++rank) { if (activeRanks[rank]) { if (!valid) { - acc = src[rank * numElements + elem_idx]; + if constexpr (kIsBf16) { + acc = (float)src[rank * numElements + elem_idx]; + } else { + acc = src[rank * numElements + elem_idx]; + } valid = true; } else { - switch (op) { - case c10d::ReduceOp::SUM: - acc += src[rank * numElements + elem_idx]; - break; - case c10d::ReduceOp::MIN: - acc = std::min(src[rank * numElements + elem_idx], - acc); - break; - case c10d::ReduceOp::MAX: - acc = std::max(src[rank * numElements + elem_idx], - acc); - break; - case c10d::ReduceOp::PRODUCT: - acc *= src[rank * numElements + elem_idx]; - break; - default: - // never - } - } - } - } - dst[elem_idx] = acc; - } -} - -namespace { - -template -void preload_reduce_kernel(const char* name) { - cudaFuncAttributes attr{}; - auto err = cudaFuncGetAttributes( - &attr, reinterpret_cast(reduceKernel)); - TORCH_CHECK(err == cudaSuccess, "Failed to preload kernel ", name, ": ", - cudaGetErrorString(err)); -} - -} // namespace - -void launchReduceKernel(at::Tensor dst, size_t pos, size_t realSize, void* src, - size_t numRanks, c10d::ReduceOp op, bool* activeRanks, - cudaStream_t stream) { - TORCH_CHECK(op == c10d::ReduceOp::SUM || op == c10d::ReduceOp::MIN || - op == c10d::ReduceOp::MAX || op == c10d::ReduceOp::PRODUCT, - "Only support SUM/MIN/MAX/PRODUCT for reduction."); - auto ptr = (char*)dst.data_ptr() + pos; - size_t num = realSize / dst.element_size(); - - switch (dst.scalar_type()) { - case c10::kByte: - reduceKernel<<<64, 256, 0, stream>>>((uint8_t*)ptr, (uint8_t*)src, - num, numRanks, op.op_, - activeRanks); - break; - case c10::kChar: - reduceKernel<<<64, 256, 0, stream>>>( - (int8_t*)ptr, (int8_t*)src, num, numRanks, op.op_, activeRanks); - break; - case c10::kShort: - reduceKernel<<<64, 256, 0, stream>>>((int16_t*)ptr, (int16_t*)src, - num, numRanks, op.op_, - activeRanks); - break; - case c10::kInt: - reduceKernel<<<64, 256, 0, stream>>>((int*)ptr, (int*)src, num, - numRanks, op.op_, activeRanks); - break; - case c10::kLong: - reduceKernel<<<64, 256, 0, stream>>>((int64_t*)ptr, (int64_t*)src, - num, numRanks, op.op_, - activeRanks); - break; - case c10::kFloat: - reduceKernel<<<64, 256, 0, stream>>>((float*)ptr, (float*)src, num, - numRanks, op.op_, activeRanks); - break; - case c10::kDouble: - reduceKernel<<<64, 256, 0, stream>>>( - (double*)ptr, (double*)src, num, numRanks, op.op_, activeRanks); - break; - case c10::kBool: - reduceKernel<<<64, 256, 0, stream>>>((bool*)ptr, (bool*)src, num, - numRanks, op.op_, activeRanks); - break; - case c10::kBFloat16: - reduceKernel<<<64, 256, 0, stream>>>((at::BFloat16*)ptr, - (at::BFloat16*)src, num, - numRanks, op.op_, activeRanks); - break; - default: - TORCH_CHECK(false, c10::str("Unsupported reduce dtype: ", - dst.scalar_type())); - } -} - -template -T applyReduceOp(const T& a, const T& b, c10d::ReduceOp op) { - switch (op) { - case c10d::ReduceOp::SUM: - return a + b; - case c10d::ReduceOp::PRODUCT: - return a * b; - case c10d::ReduceOp::MIN: - return std::min(a, b); - case c10d::ReduceOp::MAX: - return std::max(a, b); - default: - TORCH_CHECK(false, c10::str("Unsupported reduce op: ", op)); - } -} - -template -void reduceCpu(T* dst, const T* src, size_t numElements, size_t numRanks, - c10d::ReduceOp op, bool* activeRanks) { - at::parallel_for(0, numElements, 1024, [&](int64_t begin, int64_t end) { - for (int64_t i = begin; i < end; ++i) { - bool valid = false; - T acc{}; - for (int64_t rank = 0; rank < numRanks; ++rank) { - if (activeRanks[rank]) { - if (!valid) { - acc = src[i + rank * numElements]; - valid = true; + if constexpr (kIsBf16) { + float val = (float)src[rank * numElements + elem_idx]; + switch (op) { + case 0: // SUM + acc += val; + break; + case 2: // PRODUCT + acc *= val; + break; + case 3: // MIN + acc = fminf(acc, val); + break; + case 4: // MAX + acc = fmaxf(acc, val); + break; + default: + break; + } } else { - acc = - applyReduceOp(acc, src[i + rank * numElements], op); + switch (op) { + case 0: // SUM + acc += src[rank * numElements + elem_idx]; + break; + case 2: // PRODUCT + acc *= src[rank * numElements + elem_idx]; + break; + case 3: // MIN + acc = std::min( + src[rank * numElements + elem_idx], acc); + break; + case 4: // MAX + acc = std::max( + src[rank * numElements + elem_idx], acc); + break; + default: + break; + } } } } - dst[i] = acc; } - }); -} - -void launchReduceCpu(at::Tensor dst, size_t pos, size_t realSize, void* src, - size_t numRanks, c10d::ReduceOp op, bool* activeRanks) { - auto ptr = (char*)dst.data_ptr() + pos; - size_t num = realSize / dst.element_size(); - - switch (dst.scalar_type()) { - case c10::kByte: - reduceCpu((uint8_t*)ptr, (uint8_t*)src, num, numRanks, op, - activeRanks); - break; - case c10::kChar: - reduceCpu((int8_t*)ptr, (int8_t*)src, num, numRanks, op, - activeRanks); - break; - case c10::kShort: - reduceCpu((int16_t*)ptr, (int16_t*)src, num, numRanks, op, - activeRanks); - break; - case c10::kInt: - reduceCpu((int*)ptr, (int*)src, num, numRanks, op, activeRanks); - break; - case c10::kLong: - reduceCpu((int64_t*)ptr, (int64_t*)src, num, numRanks, op, - activeRanks); - break; - case c10::kFloat: - reduceCpu((float*)ptr, (float*)src, num, numRanks, op, activeRanks); - break; - case c10::kDouble: - reduceCpu((double*)ptr, (double*)src, num, numRanks, op, - activeRanks); - break; - case c10::kBool: - reduceCpu((bool*)ptr, (bool*)src, num, numRanks, op, activeRanks); - break; - default: - TORCH_CHECK(false, c10::str("Unsupported reduce dtype: ", - dst.scalar_type())); - } -} - -void preloadReduceKernels() { - preload_reduce_kernel("reduceKernel"); - preload_reduce_kernel("reduceKernel"); - preload_reduce_kernel("reduceKernel"); - preload_reduce_kernel("reduceKernel"); - preload_reduce_kernel("reduceKernel"); - preload_reduce_kernel("reduceKernel"); - preload_reduce_kernel("reduceKernel"); - preload_reduce_kernel("reduceKernel"); - preload_reduce_kernel("reduceKernel"); -} - -MooncakeWorker::MooncakeWorker(int cuda_device_index) - : cuda_device_index_(cuda_device_index) { - int deviceCount = 0; - cudaError err = cudaGetDeviceCount(&deviceCount); - if (!err && deviceCount > 0) { - // Pin memory for task array - cudaHostAlloc(&tasks_, kNumTasks_ * sizeof(Task), cudaHostAllocMapped); - cudaHostGetDevicePointer(&tasks_device_, tasks_, 0); - } else { - LOG(WARNING) << "No CUDA device found. Only the `mooncake-cpu` backend " - "can be used."; - tasks_ = new Task[kNumTasks_]; - } - for (size_t i = 0; i < kNumTasks_; ++i) { - tasks_[i].active = false; - tasks_[i].submitSequence = 0; - submitted_task_sequence_[i].store(0, std::memory_order_relaxed); - } -} - -MooncakeWorker::~MooncakeWorker() { - running_ = false; - if (worker_thread_.joinable()) { - worker_thread_.join(); - } -} - -c10::intrusive_ptr MooncakeWorker::putTaskCpu( - c10d::OpType opType, size_t tensorSize, int64_t broadcastRoot, - const std::shared_ptr& meta, - const std::shared_ptr& connection_ctx, - const std::function& - tensorToBuffer, - const std::function& - bufferToTensor) { - connection_ctx->waitUntilNewRanksConnected(); - - size_t chunkSize = ((kBufferSize - 1) / meta->size) & ~(size_t)7; - auto future = c10::make_intrusive( - c10::ListType::create(c10::TensorType::get())); - - struct IterState { - size_t currentPos = 0; - }; - auto state = std::make_shared(); - - auto processNextChunk = std::make_shared>(); - std::weak_ptr> weakProcessNextChunk = - processNextChunk; - - *processNextChunk = [this, weakProcessNextChunk, state, opType, tensorSize, - chunkSize, broadcastRoot, meta, tensorToBuffer, - bufferToTensor, future]() { - auto processNextChunk = weakProcessNextChunk.lock(); - - if (state->currentPos >= tensorSize) { - future->markCompleted(c10::IValue()); - return; + if constexpr (kIsBf16) { + dst[elem_idx] = (scalar_t)acc; + } else { + dst[elem_idx] = acc; } - - int taskId = cpuTaskCount % 2; - TORCH_CHECK(!tasks_[taskId].active); - - size_t realSize = std::min(chunkSize, tensorSize - state->currentPos); - int bufferOffset = meta->taskCount % 2; - - tasks_[taskId].opType = opType; - tasks_[taskId].tensorSize = realSize; - tasks_[taskId].broadcastRoot = broadcastRoot; - tasks_[taskId].bufferOffset = bufferOffset; - tasks_[taskId].transferGroupMeta = meta.get(); - tensorToBuffer( - (void*)meta->segmentInfos[meta->rank].send_buffer[bufferOffset], - state->currentPos, realSize); - - hasCallback_[taskId] = true; - - callbacks_[taskId] = [this, processNextChunk, state, meta, - bufferToTensor, bufferOffset, realSize, - future]() { - for (int i = 0; i < meta->size; ++i) { - meta->activeRanksTensor[i] = meta->activeRanks[i] ? 1 : 0; - } - bufferToTensor( - (void*)meta->segmentInfos[meta->rank].recv_buffer[bufferOffset], - state->currentPos, realSize); - - state->currentPos += realSize; - - (*processNextChunk)(); - }; - - tasks_[taskId].active = true; - ++cpuTaskCount; - ++meta->taskCount; - }; - - (*processNextChunk)(); - - return c10::make_intrusive(opType, future, meta); -} - -c10::intrusive_ptr MooncakeWorker::putTaskCuda( - c10d::OpType opType, size_t tensorSize, int64_t broadcastRoot, - const std::shared_ptr& meta, - const std::shared_ptr& connection_ctx, - const at::cuda::CUDAStream& issue_stream, - const std::function& tensorToBuffer, - const std::function& bufferToTensor) { - connection_ctx->waitUntilNewRanksConnected(); - - // TORCH_CHECK(tensorSize * meta->size < kBufferSize, "Too large!"); - // Alternately use even-odd items to maintain tasks - size_t chunkSize = ((kBufferSize - 1) / meta->size) & ~(size_t)7; - - // Get a non-blocking stream for enqueue: - // The incoming `issue_stream` may be the Null Stream, which enforces - // implicit synchronization semantics. Launching a spin-wait kernel - // (enqueueTaskKernel) on such a stream can introduce potential deadlock. - at::cuda::CUDAStream enq_stream = - at::cuda::getStreamFromPool(false, issue_stream.device_index()); - - // Synchronize: enq_stream waits for issue_stream - auto event_start = std::make_shared(torch::kCUDA); - event_start->record(issue_stream); - event_start->block(enq_stream); - - std::vector submitted_tasks; - submitted_tasks.reserve((tensorSize + chunkSize - 1) / chunkSize); - for (size_t pos = 0; pos < tensorSize; pos += chunkSize) { - size_t realSize = min(tensorSize, pos + chunkSize) - pos; - int taskId = cudaTaskCount % 2 + 2; - int bufferOffset = meta->taskCount % 2; - const uint64_t taskSequence = - next_cuda_task_sequence_.fetch_add(1, std::memory_order_relaxed); - submitted_tasks.push_back( - {.task_id = static_cast(taskId), .sequence = taskSequence}); - tensorToBuffer( - (void*)meta->segmentInfos[meta->rank].send_buffer[bufferOffset], - pos, realSize, enq_stream); - - hasCallback_[taskId] = false; - enqueueTaskKernel<<<1, 1, 0, enq_stream>>>( - opType, realSize, broadcastRoot, bufferOffset, taskSequence, - meta.get(), tasks_device_, meta->size, meta->activeRanksDevice, - meta->activeRanksTensor.data_ptr(), taskId); - bufferToTensor( - (void*)meta->segmentInfos[meta->rank].recv_buffer[bufferOffset], - pos, realSize, enq_stream); - - ++cudaTaskCount; - ++meta->taskCount; } - - auto event_end = std::make_shared(torch::kCUDA); - event_end->record(enq_stream); - - if (opType == c10d::OpType::BARRIER) { - return c10::make_intrusive( - opType, event_end, meta, this, std::move(submitted_tasks)); - } - return c10::make_intrusive(opType, event_end, meta, this, - std::move(submitted_tasks)); +} + +} // namespace mooncake + +// ── Kernel launch wrappers ──────────────────────────────────────── +// g++ cannot compile <<<>>> syntax, so these wrappers are compiled by +// the GPU compiler (nvcc/mcc) and provide plain C++ functions that the +// host code can call. + +#ifdef __MUSA__ +#include +#endif + +namespace mooncake { + +void launchEnqueueTaskKernel(int opType, size_t tensorSize, + int64_t broadcastRoot, int bufferOffset, + uint64_t submitSequence, void* meta, Task* tasks, + int numRanks, const bool* activeRanks, + int* activeRanksTensor, size_t taskId, + cudaStream_t stream) { + enqueueTaskKernel<<<1, 1, 0, stream>>>( + opType, tensorSize, broadcastRoot, bufferOffset, submitSequence, meta, + tasks, numRanks, activeRanks, activeRanksTensor, taskId); +} + +#define DEF_LAUNCH_REDUCE(scalar_t, suffix) \ + void launchReduceKernel_##suffix( \ + scalar_t* dst, const scalar_t* src, size_t numElements, \ + size_t numRanks, int op, bool* activeRanks, cudaStream_t stream) { \ + reduceKernel<<<64, 256, 0, stream>>>(dst, src, numElements, numRanks, \ + op, activeRanks); \ + } + +DEF_LAUNCH_REDUCE(uint8_t, uint8) +DEF_LAUNCH_REDUCE(int8_t, int8) +DEF_LAUNCH_REDUCE(int16_t, int16) +DEF_LAUNCH_REDUCE(int, int32) +DEF_LAUNCH_REDUCE(int64_t, int64) +DEF_LAUNCH_REDUCE(float, float) +DEF_LAUNCH_REDUCE(double, double) +DEF_LAUNCH_REDUCE(bool, bool) + +#undef DEF_LAUNCH_REDUCE + +void launchReduceKernel_bf16(void* dst, const void* src, size_t numElements, + size_t numRanks, int op, bool* activeRanks, + cudaStream_t stream) { +#ifdef __MUSA__ + reduceKernel<<<64, 256, 0, stream>>>((mt_bfloat16*)dst, + (const mt_bfloat16*)src, numElements, + numRanks, op, activeRanks); +#else + reduceKernel<<<64, 256, 0, stream>>>((at::BFloat16*)dst, + (const at::BFloat16*)src, numElements, + numRanks, op, activeRanks); +#endif +} + +void preloadReduceKernels() { +#ifdef __MUSA__ + // MUSA: mcc has no cudaFuncGetAttributes, kernels are JIT-compiled on + // first use. +#else + // CUDA: preload kernels to avoid JIT compilation overhead on first use. + auto preload = [](const char* name, auto kernel_ptr) { + cudaFuncAttributes attr{}; + auto err = cudaFuncGetAttributes(&attr, kernel_ptr); + TORCH_CHECK(err == cudaSuccess, "Failed to preload kernel ", name, ": ", + cudaGetErrorString(err)); + }; + preload("reduceKernel", + reinterpret_cast(reduceKernel)); + preload("reduceKernel", + reinterpret_cast(reduceKernel)); + preload("reduceKernel", + reinterpret_cast(reduceKernel)); + preload("reduceKernel", + reinterpret_cast(reduceKernel)); + preload("reduceKernel", + reinterpret_cast(reduceKernel)); + preload("reduceKernel", + reinterpret_cast(reduceKernel)); + preload("reduceKernel", + reinterpret_cast(reduceKernel)); + preload("reduceKernel", + reinterpret_cast(reduceKernel)); + preload("reduceKernel", + reinterpret_cast(reduceKernel)); +#endif } } // namespace mooncake diff --git a/mooncake-pg/src/mooncake_worker.mu b/mooncake-pg/src/mooncake_worker.mu new file mode 120000 index 00000000..73bffcdd --- /dev/null +++ b/mooncake-pg/src/mooncake_worker.mu @@ -0,0 +1 @@ +mooncake_worker.cu \ No newline at end of file diff --git a/mooncake-pg/src/mooncake_worker_host.cpp b/mooncake-pg/src/mooncake_worker_host.cpp new file mode 100644 index 00000000..96173bfa --- /dev/null +++ b/mooncake-pg/src/mooncake_worker_host.cpp @@ -0,0 +1,469 @@ +// mooncake_worker_host.cpp — Host-side code for PG collectives. +// Compiled by g++ for both CUDA and MUSA builds. Uses kernel launch wrappers +// from mooncake_worker_kernels.cuh instead of <<<>>> syntax. + +#include +#include +#include +#include +#include +#include +#ifdef MOONCAKE_EP_USE_MUSA +#include +#else +#include +#endif + +#include "pg_utils.h" + +#ifdef MOONCAKE_EP_USE_MUSA +namespace gpu_capture = at::musa; +namespace gpu_c10 = c10::musa; +#else +namespace gpu_capture = at::cuda; +namespace gpu_c10 = c10::cuda; +#endif + +namespace mooncake { + +class MooncakeWorkCpu : public ::c10d::Work { + public: + MooncakeWorkCpu(c10d::OpType opType, + c10::intrusive_ptr future, + std::shared_ptr meta) + : Work(-1, opType), + future_(std::move(future)), + meta_(std::move(meta)) {} + + bool isCompleted() override { return future_->completed(); } + + bool wait(std::chrono::milliseconds timeout) override { + future_->wait(); + return future_->completed() && !future_->hasError(); + } + + private: + c10::intrusive_ptr future_; + std::shared_ptr meta_; +}; + +class MooncakeWorkCuda : public ::c10d::Work { + public: + MooncakeWorkCuda(c10d::OpType opType, std::shared_ptr event, + std::shared_ptr meta, + const MooncakeWorker* worker, + std::vector submitted_tasks) + : Work(-1, opType), + event_(std::move(event)), + meta_(std::move(meta)), + worker_(worker), + submitted_tasks_(std::move(submitted_tasks)) {} + + bool isCompleted() override { return event_->query(); } + + bool wait(std::chrono::milliseconds timeout) override { + // Wait until the task has been submitted to TransferEngine: + // This tries to ensure that the CUDA kernels required for the transfer + // have been launched by the time `waitUntilTasksSubmitted` returns. + // + // Why is this needed? PyTorch documentation implies that collective + // operations should be enqueued when `wait()` returns. In practice, we + // found that violating this causes hangs. + // + // Our current hypothesis for the hang is: PyTorch assumes the kernels + // needed for the transfer are already launched when `wait` returns + // true. It may then launch subsequent operations after the collective + // (e.g., `.cpu()`). Such operations may acquire a process-wide lock in + // the CUDA runtime. Also, they may rely on the data produced by the + // collective, thus causing a synchronization on enq_stream. However, + // holding that runtime lock prevents cudaMemcpy(Async) in TE/TENT from + // launching. This means the transfer can't finish, and enq_stream won't + // complete. Thus, a deadlock occurs. + // (In practice, we found that replacing all cudaMemcpyAsync in TENT + // with cuMemcpyAsync actually alleviates this, which further suggests a + // deadlock in the CUDA runtime. However, that change is too invasive + // for TE/TENT, so we do not adopt it here.) + // + // Strictly speaking, the wait is needed for another reason: The current + // stream will be blocked on the event below. Any subsequent work on + // `current_stream` will wait on that event, which effectively waits for + // the task to be done. Therefore, we must ensure all kernels needed for + // the transfer task are launched BEFORE blocking the current stream, in + // case TE/TENT use `current_stream` to launch those kernels (though it + // is rare). + // + // Please note that this logic relies on the assumption that TE/TENT + // will launch all CUDA operations in `submitTransfer`. + // Unfortunately, TcpTransport in TE and TENT currently violates this + // assumption (cudaMemcpy(Async) may be called later from a callback), + // which can cause hangs in PG when a CUDA operation such as + // `x.cpu().item()` follows the collective. For TE's TcpTransport, the + // use of cudaMemcpy on the default stream may also contribute to the + // hang. + // + // Besides, for CPU-only transports (like RdmaTransport), + // waitUntilTasksSubmitted is totally unnecessary, but we keep it for + // uniform behavior to avoid invasive changes to TE/TENT. + bool submitted = true; + if (gpu_capture::currentStreamCaptureStatus() == + gpu_c10::CaptureStatus::None) { + // Normal execution: block until tasks are submitted. + submitted = + worker_->waitUntilTasksSubmitted(submitted_tasks_, timeout); + } else { + // During CUDA graph capture, kernels are recorded but not actually + // executed. The enqueueTaskKernel would never run, so + // waitUntilTasksSubmitted would hang because the CPU worker thread + // never sees task.active == true. + // + // Note that this also means NvlinkTransport (and TcpTransport too, + // of course) won't work with CUDA Graphs: Kernels launched inside + // TE/TENT can't be captured by the graph, and during replay they + // are not ordered with the graph execution. This may trigger the + // same deadlock described above. + } + if (!submitted) return false; + + // Once all tasks have been submitted, use the event to synchronize + // the current stream and the enqueue stream, but do not wait on this + // event. + // + // See PyTorch docs for more details: + // https://docs.pytorch.org/docs/stable/distributed.html#synchronous-and-asynchronous-collective-operations + // "wait() - in the case of CPU collectives, will block the process + // until the operation is completed. In the case of CUDA collectives, + // will block the currently active CUDA stream until the operation + // is completed (but will not block the CPU)." + auto current_stream = getCurrentGPUStream(); + event_->block(current_stream); + return true; + } + + protected: + std::shared_ptr event_; + std::shared_ptr meta_; + const MooncakeWorker* worker_; + std::vector submitted_tasks_; +}; + +class MooncakeBarrierWorkCuda : public MooncakeWorkCuda { + public: + using MooncakeWorkCuda::MooncakeWorkCuda; + + bool wait(std::chrono::milliseconds timeout) override { + // Skip host-side synchronization during CUDA graph capture. + // cudaEventSynchronize is not permitted while a stream is capturing. + if (gpu_capture::currentStreamCaptureStatus() != + gpu_c10::CaptureStatus::None) { + // We still need stream-level synchronization so that subsequent + // operations on the capture stream are ordered after the barrier + // task on the enqueue stream. + auto current_stream = getCurrentGPUStream(); + event_->block(current_stream); + return true; + } + + if (timeout == kNoTimeout) { + event_->synchronize(); + return true; + } + + BackoffWaiter waiter( + BackoffWaiterConfig::constantSleep(std::chrono::microseconds(10))); + return waiter.wait_for(timeout, [this] { return event_->query(); }); + } +}; + +void launchReduceKernel(at::Tensor dst, size_t pos, size_t realSize, void* src, + size_t numRanks, c10d::ReduceOp op, bool* activeRanks, + cudaStream_t stream) { + TORCH_CHECK(op == c10d::ReduceOp::SUM || op == c10d::ReduceOp::MIN || + op == c10d::ReduceOp::MAX || op == c10d::ReduceOp::PRODUCT, + "Only support SUM/MIN/MAX/PRODUCT for reduction."); + auto ptr = (char*)dst.data_ptr() + pos; + size_t num = realSize / dst.element_size(); + + switch (dst.scalar_type()) { + case c10::kByte: + launchReduceKernel_uint8((uint8_t*)ptr, (uint8_t*)src, num, + numRanks, (int)op, activeRanks, stream); + break; + case c10::kChar: + launchReduceKernel_int8((int8_t*)ptr, (int8_t*)src, num, numRanks, + (int)op, activeRanks, stream); + break; + case c10::kShort: + launchReduceKernel_int16((int16_t*)ptr, (int16_t*)src, num, + numRanks, (int)op, activeRanks, stream); + break; + case c10::kInt: + launchReduceKernel_int32((int*)ptr, (int*)src, num, numRanks, + (int)op, activeRanks, stream); + break; + case c10::kLong: + launchReduceKernel_int64((int64_t*)ptr, (int64_t*)src, num, + numRanks, (int)op, activeRanks, stream); + break; + case c10::kFloat: + launchReduceKernel_float((float*)ptr, (float*)src, num, numRanks, + (int)op, activeRanks, stream); + break; + case c10::kDouble: + launchReduceKernel_double((double*)ptr, (double*)src, num, numRanks, + (int)op, activeRanks, stream); + break; + case c10::kBool: + launchReduceKernel_bool((bool*)ptr, (bool*)src, num, numRanks, + (int)op, activeRanks, stream); + break; + case c10::kBFloat16: + launchReduceKernel_bf16(ptr, src, num, numRanks, (int)op, + activeRanks, stream); + break; + default: + TORCH_CHECK(false, c10::str("Unsupported reduce dtype: ", + dst.scalar_type())); + } +} + +template +T applyReduceOp(const T& a, const T& b, c10d::ReduceOp op) { + switch (op) { + case c10d::ReduceOp::SUM: + return a + b; + case c10d::ReduceOp::PRODUCT: + return a * b; + case c10d::ReduceOp::MIN: + return std::min(a, b); + case c10d::ReduceOp::MAX: + return std::max(a, b); + default: + TORCH_CHECK(false, c10::str("Unsupported reduce op: ", op)); + } +} + +template +void reduceCpu(T* dst, const T* src, size_t numElements, size_t numRanks, + c10d::ReduceOp op, bool* activeRanks) { + at::parallel_for(0, numElements, 1024, [&](int64_t begin, int64_t end) { + for (int64_t i = begin; i < end; ++i) { + bool valid = false; + T acc{}; + for (int64_t rank = 0; rank < numRanks; ++rank) { + if (activeRanks[rank]) { + if (!valid) { + acc = src[i + rank * numElements]; + valid = true; + } else { + acc = + applyReduceOp(acc, src[i + rank * numElements], op); + } + } + } + dst[i] = acc; + } + }); +} + +void launchReduceCpu(at::Tensor dst, size_t pos, size_t realSize, void* src, + size_t numRanks, c10d::ReduceOp op, bool* activeRanks) { + auto ptr = (char*)dst.data_ptr() + pos; + size_t num = realSize / dst.element_size(); + + switch (dst.scalar_type()) { + case c10::kByte: + reduceCpu((uint8_t*)ptr, (uint8_t*)src, num, numRanks, op, + activeRanks); + break; + case c10::kChar: + reduceCpu((int8_t*)ptr, (int8_t*)src, num, numRanks, op, + activeRanks); + break; + case c10::kShort: + reduceCpu((int16_t*)ptr, (int16_t*)src, num, numRanks, op, + activeRanks); + break; + case c10::kInt: + reduceCpu((int*)ptr, (int*)src, num, numRanks, op, activeRanks); + break; + case c10::kLong: + reduceCpu((int64_t*)ptr, (int64_t*)src, num, numRanks, op, + activeRanks); + break; + case c10::kFloat: + reduceCpu((float*)ptr, (float*)src, num, numRanks, op, activeRanks); + break; + case c10::kDouble: + reduceCpu((double*)ptr, (double*)src, num, numRanks, op, + activeRanks); + break; + case c10::kBool: + reduceCpu((bool*)ptr, (bool*)src, num, numRanks, op, activeRanks); + break; + default: + TORCH_CHECK(false, c10::str("Unsupported reduce dtype: ", + dst.scalar_type())); + } +} + +MooncakeWorker::MooncakeWorker(int cuda_device_index) + : cuda_device_index_(cuda_device_index) { + int deviceCount = 0; + cudaError err = cudaGetDeviceCount(&deviceCount); + if (!err && deviceCount > 0) { + cudaHostAlloc(&tasks_, kNumTasks_ * sizeof(Task), cudaHostAllocMapped); + cudaHostGetDevicePointer(&tasks_device_, tasks_, 0); + } else { + LOG(WARNING) << "No GPU device found. Only the `mooncake-cpu` backend " + "can be used."; + tasks_ = new Task[kNumTasks_]; + } + for (size_t i = 0; i < kNumTasks_; ++i) { + tasks_[i].active = false; + tasks_[i].submitSequence = 0; + submitted_task_sequence_[i].store(0, std::memory_order_relaxed); + } +} + +MooncakeWorker::~MooncakeWorker() { + running_ = false; + if (worker_thread_.joinable()) { + worker_thread_.join(); + } +} + +c10::intrusive_ptr MooncakeWorker::putTaskCpu( + c10d::OpType opType, size_t tensorSize, int64_t broadcastRoot, + const std::shared_ptr& meta, + const std::shared_ptr& connection_ctx, + const std::function& + tensorToBuffer, + const std::function& + bufferToTensor) { + connection_ctx->waitUntilNewRanksConnected(); + + size_t chunkSize = ((kBufferSize - 1) / meta->size) & ~(size_t)7; + auto future = c10::make_intrusive( + c10::ListType::create(c10::TensorType::get())); + + struct IterState { + size_t currentPos = 0; + }; + auto state = std::make_shared(); + + auto processNextChunk = std::make_shared>(); + std::weak_ptr> weakProcessNextChunk = + processNextChunk; + + *processNextChunk = [this, weakProcessNextChunk, state, opType, tensorSize, + chunkSize, broadcastRoot, meta, tensorToBuffer, + bufferToTensor, future]() { + auto processNextChunk = weakProcessNextChunk.lock(); + + if (state->currentPos >= tensorSize) { + future->markCompleted(c10::IValue()); + return; + } + + int taskId = cpuTaskCount % 2; + TORCH_CHECK(!tasks_[taskId].active); + + size_t realSize = std::min(chunkSize, tensorSize - state->currentPos); + int bufferOffset = meta->taskCount % 2; + + tasks_[taskId].opType = (int)opType; + tasks_[taskId].tensorSize = realSize; + tasks_[taskId].broadcastRoot = broadcastRoot; + tasks_[taskId].bufferOffset = bufferOffset; + tasks_[taskId].transferGroupMeta = meta.get(); + tensorToBuffer( + (void*)meta->segmentInfos[meta->rank].send_buffer[bufferOffset], + state->currentPos, realSize); + + hasCallback_[taskId] = true; + + callbacks_[taskId] = [this, processNextChunk, state, meta, + bufferToTensor, bufferOffset, realSize, + future]() { + for (int i = 0; i < meta->size; ++i) { + meta->activeRanksTensor[i] = meta->activeRanks[i] ? 1 : 0; + } + bufferToTensor( + (void*)meta->segmentInfos[meta->rank].recv_buffer[bufferOffset], + state->currentPos, realSize); + + state->currentPos += realSize; + + (*processNextChunk)(); + }; + + tasks_[taskId].active = true; + ++cpuTaskCount; + ++meta->taskCount; + }; + + (*processNextChunk)(); + + return c10::make_intrusive(opType, future, meta); +} + +c10::intrusive_ptr MooncakeWorker::putTaskCuda( + c10d::OpType opType, size_t tensorSize, int64_t broadcastRoot, + const std::shared_ptr& meta, + const std::shared_ptr& connection_ctx, + const GPUStream& issue_stream, + const std::function& tensorToBuffer, + const std::function& bufferToTensor) { + connection_ctx->waitUntilNewRanksConnected(); + + size_t chunkSize = ((kBufferSize - 1) / meta->size) & ~(size_t)7; + + GPUStream enq_stream = + getGPUStreamFromPool(false, issue_stream.device_index()); + + auto event_start = std::make_shared(kGPUDevice); + event_start->record(issue_stream); + event_start->block(enq_stream); + + std::vector submitted_tasks; + submitted_tasks.reserve((tensorSize + chunkSize - 1) / chunkSize); + for (size_t pos = 0; pos < tensorSize; pos += chunkSize) { + size_t realSize = std::min(tensorSize, pos + chunkSize) - pos; + int taskId = cudaTaskCount % 2 + 2; + int bufferOffset = meta->taskCount % 2; + const uint64_t taskSequence = + next_cuda_task_sequence_.fetch_add(1, std::memory_order_relaxed); + submitted_tasks.push_back( + {.task_id = static_cast(taskId), .sequence = taskSequence}); + tensorToBuffer( + (void*)meta->segmentInfos[meta->rank].send_buffer[bufferOffset], + pos, realSize, enq_stream); + + hasCallback_[taskId] = false; + launchEnqueueTaskKernel( + (int)opType, realSize, broadcastRoot, bufferOffset, taskSequence, + meta.get(), tasks_device_, meta->size, meta->activeRanksDevice, + meta->activeRanksTensor.data_ptr(), taskId, + enq_stream.stream()); + bufferToTensor( + (void*)meta->segmentInfos[meta->rank].recv_buffer[bufferOffset], + pos, realSize, enq_stream); + + ++cudaTaskCount; + ++meta->taskCount; + } + + auto event_end = std::make_shared(kGPUDevice); + event_end->record(enq_stream); + + if (opType == c10d::OpType::BARRIER) { + return c10::make_intrusive( + opType, event_end, meta, this, std::move(submitted_tasks)); + } + return c10::make_intrusive(opType, event_end, meta, this, + std::move(submitted_tasks)); +} + +} // namespace mooncake diff --git a/mooncake-pg/src/mooncake_worker_thread.cpp b/mooncake-pg/src/mooncake_worker_thread.cpp index aeaba687..488a1909 100644 --- a/mooncake-pg/src/mooncake_worker_thread.cpp +++ b/mooncake-pg/src/mooncake_worker_thread.cpp @@ -1,4 +1,4 @@ -#include +#include #include #include #include @@ -85,11 +85,12 @@ void MooncakeWorker::startWorker() { } auto group = (TransferGroupMeta*)task.transferGroupMeta; - bool skipTransfer = (task.opType == c10d::OpType::BROADCAST && - group->rank != task.broadcastRoot) || - (task.opType == c10d::OpType::SCATTER && - group->rank != task.broadcastRoot) || - task.opType == c10d::OpType::BARRIER; + bool skipTransfer = + ((c10d::OpType)task.opType == c10d::OpType::BROADCAST && + group->rank != task.broadcastRoot) || + ((c10d::OpType)task.opType == c10d::OpType::SCATTER && + group->rank != task.broadcastRoot) || + (c10d::OpType)task.opType == c10d::OpType::BARRIER; if (task_status[i].load(std::memory_order_acquire) == IDLE) { const auto submit_sequence = task.submitSequence; if (skipTransfer) { @@ -107,15 +108,17 @@ void MooncakeWorker::startWorker() { if (!group->activeRanks[j]) { continue; } - if ((task.opType == c10d::OpType::GATHER || - task.opType == c10d::OpType::REDUCE) && + if (((c10d::OpType)task.opType == + c10d::OpType::GATHER || + (c10d::OpType)task.opType == + c10d::OpType::REDUCE) && j != task.broadcastRoot) { continue; } uint64_t source = group->segmentInfos[group->rank] .send_buffer[task.bufferOffset]; - switch (task.opType) { + switch ((c10d::OpType)task.opType) { case c10d::OpType::BROADCAST: case c10d::OpType::ALLREDUCE: case c10d::OpType::ALLGATHER: @@ -136,7 +139,7 @@ void MooncakeWorker::startWorker() { group->segmentInfos[j] .recv_buffer[task.bufferOffset]; - switch (task.opType) { + switch ((c10d::OpType)task.opType) { case c10d::OpType::BROADCAST: case c10d::OpType::SCATTER: break; diff --git a/mooncake-pg/src/p2p_proxy.cpp b/mooncake-pg/src/p2p_proxy.cpp index 26e32af7..2db2fb42 100644 --- a/mooncake-pg/src/p2p_proxy.cpp +++ b/mooncake-pg/src/p2p_proxy.cpp @@ -1,8 +1,7 @@ #include #include #include -#include -#include +#include #include #include #include diff --git a/mooncake-pg/src/pg_py.cpp b/mooncake-pg/src/pg_py.cpp index b9e379a7..e39daffe 100644 --- a/mooncake-pg/src/pg_py.cpp +++ b/mooncake-pg/src/pg_py.cpp @@ -33,10 +33,17 @@ __attribute__((constructor)) static void MooncakeBackendConstructor() { kwargsCpu["devices"] = py::make_tuple("cpu"); register_backend("mooncake-cpu", py::cpp_function(createMooncakeCpuBackend), /* extended_api */ true, **kwargsCpu); +#ifndef MOONCAKE_EP_USE_MUSA py::dict kwargsCuda; kwargsCuda["devices"] = py::make_tuple("cuda"); register_backend("mooncake", py::cpp_function(createMooncakeBackend), /* extended_api */ true, **kwargsCuda); +#else + py::dict kwargsMusa; + kwargsMusa["devices"] = py::make_tuple("musa"); + register_backend("mooncake", py::cpp_function(createMooncakeBackend), + /* extended_api */ true, **kwargsMusa); +#endif } std::string getPreferredHca(c10::intrusive_ptr backend, diff --git a/mooncake-pg/tests/pg_test_utils.py b/mooncake-pg/tests/pg_test_utils.py index a5a75e37..87950c80 100644 --- a/mooncake-pg/tests/pg_test_utils.py +++ b/mooncake-pg/tests/pg_test_utils.py @@ -73,6 +73,12 @@ def temporary_env(updates: dict[str, str]): os.environ[key] = value +def musa_runtime_available(min_devices: int = 1) -> bool: + if not hasattr(torch, "musa") or not torch.musa.is_available(): + return False + return torch.musa.device_count() >= min_devices + + def cuda_runtime_available(min_devices: int = 1) -> bool: if not torch.cuda.is_available(): return False @@ -80,6 +86,18 @@ def cuda_runtime_available(min_devices: int = 1) -> bool: def require_test_device(rank: int, device_type: str) -> torch.device: + if device_type == "musa": + device_count = torch.musa.device_count() + if device_count <= 0: + raise RuntimeError( + "MUSA backend requested but no MUSA devices are available" + ) + if rank >= device_count: + raise RuntimeError( + f"rank {rank} requires a dedicated MUSA device but only {device_count} are visible" + ) + torch.musa.set_device(rank) + return torch.device("musa", rank) if device_type == "cuda": device_count = torch.cuda.device_count() if device_count <= 0: @@ -508,9 +526,16 @@ class BackendMultiProcessTestCase(MultiProcessTestCase): raise RuntimeError( f"{cls.__name__} must inherit a concrete Mooncake PG backend test base class" ) + if cls.device_type == "musa": + device_count = torch.musa.device_count() if hasattr(torch, "musa") and torch.musa.is_available() else 0 + cls.configure_for_cuda_device_count(device_count) if cls.device_type == "cuda": device_count = torch.cuda.device_count() if torch.cuda.is_available() else 0 cls.configure_for_cuda_device_count(device_count) + if cls.device_type == "musa" and not musa_runtime_available(cls.world_size): + raise unittest.SkipTest( + f"{cls.__name__} requires {cls.world_size} visible MUSA devices" + ) if cls.device_type == "cuda" and not cuda_runtime_available(cls.world_size): raise unittest.SkipTest( f"{cls.__name__} requires {cls.world_size} visible CUDA devices" @@ -582,3 +607,8 @@ class MooncakePGCPUBackendTestCase(BackendMultiProcessTestCase): class MooncakePGCUDABackendTestCase(BackendMultiProcessTestCase): backend_name = "mooncake" device_type = "cuda" + + +class MooncakePGMUSABackendTestCase(BackendMultiProcessTestCase): + backend_name = "mooncake" + device_type = "musa" diff --git a/mooncake-pg/tests/test_pg_collectives.py b/mooncake-pg/tests/test_pg_collectives.py index 0d07af00..52fbee7a 100644 --- a/mooncake-pg/tests/test_pg_collectives.py +++ b/mooncake-pg/tests/test_pg_collectives.py @@ -6,6 +6,7 @@ import torch.distributed as dist from pg_test_utils import ( MooncakePGCPUBackendTestCase, MooncakePGCUDABackendTestCase, + MooncakePGMUSABackendTestCase, MooncakePGWorkerContext, wait_until, ) @@ -226,5 +227,9 @@ class TestMooncakePGCollectivesCUDA(_CollectiveTestMixin, MooncakePGCUDABackendT world_size = 2 +class TestMooncakePGCollectivesMUSA(_CollectiveTestMixin, MooncakePGMUSABackendTestCase): + world_size = 2 + + if __name__ == "__main__": unittest.main() diff --git a/mooncake-transfer-engine/include/gpu_vendor/musa.h b/mooncake-transfer-engine/include/gpu_vendor/musa.h index 45d4428d..b53d7707 100644 --- a/mooncake-transfer-engine/include/gpu_vendor/musa.h +++ b/mooncake-transfer-engine/include/gpu_vendor/musa.h @@ -76,6 +76,7 @@ const static std::string GPU_PREFIX = "musa:"; #define cudaMemcpyAsync musaMemcpyAsync #define cudaMemcpyDefault musaMemcpyDefault #define cudaMemcpyDeviceToHost musaMemcpyDeviceToHost +#define cudaMemcpyDeviceToDevice musaMemcpyDeviceToDevice #define cudaMemcpyHostToDevice musaMemcpyHostToDevice #define cudaMemset musaMemset #define cudaMemsetAsync musaMemsetAsync @@ -89,4 +90,6 @@ const static std::string GPU_PREFIX = "musa:"; #define cudaStreamDestroy musaStreamDestroy #define cudaStreamSynchronize musaStreamSynchronize #define cudaStream_t musaStream_t +#define cudaEvent_t musaEvent_t +#define cudaHostGetDevicePointer musaHostGetDevicePointer #define cudaSuccess musaSuccess -- 2.34.1 From 40935abd441d2c93742ed44b8bc2529fceb680b7 Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Fri, 5 Jun 2026 14:15:36 +0800 Subject: [PATCH 189/382] [Wheel] Fix _parse_segment_size to support KB/MB/TB suffixes and fix handle_put None crash (#2321) * [Wheel] Fix _parse_segment_size to support KB/MB/TB suffixes and fix handle_put None crash Fixes #2320 1. Fix: _parse_segment_size now supports KB/MB/GB/TB/B/K/M/G/T suffixes with float values (e.g. "1.5gb", "512mb"), matching C++ try_string_to_byte_size. Bare float strings (e.g. "1.5", "1e9") are also handled via int(float(s)) fallback. 2. Fix: handle_put guards against value=None before calling .encode(), preventing AttributeError crash on malformed PUT requests. 3. Tests: 13 new test cases for _parse_segment_size covering all suffixes, floats, empty strings, missing numbers, and invalid input. --------- Co-authored-by: Claude Opus 4.6 --- mooncake-wheel/mooncake/mooncake_config.py | 33 ++++++--- .../mooncake/mooncake_store_service.py | 5 +- mooncake-wheel/tests/test_mooncake_config.py | 71 ++++++++++++++++++- 3 files changed, 98 insertions(+), 11 deletions(-) diff --git a/mooncake-wheel/mooncake/mooncake_config.py b/mooncake-wheel/mooncake/mooncake_config.py index db05a82f..642557cc 100644 --- a/mooncake-wheel/mooncake/mooncake_config.py +++ b/mooncake-wheel/mooncake/mooncake_config.py @@ -53,21 +53,38 @@ from typing import Optional DEFAULT_GLOBAL_SEGMENT_SIZE = 3355443200 # 3.125 GiB DEFAULT_LOCAL_BUFFER_SIZE = 1073741824 # 1.0 GiB +_SIZE_SUFFIXES = [ + ("kb", 1024), + ("mb", 1024 ** 2), + ("gb", 1024 ** 3), + ("tb", 1024 ** 4), + ("k", 1024), + ("m", 1024 ** 2), + ("g", 1024 ** 3), + ("t", 1024 ** 4), + ("b", 1), +] + + def _parse_segment_size(value) -> int: if isinstance(value, int): return value if isinstance(value, str): s = value.strip().lower() - if s.endswith("gb"): - num = s[:-2].strip() - if not num: - raise ValueError( - "Invalid segment size: missing number before 'gb'" - ) - return int(num) * 1024 * 1024 * 1024 - return int(s) + if not s: + raise ValueError("Invalid segment size: empty string") + for suffix, multiplier in _SIZE_SUFFIXES: + if s.endswith(suffix): + num = s[: -len(suffix)].strip() + if not num: + raise ValueError( + f"Invalid segment size: missing number before '{suffix}'" + ) + return int(float(num) * multiplier) + return int(float(s)) return int(value) + @dataclass class MooncakeConfig: """The configuration class for Mooncake. diff --git a/mooncake-wheel/mooncake/mooncake_store_service.py b/mooncake-wheel/mooncake/mooncake_store_service.py index fd4c1699..6ad3cb2b 100644 --- a/mooncake-wheel/mooncake/mooncake_store_service.py +++ b/mooncake-wheel/mooncake/mooncake_store_service.py @@ -461,15 +461,16 @@ class MooncakeStoreService: try: data = await request.json() key = data.get('key') - value = data.get('value').encode() + raw_value = data.get('value') - if not key or not value: + if not key or raw_value is None: return web.Response( status=400, text=json.dumps({'error': 'Missing key or value'}), content_type='application/json' ) + value = raw_value.encode() ret = self.store.put(key, value) if ret != 0: return web.Response( diff --git a/mooncake-wheel/tests/test_mooncake_config.py b/mooncake-wheel/tests/test_mooncake_config.py index 403bc326..39b57bea 100644 --- a/mooncake-wheel/tests/test_mooncake_config.py +++ b/mooncake-wheel/tests/test_mooncake_config.py @@ -3,7 +3,13 @@ import os import tempfile import unittest -from mooncake.mooncake_config import MooncakeConfig, DEFAULT_GLOBAL_SEGMENT_SIZE, DEFAULT_LOCAL_BUFFER_SIZE +from mooncake.mooncake_config import ( + MooncakeConfig, + DEFAULT_GLOBAL_SEGMENT_SIZE, + DEFAULT_LOCAL_BUFFER_SIZE, + _parse_segment_size, +) + class TestMooncakeConfig(unittest.TestCase): def setUp(self): @@ -130,5 +136,68 @@ class TestMooncakeConfig(unittest.TestCase): MooncakeConfig.load_from_env() self.assertIn("Neither the environment variable 'MOONCAKE_CONFIG_PATH' nor 'MOONCAKE_MASTER' is set.", str(cm.exception)) + +class TestParseSegmentSize(unittest.TestCase): + def test_integer_passthrough(self): + self.assertEqual(_parse_segment_size(1024), 1024) + self.assertEqual(_parse_segment_size(0), 0) + + def test_float_passthrough(self): + self.assertEqual(_parse_segment_size(1.5), 1) + + def test_bytes_string(self): + self.assertEqual(_parse_segment_size("1024"), 1024) + self.assertEqual(_parse_segment_size(" 2048 "), 2048) + + def test_kb_suffix(self): + self.assertEqual(_parse_segment_size("1kb"), 1024) + self.assertEqual(_parse_segment_size("1KB"), 1024) + self.assertEqual(_parse_segment_size("512k"), 512 * 1024) + self.assertEqual(_parse_segment_size("1.5kb"), int(1.5 * 1024)) + + def test_mb_suffix(self): + self.assertEqual(_parse_segment_size("1mb"), 1024 ** 2) + self.assertEqual(_parse_segment_size("512MB"), 512 * 1024 ** 2) + self.assertEqual(_parse_segment_size("1m"), 1024 ** 2) + + def test_gb_suffix(self): + self.assertEqual(_parse_segment_size("1gb"), 1024 ** 3) + self.assertEqual(_parse_segment_size("3GB"), 3 * 1024 ** 3) + self.assertEqual(_parse_segment_size("1g"), 1024 ** 3) + self.assertEqual(_parse_segment_size("1.5gb"), int(1.5 * 1024 ** 3)) + + def test_tb_suffix(self): + self.assertEqual(_parse_segment_size("1tb"), 1024 ** 4) + self.assertEqual(_parse_segment_size("1TB"), 1024 ** 4) + self.assertEqual(_parse_segment_size("1t"), 1024 ** 4) + + def test_b_suffix(self): + self.assertEqual(_parse_segment_size("4096b"), 4096) + self.assertEqual(_parse_segment_size("4096B"), 4096) + + def test_empty_string_raises(self): + with self.assertRaises(ValueError): + _parse_segment_size("") + with self.assertRaises(ValueError): + _parse_segment_size(" ") + + def test_missing_number_raises(self): + with self.assertRaises(ValueError): + _parse_segment_size("gb") + with self.assertRaises(ValueError): + _parse_segment_size("mb") + + def test_bare_float_string(self): + self.assertEqual(_parse_segment_size("1.5"), 1) + self.assertEqual(_parse_segment_size("1e9"), 1000000000) + + def test_invalid_string_raises(self): + with self.assertRaises(ValueError): + _parse_segment_size("abc") + + def test_whitespace_handling(self): + self.assertEqual(_parse_segment_size(" 3 gb "), 3 * 1024 ** 3) + + if __name__ == '__main__': unittest.main() -- 2.34.1 From d36a72f0e18ad6eff26dfad25d518eda5f5fb747 Mon Sep 17 00:00:00 2001 From: Xun Sun Date: Fri, 5 Jun 2026 16:00:07 +0800 Subject: [PATCH 190/382] [TE] add device API support (#2333) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(transfer-engine): extract Device API and IBGDA transport layer Move IBGDA files from mooncake-ep to mooncake-transfer-engine (git mv): - 6 headers: mooncake_ibgda/ → transport/device/ibgda/ - 1 source: mlx5gda.cpp → transport/device/mlx5gda.cpp Add new Device API layer under transport/device/: - device_transport.h: P2pTransport + RdmaTransport interfaces - device_ops.cuh: DeviceOps function pointer table (bottom IR) - comm_device.cuh, p2p_device.cuh, ibgda_device.cuh: device contexts - cuda_ops.cuh, musa_ops.cuh: platform DeviceOps implementations - ibgda_device_transport.cpp: RdmaTransport IBGDA implementation - p2p_device_transport.cpp: P2pTransport NVLink/MTLink implementation Modify transfer-engine: - transfer_engine.h/impl: add getOrCreateP2pTransport/RdmaTransport - gpu_vendor/musa.h: add MUSA API aliases - Build: link mlx5, add device/ subdirectory Minimal EP changes (include paths + build system only): - Update include paths to new transport/device/ibgda/ location - Remove mlx5gda.cpp from EP sources - Forward EP_USE_MUSA env var in BuildEpExt.cmake Root CMakeLists.txt: guard CUDAToolkit with USE_CUDA * feat(example): add device_transport_example for P2P Device API Two-rank example demonstrating the full Device API lifecycle: - Host side: P2pTransport for IPC handle exchange and peer mapping - Device side: CommCtx + mc_route_put + mc_signal for GPU-initiated P2P data transfer and notification Uses file-based IPC handle exchange (no external dependencies). Requires 2 GPUs with P2P access (NVLink or PCIe). * fix(example): handle missing TORCH_CUDA_ARCH_LIST * fix(example): convert torch arch format to CMake CUDA format * fix(example): construct CommCtx manually on host side * fix(example): add using namespace mooncake::device in kernels * fix(example): pass CommCtx by value, add barrier before cleanup - Pass CommCtx by value to kernel (CUDA copies to param space) instead of dereferencing a device pointer on host (caused segfault). - Add file-based barrier so rank 0 waits for rank 1 to finish before freeing its GDR buffer (IPC handle only valid while allocation exists). - Change default metadata_server to P2PHANDSHAKE (no etcd dependency). * fix: address review feedback — fence ordering, error checks, QP leak - musa_ops.cuh: fix fence ordering for acquire/release semantics (fence after load for acquire, before store/atomic for release) - ibgda_device_transport.cpp: check cudaMalloc return values, add num_qps >= num_ranks guard, destroy QP on rst2init failure - p2p_device_transport.cpp: add device_count > 0 guard - device_transport_example.cu: validate kDataBytes % 16 == 0 and kDataBytes <= kSignalWordOffset * style: apply clang-format to Device API files * fix: guard device transport code with USE_CUDA/USE_MUSA macros The device transport accessors (getOrCreateP2pTransport, getOrCreateRdmaTransport) and their member variables were not guarded by USE_CUDA/USE_MUSA preprocessor macros. When building with USE_CUDA=OFF (the default), the device transport source files aren't compiled but the headers and implementations still reference them, causing linker errors in CI build-flags and build jobs. * fix: set CMAKE_CUDA_STANDARD 20 and make ibgda PUBLIC - common.cmake: add CMAKE_CUDA_STANDARD 20 so nvcc compiles host code in C++20 mode, matching CMAKE_CXX_STANDARD. Fixes "starts_with is not a member of std::string" when compiling .cu files that indirectly include common.h. - transport/CMakeLists.txt: change ibgda from PRIVATE to PUBLIC so mlx5gda_* symbols are visible to downstream consumers (Go p2p store via transfer_engine). Fixes undefined reference errors for mlx5gda_destroy_qp, mlx5dv_devx_umem_reg, etc. * fix: compile mlx5gda into device_transport and link mlx5 for Go consumers The previous attempt (PUBLIC ibgda) did not work because the Go p2p-store and mooncake-store binaries link libtransfer_engine.a via hand-written cgo ldflags, which bypass CMake's target_link_libraries propagation entirely. - transport/device: compile mlx5gda.cpp directly into the device_transport OBJECT library (like every other transport module) instead of a separate ibgda STATIC lib, so mlx5gda_* symbols flow into libtransfer_engine.a and are visible to all consumers regardless of how they link. - transport: link libmlx5 (PUBLIC) since ibgda_device_transport.cpp / mlx5gda.cpp call mlx5dv_devx_* / mlx5dv_init_obj directly. - p2p-store/build.sh, mooncake-store/go/build.sh, ci.yml: add -lmlx5 to the hand-written cgo ldflags so the DevX symbols resolve. - example: set CUDA_STANDARD 20 on device_transport_example so nvcc compiles common.h (std::string::starts_with) in C++20 mode. * fix: CUDA_EXTENSIONS OFF for example, add -lm for Go consumers Follow-up to compiling mlx5gda.cpp into device_transport: - example: nvcc has no gnu++20 dialect, so CUDA_STANDARD 20 with the default CUDA_EXTENSIONS=ON fails at CMake generate ("does not know the compile flags"). Set CUDA_EXTENSIONS OFF to request plain -std=c++20. - p2p-store/build.sh, mooncake-store/go/build.sh: mlx5gda.cpp uses log2/ceil (); now that its object lives in libtransfer_engine.a, the hand- written cgo ldflags need -lm to resolve log2@GLIBC_2.29. (ci.yml already had -lm.) * fix(ci): gate Device API GPU example off by default, link mlx5 for Rust The Docker build failed at CMake generate because device_transport_example needs the CUDA20 dialect (transfer_engine.h -> common.h uses C++20 std::string::starts_with), which the older CMake in the CI image cannot enable. CUDA_EXTENSIONS OFF did not help since the limitation is the CMake version, not the dialect flavor. Gate this manual, 2-GPU example behind a new BUILD_DEVICE_TRANSPORT_EXAMPLE option (default OFF) so the default build no longer requires CUDA20. Also link mlx5 in the mooncake-store Rust build script: the IBGDA device transport (mlx5 DevX) is now compiled into transfer_engine, so the Rust test link step needs -lmlx5 to resolve mlx5dv_devx_* symbols. --- .github/workflows/ci.yml | 2 +- CMakeLists.txt | 1 + mooncake-common/common.cmake | 1 + mooncake-ep/BuildEpExt.cmake | 6 + mooncake-ep/include/mooncake_ep_buffer.h | 4 +- mooncake-ep/setup.py | 1 - mooncake-ep/src/CMakeLists.txt | 2 +- mooncake-ep/src/mooncake_ep_kernel.cu | 2 +- mooncake-p2p-store/build.sh | 2 +- mooncake-store/go/build.sh | 2 +- mooncake-store/rust/build.rs | 1 + .../example/CMakeLists.txt | 45 ++ .../example/device_transport_example.cu | 305 +++++++++++++ .../include/CMakeLists.txt | 13 + .../include/gpu_vendor/hip.h | 2 + .../include/gpu_vendor/musa.h | 26 ++ .../include/gpu_vendor/ubshmem.h | 2 + .../include/transfer_engine.h | 16 + .../include/transfer_engine_impl.h | 17 + .../include/transport/device/comm_device.cuh | 136 ++++++ .../transport/device/cuda/cuda_ops.cuh | 144 ++++++ .../include/transport/device/device_ops.cuh | 12 + .../transport/device/device_transport.h | 168 +++++++ .../transport/device/ibgda}/compiler.h | 0 .../include/transport/device/ibgda}/memheap.h | 0 .../transport/device/ibgda}/mlx5_ifc.h | 0 .../transport/device/ibgda}/mlx5_prm.h | 0 .../include/transport/device/ibgda}/mlx5gda.h | 5 + .../include/transport/device/ibgda}/os.h | 0 .../include/transport/device/ibgda_device.cuh | 206 +++++++++ .../transport/device/musa/musa_ops.cuh | 137 ++++++ .../include/transport/device/p2p_device.cuh | 46 ++ mooncake-transfer-engine/src/CMakeLists.txt | 2 +- .../src/transfer_engine.cpp | 24 + .../src/transfer_engine_impl.cpp | 18 + .../src/transport/CMakeLists.txt | 8 + .../src/transport/device/CMakeLists.txt | 22 + .../device/ibgda_device_transport.cpp | 430 ++++++++++++++++++ .../src/transport/device}/mlx5gda.cpp | 41 +- .../transport/device/p2p_device_transport.cpp | 268 +++++++++++ 40 files changed, 2100 insertions(+), 17 deletions(-) create mode 100644 mooncake-transfer-engine/example/device_transport_example.cu create mode 100644 mooncake-transfer-engine/include/transport/device/comm_device.cuh create mode 100644 mooncake-transfer-engine/include/transport/device/cuda/cuda_ops.cuh create mode 100644 mooncake-transfer-engine/include/transport/device/device_ops.cuh create mode 100644 mooncake-transfer-engine/include/transport/device/device_transport.h rename {mooncake-ep/include/mooncake_ibgda => mooncake-transfer-engine/include/transport/device/ibgda}/compiler.h (100%) rename {mooncake-ep/include/mooncake_ibgda => mooncake-transfer-engine/include/transport/device/ibgda}/memheap.h (100%) rename {mooncake-ep/include/mooncake_ibgda => mooncake-transfer-engine/include/transport/device/ibgda}/mlx5_ifc.h (100%) rename {mooncake-ep/include/mooncake_ibgda => mooncake-transfer-engine/include/transport/device/ibgda}/mlx5_prm.h (100%) rename {mooncake-ep/include/mooncake_ibgda => mooncake-transfer-engine/include/transport/device/ibgda}/mlx5gda.h (97%) rename {mooncake-ep/include/mooncake_ibgda => mooncake-transfer-engine/include/transport/device/ibgda}/os.h (100%) create mode 100644 mooncake-transfer-engine/include/transport/device/ibgda_device.cuh create mode 100644 mooncake-transfer-engine/include/transport/device/musa/musa_ops.cuh create mode 100644 mooncake-transfer-engine/include/transport/device/p2p_device.cuh create mode 100644 mooncake-transfer-engine/src/transport/device/CMakeLists.txt create mode 100644 mooncake-transfer-engine/src/transport/device/ibgda_device_transport.cpp rename {mooncake-ep/src/mooncake_ibgda => mooncake-transfer-engine/src/transport/device}/mlx5gda.cpp (91%) create mode 100644 mooncake-transfer-engine/src/transport/device/p2p_device_transport.cpp diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index dccd5e01..a2fca511 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -172,7 +172,7 @@ jobs: export LD_LIBRARY_PATH=$GITHUB_WORKSPACE/build/mooncake-common:$GITHUB_WORKSPACE/build/mooncake-store/src:$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src:$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src/common/base:$GITHUB_WORKSPACE/build/mooncake-common/etcd export CGO_ENABLED=1 export CGO_CFLAGS="-I$GITHUB_WORKSPACE/mooncake-store/include -I$GITHUB_WORKSPACE/mooncake-transfer-engine/include" - export CGO_LDFLAGS="-L$GITHUB_WORKSPACE/build/mooncake-store/src -L$GITHUB_WORKSPACE/build/mooncake-store/src/cachelib_memory_allocator -L$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src -L$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src/common/base -L$GITHUB_WORKSPACE/build/mooncake-common -L$GITHUB_WORKSPACE/build/mooncake-common/etcd -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio -letcd_wrapper -lstdc++ -lnuma -lglog -lgflags -libverbs -ljsoncpp -lzstd -lcurl -luring -lasan -lm -lgcov -lxxhash" + export CGO_LDFLAGS="-L$GITHUB_WORKSPACE/build/mooncake-store/src -L$GITHUB_WORKSPACE/build/mooncake-store/src/cachelib_memory_allocator -L$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src -L$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src/common/base -L$GITHUB_WORKSPACE/build/mooncake-common -L$GITHUB_WORKSPACE/build/mooncake-common/etcd -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio -letcd_wrapper -lstdc++ -lnuma -lglog -lgflags -libverbs -lmlx5 -ljsoncpp -lzstd -lcurl -luring -lasan -lm -lgcov -lxxhash" # Link cudart if CUDA is available (needed for D2H staging in mooncake_store) if [ -d /usr/local/cuda/lib64 ]; then export CGO_LDFLAGS="$CGO_LDFLAGS -L/usr/local/cuda/lib64 -lcudart"; fi ASAN_OPTIONS=detect_leaks=0:verify_asan_link_order=0 MC_METADATA_SERVER=http://127.0.0.1:8080/metadata go test -v ./tests/... diff --git a/CMakeLists.txt b/CMakeLists.txt index 30822b81..2872bd1a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -145,6 +145,7 @@ if (WITH_EP) "-DTORCH_CUDA_ARCH_LIST=${_torch_cuda_arch_list_pipe}" "-DSTAGING_DIR=${EP_PG_STAGING_DIR}" "-DENGINE_SO_PATH=$" + "-DEP_USE_MUSA=$,1,0>" -P "${CMAKE_CURRENT_SOURCE_DIR}/mooncake-ep/BuildEpExt.cmake" COMMENT "Building Mooncake EP Python extension(s)" DEPENDS engine diff --git a/mooncake-common/common.cmake b/mooncake-common/common.cmake index 9c2a1352..8a5308c0 100644 --- a/mooncake-common/common.cmake +++ b/mooncake-common/common.cmake @@ -1,5 +1,6 @@ set(CMAKE_C_STANDARD 99) set(CMAKE_CXX_STANDARD 20) +set(CMAKE_CUDA_STANDARD 20) option(ENABLE_DEBUG_SYMBOLS "Include debug symbols (-g) in compilation" ON) diff --git a/mooncake-ep/BuildEpExt.cmake b/mooncake-ep/BuildEpExt.cmake index d8355936..c94529cb 100644 --- a/mooncake-ep/BuildEpExt.cmake +++ b/mooncake-ep/BuildEpExt.cmake @@ -10,6 +10,7 @@ # TORCH_CUDA_ARCH_LIST - pipe-separated CUDA arch list forwarded to torch # STAGING_DIR - destination directory for the built .so files # ENGINE_SO_PATH - absolute path to the built engine.cpython-XYZ.so +# EP_USE_MUSA - set to "1" when building for MUSA (MTLink path) cmake_minimum_required(VERSION 3.16) @@ -34,6 +35,11 @@ endif() set(ENV{MAKEFLAGS} "") set(ENV{MFLAGS} "") set(ENV{TORCH_CUDA_ARCH_LIST} "${TORCH_CUDA_ARCH_LIST}") +if(EP_USE_MUSA) + set(ENV{MOONCAKE_EP_USE_MUSA} "1") +else() + unset(ENV{MOONCAKE_EP_USE_MUSA}) +endif() # --------------------------------------------------------------------------- # 2. Ensure engine.so exists in mooncake-wheel/mooncake/ for setup.py linking. diff --git a/mooncake-ep/include/mooncake_ep_buffer.h b/mooncake-ep/include/mooncake_ep_buffer.h index 3cbb5d0b..ba13407e 100644 --- a/mooncake-ep/include/mooncake_ep_buffer.h +++ b/mooncake-ep/include/mooncake_ep_buffer.h @@ -6,8 +6,8 @@ #include #include #include -#include -#include +#include +#include #include #include #include diff --git a/mooncake-ep/setup.py b/mooncake-ep/setup.py index 8a63079f..671d0f03 100644 --- a/mooncake-ep/setup.py +++ b/mooncake-ep/setup.py @@ -39,7 +39,6 @@ setup( "src/ep_py.cpp", "src/mooncake_ep_buffer.cpp", "src/mooncake_ep_kernel.cu", - "src/mooncake_ibgda/mlx5gda.cpp", ], extra_compile_args={ "cxx": [f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", "-std=c++20", "-O3", "-g0"], diff --git a/mooncake-ep/src/CMakeLists.txt b/mooncake-ep/src/CMakeLists.txt index 850f8e12..574ab514 100644 --- a/mooncake-ep/src/CMakeLists.txt +++ b/mooncake-ep/src/CMakeLists.txt @@ -1,4 +1,4 @@ -add_library(mooncake_ep ep_py.cpp mooncake_ep_buffer.cpp mooncake_ep_kernel.cu mooncake_ibgda/mlx5gda.cpp) +add_library(mooncake_ep ep_py.cpp mooncake_ep_buffer.cpp mooncake_ep_kernel.cu) set_target_properties(mooncake_ep PROPERTIES POSITION_INDEPENDENT_CODE ON) target_link_libraries(mooncake_ep PUBLIC ${TORCH_LIBRARIES} transfer_engine ibverbs mlx5) diff --git a/mooncake-ep/src/mooncake_ep_kernel.cu b/mooncake-ep/src/mooncake_ep_kernel.cu index 51979fd9..990cb187 100644 --- a/mooncake-ep/src/mooncake_ep_kernel.cu +++ b/mooncake-ep/src/mooncake_ep_kernel.cu @@ -7,7 +7,7 @@ #include #include #include -#include +#include #include namespace mooncake { diff --git a/mooncake-p2p-store/build.sh b/mooncake-p2p-store/build.sh index 72aca4b3..774e6ba6 100644 --- a/mooncake-p2p-store/build.sh +++ b/mooncake-p2p-store/build.sh @@ -35,7 +35,7 @@ EXT_LDFLAGS="-L$BUILD_DIR/mooncake-transfer-engine/src" EXT_LDFLAGS+=" -L$BUILD_DIR/mooncake-transfer-engine/src/common/base" EXT_LDFLAGS+=" -L$BUILD_DIR/mooncake-common" EXT_LDFLAGS+=" -L$BUILD_DIR/mooncake-common/src" -EXT_LDFLAGS+=" -ltransfer_engine -lbase -lasio -lstdc++ -lnuma -lglog -libverbs -ljsoncpp -lmooncake_common" +EXT_LDFLAGS+=" -ltransfer_engine -lbase -lasio -lstdc++ -lnuma -lglog -libverbs -lmlx5 -ljsoncpp -lmooncake_common -lm" if [ -d "/usr/local/cuda/lib64/stubs" ]; then EXT_LDFLAGS+=" -L/usr/local/cuda/lib64/stubs" diff --git a/mooncake-store/go/build.sh b/mooncake-store/go/build.sh index bae668d4..52304cc7 100755 --- a/mooncake-store/go/build.sh +++ b/mooncake-store/go/build.sh @@ -42,7 +42,7 @@ CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-transfer-engine/src/common/base" CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-common" CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-common/src" CGO_LDFLAGS+=" -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio -lmooncake_common -lxxhash" -CGO_LDFLAGS+=" -lstdc++ -lnuma -lglog -lgflags -libverbs -ljsoncpp -lzstd -lcurl" +CGO_LDFLAGS+=" -lstdc++ -lnuma -lglog -lgflags -libverbs -lmlx5 -ljsoncpp -lzstd -lcurl -lm" if [ -d "/usr/local/cuda/lib64" ]; then CGO_LDFLAGS+=" -L/usr/local/cuda/lib64 -lcudart" diff --git a/mooncake-store/rust/build.rs b/mooncake-store/rust/build.rs index a0a9a140..49847e46 100644 --- a/mooncake-store/rust/build.rs +++ b/mooncake-store/rust/build.rs @@ -331,6 +331,7 @@ fn main() { ("curl", &["curl"]), ("cuda", &["cuda"]), ("cudart", &["cudart"]), + ("mlx5", &["mlx5"]), // IBGDA device transport (mlx5 DevX) pulled into transfer_engine, CUDA-only ("uring", &["uring"]), ] { if has_library(&search_dirs, candidates) { diff --git a/mooncake-transfer-engine/example/CMakeLists.txt b/mooncake-transfer-engine/example/CMakeLists.txt index 15eaba45..4adfcdba 100644 --- a/mooncake-transfer-engine/example/CMakeLists.txt +++ b/mooncake-transfer-engine/example/CMakeLists.txt @@ -61,3 +61,48 @@ endif() if(USE_UBSHMEM) target_link_libraries(transfer_engine_bench PUBLIC transfer_engine) endif() + +# Device Transport Example — two-rank P2P write + signal via Device API. +# Requires 2 GPUs with P2P access. Built but not registered with CTest +# (manual execution only, like rdma_transport_test). +# +# Off by default: this .cu target pulls in transfer_engine.h, whose common.h +# uses C++20 std::string APIs (e.g. starts_with), so it needs the CUDA20 +# dialect. Older CMake (such as the CI Docker image) cannot enable CUDA20 and +# fails at generate time. Opt in explicitly with -DBUILD_DEVICE_TRANSPORT_EXAMPLE=ON +# when building manually on a host with a recent CMake and 2 GPUs. +option(BUILD_DEVICE_TRANSPORT_EXAMPLE + "Build the two-rank Device API P2P example (requires CUDA20-capable CMake and 2 GPUs)" + OFF) +if(USE_CUDA AND BUILD_DEVICE_TRANSPORT_EXAMPLE) + enable_language(CUDA) + add_executable(device_transport_example + ${WORKSPACE}/device_transport_example.cu) + set_source_files_properties( + ${WORKSPACE}/device_transport_example.cu + PROPERTIES LANGUAGE CUDA) + target_include_directories(device_transport_example PRIVATE + ${CMAKE_SOURCE_DIR}/mooncake-transfer-engine/include) + target_link_libraries(device_transport_example PUBLIC + transfer_engine gflags::gflags glog::glog) + # common.h (pulled in via transfer_engine.h) uses C++20 std::string APIs + # such as starts_with; nvcc does not inherit CMAKE_CXX_STANDARD, so set the + # CUDA standard explicitly on this target. CUDA_EXTENSIONS OFF requests + # plain -std=c++20 (nvcc has no gnu++20 dialect mapping). + set_target_properties(device_transport_example PROPERTIES + CUDA_STANDARD 20 CUDA_STANDARD_REQUIRED ON CUDA_EXTENSIONS OFF) + # Convert torch-style arch list (e.g. "8.0;9.0") to CMake CUDA format + # (e.g. "80;90") by stripping the dot. + if(TORCH_CUDA_ARCH_LIST) + set(_cuda_arch_list "") + foreach(_arch IN LISTS TORCH_CUDA_ARCH_LIST) + string(REPLACE "." "" _arch_clean "${_arch}") + list(APPEND _cuda_arch_list "${_arch_clean}") + endforeach() + set_target_properties(device_transport_example PROPERTIES + CUDA_ARCHITECTURES "${_cuda_arch_list}") + else() + set_target_properties(device_transport_example PROPERTIES + CUDA_ARCHITECTURES "80;90") + endif() +endif() diff --git a/mooncake-transfer-engine/example/device_transport_example.cu b/mooncake-transfer-engine/example/device_transport_example.cu new file mode 100644 index 00000000..f06ade3b --- /dev/null +++ b/mooncake-transfer-engine/example/device_transport_example.cu @@ -0,0 +1,305 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Device Transport Example — two-rank P2P write + signal via Device API. +// +// This example demonstrates the full Device API lifecycle: +// 1. Host side: P2pTransport for IPC handle exchange and peer mapping. +// 2. Device side: CommCtx + mc_route_put + mc_signal for GPU-initiated +// P2P data transfer and notification. +// +// Usage (two terminals on the same node): +// $ ./device_transport_example --rank=0 +// $ ./device_transport_example --rank=1 +// +// The example uses file-based IPC handle exchange (no external dependencies). +// Requires 2 GPUs with P2P access (NVLink or PCIe). + +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#include "transfer_engine.h" +#include "transport/device/comm_device.cuh" +#include "transport/device/device_transport.h" + +DEFINE_int32(rank, -1, "Rank of this process (0 or 1)"); +DEFINE_int32(world_size, 2, "Total number of ranks"); +DEFINE_int32(gpu_id, -1, "GPU ID (defaults to rank)"); +DEFINE_string(metadata_server, "P2PHANDSHAKE", + "Metadata server (P2PHANDSHAKE for no external deps)"); +DEFINE_string(local_server_name, "", + "Local server name (default: 127.0.0.1:)"); +DEFINE_string(ipc_dir, "/tmp", "Directory for IPC handle exchange files"); +DEFINE_int32(kDataBytes, 4096, "Bytes to transfer in the P2P write"); +DEFINE_int32(kSignalOffset, 0, "Offset within buffer for the signal word"); + +namespace { + +static void checkCuda(cudaError_t err, const char* msg) { + if (err != cudaSuccess) { + LOG(FATAL) << msg << ": " << cudaGetErrorString(err); + } +} + +// --------------------------------------------------------------------------- +// File-based IPC handle exchange +// --------------------------------------------------------------------------- + +static std::string ipcFilePath(int rank) { + return FLAGS_ipc_dir + "/device_api_ex_rank_" + std::to_string(rank) + + ".bin"; +} + +static void writeIpcHandle(int rank, const std::vector& handle) { + std::string path = ipcFilePath(rank); + std::ofstream ofs(path, std::ios::binary | std::ios::trunc); + CHECK(ofs) << "Failed to open " << path << " for writing"; + uint32_t size = static_cast(handle.size()); + ofs.write(reinterpret_cast(&size), sizeof(size)); + ofs.write(reinterpret_cast(handle.data()), + handle.size() * sizeof(int32_t)); + ofs.close(); + LOG(INFO) << "Rank " << rank << " wrote IPC handle (" << handle.size() + << " int32s) to " << path; +} + +static std::vector readIpcHandle(int rank) { + std::string path = ipcFilePath(rank); + // Poll until the file appears (peer may not have written it yet). + for (int attempt = 0; attempt < 300; ++attempt) { + std::ifstream ifs(path, std::ios::binary); + if (ifs) { + uint32_t size = 0; + ifs.read(reinterpret_cast(&size), sizeof(size)); + CHECK(size > 0 && size <= 256) + << "Invalid IPC handle size: " << size; + std::vector handle(size); + ifs.read(reinterpret_cast(handle.data()), + size * sizeof(int32_t)); + ifs.close(); + LOG(INFO) << "Rank " << FLAGS_rank << " read IPC handle from rank " + << rank << " (" << size << " int32s)"; + return handle; + } + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + } + LOG(FATAL) << "Timeout waiting for IPC handle from rank " << rank; + return {}; +} + +// --------------------------------------------------------------------------- +// Device-side kernel +// --------------------------------------------------------------------------- + +// GDR buffer layout: +// [0 .. kDataBytes) — data region +// [kDataBytes .. kDataBytes+4) — signal word (int32_t) +static constexpr int kSignalWordOffset = 4096; // must match kDataBytes default + +// Rank 0: P2P-write data to rank 1, then signal. +__global__ void senderKernel(mooncake::device::CommCtx ctx, int dst_rank, + int data_bytes) { + using namespace mooncake::device; + if (threadIdx.x != 0 || blockIdx.x != 0) return; + + // Write a known pattern to the peer's data region via P2P. + char* local_data = reinterpret_cast(ctx.p2p.local_base); + void* peer_data = mc_route_put(ctx, dst_rank, local_data); + if (peer_data == nullptr) { + printf("[Rank 0] FAIL: P2P not available to rank %d\n", dst_rank); + asm("trap;"); + return; + } + + // Fill local buffer with pattern, then copy to peer via P2P store. + for (int i = 0; i < data_bytes; ++i) { + local_data[i] = static_cast(i & 0xFF); + } + // Use 16-byte non-temporal stores for the bulk copy. + for (int i = 0; i < data_bytes; i += 16) { + int4 val = mc_ld_nc(reinterpret_cast(local_data + i)); + mc_st_na( + reinterpret_cast(reinterpret_cast(peer_data) + i), + val); + } + + // Signal: write 1 to the peer's signal word. + int* local_sig = reinterpret_cast(local_data + kSignalWordOffset); + *local_sig = 1; + mc_signal(ctx, dst_rank, 0 /*channel*/, 1 /*qps_per_rank*/, local_sig, 1); +} + +// Rank 1: wait for signal from rank 0, then verify data. +__global__ void receiverKernel(mooncake::device::CommCtx ctx, int src_rank, + int data_bytes) { + using namespace mooncake::device; + if (threadIdx.x != 0 || blockIdx.x != 0) return; + + char* local_data = reinterpret_cast(ctx.p2p.local_base); + int* local_sig = reinterpret_cast(local_data + kSignalWordOffset); + + // Spin-wait for the signal. + // The sender writes the signal via P2P store (mc_p2p_signal → + // mc_st_release). We use ld_acquire to observe it. + while (mc_ld_acquire(local_sig) == 0) { + __threadfence_block(); + } + + // Verify the data written by rank 0. + int mismatches = 0; + for (int i = 0; i < data_bytes; ++i) { + if (local_data[i] != static_cast(i & 0xFF)) { + ++mismatches; + } + } + if (mismatches > 0) { + printf("[Rank 1] FAIL: %d byte mismatches in received data\n", + mismatches); + } else { + printf("[Rank 1] PASS: all %d bytes match expected pattern\n", + data_bytes); + } +} + +} // namespace + +int main(int argc, char** argv) { + gflags::ParseCommandLineFlags(&argc, &argv, true); + google::InitGoogleLogging(argv[0]); + + int rank = FLAGS_rank; + int world_size = FLAGS_world_size; + CHECK(rank == 0 || rank == 1) << "--rank must be 0 or 1"; + CHECK_EQ(world_size, 2) << "This example requires exactly 2 ranks"; + CHECK_EQ(FLAGS_kDataBytes % 16, 0) << "kDataBytes must be a multiple of 16"; + CHECK_LE(FLAGS_kDataBytes, kSignalWordOffset) + << "kDataBytes cannot exceed " << kSignalWordOffset; + + int gpu_id = FLAGS_gpu_id >= 0 ? FLAGS_gpu_id : rank; + checkCuda(cudaSetDevice(gpu_id), "cudaSetDevice"); + LOG(INFO) << "Rank " << rank << " using GPU " << gpu_id; + + // ----------------------------------------------------------------------- + // 1. Create TransferEngine and get P2pTransport. + // ----------------------------------------------------------------------- + auto engine = std::make_unique(false); + + std::string local_name = FLAGS_local_server_name; + if (local_name.empty()) { + local_name = "127.0.0.1:" + std::to_string(12345 + rank); + } + engine->init(FLAGS_metadata_server, local_name); + + auto* p2p = engine->getOrCreateP2pTransport(world_size); + CHECK_NOTNULL(p2p); + + // ----------------------------------------------------------------------- + // 2. Allocate GDR buffer and fill with initial pattern. + // ----------------------------------------------------------------------- + constexpr size_t kBufSize = kSignalWordOffset + sizeof(int32_t); + void* gdr_buffer = p2p->allocateBuffer(kBufSize); + CHECK_NOTNULL(gdr_buffer); + LOG(INFO) << "Rank " << rank << " allocated GDR buffer: " << gdr_buffer; + + // Zero-initialize the signal word. + checkCuda(cudaMemset(gdr_buffer, 0, kBufSize), "cudaMemset"); + + // ----------------------------------------------------------------------- + // 3. Export IPC handle and exchange with peer. + // ----------------------------------------------------------------------- + auto my_handle = p2p->exportIpcHandle(gdr_buffer); + CHECK(!my_handle.empty()) << "exportIpcHandle returned empty vector"; + writeIpcHandle(rank, my_handle); + + // Read peer's handle. + int peer_rank = 1 - rank; + auto peer_handle = readIpcHandle(peer_rank); + + // Build remote_handles vector: index by rank. + std::vector> remote_handles(world_size); + remote_handles[rank] = my_handle; + remote_handles[peer_rank] = peer_handle; + + std::vector active_ranks_mask(world_size, 1); + + p2p->importPeerHandles(gdr_buffer, rank, world_size, remote_handles, + active_ranks_mask); + + CHECK(p2p->allPeersAccessible()) + << "P2P not accessible between rank " << rank << " and peer"; + LOG(INFO) << "Rank " << rank << " P2P peer mapping complete"; + + // ----------------------------------------------------------------------- + // 4. Build CommCtx and launch kernel. + // ----------------------------------------------------------------------- + mooncake::device::CommCtx ctx{}; + ctx.rank = rank; + ctx.p2p.available = p2p->availableTablePtr(); + ctx.p2p.peer_ptrs = p2p->peerPtrsTablePtr(); + ctx.p2p.local_base = gdr_buffer; + // IBGDA fields left as nullptr (not used for P2P-only example). + + // Pass CommCtx by value — CUDA copies it to kernel parameter space. + if (rank == 0) { + LOG(INFO) << "Rank 0 launching sender kernel..."; + senderKernel<<<1, 1>>>(ctx, peer_rank, FLAGS_kDataBytes); + } else { + LOG(INFO) << "Rank 1 launching receiver kernel..."; + receiverKernel<<<1, 1>>>(ctx, peer_rank, FLAGS_kDataBytes); + } + checkCuda(cudaDeviceSynchronize(), "cudaDeviceSynchronize"); + + // ----------------------------------------------------------------------- + // 5. Barrier: rank 1 signals completion, rank 0 waits before cleanup. + // The IPC handle is only valid while the original allocation exists, + // so rank 0 must not free its buffer until rank 1 has opened it. + // ----------------------------------------------------------------------- + if (rank == 1) { + // Signal rank 0 that we're done. + std::string done_path = FLAGS_ipc_dir + "/device_api_ex_done.bin"; + std::ofstream ofs(done_path); + ofs << "1"; + ofs.close(); + } else { + // Wait for rank 1 to finish. + std::string done_path = FLAGS_ipc_dir + "/device_api_ex_done.bin"; + for (int attempt = 0; attempt < 300; ++attempt) { + std::ifstream ifs(done_path); + if (ifs) break; + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + } + } + + // ----------------------------------------------------------------------- + // 6. Cleanup. + // ----------------------------------------------------------------------- + p2p->freeBuffer(gdr_buffer); + + // Remove IPC handle files. + std::remove(ipcFilePath(rank).c_str()); + if (rank == 0) { + std::remove((FLAGS_ipc_dir + "/device_api_ex_done.bin").c_str()); + } + + LOG(INFO) << "Rank " << rank << " done."; + return 0; +} diff --git a/mooncake-transfer-engine/include/CMakeLists.txt b/mooncake-transfer-engine/include/CMakeLists.txt index 56929077..c9486b9b 100644 --- a/mooncake-transfer-engine/include/CMakeLists.txt +++ b/mooncake-transfer-engine/include/CMakeLists.txt @@ -10,3 +10,16 @@ install(FILES transfer_metadata.h DESTINATION include) install(FILES ub_allocator.h DESTINATION include) install(FILES common/base/status.h DESTINATION include/common/base) install(FILES transport/transport.h DESTINATION include/transport) + +# Device API headers (header-only, consumed by EP kernel) +install(FILES transport/device/device_transport.h DESTINATION include/transport/device) +install(FILES transport/device/device_ops.cuh DESTINATION include/transport/device) +install(FILES transport/device/comm_device.cuh DESTINATION include/transport/device) +install(FILES transport/device/p2p_device.cuh DESTINATION include/transport/device) +install(FILES transport/device/ibgda_device.cuh DESTINATION include/transport/device) +install(FILES transport/device/cuda/cuda_ops.cuh DESTINATION include/transport/device/cuda) +install(FILES transport/device/musa/musa_ops.cuh DESTINATION include/transport/device/musa) + +# IBGDA library headers +install(DIRECTORY transport/device/ibgda/ DESTINATION include/transport/device/ibgda + FILES_MATCHING PATTERN "*.h") diff --git a/mooncake-transfer-engine/include/gpu_vendor/hip.h b/mooncake-transfer-engine/include/gpu_vendor/hip.h index b613c87a..6797fce6 100644 --- a/mooncake-transfer-engine/include/gpu_vendor/hip.h +++ b/mooncake-transfer-engine/include/gpu_vendor/hip.h @@ -1,3 +1,5 @@ +#pragma once + #include #include diff --git a/mooncake-transfer-engine/include/gpu_vendor/musa.h b/mooncake-transfer-engine/include/gpu_vendor/musa.h index b53d7707..73274ce8 100644 --- a/mooncake-transfer-engine/include/gpu_vendor/musa.h +++ b/mooncake-transfer-engine/include/gpu_vendor/musa.h @@ -1,3 +1,5 @@ +#pragma once + #include #include #include @@ -20,6 +22,11 @@ const static std::string GPU_PREFIX = "musa:"; #define CU_POINTER_ATTRIBUTE_RANGE_SIZE MU_POINTER_ATTRIBUTE_RANGE_SIZE #define CU_DEVICE_ATTRIBUTE_MEMORY_POOLS_SUPPORTED \ MU_DEVICE_ATTRIBUTE_MEMORY_POOLS_SUPPORTED +#define CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED \ + MU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED +#define CU_POINTER_ATTRIBUTE_DEVICE_ORDINAL MU_POINTER_ATTRIBUTE_DEVICE_ORDINAL + +#define CUcontext MUcontext #define CUdevice MUdevice #define CUdeviceptr MUdeviceptr @@ -93,3 +100,22 @@ const static std::string GPU_PREFIX = "musa:"; #define cudaEvent_t musaEvent_t #define cudaHostGetDevicePointer musaHostGetDevicePointer #define cudaSuccess musaSuccess +#define cudaDeviceGetAttribute musaDeviceGetAttribute +#define cudaEvent_t musaEvent_t +#define cudaMemcpyDeviceToDevice musaMemcpyDeviceToDevice +#define cudaDevAttrClockRate musaDevAttrClockRate +#define cudaLaunchConfig_t musaLaunchConfig_t +#define cudaLaunchAttribute musaLaunchAttribute +#define cudaLaunchAttributeCooperative musaLaunchAttributeCooperative +#define cudaLaunchKernelEx musaLaunchKernelEx +#define CUDA_R_16BF MUSA_R_16BF +#define CUDA_R_32F MUSA_R_32F + +// IBGDA-specific mappings +#define cuInit muInit +#define cuDevicePrimaryCtxRetain muDevicePrimaryCtxRetain +#define cuDevicePrimaryCtxRelease muDevicePrimaryCtxRelease +#define cuCtxSetCurrent muCtxSetCurrent +#define cudaHostRegisterMapped musaHostRegisterMapped +#define cudaHostRegisterIoMemory musaHostRegisterIoMemory +#define cudaHostGetDevicePointer musaHostGetDevicePointer diff --git a/mooncake-transfer-engine/include/gpu_vendor/ubshmem.h b/mooncake-transfer-engine/include/gpu_vendor/ubshmem.h index 7be526a7..584680e6 100644 --- a/mooncake-transfer-engine/include/gpu_vendor/ubshmem.h +++ b/mooncake-transfer-engine/include/gpu_vendor/ubshmem.h @@ -1,3 +1,5 @@ +#pragma once + #include #include diff --git a/mooncake-transfer-engine/include/transfer_engine.h b/mooncake-transfer-engine/include/transfer_engine.h index 65bc8741..8d5b8d2e 100644 --- a/mooncake-transfer-engine/include/transfer_engine.h +++ b/mooncake-transfer-engine/include/transfer_engine.h @@ -25,6 +25,12 @@ class TransferEngineImpl; namespace tent { class TransferEngine; }; +#if defined(USE_CUDA) || defined(USE_MUSA) +namespace device { +class P2pTransport; +class RdmaTransport; +} // namespace device +#endif using TransferRequest = Transport::TransferRequest; using TransferStatus = Transport::TransferStatus; using TransferStatusEnum = Transport::TransferStatusEnum; @@ -150,6 +156,16 @@ class TransferEngine { Transport* getTransport(const std::string& proto); +#if defined(USE_CUDA) || defined(USE_MUSA) + // Device transport accessors (P2P + IBGDA). Lazily created on first + // call and owned by the TransferEngine. These allow EP (and future + // CPU-proxy paths) to obtain device transports from an engine instance + // instead of calling the global factory functions directly. + device::P2pTransport* getOrCreateP2pTransport(int num_ranks); + device::RdmaTransport* getOrCreateRdmaTransport( + const std::vector& device_filter = {}); +#endif + /** * @brief Check if TCP is the only installed transport. * diff --git a/mooncake-transfer-engine/include/transfer_engine_impl.h b/mooncake-transfer-engine/include/transfer_engine_impl.h index db2357e2..f4584e27 100644 --- a/mooncake-transfer-engine/include/transfer_engine_impl.h +++ b/mooncake-transfer-engine/include/transfer_engine_impl.h @@ -33,6 +33,9 @@ #include "transfer_metadata.h" #include "transfer_engine.h" #include "transport/transport.h" +#if defined(USE_CUDA) || defined(USE_MUSA) +#include "transport/device/device_transport.h" +#endif #ifdef WITH_METRICS #include "ylt/metric/counter.hpp" #include "ylt/metric/histogram.hpp" @@ -335,6 +338,13 @@ class TransferEngineImpl { return multi_transports_->getTransport(proto); } +#if defined(USE_CUDA) || defined(USE_MUSA) + // Device transport accessors — lazily created, owned by this impl. + device::P2pTransport* getOrCreateP2pTransport(int num_ranks); + device::RdmaTransport* getOrCreateRdmaTransport( + const std::vector& device_filter = {}); +#endif + bool isTcpOnly() const { return multi_transports_->isTcpOnly(); } int syncSegmentCache(const std::string& segment_name = "") { @@ -411,6 +421,13 @@ class TransferEngineImpl { std::vector filter_; bool use_barex_ = false; +#if defined(USE_CUDA) || defined(USE_MUSA) + // Device transports (P2P + IBGDA) — lazily created, owned by this impl. + // Referenced by EP and future CPU-proxy paths. + std::unique_ptr p2p_transport_; + std::unique_ptr rdma_transport_; +#endif + #ifdef WITH_METRICS // Latency bucket in microseconds inline static const std::vector kTaskLatencyBuckets = { diff --git a/mooncake-transfer-engine/include/transport/device/comm_device.cuh b/mooncake-transfer-engine/include/transport/device/comm_device.cuh new file mode 100644 index 00000000..29e6c6c6 --- /dev/null +++ b/mooncake-transfer-engine/include/transport/device/comm_device.cuh @@ -0,0 +1,136 @@ +// Communication device API — top-level context and routing. +// +// CommCtx bundles all transport state the kernel needs. The kernel +// constructs one from the raw pointers passed in and calls mc_route_put / +// mc_signal / mc_red_add instead of touching transport internals directly. +#pragma once + +#include "transport/device/device_ops.cuh" +#include "transport/device/p2p_device.cuh" +#include "transport/device/ibgda_device.cuh" + +namespace mooncake { +namespace device { + +// --------------------------------------------------------------------------- +// CommCtx +// --------------------------------------------------------------------------- + +struct CommCtx { + P2PContext p2p; + IbgdaContext ibgda; + int rank; +}; + +// Construct CommCtx from the raw kernel arguments. +// raddrs/rkeys/qp_devctxs may be nullptr on MUSA (ignored). +__device__ __forceinline__ CommCtx make_comm_ctx( + void* gdr_buffer, const int32_t* nvlink_available, + void* const* ipc_peer_ptrs, void* raddrs, void* rkeys, void* qp_devctxs, + const void* rdma_send_signal_buffer, const void* rdma_recv_signal_buffer, + int rank, int num_ranks, int num_qps) { + CommCtx ctx; + ctx.rank = rank; + + ctx.p2p.available = nvlink_available; + ctx.p2p.peer_ptrs = ipc_peer_ptrs; + ctx.p2p.local_base = gdr_buffer; + + ctx.ibgda.qp_devctxs = reinterpret_cast(qp_devctxs); + ctx.ibgda.raddrs = reinterpret_cast(raddrs); + ctx.ibgda.rkeys = reinterpret_cast(rkeys); + ctx.ibgda.local_atomic_base = rdma_send_signal_buffer; + ctx.ibgda.remote_atomic_base = rdma_recv_signal_buffer; + + return ctx; +} + +// --------------------------------------------------------------------------- +// Routing helpers +// --------------------------------------------------------------------------- + +__device__ __forceinline__ bool mc_comm_p2p_available(const CommCtx& ctx, + int dst_rank) { + return mc_p2p_available(ctx.p2p, dst_rank); +} + +// Translate a local GDR pointer to the peer's mapped VA. +__device__ __forceinline__ void* mc_comm_peer_ptr(const CommCtx& ctx, + int dst_rank, + const void* local_ptr) { + return mc_p2p_peer_ptr(ctx.p2p, dst_rank, local_ptr); +} + +// --------------------------------------------------------------------------- +// mc_route_put +// +// Returns the destination pointer for a warp-cooperative copy: +// - local rank: recv_ptr itself (caller does UNROLLED_WARP_COPY) +// - P2P rank: peer-mapped recv_ptr (caller does UNROLLED_WARP_COPY) +// - IBGDA rank: nullptr (caller must stage data then call mc_rdma_put) +// --------------------------------------------------------------------------- +__device__ __forceinline__ void* mc_route_put(const CommCtx& ctx, int dst_rank, + void* recv_ptr) { + if (dst_rank == ctx.rank) return recv_ptr; + if (mc_comm_p2p_available(ctx, dst_rank)) + return mc_comm_peer_ptr(ctx, dst_rank, recv_ptr); + return nullptr; // IBGDA path +} + +// Issue an IBGDA RDMA WRITE. Call only when mc_route_put returned nullptr. +// lane_id: only lane 0 issues the WQE. +__device__ __forceinline__ void mc_rdma_put( + const CommCtx& ctx, int channel, int dst_rank, int qps_per_rank, + const void* send_ptr, + void* recv_ptr, // local VA of the recv slot (for raddr computation) + uint32_t nbytes, int lane_id) { + if (lane_id == 0) { + uint64_t recv_raddr = + ctx.ibgda.raddrs[dst_rank] + + (reinterpret_cast(recv_ptr) - + reinterpret_cast(ctx.p2p.local_base)); + mc_ibgda_put(ctx.ibgda, channel, dst_rank, ctx.rank, qps_per_rank, + send_ptr, recv_raddr, nbytes); + } +} + +// --------------------------------------------------------------------------- +// mc_signal / mc_red_add +// +// Route a signal (store) or reduction (atomic add) to dst_rank. +// sig_ptr is a local VA within the GDR buffer. +// --------------------------------------------------------------------------- + +__device__ __forceinline__ void mc_signal(const CommCtx& ctx, int dst_rank, + int channel, int qps_per_rank, + int* sig_ptr, int32_t val) { + if (dst_rank == ctx.rank) { + mc_st_release(sig_ptr, val); + return; + } + if (mc_comm_p2p_available(ctx, dst_rank)) { + mc_p2p_signal(ctx.p2p, dst_rank, sig_ptr, val); + } else { + uint64_t recv_raddr = + ctx.ibgda.raddrs[dst_rank] + + (reinterpret_cast(sig_ptr) - + reinterpret_cast(ctx.p2p.local_base)); + uint64_t laddr = + ctx.ibgda.raddrs[ctx.rank] + + (reinterpret_cast(sig_ptr) - + reinterpret_cast(ctx.ibgda.remote_atomic_base)) + + (reinterpret_cast(ctx.ibgda.local_atomic_base) - + reinterpret_cast(ctx.p2p.local_base)); + mc_ibgda_red_add(ctx.ibgda, channel, dst_rank, ctx.rank, qps_per_rank, + laddr, recv_raddr, val); + } +} + +__device__ __forceinline__ void mc_red_add(const CommCtx& ctx, int dst_rank, + int channel, int qps_per_rank, + int* sig_ptr, int32_t val) { + mc_signal(ctx, dst_rank, channel, qps_per_rank, sig_ptr, val); +} + +} // namespace device +} // namespace mooncake diff --git a/mooncake-transfer-engine/include/transport/device/cuda/cuda_ops.cuh b/mooncake-transfer-engine/include/transport/device/cuda/cuda_ops.cuh new file mode 100644 index 00000000..bb9e4d3e --- /dev/null +++ b/mooncake-transfer-engine/include/transport/device/cuda/cuda_ops.cuh @@ -0,0 +1,144 @@ +// CUDA PTX implementations of device-side memory ordering primitives. +// Included by device_ops.cuh when MOONCAKE_EP_USE_MUSA is not defined. +#pragma once + +#include +#include + +namespace mooncake { +namespace device { + +// --------------------------------------------------------------------------- +// Acquire loads — cross-GPU visibility (sys scope) +// --------------------------------------------------------------------------- +__device__ __forceinline__ int mc_ld_acquire(const int* ptr) { + int ret; + asm volatile("ld.acquire.sys.global.s32 %0, [%1];" : "=r"(ret) : "l"(ptr)); + return ret; +} + +__device__ __forceinline__ uint64_t mc_ld_acquire_u64(const uint64_t* ptr) { + uint64_t ret; + asm volatile("ld.acquire.sys.global.u64 %0, [%1];" : "=l"(ret) : "l"(ptr)); + return ret; +} + +// --------------------------------------------------------------------------- +// Release stores — cross-GPU visibility (sys scope), non-temporal (no alloc) +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_st_release(const int* ptr, int val) { + asm volatile("st.release.sys.global.L1::no_allocate.s32 [%0], %1;" + : + : "l"(ptr), "r"(val)); +} + +__device__ __forceinline__ void mc_st_release_u32(const uint32_t* ptr, + uint32_t val) { + asm volatile("st.release.sys.global.L1::no_allocate.b32 [%0], %1;" + : + : "l"(ptr), "r"(val)); +} + +__device__ __forceinline__ void mc_st_release_u64(const uint64_t* ptr, + uint64_t val) { + asm volatile("st.release.sys.global.L1::no_allocate.b64 [%0], %1;" + : + : "l"(ptr), "l"(val)); +} + +// --------------------------------------------------------------------------- +// Atomic add — release semantics, sys scope +// --------------------------------------------------------------------------- +__device__ __forceinline__ int mc_atomic_add_release(const int* ptr, int val) { + int ret; + asm volatile("atom.add.release.sys.global.s32 %0, [%1], %2;" + : "=r"(ret) + : "l"(ptr), "r"(val)); + return ret; +} + +// --------------------------------------------------------------------------- +// Non-coherent loads (read-only cache, no L1 alloc) — for bulk data reads +// --------------------------------------------------------------------------- +__device__ __forceinline__ int4 mc_ld_nc(const int4* ptr) { + int4 ret; + asm volatile( + "ld.global.nc.L1::no_allocate.L2::256B.v4.s32 {%0,%1,%2,%3}, [%4];" + : "=r"(ret.x), "=r"(ret.y), "=r"(ret.z), "=r"(ret.w) + : "l"(ptr)); + return ret; +} + +__device__ __forceinline__ int mc_ld_nc_s32(const int* ptr) { + int ret; + asm volatile("ld.global.nc.L1::no_allocate.s32 %0, [%1];" + : "=r"(ret) + : "l"(ptr)); + return ret; +} + +__device__ __forceinline__ float mc_ld_nc_f32(const float* ptr) { + float ret; + asm volatile("ld.global.nc.L1::no_allocate.f32 %0, [%1];" + : "=f"(ret) + : "l"(ptr)); + return ret; +} + +__device__ __forceinline__ int64_t mc_ld_nc_s64(const int64_t* ptr) { + int64_t ret; + asm volatile("ld.global.nc.L1::no_allocate.s64 %0, [%1];" + : "=l"(ret) + : "l"(ptr)); + return ret; +} + +// --------------------------------------------------------------------------- +// Non-temporal stores (no L1 alloc) — for bulk data writes +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_st_na(const int4* ptr, const int4& val) { + asm volatile("st.global.L1::no_allocate.v4.s32 [%0], {%1,%2,%3,%4};" + : + : "l"(ptr), "r"(val.x), "r"(val.y), "r"(val.z), "r"(val.w)); +} + +// --------------------------------------------------------------------------- +// Named barrier init — no-op on CUDA (hardware named barriers need no setup). +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_bar_init() {} + +// --------------------------------------------------------------------------- +// Named barrier (warp-group scope) — CUDA PTX bar.sync +// On MUSA this maps to __syncthreads(); see musa_ops.cuh for details. +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_bar_sync(int bar_id, int num_threads) { + asm volatile("bar.sync %0, %1;" : : "r"(bar_id), "r"(num_threads)); +} + +// --------------------------------------------------------------------------- +// Grid-level sync — cooperative_groups::this_grid().sync() +// On MUSA this is a no-op because the host always uses separate kernel +// launches (return_recv_hook=true), so SEND and RECV never run in the same +// kernel invocation. +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_grid_sync() { + cooperative_groups::this_grid().sync(); +} + +// --------------------------------------------------------------------------- +// Byte-swap helpers (for mlx5 big-endian WQE fields) +// --------------------------------------------------------------------------- +__device__ __forceinline__ uint16_t mc_bswap16(uint16_t x) { + return __byte_perm(x, x, 0x2301); +} +__device__ __forceinline__ uint32_t mc_bswap32(uint32_t x) { + return __byte_perm(x, x, 0x0123); +} +__device__ __forceinline__ uint64_t mc_bswap64(uint64_t x) { + uint32_t hi = __byte_perm((uint32_t)(x >> 32), 0, 0x0123); + uint32_t lo = __byte_perm((uint32_t)(x), 0, 0x0123); + return ((uint64_t)lo << 32) | hi; +} + +} // namespace device +} // namespace mooncake diff --git a/mooncake-transfer-engine/include/transport/device/device_ops.cuh b/mooncake-transfer-engine/include/transport/device/device_ops.cuh new file mode 100644 index 00000000..2835f815 --- /dev/null +++ b/mooncake-transfer-engine/include/transport/device/device_ops.cuh @@ -0,0 +1,12 @@ +// Device-side memory ordering primitives — platform-portable selector. +// +// This is the ONLY file in the device API that contains a platform #ifdef. +// All other device API headers include this file and use the mc_* functions. +// The kernel itself has zero platform #ifdef branches. +#pragma once + +#ifdef MOONCAKE_EP_USE_MUSA +#include "transport/device/musa/musa_ops.cuh" +#else +#include "transport/device/cuda/cuda_ops.cuh" +#endif diff --git a/mooncake-transfer-engine/include/transport/device/device_transport.h b/mooncake-transfer-engine/include/transport/device/device_transport.h new file mode 100644 index 00000000..82556e7d --- /dev/null +++ b/mooncake-transfer-engine/include/transport/device/device_transport.h @@ -0,0 +1,168 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once + +// EP DeviceTransport — platform-agnostic host-side abstraction for the two +// communication paths used by the EP kernel: +// +// P2pTransport — intra-node GPU-initiated P2P (NVLink on CUDA, MTLink on +// MUSA). Manages IPC handle exchange and peer pointer +// table. +// RdmaTransport — inter-node GPU-initiated RDMA (IBGDA / mlx5gda). +// Manages QP lifecycle, MR, and device context table. +// +// EP code includes only this header and calls the abstract interface. +// Platform-specific implementations live in device/ and are +// selected at build time by the factory functions at the bottom of this file. +// +// The header intentionally avoids including cuda_alike.h so it can be included +// from pure C++ translation units. Implementations include cuda_alike.h. + +#include +#include +#include +#include + +namespace mooncake { +namespace device { + +// --------------------------------------------------------------------------- +// P2pTransport +// +// Manages intra-node GPU P2P IPC handles and the device-visible peer pointer +// table used by the EP kernel. +// --------------------------------------------------------------------------- +class P2pTransport { + public: + virtual ~P2pTransport() = default; + + // Allocate the GDR buffer that will be shared via IPC. + // Returns a device pointer; size must be > 0. + virtual void* allocateBuffer(size_t bytes) = 0; + + // Free a buffer previously returned by allocateBuffer. + virtual void freeBuffer(void* ptr) = 0; + + // Export an IPC handle for the buffer allocated by this rank. + // Returns a byte blob (serialised as int32_t array for Python compat). + // Returns empty vector if IPC is not needed (e.g. fabric memory). + virtual std::vector exportIpcHandle(void* ptr) = 0; + + // Import peer IPC handles and populate the device-visible tables. + // remote_handles[i] is the handle exported by rank i (may be empty for + // ranks that use fabric memory or are on a different node). + // active_ranks_mask[i] == 1 means rank i is participating. + // After this call, availableTablePtr() and peerPtrsTablePtr() are valid. + virtual void importPeerHandles( + void* local_ptr, int rank, int num_ranks, + const std::vector>& remote_handles, + const std::vector& active_ranks_mask) = 0; + + // Device pointer to int32_t[num_ranks]: 1 if P2P is available to rank i. + virtual int32_t* availableTablePtr() = 0; + + // Device pointer to void*[num_ranks]: peer GDR buffer base pointers. + virtual void** peerPtrsTablePtr() = 0; + + // True if all active ranks have P2P access (fast-path condition). + virtual bool allPeersAccessible() const = 0; + + // Verify that peer-mapped memory is writable (via memcpy). + // Returns true if a small test write to each peer's mapped buffer + // succeeds. On failure, sets all_peers_accessible to false. + virtual bool verifyPeerAccess() = 0; +}; + +// --------------------------------------------------------------------------- +// RdmaLocalMetadata — exchanged between ranks during IBGDA bootstrap. +// --------------------------------------------------------------------------- +struct RdmaLocalMetadata { + int64_t raddr; + int32_t rkey; + int64_t subnet_prefix; + int64_t interface_id; + std::vector qpns; + std::vector lids; +}; + +// --------------------------------------------------------------------------- +// RdmaTransport +// +// Manages IBGDA QP lifecycle, MR registration, and the device-visible RDMA +// context tables (raddrs, rkeys, qp_devctxs) used by the EP kernel. +// --------------------------------------------------------------------------- +class RdmaTransport { + public: + virtual ~RdmaTransport() = default; + + // Initialise the RDMA transport for the given NIC. + // device_name: e.g. "mlx5_1". Pass empty string for auto-detect. + // Returns 0 on success, non-zero on failure (IBGDA disabled). + virtual int initialize(const std::string& device_name, int num_ranks, + int num_qps) = 0; + + // Register the GDR buffer for RDMA access. + virtual int registerMemory(void* ptr, size_t bytes) = 0; + + // Allocate the GPU-side control buffer (QP/CQ structures). + virtual int allocateControlBuffer() = 0; + + // Create QPs in RST→INIT state. Call after allocateControlBuffer. + // stream is a cudaStream_t / musaStream_t cast to void*. + virtual int createQueuePairs(void* stream) = 0; + + // Destroy and recreate QPs (used when active_ranks changes). + virtual int recreateQueuePairs(void* stream) = 0; + + // Connect QPs to peers using exchanged metadata. + // is_roce: true for RoCE, false for IB. + virtual int connectPeers(int local_rank, bool is_roce, + const std::vector& remote_addrs, + const std::vector& remote_keys, + const std::vector& remote_qpns, + const std::vector& remote_lids, + const std::vector& subnet_prefixes, + const std::vector& interface_ids, + const std::vector& active_ranks_mask) = 0; + + // Metadata for this rank, to be exchanged with peers. + virtual RdmaLocalMetadata localMetadata() const = 0; + + // Device pointers to the tables consumed by the EP kernel. + virtual void* raddrsPtr() = 0; // uint64_t[num_ranks] + virtual void* rkeysPtr() = 0; // uint32_t[num_ranks] + virtual void* qpDevCtxsPtr() = 0; // mlx5gda_qp_devctx[num_qps] + + virtual bool isRoce() const = 0; + virtual int gidIndex() const = 0; +}; + +// --------------------------------------------------------------------------- +// Factory functions — implemented in device_transport.cpp. +// Returns nullptr if the transport is not available on this platform. +// --------------------------------------------------------------------------- + +// Create the platform-native P2P transport (NVLink on CUDA, MTLink on MUSA). +std::unique_ptr createP2pDeviceTransport(int num_ranks); + +// Create the IBGDA RDMA transport backed by TE's RdmaContext. +// device_filter: optional whitelist of NIC names (e.g. {"mlx5_1", "mlx5_2"}). +// Empty vector = auto-detect via TE's Topology::discover() with no filter. +// Non-empty = restrict discovery to these NICs, then pick the closest one. +std::unique_ptr createIbgdaDeviceTransport( + const std::vector& device_filter = {}); + +} // namespace device +} // namespace mooncake diff --git a/mooncake-ep/include/mooncake_ibgda/compiler.h b/mooncake-transfer-engine/include/transport/device/ibgda/compiler.h similarity index 100% rename from mooncake-ep/include/mooncake_ibgda/compiler.h rename to mooncake-transfer-engine/include/transport/device/ibgda/compiler.h diff --git a/mooncake-ep/include/mooncake_ibgda/memheap.h b/mooncake-transfer-engine/include/transport/device/ibgda/memheap.h similarity index 100% rename from mooncake-ep/include/mooncake_ibgda/memheap.h rename to mooncake-transfer-engine/include/transport/device/ibgda/memheap.h diff --git a/mooncake-ep/include/mooncake_ibgda/mlx5_ifc.h b/mooncake-transfer-engine/include/transport/device/ibgda/mlx5_ifc.h similarity index 100% rename from mooncake-ep/include/mooncake_ibgda/mlx5_ifc.h rename to mooncake-transfer-engine/include/transport/device/ibgda/mlx5_ifc.h diff --git a/mooncake-ep/include/mooncake_ibgda/mlx5_prm.h b/mooncake-transfer-engine/include/transport/device/ibgda/mlx5_prm.h similarity index 100% rename from mooncake-ep/include/mooncake_ibgda/mlx5_prm.h rename to mooncake-transfer-engine/include/transport/device/ibgda/mlx5_prm.h diff --git a/mooncake-ep/include/mooncake_ibgda/mlx5gda.h b/mooncake-transfer-engine/include/transport/device/ibgda/mlx5gda.h similarity index 97% rename from mooncake-ep/include/mooncake_ibgda/mlx5gda.h rename to mooncake-transfer-engine/include/transport/device/ibgda/mlx5gda.h index bca275da..82030954 100644 --- a/mooncake-ep/include/mooncake_ibgda/mlx5gda.h +++ b/mooncake-transfer-engine/include/transport/device/ibgda/mlx5gda.h @@ -3,7 +3,12 @@ #include #include +#ifdef USE_MUSA +#include +#define cudaStream_t musaStream_t +#else #include +#endif #include #include diff --git a/mooncake-ep/include/mooncake_ibgda/os.h b/mooncake-transfer-engine/include/transport/device/ibgda/os.h similarity index 100% rename from mooncake-ep/include/mooncake_ibgda/os.h rename to mooncake-transfer-engine/include/transport/device/ibgda/os.h diff --git a/mooncake-transfer-engine/include/transport/device/ibgda_device.cuh b/mooncake-transfer-engine/include/transport/device/ibgda_device.cuh new file mode 100644 index 00000000..15fcb990 --- /dev/null +++ b/mooncake-transfer-engine/include/transport/device/ibgda_device.cuh @@ -0,0 +1,206 @@ +// IBGDA device context and operations. +// +// Wraps mlx5gda_qp_devctx and issues RDMA writes/atomics via +// device-side WQE construction. +// +// On MUSA: BF (Blue Flame) doorbell is not available (musaHostRegisterIoMemory +// doesn't support MMIO), so qp->bf is NULL and the kernel uses DBR-only mode. +// All other IBGDA logic (WQE construction, CQ polling, DBR write) is shared. +#pragma once + +#include +#include "transport/device/device_ops.cuh" + +#ifndef MOONCAKE_EP_USE_MUSA +#include +#endif +#include + +// mlx5 32-bit atomic-add WQE segment (not in mlx5gda.h; defined here for use +// in mc_ibgda_write_rdma_atomic_add_wqe). +struct mlx5_wqe_atomic_add_32_seg { + __be32 add_data; + __be32 field_boundary; + __be64 compare; +}; + +namespace mooncake { +namespace device { + +// --------------------------------------------------------------------------- +// IbgdaContext +// --------------------------------------------------------------------------- + +struct IbgdaContext { + mlx5gda_qp_devctx* qp_devctxs; // device ptr: [num_qps] + const uint64_t* raddrs; // device ptr: [num_ranks] remote GDR base + const uint32_t* rkeys; // device ptr: [num_ranks] remote rkey + const void* local_atomic_base; // local scratch base for atomic responses + const void* remote_atomic_base; // symmetric remote signal base +}; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +__device__ __forceinline__ mlx5gda_qp_devctx* mc_ibgda_channel( + const IbgdaContext& ctx, int channel, int dst_rank, int qps_per_rank) { + int qp_idx = dst_rank * qps_per_rank + (channel % qps_per_rank); + return ctx.qp_devctxs + qp_idx; +} + +__device__ __forceinline__ void mc_ibgda_lock(mlx5gda_qp_devctx* qp) { +#ifdef MOONCAKE_EP_USE_MUSA + uint32_t old; + do { + old = atomicCAS(&qp->mutex, 0u, 1u); + } while (old != 0); +#else + cuda::atomic_ref lock(qp->mutex); + while (lock.exchange(1u, cuda::memory_order_acquire) != 0u); +#endif +} + +__device__ __forceinline__ void mc_ibgda_unlock(mlx5gda_qp_devctx* qp) { +#ifdef MOONCAKE_EP_USE_MUSA + mc_st_release_u32(&qp->mutex, 0u); +#else + cuda::atomic_ref lock(qp->mutex); + lock.store(0u, cuda::memory_order_release); +#endif +} + +__device__ __forceinline__ void mc_ibgda_poll_cq(mlx5gda_qp_devctx* qp, + uint16_t expect) { + uint16_t wq_tail = qp->wq_tail; + while (static_cast(wq_tail - expect) <= 0) { + uint16_t cq_be = + *reinterpret_cast(&qp->cq->wqe_counter); + uint8_t opcode = qp->cq->op_own >> 4; + if (opcode == 0xD) + printf("[EP IBGDA] Requester error: syndrome=0x%lx\n", + qp->cq->timestamp >> 56); + if (!(opcode == 0x0 || opcode == 0xF)) { + printf("[EP IBGDA] Unexpected CQE opcode=0x%x, trapping\n", opcode); + __trap(); + } + wq_tail = mc_bswap16(cq_be) + 1; + } + if (wq_tail != qp->wq_tail) qp->wq_tail = wq_tail; +} + +__device__ __forceinline__ void mc_ibgda_post_send_db(mlx5gda_qp_devctx* qp) { + uint32_t num_posted = static_cast(qp->wq_head); + // DBR write — always done (NIC polls doorbell record in GPU memory) + mc_st_release_u32(reinterpret_cast(&qp->dbr->send_counter), + mc_bswap32(num_posted)); + // BF (Blue Flame) doorbell — only if BF register is mapped into GPU VA. + // On MUSA, musaHostRegisterIoMemory fails for MMIO addresses, so bf is + // NULL and we rely on DBR-only mode (slightly higher latency). + if (qp->bf != nullptr) { + auto* last_wqe = qp->wq + ((num_posted - 1) & qp->wqeid_mask); + mc_st_release_u64(reinterpret_cast(qp->bf + qp->bf_offset), + *reinterpret_cast(last_wqe)); + qp->bf_offset ^= MLX5GDA_BF_SIZE; + } +} + +// Issue an RDMA WRITE WQE. laddr/raddr are device VAs; keys are big-endian. +__device__ __forceinline__ void mc_ibgda_write_rdma_write_wqe( + mlx5gda_qp_devctx* qp, uint64_t laddr, __be32 lkey, uint64_t raddr, + __be32 rkey, uint32_t bytes) { + auto* wqe = reinterpret_cast( + qp->wq + (qp->wq_head & qp->wqeid_mask)); + + wqe->ctrl = {}; + wqe->ctrl.qpn_ds = mc_bswap32((qp->qpn << 8) | 3); + wqe->ctrl.fm_ce_se = MLX5_WQE_CTRL_CQ_UPDATE; + wqe->ctrl.opmod_idx_opcode = mc_bswap32( + (static_cast(qp->wq_head) << 8) | MLX5_OPCODE_RDMA_WRITE); + + wqe->raddr.raddr = mc_bswap64(raddr); + wqe->raddr.rkey = rkey; + wqe->raddr.reserved = 0; + + wqe->data.byte_count = mc_bswap32(bytes); + wqe->data.lkey = lkey; + wqe->data.addr = mc_bswap64(laddr); + + ++qp->wq_head; +} + +// Issue an RDMA ATOMIC MASKED FETCH-AND-ADD WQE (32-bit add_data). +// This matches the original CUDA IBGDA EP kernel. A regular 64-bit +// MLX5_OPCODE_ATOMIC_FA does not implement the 32-bit signal-buffer add used +// by dispatch/combine. +__device__ __forceinline__ void mc_ibgda_write_rdma_atomic_add_wqe( + mlx5gda_qp_devctx* qp, int32_t value, uint64_t laddr, __be32 lkey, + uint64_t raddr, __be32 rkey) { + auto* wqe = reinterpret_cast( + qp->wq + (qp->wq_head & qp->wqeid_mask)); + + wqe->ctrl = {}; + wqe->ctrl.qpn_ds = mc_bswap32((qp->qpn << 8) | 4); + wqe->ctrl.fm_ce_se = MLX5_WQE_CTRL_CQ_UPDATE; + wqe->ctrl.opmod_idx_opcode = + mc_bswap32(MLX5_OPCODE_ATOMIC_MASKED_FA | + (static_cast(qp->wq_head) << 8) | 0x08000000); + + wqe->raddr.raddr = mc_bswap64(raddr); + wqe->raddr.rkey = rkey; + wqe->raddr.reserved = 0; + + // atomic_seg: add_data (32-bit, big-endian), field_boundary=0, compare=0 + auto* atomic_seg = + reinterpret_cast(&wqe->atomic); + atomic_seg->add_data = mc_bswap32(static_cast(value)); + atomic_seg->field_boundary = 0; + atomic_seg->compare = 0; + + wqe->data.byte_count = mc_bswap32(static_cast(4)); + wqe->data.lkey = lkey; + wqe->data.addr = mc_bswap64(laddr); + + ++qp->wq_head; +} + +// --------------------------------------------------------------------------- +// High-level IBGDA operations +// --------------------------------------------------------------------------- + +// RDMA WRITE: send `nbytes` from `send_ptr` to `recv_ptr` on `dst_rank`. +// Must be called by lane 0 only. +__device__ __forceinline__ void mc_ibgda_put(const IbgdaContext& ctx, + int channel, int dst_rank, + int src_rank, int qps_per_rank, + const void* send_ptr, + uint64_t recv_raddr, + uint32_t nbytes) { + auto* qp = mc_ibgda_channel(ctx, channel, dst_rank, qps_per_rank); + mc_ibgda_lock(qp); + mc_ibgda_write_rdma_write_wqe(qp, reinterpret_cast(send_ptr), + mc_bswap32(ctx.rkeys[src_rank]), recv_raddr, + mc_bswap32(ctx.rkeys[dst_rank]), nbytes); + mc_ibgda_post_send_db(qp); + mc_ibgda_unlock(qp); +} + +// RDMA ATOMIC ADD: add `value` to the 32-bit word at `recv_raddr` on +// `dst_rank`. Must be called by lane 0 only. +__device__ __forceinline__ void mc_ibgda_red_add( + const IbgdaContext& ctx, int channel, int dst_rank, int src_rank, + int qps_per_rank, + uint64_t laddr, // local scratch VA for the atomic result + uint64_t recv_raddr, // remote VA of the signal word + int32_t value) { + auto* qp = mc_ibgda_channel(ctx, channel, dst_rank, qps_per_rank); + mc_ibgda_lock(qp); + mc_ibgda_write_rdma_atomic_add_wqe( + qp, value, laddr, mc_bswap32(ctx.rkeys[src_rank]), recv_raddr, + mc_bswap32(ctx.rkeys[dst_rank])); + mc_ibgda_post_send_db(qp); + mc_ibgda_unlock(qp); +} + +} // namespace device +} // namespace mooncake diff --git a/mooncake-transfer-engine/include/transport/device/musa/musa_ops.cuh b/mooncake-transfer-engine/include/transport/device/musa/musa_ops.cuh new file mode 100644 index 00000000..9d4ed201 --- /dev/null +++ b/mooncake-transfer-engine/include/transport/device/musa/musa_ops.cuh @@ -0,0 +1,137 @@ +// MUSA implementations of device-side memory ordering primitives. +// Included by device_ops.cuh when MOONCAKE_EP_USE_MUSA is defined. +// +// MUSA has no PTX-style acquire/release instructions. We emulate them with +// plain loads/stores + __threadfence_system() for cross-GPU (MTLink) +// visibility. +// +// Known MUSA SDK 4.3.3 compiler bugs to avoid: +// - atomicAdd_system / atomicCAS_system → infinite SelectionDAG loop. +// Use block-scope atomicAdd + __threadfence_system() instead. +// - Named barriers (bar.sync) → not available; use __syncthreads(). +// - cooperative_groups::this_grid().sync() → not available; host uses +// separate kernel launches (return_recv_hook=true) so grid sync is a no-op. +#pragma once + +#include + +namespace mooncake { +namespace device { + +// --------------------------------------------------------------------------- +// Acquire loads +// --------------------------------------------------------------------------- +__device__ __forceinline__ int mc_ld_acquire(const int* ptr) { + int ret = *const_cast(ptr); + __threadfence_system(); + return ret; +} + +__device__ __forceinline__ uint64_t mc_ld_acquire_u64(const uint64_t* ptr) { + uint64_t ret = *const_cast(ptr); + __threadfence_system(); + return ret; +} + +// --------------------------------------------------------------------------- +// Release stores +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_st_release(const int* ptr, int val) { + __threadfence_system(); + *const_cast(ptr) = val; +} + +__device__ __forceinline__ void mc_st_release_u32(const uint32_t* ptr, + uint32_t val) { + __threadfence_system(); + *const_cast(ptr) = val; +} + +__device__ __forceinline__ void mc_st_release_u64(const uint64_t* ptr, + uint64_t val) { + __threadfence_system(); + *const_cast(ptr) = val; +} + +// --------------------------------------------------------------------------- +// Atomic add — block-scope atomicAdd + system fence (avoids SDK bug) +// --------------------------------------------------------------------------- +__device__ __forceinline__ int mc_atomic_add_release(const int* ptr, int val) { + __threadfence_system(); + return atomicAdd(const_cast(ptr), val); +} + +// --------------------------------------------------------------------------- +// Non-coherent loads — MUSA has no nc/no_allocate cache hints; use volatile. +// int4 volatile copy not supported by MUSA compiler; copy field-by-field. +// --------------------------------------------------------------------------- +__device__ __forceinline__ int4 mc_ld_nc(const int4* ptr) { + const volatile int* vp = reinterpret_cast(ptr); + int4 ret; + ret.x = vp[0]; + ret.y = vp[1]; + ret.z = vp[2]; + ret.w = vp[3]; + return ret; +} + +__device__ __forceinline__ int mc_ld_nc_s32(const int* ptr) { + return *const_cast(ptr); +} + +__device__ __forceinline__ float mc_ld_nc_f32(const float* ptr) { + return *const_cast(ptr); +} + +__device__ __forceinline__ int64_t mc_ld_nc_s64(const int64_t* ptr) { + return *const_cast(ptr); +} + +// --------------------------------------------------------------------------- +// Non-temporal stores — MUSA has no nt/no_allocate hints; plain store. +// int4 volatile assignment not supported by MUSA compiler; copy field-by-field. +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_st_na(const int4* ptr, const int4& val) { + volatile int* vp = reinterpret_cast(const_cast(ptr)); + vp[0] = val.x; + vp[1] = val.y; + vp[2] = val.z; + vp[3] = val.w; +} + +// --------------------------------------------------------------------------- +// Named barrier — MUSA has no bar.sync. +// On MUSA, mc_bar_sync is implemented as __syncthreads() (full CTA barrier). +// Kernels that call mc_bar_sync from a subset of threads must ensure the +// remaining threads call it the same number of times from another code path. +// See mooncake_ep_kernel.cu for the warp-31 dispatch-send workaround. +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_bar_sync(int /*bar_id*/, + int /*num_threads*/) { + __syncthreads(); +} + +// --------------------------------------------------------------------------- +// Grid sync — not available on MUSA. Host always uses separate kernel +// launches, so SEND and RECV never share a kernel invocation. No-op. +// --------------------------------------------------------------------------- +__device__ __forceinline__ void mc_grid_sync() {} + +// --------------------------------------------------------------------------- +// Byte-swap helpers — MUSA has no __byte_perm; implement manually. +// --------------------------------------------------------------------------- +__device__ __forceinline__ uint16_t mc_bswap16(uint16_t x) { + return (uint16_t)(((x & 0x00FFu) << 8) | ((x & 0xFF00u) >> 8)); +} +__device__ __forceinline__ uint32_t mc_bswap32(uint32_t x) { + return ((x & 0x000000FFu) << 24) | ((x & 0x0000FF00u) << 8) | + ((x & 0x00FF0000u) >> 8) | ((x & 0xFF000000u) >> 24); +} +__device__ __forceinline__ uint64_t mc_bswap64(uint64_t x) { + uint32_t hi = mc_bswap32((uint32_t)(x >> 32)); + uint32_t lo = mc_bswap32((uint32_t)(x)); + return ((uint64_t)lo << 32) | hi; +} + +} // namespace device +} // namespace mooncake diff --git a/mooncake-transfer-engine/include/transport/device/p2p_device.cuh b/mooncake-transfer-engine/include/transport/device/p2p_device.cuh new file mode 100644 index 00000000..d47ca0a2 --- /dev/null +++ b/mooncake-transfer-engine/include/transport/device/p2p_device.cuh @@ -0,0 +1,46 @@ +// P2P device context and operations — unified NVLink (CUDA) / MTLink (MUSA). +// +// P2PContext holds the device-visible tables populated by P2pTransport on +// the host side. The kernel constructs one from the raw pointers passed in +// and calls mc_p2p_* helpers instead of accessing the tables directly. +#pragma once + +#include +#include "transport/device/device_ops.cuh" + +namespace mooncake { +namespace device { + +struct P2PContext { + const int32_t* available; // device ptr: [num_ranks], 1 = P2P reachable + void* const* peer_ptrs; // device ptr: [num_ranks], peer GDR base ptrs + void* local_base; // this rank's GDR buffer base (for offset math) +}; + +__device__ __forceinline__ bool mc_p2p_available(const P2PContext& ctx, + int dst_rank) { + return ctx.available[dst_rank] != 0 && ctx.peer_ptrs[dst_rank] != nullptr; +} + +// Translate a local pointer (within the GDR buffer) to the peer's mapped VA. +__device__ __forceinline__ void* mc_p2p_peer_ptr(const P2PContext& ctx, + int dst_rank, + const void* local_ptr) { + const auto offset = reinterpret_cast(local_ptr) - + reinterpret_cast(ctx.local_base); + return reinterpret_cast(ctx.peer_ptrs[dst_rank]) + offset; +} + +// Write a 32-bit signal value to the peer's signal slot via P2P store. +// Single-writer assumption: uses release store, not atomic add. +__device__ __forceinline__ void mc_p2p_signal(const P2PContext& ctx, + int dst_rank, + const int* local_sig_ptr, + int32_t val) { + auto* peer_sig = + reinterpret_cast(mc_p2p_peer_ptr(ctx, dst_rank, local_sig_ptr)); + mc_st_release(peer_sig, val); +} + +} // namespace device +} // namespace mooncake diff --git a/mooncake-transfer-engine/src/CMakeLists.txt b/mooncake-transfer-engine/src/CMakeLists.txt index 336a1fa6..1bd77e1d 100644 --- a/mooncake-transfer-engine/src/CMakeLists.txt +++ b/mooncake-transfer-engine/src/CMakeLists.txt @@ -64,7 +64,7 @@ endif() if(USE_CUDA) target_include_directories(transfer_engine PRIVATE /usr/local/cuda/include) - target_link_libraries(transfer_engine PUBLIC cuda cudart rt) + target_link_libraries(transfer_engine PUBLIC cuda cudart rt mlx5) if(USE_NVMEOF) target_link_libraries(transfer_engine PUBLIC nvmeof_transport cufile) endif() diff --git a/mooncake-transfer-engine/src/transfer_engine.cpp b/mooncake-transfer-engine/src/transfer_engine.cpp index 4800c005..dbb2eff3 100644 --- a/mooncake-transfer-engine/src/transfer_engine.cpp +++ b/mooncake-transfer-engine/src/transfer_engine.cpp @@ -179,6 +179,17 @@ Transport* TransferEngine::getTransport(const std::string& proto) { return impl_->getTransport(proto); } +#if defined(USE_CUDA) || defined(USE_MUSA) +device::P2pTransport* TransferEngine::getOrCreateP2pTransport(int num_ranks) { + return impl_->getOrCreateP2pTransport(num_ranks); +} + +device::RdmaTransport* TransferEngine::getOrCreateRdmaTransport( + const std::vector& device_filter) { + return impl_->getOrCreateRdmaTransport(device_filter); +} +#endif + bool TransferEngine::isTcpOnly() const { return impl_->isTcpOnly(); } int TransferEngine::syncSegmentCache(const std::string& segment_name) { @@ -580,6 +591,19 @@ Transport* TransferEngine::getTransport(const std::string& proto) { return impl_->getTransport(proto); } +#if defined(USE_CUDA) || defined(USE_MUSA) +device::P2pTransport* TransferEngine::getOrCreateP2pTransport(int num_ranks) { + if (use_tent_) return nullptr; + return impl_->getOrCreateP2pTransport(num_ranks); +} + +device::RdmaTransport* TransferEngine::getOrCreateRdmaTransport( + const std::vector& device_filter) { + if (use_tent_) return nullptr; + return impl_->getOrCreateRdmaTransport(device_filter); +} +#endif + bool TransferEngine::isTcpOnly() const { if (use_tent_) // TENT already rejects TCP loopback transfers when MC_STORE_MEMCPY diff --git a/mooncake-transfer-engine/src/transfer_engine_impl.cpp b/mooncake-transfer-engine/src/transfer_engine_impl.cpp index bbcfb8dd..08818bba 100644 --- a/mooncake-transfer-engine/src/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/src/transfer_engine_impl.cpp @@ -413,6 +413,24 @@ int TransferEngineImpl::uninstallTransport(const std::string& proto) { return 0; } +#if defined(USE_CUDA) || defined(USE_MUSA) +device::P2pTransport* TransferEngineImpl::getOrCreateP2pTransport( + int num_ranks) { + if (!p2p_transport_) { + p2p_transport_ = device::createP2pDeviceTransport(num_ranks); + } + return p2p_transport_.get(); +} + +device::RdmaTransport* TransferEngineImpl::getOrCreateRdmaTransport( + const std::vector& device_filter) { + if (!rdma_transport_) { + rdma_transport_ = device::createIbgdaDeviceTransport(device_filter); + } + return rdma_transport_.get(); +} +#endif + int TransferEngineImpl::getRpcPort() { return metadata_->localRpcMeta().rpc_port; } diff --git a/mooncake-transfer-engine/src/transport/CMakeLists.txt b/mooncake-transfer-engine/src/transport/CMakeLists.txt index 04846c25..7dcaedf7 100644 --- a/mooncake-transfer-engine/src/transport/CMakeLists.txt +++ b/mooncake-transfer-engine/src/transport/CMakeLists.txt @@ -72,3 +72,11 @@ if (USE_EFA) target_sources(transport PUBLIC $) target_link_libraries(transport PRIVATE fabric) endif() + +if(USE_CUDA OR USE_MUSA) + add_subdirectory(device) + target_sources(transport PUBLIC $) + # device_transport (ibgda_device_transport.cpp / mlx5gda.cpp) calls libmlx5 + # DevX symbols (mlx5dv_devx_*, mlx5dv_init_obj) directly. + target_link_libraries(transport PUBLIC mlx5) +endif() diff --git a/mooncake-transfer-engine/src/transport/device/CMakeLists.txt b/mooncake-transfer-engine/src/transport/device/CMakeLists.txt new file mode 100644 index 00000000..20ec48c8 --- /dev/null +++ b/mooncake-transfer-engine/src/transport/device/CMakeLists.txt @@ -0,0 +1,22 @@ +# Device transport sources — conditional on GPU vendor. +# mlx5gda.cpp (IBGDA QP lifecycle) is compiled directly into the +# device_transport OBJECT library, like every other transport module, so its +# mlx5gda_* symbols flow into transfer_engine without a separate static lib. +# This keeps them visible to consumers that link transfer_engine via hand- +# written ldflags (Go p2p-store / mooncake-store), which bypass CMake's +# target_link_libraries propagation. +set(DEVICE_TRANSPORT_SOURCES p2p_device_transport.cpp) +if(USE_CUDA OR USE_MUSA) + list(APPEND DEVICE_TRANSPORT_SOURCES ibgda_device_transport.cpp mlx5gda.cpp) +endif() + +add_library(device_transport OBJECT ${DEVICE_TRANSPORT_SOURCES}) +target_include_directories(device_transport PRIVATE ${CMAKE_SOURCE_DIR}/include) + +if(USE_CUDA) + target_include_directories(device_transport PRIVATE /usr/local/cuda/include) +endif() +if(USE_MUSA) + target_include_directories(device_transport PRIVATE /usr/local/musa/include) + target_compile_definitions(device_transport PRIVATE USE_MUSA) +endif() diff --git a/mooncake-transfer-engine/src/transport/device/ibgda_device_transport.cpp b/mooncake-transfer-engine/src/transport/device/ibgda_device_transport.cpp new file mode 100644 index 00000000..2dcdac17 --- /dev/null +++ b/mooncake-transfer-engine/src/transport/device/ibgda_device_transport.cpp @@ -0,0 +1,430 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// IBGDA device transport — wraps mlx5gda QP lifecycle behind RdmaTransport. +// +// Uses TE's Topology for NIC auto-detection when device_name is empty. +// Uses ibv_open_device / ibv_alloc_pd directly (same as the original +// MooncakeEpBuffer::init_ibgda()), but encapsulated in a reusable class. + +#include "transport/device/device_transport.h" + +#include +#include +#include +#include + +#include +#include + +#include "cuda_alike.h" +#include "transport/device/ibgda/memheap.h" +#include "transport/device/ibgda/mlx5gda.h" +#include "topology.h" + +namespace mooncake { +namespace device { + +static constexpr size_t kCtrlBufSize = 1024ULL * 1024 * 1024; // 1 GiB + +// Check if IPv6 address is IPv4-mapped (::ffff:x.x.x.x) +static bool isIpv4Mapped(const struct in6_addr* a) { + return ((a->s6_addr32[0] | a->s6_addr32[1]) == 0 && + a->s6_addr32[2] == htonl(0x0000ffff)); +} + +static int findBestGidIndex(ibv_context* ctx, uint8_t port, + const ibv_port_attr& port_attr) { + for (int i = 0; i < port_attr.gid_tbl_len; ++i) { + ibv_gid_entry entry; + if (ibv_query_gid_ex(ctx, port, i, &entry, 0)) continue; + + if (entry.gid_type == IBV_GID_TYPE_ROCE_V2) { + bool v4mapped = isIpv4Mapped( + reinterpret_cast(entry.gid.raw)); + if (v4mapped) return i; + } else if (entry.gid_type == IBV_GID_TYPE_IB) { + return i; + } + } + + return -1; +} + +// Auto-detect the best NIC for the current GPU using TE's Topology. +// filter: if non-empty, only consider NICs in this list. +static std::string autoDetectNic(const std::vector& filter) { + Topology topo; + if (topo.discover(filter) != 0) return ""; + const auto& hca_list = topo.getHcaList(); + if (hca_list.empty()) return ""; + + // Build a location string for the current GPU so Topology picks the + // topologically closest NIC. Fall back to wildcard if cudaGetDevice fails. + int device_id = 0; + cudaGetDevice(&device_id); + std::string location = "cuda:" + std::to_string(device_id); + + int idx = topo.selectDevice(location); + if (idx < 0) idx = topo.selectDevice("*"); // wildcard fallback + if (idx < 0 || idx >= static_cast(hca_list.size())) return ""; + return hca_list[idx]; +} + +class IbgdaDeviceTransportImpl : public RdmaTransport { + public: + explicit IbgdaDeviceTransportImpl(std::vector filter) + : device_filter_(std::move(filter)) {} + + ~IbgdaDeviceTransportImpl() override { teardown(); } + + int initialize(const std::string& device_name, int num_ranks, + int num_qps) override { + if (num_qps < num_ranks) { + LOG(ERROR) << "[EP IBGDA] num_qps (" << num_qps + << ") must be >= num_ranks (" << num_ranks << ")"; + return -1; + } + num_ranks_ = num_ranks; + num_qps_ = num_qps; + + std::string nic = + device_name.empty() ? autoDetectNic(device_filter_) : device_name; + if (nic.empty()) { + LOG(WARNING) << "[EP IBGDA] No RDMA NIC found"; + return -1; + } + + int num_devices = 0; + ibv_device** dev_list = ibv_get_device_list(&num_devices); + if (!dev_list) { + LOG(ERROR) << "[EP IBGDA] ibv_get_device_list failed"; + return -1; + } + + ibv_device* dev = nullptr; + for (int i = 0; i < num_devices; ++i) { + if (nic == ibv_get_device_name(dev_list[i])) { + dev = dev_list[i]; + break; + } + } + if (!dev) { + LOG(ERROR) << "[EP IBGDA] NIC '" << nic << "' not found"; + ibv_free_device_list(dev_list); + return -1; + } + + ctx_ = ibv_open_device(dev); + ibv_free_device_list(dev_list); + if (!ctx_) { + LOG(ERROR) << "[EP IBGDA] ibv_open_device failed for " << nic; + return -1; + } + + const uint8_t port = 1; + ibv_port_attr port_attr{}; + if (ibv_query_port(ctx_, port, &port_attr)) { + LOG(ERROR) << "[EP IBGDA] ibv_query_port failed"; + return -1; + } + + gid_index_ = findBestGidIndex(ctx_, port, port_attr); + if (gid_index_ < 0) { + LOG(ERROR) << "[EP IBGDA] No suitable GID on " << nic; + return -1; + } + + if (ibv_query_gid(ctx_, port, gid_index_, &gid_)) { + LOG(ERROR) << "[EP IBGDA] ibv_query_gid failed"; + return -1; + } + + is_roce_ = (port_attr.link_layer == IBV_LINK_LAYER_ETHERNET); + lid_ = port_attr.lid; + device_name_ = nic; + + pd_ = ibv_alloc_pd(ctx_); + if (!pd_) { + LOG(ERROR) << "[EP IBGDA] ibv_alloc_pd failed"; + return -1; + } + + mlx5dv_obj dv_obj{}; + dv_obj.pd.in = pd_; + dv_obj.pd.out = &mpd_; + if (mlx5dv_init_obj(&dv_obj, MLX5DV_OBJ_PD)) { + LOG(ERROR) << "[EP IBGDA] mlx5dv_init_obj failed"; + return -1; + } + + // Allocate device-visible tables + if (cudaMalloc(&raddrs_, num_ranks_ * sizeof(uint64_t)) != + cudaSuccess || + cudaMalloc(&rkeys_, num_ranks_ * sizeof(uint32_t)) != cudaSuccess || + cudaMalloc(&qp_devctxs_, num_qps_ * sizeof(mlx5gda_qp_devctx)) != + cudaSuccess) { + LOG(ERROR) << "[EP IBGDA] Failed to allocate device-visible tables"; + return -1; + } + + LOG(INFO) << "[EP IBGDA] Initialized on " << nic + << " (gid_index=" << gid_index_ << ", roce=" << is_roce_ + << ")"; + return 0; + } + + int registerMemory(void* ptr, size_t bytes) override { + mr_ = + ibv_reg_mr(pd_, ptr, bytes, + IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_READ | + IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_ATOMIC); + if (!mr_) { + LOG(ERROR) << "[EP IBGDA] ibv_reg_mr failed"; + return -1; + } + mr_ptr_ = ptr; + return 0; + } + + int allocateControlBuffer() override { + cudaError_t err = cudaMalloc(&ctrl_buf_, kCtrlBufSize); + if (err != cudaSuccess) { + LOG(ERROR) << "[EP IBGDA] cudaMalloc ctrl_buf failed: " + << cudaGetErrorString(err); + return -1; + } + + ctrl_buf_umem_ = mlx5dv_devx_umem_reg(ctx_, ctrl_buf_, kCtrlBufSize, + IBV_ACCESS_LOCAL_WRITE); + if (!ctrl_buf_umem_) { + LOG(ERROR) << "[EP IBGDA] mlx5dv_devx_umem_reg failed (errno=" + << errno << ")"; + return -1; + } + LOG(INFO) << "[EP IBGDA] ctrl_buf UMEM registered via VA path"; + + ctrl_buf_heap_ = memheap_create(kCtrlBufSize); + if (!ctrl_buf_heap_) { + LOG(ERROR) << "[EP IBGDA] memheap_create failed"; + return -1; + } + return 0; + } + + int createQueuePairs(void* stream_ptr) override { + auto stream = static_cast(stream_ptr); + for (int i = 0; i < num_qps_; ++i) { + mlx5gda_qp* qp = + mlx5gda_create_rc_qp(mpd_, ctrl_buf_, ctrl_buf_umem_, + ctrl_buf_heap_, pd_, 16384, 1, stream); + if (!qp) { + LOG(ERROR) << "[EP IBGDA] mlx5gda_create_rc_qp failed at " << i; + return -1; + } + if (mlx5gda_modify_rc_qp_rst2init(qp, 0)) { + LOG(ERROR) << "[EP IBGDA] rst2init failed at " << i; + mlx5gda_destroy_qp(ctrl_buf_heap_, qp); + return -1; + } + cudaStreamSynchronize(stream); + mlx5gda_qp_devctx devctx{ + .qpn = qp->qpn, + .wqeid_mask = qp->num_wqebb - 1, + .wq = reinterpret_cast( + static_cast(ctrl_buf_) + qp->wq_offset), + .cq = reinterpret_cast( + static_cast(ctrl_buf_) + qp->send_cq->cq_offset), + .dbr = reinterpret_cast( + static_cast(ctrl_buf_) + qp->dbr_offset), + .bf = static_cast(qp->uar->reg_addr), + }; + cudaMemcpy( + static_cast(qp_devctxs_) + i * sizeof(mlx5gda_qp_devctx), + &devctx, sizeof(mlx5gda_qp_devctx), cudaMemcpyHostToDevice); + qps_.push_back(qp); + } + return 0; + } + + int recreateQueuePairs(void* stream_ptr) override { + auto stream = static_cast(stream_ptr); + for (auto* qp : qps_) { + if (qp) mlx5gda_destroy_qp(ctrl_buf_heap_, qp); + } + qps_.clear(); + return createQueuePairs(stream_ptr); + } + + int connectPeers(int local_rank, bool is_roce, + const std::vector& remote_addrs, + const std::vector& remote_keys, + const std::vector& remote_qpns, + const std::vector& remote_lids, + const std::vector& subnet_prefixes, + const std::vector& interface_ids, + const std::vector& active_ranks_mask) override { + for (int i = 0; i < num_qps_; ++i) { + int peer_rank = i * num_ranks_ / num_qps_; + if (active_ranks_mask[peer_rank] == 0) continue; + + ibv_ah_attr ah_attr{}; + if (is_roce) { + ibv_gid remote_gid{}; + remote_gid.global.subnet_prefix = subnet_prefixes[peer_rank]; + remote_gid.global.interface_id = interface_ids[peer_rank]; + ah_attr.is_global = 1; + ah_attr.grh.dgid = remote_gid; + ah_attr.grh.sgid_index = gid_index_; + // Match the legacy IBGDA path. mlx5gda previously hard-coded + // QPC hop_limit=255 and ignored this field; after moving QP + // setup into the device transport this value reaches hardware. + ah_attr.grh.hop_limit = 255; + ah_attr.port_num = 1; + ah_attr.dlid = qps_[i]->port_attr.lid | 0xC000; + } else { + ah_attr.dlid = static_cast(remote_lids[i]); + ah_attr.port_num = 0; + } + + if (mlx5gda_modify_rc_qp_init2rtr(qps_[i], ah_attr, remote_qpns[i], + IBV_MTU_4096)) { + LOG(ERROR) << "[EP IBGDA] init2rtr failed for QP " << i + << " (roce=" << is_roce << " gid_idx=" << gid_index_ + << " remote_qpn=" << remote_qpns[i] + << " udp_sport=" << ah_attr.dlid + << " hop_limit=" << (int)ah_attr.grh.hop_limit + << ")"; + return -1; + } + if (mlx5gda_modify_rc_qp_rtr2rts(qps_[i])) { + LOG(ERROR) << "[EP IBGDA] rtr2rts failed for QP " << i; + return -1; + } + } + + // Populate device-visible raddrs/rkeys tables + for (int i = 0; i < num_ranks_; ++i) { + if (active_ranks_mask[i] == 0) continue; + uint64_t raddr = static_cast(remote_addrs[i]); + uint32_t rkey = (i == local_rank) + ? static_cast(mr_->lkey) + : static_cast(remote_keys[i]); + cudaMemcpy(static_cast(raddrs_) + i * sizeof(uint64_t), + &raddr, sizeof(uint64_t), cudaMemcpyHostToDevice); + cudaMemcpy(static_cast(rkeys_) + i * sizeof(uint32_t), &rkey, + sizeof(uint32_t), cudaMemcpyHostToDevice); + } + return 0; + } + + RdmaLocalMetadata localMetadata() const override { + RdmaLocalMetadata meta; + meta.raddr = mr_ ? reinterpret_cast(mr_->addr) : 0; + meta.rkey = mr_ ? static_cast(mr_->rkey) : 0; + meta.subnet_prefix = static_cast(gid_.global.subnet_prefix); + meta.interface_id = static_cast(gid_.global.interface_id); + for (auto* qp : qps_) { + meta.qpns.push_back(static_cast(qp->qpn)); + meta.lids.push_back(static_cast(lid_)); + } + return meta; + } + + void* raddrsPtr() override { return raddrs_; } + void* rkeysPtr() override { return rkeys_; } + void* qpDevCtxsPtr() override { return qp_devctxs_; } + bool isRoce() const override { return is_roce_; } + int gidIndex() const override { return gid_index_; } + + private: + void teardown() { + for (auto* qp : qps_) { + if (qp) mlx5gda_destroy_qp(ctrl_buf_heap_, qp); + } + qps_.clear(); + if (ctrl_buf_heap_) { + memheap_destroy(ctrl_buf_heap_); + ctrl_buf_heap_ = nullptr; + } + if (ctrl_buf_umem_) { + mlx5dv_devx_umem_dereg(ctrl_buf_umem_); + ctrl_buf_umem_ = nullptr; + } + if (ctrl_buf_) { + cudaFree(ctrl_buf_); + ctrl_buf_ = nullptr; + } + if (mr_) { + ibv_dereg_mr(mr_); + mr_ = nullptr; + } + if (raddrs_) { + cudaFree(raddrs_); + raddrs_ = nullptr; + } + if (rkeys_) { + cudaFree(rkeys_); + rkeys_ = nullptr; + } + if (qp_devctxs_) { + cudaFree(qp_devctxs_); + qp_devctxs_ = nullptr; + } + if (pd_) { + ibv_dealloc_pd(pd_); + pd_ = nullptr; + } + if (ctx_) { + ibv_close_device(ctx_); + ctx_ = nullptr; + } + } + + // IB resources + ibv_context* ctx_ = nullptr; + ibv_pd* pd_ = nullptr; + mlx5dv_pd mpd_{}; + ibv_mr* mr_ = nullptr; + void* mr_ptr_ = nullptr; + ibv_gid gid_{}; + int gid_index_ = -1; + uint16_t lid_ = 0; + bool is_roce_ = false; + std::string device_name_; + std::vector device_filter_; + + // Control buffer + void* ctrl_buf_ = nullptr; // GPU VA + mlx5dv_devx_umem* ctrl_buf_umem_ = nullptr; + memheap* ctrl_buf_heap_ = nullptr; + + // QPs + std::vector qps_; + int num_ranks_ = 0; + int num_qps_ = 0; + + // Device-visible tables + void* raddrs_ = nullptr; + void* rkeys_ = nullptr; + void* qp_devctxs_ = nullptr; +}; + +std::unique_ptr createIbgdaDeviceTransport( + const std::vector& device_filter) { + return std::make_unique(device_filter); +} + +} // namespace device +} // namespace mooncake diff --git a/mooncake-ep/src/mooncake_ibgda/mlx5gda.cpp b/mooncake-transfer-engine/src/transport/device/mlx5gda.cpp similarity index 91% rename from mooncake-ep/src/mooncake_ibgda/mlx5gda.cpp rename to mooncake-transfer-engine/src/transport/device/mlx5gda.cpp index 96e8f939..ca601b25 100644 --- a/mooncake-ep/src/mooncake_ibgda/mlx5gda.cpp +++ b/mooncake-transfer-engine/src/transport/device/mlx5gda.cpp @@ -1,14 +1,14 @@ #include -#include +#include "cuda_alike.h" #include #include -#include -#include -#include -#include +#include +#include +#include +#include template inline T IBGDA_ILOG2(T _n) { @@ -30,10 +30,22 @@ constexpr T round_up_pow2(T n) { #define IBGDA_ROUND_UP_POW2_OR_0(_n) (((_n) == 0) ? 0 : round_up_pow2(_n)) static void print_cuda_error(const char* msg) { +#ifdef USE_MUSA + const char* err_str = musaGetErrorString(musaGetLastError()); +#else const char* err_str = cudaGetErrorString(cudaGetLastError()); +#endif fprintf(stderr, "%s: %s\n", msg, err_str); } +// Create UAR for BF (Blue Flame) doorbell ringing. +// On CUDA: registers the BF MMIO region into GPU address space so the +// GPU kernel can directly write the doorbell (lowest latency). +// On MUSA: musaHostRegisterIoMemory is not supported for MMIO addresses, +// so we skip BF registration and return a UAR with reg_addr=NULL. +// The GPU kernel will use DBR-only mode (write to memory-mapped +// doorbell record, NIC polls it) — slightly higher latency but +// functionally correct. static struct mlx5dv_devx_uar* create_uar(struct ibv_context* ctx) { struct mlx5dv_devx_uar* uar = mlx5dv_devx_alloc_uar(ctx, MLX5DV_UAR_ALLOC_TYPE_BF); @@ -41,6 +53,15 @@ static struct mlx5dv_devx_uar* create_uar(struct ibv_context* ctx) { errno = EIO; return NULL; } +#ifdef USE_MUSA + // MUSA cannot map MMIO addresses into GPU VA. Skip the + // musaHostRegister(IoMemory) call entirely — attempting it + // corrupts the MUSA runtime, causing all subsequent device-side + // fill operations to fail with "illegal memory access". + // Use DBR-only mode: the kernel writes to the doorbell record + // in GPU memory instead of the BF MMIO register. + uar->reg_addr = NULL; +#else if (cudaHostRegister(uar->reg_addr, MLX5GDA_BF_SIZE * 2, cudaHostRegisterPortable | cudaHostRegisterMapped | cudaHostRegisterIoMemory) != cudaSuccess) { @@ -49,13 +70,16 @@ static struct mlx5dv_devx_uar* create_uar(struct ibv_context* ctx) { mlx5dv_devx_free_uar(uar); return NULL; } +#endif return uar; } static void destroy_uar(struct mlx5dv_devx_uar* uar) { if (!uar) return; - if (cudaHostUnregister(uar->reg_addr) != cudaSuccess) { - print_cuda_error("Failed to unregister MMIO memory"); + if (uar->reg_addr) { + if (cudaHostUnregister(uar->reg_addr) != cudaSuccess) { + print_cuda_error("Failed to unregister MMIO memory"); + } } mlx5dv_devx_free_uar(uar); } @@ -444,7 +468,8 @@ int mlx5gda_modify_rc_qp_init2rtr(struct mlx5gda_qp* qp, memcpy(DEVX_ADDR_OF(qpc, qpc, primary_address_path.rmac_47_32), &dah.av->rmac, sizeof(dah.av->rmac)); - DEVX_SET(qpc, qpc, primary_address_path.hop_limit, 255); + DEVX_SET(qpc, qpc, primary_address_path.hop_limit, + ah_attr.grh.hop_limit); DEVX_SET(qpc, qpc, primary_address_path.src_addr_index, ah_attr.grh.sgid_index); DEVX_SET(qpc, qpc, primary_address_path.udp_sport, ah_attr.dlid); diff --git a/mooncake-transfer-engine/src/transport/device/p2p_device_transport.cpp b/mooncake-transfer-engine/src/transport/device/p2p_device_transport.cpp new file mode 100644 index 00000000..23e02255 --- /dev/null +++ b/mooncake-transfer-engine/src/transport/device/p2p_device_transport.cpp @@ -0,0 +1,268 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// P2P device transport — unified NVLink (CUDA) + MTLink (MUSA) implementation. +// +// Uses cuda_alike.h so all cuda* APIs map to musa* when USE_MUSA is defined. +// No #ifdef USE_MUSA / MOONCAKE_EP_USE_MUSA in this file. + +#include "transport/device/device_transport.h" + +#include +#include +#include + +#include "cuda_alike.h" + +namespace mooncake { +namespace device { + +class P2pDeviceTransportImpl : public P2pTransport { + public: + explicit P2pDeviceTransportImpl(int num_ranks) : num_ranks_(num_ranks) { + cudaMalloc(&available_table_, num_ranks_ * sizeof(int32_t)); + cudaMemset(available_table_, 0, num_ranks_ * sizeof(int32_t)); + cudaMallocHost(&peer_ptrs_host_, num_ranks_ * sizeof(void*)); + cudaMalloc(&peer_ptrs_dev_, num_ranks_ * sizeof(void*)); + for (int i = 0; i < num_ranks_; ++i) peer_ptrs_host_[i] = nullptr; + cudaMemset(peer_ptrs_dev_, 0, num_ranks_ * sizeof(void*)); + } + + ~P2pDeviceTransportImpl() override { + if (available_table_) cudaFree(available_table_); + if (peer_ptrs_dev_) cudaFree(peer_ptrs_dev_); + if (peer_ptrs_host_) { + for (int i = 0; i < num_ranks_; ++i) { + if (peer_ptrs_host_[i] && peer_ptrs_host_[i] != local_ptr_) { + cudaIpcCloseMemHandle(peer_ptrs_host_[i]); + } + } + cudaFreeHost(peer_ptrs_host_); + } + } + + void* allocateBuffer(size_t bytes) override { + void* ptr = nullptr; + cudaError_t err = cudaMalloc(&ptr, bytes); + if (err != cudaSuccess) { + LOG(ERROR) << "[EP P2P] cudaMalloc(" << bytes + << ") failed: " << cudaGetErrorString(err); + return nullptr; + } + return ptr; + } + + void freeBuffer(void* ptr) override { cudaFree(ptr); } + + std::vector exportIpcHandle(void* ptr) override { + cudaIpcMemHandle_t handle; + cudaError_t err = cudaIpcGetMemHandle(&handle, ptr); + if (err != cudaSuccess) { + LOG(ERROR) << "[EP P2P] cudaIpcGetMemHandle failed: " + << cudaGetErrorString(err); + return {}; + } + constexpr size_t kHandleBytes = sizeof(cudaIpcMemHandle_t); + constexpr size_t kNumInt32s = + (kHandleBytes + sizeof(int32_t) - 1) / sizeof(int32_t); + std::vector result(kNumInt32s); + memcpy(result.data(), &handle, kHandleBytes); + return result; + } + + void importPeerHandles( + void* local_ptr, int rank, int num_ranks, + const std::vector>& remote_handles, + const std::vector& active_ranks_mask) override { + local_ptr_ = local_ptr; + int device_id = 0; + cudaGetDevice(&device_id); + int device_count = 0; + cudaGetDeviceCount(&device_count); + CHECK_GT(device_count, 0) << "No CUDA/MUSA devices found"; + + std::vector available(num_ranks_, 0); + available[rank] = 1; + peer_ptrs_host_[rank] = local_ptr; + + int node_id = rank / device_count; + int group_start = node_id * device_count; + int group_end = std::min(group_start + device_count, num_ranks_); + + for (int dst = group_start; dst < group_end; ++dst) { + if (active_ranks_mask[dst] == 0) continue; + if (dst == rank) continue; + + int dst_device = dst % device_count; + int can_access = 0; + cudaDeviceCanAccessPeer(&can_access, device_id, dst_device); + LOG(INFO) << "[EP P2P] rank " << rank << " (device " << device_id + << ") -> rank " << dst << " (device " << dst_device + << "): canAccessPeer=" << can_access; + if (!can_access) continue; + + cudaError_t err = cudaDeviceEnablePeerAccess(dst_device, 0); + if (err != cudaSuccess && + err != cudaErrorPeerAccessAlreadyEnabled) { + LOG(WARNING) << "[EP P2P] rank " << rank + << " failed to enable peer access to device " + << dst_device << ": " << cudaGetErrorString(err); + continue; + } + if (err == cudaErrorPeerAccessAlreadyEnabled) cudaGetLastError(); + + if (dst >= static_cast(remote_handles.size())) continue; + const auto& h = remote_handles[dst]; + if (h.empty()) continue; + + constexpr size_t kHandleBytes = sizeof(cudaIpcMemHandle_t); + constexpr size_t kNumInt32s = + (kHandleBytes + sizeof(int32_t) - 1) / sizeof(int32_t); + if (h.size() < kNumInt32s) continue; + + cudaIpcMemHandle_t handle; + memcpy(&handle, h.data(), kHandleBytes); + + void* peer_ptr = nullptr; + err = cudaIpcOpenMemHandle(&peer_ptr, handle, + cudaIpcMemLazyEnablePeerAccess); + if (err != cudaSuccess) { + LOG(WARNING) << "[EP P2P] rank " << rank + << " failed to open IPC handle for rank " << dst + << ": " << cudaGetErrorString(err); + continue; + } + LOG(INFO) << "[EP P2P] rank " << rank + << " opened IPC handle for rank " << dst + << ": peer_ptr=" << peer_ptr; + available[dst] = 1; + peer_ptrs_host_[dst] = peer_ptr; + } + + // Determine if all active ranks have P2P + all_peers_accessible_ = true; + for (int i = 0; i < num_ranks_; ++i) { + if (active_ranks_mask[i] == 0) continue; + if (!available[i] || !peer_ptrs_host_[i]) { + all_peers_accessible_ = false; + break; + } + } + // Multi-node: P2P only within a node + if (all_peers_accessible_ && num_ranks_ > 1) { + int first_node = 0 / device_count; + int last_node = (num_ranks_ - 1) / device_count; + if (first_node != last_node) all_peers_accessible_ = false; + } + + cudaMemcpy(available_table_, available.data(), + num_ranks_ * sizeof(int32_t), cudaMemcpyHostToDevice); + cudaMemcpy(peer_ptrs_dev_, peer_ptrs_host_, num_ranks_ * sizeof(void*), + cudaMemcpyHostToDevice); + } + + int32_t* availableTablePtr() override { return available_table_; } + void** peerPtrsTablePtr() override { return peer_ptrs_dev_; } + bool allPeersAccessible() const override { return all_peers_accessible_; } + + bool verifyPeerAccess() override { + if (!all_peers_accessible_) return false; + + int device_id = 0; + cudaGetDevice(&device_id); + + bool all_ok = true; + for (int i = 0; i < num_ranks_; ++i) { + if (i == device_id) continue; + if (!peer_ptrs_host_[i] || peer_ptrs_host_[i] == local_ptr_) + continue; + + // Test: write a pattern to the peer buffer via cudaMemcpy, then + // read it back. This verifies the IPC mapping is writable. + constexpr int kTestBytes = 256; + std::vector pattern(kTestBytes); + for (int j = 0; j < kTestBytes; ++j) + pattern[j] = (uint8_t)(j ^ 0xA5); + + cudaError_t err = cudaMemcpy(peer_ptrs_host_[i], pattern.data(), + kTestBytes, cudaMemcpyHostToDevice); + if (err != cudaSuccess) { + LOG(WARNING) << "[EP P2P] verifyPeerAccess: rank " << device_id + << " cannot write to peer " << i + << " mapped buffer: " << cudaGetErrorString(err); + all_ok = false; + continue; + } + + // Read back and verify + std::vector readback(kTestBytes, 0); + err = cudaMemcpy(readback.data(), peer_ptrs_host_[i], kTestBytes, + cudaMemcpyDeviceToHost); + if (err != cudaSuccess) { + LOG(WARNING) << "[EP P2P] verifyPeerAccess: rank " << device_id + << " cannot read back from peer " << i + << " mapped buffer: " << cudaGetErrorString(err); + all_ok = false; + continue; + } + + bool match = + (memcmp(readback.data(), pattern.data(), kTestBytes) == 0); + if (!match) { + LOG(WARNING) << "[EP P2P] verifyPeerAccess: rank " << device_id + << " readback mismatch from peer " << i; + all_ok = false; + } else { + LOG(INFO) << "[EP P2P] verifyPeerAccess: rank " << device_id + << " peer " << i << " OK (memcpy write/read)"; + } + } + + if (!all_ok) { + all_peers_accessible_ = false; + // Update device table to reflect failure + std::vector avail_h(num_ranks_, 0); + cudaMemcpy(avail_h.data(), available_table_, + num_ranks_ * sizeof(int32_t), cudaMemcpyDeviceToHost); + for (int i = 0; i < num_ranks_; ++i) { + if (i == device_id) continue; + if (!peer_ptrs_host_[i] || peer_ptrs_host_[i] == local_ptr_) + continue; + // Leave self as available; only clear failed peers + } + // Re-upload with all peers marked unavailable except self + std::vector cleared(num_ranks_, 0); + cleared[device_id] = 1; + cudaMemcpy(available_table_, cleared.data(), + num_ranks_ * sizeof(int32_t), cudaMemcpyHostToDevice); + } + + return all_ok; + } + + private: + int num_ranks_; + void* local_ptr_ = nullptr; + int32_t* available_table_ = nullptr; + void** peer_ptrs_host_ = nullptr; + void** peer_ptrs_dev_ = nullptr; + bool all_peers_accessible_ = false; +}; + +std::unique_ptr createP2pDeviceTransport(int num_ranks) { + return std::make_unique(num_ranks); +} + +} // namespace device +} // namespace mooncake -- 2.34.1 From 938d0c70691f76b68c7b48445776f32dc652e623 Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Sat, 6 Jun 2026 17:10:57 +0800 Subject: [PATCH 191/382] [PG] Fix null-deref on MNNVL disconnect and activeRanks leak (#2347) * [PG] Fix null-deref on MNNVL disconnect and activeRanks leak - Guard warmup_recv_region_ dereference in pollPeer disconnect path: on MNNVL/fabric clusters (skip_warmup_==true), the region is nullptr and peer disconnect from CONNECTED state crashes. - Free meta_->activeRanks in shutdown() to match the allocation at init (new[] for CPU, cudaHostAlloc for GPU). Co-Authored-By: Claude Opus 4.6 * Null activeRanksDevice after cudaFreeHost in shutdown Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 --- mooncake-pg/src/connection_poller.cpp | 6 ++++-- mooncake-pg/src/mooncake_backend.cpp | 7 +++++++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/mooncake-pg/src/connection_poller.cpp b/mooncake-pg/src/connection_poller.cpp index 9f74f62b..5d114a79 100644 --- a/mooncake-pg/src/connection_poller.cpp +++ b/mooncake-pg/src/connection_poller.cpp @@ -450,8 +450,10 @@ bool ConnectionContext::pollPeer(int pollingRank) { } // Reset warmup region - *reinterpret_cast( - &warmup_recv_region_[pollingRank]) = 0; + if (warmup_recv_region_) { + *reinterpret_cast( + &warmup_recv_region_[pollingRank]) = 0; + } // Reset P2PProxy states p2p_proxy_->resetPeerState(pollingRank); diff --git a/mooncake-pg/src/mooncake_backend.cpp b/mooncake-pg/src/mooncake_backend.cpp index 40ce0e60..524eb422 100644 --- a/mooncake-pg/src/mooncake_backend.cpp +++ b/mooncake-pg/src/mooncake_backend.cpp @@ -1008,6 +1008,13 @@ void MooncakeBackend::shutdown() { cudaFree(recv_buffer_[i]); } } + if (isCpu_) { + delete[] meta_->activeRanks; + } else { + cudaFreeHost(meta_->activeRanks); + } + meta_->activeRanks = nullptr; + meta_->activeRanksDevice = nullptr; } } -- 2.34.1 From 47b267a5a1130f34f73b69fb0e3f0f599461851c Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Sat, 6 Jun 2026 17:13:03 +0800 Subject: [PATCH 192/382] [PG] Fix data race: make running_ atomic (#2352) Co-authored-by: Claude Opus 4.6 --- mooncake-pg/include/mooncake_worker.cuh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mooncake-pg/include/mooncake_worker.cuh b/mooncake-pg/include/mooncake_worker.cuh index c075c753..e6555c83 100644 --- a/mooncake-pg/include/mooncake_worker.cuh +++ b/mooncake-pg/include/mooncake_worker.cuh @@ -167,7 +167,7 @@ class MooncakeWorker { static constexpr size_t kPingTimeoutMicroseconds_ = 100; static constexpr size_t kDrainTasksTimeoutMs = 5000; // 5s - bool running_ = false; + std::atomic running_{false}; std::atomic started_{false}; int cuda_device_index_; -- 2.34.1 From 27aaae1e948f9317eacefa987da548f5f45a4bb8 Mon Sep 17 00:00:00 2001 From: R0CKSTAR Date: Sat, 6 Jun 2026 21:17:14 +0800 Subject: [PATCH 193/382] [PG] Build the MUSA PG extension through torchada (#2353) * [PG] Build the MUSA PG extension through torchada1 Signed-off-by: Xiaodong Ye * Address review comments Signed-off-by: Xiaodong Ye --------- Signed-off-by: Xiaodong Ye --- mooncake-pg/include/mooncake_backend.h | 28 ---- mooncake-pg/include/mooncake_worker.cuh | 35 +---- mooncake-pg/setup.py | 177 ++++++++++------------- mooncake-pg/src/mooncake_backend.cpp | 107 +++++++------- mooncake-pg/src/mooncake_worker.mu | 1 - mooncake-pg/src/mooncake_worker_host.cpp | 40 ++--- mooncake-pg/src/p2p_proxy.cpp | 1 - 7 files changed, 148 insertions(+), 241 deletions(-) delete mode 120000 mooncake-pg/src/mooncake_worker.mu diff --git a/mooncake-pg/include/mooncake_backend.h b/mooncake-pg/include/mooncake_backend.h index 1b748999..81727175 100644 --- a/mooncake-pg/include/mooncake_backend.h +++ b/mooncake-pg/include/mooncake_backend.h @@ -14,38 +14,10 @@ #include #include -#ifdef MOONCAKE_EP_USE_MUSA -#include -#else #include -#endif namespace mooncake { -#ifdef MOONCAKE_EP_USE_MUSA -static inline auto getCurrentGPUStream(int device_index = -1) { - return at::musa::getCurrentMUSAStream(device_index); -} -static inline auto getGPUStreamFromPool(bool non_blocking = false, - int device_index = -1) { - return at::musa::getStreamFromPool(non_blocking, device_index); -} -static inline int currentGPUDevice() { return at::musa::current_device(); } -static inline constexpr auto kGPUDevice = c10::DeviceType::PrivateUse1; -static inline constexpr auto kGPUDeviceType = at::musa::kMUSA; -#else -static inline auto getCurrentGPUStream(int device_index = -1) { - return at::cuda::getCurrentCUDAStream(device_index); -} -static inline auto getGPUStreamFromPool(bool non_blocking = false, - int device_index = -1) { - return at::cuda::getStreamFromPool(non_blocking, device_index); -} -static inline int currentGPUDevice() { return at::cuda::current_device(); } -static inline constexpr auto kGPUDevice = torch::kCUDA; -static inline constexpr auto kGPUDeviceType = c10::DeviceType::CUDA; -#endif - // Forward declaration – MooncakeP2PShim holds a non-owning pointer to // MooncakeBackend, which is defined below. class MooncakeBackend; diff --git a/mooncake-pg/include/mooncake_worker.cuh b/mooncake-pg/include/mooncake_worker.cuh index e6555c83..d8a6c1a5 100644 --- a/mooncake-pg/include/mooncake_worker.cuh +++ b/mooncake-pg/include/mooncake_worker.cuh @@ -2,12 +2,8 @@ #define MOONCAKE_WORKER_CUH #if !defined(__MUSA__) -#ifdef MOONCAKE_EP_USE_MUSA -#include -#else -#include -#endif #include +#include #include #include #include @@ -28,31 +24,8 @@ #include #include -#ifdef MOONCAKE_EP_USE_MUSA -#define cudaDeviceSynchronize musaDeviceSynchronize -#define cudaError cudaError_t -#define cudaErrorNotReady musaErrorNotReady -#define cudaEventCreateWithFlags musaEventCreateWithFlags -#define cudaEventDestroy musaEventDestroy -#define cudaEventDisableTiming musaEventDisableTiming -#define cudaEventQuery musaEventQuery -#define cudaEventRecord musaEventRecord -#define cudaFuncAttributes musaFuncAttributes -#define cudaFuncGetAttributes musaFuncGetAttributes -#define cudaHostAlloc musaHostAlloc -#define cudaHostAllocMapped musaHostAllocMapped -#endif - namespace mooncake { -#if !defined(__MUSA__) -#ifdef MOONCAKE_EP_USE_MUSA -using GPUStream = at::musa::MUSAStream; -#else -using GPUStream = at::cuda::CUDAStream; -#endif -#endif - static constexpr size_t kBufferSize = 1u << 24; static constexpr size_t kMaxNumRanks = 64; @@ -133,11 +106,11 @@ class MooncakeWorker { c10d::OpType opType, size_t tensorSize, int64_t broadcastRoot, const std::shared_ptr& meta, const std::shared_ptr& connection_ctx, - const GPUStream& issue_stream, + const at::cuda::CUDAStream& issue_stream, const std::function& tensorToBuffer, + const at::cuda::CUDAStream&)>& tensorToBuffer, const std::function& bufferToTensor); + const at::cuda::CUDAStream&)>& bufferToTensor); void Start(); diff --git a/mooncake-pg/setup.py b/mooncake-pg/setup.py index 845960f7..37298e53 100644 --- a/mooncake-pg/setup.py +++ b/mooncake-pg/setup.py @@ -3,7 +3,23 @@ import re from setuptools import setup import torch -from torch.utils.cpp_extension import BuildExtension, CUDAExtension, CUDA_HOME + +use_musa = os.getenv("MOONCAKE_EP_USE_MUSA", "").upper() in {"1", "ON", "TRUE", "YES"} +if use_musa: + try: + import torchada # noqa: F401 + except ImportError as e: + raise ImportError( + "torchada is required to build the MUSA PG extension. " + "Please install it first using 'pip install torchada'." + ) from e + + +from torch.utils.cpp_extension import ( # noqa: E402 + BuildExtension, + CUDAExtension, + CUDA_HOME, +) torch_version = re.match(r"\d+(?:\.\d+)*", torch.__version__).group() @@ -13,67 +29,35 @@ module_name = "mooncake.pg" + version_suffix abi_flag = int(torch._C._GLIBCXX_USE_CXX11_ABI) current_dir = os.path.abspath(os.path.dirname(__file__)) -use_musa = os.getenv("MOONCAKE_EP_USE_MUSA", "").upper() in {"1", "ON", "TRUE", "YES"} +abi_define = f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}" +cxx_args = [abi_define, "-std=c++20", "-O3", "-g0"] + +cuda_libraries = ["ibverbs", "mlx5"] +cuda_library_dirs = [] if use_musa: - from torch_musa.utils.musa_extension import MUSAExtension - from torch_musa.utils.musa_extension import BuildExtension as MUSABuildExtension - - setup( - name=module_name, - ext_modules=[ - MUSAExtension( - name=module_name, - include_dirs=[ - os.path.join(current_dir, "include"), - os.path.join(current_dir, "../mooncake-transfer-engine/include"), - ], - sources=[ - "src/pg_py.cpp", - "src/mooncake_backend.cpp", - "src/p2p_proxy.cpp", - "src/mooncake_worker.mu", - "src/mooncake_worker_host.cpp", - "src/mooncake_worker_thread.cpp", - "src/connection_poller.cpp", - ], - extra_compile_args={ - "cxx": [ - f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", - "-DUSE_MUSA", - "-DMOONCAKE_EP_USE_MUSA=1", - "-std=c++20", - "-O3", - "-g0", - ], - "mcc": [ - f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", - "-DUSE_MUSA", - "-DMOONCAKE_EP_USE_MUSA=1", - "-std=c++20", - "--cuda-gpu-arch=mp_21", - "--cuda-gpu-arch=mp_31", - "-O3", - ], - }, - libraries=["ibverbs", "mlx5"], - extra_link_args=[ - "-Wl,-rpath,$ORIGIN", - "-L" + os.path.join(current_dir, "../mooncake-wheel/mooncake"), - "-Wl,--push-state,--no-as-needed", - "-l:engine.so", - "-Wl,--pop-state", - ], - ) - ], - cmdclass={"build_ext": MUSABuildExtension}, - ) + musa_defines = ["-DUSE_MUSA", "-DMOONCAKE_EP_USE_MUSA=1"] + cxx_args += musa_defines + # torchada maps the "nvcc" key to "mcc". + device_args = [ + abi_define, + *musa_defines, + "-std=c++20", + "--cuda-gpu-arch=mp_21", + "--cuda-gpu-arch=mp_31", + "-O3", + ] else: + device_args = [ + abi_define, + "-std=c++20", + "-Xcompiler", + "-O3", + "-Xcompiler", + "-g0", + ] # Link against the CUDA driver stub library if available. # Same approach as mooncake-ep/setup.py. - cuda_libraries = ["ibverbs", "mlx5"] - cuda_library_dirs = [] - if CUDA_HOME is not None: cuda_stub_dir = os.path.join(CUDA_HOME, "lib64", "stubs") cuda_stub_lib = os.path.join(cuda_stub_dir, "libcuda.so") @@ -81,50 +65,35 @@ else: cuda_libraries.insert(0, "cuda") cuda_library_dirs.append(cuda_stub_dir) - setup( - name=module_name, - ext_modules=[ - CUDAExtension( - name=module_name, - include_dirs=[ - os.path.join(current_dir, "include"), - os.path.join(current_dir, "../mooncake-transfer-engine/include"), - ], - sources=[ - "src/pg_py.cpp", - "src/mooncake_backend.cpp", - "src/p2p_proxy.cpp", - "src/mooncake_worker.cu", - "src/mooncake_worker_host.cpp", - "src/mooncake_worker_thread.cpp", - "src/connection_poller.cpp", - ], - extra_compile_args={ - "cxx": [ - f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", - "-std=c++20", - "-O3", - "-g0", - ], - "nvcc": [ - f"-D_GLIBCXX_USE_CXX11_ABI={abi_flag}", - "-std=c++20", - "-Xcompiler", - "-O3", - "-Xcompiler", - "-g0", - ], - }, - libraries=cuda_libraries, - library_dirs=cuda_library_dirs, - extra_link_args=[ - "-Wl,-rpath,$ORIGIN", - "-L" + os.path.join(current_dir, "../mooncake-wheel/mooncake"), - "-Wl,--push-state,--no-as-needed", - "-l:engine.so", - "-Wl,--pop-state", - ], - ) - ], - cmdclass={"build_ext": BuildExtension}, - ) +setup( + name=module_name, + ext_modules=[ + CUDAExtension( + name=module_name, + include_dirs=[ + os.path.join(current_dir, "include"), + os.path.join(current_dir, "../mooncake-transfer-engine/include"), + ], + sources=[ + "src/pg_py.cpp", + "src/mooncake_backend.cpp", + "src/p2p_proxy.cpp", + "src/mooncake_worker.cu", + "src/mooncake_worker_host.cpp", + "src/mooncake_worker_thread.cpp", + "src/connection_poller.cpp", + ], + extra_compile_args={"cxx": cxx_args, "nvcc": device_args}, + libraries=cuda_libraries, + library_dirs=cuda_library_dirs, + extra_link_args=[ + "-Wl,-rpath,$ORIGIN", + "-L" + os.path.join(current_dir, "../mooncake-wheel/mooncake"), + "-Wl,--push-state,--no-as-needed", + "-l:engine.so", + "-Wl,--pop-state", + ], + ) + ], + cmdclass={"build_ext": BuildExtension}, +) diff --git a/mooncake-pg/src/mooncake_backend.cpp b/mooncake-pg/src/mooncake_backend.cpp index 524eb422..9d01fa07 100644 --- a/mooncake-pg/src/mooncake_backend.cpp +++ b/mooncake-pg/src/mooncake_backend.cpp @@ -1,8 +1,4 @@ -#ifdef MOONCAKE_EP_USE_MUSA -#include -#else #include -#endif #include #include #include @@ -259,7 +255,7 @@ MooncakeBackend::MooncakeBackend( } else { for (size_t i = 0; i < 2; i++) { - cudaError err = cudaMalloc(&send_buffer_[i], kBufferSize); + cudaError_t err = cudaMalloc(&send_buffer_[i], kBufferSize); TORCH_CHECK(!err, c10::str("Failed to allocate CUDA send buffer")); int rc = engine_->registerLocalMemory(send_buffer_[i], kBufferSize, @@ -268,7 +264,7 @@ MooncakeBackend::MooncakeBackend( } for (size_t i = 0; i < 2; i++) { - cudaError err = cudaMalloc(&recv_buffer_[i], kBufferSize); + cudaError_t err = cudaMalloc(&recv_buffer_[i], kBufferSize); TORCH_CHECK(!err, c10::str("Failed to allocate CUDA recv buffer")); int rc = engine_->registerLocalMemory(recv_buffer_[i], kBufferSize, @@ -297,7 +293,7 @@ MooncakeBackend::MooncakeBackend( } auto& dev_worker_mgr = P2PDeviceWorkerManager::getInstance(); - int cuda_device_index = isCpu_ ? -1 : currentGPUDevice(); + int cuda_device_index = isCpu_ ? -1 : at::cuda::current_device(); if (isCpu_) p2p_device_worker_ = dev_worker_mgr.getCPUWorker(engine_); @@ -389,7 +385,7 @@ MooncakeBackend::MooncakeBackend( "activeRanks must be on CPU."); } else { TORCH_CHECK( - options_->activeRanks_.device().type() == kGPUDeviceType, + options_->activeRanks_.device().type() == c10::DeviceType::CUDA, "activeRanks must be on GPU."); } if (max_size != size) { @@ -399,9 +395,9 @@ MooncakeBackend::MooncakeBackend( } meta_->activeRanksTensor = options_->activeRanks_; } else { - meta_->activeRanksTensor = - at::ones({max_size}, torch::dtype(torch::kInt32) - .device(isCpu ? torch::kCPU : kGPUDevice)); + meta_->activeRanksTensor = at::ones( + {max_size}, torch::dtype(torch::kInt32) + .device(isCpu ? torch::kCPU : torch::kCUDA)); if (max_size != size) { meta_->activeRanksTensor.slice(0, size, max_size).fill_(0); } @@ -425,7 +421,7 @@ MooncakeBackend::MooncakeBackend( // Register a lightweight Backend shim so that PyTorch's P2P dispatch path // (batch_isend_irecv → _get_backend → getBackend) can find a registered // Backend for this ProcessGroup. The shim delegates send/recv back to us. - auto deviceType = isCpu ? c10::DeviceType::CPU : kGPUDeviceType; + auto deviceType = isCpu ? c10::DeviceType::CPU : c10::DeviceType::CUDA; auto shim = c10::make_intrusive(this); setBackend(deviceType, BackendType::CUSTOM, shim); #ifndef MOONCAKE_EP_USE_MUSA @@ -486,7 +482,8 @@ c10::intrusive_ptr MooncakeBackend::send( P2PProxy::OpStatus::kPending); cudaStream_t stream = nullptr; if (!isCpu_) { - auto current_stream = getCurrentGPUStream(contiguous.device().index()); + auto current_stream = + at::cuda::getCurrentCUDAStream(contiguous.device().index()); stream = current_stream.stream(); } @@ -518,7 +515,8 @@ c10::intrusive_ptr MooncakeBackend::recv( P2PProxy::OpStatus::kPending); cudaStream_t stream = nullptr; if (!isCpu_) { - auto current_stream = getCurrentGPUStream(target.device().index()); + auto current_stream = + at::cuda::getCurrentCUDAStream(target.device().index()); stream = current_stream.stream(); } @@ -553,12 +551,13 @@ c10::intrusive_ptr MooncakeBackend::broadcast( memcpy((char*)tensor.data_ptr() + pos, src, realSize); }); } else { - GPUStream stream = getCurrentGPUStream(tensor.device().index()); + at::cuda::CUDAStream stream = + at::cuda::getCurrentCUDAStream(tensor.device().index()); return worker_->putTaskCuda( c10d::OpType::BROADCAST, tensorSize, root, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { if (isRoot) { cudaMemcpyAsync(dst, (char*)tensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, @@ -566,7 +565,7 @@ c10::intrusive_ptr MooncakeBackend::broadcast( } }, [=](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemcpyAsync((char*)tensor.data_ptr() + pos, src, realSize, cudaMemcpyDeviceToDevice, enq_stream); }); @@ -592,17 +591,17 @@ c10::intrusive_ptr MooncakeBackend::allreduce( opts.reduceOp, meta_->activeRanks); }); } else { - auto stream = getCurrentGPUStream(tensor.device().index()); + auto stream = at::cuda::getCurrentCUDAStream(tensor.device().index()); return worker_->putTaskCuda( c10d::OpType::ALLREDUCE, tensorSize, 0, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemcpyAsync(dst, (char*)tensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=, this](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemsetAsync((char*)tensor.data_ptr() + pos, 0, realSize, enq_stream); launchReduceKernel(tensor, pos, realSize, src, meta_->size, @@ -633,17 +632,18 @@ c10::intrusive_ptr MooncakeBackend::allgather( } }); } else { - auto stream = getCurrentGPUStream(inputTensor.device().index()); + auto stream = + at::cuda::getCurrentCUDAStream(inputTensor.device().index()); return worker_->putTaskCuda( c10d::OpType::ALLGATHER, tensorSize, 0, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemcpyAsync(dst, (char*)inputTensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { for (const auto j : c10::irange(outputTensors_.size())) { cudaMemcpyAsync((char*)outputTensors_[j].data_ptr() + pos, (char*)src + j * realSize, realSize, @@ -673,17 +673,18 @@ c10::intrusive_ptr MooncakeBackend::_allgather_base( } }); } else { - auto stream = getCurrentGPUStream(inputBuffer.device().index()); + auto stream = + at::cuda::getCurrentCUDAStream(inputBuffer.device().index()); return worker_->putTaskCuda( c10d::OpType::_ALLGATHER_BASE, tensorSize, 0, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemcpyAsync(dst, (char*)inputBuffer.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=, this](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { for (int j = 0; j < meta_->size; ++j) { if (!meta_->activeRanks[j]) continue; cudaMemcpyAsync( @@ -718,12 +719,13 @@ c10::intrusive_ptr MooncakeBackend::_reduce_scatter_base( opts.reduceOp, meta_->activeRanks); }); } else { - auto stream = getCurrentGPUStream(inputBuffer.device().index()); + auto stream = + at::cuda::getCurrentCUDAStream(inputBuffer.device().index()); return worker_->putTaskCuda( c10d::OpType::_REDUCE_SCATTER_BASE, tensorSize, 0, meta_, connection_ctx_, stream, [=, this](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { for (int j = 0; j < meta_->size; ++j) { if (!meta_->activeRanks[j]) continue; cudaMemcpyAsync( @@ -733,7 +735,7 @@ c10::intrusive_ptr MooncakeBackend::_reduce_scatter_base( } }, [=, this](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemsetAsync((char*)outputBuffer.data_ptr() + pos, 0, realSize, enq_stream); launchReduceKernel(outputBuffer, pos, realSize, src, @@ -764,12 +766,13 @@ c10::intrusive_ptr MooncakeBackend::alltoall( } }); } else { - auto stream = getCurrentGPUStream(inputTensors[0].device().index()); + auto stream = + at::cuda::getCurrentCUDAStream(inputTensors[0].device().index()); return worker_->putTaskCuda( c10d::OpType::ALLTOALL, tensorSize, 0, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { for (const auto j : c10::irange(inputTensors.size())) { cudaMemcpyAsync((char*)dst + j * realSize, (char*)inputTensors[j].data_ptr() + pos, @@ -778,7 +781,7 @@ c10::intrusive_ptr MooncakeBackend::alltoall( } }, [=](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { for (const auto j : c10::irange(outputTensors.size())) { cudaMemcpyAsync((char*)outputTensors[j].data_ptr() + pos, (char*)src + j * realSize, realSize, @@ -797,13 +800,13 @@ c10::intrusive_ptr MooncakeBackend::barrier( connection_ctx_, [=](void*, size_t, size_t) {}, [=](void*, size_t, size_t) {}); } else { - auto device_index = currentGPUDevice(); - auto stream = getCurrentGPUStream(device_index); + auto device_index = at::cuda::current_device(); + auto stream = at::cuda::getCurrentCUDAStream(device_index); return worker_->putTaskCuda( c10d::OpType::BARRIER, kBarrierDummyTensorSize, 0, meta_, connection_ctx_, stream, - [=](void*, size_t, size_t, const GPUStream&) {}, - [=](void*, size_t, size_t, const GPUStream&) {}); + [=](void*, size_t, size_t, const at::cuda::CUDAStream&) {}, + [=](void*, size_t, size_t, const at::cuda::CUDAStream&) {}); } } @@ -829,17 +832,17 @@ c10::intrusive_ptr MooncakeBackend::reduce( } }); } else { - auto stream = getCurrentGPUStream(tensor.device().index()); + auto stream = at::cuda::getCurrentCUDAStream(tensor.device().index()); return worker_->putTaskCuda( c10d::OpType::REDUCE, tensorSize, root, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemcpyAsync(dst, (char*)tensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=, this](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { if (isRoot) { cudaMemsetAsync((char*)tensor.data_ptr() + pos, 0, realSize, enq_stream); @@ -878,17 +881,18 @@ c10::intrusive_ptr MooncakeBackend::gather( } }); } else { - auto stream = getCurrentGPUStream(inputTensor.device().index()); + auto stream = + at::cuda::getCurrentCUDAStream(inputTensor.device().index()); return worker_->putTaskCuda( c10d::OpType::GATHER, tensorSize, root, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemcpyAsync(dst, (char*)inputTensor.data_ptr() + pos, realSize, cudaMemcpyDeviceToDevice, enq_stream); }, [=](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { if (isRoot) { auto outputTensors_ = outputTensors.back(); for (const auto j : c10::irange(outputTensors_.size())) { @@ -931,12 +935,13 @@ c10::intrusive_ptr MooncakeBackend::scatter( memcpy((char*)outputTensor.data_ptr() + pos, src, realSize); }); } else { - auto stream = getCurrentGPUStream(outputTensor.device().index()); + auto stream = + at::cuda::getCurrentCUDAStream(outputTensor.device().index()); return worker_->putTaskCuda( c10d::OpType::SCATTER, tensorSize, root, meta_, connection_ctx_, stream, [=](void* dst, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { if (isRoot) { auto inputTensors_ = inputTensors.back(); for (const auto j : c10::irange(inputTensors_.size())) { @@ -948,7 +953,7 @@ c10::intrusive_ptr MooncakeBackend::scatter( } }, [=](void* src, size_t pos, size_t realSize, - const GPUStream& enq_stream) { + const at::cuda::CUDAStream& enq_stream) { cudaMemcpyAsync((char*)outputTensor.data_ptr() + pos, src, realSize, cudaMemcpyDeviceToDevice, enq_stream); }); @@ -1028,7 +1033,7 @@ void MooncakeBackend::syncActiveRanksTensor() { if (!meta_->activeRanksTensor.defined() || meta_->activeRanksTensor.size(0) != meta_->size) { meta_->activeRanksTensor = - cpu_tensor.to(isCpu_ ? torch::kCPU : kGPUDevice); + cpu_tensor.to(isCpu_ ? torch::kCPU : torch::kCUDA); return; } @@ -1109,14 +1114,15 @@ int MooncakeBackend::getNumSyncedRanks() { std::vector tensors; tensors.emplace_back(torch::tensor( connection_ctx_->getTotalConnectedPeers(), - torch::dtype(torch::kInt).device(isCpu_ ? torch::kCPU : kGPUDevice))); + torch::dtype(torch::kInt).device(isCpu_ ? torch::kCPU : torch::kCUDA))); c10d::AllreduceOptions opts{ .reduceOp = c10d::ReduceOp::MIN, }; auto work = allreduce(tensors, opts); work->wait(); if (!isCpu_) { - auto stream = getCurrentGPUStream(tensors[0].device().index()); + auto stream = + at::cuda::getCurrentCUDAStream(tensors[0].device().index()); cudaStreamSynchronize(stream); } return tensors[0].cpu().item(); @@ -1179,14 +1185,15 @@ std::vector MooncakeBackend::getPeerState(const std::vector& ranks) { std::vector tensors; tensors.emplace_back(torch::tensor( input, torch::dtype(torch::kInt) - .device(isCpu_ ? torch::kCPU : kGPUDevice))); + .device(isCpu_ ? torch::kCPU : torch::kCUDA))); c10d::AllreduceOptions opts{ .reduceOp = c10d::ReduceOp::MIN, }; auto work = allreduce(tensors, opts); work->wait(); if (!isCpu_) { - auto stream = getCurrentGPUStream(tensors[0].device().index()); + auto stream = + at::cuda::getCurrentCUDAStream(tensors[0].device().index()); cudaStreamSynchronize(stream); } bool activeRanksChanged = false; diff --git a/mooncake-pg/src/mooncake_worker.mu b/mooncake-pg/src/mooncake_worker.mu deleted file mode 120000 index 73bffcdd..00000000 --- a/mooncake-pg/src/mooncake_worker.mu +++ /dev/null @@ -1 +0,0 @@ -mooncake_worker.cu \ No newline at end of file diff --git a/mooncake-pg/src/mooncake_worker_host.cpp b/mooncake-pg/src/mooncake_worker_host.cpp index 96173bfa..db287b1f 100644 --- a/mooncake-pg/src/mooncake_worker_host.cpp +++ b/mooncake-pg/src/mooncake_worker_host.cpp @@ -8,22 +8,10 @@ #include #include #include -#ifdef MOONCAKE_EP_USE_MUSA -#include -#else #include -#endif #include "pg_utils.h" -#ifdef MOONCAKE_EP_USE_MUSA -namespace gpu_capture = at::musa; -namespace gpu_c10 = c10::musa; -#else -namespace gpu_capture = at::cuda; -namespace gpu_c10 = c10::cuda; -#endif - namespace mooncake { class MooncakeWorkCpu : public ::c10d::Work { @@ -105,8 +93,8 @@ class MooncakeWorkCuda : public ::c10d::Work { // waitUntilTasksSubmitted is totally unnecessary, but we keep it for // uniform behavior to avoid invasive changes to TE/TENT. bool submitted = true; - if (gpu_capture::currentStreamCaptureStatus() == - gpu_c10::CaptureStatus::None) { + if (at::cuda::currentStreamCaptureStatus() == + c10::cuda::CaptureStatus::None) { // Normal execution: block until tasks are submitted. submitted = worker_->waitUntilTasksSubmitted(submitted_tasks_, timeout); @@ -134,7 +122,7 @@ class MooncakeWorkCuda : public ::c10d::Work { // until the operation is completed. In the case of CUDA collectives, // will block the currently active CUDA stream until the operation // is completed (but will not block the CPU)." - auto current_stream = getCurrentGPUStream(); + auto current_stream = at::cuda::getCurrentCUDAStream(); event_->block(current_stream); return true; } @@ -153,12 +141,12 @@ class MooncakeBarrierWorkCuda : public MooncakeWorkCuda { bool wait(std::chrono::milliseconds timeout) override { // Skip host-side synchronization during CUDA graph capture. // cudaEventSynchronize is not permitted while a stream is capturing. - if (gpu_capture::currentStreamCaptureStatus() != - gpu_c10::CaptureStatus::None) { + if (at::cuda::currentStreamCaptureStatus() != + c10::cuda::CaptureStatus::None) { // We still need stream-level synchronization so that subsequent // operations on the capture stream are ordered after the barrier // task on the enqueue stream. - auto current_stream = getCurrentGPUStream(); + auto current_stream = at::cuda::getCurrentCUDAStream(); event_->block(current_stream); return true; } @@ -309,7 +297,7 @@ void launchReduceCpu(at::Tensor dst, size_t pos, size_t realSize, void* src, MooncakeWorker::MooncakeWorker(int cuda_device_index) : cuda_device_index_(cuda_device_index) { int deviceCount = 0; - cudaError err = cudaGetDeviceCount(&deviceCount); + cudaError_t err = cudaGetDeviceCount(&deviceCount); if (!err && deviceCount > 0) { cudaHostAlloc(&tasks_, kNumTasks_ * sizeof(Task), cudaHostAllocMapped); cudaHostGetDevicePointer(&tasks_device_, tasks_, 0); @@ -411,19 +399,19 @@ c10::intrusive_ptr MooncakeWorker::putTaskCuda( c10d::OpType opType, size_t tensorSize, int64_t broadcastRoot, const std::shared_ptr& meta, const std::shared_ptr& connection_ctx, - const GPUStream& issue_stream, + const at::cuda::CUDAStream& issue_stream, const std::function& tensorToBuffer, + const at::cuda::CUDAStream&)>& tensorToBuffer, const std::function& bufferToTensor) { + const at::cuda::CUDAStream&)>& bufferToTensor) { connection_ctx->waitUntilNewRanksConnected(); size_t chunkSize = ((kBufferSize - 1) / meta->size) & ~(size_t)7; - GPUStream enq_stream = - getGPUStreamFromPool(false, issue_stream.device_index()); + at::cuda::CUDAStream enq_stream = + at::cuda::getStreamFromPool(false, issue_stream.device_index()); - auto event_start = std::make_shared(kGPUDevice); + auto event_start = std::make_shared(torch::kCUDA); event_start->record(issue_stream); event_start->block(enq_stream); @@ -455,7 +443,7 @@ c10::intrusive_ptr MooncakeWorker::putTaskCuda( ++meta->taskCount; } - auto event_end = std::make_shared(kGPUDevice); + auto event_end = std::make_shared(torch::kCUDA); event_end->record(enq_stream); if (opType == c10d::OpType::BARRIER) { diff --git a/mooncake-pg/src/p2p_proxy.cpp b/mooncake-pg/src/p2p_proxy.cpp index 2db2fb42..b9e31c0e 100644 --- a/mooncake-pg/src/p2p_proxy.cpp +++ b/mooncake-pg/src/p2p_proxy.cpp @@ -8,7 +8,6 @@ #include #include #include -#include "cuda_alike.h" #include "memory_location.h" #include "pg_utils.h" -- 2.34.1 From 1352bbec43081e461356aaecf6c70cddd826b455 Mon Sep 17 00:00:00 2001 From: JimyMa <33408125+JimyMa@users.noreply.github.com> Date: Sun, 7 Jun 2026 00:08:24 +0800 Subject: [PATCH 194/382] [Doc] Fix DLSlime typo in LMDeploy docs (#2356) --- .../getting_started/examples/lmdeploy-integration-v0.9.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting_started/examples/lmdeploy-integration-v0.9.md b/docs/source/getting_started/examples/lmdeploy-integration-v0.9.md index fa8d7d11..ac6fd6fb 100644 --- a/docs/source/getting_started/examples/lmdeploy-integration-v0.9.md +++ b/docs/source/getting_started/examples/lmdeploy-integration-v0.9.md @@ -74,7 +74,7 @@ lmdeploy serve api_server Qwen/Qwen3-8B \ - The `--role` parameter sets the node role in the disaggregated system (`Prefill` for token embedding and KV cache generation). - The `--proxy-url` parameter connects the worker instance back to the proxy for coordination. - The `--backend` parameter specifies the model execution backend (e.g., `pytorch`, `turbomind`). -- The `--migration-backend` parameter defines the KV cache transport mechanism (e.g., `Mooncake` and `DlSllime`). +- The `--migration-backend` parameter defines the KV cache transport mechanism (e.g., `Mooncake` and `DLSlime`). #### Decode: -- 2.34.1 From 42f74b407c11057a9daaf4a5a8dc35724e270904 Mon Sep 17 00:00:00 2001 From: Hubert Zhang Date: Sun, 7 Jun 2026 23:31:17 +0800 Subject: [PATCH 195/382] [TE] feat(engine): add attributes in python package for representing compile flags (#2342) --- .../python-api-reference/transfer-engine.md | 29 ++++++++++++++++++ .../transfer_engine/transfer_engine_py.cpp | 30 +++++++++++++++++++ 2 files changed, 59 insertions(+) diff --git a/docs/source/python-api-reference/transfer-engine.md b/docs/source/python-api-reference/transfer-engine.md index afa90d9a..f5e7b5b0 100644 --- a/docs/source/python-api-reference/transfer-engine.md +++ b/docs/source/python-api-reference/transfer-engine.md @@ -817,3 +817,32 @@ The Transfer Engine Python API is thread-safe for most operations. However, it's 2. **Transfer Failures**: Verify target hostname is correct and network connectivity is established 3. **Memory Issues**: Ensure sufficient system memory and proper buffer alignment 4. **Performance Issues**: Check RDMA device configuration and network topology + +## Compile-time Feature Support Attributes + +The `mooncake.engine` module provides boolean attributes that indicate whether specific features were enabled during compilation: + +### Module Attributes + +- `engine.SUPPORT_CUDA`: Whether CUDA support is enabled +- `engine.SUPPORT_EFA`: Whether EFA (Elastic Fabric Adapter) support is enabled +- `engine.SUPPORT_HIP`: Whether HIP (Heterogeneous-compute Interface for Portability) support is enabled +- `engine.SUPPORT_MNNVL`: Whether MNNVL transport protocol support is enabled +- `engine.SUPPORT_INTRA_NVLINK`: Whether intra-node NVLink support is enabled + +### Usage Example + +```python +from mooncake import engine + +# Check if CUDA is supported +if engine.SUPPORT_CUDA: + print("CUDA support is available") + +# Check all features +print(f"CUDA: {engine.SUPPORT_CUDA}") +print(f"EFA: {engine.SUPPORT_EFA}") +print(f"HIP: {engine.SUPPORT_HIP}") +print(f"MNNVL: {engine.SUPPORT_MNNVL}") +print(f"Intra-NVLink: {engine.SUPPORT_INTRA_NVLINK}") +``` diff --git a/mooncake-integration/transfer_engine/transfer_engine_py.cpp b/mooncake-integration/transfer_engine/transfer_engine_py.cpp index a125760e..97875e16 100644 --- a/mooncake-integration/transfer_engine/transfer_engine_py.cpp +++ b/mooncake-integration/transfer_engine/transfer_engine_py.cpp @@ -1034,6 +1034,36 @@ void bind_coro_rpc_interface(py::module_& m) { } PYBIND11_MODULE(engine, m) { +#ifdef USE_EFA + m.attr("SUPPORT_EFA") = true; +#else + m.attr("SUPPORT_EFA") = false; +#endif + +#ifdef USE_HIP + m.attr("SUPPORT_HIP") = true; +#else + m.attr("SUPPORT_HIP") = false; +#endif + +#ifdef USE_MNNVL + m.attr("SUPPORT_MNNVL") = true; +#else + m.attr("SUPPORT_MNNVL") = false; +#endif + +#ifdef USE_INTRA_NVLINK + m.attr("SUPPORT_INTRA_NVLINK") = true; +#else + m.attr("SUPPORT_INTRA_NVLINK") = false; +#endif + +#ifdef USE_CUDA + m.attr("SUPPORT_CUDA") = true; +#else + m.attr("SUPPORT_CUDA") = false; +#endif + py::enum_ transfer_opcode( m, "TransferOpcode", py::arithmetic()); transfer_opcode.value("Read", TransferEnginePy::TransferOpcode::READ) -- 2.34.1 From bd281c16df0990785fc273ab1363f159550d90ae Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Mon, 8 Jun 2026 00:39:27 +0800 Subject: [PATCH 196/382] [Doc] Add Mooncake local CI skill docs (#2134) --- .claude/skills/mooncake-ci-local/SKILL.md | 383 +++++++----------- .../mooncake-ci-local/examples/minimal.md | 30 ++ 2 files changed, 168 insertions(+), 245 deletions(-) create mode 100644 .claude/skills/mooncake-ci-local/examples/minimal.md diff --git a/.claude/skills/mooncake-ci-local/SKILL.md b/.claude/skills/mooncake-ci-local/SKILL.md index 0729957a..546bcc24 100644 --- a/.claude/skills/mooncake-ci-local/SKILL.md +++ b/.claude/skills/mooncake-ci-local/SKILL.md @@ -1,266 +1,159 @@ --- name: mooncake-ci-local -description: Run Mooncake CI test suite locally — maps GitHub Actions CI steps to local commands. Use this skill whenever the user wants to run tests locally, reproduce a CI failure, check if their changes break tests, or run any subset of the CI test suite (C++ unit tests via ctest, Python integration tests, code format checks, or the full test pipeline). Trigger on phrases like "run tests", "run CI locally", "reproduce CI failure", "check my changes", "test before PR", "run ctest", "run python tests", "run all tests". +description: Run Mooncake pre-PR local validation through scripts/run_ci_test.sh. Use this skill whenever the user wants to validate a branch before opening or submitting a PR, run local CI, run ci test, check changes before PR, reproduce GitHub Actions locally, or force a full pre-submit verification. Trigger on phrases like "提交 PR 前验证", "run ci test", "run local CI", "check my branch", "test before PR", "pre-submit validation", and "reproduce CI locally". --- -# Mooncake CI Local Test Runner +# Mooncake Pre-PR Local Validation -You help users run the Mooncake CI test suite locally. The CI has three test layers. Map what the user wants to the right layer, check prerequisites, and run the tests. +Use `bash scripts/run_ci_test.sh` as the default entry point. This is the single local lane for PR-before-submit validation, and it already coordinates the reproducible parts of GitHub Actions. -## CI Test Layers +## Default Entry Point -### Layer 1 — C++ Unit Tests (ctest) -**CI equivalent:** `build` job in `ci.yml` — "Test (in build env) with coverage" +When the user asks for any of the following, run the repo script first instead of reconstructing the workflow by hand: -**Prerequisite services:** -```bash -# 1. etcd (port 2379) -etcd --advertise-client-urls http://127.0.0.1:2379 --listen-client-urls http://127.0.0.1:2379 & -sleep 2 -etcdctl --endpoints=http://127.0.0.1:2379 endpoint health # verify +- 提交 PR 前本地验证 +- run ci test +- run local CI +- check my branch before PR +- reproduce CI locally -# 2. HTTP metadata server (port 8080) -cd mooncake-transfer-engine/example/http-metadata-server-python -pip install aiohttp -python ./bootstrap_server.py & -cd - -``` - -**Run:** -```bash -cd build -export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib -MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -j --output-on-failure -``` - -**Run specific test:** -```bash -cd build -MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -R --output-on-failure -# List all available tests: ctest -N -``` - -### Layer 2 — Python Integration Tests -**CI equivalent:** `test-wheel-ubuntu` job — `run_tests.sh` - -**Prerequisite:** Mooncake wheel must be installed (either via `pip install` or via `make install` after build). - -**Check install:** -```bash -python -c "import mooncake; print('OK')" -which mooncake_master # must NOT be /usr/local/bin (must be from Python package) -``` - -**Run full suite:** -```bash -# Start metadata server first -mooncake_http_metadata_server --port 8080 & -sleep 1 - -cd mooncake-wheel/tests -MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 MC_FORCE_TCP=true \ - bash ../../scripts/run_tests.sh -``` - -**Individual Python tests** (all require metadata server + mooncake_master on port 50051): -```bash -# Setup shared services -mooncake_http_metadata_server --port 8080 & -mooncake_master --default_kv_lease_ttl=500 & -sleep 2 - -cd mooncake-wheel/tests -export MC_METADATA_SERVER=http://127.0.0.1:8080/metadata -export DEFAULT_KV_LEASE_TTL=500 -export MC_FORCE_TCP=true - -# Pick any test: -python test_distributed_object_store.py -python test_replicated_distributed_object_store.py -python test_put_get_tensor.py # requires torch + numpy -python test_safetensor_functions.py # requires safetensors -python test_dummy_client.py -python test_cli.py -python test_distributed_object_store_cxl.py # requires CXL build -``` - -**Transfer engine tests specifically:** -```bash -cd mooncake-wheel/tests -MC_METADATA_SERVER=http://127.0.0.1:8080/metadata MC_FORCE_TCP=true python transfer_engine_target.py & -TARGET_PID=$! -MC_METADATA_SERVER=http://127.0.0.1:8080/metadata MC_FORCE_TCP=true python transfer_engine_initiator_test.py -kill $TARGET_PID -``` - -**Scripts-based tests** (from `test-wheel-ubuntu` job): -```bash -# Tensor API perf test -export MOONCAKE_MASTER="127.0.0.1:50051" -export MOONCAKE_TE_META_DATA_SERVER="http://127.0.0.1:8080/metadata" -export MOONCAKE_PROTOCOL="tcp" -export LOCAL_HOSTNAME="127.0.0.1" -python scripts/test_tensor_api.py -n 1 -python scripts/test_async_store.py -python scripts/test_copy_move_api.py -``` - -### Layer 3 — Static Checks (no services needed) -**CI equivalent:** `clang-format` and `spell-check` jobs - -**Code format (changed files vs main):** -```bash -./scripts/code_format.sh --check --base origin/main -# Auto-fix: -./scripts/code_format.sh --base origin/main -``` - -**Spell check:** -```bash -# Requires typos tool: cargo install typos-cli -typos -``` - -**Pre-commit (runs all hooks):** -```bash -pip install pre-commit -pre-commit run --all-files -# Or just on staged files: -pre-commit run -``` - -## Build Configurations (from CI) - -If the user needs to build first, here are the CI-equivalent cmake flags: - -**Standard build with coverage (mirrors `build` job):** -```bash -mkdir build && cd build -cmake -G Ninja .. -DUSE_HTTP=ON -DUSE_CXL=ON -DUSE_ETCD=ON -DSTORE_USE_ETCD=ON -DENABLE_ASAN=ON -DCMAKE_BUILD_TYPE=Debug -cmake --build . -sudo cmake --install . -``` - -**All features ON (mirrors `build-flags` job):** -```bash -mkdir build && cd build -cmake -G Ninja .. -DUSE_ETCD=ON -DUSE_CXL=ON -DUSE_REDIS=ON -DUSE_HTTP=ON -DWITH_STORE=ON -DWITH_P2P_STORE=ON -DWITH_METRICS=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -cmake --build . -sudo cmake --install . -``` - -**Transfer engine only:** -```bash -cd mooncake-transfer-engine -mkdir build && cd build -cmake -G Ninja .. -DUSE_ETCD=OFF -DUSE_CXL=ON -DUSE_REDIS=ON -DUSE_HTTP=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -cmake --build . -``` - -## Workflow: Diagnosing and Running Tests - -### Step 1 — Understand what the user wants - -Ask (or infer from context): -- All tests, or a specific subset? -- Did a specific CI job fail? Which one? -- Is the build already done, or do they need to build first? - -### Step 2 — Check and Fix Prerequisites - -**One-command setup** — this script checks all prerequisites and auto-fixes issues: +Default command: ```bash -bash .claude/skills/mooncake-ci-local/scripts/check-prerequisites.sh +bash scripts/run_ci_test.sh ``` -**What it checks:** -1. ✓ Build directory exists -2. ✓ mooncake package installed (auto-installs via cmake --install if missing) -3. ✓ ctest available -4. ✓ Restarts all services (etcd, metadata server) in clean state -5. ✓ Verifies all services are healthy +What this script already covers: -**If you need to build first:** -```bash -mkdir build && cd build -cmake -G Ninja .. -DUSE_HTTP=ON -DUSE_ETCD=ON -DUSE_CXL=ON -DSTORE_USE_ETCD=ON -DCMAKE_BUILD_TYPE=Debug -cmake --build . -sudo cmake --install . -``` +- GitHub-like `paths-filter` against `origin/main` +- `typos` +- `scripts/code_format.sh --check` +- default CMake configure/build/install in `build-ci-local` +- `ctest` +- wheel build in `build-wheel-local` +- wheel installation validation +- `scripts/run_tests.sh` +- selected Python API and integration tests +- per-stage summary and logs under `local_test/run-ci-logs//` -**If script fails:** -- Build issues: See "Build Configurations" section below -- mooncake install fails: Try `pip install mooncake-wheel/dist/*.whl` manually -- etcd install fails: Download from https://github.com/etcd-io/etcd/releases +## Standard Agent Workflow -### Step 3 — Run and report +1. Run `bash scripts/run_ci_test.sh` from the repo root unless the user explicitly asks for a narrower subset. +2. Read the stage summary instead of dumping raw terminal output. +3. Report these items back to the user: + - passed stages + - failed stages + - blocked stages + - unsupported stages + - whether `paths-filter` skipped downstream stages + - the log directory under `local_test/run-ci-logs/...` +4. If there is a failure, inspect the corresponding stage log and summarize the root cause. -Run the relevant test layer. On failure: -1. Show the exact error message -2. Check if it's a service/env issue (most common) vs a real test failure -3. Suggest the fix (see common issues below) +## Common Options -## Common Local Test Issues - -**"mooncake_master found in /usr/local/bin" error in run_tests.sh:** -The test expects mooncake_master to come from the Python package, not a system install. -```bash -# Remove the system-installed binary: -sudo rm /usr/local/bin/mooncake_master -# Or use the wheel-installed one: -pip install mooncake-wheel/dist/*.whl -``` - -**etcd port conflict:** -```bash -pkill etcd && sleep 1 -etcd --advertise-client-urls http://127.0.0.1:2379 --listen-client-urls http://127.0.0.1:2379 & -``` - -**Metadata server port conflict:** -```bash -pkill -f bootstrap_server.py -pkill -f mooncake_http_metadata_server -``` - -**Tests hang (master not responding):** -```bash -pkill mooncake_master -sleep 2 -mooncake_master --default_kv_lease_ttl=500 & -sleep 1 -``` - -**torch/numpy not installed for tensor tests:** -```bash -pip install torch numpy safetensors packaging -``` - -**ctest shows no tests found:** -```bash -# Rebuild with unit tests enabled: -cd build -cmake .. -DBUILD_UNIT_TESTS=ON -cmake --build . -``` - -## Quick One-Liners +Force a full lane even if `paths-filter` would skip downstream stages: ```bash -# Run ALL C++ tests (after building with etcd + metadata server running): -# Note: full suite takes 5-15 minutes depending on hardware -cd build && MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -j --output-on-failure - -# Run only fast tests (skip slow integration tests): -cd build && MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -j --output-on-failure --exclude-regex "etcd|ha_test|redis" - -# Run ALL Python tests: -mooncake_http_metadata_server --port 8080 & sleep 1 && cd mooncake-wheel/tests && MC_METADATA_SERVER=http://127.0.0.1:8080/metadata MC_FORCE_TCP=true bash ../../scripts/run_tests.sh - -# Check code format (changed files only): -./scripts/code_format.sh --check --base origin/main - -# Full pre-commit check: -pre-commit run --all-files +bash scripts/run_ci_test.sh --skip-path-filter ``` + +Use another base ref: + +```bash +bash scripts/run_ci_test.sh --base origin/main +``` + +Auto-install missing dependencies: + +```bash +bash scripts/run_ci_test.sh --install-deps +``` + +Keep services running for follow-up debugging: + +```bash +bash scripts/run_ci_test.sh --keep-services +``` + +## Minimal Example + +User prompt: + +- 提交 PR 前,帮我跑一遍本地 CI 验证当前分支。 + +Expected action: + +```bash +bash scripts/run_ci_test.sh +``` + +If the user wants to ignore changed-path optimization and force the full lane: + +```bash +bash scripts/run_ci_test.sh --skip-path-filter +``` + +See also `.claude/skills/mooncake-ci-local/examples/minimal.md`. + +## How To Interpret Results + +- `passed`: the stage succeeded locally. +- `failed`: the stage reproduced a real local failure and needs investigation. +- `blocked`: local environment or dependency issue prevented execution. +- `unsupported`: intentionally not run in the local lane because it needs external platforms, special hardware, or a non-default build. + +If `paths-filter` skips downstream stages, explain that the current branch changed only non-source paths relative to the selected base. + +## Current Local Coverage + +Included by default: + +- spell check +- code format check +- default ASan CMake lane in `build-ci-local` +- `ctest` +- wheel build and installation test +- `scripts/run_tests.sh` +- selected Python API tests + +Unsupported by design in the default local lane: + +- Ascend jobs +- T-one integration jobs +- MUSA jobs +- Docker image build jobs +- CUDA 13 wheel jobs +- PG-backend tests absent from the default wheel build +- Python drain-http API stage in the local ASan lane + +## Targeted Reruns For Debugging + +Use targeted reruns only after the full script identifies a failing area, or when the user explicitly asks for a smaller scope. + +Rerun a specific C++ test pattern: + +```bash +cd build-ci-local +MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -R --output-on-failure +``` + +Rerun the Python wheel integration lane: + +```bash +source test_env/bin/activate +MC_STORE_MEMCPY=false TEST_SSD_OFFLOAD_IN_EVICT=true ./scripts/run_tests.sh +``` + +Rerun the safetensor unittest: + +```bash +source test_env/bin/activate +python -m unittest mooncake-wheel.tests.test_safetensor_functions +``` + +## Notes For The Agent + +- Prefer the repo script over rebuilding the CI workflow step by step. +- Preserve the separation between `build-ci-local` and `build-wheel-local`. +- Summarize failing stages from their logs instead of pasting raw output. +- If the user only asks whether the branch is safe before opening a PR, the default answer path is `bash scripts/run_ci_test.sh`. diff --git a/.claude/skills/mooncake-ci-local/examples/minimal.md b/.claude/skills/mooncake-ci-local/examples/minimal.md new file mode 100644 index 00000000..3d081c73 --- /dev/null +++ b/.claude/skills/mooncake-ci-local/examples/minimal.md @@ -0,0 +1,30 @@ +# Minimal Example + +Goal: validate the current branch before opening or submitting a PR. + +User prompt: + +- 提交 PR 前,帮我跑一遍本地 CI 验证当前分支。 + +Expected action: + +```bash +bash scripts/run_ci_test.sh +``` + +Force a full rerun even if `paths-filter` would skip downstream stages: + +```bash +bash scripts/run_ci_test.sh --skip-path-filter +``` + +Result location: + +- `local_test/run-ci-logs//` + +Typical report format: + +- passed stages +- failed or blocked stages +- unsupported stages +- first actionable failure and its log path -- 2.34.1 From 5893083ab13b0657d546c0bb4955a56284bbd252 Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Mon, 8 Jun 2026 09:49:51 +0800 Subject: [PATCH 197/382] [TENT] Validate minimum request size before XferDataDesc cast (#2351) Co-authored-by: Claude Opus 4.6 --- .../tent/src/runtime/control_plane.cpp | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mooncake-transfer-engine/tent/src/runtime/control_plane.cpp b/mooncake-transfer-engine/tent/src/runtime/control_plane.cpp index 49ca9abc..aa24e747 100644 --- a/mooncake-transfer-engine/tent/src/runtime/control_plane.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/control_plane.cpp @@ -243,6 +243,10 @@ void ControlService::onBootstrapRdma(const std::string_view& request, void ControlService::onSendData(const std::string_view& request, std::string& response) { + if (request.size() < sizeof(XferDataDesc)) { + response = "SendData failed: request too short"; + return; + } XferDataDesc* desc = (XferDataDesc*)request.data(); auto local_desc = manager_->getLocal().get(); auto peer_mem_addr = le64toh(desc->peer_mem_addr); @@ -263,6 +267,10 @@ void ControlService::onSendData(const std::string_view& request, void ControlService::onRecvData(const std::string_view& request, std::string& response) { + if (request.size() < sizeof(XferDataDesc)) { + response = "RecvData failed: request too short"; + return; + } XferDataDesc* desc = (XferDataDesc*)request.data(); auto local_desc = manager_->getLocal().get(); auto peer_mem_addr = le64toh(desc->peer_mem_addr); -- 2.34.1 From fc7b8eb9585d12a032e728991b6396e9ba8fb889 Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Mon, 8 Jun 2026 09:58:11 +0800 Subject: [PATCH 198/382] [TE] Fix data race, double-close, uninit, and off-by-one in RDMA transport (#2346) * [TE] Fix data race, double-close, uninit, off-by-one in RDMA transport - Remove 'static' from access_rights in registerLocalMemoryInternal to eliminate data race under concurrent registerLocalMemoryBatch. - Add event_fd_ = -1 after close(event_fd_) on 3 error paths in RdmaContext::construct() to prevent double-close in destructor. - Value-initialize comp_channel_ array to zero so partial-failure cleanup in deconstruct() sees nullptr instead of garbage pointers. - Fix off-by-one: change > to >= in doSetupConnection bounds check to prevent OOB access when qp_index equals qp_list_.size(). Co-Authored-By: Claude Opus 4.6 * [TE] Value-initialize wr_depth_list_ to fix sibling uninit bug Address review: wr_depth_list_ has the same uninitialized-array bug as comp_channel_. Partial QP creation failure leaves garbage values that corrupt the CQ outstanding counter in destructor. Co-Authored-By: Claude Opus 4.6 * Init wr_depth_list_ to nullptr, guard deconstructLocked() Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 --- .../transport/rdma_transport/rdma_context.cpp | 5 +++- .../rdma_transport/rdma_endpoint.cpp | 24 +++++++++++-------- .../rdma_transport/rdma_transport.cpp | 2 +- 3 files changed, 19 insertions(+), 12 deletions(-) diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index 46814010..5c055f47 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -209,7 +209,7 @@ int RdmaContext::construct(size_t num_cq_list, size_t num_comp_channels, } num_comp_channel_ = num_comp_channels; - comp_channel_ = new ibv_comp_channel *[num_comp_channels]; + comp_channel_ = new ibv_comp_channel *[num_comp_channels](); for (size_t i = 0; i < num_comp_channels; ++i) { comp_channel_[i] = ibv_create_comp_channel(context_); if (!comp_channel_[i]) { @@ -228,6 +228,7 @@ int RdmaContext::construct(size_t num_cq_list, size_t num_comp_channels, if (joinNonblockingPollList(event_fd_, context_->async_fd)) { LOG(ERROR) << "Failed to register context async fd to epoll"; close(event_fd_); + event_fd_ = -1; return ERR_CONTEXT; } @@ -236,6 +237,7 @@ int RdmaContext::construct(size_t num_cq_list, size_t num_comp_channels, LOG(ERROR) << "Failed to register completion channel " << i << " to epoll"; close(event_fd_); + event_fd_ = -1; return ERR_CONTEXT; } @@ -248,6 +250,7 @@ int RdmaContext::construct(size_t num_cq_list, size_t num_comp_channels, if (!cq) { PLOG(ERROR) << "Failed to create completion queue"; close(event_fd_); + event_fd_ = -1; return ERR_CONTEXT; } cq_list_[i].native = cq; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp index c1ee94d4..29882865 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp @@ -67,6 +67,7 @@ static void rememberAutoGidSelection( RdmaEndPoint::RdmaEndPoint(RdmaContext &context) : context_(context), status_(INITIALIZING), + wr_depth_list_(nullptr), active_(true), cq_outstanding_(nullptr) {} @@ -95,7 +96,7 @@ int RdmaEndPoint::construct(ibv_cq *cq, size_t num_qp_list, max_sge_per_wr_ = max_sge_per_wr; max_inline_bytes_ = max_inline_bytes; - wr_depth_list_ = new volatile int[num_qp_list]; + wr_depth_list_ = new volatile int[num_qp_list](); if (!wr_depth_list_) { LOG(ERROR) << "Failed to allocate memory for work request depth list"; return ERR_MEMORY; @@ -162,15 +163,18 @@ int RdmaEndPoint::deconstructLocked() { // Adjust cq_outstanding_ before destroying QPs, so the counter is // always corrected even if ibv_destroy_qp fails and we return early. bool displayed = false; - for (size_t i = 0; i < qp_list_.size(); ++i) { - if (wr_depth_list_[i] != 0) { - if (!displayed) { - LOG(WARNING) << "Outstanding work requests found, CQ will not " - "be generated"; - displayed = true; + if (wr_depth_list_) { + for (size_t i = 0; i < qp_list_.size(); ++i) { + if (wr_depth_list_[i] != 0) { + if (!displayed) { + LOG(WARNING) + << "Outstanding work requests found, CQ will not " + "be generated"; + displayed = true; + } + __sync_fetch_and_sub(cq_outstanding_, wr_depth_list_[i]); + wr_depth_list_[i] = 0; } - __sync_fetch_and_sub(cq_outstanding_, wr_depth_list_[i]); - wr_depth_list_[i] = 0; } } @@ -893,7 +897,7 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, uint16_t peer_lid, uint32_t peer_qp_num, int local_gid_index, std::string *reply_msg, SetupConnectionFailureInfo *failure_info) { - if (qp_index < 0 || qp_index > (int)qp_list_.size()) + if (qp_index < 0 || qp_index >= (int)qp_list_.size()) return ERR_INVALID_ARGUMENT; auto &qp = qp_list_[qp_index]; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp index 753cc1b9..6273a262 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp @@ -207,7 +207,7 @@ int RdmaTransport::registerLocalMemoryInternal(void *addr, size_t length, IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_READ; - static int access_rights = kBaseAccessRights; + int access_rights = kBaseAccessRights; if (MCIbRelaxedOrderingEnabled) { access_rights |= IBV_ACCESS_RELAXED_ORDERING; } -- 2.34.1 From 7b8f386144c23206ffe4f5cbb9e07b4f3d0c6bb6 Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Mon, 8 Jun 2026 09:58:46 +0800 Subject: [PATCH 199/382] [TE] Harden config parsing: remove exit() and guard stoi (#2344) - Remove exit(EXIT_FAILURE) on invalid MC_MTU so the process continues with the default IBV_MTU_4096 instead of crashing. - Wrap MC_HANDSHAKE_LISTEN_BACKLOG std::stoi in try-catch to match MC_PKEY_INDEX / MC_IB_TC pattern and prevent crash on non-numeric input. Co-authored-by: Claude Opus 4.6 --- mooncake-transfer-engine/src/config.cpp | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/mooncake-transfer-engine/src/config.cpp b/mooncake-transfer-engine/src/config.cpp index 44fd0393..4d987a60 100644 --- a/mooncake-transfer-engine/src/config.cpp +++ b/mooncake-transfer-engine/src/config.cpp @@ -158,7 +158,6 @@ void loadGlobalConfig(GlobalConfig& config) { else { LOG(ERROR) << "Ignore value from environment variable MC_MTU, it " "should be 512|1024|2048|4096"; - exit(EXIT_FAILURE); } } @@ -244,12 +243,18 @@ void loadGlobalConfig(GlobalConfig& config) { const char* handshake_listen_backlog = std::getenv("MC_HANDSHAKE_LISTEN_BACKLOG"); if (handshake_listen_backlog) { - int val = std::stoi(handshake_listen_backlog); - if (val > 0) { - config.handshake_listen_backlog = val; - } else { - LOG(WARNING) << "Ignore value from environment variable " - "MC_HANDSHAKE_LISTEN_BACKLOG"; + try { + int val = std::stoi(handshake_listen_backlog); + if (val > 0) { + config.handshake_listen_backlog = val; + } else { + LOG(WARNING) << "Ignore value from environment variable " + "MC_HANDSHAKE_LISTEN_BACKLOG"; + } + } catch (const std::exception& e) { + LOG(WARNING) << "Invalid MC_HANDSHAKE_LISTEN_BACKLOG environment " + "value: " + << handshake_listen_backlog << ". Error: " << e.what(); } } -- 2.34.1 From c142b40590259360196d8e504b2193382529e7b4 Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Mon, 8 Jun 2026 10:00:45 +0800 Subject: [PATCH 200/382] [TE] Fix error-path safety: freeaddrinfo leak, null deref, OOB read (#2343) - Fix freeaddrinfo leak on ERR_MALFORMED_JSON early returns in sendNotify, sendProbe, send, and exchangeMetadata. - Add null check for getSegmentDescByID in sendNotifyByID to prevent null pointer dereference on invalid segment ID. - Guard readString against zero-length network input to prevent OOB access on empty buffer. Co-authored-by: Claude Opus 4.6 --- mooncake-transfer-engine/include/common.h | 5 +++++ mooncake-transfer-engine/src/transfer_engine_impl.cpp | 4 ++++ mooncake-transfer-engine/src/transfer_metadata_plugin.cpp | 4 ++++ 3 files changed, 13 insertions(+) diff --git a/mooncake-transfer-engine/include/common.h b/mooncake-transfer-engine/include/common.h index 621c256d..5ae9a664 100644 --- a/mooncake-transfer-engine/include/common.h +++ b/mooncake-transfer-engine/include/common.h @@ -458,6 +458,11 @@ static inline std::pair readString(int fd) { return {type, ""}; } + if (length == 0) { + LOG(ERROR) << "readString: zero length from socket"; + return {type, ""}; + } + std::string str; std::vector buffer(length); n = readFully(fd, buffer.data(), length); diff --git a/mooncake-transfer-engine/src/transfer_engine_impl.cpp b/mooncake-transfer-engine/src/transfer_engine_impl.cpp index 08818bba..f7939557 100644 --- a/mooncake-transfer-engine/src/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/src/transfer_engine_impl.cpp @@ -448,6 +448,10 @@ int TransferEngineImpl::getNotifies( int TransferEngineImpl::sendNotifyByID( SegmentID target_id, TransferMetadata::NotifyDesc notify_msg) { auto desc = metadata_->getSegmentDescByID(target_id); + if (!desc) { + LOG(ERROR) << "sendNotifyByID: invalid segment ID " << target_id; + return ERR_METADATA; + } Transport::NotifyDesc peer_desc; int ret = metadata_->sendNotify(desc->name, notify_msg, peer_desc); return ret; diff --git a/mooncake-transfer-engine/src/transfer_metadata_plugin.cpp b/mooncake-transfer-engine/src/transfer_metadata_plugin.cpp index ab40c538..72daced2 100644 --- a/mooncake-transfer-engine/src/transfer_metadata_plugin.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata_plugin.cpp @@ -856,6 +856,7 @@ struct SocketHandShakePlugin : public HandShakePlugin { return 0; } if (ret == ERR_MALFORMED_JSON) { + freeaddrinfo(result); return ret; } } @@ -891,6 +892,7 @@ struct SocketHandShakePlugin : public HandShakePlugin { return 0; } if (ret == ERR_MALFORMED_JSON) { + freeaddrinfo(result); return ret; } } @@ -926,6 +928,7 @@ struct SocketHandShakePlugin : public HandShakePlugin { return 0; } if (ret == ERR_MALFORMED_JSON) { + freeaddrinfo(result); return ret; } } @@ -1034,6 +1037,7 @@ struct SocketHandShakePlugin : public HandShakePlugin { return 0; } if (ret == ERR_MALFORMED_JSON) { + freeaddrinfo(result); return ret; } } -- 2.34.1 From 5771febf4027b5cdc152bdc669425027d5477cdf Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Mon, 8 Jun 2026 10:06:20 +0800 Subject: [PATCH 201/382] [TE] Fix lock leak, missing transport_, and empty entries UB (#2349) * [TE] Fix lock leak, missing transport_, and empty entries UB Co-Authored-By: Claude Opus 4.6 * Fix pre-existing clang-format violations in dump.cpp Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 --- .../include/transfer_engine_impl.h | 6 +++++ .../src/multi_transport.cpp | 1 + .../src/transfer_metadata_dump.cpp | 22 ++++++++++--------- 3 files changed, 19 insertions(+), 10 deletions(-) diff --git a/mooncake-transfer-engine/include/transfer_engine_impl.h b/mooncake-transfer-engine/include/transfer_engine_impl.h index f4584e27..b1e4fff7 100644 --- a/mooncake-transfer-engine/include/transfer_engine_impl.h +++ b/mooncake-transfer-engine/include/transfer_engine_impl.h @@ -136,6 +136,9 @@ class TransferEngineImpl { Status submitTransferWithNotify(BatchID batch_id, const std::vector& entries, TransferMetadata::NotifyDesc notify_msg) { + if (entries.empty()) { + return Status::InvalidArgument("entries must not be empty"); + } auto target_id = entries[0].target_id; Status s = multi_transports_->submitTransfer(batch_id, entries); if (!s.ok()) { @@ -194,6 +197,9 @@ class TransferEngineImpl { Status mp_submitTransferWithNotify( BatchID batch_id, const std::vector& entries, TransferMetadata::NotifyDesc notify_msg, std::string& proto) { + if (entries.empty()) { + return Status::InvalidArgument("entries must not be empty"); + } auto target_id = entries[0].target_id; Status s = multi_transports_->mp_submitTransfer(batch_id, entries, proto); diff --git a/mooncake-transfer-engine/src/multi_transport.cpp b/mooncake-transfer-engine/src/multi_transport.cpp index dbe81ff7..7899e5f9 100644 --- a/mooncake-transfer-engine/src/multi_transport.cpp +++ b/mooncake-transfer-engine/src/multi_transport.cpp @@ -167,6 +167,7 @@ Status MultiTransport::mp_submitTransfer( assert(transport); auto& task = batch_desc.task_list[task_id]; task.batch_id = batch_id; + task.transport_ = transport; #ifdef USE_ASCEND_HETEROGENEOUS task.request = const_cast(&request); #else diff --git a/mooncake-transfer-engine/src/transfer_metadata_dump.cpp b/mooncake-transfer-engine/src/transfer_metadata_dump.cpp index dc95d697..093cd502 100644 --- a/mooncake-transfer-engine/src/transfer_metadata_dump.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata_dump.cpp @@ -24,21 +24,21 @@ void TransferMetadata::SegmentDesc::dump() const { LOG(INFO) << " protocol: " << protocol; LOG(INFO) << " topology: " << topology.toString(); LOG(INFO) << " devices: "; - for (auto &device : devices) { + for (auto& device : devices) { LOG(INFO) << " device name " << device.name << ", lid " << device.lid << ", " << device.gid; } LOG(INFO) << " buffers: "; - for (auto &buffer : buffers) { + for (auto& buffer : buffers) { LOG(INFO) << " buffer type " << buffer.name << ", address " - << (void *)buffer.addr << "--" - << (void *)(buffer.addr + buffer.length); + << (void*)buffer.addr << "--" + << (void*)(buffer.addr + buffer.length); } LOG(INFO) << " nvmeof buffers: " << nvmeof_buffers.size() << " items"; LOG(INFO) << " timestamp: " << timestamp; } -void TransferMetadata::dumpMetadataContent(const std::string &segment_name, +void TransferMetadata::dumpMetadataContent(const std::string& segment_name, uint64_t offset, uint64_t length) { thread_local uint64_t last_ts = 0; uint64_t current_ts = getCurrentTimeInNano(); @@ -47,13 +47,15 @@ void TransferMetadata::dumpMetadataContent(const std::string &segment_name, auto segment_locked = segment_lock_.tryLockShared(); auto rpc_meta_locked = rpc_meta_lock_.tryLockShared(); if (!segment_locked || !rpc_meta_locked) { + if (rpc_meta_locked) rpc_meta_lock_.unlockShared(); + if (segment_locked) segment_lock_.unlockShared(); return; } if (current_ts - last_ts > kMinDisplayThreshold || globalConfig().trace) { LOG(INFO) << "Failed to get segment descriptor for segment " - << segment_name << " address " << (void *)offset << "--" - << (void *)(offset + length); + << segment_name << " address " << (void*)offset << "--" + << (void*)(offset + length); dumpMetadataContentUnlocked(); last_ts = current_ts; } @@ -67,8 +69,8 @@ void TransferMetadata::dumpMetadataContentUnlocked() { LOG(INFO) << "TransferMetadata::dumpMetadataContent"; LOG(INFO) << "-----------------------------------------------------------"; LOG(INFO) << "=== Cached Segment Descriptors ==="; - for (auto &entry : segment_id_to_desc_map_) { - auto &desc = entry.second; + for (auto& entry : segment_id_to_desc_map_) { + auto& desc = entry.second; if (!desc) { LOG(INFO) << "segment id: " << entry.first << ", ref object nil"; } else { @@ -81,7 +83,7 @@ void TransferMetadata::dumpMetadataContentUnlocked() { LOG(INFO) << "location: " << local_rpc_meta_.ip_or_host_name << ":" << local_rpc_meta_.rpc_port; LOG(INFO) << "=== Remote RPC Routes ==="; - for (auto &entry : rpc_meta_map_) { + for (auto& entry : rpc_meta_map_) { LOG(INFO) << "segment name: " << entry.first << ", location: " << entry.second.ip_or_host_name << ":" << entry.second.rpc_port; -- 2.34.1 From 5509e640381921f94418de12e0abc1deb149ecd5 Mon Sep 17 00:00:00 2001 From: Enigmo-x <1253115086@qq.com> Date: Mon, 8 Jun 2026 10:28:54 +0800 Subject: [PATCH 202/382] feat(store): add NoF SSD deployment tools and e2e coverage (#2273) Co-authored-by: zhuwentao <1357420890@qq.com> --- .../deployment/nvmf-ssd-deployment-guide.md | 286 +++++ docs/source/getting_started/build.md | 18 + docs/source/index.md | 1 + mooncake-integration/CMakeLists.txt | 9 + mooncake-integration/store/store_py.cpp | 46 + mooncake-store/benchmarks/store_kv_bench.md | 204 ++++ mooncake-store/benchmarks/store_kv_bench.py | 1028 +++++++++++++++++ mooncake-store/include/ssd_register_client.h | 39 + mooncake-store/src/CMakeLists.txt | 1 + mooncake-store/src/ssd_register_client.cpp | 138 +++ mooncake-store/tests/e2e/readme.md | 78 +- .../tests/e2e/run_nof_heartbeat_tcp_e2e.sh | 245 ++++ mooncake-store/tests/e2e/store_client_e2e.py | 108 ++ .../mooncake/mooncake_ssd_register.py | 328 ++++++ .../mooncake/mooncake_ssd_unregister.py | 361 ++++++ mooncake-wheel/mooncake/spdk_tgt_create.py | 520 +++++++++ 16 files changed, 3409 insertions(+), 1 deletion(-) create mode 100644 docs/source/deployment/nvmf-ssd-deployment-guide.md create mode 100644 mooncake-store/benchmarks/store_kv_bench.md create mode 100644 mooncake-store/benchmarks/store_kv_bench.py create mode 100644 mooncake-store/include/ssd_register_client.h create mode 100644 mooncake-store/src/ssd_register_client.cpp create mode 100644 mooncake-store/tests/e2e/run_nof_heartbeat_tcp_e2e.sh create mode 100644 mooncake-store/tests/e2e/store_client_e2e.py create mode 100644 mooncake-wheel/mooncake/mooncake_ssd_register.py create mode 100644 mooncake-wheel/mooncake/mooncake_ssd_unregister.py create mode 100644 mooncake-wheel/mooncake/spdk_tgt_create.py diff --git a/docs/source/deployment/nvmf-ssd-deployment-guide.md b/docs/source/deployment/nvmf-ssd-deployment-guide.md new file mode 100644 index 00000000..eb855838 --- /dev/null +++ b/docs/source/deployment/nvmf-ssd-deployment-guide.md @@ -0,0 +1,286 @@ +# Mooncake NVMe-oF SSD Pool Deployment Guide + +## Overview + +This guide shows how to attach an NVMe-oF SSD pool to Mooncake Store. The +deployment has two main phases: + +- Start Mooncake services built with NoF support enabled. +- Create SPDK NVMe-oF targets on SSD pool nodes and register their namespaces + with the Mooncake master. + +After registration, the master reports the registered NVMe-oF namespaces as a +remote SSD pool in its metrics, and clients can place NoF replicas through +Mooncake Store. + +## 1. Build Mooncake with NoF Support + +Follow the "Build with NVMe-oF SSD Pool" section in the +[Build Guide](../getting_started/build.md) to install SPDK dependencies and +build Mooncake with `-DUSE_NOF=ON`. + +## 2. Deploy Mooncake Services + +### 2.1 Node Topology + +- **Mooncake service node**: 192.168.65.81. This node runs the master, metadata, and store services. +- **SSD pool nodes**: 192.168.65.56 and 192.168.65.57. These nodes provide SSD storage resources. + +### 2.2 Deploy the Master Service + +```bash +mooncake_master --rpc_address=192.168.65.81 +``` + +### 2.3 Deploy the Metadata Service + +```bash +python3 -m mooncake.http_metadata_server --host=192.168.65.81 --port=8080 +``` + +If an aiohttp-related error occurs during startup, install aiohttp: + +```bash +pip3 install aiohttp +``` + +### 2.4 Deploy the Store Service + +#### Configure `store_service.json` + +Create `store_service.json` under `/home`: + +```json +{ + "local_hostname": "localhost", + "metadata_server": "http://192.168.65.81:8080/metadata", + "master_server_address": "192.168.65.81:50051", + "protocol": "rdma", + "device_name": "mlx5_0", + "global_segment_size": "50gb", + "local_buffer_size": 0 +} +``` + +**Notes**: + +- `device_name`: Run `ibv_devices` on node 192.168.65.81 to check the RDMA device name. + +#### Start the Service + +The store service initializes the SPDK environment during startup. Configure hugepages on the store service node, 192.168.65.81: + +```bash +echo 512 > /proc/sys/vm/nr_hugepages +``` + +Only a small number of hugepages is required during startup. In most cases, 512 hugepages are sufficient. + +Start the store service: + +```bash +python3 -m mooncake.mooncake_store_service --config=/home/store_service.json --port=8081 +``` + +If a timeout error occurs during startup, check whether a proxy is configured on node 192.168.65.81. If a proxy is configured, unset the proxy configuration and try again. + +## 3. Deploy the NVMe-oF SSD Pool + +### 3.1 Prerequisites + +1. Configure passwordless SSH login from the Mooncake node, 192.168.65.81, to the SSD pool nodes, 192.168.65.56 and 192.168.65.57. + See [OpenSSH key-based authentication](https://help.ubuntu.com/community/SSH/OpenSSH/Keys). +2. Build SPDK on each SSD pool node in advance. + See [SPDK build instructions](https://github.com/spdk/spdk/blob/master/README.md#build). + +### 3.2 Install SSH Dependencies + +```bash +python3 -m pip install "paramiko>=3.4.0" +``` + +### 3.3 Deploy the SSD Pool + +#### Deployment Command + +```bash +python3 -m mooncake.spdk_tgt_create \ + --spdk_target_info="ip:192.168.65.56 path:/home/spdk pci:0000:01:00.0,0000:02:00.0" \ + --spdk_target_info="ip:192.168.65.57 path:/home/spdk" \ + --core-mask=0xff \ + --transport-type=RDMA \ + --max-queue-depth=128 \ + --max-io-qpairs-per-ctrlr=127 \ + --max-io-size=4096 \ + --in-capsule-data-size=131072 \ + --io-unit-size=131072 \ + --max-aq-depth=128 \ + --num-shared-buffers=4096 \ + --buf-cache-size=32 +``` + +#### Parameters + +| Parameter | Description | +|-----------|-------------| +| `ip` | IP address of the target node. | +| `path` | SPDK installation path on the target node. | +| `pci` | PCI addresses of SSDs to register with the target. Use commas to separate multiple PCI addresses. If this field is omitted, SPDK-ready or unmounted NVMe devices on the target node are registered. | +| `--core-mask` | CPU core mask used to start `nvmf_tgt` with `-m`. The default value is `0xff`. | + +**Tip**: Run `/path/scripts/setup.sh status` on a target node to list available PCI addresses. + +#### Transport Options + +The transport options are passed to the SPDK `nvmf_create_transport` RPC. If an option is not specified, the tool uses the default value listed below. + +| Option | Default | Description | +|--------|---------|-------------| +| `--transport-type` | `RDMA` | NVMe-oF transport type. | +| `--max-queue-depth` | `128` | Maximum number of outstanding I/O operations per queue. | +| `--max-io-qpairs-per-ctrlr` | `127` | Maximum number of I/O queue pairs per controller. | +| `--max-io-size` | `4096` | Maximum I/O size, in bytes. | +| `--in-capsule-data-size` | `131072` | Maximum in-capsule data size, in bytes. | +| `--io-unit-size` | `131072` | I/O unit size, in bytes. | +| `--max-aq-depth` | `128` | Maximum number of admin commands per admin queue. | +| `--num-shared-buffers` | `4096` | Number of pooled data buffers available to the transport. | +| `--buf-cache-size` | `32` | Number of shared buffers reserved for each poll group. | + +## 4. Register the NVMe-oF SSD Pool + +### 4.1 Register All SSDs + +```bash +python3 -m mooncake.mooncake_ssd_register \ + --master_server_address=192.168.65.81:50051 \ + --spdk_target_info="ip:192.168.65.56 path:/home/spdk" \ + --spdk_target_info="ip:192.168.65.57 path:/root/spdk" +``` + +#### Parameters + +| Parameter | Description | +|-----------|-------------| +| `--master_server_address` | IP address and port of the master service node. The default port is 50051. | +| `--spdk_target_info` | Target node information, including `ip`, the node IP address, and `path`, the SPDK installation path. | +| `--username` | SSH username used to connect to target nodes. The default value is `root`. | +| `--port` | SSH port used to connect to target nodes. The default value is `22`. | +| `--password` | SSH password used to connect to target nodes. | +| `--key-file` | SSH private key file used to connect to target nodes. | + +## 5. Unregister the NVMe-oF SSD Pool + +### 5.1 Unregister a Specific SSD + +```bash +python3 -m mooncake.mooncake_ssd_unregister \ + --master_server_address=192.168.65.81:50051 \ + --spdk_target_info="ip:192.168.65.56 ns:1 nqn:nqn.2016-06.io.spdk:cnode1" +``` + +#### Parameters + +| Parameter | Description | +|-----------|-------------| +| `--master_server_address` | IP address and port of the master service node. The default port is 50051. | +| `--spdk_target_info` | Disk information to unregister, including `ip`, the node IP address, `ns`, the namespace ID, and `nqn`, the subsystem NQN. | +| `--username` | SSH username used to connect to target nodes. The default value is `root`. | +| `--port` | SSH port used to connect to target nodes. The default value is `22`. | +| `--password` | SSH password used to connect to target nodes. | +| `--key-file` | SSH private key file used to connect to target nodes. | + +### 5.2 Get Target Disk Information + +Enter the SPDK directory on the target node and run the following commands. + +1. Show subsystem information, including NQN and namespace IDs: + +```bash +./scripts/rpc.py nvmf_get_subsystems +``` + +2. Show disk details, including block size and PCI address: + +```bash +./scripts/rpc.py bdev_get_bdevs +``` + +## 6. Performance Tests + +### 6.1 Use the Built-in Benchmark Tool + +```bash +./build/mooncake-store/benchmarks/nof_worker_pool_bench \ + --endpoints='traddr:192.168.65.56 trsvcid:4420 subnqn:nqn.2016-06.io.spdk:cnode1 trtype:RDMA adrfam:IPv4 ns:1, traddr:192.168.65.56 trsvcid:4420 subnqn:nqn.2016-06.io.spdk:cnode1 trtype:RDMA adrfam:IPv4 ns:2' \ + --op=read \ + --io_size=1048576 \ + --iodepth=8 \ + --warmup_sec=3 \ + --duration_sec=30 +``` + +#### Parameters + +| Parameter | Description | +|-----------|-------------| +| `--endpoints` | Disk information used for the test. | +| `--op` | I/O operation type, either `read` or `write`. | +| `--io_size` | Block size in bytes. | +| `--iodepth` | Read/write queue depth. | +| `--warmup_sec` | Warmup duration in seconds. | +| `--duration_sec` | Test duration in seconds. | + +### 6.2 Use NoF with vLLM + LMCache + +For the general VLLM + LMCache + Mooncake deployment flow, see +[vLLM V1 Disaggregated Serving with Mooncake Store and LMCache](../getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md). +After the NVMe-oF SSD pool is registered with Mooncake, add the NoF-specific +Mooncake configuration below. + +#### NoF Environment Variables + +```bash +export LMCACHE_CONFIG_FILE="/path/vllm-lmcache-mooncake-config.yaml" +export MC_NOF_WORKERS=4 +export MC_NOF_SUBMIT_CHUNK_BYTES=$((1 << 17)) # 128KB +export MC_NOF_INFLIGHT_BYTES_LIMIT=$((1 << 25)) # 32MB +``` + +#### NoF LMCache Configuration + +```yaml +chunk_size: 256 +remote_url: "mooncakestore://192.168.65.81:50051/" +remote_serde: "naive" +local_cpu: True +max_local_cpu_size: 8 +enable_mooncake_nof_pool: True + +extra_config: + local_hostname: "localhost" + metadata_server: "http://192.168.65.81:8080/metadata" + master_server_address: "192.168.65.81:50051" + global_segment_size: 0 + local_buffer_size: 1073741824 + protocol: "rdma" + device_name: "mlx5_0" +``` + +**Notes**: + +- `enable_mooncake_nof_pool=True` enables writing KV cache objects to the + registered NoF pool. +- `global_segment_size: 0` means the inference process does not contribute a + memory segment to the Mooncake cluster. +- Keep `local_buffer_size` non-zero because the client still needs local + staging buffers for Mooncake transfers. +- The parameters in `extra_config` should use the same Mooncake master, + metadata server, protocol, and RDMA device as the store service. + +| Environment Variable | Description | Default | +|----------------------|-------------|---------| +| `MC_NOF_WORKERS` | Number of worker threads used to process SPDK NoF I/O operations. | 4 | +| `MC_NOF_SUBMIT_CHUNK_BYTES` | Size of each I/O operation submitted to SPDK. | 128KB | +| `MC_NOF_INFLIGHT_BYTES_LIMIT` | Maximum number of in-flight I/O bytes allowed in the system. | 32MB | + +These three parameters together provide QoS control for SPDK NoF I/O. diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index ec0cd121..63f364b2 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -45,6 +45,24 @@ pip install mooncake-transfer-engine-non-cuda sudo make install ``` +### Build with NVMe-oF SSD Pool + +To enable the NVMe-oF SSD pool, install the SPDK dependencies and build +Mooncake with `USE_NOF` enabled: + +```bash +bash dependencies.sh --with-spdk + +mkdir build +cd build +cmake .. -DUSE_NOF=ON +make -j +sudo make install +``` + +`-DUSE_NOF=ON` builds the NoF registration APIs and deployment tools. Use +`-DUSE_NOF=OFF` or omit the option when the NVMe-oF SSD pool is not needed. + ## Manual Build ### Recommended Version diff --git a/docs/source/index.md b/docs/source/index.md index e5337241..3f782f81 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -142,6 +142,7 @@ troubleshooting/troubleshooting :maxdepth: 2 deployment/mooncake-store-deployment-guide +deployment/nvmf-ssd-deployment-guide ::: % Community diff --git a/mooncake-integration/CMakeLists.txt b/mooncake-integration/CMakeLists.txt index 379a5e03..b7fb78d9 100644 --- a/mooncake-integration/CMakeLists.txt +++ b/mooncake-integration/CMakeLists.txt @@ -161,6 +161,15 @@ if(WITH_STORE) "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_config.py" "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/cli.py" DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) + + if(USE_NOF) + install( + FILES + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_ssd_register.py" + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/mooncake_ssd_unregister.py" + "${CMAKE_CURRENT_SOURCE_DIR}/../mooncake-wheel/mooncake/spdk_tgt_create.py" + DESTINATION ${PYTHON_SYS_PATH}/${PYTHON_PACKAGE_NAME}) + endif() endif() install(DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/${PYTHON_PACKAGE_NAME}/ diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index 032e7528..a8b98f3a 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -11,6 +11,8 @@ #include "dummy_client.h" #include "real_client.h" #include "types.h" +#include "memory_alloc.h" +#include "ssd_register_client.h" #include // for atexit #include @@ -1642,6 +1644,21 @@ class MooncakeHostMemAllocatorPyWrapper { ~MooncakeHostMemAllocatorPyWrapper() { shm_helper_ = nullptr; } }; +uintptr_t get_alloc_func_addr() { + return reinterpret_cast(&hugepage_memory_alloc); +} + +uintptr_t get_free_func_addr() { + return reinterpret_cast(&hugepage_memory_free); +} + +class MooncakeDistributedNoFRegisterPyWrapper { + public: + std::shared_ptr register_{nullptr}; + + MooncakeDistributedNoFRegisterPyWrapper() = default; +}; + PYBIND11_MODULE(store, m) { // Object data type classification py::enum_(m, "ObjectDataType") @@ -1661,10 +1678,13 @@ PYBIND11_MODULE(store, m) { py::class_(m, "ReplicateConfig") .def(py::init<>()) .def_readwrite("replica_num", &ReplicateConfig::replica_num) + .def_readwrite("nof_replica_num", &ReplicateConfig::nof_replica_num) .def_readwrite("with_soft_pin", &ReplicateConfig::with_soft_pin) .def_readwrite("with_hard_pin", &ReplicateConfig::with_hard_pin) .def_readwrite("preferred_segments", &ReplicateConfig::preferred_segments) + .def_readwrite("preferred_nof_segments", + &ReplicateConfig::preferred_nof_segments) .def_readwrite("preferred_segment", &ReplicateConfig::preferred_segment) .def_readwrite("prefer_alloc_in_same_node", &ReplicateConfig::prefer_alloc_in_same_node) @@ -1871,6 +1891,29 @@ PYBIND11_MODULE(store, m) { }) .def_readwrite("parallelism", &ReadTargetSpec::parallelism); + py::class_( + m, "MooncakeDistributedNoFRegister") + .def(py::init<>()) + .def("real_register", + [](MooncakeDistributedNoFRegisterPyWrapper &self, + const std::string &nqn = "", size_t nsid = 1, + const std::string &traddr = "", size_t trsvcid = 4420, + uintptr_t base = 0x0, size_t size = 1024, + const std::string &master_server_addr = "127.0.0.1:50051") { + self.register_ = std::make_shared(); + return self.register_->set_register(nqn, nsid, traddr, trsvcid, + base, size, + master_server_addr); + }) + .def("real_unregister_by_endpoint", + [](MooncakeDistributedNoFRegisterPyWrapper &self, + const std::string &nqn = "", size_t nsid = 1, + const std::string &traddr = "", size_t trsvcid = 4420, + const std::string &master_server_addr = "127.0.0.1:50051") { + self.register_ = std::make_shared(); + return self.register_->set_unregister_by_endpoint( + nqn, nsid, traddr, trsvcid, master_server_addr); + }); // Create a wrapper that exposes DistributedObjectStore with Python-specific // methods // Helper function to extract PyClient shared_ptr from @@ -2845,6 +2888,9 @@ PYBIND11_MODULE(store, m) { "Bind the current thread and memory allocation preference to the " "specified NUMA node"); + m.def("get_alloc_func_addr", &get_alloc_func_addr); + m.def("get_free_func_addr", &get_free_func_addr); + // Add EngramStore bindings mooncake::engram::bind_engram_store(m); } diff --git a/mooncake-store/benchmarks/store_kv_bench.md b/mooncake-store/benchmarks/store_kv_bench.md new file mode 100644 index 00000000..4d8f85a5 --- /dev/null +++ b/mooncake-store/benchmarks/store_kv_bench.md @@ -0,0 +1,204 @@ +# `store_kv_bench.py` + +`store_kv_bench.py` is a Mooncake Store end-to-end KV benchmark tool. It talks +to a real Mooncake cluster through the Python `store` binding and can exercise +`put/get` as well as zero-copy `put_from/get_into` style APIs. + +## Scope + +This tool focuses on object-semantic benchmark scenarios: + +- Functional verification with read-after-write validation +- Dataset fill for eviction / capacity tests +- Pure write performance +- Pure read performance +- Read/write mixed mode with "existing-object read + new-object write" + +Fault injection, NoF register / unregister, heartbeat trigger, memory segment +unmount, and target-side operations are intentionally out of scope. The tool +supports phase gaps so external tools can finish those operations before the +next phase continues. + +## Supported Scenarios + +- `verify_write` + - Fixed-count write followed by full readback verification +- `fill` + - Fixed-count write used for filling a dataset / eviction watermark +- `write_perf` + - Time-based or fixed-count write benchmark +- `read_perf` + - Optional prepare-write phase, then read performance benchmark +- `mixed_rw` + - Optional prepare-write phase, then mixed "read prepared objects + write new objects" + +## APIs + +- `--io-api=plain` + - Single object: + - `put` + - `get` + - Batch: + - `put_batch` + - `get_batch` +- `--io-api=zcopy` + - Single object: + - `put_from` + - `get_into` + - Batch: + - `batch_put_from` + - `batch_get_into` + +`zcopy` mode automatically allocates temporary user buffers and registers them +with `register_buffer`. + +## Key Rules + +- Keys are generated deterministically: + - `{prefix padded/truncated to fit}{16-digit object id}` +- The same `key-prefix`, `key-size`, and `object-id-start` produce the same key sequence +- `verify` currently requires `pattern` +- Any write-involved scenario requires `value-size` to be 512-byte aligned +- `memory-replica-num` and `nof-replica-num` cannot both be `0` +- `prepare-objects` + - Controls how many objects are written by the prepare phase + - `0` means reuse `nr-objects` + +## Phase Gap + +Phase gaps are used when an external tool needs time to inject a fault or do an +unmount / remount operation. + +- `--phase-gap-mode=none` + - Continue immediately +- `--phase-gap-mode=sleep --phase-gap-sec=N` + - Sleep before the next phase +- `--phase-gap-mode=manual` + - Wait for Enter +- `--phase-gap-mode=file --phase-gap-file=/tmp/bench.ready` + - Wait until the file exists + +If `file` mode is used, make sure the marker file does not already exist before +starting the benchmark. + +## Common Examples + +### 1. Functional verification (`1+0`) + +```bash +python3 mooncake-store/benchmarks/store_kv_bench.py \ + --scenario verify_write \ + --io-api plain \ + --local-hostname 127.0.0.1:50071 \ + --metadata-server http://127.0.0.1:8080/metadata \ + --master-server 127.0.0.1:50051 \ + --protocol tcp \ + --global-segment-size $((64*1024*1024)) \ + --local-buffer-size $((32*1024*1024)) \ + --nr-objects 16 \ + --batch-size 4 \ + --key-prefix verify \ + --key-size 20 \ + --value-size 4096 \ + --memory-replica-num 1 \ + --nof-replica-num 0 \ + --verify \ + --pattern 0xab +``` + +### 2. NoF-only functional verification (`0+1`) + +```bash +python3 mooncake-store/benchmarks/store_kv_bench.py \ + --scenario verify_write \ + --io-api plain \ + --local-hostname 127.0.0.1:50071 \ + --metadata-server http://127.0.0.1:8080/metadata \ + --master-server 127.0.0.1:50051 \ + --protocol tcp \ + --global-segment-size 0 \ + --local-buffer-size $((8*1024*1024)) \ + --nr-objects 8 \ + --batch-size 2 \ + --key-prefix nofonly \ + --key-size 20 \ + --value-size 4096 \ + --memory-replica-num 0 \ + --nof-replica-num 1 \ + --verify \ + --pattern 0xcd +``` + +### 3. Read performance with automatic prepare phase + +```bash +python3 mooncake-store/benchmarks/store_kv_bench.py \ + --scenario read_perf \ + --prepare-mode auto \ + --phase-gap-mode sleep \ + --phase-gap-sec 1 \ + --io-api plain \ + --local-hostname 127.0.0.1:50071 \ + --metadata-server http://127.0.0.1:8080/metadata \ + --master-server 127.0.0.1:50051 \ + --protocol tcp \ + --nr-objects 32 \ + --batch-size 4 \ + --runtime 5 \ + --key-prefix readperf \ + --key-size 20 \ + --value-size 4096 \ + --memory-replica-num 1 \ + --nof-replica-num 0 \ + --verify \ + --pattern 0xee +``` + +### 4. Mixed read/write with initial dataset + +```bash +python3 mooncake-store/benchmarks/store_kv_bench.py \ + --scenario mixed_rw \ + --prepare-mode auto \ + --io-api zcopy \ + --local-hostname 127.0.0.1:50071 \ + --metadata-server http://127.0.0.1:8080/metadata \ + --master-server 127.0.0.1:50051 \ + --protocol tcp \ + --nr-objects 64 \ + --write-objects 4096 \ + --batch-size 4 \ + --runtime 10 \ + --rwmixread 70 \ + --key-prefix mixed \ + --key-size 20 \ + --value-size 4096 \ + --memory-replica-num 1 \ + --nof-replica-num 1 \ + --verify \ + --pattern 0x5a +``` + +In `mixed_rw`, reads are served from the prepared object set, while writes +always use fresh object ids. This keeps the workload as "existing-object read + +new-object write" and avoids key overlap between the read and write streams. + +## Output + +Each phase prints: + +- request counts +- KV counts +- miss / verify-failure counts +- bytes processed +- duration +- `req/s` +- `kv/s` +- `MiB/s` +- `lat_mean` +- `lat_p50` +- `lat_p95` +- `lat_p99` +- aggregated error counts + +An overall summary is printed after all phases complete. diff --git a/mooncake-store/benchmarks/store_kv_bench.py b/mooncake-store/benchmarks/store_kv_bench.py new file mode 100644 index 00000000..68fa639b --- /dev/null +++ b/mooncake-store/benchmarks/store_kv_bench.py @@ -0,0 +1,1028 @@ +#!/usr/bin/env python3 +"""Mooncake Store end-to-end KV benchmark.""" + +from __future__ import annotations + +import argparse +import ctypes +import logging +import math +import os +import random +import statistics +import threading +import time +from collections import Counter +from dataclasses import dataclass, field +from typing import Callable, Iterable, List, Optional +from mooncake.store import MooncakeDistributedStore, ReplicateConfig, get_alloc_func_addr, get_free_func_addr + + +LOG = logging.getLogger("store_kv_bench") + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + description="Mooncake Store end-to-end KV benchmark", + formatter_class=argparse.ArgumentDefaultsHelpFormatter, + ) + parser.add_argument( + "--scenario", + required=True, + choices=["verify_write", "fill", "write_perf", "read_perf", "mixed_rw"], + help="Benchmark scenario to execute.", + ) + + parser.add_argument("--local-hostname", default="127.0.0.1:50071") + parser.add_argument("--metadata-server", default="http://127.0.0.1:8080/metadata") + parser.add_argument("--master-server", default="127.0.0.1:50051") + parser.add_argument("--protocol", default="tcp") + parser.add_argument("--device-name", default="") + parser.add_argument("--global-segment-size", type=int, default=64 * 1024 * 1024) + parser.add_argument("--local-buffer-size", type=int, default=32 * 1024 * 1024) + parser.add_argument( + "--io-api", + choices=["plain", "zcopy"], + default="plain", + help="plain uses put/get/put_batch/get_batch, zcopy uses put_from/get_into/batch_put_from/batch_get_into", + ) + + parser.add_argument("--numjobs", type=int, default=1) + parser.add_argument("--iodepth", type=int, default=1) + parser.add_argument("--batch-size", type=int, default=1) + parser.add_argument("--runtime", type=int, default=0, help="Seconds. 0 means object-count based.") + parser.add_argument("--nr-objects", type=int, default=128) + parser.add_argument("--write-objects", type=int, default=0) + parser.add_argument( + "--prepare-objects", + type=int, + default=0, + help="Object count used by the prepare phase. 0 means reuse nr-objects.", + ) + parser.add_argument("--object-id-start", type=int, default=0) + parser.add_argument("--key-prefix", default="kvbench") + parser.add_argument("--key-size", type=int, default=20) + parser.add_argument("--value-size", type=int, default=4096) + parser.add_argument("--rand-seed", type=int, default=1) + + parser.add_argument("--memory-replica-num", type=int, default=1) + parser.add_argument("--nof-replica-num", type=int, default=0) + + parser.add_argument("--verify", action="store_true") + parser.add_argument("--pattern", default="") + parser.add_argument("--prepare-mode", choices=["auto", "none", "write"], default="auto") + parser.add_argument("--rwmixread", type=int, default=70) + + parser.add_argument( + "--phase-gap-mode", + choices=["none", "sleep", "manual", "file"], + default="none", + ) + parser.add_argument("--phase-gap-sec", type=int, default=0) + parser.add_argument("--phase-gap-file", default="") + parser.add_argument("--phase-gap-timeout-sec", type=int, default=600) + parser.add_argument("--log-level", default="INFO") + return parser + + +def setup_logging(level: str) -> None: + logging.basicConfig( + level=getattr(logging, level.upper(), logging.INFO), + format="%(asctime)s %(levelname)s %(name)s %(message)s", + datefmt="%Y-%m-%d %H:%M:%S", + ) + + +@dataclass +class PhaseStats: + name: str + request_latencies: List[float] = field(default_factory=list) + requests: int = 0 + successful_requests: int = 0 + failed_requests: int = 0 + kvs: int = 0 + successful_kvs: int = 0 + failed_kvs: int = 0 + misses: int = 0 + verify_failures: int = 0 + bytes_processed: int = 0 + error_counts: Counter = field(default_factory=Counter) + start_time: float = 0.0 + end_time: float = 0.0 + dataset_exhausted: bool = False + + +@dataclass +class RequestResult: + request_ok: bool + kv_successes: int + kv_failures: int + bytes_processed: int + successful_object_ids: List[int] = field(default_factory=list) + misses: int = 0 + verify_failures: int = 0 + error_counts: Counter = field(default_factory=Counter) + + +class PayloadFactory: + def __init__(self, value_size: int, pattern: bytes): + self.value_size = value_size + self.pattern = pattern + self._default_cache: dict[int, bytes] = {} + self._pattern_payload = self._repeat(pattern) if pattern else b"" + + def _repeat(self, token: bytes) -> bytes: + repeat = (self.value_size + len(token) - 1) // len(token) + return (token * repeat)[: self.value_size] + + def build(self, object_id: int) -> bytes: + if self.pattern: + return self._pattern_payload + fill_byte = object_id & 0xFF + payload = self._default_cache.get(fill_byte) + if payload is None: + payload = bytes([fill_byte]) * self.value_size + self._default_cache[fill_byte] = payload + return payload + + def verify_payload(self, object_id: int, payload: bytes) -> bool: + return payload == self.build(object_id) + + +class DatasetState: + def __init__(self, object_id_start: int): + self.write_lock = threading.Lock() + self.ids_lock = threading.Lock() + self.cursor_lock = threading.Lock() + self.next_write_id = object_id_start + self.prepared_ids: tuple[int, ...] = () + self.written_ids: tuple[int, ...] = () + self.read_cursor = 0 + + def reserve_write_ids(self, count: int, upper_bound: int) -> List[int]: + with self.write_lock: + if self.next_write_id >= upper_bound: + return [] + end = min(self.next_write_id + count, upper_bound) + ids = list(range(self.next_write_id, end)) + self.next_write_id = end + return ids + + def mark_prepared(self, ids: Iterable[int]) -> None: + ids = tuple(ids) + if not ids: + return + with self.ids_lock: + self.prepared_ids = self.prepared_ids + ids + self.written_ids = self.written_ids + ids + + def mark_runtime_written(self, ids: Iterable[int]) -> None: + ids = tuple(ids) + if not ids: + return + with self.ids_lock: + self.written_ids = self.written_ids + ids + + def written_count(self) -> int: + with self.ids_lock: + return len(self.written_ids) + + def snapshot_written_ids(self) -> List[int]: + with self.ids_lock: + return list(self.written_ids) + + def next_read_ids( + self, + count: int, + *, + loop: bool, + sequential: bool, + rng, + source: str = "written", + ) -> List[int]: + with self.ids_lock: + readable_ids = self.prepared_ids if source == "prepared" else self.written_ids + if not readable_ids: + return [] + if sequential: + with self.cursor_lock: + result = [] + for _ in range(count): + if self.read_cursor >= len(readable_ids): + if not loop: + break + self.read_cursor = 0 + result.append(readable_ids[self.read_cursor]) + self.read_cursor += 1 + return result + return [readable_ids[rng.randrange(len(readable_ids))] for _ in range(count)] + + +def parse_pattern(pattern_text: str) -> bytes: + if not pattern_text: + return b"" + if pattern_text.startswith("0x"): + hex_text = pattern_text[2:] + if len(hex_text) % 2 != 0: + raise ValueError("hex pattern length must be even") + return bytes.fromhex(hex_text) + return pattern_text.encode("utf-8") + + +def make_key(prefix: str, key_size: int, object_id: int) -> str: + suffix = f"{object_id:016d}" + if key_size < len(suffix): + raise ValueError(f"key_size={key_size} is smaller than suffix length {len(suffix)}") + prefix_space = key_size - len(suffix) + prefix_part = prefix[:prefix_space].ljust(prefix_space, "_") + return f"{prefix_part}{suffix}" + + +class StoreSession: + def __init__( + self, + args: argparse.Namespace, + lane_id: int, + payload_factory: PayloadFactory, + store_obj, + zcopy: Optional["ZcopyBufferView"] = None, + ): + self.args = args + self.lane_id = lane_id + self.payload_factory = payload_factory + self.store = store_obj + self.config = ReplicateConfig() + self.config.replica_num = args.memory_replica_num + self.config.nof_replica_num = args.nof_replica_num + self._zcopy = zcopy + + def close(self) -> None: + self._zcopy = None + + def put_ids(self, object_ids: List[int]) -> RequestResult: + keys = [make_key(self.args.key_prefix, self.args.key_size, object_id) for object_id in object_ids] + ret_codes = self._put_keys(keys, object_ids) + + errors: Counter = Counter() + success_ids: List[int] = [] + if self.args.io_api == "plain" and not (len(object_ids) == 1 and self.args.batch_size == 1): + ret = ret_codes[0] + if ret == 0: + success_ids.extend(object_ids) + else: + errors[ret] += 1 + else: + for object_id, ret in zip(object_ids, ret_codes): + if ret == 0: + success_ids.append(object_id) + else: + errors[ret] += 1 + request_ok = len(success_ids) == len(object_ids) + return RequestResult( + request_ok=request_ok, + kv_successes=len(success_ids), + kv_failures=len(object_ids) - len(success_ids), + bytes_processed=len(success_ids) * self.args.value_size, + successful_object_ids=success_ids, + error_counts=errors, + ) + + def get_ids(self, object_ids: List[int], verify: bool) -> RequestResult: + keys = [make_key(self.args.key_prefix, self.args.key_size, object_id) for object_id in object_ids] + errors: Counter = Counter() + kv_successes = 0 + misses = 0 + verify_failures = 0 + if self.args.io_api == "plain": + payloads = self._get_payloads_plain(keys) + for object_id, payload in zip(object_ids, payloads): + if payload in (None, b""): + misses += 1 + errors["MISS"] += 1 + continue + if verify and not self.payload_factory.verify_payload(object_id, payload): + verify_failures += 1 + errors["VERIFY_FAIL"] += 1 + continue + kv_successes += 1 + else: + assert self._zcopy is not None + lengths = self._get_lengths_zcopy(keys, len(object_ids)) + for slot, (object_id, length) in enumerate(zip(object_ids, lengths)): + if length < 0: + if self.store.isExist(keys[slot]) == 0: + misses += 1 + errors["MISS"] += 1 + else: + errors[length] += 1 + continue + payload = self._zcopy.read_bytes(slot, length) + if verify: + if length != self.args.value_size: + verify_failures += 1 + errors["VERIFY_SIZE_MISMATCH"] += 1 + continue + if not self.payload_factory.verify_payload(object_id, payload): + verify_failures += 1 + errors["VERIFY_FAIL"] += 1 + continue + kv_successes += 1 + + kv_failures = len(object_ids) - kv_successes + return RequestResult( + request_ok=(kv_failures == 0), + kv_successes=kv_successes, + kv_failures=kv_failures, + bytes_processed=kv_successes * self.args.value_size, + misses=misses, + verify_failures=verify_failures, + error_counts=errors, + ) + + def _put_keys(self, keys: List[str], object_ids: List[int]) -> List[int]: + values = [self.payload_factory.build(object_id) for object_id in object_ids] + if self.args.io_api == "plain": + if len(object_ids) == 1 and self.args.batch_size == 1: + return [self.store.put(keys[0], values[0], self.config)] + return [self.store.put_batch(keys, values, self.config)] + + assert self._zcopy is not None + ptrs = self._zcopy.fill_write_buffers(values) + sizes = [len(value) for value in values] + if len(object_ids) == 1 and self.args.batch_size == 1: + return [self.store.put_from(keys[0], ptrs[0], sizes[0], self.config)] + return list(self.store.batch_put_from(keys, ptrs, sizes, self.config)) + + def _get_payloads_plain(self, keys: List[str]) -> List[bytes]: + if len(keys) == 1 and self.args.batch_size == 1: + return [self.store.get(keys[0])] + return list(self.store.get_batch(keys)) + + def _get_lengths_zcopy(self, keys: List[str], slot_count: int) -> List[int]: + assert self._zcopy is not None + ptrs = self._zcopy.prepare_read_buffers(slot_count) + sizes = [self.args.value_size] * slot_count + if slot_count == 1 and self.args.batch_size == 1: + return [self.store.get_into(keys[0], ptrs[0], sizes[0])] + return list(self.store.batch_get_into(keys, ptrs, sizes)) + + +class ZcopyBufferPool: + def __init__(self, store_obj, value_size: int, slots: int): + self.store = store_obj + self.value_size = value_size + self.slots = slots + self.total_size = self.value_size * self.slots + self._alloc_fn = None + self._free_fn = None + self._registered = False + self.base_ptr = 0 + + alloc_addr = get_alloc_func_addr() + free_addr = get_free_func_addr() + if alloc_addr is None or free_addr is None: + raise RuntimeError("store module does not expose hugepage alloc/free helpers") + + self._alloc_fn = ctypes.CFUNCTYPE(ctypes.c_void_p, ctypes.c_size_t)( + get_alloc_func_addr() + ) + self._free_fn = ctypes.CFUNCTYPE(None, ctypes.c_void_p)( + get_free_func_addr() + ) + + raw_ptr = self._alloc_fn(self.total_size) + self.base_ptr = ctypes.cast(raw_ptr, ctypes.c_void_p).value or 0 + if self.base_ptr == 0: + raise RuntimeError( + f"direct hugepage alloc failed for zcopy pool: size={self.total_size}" + ) + ret = self.store.register_buffer(self.base_ptr, self.total_size) + if ret != 0: + failed_ptr = self.base_ptr + self._free_fn(ctypes.c_void_p(self.base_ptr)) + self.base_ptr = 0 + raise RuntimeError( + f"register_buffer failed for direct zcopy pool ptr={failed_ptr}: {ret}" + ) + self._registered = True + self._buffer = (ctypes.c_ubyte * self.total_size).from_address(self.base_ptr) + + def close(self) -> None: + self._buffer = None + if self.base_ptr: + if self._registered: + try: + self.store.unregister_buffer(self.base_ptr) + except Exception: + LOG.debug("unregister_buffer failed for direct zcopy pool ptr=%s", self.base_ptr, exc_info=True) + self._registered = False + if self._free_fn is not None: + self._free_fn(ctypes.c_void_p(self.base_ptr)) + self.base_ptr = 0 + + def slot_ptr(self, slot: int) -> int: + if slot < 0 or slot >= self.slots: + raise IndexError(f"zcopy slot {slot} is out of range [0, {self.slots})") + return self.base_ptr + slot * self.value_size + + +class ZcopyBufferView: + def __init__(self, pool: ZcopyBufferPool, slot_offset: int, slots: int): + self.pool = pool + self.slot_offset = slot_offset + self.slots = slots + + def _slot_ptr(self, slot: int) -> int: + if slot < 0 or slot >= self.slots: + raise IndexError(f"zcopy view slot {slot} is out of range [0, {self.slots})") + return self.pool.slot_ptr(self.slot_offset + slot) + + def fill_write_buffers(self, payloads: List[bytes]) -> List[int]: + ptrs: List[int] = [] + for slot, payload in enumerate(payloads): + if len(payload) > self.pool.value_size: + raise ValueError( + f"Payload size {len(payload)} exceeds slot size {self.pool.value_size}" + ) + ptr = self._slot_ptr(slot) + ctypes.memmove(ptr, payload, len(payload)) + ptrs.append(ptr) + return ptrs + + def prepare_read_buffers(self, slot_count: int) -> List[int]: + ptrs: List[int] = [] + for slot in range(slot_count): + ptr = self._slot_ptr(slot) + ctypes.memset(ptr, 0, self.pool.value_size) + ptrs.append(ptr) + return ptrs + + def read_bytes(self, slot: int, size: int) -> bytes: + if size > self.pool.value_size: + raise ValueError( + f"Read size {size} exceeds slot size {self.pool.value_size}" + ) + return ctypes.string_at(self._slot_ptr(slot), size) + + +class StoreRuntime: + def __init__(self, args: argparse.Namespace, lane_count: int): + + self.lane_count = lane_count + self.store = MooncakeDistributedStore() + setup_ret = self.store.setup( + args.local_hostname, + args.metadata_server, + args.global_segment_size, + args.local_buffer_size, + args.protocol, + args.device_name, + args.master_server, + ) + if setup_ret != 0: + raise RuntimeError(f"setup failed: {setup_ret}") + + self.zcopy_pool: Optional[ZcopyBufferPool] = None + if args.io_api == "zcopy": + slots = max(1, args.batch_size) * lane_count + self.zcopy_pool = ZcopyBufferPool( + self.store, args.value_size, slots + ) + + def make_session( + self, + args: argparse.Namespace, + lane_id: int, + payload_factory: PayloadFactory, + ) -> StoreSession: + zcopy_view: Optional[ZcopyBufferView] = None + if self.zcopy_pool is not None: + slots_per_lane = max(1, args.batch_size) + zcopy_view = ZcopyBufferView( + self.zcopy_pool, lane_id * slots_per_lane, slots_per_lane + ) + return StoreSession( + args, + lane_id, + payload_factory, + self.store, + zcopy_view, + ) + + def close(self) -> None: + if self.zcopy_pool is not None: + self.zcopy_pool.close() + self.zcopy_pool = None + if hasattr(self.store, "close"): + try: + self.store.close() + except Exception: + LOG.debug("shared store close failed", exc_info=True) + elif hasattr(self.store, "tearDownAll"): + try: + self.store.tearDownAll() + except Exception: + LOG.debug("shared tearDownAll failed", exc_info=True) + + +def merge_stats(name: str, stats_list: List[PhaseStats]) -> PhaseStats: + merged = PhaseStats(name=name) + if not stats_list: + return merged + merged.start_time = min((s.start_time for s in stats_list if s.start_time), default=0.0) + merged.end_time = max((s.end_time for s in stats_list if s.end_time), default=0.0) + for stats in stats_list: + merged.request_latencies.extend(stats.request_latencies) + merged.requests += stats.requests + merged.successful_requests += stats.successful_requests + merged.failed_requests += stats.failed_requests + merged.kvs += stats.kvs + merged.successful_kvs += stats.successful_kvs + merged.failed_kvs += stats.failed_kvs + merged.misses += stats.misses + merged.verify_failures += stats.verify_failures + merged.bytes_processed += stats.bytes_processed + merged.error_counts.update(stats.error_counts) + merged.dataset_exhausted = merged.dataset_exhausted or stats.dataset_exhausted + return merged + + +def percentile(values: List[float], p: float) -> float: + if not values: + return 0.0 + ordered = sorted(values) + if len(ordered) == 1: + return ordered[0] + rank = (len(ordered) - 1) * p + low = math.floor(rank) + high = math.ceil(rank) + if low == high: + return ordered[low] + return ordered[low] + (ordered[high] - ordered[low]) * (rank - low) + + +def summarize_stats(stats: PhaseStats) -> dict: + duration = max(stats.end_time - stats.start_time, 0.0) + return { + "requests": stats.requests, + "successful_requests": stats.successful_requests, + "failed_requests": stats.failed_requests, + "kvs": stats.kvs, + "successful_kvs": stats.successful_kvs, + "failed_kvs": stats.failed_kvs, + "misses": stats.misses, + "verify_failures": stats.verify_failures, + "bytes": stats.bytes_processed, + "duration_sec": duration, + "req_per_sec": (stats.requests / duration) if duration > 0 else 0.0, + "kv_per_sec": (stats.kvs / duration) if duration > 0 else 0.0, + "MiB_per_sec": (stats.bytes_processed / duration / (1024 * 1024)) if duration > 0 else 0.0, + "lat_mean_ms": statistics.mean(stats.request_latencies) * 1000 if stats.request_latencies else 0.0, + "lat_p50_ms": percentile(stats.request_latencies, 0.50) * 1000, + "lat_p95_ms": percentile(stats.request_latencies, 0.95) * 1000, + "lat_p99_ms": percentile(stats.request_latencies, 0.99) * 1000, + "dataset_exhausted": stats.dataset_exhausted, + "error_counts": dict(stats.error_counts), + } + + +def log_phase_stats(stats: PhaseStats) -> None: + summary = summarize_stats(stats) + LOG.info("=== phase %s ===", stats.name) + LOG.info( + "requests=%d successful_requests=%d failed_requests=%d kvs=%d successful_kvs=%d failed_kvs=%d", + summary["requests"], + summary["successful_requests"], + summary["failed_requests"], + summary["kvs"], + summary["successful_kvs"], + summary["failed_kvs"], + ) + LOG.info( + "misses=%d verify_failures=%d bytes=%d duration=%.3fs req/s=%.2f kv/s=%.2f MiB/s=%.2f", + summary["misses"], + summary["verify_failures"], + summary["bytes"], + summary["duration_sec"], + summary["req_per_sec"], + summary["kv_per_sec"], + summary["MiB_per_sec"], + ) + LOG.info( + "lat_mean=%.3fms lat_p50=%.3fms lat_p95=%.3fms lat_p99=%.3fms dataset_exhausted=%s", + summary["lat_mean_ms"], + summary["lat_p50_ms"], + summary["lat_p95_ms"], + summary["lat_p99_ms"], + summary["dataset_exhausted"], + ) + if summary["error_counts"]: + LOG.info("errors=%s", summary["error_counts"]) + + +class BenchmarkRunner: + def __init__(self, args: argparse.Namespace): + self.args = args + self.pattern = parse_pattern(args.pattern) + self.payload_factory = PayloadFactory(args.value_size, self.pattern) + self.dataset = DatasetState(args.object_id_start) + self.lane_count = args.numjobs * args.iodepth + self._sessions: Optional[List[StoreSession]] = None + self._runtime: Optional[StoreRuntime] = None + self._validate_args() + + def _validate_args(self) -> None: + if self.args.numjobs <= 0 or self.args.iodepth <= 0: + raise ValueError("numjobs and iodepth must be > 0") + if self.args.batch_size <= 0: + raise ValueError("batch-size must be > 0") + if self.args.value_size <= 0: + raise ValueError("value-size must be > 0") + if self.args.key_size <= 0: + raise ValueError("key-size must be > 0") + if self.args.nr_objects <= 0: + raise ValueError("nr-objects must be > 0") + if self.args.write_objects < 0: + raise ValueError("write-objects must be >= 0") + if self.args.prepare_objects < 0: + raise ValueError("prepare-objects must be >= 0") + if self.args.rwmixread < 0 or self.args.rwmixread > 100: + raise ValueError("rwmixread must be within [0, 100]") + if self.args.verify and not self.pattern: + raise ValueError("verify mode currently requires --pattern") + if self.args.memory_replica_num == 0 and self.args.nof_replica_num == 0: + raise ValueError("memory_replica_num and nof_replica_num cannot both be 0") + if self.args.phase_gap_mode == "sleep" and self.args.phase_gap_sec <= 0: + raise ValueError("phase-gap-sec must be > 0 when phase-gap-mode=sleep") + if self.args.phase_gap_mode == "file" and not self.args.phase_gap_file: + raise ValueError("phase-gap-file must be set when phase-gap-mode=file") + if self.args.scenario == "mixed_rw" and self.args.runtime <= 0: + raise ValueError("mixed_rw requires --runtime > 0") + if self._scenario_has_write() and self.args.value_size % 512 != 0: + raise ValueError("write-involved scenarios require value-size to be 512B aligned") + make_key(self.args.key_prefix, self.args.key_size, self.args.object_id_start) + + def _scenario_has_write(self) -> bool: + return self.args.scenario in {"verify_write", "fill", "write_perf", "mixed_rw"} + + def _write_budget(self) -> int: + return self.args.write_objects if self.args.write_objects > 0 else self.args.nr_objects + + def _prepare_budget(self) -> int: + return self.args.prepare_objects if self.args.prepare_objects > 0 else self.args.nr_objects + + def _make_sessions(self) -> List[StoreSession]: + if self._sessions is None: + self._runtime = StoreRuntime(self.args, self.lane_count) + self._sessions = [ + self._runtime.make_session(self.args, lane_id, self.payload_factory) + for lane_id in range(self.lane_count) + ] + return self._sessions + + def close(self) -> None: + if self._sessions is not None: + for session in self._sessions: + session.close() + self._sessions = None + if self._runtime is not None: + self._runtime.close() + self._runtime = None + + def _phase_gap(self, label: str) -> None: + mode = self.args.phase_gap_mode + if mode == "none": + return + LOG.info("phase gap before %s, mode=%s", label, mode) + if mode == "sleep": + LOG.info("sleeping %d seconds before %s", self.args.phase_gap_sec, label) + time.sleep(self.args.phase_gap_sec) + return + if mode == "manual": + input(f"phase '{label}' is waiting, finish external operations then press Enter to continue...") + return + deadline = time.time() + self.args.phase_gap_timeout_sec + while time.time() < deadline: + if os.path.exists(self.args.phase_gap_file): + LOG.info("detected phase gap file %s, continuing to %s", self.args.phase_gap_file, label) + return + time.sleep(1.0) + raise TimeoutError(f"timed out waiting for phase gap file {self.args.phase_gap_file}") + + def _run_threads(self, phase_name: str, worker_builder: Callable[[StoreSession, int], Callable[[PhaseStats], None]]) -> PhaseStats: + sessions = self._make_sessions() + per_lane_stats: List[Optional[PhaseStats]] = [None] * self.lane_count + threads: List[threading.Thread] = [] + + def runner(index: int, session: StoreSession) -> None: + stats = PhaseStats(name=f"{phase_name}/lane{index}") + stats.start_time = time.perf_counter() + worker_builder(session, index)(stats) + stats.end_time = time.perf_counter() + per_lane_stats[index] = stats + + for lane_id, session in enumerate(sessions): + thread = threading.Thread(target=runner, args=(lane_id, session), name=f"{phase_name}-lane{lane_id}") + threads.append(thread) + thread.start() + + for thread in threads: + thread.join() + + merged = merge_stats(phase_name, [s for s in per_lane_stats if s is not None]) + log_phase_stats(merged) + return merged + + def _record(self, stats: PhaseStats, latency: float, request: RequestResult, kv_count: int) -> None: + stats.request_latencies.append(latency) + stats.requests += 1 + stats.kvs += kv_count + if request.request_ok: + stats.successful_requests += 1 + else: + stats.failed_requests += 1 + stats.successful_kvs += request.kv_successes + stats.failed_kvs += request.kv_failures + stats.misses += request.misses + stats.verify_failures += request.verify_failures + stats.bytes_processed += request.bytes_processed + stats.error_counts.update(request.error_counts) + + def _run_fixed_write( + self, + phase_name: str, + total_objects: int, + *, + strict: bool, + write_scope: str = "runtime", + ) -> PhaseStats: + write_upper = self.dataset.next_write_id + total_objects + + def worker(session: StoreSession, _lane_id: int) -> Callable[[PhaseStats], None]: + def run(stats: PhaseStats) -> None: + while True: + object_ids = self.dataset.reserve_write_ids(self.args.batch_size, write_upper) + if not object_ids: + break + start = time.perf_counter() + result = session.put_ids(object_ids) + latency = time.perf_counter() - start + self._record(stats, latency, result, len(object_ids)) + if result.successful_object_ids: + if write_scope == "prepared": + self.dataset.mark_prepared(result.successful_object_ids) + else: + self.dataset.mark_runtime_written(result.successful_object_ids) + return run + + stats = self._run_threads(phase_name, worker) + expected = total_objects + if stats.successful_kvs < expected: + stats.dataset_exhausted = True + if strict and (stats.failed_kvs > 0 or stats.successful_kvs != expected): + raise RuntimeError( + f"{phase_name} strict write failed: expected {expected} objects, " + f"got success={stats.successful_kvs}, failed={stats.failed_kvs}" + ) + return stats + + def _run_time_based_write(self, phase_name: str, total_objects: int) -> PhaseStats: + deadline = time.time() + self.args.runtime + write_upper = self.dataset.next_write_id + total_objects + stop_event = threading.Event() + + def worker(session: StoreSession, _lane_id: int) -> Callable[[PhaseStats], None]: + def run(stats: PhaseStats) -> None: + while time.time() < deadline and not stop_event.is_set(): + object_ids = self.dataset.reserve_write_ids(self.args.batch_size, write_upper) + if not object_ids: + stats.dataset_exhausted = True + stop_event.set() + break + start = time.perf_counter() + result = session.put_ids(object_ids) + latency = time.perf_counter() - start + self._record(stats, latency, result, len(object_ids)) + if result.successful_object_ids: + self.dataset.mark_runtime_written(result.successful_object_ids) + return run + + return self._run_threads(phase_name, worker) + + def _run_read_phase( + self, + phase_name: str, + *, + verify: bool, + sequential: bool, + loop: bool, + runtime_sec: int = 0, + ) -> PhaseStats: + seed_base = self.args.rand_seed + if runtime_sec > 0: + deadline = time.time() + runtime_sec + + def worker(session: StoreSession, lane_id: int) -> Callable[[PhaseStats], None]: + rng = random.Random(seed_base + lane_id) + + def run(stats: PhaseStats) -> None: + while time.time() < deadline: + object_ids = self.dataset.next_read_ids( + self.args.batch_size, + loop=True, + sequential=sequential, + rng=rng, + source="prepared", + ) + if not object_ids: + stats.dataset_exhausted = True + break + start = time.perf_counter() + result = session.get_ids(object_ids, verify) + latency = time.perf_counter() - start + self._record(stats, latency, result, len(object_ids)) + + return run + + return self._run_threads(phase_name, worker) + + def worker(session: StoreSession, lane_id: int) -> Callable[[PhaseStats], None]: + rng = random.Random(seed_base + lane_id) + + def run(stats: PhaseStats) -> None: + while True: + object_ids = self.dataset.next_read_ids( + self.args.batch_size, + loop=loop, + sequential=sequential, + rng=rng, + source="prepared", + ) + if not object_ids: + break + start = time.perf_counter() + result = session.get_ids(object_ids, verify) + latency = time.perf_counter() - start + self._record(stats, latency, result, len(object_ids)) + + return run + + return self._run_threads(phase_name, worker) + + def _run_mixed_phase(self, phase_name: str, extra_write_budget: int) -> PhaseStats: + deadline = time.time() + self.args.runtime + write_upper = self.dataset.next_write_id + extra_write_budget + stop_event = threading.Event() + seed_base = self.args.rand_seed + + def worker(session: StoreSession, lane_id: int) -> Callable[[PhaseStats], None]: + rng = random.Random(seed_base + lane_id) + + def run(stats: PhaseStats) -> None: + while time.time() < deadline and not stop_event.is_set(): + do_read = rng.randrange(100) < self.args.rwmixread + if do_read: + object_ids = self.dataset.next_read_ids( + self.args.batch_size, + loop=True, + sequential=False, + rng=rng, + source="prepared", + ) + if not object_ids: + continue + start = time.perf_counter() + result = session.get_ids(object_ids, verify=self.args.verify) + latency = time.perf_counter() - start + self._record(stats, latency, result, len(object_ids)) + continue + + object_ids = self.dataset.reserve_write_ids(self.args.batch_size, write_upper) + if not object_ids: + stats.dataset_exhausted = True + stop_event.set() + break + start = time.perf_counter() + result = session.put_ids(object_ids) + latency = time.perf_counter() - start + self._record(stats, latency, result, len(object_ids)) + if result.successful_object_ids: + self.dataset.mark_runtime_written(result.successful_object_ids) + + return run + + return self._run_threads(phase_name, worker) + + def _maybe_prepare_dataset(self) -> Optional[PhaseStats]: + if self.args.prepare_mode == "none": + return None + if self.args.prepare_mode == "write" or self.args.scenario in {"read_perf", "mixed_rw"}: + stats = self._run_fixed_write( + "prepare_write", + self._prepare_budget(), + strict=True, + write_scope="prepared", + ) + self._phase_gap("main_run") + return stats + return None + + def run(self) -> List[PhaseStats]: + LOG.info( + "scenario=%s io_api=%s numjobs=%d iodepth=%d lanes=%d batch_size=%d value_size=%d nr_objects=%d prepare_objects=%d write_objects=%d memory_replica_num=%d nof_replica_num=%d verify=%s", + self.args.scenario, + self.args.io_api, + self.args.numjobs, + self.args.iodepth, + self.lane_count, + self.args.batch_size, + self.args.value_size, + self.args.nr_objects, + self._prepare_budget(), + self.args.write_objects, + self.args.memory_replica_num, + self.args.nof_replica_num, + self.args.verify, + ) + + phases: List[PhaseStats] = [] + if self.args.scenario == "verify_write": + phases.append( + self._run_fixed_write( + "write_verify", + self._write_budget(), + strict=True, + write_scope="prepared", + ) + ) + self._phase_gap("verify_read") + phases.append(self._run_read_phase("verify_read", verify=True, sequential=True, loop=False)) + return phases + + if self.args.scenario == "fill": + phases.append(self._run_fixed_write("fill_write", self._write_budget(), strict=False)) + return phases + + if self.args.scenario == "write_perf": + total_objects = self._write_budget() + if self.args.runtime > 0: + phases.append(self._run_time_based_write("write_perf", total_objects)) + else: + phases.append(self._run_fixed_write("write_perf", total_objects, strict=False)) + return phases + + if self.args.scenario == "read_perf": + prepared = self._maybe_prepare_dataset() + if prepared is not None: + phases.append(prepared) + phases.append( + self._run_read_phase( + "read_perf", + verify=self.args.verify, + sequential=True, + loop=(self.args.runtime > 0), + runtime_sec=self.args.runtime, + ) + ) + return phases + + prepared = self._maybe_prepare_dataset() + if prepared is not None: + phases.append(prepared) + phases.append(self._run_mixed_phase("mixed_rw", self._write_budget())) + return phases + + +def log_overall_summary(phases: List[PhaseStats]) -> None: + overall = merge_stats("overall", phases) + LOG.info("=== overall summary ===") + log_phase_stats(overall) + + +def main() -> int: + parser = build_parser() + args = parser.parse_args() + setup_logging(args.log_level) + runner: Optional[BenchmarkRunner] = None + try: + runner = BenchmarkRunner(args) + phases = runner.run() + log_overall_summary(phases) + if any(phase.verify_failures > 0 for phase in phases): + return 20 + if args.verify and any(phase.misses > 0 for phase in phases if "read" in phase.name): + return 21 + return 0 + except KeyboardInterrupt: + LOG.warning("benchmark interrupted") + return 130 + except Exception as exc: # pragma: no cover - CLI entry path + LOG.exception("benchmark failed: %s", exc) + return 1 + finally: + if runner is not None: + runner.close() + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/mooncake-store/include/ssd_register_client.h b/mooncake-store/include/ssd_register_client.h new file mode 100644 index 00000000..e8ae8cd0 --- /dev/null +++ b/mooncake-store/include/ssd_register_client.h @@ -0,0 +1,39 @@ +#ifndef MOONCAKE_SSD_REGISTER_CLIENT_H +#define MOONCAKE_SSD_REGISTER_CLIENT_H + +#include "master_client.h" +#include "types.h" +#include + +namespace mooncake { +#define OPERATION_OK 0 +#define OPERATION_FAILED -1 + +class NoFRegisterClient { + public: + NoFRegisterClient(); + ~NoFRegisterClient(); + + int set_register(const std::string &nqn, size_t nsid, + const std::string &traddr, size_t trsvcid, uintptr_t base, + size_t size, const std::string &master_server_addr); + + /** + * @brief Unregister a NoF SSD segment by its te_endpoint + * @param nqn NQN of the SSD + * @param nsid Namespace ID + * @param traddr Transport address + * @param trsvcid Transport service ID + * @param master_server_addr Master server address + * @return int OPERATION_OK or OPERATION_FAILED + */ + int set_unregister_by_endpoint(const std::string &nqn, size_t nsid, + const std::string &traddr, size_t trsvcid, + const std::string &master_server_addr); + + private: + MasterClient master_client_; +}; +} // namespace mooncake + +#endif // MOONCAKE_SSD_REGISTER_CLIENT_H diff --git a/mooncake-store/src/CMakeLists.txt b/mooncake-store/src/CMakeLists.txt index b384bf0e..2d0d8628 100644 --- a/mooncake-store/src/CMakeLists.txt +++ b/mooncake-store/src/CMakeLists.txt @@ -58,6 +58,7 @@ set(MOONCAKE_STORE_SOURCES ha_metric_manager.cpp store_c.cpp memory_alloc.cpp + ssd_register_client.cpp engram/engram_store.cpp ) diff --git a/mooncake-store/src/ssd_register_client.cpp b/mooncake-store/src/ssd_register_client.cpp new file mode 100644 index 00000000..39cc9f39 --- /dev/null +++ b/mooncake-store/src/ssd_register_client.cpp @@ -0,0 +1,138 @@ +#include "ssd_register_client.h" +#include +#include +#include + +namespace mooncake { + +NoFRegisterClient::NoFRegisterClient() + : master_client_(generate_uuid(), nullptr) {} + +NoFRegisterClient::~NoFRegisterClient() = default; + +int NoFRegisterClient::set_register(const std::string &nqn, size_t nsid, + const std::string &traddr, size_t trsvcid, + uintptr_t base, size_t size, + const std::string &master_server_addr) { + LOG(INFO) << "Registering SSD: nqn=" << nqn << ",nsid=" << nsid + << ",traddr=" << traddr << ",trsvcid=" << trsvcid + << ",master=" << master_server_addr << ",base=" << base + << ",size=" << size; + + auto err = master_client_.Connect(master_server_addr); + if (err != ErrorCode::OK) { + LOG(ERROR) << "Failed to connect to master"; + return OPERATION_FAILED; + } + + const char *trtype_env = std::getenv("MC_NOF_TRTYPE"); + std::string trtype = trtype_env ? trtype_env : "RDMA"; + std::transform( + trtype.begin(), trtype.end(), trtype.begin(), + [](unsigned char c) { return static_cast(std::toupper(c)); }); + if (trtype != "RDMA" && trtype != "TCP") { + LOG(WARNING) << "Invalid MC_NOF_TRTYPE=" << trtype + << ", fallback to RDMA"; + trtype = "RDMA"; + } + + std::string te_endpoint = "traddr:" + traddr + + " trsvcid:" + std::to_string(trsvcid) + + " subnqn:" + nqn + " trtype:" + trtype + + " adrfam:IPv4 ns:" + std::to_string(nsid); + + NoFSegment segment; + segment.base = base; + segment.size = size; + segment.id = generate_uuid(); + segment.name = te_endpoint; + segment.te_endpoint = te_endpoint; + auto mount_result = master_client_.MountNoFSegment(segment); + if (!mount_result) { + LOG(ERROR) << "mount_segment_to_master_failed "; + return OPERATION_FAILED; + } + + return OPERATION_OK; +} + +int NoFRegisterClient::set_unregister_by_endpoint( + const std::string &nqn, size_t nsid, const std::string &traddr, + size_t trsvcid, const std::string &master_server_addr) { + LOG(INFO) << "Unregistering SSD by endpoint: nqn=" << nqn + << ",nsid=" << nsid << ",traddr=" << traddr + << ",trsvcid=" << trsvcid << ",master=" << master_server_addr; + + // Connect to master server + auto err = master_client_.Connect(master_server_addr); + if (err != ErrorCode::OK) { + LOG(ERROR) << "Failed to connect to master: " << static_cast(err); + return OPERATION_FAILED; + } + + const char *trtype_env = std::getenv("MC_NOF_TRTYPE"); + std::string trtype = trtype_env ? trtype_env : "RDMA"; + std::transform( + trtype.begin(), trtype.end(), trtype.begin(), + [](unsigned char c) { return static_cast(std::toupper(c)); }); + if (trtype != "RDMA" && trtype != "TCP") { + LOG(WARNING) << "Invalid MC_NOF_TRTYPE=" << trtype + << ", fallback to RDMA"; + trtype = "RDMA"; + } + + // Build the te_endpoint string to match registered segments + std::string te_endpoint = "traddr:" + traddr + + " trsvcid:" + std::to_string(trsvcid) + + " subnqn:" + nqn + " trtype:" + trtype + + " adrfam:IPv4 ns:" + std::to_string(nsid); + + LOG(INFO) << "Built te_endpoint: " << te_endpoint; + + auto matching_segments_result = + master_client_.GetNoFSegmentsByName(te_endpoint); + if (!matching_segments_result) { + LOG(ERROR) << "Failed to get NoF segments by name: " + << static_cast(matching_segments_result.error()); + return OPERATION_FAILED; + } + + std::vector matching_segments = + matching_segments_result.value(); + LOG(INFO) << "Retrieved " << matching_segments.size() + << " mounted NoF segments for te_endpoint"; + if (matching_segments.empty()) { + LOG(ERROR) << "No segment found for te_endpoint: " << te_endpoint; + return OPERATION_FAILED; + } + + // Unmount all matching segments + bool all_unmounted = true; + for (const auto &segment : matching_segments) { + LOG(INFO) << "Found matching segment: id=" << segment.segment_id + << ", owner_client_id=" << segment.client_id; + MasterClient owner_master_client(segment.client_id, nullptr); + err = owner_master_client.Connect(master_server_addr); + if (err != ErrorCode::OK) { + LOG(ERROR) << "Failed to connect owner master client for segment " + << segment.segment_id << ": " << static_cast(err); + all_unmounted = false; + continue; + } + + auto unmount_result = + owner_master_client.UnmountNoFSegment(segment.segment_id); + if (!unmount_result) { + LOG(ERROR) << "Failed to unmount segment " << segment.segment_id + << ": " << static_cast(unmount_result.error()); + all_unmounted = false; + } else { + LOG(INFO) << "Successfully unmounted segment " << segment.segment_id + << ", owner_client_id=" << segment.client_id; + } + } + + return all_unmounted ? OPERATION_OK : OPERATION_FAILED; +} + +} // namespace mooncake diff --git a/mooncake-store/tests/e2e/readme.md b/mooncake-store/tests/e2e/readme.md index 0f7a4b39..e8ad2aed 100644 --- a/mooncake-store/tests/e2e/readme.md +++ b/mooncake-store/tests/e2e/readme.md @@ -11,6 +11,8 @@ The E2E test suite includes several executable programs designed to test differe - **e2e_rand_test**: Long-term randomized end-to-end testing. - **chaos_test**: Short-term chaos testing with predefined scenarios. - **chaos_rand_test**: Long-term randomized chaos testing with configurable parameters. +- **store_client_e2e.py**: Python `MooncakeDistributedStore` client that continuously issues `put/get` operations. +- **run_nof_heartbeat_tcp_e2e.sh**: Scripted NoF heartbeat end-to-end test using a TCP SPDK target. ## Parameters @@ -72,4 +74,78 @@ Currently it only has few test cases. Will add more in the future. 2. Run the test. **[WIP]**: -Currently it only has few test cases. Will add more in the future. \ No newline at end of file +Currently it only has few test cases. Will add more in the future. + +### run_nof_heartbeat_tcp_e2e.sh + +**Brief**: Launches a real four-component path for NoF heartbeat validation: + +- `mooncake_master` +- standalone Python HTTP metadata server +- SPDK `nvmf_tgt` with TCP transport +- Python client built on `MooncakeDistributedStore` + +The script first verifies steady-state `put/get` success with `memory + nof` replicas. It then kills the SPDK target, waits for the master heartbeat thread to emit `action=unmount_nof_segment_by_heartbeat`, and finally verifies that the client still observes successful I/O after the NoF segment is removed. + +**Prerequisites**: + +- `BUILD_DIR` points to a build tree that already contains: + - `mooncake-store/src/mooncake_master` + - `mooncake-integration/store*.so` +- SPDK has already been built under `extern/spdk` +- Python environment contains `aiohttp` because the script launches a standalone metadata process with `mooncake-wheel/mooncake/http_metadata_server.py` +- The script uses `sudo -n` to set hugepages and mount `/dev/hugepages`, so the current user must have passwordless sudo + +**Usage**: + +```bash +cd mooncake-store/tests/e2e +BUILD_DIR=/path/to/build ./run_nof_heartbeat_tcp_e2e.sh +``` + +To run in **NoF-only** mode (do not mount a local memory segment), set: + +```bash +CLIENT_GLOBAL_SEGMENT_SIZE=0 BUILD_DIR=/path/to/build ./run_nof_heartbeat_tcp_e2e.sh +``` + +To increase the amount of steady-state traffic before killing the target, set: + +```bash +PRE_FAULT_SUCCESS_TARGET=10 BUILD_DIR=/path/to/build ./run_nof_heartbeat_tcp_e2e.sh +``` + +**Notes**: + +- The client payload size defaults to `4096` bytes because the current NoF path requires 4K-aligned I/O. +- The script uses a standalone metadata server process (`mooncake-wheel/mooncake/http_metadata_server.py`) instead of the embedded master metadata server so all four components remain explicit during the test. +- In default mode, the script verifies **service continuity** after NoF unmount by checking that post-fault I/O still succeeds. +- In `CLIENT_GLOBAL_SEGMENT_SIZE=0` mode, the script verifies **NoF-only failure behavior** by checking that post-fault I/O starts failing after the NoF segment is removed. +- Logs are written under `LOG_DIR` (default `/tmp/mooncake_nof_heartbeat_e2e`) and the final pass/fail summary is printed from `summary.log`. + +### store_client_e2e.py + +**Brief**: A standalone Python workload generator built on `MooncakeDistributedStore`. It continuously issues `put/get` against the configured master/metadata pair and prints `put_ok/get_ok/put_fail/get_fail` lines that can be consumed by shell scripts. + +**Standalone Usage**: + +```bash +PYTHONPATH=/path/to/build/mooncake-integration \ +python3 store_client_e2e.py \ + --local-hostname 127.0.0.1:50071 \ + --metadata-server http://127.0.0.1:8080/metadata \ + --master-server 127.0.0.1:50051 \ + --global-segment-size 67108864 \ + --local-buffer-size 33554432 \ + --payload-size 4096 \ + --duration-sec 20 \ + --sleep-ms 200 \ + --key-prefix demo +``` + +**Key Parameters**: + +- `--global-segment-size 0`: run in NoF-only mode +- `--payload-size 4096`: keep NoF writes 4K aligned +- `--duration-sec`: total workload duration +- `--sleep-ms`: interval between operations diff --git a/mooncake-store/tests/e2e/run_nof_heartbeat_tcp_e2e.sh b/mooncake-store/tests/e2e/run_nof_heartbeat_tcp_e2e.sh new file mode 100644 index 00000000..4e748824 --- /dev/null +++ b/mooncake-store/tests/e2e/run_nof_heartbeat_tcp_e2e.sh @@ -0,0 +1,245 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) +REPO_ROOT=$(cd -- "$SCRIPT_DIR/../../.." && pwd) +BUILD_DIR=${BUILD_DIR:-"$REPO_ROOT/build"} +LOG_DIR=${LOG_DIR:-/tmp/mooncake_nof_heartbeat_e2e} +MASTER_RPC=${MASTER_RPC:-127.0.0.1:50051} +MASTER_HOST=${MASTER_RPC%:*} +MASTER_PORT=${MASTER_RPC##*:} +METADATA_HOST=${METADATA_HOST:-127.0.0.1} +METADATA_PORT=${METADATA_PORT:-8080} +METADATA_SERVER="http://$METADATA_HOST:$METADATA_PORT/metadata" +TARGET_HOST=${TARGET_HOST:-127.0.0.1} +TARGET_PORT=${TARGET_PORT:-4420} +TARGET_NQN=${TARGET_NQN:-nqn.2016-06.io.spdk:cnode1} +NOF_SIZE=${NOF_SIZE:-67108864} +PAYLOAD_SIZE=${PAYLOAD_SIZE:-4096} +CLIENT_DURATION=${CLIENT_DURATION:-0} +CLIENT_SLEEP_MS=${CLIENT_SLEEP_MS:-200} +HEARTBEAT_INTERVAL=${HEARTBEAT_INTERVAL:-2} +HEARTBEAT_TIMEOUT_MS=${HEARTBEAT_TIMEOUT_MS:-500} +HEARTBEAT_FAILURES=${HEARTBEAT_FAILURES:-3} +CLIENT_GLOBAL_SEGMENT_SIZE=${CLIENT_GLOBAL_SEGMENT_SIZE:-67108864} +CLIENT_LOCAL_BUFFER_SIZE=${CLIENT_LOCAL_BUFFER_SIZE:-33554432} +CLIENT_MEMORY_REPLICA_NUM=${CLIENT_MEMORY_REPLICA_NUM:-1} +CLIENT_NOF_REPLICA_NUM=${CLIENT_NOF_REPLICA_NUM:-1} +PRE_FAULT_SUCCESS_TARGET=${PRE_FAULT_SUCCESS_TARGET:-3} + +TARGET_PID="" +MASTER_PID="" +META_PID="" +CLIENT_PID="" + +cleanup() { + [[ -n "$CLIENT_PID" ]] && kill "$CLIENT_PID" >/dev/null 2>&1 || true + [[ -n "$MASTER_PID" ]] && kill "$MASTER_PID" >/dev/null 2>&1 || true + [[ -n "$TARGET_PID" ]] && kill "$TARGET_PID" >/dev/null 2>&1 || true + [[ -n "$META_PID" ]] && kill "$META_PID" >/dev/null 2>&1 || true +} +trap cleanup EXIT + +wait_for_pattern() { + local file=$1 + local pattern=$2 + local timeout=$3 + local waited=0 + while (( waited < timeout )); do + if grep -q "$pattern" "$file" 2>/dev/null; then + return 0 + fi + sleep 1 + waited=$((waited + 1)) + done + return 1 +} + +count_pattern() { + local file=$1 + local pattern=$2 + grep -c "$pattern" "$file" 2>/dev/null || true +} + +rm -rf "$LOG_DIR" +mkdir -p "$LOG_DIR" + +sudo -n sh -c 'echo 512 > /proc/sys/vm/nr_hugepages' +sudo -n umount /dev/hugepages >/dev/null 2>&1 || true +sudo -n mkdir -p /dev/hugepages +sudo -n mount -t hugetlbfs -o pagesize=2M,mode=1777 none /dev/hugepages +grep -E 'HugePages_Total|HugePages_Free|Hugepagesize' /proc/meminfo >"$LOG_DIR/hugepages.log" + +pkill -f '/nvmf_tgt' >/dev/null 2>&1 || true +pkill -f 'mooncake_master' >/dev/null 2>&1 || true +pkill -f 'http_metadata_server.py' >/dev/null 2>&1 || true +sleep 1 + +"$REPO_ROOT/extern/spdk/build/bin/nvmf_tgt" -m 0x1 -u --iova-mode=va --wait-for-rpc >"$LOG_DIR/target.log" 2>&1 & +TARGET_PID=$! +sleep 3 + +python3 "$REPO_ROOT/extern/spdk/scripts/rpc.py" framework_start_init >/dev/null +python3 "$REPO_ROOT/extern/spdk/scripts/rpc.py" framework_wait_init >/dev/null +python3 "$REPO_ROOT/extern/spdk/scripts/rpc.py" bdev_malloc_create -b Malloc0 64 4096 >/dev/null +python3 "$REPO_ROOT/extern/spdk/scripts/rpc.py" nvmf_create_transport -t TCP >/dev/null || true +python3 "$REPO_ROOT/extern/spdk/scripts/rpc.py" nvmf_create_subsystem "$TARGET_NQN" -a -s SPDK00000000000001 >/dev/null || true +python3 "$REPO_ROOT/extern/spdk/scripts/rpc.py" nvmf_subsystem_add_ns "$TARGET_NQN" Malloc0 >/dev/null || true +python3 "$REPO_ROOT/extern/spdk/scripts/rpc.py" nvmf_subsystem_add_listener "$TARGET_NQN" -t tcp -a "$TARGET_HOST" -s "$TARGET_PORT" >/dev/null || true + +python3 "$REPO_ROOT/mooncake-wheel/mooncake/http_metadata_server.py" --host "$METADATA_HOST" --port "$METADATA_PORT" >"$LOG_DIR/metadata.log" 2>&1 & +META_PID=$! +sleep 2 + +"$BUILD_DIR/mooncake-store/src/mooncake_master" \ + --rpc_address="$MASTER_HOST" \ + --rpc_port="$MASTER_PORT" \ + --enable_http_metadata_server=false \ + --logtostderr=true \ + --nof_heartbeat_interval_sec="$HEARTBEAT_INTERVAL" \ + --nof_heartbeat_probe_timeout_ms="$HEARTBEAT_TIMEOUT_MS" \ + --nof_heartbeat_failures_threshold="$HEARTBEAT_FAILURES" >"$LOG_DIR/master.log" 2>&1 & +MASTER_PID=$! +sleep 3 + +PYTHONPATH="$BUILD_DIR/mooncake-integration" MC_NOF_TRTYPE=TCP python3 - <"$LOG_DIR/register.log" 2>&1 +import store +ret = store.MooncakeDistributedNoFRegister().real_register( + "$TARGET_NQN", + 1, + "$TARGET_HOST", + int("$TARGET_PORT"), + 0, + int("$NOF_SIZE"), + "$MASTER_RPC", +) +print(f"register_ret {ret}", flush=True) +raise SystemExit(0 if ret == 0 else 1) +PY + +PYTHONPATH="$BUILD_DIR/mooncake-integration" python3 "$SCRIPT_DIR/store_client_e2e.py" \ + --local-hostname "127.0.0.1:50071" \ + --metadata-server "$METADATA_SERVER" \ + --master-server "$MASTER_RPC" \ + --global-segment-size "$CLIENT_GLOBAL_SEGMENT_SIZE" \ + --local-buffer-size "$CLIENT_LOCAL_BUFFER_SIZE" \ + --memory-replica-num "$CLIENT_MEMORY_REPLICA_NUM" \ + --nof-replica-num "$CLIENT_NOF_REPLICA_NUM" \ + --payload-size "$PAYLOAD_SIZE" \ + --duration-sec "$CLIENT_DURATION" \ + --sleep-ms "$CLIENT_SLEEP_MS" \ + --key-prefix "nof-heartbeat" >"$LOG_DIR/client.log" 2>&1 & +CLIENT_PID=$! + +deadline=$((SECONDS + 30)) +while (( SECONDS < deadline )); do + put_ok_count=$(count_pattern "$LOG_DIR/client.log" 'put_ok') + get_ok_count=$(count_pattern "$LOG_DIR/client.log" 'get_ok') + if (( put_ok_count >= PRE_FAULT_SUCCESS_TARGET && get_ok_count >= PRE_FAULT_SUCCESS_TARGET )); then + break + fi + sleep 1 +done + +put_ok_count=$(count_pattern "$LOG_DIR/client.log" 'put_ok') +get_ok_count=$(count_pattern "$LOG_DIR/client.log" 'get_ok') +if (( put_ok_count < PRE_FAULT_SUCCESS_TARGET || get_ok_count < PRE_FAULT_SUCCESS_TARGET )); then + echo "client did not reach enough initial success: put_ok=$put_ok_count get_ok=$get_ok_count" + exit 1 +fi + +pre_fault_line_count=$(wc -l <"$LOG_DIR/client.log") + +kill "$TARGET_PID" >/dev/null 2>&1 || true +TARGET_PID="" + +if ! wait_for_pattern "$LOG_DIR/master.log" 'action=unmount_nof_segment_by_heartbeat' $((HEARTBEAT_INTERVAL * HEARTBEAT_FAILURES + 20)); then + echo "master did not unmount nof segment by heartbeat" + exit 1 +fi + +post_unmount_line_count=$(wc -l <"$LOG_DIR/client.log") + +observation_deadline=$((SECONDS + 10)) +while (( SECONDS < observation_deadline )); do + post_unmount_successes=$(awk -v start="$post_unmount_line_count" ' + NR > start && ($0 ~ /put_ok/ || $0 ~ /get_ok/) { count++ } + END { print count + 0 } + ' "$LOG_DIR/client.log") + post_unmount_failures=$(awk -v start="$post_unmount_line_count" ' + NR > start && ($0 ~ /put_fail/ || $0 ~ /get_fail/) { count++ } + END { print count + 0 } + ' "$LOG_DIR/client.log") + + if [[ "$CLIENT_GLOBAL_SEGMENT_SIZE" -eq 0 ]]; then + (( post_unmount_failures > 0 )) && break + else + (( post_unmount_successes > 0 )) && break + fi + sleep 1 +done + +kill "$CLIENT_PID" >/dev/null 2>&1 || true +wait "$CLIENT_PID" >/dev/null 2>&1 || true +CLIENT_PID="" + +post_fault_successes=$(awk -v start="$pre_fault_line_count" ' + NR > start && ($0 ~ /put_ok/ || $0 ~ /get_ok/) { count++ } + END { print count + 0 } +' "$LOG_DIR/client.log") + +post_fault_failures=$(awk -v start="$pre_fault_line_count" ' + NR > start && ($0 ~ /put_fail/ || $0 ~ /get_fail/) { count++ } + END { print count + 0 } +' "$LOG_DIR/client.log") + +post_unmount_successes=$(awk -v start="$post_unmount_line_count" ' + NR > start && ($0 ~ /put_ok/ || $0 ~ /get_ok/) { count++ } + END { print count + 0 } +' "$LOG_DIR/client.log") + +post_unmount_failures=$(awk -v start="$post_unmount_line_count" ' + NR > start && ($0 ~ /put_fail/ || $0 ~ /get_fail/) { count++ } + END { print count + 0 } +' "$LOG_DIR/client.log") + +if [[ "$CLIENT_GLOBAL_SEGMENT_SIZE" -eq 0 ]]; then + if [[ "$post_unmount_failures" -le 0 ]]; then + echo "expected IO failures after target down in nof-only mode" + exit 1 + fi +else + if [[ "$post_unmount_successes" -le 0 ]]; then + echo "no successful IO observed after target down" + exit 1 + fi +fi + +{ + echo "=== hugepages ===" + cat "$LOG_DIR/hugepages.log" + echo "=== register ===" + cat "$LOG_DIR/register.log" + echo "=== client ===" + cat "$LOG_DIR/client.log" + echo "=== master tail ===" + tail -n 200 "$LOG_DIR/master.log" + echo "=== metadata tail ===" + tail -n 80 "$LOG_DIR/metadata.log" + echo "=== target tail ===" + tail -n 120 "$LOG_DIR/target.log" + echo "=== verdict ===" + echo "post_fault_successes=$post_fault_successes" + echo "post_fault_failures=$post_fault_failures" + echo "post_unmount_successes=$post_unmount_successes" + echo "post_unmount_failures=$post_unmount_failures" + echo "pre_fault_put_ok=$put_ok_count" + echo "pre_fault_get_ok=$get_ok_count" + echo "pre_fault_line_count=$pre_fault_line_count" + echo "post_unmount_line_count=$post_unmount_line_count" + echo "client_global_segment_size=$CLIENT_GLOBAL_SEGMENT_SIZE" + echo "client_memory_replica_num=$CLIENT_MEMORY_REPLICA_NUM" + echo "client_nof_replica_num=$CLIENT_NOF_REPLICA_NUM" +} >"$LOG_DIR/summary.log" + +cat "$LOG_DIR/summary.log" diff --git a/mooncake-store/tests/e2e/store_client_e2e.py b/mooncake-store/tests/e2e/store_client_e2e.py new file mode 100644 index 00000000..9e061677 --- /dev/null +++ b/mooncake-store/tests/e2e/store_client_e2e.py @@ -0,0 +1,108 @@ +#!/usr/bin/env python3 +import argparse +import os +import sys +import time + + +def make_payload(seq: int, size: int) -> bytes: + seed = f"value{seq}".encode() + repeat = (size // len(seed)) + 1 + return (seed * repeat)[:size] + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Continuous MooncakeDistributedStore put/get workload" + ) + parser.add_argument("--local-hostname", default="127.0.0.1:50071") + parser.add_argument("--metadata-server", default="http://127.0.0.1:8080/metadata") + parser.add_argument("--master-server", default="127.0.0.1:50051") + parser.add_argument("--protocol", default="tcp") + parser.add_argument("--device-name", default="") + parser.add_argument("--global-segment-size", type=int, default=64 * 1024 * 1024) + parser.add_argument("--local-buffer-size", type=int, default=32 * 1024 * 1024) + parser.add_argument("--payload-size", type=int, default=4096) + parser.add_argument("--duration-sec", type=int, default=20) + parser.add_argument("--sleep-ms", type=int, default=200) + parser.add_argument("--key-prefix", default="nof-e2e") + parser.add_argument("--memory-replica-num", type=int, default=1) + parser.add_argument("--nof-replica-num", type=int, default=1) + args = parser.parse_args() + + try: + import store # type: ignore + except Exception as exc: + print(f"import_fail {exc}", flush=True) + return 10 + + mc = store.MooncakeDistributedStore() + setup_ret = mc.setup( + args.local_hostname, + args.metadata_server, + args.global_segment_size, + args.local_buffer_size, + args.protocol, + args.device_name, + args.master_server, + ) + print(f"setup_ret {setup_ret}", flush=True) + if setup_ret != 0: + return setup_ret + + replicate_config = store.ReplicateConfig() + replicate_config.replica_num = args.memory_replica_num + replicate_config.nof_replica_num = args.nof_replica_num + + deadline = time.time() + args.duration_sec if args.duration_sec > 0 else None + seq = 0 + put_ok = 0 + get_ok = 0 + put_fail = 0 + get_fail = 0 + mismatch = 0 + + while deadline is None or time.time() < deadline: + seq += 1 + key = f"{args.key_prefix}-{seq}" + expected = make_payload(seq, args.payload_size) + + put_ret = mc.put(key, expected, replicate_config) + if put_ret == 0: + put_ok += 1 + print(f"put_ok seq={seq} key={key} len={len(expected)}", flush=True) + else: + put_fail += 1 + print(f"put_fail seq={seq} key={key} ret={put_ret}", flush=True) + time.sleep(args.sleep_ms / 1000.0) + continue + + actual = mc.get(key) + if actual == expected: + get_ok += 1 + print(f"get_ok seq={seq} key={key} len={len(actual)}", flush=True) + elif actual in (b"", None): + get_fail += 1 + actual_len = 0 if actual in (b"", None) else len(actual) + print(f"get_fail seq={seq} key={key} len={actual_len}", flush=True) + else: + mismatch += 1 + print( + f"data_mismatch seq={seq} key={key} expected_len={len(expected)} actual_len={len(actual)}", + flush=True, + ) + return 20 + + time.sleep(args.sleep_ms / 1000.0) + + print( + "summary " + f"put_ok={put_ok} put_fail={put_fail} " + f"get_ok={get_ok} get_fail={get_fail} mismatch={mismatch}", + flush=True, + ) + return 0 if mismatch == 0 else 20 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/mooncake-wheel/mooncake/mooncake_ssd_register.py b/mooncake-wheel/mooncake/mooncake_ssd_register.py new file mode 100644 index 00000000..3a5e6e18 --- /dev/null +++ b/mooncake-wheel/mooncake/mooncake_ssd_register.py @@ -0,0 +1,328 @@ +#!/usr/bin/env python3 +# python3 -m mooncake.mooncake_ssd_register --master_server_address=192.168.65.81:50051 --spdk_target_info="ip:192.168.65.56 path:/home/spdk" --spdk_target_info="ip:192.168.65.57 path:/root/spdk" + +import argparse +import json +import logging +import time +import re +import shlex +from typing import List, Dict, Any + +import paramiko + +from mooncake.store import MooncakeDistributedNoFRegister +from mooncake.mooncake_config import MooncakeConfig + + +class MooncakeNoFRegister: + """ + Registers SSDs from remote SPDK targets to Mooncake master server. + """ + + def __init__(self, cli_config: dict = None, spdk_targets: List[str] = None): + self.register = None + self.config_list: List[Dict[str, Any]] = [] + self.cli_config = cli_config or {} + self.spdk_targets = spdk_targets or [] + self._setup_logging() + + try: + # Only support getting SSD info from remote SPDK targets + if self.spdk_targets: + # Get SSD info from remote SPDK targets + master_server_address = self.cli_config.get('master_server_address') + if not master_server_address: + raise ValueError("master_server_address is required when using spdk_target_info") + self.config_list = self._get_remote_ssd_info(master_server_address) + else: + raise ValueError("spdk_target_info is required") + + # Apply CLI overrides to every config (if key exists) + for config in self.config_list: + for key, value in self.cli_config.items(): + if key in config: + # Convert trsvcid/nsid to int if needed + if key in ("trsvcid", "nsid"): + try: + config[key] = int(value) + except ValueError: + logging.warning(f"Invalid integer for {key}: {value}") + else: + config[key] = value + + # Remove duplicate SSDs based on their unique identifiers + self._remove_duplicate_ssds() + + logging.info("Loaded %d SSD configuration(s)", len(self.config_list)) + except Exception as e: + logging.error("Configuration load failed: %s", e) + raise + + + + def _remove_duplicate_ssds(self): + """ + Remove duplicate SSD configurations from the config list + """ + unique_configs = [] + seen_keys = set() + + for config in self.config_list: + # Generate unique key directly without calling _get_ssd_unique_key + key = (config['nqn'], config['nsid'], config['traddr'], config['trsvcid']) + if key not in seen_keys: + seen_keys.add(key) + unique_configs.append(config) + + if len(unique_configs) < len(self.config_list): + logging.info(f"Removed {len(self.config_list) - len(unique_configs)} duplicate SSD configuration(s)") + + self.config_list = unique_configs + + + + def _parse_spdk_target_info(self, target_info: str) -> Dict[str, str]: + """ + Parse spdk target info string like "ip:192.168.65.56 path:/home" + """ + result = {} + parts = re.findall(r'(\w+):([^\s]+)', target_info) + for key, value in parts: + result[key] = value + return result + + def _execute_ssh_command(self, ip: str, command: str, path: str) -> str: + """ + Execute command on remote server via SSH + """ + ssh = paramiko.SSHClient() + ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) + try: + ssh.connect( + ip, + port=int(self.cli_config.get('port', 22)), + username=self.cli_config.get('username', 'root'), + password=self.cli_config.get('password'), + key_filename=self.cli_config.get('key_file'), + timeout=10 + ) + + # Try multiple possible paths to find the RPC script + possible_paths = [ + path, # Direct path provided by user + f"{path}/spdk" # Common case: spdk is a subdirectory + ] + + for test_path in possible_paths: + full_command = f"cd {shlex.quote(test_path)} && test -f scripts/rpc.py" + stdin, stdout, stderr = ssh.exec_command(full_command, timeout=5) + if stdout.channel.recv_exit_status() == 0: + # Found the script, execute the actual command + full_command = f"cd {shlex.quote(test_path)} && {command}" + stdin, stdout, stderr = ssh.exec_command(full_command, timeout=30) + exit_status = stdout.channel.recv_exit_status() + output = stdout.read().decode('utf-8') + error = stderr.read().decode('utf-8') + if exit_status != 0: + logging.error(f"SSH command error on {ip}: {error}") + raise RuntimeError(f"SSH command failed: {error or output}") + return output + + # If we get here, none of the paths worked + raise RuntimeError(f"Could not find scripts/rpc.py in any of the possible paths: {possible_paths}") + finally: + ssh.close() + + def _get_remote_ssd_info(self, master_server_address: str) -> List[Dict[str, Any]]: + """ + Get SSD info from remote SPDK targets + """ + ssd_configs = [] + + for target_info in self.spdk_targets: + target = self._parse_spdk_target_info(target_info) + ip = target.get('ip') + path = target.get('path') + + if not ip or not path: + logging.error(f"Invalid target info: {target_info}") + continue + + logging.info(f"Getting SSD info from target: {ip} (path: {path})") + + try: + # Get subsystems info + subsystems_cmd = "./scripts/rpc.py nvmf_get_subsystems" + subsystems_output = self._execute_ssh_command(ip, subsystems_cmd, path) + subsystems = json.loads(subsystems_output) + + # Process each subsystem + for subsystem in subsystems: + if subsystem.get('subtype') != 'NVMe': + continue + + nqn = subsystem.get('nqn') + listen_addresses = subsystem.get('listen_addresses', []) + + if not nqn or not listen_addresses: + continue + + # Get transport info from first listen address + traddr = listen_addresses[0].get('traddr') + trsvcid = listen_addresses[0].get('trsvcid') + + if not traddr or not trsvcid: + continue + + # Process each namespace + namespaces = subsystem.get('namespaces', []) + for namespace in namespaces: + nsid = namespace.get('nsid') + bdev_name = namespace.get('bdev_name') + + if not nsid or not bdev_name: + continue + + # Get bdev info to calculate size + bdev_cmd = f"./scripts/rpc.py bdev_get_bdevs -b {shlex.quote(bdev_name)}" + bdev_output = self._execute_ssh_command(ip, bdev_cmd, path) + bdevs = json.loads(bdev_output) + + if not bdevs: + continue + + bdev = bdevs[0] + block_size = bdev.get('block_size', 512) + num_blocks = bdev.get('num_blocks', 0) + size = block_size * num_blocks + + # Create SSD config + ssd_config = { + 'nqn': nqn, + 'nsid': nsid, + 'traddr': traddr, + 'trsvcid': int(trsvcid), # Ensure trsvcid is integer + 'base': 0, + 'size': size, + 'master_server_address': master_server_address, + 'metadata_server': '' + } + + ssd_configs.append(ssd_config) + logging.info(f"Found SSD: nqn={nqn}, nsid={nsid}, traddr={traddr}, size={size}") + + except Exception as e: + logging.error(f"Failed to get SSD info from {ip}: {e}") + continue + + if not ssd_configs: + raise RuntimeError("No SSD information found from remote targets") + + return ssd_configs + + def _setup_logging(self): + logging.basicConfig( + level=logging.INFO, + format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' + ) + + def start_ssd_service(self): + success_count = 0 + skipped_count = 0 + failed_count = 0 + total = len(self.config_list) + + for i, cfg in enumerate(self.config_list): + try: + logging.info("Registering SSD %d/%d: nqn=%s, traddr=%s", i + 1, total, cfg.get("nqn"), cfg.get("traddr")) + + # Create register instance and register SSD + self.register = MooncakeDistributedNoFRegister() + ret = self.register.real_register( + cfg["nqn"], + cfg["nsid"], + cfg["traddr"], + cfg["trsvcid"], + cfg["base"], + cfg["size"], + cfg["master_server_address"] + ) + + if ret != 0: + raise RuntimeError(f"Registration failed with code {ret}") + + logging.info("Register SSD %d/%d succeeded", i + 1, total) + success_count += 1 + + except Exception as e: + # Check if the error is due to the segment already existing on the server + if "SEGMENT_ALREADY_EXISTS" in str(e) or "segment already exists" in str(e): + logging.info("SSD %d/%d (nqn=%s, traddr=%s) already registered on server, skipping", + i + 1, total, cfg.get("nqn"), cfg.get("traddr")) + skipped_count += 1 + else: + logging.error("Failed to register SSD %d/%d: %s", i + 1, total, e) + failed_count += 1 + + # Summary + logging.info("SSD registration summary:") + logging.info("- Total SSDs: %d", total) + logging.info("- Successfully registered: %d", success_count) + logging.info("- Already registered (skipped): %d", skipped_count) + logging.info("- Failed: %d", failed_count) + + # Return success if all SSDs were either registered or already existed + return failed_count == 0 + + +def parse_arguments(): + parser = argparse.ArgumentParser(description='Mooncake SSD Register with REST API') + parser.add_argument('--master_server_address', type=str, + help='Master server address (e.g., 192.168.65.81:50051)', + required=True) + parser.add_argument('--spdk_target_info', action='append', + help='SPDK target information (e.g., "ip:192.168.65.56 path:/home")', + required=True) + parser.add_argument('--username', type=str, default='root', + help='SSH username for target nodes (default: root)') + parser.add_argument('--port', type=int, default=22, + help='SSH port for target nodes (default: 22)') + parser.add_argument('--password', type=str, + help='SSH password for target nodes') + parser.add_argument('--key-file', type=str, + help='SSH private key file path') + parser.add_argument('-D', '--define', action='append', + help='Override configuration fields globally (e.g., -Dtrsvcid=4420)', + default=[]) + return parser.parse_args() + + +def main(): + args = parse_arguments() + + cli_config = {} + for item in args.define: + if '=' in item: + key, value = item.split('=', 1) + cli_config[key] = value + else: + logging.warning(f"Ignoring invalid CLI config: {item}") + + # Add master_server_address to cli_config if provided + if args.master_server_address: + cli_config['master_server_address'] = args.master_server_address + cli_config['username'] = args.username + cli_config['port'] = args.port + if args.password: + cli_config['password'] = args.password + if args.key_file: + cli_config['key_file'] = args.key_file + + register = MooncakeNoFRegister(cli_config, args.spdk_target_info) + success = register.start_ssd_service() + if not success: + exit(1) + +if __name__ == "__main__": + main() diff --git a/mooncake-wheel/mooncake/mooncake_ssd_unregister.py b/mooncake-wheel/mooncake/mooncake_ssd_unregister.py new file mode 100644 index 00000000..ec84b21b --- /dev/null +++ b/mooncake-wheel/mooncake/mooncake_ssd_unregister.py @@ -0,0 +1,361 @@ +#!/usr/bin/env python3 +# python3 -m mooncake.mooncake_ssd_unregister --master_server_address=192.168.65.81:50051 --spdk_target_info="ip:192.168.65.56 ns:1 nqn:nqn.2016-06.io.spdk:cnode1" + +import argparse +import logging +import json +import re +import shlex +from typing import List, Dict, Any + +import paramiko + +from mooncake.store import MooncakeDistributedNoFRegister +from mooncake.mooncake_config import MooncakeConfig + + +class MooncakeNoFUnregister: + """ + Unregisters SSDs from Mooncake master server by endpoint information. + """ + + def __init__(self, cli_config: dict = None, spdk_targets: List[str] = None): + self.register = None + self.config_list: List[Dict[str, Any]] = [] + self.cli_config = cli_config or {} + self.spdk_targets = spdk_targets or [] + self._setup_logging() + + try: + # Only support getting SSD info from command line + if self.spdk_targets: + # Get SSD info from command line parameters + master_server_address = self.cli_config.get('master_server_address') + if not master_server_address: + raise ValueError("master_server_address is required when using spdk_target_info") + self.config_list = self._parse_spdk_targets(master_server_address) + else: + raise ValueError("spdk_target_info is required") + + # Apply CLI overrides to every config (if key exists) + for config in self.config_list: + for key, value in self.cli_config.items(): + if key in config: + # Convert trsvcid/nsid to int if needed + if key in ("trsvcid", "nsid"): + try: + config[key] = int(value) + except ValueError: + logging.warning(f"Invalid integer for {key}: {value}") + else: + config[key] = value + + # Remove duplicate SSDs based on their unique identifiers + self._remove_duplicate_ssds() + + logging.info("Loaded %d SSD configuration(s) to unregister", len(self.config_list)) + except Exception as e: + logging.error("Configuration load failed: %s", e) + raise + + def _remove_duplicate_ssds(self): + """ + Remove duplicate SSD configurations from the config list + """ + unique_configs = [] + seen_keys = set() + + for config in self.config_list: + # Generate unique key directly + key = (config['nqn'], config['nsid'], config['traddr'], config['trsvcid']) + if key not in seen_keys: + seen_keys.add(key) + unique_configs.append(config) + + if len(unique_configs) < len(self.config_list): + logging.info(f"Removed {len(self.config_list) - len(unique_configs)} duplicate SSD configuration(s)") + + self.config_list = unique_configs + + def _execute_ssh_command(self, ip: str, command: str, path: str) -> str: + """ + Execute command on remote server via SSH + """ + ssh = paramiko.SSHClient() + ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) + try: + ssh.connect( + ip, + port=int(self.cli_config.get('port', 22)), + username=self.cli_config.get('username', 'root'), + password=self.cli_config.get('password'), + key_filename=self.cli_config.get('key_file'), + timeout=10 + ) + + # Try multiple possible paths to find the RPC script + possible_paths = [ + path, # Direct path provided by user + f"{path}/spdk" # Common case: spdk is a subdirectory + ] + + for test_path in possible_paths: + full_command = f"cd {shlex.quote(test_path)} && test -f scripts/rpc.py" + stdin, stdout, stderr = ssh.exec_command(full_command, timeout=5) + if stdout.channel.recv_exit_status() == 0: + # Found the script, execute the actual command + full_command = f"cd {shlex.quote(test_path)} && {command}" + stdin, stdout, stderr = ssh.exec_command(full_command, timeout=30) + exit_status = stdout.channel.recv_exit_status() + output = stdout.read().decode('utf-8') + error = stderr.read().decode('utf-8') + if exit_status != 0: + logging.error(f"SSH command error on {ip}: {error}") + raise RuntimeError(f"SSH command failed: {error or output}") + return output + + # If we get here, none of the paths worked + raise RuntimeError(f"Could not find scripts/rpc.py in any of the possible paths: {possible_paths}") + finally: + ssh.close() + + def _parse_spdk_targets(self, master_server_address: str) -> List[Dict[str, Any]]: + """ + Parse spdk target info strings like "ip:192.168.65.56" or "ip:192.168.65.56 ns:2" + or "ip:192.168.65.56 path:/home/spdk" to get actual namespaces from SPDK target + """ + ssd_configs = [] + + for target_info in self.spdk_targets: + # Parse target info + target = {} + parts = target_info.split() + for part in parts: + if ':' in part: + key, value = part.split(':', 1) + target[key.strip()] = value.strip() + + # Validate required fields + if 'ip' not in target: + raise ValueError("spdk_target_info must contain 'ip' field") + + ip = target['ip'] + specified_ns = int(target['ns']) if 'ns' in target else None + path = target.get('path') + + # We need to know the NQN to build the te_endpoint + # For now, we'll use a default NQN pattern (this should be improved) + default_nqn = "nqn.2016-06.io.spdk:cnode1" + nqn = target.get('nqn', default_nqn) + + # Default transport parameters + trsvcid = int(target.get('port', '4420')) + trtype = target.get('trtype', 'RDMA') + + # Create SSD config for each namespace (or specified ns only) + if specified_ns is not None: + # Unregister specific namespace + ssd_config = { + 'nqn': nqn, + 'nsid': specified_ns, + 'traddr': ip, + 'trsvcid': trsvcid, + 'base': 0, + 'size': 0, # Size is not needed for unregister + 'master_server_address': master_server_address, + 'metadata_server': '' + } + ssd_configs.append(ssd_config) + logging.info(f"Will unregister SSD: nqn={nqn}, nsid={specified_ns}, traddr={ip}") + elif path is not None: + # Get actual namespaces from SPDK target + logging.info(f"Getting namespace info from target: {ip} (path: {path})") + try: + # Get subsystems info + subsystems_cmd = "./scripts/rpc.py nvmf_get_subsystems" + subsystems_output = self._execute_ssh_command(ip, subsystems_cmd, path) + subsystems = json.loads(subsystems_output) + + # Process each subsystem + for subsystem in subsystems: + if subsystem.get('subtype') != 'NVMe': + continue + + subsystem_nqn = subsystem.get('nqn') + listen_addresses = subsystem.get('listen_addresses', []) + + if not subsystem_nqn or not listen_addresses: + continue + + # Get transport info from first listen address + traddr = listen_addresses[0].get('traddr') + target_trsvcid = listen_addresses[0].get('trsvcid') + + if not traddr or not target_trsvcid: + continue + + # Use the nqn from the subsystem if not specified + current_nqn = nqn if nqn != default_nqn else subsystem_nqn + current_trsvcid = int(target_trsvcid) if target_trsvcid else trsvcid + + # Process each namespace + namespaces = subsystem.get('namespaces', []) + for namespace in namespaces: + nsid = namespace.get('nsid') + + if not nsid: + continue + + # Create SSD config + ssd_config = { + 'nqn': current_nqn, + 'nsid': nsid, + 'traddr': traddr, + 'trsvcid': current_trsvcid, + 'base': 0, + 'size': 0, # Size is not needed for unregister + 'master_server_address': master_server_address, + 'metadata_server': '' + } + ssd_configs.append(ssd_config) + logging.info(f"Will unregister SSD: nqn={current_nqn}, nsid={nsid}, traddr={traddr}") + + except Exception as e: + logging.error(f"Failed to get namespace info from {ip}: {e}") + logging.warning("Falling back to unregistering default namespace (nsid=1)") + # Fall back to unregistering default namespace + ssd_config = { + 'nqn': nqn, + 'nsid': 1, + 'traddr': ip, + 'trsvcid': trsvcid, + 'base': 0, + 'size': 0, + 'master_server_address': master_server_address, + 'metadata_server': '' + } + ssd_configs.append(ssd_config) + logging.info(f"Will unregister SSD (fallback): nqn={nqn}, nsid=1, traddr={ip}") + else: + # No path provided, unregister default namespace only + logging.warning("No 'path' provided in spdk_target_info, cannot query actual namespaces from SPDK target") + logging.warning("Will unregister default namespace (nsid=1) only") + ssd_config = { + 'nqn': nqn, + 'nsid': 1, + 'traddr': ip, + 'trsvcid': trsvcid, + 'base': 0, + 'size': 0, + 'master_server_address': master_server_address, + 'metadata_server': '' + } + ssd_configs.append(ssd_config) + logging.info(f"Will unregister SSD: nqn={nqn}, nsid=1, traddr={ip}") + + if not ssd_configs: + raise RuntimeError("No SSD configuration generated from target info") + + return ssd_configs + + def _setup_logging(self): + logging.basicConfig( + level=logging.INFO, + format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' + ) + + def start_ssd_unregister_service(self): + success_count = 0 + failed_count = 0 + total = len(self.config_list) + + for i, cfg in enumerate(self.config_list): + try: + logging.info("Unregistering SSD %d/%d: nqn=%s, traddr=%s, nsid=%d", i + 1, total, cfg.get("nqn"), cfg.get("traddr"), cfg.get("nsid")) + + # Create register instance and unregister SSD + self.register = MooncakeDistributedNoFRegister() + ret = self.register.real_unregister_by_endpoint( + cfg["nqn"], + cfg["nsid"], + cfg["traddr"], + cfg["trsvcid"], + cfg["master_server_address"] + ) + + if ret != 0: + raise RuntimeError(f"Unregistration failed with code {ret}") + + logging.info("Unregister SSD %d/%d succeeded", i + 1, total) + success_count += 1 + + except Exception as e: + logging.error("Failed to unregister SSD %d/%d: %s", i + 1, total, e) + failed_count += 1 + + # Summary + logging.info("SSD unregistration summary:") + logging.info("- Total SSDs: %d", total) + logging.info("- Successfully unregistered: %d", success_count) + logging.info("- Failed: %d", failed_count) + + # Return success if all SSDs were unregistered successfully + return failed_count == 0 + + +def parse_arguments(): + parser = argparse.ArgumentParser(description='Mooncake SSD Unregister with REST API') + parser.add_argument('--master_server_address', type=str, + help='Master server address (e.g., 192.168.65.81:50051)', + default='192.168.65.81:50051') + parser.add_argument('--spdk_target_info', action='append', + help='SPDK target information (e.g., "ip:192.168.65.56 ns:2 nqn:nqn.2016-06.io.spdk:cnode1" or "ip:192.168.65.56 path:/home/spdk")', + default=None) + parser.add_argument('--username', type=str, default='root', + help='SSH username for target nodes (default: root)') + parser.add_argument('--port', type=int, default=22, + help='SSH port for target nodes (default: 22)') + parser.add_argument('--password', type=str, + help='SSH password for target nodes') + parser.add_argument('--key-file', type=str, + help='SSH private key file path') + parser.add_argument('-D', '--define', action='append', + help='Override configuration fields globally (e.g., -Dtrsvcid=4420)', + default=[]) + args = parser.parse_args() + + # If no spdk_target_info is provided, use default + if args.spdk_target_info is None: + args.spdk_target_info = ['ip:192.168.65.56 ns:1 nqn:nqn.2016-06.io.spdk:cnode1'] + + return args + + +def main(): + args = parse_arguments() + + cli_config = {} + for item in args.define: + if '=' in item: + key, value = item.split('=', 1) + cli_config[key] = value + else: + logging.warning(f"Ignoring invalid CLI config: {item}") + + # Add master_server_address to cli_config if provided + if args.master_server_address: + cli_config['master_server_address'] = args.master_server_address + cli_config['username'] = args.username + cli_config['port'] = args.port + if args.password: + cli_config['password'] = args.password + if args.key_file: + cli_config['key_file'] = args.key_file + + unregister = MooncakeNoFUnregister(cli_config, args.spdk_target_info) + success = unregister.start_ssd_unregister_service() + if not success: + exit(1) + +if __name__ == "__main__": + main() diff --git a/mooncake-wheel/mooncake/spdk_tgt_create.py b/mooncake-wheel/mooncake/spdk_tgt_create.py new file mode 100644 index 00000000..f5d85543 --- /dev/null +++ b/mooncake-wheel/mooncake/spdk_tgt_create.py @@ -0,0 +1,520 @@ +#!/usr/bin/env python3 +# Tool to remotely create SPDK targets on multiple nodes +# Usage: python3 -m mooncake.spdk_tgt_create --spdk_target_info="ip:192.168.65.56 path:/home/spdk pci:0000:01:00.0,0000:02:00.0" --spdk_target_info="ip:192.168.65.57 path:/home/spdk" + +import argparse +import logging +import paramiko +import re +import shlex +import time +from typing import List, Dict, Any, Optional + + +class SPDKTgtCreator: + """ + Remotely creates SPDK targets on multiple nodes via SSH. + """ + + DEFAULT_TRANSPORT_OPTIONS = { + 'trtype': 'RDMA', + 'max_queue_depth': 128, + 'max_io_qpairs_per_ctrlr': 127, + 'max_io_size': 4096, + 'in_capsule_data_size': 131072, + 'io_unit_size': 131072, + 'max_aq_depth': 128, + 'num_shared_buffers': 4096, + 'buf_cache_size': 32, + } + + TRANSPORT_RPC_FLAGS = { + 'trtype': '-t', + 'max_queue_depth': '-q', + 'max_io_qpairs_per_ctrlr': '-m', + 'max_io_size': '-c', + 'in_capsule_data_size': '-i', + 'io_unit_size': '-u', + 'max_aq_depth': '-a', + 'num_shared_buffers': '-n', + 'buf_cache_size': '-b', + } + + def __init__(self, spdk_targets: List[str], transport_options: Dict[str, Any] = None, core_mask: str = '0xff'): + self.spdk_targets = spdk_targets + self.core_mask = core_mask + self.transport_options = dict(self.DEFAULT_TRANSPORT_OPTIONS) + if transport_options: + self.transport_options.update(transport_options) + self._setup_logging() + self.target_configs = self._parse_spdk_targets() + + def _setup_logging(self): + logging.basicConfig( + level=logging.INFO, + format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' + ) + self.logger = logging.getLogger(self.__class__.__name__) + + def _parse_spdk_targets(self) -> List[Dict[str, Any]]: + """ + Parse SPDK target information from command line arguments. + Format: "ip: path: [pci:, ...]" + """ + target_configs = [] + + for target_info in self.spdk_targets: + target = { + 'ip': None, + 'path': None, + 'pci_devices': [] + } + + # Split the target info by spaces + parts = target_info.split() + + # Simple state machine to parse the target info + state = None # Can be 'ip', 'path', or 'pci' + + for part in parts: + if ':' in part: + # This is a key-value pair + key, value = part.split(':', 1) + key = key.strip() + value = value.strip() + + if key == 'ip': + target['ip'] = value + state = 'ip' + elif key == 'path': + target['path'] = value + state = 'path' + elif key == 'pci': + # Parse PCI devices separated by commas + if value: + # Split by commas and strip whitespace + pci_list = [dev.strip() for dev in value.split(',') if dev.strip()] + target['pci_devices'].extend(pci_list) + state = 'pci' + elif state == 'pci': + pci_list = [dev.strip() for dev in part.split(',') if dev.strip()] + target['pci_devices'].extend(pci_list) + else: + # This is a continuation of the current state + if state == 'path': + # Path might contain spaces (unlikely but possible) + target['path'] += ' ' + part + elif state == 'pci': + pci_list = [dev.strip() for dev in part.split(',') if dev.strip()] + target['pci_devices'].extend(pci_list) + + # Validate required fields + if not target['ip']: + raise ValueError("Each spdk_target_info must contain 'ip' field") + if not target['path']: + raise ValueError("Each spdk_target_info must contain 'path' field") + + target_configs.append(target) + pci_info = target['pci_devices'] if target['pci_devices'] else 'auto-discover' + self.logger.info(f"Parsed target: IP={target['ip']}, Path={target['path']}, PCI devices={pci_info}") + + return target_configs + + def _ssh_connect(self, ip: str, username: str = 'root', password: str = None, key_file: str = None) -> paramiko.SSHClient: + """ + Establish an SSH connection to the target host. + """ + ssh = paramiko.SSHClient() + ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) + + try: + if key_file: + self.logger.info(f"Connecting to {ip} using key file {key_file}") + ssh.connect(ip, username=username, key_filename=key_file) + else: + self.logger.info(f"Connecting to {ip} using password authentication") + ssh.connect(ip, username=username, password=password) + return ssh + except Exception as e: + self.logger.error(f"Failed to connect to {ip}: {e}") + raise + + def _execute_command(self, ssh: paramiko.SSHClient, command: str, working_dir: str = None, sudo: bool = False, log_errors: bool = True, timeout: Optional[int] = 30) -> tuple: + """ + Execute a command on the remote host via SSH. + """ + if working_dir: + command = f"cd {working_dir} && {command}" + + if sudo: + command = f"sudo {command}" + + self.logger.debug(f"Executing command: {command}") + + stdin, stdout, stderr = ssh.exec_command(command, timeout=timeout) + exit_status = stdout.channel.recv_exit_status() + output = stdout.read().decode('utf-8') + error = stderr.read().decode('utf-8') + + if output: + self.logger.debug(f"Command output: {output}") + if error: + self.logger.debug(f"Command error: {error}") + if exit_status != 0: + if log_errors: + self.logger.error(f"Command failed with exit code {exit_status}: {command}") + if output: + self.logger.error(f"Command output: {output}") + self.logger.error(f"Error output: {error}") + raise RuntimeError(f"Command execution failed: {error or output}") + + return output, error + + def _discover_nvme_pci_devices(self, ssh: paramiko.SSHClient) -> List[str]: + """ + Discover SPDK-ready or unmounted NVMe controller PCI addresses on the target host. + """ + self.logger.info("No PCI devices specified, discovering SPDK-ready or unmounted NVMe PCI devices") + output, _ = self._execute_command( + ssh, + r"""for dev in /sys/bus/pci/devices/*; do + class=$(cat "$dev/class" 2>/dev/null || true) + case "$class" in + 0x0108*) ;; + *) continue ;; + esac + pci=$(basename "$dev") + driver=$(basename "$(readlink "$dev/driver" 2>/dev/null)" 2>/dev/null || true) + case "$driver" in + vfio-pci|uio_pci_generic|igb_uio) + echo "USE $pci $driver" + continue + ;; + esac + has_block=0 + mounted=0 + for block in /sys/block/nvme*n*; do + [ -e "$block" ] || continue + real_device=$(readlink -f "$block/device" 2>/dev/null || true) + case "$real_device" in + *"/$pci"/*|*"/$pci") ;; + *) continue ;; + esac + has_block=1 + disk=$(basename "$block") + if lsblk -nr -o MOUNTPOINT "/dev/$disk" 2>/dev/null | grep -q '[^[:space:]]'; then + mounted=1 + fi + done + if [ "$has_block" -eq 0 ]; then + echo "SKIP $pci no_block_device" + elif [ "$mounted" -eq 0 ]; then + echo "USE $pci" + else + echo "SKIP $pci mounted" + fi +done""" + ) + + pci_devices = [] + pci_pattern = re.compile( + r'^(?:[0-9a-fA-F]{4}:)?[0-9a-fA-F]{2}:[0-9a-fA-F]{2}\.[0-7]$' + ) + for line in output.splitlines(): + fields = line.split() + if len(fields) < 2: + continue + action, pci = fields[0], fields[1] + if action == 'USE' and pci_pattern.match(pci): + pci_devices.append(pci) + elif action == 'SKIP' and pci_pattern.match(pci): + reason = ' '.join(fields[2:]) or 'not eligible' + self.logger.warning(f"Skipping NVMe PCI device {pci}: {reason}") + + if not pci_devices: + raise RuntimeError("No SPDK-ready or unmounted NVMe PCI devices found on the target host") + + self.logger.info(f"Selected NVMe PCI devices for SPDK setup: {', '.join(pci_devices)}") + return pci_devices + + def _filter_spdk_ready_pci_devices(self, ssh: paramiko.SSHClient, pci_devices: List[str], strict: bool) -> List[str]: + """ + Keep PCI devices that are bound to an SPDK-compatible userspace driver. + """ + if not pci_devices: + return [] + + pci_args = ' '.join(shlex.quote(pci) for pci in pci_devices) + output, _ = self._execute_command( + ssh, + f"""for pci in {pci_args}; do + driver=$(basename "$(readlink "/sys/bus/pci/devices/$pci/driver" 2>/dev/null)" 2>/dev/null || true) + case "$driver" in + vfio-pci|uio_pci_generic|igb_uio) echo "READY $pci $driver" ;; + "") echo "NOT_READY $pci no_driver" ;; + *) echo "NOT_READY $pci $driver" ;; + esac +done""" + ) + + ready_devices = [] + not_ready_devices = [] + for line in output.splitlines(): + fields = line.split() + if len(fields) < 3: + continue + state, pci, driver = fields[0], fields[1], fields[2] + if state == 'READY': + ready_devices.append(pci) + elif state == 'NOT_READY': + not_ready_devices.append((pci, driver)) + + if not_ready_devices: + details = ', '.join(f"{pci} ({driver})" for pci, driver in not_ready_devices) + if strict: + raise RuntimeError( + "Some PCI devices are not available to SPDK after setup.sh: " + f"{details}. Check whether they are mounted or still bound to the kernel driver." + ) + self.logger.warning(f"Skipping PCI devices not available to SPDK after setup.sh: {details}") + + if not ready_devices: + raise RuntimeError("No PCI devices are available to SPDK after setup.sh") + + self.logger.info(f"PCI devices available to SPDK: {', '.join(ready_devices)}") + return ready_devices + + def _start_spdk_tgt(self, ssh: paramiko.SSHClient, spdk_path: str) -> None: + """ + Start the SPDK NVMF target service in the background. + """ + # Check if tgt is already running + try: + self._execute_command(ssh, "pgrep -x nvmf_tgt", log_errors=False) + self.logger.info("SPDK tgt service is already running, stopping it first") + self._execute_command(ssh, "pkill -x nvmf_tgt") + time.sleep(2) # Give it time to stop + except RuntimeError: + self.logger.debug("SPDK tgt service is not running, will start it") + + # Start tgt in the background using absolute path + self.logger.info(f"Starting SPDK tgt service with core mask {self.core_mask}") + tgt_binary = f"{spdk_path}/build/bin/nvmf_tgt" + log_file = f"{spdk_path}/tgt.log" + self._execute_command( + ssh, + f"nohup {tgt_binary} -m {shlex.quote(self.core_mask)} > {log_file} 2>&1 &", + timeout=None + ) + time.sleep(3) # Give it time to start + + def _setup_spdk(self, ssh: paramiko.SSHClient, spdk_path: str, pci_devices: List[str]) -> None: + """ + Setup SPDK with the specified PCI devices. + """ + self.logger.info(f"Setting up SPDK with PCI devices: {', '.join(pci_devices)}") + pci_allowed = ' '.join(pci_devices) + setup_script = f"{spdk_path}/scripts/setup.sh" + self._execute_command(ssh, f"PCI_ALLOWED='{pci_allowed}' {setup_script}", sudo=True) + + def _format_transport_options(self) -> str: + """ + Format transport options for the SPDK nvmf_create_transport RPC. + """ + formatted_options = [] + for option, flag in self.TRANSPORT_RPC_FLAGS.items(): + value = self.transport_options[option] + formatted_options.extend([flag, shlex.quote(str(value))]) + return ' '.join(formatted_options) + + def _create_transport(self, ssh: paramiko.SSHClient, spdk_path: str) -> None: + """ + Create NVMe-oF transport for SPDK. + """ + self.logger.info(f"Creating {self.transport_options['trtype']} transport") + rpc_script = f"{spdk_path}/scripts/rpc.py" + transport_options = self._format_transport_options() + self._execute_command(ssh, f"{rpc_script} nvmf_create_transport {transport_options}") + + def _create_bdevs(self, ssh: paramiko.SSHClient, spdk_path: str, pci_devices: List[str]) -> List[str]: + """ + Create block devices for the specified PCI devices. + """ + bdevs = [] + rpc_script = f"{spdk_path}/scripts/rpc.py" + for i, pci in enumerate(pci_devices): + bdev_name = f"Nvme{i}" + self.logger.info(f"Creating bdev {bdev_name} for PCI {pci}") + self._execute_command(ssh, f"{rpc_script} bdev_nvme_attach_controller -b {bdev_name} -t PCIe -a {pci}") + bdevs.append(f"{bdev_name}n1") + self.logger.info(f"Attached PCI {pci} as bdev {bdev_name}n1") + return bdevs + + def _create_subsystem(self, ssh: paramiko.SSHClient, spdk_path: str) -> str: + """ + Create an NVMF subsystem. + """ + subsystem_nqn = "nqn.2016-06.io.spdk:cnode1" + self.logger.info(f"Creating subsystem {subsystem_nqn}") + rpc_script = f"{spdk_path}/scripts/rpc.py" + self._execute_command(ssh, f"{rpc_script} nvmf_create_subsystem {subsystem_nqn} -a -s SPDK00000000000001 -m 12") + return subsystem_nqn + + def _add_namespaces(self, ssh: paramiko.SSHClient, spdk_path: str, subsystem_nqn: str, bdevs: List[str]) -> None: + """ + Add namespaces to the subsystem. + """ + rpc_script = f"{spdk_path}/scripts/rpc.py" + for bdev in bdevs: + self.logger.info(f"Adding namespace {bdev} to subsystem {subsystem_nqn}") + self._execute_command(ssh, f"{rpc_script} nvmf_subsystem_add_ns {subsystem_nqn} {bdev}") + + def _add_listener(self, ssh: paramiko.SSHClient, spdk_path: str, subsystem_nqn: str, ip: str) -> None: + """ + Add a listener to the subsystem. + """ + trtype = self.transport_options['trtype'] + self.logger.info(f"Adding {trtype} listener on {ip}:4420") + rpc_script = f"{spdk_path}/scripts/rpc.py" + self._execute_command(ssh, f"{rpc_script} nvmf_subsystem_add_listener {subsystem_nqn} -t {shlex.quote(str(trtype))} -a {ip} -s 4420") + + def deploy_target(self, target_config: Dict[str, Any]) -> bool: + """ + Deploy SPDK target on a single node. + """ + ip = target_config['ip'] + spdk_path = target_config['path'] + pci_devices = target_config['pci_devices'] + + self.logger.info(f"Deploying SPDK target on {ip}") + + try: + # Establish SSH connection + ssh = self._ssh_connect(ip) + + try: + auto_discovered = not pci_devices + if not pci_devices: + pci_devices = self._discover_nvme_pci_devices(ssh) + else: + self.logger.info(f"Using explicitly specified PCI devices for {ip}: {', '.join(pci_devices)}") + + # Setup SPDK + self._setup_spdk(ssh, spdk_path, pci_devices) + pci_devices = self._filter_spdk_ready_pci_devices( + ssh, + pci_devices, + strict=not auto_discovered + ) + self.logger.info(f"Target {ip} will expose PCI devices: {', '.join(pci_devices)}") + + # Start tgt service + self._start_spdk_tgt(ssh, spdk_path) + + # Create transport + self._create_transport(ssh, spdk_path) + + # Create bdevs + bdevs = self._create_bdevs(ssh, spdk_path, pci_devices) + + # Create subsystem + subsystem_nqn = self._create_subsystem(ssh, spdk_path) + + # Add namespaces + self._add_namespaces(ssh, spdk_path, subsystem_nqn, bdevs) + + # Add listener + self._add_listener(ssh, spdk_path, subsystem_nqn, ip) + + self.logger.info(f"Successfully deployed SPDK target on {ip}") + return True + finally: + ssh.close() + except Exception as e: + self.logger.error(f"Failed to deploy SPDK target on {ip}: {e}") + return False + + def deploy_all_targets(self) -> bool: + """ + Deploy SPDK targets on all specified nodes. + """ + success_count = 0 + total_count = len(self.target_configs) + + for i, target_config in enumerate(self.target_configs): + self.logger.info(f"=== Deploying target {i+1}/{total_count} ===") + if self.deploy_target(target_config): + success_count += 1 + + self.logger.info("=== Deployment Summary ===") + self.logger.info(f"Total targets: {total_count}") + self.logger.info(f"Successfully deployed: {success_count}") + self.logger.info(f"Failed: {total_count - success_count}") + + return success_count == total_count + + +def parse_arguments(): + parser = argparse.ArgumentParser(description='SPDK Target Creator Tool') + parser.add_argument( + '--spdk_target_info', + action='append', + help=('SPDK target information (e.g., "ip:192.168.65.56 ' + 'path:/home/spdk pci:0000:01:00.0,0000:02:00.0"). If pci is ' + 'omitted, SPDK-ready or unmounted NVMe devices on the target are used.'), + required=True + ) + parser.add_argument('--core-mask', type=str, default='0xff', + help='CPU core mask used to start nvmf_tgt with -m (default: 0xff)') + parser.add_argument('--transport-type', type=str, default='RDMA', + help='NVMe-oF transport type for nvmf_create_transport (default: RDMA)') + parser.add_argument('--max-queue-depth', type=int, default=128, + help='Max number of outstanding I/O per queue (default: 128)') + parser.add_argument('--max-io-qpairs-per-ctrlr', type=int, default=127, + help='Max number of I/O qpairs per controller (default: 127)') + parser.add_argument('--max-io-size', type=int, default=4096, + help='Max I/O size in bytes (default: 4096)') + parser.add_argument('--in-capsule-data-size', type=int, default=131072, + help='Max in-capsule data size in bytes (default: 131072)') + parser.add_argument('--io-unit-size', type=int, default=131072, + help='I/O unit size in bytes (default: 131072)') + parser.add_argument('--max-aq-depth', type=int, default=128, + help='Max number of admin commands per admin queue (default: 128)') + parser.add_argument('--num-shared-buffers', type=int, default=4096, + help='Number of pooled data buffers available to the transport (default: 4096)') + parser.add_argument('--buf-cache-size', type=int, default=32, + help='Number of shared buffers reserved for each poll group (default: 32)') + parser.add_argument('--username', type=str, default='root', + help='SSH username for target nodes (default: root)') + parser.add_argument('--password', type=str, + help='SSH password for target nodes') + parser.add_argument('--key-file', type=str, + help='SSH private key file path') + return parser.parse_args() + + +def main(): + args = parse_arguments() + transport_options = { + 'trtype': args.transport_type, + 'max_queue_depth': args.max_queue_depth, + 'max_io_qpairs_per_ctrlr': args.max_io_qpairs_per_ctrlr, + 'max_io_size': args.max_io_size, + 'in_capsule_data_size': args.in_capsule_data_size, + 'io_unit_size': args.io_unit_size, + 'max_aq_depth': args.max_aq_depth, + 'num_shared_buffers': args.num_shared_buffers, + 'buf_cache_size': args.buf_cache_size, + } + + try: + creator = SPDKTgtCreator(args.spdk_target_info, transport_options, args.core_mask) + success = creator.deploy_all_targets() + exit(0 if success else 1) + except Exception as e: + logging.error(f"Error: {e}") + exit(1) + + +if __name__ == '__main__': + main() -- 2.34.1 From 9153043b8664a2d965fd4b013d4ede626a31dcde Mon Sep 17 00:00:00 2001 From: Jiangtian Feng Date: Mon, 8 Jun 2026 10:29:09 +0800 Subject: [PATCH 203/382] [TE] Harden TCP transport: validate remote addresses, fix idle cleanup, add TCP_NODELAY (#2314) * [TE] Harden TCP transport: validate remote addresses, fix idle cleanup, add TCP_NODELAY Fixes #2313 1. Security: ServerSession now validates remote-supplied memory addresses against registered local buffers before use, preventing arbitrary memory read/write from malicious peers. 2. Performance: Set TCP_NODELAY on all server-accept and client-connect paths to eliminate Nagle-induced latency on small control messages. 3. Correctness: Call io_context.restart() after exception in worker loop to prevent busy-spin when io_context enters stopped state. 4. Correctness: Fix cleanupIdleConnections to scan the full deque instead of only the back, so idle connections anywhere in the pool get cleaned. Co-Authored-By: Claude Opus 4.6 * [TE] Fix clang-format violations in TCP transport Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 --- .../transport/tcp_transport/tcp_transport.h | 2 + .../transport/tcp_transport/tcp_transport.cpp | 71 +++++++--- mooncake-transfer-engine/tests/CMakeLists.txt | 4 + .../tests/tcp_address_validation_test.cpp | 126 ++++++++++++++++++ 4 files changed, 184 insertions(+), 19 deletions(-) create mode 100644 mooncake-transfer-engine/tests/tcp_address_validation_test.cpp diff --git a/mooncake-transfer-engine/include/transport/tcp_transport/tcp_transport.h b/mooncake-transfer-engine/include/transport/tcp_transport/tcp_transport.h index 94dc2d9d..3e537cfe 100644 --- a/mooncake-transfer-engine/include/transport/tcp_transport/tcp_transport.h +++ b/mooncake-transfer-engine/include/transport/tcp_transport/tcp_transport.h @@ -104,6 +104,8 @@ class TcpTransport : public Transport { void startTransfer(Slice *slice); + bool validateAddress(uint64_t addr, uint64_t size) const; + const char *getName() const override { return "tcp"; } private: diff --git a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp index 9df163de..81cf9ab5 100644 --- a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp +++ b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp @@ -70,12 +70,17 @@ static bool isCudaMemory(void* addr) { // Forward declaration class TcpTransport; +using ValidateAddrFn = std::function; + // Server-side session: handles one transfer request on a persistent connection struct ServerSession : public std::enable_shared_from_this { - explicit ServerSession(std::shared_ptr socket) - : socket_(std::move(socket)) {} + explicit ServerSession(std::shared_ptr socket, + ValidateAddrFn validate_addr) + : socket_(std::move(socket)), + validate_addr_(std::move(validate_addr)) {} std::shared_ptr socket_; + ValidateAddrFn validate_addr_; SessionHeader header_; uint64_t total_transferred_bytes_; char* local_buffer_; @@ -95,8 +100,6 @@ struct ServerSession : public std::enable_shared_from_this { *socket_, asio::buffer(&header_, sizeof(SessionHeader)), [this, self](const asio::error_code& ec, std::size_t len) { if (ec || len != sizeof(SessionHeader)) { - // If client closed connection (EOF), this is normal - don't - // log if (ec.value() != asio::error::eof) { LOG(WARNING) << "ServerSession::readHeader failed. Error: " @@ -104,10 +107,20 @@ struct ServerSession : public std::enable_shared_from_this { << ", bytes read: " << len; } session_mutex_.unlock(); - return; // Don't continue, socket will be closed + return; } local_buffer_ = (char*)(le64toh(header_.addr)); + uint64_t size = le64toh(header_.size); + if (validate_addr_ && + !validate_addr_((uint64_t)local_buffer_, size)) { + LOG(ERROR) << "ServerSession: remote-supplied address 0x" + << std::hex << (uint64_t)local_buffer_ + << std::dec << " with size " << size + << " is not within any registered buffer"; + session_mutex_.unlock(); + return; + } if (header_.opcode == (uint8_t)TransferRequest::WRITE) readBody(); else @@ -492,7 +505,8 @@ struct ClientSession : public std::enable_shared_from_this { }; struct TcpContext { - TcpContext(short port) : acceptor(io_context) { + TcpContext(short port, ValidateAddrFn validate_addr) + : acceptor(io_context), validate_addr_(std::move(validate_addr)) { std::error_code ec; asio::ip::tcp::endpoint endpoint(asio::ip::tcp::v6(), port); @@ -522,11 +536,13 @@ struct TcpContext { void doAccept() { acceptor.async_accept([this](asio::error_code ec, tcpsocket socket) { if (!ec) { + asio::error_code nodelay_ec; + socket.set_option(asio::ip::tcp::no_delay(true), nodelay_ec); auto socket_ptr = std::make_shared(std::move(socket)); - auto session = std::make_shared(socket_ptr); - session->start(); // Start processing requests on this - // persistent connection + auto session = + std::make_shared(socket_ptr, validate_addr_); + session->start(); } doAccept(); }); @@ -534,6 +550,7 @@ struct TcpContext { asio::io_context io_context; asio::ip::tcp::acceptor acceptor; + ValidateAddrFn validate_addr_; }; TcpTransport::TcpTransport() : context_(nullptr), running_(false) { @@ -610,7 +627,9 @@ int TcpTransport::install(std::string& local_server_name, close(sockfd); // the above function has opened a socket LOG(INFO) << "TcpTransport: listen on port " << tcp_port; - context_ = new TcpContext(tcp_port); + context_ = new TcpContext(tcp_port, [this](uint64_t addr, uint64_t size) { + return validateAddress(addr, size); + }); running_ = true; thread_ = std::thread(&TcpTransport::worker, this); return 0; @@ -759,6 +778,7 @@ void TcpTransport::worker() { LOG(ERROR) << "TcpTransport::worker encountered an exception " "during doAccept/run: " << e.what(); + context_->io_context.restart(); } } } @@ -774,6 +794,7 @@ std::shared_ptr TcpTransport::getConnection( auto socket_ptr = std::make_shared(context_->io_context); asio::connect(*socket_ptr, endpoint_iterator); + socket_ptr->set_option(asio::ip::tcp::no_delay(true)); return socket_ptr; } catch (std::exception& e) { LOG(ERROR) @@ -826,6 +847,7 @@ std::shared_ptr TcpTransport::getConnection( new_socket = std::make_shared(context_->io_context); asio::connect(*new_socket, endpoint_iterator); + new_socket->set_option(asio::ip::tcp::no_delay(true)); } catch (std::exception& e) { LOG(ERROR) << "TcpTransport::getConnection failed to create connection to " @@ -900,9 +922,8 @@ void TcpTransport::cleanupIdleConnections() { for (auto it = connection_pool_.begin(); it != connection_pool_.end();) { auto& queue = it->second; - // Remove idle connections that exceed timeout - while (!queue.empty()) { - auto& entry = queue.back(); + for (auto entry_it = queue.begin(); entry_it != queue.end();) { + auto& entry = *entry_it; if (!entry->in_use) { auto idle_duration = std::chrono::duration_cast( @@ -913,16 +934,13 @@ void TcpTransport::cleanupIdleConnections() { asio::error_code ec; entry->socket->close(ec); } - queue.pop_back(); - } else { - break; + entry_it = queue.erase(entry_it); + continue; } - } else { - break; } + ++entry_it; } - // Remove empty endpoint queues if (queue.empty()) { it = connection_pool_.erase(it); } else { @@ -931,6 +949,21 @@ void TcpTransport::cleanupIdleConnections() { } } +bool TcpTransport::validateAddress(uint64_t addr, uint64_t size) const { + if (size == 0) return false; + if (addr + size < addr) return false; + + auto desc = metadata_->getSegmentDescByID(LOCAL_SEGMENT_ID); + if (!desc) return false; + + for (const auto& buffer : desc->buffers) { + if (buffer.addr + buffer.length < buffer.addr) continue; + if (buffer.addr <= addr && addr + size <= buffer.addr + buffer.length) + return true; + } + return false; +} + void TcpTransport::startTransfer(Slice* slice) { auto desc = metadata_->getSegmentDescByID(slice->target_id); if (!desc) { diff --git a/mooncake-transfer-engine/tests/CMakeLists.txt b/mooncake-transfer-engine/tests/CMakeLists.txt index 70f5be2a..99392508 100644 --- a/mooncake-transfer-engine/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tests/CMakeLists.txt @@ -65,6 +65,10 @@ target_link_libraries(tcp_transport_test PUBLIC transfer_engine gtest gtest_main add_test(NAME tcp_transport_test COMMAND tcp_transport_test) endif() +add_executable(tcp_address_validation_test ${WORKSPACE}/tcp_address_validation_test.cpp) +target_link_libraries(tcp_address_validation_test PUBLIC gtest gtest_main) +add_test(NAME tcp_address_validation_test COMMAND tcp_address_validation_test) + if (USE_MNNVL) add_executable(nvlink_transport_test ${WORKSPACE}/nvlink_transport_test.cpp) target_link_libraries(nvlink_transport_test PUBLIC transfer_engine gtest gtest_main ) diff --git a/mooncake-transfer-engine/tests/tcp_address_validation_test.cpp b/mooncake-transfer-engine/tests/tcp_address_validation_test.cpp new file mode 100644 index 00000000..d10966e3 --- /dev/null +++ b/mooncake-transfer-engine/tests/tcp_address_validation_test.cpp @@ -0,0 +1,126 @@ +// Copyright 2024 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include + +#include +#include + +namespace { + +struct BufferRange { + uint64_t addr; + uint64_t length; +}; + +// Mirrors TcpTransport::validateAddress logic exactly. +bool validateAddress(uint64_t addr, uint64_t size, + const std::vector& buffers) { + if (size == 0) return false; + if (addr + size < addr) return false; + + for (const auto& buffer : buffers) { + if (buffer.addr + buffer.length < buffer.addr) continue; + if (buffer.addr <= addr && addr + size <= buffer.addr + buffer.length) + return true; + } + return false; +} + +class TcpAddressValidationTest : public ::testing::Test { + protected: + std::vector buffers_; + + void SetUp() override { + buffers_ = { + {0x1000, 0x2000}, // [0x1000, 0x3000) + {0x10000, 0x100000}, // [0x10000, 0x110000) + }; + } +}; + +TEST_F(TcpAddressValidationTest, ExactMatch) { + EXPECT_TRUE(validateAddress(0x1000, 0x2000, buffers_)); + EXPECT_TRUE(validateAddress(0x10000, 0x100000, buffers_)); +} + +TEST_F(TcpAddressValidationTest, WithinBounds) { + EXPECT_TRUE(validateAddress(0x1000, 0x100, buffers_)); + EXPECT_TRUE(validateAddress(0x1500, 0x500, buffers_)); + EXPECT_TRUE(validateAddress(0x2FFF, 1, buffers_)); + EXPECT_TRUE(validateAddress(0x50000, 0x1000, buffers_)); +} + +TEST_F(TcpAddressValidationTest, OutOfBounds) { + EXPECT_FALSE(validateAddress(0x500, 0x100, buffers_)); + EXPECT_FALSE(validateAddress(0x3000, 0x100, buffers_)); + EXPECT_FALSE(validateAddress(0x5000, 0x1000, buffers_)); + EXPECT_FALSE(validateAddress(0x200000, 0x100, buffers_)); +} + +TEST_F(TcpAddressValidationTest, PartialOverlap) { + EXPECT_FALSE(validateAddress(0x2F00, 0x200, buffers_)); + EXPECT_FALSE(validateAddress(0x0F00, 0x200, buffers_)); + EXPECT_FALSE(validateAddress(0x10F000, 0x2000, buffers_)); +} + +TEST_F(TcpAddressValidationTest, ZeroSize) { + EXPECT_FALSE(validateAddress(0x1000, 0, buffers_)); + EXPECT_FALSE(validateAddress(0x0, 0, buffers_)); +} + +TEST_F(TcpAddressValidationTest, IntegerOverflow) { + EXPECT_FALSE(validateAddress(UINT64_MAX, 1, buffers_)); + EXPECT_FALSE(validateAddress(UINT64_MAX - 10, 100, buffers_)); + EXPECT_FALSE(validateAddress(1, UINT64_MAX, buffers_)); +} + +TEST_F(TcpAddressValidationTest, EmptyBufferList) { + std::vector empty; + EXPECT_FALSE(validateAddress(0x1000, 0x100, empty)); +} + +TEST_F(TcpAddressValidationTest, SingleByteAccess) { + EXPECT_TRUE(validateAddress(0x1000, 1, buffers_)); + EXPECT_TRUE(validateAddress(0x2FFF, 1, buffers_)); + EXPECT_FALSE(validateAddress(0x3000, 1, buffers_)); + EXPECT_FALSE(validateAddress(0x0FFF, 1, buffers_)); +} + +TEST_F(TcpAddressValidationTest, AdjacentBuffers) { + std::vector adjacent = { + {0x1000, 0x1000}, // [0x1000, 0x2000) + {0x2000, 0x1000}, // [0x2000, 0x3000) + }; + EXPECT_TRUE(validateAddress(0x1000, 0x1000, adjacent)); + EXPECT_TRUE(validateAddress(0x2000, 0x1000, adjacent)); + // Spanning two buffers should fail + EXPECT_FALSE(validateAddress(0x1800, 0x1000, adjacent)); +} + +TEST_F(TcpAddressValidationTest, LargeBuffer) { + std::vector large = { + {0, UINT64_MAX}, + }; + EXPECT_TRUE(validateAddress(0, 1, large)); + EXPECT_TRUE(validateAddress(0, UINT64_MAX, large)); + EXPECT_TRUE(validateAddress(UINT64_MAX - 1, 1, large)); +} + +} // namespace + +int main(int argc, char** argv) { + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); +} -- 2.34.1 From f7380ba0573678b9b79a87c3f0d0be34623e6baa Mon Sep 17 00:00:00 2001 From: Teng Ma Date: Mon, 8 Jun 2026 10:49:45 +0800 Subject: [PATCH 204/382] [Store] master log journal (#2297) * feat(master): merge master logs into a single journal directory via glog Add a MasterLogJournal module that configures glog so all master diagnostic records (LOG()/VLOG()) are merged into one chronologically ordered file inside a dedicated journal directory, instead of glog's default per-severity files scattered next to the binary. The design reuses glog rather than reimplementing a logging engine: - keeps only glog's lowest-severity sink (which already receives every record) and disables the higher-severity sinks to merge into one file - reuses size-based rotation (FLAGS_max_log_size), time-based retention (google::EnableLogCleaner), the latest-file symlink, periodic flushing (FLAGS_logbufsecs) and stderr mirroring (FLAGS_stderrthreshold) Wire it into mooncake_master via new flags (--enable_log_journal, --log_journal_dir, --log_journal_merge, --log_journal_max_file_size_mb, --log_journal_retention_days, --log_journal_also_log_to_stderr), add a unit test, and document the flags in the deployment guide. Co-authored-by: Teng Ma --- .../deployment/mooncake-store-deployment-guide.md | 3 +++ mooncake-store/src/master.cpp | 10 ++++++++++ 2 files changed, 13 insertions(+) diff --git a/docs/source/deployment/mooncake-store-deployment-guide.md b/docs/source/deployment/mooncake-store-deployment-guide.md index 641695f6..faf64c3a 100644 --- a/docs/source/deployment/mooncake-store-deployment-guide.md +++ b/docs/source/deployment/mooncake-store-deployment-guide.md @@ -39,6 +39,9 @@ This page summarizes useful flags, environment variables, and HTTP endpoints to - `--etcd_endpoints` (str, default empty unless HA config): etcd endpoints, semicolon separated. - `--cluster_id` (str, default `mooncake_cluster`): Cluster ID for persistence in HA mode. +- Logging (optional) + - The master uses glog. When `--log_dir` is set, all severities are merged into a single journal file in that directory (`mooncake_master.INFO.-

    7U*AvIE7*Fk)gr2+b4pV)te1ab*yTMNfA!<6o zx~+INqDth(k?h*p`8Ob4&Y^HC9nVf;VsiIt<>$H`d(VkC%Mr?su`JM*%|IRCjp3W1DTE zp=$au;&Ou77zjJQ-_@@Ld67S98<#U`P3bAT&=EMy9WPtU;-T~SZRbj>KipZaGHS`%Vbq5TJHXCe(+>{A~wsr1P5D$X>e!K-3=SNkn#g><6P1E z+YcUAXQKBs@o%#uu$>%213M zp~MTANt9A;NF??W<=Nh=JdmCzaP97A6Z950@KfSn@ku&Sg7n`$@UXA5p+T(oJ_q~i z?6%6UUp+(Rviuwf2c)J?X~KLFSg7e%klTXdw{zJ%!}7Ez`!DL zwRqIDV!4Jh)@gRQwvzt#W!XtU2E*u%sFPwY{fLFoiO_4!FMtd_Mf_9Rej?!XGl4Nb zy5Nde|8dmWgA2>T02+Uk`zw(AnXVI=&@fg_W)3o+|A zu}FiRmz;rVU#&Et8w)i?wK#)X%F!hX-pN>;0B`9m&I&C$z{X(G1Q3d$zRn6ByLL81 z#$Ul_D(5iBdH{i$0DlX<-F|!ZZ_tM%_zfX1*&&uesPk~w-`aP^WL+M?kY}*^_icri z-@)^vHfFkj%i#4e;$V~`VcOl)dKz?6H)#J0tawD${UOb88y4VHH@*m2C(5wTd>Ij< zif3}0-SkW^T-$}sLZaqs*}4vgcJLu)u2-qdRi$H(-(oNS17WizI-W}x-2V9`W4$T- zY?D`F_MA@1&grE2;ajKno#+AO)l#{xHk+&81M%_YIaJuOtWAc~iN~caipp%(9iK4q zH9f^doNTKTJG_MjBA?K!w}Eh6X+BUbF;$x~;iLaUDB>Y-9W;Y`Syr>i`$W-6;sgT) zglxgP8k5bM4wjPI(I5ADfiKY!n=kDuo|;h;`ppnd0fe*8_pCpp@B&w`y3l!XJ$HmQ z2}?mfFgA6K*Khv5K{r)&pYc*l@$AK}7?8#+Z&&m^)sEc6glE0d5pDC}N1Uwl;JrV} zG_D>N21uzCDannlU4PE?sSnQC4xfFjw4N}>TT9ef^S2q5t9~U_Ouy#QiF&Cl@Y3+H z+8PVl8j*vKz@u?n2b~CIKVmMmZYm{;7ny}R^vfi7y+hrxUUU+qf~1lVr(7ci$uufG zF-h}ERTe2;MpuSZVvLuhlH$akDr6amYh=g+QtQkh!=}b!qI-Zu9b(Me!!+@@7vDHOp67g?~SaHy^{?CDJL9d5;`@f)7)V z#C^PD$$$T$w|fRowkf@a*UdvN@tucT`@CkgZ*G6LbYVwNLr#O{KoCxJT=_}Yato?l zh3T^ldYa_DmB9pqT(E-Un43b!Z&|fO`RjSY*4G9VmnD}wma00Gx>zx3&-CLO^gNVy zX48|sh_I1rx%v|l4*++0oiTNKh53_61f+g+#jGWJx{q4Uw;pNr-Yk!mkV&;Gz*mBY zfz35?Fy9q_7FTEn;&))aHCS<(^mvdzt_+A#S1vo7>P}2StqWRW1XnoLT>}?! z)X98S{fS&NpL5f%bq2?Zn9b8gFlDCB2<7aG)mfu@-=FBpg_Yf zb_5z!^o{<=>SyqnDZK{ifn_VTWWWdjhrWMTjQGA1acT}QTg()rPWyJZgY6pnR2D;! zg%SRlKrpK#t`fKX{)wcz$5JWtb0+%o13=QAE3^qeRN-D3avS&qH=J|y-c2EdATDi& zdqokXG2sNu#I5lFC_%%p`f1p`>eDhdwu7K`dHZ&?E>w#6Tmol0i2A%H^3_- zftC(?H2|Nze0lFd-m)ba9PU}JIrrXf|D^hu%0+0Yw9Q6mGZg0Rw)|aoHn8_R87BMl zs-`ow=IY-{xc}Sk7DRxNn(LxzJSUvZF8$-?f&6Bsp~Iu&vk6Jl9}>*4o~w#YyL7j?fvHA{3N+}zeq-mVYa%J}8-wB1JYNFC zzP3oWWKksWwv~iuY#vc@KH-+(m|NiY{&1uA6_63#F0u4V?BZmxTfUT(m><(9(Ar*YVbKS_|%We4i-OPMLHOETU zhN5fw&-i(#OfI8pij*m6A1!{>WVH|?+I8NRkak~l11iS^kk^R09>uiW7RZr_V#s@6 zx4(?QO{*a9r>7WdcB7=9`ta5kE<8Dv^^rL<_>Y&A-!A*XQeu7EXA#Wrf*8Kq}pDo`$N6+go+jM zu1iY~F>Kd=Zn{&a=RK?CAcI((Gz?gMbIO_nJ~D!^Iu&K&-SL}O@9K64P;ig`Vw=$ezz zQf=Lbxt;M^BQv?co!<(B5n#td8H4N>P9+$Ysq;u7Ege*CJ0Pi!Bw8fjA{XV@h$Tu_?1PBoJ>q69JgST)~}l?v1Z!S zIPwQg_zJr%P(Oqvet#-(bnmiNV2!fi8g#{a5P>{q2J~j0BW>HMD+fUh4g-CvJEf;c zddZB)FUvr)K~WpubG#x8B#KwZl^wNo5BRe5>i0^oESp&CXlOORc)6J??m9dQd;Ra1 zlti2T^&-&2t&5vhP>0=Lt}}3abo5Y!`CEq@@~6=oS*CC0Yc1@MnH>jk*wTJ$KCt(d zBc78*D=%eq;6lpRC?8fwd$IIRZ(;#fAGeVvK3+@Kg_PXIvti~^b&L$hq3)E{*Y_kS zbDE)Crck_JKHv()xGUP;ZV&B?-VSfWNA-%}G?+Q@mPgdNSj$)K&e@MK?Z}taW%@%G zU(@WEtFe) zJ=2NN`_>_=BxGGN-AJUg_w(0G=4oqJw%c2oezIs787aT#6B3{iFK$!u*FFPyo~ll@ zKBW1q`hFA{adEX3SH$9^&_W`~5UNt{A$Wia>+p>2SZm;41ai4CECWYBAK_`*em0tD z*;X3zeEUBtNg~%3i{Q0b;QKh6oi-*H^8eDjr7CgrCLbprD%govv1Ph%9<{2wAcU@@A2FkEX?5 zkh@__qNHqw5W|pKuhsoXINzgl0n2iN}$*TdY||J9B(VP_tja z#J?iJoS?4YcVn~cph;@Nw&My(m@DO-DwKYMj<8o4i>c{%O0pBQ1676jK-cyfoH){k z9GLCASH*FhRz64Wh+%u|L4oG7IM|t@MM18igs+IU-XG~r9CwqexchC)`~0a>l7q$e z+M8!PW%Xj`v3l`!wvWC^TN`G7wxf~R@)ant349Su)Y>By2nSW)|0=|#{8+qe$w=S} zPD8$S?Gd^Atf=nmsqj2zeYVO^0UW9*Eyk#{A{Z_dbol_J{JK`xOAy(um0Nfp$OP04 z3A!zC3vWng?uI3CD$q5DEz*J0YE-fbMFPCc&sq1`Z<`z+3a^I6E6u^e%geIK?sG3n zVfd=lm>=LZTdLzCM2Qq9@Jt&WWQI==TIl=Q1C+-O#sxqbdT>21WHFOHl;RgEDuj&7 zm(Lx&-wz0d)DP*#;7UT!ONmzf>Gc~H5ECZ~+!y1uU6htiR<7^A?jN_Oy@~0kJ5gaS z_%Du%>rvRrE)R_y+jE6j213XAaJ6TRR_cFey9C4YV;%SL=o{#SDB&I$8JV3~0+uFM zU4t%C2QzV-L8AB)2#fSupN&Fw{V2GaG<9inxJo6SR@b$vy+gvo7U7B6S-XDF^xkvD z@+YT~b6VfLaU#qm^m*K=1I}n18jJn3>~wjR~g1&-9Jm(YqXJAk0ye zSDv`?-SNkIwOF~sGynPBDjS#Pj@@oZ&Y7@B6XTtkfMaH<)IDzt{c? zx}6rG)0bS$VUE3f=wY4A0xDmgpq9?{=5fx6>{Q|DQGSa&W6KT${3x=AEM_Q;%_NJO z(rZ{F-@VTx*5yN5qZot#mSxL~8?B_Pv|VogD}mXI0f##HvB@-dQ}R$9*un_?SFR%a z(k7E?S@<43p}bA6=yim?{M}!Mmw6R3Jno zZy!7lRY!sPay0Ik+&WTP9zCcN!K4%;b+?J5@C&&>r)Igq>hVhak< z{#IPr^gw$&F(0B*<(~NrXq%=g!nc6E3BUN`XH&DdDkpbM`(n#AXsMVrRR4(AD!tGJ z-w%WCBdyT2+PpWO2T4XC!rS+>kR7rgF#L^M>#v=c8Gcz7Ry_z`y@kxchZ>{tVz7s}N+h9zUxQ>W79mS^*-?_W)}@ejtH3e1JgnjxKa)y#`)(V< zH!#AOfY)8LfuJ16z_ZJk)W>ir8e=hcRlqru+n%jvP@YvTZK9D4R39i++SfMZCmKr| z+ER>4xV+Ezyi7cF`oCo6@c6kmzRv@Ff>UM`DHV^qdGBLW7iVE>MR=aG35XA-5Eir7 zx4S;CvrzZ4c~}b zeu!Of0x^}vs!LxWfuKOu^V&j4`kmKpeFjKC2i~vA^D`sH;HY6}eE#aRC#r4{Vze1^ zshB7e?*w0jgacj<|G}vLxQ4TX3Fi>~^I=vLR`7TW=S>H;4Xzr58=xF-OkY=_DjvfR z&lm*`&KT8F54UftF=JToEPR|7P9H70(}y8jJ-BTsL)bdVLn!e*rKq^3Ja4 z`~`^46w>_|gUQ1WxJt;MFU1WR{}*QXukb;zOaE)!4w^2lSXUiJ>%W*-lV_4D(ek!n zGorUPmGj4^ByzIK5>2`MO%Hu!!RM4Xz?Y2r$%0F^hj0g;+PxmL=5IIlSnjGxY12n4 zwA13~0dPt4VfMM@s}!#IFwq?Whx9kZ@cG5Phz8aGUI%*+UqPOWvfOJG|MRKmy=8le zAG#AmwnrsqD0k0ppfvqub6vr`K2r|6;{2SBtC=#*zUmuOa{76J%;eXhXg_2QD@D%10 z($Q|`jzw#c$yJJWi3R)DUB=0I{B3!U6v(70a zSQZ*K6~-L>FuX(cU6>1DkW@wg*r7J~nPFA20eUa*{%9$0v!i|n=Som96N)0*vp>r= zfNd1E{Q3$2>uIyMJb|b5?sU~$gC47M5sLOW^m*&g%H2{&Q^SX654X)y zACLY1rg_jCr0pIap6{5X-``;>0&-y{Am_gsKOQJlh4$A+ zW&v#cHh5JuzP=G@A_s0Wh#0!#Dp65bI*hTXy^*VIDU=fNf}}qXB^7m072KBZoQ+z)mWOw6gX^i^kG-a z1TWA?3@}esgZZ#y}*%<-5(q%OpFi|2Uie@Uf#TST$UJXq%vIep1%G#gXgs3)*sd3Ou!= z2a+b`9`goNoKGdM{Iec0BQ_kUf)EHI?kmkqhn_8m3 ziHui}`NV^`INl7@e!}S7bIk`5r4puQ?RaQie4+_jD6p6ECF@@W?w67n>U+Ett#mZ9 zZ1+LQ2Ep>U_x0_Tub1Zt#$S{$0t=!L6m=#Acdi0!$qXibx($A%b@-oD$?*8SXv5)A z#>o|1U;ZCf?wZ@rex)SbaZM zuXAj7=`}`6=L-&Fq*~Y(1Z>O2h%3gtv9r;KInf@lveh`;nOi~ zdTz>*nhOi87FQhC2l@OR^h`L3hA=PNor!i^;$6@Jl=9)Rm?>AhU3cxHKVVVdG$-9j zzgnG(U%3mIt~^O1j#DkZk19Ik?292?Ie)|qUL@fdF2&o^hEq z&c%nn{{aH)8G|!~HuTGb(+57x#5-_%XajDaYy_Pt6|QBv-1O^xIIn+N#>Owiw(#NG z!4slE*Fy?7?vz`CPsqV>NVKd$JrM_qPEx3zPpoL^*7$aewC%T!2p$GuBx%32h%z!y zR-s}K)Xyu-A!{`O)GvILr*hfd@*hidr3^hb=uAT;vk<~JC^=-9%|%lFW*Rs16>f4>7)a;|3Xf5xnY3-f&XFief-sq` zMF{!||I-*<8#&#cf!vQEx>H<;lUgE0>7&-LK*!+N0s5@9Jcw?|f z-}vov3X&>#;~uEJ6gW3J)W(Dz{lpD<^zq7Df)dtlg01d1CyL z5nls_1TX$sP;G#gQT{aWpobIhUtTcn115i=9MdRqm6*$I2p8du3Ay3*kQ%nQG1wnr zzP6AR^N^#`dB{3U3*vNmF6wh1ctPkmBLQp)(xmUn@Qq5(fRFHm2md~-q32BpgwPI| z8+hX1pgrdkW#qr$HaVk9@TUiG`wva{c3HDsl|3fV!I@NKM>2As8Bp+ZC3a?0)Mp-s zOb(BP4viU>Vg2dAPpS}hU5gne<4}&R!1dBtNH@=t+vcu?<_*#?5!SD;y#j0A6iuxA zkiUI(7hb;2d2CSXMxD!dm{7yodAKj{N+m0u5ISlIW^G$fwA zNfeX3aqU_-6UKbW+{QriX?#(9MquuQX7PCkg8TH*{F7lGBxw#&pA$j*qkQ*yWATT} zBE{x9ga z*~<~E)TKe*_H%2Lmfi)em8Opt4V%gSPrdUPKKDoXP#y~hYQmvL67DjqV}M)0f{zQn z`jTn}*DwbwWK8xvH$!`tKtyod9|f>_@;F-_F_saP3z3C~nCZ>hIIhSV8Wfa(y{_^Q zDkP3|(`1Q;}hqvOA zPsJ;9LGPYrk4}v`g1_FL9>F#X&kOBe@jef*YVJ@2)+icJC+xPeP_S$Q&=lPuTd7txspZAH+fVl-$CUR6RFWvsJ+(k2 z+)qU)1JguTF?MVH8iBVFvj#N^0XP)a&En7Q_dr|+zGSZ|S4EE{(b8;Q$7sDz-Ok>o z_ONVUK18cD{r(IVU97?K8;44vKZebwHe$d$8HE~Xd!aZDn6O>j_iR>&UcuJ*k7*J7 zTgM&&v##2Wn^ti!swrmlFLcP)EXzm&w?Y|H^&BW& zpX@8me#z=}6kUA3Xdr@qj8Tq0R*x|I5MEFKB*%@aCE_?S-evZX zuD?y#x@`9?-0bc@{@whqudfDnGh0rki_!-UN*>kXPE=GJ2u?aP_~xmcgJsMOo_19~ zI9=w_{T%(b6}NMImk%J0lk@wVh18FtKK0S>%d&nt98-#NRkS;5e6>#P~cH(?l)v3#7Ctr*4TBF~dYC2YS z^ZNl)l&k-Z({vv$MH{yxb0mZ}`lkrR`lQR9ED!AY565h-Pi|Xw828XDrPt#e>a^P~r_H`KpRnB>HAiG%_M2!WU6H;qDv>M;KkPnn{pU#9t3-aCtMjkI6q^nkrteAl zcHj?ioNhiUiZg}GS2JeI4epE#-H2B^8H#B(mf9|v7GvX34<^j;ur%1YQt9KXQH%v07XnVgO9+d?&+1#y^4&e2_hRz=jEpE%-L3ab}!u zOD04jVn95H8HEJO3y#F@9_Biq-fhAzt{MjawwYbDV{(qX@qg#Ofu)p@EWkm7lMVh` zh!(TuT%#|XM0NhZBsK{B9H>$d*?q&fZVMgqiE!5hq8ZoMa0PgnCnWg>w#sQs{-zkE z17Ih8qtN!A-*QYjyph$t6ri3=QjD4x$jzMb zVuXGBd9r(1?A_h5i|D<9W2H{k#rzihsH*}j7MCl)<5*&flK_SG} zSbGP3s!)z{qmH0xlNdYxmsrdfdd^f+A+via-)=)E(aAzUuqnwGaZ z!Ham$VonH=-;nuql60^AvIZqp?RO7Cc{j>Qec#>bfHS!v>4>9i#}5>?0()&m za^8S@Ef;VTO*U{5D=X{c;9CwsH3~TGWQL$2YP(sdz^-+}l! zO8IXcFQ_Ab^YgURna$Qaip}YUK*b-6(%8B?l z)h}5Br~YtA4Yrb|QbJ5hceo-ex?%zdre}PeYg&mo!9{w~c#8f8B!2$@c^(M0HRDO> ziwBPUJKb4%;NFw)w4kSQ*eulRcgEgD@BKESKC2Y6z7!l>kKJfZevD*oZNBBxF|XGP zJN0islNwF8xpvAf^dpYx#?d8~+%~D%Xt%Y?;VY1`f5c@J_NHum40FQr37NA4@L~*j z07zkRCR%qk4O~SL*6iy7?9$Zj@fXFh9EMSVJ^6kRneonyhv`>X#(I1QYL<6Tg~A9N z<|*xB3#(j)XEZ!Se*}GFcY8uU`SeU1&@z?~lf2+T6rZ(>*!%DJIJzHpQqM?F1Qk)v z6a|C-1|H*pz;I0eN6vkP;E5gnneyCd3)_mv+CxcnA#|&ba6|T5d2Js{6iVj3W^Qfi z*uX*7wuj<-64{@(3Bw3o>oj(=(0kk6jU~4G8ZLyPzy<=q>D%8qg)OF;~&YPO?N%*IwIavrLYhE!R3S z^&7}Ht3ShO?0G-*yEB718u^$ti?k_d2;q{AjfrT64~u(#Vx?^t`j>ymUdmW9Q;K*o zfSI)j|CvPRWv9edDY(dTFlhO<$0^JB_)%wsvuq^B@N>uMH|d<-GUgx8uT2abN)~W^ z#A*2xCIHS=-4*Rn@|*Jx1`%nXUONY`U0+&3sOvpFJ7>#b;)!_YBaeT}_UjZbUDz${ zBD`d=X(s^r^>1(+tv`<{n*zSeB{NvZ+B$y(;1-l<)zk;&P0HCVW{AgkPmze*0ce;pIV*9~(nyra;> zkLW7iXZ^*3#=!w1bSwEee-dWyFa3-A8OusVL0_sI6f`#{^w@3Pz*<>0<_`zo~Idgmk=Q| z>_lN3)^r7y=&tx_bSP&&;yT(@XB=I^kfUi} z3iZljwL&K9$`JL|Yw{BZsv0KhVmNhAVF*8Z4d-0$5`WTtY^= zMwhe4*uHDHHPG)I8(8LOt_^Q0b5hE3rV$Lsflm{_7t9pHS{ydR!C-Bl%BW^{2e#z# z0EY&MDPwJABXn(j&DqcK_ySltTU6?0m$7lt{$h@zaGT}4O2lOtA&Xbo0dKB$=M>6l0t21}T{#?t`+1}hb zr&bSlxR+b&+PBIi=ts_ph7n2{Nv_L-FjSXCW?3ZI0=RO==w4?& zvy^FjG9xdTs{Bwi1GuhW14ZUabGmRXw)#AV)@L%p_ zg0(yxHv{?P>7I^o}Q7=lg^;&sD>I*|%171`i2ueG&|wfKK)7o! zTIxTGC}Uf#Av+Z<&DegvMh?%u02vR@NaHOYk8?87Ho*l^b!MmY@fVm~{L8lIkjktxZ*r+%mjuT<88mCtp=t(M&qo$x#3L zagPP-%(;u%tV>5E!?FGqpKGZ+7S1_&L-?wd>{0OV$s|y ze~l*+N+LJVVZYEIwB6_Jkn*v&4NC^@z+Tjyz*1!b`yfwjQZqVbarBB))2{w*o! z7+TAF6i4HQM&+HxC;{=fJr9DmF(SBR{~SWkC;6DPo6qw8GV7pwS%gjm2{&syQ46fN zbYT6Q3U$7Ze!`CfHmpke8jWHPZmrPJn)o0#Q2JS}-^Kvpxt4qj>oc zd7~{}_zAcR{LE@bUDT0ekilX$Qj`jsXSp{?v@UAS$kG;g>*H0+hOF6b9=?7*fGH@?TXBt#zNI-hcp;Pxq!oQ6Xdw@<`Z-L`i$p(r{Jkwlz z1|O(C)d*INybZ&4$TfcJ|8rBCHM#Z{F1fm{V&xETA*9sYM;|uAE75ym*{w^g*wHUf>}l z`g`7X>>b-|U=;jLv)d&);mBee(1JECgZ2bsiF9R-VHpg!DG2ER0|>J_YXEs)R%nAe z4(v!QsI43jOLc5}-+bh+%u6I|6p~ZN+ZEkJzK=Ka_N+< z62}hAc49?%&~M$LyzZMzMxv*i7V}D3GE5EV3Oe4Ng=LtLu@v`K`H*xjfzDCm5(W{{k z6s{80{sf+Jf3)BE=$XV8DZ>8T>{vQ6h9H9TLq}Vtg^vS^ta0#--^#alKuJv08nycO zY|=t=K%Eml z3CA|ungl9yA*U`Slbg`#@-h^0aQWfk#}nW0QY0WdN^J4FvE`v>$GEGBpXj$x74%d~ zlL+4twt>!T7+y(U^<~vATTj>Ax0bFW6K*5}bV6SvT707lR0VNHtR{9gd{dYm z7zk}8VqtXEK8+f1#VrcGtXq+A!r`Eg-Qg&{*M+e!WeeVN+4;X`|DB@bpL5;$jbwdG zu~PV5=J3AL=>BBd?n)>73T-Hg&)2FBPw(Vw@SQ3zQ?^kYw57XM@Hf3Khs7^%+(;*v zsy`wJVIK~zoNCw`?|4bsd4K(nFnGD)u=HKbwG)wo8^-7UwsQh~jtde;Fh6q5NSK@~ zFMszt$3q!Srv|^ay6mK1m}gwKjIF)Edyu$Dpnnl;T5-VqK&~))1MiAOA8(UlaRG*r z)(pHlYiWpur%M%GNz6vljbXS8OKYMiR|MW?OPp)nOzjXftpSlgg~|+B05P9%AvdPN z!T7(Us-PZ=992q*FOWzGLc^KRHI&AFv#tziD%27PKu5xPgfEzy*x5{y@+sn9iM9vV zGlm*jLLhE7umk&}=tDPHFcd>LV)CeZwr=d|EQ&q?cfwPXj^1{Q4?#Z<(gtYM0v3_Q%HJp&K;|8L>y+j3WuEtZ^UHlhFvXe= z`}#YMQKCzs)_0vRKNObT|2%r6IGf^FQr}waym{*UFt|^rup^Ir;HcNgNxFs($+|}# zykPlkBm(#3!fX^k_P$;{-Eg38%Ncd3l5Crz5d#bPcXMfH$;al)cO=jY{~@kyOeJpe zL3jUh^C8>&s986Uv4YTq{ZK4@(SZru8VH<^=l-TYRpc67a_K6;^rG#FS9mkxN+hKTjY_yG~NiYaiRR}v0ocC9k zJ#eP2k)nR-vQ6)oU;Daqh^?UUBPDd>tm9}`*v7U`HK>I4A*ntWR6-Y87JeBAUIGg% z69iDA9orC5;H58rYBM@q2>$Qpz13 z3cnE4p^scdwrv_;X(%3+WwzORfAj96k=CjMKke7q<(VDM<& zqFB@F6zGm>?iRaURb&Y9I*6S>9$Xealp=LTF-J)A@)F!sWM4V&GWP@G_4Mp5ULdq= z(D4~(i9``FmFV_95*{oIV}(38aGNRtDGVC*)$~^w_WI0)qpoA(u1PM$OPFMMP}S{U z->E)zGTvrE8~62dS|79ItyG21Sf|hlpQX$5{EWqdo7a}-EPg(vnmJ0uoly5;e}{pp za}DQeJFe)w3VE{!iUnb)alcTSHbqI;?jTZB^Ip}gM(gz2BNsQ{JHKL(<>3%1tZi`q z&dQ(bwyaY()a)q1PRL}nwe#I~{vLrW6xUgw>LYS5%=>z=9Z6WK0K<5y)p_Tc^QnX? zCs=^?z9hq%QNlQof3xq1%5Givnir&%gOu(2fsj4Ykx|90U{jr7Uk7xWm0)8W(BpU{ z^AnM@?R0R%3Wxu(JRf5T~ld8VuB=F;LXtjzWx8>|IY(QZ^OOn>`rYej9$StBw31W6`5HnNN1*(B-~>-K-gYt*#zqNPQhY zYf`~6;Yp*;-MFQqOX?_+>NJVB`t_yG&b5Q>iDo}_)*4=n!fd`OYtNPI-ty03h=%?V z%doY9dP^WLmnZ{$In_*EC7YzQRMpyT;{)M_}ek2#Iu6= z!TV&F0?9YIOX+9lR~ zj(jRO>YFM{g7@)bA_^R{6|MVRG~Qp;vChN2q~+J0(doKrmt}KuJ#*bS0YWABW$Op2J zKb$3Q|MBqG+J0JEoe=7CGb__hmG-NUzHk< zbbt?OEg=>)n!ZyKd0xBfzIiuYM<3PA))ySFNUS^)8FAEE5#^Fz{~3tM<-@ZILszXU z?Ip#zF=@2Dh);%Q-12z66SwI~Xj@Koab`iTMDrU5*0O$7z_T&g5g$TK&ZZ`6LgkMt zaD89B*cG%R@Gqx!hwOndX2U3<2CSs*1a32S{4mVV7|Y9U(A!>v4eVCLrV$1p|xd7x*9G=eeqkdeeZ9LrKMX7uv3a%F-VVwSyAX7rC~Phu9#{3(RO=a;ppuj z2JbAeHwvtr0m)?uHKjEDSu;+Q|xVr2t8CREb)9|SV>b>?h~q>z26=|Q-Yg2 zoZpd*X+NBaN3#@;fxIFiCy6l%&wn9$toF5CL{^HH0H1Iw@nWy=p z=ujBrc|48r3x#;lDf5Y$WtJrut`F4=I)LA8aW}=0; zFgSUCw3vct!ir{`K$&yzTN729=ArM$B7^ zv;*$)Gv0^DK@*Z^T;J_1#1LfTy2}TjnKo^I=Tzx34Dq@zL+xp^*M4D?(Afo^Y|jwX zmkQ+Nv7bJ6f@LEy$PJ++(^!fJBrHy3Egp4{cc~7%(-MOm?LOMUdnlCy@a~pWZR3;k z?u7{j$6oM#qkHa^W6t9m?@qlziivR0$>H@x(59)x62|Jnh^TP(p?+)wr9^Jjq8SbR=&hE?oYUv<4Mx9Lq0#2%x$&2 zmE$iCY@`ooJes$-g~-pB=-H}PV?BCPYlLSMdk-m$tYlP&-STO8a}N3jI7xq?c$*f) z(woKj+I3bmWNou!zT_D+zeBbWAPpXtqhOQb6(R5#!|lEOv%(cI3FMjOt8{KKtipKA z9LxAic9RYX^V#Ph$|{Ro;kOz&Rh51S1Tlfl*h-pni0f|=%TRD}^-@Qu%+o{QF*ABu z-mi4RXx9So>ekP3uJ(Hxy#(!ip9}eBbPO#~=TJQQ%U$j0RPRUNBCti>s z9L~~N`Mv3KOC(Cn8!Qc$cn4a?iyahas_Dp{&*{pq-Tdc`o}-H+r{-`D_HyVTRRUFC^nsz><#VwtPX`r&Gq+{k z=zDa|SxavLQ6IlfWxbS2Cv3xd&Xqchac(9@%2kv{htGkZha*Z>XiAKVu;{JYg}N(< z@`}x@xyKI3MOH@h>V*iQq6T|P)_##^pJr;&*8r02ayHZCkv(jb8GE&l`H2?TJ{*J8 z@8*pu8N5Zr7-r16&*$GTP_en$@puYWW@xSNuuELUQ@+#yPI#OTi?MT_?+SwXJCa9z za(L(wn0O~Bm9Md_f0qxa>MMQIjcJ&3*Z6T~MX$D%;~v?P{}a|zr4a;~!k`%vVx*w* zqU-d|8Beq=6d#K=D9ptKZ<|5J2p+%osPTdZcnD(wj%GM~{j$%bI%^TGH{Kvfh`2i zQr;3T3G`up{*~6WNx+fE2)gsc7l*y~^G=fniNRRZS|hy4^vu?p1<%i}YFHza3T+K` zMMOP1HcvO^5$uTt@<7Uvo8>F0u(xUn&X4K z`q&JyyPSvyQy}ZQ0p$K%?OBpHKt3fYbHZSXQB1htk=Ze$YII*Z>18|ia^sp1(%gZa zf0s(w+KIM$t&_=fMT9E`p(3ogN=%e$T$6kXpU4b0MIH6sMWs1X-%wN za@~mp(oIcs5eCjTQzgJI^t?(XdW+DMDO>$pNFA2~fGaTcJ=UbX)*MEEch>fuB2IZx zADo-{RW1JRU)(-{m9!hRgPei>BWb|xU(XjYi^qFMZ&+>wQzoaqS!}=9M5&M&tkd!M zEv@qSa22Z#Ej11dmGviL@zcR!z&Cl?wC!DFn^x~OJ4)fgqaSxC9%QVe~(15;ehnv zT!7R5oc~*asA)q0^c6gUoR>J*+e_mBSCH(EU^vs3z6g)4VGu9a|Gfa?CuB2J$ht5< zxpJmniL~KBpgi$UXBnSXE-JZ(H^wlwjKR$m*z0=Qm^8;uXMC!WoxP#6Sm$}xd^{gd zM6d_&s7>6S@|n{@*?0Xt31w=Lg)cuIczCa5DuqZTB4`%Z2N-)*C&~qLe0JMJ7dqO2 zOZyju3Y?izh0#8MwSblW-z3=5%KG?U0wup>Y2O)?^FEh-2EJ=U?i<#b%)T*t;iGFJ z!6OgR1Sl^Nnzk;C-YW6kR2>PhQ7YtU`KAJ!LKF6@8`DYpBwEI2+Xs);yxqGQatQx8sMDq%qx z9z{4vF+1(M`BaL%33F0q$Kx_1`Y}q5xNX${|6uk=!fhUzz40FGk@uVY$RNfCOz+;- z%%$SM32RSexA&H@)ze%`2Nq>2{YQ1NoP<4!Cc4)cZXZD+nK749)G1n8wzq2>Vl=v$ zF__jdlT{5)8(gN6@K@K0oWLPbh-WOGZxO>dGmfTZ+3SnRdW)NH;hVYWY}+QiOch^h z)bQkFq7$^dizu12${eamkU4FqLfYV1m8!2Pb~MiMpIq5MlUEXJz@mXiFPz!5%E6Nd zfq`EvkX(a#B$^jwayJ8kevwYtVB@h(8)omwYXy`(6a6aoI0i6p=*~dQnYb?Bh}zFu zEk@|)r>Lo%IeS?J$2l;>{|-zRF2L`e`(S9RwGQ+K*PmlvGXN{pEvIk#9K5_3)y6DG zdXldcb{}!fDC&cJpf=I=u11b}y14kE2F|n6Fbdp0WAN)R-l9!mypI(~-DQ}C{J}Yd zKD){CVu$>_x8S%J6$$1@cpG-vim<)G@9Awu+r^moLZU;_UJb4m$m zw$U?;W?LsScE?_neAP}^TQsT=x2OsMcL7VD%Hy52U=viBs4$)h_~(|lF?XjBLEtVH zW!?A{{MOIO&yo2Cj#GC5+UFw2!l3tHPb_MTswBAS?f0#j(A+)-hjz+m05mAVt^F$8 z>}(8zrIzgN^r;hKM~yKVCYMz7%s*|BcPro_3lck(I2^NsWdHNk2$^zPQ@Okt$=rnK zt!c>h%C5LSx-NRJsaSv_Tz_)cujdg~^QM10!g`STP1F<5$f!My_;JnqD?MW`@`>SI zrmItwOUF5s%Q!~8`8?09ALMcqFg`=m6773#o)C(C+6*VHTH_Fg@azLf0Qerv;QLuxk%w2lNk zSJm_mPyC`h%58hCIX0_kg|+?M^>6bAv~8yPvpWv@Jk)>bCDqJu+pwXq;i9eAUO>rG zTh)nHa&>5&Zuv1hBf60(Z!?2fwDnd*-#VSewH0WY*pq!q-27V-@6vk}#nfwBLdS;% z=4AJi$!7OV%W^x~VIAiVclZ}oGs4&Q4Xb{n=>ma@;P=lDb4|2;{=7PTo0X!I4ZE2w zw^igR)aBA2|HHI$rLrw_s*aI5b_QYo=IICW-bq`O_~rF!)?u*EyY+mCv_+J2L}=D; z-MUJbkFU=#7g@Tt=Cg)le@H@+uP&PeI9^+qoE+A*)SF9!eW;1<@s~Ys{^y0*^?M84 z3mUAy+|1ufT2%3>J>|eAb+jfXRAuN2g2%*g?S=F7{{EvYLJ+PM#O9Ho2BR)-Y)~iOl3lDuD(2m2I*1SqjI8i^Ex}saE;LSBHGvCGPC6 zUmj)<&WD;`r5dYA-d!d$y+nt29ZnbF=9r}MGkULa4x+}YGS_c1;&mc2s_%u<^95=> z7b%6mXM81~auVF&B9PZ!@M=`ryZM{lXjX-s&2(?zOW< zY&y(D0Wa%rEpVILJ$f|6lrT6Su|M~eF>z10ZWf>SWKp+FuACog;Gb~&w}1ca`KYtZ ztMzT)=(Yy0KB=b6NDYJm4K!Od)yF$-qW)-eC3QSA%i__+2$8NQY`1*aXV5dO3}^{J zJ?s6lKpe>UmOv{XMQ+#0%9@nu?aGG_iKPpL{Idr-kRi%@z-Qw+Mf-NnX;_@0We}zi z=CZN95@sZ9d+<-Dgm{LOf4?z3_^^vV#Jsvl`w8nfd|vQv^}&KAqc3r{FEzQ$-J~~% z(z_2Za~JL+&qX|ef5JWS(q6syHBU$~_gQX1jDqgX=x1>>Q8|^~>nAT((l=v{QyoZH zQ3Gj#f@hCOSU%Okr~R}Ku#zp7PB2rpt?ro3!~LZ!6G(i13)M2OiD+)vCY}&s3Fn?c+hn#YhU$76nTGrj0TJW0qPG4Y1&p`ErlM~=!Ss^Cb zyQMse{2G3+(8Bt5nM%eLM8CidS+&RRM(P zG8400*Bce*DRs6vWGf(Z!E@bR6loJb-7iyazEzQ$_t}<5aakY6w7eeP7e@NwdA*k9 zBiDL>!15>$U-y5(4iLt-9snOZRO1R$qmF=298aDXzV5>5;5Jlp=##G2JEA6^ZruJO z+ig`(!PSFBLo*_Ud)j&IXty)f6>mDWny4qTS*B@=htHZMDy%;nKbR4bYfz=?<8SG` zjAyn1^W0q$yB&W@55&JQ5S>=qD>&LbByo9lIAtZn+084^(A~Vo}o4yR74dms47wWok9pI>C=R?$9wgnEhB>Yw!$Z>DNvpgFIOGQD_0hws$mLf+Az zHw>K1^tmE&^2r&^Rg5g@5F^;^?e~wKkFl1Mf4NAv_8RsvFUp>HdH+d8!yTnflx{7edpgUVfRr3HJ*sorXL3!jbI^%z?;^4z6jI%jiDK#WI=XH<^gHB zXxE%fDNwNSWXe{V0ln25(zAw19pAbEh}>rN?zI#;iO3yjCuAIh;u=c1+jqQ*mJoD; zb`jr6H7>cR*VV)|G;{PlC|_+P)tr_X!BGoJ(*c>?uS+ zV)}06z$&H#X4E}wCZ-Z{o$p%|j!$|n@n$xBhb0~hw%(7sKZ`dHFp=l=gk%(HW(qRR z>#Y@zDs*>UZQP~4(VBk>OiP_bjY$}t<932o&%J7*tamTI>JTqVTd4*d5C=#@lHqPZ1^BS@+I0@ZL>`}uFPMJ zRFHKAadB^0nX|Yz24*$PKydx|iF`Q!5PWuWs|z7HHk`f5m2g&oGznF>4Xf`L?V zalBB|VPvM*j#0=3)Nw^nT&~^*d{%r2@h}6qHGtztti;Q!2Qd!9KN@z4Uw-tmN{xJN z+<8U8^;ap>;!Mk%h>ItqjHDVAmD1i@kO)+z*R}8Ad_PZr27Bhx3B1rx*tISuNj`6o z&|}D7VaWeA1^lx*hRk?yh3)`T6BfyfMw~fb#VEJ0DI3&vO*s;XE($(dE~DNpl%d{m z@R1*CIVbt)!N*;M*^;mGzQ^|j4gsHV9pN`54uN9jH56{(bxeQdP*nXG*e)1(*bg;+ zFr>C}hhokAlQ3fYXb%<7w48BP7)RV^7gCZCN31ArD6^3onyq#e&NVC8NTvR(2A`vkbNjh_YLzrE9j$l`D*oFRx);pX4V-IR z#i&R(tvdn*lhc?tJ*-W1;#>|b4SVGSz1+~;?CF3ubYf~1b6e^HjOoX2Tc^FvPh<(dTesG&3Mf@PhOC^TxZCv)7Kz@=?6~=nvBUBl zIu)(fL;L`HiLP$?c^^Y~t8c|*=#%(wX(AF?_doEnmItWi_#~M0r-9Np#nlwrM*mMA0a6S-((nJsq^lc*&dxONwGKAq4AqVzn^Gb=YDd%TFc5wW=@gJz347l` zxGEoGwJzWAxP&!L40N5y7>wY0pzT4FBU+@|BZQZH)4yG+JdQ#(3|~Zl_)8v2_Kp*J znJ!lLxKkq|9 zs9hwR(nZn%Mee*-r=!ml2`+Stp3;Izs_BQj* zg}PCK(ylN=YQ1>w4u~YdTknk#KO^o@vYw{6h*BLqZCO%Vdhdt8iXGcJYF4hlnm%VF z=s)Ge*D)F+`wiiTpcrzKmYx#lNltEo3;cl1t)+FB-^g)+TAa*E=XjBInzD{f$_feU zkNxTp$gW~JKV6~in^%@1HWH-u98zQ=}$xo z@ib$N;iqwIQK_>H(YBqbqbAM@$2@Ckr8?9Vggm(wSG2e8{2Zy)ImdoxH~6mg_mn4{ zQEejN#pMlCU(o|3@%%pKH`;$1uI*?MTmt_Bk?)rX&b8|STG~%G{%y!-lLTMpU;z7+ zyw`H)^`FwW_(Q41Dq;`4*xAWH-0qR!X!Thi->l=F6}S|gh|!sEO-)#Os+GpJPv3Q$ zYPQdBizk{-8qiZ8IVvrq>&s(X-zy+{;8~LhZjMoSNg}${q9KyRfZ{vhi18u~ja5Tm z6Q6&` zM;fqIyvD^5063#Z90gDkh^NQK0(+$&Qv+=~2p`#}1AAn>t%5t#YK}~LZeqegjbvC!n~0|`Arlx*_2dzW^ZWqFw! z^;73Av>T^~pU}0VVSZMpgCoeqP=%ITJGr6ZwHU(DwPc<5quuc|?JIg!oCj;x3qD}4L^xE? z0^vFK2AtxdAm;55BSN`#$ZcX4L=-bsBvWs|l6RA*iM;&0rz zE2}S|^PL-#`}`S4Cs8g0A+JDAaf3m-`Goq{O5^{17h_V-oH_!q4gR?-z zUl(4koo?%D)0+F}DtGVe--6dV¿Av*V0wO@AD+=AM;qG*x*TgsGTHMO%vcZ{h$ z-f8|p7xGy#1TW@SFa!BU?)TCLY1A>MV9T5RC0#I%>*D=~D2exwdzgyb-gGQ6`2sdJ z6vvT~^e}V(_PclZ{NTA>bnfdY;r+Fop|StOjgo@hvfMVgSDFM#4c&{ha<~8f&B1ej zYCwCYjht{&HJ<|~IxB8^4W3zhQNwqrSd(77%DFYLiQFOfo#Cx(QDSkD%P|DILYtzH zqwWrrt3TY{yN#Y-D3QkxR(+sm#Ul}WPoY|Tm*FeRA5?T|SixNVh;kRzsjlG$BkNt_ zITxE-9>{NuE!(!GXiJXp9CpE!F<9DZn#A9u-jDWUyx{1vu#XUj(^> zU#o&YGBOaIDJ@(yQTvCaQ(zBp$(-TMQl`9kWySMF1Xtn>VQ8g~#1->Yy9R)>64YP= zz)o_O6hEu#=3m%%@D`sx`3M*=_g4g`geJ0%LvO05)iqZUqEvT9>;iNJT6$T5vr#QB zXE=9Y4`{Afn%XGc{nZ1r-L`#(IG6mwm}%W&4@eG`l*WqSc#_oqe+xy$GH}nuCcOXE zL03*(ewsoWWh#w%g^V(@?pxK~-X|X-?-0Mga?@+qp@wZcfq>ywVecP!%f!}4NY?&Y zZa=|g7E+*qd)vAIN@o1}UZ7pX%y~ZgF;3;;f#Y{udg!%<9dQT^v6cmnl8Ha0vrvZn z>!eReqyHb>T1MOF)gx&MYzZ~DcL^`^ji}&$f<}PM2Y=CYvJjeMoZfrZ0$0g%V{Y+O z)&7f;lg(x+i} z1e+Y(okwtGoX8?1!oXU>Vw%Mv4uDUOuYRe(l3+$b;+SL)beF8TmOnRPVe&{#C4p5< zOIip*J7B*4!${P^mnJ^0cSS{g|5t8yQr!`mMJTID2+gL6R*21OeGPlQhqN2g9j*6? zKFNxH(`w;R$GM8$C3hs;WB%L5Uun|4OSw84NU}p4eY(-5094ZBb*dpljN{$gb{9MV zj{LDBFMr10iDH^tL9=Z+y7SgxDfZd3*gnMCnH1vF4DUA*?|(I0WXul!1s}hDTqdW4 zK0Ve(3yKCRX`((2Ih>-0GF1&ysn>#42svCW3(xm8Pbj8>C9R*+#&r(tLfa=EAme^c(#B;umWbb(GUhsLv2 z((sQN@#$9+-4iCPFWeMo>GAZX7ua;4Vfo;296WZq4jkZCl8jf`5_fa_KI4w&2#+sP z8^O0%XR_j!QY|N^3DjT#oWkJ1G=`*#-X&%LB#wQeDik%R0fmF=R~J9nZSVQ3svXD( zJ5<^>%IX3Ari6QG!-jGXRnw8CL4)Y|GMoM*1Cf8u|)3bn^cf z6kk1|EAzzeLr2v4ZjbprbfhQ+P5Fa9>@(lC_uP*C3+6p^ZcWqv{>*+{6S3<;g*aLT zS)jX!O0);o9{DcTfKoeAUhIJM;S1|OaMtUZ^B7Kg`!YxU-MIU=23Tjc=?5aw+n)5R zU%@BHC|fNL)&~)L6Xj8K~UmIP98vB)-*Yr$O6Uu zMeYZ|o7N77y2?&i?*iz}h+-~S9mQ=nuCtDpdQJ;H)ohi>{P`z4E-6uhecL*~ouraM zOI&(lNiBb_ipKkp{hNLwIXcn7E{Hd=j`jc@bZyStnIiR)vrZ}csY-!sCZQ9ThAWk# z`LJSHBiZaAB1YygbPNzUbK^uAd1Op1sX-DHp1jb9b@0uXDSMIbt8_utQo?h{#-g$= zVT49D)0mdc8b$8pjDPS#o8DEoj^Yera0(0e972$90i+IzQ>z!a$FcOrZ0Ii(9Ma-( zrMuSwipIuxF|qWTCp^V!$zGHHMlm02M+t1^AJT3NA((A^&<d$>ecaJ~aTShN=LwrgUn34bo;|IeOj^_f7HxjWYd+`dMCz)m<@KR5^~2o8H9h zkaoj$oCb00v7oz6G|K&#kAMHJqeXNfrauy@0CFwR&(jTb;Jdj|ePZskj8hK=6F0UbPHGKs+cxwEiA zU>UL?7th<$H_3mAw27{vEM+b4N^W;y&~3cZ0Er!d8Xx+LfC-jOmFa zMQ)xjlPw~Qc@w%rZbFQ|Aj=<5P1q8OtRSo=2hBcOpFwV|_18AYxX&g3lH*B(kuZny zeh=mXTQyJo(_IGc#KLWHz?Wb1X76S$`+&7!OzdBoY8x!mYHY#(^7t^xS@O6`M63Vd z?008?$2X{lerdN1XK@N5_Z!8#>%3{qV@vI<^^@}Ru(w`DHJ$=go^$PEEcWrA>OFbs zdxCdBC+=CaQ1B;hXvtdMRP5Qn)yheRp0PL1OFuKzu&G1A_51mW(T-lnoY6vR-eea^ ztStN}t`LGH8EAXlN9uX!Ovrj5aVdNq-)~z-qii4Gns?lqnBQAld7WkWt4@C+m#8|S zg8zi<5A3cQtuL{<=t0P#ypShJv`>q@PLqF(8C*VKzuq^EbVx!>mNvg59c|$~5p+I= zK{I*M-!yqr?VBsty4A3j%slqRDOih3CeAk<-UxyCF62L0`7==8TdW^adCuaAb7{84 z?)}L8$-sb)o5Ke@w6k)U1HX#!RBII8oj$*nHMRbHx4d^90z=_>fCP4c0(?z{ZxHMIfrZ}@xQ@3sa6#$!9^ zjE{~A0mxa&gY671r%3z8SBy4!$?qZZ587`U6zPzMWSE`v5f!A3q19w+#zN{`T?#n4 zSy#^r$v_v}>4xTkmmjr9CT8j5oMDUCn-wwOFEPF5ZCW zgKWvOxk1IG6F*&rhJ6@PrHSD%h}sP|o#srvEni%1d0t2BBzmhSY7ySEqx5ta=0r2W{+$Y~0 zC@p22XZXu!Q@(O+bFG}N+(6uB`YV0s#X2zz%L-&~%E4jH;@q67(G3GfKcF#nV{Xdg)t$xW!m}#^uSN6EC($}~ z?_Q>HY_ym@x5mUTC5pR0)P*W%zu0&i{O=sXIrnjJPc(KItAqbjIa3$ks8P7!Kl(5ABRq`R0KJY6Y-<9-U~(@H!eY{ldF{V4hxjwsFih`hmYWSv?FqC2;L$&9Dq#=`Uvd;`xYfP4)c0N=N+W$PUWw0%wPL z`16>XZ=&^_;))n=_}I{x7r%ZJ;dTq%I!c-Sbp>uSx?_HfZ#*4cIUJ1CY@l9Iz)(2h@m6KeJoy7#M7;rngt7t`+Rsr|fT(0VI|;;nsi&{Do1WyId+$WF zlenFac~p|VrwRSNG*vLtQg(BVII+VS!IyR5D|ZLsL`9#(6^ZXqAp_5L3wCRQr`Hfy z8G8JO-dh1N>H%rqFW{_W@lO#*o+RVMgSRAqgOLrb2M)QCV3HWkr!@7sOf9YK<_OpT zV{fa@_`DAko(tE+Kh33wch?QMI!F!>jA>)%=cgMd2K@ty_}FO@2TbJ0xI$}Sdx{XV z-`n7tvRJF3>*jorqe@KqsHlJfo<3glkD?1p%1qqyu=nFnUe>`*<;?DxdgwBE8V%ET z+WYTk{V$@rv+7Bb^L%BX``w%*>$?v>>>L#NuRr_jC}DBInPd=q0l|?O|1+@jGojRC z!>~l|RQ=0^C^l++q*lmgM}Hf|YhN^9ZYV1;qsMVPVXn=wx1HQMn@zf9Q<9>HZ++W8 zrXUtIQx=;zT1?K5&LwJhrKw15Hs;mXBt4l4ac#O#Jd=1A_WK7tUU^@fBtPU-cjU9# z(2~V^0k+36nni#$_YJo*o*KEcSmQOR)^R(3_frP+xA0J2v+Sa*%0OXfTkZZ8fq9yG zwjfh-gKsYlURO{-XurL5I|nB$fm|Nt=|KT&h9mhbpIh%mJ$EGnHlaXqW8b5T3|t!G zWvMrx5n{(IgoKG&k+{`vsVhT?t)n4p=uKz9MQfba;CPx8 z`fJJtPhIb)&0HxINUGKugjhE35_uIqa6Tl!{Ukj?Hm2%PC`TbjxqjK%mzTv=`yNlm zD9v30qHJ`%a9eqwGu6&`$xW7S`vMfsCR81w&9sXo>x)oNFjv)X==QqJ4HEn6f(LU; z7o`=;Q_;&JrqnPk7DdBDHod@#X`c~lJ$5Q0LX6K?MSaj6T@OO}m4z`C%ec0*LT*aF zqK*2sMI|Vkqh*ORIK&e(F>V}riCK7g3y`|G{^6?`)9f7LHFllcgSag%$twz&{mL0G z<|5g}De+r^J0g6%aAWUtYTy)69i1*HhfcS?CjW+J--d0H%dMiMaNBm}&*FO!7?nKM z!~G!pb-~~JO5*B3!&qb!Y;%5nM!kTd&*0u|z7+U3N3zV3*NgU{8Un?oG{`Wg1?qxi zG1(D=D(iGhOx>I4wd<^-mJWRNo&aGu0w!Y~#v_zJc0GkCyp;WJ+qv(Fz8mAqh3wA{ zN~%?hyemjw#k!m4W}PbvIO*pmiC4=r;0tV=_pLilk_LJDmnmQaf4-TNikcT_Y^eh7 zHXl)bW3`HwPx-ti;QwvfN5YDneVDy%iS%8I%H3vmwH{Fh!&_`9uzLjJXXp`3*2|?UIUC4|+wIX^Pd;ZFihC1KVGuO5%y(cgOiv zvx)Z1vGrJ~#7tDji(d;uq3n%VJbb2QV`d=pco)xXydj!bgmb#b^qSMC`V-DWOB3D7 z2}5FFBG-7KYWCgdRIVuH(MPyE)i=oPJ*U**Prlt^Gj(h*Pz=2yasOI^Txb*{#@RVk}OJYvDxTf10MJq7-i@ zpUlyQp6Z|m7M^4>*UVwh;2+aHJp5c}m9I>cDbZhYD^H2JfM&}^Lx-=@2`X#h{Emy-hKop0KQY>mKT&$Bud=B8pvQ~{@N`S2h)C2iNCLO)q1yk=VN6FzSuQ8G zB-9jY&@&lWQ7*wVtB}_;8q(T2(gG{gwz$7>%W)Idc_V`b^#z|e- zQD26RJ9BvFrUkI52Ruk}dCWMkgWquo>$H@X{e)>1JXYv-xO@MpmP3o5SV2NJfsm6y zR$0f(5^s?5W;Zawz|u?M+Ga*r8^t1CK?l-OyL0)Oi9*70>AHj|dzUcRZ1g2NiaEG3 z6apw6@y*|3t0l_N103wVn09&yF5wV1GewexrjO_h8c!&JwpKSyW1Wy9Q>(<2O>d!7 zRy~Oo8%G!(%e2L?h^)9C;&X)FN5OE_=zYAYrrE(0%)r-HnVFCU%a;#f5@mQj(}r zznQcBGDQ^)e|}ai<}*haD*QMk_`&hVex=ckxzzVYDBtNTQ7QHF__KQ3z2RiqD z0Ca-28WuA#e3+NJ*7>jO4?6(Cfl0gtRsBT$+c@uw`khGp7_aZ#S+jZ>f)jxjms{Oh z&e4OjIK^;mP>U&jx_5%OFs@u1X}xL%cV;Xw%eEE;doARrFMHu1*7x--HlzMOQZ1~gkwtm{-;uRoLW&@e_C>d_-c0|aZPKNWilL~uSNzu>}) z2H}1d!Dy;-DQpI>gyvnd=EQ*rZZTu$V3p5}k=~Jx!=Xyo+aS~TF^``7TY4oZzJjpQdH&Gio`|vJc~gfxBBS$U<^19v*NI zmVis3AJmXe^lBJ~XJDR@DJZ9v+|KmKZp~$gBUmwI!$`XZsRF2t_}o&|wT{&aclxpB f`l;X_lL-kGnMf%KA8io`!OumrE9WZDx Date: Thu, 28 May 2026 13:39:03 +0800 Subject: [PATCH 142/382] [CI] feat: pre-release ci workflow (#2212) * Add pre-release CI workflow to validate release without PyPI Introduce a tag-triggered pre-release workflow that mirrors the three release build pipelines (CUDA 12, non-CUDA, CUDA 13), runs twine check, and uploads wheels only as workflow artifacts. Pre-release tags use semver suffixes (rc, alpha, beta, pre). Production release workflows skip tags containing a hyphen so pre-release tags do not publish to PyPI or create GitHub Releases. --------- Co-authored-by: Cursor Agent Co-authored-by: Teng Ma Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Xun Sun --- .github/workflows/pre-release.yaml | 353 ++++++++++++++++++++++++ .github/workflows/release-cuda13.yaml | 1 + .github/workflows/release-non-cuda.yaml | 1 + .github/workflows/release-npu.yaml | 3 +- .github/workflows/release.yaml | 4 +- 5 files changed, 360 insertions(+), 2 deletions(-) create mode 100644 .github/workflows/pre-release.yaml diff --git a/.github/workflows/pre-release.yaml b/.github/workflows/pre-release.yaml new file mode 100644 index 00000000..d4df2abd --- /dev/null +++ b/.github/workflows/pre-release.yaml @@ -0,0 +1,353 @@ +name: Pre-Release + +# Dry-run of the release pipelines: build wheels like Release / Release Non-CUDA / +# Release CUDA 13, validate artifacts, but do not create a GitHub Release or publish to PyPI. +# +# Trigger by pushing a pre-release tag, for example: +# git tag v1.0.0-rc1 && git push origin v1.0.0-rc1 +on: + push: + tags: + - 'v*-rc*' + - 'v*-alpha*' + - 'v*-beta*' + - 'v*-pre*' + +env: + SCCACHE_GHA_ENABLED: "true" + +jobs: + build-cuda: + name: Build (CUDA 12) + runs-on: ubuntu-22.04 + permissions: + contents: read + strategy: + matrix: + python-version: ['3.10', '3.11', '3.12', '3.13'] + env: + BUILD_WITH_EP: "1" + TORCH_CUDA_ARCH_LIST: "8.0;9.0" + steps: + - name: Checkout source + uses: actions/checkout@v4 + + - name: Set version from tag + run: echo "VERSION=${GITHUB_REF_NAME#v}" >> $GITHUB_ENV + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Free up disk space + run: | + sudo rm -rf /usr/share/dotnet + sudo rm -rf /opt/ghc + sudo rm -rf /opt/hostedtoolcache/CodeQL + sudo rm -rf /usr/local/lib/android + df -h + + - name: Install CUDA Toolkit + uses: Jimver/cuda-toolkit@v0.2.24 + with: + cuda: '12.8.1' + linux-local-args: '["--toolkit"]' + method: 'network' + sub-packages: '["nvcc", "nvrtc-dev"]' + non-cuda-sub-packages: '["libcusparse-dev", "libcublas-dev", "libcusolver-dev"]' + + - name: Run sccache-cache + uses: mozilla-actions/sccache-action@v0.0.9 + + - name: Configure sccache + uses: actions/github-script@v7 + with: + script: | + core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); + core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); + + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + + - name: Configure project + run: | + sudo apt update -y + sudo bash -x dependencies.sh -y + mkdir build + cd build + cmake .. -DBUILD_UNIT_TESTS=OFF -DUSE_HTTP=ON -DUSE_ETCD=ON -DUSE_CUDA=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.1;2.10.0;2.11.0;2.12.0" -DSTORE_USE_ETCD=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Release + shell: bash + + - name: Build project + run: | + export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LIBRARY_PATH + cd build + make -j + sudo -E make install + shell: bash + + - name: Build nvlink_allocator.so + run: | + export PATH=/usr/local/nvidia/bin:/usr/local/nvidia/lib64:$PATH + export LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LD_LIBRARY_PATH + export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LIBRARY_PATH + mkdir -p build/mooncake-transfer-engine/nvlink-allocator + cd mooncake-transfer-engine/nvlink-allocator + bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ + shell: bash + + - name: Generate Python version tag + id: generate_tag_release + run: | + echo "python_version_tag=$(echo ${{ matrix.python-version }} | tr -d '.')" >> $GITHUB_OUTPUT + shell: bash + + - name: Build Python wheel + run: | + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib + PYTHON_VERSION=${{ matrix.python-version }} OUTPUT_DIR=dist-py${{ steps.generate_tag_release.outputs.python_version_tag }} ./scripts/build_wheel.sh + env: + VERSION: ${{ env.VERSION }} + + - name: Upload Python wheel artifact + uses: actions/upload-artifact@v4 + with: + name: pre-release-cuda-py${{ steps.generate_tag_release.outputs.python_version_tag }} + path: mooncake-wheel/dist-py${{ steps.generate_tag_release.outputs.python_version_tag }}/*.whl + + build-non-cuda: + name: Build (Non-CUDA) + runs-on: ubuntu-22.04 + permissions: + contents: read + strategy: + matrix: + python-version: ['3.10', '3.11', '3.12', '3.13'] + env: + BUILD_WITH_EP: "0" + NON_CUDA_BUILD: "1" + steps: + - name: Checkout source + uses: actions/checkout@v4 + + - name: Set version from tag + run: echo "VERSION=${GITHUB_REF_NAME#v}" >> $GITHUB_ENV + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Free up disk space + run: | + sudo rm -rf /usr/share/dotnet + sudo rm -rf /opt/ghc + sudo rm -rf /opt/hostedtoolcache/CodeQL + + - name: Run sccache-cache + uses: mozilla-actions/sccache-action@v0.0.9 + + - name: Configure sccache + uses: actions/github-script@v7 + with: + script: | + core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); + core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); + + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + + - name: Configure project + run: | + sudo apt update -y + sudo bash -x dependencies.sh -y + mkdir build + cd build + cmake .. -DUSE_HTTP=ON -DUSE_ETCD=ON -DUSE_CUDA=OFF -DWITH_EP=OFF -DSTORE_USE_ETCD=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Release + shell: bash + + - name: Build project + run: | + cd build + make -j + sudo make install + shell: bash + + - name: Generate Python version tag + id: generate_tag_release + run: | + echo "python_version_tag=$(echo ${{ matrix.python-version }} | tr -d '.')" >> $GITHUB_OUTPUT + shell: bash + + - name: Build Python wheel + run: | + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib + PYTHON_VERSION=${{ matrix.python-version }} OUTPUT_DIR=dist-py${{ steps.generate_tag_release.outputs.python_version_tag }} ./scripts/build_wheel.sh + env: + VERSION: ${{ env.VERSION }} + + - name: Upload Python wheel artifact + uses: actions/upload-artifact@v4 + with: + name: pre-release-non-cuda-py${{ steps.generate_tag_release.outputs.python_version_tag }} + path: mooncake-wheel/dist-py${{ steps.generate_tag_release.outputs.python_version_tag }}/*.whl + + build-cuda13: + name: Build (CUDA 13) + runs-on: ubuntu-22.04 + permissions: + contents: read + strategy: + matrix: + python-version: ['3.10', '3.11', '3.12', '3.13'] + env: + BUILD_WITH_EP: "1" + CU13_BUILD: "1" + TORCH_CUDA_ARCH_LIST: "8.0;9.0" + steps: + - name: Checkout source + uses: actions/checkout@v4 + + - name: Set version from tag + run: echo "VERSION=${GITHUB_REF_NAME#v}" >> $GITHUB_ENV + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Free up disk space + run: | + sudo rm -rf /usr/share/dotnet + sudo rm -rf /opt/ghc + sudo rm -rf /opt/hostedtoolcache/CodeQL + sudo rm -rf /usr/local/lib/android + df -h + + - name: Install CUDA Toolkit 13 + uses: Jimver/cuda-toolkit@v0.2.29 + with: + cuda: '13.0.2' + linux-local-args: '["--toolkit"]' + method: 'network' + sub-packages: '["nvcc", "nvrtc-dev"]' + non-cuda-sub-packages: '["libcusparse-dev", "libcublas-dev", "libcusolver-dev"]' + + - name: Run sccache-cache + uses: mozilla-actions/sccache-action@v0.0.9 + + - name: Configure sccache + uses: actions/github-script@v7 + with: + script: | + core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); + core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); + + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + + - name: Configure project + run: | + sudo apt update -y + sudo bash -x dependencies.sh -y + mkdir build + cd build + cmake .. -DBUILD_UNIT_TESTS=OFF -DUSE_HTTP=ON -DUSE_ETCD=ON -DUSE_CUDA=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.0;2.9.1;2.10.0;2.11.0" -DSTORE_USE_ETCD=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Release + shell: bash + + - name: Build project + run: | + export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LIBRARY_PATH + cd build + make -j + sudo make install + shell: bash + + - name: Build nvlink_allocator.so + run: | + export PATH=/usr/local/nvidia/bin:/usr/local/nvidia/lib64:$PATH + export LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LD_LIBRARY_PATH + export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LIBRARY_PATH + mkdir -p build/mooncake-transfer-engine/nvlink-allocator + cd mooncake-transfer-engine/nvlink-allocator + bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ + shell: bash + + - name: Generate Python version tag + id: generate_tag_release + run: | + echo "python_version_tag=$(echo ${{ matrix.python-version }} | tr -d '.')" >> $GITHUB_OUTPUT + shell: bash + + - name: Build Python wheel + run: | + export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib + PYTHON_VERSION=${{ matrix.python-version }} OUTPUT_DIR=dist-py${{ steps.generate_tag_release.outputs.python_version_tag }} ./scripts/build_wheel.sh + env: + VERSION: ${{ env.VERSION }} + + - name: Upload Python wheel artifact + uses: actions/upload-artifact@v4 + with: + name: pre-release-cuda13-py${{ steps.generate_tag_release.outputs.python_version_tag }} + path: mooncake-wheel/dist-py${{ steps.generate_tag_release.outputs.python_version_tag }}/*.whl + + validate-release: + name: Validate release artifacts + needs: [build-cuda, build-non-cuda, build-cuda13] + runs-on: ubuntu-22.04 + permissions: + contents: read + steps: + - name: Checkout source + uses: actions/checkout@v4 + + - name: Download all pre-release wheel artifacts + uses: actions/download-artifact@v4 + with: + path: mooncake-wheel/dist-all + pattern: pre-release-* + + - name: Prepare wheels for validation + run: | + mkdir -p mooncake-wheel/dist-release + find mooncake-wheel/dist-all -name "*.whl" -exec cp {} mooncake-wheel/dist-release/ \; + echo "Pre-release tag: ${GITHUB_REF_NAME}" + echo "Collected wheels:" + ls -la mooncake-wheel/dist-release/ + wheel_count=$(find mooncake-wheel/dist-release -name "*.whl" | wc -l) + echo "wheel_count=${wheel_count}" >> "$GITHUB_ENV" + if [ "${wheel_count}" -lt 12 ]; then + echo "Expected at least 12 wheels (4 Python versions x 3 variants), found ${wheel_count}" + exit 1 + fi + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.12' + + - name: Validate wheels with twine + run: | + pip install twine + twine check mooncake-wheel/dist-release/*.whl + + - name: Upload validated wheels as workflow artifacts + uses: actions/upload-artifact@v4 + with: + name: pre-release-wheels-${{ github.ref_name }} + path: mooncake-wheel/dist-release/*.whl + retention-days: 14 + + - name: Pre-release validation summary + run: | + echo "## Pre-Release validation passed" >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "- Tag: \`${GITHUB_REF_NAME}\`" >> "$GITHUB_STEP_SUMMARY" + echo "- Wheels built: ${wheel_count}" >> "$GITHUB_STEP_SUMMARY" + echo "- PyPI publish: skipped (pre-release dry run)" >> "$GITHUB_STEP_SUMMARY" + echo "- GitHub Release upload: skipped (pre-release dry run)" >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/release-cuda13.yaml b/.github/workflows/release-cuda13.yaml index 0cd51990..29f23d8b 100644 --- a/.github/workflows/release-cuda13.yaml +++ b/.github/workflows/release-cuda13.yaml @@ -107,6 +107,7 @@ jobs: path: mooncake-wheel/dist-py${{ steps.generate_tag_release.outputs.python_version_tag }}/*.whl publish-release: + if: ${{ !contains(github.ref_name, '-') }} needs: build runs-on: ubuntu-22.04 permissions: diff --git a/.github/workflows/release-non-cuda.yaml b/.github/workflows/release-non-cuda.yaml index ab9b8424..15b633ec 100644 --- a/.github/workflows/release-non-cuda.yaml +++ b/.github/workflows/release-non-cuda.yaml @@ -84,6 +84,7 @@ jobs: path: mooncake-wheel/dist-py${{ steps.generate_tag_release.outputs.python_version_tag }}/*.whl publish-release: + if: ${{ !contains(github.ref_name, '-') }} needs: build runs-on: ubuntu-22.04 permissions: diff --git a/.github/workflows/release-npu.yaml b/.github/workflows/release-npu.yaml index 1494b8b7..6bebfafe 100644 --- a/.github/workflows/release-npu.yaml +++ b/.github/workflows/release-npu.yaml @@ -7,7 +7,7 @@ on: jobs: build: - if: github.repository == 'kvcache-ai/Mooncake' + if: ${{ github.repository == 'kvcache-ai/Mooncake' && !contains(github.ref_name, '-') }} runs-on: self-hosted @@ -194,6 +194,7 @@ jobs: path: mooncake-wheel/dist-npu-py${{ steps.generate_tag_release.outputs.python_version_tag }}/*.whl publish-release: + if: ${{ !contains(github.ref_name, '-') }} needs: build runs-on: ubuntu-22.04 environment: pypi diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index a9169592..1573e8bd 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -9,6 +9,8 @@ env: SCCACHE_GHA_ENABLED: "true" jobs: build: + # Skip semver pre-release tags (e.g. v1.0.0-rc1); those are handled by pre-release.yaml. + if: ${{ !contains(github.ref_name, '-') }} runs-on: ubuntu-22.04 permissions: contents: write @@ -69,7 +71,7 @@ jobs: - name: Build project run: | - export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:LIBRARY_PATH + export LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LIBRARY_PATH cd build make -j sudo -E make install -- 2.34.1 From c23d3081b1e2aa8126592ae120972d31890cc357 Mon Sep 17 00:00:00 2001 From: Kafka <92581531+ZhenyuePan@users.noreply.github.com> Date: Thu, 28 May 2026 15:09:09 +0800 Subject: [PATCH 143/382] [Doc] Clarify TENT failover poll behavior (#2208) Document enable_auto_failover_on_poll so readers understand when status polling is observational only, and add both failover test commands so the verification steps from the PR follow-up are reproducible. --- docs/source/design/tent/failover.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/source/design/tent/failover.md b/docs/source/design/tent/failover.md index 4a8c9b23..676b986a 100644 --- a/docs/source/design/tent/failover.md +++ b/docs/source/design/tent/failover.md @@ -95,6 +95,7 @@ All knobs live in the top-level `transfer-engine.json`. Defaults are safe for pr | Key | Default | Meaning | |-----|---------|---------| +| `enable_auto_failover_on_poll` | `true` | Controls whether `getTransferStatus` automatically resubmits tasks that report a recoverable `FAILED` completion. Set to `false` to make status polling observational only; internal completion paths can still trigger failover/resubmit. | | `max_failover_attempts` | `3` | Upper bound on `resubmitTransferTask` calls per task. `0` disables cross-transport failover entirely. `1` allows exactly one switch. | | `transports/rdma/rail_error_threshold` | `3` | Number of failures inside `rail_error_window_secs` that trips a rail into the paused state. | | `transports/rdma/rail_error_window_secs` | `10` | Sliding window for counting rail errors. A failure older than the window resets `error_count` to 1. | @@ -104,6 +105,7 @@ The RDMA keys are read by `RailMonitor::load`. Example: ```json { + "enable_auto_failover_on_poll": true, "max_failover_attempts": 3, "transports": { "rdma": { @@ -175,7 +177,8 @@ The TENT tests are **not** in CI today (the upstream workflow builds with `USE_T ```bash cmake -S . -B build-tent -DUSE_TENT=ON -DUSE_CUDA=OFF -cmake --build build-tent --target tent_engine_failover_e2e_test -j +cmake --build build-tent --target tent_failover_test tent_engine_failover_e2e_test -j +./build-tent/mooncake-transfer-engine/tent/tests/tent_failover_test ./build-tent/mooncake-transfer-engine/tent/tests/tent_engine_failover_e2e_test ``` -- 2.34.1 From a18fad1f30dcf9877d12275cd66c4fd75a0f35c9 Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Thu, 28 May 2026 17:50:52 +0800 Subject: [PATCH 144/382] [Security] Fix Go vulnerabilities in libetcd_wrapper.so (#2250) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Update Go toolchain and dependencies to address CVEs: - Go 1.25.9 → 1.25.10 (fixes CVE-2026-33814, CVE-2026-39836, CVE-2026-42499, CVE-2026-33811, CVE-2026-39820, CVE-2026-42501) - golang.org/x/net v0.48.0 → v0.55.0 (fixes CVE-2026-39821, CVE-2026-33814) This rebuilds libetcd_wrapper.so with patched Go stdlib and golang.org/x/net to resolve downstream vulnerability scanner findings. Co-authored-by: Claude Opus 4.7 (1M context) --- mooncake-common/etcd/go.mod | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mooncake-common/etcd/go.mod b/mooncake-common/etcd/go.mod index d1507b5d..ef5b3503 100644 --- a/mooncake-common/etcd/go.mod +++ b/mooncake-common/etcd/go.mod @@ -2,7 +2,7 @@ module github.com/kvcache-ai/Mooncake/mooncake-common/etcd go 1.25.0 -toolchain go1.25.9 +toolchain go1.25.10 require ( go.etcd.io/etcd/api/v3 v3.5.21 @@ -18,7 +18,7 @@ require ( go.uber.org/atomic v1.7.0 // indirect go.uber.org/multierr v1.6.0 // indirect go.uber.org/zap v1.17.0 // indirect - golang.org/x/net v0.48.0 // indirect + golang.org/x/net v0.55.0 // indirect golang.org/x/sys v0.39.0 // indirect golang.org/x/text v0.32.0 // indirect google.golang.org/genproto/googleapis/api v0.0.0-20251202230838-ff82c1b0f217 // indirect -- 2.34.1 From 4569ce7d96416ce80b398e9382f0c2cad74521f9 Mon Sep 17 00:00:00 2001 From: Dao007forever Date: Thu, 28 May 2026 05:30:20 -0700 Subject: [PATCH 145/382] Build tent (#2089) * Build with TENT * Fix TENT failed start * Revert * Format * Empty --- .../fabric_allocator.cmake | 1 + mooncake-transfer-engine/tent/CMakeLists.txt | 4 +- .../tent/src/runtime/transfer_engine_impl.cpp | 12 ++-- .../src/transport/nvlink/nvlink_transport.cpp | 60 ++++++++++++++++++- 4 files changed, 69 insertions(+), 8 deletions(-) diff --git a/mooncake-transfer-engine/fabric_allocator.cmake b/mooncake-transfer-engine/fabric_allocator.cmake index 52537d1b..62ee1095 100644 --- a/mooncake-transfer-engine/fabric_allocator.cmake +++ b/mooncake-transfer-engine/fabric_allocator.cmake @@ -27,6 +27,7 @@ function(add_fabric_allocator_build_target) if(FAB_ENABLE_BUILD) add_custom_command( TARGET ${FAB_TARGET_NAME} + POST_BUILD COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_CURRENT_BINARY_DIR} COMMAND bash ${FAB_BUILD_SCRIPT} ${FAB_BUILD_ARGS} ${CMAKE_CURRENT_BINARY_DIR} "${_include_dirs_str}" diff --git a/mooncake-transfer-engine/tent/CMakeLists.txt b/mooncake-transfer-engine/tent/CMakeLists.txt index 82bced27..b5bcc379 100644 --- a/mooncake-transfer-engine/tent/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/CMakeLists.txt @@ -8,4 +8,6 @@ set(CMAKE_POSITION_INDEPENDENT_CODE ON) add_subdirectory(src) add_subdirectory(plugins) -add_subdirectory(tests) +if (BUILD_UNIT_TESTS) + add_subdirectory(tests) +endif() diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index 015d0112..a8936422 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -26,7 +26,6 @@ #include "tent/common/config.h" #include "tent/common/status.h" -#include "tent/metastore/redis.h" #include "tent/runtime/control_plane.h" #include "tent/runtime/segment.h" #include "tent/runtime/segment_tracker.h" @@ -44,6 +43,11 @@ namespace mooncake { namespace tent { +namespace { +constexpr uint8_t kRedisMaxDbIndex = 255; +constexpr uint8_t kRedisDefaultDbIndex = 0; +} // namespace + struct Batch { Batch() : max_size(0) { sub_batch.fill(nullptr); } @@ -318,14 +322,14 @@ Status TransferEngineImpl::construct() { CHECK_STATUS(topology_->discover({loader})); // Validate redis_db_index range (0-255) - uint8_t db_index = REDIS_DEFAULT_DB_INDEX; + uint8_t db_index = kRedisDefaultDbIndex; if (redis_db_index_config >= 0 && - redis_db_index_config <= REDIS_MAX_DB_INDEX) { + redis_db_index_config <= kRedisMaxDbIndex) { db_index = static_cast(redis_db_index_config); } else { LOG(WARNING) << "Invalid Redis DB index: " << redis_db_index_config << ", using default " - << static_cast(REDIS_DEFAULT_DB_INDEX); + << static_cast(kRedisDefaultDbIndex); } metadata_ = std::make_shared( diff --git a/mooncake-transfer-engine/tent/src/transport/nvlink/nvlink_transport.cpp b/mooncake-transfer-engine/tent/src/transport/nvlink/nvlink_transport.cpp index 5a4d7b09..e018c91f 100644 --- a/mooncake-transfer-engine/tent/src/transport/nvlink/nvlink_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/nvlink/nvlink_transport.cpp @@ -33,6 +33,28 @@ namespace mooncake { namespace tent { +namespace { + +Status setCudaDeviceForLocation(const LocationParser& location, + int& saved_dev) { + saved_dev = -1; + CHECK_CUDA(cudaGetDevice(&saved_dev)); + if (location.index() >= 0 && saved_dev != location.index()) { + CHECK_CUDA(cudaSetDevice(location.index())); + } + return Status::OK(); +} + +Status restoreCudaDeviceForLocation(const LocationParser& location, + int saved_dev) { + if (saved_dev >= 0 && location.index() >= 0 && + saved_dev != location.index()) { + CHECK_CUDA(cudaSetDevice(saved_dev)); + } + return Status::OK(); +} + +} // namespace NVLinkTransport::NVLinkTransport() : installed_(false) {} @@ -228,10 +250,25 @@ Status NVLinkTransport::addMemoryBuffer(BufferDesc& desc, const MemoryOptions& options) { LocationParser location(desc.location); if (location.type() == "cuda") { - // If the memory region is allocated using cuMemAlloc, - // we cannot use cudaIpcGetMemHandle, so skip it + // MNNVL allocations are exported by MnnvlTransport instead of CUDA IPC. if (options.type == MNNVL) return Status::OK(); + int saved_dev = -1; + CHECK_STATUS(setCudaDeviceForLocation(location, saved_dev)); + + // VMM allocations have driver allocation handles, but + // cudaIpcGetMemHandle only supports cudaMalloc-backed pointers. + CUmemGenericAllocationHandle generic_handle; + CUresult retain_result = + cuMemRetainAllocationHandle(&generic_handle, (void*)desc.addr); + if (retain_result == CUDA_SUCCESS) { + cuMemRelease(generic_handle); + CHECK_STATUS(restoreCudaDeviceForLocation(location, saved_dev)); + LOG(INFO) << "NVLinkTransport: memory region " << (void*)desc.addr + << " is not cudaMalloc-backed; skip CUDA IPC export."; + return Status::OK(); + } + // Resolve the true cudaMalloc base address. Caching allocators // (e.g. PyTorch) sub-allocate tensors within larger cudaMalloc // segments. cudaIpcGetMemHandle returns a handle for the whole @@ -244,6 +281,7 @@ Status NVLinkTransport::addMemoryBuffer(BufferDesc& desc, LOG(ERROR) << "NVLinkTransport: cuMemGetAddressRange failed for " << "addr 0x" << std::hex << desc.addr << std::dec << " (error " << cu_err << ")"; + CHECK_STATUS(restoreCudaDeviceForLocation(location, saved_dev)); return Status::InternalError( "cuMemGetAddressRange failed" LOC_MARK); } @@ -255,12 +293,24 @@ Status NVLinkTransport::addMemoryBuffer(BufferDesc& desc, desc.addr = (uint64_t)base_ptr; desc.length = alloc_size; desc.transports.push_back(TransportType::NVLINK); + CHECK_STATUS(restoreCudaDeviceForLocation(location, saved_dev)); return Status::OK(); } } cudaIpcMemHandle_t handle; - CHECK_CUDA(cudaIpcGetMemHandle(&handle, (void*)base_ptr)); + auto cuda_err = cudaIpcGetMemHandle(&handle, (void*)base_ptr); + CHECK_STATUS(restoreCudaDeviceForLocation(location, saved_dev)); + if (cuda_err != cudaSuccess) { + LOG(ERROR) << "NVLinkTransport: cudaIpcGetMemHandle failed for " + << "addr 0x" << std::hex << desc.addr << ", base 0x" + << (uint64_t)base_ptr << std::dec << ", device " + << location.index() << ": " + << cudaGetErrorString(cuda_err); + return Status::InternalError( + std::string("cudaIpcGetMemHandle(&handle, (void*)base_ptr): ") + + cudaGetErrorString(cuda_err) + LOC_MARK); + } desc.addr = (uint64_t)base_ptr; desc.length = alloc_size; desc.shm_path = @@ -287,10 +337,14 @@ Status NVLinkTransport::removeMemoryBuffer(BufferDesc& desc) { if (location.type() == "cuda") { // Resolve base the same way we did in addMemoryBuffer, so we // remove the right entry even for sub-allocated addresses. + int saved_dev = -1; + CHECK_STATUS(setCudaDeviceForLocation(location, saved_dev)); + CUdeviceptr base_ptr = 0; size_t alloc_size = 0; CUresult cu_err = cuMemGetAddressRange(&base_ptr, &alloc_size, (CUdeviceptr)desc.addr); + CHECK_STATUS(restoreCudaDeviceForLocation(location, saved_dev)); uint64_t key = desc.addr; if (cu_err == CUDA_SUCCESS) { -- 2.34.1 From f6b4adbc9ab9ea107e28c3316ec134c535479cbd Mon Sep 17 00:00:00 2001 From: CAICAII <3360776475@qq.com> Date: Thu, 28 May 2026 21:03:07 +0800 Subject: [PATCH 146/382] [Store] Clarify cache stats semantics (#2248) Signed-off-by: CAICAIIs <3360776475@qq.com> --- .../mooncake-store-deployment-guide.md | 15 +++ docs/source/design/tent/overview.md | 8 ++ mooncake-store/include/master_client.h | 6 +- .../include/master_metric_manager.h | 14 ++- mooncake-store/src/master_metric_manager.cpp | 61 ++++++---- mooncake-store/tests/master_metrics_test.cpp | 113 ++++++++++++++---- 6 files changed, 167 insertions(+), 50 deletions(-) diff --git a/docs/source/deployment/mooncake-store-deployment-guide.md b/docs/source/deployment/mooncake-store-deployment-guide.md index 58d56a4d..641695f6 100644 --- a/docs/source/deployment/mooncake-store-deployment-guide.md +++ b/docs/source/deployment/mooncake-store-deployment-guide.md @@ -126,6 +126,21 @@ curl -s http://:9003/metrics curl -s http://:9003/metrics/summary ``` +Mooncake Store can report Store-observed cache reuse signals, such as +completed `GetReplicaList` results served from memory/SSD and current cached +object counts. These signals help operators understand reuse inside the Store, +but they are not the final request-level or token-level cache hit ratio for an +inference system. That end-to-end hit ratio should be calculated by Conductor or +the inference engine, which can observe the full request path across GPU, CPU, +and Mooncake tiers. + +For `CalcCacheStats()`, prefer the `MEMORY_CURRENT_CACHED_OBJECTS`, +`SSD_CURRENT_CACHED_OBJECTS`, and `*_HITS_PER_CURRENT_CACHED_OBJECT` enum +aliases when consuming Store-side values. The older `*_TOTAL` and `*_HIT_RATE` +names are retained for compatibility. The `*_HIT_RATE` values divide cumulative +Store-observed hits by current cached object counts, so they are not bounded +request-level hit ratios and may exceed `1.0`. + ## Client/Engine Tuning (Env Vars, with defaults) - Topology discovery (Store Client → Transfer Engine) diff --git a/docs/source/design/tent/overview.md b/docs/source/design/tent/overview.md index 0c19cdc7..13dc295e 100644 --- a/docs/source/design/tent/overview.md +++ b/docs/source/design/tent/overview.md @@ -89,6 +89,14 @@ cpp-api metrics ::: +## TENT Transport Selection + +:::{toctree} +:maxdepth: 1 + +transport-selector +::: + ## TENT Quality of Service :::{toctree} diff --git a/mooncake-store/include/master_client.h b/mooncake-store/include/master_client.h index 038b1618..5da96fba 100644 --- a/mooncake-store/include/master_client.h +++ b/mooncake-store/include/master_client.h @@ -101,9 +101,11 @@ class MasterClient { const std::vector& object_keys); /** - * @brief Calculate cache hit rate metrics + * @brief Calculate Store-observed cache reuse metrics * @param object_keys None - * @return Map containing metrics + * @return Map containing metrics. Legacy hit-rate keys describe cumulative + * Store-side hits normalized by current cached object counts, not + * end-to-end request/token hit ratios. */ [[nodiscard]] tl::expected CalcCacheStats(); diff --git a/mooncake-store/include/master_metric_manager.h b/mooncake-store/include/master_metric_manager.h index 34ad14e8..d50c61e6 100644 --- a/mooncake-store/include/master_metric_manager.h +++ b/mooncake-store/include/master_metric_manager.h @@ -58,7 +58,16 @@ class MasterMetricManager { MEMORY_HIT_RATE, SSD_HIT_RATE, OVERALL_HIT_RATE, - VALID_GET_RATE + VALID_GET_RATE, + // Clearer aliases for Store-observed counters. The legacy names above + // are kept to preserve RPC/API enum values. + MEMORY_CURRENT_CACHED_OBJECTS = MEMORY_TOTAL, + SSD_CURRENT_CACHED_OBJECTS = SSD_TOTAL, + // These values may exceed 1.0 because the numerator is cumulative + // while the denominator is the current cached object count. + MEMORY_HITS_PER_CURRENT_CACHED_OBJECT = MEMORY_HIT_RATE, + SSD_HITS_PER_CURRENT_CACHED_OBJECT = SSD_HIT_RATE, + OVERALL_HITS_PER_CURRENT_CACHED_OBJECT = OVERALL_HIT_RATE }; using CacheHitStatDict = std::unordered_map; void add_stat_to_dict(CacheHitStatDict&, CacheHitStat, double); @@ -581,7 +590,8 @@ class MasterMetricManager { ylt::metric::counter_t batch_put_revoke_items_; ylt::metric::counter_t batch_put_revoke_failed_items_; - // cache hit Statistics + // Store-observed cache reuse statistics. These counters do not represent + // end-to-end request/token-level cache hit ratio. ylt::metric::counter_t mem_cache_hit_nums_; ylt::metric::counter_t file_cache_hit_nums_; ylt::metric::gauge_t mem_cache_nums_; diff --git a/mooncake-store/src/master_metric_manager.cpp b/mooncake-store/src/master_metric_manager.cpp index da1ef239..d82d6be1 100644 --- a/mooncake-store/src/master_metric_manager.cpp +++ b/mooncake-store/src/master_metric_manager.cpp @@ -269,18 +269,23 @@ MasterMetricManager::MasterMetricManager() "master_batch_put_revoke_failed_items_total", "Total number of failed items in BatchPutRevoke requests"), - // Initialize cache hit rate metrics + // Initialize Store-observed cache reuse metrics. These are not + // end-to-end request/token-level cache hit ratio metrics. mem_cache_hit_nums_("mem_cache_hit_nums_", - "Total number of cache hits in the memory pool"), + "Total number of GetReplicaList results served from " + "the memory pool"), file_cache_hit_nums_("file_cache_hit_nums_", - "Total number of cache hits in the ssd"), + "Total number of GetReplicaList results served from " + "the SSD cache"), mem_cache_nums_("mem_cache_nums_", - "Total number of cached values in the memory pool"), + "Current number of cached values in the memory pool"), file_cache_nums_("file_cache_nums_", - "Total number of cached values in the ssd"), + "Current number of cached values in the SSD cache"), valid_get_nums_("valid_get_nums_", - "Total number of valid get operations"), - total_get_nums_("total_get_nums_", "Total number of get operations"), + "Total number of GetReplicaList operations that returned " + "at least one completed replica"), + total_get_nums_("total_get_nums_", + "Total number of GetReplicaList operations"), // Initialize Eviction Counters // total eviction @@ -506,7 +511,7 @@ void MasterMetricManager::update_metrics_for_zero_output() { batch_put_revoke_items_.inc(0); batch_put_revoke_failed_items_.inc(0); - // Update cache hit rate metrics + // Update Store-observed cache reuse metrics mem_cache_hit_nums_.inc(0); file_cache_hit_nums_.inc(0); valid_get_nums_.inc(0); @@ -762,7 +767,7 @@ int64_t MasterMetricManager::get_active_clients() { return active_clients_.value(); } -// cache hit rate metrics +// Store-observed cache reuse metrics void MasterMetricManager::inc_mem_cache_hit_nums(int64_t val) { mem_cache_hit_nums_.inc(val); } @@ -1665,25 +1670,33 @@ MasterMetricManager::calculate_cache_stats() { int64_t valid_get_nums = valid_get_nums_.value(); int64_t total_get_nums = total_get_nums_.value(); - double mem_hit_rate = 0.0; + // These values divide cumulative Store-observed hits by the current cached + // object count. They are not bounded request/token-level hit ratios; that + // end-to-end metric belongs to Conductor or the inference engine. + double mem_hits_per_current_cached_object = 0.0; if (mem_total_cache > 0) { - mem_hit_rate = static_cast(mem_cache_hits) / - static_cast(mem_total_cache); - mem_hit_rate = std::round(mem_hit_rate * 100.0) / 100.0; + mem_hits_per_current_cached_object = + static_cast(mem_cache_hits) / + static_cast(mem_total_cache); + mem_hits_per_current_cached_object = + std::round(mem_hits_per_current_cached_object * 100.0) / 100.0; } - double ssd_hit_rate = 0.0; + double ssd_hits_per_current_cached_object = 0.0; if (ssd_total_cache > 0) { - ssd_hit_rate = static_cast(ssd_cache_hits) / - static_cast(ssd_total_cache); - ssd_hit_rate = std::round(ssd_hit_rate * 100.0) / 100.0; + ssd_hits_per_current_cached_object = + static_cast(ssd_cache_hits) / + static_cast(ssd_total_cache); + ssd_hits_per_current_cached_object = + std::round(ssd_hits_per_current_cached_object * 100.0) / 100.0; } - double total_hit_rate = 0.0; + double overall_hits_per_current_cached_object = 0.0; if (total_cache > 0) { - total_hit_rate = + overall_hits_per_current_cached_object = static_cast(total_hits) / static_cast(total_cache); - total_hit_rate = std::round(total_hit_rate * 100.0) / 100.0; + overall_hits_per_current_cached_object = + std::round(overall_hits_per_current_cached_object * 100.0) / 100.0; } double valid_get_rate = 0.0; @@ -1697,10 +1710,12 @@ MasterMetricManager::calculate_cache_stats() { add_stat_to_dict(stats_dict, CacheHitStat::SSD_HITS, ssd_cache_hits); add_stat_to_dict(stats_dict, CacheHitStat::MEMORY_TOTAL, mem_total_cache); add_stat_to_dict(stats_dict, CacheHitStat::SSD_TOTAL, ssd_total_cache); - add_stat_to_dict(stats_dict, CacheHitStat::MEMORY_HIT_RATE, mem_hit_rate); - add_stat_to_dict(stats_dict, CacheHitStat::SSD_HIT_RATE, ssd_hit_rate); + add_stat_to_dict(stats_dict, CacheHitStat::MEMORY_HIT_RATE, + mem_hits_per_current_cached_object); + add_stat_to_dict(stats_dict, CacheHitStat::SSD_HIT_RATE, + ssd_hits_per_current_cached_object); add_stat_to_dict(stats_dict, CacheHitStat::OVERALL_HIT_RATE, - total_hit_rate); + overall_hits_per_current_cached_object); add_stat_to_dict(stats_dict, CacheHitStat::VALID_GET_RATE, valid_get_rate); return stats_dict; } diff --git a/mooncake-store/tests/master_metrics_test.cpp b/mooncake-store/tests/master_metrics_test.cpp index b57dcc2d..467fca69 100644 --- a/mooncake-store/tests/master_metrics_test.cpp +++ b/mooncake-store/tests/master_metrics_test.cpp @@ -1,7 +1,9 @@ #include #include +#include #include +#include #include #include #include @@ -270,6 +272,55 @@ TEST_F(MasterMetricsTest, BasicRequestTest) { TEST_F(MasterMetricsTest, CalcCacheStatsTest) { const uint64_t default_kv_lease_ttl = 100; auto& metrics = MasterMetricManager::instance(); + using CacheHitStat = MasterMetricManager::CacheHitStat; + + // These values are part of the RPC/API contract. New enum entries should + // be appended instead of renumbering existing values. + ASSERT_EQ(static_cast(CacheHitStat::MEMORY_HITS), 0); + ASSERT_EQ(static_cast(CacheHitStat::SSD_HITS), 1); + ASSERT_EQ(static_cast(CacheHitStat::MEMORY_TOTAL), 2); + ASSERT_EQ(static_cast(CacheHitStat::SSD_TOTAL), 3); + ASSERT_EQ(static_cast(CacheHitStat::MEMORY_HIT_RATE), 4); + ASSERT_EQ(static_cast(CacheHitStat::SSD_HIT_RATE), 5); + ASSERT_EQ(static_cast(CacheHitStat::OVERALL_HIT_RATE), 6); + ASSERT_EQ(static_cast(CacheHitStat::VALID_GET_RATE), 7); + + auto round_to_2 = [](double value) { + return std::round(value * 100.0) / 100.0; + }; + auto expected_ratio = [&round_to_2](double hits, double total) { + return total > 0.0 ? round_to_2(hits / total) : 0.0; + }; + auto expect_aliases = + [](const MasterMetricManager::CacheHitStatDict& stats) { + ASSERT_EQ(stats.at(CacheHitStat::MEMORY_CURRENT_CACHED_OBJECTS), + stats.at(CacheHitStat::MEMORY_TOTAL)); + ASSERT_EQ(stats.at(CacheHitStat::SSD_CURRENT_CACHED_OBJECTS), + stats.at(CacheHitStat::SSD_TOTAL)); + ASSERT_EQ( + stats.at(CacheHitStat::MEMORY_HITS_PER_CURRENT_CACHED_OBJECT), + stats.at(CacheHitStat::MEMORY_HIT_RATE)); + ASSERT_EQ( + stats.at(CacheHitStat::SSD_HITS_PER_CURRENT_CACHED_OBJECT), + stats.at(CacheHitStat::SSD_HIT_RATE)); + ASSERT_EQ( + stats.at(CacheHitStat::OVERALL_HITS_PER_CURRENT_CACHED_OBJECT), + stats.at(CacheHitStat::OVERALL_HIT_RATE)); + }; + auto expect_reuse_ratios = + [&expected_ratio](const MasterMetricManager::CacheHitStatDict& stats) { + const double memory_hits = stats.at(CacheHitStat::MEMORY_HITS); + const double ssd_hits = stats.at(CacheHitStat::SSD_HITS); + const double memory_total = stats.at(CacheHitStat::MEMORY_TOTAL); + const double ssd_total = stats.at(CacheHitStat::SSD_TOTAL); + ASSERT_EQ(stats.at(CacheHitStat::MEMORY_HIT_RATE), + expected_ratio(memory_hits, memory_total)); + ASSERT_EQ(stats.at(CacheHitStat::SSD_HIT_RATE), + expected_ratio(ssd_hits, ssd_total)); + ASSERT_EQ(stats.at(CacheHitStat::OVERALL_HIT_RATE), + expected_ratio(memory_hits + ssd_hits, + memory_total + ssd_total)); + }; // Use a wrapped master service to test the metrics manager WrappedMasterServiceConfig service_config; service_config.default_kv_lease_ttl = default_kv_lease_ttl; @@ -292,17 +343,16 @@ TEST_F(MasterMetricsTest, CalcCacheStatsTest) { ReplicateConfig config; config.replica_num = 1; - auto stats_dict = metrics.calculate_cache_stats(); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::MEMORY_HITS], 1); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::SSD_HITS], 0); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::MEMORY_TOTAL], 2); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::SSD_TOTAL], 0); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::MEMORY_HIT_RATE], - 0.5); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::SSD_HIT_RATE], 0); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::OVERALL_HIT_RATE], - 0.5); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::VALID_GET_RATE], 1); + // MasterMetricManager is a process-wide singleton and these counters do + // not have reset APIs, so assert deltas from the current baseline. + const auto base_stats = metrics.calculate_cache_stats(); + expect_aliases(base_stats); + expect_reuse_ratios(base_stats); + + const double base_memory_hits = base_stats.at(CacheHitStat::MEMORY_HITS); + const double base_memory_total = base_stats.at(CacheHitStat::MEMORY_TOTAL); + const double base_valid_get_rate = + base_stats.at(CacheHitStat::VALID_GET_RATE); auto mount_result = service_.MountSegment(segment, client_id); ASSERT_TRUE(mount_result.has_value()); @@ -311,22 +361,39 @@ TEST_F(MasterMetricsTest, CalcCacheStatsTest) { ASSERT_TRUE(put_start_result1.has_value()); auto put_end_result1 = service_.PutEnd(client_id, key, ReplicaType::MEMORY); ASSERT_TRUE(put_end_result1.has_value()); - stats_dict = metrics.calculate_cache_stats(); + auto stats_dict = metrics.calculate_cache_stats(); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::MEMORY_TOTAL], 3); + expect_aliases(stats_dict); + expect_reuse_ratios(stats_dict); + ASSERT_EQ(stats_dict[CacheHitStat::MEMORY_HITS], base_memory_hits); + ASSERT_EQ(stats_dict[CacheHitStat::MEMORY_TOTAL], base_memory_total + 1); auto get_replica_result = service_.GetReplicaList(key); + ASSERT_TRUE(get_replica_result.has_value()); stats_dict = metrics.calculate_cache_stats(); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::MEMORY_HITS], 2); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::SSD_HITS], 0); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::MEMORY_TOTAL], 3); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::SSD_TOTAL], 0); - ASSERT_NEAR(stats_dict[MasterMetricManager::CacheHitStat::MEMORY_HIT_RATE], - 0.67, 0.01); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::SSD_HIT_RATE], 0); - ASSERT_NEAR(stats_dict[MasterMetricManager::CacheHitStat::OVERALL_HIT_RATE], - 0.67, 0.01); - ASSERT_EQ(stats_dict[MasterMetricManager::CacheHitStat::VALID_GET_RATE], 1); + expect_aliases(stats_dict); + expect_reuse_ratios(stats_dict); + ASSERT_EQ(stats_dict[CacheHitStat::MEMORY_HITS], base_memory_hits + 1); + ASSERT_EQ(stats_dict[CacheHitStat::MEMORY_TOTAL], base_memory_total + 1); + ASSERT_GE(stats_dict[CacheHitStat::VALID_GET_RATE], base_valid_get_rate); + ASSERT_LE(stats_dict[CacheHitStat::VALID_GET_RATE], 1.0); + + // This value is not a bounded hit ratio: hits are cumulative while cached + // objects are a current gauge. + // Keep fetching until cumulative hits exceed current cached objects. + const auto extra_gets = std::max( + 1, static_cast(stats_dict[CacheHitStat::MEMORY_TOTAL] - + stats_dict[CacheHitStat::MEMORY_HITS]) + + 1); + for (int64_t i = 0; i < extra_gets; ++i) { + get_replica_result = service_.GetReplicaList(key); + ASSERT_TRUE(get_replica_result.has_value()); + } + stats_dict = metrics.calculate_cache_stats(); + expect_aliases(stats_dict); + expect_reuse_ratios(stats_dict); + ASSERT_GT(stats_dict[CacheHitStat::MEMORY_HITS_PER_CURRENT_CACHED_OBJECT], + 1.0); std::this_thread::sleep_for( std::chrono::milliseconds(default_kv_lease_ttl)); -- 2.34.1 From 079353e4b78369088bc82c60becabd045bef72c6 Mon Sep 17 00:00:00 2001 From: Enigmo-x <1253115086@qq.com> Date: Fri, 29 May 2026 11:31:48 +0800 Subject: [PATCH 147/382] feat(store): route NoF replicas through put and get (#2247) Co-authored-by: Enigmo-x --- mooncake-store/src/client_service.cpp | 814 ++++++++++++++++++++------ 1 file changed, 632 insertions(+), 182 deletions(-) diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 30abd6af..68614fbf 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -8,13 +8,20 @@ #include #include #include +#include #include #include #include #include #include +#include +#ifdef USE_NOF +#include +#endif #include #include +#include +#include #include #include #include @@ -38,6 +45,214 @@ using gpu_staging::CopyDeviceToHost; using gpu_staging::IsDevicePointer; using gpu_staging::SetDevice; +namespace { + +#ifdef USE_NOF +std::optional GetConfiguredNumaSocketId() { + const char* raw_value = std::getenv("MC_STORE_NUMA_SOCKET_ID"); + if (!raw_value || raw_value[0] == '\0') { + return std::nullopt; + } + + char* end_ptr = nullptr; + errno = 0; + long parsed = std::strtol(raw_value, &end_ptr, 10); + if (errno != 0 || end_ptr == raw_value || + (end_ptr != nullptr && *end_ptr != '\0') || parsed < 0 || + parsed > std::numeric_limits::max()) { + LOG(WARNING) << "Invalid MC_STORE_NUMA_SOCKET_ID=" << raw_value + << ", falling back to auto-detect"; + return std::nullopt; + } + + return static_cast(parsed); +} + +int GetCurrentNumaSocketId() { + if (numa_available() < 0) { + return 0; + } + int cpu = sched_getcpu(); + if (cpu < 0) { + return 0; + } + int node = numa_node_of_cpu(cpu); + return node < 0 ? 0 : node; +} +#endif + +struct ContiguousSliceRange { + void* ptr = nullptr; + size_t size = 0; +}; + +std::optional GetContiguousSliceRange( + std::span slices) { + if (slices.empty()) { + return std::nullopt; + } + + uintptr_t expected_ptr = 0; + uintptr_t start_ptr = 0; + size_t total_size = 0; + for (size_t i = 0; i < slices.size(); ++i) { + const auto& slice = slices[i]; + if (slice.ptr == nullptr) { + return std::nullopt; + } + + const auto current_ptr = reinterpret_cast(slice.ptr); + if (i == 0) { + start_ptr = current_ptr; + expected_ptr = current_ptr; + } else if (current_ptr != expected_ptr) { + return std::nullopt; + } + + if (slice.size > std::numeric_limits::max() - total_size) { + return std::nullopt; + } + if (slice.size > std::numeric_limits::max() - current_ptr) { + return std::nullopt; + } + + total_size += slice.size; + expected_ptr = current_ptr + slice.size; + } + + if (total_size == 0) { + return std::nullopt; + } + + return ContiguousSliceRange{.ptr = reinterpret_cast(start_ptr), + .size = total_size}; +} + +struct ReplicaTransferSummary { + size_t allocated_memory_replicas = 0; + size_t allocated_nof_replicas = 0; + size_t successful_memory_transfers = 0; + size_t successful_nof_transfers = 0; + size_t failed_memory_transfers = 0; + size_t failed_nof_transfers = 0; + ErrorCode first_error = ErrorCode::OK; + + void RecordAllocatedReplica(const Replica::Descriptor& replica) { + if (replica.is_memory_replica()) { + ++allocated_memory_replicas; + } else if (replica.is_nof_replica()) { + ++allocated_nof_replicas; + } + } + + void RecordSuccess(ReplicaType replica_type) { + if (replica_type == ReplicaType::MEMORY) { + ++successful_memory_transfers; + } else if (replica_type == ReplicaType::NOF_SSD) { + ++successful_nof_transfers; + } + } + + void RecordFailure(ReplicaType replica_type, ErrorCode error) { + if (replica_type == ReplicaType::MEMORY) { + ++failed_memory_transfers; + } else if (replica_type == ReplicaType::NOF_SSD) { + ++failed_nof_transfers; + } + if (first_error == ErrorCode::OK) { + first_error = error; + } + } +}; + +bool HasExpectedReplicaAllocation(const ReplicateConfig& config, + const ReplicaTransferSummary& summary) { + if (config.nof_replica_num == 0) { + return summary.allocated_memory_replicas > 0; + } + if (DetermineReplicaWriteMode(config) == + ReplicaWriteMode::FLEXIBLE_DUAL_REPLICA) { + return summary.allocated_memory_replicas + + summary.allocated_nof_replicas > + 0; + } + return summary.allocated_memory_replicas == config.replica_num && + summary.allocated_nof_replicas == config.nof_replica_num; +} + +// success describes whether the overall put should succeed. Reliable modes +// require all allocated replicas to complete. Flexible dual-replica mode only +// requires one replica type to succeed, so success may be true while +// revoke_type is also set for the failed side. +struct FinalizeDecision { + std::optional end_type; + std::optional revoke_type; + bool success = false; + ErrorCode error = ErrorCode::OK; +}; + +FinalizeDecision DetermineFinalizeDecision( + const ReplicateConfig& config, const ReplicaTransferSummary& summary) { + const auto write_mode = DetermineReplicaWriteMode(config); + const bool allocation_satisfied = + HasExpectedReplicaAllocation(config, summary); + + if (write_mode != ReplicaWriteMode::FLEXIBLE_DUAL_REPLICA) { + const bool all_transfers_succeeded = + summary.successful_memory_transfers == + summary.allocated_memory_replicas && + summary.successful_nof_transfers == + summary.allocated_nof_replicas && + summary.failed_memory_transfers == 0 && + summary.failed_nof_transfers == 0; + if (allocation_satisfied && all_transfers_succeeded) { + return {.end_type = ReplicaType::ALL, + .revoke_type = std::nullopt, + .success = true, + .error = ErrorCode::OK}; + } + return {.end_type = std::nullopt, + .revoke_type = ReplicaType::ALL, + .success = false, + .error = allocation_satisfied + ? (summary.first_error == ErrorCode::OK + ? ErrorCode::TRANSFER_FAIL + : summary.first_error) + : ErrorCode::NO_AVAILABLE_HANDLE}; + } + + const bool memory_succeeded = summary.successful_memory_transfers > 0; + const bool nof_succeeded = summary.successful_nof_transfers > 0; + + if (memory_succeeded && nof_succeeded) { + return {.end_type = ReplicaType::ALL, + .revoke_type = std::nullopt, + .success = true, + .error = ErrorCode::OK}; + } + if (memory_succeeded) { + return {.end_type = ReplicaType::MEMORY, + .revoke_type = ReplicaType::NOF_SSD, + .success = true, + .error = ErrorCode::OK}; + } + if (nof_succeeded) { + return {.end_type = ReplicaType::NOF_SSD, + .revoke_type = ReplicaType::MEMORY, + .success = true, + .error = ErrorCode::OK}; + } + + return {.end_type = std::nullopt, + .revoke_type = ReplicaType::ALL, + .success = false, + .error = summary.first_error == ErrorCode::OK + ? ErrorCode::NO_AVAILABLE_HANDLE + : summary.first_error}; +} + +} // namespace + [[nodiscard]] size_t CalculateSliceSize(const std::vector& slices) { size_t slice_size = 0; for (const auto& slice : slices) { @@ -582,9 +797,17 @@ void Client::InitTransferSubmitter() { // Initialize TransferSubmitter after transfer engine is ready // Keep using logical local_hostname for name-based behaviors; endpoint is // used separately where needed. +#ifdef USE_NOF + int numa_socket_id = + GetConfiguredNumaSocketId().value_or(GetCurrentNumaSocketId()); + transfer_submitter_ = std::make_unique( + *transfer_engine_, storage_backend_, local_hostname_, + metrics_ ? &metrics_->transfer_metric : nullptr, numa_socket_id); +#else transfer_submitter_ = std::make_unique( *transfer_engine_, storage_backend_, local_hostname_, metrics_ ? &metrics_->transfer_metric : nullptr); +#endif } std::optional> Client::Create( @@ -1112,8 +1335,21 @@ std::vector> Client::BatchGet( } // Submit transfer operation asynchronously - auto future = transfer_submitter_->submit(replica, slices_it->second, - TransferRequest::READ); + std::optional future; + if (replica.is_nof_replica()) { + auto contiguous_range = GetContiguousSliceRange(slices_it->second); + if (!contiguous_range.has_value()) { + LOG(ERROR) << "NoF transfer requires contiguous slices"; + results[i] = tl::unexpected(ErrorCode::INVALID_PARAMS); + continue; + } + future = transfer_submitter_->submit( + replica, slices_it->second, TransferRequest::READ, + contiguous_range->ptr, contiguous_range->size); + } else { + future = transfer_submitter_->submit(replica, slices_it->second, + TransferRequest::READ); + } if (!future) { // Release cache block if submit failed if (hot_cache_ && cache_used) { @@ -1247,6 +1483,11 @@ tl::expected Client::Put(const ObjectKey& key, return tl::unexpected(err); } + ReplicaTransferSummary transfer_summary; + for (const auto& replica : start_result.value()) { + transfer_summary.RecordAllocatedReplica(replica); + } + // Record Put transfer latency (all replicas) auto t0_put = std::chrono::steady_clock::now(); @@ -1266,19 +1507,17 @@ tl::expected Client::Put(const ObjectKey& key, } for (const auto& replica : start_result.value()) { - if (replica.is_memory_replica()) { + if (replica.is_memory_replica() || replica.is_nof_replica()) { // Transfer data using allocated handles from all replicas + const auto replica_type = replica.is_memory_replica() + ? ReplicaType::MEMORY + : ReplicaType::NOF_SSD; ErrorCode transfer_err = TransferWrite(replica, slices); if (transfer_err != ErrorCode::OK) { - // Revoke put operation - auto revoke_result = - master_client_.PutRevoke(key, ReplicaType::MEMORY); - if (!revoke_result) { - LOG(ERROR) << "Failed to revoke put operation"; - return tl::unexpected(revoke_result.error()); - } - return tl::unexpected(transfer_err); + transfer_summary.RecordFailure(replica_type, transfer_err); + continue; } + transfer_summary.RecordSuccess(replica_type); } } @@ -1289,12 +1528,30 @@ tl::expected Client::Put(const ObjectKey& key, metrics_->transfer_metric.put_latency_us.observe(us_put); } - // End put operation - auto end_result = master_client_.PutEnd(key, ReplicaType::MEMORY); - if (!end_result) { - ErrorCode err = end_result.error(); - LOG(ERROR) << "Failed to end put operation: " << err; - return tl::unexpected(err); + const auto finalize_decision = + DetermineFinalizeDecision(config, transfer_summary); + + if (finalize_decision.end_type.has_value()) { + auto end_result = + master_client_.PutEnd(key, *finalize_decision.end_type); + if (!end_result) { + ErrorCode err = end_result.error(); + LOG(ERROR) << "Failed to end put operation: " << err; + return tl::unexpected(err); + } + } + + if (finalize_decision.revoke_type.has_value()) { + auto revoke_result = + master_client_.PutRevoke(key, *finalize_decision.revoke_type); + if (!revoke_result) { + LOG(ERROR) << "Failed to revoke put operation"; + return tl::unexpected(revoke_result.error()); + } + } + + if (!finalize_decision.success) { + return tl::unexpected(finalize_decision.error); } return {}; @@ -1421,23 +1678,34 @@ enum class PutOperationState { class PutOperation { public: + struct PendingTransferRecord { + ReplicaType replica_type; + TransferFuture future; + + PendingTransferRecord(ReplicaType type, + TransferFuture&& transfer_future) + : replica_type(type), future(std::move(transfer_future)) {} + }; + PutOperation(std::string_view k, const std::vector& s) : key(k), slices(s) { - value_length = CalculateSliceSize(slices); // Initialize with a pending error state to ensure result is always set result = tl::unexpected(ErrorCode::INTERNAL_ERROR); } std::string key; std::vector slices; - size_t value_length; std::vector> batched_slices; // Enhanced state tracking PutOperationState state = PutOperationState::PENDING; tl::expected result; std::vector replicas; - std::vector pending_transfers; + std::vector pending_transfers; + + size_t requested_memory_replicas = 0; + size_t requested_nof_replicas = 0; + ReplicaTransferSummary transfer_summary; // Error context for debugging std::optional failure_context; @@ -1449,6 +1717,15 @@ class PutOperation { failure_context.reset(); } + void SetTerminalError(ErrorCode error, PutOperationState terminal_state, + const std::string& context = "") { + state = terminal_state; + result = tl::unexpected(error); + if (!context.empty()) { + failure_context = context; + } + } + void SetError(ErrorCode error, const std::string& context = "") { result = tl::unexpected(error); if (!context.empty()) { @@ -1466,6 +1743,36 @@ class PutOperation { } } + void AppendFailureContext(const std::string& context) { + if (context.empty()) { + return; + } + if (!failure_context.has_value()) { + failure_context = context; + return; + } + failure_context = *failure_context + "; " + context; + } + + void InitializeRequestedReplicas(const ReplicateConfig& config) { + requested_memory_replicas = config.replica_num; + requested_nof_replicas = config.nof_replica_num; + } + + ReplicateConfig ToReplicateConfig() const { + ReplicateConfig config; + config.replica_num = requested_memory_replicas; + config.nof_replica_num = requested_nof_replicas; + return config; + } + + void RecordAllocatedReplicas() { + transfer_summary = ReplicaTransferSummary{}; + for (const auto& replica : replicas) { + transfer_summary.RecordAllocatedReplica(replica); + } + } + bool IsResolved() const { return state != PutOperationState::PENDING; } bool IsSuccessful() const { @@ -1519,11 +1826,22 @@ void Client::StartBatchPut(std::vector& ops, // Process individual responses with robust error handling for (size_t i = 0; i < ops.size(); ++i) { + ops[i].InitializeRequestedReplicas(config); if (!start_responses[i]) { - ops[i].SetError(start_responses[i].error(), - "Master failed to start put operation"); + ops[i].SetTerminalError(start_responses[i].error(), + PutOperationState::MASTER_FAILED, + "Master failed to start put operation"); } else { ops[i].replicas = start_responses[i].value(); + ops[i].RecordAllocatedReplicas(); + if (!HasExpectedReplicaAllocation(config, + ops[i].transfer_summary)) { + ops[i].SetTerminalError(ErrorCode::NO_AVAILABLE_HANDLE, + PutOperationState::MASTER_FAILED, + "Allocated replicas do not satisfy " + "requested replica policy"); + continue; + } // Operation continues to next stage - result remains INTERNAL_ERROR // until fully successful VLOG(1) << "Successfully started put for key " << ops[i].key @@ -1582,8 +1900,9 @@ void Client::SubmitTransfers(std::vector& ops) { if (!transfer_submitter_) { LOG(ERROR) << "TransferSubmitter not initialized"; for (auto& op : ops) { - op.SetError(ErrorCode::INVALID_PARAMS, - "TransferSubmitter not initialized"); + op.SetTerminalError(ErrorCode::INVALID_PARAMS, + PutOperationState::TRANSFER_FAILED, + "TransferSubmitter not initialized"); } return; } @@ -1596,14 +1915,12 @@ void Client::SubmitTransfers(std::vector& ops) { // Skip operations that don't have replicas (failed in StartBatchPut) if (op.replicas.empty()) { - op.SetError(ErrorCode::INTERNAL_ERROR, - "No replicas available for transfer"); + op.SetTerminalError(ErrorCode::INTERNAL_ERROR, + PutOperationState::MASTER_FAILED, + "No replicas available for transfer"); continue; } - bool all_transfers_submitted = true; - std::string failure_context; - // We must deal with disk replica first, then the disk putrevoke/putend // can be called surely if (storage_backend_) { @@ -1621,31 +1938,48 @@ void Client::SubmitTransfers(std::vector& ops) { for (size_t replica_idx = 0; replica_idx < op.replicas.size(); ++replica_idx) { const auto& replica = op.replicas[replica_idx]; - if (replica.is_memory_replica()) { - auto submit_result = transfer_submitter_->submit( - replica, op.slices, TransferRequest::WRITE); + if (replica.is_memory_replica() || replica.is_nof_replica()) { + const auto replica_type = replica.is_memory_replica() + ? ReplicaType::MEMORY + : ReplicaType::NOF_SSD; + std::optional submit_result; + if (replica.is_nof_replica()) { + auto contiguous_range = GetContiguousSliceRange(op.slices); + if (!contiguous_range.has_value()) { + std::string failure_context = + "NoF transfer requires contiguous slices for " + "replica " + + std::to_string(replica_idx); + op.transfer_summary.RecordFailure( + replica_type, ErrorCode::INVALID_PARAMS); + op.AppendFailureContext(failure_context); + continue; + } + submit_result = transfer_submitter_->submit( + replica, op.slices, TransferRequest::WRITE, + contiguous_range->ptr, contiguous_range->size); + } else { + submit_result = transfer_submitter_->submit( + replica, op.slices, TransferRequest::WRITE); + } if (!submit_result) { - failure_context = "Failed to submit transfer for replica " + - std::to_string(replica_idx); - all_transfers_submitted = false; - break; + std::string failure_context = + "Failed to submit transfer for replica " + + std::to_string(replica_idx); + op.transfer_summary.RecordFailure(replica_type, + ErrorCode::TRANSFER_FAIL); + op.AppendFailureContext(failure_context); + continue; } op.pending_transfers.emplace_back( - std::move(submit_result.value())); + replica_type, std::move(submit_result.value())); } } - if (!all_transfers_submitted) { - LOG(ERROR) << "Transfer submission failed for key " << op.key - << ": " << failure_context; - op.SetError(ErrorCode::TRANSFER_FAIL, failure_context); - op.pending_transfers.clear(); - } else { - VLOG(1) << "Successfully submitted " << op.pending_transfers.size() - << " transfers for key " << op.key; - } + VLOG(1) << "Submitted " << op.pending_transfers.size() + << " transfers for key " << op.key; } } @@ -1656,154 +1990,223 @@ void Client::WaitForTransfers(std::vector& ops) { continue; } - // Skip operations with no pending transfers (failed in SubmitTransfers) - if (op.pending_transfers.empty()) { - op.SetError(ErrorCode::INTERNAL_ERROR, - "No pending transfers to wait for"); - continue; - } - - bool all_transfers_succeeded = true; - ErrorCode first_error = ErrorCode::OK; - size_t failed_transfer_idx = 0; - for (size_t i = 0; i < op.pending_transfers.size(); ++i) { - ErrorCode transfer_result = op.pending_transfers[i].get(); + auto& pending_transfer = op.pending_transfers[i]; + ErrorCode transfer_result = pending_transfer.future.get(); if (transfer_result != ErrorCode::OK) { - if (all_transfers_succeeded) { - // Record the first error for reporting - first_error = transfer_result; - failed_transfer_idx = i; - all_transfers_succeeded = false; - } - // Continue waiting for all transfers to avoid resource leaks + op.transfer_summary.RecordFailure(pending_transfer.replica_type, + transfer_result); + std::string error_context = + "Transfer " + std::to_string(i) + " failed"; + op.AppendFailureContext(error_context); + } else { + op.transfer_summary.RecordSuccess( + pending_transfer.replica_type); } } - if (all_transfers_succeeded) { - VLOG(1) << "All transfers completed successfully for key " - << op.key; - // Transfer phase successful - continue to finalization - // Note: Don't mark as SUCCESS yet, need to complete finalization - } else { - std::string error_context = - "Transfer " + std::to_string(failed_transfer_idx) + " failed"; - LOG(ERROR) << "Transfer failed for key " << op.key << ": " - << toString(first_error) << " (" << error_context << ")"; - op.SetError(first_error, error_context); - } + VLOG(1) << "Transfers finished for key " << op.key << ", success(mem=" + << op.transfer_summary.successful_memory_transfers + << ", nof=" << op.transfer_summary.successful_nof_transfers + << "), fail(mem=" << op.transfer_summary.failed_memory_transfers + << ", nof=" << op.transfer_summary.failed_nof_transfers << ")"; } } void Client::FinalizeBatchPut(std::vector& ops) { - // For each operation, - // If transfers completed successfully, we need to call BatchPutEnd - // If the operation failed but has allocated replicas, we need to call - // BatchPutRevoke + struct BatchFinalizeGroup { + std::vector keys; + std::vector indices; + }; - std::vector successful_keys; - std::vector successful_indices; - std::vector failed_keys; - std::vector failed_indices; + BatchFinalizeGroup end_all_group; + BatchFinalizeGroup end_memory_group; + BatchFinalizeGroup end_nof_group; + BatchFinalizeGroup revoke_all_group; + BatchFinalizeGroup revoke_memory_group; + BatchFinalizeGroup revoke_nof_group; - // Reserve space to avoid reallocations - successful_keys.reserve(ops.size()); - successful_indices.reserve(ops.size()); - failed_keys.reserve(ops.size()); - failed_indices.reserve(ops.size()); + std::vector pending_finalize_actions(ops.size(), 0); + std::vector should_succeed(ops.size(), false); + std::vector terminal_errors(ops.size(), ErrorCode::OK); + std::vector> finalize_rpc_errors(ops.size()); + + auto add_group_entry = [](BatchFinalizeGroup& group, const std::string& key, + size_t index) { + group.keys.emplace_back(key); + group.indices.emplace_back(index); + }; + + auto add_finalize_action = + [&](const std::optional& replica_type, bool is_end, + const std::string& key, size_t index) { + if (!replica_type.has_value()) { + return; + } + switch (*replica_type) { + case ReplicaType::ALL: + add_group_entry(is_end ? end_all_group : revoke_all_group, + key, index); + ++pending_finalize_actions[index]; + break; + case ReplicaType::MEMORY: + add_group_entry( + is_end ? end_memory_group : revoke_memory_group, key, + index); + ++pending_finalize_actions[index]; + break; + case ReplicaType::NOF_SSD: + add_group_entry(is_end ? end_nof_group : revoke_nof_group, + key, index); + ++pending_finalize_actions[index]; + break; + default: + LOG(ERROR) << "Unexpected replica type in batch finalize: " + << *replica_type; + finalize_rpc_errors[index] = ErrorCode::INVALID_PARAMS; + break; + } + }; + + auto complete_finalize_action = [&](size_t index) { + if (pending_finalize_actions[index] > 0) { + --pending_finalize_actions[index]; + } + }; for (size_t i = 0; i < ops.size(); ++i) { auto& op = ops[i]; - - // Check if operation completed transfers successfully and needs - // finalization - if (!op.IsResolved() && !op.replicas.empty() && - !op.pending_transfers.empty()) { - // Transfers completed, needs BatchPutEnd - successful_keys.emplace_back(op.key); - successful_indices.emplace_back(i); - } else if (op.state != PutOperationState::PENDING && - !op.replicas.empty()) { - // Operation failed but has allocated replicas, needs BatchPutRevoke - failed_keys.emplace_back(op.key); - failed_indices.emplace_back(i); + if (op.IsResolved()) { + if (!op.IsSuccessful() && !op.replicas.empty()) { + terminal_errors[i] = op.result.has_value() + ? ErrorCode::INTERNAL_ERROR + : op.result.error(); + add_finalize_action(ReplicaType::ALL, false, op.key, i); + } + continue; } - // Operations without replicas (early failures) don't need finalization + if (op.replicas.empty()) { + op.SetTerminalError(ErrorCode::INTERNAL_ERROR, + PutOperationState::MASTER_FAILED, + "Operation has no replicas to finalize"); + continue; + } + + const auto finalize_decision = DetermineFinalizeDecision( + op.ToReplicateConfig(), op.transfer_summary); + should_succeed[i] = finalize_decision.success; + terminal_errors[i] = finalize_decision.error; + add_finalize_action(finalize_decision.end_type, true, op.key, i); + add_finalize_action(finalize_decision.revoke_type, false, op.key, i); } - // Process successful operations - if (!successful_keys.empty()) { - auto end_responses = master_client_.BatchPutEnd(successful_keys); - if (end_responses.size() != successful_keys.size()) { - LOG(ERROR) << "BatchPutEnd response size mismatch: expected " - << successful_keys.size() << ", got " - << end_responses.size(); - for (size_t idx : successful_indices) { - ops[idx].SetError(ErrorCode::RPC_FAIL, - "BatchPutEnd response size mismatch"); - } - } else { - // Process individual responses - for (size_t i = 0; i < end_responses.size(); ++i) { - const size_t op_idx = successful_indices[i]; - if (!end_responses[i]) { - LOG(ERROR) << "Failed to finalize put for key " - << successful_keys[i] << ": " - << toString(end_responses[i].error()); - ops[op_idx].SetError(end_responses[i].error(), - "BatchPutEnd failed"); - } else { - // Operation fully successful - ops[op_idx].SetSuccess(); - VLOG(1) << "Successfully completed put for key " - << successful_keys[i]; - } - } + auto process_end_group = [&](BatchFinalizeGroup& group, + ReplicaType replica_type) { + if (group.keys.empty()) { + return; } - } + auto responses = master_client_.BatchPutEnd(group.keys, replica_type); + if (responses.size() != group.keys.size()) { + for (size_t idx : group.indices) { + finalize_rpc_errors[idx] = ErrorCode::RPC_FAIL; + complete_finalize_action(idx); + } + return; + } + for (size_t i = 0; i < responses.size(); ++i) { + const size_t op_idx = group.indices[i]; + if (!responses[i]) { + finalize_rpc_errors[op_idx] = responses[i].error(); + LOG(ERROR) << "Failed to BatchPutEnd key " << group.keys[i] + << ": " << toString(responses[i].error()); + complete_finalize_action(op_idx); + continue; + } + complete_finalize_action(op_idx); + } + }; - // Process failed operations that need cleanup - if (!failed_keys.empty()) { - auto revoke_responses = master_client_.BatchPutRevoke(failed_keys); - if (revoke_responses.size() != failed_keys.size()) { - LOG(ERROR) << "BatchPutRevoke response size mismatch: expected " - << failed_keys.size() << ", got " - << revoke_responses.size(); - // Mark all failed operations with revoke RPC failure - for (size_t idx : failed_indices) { - ops[idx].SetError(ErrorCode::RPC_FAIL, - "BatchPutRevoke response size mismatch"); - } - } else { - // Process individual revoke responses - for (size_t i = 0; i < revoke_responses.size(); ++i) { - const size_t op_idx = failed_indices[i]; - if (!revoke_responses[i]) { - LOG(ERROR) - << "Failed to revoke put for key " << failed_keys[i] - << ": " << toString(revoke_responses[i].error()); - // Preserve original error but note revoke failure in - // context - std::string original_context = - ops[op_idx].failure_context.value_or("unknown error"); - ops[op_idx].failure_context = - original_context + "; revoke also failed"; - } else { - LOG(INFO) << "Successfully revoked failed put for key " - << failed_keys[i]; - } - } + auto process_revoke_group = [&](BatchFinalizeGroup& group, + ReplicaType replica_type) { + if (group.keys.empty()) { + return; } - } + auto responses = + master_client_.BatchPutRevoke(group.keys, replica_type); + if (responses.size() != group.keys.size()) { + for (size_t idx : group.indices) { + finalize_rpc_errors[idx] = ErrorCode::RPC_FAIL; + complete_finalize_action(idx); + } + return; + } + for (size_t i = 0; i < responses.size(); ++i) { + const size_t op_idx = group.indices[i]; + if (!responses[i]) { + finalize_rpc_errors[op_idx] = responses[i].error(); + LOG(ERROR) << "Failed to BatchPutRevoke key " << group.keys[i] + << ": " << toString(responses[i].error()); + complete_finalize_action(op_idx); + continue; + } + complete_finalize_action(op_idx); + } + }; - // Ensure all operations have definitive results - for (auto& op : ops) { - if (!op.IsResolved()) { - op.SetError(ErrorCode::INTERNAL_ERROR, - "Operation not resolved after finalization"); - LOG(ERROR) << "Operation for key " << op.key - << " was not properly resolved"; + process_end_group(end_all_group, ReplicaType::ALL); + process_end_group(end_memory_group, ReplicaType::MEMORY); + process_end_group(end_nof_group, ReplicaType::NOF_SSD); + process_revoke_group(revoke_all_group, ReplicaType::ALL); + process_revoke_group(revoke_memory_group, ReplicaType::MEMORY); + process_revoke_group(revoke_nof_group, ReplicaType::NOF_SSD); + + auto append_finalize_error_context = [&](PutOperation& op, size_t index) { + if (finalize_rpc_errors[index].has_value()) { + op.AppendFailureContext("Batch finalization RPC failed: " + + toString(*finalize_rpc_errors[index])); } + if (pending_finalize_actions[index] != 0) { + op.AppendFailureContext( + "Operation has unfinished finalize actions"); + } + }; + + for (size_t i = 0; i < ops.size(); ++i) { + auto& op = ops[i]; + if (op.IsResolved()) { + if (!op.IsSuccessful()) { + append_finalize_error_context(op, i); + } + continue; + } + if (finalize_rpc_errors[i].has_value() || + pending_finalize_actions[i] != 0) { + if (!should_succeed[i] && terminal_errors[i] != ErrorCode::OK) { + append_finalize_error_context(op, i); + op.SetTerminalError( + terminal_errors[i], PutOperationState::TRANSFER_FAILED, + op.failure_context.value_or( + "Replica transfer failed before finalize")); + } else if (finalize_rpc_errors[i].has_value()) { + op.SetTerminalError(*finalize_rpc_errors[i], + PutOperationState::FINALIZE_FAILED, + "Batch finalization RPC failed"); + } else { + op.SetTerminalError( + ErrorCode::INTERNAL_ERROR, + PutOperationState::FINALIZE_FAILED, + "Operation has unfinished finalize actions"); + } + continue; + } + if (should_succeed[i]) { + op.SetSuccess(); + continue; + } + op.SetTerminalError(terminal_errors[i], + PutOperationState::TRANSFER_FAILED, + op.failure_context.value_or( + "Replica transfer failed before finalize")); } } @@ -1984,6 +2387,7 @@ std::vector> Client::BatchPutWhenPreferSameNode( merged_ops.emplace_back(op.key, op.slices); auto& merged_op = merged_ops.back(); merged_op.replicas = op.replicas; + merged_op.transfer_summary.allocated_memory_replicas = 1; auto submit_result = transfer_submitter_->submit_batch( op.replicas, op.batched_slices, TransferRequest::WRITE); if (!submit_result) { @@ -1991,12 +2395,14 @@ std::vector> Client::BatchPutWhenPreferSameNode( all_transfers_submitted = false; } else { merged_op.pending_transfers.emplace_back( - std::move(submit_result.value())); + ReplicaType::MEMORY, std::move(submit_result.value())); } if (!all_transfers_submitted) { LOG(ERROR) << "Transfer submission failed for key " << op.key << ": " << failure_context; - merged_op.SetError(ErrorCode::TRANSFER_FAIL, failure_context); + merged_op.transfer_summary.RecordFailure(ReplicaType::MEMORY, + ErrorCode::TRANSFER_FAIL); + merged_op.failure_context = failure_context; merged_op.pending_transfers.clear(); } else { VLOG(1) << "Successfully submitted " @@ -2009,7 +2415,8 @@ std::vector> Client::BatchPutWhenPreferSameNode( auto& memory_descriptor = op.replicas[0].get_memory_descriptor(); auto& buffer_descriptor = memory_descriptor.buffer_descriptor; auto seg = buffer_descriptor.transport_endpoint_; - seg_to_ops.at(seg).state = op.state; + seg_to_ops.at(seg).transfer_summary = op.transfer_summary; + seg_to_ops.at(seg).failure_context = op.failure_context; } for (auto& op : ops) { if (op.IsResolved()) { @@ -2018,10 +2425,16 @@ std::vector> Client::BatchPutWhenPreferSameNode( auto& memory_descriptor = op.replicas[0].get_memory_descriptor(); auto& buffer_descriptor = memory_descriptor.buffer_descriptor; auto seg = buffer_descriptor.transport_endpoint_; - op.state = seg_to_ops.at(seg).state; - auto state = std::make_shared(); - auto future = TransferFuture(state); - op.pending_transfers.emplace_back(std::move(future)); + op.transfer_summary.successful_memory_transfers = + seg_to_ops.at(seg).transfer_summary.successful_memory_transfers > 0 + ? 1 + : 0; + op.transfer_summary.failed_memory_transfers = + seg_to_ops.at(seg).transfer_summary.failed_memory_transfers > 0 ? 1 + : 0; + op.transfer_summary.first_error = + seg_to_ops.at(seg).transfer_summary.first_error; + op.failure_context = seg_to_ops.at(seg).failure_context; } auto us = std::chrono::duration_cast( std::chrono::steady_clock::now() - t0) @@ -2043,6 +2456,12 @@ std::vector> Client::BatchPut( } std::vector ops = CreatePutOperations(keys, batched_slices); if (client_cfg.prefer_alloc_in_same_node) { + if (client_cfg.nof_replica_num > 0) { + LOG(ERROR) << "prefer_alloc_in_same_node is not supported with " + "NoF replicas"; + return std::vector>( + keys.size(), tl::unexpected(ErrorCode::INVALID_PARAMS)); + } if (client_cfg.replica_num != 1) { LOG(ERROR) << "prefer_alloc_in_same_node is not supported with " "replica_num != 1"; @@ -2804,8 +3223,20 @@ ErrorCode Client::TransferData(const Replica::Descriptor& replica_descriptor, return ErrorCode::INVALID_PARAMS; } - auto future = - transfer_submitter_->submit(replica_descriptor, slices, op_code); + std::optional future; + if (replica_descriptor.is_nof_replica()) { + auto contiguous_range = GetContiguousSliceRange(slices); + if (!contiguous_range.has_value()) { + LOG(ERROR) << "NoF transfer requires contiguous slices"; + return ErrorCode::INVALID_PARAMS; + } + future = transfer_submitter_->submit(replica_descriptor, slices, + op_code, contiguous_range->ptr, + contiguous_range->size); + } else { + future = + transfer_submitter_->submit(replica_descriptor, slices, op_code); + } if (!future) { LOG(ERROR) << "Failed to submit transfer operation"; return ErrorCode::TRANSFER_FAIL; @@ -2847,9 +3278,15 @@ ErrorCode Client::TransferRead(const Replica::Descriptor& replica_descriptor, if (replica_descriptor.is_memory_replica()) { auto& mem_desc = replica_descriptor.get_memory_descriptor(); total_size = mem_desc.buffer_descriptor.size_; - } else { + } else if (replica_descriptor.is_nof_replica()) { + auto& nof_desc = replica_descriptor.get_nof_descriptor(); + total_size = nof_desc.buffer_descriptor.size_; + } else if (replica_descriptor.is_disk_replica()) { auto& disk_desc = replica_descriptor.get_disk_descriptor(); total_size = disk_desc.object_size; + } else if (replica_descriptor.is_local_disk_replica()) { + auto& disk_desc = replica_descriptor.get_local_disk_descriptor(); + total_size = disk_desc.object_size; } size_t slices_size = CalculateSliceSize(slices); @@ -3251,6 +3688,7 @@ tl::expected Client::GetPreferredReplica( } } + // Prefer local MEMORY replicas first for (const auto& rep : replica_list) { if (rep.is_memory_replica()) { const auto& mem_desc = rep.get_memory_descriptor(); @@ -3262,6 +3700,18 @@ tl::expected Client::GetPreferredReplica( } } + // Then prefer local NOF_SSD replicas + for (const auto& rep : replica_list) { + if (rep.is_nof_replica()) { + const auto& nof_desc = rep.get_nof_descriptor(); + const std::string& endpoint = + nof_desc.buffer_descriptor.transport_endpoint_; + if (local_endpoints.count(endpoint)) { + return rep; + } + } + } + return replica_list[0]; } -- 2.34.1 From c39117647762a09d9f2d4426e72b4cc419b75461 Mon Sep 17 00:00:00 2001 From: Chizheng Fang <93508110+fcczzz@users.noreply.github.com> Date: Fri, 29 May 2026 17:35:22 +0800 Subject: [PATCH 148/382] [Doc] Split LMCache vLLM MP and non-MP guides (#2268) * [Doc] Split LMCache vLLM MP and non-MP guides * Simplified relative link to lmcache-integration.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Simplified relative link to lmcache-integration.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [Doc] clarify LMCache Mooncake build requirements --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- .../examples/lmcache-integration.md | 2 + .../examples/vllm-integration/index.md | 2 +- .../vllmv1-lmcache-integration.md | 262 +++++++++--------- .../vllmv1-lmcache-mp-integration.md | 193 +++++++++++++ 4 files changed, 332 insertions(+), 127 deletions(-) create mode 100644 docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-mp-integration.md diff --git a/docs/source/getting_started/examples/lmcache-integration.md b/docs/source/getting_started/examples/lmcache-integration.md index 7dbcef46..b15da526 100644 --- a/docs/source/getting_started/examples/lmcache-integration.md +++ b/docs/source/getting_started/examples/lmcache-integration.md @@ -25,6 +25,8 @@ For a complete deployment guide with step-by-step instructions, see: 👉 **[vLLM V1 Disaggregated Serving with Mooncake Store and LMCache](vllm-integration/vllmv1-lmcache-integration.md)** +👉 **[vLLM V1 Disaggregated Serving with Mooncake Store and LMCache \[MP\]](vllm-integration/vllmv1-lmcache-mp-integration.md)** + ## Performance Benchmarking and Results To illustrate the benefits of this collaboration, a comprehensive performance evaluation of the integrated vLLM, LMCache, and Mooncake Store was conducted. The selected experimental conditions simulate realistic LLM deployment scenarios, specifically comparing the initial cold start with subsequent cache-hit performance to demonstrate the advantages of KVCache reuse. diff --git a/docs/source/getting_started/examples/vllm-integration/index.md b/docs/source/getting_started/examples/vllm-integration/index.md index 8ead4680..021b9748 100644 --- a/docs/source/getting_started/examples/vllm-integration/index.md +++ b/docs/source/getting_started/examples/vllm-integration/index.md @@ -5,9 +5,9 @@ vllm-mooncakestoreconnector vllmv1-lmcache-integration +vllmv1-lmcache-mp-integration vllm-integration-v0.2 vllm-integration-v0.3 vllm-integration-v1.0 :::: - diff --git a/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md b/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md index 3a9ee791..024bd760 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md +++ b/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-integration.md @@ -2,150 +2,163 @@ ## Overview -This guide shows a single-machine 1-prefill/1-decode deployment using vLLM V1, -LMCache's multiprocess server, and Mooncake Store as the LMCache L2 backend. +This guide shows a two-machine 1-prefill/1-decode deployment using vLLM V1, +LMCache's non-MP `LMCacheConnectorV1`, and Mooncake Store as LMCache's remote +storage backend. -In this setup, one machine runs Mooncake master, one LMCache MP server, the -disaggregated proxy, the prefiller vLLM instance, and the decoder vLLM -instance. The prefiller and decoder should use different GPUs. +LMCache supports both non-MP mode and MP mode with Mooncake Store. This page +covers the non-MP path, where each vLLM instance loads an LMCache YAML config +through `LMCACHE_CONFIG_FILE` and connects directly to Mooncake Store with +`remote_url: "mooncakestore://..."`. For the LMCache multiprocess server path, +see [vLLM V1 Disaggregated Serving with Mooncake Store and LMCache \[MP\]](vllmv1-lmcache-mp-integration.md). -This example uses `"metadata_server":"P2PHANDSHAKE"` for Mooncake transfer -metadata, so the Mooncake HTTP metadata server is not needed. If you switch to -HTTP metadata, remember that Mooncake's HTTP metadata server also defaults to -`8080`, which conflicts with LMCache's HTTP API on a single host. +The examples below use: + +- Machine A: Mooncake master and vLLM decoder +- Machine B: vLLM prefiller +- Mooncake master RPC address: `{IP of Machine A}:50051` +- Mooncake HTTP metadata endpoint: `http://{IP of Machine A}:8080/metadata` +- RDMA device: `{RDMA device}` + +Replace these placeholders with the IP addresses, hostname, and RDMA device for +your environment. ## Prerequisites -Install Mooncake, vLLM, and LMCache on the machine. The example assumes an RDMA -deployment and uses: +Install Mooncake, vLLM, and LMCache on both machines. For installation details, +refer to the official documentation of each project: -- local host address: `{IP of Machine}` -- RDMA device: `{RDMA device}` -- LMCache checkout path: `/path/to/LMCache` - -Replace these values with the local hostname/IP, RDMA device, and LMCache checkout path for your environment. - -The `mooncake_store` MP L2 adapter requires LMCache's `lmcache_mooncake` C++ -extension. When building LMCache from source, enable Mooncake support, for -example: - -```bash -BUILD_MOONCAKE=1 \ -MOONCAKE_INCLUDE_DIR=/path/to/mooncake/include \ -MOONCAKE_LIB_DIR=/path/to/mooncake/lib \ -pip install -e /path/to/LMCache --verbose -``` +- [Mooncake build guide](../../build.md) +- [LMCache installation](https://docs.lmcache.ai/getting_started/installation.html) +- [vLLM installation](https://docs.vllm.ai/en/latest/getting_started/installation/) ## Deployment -### 1. Start Mooncake Master +### 1. Start Mooncake Master on Machine A ```bash mooncake_master -v=1 \ --rpc_port=50051 \ - --metrics_port=9003 + --metrics_port=9003 \ + --enable_http_metadata_server=true \ + --http_metadata_server_host=0.0.0.0 \ + --http_metadata_server_port=8080 ``` -### 2. Start the LMCache Multiprocess Server +### 2. Configure and Start the vLLM Decoder on Machine A -Start one LMCache MP server and configure Mooncake Store as the L2 adapter. +Modify the vLLM disaggregated prefill launcher to use a Mooncake-backed LMCache +config for the decoder. -```bash -lmcache server \ - --host 127.0.0.1 \ - --port 5555 \ - --http-host 127.0.0.1 \ - --http-port 8080 \ - --l1-size-gb 32 \ - --eviction-policy LRU \ - --no-l1-use-lazy \ - --l2-adapter '{ - "type": "mooncake_store", - "local_hostname": "{IP of Machine}", - "metadata_server": "P2PHANDSHAKE", - "protocol": "rdma", - "rdma_devices": "{RDMA device}", - "global_segment_size": "32212254720", - "local_buffer_size": "1073741824", - "master_server_addr": "127.0.0.1:50051" - }' +The decoder should continue to use `LMCacheConnectorV1` with `kv_role` set to +`kv_consumer`. + +```diff +diff --git a/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh b/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh +--- a/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh ++++ b/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh +@@ + elif [[ $1 == "decoder" ]]; then + # Decoder listens on port 8200 +- decode_config_file=$SCRIPT_DIR/configs/lmcache-decoder-config.yaml ++ decode_config_file=$SCRIPT_DIR/configs/mooncake-decoder-config.yaml ``` -### 3. Start the Disaggregated Proxy +Create `configs/mooncake-decoder-config.yaml`: -The proxy receives client requests, sends prefill requests to the prefiller, -sends decode requests to the decoder, and receives LMCache request telemetry -from the prefiller. +```yaml +chunk_size: 256 +remote_url: "mooncakestore://{IP of Machine A}:50051/" +remote_serde: "naive" +local_cpu: False +max_local_cpu_size: 100 + +extra_config: + local_hostname: "{IP of Machine A}" + metadata_server: "http://{IP of Machine A}:8080/metadata" + protocol: "rdma" + device_name: "{RDMA device}" + master_server_address: "{IP of Machine A}:50051" + global_segment_size: 32212254720 # 30GB + local_buffer_size: 1073741824 # 1GB + transfer_timeout: 1 + save_chunk_meta: False +``` + +Launch the decoder: ```bash -python /path/to/LMCache/examples/disagg_prefill_mp/disagg_proxy_server.py \ - --host 127.0.0.1 \ - --port 8000 \ - --prefiller-host 127.0.0.1 \ +bash disagg_vllm_launcher.sh decoder Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 +``` + +### 3. Configure and Start the vLLM Prefiller on Machine B + +Modify the launcher to use a Mooncake-backed LMCache config for the prefiller. + +The prefiller should continue to use `LMCacheConnectorV1` with `kv_role` set to +`kv_producer`. + +```diff +diff --git a/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh b/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh +--- a/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh ++++ b/examples/lmcache/disagg_prefill_lmcache_v1/disagg_vllm_launcher.sh +@@ + if [[ $1 == "prefiller" ]]; then + # Prefiller listens on port 8100 +- prefill_config_file=$SCRIPT_DIR/configs/lmcache-prefiller-config.yaml ++ prefill_config_file=$SCRIPT_DIR/configs/mooncake-prefiller-config.yaml +``` + +Create `configs/mooncake-prefiller-config.yaml`: + +```yaml +chunk_size: 256 +remote_url: "mooncakestore://{IP of Machine A}:50051/" +remote_serde: "naive" +local_cpu: False +max_local_cpu_size: 100 + +extra_config: + local_hostname: "{IP of Machine B}" + metadata_server: "http://{IP of Machine A}:8080/metadata" + protocol: "rdma" + device_name: "{RDMA device}" + master_server_address: "{IP of Machine A}:50051" + global_segment_size: 32212254720 # 30GB + local_buffer_size: 1073741824 # 1GB + transfer_timeout: 1 + save_chunk_meta: False +``` + +Launch the prefiller: + +```bash +bash disagg_vllm_launcher.sh prefiller Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 +``` + +### 4. Start the Disaggregated Proxy + +Use the LMCache [`disagg_proxy_server.py`](https://github.com/LMCache/LMCache/blob/dev/examples/disagg_prefill/disagg_proxy_server.py) to route requests between the prefiller and decoder. According to [LMCache/LMCache#1342](https://github.com/LMCache/LMCache/issues/1342), when using Mooncake Store as the backend, comment out the `wait_decode_kv_ready(...)` call in the proxy before starting it. + +```bash +python3 disagg_proxy_server.py \ + --host 0.0.0.0 \ + --port 9000 \ + --prefiller-host {IP of Machine B} \ --prefiller-port 8100 \ - --decoder-host 127.0.0.1 \ - --decoder-port 8200 \ - --telemetry-port 5768 + --decoder-host {IP of Machine A} \ + --decoder-port 8200 ``` -### 4. Start the vLLM Prefiller - -The prefiller reports request telemetry back to the proxy so the proxy knows -when KV cache storage has completed. - -```bash -CUDA_VISIBLE_DEVICES=0 \ -LMCACHE_REQUEST_TELEMETRY_TYPE=fastapi \ -LMCACHE_REQUEST_TELEMETRY_ENDPOINT=http://127.0.0.1:5768/api/v1/telemetry \ -vllm serve Qwen/Qwen3-4B \ - --host 127.0.0.1 \ - --port 8100 \ - --gpu-memory-utilization 0.8 \ - --no-enable-log-requests \ - --no-enable-prefix-caching \ - --kv-transfer-config '{ - "kv_connector": "LMCacheMPConnector", - "kv_role": "kv_both", - "kv_connector_extra_config": { - "lmcache.mp.host": "tcp://127.0.0.1", - "lmcache.mp.port": 5555 - } - }' -``` - -### 5. Start the vLLM Decoder - -The decoder connects to the same local LMCache MP server. It does not need -request telemetry environment variables; only the prefiller reports the "KV -cache is stored" event back to the proxy. - -```bash -CUDA_VISIBLE_DEVICES=1 \ -vllm serve Qwen/Qwen3-4B \ - --host 127.0.0.1 \ - --port 8200 \ - --gpu-memory-utilization 0.8 \ - --no-enable-log-requests \ - --no-enable-prefix-caching \ - --kv-transfer-config '{ - "kv_connector": "LMCacheMPConnector", - "kv_role": "kv_both", - "kv_connector_extra_config": { - "lmcache.mp.host": "tcp://127.0.0.1", - "lmcache.mp.port": 5555 - } - }' -``` - -### 6. Send a Test Request +### 5. Send a Test Request Send traffic to the proxy, not directly to either vLLM instance. ```bash -curl -N http://127.0.0.1:8000/v1/chat/completions \ +curl -N http://{Proxy IP}:9000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ - "model": "Qwen/Qwen3-4B", + "model": "Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4", "messages": [ { "role": "user", @@ -162,20 +175,17 @@ curl -N http://127.0.0.1:8000/v1/chat/completions \ When changing ports away from these defaults, update all dependent settings together: -- Mooncake master `--rpc_port` must match LMCache `master_server_addr`. -- The vLLM prefiller and decoder should connect to the local LMCache MP server - via `kv_connector_extra_config.lmcache.mp.host` and - `kv_connector_extra_config.lmcache.mp.port`. +- Mooncake master `--rpc_port` must match `remote_url` and + `extra_config.master_server_address`. +- `extra_config.metadata_server` must point to the Mooncake HTTP metadata + endpoint when HTTP metadata is used. +- Decoder and prefiller `device_name`, `protocol`, `global_segment_size`, and + `local_buffer_size` should be set for the local hardware and workload. - Proxy `--prefiller-port` and `--decoder-port` must match the two vLLM - `--port` values. -- `LMCACHE_REQUEST_TELEMETRY_ENDPOINT` on the prefiller must point to the proxy - telemetry endpoint. -- If `metadata_server` is changed from `P2PHANDSHAKE` to an HTTP metadata URL, - enable Mooncake HTTP metadata server and make sure its port does not conflict - with LMCache `--http-port`. + instance ports. ## Additional Resources -* [Mooncake x LMCache: Unite to Pioneer KVCache-Centric LLM Serving System](../../../getting_started/examples/lmcache-integration.md) -* [LMCache MP `mooncake_store` L2 adapter](https://docs.lmcache.ai/mp/l2_storage.html#mooncake-store-mooncake-store-native-connector) -* [LMCache multiprocess disaggregated prefill example](https://github.com/LMCache/LMCache/tree/dev/examples/disagg_prefill_mp) +* [Mooncake x LMCache: Unite to Pioneer KVCache-Centric LLM Serving System](../lmcache-integration.md) +* [Using Mooncake in LMCache](https://docs.lmcache.ai/kv_cache/storage_backends/mooncake.html) +* [Using LMCache in vLLM](https://github.com/vllm-project/vllm/tree/main/examples/others/lmcache) diff --git a/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-mp-integration.md b/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-mp-integration.md new file mode 100644 index 00000000..d728753a --- /dev/null +++ b/docs/source/getting_started/examples/vllm-integration/vllmv1-lmcache-mp-integration.md @@ -0,0 +1,193 @@ +# vLLM V1 Disaggregated Serving with Mooncake Store and LMCache [MP] + +## Overview + +This guide shows a single-machine 1-prefill/1-decode deployment using vLLM V1, +LMCache's multiprocess server, and Mooncake Store as the LMCache L2 backend. + +LMCache supports both non-MP mode and MP mode with Mooncake Store. This page +covers the MP path, where vLLM instances connect to an LMCache server through +`LMCacheMPConnector`, and the LMCache server connects to Mooncake Store through +the `mooncake_store` L2 adapter. For the non-MP `LMCacheConnectorV1` path, see +[vLLM V1 Disaggregated Serving with Mooncake Store and LMCache](vllmv1-lmcache-integration.md). + +In this setup, one machine runs Mooncake master, one LMCache MP server, the +disaggregated proxy, the prefiller vLLM instance, and the decoder vLLM +instance. The prefiller and decoder should use different GPUs. + +This example uses `"metadata_server":"P2PHANDSHAKE"` for Mooncake transfer +metadata, so the Mooncake HTTP metadata server is not needed. If you switch to +HTTP metadata, remember that Mooncake's HTTP metadata server also defaults to +`8080`, which conflicts with LMCache's HTTP API on a single host. + +## Prerequisites + +Install Mooncake, vLLM, and LMCache on the machine. The example assumes an RDMA +deployment and uses: + +- local host address: `{IP of Machine}` +- RDMA device: `{RDMA device}` +- LMCache checkout path: `/path/to/LMCache` + +Replace these values with the local hostname/IP, RDMA device, and LMCache +checkout path for your environment. + +LMCache requirement: This example requires LMCache v0.4.5 or later. LMCache +must also be built from source with Mooncake support enabled, because the +`mooncake_store` MP L2 adapter depends on the optional +`lmcache.lmcache_mooncake` C++ extension. + +The standard prebuilt LMCache wheels currently include the Python Mooncake +adapter files, but do not include the optional `lmcache.lmcache_mooncake` +native extension. + +```bash +BUILD_MOONCAKE=1 \ +MOONCAKE_INCLUDE_DIR=/path/to/mooncake/include \ +MOONCAKE_LIB_DIR=/path/to/mooncake/lib \ +pip install -e /path/to/LMCache --verbose +``` + +## Deployment + +### 1. Start Mooncake Master + +```bash +mooncake_master -v=1 \ + --rpc_port=50051 \ + --metrics_port=9003 +``` + +### 2. Start the LMCache Multiprocess Server + +Start one LMCache MP server and configure Mooncake Store as the L2 adapter. + +```bash +lmcache server \ + --host 127.0.0.1 \ + --port 5555 \ + --http-host 127.0.0.1 \ + --http-port 8080 \ + --l1-size-gb 32 \ + --eviction-policy LRU \ + --no-l1-use-lazy \ + --l2-adapter '{ + "type": "mooncake_store", + "local_hostname": "{IP of Machine}", + "metadata_server": "P2PHANDSHAKE", + "protocol": "rdma", + "rdma_devices": "{RDMA device}", + "global_segment_size": 32212254720, + "local_buffer_size": 1073741824, + "master_server_addr": "127.0.0.1:50051" + }' +``` + +### 3. Start the Disaggregated Proxy + +The proxy receives client requests, sends prefill requests to the prefiller, +sends decode requests to the decoder, and receives LMCache request telemetry +from the prefiller. + +```bash +python /path/to/LMCache/examples/disagg_prefill_mp/disagg_proxy_server.py \ + --host 127.0.0.1 \ + --port 8000 \ + --prefiller-host 127.0.0.1 \ + --prefiller-port 8100 \ + --decoder-host 127.0.0.1 \ + --decoder-port 8200 \ + --telemetry-port 5768 +``` + +### 4. Start the vLLM Prefiller + +The prefiller reports request telemetry back to the proxy so the proxy knows +when KV cache storage has completed. + +```bash +CUDA_VISIBLE_DEVICES=0 \ +LMCACHE_REQUEST_TELEMETRY_TYPE=fastapi \ +LMCACHE_REQUEST_TELEMETRY_ENDPOINT=http://127.0.0.1:5768/api/v1/telemetry \ +vllm serve Qwen/Qwen3-4B \ + --host 127.0.0.1 \ + --port 8100 \ + --gpu-memory-utilization 0.8 \ + --no-enable-log-requests \ + --no-enable-prefix-caching \ + --kv-transfer-config '{ + "kv_connector": "LMCacheMPConnector", + "kv_role": "kv_both", + "kv_connector_extra_config": { + "lmcache.mp.host": "tcp://127.0.0.1", + "lmcache.mp.port": 5555 + } + }' +``` + +### 5. Start the vLLM Decoder + +The decoder connects to the same local LMCache MP server. It does not need +request telemetry environment variables; only the prefiller reports the "KV +cache is stored" event back to the proxy. + +```bash +CUDA_VISIBLE_DEVICES=1 \ +vllm serve Qwen/Qwen3-4B \ + --host 127.0.0.1 \ + --port 8200 \ + --gpu-memory-utilization 0.8 \ + --no-enable-log-requests \ + --no-enable-prefix-caching \ + --kv-transfer-config '{ + "kv_connector": "LMCacheMPConnector", + "kv_role": "kv_both", + "kv_connector_extra_config": { + "lmcache.mp.host": "tcp://127.0.0.1", + "lmcache.mp.port": 5555 + } + }' +``` + +### 6. Send a Test Request + +Send traffic to the proxy, not directly to either vLLM instance. + +```bash +curl -N http://127.0.0.1:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Qwen/Qwen3-4B", + "messages": [ + { + "role": "user", + "content": "Explain how KV cache reuse helps long-context serving." + } + ], + "max_tokens": 128, + "temperature": 0.7 + }' +``` + +## Port and Configuration Checklist + +When changing ports away from these defaults, update all dependent settings +together: + +- Mooncake master `--rpc_port` must match LMCache `master_server_addr`. +- The vLLM prefiller and decoder should connect to the local LMCache MP server + via `kv_connector_extra_config.lmcache.mp.host` and + `kv_connector_extra_config.lmcache.mp.port`. +- Proxy `--prefiller-port` and `--decoder-port` must match the two vLLM + `--port` values. +- `LMCACHE_REQUEST_TELEMETRY_ENDPOINT` on the prefiller must point to the proxy + telemetry endpoint. +- If `metadata_server` is changed from `P2PHANDSHAKE` to an HTTP metadata URL, + enable Mooncake HTTP metadata server and make sure its port does not conflict + with LMCache `--http-port`. + +## Additional Resources + +* [Mooncake x LMCache: Unite to Pioneer KVCache-Centric LLM Serving System](../lmcache-integration.md) +* [LMCache MP `mooncake_store` L2 adapter](https://docs.lmcache.ai/mp/l2_storage.html#mooncake-store-mooncake-store-native-connector) +* [LMCache multiprocess disaggregated prefill example](https://github.com/LMCache/LMCache/tree/dev/examples/disagg_prefill_mp) -- 2.34.1 From c876cbe1c7a520d17fa9381e72f60503dd24bd84 Mon Sep 17 00:00:00 2001 From: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com> Date: Fri, 29 May 2026 17:38:26 +0800 Subject: [PATCH 149/382] Fix engine.so runtime dependency (#2255) Signed-off-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com> --- mooncake-ep/setup.py | 2 ++ mooncake-pg/setup.py | 2 ++ 2 files changed, 4 insertions(+) diff --git a/mooncake-ep/setup.py b/mooncake-ep/setup.py index 75a2fe62..8a63079f 100644 --- a/mooncake-ep/setup.py +++ b/mooncake-ep/setup.py @@ -50,7 +50,9 @@ setup( extra_link_args=[ "-Wl,-rpath,$ORIGIN", "-L" + os.path.join(current_dir, "../mooncake-wheel/mooncake"), + "-Wl,--push-state,--no-as-needed", "-l:engine.so", + "-Wl,--pop-state", ], ) ], diff --git a/mooncake-pg/setup.py b/mooncake-pg/setup.py index 8a393773..595b11a7 100644 --- a/mooncake-pg/setup.py +++ b/mooncake-pg/setup.py @@ -64,7 +64,9 @@ setup( extra_link_args=[ "-Wl,-rpath,$ORIGIN", "-L" + os.path.join(current_dir, "../mooncake-wheel/mooncake"), + "-Wl,--push-state,--no-as-needed", "-l:engine.so", + "-Wl,--pop-state", ], ) ], -- 2.34.1 From daeef0053da2e6b4e97914e7c471bf71cca0ec54 Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Fri, 29 May 2026 19:08:05 +0800 Subject: [PATCH 150/382] [Store] Implement tenant metadata map isolation (#2232) --- mooncake-store/include/master_service.h | 329 ++- mooncake-store/include/types.h | 4 + mooncake-store/src/master_service.cpp | 1823 +++++++++-------- .../snapshot/snapshot_child_process_test.cpp | 15 +- mooncake-store/tests/master_service_test.cpp | 320 +++ 5 files changed, 1591 insertions(+), 900 deletions(-) diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index 893a37e5..e6e7f175 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -1,5 +1,6 @@ #pragma once +#include #include #include #include @@ -163,6 +164,8 @@ class MasterService { * @return ErrorCode::OK if exists, otherwise return other ErrorCode */ auto ExistKey(const std::string& key) -> tl::expected; + auto ExistKey(const std::string& key, const std::string& tenant_id) + -> tl::expected; std::vector> BatchExistKey( const std::vector& keys); @@ -256,6 +259,11 @@ class MasterService { -> tl::expected< std::unordered_map>, ErrorCode>; + auto GetReplicaListByRegex(const std::string& regex_pattern, + const std::string& tenant_id) + -> tl::expected< + std::unordered_map>, + ErrorCode>; /** * @brief Get list of replicas for an object @@ -265,6 +273,8 @@ class MasterService { */ auto GetReplicaList(const std::string& key) -> tl::expected; + auto GetReplicaList(const std::string& key, const std::string& tenant_id) + -> tl::expected; /** * @brief Start a put operation for an object @@ -277,6 +287,10 @@ class MasterService { auto PutStart(const UUID& client_id, const std::string& key, const uint64_t slice_length, const ReplicateConfig& config) -> tl::expected, ErrorCode>; + auto PutStart(const UUID& client_id, const std::string& key, + const std::string& tenant_id, const uint64_t slice_length, + const ReplicateConfig& config) + -> tl::expected, ErrorCode>; /** * @brief Complete a put operation, replica_type indicates the type of @@ -286,6 +300,9 @@ class MasterService { */ auto PutEnd(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected; + auto PutEnd(const UUID& client_id, const std::string& key, + const std::string& tenant_id, ReplicaType replica_type) + -> tl::expected; /** * @brief Adds a replica instance associated with the given client and key. @@ -332,6 +349,10 @@ class MasterService { auto UpsertStart(const UUID& client_id, const std::string& key, const uint64_t slice_length, const ReplicateConfig& config) -> tl::expected, ErrorCode>; + auto UpsertStart(const UUID& client_id, const std::string& key, + const std::string& tenant_id, const uint64_t slice_length, + const ReplicateConfig& config) + -> tl::expected, ErrorCode>; /** * @brief Complete an upsert operation. Delegates to PutEnd. @@ -446,6 +467,8 @@ class MasterService { */ auto Remove(const std::string& key, bool force = false) -> tl::expected; + auto Remove(const std::string& key, const std::string& tenant_id, + bool force = false) -> tl::expected; /** * @brief Removes objects from the master whose keys match a regex pattern. @@ -456,6 +479,8 @@ class MasterService { */ auto RemoveByRegex(const std::string& str, bool force = false) -> tl::expected; + auto RemoveByRegex(const std::string& str, const std::string& tenant_id, + bool force = false) -> tl::expected; /** * @brief Remove all objects and their replicas @@ -726,6 +751,11 @@ class MasterService { void JobDispatchThreadFunc(); // Internal data structures + struct ObjectIdentity { + std::string tenant_id; + std::string user_key; + }; + struct ObjectMetadata { // RAII-style metric management ~ObjectMetadata() { @@ -743,12 +773,15 @@ class MasterService { size_t value_length, std::vector&& reps, bool enable_soft_pin, bool enable_hard_pin = false, ObjectDataType data_type_ = ObjectDataType::UNKNOWN, - std::string group_id_ = "") + std::string group_id_ = "", std::string tenant_id_ = "default", + std::string user_key_ = {}) : client_id(client_id_), put_start_time(put_start_time_), size(value_length), data_type(data_type_), group_id(std::move(group_id_)), + tenant_id(std::move(tenant_id_)), + user_key(std::move(user_key_)), lease_timeout(), soft_pin_timeout(std::nullopt), hard_pinned(enable_hard_pin), @@ -773,6 +806,8 @@ class MasterService { const size_t size; const ObjectDataType data_type{ObjectDataType::UNKNOWN}; const std::string group_id; + const std::string tenant_id; + const std::string user_key; mutable SpinLock lock; // Default constructor, creates a time_point representing @@ -1069,20 +1104,28 @@ class MasterService { static constexpr size_t kNumShards = 1024; // Number of metadata shards + struct TenantState { + std::unordered_map metadata; + std::unordered_set processing_keys; + std::unordered_map + replication_tasks; + std::unordered_map offloading_tasks; + std::unordered_map promotion_tasks; + + std::unordered_map> + group_members; // group_id → set of keys + + bool Empty() const { + return metadata.empty() && processing_keys.empty() && + replication_tasks.empty() && offloading_tasks.empty() && + promotion_tasks.empty() && group_members.empty(); + } + }; + // Sharded metadata maps and their mutexes struct MetadataShard { mutable SharedMutex mutex; - std::unordered_map metadata - GUARDED_BY(mutex); - std::unordered_set processing_keys GUARDED_BY(mutex); - std::unordered_map replication_tasks - GUARDED_BY(mutex); - std::unordered_map offloading_tasks - GUARDED_BY(mutex); - std::unordered_map promotion_tasks - GUARDED_BY(mutex); - std::unordered_map> - group_members GUARDED_BY(mutex); + std::unordered_map tenants GUARDED_BY(mutex); }; std::array metadata_shards_; @@ -1092,6 +1135,17 @@ class MasterService { GUARDED_BY(group_routing_mutex_); mutable std::shared_mutex group_routing_mutex_; + static constexpr size_t kObjectOperationLockStripes = 4096; + + struct ObjectOperationLock { + std::unique_lock lock; + }; + + ObjectOperationLock AcquireObjectOperationLock(const std::string& tenant_id, + const std::string& key); + + std::array object_operation_locks_; + // For accessing a metadata shard with read-write permission class MetadataShardAccessorRW { public: @@ -1130,22 +1184,59 @@ class MasterService { SharedMutexLocker lock_; }; - // Helper to get shard index from key + static ObjectIdentity MakeObjectIdentity( + const std::string& user_key, const std::string& tenant_id = "default") { + return {NormalizeTenantId(tenant_id), user_key}; + } + + static std::string MakeTenantScopedKey(const std::string& tenant_id, + const std::string& key) { + const auto normalized_tenant = NormalizeTenantId(tenant_id); + std::string scoped_key; + scoped_key.reserve(normalized_tenant.size() + key.size() + 1); + scoped_key.append(normalized_tenant); + scoped_key.push_back('\0'); + scoped_key.append(key); + return scoped_key; + } + + // Helper to get shard index from tenant-scoped object identity. + size_t getShardIndex(const std::string& tenant_id, + const std::string& user_key) const { + const auto normalized_tenant = NormalizeTenantId(tenant_id); + if (normalized_tenant == "default") { + return std::hash{}(user_key) % kNumShards; + } + size_t seed = std::hash{}(normalized_tenant); + boost::hash_combine(seed, user_key); + return seed % kNumShards; + } + + // Legacy helper routes plain keys to the default tenant. size_t getShardIndex(const std::string& key) const { return std::hash{}(key) % kNumShards; } size_t getMetadataShardIndex(const std::string& key) const; - void RegisterGroupMember(MetadataShard& shard, const std::string& key, + size_t getMetadataShardIndex(const std::string& tenant_id, + const std::string& key) const; + std::optional GetGroupRoute(const std::string& tenant_id, + const std::string& key) const; + void RegisterGroupMember(TenantState& tenant_state, + const std::string& tenant_id, + const std::string& key, const std::string& group_id); - void UnregisterGroupMember(MetadataShard& shard, const std::string& key, + void UnregisterGroupMember(TenantState& tenant_state, + const std::string& tenant_id, + const std::string& key, const std::string& group_id); - std::unordered_map::iterator - EraseMetadataEntry( - MetadataShard& shard, - std::unordered_map::iterator it); + std::unordered_map::iterator EraseMetadata( + TenantState& tenant_state, + std::unordered_map::iterator it, + const std::string& tenant_id); void RebuildGroupRoutingIndex(); - void GrantLeaseForGroup(const MetadataShard& shard, const std::string& key, + void GrantLeaseForGroup(const TenantState& tenant_state, + const std::string& key, const ObjectMetadata& metadata) const; // Helper to clean up stale handles pointing to unmounted segments @@ -1160,6 +1251,7 @@ class MasterService { MetadataShardAccessorRW& shard, const UUID& client_id, const std::string& key, uint64_t value_length, const ReplicateConfig& config, const std::string& group_id, + const std::string& tenant_id, const std::chrono::system_clock::time_point& now) -> tl::expected, ErrorCode>; @@ -1186,8 +1278,8 @@ class MasterService { bool ProbeNoFSegment(const std::string& te_endpoint, std::string* error_reason); - tl::expected PushOffloadingQueue(const std::string& key, - Replica& replica); + tl::expected PushOffloadingQueue( + const ObjectIdentity& object_id, Replica& replica); // Graceful unmount scheduler class GracefulUnmountScheduler { @@ -1225,8 +1317,8 @@ class MasterService { * Caller is responsible for refcnt-pinning the source replica and * recording the task in the shard's promotion_tasks map. */ - tl::expected PushPromotionQueue(const std::string& key, - Replica& source_replica); + tl::expected PushPromotionQueue( + const ObjectIdentity& object_id, Replica& source_replica); /** * @brief Helper invoked from GetReplicaList when an only-LOCAL_DISK key is @@ -1236,7 +1328,17 @@ class MasterService { * map. Acquires its own RW shard accessor; safe to call after * GetReplicaList's RO accessor has been released. */ - void TryPushPromotionQueue(const std::string& key); + void TryPushPromotionQueue(const ObjectIdentity& object_id); + + // Erase any in-flight PromotionTask for `key` and decrement the + // cluster-wide in-flight counter. Safe no-op if no task exists. + void ErasePromotionTaskIfPresent(TenantState& tenant_state, + const std::string& key) + NO_THREAD_SAFETY_ANALYSIS { + if (tenant_state.promotion_tasks.erase(key) > 0) { + promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + } + } // Lease related members const uint64_t default_kv_lease_ttl_; // in milliseconds @@ -1275,19 +1377,37 @@ class MasterService { class MetadataAccessorRW { public: MetadataAccessorRW(MasterService* service, const std::string& key) + : MetadataAccessorRW(service, MakeObjectIdentity(key)) {} + + MetadataAccessorRW(MasterService* service, + const ObjectIdentity& object_id) : service_(service), - key_(key), - shard_idx_(service_->getMetadataShardIndex(key)), + object_id_(object_id), + shard_idx_(service_->getMetadataShardIndex(object_id_.tenant_id, + object_id_.user_key)), shard_guard_(service_, shard_idx_), - it_(shard_guard_->metadata.find(key)), - processing_it_(shard_guard_->processing_keys.find(key)), - replication_task_it_(shard_guard_->replication_tasks.find(key)) { + tenant_it_(shard_guard_->tenants.find(object_id_.tenant_id)), + tenant_state_(tenant_it_ == shard_guard_->tenants.end() + ? nullptr + : &tenant_it_->second), + it_(tenant_state_ == nullptr + ? ObjectMetadataIterator{} + : tenant_state_->metadata.find(object_id_.user_key)), + processing_it_(tenant_state_ == nullptr + ? ProcessingIterator{} + : tenant_state_->processing_keys.find( + object_id_.user_key)), + replication_task_it_(tenant_state_ == nullptr + ? ReplicationTaskIterator{} + : tenant_state_->replication_tasks.find( + object_id_.user_key)) { // Automatically clean up invalid handles (memory replicas only). // Note: We only check memory replicas here to avoid lock order // violation (client_mutex_ must be acquired before metadata shard). // local_disk replicas are cleaned up by ClearInvalidHandles() in // ClientMonitorFunc. - if (it_ != shard_guard_->metadata.end()) { + if (tenant_state_ != nullptr && + it_ != tenant_state_->metadata.end()) { // Erase invalid memory replicas (those with unmounted // segments). No client_mutex_ needed since we only check memory // replicas. @@ -1296,32 +1416,48 @@ class MasterService { }); // If no valid replicas remain, delete the whole object. if (!it_->second.IsValid()) { + const bool had_processing = + processing_it_ != tenant_state_->processing_keys.end(); this->Erase(); - if (processing_it_ != shard_guard_->processing_keys.end()) { + if (tenant_state_ != nullptr && had_processing) { this->EraseFromProcessing(); } + if (tenant_state_ != nullptr) { + service_->ErasePromotionTaskIfPresent( + *tenant_state_, object_id_.user_key); + MaybeEraseEmptyTenant(); + } } } } // Check if metadata exists bool Exists() const NO_THREAD_SAFETY_ANALYSIS { - return it_ != shard_guard_->metadata.end() && it_->second.IsValid(); + return tenant_state_ != nullptr && + it_ != tenant_state_->metadata.end() && + it_->second.IsValid(); } bool InProcessing() const NO_THREAD_SAFETY_ANALYSIS { - return processing_it_ != shard_guard_->processing_keys.end(); + return tenant_state_ != nullptr && + processing_it_ != tenant_state_->processing_keys.end(); } bool HasReplicationTask() const NO_THREAD_SAFETY_ANALYSIS { - return replication_task_it_ != - shard_guard_->replication_tasks.end(); + return tenant_state_ != nullptr && + replication_task_it_ != + tenant_state_->replication_tasks.end(); } MetadataShardAccessorRW& GetShard() NO_THREAD_SAFETY_ANALYSIS { return shard_guard_; } + TenantState& GetTenantState() NO_THREAD_SAFETY_ANALYSIS { + EnsureTenantState(); + return *tenant_state_; + } + // Get metadata (only call when Exists() is true) ObjectMetadata& Get() NO_THREAD_SAFETY_ANALYSIS { return it_->second; } @@ -1331,44 +1467,80 @@ class MasterService { // Delete current metadata (for PutRevoke or Remove operations) void Erase() NO_THREAD_SAFETY_ANALYSIS { - it_ = service_->EraseMetadataEntry(shard_guard_.get(), it_); + service_->EraseMetadata(*tenant_state_, it_, object_id_.tenant_id); + it_ = tenant_state_->metadata.end(); + MaybeEraseEmptyTenant(); } void EraseFromProcessing() NO_THREAD_SAFETY_ANALYSIS { - shard_guard_->processing_keys.erase(processing_it_); - processing_it_ = shard_guard_->processing_keys.end(); + tenant_state_->processing_keys.erase(processing_it_); + processing_it_ = tenant_state_->processing_keys.end(); + MaybeEraseEmptyTenant(); } void EraseReplicationTask() NO_THREAD_SAFETY_ANALYSIS { - shard_guard_->replication_tasks.erase(replication_task_it_); - replication_task_it_ = shard_guard_->replication_tasks.end(); + tenant_state_->replication_tasks.erase(replication_task_it_); + replication_task_it_ = tenant_state_->replication_tasks.end(); + MaybeEraseEmptyTenant(); } void Create(const UUID& client_id, uint64_t total_length, std::vector replicas, bool enable_soft_pin, bool enable_hard_pin = false, - ObjectDataType data_type = ObjectDataType::UNKNOWN) { + ObjectDataType data_type = ObjectDataType::UNKNOWN, + std::string group_id = "") { if (Exists()) { throw std::logic_error("Already exists"); } const auto now = std::chrono::system_clock::now(); - auto result = shard_guard_->metadata.emplace( - std::piecewise_construct, std::forward_as_tuple(key_), - std::forward_as_tuple(client_id, now, total_length, - std::move(replicas), enable_soft_pin, - enable_hard_pin, data_type)); + EnsureTenantState(); + auto result = tenant_state_->metadata.emplace( + std::piecewise_construct, + std::forward_as_tuple(object_id_.user_key), + std::forward_as_tuple( + client_id, now, total_length, std::move(replicas), + enable_soft_pin, enable_hard_pin, data_type, group_id, + object_id_.tenant_id, object_id_.user_key)); it_ = result.first; } private: + using ObjectMetadataIterator = + std::unordered_map::iterator; + using ProcessingIterator = std::unordered_set::iterator; + using ReplicationTaskIterator = + std::unordered_map::iterator; + + void EnsureTenantState() NO_THREAD_SAFETY_ANALYSIS { + if (tenant_state_ != nullptr) { + return; + } + auto result = + shard_guard_->tenants.try_emplace(object_id_.tenant_id); + tenant_it_ = result.first; + tenant_state_ = &tenant_it_->second; + it_ = tenant_state_->metadata.end(); + processing_it_ = tenant_state_->processing_keys.end(); + replication_task_it_ = tenant_state_->replication_tasks.end(); + } + + void MaybeEraseEmptyTenant() NO_THREAD_SAFETY_ANALYSIS { + if (tenant_state_ == nullptr || !tenant_state_->Empty()) { + return; + } + shard_guard_->tenants.erase(tenant_it_); + tenant_state_ = nullptr; + } + MasterService* service_; - std::string key_; + ObjectIdentity object_id_; size_t shard_idx_; MetadataShardAccessorRW shard_guard_; - std::unordered_map::iterator it_; - std::unordered_set::iterator processing_it_; - std::unordered_map::iterator - replication_task_it_; + std::unordered_map::iterator tenant_it_; + TenantState* tenant_state_; + ObjectMetadataIterator it_; + ProcessingIterator processing_it_; + ReplicationTaskIterator replication_task_it_; }; class MetadataSerializer { @@ -1416,20 +1588,37 @@ class MasterService { class MetadataAccessorRO { public: MetadataAccessorRO(const MasterService* service, const std::string& key) + : MetadataAccessorRO(service, MakeObjectIdentity(key)) {} + + MetadataAccessorRO(const MasterService* service, + const ObjectIdentity& object_id) : service_(service), - key_(key), - shard_idx_(service_->getMetadataShardIndex(key)), + object_id_(object_id), + shard_idx_(service_->getMetadataShardIndex(object_id_.tenant_id, + object_id_.user_key)), shard_guard_(service_, shard_idx_), - it_(shard_guard_->metadata.find(key)), - processing_it_(shard_guard_->processing_keys.find(key)) {} + tenant_it_(shard_guard_->tenants.find(object_id_.tenant_id)), + tenant_state_(tenant_it_ == shard_guard_->tenants.end() + ? nullptr + : &tenant_it_->second), + it_(tenant_state_ == nullptr + ? ObjectMetadataConstIterator{} + : tenant_state_->metadata.find(object_id_.user_key)), + processing_it_(tenant_state_ == nullptr + ? ProcessingConstIterator{} + : tenant_state_->processing_keys.find( + object_id_.user_key)) {} // Check if metadata exists bool Exists() const NO_THREAD_SAFETY_ANALYSIS { - return it_ != shard_guard_->metadata.end() && it_->second.IsValid(); + return tenant_state_ != nullptr && + it_ != tenant_state_->metadata.end() && + it_->second.IsValid(); } bool InProcessing() const NO_THREAD_SAFETY_ANALYSIS { - return processing_it_ != shard_guard_->processing_keys.end(); + return tenant_state_ != nullptr && + processing_it_ != tenant_state_->processing_keys.end(); } // Get metadata (only call when Exists() is true) @@ -1441,13 +1630,24 @@ class MasterService { return shard_guard_; } + const TenantState* GetTenantState() const NO_THREAD_SAFETY_ANALYSIS { + return tenant_state_; + } + private: + using ObjectMetadataConstIterator = + std::unordered_map::const_iterator; + using ProcessingConstIterator = + std::unordered_set::const_iterator; + const MasterService* service_; - const std::string key_; + const ObjectIdentity object_id_; const size_t shard_idx_; MetadataShardAccessorRO shard_guard_; - std::unordered_map::const_iterator it_; - std::unordered_set::const_iterator processing_it_; + std::unordered_map::const_iterator tenant_it_; + const TenantState* tenant_state_; + ObjectMetadataConstIterator it_; + ProcessingConstIterator processing_it_; }; friend class MetadataAccessorRW; @@ -1658,8 +1858,13 @@ class MasterService { std::optional SelectDrainTargetForKey( const ObjectMetadata& metadata, const std::string& source_segment, const std::vector& requested_targets); - std::string MakeDrainUnitKey(const std::string& key, + std::string MakeDrainUnitKey(const std::string& tenant_id, + const std::string& key, const std::string& source_segment) const; + std::string MakeDrainUnitKey(const std::string& key, + const std::string& source_segment) const { + return MakeDrainUnitKey("default", key, source_segment); + } std::thread job_dispatch_thread_; std::atomic job_dispatch_running_{false}; diff --git a/mooncake-store/include/types.h b/mooncake-store/include/types.h index b93e4882..f6c326f8 100644 --- a/mooncake-store/include/types.h +++ b/mooncake-store/include/types.h @@ -222,6 +222,10 @@ static constexpr size_t DEFAULT_LOCAL_BUFFER_SIZE = 1024 * 1024 * 16; // 16MB constexpr const char* DEFAULT_PROTOCOL = "tcp"; constexpr const char* DEFAULT_MASTER_SERVER_ADDR = "127.0.0.1:50051"; +inline std::string NormalizeTenantId(const std::string& tenant_id) { + return tenant_id.empty() ? "default" : tenant_id; +} + // Store client configuration validation limits static constexpr size_t MIN_SEGMENT_SIZE = 1024; // 1KB static constexpr size_t MAX_SEGMENT_SIZE = 1024ULL * 1024 * 1024 * 1024; // 1TB diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 6177989c..54589cce 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -165,10 +165,10 @@ MasterService::MasterService(const MasterServiceConfig& config) config.snapshot_catalog_store_connstring), put_start_discard_timeout_sec_(config.put_start_discard_timeout_sec), put_start_release_timeout_sec_(config.put_start_release_timeout_sec), + task_manager_(config.task_manager_config), cxl_path_(config.cxl_path), cxl_size_(config.cxl_size), - enable_cxl_(config.enable_cxl), - task_manager_(config.task_manager_config) { + enable_cxl_(config.enable_cxl) { if (enable_snapshot_ || enable_snapshot_restore_) { try { auto object_store_type = @@ -590,90 +590,116 @@ MasterService::getAliveClientsSnapshot() const { } size_t MasterService::getMetadataShardIndex(const std::string& key) const { + return getMetadataShardIndex("default", key); +} + +size_t MasterService::getMetadataShardIndex(const std::string& tenant_id, + const std::string& key) const { + const auto normalized_tenant = NormalizeTenantId(tenant_id); std::shared_lock lock(group_routing_mutex_); - auto it = object_group_ids_.find(key); + auto it = + object_group_ids_.find(MakeTenantScopedKey(normalized_tenant, key)); if (it == object_group_ids_.end()) { - return getShardIndex(key); + return getShardIndex(normalized_tenant, key); } return getShardIndex(it->second); } -void MasterService::RegisterGroupMember(MetadataShard& shard, +std::optional MasterService::GetGroupRoute( + const std::string& tenant_id, const std::string& key) const { + const auto normalized_tenant = NormalizeTenantId(tenant_id); + std::shared_lock lock(group_routing_mutex_); + auto it = + object_group_ids_.find(MakeTenantScopedKey(normalized_tenant, key)); + if (it == object_group_ids_.end()) { + return std::nullopt; + } + return it->second; +} + +MasterService::ObjectOperationLock MasterService::AcquireObjectOperationLock( + const std::string& tenant_id, const std::string& key) { + const auto scoped_key = MakeTenantScopedKey(tenant_id, key); + const auto stripe_idx = + std::hash{}(scoped_key) % kObjectOperationLockStripes; + return {std::unique_lock(object_operation_locks_[stripe_idx])}; +} + +void MasterService::RegisterGroupMember(TenantState& tenant_state, + const std::string& tenant_id, const std::string& key, const std::string& group_id) { if (group_id.empty()) { return; } + const auto normalized_tenant = NormalizeTenantId(tenant_id); std::unique_lock lock(group_routing_mutex_); - object_group_ids_[key] = group_id; - groups_needing_lease_refresh_.insert(group_id); - shard.group_members[group_id].insert(key); + object_group_ids_[MakeTenantScopedKey(normalized_tenant, key)] = group_id; + groups_needing_lease_refresh_.insert( + MakeTenantScopedKey(normalized_tenant, group_id)); + tenant_state.group_members[group_id].insert(key); } -void MasterService::UnregisterGroupMember(MetadataShard& shard, +void MasterService::UnregisterGroupMember(TenantState& tenant_state, + const std::string& tenant_id, const std::string& key, const std::string& group_id) { if (group_id.empty()) { return; } + const auto normalized_tenant = NormalizeTenantId(tenant_id); bool group_empty = false; - auto group_it = shard.group_members.find(group_id); - if (group_it != shard.group_members.end()) { + auto group_it = tenant_state.group_members.find(group_id); + if (group_it != tenant_state.group_members.end()) { group_it->second.erase(key); if (group_it->second.empty()) { - shard.group_members.erase(group_it); + tenant_state.group_members.erase(group_it); group_empty = true; } } std::unique_lock lock(group_routing_mutex_); - auto route_it = object_group_ids_.find(key); + auto route_it = + object_group_ids_.find(MakeTenantScopedKey(normalized_tenant, key)); if (route_it != object_group_ids_.end() && route_it->second == group_id) { object_group_ids_.erase(route_it); } if (group_empty) { - groups_needing_lease_refresh_.erase(group_id); + groups_needing_lease_refresh_.erase( + MakeTenantScopedKey(normalized_tenant, group_id)); } } std::unordered_map::iterator -MasterService::EraseMetadataEntry( - MetadataShard& shard, - std::unordered_map::iterator it) { - if (it == shard.metadata.end()) { - return it; - } +MasterService::EraseMetadata( + TenantState& tenant_state, + std::unordered_map::iterator it, + const std::string& tenant_id) { const std::string key = it->first; const std::string group_id = it->second.group_id; - // Keep the route visible until metadata is gone. Concurrent writers that - // saw the old route will either serialize on this shard or retry after the - // route changes, instead of creating a second copy on the key shard. - auto next = shard.metadata.erase(it); - if (shard.promotion_tasks.erase(key) > 0) { - promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); - } - UnregisterGroupMember(shard, key, group_id); + auto next = tenant_state.metadata.erase(it); + UnregisterGroupMember(tenant_state, tenant_id, key, group_id); return next; } void MasterService::RebuildGroupRoutingIndex() { std::unordered_map rebuilt_group_ids; std::unordered_set groups_needing_refresh; - // Snapshot restore rebuilds this derived index after all metadata shards - // have been deserialized. Take shard locks here so the helper does not - // depend on callers touching guarded shard state directly. for (size_t shard_idx = 0; shard_idx < kNumShards; ++shard_idx) { MetadataShardAccessorRW shard(this, shard_idx); - shard->group_members.clear(); - for (const auto& [key, metadata] : shard->metadata) { - if (!metadata.IsGrouped()) { - continue; + for (auto& [tenant_id, tenant_state] : shard->tenants) { + tenant_state.group_members.clear(); + for (const auto& [key, metadata] : tenant_state.metadata) { + if (!metadata.IsGrouped()) { + continue; + } + tenant_state.group_members[metadata.group_id].insert(key); + rebuilt_group_ids[MakeTenantScopedKey(tenant_id, key)] = + metadata.group_id; + groups_needing_refresh.insert( + MakeTenantScopedKey(tenant_id, metadata.group_id)); } - shard->group_members[metadata.group_id].insert(key); - rebuilt_group_ids[key] = metadata.group_id; - groups_needing_refresh.insert(metadata.group_id); } } - { std::unique_lock lock(group_routing_mutex_); object_group_ids_ = std::move(rebuilt_group_ids); @@ -681,7 +707,7 @@ void MasterService::RebuildGroupRoutingIndex() { } } -void MasterService::GrantLeaseForGroup(const MetadataShard& shard, +void MasterService::GrantLeaseForGroup(const TenantState& tenant_state, const std::string& key, const ObjectMetadata& metadata) const { if (!metadata.IsGrouped()) { @@ -693,24 +719,25 @@ void MasterService::GrantLeaseForGroup(const MetadataShard& shard, default_kv_soft_pin_ttl_); if (!needs_refresh) { std::shared_lock lock(group_routing_mutex_); - needs_refresh = groups_needing_lease_refresh_.find(metadata.group_id) != + needs_refresh = groups_needing_lease_refresh_.find(MakeTenantScopedKey( + metadata.tenant_id, metadata.group_id)) != groups_needing_lease_refresh_.end(); } if (!needs_refresh) { return; } - auto group_it = shard.group_members.find(metadata.group_id); - if (group_it == shard.group_members.end()) { + auto group_it = tenant_state.group_members.find(metadata.group_id); + if (group_it == tenant_state.group_members.end()) { metadata.GrantLease(default_kv_lease_ttl_, default_kv_soft_pin_ttl_); return; } for (const auto& member_key : group_it->second) { - auto member_it = shard.metadata.find(member_key); - if (member_it != shard.metadata.end()) { - member_it->second.GrantLease(default_kv_lease_ttl_, - default_kv_soft_pin_ttl_); + auto mit = tenant_state.metadata.find(member_key); + if (mit != tenant_state.metadata.end()) { + mit->second.GrantLease(default_kv_lease_ttl_, + default_kv_soft_pin_ttl_); } } if (group_it->second.find(key) == group_it->second.end()) { @@ -718,7 +745,8 @@ void MasterService::GrantLeaseForGroup(const MetadataShard& shard, } { std::unique_lock lock(group_routing_mutex_); - groups_needing_lease_refresh_.erase(metadata.group_id); + groups_needing_lease_refresh_.erase( + MakeTenantScopedKey(metadata.tenant_id, metadata.group_id)); } } @@ -730,18 +758,28 @@ void MasterService::ClearInvalidHandles( const std::unordered_set>& alive_clients) { for (size_t i = 0; i < kNumShards; i++) { MetadataShardAccessorRW shard(this, i); - auto it = shard->metadata.begin(); - while (it != shard->metadata.end()) { - if (CleanupStaleHandles(it->second, alive_clients)) { - // If the object is empty, we need to erase the iterator and - // also erase the key from processing_keys, - // replication_tasks, offloading_tasks, and promotion_tasks. - shard->processing_keys.erase(it->first); - shard->replication_tasks.erase(it->first); - shard->offloading_tasks.erase(it->first); - it = EraseMetadataEntry(shard.get(), it); + for (auto tenant_it = shard->tenants.begin(); + tenant_it != shard->tenants.end();) { + auto& tenant_state = tenant_it->second; + auto it = tenant_state.metadata.begin(); + while (it != tenant_state.metadata.end()) { + if (CleanupStaleHandles(it->second, alive_clients)) { + tenant_state.processing_keys.erase(it->first); + tenant_state.replication_tasks.erase(it->first); + tenant_state.offloading_tasks.erase(it->first); + if (tenant_state.promotion_tasks.erase(it->first) > 0) { + promotion_in_flight_.fetch_sub( + 1, std::memory_order_relaxed); + } + it = EraseMetadata(tenant_state, it, tenant_it->first); + } else { + ++it; + } + } + if (tenant_state.Empty()) { + tenant_it = shard->tenants.erase(tenant_it); } else { - ++it; + ++tenant_it; } } } @@ -890,8 +928,14 @@ auto MasterService::UnmountNoFSegment(const UUID& segment_id, auto MasterService::ExistKey(const std::string& key) -> tl::expected { + return ExistKey(key, "default"); +} + +auto MasterService::ExistKey(const std::string& key, + const std::string& tenant_id) + -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRO accessor(this, key); + MetadataAccessorRO accessor(this, MakeObjectIdentity(key, tenant_id)); if (!accessor.Exists()) { VLOG(1) << "key=" << key << ", info=object_not_found"; return false; @@ -901,7 +945,13 @@ auto MasterService::ExistKey(const std::string& key) if (metadata.HasReplica(&Replica::fn_is_completed)) { // Grant a lease to the object as it may be further used by the // client. - GrantLeaseForGroup(accessor.GetShard().get(), key, metadata); + auto* ts = accessor.GetTenantState(); + if (ts) { + GrantLeaseForGroup(*ts, key, metadata); + } else { + metadata.GrantLease(default_kv_lease_ttl_, + default_kv_soft_pin_ttl_); + } return true; } @@ -923,8 +973,14 @@ auto MasterService::GetAllKeys() std::vector all_keys; for (size_t i = 0; i < kNumShards; i++) { MetadataShardAccessorRO shard(this, i); - for (const auto& item : shard->metadata) { - all_keys.push_back(item.first); + auto tenant_it = shard->tenants.find("default"); + if (tenant_it == shard->tenants.end()) { + continue; + } + for (const auto& item : tenant_it->second.metadata) { + all_keys.push_back(item.second.user_key.empty() + ? item.first + : item.second.user_key); } } return all_keys; @@ -1168,6 +1224,14 @@ auto MasterService::BatchReplicaClear( } auto MasterService::GetReplicaListByRegex(const std::string& regex_pattern) + -> tl::expected< + std::unordered_map>, + ErrorCode> { + return GetReplicaListByRegex(regex_pattern, "default"); +} + +auto MasterService::GetReplicaListByRegex(const std::string& regex_pattern, + const std::string& tenant_id) -> tl::expected< std::unordered_map>, ErrorCode> { @@ -1183,10 +1247,14 @@ auto MasterService::GetReplicaListByRegex(const std::string& regex_pattern) } std::shared_lock shared_lock(snapshot_mutex_); + const auto normalized_tenant = NormalizeTenantId(tenant_id); for (size_t i = 0; i < kNumShards; ++i) { MetadataShardAccessorRO shard(this, i); - - for (const auto& [key, metadata] : shard->metadata) { + auto tenant_it = shard->tenants.find(normalized_tenant); + if (tenant_it == shard->tenants.end()) { + continue; + } + for (const auto& [key, metadata] : tenant_it->second.metadata) { if (std::regex_search(key, pattern)) { std::vector replica_list; metadata.VisitReplicas( @@ -1203,7 +1271,7 @@ auto MasterService::GetReplicaListByRegex(const std::string& regex_pattern) } results.emplace(key, std::move(replica_list)); - GrantLeaseForGroup(shard.get(), key, metadata); + GrantLeaseForGroup(tenant_it->second, key, metadata); } } } @@ -1213,12 +1281,19 @@ auto MasterService::GetReplicaListByRegex(const std::string& regex_pattern) auto MasterService::GetReplicaList(const std::string& key) -> tl::expected { + return GetReplicaList(key, "default"); +} + +auto MasterService::GetReplicaList(const std::string& key, + const std::string& tenant_id) + -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); + const auto object_id = MakeObjectIdentity(key, tenant_id); GetReplicaListResponse resp({}, default_kv_lease_ttl_); bool promotion_eligible = false; { - MetadataAccessorRO accessor(this, key); + MetadataAccessorRO accessor(this, object_id); MasterMetricManager::instance().inc_total_get_nums(); @@ -1248,7 +1323,13 @@ auto MasterService::GetReplicaList(const std::string& key) MasterMetricManager::instance().inc_valid_get_nums(); // Grant a lease to the object so it will not be removed // when the client is reading it. - GrantLeaseForGroup(accessor.GetShard().get(), key, metadata); + auto* ts = accessor.GetTenantState(); + if (ts) { + GrantLeaseForGroup(*ts, key, metadata); + } else { + metadata.GrantLease(default_kv_lease_ttl_, + default_kv_soft_pin_ttl_); + } // Promotion-on-hit eligibility: only when no MEMORY replica is // present but at least one LOCAL_DISK replica is. Decided here while @@ -1267,7 +1348,7 @@ auto MasterService::GetReplicaList(const std::string& key) } // RO accessor released. Safe to take a fresh RW accessor now. if (promotion_eligible) { - TryPushPromotionQueue(key); + TryPushPromotionQueue(object_id); } return resp; } @@ -1276,8 +1357,19 @@ auto MasterService::AllocateAndInsertMetadata( MetadataShardAccessorRW& shard, const UUID& client_id, const std::string& key, uint64_t value_length, const ReplicateConfig& config, const std::string& group_id, + const std::string& tenant_id, const std::chrono::system_clock::time_point& now) -> tl::expected, ErrorCode> { + auto& tenant_state = shard->tenants[tenant_id]; + if (tenant_state.metadata.contains(key)) { + LOG(INFO) << "key=" << key << ", info=object_already_exists"; + return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); + } + if (GetGroupRoute(tenant_id, key).has_value()) { + LOG(INFO) << "key=" << key << ", info=object_already_exists"; + return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); + } + std::vector replicas; const auto write_mode = DetermineReplicaWriteMode(config); size_t allocated_memory_replicas = 0; @@ -1404,16 +1496,17 @@ auto MasterService::AllocateAndInsertMetadata( } } - // Publish the route before metadata while holding the target shard lock, so - // same-key callers serialize on this shard instead of missing the grouped - // object and writing a duplicate on the natural key shard. - RegisterGroupMember(shard.get(), key, group_id); - shard->metadata.emplace( + auto [it, inserted] = tenant_state.metadata.emplace( std::piecewise_construct, std::forward_as_tuple(key), std::forward_as_tuple(client_id, now, value_length, std::move(replicas), config.with_soft_pin, config.with_hard_pin, - config.data_type, group_id)); - shard->processing_keys.insert(key); + config.data_type, group_id, tenant_id, key)); + if (!inserted) { + LOG(INFO) << "key=" << key << ", info=object_already_exists"; + return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); + } + RegisterGroupMember(tenant_state, tenant_id, key, group_id); + tenant_state.processing_keys.insert(key); return replica_list; } @@ -1422,6 +1515,15 @@ auto MasterService::PutStart(const UUID& client_id, const std::string& key, const uint64_t slice_length, const ReplicateConfig& config) -> tl::expected, ErrorCode> { + return PutStart(client_id, key, "default", slice_length, config); +} + +auto MasterService::PutStart(const UUID& client_id, const std::string& key, + const std::string& tenant_id, + const uint64_t slice_length, + const ReplicateConfig& config) + -> tl::expected, ErrorCode> { + const auto object_id = MakeObjectIdentity(key, tenant_id); if ((config.replica_num == 0 && config.nof_replica_num == 0) || key.empty() || slice_length == 0) { LOG(ERROR) << "key=" << key << ", replica_num=" << config.replica_num @@ -1464,106 +1566,95 @@ auto MasterService::PutStart(const UUID& client_id, const std::string& key, } const std::string group_id = group_id_result.value(); - auto alive_clients = getAliveClientsSnapshot(); - std::shared_lock shared_lock(snapshot_mutex_); + [[maybe_unused]] auto object_operation_lock = + AcquireObjectOperationLock(object_id.tenant_id, object_id.user_key); const auto now = std::chrono::system_clock::now(); + std::optional retry_shard_idx; + { + auto alive_clients = getAliveClientsSnapshot(); + std::shared_lock shared_lock(snapshot_mutex_); + const size_t lookup_shard_idx = + getMetadataShardIndex(object_id.tenant_id, object_id.user_key); + MetadataShardAccessorRW shard(this, lookup_shard_idx); + auto& tenant_state = shard->tenants[object_id.tenant_id]; - auto prepare_existing = - [this, &alive_clients, &key, &now]( - MetadataShardAccessorRW& shard, - std::unordered_map::iterator& it) - -> tl::expected { - if (it == shard->metadata.end()) { - return {}; - } - - if (CleanupStaleHandles(it->second, alive_clients)) { - shard->processing_keys.erase(key); - shard->replication_tasks.erase(key); - shard->offloading_tasks.erase(key); - it = EraseMetadataEntry(shard.get(), it); - return {}; - } - - auto& metadata = it->second; - // If the object's PutStart expired and has not completed any replicas, - // discard it and allow the new PutStart to go. - if (!metadata.HasReplica(&Replica::fn_is_completed) && - metadata.put_start_time + put_start_discard_timeout_sec_ < now) { - auto replicas = metadata.PopReplicas(&Replica::fn_is_processing); - if (!replicas.empty()) { - std::lock_guard lock(discarded_replicas_mutex_); - discarded_replicas_.emplace_back( - std::move(replicas), - metadata.put_start_time + put_start_release_timeout_sec_); + auto it = tenant_state.metadata.find(key); + if (it != tenant_state.metadata.end()) { + if (CleanupStaleHandles(it->second, alive_clients)) { + tenant_state.processing_keys.erase(key); + tenant_state.replication_tasks.erase(key); + tenant_state.offloading_tasks.erase(key); + ErasePromotionTaskIfPresent(tenant_state, key); + EraseMetadata(tenant_state, it, object_id.tenant_id); + it = tenant_state.metadata.end(); + } else { + auto& metadata = it->second; + if (metadata.HasReplica(&Replica::fn_is_completed) || + metadata.put_start_time + put_start_discard_timeout_sec_ >= + now) { + LOG(INFO) + << "key=" << key << ", info=object_already_exists"; + return tl::make_unexpected( + ErrorCode::OBJECT_ALREADY_EXISTS); + } + auto replicas = + metadata.PopReplicas(&Replica::fn_is_processing); + if (!replicas.empty()) { + std::lock_guard lock(discarded_replicas_mutex_); + discarded_replicas_.emplace_back( + std::move(replicas), + metadata.put_start_time + + put_start_release_timeout_sec_); + } + tenant_state.processing_keys.erase(key); + EraseMetadata(tenant_state, it, object_id.tenant_id); + it = tenant_state.metadata.end(); } - shard->processing_keys.erase(key); - it = EraseMetadataEntry(shard.get(), it); - return {}; } + if (it == tenant_state.metadata.end()) { + const size_t target_shard_idx = + group_id.empty() + ? getShardIndex(object_id.tenant_id, object_id.user_key) + : getShardIndex(group_id); + if (target_shard_idx != lookup_shard_idx) { + retry_shard_idx = target_shard_idx; + if (tenant_state.Empty()) { + shard->tenants.erase(object_id.tenant_id); + } + } else { + return AllocateAndInsertMetadata(shard, client_id, key, + slice_length, config, group_id, + object_id.tenant_id, now); + } + } + } + std::shared_lock shared_lock(snapshot_mutex_); + MetadataShardAccessorRW shard(this, retry_shard_idx.value()); + auto& retry_tenant_state = shard->tenants[object_id.tenant_id]; + if (GetGroupRoute(object_id.tenant_id, object_id.user_key).has_value() || + retry_tenant_state.metadata.contains(key)) { LOG(INFO) << "key=" << key << ", info=object_already_exists"; return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); - }; - - auto put_with_locked_target = [&](MetadataShardAccessorRW& target_shard) - -> tl::expected, ErrorCode> { - auto target_it = target_shard->metadata.find(key); - auto result = prepare_existing(target_shard, target_it); - if (!result) { - return tl::make_unexpected(result.error()); - } - return AllocateAndInsertMetadata(target_shard, client_id, key, - slice_length, config, group_id, now); - }; - - auto put_with_locked_shards = [&](MetadataShardAccessorRW& existing_shard, - MetadataShardAccessorRW& target_shard) - -> tl::expected, ErrorCode> { - auto existing_it = existing_shard->metadata.find(key); - auto result = prepare_existing(existing_shard, existing_it); - if (!result) { - return tl::make_unexpected(result.error()); - } - return put_with_locked_target(target_shard); - }; - - while (true) { - const size_t existing_shard_idx = getMetadataShardIndex(key); - const size_t target_shard_idx = - group_id.empty() ? getShardIndex(key) : getShardIndex(group_id); - - if (existing_shard_idx == target_shard_idx) { - MetadataShardAccessorRW shard(this, target_shard_idx); - if (getMetadataShardIndex(key) != existing_shard_idx) { - continue; - } - return put_with_locked_target(shard); - } - - if (existing_shard_idx < target_shard_idx) { - MetadataShardAccessorRW existing_shard(this, existing_shard_idx); - MetadataShardAccessorRW target_shard(this, target_shard_idx); - if (getMetadataShardIndex(key) != existing_shard_idx) { - continue; - } - return put_with_locked_shards(existing_shard, target_shard); - } - - MetadataShardAccessorRW target_shard(this, target_shard_idx); - MetadataShardAccessorRW existing_shard(this, existing_shard_idx); - if (getMetadataShardIndex(key) != existing_shard_idx) { - continue; - } - return put_with_locked_shards(existing_shard, target_shard); } + return AllocateAndInsertMetadata(shard, client_id, key, slice_length, + config, group_id, object_id.tenant_id, + now); } auto MasterService::PutEnd(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected { + return PutEnd(client_id, key, "default", replica_type); +} + +auto MasterService::PutEnd(const UUID& client_id, const std::string& key, + const std::string& tenant_id, + ReplicaType replica_type) + -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + const auto object_id = MakeObjectIdentity(key, tenant_id); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { LOG(ERROR) << "key=" << key << ", error=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -1597,18 +1688,19 @@ auto MasterService::PutEnd(const UUID& client_id, const std::string& key, [](Replica& replica) { replica.mark_complete(); }); if (enable_offload_ && !offload_on_evict_) { - auto& shard = accessor.GetShard(); + auto& tenant_state = accessor.GetTenantState(); metadata.VisitReplicas( [](const Replica& replica) { return replica.is_completed() && replica.is_memory_replica(); }, - [this, &key, &shard](Replica& replica) { - auto result = PushOffloadingQueue(key, replica); + [this, &object_id, &tenant_state](Replica& replica) { + auto result = PushOffloadingQueue(object_id, replica); if (result) { replica.inc_refcnt(); - shard->offloading_tasks.emplace( - key, OffloadingTask{replica.id(), - std::chrono::system_clock::now()}); + tenant_state.offloading_tasks.emplace( + object_id.user_key, + OffloadingTask{replica.id(), + std::chrono::system_clock::now()}); } }); } @@ -1774,6 +1866,15 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, const uint64_t slice_length, const ReplicateConfig& config) -> tl::expected, ErrorCode> { + return UpsertStart(client_id, key, "default", slice_length, config); +} + +auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, + const std::string& tenant_id, + const uint64_t slice_length, + const ReplicateConfig& config) + -> tl::expected, ErrorCode> { + const auto object_id = MakeObjectIdentity(key, tenant_id); // --- Parameter validation (same as PutStart) --- if ((config.replica_num == 0 && config.nof_replica_num == 0) || key.empty() || slice_length == 0) { @@ -1815,105 +1916,49 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, if (!group_id_result) { return tl::make_unexpected(group_id_result.error()); } - const bool has_requested_group_id = config.group_ids.has_value(); - const std::string requested_group_id = group_id_result.value(); + const std::string group_id = group_id_result.value(); - // --- Lock acquisition --- - // snapshot_mutex_ (shared): allows concurrent reads/writes, blocks only - // during full metadata snapshots. - // shard lock (exclusive via MetadataShardAccessorRW): serializes all - // operations on keys that hash to the same shard. - auto alive_clients = getAliveClientsSnapshot(); - std::shared_lock shared_lock(snapshot_mutex_); + [[maybe_unused]] auto object_operation_lock = + AcquireObjectOperationLock(object_id.tenant_id, object_id.user_key); const auto now = std::chrono::system_clock::now(); + std::optional case_a_retry_shard_idx; + { + // --- Lock acquisition --- + auto alive_clients = getAliveClientsSnapshot(); + std::shared_lock shared_lock(snapshot_mutex_); + // Use getMetadataShardIndex to find the object at its current shard + // (handles both grouped and ungrouped routing). + const size_t lookup_shard_idx = + getMetadataShardIndex(object_id.tenant_id, object_id.user_key); + MetadataShardAccessorRW shard(this, lookup_shard_idx); + auto& tenant_state = shard->tenants[object_id.tenant_id]; - auto cleanup_existing_if_stale = - [this, &alive_clients, &key]( - MetadataShardAccessorRW& shard, - std::unordered_map::iterator& it) - -> bool { - if (it != shard->metadata.end() && + auto it = tenant_state.metadata.find(key); + + // --- Step 0: stale handle cleanup --- + if (it != tenant_state.metadata.end() && CleanupStaleHandles(it->second, alive_clients)) { - shard->processing_keys.erase(key); - shard->replication_tasks.erase(key); - shard->offloading_tasks.erase(key); - EraseMetadataEntry(shard.get(), it); - it = shard->metadata.end(); - return true; + tenant_state.processing_keys.erase(key); + ErasePromotionTaskIfPresent(tenant_state, key); + EraseMetadata(tenant_state, it, object_id.tenant_id); + it = tenant_state.metadata.end(); } - return false; - }; - - while (true) { - const size_t existing_shard_idx = getMetadataShardIndex(key); - const size_t requested_shard_idx = - has_requested_group_id && !requested_group_id.empty() - ? getShardIndex(requested_group_id) - : getShardIndex(key); - const size_t target_shard_idx = requested_shard_idx; - - std::optional first_shard_guard; - std::optional second_shard_guard; - MetadataShardAccessorRW* existing_shard = nullptr; - MetadataShardAccessorRW* target_shard = nullptr; - if (existing_shard_idx == target_shard_idx) { - first_shard_guard.emplace(this, target_shard_idx); - target_shard = &*first_shard_guard; - } else if (existing_shard_idx < target_shard_idx) { - first_shard_guard.emplace(this, existing_shard_idx); - second_shard_guard.emplace(this, target_shard_idx); - existing_shard = &*first_shard_guard; - target_shard = &*second_shard_guard; - } else { - first_shard_guard.emplace(this, target_shard_idx); - second_shard_guard.emplace(this, existing_shard_idx); - target_shard = &*first_shard_guard; - existing_shard = &*second_shard_guard; - } - - if (getMetadataShardIndex(key) != existing_shard_idx) { - continue; - } - - MetadataShardAccessorRW* shard_for_key = target_shard; - std::unordered_map::iterator it; - - if (existing_shard != nullptr) { - auto existing_it = (*existing_shard)->metadata.find(key); - cleanup_existing_if_stale(*existing_shard, existing_it); - if (existing_it != (*existing_shard)->metadata.end()) { - if (has_requested_group_id) { - LOG(ERROR) << "key=" << key - << ", error=group_membership_is_immutable"; - return tl::make_unexpected(ErrorCode::INVALID_PARAMS); - } - shard_for_key = existing_shard; - it = existing_it; - } else { - it = (*target_shard)->metadata.find(key); - cleanup_existing_if_stale(*target_shard, it); - } - } else { - it = (*target_shard)->metadata.find(key); - cleanup_existing_if_stale(*target_shard, it); - } - - MetadataShardAccessorRW& shard = *shard_for_key; // --- Step 1: safety checks and preemption (only if key exists) --- - if (it != shard->metadata.end()) { + if (it != tenant_state.metadata.end()) { auto& metadata = it->second; - if (has_requested_group_id && - metadata.group_id != requested_group_id) { + // Reject if the caller tries to change group membership. + // Group membership is immutable while an object exists. + if (config.group_ids.has_value() && metadata.group_id != group_id) { LOG(ERROR) << "key=" << key << ", error=group_membership_is_immutable"; return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } // Reject if a Copy/Move task is actively reading this key's - // replicas. Writing during replication would corrupt the copy. - if (shard->replication_tasks.count(key) > 0) { + // replicas. + if (tenant_state.replication_tasks.count(key) > 0) { LOG(INFO) << "key=" << key << ", error=object_has_replication_task"; return tl::make_unexpected( @@ -1921,7 +1966,7 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, } // Reject if an offload-to-disk task is in progress (same reason). - if (shard->offloading_tasks.count(key) > 0) { + if (tenant_state.offloading_tasks.count(key) > 0) { LOG(INFO) << "key=" << key << ", error=object_has_offloading_task"; return tl::make_unexpected( @@ -1933,7 +1978,7 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, // with a TTL so they are not freed while the old writer may still // be doing RDMA writes. Unlike PutStart (which only preempts after // a timeout), UpsertStart preempts immediately. - if (shard->processing_keys.count(key) > 0) { + if (tenant_state.processing_keys.count(key) > 0) { auto processing_replicas = metadata.PopReplicas(&Replica::fn_is_processing); if (!processing_replicas.empty()) { @@ -1942,112 +1987,141 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, std::move(processing_replicas), now + put_start_release_timeout_sec_); } - shard->processing_keys.erase(key); + tenant_state.processing_keys.erase(key); // If no COMPLETE replicas survive the preemption, this key // effectively does not exist — fall through to Case A. if (!metadata.HasReplica(&Replica::fn_is_completed)) { - it = EraseMetadataEntry(shard.get(), it); - it = shard->metadata.end(); - if (!has_requested_group_id && - shard_for_key != target_shard) { - continue; - } + ErasePromotionTaskIfPresent(tenant_state, key); + EraseMetadata(tenant_state, it, object_id.tenant_id); + it = tenant_state.metadata.end(); } } } // --- Case A: key does not exist (or was erased above) --- // Allocate fresh buffers, identical to PutStart. - if (it == shard->metadata.end()) { + if (it == tenant_state.metadata.end()) { VLOG(1) << "key=" << key << ", action=upsert_start_case_a"; - const std::string group_id = - has_requested_group_id ? requested_group_id : ""; - return AllocateAndInsertMetadata( - shard, client_id, key, slice_length, config, group_id, now); - } - - // --- Step 2: key exists with COMPLETE replicas → Case B or C --- - auto& metadata = it->second; - - // Reject if any reader holds a reference (refcnt > 0). Overwriting a - // buffer that an RDMA read is streaming from would cause data - // corruption. The client should retry after readers finish. - if (metadata.HasReplica(&Replica::fn_is_busy)) { - LOG(INFO) << "key=" << key << ", error=object_replica_busy"; - return tl::make_unexpected(ErrorCode::OBJECT_REPLICA_BUSY); - } - - if (metadata.size == slice_length) { - // --- Case B: same size — in-place update --- - // Reuse existing buffer addresses. No allocation or deallocation. - // The client will RDMA-write new data to the same addresses. - // - // hard_pinned is const and preserved automatically — upsert does - // not change the eviction protection level of an existing object. - metadata.client_id = client_id; - metadata.put_start_time = now; - - // Reconcile soft_pin state with the incoming config. - { - SpinLocker locker(&metadata.lock); - if (config.with_soft_pin && !metadata.soft_pin_timeout) { - metadata.soft_pin_timeout.emplace(); - MasterMetricManager::instance().inc_soft_pin_key_count(1); - } else if (!config.with_soft_pin && metadata.soft_pin_timeout) { - metadata.soft_pin_timeout.reset(); - MasterMetricManager::instance().dec_soft_pin_key_count(1); + const size_t case_a_shard_idx = + group_id.empty() + ? getShardIndex(object_id.tenant_id, object_id.user_key) + : getShardIndex(group_id); + if (case_a_shard_idx != lookup_shard_idx) { + case_a_retry_shard_idx = case_a_shard_idx; + if (tenant_state.Empty()) { + shard->tenants.erase(object_id.tenant_id); } + } else { + return AllocateAndInsertMetadata(shard, client_id, key, + slice_length, config, group_id, + object_id.tenant_id, now); + } + } else { + // --- Step 2: key exists with COMPLETE replicas → Case B or C --- + auto& metadata = it->second; + + // Reject if any reader holds a reference (refcnt > 0). Overwriting + // a buffer that an RDMA read is streaming from would cause data + // corruption. The client should retry after readers finish. + if (metadata.HasReplica(&Replica::fn_is_busy)) { + LOG(INFO) << "key=" << key << ", error=object_replica_busy"; + return tl::make_unexpected(ErrorCode::OBJECT_REPLICA_BUSY); } - // Mark COMPLETE → PROCESSING so readers won't see stale data - // mid-transfer. The key becomes unreadable until UpsertEnd. - metadata.VisitReplicas( - &Replica::fn_is_completed, - [](Replica& replica) { replica.mark_processing(); }); + if (metadata.size == slice_length) { + // --- Case B: same size — in-place update --- + // Reuse existing buffer addresses. No allocation or + // deallocation. The client will RDMA-write new data to the same + // addresses. + // + // hard_pinned is const and preserved automatically — upsert + // does not change the eviction protection level of an existing + // object. + metadata.client_id = client_id; + metadata.put_start_time = now; - shard->processing_keys.insert(key); + // Reconcile soft_pin state with the incoming config. + { + SpinLocker locker(&metadata.lock); + if (config.with_soft_pin && !metadata.soft_pin_timeout) { + metadata.soft_pin_timeout.emplace(); + MasterMetricManager::instance().inc_soft_pin_key_count( + 1); + } else if (!config.with_soft_pin && + metadata.soft_pin_timeout) { + metadata.soft_pin_timeout.reset(); + MasterMetricManager::instance().dec_soft_pin_key_count( + 1); + } + } - // Return the existing descriptors — same buffer addresses as - // before. - std::vector replica_list; - const auto& all_replicas = metadata.GetAllReplicas(); - replica_list.reserve(all_replicas.size()); - for (const auto& replica : all_replicas) { - replica_list.emplace_back(replica.get_descriptor()); + // Mark COMPLETE → PROCESSING so readers won't see stale data + // mid-transfer. The key becomes unreadable until UpsertEnd. + metadata.VisitReplicas( + &Replica::fn_is_completed, + [](Replica& replica) { replica.mark_processing(); }); + + tenant_state.processing_keys.insert(key); + + // Return the existing descriptors — same buffer addresses as + // before. + std::vector replica_list; + const auto& all_replicas = metadata.GetAllReplicas(); + replica_list.reserve(all_replicas.size()); + for (const auto& replica : all_replicas) { + replica_list.emplace_back(replica.get_descriptor()); + } + + VLOG(1) << "key=" << key + << ", action=upsert_start_case_b_inplace"; + return replica_list; } - VLOG(1) << "key=" << key << ", action=upsert_start_case_b_inplace"; - return replica_list; + // --- Case C: different size — discard old replicas and reallocate + // --- Old buffers cannot be reused. Move them to + // discarded_replicas_ for delayed release (readers may still hold + // descriptors without refcnt), then allocate fresh buffers at the + // new size. + // + // Preserve hard_pin and soft_pin from the old metadata so that + // eviction protection survives a size-changing upsert (RFC §2.2.2). + ReplicateConfig merged_config = config; + merged_config.with_hard_pin = + merged_config.with_hard_pin || metadata.IsHardPinned(); + merged_config.with_soft_pin = + merged_config.with_soft_pin || metadata.IsSoftPinned(); + + const std::string existing_group_id = metadata.group_id; + auto old_replicas = metadata.PopReplicas(); + if (!old_replicas.empty()) { + std::lock_guard lock(discarded_replicas_mutex_); + discarded_replicas_.emplace_back( + std::move(old_replicas), + now + put_start_release_timeout_sec_); + } + EraseMetadata(tenant_state, it, object_id.tenant_id); + + VLOG(1) << "key=" << key + << ", action=upsert_start_case_c_reallocate"; + return AllocateAndInsertMetadata( + shard, client_id, key, slice_length, merged_config, + existing_group_id, object_id.tenant_id, now); } - - // --- Case C: different size — discard old replicas and reallocate --- - // Old buffers cannot be reused. Move them to discarded_replicas_ for - // delayed release (readers may still hold descriptors without refcnt), - // then allocate fresh buffers at the new size. - // - // Preserve hard_pin and soft_pin from the old metadata so that eviction - // protection survives a size-changing upsert (RFC §2.2.2). - ReplicateConfig merged_config = config; - merged_config.with_hard_pin = - merged_config.with_hard_pin || metadata.IsHardPinned(); - merged_config.with_soft_pin = - merged_config.with_soft_pin || metadata.IsSoftPinned(); - const std::string group_id = - has_requested_group_id ? requested_group_id : metadata.group_id; - - auto old_replicas = metadata.PopReplicas(); - if (!old_replicas.empty()) { - std::lock_guard lock(discarded_replicas_mutex_); - discarded_replicas_.emplace_back( - std::move(old_replicas), now + put_start_release_timeout_sec_); - } - EraseMetadataEntry(shard.get(), it); - - VLOG(1) << "key=" << key << ", action=upsert_start_case_c_reallocate"; - return AllocateAndInsertMetadata(shard, client_id, key, slice_length, - merged_config, group_id, now); } + std::shared_lock shared_lock(snapshot_mutex_); + MetadataShardAccessorRW shard(this, case_a_retry_shard_idx.value()); + auto& retry_tenant_state = shard->tenants[object_id.tenant_id]; + const auto current_route = + GetGroupRoute(object_id.tenant_id, object_id.user_key); + if (current_route.has_value() || + retry_tenant_state.metadata.contains(key)) { + LOG(INFO) << "key=" << key << ", info=object_already_exists"; + return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); + } + return AllocateAndInsertMetadata(shard, client_id, key, slice_length, + config, group_id, object_id.tenant_id, + now); } auto MasterService::UpsertEnd(const UUID& client_id, const std::string& key, @@ -2229,8 +2303,8 @@ tl::expected MasterService::CopyStart( } // Create replication task for tracking. - auto& shard = accessor.GetShard(); - shard->replication_tasks.emplace( + auto& tenant_state = accessor.GetTenantState(); + tenant_state.replication_tasks.emplace( std::piecewise_construct, std::forward_as_tuple(key), std::forward_as_tuple(client_id, std::chrono::system_clock::now(), ReplicationTask::Type::COPY, source->id(), @@ -2444,8 +2518,8 @@ tl::expected MasterService::MoveStart( } // Create replication task for tracking. - auto& shard = accessor.GetShard(); - shard->replication_tasks.emplace( + auto& tenant_state = accessor.GetTenantState(); + tenant_state.replication_tasks.emplace( std::piecewise_construct, std::forward_as_tuple(key), std::forward_as_tuple(client_id, std::chrono::system_clock::now(), ReplicationTask::Type::MOVE, source->id(), @@ -2602,8 +2676,14 @@ tl::expected MasterService::MoveRevoke( auto MasterService::Remove(const std::string& key, bool force) -> tl::expected { + return Remove(key, "default", force); +} + +auto MasterService::Remove(const std::string& key, const std::string& tenant_id, + bool force) -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + const auto object_id = MakeObjectIdentity(key, tenant_id); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { VLOG(1) << "key=" << key << ", error=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -2632,13 +2712,20 @@ auto MasterService::Remove(const std::string& key, bool force) return tl::make_unexpected(ErrorCode::OBJECT_HAS_REPLICATION_TASK); } - // Remove object metadata + auto& tenant_state = accessor.GetTenantState(); + ErasePromotionTaskIfPresent(tenant_state, key); accessor.Erase(); return {}; } auto MasterService::RemoveByRegex(const std::string& regex_pattern, bool force) -> tl::expected { + return RemoveByRegex(regex_pattern, "default", force); +} + +auto MasterService::RemoveByRegex(const std::string& regex_pattern, + const std::string& tenant_id, bool force) + -> tl::expected { long removed_count = 0; std::regex pattern; @@ -2651,10 +2738,17 @@ auto MasterService::RemoveByRegex(const std::string& regex_pattern, bool force) } std::shared_lock shared_lock(snapshot_mutex_); + const auto normalized_tenant = NormalizeTenantId(tenant_id); for (size_t i = 0; i < kNumShards; ++i) { MetadataShardAccessorRW shard(this, i); + auto tenant_it = shard->tenants.find(normalized_tenant); + if (tenant_it == shard->tenants.end()) { + continue; + } + auto& tenant_state = tenant_it->second; - for (auto it = shard->metadata.begin(); it != shard->metadata.end();) { + for (auto it = tenant_state.metadata.begin(); + it != tenant_state.metadata.end();) { if (std::regex_search(it->first, pattern)) { if (!force && !it->second.IsLeaseExpired()) { VLOG(1) << "key=" << it->first @@ -2677,7 +2771,7 @@ auto MasterService::RemoveByRegex(const std::string& regex_pattern, bool force) ++it; continue; } - if (metadata_shards_[i].replication_tasks.contains(it->first)) { + if (tenant_state.replication_tasks.contains(it->first)) { LOG(WARNING) << "key=" << it->first << ", matched by regex, but has replication " "task. Skipping removal."; @@ -2687,12 +2781,16 @@ auto MasterService::RemoveByRegex(const std::string& regex_pattern, bool force) VLOG(1) << "key=" << it->first << " matched by regex. Removing."; - it = EraseMetadataEntry(shard.get(), it); + ErasePromotionTaskIfPresent(tenant_state, it->first); + it = EraseMetadata(tenant_state, it, normalized_tenant); removed_count++; } else { ++it; } } + if (tenant_state.Empty()) { + shard->tenants.erase(tenant_it); + } } VLOG(1) << "action=remove_by_regex, pattern=" << regex_pattern @@ -2710,29 +2808,28 @@ long MasterService::RemoveAll(bool force) { for (size_t i = 0; i < kNumShards; i++) { MetadataShardAccessorRW shard(this, i); - if (shard->metadata.empty()) { - continue; - } - - // Only remove completed objects with expired leases (unless force=true) - auto it = shard->metadata.begin(); - while (it != shard->metadata.end()) { - /** - * The reason the force operation here does not bypass the replica - * check is that put operations (which could also be copy or move) - * and remove operations might be happening concurrently, making it - * extremely dangerous to perform a direct removal at this point. - */ - if ((force || it->second.IsLeaseExpired(now)) && - it->second.AllReplicas(&Replica::fn_is_completed) && - !shard->replication_tasks.contains(it->first)) { - auto mem_rep_count = - it->second.CountReplicas(&Replica::fn_is_memory_replica); - total_freed_size += it->second.size * mem_rep_count; - it = EraseMetadataEntry(shard.get(), it); - removed_count++; + for (auto tenant_it = shard->tenants.begin(); + tenant_it != shard->tenants.end();) { + auto& tenant_state = tenant_it->second; + auto it = tenant_state.metadata.begin(); + while (it != tenant_state.metadata.end()) { + if ((force || it->second.IsLeaseExpired(now)) && + it->second.AllReplicas(&Replica::fn_is_completed) && + !tenant_state.replication_tasks.contains(it->first)) { + auto mem_rep_count = it->second.CountReplicas( + &Replica::fn_is_memory_replica); + total_freed_size += it->second.size * mem_rep_count; + ErasePromotionTaskIfPresent(tenant_state, it->first); + it = EraseMetadata(tenant_state, it, tenant_it->first); + removed_count++; + } else { + ++it; + } + } + if (tenant_state.Empty()) { + tenant_it = shard->tenants.erase(tenant_it); } else { - ++it; + ++tenant_it; } } } @@ -2771,9 +2868,17 @@ auto MasterService::BatchRemove(const std::vector& keys, for (const auto& [original_idx, key_ptr] : key_group) { const std::string& key = *key_ptr; - auto it = shard->metadata.find(key); + auto tenant_it = shard->tenants.find("default"); + if (tenant_it == shard->tenants.end()) { + VLOG(1) << "key=" << key << ", error=object_not_found"; + results[original_idx] = + tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); + continue; + } + auto& tenant_state = tenant_it->second; + auto it = tenant_state.metadata.find(key); - if (it == shard->metadata.end()) { + if (it == tenant_state.metadata.end()) { VLOG(1) << "key=" << key << ", error=object_not_found"; results[original_idx] = tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -2782,10 +2887,11 @@ auto MasterService::BatchRemove(const std::vector& keys, // Clean up stale replica handles (consistent with single Remove) if (CleanupStaleHandles(it->second, alive_clients)) { - shard->processing_keys.erase(key); - shard->replication_tasks.erase(key); - shard->offloading_tasks.erase(key); - EraseMetadataEntry(shard.get(), it); + tenant_state.processing_keys.erase(key); + tenant_state.replication_tasks.erase(key); + tenant_state.offloading_tasks.erase(key); + ErasePromotionTaskIfPresent(tenant_state, key); + EraseMetadata(tenant_state, it, "default"); results[original_idx] = tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); continue; @@ -2812,7 +2918,7 @@ auto MasterService::BatchRemove(const std::vector& keys, continue; } - if (shard->replication_tasks.contains(key)) { + if (tenant_state.replication_tasks.contains(key)) { LOG(ERROR) << "key=" << key << ", error=object_has_replication_task"; results[original_idx] = @@ -2821,7 +2927,11 @@ auto MasterService::BatchRemove(const std::vector& keys, } // Remove object metadata - EraseMetadataEntry(shard.get(), it); + ErasePromotionTaskIfPresent(tenant_state, key); + EraseMetadata(tenant_state, it, "default"); + if (tenant_state.Empty()) { + shard->tenants.erase(tenant_it); + } results[original_idx] = {}; // Success } } @@ -2849,7 +2959,9 @@ size_t MasterService::GetKeyCount() const { size_t total = 0; for (size_t i = 0; i < kNumShards; i++) { MetadataShardAccessorRO shard(this, i); - total += shard->metadata.size(); + for (const auto& [tenant_id, tenant_state] : shard->tenants) { + total += tenant_state.metadata.size(); + } } return total; } @@ -2969,15 +3081,15 @@ auto MasterService::OffloadObjectHeartbeat(const UUID& client_id, for (auto& [key, size] : offloading_objects_copy) { MetadataAccessorRW accessor(this, key); if (accessor.Exists()) { - auto& shard = accessor.GetShard(); - auto task_it = shard->offloading_tasks.find(key); - if (task_it != shard->offloading_tasks.end()) { + auto& tenant_state = accessor.GetTenantState(); + auto task_it = tenant_state.offloading_tasks.find(key); + if (task_it != tenant_state.offloading_tasks.end()) { auto source = accessor.Get().GetReplicaByID(task_it->second.source_id); if (source) { source->dec_refcnt(); } - shard->offloading_tasks.erase(task_it); + tenant_state.offloading_tasks.erase(task_it); } } } @@ -3032,15 +3144,15 @@ auto MasterService::NotifyOffloadSuccess( MetadataAccessorRW accessor(this, key); if (accessor.Exists()) { auto& obj_metadata = accessor.Get(); - auto& shard = accessor.GetShard(); - auto task_it = shard->offloading_tasks.find(key); - if (task_it != shard->offloading_tasks.end()) { + auto& tenant_state = accessor.GetTenantState(); + auto task_it = tenant_state.offloading_tasks.find(key); + if (task_it != tenant_state.offloading_tasks.end()) { auto source = obj_metadata.GetReplicaByID(task_it->second.source_id); if (source != nullptr) { source->dec_refcnt(); } - shard->offloading_tasks.erase(task_it); + tenant_state.offloading_tasks.erase(task_it); } } } @@ -3059,7 +3171,13 @@ auto MasterService::NotifyOffloadSuccess( } tl::expected MasterService::PushOffloadingQueue( - const std::string& key, Replica& replica) { + const ObjectIdentity& object_id, Replica& replica) { + if (object_id.tenant_id != "default") { + VLOG(1) << "key=" << object_id.user_key + << ", tenant_id=" << object_id.tenant_id + << ", action=skip_offload_for_non_default_tenant"; + return tl::make_unexpected(ErrorCode::UNABLE_OFFLOADING); + } const auto& segment_names = replica.get_segment_names(); if (segment_names.empty()) { return {}; @@ -3093,9 +3211,9 @@ tl::expected MasterService::PushOffloadingQueue( return tl::make_unexpected(ErrorCode::KEYS_ULTRA_LIMIT); } auto res = local_disk_segment_it->second->offloading_objects.emplace( - key, replica.get_descriptor() - .get_memory_descriptor() - .buffer_descriptor.size_); + object_id.user_key, replica.get_descriptor() + .get_memory_descriptor() + .buffer_descriptor.size_); if (!res.second) { return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); } @@ -3109,7 +3227,7 @@ tl::expected MasterService::PushOffloadingQueue( // holder via the LOCAL_DISK replica's embedded client_id rather than via // the segment-name reverse lookup. tl::expected MasterService::PushPromotionQueue( - const std::string& key, Replica& source_replica) { + const ObjectIdentity& object_id, Replica& source_replica) { auto holder_id = source_replica.get_local_disk_client_id(); if (!holder_id.has_value()) { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); @@ -3128,19 +3246,21 @@ tl::expected MasterService::PushPromotionQueue( } MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); auto res = local_disk_segment_it->second->promotion_objects.emplace( - key, static_cast(source_replica.get_descriptor() - .get_local_disk_descriptor() - .object_size)); + object_id.user_key, + static_cast(source_replica.get_descriptor() + .get_local_disk_descriptor() + .object_size)); if (!res.second) { return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); } return {}; } -void MasterService::TryPushPromotionQueue(const std::string& key) { +void MasterService::TryPushPromotionQueue(const ObjectIdentity& object_id) { if (!promotion_on_hit_ || !promotion_sketch_) { return; } + const auto& key = object_id.user_key; // Frequency gate: bump and compare against the threshold. The sketch // returns uint8_t (saturating at 255); promotion_admission_threshold_ @@ -3164,16 +3284,16 @@ void MasterService::TryPushPromotionQueue(const std::string& key) { // Acquire a fresh RW shard accessor for dedup, refcnt-pin, and task // record. Safe to call here because GetReplicaList has already released // its RO accessor. - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { return; } auto& metadata = accessor.Get(); - auto& shard = accessor.GetShard(); + auto& tenant_state = accessor.GetTenantState(); // Dedup: don't queue twice if a promotion is already in flight or if a // MEMORY replica has appeared since GetReplicaList observed only-disk. - if (shard->promotion_tasks.count(key) > 0) { + if (tenant_state.promotion_tasks.count(key) > 0) { return; } if (metadata.HasReplica(&Replica::fn_is_memory_replica)) { @@ -3208,7 +3328,7 @@ void MasterService::TryPushPromotionQueue(const std::string& key) { source->get_descriptor().get_local_disk_descriptor().object_size; // Try to enqueue on the holder client. On failure, drop the refcnt back. - auto push_result = PushPromotionQueue(key, *source); + auto push_result = PushPromotionQueue(object_id, *source); if (!push_result) { source->dec_refcnt(); VLOG(1) << "promotion_push_failed key=" << key @@ -3223,7 +3343,7 @@ void MasterService::TryPushPromotionQueue(const std::string& key) { // Record the in-flight task. alloc_id is filled in by // PromotionAllocStart once the new MEMORY replica is staged. - shard->promotion_tasks.emplace( + tenant_state.promotion_tasks.emplace( key, PromotionTask{.source_id = source->id(), .alloc_id = 0, .object_size = object_size, @@ -3279,14 +3399,14 @@ auto MasterService::PromotionAllocStart( // stall AllocStart past put_start_release_timeout_sec_). If we // allocated and AddReplicas'd anyway, the staged PROCESSING MEMORY // replica would have no PromotionTask pointing at it: the generic - // PROCESSING reaper iterates shard->processing_keys (never + // PROCESSING reaper iterates tenant_state.processing_keys (never // populated by promotion) and the promotion-task reaper would have // nothing left to iterate, leaking the buffer until the object is // removed or evicted. The shard mutex is held for the rest of this // function, so the iterator stays valid across the allocation step. - auto& shard = accessor.GetShard(); - auto task_it = shard->promotion_tasks.find(key); - if (task_it == shard->promotion_tasks.end()) { + auto& tenant_state = accessor.GetTenantState(); + auto task_it = tenant_state.promotion_tasks.find(key); + if (task_it == tenant_state.promotion_tasks.end()) { return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); } @@ -3364,15 +3484,15 @@ auto MasterService::NotifyPromotionSuccess(const UUID& client_id, return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); } auto& metadata = accessor.Get(); - auto& shard = accessor.GetShard(); + auto& tenant_state = accessor.GetTenantState(); // Look up the in-flight task to find the exact replica we staged. A // concurrent Put on this key may have created other PROCESSING MEMORY // replicas, so we must not just "mark first PROCESSING memory // complete" — that would risk committing someone else's half-written // replica. - auto task_it = shard->promotion_tasks.find(key); - if (task_it == shard->promotion_tasks.end() || + auto task_it = tenant_state.promotion_tasks.find(key); + if (task_it == tenant_state.promotion_tasks.end() || task_it->second.alloc_id == 0) { return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); } @@ -3395,7 +3515,7 @@ auto MasterService::NotifyPromotionSuccess(const UUID& client_id, if (source != nullptr) { source->dec_refcnt(); } - shard->promotion_tasks.erase(task_it); + tenant_state.promotion_tasks.erase(task_it); promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); // Erase the per-client promotion_objects entry (best-effort; the @@ -3427,10 +3547,10 @@ auto MasterService::NotifyPromotionFailure(const UUID& client_id, return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); } auto& metadata = accessor.Get(); - auto& shard = accessor.GetShard(); + auto& tenant_state = accessor.GetTenantState(); - auto task_it = shard->promotion_tasks.find(key); - if (task_it == shard->promotion_tasks.end()) { + auto task_it = tenant_state.promotion_tasks.find(key); + if (task_it == tenant_state.promotion_tasks.end()) { // No task to release. Either the reaper already swept it, or the // client never had a task here. Return OK to keep this RPC // idempotent — repeated failure notifications on the same key @@ -3452,7 +3572,7 @@ auto MasterService::NotifyPromotionFailure(const UUID& client_id, if (task_it->second.alloc_id != 0) { metadata.EraseReplicaByID(task_it->second.alloc_id); } - shard->promotion_tasks.erase(task_it); + tenant_state.promotion_tasks.erase(task_it); promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); // Clear the holder's per-client promotion_objects entry. Same @@ -3476,13 +3596,7 @@ auto MasterService::NotifyPromotionFailure(const UUID& client_id, void MasterService::EvictionThreadFunc() { VLOG(1) << "action=eviction_thread_started"; - // Start with an already-elapsed window so the first loop iteration - // (after kEvictionThreadSleepMs) triggers DiscardExpiredProcessingReplicas. - // Without this, a task admitted shortly after thread startup can survive - // the first reaper cycle and not be cleaned until ~2s later, causing - // promotion-on-hit tests that sleep for 2s to flake. - auto last_discard_time = - std::chrono::system_clock::now() - put_start_release_timeout_sec_; + auto last_discard_time = std::chrono::system_clock::now(); while (eviction_running_) { const auto now = std::chrono::system_clock::now(); double used_ratio = @@ -3545,166 +3659,138 @@ void MasterService::DiscardExpiredProcessingReplicas( const std::chrono::system_clock::time_point& now) { std::list discarded_replicas; - // Part 1: Discard expired PutStart operations. - for (auto key_it = shard->processing_keys.begin(); - key_it != shard->processing_keys.end();) { - auto it = shard->metadata.find(*key_it); - if (it == shard->metadata.end()) { - // The key has been removed from metadata. This should be - // impossible. - LOG(ERROR) << "Key " << *key_it - << " was removed while in processing"; - key_it = shard->processing_keys.erase(key_it); - continue; - } + for (auto tenant_it = shard->tenants.begin(); + tenant_it != shard->tenants.end();) { + auto& tenant_state = tenant_it->second; - auto& metadata = it->second; - // If the object is not valid or not in processing state, just - // remove it from the processing set. - if (!metadata.IsValid() || - metadata.AllReplicas(&Replica::fn_is_completed)) { - if (!metadata.IsValid()) { - EraseMetadataEntry(shard.get(), it); + for (auto key_it = tenant_state.processing_keys.begin(); + key_it != tenant_state.processing_keys.end();) { + auto it = tenant_state.metadata.find(*key_it); + if (it == tenant_state.metadata.end()) { + LOG(ERROR) << "Key " << *key_it + << " was removed while in processing"; + key_it = tenant_state.processing_keys.erase(key_it); + continue; } - key_it = shard->processing_keys.erase(key_it); - continue; + + auto& metadata = it->second; + if (!metadata.IsValid() || + metadata.AllReplicas(&Replica::fn_is_completed)) { + if (!metadata.IsValid()) { + EraseMetadata(tenant_state, it, tenant_it->first); + } + key_it = tenant_state.processing_keys.erase(key_it); + continue; + } + + const auto ttl = + metadata.put_start_time + put_start_release_timeout_sec_; + if (ttl < now) { + auto replicas = + metadata.PopReplicas(&Replica::fn_is_processing); + if (!replicas.empty()) { + discarded_replicas.emplace_back(std::move(replicas), ttl); + } + if (!metadata.IsValid()) { + EraseMetadata(tenant_state, it, tenant_it->first); + } + key_it = tenant_state.processing_keys.erase(key_it); + continue; + } + key_it++; } - // If the object's PutStart timedout, discard and release it's - // space. Note that instead of releasing the space directly, we - // insert the replicas into the discarded list so that the - // discarding and releasing operations can be recorded in - // statistics. - const auto ttl = - metadata.put_start_time + put_start_release_timeout_sec_; - if (ttl < now) { - auto replicas = metadata.PopReplicas(&Replica::fn_is_processing); + for (auto task_it = tenant_state.replication_tasks.begin(); + task_it != tenant_state.replication_tasks.end();) { + auto metadata_it = tenant_state.metadata.find(task_it->first); + if (metadata_it == tenant_state.metadata.end()) { + LOG(ERROR) << "Key " << task_it->first + << " was removed with ongoing replication task"; + task_it = tenant_state.replication_tasks.erase(task_it); + continue; + } + + const auto ttl = + task_it->second.start_time + put_start_release_timeout_sec_; + if (ttl > now) { + task_it++; + continue; + } + + auto& metadata = metadata_it->second; + auto source = metadata.GetReplicaByID(task_it->second.source_id); + if (source != nullptr) { + source->dec_refcnt(); + } + + auto& replica_ids = task_it->second.replica_ids; + auto replicas = + metadata.PopReplicas([&replica_ids](const Replica& replica) { + auto it = std::find(replica_ids.begin(), replica_ids.end(), + replica.id()); + return it != replica_ids.end(); + }); if (!replicas.empty()) { discarded_replicas.emplace_back(std::move(replicas), ttl); } - if (!metadata.IsValid()) { - // All replicas of this object are discarded, just - // remove the whole object. - EraseMetadataEntry(shard.get(), it); + EraseMetadata(tenant_state, metadata_it, tenant_it->first); } - - key_it = shard->processing_keys.erase(key_it); - continue; + task_it = tenant_state.replication_tasks.erase(task_it); } - key_it++; - } - - // Part 2: Discard expired CopyStart/MoveStart operations. - for (auto task_it = shard->replication_tasks.begin(); - task_it != shard->replication_tasks.end();) { - auto metadata_it = shard->metadata.find(task_it->first); - if (metadata_it == shard->metadata.end()) { - // The key has been removed from metadata. This should be - // impossible. - LOG(ERROR) << "Key " << task_it->first - << " was removed with ongoing replication task"; - task_it = shard->replication_tasks.erase(task_it); - continue; - } - - const auto ttl = - task_it->second.start_time + put_start_release_timeout_sec_; - if (ttl > now) { - // The task is not expired, skip it. - task_it++; - continue; - } - - auto& metadata = metadata_it->second; - - // Release source refcnt. - auto source = metadata.GetReplicaByID(task_it->second.source_id); - if (source != nullptr) { - source->dec_refcnt(); - } - - // Discard allocated replicas. - auto& replica_ids = task_it->second.replica_ids; - auto replicas = - metadata.PopReplicas([&replica_ids](const Replica& replica) { - auto it = std::find(replica_ids.begin(), replica_ids.end(), - replica.id()); - return it != replica_ids.end(); - }); - if (!replicas.empty()) { - discarded_replicas.emplace_back(std::move(replicas), ttl); - } - - // Check whether the object is still valid. - if (!metadata.IsValid()) { - EraseMetadataEntry(shard.get(), metadata_it); - } - - task_it = shard->replication_tasks.erase(task_it); - } - - // Part 3: Discard expired offloading operations. - for (auto task_it = shard->offloading_tasks.begin(); - task_it != shard->offloading_tasks.end();) { - const auto ttl = - task_it->second.start_time + put_start_release_timeout_sec_; - if (ttl > now) { - task_it++; - continue; - } - - auto metadata_it = shard->metadata.find(task_it->first); - if (metadata_it != shard->metadata.end()) { - auto source = - metadata_it->second.GetReplicaByID(task_it->second.source_id); - if (source != nullptr) { - source->dec_refcnt(); + for (auto task_it = tenant_state.offloading_tasks.begin(); + task_it != tenant_state.offloading_tasks.end();) { + const auto ttl = + task_it->second.start_time + put_start_release_timeout_sec_; + if (ttl > now) { + task_it++; + continue; } + auto metadata_it = tenant_state.metadata.find(task_it->first); + if (metadata_it != tenant_state.metadata.end()) { + auto source = metadata_it->second.GetReplicaByID( + task_it->second.source_id); + if (source != nullptr) { + source->dec_refcnt(); + } + } + LOG(WARNING) << "Offloading task expired for key: " + << task_it->first; + task_it = tenant_state.offloading_tasks.erase(task_it); } - LOG(WARNING) << "Offloading task expired for key: " << task_it->first; - task_it = shard->offloading_tasks.erase(task_it); - } + for (auto task_it = tenant_state.promotion_tasks.begin(); + task_it != tenant_state.promotion_tasks.end();) { + const auto ttl = + task_it->second.start_time + put_start_release_timeout_sec_; + if (ttl > now) { + task_it++; + continue; + } + auto metadata_it = tenant_state.metadata.find(task_it->first); + if (metadata_it != tenant_state.metadata.end()) { + auto source = metadata_it->second.GetReplicaByID( + task_it->second.source_id); + if (source != nullptr) { + source->dec_refcnt(); + } + if (task_it->second.alloc_id != 0) { + metadata_it->second.EraseReplicaByID( + task_it->second.alloc_id); + } + } + LOG(WARNING) << "Promotion task expired for key: " + << task_it->first; + task_it = tenant_state.promotion_tasks.erase(task_it); + promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + } - // Part 4: Discard expired promotion-on-hit tasks. For each: - // - Drop the source LOCAL_DISK refcnt so the source can be - // evicted normally. - // - If a PROCESSING MEMORY replica was staged (alloc_id != 0), - // pop it via EraseReplicaByID. The staged replica is not in - // shard->processing_keys, so this is the only place (besides - // NotifyPromotionFailure) that reaps it; without this the - // buffer leaks until the object is removed or evicted. - // - Erase the task entry and decrement the in-flight counter. - // task.start_time is set at admission and reset at AllocStart, so - // queue-wait (alloc_id == 0) and active-transfer (alloc_id != 0) - // phases each get a full put_start_release_timeout_sec_ window. - // The per-client promotion_objects map is GC'd on the next - // heartbeat (entries for vanished tasks are harmless — Notify will - // return REPLICA_IS_NOT_READY since the task entry is gone). - for (auto task_it = shard->promotion_tasks.begin(); - task_it != shard->promotion_tasks.end();) { - const auto ttl = - task_it->second.start_time + put_start_release_timeout_sec_; - if (ttl > now) { - task_it++; - continue; + if (tenant_state.Empty()) { + tenant_it = shard->tenants.erase(tenant_it); + } else { + ++tenant_it; } - auto metadata_it = shard->metadata.find(task_it->first); - if (metadata_it != shard->metadata.end()) { - auto source = - metadata_it->second.GetReplicaByID(task_it->second.source_id); - if (source != nullptr) { - source->dec_refcnt(); - } - if (task_it->second.alloc_id != 0) { - metadata_it->second.EraseReplicaByID(task_it->second.alloc_id); - } - } - LOG(WARNING) << "Promotion task expired for key: " << task_it->first; - task_it = shard->promotion_tasks.erase(task_it); - promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); } if (!discarded_replicas.empty()) { @@ -4643,33 +4729,26 @@ bool MasterService::TryRestoreStateFromSnapshot( if (!skip_cleanup) { auto cleanup_now = now; for (auto& shard : metadata_shards_) { - for (auto it = shard.metadata.begin(); - it != shard.metadata.end();) { - if (it->second.HasDiffRepStatus( - ReplicaStatus::COMPLETE) || - (it->second.IsLeaseExpired(cleanup_now) && - !it->second.IsSoftPinned(cleanup_now))) { - VLOG(1) - << "clear metadata key=" << it->first - << " ,lease_timeout=" - << std::chrono::duration_cast< - std::chrono::milliseconds>( - it->second.lease_timeout - .time_since_epoch()) - .count() - << " ,soft_pin_timeout=" - << (it->second.soft_pin_timeout.has_value() - ? std::to_string( - std::chrono::duration_cast< - std::chrono::milliseconds>( - it->second.soft_pin_timeout - .value() - .time_since_epoch()) - .count()) - : "null"); - it = EraseMetadataEntry(shard, it); + for (auto tenant_it = shard.tenants.begin(); + tenant_it != shard.tenants.end();) { + auto& tenant_state = tenant_it->second; + for (auto it = tenant_state.metadata.begin(); + it != tenant_state.metadata.end();) { + if (it->second.HasDiffRepStatus( + ReplicaStatus::COMPLETE) || + (it->second.IsLeaseExpired(cleanup_now) && + !it->second.IsSoftPinned(cleanup_now))) { + VLOG(1) << "clear metadata key=" << it->first; + it = EraseMetadata(tenant_state, it, + tenant_it->first); + } else { + ++it; + } + } + if (tenant_state.Empty()) { + tenant_it = shard.tenants.erase(tenant_it); } else { - ++it; + ++tenant_it; } } } @@ -4682,30 +4761,33 @@ bool MasterService::TryRestoreStateFromSnapshot( } for (auto& shard : metadata_shards_) { - for (auto it = shard.metadata.begin(); - it != shard.metadata.end();) { - for (auto& replica : it->second.GetAllReplicas()) { - if (!replica.get_descriptor().is_memory_replica()) { - continue; + for (auto& [tenant_id, tenant_state] : shard.tenants) { + for (auto it = tenant_state.metadata.begin(); + it != tenant_state.metadata.end();) { + for (auto& replica : it->second.GetAllReplicas()) { + if (!replica.get_descriptor().is_memory_replica()) { + continue; + } + auto temp_segment_names = + replica.get_segment_names(); + if (temp_segment_names.empty()) { + continue; + } + if (!temp_segment_names[0].has_value()) { + continue; + } + auto buffer_descriptor = + replica.get_descriptor() + .get_memory_descriptor() + .buffer_descriptor; + MasterMetricManager::instance() + .inc_allocated_mem_size( + temp_segment_names[0].value(), + static_cast( + buffer_descriptor.size_)); } - auto temp_segment_names = replica.get_segment_names(); - if (temp_segment_names.empty()) { - continue; - } - - std::string temp_segment_name; - if (temp_segment_names[0].has_value()) { - temp_segment_name = temp_segment_names[0].value(); - } - - auto buffer_descriptor = replica.get_descriptor() - .get_memory_descriptor() - .buffer_descriptor; - MasterMetricManager::instance().inc_allocated_mem_size( - temp_segment_name, - static_cast(buffer_descriptor.size_)); + ++it; } - ++it; } } @@ -4818,11 +4900,6 @@ void MasterService::BatchEvict(double evict_ratio_target, }); }; - struct EvictionResult { - uint64_t freed_bytes{0}; - long evicted_objects{0}; - }; - // --- Offload-on-evict support --- long offload_queued_this_cycle = 0; long offload_deferred_count = 0; @@ -4840,13 +4917,18 @@ void MasterService::BatchEvict(double evict_ratio_target, // Returns freed bytes. Returns 0 if offload-queued and no additional // replicas were evicted (all MEMORY replicas of the key are now pinned). auto try_evict_or_offload = - [&, this](const std::string& key, ObjectMetadata& metadata, - MetadataShardAccessorRW& shard) -> uint64_t { + [&, this](const std::string& tenant_id, const std::string& key, + ObjectMetadata& metadata, + TenantState& tenant_state) -> uint64_t { if (!offload_on_evict_) { // Original behavior return metadata.size * evict_replicas(metadata); } + if (tenant_id != "default") { + return 0; + } + // LOCAL_DISK replica already exists — safe to delete MEMORY immediately if (has_local_disk_replica(metadata)) { return metadata.size * evict_replicas(metadata); @@ -4867,12 +4949,14 @@ void MasterService::BatchEvict(double evict_ratio_target, return r.is_memory_replica() && r.is_completed() && r.get_refcnt() == 0; }, - [this, &key, &shard, &queued, &now](Replica& replica) { + [this, &tenant_id, &key, &tenant_state, &queued, + &now](Replica& replica) { if (queued) return; // only need to pin one replica for offload - auto result = PushOffloadingQueue(key, replica); + auto result = PushOffloadingQueue( + MakeObjectIdentity(key, tenant_id), replica); if (result) { replica.inc_refcnt(); - shard->offloading_tasks.emplace( + tenant_state.offloading_tasks.emplace( key, OffloadingTask{replica.id(), now}); queued = true; } @@ -4898,24 +4982,32 @@ void MasterService::BatchEvict(double evict_ratio_target, return 0; }; + struct EvictionResult { + uint64_t freed_bytes{0}; + long evicted_objects{0}; + }; + auto try_evict_group_or_object = - [&, this](const std::string& key, ObjectMetadata& metadata, - MetadataShardAccessorRW& shard, + [&, this](const std::string& tenant_id, const std::string& key, + ObjectMetadata& metadata, MetadataShardAccessorRW& shard, + TenantState& tenant_state, bool allow_soft_pinned) -> EvictionResult { if (!metadata.IsGrouped()) { - uint64_t freed = try_evict_or_offload(key, metadata, shard); + uint64_t freed = + try_evict_or_offload(tenant_id, key, metadata, tenant_state); return {.freed_bytes = freed, .evicted_objects = freed > 0 ? 1 : 0}; } - auto group_it = shard->group_members.find(metadata.group_id); - if (group_it == shard->group_members.end()) { - uint64_t freed = try_evict_or_offload(key, metadata, shard); + auto group_it = tenant_state.group_members.find(metadata.group_id); + if (group_it == tenant_state.group_members.end()) { + uint64_t freed = + try_evict_or_offload(tenant_id, key, metadata, tenant_state); return {.freed_bytes = freed, .evicted_objects = freed > 0 ? 1 : 0}; } for (const auto& member_key : group_it->second) { - auto member_it = shard->metadata.find(member_key); - if (member_it != shard->metadata.end() && + auto member_it = tenant_state.metadata.find(member_key); + if (member_it != tenant_state.metadata.end() && !member_it->second.IsLeaseExpired(now)) { return {}; } @@ -4925,8 +5017,8 @@ void MasterService::BatchEvict(double evict_ratio_target, std::vector member_keys(group_it->second.begin(), group_it->second.end()); for (const auto& member_key : member_keys) { - auto member_it = shard->metadata.find(member_key); - if (member_it == shard->metadata.end()) { + auto member_it = tenant_state.metadata.find(member_key); + if (member_it == tenant_state.metadata.end()) { continue; } auto& member_metadata = member_it->second; @@ -4937,16 +5029,14 @@ void MasterService::BatchEvict(double evict_ratio_target, continue; } - uint64_t freed = - try_evict_or_offload(member_key, member_metadata, shard); + uint64_t freed = try_evict_or_offload( + tenant_id, member_key, member_metadata, tenant_state); result.freed_bytes += freed; if (freed > 0) { result.evicted_objects++; } - // The caller owns the iterator for the trigger key and erases it - // after this helper returns; erase only peer members here. if (member_key != key && !member_metadata.IsValid()) { - EraseMetadataEntry(shard.get(), member_it); + EraseMetadata(tenant_state, member_it, tenant_id); } } return result; @@ -4965,9 +5055,11 @@ void MasterService::BatchEvict(double evict_ratio_target, // in later evictions. DiscardExpiredProcessingReplicas(shard, now); - // object_count must be updated at beginning as it will be used later - // to compute ideal_evict_num - object_count += shard->metadata.size(); + size_t shard_object_count = 0; + for (const auto& [tenant_id, tenant_state] : shard->tenants) { + shard_object_count += tenant_state.metadata.size(); + } + object_count += shard_object_count; // To achieve evicted_count / object_count = evict_ratio_target, // ideally how many object should be evicted in this shard @@ -4976,30 +5068,25 @@ void MasterService::BatchEvict(double evict_ratio_target, std::vector candidates; // can be removed - for (auto it = shard->metadata.begin(); it != shard->metadata.end(); - it++) { - // Hard-pinned objects are never evicted - if (it->second.IsHardPinned()) { - continue; - } - // Skip objects that are not expired or have incomplete replicas - if (!it->second.IsLeaseExpired(now) || - !can_evict_replicas(it->second)) { - continue; - } - if (!it->second.IsSoftPinned(now)) { - if (ideal_evict_num > 0) { - // first pass candidates - candidates.push_back(it->second.lease_timeout); - } else { - // No need to evict any object in this shard, put to - // second pass candidates - no_pin_objects.push_back(it->second.lease_timeout); + for (const auto& [tenant_id, tenant_state] : shard->tenants) { + for (auto it = tenant_state.metadata.begin(); + it != tenant_state.metadata.end(); it++) { + if (it->second.IsHardPinned()) { + continue; + } + if (!it->second.IsLeaseExpired(now) || + !can_evict_replicas(it->second)) { + continue; + } + if (!it->second.IsSoftPinned(now)) { + if (ideal_evict_num > 0) { + candidates.push_back(it->second.lease_timeout); + } else { + no_pin_objects.push_back(it->second.lease_timeout); + } + } else if (allow_evict_soft_pinned_objects_) { + soft_pin_objects.push_back(it->second.lease_timeout); } - } else if (allow_evict_soft_pinned_objects_) { - // second pass candidates, only if - // allow_evict_soft_pinned_objects_ is true - soft_pin_objects.push_back(it->second.lease_timeout); } } @@ -5011,34 +5098,39 @@ void MasterService::BatchEvict(double evict_ratio_target, candidates.begin() + (evict_num - 1), candidates.end()); auto target_timeout = candidates[evict_num - 1]; - // Evict objects with lease timeout less than or equal to target. - auto it = shard->metadata.begin(); - while (it != shard->metadata.end()) { - // Skip objects that are not allowed to be evicted in the first - // pass - if (it->second.IsHardPinned() || - !it->second.IsLeaseExpired(now) || - it->second.IsSoftPinned(now) || - !can_evict_replicas(it->second)) { - ++it; - continue; - } - if (it->second.lease_timeout <= target_timeout) { - // Evict this object (or defer for offload) - auto evict_result = - try_evict_group_or_object(it->first, it->second, shard, - /*allow_soft_pinned=*/false); - total_freed_size += evict_result.freed_bytes; - if (it->second.IsValid() == false) { - it = EraseMetadataEntry(shard.get(), it); + for (auto tenant_it = shard->tenants.begin(); + tenant_it != shard->tenants.end();) { + auto& tenant_state = tenant_it->second; + auto it = tenant_state.metadata.begin(); + while (it != tenant_state.metadata.end()) { + if (it->second.IsHardPinned() || + !it->second.IsLeaseExpired(now) || + it->second.IsSoftPinned(now) || + !can_evict_replicas(it->second)) { + ++it; + continue; + } + if (it->second.lease_timeout <= target_timeout) { + auto evict_result = try_evict_group_or_object( + tenant_it->first, it->first, it->second, shard, + tenant_state, /*allow_soft_pinned=*/false); + total_freed_size += evict_result.freed_bytes; + if (it->second.IsValid() == false) { + it = EraseMetadata(tenant_state, it, + tenant_it->first); + } else { + ++it; + } + shard_evicted_count += evict_result.evicted_objects; } else { + no_pin_objects.push_back(it->second.lease_timeout); ++it; } - shard_evicted_count += evict_result.evicted_objects; + } + if (tenant_state.Empty()) { + tenant_it = shard->tenants.erase(tenant_it); } else { - // second pass candidates - no_pin_objects.push_back(it->second.lease_timeout); - ++it; + ++tenant_it; } } evicted_count += shard_evicted_count; @@ -5079,27 +5171,38 @@ void MasterService::BatchEvict(double evict_ratio_target, for (size_t i = 0; i < kNumShards && target_evict_num > 0; i++) { MetadataShardAccessorRW shard(this, (start_idx + i) % kNumShards); - auto it = shard->metadata.begin(); - while (it != shard->metadata.end() && target_evict_num > 0) { - if (!it->second.IsHardPinned() && - it->second.IsLeaseExpired(now) && - it->second.lease_timeout <= target_timeout && - !it->second.IsSoftPinned(now) && - can_evict_replicas(it->second)) { - // Evict this object (or defer for offload) - auto evict_result = try_evict_group_or_object( - it->first, it->second, shard, - /*allow_soft_pinned=*/false); - total_freed_size += evict_result.freed_bytes; - if (it->second.IsValid() == false) { - it = EraseMetadataEntry(shard.get(), it); + for (auto tenant_it = shard->tenants.begin(); + tenant_it != shard->tenants.end() && + target_evict_num > 0;) { + auto& tenant_state = tenant_it->second; + auto it = tenant_state.metadata.begin(); + while (it != tenant_state.metadata.end() && + target_evict_num > 0) { + if (!it->second.IsHardPinned() && + it->second.IsLeaseExpired(now) && + it->second.lease_timeout <= target_timeout && + !it->second.IsSoftPinned(now) && + can_evict_replicas(it->second)) { + auto evict_result = try_evict_group_or_object( + tenant_it->first, it->first, it->second, shard, + tenant_state, /*allow_soft_pinned=*/false); + total_freed_size += evict_result.freed_bytes; + if (!it->second.IsValid()) { + it = EraseMetadata(tenant_state, it, + tenant_it->first); + } else { + ++it; + } + evicted_count += evict_result.evicted_objects; + target_evict_num -= evict_result.evicted_objects; } else { ++it; } - evicted_count += evict_result.evicted_objects; - target_evict_num -= evict_result.evicted_objects; + } + if (tenant_state.Empty()) { + tenant_it = shard->tenants.erase(tenant_it); } else { - ++it; + ++tenant_it; } } } @@ -5125,33 +5228,41 @@ void MasterService::BatchEvict(double evict_ratio_target, MetadataShardAccessorRW shard(this, (start_idx + i) % kNumShards); - auto it = shard->metadata.begin(); - while (it != shard->metadata.end() && target_evict_num > 0) { - // Skip hard-pinned or not-yet-expired objects - if (it->second.IsHardPinned() || - !it->second.IsLeaseExpired(now) || - !can_evict_replicas(it->second)) { - ++it; - continue; - } - // Evict objects with 1). no soft pin OR 2). with soft pin - // and lease timeout less than or equal to target. - if (!it->second.IsSoftPinned(now) || - it->second.lease_timeout <= soft_target_timeout) { - // Evict this object (or defer for offload) - auto evict_result = try_evict_group_or_object( - it->first, it->second, shard, - /*allow_soft_pinned=*/true); - total_freed_size += evict_result.freed_bytes; - if (it->second.IsValid() == false) { - it = EraseMetadataEntry(shard.get(), it); + for (auto tenant_it = shard->tenants.begin(); + tenant_it != shard->tenants.end() && + target_evict_num > 0;) { + auto& tenant_state = tenant_it->second; + auto it = tenant_state.metadata.begin(); + while (it != tenant_state.metadata.end() && + target_evict_num > 0) { + if (it->second.IsHardPinned() || + !it->second.IsLeaseExpired(now) || + !can_evict_replicas(it->second)) { + ++it; + continue; + } + if (!it->second.IsSoftPinned(now) || + it->second.lease_timeout <= soft_target_timeout) { + auto evict_result = try_evict_group_or_object( + tenant_it->first, it->first, it->second, shard, + tenant_state, /*allow_soft_pinned=*/true); + total_freed_size += evict_result.freed_bytes; + if (!it->second.IsValid()) { + it = EraseMetadata(tenant_state, it, + tenant_it->first); + } else { + ++it; + } + evicted_count += evict_result.evicted_objects; + target_evict_num -= evict_result.evicted_objects; } else { ++it; } - evicted_count += evict_result.evicted_objects; - target_evict_num -= evict_result.evicted_objects; + } + if (tenant_state.Empty()) { + tenant_it = shard->tenants.erase(tenant_it); } else { - ++it; + ++tenant_it; } } } @@ -5230,7 +5341,9 @@ void MasterService::NoFBatchEvict(double evict_ratio_target, MetadataShardAccessorRW shard( this, (start_idx + i) % metadata_shards_.size()); DiscardExpiredProcessingReplicas(shard, now); - object_count += shard->metadata.size(); + for (const auto& [tenant_id, tenant_state] : shard->tenants) { + object_count += tenant_state.metadata.size(); + } const long ideal_evict_num = std::ceil(object_count * evict_ratio_target) - evicted_count; @@ -5239,32 +5352,43 @@ void MasterService::NoFBatchEvict(double evict_ratio_target, } long shard_evicted_count = 0; - for (auto it = shard->metadata.begin(); - it != shard->metadata.end() && + for (auto tenant_it = shard->tenants.begin(); + tenant_it != shard->tenants.end() && shard_evicted_count < ideal_evict_num;) { - auto& metadata = it->second; - if (metadata.IsHardPinned() || !metadata.IsLeaseExpired(now) || - metadata.IsSoftPinned(now)) { - ++it; - continue; - } + auto& tenant_state = tenant_it->second; + for (auto it = tenant_state.metadata.begin(); + it != tenant_state.metadata.end() && + shard_evicted_count < ideal_evict_num;) { + auto& metadata = it->second; + if (metadata.IsHardPinned() || !metadata.IsLeaseExpired(now) || + metadata.IsSoftPinned(now)) { + ++it; + continue; + } - const size_t erased = - metadata.EraseReplicas([](const Replica& replica) { - return replica.is_nof_replica() && replica.is_completed() && - replica.get_refcnt() == 0; - }); - if (erased == 0) { - ++it; - continue; - } + const size_t erased = + metadata.EraseReplicas([](const Replica& replica) { + return replica.is_nof_replica() && + replica.is_completed() && + replica.get_refcnt() == 0; + }); + if (erased == 0) { + ++it; + continue; + } - total_freed_size += metadata.size * erased; - shard_evicted_count++; - if (!metadata.IsValid()) { - it = EraseMetadataEntry(shard.get(), it); + total_freed_size += metadata.size * erased; + shard_evicted_count++; + if (!metadata.IsValid()) { + it = EraseMetadata(tenant_state, it, tenant_it->first); + } else { + ++it; + } + } + if (tenant_state.Empty()) { + tenant_it = shard->tenants.erase(tenant_it); } else { - ++it; + ++tenant_it; } } evicted_count += shard_evicted_count; @@ -5650,7 +5774,7 @@ MasterService::MetadataSerializer::Serialize() { // First count non-empty shards size_t valid_shards = 0; for (size_t i = 0; i < kNumShards; ++i) { - if (!service_->metadata_shards_[i].metadata.empty()) { + if (!service_->metadata_shards_[i].tenants.empty()) { valid_shards++; } } @@ -5663,7 +5787,7 @@ MasterService::MetadataSerializer::Serialize() { const auto& shard = service_->metadata_shards_[shard_idx]; // Skip if shard is empty - if (shard.metadata.empty()) { + if (shard.tenants.empty()) { continue; } @@ -5834,15 +5958,13 @@ MasterService::MetadataSerializer::Deserialize( auto next_id = replica_next_id_obj->as(); Replica::next_id_.store(next_id); LOG(INFO) << "Restored Replica::next_id_ to " << next_id; - service_->RebuildGroupRoutingIndex(); return {}; } void MasterService::MetadataSerializer::Reset() { for (auto& shard : service_->metadata_shards_) { - shard.metadata.clear(); - shard.group_members.clear(); + shard.tenants.clear(); } { std::unique_lock lock( @@ -5865,29 +5987,46 @@ MasterService::MetadataSerializer::SerializeShard(const MetadataShard& shard, // Serialize metadata packer.pack("metadata"); - packer.pack_array(shard.metadata.size()); - - // Sort keys to ensure consistent serialization order. - // NOTE: sort may be slow for large shards. - std::vector sorted_keys; - sorted_keys.reserve(shard.metadata.size()); - for (const auto& [key, metadata] : shard.metadata) { - sorted_keys.push_back(key); + size_t metadata_count = 0; + for (const auto& [tenant_id, tenant_state] : shard.tenants) { + metadata_count += tenant_state.metadata.size(); } - std::sort(sorted_keys.begin(), sorted_keys.end()); + packer.pack_array(metadata_count); - for (const auto& key : sorted_keys) { - const auto& metadata = shard.metadata.at(key); - // Each metadata item format: [key, metadata_object] - packer.pack_array(2); - packer.pack(key); + // Sort tenant/key pairs to ensure consistent serialization order. + // NOTE: sort may be slow for large shards. + struct SortedEntry { + std::string tenant_id; + std::string key; + const ObjectMetadata* metadata; + }; + std::vector sorted_entries; + sorted_entries.reserve(metadata_count); + for (const auto& [tenant_id, tenant_state] : shard.tenants) { + for (const auto& [key, metadata] : tenant_state.metadata) { + sorted_entries.push_back({tenant_id, key, &metadata}); + } + } + std::sort(sorted_entries.begin(), sorted_entries.end(), + [](const SortedEntry& lhs, const SortedEntry& rhs) { + if (lhs.tenant_id != rhs.tenant_id) { + return lhs.tenant_id < rhs.tenant_id; + } + return lhs.key < rhs.key; + }); - auto result = SerializeMetadata(metadata, packer); + for (const auto& entry : sorted_entries) { + // Each metadata item format: [tenant_id, key, metadata_object]. + packer.pack_array(3); + packer.pack(entry.tenant_id); + packer.pack(entry.key); + + auto result = SerializeMetadata(*entry.metadata, packer); if (!result) { return tl::make_unexpected(SerializationError( result.error().code, fmt::format("Failed to serialize metadata for key '{}': {}", - key, result.error().message))); + entry.key, result.error().message))); } } @@ -5916,8 +6055,7 @@ MasterService::MetadataSerializer::DeserializeShard(const msgpack::object& obj, } // Clear existing data - shard.metadata.clear(); - shard.group_members.clear(); + shard.tenants.clear(); // Deserialize metadata if (metadata_array == nullptr || @@ -5927,21 +6065,33 @@ MasterService::MetadataSerializer::DeserializeShard(const msgpack::object& obj, "Missing or invalid 'metadata' field in shard")); } - shard.metadata.reserve(metadata_array->via.array.size); + shard.tenants.reserve(metadata_array->via.array.size); for (uint32_t j = 0; j < metadata_array->via.array.size; ++j) { const msgpack::object& item = metadata_array->via.array.ptr[j]; - if (item.type != msgpack::type::ARRAY || item.via.array.size != 2) { + if (item.type != msgpack::type::ARRAY || + (item.via.array.size != 2 && item.via.array.size != 3)) { return tl::make_unexpected(SerializationError( ErrorCode::DESERIALIZE_FAIL, - "Invalid metadata item format: expected [key, metadata]")); + "Invalid metadata item format: expected [key, metadata] or " + "[tenant_id, key, metadata]")); } - std::string key = item.via.array.ptr[0].as(); - const msgpack::object& value_obj = item.via.array.ptr[1]; + std::string tenant_id = "default"; + std::string key; + const msgpack::object* value_obj = nullptr; + if (item.via.array.size == 2) { + key = item.via.array.ptr[0].as(); + value_obj = &item.via.array.ptr[1]; + } else { + tenant_id = + NormalizeTenantId(item.via.array.ptr[0].as()); + key = item.via.array.ptr[1].as(); + value_obj = &item.via.array.ptr[2]; + } - auto metadata_result = DeserializeMetadata(value_obj); + auto metadata_result = DeserializeMetadata(*value_obj); if (!metadata_result) { LOG(ERROR) << "Failed to deserialize metadata for key: " << key << ": " << metadata_result.error().message; @@ -5949,14 +6099,16 @@ MasterService::MetadataSerializer::DeserializeShard(const msgpack::object& obj, } auto metadata_ptr = std::move(metadata_result.value()); - auto [it, inserted] = shard.metadata.emplace( + auto& tenant_state = shard.tenants[tenant_id]; + const std::string user_key = key; + auto [it, inserted] = tenant_state.metadata.emplace( std::piecewise_construct, std::forward_as_tuple(std::move(key)), std::forward_as_tuple( metadata_ptr->client_id, metadata_ptr->put_start_time, metadata_ptr->size, metadata_ptr->PopReplicas(), metadata_ptr->soft_pin_timeout.has_value(), metadata_ptr->IsHardPinned(), metadata_ptr->data_type, - metadata_ptr->group_id)); + metadata_ptr->group_id, tenant_id, user_key)); it->second.lease_timeout = metadata_ptr->lease_timeout; it->second.soft_pin_timeout = metadata_ptr->soft_pin_timeout; @@ -6045,9 +6197,7 @@ MasterService::MetadataSerializer::DeserializeMetadata( } // Need at least 7 elements: client_id, put_start_time, size, lease_timeout, - // has_soft_pin_timeout, soft_pin_timeout, replicas_count. - // Optional fields are decoded by type for backward compatibility: - // data_type appears before replicas; hard_pinned and group_id trail them. + // has_soft_pin_timeout, soft_pin_timeout, replicas_count if (obj.via.array.size < 7) { return tl::unexpected(SerializationError( ErrorCode::DESERIALIZE_FAIL, @@ -6083,9 +6233,8 @@ MasterService::MetadataSerializer::DeserializeMetadata( // Format detection: // v1: 7 + replicas_count, no optional fields // v2: 8 + replicas_count, either data_type or hard_pinned - // v3: 9 + replicas_count, data_type + hard_pinned or - // hard_pinned + group_id - // v4: 10 + replicas_count, data_type + hard_pinned + group_id + // v3: 9 + replicas_count, data_type + hard_pinned or hard_pinned + + // group_id v4: 10 + replicas_count, data_type + hard_pinned + group_id constexpr uint32_t kBaseFieldCount = 7; constexpr uint32_t kMaxOptionalFieldCount = 3; const uint32_t total_elements = obj.via.array.size; @@ -6127,11 +6276,6 @@ MasterService::MetadataSerializer::DeserializeMetadata( if (index < obj.via.array.size && array[index].type == msgpack::type::STR) { group_id = array[index++].as(); } - if (index != obj.via.array.size) { - return tl::unexpected(SerializationError( - ErrorCode::DESERIALIZE_FAIL, - "deserialize ObjectMetadata optional field type mismatch")); - } // Create ObjectMetadata instance bool enable_soft_pin = has_soft_pin_timeout; @@ -6483,8 +6627,11 @@ tl::expected MasterService::CancelDrainJob( } std::string MasterService::MakeDrainUnitKey( - const std::string& key, const std::string& source_segment) const { - return std::to_string(key.size()) + ":" + key + ":" + source_segment; + const std::string& tenant_id, const std::string& key, + const std::string& source_segment) const { + const auto normalized_tenant = NormalizeTenantId(tenant_id); + return std::to_string(normalized_tenant.size()) + ":" + normalized_tenant + + ":" + std::to_string(key.size()) + ":" + key + ":" + source_segment; } std::optional MasterService::SelectDrainTargetForKey( @@ -6599,40 +6746,45 @@ void MasterService::ScheduleDrainJobTasks(DrainJob& job) { std::shared_lock shared_lock(snapshot_mutex_); for (size_t i = 0; i < kNumShards; ++i) { MetadataShardAccessorRO shard(this, i); - for (const auto& [key, metadata] : shard->metadata) { - for (const auto& source_segment : job.request.segments) { - const auto unit_key = MakeDrainUnitKey(key, source_segment); - if (job.completed_unit_keys.contains(unit_key) || - active_unit_keys.contains(unit_key) || - job.terminal_failed_unit_keys.contains(unit_key)) { - continue; - } + for (const auto& [tenant_id, tenant_state] : shard->tenants) { + for (const auto& [key, metadata] : tenant_state.metadata) { + for (const auto& source_segment : job.request.segments) { + const auto unit_key = + MakeDrainUnitKey(tenant_id, key, source_segment); + if (job.completed_unit_keys.contains(unit_key) || + active_unit_keys.contains(unit_key) || + job.terminal_failed_unit_keys.contains(unit_key)) { + continue; + } - const auto replica_segments = - metadata.GetReplicaSegmentNames(); - if (std::find(replica_segments.begin(), - replica_segments.end(), - source_segment) == replica_segments.end()) { - continue; - } + const auto replica_segments = + metadata.GetReplicaSegmentNames(); + if (std::find(replica_segments.begin(), + replica_segments.end(), source_segment) == + replica_segments.end()) { + continue; + } - if (metadata.IsHardPinned() || !metadata.IsLeaseExpired() || - !metadata.AllReplicas(&Replica::fn_is_completed) || - shard->replication_tasks.contains(key)) { - blocked_unit_keys.insert(unit_key); - continue; - } + if (tenant_id != "default" || metadata.IsHardPinned() || + !metadata.IsLeaseExpired() || + !metadata.AllReplicas(&Replica::fn_is_completed) || + tenant_state.replication_tasks.contains(key)) { + blocked_unit_keys.insert(unit_key); + continue; + } - auto target = SelectDrainTargetForKey( - metadata, source_segment, job.request.target_segments); - if (!target.has_value()) { - blocked_unit_keys.insert(unit_key); - continue; - } + auto target = SelectDrainTargetForKey( + metadata, source_segment, + job.request.target_segments); + if (!target.has_value()) { + blocked_unit_keys.insert(unit_key); + continue; + } - if (plans.size() < slots) { - plans.push_back({key, source_segment, *target, - metadata.size, unit_key}); + if (plans.size() < slots) { + plans.push_back({key, source_segment, *target, + metadata.size, unit_key}); + } } } } @@ -6684,15 +6836,18 @@ bool MasterService::MaybeCompleteDrainJob(DrainJob& job) { std::shared_lock shared_lock(snapshot_mutex_); for (size_t i = 0; i < kNumShards; ++i) { MetadataShardAccessorRO shard(this, i); - for (const auto& [key, metadata] : shard->metadata) { - const auto replica_segments = metadata.GetReplicaSegmentNames(); - for (const auto& source_segment : job.request.segments) { - if (std::find(replica_segments.begin(), - replica_segments.end(), - source_segment) != replica_segments.end()) { - remaining_segments.insert(source_segment); - remaining_unit_keys.insert( - MakeDrainUnitKey(key, source_segment)); + for (const auto& [tenant_id, tenant_state] : shard->tenants) { + for (const auto& [key, metadata] : tenant_state.metadata) { + const auto replica_segments = + metadata.GetReplicaSegmentNames(); + for (const auto& source_segment : job.request.segments) { + if (std::find(replica_segments.begin(), + replica_segments.end(), source_segment) != + replica_segments.end()) { + remaining_segments.insert(source_segment); + remaining_unit_keys.insert(MakeDrainUnitKey( + tenant_id, key, source_segment)); + } } } } diff --git a/mooncake-store/tests/ha/snapshot/snapshot_child_process_test.cpp b/mooncake-store/tests/ha/snapshot/snapshot_child_process_test.cpp index e0b0e9de..0fd33c22 100644 --- a/mooncake-store/tests/ha/snapshot/snapshot_child_process_test.cpp +++ b/mooncake-store/tests/ha/snapshot/snapshot_child_process_test.cpp @@ -159,7 +159,10 @@ class SnapshotChildProcessTest : public ::testing::Test { size_t shard_idx = svc->getShardIndex(key); auto& shard = svc->metadata_shards_[shard_idx]; SharedMutexLocker lock(&shard.mutex, shared_lock_t{}); - return shard.metadata.find(key) != shard.metadata.end(); + auto tenant_it = shard.tenants.find("default"); + return tenant_it != shard.tenants.end() && + tenant_it->second.metadata.find(key) != + tenant_it->second.metadata.end(); } uint32_t GetShardIndexForTest(const std::string& key) { @@ -175,9 +178,13 @@ class SnapshotChildProcessTest : public ::testing::Test { bool ObjectIsGroupedInMetadata(const std::string& key, size_t shard_idx) { auto& shard = service_->metadata_shards_[shard_idx]; SharedMutexLocker lock(&shard.mutex, shared_lock_t{}); - auto it = shard.metadata.find(key); - EXPECT_NE(it, shard.metadata.end()); - return it != shard.metadata.end() && it->second.IsGrouped(); + for (const auto& [tenant_id, tenant_state] : shard.tenants) { + auto it = tenant_state.metadata.find(key); + if (it != tenant_state.metadata.end()) { + return it->second.IsGrouped(); + } + } + return false; } std::string FindGroupIdOnDifferentShard(MasterService* svc, diff --git a/mooncake-store/tests/master_service_test.cpp b/mooncake-store/tests/master_service_test.cpp index b7c95c0c..1cf903c6 100644 --- a/mooncake-store/tests/master_service_test.cpp +++ b/mooncake-store/tests/master_service_test.cpp @@ -5,6 +5,7 @@ #include #include +#include #include #include #include @@ -619,6 +620,180 @@ TEST_F(MasterServiceTest, GroupedObjectRoutesKeyLevelLookupAndRemove) { EXPECT_FALSE(exists_after_remove.value()); } +TEST_F(MasterServiceTest, GroupRoutingIsTenantScopedForSameUserKey) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "tenant_grouped_shared_user_key"; + const std::string tenant_a = "tenant_group_route_a"; + const std::string tenant_b = "tenant_group_route_b"; + const std::string group_a = FindGroupIdOnDifferentShard(key); + std::string group_b; + for (int i = 0; i < 10000; ++i) { + group_b = key + "_tenant_b_group_" + std::to_string(i); + if (std::hash{}(group_b) % 1024 != + std::hash{}(group_a) % 1024) { + break; + } + } + + ReplicateConfig config_a; + config_a.replica_num = 1; + config_a.group_ids = std::vector{group_a}; + ReplicateConfig config_b; + config_b.replica_num = 1; + config_b.group_ids = std::vector{group_b}; + + ASSERT_TRUE(service_->PutStart(client_id, key, tenant_a, 1024, config_a) + .has_value()); + ASSERT_TRUE(service_->PutEnd(client_id, key, tenant_a, ReplicaType::MEMORY) + .has_value()); + ASSERT_TRUE(service_->PutStart(client_id, key, tenant_b, 2048, config_b) + .has_value()); + ASSERT_TRUE(service_->PutEnd(client_id, key, tenant_b, ReplicaType::MEMORY) + .has_value()); + + EXPECT_TRUE(service_->ExistKey(key, tenant_a).value_or(false)); + EXPECT_TRUE(service_->ExistKey(key, tenant_b).value_or(false)); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_a).has_value()); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_b).has_value()); + + ASSERT_TRUE(service_->Remove(key, tenant_a, /*force=*/true).has_value()); + EXPECT_FALSE(service_->GetReplicaList(key, tenant_a).has_value()); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_b).has_value()); +} + +TEST_F(MasterServiceTest, + ConcurrentGroupedAndUngroupedFirstCreateDoesNotDuplicateMetadata) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "concurrent_grouped_ungrouped_first_create"; + const std::string tenant_id = "tenant_concurrent_first_create"; + ReplicateConfig ungrouped_config; + ungrouped_config.replica_num = 1; + ReplicateConfig grouped_config; + grouped_config.replica_num = 1; + grouped_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(key)}; + + static constexpr size_t kThreadCount = 16; + std::atomic ready{0}; + std::atomic start{false}; + std::vector put_start_success(kThreadCount, 0); + std::vector put_end_success(kThreadCount, 0); + std::vector threads; + threads.reserve(kThreadCount); + + for (size_t i = 0; i < kThreadCount; ++i) { + threads.emplace_back([&, i]() { + ready.fetch_add(1, std::memory_order_acq_rel); + while (!start.load(std::memory_order_acquire)) { + std::this_thread::yield(); + } + const auto& config = + (i % 2 == 0) ? grouped_config : ungrouped_config; + auto put_start = + service_->PutStart(client_id, key, tenant_id, 1024, config); + put_start_success[i] = put_start.has_value() ? 1 : 0; + if (put_start.has_value()) { + put_end_success[i] = service_->PutEnd(client_id, key, tenant_id, + ReplicaType::MEMORY) + .has_value() + ? 1 + : 0; + } else { + EXPECT_EQ(ErrorCode::OBJECT_ALREADY_EXISTS, put_start.error()); + } + }); + } + + while (ready.load(std::memory_order_acquire) < kThreadCount) { + std::this_thread::yield(); + } + start.store(true, std::memory_order_release); + for (auto& thread : threads) { + thread.join(); + } + + EXPECT_EQ(std::count(put_start_success.begin(), put_start_success.end(), 1), + 1); + EXPECT_EQ(std::count(put_end_success.begin(), put_end_success.end(), 1), 1); + EXPECT_EQ(service_->GetKeyCount(), 1u); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_id).has_value()); +} + +TEST_F(MasterServiceTest, + ConcurrentDifferentGroupedFirstCreateDoesNotDuplicateMetadata) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "concurrent_different_grouped_first_create"; + const std::string tenant_id = "tenant_concurrent_grouped_first_create"; + const std::string group_a = FindGroupIdOnDifferentShard(key); + std::string group_b; + for (int i = 0; i < 10000; ++i) { + group_b = key + "_other_group_" + std::to_string(i); + if (std::hash{}(group_b) % 1024 != + std::hash{}(group_a) % 1024) { + break; + } + } + ReplicateConfig config_a; + config_a.replica_num = 1; + config_a.group_ids = std::vector{group_a}; + ReplicateConfig config_b; + config_b.replica_num = 1; + config_b.group_ids = std::vector{group_b}; + + static constexpr size_t kThreadCount = 16; + std::atomic ready{0}; + std::atomic start{false}; + std::vector put_start_success(kThreadCount, 0); + std::vector put_end_success(kThreadCount, 0); + std::vector threads; + threads.reserve(kThreadCount); + + for (size_t i = 0; i < kThreadCount; ++i) { + threads.emplace_back([&, i]() { + ready.fetch_add(1, std::memory_order_acq_rel); + while (!start.load(std::memory_order_acquire)) { + std::this_thread::yield(); + } + const auto& config = (i % 2 == 0) ? config_a : config_b; + auto put_start = + service_->PutStart(client_id, key, tenant_id, 1024, config); + put_start_success[i] = put_start.has_value() ? 1 : 0; + if (put_start.has_value()) { + put_end_success[i] = service_->PutEnd(client_id, key, tenant_id, + ReplicaType::MEMORY) + .has_value() + ? 1 + : 0; + } else { + EXPECT_EQ(ErrorCode::OBJECT_ALREADY_EXISTS, put_start.error()); + } + }); + } + + while (ready.load(std::memory_order_acquire) < kThreadCount) { + std::this_thread::yield(); + } + start.store(true, std::memory_order_release); + for (auto& thread : threads) { + thread.join(); + } + + EXPECT_EQ(std::count(put_start_success.begin(), put_start_success.end(), 1), + 1); + EXPECT_EQ(std::count(put_end_success.begin(), put_end_success.end(), 1), 1); + EXPECT_EQ(service_->GetKeyCount(), 1u); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_id).has_value()); +} + TEST_F(MasterServiceTest, ExpiredGroupedPutCanBeReplacedByUngroupedPut) { auto service_config = MasterServiceConfig::builder() .set_put_start_discard_timeout_sec(0) @@ -647,6 +822,52 @@ TEST_F(MasterServiceTest, ExpiredGroupedPutCanBeReplacedByUngroupedPut) { EXPECT_TRUE(service_->ExistKey(key).value_or(false)); } +TEST_F(MasterServiceTest, BatchRemoveUnregistersGroupedRoute) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "batch_remove_grouped_route"; + ReplicateConfig grouped_config; + grouped_config.replica_num = 1; + grouped_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(key)}; + PutCompletedObject(*service_, client_id, key, grouped_config); + + auto remove_results = + service_->BatchRemove(std::vector{key}, /*force=*/true); + ASSERT_EQ(remove_results.size(), 1u); + ASSERT_TRUE(remove_results[0].has_value()); + + ReplicateConfig ungrouped_config; + ungrouped_config.replica_num = 1; + PutCompletedObject(*service_, client_id, key, ungrouped_config); + EXPECT_TRUE(service_->GetReplicaList(key).has_value()); +} + +TEST_F(MasterServiceTest, RemoveByRegexUnregistersGroupedRoute) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "regex_remove_grouped_route"; + ReplicateConfig grouped_config; + grouped_config.replica_num = 1; + grouped_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(key)}; + PutCompletedObject(*service_, client_id, key, grouped_config); + + auto removed = service_->RemoveByRegex("^regex_remove_grouped_route$", + /*force=*/true); + ASSERT_TRUE(removed.has_value()); + EXPECT_EQ(removed.value(), 1); + + ReplicateConfig ungrouped_config; + ungrouped_config.replica_num = 1; + PutCompletedObject(*service_, client_id, key, ungrouped_config); + EXPECT_TRUE(service_->GetReplicaList(key).has_value()); +} + TEST_F(MasterServiceTest, GroupedLeaseRefreshNearExpiryProtectsCurrentMembers) { auto service_config = MasterServiceConfig::builder().set_default_kv_lease_ttl(200).build(); @@ -813,6 +1034,28 @@ TEST_F(MasterServiceTest, IncompleteGroupedUpsertCanBecomeUngrouped) { EXPECT_TRUE(service_->ExistKey(key).value_or(false)); } +TEST_F(MasterServiceTest, UpsertRejectsExistingUngroupedToGrouped) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "upsert_ungrouped_to_grouped"; + ReplicateConfig ungrouped_config; + ungrouped_config.replica_num = 1; + PutCompletedObject(*service_, client_id, key, ungrouped_config); + + ReplicateConfig grouped_config; + grouped_config.replica_num = 1; + grouped_config.group_ids = + std::vector{FindGroupIdOnDifferentShard(key)}; + auto upsert_start = + service_->UpsertStart(client_id, key, 2048, grouped_config); + ASSERT_FALSE(upsert_start.has_value()); + EXPECT_EQ(ErrorCode::INVALID_PARAMS, upsert_start.error()); + + EXPECT_TRUE(service_->GetReplicaList(key).has_value()); +} + TEST_F(MasterServiceTest, GroupedEvictionExpandsSafeMembersAndSkipsLeasedGroup) { auto service_config = @@ -1084,6 +1327,83 @@ TEST_F(MasterServiceTest, PutStartEndFlow) { EXPECT_EQ(ReplicaStatus::COMPLETE, replica_list[0].status); } +TEST_F(MasterServiceTest, TenantPutGetRemoveIsolatesSameUserKey) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "shared_user_key"; + const std::string tenant_a = "tenant_a"; + const std::string tenant_b = "tenant_b"; + ReplicateConfig config; + config.replica_num = 1; + + ASSERT_TRUE( + service_->PutStart(client_id, key, tenant_a, 1024, config).has_value()); + ASSERT_TRUE(service_->PutEnd(client_id, key, tenant_a, ReplicaType::MEMORY) + .has_value()); + ASSERT_TRUE( + service_->PutStart(client_id, key, tenant_b, 2048, config).has_value()); + ASSERT_TRUE(service_->PutEnd(client_id, key, tenant_b, ReplicaType::MEMORY) + .has_value()); + + EXPECT_FALSE(service_->GetReplicaList(key).has_value()); + EXPECT_FALSE(service_->ExistKey(key).value()); + EXPECT_TRUE(service_->ExistKey(key, tenant_a).value()); + EXPECT_TRUE(service_->ExistKey(key, tenant_b).value()); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_a).has_value()); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_b).has_value()); + EXPECT_EQ(service_->GetKeyCount(), 2u); + + ASSERT_TRUE(service_->Remove(key, tenant_a, /*force=*/true).has_value()); + EXPECT_FALSE(service_->GetReplicaList(key, tenant_a).has_value()); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_b).has_value()); + EXPECT_EQ(service_->GetKeyCount(), 1u); +} + +TEST_F(MasterServiceTest, RegexOperationsAreTenantScoped) { + std::unique_ptr service_(new MasterService()); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*service_); + const UUID client_id = generate_uuid(); + + const std::string key = "regex_shared_key"; + const std::string tenant_a = "tenant_regex_a"; + const std::string tenant_b = "tenant_regex_b"; + ReplicateConfig config; + config.replica_num = 1; + + ASSERT_TRUE(service_->PutStart(client_id, key, 1024, config).has_value()); + ASSERT_TRUE( + service_->PutEnd(client_id, key, ReplicaType::MEMORY).has_value()); + ASSERT_TRUE( + service_->PutStart(client_id, key, tenant_a, 1024, config).has_value()); + ASSERT_TRUE(service_->PutEnd(client_id, key, tenant_a, ReplicaType::MEMORY) + .has_value()); + ASSERT_TRUE( + service_->PutStart(client_id, key, tenant_b, 1024, config).has_value()); + ASSERT_TRUE(service_->PutEnd(client_id, key, tenant_b, ReplicaType::MEMORY) + .has_value()); + + auto default_matches = service_->GetReplicaListByRegex("^regex_shared"); + ASSERT_TRUE(default_matches.has_value()); + EXPECT_EQ(default_matches->size(), 1); + + auto remove_default = + service_->RemoveByRegex("^regex_shared", /*force=*/true); + ASSERT_TRUE(remove_default.has_value()); + EXPECT_EQ(remove_default.value(), 1); + EXPECT_FALSE(service_->GetReplicaList(key).has_value()); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_a).has_value()); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_b).has_value()); + + auto remove_tenant_a = + service_->RemoveByRegex("^regex_shared", tenant_a, /*force=*/true); + ASSERT_TRUE(remove_tenant_a.has_value()); + EXPECT_EQ(remove_tenant_a.value(), 1); + EXPECT_FALSE(service_->GetReplicaList(key, tenant_a).has_value()); + EXPECT_TRUE(service_->GetReplicaList(key, tenant_b).has_value()); +} + TEST_F(MasterServiceTest, PutWithPreferredSegment) { // For backward compatibility, test the deprecated single preferred_segment std::unique_ptr service_(new MasterService()); -- 2.34.1 From daa44a477aced13932b8e7e0cbabba145b4f5d15 Mon Sep 17 00:00:00 2001 From: Yuhui Liang <86229177+yuhuiaws@users.noreply.github.com> Date: Sun, 31 May 2026 07:30:40 +0800 Subject: [PATCH 151/382] [EFA] Add MC_EFA_CQ_THREADS env var to cap CQ poller threads (#2113) * [EFA] Add MC_EFA_CQ_THREADS env var and reduce idle CPU spin Two changes to EFA transport CQ polling: 1. Add MC_EFA_CQ_THREADS environment variable to cap the number of CQ polling threads. When running multiple EFA consumers (e.g. KV transfer + DeepEP all-to-all) in the same process, each creates threads per context. This allows limiting contention. 2. Replace std::this_thread::yield() with sleep_for(10us) in the idle path of workerThreadFunc. yield() on Linux compiles to sched_yield() which busy-spins at 100% CPU when there is no CQ work, wasting cores that could serve other EFA consumers. Co-Authored-By: Claude Opus 4.6 * refactor: move MC_EFA_CQ_THREADS to Environ singleton, revert yield change Address reviewer feedback: - Register MC_EFA_CQ_THREADS in Environ with GetInt (default 0 = unset) - Use Environ::Get().GetEfaCqThreads() instead of raw std::getenv/stoull - Revert yield() -> sleep_for() change (keep original yield behavior) - Update comment to explain when/why the cap is useful Co-Authored-By: Claude Opus 4.6 * docs: add MC_EFA_CQ_THREADS documentation to EFA transport guide Co-Authored-By: Claude Opus 4.6 * fix: change MC_EFA_CQ_THREADS default to 1 to reduce idle CPU spin Benchmarks on p5.48xlarge show cap=1 reaches 99.93% of peak GPU-to-GPU throughput (386.22 vs 386.48 GB/s) while freeing 31 cores from busy-spin. Set MC_EFA_CQ_THREADS=0 to restore the legacy one-poller-per-context behavior. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Ubuntu Co-authored-by: Claude Opus 4.6 --- docs/source/design/transfer-engine/efa_transport.md | 9 +++++++++ mooncake-common/include/environ.h | 2 ++ mooncake-common/src/environ.cpp | 1 + .../src/transport/efa_transport/efa_transport.cpp | 8 +++++++- 4 files changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/source/design/transfer-engine/efa_transport.md b/docs/source/design/transfer-engine/efa_transport.md index 669d3a28..bc2a1479 100644 --- a/docs/source/design/transfer-engine/efa_transport.md +++ b/docs/source/design/transfer-engine/efa_transport.md @@ -523,6 +523,15 @@ export LD_LIBRARY_PATH=/opt/amazon/efa/lib:$LD_LIBRARY_PATH > **Note on additional `MC_*` knobs:** `MC_NUM_CQ_PER_CTX`, `MC_MAX_WR`, `MC_MAX_CQE_PER_CTX`, `MC_SLICE_SIZE`, and `MC_EFA_STRIPING_THRESHOLD` are **not** required at typical PD-disagg loads — the SRD shared-endpoint refactor (#1944) makes them redundant up to high concurrency on 1k/1k traffic. Treat them as emergency switches for CQ-overflow or long-running drift symptoms. +> **`MC_EFA_CQ_THREADS`** — caps the number of CQ polling threads spawned by the EFA transport. Default is `1`, which reaches 99.93% of peak GPU-to-GPU throughput while saving CPU for other workloads. Set to `0` to disable the cap (one poller per EFA context — the legacy behavior). Higher values (e.g., `MC_EFA_CQ_THREADS=4`) are available as an escape hatch for throughput tuning but rarely help in practice. +> +> ```bash +> export MC_EFA_CQ_THREADS=1 # default: single CQ poller (recommended) +> export MC_EFA_CQ_THREADS=0 # disable cap: one poller per EFA context (legacy) +> ``` +> +> If the value exceeds the number of EFA contexts, it is safely ignored (no excess threads are created). + ### 3. Prefill Instance ```bash diff --git a/mooncake-common/include/environ.h b/mooncake-common/include/environ.h index 56106145..474af488 100644 --- a/mooncake-common/include/environ.h +++ b/mooncake-common/include/environ.h @@ -50,6 +50,7 @@ class Environ { bool GetIntraNvlink() const { return intra_nvlink_; } bool GetPathRoundrobin() const { return path_roundrobin_; } bool GetWithNvidiaPeermem() const { return with_nvidia_peermem_; } + int GetEfaCqThreads() const { return efa_cq_threads_; } private: Environ(); @@ -101,6 +102,7 @@ class Environ { bool intra_nvlink_; bool path_roundrobin_; bool with_nvidia_peermem_; + int efa_cq_threads_; }; } // namespace mooncake diff --git a/mooncake-common/src/environ.cpp b/mooncake-common/src/environ.cpp index cb2bfa2d..e45277a0 100644 --- a/mooncake-common/src/environ.cpp +++ b/mooncake-common/src/environ.cpp @@ -82,6 +82,7 @@ Environ::Environ() { intra_nvlink_ = GetBool("MC_INTRA_NVLINK", false); path_roundrobin_ = GetBool("MC_PATH_ROUNDROBIN", false); with_nvidia_peermem_ = GetBool("WITH_NVIDIA_PEERMEM", true); + efa_cq_threads_ = GetInt("MC_EFA_CQ_THREADS", 1); } } // namespace mooncake diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp index 3106de71..f9d876c3 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_transport.cpp @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -32,6 +33,7 @@ #include "common.h" #include "config.h" +#include "environ.h" #include "memory_location.h" #include "topology.h" #include "transport/efa_transport/efa_context.h" @@ -105,8 +107,12 @@ void EfaTransport::startWorkerThreads() { if (worker_running_) return; worker_running_ = true; - // One poller thread per context for responsive CQ draining under load + // MC_EFA_CQ_THREADS caps CQ poller count (default 1). Set 0 to disable cap. size_t num_threads = context_list_.size(); + int cq_cap = Environ::Get().GetEfaCqThreads(); + if (cq_cap > 0 && static_cast(cq_cap) < num_threads) { + num_threads = static_cast(cq_cap); + } for (size_t i = 0; i < num_threads; i++) { worker_threads_.emplace_back(&EfaTransport::workerThreadFunc, this, i); } -- 2.34.1 From 766b83a6fd6d9664cbee0c5794646877f691a490 Mon Sep 17 00:00:00 2001 From: andyluo7 <43718156+andyluo7@users.noreply.github.com> Date: Sun, 31 May 2026 10:00:02 -0700 Subject: [PATCH 152/382] [TransferEngine][ROCm] Add HIP dmabuf MR registration for AMD GPUs (fixes #751) (#2225) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [TransferEngine][ROCm] Add HIP dmabuf MR registration for AMD GPUs Fixes #751. Adds a parallel `#elif defined(USE_HIP)` branch in RdmaContext::registerMemoryRegionInternal that mirrors the existing CUDA dmabuf path (added by #704) using ROCm's `hsa_amd_portable_export_dmabuf()` instead of `cuMemGetHandleForAddressRange(...DMA_BUF_FD...)`. This lets Mooncake register AMD GPU memory for RDMA without requiring an nvidia-peermem-equivalent kernel module — the path UCX's ROCm backend (uct/rocm/base/rocm_base.c) already uses successfully. Same host-vs-device split as the CUDA branch: `hipPointerGetAttributes` detects host memory and falls back to `ibv_reg_mr`; device/managed memory goes through the dmabuf path. `hipMemGetAddressRange` is used to get the true allocation base because `addr` may sit at an offset within a larger hipMalloc block (caching allocators pack tensors). CMake: added `hsa-runtime64` to the HIP link line in mooncake-transfer-engine/src/CMakeLists.txt. Validation: - Standalone dmabuf probe verified PASS on: * AMD MI355X (gfx950) + Pensando ionic + ROCm 7.2.2 * AMD MI300X (gfx942) + Broadcom Thor2 (bnxt_re) + ROCm 7.0.2 Probe source + container recipe: https://github.com/andyluo7/dynamo/blob/amd-poc-consumer-polish/amd-mi355x-poc/advanced/debug-probes/dmabuf_register_probe.cpp - Standalone compile check confirms all HIP/HSA/ibverbs symbols in the new branch resolve and link cleanly with hsa-runtime64 + libibverbs. End-to-end SGLang+Mooncake disagg validation (T3) on MI355X+ionic will follow in a comment once a full Mooncake build with submodules completes. CC @misterwilliam @stmatengss @alogfans (active on #751) Closes #751 --------- Signed-off-by: Andy Luo Signed-off-by: Andy Luo Co-authored-by: Claude Sonnet 4 --- .typos.toml | 5 +- mooncake-transfer-engine/src/CMakeLists.txt | 16 ++ .../transport/rdma_transport/rdma_context.cpp | 184 ++++++++++++++++++ 3 files changed, 204 insertions(+), 1 deletion(-) diff --git a/.typos.toml b/.typos.toml index b3fe85df..5ba9fb61 100644 --- a/.typos.toml +++ b/.typos.toml @@ -1,11 +1,14 @@ [default] -extend-ignore-words = ["CANN", "ASO", "fre", "wqs"] +extend-ignore-words = ["CANN", "ASO", "fre", "wqs", "hsa"] [default.extend-words] CANN = "CANN" ASO = "ASO" fre = "fre" wqs = "wqs" +# AMD HSA runtime symbol prefix (hsa_*, hsaRes, hsaErr, etc.) — used by the +# ROCm dmabuf MR registration path. +hsa = "hsa" [files] extend-exclude = [ diff --git a/mooncake-transfer-engine/src/CMakeLists.txt b/mooncake-transfer-engine/src/CMakeLists.txt index 80173547..336a1fa6 100644 --- a/mooncake-transfer-engine/src/CMakeLists.txt +++ b/mooncake-transfer-engine/src/CMakeLists.txt @@ -96,6 +96,22 @@ endif() if(USE_HIP) target_include_directories(transfer_engine PRIVATE ${HIP_INCLUDE_DIRS}) target_link_libraries(transfer_engine PUBLIC hip::host rt) + + # Optional dmabuf MR registration path (requires hsa-runtime64 and a + # kernel with CONFIG_PCI_P2PDMA + CONFIG_DMABUF_MOVE_NOTIFY). + option(USE_HIP_DMABUF "Enable HIP dmabuf RDMA MR registration" ON) + if(USE_HIP_DMABUF) + find_package(hsa-runtime64 CONFIG) + if(hsa-runtime64_FOUND) + target_compile_definitions(transfer_engine PRIVATE USE_HIP_DMABUF) + target_link_libraries(transfer_engine PUBLIC hsa-runtime64::hsa-runtime64) + message(STATUS "HIP dmabuf MR registration enabled (hsa-runtime64 found)") + else() + message(STATUS "HIP dmabuf MR registration disabled (hsa-runtime64 not found)") + endif() + else() + message(STATUS "HIP dmabuf MR registration disabled (USE_HIP_DMABUF=OFF)") + endif() endif() if(USE_MLU) diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index 1b8b79e2..79cbb780 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -30,6 +30,14 @@ #include "config.h" #include "cuda_alike.h" #include "environ.h" +#if defined(USE_HIP_DMABUF) +#include + +#include + +#include +#include +#endif #include "transport/rdma_transport/endpoint_store.h" #include "transport/rdma_transport/rdma_endpoint.h" #include "transport/rdma_transport/rdma_transport.h" @@ -50,6 +58,89 @@ bool containsAddress(const MemoryRegionMeta ®ion, uintptr_t addr) { const auto region_length = static_cast(region.mr->length); return region_start <= addr && addr - region_start < region_length; } + +#if defined(USE_HIP_DMABUF) +// Returns true when the kernel has CONFIG_PCI_P2PDMA and +// CONFIG_DMABUF_MOVE_NOTIFY enabled, which are required for ibv_reg_dmabuf_mr +// to produce working RDMA transfers (not just successful registration). +// Result is cached after the first call. +bool isKernelDmabufSupported() { + static const bool supported = []() { + if (const char *env = std::getenv("MOONCAKE_DISABLE_HIP_DMABUF")) { + if (std::string(env) != "0") { + LOG(INFO) + << "HIP dmabuf disabled via MOONCAKE_DISABLE_HIP_DMABUF"; + return false; + } + } + struct utsname uts{}; + std::string release; + if (uname(&uts) == 0) release = uts.release; + + const char *needles[] = {"CONFIG_PCI_P2PDMA=y", + "CONFIG_DMABUF_MOVE_NOTIFY=y"}; + std::vector found(2, false); + + const std::vector candidates = { + "/proc/config.gz", // compressed; checked below via gzip-magic skip + "/boot/config-" + release, + "/usr/src/linux-" + release + "/.config", + "/usr/src/linux/.config", + "/usr/lib/modules/" + release + "/config", + "/usr/lib/ostree-boot/config-" + release, + "/usr/lib/kernel/config-" + release, + "/usr/src/linux-headers-" + release + "/.config", + "/lib/modules/" + release + "/build/.config", + }; + + for (const auto &path : candidates) { + std::ifstream f(path); + if (!f.good()) continue; + // /proc/config.gz is gzipped; we skip it here (the kallsyms + // fallback below covers that case in practice). + if (path.find(".gz") != std::string::npos) continue; + std::string line; + while (std::getline(f, line)) { + for (size_t i = 0; i < 2; ++i) { + if (!found[i] && line.find(needles[i]) != std::string::npos) + found[i] = true; + } + if (found[0] && found[1]) break; + } + if (found[0] && found[1]) break; + } + + // Fallback: probe /proc/kallsyms for the corresponding kernel symbols. + if (!found[0] || !found[1]) { + std::ifstream f("/proc/kallsyms"); + if (f.good()) { + std::string line; + while (std::getline(f, line)) { + if (!found[0] && + line.find("pci_p2pdma") != std::string::npos) + found[0] = true; + if (!found[1] && + line.find("dma_buf_move_notify") != std::string::npos) + found[1] = true; + if (found[0] && found[1]) break; + } + } + } + + bool ok = found[0] && found[1]; + if (!ok) { + LOG(WARNING) + << "Kernel lacks CONFIG_PCI_P2PDMA / CONFIG_DMABUF_MOVE_NOTIFY " + << "(p2pdma=" << found[0] << " move_notify=" << found[1] + << "); HIP dmabuf MR registration disabled, falling back to " + << "ibv_reg_mr() (which requires an amdgpu peermem driver). " + << "Rebuild kernel with both options for GPU-direct RDMA."; + } + return ok; + }(); + return supported; +} +#endif // USE_HIP_DMABUF } // namespace RdmaContext::RdmaContext(RdmaTransport &engine, const std::string &device_name) @@ -338,6 +429,99 @@ int RdmaContext::registerMemoryRegionInternal(void *addr, size_t length, cuDevicePrimaryCtxRelease(cuDev); #endif } +#elif defined(USE_HIP_DMABUF) + hipPointerAttribute_t hipAttr{}; + hipError_t hipRes = hipPointerGetAttributes(&hipAttr, addr); + + if (hipRes != hipSuccess || hipAttr.type == hipMemoryTypeHost || + hipAttr.type == hipMemoryTypeUnregistered) { + // Host memory — standard ibv_reg_mr() path. + mrMeta.addr = addr; + mrMeta.mr = ibv_reg_mr(pd_, addr, length, access); + } else if (hipAttr.type == hipMemoryTypeManaged) { + // Managed (unified) memory pages can migrate between host and device; + // hsa_amd_portable_export_dmabuf captures the device-side handle at + // export time only, making the dmabuf fd stale after migration. Fall + // back to ibv_reg_mr() for safety. + LOG(WARNING) << "HIP managed memory at " << (uintptr_t)addr + << " — dmabuf export skipped (pages may migrate); " + "falling back to ibv_reg_mr"; + mrMeta.addr = addr; + mrMeta.mr = ibv_reg_mr(pd_, addr, length, access); + } else if (hipAttr.type == hipMemoryTypeDevice && + !isKernelDmabufSupported()) { + // Kernel lacks CONFIG_PCI_P2PDMA / CONFIG_DMABUF_MOVE_NOTIFY — + // ibv_reg_dmabuf_mr may succeed but transfers will silently fail. + // Fail at registration time instead. + mrMeta.addr = addr; + mrMeta.mr = ibv_reg_mr(pd_, addr, length, access); + } else if (hipAttr.type == hipMemoryTypeDevice) { + // Device memory + kernel support — export dmabuf fd and register. + // Pin to the owning device for the duration of the export calls. + struct HipDeviceGuard { + int prev_device = 0; + bool need_restore = false; + bool set_ok = false; + explicit HipDeviceGuard(int target_device) { + if (hipGetDevice(&prev_device) == hipSuccess) { + need_restore = (prev_device != target_device); + } + set_ok = (hipSetDevice(target_device) == hipSuccess); + } + ~HipDeviceGuard() { + if (need_restore) { + (void)hipSetDevice(prev_device); + } + } + } dev_guard(hipAttr.device); + if (!dev_guard.set_ok) { + LOG(ERROR) << "Failed to set HIP device to " << hipAttr.device + << " for dmabuf export of " << (uintptr_t)addr; + return ERR_CONTEXT; + } + + // Get the allocation base + size, since `addr` may sit at an offset + // within a larger hipMalloc block (caching allocators pack tensors). + hipDeviceptr_t allocBase = nullptr; + size_t allocSize = 0; + hipRes = hipMemGetAddressRange(&allocBase, &allocSize, + reinterpret_cast(addr)); + if (hipRes != hipSuccess) { + LOG(ERROR) << "Failed to call hipMemGetAddressRange for " + << (uintptr_t)addr + << " hip error=" << hipGetErrorString(hipRes); + return ERR_CONTEXT; + } + + int dmabuf_fd = -1; + uint64_t hsa_dmabuf_offset = 0; + hsa_status_t hsaRes = hsa_amd_portable_export_dmabuf( + allocBase, allocSize, &dmabuf_fd, &hsa_dmabuf_offset); + if (hsaRes != HSA_STATUS_SUCCESS) { + const char *hsaErr = nullptr; + hsa_status_string(hsaRes, &hsaErr); + LOG(ERROR) << "Failed to retrieve dmabuf for " << (uintptr_t)addr + << " base=" << (uintptr_t)allocBase + << " size=" << allocSize + << " hsa error=" << (hsaErr ? hsaErr : "unknown"); + return ERR_CONTEXT; + } + + mrMeta.addr = addr; + // Offset within the dmabuf-backed region: distance from the + // allocation base, plus any offset hsa returned for the export. + uint64_t reg_offset = + (uintptr_t)addr - (uintptr_t)allocBase + hsa_dmabuf_offset; + mrMeta.mr = ibv_reg_dmabuf_mr(pd_, reg_offset, length, (uintptr_t)addr, + dmabuf_fd, access); + const int regErrno = errno; + if (close(dmabuf_fd) != 0) { + PLOG(WARNING) << "Failed to close dmabuf fd"; + } + if (!mrMeta.mr) { + errno = regErrno; + } + } #else mrMeta.addr = addr; mrMeta.mr = ibv_reg_mr(pd_, addr, length, access); -- 2.34.1 From 591aecdecacb7854834ed8f5075d1192a45f0942 Mon Sep 17 00:00:00 2001 From: Chuang Zhang Date: Mon, 1 Jun 2026 10:29:39 +0800 Subject: [PATCH 153/382] [Store] enables the Ubtransport for Mooncake Store And optimize UrmaEndpoint (#2196) --- .github/workflows/ci.yml | 2 +- mooncake-common/FindUrma.cmake | 2 +- mooncake-p2p-store/build.sh | 4 + mooncake-store/include/real_client.h | 11 + mooncake-store/src/client_service.cpp | 17 ++ mooncake-store/src/real_client.cpp | 4 + mooncake-store/src/utils.cpp | 13 +- .../include/CMakeLists.txt | 1 + .../transport/kunpeng_transport/ub_context.h | 6 + .../include/transport/transport.h | 1 + .../include/ub_allocator.h | 11 + .../kunpeng_transport/CMakeLists.txt | 2 +- .../kunpeng_transport/ub_allocator.cpp | 76 +++++ .../kunpeng_transport/ub_context.cpp | 40 ++- .../kunpeng_transport/urma/mock_urma.cpp | 259 +++++++++++------- .../kunpeng_transport/urma/urma_endpoint.cpp | 17 +- scripts/build_wheel.sh | 1 + 17 files changed, 364 insertions(+), 103 deletions(-) create mode 100644 mooncake-transfer-engine/include/ub_allocator.h create mode 100644 mooncake-transfer-engine/src/transport/kunpeng_transport/ub_allocator.cpp diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 27555d70..003c1c00 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -184,7 +184,7 @@ jobs: cd build export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib ldconfig -v || echo "always continue" - MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -j --output-on-failure -E ub_transport_test + MC_METADATA_SERVER=http://127.0.0.1:8080/metadata DEFAULT_KV_LEASE_TTL=500 ctest -j --output-on-failure shell: bash - name: Drain HTTP E2E test diff --git a/mooncake-common/FindUrma.cmake b/mooncake-common/FindUrma.cmake index d2d93cc3..0af8d1a7 100644 --- a/mooncake-common/FindUrma.cmake +++ b/mooncake-common/FindUrma.cmake @@ -4,7 +4,7 @@ include(FetchContent) FetchContent_Declare( urma GIT_REPOSITORY https://atomgit.com/openeuler/umdk.git - GIT_TAG v25.12.0 + GIT_TAG v25.12.0.B081 ) FetchContent_MakeAvailable(urma) diff --git a/mooncake-p2p-store/build.sh b/mooncake-p2p-store/build.sh index 66eff5f1..72aca4b3 100644 --- a/mooncake-p2p-store/build.sh +++ b/mooncake-p2p-store/build.sh @@ -53,6 +53,10 @@ if [ -d "/usr/local/musa/lib" ]; then EXT_LDFLAGS+=" -L/usr/local/musa/lib -lmusart" fi +if [ -e "/usr/lib64/liburma.so" ]; then + EXT_LDFLAGS+=" -L/usr/lib64 -lurma" +fi + if [ "$USE_ETCD" = "ON" ]; then if [ "$USE_ETCD_LEGACY" = "ON" ]; then EXT_LDFLAGS+=" -letcd-cpp-api -lprotobuf -lgrpc++ -lgrpc" diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index bcf9b674..0f73eddd 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -766,11 +766,22 @@ class RealClient : public PyClient { } }; + struct UbSegmentDeleter { + size_t size = 0; + std::string protocol = "ub"; + void operator()(void *ptr) const { + if (ptr && size > 0) { + free_memory(protocol.c_str(), ptr); + } + } + }; + std::vector> hugepage_segment_ptrs_; std::vector> segment_ptrs_; std::vector> ascend_segment_ptrs_; + std::vector> ub_segment_ptrs_; std::string protocol; std::string device_name; std::string local_hostname; diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 68614fbf..c1badfd2 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -784,6 +784,23 @@ ErrorCode Client::InitTransferEngine( LOG(ERROR) << "Failed to install CXL transport"; return ErrorCode::INTERNAL_ERROR; } + } else if (protocol == "ub") { + auto deviceName = device_names.value_or("bonding_dev_0"); + LOG(ERROR) << "ub protocol entable device names is " << deviceName; + auto devices = splitString(deviceName, ',', true); + auto topology = transfer_engine_->getLocalTopology(); + if (topology) { + topology->discover(devices); + LOG(INFO) << "Topology discovery complete with specified " + "devices. Found " + << topology->getHcaList().size() << " HCAs"; + } + transport = transfer_engine_->installTransport("ub", nullptr); + if (!transport) { + LOG(ERROR) << "Failed to install ub transport with specified " + "devices"; + return ErrorCode::INTERNAL_ERROR; + } } else { LOG(ERROR) << "unsupported_protocol protocol=" << protocol; return ErrorCode::INVALID_PARAMS; diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index b0cab936..0af42bbc 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -850,6 +850,9 @@ tl::expected RealClient::setup_internal( if (this->protocol == "ascend" || this->protocol == "ubshmem") { ascend_segment_ptrs_.emplace_back( ptr, AscendSegmentDeleter{this->protocol}); + } else if (this->protocol == "ub") { + ub_segment_ptrs_.emplace_back(ptr, + UbSegmentDeleter{mapped_size}); } else if (!seg_numa_nodes.empty() || should_use_hugepage) { // NUMA-segmented or hugepage: track as mmap allocation for // munmap cleanup @@ -1106,6 +1109,7 @@ tl::expected RealClient::tearDownAll_internal() { client_buffer_allocator_.reset(); port_binder_.reset(); hugepage_segment_ptrs_.clear(); + ub_segment_ptrs_.clear(); segment_ptrs_.clear(); local_hostname = ""; device_name = ""; diff --git a/mooncake-store/src/utils.cpp b/mooncake-store/src/utils.cpp index 2b9639fd..810a7cf8 100644 --- a/mooncake-store/src/utils.cpp +++ b/mooncake-store/src/utils.cpp @@ -2,6 +2,7 @@ #include "mmap_arena.h" #include "config.h" #include "common.h" +#include "ub_allocator.h" #include #include @@ -117,6 +118,11 @@ void *allocate_buffer_allocator_memory(size_t total_size, return ascend_allocate_memory(total_size, protocol); } #endif +#if defined(USE_UB) + if (protocol == "ub") { + return mooncake::ub_allocate_memory(alignment, total_size); + } +#endif #ifdef USE_NOF if (use_spdk_dma && total_size > 0) { return mooncake::SpdkWrapper::GetInstance().Alloc(total_size, alignment, @@ -371,7 +377,12 @@ void free_memory(const std::string &protocol, void *ptr) { return ascend_free_memory(protocol, ptr); } #endif - +#if defined(USE_UB) + if (protocol == "ub") { + mooncake::ub_free_memory(ptr); + return; + } +#endif free(ptr); } diff --git a/mooncake-transfer-engine/include/CMakeLists.txt b/mooncake-transfer-engine/include/CMakeLists.txt index 4f52e1e5..56929077 100644 --- a/mooncake-transfer-engine/include/CMakeLists.txt +++ b/mooncake-transfer-engine/include/CMakeLists.txt @@ -7,5 +7,6 @@ install(FILES multi_transport.h DESTINATION include) install(FILES topology.h DESTINATION include) install(FILES transfer_engine.h DESTINATION include) install(FILES transfer_metadata.h DESTINATION include) +install(FILES ub_allocator.h DESTINATION include) install(FILES common/base/status.h DESTINATION include/common/base) install(FILES transport/transport.h DESTINATION include/transport) diff --git a/mooncake-transfer-engine/include/transport/kunpeng_transport/ub_context.h b/mooncake-transfer-engine/include/transport/kunpeng_transport/ub_context.h index 7240ff7b..fd7f9371 100644 --- a/mooncake-transfer-engine/include/transport/kunpeng_transport/ub_context.h +++ b/mooncake-transfer-engine/include/transport/kunpeng_transport/ub_context.h @@ -83,6 +83,7 @@ class UbEndpointStore { virtual std::shared_ptr insertEndpoint( const std::string& peer_nic_path, UbContext* context) = 0; virtual int deleteEndpoint(const std::string& peer_nic_path) = 0; + virtual int deleteEndpointByPtr(UbEndPoint* point_ptr) = 0; virtual void evictEndpoint() = 0; virtual void reclaimEndpoint() = 0; virtual size_t getSize() = 0; @@ -102,6 +103,7 @@ class UbSIEVEEndpointStore : public UbEndpointStore { std::shared_ptr insertEndpoint(const std::string& peer_nic_path, UbContext* context) override; int deleteEndpoint(const std::string& peer_nic_path) override; + int deleteEndpointByPtr(UbEndPoint* point_ptr) override; void evictEndpoint() override; void reclaimEndpoint() override; size_t getSize() override; @@ -229,6 +231,10 @@ class UbContext { return endpoint_store_->deleteEndpoint(peer_nic_path); } + int deleteEndpointByPtr(UbEndPoint* point_ptr) { + return endpoint_store_->deleteEndpointByPtr(point_ptr); + } + int disconnectAllEndpoints() { return endpoint_store_->disconnect(); } // Device name, such as `mlx5_3` diff --git a/mooncake-transfer-engine/include/transport/transport.h b/mooncake-transfer-engine/include/transport/transport.h index 60f46fba..aca0a0d1 100644 --- a/mooncake-transfer-engine/include/transport/transport.h +++ b/mooncake-transfer-engine/include/transport/transport.h @@ -132,6 +132,7 @@ class Transport { uint32_t max_retry_cnt; void *r_seg; void *l_seg; + void *endpoint; } ub; struct { void *dest_addr; diff --git a/mooncake-transfer-engine/include/ub_allocator.h b/mooncake-transfer-engine/include/ub_allocator.h new file mode 100644 index 00000000..a753f421 --- /dev/null +++ b/mooncake-transfer-engine/include/ub_allocator.h @@ -0,0 +1,11 @@ +#pragma once + +namespace mooncake { + +void* ub_allocate_memory(size_t alignment, size_t total_size); + +void ub_free_memory(void* ptr); + +bool ub_is_store_memory(void* addr, size_t length); + +} // namespace mooncake \ No newline at end of file diff --git a/mooncake-transfer-engine/src/transport/kunpeng_transport/CMakeLists.txt b/mooncake-transfer-engine/src/transport/kunpeng_transport/CMakeLists.txt index c8616968..5583b7d8 100644 --- a/mooncake-transfer-engine/src/transport/kunpeng_transport/CMakeLists.txt +++ b/mooncake-transfer-engine/src/transport/kunpeng_transport/CMakeLists.txt @@ -1,4 +1,4 @@ -file(GLOB UB_SOURCES "*.cpp" "urma/urma_endpoint.cpp") +file(GLOB UB_SOURCES "*.cpp" "urma/urma_endpoint.cpp" "ub_allocator.cpp") # Check if liburma.so exists find_library(URMA_LIBRARY urma PATHS /usr/lib64) diff --git a/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_allocator.cpp b/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_allocator.cpp new file mode 100644 index 00000000..609ecf98 --- /dev/null +++ b/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_allocator.cpp @@ -0,0 +1,76 @@ +#include +#include +#include +#include +#include +#include +#include + +#include "ub_allocator.h" + +namespace mooncake { +struct UbStoreMemRange { + void* base; + size_t size; +}; +std::mutex g_ub_store_mem_mutex; +std::vector g_ub_store_mem_ranges; + +size_t remove_store_memory_range(void* ptr) { + std::lock_guard store_lock(g_ub_store_mem_mutex); + + auto it = std::find_if( + g_ub_store_mem_ranges.begin(), g_ub_store_mem_ranges.end(), + [ptr](const UbStoreMemRange& range) { return range.base == ptr; }); + + if (it == g_ub_store_mem_ranges.end()) { + LOG(ERROR) << "failed for UB protocol, addr at " << ptr; + return 0; + } + + size_t sz = it->size; + g_ub_store_mem_ranges.erase(it); + return sz; +} + +void* ub_allocate_memory(size_t alignment, size_t total_size) { + void* ptr = numa_alloc_local(total_size); + if (!ptr) { + LOG(ERROR) << "failed for UB protocol, size=" << total_size + << ", alignment : " << alignment; + return nullptr; + } + LOG(INFO) << "UB: allocated total size : " << total_size + << ", alignment : " << alignment << " addr at " << ptr; + + std::lock_guard store_lock(g_ub_store_mem_mutex); + g_ub_store_mem_ranges.push_back({ptr, total_size}); + + return ptr; +} + +void ub_free_memory(void* ptr) { + if (!ptr) { + return; + } + auto size = remove_store_memory_range(ptr); + numa_free(ptr, size); + LOG(INFO) << "UB: freed bytes at " << ptr; +} + +bool ub_is_store_memory(void* addr, size_t length) { + if (!addr || length == 0) return false; + auto addr_start = reinterpret_cast(addr); + uintptr_t addr_end = addr_start + length; + std::lock_guard lock(g_ub_store_mem_mutex); + for (const auto& range : g_ub_store_mem_ranges) { + auto range_start = reinterpret_cast(range.base); + uintptr_t range_end = range_start + range.size; + if (addr_start >= range_start && addr_end <= range_end) { + return true; + } + } + return false; +} + +} // namespace mooncake \ No newline at end of file diff --git a/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_context.cpp b/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_context.cpp index 4814a183..ac10faa5 100644 --- a/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_context.cpp +++ b/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_context.cpp @@ -80,6 +80,29 @@ int UbSIEVEEndpointStore::deleteEndpoint(const std::string& peer_nic_path) { return 0; } +int UbSIEVEEndpointStore::deleteEndpointByPtr(UbEndPoint* point_ptr) { + RWSpinlock::WriteGuard guard(endpoint_map_lock_); + for (auto iter = endpoint_map_.begin(); iter != endpoint_map_.end(); + iter++) { + if (iter->second.first.get() == point_ptr) { + std::string peer_nic_path = iter->first; + iter->second.first->deconstruct(); + waiting_list_len_++; + waiting_list_.insert(iter->second.first); + auto fifo_iter = fifo_map_[peer_nic_path]; + if (hand_.has_value() && hand_.value() == fifo_iter) { + fifo_iter == fifo_list_.begin() ? hand_ = std::nullopt + : hand_ = std::prev(fifo_iter); + } + fifo_list_.erase(fifo_iter); + fifo_map_.erase(peer_nic_path); + endpoint_map_.erase(iter); + return 0; + } + } + return 0; +} + void UbSIEVEEndpointStore::evictEndpoint() { if (fifo_list_.empty()) { return; @@ -246,6 +269,12 @@ int UbWorkerPool::submitPostSend( auto targetSegment = peer_segment_desc->buffers[buffer_id].tseg[device_id]; slice->ub.r_seg = context_.retrieveRemoteSeg(targetSegment); + if (!slice->ub.r_seg) { + LOG(ERROR) << "[UB] retrieveRemoteSeg failed for target_id=" + << slice->target_id << " buffer_id=" << buffer_id + << " device_id" << device_id + << " dest_addr=" << slice->ub.dest_addr; + } auto peer_nic_path = MakeNicPath(peer_segment_desc->name, peer_segment_desc->devices[device_id].name); @@ -333,7 +362,7 @@ void UbWorkerPool::performPostSend(int thread_id) { } if (!endpoint->active()) { if (endpoint->inactiveTime() > 1.0) - context_.deleteEndpoint(entry.first); + context_.deleteEndpointByPtr(endpoint.get()); // enable for re-establishation for (auto& slice : entry.second) failed_slice_list.push_back(slice); entry.second.clear(); @@ -355,6 +384,10 @@ void UbWorkerPool::performPostSend(int thread_id) { entry.second.clear(); continue; } + // Set endpoint pointer for each slice before submitting + for (auto& slice : entry.second) { + slice->ub.endpoint = endpoint.get(); + } endpoint->submitPostSend(entry.second, failed_slice_list); #endif } @@ -392,9 +425,12 @@ void UbWorkerPool::performPoll(int thread_id) { << context_.nicPath() << ", mark it inactive"; context_.set_active(false); } - context_.deleteEndpoint(slice->peer_nic_path); slice->ub.retry_cnt++; if (slice->ub.retry_cnt >= slice->ub.max_retry_cnt) { + if (slice->ub.endpoint) { + auto ptr = static_cast(slice->ub.endpoint); + context_.deleteEndpointByPtr(ptr); + } slice->markFailed(); processed_slice_count_++; } else { diff --git a/mooncake-transfer-engine/src/transport/kunpeng_transport/urma/mock_urma.cpp b/mooncake-transfer-engine/src/transport/kunpeng_transport/urma/mock_urma.cpp index 020c3a6b..c392577a 100644 --- a/mooncake-transfer-engine/src/transport/kunpeng_transport/urma/mock_urma.cpp +++ b/mooncake-transfer-engine/src/transport/kunpeng_transport/urma/mock_urma.cpp @@ -1,16 +1,26 @@ #include "urma_api.h" -#include -#include +#include +#include #include +#include +#include #include +#include +#include namespace { -std::mutex mock_mutex; + +struct JfcState { + std::mutex mutex; + std::deque pending_ctx; +}; + +std::shared_mutex g_rw_mutex; bool initialized = false; std::vector device_list; std::map context_map; std::map jfce_map; -std::map> jfc_user_ctx_map; +std::map jfc_state_map; std::map jfr_map; std::map seg_map; std::map jetty_map; @@ -32,10 +42,11 @@ urma_eid_info_t mock_eid_info = { .eid = {{0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10}}, .eid_index = 0}; + } // namespace urma_status_t urma_init(urma_init_attr_t *init_attr) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (initialized) { return URMA_EEXIST; } @@ -44,7 +55,7 @@ urma_status_t urma_init(urma_init_attr_t *init_attr) { } urma_status_t urma_uninit(void) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); initialized = false; for (auto device : device_list) { delete device; @@ -52,7 +63,10 @@ urma_status_t urma_uninit(void) { device_list.clear(); context_map.clear(); jfce_map.clear(); - jfc_user_ctx_map.clear(); + for (auto &kv : jfc_state_map) { + delete kv.second; + } + jfc_state_map.clear(); jfr_map.clear(); seg_map.clear(); jetty_map.clear(); @@ -61,53 +75,81 @@ urma_status_t urma_uninit(void) { } urma_device_t **urma_get_device_list(int *num_devices) { - std::lock_guard lock(mock_mutex); - if (!initialized) { - *num_devices = 0; - return nullptr; + { + std::shared_lock lock(g_rw_mutex); + if (!initialized) { + *num_devices = 0; + return nullptr; + } + if (!device_list.empty()) { + *num_devices = device_list.size(); + urma_device_t **devices = new urma_device_t *[device_list.size()]; + for (size_t i = 0; i < device_list.size(); ++i) { + devices[i] = device_list[i]; + } + return devices; + } } - - if (device_list.empty()) { - urma_device_t *device = new urma_device_t; - strcpy(device->name, "mock_urma_device"); - strcpy(device->path, "/sys/class/infiniband/mock_device"); - device->type = URMA_TRANSPORT_UB; - device->ops = nullptr; - device->sysfs_dev = nullptr; - device_list.push_back(device); + { + std::unique_lock write_lock(g_rw_mutex); + if (!initialized) { + *num_devices = 0; + return nullptr; + } + if (device_list.empty()) { + urma_device_t *device = new urma_device_t; + strcpy(device->name, "mock_urma_device"); + strcpy(device->path, "/sys/class/infiniband/mock_device"); + device->type = URMA_TRANSPORT_UB; + device->ops = nullptr; + device->sysfs_dev = nullptr; + device_list.push_back(device); + } + *num_devices = device_list.size(); + urma_device_t **devices = new urma_device_t *[device_list.size()]; + for (size_t i = 0; i < device_list.size(); ++i) { + devices[i] = device_list[i]; + } + return devices; } - - *num_devices = device_list.size(); - urma_device_t **devices = new urma_device_t *[device_list.size()]; - for (size_t i = 0; i < device_list.size(); ++i) { - devices[i] = device_list[i]; - } - return devices; } urma_device_t *urma_get_device_by_name(const char *name) { - std::lock_guard lock(mock_mutex); - if (!initialized) { - return nullptr; - } - - if (device_list.empty()) { - auto *device = new urma_device_t; - strcpy(device->name, "mock_urma_device"); - strcpy(device->path, "/sys/class/infiniband/mock_device"); - device->type = URMA_TRANSPORT_UB; - device->ops = nullptr; - device->sysfs_dev = nullptr; - device_list.push_back(device); - } - - for (auto device : device_list) { - if (strcmp(device->name, name) == 0) { - return device; + { + std::shared_lock lock(g_rw_mutex); + if (!initialized) { + return nullptr; + } + if (!device_list.empty()) { + for (auto device : device_list) { + if (strcmp(device->name, name) == 0) { + return device; + } + } + return device_list[0]; } } - - return device_list.empty() ? nullptr : device_list[0]; + { + std::unique_lock write_lock(g_rw_mutex); + if (!initialized) { + return nullptr; + } + if (device_list.empty()) { + auto *device = new urma_device_t; + strcpy(device->name, "mock_urma_device"); + strcpy(device->path, "/sys/class/infiniband/mock_device"); + device->type = URMA_TRANSPORT_UB; + device->ops = nullptr; + device->sysfs_dev = nullptr; + device_list.push_back(device); + } + for (auto device : device_list) { + if (strcmp(device->name, name) == 0) { + return device; + } + } + return device_list.empty() ? nullptr : device_list[0]; + } } void urma_free_device_list(urma_device_t **device_list) { @@ -118,7 +160,6 @@ void urma_free_device_list(urma_device_t **device_list) { urma_status_t urma_query_device(urma_device_t *device, urma_device_attr_t *attr) { - std::lock_guard lock(mock_mutex); if (!device || !attr) { return URMA_EINVAL; } @@ -129,7 +170,6 @@ urma_status_t urma_query_device(urma_device_t *device, } urma_eid_info_t *urma_get_eid_list(urma_device_t *device, uint32_t *eid_cnt) { - std::lock_guard lock(mock_mutex); if (!device || !eid_cnt) { return nullptr; } @@ -146,7 +186,7 @@ void urma_free_eid_list(urma_eid_info_t *eid_list) { } urma_context_t *urma_create_context(urma_device_t *device, uint32_t eid_index) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!device) { return nullptr; } @@ -158,7 +198,7 @@ urma_context_t *urma_create_context(urma_device_t *device, uint32_t eid_index) { } urma_status_t urma_delete_context(urma_context_t *ctx) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!ctx || context_map.find(ctx) == context_map.end()) { return URMA_EINVAL; } @@ -168,7 +208,7 @@ urma_status_t urma_delete_context(urma_context_t *ctx) { } urma_jfce_t *urma_create_jfce(urma_context_t *ctx) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!ctx || context_map.find(ctx) == context_map.end()) { return nullptr; } @@ -178,7 +218,7 @@ urma_jfce_t *urma_create_jfce(urma_context_t *ctx) { } urma_status_t urma_delete_jfce(urma_jfce_t *jfce) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!jfce || jfce_map.find(jfce) == jfce_map.end()) { return URMA_EINVAL; } @@ -188,7 +228,7 @@ urma_status_t urma_delete_jfce(urma_jfce_t *jfce) { } urma_jfc_t *urma_create_jfc(urma_context_t *ctx, urma_jfc_cfg_t *cfg) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!ctx || !cfg || context_map.find(ctx) == context_map.end()) { return nullptr; } @@ -201,22 +241,23 @@ urma_jfc_t *urma_create_jfc(urma_context_t *ctx, urma_jfc_cfg_t *cfg) { jfc->comp_events_acked = 0; jfc->async_events_acked = 0; jfc->jfc_cfg = *cfg; - jfc_user_ctx_map[jfc] = std::vector(); + jfc_state_map[jfc] = new JfcState(); return jfc; } urma_status_t urma_delete_jfc(urma_jfc_t *jfc) { - std::lock_guard lock(mock_mutex); - if (!jfc || jfc_user_ctx_map.find(jfc) == jfc_user_ctx_map.end()) { + std::unique_lock lock(g_rw_mutex); + if (!jfc || jfc_state_map.find(jfc) == jfc_state_map.end()) { return URMA_EINVAL; } - jfc_user_ctx_map.erase(jfc); + delete jfc_state_map[jfc]; + jfc_state_map.erase(jfc); delete jfc; return URMA_SUCCESS; } urma_jfr_t *urma_create_jfr(urma_context_t *ctx, urma_jfr_cfg_t *cfg) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!ctx || !cfg || context_map.find(ctx) == context_map.end()) { return nullptr; } @@ -226,7 +267,7 @@ urma_jfr_t *urma_create_jfr(urma_context_t *ctx, urma_jfr_cfg_t *cfg) { } urma_status_t urma_delete_jfr(urma_jfr_t *jfr) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!jfr || jfr_map.find(jfr) == jfr_map.end()) { return URMA_EINVAL; } @@ -236,7 +277,7 @@ urma_status_t urma_delete_jfr(urma_jfr_t *jfr) { } urma_target_seg_t *urma_register_seg(urma_context_t *ctx, urma_seg_cfg_t *cfg) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!ctx || !cfg || context_map.find(ctx) == context_map.end()) { return nullptr; } @@ -252,7 +293,7 @@ urma_target_seg_t *urma_register_seg(urma_context_t *ctx, urma_seg_cfg_t *cfg) { } urma_status_t urma_unregister_seg(urma_target_seg_t *seg) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!seg || seg_map.find(seg) == seg_map.end()) { return URMA_EINVAL; } @@ -264,7 +305,7 @@ urma_status_t urma_unregister_seg(urma_target_seg_t *seg) { urma_target_seg_t *urma_import_seg(urma_context_t *ctx, urma_seg_t *seg, urma_token_t *token_value, uint64_t addr, urma_import_seg_flag_t flag) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!ctx || !seg || !token_value || context_map.find(ctx) == context_map.end()) { return nullptr; @@ -277,7 +318,7 @@ urma_target_seg_t *urma_import_seg(urma_context_t *ctx, urma_seg_t *seg, } urma_status_t urma_unimport_seg(urma_target_seg_t *tseg) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!tseg || seg_map.find(tseg) == seg_map.end()) { return URMA_EINVAL; } @@ -288,8 +329,11 @@ urma_status_t urma_unimport_seg(urma_target_seg_t *tseg) { urma_status_t urma_get_async_event(urma_context_t *ctx, urma_async_event_t *event) { - std::lock_guard lock(mock_mutex); - if (!ctx || !event || context_map.find(ctx) == context_map.end()) { + if (!ctx || !event) { + return URMA_EINVAL; + } + std::shared_lock lock(g_rw_mutex); + if (context_map.find(ctx) == context_map.end()) { return URMA_EINVAL; } return URMA_ETIMEOUT; @@ -298,7 +342,7 @@ urma_status_t urma_get_async_event(urma_context_t *ctx, void urma_ack_async_event(urma_async_event_t *event) {} urma_jetty_t *urma_create_jetty(urma_context_t *ctx, urma_jetty_cfg_t *cfg) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!ctx || !cfg || context_map.find(ctx) == context_map.end()) { return nullptr; } @@ -314,7 +358,7 @@ urma_jetty_t *urma_create_jetty(urma_context_t *ctx, urma_jetty_cfg_t *cfg) { } urma_status_t urma_delete_jetty(urma_jetty_t *jetty) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!jetty || jetty_map.find(jetty) == jetty_map.end()) { return URMA_EINVAL; } @@ -324,7 +368,7 @@ urma_status_t urma_delete_jetty(urma_jetty_t *jetty) { } urma_status_t urma_unbind_jetty(urma_jetty_t *jetty) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!jetty || jetty_map.find(jetty) == jetty_map.end()) { return URMA_EINVAL; } @@ -335,7 +379,7 @@ urma_status_t urma_unbind_jetty(urma_jetty_t *jetty) { urma_target_jetty_t *urma_import_jetty(urma_context_t *ctx, urma_rjetty_t *rjetty, urma_token_t *token_value) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!ctx || !rjetty || !token_value || context_map.find(ctx) == context_map.end()) { return nullptr; @@ -348,7 +392,7 @@ urma_target_jetty_t *urma_import_jetty(urma_context_t *ctx, } urma_status_t urma_unimport_jetty(urma_target_jetty_t *tjetty) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!tjetty || target_jetty_map.find(tjetty) == target_jetty_map.end()) { return URMA_EINVAL; } @@ -359,7 +403,7 @@ urma_status_t urma_unimport_jetty(urma_target_jetty_t *tjetty) { urma_status_t urma_bind_jetty(urma_jetty_t *jetty, urma_target_jetty_t *tjetty) { - std::lock_guard lock(mock_mutex); + std::unique_lock lock(g_rw_mutex); if (!jetty || !tjetty || jetty_map.find(jetty) == jetty_map.end() || target_jetty_map.find(tjetty) == target_jetty_map.end()) { return URMA_EINVAL; @@ -369,7 +413,7 @@ urma_status_t urma_bind_jetty(urma_jetty_t *jetty, } urma_status_t urma_modify_jetty(urma_jetty_t *jetty, urma_jetty_attr_t *attr) { - std::lock_guard lock(mock_mutex); + std::shared_lock lock(g_rw_mutex); if (!jetty || !attr || jetty_map.find(jetty) == jetty_map.end()) { return URMA_EINVAL; } @@ -378,19 +422,37 @@ urma_status_t urma_modify_jetty(urma_jetty_t *jetty, urma_jetty_attr_t *attr) { urma_status_t urma_post_jetty_send_wr(urma_jetty_t *jetty, urma_jfs_wr_t *wr, urma_jfs_wr_t **bad_wr) { - std::lock_guard lock(mock_mutex); - if (!jetty || !wr || jetty_map.find(jetty) == jetty_map.end()) { - if (bad_wr) { - *bad_wr = wr; + { + std::shared_lock lock(g_rw_mutex); + if (!jetty || !wr || jetty_map.find(jetty) == jetty_map.end()) { + if (bad_wr) { + *bad_wr = wr; + } + return URMA_EINVAL; } - return URMA_EINVAL; } - urma_jfs_wr_t *current_wr = wr; - while (current_wr) { - jfc_user_ctx_map[jetty->jetty_cfg.jfs_cfg.jfc].push_back( - current_wr->user_ctx); - current_wr = current_wr->next; + urma_jfc_t *jfc = jetty->jetty_cfg.jfs_cfg.jfc; + JfcState *state = nullptr; + { + std::shared_lock lock(g_rw_mutex); + auto it = jfc_state_map.find(jfc); + if (it == jfc_state_map.end()) { + if (bad_wr) { + *bad_wr = wr; + } + return URMA_EINVAL; + } + state = it->second; + } + + { + std::lock_guard jfc_lock(state->mutex); + urma_jfs_wr_t *current_wr = wr; + while (current_wr) { + state->pending_ctx.push_back(current_wr->user_ctx); + current_wr = current_wr->next; + } } if (bad_wr) { @@ -400,18 +462,27 @@ urma_status_t urma_post_jetty_send_wr(urma_jetty_t *jetty, urma_jfs_wr_t *wr, } int urma_poll_jfc(urma_jfc_t *jfc, int num_entries, urma_cr_t *cr_list) { - std::lock_guard lock(mock_mutex); - if (!jfc || !cr_list || - jfc_user_ctx_map.find(jfc) == jfc_user_ctx_map.end()) { - return -1; + JfcState *state = nullptr; + { + std::shared_lock lock(g_rw_mutex); + auto it = jfc_state_map.find(jfc); + if (it == jfc_state_map.end()) { + return -1; + } + state = it->second; } - int available = jfc_user_ctx_map[jfc].size(); - int num_completed = std::min(num_entries, available); - for (int i = 0; i < num_completed; ++i) { - cr_list[i].status = URMA_CR_SUCCESS; - cr_list[i].user_ctx = jfc_user_ctx_map[jfc][i]; + + int num_completed = 0; + { + std::lock_guard jfc_lock(state->mutex); + int available = static_cast(state->pending_ctx.size()); + num_completed = std::min(num_entries, available); + for (int i = 0; i < num_completed; ++i) { + cr_list[i].status = URMA_CR_SUCCESS; + cr_list[i].user_ctx = state->pending_ctx[i]; + } + state->pending_ctx.erase(state->pending_ctx.begin(), + state->pending_ctx.begin() + num_completed); } - jfc_user_ctx_map[jfc].erase(jfc_user_ctx_map[jfc].begin(), - jfc_user_ctx_map[jfc].begin() + num_completed); return num_completed; } diff --git a/mooncake-transfer-engine/src/transport/kunpeng_transport/urma/urma_endpoint.cpp b/mooncake-transfer-engine/src/transport/kunpeng_transport/urma/urma_endpoint.cpp index f7080a76..0d9b3c61 100644 --- a/mooncake-transfer-engine/src/transport/kunpeng_transport/urma/urma_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/kunpeng_transport/urma/urma_endpoint.cpp @@ -420,13 +420,16 @@ int UrmaContext::openDevice(const std::string& device_name, uint8_t port, return ERR_CONTEXT; } for (int p = 0; p < MAX_PORT_CNT; p++) { - if (dev_attr_.port_attr[p].state == URMA_PORT_ACTIVE) { + auto port_attr = dev_attr_.port_attr[p]; + if (port_attr.state == URMA_PORT_ACTIVE || + port_attr.state == URMA_PORT_ACTIVE_DEFER) { port_ = p; break; } } if (dev_attr_.port_cnt != 0 && - dev_attr_.port_attr[port_].state != URMA_PORT_ACTIVE) { + dev_attr_.port_attr[port_].state != URMA_PORT_ACTIVE && + dev_attr_.port_attr[port_].state != URMA_PORT_ACTIVE_DEFER) { LOG(WARNING) << "Device " << device_name << " not found active port"; if (urma_delete_context(context)) { @@ -528,9 +531,9 @@ int UrmaContext::poll(int num_entries, Transport::Slice** slices, if (!slice) { continue; } + slices[i] = slice; if (cr[i].status == URMA_CR_SUCCESS) { slice->markSuccess(); - slices[i] = slice; continue; } if (cr[i].status != URMA_CR_WR_FLUSH_ERR || @@ -886,6 +889,10 @@ int UrmaEndpoint::submitPostSend( wr.flag.bs.inline_flag = 0; // Check if the jetty is in the imported_jetty_map_ auto it = imported_jetty_map_.find(jetty_list_[jetty_index]); + if (it == imported_jetty_map_.end()) { + LOG(ERROR) << "Jetty not imported for endpoint, tjetty is nullptr" + << jetty_index << ", local_nic="; + } if (it != imported_jetty_map_.end()) { wr.tjetty = it->second; } else { @@ -895,6 +902,8 @@ int UrmaEndpoint::submitPostSend( slice->ts = getCurrentTimeInNano(); slice->status = Transport::Slice::POSTED; slice->ub.jetty_depth = &wr_depth_list_[jetty_index]; + // Set endpoint pointer for each slice before submitting + slice->ub.endpoint = this; } __sync_fetch_and_add(&wr_depth_list_[jetty_index], wr_count); __sync_fetch_and_add(jfc_outstanding_, wr_count); @@ -966,6 +975,8 @@ int UrmaEndpoint::doSetupConnection(int jetty_index, rjetty.jetty_id.eid = eid; rjetty.trans_mode = URMA_TM_RC; rjetty.type = URMA_JETTY; + rjetty.tp_type = URMA_CTP; + rjetty.flag.value = 0; LOG(INFO) << "Peer jetty id = " << peer_jetty_num; urma_target_jetty_t* imported_jetty = urma_import_jetty(context_->urma_context_, &rjetty, &urma_token); diff --git a/scripts/build_wheel.sh b/scripts/build_wheel.sh index feff9662..fdd5737f 100755 --- a/scripts/build_wheel.sh +++ b/scripts/build_wheel.sh @@ -363,6 +363,7 @@ ${AUDITWHEEL_CMD} repair ${OUTPUT_DIR}/*.whl \ --exclude libllm_datadist*.so \ --exclude ascend_transport*.so \ --exclude libaccl_barex.so* \ + --exclude liburma.so* \ -w ${REPAIRED_DIR}/ --plat ${PLATFORM_TAG} # Inject CUDA extensions into the repaired wheel. patchelf (used by auditwheel) -- 2.34.1 From 579d42d35437c45427852fc9233850a8427effe0 Mon Sep 17 00:00:00 2001 From: Yaoming Zhan Date: Sun, 31 May 2026 19:31:44 -0700 Subject: [PATCH 154/382] [Store] L2->L1 promotion-on-hit: observability metrics + max_per_heartbeat knob (#2176) * [Store] L2->L1 promotion-on-hit: Tier A observability + max_per_heartbeat knob Adds Prometheus metrics for the promotion-on-hit funnel and exposes the previously-hardcoded kMaxPerHeartbeat as a config knob. Metrics (master_promotion_*) ---------------------------- Funnel: - promotion_in_flight (gauge): mirror of promotion_in_flight_ - promotion_admitted_total: tasks past all gates, enqueued - promotion_completed_total: NotifyPromotionSuccess success - promotion_completed_bytes_total: bytes promoted (sum of source object_size) - promotion_failed_total: NotifyPromotionFailure accepted - promotion_expired_total: reaper Part 4 sweeps Rejection (per gate): - promotion_rejected_frequency_total: below admission threshold - promotion_rejected_watermark_total: DRAM at or above high watermark - promotion_rejected_cap_total: promotion_in_flight at queue limit Together: admitted = completed + failed + expired + in_flight. Wires a working 'promotion feature health' Grafana panel: rate(admitted) vs rate(completed) shows yield; rate(rejected_*) breaks down where work is dropping; in_flight / promotion_queue_limit shows saturation. promotion_max_per_heartbeat knob -------------------------------- The previous compile-time constant kMaxPerHeartbeat = 1 (in PromotionObjectHeartbeat) capped per-client throughput at ~6 promotions/min with the default 10s heartbeat, making promotion_queue_limit = 50000 mostly theoretical. Exposed as MasterServiceConfig::promotion_max_per_heartbeat, wired through the flag parser (FLAGS_promotion_max_per_heartbeat) and default_config.GetUInt32 path. Constructor clamps 0 -> 1 so a mistyped config doesn't silently halt promotion delivery. Startup log now includes max_per_heartbeat= alongside threshold= / queue_limit=. Default stays 1 (no behavior change on existing deployments). Operators with small objects + RDMA-rich clusters can raise it; the new MaxPerHeartbeatKnobControlsBatchSize test exercises 3. Tests ----- - MetricsFunnelTracksSuccessfulPromotion: single full lifecycle bumps admitted/completed/completed_bytes correctly and brings in_flight back to baseline. - MetricsRejectionCountersIncrementOnGateMiss: each of frequency and cap counters increments when its branch fires. - MaxPerHeartbeatKnobControlsBatchSize: knob=3, 5 admitted tasks drain across 3+2+0 heartbeats. - MaxPerHeartbeatZeroClampsToOne: pathological config clamps to 1. Suite: 32/32 pass (was 28; +4 new). No behavior change in existing tests. * [Store] L2->L1 promotion-on-hit: cover watermark gate in rejection test MetricsRejectionCountersIncrementOnGateMiss claimed coverage of all three rejection counters but only exercised frequency and cap. Add a sub-case that forces the watermark gate by configuring eviction_high_watermark_ratio = 0.0, asserting that promotion_rejected_watermark_total increments. * [Store] L2->L1 promotion-on-hit: cover RemoveAll/BatchRemove cleanup upstream #2180 introduced EraseMetadataEntry as the centralized metadata-erase helper and routed RemoveAll, BatchRemove, and RemoveByRegex through it, so promotion_tasks cleanup on those paths is already correct on main. The metric instrumentation (dec_promotion_in_flight + inc_promotion_cancelled) is wired into EraseMetadataEntry in this branch's earlier commit so every site that erases metadata bumps the funnel counters consistently. Add regression tests for the three paths so any future refactor that reintroduces a metadata.erase without going through EraseMetadataEntry will fail the suite: - RemoveAllErasesPromotionTask - BatchRemoveErasesPromotionTask (normal-completion branch) - BatchRemoveStaleHandleErasesPromotionTask * [Store] L2->L1 promotion-on-hit: bump reaper-test sleep margin to 3s Three tests configure put_start_release_timeout_sec=1 and then sleep 2s waiting for the eviction-thread reaper to expire the promotion task. The reaper schedule is `now - last_discard_time > put_start_release_timeout_sec_` (strict greater-than), so a 1s release with a 2s sleep leaves only ~1s margin between the reaper firing and the assertion. CI runs observed intermittent failures when scheduling jitter erased that margin. Bump the sleep to 3s in the three affected tests so the margin is ~2s. Configuration values unchanged. Affected: - StalePromotionReaper - RemoveDuringPromotion - AllocStartRejectsReapedTask Verified 5 consecutive clean runs of all three under -j1 build. --- mooncake-store/include/master_config.h | 13 + .../include/master_metric_manager.h | 37 ++ mooncake-store/include/master_service.h | 3 + mooncake-store/src/master.cpp | 15 + mooncake-store/src/master_metric_manager.cpp | 144 ++++++ mooncake-store/src/master_service.cpp | 40 +- .../tests/promotion_on_hit_test.cpp | 481 +++++++++++++++++- 7 files changed, 715 insertions(+), 18 deletions(-) diff --git a/mooncake-store/include/master_config.h b/mooncake-store/include/master_config.h index 1f1bc5a9..6dcf1de8 100644 --- a/mooncake-store/include/master_config.h +++ b/mooncake-store/include/master_config.h @@ -109,6 +109,12 @@ struct MasterConfig { bool promotion_on_hit = false; uint32_t promotion_admission_threshold = 2; uint32_t promotion_queue_limit = 50000; + // Max promotion tasks PromotionObjectHeartbeat returns to a single + // client per call. Each task is a synchronous SSD-read + RDMA-write + // on the client; serializing them avoids blocking past the client- + // liveness window. Default 1 is conservative; small-object or RDMA- + // rich clusters may safely raise it. + uint32_t promotion_max_per_heartbeat = 1; }; class MasterServiceSupervisorConfig { @@ -182,6 +188,7 @@ class MasterServiceSupervisorConfig { bool promotion_on_hit = false; uint32_t promotion_admission_threshold = 2; uint32_t promotion_queue_limit = 50000; + uint32_t promotion_max_per_heartbeat = 1; MasterServiceSupervisorConfig() = default; // From MasterConfig @@ -209,6 +216,7 @@ class MasterServiceSupervisorConfig { promotion_on_hit = config.promotion_on_hit; promotion_admission_threshold = config.promotion_admission_threshold; promotion_queue_limit = config.promotion_queue_limit; + promotion_max_per_heartbeat = config.promotion_max_per_heartbeat; rpc_port = static_cast(config.rpc_port); rpc_thread_num = static_cast(config.rpc_thread_num); @@ -350,6 +358,7 @@ class WrappedMasterServiceConfig { bool promotion_on_hit = false; uint32_t promotion_admission_threshold = 2; uint32_t promotion_queue_limit = 50000; + uint32_t promotion_max_per_heartbeat = 1; std::string ha_backend_type = "etcd"; std::string ha_backend_connstring; std::string cluster_id = DEFAULT_CLUSTER_ID; @@ -417,6 +426,7 @@ class WrappedMasterServiceConfig { promotion_on_hit = config.promotion_on_hit; promotion_admission_threshold = config.promotion_admission_threshold; promotion_queue_limit = config.promotion_queue_limit; + promotion_max_per_heartbeat = config.promotion_max_per_heartbeat; ha_backend_type = config.ha_backend_type; ha_backend_connstring = ResolveConfiguredHABackendConnstring( ha_backend_type, config.ha_backend_connstring, @@ -502,6 +512,7 @@ class WrappedMasterServiceConfig { promotion_on_hit = config.promotion_on_hit; promotion_admission_threshold = config.promotion_admission_threshold; promotion_queue_limit = config.promotion_queue_limit; + promotion_max_per_heartbeat = config.promotion_max_per_heartbeat; ha_backend_type = config.ha_backend_type; ha_backend_connstring = ResolveConfiguredHABackendConnstring( ha_backend_type, config.ha_backend_connstring, @@ -886,6 +897,7 @@ class MasterServiceConfig { bool promotion_on_hit = false; uint32_t promotion_admission_threshold = 2; uint32_t promotion_queue_limit = 50000; + uint32_t promotion_max_per_heartbeat = 1; std::string ha_backend_type = "etcd"; std::string ha_backend_connstring; std::string cluster_id = DEFAULT_CLUSTER_ID; @@ -949,6 +961,7 @@ class MasterServiceConfig { promotion_on_hit = config.promotion_on_hit; promotion_admission_threshold = config.promotion_admission_threshold; promotion_queue_limit = config.promotion_queue_limit; + promotion_max_per_heartbeat = config.promotion_max_per_heartbeat; ha_backend_type = config.ha_backend_type; ha_backend_connstring = config.ha_backend_connstring; cluster_id = config.cluster_id; diff --git a/mooncake-store/include/master_metric_manager.h b/mooncake-store/include/master_metric_manager.h index d50c61e6..582b1569 100644 --- a/mooncake-store/include/master_metric_manager.h +++ b/mooncake-store/include/master_metric_manager.h @@ -295,6 +295,31 @@ class MasterMetricManager { int64_t get_put_start_release_cnt(); int64_t get_put_start_discarded_staging_size(); + // Promotion-on-hit Metrics + void inc_promotion_in_flight(int64_t val = 1); + void dec_promotion_in_flight(int64_t val = 1); + void inc_promotion_admitted(int64_t val = 1); + void inc_promotion_completed(int64_t val = 1); + void inc_promotion_completed_bytes(int64_t bytes); + void inc_promotion_expired(int64_t val = 1); + void inc_promotion_failed(int64_t val = 1); + void inc_promotion_cancelled(int64_t val = 1); + void inc_promotion_rejected_frequency(int64_t val = 1); + void inc_promotion_rejected_watermark(int64_t val = 1); + void inc_promotion_rejected_cap(int64_t val = 1); + + // Promotion-on-hit Metrics Getters + int64_t get_promotion_in_flight(); + int64_t get_promotion_admitted(); + int64_t get_promotion_completed(); + int64_t get_promotion_completed_bytes(); + int64_t get_promotion_expired(); + int64_t get_promotion_failed(); + int64_t get_promotion_cancelled(); + int64_t get_promotion_rejected_frequency(); + int64_t get_promotion_rejected_watermark(); + int64_t get_promotion_rejected_cap(); + // CopyStart, CopyEnd, CopyRevoke, MoveStart, MoveEnd, MoveRevoke Metrics void inc_copy_start_requests(int64_t val = 1); void inc_copy_start_failures(int64_t val = 1); @@ -632,6 +657,18 @@ class MasterMetricManager { ylt::metric::counter_t put_start_release_cnt_; ylt::metric::gauge_t put_start_discarded_staging_size_; + // Promotion-on-hit Metrics + ylt::metric::gauge_t promotion_in_flight_metric_; + ylt::metric::counter_t promotion_admitted_; + ylt::metric::counter_t promotion_completed_; + ylt::metric::counter_t promotion_completed_bytes_; + ylt::metric::counter_t promotion_expired_; + ylt::metric::counter_t promotion_failed_; + ylt::metric::counter_t promotion_cancelled_; + ylt::metric::counter_t promotion_rejected_frequency_; + ylt::metric::counter_t promotion_rejected_watermark_; + ylt::metric::counter_t promotion_rejected_cap_; + // Snapshot Metrics ylt::metric::histogram_t snapshot_duration_ms_; ylt::metric::counter_t snapshot_success_; diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index e6e7f175..e33fe41d 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -1337,6 +1337,8 @@ class MasterService { NO_THREAD_SAFETY_ANALYSIS { if (tenant_state.promotion_tasks.erase(key) > 0) { promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + MasterMetricManager::instance().dec_promotion_in_flight(); + MasterMetricManager::instance().inc_promotion_cancelled(); } } @@ -1713,6 +1715,7 @@ class MasterService { bool promotion_on_hit_{false}; uint32_t promotion_admission_threshold_{2}; uint32_t promotion_queue_limit_{50000}; + uint32_t promotion_max_per_heartbeat_{1}; // Global in-flight task counter, checked against promotion_queue_limit_ // as the gate cap. Promotion specifically targets skewed // access (hot keys re-accessed after eviction), so the global counter diff --git a/mooncake-store/src/master.cpp b/mooncake-store/src/master.cpp index 8c559c07..ec9545b2 100644 --- a/mooncake-store/src/master.cpp +++ b/mooncake-store/src/master.cpp @@ -134,6 +134,12 @@ DEFINE_uint32(promotion_admission_threshold, 2, "(set 1 to disable second-touch gating)"); DEFINE_uint32(promotion_queue_limit, 50000, "Max in-flight promotion tasks across all shards"); +DEFINE_uint32(promotion_max_per_heartbeat, 1, + "Max promotion tasks returned to a single client per " + "PromotionObjectHeartbeat call. Each task is a synchronous " + "SSD-read + RDMA-write on the client; serializing them avoids " + "blocking past the client-liveness window. Default 1 is " + "conservative."); DEFINE_string(ha_backend_type, "etcd", "HA backend type, e.g. etcd | redis | k8s"); DEFINE_string(ha_backend_connstring, "", @@ -358,6 +364,9 @@ void InitMasterConf(const mooncake::DefaultConfig& default_config, default_config.GetUInt32("promotion_queue_limit", &master_config.promotion_queue_limit, FLAGS_promotion_queue_limit); + default_config.GetUInt32("promotion_max_per_heartbeat", + &master_config.promotion_max_per_heartbeat, + FLAGS_promotion_max_per_heartbeat); default_config.GetString("ha_backend_type", &master_config.ha_backend_type, FLAGS_ha_backend_type); default_config.GetString("ha_backend_connstring", @@ -638,6 +647,12 @@ void LoadConfigFromCmdline(mooncake::MasterConfig& master_config, !conf_set) { master_config.promotion_queue_limit = FLAGS_promotion_queue_limit; } + if ((google::GetCommandLineFlagInfo("promotion_max_per_heartbeat", &info) && + !info.is_default) || + !conf_set) { + master_config.promotion_max_per_heartbeat = + FLAGS_promotion_max_per_heartbeat; + } // Clamp promotion_admission_threshold into the sketch counter's // representable range. The CountMinSketch uses 8-bit saturating // counters (max 255) so any threshold beyond that would silently diff --git a/mooncake-store/src/master_metric_manager.cpp b/mooncake-store/src/master_metric_manager.cpp index d82d6be1..cdaec57b 100644 --- a/mooncake-store/src/master_metric_manager.cpp +++ b/mooncake-store/src/master_metric_manager.cpp @@ -329,6 +329,46 @@ MasterMetricManager::MasterMetricManager() "master_put_start_discarded_staging_size", "Total size of memory replicas in discarded but not yet released " "PutStart operations"), + + // Promotion-on-hit Metrics + promotion_in_flight_metric_( + "master_promotion_in_flight", + "Current number of in-flight L2->L1 promotion tasks"), + promotion_admitted_( + "master_promotion_admitted_total", + "Total promotion tasks admitted past all gates and enqueued"), + promotion_completed_( + "master_promotion_completed_total", + "Total promotion tasks committed via NotifyPromotionSuccess"), + promotion_completed_bytes_( + "master_promotion_completed_bytes_total", + "Total bytes promoted from LOCAL_DISK to MEMORY"), + promotion_expired_("master_promotion_expired_total", + "Total promotion tasks expired via the reaper " + "(put_start_release_timeout_sec)"), + promotion_failed_( + "master_promotion_failed_total", + "Total promotion tasks aborted by holder via " + "NotifyPromotionFailure (holder reported a downstream failure)"), + promotion_cancelled_( + "master_promotion_cancelled_total", + "Total promotion tasks removed because the prerequisite went " + "away: object removal mid-flight (Remove / UpsertStart / etc.), " + "holder-client expiry (ClearInvalidHandles), or staged replica " + "lost (NotifyPromotionSuccess committed=false)"), + promotion_rejected_frequency_( + "master_promotion_rejected_frequency_total", + "Promotion attempts rejected because CountMinSketch frequency " + "was below promotion_admission_threshold"), + promotion_rejected_watermark_( + "master_promotion_rejected_watermark_total", + "Promotion attempts rejected because DRAM was at or above the " + "eviction high watermark"), + promotion_rejected_cap_( + "master_promotion_rejected_cap_total", + "Promotion attempts rejected because promotion_in_flight was at " + "promotion_queue_limit"), + // Snapshot Metrics snapshot_duration_ms_( "master_snapshot_duration_ms", @@ -417,8 +457,18 @@ void MasterMetricManager::update_metrics_for_zero_output() { mem_cache_nums_.update(0); file_cache_nums_.update(0); put_start_discarded_staging_size_.update(0); + promotion_in_flight_metric_.update(0); // Update Counters (use inc(0) to mark as changed) + promotion_admitted_.inc(0); + promotion_completed_.inc(0); + promotion_completed_bytes_.inc(0); + promotion_expired_.inc(0); + promotion_failed_.inc(0); + promotion_cancelled_.inc(0); + promotion_rejected_frequency_.inc(0); + promotion_rejected_watermark_.inc(0); + promotion_rejected_cap_.inc(0); put_start_requests_.inc(0); put_start_failures_.inc(0); put_start_alloc_failures_.inc(0); @@ -1024,6 +1074,41 @@ void MasterMetricManager::inc_put_start_release_cnt(int64_t count, put_start_discarded_staging_size_.dec(size); } +// --- Promotion-on-hit Metrics --- +void MasterMetricManager::inc_promotion_in_flight(int64_t val) { + promotion_in_flight_metric_.inc(val); +} +void MasterMetricManager::dec_promotion_in_flight(int64_t val) { + promotion_in_flight_metric_.dec(val); +} +void MasterMetricManager::inc_promotion_admitted(int64_t val) { + promotion_admitted_.inc(val); +} +void MasterMetricManager::inc_promotion_completed(int64_t val) { + promotion_completed_.inc(val); +} +void MasterMetricManager::inc_promotion_completed_bytes(int64_t bytes) { + promotion_completed_bytes_.inc(bytes); +} +void MasterMetricManager::inc_promotion_expired(int64_t val) { + promotion_expired_.inc(val); +} +void MasterMetricManager::inc_promotion_failed(int64_t val) { + promotion_failed_.inc(val); +} +void MasterMetricManager::inc_promotion_cancelled(int64_t val) { + promotion_cancelled_.inc(val); +} +void MasterMetricManager::inc_promotion_rejected_frequency(int64_t val) { + promotion_rejected_frequency_.inc(val); +} +void MasterMetricManager::inc_promotion_rejected_watermark(int64_t val) { + promotion_rejected_watermark_.inc(val); +} +void MasterMetricManager::inc_promotion_rejected_cap(int64_t val) { + promotion_rejected_cap_.inc(val); +} + void MasterMetricManager::set_snapshot_duration_ms(int64_t size) { snapshot_duration_ms_.observe(size); } @@ -1376,6 +1461,38 @@ int64_t MasterMetricManager::get_put_start_discarded_staging_size() { return put_start_discarded_staging_size_.value(); } +// --- Promotion-on-hit Metrics Getters --- +int64_t MasterMetricManager::get_promotion_in_flight() { + return promotion_in_flight_metric_.value(); +} +int64_t MasterMetricManager::get_promotion_admitted() { + return promotion_admitted_.value(); +} +int64_t MasterMetricManager::get_promotion_completed() { + return promotion_completed_.value(); +} +int64_t MasterMetricManager::get_promotion_completed_bytes() { + return promotion_completed_bytes_.value(); +} +int64_t MasterMetricManager::get_promotion_expired() { + return promotion_expired_.value(); +} +int64_t MasterMetricManager::get_promotion_failed() { + return promotion_failed_.value(); +} +int64_t MasterMetricManager::get_promotion_cancelled() { + return promotion_cancelled_.value(); +} +int64_t MasterMetricManager::get_promotion_rejected_frequency() { + return promotion_rejected_frequency_.value(); +} +int64_t MasterMetricManager::get_promotion_rejected_watermark() { + return promotion_rejected_watermark_.value(); +} +int64_t MasterMetricManager::get_promotion_rejected_cap() { + return promotion_rejected_cap_.value(); +} + // CopyStart, CopyEnd, CopyRevoke, MoveStart, MoveEnd, MoveRevoke Metrics void MasterMetricManager::inc_copy_start_requests(int64_t val) { copy_start_requests_.inc(val); @@ -1648,6 +1765,18 @@ std::string MasterMetricManager::serialize_metrics() { serialize_metric(put_start_release_cnt_); serialize_metric(put_start_discarded_staging_size_); + // Serialize Promotion-on-hit Metrics + serialize_metric(promotion_in_flight_metric_); + serialize_metric(promotion_admitted_); + serialize_metric(promotion_completed_); + serialize_metric(promotion_completed_bytes_); + serialize_metric(promotion_expired_); + serialize_metric(promotion_failed_); + serialize_metric(promotion_cancelled_); + serialize_metric(promotion_rejected_frequency_); + serialize_metric(promotion_rejected_watermark_); + serialize_metric(promotion_rejected_cap_); + // Serialize Snapshot Metrics serialize_metric(snapshot_duration_ms_); serialize_metric(snapshot_success_); @@ -2281,6 +2410,21 @@ std::string MasterMetricManager::get_summary_string( << put_start_discard_cnt << ", StagingSize=" << byte_size_to_string(put_start_discarded_staging_size); + // Promotion-on-hit summary (counters are cumulative, not deltas; the + // gauge is current-state). + ss << " | Promotion: " + << "in_flight=" << promotion_in_flight_metric_.value() << ", " + << "admitted=" << promotion_admitted_.value() << ", " + << "completed=" << promotion_completed_.value() << ", " + << "failed=" << promotion_failed_.value() << ", " + << "cancelled=" << promotion_cancelled_.value() << ", " + << "expired=" << promotion_expired_.value() << ", " + << "bytes=" << byte_size_to_string(promotion_completed_bytes_.value()) + << ", " + << "rejected(freq/wm/cap)=" << promotion_rejected_frequency_.value() + << "/" << promotion_rejected_watermark_.value() << "/" + << promotion_rejected_cap_.value(); + // Snapshot summary ss << " | Snapshots: " << "Success=" << snapshot_success_.value() << ", " diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 54589cce..ba3a6992 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -258,6 +258,12 @@ MasterService::MasterService(const MasterServiceConfig& config) promotion_on_hit_ = enable_offload_ && config.promotion_on_hit; promotion_admission_threshold_ = config.promotion_admission_threshold; promotion_queue_limit_ = config.promotion_queue_limit; + promotion_max_per_heartbeat_ = config.promotion_max_per_heartbeat; + // Clamp to >=1: 0 would make PromotionObjectHeartbeat return an empty + // batch every call, silently disabling promotion delivery. + if (promotion_max_per_heartbeat_ == 0) { + promotion_max_per_heartbeat_ = 1; + } // Defense-in-depth clamp: master.cpp clamps threshold into [1, 255] // at flag-parse time, but direct MasterServiceConfig construction // (tests, embedded users) bypasses that. Without the clamp here, @@ -280,7 +286,9 @@ MasterService::MasterService(const MasterServiceConfig& config) LOG(INFO) << "Promotion-on-hit mode enabled: LOCAL_DISK-only Gets " "will queue async promotion to MEMORY (threshold=" << promotion_admission_threshold_ - << ", queue_limit=" << promotion_queue_limit_ << ")"; + << ", queue_limit=" << promotion_queue_limit_ + << ", max_per_heartbeat=" << promotion_max_per_heartbeat_ + << ")"; } eviction_running_ = true; @@ -767,10 +775,7 @@ void MasterService::ClearInvalidHandles( tenant_state.processing_keys.erase(it->first); tenant_state.replication_tasks.erase(it->first); tenant_state.offloading_tasks.erase(it->first); - if (tenant_state.promotion_tasks.erase(it->first) > 0) { - promotion_in_flight_.fetch_sub( - 1, std::memory_order_relaxed); - } + ErasePromotionTaskIfPresent(tenant_state, it->first); it = EraseMetadata(tenant_state, it, tenant_it->first); } else { ++it; @@ -3269,6 +3274,7 @@ void MasterService::TryPushPromotionQueue(const ObjectIdentity& object_id) { // bypass the gate entirely since freq is uint8_t) cannot reach here. const uint8_t freq = promotion_sketch_->increment(key); if (freq < promotion_admission_threshold_) { + MasterMetricManager::instance().inc_promotion_rejected_frequency(); return; } @@ -3278,6 +3284,7 @@ void MasterService::TryPushPromotionQueue(const ObjectIdentity& object_id) { const double used_ratio = MasterMetricManager::instance().get_global_mem_used_ratio(); if (used_ratio >= eviction_high_watermark_ratio_) { + MasterMetricManager::instance().inc_promotion_rejected_watermark(); return; } @@ -3309,6 +3316,7 @@ void MasterService::TryPushPromotionQueue(const ObjectIdentity& object_id) { // relaxed because the value is purely advisory. if (promotion_in_flight_.load(std::memory_order_relaxed) >= promotion_queue_limit_) { + MasterMetricManager::instance().inc_promotion_rejected_cap(); return; } @@ -3350,6 +3358,8 @@ void MasterService::TryPushPromotionQueue(const ObjectIdentity& object_id) { .start_time = std::chrono::system_clock::now(), .holder_id = holder_id}); promotion_in_flight_.fetch_add(1, std::memory_order_relaxed); + MasterMetricManager::instance().inc_promotion_in_flight(); + MasterMetricManager::instance().inc_promotion_admitted(); VLOG(1) << "promotion_queued key=" << key << " size=" << object_size; } @@ -3365,17 +3375,16 @@ auto MasterService::PromotionObjectHeartbeat(const UUID& client_id) return tl::make_unexpected(ErrorCode::SEGMENT_NOT_FOUND); } MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); - // Return at most kMaxPerHeartbeat tasks. Each task does a - // synchronous SSD read + RDMA write on the client side; allowing + // Return at most promotion_max_per_heartbeat_ tasks. Each task does + // a synchronous SSD read + RDMA write on the client side; allowing // more than one per heartbeat risks blocking past the client- // liveness window and the master marking the client dead. The rest // stay queued in promotion_objects for subsequent heartbeats. The // cap must live here (server side) rather than on the client so // leftover work isn't silently dropped. - constexpr size_t kMaxPerHeartbeat = 1; auto& src = local_disk_segment_it->second->promotion_objects; std::unordered_map result; - while (result.size() < kMaxPerHeartbeat && !src.empty()) { + while (result.size() < promotion_max_per_heartbeat_ && !src.empty()) { auto node = src.extract(src.begin()); result.insert(std::move(node)); } @@ -3515,8 +3524,17 @@ auto MasterService::NotifyPromotionSuccess(const UUID& client_id, if (source != nullptr) { source->dec_refcnt(); } + const uint64_t completed_bytes = task_it->second.object_size; tenant_state.promotion_tasks.erase(task_it); promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + MasterMetricManager::instance().dec_promotion_in_flight(); + if (committed) { + MasterMetricManager::instance().inc_promotion_completed(); + MasterMetricManager::instance().inc_promotion_completed_bytes( + static_cast(completed_bytes)); + } else { + MasterMetricManager::instance().inc_promotion_cancelled(); + } // Erase the per-client promotion_objects entry (best-effort; the // heartbeat may have already drained it). @@ -3574,6 +3592,8 @@ auto MasterService::NotifyPromotionFailure(const UUID& client_id, } tenant_state.promotion_tasks.erase(task_it); promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + MasterMetricManager::instance().dec_promotion_in_flight(); + MasterMetricManager::instance().inc_promotion_failed(); // Clear the holder's per-client promotion_objects entry. Same // best-effort cleanup pattern as NotifyPromotionSuccess — the @@ -3784,6 +3804,8 @@ void MasterService::DiscardExpiredProcessingReplicas( << task_it->first; task_it = tenant_state.promotion_tasks.erase(task_it); promotion_in_flight_.fetch_sub(1, std::memory_order_relaxed); + MasterMetricManager::instance().dec_promotion_in_flight(); + MasterMetricManager::instance().inc_promotion_expired(); } if (tenant_state.Empty()) { diff --git a/mooncake-store/tests/promotion_on_hit_test.cpp b/mooncake-store/tests/promotion_on_hit_test.cpp index 73cc2544..89e2fa83 100644 --- a/mooncake-store/tests/promotion_on_hit_test.cpp +++ b/mooncake-store/tests/promotion_on_hit_test.cpp @@ -306,6 +306,9 @@ TEST_F(PromotionOnHitTest, StalePromotionReaper) { ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_cold", 1024, ctx.segment_name)); + auto& mm = MasterMetricManager::instance(); + const int64_t expired_pre = mm.get_promotion_expired(); + // Trigger #1: enqueue, then drain the per-segment queue. Drain leaves // the per-shard PromotionTask intact (the heartbeat is best-effort GC, // not the authoritative state). @@ -328,10 +331,13 @@ TEST_F(PromotionOnHitTest, StalePromotionReaper) { << "Dedup gate should block re-enqueue while task is in flight"; } - // Wait past the staleness window; the eviction thread reaps the task. - // Eviction loop sleeps for kEvictionThreadSleepMs (10 ms), so 2s wall - // clock gives ~200 attempts — plenty. - std::this_thread::sleep_for(std::chrono::seconds(2)); + // Wait past the staleness window. The eviction thread reaps every + // kEvictionThreadSleepMs (10 ms) but gated by + // `now - last_discard_time > put_start_release_timeout_sec_` (strict). + // With release=1s a 2s sleep leaves only ~1s margin between reaper + // firing and the wake-up, which has flaked under CI load. 3s gives + // ~2s of margin and matches the schedule's strict-greater comparison. + std::this_thread::sleep_for(std::chrono::seconds(3)); // Trigger #3: with the task reaped, dedup is unblocked and a fresh // GetReplicaList must enqueue again. @@ -344,6 +350,9 @@ TEST_F(PromotionOnHitTest, StalePromotionReaper) { << "After reap, a fresh read must re-enqueue the same key"; } + EXPECT_EQ(mm.get_promotion_expired() - expired_pre, 1) + << "Reaper expiry must bump promotion_expired"; + service->RemoveAll(); } @@ -368,6 +377,9 @@ TEST_F(PromotionOnHitTest, RemoveDuringPromotion) { ASSERT_TRUE(InjectLocalDiskReplica(*service, ctx.client_id, "k_cold", 1024, ctx.segment_name)); + auto& mm = MasterMetricManager::instance(); + const int64_t cancelled_pre = mm.get_promotion_cancelled(); + // Queue a promotion task. { auto r = service->GetReplicaList("k_cold"); @@ -381,6 +393,9 @@ TEST_F(PromotionOnHitTest, RemoveDuringPromotion) { ASSERT_TRUE(rm.has_value()) << "Remove on a LOCAL_DISK-only key with a queued promotion should " << "succeed (all replicas COMPLETE); error=" << rm.error(); + EXPECT_EQ(mm.get_promotion_cancelled() - cancelled_pre, 1) + << "Remove of a key with an in-flight promotion task must bump " + << "promotion_cancelled"; // NotifyPromotionSuccess on the now-removed key must surface the missing // metadata cleanly, not crash. @@ -389,8 +404,9 @@ TEST_F(PromotionOnHitTest, RemoveDuringPromotion) { EXPECT_EQ(notify.error(), ErrorCode::OBJECT_NOT_FOUND); // Wait for the reaper; it must tolerate the missing metadata entry - // (the source replica it would dec_refcnt is already gone). - std::this_thread::sleep_for(std::chrono::seconds(2)); + // (the source replica it would dec_refcnt is already gone). 3s for + // CI-safe margin over the 1s release timeout. + std::this_thread::sleep_for(std::chrono::seconds(3)); // Re-injecting the key and re-triggering must work end-to-end, proving // the per-shard PromotionTask was reaped (not stuck). @@ -547,6 +563,9 @@ TEST_F(PromotionOnHitTest, QueueLimitRejectsBeyondCap) { ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, k2, 1024, seg.segment_name)); + auto& mm = MasterMetricManager::instance(); + const int64_t cap_rej_pre = mm.get_promotion_rejected_cap(); + // First read on k1 enqueues a task in shard S. auto r1 = service->GetReplicaList(k1); ASSERT_TRUE(r1.has_value()); @@ -568,6 +587,8 @@ TEST_F(PromotionOnHitTest, QueueLimitRejectsBeyondCap) { << "k1 was read first and should be the surviving task"; EXPECT_EQ(heartbeat->count(k2), 0u) << "k2 was rejected by the cap gate; should not appear"; + EXPECT_EQ(mm.get_promotion_rejected_cap() - cap_rej_pre, 1) + << "k2's rejection must increment promotion_rejected_cap"; service->RemoveAll(); } @@ -1008,9 +1029,9 @@ TEST_F(PromotionOnHitTest, AllocStartRejectsReapedTask) { // removed. We can't easily poll for reap externally (the only // user-facing observable would be re-admitting through the gate, // which would create a fresh task and defeat the test), so use a - // fixed sleep with margin. Matches the StalePromotionReaper pattern - // (TTL=1s, sleep=2s). - std::this_thread::sleep_for(std::chrono::seconds(2)); + // fixed sleep with margin. 3s sleep over 1s release timeout matches + // StalePromotionReaper's CI-safe margin. + std::this_thread::sleep_for(std::chrono::seconds(3)); // AllocStart on a reaped task must reject without allocating. // Allocating would leave an orphaned PROCESSING MEMORY replica @@ -1133,6 +1154,9 @@ TEST_F(PromotionOnHitTest, NotifyFailureReleasesStateImmediately) { ASSERT_TRUE(seg_baseline.has_value()); const size_t used_baseline = seg_baseline->first; + auto& mm = MasterMetricManager::instance(); + const int64_t failed_pre = mm.get_promotion_failed(); + // Admit + stage k_a. promotion_in_flight_ goes 0 -> 1. { auto r = service->GetReplicaList("k_a"); @@ -1154,6 +1178,8 @@ TEST_F(PromotionOnHitTest, NotifyFailureReleasesStateImmediately) { ASSERT_TRUE(failure.has_value()) << "NotifyPromotionFailure on a valid in-flight task from the " << "legitimate holder must succeed; error=" << failure.error(); + EXPECT_EQ(mm.get_promotion_failed() - failed_pre, 1) + << "holder-reported failure must bump promotion_failed"; // The staged buffer must be freed back to the DRAM allocator. If // this fires, NotifyPromotionFailure did not pop the staged replica @@ -1387,6 +1413,9 @@ TEST_F(PromotionOnHitTest, ClientExpiryClearsPromotionTask) { ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_cold", 1024, holder.segment_name)); + auto& mm = MasterMetricManager::instance(); + const int64_t cancelled_pre = mm.get_promotion_cancelled(); + // Admit the promotion. promotion_in_flight_ goes 0 -> 1. { auto r = service->GetReplicaList("k_cold"); @@ -1464,6 +1493,9 @@ TEST_F(PromotionOnHitTest, ClientExpiryClearsPromotionTask) { << "saturated by the dead holder's task for " << "put_start_release_timeout_sec_ seconds, and this admission " << "is dropped."; + EXPECT_EQ(mm.get_promotion_cancelled() - cancelled_pre, 1) + << "Holder-client expiry mid-promotion must bump " + << "promotion_cancelled"; service->RemoveAll(); } @@ -1612,6 +1644,437 @@ TEST_F(PromotionOnHitTest, RemoveByRegexErasesPromotionTask) { service->RemoveAll(); } +// RemoveAll on a key with an in-flight PromotionTask must drop the +// task entry alongside the metadata. Same shape as +// RemoveErasesPromotionTask but exercises the bulk-erase loop in +// MasterService::RemoveAll, which iterates every shard and erases +// metadata entries directly. +TEST_F(PromotionOnHitTest, RemoveAllErasesPromotionTask) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; + config.default_kv_lease_ttl = 5000; + config.put_start_release_timeout_sec = 300; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_first", + 1024, holder.segment_name)); + + auto& mm = MasterMetricManager::instance(); + const int64_t cancelled_pre = mm.get_promotion_cancelled(); + + { + auto r = service->GetReplicaList("k_first"); + ASSERT_TRUE(r.has_value()); + } + { + auto pending = service->PromotionObjectHeartbeat(holder.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->count("k_first"), 1u); + } + + auto removed = service->RemoveAll(/*force=*/true); + EXPECT_GE(removed, 1) << "RemoveAll should erase k_first"; + + EXPECT_EQ(mm.get_promotion_cancelled() - cancelled_pre, 1) + << "RemoveAll on a key with an in-flight promotion must route " + << "through EraseMetadataEntry and bump promotion_cancelled_total."; + + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_second", + 1024, holder.segment_name)); + { + auto r = service->GetReplicaList("k_second"); + ASSERT_TRUE(r.has_value()); + } + auto pending_post = service->PromotionObjectHeartbeat(holder.client_id); + ASSERT_TRUE(pending_post.has_value()); + EXPECT_EQ(pending_post->count("k_second"), 1u) + << "k_second must be admittable after RemoveAll of k_first — " + << "otherwise queue_limit=1 stays saturated until reaper TTL."; + + service->RemoveAll(/*force=*/true); +} + +// BatchRemove normal-completion path on a key with an in-flight +// PromotionTask must drop the task entry. ReMountSegment registers the +// holder in ok_client_ so CleanupStaleHandles returns false and +// BatchRemove takes the non-stale branch. +TEST_F(PromotionOnHitTest, BatchRemoveErasesPromotionTask) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; + config.default_kv_lease_ttl = 5000; + config.put_start_release_timeout_sec = 300; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + { + Segment seg_a = MakeSegment("seg_a", kDefaultSegmentBase, seg_size); + seg_a.id = holder.segment_id; + std::vector segs{seg_a}; + auto remount = service->ReMountSegment(segs, holder.client_id); + ASSERT_TRUE(remount.has_value()) << "ReMount failed"; + } + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_first", + 1024, holder.segment_name)); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_second", + 1024, holder.segment_name)); + + auto& mm = MasterMetricManager::instance(); + const int64_t cancelled_pre = mm.get_promotion_cancelled(); + + { + auto r = service->GetReplicaList("k_first"); + ASSERT_TRUE(r.has_value()); + } + { + auto pending = service->PromotionObjectHeartbeat(holder.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->count("k_first"), 1u); + } + + auto results = service->BatchRemove({"k_first"}, /*force=*/true); + ASSERT_EQ(results.size(), 1u); + EXPECT_TRUE(results[0].has_value()) + << "BatchRemove should succeed; error=" << results[0].error(); + + EXPECT_EQ(mm.get_promotion_cancelled() - cancelled_pre, 1) + << "BatchRemove normal path on a key with an in-flight promotion " + << "must bump promotion_cancelled_total."; + + { + auto r = service->GetReplicaList("k_second"); + ASSERT_TRUE(r.has_value()); + } + auto pending_post = service->PromotionObjectHeartbeat(holder.client_id); + ASSERT_TRUE(pending_post.has_value()); + EXPECT_EQ(pending_post->count("k_second"), 1u) + << "k_second must be admittable after BatchRemove of k_first — " + << "otherwise queue_limit=1 stays saturated until reaper TTL."; + + service->RemoveAll(/*force=*/true); +} + +// BatchRemove stale-handle path on a key with an in-flight +// PromotionTask must drop the task entry. The holder is mounted via +// PrepareSegment only (no ReMount), so its client is absent from +// ok_client_; BatchRemove's CleanupStaleHandles then erases the +// LOCAL_DISK replica and the stale-handle branch fires. +TEST_F(PromotionOnHitTest, BatchRemoveStaleHandleErasesPromotionTask) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_queue_limit = 1; + config.default_kv_lease_ttl = 5000; + config.put_start_release_timeout_sec = 300; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto holder = + PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, holder.client_id, "k_first", + 1024, holder.segment_name)); + + auto& mm = MasterMetricManager::instance(); + const int64_t cancelled_pre = mm.get_promotion_cancelled(); + + { + auto r = service->GetReplicaList("k_first"); + ASSERT_TRUE(r.has_value()); + } + { + auto pending = service->PromotionObjectHeartbeat(holder.client_id); + ASSERT_TRUE(pending.has_value()); + EXPECT_EQ(pending->count("k_first"), 1u); + } + + auto results = service->BatchRemove({"k_first"}, /*force=*/true); + ASSERT_EQ(results.size(), 1u); + EXPECT_FALSE(results[0].has_value()) + << "stale-handle path should report OBJECT_NOT_FOUND once the " + << "LOCAL_DISK replica is wiped."; + + EXPECT_EQ(mm.get_promotion_cancelled() - cancelled_pre, 1) + << "BatchRemove stale-handle path must also bump " + << "promotion_cancelled_total."; + + auto second_holder = PrepareSegment( + *service, "seg_b", kDefaultSegmentBase + seg_size, seg_size); + { + Segment seg_b = + MakeSegment("seg_b", kDefaultSegmentBase + seg_size, seg_size); + seg_b.id = second_holder.segment_id; + std::vector segs{seg_b}; + auto remount = service->ReMountSegment(segs, second_holder.client_id); + ASSERT_TRUE(remount.has_value()) << "ReMount failed"; + } + ASSERT_TRUE(InjectLocalDiskReplica(*service, second_holder.client_id, + "k_second", 1024, + second_holder.segment_name)); + { + auto r = service->GetReplicaList("k_second"); + ASSERT_TRUE(r.has_value()); + } + auto pending_post = + service->PromotionObjectHeartbeat(second_holder.client_id); + ASSERT_TRUE(pending_post.has_value()); + EXPECT_EQ(pending_post->count("k_second"), 1u) + << "k_second must be admittable after the stale-handle " + << "BatchRemove — otherwise queue_limit=1 stays saturated until " + << "reaper TTL."; + + service->RemoveAll(/*force=*/true); +} + +// The funnel metrics (admitted, completed, completed_bytes, in_flight) +// track a successful promotion lifecycle end-to-end: a single +// admission bumps admitted+1 and in_flight+1; NotifyPromotionSuccess +// bumps completed+1 and completed_bytes+object_size and brings +// in_flight back to 0. +TEST_F(PromotionOnHitTest, MetricsFunnelTracksSuccessfulPromotion) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + constexpr int64_t kObjBytes = 4096; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k_hot", + kObjBytes, seg.segment_name)); + + auto& mm = MasterMetricManager::instance(); + const int64_t admitted_pre = mm.get_promotion_admitted(); + const int64_t completed_pre = mm.get_promotion_completed(); + const int64_t bytes_pre = mm.get_promotion_completed_bytes(); + const int64_t in_flight_pre = mm.get_promotion_in_flight(); + + // Admit. + { + auto r = service->GetReplicaList("k_hot"); + ASSERT_TRUE(r.has_value()); + } + EXPECT_EQ(mm.get_promotion_admitted() - admitted_pre, 1); + EXPECT_EQ(mm.get_promotion_in_flight() - in_flight_pre, 1); + + // Drive AllocStart + NotifyPromotionSuccess. + auto alloc = + service->PromotionAllocStart(seg.client_id, "k_hot", kObjBytes, {}); + ASSERT_TRUE(alloc.has_value()); + auto notify = service->NotifyPromotionSuccess(seg.client_id, "k_hot"); + ASSERT_TRUE(notify.has_value()); + + EXPECT_EQ(mm.get_promotion_completed() - completed_pre, 1); + EXPECT_EQ(mm.get_promotion_completed_bytes() - bytes_pre, kObjBytes); + EXPECT_EQ(mm.get_promotion_in_flight() - in_flight_pre, 0); + + service->RemoveAll(); +} + +// Each rejection gate (frequency / watermark / cap) increments its +// own counter when its branch fires. +TEST_F(PromotionOnHitTest, MetricsRejectionCountersIncrementOnGateMiss) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + // Threshold > 1 so the first Get is rejected on frequency. + config.promotion_admission_threshold = 2; + config.promotion_queue_limit = 1; + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k_a", 1024, + seg.segment_name)); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k_b", 1024, + seg.segment_name)); + + auto& mm = MasterMetricManager::instance(); + const int64_t freq_pre = mm.get_promotion_rejected_frequency(); + const int64_t cap_pre = mm.get_promotion_rejected_cap(); + + // 1st Get on k_a: freq=1, threshold=2 → rejected on frequency. + { + auto r = service->GetReplicaList("k_a"); + ASSERT_TRUE(r.has_value()); + } + EXPECT_EQ(mm.get_promotion_rejected_frequency() - freq_pre, 1); + + // 2nd Get on k_a: freq=2, admits. Now in-flight = 1 == limit. + { + auto r = service->GetReplicaList("k_a"); + ASSERT_TRUE(r.has_value()); + } + // Get on k_b: freq=1, threshold=2 → rejected on frequency. + { + auto r = service->GetReplicaList("k_b"); + ASSERT_TRUE(r.has_value()); + } + // Get on k_b again: freq=2, gets past frequency, but cap=1 + // saturated → rejected on cap. + { + auto r = service->GetReplicaList("k_b"); + ASSERT_TRUE(r.has_value()); + } + EXPECT_EQ(mm.get_promotion_rejected_cap() - cap_pre, 1); + + service->RemoveAll(); + + // Watermark gate uses a fresh service configured with + // eviction_high_watermark_ratio = 0.0 so any non-negative DRAM + // usage trips it. threshold = 1 makes the first Get clear the + // frequency gate and reach the watermark check. + MasterServiceConfig wm_config; + wm_config.enable_offload = true; + wm_config.promotion_on_hit = true; + wm_config.promotion_admission_threshold = 1; + wm_config.promotion_queue_limit = 50; + wm_config.eviction_high_watermark_ratio = 0.0; + wm_config.default_kv_lease_ttl = 2000; + auto wm_service = std::make_unique(wm_config); + + auto wm_seg = + PrepareSegment(*wm_service, "wm_seg", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*wm_service, wm_seg.client_id, "k_w", + 1024, wm_seg.segment_name)); + + const int64_t wm_pre = mm.get_promotion_rejected_watermark(); + { + auto r = wm_service->GetReplicaList("k_w"); + ASSERT_TRUE(r.has_value()); + } + EXPECT_EQ(mm.get_promotion_rejected_watermark() - wm_pre, 1); + + wm_service->RemoveAll(); +} + +// promotion_max_per_heartbeat controls how many tasks +// PromotionObjectHeartbeat returns per call. Set the knob to 3 and +// verify the master returns up to 3 tasks per heartbeat. +TEST_F(PromotionOnHitTest, MaxPerHeartbeatKnobControlsBatchSize) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_max_per_heartbeat = 3; // raise from default 1 + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + + // Admit 5 tasks on the same holder. + for (int i = 0; i < 5; ++i) { + const auto key = "k_" + std::to_string(i); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, key, 1024, + seg.segment_name)); + auto r = service->GetReplicaList(key); + ASSERT_TRUE(r.has_value()); + } + + // First heartbeat: cap=3. + auto first = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(first.has_value()); + EXPECT_EQ(first->size(), 3u) + << "expected promotion_max_per_heartbeat=3 keys, got " << first->size(); + + // Second heartbeat: 2 leftover. + auto second = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(second.has_value()); + EXPECT_EQ(second->size(), 2u); + + // Third heartbeat: empty. + auto third = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(third.has_value()); + EXPECT_EQ(third->size(), 0u); + + service->RemoveAll(); +} + +// promotion_max_per_heartbeat = 0 must be clamped to 1 by the +// MasterService constructor; otherwise PromotionObjectHeartbeat would +// return an empty batch every call and silently disable promotion +// delivery. +TEST_F(PromotionOnHitTest, MaxPerHeartbeatZeroClampsToOne) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.promotion_max_per_heartbeat = 0; // pathological + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k1", 1024, + seg.segment_name)); + { + auto r = service->GetReplicaList("k1"); + ASSERT_TRUE(r.has_value()); + } + + // If max_per_heartbeat=0 weren't clamped, this would return empty. + auto hb = service->PromotionObjectHeartbeat(seg.client_id); + ASSERT_TRUE(hb.has_value()); + EXPECT_EQ(hb->size(), 1u) + << "max_per_heartbeat=0 must clamp to 1 so promotion delivery " + << "isn't silently disabled"; + + service->RemoveAll(); +} + +// Remove on a key mid-promotion must bump promotion_cancelled so the +// funnel invariant +// admitted = completed + failed + expired + cancelled + in_flight +// holds. Exercises the EraseMetadataEntry path. +TEST_F(PromotionOnHitTest, MetricsRemoveMidPromotionCountsAsCancelled) { + MasterServiceConfig config; + config.enable_offload = true; + config.promotion_on_hit = true; + config.promotion_admission_threshold = 1; + config.default_kv_lease_ttl = 2000; + auto service = std::make_unique(config); + + constexpr size_t seg_size = 1024 * 1024 * 16; + auto seg = PrepareSegment(*service, "seg_a", kDefaultSegmentBase, seg_size); + ASSERT_TRUE(InjectLocalDiskReplica(*service, seg.client_id, "k_drop", 1024, + seg.segment_name)); + + auto& mm = MasterMetricManager::instance(); + const int64_t admitted_pre = mm.get_promotion_admitted(); + const int64_t cancelled_pre = mm.get_promotion_cancelled(); + const int64_t in_flight_pre = mm.get_promotion_in_flight(); + + // Admit a promotion. in_flight goes from 0 to 1. + { + auto r = service->GetReplicaList("k_drop"); + ASSERT_TRUE(r.has_value()); + } + ASSERT_EQ(mm.get_promotion_admitted() - admitted_pre, 1); + ASSERT_EQ(mm.get_promotion_in_flight() - in_flight_pre, 1); + + auto rm = service->Remove("k_drop", /*force=*/true); + ASSERT_TRUE(rm.has_value()) << "error=" << rm.error(); + + EXPECT_EQ(mm.get_promotion_in_flight() - in_flight_pre, 0); + EXPECT_EQ(mm.get_promotion_cancelled() - cancelled_pre, 1); + + service->RemoveAll(); +} + } // namespace mooncake::test int main(int argc, char** argv) { -- 2.34.1 From ecfa92d518b7412d97c0f9716222a7f6a191cf0a Mon Sep 17 00:00:00 2001 From: Willow Lopez <100782273+Oxygen56@users.noreply.github.com> Date: Mon, 1 Jun 2026 11:23:58 +0800 Subject: [PATCH 155/382] fix(metrics): show actual client-reported SSD capacity instead of infinite (#2278) * fix(metrics): show actual client-reported SSD capacity instead of infinite When no global file segment size limit is configured, dfs_capacity_unlimited_ is set to true, causing the metrics display to show 'infinite' for SSD Storage capacity regardless of actual capacity reported by clients via ReportSsdCapacity. Fix the display logic to only show 'infinite' when clients have reported NO capacity (file_capacity == 0). If clients have reported their actual SSD capacity via ReportSsdCapacity, use that value instead. This respects per-client SSD capacity limits even when no global limit is configured. Co-Authored-By: Claude Opus 4.8 * fix: align get_total_file_capacity and get_global_file_used_ratio with display logic Address review feedback from gemini-code-assist: update programmatic APIs to use the same (dfs_capacity_unlimited_ && file_capacity == 0) condition as the display, so they don't unconditionally return unlimited when clients have reported actual capacity. * Apply suggestions from code review Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: Autumn Co-authored-by: Claude Opus 4.8 Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- mooncake-store/src/master_metric_manager.cpp | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mooncake-store/src/master_metric_manager.cpp b/mooncake-store/src/master_metric_manager.cpp index cdaec57b..7ac34129 100644 --- a/mooncake-store/src/master_metric_manager.cpp +++ b/mooncake-store/src/master_metric_manager.cpp @@ -765,14 +765,14 @@ int64_t MasterMetricManager::get_allocated_file_size() { } int64_t MasterMetricManager::get_total_file_capacity() { - if (dfs_capacity_unlimited_) { + if (dfs_capacity_unlimited_ && file_total_capacity_.value() == 0) { return std::numeric_limits::max(); } return file_total_capacity_.value(); } double MasterMetricManager::get_global_file_used_ratio(void) { - if (dfs_capacity_unlimited_) { + if (dfs_capacity_unlimited_ && file_total_capacity_.value() == 0) { return 0.0; } double allocated = file_allocated_size_.value(); @@ -2166,9 +2166,7 @@ std::string MasterMetricManager::get_summary_string( ss << " (" << std::fixed << std::setprecision(1) << ((double)mem_allocated / (double)mem_capacity * 100.0) << "%)"; } - int64_t file_display_capacity = dfs_capacity_unlimited_ - ? std::numeric_limits::max() - : file_capacity; + int64_t file_display_capacity = get_total_file_capacity(); ss << " | NVMe-oF SSD: " << byte_size_to_string(nof_allocated) << " / " << byte_size_to_string(nof_capacity); if (nof_capacity > 0) { -- 2.34.1 From 6cd60d651678326f2c9c6658b96a8a35831ad249 Mon Sep 17 00:00:00 2001 From: Aoi Date: Mon, 1 Jun 2026 15:27:08 +0800 Subject: [PATCH 156/382] [TE][Store] Fix IPv6 address parsing in connection endpoints (#2184) --- mooncake-store/src/master_service.cpp | 15 +-- mooncake-store/src/real_client.cpp | 21 ++- .../master_service_test_for_snapshot.cpp | 111 ++++++++++++++++ mooncake-store/tests/master_service_test.cpp | 111 ++++++++++++++++ mooncake-transfer-engine/include/common.h | 27 ++++ .../tent/src/common/ip.cpp | 84 +++++++----- .../tent/tests/CMakeLists.txt | 6 + .../tent/tests/ip_utils_test.cpp | 125 ++++++++++++++++++ .../tests/common_test.cpp | 21 +++ 9 files changed, 467 insertions(+), 54 deletions(-) create mode 100644 mooncake-transfer-engine/tent/tests/ip_utils_test.cpp diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index ba3a6992..aa435920 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -15,6 +15,7 @@ #include #include "master_metric_manager.h" +#include "common.h" #include "segment.h" #ifdef USE_NOF #include "spdk/spdk_wrapper.h" @@ -512,7 +513,8 @@ auto MasterService::MountNoFSegment(const NoFSegment& segment, ScopedNoFSegmentAccess nof_segment_access = nof_segment_manager_.getNoFSegmentAccess(); - LOG(INFO) << "NoF segment mount: " << "client_id=" << client_id + LOG(INFO) << "NoF segment mount: " + << "client_id=" << client_id << ", action=mount_segment, segment_name=" << segment.name; auto err = nof_segment_access.MountSegment(segment, client_id); @@ -1075,13 +1077,7 @@ auto MasterService::QueryIp(const UUID& client_id) unique_ips.reserve(segments.size()); for (const auto& segment : segments) { if (!segment.te_endpoint.empty()) { - size_t colon_pos = segment.te_endpoint.find(':'); - if (colon_pos != std::string::npos) { - std::string ip = segment.te_endpoint.substr(0, colon_pos); - unique_ips.emplace(ip); - } else { - unique_ips.emplace(segment.te_endpoint); - } + unique_ips.emplace(getHostNameWithoutPort(segment.te_endpoint)); } } @@ -5320,7 +5316,8 @@ void MasterService::BatchEvict(double evict_ratio_target, MasterMetricManager::instance().inc_eviction_fail(); MasterMetricManager::instance().inc_mem_eviction_fail(); } - VLOG(1) << "action=evict_objects" << ", evicted_count=" << evicted_count + VLOG(1) << "action=evict_objects" + << ", evicted_count=" << evicted_count << ", offload_deferred=" << offload_deferred_count << ", offload_cap_forced=" << offload_cap_forced_count << ", offload_push_failed_forced=" << offload_push_failed_forced diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 0af42bbc..9f017345 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -22,6 +22,7 @@ #include "real_client.h" #include "client_buffer.hpp" +#include "common.h" #include "config.h" #include "mutex.h" #include "types.h" @@ -667,14 +668,13 @@ tl::expected RealClient::setup_internal( // Check if hostname already contains a port const std::string &hostname = local_hostname; - size_t colon_pos = hostname.find(':'); - bool user_specified_port = (colon_pos != std::string::npos); + bool user_specified_port = hasExplicitPort(hostname); if (user_specified_port) { // User specified port, no retry needed this->local_hostname = local_hostname; - this->local_rpc_addr = - hostname.substr(0, colon_pos + 1) + std::to_string(local_rpc_port); + this->local_rpc_addr = buildHostNameWithPort( + getHostNameWithoutPort(hostname), local_rpc_port); auto client_opt = mooncake::Client::Create( this->local_hostname, metadata_server, protocol, device_name, master_server_addr, transfer_engine, {{"client_mode", "real"}}); @@ -707,9 +707,9 @@ tl::expected RealClient::setup_internal( continue; } - this->local_hostname = hostname + ":" + std::to_string(port); + this->local_hostname = buildHostNameWithPort(hostname, port); this->local_rpc_addr = - hostname + ":" + std::to_string(local_rpc_port); + buildHostNameWithPort(hostname, local_rpc_port); auto client_opt = mooncake::Client::Create( this->local_hostname, metadata_server, protocol, device_name, master_server_addr, transfer_engine, {{"client_mode", "real"}}); @@ -903,13 +903,8 @@ tl::expected RealClient::setup_internal( LOG(INFO) << "Offload RPC server started on port " << offload_rpc_port_; // Build local_rpc_addr from hostname + auto-allocated port - std::string rpc_host = this->local_hostname; - auto pos = rpc_host.find(':'); - if (pos != std::string::npos) { - rpc_host = rpc_host.substr(0, pos); - } - this->local_rpc_addr = - rpc_host + ":" + std::to_string(offload_rpc_port_); + this->local_rpc_addr = buildHostNameWithPort( + getHostNameWithoutPort(this->local_hostname), offload_rpc_port_); } if (enable_ssd_offload) { auto file_storage_config = FileStorageConfig::FromEnvironment(); diff --git a/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot.cpp b/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot.cpp index ec9f290b..6ec222f4 100644 --- a/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot.cpp +++ b/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot.cpp @@ -2156,6 +2156,117 @@ TEST_F(MasterServiceSnapshotTest, << "Client with all empty te_endpoints should have empty IP vector"; } +TEST_F(MasterServiceSnapshotTest, BatchQueryIpBracketedIpv6Test) { + service_.reset(new MasterService()); + const UUID client_id = generate_uuid(); + + // Mount a segment with a bracketed IPv6 endpoint + constexpr size_t buffer = 0x300000000; + constexpr size_t size = 1024 * 1024 * 16; + Segment segment = MakeSegment("test_segment", buffer, size); + segment.te_endpoint = "[::1]:17813"; + auto mount_result = service_->MountSegment(segment, client_id); + ASSERT_TRUE(mount_result.has_value()); + + std::vector client_ids = {client_id}; + auto query_result = service_->BatchQueryIp(client_ids); + ASSERT_TRUE(query_result.has_value()); + + const auto& results = query_result.value(); + auto it = results.find(client_id); + ASSERT_NE(it, results.end()); + + const auto& ip_addresses = it->second; + ASSERT_EQ(1u, ip_addresses.size()); + EXPECT_EQ("::1", ip_addresses[0]); +} + +TEST_F(MasterServiceSnapshotTest, BatchQueryIpLinkLocalIpv6WithScopeTest) { + service_.reset(new MasterService()); + const UUID client_id = generate_uuid(); + + // Mount a segment with a link-local IPv6 address with scope ID + constexpr size_t buffer = 0x300000000; + constexpr size_t size = 1024 * 1024 * 16; + Segment segment = MakeSegment("test_segment", buffer, size); + segment.te_endpoint = "fe80::a236:bcff:fecb:a1be%eno2:15773"; + auto mount_result = service_->MountSegment(segment, client_id); + ASSERT_TRUE(mount_result.has_value()); + + std::vector client_ids = {client_id}; + auto query_result = service_->BatchQueryIp(client_ids); + ASSERT_TRUE(query_result.has_value()); + + const auto& results = query_result.value(); + auto it = results.find(client_id); + ASSERT_NE(it, results.end()); + + const auto& ip_addresses = it->second; + ASSERT_EQ(1u, ip_addresses.size()); + EXPECT_EQ("fe80::a236:bcff:fecb:a1be%eno2", ip_addresses[0]); +} + +TEST_F(MasterServiceSnapshotTest, BatchQueryIpIpv6NoPortTest) { + service_.reset(new MasterService()); + const UUID client_id = generate_uuid(); + + // Mount a segment with an IPv6 address without port + constexpr size_t buffer = 0x300000000; + constexpr size_t size = 1024 * 1024 * 16; + Segment segment = MakeSegment("test_segment", buffer, size); + segment.te_endpoint = "::1"; + auto mount_result = service_->MountSegment(segment, client_id); + ASSERT_TRUE(mount_result.has_value()); + + std::vector client_ids = {client_id}; + auto query_result = service_->BatchQueryIp(client_ids); + ASSERT_TRUE(query_result.has_value()); + + const auto& results = query_result.value(); + auto it = results.find(client_id); + ASSERT_NE(it, results.end()); + + const auto& ip_addresses = it->second; + ASSERT_EQ(1u, ip_addresses.size()); + EXPECT_EQ("::1", ip_addresses[0]); +} + +TEST_F(MasterServiceSnapshotTest, BatchQueryIpMixedIpv4AndIpv6Test) { + service_.reset(new MasterService()); + const UUID client_id = generate_uuid(); + + // Mount segments with IPv4 and IPv6 endpoints for the same client + constexpr size_t buffer1 = 0x300000000; + constexpr size_t buffer2 = 0x400000000; + constexpr size_t size = 1024 * 1024 * 16; + + Segment segment1 = MakeSegment("segment1", buffer1, size); + segment1.te_endpoint = "192.168.1.1:12345"; + auto mount_result1 = service_->MountSegment(segment1, client_id); + ASSERT_TRUE(mount_result1.has_value()); + + Segment segment2 = MakeSegment("segment2", buffer2, size); + segment2.te_endpoint = "[::1]:17813"; + auto mount_result2 = service_->MountSegment(segment2, client_id); + ASSERT_TRUE(mount_result2.has_value()); + + std::vector client_ids = {client_id}; + auto query_result = service_->BatchQueryIp(client_ids); + ASSERT_TRUE(query_result.has_value()); + + const auto& results = query_result.value(); + auto it = results.find(client_id); + ASSERT_NE(it, results.end()); + + const auto& ip_addresses = it->second; + ASSERT_EQ(2u, ip_addresses.size()); + + std::unordered_set ip_set(ip_addresses.begin(), + ip_addresses.end()); + EXPECT_NE(ip_set.find("192.168.1.1"), ip_set.end()); + EXPECT_NE(ip_set.find("::1"), ip_set.end()); +} + TEST_F(MasterServiceSnapshotTest, PutStartExpiringTest) { // Reset storage space metrics. MasterMetricManager::instance().reset_allocated_mem_size(); diff --git a/mooncake-store/tests/master_service_test.cpp b/mooncake-store/tests/master_service_test.cpp index 1cf903c6..7e296ec5 100644 --- a/mooncake-store/tests/master_service_test.cpp +++ b/mooncake-store/tests/master_service_test.cpp @@ -4149,6 +4149,117 @@ TEST_F(MasterServiceTest, BatchQueryIpMultipleSegmentsEmptyTeEndpointTest) { << "Client with all empty te_endpoints should have empty IP vector"; } +TEST_F(MasterServiceTest, BatchQueryIpBracketedIpv6Test) { + std::unique_ptr service_(new MasterService()); + const UUID client_id = generate_uuid(); + + // Mount a segment with a bracketed IPv6 endpoint + constexpr size_t buffer = 0x300000000; + constexpr size_t size = 1024 * 1024 * 16; + Segment segment = MakeSegment("test_segment", buffer, size); + segment.te_endpoint = "[::1]:17813"; + auto mount_result = service_->MountSegment(segment, client_id); + ASSERT_TRUE(mount_result.has_value()); + + std::vector client_ids = {client_id}; + auto query_result = service_->BatchQueryIp(client_ids); + ASSERT_TRUE(query_result.has_value()); + + const auto& results = query_result.value(); + auto it = results.find(client_id); + ASSERT_NE(it, results.end()); + + const auto& ip_addresses = it->second; + ASSERT_EQ(1u, ip_addresses.size()); + EXPECT_EQ("::1", ip_addresses[0]); +} + +TEST_F(MasterServiceTest, BatchQueryIpLinkLocalIpv6WithScopeTest) { + std::unique_ptr service_(new MasterService()); + const UUID client_id = generate_uuid(); + + // Mount a segment with a link-local IPv6 address with scope ID + constexpr size_t buffer = 0x300000000; + constexpr size_t size = 1024 * 1024 * 16; + Segment segment = MakeSegment("test_segment", buffer, size); + segment.te_endpoint = "fe80::a236:bcff:fecb:a1be%eno2:15773"; + auto mount_result = service_->MountSegment(segment, client_id); + ASSERT_TRUE(mount_result.has_value()); + + std::vector client_ids = {client_id}; + auto query_result = service_->BatchQueryIp(client_ids); + ASSERT_TRUE(query_result.has_value()); + + const auto& results = query_result.value(); + auto it = results.find(client_id); + ASSERT_NE(it, results.end()); + + const auto& ip_addresses = it->second; + ASSERT_EQ(1u, ip_addresses.size()); + EXPECT_EQ("fe80::a236:bcff:fecb:a1be%eno2", ip_addresses[0]); +} + +TEST_F(MasterServiceTest, BatchQueryIpIpv6NoPortTest) { + std::unique_ptr service_(new MasterService()); + const UUID client_id = generate_uuid(); + + // Mount a segment with an IPv6 address without port + constexpr size_t buffer = 0x300000000; + constexpr size_t size = 1024 * 1024 * 16; + Segment segment = MakeSegment("test_segment", buffer, size); + segment.te_endpoint = "::1"; + auto mount_result = service_->MountSegment(segment, client_id); + ASSERT_TRUE(mount_result.has_value()); + + std::vector client_ids = {client_id}; + auto query_result = service_->BatchQueryIp(client_ids); + ASSERT_TRUE(query_result.has_value()); + + const auto& results = query_result.value(); + auto it = results.find(client_id); + ASSERT_NE(it, results.end()); + + const auto& ip_addresses = it->second; + ASSERT_EQ(1u, ip_addresses.size()); + EXPECT_EQ("::1", ip_addresses[0]); +} + +TEST_F(MasterServiceTest, BatchQueryIpMixedIpv4AndIpv6Test) { + std::unique_ptr service_(new MasterService()); + const UUID client_id = generate_uuid(); + + // Mount segments with IPv4 and IPv6 endpoints for the same client + constexpr size_t buffer1 = 0x300000000; + constexpr size_t buffer2 = 0x400000000; + constexpr size_t size = 1024 * 1024 * 16; + + Segment segment1 = MakeSegment("segment1", buffer1, size); + segment1.te_endpoint = "192.168.1.1:12345"; + auto mount_result1 = service_->MountSegment(segment1, client_id); + ASSERT_TRUE(mount_result1.has_value()); + + Segment segment2 = MakeSegment("segment2", buffer2, size); + segment2.te_endpoint = "[::1]:17813"; + auto mount_result2 = service_->MountSegment(segment2, client_id); + ASSERT_TRUE(mount_result2.has_value()); + + std::vector client_ids = {client_id}; + auto query_result = service_->BatchQueryIp(client_ids); + ASSERT_TRUE(query_result.has_value()); + + const auto& results = query_result.value(); + auto it = results.find(client_id); + ASSERT_NE(it, results.end()); + + const auto& ip_addresses = it->second; + ASSERT_EQ(2u, ip_addresses.size()); + + std::unordered_set ip_set(ip_addresses.begin(), + ip_addresses.end()); + EXPECT_NE(ip_set.find("192.168.1.1"), ip_set.end()); + EXPECT_NE(ip_set.find("::1"), ip_set.end()); +} + TEST_F(MasterServiceTest, PutStartExpiringTest) { // Reset storage space metrics. MasterMetricManager::instance().reset_allocated_mem_size(); diff --git a/mooncake-transfer-engine/include/common.h b/mooncake-transfer-engine/include/common.h index f98ba147..621c256d 100644 --- a/mooncake-transfer-engine/include/common.h +++ b/mooncake-transfer-engine/include/common.h @@ -267,6 +267,33 @@ static inline std::pair parseHostNameWithPort( getPortFromString(server_name.substr(colon_pos + 1), port)}; } +static inline bool hasExplicitPort(const std::string &server_name) { + auto result = extractIPv6HostAndPort(server_name); + if (result.matched) { + return !result.port.empty(); + } + return server_name.rfind(':') != std::string::npos; +} + +static inline std::string getHostNameWithoutPort( + const std::string &server_name) { + auto result = extractIPv6HostAndPort(server_name); + if (result.matched) { + return std::move(result.host); + } + + const size_t colon_pos = server_name.rfind(':'); + if (colon_pos == std::string::npos) { + return server_name; + } + return server_name.substr(0, colon_pos); +} + +static inline std::string buildHostNameWithPort(const std::string &host, + uint16_t port) { + return maybeWrapIpV6(host) + ":" + std::to_string(port); +} + static inline uint16_t parsePortAndDevice(std::string_view suffix, uint16_t default_port, int *device_id) { diff --git a/mooncake-transfer-engine/tent/src/common/ip.cpp b/mooncake-transfer-engine/tent/src/common/ip.cpp index 299c0fa6..a1323806 100644 --- a/mooncake-transfer-engine/tent/src/common/ip.cpp +++ b/mooncake-transfer-engine/tent/src/common/ip.cpp @@ -13,6 +13,8 @@ // limitations under the License. #include "tent/common/utils/ip.h" +#include "common.h" + #include namespace mooncake { @@ -92,42 +94,60 @@ Status checkLocalIpAddress(std::string &hostname, bool &ipv6) { std::pair parseHostNameWithPort(const std::string &url, uint16_t default_port) { uint16_t port = default_port; + if (url.empty()) return std::make_pair(url, port); - // Check if url is valid IPv6 address - in6_addr addr; - if (inet_pton(AF_INET6, url.c_str(), &addr) == 1) return {url, port}; - - // Check if url is valid IPv6 address with port - size_t start_pos = 0; - size_t end_pos = url.find_last_of(']'); - size_t port_pos = url.find_last_of(':'); - if (url.front() == '[' && end_pos != std::string::npos && - port_pos > end_pos) { - auto ip = url.substr(start_pos + 1, end_pos - start_pos - 1); - std::string port_str = url.substr(port_pos + 1); - int val = std::atoi(port_str.c_str()); - if (val <= 0 || val > 65535) { - LOG(WARNING) << "Illegal port number in " << url - << ". Use default port " << port << " instead"; - } else { - port = static_cast(val); + // Helper: check if a string is a valid numeric IP (IPv4 or IPv6), + // automatically stripping scope IDs (e.g., %eth0) before validation + auto isNumericAddress = [](const std::string &host) -> bool { + std::string check = host; + auto pct = host.find('%'); + if (pct != std::string::npos) { + if (host.find(':', pct) != std::string::npos) { + return false; + } + check = host.substr(0, pct); } - return std::make_pair(port_str, port); + struct addrinfo hints, *res; + memset(&hints, 0, sizeof(hints)); + hints.ai_flags = AI_NUMERICHOST; + hints.ai_family = AF_UNSPEC; + hints.ai_socktype = SOCK_STREAM; + int ret = getaddrinfo(check.c_str(), nullptr, &hints, &res); + if (ret == 0) freeaddrinfo(res); + return ret == 0; + }; + + if (isNumericAddress(url)) return {url, port}; + + size_t end_pos = url.find_last_of(']'); + if (url.front() == '[' && end_pos != std::string::npos) { + std::string ip = url.substr(1, end_pos - 1); + if (!isNumericAddress(ip)) { + LOG(WARNING) << "Illegal bracketed IPv6 address: " << ip; + return std::make_pair(std::string(), port); + } + if (end_pos + 1 < url.size() && url[end_pos + 1] == ':') { + port = getPortFromString(url.substr(end_pos + 2), default_port); + } + return std::make_pair(ip, port); } - // Check if url has port field - auto pos = url.find(':'); - if (pos == url.npos) return std::make_pair(url, port); - auto trimmed_server_name = url.substr(0, pos); - auto port_str = url.substr(pos + 1); - int val = std::atoi(port_str.c_str()); - if (val <= 0 || val > 65535) - LOG(WARNING) << "Illegal port number in " << url - << ". Use default port " << port << " instead"; - else - port = (uint16_t)val; + auto pos = url.rfind(':'); + if (pos != std::string::npos) { + std::string host_part = url.substr(0, pos); + std::string port_str = url.substr(pos + 1); - return std::make_pair(trimmed_server_name, port); + if (!isNumericAddress(host_part) && + host_part.find(':') != std::string::npos) { + LOG(WARNING) << "Illegal IPv6 address: " << host_part; + return std::make_pair(std::string(), port); + } + + return std::make_pair(host_part, + getPortFromString(port_str, default_port)); + } + + return std::make_pair(url, port); } std::string buildIpAddrWithPort(const std::string &hostname, uint16_t port, @@ -138,4 +158,4 @@ std::string buildIpAddrWithPort(const std::string &hostname, uint16_t port, return hostname + ":" + std::to_string(port); } } // namespace tent -} // namespace mooncake \ No newline at end of file +} // namespace mooncake diff --git a/mooncake-transfer-engine/tent/tests/CMakeLists.txt b/mooncake-transfer-engine/tent/tests/CMakeLists.txt index be402a67..43ab57c5 100644 --- a/mooncake-transfer-engine/tent/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tent/tests/CMakeLists.txt @@ -20,6 +20,12 @@ target_include_directories(metrics_config_loader_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) add_test(NAME metrics_config_loader_test COMMAND metrics_config_loader_test) +add_executable(tent_ip_utils_test ip_utils_test.cpp) +target_link_libraries(tent_ip_utils_test PRIVATE tent_common gtest gtest_main) +target_include_directories(tent_ip_utils_test + PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../include) +add_test(NAME tent_ip_utils_test COMMAND tent_ip_utils_test) + add_executable(request_merge_test request_merge_test.cpp) target_link_libraries(request_merge_test PRIVATE gtest gtest_main tent_link_group) diff --git a/mooncake-transfer-engine/tent/tests/ip_utils_test.cpp b/mooncake-transfer-engine/tent/tests/ip_utils_test.cpp new file mode 100644 index 00000000..b6cc6b45 --- /dev/null +++ b/mooncake-transfer-engine/tent/tests/ip_utils_test.cpp @@ -0,0 +1,125 @@ +// Copyright 2025 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include + +#include "tent/common/utils/ip.h" + +namespace mooncake { +namespace tent { +namespace { + +TEST(TentIpUtilsTest, ParsesBracketedIpv6WithPort) { + auto [host, port] = parseHostNameWithPort("[2001:db8::1]:9000", 1234); + + EXPECT_EQ(host, "2001:db8::1"); + EXPECT_EQ(port, 9000); +} + +TEST(TentIpUtilsTest, ParsesBracketedIpv6WithoutPort) { + auto [host, port] = parseHostNameWithPort("[2001:db8::1]", 1234); + + EXPECT_EQ(host, "2001:db8::1"); + EXPECT_EQ(port, 1234); +} + +TEST(TentIpUtilsTest, ParsesRawIpv6WithoutPort) { + auto [host, port] = parseHostNameWithPort("2001:db8::1", 1234); + + EXPECT_EQ(host, "2001:db8::1"); + EXPECT_EQ(port, 1234); +} + +TEST(TentIpUtilsTest, ParsesIpv4WithPort) { + auto [host, port] = parseHostNameWithPort("192.168.1.10:9000", 1234); + + EXPECT_EQ(host, "192.168.1.10"); + EXPECT_EQ(port, 9000); +} + +TEST(TentIpUtilsTest, ParsesHostnameWithPort) { + auto [host, port] = parseHostNameWithPort("example.com:9000", 1234); + + EXPECT_EQ(host, "example.com"); + EXPECT_EQ(port, 9000); +} + +TEST(TentIpUtilsTest, InvalidBracketedIpv6UsesDefaultPort) { + auto [host, port] = parseHostNameWithPort("[2001::db8::1]:9000", 1234); + + EXPECT_TRUE(host.empty()); + EXPECT_EQ(port, 1234); +} + +TEST(TentIpUtilsTest, InvalidRawIpv6UsesDefaultPort) { + auto [host, port] = parseHostNameWithPort("2001::db8::1", 1234); + + EXPECT_TRUE(host.empty()); + EXPECT_EQ(port, 1234); +} + +TEST(TentIpUtilsTest, BracketedIpv6WithInvalidPortUsesDefaultPort) { + auto [host, port] = parseHostNameWithPort("[2001:db8::1]:bad", 1234); + + EXPECT_EQ(host, "2001:db8::1"); + EXPECT_EQ(port, 1234); +} + +TEST(TentIpUtilsTest, EmptyInputUsesDefaultPort) { + auto [host, port] = parseHostNameWithPort("", 1234); + + EXPECT_TRUE(host.empty()); + EXPECT_EQ(port, 1234); +} + +TEST(TentIpUtilsTest, BuildsBracketedIpv6Endpoint) { + EXPECT_EQ(buildIpAddrWithPort("2001:db8::1", 9000, true), + "[2001:db8::1]:9000"); +} + +TEST(TentIpUtilsTest, ParsesScopeIdWithoutPort) { + auto [host, port] = + parseHostNameWithPort("fe80::a236:bcff:fecb:a1be%eno2", 1234); + + EXPECT_EQ(host, "fe80::a236:bcff:fecb:a1be%eno2"); + EXPECT_EQ(port, 1234); +} + +TEST(TentIpUtilsTest, ParsesScopeIdWithPort) { + auto [host, port] = + parseHostNameWithPort("fe80::a236:bcff:fecb:a1be%eno2:15773", 1234); + + EXPECT_EQ(host, "fe80::a236:bcff:fecb:a1be%eno2"); + EXPECT_EQ(port, 15773); +} + +TEST(TentIpUtilsTest, ParsesBracketedScopeIdWithPort) { + auto [host, port] = + parseHostNameWithPort("[fe80::a236:bcff:fecb:a1be%eno2]:15773", 1234); + + EXPECT_EQ(host, "fe80::a236:bcff:fecb:a1be%eno2"); + EXPECT_EQ(port, 15773); +} + +TEST(TentIpUtilsTest, ScopeIdWithInvalidPortUsesDefault) { + auto [host, port] = + parseHostNameWithPort("fe80::a236:bcff:fecb:a1be%eno2:bad", 1234); + + EXPECT_EQ(host, "fe80::a236:bcff:fecb:a1be%eno2"); + EXPECT_EQ(port, 1234); +} + +} // namespace +} // namespace tent +} // namespace mooncake diff --git a/mooncake-transfer-engine/tests/common_test.cpp b/mooncake-transfer-engine/tests/common_test.cpp index 887ed426..b07971be 100644 --- a/mooncake-transfer-engine/tests/common_test.cpp +++ b/mooncake-transfer-engine/tests/common_test.cpp @@ -112,6 +112,27 @@ TEST(ParseHostNameWithPort, HostWithoutPort) { EXPECT_EQ(port, kDefaultPort); } +TEST(HostPortHelpers, DetectsExplicitPort) { + EXPECT_TRUE(hasExplicitPort("8.8.8.8:4321")); + EXPECT_TRUE(hasExplicitPort("example.com:4321")); + EXPECT_TRUE(hasExplicitPort("[2001:db8::1]:4321")); + EXPECT_FALSE(hasExplicitPort("2001:db8::1")); + EXPECT_FALSE(hasExplicitPort("[2001:db8::1]")); +} + +TEST(HostPortHelpers, ExtractsHostWithoutPort) { + EXPECT_EQ(getHostNameWithoutPort("8.8.8.8:4321"), "8.8.8.8"); + EXPECT_EQ(getHostNameWithoutPort("example.com:4321"), "example.com"); + EXPECT_EQ(getHostNameWithoutPort("[2001:db8::1]:4321"), "2001:db8::1"); + EXPECT_EQ(getHostNameWithoutPort("[2001:db8::1]"), "2001:db8::1"); + EXPECT_EQ(getHostNameWithoutPort("2001:db8::1"), "2001:db8::1"); +} + +TEST(HostPortHelpers, BuildsBracketedIpv6Endpoint) { + EXPECT_EQ(buildHostNameWithPort("2001:db8::1", 4321), "[2001:db8::1]:4321"); + EXPECT_EQ(buildHostNameWithPort("8.8.8.8", 4321), "8.8.8.8:4321"); +} + //------------------------------------------------------------------------------ // parsePortAndDevice //------------------------------------------------------------------------------ -- 2.34.1 From bc17c9b60a31b92717fcaad8af2f91549957ac5e Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Mon, 1 Jun 2026 16:49:58 +0800 Subject: [PATCH 157/382] Fix P2PHANDSHAKE in dual-NIC container setups via MC_RDMA_BIND_ADDRESS (#2280) * Initial plan * Add MC_RDMA_BIND_ADDRESS support for dual-NIC P2PHANDSHAKE setups In dual-NIC environments where TCP and RDMA use separate interfaces, P2PHANDSHAKE mode previously required using a single IP for both TCP handshake and RDMA NIC paths, causing conflicts. This change adds MC_RDMA_BIND_ADDRESS env var support: - When set, RDMA NIC paths use the RDMA-reachable IP - TCP P2P routing continues using the local_server_name IP - Segment descriptors carry rdma_server_name for consistent NIC path construction on both sides - P2P metadata exchange caches RDMA->TCP address mapping so subsequent handshakes resolve to TCP-routable addresses * Changes before error encountered Agent-Logs-Url: https://github.com/kvcache-ai/Mooncake/sessions/fc2826eb-a0ae-450f-b1f4-4ab94269d97a * Apply dual-NIC (MC_RDMA_BIND_ADDRESS) support to TENT transport and update Chinese docs --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- .../python-api-reference/transfer-engine.md | 1 + .../zh_archive/transfer-engine-python.md | 1 + .../include/transfer_metadata.h | 13 +++++++ .../transport/rdma_transport/rdma_transport.h | 5 +++ .../src/transfer_metadata.cpp | 36 ++++++++++++++++++- .../src/transfer_metadata_dump.cpp | 3 ++ .../transport/efa_transport/efa_context.cpp | 5 +-- .../kunpeng_transport/ub_context.cpp | 4 +-- .../transport/rdma_transport/rdma_context.cpp | 2 +- .../rdma_transport/rdma_transport.cpp | 21 +++++++++++ .../transport/rdma_transport/worker_pool.cpp | 4 +-- .../tent/include/tent/runtime/segment.h | 19 ++++++++++ .../tent/transport/rdma/rdma_transport.h | 5 +++ .../tent/src/transport/rdma/endpoint.cpp | 9 ++--- .../src/transport/rdma/rdma_transport.cpp | 25 ++++++++++++- .../tent/src/transport/rdma/workers.cpp | 6 ++-- 16 files changed, 144 insertions(+), 15 deletions(-) diff --git a/docs/source/python-api-reference/transfer-engine.md b/docs/source/python-api-reference/transfer-engine.md index 71136f14..afa90d9a 100644 --- a/docs/source/python-api-reference/transfer-engine.md +++ b/docs/source/python-api-reference/transfer-engine.md @@ -615,6 +615,7 @@ The Transfer Engine respects the following environment variables: - `MC_METADATA_SERVER`: Default metadata server address - `MC_LEGACY_RPC_PORT_BINDING`: Enables legacy RPC port binding behavior - `MC_TCP_BIND_ADDRESS`: Specifies the TCP bind address +- `MC_RDMA_BIND_ADDRESS`: Specifies the RDMA bind address for NIC path construction in dual-NIC environments. When set, RDMA NIC paths use this address while TCP handshake uses the address from `local_hostname`. This is useful when TCP and RDMA traffic use separate network interfaces (e.g., `eth0` for TCP and `rdma-net1` for RDMA). - `MC_CUSTOM_TOPO_JSON`: Path to custom topology JSON file - `MC_TE_METRIC`: Enables metrics reporting (set to "1", "true", "yes", or "on"). **Note:** Not supported when using Transfer Engine TENT. - `MC_TE_METRIC_INTERVAL_SECONDS`: Sets metrics reporting interval in seconds diff --git a/docs/source/zh_archive/transfer-engine-python.md b/docs/source/zh_archive/transfer-engine-python.md index 18a98189..7398a052 100644 --- a/docs/source/zh_archive/transfer-engine-python.md +++ b/docs/source/zh_archive/transfer-engine-python.md @@ -287,6 +287,7 @@ TransferOpcode.WRITE # 写操作 - `MC_METADATA_SERVER`: 默认元数据服务器地址 - `MC_LEGACY_RPC_PORT_BINDING`: 启用传统RPC端口绑定行为 - `MC_TCP_BIND_ADDRESS`: 指定TCP绑定地址 +- `MC_RDMA_BIND_ADDRESS`: 指定RDMA绑定地址,用于双网卡环境下的NIC路径构建。设置后,RDMA NIC路径使用此地址,而TCP握手使用 `local_hostname` 中的地址。适用于TCP和RDMA流量使用不同网络接口的场景(例如 `eth0` 用于TCP,`rdma-net1` 用于RDMA)。 - `MC_CUSTOM_TOPO_JSON`: 自定义拓扑JSON文件路径 - `MC_TE_METRIC`: 启用指标报告(设置为"1"、"true"、"yes"或"on") - `MC_TE_METRIC_INTERVAL_SECONDS`: 设置指标报告间隔(秒) diff --git a/mooncake-transfer-engine/include/transfer_metadata.h b/mooncake-transfer-engine/include/transfer_metadata.h index 863959fb..0233a35c 100644 --- a/mooncake-transfer-engine/include/transfer_metadata.h +++ b/mooncake-transfer-engine/include/transfer_metadata.h @@ -104,6 +104,19 @@ class TransferMetadata { int tcp_data_port; + // In dual-NIC setups (MC_RDMA_BIND_ADDRESS), the RDMA-reachable + // address may differ from the TCP-routable segment name. When + // non-empty, NIC paths are constructed using this value instead + // of `name`. + std::string rdma_server_name; + + // Returns the server name to use for NIC path construction. + // Uses rdma_server_name when available, otherwise falls back + // to name. + const std::string &nicPathServerName() const { + return rdma_server_name.empty() ? name : rdma_server_name; + } + void dump() const; }; diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h index d292a27e..11997831 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h @@ -129,6 +129,11 @@ class RdmaTransport : public Transport { private: std::vector> context_list_; std::shared_ptr local_topology_; + // When MC_RDMA_BIND_ADDRESS is set in a dual-NIC environment, + // rdma_server_name_ holds the RDMA-reachable address (e.g. + // "192.168.0.y:port") for NIC path construction, while + // local_server_name_ keeps the TCP-reachable address for P2P routing. + std::string rdma_server_name_; }; using TransferRequest = Transport::TransferRequest; diff --git a/mooncake-transfer-engine/src/transfer_metadata.cpp b/mooncake-transfer-engine/src/transfer_metadata.cpp index 10311838..f2ed39c3 100644 --- a/mooncake-transfer-engine/src/transfer_metadata.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata.cpp @@ -219,6 +219,9 @@ static int encodeMultiProtocolSegmentDesc( const TransferMetadata::SegmentDesc &desc, Json::Value &segmentJSON) { // Multi-protocol encoding for CXL+TCP or CXL+RDMA combination segmentJSON["name"] = desc.name; + if (!desc.rdma_server_name.empty()) { + segmentJSON["rdma_server_name"] = desc.rdma_server_name; + } Json::Value protocolJSON(Json::arrayValue); for (const auto &proto : protocols) { if (proto == "rdma") { @@ -315,6 +318,9 @@ int TransferMetadata::encodeSegmentDesc(const SegmentDesc &desc, segmentJSON["protocol"] = desc.protocol; segmentJSON["tcp_data_port"] = desc.tcp_data_port; segmentJSON["timestamp"] = getCurrentDateTime(); + if (!desc.rdma_server_name.empty()) { + segmentJSON["rdma_server_name"] = desc.rdma_server_name; + } if (segmentJSON["protocol"] == "rdma" || segmentJSON["protocol"] == "barex" || @@ -508,6 +514,8 @@ decodeMultiProtocolSegmentDesc(Json::Value &segmentJSON, desc->tcp_data_port = segmentJSON["tcp_data_port"].asInt(); if (segmentJSON.isMember("timestamp")) desc->timestamp = segmentJSON["timestamp"].asString(); + if (segmentJSON.isMember("rdma_server_name")) + desc->rdma_server_name = segmentJSON["rdma_server_name"].asString(); for (const auto &protocolStr : segmentJSON["protocol"]) { std::string proto = protocolStr.asString(); @@ -651,6 +659,8 @@ TransferMetadata::decodeSegmentDesc(Json::Value &segmentJSON, desc->tcp_data_port = segmentJSON["tcp_data_port"].asInt(); if (segmentJSON.isMember("timestamp")) desc->timestamp = segmentJSON["timestamp"].asString(); + if (segmentJSON.isMember("rdma_server_name")) + desc->rdma_server_name = segmentJSON["rdma_server_name"].asString(); if (desc->protocol == "rdma" || desc->protocol == "barex" || desc->protocol == "efa") { @@ -897,7 +907,31 @@ std::shared_ptr TransferMetadata::getSegmentDesc( } } - return decodeSegmentDesc(peer_json, segment_name); + auto result = decodeSegmentDesc(peer_json, segment_name); + + // In P2P mode with dual-NIC setups (MC_RDMA_BIND_ADDRESS), the peer's + // segment descriptor may contain an rdma_server_name that differs from + // the TCP-routable segment_name. Cache the mapping so subsequent + // sendHandshake() calls can resolve the peer's TCP address from the + // RDMA server name extracted from NIC paths. + if (p2p_handshake_mode_ && result && + !result->rdma_server_name.empty() && + result->rdma_server_name != segment_name) { + auto [tcp_ip, tcp_port] = parseHostNameWithPort(segment_name); + RWSpinlock::WriteGuard guard(rpc_meta_lock_); + if (!rpc_meta_map_.count(result->rdma_server_name)) { + RpcMetaDesc meta; + meta.ip_or_host_name = tcp_ip; + meta.rpc_port = tcp_port; + meta.sockfd = -1; + rpc_meta_map_[result->rdma_server_name] = meta; + LOG(INFO) << "P2P: cached RDMA->TCP mapping: " + << result->rdma_server_name << " -> " << tcp_ip + << ":" << tcp_port; + } + } + + return result; } int TransferMetadata::syncSegmentCache(const std::string &segment_name) { diff --git a/mooncake-transfer-engine/src/transfer_metadata_dump.cpp b/mooncake-transfer-engine/src/transfer_metadata_dump.cpp index 4035cdfb..dc95d697 100644 --- a/mooncake-transfer-engine/src/transfer_metadata_dump.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata_dump.cpp @@ -18,6 +18,9 @@ namespace mooncake { void TransferMetadata::SegmentDesc::dump() const { LOG(INFO) << " segment name: " << name; + if (!rdma_server_name.empty() && rdma_server_name != name) { + LOG(INFO) << " rdma server name: " << rdma_server_name; + } LOG(INFO) << " protocol: " << protocol; LOG(INFO) << " topology: " << topology.toString(); LOG(INFO) << " devices: "; diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp index b20787b0..9f0b6fab 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp @@ -665,8 +665,9 @@ int EfaContext::submitPostSend( slice->rdma.dest_rkey = peer_segment_desc->buffers[buffer_id].rkey[device_id]; - std::string peer_nic_path = peer_segment_desc->name + "@" + - peer_segment_desc->devices[device_id].name; + std::string peer_nic_path = + peer_segment_desc->nicPathServerName() + "@" + + peer_segment_desc->devices[device_id].name; slice->peer_nic_path = peer_nic_path; slices_by_peer[peer_nic_path].push_back(slice); } diff --git a/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_context.cpp b/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_context.cpp index ac10faa5..d5242ea3 100644 --- a/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_context.cpp +++ b/mooncake-transfer-engine/src/transport/kunpeng_transport/ub_context.cpp @@ -276,7 +276,7 @@ int UbWorkerPool::submitPostSend( << " dest_addr=" << slice->ub.dest_addr; } auto peer_nic_path = - MakeNicPath(peer_segment_desc->name, + MakeNicPath(peer_segment_desc->nicPathServerName(), peer_segment_desc->devices[device_id].name); slice->peer_nic_path = peer_nic_path; int shard_id = (slice->target_id * 10007 + device_id) % kShardCount; @@ -486,7 +486,7 @@ void UbWorkerPool::redispatch(std::vector& slice_list, peer_segment_desc->buffers[buffer_id].tseg[device_id]; slice->ub.r_seg = context_.retrieveRemoteSeg(targetSegment); auto peer_nic_path = - MakeNicPath(peer_segment_desc->name, + MakeNicPath(peer_segment_desc->nicPathServerName(), peer_segment_desc->devices[device_id].name); slice->peer_nic_path = peer_nic_path; collective_slice_queue_[thread_id][peer_nic_path].push_back(slice); diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index 79cbb780..08092ad6 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -658,7 +658,7 @@ size_t RdmaContext::getTotalQPNumber() const { } std::string RdmaContext::nicPath() const { - return MakeNicPath(engine_.local_server_name_, device_name_); + return MakeNicPath(engine_.rdma_server_name_, device_name_); } std::string RdmaContext::gid() const { diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp index 2c20664c..49f622d8 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -102,6 +103,21 @@ int RdmaTransport::install(std::string &local_server_name, local_server_name_ = local_server_name; local_topology_ = topo; + // In dual-NIC environments (e.g. separate TCP and RDMA interfaces), + // MC_RDMA_BIND_ADDRESS allows NIC paths to use an RDMA-reachable IP + // while local_server_name_ keeps the TCP-reachable address for P2P. + const char *rdma_bind_addr = std::getenv("MC_RDMA_BIND_ADDRESS"); + if (rdma_bind_addr && rdma_bind_addr[0] != '\0') { + auto [host_name, port] = parseHostNameWithPort(local_server_name); + rdma_server_name_ = + std::string(rdma_bind_addr) + ":" + std::to_string(port); + LOG(INFO) << "RdmaTransport: using RDMA bind address " + << rdma_server_name_ + << " (TCP address: " << local_server_name_ << ")"; + } else { + rdma_server_name_ = local_server_name_; + } + auto ret = initializeRdmaResources(); if (ret) { LOG(ERROR) << "RdmaTransport: cannot initialize RDMA resources"; @@ -362,6 +378,11 @@ int RdmaTransport::allocateLocalSegmentID() { auto desc = metadata_->getSegmentDesc(local_server_name_); if (!desc) desc = std::make_shared(); desc->name = local_server_name_; + // Store RDMA server name for dual-NIC setups; when it differs from + // local_server_name_ the peer will use it for NIC path construction. + if (rdma_server_name_ != local_server_name_) { + desc->rdma_server_name = rdma_server_name_; + } #ifdef ENABLE_MULTI_PROTOCOL if (!desc->protocol.empty()) desc->protocol += ","; desc->protocol += "rdma"; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp index 788edf6b..ee028307 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/worker_pool.cpp @@ -143,7 +143,7 @@ int WorkerPool::submitPostSend( slice->rdma.dest_rkey = peer_segment_desc->buffers[buffer_id].rkey[device_id]; auto peer_nic_path = - MakeNicPath(peer_segment_desc->name, + MakeNicPath(peer_segment_desc->nicPathServerName(), peer_segment_desc->devices[device_id].name); slice->peer_nic_path = peer_nic_path; int shard_id = (slice->target_id * 10007 + device_id) % kShardCount; @@ -380,7 +380,7 @@ void WorkerPool::redispatch(std::vector &slice_list, slice->rdma.dest_rkey = peer_segment_desc->buffers[buffer_id].rkey[device_id]; auto peer_nic_path = - MakeNicPath(peer_segment_desc->name, + MakeNicPath(peer_segment_desc->nicPathServerName(), peer_segment_desc->devices[device_id].name); slice->peer_nic_path = peer_nic_path; collective_slice_queue_[thread_id][peer_nic_path].push_back(slice); diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/segment.h b/mooncake-transfer-engine/tent/include/tent/runtime/segment.h index 21ee86e4..d9c60507 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/segment.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/segment.h @@ -169,12 +169,25 @@ struct SegmentDesc { std::string rpc_server_addr; std::variant detail; + // In dual-NIC setups (MC_RDMA_BIND_ADDRESS), the RDMA-reachable + // address may differ from the TCP-routable segment name. When + // non-empty, NIC paths are constructed using this value instead + // of `name`. + std::string rdma_server_name; + public: BufferDesc* findBuffer(uint64_t base, uint64_t length); DeviceDesc* findDevice(const std::string& name); const MemorySegmentDesc& getMemory() const { return std::get(detail); } + + // Returns the server name to use for NIC path construction. + // Uses rdma_server_name when available, otherwise falls back + // to name. + const std::string& nicPathServerName() const { + return rdma_server_name.empty() ? name : rdma_server_name; + } }; inline void to_json(json& j, const SegmentDesc& s) { @@ -182,6 +195,9 @@ inline void to_json(json& j, const SegmentDesc& s) { {"type", s.type}, {"machine_id", s.machine_id}, {"rpc_server_addr", s.rpc_server_addr}}; + if (!s.rdma_server_name.empty()) { + j["rdma_server_name"] = s.rdma_server_name; + } if (s.type == SegmentType::Memory) { j["detail"] = std::get(s.detail); } else { @@ -194,6 +210,9 @@ inline void from_json(const json& j, SegmentDesc& s) { j.at("type").get_to(s.type); j.at("machine_id").get_to(s.machine_id); j.at("rpc_server_addr").get_to(s.rpc_server_addr); + if (j.contains("rdma_server_name")) { + j.at("rdma_server_name").get_to(s.rdma_server_name); + } if (s.type == SegmentType::Memory) { s.detail = j.at("detail").get(); } else { diff --git a/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h b/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h index 84ed23df..2a739579 100644 --- a/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h +++ b/mooncake-transfer-engine/tent/include/tent/transport/rdma/rdma_transport.h @@ -120,6 +120,11 @@ class RdmaTransport : public Transport { bool installed_; std::shared_ptr conf_; std::string local_segment_name_; + // When MC_RDMA_BIND_ADDRESS is set in a dual-NIC environment, + // rdma_server_name_ holds the RDMA-reachable address for NIC path + // construction, while local_segment_name_ keeps the TCP-reachable + // address for P2P routing. + std::string rdma_server_name_; std::shared_ptr local_topology_; std::shared_ptr metadata_; LocalBufferManager local_buffer_manager_; diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp index 690ed41d..7a02d382 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/endpoint.cpp @@ -340,7 +340,7 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, } local_desc.local_nic_path = - MakeNicPath(transport.local_segment_name_, context_->name()); + MakeNicPath(transport.rdma_server_name_, context_->name()); local_desc.peer_nic_path = MakeNicPath(peer_server_name, peer_nic_name); qp_num = qpNum(); local_desc.qp_num = qp_num; @@ -350,7 +350,8 @@ Status RdmaEndPoint::connect(const std::string& peer_server_name, same_nic = (local_desc.local_nic_path == local_desc.peer_nic_path); is_self = - (!same_nic && peer_server_name == transport.local_segment_name_); + (!same_nic && (peer_server_name == transport.local_segment_name_ || + peer_server_name == transport.rdma_server_name_)); } // ===== Phase 2: Bootstrap (unlocked) ===== @@ -463,7 +464,7 @@ Status RdmaEndPoint::accept(const BootstrapDesc& peer_desc, << " (duplicate bootstrap, reusing connection)"; auto& transport = context_->transport_; local_desc.local_nic_path = - MakeNicPath(transport.local_segment_name_, context_->name()); + MakeNicPath(transport.rdma_server_name_, context_->name()); local_desc.peer_nic_path = peer_desc.local_nic_path; local_desc.qp_num = qpNum(); local_desc.local_lid = context_->lid(); @@ -495,7 +496,7 @@ Status RdmaEndPoint::accept(const BootstrapDesc& peer_desc, return mooncake::tent::Status::InvalidArgument( "Invalid peer path" LOC_MARK); local_desc.local_nic_path = - MakeNicPath(transport.local_segment_name_, context_->name()); + MakeNicPath(transport.rdma_server_name_, context_->name()); local_desc.peer_nic_path = peer_nic_path; local_desc.qp_num = qpNum(); local_desc.local_lid = context_->lid(); diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp index f12e6427..0bf8733a 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp @@ -22,11 +22,13 @@ #include #include +#include #include #include #include #include "tent/common/status.h" +#include "tent/common/utils/ip.h" #include "tent/transport/rdma/buffers.h" #include "tent/transport/rdma/endpoint_store.h" #include "tent/transport/rdma/workers.h" @@ -213,6 +215,22 @@ Status RdmaTransport::install(std::string& local_segment_name, metadata_ = metadata; local_segment_name_ = local_segment_name; local_topology_ = local_topology; + + // In dual-NIC environments (e.g. separate TCP and RDMA interfaces), + // MC_RDMA_BIND_ADDRESS allows NIC paths to use an RDMA-reachable IP + // while local_segment_name_ keeps the TCP-reachable address for P2P. + const char *rdma_bind_addr = std::getenv("MC_RDMA_BIND_ADDRESS"); + if (rdma_bind_addr && rdma_bind_addr[0] != '\0') { + auto [host_name, port] = parseHostNameWithPort(local_segment_name); + rdma_server_name_ = + std::string(rdma_bind_addr) + ":" + std::to_string(port); + LOG(INFO) << "RdmaTransport(TENT): using RDMA bind address " + << rdma_server_name_ + << " (TCP address: " << local_segment_name_ << ")"; + } else { + rdma_server_name_ = local_segment_name_; + } + local_buffer_manager_.setTopology(local_topology); context_set_.clear(); for (size_t i = 0; i < local_topology_->getNicCount(); ++i) { @@ -488,6 +506,11 @@ Status RdmaTransport::setupLocalSegment() { auto& manager = metadata_->segmentManager(); auto segment = manager.getLocal(); assert(segment); + // Store RDMA server name for dual-NIC setups; when it differs from + // local_segment_name_ the peer will use it for NIC path construction. + if (rdma_server_name_ != local_segment_name_) { + segment->rdma_server_name = rdma_server_name_; + } auto& detail = std::get(segment->detail); for (auto& context : context_set_) { if (context->status() != RdmaContext::DEVICE_ENABLED) continue; @@ -576,7 +599,7 @@ std::shared_ptr RdmaTransport::getEndpoint(SegmentID target_id, return nullptr; } std::shared_ptr endpoint; - std::string peer_name = MakeNicPath(segment_desc->name, target_dev_name); + std::string peer_name = MakeNicPath(segment_desc->nicPathServerName(), target_dev_name); endpoint = context->endpointStore()->getOrInsert(peer_name); if (!endpoint) { LOG(ERROR) << "Cannot allocate endpoint " << peer_name; diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp index 0d7996f5..9b9d2bc3 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/workers.cpp @@ -230,7 +230,8 @@ Status Workers::cancel(RdmaSliceList& slice_list) { } std::shared_ptr Workers::getEndpoint(Workers::PostPath path) { - std::string rpc_server_addr, target_seg_name, target_dev_name; + std::string rpc_server_addr, target_seg_name, target_dev_name, + target_nic_path_name; RouteHint hint; auto& segment_manager = transport_->metadata_->segmentManager(); auto target_id = path.remote_segment_id; @@ -248,6 +249,7 @@ std::shared_ptr Workers::getEndpoint(Workers::PostPath path) { rpc_server_addr = segment->rpc_server_addr; } target_seg_name = segment->name; + target_nic_path_name = segment->nicPathServerName(); target_dev_name = hint.topo->getNicName(device_id); if (target_seg_name.empty() || target_dev_name.empty()) { return Status::NeedsRefreshCache( @@ -268,7 +270,7 @@ std::shared_ptr Workers::getEndpoint(Workers::PostPath path) { // connection unavailable } std::shared_ptr endpoint; - auto peer_name = MakeNicPath(target_seg_name, target_dev_name); + auto peer_name = MakeNicPath(target_nic_path_name, target_dev_name); endpoint = context->endpointStore()->getOrInsert(peer_name); if (!endpoint) { LOG(ERROR) << "Cannot allocate endpoint " << peer_name; -- 2.34.1 From b6386a103d9f70f746f9fde2979c489dd59cda06 Mon Sep 17 00:00:00 2001 From: Lewis <63569348+TTThanos@users.noreply.github.com> Date: Mon, 1 Jun 2026 16:56:19 +0800 Subject: [PATCH 158/382] [TE] IntraNode NVLink transport: update cuMemcpyAsync to BatchAsyc for CUDA version >= 12.8 (#2251) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Update intraNode nvlink transport from MemcpuAsync to BatchAsync * Code format update for intraNode nvlink * Change addr to base_addr for register and unregister --------- Co-authored-by: 百麒 --- .../intranode_nvlink_transport.cpp | 214 ++++++++++++++---- 1 file changed, 167 insertions(+), 47 deletions(-) diff --git a/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp b/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp index 6464bc20..b4fbd808 100644 --- a/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp +++ b/mooncake-transfer-engine/src/transport/intranode_nvlink_transport/intranode_nvlink_transport.cpp @@ -24,6 +24,7 @@ #include #include #include +#include #include "common.h" #include "common/serialization.h" @@ -44,24 +45,26 @@ struct CudaStreamNVLinkRAII { } ~CudaStreamNVLinkRAII() { cudaStreamDestroy(stream_); } }; + static thread_local CudaStreamNVLinkRAII tl_nvlink_stream; -// Thread-local CUDA event used to synchronize the NVLink stream with -// the default (legacy) CUDA stream before issuing cudaMemcpyAsync. -// This prevents the NVLink stream from reading source data that -// PyTorch has not yet finished writing on the default stream. -struct CudaSyncEventRAII { +/// Thread-local CUDA event for GPU-level stream synchronization. +/// Used to establish a GPU-visible dependency between cudaStreamPerThread +/// and nvlink_stream, which is required by cudaMemcpyBatchAsync's +/// srcAccessOrderStream attribute for cross-stream P2P copies. +struct CudaEventNVLinkRAII { cudaEvent_t event_; - CudaSyncEventRAII() { + CudaEventNVLinkRAII() { auto err = cudaEventCreateWithFlags(&event_, cudaEventDisableTiming); if (err != cudaSuccess) { - LOG(FATAL) << "Failed to create NVLink sync CUDA event: " << err + LOG(FATAL) << "Failed to create NVLink CUDA sync event: " << err << " - " << cudaGetErrorString(err); } } - ~CudaSyncEventRAII() { cudaEventDestroy(event_); } + ~CudaEventNVLinkRAII() { cudaEventDestroy(event_); } }; -static thread_local CudaSyncEventRAII tl_nvlink_sync_event; + +static thread_local CudaEventNVLinkRAII tl_nvlink_sync_event; } // anonymous namespace static bool checkCudaErrorReturn(cudaError_t result, const char *message) { @@ -74,6 +77,112 @@ static bool checkCudaErrorReturn(cudaError_t result, const char *message) { } namespace mooncake { + +typedef Transport::Slice Slice; + +/// Submit batched memcpy operations using cudaMemcpyBatchAsync when available +/// (CUDA 12.8+), falling back to per-slice cudaMemcpyAsync otherwise. +/// Uses cudaMemcpySrcAccessOrderStream attribute to respect stream access +/// ordering semantics. Individual slice errors are tracked so that slices +/// whose memcpy failed are marked as FAILED while successful ones are POSTED. +static void submitBatchMemcpy(const std::vector &slices, + const std::vector &srcs, + const std::vector &dsts, + const std::vector &sizes, + cudaStream_t stream) { + if (slices.empty()) return; + + const size_t count = slices.size(); + cudaError_t err = cudaSuccess; + + // Log the active memcpy path once per process lifetime + static const bool logged_once = [] { +#if CUDART_VERSION >= 13000 + LOG(INFO) << "IntraNodeNvlinkTransport: using cudaMemcpyBatchAsync " + << "(CUDA >= 13.0 path)"; +#elif CUDART_VERSION >= 12080 + LOG(INFO) << "IntraNodeNvlinkTransport: using cudaMemcpyBatchAsync " + << "(CUDA >= 12.8 path)"; +#else + LOG(INFO) + << "IntraNodeNvlinkTransport: using per-slice cudaMemcpyAsync " + << "(CUDA < 12.8 fallback path)"; +#endif + return true; + }(); + (void)logged_once; + +#if CUDART_VERSION >= 12080 + // Use srcAccessOrderStream for P2P copies. The GPU-level dependency + // established by cudaEventRecord + cudaStreamWaitEvent in the caller + // ensures the source data is visible through nvlink_stream's access + // order, satisfying srcAccessOrderStream's requirement. + cudaMemcpyAttributes attr{}; + attr.srcAccessOrder = cudaMemcpySrcAccessOrderStream; + size_t attrs_idx = 0; + // cudaMemcpyBatchAsync in CUDA 12.8 takes non-const size_t* for sizes + std::vector mutable_sizes(sizes); +#endif + +#if CUDART_VERSION >= 13000 + err = cudaMemcpyBatchAsync(const_cast(dsts.data()), + const_cast(srcs.data()), + mutable_sizes.data(), static_cast(count), + &attr, &attrs_idx, 1, stream); +#elif CUDART_VERSION >= 12080 + { + size_t fail_idx = count; + err = cudaMemcpyBatchAsync( + const_cast(dsts.data()), const_cast(srcs.data()), + mutable_sizes.data(), static_cast(count), &attr, &attrs_idx, + 1, &fail_idx, stream); + if (err != cudaSuccess) { + if (fail_idx < count) { + LOG(ERROR) << "IntraNodeNvlinkTransport: cudaMemcpyBatchAsync " + << "failed at index " << fail_idx + << " (src=" << srcs[fail_idx] + << ", dst=" << dsts[fail_idx] + << ", size=" << sizes[fail_idx] + << "): " << cudaGetErrorString(err); + } else { + LOG(ERROR) << "IntraNodeNvlinkTransport: cudaMemcpyBatchAsync " + << "failed: " << cudaGetErrorString(err); + } + } + } +#else + // Fallback for CUDA < 12.8: submit each memcpy individually + for (size_t i = 0; i < count; ++i) { + auto single_err = cudaMemcpyAsync(dsts[i], srcs[i], sizes[i], + cudaMemcpyDefault, stream); + if (single_err != cudaSuccess) { + LOG(ERROR) << "IntraNodeNvlinkTransport: cudaMemcpyAsync failed at " + << "index " << i << ": " + << cudaGetErrorString(single_err); + slices[i]->markFailed(); + continue; + } + slices[i]->status = Slice::POSTED; + slices[i]->local.cuda_stream = (void *)stream; + } + return; // Slice states already set above +#endif + + // For cudaMemcpyBatchAsync paths, update slice states based on result + if (err != cudaSuccess) { + for (size_t i = 0; i < count; ++i) { + if (slices[i]->status == Slice::PENDING) { + slices[i]->markFailed(); + } + } + } else { + for (size_t i = 0; i < count; ++i) { + slices[i]->status = Slice::POSTED; + slices[i]->local.cuda_stream = (void *)stream; + } + } +} + static int getNumDevices() { static int cached_num_devices = -1; if (cached_num_devices == -1) { @@ -210,11 +319,16 @@ Status IntraNodeNvlinkTransport::submitTransfer( // Synchronize with the caller's CUDA stream before issuing any memcpy. // PyTorch uses cudaStreamPerThread (per-thread default stream), NOT the - // legacy default stream (nullptr). Recording the event on - // cudaStreamPerThread ensures that all previously submitted PyTorch - // operations on source/dest buffers have completed before cudaMemcpyAsync - // starts on tl_nvlink_stream. Using nullptr (legacy default stream) would - // miss PyTorch's work and could cause deadlocks with blocking streams. + // legacy default stream (nullptr). Recording an event on + // cudaStreamPerThread and making nvlink_stream wait for it ensures that all + // PyTorch GPU operations on source/dest buffers complete before the NVLink + // memcpy starts. This GPU-level dependency is also required by + // cudaMemcpyBatchAsync's srcAccessOrderStream attribute, which needs + // the source data to be visible through the nvlink_stream's access order. + // + // Do NOT use the legacy default stream (nullptr) for cudaEventRecord, + // as it would trigger implicit synchronization with blocking streams and + // could cause deadlocks. cudaStream_t stream = tl_nvlink_stream.stream_; cudaError_t sync_err = cudaEventRecord(tl_nvlink_sync_event.event_, cudaStreamPerThread); @@ -233,6 +347,11 @@ Status IntraNodeNvlinkTransport::submitTransfer( std::string(cudaGetErrorString(sync_err))); } + // Phase 1: Prepare slices and collect memcpy parameters + std::vector dsts, srcs; + std::vector sizes; + std::vector slices; + for (auto &request : entries) { TransferTask &task = batch_desc.task_list[task_id]; ++task_id; @@ -254,24 +373,22 @@ Status IntraNodeNvlinkTransport::submitTransfer( slice->ts = getCurrentTimeInNano(); task.slice_list.push_back(slice); __sync_fetch_and_add(&task.slice_count, 1); - cudaStream_t stream = tl_nvlink_stream.stream_; - cudaError_t err; - if (slice->opcode == TransferRequest::READ) - err = cudaMemcpyAsync(slice->source_addr, - (void *)slice->local.dest_addr, slice->length, - cudaMemcpyDefault, stream); - else - err = cudaMemcpyAsync((void *)slice->local.dest_addr, - slice->source_addr, slice->length, - cudaMemcpyDefault, stream); - if (err != cudaSuccess) { - slice->markFailed(); - } else { - slice->status = Slice::POSTED; - slice->local.cuda_stream = (void *)stream; - } + + void *src = (request.opcode == TransferRequest::READ) + ? (void *)slice->local.dest_addr + : (void *)slice->source_addr; + void *dst = (request.opcode == TransferRequest::READ) + ? slice->source_addr + : (void *)slice->local.dest_addr; + srcs.push_back(src); + dsts.push_back(dst); + sizes.push_back(slice->length); + slices.push_back(slice); } + // Phase 2: Submit all memcpy operations + submitBatchMemcpy(slices, srcs, dsts, sizes, stream); + return Status::OK(); } @@ -319,7 +436,7 @@ Status IntraNodeNvlinkTransport::getTransferStatus(BatchID batch_id, Status IntraNodeNvlinkTransport::submitTransferTask( const std::vector &task_list) { // Synchronize with the caller's CUDA stream before issuing any memcpy. - // See submitTransfer() for detailed rationale on using cudaStreamPerThread. + // See submitTransfer() for detailed rationale. cudaStream_t stream = tl_nvlink_stream.stream_; cudaError_t sync_err = cudaEventRecord(tl_nvlink_sync_event.event_, cudaStreamPerThread); @@ -338,6 +455,11 @@ Status IntraNodeNvlinkTransport::submitTransferTask( std::string(cudaGetErrorString(sync_err))); } + // Phase 1: Prepare slices and collect memcpy parameters + std::vector dsts, srcs; + std::vector sizes; + std::vector slices; + for (size_t index = 0; index < task_list.size(); ++index) { assert(task_list[index]); auto &task = *task_list[index]; @@ -361,24 +483,22 @@ Status IntraNodeNvlinkTransport::submitTransferTask( slice->ts = getCurrentTimeInNano(); task.slice_list.push_back(slice); __sync_fetch_and_add(&task.slice_count, 1); - cudaStream_t stream = tl_nvlink_stream.stream_; - cudaError_t err; - if (slice->opcode == TransferRequest::READ) - err = cudaMemcpyAsync(slice->source_addr, - (void *)slice->local.dest_addr, slice->length, - cudaMemcpyDefault, stream); - else - err = cudaMemcpyAsync((void *)slice->local.dest_addr, - slice->source_addr, slice->length, - cudaMemcpyDefault, stream); - if (err != cudaSuccess) { - slice->markFailed(); - } else { - slice->status = Slice::POSTED; - slice->local.cuda_stream = (void *)stream; - } + void *src = (request.opcode == TransferRequest::READ) + ? (void *)slice->local.dest_addr + : (void *)slice->source_addr; + void *dst = (request.opcode == TransferRequest::READ) + ? slice->source_addr + : (void *)slice->local.dest_addr; + srcs.push_back(src); + dsts.push_back(dst); + sizes.push_back(slice->length); + slices.push_back(slice); } + + // Phase 2: Submit all memcpy operations + submitBatchMemcpy(slices, srcs, dsts, sizes, stream); + return Status::OK(); } -- 2.34.1 From 3a1117bccc9e37e677ee2bb52c769d74e8edee49 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E7=8E=8B=E9=B9=A4=E7=94=B7?= Date: Mon, 1 Jun 2026 19:32:57 +0800 Subject: [PATCH 159/382] [Build] fix(wheel): exclude libfabric/libefa from auditwheel bundle to avoid dual-libfabric EFA conflict (#2271) * fix(wheel): exclude libfabric/libefa from auditwheel bundle The wheel build runs `auditwheel repair`, which by default grafts every non-excluded shared library the extension links against into the wheel's `.libs/` directory and rewrites RPATH to prefer the bundled copy. The exclude list already carves out the system RDMA/EFA stack (libibverbs, libmlx5, libnuma, libcuda, ...) but missed libfabric and libefa. As a result a pip-installed wheel ships its own `mooncake_transfer_engine.libs/libfabric-.so.1` and `engine.so` loads that instead of the system `/opt/amazon/efa/lib/libfabric.so.1`. On AWS EFA hosts this puts two independent libfabric instances in one process: Mooncake uses the bundled one, while aws-ofi-nccl (loaded by NCCL) uses the system one. Each instance runs its own `ofi_hmem_init` and opens EFA devices independently. When Mooncake initializes first it claims the EFA device context; aws-ofi-nccl's later `fi_getinfo` then returns "provider efa output empty list" and NCCL silently falls back to the TCP provider (169.254.170.x), which hangs cross-node collectives such as `all_gather_object`. libfabric is the one library that MUST be shared with the system aws-ofi-nccl plugin, so it has to come from the system just like libibverbs/libmlx5 already do. Excluding it (and libefa) makes the wheel load the same libfabric the rest of the EFA stack uses, eliminating the dual-instance conflict. --------- Co-authored-by: Claude Opus 4.8 --- .../design/transfer-engine/efa_transport.md | 32 +++++++++++++++++-- scripts/build_wheel.sh | 2 ++ 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/docs/source/design/transfer-engine/efa_transport.md b/docs/source/design/transfer-engine/efa_transport.md index bc2a1479..1f811a9e 100644 --- a/docs/source/design/transfer-engine/efa_transport.md +++ b/docs/source/design/transfer-engine/efa_transport.md @@ -80,6 +80,34 @@ cp mooncake-common/libasio.so ../mooncake-wheel/mooncake/ pip install -e ../mooncake-wheel --no-build-isolation ``` +### 3. Building a Distributable Wheel (optional) + +To produce a relocatable wheel for distribution (instead of the editable +install above), use `scripts/build_wheel.sh`, which runs `auditwheel +repair` to bundle non-system dependencies: + +```bash +# After the cmake/make build above completes: +PYTHON_VERSION=3.13 BUILD_DIR=build bash scripts/build_wheel.sh 3.13 dist +pip install dist/mooncake_transfer_engine-*.whl +``` + +> **Important (EFA builds):** `auditwheel repair` excludes `libfabric` +> and `libefa` from the wheel so they resolve to the system EFA +> installation (`/opt/amazon/efa/lib`) at runtime. This is required +> because the in-process `aws-ofi-nccl` plugin (loaded by NCCL) links the +> **same** system `libfabric`. If the wheel bundled its own copy, the +> process would load two independent libfabric instances — Mooncake's +> bundled one and NCCL's system one — and whichever initializes first +> claims the EFA device, leaving the other with an empty provider list +> (`fi_getinfo: provider efa output empty list`). NCCL then silently +> falls back to the TCP provider and cross-node collectives such as +> `all_gather_object` hang. Excluding libfabric/libefa (see +> `scripts/build_wheel.sh`) keeps a single shared libfabric in the +> process. If you are on an older Mooncake build whose wheel still bundles +> libfabric, force the system copy with +> `export LD_PRELOAD=/opt/amazon/efa/lib/libfabric.so.1` as a workaround. + ## Verification Test EFA transport initialization: @@ -495,9 +523,9 @@ SGLang's PD-disaggregation Mooncake integration reads the transport from `MOONCA ### 1. Apply EFA Patch (only if SGLang version predates PR #25083) -Older SGLang releases hardcode `"rdma"` in the transfer engine init. Since [SGLang PR #25083](https://github.com/sgl-project/sglang/pull/25083) the protocol is read from `MOONCAKE_PROTOCOL`, so once that PR is in your build (or upstream `main`) **this step is unnecessary** — skip to step 2. +Older SGLang releases hardcode `"rdma"` in the transfer engine init. [SGLang PR #25083](https://github.com/sgl-project/sglang/pull/25083) has been **merged into SGLang `main`**, so the protocol is now read from `MOONCAKE_PROTOCOL`. If your SGLang build includes that PR (any recent `main` or release built after it), **this step is unnecessary** — skip to step 2. -If you are pinned to an older release, apply the [patch script](https://github.com/whn09/kimi-k2-sglang): +Only if you are pinned to an older release that predates PR #25083, apply the [patch script](https://github.com/whn09/kimi-k2-sglang): ```bash bash patch_sglang_efa.sh diff --git a/scripts/build_wheel.sh b/scripts/build_wheel.sh index fdd5737f..7b6b9d73 100755 --- a/scripts/build_wheel.sh +++ b/scripts/build_wheel.sh @@ -280,6 +280,8 @@ else fi ${AUDITWHEEL_CMD} repair ${OUTPUT_DIR}/*.whl \ --exclude libcurl.so* \ + --exclude libfabric.so* \ + --exclude libefa.so* \ --exclude libibverbs.so* \ --exclude libmlx5.so* \ --exclude libnuma.so* \ -- 2.34.1 From e9aa93592b2063955edbf1ab70978facf473f4f1 Mon Sep 17 00:00:00 2001 From: jinke <1534057243@qq.com> Date: Tue, 2 Jun 2026 10:15:24 +0800 Subject: [PATCH 160/382] [TENT] remote redis dependency (#2109) --- .../tent/config/transfer-engine.json | 1 - .../tent/include/tent/runtime/control_plane.h | 8 --- .../tent/include/tent/runtime/metastore.h | 11 ---- .../include/tent/runtime/segment_registry.h | 7 --- .../tent/src/runtime/control_plane.cpp | 16 +----- .../tent/src/runtime/metastore.cpp | 55 +++++++++++++------ .../tent/src/runtime/segment_registry.cpp | 15 ----- .../tent/src/runtime/transfer_engine_impl.cpp | 42 +------------- 8 files changed, 42 insertions(+), 113 deletions(-) diff --git a/mooncake-transfer-engine/tent/config/transfer-engine.json b/mooncake-transfer-engine/tent/config/transfer-engine.json index e5bdbbd1..bfe46b2e 100644 --- a/mooncake-transfer-engine/tent/config/transfer-engine.json +++ b/mooncake-transfer-engine/tent/config/transfer-engine.json @@ -2,7 +2,6 @@ "local_segment_name": "", "metadata_type": "p2p", "metadata_servers": "127.0.0.1:2379", - "redis_db_index": 0, "rpc_server_hostname": "127.0.0.1", "rpc_server_port": 0, "topology": { diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/control_plane.h b/mooncake-transfer-engine/tent/include/tent/runtime/control_plane.h index e7ce7ffe..a4739f27 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/control_plane.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/control_plane.h @@ -114,14 +114,6 @@ class ControlService { ControlService(const std::string& type, const std::string& servers, TransferEngineImpl* impl); - ControlService(const std::string& type, const std::string& servers, - const std::string& password, uint8_t db_index, - TransferEngineImpl* impl); - - ControlService(const std::string& type, const std::string& servers, - const std::string& username, const std::string& password, - uint8_t db_index, TransferEngineImpl* impl); - ~ControlService(); ControlService(const ControlService&) = delete; diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/metastore.h b/mooncake-transfer-engine/tent/include/tent/runtime/metastore.h index a70fdd8f..ab383b6d 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/metastore.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/metastore.h @@ -26,17 +26,6 @@ struct MetaStore { static std::shared_ptr Create(const std::string &type, const std::string &servers); - static std::shared_ptr Create(const std::string &type, - const std::string &servers, - const std::string &password, - uint8_t db_index); - - static std::shared_ptr Create(const std::string &type, - const std::string &servers, - const std::string &username, - const std::string &password, - uint8_t db_index); - MetaStore() {} virtual ~MetaStore() {} diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/segment_registry.h b/mooncake-transfer-engine/tent/include/tent/runtime/segment_registry.h index 97c586d0..46d5dc56 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/segment_registry.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/segment_registry.h @@ -53,13 +53,6 @@ class CentralSegmentRegistry : public SegmentRegistry { public: CentralSegmentRegistry(const std::string &type, const std::string &servers); - CentralSegmentRegistry(const std::string &type, const std::string &servers, - const std::string &password, uint8_t db_index); - - CentralSegmentRegistry(const std::string &type, const std::string &servers, - const std::string &username, - const std::string &password, uint8_t db_index); - virtual ~CentralSegmentRegistry() {} public: diff --git a/mooncake-transfer-engine/tent/src/runtime/control_plane.cpp b/mooncake-transfer-engine/tent/src/runtime/control_plane.cpp index 384828e4..49ca9abc 100644 --- a/mooncake-transfer-engine/tent/src/runtime/control_plane.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/control_plane.cpp @@ -156,26 +156,12 @@ Status ControlClient::unpinStageBuffer(const std::string& server_addr, ControlService::ControlService(const std::string& type, const std::string& servers, TransferEngineImpl* impl) - : ControlService(type, servers, "", "", 0, impl) {} - -ControlService::ControlService(const std::string& type, - const std::string& servers, - const std::string& password, uint8_t db_index, - TransferEngineImpl* impl) - : ControlService(type, servers, "", password, db_index, impl) {} - -ControlService::ControlService(const std::string& type, - const std::string& servers, - const std::string& username, - const std::string& password, uint8_t db_index, - TransferEngineImpl* impl) : bootstrap_callback_(nullptr), notify_callback_(nullptr), impl_(impl) { if (type == "p2p") { auto agent = std::make_unique(); manager_ = std::make_unique(std::move(agent)); } else { - auto agent = std::make_unique( - type, servers, username, password, db_index); + auto agent = std::make_unique(type, servers); manager_ = std::make_unique(std::move(agent)); } rpc_server_ = std::make_shared(); diff --git a/mooncake-transfer-engine/tent/src/runtime/metastore.cpp b/mooncake-transfer-engine/tent/src/runtime/metastore.cpp index f797b4d8..75b92d62 100644 --- a/mooncake-transfer-engine/tent/src/runtime/metastore.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/metastore.cpp @@ -27,23 +27,9 @@ namespace mooncake { namespace tent { -std::shared_ptr MetaStore::Create(const std::string &type, - const std::string &servers) { - return Create(type, servers, "", "", 0); -} -std::shared_ptr MetaStore::Create(const std::string &type, - const std::string &servers, - const std::string &password, - uint8_t db_index) { - return Create(type, servers, "", password, db_index); -} - -std::shared_ptr MetaStore::Create(const std::string &type, - const std::string &servers, - const std::string &username, - const std::string &password, - uint8_t db_index) { +std::shared_ptr MetaStore::Create(const std::string& type, + const std::string& servers) { std::shared_ptr plugin; #ifdef USE_ETCD if (type == "etcd") { @@ -52,6 +38,43 @@ std::shared_ptr MetaStore::Create(const std::string &type, #endif // USE_ETCD #ifdef USE_REDIS if (type == "redis") { + // Get Redis password from environment variable for security + std::string password; + const char* env_password = std::getenv("MC_REDIS_PASSWORD"); + if (env_password && *env_password) { + password = env_password; + } + + std::string username; + const char* env_username = std::getenv("MC_REDIS_USERNAME"); + if (env_username && *env_username) { + username = env_username; + } + + // Get Redis DB index from environment variable + int redis_db_index = REDIS_DEFAULT_DB_INDEX; + const char* env_db_index = std::getenv("MC_REDIS_DB_INDEX"); + if (env_db_index && *env_db_index) { + try { + redis_db_index = std::stoi(env_db_index); + } catch (const std::exception& e) { + LOG(WARNING) + << "Invalid MC_REDIS_DB_INDEX environment variable: " + << env_db_index << ", using default " + << static_cast(REDIS_DEFAULT_DB_INDEX); + } + } + + // Validate redis_db_index range (0-255) + uint8_t db_index = REDIS_DEFAULT_DB_INDEX; + if (redis_db_index >= 0 && redis_db_index <= REDIS_MAX_DB_INDEX) { + db_index = static_cast(redis_db_index); + } else { + LOG(WARNING) << "Invalid Redis DB index: " << redis_db_index + << ", using default " + << static_cast(REDIS_DEFAULT_DB_INDEX); + } + auto redis_plugin = std::make_shared(); auto status = redis_plugin->connect(servers, username, password, db_index); diff --git a/mooncake-transfer-engine/tent/src/runtime/segment_registry.cpp b/mooncake-transfer-engine/tent/src/runtime/segment_registry.cpp index f6f85971..3f932b4c 100644 --- a/mooncake-transfer-engine/tent/src/runtime/segment_registry.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/segment_registry.cpp @@ -34,21 +34,6 @@ CentralSegmentRegistry::CentralSegmentRegistry(const std::string &type, plugin_ = MetaStore::Create(type, servers); } -CentralSegmentRegistry::CentralSegmentRegistry(const std::string &type, - const std::string &servers, - const std::string &password, - uint8_t db_index) { - plugin_ = MetaStore::Create(type, servers, "", password, db_index); -} - -CentralSegmentRegistry::CentralSegmentRegistry(const std::string &type, - const std::string &servers, - const std::string &username, - const std::string &password, - uint8_t db_index) { - plugin_ = MetaStore::Create(type, servers, username, password, db_index); -} - Status CentralSegmentRegistry::getSegmentDesc(SegmentDescRef &desc, const std::string &segment_name) { if (!plugin_) diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index a8936422..01f0a596 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -277,32 +277,6 @@ Status TransferEngineImpl::construct() { auto metadata_type = conf_->get("metadata_type", "p2p"); auto metadata_servers = conf_->get("metadata_servers", ""); - // Get Redis password from environment variable for security - std::string redis_password; - const char* env_password = std::getenv("MC_REDIS_PASSWORD"); - if (env_password && *env_password) { - redis_password = env_password; - } - - std::string redis_username; - const char* env_username = std::getenv("MC_REDIS_USERNAME"); - if (env_username && *env_username) { - redis_username = env_username; - } - - // Get Redis DB index from environment variable or config - int redis_db_index_config = conf_->get("redis_db_index", 0); - const char* env_db_index = std::getenv("MC_REDIS_DB_INDEX"); - if (env_db_index && *env_db_index) { - try { - redis_db_index_config = std::stoi(env_db_index); - } catch (const std::exception& e) { - LOG(WARNING) << "Invalid REDIS_DB_INDEX environment variable: " - << env_db_index - << ", using config value: " << redis_db_index_config; - } - } - setLogLevel(conf_->get("log_level", "info")); hostname_ = conf_->get("rpc_server_hostname", ""); local_segment_name_ = conf_->get("local_segment_name", ""); @@ -321,20 +295,8 @@ Status TransferEngineImpl::construct() { auto loader = &Platform::getLoader(conf_); CHECK_STATUS(topology_->discover({loader})); - // Validate redis_db_index range (0-255) - uint8_t db_index = kRedisDefaultDbIndex; - if (redis_db_index_config >= 0 && - redis_db_index_config <= kRedisMaxDbIndex) { - db_index = static_cast(redis_db_index_config); - } else { - LOG(WARNING) << "Invalid Redis DB index: " << redis_db_index_config - << ", using default " - << static_cast(kRedisDefaultDbIndex); - } - - metadata_ = std::make_shared( - metadata_type, metadata_servers, redis_username, redis_password, - db_index, this); + metadata_ = + std::make_shared(metadata_type, metadata_servers, this); CHECK_STATUS(metadata_->start(port_, ipv6_)); -- 2.34.1 From 9b50b354b39403659ed796ef1bf26ac5640c5804 Mon Sep 17 00:00:00 2001 From: CAICAII <3360776475@qq.com> Date: Tue, 2 Jun 2026 10:33:08 +0800 Subject: [PATCH 161/382] [Store] Robustify ConfigDict size parsing (#2206) --- mooncake-store/include/utils.h | 29 +++++-- mooncake-store/src/real_client.cpp | 67 ++++++++------- mooncake-store/tests/pybind_client_test.cpp | 82 ++++++++++++++++--- mooncake-store/tests/utils_test.cpp | 14 ++++ .../tests/test_distributed_object_store.py | 40 +++++++++ 5 files changed, 179 insertions(+), 53 deletions(-) diff --git a/mooncake-store/include/utils.h b/mooncake-store/include/utils.h index 8e4d0994..d89c952f 100644 --- a/mooncake-store/include/utils.h +++ b/mooncake-store/include/utils.h @@ -171,14 +171,15 @@ std::string expected_to_str(const tl::expected& expected) { } /** - * @brief Convert a string representation of size to bytes + * @brief Parse a string representation of size to bytes * @param str String representation of size (e.g., "1.5 GB", "1024 MB", * "1048576") - * @return uint64_t Number of bytes, or 0 if parsing fails + * @return Parsed byte size, or std::nullopt if parsing fails */ -[[nodiscard]] inline uint64_t string_to_byte_size(const std::string& str) { +[[nodiscard]] inline std::optional try_string_to_byte_size( + const std::string& str) { if (str.empty()) { - return 0; + return std::nullopt; } // Create a copy for manipulation @@ -189,7 +190,7 @@ std::string expected_to_str(const tl::expected& expected) { s.erase(s.find_last_not_of(" \t\r\n") + 1); if (s.empty()) { - return 0; + return std::nullopt; } // Handle special case for "infinite" @@ -204,7 +205,10 @@ std::string expected_to_str(const tl::expected& expected) { try { value = std::stod(s, &pos); } catch (const std::exception&) { - return 0; // Failed to parse number + return std::nullopt; // Failed to parse number + } + if (value < 0) { + return std::nullopt; } if (pos >= s.length()) { @@ -238,10 +242,21 @@ std::string expected_to_str(const tl::expected& expected) { return static_cast(value); } else { // Unknown unit - return 0; + return std::nullopt; } } +/** + * @brief Convert a string representation of size to bytes + * @param str String representation of size (e.g., "1.5 GB", "1024 MB", + * "1048576") + * @return uint64_t Number of bytes, or 0 if parsing fails + */ +[[nodiscard]] inline uint64_t string_to_byte_size(const std::string& str) { + auto parsed = try_string_to_byte_size(str); + return parsed.value_or(0); +} + /** * @brief Convert a boolean-like string to a bool * @param str String representation ("1"/"true"/"yes"/"on" or diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 9f017345..dbc126da 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -954,30 +954,21 @@ inline std::string get_config(const ConfigDict &config, const std::string &key, return (it != config.end()) ? it->second : default_value; } -inline size_t get_config_size(const ConfigDict &config, const std::string &key, - size_t default_value) { +inline std::optional get_config_size(const ConfigDict &config, + const std::string &key, + size_t default_value) { auto it = config.find(key); if (it == config.end()) { return default_value; } - const std::string &value = it->second; - // Check for negative numbers (stoull incorrectly parses "-1" as large val) - if (!value.empty() && value[0] == '-') { - LOG(WARNING) << "Invalid negative value for config key '" << key - << "': " << value << ", using default: " << default_value; - return default_value; - } - try { - return std::stoull(value); - } catch (const std::invalid_argument &e) { - LOG(WARNING) << "Invalid non-numeric value for config key '" << key - << "': " << value << ", using default: " << default_value; - return default_value; - } catch (const std::out_of_range &e) { - LOG(WARNING) << "Value out of range for config key '" << key - << "': " << value << ", using default: " << default_value; - return default_value; + + auto parsed_size_opt = try_string_to_byte_size(it->second); + if (!parsed_size_opt.has_value()) { + LOG(ERROR) << "Invalid size value for config key '" << key + << "': " << it->second; + return std::nullopt; } + return static_cast(parsed_size_opt.value()); } } // namespace @@ -999,10 +990,18 @@ tl::expected RealClient::setup_internal( } // Extract optional parameters with defaults - size_t global_segment_size = get_config_size( + auto global_segment_size_opt = get_config_size( config, CONFIG_KEY_GLOBAL_SEGMENT_SIZE, DEFAULT_GLOBAL_SEGMENT_SIZE); - size_t local_buffer_size = get_config_size( + if (!global_segment_size_opt.has_value()) { + return tl::unexpected(ErrorCode::INVALID_PARAMS); + } + auto local_buffer_size_opt = get_config_size( config, CONFIG_KEY_LOCAL_BUFFER_SIZE, DEFAULT_LOCAL_BUFFER_SIZE); + if (!local_buffer_size_opt.has_value()) { + return tl::unexpected(ErrorCode::INVALID_PARAMS); + } + size_t global_segment_size = global_segment_size_opt.value(); + size_t local_buffer_size = local_buffer_size_opt.value(); std::string protocol = get_config(config, CONFIG_KEY_PROTOCOL, DEFAULT_PROTOCOL); std::string rdma_devices = get_config(config, CONFIG_KEY_RDMA_DEVICES); @@ -1011,19 +1010,19 @@ tl::expected RealClient::setup_internal( std::string ipc_socket_path = get_config(config, CONFIG_KEY_IPC_SOCKET_PATH); - // Validate size parameters are within acceptable ranges - if (global_segment_size < MIN_SEGMENT_SIZE || - global_segment_size > MAX_SEGMENT_SIZE) { - LOG(ERROR) << "Invalid " << CONFIG_KEY_GLOBAL_SEGMENT_SIZE << ": " - << global_segment_size << ", must be between " - << MIN_SEGMENT_SIZE << " and " << MAX_SEGMENT_SIZE; - return tl::unexpected(ErrorCode::INVALID_PARAMS); - } - if (local_buffer_size < MIN_SEGMENT_SIZE || - local_buffer_size > MAX_SEGMENT_SIZE) { - LOG(ERROR) << "Invalid " << CONFIG_KEY_LOCAL_BUFFER_SIZE << ": " - << local_buffer_size << ", must be between " - << MIN_SEGMENT_SIZE << " and " << MAX_SEGMENT_SIZE; + // A size of 0 keeps the pure client/server setup semantics. + auto validate_size = [](const char *key, size_t value) { + if ((value != 0 && value < MIN_SEGMENT_SIZE) || + value > MAX_SEGMENT_SIZE) { + LOG(ERROR) << "Invalid " << key << ": " << value + << ", must be 0 or between " << MIN_SEGMENT_SIZE + << " and " << MAX_SEGMENT_SIZE; + return false; + } + return true; + }; + if (!validate_size(CONFIG_KEY_GLOBAL_SEGMENT_SIZE, global_segment_size) || + !validate_size(CONFIG_KEY_LOCAL_BUFFER_SIZE, local_buffer_size)) { return tl::unexpected(ErrorCode::INVALID_PARAMS); } diff --git a/mooncake-store/tests/pybind_client_test.cpp b/mooncake-store/tests/pybind_client_test.cpp index dec33176..28cf616b 100644 --- a/mooncake-store/tests/pybind_client_test.cpp +++ b/mooncake-store/tests/pybind_client_test.cpp @@ -85,6 +85,23 @@ class RealClientTest : public ::testing::Test { 0); } + ConfigDict MakeConfigDict(const std::string& local_hostname, + const std::string& global_segment_size, + const std::string& local_buffer_size) const { + const std::string rdma_devices = (FLAGS_protocol == std::string("rdma")) + ? FLAGS_device_name + : std::string(""); + ConfigDict config; + config[CONFIG_KEY_LOCAL_HOSTNAME] = local_hostname; + config[CONFIG_KEY_METADATA_SERVER] = "P2PHANDSHAKE"; + config[CONFIG_KEY_GLOBAL_SEGMENT_SIZE] = global_segment_size; + config[CONFIG_KEY_LOCAL_BUFFER_SIZE] = local_buffer_size; + config[CONFIG_KEY_PROTOCOL] = FLAGS_protocol; + config[CONFIG_KEY_RDMA_DEVICES] = rdma_devices; + config[CONFIG_KEY_MASTER_SERVER_ADDR] = master_address_; + return config; + } + std::string CreateTempSegmentFile(size_t size) { std::string path = "/tmp/mooncake_real_client_segment_XXXXXX"; int fd = mkstemp(path.data()); @@ -889,22 +906,12 @@ TEST_F(RealClientTest, SetupWithConfigDict) { master_address_ = master_.master_address(); LOG(INFO) << "Started in-proc master at " << master_address_; - // Setup the client using ConfigDict - const std::string rdma_devices = (FLAGS_protocol == std::string("rdma")) - ? FLAGS_device_name - : std::string(""); - ConfigDict config; auto result = py_client_->setup_internal(config); ASSERT_FALSE(result.has_value()) << "Setup with empty config should fail"; - config[CONFIG_KEY_LOCAL_HOSTNAME] = "localhost:17813"; - config[CONFIG_KEY_METADATA_SERVER] = "P2PHANDSHAKE"; - config[CONFIG_KEY_GLOBAL_SEGMENT_SIZE] = std::to_string(16 * 1024 * 1024); - config[CONFIG_KEY_LOCAL_BUFFER_SIZE] = std::to_string(16 * 1024 * 1024); - config[CONFIG_KEY_PROTOCOL] = FLAGS_protocol; - config[CONFIG_KEY_RDMA_DEVICES] = rdma_devices; - config[CONFIG_KEY_MASTER_SERVER_ADDR] = master_address_; + config = MakeConfigDict("localhost:17813", std::to_string(16 * 1024 * 1024), + std::to_string(16 * 1024 * 1024)); result = py_client_->setup_internal(config); ASSERT_TRUE(result.has_value()) << "Setup with ConfigDict should succeed"; @@ -929,6 +936,57 @@ TEST_F(RealClientTest, SetupWithConfigDict) { EXPECT_EQ(retrieved_data, test_data) << "Retrieved data should match"; } +TEST_F(RealClientTest, SetupWithConfigDictHumanReadableSizes) { + ASSERT_TRUE(master_.Start(InProcMasterConfigBuilder().build())) + << "Failed to start in-proc master"; + master_address_ = master_.master_address(); + + ConfigDict config = MakeConfigDict("localhost:17814", "16MB", "16 MB"); + auto result = py_client_->setup_internal(config); + ASSERT_TRUE(result.has_value()) + << "Setup should accept human-readable size strings"; +} + +TEST_F(RealClientTest, SetupWithConfigDictAllowsZeroSizes) { + ASSERT_TRUE(master_.Start(InProcMasterConfigBuilder().build())) + << "Failed to start in-proc master"; + master_address_ = master_.master_address(); + + ConfigDict config = MakeConfigDict("localhost:17815", "0", "0"); + auto result = py_client_->setup_internal(config); + ASSERT_TRUE(result.has_value()) + << "Setup should preserve zero-size pure client/server semantics"; +} + +TEST_F(RealClientTest, ErrSetupWithInvalidConfigDictSize) { + GLogMuter muter; + ASSERT_TRUE(master_.Start(InProcMasterConfigBuilder().build())) + << "Failed to start in-proc master"; + master_address_ = master_.master_address(); + + struct InvalidSizeCase { + const char* local_hostname; + const char* global_segment_size; + const char* local_buffer_size; + }; + + const InvalidSizeCase invalid_size_cases[] = { + {"localhost:17816", "50%", "16MB"}, + {"localhost:17817", "16MB", "16XB"}, + {"localhost:17818", "-5", "16MB"}, + }; + + for (const auto& test_case : invalid_size_cases) { + ConfigDict config = MakeConfigDict(test_case.local_hostname, + test_case.global_segment_size, + test_case.local_buffer_size); + auto result = py_client_->setup_internal(config); + EXPECT_FALSE(result.has_value()) + << "Invalid explicit size values should fail instead of being " + "partially parsed or silently defaulted"; + } +} + TEST_F(RealClientTest, ErrSetupWithInvalidArgument) { GLogMuter muter; // Case 1: Setup with unreachable master address diff --git a/mooncake-store/tests/utils_test.cpp b/mooncake-store/tests/utils_test.cpp index 32f703f4..b3102fd9 100644 --- a/mooncake-store/tests/utils_test.cpp +++ b/mooncake-store/tests/utils_test.cpp @@ -23,6 +23,20 @@ TEST(UtilsTest, ByteSizeToString) { EXPECT_EQ(byte_size_to_string(15 * 1024 * 1024 + 44048), "15.04 MB"); } +TEST(UtilsTest, StringToByteSize) { + auto parsed = try_string_to_byte_size("16 MB"); + ASSERT_TRUE(parsed.has_value()); + EXPECT_EQ(parsed.value(), 16ULL * 1024 * 1024); + + parsed = try_string_to_byte_size("0"); + ASSERT_TRUE(parsed.has_value()); + EXPECT_EQ(parsed.value(), 0); + + EXPECT_FALSE(try_string_to_byte_size("-5").has_value()); + EXPECT_FALSE(try_string_to_byte_size("16XB").has_value()); + EXPECT_EQ(string_to_byte_size("-5"), 0); +} + TEST(UtilsTest, StringToBool) { EXPECT_EQ(string_to_bool("1"), true); EXPECT_EQ(string_to_bool("true"), true); diff --git a/mooncake-wheel/tests/test_distributed_object_store.py b/mooncake-wheel/tests/test_distributed_object_store.py index 52530d3b..9e5d9e96 100644 --- a/mooncake-wheel/tests/test_distributed_object_store.py +++ b/mooncake-wheel/tests/test_distributed_object_store.py @@ -38,6 +38,46 @@ def get_client(store, local_buffer_size_param=None): if retcode: raise RuntimeError(f"Failed to setup store client. Return code: {retcode}") + +def get_config_dict(global_segment_size, local_buffer_size): + """Build a config dictionary for the MooncakeDistributedStore setup wrapper.""" + return { + "local_hostname": os.getenv("LOCAL_HOSTNAME", "localhost"), + "metadata_server": os.getenv( + "MC_METADATA_SERVER", "http://127.0.0.1:8080/metadata" + ), + "global_segment_size": global_segment_size, + "local_buffer_size": local_buffer_size, + "protocol": os.getenv("PROTOCOL", "tcp"), + "rdma_devices": os.getenv("DEVICE_NAME", "ibp6s0"), + "master_server_addr": os.getenv("MASTER_SERVER", "127.0.0.1:50051"), + } + + +class TestConfigDictSetup(unittest.TestCase): + """Test configuration-dictionary setup through the Python store wrapper.""" + + def test_human_readable_sizes(self): + store = MooncakeDistributedStore() + self.addCleanup(store.close) + + retcode = store.setup(get_config_dict("16MB", "16 MB")) + self.assertEqual(retcode, 0) + + test_data = b"test_config_dict_human_readable_value" + key = f"test_config_dict_human_readable_key_{os.getpid()}" + + self.assertEqual(store.put(key, test_data), 0) + self.assertEqual(store.get(key), test_data) + + def test_unsupported_percentage_size(self): + store = MooncakeDistributedStore() + self.addCleanup(store.close) + + retcode = store.setup(get_config_dict("50%", "16MB")) + self.assertNotEqual(retcode, 0) + + class TestZeroLocalBufferSize(unittest.TestCase): """Test class for zero local buffer size scenarios.""" -- 2.34.1 From 94b9a6bc3ee7c4cb343cf433c2742b1d8c7560c2 Mon Sep 17 00:00:00 2001 From: leonzzhu Date: Tue, 2 Jun 2026 10:43:16 +0800 Subject: [PATCH 162/382] [Store] fix: unify default cluster_namespace to match master's DEFAULT_CLUSTER_ID (#2244) The client-side ResolveClusterNamespace() in etcd leader coordinator, redis leader coordinator, and redis snapshot catalog store all used 'mooncake' as the fallback default, while the master-side --cluster_id flag defaults to 'mooncake_cluster' (DEFAULT_CLUSTER_ID). This inconsistency caused clients to look up a different etcd/redis key than what the master registered, making them unable to discover the master when both sides use defaults. Fix by replacing the hardcoded 'mooncake' fallback with the DEFAULT_CLUSTER_ID constant ('mooncake_cluster') so that client and master use the same namespace out of the box. Also update documentation to reflect the corrected default value. --- docs/source/design/mooncake-store.md | 2 +- docs/source/zh_archive/mooncake-store.md | 2 +- .../backends/etcd/etcd_leader_coordinator.cpp | 4 +++- .../backends/redis/redis_leader_coordinator.cpp | 4 +++- .../redis/redis_snapshot_catalog_store.cpp | 14 +++++++++++--- 5 files changed, 19 insertions(+), 7 deletions(-) diff --git a/docs/source/design/mooncake-store.md b/docs/source/design/mooncake-store.md index f12099e1..3e44f270 100644 --- a/docs/source/design/mooncake-store.md +++ b/docs/source/design/mooncake-store.md @@ -836,7 +836,7 @@ The HTTP metadata server can be configured using the following parameters: - **`http_metadata_server_port`** (integer, default: `8080`): Specifies the TCP port on which the HTTP metadata server will listen for incoming connections. This port must be available and not conflict with other services. - **`http_metadata_server_host`** (string, default: `"0.0.0.0"`): Specifies the host address for the HTTP metadata server to bind to. Use `"0.0.0.0"` to listen on all available network interfaces, or specify a specific IP address for security purposes. #### Environment Variables -- MC_STORE_CLUSTER_ID: Identify the metadata when multiple cluster share the same master, default 'mooncake'. +- MC_STORE_CLUSTER_ID: Identify the metadata when multiple cluster share the same master, default 'mooncake_cluster'. - MC_STORE_MEMCPY: Enables or disables local memcpy optimization, set to 1/true to enable, 0/false to disable. - MC_STORE_CLIENT_METRIC: Enables client metric reporting, enabled by default; set to 0/false to disable. - MC_STORE_CLIENT_METRIC_INTERVAL: Reporting interval in seconds, default 0 (collects but does not report). diff --git a/docs/source/zh_archive/mooncake-store.md b/docs/source/zh_archive/mooncake-store.md index 841c9301..768b310a 100644 --- a/docs/source/zh_archive/mooncake-store.md +++ b/docs/source/zh_archive/mooncake-store.md @@ -721,7 +721,7 @@ HTTP 元数据服务器可通过以下参数进行配置: #### 环境变量说明 -- **MC_STORE_CLUSTER_ID**: 在多集群复用 master 场景下标识元数据, 默认 'mooncake' +- **MC_STORE_CLUSTER_ID**: 在多集群复用 master 场景下标识元数据, 默认 'mooncake_cluster' - **MC_STORE_MEMCPY**: 控制是否启用本地 memcpy 优化, 1/true 启用, 0/false 禁用 - **MC_STORE_CLIENT_METRIC**: 启用客户端指标上报, 默认启用;设为 0/false 禁用 - **MC_STORE_CLIENT_METRIC_INTERVAL**: 指标上报间隔(秒), 默认 0(仅收集不上报) diff --git a/mooncake-store/src/ha/leadership/backends/etcd/etcd_leader_coordinator.cpp b/mooncake-store/src/ha/leadership/backends/etcd/etcd_leader_coordinator.cpp index 58c1e238..3facf9d2 100644 --- a/mooncake-store/src/ha/leadership/backends/etcd/etcd_leader_coordinator.cpp +++ b/mooncake-store/src/ha/leadership/backends/etcd/etcd_leader_coordinator.cpp @@ -11,6 +11,8 @@ #include #include +#include "types.h" + namespace mooncake { namespace ha { namespace backends { @@ -369,7 +371,7 @@ ClusterNamespace EtcdLeaderCoordinator::ResolveClusterNamespace( if (env_cluster_id != nullptr && std::strlen(env_cluster_id) > 0) { resolved_namespace = env_cluster_id; } else { - resolved_namespace = "mooncake"; + resolved_namespace = DEFAULT_CLUSTER_ID; } return resolved_namespace; } diff --git a/mooncake-store/src/ha/leadership/backends/redis/redis_leader_coordinator.cpp b/mooncake-store/src/ha/leadership/backends/redis/redis_leader_coordinator.cpp index 57a5ce3a..e8539c01 100644 --- a/mooncake-store/src/ha/leadership/backends/redis/redis_leader_coordinator.cpp +++ b/mooncake-store/src/ha/leadership/backends/redis/redis_leader_coordinator.cpp @@ -11,6 +11,8 @@ #include #include + +#include "types.h" #ifdef STORE_USE_REDIS #include #endif @@ -665,7 +667,7 @@ ClusterNamespace RedisLeaderCoordinator::ResolveClusterNamespace( if (env_cluster_id != nullptr && std::strlen(env_cluster_id) > 0) { resolved_namespace = env_cluster_id; } else { - resolved_namespace = "mooncake"; + resolved_namespace = DEFAULT_CLUSTER_ID; } return resolved_namespace; } diff --git a/mooncake-store/src/ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.cpp b/mooncake-store/src/ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.cpp index 8b04f855..ddbd0fa3 100644 --- a/mooncake-store/src/ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.cpp +++ b/mooncake-store/src/ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.cpp @@ -1,11 +1,14 @@ #include "ha/snapshot/catalog/backends/redis/redis_snapshot_catalog_store.h" +#include #include #include #include #include #include + +#include "types.h" #ifdef STORE_USE_REDIS #include #endif @@ -315,10 +318,15 @@ ErrorCode RedisSnapshotCatalogStore::Delete(const SnapshotId& snapshot_id) { ClusterNamespace RedisSnapshotCatalogStore::ResolveClusterNamespace( const ClusterNamespace& cluster_namespace) { - if (cluster_namespace.empty()) { - return "mooncake"; + if (!cluster_namespace.empty()) { + return cluster_namespace; } - return cluster_namespace; + + const char* env_cluster_id = std::getenv("MC_STORE_CLUSTER_ID"); + if (env_cluster_id != nullptr && *env_cluster_id != '\0') { + return env_cluster_id; + } + return DEFAULT_CLUSTER_ID; } std::string RedisSnapshotCatalogStore::BuildLatestKey( -- 2.34.1 From 6f22861dfd6f1b34be6e233383c8327a96f09b98 Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Tue, 2 Jun 2026 11:27:20 +0800 Subject: [PATCH 163/382] =?UTF-8?q?[TE]=20fix:=20pass=20default=20port=20w?= =?UTF-8?q?hen=20parsing=20TENT=20RDMA=20bind=20a=E2=80=A6=20(#2289)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * [TransferEngine] fix: pass default port when parsing TENT RDMA bind address * [TransferEngine] fix: read TENT RDMA bind address from config --- .../src/transfer_metadata.cpp | 7 +++--- .../transport/efa_transport/efa_context.cpp | 6 ++--- .../tent/src/common/config.cpp | 1 + .../src/transport/rdma/rdma_transport.cpp | 19 ++++++++------- .../transfer_engine_config_override_test.cpp | 23 +++++++++++++++++++ 5 files changed, 41 insertions(+), 15 deletions(-) diff --git a/mooncake-transfer-engine/src/transfer_metadata.cpp b/mooncake-transfer-engine/src/transfer_metadata.cpp index f2ed39c3..b1a46aaa 100644 --- a/mooncake-transfer-engine/src/transfer_metadata.cpp +++ b/mooncake-transfer-engine/src/transfer_metadata.cpp @@ -914,8 +914,7 @@ std::shared_ptr TransferMetadata::getSegmentDesc( // the TCP-routable segment_name. Cache the mapping so subsequent // sendHandshake() calls can resolve the peer's TCP address from the // RDMA server name extracted from NIC paths. - if (p2p_handshake_mode_ && result && - !result->rdma_server_name.empty() && + if (p2p_handshake_mode_ && result && !result->rdma_server_name.empty() && result->rdma_server_name != segment_name) { auto [tcp_ip, tcp_port] = parseHostNameWithPort(segment_name); RWSpinlock::WriteGuard guard(rpc_meta_lock_); @@ -926,8 +925,8 @@ std::shared_ptr TransferMetadata::getSegmentDesc( meta.sockfd = -1; rpc_meta_map_[result->rdma_server_name] = meta; LOG(INFO) << "P2P: cached RDMA->TCP mapping: " - << result->rdma_server_name << " -> " << tcp_ip - << ":" << tcp_port; + << result->rdma_server_name << " -> " << tcp_ip << ":" + << tcp_port; } } diff --git a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp index 9f0b6fab..54807e77 100644 --- a/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp +++ b/mooncake-transfer-engine/src/transport/efa_transport/efa_context.cpp @@ -665,9 +665,9 @@ int EfaContext::submitPostSend( slice->rdma.dest_rkey = peer_segment_desc->buffers[buffer_id].rkey[device_id]; - std::string peer_nic_path = - peer_segment_desc->nicPathServerName() + "@" + - peer_segment_desc->devices[device_id].name; + std::string peer_nic_path = peer_segment_desc->nicPathServerName() + + "@" + + peer_segment_desc->devices[device_id].name; slice->peer_nic_path = peer_nic_path; slices_by_peer[peer_nic_path].push_back(slice); } diff --git a/mooncake-transfer-engine/tent/src/common/config.cpp b/mooncake-transfer-engine/tent/src/common/config.cpp index c6703531..0a48e997 100644 --- a/mooncake-transfer-engine/tent/src/common/config.cpp +++ b/mooncake-transfer-engine/tent/src/common/config.cpp @@ -85,6 +85,7 @@ Status ConfigHelper::loadFromEnv(Config& config) { } // Legacy keys for backward compatibility (MC_* env vars) + setConfig(config, "MC_RDMA_BIND_ADDRESS", "transports/rdma/bind_address"); setConfig(config, "MC_NUM_CQ_PER_CTX", "transports/rdma/device/num_cq_list"); setConfig(config, "MC_NUM_COMP_CHANNELS_PER_CTX", diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp index 0bf8733a..55a3a34b 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp @@ -217,13 +217,15 @@ Status RdmaTransport::install(std::string& local_segment_name, local_topology_ = local_topology; // In dual-NIC environments (e.g. separate TCP and RDMA interfaces), - // MC_RDMA_BIND_ADDRESS allows NIC paths to use an RDMA-reachable IP - // while local_segment_name_ keeps the TCP-reachable address for P2P. - const char *rdma_bind_addr = std::getenv("MC_RDMA_BIND_ADDRESS"); - if (rdma_bind_addr && rdma_bind_addr[0] != '\0') { - auto [host_name, port] = parseHostNameWithPort(local_segment_name); - rdma_server_name_ = - std::string(rdma_bind_addr) + ":" + std::to_string(port); + // transports/rdma/bind_address allows NIC paths to use an RDMA-reachable + // IP while local_segment_name_ keeps the TCP-reachable address for P2P. + const auto rdma_bind_addr = conf_->get("transports/rdma/bind_address", ""); + if (!rdma_bind_addr.empty()) { + const auto default_port = + conf_->get("rpc_server_port", static_cast(0)); + auto [host_name, port] = + parseHostNameWithPort(local_segment_name, default_port); + rdma_server_name_ = rdma_bind_addr + ":" + std::to_string(port); LOG(INFO) << "RdmaTransport(TENT): using RDMA bind address " << rdma_server_name_ << " (TCP address: " << local_segment_name_ << ")"; @@ -599,7 +601,8 @@ std::shared_ptr RdmaTransport::getEndpoint(SegmentID target_id, return nullptr; } std::shared_ptr endpoint; - std::string peer_name = MakeNicPath(segment_desc->nicPathServerName(), target_dev_name); + std::string peer_name = + MakeNicPath(segment_desc->nicPathServerName(), target_dev_name); endpoint = context->endpointStore()->getOrInsert(peer_name); if (!endpoint) { LOG(ERROR) << "Cannot allocate endpoint " << peer_name; diff --git a/mooncake-transfer-engine/tent/tests/transfer_engine_config_override_test.cpp b/mooncake-transfer-engine/tent/tests/transfer_engine_config_override_test.cpp index 706f7a53..8704924b 100644 --- a/mooncake-transfer-engine/tent/tests/transfer_engine_config_override_test.cpp +++ b/mooncake-transfer-engine/tent/tests/transfer_engine_config_override_test.cpp @@ -260,6 +260,29 @@ class TestHttpMetadataServer { }; #endif +TEST(TransferEngineConfigOverrideTest, + LegacyRdmaBindAddressEnvLoadsIntoTentConfig) { + EnvVarGuard guard("MC_RDMA_BIND_ADDRESS", "10.0.0.2"); + + Config config; + ASSERT_TRUE(ConfigHelper().loadFromEnv(config).ok()); + + EXPECT_EQ(config.get("transports/rdma/bind_address", ""), "10.0.0.2"); +} + +TEST(TransferEngineConfigOverrideTest, + LegacyRdmaBindAddressEnvOverridesMcTentConf) { + EnvVarGuard conf_guard( + "MC_TENT_CONF", + R"({"transports":{"rdma":{"bind_address":"10.0.0.1"}}})"); + EnvVarGuard bind_guard("MC_RDMA_BIND_ADDRESS", "10.0.0.2"); + + Config config; + ASSERT_TRUE(ConfigHelper().loadFromEnv(config).ok()); + + EXPECT_EQ(config.get("transports/rdma/bind_address", ""), "10.0.0.2"); +} + TEST(TransferEngineConfigOverrideTest, ExplicitMetadataOverridesDriveSuccessfulHttpInitialization) { #ifdef _WIN32 -- 2.34.1 From b4f47b40343670dfb64b4752fabe38f16a378594 Mon Sep 17 00:00:00 2001 From: Colors-111 <70190328+Colors-111@users.noreply.github.com> Date: Tue, 2 Jun 2026 12:20:08 +0800 Subject: [PATCH 164/382] [Store] RemoveAll not deleting SSD offload files, enable storage_backend_->RemoveAll() in Client::RemoveAll (#2283) * Enable storage_backend_->RemoveAll() in Client::RemoveAll * complete --------- Co-authored-by: ruanzhao --- mooncake-store/src/client_service.cpp | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index c1badfd2..e968e6d3 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -2528,10 +2528,11 @@ tl::expected Client::RemoveByRegex(const ObjectKey& str, } tl::expected Client::RemoveAll(bool force) { - // if (storage_backend_) { - // storage_backend_->RemoveAll(); - // } - return master_client_.RemoveAll(force); + auto result = master_client_.RemoveAll(force); + if (result && storage_backend_) { + storage_backend_->RemoveAll(); + } + return result; } std::vector> Client::BatchRemove( -- 2.34.1 From a953dcb110c09994d37c1fe0b30d90b8dc71bb62 Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Tue, 2 Jun 2026 14:14:24 +0800 Subject: [PATCH 165/382] [Store] Propagate tenant identity through object RPCs (#2288) --- mooncake-integration/store/store_py.cpp | 7 +- mooncake-store/include/client_service.h | 6 +- mooncake-store/include/dummy_client.h | 3 +- mooncake-store/include/master_client.h | 10 +- mooncake-store/include/master_service.h | 42 +++++- mooncake-store/include/pyclient.h | 3 +- mooncake-store/include/real_client.h | 6 +- mooncake-store/include/rpc_service.h | 74 ++++++---- mooncake-store/include/types.h | 1 + mooncake-store/src/client_service.cpp | 12 +- mooncake-store/src/master_client.cpp | 50 +++---- mooncake-store/src/master_service.cpp | 136 +++++++++++++++++-- mooncake-store/src/real_client.cpp | 22 +-- mooncake-store/src/rpc_service.cpp | 126 ++++++++++------- mooncake-store/tests/master_service_test.cpp | 125 +++++++++++++++++ 15 files changed, 484 insertions(+), 139 deletions(-) diff --git a/mooncake-integration/store/store_py.cpp b/mooncake-integration/store/store_py.cpp index a957c7b7..032e7528 100644 --- a/mooncake-integration/store/store_py.cpp +++ b/mooncake-integration/store/store_py.cpp @@ -1919,7 +1919,8 @@ PYBIND11_MODULE(store, m) { const std::string &master_server_addr = "127.0.0.1:50051", const py::object &engine = py::none(), bool enable_ssd_offload = false, - const std::string &ssd_offload_path = "") { + const std::string &ssd_offload_path = "", + const std::string &tenant_id = "default") { auto real_client = self.init_real_client(); std::shared_ptr transfer_engine = nullptr; @@ -1931,14 +1932,14 @@ PYBIND11_MODULE(store, m) { local_hostname, metadata_server, global_segment_size, local_buffer_size, protocol, rdma_devices, master_server_addr, transfer_engine, "", enable_ssd_offload, - ssd_offload_path); + ssd_offload_path, tenant_id); }, py::arg("local_hostname"), py::arg("metadata_server"), py::arg("global_segment_size"), py::arg("local_buffer_size"), py::arg("protocol"), py::arg("rdma_devices"), py::arg("master_server_addr"), py::arg("engine") = py::none(), py::arg("enable_ssd_offload") = false, - py::arg("ssd_offload_path") = "") + py::arg("ssd_offload_path") = "", py::arg("tenant_id") = "default") .def( "setup", [](MooncakeStorePyWrapper &self, const py::dict &config_dict) { diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index c005b02e..57ee11a7 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -86,7 +86,8 @@ class Client { const std::optional& device_names = std::nullopt, const std::string& master_server_entry = kDefaultMasterAddress, const std::shared_ptr& transfer_engine = nullptr, - std::map labels = {}); + std::map labels = {}, + const std::string& tenant_id = "default"); /** * @brief Retrieves data for a given key @@ -627,7 +628,8 @@ class Client { */ Client(const std::string& local_hostname, const std::string& metadata_connstring, const std::string& protocol, - const std::map& labels = {}); + const std::map& labels = {}, + const std::string& tenant_id = "default"); private: /** diff --git a/mooncake-store/include/dummy_client.h b/mooncake-store/include/dummy_client.h index 3eedb17d..8c2b5c5e 100644 --- a/mooncake-store/include/dummy_client.h +++ b/mooncake-store/include/dummy_client.h @@ -29,7 +29,8 @@ class DummyClient : public PyClient { const std::shared_ptr &transfer_engine, const std::string &ipc_socket_path, bool enable_ssd_offload = false, - const std::string &ssd_offload_path = "") { + const std::string &ssd_offload_path = "", + const std::string &tenant_id = "default") { // Dummy client does not support real setup return -1; }; diff --git a/mooncake-store/include/master_client.h b/mooncake-store/include/master_client.h index 5da96fba..a10fd15c 100644 --- a/mooncake-store/include/master_client.h +++ b/mooncake-store/include/master_client.h @@ -52,8 +52,11 @@ inline void MaybeEnableRdmaSocketConfig(SocketConfigVariant& socket_config) { */ class MasterClient { public: - MasterClient(const UUID& client_id, MasterClientMetric* metrics = nullptr) - : client_id_(client_id), metrics_(metrics) { + MasterClient(const UUID& client_id, MasterClientMetric* metrics = nullptr, + std::string tenant_id = "default") + : client_id_(client_id), + tenant_id_(NormalizeTenantId(std::move(tenant_id))), + metrics_(metrics) { coro_io::client_pool::pool_config pool_conf{}; @@ -635,6 +638,9 @@ class MasterClient { // The client identification. const UUID client_id_; + // Tenant identity for this client instance. + const std::string tenant_id_; + // Metrics for tracking RPC operations MasterClientMetric* metrics_; std::shared_ptr> diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index e33fe41d..0580000c 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -318,6 +318,9 @@ class MasterService { */ auto PutRevoke(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected; + auto PutRevoke(const UUID& client_id, const std::string& key, + const std::string& tenant_id, ReplicaType replica_type) + -> tl::expected; /** * @brief Complete a batch of put operations @@ -327,6 +330,10 @@ class MasterService { std::vector> BatchPutEnd( const UUID& client_id, const std::vector& keys, ReplicaType replica_type = ReplicaType::ALL); + std::vector> BatchPutEnd( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id, + ReplicaType replica_type = ReplicaType::ALL); /** * @brief Revoke a batch of put operations @@ -336,6 +343,10 @@ class MasterService { std::vector> BatchPutRevoke( const UUID& client_id, const std::vector& keys, ReplicaType replica_type = ReplicaType::ALL); + std::vector> BatchPutRevoke( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id, + ReplicaType replica_type = ReplicaType::ALL); /** * @brief Start an upsert operation. If the key does not exist, behaves @@ -359,6 +370,9 @@ class MasterService { */ auto UpsertEnd(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected; + auto UpsertEnd(const UUID& client_id, const std::string& key, + const std::string& tenant_id, ReplicaType replica_type) + -> tl::expected; /** * @brief Revoke an upsert operation. Delegates to PutRevoke. @@ -366,6 +380,9 @@ class MasterService { auto UpsertRevoke(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected; + auto UpsertRevoke(const UUID& client_id, const std::string& key, + const std::string& tenant_id, ReplicaType replica_type) + -> tl::expected; /** * @brief Start a batch of upsert operations. @@ -375,18 +392,30 @@ class MasterService { const std::vector& keys, const std::vector& slice_lengths, const ReplicateConfig& config); + std::vector, ErrorCode>> + BatchUpsertStart(const UUID& client_id, + const std::vector& keys, + const std::string& tenant_id, + const std::vector& slice_lengths, + const ReplicateConfig& config); /** * @brief Complete a batch of upsert operations. Delegates to BatchPutEnd. */ std::vector> BatchUpsertEnd( const UUID& client_id, const std::vector& keys); + std::vector> BatchUpsertEnd( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id); /** * @brief Revoke a batch of upsert operations. Delegates to BatchPutRevoke. */ std::vector> BatchUpsertRevoke( const UUID& client_id, const std::vector& keys); + std::vector> BatchUpsertRevoke( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id); /** * @brief Evict a disk replica for a key (triggered by client-side disk @@ -483,12 +512,20 @@ class MasterService { bool force = false) -> tl::expected; /** - * @brief Remove all objects and their replicas + * @brief Remove all objects and their replicas across all tenants. * @param force If true, skip lease and replication task checks. * @return return the number of objects removed */ long RemoveAll(bool force = false); + /** + * @brief Remove all objects and their replicas for a single tenant. + * @param tenant_id The tenant whose objects should be removed. + * @param force If true, skip lease and replication task checks. + * @return return the number of objects removed + */ + long RemoveAll(const std::string& tenant_id, bool force = false); + /** * @brief Batch remove objects and their replicas * @param keys The list of keys to remove. @@ -497,6 +534,9 @@ class MasterService { */ auto BatchRemove(const std::vector& keys, bool force = false) -> std::vector>; + auto BatchRemove(const std::vector& keys, + const std::string& tenant_id, bool force = false) + -> std::vector>; /** * @brief Get the count of keys diff --git a/mooncake-store/include/pyclient.h b/mooncake-store/include/pyclient.h index 4dcca4b5..29852b5f 100644 --- a/mooncake-store/include/pyclient.h +++ b/mooncake-store/include/pyclient.h @@ -215,7 +215,8 @@ class PyClient { const std::string &master_server_addr, const std::shared_ptr &transfer_engine, const std::string &ipc_socket_path, bool enable_ssd_offload = false, - const std::string &ssd_offload_path = "") = 0; + const std::string &ssd_offload_path = "", + const std::string &tenant_id = "default") = 0; virtual int setup_dummy(size_t mem_pool_size, size_t local_buffer_size, const std::string &server_address, diff --git a/mooncake-store/include/real_client.h b/mooncake-store/include/real_client.h index 0f73eddd..00bfd813 100644 --- a/mooncake-store/include/real_client.h +++ b/mooncake-store/include/real_client.h @@ -82,7 +82,8 @@ class RealClient : public PyClient { const std::shared_ptr &transfer_engine = nullptr, const std::string &ipc_socket_path = "", bool enable_ssd_offload = false, - const std::string &ssd_offload_path = ""); + const std::string &ssd_offload_path = "", + const std::string &tenant_id = "default"); int setup_dummy(size_t mem_pool_size, size_t local_buffer_size, const std::string &server_address, @@ -503,7 +504,8 @@ class RealClient : public PyClient { const std::shared_ptr &transfer_engine = nullptr, const std::string &ipc_socket_path = "", int local_rpc_port = 50052, bool enable_ssd_offload = false, bool start_offload_rpc_server = false, - const std::string &ssd_offload_path = ""); + const std::string &ssd_offload_path = "", + const std::string &tenant_id = "default"); // Overload that accepts a configuration dictionary tl::expected setup_internal(const ConfigDict &config); diff --git a/mooncake-store/include/rpc_service.h b/mooncake-store/include/rpc_service.h index 2228d381..b233ed4d 100644 --- a/mooncake-store/include/rpc_service.h +++ b/mooncake-store/include/rpc_service.h @@ -30,13 +30,15 @@ class WrappedMasterService { ~WrappedMasterService(); - tl::expected ExistKey(const std::string& key); + tl::expected ExistKey( + const std::string& key, const std::string& tenant_id = "default"); tl::expected CalcCacheStats(); std::vector> BatchExistKey( - const std::vector& keys); + const std::vector& keys, + const std::string& tenant_id = "default"); tl::expected< std::unordered_map, boost::hash>, @@ -50,73 +52,91 @@ class WrappedMasterService { tl::expected< std::unordered_map>, ErrorCode> - GetReplicaListByRegex(const std::string& str); + GetReplicaListByRegex(const std::string& str, + const std::string& tenant_id = "default"); tl::expected GetReplicaList( - const std::string& key); + const std::string& key, const std::string& tenant_id = "default"); std::vector> - BatchGetReplicaList(const std::vector& keys); + BatchGetReplicaList(const std::vector& keys, + const std::string& tenant_id = "default"); tl::expected, ErrorCode> PutStart( const UUID& client_id, const std::string& key, - const uint64_t slice_length, const ReplicateConfig& config); + const uint64_t slice_length, const ReplicateConfig& config, + const std::string& tenant_id = "default"); tl::expected PutEnd( const UUID& client_id, const std::string& key, - ReplicaType replica_type = ReplicaType::ALL); + ReplicaType replica_type = ReplicaType::ALL, + const std::string& tenant_id = "default"); tl::expected PutRevoke( const UUID& client_id, const std::string& key, - ReplicaType replica_type = ReplicaType::ALL); + ReplicaType replica_type = ReplicaType::ALL, + const std::string& tenant_id = "default"); std::vector, ErrorCode>> BatchPutStart(const UUID& client_id, const std::vector& keys, const std::vector& slice_lengths, - const ReplicateConfig& config); + const ReplicateConfig& config, + const std::string& tenant_id = "default"); std::vector> BatchPutEnd( const UUID& client_id, const std::vector& keys, - ReplicaType replica_type = ReplicaType::ALL); + ReplicaType replica_type = ReplicaType::ALL, + const std::string& tenant_id = "default"); std::vector> BatchPutRevoke( const UUID& client_id, const std::vector& keys, - ReplicaType replica_type = ReplicaType::ALL); + ReplicaType replica_type = ReplicaType::ALL, + const std::string& tenant_id = "default"); tl::expected, ErrorCode> UpsertStart( const UUID& client_id, const std::string& key, - const uint64_t slice_length, const ReplicateConfig& config); + const uint64_t slice_length, const ReplicateConfig& config, + const std::string& tenant_id = "default"); - tl::expected UpsertEnd(const UUID& client_id, - const std::string& key, - ReplicaType replica_type); + tl::expected UpsertEnd( + const UUID& client_id, const std::string& key, + ReplicaType replica_type = ReplicaType::ALL, + const std::string& tenant_id = "default"); - tl::expected UpsertRevoke(const UUID& client_id, - const std::string& key, - ReplicaType replica_type); + tl::expected UpsertRevoke( + const UUID& client_id, const std::string& key, + ReplicaType replica_type = ReplicaType::ALL, + const std::string& tenant_id = "default"); std::vector, ErrorCode>> BatchUpsertStart(const UUID& client_id, const std::vector& keys, const std::vector& slice_lengths, - const ReplicateConfig& config); + const ReplicateConfig& config, + const std::string& tenant_id = "default"); std::vector> BatchUpsertEnd( - const UUID& client_id, const std::vector& keys); + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id = "default"); std::vector> BatchUpsertRevoke( - const UUID& client_id, const std::vector& keys); + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id = "default"); - tl::expected Remove(const std::string& key, - bool force = false); + tl::expected Remove( + const std::string& key, bool force = false, + const std::string& tenant_id = "default"); - tl::expected RemoveByRegex(const std::string& str, - bool force = false); + tl::expected RemoveByRegex( + const std::string& str, bool force = false, + const std::string& tenant_id = "default"); - long RemoveAll(bool force = false); + long RemoveAll(bool force = false, + const std::string& tenant_id = "default"); std::vector> BatchRemove( - const std::vector& keys, bool force = false); + const std::vector& keys, bool force = false, + const std::string& tenant_id = "default"); tl::expected MountSegment(const Segment& segment, const UUID& client_id); diff --git a/mooncake-store/include/types.h b/mooncake-store/include/types.h index f6c326f8..e7a566a8 100644 --- a/mooncake-store/include/types.h +++ b/mooncake-store/include/types.h @@ -215,6 +215,7 @@ constexpr const char* CONFIG_KEY_PROTOCOL = "protocol"; constexpr const char* CONFIG_KEY_RDMA_DEVICES = "rdma_devices"; constexpr const char* CONFIG_KEY_MASTER_SERVER_ADDR = "master_server_addr"; constexpr const char* CONFIG_KEY_IPC_SOCKET_PATH = "ipc_socket_path"; +constexpr const char* CONFIG_KEY_TENANT_ID = "tenant_id"; // Store client configuration defaults static constexpr size_t DEFAULT_GLOBAL_SEGMENT_SIZE = 1024 * 1024 * 16; // 16MB diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index e968e6d3..47ca39e2 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -272,11 +272,13 @@ FinalizeDecision DetermineFinalizeDecision( Client::Client(const std::string& local_hostname, const std::string& metadata_connstring, const std::string& protocol, - const std::map& labels) + const std::map& labels, + const std::string& tenant_id) : client_id_(generate_uuid()), metrics_(ClientMetric::Create(merge_labels(labels))), master_client_(client_id_, - metrics_ ? &metrics_->master_client_metric : nullptr), + metrics_ ? &metrics_->master_client_metric : nullptr, + tenant_id), local_hostname_(local_hostname), metadata_connstring_(metadata_connstring), protocol_(protocol), @@ -832,9 +834,9 @@ std::optional> Client::Create( const std::string& protocol, const std::optional& device_names, const std::string& master_server_entry, const std::shared_ptr& transfer_engine, - std::map labels) { - auto client = std::shared_ptr( - new Client(local_hostname, metadata_connstring, protocol, labels)); + std::map labels, const std::string& tenant_id) { + auto client = std::shared_ptr(new Client( + local_hostname, metadata_connstring, protocol, labels, tenant_id)); ErrorCode err = client->ConnectToMaster(master_server_entry); if (err != ErrorCode::OK) { diff --git a/mooncake-store/src/master_client.cpp b/mooncake-store/src/master_client.cpp index d22a697c..e61f7b23 100644 --- a/mooncake-store/src/master_client.cpp +++ b/mooncake-store/src/master_client.cpp @@ -445,7 +445,8 @@ tl::expected MasterClient::ExistKey( ScopedVLogTimer timer(1, "MasterClient::ExistKey"); timer.LogRequest("object_key=", object_key); - auto result = invoke_rpc<&WrappedMasterService::ExistKey, bool>(object_key); + auto result = invoke_rpc<&WrappedMasterService::ExistKey, bool>(object_key, + tenant_id_); timer.LogResponseExpected(result); return result; } @@ -456,7 +457,7 @@ std::vector> MasterClient::BatchExistKey( timer.LogRequest("keys_count=", object_keys.size()); auto result = invoke_batch_rpc<&WrappedMasterService::BatchExistKey, bool>( - object_keys.size(), object_keys); + object_keys.size(), object_keys, tenant_id_); timer.LogResponse("result=", result.size(), " keys"); return result; } @@ -506,7 +507,8 @@ MasterClient::GetReplicaListByRegex(const std::string& str) { auto result = invoke_rpc< &WrappedMasterService::GetReplicaListByRegex, - std::unordered_map>>(str); + std::unordered_map>>( + str, tenant_id_); timer.LogResponseExpected(result); return result; @@ -518,7 +520,7 @@ tl::expected MasterClient::GetReplicaList( timer.LogRequest("object_key=", object_key); auto result = invoke_rpc<&WrappedMasterService::GetReplicaList, - GetReplicaListResponse>(object_key); + GetReplicaListResponse>(object_key, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -529,8 +531,8 @@ MasterClient::BatchGetReplicaList(const std::vector& object_keys) { timer.LogRequest("keys_count=", object_keys.size()); auto result = invoke_batch_rpc<&WrappedMasterService::BatchGetReplicaList, - GetReplicaListResponse>(object_keys.size(), - object_keys); + GetReplicaListResponse>( + object_keys.size(), object_keys, tenant_id_); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -549,7 +551,7 @@ MasterClient::PutStart(const std::string& key, auto result = invoke_rpc<&WrappedMasterService::PutStart, std::vector>( - client_id_, key, total_slice_length, config); + client_id_, key, total_slice_length, config, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -574,7 +576,7 @@ MasterClient::BatchPutStart( auto result = invoke_batch_rpc<&WrappedMasterService::BatchPutStart, std::vector>( - keys.size(), client_id_, keys, total_slice_lengths, config); + keys.size(), client_id_, keys, total_slice_lengths, config, tenant_id_); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -585,7 +587,7 @@ tl::expected MasterClient::PutEnd(const std::string& key, timer.LogRequest("key=", key); auto result = invoke_rpc<&WrappedMasterService::PutEnd, void>( - client_id_, key, replica_type); + client_id_, key, replica_type, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -596,7 +598,7 @@ std::vector> MasterClient::BatchPutEnd( timer.LogRequest("keys_count=", keys.size()); auto result = invoke_batch_rpc<&WrappedMasterService::BatchPutEnd, void>( - keys.size(), client_id_, keys, replica_type); + keys.size(), client_id_, keys, replica_type, tenant_id_); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -607,7 +609,7 @@ tl::expected MasterClient::PutRevoke( timer.LogRequest("key=", key); auto result = invoke_rpc<&WrappedMasterService::PutRevoke, void>( - client_id_, key, replica_type); + client_id_, key, replica_type, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -618,7 +620,7 @@ std::vector> MasterClient::BatchPutRevoke( timer.LogRequest("keys_count=", keys.size()); auto result = invoke_batch_rpc<&WrappedMasterService::BatchPutRevoke, void>( - keys.size(), client_id_, keys, replica_type); + keys.size(), client_id_, keys, replica_type, tenant_id_); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -637,7 +639,7 @@ MasterClient::UpsertStart(const std::string& key, auto result = invoke_rpc<&WrappedMasterService::UpsertStart, std::vector>( - client_id_, key, total_slice_length, config); + client_id_, key, total_slice_length, config, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -662,7 +664,7 @@ MasterClient::BatchUpsertStart( auto result = invoke_batch_rpc<&WrappedMasterService::BatchUpsertStart, std::vector>( - keys.size(), client_id_, keys, total_slice_lengths, config); + keys.size(), client_id_, keys, total_slice_lengths, config, tenant_id_); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -673,7 +675,7 @@ tl::expected MasterClient::UpsertEnd( timer.LogRequest("key=", key); auto result = invoke_rpc<&WrappedMasterService::UpsertEnd, void>( - client_id_, key, replica_type); + client_id_, key, replica_type, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -684,7 +686,7 @@ std::vector> MasterClient::BatchUpsertEnd( timer.LogRequest("keys_count=", keys.size()); auto result = invoke_batch_rpc<&WrappedMasterService::BatchUpsertEnd, void>( - keys.size(), client_id_, keys); + keys.size(), client_id_, keys, tenant_id_); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -695,7 +697,7 @@ tl::expected MasterClient::UpsertRevoke( timer.LogRequest("key=", key); auto result = invoke_rpc<&WrappedMasterService::UpsertRevoke, void>( - client_id_, key, replica_type); + client_id_, key, replica_type, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -707,7 +709,7 @@ std::vector> MasterClient::BatchUpsertRevoke( auto result = invoke_batch_rpc<&WrappedMasterService::BatchUpsertRevoke, void>( - keys.size(), client_id_, keys); + keys.size(), client_id_, keys, tenant_id_); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -717,7 +719,8 @@ tl::expected MasterClient::Remove(const std::string& key, ScopedVLogTimer timer(1, "MasterClient::Remove"); timer.LogRequest("key=", key, ", force=", force); - auto result = invoke_rpc<&WrappedMasterService::Remove, void>(key, force); + auto result = + invoke_rpc<&WrappedMasterService::Remove, void>(key, force, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -727,8 +730,8 @@ tl::expected MasterClient::RemoveByRegex( ScopedVLogTimer timer(1, "MasterClient::RemoveByRegex"); timer.LogRequest("key=", str, ", force=", force); - auto result = - invoke_rpc<&WrappedMasterService::RemoveByRegex, long>(str, force); + auto result = invoke_rpc<&WrappedMasterService::RemoveByRegex, long>( + str, force, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -737,7 +740,8 @@ tl::expected MasterClient::RemoveAll(bool force) { ScopedVLogTimer timer(1, "MasterClient::RemoveAll"); timer.LogRequest("action=remove_all_objects, force=", force); - auto result = invoke_rpc<&WrappedMasterService::RemoveAll, long>(force); + auto result = + invoke_rpc<&WrappedMasterService::RemoveAll, long>(force, tenant_id_); timer.LogResponseExpected(result); return result; } @@ -748,7 +752,7 @@ std::vector> MasterClient::BatchRemove( timer.LogRequest("keys_count=", keys.size(), ", force=", force); auto result = invoke_batch_rpc<&WrappedMasterService::BatchRemove, void>( - keys.size(), keys, force); + keys.size(), keys, force, tenant_id_); timer.LogResponse("result=", result.size(), " operations"); return result; } diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index aa435920..77f734d9 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -1773,8 +1773,15 @@ auto MasterService::AddReplica(const UUID& client_id, const std::string& key, auto MasterService::PutRevoke(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected { + return PutRevoke(client_id, key, "default", replica_type); +} + +auto MasterService::PutRevoke(const UUID& client_id, const std::string& key, + const std::string& tenant_id, + ReplicaType replica_type) + -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, MakeObjectIdentity(key, tenant_id)); if (!accessor.Exists()) { LOG(INFO) << "key=" << key << ", info=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -1830,10 +1837,16 @@ auto MasterService::PutRevoke(const UUID& client_id, const std::string& key, std::vector> MasterService::BatchPutEnd( const UUID& client_id, const std::vector& keys, ReplicaType replica_type) { + return BatchPutEnd(client_id, keys, "default", replica_type); +} + +std::vector> MasterService::BatchPutEnd( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id, ReplicaType replica_type) { std::vector> results; results.reserve(keys.size()); for (const auto& key : keys) { - results.emplace_back(PutEnd(client_id, key, replica_type)); + results.emplace_back(PutEnd(client_id, key, tenant_id, replica_type)); } return results; } @@ -1841,10 +1854,17 @@ std::vector> MasterService::BatchPutEnd( std::vector> MasterService::BatchPutRevoke( const UUID& client_id, const std::vector& keys, ReplicaType replica_type) { + return BatchPutRevoke(client_id, keys, "default", replica_type); +} + +std::vector> MasterService::BatchPutRevoke( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id, ReplicaType replica_type) { std::vector> results; results.reserve(keys.size()); for (const auto& key : keys) { - results.emplace_back(PutRevoke(client_id, key, replica_type)); + results.emplace_back( + PutRevoke(client_id, key, tenant_id, replica_type)); } return results; } @@ -2128,13 +2148,27 @@ auto MasterService::UpsertStart(const UUID& client_id, const std::string& key, auto MasterService::UpsertEnd(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected { - return PutEnd(client_id, key, replica_type); + return UpsertEnd(client_id, key, "default", replica_type); +} + +auto MasterService::UpsertEnd(const UUID& client_id, const std::string& key, + const std::string& tenant_id, + ReplicaType replica_type) + -> tl::expected { + return PutEnd(client_id, key, tenant_id, replica_type); } auto MasterService::UpsertRevoke(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected { - return PutRevoke(client_id, key, replica_type); + return UpsertRevoke(client_id, key, "default", replica_type); +} + +auto MasterService::UpsertRevoke(const UUID& client_id, const std::string& key, + const std::string& tenant_id, + ReplicaType replica_type) + -> tl::expected { + return PutRevoke(client_id, key, tenant_id, replica_type); } std::vector, ErrorCode>> @@ -2142,6 +2176,15 @@ MasterService::BatchUpsertStart(const UUID& client_id, const std::vector& keys, const std::vector& slice_lengths, const ReplicateConfig& config) { + return BatchUpsertStart(client_id, keys, "default", slice_lengths, config); +} + +std::vector, ErrorCode>> +MasterService::BatchUpsertStart(const UUID& client_id, + const std::vector& keys, + const std::string& tenant_id, + const std::vector& slice_lengths, + const ReplicateConfig& config) { if (keys.size() != slice_lengths.size()) { LOG(ERROR) << "BatchUpsertStart: keys.size()=" << keys.size() << " != slice_lengths.size()=" << slice_lengths.size(); @@ -2163,20 +2206,32 @@ MasterService::BatchUpsertStart(const UUID& client_id, results.reserve(keys.size()); for (size_t i = 0; i < keys.size(); ++i) { auto key_config = config.ForSingleKey(i); - results.emplace_back( - UpsertStart(client_id, keys[i], slice_lengths[i], key_config)); + results.emplace_back(UpsertStart(client_id, keys[i], tenant_id, + slice_lengths[i], key_config)); } return results; } std::vector> MasterService::BatchUpsertEnd( const UUID& client_id, const std::vector& keys) { - return BatchPutEnd(client_id, keys); + return BatchUpsertEnd(client_id, keys, "default"); +} + +std::vector> MasterService::BatchUpsertEnd( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id) { + return BatchPutEnd(client_id, keys, tenant_id); } std::vector> MasterService::BatchUpsertRevoke( const UUID& client_id, const std::vector& keys) { - return BatchPutRevoke(client_id, keys); + return BatchUpsertRevoke(client_id, keys, "default"); +} + +std::vector> MasterService::BatchUpsertRevoke( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id) { + return BatchPutRevoke(client_id, keys, tenant_id); } auto MasterService::EvictDiskReplica(const UUID& client_id, @@ -2802,8 +2857,6 @@ auto MasterService::RemoveByRegex(const std::string& regex_pattern, long MasterService::RemoveAll(bool force) { long removed_count = 0; uint64_t total_freed_size = 0; - // Store the current time to avoid repeatedly - // calling std::chrono::steady_clock::now() std::shared_lock shared_lock(snapshot_mutex_); auto now = std::chrono::system_clock::now(); @@ -2841,10 +2894,60 @@ long MasterService::RemoveAll(bool force) { return removed_count; } +long MasterService::RemoveAll(const std::string& tenant_id, bool force) { + long removed_count = 0; + uint64_t total_freed_size = 0; + // Store the current time to avoid repeatedly + // calling std::chrono::steady_clock::now() + std::shared_lock shared_lock(snapshot_mutex_); + auto now = std::chrono::system_clock::now(); + const auto normalized_tenant = NormalizeTenantId(tenant_id); + + for (size_t i = 0; i < kNumShards; i++) { + MetadataShardAccessorRW shard(this, i); + auto tenant_it = shard->tenants.find(normalized_tenant); + if (tenant_it == shard->tenants.end()) { + continue; + } + auto& tenant_state = tenant_it->second; + auto it = tenant_state.metadata.begin(); + while (it != tenant_state.metadata.end()) { + if ((force || it->second.IsLeaseExpired(now)) && + it->second.AllReplicas(&Replica::fn_is_completed) && + !tenant_state.replication_tasks.contains(it->first)) { + auto mem_rep_count = + it->second.CountReplicas(&Replica::fn_is_memory_replica); + total_freed_size += it->second.size * mem_rep_count; + ErasePromotionTaskIfPresent(tenant_state, it->first); + it = EraseMetadata(tenant_state, it, normalized_tenant); + removed_count++; + } else { + ++it; + } + } + if (tenant_state.Empty()) { + shard->tenants.erase(tenant_it); + } + } + + VLOG(1) << "action=remove_all_objects" + << ", tenant_id=" << normalized_tenant + << ", removed_count=" << removed_count + << ", total_freed_size=" << total_freed_size; + return removed_count; +} + auto MasterService::BatchRemove(const std::vector& keys, bool force) -> std::vector> { + return BatchRemove(keys, "default", force); +} + +auto MasterService::BatchRemove(const std::vector& keys, + const std::string& tenant_id, bool force) + -> std::vector> { std::vector> results(keys.size()); + const auto normalized_tenant = NormalizeTenantId(tenant_id); // Group keys by shard to reduce lock contention std::unordered_map& keys, std::min(keys.size(), static_cast(kNumShards))); for (size_t i = 0; i < keys.size(); ++i) { - size_t shard_idx = getMetadataShardIndex(keys[i]); + size_t shard_idx = getMetadataShardIndex(normalized_tenant, keys[i]); keys_by_shard[shard_idx].emplace_back(i, &keys[i]); } @@ -2869,7 +2972,7 @@ auto MasterService::BatchRemove(const std::vector& keys, for (const auto& [original_idx, key_ptr] : key_group) { const std::string& key = *key_ptr; - auto tenant_it = shard->tenants.find("default"); + auto tenant_it = shard->tenants.find(normalized_tenant); if (tenant_it == shard->tenants.end()) { VLOG(1) << "key=" << key << ", error=object_not_found"; results[original_idx] = @@ -2892,7 +2995,10 @@ auto MasterService::BatchRemove(const std::vector& keys, tenant_state.replication_tasks.erase(key); tenant_state.offloading_tasks.erase(key); ErasePromotionTaskIfPresent(tenant_state, key); - EraseMetadata(tenant_state, it, "default"); + EraseMetadata(tenant_state, it, normalized_tenant); + if (tenant_state.Empty()) { + shard->tenants.erase(tenant_it); + } results[original_idx] = tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); continue; @@ -2929,7 +3035,7 @@ auto MasterService::BatchRemove(const std::vector& keys, // Remove object metadata ErasePromotionTaskIfPresent(tenant_state, key); - EraseMetadata(tenant_state, it, "default"); + EraseMetadata(tenant_state, it, normalized_tenant); if (tenant_state.Empty()) { shard->tenants.erase(tenant_it); } diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index dbc126da..9b41e02a 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -633,7 +633,7 @@ tl::expected RealClient::setup_internal( const std::shared_ptr &transfer_engine, const std::string &ipc_socket_path, int local_rpc_port, bool enable_ssd_offload, bool start_offload_rpc_server, - const std::string &ssd_offload_path) { + const std::string &ssd_offload_path, const std::string &tenant_id) { this->protocol = protocol; this->ipc_socket_path_ = ipc_socket_path; const bool should_use_hugepage = @@ -677,7 +677,8 @@ tl::expected RealClient::setup_internal( getHostNameWithoutPort(hostname), local_rpc_port); auto client_opt = mooncake::Client::Create( this->local_hostname, metadata_server, protocol, device_name, - master_server_addr, transfer_engine, {{"client_mode", "real"}}); + master_server_addr, transfer_engine, {{"client_mode", "real"}}, + tenant_id); if (!client_opt) { LOG(ERROR) << "Failed to create client"; return tl::unexpected(ErrorCode::INVALID_PARAMS); @@ -712,7 +713,8 @@ tl::expected RealClient::setup_internal( buildHostNameWithPort(hostname, local_rpc_port); auto client_opt = mooncake::Client::Create( this->local_hostname, metadata_server, protocol, device_name, - master_server_addr, transfer_engine, {{"client_mode", "real"}}); + master_server_addr, transfer_engine, {{"client_mode", "real"}}, + tenant_id); if (client_opt) { client_ = *client_opt; success = true; @@ -939,11 +941,12 @@ int RealClient::setup_real( const std::string &master_server_addr, const std::shared_ptr &transfer_engine, const std::string &ipc_socket_path, bool enable_ssd_offload, - const std::string &ssd_offload_path) { + const std::string &ssd_offload_path, const std::string &tenant_id) { return to_py_ret(setup_internal( local_hostname, metadata_server, global_segment_size, local_buffer_size, protocol, rdma_devices, master_server_addr, transfer_engine, - ipc_socket_path, 50052, enable_ssd_offload, true, ssd_offload_path)); + ipc_socket_path, 50052, enable_ssd_offload, true, ssd_offload_path, + tenant_id)); } namespace { @@ -1034,6 +1037,7 @@ tl::expected RealClient::setup_internal( } std::string ssd_offload_path = get_config(config, "ssd_offload_path"); + std::string tenant_id = get_config(config, CONFIG_KEY_TENANT_ID, "default"); std::string enable_ssd_offload_str = get_config(config, "enable_ssd_offload", "false"); @@ -1043,10 +1047,10 @@ tl::expected RealClient::setup_internal( bool enable_ssd_offload = (enable_ssd_offload_str == "true" || enable_ssd_offload_str == "1"); - return setup_internal(local_hostname, metadata_server, global_segment_size, - local_buffer_size, protocol, rdma_devices, - master_server_addr, nullptr, ipc_socket_path, 50052, - enable_ssd_offload, true, ssd_offload_path); + return setup_internal( + local_hostname, metadata_server, global_segment_size, local_buffer_size, + protocol, rdma_devices, master_server_addr, nullptr, ipc_socket_path, + 50052, enable_ssd_offload, true, ssd_offload_path, tenant_id); } tl::expected RealClient::initAll_internal( diff --git a/mooncake-store/src/rpc_service.cpp b/mooncake-store/src/rpc_service.cpp index 5ad88374..ed94a3f3 100644 --- a/mooncake-store/src/rpc_service.cpp +++ b/mooncake-store/src/rpc_service.cpp @@ -434,7 +434,8 @@ void MasterAdminServer::InitHttpServer() { } auto key = req.get_query_value("key"); - auto get_result = service->GetReplicaList(std::string(key)); + auto get_result = + service->GetReplicaList(std::string(key), "default"); resp.add_header("Content-Type", "text/plain; version=0.0.4"); if (get_result) { std::string ss; @@ -696,7 +697,7 @@ void MasterAdminServer::InitHttpServer() { return; } - auto results = service->BatchGetReplicaList(keys); + auto results = service->BatchGetReplicaList(keys, "default"); const size_t n = std::min(keys.size(), results.size()); std::string body; body.reserve(n * 512); @@ -754,22 +755,26 @@ WrappedMasterService::CalcCacheStats() { } tl::expected WrappedMasterService::ExistKey( - const std::string& key) { + const std::string& key, const std::string& tenant_id) { return execute_rpc( - "ExistKey", [&] { return master_service_.ExistKey(key); }, + "ExistKey", [&] { return master_service_.ExistKey(key, tenant_id); }, [&](auto& timer) { timer.LogRequest("key=", key); }, [] { MasterMetricManager::instance().inc_exist_key_requests(); }, [] { MasterMetricManager::instance().inc_exist_key_failures(); }); } std::vector> WrappedMasterService::BatchExistKey( - const std::vector& keys) { + const std::vector& keys, const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchExistKey"); const size_t total_keys = keys.size(); timer.LogRequest("keys_count=", total_keys); MasterMetricManager::instance().inc_batch_exist_key_requests(total_keys); - auto result = master_service_.BatchExistKey(keys); + std::vector> result; + result.reserve(keys.size()); + for (const auto& key : keys) { + result.emplace_back(master_service_.ExistKey(key, tenant_id)); + } size_t failure_count = 0; for (size_t i = 0; i < result.size(); ++i) { @@ -876,10 +881,11 @@ WrappedMasterService::BatchReplicaClear( tl::expected>, ErrorCode> -WrappedMasterService::GetReplicaListByRegex(const std::string& str) { +WrappedMasterService::GetReplicaListByRegex(const std::string& str, + const std::string& tenant_id) { return execute_rpc( "GetReplicaListByRegex", - [&] { return master_service_.GetReplicaListByRegex(str); }, + [&] { return master_service_.GetReplicaListByRegex(str, tenant_id); }, [&](auto& timer) { timer.LogRequest("Regex=", str); }, [] { MasterMetricManager::instance() @@ -892,9 +898,11 @@ WrappedMasterService::GetReplicaListByRegex(const std::string& str) { } tl::expected -WrappedMasterService::GetReplicaList(const std::string& key) { +WrappedMasterService::GetReplicaList(const std::string& key, + const std::string& tenant_id) { return execute_rpc( - "GetReplicaList", [&] { return master_service_.GetReplicaList(key); }, + "GetReplicaList", + [&] { return master_service_.GetReplicaList(key, tenant_id); }, [&](auto& timer) { timer.LogRequest("key=", key); }, [] { MasterMetricManager::instance().inc_get_replica_list_requests(); }, [] { @@ -903,8 +911,8 @@ WrappedMasterService::GetReplicaList(const std::string& key) { } std::vector> -WrappedMasterService::BatchGetReplicaList( - const std::vector& keys) { +WrappedMasterService::BatchGetReplicaList(const std::vector& keys, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchGetReplicaList"); const size_t total_keys = keys.size(); timer.LogRequest("keys_count=", total_keys); @@ -915,7 +923,7 @@ WrappedMasterService::BatchGetReplicaList( results.reserve(keys.size()); for (const auto& key : keys) { - results.emplace_back(master_service_.GetReplicaList(key)); + results.emplace_back(master_service_.GetReplicaList(key, tenant_id)); } size_t failure_count = 0; @@ -951,12 +959,13 @@ WrappedMasterService::BatchGetReplicaList( tl::expected, ErrorCode> WrappedMasterService::PutStart(const UUID& client_id, const std::string& key, const uint64_t slice_length, - const ReplicateConfig& config) { + const ReplicateConfig& config, + const std::string& tenant_id) { return execute_rpc( "PutStart", [&] { - return master_service_.PutStart(client_id, key, slice_length, - config); + return master_service_.PutStart(client_id, key, tenant_id, + slice_length, config); }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, @@ -967,10 +976,14 @@ WrappedMasterService::PutStart(const UUID& client_id, const std::string& key, } tl::expected WrappedMasterService::PutEnd( - const UUID& client_id, const std::string& key, ReplicaType replica_type) { + const UUID& client_id, const std::string& key, ReplicaType replica_type, + const std::string& tenant_id) { return execute_rpc( "PutEnd", - [&] { return master_service_.PutEnd(client_id, key, replica_type); }, + [&] { + return master_service_.PutEnd(client_id, key, tenant_id, + replica_type); + }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, ", replica_type=", replica_type); @@ -980,10 +993,14 @@ tl::expected WrappedMasterService::PutEnd( } tl::expected WrappedMasterService::PutRevoke( - const UUID& client_id, const std::string& key, ReplicaType replica_type) { + const UUID& client_id, const std::string& key, ReplicaType replica_type, + const std::string& tenant_id) { return execute_rpc( "PutRevoke", - [&] { return master_service_.PutRevoke(client_id, key, replica_type); }, + [&] { + return master_service_.PutRevoke(client_id, key, tenant_id, + replica_type); + }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, ", replica_type=", replica_type); @@ -996,7 +1013,8 @@ std::vector, ErrorCode>> WrappedMasterService::BatchPutStart(const UUID& client_id, const std::vector& keys, const std::vector& slice_lengths, - const ReplicateConfig& config) { + const ReplicateConfig& config, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchPutStart"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys); @@ -1023,7 +1041,7 @@ WrappedMasterService::BatchPutStart(const UUID& client_id, for (size_t i = 0; i < keys.size(); ++i) { auto key_config = new_config.ForSingleKey(i); auto result = master_service_.PutStart( - client_id, keys[i], slice_lengths[i], key_config); + client_id, keys[i], tenant_id, slice_lengths[i], key_config); results.emplace_back(result); if ((i == 0) && result.has_value()) { std::string preferred_segment; @@ -1045,7 +1063,7 @@ WrappedMasterService::BatchPutStart(const UUID& client_id, for (size_t i = 0; i < keys.size(); ++i) { auto key_config = config.ForSingleKey(i); results.emplace_back(master_service_.PutStart( - client_id, keys[i], slice_lengths[i], key_config)); + client_id, keys[i], tenant_id, slice_lengths[i], key_config)); } } @@ -1088,7 +1106,7 @@ WrappedMasterService::BatchPutStart(const UUID& client_id, std::vector> WrappedMasterService::BatchPutEnd( const UUID& client_id, const std::vector& keys, - ReplicaType replica_type) { + ReplicaType replica_type, const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchPutEnd"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys); @@ -1099,7 +1117,7 @@ std::vector> WrappedMasterService::BatchPutEnd( for (const auto& key : keys) { results.emplace_back( - master_service_.PutEnd(client_id, key, replica_type)); + master_service_.PutEnd(client_id, key, tenant_id, replica_type)); } size_t failure_count = 0; @@ -1128,7 +1146,7 @@ std::vector> WrappedMasterService::BatchPutEnd( std::vector> WrappedMasterService::BatchPutRevoke( const UUID& client_id, const std::vector& keys, - ReplicaType replica_type) { + ReplicaType replica_type, const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchPutRevoke"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys); @@ -1139,7 +1157,7 @@ std::vector> WrappedMasterService::BatchPutRevoke( for (const auto& key : keys) { results.emplace_back( - master_service_.PutRevoke(client_id, key, replica_type)); + master_service_.PutRevoke(client_id, key, tenant_id, replica_type)); } size_t failure_count = 0; @@ -1169,12 +1187,13 @@ std::vector> WrappedMasterService::BatchPutRevoke( tl::expected, ErrorCode> WrappedMasterService::UpsertStart(const UUID& client_id, const std::string& key, const uint64_t slice_length, - const ReplicateConfig& config) { + const ReplicateConfig& config, + const std::string& tenant_id) { return execute_rpc( "UpsertStart", [&] { - return master_service_.UpsertStart(client_id, key, slice_length, - config); + return master_service_.UpsertStart(client_id, key, tenant_id, + slice_length, config); }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, @@ -1185,10 +1204,14 @@ WrappedMasterService::UpsertStart(const UUID& client_id, const std::string& key, } tl::expected WrappedMasterService::UpsertEnd( - const UUID& client_id, const std::string& key, ReplicaType replica_type) { + const UUID& client_id, const std::string& key, ReplicaType replica_type, + const std::string& tenant_id) { return execute_rpc( "UpsertEnd", - [&] { return master_service_.UpsertEnd(client_id, key, replica_type); }, + [&] { + return master_service_.UpsertEnd(client_id, key, tenant_id, + replica_type); + }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, ", replica_type=", replica_type); @@ -1198,11 +1221,13 @@ tl::expected WrappedMasterService::UpsertEnd( } tl::expected WrappedMasterService::UpsertRevoke( - const UUID& client_id, const std::string& key, ReplicaType replica_type) { + const UUID& client_id, const std::string& key, ReplicaType replica_type, + const std::string& tenant_id) { return execute_rpc( "UpsertRevoke", [&] { - return master_service_.UpsertRevoke(client_id, key, replica_type); + return master_service_.UpsertRevoke(client_id, key, tenant_id, + replica_type); }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, @@ -1215,13 +1240,14 @@ tl::expected WrappedMasterService::UpsertRevoke( std::vector, ErrorCode>> WrappedMasterService::BatchUpsertStart( const UUID& client_id, const std::vector& keys, - const std::vector& slice_lengths, const ReplicateConfig& config) { + const std::vector& slice_lengths, const ReplicateConfig& config, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchUpsertStart"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys); MasterMetricManager::instance().inc_batch_put_start_requests(total_keys); - auto results = master_service_.BatchUpsertStart(client_id, keys, + auto results = master_service_.BatchUpsertStart(client_id, keys, tenant_id, slice_lengths, config); size_t failure_count = 0; @@ -1249,13 +1275,14 @@ WrappedMasterService::BatchUpsertStart( } std::vector> WrappedMasterService::BatchUpsertEnd( - const UUID& client_id, const std::vector& keys) { + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchUpsertEnd"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys); MasterMetricManager::instance().inc_batch_put_end_requests(total_keys); - auto results = master_service_.BatchUpsertEnd(client_id, keys); + auto results = master_service_.BatchUpsertEnd(client_id, keys, tenant_id); size_t failure_count = 0; for (size_t i = 0; i < results.size(); ++i) { @@ -1283,13 +1310,15 @@ std::vector> WrappedMasterService::BatchUpsertEnd( std::vector> WrappedMasterService::BatchUpsertRevoke(const UUID& client_id, - const std::vector& keys) { + const std::vector& keys, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchUpsertRevoke"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys); MasterMetricManager::instance().inc_batch_put_revoke_requests(total_keys); - auto results = master_service_.BatchUpsertRevoke(client_id, keys); + auto results = + master_service_.BatchUpsertRevoke(client_id, keys, tenant_id); size_t failure_count = 0; for (size_t i = 0; i < results.size(); ++i) { @@ -1316,19 +1345,19 @@ WrappedMasterService::BatchUpsertRevoke(const UUID& client_id, } tl::expected WrappedMasterService::Remove( - const std::string& key, bool force) { + const std::string& key, bool force, const std::string& tenant_id) { return execute_rpc( - "Remove", [&] { return master_service_.Remove(key, force); }, + "Remove", [&] { return master_service_.Remove(key, tenant_id, force); }, [&](auto& timer) { timer.LogRequest("key=", key, ", force=", force); }, [] { MasterMetricManager::instance().inc_remove_requests(); }, [] { MasterMetricManager::instance().inc_remove_failures(); }); } tl::expected WrappedMasterService::RemoveByRegex( - const std::string& str, bool force) { + const std::string& str, bool force, const std::string& tenant_id) { return execute_rpc( "RemoveByRegex", - [&] { return master_service_.RemoveByRegex(str, force); }, + [&] { return master_service_.RemoveByRegex(str, tenant_id, force); }, [&](auto& timer) { timer.LogRequest("regex=", str, ", force=", force); }, @@ -1336,23 +1365,24 @@ tl::expected WrappedMasterService::RemoveByRegex( [] { MasterMetricManager::instance().inc_remove_by_regex_failures(); }); } -long WrappedMasterService::RemoveAll(bool force) { +long WrappedMasterService::RemoveAll(bool force, const std::string& tenant_id) { ScopedVLogTimer timer(1, "RemoveAll"); timer.LogRequest("action=remove_all_objects, force=", force); MasterMetricManager::instance().inc_remove_all_requests(); - long result = master_service_.RemoveAll(force); + long result = master_service_.RemoveAll(tenant_id, force); timer.LogResponse("items_removed=", result); return result; } std::vector> WrappedMasterService::BatchRemove( - const std::vector& keys, bool force) { + const std::vector& keys, bool force, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "BatchRemove"); const size_t total_keys = keys.size(); timer.LogRequest("keys_count=", total_keys, ", force=", force); MasterMetricManager::instance().inc_remove_requests(total_keys); - auto results = master_service_.BatchRemove(keys, force); + auto results = master_service_.BatchRemove(keys, tenant_id, force); size_t failure_count = 0; for (const auto& result : results) { diff --git a/mooncake-store/tests/master_service_test.cpp b/mooncake-store/tests/master_service_test.cpp index 7e296ec5..cff39b0a 100644 --- a/mooncake-store/tests/master_service_test.cpp +++ b/mooncake-store/tests/master_service_test.cpp @@ -1404,6 +1404,131 @@ TEST_F(MasterServiceTest, RegexOperationsAreTenantScoped) { EXPECT_TRUE(service_->GetReplicaList(key, tenant_b).has_value()); } +TEST_F(MasterServiceTest, TenantBatchUpsertAndRevokeAreScoped) { + auto svc = std::make_unique(); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*svc); + const UUID client_id = generate_uuid(); + + const std::vector keys = {"tenant_batch_upsert_key_a", + "tenant_batch_upsert_key_b"}; + const std::vector sizes = {1024, 2048}; + const std::string tenant_a = "tenant_batch_upsert_a"; + const std::string tenant_b = "tenant_batch_upsert_b"; + + ReplicateConfig config; + config.replica_num = 1; + + auto tenant_a_results = + svc->BatchUpsertStart(client_id, keys, tenant_a, sizes, config); + ASSERT_EQ(tenant_a_results.size(), keys.size()); + for (const auto& result : tenant_a_results) { + ASSERT_TRUE(result.has_value()); + } + auto tenant_a_end = svc->BatchUpsertEnd(client_id, keys, tenant_a); + ASSERT_EQ(tenant_a_end.size(), keys.size()); + for (const auto& result : tenant_a_end) { + ASSERT_TRUE(result.has_value()); + } + + auto tenant_b_results = + svc->BatchUpsertStart(client_id, keys, tenant_b, sizes, config); + ASSERT_EQ(tenant_b_results.size(), keys.size()); + for (const auto& result : tenant_b_results) { + ASSERT_TRUE(result.has_value()); + } + auto tenant_b_end = svc->BatchUpsertEnd(client_id, keys, tenant_b); + ASSERT_EQ(tenant_b_end.size(), keys.size()); + for (const auto& result : tenant_b_end) { + ASSERT_TRUE(result.has_value()); + } + + for (const auto& key : keys) { + EXPECT_FALSE(svc->GetReplicaList(key).has_value()); + EXPECT_TRUE(svc->GetReplicaList(key, tenant_a).has_value()); + EXPECT_TRUE(svc->GetReplicaList(key, tenant_b).has_value()); + } + + const std::string revoke_key = "tenant_batch_upsert_revoke_key"; + auto revoke_start = + svc->UpsertStart(client_id, revoke_key, tenant_a, 1024, config); + ASSERT_TRUE(revoke_start.has_value()); + ASSERT_TRUE( + svc->UpsertRevoke(client_id, revoke_key, tenant_a, ReplicaType::MEMORY) + .has_value()); + EXPECT_FALSE(svc->GetReplicaList(revoke_key, tenant_a).has_value()); +} + +TEST_F(MasterServiceTest, TenantBatchRemoveAndRemoveAllAreScoped) { + auto svc = std::make_unique(); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*svc); + const UUID client_id = generate_uuid(); + + const std::string shared_key = "tenant_batch_remove_shared_key"; + const std::string tenant_a = "tenant_batch_remove_a"; + const std::string tenant_b = "tenant_batch_remove_b"; + + ReplicateConfig config; + config.replica_num = 1; + + ASSERT_TRUE(svc->PutStart(client_id, shared_key, 1024, config).has_value()); + ASSERT_TRUE( + svc->PutEnd(client_id, shared_key, ReplicaType::MEMORY).has_value()); + ASSERT_TRUE(svc->PutStart(client_id, shared_key, tenant_a, 1024, config) + .has_value()); + ASSERT_TRUE( + svc->PutEnd(client_id, shared_key, tenant_a, ReplicaType::MEMORY) + .has_value()); + ASSERT_TRUE(svc->PutStart(client_id, shared_key, tenant_b, 1024, config) + .has_value()); + ASSERT_TRUE( + svc->PutEnd(client_id, shared_key, tenant_b, ReplicaType::MEMORY) + .has_value()); + + auto remove_a = svc->BatchRemove({shared_key}, tenant_a, /*force=*/true); + ASSERT_EQ(remove_a.size(), 1u); + ASSERT_TRUE(remove_a[0].has_value()); + EXPECT_FALSE(svc->GetReplicaList(shared_key, tenant_a).has_value()); + EXPECT_TRUE(svc->GetReplicaList(shared_key).has_value()); + EXPECT_TRUE(svc->GetReplicaList(shared_key, tenant_b).has_value()); + + EXPECT_EQ(svc->RemoveAll(tenant_b, /*force=*/true), 1); + EXPECT_FALSE(svc->GetReplicaList(shared_key, tenant_b).has_value()); + EXPECT_TRUE(svc->GetReplicaList(shared_key).has_value()); + + EXPECT_EQ(svc->RemoveAll(/*force=*/true), 1); + EXPECT_FALSE(svc->GetReplicaList(shared_key).has_value()); +} + +TEST_F(MasterServiceTest, LegacyRemoveAllRemovesAllTenants) { + auto svc = std::make_unique(); + [[maybe_unused]] const auto context = PrepareSimpleSegment(*svc); + const UUID client_id = generate_uuid(); + + const std::string key = "legacy_remove_all_shared_key"; + const std::string tenant_a = "legacy_remove_all_a"; + const std::string tenant_b = "legacy_remove_all_b"; + + ReplicateConfig config; + config.replica_num = 1; + + ASSERT_TRUE(svc->PutStart(client_id, key, 1024, config).has_value()); + ASSERT_TRUE(svc->PutEnd(client_id, key, ReplicaType::MEMORY).has_value()); + ASSERT_TRUE( + svc->PutStart(client_id, key, tenant_a, 1024, config).has_value()); + ASSERT_TRUE( + svc->PutEnd(client_id, key, tenant_a, ReplicaType::MEMORY).has_value()); + ASSERT_TRUE( + svc->PutStart(client_id, key, tenant_b, 1024, config).has_value()); + ASSERT_TRUE( + svc->PutEnd(client_id, key, tenant_b, ReplicaType::MEMORY).has_value()); + + EXPECT_EQ(svc->RemoveAll(/*force=*/true), 3); + EXPECT_FALSE(svc->GetReplicaList(key).has_value()); + EXPECT_FALSE(svc->GetReplicaList(key, tenant_a).has_value()); + EXPECT_FALSE(svc->GetReplicaList(key, tenant_b).has_value()); + EXPECT_EQ(svc->RemoveAll(/*force=*/true), 0); +} + TEST_F(MasterServiceTest, PutWithPreferredSegment) { // For backward compatibility, test the deprecated single preferred_segment std::unique_ptr service_(new MasterService()); -- 2.34.1 From 2e4a4fe1cc0063aaf7ac77d836c5caa3c5fb0801 Mon Sep 17 00:00:00 2001 From: Aoi Date: Tue, 2 Jun 2026 14:50:43 +0800 Subject: [PATCH 166/382] [Doc] add vLLM scenario-based landing pages and archive legacy docs (#2262) * [Doc] add vLLM scenario-based landing pages and archive legacy docs * fix(docs): remove duplicate TENT section, fix num_workers indentation and benchmark version - Remove duplicated "TENT Transport Selector" section in tent overview - Fix num_workers indentation to reflect it's a top-level JSON key, not nested under kv_role - Correct benchmark backend from V0 to V1 for vllm-benchmark-results-v1 --- docs/source/design/architecture.md | 2 +- docs/source/design/mooncake-store.md | 4 +- docs/source/getting_started/build.md | 9 +- .../vllm-integration/disagg-prefill-decode.md | 337 +++++++++++++++ .../examples/vllm-integration/index.md | 65 ++- .../vllm-integration/kv-cache-storage.md | 409 ++++++++++++++++++ .../vllm-integration/vllm-integration-v0.2.md | 19 +- .../vllm-integration/vllm-integration-v0.3.md | 74 +++- .../vllm-integration/vllm-integration-v1.0.md | 7 +- .../vllm-mooncakestoreconnector.md | 9 +- docs/source/index.md | 14 + docs/source/performance/vllm/index.md | 18 + 12 files changed, 940 insertions(+), 27 deletions(-) create mode 100644 docs/source/getting_started/examples/vllm-integration/disagg-prefill-decode.md create mode 100644 docs/source/getting_started/examples/vllm-integration/kv-cache-storage.md create mode 100644 docs/source/performance/vllm/index.md diff --git a/docs/source/design/architecture.md b/docs/source/design/architecture.md index a3edfe8d..434def0f 100644 --- a/docs/source/design/architecture.md +++ b/docs/source/design/architecture.md @@ -12,7 +12,7 @@ Mooncake: ## Architectural Overview ![architecture](../image/mooncake-store.png) -- Mooncake provides object-level operations, i.e. `Get/Put/List/Del`, and also supports dynamically configurating replication strategies (`Replicate` operations); +- Mooncake provides object-level operations, i.e. `Get/Put/List/Del`, and also supports dynamically configuring replication strategies (`Replicate` operations); - Mooncake supports zero-copy and multi-NIC data transfer over VRAM/DRAM/NVMe SSD. This feature is supported by Transfer Engine, which has been open-sourced; - **The master node** centrally manages the mappings of objects to VRAM/DRAM/NVM buffers. The master node also drives **managed pool buffer nodes** to achieve data transfer by calling Transfer Engine's APIs; - **Managed pool buffer nodes** mainly provide DRAM space for storing objects. diff --git a/docs/source/design/mooncake-store.md b/docs/source/design/mooncake-store.md index 3e44f270..51e1754c 100644 --- a/docs/source/design/mooncake-store.md +++ b/docs/source/design/mooncake-store.md @@ -6,7 +6,7 @@ Mooncake Store is a high-performance **distributed key-value (KV) cache storage Unlike traditional caching systems such as Redis or Memcached, Mooncake Store is positioned as **a distributed KV cache rather than a generic caching system**. The key difference is that in the latter, the key is derived from the value through hashing, so value is immutable after inserting (although the key/value pair may be evicted). -Mooncake Store provides low-level object storage and management capabilities, including configurable caching and eviction strategies that offers high memory efficiency and is specifically designed to accelerate LLM inference performance. +Mooncake Store provides low-level object storage and management capabilities, including configurable caching and eviction strategies that offer high memory efficiency and is specifically designed to accelerate LLM inference performance. Key features of Mooncake Store include: - **Object-level storage operations**: Mooncake Store provides simple and easy-to-use object-level APIs, including `Put`, `Get`, and `Remove` operations. @@ -968,7 +968,7 @@ retcode = store.setup( The absence of error messages indicates successful data transfer. ### Starting the Client as Standalone Process and accessing via RPC -To start a RPC type **real** `Client` as a standalone process, you can use the following command: +To start an RPC type **real** `Client` as a standalone process, you can use the following command: ```bash ./build/mooncake-store/src/mooncake_client \ diff --git a/docs/source/getting_started/build.md b/docs/source/getting_started/build.md index d83c2e6a..af50b0a4 100644 --- a/docs/source/getting_started/build.md +++ b/docs/source/getting_started/build.md @@ -20,7 +20,7 @@ pip install mooncake-transfer-engine-non-cuda > **Note**: The CUDA version includes Mooncake-EP and GPU topology detection, requiring CUDA 12.1+. The non-CUDA version is for environments without CUDA dependencies. > **Note**: MLU support is currently source-build only. If you need Cambricon MLU memory support, install Neuware and build with `-DUSE_MLU=ON`. -## Automatic +## Automatic Build ### Recommended Version - OS: Ubuntu 22.04 LTS+ @@ -45,7 +45,7 @@ pip install mooncake-transfer-engine-non-cuda sudo make install ``` -## Manual +## Manual Build ### Recommended Version - cmake: 3.22.x @@ -105,7 +105,10 @@ pip install mooncake-transfer-engine-non-cuda export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/cuda/lib64 export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64 ``` - > **Note:** Mooncake could use the DMA-BUF path for GPU-Direct RDMA, which does **not** require the `nvidia-peermem` kernel module. If you prefer the DMA-BUF path, please set the runtime environment variable `WITH_NVIDIA_PEERMEM=0` before starting Mooncake. If you prefer the legacy `ibv_reg_mr` path (which requires `nvidia-peermem`), set the runtime environment variable `WITH_NVIDIA_PEERMEM=1`. See Section 3.7 of https://docs.nvidia.com/cuda/gpudirect-rdma/ for instructions on installing `nvidia-peermem`. + ```{admonition} GPU-Direct RDMA + :class: note + Mooncake could use the DMA-BUF path for GPU-Direct RDMA, which does **not** require the `nvidia-peermem` kernel module. If you prefer the DMA-BUF path, please set the runtime environment variable `WITH_NVIDIA_PEERMEM=0` before starting Mooncake. If you prefer the legacy `ibv_reg_mr` path (which requires `nvidia-peermem`), set the runtime environment variable `WITH_NVIDIA_PEERMEM=1`. See Section 3.7 of https://docs.nvidia.com/cuda/gpudirect-rdma/ for instructions on installing `nvidia-peermem`. + ``` 3. If you want to compile the Moore Mthreads GPUDirect support module, first follow the instructions in https://docs.mthreads.com/musa-sdk/musa-sdk-doc-online/install_guide to install MUSA. After that: 1) Install `mthreads-peermem` for enabling GPU-Direct RDMA diff --git a/docs/source/getting_started/examples/vllm-integration/disagg-prefill-decode.md b/docs/source/getting_started/examples/vllm-integration/disagg-prefill-decode.md new file mode 100644 index 00000000..c8f0ab24 --- /dev/null +++ b/docs/source/getting_started/examples/vllm-integration/disagg-prefill-decode.md @@ -0,0 +1,337 @@ +# Disaggregated Prefill-Decode with MooncakeConnector + +## Overview + +This guide demonstrates how to use `MooncakeConnector` with vLLM for disaggregated Prefill-Decode (PD) serving. `MooncakeConnector` enables direct cross-node KV cache transfer between prefill and decode instances using RDMA technology, achieving up to **142.25 GB/s** peak bandwidth (71.1% utilization of 8x RoCE). + +For more details about Mooncake, please refer to [Mooncake project](https://github.com/kvcache-ai/Mooncake) and [Mooncake documents](https://kvcache-ai.github.io/Mooncake/). + +--- + +## Choose Your vLLM Backend + +| Backend | vLLM Version | Status | Guide | +|---------|-------------|--------|-------| +| **vLLM V1** | Latest | Recommended | [Jump to V1 guide](#using-vllm-v1-recommended) | +| **vLLM V0** | ≤ v0.6.4.post1 | Legacy | [Jump to V0 guide](#using-vllm-v0-legacy) | + +```{admonition} New Users +:class: tip +If you are starting a new deployment, use the **vLLM V1** backend. V0 support is maintained for existing deployments only. +``` + +--- + +## Using vLLM V1 (Recommended) + +This section covers `MooncakeConnector` integration with vLLM V1 backend. The integration enables efficient cross-node KV cache transfer via RDMA. + +### Installation + +#### Prerequisites + +Install mooncake-transfer-engine through pip: + +```bash +pip install mooncake-transfer-engine +``` + +```{note} +If you encounter problems such as missing `lib*.so`, uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](../../build.md). +``` + +#### Install vLLM + +Refer to [vLLM official installation guide](https://docs.vllm.ai/en/latest/getting_started/installation.html) for the latest installation instructions. + +### Usage + +#### Basic Setup (Different Nodes) + +**Prefiller Node** (192.168.0.2): + +```bash +vllm serve Qwen/Qwen2.5-7B-Instruct \ + --port 8010 \ + --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer"}' +``` + +**Decoder Node** (192.168.0.3): + +```bash +vllm serve Qwen/Qwen2.5-7B-Instruct \ + --port 8020 \ + --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_consumer"}' +``` + +**Proxy Server:** + +```bash +# In vllm root directory. +python tests/v1/kv_connector/nixl_integration/toy_proxy_server.py \ + --prefiller-host 192.168.0.2 --prefiller-port 8010 \ + --decoder-host 192.168.0.3 --decoder-port 8020 +``` + +> NOTE: The Mooncake Connector currently uses the proxy from nixl_integration. This will be replaced with a self-developed proxy in the future. + +Now you can send requests to the proxy server through port 8000. + +**Test:** + +```bash +curl http://127.0.0.1:8000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Qwen/Qwen2.5-7B-Instruct", + "messages": [ + {"role": "user", "content": "Tell me a long story about artificial intelligence."} + ] + }' +``` + +#### Advanced Configuration + +**With Tensor Parallelism:** + +Prefiller: + +```bash +CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \ +vllm serve Qwen/Qwen2.5-7B-Instruct \ + --port 8010 \ + --tensor-parallel-size 8 \ + --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer"}' +``` + +Decoder: + +```bash +CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \ +vllm serve Qwen/Qwen2.5-7B-Instruct \ + --port 8020 \ + --tensor-parallel-size 8 \ + --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_consumer"}' +``` + +#### Configuration Parameters + +- `--kv-transfer-config`: JSON string to configure the KV transfer connector + - `kv_connector`: Set to `"MooncakeConnector"` + - `kv_role`: Role of the instance + - `kv_producer`: For prefiller instances that generate KV caches + - `kv_consumer`: For decoder instances that consume KV caches + - `kv_both`: Enables symmetric functionality (experimental) + - `num_workers`: Thread pool size in each prefiller worker to send kvcache (default 10) + +### Environment Variables + +- `VLLM_MOONCAKE_BOOTSTRAP_PORT`: Port for Mooncake bootstrap server (default: 8998) + - Required only for prefiller instances + - Each vLLM worker needs a unique port on its host + - For TP/DP deployments, each worker's port is computed as: `base_port + dp_rank * tp_size + tp_rank` +- `VLLM_MOONCAKE_ABORT_REQUEST_TIMEOUT`: Timeout (in seconds) for automatically releasing KV cache (default: 480) + - Used when a request is aborted to prevent holding resources indefinitely + +### Performance + +For detailed performance benchmarks and results, see the [vLLM Benchmark](../../../performance/vllm-v1-support-benchmark.md) documentation. + +--- + +## Using vLLM V0 (Legacy) + +```{admonition} Legacy Backend +:class: warning +This section is for vLLM V0 backend (≤ v0.6.4.post1). For new deployments, use the [V1 backend](#using-vllm-v1-recommended) above. +``` + +This integration is based on [PR 10502](https://github.com/vllm-project/vllm/pull/10502) and [PR 10884](https://github.com/vllm-project/vllm/pull/10884). Preview benchmark results are available at [vLLM Benchmark Results V0.2](../../../performance/vllm-benchmark-results-v0.2.md). + +### Installation + +#### Prerequisite + +```bash +pip3 install mooncake-transfer-engine +``` + +```{note} +- If you encounter problems such as missing `lib*.so`, uninstall this package by `pip3 uninstall mooncake-transfer-engine`, and build the binaries manually according to the [instructions](../../build.md). +- For vLLM version ≤ v0.8.4, it requires `mooncake-transfer-engine ≤ 0.3.3.post2`. In the latest release, the interface `mooncake_vllm_adaptor` has been deprecated. +``` + +#### Install vLLM + +**1. Clone vLLM from official repo:** + +```bash +git clone git@github.com:vllm-project/vllm.git +``` + +**2. Build from source (Include C++ and CUDA code):** + +```bash +cd vllm +pip3 uninstall vllm -y +pip3 install -e . +``` + +```{tip} +If the build fails, try upgrading cmake: `pip3 install cmake --upgrade`. +``` + +If you encounter any problems, refer to the [vLLM official compilation guide](https://docs.vllm.ai/en/v0.6.4.post1/getting_started/installation.html#install-the-latest-code). + +### Configuration + +#### Prepare configuration file over RDMA + +Create a `mooncake.json` file for both Prefill and Decode instances. Use the identical config file on both sides. + +```json +{ + "prefill_url": "192.168.0.137:13003", + "decode_url": "192.168.0.139:13003", + "metadata_server": "192.168.0.139:2379", + "metadata_backend": "etcd", + "protocol": "rdma", + "device_name": "erdma_0" +} +``` + +- `prefill_url`: The IP address and port of the Prefill node (port is used to communicate with metadata server). +- `decode_url`: The IP address and port of the Decode node. If running prefill and decode on the same node, set a different port (at least 50 apart from `prefill_url` port) to avoid conflicts. +- `metadata_server`: The metadata server address. Supports `etcd`, `redis`, and `http` backends. Example: `"etcd://192.168.0.137:2379"`, `"redis://192.168.0.137:6379"`, `"http://192.168.0.137:8080/metadata"`. +- `metadata_backend`: Currently supports `"etcd"`, `"redis"`, and `"http"`. If absent and `metadata_server` has no prefix, defaults to `"etcd"`. This parameter will be deprecated in a future version. +- `protocol`: `"rdma"` or `"tcp"`. +- `device_name`: Required when protocol is `"rdma"`. Multiple NICs can be separated by commas (`"erdma_0,erdma_1"`). + +#### Prepare configuration file over TCP + +```json +{ + "prefill_url": "192.168.0.137:13003", + "decode_url": "192.168.0.139:13003", + "metadata_server": "192.168.0.139:2379", + "metadata_backend": "etcd", + "protocol": "tcp", + "device_name": "" +} +``` + +### Run Example + +Change the IP addresses and ports according to your environment. + +```bash +# Begin from root of your cloned repo! + +# 1. Start the etcd server +etcd --listen-client-urls http://0.0.0.0:2379 --advertise-client-urls http://localhost:2379 + +# 2. Run on the prefilling side (producer role) +MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8100 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}' + +# 3. Run on the decoding side (consumer role) +MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8200 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2,"kv_buffer_size":2e9}' +``` + +**Key parameters:** +- `MOONCAKE_CONFIG_PATH`: Path to the mooncake.json configuration file. +- `VLLM_USE_MODELSCOPE`: Optional. Remove if you have HuggingFace access. +- `--kv-transfer-config`: Connector configuration + - `kv_connector`: `"MooncakeConnector"` + - `kv_role`: `"kv_producer"` or `"kv_consumer"` + - `kv_rank`: 0 for producer, 1 for consumer + - `kv_parallel_size`: Fixed to 2 currently + - `kv_buffer_size`: KVCache lookup buffer size; increase for longer prompts. If OOM occurs, decrease `--gpu-memory-utilization`. + - `kv_ip` and `kv_port`: Used to specify the IP address and port of the master node for `"PyNcclConnector"` distributed setup. Not used for `"MooncakeConnector"` currently. Instead, `"MooncakeConnector"` uses a config file to set up the distributed connection. +- `--tensor-parallel-size` / `-tp`: Supported. If running on the same node, set different `CUDA_VISIBLE_DEVICES`. + +```{note} +If running prefill and decode on the same node, set a different port for `decode_url`. To avoid port conflicts, ensure the decode port differs by at least 50 from the `prefill_url` port (e.g., `"decode_url": "192.168.0.137:13103"`). If the same URL is set for both, the port of `decode_url` will be automatically incremented by 100. +``` + +**Proxy Server:** + +```bash +python3 proxy_server.py +``` + +```python +# proxy_server.py +import os +import aiohttp +from quart import Quart, make_response, request + +AIOHTTP_TIMEOUT = aiohttp.ClientTimeout(total=6 * 60 * 60) +app = Quart(__name__) + +async def forward_request(url, data): + async with aiohttp.ClientSession(timeout=AIOHTTP_TIMEOUT) as session: + headers = {"Authorization": f"Bearer {os.environ.get('OPENAI_API_KEY')}"} + async with session.post(url=url, json=data, headers=headers) as response: + if response.status == 200: + async for chunk_bytes in response.content.iter_chunked(1024): + yield chunk_bytes + +@app.route('/v1/completions', methods=['POST']) +async def handle_request(): + try: + original_request_data = await request.get_json() + prefill_request = original_request_data.copy() + prefill_request['max_tokens'] = 1 # prefill only + async for _ in forward_request('http://localhost:8100/v1/completions', prefill_request): + continue + generator = forward_request('http://192.168.0.139:8200/v1/completions', # Change IP + original_request_data) + response = await make_response(generator) + response.timeout = None + return response + except Exception as e: + import sys, traceback + exc_info = sys.exc_info() + print("Error occurred in disagg prefill proxy server") + print(e) + print("".join(traceback.format_exception(*exc_info))) + +if __name__ == '__main__': + app.run(host="0.0.0.0", port=8000) +``` + +> Be sure to change the IP address in the proxy server code. + +### Test + +```bash +curl -s http://localhost:8000/v1/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4", + "prompt": "San Francisco is a", + "max_tokens": 1000 + }' +``` + +--- + +## Troubleshooting + +- If you encounter connection issues, check that: + - All nodes can reach each other over the network + - Firewall rules allow traffic on the specified ports + - RDMA devices are properly configured and listed in `device_name` +- For missing library errors, rebuild `mooncake-transfer-engine` from source +- Enable debug logging with `VLLM_LOGGING_LEVEL=DEBUG` for detailed diagnostics +- For production deployments, consider using a more robust proxy solution diff --git a/docs/source/getting_started/examples/vllm-integration/index.md b/docs/source/getting_started/examples/vllm-integration/index.md index 021b9748..b2a49474 100644 --- a/docs/source/getting_started/examples/vllm-integration/index.md +++ b/docs/source/getting_started/examples/vllm-integration/index.md @@ -1,13 +1,64 @@ -# vLLM Disaggregated Serving +# Mooncake x vLLM Integration + +## Overview + +Mooncake integrates with vLLM to accelerate large language model serving through high-performance KV cache transfer and shared storage. The integration supports two primary scenarios: + +- **Disaggregated Prefill-Decode Serving**: Seamlessly split prefill and decode across nodes using `MooncakeConnector`, with RDMA-powered cross-node KV cache transfer achieving up to **142.25 GB/s** peak bandwidth (71.1% utilization of 8x RoCE). Transfer overhead is negligible — for 32K-token prompts (4.50 GB of KV data), transfer takes only **31.65 ms**, accounting for just **4.2%** of total TTFT. +- **KV Cache Storage & Sharing**: Extend effective KV cache capacity via `MooncakeStore` / `MooncakeStoreConnector`, with hash-based prefix caching that enables multiple vLLM instances to share cached KV blocks. Supports CPU/Disk offloading and dynamic XpYd topologies at runtime. + +| Scenario | Guide | vLLM Backend | +|----------|-------|-------------| +| PD Disaggregation (KV transfer) | [Disaggregated Prefill-Decode](disagg-prefill-decode) | V1 ✅ / V0 ⚠️ | +| KV Cache Storage & Sharing | [KV Cache Storage with MooncakeStore](kv-cache-storage) | V1 ✅ / V0 ⚠️ | + +```{admonition} New to Mooncake + vLLM? +:class: tip +Start with the V1 guides above. Legacy V0 documentation is available for existing deployments only. +``` + +--- + +## Getting Started + +### Disaggregated Prefill-Decode + +Direct KV cache transfer between prefill and decode nodes via `MooncakeConnector` using RDMA. ::::{toctree} :maxdepth: 1 -vllm-mooncakestoreconnector -vllmv1-lmcache-integration -vllmv1-lmcache-mp-integration -vllm-integration-v0.2 -vllm-integration-v0.3 -vllm-integration-v1.0 +disagg-prefill-decode :::: +### KV Cache Storage & Sharing + +Distributed KV cache storage via `MooncakeStore` / `MooncakeStoreConnector` for offloading, prefix caching, and cross-instance sharing. + +::::{toctree} +:maxdepth: 1 + +kv-cache-storage +:::: + +### LMCache-Based Disaggregated Serving + +Disaggregated prefill-decode using LMCache with Mooncake Store as the remote storage backend. Supports both non-MP (`LMCacheConnectorV1`) and MP (`LMCacheMPConnector`) paths. + +::::{toctree} +:maxdepth: 1 + +vllmv1-lmcache-integration +vllmv1-lmcache-mp-integration +:::: + +--- + +## Archived Documentation + +The following pages are from earlier versions of the integration and are no longer maintained. All content has been consolidated into the scenario-based guides above. + +- [MooncakeStoreConnector (Original)](vllm-mooncakestoreconnector) +- [vLLM V0 PD Disaggregation Demo (Original)](vllm-integration-v0.2) +- [vLLM V0 MooncakeStore (Original)](vllm-integration-v0.3) +- [vLLM V1 PD Disaggregation (Original)](vllm-integration-v1.0) diff --git a/docs/source/getting_started/examples/vllm-integration/kv-cache-storage.md b/docs/source/getting_started/examples/vllm-integration/kv-cache-storage.md new file mode 100644 index 00000000..c25427ad --- /dev/null +++ b/docs/source/getting_started/examples/vllm-integration/kv-cache-storage.md @@ -0,0 +1,409 @@ +# KV Cache Storage & Sharing with MooncakeStore + +## Overview + +This guide demonstrates how to use `MooncakeStore` / `MooncakeStoreConnector` with vLLM to build a distributed KV cache storage pool. It enables KV cache offloading to CPU/SSD, hash-based prefix caching across multiple vLLM instances, and flexible XpYd disaggregated deployment — where you can dynamically adjust prefill and decode group sizes at runtime. + +Compared to Redis-based backends, MooncakeStore achieves significantly lower TTFT (e.g., **~32% improvement** in mean TTFT for 2P2D tp=2 under RDMA). See [benchmark results](../../../performance/vllm-benchmark-results-v1.md) for details. + +--- + +## Choose Your vLLM Backend + +| Backend | Connector | vLLM Version | Status | Guide | +|---------|-----------|-------------|--------|-------| +| **vLLM V1** | `MooncakeStoreConnector` | Latest | Recommended | [Jump to V1 guide](#using-vllm-v1-recommended) | +| **vLLM V0** | `MooncakeStore` | ≤ v0.6.4.post1 | Legacy | [Jump to V0 guide](#using-vllm-v0-legacy) | + +```{admonition} New Users +:class: tip +If you are starting a new deployment, use the **vLLM V1** backend with `MooncakeStoreConnector`. V0 support is maintained for existing deployments only. +``` + +Key differences from v0.x to v1: +- **XpYd support and orchestration**: Dynamically change the population of prefill and decode groups +- **More stable and fault-tolerant**: A sudden crash of a single vLLM instance is tolerable; instance-to-instance connections are removed, so each instance works as a vanilla vLLM instance capable of handling requests independently + +--- + +## Using vLLM V1 (Recommended) + +This section covers `MooncakeStoreConnector` — the new vLLM KV connector that uses `MooncakeDistributedStore` as a shared KV cache pool. It enables: + +- **CPU/Disk offloading**: Extend effective KV cache capacity by offloading to CPU memory or SSD via Mooncake's transfer engine. +- **Hash-based prefix caching across instances**: Multiple vLLM instances share cached KV blocks through the store using block-hash deduplication. +- **Flexible deployment**: Works as a single-node KV cache extension (`kv_both`), or in disaggregated prefill-decode setups (`kv_producer` / `kv_consumer`). + +### Deployment + +#### 1. Prerequisites + +- [vLLM](https://github.com/vllm-project/vllm) is installed +- [Mooncake](https://github.com/kvcache-ai/Mooncake) is installed + +Refer to the [vLLM official repository](https://github.com/vllm-project/vllm) and [Mooncake official repository](https://github.com/kvcache-ai/Mooncake) for installation instructions and building from source. + +#### 2. Mooncake Master Server + +**Start:** + +```shell +mooncake_master --port 50063 +``` + +**Configure Mooncake**: Create a JSON configuration file (e.g., `mooncake_config.json`): + +```json +{ + "metadata_server": "http://127.0.0.1:8092/metadata", + "master_server_address": "127.0.0.1:50063", + "global_segment_size": "0", + "local_buffer_size": "2147483648", + "protocol": "rdma", + "device_name": "" +} +``` + +**Set environment variable:** + +```shell +export MOONCAKE_CONFIG_PATH=/path/to/mooncake_config.json +``` + +#### 3. Usage + +**3.1 Single-Node KV Cache Offloading** (`kv_both`): + +```shell +MOONCAKE_CONFIG_PATH=mooncake_config.json \ +vllm serve meta-llama/Llama-3.1-8B-Instruct \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_both"}' +``` + +**3.2 XpYd Disaggregated Prefill-Decode** (`kv_producer/kv_consumer`): + +Prefill Node: + +```shell +MOONCAKE_CONFIG_PATH=mooncake_config.json \ +VLLM_MOONCAKE_BOOTSTRAP_PORT=50052 \ +vllm serve meta-llama/Llama-3.1-8B-Instruct \ + --port 8100 \ + --kv-transfer-config '{ + "kv_connector": "MultiConnector", + "kv_role": "kv_producer", + "kv_connector_extra_config": { + "connectors": [ + { + "kv_connector": "MooncakeConnector", + "kv_role": "kv_producer" + }, + { + "kv_connector": "MooncakeStoreConnector", + "kv_role": "kv_producer" + } + ] + } + }' +``` + +Decode Node: + +```shell +MOONCAKE_CONFIG_PATH=mooncake_config.json \ +VLLM_MOONCAKE_BOOTSTRAP_PORT=50053 \ +vllm serve meta-llama/Llama-3.1-8B-Instruct \ + --port 8200 \ + --kv-transfer-config '{ + "kv_connector": "MultiConnector", + "kv_role": "kv_consumer", + "kv_connector_extra_config": { + "connectors": [ + { + "kv_connector": "MooncakeConnector", + "kv_role": "kv_consumer" + }, + { + "kv_connector": "MooncakeStoreConnector", + "kv_role": "kv_consumer" + } + ] + } + }' +``` + +Proxy: + +```shell +python examples/disaggregated/disaggregated_serving/mooncake_connector/mooncake_connector_proxy.py \ + --prefill http://192.168.0.2:8100 \ + --decode http://192.168.0.3:8200 +``` + +> When running with data parallelism, set a fixed `PYTHONHASHSEED` so that block hashes are consistent across DP ranks: +> +> ```shell +> PYTHONHASHSEED=0 vllm serve ... +> ``` +> +> Without this, identical prompts may produce different block hashes on different DP ranks, preventing cross-instance prefix cache hits. + +--- + +## Using vLLM V0 (Legacy) + +```{admonition} Legacy Backend +:class: warning +This section is for vLLM V0 backend with `MooncakeStore`. For new deployments, use the [V1 backend with `MooncakeStoreConnector`](#using-vllm-v1-recommended) above. +``` + +This integration is based on [PR 10502](https://github.com/vllm-project/vllm/pull/10502) and [PR 12957](https://github.com/vllm-project/vllm/pull/12957) to support KVCache transfer for intra-node and inter-node disaggregated serving. + +### Installation + +#### Prerequisite + +```bash +pip3 install mooncake-transfer-engine +``` + +```{note} +- If you encounter problems such as missing `lib*.so`, uninstall by `pip3 uninstall mooncake-transfer-engine`, and build manually according to the [instructions](../../build.md). +- For vLLM version ≤ v0.8.4, it requires `mooncake-transfer-engine ≤ 0.3.3.post2`. The interface `mooncake_vllm_adaptor` has been deprecated in the latest release. +``` + +#### Install vLLM + +**1. Clone vLLM:** + +```bash +git clone git@github.com:vllm-project/vllm.git +``` + +**2. Build from source:** + +```bash +cd vllm +pip3 install -e . +``` + +If you encounter problems, refer to the [vLLM official compilation guide](https://docs.vllm.ai/en/latest/getting_started/installation/index.html). + +### Configuration + +#### Prepare configuration for RDMA + +Create a `mooncake.json` file: + +```json +{ + "local_hostname": "192.168.0.137", + "metadata_server": "etcd://192.168.0.137:2379", + "protocol": "rdma", + "device_name": "erdma_0", + "master_server_address": "192.168.0.137:50001" +} +``` + +- `local_hostname`: The IP address of the current node. All prefill and decode instances on the same node can share this config. +- `metadata_server`: The metadata server. Supports `etcd`, `redis`, and `http` backends. +- `protocol`: `"rdma"` or `"tcp"`. +- `device_name`: Required for RDMA. Multiple NICs separated by commas (`"erdma_0,erdma_1"`). +- `master_server_address`: The IP address and port of the MooncakeStore master daemon. + +#### Prepare configuration for TCP + +```json +{ + "local_hostname": "192.168.0.137", + "metadata_server": "etcd://192.168.0.137:2379", + "protocol": "tcp", + "device_name": "", + "master_server_address": "192.168.0.137:50001" +} +``` + +### Run Example + +Change the IP addresses and ports according to your environment. `VLLM_USE_V1=0` is required for vLLM V0 backend. + +```bash +# Begin from root of your cloned repo! + +# 1. Start the etcd server +etcd --listen-client-urls http://0.0.0.0:2379 --advertise-client-urls http://localhost:2379 +# You may need to terminate other etcd processes before running the above command + +# 2. Start the mooncake_master server +mooncake_master --port 50001 +# If some vllm instances exit unexpectedly, some connection metadata will be +# corrupted since they are not properly cleaned. In that case, we recommend +# you restart the mooncake_master before running another test. + +# 3. Run multiple vllm instances +# kv_producer role +MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8100 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' + +CUDA_VISIBLE_DEVICES=1 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8101 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' + +CUDA_VISIBLE_DEVICES=2 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8102 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' + +CUDA_VISIBLE_DEVICES=3 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8103 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' + +# kv_consumer role +CUDA_VISIBLE_DEVICES=4 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8200 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' + +CUDA_VISIBLE_DEVICES=5 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8201 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' + +CUDA_VISIBLE_DEVICES=6 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8202 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' + +CUDA_VISIBLE_DEVICES=7 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8203 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' +``` + +**Key parameters:** +- `MOONCAKE_CONFIG_PATH`: Path to the mooncake.json configuration file. +- `VLLM_USE_MODELSCOPE`: Optional. Remove if you have HuggingFace access. +- `VLLM_USE_V1=0`: Required since the disaggregated feature is currently only supported on V0 vLLM. You can also `export` this configuration to the env instead of putting it in front of every command. +- `--model`: The model to use. +- `--port`: The vllm service port on which to listen. +- `--max-model-len`: The maximum length of the model. +- `--tensor-parallel-size` / `-tp`: Supported. All instances should have the same tensor_parallel_size. If running prefill and decode on the same node, set different `CUDA_VISIBLE_DEVICES` (e.g., `CUDA_VISIBLE_DEVICES=0,1` for prefill and `CUDA_VISIBLE_DEVICES=2,3` for decode). +- `--kv-transfer-config`: Set `kv_connector` to `"MooncakeStoreConnector"`, `kv_role` to `"kv_producer"`, `"kv_consumer"`, or `"kv_both"`. +- If some vLLM instances exit unexpectedly, connection metadata may be corrupted. Restart `mooncake_master` before another test. + +```bash +# 5. Start the proxy server +cd vllm +python3 examples/online_serving/disagg_examples/disagg_proxy_demo.py \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --prefill localhost:8100 localhost:8101 \ + --decode localhost:8200 localhost:8201 \ + --port 8000 +``` + +- `--model`: The model and tokenizer used by the proxy server. +- `--port`: The proxy server port on which to listen. +- `--prefill` / `-p`: IP and port of the vllm prefill instances. +- `--decode` / `-d`: IP and port of the vllm decode instances. + +#### Dynamic XpYd Adjustment + +To dynamically adjust prefill and decode instances at runtime: + +```bash +export ADMIN_API_KEY="xxxxxxxx" + +# or add it before the command: +ADMIN_API_KEY="xxxxxxxx" python3 vllm/examples/online_serving/disagg_examples/disagg_demo.py \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --prefill localhost:8100 localhost:8101 \ + --decode localhost:8200 localhost:8201 \ + --port 8000 \ + --scheduling round_robin + +# Add instances to groups dynamically +curl -X POST "http://localhost:8000/instances/add" \ + -H "Content-Type: application/json" \ + -H "X-API-Key: $ADMIN_API_KEY" \ + -d '{"type": "prefill", "instance": "localhost:8102"}' + +curl -X POST "http://localhost:8000/instances/add" \ + -H "Content-Type: application/json" \ + -H "X-API-Key: $ADMIN_API_KEY" \ + -d '{"type": "prefill", "instance": "localhost:8103"}' + +curl -X POST "http://localhost:8000/instances/add" \ + -H "Content-Type: application/json" \ + -H "X-API-Key: $ADMIN_API_KEY" \ + -d '{"type": "decode", "instance": "localhost:8202"}' + +curl -X POST "http://localhost:8000/instances/add" \ + -H "Content-Type: application/json" \ + -H "X-API-Key: $ADMIN_API_KEY" \ + -d '{"type": "decode", "instance": "localhost:8203"}' + +# Get proxy status +curl localhost:8000/status | jq +``` + +```{note} +Mooncake team provides this simple round-robin proxy as a demo. In production, you can implement custom global proxy strategies. +``` + +**Be sure to change the IP address in the commands.** + +### Test + +```bash +curl -s http://localhost:8000/v1/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4", + "prompt": "San Francisco is a", + "max_tokens": 1000 + }' +``` + +- If you are not testing on the proxy server, change `localhost` to the IP address of the proxy server. + +--- + +## Performance + +| Scenario | Document | +|----------|----------| +| V1 MooncakeStoreConnector vs Redis | [Benchmark V1](../../../performance/vllm-benchmark-results-v1.md) | +| V0 MooncakeStore vs Redis | [Benchmark V0](../../../performance/vllm-benchmark-results-v0.2.md) | + +--- + +## Troubleshooting + +- If you encounter connection issues, check that: + - All nodes can reach each other over the network + - Firewall rules allow traffic on the specified ports + - RDMA devices are properly configured + - `mooncake_master` is running and reachable +- For missing library errors, rebuild `mooncake-transfer-engine` from source +- If vLLM instances exit unexpectedly, restart `mooncake_master` to clean up corrupted metadata +- Enable debug logging with `VLLM_LOGGING_LEVEL=DEBUG` for detailed diagnostics diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.2.md b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.2.md index babb851b..6a567dd7 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.2.md +++ b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.2.md @@ -1,5 +1,10 @@ # vLLM V0 Disaggregated Serving Demo +```{admonition} Archived +:class: warning +This page has been **consolidated** into the unified [Disaggregated Prefill-Decode](disagg-prefill-decode) guide (see the V0 Legacy section). Please use that guide for up-to-date information. +``` + ## Overview This is the latest version of mooncake-transfer-engine integration doc with the vLLM project based on [PR 10502](https://github.com/vllm-project/vllm/pull/10502) and [PR 10884](https://github.com/vllm-project/vllm/pull/10884) (vllm version: v0.6.4.post1/main) to accelerate KVCache transfer for inter-node disaggregated serving scenario. We have run some experiments to obtain some [preview benchmark results](../../../performance/vllm-benchmark-results-v0.2.md). More benchmark results will be released in due time. @@ -89,10 +94,20 @@ etcd --listen-client-urls http://0.0.0.0:2379 --advertise-client-urls http://loc # You may need to terminate other etcd processes before running the above command # 2. Run on the prefilling side (producer role) -MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8100 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}' +MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8100 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_producer","kv_rank":0,"kv_parallel_size":2,"kv_buffer_size":2e9}' # 3. Run on the decoding side (consumer role) -MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8200 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2,"kv_buffer_size":2e9}' +MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_MODELSCOPE=True python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8200 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeConnector","kv_role":"kv_consumer","kv_rank":1,"kv_parallel_size":2,"kv_buffer_size":2e9}' ``` - `MOONCAKE_CONFIG_PATH` is the path to the mooncake.json configuration file. diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.3.md b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.3.md index dadb8544..42e34106 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.3.md +++ b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v0.3.md @@ -1,5 +1,10 @@ # vLLM V0 Disaggregated Serving with MooncakeStore +```{admonition} Archived +:class: warning +This page has been **consolidated** into the unified [KV Cache Storage & Sharing](kv-cache-storage) guide (see the V0 Legacy section). Please use that guide for up-to-date information. +``` + ## Overview This is the latest version of the MooncakeStore integration doc with the vLLM project based on [PR 10502](https://github.com/vllm-project/vllm/pull/10502) and [PR 12957](https://github.com/vllm-project/vllm/pull/12957) to support KVCache transfer for intra-node and inter-node disaggregated serving scenario. Benchmark results will be released soon. @@ -91,22 +96,62 @@ mooncake_master --port 50001 # 3. Run multiple vllm instances # kv_producer role -MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8100 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' +MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8100 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' -CUDA_VISIBLE_DEVICES=1 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8101 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' +CUDA_VISIBLE_DEVICES=1 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8101 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' -CUDA_VISIBLE_DEVICES=2 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8102 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' +CUDA_VISIBLE_DEVICES=2 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8102 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' -CUDA_VISIBLE_DEVICES=3 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8103 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' +CUDA_VISIBLE_DEVICES=3 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8103 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_producer"}' # kv_consumer role -CUDA_VISIBLE_DEVICES=4 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8200 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' +CUDA_VISIBLE_DEVICES=4 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8200 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' -CUDA_VISIBLE_DEVICES=5 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8201 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' +CUDA_VISIBLE_DEVICES=5 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8201 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' -CUDA_VISIBLE_DEVICES=6 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8202 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' +CUDA_VISIBLE_DEVICES=6 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8202 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' -CUDA_VISIBLE_DEVICES=7 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --port 8203 --max-model-len 10000 --gpu-memory-utilization 0.8 --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' +CUDA_VISIBLE_DEVICES=7 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python3 -m vllm.entrypoints.openai.api_server \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --port 8203 \ + --max-model-len 10000 \ + --gpu-memory-utilization 0.8 \ + --kv-transfer-config '{"kv_connector":"MooncakeStoreConnector","kv_role":"kv_consumer"}' ``` - `MOONCAKE_CONFIG_PATH` is the path to the mooncake.json configuration file. @@ -127,7 +172,11 @@ CUDA_VISIBLE_DEVICES=7 MOONCAKE_CONFIG_PATH=./mooncake.json VLLM_USE_V1=0 python ```bash # 4. Start the proxy server cd vllm -python3 examples/online_serving/disagg_examples/disagg_proxy_demo.py --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --prefill localhost:8100 localhost:8101 --decode localhost:8200 localhost:8201 --port 8000 +python3 examples/online_serving/disagg_examples/disagg_proxy_demo.py \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --prefill localhost:8100 localhost:8101 \ + --decode localhost:8200 localhost:8201 \ + --port 8000 ``` - The `--model` parameter specifies the model to use, also specifies the tokenizer used by the proxy server. @@ -139,7 +188,12 @@ python3 examples/online_serving/disagg_examples/disagg_proxy_demo.py --model Qwe # If you want to dynamically adjust the instances of p-nodes and d-nodes during runtime, you need to configure this environment variables. export ADMIN_API_KEY="xxxxxxxx" # or add it before the command: -ADMIN_API_KEY="xxxxxxxx" python3 vllm/examples/online_serving/disagg_examples/disagg_demo.py --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 --prefill localhost:8100 localhost:8101 --decode localhost:8200 localhost:8201 --port 8000 --scheduling round_robin +ADMIN_API_KEY="xxxxxxxx" python3 vllm/examples/online_serving/disagg_examples/disagg_demo.py \ + --model Qwen/Qwen2.5-7B-Instruct-GPTQ-Int4 \ + --prefill localhost:8100 localhost:8101 \ + --decode localhost:8200 localhost:8201 \ + --port 8000 \ + --scheduling round_robin # Then use this command to add instances into prefill group or decode group curl -X POST "http://localhost:8000/instances/add" -H "Content-Type: application/json" -H "X-API-Key: $ADMIN_API_KEY" -d '{"type": "prefill", "instance": "localhost:8102"}' diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v1.0.md b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v1.0.md index 84d3e2bc..6e7ad79e 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllm-integration-v1.0.md +++ b/docs/source/getting_started/examples/vllm-integration/vllm-integration-v1.0.md @@ -1,5 +1,10 @@ # vLLM v1 backend Disaggregated Serving with MooncakeConnector +```{admonition} Archived +:class: warning +This page has been **consolidated** into the unified [Disaggregated Prefill-Decode](disagg-prefill-decode) guide. Please use that guide for up-to-date information. +``` + ## Overview This guide demonstrates how to use the MooncakeConnector with vLLM v1 backend for disaggregated serving in Prefill-Decode separation architecture. The integration enables efficient cross-node KV cache transfer using RDMA technology. @@ -100,7 +105,7 @@ vllm serve Qwen/Qwen2.5-7B-Instruct \ - `kv_producer`: For prefiller instances that generate KV caches - `kv_consumer`: For decoder instances that consume KV caches - `kv_both`: Enables symmetric functionality (experimental) - - `num_workers`: Thread pool size in each prefiller worker to send kvcache (default 10) + - `num_workers`: Thread pool size in each prefiller worker to send kvcache (default 10) ## Environment Variables diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md b/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md index 50a87f05..8cc122f3 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md +++ b/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md @@ -1,5 +1,10 @@ # Guide: vLLM MooncakeStoreConnector +```{admonition} Archived +:class: warning +This page has been **consolidated** into the unified [KV Cache Storage & Sharing](kv-cache-storage) guide (see the V1 Recommended section). Please use that guide for up-to-date information. +``` + ## Overview This document describes how to deploy vLLM's `MooncakeStoreConnector`. `MooncakeStoreConnector` is a new vLLM's KV connector that uses `MooncakeDistributedStore` as a shared KV cache pool. It enables: @@ -114,7 +119,9 @@ vllm serve meta-llama/Llama-3.1-8B-Instruct \ Proxy: ```shell -python examples/disaggregated/disaggregated_serving/mooncake_connector/mooncake_connector_proxy.py --prefill http://192.168.0.2:8100 --decode http://192.168.0.3:8200 +python examples/disaggregated/disaggregated_serving/mooncake_connector/mooncake_connector_proxy.py \ + --prefill http://192.168.0.2:8100 \ + --decode http://192.168.0.3:8200 ``` diff --git a/docs/source/index.md b/docs/source/index.md index e551f369..19ff330e 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -83,6 +83,7 @@ getting_started/examples/vllm-integration/index :caption: Performance :maxdepth: 1 +performance/vllm/index performance/sglang-benchmark-results-v1 performance/vllm-benchmark-results-v0.2 performance/vllm-benchmark-results-v1 @@ -152,3 +153,16 @@ deployment/mooncake-store-deployment-guide community/governance ::: + +% Archived content + +:::{toctree} +:caption: Archived +:maxdepth: 1 + +getting_started/examples/vllm-integration/vllm-mooncakestoreconnector +getting_started/examples/vllm-integration/vllm-integration-v0.2 +getting_started/examples/vllm-integration/vllm-integration-v0.3 +getting_started/examples/vllm-integration/vllm-integration-v1.0 +getting_started/examples/vllm-integration/vllmv1-lmcache-integration +::: diff --git a/docs/source/performance/vllm/index.md b/docs/source/performance/vllm/index.md new file mode 100644 index 00000000..6e5ea118 --- /dev/null +++ b/docs/source/performance/vllm/index.md @@ -0,0 +1,18 @@ +# vLLM Performance Benchmarks + +Benchmarks evaluating Mooncake's integration with vLLM across different backends and scenarios. + +| Document | Backend | Key Findings | +|----------|---------|---------------| +| [vLLM V1 + MooncakeConnector](../vllm-v1-support-benchmark) | vLLM V1 | 1P1D PD disaggregation on H800 with 8x RoCE: **142.25 GB/s** peak transfer bandwidth (71.1% of theoretical), KV transfer overhead just **4.2%** of total TTFT at 32K tokens | +| [vLLM V1 + MooncakeStore vs Redis](../vllm-benchmark-results-v1) | vLLM V1 | MooncakeStore RDMA consistently outperforms Redis across all XpYd topologies — e.g., **~32% lower** mean TTFT in 2P2D tp=2 | +| [vLLM V0 + MooncakeConnector (Legacy)](../vllm-benchmark-results-v0.2) | vLLM V0 | TP=4 reduces TTFT by ~80% vs TP=1; RDMA provides significant latency advantage over TCP across varying QPS and input lengths | + +:::{toctree} +:maxdepth: 1 +:hidden: + +../vllm-v1-support-benchmark +../vllm-benchmark-results-v1 +../vllm-benchmark-results-v0.2 +::: -- 2.34.1 From 6a18e79eeae5698f141f517ff8437a17ca97d441 Mon Sep 17 00:00:00 2001 From: LZW <99333079+Lin-z-w@users.noreply.github.com> Date: Tue, 2 Jun 2026 20:46:36 +0800 Subject: [PATCH 167/382] [Store] Support tenant-aware async storage tasks (#2294) --- mooncake-store/include/client_service.h | 46 +++- mooncake-store/include/file_storage.h | 4 +- mooncake-store/include/master_client.h | 56 +++- mooncake-store/include/master_service.h | 64 ++++- mooncake-store/include/rpc_service.h | 39 ++- mooncake-store/include/segment.h | 8 +- mooncake-store/include/task_manager.h | 6 +- mooncake-store/include/types.h | 45 ++++ mooncake-store/src/client_service.cpp | 103 +++++-- mooncake-store/src/file_storage.cpp | 153 ++++++++--- mooncake-store/src/master_client.cpp | 207 ++++++++++---- mooncake-store/src/master_service.cpp | 255 ++++++++++++++---- mooncake-store/src/real_client.cpp | 5 +- mooncake-store/src/rpc_service.cpp | 111 +++++--- mooncake-store/src/segment.cpp | 72 ++++- mooncake-store/src/utils.cpp | 5 +- .../tests/file_storage_promotion_test.cpp | 47 +++- .../master_service_test_for_snapshot.cpp | 15 +- .../master_service_test_for_snapshot_base.h | 9 +- mooncake-store/tests/master_service_test.cpp | 119 +++++++- .../tests/offload_on_evict_test.cpp | 6 +- .../tests/promotion_on_hit_test.cpp | 52 ++-- 22 files changed, 1125 insertions(+), 302 deletions(-) diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h index 57ee11a7..d644610d 100644 --- a/mooncake-store/include/client_service.h +++ b/mooncake-store/include/client_service.h @@ -65,6 +65,7 @@ class Client { virtual ~Client(); const UUID& getClientId() const { return client_id_; } + const std::string& tenant_id() const { return master_client_.tenant_id(); } /** * @brief Creates and initializes a new Client instance @@ -145,6 +146,9 @@ class Client { */ std::vector> BatchQuery( const std::vector& object_keys); + std::vector> BatchQuery( + const std::vector& object_keys, + const std::string& tenant_id); /** * @brief Batch clear KV cache for specified object keys on a specific @@ -275,9 +279,15 @@ class Client { */ tl::expected EvictDiskReplica(const std::string& key, ReplicaType replica_type); + tl::expected EvictDiskReplica(const std::string& key, + const std::string& tenant_id, + ReplicaType replica_type); std::vector> BatchEvictDiskReplica( const std::vector& keys, ReplicaType replica_type); + std::vector> BatchEvictDiskReplica( + const std::vector& keys, const std::string& tenant_id, + ReplicaType replica_type); /** * @brief Registers a memory segment to master for allocation @@ -362,6 +372,9 @@ class Client { */ tl::expected CreateCopyTask( const std::string& key, const std::vector& targets); + tl::expected CreateCopyTask( + const std::string& key, const std::string& tenant_id, + const std::vector& targets); /** * @brief Create a move task to move an object's replica from source segment @@ -375,6 +388,10 @@ class Client { tl::expected CreateMoveTask(const std::string& key, const std::string& source, const std::string& target); + tl::expected CreateMoveTask(const std::string& key, + const std::string& tenant_id, + const std::string& source, + const std::string& target); /** * @brief Query a task by task id @@ -401,25 +418,25 @@ class Client { * set of non-offloaded objects. * @param enable_offloading Indicates whether offloading is enabled for this * segment. - * @param offloading_objects On return, contains a map from object key to - * size (in bytes) for all objects that require offload. + * @param offloading_objects On return, contains the tenant-scoped object + * tasks that require offload. */ tl::expected OffloadObjectHeartbeat( bool enable_offloading, - std::unordered_map& offloading_objects); + std::vector& offloading_objects); tl::expected ReportSsdCapacity( int64_t ssd_total_capacity_bytes); /** * @brief Heartbeat-driven pull of pending L2->L1 promotion work for this - * client. Mirror of OffloadObjectHeartbeat. Returns key->size pairs the + * client. Mirror of OffloadObjectHeartbeat. Returns tenant-scoped tasks the * caller (FileStorage) must read from local SSD and stage as MEMORY * replicas via PromotionAllocStart + NotifyPromotionSuccess. */ // Virtual to enable subclassing in unit tests. virtual tl::expected PromotionObjectHeartbeat( - std::unordered_map& promotion_objects); + std::vector& promotion_objects); /** * @brief Stage a PROCESSING MEMORY replica for an existing key during @@ -429,6 +446,10 @@ class Client { virtual tl::expected PromotionAllocStart(const std::string& key, uint64_t size, const std::vector& preferred_segments); + virtual tl::expected + PromotionAllocStart(const std::string& key, const std::string& tenant_id, + uint64_t size, + const std::vector& preferred_segments); /** * @brief Commit a staged MEMORY replica to COMPLETE; called after the @@ -436,6 +457,8 @@ class Client { */ virtual tl::expected NotifyPromotionSuccess( const std::string& key); + virtual tl::expected NotifyPromotionSuccess( + const std::string& key, const std::string& tenant_id); /** * @brief Release master-side promotion task after a client-side failure @@ -443,6 +466,8 @@ class Client { */ virtual tl::expected NotifyPromotionFailure( const std::string& key); + virtual tl::expected NotifyPromotionFailure( + const std::string& key, const std::string& tenant_id); /** * @brief Write `slices` into the memory replica described by @@ -483,6 +508,9 @@ class Client { tl::expected NotifyOffloadSuccess( const std::vector& keys, const std::vector& metadatas); + tl::expected NotifyOffloadSuccess( + const std::vector& tasks, + const std::vector& metadatas); /** * @brief Fetch tasks assigned to a client @@ -844,6 +872,10 @@ class Client { tl::expected Copy(const std::string& key, const std::string& source, const std::vector& targets); + tl::expected Copy(const std::string& key, + const std::string& tenant_id, + const std::string& source, + const std::vector& targets); /** * @brief Move an object's replica from source segment to target segment @@ -855,6 +887,10 @@ class Client { tl::expected Move(const std::string& key, const std::string& source, const std::string& target); + tl::expected Move(const std::string& key, + const std::string& tenant_id, + const std::string& source, + const std::string& target); // Task thread pool for async task execution ThreadPool task_thread_pool_; diff --git a/mooncake-store/include/file_storage.h b/mooncake-store/include/file_storage.h index db56d313..6aa2ac3d 100644 --- a/mooncake-store/include/file_storage.h +++ b/mooncake-store/include/file_storage.h @@ -75,7 +75,7 @@ class FileStorage { * @return tl::expected indicating operation status. */ tl::expected OffloadObjects( - const std::unordered_map& offloading_objects); + const std::vector& offloading_objects); /** * @brief Performs a heartbeat operation for the FileStorage component. @@ -107,7 +107,7 @@ class FileStorage { std::unordered_map& batch_object); tl::expected BatchQuerySegmentSlices( - const std::vector& keys, + const std::vector& keys, const std::string& tenant_id, std::unordered_map>& batched_slices); tl::expected RegisterLocalMemory(); diff --git a/mooncake-store/include/master_client.h b/mooncake-store/include/master_client.h index a10fd15c..cad41592 100644 --- a/mooncake-store/include/master_client.h +++ b/mooncake-store/include/master_client.h @@ -76,6 +76,8 @@ class MasterClient { } ~MasterClient(); + const std::string& tenant_id() const { return tenant_id_; } + MasterClient(const MasterClient&) = delete; MasterClient& operator=(const MasterClient&) = delete; @@ -146,6 +148,8 @@ class MasterClient { */ [[nodiscard]] tl::expected GetReplicaList(const std::string& object_key); + [[nodiscard]] tl::expected + GetReplicaList(const std::string& object_key, const std::string& tenant_id); /** * @brief Retrieves replica lists for object keys that match a regex @@ -167,6 +171,9 @@ class MasterClient { */ [[nodiscard]] std::vector> BatchGetReplicaList(const std::vector& object_keys); + [[nodiscard]] std::vector> + BatchGetReplicaList(const std::vector& object_keys, + const std::string& tenant_id); /** * @brief Starts a put operation @@ -406,8 +413,7 @@ class MasterClient { * @param enable_offloading Indicates whether persistence is enabled for * this segment. */ - [[nodiscard]] tl::expected, - ErrorCode> + [[nodiscard]] tl::expected, ErrorCode> OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading); [[nodiscard]] tl::expected ReportSsdCapacity( @@ -423,15 +429,17 @@ class MasterClient { [[nodiscard]] tl::expected NotifyOffloadSuccess( const UUID& client_id, const std::vector& keys, const std::vector& metadatas); + [[nodiscard]] tl::expected NotifyOffloadSuccess( + const UUID& client_id, const std::vector& tasks, + const std::vector& metadatas); /** * @brief Heartbeat-driven pull of pending L2->L1 promotion work for a - * client. Returns key->size pairs the caller should read from local + * client. Returns tenant-scoped tasks the caller should read from local * SSD and stage as MEMORY replicas via PromotionAllocStart + * NotifyPromotionSuccess. */ - [[nodiscard]] tl::expected, - ErrorCode> + [[nodiscard]] tl::expected, ErrorCode> PromotionObjectHeartbeat(const UUID& client_id); /** @@ -443,6 +451,10 @@ class MasterClient { PromotionAllocStart(const UUID& client_id, const std::string& key, uint64_t size, const std::vector& preferred_segments); + [[nodiscard]] tl::expected + PromotionAllocStart(const UUID& client_id, const std::string& key, + const std::string& tenant_id, uint64_t size, + const std::vector& preferred_segments); /** * @brief Release master-side promotion task state after a client-side @@ -451,6 +463,9 @@ class MasterClient { */ [[nodiscard]] tl::expected NotifyPromotionFailure( const UUID& client_id, const std::string& key); + [[nodiscard]] tl::expected NotifyPromotionFailure( + const UUID& client_id, const std::string& key, + const std::string& tenant_id); /** * @brief Commit a staged MEMORY replica to COMPLETE; called after the @@ -458,6 +473,9 @@ class MasterClient { */ [[nodiscard]] tl::expected NotifyPromotionSuccess( const UUID& client_id, const std::string& key); + [[nodiscard]] tl::expected NotifyPromotionSuccess( + const UUID& client_id, const std::string& key, + const std::string& tenant_id); /** * @brief Start a copy operation @@ -470,6 +488,10 @@ class MasterClient { [[nodiscard]] tl::expected CopyStart( const std::string& key, const std::string& src_segment, const std::vector& tgt_segments); + [[nodiscard]] tl::expected CopyStart( + const std::string& key, const std::string& tenant_id, + const std::string& src_segment, + const std::vector& tgt_segments); /** * @brief End a copy operation @@ -477,6 +499,8 @@ class MasterClient { * @return tl::expected indicating success/failure */ [[nodiscard]] tl::expected CopyEnd(const std::string& key); + [[nodiscard]] tl::expected CopyEnd( + const std::string& key, const std::string& tenant_id); /** * @brief Revoke a copy operation @@ -485,6 +509,8 @@ class MasterClient { */ [[nodiscard]] tl::expected CopyRevoke( const std::string& key); + [[nodiscard]] tl::expected CopyRevoke( + const std::string& key, const std::string& tenant_id); /** * @brief Start a move operation @@ -497,6 +523,9 @@ class MasterClient { [[nodiscard]] tl::expected MoveStart( const std::string& key, const std::string& src_segment, const std::string& tgt_segment); + [[nodiscard]] tl::expected MoveStart( + const std::string& key, const std::string& tenant_id, + const std::string& src_segment, const std::string& tgt_segment); /** * @brief End a move operation @@ -504,6 +533,8 @@ class MasterClient { * @return tl::expected indicating success/failure */ [[nodiscard]] tl::expected MoveEnd(const std::string& key); + [[nodiscard]] tl::expected MoveEnd( + const std::string& key, const std::string& tenant_id); /** * @brief Revoke a move operation @@ -512,6 +543,8 @@ class MasterClient { */ [[nodiscard]] tl::expected MoveRevoke( const std::string& key); + [[nodiscard]] tl::expected MoveRevoke( + const std::string& key, const std::string& tenant_id); /** * @brief Create a task to copy an object's replica to target segments @@ -522,6 +555,9 @@ class MasterClient { */ [[nodiscard]] tl::expected CreateCopyTask( const std::string& key, const std::vector& targets); + [[nodiscard]] tl::expected CreateCopyTask( + const std::string& key, const std::string& tenant_id, + const std::vector& targets); /** * @brief Create a task to move an object's replica from source segment to @@ -535,6 +571,9 @@ class MasterClient { [[nodiscard]] tl::expected CreateMoveTask( const std::string& key, const std::string& source, const std::string& target); + [[nodiscard]] tl::expected CreateMoveTask( + const std::string& key, const std::string& tenant_id, + const std::string& source, const std::string& target); /** * @brief Query a task by task id @@ -570,6 +609,9 @@ class MasterClient { */ [[nodiscard]] tl::expected EvictDiskReplica( const std::string& key, ReplicaType replica_type); + [[nodiscard]] tl::expected EvictDiskReplica( + const std::string& key, const std::string& tenant_id, + ReplicaType replica_type); /** * @brief Batch notify master that disk replicas were evicted locally. @@ -581,6 +623,10 @@ class MasterClient { [[nodiscard]] std::vector> BatchEvictDiskReplica(const std::vector& keys, ReplicaType replica_type); + [[nodiscard]] std::vector> + BatchEvictDiskReplica(const std::vector& keys, + const std::string& tenant_id, + ReplicaType replica_type); private: /** diff --git a/mooncake-store/include/master_service.h b/mooncake-store/include/master_service.h index 0580000c..38193d21 100644 --- a/mooncake-store/include/master_service.h +++ b/mooncake-store/include/master_service.h @@ -309,6 +309,9 @@ class MasterService { */ auto AddReplica(const UUID& client_id, const std::string& key, Replica& replica) -> tl::expected; + auto AddReplica(const UUID& client_id, const std::string& key, + const std::string& tenant_id, Replica& replica) + -> tl::expected; /** * @brief Revoke a put operation, replica_type indicates the type of @@ -428,6 +431,10 @@ class MasterService { auto EvictDiskReplica(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected; + auto EvictDiskReplica(const UUID& client_id, const std::string& key, + const std::string& tenant_id, + ReplicaType replica_type) + -> tl::expected; /** * @brief Batch evict disk replicas for multiple keys. @@ -439,6 +446,9 @@ class MasterService { std::vector> BatchEvictDiskReplica( const UUID& client_id, const std::vector& keys, ReplicaType replica_type); + std::vector> BatchEvictDiskReplica( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id, ReplicaType replica_type); /** * @brief Start a copy operation @@ -457,12 +467,22 @@ class MasterService { const UUID& client_id, const std::string& key, const std::string& src_segment, const std::vector& tgt_segments); + tl::expected CopyStart( + const UUID& client_id, const std::string& key, + const std::string& tenant_id, const std::string& src_segment, + const std::vector& tgt_segments); tl::expected CopyEnd(const UUID& client_id, const std::string& key); + tl::expected CopyEnd(const UUID& client_id, + const std::string& key, + const std::string& tenant_id); tl::expected CopyRevoke(const UUID& client_id, const std::string& key); + tl::expected CopyRevoke(const UUID& client_id, + const std::string& key, + const std::string& tenant_id); /** * @brief Start a move operation @@ -480,12 +500,22 @@ class MasterService { tl::expected MoveStart( const UUID& client_id, const std::string& key, const std::string& src_segment, const std::string& tgt_segment); + tl::expected MoveStart( + const UUID& client_id, const std::string& key, + const std::string& tenant_id, const std::string& src_segment, + const std::string& tgt_segment); tl::expected MoveEnd(const UUID& client_id, const std::string& key); + tl::expected MoveEnd(const UUID& client_id, + const std::string& key, + const std::string& tenant_id); tl::expected MoveRevoke(const UUID& client_id, const std::string& key); + tl::expected MoveRevoke(const UUID& client_id, + const std::string& key, + const std::string& tenant_id); /** * @brief Remove an object and its replicas @@ -581,7 +611,7 @@ class MasterService { * segment. */ auto OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading) - -> tl::expected, ErrorCode>; + -> tl::expected, ErrorCode>; auto ReportSsdCapacity(const UUID& client_id, int64_t ssd_total_capacity_bytes) @@ -599,16 +629,20 @@ class MasterService { const UUID& client_id, const std::vector& keys, const std::vector& metadatas) -> tl::expected; + auto NotifyOffloadSuccess( + const UUID& client_id, const std::vector& tasks, + const std::vector& metadatas) + -> tl::expected; /** * @brief Heartbeat-driven pull of pending promotion work for a client. - * Returns the per-client promotion_objects map (key -> object size) and - * clears it. The per-shard promotion_tasks map remains populated as the - * source of truth until NotifyPromotionSuccess commits the new MEMORY - * replica. + * Returns tenant-scoped promotion tasks for the holder client and clears + * its per-client promotion_objects queue. The per-shard promotion_tasks + * map remains populated as the source of truth until NotifyPromotionSuccess + * commits the new MEMORY replica. */ auto PromotionObjectHeartbeat(const UUID& client_id) - -> tl::expected, ErrorCode>; + -> tl::expected, ErrorCode>; /** * @brief Stage a PROCESSING MEMORY replica for an existing key. Allocates @@ -626,6 +660,10 @@ class MasterService { uint64_t size, const std::vector& preferred_segments) -> tl::expected; + auto PromotionAllocStart(const UUID& client_id, const std::string& key, + const std::string& tenant_id, uint64_t size, + const std::vector& preferred_segments) + -> tl::expected; /** * @brief Commit a staged MEMORY replica to COMPLETE; decrement source @@ -634,6 +672,9 @@ class MasterService { */ auto NotifyPromotionSuccess(const UUID& client_id, const std::string& key) -> tl::expected; + auto NotifyPromotionSuccess(const UUID& client_id, const std::string& key, + const std::string& tenant_id) + -> tl::expected; /** * @brief Holder-side failure notification: the client got past @@ -655,6 +696,9 @@ class MasterService { */ auto NotifyPromotionFailure(const UUID& client_id, const std::string& key) -> tl::expected; + auto NotifyPromotionFailure(const UUID& client_id, const std::string& key, + const std::string& tenant_id) + -> tl::expected; /** * @brief Create a copy task to copy an object's replicas to target segments @@ -662,6 +706,9 @@ class MasterService { */ tl::expected CreateCopyTask( const std::string& key, const std::vector& targets); + tl::expected CreateCopyTask( + const std::string& key, const std::string& tenant_id, + const std::vector& targets); /** * @brief Create a move task to move an object's replica from source segment @@ -671,6 +718,10 @@ class MasterService { tl::expected CreateMoveTask(const std::string& key, const std::string& source, const std::string& target); + tl::expected CreateMoveTask(const std::string& key, + const std::string& tenant_id, + const std::string& source, + const std::string& target); /** * @brief Create a drain job to gracefully evacuate one or more segments. @@ -1860,6 +1911,7 @@ class MasterService { struct ActiveDrainTask { UUID task_id; + std::string tenant_id; std::string key; std::string source_segment; std::string target_segment; diff --git a/mooncake-store/include/rpc_service.h b/mooncake-store/include/rpc_service.h index b233ed4d..90726f17 100644 --- a/mooncake-store/include/rpc_service.h +++ b/mooncake-store/include/rpc_service.h @@ -184,29 +184,32 @@ class WrappedMasterService { tl::expected MountLocalDiskSegment(const UUID& client_id, bool enable_offloading); - tl::expected, ErrorCode> + tl::expected, ErrorCode> OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading); tl::expected ReportSsdCapacity( const UUID& client_id, int64_t ssd_total_capacity_bytes); tl::expected NotifyOffloadSuccess( - const UUID& client_id, const std::vector& keys, + const UUID& client_id, const std::vector& tasks, const std::vector& metadatas); // Promotion-on-hit RPCs. - tl::expected, ErrorCode> + tl::expected, ErrorCode> PromotionObjectHeartbeat(const UUID& client_id); tl::expected PromotionAllocStart( - const UUID& client_id, const std::string& key, uint64_t size, + const UUID& client_id, const std::string& key, + const std::string& tenant_id, uint64_t size, const std::vector& preferred_segments); tl::expected NotifyPromotionSuccess( - const UUID& client_id, const std::string& key); + const UUID& client_id, const std::string& key, + const std::string& tenant_id); tl::expected NotifyPromotionFailure( - const UUID& client_id, const std::string& key); + const UUID& client_id, const std::string& key, + const std::string& tenant_id); tl::expected CreateDrainJob( const CreateDrainJobRequest& request); @@ -220,9 +223,11 @@ class WrappedMasterService { tl::expected QuerySegmentStatusById( const UUID& segment_id); tl::expected CreateCopyTask( - const std::string& key, const std::vector& targets); + const std::string& key, const std::string& tenant_id, + const std::vector& targets); tl::expected CreateMoveTask(const std::string& key, + const std::string& tenant_id, const std::string& source, const std::string& target); @@ -236,32 +241,38 @@ class WrappedMasterService { tl::expected CopyStart( const UUID& client_id, const std::string& key, - const std::string& src_segment, + const std::string& tenant_id, const std::string& src_segment, const std::vector& tgt_segments); tl::expected CopyEnd(const UUID& client_id, - const std::string& key); + const std::string& key, + const std::string& tenant_id); tl::expected CopyRevoke(const UUID& client_id, - const std::string& key); + const std::string& key, + const std::string& tenant_id); tl::expected MoveStart( const UUID& client_id, const std::string& key, - const std::string& src_segment, const std::string& tgt_segment); + const std::string& tenant_id, const std::string& src_segment, + const std::string& tgt_segment); tl::expected MoveEnd(const UUID& client_id, - const std::string& key); + const std::string& key, + const std::string& tenant_id); tl::expected MoveRevoke(const UUID& client_id, - const std::string& key); + const std::string& key, + const std::string& tenant_id); tl::expected EvictDiskReplica(const UUID& client_id, const std::string& key, + const std::string& tenant_id, ReplicaType replica_type); std::vector> BatchEvictDiskReplica( const UUID& client_id, const std::vector& keys, - ReplicaType replica_type); + const std::string& tenant_id, ReplicaType replica_type); private: MasterService master_service_; diff --git a/mooncake-store/include/segment.h b/mooncake-store/include/segment.h index 8fcd2f87..0d8aa1c1 100644 --- a/mooncake-store/include/segment.h +++ b/mooncake-store/include/segment.h @@ -86,14 +86,14 @@ struct LocalDiskSegment { mutable Mutex offloading_mutex_; bool enable_offloading; int64_t ssd_total_capacity_bytes = 0; // last reported by client heartbeat - std::unordered_map GUARDED_BY(offloading_mutex_) - offloading_objects; + std::unordered_map GUARDED_BY( + offloading_mutex_) offloading_objects; // Promotion-on-hit pending work for this client. Populated by master's // TryPushPromotionQueue when a Get hits a LOCAL_DISK-only key on this // client. Drained by PromotionObjectHeartbeat. Same locking as // offloading_objects (offloading_mutex_). - std::unordered_map GUARDED_BY(offloading_mutex_) - promotion_objects; + std::unordered_map GUARDED_BY( + offloading_mutex_) promotion_objects; explicit LocalDiskSegment(bool enable_offloading) : enable_offloading(enable_offloading) {} diff --git a/mooncake-store/include/task_manager.h b/mooncake-store/include/task_manager.h index d12152ed..8f4578ce 100644 --- a/mooncake-store/include/task_manager.h +++ b/mooncake-store/include/task_manager.h @@ -92,18 +92,20 @@ struct Task { }; struct ReplicaCopyPayload { + std::string tenant_id = "default"; std::string key; std::string source; std::vector targets; }; -YLT_REFL(ReplicaCopyPayload, key, source, targets); +YLT_REFL(ReplicaCopyPayload, tenant_id, key, source, targets); struct ReplicaMovePayload { + std::string tenant_id = "default"; std::string key; std::string source; std::string target; }; -YLT_REFL(ReplicaMovePayload, key, source, target); +YLT_REFL(ReplicaMovePayload, tenant_id, key, source, target); template struct TaskPayloadTraits; diff --git a/mooncake-store/include/types.h b/mooncake-store/include/types.h index e7a566a8..18ab8ed7 100644 --- a/mooncake-store/include/types.h +++ b/mooncake-store/include/types.h @@ -227,6 +227,51 @@ inline std::string NormalizeTenantId(const std::string& tenant_id) { return tenant_id.empty() ? "default" : tenant_id; } +inline std::string MakeTenantScopedStorageKey(const std::string& tenant_id, + const std::string& key) { + const auto normalized_tenant = NormalizeTenantId(tenant_id); + std::string scoped_key; + scoped_key.reserve(normalized_tenant.size() + key.size() + 1); + scoped_key.append(normalized_tenant); + scoped_key.push_back('\0'); + scoped_key.append(key); + return scoped_key; +} + +inline std::pair ParseTenantScopedStorageKey( + const std::string& storage_key) { + const auto separator = storage_key.find('\0'); + if (separator == std::string::npos) { + return {"default", storage_key}; + } + return {NormalizeTenantId(storage_key.substr(0, separator)), + storage_key.substr(separator + 1)}; +} + +struct OffloadTaskItem { + std::string tenant_id; + std::string key; + int64_t size; + + bool operator==(const OffloadTaskItem& other) const { + return tenant_id == other.tenant_id && key == other.key && + size == other.size; + } +}; +YLT_REFL(OffloadTaskItem, tenant_id, key, size); + +struct PromotionTaskItem { + std::string tenant_id; + std::string key; + int64_t size; + + bool operator==(const PromotionTaskItem& other) const { + return tenant_id == other.tenant_id && key == other.key && + size == other.size; + } +}; +YLT_REFL(PromotionTaskItem, tenant_id, key, size); + // Store client configuration validation limits static constexpr size_t MIN_SEGMENT_SIZE = 1024; // 1KB static constexpr size_t MAX_SEGMENT_SIZE = 1024ULL * 1024 * 1024 * 1024; // 1TB diff --git a/mooncake-store/src/client_service.cpp b/mooncake-store/src/client_service.cpp index 47ca39e2..6bebc4fe 100644 --- a/mooncake-store/src/client_service.cpp +++ b/mooncake-store/src/client_service.cpp @@ -1014,9 +1014,14 @@ tl::expected Client::Query( std::vector> Client::BatchQuery( const std::vector& object_keys) { + return BatchQuery(object_keys, master_client_.tenant_id()); +} + +std::vector> Client::BatchQuery( + const std::vector& object_keys, const std::string& tenant_id) { std::chrono::steady_clock::time_point start_time = std::chrono::steady_clock::now(); - auto response = master_client_.BatchGetReplicaList(object_keys); + auto response = master_client_.BatchGetReplicaList(object_keys, tenant_id); // Check if we got the expected number of responses if (response.size() != object_keys.size()) { @@ -2547,11 +2552,23 @@ tl::expected Client::EvictDiskReplica( return master_client_.EvictDiskReplica(key, replica_type); } +tl::expected Client::EvictDiskReplica( + const std::string& key, const std::string& tenant_id, + ReplicaType replica_type) { + return master_client_.EvictDiskReplica(key, tenant_id, replica_type); +} + std::vector> Client::BatchEvictDiskReplica( const std::vector& keys, ReplicaType replica_type) { return master_client_.BatchEvictDiskReplica(keys, replica_type); } +std::vector> Client::BatchEvictDiskReplica( + const std::vector& keys, const std::string& tenant_id, + ReplicaType replica_type) { + return master_client_.BatchEvictDiskReplica(keys, tenant_id, replica_type); +} + std::vector Client::GetNicNumaNodes() const { std::set nodes; if (!transfer_engine_) return {}; @@ -2875,8 +2892,7 @@ tl::expected Client::MountLocalDiskSegment( } tl::expected Client::OffloadObjectHeartbeat( - bool enable_offloading, - std::unordered_map& offloading_objects) { + bool enable_offloading, std::vector& offloading_objects) { auto response = master_client_.OffloadObjectHeartbeat(client_id_, enable_offloading); if (!response) { @@ -2928,8 +2944,14 @@ tl::expected Client::NotifyOffloadSuccess( return response; } +tl::expected Client::NotifyOffloadSuccess( + const std::vector& tasks, + const std::vector& metadatas) { + return master_client_.NotifyOffloadSuccess(client_id_, tasks, metadatas); +} + tl::expected Client::PromotionObjectHeartbeat( - std::unordered_map& promotion_objects) { + std::vector& promotion_objects) { auto response = master_client_.PromotionObjectHeartbeat(client_id_); if (!response) { return tl::make_unexpected(response.error()); @@ -2946,16 +2968,34 @@ Client::PromotionAllocStart( preferred_segments); } +tl::expected +Client::PromotionAllocStart( + const std::string& key, const std::string& tenant_id, uint64_t size, + const std::vector& preferred_segments) { + return master_client_.PromotionAllocStart(client_id_, key, tenant_id, size, + preferred_segments); +} + tl::expected Client::NotifyPromotionSuccess( const std::string& key) { return master_client_.NotifyPromotionSuccess(client_id_, key); } +tl::expected Client::NotifyPromotionSuccess( + const std::string& key, const std::string& tenant_id) { + return master_client_.NotifyPromotionSuccess(client_id_, key, tenant_id); +} + tl::expected Client::NotifyPromotionFailure( const std::string& key) { return master_client_.NotifyPromotionFailure(client_id_, key); } +tl::expected Client::NotifyPromotionFailure( + const std::string& key, const std::string& tenant_id) { + return master_client_.NotifyPromotionFailure(client_id_, key, tenant_id); +} + ErrorCode Client::PromotionWrite(const Replica::Descriptor& memory_descriptor, std::vector& slices) { return TransferWrite(memory_descriptor, slices); @@ -2966,12 +3006,24 @@ tl::expected Client::CreateCopyTask( return master_client_.CreateCopyTask(key, targets); } +tl::expected Client::CreateCopyTask( + const std::string& key, const std::string& tenant_id, + const std::vector& targets) { + return master_client_.CreateCopyTask(key, tenant_id, targets); +} + tl::expected Client::CreateMoveTask( const std::string& key, const std::string& source, const std::string& target) { return master_client_.CreateMoveTask(key, source, target); } +tl::expected Client::CreateMoveTask( + const std::string& key, const std::string& tenant_id, + const std::string& source, const std::string& target) { + return master_client_.CreateMoveTask(key, tenant_id, source, target); +} + tl::expected Client::ExecuteReplicaTransfer( const std::string& key, const std::string& action_name, std::function()> end_fn, @@ -3032,11 +3084,18 @@ tl::expected Client::ExecuteReplicaTransfer( tl::expected Client::Copy( const std::string& key, const std::string& source, const std::vector& targets) { + return Copy(key, master_client_.tenant_id(), source, targets); +} + +tl::expected Client::Copy( + const std::string& key, const std::string& tenant_id, + const std::string& source, const std::vector& targets) { LOG(INFO) << "action=replica_copy_start" << ", key=" << key << ", targets_count=" << targets.size(); // Call CopyStart first - it validates existence and allocates replicas - auto start_result = master_client_.CopyStart(key, source, targets); + auto start_result = + master_client_.CopyStart(key, tenant_id, source, targets); if (!start_result.has_value()) { ErrorCode error = start_result.error(); LOG(ERROR) << "action=replica_copy_failed" << ", key=" << key @@ -3050,7 +3109,7 @@ tl::expected Client::Copy( LOG(INFO) << "action=replica_copy_skipped" << ", key=" << key << ", info=target_replicas_already_exist"; // Target replicas already exist, consider it success - auto copy_end_result = master_client_.CopyEnd(key); + auto copy_end_result = master_client_.CopyEnd(key, tenant_id); if (!copy_end_result.has_value()) { ErrorCode error = copy_end_result.error(); LOG(ERROR) << "action=replica_copy_failed" << ", key=" << key @@ -3061,9 +3120,9 @@ tl::expected Client::Copy( } auto result = ExecuteReplicaTransfer( - key, "copy", [&]() { return master_client_.CopyEnd(key); }, - [&]() { return master_client_.CopyRevoke(key); }, response.source, - response.targets); + key, "copy", [&]() { return master_client_.CopyEnd(key, tenant_id); }, + [&]() { return master_client_.CopyRevoke(key, tenant_id); }, + response.source, response.targets); if (result.has_value()) { LOG(INFO) << "action=replica_copy_success" << ", key=" << key @@ -3076,12 +3135,20 @@ tl::expected Client::Copy( tl::expected Client::Move(const std::string& key, const std::string& source, const std::string& target) { + return Move(key, master_client_.tenant_id(), source, target); +} + +tl::expected Client::Move(const std::string& key, + const std::string& tenant_id, + const std::string& source, + const std::string& target) { LOG(INFO) << "action=replica_move_start" << ", key=" << key << ", source_segment=" << source << ", target_segment=" << target; // Call MoveStart first - it validates existence and allocates replica if // needed - auto move_start_result = master_client_.MoveStart(key, source, target); + auto move_start_result = + master_client_.MoveStart(key, tenant_id, source, target); if (!move_start_result.has_value()) { ErrorCode error = move_start_result.error(); LOG(ERROR) << "action=replica_move_failed" << ", key=" << key @@ -3095,7 +3162,7 @@ tl::expected Client::Move(const std::string& key, LOG(INFO) << "action=replica_move_skipped" << ", key=" << key << ", info=target_replica_already_exists"; // Target already exists, consider it success - auto move_end_result = master_client_.MoveEnd(key); + auto move_end_result = master_client_.MoveEnd(key, tenant_id); if (!move_end_result.has_value()) { ErrorCode error = move_end_result.error(); LOG(ERROR) << "action=replica_move_failed" << ", key=" << key @@ -3108,9 +3175,9 @@ tl::expected Client::Move(const std::string& key, std::vector targets = {response.target.value()}; auto result = ExecuteReplicaTransfer( - key, "move", [&]() { return master_client_.MoveEnd(key); }, - [&]() { return master_client_.MoveRevoke(key); }, response.source, - targets); + key, "move", [&]() { return master_client_.MoveEnd(key, tenant_id); }, + [&]() { return master_client_.MoveRevoke(key, tenant_id); }, + response.source, targets); if (result.has_value()) { LOG(INFO) << "action=replica_move_success" << ", key=" << key @@ -3383,8 +3450,8 @@ void Client::ExecuteTask(const ClientTask& client_task) { case TaskType::REPLICA_COPY: { ReplicaCopyPayload payload; struct_json::from_json(payload, assignment.payload); - auto copy_result = - Copy(payload.key, payload.source, payload.targets); + auto copy_result = Copy(payload.key, payload.tenant_id, + payload.source, payload.targets); if (copy_result.has_value()) { result = ErrorCode::OK; } else { @@ -3395,8 +3462,8 @@ void Client::ExecuteTask(const ClientTask& client_task) { case TaskType::REPLICA_MOVE: { ReplicaMovePayload payload; struct_json::from_json(payload, assignment.payload); - auto move_result = - Move(payload.key, payload.source, payload.target); + auto move_result = Move(payload.key, payload.tenant_id, + payload.source, payload.target); if (move_result.has_value()) { result = ErrorCode::OK; } else { diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index a7623f03..39653822 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -18,6 +18,26 @@ using gpu_staging::CopyDeviceToHost; using gpu_staging::IsDevicePointer; using gpu_staging::SetDevice; +namespace { + +std::vector BuildOffloadTasksFromStorageKeys( + const std::vector& storage_keys, + const std::vector& metadatas) { + std::vector tasks; + tasks.reserve(storage_keys.size()); + for (size_t i = 0; i < storage_keys.size(); ++i) { + auto [tenant_id, key] = ParseTenantScopedStorageKey(storage_keys[i]); + const int64_t size = + i < metadatas.size() ? metadatas[i].data_size : int64_t{0}; + tasks.push_back(OffloadTaskItem{.tenant_id = std::move(tenant_id), + .key = std::move(key), + .size = size}); + } + return tasks; +} + +} // namespace + FileStorageConfig FileStorageConfig::FromEnvironment() { FileStorageConfig config; @@ -264,8 +284,9 @@ tl::expected FileStorage::Init() { for (auto& metadata : metadatas) { metadata.transport_endpoint = local_rpc_addr_; } + auto tasks = BuildOffloadTasksFromStorageKeys(keys, metadatas); auto add_object_result = - client_->NotifyOffloadSuccess(keys, metadatas); + client_->NotifyOffloadSuccess(tasks, metadatas); if (!add_object_result) { LOG(ERROR) << "Failed to add object to master: " << add_object_result.error(); @@ -339,15 +360,26 @@ tl::expected FileStorage::BatchGet( } tl::expected FileStorage::OffloadObjects( - const std::unordered_map& offloading_objects) { + const std::vector& offloading_objects) { if (offloading_objects.empty()) { return {}; } + std::unordered_map storage_object_sizes; + std::unordered_map task_by_storage_key; + storage_object_sizes.reserve(offloading_objects.size()); + task_by_storage_key.reserve(offloading_objects.size()); + for (const auto& task : offloading_objects) { + const auto storage_key = + MakeTenantScopedStorageKey(task.tenant_id, task.key); + storage_object_sizes.emplace(storage_key, task.size); + task_by_storage_key.emplace(storage_key, task); + } + std::vector> buckets_keys; if (auto bucket_backend = std::dynamic_pointer_cast(storage_backend_)) { auto allocate_res = bucket_backend->AllocateOffloadingBuckets( - offloading_objects, buckets_keys); + storage_object_sizes, buckets_keys); if (!allocate_res) { LOG(ERROR) << "AllocateOffloadingBuckets failed with error: " << allocate_res.error(); @@ -355,21 +387,32 @@ tl::expected FileStorage::OffloadObjects( } } else { std::vector keys; - keys.reserve(offloading_objects.size()); - for (const auto& it : offloading_objects) { + keys.reserve(storage_object_sizes.size()); + for (const auto& it : storage_object_sizes) { keys.emplace_back(it.first); } buckets_keys.emplace_back(std::move(keys)); } auto complete_handler = - [this](const std::vector& keys, - std::vector& metadatas) -> ErrorCode { + [this, &task_by_storage_key]( + const std::vector& keys, + std::vector& metadatas) -> ErrorCode { VLOG(1) << "Success to store objects, keys count: " << keys.size(); for (auto& metadata : metadatas) { metadata.transport_endpoint = local_rpc_addr_; } - auto result = client_->NotifyOffloadSuccess(keys, metadatas); + std::vector tasks; + tasks.reserve(keys.size()); + for (const auto& key : keys) { + auto it = task_by_storage_key.find(key); + if (it == task_by_storage_key.end()) { + LOG(ERROR) << "Offload task not found for storage key"; + return ErrorCode::INVALID_KEY; + } + tasks.push_back(it->second); + } + auto result = client_->NotifyOffloadSuccess(tasks, metadatas); if (!result) { LOG(ERROR) << "NotifyOffloadSuccess failed with error: " << result.error(); @@ -380,24 +423,63 @@ tl::expected FileStorage::OffloadObjects( for (const auto& keys : buckets_keys) { std::unordered_map> batch_object; - auto query_result = BatchQuerySegmentSlices(keys, batch_object); - if (!query_result) { - LOG(ERROR) << "BatchQuerySlices failed with error: " - << query_result.error(); + std::unordered_map> + storage_keys_by_tenant; + for (const auto& storage_key : keys) { + const auto it = task_by_storage_key.find(storage_key); + if (it != task_by_storage_key.end()) { + storage_keys_by_tenant[it->second.tenant_id].push_back( + storage_key); + } + } + for (const auto& [tenant_id, storage_keys] : storage_keys_by_tenant) { + std::vector user_keys; + user_keys.reserve(storage_keys.size()); + for (const auto& storage_key : storage_keys) { + user_keys.push_back(task_by_storage_key[storage_key].key); + } + std::unordered_map> + user_batch_object; + auto query_result = BatchQuerySegmentSlices(user_keys, tenant_id, + user_batch_object); + if (!query_result) { + LOG(ERROR) << "BatchQuerySlices failed with error: " + << query_result.error(); + continue; + } + for (size_t i = 0; i < storage_keys.size(); ++i) { + auto it = user_batch_object.find(user_keys[i]); + if (it != user_batch_object.end()) { + batch_object.emplace(storage_keys[i], + std::move(it->second)); + } + } + } + if (batch_object.empty()) { continue; } auto eviction_handler = [this](const std::vector& evicted_keys) { if (evicted_keys.empty()) return; - auto results = client_->BatchEvictDiskReplica( - evicted_keys, ReplicaType::LOCAL_DISK); - for (size_t i = 0; i < results.size(); ++i) { - if (!results[i]) { - LOG(WARNING) - << "Failed to notify master about evicted local disk " - "key: " - << evicted_keys[i] << ", error: " << results[i].error(); + std::unordered_map> + keys_by_tenant; + for (const auto& storage_key : evicted_keys) { + auto [tenant_id, key] = + ParseTenantScopedStorageKey(storage_key); + keys_by_tenant[tenant_id].push_back(key); + } + for (const auto& [tenant_id, keys] : keys_by_tenant) { + auto results = client_->BatchEvictDiskReplica( + keys, tenant_id, ReplicaType::LOCAL_DISK); + for (size_t i = 0; i < results.size(); ++i) { + if (!results[i]) { + LOG(WARNING) + << "Failed to notify master about evicted local " + "disk key: " + << keys[i] << ", tenant_id=" << tenant_id + << ", error: " << results[i].error(); + } } } }; @@ -521,7 +603,7 @@ tl::expected FileStorage::Heartbeat() { }); } - std::unordered_map + std::vector offloading_objects; // Objects selected for offloading // === STEP 1: Send heartbeat and get offloading decisions === @@ -605,7 +687,7 @@ tl::expected FileStorage::ProcessPromotionTasks() { return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } - std::unordered_map promotion_objects; + std::vector promotion_objects; auto heartbeat_result = client_->PromotionObjectHeartbeat(promotion_objects); if (!heartbeat_result) { @@ -636,7 +718,11 @@ tl::expected FileStorage::ProcessPromotionTasks() { // work stays queued in the master's promotion_objects map and is // returned on subsequent heartbeats; we process whatever we received // here without a second client-side cap. - for (const auto& [key, size] : promotion_objects) { + for (const auto& task : promotion_objects) { + const auto& key = task.key; + const auto& tenant_id = task.tenant_id; + const int64_t size = task.size; + const auto storage_key = MakeTenantScopedStorageKey(tenant_id, key); if (size <= 0) { LOG(WARNING) << "Skipping promotion for key=" << key << " with non-positive size=" << size; @@ -644,7 +730,7 @@ tl::expected FileStorage::ProcessPromotionTasks() { } auto alloc_result = client_->PromotionAllocStart( - key, static_cast(size), preferred_segments); + key, tenant_id, static_cast(size), preferred_segments); if (!alloc_result) { // AllocStart failed (typically NO_AVAILABLE_HANDLE under // DRAM pressure). No staged buffer to release, but the @@ -657,7 +743,7 @@ tl::expected FileStorage::ProcessPromotionTasks() { VLOG(1) << "PromotionAllocStart failed for key=" << key << ", error=" << alloc_result.error() << " (likely no free DRAM); releasing master slot"; - auto release = client_->NotifyPromotionFailure(key); + auto release = client_->NotifyPromotionFailure(key, tenant_id); if (!release) { VLOG(1) << "Promotion: NotifyPromotionFailure failed for key=" << key << ", error=" << release.error() @@ -673,8 +759,8 @@ tl::expected FileStorage::ProcessPromotionTasks() { // throttling or RDMA flakes saturate promotion_queue_limit_ // for the full reaper TTL. NotifyPromotionFailure is // idempotent and best-effort — the reaper is the long-stop. - auto release_master_state = [this, &key]() { - auto release = client_->NotifyPromotionFailure(key); + auto release_master_state = [this, &key, &tenant_id]() { + auto release = client_->NotifyPromotionFailure(key, tenant_id); if (!release) { VLOG(1) << "Promotion: NotifyPromotionFailure failed for key=" << key << ", error=" << release.error() @@ -686,7 +772,7 @@ tl::expected FileStorage::ProcessPromotionTasks() { // from the local SSD backend into it. AllocateBatch returns a // shared_ptr whose BufferHandles RAII-release the // staging space when the local goes out of scope. - std::vector single_key{key}; + std::vector single_key{storage_key}; std::vector single_size{size}; auto allocate_res = AllocateBatch(single_key, single_size); if (!allocate_res) { @@ -707,7 +793,7 @@ tl::expected FileStorage::ProcessPromotionTasks() { // (b) TE-write from the staging slice into the freshly-allocated // MEMORY replica. Slice ptr may have been bumped by O_DIRECT offset // correction in BatchLoad, so re-read it from the slice map. - auto slice_it = staging->slices.find(key); + auto slice_it = staging->slices.find(storage_key); if (slice_it == staging->slices.end()) { LOG(WARNING) << "Promotion: staging slice missing for key=" << key; release_master_state(); @@ -725,7 +811,7 @@ tl::expected FileStorage::ProcessPromotionTasks() { // (c) Commit. Master flips the PROCESSING replica to COMPLETE and it // becomes visible to readers. - auto notify_res = client_->NotifyPromotionSuccess(key); + auto notify_res = client_->NotifyPromotionSuccess(key, tenant_id); if (!notify_res) { // The write landed but the commit failed. We can't retry the // commit (the success path is one-shot via alloc_id), and we @@ -775,9 +861,9 @@ tl::expected FileStorage::BatchLoad( } tl::expected FileStorage::BatchQuerySegmentSlices( - const std::vector& keys, + const std::vector& keys, const std::string& tenant_id, std::unordered_map>& batched_slices) { - auto batched_query_results = client_->BatchQuery(keys); + auto batched_query_results = client_->BatchQuery(keys, tenant_id); if (batched_query_results.empty()) return tl::make_unexpected(ErrorCode::INVALID_REPLICA); for (size_t i = 0; i < batched_query_results.size(); ++i) { @@ -958,8 +1044,9 @@ tl::expected FileStorage::ReRegisterOffloadedObjects() { for (auto& metadata : metadatas) { metadata.transport_endpoint = local_rpc_addr_; } + auto tasks = BuildOffloadTasksFromStorageKeys(keys, metadatas); auto add_object_result = - client_->NotifyOffloadSuccess(keys, metadatas); + client_->NotifyOffloadSuccess(tasks, metadatas); if (!add_object_result) { total_failures++; LOG(ERROR) diff --git a/mooncake-store/src/master_client.cpp b/mooncake-store/src/master_client.cpp index e61f7b23..dc928286 100644 --- a/mooncake-store/src/master_client.cpp +++ b/mooncake-store/src/master_client.cpp @@ -516,23 +516,35 @@ MasterClient::GetReplicaListByRegex(const std::string& str) { tl::expected MasterClient::GetReplicaList( const std::string& object_key) { + return GetReplicaList(object_key, tenant_id_); +} + +tl::expected MasterClient::GetReplicaList( + const std::string& object_key, const std::string& tenant_id) { ScopedVLogTimer timer(1, "MasterClient::GetReplicaList"); - timer.LogRequest("object_key=", object_key); + timer.LogRequest("object_key=", object_key, ", tenant_id=", tenant_id); auto result = invoke_rpc<&WrappedMasterService::GetReplicaList, - GetReplicaListResponse>(object_key, tenant_id_); + GetReplicaListResponse>(object_key, tenant_id); timer.LogResponseExpected(result); return result; } std::vector> MasterClient::BatchGetReplicaList(const std::vector& object_keys) { + return BatchGetReplicaList(object_keys, tenant_id_); +} + +std::vector> +MasterClient::BatchGetReplicaList(const std::vector& object_keys, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "MasterClient::BatchGetReplicaList"); - timer.LogRequest("keys_count=", object_keys.size()); + timer.LogRequest("keys_count=", object_keys.size(), + ", tenant_id=", tenant_id); auto result = invoke_batch_rpc<&WrappedMasterService::BatchGetReplicaList, GetReplicaListResponse>( - object_keys.size(), object_keys, tenant_id_); + object_keys.size(), object_keys, tenant_id); timer.LogResponse("result=", result.size(), " operations"); return result; } @@ -921,11 +933,18 @@ tl::expected MasterClient::MountLocalDiskSegment( tl::expected MasterClient::CreateCopyTask( const std::string& key, const std::vector& targets) { - ScopedVLogTimer timer(1, "MasterClient::CreateCopyTask"); - timer.LogRequest("key=", key, ", targets_size=", targets.size()); + return CreateCopyTask(key, tenant_id_, targets); +} - auto result = - invoke_rpc<&WrappedMasterService::CreateCopyTask, UUID>(key, targets); +tl::expected MasterClient::CreateCopyTask( + const std::string& key, const std::string& tenant_id, + const std::vector& targets) { + ScopedVLogTimer timer(1, "MasterClient::CreateCopyTask"); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id, + ", targets_size=", targets.size()); + + auto result = invoke_rpc<&WrappedMasterService::CreateCopyTask, UUID>( + key, tenant_id, targets); timer.LogResponseExpected(result); return result; } @@ -933,25 +952,32 @@ tl::expected MasterClient::CreateCopyTask( tl::expected MasterClient::CreateMoveTask( const std::string& key, const std::string& source, const std::string& target) { + return CreateMoveTask(key, tenant_id_, source, target); +} + +tl::expected MasterClient::CreateMoveTask( + const std::string& key, const std::string& tenant_id, + const std::string& source, const std::string& target) { ScopedVLogTimer timer(1, "MasterClient::CreateMoveTask"); - timer.LogRequest("key=", key, ", source=", source, ", target=", target); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id, + ", source=", source, ", target=", target); auto result = invoke_rpc<&WrappedMasterService::CreateMoveTask, UUID>( - key, source, target); + key, tenant_id, source, target); timer.LogResponseExpected(result); return result; } -tl::expected, ErrorCode> +tl::expected, ErrorCode> MasterClient::OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading) { ScopedVLogTimer timer(1, "MasterClient::OffloadObjectHeartbeat"); timer.LogRequest("client_id=", client_id, ", enable_offloading=", enable_offloading); - auto result = invoke_rpc<&WrappedMasterService::OffloadObjectHeartbeat, - std::unordered_map>( - client_id, enable_offloading); + auto result = + invoke_rpc<&WrappedMasterService::OffloadObjectHeartbeat, + std::vector>(client_id, enable_offloading); return result; } @@ -967,56 +993,91 @@ tl::expected MasterClient::ReportSsdCapacity( tl::expected MasterClient::NotifyOffloadSuccess( const UUID& client_id, const std::vector& keys, const std::vector& metadatas) { + std::vector tasks; + tasks.reserve(keys.size()); + for (const auto& key : keys) { + tasks.push_back( + OffloadTaskItem{.tenant_id = tenant_id_, .key = key, .size = 0}); + } + return NotifyOffloadSuccess(client_id, tasks, metadatas); +} + +tl::expected MasterClient::NotifyOffloadSuccess( + const UUID& client_id, const std::vector& tasks, + const std::vector& metadatas) { ScopedVLogTimer timer(1, "MasterClient::NotifyOffloadSuccess"); - timer.LogRequest("client_id=", client_id, ", keys_count=", keys.size(), + timer.LogRequest("client_id=", client_id, ", tasks_count=", tasks.size(), ", metadatas_count=", metadatas.size()); auto result = invoke_rpc<&WrappedMasterService::NotifyOffloadSuccess, void>( - client_id, keys, metadatas); + client_id, tasks, metadatas); timer.LogResponseExpected(result); return result; } -tl::expected, ErrorCode> +tl::expected, ErrorCode> MasterClient::PromotionObjectHeartbeat(const UUID& client_id) { ScopedVLogTimer timer(1, "MasterClient::PromotionObjectHeartbeat"); timer.LogRequest("client_id=", client_id); return invoke_rpc<&WrappedMasterService::PromotionObjectHeartbeat, - std::unordered_map>(client_id); + std::vector>(client_id); } tl::expected MasterClient::PromotionAllocStart( const UUID& client_id, const std::string& key, uint64_t size, const std::vector& preferred_segments) { + return PromotionAllocStart(client_id, key, tenant_id_, size, + preferred_segments); +} + +tl::expected +MasterClient::PromotionAllocStart( + const UUID& client_id, const std::string& key, const std::string& tenant_id, + uint64_t size, const std::vector& preferred_segments) { ScopedVLogTimer timer(1, "MasterClient::PromotionAllocStart"); - timer.LogRequest("client_id=", client_id, ", key=", key, ", size=", size, + timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id, ", size=", size, ", preferred_count=", preferred_segments.size()); auto result = invoke_rpc<&WrappedMasterService::PromotionAllocStart, - PromotionAllocStartResponse>(client_id, key, size, - preferred_segments); + PromotionAllocStartResponse>( + client_id, key, tenant_id, size, preferred_segments); timer.LogResponseExpected(result); return result; } tl::expected MasterClient::NotifyPromotionSuccess( const UUID& client_id, const std::string& key) { + return NotifyPromotionSuccess(client_id, key, tenant_id_); +} + +tl::expected MasterClient::NotifyPromotionSuccess( + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "MasterClient::NotifyPromotionSuccess"); - timer.LogRequest("client_id=", client_id, ", key=", key); + timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id); auto result = invoke_rpc<&WrappedMasterService::NotifyPromotionSuccess, void>( - client_id, key); + client_id, key, tenant_id); timer.LogResponseExpected(result); return result; } tl::expected MasterClient::NotifyPromotionFailure( const UUID& client_id, const std::string& key) { + return NotifyPromotionFailure(client_id, key, tenant_id_); +} + +tl::expected MasterClient::NotifyPromotionFailure( + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "MasterClient::NotifyPromotionFailure"); - timer.LogRequest("client_id=", client_id, ", key=", key); + timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id); auto result = invoke_rpc<&WrappedMasterService::NotifyPromotionFailure, void>( - client_id, key); + client_id, key, tenant_id); timer.LogResponseExpected(result); return result; } @@ -1024,13 +1085,21 @@ tl::expected MasterClient::NotifyPromotionFailure( tl::expected MasterClient::CopyStart( const std::string& key, const std::string& src_segment, const std::vector& tgt_segments) { + return CopyStart(key, tenant_id_, src_segment, tgt_segments); +} + +tl::expected MasterClient::CopyStart( + const std::string& key, const std::string& tenant_id, + const std::string& src_segment, + const std::vector& tgt_segments) { ScopedVLogTimer timer(1, "MasterClient::CopyStart"); - timer.LogRequest("key=", key, ", src_segment=", src_segment, + timer.LogRequest("key=", key, ", tenant_id=", tenant_id, + ", src_segment=", src_segment, ", tgt_segments_count=", tgt_segments.size()); auto result = invoke_rpc<&WrappedMasterService::CopyStart, CopyStartResponse>( - client_id_, key, src_segment, tgt_segments); + client_id_, key, tenant_id, src_segment, tgt_segments); timer.LogResponseExpected(result); return result; } @@ -1048,11 +1117,16 @@ tl::expected MasterClient::QueryTask( } tl::expected MasterClient::CopyEnd(const std::string& key) { - ScopedVLogTimer timer(1, "MasterClient::CopyEnd"); - timer.LogRequest("key=", key); + return CopyEnd(key, tenant_id_); +} - auto result = - invoke_rpc<&WrappedMasterService::CopyEnd, void>(client_id_, key); +tl::expected MasterClient::CopyEnd( + const std::string& key, const std::string& tenant_id) { + ScopedVLogTimer timer(1, "MasterClient::CopyEnd"); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id); + + auto result = invoke_rpc<&WrappedMasterService::CopyEnd, void>( + client_id_, key, tenant_id); timer.LogResponseExpected(result); return result; } @@ -1069,11 +1143,16 @@ tl::expected, ErrorCode> MasterClient::FetchTasks( } tl::expected MasterClient::CopyRevoke(const std::string& key) { - ScopedVLogTimer timer(1, "MasterClient::CopyRevoke"); - timer.LogRequest("key=", key); + return CopyRevoke(key, tenant_id_); +} - auto result = - invoke_rpc<&WrappedMasterService::CopyRevoke, void>(client_id_, key); +tl::expected MasterClient::CopyRevoke( + const std::string& key, const std::string& tenant_id) { + ScopedVLogTimer timer(1, "MasterClient::CopyRevoke"); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id); + + auto result = invoke_rpc<&WrappedMasterService::CopyRevoke, void>( + client_id_, key, tenant_id); timer.LogResponseExpected(result); return result; } @@ -1081,33 +1160,50 @@ tl::expected MasterClient::CopyRevoke(const std::string& key) { tl::expected MasterClient::MoveStart( const std::string& key, const std::string& src_segment, const std::string& tgt_segment) { + return MoveStart(key, tenant_id_, src_segment, tgt_segment); +} + +tl::expected MasterClient::MoveStart( + const std::string& key, const std::string& tenant_id, + const std::string& src_segment, const std::string& tgt_segment) { ScopedVLogTimer timer(1, "MasterClient::MoveStart"); - timer.LogRequest("key=", key, ", src_segment=", src_segment, + timer.LogRequest("key=", key, ", tenant_id=", tenant_id, + ", src_segment=", src_segment, ", tgt_segment=", tgt_segment); auto result = invoke_rpc<&WrappedMasterService::MoveStart, MoveStartResponse>( - client_id_, key, src_segment, tgt_segment); + client_id_, key, tenant_id, src_segment, tgt_segment); timer.LogResponseExpected(result); return result; } tl::expected MasterClient::MoveEnd(const std::string& key) { - ScopedVLogTimer timer(1, "MasterClient::MoveEnd"); - timer.LogRequest("key=", key); + return MoveEnd(key, tenant_id_); +} - auto result = - invoke_rpc<&WrappedMasterService::MoveEnd, void>(client_id_, key); +tl::expected MasterClient::MoveEnd( + const std::string& key, const std::string& tenant_id) { + ScopedVLogTimer timer(1, "MasterClient::MoveEnd"); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id); + + auto result = invoke_rpc<&WrappedMasterService::MoveEnd, void>( + client_id_, key, tenant_id); timer.LogResponseExpected(result); return result; } tl::expected MasterClient::MoveRevoke(const std::string& key) { - ScopedVLogTimer timer(1, "MasterClient::MoveRevoke"); - timer.LogRequest("key=", key); + return MoveRevoke(key, tenant_id_); +} - auto result = - invoke_rpc<&WrappedMasterService::MoveRevoke, void>(client_id_, key); +tl::expected MasterClient::MoveRevoke( + const std::string& key, const std::string& tenant_id) { + ScopedVLogTimer timer(1, "MasterClient::MoveRevoke"); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id); + + auto result = invoke_rpc<&WrappedMasterService::MoveRevoke, void>( + client_id_, key, tenant_id); timer.LogResponseExpected(result); return result; } @@ -1124,24 +1220,37 @@ tl::expected MasterClient::MarkTaskToComplete( tl::expected MasterClient::EvictDiskReplica( const std::string& key, ReplicaType replica_type) { + return EvictDiskReplica(key, tenant_id_, replica_type); +} + +tl::expected MasterClient::EvictDiskReplica( + const std::string& key, const std::string& tenant_id, + ReplicaType replica_type) { ScopedVLogTimer timer(1, "MasterClient::EvictDiskReplica"); - timer.LogRequest("key=", key, ", replica_type=", replica_type); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id, + ", replica_type=", replica_type); auto result = invoke_rpc<&WrappedMasterService::EvictDiskReplica, void>( - client_id_, key, replica_type); + client_id_, key, tenant_id, replica_type); timer.LogResponseExpected(result); return result; } std::vector> MasterClient::BatchEvictDiskReplica( const std::vector& keys, ReplicaType replica_type) { + return BatchEvictDiskReplica(keys, tenant_id_, replica_type); +} + +std::vector> MasterClient::BatchEvictDiskReplica( + const std::vector& keys, const std::string& tenant_id, + ReplicaType replica_type) { ScopedVLogTimer timer(1, "MasterClient::BatchEvictDiskReplica"); - timer.LogRequest("keys_count=", keys.size(), + timer.LogRequest("keys_count=", keys.size(), ", tenant_id=", tenant_id, ", replica_type=", replica_type); auto result = invoke_batch_rpc<&WrappedMasterService::BatchEvictDiskReplica, void>( - keys.size(), client_id_, keys, replica_type); + keys.size(), client_id_, keys, tenant_id, replica_type); timer.LogResponse("result=", result.size(), " operations"); return result; } diff --git a/mooncake-store/src/master_service.cpp b/mooncake-store/src/master_service.cpp index 77f734d9..20635b52 100644 --- a/mooncake-store/src/master_service.cpp +++ b/mooncake-store/src/master_service.cpp @@ -1728,8 +1728,14 @@ auto MasterService::PutEnd(const UUID& client_id, const std::string& key, auto MasterService::AddReplica(const UUID& client_id, const std::string& key, Replica& replica) -> tl::expected { + return AddReplica(client_id, key, "default", replica); +} + +auto MasterService::AddReplica(const UUID& client_id, const std::string& key, + const std::string& tenant_id, Replica& replica) + -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, MakeObjectIdentity(key, tenant_id)); if (!accessor.Exists()) { accessor.Create( client_id, @@ -2238,9 +2244,19 @@ auto MasterService::EvictDiskReplica(const UUID& client_id, const std::string& key, ReplicaType replica_type) -> tl::expected { - MetadataAccessorRW accessor(this, key); + return EvictDiskReplica(client_id, key, "default", replica_type); +} + +auto MasterService::EvictDiskReplica(const UUID& client_id, + const std::string& key, + const std::string& tenant_id, + ReplicaType replica_type) + -> tl::expected { + const auto object_id = MakeObjectIdentity(key, tenant_id); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { - LOG(INFO) << "key=" << key << ", info=object_not_found_for_eviction"; + LOG(INFO) << "key=" << key << ", tenant_id=" << object_id.tenant_id + << ", info=object_not_found_for_eviction"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); } @@ -2272,10 +2288,17 @@ auto MasterService::EvictDiskReplica(const UUID& client_id, std::vector> MasterService::BatchEvictDiskReplica( const UUID& client_id, const std::vector& keys, ReplicaType replica_type) { + return BatchEvictDiskReplica(client_id, keys, "default", replica_type); +} + +std::vector> MasterService::BatchEvictDiskReplica( + const UUID& client_id, const std::vector& keys, + const std::string& tenant_id, ReplicaType replica_type) { std::vector> results; results.reserve(keys.size()); for (const auto& key : keys) { - results.push_back(EvictDiskReplica(client_id, key, replica_type)); + results.push_back( + EvictDiskReplica(client_id, key, tenant_id, replica_type)); } return results; } @@ -2284,7 +2307,15 @@ tl::expected MasterService::CopyStart( const UUID& client_id, const std::string& key, const std::string& src_segment, const std::vector& tgt_segments) { + return CopyStart(client_id, key, "default", src_segment, tgt_segments); +} + +tl::expected MasterService::CopyStart( + const UUID& client_id, const std::string& key, const std::string& tenant_id, + const std::string& src_segment, + const std::vector& tgt_segments) { std::shared_lock shared_lock(snapshot_mutex_); + const auto object_id = MakeObjectIdentity(key, tenant_id); { ScopedSegmentAccess segment_access = segment_manager_.getSegmentAccess(); @@ -2302,7 +2333,7 @@ tl::expected MasterService::CopyStart( } } } - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { LOG(ERROR) << "key=" << key << ", object not found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -2378,8 +2409,14 @@ tl::expected MasterService::CopyStart( tl::expected MasterService::CopyEnd(const UUID& client_id, const std::string& key) { + return CopyEnd(client_id, key, "default"); +} + +tl::expected MasterService::CopyEnd( + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, MakeObjectIdentity(key, tenant_id)); if (!accessor.Exists()) { LOG(ERROR) << "key=" << key << ", error=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -2449,8 +2486,14 @@ tl::expected MasterService::CopyEnd(const UUID& client_id, tl::expected MasterService::CopyRevoke( const UUID& client_id, const std::string& key) { + return CopyRevoke(client_id, key, "default"); +} + +tl::expected MasterService::CopyRevoke( + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, MakeObjectIdentity(key, tenant_id)); if (!accessor.Exists()) { LOG(ERROR) << "key=" << key << ", error=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -2503,7 +2546,14 @@ tl::expected MasterService::CopyRevoke( tl::expected MasterService::MoveStart( const UUID& client_id, const std::string& key, const std::string& src_segment, const std::string& tgt_segment) { + return MoveStart(client_id, key, "default", src_segment, tgt_segment); +} + +tl::expected MasterService::MoveStart( + const UUID& client_id, const std::string& key, const std::string& tenant_id, + const std::string& src_segment, const std::string& tgt_segment) { std::shared_lock shared_lock(snapshot_mutex_); + const auto object_id = MakeObjectIdentity(key, tenant_id); if (src_segment == tgt_segment) { LOG(ERROR) << "key=" << key << ", move_tgt=" << tgt_segment << " cannot be the same as move_src=" << src_segment; @@ -2525,7 +2575,7 @@ tl::expected MasterService::MoveStart( } } - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { LOG(ERROR) << "key=" << key << ", object not found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -2593,8 +2643,14 @@ tl::expected MasterService::MoveStart( tl::expected MasterService::MoveEnd(const UUID& client_id, const std::string& key) { + return MoveEnd(client_id, key, "default"); +} + +tl::expected MasterService::MoveEnd( + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, MakeObjectIdentity(key, tenant_id)); if (!accessor.Exists()) { LOG(ERROR) << "key=" << key << ", error=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -2679,8 +2735,14 @@ tl::expected MasterService::MoveEnd(const UUID& client_id, tl::expected MasterService::MoveRevoke( const UUID& client_id, const std::string& key) { + return MoveRevoke(client_id, key, "default"); +} + +tl::expected MasterService::MoveRevoke( + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, MakeObjectIdentity(key, tenant_id)); if (!accessor.Exists()) { LOG(ERROR) << "key=" << key << ", error=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -3153,7 +3215,7 @@ auto MasterService::MountLocalDiskSegment(const UUID& client_id, auto MasterService::OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading) - -> tl::expected, ErrorCode> { + -> tl::expected, ErrorCode> { std::shared_lock shared_lock(snapshot_mutex_); ScopedLocalDiskSegmentAccess local_disk_segment_access = segment_manager_.getLocalDiskSegmentAccess(); @@ -3165,12 +3227,20 @@ auto MasterService::OffloadObjectHeartbeat(const UUID& client_id, << client_id; return tl::make_unexpected(ErrorCode::SEGMENT_NOT_FOUND); } - std::unordered_map offloading_objects_copy; + std::unordered_map offloading_objects_copy; { MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); local_disk_segment_it->second->enable_offloading = enable_offloading; if (enable_offloading) { - return std::move(local_disk_segment_it->second->offloading_objects); + std::vector result; + result.reserve( + local_disk_segment_it->second->offloading_objects.size()); + for (const auto& [_, task] : + local_disk_segment_it->second->offloading_objects) { + result.push_back(task); + } + local_disk_segment_it->second->offloading_objects.clear(); + return result; } // Offloading is disabled: clear the pending queue to prevent // unbounded growth that would trigger KEYS_ULTRA_LIMIT in @@ -3185,11 +3255,13 @@ auto MasterService::OffloadObjectHeartbeat(const UUID& client_id, std::move(local_disk_segment_it->second->offloading_objects); } - for (auto& [key, size] : offloading_objects_copy) { - MetadataAccessorRW accessor(this, key); + for (auto& [_, task] : offloading_objects_copy) { + const auto object_id = MakeObjectIdentity(task.key, task.tenant_id); + MetadataAccessorRW accessor(this, object_id); if (accessor.Exists()) { auto& tenant_state = accessor.GetTenantState(); - auto task_it = tenant_state.offloading_tasks.find(key); + auto task_it = + tenant_state.offloading_tasks.find(object_id.user_key); if (task_it != tenant_state.offloading_tasks.end()) { auto source = accessor.Get().GetReplicaByID(task_it->second.source_id); @@ -3242,17 +3314,35 @@ auto MasterService::NotifyOffloadSuccess( const UUID& client_id, const std::vector& keys, const std::vector& metadatas) -> tl::expected { - for (size_t i = 0; i < keys.size(); ++i) { - const auto& key = keys[i]; + std::vector tasks; + tasks.reserve(keys.size()); + for (const auto& key : keys) { + tasks.push_back( + OffloadTaskItem{.tenant_id = "default", .key = key, .size = 0}); + } + return NotifyOffloadSuccess(client_id, tasks, metadatas); +} + +auto MasterService::NotifyOffloadSuccess( + const UUID& client_id, const std::vector& tasks, + const std::vector& metadatas) + -> tl::expected { + if (tasks.size() != metadatas.size()) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + for (size_t i = 0; i < tasks.size(); ++i) { + const auto& task = tasks[i]; const auto& metadata = metadatas[i]; + const auto object_id = MakeObjectIdentity(task.key, task.tenant_id); // Release refcnt and clear offloading task. { - MetadataAccessorRW accessor(this, key); + MetadataAccessorRW accessor(this, object_id); if (accessor.Exists()) { auto& obj_metadata = accessor.Get(); auto& tenant_state = accessor.GetTenantState(); - auto task_it = tenant_state.offloading_tasks.find(key); + auto task_it = + tenant_state.offloading_tasks.find(object_id.user_key); if (task_it != tenant_state.offloading_tasks.end()) { auto source = obj_metadata.GetReplicaByID(task_it->second.source_id); @@ -3267,10 +3357,13 @@ auto MasterService::NotifyOffloadSuccess( // Add LOCAL_DISK replica. Replica replica(client_id, metadata.data_size, metadata.transport_endpoint, ReplicaStatus::COMPLETE); - auto res = AddReplica(client_id, key, replica); + auto res = AddReplica(client_id, object_id.user_key, + object_id.tenant_id, replica); if (!res && res.error() != ErrorCode::OBJECT_NOT_FOUND) { LOG(ERROR) << "Failed to add replica: error=" << res.error() - << ", client_id=" << client_id << ", key=" << key; + << ", client_id=" << client_id + << ", tenant_id=" << object_id.tenant_id + << ", key=" << object_id.user_key; return tl::make_unexpected(res.error()); } } @@ -3279,12 +3372,6 @@ auto MasterService::NotifyOffloadSuccess( tl::expected MasterService::PushOffloadingQueue( const ObjectIdentity& object_id, Replica& replica) { - if (object_id.tenant_id != "default") { - VLOG(1) << "key=" << object_id.user_key - << ", tenant_id=" << object_id.tenant_id - << ", action=skip_offload_for_non_default_tenant"; - return tl::make_unexpected(ErrorCode::UNABLE_OFFLOADING); - } const auto& segment_names = replica.get_segment_names(); if (segment_names.empty()) { return {}; @@ -3317,10 +3404,14 @@ tl::expected MasterService::PushOffloadingQueue( offloading_queue_limit_) { return tl::make_unexpected(ErrorCode::KEYS_ULTRA_LIMIT); } + const int64_t size = replica.get_descriptor() + .get_memory_descriptor() + .buffer_descriptor.size_; auto res = local_disk_segment_it->second->offloading_objects.emplace( - object_id.user_key, replica.get_descriptor() - .get_memory_descriptor() - .buffer_descriptor.size_); + MakeTenantScopedStorageKey(object_id.tenant_id, object_id.user_key), + OffloadTaskItem{.tenant_id = object_id.tenant_id, + .key = object_id.user_key, + .size = size}); if (!res.second) { return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); } @@ -3353,10 +3444,13 @@ tl::expected MasterService::PushPromotionQueue( } MutexLocker locker(&local_disk_segment_it->second->offloading_mutex_); auto res = local_disk_segment_it->second->promotion_objects.emplace( - object_id.user_key, - static_cast(source_replica.get_descriptor() - .get_local_disk_descriptor() - .object_size)); + MakeTenantScopedStorageKey(object_id.tenant_id, object_id.user_key), + PromotionTaskItem{ + .tenant_id = object_id.tenant_id, + .key = object_id.user_key, + .size = static_cast(source_replica.get_descriptor() + .get_local_disk_descriptor() + .object_size)}); if (!res.second) { return tl::make_unexpected(ErrorCode::OBJECT_ALREADY_EXISTS); } @@ -3466,7 +3560,7 @@ void MasterService::TryPushPromotionQueue(const ObjectIdentity& object_id) { } auto MasterService::PromotionObjectHeartbeat(const UUID& client_id) - -> tl::expected, ErrorCode> { + -> tl::expected, ErrorCode> { std::shared_lock shared_lock(snapshot_mutex_); ScopedLocalDiskSegmentAccess local_disk_segment_access = segment_manager_.getLocalDiskSegmentAccess(); @@ -3485,10 +3579,10 @@ auto MasterService::PromotionObjectHeartbeat(const UUID& client_id) // cap must live here (server side) rather than on the client so // leftover work isn't silently dropped. auto& src = local_disk_segment_it->second->promotion_objects; - std::unordered_map result; + std::vector result; while (result.size() < promotion_max_per_heartbeat_ && !src.empty()) { auto node = src.extract(src.begin()); - result.insert(std::move(node)); + result.push_back(std::move(node.mapped())); } return result; } @@ -3497,8 +3591,17 @@ auto MasterService::PromotionAllocStart( const UUID& client_id, const std::string& key, uint64_t size, const std::vector& preferred_segments) -> tl::expected { + return PromotionAllocStart(client_id, key, "default", size, + preferred_segments); +} + +auto MasterService::PromotionAllocStart( + const UUID& client_id, const std::string& key, const std::string& tenant_id, + uint64_t size, const std::vector& preferred_segments) + -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + const auto object_id = MakeObjectIdentity(key, tenant_id); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); } @@ -3516,7 +3619,7 @@ auto MasterService::PromotionAllocStart( // removed or evicted. The shard mutex is held for the rest of this // function, so the iterator stays valid across the allocation step. auto& tenant_state = accessor.GetTenantState(); - auto task_it = tenant_state.promotion_tasks.find(key); + auto task_it = tenant_state.promotion_tasks.find(object_id.user_key); if (task_it == tenant_state.promotion_tasks.end()) { return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); } @@ -3589,8 +3692,16 @@ auto MasterService::PromotionAllocStart( auto MasterService::NotifyPromotionSuccess(const UUID& client_id, const std::string& key) -> tl::expected { + return NotifyPromotionSuccess(client_id, key, "default"); +} + +auto MasterService::NotifyPromotionSuccess(const UUID& client_id, + const std::string& key, + const std::string& tenant_id) + -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + const auto object_id = MakeObjectIdentity(key, tenant_id); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); } @@ -3602,7 +3713,7 @@ auto MasterService::NotifyPromotionSuccess(const UUID& client_id, // replicas, so we must not just "mark first PROCESSING memory // complete" — that would risk committing someone else's half-written // replica. - auto task_it = tenant_state.promotion_tasks.find(key); + auto task_it = tenant_state.promotion_tasks.find(object_id.user_key); if (task_it == tenant_state.promotion_tasks.end() || task_it->second.alloc_id == 0) { return tl::make_unexpected(ErrorCode::REPLICA_IS_NOT_READY); @@ -3648,7 +3759,8 @@ auto MasterService::NotifyPromotionSuccess(const UUID& client_id, auto it = client_local_disk_segment.find(client_id); if (it != client_local_disk_segment.end()) { MutexLocker locker(&it->second->offloading_mutex_); - it->second->promotion_objects.erase(key); + it->second->promotion_objects.erase(MakeTenantScopedStorageKey( + object_id.tenant_id, object_id.user_key)); } } @@ -3661,15 +3773,23 @@ auto MasterService::NotifyPromotionSuccess(const UUID& client_id, auto MasterService::NotifyPromotionFailure(const UUID& client_id, const std::string& key) -> tl::expected { + return NotifyPromotionFailure(client_id, key, "default"); +} + +auto MasterService::NotifyPromotionFailure(const UUID& client_id, + const std::string& key, + const std::string& tenant_id) + -> tl::expected { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRW accessor(this, key); + const auto object_id = MakeObjectIdentity(key, tenant_id); + MetadataAccessorRW accessor(this, object_id); if (!accessor.Exists()) { return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); } auto& metadata = accessor.Get(); auto& tenant_state = accessor.GetTenantState(); - auto task_it = tenant_state.promotion_tasks.find(key); + auto task_it = tenant_state.promotion_tasks.find(object_id.user_key); if (task_it == tenant_state.promotion_tasks.end()) { // No task to release. Either the reaper already swept it, or the // client never had a task here. Return OK to keep this RPC @@ -3708,7 +3828,8 @@ auto MasterService::NotifyPromotionFailure(const UUID& client_id, auto it = client_local_disk_segment.find(client_id); if (it != client_local_disk_segment.end()) { MutexLocker locker(&it->second->offloading_mutex_); - it->second->promotion_objects.erase(key); + it->second->promotion_objects.erase(MakeTenantScopedStorageKey( + object_id.tenant_id, object_id.user_key)); } } @@ -5049,10 +5170,6 @@ void MasterService::BatchEvict(double evict_ratio_target, return metadata.size * evict_replicas(metadata); } - if (tenant_id != "default") { - return 0; - } - // LOCAL_DISK replica already exists — safe to delete MEMORY immediately if (has_local_disk_replica(metadata)) { return metadata.size * evict_replicas(metadata); @@ -6442,12 +6559,19 @@ std::string MasterService::FormatTimestamp( tl::expected MasterService::CreateCopyTask( const std::string& key, const std::vector& targets) { + return CreateCopyTask(key, "default", targets); +} + +tl::expected MasterService::CreateCopyTask( + const std::string& key, const std::string& tenant_id, + const std::vector& targets) { std::shared_lock shared_lock(snapshot_mutex_); + const auto object_id = MakeObjectIdentity(key, tenant_id); if (targets.empty()) { LOG(ERROR) << "key=" << key << ", error=empty_targets"; return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } - MetadataAccessorRO accessor(this, key); + MetadataAccessorRO accessor(this, object_id); if (!accessor.Exists()) { VLOG(1) << "key=" << key << ", info=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -6490,7 +6614,8 @@ tl::expected MasterService::CreateCopyTask( } return task_manager_.get_write_access() .submit_task_typed( - select_client, {.key = key, + select_client, {.tenant_id = object_id.tenant_id, + .key = object_id.user_key, .source = selected_source_segment, .targets = targets}); } @@ -6498,8 +6623,15 @@ tl::expected MasterService::CreateCopyTask( tl::expected MasterService::CreateMoveTask( const std::string& key, const std::string& source, const std::string& target) { + return CreateMoveTask(key, "default", source, target); +} + +tl::expected MasterService::CreateMoveTask( + const std::string& key, const std::string& tenant_id, + const std::string& source, const std::string& target) { std::shared_lock shared_lock(snapshot_mutex_); - MetadataAccessorRO accessor(this, key); + const auto object_id = MakeObjectIdentity(key, tenant_id); + MetadataAccessorRO accessor(this, object_id); if (!accessor.Exists()) { VLOG(1) << "key=" << key << ", info=object_not_found"; return tl::make_unexpected(ErrorCode::OBJECT_NOT_FOUND); @@ -6545,7 +6677,10 @@ tl::expected MasterService::CreateMoveTask( return task_manager_.get_write_access() .submit_task_typed( - select_client, {.key = key, .source = source, .target = target}); + select_client, {.tenant_id = object_id.tenant_id, + .key = object_id.user_key, + .source = source, + .target = target}); } tl::expected MasterService::QueryTask( @@ -6851,6 +6986,7 @@ void MasterService::ScheduleDrainJobTasks(DrainJob& job) { } struct DrainPlan { + std::string tenant_id; std::string key; std::string source_segment; std::string target_segment; @@ -6890,7 +7026,7 @@ void MasterService::ScheduleDrainJobTasks(DrainJob& job) { continue; } - if (tenant_id != "default" || metadata.IsHardPinned() || + if (metadata.IsHardPinned() || !metadata.IsLeaseExpired() || !metadata.AllReplicas(&Replica::fn_is_completed) || tenant_state.replication_tasks.contains(key)) { @@ -6907,8 +7043,8 @@ void MasterService::ScheduleDrainJobTasks(DrainJob& job) { } if (plans.size() < slots) { - plans.push_back({key, source_segment, *target, - metadata.size, unit_key}); + plans.push_back({tenant_id, key, source_segment, + *target, metadata.size, unit_key}); } } } @@ -6919,11 +7055,12 @@ void MasterService::ScheduleDrainJobTasks(DrainJob& job) { job.blocked_units = blocked_unit_keys.size(); for (const auto& plan : plans) { - auto task_id = - CreateMoveTask(plan.key, plan.source_segment, plan.target_segment); + auto task_id = CreateMoveTask(plan.key, plan.tenant_id, + plan.source_segment, plan.target_segment); if (task_id.has_value()) { ActiveDrainTask active_task; active_task.task_id = task_id.value(); + active_task.tenant_id = plan.tenant_id; active_task.key = plan.key; active_task.source_segment = plan.source_segment; active_task.target_segment = plan.target_segment; diff --git a/mooncake-store/src/real_client.cpp b/mooncake-store/src/real_client.cpp index 9b41e02a..975f04f1 100644 --- a/mooncake-store/src/real_client.cpp +++ b/mooncake-store/src/real_client.cpp @@ -5483,15 +5483,18 @@ RealClient::batch_get_into_offload_object_internal( offload_rpc_read_count_.fetch_add(1, std::memory_order_relaxed); auto start_time = std::chrono::steady_clock::now(); std::vector keys; + std::vector storage_keys; std::vector sizes; for (const auto &object_it : objects) { keys.emplace_back(object_it.first); + storage_keys.emplace_back( + MakeTenantScopedStorageKey(client_->tenant_id(), object_it.first)); int64_t total = 0; for (const auto &s : object_it.second) total += s.size; sizes.emplace_back(total); } auto batchGetResp = client_requester_->batch_get_offload_object( - target_rpc_service_addr, keys, sizes); + target_rpc_service_addr, storage_keys, sizes); if (!batchGetResp) { LOG(ERROR) << "Batch get offload object failed with error: " << batchGetResp.error(); diff --git a/mooncake-store/src/rpc_service.cpp b/mooncake-store/src/rpc_service.cpp index ed94a3f3..c5831d22 100644 --- a/mooncake-store/src/rpc_service.cpp +++ b/mooncake-store/src/rpc_service.cpp @@ -1535,17 +1535,18 @@ WrappedMasterService::GetNoFSegmentsByName(const std::string& segment_name) { } tl::expected WrappedMasterService::CopyStart( - const UUID& client_id, const std::string& key, + const UUID& client_id, const std::string& key, const std::string& tenant_id, const std::string& src_segment, const std::vector& tgt_segments) { return execute_rpc( "CopyStart", [&] { - return master_service_.CopyStart(client_id, key, src_segment, - tgt_segments); + return master_service_.CopyStart(client_id, key, tenant_id, + src_segment, tgt_segments); }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id, ", src_segment=", src_segment, ", tgt_segments_count=", tgt_segments.size()); }, @@ -1554,39 +1555,45 @@ tl::expected WrappedMasterService::CopyStart( } tl::expected WrappedMasterService::CopyEnd( - const UUID& client_id, const std::string& key) { + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { return execute_rpc( - "CopyEnd", [&] { return master_service_.CopyEnd(client_id, key); }, + "CopyEnd", + [&] { return master_service_.CopyEnd(client_id, key, tenant_id); }, [&](auto& timer) { - timer.LogRequest("client_id=", client_id, ", key=", key); + timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id); }, [] { MasterMetricManager::instance().inc_copy_end_requests(); }, [] { MasterMetricManager::instance().inc_copy_end_failures(); }); } tl::expected WrappedMasterService::CopyRevoke( - const UUID& client_id, const std::string& key) { + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { return execute_rpc( "CopyRevoke", - [&] { return master_service_.CopyRevoke(client_id, key); }, + [&] { return master_service_.CopyRevoke(client_id, key, tenant_id); }, [&](auto& timer) { - timer.LogRequest("client_id=", client_id, ", key=", key); + timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id); }, [] { MasterMetricManager::instance().inc_copy_revoke_requests(); }, [] { MasterMetricManager::instance().inc_copy_revoke_failures(); }); } tl::expected WrappedMasterService::MoveStart( - const UUID& client_id, const std::string& key, + const UUID& client_id, const std::string& key, const std::string& tenant_id, const std::string& src_segment, const std::string& tgt_segment) { return execute_rpc( "MoveStart", [&] { - return master_service_.MoveStart(client_id, key, src_segment, - tgt_segment); + return master_service_.MoveStart(client_id, key, tenant_id, + src_segment, tgt_segment); }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id, ", src_segment=", src_segment, ", tgt_segment=", tgt_segment); }, @@ -1595,38 +1602,45 @@ tl::expected WrappedMasterService::MoveStart( } tl::expected WrappedMasterService::MoveEnd( - const UUID& client_id, const std::string& key) { + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { return execute_rpc( - "MoveEnd", [&] { return master_service_.MoveEnd(client_id, key); }, + "MoveEnd", + [&] { return master_service_.MoveEnd(client_id, key, tenant_id); }, [&](auto& timer) { - timer.LogRequest("client_id=", client_id, ", key=", key); + timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id); }, [] { MasterMetricManager::instance().inc_move_end_requests(); }, [] { MasterMetricManager::instance().inc_move_end_failures(); }); } tl::expected WrappedMasterService::MoveRevoke( - const UUID& client_id, const std::string& key) { + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { return execute_rpc( "MoveRevoke", - [&] { return master_service_.MoveRevoke(client_id, key); }, + [&] { return master_service_.MoveRevoke(client_id, key, tenant_id); }, [&](auto& timer) { - timer.LogRequest("client_id=", client_id, ", key=", key); + timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id); }, [] { MasterMetricManager::instance().inc_move_revoke_requests(); }, [] { MasterMetricManager::instance().inc_move_revoke_failures(); }); } tl::expected WrappedMasterService::EvictDiskReplica( - const UUID& client_id, const std::string& key, ReplicaType replica_type) { + const UUID& client_id, const std::string& key, const std::string& tenant_id, + ReplicaType replica_type) { return execute_rpc( "EvictDiskReplica", [&] { - return master_service_.EvictDiskReplica(client_id, key, + return master_service_.EvictDiskReplica(client_id, key, tenant_id, replica_type); }, [&](auto& timer) { timer.LogRequest("client_id=", client_id, ", key=", key, + ", tenant_id=", tenant_id, ", replica_type=", replica_type); }, [] { @@ -1640,15 +1654,16 @@ tl::expected WrappedMasterService::EvictDiskReplica( std::vector> WrappedMasterService::BatchEvictDiskReplica( const UUID& client_id, const std::vector& keys, - ReplicaType replica_type) { + const std::string& tenant_id, ReplicaType replica_type) { ScopedVLogTimer timer(1, "BatchEvictDiskReplica"); const size_t total_keys = keys.size(); timer.LogRequest("client_id=", client_id, ", keys_count=", total_keys, + ", tenant_id=", tenant_id, ", replica_type=", replica_type); MasterMetricManager::instance().inc_evict_disk_replica_requests(); - auto results = - master_service_.BatchEvictDiskReplica(client_id, keys, replica_type); + auto results = master_service_.BatchEvictDiskReplica( + client_id, keys, tenant_id, replica_type); size_t failure_count = 0; for (size_t i = 0; i < results.size(); ++i) { @@ -1670,12 +1685,14 @@ WrappedMasterService::BatchEvictDiskReplica( } tl::expected WrappedMasterService::CreateCopyTask( - const std::string& key, const std::vector& targets) { + const std::string& key, const std::string& tenant_id, + const std::vector& targets) { return execute_rpc( "CreateCopyTask", - [&] { return master_service_.CreateCopyTask(key, targets); }, + [&] { return master_service_.CreateCopyTask(key, tenant_id, targets); }, [&](auto& timer) { - timer.LogRequest("key=", key, ", targets_size=", targets.size()); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id, + ", targets_size=", targets.size()); }, [] { MasterMetricManager::instance().inc_create_copy_task_requests(); }, [] { @@ -1684,14 +1701,17 @@ tl::expected WrappedMasterService::CreateCopyTask( } tl::expected WrappedMasterService::CreateMoveTask( - const std::string& key, const std::string& source, - const std::string& target) { + const std::string& key, const std::string& tenant_id, + const std::string& source, const std::string& target) { return execute_rpc( "CreateMoveTask", - [&] { return master_service_.CreateMoveTask(key, source, target); }, + [&] { + return master_service_.CreateMoveTask(key, tenant_id, source, + target); + }, [&](auto& timer) { - timer.LogRequest("key=", key, ", source=", source, - ", target=", target); + timer.LogRequest("key=", key, ", tenant_id=", tenant_id, + ", source=", source, ", target=", target); }, [] { MasterMetricManager::instance().inc_create_move_task_requests(); }, [] { @@ -1799,8 +1819,7 @@ tl::expected WrappedMasterService::MountLocalDiskSegment( return result; } -tl::expected>, - ErrorCode> +tl::expected, ErrorCode> WrappedMasterService::OffloadObjectHeartbeat(const UUID& client_id, bool enable_offloading) { ScopedVLogTimer timer(1, "OffloadObjectHeartbeat"); @@ -1820,18 +1839,18 @@ tl::expected WrappedMasterService::ReportSsdCapacity( } tl::expected WrappedMasterService::NotifyOffloadSuccess( - const UUID& client_id, const std::vector& keys, + const UUID& client_id, const std::vector& tasks, const std::vector& metadatas) { ScopedVLogTimer timer(1, "NotifyOffloadSuccess"); timer.LogRequest("action=notify_offload_success"); auto result = - master_service_.NotifyOffloadSuccess(client_id, keys, metadatas); + master_service_.NotifyOffloadSuccess(client_id, tasks, metadatas); timer.LogResponseExpected(result); return result; } -tl::expected, ErrorCode> +tl::expected, ErrorCode> WrappedMasterService::PromotionObjectHeartbeat(const UUID& client_id) { ScopedVLogTimer timer(1, "PromotionObjectHeartbeat"); timer.LogRequest("action=promotion_object_heartbeat"); @@ -1840,30 +1859,34 @@ WrappedMasterService::PromotionObjectHeartbeat(const UUID& client_id) { tl::expected WrappedMasterService::PromotionAllocStart( - const UUID& client_id, const std::string& key, uint64_t size, - const std::vector& preferred_segments) { + const UUID& client_id, const std::string& key, const std::string& tenant_id, + uint64_t size, const std::vector& preferred_segments) { ScopedVLogTimer timer(1, "PromotionAllocStart"); timer.LogRequest("action=promotion_alloc_start"); - auto result = master_service_.PromotionAllocStart(client_id, key, size, - preferred_segments); + auto result = master_service_.PromotionAllocStart(client_id, key, tenant_id, + size, preferred_segments); timer.LogResponseExpected(result); return result; } tl::expected WrappedMasterService::NotifyPromotionSuccess( - const UUID& client_id, const std::string& key) { + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "NotifyPromotionSuccess"); timer.LogRequest("action=notify_promotion_success"); - auto result = master_service_.NotifyPromotionSuccess(client_id, key); + auto result = + master_service_.NotifyPromotionSuccess(client_id, key, tenant_id); timer.LogResponseExpected(result); return result; } tl::expected WrappedMasterService::NotifyPromotionFailure( - const UUID& client_id, const std::string& key) { + const UUID& client_id, const std::string& key, + const std::string& tenant_id) { ScopedVLogTimer timer(1, "NotifyPromotionFailure"); timer.LogRequest("action=notify_promotion_failure"); - auto result = master_service_.NotifyPromotionFailure(client_id, key); + auto result = + master_service_.NotifyPromotionFailure(client_id, key, tenant_id); timer.LogResponseExpected(result); return result; } diff --git a/mooncake-store/src/segment.cpp b/mooncake-store/src/segment.cpp index b5b9b024..57769c8c 100644 --- a/mooncake-store/src/segment.cpp +++ b/mooncake-store/src/segment.cpp @@ -20,6 +20,11 @@ bool HasAllocator(const AllocatorManager& allocator_manager, allocators->end(); } +bool IsMsgpackInteger(const msgpack::object& object) { + return object.type == msgpack::type::POSITIVE_INTEGER || + object.type == msgpack::type::NEGATIVE_INTEGER; +} + } // namespace ErrorCode ScopedSegmentAccess::MountSegment(const Segment& segment, @@ -573,10 +578,10 @@ SegmentSerializer::Serialize() { segment_manager_->client_local_disk_segment_.at(client_uuid); packer.pack(UuidToString(client_uuid)); - // Serialize LocalDiskSegment: [enable_offloading, count, key1, ts1, - // key2, ts2, ...] Sort keys to ensure determinism + // Serialize LocalDiskSegment: [enable_offloading, count, storage_key1, + // task1, storage_key2, task2, ...] Sort keys to ensure determinism. std::vector sorted_keys; - for (const auto& [key, ts] : segment->offloading_objects) { + for (const auto& [key, _] : segment->offloading_objects) { sorted_keys.push_back(key); } std::sort(sorted_keys.begin(), sorted_keys.end()); @@ -587,7 +592,11 @@ SegmentSerializer::Serialize() { for (const auto& key : sorted_keys) { packer.pack(key); - packer.pack(segment->offloading_objects.at(key)); + const auto& task = segment->offloading_objects.at(key); + packer.pack_array(3); + packer.pack(task.tenant_id); + packer.pack(task.key); + packer.pack(task.size); } } @@ -902,8 +911,8 @@ tl::expected SegmentSerializer::Deserialize( client_uuid_str))); } - // Parse LocalDiskSegment array: [enable_offloading, count, key1, - // ts1, ...] + // Parse LocalDiskSegment array: [enable_offloading, count, + // storage_key1, task1, ...] if (client_value.type != msgpack::type::ARRAY || client_value.via.array.size < 2) { return tl::unexpected( @@ -921,19 +930,62 @@ tl::expected SegmentSerializer::Deserialize( // Parse offloading_objects for (uint64_t k = 0; k < count; ++k) { size_t key_idx = 2 + k * 2; - size_t ts_idx = 2 + k * 2 + 1; - if (ts_idx >= client_value.via.array.size) { + size_t task_idx = 2 + k * 2 + 1; + if (task_idx >= client_value.via.array.size) { return tl::unexpected( SerializationError(ErrorCode::DESERIALIZE_FAIL, "deserialize local_disk_segments " "offloading_objects out of bounds")); } + if (client_value.via.array.ptr[key_idx].type != + msgpack::type::STR) { + return tl::unexpected(SerializationError( + ErrorCode::DESERIALIZE_FAIL, + "deserialize local_disk_segments offloading key is " + "not string")); + } std::string key( client_value.via.array.ptr[key_idx].via.str.ptr, client_value.via.array.ptr[key_idx].via.str.size); - int64_t ts = client_value.via.array.ptr[ts_idx].as(); - segment->offloading_objects[key] = ts; + const auto& task_obj = client_value.via.array.ptr[task_idx]; + if (task_obj.type == msgpack::type::ARRAY && + task_obj.via.array.size == 3) { + if (task_obj.via.array.ptr[0].type != msgpack::type::STR || + task_obj.via.array.ptr[1].type != msgpack::type::STR) { + return tl::unexpected(SerializationError( + ErrorCode::DESERIALIZE_FAIL, + "deserialize local_disk_segments offloading task " + "fields are not strings")); + } + if (!IsMsgpackInteger(task_obj.via.array.ptr[2])) { + return tl::unexpected(SerializationError( + ErrorCode::DESERIALIZE_FAIL, + "deserialize local_disk_segments offloading task " + "size is not integer")); + } + OffloadTaskItem task; + task.tenant_id = + task_obj.via.array.ptr[0].as(); + task.key = task_obj.via.array.ptr[1].as(); + task.size = task_obj.via.array.ptr[2].as(); + segment->offloading_objects[key] = std::move(task); + } else { + // Backward compatibility for snapshots whose + // offloading_objects value was key -> size. + if (!IsMsgpackInteger(task_obj)) { + return tl::unexpected(SerializationError( + ErrorCode::DESERIALIZE_FAIL, + "deserialize local_disk_segments legacy " + "offloading size is not integer")); + } + auto [tenant_id, user_key] = + ParseTenantScopedStorageKey(key); + segment->offloading_objects[key] = + OffloadTaskItem{.tenant_id = std::move(tenant_id), + .key = std::move(user_key), + .size = task_obj.as()}; + } } segment_manager_->client_local_disk_segment_[client_id] = diff --git a/mooncake-store/src/utils.cpp b/mooncake-store/src/utils.cpp index 810a7cf8..d1f03549 100644 --- a/mooncake-store/src/utils.cpp +++ b/mooncake-store/src/utils.cpp @@ -559,8 +559,9 @@ static std::string SanitizeKey(const std::string &key) { for (char c : key) { // Replace invalid characters with underscore - sanitized_key.push_back( - kInvalidChars.find(c) != std::string_view::npos ? '_' : c); + const bool invalid = + c == '\0' || kInvalidChars.find(c) != std::string_view::npos; + sanitized_key.push_back(invalid ? '_' : c); } return sanitized_key; } diff --git a/mooncake-store/tests/file_storage_promotion_test.cpp b/mooncake-store/tests/file_storage_promotion_test.cpp index 187f00d6..a553d92d 100644 --- a/mooncake-store/tests/file_storage_promotion_test.cpp +++ b/mooncake-store/tests/file_storage_promotion_test.cpp @@ -30,12 +30,12 @@ class FakeClient : public Client { /*labels=*/{}) {} // Drives the queue returned to the heartbeat caller. - std::unordered_map heartbeat_queue; + std::vector heartbeat_queue; tl::expected heartbeat_result = tl::expected{}; tl::expected PromotionObjectHeartbeat( - std::unordered_map& promotion_objects) override { + std::vector& promotion_objects) override { heartbeat_calls.fetch_add(1); if (!heartbeat_result.has_value()) { return tl::make_unexpected(heartbeat_result.error()); @@ -50,8 +50,8 @@ class FakeClient : public Client { promotion_objects.clear(); while (promotion_objects.size() < kMaxPerHeartbeat && !heartbeat_queue.empty()) { - auto node = heartbeat_queue.extract(heartbeat_queue.begin()); - promotion_objects.insert(std::move(node)); + promotion_objects.push_back(std::move(heartbeat_queue.back())); + heartbeat_queue.pop_back(); } return {}; } @@ -65,6 +65,13 @@ class FakeClient : public Client { tl::expected PromotionAllocStart( const std::string& key, uint64_t size, const std::vector& preferred_segments) override { + return PromotionAllocStart(key, "default", size, preferred_segments); + } + + tl::expected PromotionAllocStart( + const std::string& key, const std::string& tenant_id, uint64_t size, + const std::vector& preferred_segments) override { + (void)tenant_id; (void)size; (void)preferred_segments; alloc_calls.fetch_add(1); @@ -99,6 +106,12 @@ class FakeClient : public Client { tl::expected NotifyPromotionSuccess( const std::string& key) override { + return NotifyPromotionSuccess(key, "default"); + } + + tl::expected NotifyPromotionSuccess( + const std::string& key, const std::string& tenant_id) override { + (void)tenant_id; notify_calls.fetch_add(1); notify_keys.push_back(key); auto it = notify_overrides.find(key); @@ -116,6 +129,12 @@ class FakeClient : public Client { // notify the master. tl::expected NotifyPromotionFailure( const std::string& key) override { + return NotifyPromotionFailure(key, "default"); + } + + tl::expected NotifyPromotionFailure( + const std::string& key, const std::string& tenant_id) override { + (void)tenant_id; notify_failure_calls.fetch_add(1); notify_failure_keys.push_back(key); return {}; @@ -185,7 +204,12 @@ class FileStoragePromotionTest : public ::testing::Test { tl::expected last_res{}; while (!remaining.empty()) { std::string before = fake->last_alloc_key; - fake->heartbeat_queue = remaining; + fake->heartbeat_queue.clear(); + fake->heartbeat_queue.reserve(remaining.size()); + for (const auto& [key, size] : remaining) { + fake->heartbeat_queue.push_back(PromotionTaskItem{ + .tenant_id = "default", .key = key, .size = size}); + } last_res = CallProcessPromotionTasks(); if (!last_res.has_value()) return last_res; if (fake->last_alloc_key == before || @@ -231,7 +255,9 @@ TEST_F(FileStoragePromotionTest, HeartbeatHardErrorPropagates) { // Non-positive size in queue: skip that key, continue. TEST_F(FileStoragePromotionTest, NonPositiveSizeSkipped) { - fake->heartbeat_queue = {{"k_bad", 0}, {"k_good", 1024}}; + fake->heartbeat_queue = { + {.tenant_id = "default", .key = "k_bad", .size = 0}, + {.tenant_id = "default", .key = "k_good", .size = 1024}}; auto res = CallProcessPromotionTasks(); EXPECT_TRUE(res.has_value()); // Only k_good should reach AllocStart. @@ -255,7 +281,8 @@ TEST_F(FileStoragePromotionTest, AllocStartFailureSkipsKey) { // BatchLoad failure (SSD file missing): no PromotionWrite, no Notify. // Master-side reaper handles the orphaned PROCESSING replica. TEST_F(FileStoragePromotionTest, BatchLoadFailureLeavesNoNotify) { - fake->heartbeat_queue = {{"k_missing", 1024}}; + fake->heartbeat_queue = { + {.tenant_id = "default", .key = "k_missing", .size = 1024}}; // Default alloc succeeds; BatchLoad will fail because there's no file // at data_path/k_missing for the storage backend to read. auto res = CallProcessPromotionTasks(); @@ -270,7 +297,8 @@ TEST_F(FileStoragePromotionTest, BatchLoadFailureLeavesNoNotify) { // PromotionWrite failure: no Notify. TEST_F(FileStoragePromotionTest, TransferWriteFailureLeavesNoNotify) { - fake->heartbeat_queue = {{"k_te_fail", 1024}}; + fake->heartbeat_queue = { + {.tenant_id = "default", .key = "k_te_fail", .size = 1024}}; fake->default_write_result = ErrorCode::TRANSFER_FAIL; auto res = CallProcessPromotionTasks(); EXPECT_TRUE(res.has_value()); @@ -313,7 +341,8 @@ TEST_F(FileStoragePromotionTest, PerKeyFailuresAreIndependent) { // put_start_release_timeout_sec_ (~10 min default), turning a transient // DRAM-pressure spike into a sustained outage of promotion_queue_limit_. TEST_F(FileStoragePromotionTest, AllocStartFailureNotifiesMaster) { - fake->heartbeat_queue = {{"k_alloc_fail", 1024}}; + fake->heartbeat_queue = { + {.tenant_id = "default", .key = "k_alloc_fail", .size = 1024}}; fake->alloc_overrides["k_alloc_fail"] = ErrorCode::NO_AVAILABLE_HANDLE; auto res = CallProcessPromotionTasks(); EXPECT_TRUE(res.has_value()); diff --git a/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot.cpp b/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot.cpp index 6ec222f4..d9d371b5 100644 --- a/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot.cpp +++ b/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot.cpp @@ -1,5 +1,6 @@ #include "master_service_test_for_snapshot_base.h" +#include #include #include #include @@ -2480,8 +2481,11 @@ TEST_F(MasterServiceSnapshotTest, OffloadObjectHeartbeat) { } ASSERT_EQ(res->size(), keys.size()); for (auto& key : keys) { - ASSERT_TRUE(res.value().find(key) != res.value().end()); - ASSERT_EQ(res.value().find(key)->second, 1024); + auto it = std::find_if( + res->begin(), res->end(), + [&key](const OffloadTaskItem& task) { return task.key == key; }); + ASSERT_TRUE(it != res->end()); + ASSERT_EQ(it->size, 1024); } keys.clear(); @@ -2497,8 +2501,11 @@ TEST_F(MasterServiceSnapshotTest, OffloadObjectHeartbeat) { } ASSERT_EQ(res->size(), keys.size()); for (auto& key : keys) { - ASSERT_TRUE(res.value().find(key) != res.value().end()); - ASSERT_EQ(res.value().find(key)->second, 1024); + auto it = std::find_if( + res->begin(), res->end(), + [&key](const OffloadTaskItem& task) { return task.key == key; }); + ASSERT_TRUE(it != res->end()); + ASSERT_EQ(it->size, 1024); } } diff --git a/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot_base.h b/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot_base.h index d2edb675..c59a760b 100644 --- a/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot_base.h +++ b/mooncake-store/tests/ha/snapshot/master_service_test_for_snapshot_base.h @@ -72,8 +72,8 @@ class MasterServiceSnapshotTestBase : public ::testing::Test { // LocalDiskSegment state for comparison struct LocalDiskSegmentState { bool enable_offloading = false; - std::map - offloading_objects; // key -> timestamp (sorted) + std::map + offloading_objects; // storage key -> task (sorted) }; // Task state for comparison @@ -247,9 +247,8 @@ class MasterServiceSnapshotTestBase : public ::testing::Test { seg_state.enable_offloading = segment->enable_offloading; // Copy offloading_objects to sorted map std::lock_guard lock(segment->offloading_mutex_); - for (const auto& [key, timestamp] : - segment->offloading_objects) { - seg_state.offloading_objects[key] = timestamp; + for (const auto& [key, task] : segment->offloading_objects) { + seg_state.offloading_objects[key] = task; } state.local_disk_segments[client_id] = std::move(seg_state); } diff --git a/mooncake-store/tests/master_service_test.cpp b/mooncake-store/tests/master_service_test.cpp index cff39b0a..46526fc7 100644 --- a/mooncake-store/tests/master_service_test.cpp +++ b/mooncake-store/tests/master_service_test.cpp @@ -137,10 +137,13 @@ class MasterServiceTest : public ::testing::Test { ReplicaMovePayload payload; struct_json::from_json(payload, assignment.payload); - auto move_start = service.MoveStart(client_id, payload.key, - payload.source, payload.target); + auto move_start = + service.MoveStart(client_id, payload.key, payload.tenant_id, + payload.source, payload.target); EXPECT_TRUE(move_start.has_value()); - EXPECT_TRUE(service.MoveEnd(client_id, payload.key).has_value()); + EXPECT_TRUE( + service.MoveEnd(client_id, payload.key, payload.tenant_id) + .has_value()); TaskCompleteRequest complete_request; complete_request.id = assignment.id; @@ -199,6 +202,27 @@ class MasterServiceTest : public ::testing::Test { void TearDown() override { google::ShutdownGoogleLogging(); } }; +TEST(TenantScopedStorageKeyTest, RoundTripsAndParsesLegacyKeys) { + const auto scoped = + MakeTenantScopedStorageKey("tenant:with:colon", "path/key:with:colon"); + EXPECT_NE(scoped.find('\0'), std::string::npos); + + auto [tenant_id, key] = ParseTenantScopedStorageKey(scoped); + EXPECT_EQ(tenant_id, "tenant:with:colon"); + EXPECT_EQ(key, "path/key:with:colon"); + + auto [default_tenant, default_key] = ParseTenantScopedStorageKey("raw_key"); + EXPECT_EQ(default_tenant, "default"); + EXPECT_EQ(default_key, "raw_key"); + + std::string legacy = "legacy_tenant"; + legacy.push_back('\0'); + legacy.append("legacy_key"); + auto [legacy_tenant, legacy_key] = ParseTenantScopedStorageKey(legacy); + EXPECT_EQ(legacy_tenant, "legacy_tenant"); + EXPECT_EQ(legacy_key, "legacy_key"); +} + std::string GenerateKeyForSegment(const UUID& client_id, const std::unique_ptr& service, const std::string& segment_name) { @@ -4598,8 +4622,11 @@ TEST_F(MasterServiceTest, OffloadObjectHeartbeat) { } ASSERT_EQ(res->size(), keys.size()); for (auto& key : keys) { - ASSERT_TRUE(res.value().find(key) != res.value().end()); - ASSERT_EQ(res.value().find(key)->second, 1024); + auto it = std::find_if( + res->begin(), res->end(), + [&key](const OffloadTaskItem& task) { return task.key == key; }); + ASSERT_TRUE(it != res->end()); + ASSERT_EQ(it->size, 1024); } keys.clear(); @@ -4615,8 +4642,11 @@ TEST_F(MasterServiceTest, OffloadObjectHeartbeat) { } ASSERT_EQ(res->size(), keys.size()); for (auto& key : keys) { - ASSERT_TRUE(res.value().find(key) != res.value().end()); - ASSERT_EQ(res.value().find(key)->second, 1024); + auto it = std::find_if( + res->begin(), res->end(), + [&key](const OffloadTaskItem& task) { return task.key == key; }); + ASSERT_TRUE(it != res->end()); + ASSERT_EQ(it->size, 1024); } } @@ -5228,6 +5258,81 @@ TEST_F(MasterServiceTest, FetchTasksReturnsAssignedTasksOnlyAndDrainsQueue) { EXPECT_TRUE(fetch0_again->empty()); } +TEST_F(MasterServiceTest, TenantTasksCarryTenantInPayload) { + auto service = std::make_unique(); + const auto ctx0 = PrepareSimpleSegment(*service, "segment_0", 0x300000000, + kDefaultSegmentSize); + [[maybe_unused]] const auto ctx1 = PrepareSimpleSegment( + *service, "segment_1", 0x400000000, kDefaultSegmentSize); + + const UUID put_client_id = generate_uuid(); + const std::string key = "tenant_task_key"; + const std::string tenant_id = "tenant_for_async_task"; + + ReplicateConfig config; + config.replica_num = 1; + config.preferred_segment = "segment_0"; + + ASSERT_TRUE(service + ->PutStart(put_client_id, key, tenant_id, + /*slice_length=*/1024, config) + .has_value()); + ASSERT_TRUE( + service->PutEnd(put_client_id, key, tenant_id, ReplicaType::MEMORY) + .has_value()); + + auto copy_task_id = service->CreateCopyTask(key, tenant_id, {"segment_1"}); + ASSERT_TRUE(copy_task_id.has_value()); + auto move_task_id = + service->CreateMoveTask(key, tenant_id, "segment_0", "segment_1"); + ASSERT_TRUE(move_task_id.has_value()); + + auto fetched = service->FetchTasks(ctx0.client_id, /*batch_size=*/16); + ASSERT_TRUE(fetched.has_value()); + ASSERT_EQ(fetched->size(), 2u); + + bool saw_copy = false; + bool saw_move = false; + for (const auto& assignment : *fetched) { + if (assignment.id == copy_task_id.value()) { + ReplicaCopyPayload payload; + struct_json::from_json(payload, assignment.payload); + EXPECT_EQ(payload.tenant_id, tenant_id); + EXPECT_EQ(payload.key, key); + saw_copy = true; + } else if (assignment.id == move_task_id.value()) { + ReplicaMovePayload payload; + struct_json::from_json(payload, assignment.payload); + EXPECT_EQ(payload.tenant_id, tenant_id); + EXPECT_EQ(payload.key, key); + saw_move = true; + } + } + EXPECT_TRUE(saw_copy); + EXPECT_TRUE(saw_move); +} + +TEST_F(MasterServiceTest, LegacyTaskPayloadDefaultsTenant) { + ReplicaCopyPayload copy_payload; + struct_json::from_json( + copy_payload, + R"({"key":"legacy_copy_key","source":"segment_0","targets":["segment_1"]})"); + EXPECT_EQ(copy_payload.tenant_id, "default"); + EXPECT_EQ(copy_payload.key, "legacy_copy_key"); + EXPECT_EQ(copy_payload.source, "segment_0"); + ASSERT_EQ(copy_payload.targets.size(), 1u); + EXPECT_EQ(copy_payload.targets[0], "segment_1"); + + ReplicaMovePayload move_payload; + struct_json::from_json( + move_payload, + R"({"key":"legacy_move_key","source":"segment_0","target":"segment_1"})"); + EXPECT_EQ(move_payload.tenant_id, "default"); + EXPECT_EQ(move_payload.key, "legacy_move_key"); + EXPECT_EQ(move_payload.source, "segment_0"); + EXPECT_EQ(move_payload.target, "segment_1"); +} + TEST_F(MasterServiceTest, FetchTasksRespectsBatchSize) { std::unique_ptr service_(new MasterService()); diff --git a/mooncake-store/tests/offload_on_evict_test.cpp b/mooncake-store/tests/offload_on_evict_test.cpp index c002d285..18fab134 100644 --- a/mooncake-store/tests/offload_on_evict_test.cpp +++ b/mooncake-store/tests/offload_on_evict_test.cpp @@ -68,7 +68,11 @@ class OffloadOnEvictTest : public ::testing::Test { if (!res) { return {}; } - return std::move(res.value()); + std::unordered_map queued; + for (const auto& task : res.value()) { + queued[task.key] = task.size; + } + return queued; } template diff --git a/mooncake-store/tests/promotion_on_hit_test.cpp b/mooncake-store/tests/promotion_on_hit_test.cpp index 89e2fa83..a9970c37 100644 --- a/mooncake-store/tests/promotion_on_hit_test.cpp +++ b/mooncake-store/tests/promotion_on_hit_test.cpp @@ -7,6 +7,7 @@ #include #include +#include #include #include #include @@ -17,6 +18,13 @@ namespace mooncake::test { +size_t CountPromotionTask(const std::vector& tasks, + const std::string& key) { + return std::count_if( + tasks.begin(), tasks.end(), + [&key](const PromotionTaskItem& task) { return task.key == key; }); +} + class PromotionOnHitTest : public ::testing::Test { protected: void SetUp() override { @@ -193,7 +201,7 @@ TEST_F(PromotionOnHitTest, MemoryReplicaPresentNoPromotion) { service->RemoveAll(); } -// PromotionObjectHeartbeat returns an empty map when called against a +// PromotionObjectHeartbeat returns an empty task list when called against a // client that has no LocalDiskSegment registered. TEST_F(PromotionOnHitTest, HeartbeatReturnsErrorForUnknownClient) { MasterServiceConfig config; @@ -275,7 +283,7 @@ TEST_F(PromotionOnHitTest, RacingReadersDedup) { EXPECT_EQ(pending->size(), 1u) << "Concurrent readers (" << kThreads << " x " << kReadsPerThread << ") must dedupe to a single promotion task"; - ASSERT_TRUE(pending->count("k_cold")); + ASSERT_TRUE(CountPromotionTask(*pending, "k_cold")); service->RemoveAll(); } @@ -583,9 +591,9 @@ TEST_F(PromotionOnHitTest, QueueLimitRejectsBeyondCap) { EXPECT_EQ(heartbeat->size(), 1u) << "promotion_queue_limit=1 should admit only the first task " << "globally; k2's enqueue must be dropped"; - EXPECT_EQ(heartbeat->count(k1), 1u) + EXPECT_EQ(CountPromotionTask(*heartbeat, k1), 1u) << "k1 was read first and should be the surviving task"; - EXPECT_EQ(heartbeat->count(k2), 0u) + EXPECT_EQ(CountPromotionTask(*heartbeat, k2), 0u) << "k2 was rejected by the cap gate; should not appear"; EXPECT_EQ(mm.get_promotion_rejected_cap() - cap_rej_pre, 1) << "k2's rejection must increment promotion_rejected_cap"; @@ -627,14 +635,14 @@ TEST_F(PromotionOnHitTest, HeartbeatBoundedBatchPreservesLeftovers) { ASSERT_TRUE(tick1.has_value()); EXPECT_EQ(tick1->size(), 1u) << "heartbeat should return at most kMaxPerHeartbeat=1 entry"; - std::string first_key = tick1->begin()->first; + std::string first_key = tick1->begin()->key; EXPECT_TRUE(std::find(keys.begin(), keys.end(), first_key) != keys.end()); // Second heartbeat returns another key (a different one). auto tick2 = service->PromotionObjectHeartbeat(seg.client_id); ASSERT_TRUE(tick2.has_value()); EXPECT_EQ(tick2->size(), 1u); - std::string second_key = tick2->begin()->first; + std::string second_key = tick2->begin()->key; EXPECT_NE(second_key, first_key) << "second heartbeat must drain a different leftover key, not " << "re-return the one already extracted"; @@ -643,7 +651,7 @@ TEST_F(PromotionOnHitTest, HeartbeatBoundedBatchPreservesLeftovers) { auto tick3 = service->PromotionObjectHeartbeat(seg.client_id); ASSERT_TRUE(tick3.has_value()); EXPECT_EQ(tick3->size(), 1u); - std::string third_key = tick3->begin()->first; + std::string third_key = tick3->begin()->key; EXPECT_NE(third_key, first_key); EXPECT_NE(third_key, second_key); @@ -808,8 +816,8 @@ TEST_F(PromotionOnHitTest, QueueLimitRejectsCrossShard) { << " must be rejected by the global gate. A per-shard heuristic " << "would admit it here since the destination shard's local " << "count is 0."; - EXPECT_EQ(heartbeat->count(k1), 1u); - EXPECT_EQ(heartbeat->count(k2), 0u); + EXPECT_EQ(CountPromotionTask(*heartbeat, k1), 1u); + EXPECT_EQ(CountPromotionTask(*heartbeat, k2), 0u); service->RemoveAll(); } @@ -984,7 +992,7 @@ TEST_F(PromotionOnHitTest, NotifySuccessDecrementsCounter) { << "in-flight counter must decrement on the NotifyPromotionSuccess " << "success path. Without fetch_sub, the cap stays saturated and " << "k_second is silently dropped."; - EXPECT_EQ(pending->count("k_second"), 1u); + EXPECT_EQ(CountPromotionTask(*pending, "k_second"), 1u); service->RemoveAll(); } @@ -1201,7 +1209,7 @@ TEST_F(PromotionOnHitTest, NotifyFailureReleasesStateImmediately) { } auto heartbeat = service->PromotionObjectHeartbeat(seg.client_id); ASSERT_TRUE(heartbeat.has_value()); - EXPECT_EQ(heartbeat->count("k_b"), 1u) + EXPECT_EQ(CountPromotionTask(*heartbeat, "k_b"), 1u) << "k_b admission must succeed after k_a's failure released the " << "slot. If this fires, NotifyPromotionFailure did not decrement " << "promotion_in_flight_, and transient client-side errors " @@ -1452,7 +1460,7 @@ TEST_F(PromotionOnHitTest, ClientExpiryClearsPromotionTask) { auto pending_pre = service->PromotionObjectHeartbeat(second_holder.client_id); ASSERT_TRUE(pending_pre.has_value()); - EXPECT_EQ(pending_pre->count("k_other"), 0u) + EXPECT_EQ(CountPromotionTask(*pending_pre, "k_other"), 0u) << "Sanity: queue_limit=1 should block the second admission " << "while the first task is in flight."; @@ -1486,7 +1494,7 @@ TEST_F(PromotionOnHitTest, ClientExpiryClearsPromotionTask) { auto pending_post = service->PromotionObjectHeartbeat(second_holder.client_id); ASSERT_TRUE(pending_post.has_value()); - EXPECT_EQ(pending_post->count("k_other"), 1u) + EXPECT_EQ(CountPromotionTask(*pending_post, "k_other"), 1u) << "After the holder expired, ClearInvalidHandles must have " << "erased its promotion_tasks entry and decremented " << "promotion_in_flight_. Otherwise the global cap remains " @@ -1567,7 +1575,7 @@ TEST_F(PromotionOnHitTest, RemoveErasesPromotionTask) { { auto pending = service->PromotionObjectHeartbeat(holder.client_id); ASSERT_TRUE(pending.has_value()); - EXPECT_EQ(pending->count("k_first"), 1u); + EXPECT_EQ(CountPromotionTask(*pending, "k_first"), 1u); } // Remove k_first with force=true. With the fix, this also wipes @@ -1586,7 +1594,7 @@ TEST_F(PromotionOnHitTest, RemoveErasesPromotionTask) { } auto pending_post = service->PromotionObjectHeartbeat(holder.client_id); ASSERT_TRUE(pending_post.has_value()); - EXPECT_EQ(pending_post->count("k_second"), 1u) + EXPECT_EQ(CountPromotionTask(*pending_post, "k_second"), 1u) << "k_second must be admittable after Remove of k_first — Remove " << "must erase the in-flight promotion_tasks entry and decrement " << "promotion_in_flight_, otherwise queue_limit=1 stays saturated."; @@ -1636,7 +1644,7 @@ TEST_F(PromotionOnHitTest, RemoveByRegexErasesPromotionTask) { } auto pending_post = service->PromotionObjectHeartbeat(holder.client_id); ASSERT_TRUE(pending_post.has_value()); - EXPECT_EQ(pending_post->count("other_k2"), 1u) + EXPECT_EQ(CountPromotionTask(*pending_post, "other_k2"), 1u) << "other_k2 must be admittable after RemoveByRegex of regex_k1 " << "— RemoveByRegex must erase the in-flight promotion_tasks " << "entry. Otherwise queue_limit=1 stays saturated."; @@ -1675,7 +1683,7 @@ TEST_F(PromotionOnHitTest, RemoveAllErasesPromotionTask) { { auto pending = service->PromotionObjectHeartbeat(holder.client_id); ASSERT_TRUE(pending.has_value()); - EXPECT_EQ(pending->count("k_first"), 1u); + EXPECT_EQ(CountPromotionTask(*pending, "k_first"), 1u); } auto removed = service->RemoveAll(/*force=*/true); @@ -1693,7 +1701,7 @@ TEST_F(PromotionOnHitTest, RemoveAllErasesPromotionTask) { } auto pending_post = service->PromotionObjectHeartbeat(holder.client_id); ASSERT_TRUE(pending_post.has_value()); - EXPECT_EQ(pending_post->count("k_second"), 1u) + EXPECT_EQ(CountPromotionTask(*pending_post, "k_second"), 1u) << "k_second must be admittable after RemoveAll of k_first — " << "otherwise queue_limit=1 stays saturated until reaper TTL."; @@ -1739,7 +1747,7 @@ TEST_F(PromotionOnHitTest, BatchRemoveErasesPromotionTask) { { auto pending = service->PromotionObjectHeartbeat(holder.client_id); ASSERT_TRUE(pending.has_value()); - EXPECT_EQ(pending->count("k_first"), 1u); + EXPECT_EQ(CountPromotionTask(*pending, "k_first"), 1u); } auto results = service->BatchRemove({"k_first"}, /*force=*/true); @@ -1757,7 +1765,7 @@ TEST_F(PromotionOnHitTest, BatchRemoveErasesPromotionTask) { } auto pending_post = service->PromotionObjectHeartbeat(holder.client_id); ASSERT_TRUE(pending_post.has_value()); - EXPECT_EQ(pending_post->count("k_second"), 1u) + EXPECT_EQ(CountPromotionTask(*pending_post, "k_second"), 1u) << "k_second must be admittable after BatchRemove of k_first — " << "otherwise queue_limit=1 stays saturated until reaper TTL."; @@ -1795,7 +1803,7 @@ TEST_F(PromotionOnHitTest, BatchRemoveStaleHandleErasesPromotionTask) { { auto pending = service->PromotionObjectHeartbeat(holder.client_id); ASSERT_TRUE(pending.has_value()); - EXPECT_EQ(pending->count("k_first"), 1u); + EXPECT_EQ(CountPromotionTask(*pending, "k_first"), 1u); } auto results = service->BatchRemove({"k_first"}, /*force=*/true); @@ -1828,7 +1836,7 @@ TEST_F(PromotionOnHitTest, BatchRemoveStaleHandleErasesPromotionTask) { auto pending_post = service->PromotionObjectHeartbeat(second_holder.client_id); ASSERT_TRUE(pending_post.has_value()); - EXPECT_EQ(pending_post->count("k_second"), 1u) + EXPECT_EQ(CountPromotionTask(*pending_post, "k_second"), 1u) << "k_second must be admittable after the stale-handle " << "BatchRemove — otherwise queue_limit=1 stays saturated until " << "reaper TTL."; -- 2.34.1 From 1ca541045407338450a07cc671003be699037658 Mon Sep 17 00:00:00 2001 From: Aoi Date: Wed, 3 Jun 2026 00:02:05 +0800 Subject: [PATCH 168/382] [Build] Disable debug symbols (-g) in default compilation flags (#2285) Keep debug symbols enabled by default for local developer builds. CI test workflows pass -DENABLE_DEBUG_SYMBOLS=OFF to reduce binary sizes during testing. Release workflows are unchanged and retain debug symbols. --- .github/workflows/ci.yml | 14 +++++++------- .github/workflows/ci_ascend.yml | 3 ++- .github/workflows/ci_cu13.yml | 3 ++- mooncake-common/common.cmake | 11 +++++++++-- 4 files changed, 20 insertions(+), 11 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 003c1c00..f549bf38 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -108,7 +108,7 @@ jobs: sudo bash -x dependencies.sh -y mkdir build cd build - cmake -G Ninja .. -DUSE_HTTP=ON -DUSE_CXL=ON -DUSE_UB=ON -DUSE_ETCD=ON -DSTORE_USE_ETCD=ON -DENABLE_ASAN=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Debug + cmake -G Ninja .. -DUSE_HTTP=ON -DUSE_CXL=ON -DUSE_UB=ON -DUSE_ETCD=ON -DSTORE_USE_ETCD=ON -DENABLE_ASAN=ON -DENABLE_SCCACHE=ON -DCMAKE_BUILD_TYPE=Debug -DENABLE_DEBUG_SYMBOLS=OFF shell: bash - name: Build project @@ -297,7 +297,7 @@ jobs: bash -x dependencies.sh -y mkdir build cd build - cmake -G Ninja .. -DUSE_MUSA=ON -DUSE_MNNVL=ON -DUSE_ETCD=ON -DSTORE_USE_ETCD=ON -DUSE_CXL=ON -DUSE_TCP=ON -DBUILD_UNIT_TESTS=OFF -DBUILD_EXAMPLES=OFF + cmake -G Ninja .. -DUSE_MUSA=ON -DUSE_MNNVL=ON -DUSE_ETCD=ON -DSTORE_USE_ETCD=ON -DUSE_CXL=ON -DUSE_TCP=ON -DBUILD_UNIT_TESTS=OFF -DBUILD_EXAMPLES=OFF -DENABLE_DEBUG_SYMBOLS=OFF shell: bash - name: Build project @@ -553,7 +553,7 @@ jobs: cd build export PATH=/usr/local/nvidia/bin:/usr/local/nvidia/lib64:$PATH export LD_LIBRARY_PATH=/usr/local/cuda/lib64/stubs:$LD_LIBRARY_PATH - cmake -G Ninja .. -DUSE_ETCD=OFF -DUSE_CXL=ON -DUSE_REDIS=ON -DUSE_HTTP=ON -DWITH_METRICS=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DENABLE_SCCACHE=ON -DUSE_CUDA=OFF -DUSE_MNNVL=OFF -DUSE_UB=OFF -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/cuda/lib64/stubs" + cmake -G Ninja .. -DUSE_ETCD=OFF -DUSE_CXL=ON -DUSE_REDIS=ON -DUSE_HTTP=ON -DWITH_METRICS=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DENABLE_SCCACHE=ON -DUSE_CUDA=OFF -DUSE_MNNVL=OFF -DUSE_UB=OFF -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/cuda/lib64/stubs" -DENABLE_DEBUG_SYMBOLS=OFF cmake --build . sudo cmake --install . df -h @@ -563,7 +563,7 @@ jobs: run: | mkdir build cd build - cmake -G Ninja .. -DUSE_ETCD=ON -DUSE_CXL=ON -DUSE_REDIS=ON -DUSE_HTTP=ON -DWITH_STORE=ON -DWITH_P2P_STORE=ON -DWITH_METRICS=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DENABLE_SCCACHE=ON -DUSE_CUDA=ON -DUSE_MNNVL=OFF -DUSE_UB=OFF -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/cuda/lib64/stubs" + cmake -G Ninja .. -DUSE_ETCD=ON -DUSE_CXL=ON -DUSE_REDIS=ON -DUSE_HTTP=ON -DWITH_STORE=ON -DWITH_P2P_STORE=ON -DWITH_METRICS=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DENABLE_SCCACHE=ON -DUSE_CUDA=ON -DUSE_MNNVL=OFF -DUSE_UB=OFF -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/cuda/lib64/stubs" -DENABLE_DEBUG_SYMBOLS=OFF shell: bash # TODO: lack USE_NVMEOF,USE_MNNVL @@ -580,7 +580,7 @@ jobs: - name: Configure project with unit tests and examples run: | cd build - cmake -G Ninja .. -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DWITH_STORE_RUST=ON -DENABLE_SCCACHE=ON + cmake -G Ninja .. -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DWITH_STORE_RUST=ON -DENABLE_SCCACHE=ON -DENABLE_DEBUG_SYMBOLS=OFF shell: bash - name: Build project with unit tests and examples @@ -613,7 +613,7 @@ jobs: run: | cd build rm -r */tests - cmake -G Ninja .. -DBUILD_UNIT_TESTS=OFF -DBUILD_EXAMPLES=OFF -DUSE_HTTP=ON -DENABLE_SCCACHE=ON -DUSE_CXL=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.1;2.10.0;2.11.0;2.12.0" + cmake -G Ninja .. -DBUILD_UNIT_TESTS=OFF -DBUILD_EXAMPLES=OFF -DUSE_HTTP=ON -DENABLE_SCCACHE=ON -DUSE_CXL=ON -DWITH_EP=ON -DEP_TORCH_VERSIONS="2.9.1;2.10.0;2.11.0;2.12.0" -DENABLE_DEBUG_SYMBOLS=OFF shell: bash - name: Build project @@ -629,7 +629,7 @@ jobs: run: | mkdir build-tent cd build-tent - cmake -G Ninja .. -DUSE_TENT=ON -DUSE_HTTP=ON -DENABLE_SCCACHE=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON + cmake -G Ninja .. -DUSE_TENT=ON -DUSE_HTTP=ON -DENABLE_SCCACHE=ON -DBUILD_UNIT_TESTS=ON -DBUILD_EXAMPLES=ON -DENABLE_DEBUG_SYMBOLS=OFF shell: bash - name: Build project with TENT diff --git a/.github/workflows/ci_ascend.yml b/.github/workflows/ci_ascend.yml index 28df9b2f..30264db8 100644 --- a/.github/workflows/ci_ascend.yml +++ b/.github/workflows/ci_ascend.yml @@ -163,7 +163,8 @@ jobs: cmake .. \ -DUSE_ASCEND_DIRECT=ON \ -DBUILD_EXAMPLES=OFF \ - -DBUILD_UNIT_TESTS=OFF + -DBUILD_UNIT_TESTS=OFF \ + -DENABLE_DEBUG_SYMBOLS=OFF - name: Build shell: bash diff --git a/.github/workflows/ci_cu13.yml b/.github/workflows/ci_cu13.yml index ba28fa49..a4564bc4 100644 --- a/.github/workflows/ci_cu13.yml +++ b/.github/workflows/ci_cu13.yml @@ -83,7 +83,8 @@ jobs: -DBUILD_BENCHMARK=ON \ -DUSE_CUDA=ON \ -DCMAKE_BUILD_TYPE=Release \ - -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/cuda/lib64/stubs" + -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/cuda/lib64/stubs" \ + -DENABLE_DEBUG_SYMBOLS=OFF shell: bash - name: Build project diff --git a/mooncake-common/common.cmake b/mooncake-common/common.cmake index d20b2ccd..9c2a1352 100644 --- a/mooncake-common/common.cmake +++ b/mooncake-common/common.cmake @@ -1,8 +1,15 @@ set(CMAKE_C_STANDARD 99) set(CMAKE_CXX_STANDARD 20) -set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -Wall -Wextra -Wno-unused-parameter -fPIC") -set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -g -Wall -Wextra -Wno-unused-parameter -fPIC") +option(ENABLE_DEBUG_SYMBOLS "Include debug symbols (-g) in compilation" ON) + +set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -Wextra -Wno-unused-parameter -fPIC") +set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wall -Wextra -Wno-unused-parameter -fPIC") + +if(ENABLE_DEBUG_SYMBOLS) + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g") + set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -g") +endif() if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fcoroutines") -- 2.34.1 From 4c6a5367dd4e669b7caefb1e1f33c45596dac42e Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Wed, 3 Jun 2026 09:41:23 +0800 Subject: [PATCH 169/382] [TE] Fix TCP connection pool SIGSEGV by deferring cleanup with asio::post (#2174) * Use asio::post to defer cleanup * Update lambda binding * reformat --- .../transport/tcp_transport/tcp_transport.cpp | 98 ++++++++++++++----- 1 file changed, 73 insertions(+), 25 deletions(-) diff --git a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp index 83e213a3..e7472608 100644 --- a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp +++ b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp @@ -278,9 +278,15 @@ struct ClientSession : public std::enable_shared_from_this { << "ClientSession::writeHeader failed. Error: " << ec.message() << " (value: " << ec.value() << ")" << ", bytes written: " << len; - if (on_finalize_) on_finalize_(TransferStatusEnum::FAILED); - session_mutex_.unlock(); - if (on_complete_) on_complete_(); + asio::post( + socket_->get_executor(), + [this, self, on_finalize = std::move(on_finalize_), + on_complete = std::move(on_complete_)]() { + if (on_finalize) + on_finalize(TransferStatusEnum::FAILED); + session_mutex_.unlock(); + if (on_complete) on_complete(); + }); return; } if (header_.opcode == (uint8_t)TransferRequest::WRITE) @@ -298,9 +304,14 @@ struct ClientSession : public std::enable_shared_from_this { size_t buffer_size = std::min(kDefaultBufferSize, size - total_transferred_bytes_); if (buffer_size == 0) { - if (on_finalize_) on_finalize_(TransferStatusEnum::COMPLETED); - session_mutex_.unlock(); - if (on_complete_) on_complete_(); + asio::post(socket_->get_executor(), + [this, self, on_finalize = std::move(on_finalize_), + on_complete = std::move(on_complete_)]() { + if (on_finalize) + on_finalize(TransferStatusEnum::COMPLETED); + session_mutex_.unlock(); + if (on_complete) on_complete(); + }); return; } @@ -328,14 +339,22 @@ struct ClientSession : public std::enable_shared_from_this { << " using buffer " << static_cast(dram_buffer) << ". Error: " << ec.message() << " (value: " << ec.value() << ")"; - if (on_finalize_) on_finalize_(TransferStatusEnum::FAILED); - if (on_complete_) on_complete_(); + // Post entire cleanup to ensure it runs after callback + // returns + asio::post(socket_->get_executor(), + [this, self, dram_buffer, is_cuda_memory, + on_finalize = std::move(on_finalize_), + on_complete = std::move(on_complete_)]() { + if (on_finalize) + on_finalize(TransferStatusEnum::FAILED); #if defined(USE_CUDA) || defined(USE_MUSA) || defined(USE_HIP) || \ defined(USE_MLU) || defined(USE_MACA) || defined(USE_HYGON) || \ defined(USE_COREX) - if (is_cuda_memory) delete[] dram_buffer; + if (is_cuda_memory) delete[] dram_buffer; #endif - session_mutex_.unlock(); + session_mutex_.unlock(); + if (on_complete) on_complete(); + }); return; } @@ -351,11 +370,19 @@ struct ClientSession : public std::enable_shared_from_this { << "ClientSession::readBody failed to copy to CUDA " "memory. " << "Error: " << cudaGetErrorString(cuda_status); - if (on_finalize_) - on_finalize_(TransferStatusEnum::FAILED); - if (on_complete_) on_complete_(); - delete[] dram_buffer; - session_mutex_.unlock(); + // Post entire cleanup to ensure it runs after callback + // returns + asio::post( + socket_->get_executor(), + [this, self, dram_buffer, + on_finalize = std::move(on_finalize_), + on_complete = std::move(on_complete_)]() { + if (on_finalize) + on_finalize(TransferStatusEnum::FAILED); + delete[] dram_buffer; + session_mutex_.unlock(); + if (on_complete) on_complete(); + }); return; } delete[] dram_buffer; @@ -374,9 +401,15 @@ struct ClientSession : public std::enable_shared_from_this { size_t buffer_size = std::min(kDefaultBufferSize, size - total_transferred_bytes_); if (buffer_size == 0) { - if (on_finalize_) on_finalize_(TransferStatusEnum::COMPLETED); - session_mutex_.unlock(); - if (on_complete_) on_complete_(); + // Post cleanup to ensure it runs after callback returns + asio::post(socket_->get_executor(), + [this, self, on_finalize = std::move(on_finalize_), + on_complete = std::move(on_complete_)]() { + if (on_finalize) + on_finalize(TransferStatusEnum::COMPLETED); + session_mutex_.unlock(); + if (on_complete) on_complete(); + }); return; } @@ -394,10 +427,17 @@ struct ClientSession : public std::enable_shared_from_this { LOG(ERROR) << "ClientSession::writeBody failed to copy from " "CUDA memory. " << "Error: " << cudaGetErrorString(cuda_status); - if (on_finalize_) on_finalize_(TransferStatusEnum::FAILED); - if (on_complete_) on_complete_(); - session_mutex_.unlock(); - delete[] dram_buffer; + // Post entire cleanup to ensure it runs after callback returns + asio::post(socket_->get_executor(), + [this, self, dram_buffer, + on_finalize = std::move(on_finalize_), + on_complete = std::move(on_complete_)]() { + if (on_finalize) + on_finalize(TransferStatusEnum::FAILED); + delete[] dram_buffer; + session_mutex_.unlock(); + if (on_complete) on_complete(); + }); return; } } @@ -421,9 +461,17 @@ struct ClientSession : public std::enable_shared_from_this { << " using buffer " << static_cast(dram_buffer) << ". Error: " << ec.message() << " (value: " << ec.value() << ")"; - if (on_finalize_) on_finalize_(TransferStatusEnum::FAILED); - if (on_complete_) on_complete_(); - session_mutex_.unlock(); + // Post entire cleanup to ensure it runs after callback + // returns + asio::post( + socket_->get_executor(), + [this, self, on_finalize = std::move(on_finalize_), + on_complete = std::move(on_complete_)]() { + if (on_finalize) + on_finalize(TransferStatusEnum::FAILED); + session_mutex_.unlock(); + if (on_complete) on_complete(); + }); return; } total_transferred_bytes_ += transferred_bytes; -- 2.34.1 From 579d283d83cf369cc9bb9a07b630c9923ada9776 Mon Sep 17 00:00:00 2001 From: lujh <101535776+LujhCoconut@users.noreply.github.com> Date: Wed, 3 Jun 2026 10:31:07 +0800 Subject: [PATCH 170/382] [Store](refactor) Extract 3FS logic into DistributedStorageBackend (#2234) --- .github/workflows/ci.yml | 2 +- mooncake-store/go/build.sh | 2 +- .../distributed/distributed_storage_backend.h | 67 +++ .../include/storage/distributed/fs_adapter.h | 112 +++++ .../storage/distributed/hf3fs_adapter.h | 51 +++ mooncake-store/include/storage_backend.h | 52 +-- mooncake-store/include/types.h | 8 + mooncake-store/src/CMakeLists.txt | 2 + mooncake-store/src/file_storage.cpp | 2 + .../distributed_storage_backend.cpp | 320 +++++++++++++++ .../src/storage/distributed/hf3fs_adapter.cpp | 382 ++++++++++++++++++ mooncake-store/src/storage_backend.cpp | 74 ++-- 12 files changed, 995 insertions(+), 79 deletions(-) create mode 100644 mooncake-store/include/storage/distributed/distributed_storage_backend.h create mode 100644 mooncake-store/include/storage/distributed/fs_adapter.h create mode 100644 mooncake-store/include/storage/distributed/hf3fs_adapter.h create mode 100644 mooncake-store/src/storage/distributed/distributed_storage_backend.cpp create mode 100644 mooncake-store/src/storage/distributed/hf3fs_adapter.cpp diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f549bf38..8134836c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -172,7 +172,7 @@ jobs: export LD_LIBRARY_PATH=$GITHUB_WORKSPACE/build/mooncake-common:$GITHUB_WORKSPACE/build/mooncake-store/src:$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src:$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src/common/base:$GITHUB_WORKSPACE/build/mooncake-common/etcd export CGO_ENABLED=1 export CGO_CFLAGS="-I$GITHUB_WORKSPACE/mooncake-store/include -I$GITHUB_WORKSPACE/mooncake-transfer-engine/include" - export CGO_LDFLAGS="-L$GITHUB_WORKSPACE/build/mooncake-store/src -L$GITHUB_WORKSPACE/build/mooncake-store/src/cachelib_memory_allocator -L$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src -L$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src/common/base -L$GITHUB_WORKSPACE/build/mooncake-common -L$GITHUB_WORKSPACE/build/mooncake-common/etcd -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio -letcd_wrapper -lstdc++ -lnuma -lglog -lgflags -libverbs -ljsoncpp -lzstd -lcurl -luring -lasan -lm -lgcov" + export CGO_LDFLAGS="-L$GITHUB_WORKSPACE/build/mooncake-store/src -L$GITHUB_WORKSPACE/build/mooncake-store/src/cachelib_memory_allocator -L$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src -L$GITHUB_WORKSPACE/build/mooncake-transfer-engine/src/common/base -L$GITHUB_WORKSPACE/build/mooncake-common -L$GITHUB_WORKSPACE/build/mooncake-common/etcd -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio -letcd_wrapper -lstdc++ -lnuma -lglog -lgflags -libverbs -ljsoncpp -lzstd -lcurl -luring -lasan -lm -lgcov -lxxhash" # Link cudart if CUDA is available (needed for D2H staging in mooncake_store) if [ -d /usr/local/cuda/lib64 ]; then export CGO_LDFLAGS="$CGO_LDFLAGS -L/usr/local/cuda/lib64 -lcudart"; fi ASAN_OPTIONS=detect_leaks=0:verify_asan_link_order=0 MC_METADATA_SERVER=http://127.0.0.1:8080/metadata go test -v ./tests/... diff --git a/mooncake-store/go/build.sh b/mooncake-store/go/build.sh index 9020a280..bae668d4 100755 --- a/mooncake-store/go/build.sh +++ b/mooncake-store/go/build.sh @@ -41,7 +41,7 @@ CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-transfer-engine/src" CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-transfer-engine/src/common/base" CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-common" CGO_LDFLAGS+=" -L${BUILD_DIR}/mooncake-common/src" -CGO_LDFLAGS+=" -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio -lmooncake_common" +CGO_LDFLAGS+=" -lmooncake_store -lcachelib_memory_allocator -ltransfer_engine -lbase -lasio -lmooncake_common -lxxhash" CGO_LDFLAGS+=" -lstdc++ -lnuma -lglog -lgflags -libverbs -ljsoncpp -lzstd -lcurl" if [ -d "/usr/local/cuda/lib64" ]; then diff --git a/mooncake-store/include/storage/distributed/distributed_storage_backend.h b/mooncake-store/include/storage/distributed/distributed_storage_backend.h new file mode 100644 index 00000000..d4c935bb --- /dev/null +++ b/mooncake-store/include/storage/distributed/distributed_storage_backend.h @@ -0,0 +1,67 @@ +#pragma once + +#include + +#include "fs_adapter.h" +#include "storage_backend.h" + +namespace mooncake { + +struct DistributedStorageConfig { + std::string fsdir = "distributed_dir"; + std::string fs_adapter_type = "hf3fs"; + bool enable_health_check = false; + int hash_bucket_count = 256; + + bool Validate() const; + static DistributedStorageConfig FromEnvironment(); +}; + +/** + * @brief Distributed filesystem storage backend. + * + * Implements StorageBackendInterface, delegating I/O to a FileSystemAdapter. + * Does not handle eviction (DFS manages its own space). + */ +class DistributedStorageBackend : public StorageBackendInterface { + public: + DistributedStorageBackend( + const FileStorageConfig& file_storage_config, + const DistributedStorageConfig& distributed_config, + std::unique_ptr fs_adapter); + + tl::expected Init() override; + + tl::expected BatchOffload( + const std::unordered_map>& batch_object, + std::function& keys, + std::vector& metadatas)> + complete_handler, + std::function& evicted_keys)> + eviction_handler = nullptr) override; + + tl::expected BatchLoad( + std::unordered_map& batched_slices) override; + + tl::expected IsExist(const std::string& key) override; + + tl::expected IsEnableOffloading() override; + + tl::expected ScanMeta( + const std::function& keys, + std::vector& metadatas)>& handler) override; + + private: + std::string GetObjectPath(const std::string& key) const; + static std::string EscapeFilename(const std::string& key); + static std::string UnescapeFilename(const std::string& name); + + std::unique_ptr fs_adapter_; + DistributedStorageConfig distributed_config_; + std::string root_dir_; + int hash_bucket_count_; + bool initialized_ = false; +}; + +} // namespace mooncake diff --git a/mooncake-store/include/storage/distributed/fs_adapter.h b/mooncake-store/include/storage/distributed/fs_adapter.h new file mode 100644 index 00000000..defda616 --- /dev/null +++ b/mooncake-store/include/storage/distributed/fs_adapter.h @@ -0,0 +1,112 @@ +#pragma once + +#include +#include + +#include +#include +#include + +#include "types.h" + +namespace mooncake { + +struct FileInfo { + std::string name; + size_t size; +}; + +/** + * @brief Abstract interface for distributed filesystem adapters. + * + * Encapsulates file-level I/O differences across DFS implementations + * (3FS, CephFS, JuiceFS, etc.). DistributedStorageBackend depends only + * on this interface and is unaware of the concrete DFS. + */ +class FileSystemAdapter { + public: + virtual ~FileSystemAdapter() = default; + + // === File I/O === + + virtual tl::expected WriteFile( + const std::string& path, std::span data) = 0; + + // Read file into a pre-allocated buffer (zero-copy into Slice.ptr) + virtual tl::expected ReadFile(const std::string& path, + void* buf, size_t len) = 0; + + virtual tl::expected VectorWriteFile( + const std::string& path, const iovec* iov, int iovcnt, + off_t offset) = 0; + + virtual tl::expected VectorReadFile( + const std::string& path, const iovec* iov, int iovcnt, + off_t offset) = 0; + + // === File management === + + virtual tl::expected DeleteFile( + const std::string& path) = 0; + + virtual tl::expected FileExists( + const std::string& path) = 0; + + virtual tl::expected, ErrorCode> ListFiles( + const std::string& dir) = 0; + + virtual tl::expected GetFileSize( + const std::string& path) { + struct stat st; + if (::stat(path.c_str(), &st) != 0) { + if (errno == ENOENT) { + return tl::make_unexpected(ErrorCode::FILE_NOT_FOUND); + } + return tl::make_unexpected(ErrorCode::FILE_READ_FAIL); + } + return static_cast(st.st_size); + } + + // === Batch operations (default implementations, adapters may override) === + + virtual tl::expected DeleteFiles( + const std::vector& paths) { + for (const auto& path : paths) { + auto result = DeleteFile(path); + if (!result) return result; + } + return {}; + } + + virtual tl::expected, ErrorCode> ListFilesWithInfo( + const std::string& dir) { + auto files = ListFiles(dir); + if (!files) return tl::make_unexpected(files.error()); + + std::vector result; + result.reserve(files->size()); + for (const auto& name : *files) { + std::string full_path = dir + "/" + name; + auto size = GetFileSize(full_path); + if (size) { + result.push_back({name, *size}); + } else if (size.error() != ErrorCode::FILE_NOT_FOUND) { + return tl::make_unexpected(size.error()); + } + } + return result; + } + + // === Lifecycle === + + virtual tl::expected Init( + const std::string& mount_path) = 0; + + virtual tl::expected Shutdown() = 0; + + // === Identity === + + virtual const char* GetName() const = 0; +}; + +} // namespace mooncake diff --git a/mooncake-store/include/storage/distributed/hf3fs_adapter.h b/mooncake-store/include/storage/distributed/hf3fs_adapter.h new file mode 100644 index 00000000..4b0fa05f --- /dev/null +++ b/mooncake-store/include/storage/distributed/hf3fs_adapter.h @@ -0,0 +1,51 @@ +#pragma once + +#include + +#include "storage/distributed/fs_adapter.h" + +namespace mooncake { + +// Forward declaration: avoid including hf3fs/hf3fs.h in the header. +// Full type is only used in hf3fs_adapter.cpp. +class USRBIOResourceManager; + +class Hf3fsAdapter : public FileSystemAdapter { + public: + Hf3fsAdapter(); + ~Hf3fsAdapter() override; // defined in .cpp (needs complete type for + // unique_ptr deleter) + + tl::expected WriteFile( + const std::string& path, std::span data) override; + + tl::expected ReadFile(const std::string& path, void* buf, + size_t len) override; + + tl::expected VectorWriteFile(const std::string& path, + const iovec* iov, + int iovcnt, + off_t offset) override; + + tl::expected VectorReadFile(const std::string& path, + const iovec* iov, int iovcnt, + off_t offset) override; + + tl::expected DeleteFile(const std::string& path) override; + + tl::expected FileExists(const std::string& path) override; + + tl::expected, ErrorCode> ListFiles( + const std::string& dir) override; + + tl::expected Init(const std::string& mount_path) override; + + tl::expected Shutdown() override; + + const char* GetName() const override { return "hf3fs"; } + + private: + std::unique_ptr resource_manager_; +}; + +} // namespace mooncake diff --git a/mooncake-store/include/storage_backend.h b/mooncake-store/include/storage_backend.h index be297450..5fd071b5 100644 --- a/mooncake-store/include/storage_backend.h +++ b/mooncake-store/include/storage_backend.h @@ -155,7 +155,12 @@ struct OffloadMetadata { enum class FileMode { Read, Write }; -enum class StorageBackendType { kFilePerKey, kBucket, kOffsetAllocator }; +enum class StorageBackendType { + kFilePerKey, + kBucket, + kOffsetAllocator, + kDistributed +}; static constexpr size_t kKB = 1024; static constexpr size_t kMB = kKB * 1024; @@ -294,40 +299,25 @@ class StorageBackendInterface { */ class StorageBackend { public: -/** - * @brief Constructs a new StorageBackend instance - * @param root_dir Root directory path for object storage - * @param fsdir subdirectory name - * @note Directory existence is not checked in constructor - */ -#ifdef USE_3FS - explicit StorageBackend(const std::string& root_dir, - const std::string& fsdir, bool is_3fs_dir, - bool enable_eviction = true) - : root_dir_(root_dir), - fsdir_(fsdir), - is_3fs_dir_(is_3fs_dir), - enable_eviction_(enable_eviction) { - resource_manager_ = std::make_unique(); - Hf3fsConfig config; - config.mount_root = root_dir; - resource_manager_->setDefaultParams(config); - } -#else + /** + * @brief Constructs a new StorageBackend instance + * @param root_dir Root directory path for object storage + * @param fsdir subdirectory name + * @note Directory existence is not checked in constructor + */ explicit StorageBackend(const std::string& root_dir, const std::string& fsdir, bool enable_eviction = true) : root_dir_(root_dir), fsdir_(fsdir), enable_eviction_(enable_eviction) {} -#endif /** * @brief Factory method to create a StorageBackend instance * @param root_dir Root directory path for object storage * @param fsdir subdirectory name * @param enable_eviction Whether to enable disk eviction feature (default: - * true) Note: Eviction is automatically disabled for 3FS mode + * true) Note: Eviction is controlled by the enable_eviction parameter * @return shared_ptr to new instance or nullptr if directory is invalid * * Performs validation of the root directory before creating the instance: @@ -352,15 +342,8 @@ class StorageBackend { fs::path root_path(root_dir); std::string real_fsdir = "moon_" + fsdir; -#ifdef USE_3FS - bool is_3fs_dir = fs::exists(root_path / "3fs-virt") && - fs::is_directory(root_path / "3fs-virt"); - return std::make_shared(root_dir, real_fsdir, - is_3fs_dir, enable_eviction); -#else return std::make_shared(root_dir, real_fsdir, enable_eviction); -#endif } /** @@ -476,12 +459,6 @@ class StorageBackend { true}; // User-configurable flag to enable/disable eviction bool use_uring_{false}; // Use io_uring for file I/O -#ifdef USE_3FS - bool is_3fs_dir_{false}; // Flag to indicate if the storage is using 3FS - // directory structure - std::unique_ptr resource_manager_; -#endif - private: // File write queue for disk eviction - tracks files in FIFO order std::list file_write_queue_; @@ -583,8 +560,7 @@ class StorageBackend { /** * @brief Checks if disk eviction is enabled for this storage backend. - * @return true if eviction is enabled (local mode), false if disabled (3FS - * mode). + * @return true if eviction is enabled, false otherwise. */ bool IsEvictionEnabled() const; diff --git a/mooncake-store/include/types.h b/mooncake-store/include/types.h index 18ab8ed7..801693ee 100644 --- a/mooncake-store/include/types.h +++ b/mooncake-store/include/types.h @@ -398,6 +398,14 @@ enum class ErrorCode : int32_t { TASK_PENDING_LIMIT_EXCEEDED = -1401, ///< Total pending tasks exceed the limit. JOB_NOT_FOUND = -1402, ///< Job not found. + + // DFS errors (Range: -1600 to -1699) + DFS_NETWORK_TIMEOUT = -1600, ///< DFS network timeout. + DFS_SERVICE_UNAVAILABLE = -1601, ///< DFS service unavailable. + DFS_QUOTA_EXCEEDED = -1602, ///< DFS quota exceeded. + DFS_PERMISSION_DENIED = -1603, ///< DFS permission denied. + DFS_STALE_HANDLE = -1604, ///< DFS file handle expired. + DFS_PARTIAL_WRITE = -1605, ///< DFS partial write success. }; int32_t toInt(ErrorCode errorCode) noexcept; diff --git a/mooncake-store/src/CMakeLists.txt b/mooncake-store/src/CMakeLists.txt index abd184e2..b384bf0e 100644 --- a/mooncake-store/src/CMakeLists.txt +++ b/mooncake-store/src/CMakeLists.txt @@ -27,6 +27,7 @@ set(MOONCAKE_STORE_SOURCES http_metadata_server.cpp file_storage.cpp serialize/serializer.cpp + storage/distributed/distributed_storage_backend.cpp ha/leadership/leader_coordinator_factory.cpp ha/leadership/backends/etcd/etcd_leader_coordinator.cpp ha/common/redis/redis_connection.cpp @@ -116,6 +117,7 @@ endif() if(USE_3FS) add_subdirectory(hf3fs) list(APPEND MOONCAKE_STORE_SOURCES ${HF3FS_SOURCES}) + list(APPEND MOONCAKE_STORE_SOURCES storage/distributed/hf3fs_adapter.cpp) find_library( HF3FS_API_LIB hf3fs_api_shared PATHS /usr/lib diff --git a/mooncake-store/src/file_storage.cpp b/mooncake-store/src/file_storage.cpp index 39653822..4830c13e 100644 --- a/mooncake-store/src/file_storage.cpp +++ b/mooncake-store/src/file_storage.cpp @@ -52,6 +52,8 @@ FileStorageConfig FileStorageConfig::FromEnvironment() { } else if (storage_backend_descriptor == "offset_allocator_storage_backend") { config.storage_backend_type = StorageBackendType::kOffsetAllocator; + } else if (storage_backend_descriptor == "distributed_storage_backend") { + config.storage_backend_type = StorageBackendType::kDistributed; } else { LOG(ERROR) << "Unknown storage backend."; } diff --git a/mooncake-store/src/storage/distributed/distributed_storage_backend.cpp b/mooncake-store/src/storage/distributed/distributed_storage_backend.cpp new file mode 100644 index 00000000..7c8e6ffb --- /dev/null +++ b/mooncake-store/src/storage/distributed/distributed_storage_backend.cpp @@ -0,0 +1,320 @@ +#include "storage/distributed/distributed_storage_backend.h" + +#include + +#include +#include +#include +#include + +#include "utils.h" + +namespace mooncake { + +// === DistributedStorageConfig === + +bool DistributedStorageConfig::Validate() const { + if (fsdir.empty()) { + LOG(ERROR) << "DistributedStorageConfig: fsdir is empty"; + return false; + } + if (!std::filesystem::path(fsdir).is_absolute()) { + LOG(ERROR) + << "DistributedStorageConfig: fsdir must be an absolute path: " + << fsdir; + return false; + } + if (fs_adapter_type.empty()) { + LOG(ERROR) << "DistributedStorageConfig: fs_adapter_type is empty"; + return false; + } + if (fs_adapter_type != "hf3fs") { + LOG(ERROR) << "DistributedStorageConfig: unsupported fs_adapter_type: " + << fs_adapter_type; + return false; + } + if (hash_bucket_count <= 0) { + LOG(ERROR) << "DistributedStorageConfig: hash_bucket_count must > 0"; + return false; + } + return true; +} + +DistributedStorageConfig DistributedStorageConfig::FromEnvironment() { + DistributedStorageConfig config; + config.fsdir = + GetEnvStringOr("MOONCAKE_DISTRIBUTED_ROOT_DIR", config.fsdir); + if (!std::filesystem::path(config.fsdir).is_absolute()) { + config.fsdir = std::filesystem::absolute(config.fsdir).string(); + } + config.fs_adapter_type = + GetEnvStringOr("MOONCAKE_DISTRIBUTED_FS_TYPE", config.fs_adapter_type); + config.enable_health_check = + GetEnvOr("MOONCAKE_DISTRIBUTED_HEALTH_CHECK", false); + config.hash_bucket_count = + GetEnvOr("MOONCAKE_DISTRIBUTED_HASH_BUCKET_COUNT", 256); + return config; +} + +// === DistributedStorageBackend === + +DistributedStorageBackend::DistributedStorageBackend( + const FileStorageConfig& file_storage_config, + const DistributedStorageConfig& distributed_config, + std::unique_ptr fs_adapter) + : StorageBackendInterface(file_storage_config), + fs_adapter_(std::move(fs_adapter)), + distributed_config_(distributed_config), + root_dir_(distributed_config.fsdir), + hash_bucket_count_(distributed_config.hash_bucket_count) {} + +tl::expected DistributedStorageBackend::Init() { + if (initialized_) { + LOG(WARNING) << "DistributedStorageBackend is already initialized"; + return {}; + } + + auto init_result = fs_adapter_->Init(root_dir_); + if (!init_result) return init_result; + + // Ensure root directory exists before health check + std::error_code ec; + std::filesystem::create_directories(root_dir_, ec); + if (ec) { + LOG(ERROR) << "Failed to create root directory " << root_dir_ << ": " + << ec.message(); + return tl::make_unexpected(ErrorCode::FILE_WRITE_FAIL); + } + + if (distributed_config_.enable_health_check) { + std::string probe_path = + fmt::format("{}/.mooncake_health_probe_{}", root_dir_, + UuidToString(generate_uuid())); + std::string probe_data = "health_check"; + auto write_result = fs_adapter_->WriteFile( + probe_path, + std::span(probe_data.data(), probe_data.size())); + if (!write_result) { + LOG(ERROR) << "DFS health check failed (write): " + << static_cast(write_result.error()); + return tl::make_unexpected(write_result.error()); + } + + std::vector read_buf(probe_data.size()); + auto read_result = + fs_adapter_->ReadFile(probe_path, read_buf.data(), read_buf.size()); + if (!read_result || *read_result != probe_data.size() || + std::string(read_buf.data(), read_buf.size()) != probe_data) { + LOG(ERROR) << "DFS health check failed (read back mismatch)"; + auto del_err = fs_adapter_->DeleteFile(probe_path); + if (!del_err) { + LOG(WARNING) << "Failed to delete health-check probe: " + << static_cast(del_err.error()); + } + return tl::make_unexpected(ErrorCode::DFS_SERVICE_UNAVAILABLE); + } + + fs_adapter_->DeleteFile(probe_path); + LOG(INFO) << "DFS health check passed, adapter=" + << fs_adapter_->GetName(); + } + + // Ensure hash bucket directories exist + for (int i = 0; i < hash_bucket_count_; ++i) { + std::string bucket_dir = fmt::format("{}/{:02x}", root_dir_, i); + std::filesystem::create_directories(bucket_dir, ec); + if (ec) { + LOG(ERROR) << "Failed to create bucket directory " << bucket_dir + << ": " << ec.message(); + return tl::make_unexpected(ErrorCode::FILE_WRITE_FAIL); + } + } + + initialized_ = true; + return {}; +} + +tl::expected DistributedStorageBackend::BatchOffload( + const std::unordered_map>& batch_object, + std::function& keys, + std::vector& metadatas)> + complete_handler, + std::function& evicted_keys)> + eviction_handler) { + if (!initialized_) { + LOG(ERROR) << "DistributedStorageBackend is not initialized"; + return tl::make_unexpected(ErrorCode::INTERNAL_ERROR); + } + + if (eviction_handler) { + LOG_FIRST_N(WARNING, 1) + << "DistributedStorageBackend does not support eviction, " + "eviction_handler ignored"; + } + + std::vector success_keys; + std::vector success_metas; + + for (const auto& [key, slices] : batch_object) { + auto path = GetObjectPath(key); + + std::vector iovs; + for (const auto& slice : slices) { + iovs.push_back({slice.ptr, slice.size}); + } + + auto result = + fs_adapter_->VectorWriteFile(path, iovs.data(), iovs.size(), 0); + if (!result) { + LOG(WARNING) << "Failed to offload key " << key << ": " + << static_cast(result.error()); + continue; + } + + success_keys.push_back(key); + StorageObjectMetadata meta{-1, 0, static_cast(key.size()), + static_cast(*result), ""}; + success_metas.push_back(meta); + } + + if (!success_keys.empty()) { + auto err = complete_handler(success_keys, success_metas); + if (err != ErrorCode::OK) { + return tl::make_unexpected(err); + } + } + + return static_cast(success_keys.size()); +} + +tl::expected DistributedStorageBackend::BatchLoad( + std::unordered_map& batched_slices) { + if (!initialized_) { + LOG(ERROR) << "DistributedStorageBackend is not initialized"; + return tl::make_unexpected(ErrorCode::INTERNAL_ERROR); + } + + for (auto& [key, slice] : batched_slices) { + auto path = GetObjectPath(key); + + auto result = fs_adapter_->ReadFile(path, slice.ptr, slice.size); + if (!result) { + return tl::make_unexpected(result.error()); + } + if (*result != slice.size) { + return tl::make_unexpected(ErrorCode::FILE_READ_FAIL); + } + } + return {}; +} + +tl::expected DistributedStorageBackend::IsExist( + const std::string& key) { + if (!initialized_) { + LOG(ERROR) << "DistributedStorageBackend is not initialized"; + return tl::make_unexpected(ErrorCode::INTERNAL_ERROR); + } + + auto path = GetObjectPath(key); + return fs_adapter_->FileExists(path); +} + +tl::expected DistributedStorageBackend::IsEnableOffloading() { + return true; +} + +tl::expected DistributedStorageBackend::ScanMeta( + const std::function< + ErrorCode(const std::vector& keys, + std::vector& metadatas)>& handler) { + if (!initialized_) { + LOG(ERROR) << "DistributedStorageBackend is not initialized"; + return tl::make_unexpected(ErrorCode::INTERNAL_ERROR); + } + + std::vector batch_keys; + std::vector batch_metas; + const size_t batch_limit = static_cast(std::max( + 1, file_storage_config_.scanmeta_iterator_keys_limit)); + + for (int i = 0; i < hash_bucket_count_; ++i) { + std::string bucket_dir = fmt::format("{}/{:02x}", root_dir_, i); + auto file_infos = fs_adapter_->ListFilesWithInfo(bucket_dir); + if (!file_infos) { + if (file_infos.error() == ErrorCode::FILE_NOT_FOUND) { + continue; + } + LOG(ERROR) << "Failed to list files in bucket " << bucket_dir + << ": " << static_cast(file_infos.error()); + return tl::make_unexpected(file_infos.error()); + } + + for (const auto& info : *file_infos) { + std::string key = UnescapeFilename(info.name); + batch_keys.push_back(key); + StorageObjectMetadata meta{-1, 0, static_cast(key.size()), + static_cast(info.size), ""}; + batch_metas.push_back(meta); + + if (batch_keys.size() >= batch_limit) { + auto err = handler(batch_keys, batch_metas); + if (err != ErrorCode::OK) return tl::make_unexpected(err); + batch_keys.clear(); + batch_metas.clear(); + } + } + } + if (!batch_keys.empty()) { + auto err = handler(batch_keys, batch_metas); + if (err != ErrorCode::OK) return tl::make_unexpected(err); + } + return {}; +} + +// === Key -> Path mapping === + +std::string DistributedStorageBackend::GetObjectPath( + const std::string& key) const { + uint64_t hash = XXH64(key.data(), key.size(), 0); + std::string bucket = fmt::format("{:02x}", hash % hash_bucket_count_); + std::string safe_key = EscapeFilename(key); + return (std::filesystem::path(root_dir_) / bucket / safe_key).string(); +} + +std::string DistributedStorageBackend::EscapeFilename(const std::string& key) { + std::string result; + result.reserve(key.size() + 16); + for (unsigned char c : key) { + if (c == '@' || c == ':' || c == '/' || c == '\\' || c == '%' || + c < 0x20 || c > 0x7e) { + result += fmt::format("%{:02x}", static_cast(c)); + } else { + result += static_cast(c); + } + } + return result; +} + +std::string DistributedStorageBackend::UnescapeFilename( + const std::string& name) { + auto is_hex = [](char c) { + return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || + (c >= 'A' && c <= 'F'); + }; + std::string result; + result.reserve(name.size()); + for (size_t i = 0; i < name.size(); ++i) { + if (name[i] == '%' && i + 2 < name.size() && is_hex(name[i + 1]) && + is_hex(name[i + 2])) { + char hex[3] = {name[i + 1], name[i + 2], 0}; + unsigned long val = strtoul(hex, nullptr, 16); + result += static_cast(val); + i += 2; + } else { + result += name[i]; + } + } + return result; +} + +} // namespace mooncake diff --git a/mooncake-store/src/storage/distributed/hf3fs_adapter.cpp b/mooncake-store/src/storage/distributed/hf3fs_adapter.cpp new file mode 100644 index 00000000..a4ccf3b5 --- /dev/null +++ b/mooncake-store/src/storage/distributed/hf3fs_adapter.cpp @@ -0,0 +1,382 @@ +#include "storage/distributed/hf3fs_adapter.h" + +#include +#include +#include + +#include +#include +#include + +#include "hf3fs/hf3fs.h" + +namespace mooncake { + +// Destructor must be defined in .cpp where USRBIOResourceManager is complete. +Hf3fsAdapter::~Hf3fsAdapter() = default; + +Hf3fsAdapter::Hf3fsAdapter() = default; + +tl::expected Hf3fsAdapter::Init( + const std::string& mount_path) { + resource_manager_ = std::make_unique(); + Hf3fsConfig config{}; + config.mount_root = mount_path; + resource_manager_->setDefaultParams(config); + return {}; +} + +tl::expected Hf3fsAdapter::Shutdown() { + resource_manager_.reset(); + return {}; +} + +tl::expected Hf3fsAdapter::WriteFile( + const std::string& path, std::span data) { + int fd = open(path.c_str(), O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644); + if (fd < 0) return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + + if (hf3fs_reg_fd(fd, 0) > 0) { + close(fd); + ::unlink(path.c_str()); + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + auto* resource = resource_manager_->getThreadResource(); + if (!resource || !resource->initialized) { + hf3fs_dereg_fd(fd); + close(fd); + ::unlink(path.c_str()); + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + auto& threefs_iov = resource->iov_; + auto& ior_write = resource->ior_write_; + const char* data_ptr = data.data(); + size_t length = data.size(); + size_t total_written = 0; + off_t offset = 0; + + while (total_written < length) { + size_t chunk = + std::min(length - total_written, resource->config_.iov_size); + memcpy(threefs_iov.base, data_ptr + total_written, chunk); + + int ret = hf3fs_prep_io(&ior_write, &threefs_iov, false, + threefs_iov.base, fd, offset, chunk, nullptr); + if (ret < 0) break; + + ret = hf3fs_submit_ios(&ior_write); + if (ret < 0) break; + + struct hf3fs_cqe cqe; + ret = hf3fs_wait_for_ios(&ior_write, &cqe, 1, 1, nullptr); + if (ret < 0 || cqe.result < 0) break; + + size_t bytes_written = cqe.result; + total_written += bytes_written; + offset += bytes_written; + if (bytes_written < chunk) break; + } + + hf3fs_dereg_fd(fd); + close(fd); + + if (total_written != length) { + auto unlink_ret = ::unlink(path.c_str()); + if (unlink_ret != 0) { + LOG(WARNING) << "Failed to clean up partial write: " << path; + } + return tl::make_unexpected(ErrorCode::FILE_WRITE_FAIL); + } + return total_written; +} + +tl::expected Hf3fsAdapter::ReadFile(const std::string& path, + void* buf, size_t len) { + if (!buf && len > 0) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + + int fd = open(path.c_str(), O_RDONLY | O_CLOEXEC); + if (fd < 0) { + if (errno == ENOENT) { + return tl::make_unexpected(ErrorCode::FILE_NOT_FOUND); + } + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + if (hf3fs_reg_fd(fd, 0) > 0) { + close(fd); + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + auto* resource = resource_manager_->getThreadResource(); + if (!resource || !resource->initialized) { + hf3fs_dereg_fd(fd); + close(fd); + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + auto& threefs_iov = resource->iov_; + auto& ior_read = resource->ior_read_; + char* dest = static_cast(buf); + size_t total_read = 0; + off_t offset = 0; + + while (total_read < len) { + size_t chunk = std::min(len - total_read, resource->config_.iov_size); + + int ret = hf3fs_prep_io(&ior_read, &threefs_iov, true, threefs_iov.base, + fd, offset, chunk, nullptr); + if (ret < 0) break; + + ret = hf3fs_submit_ios(&ior_read); + if (ret < 0) break; + + struct hf3fs_cqe cqe; + ret = hf3fs_wait_for_ios(&ior_read, &cqe, 1, 1, nullptr); + if (ret < 0 || cqe.result < 0) break; + + size_t bytes_read = cqe.result; + if (bytes_read == 0) break; + + memcpy(dest + total_read, threefs_iov.base, bytes_read); + total_read += bytes_read; + offset += bytes_read; + if (bytes_read < chunk) break; + } + + hf3fs_dereg_fd(fd); + close(fd); + + if (total_read != len) { + return tl::make_unexpected(ErrorCode::FILE_READ_FAIL); + } + return total_read; +} + +tl::expected Hf3fsAdapter::VectorWriteFile( + const std::string& path, const iovec* iov, int iovcnt, off_t offset) { + for (int i = 0; i < iovcnt; ++i) { + if (!iov[i].iov_base && iov[i].iov_len > 0) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + } + + int fd = open(path.c_str(), O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644); + if (fd < 0) return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + + if (hf3fs_reg_fd(fd, 0) > 0) { + close(fd); + ::unlink(path.c_str()); + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + auto* resource = resource_manager_->getThreadResource(); + if (!resource || !resource->initialized) { + hf3fs_dereg_fd(fd); + close(fd); + ::unlink(path.c_str()); + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + auto& threefs_iov = resource->iov_; + auto& ior_write = resource->ior_write_; + + size_t total_length = 0; + for (int i = 0; i < iovcnt; ++i) total_length += iov[i].iov_len; + + size_t total_written = 0; + off_t current_offset = offset; + size_t remaining = total_length; + int iov_idx = 0; + size_t iov_off = 0; + + while (remaining > 0) { + size_t chunk = std::min(remaining, resource->config_.iov_size); + + // Copy from iovec to shared buffer + size_t copied = 0; + char* dest = reinterpret_cast(threefs_iov.base); + while (copied < chunk && iov_idx < iovcnt) { + size_t n = std::min(chunk - copied, iov[iov_idx].iov_len - iov_off); + memcpy(dest + copied, + static_cast(iov[iov_idx].iov_base) + iov_off, n); + copied += n; + iov_off += n; + if (iov_off >= iov[iov_idx].iov_len) { + iov_idx++; + iov_off = 0; + } + } + + int ret = + hf3fs_prep_io(&ior_write, &threefs_iov, false, threefs_iov.base, fd, + current_offset, chunk, nullptr); + if (ret < 0) break; + ret = hf3fs_submit_ios(&ior_write); + if (ret < 0) break; + struct hf3fs_cqe cqe; + ret = hf3fs_wait_for_ios(&ior_write, &cqe, 1, 1, nullptr); + if (ret < 0 || cqe.result < 0) break; + + size_t bytes_written = cqe.result; + total_written += bytes_written; + current_offset += bytes_written; + remaining -= bytes_written; + if (bytes_written < chunk) break; + } + + hf3fs_dereg_fd(fd); + close(fd); + + if (total_written != total_length) { + auto unlink_ret = ::unlink(path.c_str()); + if (unlink_ret != 0) { + LOG(WARNING) << "Failed to clean up partial write: " << path; + } + return tl::make_unexpected(ErrorCode::FILE_WRITE_FAIL); + } + return total_written; +} + +tl::expected Hf3fsAdapter::VectorReadFile( + const std::string& path, const iovec* iov, int iovcnt, off_t offset) { + for (int i = 0; i < iovcnt; ++i) { + if (!iov[i].iov_base && iov[i].iov_len > 0) { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + } + + int fd = open(path.c_str(), O_RDONLY | O_CLOEXEC); + if (fd < 0) { + if (errno == ENOENT) { + return tl::make_unexpected(ErrorCode::FILE_NOT_FOUND); + } + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + if (hf3fs_reg_fd(fd, 0) > 0) { + close(fd); + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + auto* resource = resource_manager_->getThreadResource(); + if (!resource || !resource->initialized) { + hf3fs_dereg_fd(fd); + close(fd); + return tl::make_unexpected(ErrorCode::FILE_OPEN_FAIL); + } + + auto& threefs_iov = resource->iov_; + auto& ior_read = resource->ior_read_; + + size_t total_length = 0; + for (int i = 0; i < iovcnt; ++i) total_length += iov[i].iov_len; + + size_t total_read = 0; + off_t current_offset = offset; + size_t remaining = total_length; + int iov_idx = 0; + size_t iov_off = 0; + + while (remaining > 0) { + size_t chunk = std::min(remaining, resource->config_.iov_size); + + int ret = hf3fs_prep_io(&ior_read, &threefs_iov, true, threefs_iov.base, + fd, current_offset, chunk, nullptr); + if (ret < 0) break; + ret = hf3fs_submit_ios(&ior_read); + if (ret < 0) break; + struct hf3fs_cqe cqe; + ret = hf3fs_wait_for_ios(&ior_read, &cqe, 1, 1, nullptr); + if (ret < 0 || cqe.result < 0) break; + + size_t bytes_read = cqe.result; + if (bytes_read == 0) break; + + // Copy from shared buffer to iovec + size_t to_copy = bytes_read; + char* src = reinterpret_cast(threefs_iov.base); + while (to_copy > 0 && iov_idx < iovcnt) { + size_t n = std::min(to_copy, iov[iov_idx].iov_len - iov_off); + memcpy(static_cast(iov[iov_idx].iov_base) + iov_off, src, n); + src += n; + to_copy -= n; + total_read += n; + remaining -= n; + current_offset += n; + iov_off += n; + if (iov_off >= iov[iov_idx].iov_len) { + iov_idx++; + iov_off = 0; + } + } + if (bytes_read < chunk) break; + } + + hf3fs_dereg_fd(fd); + close(fd); + + if (total_read != total_length) { + return tl::make_unexpected(ErrorCode::FILE_READ_FAIL); + } + return total_read; +} + +tl::expected Hf3fsAdapter::DeleteFile( + const std::string& path) { + if (::unlink(path.c_str()) != 0) { + if (errno == ENOENT) { + return tl::make_unexpected(ErrorCode::FILE_NOT_FOUND); + } + return tl::make_unexpected(ErrorCode::FILE_WRITE_FAIL); + } + return {}; +} + +// Note: FileExists and DeleteFile use POSIX access()/unlink() via the +// kernel VFS mount. This assumes 3FS's FUSE mount namespace is consistent +// with the USRBIO namespace used for read/write I/O. + +tl::expected Hf3fsAdapter::FileExists( + const std::string& path) { + if (::access(path.c_str(), F_OK) == 0) { + return true; + } + if (errno == ENOENT) { + return false; + } + return tl::make_unexpected(ErrorCode::FILE_READ_FAIL); +} + +tl::expected, ErrorCode> Hf3fsAdapter::ListFiles( + const std::string& dir) { + DIR* d = opendir(dir.c_str()); + if (!d) { + if (errno == ENOENT) { + return tl::make_unexpected(ErrorCode::FILE_NOT_FOUND); + } + return tl::make_unexpected(ErrorCode::FILE_READ_FAIL); + } + std::vector result; + + struct dirent* entry; + while ((entry = readdir(d)) != nullptr) { + std::string name = entry->d_name; + if (name == "." || name == "..") continue; + if (entry->d_type == DT_DIR) continue; + if (entry->d_type == DT_UNKNOWN) { + struct stat st; + if (::stat((dir + "/" + name).c_str(), &st) == 0 && + S_ISDIR(st.st_mode)) + continue; + } + result.push_back(name); + } + closedir(d); + return result; +} + +} // namespace mooncake diff --git a/mooncake-store/src/storage_backend.cpp b/mooncake-store/src/storage_backend.cpp index c269d477..68d5b216 100644 --- a/mooncake-store/src/storage_backend.cpp +++ b/mooncake-store/src/storage_backend.cpp @@ -21,6 +21,7 @@ #include "utils.h" #include +#include "storage/distributed/distributed_storage_backend.h" namespace mooncake { @@ -104,23 +105,10 @@ void StorageBackend::RecalculateAvailableSpace() { } } -bool StorageBackend::IsEvictionEnabled() const { - // First check user configuration - if (!enable_eviction_) { - return false; - } - -#ifdef USE_3FS - // Eviction is only enabled for local storage, not for 3FS - return !is_3fs_dir_; -#else - // If 3FS is not compiled in, eviction is enabled if user config allows - return true; -#endif -} +bool StorageBackend::IsEvictionEnabled() const { return enable_eviction_; } tl::expected StorageBackend::Init(uint64_t quota_bytes = 0) { - // Skip eviction initialization for 3FS mode + // Skip eviction initialization if disabled if (!IsEvictionEnabled()) { initialized_.store(true, std::memory_order_release); return {}; @@ -208,7 +196,7 @@ tl::expected StorageBackend::Init(uint64_t quota_bytes = 0) { if (total_space_ >= used_space_) { RecalculateAvailableSpace(); } else { - // Only enable eviction for local storage, not for 3FS + // Only enable eviction for local storage if (IsEvictionEnabled()) { eviction_needed = true; available_space_ = -1; @@ -217,10 +205,11 @@ tl::expected StorageBackend::Init(uint64_t quota_bytes = 0) { << ") exceeds the new quota (" << total_space_ << "). Eviction will be triggered after initial setup."; } else { - // For 3FS mode, just log a warning but don't trigger eviction + // Eviction disabled, just log a warning but don't trigger + // eviction LOG(WARNING) << "Existing used space (" << used_space_ << ") exceeds the new quota (" << total_space_ - << "). Eviction is disabled for 3FS mode."; + << "). Eviction is disabled."; RecalculateAvailableSpace(); // Still calculate available space } } @@ -496,7 +485,7 @@ void StorageBackend::RemoveFile(const std::string& path) { std::this_thread::sleep_for( std::chrono::microseconds(50)); // sleep for 50 us - // For 3FS mode, use original logic (no queue tracking) + // Eviction disabled, use simple delete (no queue tracking) if (!IsEvictionEnabled()) { if (fs::exists(path)) { std::error_code ec; @@ -552,7 +541,7 @@ void StorageBackend::RemoveByRegex(const std::string& regex_pattern) { return; } - // For 3FS mode, use original logic (no queue tracking) + // Eviction disabled, use simple delete (no queue tracking) if (!IsEvictionEnabled()) { fs::path storage_root = fs::path(root_dir_) / fsdir_; if (!fs::exists(storage_root) || !fs::is_directory(storage_root)) { @@ -625,7 +614,7 @@ void StorageBackend::RemoveByRegex(const std::string& regex_pattern) { void StorageBackend::RemoveAll() { namespace fs = std::filesystem; - // For 3FS mode, use original logic (no queue tracking) + // Eviction disabled, use simple delete (no queue tracking) if (!IsEvictionEnabled()) { // Iterate through the root directory and remove all files for (const auto& entry : fs::directory_iterator(root_dir_)) { @@ -788,18 +777,6 @@ std::unique_ptr StorageBackend::create_file( return nullptr; } -#ifdef USE_3FS - if (is_3fs_dir_) { - if (hf3fs_reg_fd(fd, 0) > 0) { - close(fd); - return nullptr; - } - return resource_manager_ ? std::make_unique( - path, fd, resource_manager_.get()) - : nullptr; - } -#endif - #ifdef USE_URING if (use_uring_) { // use_direct_io mirrors the O_DIRECT flag: true for reads, false for @@ -836,8 +813,7 @@ bool StorageBackend::CheckDiskSpace(size_t required_size) { FileRecord StorageBackend::EvictFile() { // Eviction is only enabled for local storage if (!IsEvictionEnabled()) { - LOG(WARNING) - << "Eviction is disabled for 3FS mode. Cannot evict files."; + LOG(WARNING) << "Eviction is disabled. Cannot evict files."; return {}; } @@ -909,8 +885,7 @@ FileRecord StorageBackend::SelectFileToEvictByFIFO() { tl::expected, ErrorCode> StorageBackend::EnsureDiskSpace(size_t required_size) { std::vector evicted_keys; - // If eviction is disabled (3FS mode), skip space checking and eviction - // Let 3FS filesystem handle space management itself + // If eviction is disabled, skip space checking and eviction if (!IsEvictionEnabled()) { return evicted_keys; } @@ -3221,9 +3196,30 @@ CreateStorageBackend(const FileStorageConfig& config) { case StorageBackendType::kOffsetAllocator: { return std::make_shared(config); } + case StorageBackendType::kDistributed: { + auto distributed_config = + DistributedStorageConfig::FromEnvironment(); + if (!distributed_config.Validate()) { + throw std::invalid_argument( + "Invalid DistributedStorage configuration"); + } + std::unique_ptr adapter; + if (distributed_config.fs_adapter_type == "hf3fs") { +#ifdef USE_3FS + adapter = std::make_unique(); +#else + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); +#endif + } else { + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); + } + return std::make_shared( + config, distributed_config, std::move(adapter)); + } default: { - LOG(FATAL) << "Unsupported backend type"; - return tl::make_unexpected(ErrorCode::INTERNAL_ERROR); + LOG(ERROR) << "Unsupported backend type: " + << static_cast(config.storage_backend_type); + return tl::make_unexpected(ErrorCode::INVALID_PARAMS); } } } -- 2.34.1 From a2db8c05e2a48fff8a2b6838e061e260adec307c Mon Sep 17 00:00:00 2001 From: Feng Ren Date: Wed, 3 Jun 2026 17:07:38 +0800 Subject: [PATCH 171/382] [TENT] Add policy name binding to transport selector (#2295) * [TENT] Add policy name binding to transport selector Add ability to bind a request to a specific transport policy by name, making the policy's "name" field in configuration actually useful. Changes: - Add optional `policy_name` field to Request struct (types.h) - Add optional `policy_name` field to SelectionContext (transport_selector.h) - Modify matchesPolicy() to prioritize exact policy name matching when context.policy_name is specified - Pass policy_name from request to context in transfer_engine_impl.cpp When a request specifies policy_name, the selector will only match the policy with that exact name, ignoring other matching conditions (segment_type, priority, memory_pattern, etc.). Co-Authored-By: Claude Opus 4.8 (1M context) * Reformat * Update mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Reformat --------- Co-authored-by: Claude Opus 4.8 (1M context) Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- mooncake-transfer-engine/tent/include/tent/common/types.h | 3 +++ .../tent/include/tent/runtime/transport_selector.h | 2 ++ .../tent/src/runtime/transfer_engine_impl.cpp | 1 + .../tent/src/runtime/transport_selector.cpp | 7 +++++++ 4 files changed, 13 insertions(+) diff --git a/mooncake-transfer-engine/tent/include/tent/common/types.h b/mooncake-transfer-engine/tent/include/tent/common/types.h index f999ddb8..c324caac 100644 --- a/mooncake-transfer-engine/tent/include/tent/common/types.h +++ b/mooncake-transfer-engine/tent/include/tent/common/types.h @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -51,6 +52,8 @@ struct Request { size_t length; int priority = PRIO_HIGH; // Request priority (PRIO_HIGH, PRIO_MEDIUM, PRIO_LOW) + std::optional + policy_name; // Optional: bind to specific policy by name }; enum TransferStatusEnum { diff --git a/mooncake-transfer-engine/tent/include/tent/runtime/transport_selector.h b/mooncake-transfer-engine/tent/include/tent/runtime/transport_selector.h index 7fc0dccf..e0f085de 100644 --- a/mooncake-transfer-engine/tent/include/tent/runtime/transport_selector.h +++ b/mooncake-transfer-engine/tent/include/tent/runtime/transport_selector.h @@ -80,6 +80,8 @@ struct SelectionContext { buffer_transports; // Pointer to transports in buffer size_t transfer_size; // Transfer size in bytes int priority_level; // Request priority level (lower = more urgent) + std::optional + policy_name; // Optional: bind to specific policy by name }; /** diff --git a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp index 01f0a596..db214e0c 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transfer_engine_impl.cpp @@ -861,6 +861,7 @@ SelectionResult TransferEngineImpl::getTransportType(const Request& request, ctx.transfer_size = request.length; ctx.priority_level = request.priority; // Use request priority for selection + ctx.policy_name = request.policy_name; // Optional: bind to specific policy if (desc->type == SegmentType::File) { // File segment: use selector with empty buffer_transports diff --git a/mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp b/mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp index 9cf0a430..52b697e7 100644 --- a/mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp +++ b/mooncake-transfer-engine/tent/src/runtime/transport_selector.cpp @@ -237,6 +237,13 @@ bool TransportSelector::matchesMemoryPattern(const std::string& pattern, bool TransportSelector::matchesPolicy(const SelectionPolicy& policy, const SelectionContext& context) const { + // If context specifies a policy_name, only match that exact policy with + // matching segment type + if (context.policy_name.has_value()) { + return context.policy_name.value() == policy.name && + policy.segment_type == context.segment_type; + } + // Check segment type if (policy.segment_type != context.segment_type) { return false; -- 2.34.1 From 82a3416ea6c3c44839cc67bccad31665f3907ca3 Mon Sep 17 00:00:00 2001 From: Aoi Date: Wed, 3 Jun 2026 17:10:40 +0800 Subject: [PATCH 172/382] [Doc] reorganize API reference with Python/C++/HTTP sub-indices (#2263) --- docs/source/api-reference/cpp/index.md | 16 ++ .../api-reference/cpp/mooncake-store.md | 178 ++++++++++++ docs/source/api-reference/http/index.md | 13 + docs/source/api-reference/python/index.md | 16 ++ docs/source/design/mooncake-store.md | 255 ++++-------------- docs/source/design/tent/overview.md | 6 +- docs/source/design/transfer-engine/index.md | 14 +- .../examples/lmcache-integration.md | 7 +- .../examples/vllm-integration/index.md | 11 - .../vllm-mooncakestoreconnector.md | 2 +- docs/source/index.md | 30 +-- 11 files changed, 299 insertions(+), 249 deletions(-) create mode 100644 docs/source/api-reference/cpp/index.md create mode 100644 docs/source/api-reference/cpp/mooncake-store.md create mode 100644 docs/source/api-reference/http/index.md create mode 100644 docs/source/api-reference/python/index.md diff --git a/docs/source/api-reference/cpp/index.md b/docs/source/api-reference/cpp/index.md new file mode 100644 index 00000000..35e77191 --- /dev/null +++ b/docs/source/api-reference/cpp/index.md @@ -0,0 +1,16 @@ +# C/C++ API Reference + +| Module | Description | +|--------|-------------| +| [Transfer Engine C++ API](../../design/transfer-engine/cpp-api) | `TransferEngine` class — memory registration, batch transfer, segment management, RDMA transport | +| [TENT C++ API](../../design/tent/cpp-api) | `mooncake::tent::TransferEngine` — next-gen transfer engine with automatic transport selection and fault tolerance | +| [Mooncake Store Client C++ API](mooncake-store) | `Client` class — `Put`/`Get`/`Remove`/`Replicate` operations, `BufferAllocatorBase` interface | + +:::{toctree} +:maxdepth: 1 +:hidden: + +../../design/transfer-engine/cpp-api +../../design/tent/cpp-api +mooncake-store +::: diff --git a/docs/source/api-reference/cpp/mooncake-store.md b/docs/source/api-reference/cpp/mooncake-store.md new file mode 100644 index 00000000..41c3028a --- /dev/null +++ b/docs/source/api-reference/cpp/mooncake-store.md @@ -0,0 +1,178 @@ +# Mooncake Store C++ API Reference + +## Client C++ API + +### Constructor and Initialization `Init` + +```C++ +ErrorCode Init(const std::string& local_hostname, + const std::string& metadata_connstring, + const std::string& protocol, + void** protocol_args, + const std::string& master_server_entry); +``` + +Initializes the Mooncake Store client. The parameters are as follows: +- `local_hostname`: The `IP:Port` of the local machine or an accessible domain name (default value used if port is not included) +- `metadata_connstring`: The address of the metadata service (e.g., etcd/Redis) required for Transfer Engine initialization +- `protocol`: The protocol supported by the Transfer Engine, including RDMA and TCP +- `protocol_args`: Protocol parameters required by the Transfer Engine +- `master_server_entry`: The address information of the Master (`IP:Port` for default mode and `etcd://IP:Port;IP:Port;...;IP:Port` for high availability mode) + +### Get + +```C++ +tl::expected Get(const std::string& object_key, + std::vector& slices); +``` + +`Get` retrieves the value of `object_key` into the provided `slices`. The returned data is guaranteed to be complete and correct. Each slice must reference local DRAM/VRAM memory that has been pre-registered with `registerLocalMemory(addr, len)` (not the global segments that contribute to the distributed memory pool). When persistence is enabled and the requested data is not found in the distributed memory pool, `Get` will fall back to loading the data from SSD. + +### Put + +```C++ +tl::expected Put(const ObjectKey& key, + std::vector& slices, + const ReplicateConfig& config); +``` + +`Put` stores the value associated with `key` in the distributed memory pool. The `config` parameter allows specifying the required number of replicas as well as the preferred segment for storing the value. When persistence is enabled, `Put` also asynchronously triggers a persistence operation to SSD. + +**Replication Guarantees and Best Effort Behavior:** +- Each slice of an object is guaranteed to be replicated to different segments, ensuring distribution across separate storage nodes +- Different slices from different objects may be placed in the same segment +- Replication operates on a best-effort basis: if insufficient space is available for all requested replicas, the object will still be written with as many replicas as possible + +The data structure details of `ReplicateConfig` are as follows: + +```C++ +struct ReplicateConfig { + size_t replica_num{1}; // Total number of replicas for the object + bool with_soft_pin{false}; // Whether to enable soft pin mechanism for this object + bool with_hard_pin{false}; // Whether to enable hard pin (never evicted) + std::string preferred_segment{}; // Preferred segment for allocation +}; +``` + +### Upsert + +```C++ +tl::expected Upsert(const ObjectKey& key, + std::vector& slices, + const ReplicateConfig& config); + +std::vector> BatchUpsert( + const std::vector& keys, + std::vector>& batched_slices, + const ReplicateConfig& config); +``` + +`Upsert` inserts `key` if it does not exist and updates the existing object if +it does. It uses the same replication configuration model as `Put`, while +allowing the store to reuse existing placement for in-place updates when the +current layout permits it. `BatchUpsert` performs the same operation for +multiple keys using a shared replication configuration. + +### Remove + +```C++ +tl::expected Remove(const ObjectKey& key); +``` + +Used to delete the object corresponding to the specified key. This interface marks all data replicas associated with the key in the storage engine as deleted, without needing to communicate with the corresponding storage node (Client). + +### CreateCopyTask + +```C++ +tl::expected CreateCopyTask( + const std::string& key, + const std::vector& targets); +``` + +`CreateCopyTask` creates an asynchronous copy task that will be executed by the client's task execution system. This is useful when you want to submit multiple copy operations without waiting for each one to complete. The task is submitted to the master service, assigned a unique task ID, and executed asynchronously by an available client. The task status can be queried using `QueryTask`. + +**Task Execution and Result Reporting:** +1. **Task Assignment**: The master service assigns the task to an available client during the client's periodic ping operation +2. **Task Execution**: The assigned client executes the copy operation asynchronously in a background thread pool +3. **Result Reporting**: Upon completion (success or failure), the client automatically reports the result to the master service via `MarkTaskToComplete`: + - On success: `status = SUCCESS`, `message = "Task completed successfully"` + - On failure: `status = FAILED`, `message = ` +4. **Status Query**: You can query the task status at any time using `QueryTask` to monitor progress + +### CreateMoveTask + +```C++ +tl::expected CreateMoveTask( + const std::string& key, + const std::string& source, + const std::string& target); +``` + +`CreateMoveTask` creates an asynchronous move task that will be executed by the client's task execution system. This is useful when you want to submit multiple move operations without waiting for each one to complete. The task is submitted to the master service, assigned a unique task ID, and executed asynchronously by an available client. The task status can be queried using `QueryTask`. + +**Task Execution and Result Reporting:** +1. **Task Assignment**: The master service assigns the task to an available client during the client's periodic ping operation +2. **Task Execution**: The assigned client executes the move operation asynchronously in a background thread pool +3. **Result Reporting**: Upon completion (success or failure), the client automatically reports the result to the master service via `MarkTaskToComplete`: + - On success: `status = SUCCESS`, `message = "Task completed successfully"` + - On failure: `status = FAILED`, `message = ` +4. **Status Query**: You can query the task status at any time using `QueryTask` to monitor progress + +### QueryTask + +```C++ +tl::expected QueryTask(const UUID& task_id); +``` + +`QueryTask` queries the status of an asynchronous task (copy or move). This allows you to monitor the progress of task-based operations. The response includes task status, type, creation time, last update time, assigned client, and status message. + +The data structure details of `QueryTaskResponse` are as follows: + +```C++ +struct QueryTaskResponse { + UUID id; // Task UUID + TaskType type; // Task type (REPLICA_COPY or REPLICA_MOVE) + TaskStatus status; // Task status (PENDING, PROCESSING, SUCCESS, or FAILED) + int64_t created_at_ms_epoch; // Task creation timestamp in milliseconds + int64_t last_updated_at_ms_epoch; // Last update timestamp in milliseconds + UUID assigned_client; // UUID of the client assigned to execute the task + std::string message; // Status message or error description +}; +``` + +### BatchQueryIp + +```C++ +tl::expected, boost::hash>, ErrorCode> +BatchQueryIp(const std::vector& client_ids); +``` + +Used to batch query the IP addresses for multiple client IDs. For each client ID in the input list, this interface retrieves the unique IP addresses from all segments mounted by that client. The operation is performed on the Master Service and returns a map from client ID to their IP address lists. Only client IDs that have successfully mounted segments are included in the result map. This is useful for discovering the network locations of storage nodes in the cluster. + +### BatchReplicaClear + +```C++ +tl::expected, ErrorCode> +BatchReplicaClear(const std::vector& object_keys, + const UUID& client_id, + const std::string& segment_name); +``` + +Used to batch clear replicas for multiple object keys belonging to a specific client ID. This interface allows clearing replicas either on a specific segment or across all segments. If segment_name is empty, all replicas of the specified objects are cleared (the objects are deleted entirely). If segment_name is provided, only replicas located on that specific segment are cleared. The operation is performed on the Master Service and returns a list of object keys that were successfully cleared. Only objects that belong to the specified `client_id`, have expired leases, and meet the clearing criteria are processed. This is useful for managing storage resources and cleaning up data on specific storage nodes. + +### QueryByRegex + +```C++ +tl::expected>, ErrorCode> +QueryByRegex(const std::string& str); +``` + +Used to query the replica information for all objects whose keys match the given regular expression. This is useful for batch operations or for retrieving a group of related objects. The operation is performed on the Master and returns a map of keys to their replica lists. + +### RemoveByRegex + +```C++ +tl::expected RemoveByRegex(const ObjectKey& str); +``` + +Used to delete all objects from the store whose keys match the specified regular expression. This provides a powerful way to perform bulk deletions. The command returns the number of objects that were successfully removed. diff --git a/docs/source/api-reference/http/index.md b/docs/source/api-reference/http/index.md new file mode 100644 index 00000000..893a1d94 --- /dev/null +++ b/docs/source/api-reference/http/index.md @@ -0,0 +1,13 @@ +# HTTP + +| Module | Description | +|--------|-------------| +| [HTTP Service](../../http-api-reference/http-service) | RESTful endpoints for cluster management, metrics, and data inspection | +| [Conductor Indexer API](../../design/conductor/indexer-api-design) | Cache-aware routing: service registration, KV event subscription, prefix cache-hit query APIs | + +:::{toctree} +:maxdepth: 1 +:hidden: + +../../http-api-reference/http-service +::: diff --git a/docs/source/api-reference/python/index.md b/docs/source/api-reference/python/index.md new file mode 100644 index 00000000..75ca5c5a --- /dev/null +++ b/docs/source/api-reference/python/index.md @@ -0,0 +1,16 @@ +# Python + +| Module | Description | +|--------|-------------| +| [Mooncake Store](../../python-api-reference/mooncake-store) | Distributed KV cache storage client — `put`/`get`/`remove`/`replicate` operations | +| [Transfer Engine](../../python-api-reference/transfer-engine) | High-performance RDMA/TCP data transfer between nodes | +| [EP Backend](../../python-api-reference/ep-backend) | Expert-parallel backend for large MoE model deployment | + +:::{toctree} +:maxdepth: 1 +:hidden: + +../../python-api-reference/mooncake-store +../../python-api-reference/transfer-engine +../../python-api-reference/ep-backend +::: diff --git a/docs/source/design/mooncake-store.md b/docs/source/design/mooncake-store.md index 51e1754c..254385bd 100644 --- a/docs/source/design/mooncake-store.md +++ b/docs/source/design/mooncake-store.md @@ -48,198 +48,33 @@ If the current leader fails or becomes partitioned from the network, the remaini In both modes, the leader monitors the health of all client nodes through periodic heartbeats. If a client crashes or becomes unreachable, the leader quickly detects the failure and takes appropriate action. When a client node recovers or reconnects, it can automatically rejoin the cluster without manual intervention. +(client-c-api)= ## Client C++ API -### Constructor and Initialization `Init` +The `Client` class provides the primary interface for Mooncake Store operations: -```C++ -ErrorCode Init(const std::string& local_hostname, - const std::string& metadata_connstring, - const std::string& protocol, - void** protocol_args, - const std::string& master_server_entry); -``` +| API | Description | +|-----|-------------| +| `Init` | Initialize the client with metadata server, protocol, and master address | +| `Get` | Retrieve object data into pre-registered local memory slices | +| `Put` | Store object data with configurable replication and persistence | +| `Upsert` / `BatchUpsert` | Insert or update with existing placement reuse | +| `Remove` | Delete an object and all its replicas | +| `CreateCopyTask` / `CreateMoveTask` | Asynchronous cross-node data transfer | +| `QueryTask` | Monitor the status of async copy/move tasks | +| `BatchQueryIp` | Discover network locations of storage nodes | +| `BatchReplicaClear` | Batch clear replicas on specific segments | +| `QueryByRegex` / `RemoveByRegex` | Query or delete objects matching a regex | -Initializes the Mooncake Store client. The parameters are as follows: -- `local_hostname`: The `IP:Port` of the local machine or an accessible domain name (default value used if port is not included) -- `metadata_connstring`: The address of the metadata service (e.g., etcd/Redis) required for Transfer Engine initialization -- `protocol`: The protocol supported by the Transfer Engine, including RDMA and TCP -- `protocol_args`: Protocol parameters required by the Transfer Engine -- `master_server_entry`: The address information of the Master (`IP:Port` for default mode and `etcd://IP:Port;IP:Port;...;IP:Port` for high availability mode) +For full API signatures, parameter details, and usage examples, see the [Mooncake Store C++ API Reference](../api-reference/cpp/mooncake-store.md). -### Get - -```C++ -tl::expected Get(const std::string& object_key, - std::vector& slices); -``` - -![mooncake-store-simple-get](../image/mooncake-store-simple-get.png) - -`Get` retrieves the value of `object_key` into the provided `slices`. The returned data is guaranteed to be complete and correct. Each slice must reference local DRAM/VRAM memory that has been pre-registered with `registerLocalMemory(addr, len)` (not the global segments that contribute to the distributed memory pool). When persistence is enabled and the requested data is not found in the distributed memory pool, `Get` will fall back to loading the data from SSD. - -### Put - -```C++ -tl::expected Put(const ObjectKey& key, - std::vector& slices, - const ReplicateConfig& config); -``` - -![mooncake-store-simple-put](../image/mooncake-store-simple-put.png) - -`Put` stores the value associated with `key` in the distributed memory pool. The `config` parameter allows specifying the required number of replicas as well as the preferred segment for storing the value. When persistence is enabled, `Put` also asynchronously triggers a persistence operation to SSD. - -**Replication Guarantees and Best Effort Behavior:** -- Each slice of an object is guaranteed to be replicated to different segments, ensuring distribution across separate storage nodes -- Different slices from different objects may be placed in the same segment -- Replication operates on a best-effort basis: if insufficient space is available for all requested replicas, the object will still be written with as many replicas as possible - -The data structure details of `ReplicateConfig` are as follows: - -```C++ -struct ReplicateConfig { - size_t replica_num{1}; // Total number of replicas for the object - bool with_soft_pin{false}; // Whether to enable soft pin mechanism for this object - bool with_hard_pin{false}; // Whether to enable hard pin (never evicted) - std::string preferred_segment{}; // Preferred segment for allocation -}; -``` - -### Upsert - -```C++ -tl::expected Upsert(const ObjectKey& key, - std::vector& slices, - const ReplicateConfig& config); - -std::vector> BatchUpsert( - const std::vector& keys, - std::vector>& batched_slices, - const ReplicateConfig& config); -``` - -`Upsert` inserts `key` if it does not exist and updates the existing object if -it does. It uses the same replication configuration model as `Put`, while -allowing the store to reuse existing placement for in-place updates when the -current layout permits it. `BatchUpsert` performs the same operation for -multiple keys using a shared replication configuration. - -### Remove - -```C++ -tl::expected Remove(const ObjectKey& key); -``` - -Used to delete the object corresponding to the specified key. This interface marks all data replicas associated with the key in the storage engine as deleted, without needing to communicate with the corresponding storage node (Client). - -### CreateCopyTask - -```C++ -tl::expected CreateCopyTask( - const std::string& key, - const std::vector& targets); -``` - -![mooncake-store-create-copy-task](../image/mooncake-store-client-create-copy-task.png) - -`CreateCopyTask` creates an asynchronous copy task that will be executed by the client's task execution system. This is useful when you want to submit multiple copy operations without waiting for each one to complete. The task is submitted to the master service, assigned a unique task ID, and executed asynchronously by an available client. The task status can be queried using `QueryTask`. - -**Task Execution and Result Reporting:** -1. **Task Assignment**: The master service assigns the task to an available client during the client's periodic ping operation -2. **Task Execution**: The assigned client executes the copy operation asynchronously in a background thread pool -3. **Result Reporting**: Upon completion (success or failure), the client automatically reports the result to the master service via `MarkTaskToComplete`: - - On success: `status = SUCCESS`, `message = "Task completed successfully"` - - On failure: `status = FAILED`, `message = ` -4. **Status Query**: You can query the task status at any time using `QueryTask` to monitor progress - -### CreateMoveTask - -```C++ -tl::expected CreateMoveTask( - const std::string& key, - const std::string& source, - const std::string& target); -``` - -![mooncake-store-create-move-task](../image/mooncake-store-client-create-move-task.png) - -`CreateMoveTask` creates an asynchronous move task that will be executed by the client's task execution system. This is useful when you want to submit multiple move operations without waiting for each one to complete. The task is submitted to the master service, assigned a unique task ID, and executed asynchronously by an available client. The task status can be queried using `QueryTask`. - -**Task Execution and Result Reporting:** -1. **Task Assignment**: The master service assigns the task to an available client during the client's periodic ping operation -2. **Task Execution**: The assigned client executes the move operation asynchronously in a background thread pool -3. **Result Reporting**: Upon completion (success or failure), the client automatically reports the result to the master service via `MarkTaskToComplete`: - - On success: `status = SUCCESS`, `message = "Task completed successfully"` - - On failure: `status = FAILED`, `message = ` -4. **Status Query**: You can query the task status at any time using `QueryTask` to monitor progress - -### QueryTask - -```C++ -tl::expected QueryTask(const UUID& task_id); -``` - -`QueryTask` queries the status of an asynchronous task (copy or move). This allows you to monitor the progress of task-based operations. The response includes task status, type, creation time, last update time, assigned client, and status message. - -The data structure details of `QueryTaskResponse` are as follows: - -```C++ -struct QueryTaskResponse { - UUID id; // Task UUID - TaskType type; // Task type (REPLICA_COPY or REPLICA_MOVE) - TaskStatus status; // Task status (PENDING, PROCESSING, SUCCESS, or FAILED) - int64_t created_at_ms_epoch; // Task creation timestamp in milliseconds - int64_t last_updated_at_ms_epoch; // Last update timestamp in milliseconds - UUID assigned_client; // UUID of the client assigned to execute the task - std::string message; // Status message or error description -}; -``` - -### BatchQueryIp - -```C++ -tl::expected, boost::hash>, ErrorCode> -BatchQueryIp(const std::vector& client_ids); -``` - -Used to batch query the IP addresses for multiple client IDs. For each client ID in the input list, this interface retrieves the unique IP addresses from all segments mounted by that client. The operation is performed on the Master Service and returns a map from client ID to their IP address lists. Only client IDs that have successfully mounted segments are included in the result map. This is useful for discovering the network locations of storage nodes in the cluster. - -### BatchReplicaClear - -```C++ -tl::expected, ErrorCode> -BatchReplicaClear(const std::vector& object_keys, - const UUID& client_id, - const std::string& segment_name); -``` - -Used to batch clear replicas for multiple object keys belonging to a specific client ID. This interface allows clearing replicas either on a specific segment or across all segments. If segment_name is empty, all replicas of the specified objects are cleared (the objects are deleted entirely). If segment_name is provided, only replicas located on that specific segment are cleared. The operation is performed on the Master Service and returns a list of object keys that were successfully cleared. Only objects that belong to the specified `client_id`, have expired leases, and meet the clearing criteria are processed. This is useful for managing storage resources and cleaning up data on specific storage nodes. - -### QueryByRegex - -```C++ -tl::expected>, ErrorCode> -QueryByRegex(const std::string& str); -``` - -Used to query the replica information for all objects whose keys match the given regular expression. This is useful for batch operations or for retrieving a group of related objects. The operation is performed on the Master and returns a map of keys to their replica lists. - -### RemoveByRegex - -```C++ -tl::expected RemoveByRegex(const ObjectKey& str); -``` - -Used to delete all objects from the store whose keys match the specified regular expression. This provides a powerful way to perform bulk deletions. The command returns the number of objects that were successfully removed. - -### Master Service +## Master Service The cluster's available resources are viewed as a large resource pool, managed centrally by a Master process for space allocation and guiding data replication **Note: The Master Service does not take over any data flow, only providing corresponding metadata information.** -#### Snapshot & Restore +### Snapshot & Restore To reduce cache warm-up time after a master restart, the Master Service supports periodic snapshots of its in-memory metadata and recovery from these snapshots. @@ -255,7 +90,7 @@ To reduce cache warm-up time after a master restart, the Master Service supports > > The snapshot storage location is **exclusively managed** by the Mooncake snapshot system. Old snapshots are automatically deleted during cleanup. **DO NOT store other files in this location.** Use a dedicated, isolated storage for snapshots. -#### Master Service APIs +### Master Service APIs The protobuf definition between Master and Client is as follows: @@ -498,7 +333,7 @@ message UnMountSegmentResponse { When the space needs to be released, this interface is used to remove the previously mounted resources from the Master Service. -#### Object Information Maintenance +### Object Information Maintenance The Master Service needs to maintain mappings related to buffer allocators and object metadata to efficiently manage memory resources and precisely control replica states in multi-replica scenarios. Additionally, the Master Service uses read-write locks to protect critical data structures, ensuring data consistency and security in multi-threaded environments. The following are the interfaces maintained by the Master Service for storage space information: @@ -590,7 +425,7 @@ tl::expected RemoveByRegex(const std::string& str); The Client requests the Master Service to delete all replicas corresponding to the specified key or for all object keys that match the specified regular expression. -### Buffer Allocator +## Buffer Allocator The buffer allocator serves as a low-level memory management component within the Mooncake Store system, primarily responsible for efficient memory allocation and deallocation. It builds upon underlying memory allocators to perform its functions. @@ -620,7 +455,7 @@ class BufferAllocatorBase { 3. **`deallocate` Function**: This function is automatically triggered by the `BufHandle` destructor. It calls the internal allocator to release the associated memory and updates the handle’s status to `BufStatus::UNREGISTERED`. -### AllocationStrategy +## AllocationStrategy AllocationStrategy is a strategy class for efficiently managing memory resource allocation and replica storage location selection in a distributed environment. It is mainly used in the following scenarios: - Determining the allocation locations for object storage replicas. - Selecting suitable read/write paths among multiple replicas. @@ -630,7 +465,7 @@ AllocationStrategy is used in conjunction with the Master Service and the underl - Master Service: Determines the target locations for replica allocation via `AllocationStrategy`. - Buffer Allocator: Executes the actual memory allocation and release tasks. -#### APIs +### APIs `Allocate`: Finds suitable storage segments from available storage resources to allocate space of a specified size for multiple replicas. Uses best-effort semantics, meaning it allocates as many replicas as possible even if the full requested count cannot be satisfied. @@ -652,7 +487,7 @@ virtual tl::expected, ErrorCode> Allocate( - On success: vector of allocated replicas (may be fewer than requested due to resource constraints, but at least 1) - On failure: ErrorCode::NO_AVAILABLE_HANDLE if no replicas can be allocated, ErrorCode::INVALID_PARAMS for invalid configuration -#### Allocation Strategies +### Allocation Strategies Mooncake Store provides multiple built-in allocation strategies to control how storage space is distributed across segments. Users can select a strategy via the `--allocation_strategy` flag when starting the master service: @@ -662,7 +497,7 @@ Mooncake Store provides multiple built-in allocation strategies to control how s Valid values are: `random` (default), `free_ratio_first`, `cxl` (case-sensitive). -##### How to Choose +#### How to Choose | Strategy | Best For | Trade-off | |---|---|---| @@ -678,7 +513,7 @@ Valid values are: `random` (default), `free_ratio_first`, `cxl` (case-sensitive) **Use `cxl`** only when your hardware includes CXL (Compute Express Link) memory devices and you want to allocate data exclusively on CXL segments. -##### Strategy Details +#### Strategy Details **`random` — RandomAllocationStrategy** @@ -715,13 +550,13 @@ Specialized for CXL (Compute Express Link) memory hardware. Unlike the other str Limitations: This strategy only supports single-replica allocation (does not distribute across multiple segments) and does not support the `AllocateFrom()` interface. -### Eviction Policy +## Eviction Policy When a `PutStart` request fails due to insufficient memory, or when the eviction thread detects that space usage has reached the configured high watermark (95% by default, configurable via `-eviction_high_watermark_ratio`), an eviction task is triggered to free up space by evicting a portion of objects (5% by default, configurable via `-eviction_ratio`). Similar to `Remove`, evicted objects are simply marked as deleted, with no data transfer required. Currently, an approximate LRU policy is adopted, where the least recently used objects are preferred for eviction. To avoid data races and corruption, objects currently being read or written by clients should not be evicted. For this reason, objects that have leases or have not been marked as complete by `PutEnd` requests will be ignored by the eviction task. -### Lease +## Lease To avoid data conflicts, a per-object lease is granted whenever an `ExistKey` request or a `GetReplicaListRequest` request succeeds. While the lease is active, the object is protected from `Remove`, `RemoveAll`, and `Eviction` operations. Specifically, a `Remove` request targeting a leased object will fail, and a `RemoveAll` request will only delete objects without an active lease. This ensures that the object’s data can be safely read as long as the lease has not expired. @@ -729,7 +564,7 @@ However, if the lease expires before a `Get` operation finishes reading the data The default lease TTL is 5 seconds and is configurable via a startup parameter of `master_service`. -### Soft Pin +## Soft Pin For important and frequently used objects, such as system prompts, Mooncake Store provides a soft pin mechanism. When putting an object, it can be configured to enable soft pin. During eviction, objects that are not soft pinned are prioritized for eviction. Soft pinned objects are only evicted when memory is insufficient and no other objects are eligible for eviction. @@ -743,7 +578,7 @@ There are two startup parameters in `master_service` related to the soft pin mec Notably, soft pinned objects can still be removed using APIs such as `Remove` or `RemoveAll`. -### Hard Pin +## Hard Pin For objects that must never be evicted under any circumstances (e.g., model weights, critical metadata), Mooncake Store provides a hard pin mechanism. Unlike soft pin, hard-pinned objects are permanently protected from eviction — they will never be selected as eviction candidates regardless of memory pressure. @@ -755,23 +590,23 @@ Key differences from soft pin: - Hard-pinned objects are completely skipped during eviction. Soft-pinned objects may still be evicted when no other candidates are available. - Hard pin is immutable once set. Soft pin status is automatically refreshed on access. -### Zombie Object Cleanup +## Zombie Object Cleanup If a Client crashes or experiences a network failure after sending a `PutStart` request but before it can send the corresponding `PutEnd` or `PutRevoke` request to the Master, the object initiated by `PutStart` enters a "zombie" state—rendering it neither usable nor deletable. The existence of such "zombie objects" not only consumes storage space but also prevents subsequent `Put` operations on the same keys. To mitigate these issues, the Master records the start time of each `PutStart` request and employs two timeout thresholds—`put_start_discard_timeout` and `put_start_release_timeout`—to clean up zombie objects. -#### `PutStart` Preemption +### `PutStart` Preemption If an object receives neither a `PutEnd` nor a `PutRevoke` request within `put_start_discard_timeout` (default: 30 seconds) after its `PutStart`, any subsequent `PutStart` request for the same object will be allowed to "preempt" the previous `PutStart`. This enables the new request to proceed with writing the object, thereby preventing a single faulty Client from permanently blocking access to that object. Note that during such preemption, the storage space allocated by the old `PutStart` is not reused; instead, new space is allocated for the preempting `PutStart`. The space previously allocated by the old `PutStart` will be reclaimed via the mechanism described below. -#### Space Reclaim +### Space Reclaim Replica space allocated during a `PutStart` is considered releasable by the Master if the write operation is neither completed (via `PutEnd`) nor canceled (via `PutRevoke`) within `put_start_release_timeout` (default: 10 minutes) after the `PutStart`. When object eviction is triggered—either due to allocation failures or because storage utilization exceeds the configured threshold—these releasable replica spaces are prioritized for release to reclaim storage capacity. -### Preferred Segment Allocation +## Preferred Segment Allocation Mooncake Store provides a **preferred segment allocation** feature that allows users to specify a preferred storage segment (node) for object allocation. This feature is particularly useful for optimizing data locality and reducing network overhead in distributed scenarios. -#### How It Works +### How It Works The preferred segment allocation feature is implemented through the `AllocationStrategy` system and is controlled via the `preferred_segment` field in the `ReplicateConfig` structure: @@ -795,23 +630,23 @@ When a `Put` operation is initiated with a non-empty `preferred_segment` value, - **Data Locality**: By preferring local segments, applications can reduce network traffic and improve access performance for frequently used data. - **Load Balancing**: Applications can distribute data across specific nodes to achieve better load distribution. -### Multi-layer Storage Support +## Multi-layer Storage Support This system provides support for a hierarchical cache architecture, enabling efficient data access through a combination of in-memory caching and persistent storage. Data is initially stored in memory cache and asynchronously backed up to a Distributed File System (DFS), forming a two-tier "memory-SSD persistent storage" cache structure. -#### Enabling Persistence Functionality +### Enabling Persistence Functionality When the user specifies `--root_fs_dir=/path/to/dir` when starting the master, and this path is a valid DFS-mounted directory on all machines where the clients reside, Mooncake Store's tiered caching functionality will work properly. Additionally, during master initialization, a `cluster_id` is loaded. This ID can be specified during master initialization (`--cluster_id=xxxx`). If not specified, the default value `mooncake_cluster` will be used. Subsequently, the root directory for client persistence will be `/`. ​Note​​: When enabling this feature, the user must ensure that the DFS-mounted directory (`root_fs_dir=/path/to/dir`) is valid and consistent across all client hosts. If some clients have invalid or incorrect mount paths, it may cause abnormal behavior in Mooncake Store. -#### Persistent Storage Space Configuration​ +### Persistent Storage Space Configuration​ Mooncake provides configurable DFS available space. Users can specify `--global_file_segment_size=1048576` when starting the master, indicating a maximum usable space of 1MB on DFS. The current default setting is the maximum value of int64 (as we generally do not restrict DFS storage usage), which is displayed as `infinite` in `mooncake_maseter`'s console logs. **Notice** The DFS cache space configuration must be used together with the `--root_fs_dir` parameter. Otherwise, you will observe that the `SSD Storage` usage consistently shows: `0 B / 0 B` **Notice** The capability for file eviction on DFS has not been provided yet -#### Data Access Mechanism +### Data Access Mechanism The persistence feature also follows Mooncake Store's design principle of separating control flow from data flow. The read/write operations of kvcache objects are completed on the client side, while the query and management functions of kvcache objects are handled on the master side. In the file system, the key -> kvcache object index information is maintained by a fixed indexing mechanism, with each file corresponding to one kvcache object (the filename serves as the associated key name). @@ -820,7 +655,7 @@ After enabling the persistence feature: - For each `Put` or `BatchPut` operation, both a synchronous memory pool write operation and an asynchronous DFS persistence operation will be initiated. - For each `Get` or `BatchGet` operation, if the corresponding kvcache is not found in the memory pool, the system will attempt to read the file data from DFS and return it to the user. -#### 3FS USRBIO Plugin (Experimental) +### 3FS USRBIO Plugin (Experimental) ```{note} This integration is **experimental** and incomplete; see the plugin page for details before relying on it. @@ -828,15 +663,15 @@ This integration is **experimental** and incomplete; see the plugin page for det If you need to use 3FS's native API (USRBIO) to achieve high-performance persistent file reads and writes, you can refer to the configuration instructions in this document [3FS USRBIO Plugin](../getting_started/plugin-usage/3FS-USRBIO-Plugin.md). -### Builtin Metadata Server +## Builtin Metadata Server Mooncake Store provides a built-in HTTP metadata server as an alternative to etcd for storing cluster metadata. This feature is particularly useful for development environments or scenarios where etcd is not available. -#### Configuration Parameters +### Configuration Parameters The HTTP metadata server can be configured using the following parameters: - **`enable_http_metadata_server`** (boolean, default: `false`): Enables the built-in HTTP metadata server instead of using etcd. When set to `true`, the master service will start an embedded HTTP server that handles metadata operations. - **`http_metadata_server_port`** (integer, default: `8080`): Specifies the TCP port on which the HTTP metadata server will listen for incoming connections. This port must be available and not conflict with other services. - **`http_metadata_server_host`** (string, default: `"0.0.0.0"`): Specifies the host address for the HTTP metadata server to bind to. Use `"0.0.0.0"` to listen on all available network interfaces, or specify a specific IP address for security purposes. -#### Environment Variables -- MC_STORE_CLUSTER_ID: Identify the metadata when multiple cluster share the same master, default 'mooncake_cluster'. +### Environment Variables +- MC_STORE_CLUSTER_ID: Identify the metadata when multiple cluster share the same master, default 'mooncake'. - MC_STORE_MEMCPY: Enables or disables local memcpy optimization, set to 1/true to enable, 0/false to disable. - MC_STORE_CLIENT_METRIC: Enables client metric reporting, enabled by default; set to 0/false to disable. - MC_STORE_CLIENT_METRIC_INTERVAL: Reporting interval in seconds, default 0 (collects but does not report). @@ -846,7 +681,7 @@ The HTTP metadata server can be configured using the following parameters: - MC_STORE_HUGEPAGE_SIZE: Specifies the page size of the huge page to use, default 2M. - MC_MMAP_ARENA_POOL_SIZE: Size of the pre-allocated arena pool for mmap buffer allocations. Accepts human-readable sizes (e.g., `"8gb"`, `"20gb"`). Providing this variable explicitly enables the arena; when enabled via gflag without an env override, the default pool size is `8gb`. The arena is allocated once at first use and serves subsequent allocations via lock-free atomic bump pointer (~50ns per allocation vs ~1000ns for direct mmap). - MC_DISABLE_MMAP_ARENA: Set to `1` to disable the arena allocator and fall back to per-call `mmap()`, even if the arena was explicitly requested. Also accepts `true`, `yes`, or `on`. This must be set before the first Mooncake mmap-buffer allocation in the process. Useful for debugging or memory-constrained environments where pre-allocating a pool is not desirable. -#### Usage Example +### Usage Example To start the master service with the HTTP metadata server enabled: ```bash ./build/mooncake-store/src/mooncake_master \ @@ -857,6 +692,8 @@ To start the master service with the HTTP metadata server enabled: When enabled, the HTTP metadata server will start automatically and provide metadata services for the Mooncake Store cluster. This eliminates the need for an external etcd deployment, simplifying the setup process for development and testing environments. Note that the HTTP metadata server is designed for single-node deployments and does not provide the high availability features that etcd offers. For production environments requiring high availability, etcd is still the recommended choice. +For detailed guidance on monitoring master metrics, Prometheus endpoints, and health checks, see the [Observability guide](../getting_started/observability.md). + ## Mooncake Store Python API **Complete Python API Documentation**: [https://kvcache-ai.github.io/Mooncake/python-api-reference/mooncake-store.html](https://kvcache-ai.github.io/Mooncake/python-api-reference/mooncake-store.html) diff --git a/docs/source/design/tent/overview.md b/docs/source/design/tent/overview.md index 13dc295e..32887435 100644 --- a/docs/source/design/tent/overview.md +++ b/docs/source/design/tent/overview.md @@ -75,11 +75,7 @@ The design favors predictable behavior and operational simplicity over manual tu ## TENT C++ API Reference -:::{toctree} -:maxdepth: 1 - -cpp-api -::: +[TENT C++ API Reference](cpp-api.md) ## TENT Metrics System diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index f6bff1e1..5e837535 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -314,11 +314,7 @@ For advanced users, TransferEngine provides the following advanced runtime optio ## C++ API Reference -::::{toctree} -:maxdepth: 1 - -cpp-api -:::: +For the complete C++ API reference, see [Transfer Engine C++ API](../../api-reference/cpp/index). ## EFA Transport (AWS) @@ -347,6 +343,14 @@ kunpeng_ub_transport sunrise_link_transport ::: +## Supported Protocols + +:::{toctree} +:maxdepth: 1 + +../../getting_started/supported-protocols +::: + ## Benchmark and Tuning Guide :::{toctree} diff --git a/docs/source/getting_started/examples/lmcache-integration.md b/docs/source/getting_started/examples/lmcache-integration.md index b15da526..f5ac2948 100644 --- a/docs/source/getting_started/examples/lmcache-integration.md +++ b/docs/source/getting_started/examples/lmcache-integration.md @@ -23,9 +23,12 @@ This strategic alliance delivers substantial mutual technological advancements: For a complete deployment guide with step-by-step instructions, see: -👉 **[vLLM V1 Disaggregated Serving with Mooncake Store and LMCache](vllm-integration/vllmv1-lmcache-integration.md)** +:::{toctree} +:maxdepth: 1 -👉 **[vLLM V1 Disaggregated Serving with Mooncake Store and LMCache \[MP\]](vllm-integration/vllmv1-lmcache-mp-integration.md)** +../examples/vllm-integration/vllmv1-lmcache-integration +../examples/vllm-integration/vllmv1-lmcache-mp-integration +::: ## Performance Benchmarking and Results diff --git a/docs/source/getting_started/examples/vllm-integration/index.md b/docs/source/getting_started/examples/vllm-integration/index.md index b2a49474..56e941d7 100644 --- a/docs/source/getting_started/examples/vllm-integration/index.md +++ b/docs/source/getting_started/examples/vllm-integration/index.md @@ -41,17 +41,6 @@ Distributed KV cache storage via `MooncakeStore` / `MooncakeStoreConnector` for kv-cache-storage :::: -### LMCache-Based Disaggregated Serving - -Disaggregated prefill-decode using LMCache with Mooncake Store as the remote storage backend. Supports both non-MP (`LMCacheConnectorV1`) and MP (`LMCacheMPConnector`) paths. - -::::{toctree} -:maxdepth: 1 - -vllmv1-lmcache-integration -vllmv1-lmcache-mp-integration -:::: - --- ## Archived Documentation diff --git a/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md b/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md index 8cc122f3..d76f5646 100644 --- a/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md +++ b/docs/source/getting_started/examples/vllm-integration/vllm-mooncakestoreconnector.md @@ -116,7 +116,7 @@ vllm serve meta-llama/Llama-3.1-8B-Instruct \ }' ``` -Proxy: +Proxy: ```shell python examples/disaggregated/disaggregated_serving/mooncake_connector/mooncake_connector_proxy.py \ diff --git a/docs/source/index.md b/docs/source/index.md index 19ff330e..e5337241 100644 --- a/docs/source/index.md +++ b/docs/source/index.md @@ -69,12 +69,12 @@ getting_started/build getting_started/quick-start getting_started/supported-protocols getting_started/observability -getting_started/plugin-usage/3FS-USRBIO-Plugin -getting_started/examples/lmcache-integration -getting_started/examples/lmdeploy-integration-v0.9 getting_started/examples/sglang-integration-v1 getting_started/examples/sglang-integration/index getting_started/examples/vllm-integration/index +Mooncake x LMCache Integration +getting_started/examples/lmdeploy-integration-v0.9 +getting_started/plugin-usage/3FS-USRBIO-Plugin ::: % Making the most out of Mooncake @@ -95,18 +95,6 @@ performance/ssd-offload-benchmark-results performance/storage-benchmark ::: -% API Documentation - -:::{toctree} -:caption: Python API Reference -:maxdepth: 1 - -python-api-reference/mooncake-store -python-api-reference/transfer-engine -http-api-reference/http-service -python-api-reference/ep-backend -::: - % Explanation of Mooncake internals :::{toctree} @@ -126,6 +114,17 @@ design/conductor/conductor-architecture-design design/conductor/indexer-api-design ::: +% API Documentation + +:::{toctree} +:caption: API Reference +:maxdepth: 2 + +api-reference/python/index +api-reference/cpp/index +api-reference/http/index +::: + % Q&A for Mooncake :::{toctree} @@ -164,5 +163,4 @@ getting_started/examples/vllm-integration/vllm-mooncakestoreconnector getting_started/examples/vllm-integration/vllm-integration-v0.2 getting_started/examples/vllm-integration/vllm-integration-v0.3 getting_started/examples/vllm-integration/vllm-integration-v1.0 -getting_started/examples/vllm-integration/vllmv1-lmcache-integration ::: -- 2.34.1 From de19e8fbdf408d1aaf4d4d2a715538ce6658c835 Mon Sep 17 00:00:00 2001 From: ertcmm <104262261+ertcmm@users.noreply.github.com> Date: Wed, 3 Jun 2026 18:21:49 +0800 Subject: [PATCH 173/382] [Store] Support SSD offload configuration in standalone store service (#2261) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit and Python setup Co-authored-by: 张翔云 --- mooncake-wheel/mooncake/mooncake_config.py | 16 ++++++++++++++-- .../mooncake/mooncake_store_service.py | 5 ++++- mooncake-wheel/tests/test_mooncake_config.py | 14 +++++++++++++- 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/mooncake-wheel/mooncake/mooncake_config.py b/mooncake-wheel/mooncake/mooncake_config.py index b4920d0f..db05a82f 100644 --- a/mooncake-wheel/mooncake/mooncake_config.py +++ b/mooncake-wheel/mooncake/mooncake_config.py @@ -85,6 +85,8 @@ class MooncakeConfig: (e.g., "mlx5_0", "erdma_0", or "auto-discovery"). Required when protocol is "rdma", optional for other protocols. master_server_address (str): The address of the master server. + enable_ssd_offload (bool): Enable SSD offload. Default is False. + ssd_offload_path (str): The path to the SSD directory for offloading. Example of configuration file: { @@ -94,7 +96,9 @@ class MooncakeConfig: "local_buffer_size": 1073741824, "protocol": "tcp", "device_name": "", - "master_server_address": "localhost:8081" + "master_server_address": "localhost:8081", + "enable_ssd_offload": true, + "ssd_offload_path": "/nvme/mooncake_offload" } For RDMA: @@ -105,7 +109,9 @@ class MooncakeConfig: "local_buffer_size": 1073741824, "protocol": "rdma", "device_name": "mlx5_0", - "master_server_address": "master:8081" + "master_server_address": "master:8081", + "enable_ssd_offload": true, + "ssd_offload_path": "/nvme/mooncake_offload" } """ local_hostname: str @@ -115,6 +121,8 @@ class MooncakeConfig: protocol: str device_name: Optional[str] master_server_address: str + enable_ssd_offload: bool = False + ssd_offload_path: str = "" @staticmethod def from_file(file_path: str) -> 'MooncakeConfig': @@ -141,6 +149,8 @@ class MooncakeConfig: protocol=config.get("protocol", "tcp"), device_name=config.get("device_name", ""), master_server_address=config.get("master_server_address"), + enable_ssd_offload=bool(config.get("enable_ssd_offload", False)), + ssd_offload_path=str(config.get("ssd_offload_path", "")), ) @staticmethod @@ -167,5 +177,7 @@ class MooncakeConfig: protocol=os.getenv("MOONCAKE_PROTOCOL", "tcp"), device_name=os.getenv("MOONCAKE_DEVICE", ""), master_server_address=os.getenv("MOONCAKE_MASTER"), + enable_ssd_offload=os.getenv("MOONCAKE_OFFLOAD_ENABLED", "false").lower() in ("true", "1"), + ssd_offload_path=os.getenv("MOONCAKE_OFFLOAD_FILE_STORAGE_PATH", ""), ) return MooncakeConfig.from_file(config_file_path) \ No newline at end of file diff --git a/mooncake-wheel/mooncake/mooncake_store_service.py b/mooncake-wheel/mooncake/mooncake_store_service.py index ca5ddcbf..fd4c1699 100644 --- a/mooncake-wheel/mooncake/mooncake_store_service.py +++ b/mooncake-wheel/mooncake/mooncake_store_service.py @@ -128,7 +128,10 @@ class MooncakeStoreService: self.config.local_buffer_size, self.config.protocol, self.config.device_name, - self.config.master_server_address + self.config.master_server_address, + None, + self.config.enable_ssd_offload, + self.config.ssd_offload_path ) if ret != 0: diff --git a/mooncake-wheel/tests/test_mooncake_config.py b/mooncake-wheel/tests/test_mooncake_config.py index 497da0c8..403bc326 100644 --- a/mooncake-wheel/tests/test_mooncake_config.py +++ b/mooncake-wheel/tests/test_mooncake_config.py @@ -19,7 +19,9 @@ class TestMooncakeConfig(unittest.TestCase): "global_segment_size": 3355443200, "local_buffer_size": 1073741824, "protocol": "tcp", - "device_name": "eth0" + "device_name": "eth0", + "enable_ssd_offload": True, + "ssd_offload_path": "/nvme/mooncake_offload" } def tearDown(self): @@ -42,6 +44,8 @@ class TestMooncakeConfig(unittest.TestCase): self.assertEqual(config.local_buffer_size, 1073741824) self.assertEqual(config.protocol, "tcp") self.assertEqual(config.device_name, "eth0") + self.assertEqual(config.enable_ssd_offload, True) + self.assertEqual(config.ssd_offload_path, "/nvme/mooncake_offload") def test_load_with_default_values(self): """Test loading configuration with default values""" @@ -57,6 +61,8 @@ class TestMooncakeConfig(unittest.TestCase): self.assertEqual(config.local_buffer_size, DEFAULT_LOCAL_BUFFER_SIZE) self.assertEqual(config.protocol, "tcp") self.assertEqual(config.device_name, "") + self.assertEqual(config.enable_ssd_offload, False) + self.assertEqual(config.ssd_offload_path, "") def test_missing_required_field(self): """Test missing required field""" @@ -93,6 +99,8 @@ class TestMooncakeConfig(unittest.TestCase): os.environ['MOONCAKE_GLOBAL_SEGMENT_SIZE'] = str(self.valid_config["global_segment_size"]) os.environ['MOONCAKE_PROTOCOL'] = self.valid_config["protocol"] os.environ['MOONCAKE_DEVICE'] = self.valid_config["device_name"] + os.environ['MOONCAKE_OFFLOAD_ENABLED'] = str(self.valid_config["enable_ssd_offload"]) + os.environ['MOONCAKE_OFFLOAD_FILE_STORAGE_PATH'] = self.valid_config["ssd_offload_path"] try: config = MooncakeConfig.load_from_env() @@ -102,6 +110,8 @@ class TestMooncakeConfig(unittest.TestCase): self.assertEqual(config.global_segment_size, self.valid_config["global_segment_size"]) self.assertEqual(config.protocol, self.valid_config["protocol"]) self.assertEqual(config.device_name, self.valid_config["device_name"]) + self.assertEqual(config.enable_ssd_offload, self.valid_config["enable_ssd_offload"]) + self.assertEqual(config.ssd_offload_path, self.valid_config["ssd_offload_path"]) finally: # Clean up environment variable @@ -111,6 +121,8 @@ class TestMooncakeConfig(unittest.TestCase): del os.environ['MOONCAKE_GLOBAL_SEGMENT_SIZE'] del os.environ['MOONCAKE_PROTOCOL'] del os.environ['MOONCAKE_DEVICE'] + del os.environ['MOONCAKE_OFFLOAD_ENABLED'] + del os.environ['MOONCAKE_OFFLOAD_FILE_STORAGE_PATH'] def test_load_from_env_missing(self): """Test loading configuration from environment variable when not set""" -- 2.34.1 From 8db42ca973ccb45f7818bfcc2925157e7a74d969 Mon Sep 17 00:00:00 2001 From: XiaoTian <47113549+gogongxt@users.noreply.github.com> Date: Wed, 3 Jun 2026 20:26:20 +0800 Subject: [PATCH 174/382] [TransferEngine] Make TCP transport slice size configurable via MC_TCP_SLICE_SIZE (#2308) The TCP transport previously hardcoded a 64KB (65536 bytes) slice size for splitting large transfers into socket read/write operations. This commit makes it configurable via the MC_TCP_SLICE_SIZE environment variable, consistent with the RDMA transport's MC_SLICE_SIZE naming. Usage: export MC_TCP_SLICE_SIZE=1048576 # 1MB slices Default: 65536 (64KB, unchanged) --- docs/source/design/transfer-engine/index.md | 1 + .../transport/tcp_transport/tcp_transport.cpp | 21 ++++++++++++++----- 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index 5e837535..37eb37cc 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -311,6 +311,7 @@ For advanced users, TransferEngine provides the following advanced runtime optio - `WITH_NVIDIA_PEERMEM` When set to `1`, `ON`, or `TRUE`, Mooncake uses `ibv_reg_mr()` directly for GPU memory registration (requires the `nvidia-peermem` kernel module). By default (unset or `0`), Mooncake uses the DMA-BUF path which does not require `nvidia-peermem`. - `MC_ENDPOINT_STORE_TYPE` Choose FIFO Endpoint Store (`FIFO`) or Sieve Endpoint Store (`SIEVE`), default is `SIEVE`. - `MC_TCP_ENABLE_CONNECTION_POOL` Enable TCP Connection Pool to avoid excessive sockets. +- `MC_TCP_SLICE_SIZE` The segmentation granularity (in bytes) of TCP transport for splitting large transfers into socket read/write operations. Corresponds to `MC_SLICE_SIZE` for RDMA. Default value 65536 (64KB). ## C++ API Reference diff --git a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp index e7472608..9df163de 100644 --- a/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp +++ b/mooncake-transfer-engine/src/transport/tcp_transport/tcp_transport.cpp @@ -23,6 +23,7 @@ #include #include #include +#include #include #include @@ -36,7 +37,17 @@ namespace mooncake { using tcpsocket = asio::ip::tcp::socket; -const static size_t kDefaultBufferSize = 65536; +static size_t getChunkSize() { + static const size_t val = [] { + const char* env = std::getenv("MC_TCP_SLICE_SIZE"); + if (env) { + size_t v = std::stoull(env); + if (v > 0) return v; + } + return size_t(65536); // 64KB default + }(); + return val; +} struct SessionHeader { uint64_t size; @@ -110,7 +121,7 @@ struct ServerSession : public std::enable_shared_from_this { char* addr = local_buffer_; size_t buffer_size = - std::min(kDefaultBufferSize, size - total_transferred_bytes_); + std::min(getChunkSize(), size - total_transferred_bytes_); if (buffer_size == 0) { session_mutex_.unlock(); // Transfer complete, wait for next request on this connection @@ -171,7 +182,7 @@ struct ServerSession : public std::enable_shared_from_this { char* addr = local_buffer_; size_t buffer_size = - std::min(kDefaultBufferSize, size - total_transferred_bytes_); + std::min(getChunkSize(), size - total_transferred_bytes_); if (buffer_size == 0) { session_mutex_.unlock(); // Transfer complete, wait for next request on this connection @@ -302,7 +313,7 @@ struct ClientSession : public std::enable_shared_from_this { char* addr = local_buffer_; size_t buffer_size = - std::min(kDefaultBufferSize, size - total_transferred_bytes_); + std::min(getChunkSize(), size - total_transferred_bytes_); if (buffer_size == 0) { asio::post(socket_->get_executor(), [this, self, on_finalize = std::move(on_finalize_), @@ -399,7 +410,7 @@ struct ClientSession : public std::enable_shared_from_this { char* addr = local_buffer_; size_t buffer_size = - std::min(kDefaultBufferSize, size - total_transferred_bytes_); + std::min(getChunkSize(), size - total_transferred_bytes_); if (buffer_size == 0) { // Post cleanup to ensure it runs after callback returns asio::post(socket_->get_executor(), -- 2.34.1 From 4d7c1a19b021ed1fd2c95a4813ab5f57bb188fd9 Mon Sep 17 00:00:00 2001 From: Vincent Date: Wed, 3 Jun 2026 20:52:24 +0800 Subject: [PATCH 175/382] [TE] fix: improve auto gid selection and retry (#2269) --- docs/source/design/transfer-engine/index.md | 1 + docs/source/zh_archive/transfer-engine.md | 1 + mooncake-transfer-engine/include/config.h | 1 + .../transport/rdma_transport/rdma_context.h | 24 +- .../transport/rdma_transport/rdma_endpoint.h | 20 +- .../transport/rdma_transport/rdma_gid_probe.h | 247 ++++++++++ .../transport/rdma_transport/rdma_transport.h | 8 +- mooncake-transfer-engine/src/config.cpp | 12 + .../transport/rdma_transport/rdma_context.cpp | 285 ++++++++--- .../rdma_transport/rdma_endpoint.cpp | 402 +++++++++++----- .../rdma_transport/rdma_transport.cpp | 30 +- .../src/transport/rdma/rdma_transport.cpp | 43 +- mooncake-transfer-engine/tests/CMakeLists.txt | 21 +- .../tests/config_test.cpp | 28 +- .../tests/rdma_context_reprobe_test.cpp | 144 ++++++ .../tests/rdma_endpoint_reestablish_test.cpp | 452 +++++++++++++----- .../tests/rdma_gid_probe_test.cpp | 402 ++++++++++++++++ 17 files changed, 1811 insertions(+), 310 deletions(-) create mode 100644 mooncake-transfer-engine/include/transport/rdma_transport/rdma_gid_probe.h create mode 100644 mooncake-transfer-engine/tests/rdma_context_reprobe_test.cpp create mode 100644 mooncake-transfer-engine/tests/rdma_gid_probe_test.cpp diff --git a/docs/source/design/transfer-engine/index.md b/docs/source/design/transfer-engine/index.md index 37eb37cc..e8d0ec27 100644 --- a/docs/source/design/transfer-engine/index.md +++ b/docs/source/design/transfer-engine/index.md @@ -291,6 +291,7 @@ For advanced users, TransferEngine provides the following advanced runtime optio - `MC_WORKERS_PER_CTX` The number of asynchronous worker threads corresponding to each device instance - `MC_SLICE_SIZE` The segmentation granularity of user requests in Transfer Engine - `MC_RETRY_CNT` The maximum number of retries in Transfer Engine +- `MC_AUTO_GID_MAX_RETRIES` The maximum number of automatic local GID reprobe retries during classic RDMA handshake recovery. Default value 2. Set to 0 to disable automatic GID retry. - `MC_LOG_LEVEL` This option can be set as `TRACE`/`INFO`/`WARNING`/`ERROR` (see [glog doc](https://github.com/google/glog/blob/master/docs/logging.md)), and more detailed logs will be output during runtime - `MC_DISABLE_METACACHE` Disable local meta cache to prevent transfer failure due to dynamic memory registrations, which may downgrades the performance - `MC_HANDSHAKE_LISTEN_BACKLOG` The backlog size of socket listening for handshaking, default value is 128 diff --git a/docs/source/zh_archive/transfer-engine.md b/docs/source/zh_archive/transfer-engine.md index 73d04aaf..a0bb0492 100644 --- a/docs/source/zh_archive/transfer-engine.md +++ b/docs/source/zh_archive/transfer-engine.md @@ -407,6 +407,7 @@ int init(const std::string &metadata_conn_string, - `MC_WORKERS_PER_CTX` 每个设备实例对应的异步工作线程数量 - `MC_SLICE_SIZE` Transfer Engine 中用户请求的切分粒度 - `MC_RETRY_CNT` Transfer Engine 中最大重试次数 +- `MC_AUTO_GID_MAX_RETRIES` classic RDMA 握手恢复过程中自动重探测本地 GID 的最大重试次数,默认值 2。设置为 0 可以关闭自动 GID 重试。 - `MC_LOG_LEVEL` 该选项可以设置成`TRACE`/`INFO`/`WARNING`/`ERROR`(详情见 [glog doc](https://github.com/google/glog/blob/master/docs/logging.md)),则在运行时会输出更详细的日志 - `MC_HANDSHAKE_LISTEN_BACKLOG` 监听握手连接的 backlog 大小, 默认值 128 - `MC_HANDSHAKE_MAX_LENGTH` P2P 模式下握手消息的最大长度(字节)。有效范围:1MB 到 128MB。默认值为 1MB (1048576 字节)。当单个 RDMA 实例注册大量内存缓冲区(>10,000)时,需要增大此值以避免握手失败。示例:设置为 10485760 表示 10MB diff --git a/mooncake-transfer-engine/include/config.h b/mooncake-transfer-engine/include/config.h index 32844b61..8cd09515 100644 --- a/mooncake-transfer-engine/include/config.h +++ b/mooncake-transfer-engine/include/config.h @@ -50,6 +50,7 @@ struct GlobalConfig { int workers_per_ctx = 2; size_t slice_size = 65536; int retry_cnt = 9; + int auto_gid_max_retries = 2; int handshake_listen_backlog = 128; bool metacache = true; int log_level = google::INFO; diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h index 3818e1b4..4fa4088a 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_context.h @@ -29,11 +29,13 @@ #include #include #include +#include #include #include #include #include "common.h" +#include "rdma_gid_probe.h" #include "rdma_transport.h" #include "transport/transport.h" @@ -41,6 +43,7 @@ namespace mooncake { class RdmaEndPoint; class RdmaTransport; +class RdmaContextTestPeer; class WorkerPool; class EndpointStore; @@ -51,6 +54,11 @@ enum class GidNetworkState { GID_NOT_FOUND = 2 // No suitable GID found }; +struct GidSelectionSnapshot { + std::string gid; + int gid_index = -1; +}; + struct RdmaCq { RdmaCq() : native(nullptr), outstanding(0) {} ibv_cq *native; @@ -68,6 +76,8 @@ struct MemoryRegionMeta { // including Memory Region, CQ, EndPoint (QPs), etc. class RdmaContext { public: + friend class RdmaContextTestPeer; + RdmaContext(RdmaTransport &engine, const std::string &device_name); ~RdmaContext(); @@ -148,7 +158,16 @@ class RdmaContext { std::string gid() const; - int gidIndex() const { return gid_index_; } + GidSelectionSnapshot gidSelection() const; + + int gidIndex() const; + + bool autoGidSelectionEnabled() const { return auto_gid_selection_enabled_; } + + bool reprobeAutoGid( + const GidSelectionSnapshot &expected_selection, + const std::vector &tried_selections = {}, + std::string *previous_gid = nullptr, std::string *next_gid = nullptr); ibv_context *context() const { return context_; } @@ -215,6 +234,9 @@ class RdmaContext { ibv_mtu active_mtu_; uint8_t num_lag_ports_ = 0; // 0/1 = not in LAG; ≥2 = LAG active ibv_gid gid_; + mutable std::mutex gid_lock_; + mutable std::mutex gid_reprobe_lock_; + bool auto_gid_selection_enabled_ = false; RWSpinlock memory_regions_lock_; MemoryRegionMap memory_region_map_; diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h index 089f2c18..d1ae09df 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_endpoint.h @@ -148,15 +148,31 @@ class RdmaEndPoint { size_t getQPNumber() const; private: + enum class SetupConnectionFailureStage { + kNone, + kPeerValidation, + kReset, + kInit, + kRtr, + kRts, + }; + + struct SetupConnectionFailureInfo { + SetupConnectionFailureStage stage = SetupConnectionFailureStage::kNone; + int sys_errno = 0; + }; + std::vector qpNum() const; int doSetupConnection(const std::string &peer_gid, uint16_t peer_lid, std::vector peer_qp_num_list, - std::string *reply_msg = nullptr); + std::string *reply_msg = nullptr, + SetupConnectionFailureInfo *failure_info = nullptr); int doSetupConnection(int qp_index, const ibv_gid &peer_gid, uint16_t peer_lid, uint32_t peer_qp_num, - std::string *reply_msg = nullptr); + int local_gid_index, std::string *reply_msg = nullptr, + SetupConnectionFailureInfo *failure_info = nullptr); private: static constexpr uint64_t kWaitExistingHandshakeTimeoutNano = diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_gid_probe.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_gid_probe.h new file mode 100644 index 00000000..a4ee60db --- /dev/null +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_gid_probe.h @@ -0,0 +1,247 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#ifndef RDMA_GID_PROBE_H +#define RDMA_GID_PROBE_H + +#include +#include + +#include +#include +#include +#include +#include + +namespace mooncake { + +enum class AutoGidCandidateClass { + kNetworkRoutable = 0, + kNoNetworkRoutable = 1, + kNetworkDegraded = 2, + kNoNetworkDegraded = 3, + kFallbackNonzero = 4, +}; + +enum class AutoGidRetryAction { + kDoNotRetry = 0, + kRetryWithReprobedGid = 1, + kRetryWithObservedChange = 2, +}; + +struct AutoGidCandidate { + int gid_index = -1; + std::string gid; + uint32_t gid_type = 0; + bool has_network_device = false; + bool is_ipv4_mapped = false; + bool is_link_local_ipv6 = false; + bool is_overlay_network = false; + bool is_overlay_ipv4 = false; + bool is_null_gid = false; + bool query_succeeded = true; +}; + +struct AutoGidSelection { + int gid_index = -1; + std::string gid; + AutoGidCandidateClass candidate_class = + AutoGidCandidateClass::kFallbackNonzero; +}; + +struct AutoGidSelectionIdentity { + int gid_index = -1; + std::string gid; +}; + +inline const char* autoGidCandidateClassToString( + AutoGidCandidateClass candidate_class) { + switch (candidate_class) { + case AutoGidCandidateClass::kNetworkRoutable: + return "network-routable"; + case AutoGidCandidateClass::kNoNetworkRoutable: + return "no-network-routable"; + case AutoGidCandidateClass::kNetworkDegraded: + return "network-degraded"; + case AutoGidCandidateClass::kNoNetworkDegraded: + return "no-network-degraded"; + case AutoGidCandidateClass::kFallbackNonzero: + return "fallback-nonzero"; + } + return "unknown"; +} + +inline std::optional classifyAutoGidCandidate( + const AutoGidCandidate& candidate) { + if (!candidate.query_succeeded || candidate.gid_index < 0 || + candidate.is_null_gid) { + return std::nullopt; + } + + if (candidate.gid_type != IBV_GID_TYPE_ROCE_V2 && + candidate.gid_type != IBV_GID_TYPE_IB) { + return std::nullopt; + } + + const bool is_roce_v2 = candidate.gid_type == IBV_GID_TYPE_ROCE_V2; + const bool is_overlay = + is_roce_v2 && (candidate.is_overlay_network || + (candidate.is_ipv4_mapped && candidate.is_overlay_ipv4)); + const bool is_link_local = + is_roce_v2 && !candidate.is_ipv4_mapped && candidate.is_link_local_ipv6; + const bool is_degraded = is_overlay || is_link_local; + + if (candidate.has_network_device) { + return is_degraded ? AutoGidCandidateClass::kNetworkDegraded + : AutoGidCandidateClass::kNetworkRoutable; + } + + return is_degraded ? AutoGidCandidateClass::kNoNetworkDegraded + : AutoGidCandidateClass::kNoNetworkRoutable; +} + +inline int autoGidCandidateClassPriority( + AutoGidCandidateClass candidate_class) { + switch (candidate_class) { + case AutoGidCandidateClass::kNetworkRoutable: + return 0; + case AutoGidCandidateClass::kNoNetworkRoutable: + return 1; + case AutoGidCandidateClass::kNetworkDegraded: + return 2; + case AutoGidCandidateClass::kNoNetworkDegraded: + return 3; + case AutoGidCandidateClass::kFallbackNonzero: + return 4; + } + return 5; +} + +inline std::vector rankAutoGidCandidates( + const std::vector& candidates) { + std::vector ranked; + int first_query_success_fallback = -1; + std::string first_query_success_fallback_gid; + + for (const auto& candidate : candidates) { + auto candidate_class = classifyAutoGidCandidate(candidate); + if (candidate_class.has_value()) { + ranked.push_back( + {candidate.gid_index, candidate.gid, *candidate_class}); + continue; + } + + if (candidate.query_succeeded && !candidate.is_null_gid && + first_query_success_fallback < 0) { + first_query_success_fallback = candidate.gid_index; + first_query_success_fallback_gid = candidate.gid; + } + } + + std::stable_sort( + ranked.begin(), ranked.end(), + [](const AutoGidSelection& lhs, const AutoGidSelection& rhs) { + int lhs_priority = + autoGidCandidateClassPriority(lhs.candidate_class); + int rhs_priority = + autoGidCandidateClassPriority(rhs.candidate_class); + if (lhs_priority != rhs_priority) { + return lhs_priority < rhs_priority; + } + return lhs.gid_index < rhs.gid_index; + }); + + if (first_query_success_fallback >= 0) { + ranked.push_back({first_query_success_fallback, + first_query_success_fallback_gid, + AutoGidCandidateClass::kFallbackNonzero}); + } + + return ranked; +} + +inline std::optional selectBestAutoGidCandidate( + const std::vector& candidates) { + auto ranked = rankAutoGidCandidates(candidates); + if (ranked.empty()) { + return std::nullopt; + } + return ranked.front(); +} + +inline bool shouldAttemptAutoGidHandshakeRetry(bool auto_gid_selection_enabled, + int retry_count, int max_retries, + bool failure_happened_at_rtr, + int sys_errno) { + return auto_gid_selection_enabled && retry_count < max_retries && + failure_happened_at_rtr && sys_errno == EINVAL; +} + +inline bool didAutoGidSelectionChange(int previous_gid_index, + std::string_view previous_gid, + int current_gid_index, + std::string_view current_gid) { + return previous_gid_index != current_gid_index || + previous_gid != current_gid; +} + +inline bool matchesAutoGidSelection(const AutoGidSelectionIdentity& identity, + const AutoGidSelection& selection) { + return identity.gid_index == selection.gid_index && + identity.gid == selection.gid; +} + +inline bool hasTriedAutoGidSelection( + const std::vector& tried_selections, + const AutoGidSelection& selection) { + return std::any_of(tried_selections.begin(), tried_selections.end(), + [&](const AutoGidSelectionIdentity& identity) { + return matchesAutoGidSelection(identity, selection); + }); +} + +inline std::optional reselectAutoGidCandidate( + const std::vector& candidates, int current_gid_index, + std::string_view current_gid, + const std::vector& tried_selections = {}) { + auto ranked = rankAutoGidCandidates(candidates); + for (const auto& selection : ranked) { + if (!didAutoGidSelectionChange(current_gid_index, current_gid, + selection.gid_index, selection.gid)) { + continue; + } + if (hasTriedAutoGidSelection(tried_selections, selection)) { + continue; + } + return selection; + } + return std::nullopt; +} + +inline AutoGidRetryAction decideAutoGidRetryAction( + bool reprobe_changed, int previous_gid_index, std::string_view previous_gid, + int current_gid_index, std::string_view current_gid) { + if (reprobe_changed) { + return AutoGidRetryAction::kRetryWithReprobedGid; + } + if (didAutoGidSelectionChange(previous_gid_index, previous_gid, + current_gid_index, current_gid)) { + return AutoGidRetryAction::kRetryWithObservedChange; + } + return AutoGidRetryAction::kDoNotRetry; +} + +} // namespace mooncake + +#endif // RDMA_GID_PROBE_H diff --git a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h index 11997831..c999b89c 100644 --- a/mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h +++ b/mooncake-transfer-engine/include/transport/rdma_transport/rdma_transport.h @@ -36,11 +36,13 @@ namespace mooncake { class RdmaContext; class RdmaEndPoint; class TransferMetadata; +class RdmaTransportTestPeer; class WorkerPool; class RdmaTransport : public Transport { friend class RdmaContext; friend class RdmaEndPoint; + friend class RdmaTransportTestPeer; friend class WorkerPool; public: @@ -101,6 +103,9 @@ class RdmaTransport : public Transport { private: int allocateLocalSegmentID(); + int refreshLocalDeviceDesc(const std::string &device_name, uint16_t lid, + const std::string &gid); + int preTouchMemory(void *addr, size_t length); public: @@ -134,6 +139,7 @@ class RdmaTransport : public Transport { // "192.168.0.y:port") for NIC path construction, while // local_server_name_ keeps the TCP-reachable address for P2P routing. std::string rdma_server_name_; + std::mutex local_desc_lock_; }; using TransferRequest = Transport::TransferRequest; @@ -144,4 +150,4 @@ using BatchID = Transport::BatchID; } // namespace mooncake -#endif // RDMA_TRANSPORT_H_ \ No newline at end of file +#endif // RDMA_TRANSPORT_H_ diff --git a/mooncake-transfer-engine/src/config.cpp b/mooncake-transfer-engine/src/config.cpp index 1042fc9c..a7f84fc1 100644 --- a/mooncake-transfer-engine/src/config.cpp +++ b/mooncake-transfer-engine/src/config.cpp @@ -224,6 +224,18 @@ void loadGlobalConfig(GlobalConfig& config) { << "Ignore value from environment variable MC_RETRY_CNT"; } + const char* auto_gid_max_retries_env = + std::getenv("MC_AUTO_GID_MAX_RETRIES"); + if (auto_gid_max_retries_env) { + int val = atoi(auto_gid_max_retries_env); + if (val >= 0 && val <= 16) { + config.auto_gid_max_retries = val; + } else { + LOG(WARNING) << "Ignore value from environment variable " + "MC_AUTO_GID_MAX_RETRIES"; + } + } + const char* disable_metacache = std::getenv("MC_DISABLE_METACACHE"); if (disable_metacache) { config.metacache = false; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp index 08092ad6..46814010 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_context.cpp @@ -16,7 +16,9 @@ #include #include +#include #include +#include #include #include @@ -39,6 +41,7 @@ #include #endif #include "transport/rdma_transport/endpoint_store.h" +#include "transport/rdma_transport/rdma_gid_probe.h" #include "transport/rdma_transport/rdma_endpoint.h" #include "transport/rdma_transport/rdma_transport.h" #include "transport/rdma_transport/worker_pool.h" @@ -132,7 +135,7 @@ bool isKernelDmabufSupported() { LOG(WARNING) << "Kernel lacks CONFIG_PCI_P2PDMA / CONFIG_DMABUF_MOVE_NOTIFY " << "(p2pdma=" << found[0] << " move_notify=" << found[1] - << "); HIP dmabuf MR registration disabled, falling back to " + << "), HIP dmabuf MR registration disabled, falling back to " << "ibv_reg_mr() (which requires an amdgpu peermem driver). " << "Rebuild kernel with both options for GPU-direct RDMA."; } @@ -141,6 +144,17 @@ bool isKernelDmabufSupported() { return supported; } #endif // USE_HIP_DMABUF + +std::string gidBytesToString(const uint8_t *raw) { + std::string gid_str; + char buf[16] = {0}; + const static size_t kGidLength = 16; + for (size_t i = 0; i < kGidLength; ++i) { + snprintf(buf, sizeof(buf), "%02x", raw[i]); + gid_str += i == 0 ? buf : std::string(":") + buf; + } + return gid_str; +} } // namespace RdmaContext::RdmaContext(RdmaTransport &engine, const std::string &device_name) @@ -661,16 +675,16 @@ std::string RdmaContext::nicPath() const { return MakeNicPath(engine_.rdma_server_name_, device_name_); } -std::string RdmaContext::gid() const { - std::string gid_str; - char buf[16] = {0}; - const static size_t kGidLength = 16; - for (size_t i = 0; i < kGidLength; ++i) { - sprintf(buf, "%02x", gid_.raw[i]); - gid_str += i == 0 ? buf : std::string(":") + buf; - } +std::string RdmaContext::gid() const { return gidSelection().gid; } - return gid_str; +GidSelectionSnapshot RdmaContext::gidSelection() const { + std::lock_guard guard(gid_lock_); + return {gidBytesToString(gid_.raw), gid_index_}; +} + +int RdmaContext::gidIndex() const { + std::lock_guard guard(gid_lock_); + return gid_index_; } ibv_cq *RdmaContext::cq() { @@ -711,16 +725,53 @@ static std::string readGidNdev(const std::string &device_name, uint8_t port, return ndev; } -// Returns 1 if the GID has an associated network device, 0 otherwise. -static int hasNetworkDevice(const std::string &device_name, uint8_t port, - int gid_index) { - return !readGidNdev(device_name, port, gid_index).empty() ? 1 : 0; +static inline bool isOverlayNetwork(const std::string &ndev) { + return ndev.find("flannel") == 0 || ndev.find("cni") == 0 || + ndev.find("calico") == 0 || ndev.find("vxlan") == 0 || + ndev.find("docker") == 0 || ndev == "tunl0"; +} + +static inline bool isOverlayIPv4(const struct in6_addr *addr) { + if (!ipv6_addr_v4mapped(addr)) return false; + + uint32_t ipv4 = ntohl(addr->s6_addr32[3]); + uint8_t octet1 = (ipv4 >> 24) & 0xFF; + uint8_t octet2 = (ipv4 >> 16) & 0xFF; + + if (octet1 == 10) return true; + if (octet1 == 172 && octet2 >= 16 && octet2 <= 31) return true; + if (octet1 == 100 && octet2 >= 64 && octet2 <= 127) return true; + return false; +} + +static inline bool isLinkLocalIpv6(const struct in6_addr *addr) { + return IN6_IS_ADDR_LINKLOCAL(addr); } static const char *GidNetworkStateToString(GidNetworkState state) { - return (state == GidNetworkState::GID_WITH_NETWORK) - ? "with network device" - : "without network device"; + switch (state) { + case GidNetworkState::GID_WITH_NETWORK: + return "with network device"; + case GidNetworkState::GID_WITHOUT_NETWORK: + return "without network device"; + case GidNetworkState::GID_NOT_FOUND: + return "not found"; + } + return "unknown"; +} + +static GidNetworkState autoGidStateFromSelection( + const AutoGidSelection &selection) { + switch (selection.candidate_class) { + case AutoGidCandidateClass::kNetworkRoutable: + case AutoGidCandidateClass::kNetworkDegraded: + return GidNetworkState::GID_WITH_NETWORK; + case AutoGidCandidateClass::kNoNetworkRoutable: + case AutoGidCandidateClass::kNoNetworkDegraded: + case AutoGidCandidateClass::kFallbackNonzero: + return GidNetworkState::GID_WITHOUT_NETWORK; + } + return GidNetworkState::GID_NOT_FOUND; } GidNetworkState RdmaContext::findBestGidIndex(const std::string &device_name, @@ -728,67 +779,167 @@ GidNetworkState RdmaContext::findBestGidIndex(const std::string &device_name, ibv_port_attr &port_attr, uint8_t port, int &gid_index) { gid_index = -1; - int i; - struct ibv_gid_entry gid_entry; - int fallback_ipv4_gid_without_network = -1; - int fallback_ipv6_gid_with_network = -1; - int fallback_ipv6_gid_without_network = -1; - GidNetworkState state = GidNetworkState::GID_NOT_FOUND; + std::vector candidates; + candidates.reserve(port_attr.gid_tbl_len); - for (i = 0; i < port_attr.gid_tbl_len; i++) { + for (int i = 0; i < port_attr.gid_tbl_len; i++) { + AutoGidCandidate candidate; + candidate.gid_index = i; + + struct ibv_gid_entry gid_entry; if (ibv_query_gid_ex(context, port, i, &gid_entry, 0)) { - // Reached end of valid GID indices - break; - } - - if (gid_entry.gid_type != IBV_GID_TYPE_ROCE_V2 && - gid_entry.gid_type != IBV_GID_TYPE_IB) { + candidate.query_succeeded = false; + candidates.push_back(candidate); continue; } - const bool is_ipv4_gid = - gid_entry.gid_type == IBV_GID_TYPE_ROCE_V2 && - ipv6_addr_v4mapped((struct in6_addr *)gid_entry.gid.raw); - const bool has_network_device = hasNetworkDevice(device_name, port, i); + const auto *gid_addr = + reinterpret_cast(gid_entry.gid.raw); + std::string ndev = readGidNdev(device_name, port, i); + candidate.gid = gidBytesToString(gid_entry.gid.raw); + candidate.gid_type = gid_entry.gid_type; + candidate.has_network_device = !ndev.empty(); + candidate.is_ipv4_mapped = ipv6_addr_v4mapped(gid_addr); + candidate.is_link_local_ipv6 = isLinkLocalIpv6(gid_addr); + candidate.is_overlay_network = + candidate.has_network_device && isOverlayNetwork(ndev); + candidate.is_overlay_ipv4 = + candidate.is_ipv4_mapped && isOverlayIPv4(gid_addr); + candidate.is_null_gid = isNullGid(&gid_entry.gid); + candidates.push_back(candidate); + } - if (is_ipv4_gid) { - if (has_network_device) { - gid_index = i; - return GidNetworkState::GID_WITH_NETWORK; - } - if (fallback_ipv4_gid_without_network < 0) { - gid_index = i; - fallback_ipv4_gid_without_network = i; - state = GidNetworkState::GID_WITHOUT_NETWORK; - } + auto selection = selectBestAutoGidCandidate(candidates); + if (!selection.has_value()) { + return GidNetworkState::GID_NOT_FOUND; + } + + gid_index = selection->gid_index; + VLOG(1) << "Selected auto GID[" << gid_index << "] on " << device_name + << " with class " + << autoGidCandidateClassToString(selection->candidate_class); + return autoGidStateFromSelection(*selection); +} + +bool RdmaContext::reprobeAutoGid( + const GidSelectionSnapshot &expected_selection, + const std::vector &tried_selections, + std::string *previous_gid, std::string *next_gid) { + std::lock_guard reprobe_guard(gid_reprobe_lock_); + std::string current_gid_string; + std::string next_gid_string; + int current_gid_index = -1; + int next_gid_index = -1; + uint16_t current_lid = 0; + ibv_context *current_context = nullptr; + uint8_t current_port = 0; + AutoGidCandidateClass next_candidate_class = + AutoGidCandidateClass::kFallbackNonzero; + { + std::lock_guard guard(gid_lock_); + if (!auto_gid_selection_enabled_ || !context_) { + return false; + } + + current_gid_index = gid_index_; + current_gid_string = gidBytesToString(gid_.raw); + current_lid = lid_; + current_context = context_; + current_port = port_; + if (current_gid_index != expected_selection.gid_index || + current_gid_string != expected_selection.gid) { + return false; + } + } + + ibv_port_attr port_attr; + if (ibv_query_port(current_context, current_port, &port_attr)) { + PLOG(WARNING) << "Failed to reprobe port attributes on " << device_name_ + << "/" << static_cast(current_port); + return false; + } + + std::vector candidates; + candidates.reserve(port_attr.gid_tbl_len); + for (int i = 0; i < port_attr.gid_tbl_len; ++i) { + AutoGidCandidate candidate; + candidate.gid_index = i; + + struct ibv_gid_entry gid_entry; + if (ibv_query_gid_ex(current_context, current_port, i, &gid_entry, 0)) { + candidate.query_succeeded = false; + candidates.push_back(candidate); continue; } - if (has_network_device && fallback_ipv6_gid_with_network < 0) { - fallback_ipv6_gid_with_network = i; - } + const auto *gid_addr = + reinterpret_cast(gid_entry.gid.raw); + std::string ndev = readGidNdev(device_name_, current_port, i); + candidate.gid = gidBytesToString(gid_entry.gid.raw); + candidate.gid_type = gid_entry.gid_type; + candidate.has_network_device = !ndev.empty(); + candidate.is_ipv4_mapped = ipv6_addr_v4mapped(gid_addr); + candidate.is_link_local_ipv6 = isLinkLocalIpv6(gid_addr); + candidate.is_overlay_network = + candidate.has_network_device && isOverlayNetwork(ndev); + candidate.is_overlay_ipv4 = + candidate.is_ipv4_mapped && isOverlayIPv4(gid_addr); + candidate.is_null_gid = isNullGid(&gid_entry.gid); + candidates.push_back(candidate); + } - if (!has_network_device && fallback_ipv6_gid_without_network < 0) { - fallback_ipv6_gid_without_network = i; + auto selection = reselectAutoGidCandidate( + candidates, current_gid_index, current_gid_string, tried_selections); + if (!selection.has_value()) { + if (next_gid) { + *next_gid = current_gid_string; + } + return false; + } + + ibv_gid new_gid = {}; + if (ibv_query_gid(current_context, current_port, selection->gid_index, + &new_gid)) { + return false; + } + if (isNullGid(&new_gid)) { + return false; + } + + if (previous_gid) { + *previous_gid = current_gid_string; + } + next_gid_string = gidBytesToString(new_gid.raw); + + int publish_ret = engine_.refreshLocalDeviceDesc(device_name_, current_lid, + next_gid_string); + if (publish_ret) { + LOG(ERROR) << "Failed to refresh local device descriptor for " + << device_name_ << ": " << publish_ret; + return false; + } + + { + std::lock_guard guard(gid_lock_); + gid_ = new_gid; + gid_index_ = selection->gid_index; + next_gid_index = selection->gid_index; + next_candidate_class = selection->candidate_class; + if (next_gid) { + *next_gid = next_gid_string; } } - if (fallback_ipv4_gid_without_network >= 0) { - gid_index = fallback_ipv4_gid_without_network; - return GidNetworkState::GID_WITHOUT_NETWORK; + if (next_gid_string.empty()) { + return false; } - if (fallback_ipv6_gid_with_network >= 0) { - gid_index = fallback_ipv6_gid_with_network; - return GidNetworkState::GID_WITH_NETWORK; - } - - if (fallback_ipv6_gid_without_network >= 0) { - gid_index = fallback_ipv6_gid_without_network; - return GidNetworkState::GID_WITHOUT_NETWORK; - } - - return state; + LOG(WARNING) << "Auto GID reprobe switched " << device_name_ << "/" + << static_cast(port_) << " from index " + << current_gid_index << " (" << current_gid_string << ") to " + << next_gid_index << " (" << next_gid_string << "), class " + << autoGidCandidateClassToString(next_candidate_class); + return true; } int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, @@ -950,6 +1101,7 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, updateGlobalConfig(device_attr); GidNetworkState gid_state; + auto_gid_selection_enabled_ = gid_index < 0; if (gid_index < 0) { int found_gid_index = -1; gid_state = findBestGidIndex(device_name, context, port_attr, port, @@ -967,7 +1119,7 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, } } else { // Also check network state for user-specified GID - bool has_ndev = hasNetworkDevice(device_name, port, gid_index); + bool has_ndev = !readGidNdev(device_name, port, gid_index).empty(); if (!has_ndev) { LOG(WARNING) << "User-specified GID index " << gid_index << " on " << device_name << "/" << port @@ -1001,7 +1153,10 @@ int RdmaContext::openRdmaDevice(const std::string &device_name, uint8_t port, lid_ = attr.lid; active_mtu_ = attr.active_mtu; active_speed_ = attr.active_speed; - gid_index_ = gid_index; + { + std::lock_guard guard(gid_lock_); + gid_index_ = gid_index; + } ibv_free_device_list(devices); return 0; diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp index 34271216..c1ee94d4 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_endpoint.cpp @@ -17,6 +17,7 @@ #include #include +#include #include #include #include @@ -28,11 +29,40 @@ #include "common.h" #include "config.h" +#include "transport/rdma_transport/rdma_gid_probe.h" namespace mooncake { -const static uint8_t MAX_HOP_LIMIT = 16; -const static uint8_t TIMEOUT = 14; -const static uint8_t RETRY_CNT = 7; +constexpr uint8_t kMaxHopLimit = 16; +constexpr uint8_t kTimeout = 14; +constexpr uint8_t kRetryCount = 7; + +static GidSelectionSnapshot fillLocalHandshakeDesc( + RdmaContext &context, const std::string &peer_nic, + const std::vector &qp_num, + RdmaEndPoint::HandShakeDesc &local_desc) { + auto gid_selection = context.gidSelection(); + local_desc.local_nic_path = context.nicPath(); + local_desc.local_lid = context.lid(); + local_desc.local_gid = gid_selection.gid; + local_desc.peer_nic_path = peer_nic; + local_desc.qp_num = qp_num; + local_desc.reply_msg.clear(); + return gid_selection; +} + +static void rememberAutoGidSelection( + std::vector &attempted_selections, + const GidSelectionSnapshot &selection) { + auto already_attempted = + std::any_of(attempted_selections.begin(), attempted_selections.end(), + [&](const AutoGidSelectionIdentity &attempted) { + return attempted.gid_index == selection.gid_index && + attempted.gid == selection.gid; + }); + if (!already_attempted) { + attempted_selections.push_back({selection.gid_index, selection.gid}); + } +} RdmaEndPoint::RdmaEndPoint(RdmaContext &context) : context_(context), @@ -262,6 +292,8 @@ int RdmaEndPoint::setupConnectionsByActive() { HandShakeDesc local_desc, peer_desc; std::string peer_server_name, peer_nic_name; bool do_rpc = false; + int auto_gid_retry_count = 0; + std::vector attempted_auto_gid_selections; { RWSpinlock::WriteGuard guard(lock_); @@ -290,12 +322,6 @@ int RdmaEndPoint::setupConnectionsByActive() { disconnectUnlocked(); return ERR_INVALID_ARGUMENT; } - - local_desc.local_nic_path = context_.nicPath(); - local_desc.local_lid = context_.lid(); - local_desc.local_gid = context_.gid(); - local_desc.peer_nic_path = peer_nic_path_; - local_desc.qp_num = qpNum(); } } @@ -332,96 +358,162 @@ int RdmaEndPoint::setupConnectionsByActive() { return connected() ? 0 : ERR_ENDPOINT; } - // Perform the RPC without holding the lock to avoid deadlock and allow - // "simultaneous open" handshake handling. - int rc = context_.engine().sendHandshake(peer_server_name, local_desc, - peer_desc); + for (;;) { + std::vector local_qp_num; + { + RWSpinlock::ReadGuard guard(lock_); + local_qp_num = qpNum(); + } + auto local_gid_selection = fillLocalHandshakeDesc( + context_, peer_nic_path_, local_qp_num, local_desc); + rememberAutoGidSelection(attempted_auto_gid_selections, + local_gid_selection); + peer_desc = HandShakeDesc(); - // We should check the RPC return code before comparing `peer_qp_num_list_` - // with `peer_desc.qp_num`, since a failed RPC may result in an - // invalid `peer_desc.qp_num`. - // - // If the RPC is failed, even if the state is CONNECTED, (which means - // it is handled by setupConnectionsByPassive in another thread during the - // RPC, or "simultaneous open"), we should resetConnection to be safe. - // Because we're not sure whether the peer needs a connection - // re-establishment. (We don't know `peer_desc.qp_num`) - if (rc) { - RWSpinlock::WriteGuard write_guard(lock_); - resetConnection("handshake RPC failure"); - return rc; - } + // Perform the RPC without holding the lock to avoid deadlock and allow + // "simultaneous open" handshake handling. + int rc = context_.engine().sendHandshake(peer_server_name, local_desc, + peer_desc); - // Re-acquire lock after RPC to finalize state transition - RWSpinlock::WriteGuard guard(lock_); - - // Handle simultaneous open: if the peer initiates a connection during our - // RPC and it is passively established in setupConnectionsByPassive, simply - // reuse the existing endpoint. - if (connected()) { - if (peer_qp_num_list_ == peer_desc.qp_num) { - LOG(INFO) << "Received same peer QP numbers, reusing connection."; - return 0; + // We should check the RPC return code before comparing + // `peer_qp_num_list_` with `peer_desc.qp_num`, since a failed RPC may + // result in an invalid `peer_desc.qp_num`. + // + // If the RPC is failed, even if the state is CONNECTED, (which means + // it is handled by setupConnectionsByPassive in another thread during + // the RPC, or "simultaneous open"), we should resetConnection to be + // safe. Because we're not sure whether the peer needs a connection + // re-establishment. (We don't know `peer_desc.qp_num`) + if (rc) { + RWSpinlock::WriteGuard write_guard(lock_); + resetConnection("handshake RPC failure"); + return rc; } - // This mismatch scenario should be rare. It may occur when a peer - // first sends us an Active RPC and establishes a connection, - // then restarts, and eventually accepts and responds to our - // Active RPC. - LOG(WARNING) << "Peer QP list mismatch on connected endpoint, " - "re-establishing connection: " - << toString(); + bool retry_with_new_gid = false; + { + // Re-acquire lock after RPC to finalize state transition + RWSpinlock::WriteGuard guard(lock_); - int ret = resetConnection("re-establishing connection (active)"); - if (ret) return ret; - } - - if (!peer_desc.reply_msg.empty()) { - LOG(ERROR) << "Rejected handshake request by peer " - << local_desc.peer_nic_path; - disconnectUnlocked(); - return ERR_REJECT_HANDSHAKE; - } - - if (peer_desc.local_nic_path != peer_nic_path_ || - peer_desc.peer_nic_path != local_desc.local_nic_path) { - LOG(ERROR) << "Invalid argument: received packet mismatch, " - "local.local_nic_path: " - << local_desc.local_nic_path - << ", local.peer_nic_path: " << local_desc.peer_nic_path - << ", peer.local_nic_path: " << peer_desc.local_nic_path - << ", peer.peer_nic_path: " << peer_desc.peer_nic_path; - disconnectUnlocked(); - return ERR_REJECT_HANDSHAKE; - } - - if (!peer_desc.local_gid.empty()) { - int ret = doSetupConnection(peer_desc.local_gid, peer_desc.local_lid, - peer_desc.qp_num); - if (ret != 0) { - resetConnection("failed connection setup (active)"); - } - return ret; - } else { - auto segment_desc = - context_.engine().meta()->getSegmentDescByName(peer_server_name); - if (segment_desc) { - for (auto &nic : segment_desc->devices) { - if (nic.name == peer_nic_name) { - int ret = - doSetupConnection(nic.gid, nic.lid, peer_desc.qp_num); - if (ret != 0) { - resetConnection("failed connection setup (active)"); - } - return ret; + // Handle simultaneous open: if the peer initiates a connection + // during our RPC and it is passively established in + // setupConnectionsByPassive, simply reuse the existing endpoint. + if (connected()) { + if (peer_qp_num_list_ == peer_desc.qp_num) { + LOG(INFO) + << "Received same peer QP numbers, reusing connection."; + return 0; } + + // This mismatch scenario should be rare. It may occur when a + // peer first sends us an Active RPC and establishes a + // connection, then restarts, and eventually accepts and + // responds to our Active RPC. + LOG(WARNING) << "Peer QP list mismatch on connected endpoint, " + "re-establishing connection: " + << toString(); + + int ret = + resetConnection("re-establishing connection (active)"); + if (ret) return ret; + } + + if (!peer_desc.reply_msg.empty()) { + LOG(ERROR) << "Rejected handshake request by peer " + << local_desc.peer_nic_path; + disconnectUnlocked(); + return ERR_REJECT_HANDSHAKE; + } + + if (peer_desc.local_nic_path != peer_nic_path_ || + peer_desc.peer_nic_path != local_desc.local_nic_path) { + LOG(ERROR) << "Invalid argument: received packet mismatch, " + "local.local_nic_path: " + << local_desc.local_nic_path + << ", local.peer_nic_path: " + << local_desc.peer_nic_path + << ", peer.local_nic_path: " + << peer_desc.local_nic_path + << ", peer.peer_nic_path: " + << peer_desc.peer_nic_path; + disconnectUnlocked(); + return ERR_REJECT_HANDSHAKE; + } + + int ret = ERR_DEVICE_NOT_FOUND; + std::string failure_message; + SetupConnectionFailureInfo failure_info; + if (!peer_desc.local_gid.empty()) { + ret = doSetupConnection(peer_desc.local_gid, + peer_desc.local_lid, peer_desc.qp_num, + &failure_message, &failure_info); + } else { + auto segment_desc = + context_.engine().meta()->getSegmentDescByName( + peer_server_name); + if (segment_desc) { + for (auto &nic : segment_desc->devices) { + if (nic.name == peer_nic_name) { + ret = doSetupConnection( + nic.gid, nic.lid, peer_desc.qp_num, + &failure_message, &failure_info); + break; + } + } + } + } + + if (ret == 0) { + return 0; + } + + if (shouldAttemptAutoGidHandshakeRetry( + context_.autoGidSelectionEnabled(), auto_gid_retry_count, + globalConfig().auto_gid_max_retries, + failure_info.stage == SetupConnectionFailureStage::kRtr, + failure_info.sys_errno)) { + std::string previous_gid; + std::string next_gid; + bool reprobe_changed = context_.reprobeAutoGid( + local_gid_selection, attempted_auto_gid_selections, + &previous_gid, &next_gid); + auto current_gid_selection = context_.gidSelection(); + auto retry_action = decideAutoGidRetryAction( + reprobe_changed, local_gid_selection.gid_index, + local_gid_selection.gid, current_gid_selection.gid_index, + current_gid_selection.gid); + if (retry_action != AutoGidRetryAction::kDoNotRetry) { + int reset_ret = resetConnection( + retry_action == + AutoGidRetryAction::kRetryWithReprobedGid + ? "retry after auto GID reprobe (active)" + : "retry with externally reprobed GID (active)"); + if (reset_ret) return reset_ret; + status_.store(CONNECTING, std::memory_order_relaxed); + ++auto_gid_retry_count; + retry_with_new_gid = true; + LOG(WARNING) + << "Retry active handshake with updated local GID on " + << context_.deviceName() << ": " + << local_gid_selection.gid << " -> " + << current_gid_selection.gid << " (attempt " + << auto_gid_retry_count << "/" + << globalConfig().auto_gid_max_retries << ")"; + } + } + + if (!retry_with_new_gid) { + if (ret == ERR_DEVICE_NOT_FOUND) { + LOG(ERROR) << "Peer NIC " << peer_nic_name + << " not found in " << peer_server_name; + disconnectUnlocked(); + } else { + resetConnection("failed connection setup (active)"); + } + return ret; } } } - LOG(ERROR) << "Peer NIC " << peer_nic_name << " not found in " - << peer_server_name; - disconnectUnlocked(); - return ERR_DEVICE_NOT_FOUND; } int RdmaEndPoint::setupConnectionsByPassive(const HandShakeDesc &peer_desc, @@ -430,11 +522,8 @@ int RdmaEndPoint::setupConnectionsByPassive(const HandShakeDesc &peer_desc, if (connected()) { // If already connected with the same peer QP info, return success if (peer_qp_num_list_ == peer_desc.qp_num) { - local_desc.local_nic_path = context_.nicPath(); - local_desc.local_lid = context_.lid(); - local_desc.local_gid = context_.gid(); - local_desc.peer_nic_path = peer_nic_path_; - local_desc.qp_num = qpNum(); + fillLocalHandshakeDesc(context_, peer_nic_path_, qpNum(), + local_desc); LOG(INFO) << "Received same peer QP numbers, reusing connection."; return 0; } @@ -473,38 +562,82 @@ int RdmaEndPoint::setupConnectionsByPassive(const HandShakeDesc &peer_desc, return ERR_INVALID_ARGUMENT; } - local_desc.local_nic_path = context_.nicPath(); - local_desc.local_lid = context_.lid(); - local_desc.local_gid = context_.gid(); - local_desc.peer_nic_path = peer_nic_path_; - local_desc.qp_num = qpNum(); + status_.store(CONNECTING, std::memory_order_relaxed); + + auto attempt_setup_with_peer = [&](const std::string &peer_gid, + uint16_t peer_lid) -> int { + int auto_gid_retry_count = 0; + std::vector attempted_auto_gid_selections; + for (;;) { + auto local_gid_selection = fillLocalHandshakeDesc( + context_, peer_nic_path_, qpNum(), local_desc); + rememberAutoGidSelection(attempted_auto_gid_selections, + local_gid_selection); + + SetupConnectionFailureInfo failure_info; + int ret = doSetupConnection(peer_gid, peer_lid, peer_desc.qp_num, + &local_desc.reply_msg, &failure_info); + if (ret == 0) { + return 0; + } + + if (!shouldAttemptAutoGidHandshakeRetry( + context_.autoGidSelectionEnabled(), auto_gid_retry_count, + globalConfig().auto_gid_max_retries, + failure_info.stage == SetupConnectionFailureStage::kRtr, + failure_info.sys_errno)) { + resetConnection("failed connection setup (passive)"); + return ret; + } + + std::string previous_gid; + std::string next_gid; + bool reprobe_changed = context_.reprobeAutoGid( + local_gid_selection, attempted_auto_gid_selections, + &previous_gid, &next_gid); + auto current_gid_selection = context_.gidSelection(); + auto retry_action = decideAutoGidRetryAction( + reprobe_changed, local_gid_selection.gid_index, + local_gid_selection.gid, current_gid_selection.gid_index, + current_gid_selection.gid); + if (retry_action == AutoGidRetryAction::kDoNotRetry) { + resetConnection("failed connection setup (passive)"); + return ret; + } + + int reset_ret = resetConnection( + retry_action == AutoGidRetryAction::kRetryWithReprobedGid + ? "retry after auto GID reprobe (passive)" + : "retry with externally reprobed GID (passive)"); + if (reset_ret) return reset_ret; + status_.store(CONNECTING, std::memory_order_relaxed); + ++auto_gid_retry_count; + LOG(WARNING) << "Retry passive handshake with updated local GID on " + << context_.deviceName() << ": " + << local_gid_selection.gid << " -> " + << current_gid_selection.gid << " (attempt " + << auto_gid_retry_count << "/" + << globalConfig().auto_gid_max_retries << ")"; + } + }; if (!peer_desc.local_gid.empty()) { - int ret = doSetupConnection(peer_desc.local_gid, peer_desc.local_lid, - peer_desc.qp_num, &local_desc.reply_msg); - if (ret != 0) { - resetConnection("failed connection setup (passive)"); - } - return ret; + return attempt_setup_with_peer(peer_desc.local_gid, + peer_desc.local_lid); } else { auto segment_desc = context_.engine().meta()->getSegmentDescByName(peer_server_name); if (segment_desc) { for (auto &nic : segment_desc->devices) { if (nic.name == peer_nic_name) { - int ret = - doSetupConnection(nic.gid, nic.lid, peer_desc.qp_num, - &local_desc.reply_msg); - if (ret != 0) { - resetConnection("failed connection setup (passive)"); - } - return ret; + return attempt_setup_with_peer(nic.gid, nic.lid); } } } } local_desc.reply_msg = "Peer nic not found in that server: " + peer_nic_path_; + status_.store(UNCONNECTED, std::memory_order_relaxed); LOG(ERROR) << local_desc.reply_msg; return ERR_DEVICE_NOT_FOUND; } @@ -714,13 +847,18 @@ static int parseGidString(const std::string &gid_str, ibv_gid &gid_out) { int RdmaEndPoint::doSetupConnection(const std::string &peer_gid, uint16_t peer_lid, std::vector peer_qp_num_list, - std::string *reply_msg) { + std::string *reply_msg, + SetupConnectionFailureInfo *failure_info) { if (qp_list_.size() != peer_qp_num_list.size()) { std::string message = "QP count mismatch in peer and local endpoints, check " "MC_MAX_EP_PER_CTX"; LOG(ERROR) << "[Handshake] " << message; if (reply_msg) *reply_msg = message; + if (failure_info) { + failure_info->stage = SetupConnectionFailureStage::kPeerValidation; + failure_info->sys_errno = 0; + } return ERR_INVALID_ARGUMENT; } @@ -731,12 +869,18 @@ int RdmaEndPoint::doSetupConnection(const std::string &peer_gid, std::string message = "Invalid peer GID: " + peer_gid; LOG(ERROR) << "[Handshake] " << message; if (reply_msg) *reply_msg = message; + if (failure_info) { + failure_info->stage = SetupConnectionFailureStage::kPeerValidation; + failure_info->sys_errno = 0; + } return ret; } + int local_gid_index = context_.gidIndex(); for (int qp_index = 0; qp_index < (int)qp_list_.size(); ++qp_index) { int ret = doSetupConnection(qp_index, peer_gid_raw, peer_lid, - peer_qp_num_list[qp_index], reply_msg); + peer_qp_num_list[qp_index], local_gid_index, + reply_msg, failure_info); if (ret) return ret; } @@ -747,7 +891,8 @@ int RdmaEndPoint::doSetupConnection(const std::string &peer_gid, int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, uint16_t peer_lid, uint32_t peer_qp_num, - std::string *reply_msg) { + int local_gid_index, std::string *reply_msg, + SetupConnectionFailureInfo *failure_info) { if (qp_index < 0 || qp_index > (int)qp_list_.size()) return ERR_INVALID_ARGUMENT; auto &qp = qp_list_[qp_index]; @@ -761,6 +906,10 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, std::string message = "Failed to modify QP to RESET"; PLOG(ERROR) << "[Handshake] " << message; if (reply_msg) *reply_msg = message + ": " + strerror(errno); + if (failure_info) { + failure_info->stage = SetupConnectionFailureStage::kReset; + failure_info->sys_errno = errno; + } return ERR_ENDPOINT; } @@ -779,6 +928,10 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, "Failed to modify QP to INIT, check local context port num"; PLOG(ERROR) << "[Handshake] " << message; if (reply_msg) *reply_msg = message + ": " + strerror(errno); + if (failure_info) { + failure_info->stage = SetupConnectionFailureStage::kInit; + failure_info->sys_errno = errno; + } return ERR_ENDPOINT; } @@ -790,8 +943,8 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, attr.path_mtu = globalConfig().mtu_length; attr.ah_attr.grh.dgid = peer_gid; // TODO gidIndex and portNum must fetch from REMOTE - attr.ah_attr.grh.sgid_index = context_.gidIndex(); - attr.ah_attr.grh.hop_limit = MAX_HOP_LIMIT; + attr.ah_attr.grh.sgid_index = local_gid_index; + attr.ah_attr.grh.hop_limit = kMaxHopLimit; // Set traffic class if configured (-1 means use default) if (globalConfig().ib_traffic_class >= 0) { attr.ah_attr.grh.traffic_class = @@ -816,14 +969,18 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, "Failed to modify QP to RTR, check mtu, gid, peer lid, peer qp num"; PLOG(ERROR) << "[Handshake] " << message; if (reply_msg) *reply_msg = message + ": " + strerror(errno); + if (failure_info) { + failure_info->stage = SetupConnectionFailureStage::kRtr; + failure_info->sys_errno = errno; + } return ERR_ENDPOINT; } // RTR -> RTS memset(&attr, 0, sizeof(attr)); attr.qp_state = IBV_QPS_RTS; - attr.timeout = TIMEOUT; - attr.retry_cnt = RETRY_CNT; + attr.timeout = kTimeout; + attr.retry_cnt = kRetryCount; attr.rnr_retry = 7; // or 7,RNR error attr.sq_psn = 0; attr.max_rd_atomic = 16; @@ -835,6 +992,10 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, std::string message = "Failed to modify QP to RTS"; PLOG(ERROR) << "[Handshake] " << message; if (reply_msg) *reply_msg = message + ": " + strerror(errno); + if (failure_info) { + failure_info->stage = SetupConnectionFailureStage::kRts; + failure_info->sys_errno = errno; + } return ERR_ENDPOINT; } @@ -900,4 +1061,5 @@ int RdmaEndPoint::doSetupConnection(int qp_index, const ibv_gid &peer_gid, return 0; } + } // namespace mooncake diff --git a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp index 49f622d8..753cc1b9 100644 --- a/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp +++ b/mooncake-transfer-engine/src/transport/rdma_transport/rdma_transport.cpp @@ -375,7 +375,7 @@ int RdmaTransport::unregisterLocalMemoryInternal(void *addr, } int RdmaTransport::allocateLocalSegmentID() { - auto desc = metadata_->getSegmentDesc(local_server_name_); + auto desc = metadata_->getSegmentDescByID(LOCAL_SEGMENT_ID); if (!desc) desc = std::make_shared(); desc->name = local_server_name_; // Store RDMA server name for dual-NIC setups; when it differs from @@ -402,6 +402,34 @@ int RdmaTransport::allocateLocalSegmentID() { return 0; } +int RdmaTransport::refreshLocalDeviceDesc(const std::string &device_name, + uint16_t lid, + const std::string &gid) { + std::lock_guard guard(local_desc_lock_); + auto original_desc = metadata_->getSegmentDescByID(LOCAL_SEGMENT_ID); + if (!original_desc) { + return ERR_ADDRESS_NOT_REGISTERED; + } + + auto updated_desc = std::make_shared(*original_desc); + for (auto &device : updated_desc->devices) { + if (device.name != device_name) continue; + device.lid = lid; + device.gid = gid; + metadata_->addLocalSegment(LOCAL_SEGMENT_ID, local_server_name_, + std::move(updated_desc)); + int ret = metadata_->updateLocalSegmentDesc(); + if (ret) { + auto rollback_desc = original_desc; + metadata_->addLocalSegment(LOCAL_SEGMENT_ID, local_server_name_, + std::move(rollback_desc)); + } + return ret; + } + + return ERR_DEVICE_NOT_FOUND; +} + int RdmaTransport::registerLocalMemoryBatch( const std::vector &buffer_list, const std::string &location) { diff --git a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp index 55a3a34b..4a80f357 100644 --- a/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp +++ b/mooncake-transfer-engine/tent/src/transport/rdma/rdma_transport.cpp @@ -21,9 +21,11 @@ #include #include +#include #include #include #include +#include #include #include @@ -47,6 +49,39 @@ namespace mooncake { namespace tent { + +namespace { + +uint16_t getRdmaBindDefaultPort(const Config& config) { + constexpr const char* kKey = "rpc_server_port"; + if (!config.contains(kKey)) return 0; + + json raw_value = config.get(kKey, json()); + if (raw_value.is_number_integer() || raw_value.is_number_unsigned()) { + long long value = raw_value.get(); + if (value >= 0 && value <= static_cast( + std::numeric_limits::max())) { + return static_cast(value); + } + return 0; + } + + if (raw_value.is_string()) { + const std::string string_value = raw_value.get(); + char* end = nullptr; + errno = 0; + unsigned long value = std::strtoul(string_value.c_str(), &end, 10); + if (errno == 0 && end != string_value.c_str() && *end == '\0' && + value <= std::numeric_limits::max()) { + return static_cast(value); + } + } + + return 0; +} + +} // namespace + static Status configureLaneCount(std::shared_ptr conf, std::shared_ptr params) { constexpr int kUnset = -1; @@ -217,12 +252,12 @@ Status RdmaTransport::install(std::string& local_segment_name, local_topology_ = local_topology; // In dual-NIC environments (e.g. separate TCP and RDMA interfaces), - // transports/rdma/bind_address allows NIC paths to use an RDMA-reachable - // IP while local_segment_name_ keeps the TCP-reachable address for P2P. + // transports/rdma/bind_address allows NIC paths to use an + // RDMA-reachable IP while local_segment_name_ keeps the + // TCP-reachable address for P2P. const auto rdma_bind_addr = conf_->get("transports/rdma/bind_address", ""); if (!rdma_bind_addr.empty()) { - const auto default_port = - conf_->get("rpc_server_port", static_cast(0)); + const uint16_t default_port = getRdmaBindDefaultPort(*conf_); auto [host_name, port] = parseHostNameWithPort(local_segment_name, default_port); rdma_server_name_ = rdma_bind_addr + ":" + std::to_string(port); diff --git a/mooncake-transfer-engine/tests/CMakeLists.txt b/mooncake-transfer-engine/tests/CMakeLists.txt index 0dafe93e..7fc5af5d 100644 --- a/mooncake-transfer-engine/tests/CMakeLists.txt +++ b/mooncake-transfer-engine/tests/CMakeLists.txt @@ -36,9 +36,16 @@ target_link_libraries(rdma_loopback_test PUBLIC transfer_engine gtest gtest_main # add_test(NAME rdma_loopback_test COMMAND rdma_loopback_test) # This test verifies endpoint re-establishment in RDMATransport. -# Intended for manual testing only. add_executable(rdma_endpoint_reestablish_test ${WORKSPACE}/rdma_endpoint_reestablish_test.cpp) target_link_libraries(rdma_endpoint_reestablish_test PUBLIC transfer_engine gtest gtest_main ) +if (UNIX AND NOT APPLE) + target_link_options(rdma_endpoint_reestablish_test PRIVATE + "-Wl,--wrap=ibv_modify_qp" + "-Wl,--wrap=ibv_query_gid" + "-Wl,--wrap=_ibv_query_gid_ex") +endif() +add_test(NAME rdma_endpoint_reestablish_test COMMAND rdma_endpoint_reestablish_test) +set_tests_properties(rdma_endpoint_reestablish_test PROPERTIES LABELS "rdma") if (USE_CXL) add_executable(cxl_transport_test ${WORKSPACE}/cxl_transport_test.cpp) @@ -98,6 +105,18 @@ add_executable(transfer_metadata_test ${WORKSPACE}/transfer_metadata_test.cpp) target_link_libraries(transfer_metadata_test PUBLIC transfer_engine gtest gtest_main) add_test(NAME transfer_metadata_test COMMAND transfer_metadata_test) +add_executable(config_test ${WORKSPACE}/config_test.cpp) +target_link_libraries(config_test PUBLIC transfer_engine gtest gtest_main) +add_test(NAME config_test COMMAND config_test) + +add_executable(rdma_gid_probe_test ${WORKSPACE}/rdma_gid_probe_test.cpp) +target_link_libraries(rdma_gid_probe_test PUBLIC transfer_engine gtest gtest_main) +add_test(NAME rdma_gid_probe_test COMMAND rdma_gid_probe_test) + +add_executable(rdma_context_reprobe_test ${WORKSPACE}/rdma_context_reprobe_test.cpp) +target_link_libraries(rdma_context_reprobe_test PUBLIC transfer_engine gtest gtest_main) +add_test(NAME rdma_context_reprobe_test COMMAND rdma_context_reprobe_test) + add_executable(topology_test ${WORKSPACE}/topology_test.cpp) target_link_libraries(topology_test PUBLIC transfer_engine gtest gtest_main) add_test(NAME topology_test COMMAND topology_test) diff --git a/mooncake-transfer-engine/tests/config_test.cpp b/mooncake-transfer-engine/tests/config_test.cpp index ba108d9f..4b58ab70 100644 --- a/mooncake-transfer-engine/tests/config_test.cpp +++ b/mooncake-transfer-engine/tests/config_test.cpp @@ -23,7 +23,10 @@ namespace { class PkeyIndexEnvTest : public ::testing::Test { protected: - void TearDown() override { ::unsetenv("MC_PKEY_INDEX"); } + void TearDown() override { + ::unsetenv("MC_PKEY_INDEX"); + ::unsetenv("MC_AUTO_GID_MAX_RETRIES"); + } }; TEST_F(PkeyIndexEnvTest, DefaultIsZeroWhenUnset) { @@ -79,5 +82,28 @@ TEST_F(PkeyIndexEnvTest, EmptyStringKeepsDefault) { EXPECT_EQ(config.pkey_index, 4); } +TEST_F(PkeyIndexEnvTest, AutoGidRetriesDefaultsToTwoWhenUnset) { + ::unsetenv("MC_AUTO_GID_MAX_RETRIES"); + GlobalConfig config; + config.auto_gid_max_retries = 2; + loadGlobalConfig(config); + EXPECT_EQ(config.auto_gid_max_retries, 2); +} + +TEST_F(PkeyIndexEnvTest, AutoGidRetriesAcceptsValidOverride) { + ASSERT_EQ(::setenv("MC_AUTO_GID_MAX_RETRIES", "0", 1), 0); + GlobalConfig config; + loadGlobalConfig(config); + EXPECT_EQ(config.auto_gid_max_retries, 0); +} + +TEST_F(PkeyIndexEnvTest, AutoGidRetriesRejectsOutOfRangeOverride) { + ASSERT_EQ(::setenv("MC_AUTO_GID_MAX_RETRIES", "99", 1), 0); + GlobalConfig config; + config.auto_gid_max_retries = 5; + loadGlobalConfig(config); + EXPECT_EQ(config.auto_gid_max_retries, 5); +} + } // namespace } // namespace mooncake diff --git a/mooncake-transfer-engine/tests/rdma_context_reprobe_test.cpp b/mooncake-transfer-engine/tests/rdma_context_reprobe_test.cpp new file mode 100644 index 00000000..319d57b2 --- /dev/null +++ b/mooncake-transfer-engine/tests/rdma_context_reprobe_test.cpp @@ -0,0 +1,144 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include + +#include +#include +#include +#include + +#include "common.h" +#include "error.h" +#include "transfer_metadata.h" +#include "transport/rdma_transport/rdma_context.h" +#include "transport/rdma_transport/rdma_transport.h" + +#if defined(__has_feature) +#define MC_HAS_FEATURE(x) __has_feature(x) +#else +#define MC_HAS_FEATURE(x) 0 +#endif +#if defined(__SANITIZE_ADDRESS__) || MC_HAS_FEATURE(address_sanitizer) +#include +#define MC_LSAN_IGNORE_OBJECT(p) __lsan_ignore_object(p) +#else +#define MC_LSAN_IGNORE_OBJECT(p) ((void)(p)) +#endif + +using namespace mooncake; + +namespace mooncake { + +class RdmaTransportTestPeer { + public: + static void bindMetadata(RdmaTransport &transport, + std::shared_ptr metadata, + std::string local_server_name) { + transport.metadata_ = std::move(metadata); + transport.local_server_name_ = std::move(local_server_name); + } +}; + +class RdmaContextTestPeer { + public: + static void seedAutoGidState(RdmaContext &context, ibv_context *verbs_ctx, + uint8_t port, uint16_t lid, const ibv_gid &gid, + int gid_index) { + context.context_ = verbs_ctx; + context.port_ = port; + context.lid_ = lid; + context.gid_ = gid; + context.gid_index_ = gid_index; + context.auto_gid_selection_enabled_ = true; + } + + static void disableContextForTeardown(RdmaContext &context) { + context.context_ = nullptr; + } +}; + +} // namespace mooncake + +namespace { + +ibv_gid makeGid(const std::array &bytes) { + ibv_gid gid = {}; + std::memcpy(gid.raw, bytes.data(), bytes.size()); + return gid; +} + +std::string formatGid(const std::array &bytes) { + std::string gid; + char buf[4] = {0}; + for (size_t i = 0; i < bytes.size(); ++i) { + std::snprintf(buf, sizeof(buf), "%02x", bytes[i]); + gid += i == 0 ? buf : std::string(":") + buf; + } + return gid; +} + +constexpr std::array kCurrentGid = { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x11}; +class RdmaContextReprobeTest : public ::testing::Test { + protected: + void SetUp() override { + transport_ = new RdmaTransport(); + MC_LSAN_IGNORE_OBJECT(transport_); + metadata_ = std::make_shared(P2PHANDSHAKE); + RdmaTransportTestPeer::bindMetadata(*transport_, metadata_, + "local-rdma-segment"); + + auto local_desc = std::make_shared(); + local_desc->name = "local-rdma-segment"; + local_desc->protocol = "rdma"; + local_desc->devices.push_back( + {"synthetic0", 23, formatGid(kCurrentGid), ""}); + ASSERT_EQ( + metadata_->addLocalSegment(LOCAL_SEGMENT_ID, "local-rdma-segment", + std::move(local_desc)), + 0); + + context_ = new RdmaContext(*transport_, "synthetic0"); + MC_LSAN_IGNORE_OBJECT(context_); + RdmaContextTestPeer::seedAutoGidState( + *context_, reinterpret_cast(0x1), /*port=*/1, + /*lid=*/23, makeGid(kCurrentGid), /*gid_index=*/0); + } + + std::shared_ptr localDesc() const { + return metadata_->getSegmentDescByID(LOCAL_SEGMENT_ID); + } + + RdmaTransport *transport_ = nullptr; + std::shared_ptr metadata_; + RdmaContext *context_ = nullptr; +}; + +TEST_F(RdmaContextReprobeTest, + ReprobeStopsWhenExpectedSelectionDoesNotMatchCurrentState) { + auto before_desc = localDesc(); + ASSERT_TRUE(before_desc); + + bool changed = context_->reprobeAutoGid({formatGid(kCurrentGid), 9}, {}); + + EXPECT_FALSE(changed); + EXPECT_EQ(context_->gidIndex(), 0); + EXPECT_EQ(context_->gid(), formatGid(kCurrentGid)); + auto after_desc = localDesc(); + EXPECT_EQ(after_desc.get(), before_desc.get()); +} + +} // namespace diff --git a/mooncake-transfer-engine/tests/rdma_endpoint_reestablish_test.cpp b/mooncake-transfer-engine/tests/rdma_endpoint_reestablish_test.cpp index 3498b9bf..77f58192 100644 --- a/mooncake-transfer-engine/tests/rdma_endpoint_reestablish_test.cpp +++ b/mooncake-transfer-engine/tests/rdma_endpoint_reestablish_test.cpp @@ -16,49 +16,169 @@ * RDMA Endpoint Re-establishment Test * * Purpose: - * This test verifies that TE correctly handles endpoint re-establish - * during simulated Initiator restarts. + * This test verifies that TE correctly handles endpoint re-establish during + * simulated initiator restarts, and that classic RDMA can recover from a first + * RTR/EINVAL by reprobeing the next auto-selected local GID. * * How to run: - * 1. Start etcd: - * etcd --listen-client-urls http://127.0.0.1:18222 \ - * --advertise-client-urls http://127.0.0.1:18222 - * - * 2. Run test - * sudo env MC_METADATA_SERVER=127.0.0.1:18222 \ - * MC_TARGET_SERVER_NAME=127.0.0.1:12345 \ - * MC_INITIATOR_SERVER_NAME=127.0.0.1:12346 \ - * MC_TARGET_DEVICE_NAME=erdma_0 MC_INITIATOR_DEVICE_NAME=erdma_1 \ - * ./build/mooncake-transfer-engine/tests/rdma_endpoint_reestablish_test + * sudo env MC_METADATA_SERVER=P2PHANDSHAKE \ + * MC_TARGET_SERVER_NAME=127.0.0.1:12345 \ + * MC_INITIATOR_SERVER_NAME=127.0.0.1:12346 \ + * MC_TARGET_DEVICE_NAME=erdma_0 MC_INITIATOR_DEVICE_NAME=erdma_1 \ + * ./build/mooncake-transfer-engine/tests/rdma_endpoint_reestablish_test */ -#include -#include -#include -#include -#include -#include - +#include +#include #include #include #include #include +#include #include +#include +#include #include +#include +#include +#include +#include +#include +#include +#include + +#include "common.h" #include "transfer_engine.h" #include "transport/transport.h" -#include "common.h" using namespace mooncake; -// Size of the pre-registered memory. -constexpr size_t kRAMBufSize = 256ull << 24; // 256 MB +namespace { -// Actual data payload size for RDMA Read/Write. -constexpr size_t kDataLength = 16ull << 24; // 16MB +constexpr size_t kRAMBufSize = 256ull << 24; +constexpr size_t kDataLength = 16ull << 24; -std::string formatDeviceNames(const std::string &device_names) { +bool usesP2PHandshake(const std::string& metadata_server) { + return metadata_server == P2PHANDSHAKE; +} + +std::vector getAvailableRdmaDevices() { + int num_devices = 0; + ibv_device** device_list = ibv_get_device_list(&num_devices); + std::vector devices; + if (device_list == nullptr) { + return devices; + } + devices.reserve(num_devices); + for (int i = 0; i < num_devices; ++i) { + devices.emplace_back(ibv_get_device_name(device_list[i])); + } + ibv_free_device_list(device_list); + return devices; +} + +struct RtrFaultInjectionState { + std::mutex mu; + bool synthetic_gid_swap_enabled = false; + std::string synthetic_gid_device; + bool fail_first_rtr_einval = false; + std::string fail_rtr_device; + int injected_failures = 0; + std::unordered_map> rtr_sgid_history; + std::unordered_map> rtr_gid_history; +} g_rtr_fault_injection_state; + +std::string formatGidBytes(const uint8_t* raw) { + std::ostringstream oss; + oss << std::hex << std::setfill('0'); + for (size_t i = 0; i < 16; ++i) { + if (i != 0) { + oss << ":"; + } + oss << std::setw(2) << static_cast(raw[i]); + } + return oss.str(); +} + +void resetRtrFaultInjectionState() { + std::lock_guard guard(g_rtr_fault_injection_state.mu); + g_rtr_fault_injection_state.synthetic_gid_swap_enabled = false; + g_rtr_fault_injection_state.synthetic_gid_device.clear(); + g_rtr_fault_injection_state.fail_first_rtr_einval = false; + g_rtr_fault_injection_state.fail_rtr_device.clear(); + g_rtr_fault_injection_state.injected_failures = 0; + g_rtr_fault_injection_state.rtr_sgid_history.clear(); + g_rtr_fault_injection_state.rtr_gid_history.clear(); +} + +void configureRtrFaultInjection(const std::string& device_name) { + std::lock_guard guard(g_rtr_fault_injection_state.mu); + g_rtr_fault_injection_state.synthetic_gid_swap_enabled = true; + g_rtr_fault_injection_state.synthetic_gid_device = device_name; + g_rtr_fault_injection_state.fail_first_rtr_einval = true; + g_rtr_fault_injection_state.fail_rtr_device = device_name; + g_rtr_fault_injection_state.injected_failures = 0; + g_rtr_fault_injection_state.rtr_sgid_history.clear(); + g_rtr_fault_injection_state.rtr_gid_history.clear(); +} + +std::vector getRtrSgidHistory(const std::string& device_name) { + std::lock_guard guard(g_rtr_fault_injection_state.mu); + auto iter = g_rtr_fault_injection_state.rtr_sgid_history.find(device_name); + if (iter == g_rtr_fault_injection_state.rtr_sgid_history.end()) { + return {}; + } + return iter->second; +} + +int getInjectedFailureCount() { + std::lock_guard guard(g_rtr_fault_injection_state.mu); + return g_rtr_fault_injection_state.injected_failures; +} + +std::vector getRtrGidHistory(const std::string& device_name) { + std::lock_guard guard(g_rtr_fault_injection_state.mu); + auto iter = g_rtr_fault_injection_state.rtr_gid_history.find(device_name); + if (iter == g_rtr_fault_injection_state.rtr_gid_history.end()) { + return {}; + } + return iter->second; +} + +void recordRtrAttempt(const std::string& device_name, int sgid_index, + const std::string& gid) { + std::lock_guard guard(g_rtr_fault_injection_state.mu); + g_rtr_fault_injection_state.rtr_sgid_history[device_name].push_back( + sgid_index); + g_rtr_fault_injection_state.rtr_gid_history[device_name].push_back(gid); +} + +int maybeSwapSyntheticGidIndex(const std::string& device_name, int gid_index) { + std::lock_guard guard(g_rtr_fault_injection_state.mu); + if (!g_rtr_fault_injection_state.synthetic_gid_swap_enabled || + g_rtr_fault_injection_state.synthetic_gid_device != device_name) { + return gid_index; + } + if (gid_index == 0) return 1; + if (gid_index == 1) return 0; + return gid_index; +} + +bool shouldInjectRtrEinval(const std::string& device_name, int sgid_index) { + std::lock_guard guard(g_rtr_fault_injection_state.mu); + if (!g_rtr_fault_injection_state.fail_first_rtr_einval || + g_rtr_fault_injection_state.fail_rtr_device != device_name || + sgid_index != 0) { + return false; + } + g_rtr_fault_injection_state.fail_first_rtr_einval = false; + g_rtr_fault_injection_state.synthetic_gid_swap_enabled = false; + ++g_rtr_fault_injection_state.injected_failures; + return true; +} + +std::string formatDeviceNames(const std::string& device_names) { std::stringstream ss(device_names); std::string item; std::vector tokens; @@ -76,7 +196,7 @@ std::string formatDeviceNames(const std::string &device_names) { return formatted; } -std::string makeNicPriorityMatrix(const std::string &device_name) { +std::string makeNicPriorityMatrix(const std::string& device_name) { auto formatted_devices = formatDeviceNames(device_name); return "{\"cpu:0\": [[" + formatted_devices + "],[]], " @@ -84,8 +204,8 @@ std::string makeNicPriorityMatrix(const std::string &device_name) { formatted_devices + "],[]]}"; } -void wait_for_transfer(TransferEngine *engine, BatchID batch_id, - const std::string &op_name) { +void waitForTransfer(TransferEngine* engine, BatchID batch_id, + const std::string& op_name) { bool completed = false; TransferStatus status; while (!completed) { @@ -103,27 +223,26 @@ void wait_for_transfer(TransferEngine *engine, BatchID batch_id, struct TEContext { std::unique_ptr engine_{}; - uint8_t *local_addr_{}; + uint8_t* local_addr_{}; bool segment_opened_{false}; SegmentHandle segment_handle_{}; uint64_t remote_base_{}; - TEContext(const std::string &local_server_name, - const std::string &metadata_server, const std::string &segment_id, - const std::string &device_name) { + TEContext(const std::string& local_server_name, + const std::string& metadata_server, const std::string& segment_id, + const std::string& device_name) { engine_ = std::make_unique(false); auto hostname_port = parseHostNameWithPort(local_server_name); engine_->init(metadata_server, local_server_name, hostname_port.first, hostname_port.second); auto nic_priority_matrix = makeNicPriorityMatrix(device_name); - void *args[2] = {const_cast(nic_priority_matrix.c_str()), + void* args[2] = {const_cast(nic_priority_matrix.c_str()), nullptr}; - - Transport *xport = engine_->installTransport("rdma", args); + Transport* xport = engine_->installTransport("rdma", args); LOG_ASSERT(xport); - local_addr_ = static_cast(numa_alloc_onnode(kRAMBufSize, 0)); + local_addr_ = static_cast(numa_alloc_onnode(kRAMBufSize, 0)); memset(local_addr_, 0, kDataLength); int rc = @@ -145,16 +264,21 @@ struct TEContext { numa_free(local_addr_, kRAMBufSize); if (segment_opened_) engine_->closeSegment(segment_handle_); } + + std::string localSegmentName() const { + return engine_->getLocalIpAndPort(); + } }; class RDMAEndpointReestablishTest : public ::testing::Test { protected: void SetUp() override { + resetRtrFaultInjectionState(); google::InitGoogleLogging("RDMAEndpointReestablishTest"); FLAGS_logtostderr = true; - const char *env = std::getenv("MC_METADATA_SERVER"); - metadata_server = env ? env : "127.0.0.1:18222"; + const char* env = std::getenv("MC_METADATA_SERVER"); + metadata_server = env ? env : P2PHANDSHAKE; LOG(INFO) << "metadata_server: " << metadata_server; env = std::getenv("MC_TARGET_SERVER_NAME"); @@ -165,16 +289,99 @@ class RDMAEndpointReestablishTest : public ::testing::Test { initiator_server_name = env ? env : "127.0.0.1:12346"; LOG(INFO) << "initiator_server_name: " << initiator_server_name; + auto devices = getAvailableRdmaDevices(); + if (devices.size() < 2) { + GTEST_SKIP() << "Need at least two RDMA devices, found " + << devices.size(); + } + env = std::getenv("MC_TARGET_DEVICE_NAME"); - target_device_name = env ? env : "erdma_0"; + target_device_name = env ? env : devices[0]; LOG(INFO) << "target_device_name: " << target_device_name; env = std::getenv("MC_INITIATOR_DEVICE_NAME"); - initiator_device_name = env ? env : "erdma_1"; + initiator_device_name = env ? env : devices[1]; LOG(INFO) << "initiator_device_name: " << initiator_device_name; } - void TearDown() override { google::ShutdownGoogleLogging(); } + void TearDown() override { + google::ShutdownGoogleLogging(); + resetRtrFaultInjectionState(); + } + + void runEndpointReestablishScenario(const std::string& target_device, + const std::string& initiator_device) { + LOG(INFO) << "========== Setting up Target =========="; + TEContext target_ctx(target_server_name, metadata_server, "", + target_device); + const std::string target_segment_name = + usesP2PHandshake(metadata_server) ? target_ctx.localSegmentName() + : target_server_name; + LOG(INFO) << "Resolved target segment name: " << target_segment_name; + LOG(INFO) + << "Target is up. Waiting for RDMA connections and operations..."; + + LOG(INFO) << "========== Phase 1: Start, Connect & Write =========="; + { + TEContext init_ctx(initiator_server_name, metadata_server, + target_segment_name, initiator_device); + for (size_t i = 0; i < kDataLength; ++i) { + init_ctx.local_addr_[i] = static_cast(i % 256); + } + + LOG(INFO) << "Writing " << kDataLength << " bytes to Target..."; + auto batch_id = init_ctx.engine_->allocateBatchID(1); + TransferRequest entry; + entry.opcode = TransferRequest::WRITE; + entry.length = kDataLength; + entry.source = init_ctx.local_addr_; + entry.target_id = init_ctx.segment_handle_; + entry.target_offset = init_ctx.remote_base_; + + Status s = init_ctx.engine_->submitTransfer(batch_id, {entry}); + ASSERT_EQ(s, Status::OK()); + waitForTransfer(init_ctx.engine_.get(), batch_id, "WRITE"); + LOG(INFO) << "Phase 1: Write Completed. Tearing down connection..."; + } + + LOG(INFO) << "Simulating Initiator Crash/Restart... Waiting 2 seconds."; + sleep(2); + + LOG(INFO) + << "========== Phase 2: Restart, Re-establish Endpoint & Read " + "=========="; + { + TEContext init_ctx(initiator_server_name, metadata_server, + target_segment_name, initiator_device); + LOG(INFO) << "Reading data back over new Endpoint..."; + auto batch_id = init_ctx.engine_->allocateBatchID(1); + TransferRequest entry; + entry.opcode = TransferRequest::READ; + entry.length = kDataLength; + entry.source = init_ctx.local_addr_; + entry.target_id = init_ctx.segment_handle_; + entry.target_offset = init_ctx.remote_base_; + + Status s = init_ctx.engine_->submitTransfer(batch_id, {entry}); + ASSERT_EQ(s, Status::OK()); + waitForTransfer(init_ctx.engine_.get(), batch_id, "READ"); + + bool ok = true; + for (size_t i = 0; i < kDataLength; ++i) { + if (init_ctx.local_addr_[i] != static_cast(i % 256)) { + ok = false; + LOG(ERROR) << "Data mismatch at offset " << i + << ", expected " << (i % 256) << ", got " + << (int)init_ctx.local_addr_[i]; + break; + } + } + + ASSERT_TRUE(ok) << "Endpoint Reconstruction Verification Failed!"; + LOG(INFO) << ">>> ENDPOINT RECONSTRUCTION VERIFICATION: " + "\033[32mSUCCESS\033"; + } + } std::string metadata_server; std::string target_server_name; @@ -184,77 +391,94 @@ class RDMAEndpointReestablishTest : public ::testing::Test { }; TEST_F(RDMAEndpointReestablishTest, EndpointReestablish) { - // 1. Setup Target (will stay alive until test function exits) - LOG(INFO) << "========== Setting up Target =========="; - TEContext target_ctx(target_server_name, metadata_server, "", - target_device_name); - LOG(INFO) << "Target is up. Waiting for RDMA connections and operations..."; - - // 2. Phase 1: Initiator Start, Connect & Write - LOG(INFO) << "========== Phase 1: Start, Connect & Write =========="; - { - TEContext init_ctx(initiator_server_name, metadata_server, - target_server_name, initiator_device_name); - - // Fill buffer with test pattern - for (size_t i = 0; i < kDataLength; ++i) { - init_ctx.local_addr_[i] = static_cast(i % 256); - } - - LOG(INFO) << "Writing " << kDataLength << " bytes to Target..."; - auto batch_id = init_ctx.engine_->allocateBatchID(1); - TransferRequest entry; - entry.opcode = TransferRequest::WRITE; - entry.length = kDataLength; - entry.source = init_ctx.local_addr_; - entry.target_id = init_ctx.segment_handle_; - entry.target_offset = init_ctx.remote_base_; - - Status s = init_ctx.engine_->submitTransfer(batch_id, {entry}); - ASSERT_EQ(s, Status::OK()); - - wait_for_transfer(init_ctx.engine_.get(), batch_id, "WRITE"); - - LOG(INFO) << "Phase 1: Write Completed. Tearing down connection..."; - } - - LOG(INFO) << "Simulating Initiator Crash/Restart... Waiting 2 seconds."; - sleep(2); - - // 3. Phase 2: Restart, Re-establish Endpoint & Read - LOG(INFO) << "========== Phase 2: Restart, Re-establish Endpoint & Read " - "=========="; - { - TEContext init_ctx(initiator_server_name, metadata_server, - target_server_name, initiator_device_name); - - LOG(INFO) << "Reading data back over new Endpoint..."; - auto batch_id = init_ctx.engine_->allocateBatchID(1); - TransferRequest entry; - entry.opcode = TransferRequest::READ; - entry.length = kDataLength; - entry.source = init_ctx.local_addr_; - entry.target_id = init_ctx.segment_handle_; - entry.target_offset = init_ctx.remote_base_; - - Status s = init_ctx.engine_->submitTransfer(batch_id, {entry}); - ASSERT_EQ(s, Status::OK()); - - wait_for_transfer(init_ctx.engine_.get(), batch_id, "READ"); - - bool ok = true; - for (size_t i = 0; i < kDataLength; ++i) { - if (init_ctx.local_addr_[i] != static_cast(i % 256)) { - ok = false; - LOG(ERROR) << "Data mismatch at offset " << i << ", expected " - << (i % 256) << ", got " - << (int)init_ctx.local_addr_[i]; - break; - } - } - - ASSERT_TRUE(ok) << "Endpoint Reconstruction Verification Failed!"; - LOG(INFO) - << ">>> ENDPOINT RECONSTRUCTION VERIFICATION: \033[32mSUCCESS\033"; - } + runEndpointReestablishScenario(target_device_name, initiator_device_name); +} + +TEST_F(RDMAEndpointReestablishTest, EndpointReestablishReverseDevices) { + runEndpointReestablishScenario(initiator_device_name, target_device_name); +} + +TEST_F(RDMAEndpointReestablishTest, ActiveHandshakeRetriesAfterAutoGidReprobe) { + configureRtrFaultInjection(initiator_device_name); + runEndpointReestablishScenario(target_device_name, initiator_device_name); + + EXPECT_EQ(getInjectedFailureCount(), 1); + auto sgid_history = getRtrSgidHistory(initiator_device_name); + auto gid_history = getRtrGidHistory(initiator_device_name); + ASSERT_EQ(sgid_history.size(), gid_history.size()); + ASSERT_GE(sgid_history.size(), 2u); + EXPECT_EQ(sgid_history.front(), 0); + EXPECT_FALSE(gid_history.front().empty()); + EXPECT_TRUE(std::any_of( + gid_history.begin() + 1, gid_history.end(), + [&](const std::string& gid) { return gid != gid_history.front(); })); +} + +TEST_F(RDMAEndpointReestablishTest, + PassiveHandshakeRetriesAfterAutoGidReprobe) { + configureRtrFaultInjection(target_device_name); + runEndpointReestablishScenario(target_device_name, initiator_device_name); + + EXPECT_EQ(getInjectedFailureCount(), 1); + auto sgid_history = getRtrSgidHistory(target_device_name); + auto gid_history = getRtrGidHistory(target_device_name); + ASSERT_EQ(sgid_history.size(), gid_history.size()); + ASSERT_GE(sgid_history.size(), 2u); + EXPECT_EQ(sgid_history.front(), 0); + EXPECT_FALSE(gid_history.front().empty()); + EXPECT_TRUE(std::any_of( + gid_history.begin() + 1, gid_history.end(), + [&](const std::string& gid) { return gid != gid_history.front(); })); +} + +} // namespace + +extern "C" int __real__ibv_query_gid_ex(ibv_context* context, uint8_t port_num, + int gid_index, + struct ibv_gid_entry* entry, + uint32_t flags, size_t entry_size); + +extern "C" int __wrap__ibv_query_gid_ex(ibv_context* context, uint8_t port_num, + int gid_index, + struct ibv_gid_entry* entry, + uint32_t flags, size_t entry_size) { + const std::string device_name = ibv_get_device_name(context->device); + int wrapped_gid_index = maybeSwapSyntheticGidIndex(device_name, gid_index); + return __real__ibv_query_gid_ex(context, port_num, wrapped_gid_index, entry, + flags, entry_size); +} + +extern "C" int __real_ibv_query_gid(ibv_context* context, uint8_t port_num, + int gid_index, union ibv_gid* gid); + +extern "C" int __wrap_ibv_query_gid(ibv_context* context, uint8_t port_num, + int gid_index, union ibv_gid* gid) { + const std::string device_name = ibv_get_device_name(context->device); + int wrapped_gid_index = maybeSwapSyntheticGidIndex(device_name, gid_index); + return __real_ibv_query_gid(context, port_num, wrapped_gid_index, gid); +} + +extern "C" int __real_ibv_modify_qp(ibv_qp* qp, ibv_qp_attr* attr, + int attr_mask); + +extern "C" int __wrap_ibv_modify_qp(ibv_qp* qp, ibv_qp_attr* attr, + int attr_mask) { + if (qp != nullptr && attr != nullptr && attr->qp_state == IBV_QPS_RTR && + (attr_mask & IBV_QP_AV)) { + const std::string device_name = + ibv_get_device_name(qp->context->device); + int sgid_index = attr->ah_attr.grh.sgid_index; + union ibv_gid actual_gid = {}; + std::string gid_string; + if (__real_ibv_query_gid(qp->context, attr->ah_attr.port_num, + sgid_index, &actual_gid) == 0) { + gid_string = formatGidBytes(actual_gid.raw); + } + recordRtrAttempt(device_name, sgid_index, gid_string); + if (shouldInjectRtrEinval(device_name, sgid_index)) { + errno = EINVAL; + return -1; + } + } + return __real_ibv_modify_qp(qp, attr, attr_mask); } diff --git a/mooncake-transfer-engine/tests/rdma_gid_probe_test.cpp b/mooncake-transfer-engine/tests/rdma_gid_probe_test.cpp new file mode 100644 index 00000000..740222bc --- /dev/null +++ b/mooncake-transfer-engine/tests/rdma_gid_probe_test.cpp @@ -0,0 +1,402 @@ +// Copyright 2026 KVCache.AI +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include + +#include + +#include "transport/rdma_transport/rdma_gid_probe.h" + +using namespace mooncake; + +namespace { + +AutoGidCandidate makeCandidate(int gid_index, uint32_t gid_type, + bool has_network_device, bool is_ipv4_mapped, + bool is_link_local_ipv6, + bool is_overlay_network = false, + bool is_overlay_ipv4 = false, + bool is_null_gid = false, + bool query_succeeded = true, + std::string gid = "") { + AutoGidCandidate candidate; + candidate.gid_index = gid_index; + candidate.gid = + gid.empty() ? "gid-" + std::to_string(gid_index) : std::move(gid); + candidate.gid_type = gid_type; + candidate.has_network_device = has_network_device; + candidate.is_ipv4_mapped = is_ipv4_mapped; + candidate.is_link_local_ipv6 = is_link_local_ipv6; + candidate.is_overlay_network = is_overlay_network; + candidate.is_overlay_ipv4 = is_overlay_ipv4; + candidate.is_null_gid = is_null_gid; + candidate.query_succeeded = query_succeeded; + return candidate; +} + +TEST(RdmaGidProbeTest, PrefersNetworkBackedRoutableCandidate) { + std::vector candidates = { + makeCandidate(/*gid_index=*/0, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 1); + EXPECT_EQ(selection->candidate_class, + AutoGidCandidateClass::kNetworkRoutable); +} + +TEST(RdmaGidProbeTest, DemotesLinkLocalBehindRoutableNetworkCandidate) { + std::vector candidates = { + makeCandidate(/*gid_index=*/0, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/false, + /*is_link_local_ipv6=*/true), + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 1); + EXPECT_EQ(selection->candidate_class, + AutoGidCandidateClass::kNetworkRoutable); +} + +TEST(RdmaGidProbeTest, DemotesOverlayCandidateBehindNormalNetworkCandidate) { + std::vector candidates = { + makeCandidate(/*gid_index=*/0, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/true), + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 1); +} + +TEST(RdmaGidProbeTest, + PrefersNoNetworkRoutableOverDegradedNetworkBackedCandidate) { + std::vector candidates = { + makeCandidate(/*gid_index=*/0, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/false, + /*is_link_local_ipv6=*/true), + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 1); + EXPECT_EQ(selection->candidate_class, + AutoGidCandidateClass::kNoNetworkRoutable); +} + +TEST(RdmaGidProbeTest, KeepsNoNetworkFallbackAsLastResort) { + std::vector candidates = { + makeCandidate(/*gid_index=*/3, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 3); + EXPECT_EQ(selection->candidate_class, + AutoGidCandidateClass::kNoNetworkRoutable); +} + +TEST(RdmaGidProbeTest, FallsBackToFirstNonzeroCandidateWhenNeeded) { + std::vector candidates = { + makeCandidate(/*gid_index=*/0, IBV_GID_TYPE_ROCE_V1, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/false, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/false, + /*is_overlay_ipv4=*/false, + /*is_null_gid=*/false), + makeCandidate(/*gid_index=*/2, IBV_GID_TYPE_ROCE_V1, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/false, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/false, + /*is_overlay_ipv4=*/false, + /*is_null_gid=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 0); + EXPECT_EQ(selection->candidate_class, + AutoGidCandidateClass::kFallbackNonzero); +} + +TEST(RdmaGidProbeTest, DoesNotTreatIbCandidateAsLinkLocalIpv6Penalty) { + std::vector candidates = { + makeCandidate(/*gid_index=*/0, IBV_GID_TYPE_IB, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/false, + /*is_link_local_ipv6=*/true), + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 0); + EXPECT_EQ(selection->candidate_class, + AutoGidCandidateClass::kNetworkRoutable); +} + +TEST(RdmaGidProbeTest, SkipsInvalidAndNullCandidates) { + std::vector candidates = { + makeCandidate(/*gid_index=*/0, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/false, + /*is_overlay_ipv4=*/false, + /*is_null_gid=*/false, + /*query_succeeded=*/false), + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/false, + /*is_overlay_ipv4=*/false, + /*is_null_gid=*/true), + makeCandidate(/*gid_index=*/2, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 2); +} + +TEST(RdmaGidProbeTest, KeepsStableOrderingWithinSameCandidateClass) { + std::vector candidates = { + makeCandidate(/*gid_index=*/3, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto ranked = rankAutoGidCandidates(candidates); + ASSERT_EQ(ranked.size(), 2u); + EXPECT_EQ(ranked[0].gid_index, 1); + EXPECT_EQ(ranked[1].gid_index, 3); +} + +TEST(RdmaGidProbeTest, ReprobeStillPicksBestCandidateFromFreshSnapshot) { + std::vector candidates = { + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + makeCandidate(/*gid_index=*/3, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = selectBestAutoGidCandidate(candidates); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 1); + EXPECT_EQ(selection->candidate_class, + AutoGidCandidateClass::kNetworkRoutable); +} + +TEST(RdmaGidProbeTest, ReprobeDetectsSameIndexGidRefresh) { + std::vector candidates = { + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/false, + /*is_overlay_ipv4=*/false, + /*is_null_gid=*/false, + /*query_succeeded=*/true, + /*gid=*/"00:11:22"), + makeCandidate(/*gid_index=*/3, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false), + }; + + auto selection = reselectAutoGidCandidate( + candidates, /*current_gid_index=*/1, /*current_gid=*/"00:11:21"); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 1); + EXPECT_EQ(selection->gid, "00:11:22"); + EXPECT_EQ(selection->candidate_class, + AutoGidCandidateClass::kNetworkRoutable); +} + +TEST(RdmaGidProbeTest, ReprobeSkipsRetryWhenBestSelectionDidNotChange) { + std::vector candidates = { + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/false, + /*is_overlay_ipv4=*/false, + /*is_null_gid=*/false, + /*query_succeeded=*/true, + /*gid=*/"00:11:22"), + }; + + auto selection = reselectAutoGidCandidate( + candidates, /*current_gid_index=*/1, /*current_gid=*/"00:11:22"); + EXPECT_FALSE(selection.has_value()); +} + +TEST(RdmaGidProbeTest, ReprobeSkipsAlreadyTriedCandidates) { + std::vector candidates = { + makeCandidate(/*gid_index=*/1, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/true, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/false, + /*is_overlay_ipv4=*/false, + /*is_null_gid=*/false, + /*query_succeeded=*/true, + /*gid=*/"00:11:22"), + makeCandidate(/*gid_index=*/3, IBV_GID_TYPE_ROCE_V2, + /*has_network_device=*/false, + /*is_ipv4_mapped=*/true, + /*is_link_local_ipv6=*/false, + /*is_overlay_network=*/false, + /*is_overlay_ipv4=*/false, + /*is_null_gid=*/false, + /*query_succeeded=*/true, + /*gid=*/"00:11:33"), + }; + + std::vector tried = { + {1, "00:11:22"}, + }; + auto selection = reselectAutoGidCandidate( + candidates, /*current_gid_index=*/1, /*current_gid=*/"00:11:21", tried); + ASSERT_TRUE(selection.has_value()); + EXPECT_EQ(selection->gid_index, 3); + EXPECT_EQ(selection->gid, "00:11:33"); +} + +TEST(RdmaGidProbeTest, DetectsSameIndexGidByteChangesAsSelectionChanges) { + EXPECT_TRUE(didAutoGidSelectionChange(/*previous_gid_index=*/1, + /*previous_gid=*/"00:11:22", + /*current_gid_index=*/1, + /*current_gid=*/"00:11:23")); + + EXPECT_FALSE(didAutoGidSelectionChange(/*previous_gid_index=*/1, + /*previous_gid=*/"00:11:22", + /*current_gid_index=*/1, + /*current_gid=*/"00:11:22")); +} + +TEST(RdmaGidProbeTest, HandshakeRetryRespectsConfiguredRetryBudget) { + EXPECT_TRUE(shouldAttemptAutoGidHandshakeRetry( + /*auto_gid_selection_enabled=*/true, + /*retry_count=*/0, + /*max_retries=*/2, + /*failure_happened_at_rtr=*/true, EINVAL)); + + EXPECT_TRUE(shouldAttemptAutoGidHandshakeRetry( + /*auto_gid_selection_enabled=*/true, + /*retry_count=*/1, + /*max_retries=*/2, + /*failure_happened_at_rtr=*/true, EINVAL)); + + EXPECT_FALSE(shouldAttemptAutoGidHandshakeRetry( + /*auto_gid_selection_enabled=*/false, + /*retry_count=*/0, + /*max_retries=*/2, + /*failure_happened_at_rtr=*/true, EINVAL)); + + EXPECT_FALSE(shouldAttemptAutoGidHandshakeRetry( + /*auto_gid_selection_enabled=*/true, + /*retry_count=*/2, + /*max_retries=*/2, + /*failure_happened_at_rtr=*/true, EINVAL)); + + EXPECT_FALSE(shouldAttemptAutoGidHandshakeRetry( + /*auto_gid_selection_enabled=*/true, + /*retry_count=*/0, + /*max_retries=*/0, + /*failure_happened_at_rtr=*/true, EINVAL)); +} + +TEST(RdmaGidProbeTest, HandshakeRetryOnlyTriggersForRtrEinval) { + EXPECT_FALSE(shouldAttemptAutoGidHandshakeRetry( + /*auto_gid_selection_enabled=*/true, + /*retry_count=*/0, + /*max_retries=*/2, + /*failure_happened_at_rtr=*/false, EINVAL)); + + EXPECT_FALSE(shouldAttemptAutoGidHandshakeRetry( + /*auto_gid_selection_enabled=*/true, + /*retry_count=*/0, + /*max_retries=*/2, + /*failure_happened_at_rtr=*/true, ENOENT)); +} + +TEST(RdmaGidProbeTest, RetryActionRequiresObservedOrReprobedChange) { + EXPECT_EQ(decideAutoGidRetryAction( + /*reprobe_changed=*/false, /*previous_gid_index=*/1, + /*previous_gid=*/"00:11:22", /*current_gid_index=*/1, + /*current_gid=*/"00:11:22"), + AutoGidRetryAction::kDoNotRetry); + + EXPECT_EQ(decideAutoGidRetryAction( + /*reprobe_changed=*/true, /*previous_gid_index=*/1, + /*previous_gid=*/"00:11:22", /*current_gid_index=*/1, + /*current_gid=*/"00:11:23"), + AutoGidRetryAction::kRetryWithReprobedGid); + + EXPECT_EQ(decideAutoGidRetryAction( + /*reprobe_changed=*/false, /*previous_gid_index=*/1, + /*previous_gid=*/"00:11:22", /*current_gid_index=*/1, + /*current_gid=*/"00:11:23"), + AutoGidRetryAction::kRetryWithObservedChange); +} + +} // namespace -- 2.34.1 From c6152c9ca3c8ca0f87b47e893199bd2bd0dca414 Mon Sep 17 00:00:00 2001 From: fatSheep <50252876+00fish0@users.noreply.github.com> Date: Thu, 4 Jun 2026 10:00:08 +0800 Subject: [PATCH 176/382] [Build] Allow building tebench without USE_TENT (#2322) Separate the tebench benchmark from the TENT build so it can be compiled with only the classic Transfer Engine backend. - Gate the benchmark subdirectory solely on BUILD_BENCHMARK instead of also requiring USE_TENT. - Drop tent_backend.cpp from the sources and only link tent_link_group / define USE_TENT when USE_TENT is enabled; otherwise expose just the tent/include header path for the header-only helpers used by the classic backend. - Guard the TENT backend include and runner in main.cpp, returning an error when a TENT-only backend is requested in a non-TENT build. --- mooncake-transfer-engine/CMakeLists.txt | 11 ++++++++--- .../benchmark/CMakeLists.txt | 18 +++++++++++++++++- mooncake-transfer-engine/benchmark/main.cpp | 14 ++++++++++++-- 3 files changed, 37 insertions(+), 6 deletions(-) diff --git a/mooncake-transfer-engine/CMakeLists.txt b/mooncake-transfer-engine/CMakeLists.txt index c5ab0e3c..121c3972 100644 --- a/mooncake-transfer-engine/CMakeLists.txt +++ b/mooncake-transfer-engine/CMakeLists.txt @@ -74,7 +74,12 @@ endif() if (USE_TENT) add_subdirectory(tent) - if (BUILD_BENCHMARK) - add_subdirectory(benchmark) - endif() +endif() + +# tebench only needs the classic Transfer Engine for its default "classic" +# backend; the TENT backend is compiled in conditionally (see benchmark/ +# CMakeLists.txt and main.cpp). Keep it gated solely on BUILD_BENCHMARK so it +# can be built without enabling USE_TENT. +if (BUILD_BENCHMARK) + add_subdirectory(benchmark) endif() diff --git a/mooncake-transfer-engine/benchmark/CMakeLists.txt b/mooncake-transfer-engine/benchmark/CMakeLists.txt index dd222719..e2c80f0d 100644 --- a/mooncake-transfer-engine/benchmark/CMakeLists.txt +++ b/mooncake-transfer-engine/benchmark/CMakeLists.txt @@ -17,8 +17,24 @@ else() endif() file(GLOB TEBENCH_SOURCES "*.cpp") +# The TENT backend is only available when USE_TENT is enabled; drop its +# translation unit (which pulls in tent/ headers) from non-TENT builds. +if(NOT USE_TENT) + list(REMOVE_ITEM TEBENCH_SOURCES "${CMAKE_CURRENT_SOURCE_DIR}/tent_backend.cpp") +endif() add_executable(tebench ${TEBENCH_SOURCES}) -target_link_libraries(tebench PUBLIC transfer_engine tent_link_group) +target_link_libraries(tebench PUBLIC transfer_engine) +if(USE_TENT) + target_compile_definitions(tebench PRIVATE USE_TENT) + # tent_link_group's interface brings in the tent/ header search path. + target_link_libraries(tebench PUBLIC tent_link_group) +else() + # The classic backend still uses a couple of header-only helpers that live + # under tent/include (SimpleRandom in utils.h, bindToSocket in te_backend.cpp). + # Expose just the header path so tebench builds without the TENT library. + target_include_directories(tebench PRIVATE + "${CMAKE_CURRENT_SOURCE_DIR}/../tent/include") +endif() if(USE_CUDA) target_link_libraries(tebench PUBLIC CUDA::cudart) endif() diff --git a/mooncake-transfer-engine/benchmark/main.cpp b/mooncake-transfer-engine/benchmark/main.cpp index bb3b792b..b6ba3878 100644 --- a/mooncake-transfer-engine/benchmark/main.cpp +++ b/mooncake-transfer-engine/benchmark/main.cpp @@ -16,7 +16,9 @@ #include "bench_runner.h" #include "te_backend.h" +#ifdef USE_TENT #include "tent_backend.h" +#endif using namespace mooncake::tent; @@ -101,10 +103,18 @@ int main(int argc, char* argv[]) { gflags::ParseCommandLineFlags(&argc, &argv, true); XferBenchConfig::loadFromFlags(); std::unique_ptr runner; - if (XferBenchConfig::backend == "classic") + if (XferBenchConfig::backend == "classic") { runner = std::make_unique(); - else + } else { +#ifdef USE_TENT runner = std::make_unique(); +#else + LOG(ERROR) << "Backend '" << XferBenchConfig::backend + << "' requires building with -DUSE_TENT=ON; only the " + "'classic' backend is available in this build"; + return EXIT_FAILURE; +#endif + } if (XferBenchConfig::target_seg_name.empty()) { std::cout << "\033[33mTo start initiators, run " << std::endl << " ./tebench --target_seg_name=" -- 2.34.1 From 5dbc1b7088df3169efea72eb15304c19fc19efc3 Mon Sep 17 00:00:00 2001 From: Aoi Date: Thu, 4 Jun 2026 10:54:51 +0800 Subject: [PATCH 177/382] fix(ci): move sccache --show-stats to after build steps (#2303) * fix(ci): move sccache --show-stats to after build steps The "Run sccache stat for check" step was running before any build step, so it always reported 0 cache hits and 0 cache misses. Move it to after the build so it shows actual sccache statistics. * fix(ci): move sccache stat after nvlink_allocator build in first job In the first `build` job, the sccache stats step was placed after `Build project` but before `Build nvlink_allocator.so`, missing the nvlink_allocator compilation from the cache statistics. Move it after `Build nvlink_allocator.so` to match the ordering in the `build-flags` job and capture all compilation steps. --- .github/workflows/ci.yml | 16 ++++++++-------- .github/workflows/ci_cu13.yml | 8 ++++---- .github/workflows/pre-release.yaml | 24 ++++++++++++------------ .github/workflows/release-cuda13.yaml | 8 ++++---- .github/workflows/release-non-cuda.yaml | 8 ++++---- .github/workflows/release.yaml | 8 ++++---- 6 files changed, 36 insertions(+), 36 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8134836c..dccd5e01 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -98,10 +98,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Configure project with coverage support run: | sudo apt update -y @@ -126,6 +122,10 @@ jobs: bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Start Metadata Server run: | cd mooncake-transfer-engine/example/http-metadata-server-python @@ -531,10 +531,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Install dependencies run: | sudo apt update -y @@ -655,6 +651,10 @@ jobs: bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Generate Python version tag id: generate_tag_flags run: | diff --git a/.github/workflows/ci_cu13.yml b/.github/workflows/ci_cu13.yml index a4564bc4..1e6b5e68 100644 --- a/.github/workflows/ci_cu13.yml +++ b/.github/workflows/ci_cu13.yml @@ -52,10 +52,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Install dependencies run: | sudo apt update -y @@ -107,6 +103,10 @@ jobs: bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Generate Python version tag id: generate_tag run: | diff --git a/.github/workflows/pre-release.yaml b/.github/workflows/pre-release.yaml index d4df2abd..718b37f1 100644 --- a/.github/workflows/pre-release.yaml +++ b/.github/workflows/pre-release.yaml @@ -67,10 +67,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Configure project run: | sudo apt update -y @@ -98,6 +94,10 @@ jobs: bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Generate Python version tag id: generate_tag_release run: | @@ -156,10 +156,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Configure project run: | sudo apt update -y @@ -176,6 +172,10 @@ jobs: sudo make install shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Generate Python version tag id: generate_tag_release run: | @@ -246,10 +246,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Configure project run: | sudo apt update -y @@ -277,6 +273,10 @@ jobs: bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Generate Python version tag id: generate_tag_release run: | diff --git a/.github/workflows/release-cuda13.yaml b/.github/workflows/release-cuda13.yaml index 29f23d8b..f7bf7539 100644 --- a/.github/workflows/release-cuda13.yaml +++ b/.github/workflows/release-cuda13.yaml @@ -55,10 +55,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Configure project run: | sudo apt update -y @@ -86,6 +82,10 @@ jobs: bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Generate Python version tag id: generate_tag_release run: | diff --git a/.github/workflows/release-non-cuda.yaml b/.github/workflows/release-non-cuda.yaml index 15b633ec..bed7c71b 100644 --- a/.github/workflows/release-non-cuda.yaml +++ b/.github/workflows/release-non-cuda.yaml @@ -43,10 +43,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Configure project run: | sudo apt update -y @@ -63,6 +59,10 @@ jobs: sudo make install shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Generate Python version tag id: generate_tag_release run: | diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index 1573e8bd..84eea68c 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -56,10 +56,6 @@ jobs: core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || ''); core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); - - name: Run sccache stat for check - shell: bash - run: ${SCCACHE_PATH} --show-stats - - name: Configure project run: | sudo apt update -y @@ -87,6 +83,10 @@ jobs: bash build.sh ../../build/mooncake-transfer-engine/nvlink-allocator/ shell: bash + - name: Run sccache stat for check + shell: bash + run: ${SCCACHE_PATH} --show-stats + - name: Generate Python version tag id: generate_tag_release run: | -- 2.34.1 From f23575169d5952ba4c341ca29fd3e388b51ae98b Mon Sep 17 00:00:00 2001 From: ykwd Date: Thu, 4 Jun 2026 11:01:39 +0800 Subject: [PATCH 178/382] [Docs][1/N] Refactor Readme: update readme top link and badges (#2304) * Update readme top icons * remove duplicate links * update pypi icon * fix format error * fix a link error --------- Co-authored-by: Ke Yang --- README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 925d9e33..f88c0da0 100644 --- a/README.md +++ b/README.md @@ -6,22 +6,22 @@ Paper | Slides | Traces - | Technical Report - | Blog + | Documentation + | Blog | Slack

    - [![Docs](https://img.shields.io/badge/docs-live-brightgreen)](https://kvcache-ai.github.io/Mooncake/) - [![PyPI](https://img.shields.io/pypi/v/mooncake-transfer-engine)](https://pypi.org/project/mooncake-transfer-engine) - [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mooncake-transfer-engine)](https://pypi.org/project/mooncake-transfer-engine) - [![CUDA <=12.9](https://img.shields.io/static/v1?label=CUDA&message=%3C%3D12.9&color=76B900)](https://pypi.org/project/mooncake-transfer-engine) - [![CUDA 13.0/13.1](https://img.shields.io/static/v1?label=CUDA&message=13.0%2F13.1&color=76B900)](https://pypi.org/project/mooncake-transfer-engine-cuda13) - [![PyPI - Downloads](https://img.shields.io/pypi/dm/mooncake-transfer-engine)](https://pypi.org/project/mooncake-transfer-engine) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/kvcache-ai/Mooncake) + [![PyPI - Downloads](https://static.pepy.tech/badge/mooncake-transfer-engine?period=month)](https://pypi.org/project/mooncake-transfer-engine) [![GitHub commit activity](https://img.shields.io/github/commit-activity/w/kvcache-ai/Mooncake)](https://github.com/kvcache-ai/Mooncake/graphs/commit-activity) [![license](https://img.shields.io/github/license/kvcache-ai/mooncake.svg)](https://github.com/kvcache-ai/Mooncake/blob/main/LICENSE-APACHE) +
    + [![PyPI](https://img.shields.io/pypi/v/mooncake-transfer-engine)](https://pypi.org/project/mooncake-transfer-engine) + [![PyPI CUDA <=12.9](https://img.shields.io/static/v1?label=pypi&message=CUDA%20%3C%3D12.9&color=76B900)](https://pypi.org/project/mooncake-transfer-engine) + [![PyPI CUDA 13.0/13.1](https://img.shields.io/static/v1?label=pypi&message=CUDA%2013.0%2F13.1&color=76B900)](https://pypi.org/project/mooncake-transfer-engine-cuda13) + [![PyPI Non-CUDA](https://img.shields.io/static/v1?label=pypi&message=non-CUDA&color=00BFFF)](https://pypi.org/project/mooncake-transfer-engine-non-cuda/)
    -- 2.34.1 From 6c51df326a0bbd502071a42b7505e4dde719d626 Mon Sep 17 00:00:00 2001 From: ykwd Date: Thu, 4 Jun 2026 11:08:04 +0800 Subject: [PATCH 179/382] [Docs] [2/N] Refactor Readme: merges the Showcase and Components sections (#2312) --- README.md | 155 ++++++++++++++++++++++++------------------------------ 1 file changed, 69 insertions(+), 86 deletions(-) diff --git a/README.md b/README.md index f88c0da0..43263ff1 100644 --- a/README.md +++ b/README.md @@ -74,26 +74,84 @@ Mooncake features a KVCache-centric disaggregated architecture that separates th The core of Mooncake is its KVCache-centric scheduler, which balances maximizing overall effective throughput while meeting latency-related Service Level Objectives (SLOs). Unlike traditional studies that assume all requests will be processed, Mooncake faces challenges in highly overloaded scenarios. To mitigate these, we developed a prediction-based early rejection policy. Experiments show that Mooncake excels in long-context scenarios. Compared to the baseline method, Mooncake can achieve up to a 525% increase in throughput in certain simulated scenarios while adhering to SLOs. Under real workloads, Mooncake’s innovative architecture enables Kimi to handle 75% more requests. -

    haHMQL>Yx=-3v=1Qo=R zPc^9TMe|L!w0v8hWBz7ieI&BHwaIwOxDNjMAA5yh>7!B>A1)~x1LZq#^y~7sQ)oJ* zkw^80^EmG73dhPlTeh#(9Fy;2BlZ=>XW0$L1&oqGF^KZtuJarBso*7Q+g!EF@S%9P zPDVuscV2%uboJgbs2z7pK>2R9zBksyWp`1YF!jUx&Pb}mZej>mWTwQ+C;y2!^SIaY zTK&DX@jZx4Ia@aBZJew6`XBc0qvp3o3Ph?#;wuopN{2p^{GBXb^B9lfSVGppHuTs%iox}*w|O{I4%>Ld-<`FTR-GGnPQ1U)nP6Zh1JMwDd2xuW6|gWXpUEL6ALmtppL&wEa2*<<+UNo6(O<>!q4 zGLMG?tQPbE~o?>TNtlZ`u~RAjiSCWf*K=BVIwkXF580 zk$}*j$a&MzJPa0}LC<>m20=5y;koawYFyG0B_N#JXfasnGTe3W^`_RHt$|#(6i{agcrx=}B@1ZMnd1HSBGu}}( z!D~^YVkB1&!neX1M8t;wEqp9#z`-cp)!->oWv3KFHD1K)V6hpV0u~FAwz> zN6|YLqde)e^S9rJ1E6@(O13U$!mo!#GUxBOnIah}kHI3v+6xAT1Me!loG3os_AAQ9 zS$tLha2I#6JF9adN|C)W#(AtwR3kd@v1vzaMqKpnRYssGN}`HTS_$n^$>nB_1y2rU&BN41>dRbPf?vBj2XH+m2Nh~ z1Pgx;)}^^pW1&MW+S}ebK6~~3YH91{fgHdvE3Fr+!ky2>=#_-V%JNP&Wf?rjKhgk+aJ7H zR^2&)?J6LSuzP{lVziEzpC*Y5b906C8c3LBiIYw!LfgUb-NDiaR=>A5EsHG8XOhG0 z8xbFVu)PFB@2{-K)PBm?ie;k@=HTdMLbwcI;0Bbkf>-E>h|s}~fFMAKOku)rP`ejG zl@0zAj4T<>5WNe)J;5B2-hd(daqS)hJwM}?m*iQm0LAAgs80n&9RE*J*~z|M1?+~y zT8w!(#qpb83P|RuFVqq?(?n7y^V3Vyi|IcRnB-X4rc$D&xRw@#^vB+hz6MQS19r|* zwuFLkHSNaB`{C6WkY!{qnh9${Fl(+G>d`m$XO_grOAS5Kxo#qhv8oRaaaHcGPQJQb zjD^Y2&TU%Wn_0v`)E~8!$%;9Tw;orp&79w|gwQzr$1a^b-hBZz*d4iWnT1Y=$4i~m zl8O1D_8xz**io%rC(w|*XBmaUA%mRo+nGw3r_G=xJDpb_eM6LcJiMxh^j4$W@(ij5 z7|5yOlAit6p$FqlKB#NIMstL{+r2Gn#YDogV@QIX_A2I+ot^#POv1!9%5%)qwJfp} zMyu5ZsqzA2zP~Mcj^DD|@lA$lSt-A~&2%E)Mc*$xg{na(Q3w=U3?^7kwIkuaS$01xq&4DkK85qohie&>#Pcq;0~;mm0=ILjSb!6_%{) zFMpNm{iok7ee1|2qki8NkdUeAM`zrXPq3eDYfXOqsIJeLyo=E0-!S0&t8q`MoJYg( zI<8!$@x39<9ut$7W^dnn8F|*w+VPL8OBXL*ig3m`o_R1b=dP0vbt^S{i3qxgy2|U8 zJ&Clw`Yb=fTX&m!r5ob+L|?oy?Z3;^Otba3FVC`OnZ%f;$Z4!e2sHl2A}_5d08AsSnGp; z7~ofT{gUQ8f8L;bx)SLN!Lg0uJol4=SD7(LwB&`yOwYZ z9!vqJOx@7mY;1a!4J>Z0&5#`!o@{GgWqK9EE+^I&BL}iK zm1QliI=k#uLNnF-IsB~*PujlIs_}B%&n$ot(?P#>+ahB&lyR}%LPNlyByM7L&{`Pv6}?b8jDd%=o9Nw z0=nAIFcaS~IW^(u9|h-@l*lNGDGa^0%bZ7fGD3`y({Qy_$o(C)p&2t5Ww)Aht`8m_ zz^DKUC+Al9*Ls2+R{JewQ=kbZPj1VC)=OMfpdO{3NAyr}2d?w@{e9piAvzVZJChAka#5>m&3}pAHrbrs`ezo1N5(x9M!PcZheq=vR{@xOU z*;j*>FkFJerwUc>fWY(Hn$ilB$pzF*W!4+_nvwcNhZZa5Cq3~S7uPIYra^>trPg>H6qH#2Ir?ryk1CV|8Xwm1_oXsXOrbAy3|-_5&9(vRdbQ zHn!5xx+KxECma`-;~E%8R#BUuC*@6q5Vi_PM`T%8h#V#E~_Uf+S5IADdJ6P2r#D zF+m1i4c&UxoDV!1?7q`Q)&c{J%d9^OHrOAC-XScXK4|YJ`iBO>r3oKqBKZ)>gp(gJ zQT-&Cj?-UiXV4p@e?}x?pX*}xZT*^~wi|^ks!q+oPYngQ9$Y#0i7k4??Q-LYr@svc zg06M_T0FE!iun2(xc!z?&0e5dhY;mcFhri35S^di6zoI(Ap8SN@}a3>O-*^z1t_QT zUlJ3?#Hc=r1|L5hlYtxmv}j|^VE#dXJ9A2Ait56Nw_%OHoFrya=NNbBKxp{Fxc zgwqmX2Cag$(2KGf=vC?J_df$#T(7+pOu^M1{@72KU+eGdO;4UTv9SAI1)JSn?RO1S zyQb9BV=iQ&k|*7XTOkgG;~sk5^SWkMg(;BjXP?04XA<~fUPsuCKAT9hc4SX}Y^NeU zAOa~+^D(s5T$|FmW2vV6_5B(>4-#JSG<@B>Sg>B)$N2X({Tt`E5dqEW+@e3)b}jK} zXf4|AuPGN#KC6ejEr_d10xpFD#mR)Ts`Cf?udV+{g$70mZ@Ob%5+6_TckJCY#&GjB zEGK)g-O8uto;X zo4sexO@Z1&5T50g(VBqw^PPq%4@>k+E z)BXRm1WTRP4h6HYoozSD6U$uw+s*FbVT4hIIx-C9_s8ddh+cF1ZmJS)*P)o&Mk2HUnHZpKd)Bui z1L{nR+)S*9L@Hibo=qIeVvzc`TLoom%SatUr(cPnVjYtQzw=b0mz|~rQ|Lt8_WnvC zD6zz5OYP?FhEkoc8?h+5G@wod?U4p$24TbLV6T!|3g*8ZZfKrxfm|N&o+v-jffN)j zaAedl&?ulSqfXA;8oz8gzGT;8p&TKDbYpO@(H^Dc^&n(U5X5~(8z3!<34dUO6ms#3 zqGjLQmfmx)9ND(v*`@WsKFle%Qbc}?EqR_FrFN7|J-1AJ*U1p(_kAfBrW;Rd^Y)6L za@`ij`&NnKtjAZIAn$p#>dCH->nx`x5!k&lLuy^}iVr&4YimF6fuE1s(751op? zY>{T_;_ZJ|(Oq-JmPm5sCQNa>2m{&HR;LaoTrq?GwL7{RbOU1Y7_SEm^@I9zbO5$3}Nivs6?^^d@AD zp?ejY@Z@cA-fZUTpViccPr;9cMLblkoE5e#mJ25;!5G_$hg%e_qx+TU96p%meQtSf zmN2_W9Jb0=UAA-j~$8oTq*(`$nOF z;(DK@-u1S)ZF76h#0!{z#A;Q44~G-SeZy6ZOKM`B^uzO3@bvs1oGsWd6=!z-^?lcpzs^HIUkB zykioe&^nh|*VI3P?{(gn%+ibBSYz&aJ9*B2_QpB-oaKi8yO5&<-)fv z5=zQre)xjG{(w@HkS?jCncjNbp0zsUc6loc@2_n9SO0GnQU0x=4^B9dFYz*-Lf6gy za$x)H7o?76i8pwh?vS-R2mTqfrzetxP<~G@PSd5q!F0%8R>@yKAXnZzOh)m8ZkB2S z_^tBKl8`gTURkVlNw+$+S1IXJ$R3)VPk+cFKvo{VfJPJrtMe$7FV#HEo8N0f-1238Gy7=( zD6{D^hli>Do=^Y4)n00SW7#8l6e=tr8KDinF#`g{PiO~-A z--Pyf;Nw;bl#lVV_d-3PlgPoXQ#Uk^DUBT)kRuZPRePHspZ`o(!Jr#VI6RV1dzi<2 zqlC78m8yg!g^`<5M?M<_X41YGG#Q+rr^B?16r>8^m4RX#w~H32uYvPZj$#V3-Ou7?%`FIVqe{?lDoTMiR}PiD~tOV9|m(+6hPa zXc@dg43gr<3F`V0?WojA;M+eLZa++-i+l>T`#e|~q7yIchMGpiSsdi+wVrhIuUkH> zOV$fKX55tYmGR>9?CBpEIB?jc2QGRMkDdYY^R0&9N3BT5`hYQ;cf?XE_@d}?wU^_d zi_nTpmjJKRCU6Cna_*yF}B`9%ro^)N?`K|Ai+U(yX`9_Pnj=wy4w1&;W^i zLHol`hV?u=-n=reO;k#AG+gvr$W-e5b5*9;UAGZBBD!8I+p-wvC;`13qm#|b{Q2~j z&hF9_5}6mQCeLo!$qOvYk_8bgiZ(qx}IpR8zbZgLISJMpplUgOUT_(!wTfE(F{(<+%IMJ zfvun7xcXKIABK`o)kz@nolw`MxHlGE(!g~{v*73 z;9{l7V-<|q3EhmlONh%UVDI)cDc=ldnAWnRr2#zc-kCQutiSgoKTG8SGR*0j(o1ou zW>IGS@-dXV65fRTD7$5)7r@UV15IWx;aPz!LZptDN_F`WMoW_qyAdjCAbSPZ;T}!v zJ^6T}eZ5I&ddS@gm27m0W#n4k#uIMAbEy5Ar_-S{yeU$tNl97!y@PbL9usa7#3tYF&SP>1sXkc`Ua-So;3je5q^BCCMzD{NB;>lF3vG^f!z>%wu(V!@96P_>RqSF^1 zf6({KkX^>e>p$>CF6eg0EPdRg6DZlV0e!OjrgRA0=C z4q}O;Gk^~896Z_7^^IAeb5K1uulfFT#m5g_Nb8vQH$P6*#CD{vAkNLjBRDR>z)m_U z3DK2~*z9BU9sfIL&~LR6uu&%;Q=N}8OQL)n%` zrCuH^Tjmpr(}{R8`N2047qpJ84@)(g`?fPPp_O&B29a*~j?85?Rj;NbAcME&m|u=p z@cUT~t7&Op16*zs<;_Kk2aR*8VUc!+Jy2=i7(e`S<6_F>Ek@lo*1#TVBb~>=F`>kR zcTc@K3k)G?%m#O5d;PD1kg)^t12(xso-a5{3*-NmM+6- zf{mSFGP!*z+|DIfZEAO?vQ|ynP80dwFslw z1hn^Xst6=*|F=^CASE!5?c1Fc$XKk+9iFOseYyZxH~_2b68NP)S2inFi7_~YbV>49 zI%l{{8^FuohRj)J`_UvL+t+mr?f$jI!QBkc*lnk7t#xmu{CcBzVNeEpT_fuP07}pz zPUb1T3m|B=u<0y-m%VDTq^jf5x4&Aqt*B0!UpL%iHkPM=XY_g>R@_XqT{rLd(!zcI zJ|$zLi{EW1*s4@F)f3?`M*~3DBDG#s8d@JHL0r`kvZ_szy zd!fx$_nG|Ae|(~pq3RF5&6o~7M61c13i*23-c;c1eXykX&g)Ndw1#bh_8>=YqSp&o>RHi9@GUi1_%3I9QlvAeECX8PhpZBNKt>o(_%>gxZ)Z zn)>#9;Lv_BaTC;5d?J2>|MXEXh$|;6!~4I~Z5+FCpw)qR(+}1ZAQO|LLU92rsC6#$ zJ@$BM1auE}`_uo^cyY8$yMHhUhg9a(8Qv;0Nc(_|vsOgx&wtPVHAgcpeq|eCbwEzZ zKwMR#_z=c=U8`#ra#^$&34MTERsQrG8QPhYW1#l1gS_?G;;$jcn+gep2RY`~0*AFA zQ7^W&jrx@O9$tJZUj5c@120so(*C5MQ&_NwQywqRqeGOK{{>LpT8 zwv>}V+J3^{pz7aDiDV2P8%=)I9 zjQz)3ws(d@8?XC^Cl@B0e}$|LZjnA&qGqwD^%^XBqs#FT^O`{Z(6sXWH_5+WMY2dG z(iA6B+j2#e7S_-kv9-weG~f#E=imn=O5Ow2(ceIGdMsBXqM(D)mounvB2Eqe>M7lV zv*M?a*>oQ=y;Q$~{B)aV-zlR0+qe!<+73K5dC%j-g$3fJ<{?%$#iAaj;?wiIX}fV! zl%v<^;4?{0d#{FR?O%O?@u z4gRVRp+WU=9F)NCn+giOShNUnl_+Wz2`+i_Nru!4db_>zD(Gh@L1*?zXb&$TvCPCJ zs_dCE8Aagb)qWCPjE1Hs;|*LKAaO>|S&q*=?}I+I`4NGbY3|>riwHRxF<~y5XJ)23 z>;f)@Z0N`nudfw)v1+obUPW25*VS!0^dLKc`XF&vb}FDe1mDUFeuPQVRHYU)?M@RsJZilFq8#fN zM}b1b()o~m>p&qHyJ|GLPKrY-26l^2buKJY3ZMi^x^$Mm_tE#Bib0GQDrHa!%6yN} z4dUo}emLb~LU! zn1{PlI3}F53ztA|C!g_O*R6js5WisQ*9+i$Olcij1=ntoHnU~vVGPdwT-@cYy)^mm z^bJENK1ZV=Ki9j$U;w0Jhd-0pDzO5j$}_rxAUOLz}rg7FH}p%nxS1uQ=7>rJcBw|2I4ULm;)#w|*SpS%-ko z=e`=i--tOY(cb+(3t-U!U?;-69E>d@uR3+lg1>kHOpGb78aEcXTL8yWYYi$6fpDAhXd`mr;o@-v9Voach`IG9CO%H5 zp1uAO{0NsZ62V0<#-BUqB4aOu!(O5GtUjGQo7X`wq&@FGo?)vK!9$y`CWErBJfghOUq4Y$ zT(>a582*l-zP>E6yl%ql*$rPGk0~1hm=KAbs5X|iV9=xZIlz8IX#C5e z|F4f=nYrIlv5BecAo_1y5G_6b?7^vyD9TF9nd(Faxv0UP)t4&dGUfS)uVh$xUft*m z@5LBXh&NNi`p zs!f{PR2cnXFt>uQ=_fS5o&ct}2v+Dir6R_uX77;r7K_p04J|=66nvgKmJg8o-f;-d z?RzlWAofe?FllY@+bF8=FAQL%KcF|5SA(k&H_|ynILX&pfM)yi2f}~6~e{*ejdB;yc7?GEX7J0x$nQutaARrr7c1+Ze*$QK4n2;=N5)dO8us> zplH{v_Ya50N3kzx&jQc&J^#eod2H;0)0=&v$Z}t=oDSXT$K}Rc##2j{jWkR$)_^i# z7@wSTI=2Zlx^1Aa_Wb=@}s+m z2%n9<(;=^Tc{|Dm>yZMpZlmk><1y&>h5gl|nT9xZDb(fu{r#!2@4q<6Ba9#Dp}Os6 zh3tMp|6{gEf5Fvew>|0caNoc~1d^wxOQ>sLeojKXuE3T)x za9&dC+sl|&BgEv%ed3c?Zz+UwXp}yLsgiRySHIWwpN3Q2^cuK90OUQ}SwRkw@xBw- z$XrO-ZFRyU{c)T$YQC4&j>+Ck?`eslf^Vmq8vSl2W*O9`{u%r4nc?`}m(haTk8(fZ zy>kHu!;Yqg!(*>PIW|7QR)S)sEJm?!L{+_7&8m&gV>m~K+R*39RK_A!L033LhROA9 z?$mSjH!Jk;*23@AbtaQ56H&x3snqnp7c{}p-8zQ$3f6QOR@E2kNR4PHO8q|_p7@X4 zhm6lg~jHza~#cYQ9}ajon1#KSy69x30Vo-OI+| z&U0PK^x`$SgUq?y!X}QumZga?UuvC1oO37RGFbL4{gT2WXTX{%?XSwX#b?&uM;<$_ zVpaqZ?vLiyr(Lop%AP<*`CE%*)yL@-c_K|Y!ru$FOZVt3#t^pMU8*?uk7wGx*SM2J z&tobYJ0rvBh1LNCAOoX0aI?W@`54Q!*%IO3{a$N2LX5ZPa?V8RtONVB{gBUwbA2c0 zF*eR{ro_do(Ow4MvlQ9;Q=n03I0QZx$QJe)O`zm6VU=FdbF|j4?PF*G&_HIFf}Uq< z@hl(P&kwjA+iqvEZ*0@gw~_!Bq@~^mG{1xKpSh1W8J>OgoxerS%$o1-ZlK3?y6K8+ zKx%hI%aQk$s4jFz7#3k*GNUUW!hd(vatZ+$+PzQlkP>MD*f^uW(t1!^3{WeNjG^sO ziP{wv8gr#;r@p>&LtkVLF%GdXQQXo*a@pLpdRiI~rfJOBEvB8k>ae24 zhxq1<{AM2G`WBL7Xtc7;eL9L{W!x4-eAnX;ueYkaHb=xq-`q#vowi^?SJ5dgFFG}G z%M5b2fHOeff3)ocbXA{ieeOD+;v=KiQHPZ+cPA~3Kpr%oe8;<33Vt=2+i5`1d$CkA zwt>GbO`m8;-((-~^6u~J)Rpgd-G>$M*dYy(A2lH$ECD%=4xA;c9*Xon25v0t?(kF- z@7wZp1tt}BrfszLw;J{R#={f%ncaBP#m>ZEe{&px5PrSJA7K7w08dmLcrk9Cs*(k` zKVfM{b#vKdr)gIFT%O(AfpaE4gbs0P*{ws3#+3ji5Fji+5@Ce&^KOJGzg+q(a&epv z`2E))K$CU*1*y|l1<1%x;jp2HXsIee%7bWS)f!8F2Wu|??bP(+Cc^JJfc?J?r?dT$ zz!XXlF7noGZuNFo7JyOxG7lpf0cbkXcS|n7UpP(bV@%aLdY6&Wc=mo?%?ENI*hlAc zW9;E#9uY^kSLAWJLTc;^i-R>Rx`$DO^4-iDIxc^|vo}hL|H?Rh0VdaM|95;qk2|X_ zOleJ*ZYRtrP_6r^W#6|kl<4U1)xIuWsz2o_QKWKoXeEu=<};G^%d1d@%M2Fn@L}BU zB)9_3WeAl?Z4afPL)(a0y3PEPHOuAU7$jf*7d6;{n_F1!^Zdaz zmCEjq`3@uM324e_7T5PkxIN;NB&8&!MLCG83XE7-m<3{fEuF>|4_H2y!BVd1GfnLRxpO8B`8;M&$iQ1ANUPkv+B%>E%eaP zw$k{&xeDWUrd}LmMc`Kw+Kad(T9$N@@&2t-XJD?W|a_k#=* zZiC{WnW3WjnzgIyEFVpl%Kj-(iAuSi379{2K|lLpT%W)E?LlI5Y7>e)BL$$k*3-Y& zANxoKK;Or_`4jVO#O-z=S#Ab_VLuk<50nGPP z8nR`xKNX1M?x+6!&*d85_F8TbnBN4wee}w^rv|Ie1oNI z!ypEAL#3QrK3|in791WYRE9#A_0@m)_M2;FJ6Bjxs@47OFlAr3-KgI6pW+?hd`hT& ziW!Hpkfi^%dT;Xi$Tt#)N_Djgvj1n>NGVT4b^17aIe+o1A>m-vr%q>Q*2Dzo3zcOY zLT|ZBymT}F%!h5ev-tPI4-MI%d!{3kWE(Sk*V+w}?DrK5>B(LAOtI0`cr+|mQ9rcn zrHnux4}m#5ZIyD>9rHwP6zh)&N!5Ar!Otx!ulGC!h(CQQMl0pEAJ_&4wD!?5B!=cF zD<}l=v3pCPXJL(bL)e^jWIpS+ujU{Bq7Ts4cbcpsb~-{MdoK6)x*2+S*57_RC}7|$ z)X?KF-PmB8Da`)toQ&mqC7Bvh8q84&aVMspL zUu1$-C#z#gO-)KR%o-f=5Yy$*?u@I5UN^H4ypF+5rKU68pI_eW_aUDz)iYUnZkQ!~ zD?4ku-)V#pF?}mPdV*a79zjLMQ1Tj9TRaZnq5Nj2Y@COOubm_{mC%JMc;<)aO~}QP zpNFF6T_emi)nIi_R}N-GejL^=^p3G{?jDGqDV=3f5y7M%TA~htoRRaV2m9eJj<>*- z?qs4bdO^SUp+OW4 zsRWg4uReBth*6{ycUanBY!53;dB4$xU>pUg)8chD%P0Uz^8U0?o~zFzx^fC7mK?ZPNy9*4!k5z9pO?Gad|2h4;|_$26i z0?}dW$nU}4);l8q%oy*R{mp&4Fe&$=-+f((q>nhDgnGUh2ey?xFW?y^I{BwO54XE8 zVnzupUA^(wy-)$jNv78m5}9|(seJdmXY(iQFo$*A?79F}B07f1XkVCUFlt3Q4yr8woZ?apM{PmSq#zWqEknUF>t%ilzxIY4N! zG9xK^2WZCKFz!L*#aVA7DT{PXTO`|lWJ-$Ldmkt3e%Y_q%3X~^Wc@3)Re&0FN>8tdBB{908^i=U^u=wzbnt)TzR88H`|wRN`N$sYpX zMpl?`8C}-4h$k|f@1?lXV$zgn9hRFh=R;%gDYI(F)6Nk5m!tA`%dPu!x(NH(YoE)# z38GYE9FP;0n6*;aGe)+c<6f1YD>S#1%dg4UAmpKKl2TtHZSr|=n90;c99J?g|JCeP zjY>bz3YPf4M~~6y7nL_xoG(9`HAN-c4(Ik{J}Q5+rG1E26Vsd z4%R+l3W@Az4eEyZ+Zq-{L-QMGnu0|zF)o_XYhMi|+c~z+xJ#UK$H+HuHB`O1Uz821 z!aSnequJ^d24}bWzg2#I_xH(m~$@P70zOxVmHCWM^+bz{2@G4pht%E4rN|=-USoRK3I`f{^smg5<8_TRx=a)O&foeQJy*}Es%z8lYb zahp?BcNxF_x$Q%+V5(?~Q&P1U|0a|}mV@okHMbZ{<@HdaPjqSWf zZ)$D+RZKRyWr#W60KjM^8rGI=taQAMz(SoY)b5<4Mgt_#`|79k)il$S6{w?KV&RDG zoc(mU;h3RgOO0P&>A`f{S|oRiBorM-5YO_nHf4cZ3heW)tMH1(WPUrt0SFesP-Im~ zU;^28jh+5$8RDa257uiG4-yvd;}Cn8zh1(Z&k(VNsLo`h*4lX-9^be&75XI8xYJT=87a=xxjxJm>pJGZl|vdthmSd#bT|ox!p4qVc~b+Xjo& z+FVPlm>oOL1U}i(L#&>r(w|S?nou7=y4+f?hud1zhu8jcHYnQkzeuq*J$Y?1KH={w6LJ{51|8(po#8OE7FzaG~1+r;Vv(LK`etSj2m1s#W??T11#hk2##*Qp= zi`**gPDLX(uge#&&K}Q1GT~)?66Diz5c5?UlsVz+!aX2f;P>;qZE;mRUJ}&_r%~+I zn2~)SZ`5b{8tivfqz{f@9vCrXt!U=b;ZQ@rNpEgJ77jL1NP6%Yj+$f$M4dGuhzJQh zzcsbL_&)#>LF~R6uNPZ8fAH<*U3s!ki`tDUSnjbMJARVk^=lHQRhe>$+X@GKh!D}0 zk~gnLb{KyXjpgpzfmr4qlx&tE{RITaO%0%v}^er&7FX1r80sQdZR~ zK4K`<_A6oYKPntw$)SkERK|o)dHrlzZk*$9+qR9T*}%4H)si)f8ZQz_*UySNCD>ph zw;3fNetKHk8?V1kcHtsLxaSuK{R4=)0m^*odQXGz{LigsUxA4wep$6rf|Sg9{d!y-T)zDNp_~{a?MIFrLH`AJWnaylB?$BXs8OSUVOYxKNt1Zz#qkRk zD)it(58ZR`J-BH7hV@gYPWAY4{$oauCXwy#-MY>EawhNE@VQ^&r%ajBef#tk;gY5G zWZqoJWESG2KUO*TXN^zCx3vnpe|%T#aG`$hzU%H$Z;AP(m$sWUX;OiF`N8M)hjP}k z`0NF;Iot2PefvK7^ixXl;z@<_gQ9F`J;j< z9x!O+h>=e|`Q+K&5hn~BUsa=~siXSlUpF%a3{zTGtx|;s$_J_& zH)*n9{(LGCMFcJLT(E(pj)959v(G%ccI{e9{^P}epsV6J70KoBP_=RuI=Sw<_gNE8a^_2uH1zRoPmIF&|p8R zv$$Rad;Ezf#=Q3)*Jg?_zK7MM$^3cq3Kc4dWv{;W8V6*=koERU1R}=R_DPdIt5drU zNYpPwLLPqT;d{)V^a_1GWvaZN6hC}?9Le2#cF$ihzuOHreEP{JojP5~n>*F2SH~5O zj}kpRvBipuB~1WzbT)0-vUR|K0b|F$f9|=}D8OV`O`0^O=dNJE0+|2$8*lo1l#rw> zZW=RYw7RRpuy1N%k43zf)Vp`@MT-`dE?sKNmMsXNml1_%iOyegj24bjtq}P7n{N&u zJ|Y)b-8p{XME^f283+XJE2t35K4ty7f877V3*$!=*$pS{Vw+Z^ zh9pxIloC~{ge-cjA6Xtp5*w#YbH593N1xfLQ-!2 zcmAY1X?aGaWPsztM5VIwH+e@3%{EU-(#rCi?0==30Qg?N;Dan0dNBCwJMWHye3f&n zE?>Ul_uaoU%BxqeUMdsDPvjpwqI6gZ!FaQ8ze*&9A6-%C*B9mD8}kq9%pMid;uP?Q zZ6P1=H_yobE{G@_(vFbfNB)-$r@{AH1(agA-82UNcmK%I!J| z$fX$1BK7oANE8uc&jDoIN=lm(3e{tx*;gsBY z82rKUAy?i!si`Ty{kHw-!9&_!+R;>K<;NeMG#@&0>>%G?J#zTS(Stnd8Sv=iPY-&5 z+g%%`kgAa*xpFbT#j3Sy8#Zbxwk%w5=*%U(JWi=nt0Q8vxSJ*-+sj6rlc*H@uW2eY!~z@{ zr24h2~)r|8vaHv4vp4&EWL?8CW?!NovS>*-7Z9x8_yDq(~PV?re;eZ8s7-U`V z!6m3`QDKp5%!#Kz|B96?2u#rVSB0o;pIkuZd4tV){zVYwGR-*%MB&Dgt8T3nZr33o ze8yzwjWw>DF@wnngzb?L-?JxPxUeZxnUwv9W7Fp5YE(a?V!5bi%K7_+VRZd~aJJ)T zQ0Ynjhnlxv5P}0vc>V0cZY7mT{cSWCbMXUzhcJuaI99R+NSI(W$qG_}teou^0k2aV}nm(lSEYf{WJv56mjM5j4+^y}AG zi3#QrKKkO27ujZdi#~n&D3Ma(^-0A_m2T*E!vzf*lqyx4bYmT#GIi?o8F*|gV+7oG zWyzfQop&B#s_A%6MdU>3-_lx35eH@!70csp1XYx;1Ffkl8W4R#lri z^~f_-DplczoQ>r6Tda8To;`bzzjxn0?k1cA%vnZ{8I5q=dUfu&^N!5+i*9qmz(*d# z5JK^}=bl9jc~c@6k!XCCsC1<<{6T?Krc7C=B)4Di_w3!XcmLi89{4-PB9EB;Q7m4i zYSjS)9?I5!jdM>V>2KP+Ws6(q^*ed;hE&lJ8KZTx_mQFg30l|;Y z%wUQMXeDl6d*h8fbkQna6e_mJG&V>i)s6LxR29_b+2@@7=)i%Z9W9AC)42BP-J25o z_U+xgd9(Y6;v9pYfBtzGK=P%RU*Gj1wZ-tDQ31fX?ezEU>CJt z2GDh^Vv(U*BPK~SV3nRNuvk=?Dl`VUq#7G9uA-;Gw{XbB zH$W>ZJ!UbXEJIDbDRg9Cd+}s^N;xUk&R;3^505j;bmdBwUmP+dD57>EZjPUgKj}I7 zD~-kcBMu5@^Ni`xw&`_Gp{`P;%Fn;>Ld2iq5q~yx=(2s%t-8Wc$bp~J{~=VS|2?7% zFYtd({}2k&X!qD^#vy;lcm6^E-7U>pzX&)!8;ch&K6>;xvQIww1cyJ$$l*!;Nyhg& zpcDcTJ{yRu$m#1B8Luq&PlT`Z)A(OjAeV!G8nqh~*6wNT7hE94CAa{#A}AIfI)k9ZneX<`v~`NsmgA{QCKp~U!m{7Wq4 zPjN?Z?N&51ezL{Cm6J(stotV${-2DWKd9eq_|PN8+n4iKiv80O(-vr7-L>nNEHh{3 z&TX?*Tl{3>FH@~ZGXG?JnIFbK8q8-ZAVn(duC^)*OXmL{Y`=~#J1i`-#sA!I7SH?U`?=pNkyuaz zw&}D({@LOa3BwDI5@CEI{(r6gVvZQw)$M88uT`g=df|&p{oLwg6?NFi{hp$LZimon zJHCaTP&)bj^2^Fr?XIA-B=``hqJjTWAjNa1rf%4@nei9uF@X|*#){V_z^53~IHYgf zF!7)NG&Sgl4Ot2n6OZlPf2eKyE7;~fUl&Ii89R3Tc1!=;Mb)&CrQVaHM=8kdiF!8p zpAx_ErXJgO?9dlrSm4A!*?oH(U)cQMfrHW#-j-Umupf)(){T4q{CQu0W7rcK&FgDJE)I6{S24^zFFkEC}wChpIyN#`w#5x+Nl%rXi*_F>qM>> zpxpH$jlrHJEt%(c)khM(%y0*ioJi!YRy98G(Ud$~P}5aJU!E|qs`QTSd!BfLZaHL9 z%auL(^wV7S9gF8$S*Lp0PMu1ZE}eY*1m6##DlB-@M?|w_Hp*>*QqNKe7Wv0TY(vEX zQgLPm4z8~#>-?1|&k9PC!p9wGP>g6F5CdQ@UZwhRZr4pX@PG4*G8Qh19cP9TEit|% zv3gaCrp*fcbMnWZa_u>iXi_^RPaajS`Nz*+elVs~s2T-|wPgHk=f{AP2#G%hSyUq0 zz*HqcJi!gT$5_dd`r3$+%%o&|rNv@9e%r=5;_I&$EnN5wV<=vE`IVb{-pu4*OO`A_ znE4kU9r!4yNXipzKvSFSIA04+rO3U>_+F}DA?|)v{Mb=?zuY%)6b6e!kj{oO4L>by z-MaL3$n@*qpU3rjR%b4JgO?zx(cJ_?0bNwnK-@RU{=&xBJ%ZTfx8Ww%Y_?{FCGJ-FM#p=9_OO zPMjbVivy;8ZPTV*D<07&;VU=BV4SlU;sNrejhk3|d{V#6W<{>6V8i_%w%^3tZ@*1U zel&3+_~gPNxW486k^ArEJx~%o-K&i4i<(_*5tQKDF%~Q;#e^(AvY=P*zKm0FVfVj` zjm6F^T&Qc8E^IGf_6zSKW#b>kj|dogjamw%B|rYK_1CSIXigTD6a|ud6%;!q77pP zzr@ML4TOD4pWb~+mMG!=36F2G1^oW~`x6^v`L#{5eW!lK$vi$J`CkYjzFa9#G|Sn^_#$JSM!IbB z+Ea=x{Q9=lqg=;{Ur9>!wY;zX8GG)9>p3RrI)r4*Hi98kCO+x~pt?~e6|8;ffTdN1 z_LStb8LX*vlL(LvpQ2XK1Ouz2*YTAa^3RDMoRZ=R-b~v_^WD5j(kOLN3aSnQe50Dl z-;e|TuFDI_4IzJXf$GJ}f@_w>m;SVr_Z7En-f}v8QLyS)+-K!)+^iBs6DjIano-Js z$2Sg{{wKri5(~*&Ip=R{d6_)tZ<(rJNh>FeD>>hB{MXh((^ZS%hyzQ|ZWFNS?m?ZIc|K!BCvT}?um&(XLr{sUi`Zc@VCc&7~q#j|% zcY$R8dwhoORaq3vc&mQLiuhZ)aWlbWe3hb%#f-cKLU6uEDu{pTS2C6z;gf=a+1l?< zKmA1SHNE4?H9GKDoCJjICrA9a|FyRI<(J|@NWyo{mZ+@%R{Qn%)CoSReaTX*U(*A; zYS0#QLKvT_&ai#q01-&ykGvIEIWm&*C7HzEkYyKwuFLe<4N{07#i!JWKoFnsr+7C0 z2t@eWvi7S3%f`U}j(@uN!6?;>Jn0-vx%r=rMok*sc+-vK4t{pCer)`A-P@ z_8;oo?^cQkA2ho+)$J;1v5fD&{UKL8_1KXkRjXEM+opBV62?Uf~ICs>DpP9HG@+|kNJSfN1&j$2o(c%cvACW7)X58>UBKM=PZ3- zKXZ&6+P8ZjqgoI{kXJ3F+{ofLH8(qyVX_*j^@HGl0&npTOUUK8pa9E=7cW+fkTasv zM^RHvaZB@lC%PASu}I)rtZkYTC1hfDELBtsflDs6FKy^^n;HorIT9RS2T*EfW|fKg`4>*I@vTwsXc82oc4N02@)pSFFwn6YbDm&`ztBaRMGluMOsmL| zjntBUd2ie{GBOfA9RebLTF~^S%b-&m(-Z)L^yJLdvvh z)A-gy^Gg`qcyX7mUHwn$k;@Bnp{ePbvI=3j|n5K*%Ry3sbi|MpMU*TGAR{{ zAU-c{b}>8q_U*sql8Z4gm`lqPyMIK%>*!S1&%gh^Ywemfd@YFXk|KqRAj&Hk7Cbsk zt>2g~P0X;+yahkdBdVM#8q6(;f(7!Qf8P1-Mj6T<{^uvDe#KZhLHw9=94GLPoZ|SV z04E{y=%bJFSzk(BarqVh{P>@|J`>?vJ*QNuT!k8@J(WnX+ZXG9KT$g;$*9MRTLFL#QDBf8Vuh^{UlO3WGn37Udil*Ly}@ zSVtE-K0A0K*1W|fqMmgB8?;}ciky4bG|Na&L_6QL5oE|e!q>N}$&M$sY-I!r5XwnU zPcs1~7HMvEL^0J6ezJ5aUq#zbjn|`8!4XC0*}{2q9ch>@2~P z=dHXB>+*tZ40u=xE31-7>Ww8NR7P|tEJ_qR6~1x}-&Eia%KQ<&SQxS&YK1Q)PSNa}aw@E~w}lTpRXZ|*qjO|;6Ugp!i@2QE{F0SZlJwUB5W0+?icNy#%VXkL^2f5G47 zLwxY$Cy~*gKqCJ;e})l~Lu!eVbP!tYt3opQ|CjZP=qdf5jsNNE*NhC~(>4Dwf0N1X ze={|Uf2f?S_3NHAXC(PQ*+1qdA;|^ie=oy2;vdBi_`&sp>WDw;D>#%me}Dv~s}wiO ze=Y8CSr9);WC5S|8)blrvyp?pGE||`lo>zf{{SC-9y6?S;)`DpjQ`#G6~1aYDSpJl zEwhk{{PSP7Un7}~|9`IiiWC1W?#*3M`-=FxK|8)E1pryto&hAm;9i(t0%B?T9u<#9SSw$<;wuC9CK&7w5dO4uZd>tI?4ME?~ zP>WYqBKG#^_ia~DWVCDBV%VFnN-O!X*zJG4t=(lEH*fuolOzwc=FVRT22I?%6zCRV zl1C0Pn9lbvn6zT%jH%~Vts;?F?3-^FUw^|*dMPF)#yb`Kn>PK+_LsG_b53mc?|T$B z)}CtCtTFkMi2=P>Z0zW_d-dth3oKmY@l0Xo&fl2SjS2Sg|F{n)?%uVF`yOsj;eYVJ z;h8f&uUTEMB4hEBk3D+B%{{+cyx2|Sy93u;f8)jMySq)j%#{h$Il!6Y1bk&+%i<`xFg@B!V_L4#jl^eon&;EWwlRIgr@ z;$-=Xw_2n`Uc_|sn)}84Pfv*uO$icmAni`nv_%&9?6509JAQJ&;~t(a3NEN%D@t|@ zmBzp2vak zRfSFobzIAufSkm{``2{s^2!@Sk>$Oox85ByWH6($!IV*=niBu$6K-4uaO~KL9yfO% z|KZ00o{i0MT95z*2Q!_|k-*jK4BS#^DAcJ|P>?l1x;} zMIB45L;m{gb1G7j^hG%#zBDTwmBW^Jn9uYhDO+)Sol>FviRH_4HEbAM|CBDCQmJC% z`=4TUHJ{?~Y18st+RkNRoH%qaG4tDakKTz=#o}w!b!U}9ezf4~h<}ji$G4D!{hGC^ z4XB0w5#en7t-b;q5{ccv@1n4G{IQX#(nsS8EbsuY>lZwOZU_xq%9YBH_0na_mTA$l z#r4l)7Ymm307Rn1ceB z1XDRh@EuVYAz1^&_XGoK@2qVXS>ELoeDws-FyP_We6hin&vkpKGh4PLy$ z+2z>~$-vUUKVDzFBLHD?gZMG6b{z)ZAJi}MVSE}*_D!0|NDO-B*=Gk2cJ>V6q{BkF zR=*@F-QuTrpFX^?$c&Ryr+(hGD+3wC7yhG1zh@iq-hKOsn`p<@)7fF=hy+Q2X7VKF ztY61B{)wf_ejGY<#E(BNS(mnM&u+c~Ys?K37@e>}{u?)}ci3{}tc%t&q5@XG$^6|v zq~P^Rl`FYrk^kL43}S*I630(i$FV9Xwtd?US&&FD)e9fv6|f3(V%9X*LP#XmrKd|& zy*%qHRjg#15 zpA#crPK5*C0Fsa`rj)YQdaN*;)C-puTwwgZYiB5EWRv(Cz6wZ-3^yY(kDq~tsQsF! z0MqrAbCZAMG->d4vVS6c%gw|`WmHN{i;_A!2DWJ^FN8?(iua2RZ6hOf@hzKH zQUQk;udwoQET&cCmEyd4^3^*}pKgxu#o59s;=g9K=6|BJ)?oVDj%8wEKGC&p#>w>b z4O_SUdgSoY{Q2|o5Q8~s+&^TpP?BpPj1o`jutcv)*;_zc;G<&3og)5Ra@LYtfTtX(?9|< z&UjCImB8v(F=J#vlVnQDvEwHgJ*Z)6eEqFfP3svEB{aeTY~QxEwt?kAApXJRRjXEW z$F_|M89~I;E zNY$+<7-&-aGH;WqC1UPAiTG1!`}Q5_>1j+iP_EorRV!EJJ_VjEQkJ9r@^Hy&16{g# zK#Ymm)6&!MKa)-|FB+|e^r(KFzY|!qhHlNk|D-5UH#WiX!6cda?g-~vzR-5yz(FQ& z1Jn6SSSDH_RZ2}JJfAXU>I*}L^ytB(8}6*EYD8J_?}E->{z-}-^N87JMa*jk9C*p9 zGCV%JQ^9n6uV3&(GU|}i<%J!B1)A0WqL3i6Gh@`5FT?T$A4p<9@JK=~w;p5I!&WkVo1IM4_#GH{)q-bG!Iw))f z_1K?K`Rtz`)oXBJ-u!wMoMq6ng9i?Hh#P4==cd%j;JtP3l;_Q!dDdBF<$vSSq;bQ? z9v$%P^DlD4jTDa*{ylx#rR`b^$G&Nzjf5t$=y+*6)l0arIdb^WVN7?Y^FCa8u|@9# zv%LEHPI=zxtuCuUBcQmaNaUdbU#0Ev_1-g~#nh0XWu--`%IhYucNq*#Rt<+&yx zn;<~St>+VW-!7P2Ubi&5isFb=sE|GWrAq?cF`(;Q3Z-75D_ys=B zmnUz2q)xjvUI`1w@-u)EUg0m3~~`+wGVnN ztY<-WN0$m+T5uBQ4pI7Z1G5kQq=gufoI$pZ;MNd5DlO39R zSr@b*aeBn^xw;T!vUEd}O6ilg6<-~&TbJJ;9yhz4xJ2Fa?P#u{7Uu(>r z>x|fCP0=VUAd?nUlB{?$#X@6S#$~^L_=EGWlctD(<-uSfMP|q11?=k#QWaPl;A0F9 z(jg|6VE%=7-g(E5)gVyJ`D-iS=8e#_1RDHsjjIFx$@qxK<&G2AI}vJ)v3R-i<;+9_ zbxpSE{2iTbwC~!rQ#4}nQf2J9cR(t;f6gjXRz~MI|H{?!h2n?B<0nt<)DI_2cy93E z;lqb>F6Kl$efkXk-EqgAUAkQT>T55TDQ)8doWRcCyLG?et68%IV?JSe2nJ7a6_GD* zUOof^zAm~#{ycSexsoMIxj9xS`}y6vdsGER`G0i&_4;M9rYD|!0wT`y*8aDC^2w+2 ziPf*ysqv3@>D=Y6!gnz@#MrU#)7hZ{Iw+lBmJQ^2S*c3p%91qmfBltt8L%x@f>~f~ z!`F%`6R_Vb=U*=JyLazCYt}3zec`=JdXb71E6&C3ci(<1R<_9cSZvF$Oj@ZR;-H_RITRx-5>!%gT=mJof2!a zYs=Q{hNK3L6Qkpg@=cNzr{fPl98Vw(9X9;8ojbT&oX#?1+8uY?b@kO<`4)^_jOqAb z{Kc4}`1Zz#Cl5Ptn_`YxV8*iHR)t5$5Omn5?X0qGj#fx*mT>RZ) zgmoTeG5`3n&Q3i_Eo;Fnr-a8(KR$wiv^;*^LHzy_JwejT47q@)``l(Zz@$hhnaHe(!T7n%UZT-K~$Yzzy7?r^CTw@qQ>95@4m~l%=AV5 z{L3$-3>`i^e}Vi33Kr6c!2$(1>LKa;nH`Yt1D#XxoSHSNwQSL{L;K4Zl~=5I(fakz zpErM=gIa*e`w_nDiu^Nr^ccry$|(2`ee=x%1qza)|M~ONQ)}`u+8D#+Y}8qN`t)fv zYu9Arr8aHbwrtrdEj>N*KObSn(z7d8tX`vf^A;^Gy|jI+R;`N_FLr+Y3+B$7D=5*@ z;wI1_o_qeeLIv}8=+FTN@EWIPY_3q@;^r5(ZQHgk_snOUdE-qtF=8zn{~bGjyX&qy zixeqVr*7R=ty^7jWhYK`Ox4RVf=(s)IR1nWKP*s?fBBvr^en1*RE1p(d?KlL@7@Y5 zJ1iiNr4K#$(AgC#Fy~F{Hf_M98?{1(bMCq4UcM}4A!rq4o&=oFlHs3*4H{s}^l4wz zs#WX!^Xl;y9QDgxVqj;8{|EkeyHx%0*loAo3bpgkukZe0?MD!@SXj@PF~j-)u;hmz z$@mW*JeYzF8#F|anZvr@cw?Ef$~J7&h_@V@HM_WQk-~NA*5jL1Dv}hRu!`_4wjU+7 zi5nqSO|a7`|}@bSkU=Y;1WDh`4WS+#FF6L53>@PLOaRIJ4G(XCpv;itS*v9$ zqPrEB0_Rkyz~o6sjvR$K3$n@h?j@s_N%WS!w}8*>y8AhqKfZYX{jszxC~ea0VnO|a z;P{-m3!ix=q4L7>gH=C4EyQ=jZNS6aLDi^PlkjfYvL&H%&e>=4t$R9;jez`PF!I%i zkt4kQ9^x~xVf;k=4;?;q$L)V%HuCc2&SC^?o3^cqz-l$Bz4_*w6#nef&!ETh+m0PV z<}dWW(dC?%Yw5D3?K^ZN{+nNX2@yzl=NV_7annsV@-4Sy|BQb39p-t$oNWFFKiU6W zRWQAIkzz&FvRbvd(puJqO)nb!++aRV;RN8|E&`un$ppfh)obQi12=l!szmg^JpN?5 z)uF9gqxS3k(I)qYy0zikAwI`wJBvlhS^V!V14J81{>Oxz_<^FV_(;0?F!>btPR|8Q z&!37wJunvctJ<*k|k zCL_`7=I+>Hoh(+=c=PD{oQh|2)5U!+NRvMQm&)okR~L9N1c)u8XRPE=z?VZkLnolYX)@gsbdpw|Yb+Ek`$6(8b7RliMkMytzPH)oI8Jks9CQmzt6ugB)>j*DI@Q^^LEj|+uFlky9ksjUjpyRBJ&UR zYup2~*!K7V&C?|rP*;Tk&XVW6dU+ptES?v~9$~tF8weE7mHT+1e}1xk^5he|9v_c$ ze&IF};hgw#LJ~e2WI8T07RrOU^qTT;tzC1aq$B>`&t(ZFpPj!zGtD>uqd=`jyyQX` z;8vDc`)z0Bhztfa?b{#Qy!FJQMS0q{cMS7DMq=a#DHSUvF0!iyl@|y$Y0YBhoSHu< z5zwEzevbHiIhg0o$M-%G%)BX8mKqtT4ND>5SfcA^Bc&AaR~9hZ_N1tOZP(+%qzoUi zP|!B4kQ66k%(KV_@!7Ivt1$VG@F}m5z(4WW!Tkq}k7zi)?>hfI`}eVHagsHDtS{2) zOciK^^~>DJ+#qn2`|`^#AAIP+^UkYBVGc;2{PUxBZQJqi&%o*TJkjpbw%oUaz?ZJx z96FT4z~MuOX#Pun_<;{h)UIP+$4K&zz6OdP5;4T2&V1+b!#1i#Kd!G`SbK8sS%&Q+ zscaHdRKhv^Z~4MSfc9u72wTNtf4l!)KGRFtDN`m7dGQ5FhCr?fajq=2v-&(L3hKW zk3Gh0nzO!~MIfDjUR`A-GCuzJqqgnZ?%1(C$^WiAIFg5MORir|csi%9^rQH z)}3$UNQ-s9@lqcne}4Gk`z1eoKXb;+rWZDa23Zb*({y>m!?$C{J6>`59KLBxRtEDw z7v|y8wryK*5bv9{Zr$qr_dmFy+x3Dqe=yFLm!c(X1$4o}wW4J)4#}=Tq~aK`9XrmM zGuMoa6)jSfJ0j*mz}OYbe{R#JH8*m6DH=jkDGMT!>XQiE5p@bL0wOWU++ z!~OPk*I$pc=|V&#_U+lnOz(VYe9oLXojZ3vaNq!7g$fm*&jKbVP8^@`;e>??7tjcc z6)h$@*nafLrR~}+Tegf!IJ zUIOPs=)dpYgUYVgT+JOgiDa>GGGz;%9-mn0jX?r|h?+PI;p1J63Ubb5^c);Ne)zDT z&jcploBs_s#*BT(sVq4g_)L6WMLT@-$gbaa(OX0naAT}OalKwnB6kHVfBCt0&tCKA z&41;!SNY(s3RAzi>>?{8v32{F?%i(m*d~L)I0Xyl-@A7ovHip1#WQBkys&8#z6eFT zi}*X?;w{GRiV9wmR6DLlnR;`Bn z=a1WO&6+ii9X(Fp5T6rc>M!T7O0f8+D*+DdO_@CT_S?BY51)*Suez2F9U{&Xt=qQI zzB!#f!LD7e=4p~rIC|!0qP$#3j~&^vZR^mXL+Rh)V`kN>SCfXYe)sR&cidiN#XPrA z@WP8R1oyhDuUY=fa`;$Njnlots2uKj=sH=ja6!xFEsh^MK@jB8#bvO-pU`q!7A;zM z?KM~L(+8E~Ol-zS(dcmDrTPy(7(ajhf*+PFDas4L=53yG6R}oG?1;$>Y|rX{9wW4C z*N)Gmp^4`kd_;{ogXmyg{N1xA@HsFV&CK@oC)=-S3R<|~i+^(al>j_Ub;O@z`f*i;-B1Um@kLM; zF8Ye6DB&av60?K{`HZ6xb)C`_JAM-M**q1=ey5g5{I=~kZUtyxR0jmgmn%mL0#NOZ zK?q8t3m{j6dUgM`b*s)VBi`jr$9`J2)V#t08YxC#q|)a@Ck0*4@}$C0{sfbQ zAuo?8FjY|Rd~nNuG9{w`!yy%g=43*`;?7=1@lK>f{=E68eK7^pAW{KEC1v~=u1!7O zQAjIr@Gc4Jm4H9WIqAqEa~y^4yzAZti@uZl>^eb65;Jq8I>Co9*RNX<;44pnCM0{< zw}C1ORNGZq;>0eEtU|Glcc2(Hg0obqLLx?RbU;ICMp~ZH(ZJ&${?@(}9U@Ed1b2q| zjw}Tpe{?{XuGgjN!nDdzLCHA z#pP<$L?h<8b%00sX0x1{eE!8oO*ulI^h&k1d{2C|2vBACO*%fPlaMh!)+=aI!WV>r zC@K{-Dz1O31WfkxuSJToSW=18bbL7xB;d z{`3CI^|SLAiZNhSv}kegcm2NGkKtnP zP6?0CifkFnf9Z7r=dVzdbrw?QAtgvm=U-`2udJ1#KmZ^F zYuTE|A*EdlxdEWnuZ0l_LY7ZsvC0)IQenJ;#zPK$D3m0!t^Fbs`G>nr6|)erasj+N z>;(cmMS|t&NXKEB=}9iRr1_vh&;GP@+28KFZ`SNtJYX0!Xwd7gzYhL{2_JFoihTPH z?I(Tu89gr;<%XjLfkWJ%iw_M;SE?9Y@tkv92S-+yn^u;Kdk>mM2LNR!4Fw$PLfnfTs*d-d+c zdoJ7sOc+1@_B-yxg3<513(1nji+AbV8F>)>`neMCGFTOD-L|c)K06}kIYXnDy?u@U zaQu>=e#9F*6aDNnP4Z%B#?Zq2@S%h5ACFJaDwIE)HDcSgm6}WK!arVeVNAE-OUevVZL*W|_!9tHe7Z39BdlU! zBV&jM4~&_mWzaK&2pB#)GIQq470Xu)8Z`Lz*Iz}}qE2P_7aiC|F{pG8%HZtBsZIFi z<&P#z4XFPr;_796H2fIpKhe3%q_^hBZK_K4tqQ{#7d+i8bgj zGd@XD7Bff%mli?$H6?PDd>Aw$>9rb)om1g#C~*q=b@QfLHERTbQlU-BiA-$6rcIQU z6O4b%?pt}QYGq?ZAaT?yzdE+KNi1TkHx3IH+eT-o4zP(-SsyxE@e5@Qg?OB4vfW zX2pmOzYoXJ)4?V~xtN-fdvcDUd-v>n>glH$h=oy)3>?5MZp8nln{HaReEHI4%L<*L zIhi|m>C7$*(Yb2%DsEp5HDmnvaok;C1!E9?T=FAiIMVTc%`?wDqu5Qv9vL{0w*U~< zlGQ(QEGbHlF>V4~;L~@5)9G$lAze#Ztz5mjMDgMz;xUc#jl~WgJjjDwdOA9G=*ZWn z($mto_p8Q-oeh8d`0;b*%2K-6m;CgTC~exfXWw2~%6BMYPg8;fxtxNN!pF#wMWN2xryd>My~A^I>CA~8OT)v41} z>(;C#w{PFR>(;H~n~2<*uU@nIy6dh(VA=8&H{8$-I9&w_l_gQgX3Rc#VE_FO+|T%^ zj_o^q`st@>kwX*;GSh@l;eRrOWM;Afj(5L@!TAG20rr0 zu@ioS9b*(C&S&AC9KBa{?o1_G!ZO|fDsC4~Kv5l6UDbu24vg)4%PqW|Np;W#hKyGS z@p=9AbPmdG?jLg<`=aRE$}#`b^+O`=obknUZP;QZ;|!^oF>?k96q__@G8<9-x_|!l zuYZ$S_N=mv8Y-^9hZ_xK-P$$i?bEj}bGjcqbZGzneJfY4y8ar4%JLQB2EO|T7UCqD z{AbZ&O+R+zr`31t(4L+S&H;3l(iKJfrPXZOyz#A(Z*f$aIAJ2q*-FLprm9cc7sodx zauGcp_?GzBm&tbTUbtW(O%m%jZQAtOtFL-ne1~&ZDIV{?{=@OQ!$1Sn-y1 z4=wB0%}kcYXEGRK)~Hcq;%D5r558xv1aqub{6QriYu>yC_+NeX73*yNcfuyAO|K*8 zul-E_XfK=p-BS_&Z1t;L_pV?xjX|_HzL=Z_pMnwp2;YnUF@KkF{G9DqWy#ZWn)aLI zALp+E>Tn2%CgVGQE@HTAW!o?AxizwFzjEN3gVQHxClzN@B`#7aY0-|=PJH{W-*WMV z6=~tyrSZQ#^l+~}xAp9EYp=ey_U_xiS8vw0_3m@q)M+!IWZfrmK1VN8kOw@KD>Cz}jFLt={>ayi4cj|iW z*sI*ziSy@%WiV&d|sf1QRhH@sY*pUI!+; zB7p>#-F)=`|KMH+cmIU^?W#kFx@?hu6bj}vV<3Ooa^m>OOPXEu^@2GiOO+s>WR7eS z2|N*OhAK>%F0yz@MCC1AeIkfU1M5UAFK?VO5Tal{PJaxnVtiYk0(7V3D@ZcC`SUT{ z%8WpkUsGXE+SpWiSO`Z7WDb8G7?wi5)sCdBu;p_pr8+{t3hFC(y>@t+B#0Y=vNb>-_ItDMF7zaUwBrPWpQzgJE$|B+!HY594BL0*$iGV+4@q^l?#Sd9zcu${U zYZWhAEWvOCDpnRM#xk1KKl#y=TxB&`L%76`e>ydudS>GK_JE*6m_k=&Xsv>5IYmz) zM_epdzX84@y0sxXx#GqWUO#(|ADK=bKK(9D_2JKn~gujR@&HV zb7nx)RjBr;N!gW!zx-gzM98+zt5{G zpWbA^7dFc)AwDx5s77OnRho=h^{bUiW%`G1XwK!XUnNb<2`A%|qB5Ql;`?6m`B&Kb zj!eD>{{8?Ox$ofep4)HwSi5)cwtd@nLkM6TpN|mNuA>na@4Yvc44%ME{d_9BH{Nu! ze_A-gFK46e$XcJC4i}Ff=MZjZ;s4cFvoRh@++^9LA>t-9Bgs3A>(_5Ed)@0GCC)y( z0=}V0+WIs{&W4|pzXYsebWD_YW?l!d>z8IWZO z$nl-O>rkr1+#6!<`0gLZ(K8(kZwdpO8Qn%BS>? zB<=iykSzg!-dgaKl`B`0?*3sTYF~)+i7GB*mFp$t7TZhm&#s-nb4DbUZ_uz7GZ)Ls z|CIRde?i(3@#onQ2_Zo6>plcu%965!J(MQZuQ&kWKpnq?{&4}WU!1+N*e1jgFd41P zoqBwsNz_f!)vyyNaJ#gtQr6zc(XLBS%7Jw9ldvbN+tS6z{b=HZr7Lh~O8y@@0fDWqnp(hfB0tx-twrDdj^A!J1$7Gp3uZ=mps1SK|X zSU+{jWVy|jqQ#2Uao~P4HSisnJzC$s15SovvNG zwwb^kc%ergd4#X~U>otuTtRNNI{pH)%)-b>s=sJa$2Y=y@3DII8ntTGu2rjM)hbn> z>V)~OGKIK*=5aCo5uZ-_^r}vsDpWX|7jqlBG&bo-(CQK#7sh1qKXw z`0jh|feFKQr+)sq1%lkwmGFemZcbE}LTIWl*i+zgzVJA1B~qo>m~Yt+Pdym=&! zxpHy5z`eX>!-OCnKhHfkm~q6s>-^G7{}?f1#M$&ZYxIGpm}X=epLD{v+yQ?4aWK1{ zT_wiz<bmf&-@B$9KGwxlpE^xH_2UyiGKk}IJ>4Rj$uDaM& z6X74tAqXQ$OWfD`mG8i`V}b2krY7QsKWDqutJd)OdO9F@vcMo6O?X}k|T(qcM zIo`C7z4rQR3_Z1GWDH`li62f}wsa}5{rcT9dCKGl4I1!L0|j{k+^=8%`3vTkDN{;A zI>KQcFc2S6)91V3U#fJeNs}knsiQMG1$k3tz{3OXzWXlh;w1F>6oODo@#4jBKCwzf zarw+RiyAdl5sK_{nW6$D`DDF_;L5#a}VvzLgi!U=Jo4!1UQi+ z0@i+E%X>ZCq>)Jc%>44p3omTKSWpO*Eqhk)KD`$$T*MqSzuMQ)LCB*i&}|9%bE*fD zFVa2!_+w+mvR zY7T3pUU>fbe0fb{B0<;5TW`IUJ9n-^1q#icJ=-isIB37f(~XM;#%waO)YVBNe_D}1 zwCMN~WQF|6cK=vK1(K2fRZ&6^b1#^!eysrcvX0ncK0?>2{jhzpVIVpp2Tb_UX-~Tl zlB6AwRmQ2>7Aq@=a_#@w_G@L6@tt{Kc!=)~FhXKatihl7r!99dT*3ki4jE+%R4Fj@O8Np=` zTlw?kQ+v|X39+?nxuUhBqwwVbRtBhtFYX#ygr5~BS`gt#ixEKK+mfYAl`DVt+2zh= zRf>0jF##EzoDwfpve?_BMieNJSFSb=x#%>a7`V1jo?qd!(74c``e1+`7&2Y)FB>Gn zzNF}qAytpxHF*ewALLA)hFQlC1;aa4D@75DQVUsy3rJ9tPHR?R+XsW$vw8yGT8-F> z8A_#1EWok&L9ZB@n2|?N0Va}0(-iB`rKFUiW=MX^)~zh!VsyuRI%$foQFQ7bld2HX zR6@7)zs2-xhEVhhk#hPNups3kpmZOuI3bTekAKpGeYH4kv)GUPZYWkDY`=_*J^I9B z42zNhcfr!wMHi%$FN1_T);)<#bo_C7U!w+nY*SlkVKM6hqyE_c$^1Qj04r9k zsOjaQE2D{JbX31=YuAdAK|1c?&lizkaUnBX%#|Qk3MR=~gfmpe7RN_~aw1?0ujVlw z29IP&Z`Q17_nU4~c^1p{lF|8vL=SIiPH^d1BURH&#dvIncvw#DH^QIv*(C0wDTtWb zZ@@+SvIo0@*fea7Yd*Gl(`Lu#AcGxVg41Zr7A+{VapMMO$Luylf8MDs1iXG_Au%p9CF=#bK6(Cy=h@}SVz(Q+Gb+Sj-EZc;oxeAw_?Zt2I%x#T*3Hq?G)mDI9vP1z=+mjii@SThes^sDjYMu?s#L8CzR;|l zvSycSJkyQdfPZHDCF&((4S+3^0NMQm4#ip2;t6RJU?qH6K?X~3{xyaHzGA{?Bp*L1 z$n`2Oo3fY%qjOmsGLDy0o;&Kgsx_n#A7>S8*kiZv1r(9xO_uPuE$|LSB3Q zb=o=z?jP1F6N|~xFaFJt6H&^Ff7hLNI2iY_$^NHoDhF{}=)E2|V4$I~uX7VW#fz1= z`R1N%?AyQh*Izfo0Zp4XZzP4!L7Uxd`fr>Y>D|0ovUj)U5eWS+Q!O7jU5N)}g81Q~ z)4T7EA*W8A+I$Gj@j;CE4;(nq=EY159{im7&~1&SGFC+d6Z!M!qX&VjCZi!g*b?AJ z{`bV3_@oQR{4RWBsYG{^iUs_I&qbXdPf3(^%E=;nyCoLm^`C;TzxFDJMbA~fA#*qT zxBLHw0MGt8k;n;l(?b8zlTSTCI=wQop;=hhaKNE8n;IY6%CUcJowqj|LnS< zwqc$yd?uD*mUA*`EIhB^t~1#`(o(C|-yV3ttQKVpFbHPy7Z#U6RjWSy@BlL^)BikK z?a{LrdiU+yyM?E(NNQ272U{d%(_4qU^q%xVmqyjg2on`XJr8(QL^EYo>L9@#7Ma6Eca^g#+ zX%0E?>x%6sBC51Z|A<%;eo*E#{QnqVP;TF;@J*@lag?O`RjO;`46aL`z&|jfB?0Pv z4P25MQ$%B_jLZ;@3ToftzwjA_)yVJQ<@C+32LIc6UxfM3vD>a1;dc99Z(Fx|dBghW zl_*x6cM=)Js(}_1gpGY*hWECHz4gvtZ@*h6+5~2r=2REZdYR?OQMz5EO-w{XYY|Jt zj%wT@w*)-akCiS-yTc)9)%z=|NK)gl3J)w9_)+_|~79ZK}O)^UwLpJK}59~Bls7Ui>0ClP-VGx3d$iHL1Z{)R6r%octn zTcc*JJb8HFm(Z=I^U#miCixy>YFq}XGGzaGe1HPpKpB2A{~x|z%v%+sb*aF&C(jr& z<~K9L7yDy2~cJ5@WOqsIof9B^Re)w2Bqb60qqV8T#Fgp9ufdhjbl@9PZ zV&8mo5B68DT0QoI_myEqtgQUAZ{L0{P$OO{XxGmXzVQ^AgPWwmuyXs8>Q}HjJ|HWr zgZxhN&hDpVU<0rT!#)^ z_$8!b!+Ir!_-t{2rOMtNJ$n4O3GRRQk8reP#*d||hA6IW{@Bs~p#7eC<{5qZ-9k3w z3wUrSK`j9S(iG~`r%%sbJsLE)z)O2RYsZyu^e6o138W%tR<2Tc$&WutOpEeZi2t2C z2&4qF*G2f8zXC#bkj{oLv0w@C4JcN@hQ098JpPo7^DzHx+O!FO0!A7qNcTU=jQ}eL z%;KM9{$zWEo$C4x6mT*T@O*@? z<4mx`IAaRAl7^p_mcDlFI*=IF!L+!lQ_~ai2miLe-g;)?A|CZ- zowX^e-!slQlNxVrvF!z=+H5IT_AKfGFy`x05{dfn_^NyaS!avS#fum7Egpn>_vuZy zy*bbzS=oXKyhwt;%H=D#DG;pt?)2@qL&x?gV?4{Z-+l*%x8D;dPfVRU6$Y(ZwdMs| z+`^kIU9OU?7}w#v zEUkS;u1clSGTSeErVcy>jY~$y2TyS@+g52UIpW{_6X9b#rHCJy#-md1HSA!VeEf|3 z9}+-=948RNXV>HBKh=J-;hQeg=e(`fl&Gr1OYs@+4V&@)urJ1NFU?-yAAwk%z@Jut zICqEK**Z9q64vtv4y~lgB)?M7y}so}2XD@w%+RSuLK0u^ICgyD{8`^DocqnfufAC@ zXW_iBzF9cun}xF$EttD-!R!U|zvSB`UoZS>-rQMCL@r>%gO;Ic<#ym~9$r!-+-`Fy zQuU$24-DAG0yY2G2T$ZuQ^$`V`_0#LeqOfZnWr9Q92fm83_rD+vQSLKrqBFx>yB+W zpjx%6^ql}~4oal0*V~2Wn!s2o5;n6<>AhlJpka?LKh}P%gEXww_aZ4#v16o>4DIma zn@URp6H<;IW6~u-d6($ZPd%#hFb?H%G9~`%&|!j0TELf?7C$iU0Ml{Y8X@>b068P7NzKKa!~D_WEzQ-hRD^RLCyv zxy|AA6%3ST3z>i%eRx2|f<@FEA7P0z^%H$V$76fW$oJ#LEgE*Zszj;Ms+0TgJ#z2e z8-8387FYYq#J7(rI3xmKk|+P!wjUpC+xFcUce6VE^zaf+R=?E0;*F2*SUt$S9(l>f zPgCUO%`f5cO%1?n6Fuhh)J9E?FIkd&eqqS2m9gR(CsXGAka9+Wc3wCHBHqEe@WKnpn9V!drUqeQTCN}e z*_IOA8W0nh$E|ZpK0dqp2;VugtD^S9>|@Nk&%EZG7MnJ^n3tWwFIl44XOljySgDfn z%u>IkoD$!_Vvi4e)Nae;S6_Y2r~mvXP|ly+^z`(LF202Cn_>+7IUv~g2aHMJpMV{P zd5de=s+D>zPn_uAzaMKkAv^@nojY&xXP@D=3z*E_%AHs5JUJv5V_L2gCp6caIbOqH z+qZAeYb=4Ltmj`IY3P|&Ja)@1w+tCF#KOi+iT|=#n!rD*QY$GY;bO-J(NrGxg#KYLZ>36=ApFuxFEWA~eB}kDG;hTfS6t2> z&;N9tnP6B4Fq)yXN^FWGKx*na;Pd+!0LMRl$HzvmndMFBN7P=Rwm5EZN_#tJqN z1Q8q7#NJ|xSFifN_7+PLODwU(8%=CzV(g*<_5uPyv0woUDwY@vNpNgS`f&DcyN?_< z?hz`3x!%bl6Up5F#83Dqytv(V+cKu3Ksxvk&TnPBT9p>22>A1z1X8Kr#{HCLlRa&S zyleXfEam#Iwe3=At5fSx7XDmsJ$BgO-2#R@l;rwll5E~&R|E!77C%y;vML@3vDv(R zS-(^hKcP-V{Xg3^Lgt_7KP_4SM6+%i3Kd})w94CNLddf-pK&g7vfDuHS9AXMu z74g|tg5?%5#r#T>?nGt$wV`qq{-lJMt_)>k;>2&@tB4kfC5lW81EuNHx%R_SU$#{-heQ+q zM?uy)2BT({GVz> ztcA}XTgb3}>E%~cvfRKMM$P$0{qpVwMH(6wz4z{Wflm>IkpRj}MiuzlG5^5N`IAVv zs)*0`!avqki63$)ZPNZ_Rq%fS-zS334E~w_Wjo7&0UG?u<3Ep2vO08FmhW^>^sP7F zmZ<$RgU_}}7X=3_i9fHytk<`XiXbUkQXjj4pZQXW+{Ae8)z_K2g6SSsSb^a`zNq>E z;=wlakM@zaM%gLv2)xY16Jb%GK7EwlQu|$Zojwp)*sxfSA$)`OJ8-~2JquSZ^AL*a z$MMOddQ45iW!TnRZK)|In2z+p2Y50zkDvPl#OSt|AI(-*=~ zzri;6Q@`u1gQGaU!c9*hW&XiZB0y*PdHdyq3lTr|yxe1Nz)#+d{*5o}tnW|rkVrm& zIe%z4rb>O*U58Ecl5OBCinx!&_d&IkHF(sL|1F6?qxQSS7F&_R9LZa4wG}T7&HZw2 zrc_XHXtDoXamAISs#L{jdsqBbN==V z$aHcb>hfO#I@oC!d^rby10e*xe_?i5K)cTSfBPrym6PAFn9mB0+H0@7|NO^io*4Jc zGmm!YxU7Z=pe($av11=$vs>3L?i!R1pR2C^ZxT~{h>A2)X!7JKoD_W=3DVYetaKM8@7E*cG{dme_t5 zju|uWqDzFnXd(8a!ap)H z>wm12d~T9fEvE8)&4aVT|4_~>7zTBzgwW!_(49}T#* z*jMCFA~xmzfyqvTcO-Gj_c`)<2&OOnaCu;R=!&IAD(%*w~x zq%`k8-hO-U-s>E8_~C~gdMGD6yuQtK!O)>QZ`^NVX^r+Pt=j9}z3zewE});}>ixu% zPVCgV)1W~+?mToT&zW*wM?vP*I{tV)y~~0_it#gl-dv^wWmJP!bKmPHL zd2ycS5tt&4cllRay$f?~fV%Cr+n##rDKR)zL!%TZ!=P5eV5#}^3D*%K4Fe=>7Z zn3zn8Peo%EKSpImx9*B?eAd@1?70u!uz1l)C!NH14+rfyc;{g|Gtt8a8?688tJA63 zF`hRr-AcP90ZEj@0OSUqoCJPiq)=NFyN)WIeAY%N@B$y+Z;hv7)vw$ zY=r-VzcSrFJCVIg{97=fvf43zD@lcFH#upN0@VcqVkUJW%AU4FMLBF`De?RQB6`y` zIMd@I6|}roIXVb++>P;P&Uiy5nc%le7unBJ#D|bfuIj?WX;n`tEYOUAS-70oKmV7% zTz=(mrcRmi`s;5nk6@(BXRa7`(}}HUC491FZZgDW9CjH#ee2XIeAb&DmaA6Y-yl~m zB;Sh)0;R!MC5ATwKLlcpb{X!S;E#;X_@rNp{F9w~b?&s9(>Z4mpoMaUXla-_^?6YW zOF!kywKg(rr&U�N=G)#GQN<D#j2mAwbLS>CEYuE%CChsY;5r0Huk&|5u;n~=hlDs!jZSmh7hBgHx zdQpF*I2aPTq3k?VdZ{XP>(+%W!{lK@Y4ujDS6h8Ga^HG;rZ|dj^&2y1gqE(d!rJ+1 z?0-()i2S5MjG_|sYqq7o{N*oKTzTcxzf5`U^*28IG{v@P^I!gO70U)&BsDgP4Q+-^ z+98{-YN3w&!w)}v8Ro>b{|TyKCIp7_sSb&ksr|aTdTnW6_${_y-V>pl<5T4fc5wP) z5>?FUjsG}#YTu#5kRd}T$uy|USGx1gI|Ice^Upmy0cykj(5CRqWxmJcnkXH5*r7}! zC@$g-9>D|)apZk8CGow_H{YzQt0M&&a%8=F(s+nnRNrP2GGMDp#E;@rj!N<42Ale~ zu3fqUVDqd=I0+Q~c&5I4k7&PQ;Eu@RRjV?K5Xxyayz6NG;0wx^YBEGNMb94WRQv}& z*#G_q?!EiYyJpUudB6cbU_J*v`n$_6G^`PwR^>Trb#!43Hi8$(%&FL}ZJW9C7X0m# zPgLuT;|F;`+dtNn_@@+8krOSG@WVz2Ey=lmJi1+Axu{)rrIp!$C9#cPEDs+_=r)VHXW2%154gx29}qAvf{K}j=P`On zX)by}SgvCStd9ESOIs^0zry4RPxt7}NA++EEXv$eAQ$^OuF#RkQ!#{V^XI2L&ty70 z9ho^N>bLsDlTYJHn}ZhPH@xvNeE3j%16S~T4-;U>$GY;w8Q~?2l;iPV91z*pWErjK zY%nN}Zjy{g;V4vf=4q#%aKg#uYCO-NN$J|({r;4b|I3%`!t(JF>(Qe66>o));>yZ3 z{DvD92HEj9GXZ;b;Yx-}u~Ji%SN-;TiVfnShKG8zDE=Ac&|?f5=~>K0uD-@X8NxvH zC=4JDp{!uNi#kEZ5THNSft`M;!Wy{j_M4vUFqG5EGuR?eG!euksY1nFBLI2^B0VHR zI8^Vt%kcj?<@5y$7;R{HX#8Ue6X7pjV~y1aHdQz<60C{QICVNO7P$PcXM&6`FOwz* zD0tbECnJ%4rwA#|PqOw~p7r*8u0rA8x{8-zjkoRPa$8s!*!1Wa+u)!mWBPzV2E)Dtp@c4O@B|1Q;*; z-U_wbZC74_2PYl=ggZqxdkCNrG zSW&}5V@meM6F0$!EMll@mxg!Wt6EJjK2XcZibFK>=U;AkV^;Nl9;0{@N)~>Ctn|;( z5dtQk>ra&;d--X6e%r+Vt~^NwH`XkuHhiP==k0db;k46F=d$*JF%R%jHUY!1aQ+;+ zP0$9zMDZyXRPgPKu<0DGxUG({_K~SyZgl#$qiLoZn?!xH{Cpb{CEoCK(3bq~X&h#d_jzV$x()?48J+@y(I z>R)&5wZV;CX76Gad1kv8zC+bDm(Jg~kmk|LefHV=lTZHk@kbvq&lg2lI6C~`fc-DN z^w;N|doH`?RyluG1c`KnkmT`?ej-ug(i9vbJ{2k7ZUVZg78*a1_9~qfDe=cA2W6Ax` zBkRAt?mC9JRF&1tliF_d=+Orsd@xV`;@a{_ikD)d+AG(0vbgG+ zYp>m?-^R!O>{vcg$NN@~J^D!QY6hXMyY9M<9XjUi3!A;fK_iQQ`$xoxU3Wp5gH!vZ z2zA653gcz_?z0c?>GNfS*Is{3wO~L#CdiXD=bn3RnhL-LA`6CWg#Y172^8J_$I%BK zbnx9w88d6vuP)YXR(bq0dUDAnmv-;bU0hU2DvuJ3N|nfu)G8;XxOs$5*5M!Ki(y8U zgAP07xrq~bF6OuY{adOZ5DGFOBM~@w@F0X)#7{`M+kAY__M}DS-0fIg@k0s$C3>mT z(7p)j#9l7cPU4TO?DHZ3Ek0;w{1N`~kAK2U5qs}Fim?Cd#~(jD*0b-Sv{lUy4?N(K zODTPyB^Zk&=yafo+S^2>N_aK&CS! z=S6YZ=;LqfhJPp;@k65U<FfVM2kV}#Of zRcW2f#G%0uq9^96CQ9kNCC9g?B2n4kfBaXT%?72s>Z+Z9Qn|+XGN7zi@Kdr$@hmtqm(Kc z|MT{5c+;UmYMB}JZo2vA3(h-_PvLy@(Z_`HLu2^b`WWUw*?r`046>pWKyh$oaS-Nw zkVUJ;@$*+6yjM=un{U0TSP*=ize~S-#G-)KGW)|1Lc%fuSK@sQ)KDm7lgEnuofA$= z#ku+KyIVT#km;hTA8VK8f{R3DJZcqJbkf z?~5`sA@$24Gf;T$SIRLm`KRK4wo+{86gfUPeh5OQs(NJX_^3sS^Q^?GomTzciYxMg z+X)jV8V}4S{*gyGq_8w;zq%nHOEg#@8UC}1vvvRgKmbWZK~#&8)7Zu~U3)pTRQllk z4eH&hc(qQQ>CizIs;bB6)z$5mZ9i|mo021%Ox1C-`ppSN z{3C;ZqW_HkE!uz_iqB%jI<5*_xQTxve)!my3`%F@q7?Bd&&c`||KUS%W-Isy+ot}3 zdWF9Zd#Ymf4lH4Ei|scEYMt?it)UwE--^Z*G3qopJ;HJ`HZR1rNB_f&^gOqZghml~iwK7) zq&NkTQ3x^L^Y3oDMU^PX(xc-Z;AKX(i65PE@c|7#K+Bj-g!+aXuCJ=9DsR5orVl*$ zFnuI|wQXDPvfHS~#ytce5khHb+(M^>}Ii+-rBEtHe^eoAedV}5q* zWfxxr6CzgU39{67%LKcJbVPN=Bgb(K3tdMNlJc8fA&1G$%r>G~Ejhy5Zh>eC3{Fdh zWPGSCCK*q&#rs+Sfg}q!t}&h{`j9Mfdw(}DTW0sjGnk- zM2Rs_AbfJ7{knvBvdGsSQkNA12_$K^Um z-K~z?mvRawDUX_{;w#XtMhvSSxr==xV1=I`U7k4tuJ}#$RkxIn+?Pty zAeHo~X0-N>WMjkat9H9>=UskTo1grwseeA7Q#0v>hVFF@^{b0Q;QPm6W{vKjaiOQa z5r0zA3BDHND(@{`rG<+9|TtE<{ZD zOx%DUn0$(2BoaStG{=wrUm0FjS!I=5Zn?!fjr`Ae9JkQBtg(8h&a0^#w1+G0t0nw6 z{sz7%$@5;!9|tgXDBm}EZt`Sq*t|IH1ztqma*NG3+k6Yo&%x(j$gQJC^L!=e)iw%8 z!>hd8?z?xZtE;W9Y1e)k(Qx=PFZ<0r3;ESoeCpz%hadiM_6NM=Ti?Bo=R^kK9G8@C z`=5)qr=EH0{(J9Zj*mBI&LrOI>bs5DWmle7K*JelpEdja*-XH%2$RPu$1mO*=Iz(# zpMPP>l*#YBJ*#z_+HPID4IVOta~PP-m^qU*w_Pa|^&RzlrM z3I7~l>E)^;k2r#Rlw2BcbM*c9-sS#QUAL}VZoN&vjW>oE-hzJ|e;MWBBbfK!ckk4x ze|c@@OnBFK?KWb!UHkYk+q2I+lL=6I_Uh%@RkS(Iq~pjVj^zE3i4!M2KXoe4+cKAb zmu__fx7m7wjW))jV*C#uKBBI!p4&F?X1l*{Y%S@)SIpT76Ldx#~%2BYx&t|LdHh2m$Qb=LbIe7IJxUPb@Veg_R2G@6N2 zs;jxel11(~i|v<3ndU{LVcO7P!|@(8m(|BNQ~w|$ic>sM?%J*E=+U=v zZm^Xf+t1IhB0jg@ZhL0`okA5(p28Jac!x7qyLuA#ZpVcuR7@`hmz?_L*-B+`nMv!(kYSf zee@I*PIbCM!1+3c3BOg|&r&9Y610fuNKav?D9c-F8#)?k32ta3Gnq#&?b981I2~aG zLzr#t2+3{C=GNBwu3EV~=gTjc%uTW!zKy^sk#ukermDoJQq6(FDsMaWRpA}B-|>YP zo(F%!4L3UUkVAGJHjM9q%P%sxVg2sgcDDE>d7ZLw2T=uX6n?b zxa+ptZ|8HGW5BgJ8uh|{-GO}=J*RB+bb?>#- z5^3*pb5&f*<<_m++Bud<{H3kPUxh)g(J@92@c}0E%H4K=9K$7zwuw�}1TpWZ31? zzj(qU9Tj{!GL?jLO#?wl$BHi7D_(&guH{B6DLih+2N;CUoNle#sIl_|@|-ygHc1^B zR-N+)pEo}6ZI#-hBMSU#9<-sxchHwg%jFF>-0+Y?4;@C0uiDACucVqc1G!rQ;qZ^_ zwfBUN|9#@!r2X>I9Mu3HY`*EH)Hm5bsGtZq$h-elMe@*{+?;%7B@$vo;_$~B{VzB0 zz^AD%CRFy>d++BaOn9HKoK#nHpw+cnP0fg1cV)2Sp1bd%!NUK(`|hKDIoNBIC&T9? z%vDx(^G!GEv3Z^C@=824r7#AckkWOO&E&OqDlys@Mu%7RYlet0U?)eHN?(0Bx9FcD zf8zg>zy1{&p>)&GcH z9{=`=h<$DQ1>YVm9A9vF)|p7D|AJeNPy13ivaCsQo3&q3;`)G0?qq9RfH1SAi9gK& zd?o4jKE~QIpt$DesH8p${1cA*>Am+p(5@XXbMY~jwuc}2<3T%a`|C@7Rp#yThQ;^X zcmJ`+pVYoRuWLD`%MDjw{%Z<~h_rFd?k&{;_xx1@s%5n>J5ales~7Pq1Ifti(1q;b zM;vy;&9^B0C{W$7^?)7j_|vU@cqLF^OojX2_JfADVi;Xvp;+$UzrU=js)E?seK%;} zp%8rtw}$uNRT zdQ*}i3%OjuC=N=Ids6K%nX7Dsuw1MW2bM(*4GhJd*>oPEaWKmF@qHfe}@qB;_pb(X)>lht7BbXjp`kk<ngsNI%lUHcL13sT!0(|QLiyNxyAMDVVS^>^e(6sa-f`!` zyY6WCVh)X_rna_fp>BPYTQ}Ub(Z)OPe?X7=9`t)P46@&Ld-WId_1q2s8sEW06n+6G`hxTwjw(G=!g`2>?8?)%(QLS1NmsGaAujA2@+i#9v z?zHlv-`w5mP<@qLevrlTocm82m1k z75M|jv4HA~79{={MXmb46#UWaAdmt~Q=R>f&!2gto;fd`dg95apL%kOEw)TG>9_@d zx7|kC!MGNJ)@vIvw)y$9T_XNmw~ZW0K`jaz*GcReKWzqo<;sl~bNuAz&&cKH=ct$G zNs<~#vvAJD79Z~$IA8#y9!fO{L|MRJZMDu^yrnWO9mn6gt~{mDO$eH8nWlnlq6T^H ztPCE!lN4AL8R`*LInLFZI?A{tTuew%K#X@3=CDew>fTfaat3WxAjzSVbGoo_qJ zMgEJk-y(l=rkAQ>*hmj?NTI4R$LC5uR0o$P?UxS=GLIoDa{RC>;y-U+J({>Ai0eD z3>i9P=*~k)iTDwp@Q|FE+Ao(RYxU|G{vi?kh{ZU+5~Vh6NA5O~tZ+N5a9r|Fsk}2U zgbW=f#j&VbSjt;$xs@g5=sZ5jSbdE(cI&c7_@CXpeSzRQ;nDIs1~sHkRB0V@BLCqT zS5LXFs9$Mu`^9y17qltFGS^a8XT;A&%-o@xqPhZ8%O19QgB}|9tB6 z&mVW(e~ceHju~stIp-W+b<6ofntMoU45OomH5(4U0ofp{N?RS=X3H^HIKSc z0#hIu{vn+pWTXR}L4ybLAs$|+*DTaDVUL_oukLi|Mg?OtCm>-bdj$}2Nc z9$-cP2lGg0E!UyKze?0LEK>bO`z4iO5#9xzF=HkVDeN<96!SbGxcTOrGkyUA_>BAV zoso||{%GRF37c%%pRqF(4I9ScynrI;x&$0C@2`}7kIOHf*Q<&`pT zRN~V%Qy|5EvHeP$6srFbE2-9REP=n2`YrT7C;>?gs0ltS^Y{<^@DD0-{JeeT5lQaf z(SBRxk3}uw7uzo$z&}O*S3ICR9bdSa36Rcf)vM#o%>QyBi=_CEyI7-?K;?Xz4bFEt zHUF;@kNxGvS7;oH%faf>Lu1GFUUvgupL2SwPhzQ$r))`|Ghch{wE|Jz14+)1axQu4 z!mLHUR}n=KO2Vni)IbP_&Jtg2U70r`c=$jq1)%Dh-S*r^8>$ghX{5lzJUq2f>fD*v zr1ZKX3jk-GaoUL|omwLc@RJ^6-v2-*x>J}_7kG<+Q zmrGQ=xkgx!Wz>U56`~xaSa7UIT{2%;4COg~_zN=NaC~P3Bt@zC_##Apa!h+H2_O{W zIjo1TpW7nRtM&OT-n21aW=&?e0(0>yubrs4Jm3)#vDDQ9PO0H(S!L)NO+Oy=0h$`3 zh8Hj9t$xy7RM+#Av=l#a4oC2n%S>y(X7_GyzQt5_VQ*=#Jw|d5TtqStgQ)toiP9r3 z+LM12S7?*9Bs2-=MS8^P` z%GFm<2Uxd`2QW}&%=ff@7;SX zz9D_?&(FR0p8G&&$?KPRatf)!SHvYJC+-sV@zHT%=9L`Ph+M4{{VLn596$4aWJ#t) z{O0G+)>Of#esS{OjSnAnObHZ@Y7!!w@z3H%{B!@yH;w(%y#EITAkeUD*Act&MY&a0 zTKVNyrpI+e;DZ?aAA0EFDSw%ayBIa5VAOAPg0H;t^2FyReE<8Ki)Z8Z75M6xq%4n3 z?dANJ;(xTuDy=}0|C`{0l6p2|jK3uRSY7gnWF`0Q&YxT4AO5j5Zmq$#;GevGwdnt^ zo;>U+A8PgP2%V8Xuqm6vCyw{dKKKn zzj1v2UunPGhJ1MJSe_5yvX1)@!O-qW@%hk04>CU>zUtq9(g zz!8IgaO9vL9vLYTi)^;EaK7T7kax>hyA@u!zOG*9LRHnw=&Qk`h*2at_M??Ihtuz=*^yNJdR5ha<#m$-0!_syAaYS*y~j{dSR!BY@ZuBiq-+cB1gAC!eObt1pULZ?HT zSEi&?30V@s>Z->ddm=H+@z7>skV3Cek+4;sZ=&0=tt)buv527K~x7m zTJgW$6?b~hAN-IH|E$-y@4ffl%ZFM}{KOMaB5O!-0D*cVC9h?Y0-wtO=Tm*2s#D6e zxU5zz>`*fIzfxGBnNxI6AL%)NLD+{{W$~%Bg8#*gg&*VdkRc5J`qz&aEQr}jlqFq` ze*~F*8ZecJCkRY4$474Z^yxW&VR{MvoW?SLElKtDbwc!IL4934A|PZFwaOXfk|?hx zAC{)SHl0W(UAYG8)O2j$K??*wneE+eR{XusSN z=!wW9QjP{P-Bb{-Yh#HB(6&n{b|Wg#hj}I#BGal z$p=;KOADDnGHbu7 z#r>!FcnSZ=Wl=!;v6EaE$vHlG{6%T~$UOTh;fbW$LWWOq>rHYlss1-hPyLS|l8+nt zC+FWtYh!$4CjM4a@qchbAm=Y7js2fw+Z=Pb&Uto}8cPqoC3# z6G>Dt@||BHeUUp5z7b83nDzDATL#wnG5jg$H&orb-yd{^=;Q^$3aMk zB&i^k_G>qBl0Z{PX$mT^^_+kQ8n`=JX4d(K9=Oj+ z)lGxH`^a5?_LHNTgjgq~#-nf*4_+)TFVbL$E)dbtVY&9tKKZ!VfRCIQYoZeV!Httm zov>+1{ezVHb}m|4>D3!3faie;`9zvb*O7*~Ri(4fK8vZTb+xOZf%5$iJj|0OVTH7* zfE}l@Y>9-n+)r{2r4)+6SC$#y&eOL4FqShdZV{6#SuaMFcCv)0K?gQ;5EB^DqKE*1 z|6s?l5f5l=fsrihlK-*W7D>BCI^5`S45FGzQzfO9Ts@D4H&7ylx@Aq4_lYi!zN)Ka zpaqa&<_=v@X({sw-GhF3fcPg9;L204@~OZz5Sp4leVXv&2}h|?8UCVm>c@zY?rh8l zq+GRd;eso#Xn1Zii8Zxt;Lkggyg>3+#}#I6v)vB6?@l^JtDkwg#K$$hTtt}jq#3S- zMb)x3b+5iU^ZW}XIzCUs5H!hg|B|v3dr_ zV#&)hOK!gwC@p?+XJe4S7njEWJr&iPU6C+hTxRa z&65>SB>tSsiE`ZAx0?$zt5ioUx3+SjuI7wB1DU~ zlYhi2kd#Y)25Wpte%X}{7SK-S>r{H3ITZf z%D?tc&{O-R-;~flxBbcyTGH`R8r?r={qNC!neS%)ym@cE`PPw#AI?YGA(j~X(DU?D zPcge6wa4i+U+NM2tcu~!TY+lIfS_q1 zu_gatGnwWqnMU!W43%1AwKecR8#8CT!DC;BM^>2>{$kD-cWI^x14)Z}L)WqVY)4#q z%>2t-_yg{d}m;tT6pjeD@KhOm z?}{t0;A^<_QzRU+OJs3jwb#IPy-uKVk{YdJCD8aleCjr;jbcIO!yn%hqUnv{9S=| zVum05EA7{hF_LI18rq;yShbFH=%ts3N{y~4wXH6WMLDVwQrMt(r3XCojMMm7`5t@j z&o>Tq^dnubF2F`+&7C**|2yHh!w)}%a)!{eepFLU4RaT$_vQ4Es}uKJss)OWYNwKF z2?iMyF*=RYY!A|1J$>4wGtNHmh8u5f&1{co(ttnr#WfOxDBpBE?${q6b;O}+=_(t? zWo;aN)ZqgMZasYDD8~F`1RbLWF<_xS;mdgo4nO#SlmF`^ovGNV;CdU?@!}02Tnyt0 zHpHK^96uI|E0k*LkN~%V8%9b*(rDx=KFw#RLECp)rPIfse2h^fv~E{>>~SaD`1{|5 z@ltK414m&nHB8_`3wu0Z!G9GD{&@uYo$7RsrE0(}x?dMpL2QI`Mbeyf?2;mlc}R`+ zfkZP`j}+xoy;X~e0PsaiDV(mzM8+_*LZ#_Lc&jNq0VD)l~f{XG*#X^X( ziT^1p(mJmRLQvKsCy2lj+wa0p{`TUBAHTL?Uy9YrHoQU1SHIn(`xg6*>a>d9tt-!; z_sItzOzyMp93Cpvh*zl=X&Q!l$gyLDc_L5KU?xI(XJjFhh>$BH zybT4rR(RGyQ2pAhhHVG&Y`XGVw_dRQ$Wq5;%57>(pMBM8`rGBj%cd~4=b;S|IM!^u z(ZYQXDbJZ(?%2M(XhF@J@7BIOr|PJKN^59F3Au|uyRP+S{TE(yea%rP&0A$T)#Q60 zx1Rn%m0u-r4N6$5qh{p{)v!`r=dv63FZul0dWf5VaW$#^3SsE5VWalmi!XHi_@_S^ zxb3zpueg%p!SNILV#wmMM7Xx?|Ab}!&GC`ZpJbKkA9>!qV>fc`0mNjIh$Z~+kM|`f zDe-5mJsBII22Q0j|5ytaAf@KvB1^#!(*j@G3;5E=e&gdWnnkv`|C{qSZzGnDI{N6} z|KShZAHDwi>zNmCuf0a~TB{f54}2tk(sL8j> z{8Qnd{8Q+k#o3YbuW!FUIp!Fq3g*1}ch_G(cHFp8dynGGdGVq}Tv<ZKAL~_hwHW{Zn)WMhOO79hX$&5Srvf*)s-;DH zP#o*O{v6smcFQKO80;oeeGbF{A3;T<1%#uL0u*#)DnT%r7M~aJSnnz;@4w%E*WYl% zlqr9CaLhwH4%$KZI!XwiW#NJa=luK}Hb~d&v#|)b)U|6D_V@tz?AfzBb<+H%?6O7+ zFU6d4@@b+WTR6z-yh}9M;n2y+jD@Gat|tx<%HxdOH4MwQ_kd+iN|&)zsl@axr<`(% zOP~Zqnwz`$44+hTh3<|R+S2YsvR<36yVlNPt09m0|$b_Bb`n(IF%+gfLJ{6NN|K*3)UuJ&%;m6>6f>LRVVu}I^>oi8*5DVF*`sKX;kV6l?;_}O}>eLtRy8AAsqQFoVg9&iqq6K`w z3B4Vb?QqO7$0*6TE3Lc|v3p1naQ?nE&+n7iJ1HP~G?1rcdO@GGb#`3|39(?&_movi? z)v?#8QHZL>TwGQzZr}z5)p7m>7f>AlDlfNFa`J;wQ~RaHNgc4&HuQ#HeKi+iWn)3EV zse=wYWc;J!F1S$KC>!`9Wg8>**k)n&AHGj^-F4UTxXq1!xN-jj4p6H36A2)H@%b0$ zp2L8&(^zHkV?PB3sS0W@a)J4TCatwEf2hvFC%Gs&Oc_p&TqQ~2U+{|yPe1Jp?wvpT z%(F}_j&L5Ii1ArjtnIuiGvL|)uaLn%PdxqvYF1ux<*sY2!A9z-D(zSPHz6ys#)!`x zKQeOu;%Phlo9b7&u@u@bc~QSIAq+`nWi$8^(86vy&z4xi)o)(EO0#Kycl#Aw&=8il z{hRNYiMZGn)v|0O!TyJeHe9ySeoZM>Nb70}A%|=TufAI6XC5D?vM4O`#(O)q2t^Mc zI5BEOC%6rKt9$nQGv9vuooAkTcIpe$KKk(En%bJ~b=`Iwx$A13J2N#}A|QN?J20Kt zeUKMd%s%z`GT=dm2XCAhzqt&Y%ykgPntTRtN;lyKS zz4P8Z_doE~j5qn_3lsbG=vFsu*wEEFcY-iizjj-&jT#}=s@Cqk&g(Bf&lJ>8KK0a# z(_WhO-g~??-Dmy2n{KpGuU^i?#c= z6OKRjs3VV%tDNVyT<#vd;sR&1$l~?L`)|u;mHBU))wT7;$sy10}xh`w0!H_-Ws+j?AkG<9! zxGjUWQP<3FrtSzcc=oAP%`)wlP59Qb=Jl9TdASa?KRTqg{}$z$Z|U%`fXQ!4rMCJ43&VVp)V+6}JAoBo z{FO>uY$(+{k1}vz)hb=fk4>z4?_&*7?l`oxReE7lcB)!v>D!yF-`PnL3edk?IAxX_gtslgcnOppVYHW>4%-Gw%0LVwI}4vKa>?%B`kC=vR=%+Hd~ceV~$!7RPp>-h4rFeXC_(S zbN4-UGjG;Azxd@Z0xbwoX5F>dz2@qxH|^g)(-m6epOtwv(O+kiB5RD}OVK}M1S<)! zWHrHF(f{TW%+vAf8@6BdHY`EF`1V<)ewAh~{d*mMMVdvukbmt!w!%v-oj)K+kR*F8 zDPM-+ng8SX!k!lSH{oYqui#rkmB2=DNReX(Vi9E=mw#XTjrz4B)Z|S5E!yuYojQ#f z^S~Z^?eX%9FEi5jiwhO@BI$5td!2RGz4n@G2>YCWwBMtSJnHk$=I{dh8E2lZtj6u@ zU*CRla+CHIgp2)~Bv`n}@rjL^CmW0O;WeFz#8TOiRKCQykVM+{>eoj9Hp%nYaQRd; z@Ok!9>Aae3xmdHw7dkn$!imeaZ~sRgk~D`9o;h>o6xsm1aRaaeQg*AKa zxyL-d7ie9q_yBFd2J-HE>_I`6FeG3jC6Xg0)_vFQ169<0`nSJ@3>ved4cA_CEgE^X z7<l z>oi?TCS<7)v39R!GcrP~GkvdzR-lu}^xfj}=A zAr+@O7?q%Kn>MX4zwELQU=70i@6V<>Y)JLVC!ZYr6GFJyYDkH_lm@7UR5tVBS!bS^ z2V0I%S6814@SR%lk;!v`dGFm>aDzPO?t}{Xk;?|5!f;6x7dN>=`q_vG#}#wtd>Qhw ze&h8SZMffEQ$x_6fByM-5lS(h60HVZTJ*8Tq>Bxf3*BmrIVaH2W`wlnJ?e;0rg~KY z=Qi>GUq9hHQ(D4N6DCYh0999Syz#~i;L3tbDe=~}wl#qJ?dO&i#geLq0RslWagROs z%uNo!SXWS@l-Ekk{NwG!knM_;@sp~AQa(3Yu3$V(XKf=he%Q}RWQCNf~ zfmka;TC(`82vXYZi*B94f4^U6zM`z;(?RhPMR7%KXvMLgJlw>349~)UQ{i;X-pPeP)N2#C~ zBoGe%E#eC}Em`cvZdSkSB_{SyWQO+KoZLT2dn(Z6C*fq;5?n>d2?@bmoA6KNBh_*V zDo4zzB2(aB;GA(tzR(Xyv6s>!sX|LzMo7Yu6Znm!T)%m4UWARM#D9)ZtY5pLl8S>Mg{Or2lUYjmSY)9_@apT5uFScv9ZjV0tDC)wH#Kb&;OsGcccQXIyDUO*CPZaf3BuhoY1tSrWVmiA%pqh|X<+-hU zZNhj*>)!oH2{_i3y3FACbrB;dv4YQ?_S)K3oYON)3ez3$_X91=aoC}KdtP#>tM7j1 zS!Z&k!1C#*pHg0h8ydd+>MOpPq|24+>bknRLwHr-;6v6~XYEKCHhgC$6Qal-d&Cfc z^y@#x%Y3@|5X88i;g<67T}JG;|9)F+x#i@^lMyJu*3{S6A9mPbyu85MO~~_@=bT3kjrw-HP=M$%o#Hao~!tWnO4`fU6=!< zLk>F({!oHH{uwrWmreU`3Ic;2s-&zIAaLP@40h-OmKk?ZWJ z>UZeSp}s1sVH${LVU4-EWQzNIS24_@Em=6+A^s;Qd%{B)ebnvKAd(phv<&yY@-`{{U*&aJ~YygIJ_)yeq zYWCfCzbM?ZcW=I!eb7M%Z98z_^2;r6|5fieXwc`Mf3AWB|FcJI1FrHsjof#=AN=5e zi0$?rbYV~2o_XdOWU|fIgFZjhT#GtyG&*+d6RMXmCWUxa8-D(ghF7!cBHuwJ)TWmgQ z(j;J*>)b}vbgip9?zrPFzWCyE&pBuJ-F9!qNJmXIgLiMd@rK|F@xwH<+?)q61`30x z*HBF^%y*;xvKt86a4j_9AGwhjWiy1ZLpe@~buOah$SUHK(gdH~u%a<6g0X}++l}HU z3Xw`F{)3n;M?}vgf;2m3=y+vlIKSQ5lN}CSgf73{$;c% zFG})&5V-pFL0MOs*_ALG%owmxl3Bqppok9$$8s$JWPGu!n5;|%Iim#O4Suwcm{P!( zP#j{Q6vab`RAUCdaa`CugFoUGeA^O*;E$-7Od*qrRHz(*^nTFZolAKF#O>E!0%N)$EyvMgG(K*E^`wi|&A z&{;SF1XoM)Mj~OOqAV%Mut<@7um4U!S8tE2RF03liWxsg$&oX4c}@l2SqP?T&H3km+Q@_=x3Zgg*!j7jHe!kNFj_hc z6&oQ0p*#`0n)45YY@J|>{xMD?|K#|ATvU`%#`RiQ=}ZHC*({<$<6UeX@bwLOf3{crm%!K z2}|=&)Nho^@q=XG=O{UHD9q?I!gm%a-?d-4FxDwr)fJ{ugJIX;gcKb|ib#kEJQ zbS8nf?Czff4vGyfBStY$T_Jhnu|ARPVd*eq1N;F%in_ zvHKqI&o#4wJ$xXed-v{w^glt@w0iKtF(Ec>+O$nK-PE9g9P8?R_uH2TD+UZ4fG{`J z12p{81V8ymcFK9YR#RW!9dZEaDoN`azFrYlTyqT%GO$$5m^nlA^2MOxAHMOkna6Gf z;wvYH!A$r%m!ZSPTg zw`!D)+_AspcP9xql{1oXB`#SS{A(&zkR7A9wtJ zFw&7CEc@YtT*7{`1E`3fJc@A7fd?KGI1JW=f5MZ=?b$h{i#T|_TCiaLDJP%Yx^*pO zRXl#Gt5;fSWxj`s;TTKW-S^y`YQSb(aK4_7;R5-yPe1+fkB?!_+M@qKX;s_mjMLAE zqh#)r!vt~KA1?kwf&$MbF2>Wl=-=*@EHul?ZU&Zz| z+px{0vP3K4UFO-1`tJ3aajHGG9~d(xlM(d92@`wv)DRDFQNI!YR64_3ix%lDKjdSb zdp?W`jUmB0{6BN%Ole{~t9S1v{ewz1l*f;M#Ac_#$L9Em;ir2*%1+!S`Y#cKKKxi{{&jt%eqb5HisT^7(P7tlv8YYO_Tm1 zY8<6JV4wPG$4_hn8-pF#d((|-HNiaAMgN0f|5xAtzysJjmzK5nCnrw94`uJPN-F>zqGqEm4tT ztV8mP;t-xJj_eAM&6gG@3Be18ExA}fTAb#!R!XtwoEafsPsT}JgoUr632m`bqKbvGD}f1$ZxbN-SOIN1Tje(;yZY>7-9k4*SqIizOzRt`Eq8iR^0 zI11MIA@AKv`*nR03lft>#%B^Mo|Op6z|Z49{I7MmO^W2POgPak@)xqCl@;;Bn|&@V zdyD*|l*VLaq%0ACc}6)>5h&INni7Lqo^JgWecH8Rt^8k1tvndSwi30tv{s=mDYd!~ z2#KiQOy010{hFMfIyC@ef(AD=V@?EMHgh6x3c) z?MX7ATf|SIQN(LW7w}y?t&)`Yf`3Rh$1kJ^FUm)8?YHEgNSBNSn)|=VzeW5Uxrm?F zf{$It86j8tP3%AsN-@R!8@YyBhS zJQR#yB+2%Y_~(UUBdpOnaa2Q1JGp4aKQ}?*S)vgN)_K3Ve~R_n+&?nMOQ5m-w*HCw z1;YhR-K&aW89u)`ey+}$nFM9>{!W&D*M8-voDoFx}>*RYZJ;-YX0un%3R-n`SR}|!eoLkUSr%vTEuS189n{Bq)R$FZu@5jFX?t4!@`^+-!+U}%} zYzvPCa|53%)v14(`udF5Iq~b>qkfkWyYlrA1fF=}i4Q)U-D}NWeb-yBNIuHX`SQ!r zx8C~38*kvK)jM_W+P!|8t+!gP<8nFwd+xpOh3BWtnDHiMyLIh0V&tyt_34X$9)IG= z4?p;zSFc{{t+$?E_X=x7si-BR>GTF>!usmA(YL`Imwd$&C^~KhQMX%)TuLO%)p6Vx^x*aVwac^^NGhEpZ)PiJa^X@ zhM9jU`TA?qpMQQD4_3%;&=j;+u3zxy&7XVk-S^^wt+(EK)m2xu9*H2dK zyh@)w>oIQCu5CN;d72CUI}I7i>neay7L^Pb{pX1%@FwqPpMA=MIdkUBnKbFSC!c)! z#g|^>o2wgbw9&@>`mNr1wH&|be|W1AFahf4UpzPYxv5j8zWCC#4jq@$=tvY52Jqob%?+VL~RXeTOd##bX zj#y#&6^ixyt=jM1ci+t&V;*qh)iVs?R6h4VwD{=4pYQk+8|$vyH@?0Z?Ki?h6=e3r z=iGVopP!1+yl(Qs7jr&mCX1fEdW{${f~ShYKdu*~<=L4tXJSZgbiV}6#l z?b;39L#1AL%Jee|P%>9JqR$Feq>Z(lZqxj7EfBDti=veT7=hZsZcT-FEBB0O! z06+jqL_t(ve);7?1%oV)JTm^_hsM79{(H-H?AW76k3N0+>@;X_6`#1aL>@ojZ?xfh z)26+I?YE7-Eyq^_$l9+otG`wJ7weali2u|s2n+gTmnzn8v|p(WZ#sSgUujS&l?VUe zDD|@qr3Y`gQj)mc?I=n_imFa;w`x%#zndEVkN9uWexoIZc+TIuT7@A?YQN3##itp6 zK-%`!7tv5XZ~BDb94ca)E7*unxh|_;tCk%BS<(fO?A2SWGLj6tCnKp0B~?0mDxE|e zz=e32nd1vnm8Cdkn@v8_0X_l(p_HPCTOfGgQh%xL4F7>0Ths^ILU~GM_MaP)vJk84! zQRKCThh?rN#efYMb}yTD7Wg5FPr^S&vK~q`_75^r?6{*8{3CK?LzaOpO5`u={~5R- zjdX5}R7zL`FTo$Z7!DIvoct;$1ipdUR<9oYo9YfYbCpj1n1~?e>mVa%T$As^Ui+Kr|0k5Yw;f>maFqZGcGS@k1O1e{;q;NCtnoIu_vz$&QO8 z=mFvF@INv*+kUq&pw=Go@48AQ|9S1#JX8M`oI_=77mqL4oY!w&Q!+R^{zlEPR%~YN zE6ff51ahn+it_e_x*XqGas-LFf5a!esalq&@K1msX@7oA|6If`_RoP^#K-^OXElY^ zM)fPnY$^H&T2a5jKcs?x3J-Q9B>#wh1_FMBNoI~RLQAq3$J_iIKltaz-xOtbiZxXH zW1NH^3X58j8fWDGVUOc4tkkCoI-^(MpZmW#eo;%4`c1u>DlGZ;8{+5j--!G_?)dv% z`&F$)bUHrs`gLul_>Y>6byP%?`i=gLr^weM&-IOP8)?boMTsq>QUaTHGx6Z4!$ zkY!zJI^d~C&{c&_7pkh_oRWU$tCuI(iR$;<(F9WTS_wZtw9tPAn!8p7q5Q6+sA~dv zrpU@qw%k4YOv^_SQ3X&-rik$;b%LA}Mhf^sjw|!2@YZKxh0$9+k43(V9x%)f(|{(Laq`VgEG2x7ii`$mjlHM$8jWJOTfF_St94wqG{q zUyRS@_>mG?VB3U$;75FF?J$yX#*OV_{Wd~#8wun{2R}o=6{+gsANXp)Usu0@me!7c zV+MRwajM_{MeUaXUB+>iTW-1Vzda)>k~161{lDLT%H%1$pm^6^cRG6iz1r_R_uRuU z2W+2w@=3mP9{tjR_oD%ou_ z#K3G>t8 zDXTs3$x_9MZ(73t%92LijL8=ud29myi0e9vrGOv&C2V}IR43m78!xho`jUBzqGCWU-t(;#4KIx+5h*d7SNc!&l z`MdVpQvVb2lcG_*sXDuepXj^T{_gxa{PVxD{kC1JA5(#I(lh^+2~GNk7V9@UF$DmN zdO-Dojh8ZCh4YDnk~J=nA3-@Rt54LqP#oyfczA+C+6VZ6mz!^reaj+S}5gHlK@l#bGG}n2m$(#ESZ+@0fEb3A*v4h+Raa(JX& zm0(K2pTzLLcEJgYNRxG{o)DIpw{(`3EhHcyo{3mhj-T}U?o&dLsz(oprj%u8~zFWf>Hm=+V8Hr?)vb<53jt+ zDzCosN~_x1=I!^PhaZ|eX)?C+!W!4D|8DJ<{^8tn&!rZ4Ux`VA;Q4>6{puP{{|U)7 zbPFp&jU*OrXCdtbMXLVtL?oKEU&U3{|GPP;WhsOu>as1x|F>joT>l3nDf#p4)U!|d;RbRzI2pg%eY zhC6g!?Y=Y{}PgW<^%9cWl0R~zE5 zwwbei$1OrptOQ7XzZ= zReaNfi+^pB#*@)GzF_#Ks|TUbB%fOz<9H@U^@6A#p`isycRUrL`+jg!zp$f~YDm zUnjy1D6mPV6c6x{5~_lQvb4BC+W)+!qcbI0N`ypIgd#B^UG)IX?EEJfqEo!hmbLBk zzLNZxCkjQ%Wt|I!F#91JzER#ky8Y((cD+`SB0k?8e+%AF^?Y~!{N4F8%_CY!^igq$ z`mX&ZuY7m@oX!g54`M?H2YL=IijnO4B2S@X*MY(NI$rAxCn4y#nO$Eaq_RFO9Xi6oyFA68Mwzt@y*Ih#&l;ewC3H)o-x< zw)nU)#}BY%b%DRFQ6*WR;{PT2Cu%3--M9M>1E7%BbQLpKLz! z^wWnQegt0PrJJ7}`?Kcws%b41{46Ds(@`K2qkjLf?f2SiuYGpHvkY)taNY$=kDq^j z`=u%|Gy?ux$A7VX0g|^b`>_%~|CIKtYe^QjrmX&o?Uyup4^Fl)OjN|G0cnZLWBku$ zsjf~L+Hn7@K%$VJjX;cX3hl*3ui`6}r%fq7xv1tTMFX;Oed+N56d9i!SDNwo^oyR48HUqvJtixV>6-wZ%PzYO|k~$aA z842l<3Ic&hB1#iFSsd{fr*;9pPJ;jiG6;wbWRxy4GDM06%f8a&O$@dEG!lVWjC4pu z1z4rxfsg;gr@#bDm7{6q$z5O1hEjZM-VU*mWe-4a>OVDrr221 zZNy)tDQ}1JtD?L*5*p!j~DUYrps9>o7uRIA^h zG5?tHhfNSn)kaCJ_9#O^JRy8)STJ$?YE-{rwehs6=&rC$_2`1iO-teY#EL4%FN~nL zcz9t$bc~p#v6H+ZqKbs2onIC4GycMNdfEX0@8%CHV8uzKAyzrRVm}#@;fuM9`4;?} zV_PZqiM0_^!&p)I6(y|%6rLQBZTf|q7 zr>w6(zNog{X{NgXU#JfAX!TZEdJTZJgu2GB?kOp=QD7EJgYUIQP=96$VHvBbpN zi9h&CE#RXl{Et}h5C1pukNv7qwD6A=Ho}({XE*T=$whpLHS>S)hY0}TAB(ksFFTX1 z;c97X(LYW2NBvUQP2$Ho1tN{*(uW^@@c!)Cz1QyDrnP4DP+DXEI7NxnZq%=gj#N9K z3IE7UYx_r5+5f-%<*#^+1(;9&_G$a}?JWavvO33)_{{ykr218H;|ty97^c=w%m4c>`qfV8w2pNgJ;dHa3)op&}^ukUA{ zf5Asv&OiS=)sE{Wcs8luJnNs-ewmUfB4X~#ybPj9h*p?bDLN|s=QnJ>#r`>r!ipyR z70Cay_S?31f0`jRICr|mv%FB*OP(Rg^(*r{ETRfBDK_%>k%-b&YlhKw z&L8>QKQ1dplFS!pr~1t+C97Ygk{64J_WqNPoTRK(qUPHeslg!@V_blmg=`f{}l0|6pQOiCudFkld`Kyyq>^6b|b{|_$l(w z9D!7-Sru`pQ2viuieJ}SyKCrVo@~r&;R!J%M7>=j(Kq8?z{1EVum~}yx+OMxWaxWOWMgIBmS5aaggO8KJXB`|UZ2;1;9(O z;$jI@j%8PgMw0y_eaVJ$Y8v3vTUDjpL_8`&ko*= zKQ`w{JpNXv&-&=vB~$traZrBIzw`W!481qk`LmJ@`$V)cKZ&Hv_#A&cZT?)Irs;#b z`tRz$Q~bmzM*RF*`7ie|a{DFce{*~|ef(X<=P^I&{;;KYq#|{bN2euMJN6o0XIezV*dO zQrEzDnD>7Ufjsn43M&e*ReBP&(dTmbkBa}&f^OtPu_uE6{`qJAx%=L$a7@42D_7nSW_LH_Wqa`PXq*?9QXHp)9{}N||tO1vwZ0vIz|4Q>; zbx{TTH;(K6>c3jjI{s?P>-f7BAFee2#d8TYtN*G3#+s#OM63U1C|dn@^GVwk)8jxI~!|eoO8p3RL7XAq)Ep|5}YD) zn#g%QpQDtCZPvR`wkeYz-A5wh=u7?M5vmUWUX>txS5_FZPIW6a$9o!=k7|L^O*?tR^R-se2$JkNQ~q{#N9 zr9JPmUL6`#a7ont5HBGk?=pjR{gL;Wk@DFW-z8zO`!3NWm$!4>_;d|kUeZlDak_$y zEp%TP&zxXCNtQTp>^^auFBZ8kKUV(OOVhj7PZ7M*0igd$De(gfVrS^HUB#3o{4%;_ zRFQL9iVV7A4Pgc(V<>BRMgfdnFw-GlJixN-e6oy_g3Noc*AN*d$9F&Jn2{Uqi6V_4 z^L5NNsj-DB?qz5AT}UqI4x_))7YOv^&suX?9iiO4xrbyU1uQ=t+$GI^;g6a65iIzo z-b;rm+B%;N5qfv8Fr%DJq>=uPDD}m4@7sZ#?kOpC#&oa8?-JjCalsIhDFx| zmAV4*2(^|Bf#O;F@5Zpq>&F%r@)&dJH~b{UWRkrWFNpyqvwp9`Wr6A?b;#V#dsl+j z!-EZCyoCI!5+Ti@j*sxl@O7@3L0zf1QTm5L^;v@2(UPr++vg&2UKeKYG0=E)n9D4h z+f&&89X|R1$K5-yxdsaHW~VahR@{C@Qroq;ql<_HrqCZ>afQ?I$WT`>47X(f1Zt_)zDG^pMj zi+(V94|tyM4YtIhnY~E6^Myd!fxUIIrzlX~)2lP2HgS4zd^)*&NJ`U~5psLhxRQ0* z7TP>6BQ&PZ>?y)uT{5*fwo)^#<%cHkTZgw?3JCn~)!ocwrB1CQK`UE3t72x%EndzK z_#%XnJI?g)`m4vK7-{L>#aV{Fmt$!=;<)(f+3?Z%H(s{5D>#dT%EgNZmi2V}9uL3U z@MA+Hkqt-qRZp;Xp0URd> zocT@^5+-LnN~6>uCh!wouQ^E-s^_u)Q&{qV7dVuvEZn}pSo*J@>y#O)o~aZKWT4?>ImW>mlm59gHh-!rg61*%*sF6pV=UP8yt5`pQoAO zKxU?aC5k(DTgAWYdQx&Sr5}(K|bW(5r>V#XF;ix6@Vug`jCs;I;U=?Jlj_?~y~3 z1+R^_XHSZ}NiptnoNDCoeCQX9h|vU@ z6RGP8Q)WJ&~Hqh!Sqkp)Nl&FCtY-GeIr5*IzqpR{u z$>hCpG^x~o{1=kz7@MRdyNvpW_$3JAgi-6pxz87d#O+_%LV_NYN7P^}j6rj*Jl9@Q ztZ2G_JylP_GAVpY!*2LfQjIG0JWqQ?vrKi;XS^~>M{IC)+R9wqF8xcG{E1Dm6z!|l z6vXoek;I8}H)M=BrWVxhN+-yrnP2Tv*vfB1S2gNs4 zmp<~})4D)@$VNZK9O++C4my+CrHvuv53oN@z<;<&@^Tu`#nHTM)(D=}%c18N6BDeC zu->Hmvvx&yk`G2@MyGaS#-UG}>mUYxNN&co5 zHzGM-$~(y5Eb8DU=7Lhl)Bse~z>%Q(rn)P#mZPj)XzvP)~VVd3sU-V`wNu7YCt{&Xl_ezV%4`svh{+^GvG?khTs%7wOG|0bCV-#Fg?q-_`f z#zJ^GyV2o*`gK;*t8@lildZ7~M%;F7;OKN;>h~11a+~d_Hq}ce`T7OKEKS=~SKWUR zt{8))+Pd4eK87VY@sJogs6_SK?GZ_qzjNL0bXS62i~jwOt*xuqslwwMnc?^;zmGjx zcO5~nbm*oI8rPeBFrY^?&5JpyR`1Jr-v389og4aPkiyA}eYl+maJ#{6n5aE+ z0Yib3L?bzotZ#9K4PS(}9*5OxT}mBB8?OI!ABxJg0*r7ih~_c-}M7%zEd)d zxDZ#^kW1wkD=65;gxs(q(I5T0V|fzqdxX72lLRc@+a}DY=per@?-{Wpw5 z{M%Wu;~0N|z0H!a`vo<_?CFZ;DwQ18$%=b2r#CzIMR3gcJGml zN>|%tl(+V>1$xr!uRU}<)R>tO^)2mmS8)s>*^}lMXjE_Q=RgBkVn}rwFsFZc^;(d# zr}`kVaBJ+(=}fdE7a@M3>3-p%lbL`vkZ;X1f!&JazrW{+NG-Ry}#_P z)HHo%3V|PDH)>qr0!LtsQSclVEhtfifU-dyRxoA~(qN@fclU_NX16NT{V#)3*l{1%RADrL=D;Jw zE=!b)WG=n}oLTPJLjUM@BTcGT4}gZ)sEk0Tz^JNp$0Nk|ORfo4yuc*BVNLixSp;%A z;10CSfM0^%>_n2|8+X(g#Xl=bAl5NXCd&bWdQVkG6@YSr*>Q--r=-iZL#W9EY{eES zJpohENr-t8%wF?h*4Xx9zw*&)IrAjI7)*yTl5=b8lPB5yZVw%9RcFLvNnH&2FfOlo+KU4dc5w*HXMJ(9MOFvjJ|(0A*; z8QJ_aD-(88FAnfGLLv)4_m7}6rjRDqXy{S$<3@64-m^FQs=RZDPHV^zI1>K4r+Wde zr@2oJ#4?eLGK`~S&HvIlx^7S48%WNPs~zsu!?UckJA6&ra?z?w?H0n8VfD_ z82Si9^qyA4RV{q7cyjmB8Z*$%L@uKk0O!6i6889fAYI=ZUvn>Mo2FlSc35srUTGy< zH4#Ayf`U!n%+E-EH+ZzQh4|)7rkmXtB`GNU?0LGjXrsUa=QJ9yaOEN~pw)Ox72a^X z!}AlA?MwY|{P{F6Wt*vcp11}sWP0dy6ryVwNdBsR_hLRysvqr?dBhz0G;f3A&_FIG zJqnBSD*z>stQE3@!n(sEVCoj>Mtl^V9hX|O3OA`{TR-r7T-gx%WI7uxYYKzO0L{{e z^fhm)C5}7cNj^zcAEtsz49aMsDNwmK1Wxo_Lr(_f8Gs2~1O0K4;yAPHbf zF(`$6xvkHxFUaTrz(i`;U6J+h=<890npm{80q?Q?0n|SP*P3Zv&Z9{O@=yN&l&CtD z>3*JCQ;G5hm6WPT`F`3c>=4mogAMYRK6B@5DAE!!DP`lX+Z7GAl!&->w4&rSvR0u{ z^Gtf;%O(B6;A&BE6~AiJY+pf{#lfgsgKhG6mW8Jd6trBHX;nUQq3xeVFg;O5wc(XL zM89O;hLUjHNB3+N^htswnSJ-JVp)bd+|=pc0e#$z`j!5AHhWv)1Acs zf_4fw)|%W=vGBvqQ-&L%q0>|!Ec7G~C#xICrH8}b0<$Urt7#Z7DT$KM9;IM=2?%j0 zlq1G4HpZoJ3feH+-#gy3IFn}x>NVxPmX+*$=pJGin7a9J$NmUAdi0`K=EOY$T?u{Q zFUkj*IVr{7=(4Hzx$Q-S_VJY+=r(wN3A2U1dQc8}k+_cK58-p5IQUnMlu}W)?Edj( z4>r;Al`yOC<3}iOl507qc1GUrXpcyEEWTnK(E3!38tJSnjy-Co)SJ3tELeX~Y&FC7 zF8b9v#=cfJ-}!uY0d(aZ=?KZ89pYxQjzRV8zquJ!&Wn^SeP*`xMg8pi9_{>*FZ@lv zxv$sL^K8aa1Q+;j-^s}rSi-w^|1#ErlbLzKwaE2r?5JzxpXfz0f7|I$zy{M0NEqe% z$dKxnWo%9kUN=8mo8-u4le?Tbg$eNu@ksj*lpC*xAG*U&52H}ffPUL4KXNXr*L&~) zb#~0}$NPt8C9b{$uASV=xCi})g4WH==1&w%oH&hnU|Se^fXoS5+_5O&u>W85)jZLMg#gZ*>3b= zGvWN7X5hF3^<&+FFcVG^poX_FP1z>JR!1JRwa!OT3)(QdwJpodAJy;+ z=;9m5vypEG;^ZqPu}ajH*WEX z&KCQ@>v}DR4Fa5H8wHBM^4y-95Yw&3IaP`b_A{N^CicR2qODY5wrzF1W{}S&Hgk^@ zLou%5=OgMz3)ox*4aot!nP9gT0%y(uu2?so#s;_%7Q(4%EY`TmQRosr9<#HsNi*`V z^@+isYbjBe8n0fM*&}be+cfN`LA#^_CYIjBvuU{^!CKL=jIVF+bqZskDE^=ew0b~p zy;=SJS2+B!T-!0lb@gL%Vo#o{-znlOv+rFIHD5yhjPNi$vbMIP1AP<*aR@z9_cS#l zF)RfD5B@u+ku}DaWsRCSCY*RzU0WNVWolMKM)-Rcs>rFjNU3zU|wtoDg9OWXdmL)rnn`e8b~WrJhUa*Ll7ERY5Q!Nmx@GZ9FA8SHL=e zcB#M9>W)`vfS0f3+GXZ0^@;`dpmv2fO_$TUs-x>G)0`;neGCKnuEG9a)>7;!S_ZPp zzBWzI)J!U_`H{7P4wYFtaM(7%^#iukN21_JrPYLNSpNP+uOU;~^*S}77G*Hwyn4rr zr=EqUy&42-lJ>ah%V>W{Q>K5D_}o;a7Ne5%>ur>CT{+cBEII_*B%8iE-pY78W=N=k zF`S782!(7qupuM2G|0_FsbAH}i~|Q4P5YSl`iRdz|M37Nw@)c~Lh42QI1uwvvV)T5 zvwA9SFmeCY!rCfkD#Iqrxtao9hVW(dPH$$`<5GtFH&Cv|D`sThjhfaEvUFj((opNe z(wXS{9GuppM)h`8)3J_qMF!=tqLL4;@vKr=-u9k3fHx zFZRu3Yc{)Q{*WFA-ro4CDOI1Oz2hVwnx>gXe_7|&seMk5;+?f zjyyNp71B`{4|no7{9>%g1mQ*!)*mO!crCjxTG`sJKhUC$c5-0woe4vyDN;IA=<9F1 zii2)j-D!$AxSP;&C!y=#bs6D^VKs2F6<^0gf91okth$%8$^QLWCpx6T|A3#8d_Q|M zH&?yfIX>j~8A)Bzlf%A1=02>t7In@&%ndck+tU599Bz2?p4tsRaA8Ek*NS<&Jdal*4?4QyIZhlr}&#rgKWobX%9Us^QmgBM5U>(q{8b+UDt%5PZv+ zU_m{t*KZS~h$pF>_b%ecG{D_u!uu!$u7WT5LbxRFFdFo)5B!Ea@Um3d)y;t~1xq4mr3f=i~#5?(Ox|%0dI*^n5Yg_K* zNv5UtC)o-Wu!Z0qZQpru%lE$^(T|NQx9m?j=a9QMcr*c+pzKc49q)c#^b6A9;_wt~ zGa&@NqHtaLKl~C!Jwx#LH>-WobWY|VBKCDR2mi|BhyIGFVg;P5%rm7NSn^%@jl@sS zxzF%a7NaA*%KI-xXp=j$f2PBQoB)Tc62)ZY$^uh9V0i#HVe zx46Nl6g-W--mBkK@ce=s|2eCfIag%lh0Ql>e`yeJn#tVJGB_+3Pzrc|yFzMsu(Kpb zwie_u$6-t3t{-Rk4Tod&3hG|GP<3*Zvp+yE7`ewgyy@56!b#CbE?eSE!(2)%Z^U0C zO*1@~70=9n_;BxDAw6TDRHEAs8n3%)PBP#Gi-*m)k-u^>H2s~MQ`iBmonv=*eWn7V zse-JSz$(rc$f9yW6q@LM@e#)o^YqjDaUk<%SH`U~m)?NGyM;z&5*46nlQIW*mLAu3 zVhZ5ON}=-Qy~-+=9WMTpXer~*kbl9$&Y!wge|AGUuL-96T-KSsUq5d~|KLx4^6)K_ z?VQIqJ->L-0gGK%jubvubnDaInEwy;(!J81aOb)V;*CG^!!Y%aeGPlrF&}j-leTuR zu86h@&xfbI`KQ-~J&48t!o~{HE;8M)^=)XO5OoZJQbDD=dNSVSj|$kw>|Mv!>r!kM zN*tpR+6{{P?8nSu11jO_|M7QYO+FS#7KdlGiVrL0Cbva3-)#R>c zKF|A1?fzW;Mg1|vYhCKe4z9uGXpcJdVtddhiA;Ex%FkBy>jBOzx9>Nm+_b@I@~Dj# z=2j$jL1u?HnAvP9jAY@*y^cRLOfq2gVt;Oxyq~s0{G~A$)o6_R===OyCJ$(9B4b?B zKuBWAv$7l5yg)h^K{n8yF03yZpJXkVxPO}VZY52bpK3LU)y8M$AfSF}lhI)bOJ5iu zBA_yngsd-R`lc?<)>ozHK4Oqk@-#3<BTGJAeMxTEriAhB zV2$54Uk9D!j;C0y?57caF+%;WegDGhnoSRY22w)g;86 zVwxOe`?~angnZRZBti}w)WH1go#aCCe&}OeF#<6?ll3!gi5;6o;a?gQb1h9t&VgI)q%6^Zu7rx%z2g9DT%2Uq*#=hO-3H?4 zPwv^lZBF_TLJcNp6FSrtdPsU3xwZDvt%>YrU#oN4i|#_{3!`;7_joW2Y*4N0dR^?j zKB=G9UBbANX=Z{z<@i=A{YcS$u7YL_HXd7%mKG2pY$&2wmbcD7!dwPsQ{#8 z&}%w~kMP$E2hT2(!ollfoxeBN@Kq=0pC@Lp|A`#N7SjGgadZ3x9JA-1M-Cwl7&&v? z+Q@uE-9D!n^4Umfb`*^aG=C0dXbUMZSOZSXL4$U2`9S*bbQhuZ@}L{X@gtj0>K&+iN`O{5 z8#UlbqTb|sMroVklng!w?0Qls#S1JAVaM(gCJ80FJygYAGfi_tv~O=q~Y%9uwU0bsZb4k zxSuBvdkfwj^O9}igqy2O67?uTbTudg*c3!U07J6uMT%W_Ru3cc}?J1WJO=xDiR+B_C~X1(Hs(_({M0Dnd82<<~xceD9K^a*n=rSGxka z$1!{)w;tPe$_u86SX(*-xSllcSudCc^l` zC)+mf3&+F}D9Dc_|2eX_#?F%@)b8_g@KEp-ooa^mQ1C5?0;jka^OUCI;j}^tme;;H zV@V3x#*FBED3P$wPt44E@Qi?&WMIi3FKnaS@8~{EHgWiUN}Yv5&?y4Akg~UdxTmsy zN4G<@ml{yN@N3k=7;1M%koO?whcHkr`!oVhd?w1m%BC;zQ~10Y1=`;nRDuOt39< zR&2kId_>1d(72NJ?Hf};G5+r(;p?qlkQ2quW8jZ^`!!QJZ-Dl0URXp z$0c7S`IWniI6zy^tp~8c6I!61#RyiQF663hJ(B%*y#iVaAFzt;MlemKU0^QlK>G)<}EouR*L39$VPrG@hrbF)Tp@=`#0!e;q z!2SjGdZ&eo(~T+w{PX-)e>Xm#8%AJgxP!0Jwj4hSj|@~l+FO`}&XvVyUlenX!^jc- z{A*3ObRifAs+$j@DTsW#haghr2<})f!2F{-tE;`odOzOBOPtM%bXud2avZjFbrf}i z2T%zrykfBUiPM?c;MTV!$~>vOyXWGOgUOxQ2YzRBd*C0bZy!3@?bIDqm7SE&jQNag z%SR4%mCN)$!_jAt`B~g?d-(R*ri3T+V1W`ipi?~rdXJi;@%np^yu63H-yni$Jik@o zjK%qPtCeUhm%7@@%m))2;D?q>MetXhL7ymhg!` zm#NfSd)$GJZ5e!?h6Jq0=m(XM$|SJHKk7&{>#<(Yc`rwjo1vw}o^^QS@D9~Ek?6zc z(RUPl|4qxaNGcf27+Oc8(cWb7Ho`*LFn&R19SQ*m-cq8}MZxrgVv`(ViXn{97Nvvh zK`x+4A2n_La6kw+I%pH$;HQE7S{}< zUiX@e1^e;72W}t!W6xqpY`7P=a%-OLTq?u&slSBDPbj~~!A^TaC}aaw9PhVMX`|Qi z0o_<2Q&L3{$dhjd9#b^Wfoay4T^E(6q$T7sg%A?t|%S2 z_*HM@YKl|`V2Yw}y*Z}M(x@~c!Kp!chhxp+O_m3j~HJ4?j$SpUDr{J&+* z#-qf7p1-QH@Mq_guEiuu+IC6BTcXzF0<_j0hy&(NJ6Zc~E0Q)5BlN@b^&baH!}cqj z%ZcToWK%`M?gwE=4vnfef7nf|P;Jp2Hfd{UOGN^PkmCCH3)1f$E|D`(n7>Q+GT4*X z4-1D?HMoKizZWJ`l5eSAf3Fqf9o75os5o083OI7#?C{VdcnEXm9}|tKqIo-M>&4CV zE?;());d=`KY=F1yD&14wwir}LN5&?UOqM&bLF=gVC#KX4t_6fdfv^qa2c9gLl4(e z1#oFiT{|^aX6MhiIAQd&nkjwkYP<$NsfL8%rRTg6kNU^rvJE9fYA4jwXKeq53I6>p>EmqF??^QvL9mjC=HdD zE>;p*hUm=ePwh`;8%_6!j9TrI`t~zRdZ-o8tIhKxXXLbDsdeidt@<&MjfX#$RMr;b zT8zA3>Ah}-M*L7evPxZ^I$#PjeUr_o)vG#6QP7-J>(=RFeN|&gVsEZ4_1Y_#zUsxQ zyOKASX#b;>5%aBPh5F%w82q_^=>;CoQsuL~iswT&LYf+8J*Dh_XWdX|J&^LoQyE8G zJ`!;7YX2%Ql?H?OqNw3GPqBz_w4i4JtJbz%7)MuKI()QX(Zxil3XZ!7W zMCo{+V#@RNJh%QMx3QvQ&`H3m$Yu^9r^t4hr?_s?Rfx@A2yyNoh9)uGtD^u3>N(+nenIj~yEp z0gnX580K=I<~N(*j1~CoQ_Jabz+yV_!RS<^`yzA0vX}eBEmswSv8mF6_m{_#KQ5gW zy9w2BE5*<39w>Mx-zm7?#q*E_;8@&?I z8;S>Eb`!Mj{t~h;4rqn6*w-GchjhS%wyatUv`K1^1-Yd;eQAi)L-0QLamF=NpwQy^ zen3FbQ;UQOzjm*5y9@Kbd!<5`5MRub8ozc11A61@sq5H&Hs2_C@tAYP!nGh#3XfT3 zZP5Rsf~Me00mnY$s^tr9ZqXOZk5AJ!rI4U+ZU?U7(^KN2yZH$c;g4hT`dgx3F%Se@D*|ID7sg(|_QgOf*{QjreHt z-)vwlcg~;JMj`n2Z_{Inf#a@21M!0S8!2UT#8jOr5)GLAl zTCH5yN&RTCvk8LM>itwJrKHsS^1i1Zu3&(5X`6!bYryjI4z2Lh($++$p653MxEw_= zVRj=b6kiSrg$TZN%wmtOQQ*s$3x9+Dhz*y&Tu6!;t>@y8Zy8~|bdI*|g6gAn?r8;S z0yN=WK^VEv@GBw1TkCZaLV{V6b%dgXbzNa;XZRRYMjKR@-IPy!Y*s^n( zBs~-efB5@_oYr|C+BQM<&c$59V|tR<4^H$Uiv%IncQL7I+=NxJa3MU6AZZTOCxN?> z_dy;pa`6%yDHN8r3O`Kf;-Va)`!EwIX~%wT??@|cg3`PtjUH7RV`8 z<0wQtj?sp~B!jGENn(~+R5W72HM^m3+B`C`Klw(ZOu~idvF8K7H3jNU1INw0aKW@a2QMQHnt}mO2zESj}+&=^8 z1#Oq#EHAgtx)CG28DHLHah#=@f8zB-U*XmdRF{nlBBsqBJO=imn$4DyjRk>7Fa>^; z&reK3l$S`b1;*=g0ll)Dh{%4#P0#k-O0#Q0`-IYK+~9OvqT6OjRn{6>$C?2#_e+iG z=*#Q-;YRA^+ZdW=o)Y|=WBTbez+GfVnE}XrNR#o}WsaZ!D;l9=7om8!BZBJ^QY`Lj zv0J_F+2oT`6DWh5Bxo7A@yqvi~-IK_2XaPZp5KN`<6NWi68>D?P z#k(#cu!8+jouSAr$t3c_YW0!yvcySx--(KHXhklS*CqSOT0iuPCBkz+AL#vvd3Uk7i!}v7YDN>G;WaEoDFsk-#0J z(KtX5?@k1+sU9slULo;Q7}QrO6#W9Y{3tpmGu(Bf9yvtpv<sFx@L z_iq9g?Z-zre01?#%)P3w?@6V>Uz?`p+t)9Lau9l@<}FC~3a-l$8!SFi`k=qkUZuq* zx1$eroBi#lLSo=kiKAv|U8p0hfs-v;!Bf1pE>!QTn9w096I3ylI5oeapRQlB*`d1A zj`*^GAsVybLia^b+-J|mxk=YEL&O6z@^UdZl5gy2yDqR`RfSn5{0Sk;+OG$wA;@JX0G47=TIVGQ?|CLRzEHU-FcuOMZMJpcd&f%t^7W!L zt%pV8Bd4K-U(=mF-3F&5G{(cHI{Pg;TeaAxmk{q}H{9Z-cXF_NV1At5^FSmKYwrqGC?3 zNcbu{Nxg2%4LaHW657;*~JuPgA##l%GsyJS8o^j zmwVLnrxyT@jL8l>UK)JSeexyu3$K`J{XMRa-4If~rRjGdWAF-kqpqcVQq8%)Dw98u z!Bp<4X200qy@V>scj@MZmIkM+gMM2Q;!c_4F1?m!bCj|DG$Z%IWeeMM)4d&Dd-sZ4 zEuL$u8miXy(T^bY!%lnamrzX9*cuh8{&Ww<2RmD+r!(XYuOkLO$hX&le_DSI<~Sz& zZ>2o40NO#UZz8B*r&{4Cf|^a~sjz#b-*oAGpX5W5^&?cK8(QngKkzC^3x!*u8$1pa z61sU`>3I%?5DZ%z9FXnnxEASLzl-e*pKWPbaSyww{Hp2?8ZT8o;s1Q^dRlWOn7NLk zm|qwzPYJLp7E5_a+w0uW?)AFjQUtI$EE`)AKd-kDUjG>4k1%LXnYr6*U4)aj1C<}N_<}B<;snZIkk7Qz8P=F!n2`a7ij$EQHNS#`rN&r&va=zTX40dPT^QeMA zy5PH#Ejo|xNmKl&TH$3Qo9ld)u+p)xVD5+|ecwBcS)%T{CP+ZJ!8wee3v9G+0 z?b=a?3W?&+o-jz&ZwRzwUL1?8*wGL!8Q8tRn3EU9Z<84=7g&adsw3aPC0FE z7^E2+wz6xHqu@&PfQUyTh443E57z(fsx=DW@z{`API8|Je;ii_aWsA{WOMA+AcX^U zVc)@KqWi(*IN)hS4m`s8k&qi{*pu|Q)=37c9yPV>3s>?f$|r$63#*d%R#;*Qe*m!* zXj#&YN<`8=GFc-aJd1xBbZvK$dR{x<>Z=<1>4d7Zp|HW&9c0idX~J8Cvqo<0*(Nu( z`7;0`(+2HqR`@fN*^yrLAjl%iX&?(N*RtZgG)Y`->ah$(KjziNFRVk8l4!4mYNf2{ z%zLf~&MW z|I^jX+3Hk0rEPmpOy_!qD;Z7suOXC~UyAE@1@4hpAXn(6>+d$x_*!D<9}OtcnCX}_ zf?L!~Rkj@*Z{}v@1{V4C#3+vnM4gZ2U^S9l5M|w8(f&*~cHya4Q1Wi3Jzgx)cDN(w&oH=&- z8;&iiEO!y?4DUKQJ2x-tv4VW#o$wVGlDgA3eW^pC#IQ@M&V4zr0UBXV7t&hBsI6#S z`ww|HL7FA0HdmK-U2|dqbEAeOw$$4t401skJw508A&*az@^_GOKbuYDfZRw~Qk6*c zAR*L9JMB$TlTWy2C|g>b;BOA=^26G%)hV!0#Hkh_7IjI0v+el|u?OyAF8cGu)rUA$PMc~@<7bq{S}3faKVOA1uey%<}vLR zC|?TZ(UYH-=l*UY>M7w43}^rB{(p-XBe3j%OQF0tXb>;Z`^U{+mO|r}f!}`dCox7C zLpb{rvv;-=1K$n2LDNH$v3pCP><#NuXB%4Lg>c~F;h_vSgN-?qr9d8$x13}VH-9U)*LHIA_ zR&4z07t95JFE;Z%iM#1xd=Q8Z+UQ3$En`Tl#$^ZVr=;`3Cgc-(6g=%*06K)hXinJ6 z?}f4U4(mDIja?mggSn4SE?8RS4&A*xtA6?9a=D&9RTArz*H|M>8s8W3G0x35pBgV~ zCgoMHE~Re>FC?AHVTWek!itTv75kKV{nzq4qG)?K{W}EzIV$>5*7PnVEUqvx&kmWJ zGJtt*PAiuQyKk$`znUy02-m#jy88XS(sf;uW94=0bhgiUdAAFNdCi0sQIh&JONg#X zRMg1mg$INH8*X42jT6Q!AUY&Ouh)wIC%lXyJ100a|McDXf}3*FGPjoWNW4XTJso`j^99H<(-evP1`EFJuGoufR zYz#TQ9vzNk78Rll6fY$nE}RE=#Lm8h=44|(lUt!$l%MR>f?UHNz+5$OV?4e)up^zy z>28{Ec)>3bLDU7Vp0CE^brjnfk;Q zt0c?bU*mTKFm{!YKV(7=93g1&RnItn{6WkW!>_0lp5b=Vmy1$#GxoJ2y=6pdE~f4|L#dXl$7rofWxH1h6p1<^)5+Hv^83-sVugMiY}Hkwc+QAa<~F17!Yc${M5VT2#@)z{kkTZ zOqb+-2DsU{C3|3p#O^LT;prfgbP8yc^zUf}UDU z1cRE|a`zMg?$tHI^Pta%<3I|2cpr<|zuc|{5OIGI^c6Fejg|?gO_C_##?idMv|f0@ z(d<)yQmrvaXlUD#826N>)In~LE$X}PF29t>j2lgbOs7mXMelR+)+0ZJi?!kEBq_lHwc;-@RzUu*AZ8&h#l zVG5J-Y!93t#Q9qW8$D9WwY#7~zh)ja)1)y&nP-k!^E%!)J#{fwP-uHEyxeinP`*4= zI@8-Z&VWT+6v_;qNN>Y&nTds*Q~tedzjEou8LDSQDA`3~Su6rdwLiA*t$k>S^-(_F7rOgcE#m`n8LrlK8H9HWo>0FKW_6m3%*R6jMzz;s0!ykPo{8nEIN4;|%S4pBVk& zZJ&63I&NqL9BmRx)TuelmkRau{XSkJy6=DWOjRbj5Pd66mYB~r>!fxI4+~ovfj`e6 z+1-4k>RAQ-l>Ok#O!C-28&pcneVX*08IYRlGQL8Z%63yODj8;|TQl$y+Ct}#^PTM+ z$2pWj|Ar}~?;qa}KJJ%_j{|=GtGRJ)oWAxRH^2IaZCcBP*?nbj4!#)Y^2VLkGF0?J zKp@L6oA){3m_4HFUn=>6zNHVPu5v$YGm5`sAs!cv*P>HasE$_Bpzr$x_5QUtc{{r7 zLrFNsu!Aa;k4J3R>kmV4vWB3V!mH0cQu2jL3%2osN=H*z{pB?2m>X4lNKL8 z*-W~}A_XHE4GAH@ks=~+p&m|7Lo|*RV)*&cU@6C`={m+1`u#y)X?`6e`%~72&E9&n z#A!yohw(6&q}87(KlZx7_Nt_3bYOL(34JQLBs8YPbIf@{rTmpYjqc`5Du zg*MWHa4gyOWcKfkNU5Zuq}#JT)chkFjSuP1_0e8#?DmdzjXl3Yoxs;9i5zWc{d@B7 zE!f&N(PMR@`dP%t&xsSv$IOzJ`0KSO5Wja+d}9!W=dLP;o0O%+mBZU7_RF#hDn;$h zN?uD@gwg|RXilIuUo)ou5?J6HXc`~tLZQThDfZvj#asyWpxsP3pa7khs;x5TuHz%k z{r!g(s9Zoy)#lvc`5Y?aGgY*dqSt%@pr~Ahx%ryL(RnWHh23HLVZ%pL-+aMP?1=uY z=R%0@h;L=KxC%WUanQ8v@+9ExozIl@51kkL0U_!o@vU}l9=(c)c~~U)xZ@?+Hx^zv z1F5VJ^F|$pg0}E+DTXu7+^JzIXKpXf3xAuWslak*h)c{Ru$>FIXrsiYk5P|b4x6&!2@N2 z`rW`*dHFut@x$LQ^FE4{hSsN=4PNRUyyZmUVccit4&by4rHduV+?Bv^$vj$hW3tQE z(LA01(lo@)O`;o@E-FM5dk7I6V%Lrl$}hKv(Owv*AmqGT{rOitSaqYmWH;?1^%(^I1$nIR6TOeH z(t8ex6)Txpa?Sk2WY>7tiAK$0J2)_9^X?w0Va{DNra)OKAM5yqy<@2Y23H z1_rC6=}==YNjxw6S#Yvr=Gy?lZi$(za(Tg+MG8>l7Y$=XOULs{k@qjOt!SWXuDP{a zS*{+%eq9_o=X+_w*#NVg9{Yf&#y0vM82i@n6ZD3kQ@3;A;;*n4Rr5nD&AtWnT_P2u ze@Y|4J|GIP%%;YQ-T0G|j@Lp7JSUpIsI%{oUs1&`5bpFvXH7r&0NGN6kmvk1tR-6O zT=w=)mV9|d!c=c+EVhKh-`V$I`w;?sIn~{6#;Bm2_0NaJG>m6FByO8sUs8$l|7bc7 zcPbzM|5rjqBrCf@va<>2^tMNwA}eyTvsX6fgpiDry`7M3vdK7+J+rrC?{&ZK^&#Tekf<($zT(fWF6$tO=b(@h%tr83fz} z6HJ)V&OEimCxZWS(U^_%|E=S=D?fTf-_@2ReGFVbX5?|(g7bM$@-?zgg`}kcuEwsE z&&RIb5ACC4EJDnB!71`fh%MCMA~V&J5Zo2gG6M}brZRc%I|{RIHs98a)L9eoOa>29 z&5FmHKizDLx&o8FvcWUSoy!h*tor;B-@K@dvpaR5hDHlOx15(5xQje+i-VryWe40v^oA+&m4gDuF@ddQZ%@3Yjhiliuda)SI`bYuf-PstFtMZ)tt3R z{Uxs`o!pe|f}H`#kcF z26H9+EI-FWdq%U_R#)AWjcofdQx?cHkSr0-3dp`gAR@4fSNIN`OV;A5@Jz1o+{$KU^paljd+C8#dz8rB z$%2~`&%)%t#voPpq`KC4y3HUTmB__%(44l@M}E6B;U%(0`a(Q8GxeKEYwon%-lv?X zxee!8Z5|GbM9uB-UUGloS3G&~y0Zp9Uc?srUy|K4{LqTTnV-zkgsoz)AeM2xt0oRA zbHOpQxL-&>dysWM`sn60U42fUfg@atzk|s;!!0Mz%_6fOuQt}wfIh-`J0+t567g0+ z&OVW7ms|VTWDxnPD;f6rsUi0KlTEQBs`uVGnu|v^*^@QAj|Gdx0W@E%umC}7=WTeJOB{F^DO z8$Bk;vwksfrP0$(&{EHFZE^+xLTE&`&?!BtS59C0!u7gt3J z?NM_2787<)BFJ1;R52Q81n)xnQrAwupu=10+M#Zt`Ep-FYv>5nYOjy_j^fCZYJ(;> zBLYn={z|jX7X$wcL{l%?o5jW8&8SFhJh=Yx^C=^botI?QBKH)>w!_|Wq>K?bfLbs(ppFPo5Dj<53=)TY@RdO_?tc087gpY0k_myL%4(T0+wIQkD$wH3smpD531p50Q`#T~4P_;C+B zkMwJ_Y87psT9WfpR^h{5}>tN-!7lpW-QBAUFzDmsEv`!)>#;G8qu^OcUv8uZ!% zT7Ok|0=jB9NIuwohscJjWp`Cox!^asFU5f#94OH|_u1`_kRQawoR5y#JSh2x0@~@v z&EYgIqHEi%&4~5iCPSBkI5#+qh7@t4jlaUq*!7Yx9OD22y*%dG_=O4OGGzc?4)X61;EqDBkNLT&=*+;-_y0*7F39 zIs0L!vv|@57+~Pd_?vobEZ_dddJyrwJ2_@l0?6&yG3k_Gv0F^TX}Q^#MP^9eNBmw3 z3K!2&^y5}xdoO>K!+D^XfWlI-sts|ze9;Q%1dTnHP%&%{^V{V>90ui{Rf1DtckAvG z*Q!rVXxpQ2Va~tmNvg{5{+lvqg$19PiPi$9t#5Z z*iY8bcjVe~HZOfu$Hyg&Xa_HV_LFNLykU0_M1$E#CcqDurUc6&U0N>eA!j>M3G}2) z4E<>gk-x2fkV(qYkjx%hT0s0I3khva39Rc@W&{)HBT0STIgoB2Iw%4s4z3{S=PKu^ zM&Dr-=R&E7|9p7+yKC?jhmO8_4#f%lMOSu*z1~@rFNXK7AMvE3DzFTgQC<56qm#RS z_5hriw_OUFSw}x7O4_@f&v7Arh7$7jf#v4|Y|v&c)GipC@luMgoOS&+2Mux~*Odo<*k*7Awi(9>9{zlkG>fJqB+;I^D_XCuG z<^_AO!!aI1^lJ?R`n&~)wMj}y1x`qyUS3}vp|2#e%c5@-K7}z@h7-vVp3@| z&5ZSx?!Pkn;~m0U%VLEl9?lGT-52|4NA~cRW&RX?M!E!tY$5%@(`8qu;vq~i3p@|t zPi<+!VtZ$%5)!-N4_#gP>CNAj&Fd;Gh{ffB4x`hlBUEPcf0$cjPinpsG)osXdL~n@ zWUsXqOx2MZX?>#XUqk#TIOK~efWBr)SHINfnV>!}a_VmxD>@tXd_PyT5({1!x+TQL zIm_}OVgRDw0NR}LpRjHG>(V1$Tx`>}YSwI;^nwvR@Oy4spCpFiSP3$pdwQ$vTgQFq ziFh!_EUpo4ps&LSMaoW~!sm8Iv1#n~wJI2R`RIpAve#j}Zi&UapjZNHBEI;6(I}zJ zG*9AOkSP0epON9iya8OujJ&LCcSSVa%m@>~p-f1wYaf7LY-z~}`vQ50{w{zz9=HTs zrk#+JXwvTWw9e>os(X!J>L$>ZXVl|uj0+YT3~TChfU_voU)pCn}3?iDH$(h)rZ{P%FM^4}@i!Ke-pQIj-BFB@Z#%fp}C zO;>MH&doS=-D37%DuRXRJmvVtDeAtbi7_w0wlUik0qsYszz#9Tf5;e&OZ|wekoAJ< zcxXjcW+;RG{uZY7i)wD~C5=4(^<8fBH05Z%ZhABF%5dsa28WsWiPV@c_37p4vVkm zS34TfL-yzCmbLI7=HQxy=8z~j`Sy%W6rm$39IvRnhu+ApO|ZnX?jF>LoXbf?5Z-@r zEn1|(58OBC?=6-3@u7aOPM?<-GMdp^5W3;O_Ug1>Qs9ovZfy+s(u}f9U8=p?Mkpn3 zdWD|T;5W?Q!&Xf4fQ8g?H^jxfcmRMe>$$<|p@1>oKRd|O8ms{E&gX-c#2BjSwDHT=F0w{*3K1|sisX7y^Ya+hG7=pD6*}%p zUW-7IxT9v|H4=V-s`Zj{j>rUbYU)(_y#X1qtDJW2>UbpB^BME<&ahw`Beh?tz_+Y z_Ghpe3%ZP?QQJ`JRw`d*fidZgbo&Z&xI(pY>A@b{@n*^nE9EXxyPlD@jJ8~kG#3bJ z?MPsck8d`aHz!09TtJGd-gk$$#DJv!j!8`mp-jQr3l|Izo}&h}L^acsOFVeeVP{m)BeJR0`c4bSOG zOOZ8toVpq@%bXD%_4ewLJxH=kVZuZd)jys2ksX+twjasC-~!e z>)806gV>qJ|BJg6Bg%{4?V&;(^mguoPXcy66PG3*gdG;^95s@`731XTbUvAY(#B!L)z2r-&`JI*cr$n{dgf1B*ZLfE6|oQI8>ifxJM3Mq zNM6mh-6PklMW}}l%l%NHH6FcwHk0z0G%mL9tGC+T>?4It=Zs0}*%1KcZuBp#y9q_v za#z&ThFCbP0L4mdA*x%SP4_NCd9a&TA(5=ATYR-Wex<&zLes@Y1RvONKjNZjeZg)R zaGx{T!?j1w-i(Su$Krfnrw0snbkY|%>)lS8NDIUZRAEsLt%=Bjsc%8o6g-2=(TptS zsEjZ5@rb`fVMsh`A02V)|r-TI?P9k(?F<{a{cWy|KQ>tfJzhsfTV zTj`*X5hEmV1T3l=PO1!HA=%Dml148Twi@|z`2j=N^0RF(*NfnTUSw|!Y1)V?<#k&1 z2RKz{!YhsmL1(*$-jg-DiJ*h2d9eTlseCXlC!UT~1}m;s zDKsjS9(}cS%!h$v_!AlP%jqZ5VQwW62Eln_K^e4c^j^b6+uT? z+Km6f?BMeZ4<$8uBx~C`wj+_0(686N1zD*45pw=3N)Ir=QhJB4Z6^*b1CdtS>2d2IF&$CfBJFG?n=tUx;x+Haa7p_?i*%$d^{<#>u+5} zV8w;tA*pQr@gA{K}P{mtECM?6Em zeu*q}(rC4u+;3w>GXlqQ&>3(?1yJO=w*}>8FIjn}i4WeZ1!rgctUnvorM>NBijxE2 z)Irb-b5V~%th>TJ+N&N1#ul^rBayTzxJ7b7|LZOz7sa4jCe?Z}i_k2@Uh$FDG?bGa z>s=>pU$L1JOmw5tv&7RV?AN*odDu3Z7v(;oxGmG|=#F@JUs3dy#y5k9I2PaApOD)N z@ptT{UqmggBvDhfkWGiD42?!~x_AXRO=lozd4=~p?;T|IogX|t{r7cy8`4P8ga{Z3 zp>5`KjEiE;nY*W)Gx|Bh$mK*k|DK~vvT=wiytFNIL{@=!#qJzs|G5MBbr@fft1;VQ zj(zh^_L1G3@@SiAG_Gi;c7-Pphm~`^mO=XvMqKI!rMaGm+TCg?T^o2nN<#qqpQSAmSoAG;IJ8vUK!YkAPD?HX^Uemq^-$i@^P%+OB+ zF8g24{?5gv5x9cSqjl~go@cqV;2jA4uh7%4pxSv%xLodJs1G5-}>sE_0c7?6c|5QF?6>Et$u5Q(jCoLdWL>+wDi3f_Oiv_KRWHny=jGh zC3Gzio$4Mq9drM&690Q57|qtLEN9+}RDk3=bZ<+vE3Nh-MF*o@vb~A(O`b1=9S-3u ztjROj_r!aFZ<@p1xzaKaCC(gXTxKgn?5RxW$7S-+tx~h0Uq%)~g&u!W;z1B%sL4sb zs;jpRJl8M+xH{JG7sqKOeyLsZ6;7@#%!ysKmTIysyi%f)Z-rvw;>brfWb#z4O3n9I zwkr8k&;m`71Bpu{&0;|b5v46;o5vbEZ#nNh;SA+|Nc_HTY(E z-f6{U{nu;5-8l@hid*JjG&8G~d64pjW@GbH?|X)V4{R}f)OX7G(CU$SdK>Lj_PqF+LIr+!>G_+#~ZBr{->>LKu2bh@HMvBhIP>WX_U!=mb9<(nv`-VfS35@ zN@=+lYg&-T9w594U}X|<3HaAWAFi7(CU9TV`n{X+WTPfm$#*$ zJ)eF(&ow~B|F94o8Dh->tf^!P@Z63zReW7@kkmLXjoS!CJcsf*Y&pLDY1}pfM+9z@ z6Hx9Lv^jTXXmU3uO(6T`gEov!|Em-jWGY-uA03TS7}i=HdXm^iyBwx}%j*$iZPi%{ zpRwrsnTQ28B!+J)(KQLeOF_5r&@n#%EkeL%i?U{1p)%I1vWB+pb9XQ%+z1Wf>M}JP<^cy`_it z5GPP4MZPXq@ecof5WOpY76MFCxx3rLiH{>N<(+JIvMI!VExAqH zZpy5$hTkQwAUWzve(;qVYq>XoyrWIz$Z)9MyOg)h1>=?kYT~a^p^m^F zagY1qdrq?;uKY(d_~M#Xda^kz2X4+I_rzS!s(lPPEihnvgS}LjG`|AUDF#WCgwRz} z#$@)d4?o@SS?I1Ier~2HVVuo=UGW?_rCCh|SPwCb>Fj-wt6;gT#fs;vf3^bJxfeHL z#mAGYQlQo58b0b6ac|lleX7CF2?aN;x2_EZq|?iDb8UK2FC0n8d+vRBIyW>aOiZ6F zknin9d+Fd&WVW4#9J$9eX%$ixGpkHl@euKSw_0*gBFNwxp&CiE$So3i|efc0N9q0^4}qhxP#^72v> zfl@#k@_JgzZ(DqA#mBLP_BMQr!8jz(O-02f&rS*K!YD{aFEH-Iph)(o@W=xmq*V2c zcqhBV{9*Fq@{pPzd(_9S_cSzU8t+iMke=O+ZchfOgkNNzum#9 zdfeg^bOvB(3k#8b3RJ=t`mex!pR_leiaj_*TLd|TZqh%E;Qfqj zGktQpxxs(ZoWqAcFgClr5*7jC@_c(T3U@yi-UO8KF01T)SXbs;fdg0WvGNBL2|DOB zcmFY=UTDCK%E?_}-qw|pEKpd-B`G5-?>TlGX_`(-cQexnANyxm=E3L!aIbh^{M1rgn_Z)S~`V6 z#wAWvtTnaoltjlb{(I5bh->^~mL(M%1JvsL^5&JB86n_sa8l&dftNqT|M^;WCp$(>%%F4aHiPv4t|$Z_ty1otaz(0&22>^ z-PKyDea3{hRCej-z}w(NNB{kQ#^w}bh?m!6T*!ZD{Zx`+A6PXJvDP$~*)x42K5vPg zHQAba7)kIhR&t&}ltKrI5y#fXdK8F1W-(wq8M{tkH78>-P|#Qex-W%1Wf!wtV>Qm+ zuvnwsjiYa+3(dd=Kt6Y;eY-DDi;ldR>)C9Jm;3gtuV$H*G=q}6KK{$`P36h#%;y7v zZZuABG(cM>L*fgHa-V&WmAm!uirl5*s|^j2A6FLSgZD-4yNBi)dpL2=4eYD*eOjB> zcfzV9yO~D+r<5>F`naL%DDEqd^{;dP^=^G#2{MIFSAp3QU;g*rnUfPeO%^J>rqXkA zoTw0(#$>UL0M~}UMp90ZghG%HbfLW5nO30cP$6YZIbGukpD}yh^MiFWc z@QygU!!xFkXKR{2p{Sxzk+YXOR=c3N2~5yO4pJ-@97=52K}P&f^?tX?vDXgQcu*UN zi6t{RTmF$R*dju5xknBNnBuETaK@i9|UZLIYJ3FB~0ww|Qog zy^NTbyixZsM;HxR8VGrL$tj`CvF%TwB%gi6IjcR%&unN9vHSgN!Q`q$o$u5Wd&o~s zV-(keuWKUw(7Wu*2$V(JR;CM6Yh+wHiz@x;H1YEKGFwzcs~Y zCP5>dwRlpIMOZ^x|6bD?4@erd2xKnw)8;q$Ew;XbjA;V)Cqs{=7A+9~rsD02E#9P5Zjnf@(gu=tZgrS9W}iA03mV zC=@IGWfL510YhX{pPp4KxSvaEZo!qFt|W)|zju<~eSG+UJQD z+6iq4wUB~v>6t~{Icg=lLypGewECy;+Gjb8f4g@>-&?+?kOBN}7n49~TD)zGmtIWh z2TJYJrP}2DrxHR%k{;(3X~8RO78VDTPC@K5PKIwF7nf^dfai~3r&?g5L8!q=oFx}9 zYBm8_Voyio1Jd)ecP6 zDLOG>^EmRSF9bL@IngGKYyOJ|YxWzf8^G1?KpWuto7HDv{wz#dh!~nayeUVFY`??D}h*}zlRuR@8Zh;9`C^@2K9#MB=MD81% zr^1<%Uzd4*0E1w{4b`sY*53nbF9z|U+wy1l-=jD9WT+WQ+i0a;lBr0jEf+M0r;-@6 z>K_^&4XA>ejR4ysZs}UBMd#IHio(t21Xvp55m}gMNJe!K7gRSY^TjmU^z82)#X?ZY zX7bk18Kacq;aYsd*RYFgQMD;t%MjV3Aett8C6m{TD6679Y&J(%_t9%jAc0+ zeNqZ>9seO0d2}iMDlG>&pG9Z;5_UMp29Fohp-)_=@j z4!BpYXud4F@{ly}7blfQxlkZ<>T0`ur9Ss+%XzyRhu%h(-VGPE8D08%xF(LK4%USy zza&v5%4+=5xF%#(+fmftpHKY3LXspi-OWgKo#b-p=2xYiaKSH(vb(o9vT1If=&_=I zGfU03OskY(_}rP_&Ts!BZB%nmw@(&?)yE`|I^@8qomIs8jzVaK;47d{@=yzw>DoRU zN#~NB?0g9vq@U1tE5D4U+R4+t^$Rotdv+_9)#iMtWCSK% z+>;9YR6_ZaDO(8iZBq4Im)1Jjf#~OuW-DMBU$0V`J7yx8UrZk?m`bskc7XI_L{5g62Xeo-ry8OZ1TXxI&eidBMEQO7 zhC%0yLiq$wEY*T^&%5T8y4;4Nm86;#ca>D>Qk^1!+txhUW!~WPT ztiaOZ*9_5b>i+eXHeOWW`!jNDKQoJ;6u@@5Wy1+s242sg?kj)jX-f|Okb&6k|0J^7 zup2w%b4$qh+V@{mmBPiNyLj{KG~$QeCp?$h&i%(oTevDI50rV2S@%=9L-4c7ikg>} zW&fWAurGB|#-T%+An4vHnkSijp3_Db%1!C?dPkBR z!C2SApnKY7fp|9T!ke_q*H{5xM_>?&GV?r27}s(Mdfr46qTeYF5ZI*{`Exwb^=qST z)jdvfp(|<}Nb9=)=H+E`7i&U*0^Q)^r1h6yCwVE^sG$V1k@bG60rO`nkS^ai}b+uBj^1>1lB!&K=gqDeoc! zf(3D~LbG6r_Vkcbale{_dLCGIERXPS+V;rQ&5!s=Ro;gaTO^0N(Khsrdf7V^pvzvg zmcbm!l1PK&L3cY`fU!x8fafJ}o-v}-pAq7-SoD6@GGng_GCW5XurUpu?i-`6-vYo3(k>TJPyPGsa^7OPy>u20xGVqj>RBj@uiMypUHz(Oi(HK!Utk7 zhA$gkdErs%Di(0=oe0>Qr>pr@X_#u9#BV#A+`&1fIcZDYS(a+4&j|+KZb+@wX2(=3 zTZ|N*bl57&>?KFQ#IX$$%dw&~mT=X68l||b{8-7wpDFuWbSKF&8HsfkMTf~x9c8k* z1r+03=SRk!%EOOvGGmcY*Q7h}o`0kBLA|2`S=vH{zTPFQ|oTLvv@I;JT#~vk2tR z?5INCzWA^-LEjUx;1KV~QUBi~*@v(GvpE`PMbs-1CG`K7Pm?m`OBSupkrDr} zujN~m26`m2r_`G}VU6gUNv^*7o$y1rlFjv^$uIr4I&O^33k^iyI5op%eqki zD51D9zKgNqC#1JKpi29{EX8(Z&?o?>a7gCxFL#FzvQhvadj=sp6LQx$qg9V}zxpza z4(i5MIi$CThxhGBe>w^e9`-j4I^YyXd2Qru(_1i-1*ll2X0n??+<(<9)Zro$e8%y| z%2u%m;w=3c2-5J_kOFe=pM!}hCzMMsy*dEr>2}TPQ;X?kiBAt8DJuKu`Bt;3c7WNY z*{neG@VVCZXvD#_thW&ZcqcAs6tOQ}s9v5q;KndKW66s(vEnbpB3{fooBST$mk{@v zHXf8^djA!!#Ed4@UWp+xMYjXMe#cL;U%u4t7hw^6l6*}!>WxOURshp$KZ?**ttf8l z*GygiB7P{}p$#AJi9fps%8YN26ceyBF5CKg6qb@g~Dv z@y!dz%|sWB70>ue716jb(BusHvhA}Mro+v{(0pmOAoA%)0~5(mRegIkP6ZqKsgR}r zf>jH%0ZJimZn%(S_MmCqf$~d0a81rytvy4(!e8gl&CdUHrH_y&0}}acm75zCHpYAO zje)>2@h1hS$`3!E83?L&r4GB8V_1RPr@W)h_&nD--5ObR3Y^_OXxKRQ)Mo&uFp5^_ za7R_(Wq%bK=xy@2zGFr}NIE8a8l@3c=8v z6jY4#p^pMRx7G+99DE#=;~jiIg&YIngYNI#D+%$1lB z!rSy8lk6iWhOU*`Wd`MT=)M9))#h1{Xg9UEczcgT(gma3Vn9TQhJFDjlp`}QA{B7< zQv@PB|0G;yMnvwE8tiE{VtcOg%eFysPi1AS%k~Stj@6-h_(k3T&`fc~Pm9S?Q83^2 zsXRRQWY*m@f`{%wcJXhrhC`Ny^-tDgh9Rk4DH`uSXWf^=xdK@u`!7!{wyymOTXs81 zaucLhm>2Wey73F&SY_z_Ya(Lf9r_S_p9R7fcXR!3wdsr09|tV~S_Y9!|9+PpIG|Gy zSVIge>AjibwYj^xUzhd_l8Tm^yMRiyuBd`A{JcE9^xOE+gQrQFZyqIM`x8{IK5igS zk4s#xlF?)six1KO3{a_ z0HP(2-4HE=+yxV6+eYM-H$J5NGRA^Vd^vo$Km=7ze3=2N^(eTMawlxu!RJhjrV;6M zm8f~|Qy>a>^nVhI+HVa`K+uIpYXA(pJQRk}n z2}DC3b33nV0N?bzbl;p_3aE1C5fQNU)#tALjpAo=K?=ZFl1tOVQ*?nfLOcx-`}&o65Zq#PNC+pkb;`d>fE+ zP@WeJiF?QRO*Ojj+0P`l^342lTdl=cB?6Gl1Ow~V8_<_v7%E3ylM;k-yuO>e7B3>5 zGLPBV`cJ4+wj#EEU|H_V13pFN6z6Fm-r%ue@v#E1CQdj%t}UUO^A)v<1;;(BvOX(VpQ2nsw3p}z~eD>RgV*99tg zRzO9)3!&OoC%p9F8I*zvB-8n#%C2%H%iC?sbK%fr!e5ji9x;PuwceDa^2;Y>g;h<% zZMlgKjmZDFz^~8ye|U#n$q|zZ%GTXK61hnykLKNtCz@wFg0zjeBc+kuSEnNn0k0Ly zI0M~43Bz(1GS9Ve0)nZca*%0YaXb!fZY+xr9y*kJWUvmRFsP?=_)7^`2UR9{576vJ z1LL3#*nlHs1M!R-XCMc=IODZJIHpS1g$8=CxF z*l5}O^q(5Q{;9GA`JUECFX;RC8R&^p8Pxq(JKNJF!?Ppv(1yQz`zDo9DbJPGyUbe~w`Sgux#DR)5`+HH{m1$A?|27SDI*O;1E5hd;er z5ul?KOi42HNJn)^wEIvZiWuD)d;;86LcbQTzE@FdOvF*0sXTqmppnMPB=BW-U5IIY z*J9(OH?^s1WjD?*;8pK;srQzQEZ{MYRXd0HSE|HdRNmX=AJ<*&zeTzv>o$1t0e^@< zU|MBYAZbk zoX&Vbjz>2fJo#(h6U|h<{9uL}CFm$QUOJ5r%_yyS^GBiIpYotsNZm^-FdHSIr+aCvO|DaQDz4F+>^;MdQb8=1X{ zh%1E_Yy5@!zgyDYEuGyYijw-18G=v0IiclGz^2i6fT}`>fiXA4GfwrdTfbTs$e{|Q zq$25;;kDZY9CK=arssQpH(y(65IV>@~*wbPqwB%`fpy?zsm>jwOHE4iK9}p z=X$}I&cnqf7d7|a1lse@cB^!c zojjc3mAVQ3Nwv)jb|hQVO`57HeN5w%c5PUY+m4&}Z9WziDIGo`fHwXI-!p`0aa=XC)BXc4j*mT-6yvPLFnGmxXt_uW|jZ z#W0hM31-BkvtbFJmw&qn@uy6gY;P2Z7-q_NudU+woU`2NN#;-Pr?xQk4GWjBi!e6| z$ldfv*Uoq6{TmoC8!KY! zwS0|vPVN-GP9vaL^$5t4rD8|zN%<7m9mQk5YQN;t4UXOv&<}T!+TL)8fhIuF)koP$ z$ z3MEw#KfxYs6Lbq%HybN2*=p#mDf`;-PY*SKnwDjA-%=+dc*N8>6lBr#s{-9f9z6Xr zWMLYIfr{fT{!)9R;O&Yhh;!iX7;K%mbl4ah@pt~jh%m$b;{-rZS|!yV=&me5iXcaq zW%~Aby=O1Jgo}@KYeROEcFajGrt5JrmNZr=e)8|&esP(n3NrsbXuxy%>OLO0_FOTN zEQnVqKM7g3wo%|W41|QZpg(f&=`~q{wO(Q2{D-e1?h>_0`81_n_Q9;P!;?eInD?m1 zHZ&8!;_DHq1fLUFU$Kpm`F*$EEXN|uL2#qY)OF=_@NwydUCzeP6qmD7`mA4XY|z0B=WZ$YPhiCEeBh6R?@$~ip95D9(hV~TxPI!9t;(5 zqmT*cW1U}bnmPJ#4%<+T=A;}^a@p{cp~7!CFRytU+^gj|0ijttHTU19G1FrXcUgXT zy%ru}A?4#&==)t_Z+(4047ps8HnfvG`VSLS^nzXQFc^NKy3}gT&$=uVXKR|0%tOCf zlw$BcS1yj-qw5r2I(qWABNQ5H96ndl#7N2|@*k6Z5Fn315$lEl#W6olM#ZeXDPj&m z6hpP>{}hF2kOg_vg#UnPA!FypkL1lyFx+_a-`j*OTd&zi+3BZgeW9QReo{2?*2f`w zO)0B4m6Hu0Vn||#CXjKV7S>kC8?qba>W{UIixr;nWnRc*6I|H2H~yq{;dmT=iuhmP zN;{xAYjwc4*@Dv@d#dQCAgJV9tKHhYG^wO!Y$#HayD6F)HU4}5;fNkefNatp7wWxY zI@$n3#DhQDM@l^Zo#lu_x5PnLBCPsjqSP|voZfZnYjF1swsDV%)6e+}mXx{siDOo2 zWWMchyvj7~n;>Vk{g0E&>kgAvJ?=dw!T?R}AG%z`C3fwOug9%tME?nO`zMJS55m3;I+qz7*3>#V-G`E{+yqa z^0Lz2F)%2Ixvrud=eh+Emd2^EHZuRNuASkcr2jnII0zWhQ9G{=y8HlU!d8EL}e-xTSL zsOR9ZB}96NsDWNLL))b#SKxI?;-{d5pbe8w-OG!&cyL#vrpkpnKISy3Z9MX&?q8BH zCg2v}zRq8M{15X-(BahirFUdkIbn{g)4yh-e-0-}i#A~X;6`K+-a*uQkfcr`S)z+r zM^He6%fa{V9GI0%3OUbb$}fcBQeMy0*r*m!Pp?0^=I|(7q4W_YNh|m}={g~kh+HQr zw(kU5ZbJ33SNWvY9TI4x=+M|+3Wi^}O?1YPmLQ{tGu^TOVrQiL&MB!qM#M-Q(EN86 zKmlxU2fFj-_^b2Vup~qS3BR*&%ngKY_ky=*r$*6Hr3IP0h9OGE%p~pN2kIe~B<)xA zc!#xO8q@+|J*}|Z2S(Or8uebmCegUQgEX*!r@!V!PwabtC(X8=e1$4TBY zIgKY-Gjm;J>L)XHA+XNk7<%cBQ*@gcaVO z+BA6fc4~mZD1FEVkF6tLdui<~~B4;J= zdMc{8XO7h&AsgQ3N0AvXGMOk^)7Ew^w|9;)0>i6g5b!lNOYV zr`zS-m*0WiaNhuiId|xX-DV`gR$cMi1vOyoNQpgf zSS;o8S3Qdt`GF&CIH-9@@vz;YQy+9zdm2SWbg@p!<+3RNDI!7QN=- zEcq|{>~qaKptSaxYOFZ~?rgdm^PM^)S z@1Z@FdW-AZ9rQoqp0Yw_NAYE?k>X*N`8v>+#O#pnN6wsM{?8eTI zq>v=S8QA1Swi8u+;tZ-@S+3g@Z4B0Z-edBV5(VRY@K~4?a;m69Cz^e9Sdx9|MmIE` zG(GA_{Q&OmS^sw^@&hCIk07ZNe3}%Kw2gmtE8FG9=bU>Io7*h`VyEk;-!_wvA4Bd( zRB3e=4XqWSyZu>Dg^kqj7Mc^g>j;%r4?ihbeTuPJ1t(~2j0UShBD7U3iXAFfC+_L$ z1z*i+ao!!Cqh2x8@?E@HjE{l4l8lKTie$`AOo4iN)z?o|;Q39=ZRiLx1x`gU+*bXCsw_jl#l1@Z@S}J*MEVJ>KW)DN*_fzojiD zAF3#`F&fAwR|*g2?luuvAJh)^@eHnl^i;H4;#*+|=V0)<38t!==i)d8P0EDo;~%~3 z?PUfkm7?cbmfF~7F$_S?A3;O-K6xc#Qc0rtG(Rj4e-6K?O1Bt6o7JSEzzXUSANWn_3_4a-moOt zd!F+5^dE+mtJ~lG98?p0Im}@i?bVc2ANIT?&edDLS<3mf3A26W`bOye`-Uxz)gHe6 zh|QwzrwX~N-xWej@!fK;yIkntl=~|QqHF0ktBuCBV|^@zYz)!7C|(583Z|3-el}tg zWud2Kfbw2msUpw-qyzttrtk2l^8NoWDxydwnWwBn)Da@{uI$9g&Zt9HWN*beO7b?3 zJu*&qNV18X>=h?uWgL62bF6bX=iJ}(`8^)LKjD6y*SW6O^_;g}pP8h^Tn{M>*lj~c zd-if7{JvuHzQp}xbP~7*d>?v7NeKD|_|jKyu!D=GSX4EQ@1x2_Z*zi?Hir)C7;ee@X4w};si{1<%@bpJptjHr#z=*ox8{Op z8Knvlz@#Y<8tfaNx4MHC9m|_Tp!m2!(7n0nNB19^M^!!NLC~K?6zxF~xq)P4@%EjFB5;2YjMw+2HTRSJ~!jUzBWx(96mC&PCeOF3`7+=cycv zm%npSdj7O+GIJ`BgqAf-P4Uu`M|qPoPF^X>W{-recx^2yXebP;`pr!f#J3`q^~|19)TIb#Cq8Let?f>B{tg}xT?@gqwZMrJnwPnN z7nw)bGFq@jtd&y5H|6uAeZnw}M##x)1k!fH4|(=2{tC^lxUqg|w_54C@On&1Pfy*+ z6{Dm~k&64e$nSHRIER(&y3Y)6YEXMkvg8OpL&??9dhxdN$OGuS;3J|NsUn!nTXaU} zh3b^jzWZh6y4^lBWZO4hu?r?DG zB6OOqPJnlGh0W>lD3!>~a=@OmA%?sWmdQn~+y1^ja6sx+Ui%glpnkW}##H&ikbThR zLV0zo)7nLUA~5du`5wDG;da>gjl=jZ+!OErN~kaU^99><}>wqpD*0ClP^Dd*|2;!{7SPB(32fO89As{v+2}ywsh}z zI*EJwMRhLsK|CU2V3DbblxYXGsy$$Apg5k&-F@(ijhKq@ypJRPoc^6+36;2c>9O?5 z*zel@_*7brd|9}dd#+U~wkER=#&NrOF8oaXcas*T>Gv+Im^K;Y)!uyyYnvMF#^8lN zl+0tH`Om-Ko7V1EQ9{nOn}fY=r-2hjGBfsaLc1x|4C&2qi5vxg>nxGRWtr`CSYL()f zWk`y`uk3XtYKF>zHE2XjMKL}`w|8FL=f(=$r>R<;^xwM(5&3LW-7O_x{v5Yz|M=xO zi;WICuZv0u)3P8k00**7iDW?_6bv-QS5M@7LQo^U-@_yJf|oJ6pGx@cjd;7d#Tx00 z86|O!fX1$}6YS^VNGcQmOO=_iq|2eXR#x91;b-Sl3hx&g_kFMY8vNyf)UP5pxu(S+ zvO(K9KnB@u?qy*owy4#I45$53wp5sx=r*T(Pt^WghM^=~kkxj+v$yA2g$GDfynXB$ z^YxKM9s9sHdrT0I``oyIWS^Qx)!A--UY z(8M+q`>c}3xrf`|aD%`Edp~b3e{aT!6(~Mz?~XXVR&(V|oQ&L6qw25lWvxFcFU^=% z(g({9*OJVi4xHQ&mZ#hum-;4n@$#!Tgt1>*v&;L zE&*b3I;M^>^h>x@**KYjqnl$IisCQrW7iNQx8JjDIyC-s8_0`sa#}iN@!&DnjqmeX z*OiSd2I7l`5h+)o3M}KQ`R#Vi()Dikgb2u>MOlG3>*`dTg39Hy-Nj;m3Q=4A+HU_T z%d5j^m%fyB8;YApVhrcqm(0>!YWq#u)rTA2<*(*8b(BJO8#R9(P_VtRAGUICj>7hM-`Z56!uckaRM*N5fq#dl*$l1@{~ zg2m_BS2C=YnZe9YD<$k&B=j&+2rXQ# z`n@9ZFF*zHTmE&3s|6O-1oy&@i(7}5N<){{dz-Wb$0htV zf10TMFw0kP=o`*AeEwZGhq-o_Vw1zrURy>!8{G2R8ES-O#d7L$f~+$))be|=cuSzh z%!dP5MzF;;jphr9O?5)VCyPF<0;lA3)aZ1XD!h)+|NMvnC zoq)Z7mm^9va7{uWw@kZW?&H)VWO5$b75luzSNZ~*5Z0tB=1u2V3Pd{N2Z?)k4zqaA zJBnf%#G6%MKG(>S==_TjB4?4l`&}XbhO3J%Z4~g5&$!G_%JDdGyqbP2W5Gc={m=9_ z&p)gJ3po2M=2lk`5l)@x#T9qJ3Ejc7(A=|jtI(78>EfXGX1KUsmR0lXny*< zUWgG@xyx+q2|sCxsp$%hi>9uBo@+by_=pcngwdGJC(`>NBmico{&tk`;afUfN)(z* zreDzeIZk@O5}hmj1b-EUsvp9tG5uu+Q)ruqmbSOw-4C6ntCwEsQub;HTyzm*u@~ek zm4-oWl!GX667Ri=z8Xs;G#uRYZMOQc*n}=685)YF+j;sj;^qa;@&akwuqsL(_CTIC zvIDwNRr^~9?Znl;!7O9Z5YFh;hccsRre(S%8M z2?|%m*Sivpnp6Q@_bK9F-~20+{p6iZfqBfoF{!F?E^eEBY6PZuv#km-9i?_jujr?R zXWiZF{QHh*4Ek@VDy!p28fk_5vtJN9Cg9;3Wd?J?;+=z{@ScMQIMIe0E!Lg?>2MS^yP3hv{AmyFy|A}fT%gFHWC zEh?3wg19)f4v3$SPDovd#+{252ri|mkBM7m4m<;krQe=EH(iryEFbl4B{M_w^i;cW_PpwJ)(tkOZ+74rE1w%SDl(gB>w@c^(ocHml@;@KW2IejCNm zhtsUqdFVn)yrbCfYlurLq{yd1v2nm!t+aIJTt*#_$sKyf0OTPQxrF}EYk_RX48^_l zzSOaPyL&XyOW#A5^e{(o?A1U7B~H-c(^aIMcv%`$nUA!m!%j_VlS;Bd<5fO|eP3H=24vDQQm_jyQn- zevJeS1nnK6n4pUfUR$7&4}1Knv-HK1F5ng)cm+Wbnu#@NYz85fWBUA!v7uZ*jJ?l! z72O-ID8g`jrs_WO&Yz%8QwoD1XGNZHx3qEP58Ixm;rqx3@(_gQjm zJB~ij5?>|&Y7AIH>$jZTcNeq1!>Dx9&`)7DvAr03#R%8{FbXQURm>-YdcnE!ul8Ka zRTi|AvQU9fG@i53U+-U^FW66RjY&}5oZ;u9+3822H}_>@wxx8HzZC*CqWV%9(nLI; z_aif?{ihd`FzFg)%=D7?$o#~U-QP-tii9XqtGT=JvZCt!fArEtxwmwa#2yq{l9>;zlZndZJ>M*K-)N?bp zAwHhLWrecv>Bo=(@@s$)5bhQ)cb|#oeScR0+X19ves{3zRA!!ITMPrL2-5=cZ)=P3 zRjBxhR+o%qHieraL|D+}Nc5>BaX|D(RxftHh_rYL!~VEn_B58lr7(Ru-?Gv-6&D%s z4io_;4iI;lczdmaRQ1f@79HY%Fj~fpG|`Kogf%&J#9@x>sFt3Q;A1kq-DlBzq?M*Z zfsvyCjUT_jYbXcHpB}=135<%M;LAaKhaAZuiJ!ut8+Hp%WYpQTQyRftq4gxb52wX2 zT;Ozr#hvc0BOR^c+^_{dH+K$|0-t5iRZ^P|%&>&#!2a2!E=RLzw=6ZH;LlXDD|}S3 z>2!udk&Q~~Fvhc@?I?&WB-wf4-1!$Nd<4euRWXT~2nb)P@owWz;D#2x@f!77>`0Z? zyJruE{6x8Q;q@pHG>@K(F}%l&w~tn#tW|^4>damnIZu|WXjRSo0-(;}`id&;W1R}p zZ}IuUj^hmN-w`ssaVQ04dA!zVrwADC?SA;3LYxBJN0+P@fZziyp9}6fK0VMIE(6#< z2f)End2BMm+E1V?62UU$Z%A+jYbb_OBPc zcu5j8D^ht9zLmmm6!gtv9KSWTw$6!sg4t{IrkYdgP&W74m*@92QQ!JegK*Yy2x;-R zMAQ2$WF_KDW=#lPoc83`wVQ)a{LL8&O~KBcudBrTW;|B4xe%udsh4~TB+e(2(`)Qx=nilL)pmDmU@Hh`!a1c zhnRvhmvB4)S=@F}#@h=UiRmr>FQrnbmt9Fm$He|N5q-SB=IZ( zyMVqPWt167X6pL!?|P%dF1(nCn!y&>$t4cubU6Rho5&y)x{=OPN4rhQ9158eLuHh) zT`oWunG(B|{kM{2P8xxV)c#qSgWAP{gj*S*rnSYmJBlO8Xi0Q|;T%HL3UG!0qj`K=ivv z-u*qnu;{P;63Fib4bG1!pQC&y7J81F4jfgF4%`?2+Jg(?TiQ_#cwlG2{im$@;CT}X zoniP!St_`)8pDf1EJK6&@}F&ZC=>m1J29Kj|HAjrahB4O(!EbCDAxA6s9CMocHc-Z z790&!txkSAA`M;}$`=Ngk)*^cx?(8naSHrvwa2zX_qNbAROIpscAZf)N7S~7)<3T8 zkb-;!8SQ52mO=IqMSzZkd&+zwu+f57*X}p%g!(giup3cVrllaJt_dgHJU!Y8>{E&hZdaf$)8(Tqty%PE@G-1UXj>OjsG8ERa(gEl%E0tm0yv+~g-`tQle>o{gd^r8brE$RD6SeG!Y-`{ zP;LXv1`5d=X{e!zU*bAYQeHth3=nc%=Q+58{QA3qH%SedfGh!|uPO9fy}{85DCQ^G zBG|LBgpMD=AeG`mI6dBrUBc`~kvQ7ud1HuFl%14qPrw%%Vi(B%_;IjGp+_;XUdncj9Tr<#ZQ8ucC}k+cmwX&Z9PU~>I^DA4I&v6ptLd|y!0I^gB$_5-w)gS^Me zI%Bc=mIQK^AF0U-bN7AXk0YDE!+{oNig!*h1;C~q;^eXy1T{Q3r;hu%`8o!6fb%%G zbdoT9H~;70_IAMC z@RoxK%Blt#+O?1Qp8z=(VQdDJ=DOa>M~hlYrKjTJ;;;UIawegl+dYO33sb+`--TM=7Kk@A5a48exy$kBG@nTTYZ6?S z_`+~~#W5jZcQZx4@@?s@cd@4Z_w(~Nn}TT3T-8MfWe>RD<4yOt$AxdIaJ>)eFS<5h ziX6IB1tV7hteqH8k*CU{^9b1g#+Y9vxiG5L3v4mP@hmM-xy@%Jb z!6rXbePu<;;orhnzj?)b%U4(`-Pttf)o$W9g*nH$okLdJOPs;HZwj)XQ1M$_XA=eF z$}aLw-yfMqsxen*Mq~S$ReesVa|mz?T*8v}K;jNKz=MzJdp@u`ZTE4mUPT2D-Q`p6 zorkk74rvd}f(l)52anS<1(lr|aw?p*XbI9*kXyXYZFGg}sT0y$Jn5}>+X-9RVoSo2 zpGV(00`#!;q9)E*{%<$B3AU^d80?WX1qV)7`F7ZmLKarKY_Z}0g?aYQQ7>(GLeZok zG{)u}(zvj{kR}}OE!;NVE$YubP)J1{_n{&d=#L3dM1sE#{$lDknZt+=@~S5qJ1ml_ z&;%JEC;9j0bvR ze&{!=LZ?`^UMtB@a;WOfV}RrjdPLr8I7P>N^U@XjyIvP?IzDuWCb74p=wBFkiCjmy zk^I25n;S?@k(3E(?nVEG)9NT<4!hJJTv&>-Ix@r~?5})#b1Z5Ayhkac$l4v0zhpw3 z>-L$#E-?kZ=s{URwFL7^(Mxqga-_Z0xns4p9Pnt?EO~gXHb4UCKQqSzs3`S`p$-sq zJ^d2a0ila>h!OY82bsvO-Oj!3M<-8EPjU{=a4LZRh~bf;G|av`p*Y}RM{N;Cx01J= z2Q*59Z}u>f>8!0Fs6sm0v&9K$WB*{U0xRHj5BncC?3_$qrw;;38HW6MTY*Y^6|U!M z4D9USv$O2)?3bsFZM~ku@o++*-`gkW-4A5Qg{CjMU~Fd*W1!^eTcB1Ovf(Vf;4AoM z76RP)7k%aN&W4@*2CwGsCP%aPa!U7mtw+9ns@turuP6_!9(aH~kmFYNm9^wOx{mGW z6D=sS3vq`jPxD)jN<$BSdkoMxuzcncHc{%_f?ww7VLCTY;tx@?#G}1P)NXq;p5FJhShlNSf>kG*w}PMM4!T=>Roel;dyYEiPC zi0vKH-$@iX04cjKqYIG=*H37@42!xcopO@=M&Uaxk?;O*%b5vEaUIIntAv&`6GhEv zPamUAbm7$*o-%maCPMCwy4K-`c2&pS%iCmbuH-blxZ0UwbG@kSMD5;BVR|g>(aY#m z=5|!c@UpZ}o%W)uEMZJw(%anpNw}wJ{x8Ck$o{@P!IFc!1>UE<+}Lp+4k+k{eQWG39H<2HgZRA?GkEkC38Ax2%97Z zB;)xUE^a4bDeMaO<;hO2OI@Pz3kj5wv<|ornHlqVN!(*pzHcFVVN@NY>s^z~!*V@sp<6)u{pwozNo06%nD@7< ziSGy^;CyS`w${L_cK`O@*(fj$EYPiXAa_D?ky~V5qHO1dW9> zd)?^FTQtW1M9+A)dsALUwmOqr?Z7z99(_03eMl)x*LkOWINdu>Aadt{mYhs1HYwvo z67~1ylFNn9QcOXyID*dNy}K1=5@NObu1WM{Jv~DGoi?#n*HGou_h;YWN3wvEC+Bk0 zqxtUmTW^_j5(?S9c4zpf^KP>nL(@j@-%{*`YjBYj;_*7o@fb>GHy}msb$Ff#ndAvN zyXIiT8M>7TdE9&lfRdb|Nr}G_8us)&U}S(qeq2;nZ?6^ z^gf&^5C!=cPFbO#DM`>iPMR-CZv6{t>!%F)UE03Daq87cX@cX&7Sw^`9H!g-=F&Xf z-U6i3{u*z$;174`vLL6xG+OsbbveEC%HnHWF+S-HZC~M zL7!C^-B|}nN8LDcmO?FdLQem&R3{!zrI^wFPGaZFJ(ax8kJt40znypAzoMariKl*K zeH+<#NTGve2X9d{iAhpbndO2|y8ga58e{K9N4b|6G2oB+R1td@z0^Jj$5;JrRSxga zSN_4WKKw`4cYq4Sz9}JzIj*_=@VF4_tuxpsX zbo7M1YT);I&WR?w?mMlDDg$fXWgqNhB&P6f*yK^hc-%iSu$$8rjH_ey;kO2&I ziS7GHfwIKd9H3Y9@HD7Si=yKdXwI{c_6{(w8ZW+jW5*cP+HA`Ah>h|fnOZrylTH_q z$d}~4*g7mNvQ>l5gVU*5g}v~ts0CqXgK`BLi+ND+)yWDFe7bq$jR5ts9c{9EVn(HY zsiZd7%hm-Xq=j4ltVu+f(`K6%YPhtrmC4%>%KS<*4Zl7j|Gy8~0DFAVAV9#KkV;EF zOF{+-W2X{F6?FG?oYnMVSTu-*h!ewJ)c0+o9Udv#47GcN zYP<^a+v8d1KiQnv9QD=C9ujg8RF*>0eorE(`-oW@+h~wyh=GOLZJPI$Db~y8P2O+9 z`ea8e7fmc?w2u$Vk@!^XkN-D8AF&NvhrA~DctR4MFwsYUd2ZuBOL%8j_IoW_@yHIn zS494>nQj}XcQoQTllBF3$Uy8ZVDy`b6n6WUMN`Hvw0Wtli1(gr?!VtvB&{D`YUn~Z zds$rG@T~f9?{Y+Dfv_&m1?Ip(MSr)nPa86F>Mwr2Y%`l_3QY{rh?LscdAj(fJD&2( zXyCCVinJ@X7L>hLwP=>q1J})*g$|-)C=>k=9q>EEK2QN5$LENDmg1s6{IO%b&J0n=T*12xgo4{nYI2Ig0Q{cC z;|Q}}@BuBRdUCGoxM~C?Awejvhj0;O)LD*1$9G=uL3oDVrIOsa>QEyw4n8~Yk^Fp2^@q05YtDRqtls}NIYEvgS}5xYZ%^wZfUF^^+k#b(ULN)M^#05I#sYdK?; zKM<#Z{k*A#wU|s*iw9y0_3T$l^o$Ea!`r)YRV%PSco!G`xW;|+uyONs9pbhl87kBsI^-k z>M~W#hKKl*u+@N~8&6$>VvT<94AMlzV%*hnnQgnBZH}~+fxKgQ%F0Jj7>fQCM4teM zu@7LVfx7Ge1U4$jk9Ft~lI~mELUaeR>zR9X!o@YdJ(G$1W{%d!j5W`Ha*w-87g4@rs^Rv0CMI5> zFhmmQM+p3?OL?;;LrGX5yI3MWWY*c%GqIIEvhvSJ*RkodXQW?KC>DphuYw)nP~M$h z&W6g#Ion=1&BU$;-ptZ)_Nqgb=cTh1HJP{W|Nkt2$Gcsh>C0HO5lZag!;)(y%t=og zEe=Y8=gbCr-JL9dLsuk1&KNGuW0`YBLL+PAs~EmojHb^EF{hgh^neLiO5KowujNA{ zN9@&@DkknEp?{pn0kYQpiTYenJm?q9y=uzUdc~VD|c)q{#0rmAdt!FCcy<8K&ZqLM*GRmTI7oI4ye$j|-Slq=gr(!ktKP1X#&ZkNc&{LFcJ-*KH4{K95%|J3Y!maml*B^+9bCn73&Jb%bum&)C zr(vHI#5~?6?73mjPY>pmq28jp5D-g#Oa8eu@py-&`bTeC;3cljcmLdr$K@Zg8bzFb zB0R=fGHe(5FC*~kS{+i}$aj~q<4LlhOR-NV8ipZpN<2-Ob7Q-v*fhbSmPLB@H7|k{ zqU>P3<#KI*TQchSrC;e?bfwF#1ORPhOXW*lGVAU^*A<3vkr{iGO57CqNTU1%+a-bo zD?)b1qeA_A=4B-5zeoqfUJxtd2)ZrRDMVp{Jz5<`#d zhCfm)DWgiowGFN(8~`t^j@r08g7x{G4*kxF0qOA?d|0iM z4;e~LWudQc?D&vF>(cgW*&t3fFWww&HsX&5ai}ar`Uq-Oe^3+PQ{;dK!0w zLo)H7Pp^p{*`4qO?w-$avLWbQ#pgCncAYuGG`W#u#JuY2z6s{J_gUcjPiNh@EM+Ss!4zPzTM=CI{*=svp@ z+XxG$Eu+DyK5)CP_`h5AWua`m$320i$T^{Da1L|gh&ZMICo3qZh*~9th>+xkkmH%2 z;q;LA6HhXp7Kb9e5(>Rp$|_MkGomMf7qO0lU_16NHd&h?)b}AOY3DsfW-wP5*ck@B zDMdI6)yq9*9^%2N^Xw%>Tlz#--20cDD0IIQE^(;P`vZEJYY zcIH#0JK*#4m+=s+2Z3~XKr%ptekOVyb?^PjFg?>&$hWJztZ4aAlW=qB8bb6M`1!x9 zA14%2fYLhy#$B0!@8naW`|d;LX6^gXxCj1 zOTPR#QSki=nshpU>3(0acThq@K1|)BV)wbXeuPhD_nf$rQ>@058qfL1Y9&K`c6!2& zIwh1IyZHZ-?AXUEL_P1zn7%a}GOA63v5LU`I_SBLahdlpkJ5@sxwo!>Ll2j_6u(NW zM6P}lflZ{Q!{+LK=4Q?^y=J~-(|x}4UeM_~YuN6c3FO)Hs&%n_zY9?+9F$dh6|ozd zju4APe?(r`I--yL&ZF#ywZBQN4wx#vr)>w$KR<6awF6!$K2_^WK_B@#hfn_WT`Z;T z7B(;o2DWE@zF6yBAno>c^=EnLHB3NCR-9G zV`Ffrj)ZkdFzy*2BqF96TE5FM*v234t(U#P5+y?BC4&}?9HW(a#{2{m80P!f&pxoe zYEEHh1wQ5>13ZhohP3^B5txI-LxJ9MV6^ER&>P4JW@3nNf-DAEfl8&_k57#H9ox!Y zR&e%zo@^q#fZ32rC=!>Z-JuOnbBx+hZ=epUsEs7LJ`G9hg{e|s^f)jd7*g|&$1&N)Hy+y0Cn2a^domW!|KcP_ zsf+#$6_Yz0pifGbU~p%3xzGvqsCh+q>*kMxPzZ`6tX}jmELYXBi#I#~&eZ!-c-_41 z@l@ft?3h0MOh)usuN1UDH-Or+1k3%t$`5LM&)HgqD;GJRv8VkfR`c{W{j_`n{$7{7 z5+4_W5Pg~8tn}aWtJ*uA=cK~UOBjvbio})6E{NYXPvUfBP!-Y&71=c}xgnmM!Nw*X zg0_ym^qhP5*MTN0`@}hI+!(UJXc8o06cBr9+!WL6 zUdD!ELUxP0``w_;3DT1(F3S@7VvHlN2wwd~NYzZngkecAAl6?yT6?aDn2Hkd6;@de zzgEzuzuN~;-`)bYir$3;twmu12;r!CcaWX=^W@q!s%%@LBmOuY7JSmxysNIKG>W+h zI1$5W!9;x$+HjCw5xQ9ba2-?~SJCyylXjWpON=ibzdtrY#_{u{(9BThz}O~kt&gIH zDVb`hfu&Uc&$*aq{V8A=f>U1gv(`1$V(#Z+LbP-m{B9Cte2=Em0;Y|Z2N$nIcuiLM z_`NEh_ORYLk*@~Qw>;MTqM>s^wAaF1HD7Aa%nUHt>!0fG;HSMT`G-n>Fd z@tJDbgyC?tP@6ci@clf~eQ9Uh+FP^_t7TBHmKE?;uI5?0yvkg`#U7KX)L2;sz>OPo=6&_gV@xAqrf+$t`$@siZ)La%q4*j_Z+@j1DF<XE1miw%+G(JhYmE63vJu0yhB$NuimvSRRud0yrQx*xD1?h;9JH&FF;AM#W$$t~ql z*jtKR3R(2u))M=ss=m5M;JBLw0fxlA!z73X6dtY+-lt2+Lr?}%Dtp;AJVxj6zitSf z0p9*TzjGXl3U#VlKRhO;%@6Z>-x>;gSft2=w7bqs*NiJSyuAr#3)9Ij*Rbc8^><~a z*y)h7@Tq-q8}3UEa}a{}dzkOC>Oy$Xn{#^E%1q=av`ZIdOtFq)45U(sKStn1m0Fai ziGAss9%{56@FCcu_2+OWf@Zai&%wwYYolUG3agIvW6jyGmWUDr(Uc6}e6EmFlE%Fx z9eWs}GWB-0l<>cuV|W6mSg`AIh)1T6wH%GSR)r#zfm)Cv;p$*g?W(3SL(j#ufC1QH z)S1p_=`!CT9VYSZ(DF}DG~WE~zQsW@y@;6G^x}}r37()nE_tHxdZ+0cGE&Jaq&gWx z_X7+z&2HdsgoQD^pN{4nZEqgxW`lOxro(_|g$hI@>{>&%%jR z6sI!T%+j3KyI2%;WZ+jBC8ZmIT?(HZZk%BVTk5JDc%S=|mkzimUpwY>vC_Q11b)C= zp}eMOpS@j^e2xmG>K>m5S14kWn#f7)Af$Zj(Z`L4m8_~GJ#fPkUoIk@!O(Bum;6^OqX zLDk*K%wAsBuR^HvC+*_z@>;xuQqABDP+L|=KkgPh{O)9L^$WsNo3)xa-SnVH$Vu`= z?nqER@wj|i9BUt<(CwaEQ+Rt(ju#QjWsZ)ebfDQ;^mmQe8QMvV!Gb_qNZ5xK_QNR3 z6N&8>|3#&E{9Uy{&d^r5Zwv0ER6F-PamG2vSs0B@Z^$h^EEGqbKhMZT5{D*(*CEZD zh_MZMN(f_e+F8;LqZn>~H|<|JjuPbg-qb z&>GUSDsgRN**X7baO30@OfgBn*XQXn|YsCDm8Zht{t>ToByL% z4=kElUrarn(qqja9;-NHp}Hv6{jZiJ_opPv{ouI3Hb~0n!dI`$tCm+VS!W9UUa2s@ z7U^L3R=9wP6mKcHa$fKl#w~V{m$}J*^`IvyjN*PQdu^6i$zq+ zp#3T@(G1eq{$4KSY4{kg$=@w;yQ}S=lsybRMX|@g!7wuW*hq|5?c*1U2wg@zxoigX z;Q~QZ)WDV*c~9+5 zOfTFRmZK8OdQuSiUrvXRh{-1j&SbalkFr}5(tkQb`LYS<;{)@gtFdz8*GWETi&>7a zm=9&GQJJEaGLYFZVga z8LH<2t7EYkK-zu`xl?w3*|PA5`EJH?>xXr{f#iy66wOs=`nOa)7wx$F&UTD z2u1C4q2>@L8DM`uKmop=m&!t!w@udpCsh1#0m51fUci$0`&=6Hf2nHaVwwBp-+502 zc{LuivrAH;Y8iT-2hl<&6I*9cb%>f%hO2`|FR_;-A8>01URSZ2eqfR}yQK#hvHzT; znU>~x-PU>GXFR?=K0@URdIB>j4tp(sZS2!!w9{t;=-<<4koKdbD;nKTUv)>63u!KO z!G63eGJMu15+vLM3>rSDxtz{;TX*`ndvd36?uQw@e2&u7kWxSC?lbo@7P~`Pg#TLU z$I~zvm8C}sk0ZK{4{@~iD2yn7XKC~5^Xm8)<6Q`q)Y<^gAbb}-7F%wMZNql_T%X@{ zSA52O=Lu!*btmq_Gt!e&k#o*0)^8F-c=S6G18i`@3gIc$^~Kzo%AD7K#;Ki3C!@IW z4$%pUCGqkkMZ#aBpm(tZBe-Gc&P54vZON=+iM9*ChQ^)HE5KD1AQD~)qk>S-5C0^MbdSp-it zolpHd60S8hUpe0N`bi9&4eHX+P??q24$CvP%gO}0>X=D~)Bh0SOMz^d|hUE2Sg!$+%*WMCC{7u<(&4#tHc$U0wWm~Qy%6dH{B z&~0~Xao7EA8+whyihv{IcdHgt%WSC%6^OXi6|q{{%^#RZBao5Gd+!fRltyI`8`!kV z1(u?U>2%4L$$w*RSl*LIXjQ+%lP>H&W(Ij!b=UCusakOU1Ph`AyX}wbuADrl-F67$ z*j{i~wxH-Y4aEoZQ&v1FA){r4=QUBUTO|~$`A=}`^t_cJ66w;(S&VB3hBRIVDx=*I zYnX>dpA=_<6qVx&&lRg{kwgVLFjnG)G3}VAaf00oBMUA8ud-yh%DcvQA5*w7Xv&=U z(0@#HzZUPywx}$>d`u%%^YzH2^Zt3D4+)ZrP?RhT_3pO*JkU$)0`{)Z7ky0h`KO0U zud$>LZ|6&&#Stdyo>G#sfHAw50O%c=J=0m|u1^X2z92SpatTl>vkv==`PJpAW|h)f zbKZ>62e%y~a_PvZJ$Em6V|ONAYE!9GswCatszZ9B@#f1{i~iv}J!!#yl~N9hZ}ZCZ zm87-fK2da_=-g`<%eL#c@Ao}FXVt%ua&en)e^gngGOr=#PwtEE+-h8~M^7hfwl}j; z^!&%uXH5)8(czSkI?fd$3^^YEwYD$vM=52qAX$eOVtPuI*DyYUfGgmVetU6rHA-~i zR>O7mqjgD*G^AqD8QF_69a1rouacTWRPG3d=(%oxQuW7Jq-gIgSGKIPToG9XG{K* z|71JDM2Tw+W3P}vZHpKn-6+-nQYr4{!pRR?}k;QqIBUnFjQ)^*R$NeXV!H1PYvioCC*(9EGDPz{mB z9KsyARP~lk-}kx~73<7n_IKvB8ce~Gnty6bMGw`G&@R)mHFerLI{%-)^Jpt}9Q*i{ zL7IucKE0Ko$+$szzGKVWU0UK1VYoET_YaPk_ec|1DmwG@RWIC}ZVd)l7Bo!6O6b96 z?8l6v+^_sCpa#5@*VNZ-0YfYbk7C2u@N`FRa?B?ofaGwm?njP5cg=ANh9y(ha{Zfl zd~2}Z_>@c_Nj|dje4H`hWXHO4Ds39+f1&~Vqej~~Y&^j5Y<42S)l{&m5l5T6p!)2O z60o~N8H9l^sY4nkW%l!(JganoP<6u-jeWej{Pe9-t1)`bUSFZP3zeXQ`C8Xeg*je{ zUKd}q{-hVZ_Oq;2J@}3C9uX&IvHY$YW*8`Jo{#J-7MK6N+%rv@y*LnUPa z(Y6@@V27|5i8F|f+RIBMjJ26{;8wX_T*O@SeT$!8sYf#I)ekEgQ%h>pOMTl2`J(-H zg12I?7It*!$?ID*du_+ztK)N&Si5_o;?+G(rAeRuw%2`%fS0}IC1>e=3WFCl%m`{< zUg){V{#`2f=~)7#p!?U^7ZVNxZHml!29vS`Rxy1UrbKoe&U@Pjqq1BPUs&Ww6bIza z4gM~_kc_Z-;wTwy-Mtk*F;&62&t{Y1`&~Bi)SZ62WMLV-mg`NJ8Dc%xf+K|^RP54= zDqHqoZjwGDdRL)K;xvxn`tKYfJql)?c?DBO`q|+xfqideK)g*BpI`hvQyutC-2}OJ ztAJP}6X_nB=dWxTa0^R)8%YlVAcaDBsz?*FX2^AxX7>v>mp))_gW7E1qB-5r!cGiF z@yd}gB(7sK!Sekb1<(??`R6t`M$*|qY~qEGoAJBs>(pZzvN^SJaiudpwN4f4Hw!&h zJOAMlL~L694ZM?yjY2}MRXObIDv z#AuLhgd2?Qzwht5{*U%xPxb)L`<(mS_v_eIq|Xi=ZKxkO@lz$8;KmoyVK9CY9rbm2 z=ih&}rhZ0x%GPSqsoQL8zO+Le>wb_EdPSSO{$KH&YOZvQ@XjNu(!q_^<{t$T1*7+? z00Fio-nQGE{9yLwU3W2ZN}yUs^yUXSYgGB!wKBUP1;&#a^S#}o=KPb1) z6X20tc(E<<`CC#c&oSJUGW??HrPYDO?ODp=W@94DPnv6Zw~_?0>U9NltMtvk+YsD$ zK4druHS5IlYYe^V>Tdtr&y~hu%*aAc?Ri7k$G!#&|93}%W#sfdSq)-5c{Zq5cKW|ctKC?{XA z8F!(LCN;UelM%olyo8FI=|*MU!c3R;%GK|StRPBE#3M<3HuOOnf8I;9+0oq}Dr8j+ znw5cGf#;oHo4CXHg!?VEwQ4)A+b4JE=HJe3n$xdAWPJ|Qc;E9*mmahQEo{}M<@<=* z8AZqiD33q=Nyl&#_E7dySkB=|H+LG53u$En@;5r2?YA6C6}>yF=#iGV%U=%~xll*d zKyR$a1Agox@2*!&DxUda3<7>=W1Y#VZ{>J3coIz62vJFn^zdJeJ(g)7Di6E;`HKIN z$`ZmRLqJa(@?|wuP#Ecwq|^M4D`3Dy_AAh)!4Ms6(E0I6r!n zNhWc<<$7iRRZ7}^Q*#bjkKtzJm8{o&_-hMvV6$VNDA~FPdGj1{D^AER=@6{$!t(Gv z4?M6QkT+VZ?WmP97%K0iP?RGP&U)9BM_c>fNCxi+_%RRS8XMG-P42Hq6eW;+Vb#Kn z4ya!p4ghJE{@s-S4j(}rI}-@lR@hc5MhN6v_O?0w!#^BelOHq?dZpCVM&vzC^~y+V zljKu?za&h~`P}D}g$UgH5rWe*Gee{vp5orz<(b=gCI0HSsHw6ZTbuOK<8}(cb;+F@ z5Z}d(;AJV0SaCJWCpvHAQgFvJ4HB#{<;d@U@E^+5;1dmH>(`ZcoL^je&HGm-G;BVouZ-`6q2^R6Cb==2fF!_HCi-gqCBUhu7C zxYJRIWDt++jj0Ge{xnU_=gb?!|5QS5-26e^(>iW?c4z%f_BW zS}HbF;^~9`O6!Z<=6jv)4NC$w@?k7)y7xR2sFC=NOh5O)@m8&!uc*Sr30$%)Xz!I? zGePE|;k~{Z@;KZN%7zn)?%}$GLH7)I_kSkBupqP=LH3tEaR|i75dGs5mfGKhm;jSV z>LcwV#fW?SQ-^KP=CR}P_>OvkG1jHsLfl8F7&LO76RAaarM?B*^t}kq0zHG1GY=pu zq7{MT3WO%BfpccdM&D})c0T{bDy{mq9m-zhvhN||h^QK~F8z+v;9YWorMu)Rt6skz z`ToN@+O$yqFq#z=lD?_*+u9=)d2%Iulg(LByiWOUt9|D4P_*w0n;Y;9L9_YRadcL11c$^=!cq{rIAM~7B*{dpfB*++3#*O!Y) zq{X-J_23Sq_c*^BoXq)HIle}+1?}#9ONZfAZfdtsVA=$J`AU{Wc{!WTXlESX^8J1< zIHiRDW~;gha_}UlF77@{jci5$tCJ(T*R1TeM0Z@g1Eb9?eU`hd1QGfZE0wsALR?UuA&Fjwa4%$zunff!1wWX~8$>F8Ay6QH9l#xi2_`dx(3<{N}c;s$xCv0SX z(9uXJnnp9e|8O*o_K|ThrL;E|R#$nIqL$yGT&)0#(igdBvV-^D8#GgEZKy!~mZvs< zmjkn8bG9(j;}d7@c~@!P0gT?KX3l%?Sqj0@@*ot=OqthD*!yTZ6qzK4)oh#_FJsf% z?sv$pl{vfEtR7|!W6av?xeatt>s}C$YMOrpj%b-3_z%v)&qW*S97YG5O)h8MNEcyf z6_y~&Qw%Is6}q;!+stXC9q|5Igse*P2=KRj@f@fw(W?V_v53{RZ13EdF_Mw(su{cM2;2ZV%k|6c{?hl9N5!FN++ImY-&f7bFx*0yS z&*Dq@i8dY5q&i5XoZgYM1BLfzQVam%VT*IwY^!e-zfNwn*3ZGbAeGs*}x#3KK`hPt!KPQNzWF438ewIj&Kqler zB|jF14oHQ3SOVxKP*zHdS#+$-rr#)N_3p`+mKKwr2J&jNzA#OJ8VY*#CTW8w!cLqJc|M~D60DDZg z;KOFglwel@jM#s4_^FA}l(sE z*(=UeRFoV$*IbaOapsMxdkX%1B8im5m2yuxmL0d24*f0(6jP#^zKPuDS~G-rZI0k> zoPH(vBx217daBry%<}3C5hzMy1d!{nk;M!$bp7VvjU{VBg@6)a(1+CK7i4PB5R8zk zv8Cpdc6|E8xHm@Hzy?u3VkXuR_#xJawYaR}v)&GqJUQI9`T*=obIWd<9K4ilThfmn zbe>!!?W?Gf9?F|1)1AcCTSISw<`L+C$Lo?UeyEz;E5!VDD|`)PbrzmYq9V=!Iwdoi zg>YH@Cn`r|uf{Giq^sshTObt80d8R;Ht(@sQ{j72?Vjv}OM`x!Is%e@0of-?^9-{E z{0P+8;&}Y(CUg$TOd_*iNc+%7E*;K-WbT}bYSHVgtG}DarvVxjZwN|a`xqR~^)3AE z&sXm*tUK0|4?sUjw>0(*=yEKPX&m$TZ}S{Gy;$P2GsKt)Zv|yFQ+3S3wcd9auN#5@ ze(3hIpveakx7YU=wa$N$0=DQ>Y6k68)eIW^k}w`vcqTjv=bs5P+e8d^$e0n?0c7h= z1fhM4?f|*z)xGqKw3#zkdHjd~{t3^1y#ZS>2<#pWLV@Im5snWQj1C@cnrzzaJ?)lx zI~J+fI+#wW5fog-6Zwqg=~LpOj~{+n^wulA=@%ru`z-NRB`ur*Xr@L(8SF^Y04WCXIm3H z#r$vTsMH%Cw1k+ir~BpocT_rYam~4SaEDL+B*L!!yPXIwqh}+l7#41(p9NM;Rk?Zm zMnuzIAgeK6*K^7K=6RhZ`kB@!NIZJyRj=R7g0NnqG#v3W-L%GlZRTKkjUSyx_a~~)vPrGpF%XJDaN&Z|EwWD#|5z8QU^7M#_TjXP(*X zFZUiZ-hN{2cqKclO7<8}5Z_(Q5+y=Eg2Z3q?WUEkv_|D+Lss=cH3Y-kkki(S0Sk^o zq8%uIY~)i^JRh$sA8{UvgAx4j0VJnQx|o@&-5svW|5%U0gIkPV5py>JABj{Yf`b?L zP#DLtPt_Z4hDZByyj1%Jt*s+(6$FGZt?SNF*Fg?MY09#1S2JQ>=U-Num%pqW1BpF> zn*mY&gr**AQVMnBO6X#USYx2yhT7GqRgR>C2o)qWLtcy+Vy+VBK=DqtORy;9hwUbu zrho;Ugj4z#`9y1Xtr^GRVw0|@d}E<^`@oR6M{)oPQ=U|Z@nHlAQk|J_r)siyvKr@} z3F9D5{x7U_X7@HgCEwsC$g5{JeXO|F^mdb#X&^pvGGc3w#^Seh=&Oib}nr5^F*4;J@ zy>`8x!<1(ocF$CHY*Od%vCH3KlS=Pzr8kkSD8)jbf1}8-zMLfl>bRAfoGUl0_4E!} zl&IO{R8z+eX{m{JrlTG``dJpgY+h5ECtT)H&|og^dqD>&EyyjSlU;UYiSpw>7C%E< z@#27XA`YrW#W~;B;;Ia(VWN9__#}uD4QRc;*I<|fPq{K4P$vOJ?+KtJ^XZ?+BztGS z$6*H!`N;ySdI3_4190_eC>Iu!xyt|%hlUKwc;lr>dlcN#GjIp6tPGH~{vk1>chDJY z^56Gxdt#nFfg4G$KJSl75?$2PB_;sgeD7r~JJv`UEZ>kdN7vCo{uUw`Y6#>QtlNgz zj1j4P96k>lWgT<4k|wGU&}U30ZP7I?=W;NqSkbtl;TuH6JC4k706sgb&7inE7Z4mZK^K= zdhY%s#9Q1eUd72K zYe7)M<|p;+pliCN_BW8Lb$#7)<1YLRcSUD~h^}rnKOMKQt_i|Ml_v{&bKELvVuI`5oYq*G&3*4|kd3mV8m1j&%0`C4G!6 zX?62QMl{3M7Ta}?0y!x`oc+~AMq@D{UdV?6rup2lY5o2i?APy-iK3Waw#UXBbMzpD zNZbrRBvt-8DW-(HH{`$nF_x9HCKCsm1>kjd(Q#T^I}ag|Ii7(*A7R4yUuLHEX>+x1 zeK3Lvu7Hd%#aq3or9aoiXA)~)|3uW0E7oTSmnAkfGekfinFSCQAJez-V=ziHMXM(e z1O+fOC&E~N!d(W&4zQyKQV0LXYCu3e^Re7@yCO&(jkYMBcCpNm&xc$0z zWZc6uuMX#3GF1H<`RK?Wh>ClX%+*fZVK@M)Mrk~x9~zBFY5-IO~r&v6mMcA38+NzmaZinMY*~wx38$iPz41u*3yH*A4R^(+;GS|!tfA;mB@KOoL@(6X_9WOQEJ}l;(G%%G=^orrz z8uZc}#N6b=t+{i=Xx>W=IP0rPwin}?;lVi6LEqoiKYQ8BB*P+58YHxQW8_8x*fgY! zRx@qwkTL06Qi&M8F!RM!3hdN8u}iZeFj1d7wk7)Is_K* zi~ns}a0q9h>-i=w1%G2YeB$ML%OX1Zu?DXkl6|;q8a#@MTAuzD+gxhQTFm#Xlfs$22> z5&#%6V2FV}UK(hh2f89(XCkq+T-ZVSNtDomOq-{L5n095=(kPvp!Nl_$x4#!dn+M& za3`aiA*p89n!HN)B_+AZhPf`3GeK%SkrWJBhUddBvZ`FISC~?uhd^R)tpQnqO^JBEWs>1|% zHTLalpHv8~B_9%aGX(+-LPUaJ=Xy9}5RybMGHyAqfE;`oaZdaY@|56Jx|L0Gj~4z> z%dd?yue+vtpf9(!p=CLJ=QH(EE2{^qC&Qf)n@}L~y z8u18D_m2xW8X-buNf=y9Cj1RSfJ9b=o&prli{RemtX$jhh%T56RZC9@CfdNg@|eBK z08wig@&;jq$ZT+ra58q$OTFp!62@<8dz|50v(!YNmFnfG5et|WkX*XgZ1wBxpdcV% zbKuH|V`h?)$~ikFLfGk#2U<>#yTs4j+y2$u@Y;#JV((6i@{awfDo4>pltdi~nV}IA z)MUKWaO3yy9&=%5rM@gGvpZG6jpa#_MxAfQSf7ak{hu!siqI&B}tzv4xP>I`lsrSu`#WJ&d{ zv!PA}eU2;-oEB}bz2+mVI5_={aYRs>j4VG`Vs8IaXfGc=8f`XgdE-&MrHq44$(q`M zT#snNYzTEFwZ-P`x$Kk=7QZHl*Tw9>kss+n_QwZYjuW5rt4HQaUU*7Xym%p+kmStW z6D2kyECgF%KY*&MnMwAHS`bdKZsrzb0WvMD6^GPZsf4N;JA{=(VKG2iD-cD!%$i@J>t5 z&7?*jk<~>?||9lNDK>o_sbrTtKs!@#V4mhy8s?di;Dz z9Fc-DKj=N=t0k)+?lT}Zwsy6j@%qS5wQPyRNkZClanI7LG@4osspkP?xG{}I_M!uk zjFeZ2Y~MG#EUbSy2TlFJG1N@%Tvy-@SIBYb6ogCMsi`b6b9+_P`$VZEeR2IOx00`? z0nznb{GM=jNqf>&h@sSc+TU|VKzR87gw#vO^6@YD9_u+qO>D$X2pX8E5%|4N1oE-9 zj5s?x401Z!ae7Lp^4sr9*3CXq#iRkJDk%uR7m9+ZOSM6qNRw=7CYfe6v`$GRtpJ{H zW8eDkj#B0dr@Ee;p9`k`(eeVDW+@Xu6;(Yi|3|FGdesheSPUA0$r-h|No|O&ZV~Mv zMoGwjcqdZ|G90%LL;Rv!bt5p{f^hv~V>L1S8$YP4t#@@kx*j6`ca?W~dGrNM`E653 z89k?gsnK1V!8kg°e5?DAUNhO%(UHMJVfFE~@Dvabs7(Mn}fiq|^Zd%J~Cp3;2s zcKSE=sBhh{*;~59ZWAdBijx(do^D3#ZY95X?02&AXB5zrz3{m(<<1j@0Y!ID4KMmG z(QKa#y0#3c(a*|ODPe2hQON}D55C`*Wpi)>H$gu#XvC=CcJq?jTd`-`z@2(Vz*IJB2Lvs=7$!^)m)R}Hc?d;AhO5x1?C}$U=RaNOcP7LIijycw$*^&R<$wx-gPhRm2-{*-@p+ihbl$AU z{iOh!nh3pL4=P9N!P<9)p+!H}noppQi5v1)gGBf6YS_OSMaQ+ z)=K+SK?SBl7dh!umG@_~(BYTg>tW;~%lfHWjFh=uJ2wkDJb&{OLPmoS;s2|#03=S+ zQL3qhc=Bb#Oh5mIEDbG<=wZ^O*K>bk_lMm3U1Hw5#V!QaB@7;6WRZ5poXIWxj5c{y#_B1WRE;X z2|m1$Ec~kUlHtR%1#k8pT$0+fEj&P4ekMn@I5j)D)T*hDzfhCal-8guAz-P2k)|Nz zif|JYD`hp2el0Zf&r_}Wi$0O-g946_p0oOP?n7Rka$v+XIPqcF;{RB(H}7|TJp|>@ zpYU*vmKqlcwcmOFXWY3s|2Ef&;<5vNmd%H%8IhLtAR{`O%+1g`n=63*HuKST&$6yL zXgWT<3wfdtQv%?e1QYTZgB;3c36I+R9^u+wS9E)bYTb32<>mAO8Zf(4NKP2GO?EYN za~u1+R|=dJ{~I=Cg8b$F6qyu^iFiX@sLv^BSa)4U=6TciTydgRxiIAo0rR-y7GA*- zC`6U6GD@$u=`GWoPxa!zr~H2_TzM*sPSs$$KWt=hjU}Z zXTp}jtuckIqdJ(?F3U{Lzf#ivjuFYT&*p9ZldO6} zE9fAM5IpXX(+}@{46$y$8KS=R1I}PeRmF{>dxO>WCC7|_7nzb4&*r7?b0skpLTF8| zM2k=Q4Td;>HWcumHbhNt`MFL=;PaGgP{Fpm$uZ;`FI~in<}kAIWPd>-ExlpAui3Q! zo8+`No^!j4d#T{9rr@I_uTr+B(S<*-*IC|E#E56Pu(Lmqk|t%LAMDaxPZW=5i=?B@ z{PCG`-y)CykuIZrJ*R&MH@lA&xg09ZXnReUe}`Wkefj0&dsM6*{OR3K z!k-iT#AL_=R)W{I$bUrz9IHOfn}aDMOZPMlFx<$?)--s77aiU!;_(?5GA(i=@$6^| z^u239PVwM+{m{4N`l*0Mv)x-%?jN`3jOEfLGo|NTsbiyFSxJuF@ZoX|jeRZ3_wN{6 z}&Nf4YuqF~*F z)|YsC)szILExR$LB4*G4UQ>k>za<6jiJL<5UO61R4t;g#`~6K3=$rS9rNKY zVu$z}&af9pOUw)YwlW4UTHhJz#hPh6iDNLBc|}Wjy^Ug++go3W=f}m$2c2ngKv*J+ z_MM2Ks5@4RJFC4~X>)IOL=LnGS`vK{1#jHN;<=xv$05~;Y*0Fn!X6%%bU9|y=$~qv znoYPSc4=6f0JQpHP5)5rQ_Mn91Ltj58m&IjD|Q#@rhvrP*UHxaOPw!i=L{kW^B0ap zEXU3dg|`WLqi=kE8;cNx_(bmxhWn)hae6vzpaM~Tu^MBZnBAz9?=IE?BT4BCTeiuSisu9Ge(FRP(fK8yB+Ha9$HyH;Y{7-#)sgncMogi zNz)Kdr3)Wig(ywd<8HzOW_gM_L~K$a89arFuR~e!#M+NPXe$pBBcTw=Q?QB}wn#%2qt8^HA(E zo-U2gJp3Coq{_Wo7-SAaeg6G${|>3tf4ESt-^b@<|37s26r2EeX?wBB$Xr5r#R*rDn;%{fzzD;Nmu2y>jw#h_C>^cbm zxe^cMa|UA$USRTHPF^Hdtu9zCv)!3y4*nfu{|#8cp!(Pa{iiaHcaxtwrN0%eel~lxj^|w(K6(-C6xI4Dm<)A_?k>pTH z=RnEH$qzyu?4G_jay3qcGi^Ci`Jk+rpU9g;ldPk)9P6G`qq{@~us$Qctb)Kbqt7jz zzbkVS-kT|)<)RNbqfE**`^ULuPCeN?48bT9&A?y#HBxN%$`TUoWyUmbn^$&s7Ds<7 zI$`X}7WVIb5KGa9^Wh`>{&t@+LuT2roB~Gqsz(a<_ihkMTG$J4i3_EnS|z_OmS(D} z2_gksEf}yhZ=&5aWs9J=K<(W)!Nt=)PiKx{u1dKQ5S#z_j`l*8BT%sfK6RkfS zoW_O)%RCxC^ujw()y*mJ!NqNHjF{Eg?5w;(d)0fiYdU1_ z*KuP+6+?+7aK_BaITZ>3hz+7?&j%Dth-Z!fnF@RzTPp#1<%=MS<3hTRC4LCnNc&I_2B+z#jNOSFHctr5p(#7HTO;EYB6nn z<0Ihk!s{Ze1RJ(i?B>#x&_`#MPdEhlfnHz95#J$aZFj9Y%@+b{MU7fsCD5nhC%-@|;(rUV zksmB`p|o+zN@os;hS^Xt$QkBwOPeB$YhV3*tkE%QlemF`Jz-AzfPy(q8Px>t3lvzv z;0m0P6&S*8=*4__Y2P;N=wXf!Ih`v9Uygs z%;`Z;Tjp_$5al!Qgr+YqOWg5(*MqW-0C67r!DYQ+`kv6le^Za2knN6x>DM zh|$S=H;c}qLY!$9MyxRce8L&TB!#zAKM&}>g!AL0QgLVyjaA}AEi@h)zsNJW^ zO79=V)GPU5PZWYJ32jeP2)@7#!Nxxs!;;;E0hinKo-Jd}&OqR{Em4S()L-b2JIUVs zmStvIAPb{~CDt!Z9e~JQ0Ypj`CCL^(x3@{HP<_P-H1PTnkcM0c7(w=zLzEiBWMC}L z30Sy1Y!Sf*O&oK{|8x>mffT_9d+T4ZjMdSG)mzC&N`6s-Zg*H^!n zihr+j-bzYDvq?|JDT`L2cs9T`sm{ z5}lXUXd|wAPP^Vt9qV>V{VC7+{Sw9H`HBHneL6D|KL4V7>{Gv;pTnA=?xWC7rDwoO zllOk_PClb_{VDVs`Z|9xMkTitn#^?vyImhj7c$NkW|oP!b%WN#TRfGdVQK{559wJ- zWWUIf^u61pzco(Pt={1x&wCqJH}Sv}92>-8Qo_Y%Gqh8|Eta#Df-nx0flRz3c2;Ct z)P@%ReI@!s`Mtf$mxfdjqkoT$4x+^$Gani{>;p?I;K_I(Qt<_>*e(f3npEU;i&90o zZKjue*5}J+<=YemmDayI9+U* zHQe8PNb76s2aeY-$WMoHiP4VL7Musxs|AL=8}g*x;>_#U;cA9H8l8>fuPGuvDErs;>80z_T95-kx9PJYzXO`Vn3je7%dcH|~B z1}sqAF{O<2V46K!fx+i#KKcFEWyt04`}Bx%)wiXlS8*oguNm`VLtS0Bz?Wh6T&b_u zl;6R%Nr}<=Cw9KPQ^;TNdH0oSJU?>Ez164SlS+aC#*Pj}nBm?}du4%>c6{Cr^B~MK z78DM?*;lhE?+9pf*a}b>Ze{1uS{4|Wl1uOB;M4j#L1;}xK8U8NG7~yW%@BdH|Gl2T zac$72C)3ZU5W4)Gd$s#6o2+aApawwH#m#t{77oDPfvOXA2*yOERCqO>ynay{hxzu5 zgg;Kn)WGxp;+~Xqcr?*GIjE06F%yGMQz`6q#lrr=0>?Xqq1%O@H>RGkDHk;{8GpOu z|L9w2J?7Jw$OTeP-M75ZYC)0|y|k|0^^2$F?-u0lk>eMh!6`-t#PH&c`nJlocf;05 zil?~Y1_5T^V{E1W{l~{#gW|mUFV}CC&&lY0R_CjpJdBT?doH1p1^`ag&rgS^0;rft z%P<*OMH1-2F1Zc#s#vS;bS0|T4!E+2KJdZ9I+BClxbMFbDEt?Qnv1*f*4s!kK>rnu zajN<_UG+z}E^k+h6qhwPK!t)^?-b3;5}{)oDYJ~sx!=mA*gBIqa;{;)Chy{9wa5Ul2SlB zDL2*sjGuTtk)2E`-fott<%voK!0MP;FV)5r!_ExSF3g+3FR#21e=Mfn&5L=L-a0@u z&&aD}Wl}^D$&Ihe|1}2TzC>Fv?+_AJf=kQwyK)D+6C^7?$qq8}6aQ?~Z!#wmT6L`L z{8C*FQ%5p_Sxga1X)_mI2P>&hbO7-w{@{;a`%SlCrBQ2N^{R|>p;*7;)6K0;clz6i zRGL%%?QglDg-?P0ZRGA^PGKCTSOtN6v?$sCRNUqByP6P@2SVOF@$%=CoiA~%>98ao z=1^My{bjQIkAty%GAYE%8ttpp`DP~t@Cjr>H~7R=pW^j9AwIchkDZj3uZ#SY zuX$ytl$VK+3iH{jy$)xv!USXc*oXQw(q&e{P?g9w9+-P3-Iw9LmFE1J4*Ba;UuNQu z#lL-g$cW@3yl+f~YnhkMmlbC)N{*<-Bfc=yE6G9ffPvlJwVR6RW`7sAgE&&D2_i(7 z3nXW66o;dS81ulVblSwK{r(3xd{}U19VC5X)Fl%X{S`ky>D*>0 zy=qm>a3g;1BI(zSS6Kx&oCtA2k5i`gH!2gKF7h-CdqlN&dSC)<;mYJ1{gXV%J8vuq zO(F*iEeMZsza&G6C#w8VOe)TRa4Tff<*d?g#X_^>&U~$7Np`w<0iobx=@hXQJzIMB zyJCsWCM@o@zRcoJVW_Iwdmi&{RUKd7@;tN38!z~q_&S}4>LtExoEJw0=DtO&1kh^F z3F$PM#LB3dUt{fVLT$eiF&>GLwF%<_#C-iT{J&qbg)q`k#NHwCZXr6(2)zD7@KC%z zfuDtyE_~v|D=HxC+Gpxu;;l9h3o{Fw+VX}b#BCYdF9E9KuD||%vvvdBUOM~U#qoZq z<3o&Vv~!W5V2P$`)Yr@J*K`e^I^ja}aiXz2igNW53hT7C4cF~yFZ@_u$`76?cFXbA zjOL7rkeml*?3YpKc3T54EDsl;bvdw;D1jgVS+&8^nE3Cz?RjB zo;2Q0qihb1QXz))eIT|Y`8c3b9Z5Y8qb7D+6W<5 zf-5ZD9czOs=$t1VD6V3Yg)>W=csgZ>e=RCqgeDLppLgWj`o_N@kJ4%0}SWkB7 zU1(R|F_zfhWfPM7M~Llu4cy?;+RVz(&V9y=ANAC7oqHX8)jde45tpNZ2|Dk}j73?M zS6Hd|x|*naQ;A=4%J_fmj%<7XpI;Xw?d_NJ!V>1YVg9^`{k47~3uH)CR`J*cZSqIJ zGlrG-uV*0VpIbdz{9rZ)_*Ov3A!@TT%gkVrQrR`{#^-s%rK3$XtWkC#95*Y?2uQS) zRx4!9c_3vxjQY;lw|(yoF z4ok=Wvl^BeTcoe4svRIT^wn;6-nx31eaWuF(5_Ed7D*^Ju|iR^KdTSeSCm~J2PNyISpG;GzZsq1$L2{r;NajZJNY1(td4EPcwmV5w;cL2uK=&2Y~FYWOWXFZuNDP!?naOO@U#cjDq~wopx)X zf&FzcXCLf#=Qw1pnZ2U^ypC`!nRhJtvs9lxd>mQ*;`V!Ec*yfF=wo!X^Aps4S%9|> z$T`dj^&#wc>Vr}uJF!~K|J5`(n>I&0dAb*+GpdpFi@(r>la@M_=Si|J+1xe(_& zn-G43Jzm+F-j;W3aMi#N_EU3zm@dtfXf8D*)p~8h-Zj2Re2QYx<7vBI5Kgp`C2OO& zBq<^FeA53_*ej!L`QR{4=q3#MazlPq>F0EBO4^WFi(Q1>R=}1MKgs8lDHq=ty#Eet z)_O$1PO#7t?=P%DcpaQd2on97$ZL1CQ6C{*j8Dch-WikY_x6n67DDMa4PO&Ww}$2r zmPXadF(*=PM-eD}J75b`O(@3eJ^e9({Llm^|CujIJTT^KW+d=#BQ%@@Jf~IjO1|4* zcTf|F)H%APUp1#RBw3zuJ83>Oc5oM{d>KqZ*O>LB)S4a1X&1YyypI;VR;PZD6;HFT zsyU0^6E~b%b}7US(%*C*-1L9iwn&>H==Pg5T`ltrxtRHL{B^J8@F!6XiQhCo(E$e5 zf2Jp47g0nBy2sTcAQ;cK0*M!6e|24oW5LGtX1eieCcW}N30`R?U>j5mg z%7-YikVwWJ82MbrTD36dPvRD}@NE;Is?S+kmgc2(lNFjDZ$xPWwc38V5xlRM8_Br9UDGW0Wv_lW#qfC;|`*|SA&U(90>QKa)= zHt59dM9{abkie3y;0b68Qhn32<_uBxX2I=SMAbu!!*b}Y)8c5h7o7qd`E-Dx@+AWi z);lT?UQV%RAikLIOzs1z_l35iRbB%&jw#*V`xx5w2_}iJJ3;QI!pHU2E8**U#s~qE6jdhN(m_)6W(eI}=&hm##YDTk>HdUq zOzeb|8OYte(RVKf)%Eeu*R2+gN4VOp4gXH@7jM+wu))xqbeO%b8S`V}sR!0HTYfRZ zDEG6+an_-Fs2ts7R?lrWZfF0Bpi-0F&+cn6f0icoA* z{F$HzqYQ8JJaOI|i;j|qXP!86iY#l|QWf!wbstkvutY}a{B?(CP^|V(o%`uO4>}cp zwJs}%5cddlLNNPan8kbJ;U-T9w2t_eq-2SYN0O4Tt(vNutfcgz9XXAAGPO_X^)G%l zkJkeIdB%bq9^SkQelK#~75DDJk1$VSOKxCfA#}ZvtZ6>0^$9H$}FzBWpxn>87n3kD^yhuloPV)t^@BGTm?Dl(_6-@ z+2P)l3}mqNx}wvZ~BM& z5?iVN3*T2B*qq923q~AQplT&vKhghn;q|%ulGY*~{%6VG4Mr9Z_pw|}(lHTLr6wW0LzInQ#^If5&`?i`u4sv_- z>qUJ31gJ*G?*p6q@AX**!dyeL4zaj|I^vY;{JFSt+w02@-VgWZOuV19Q00Jp?0 z_PPe%!&pAW8u<@lE~(94OnjGQLy?Ny-4CaSHr21xNw}%2GSagVC*U~b*7w- zp5$@w4~9BwU1nP``K+$DmjBjo6asQ-5G61h&NzXMPL)B$w8fj?tRb(VkE%Ze#N{onbWsw?bL5&#LvdC(~tGb;}yzH zfy@xxz_lyv@9@H2<{PH}v&S9wI#(|E#wiQ|>zMO3Sgt?Tyrkr%=E*TbLTZ1H1mHAd zT;|YT@@v9Mkf|?+bjDMJcjHJ!%Nj3`s$6p2`O=uzYvRmegvKG@-p2%}tk9p=6Hw#H z4E!c<{Y3}3t-#nSe#$jmc(O>&{MFOC!B45>SEo2x=!;8Fy^_rBfJ1^~3uR78_&>V;I)ilBl_q%n=sY=#-9SFXbP!+&xsKC( z2M2|p6dEa}!o$&orTBqKGHWyZ;J~2(6ud*tt@61Uce3^k_MXQ8F|22HVEu2ayz!u= zPkA&)PJ`gLdi3$Q4_zKZ3)}Q5sD{-3m+Is3V|KXnVS25j6`@xdmoWCxH#*hQXToDc z&OFLEPEa@VKbns&$7{7iTykCPiCze6za4zQBxQ$OqHrS``9-|f*Y4eioOqv@GX7Jf9E35&} zOu`q^AC%eVXbtYfIM5lGV5S`~X&(9Z3I+UNyQYm-5-^xsHz0yvpaA4CUXEZ2kGf2e zOWXoobH3pNt|+)dzt~?Yab|Y$1%VGI;Gdz5wVz=*P|Wd)moqSm!2Wyyx=gTPOE)jU zn#%0x$gz%>LDv)rnxwd>=lm33_a59UV(q@}arvO?=l{y+u{Y5(_R4xYfWv$P8MO0` zC2AD@cHwg@rxP zzgXQe^)hh+c3eeOL!}3;n#NszIlUA~mG^Jvg}OM*u~FtTl7>4~z1 z+?4=|=4*x#jwZaw-HX4~tOZeP9EQ;k)a)oD@FI5x!d}%{T?OZ?I+TNUeC1XL8maGu zIN2BB5!w?iCY@`#>;cCIE+<18{&El%=T8L-9tdPju7GLId>%}x&Pdb7r43r6j3tDj3M&4-F z3iW*d#H;2nNSv>DYCLaafcu(N8UsxWu-l8Ja+uf7v zmkcZsj1J>$9wJLlm{KQA}g-9)p!N1K|FG2)6<=KmN%m zRt2krm4KnR1Y9`7OcKfo1F+u)FIrG$RI3QoH$jjUFh!pCRgT4YV2w}lVReq9*`L8egJ9f%R z&DB^~IcS!pUbS2V@<(x#{4Yby!|t5)fQgxKD1P`0|LFM6bjde#kuHGv-fH`=QnLd* zeWTgx>oezc?2dLS{6qiNtBI3+hWuO2H-FC%mP!s@Wp4RSj8SpJ{jv0XiB2W{*4Dgr62gS`Q-yx87tC z9c4BqgNlL{9{fL=&ibwC|9j&KC@Kml(xq>tOQdT#{4-a}@c{oc$X`nyo$nOIN#OO%52U#wc;skVLbz!2vm&4|$nI!4UmB$eaY@U9 z&02~B#anwB3XhrY6e{ukrjq%|vG_)$HlQhcmo1vi?~EFfkgyDVT7UALcI;_ma3QM% z)pP&%!h?^}@Z(gpJs%E1If|KE!F+YczFt+z&s7VUIjLX?Y%D92#m*c2uZ_k($qK58 zLd66=hrze*zaTs%)Q$O0xGDH3kaBo(gur!c&9$7;PrcZg2((&PB5T zL`tl=-cD|_Q^a|{1bFpLbg5%E)mAG^kLho2tQGUwq&@Cq+@Q;d*Loe1PiSvAmydxq zD#uRT0^_8y_Y#)myt@RZy#ZVaM1yPpPh?rNr;V5Fo8WWuoXeO%rr{rW_AIos+C95Z zd@Ad7cW2y>zpOl@w*lINi`}Cr!Z)<2Hu7>Cd;1D@7;kowjec-{FDO0#pZ3>C=G(9? zBZ9n#?`vzAj|K(>ViyRv^LN5&zKuU5gS!&Bp<~ILIK^&Knz1OCKawC%s_P^8DJ>>b zd?ALLUXIvTImbrp`UZaD`glCjm`xMcETETnUz>IO7p?r6!Y?$*KYRHkd;;Igp@wPL ze6z)g8qB~pUSWJ6L!pN#QU5q;XFTtr9&ahsxgHIhexo5Mo%u=X^jSz@&r*qS#e=+^ z8x#B@4QGf@xgAD|u9Vf$D2X1OcbC(5Uo@_NR8_B$W|oYQmTdjhRs8+lPv>3D2R{~i zs(0d*O}Gu}31BG(J=u7#$Iu15TM}?fZVqHku=G2TgkF?Me~gsT+@|b)SLbx#+PZQ# zA8Ri7O%EUAmqtW};F(jy^gYEwuUgO0TV|O4bY8X<@im>DuB=7PUo!SfZ4tlBA4Z7U zCU3e^VZwgv%ZoJm4E$rI9Cflk$>ed+UphN!nVjYSwO5Q4Ve-N`R~;?7bd=Ek5xF02 zWXaQiQS;2Bms-%+#WcL>Ft|3_ZZh`u?nLkxxfm!rD!FJ<^)bKz*)j@ucyeHCWsK(^ zc++Q?E}c8vlgMI<9ug=uGoCCXw>S**iMhpZqB1m;9A-c84cPLhh$(M|=$V{j_}iLt zI?~kO1^v$Vqr%Gvf6LGg8~EtI4&m{JS}Cj+PYgg0lD?|a*==FHz}U*-nj5qtRnmCv zJ085DW&T(`wx-#_Os*814rxbejtTMhyKAUVLqCufT{xzNju*H*ON*-!u4$lU_*OA% zMtTDK!-d_VIKDxC^oc4c6le$soT=bc1r-Coqrnf=;LItP2iVj=Gh9@KT`*K!e+kxw zoF>OTCPbs9H8IiOn&+7TT|YdnN%ZVGNX$+>_ztu(uXE>e3c$_?RlN4FX5jt??BRS- z9`v^NCJk2J@Zb0kJbfzr{RI-1Mkr1M8hYF1tX3ua7~wE+W2 zMynGs2=~77a!Y;)&kfjpLy{(Vaf8m!2_JGU=cl2YiS3U|`^>8!-mlpKuBn<6v-swE zo>{9edK-=rq6tlLq8zdo&2~BWH(KWN{)5#EWMj-|CcrM=O;(t|rx7GCk?>AYXZ4=@r$=Yix?YbDPNN8P0ET(uIY)HR@ zo;{ec^jnK`G#*KBnITXf1`JG4E*$`Pm(E`Dip)YgbqRxa#O5}=kc&DXD-Yu|>g zS`R-tIrXY)ML*w7R>w zJ!cGkvcu|QO}jNAQ4}H}6?s^8Y#wxBD%=#ilJ;*X z6e;A;uQw*}ubz-}kJ6ZzJtR4|!Z@ljYAG8&Ce53ZSa#xinX8ANvXbp|2{>smpsy(E zkEZ(}@M*p7F2j!s(@w{dZS3S<9@*KtDGIj34&dIe!p-oI)xf55A^z_)G3b;B$`AfS z2Nc|PaFL)$YEkguwc3C^|BoWk)vZw>$Um<042__M4BVql&vptGI0~GLtg*AGcQ9#5j<~w2FPkC2BaC*3h?FkL5Yv!8w$?F5vV;<)_16 z)Iw5F2<`Rjh>F6hV>{;L_-O6p-}EzSLW$9eY;GCys?(r9D+j^n=L{vS&1+>Fl7^wk zlZwmfT>j7GINe@Z(6$*f;$6D7};!k@3#rD>9{0-dBy|TrRWu&Mank$8H-n zC+l?ax5NjAI^h3UxiOiT)$pKOSWzE4M`H8V93P*{cZi}US~HH>zPx=|Z*=zZT4~Fc z1@T!9%a4tJBD8LSWhe4z^AYr#-k~QtCG0zme?EYYsqec@$*s_MZDjq3-C2M4(5B#N z&W%tjF3#%5X&7Vl$^CeN=#jeY=x+>Ju5XjLVO{pR zxj#aC)CyO$S`B^4a-9<7e;8*pq<~#TDwMV#zocrX-+zg(Ci3Iuubd7zQRG(*hTXvC zGJjdeq_b{6g{X(j=QI4tMz?h8^2UfI>9{F?zZtju)%4ud1l!&R%Y{iBB*VF)YqSG zxzgPXJ19JH9h0_r>KYNJGdK;mawjJ~mjuXxE))nZbFCWzcGvGWVjSyTm!<5Mv%2Ff zKOgBI&HMjbFvl-7x?^4;2`BT(dkwA&a&|{eyi1ryl{i2x#E}Kglt&Xd+ZdgoGYUKd zsG7G?_ZNXRd^(p6h{#n&uHE|r;``sd@2kJ`#9ro{IIcSm#jpl z0*d%9ZR49etL8v_+u)L(5ml>5J%35lpjk=*nZfg*YnLhesPehaCaf_HgMCSEg{bS~Xk z=VILTwCPZ0Q|w64sx@%(BfMmQ_|Q>{G{|u!4KvblP{J`^v)+E>ZzQeu-U_4UyKcGd%mJ@fIu}ePaS} zl#d@L|3uKeFfLI9ET0HB8I@%&M}Tk$9Z5;X(OE?n(b7PY@l~sPBPJ*XQXKIa>*^p> zx^Gv2E0wvXA+s)wVtIGmdFMgE^pJk&BY$@Zs|{B5U0T<_B6oZ#Zkq{^`b-Tq$xI`G zOxOW_lx>7!7)eyvPsI_>ci*+?URe0Er9Vs2Rp9v7Z_xenfF(`OAj2n7emS6 z5!~%^4IQmO6PuMCjgy$pVa+QB9nzJ0RZcu~f3c6p$)2-Ogvo^^IFU8tN{;vF!XHuQ zJ&64g?XHu({UirhxO1=V#NyMthk)Yd>c=yn)#*=3->jd6f(f1Rx1rLQtxXB_2d_ea zz1XXz3d^5KU6!U{0>q5Y_ivm#97WhWYH+Ssds%_wy( z!aF0@j-2fY%Q&8o5UJ-Tz4eh5!wt>a`# zwwKA$I*GLCyyoyh>k?YMC#R*ad*wytiPaDG8eEX|&J}IE3><#5HN^8;c_fB{w$co& zhV$HlCiM#sz?>Q6-)0fulIX3wY>uV&NOPNpRf&6L3fa~+EX5T;Ue0R=A9ffhpHa+d zvmca59nV=|8-@2i8I3&JjNuR91ZuIkh@ zR)>>e7Jp4S7UDG+_L9Dh%4`ePWfoBYBe#3+&B=Hs4vv%L(*|^|OK=%;rCL3+R}@%A zDIx#6kV0L2XjI?sz+P=J=L=n_&z?UtESDBusw~C+)I-eD#)jn-@UF+n zb!$wsTRx118w|E9AoJt4FR#4sIck~|h;9_z=SpWW!-j&SeV!3BJiE{JrLa=FFeP zV6|LIL_~yX{}Lwcn>q&@{d#em3L)1H_`EH21eqY<5!?Cc-B>u5^4^6#!DT+l`=PE` z?9RV}GxJXnZ|ogdP6rVNgR@FIU#?1v&b8W51@&9T#CeQrVUisLJ4cU zh|J9#(d3o8t>Aj(N6A)ylxNMLGqU730C43x@l8ZV8sqBLtz9RZwRHl6C}0KHEsGH_ z588|9W*!CKo1XT(`K&Z?PORQ_mEox%NYgFY*-(j9J3Uzu-OVwRcv?Jxby^EEP?$|Dv$d_){bs%S+ssU@&29peuKMH zVE~e~sAgM%M#CdSTj-#M%=;veU+&X2`KIDJawNvOwj~{y%3hlNH;^ZKb#&5E`;Rp= zqXsQX9I>>6>Eq+>6(}Xw^s}|pp|{YHv(HNFXNWs+p{D(LCcdR{!y-Eh$yG^qGre1v#Iv<1_vzIR3mz5CmiX{7R7yGVQI(+>n(4lklmG_@M*8h6? z`9U%bzs&QtFOL=i{I>Jz)PT&nzAe&H)Za|ed?jB1mn)epci!21r ztSrf>ZCgY~iZ6a*7sB^DIy6$&?n04s-9OA!eT}VvoFU_n=Yl~eMz=Q9*oXWQ1k>JRnuz7lk0p2uO5dRo zlHKmvw?1mOU(M@g;$l!)*um4Qb$N%2e)!sD6W6s>%CIc*=+EEZzPIe97f!P!B|NEM z8h^Ld#005#pGJNOe9Vy9M6T%96UWV|f!4dL)td@WNY6TsqG?v+3+Vo?FM{Xde9TYj zEl{ch7MJ&`A`L7Vu`NHvTK(P?_yVRSc_? zoi3_7;oQhgYbYdb;Q7ZvP0U@xK3@ z)1y#qFR{afEbU%eR+^+E?D`zAuM^JZ-}98N#`em&dbUv6yMp;4)-p9VPy}v2c!40S zCZ`HnxObO$56_G)*&XJs(a7T)NyWjpCzQk8)#yoDR-CUMhlDZQs~>>wmw_A~L>Y^M zC$)mJHiHGCd2B)ZHSY;s*3=5?-&Hn?z_@-}rnJC^WVU(OzW#=TW0oCSfQ1#Y$jDu< zrj8RZK2Xo$u|Ii$oOk$_^UUyu)AoP5`*M%OFrTiSlEnLm;lHUzWmdpAJ4%aY=W7rc zKDi}Y{do#Q#If4t@V>dvw5CR(#NocbDk{6?*S~x{yMwop_a;Ej){udFn#ekv?#<|D z@7PhKVpBuhU4r_<^^aanjN?D8d-i9-ZD2ZgRb$4h^1t$(Uw9Ev{zEz+c*g!?(+i0a z_zs;6&Nki-fuA<*;aKw<44wY+WJ3j=vaIHI|7?ju@VL0oxERgUIbSKQWe3Z17R-%S zYKIeRYfw<<8g%Owu`m;UOvLaKmbj5%EqL8;=T6=6C!^U7GvplJXi-I6E)D*7z-Wee zl{zLB=XaS;L>F);WPKpM*h@oO;d=r&FG>-g*Nfwz<5{xzs@JW{lz%nw5niAo6%Cy+*XphuoIb$Gbc>GMAol~zKJkm^O@u2K`>V>MZiBr z@^8{L3z(Hm&t}VS00&uy9=|NiSJnBLr9%g1nW)Dtbf~UOqI_}A;|8~;KKC(@4%WOF zroKwEGeurZuH2MAmo(^+n>N2zL=82fi`GfbN~WygEQzDC-d<$WUfK|fLyy3RULaoa^%U9+A6PRl>S z-m7QG3Qk&a=>gMr!i7h`X-UwWN1P8ahy~IjggrlwyhORVgk1%}7_P1B=FspB--M#N zPa3Ogh*zFS66j^TmjLJ4jAmP~>s&T(aGGnLYgw&R4FB~yLr6+CrQQ;ktS-stSlO?> zct_x)n%CtuXDocITQLjH9gzSk2fq`P`4ugCX#>P{kB)%Vf=ag#uN^3H<$Wpdqye}6 zkX`tuoViunDF}acSQgD)GO^fg!=-{)l;N;m;8bn0&11hO<~GL?F;_14LX*1pU-i`f zy_a0idiA*~d)0V!H?Pw-cfu=vGWAYDux%)!!N^3{3fjeLf;ZC0;{j2;NAi)2Kjzs> zAwI@)5D7}#2`O@mr^{P3f5)&3Ly#mj?3GH!{f^+L4`1C>IUDxx9~?;AwIu|TjrcKq zqBrJe;`;t;`3EQ853ZXC>efx1>-A;&)~k{9IWHH4wK`kFaW4@c-jBRy>M~pkcx`oi z-9G)yOXbyIEGd+{9}%>miEQ#-Gg4u(i5NMRTfMi1ID`%@6$7}4&6M|L44!uwPowz) zP2{so<*|$&ckjl7TzDMT$wSaDrD*B2aL2J>Pv`%nTlGVYBYqo~{}$!Us>bwF%=b`-Hm_&u4zwb`92PzcmcuR==0AZ!+ zDAQl+vR7j4;O9e{@Rl@1pbcTuB4eT)));^GG7f;V`U5(YwfhkJF(KazFa2lX7<_cK zn|^fJrLAe~W+1fCa=mSys^Jodu6#ua+_Qn7M8!chWs?{WnNM1Qsq584o4v8HWWA3fJSV11`Cg<4^+$TxV*41-~EUaf7dT zUQjIMsJb55Wk9%fvD9fp_nIkw7pb}trf(?dE|x|kWT&y=Y+9xQM%uuDpJzb*&6!%# zm_IdaM7XePw@H-{KNzFMOZZ$T&%jZpqytqIaF=OINU7MRGnQr)$y#A?p27(C98PhT z%=&TT=D!nX^9Ja(H8F@7zC4LwexZJaZXzE;_5u29IgKn&TuTlZMvxv&6pWF6piJ!8nOR9*8Pae7DTIY+6y-x6i)=0<$67qq*mhgh^J^@0+L4IA{i05ZG8wv@1UrJ&-n=d!O%n% zg5LmfaUUN;D)C#$JX>Vg$64h~bgq?>h2k*bYV+?Xrp&ohN$oh1} z6nx?vS>D7&gcw7+t|ET?PJ5B}^MIkkXLK0nM{$ao@R*nrZ1!XGN7Sn)^j98x&q*FfjZ*YK#ZBv|3=T|r^+ZrF3uvXg4~kL6HC}c z;Z9U`vp=9b`vJjpyxwW-3-25E$0-YIhO#DQf=;^f$>&aE=GOB~b;Bi9>I?k0jkx)x z*l*s)-^UtK;>o-^TqG?&XrRDqtOKGx%__;2v#dt8u?M}vRGjn&SUnuLc8`Jx(y zBCiJ#gU$;w2~?6l+&d&wQlMluT!*nhdAA~_Arq!grBr}9$am}0~D?6F*nv;?LXi1A)9TMBUrLUHYB`g3B zpmr531DMrhA}#dmJ9Ipdvedf$;q)M9U^1#zgdhO*=uhATd-88F@)^9>0N!5hNRAUm zsnt8p<3~ZzJtE*LMl^7d*l-%TrwOl(jEK3$QW`59t%_bZ7(-Srxre)cNU=Eu-kII` zwBP5L0>8u&@fQd`Ydhd?z;rOeLq1{HlNHWy1u)S|b|*fFoK zgUsf0#6k1b%*SG%P2N~%d3QUW(FUjhZ30tW!B=-Pzq_emjEj`Y8X;%Wz>92VEGQce z{(NNY7swb?OYTt=Dg45+!N(d5NJ4gQx!M8gEYsKDsjneh86D!bTzNf1e;kI=hc&l< z!}tyW?=L{xF%fo3b=kz8xCf)izej!AncRU+r3`qcl*^Zo`RUUS@m4`@^opd?)rc!@h!=w$^*0I9fB+6~zdq$|Ja zZiJF++c(7()r+mgU<;~d_=9x>J+w(J;KWxX#zM|-*r*C&N1SraI_CwMRUYBBdIm!o zIk4Xgvo42dTCry^9FgO6`)VXz+T>&F_T{t*VQCqa@=PS42MPJwGp;_JGd(?%FJ92G= z->knEYMr0`HU8TtJJrWd)_oq9vtJ(O1p5ZkzYzc2`V1eXP%(+!`u^J=%~Eh} z6zsI2pTLAdu4vv2^NIi$+TpmI5AUrt+XFl~zrf*U5j-neI-e^ICM|fHr@VQ6nm%>Z zXM2|#U-G?%8UBA3K)*}lr9XLuEV=Nb;`Ig9f{mFnxeHGu;OvCvW$>ssuV55StN#N6 zwC%`o5w$_PK06!w+$C{u#_4C#+L0oiJ}zKU0SAQ=`{nG6vY0F4|#sCb|46j zPca|Uc5{+Pth&4it#ZG!@r&M?97c5%Nd_3`KkchvE|yYa3EUl>7h7=r8xtrO*!|Sm zwkzB_l*S+mrt?T`&hcb|@xP1%F78(41Y`GA}Ll%T^z+NjQ!>eE2b;;TT+F8x;LU z*&&mC?<6krkq<(;{zOp36y4QNJkuPLxh51Ymrfo9Sf0giNdL*20I2AfbkZL#iUwZP z5OKZzgmbO{yke`*ncD6z#xN77A{XJ;U%ww%|7x|kl?cWy0D)^XoNl{{qXW0k69G+h z_lMHKl4N7_pN+}vzt5k3y~awnj!0bVWWv3QD8@f?#YRU21wY$!AH?--2EV5UURN1!jaff0AlA!}EOv?~chg@!)&$gCU^|zpBH%1?LN`hm$lUGjH>% zP(SxB;#lBhU8wY#WF=zRU9X0KW0}B;GvbabFJ>m?i-1mG9);9{ng5pAN~_&-UR}^2 zAZzCB=#rrGZ0In`6#*2l=RX;~qrH_)+(8Z3Vl~T0!2Hy`;IhahTfX*10IDLYU|ZZQ zGH-w|^zc)Pd<Q@J$@?4!?+YMH|= zU=V?lPH!MDDnS@JVQtk=t-Bf`kB?W6*Qm|ok&!5S)t;;M82~zl7k|j*@amneZ@s!w zt27Rg3ixdF+qSaZ5Z0jP zEwOOU0Fy(;oCMPK4aE)}^j4kYv_;V1Uv5Tl%jr<&=4iHe$5To+--~>mX5?K3MEkI7 zz$N=Sf>p}7eLWzm!Zh$;Xv1-)Z1%ZSQ%t5CjU2usAEbcVPK*U$2grTnfqS(X59(xC z4eArg{^;+A@#YU+_zM?2=Jkcbc?I#{ z+k%_e{XTa|944ha5Me{f7?5>y<)3PFI+~p9ou>Yy`2MmsTa4nF4#A`?McdNBs4#H4 z+$_#QxWxr`NxwJq#}Y^k#`(&v4&>`7puW>lBLj~7NO~#BzJ*+S<0Te&+W%|6F*y%? zi3K?=SAG+Z)?@?^ljl$?co7ppO$*saz8_dQDKQQBEqX@Fpur0fbI-|LL%|v42M(OU zwVZA!F<-^`LABpfq#YJ~_AcjREgYwMiA)laWTVvid8aU1xn$$hDU(VhEdA7L=3Ylk zvz-;6_0{_2TT3MkmO3vE1)AB4a5@dZsmn_Yfpenz4#(`tOIsRLLzJ234ETN1_jR}7 zyGpKIXLnpIGEqf+-x|gAa~PFa+r+;PPF2dlwZru7%CNL!)s&A+Qp9(n6O8m8|0pus zRioR`=QNW`szdK!8e9aZ1c(weZ$JPx>n}pjBu~2*26>7N+`rAeZF?OX8$oQ)Chzf&7QNJctv%CpHXr(i6&-e1}z$NxsMxu| zd)7-6K*N+7qrk6Bd{KEzDfX$Mg-fRhy1qmV`DKG1|M#WbPph|%6v8xon7*JnRq?Ql z1=qBnANJU>l$D5#2a3-J-?w(=q{z_AfrM1Q9Qy`_UEUiWjZ8?atY!S}SXLfr;L&cb zLHBXM9rS1-Y9oUxM)%pH1 zywli8>G>19BaL?WSooOT@oQwfedn$;`e>LJ8RLuvsRF9Ml_W@L?_dzc5~4S9*PSgx zl5%}-ovtGX3&Wf(KC2G+i+jyi#*}T}&|wKvu{)n=@fK=W?W@Z~9(O_g`i+V1t~0#5 z7kzuiud=GhZ}Zne@4TD1QIP*ISM6#HPFxA)@Fa#(%l}-D>ojYdlTd!3Z-^=Ap@^i8 zG^xtJ4D)7PKIJ8%V`~WUyrK8!;{G<&VW@uVrZesrw*O$cGJro9z_&vR5#atuyLpxR zkI4T$<;EwrWT`S2>vs_x4HUghT4 zbExLmHHGIky)mWKfK#QHlNaOo>%$pHCBAve#~NME`F8q2=>5S&Hqv8`KRu|~ouuw= zzY(P5uRFZdNoKo)1aC?H_PAZTL($zYgsS=QyjfQ9A#1odi7(tMf9- z(Us1d>n$w;hb0ESe-6c2{R>yzNz#(`js>q8dPa<{IC5Oog1aVX@ZOKO7^$yZf{yKu zB=?MS&iL>Xn`fEEi#-qe7@e2btv|f2DT%9Sb~p$}`fQ35@x?*Jqs4Rwf~^h(GQ!cA zV`#eV>ZC{6%Rin-hXLX@2Z>*ULl)LZ^kM3t-xfLNuQZauy`+6 z&0fe>VUrV=%g0qjfJ!0VziRYWWcMSfNjd+n@ArDaHs9WUD&Z|#j77P!rMsKP@OkM- zB;NViY|#h)$65ICmFhGCsp@cTsC2EGz8NeM=m03aGMLt|ZaP!G_0n(Zc5mgg3Xa!; zPC-67Ut--2D6)D{MuO4=ml;et^zR$ckj0=49@XqkfMXv+*oz_6VC8rG>HKWoJ{X-|<6AMf42kWi6wEpA>j1DLcoD5p!|1WpSDV#cp#W zTi!v<{cjniGFq-#Q*uJa^L6CUW?)2YlPBKe@D%Si_tyq`ng(BP^ihdcWm&tygK%)yT*4{XZ!lF+?~?+lLcd8Fr(!3Fm?^H zwTifIL9TA=67WqIkdshXIZxb!QeEn0}wXJ>F@M%3JEF|R$)CmsmHaf1E&!ELPj5PUD-TSgZC z6b{`HKe~Kdd+QkcGW?KuUPE+03flbezRzF4N_Cq3siul%P)M1i& zv*JZ|J>l);0bT&7Us;F~|Ml##jW0p_?)kOn0RLp4f4YctBJhga*+7{!SVF{d>0dgZ z#mjq>iYh0KUIw+4WFN@ZU2d+NWLQg8&pwK?%M1a<)Shu`X7OWn@BPB=zcv2K0r1fl zqpacEOtbuu)o;`HgZ=A+`i&dQ2@zqW6s9x$7wh9*2FZCZhB1e&ldv1=&)!R#oo*yb zF;n1+#}nfQCQq*Loehst*0k@7+%i!Se3Z*nSZ)=a;^8;v!c?_h_DX(N_P;R(w*+#i z${Qv3M^=u%ZJB&vv$+RYEDqda->Y4ko`njSG3Fw46($~KpM@q%|M_zwFCu^Ixzd9S z^vRDy_2TAXaFHaQ%Uvn$UtiP?{(rW$$N5*Clhki22EjQmq>8E9sSF5nu0^=c^knb6 z zG)wxsgBTxys>!1pFaZr=^wBz3lPz8FQrFEiLMKW<=l0wXFJ2`sW14*c<{w?q58g(B z3Tvx8?t1-CoPgr5d>Swq?D31l5VN)3IBMSQll1VcHZ(uk`8qk3ng!hzP?KBHBjSbg zg;*tzu$IjkuPZIs?(1tc0!dbR%Kv)+X}lo>Qs4Z8UM*ioKw_BaG;Q`wdSfgsomB!P zjczeFgp=kPgM1EpKW&vO?Ta08bjf)j-SHkw;o>g*!eqrR3l7E)n4SynHER1P8-B2I zcxk&r+F~;jpx&qC3?4YK+<=bv38%s5;mStn*3{%4KP%dXkA#%FA0$Ib@gjC|e1r#( zInW+(RqPiy)X;9KiuO-F;iMGGI60Z3oY@qcY(I1Y8*O#j(0?k=<>}=mW_M_bfb2{@ z`AJ;w#5zh}V}_T{ra;GMkn%KTTwENK&<2KX!eI60q)-&ItF7VL*?4u%^F}*`uC@`H z0m#XQCSbm(FzRksp2vsXng!;ie+-JttmPF^CR2gK7BXkncA9 z{2pr_chZ7B-e&U&vD$NL^YMQko4-0m)?5=ze@So?|EavnZf|+o)Frjr`DCcLdQxU~ zz?9ih$V`NC_J|258}9l2u6?&j%8}F96Ruj%NrWHw4(O4SL}y7N|G(Z^XR|lVwiz@h zjN=b)6_5LqJ~}FusMO>-`LONh%)D%*jA9ksIDXWm%Qqv4lxwNmj31(tBAcn)9)Gva zn1fxZL9Bla@3!xIA%#nhgRV7z@dj^-EYE4)jDr8v>b+A*K&{W>ueTJpS9yth>jA26 zOzpoj4{Ec;fQuyfvITr`0&2U)K-!v@UhKSirIGyBs9q3&-vq&~nvSvwmv&3IW!|Jf zT753gEpQ^xb-sB+vEq80cbNe8VfV56d$2LNJ{!4b4B3VOu167p7q=<`&MiOymrJ}w zyVF@%-5$MuZgh-A&`#aVJD;UJ!nSi1)XQ+#Ktg&$vdLkzKpMV|6uDmAKwK{%vlKhc z7||yPeax0?&b-)1gWLb!&HDgsa8;FES>3#KjB`0IV|>Dd_o zK1Q5yuE;v=wC=f3wN~wU(7WWXxK)G19T6{h7W$h>!>au7{U<1?*rkyE5YpuI2L|-aM6`f06QK4r&wmc{{4wVuGVl%RBI7mnLhW-YKQ)o`A>?HNSPYSIk0cv(Zo2@g7#QQlSo$6Cv zuxeKLVTnyY@<68wv~{$b?1)6?K^2dvkPVUwksenm7lSr4<)5#2T#4t5W6)(ms2%c} z@&2^v3C$9|j%H`+66ZNc=a0H7X|GL@HRl0W$%f~G{? z3mgr%u6Da27o?7|Udm{e0$HZIzYZI|lCMXM8ZiERBF9Lwc z>IZJQ$Ie7jvn3WAkl$9ASEukVb8t~FW;p0D8FLrdsnzp4)GAA6DO#z04Hg@ zj0Hjc`+AX0@1{(<^@E?v_-w?bWTDe2Wv&1HD9k3e;l+PJ07Zm|gkgxv(yEL{|G@52ODkJ{~XW}GiT&G4^Q(5LsmX*aqHXGzYLt5cSL z>_n_}-OK7S3~b&`pZo0dYh=lD)T!|3Ugg@y%3Z^|P5nUbA5SkfE4m^CSWiqvr~3M0 zn-(UFEo&uY=I>3l`>a-;1cY2qKG25*tZ;g>1a{82xZgIH$v#Sr`k7=FNV6SGyWtB`; zzU(fxJ5GU6`HGjr51UP0O0`*Je~Is9edl*FRhnAyJMk9fzUQK1tX>@R@GWHXy86~T zk4ed4p7RWE{8F6as%1Z0z+b(Zt+Dtqk*%X??aV*z`}4!1&F2}KDPV+7A-Tij zMCfWxIGelDe#rA*1<`URt{X7dW*=NH(z5k*Xs@ZUKUAkNOdJXiid8r{UOV(DY+2r- z*oPM27Pk2UFhisyY?`)OUD;g(}2gE&?{Gp_03(@E}d z`<1*gQ?WESJv?~wvH4XQ)rV~S));c{bOv}8_FGWo@|^q8F#Z;FX-IN!x*P^ahGdeq zXDTZ_>VJt}WynD}&uYW7F}NcVzr*-`omy=yF-o*C>$tiG+k(&7U<}&&02xc?{Gm*! z*^E7EAht)5L?UbpTM#M;q!gkZXa(Sk?_s3)4UY*#hdy1y|2dY7s%Uz_(`I$ncTF>Q zF<@Pl7r^d?B`o2mZiY8Jx<|b8W$GKCqn2m;(JN^S3yZWwSoCubear}wFSIjc@o+g%(h z)gn5D*{yvV5R_DXBM*oyY?wBhT~qq|)N5iL#-=2`?tb9(GXiw-geVa@@kj$u-TEju zTeX~6L0!}b1!qY6Y!quZ;BWmpIwobo4zs-f0qZETgUM&Y@8Ta-`^tRlBZr(=VlI#Y z-7HY?G`GHB*~_}P?2Bk1^iv{OOyLb~N<32=a?))l2;C@zIC$??C{WOORP@nP^lACx zw_szT?z}6y9=LCs%Wjv<#$f-L5V}ICB=I!3>|<^nowt}?5VoZdj-7#BsuNj>BU#Kt zFwvIn-5D69&k)#tTgYCZ5^B*=PZ72 zTx@cS(ztD%zv5j@poM$Pv#(#gM%2-4mx}@Iaxcj+GoV5@e**p+P2hnrDV(8+@d?>$ z$Ah2t3`Ya5*BNJagsW6AHjGXHYUwd|=K6U89#Z^Ls!m=^8g1OQs-Fb7;siG>kRJ`D zn(uskLtqR1M7^HRy_ZC;*zSE*Ka)j}f>! zByZWb`Uu6V;-KOFeuNxmkqTaeHRKH1$&I4c29ZWKI*$lARh-<%qw7s)vzFh-R(Bqs z58{57XEqQ!1tE@Oi?V%B{FnU*tq!AoC>ofH{=3-M?q2fT@EAvFxkqk^K%w^Sl9~3E z`#GM*b>bsxfz>;G%k#&EOuKYq{|fPaoB)n_7N{seYLvcQS;dm{q|lBqg~Bl}U9E4` zperTGMMZi@R<+;3j$N5$--Um(FvFT=&Ez1l48(r`#cvN&gT1!v%{V)2vq>-eQ;-)4 zkQyZMc~8Mik{+whvq0&*$yF~EkHRxpG_afv+TQo*tif>j&*NVqf0f=b&1ShVj;c<( zm+U5Ttv=e){I7RXyQl+dDN`m^i+z~%VN@H_iCAUP^tYBi%oAGn&cgcr7P$E_*wx|3 zRDn+~i-(|%oF+R(vk$D#1A!i;D8Pd2l9Xk@PgUdXQ@EH?CbP%dSk`H-6q>t?#=!H9c6i}B;71l+KjV~r&4Cltg}r<>a?O0xQ)Rc z4+PmDsa_K&>Ks8`dEK$HJ)+!t;^>Fk@Zc@fryIDrsnP)iX#rdQJ;yN9BCJoLeNBDK z(*Gv1uhr*OL+ffzo2;ieyFb2-I7G1|P3HxO_Ov_W(OEp@Of| zKnlUa+N%=(2g5)-zkiSbG9#@OA^T%IGlkQ-o7Q^|#Xj@*I{MCw=Fgvx{Xmyy$RDv_ z20Tvnn#-@f{KBP|U%X_+^(&4%X5pRgcGNK^9kuA#g)tq368U>3O}mKv#n-HLmdFw* zZ1=GFJCe}C>4YjxC3Qu4^EQ9M-YvfBBQ=KJ3MOGhg^>o4qBM-5{gbC;H%sCfMSu zcQ!d{|L{lt`|Q2X3ts|PLg!57Rpf9Z*h z6MVGD(paJvMH$%Mj;-6SUVL3(EWP|kS6*@H(qm6JDpR+aziMQ1?5t0m!|$!wIyOP= zUwYDmIoVP6S^5Qmo_02W`8M2pUh|RVH>`v|gDRDFl9@Yi&eQ(%$xr;f$IYJ2Z8Y@Z z{LLM)=};-y?7u#A_WNJ`@#QzJP#ZBQhtq81bLY)I?N5L0*Pi`YzBEfO7-@h0e)!Fw ze9vn?21v#*e%hZr{x@IvYZsjJqqqO%f077V+5%DKfP?mZ(c7MPuLs|)=5MkfDh&Md z#*Lfb{SP1d?0}qF71wtB9au9E zg>mQm9`{>+_Urfg#d|PZWz^2!-dG!^%G^@aLnq_sec^krdfr?3MzI5=@#NUv`|kbT z^WQwd-LUQa-MV$_``+-eGvDzAzJ6Ttr*dplTiR#edCz?L(;oMm4-X%_f_g9hyf^;2 z=N}X?>f_^FFTKb8PNMgUk4R%D^Eci*5|bQ%vZz%iIL;{ayYQGpY8365_?aQDx?8^= zf5~M(|ADi<@cyqYz3h5rt$YLplA5~4Nn+*kzw@FebLI=W6(1D`oD*ND4b$}-w!HgQ zANuUO8Y@zWX)Lwzu?If#otDijQ9WQ(Ly0vwQ(mMa|KfXex`w_P$=@=_>G)KNN!t?VO~VEtQ|b`;USNE;Fmvmw(_^s12suaKSkj zFI&32)q?tE6c2mSgK;o4b7B**#BY%JI23<*V8JV&^~Rg7TW)>OBO3c`8M$jATZ0c4 z{k-?pA3o)m?{~~eN7|~!ko>J`!zr(M-KL5V2J^ckw+;9K+<^Kx_{a*gV ztKRzZ|7^~5WMuvLOIvysoDKuD`i2$w2k2sNS6_TB-4#Ckn;NZrq+0tmEU~Qn&cAuj z{eIyd3l7|u78JGf*V;q!!BtZt$o8@KyyjzGsZu@-Xm$4skAKjkPJ384e>bn&^t#jE zR%`Eqlf6`e?Vg>+>?v2?eBbcBum0Ez-}DF2nbAHST7P{+oAEK8LbCX~iye=DG!>RT zE_ZS3`e`qC;v=8(&{-3l^ds|V)?ZYTMH_$j-fti3XOjjy5r9i$Dh4hh&k+g;zVR&7 zo#b5hE1&&OH(t~IG`mtMf%N6m$UnoZp0KNj{__2fI_XGIr^H{FNuaSGg>^Za*kuC2gJfz4oO5s?qcpSkd#uld-EUjGLbzxwl+F%QZA z=Hf3>+1>mZR=GawC=paiCKB@q>MNeIDnoZjRxAJMU>MG{{22v%z>%xI=rM1u6zn9y zSF94Q-PcM!WkRyK^3Lb11Sp_8Tf3(P;qHT9-MCCh<^h(}(Tk0%enVakZlX`amjg}N zR1uB&ZV6zbgFySvwCvD~jv? z-}~sj_a;R}!QKEtMU5J}vBz!_jlG+g#GcrVvBq9V?1~y&G=jZFL_iTirT1Q>J>cE{ z=W}N7eV3v!zx?;X-rd=mbH3+0=giEuSu$n(=-Ckt3BU9ppeV$xXuL}NXi?;_U<~c5 z`9Viqwna2~=nQ$Svu1@~k`Me^g$ei**JU9Hv&3P=gB$zkMb_V36@&`)CO;nJc^KOn%%yIIJoWCQH$Ka!p)xC!5=-pYtyk-)^A0)qlzp7&Itm#%o^`lnufcxYIl`qJ2xoIC8f^+~@wvU892z)t*go9^`|-hJZ< z#>6u~5O`B1Mrg$TW?OeV`l6q0J)pO?BR8%jz=k7Mac7>Byb;%;O4n4y^FkH*f4?!} z2F`GpJ~rx?b=`*b9~*gl%{-2zt0;5T>Or?W_rm?J^WjE7lE9z%*(wz3HmrB@ACB1P z=erw_>8yw*E%3vC4AF3gxF}e~_^Af<%gD|8&FpbQaytk2?2fNe!M~Ghfs3EZh~Um@TQ-PCYq=(=COKaV~Z0Ff9tahtqJA z5$}=L>Kt+gexgR>Elcch^L)U8JAC!&xajT4067vd_3P=IZPx?o5nqBQnJZr8z>)dI z2XH1N=@FPF3<6C46F(yP{`eJ~G-uYAw-4$HR0sz4grdC;aY6>leGdz$KIX z{p-p1EKDjwt8&fTHQl_==EC!8t!*ztks?2WSSx-P&RulpMGt=b1`mc5mjId!vPeE( zxpMU**FOE;i^I;j<>aoLbV~8-NRCQi?2_lhQVVat=;2R>d=@;ZP|#_KO+Z$zSUu>P zr}+BM**BfkWmDc$^>8VEzw?m?KxR$p!Uf+Ay8h{B{`zWAk^lij&H==|rb~Z*6OVE3 zbM!z8L?ayjuW`B0KOS@Y1rN-aJUgMtY8Q3VrhFdC9r6B{dxl=qqC?A=BfDb!*vXgv z{N_b-eScUOQBo(AHh(Od%Z>YN!llPtfB0GZ`+`;QNe}HIXpaSe zmAFX7flrz6j8qmOcyVW5{tpY$5k-j70#mgCUzI2}{rloKc+t`TL~gHd$R_)x{x|#R zR}dnKLZL(F)*;tYN&b{^Q!hRG`UP_tHG)p2S=eF$6?s5I>bQ~L{^7Wr4nK4MV=p?a zynl2uZ6?_%>cMnFmv+kR2E6`7p5SvK=VFwn63jL-x!H{ zVYq|*SySfz`MmqQ(h-@+gCNptr6EDM4cEJGI{V&lc~-?t$XF|WDYAmc@qYWzCqqV@ zb;HR`S~gUbLFkz}&#i=hOOnEs;sDax0zI%#a zn-Uv3&#LLap1k$B$8Mo(E}}@*1*{S8e|hd+mn`7+qX5I`P(gjA5rJ*5j^-s_nT`w4 zKIf?X8 zksbG}G1)dTy2?n##nIyh4x%>3CmsY%Jk#OTg{Zd7M>T@KiB$P@1fo?~GH2YCBz>%N;2 zzsQGnowBO%CLId)qt&7=E!sETe8)|hgaVJUdrK!%MdGX)ObfxO|9$Sk*XPXSa*1Y( zB#yFaP6{&}pUNx}(z0{~ROOib>#|4ATW9<{`2N@CP3O`{s?Ts51hH5hQC*HsVhI(Q zSLq=Iz9@J3!{?`D{FCu>8awHqfxK8!7>OSv!mivSMI$otmCIJXJZK2l6Uh?%;{gcF zG0|An>TIy57?D-?zlf#0@->p8g;lh|#rR$01@26~kXZ7$1`X@C?bM=C^M*|6$|Xzg zXQcR5S;${Ff9adge{k?g`;^D8qXQ(UB$n8e>bM%h<%7X@z3c%2Dk;@(@0~V1_s^#Q zjTRuq@8Em>xnS;MGEz1{19^>_`5pJ#uKVU)makax`TJwX`AKr^$pGbYZ$9_IL8tD$ zQP0j{rTx2b=wH}pyKMYSe6mabyUi|}e?9yg5oPVOI)TS7_d9t~=E=W2f50#HVYZaI z4eIOk+#Qrg<1zKWZMw9g3=0A%r(=bA*?3ZM=`dTG+>U;~IDUTazSrhVn`7BhHeuR6 zlbKeGd$*}H>%>j$V_DS$e|qw=2hLfq|Mf&^J^zsbsStny#8JqtTF(6s`GM`l^A>5f z$@~)z@1F0p=QbPl?ozpO#i;kjjr;6tic5?|8{eALNP6dU0G6qzx=-1#uL72dE z90F80XyL*Sy@YEG;V}og89anYvsYs#Nv7ZBBtdCmh8t-q;n#Vr^AbPs98ks2qQm`1 ze#j>uSy!5FAe8t)9{n8uWYG)E+}*YXLomt%fAAB}!!jmA9cChgFJ-_fNK*%lq3Mph zy(UqQNTQ8uN}io@tSLN5_Sj8>#UB;L@0-tlc>E=Y#_XNuFUkMc)9;6{BqP_NL$h9e zHc8b;x}>SfC2-;ebl@}-d0q^={o;o{`PXLv$hghwJ-H4$Rh7$EK62et@4o!Wc{lx{ zYtN4M_o!T5nY&(=7Uhea4SxBz*zws7XZ-27J&qn&&A*2LgVZ&n|NYHv z7u-K{;;bkfA}O5Yi!H|(@rw_}{B`KH%{zE|kSByMJLdX$u0)(kp+CBA=nnqJd^Y~F zV{Sg|%>7Qh_y~k@O#c0NH3X>Hzl2dKV9#inHF@4G=iakAW}`@u5Y2sYXNyvepr=fj zar4=CfBX3)cap7&O-E&6#nR<}{q3M(Z;UwWre8AIID;Pi#*C~{PnNh7$vr8Z&hL^~ zzw$$upito?hP>vcVEu=5{WG8Ac1OzQ0mO#gYgXgw+T|&FI6wVxwS1$1+g{8dSR0F@ zgfa||e6D%xCMAB1Yj5CVZ!%c#DsD%;E7lu=NbAtOEuvr)FOA%SVV3Yx{W?G@#aHm3 z=#PL%60(R>7=@C+ zs^4$Fm~j5Sm)|+$y4IarCjSQ;_$Ple?Kg*BvzR%<*{2ASSYjpYY|{;PzZo^*g8i?$ z@`LsKIi8oMCJn?h<1wSK>fQ@s0{l%o;9P-D? zm8(L8hMGA*YkA;L)$_@}MqG8mtv9`RX^LMGvlG0c{~Is@sE%LC+pW4}_%r!y_h0cP z7u&LR;@81T&@Zg&_+>mSSR|(SMSn#tnzps#7nMz#HA1f!Lw+!Ri~b*!C73osA-~xF z&YQpZo#&VqL?GG|{@JVVW=TH1Q)(anF=*)gxr=qhXa{V`lKEx$qn;3vqbhgacgL>1 zI@`Ya)v(k{=06`kPA@`*`ZE7X45aoGY{*}^eC4FcxSo)QOoZjRjP5JEnzClX=xO)>0u@Eu9P+_gt(#)-Pe=B~h(Dh6mmB|a8F`2q zhPieM!AVK}D}#oTV)O_`8L53X{xOHW)vNEh;;|j}-mWF@%$sKWnfjMOH$FA^jyN5% zq@qKJ>$JzlI^AQu{3b`lkVu$+sNpq}1Z6v>?sOl5HZ;z3TQeBp5Cf zhOB6x@k|S1BeaArig-9uW1Y#z=yW{s;o7QMYy9aOM-G(0@55_xrj0y0deijJ&?V-z z@q4QCW5&KLBgWBdRGjBl4OTmIfZKS>!9gVf7z2lnGM$sS9I;X_6F5R~e-)c-Oq!rX(*o8Zp!`>L>1?bH4GIPlD@#`awKKr25|B@g6 zhrjj3#Bo!R0L1e6ZPudEF&7`Y+4h^%s#|B;*VF#_=$qUF3;J^02=vNJ#BcM~jR7l! zT&{Vm#=9Nff477CZq&0&bS6~gnwb;k-g4f(e5DonfioND%$hj6#D9w4HE?>nbX9Ud{u;&;)Eh0oplD%@o48bM-64=C$$Z7{509J=t#L^r9l}3B$6s~(-~g6Jx}xInU+>FZ zoY~XnrS^?Pm-wMDRW*CY{0IK{*yFdpaQLbF?ElNX>od0~aLWAukYDfXYqqOa^9m}T zJBg#IF2%pemR+~#-xKq>j)&YXy}NMz$gYejuIDX!_O6%WJQ0POfm+seo7DaF70317 zsV5h*r%#&2?UM(tdXj6+`U;0xY)^B;hK|68x9ZyyVIkk%Fvw7lz7PV}>3>_bZ?>{> zHD6E1t>Q9Ge(ucq&)v;^ov_v+qcR$`Y;?(eXZPBE6M~L})fX1H=<|5G1}y-yqvMh5 zp1yq0`RFgjuTe??_|dbKMpgW9|GQ572Kwsr3krw-6wiO78Bn}RuQzXep1@2NVAJu415=!#LR>cb(`ex~ji`k^1FCrNyvD-lzN1Y8jeUSz1RFrik z7VQzI(D+~6iGGgV;HUg(IsPb!V=TIoqqODF8%mlF;hSWAJm!u zu%$HexfH(reB$~0Up{TZEbyZ+7`A@-7d)HkpN|aXN(UR7YJy9ORb)|DRcos5zWnjU z^BBl7LX>|hD*4cuVc-7lus_Y5G&}Jbwre#e_*ay*VtM74BgQBEFaP}wZcjyp2sHyt zWo4j?{Ke(A=l=TY_%RcyU#}W^%>wVu@S_A<6J5NG;wL}yIj+Iqa0X{M%h8SVr20kh z&pE@PlKtXL#rQFk&fVuyn9)k@44C*AkpSqZg*Z~+7UGr1-=fDPxL(|28#gnc5NJYp zRJ3SFV*Hh|lTmLB2L_w?*+q`x3x(S8xD6>TM6>l+)ucK13~9AvcGd8IN0-U-c#Oh= zUc#6$ekKx9$%LO>CK$JVQ((~@n}Ot5my{%nPm1kf+wZsZk_glQt6Z!pjHy+3*@F|tQ{MQj@{bz=+!mE za#*Y8OkeorM_*aLWVU-H-q-L|+QfgLvPs;{PdMpCK~ILAqvi3dzI?@lS3ddO{O_Vd zIF6DH$8VAkuvL{+4`2IaieE_aJf@I);OfVnV;nu!C95J`=rGAtxpLDr(J|=y!R7Ir z>Nf+R083C96a4dlvO-d6GtVnS) zlt(7SRa5Y<0wHLNK3f*QiBZD*!ONd0wg2W1_3@L%FAu?+m@7PT{nKOpjL2H}_c?yx zz9;O#@f5!gUH#aKc(5zuUsG~S2Hd4*hgx+E*mC?d`|rKi{{RnKX2x|IWy;S19skwg|lbQn>Bs@l4Z-+iQmMa^(+^1!X`ty zb>f%yVf?7c>%=e3WIJCx`4RD}+-QHIF0&dh#_yUHm7_iy`}@PL`z~f2DAsSjZI@R4 z_S=4){&&e7E?Pwdr@GV6g_C0>!{wM5FYTq9G<6}JBw~qW~ z%^P;ztRp9)mWAzzfp+twfo;66TkFdnSY|A ztp6=K zpR8OOb0}C;E|9a^&-&W`BAGHF8+(&{EHAY`ssDZK=I53#^A*?3f1n^=(V|WB>z=!G z&qMpSZr6hGL;J36_C9LYs~^9hQByv53U`R*e->YZ?{O$t8b9;hiWT3jAd@m(+-%vd z#f^h6`^jMgTD55c{`Os3?Qzs@e|-ExrhGzFmbVzcFWmFW+WoJVDGS|cDfYiR?X}Ha zZ(Vov1qUBG7tCBx>VG*3|F1kUgpCq@OP z^llwDcHXOPcH82T`_H`Q*-My(2QTg2XR}iL@;%yZciUpq?KU~>>J#pN|K<~ad*nub zVU*Yu9`f6DZF%-hzvP1!VAeMNaiox&K4Erw{CY$hvn6;UjqEiD!kf;#Z^?r1EX#k_ z{xoad z()gb&iJxXo8y$DiA;GUAKvolr+3`?&nxd5dd)_gbtdbRw>)3DawXJBAYx@Q zcgA%m@dhYKNaXw(I(~lc!JBNyr)tR(J`ntjdhg3Mm8;6*7gEZ{he%7~tN&l)X8<_% z*3%MF>nLr3_WwM7_F`DnN+uf(wyLbdmqJBojFy6dq>;b~+&0ez%0w1^!&c-^M-<4> zD4}~*V-FtbA{bJ0?#vMje2;vJ!F##R^Xb#P`)^uOh|GIxq%1wO~gflxLlAy5wY^<40h)pzWT963z&on^_l!6 zM(?l?KT*W84hDGo`jP9O`R!lNLT_A$=FCs>9=z)DMcxn&E5aDt0k7MFFL+i$5e9O2 z68?s#F1hQ>tl~Km2NRj()vi#_iop%-kBL?oV$90_$`vbLe&}DvTy&UTUF4tb`>%Mc z8h_-BF-{0bC8Ht`MlYW5GdYE-r6g%qex@{#l=wjaL3i(Ek6-)DCE+>|4M}WZi+WU7 zz?2^*c4l;&(vhDG+xNtQ`<=Ka{)Aozt`A-PB=>SH7K=D?xn?aIHfi024{=wnSOx#8 zBQ?+c@bT}x^6y>tW$w?+e+VH*XkwF#XwIZv2Sa|;ZStQG&k_6|?H@QIGaig|KfL??UGT2^}kM|)8a%9rg!sd_W~f{B48@=aEptqWyJ{^@?;4+zFR z`a=F%@f*k!D8-PEPE6LLzwxLz1(Pk+TiOt+Gt5#Ef2n;2KzaKKUhq(&6u%0!@l>#q z+)YaSp|mcNFZ4fThp$IZASvoibKiRQgDrRNCEdQgv8J$k&D(=NpokePTIjrK`|g`{ z;$mppSE$6Qsl*RC@t;rL9X0GNr+ZX2tBBubt($Tl-nCc9m5VFC{OsG;A06s!AR%8- zxxxpp3>!P_>&>>?IMwf`2zhgqM>Ic^)uMGX&gZ-J?y`KziZR2-z5eK1t5-pZ@s)`B z;FV9l8uoS1?XxG*;3ohoKO{xN7a_QtMF0Rm07*naROGN*uR$HA&Y3!XCUc=jqZI7w zU->tleCN0exIG>eGQIB8?|$?7_=KbszYJLh9=ub#uI=W{T=3y5!zPTG5;$}9n>L7^ z&+HAFRKN6#u7QX2+jRSGtvj}=RjbB?u~Ytb?bDxsG*15scoY~r>RT#P9N?n^%L0{6 zcr3M_d7hqojtSnAKGz2Sj(cwX>no4*6iNZPkB59V@`Ev2Q*1vQHfwmn9j9*Fe@ov2 zSy_4ipM^FFgT+$0U=ekguL7)s(t zg-k?a-}t z`&PAU*W}jgBiB7U`lB%ke^_4_JMx=i`=%GsUJZTb{ck>>Q1YM1mTS?z#cl`nX}3`u zUfBHbl}{#oIXPT0SHDREb>jjlUzm~c6BK^@ui@Mywc>*XbA=E5@`~dmB})DK&Cf=@ zHzs(_{0A#8zU!=Q2J|BHn##(%FMIU$$KQ^;h(Q!n8$8=*?`@?wYyU01!Tz^y{W_a& z+il9%Gx_F)5Lk-N^v09#9dqH~Ng~N?y!^MJoaC(2|JH9%XN!Km7RUFkJ*6(vmpMkj z;vu%wzog()Nwxo!0wuD>in2n7Xmd4Wrq%af585Iq)}rSTfWa` zx&PCrbl$WRnPWt&k*OA}sfabpmi50`k3fN_AJyb?N1b8qOWl zK*zf*YrgoXG~-yQ{jWITk|UYWH*4vO{A{s(&wJmxj?3#!+AvRcoHkeRCU5O4|8`+A z5Hc}}rY{&0A?+K4@$_rQuB|bo#6O2-Oo>-8!jwSh);jTl~Se_`00BlM{xzu5mCdEP-xt{47Ojeq0b z-H$%|!27RyEW9wefsM!1@i;1+QC->iD)5Tm4MR%v*=y9S{xR z@nOclyVdQ(|2TdYr8QIng@NGwU&qfeBO_91q@xc&jaLe2>98l9A(UonIV!txo}hJM zcDc`R$WmD@`GS|!^m`yXl+g;GUL)6zOgOwzPyK^!>Y?xeYfj86h`CSg|^UdcI*)V|2S9o*qhg)&s<%n-KkNF@%w47!~1x#BRN^|)-^=oaiUH9o<&*J&6kk7%yXYOB5 zz4!AA4^<>-a+`nsnYX_gIl&Z#CeNv8(YEQXKkLiK7dXS=^lajnlNFKU^J%pqf|jm9 z<9bOl>mFNm+x?Kfn{3yuRi{?9Yt&?}R%SE#;^VJ;`Y8BE&o`sKO^#t5piI2g_))HV zd%mCN?B%(8Ue#GB%R1_}&tBV{dd2aS)42;(oMiIyGZI)xL}J5c^)I^f^lb+8CSlde zRrg%>x7QwjOC3=Qtp0WbXE@tdI46d_goo9zLsX|ANr^#!R7%j88eqkXm?yOLU^4RG zN!1JAEnEIfdUg(B7!;zqY}z4T(}yfbV!B|fg)SW-+8SWcURCwd!`u=?i$awi0Jh$x zck}km^nX*x9nAs?LoNHMe#Oesbj^aJ6%{L%RzByQ9ND3o$41O*KzzOiaKYqm~CrN%-giOz%5{fv;HhR(`a`n2G z(g2;>=i(s*WQS8Lfh6!`?E^<3&3QhRV}SaGi$4~M28$m6QMXv#fBzjH{^QdjIVsM) z`|7Y$|9D*O`gLfAu=jU|5%Jvsa8?a>LS~tDcn9V*H|j z4^>?G$c3#swn&5yIC!V+cI|!5uWzeb83TflPX#@3`-_)9e4gS9g@>$g+4 z_Lu$ryfz(M#Ti0j;6Z)4XLZA=cR4E!8Chhb@FYTf*fl1t(3P?yfmsH?fG)mKzxQ#w zH>g`@>53IMo^sc)w?{&_0-}AJKWpKtRjX^(u7w}0UcH7XdrDG}Nc?u~+4<7HUC^p^ zGc@80N1Sy456azt`D05LF73#-ErKJwX_o`~jCk+!o?CBx@Tq%my7k7ka*T;0zP8w| z=VcFH@T>k8aZfCxv{2xLaNe8+&06u0{yh`@HNj+`FMZ~1_{5r=ZxNcHdkssU%PI-({DJDX+Fn%I!@C<{){Pe zxzxDvHr@0eYOv`4c%lIm{>GsczW~!k$(V|xGM!^7ey#ZB_~M02=FMEh4EfdZ3vcQ6 zc(HydhYVJBjw}@wvnJ2~%XxS6pj@gl4)TZaO}6M-QBm{9#4m#6uLzUM!qT}@m`^JC zC)lN@#AD6cHLe}}`;Oh)>OX7uzl6+ZZ;xiKA6^~bbn6~UR+yK5dT!g}vIo!R-XK+p zWZD+it5>X9G;eA17LDUl9BK-Ku78HLtWhrXck9{tvPUjx+NOyho&NKncc1^@-YXwp zy6`*BKq7t(0{JIHvHzu|;_}>W7d#ZutMT(alnd`Xt)^$tL?rV;R1Lc3S=31;LL!x? z%D3PR9TTf+*RFlebtg|5H+{^oaiL%S`LYl1Kla?JThwCoR630Km6Mi2;l$q`_0tmu zHmFyRJMGt=ddKj0KBxBO0|P7abdJn~%9KeoHSn`?Z%pAC<7bcl#PfCyvwbk% zz(aPnrC>Mw+bU(5c8xja*G#j9aO_}7A@w2 zUe)q`O~)3_S&o__|KHIoS--|tMjr)`R*YW}J=WX|P!_1#2|jY_iJ41U<_+=Ho_ES}mG1r%~jnANruS9y@Z z`|sKLk*9G20Fgq;$FFPVgUeOy^pmZ4HSf=7-#cmSG}V|MM~ml2mNJ2Vg(q%#o~fyi zzxc55_-cOWvB{90e=%QZ1AeLfv3PO(T)DJz{)~nAKg8QVawwl`*rdL12=d(}cf_f2-Va z%Ktom4pQaHB}micSmX`A`ag}IYlbLH;4k@HvWRV4HZ6_{jT|~dQZMgrS=}iSPh>+! zwFW5_e*qOQU0=R@W=CVg92=Ie7Exr1N*F8<5(r{@4B~^%1k`56_m~UH1af&qNU2LA z#Y^-lJy};SUu|pSPsSwcfGH9R1b)R}X#|d2Gt#yq;2& zEHzTn$$!UNT0wpekp!UuLj_?=yj3fa;&MRhc7iBCuvson^AOU1{~aE>=E)$BT*`Xy zm0_n`eJo!OwUsFpK7DJ%!g#8JLN@8S!mbDRQ=I!zz)ceni>oC3MCrpWtls7 zZ-2$#E@;`QWjgyBctD?RciHl~Uo)-dD)L2Ug+6uLi;N^&OwR^|>$t3PKjlTClouHMi-)H2AF=bltPjE>;Ifm(4m{@yJCj+i)sE z!TeEY9rV@<@85g*qr4r;H)7(vxPtGf4S#DimsEas%HEr8&24H=iNiPWVso z6BRN3^NFtx8k*`&H)9cXZvT_^pkF5o7cb!lLn%y#AVWf;?Pkxm3ry*WtXY$0GuaAL zPDWywK*4-|qu!kZ8`+I7FeorIaVZY}QJP4N9b8e!K~@$3c4>+2?~h*!h#naVXKd=W z9xiwn_oEUqXYKYg%!(>bHpbl;jTMwYT>>-i6ND%y#c#2F2&D9;X17%0C((}>_-WOl zbjL@Qk21arR~L+7=;;-yH^Swf_lwzHr}b>>xkdxdOuxGJuvBh9{4m?x7?< zSUdGk$MQKel&AWoSf7F0{`90ho@chvV1&$2F?rln38G-sFPIX3^^nAoLhh_oO10k!~2hlz7OiM_J`{oZ9BCB?d)mu7tUUcz$8D6&!2I_$<13eE8{;u z@W8%(_TGN}^m)7`q)ux-x6LlSANufCilQq97;NgF`bD9@H<-5?(CfW_d@O!5nK)ze z%%-iHgvU70k$~VQOv>?M!Ods#dB6B*UpNC<+IDSy*+b`YpIi$ge&@|zIOXeUS^JNS zLax`2n{2Vu<^hbAaYW+2C-1?u%s6=&{|1AY;USpaV|EKgnzLFDi*e zBPdZFzm@@2C3hW=#v$tf`JVG|4BMr~M5Z8#D+2k2^A=AXJALAqNtJ$RCh~-Qc&po> z7PnYU{1Nesl`v9GuB!P9h?R5`4E$!v%*yg_)CPHE-1 z1=_9$_3iu9?Rn@j#;+OwC4P{s($!Vhoq3OQ*_7dLw^8fM9zM5To!Yt@xp;+Y<|1aj z5lN~Jp1a-wA8>ww;NNB&{0E$*4+JP0HjSD-Y4*ljcc&O_qwTIDl$h*d>-1|+{OQpH zNyd}w)~j>+pN>0qm*1{-FS11D1+y2m?a(sL{Byjox{yzA#^Dk_=iGWq`_8SsRKd;3 z#tnXR=jqp-cGpL*eVQ~$N%W8Uo&asIg{~KIkeul^b}bl6U1yaO zIaM6>k=kc=BS*+bfJ_8+Yq!s66He#}fg#`fX(ZAD#`u$)RsxHwcC=^0;du6l2!L^a z?;a&aG)9>Te@Q-hs^yEj8o#-t2uUo6PW@N0{TKLuj9(;!t7GSciuD|S9$`+wZ`ItR>Q23sO#?%4l?y)gN$!S9l&jatpGG1UK{IQ_!}+Y*^f zC4(4qmiNE2CeKoWe^D`_%xyWKcb$4pwZP6gYb%-}?|2Tf;3bQu4;wNe`kr9;}2hu^+`)7CS zF;n$D!_Up?vME37M4!JLOsQrqZkvKB2-FyfWH0^#5ZRJ^%P2o0CE6n;1PMw=I}j0r zF)mTjyO2A5nZ@V7B8A?Dc`ztVzk`m{>(Hh9zL(fg6$s%qg=|UT+J-Xk1Y>jG@O#`{7>}vs_-2EPe2`V7%1EahX9`v@{+7Euiwj? zhQI`E5c{U7k_N&|3Ko1s8p){|49r5SjxG77{g(t4vLXXtf%^FMk-Hw)2N4t<`jii@ zCwrr0ZWq1^hBzR(qLdpR;x8=aYR&A)vnPG+SLj&Ci%n!WeNEx$)@>LJMO;K z0VnP`_?}m#NYk`o(pO$>6>z>{=EOPaG)WoBid^p3SD(4ekzUue$YM|_2< z`EwYbF*k&2W=@~Cc1KlA#NnzwDDpGCh$AHVCtJ9CD^m$5o_<4aMt(VpIX z{AV!1u{Fu`KH=`(8s++K!v;2A)9|12Kce!ECkvP5akpZeZ;SXe#L29mQ^_rW` zymz6a+aOr({F?8j51&)NZe2aWI4R7YyXa=bWX5>GpDh!14mb;+QpwD8X46S-%~2+XnVY z4PVDL{a^4!{92LAEnTwwk(;=io#W#dQ^wAm@FhrAso&sICL49@)js;1gGrzSdQk@< z3JYlRqyLoZH&m74E2`~>^cZwn>{5K*V%7MicU2S zmU1Wu@8NWjqDHn50mYpo@GDziGrmG4yB1CH%Sb5VxxjUeFEOja0{A|l~fzLDT_49!m9M!{tc6`aYXQv5#$SoNR z6enF<*t%0Ie3Y?^5{mNC+kJ~Jypw`BmxRWvHTn8=YuD@E-jJ35z#1BL6kCFDGD*}5 zez2icyBOCN@sqx6F;Dx0Ok;F-A!P+A^yp2`an8(9jfoZCle+TZi<-4)inLPv&Ya*^ zbrli-*y^>@CL9H1l%w@Gh&pb>{Ty-y|ANdhYnqKo_)g@mA?G>`7!nM56dx%Rh*Bbq zEtLicc16cg?J%ug2z z2yWWM85*maV6ITuc*|}*x9kS1@>3SSwd>aC(2Yk+ivC%|uLlQVkKX*$g5uRTQaIkI zd7~=^UC5NWanVrJWSl;U=hD?#^8c-N+LXIv9*g!NjUU=~ZyVW?r&wU-#F^oncm+

  • G3DXf%wU;!)Co6jZ_vZeLH9}V(dV+&P|}j+D_P{ zAsVrYy=3X;5zw#Ola_6m=RiFJz7MLNR6J(uCtk2y4XA5*@s(=#7b2La(e)AoP`Zxn zL|GcYhDc433-+jNlH{teQ_b$zP$8d8Zwo}2S**>QKK-DU85Nq400F}eIA8H9&xXL+ zbSU^8eSLK8Z|5EeDNDBSRK4X*T>C=~!#SK*?}Xf#8lMXF0+m6T2TTVhEGx-8zF1X0 ziShNl7EDzdV#0+tM{PepmW*-o>nYz)TX|3&yit{Q(~#o@`hKno^2Z||&5v*4WSgJ= z8i1grcpo?X=@?AuKC7vhtnbGs4aUZ7^tr&+TG7l20pR-cZcKqiiQRqtsJ2(zU`GWs z)~82W*;N9s63be1Vm-_Dt>ix&MI;>n{ly- zlJ5`@{61XB0JJ14w%j!T_)0&!N-s5wE&f}y@(x1GiCRrAQ|#Ln>SU-nz%sbM!tsU*t-B^;OfAV7hJd1D{(~agO!3g0d^Q)%SZ( z@Jpt~v;Sr}DmOo;1o@>aY`XrYR1|{AasI7}fyLwXT=tdH$wbE(4wCnlpx?d9g23m< zmB^vjn3oe{&il)lCc&39yAi|nCXb+IRc&=X;XhNpjLw zVMlq&p02#Q$cs;bVBgxCR>)cm8Stv;mo9=VJKW~T03OF=bj`_wtIHGS&gl$%qYxOBNB z1ltMw0R_<<^i-urx(v|h+qcIlxbWZzsl5%eeRrew?XRHuV_XJH9l;@P$&r62)7N7c z&PmsMn{5GAvC&-_?c=yk+rR%%sg>o56n$7obHx!j_?*)A%#qfHpsTF7M?bvKVUzd3 zD2mMUz@=e(WqHD!&*e@vfJ<63;Y54@-OLvW^gFlYdKX9_u#A9zkUDHPy%(wJShG~e zvsw;Gj3Ku@#&GGuv~C%0`F!LIhSY|-?tQ!p@X5q0!{$hn@J{AU)D4C$0yPizv8Yqs z=snEOFC6#O{_$*nuVh0GU?6VK&UBu_{p=9$ar;eJeP^jvw*>Pr1vUh0fwh}!4|5<# zXDK%q^hR?Uo!=xSX2=oAk*>#(4>;lnUjlpQQtD)*R_cE45`CHW2-y9V6a*9Ava3Oc zUUCwec1}IM&6yi&MvmmncEei5V=aO^yjYM5uVHH4&NbP=ACA09JB0tzJJG0)}zzoTd^`m_O$?4 z@d({K*X{7au><~E7#ZJ@;ekcEvNng+F5PV({1o#oX`XXT#z$a?0Pji_cX16VARohV zJCj|65u_!!irK|9m$6OV3Q)B{cQP2drW?a3_F(O0M5qkC7(Xd|Mas7|T!8@*CDS8p zXMj4(Qxk{(^bDlfOK};jn9MH=`6m4<1AkoZg!+{kF_j>lP(wGzXN*+5`9Z&y?u>pj zNGNta$iDm_)=FvZJMFV{E6YsJldAXGJh3D!8zF)?8T0vs2@pOeO3|JKC6kd9E+}^O z*(U?Z>*&^QYIhZ~q~&|tR+gAYDS3IE{fqT0si!L?yi_@OJ9yS!s?Mw5abYgP9LCoZ zIFMc~u~tLbl&9Fd6Z5k1FIzQ;Xp}T*`|eTo?d`y^E3PU%C$xyAx%9zd^zvz9PQN ztYz+wkHk#&+reH$1+W=2?Y`%5IQfK+6E_#3RxGHblTG3uEq7+ zBKhin42r}~~G%KqdN_(b$G-Zlb99t9XUA*CrmNL!kAF3+;8I{To@~15bA}#QFk7{PB1qvXJtQ$D z)@bMvRbCb;`r?1Dd;iL_WJI6Av`uNV;7+?%%vyuVS`pSD zKI=-aItyQrnE?n?np3_;X3o%!DILa2andmwu0+OBF_9mEAD0xyCOQ#?hRjCSNz{t>3{nY8&vHH2i$pOXV-wsW!;3lVxy=k^QsUM&9IbF zthT1QA`qgrv>4<(hD{_Mx}p$RPA7@@7vbf(iriA17{|L#1d>uaZp+}%8=7Lh#Uv=C ze%=%LBewy5JUK3U`*h{BkwyHvw@zK+>I8?@d)i(Cq1tf~M-}g&iWF}SkU;eg1fV+t zKh&_~#4eo%lZv6j1Gn6vz8m&BC7!Zh*8=3sPMORL&b}oFH%W}8x7{A#hABqAY89#N z*+6U`n8|J9tSzOCB-RRY;3$kvsXy*hj-FZOKZt304a1$i=`Di1WC=T#*dZ^wDZLP+ znlMhkGc7HUR%3vwnG&6rny=X&t{Ib7%05^9LvsK6p~va|pG4Oe&D>2dBj;?2>JA6h zS||{U-!4(Il9|0lIHCbublk`Vt)OxAq%}cLYQ>BJ>SU?u{MO2D3g+m7S8XpNJP}Q4 z*;k5%mo&q+P1DvOGsZaemdyN)EOoqBZbf{%X|e8)_TiTlC1V;PLyHoWWUzUX@1f2) zAV%RGmHdAA;$N!nmP(Ab;6DGFm6I3fE>4!rIqGB!Z^gu+djTu`NzXe(zneqN<7DJx ztDk_~z@IA`N0+zDR*`>%MWTLk1?7jUx^I;VX4gg_(x2JC7~lxiebA?sjQ7`t{&h85 z{+LcnO2Qbw#^Ka5JT~dsKV9w>)JYJ(LL{g25)OA+?IO@Lh5v<89MU}}ffiC7jNO@( zn7%g&JlWV;KGopL470~iNL&p)JcE2PWV@c~*BN#1?-U!FzKg1OqEA_J>dTp^{7Wy{ z-PK=r=xg}y;PVA6>ulA3!a2eEsjP2Z{ck_`Uu~@1$0zu2Zj6iFjudhQtRa?9GP&*@ zJ1)bxeCYX?oWAE5`pfz!7E!hJ_B3AF$aohoO{9VLsQ@V;Utc z`?&lD8Hmr^v90Y#!d43p?+VP~I5H8+*?EC>&=uALjulkMaR9=1`KIUCXME#0#wBpO z!tC;Q?v+r-{>t^b`H8I4J5xARvIi3Gjc?XO{mbKzKrRYMtww(S4_ad>w!QI^U?3n3b_v5|3kbKQY|D0C0UD z;1%1Uy_9v%pOu%D>SGGe<@tZS%W44j+E#w*eL;d7&QrSgZGvc!b#BLnuLP*l1s;o> ze(}~>c@7KTF=FBq-Dz3n!PLm$`;*U+-(D-b&VEzvd$}Ar(%I5v*f_0+gg9e{^N_ry zci6MqCcl@obN7*B7c395nuBmKs@Xkf6e#_~9V@9?5)%JWZb&>KONhe~d;6OlM1U^H z7$)cZMI`jaQ*`Ytu*6!fIEYmGXQ4q-Pq96_+KHUii{&=cb|wY(L%cZ*^%H!oF2V84 zx9(^wb{&@NZGRi9<9s1TFjN}`@Rc)3C5fFi7`4!t+{>Fpzr=-~5M_^!j_jeT?1o`d z5bNL{f=cYp;O{~qkNJA1u|OsHBRXn&M8zcRlM zJ4S6EyH&SyL``-)lssP2K4x`1;soAg9l-iQS|L3*3UA?QL?v~>M3famxOT$S--HJ~ zj}nR(-1@UZL6b<+>9kZ#e&V0B@{0afO>M|rzk7+tvfS&k67h$NVY3_Zj!krz-1NsW zF7J!Ph*8>DtWBh#SA_N!gZW?|=WL7g(F5xKBtrZw;Q{IdOswHEzuv`(TpZI&j0VDM zkIzv4e=NDqNM5t8(;nPqcbr^34nOW);TM^K?M@6SdkiH`Z|A-?@p+9irVrm1cTGja zqoCc5+M$(4n^CpKFBTJlI>ZPGzIP={+xTDW*CAZRroX{hdx9ha3#X@PF%q;d#+Lt4 zAIb4^Gn|B8fa+@O@kAROvtvVf`T6)3nUvf#{G_P(OTD)XyUkPq1)v0Mm_b4d5{P%p z&n}GYYEP@lrWzN)2mHhnSKZMf&5CjVx*O2+PH<7EvT^KNf>|qv(~iU3M8^Jwk0Y3> z;bz{73Xs6t|EEm^<0XsP8H<<^Q|vwL4M9lPh2O9A=)mIYH4Usa$M8-=`qe0qa-Fqi zRl>+#&ROf3(SQ|W&s|&=c|qScjap>ki-w_#pDp?uBl2OD@r37ON)^VsxBR?p<*paJ zGqGqhR=65EY4u)RyulUof#aVyFa{p?dqSEsg1{@|6 z{$VG5%Fl{}-16Vw7$5B0n4}HNEU6Xiu^^~clk@)kPnBS~A*_5<1u1wiJ6mw-x8~pVtUMM5?s`6s)83pUZ%zi>aU4f3kS7a$Q%w+dE3G@*o=2a0 zVXU#h)$Nq_c%)$L+wXSFTgf03b3J}YEzFrI zKGo$EbsrRX+9+|N8)#xh-LZA-ciXb^u>815@NoMKbHR=5@(IEUYo~SnOLz_oz^PCT z8j#3P(dQQq;=cWGH|^wuDq9+!iZQ#}gAqAi)Kdd)-P?u%5%zOuBk%9&Xc&MoaN+;lIi9VenD|nzSPhQx6W*r z&XP=0Nt+=RkGH<7f*$+Sga12V$pg>3bPb3l-T^LAyE9GIS6<^ zv$^AU2?!Vnd^o<_hTgcg;N$9FKi)Y#u9)8>n?L@^4dCtIK?R#%7E7Suj|kE=THer4 zDVtZdm?%=ynJA@j0X`}IL_=x0!SPGgn*pSiKmAUfRk6MellUrnZx}p@MN+yZzwCEB zW=fp5tz7CItNS?m!ncl%M{f34u8ae(j&D8vDW2c5PwsAF8@F^*gZ2z4=Ir0beiEmJ zbK(5R)|YU5vB~uq)p6PzACDN(n8mv2&ATL`mTofCeVUWsdG3RM z;L!%HuidpC?3VsG_ajZf=9PwIf^VPTn`Wk~QI}VRK z!kYNL>gQV>GYgPfzs+3NXKi23Opde3@xP>eSeKwWF~4_uh$fY|WC#j8uzxtXx;v-k z?n>|Y5hClUKt@CYec6Gq4~(graJujR@K^?2Y9s_{8!OWKT<1$Xay)KSNR%wSzWuv` zvVZtX`zWz777$t*Ym*ajN%?r?L=kYxfwMh54vuw>usXgEsfQDBspv2pOOJTbi+V-` zk~Fp&A=>4diFMN)it-xEKtyc2tk8IwS_}{|^a#`>8VdYhy#O?W|C!E_Nt`6XPCM#6 zj6y2o1n`0;dDB~EO4BSCoMpArs1{w6sAmoH5);Ko{u&&KVGgh?4@CT3X+A%}F?$kJ z`#~T~e0w)*uOjal_1llQ74Lr~F@6!*o z^)lped|n&i9i<^UAjjkbr4lPwXT*N1R`%3)S_IC(9(9#sQkI5~B-Gr3PUcuzRE}rs4Xr;{ zqJzXu?*Oh{V`jp~d8yv~^vI&_!5Z)~_Zo8~B2%ljV-0#2@++NZwc| z7V{gSTz6LqU-jCDYX3?_7ATgGbfSw=duk=yG3Lq@eT$O9eUj%6NF;tM-W@ya*_^M| z%uqn(#yhz$8lxyu8`oVQ#=Zo31Qopl_f}vhpHh!njTiKOU)fMUam3!rt z3z3Jqz}vM5M1)aS%{)h5O4#K*QMzN8pgSYzO{84cn=b$vs=2dQ`3I_&=}oe1!ZqmD zkq^{yFZY%|`t0~{BhGDM#^V{+PYGI)zRPzZP7m?uu`&Dk5i|ie73SxZ9qUcKSIeaC zIwAXsucbqY6FC-<1odn0rCtQQq53gmll^oK)lG66vT~n&e0%Eoa_!RR;ty&2<@xe8 zdUj8-JPq5eOd+wOyWi!U&ErWR3XP-k@p=R8hMWg>k>1U9oW^n8kLYKJvR13jjU?SB zjbS$hPik`bmNDKMd8;LumU-X||bcfVpeAo2B0cbJn2y^fbap_MIC#dyxy5 zmmAApXJ!-dgh^kJUswuvUd>y`ydKfGSjXXTtu0em*;KA)EVu9+(e zda~DR;&!Pee_7MsdZUn?n(lQ{NH*Qmx(PYpyGy1bA(TIJ(oZdqyDD>GWLz1>62&4% z7vtPWbqVI>Ok%4!6&_~tBu5@u!u=cU^!+h?YPlR%GGsev+t?%RG__F7B zTg`W-TuBy+%40%|Lr+-g*bDNWJKO* zg{89CC7fFIZ}fg2^#WUPB~nBL-KpILyIe?-sjCYFY5ure4^T|Eqk%yhxDWz=JgOkx zi@bqv2u7caN&)MIb8j#imV;fe+~MxgR&_FSNzRf9^Ux=B+%K?_9Mkqo>knrCY5?dF zK6?zijHeXZ3Lwtvczr1Mc)gQNOp^b1A&8MknMyNn#u(v_t>QF0Mvq|6h}vEJRds3z z>G5eOx47AM*N;X%>8l_wUjIGu++5#5TZGr?>pV{~vJ&EsPgHFa{gcU+UPG^fLpx8i zJn7Plek+`7STEeT{$grySD6vzspPtGc<`vsMVwMf_oh=d(n{p+eX5wMTC7rb>m>7U z^~adcjvKzidYO7aV$?4#$z3KgvH*BT!t{)t5y=`xW2&$*PO-Su3{C`PHaPG9og5cY z?5U6S$)_&z_+84)d;7#qt-RadUi~vh1d$3;$5mCx1e>)sXl7>o%a?jpe~mTM(E_b@ zmg7&Q1h=Rj8foA2!v%Y5il?vYn)C+dSmSK{Sl{QYFv!-{@gB8an=I-fafr3To!VV@ z{cF5_(iKN5c009}hPf2zBv>BmTU090F@!R}_^ClRbH_EY^(zoSjoWcVP!;zQ`)^BT5FY5sHjq#JsuEl#y=Obg@!y2Mh zJW#~=!+7h4YQHen<3uAdXicF1Ciq* z%lM7C&&VTIMlK$KM;-vd{PRu>x?W;y@OEs<_c%yz{Mx()@@akRJcxH#(JutYFA2G3 zGPMRWd^TD6AAm%iBzxq_F33X9eBl!J$4-i>E z{D?%yDYn|3R_Y8$eT5%LQ?Gx+_|<-;pU4Q(5o%eipiJGFp^G3RbwlpCkWa;D@5|Y= zW=ETon<8U=zY1JYb^50<_vxffp%9PE12q3LBIf^| z@mr^H4XFz&f^~xy^;sR%k@+`yj^y=IgUy_=rG_xD zaWUsRekXwjrSf}Gthje;ihwS?I$#Q_y#2X^k!Rtg#a(-!?9|kdsQtWTh9=>)v{$$Y z8h9ta*@xMM0dOtBi><%kaNQCawkkYc`!zpP=BM+2$->Ne`3uG@d6H7mh#(fg%pCuN zJR(m0l#wF@03mF}(lEbRNs(wAK?72D?pU>W;hM$2WM`%m7WNp{XhM{R0bvbIv}=b1 zo9kBw9$3-XP2M8Xq6OvD8T`|8EN?DGdZFveD( zz7W>z%`p21NsABk{)U}803IMZ$^tgS0TcjsikKg)!-*oY0CJ}&9z#66jzp+Lh%D7g zkt0o{ltLt?Fo8Kdk!+FVK#E?3NVb2g(j14`qoS}EIcy~c0MUfIsP4>^XN;WdaEiIs zPyKF=*cMEytwfgCL<2A4NCaIfa#+x|QW_>v&EJ zAa{!5F-Y@izm6)5*CSJz?Eyb_i2PVp5XLDZdR@udmPnq8tFYA`M*}r zIz~2)L?xx75kZ0x3^ZC600eE7nVcm>>;UB|CJwyid@9RVNbJz1W%1(0;!%lh7yvKg zNCd%$HBtmY+loG(N!T?_oYof}5p;*IQLn1#;|cKsh=$vvI3> znmMU@rK=yf7@By@8Z1;IM3!o$$dM*eN+A+cn7|w!Z2mHJsiQ=LcIethE4fdQh896% z3L}U235BuJ81yA{Azf2Ee%t2MX23{Pz$}@${K%1`k|hy@N;7C#01(1fEERk_fF}8m z$i*ZV;-1$z^iY%{2*8IwWdc3sOE7DDJ&AZSa~;-G!%~f>tTnuE4{D>dW|~to=u)lM zzHmf1b=2451w}w6Z`-R8rj1qWzok^20eg@*K4a|Dfc+KPGjb=87jD&b=>QPeG8Wb=;6tLc( zz3+=xpIh|P#HT*~u3F7no~8QJ@}y3TmOS;EHdloaBEn#Oy*T~;1HIg@X7*rAPWA_< zz9=mqOR-YF9gVp!L-ZEwK=|3RrpQt|nV}x*aC?~|iAQ@0@&f|}z34i|{92ws#e6o6 zCyHw~pSsM-5w);-RMFNlW2_f$_0ql)x#kqFP|`}6QA8v(CloEJ z`6Jpzn+Yj|_<3Wt4K3pjGO%8ZnXZW!?jYP9zZrYXU}yZ{!ESt7zgUwzMr-?3r7Bep zWbYFt1$g8_Q%NI~)CeAP8A(0`CK#;|6A09kP3t3NxctBItNS7H=JXIRp-f-=e(04u z`3xmJUc1C=P2zQ_XrL4}9H4mj=T%Q+DV{Gt6t-4@yJz8k0!-vRXN?q(tQm@y&Rvt0 zv9DrMd1&I@7s}5+H1Ee>48V_Bcr>}Ggeiy(Ce{nbVWKy(X(4YhaJ85NHShN~rl$*_ zVk+MRc-{xrPEi=+(Y~w?pB`UkW5JmI2vCQkeeAZ)G!6IpXc@&GA%2!){)ym*Nx5b59x>4luMEjX$$26)G*6Jf!G z4mfhN0EWnTgao@P*;_Si-ty|<4@rmA7|76{bb1d_MhZ?fB`O_5ylfETFwje+GBN-t zHTclMLsC0E*d;IrPfkV%o+sZQv3<*iu%o5(ejfMvht!u%EO+lK?>HFb3Wp$_dYtR% zxv9NtKwQ0l4(VTP)J9%?!?w&rJbi$oa03k{np5Xlvxy!;*t3q3pgaIF$5H;mBN*bf%yMY-F_Ui;IB*mt1k95+ckC}0 z*iW1|h7uK?p_SK|F`&Y`2=G9O_YxfjI)XA5z4jd?I}8$o#_C!-c)*we=n2z?kR3cW znOjHiSm{mey^wm6!!X>+EYj^u&Vu#YFXTvO2ZK$tk(@rntFP!Ggrf>`3O-!qXoz(1 z;stL!@ueqI7u8)-wx`|L?Oswbdi21=ie>M8&3i}E@xo0VIS3u6j!9(p#7ZKhRU}m; zgT_1Ypy(TY$}TFFg|<(bO|mt7N#1~j(t(Nje_SwE-4H~)Br-moG9vjbMl5>lgg{fwv=o%Y$72rr()`+E7>Pg) zKp$3X*)K&b+7_~@q(x5s;TyWp0BB~6AAQGT&lf8Zn+PWy6%AQEng=k7F{17PXe>Of z)|^@n)Y`A32}XJJV&k76snG?q6y7J&u1@1-9Z%~{*{m)9Rd?Kr=j5Ng>!wAsrb0*V zy%}jw+er3#G9o5ut0xzVXsAc=6YnN{nTF=n$o74wlFvq7RH>m0U z>|7kn;xYKWd$adv?Jr-cT%ATWYd5J`uSv~PrAujB$iPdzOjI^gtTkjF*zM0guza2& z%*xuAvo}Ww>o%@Y(th*L$&zz8c=*WCjFhbWynK9(uVR(*cmiMej~)4A>Ac@Dr+s>I zTB)+78nBu^M93p3?W@E-dT-z#Eym-ZG<;o?Nb?|y~^V;p}e%rJ+9r0viw&U_iLL#-}A`>IP-sLnh}=_+{(*U5E1XD^;u1 zu4}87?VG@geT)$z6^HV?gzwGVkNdz#I3}vER;zOD#wOwTK!#Hn9W&U(JKp?fcap%4 z9{pq4yfs_a?#z(x)TVw-ya!sD*hp>eI?`8=kChcCPwDy~I<`ryGs9)FP{#fy0}0_Z*?Sw%yyU9yo4joY?t z=vF5;ZP47ikwzrR$w>{or>A7)<>p~bRjN_}r<&{u2M->eGk)Q=b-Qp|5lzs(TWdCf zga?7+#R$RiE*PC0QEDOF>%@ea1Q_%DV2_cP=SGMkjDltJ*J9MC?b_pwdfQAO(OJ?S z9T^j>?oQ53-khAWX-{rmUfHr`Dpo01uVtMYb-j09B%We;;?;YA*5g0|Gt>6r>l+B8 zWT}!>YgG#GBCyNKOxcSbUZiLd^fA0Z1Y_n6R@h~2U%v~xtdeC)nq8J3yc1->n}IQY zLecV-y)oXPS>PV3@$2vQAtQLZUo_kW|NNz?ygdma#`v7HTj~c%Wrxjdi)BkfXu`& zDwCpwpzDP=rFuHraQxJeCydm3ZCe;w$M!FJIzvUwF}F^Sv^<0Z~We5A5MQk7>v z8~Oc5Z+-CW<3|o3X4vaiEF1Cm%hwKj5GG+Skv#E~y0U_SQitkboKmrh6vS2p!2ZdY zBOS$4>Waj}6jHCNASQn*8)Eg}CzySIWCfj2kDJfJki_E@TM-s-Lt!+v~#k zc>Ny?ej$5Pw8&_`KzA3q0S>X)_tmiM>E z#hq|DLo4MVNo*kiAUu#)|GMszXGX4vp=}0d{DPrM&B|vDJZWY0m} zFT4p)%U5s=aRYPC*oD(a&R@T56HZRURRa%}E`3hA?5+WLmfkVfn_>$=&H^NpIQeF7oxL8yQ{Pg~2K{aBnpcOuyDu{bdXkKIarA4GpKRRHo2mdOhd+;5IBn$I z^-DJ+;WTIM#3?pUJOAV%!!C@MaP6J8x;u^k@TYKhN-S3f-RayRJ)}h>auE>EPa|f2 z|KbnhEx;pPz-zHCbkjxsA)F6x%ZwS8mY�eLz?Nk(6 z|6eX6=9ZnB&;M~T<7Af7cBSvl+F!}OycsWk8Ef$Xx<|#s{1@uIxii@9bf;mgVU4FR z8k|G)i1{utzI7tq7qGA`HDOFVWJJE?$1tMIQ-#?)k41{CUAkn~)-427uT$^bi!K}U z@w;nRcrQ#&9rZ0QXXprZHK5PWI{-JO%b1IiSU_cti&Rv=V9ConfEmL9GcN-90cj*K zmLUF$0N<9ru*PNY9h*07UcaVHnX-+VpIEI{Ed>e2qY5Bu@8DqogqTu^hX)7Tv|6DV zc-`&zXzN4Q3=;egpS*YJ%|lC<_HLvIQmQW+){`-?=D6pOx@+gQjlXAS@2gm`Qr$+) zYt^f70<*yxd@LrjWfng+l`H$pJn{JcDT%C~s25+%!&aep;~NSNY&%b~*uive7!6gfi>Q=$EPGdHeT zm6e%RuW|E+O`7YcfoSEe{hH!dhUs!eij?uY97xtJ`)$|Gt+?$_zj5=%Em})f!ycU) zi3uTSY+)>$$z0U9@SVr`>nBs3LJ--r zvCgy-a;*QWf`Hx{TT`z^0;PV>F(}1P{L!OFvG>@r;rB9S$~0_d_8wj&_K$MtsiEQG z!;1Kg;Va}Xa8^eA`?E4OtX_#uUcX5Tbn@6Yu;aw$wT15FfBfosa#yJ2J8Ym$g!f*%_k<>ls>(NwmaG&*v!@%I%eJGS8- zRMS=+BqzR6j7I0&k$3OfiQN(AUQMc8qkgkyWfILy5Xa3G_6AH;+`U0K2pf*0mOVf$ zR)4gHHH@6mk;h>}QCLGB+dDO7gdDaNk0`>OKpcrWmovaJ^N0CY-*XA+fS1;RG;{Pk zoo!$eDdhpYYtGC?v#8q;0TaXZB`E}Ncm+p8Uv)@A_xzZ8B^|+6In*RNq zr@p$M;h6*3VW!;V^ zuXzopNhk`J5`zbS-njRpq#8*lpV4Odg22T`F%dyrLs-3FEl-uHLQ=zH?u&Q4yKvgl+x`~O&`5&5mH}oY!D#|=Yt57usR&$Z2H7<(a&({@vHeEGgn(yZ>Jt!NQ-sdX6`lFcyYoCi!ciV1kr|V zrHptJj3u#T0YXhKUJO34?=&TKn+6Wy45mntEo*jG>e|W_+l5huKnuo|R=3nu21}4tf`^|;ZCt<3~#PaTK3Yy?^9)mi!!b2R`tXiw~cMIJ&uY~!E?(HzQC^pMgtn}5q z6^Z35B8C|gM!kH`?Kqx3nD5<(L1ScRX7p}eLkw~47Oa%>aA+7C>hk`xkIbDsPFDn_ zHR{ysGvt~butoN8y49oKMa&*;>38IZZ@vA5dGG|wnf)($?5$69PNWzXGp9gip>MEcc$ zLdx;@2|QA^w7b40bsD7*Q`1R4wMiu*zsDH04V4d1bevF z#;7K|^#4n)yN%t_CyxHBe!(LEx|18fHaYmr zbED?1PCcvFRYUJbqZqyYr|ZQ|4*b~z(8)hyC&y==Wu$ahot%xnVA`bZo7RgLMuRJf zX!PqJK%-aHX_TA{>Drc~iH?pOI`s8>uZ{iU!;IA3WIzlRld7E5{mi?bc?J7zC?!sv zQi8F@?&ynGpWV224JI05eW5H`tXTUl*d5(e(hyNSek|W&je803rjE(!vWoE;;Uf8zC$tUeY2Rtm7Fc;+CMC@)0n5^unu#U(5qO zT}PBoA2m;$QaJ5`jwMP&UAJ;@{^sW2x4(Sn``g#;l3-9mS)%f><}Z-<4S4#BdoCVy zoAuBEsggH5GDPdOjGN?d-v0?6I~OIaBH3_eHU86S!(Z`!$9HZ?eRcSU zG=zyZhYSWRT8}G4lRlr>`jlqf`gJaR{{a>5MZa|0dvf+gPLQMMjJ}=#g1-w}R&JX! zb`h?B0qocU39IDIdysJFLYsrsj+pQ^r!+sKf9yFF#;;+Nx2!hZDKjnGDPoY~#dN3A zrEUy$r~Ns(&)xVIuVpw4+|-rbxnWnwGg=?ai~Z_FnrLPIfdCi{jq5&6L&F8)#G-vc z{&~!T-xqIqZscF9*GkfyBEy6-ngCC}pa~ojlxj|Fq6OtjoY$a2X9bOiEG`IUe-j2+ zuw7!Kh-~h50-H%jebA@31z!b>`d??Qdi$ZzaEd`RB#x57 z@69mBHHW$7`F@Pr4pG^fBd9vnl{KR@bWY5M3qYa3ri8{vfC z-}3&Jt&oT}r?3%^hkCZJwv}pXCFC&D0XUoS_#K#R5?{BGdpQGuEo-)S^e<-sCVNq_ z)dG2l(Xz(;Z;?+xV@P{Y#(2Pj{=_atCDzolVwgKa{^9Nx78FaJX)Bu$s3BFjlt}Uu zLyeX_HX36mWxG279<|kb82g8|2H<0k9zHyK+-TwK(eGl&JubKiYq*9H&aUVD_=CZ7 zv0lvRw9CP~TwcpSVOp{|j=_SG7~X0SNLos=6%%l_pLZaS_1ds}Szhh|UdsT6wlK$; zmHYz$vuyfRG=Ir6Ytoo&dvzK+;^Va|ev?Z8gykQ|S-tcZoOll#+!rV9#2Z6gHnR(Z z_u^D~0L(ur_5}OLEO&e;MFlau$wTUdgd6UEEan9V^s}e?B_7!xn;h_EWu_m@4SW?D z$m!!pU43SUnd3&{w2D=bYVO{-{o@y&{NP!`55HbSzO(%N+*kiP?A{^$SNyWzz<%@F zEQUvq99c4F<~>6OyfEw*JkCH1S^qp`++WV^jLQx9+^n3-(H18mm={)OPwFe?5N3}b zjqlL}@?e7_CilSpsiVIZ{Jy;z4Bf*YJ2vi%k1%uU&lAV+?1e}KfUptLh&NumZNOQ{ z+qXIVAOSLSk9VSlGp^+e=MKB1H%{}!7liE_ekal>ZB*n5x3A{+a;xyhnZF=5EdG}H zOAW0k>r%*)H)2?-We-nd+xMi@*9APX6l+sEi_o0VzxDV7Lr!n=^}Da)5*4cE@%w); zOeTK!`IX%}%$YJSMkS^Bc^u0qlCjJ}xH75pH!zl&x9qG;jOAFUvj6cpCr)V4VI>9x@9j1Zud3Rd@l{_+Td&bBs9n{ql_$hhEeZS0YS5v1P`+Uw3qJ z;?sBUUcO+C)5!tbzR~n~V9@B7cRlf&_g>?*3>_F`H2RO%^*3G6{o|LOmTMWr?#)b}KIZ#d`u4y;g9!t3*nq_D=(UIMzVGq@ zD;F=y-JcDBw3po-&ARvU0WS={4NBD1{PUObUw(Y^`Q7JC8mq5mpb!s^tzWh5%hzAH zwf~uTgwV4R_rh;^l-l>}^P9gaw^Un&IY7ihS;HaYI7Z?JhNTuB%OR9FGV-Y8r^DAk zuQ_QeR8opQio0{2&O6z#*@b>_zO?F>bvTod8&L3te=8TQOWn4INa63%|1`?NFGX*;RC$4!FcaMQ#@C?S`)C`8?k7FZV9*tLEpeIuDbNJxlr~mTCj`hi86GIWm zFL=-V=pSFcGzt$LczRnqbLONL6k!W@(RpG(*TN(G6yRmbSF<~&5diez5U>) zxLyFI03a}CXAOPko-CRpq^)ar;93SZ#hjJGCW9oEFhl?M$G*gc2_6uO-hok%Mj_i5 zuZ&*%o4F_{3;9p)duq>tU8N`BEeu@C$k?4Ju7Q`p#8wOFPG3I%J@G(d5jMh@0m=k0 z&)xyO>W2tcGdr?oW^0m1U;u1fy$!dxsmCUY=n?DEWlKZ9`S-2*suKXrkB~T<)PMNI z2xw!1Q@462thD9W;6i~v_Swo7>YqSe@N!hjEH$*XO`T&H18J$`9LO_-k=trs2>kH3Et zkBHVN)m}x4JmFZNzfnnkjU+SKSX#SrO>A-zX z-OI`$ik>O78lK) ziW6+&>osoHq}7R#D_5(5#|@}NjwY#ZxQ3aL$*@v1;W7|PX}Ki#YvfX-g>seL%gnF+ zb=c!KUYWBui?DbxEMNHZb-g;RTNQgk&%pd1U%_QGJHw+C!eJ5Ixhcp zKKg5 zK*exy9sykJpNfK z;H47^=hFEQ7Dn0Qvg@bvLIIlQBj-+10> z;ZCkAn7V7*W2%#jU%VOyUrQIv8TIiyqJ&1v5)2ysrTd0SqZ^o%`NFn!tM3}vd-K{g zzQX;xW7Eb5uelflKWqyq-uQENN0E_C`u>aG=FQgBPu+2S!FvPntY5J#oT$h!z#OTM z*BNvq1;yXaEssFJ1b{W^;*HZw=h7g>a9-_8-4CRJ)xZgXmLQHl^IB1P%|PnmK0P*57wX!#MzFvyd>yD!dtwN3!IY;4BWBKY|Hox|r~`-C8xY2@mHG z7b#|qTHp|J6Yk=9jKou!AL>r%vA<4RCa%eew_NeQ*o{vNx&F~1=U;Pn&HCQ+kAUG* zm!PU%r*fp1$dGE*sY)9ZGyJ-!q#9kO^^Uz@rIb$iV!GFPJ~rKE>_?{Na(n~itjK!pWn9JlF1x3I_9E+--3r%~UC zUp$(D{TxesT{I9!^Ne%VFAMghBv-Fh+x!qS1`cIE9_8PVN$rUDf4*Zy0I*(|r{hrm zL73-cWsUyyJs_yR%1)(e3@lrp!B=S)bEi%iW538qM=&|(q9JWgI;DQI7Q0h+uK#Vx z&y&Zd?@pn4PWHaHAOGv?-~R|O(&FVaR120eZ$`EY7=Sk>+C7S(w;6ySb!^e$3-TJpZ{@khE&c=HdX}fnXm^N|DXCG|a z@OueP1#dw_!ZqG0z?2i*M3A@$)#uVHo3-nJZ`5zyxOVclBX(@^zGs=46+3dB`_p)E z+DaIYFg)kN%UU6OO`9Wo>y|9m**lQE@0}+eM)ph+5)x|Ft=IOH&bal;YR(ut>f*oL z0=@JKCJ>6*<6>7jx}4p+e1+I`tNGJ@oH6!$V`Y~H&01V__k(RZb}3)6!sc~rCXD!W z(zl7j)HO;2qzC_BJ71xxMbq68XXpS9Y-atrZ;Ry)J7;FG*eiUNS8eNPcq1uZ_ zVC({L!{)8e8#JU<`;)8Hsek0q!EKw?e(~z_crA+ne1hl8H(t8wp(o9zPvQUd=s_ZV zn^xi%MXVRC#Soj&OOa<~{{YkL!pjck1qM4Sb@g2jVmJx-{7bJEnF9F%2FN3i8o%ZX zQfxKz*KEO@<5cPh&Ae$7a0QS&23Vv>&kHZZ-lIYDmgxU$7yqjJKlUE%|ACM!Gc*Vc z<`l5j^1Nvir;qtglo(v-w{BdgcSl*+9r;#B)Qqv;Nd>t#V4}9zy}`t9K4U5N2G~{N z1vlf`F2;>5W8NULUeudH;?!FDlITcIy(l4=#`XIf_SZOSvFsY+$Oz8S+qrqg%H?qUZAk3?7Q(!VOKy4ksuCrN1I8F|Lv_KV#bN~kF1iosBpR0- z#kMY}4-d}p<%`kpO{5DHi6N$@yYZ>ZTX$|J5pU-QI6!+x2*Y{kO0 z?K-sp+aI=~ARPJ51V$p>v1tgOyu4NCmep!jO5TwU!jGQ#3O7;3BLxtyUbL=l=a$SH z7`UL6K|;_0;4>CI2X{vbm&{nPe)(pW!bD)dOj&yA?XfHG&|_IVd-b;95v@r}ks_D| zf5<%-w>YT@KH#!_XptO)V0Te&hkM4P$6Tl z-}NCLtdLNNPuZHPkBQB%g1_GoLenFTC8Y1keC@9H#N0-0Rt=ihdE}G3(Fdeo;X62E z-k*pf$q!^yQvG(_pEO=?( zK>XH<9p2d6MNDK<#tXdQKQ12H|3fPlRjX6^@S&rbsaYtj)xdlXGpSl70-&i>ts)c7 zue4B)guiy@2Rh*--nG>n`C4fbAYc&;cot3mO?Rj5 z8kr~#xWKynvHj8`?k7*?hW-O|UDnw^nhK$OrUo-$=iVUNM9z2XsH%%L@EBnd&O zw5>OqY$?v6047YLWnYC}=TN+x0Bj%DYaO=m7-l<;bSabt=)*|jH77AzLrmbtI8}vR z7druSe*%p0QCO3IDNfpktkO+H(B5z)jd$!t!-+Gvc}Ac9uRS~*&-!w6`q*!V++uzL z|I~tDBK?AdapI8on!6srCZ4$F3rt7)U-`V=rv&QrMKRF-xdA%0s`82#iBV2XHMyS)_HImFC@l;XekK4?HmCeH$(-2 zb*x>tzV37TveNbC3>4{7;fJtp*%EF7fEgc}wrS7JYBg*2zv}wFmtVVR=9F6X8&N6V zbM|F@`s%Y(VqL#!%jdrsUA=Z4qd9i$w4Qwi-*D%|uRnYH$p`Uxf91;6h?MNX7phjK zY@%cj8k4>LgOI)Zuk4TPF<@NPGTHMp-1EXqaXACFC|&d0qCLBIR-;oou)fMr6LWTQfW;`A8A_{<)7to@KTY;apWlLG-!;yWUKb4Ty^*T zr=D?+4zF&bCSA_zJ#28F6^p!^y-VlM(tt==vJo!jPE*1Q14JW1Mh^D!NtLf1{*Zu; z!13;@#2fW`J(aYeA6v2~e=kyz>_a0Eo-os!D zT(p-D{`Qz)aXIhdw?C4(fm^gcxn+k_TBDQScLy=E#*G?s>)q(&!XUP~I_TtgJ@ayL zJJ$vtg}?{|tWJX_Xmnh{xN_Ki(&)5equ+h$`73`}K+V#5Gszq;jV!%p*h5#|d7pQr z)o)<;-UEhT-Vg8KpcFh~K6=-?oI$i*;}ybBUiljpXa#R~RLh%1?6jWe2X;qE-mx;8 zBb+GCD}=gnZ*XETQJ3E~T<;Co?h=U9%h3zq&kAzfQ%FMxZw`R0C6CygsGQ-0XCk_s ze=;UqfQ7U>W~}^W>awoqpCSRwo$xE}=E)`noLV~?Y=XpbLClC(M&}3xxna4p~jD2?^E@xnq^#x&Gju#R*89WIwD;p0JYu2k$ zyIwaCK6d$wVZt%*O~U02WXVY&sdSw9Xvn><-&~?(33$f|z>7oQ#WawRH6|l)I?KNt6TY$9&OhwUjN2DA17~1ZPBS2O%PR!lNxvIabj-v z!5$Z$+Uv3&yuhRhk*>R5x$f4pAH>TnqDTC@x9yQl5hP30V4sJ3{x9ABK5r)LTI0C< zkbHLJBYxJutP%fXl2EOn#ffz#2{Zq?M zb7+%7${qvGc!E22nnQ^S4twn;zSh3!_wB`s7v#5z-yHlgoj0HsNT+RTdUej9F+&9ok5lFs%J))s+dCqHS?<_8=p~1<9E`n1M zbOwndD9|IC;I3D1WD{)ueMj+<#pHuZsJQIG5D)$_6+s%Wah+am{Mts>iK1ptavG=r zz?!u%wB`pbdg7VLXS?q<@ap_z$s!tpTF^`D@&=% zf;}(kinmlS-U(xz+3EXak_85-y=bK99hj~)IQ1@T(CvM(%R(R1IRQ_2mlfIc7~{N_ zLDz5;9^+tMI%hStN8frY! zibVjCg0>3aZ9$}oqckq?4?7h63(6#%^WzwKBcNHEc6A#!B@(4&%U3+5yLU!}%_-l1 z>#@-X*@FHMmtSb;DYF*3NE+>t2VVOQlw_R1{#!|6}we#Y7tnxc7Y-C zFOL`if_d)%=EAz;qyp#s4Vs!`N3+;6{2lT3tC+a}VL-;`Dqb8h)?CYAjSSAbuG06) z>pq#W_@3upZ+q%#Muj&FM~>ht_t=8%Q^=lwox*op1eZ~mJ!p(oLc&MSJwXBMGy7hM z`T1mX=7fHNTJNGabn@p(3j!pdb3z?!av0V8FUU|oSeg%$HTl5}$ zRqK;tN9Pkpm>buz#a0=cUX~IPx}DeW?&n{}D=l6&iWfJZjqqVHp!;9|06+jqL_t(E zl-hY#e$^zO!f$FVrB*AB#3`#OCXs@+V%L6l&lUW6u9T|e#0;8i+>O(-+ftM z{DVejW`%9j*6pTj%6N0k)Kj|m)DcofN4oazzlw`w=k+BOm9}Lqux1-EBq0fuQtPLk zMQA16vSJtzB-rfQ8E?EkbxDfb>a z5-AF>`_WGN_6wQD0@qs~eD&Sf)W0?_uqGS#umk6(Sl3^RrRt=&K;AO7N- zB}&=b#YKwXgCqPlO4C*+PT83L*4SydD8rna>evMWb^>U0@shkV@Vuo#`2o-b$a|lE zmO+dWjvZ^-ra=LO#OU|~UK0{<@_GMTx8Yg_F&G4RwqL*HLHyqIeK?w@jEa~5 zj{Fh+t}TkLeP9sIian&?My*e2{@%>zKK%X3%wIC(KKjL zw{y=93`t@}Y5(hb5+h1{NhCGJ3{`1s6PRCjk%ie2i1e%C4Ftz8HUZNM>m_R`DxXyD znQ!lxvqyYQqkfaxhye=|{^Ac0^|`tyuVp|921dy0)NW8cobVKzaQdPnsXI4OL;>CD zzPC+x;wI*=L3ffjt;7o53H=8(f@}OMl~@WzQcByeUmJ7q;9-eP11MwdO~uzqJ}N%b~&emwgLctrJ<$h=+QsYQueaFOki=YX`l(7{^mY30WkR2sAb){jcSsF zF;j1zWsD)!Fmnf)&g7ABl^VN8Q72$)*m0sN^=-PmD6DP@t= zQ3NrW_}!OmYiL@u>(HTV4=K_@V#TCh7hgsV@Eg~x#CK9P;Lpf`Tt?;=aVAy_9CP67 zqye*#KUNJ76llZQKaCGpS({4fg6UJJN0;q-USC{;kRs}_iGMZg)Wvrqxd0S-8xzpz`sL~_8Ruu_3|KqIASJcvtS{qW+h1@kvaYcZX*rFH74Ld zcfutM4P$o#q2^?F5?I}4_8C~&wgT?A-+A66KYcw5r_jNYe|!smQ@457!X)h0odXR` zc4SO9w19W-h&8mtfl>iL8QBErKhlN%$zGu|33oQNnC;cOge37xcp}gc&2;Zu5zQnI zDF{ECNjgt)b=QrYc&JTE#Y<8DF(t`KxpJjZT9{f(ii7}zKzzT=5iqL``-{$BF)j=R2)XJz$^z3<(uxv7IE}hgGO3jX$T~R% zz*r>YP8ZUKAS_)b(7{-mvo}w}{y&kuQvyjGs?9N@HI%g;pEUu}%%Blm0vPAu*Jq}M z<_~hXTizJvuQ)li9Mt;_%3ZA1D-n04+LM3GrLf4*sN|z}s^DIsu^=(3%>4Ft0qmmdZwtf|tThh7G26ee;WM%L0;tEPv*9zFBCWU3 zV5FXkNav9HU~XWj9R1^{*J-R#ycPJHv~0U{{wxBXyYt5U?CkRfUll$-mx=(1H%;^- zDt#`!iss&0K_Qw(WgU+l!?dH4d1eo#3hogAbKamUkUgPI-nK=uch*G%=S}^QMe9~A z-??>5oraCfruCAuZ|~x{Gwhb}+;hM{Q)Inhd*@bjHo~Q27xS)ZctJe%8pL+@`soz| zW*hkf=bjhd)E^2Bu&Ii%)=FV&&CEy!!Wgn?J!S1hYc#*cgk{f4_ctwv4T2gv0!ai< z5EdRkuDPp(Z_TPtCgZBpc4|s4q$UY|y#Xs=UZ`oz;GvL2nRviy3kVd9 zxblP$MgwR?4R}bU0g*DS(B$TSG=oK$a|X(7@H~E%;%`I9>KKh*0N+NbNq)eoI}-0l`AxB z*F?Xt02uE6?b*GzdYvTr1))?#XxoFZq`8~wumJ$V=IxvOz9jH`0dFVPs9PBTW{*^w zxkbmu=H%1QwlG($UP1hda3DLkaDSN7NS_cWRu3T`dSky z!&me6jqqZN8FPM*0upj$cf!dYCQKm`uO{J#6yH>4z{0<4^X|9q|Mb(Rzw0{S)N_ZN z*`!r{DS{C0V}Vj|1@nW_J;|BjhW_chxx4*q9AsdXv(wG|Yglv!FmFq{E-j7H_B~tf ztpI~ZHUU*m9-!1MsBLv!Xr)WF3>SJ+Uq`a30cl$DPewn%a)H7o@bub6bf}ef^SO?RnbpzRipij1g?#wR`{u8hS zPZ;zqI@n!SDE}3z#ouMwebJs>nTHP@CL#H;jvV9A%NQ3t0zlKZKdqI7B!Xnp#E!uH zg8ln)eKGb|K5!#7k}~+^UP3Ye3>HQ!FbFzJ2B6Eh15}!@xhjl_;A!2th13NFT876` zWlP1-zLb}P&&MPI$=?py%V}F5>;5emqwDBe2DcG;P!Slg@v0xsO6w_;LCXc*;dDZY zZ63kL=VDv`K1nBo4yh3to0LW41VjC_v0qCr+ML|EdhOb>NqouDWlryP!L%{oa&zYR zQ8z#G3{JC!-=TFj9hgI|!9eO#Eei}7EP1&(S?T7DDYjYDR+#O&-fin&}W(3j(x0*Uuh{BzsOpSc>f7`yi3o zJN@iFr4!5XU3$bkecbm~-+8~pGiUO6d~$^~tx=~Q9xNArTyoaeCPser?v(GpMp|HN z@Z1&J&Apl4t2!v!vug(o@dP_v04*jEnzoMU0}>1codjJ3Dip;l)yRLLtlhDts1aEJ ztR^lgCROvMK4BQtLq^h?;0mKZ-o=oxgL!$G>1mmJQud@KXQyWrvEQRGpQZ4Ba)gI? zXAL%nM7=|=L)T!*1C%-djS0&hzw{;FqxKuG|7$kEqyOuRxyd`W zHg0($OEsRdj`_&Zqqz1b7+DoIp6 z8a-`KMrv}}o|Np2Oq^d~r=w3_okoZ4@!JbuOej$TQyOLaqcR7R&f;>0zi@X%9`Iv# z%3sRH)~pfd4CIQciKcy!BJ@TiE#g(3q$Su{>GP7|OMbFl(Uo^Jg+ zIhQj4m@#^O-)qmo^nNo&&PSd#L+ewUhhIj3i32l-ZZ?jcn^NuH*b2zX)y;JTE)u`5 zNJy$#UK$rlGVk882bVLj=@2ruDm6?IGax)7B-M0}=D`U1QsJpw)BBCqEd^Qtm@7eQ;kP+!ejc}A$;_2md$P1%WZgQBN5N;uRSyp0!!}1EU%$W6Zkty zoj^8$CXqOyq^wKh$>h{|c=M;HtJSH(?Ag6|%oNuUiZ{PCM@;Jbtxq#pXBe`H0s$=+_t)Dbk=-UBQ!0HY0_| ziU0}Dct<)CJQ0jvlP7OYMUm8tLS)m6twSYEuW2DmCY`YHO$G;v%@U=|xiFA!dsrx_ z(5vE=hS{hQzYw4yvpFjD>nnzDPJOz19kYvI)qK6NC|a}#-rwQ-Q<1;F#Yz>I?D>WX zeB~$H>m(K|l7hY9eUIQh$P)_{v~IfF(SX_!ub7UtDA zNW~+DuFh78N?kh-&p=ogf6xPlg=bvaiHYB0cKPoeaMMgesd@YeBo84#Xp>q&# zg0Wm!!OENB-wV-{cne3J=C?PvfR{M;B=7LaOnS;!j2$5Qd|uj6iiJuD=ky=A^ygXl zexbk+^P7*|_xEQXJ?G-f2HkjPy+(~SGvu6o`veT3O7$8%V?n=?_C@3<^7@1Im%L>Y z{h|o?Dm80M_6*c61YiwK|M5To$EsCY#4=jW7R<3o;fcxlqQn41AlTD89YwmuWm( z#^C)@ipIU~*D0kMaA24pV`4zDg{5GKsjNiXe{B4MfKeHyV6>Lol*Z!m^U%3x?}y*~ zNMc5uH1#T{cu0cRBBqW0Zq|e`n8Q_{$D$M_qH*o8?Rz8-vqz=p%PR=|@w0^ZjRXHX z_Cjd%#Q3TJA^bX}ejc3hXdE+0A8$043oE1!i|-+(rxo%R?*G7BO4Il-Wc)_9mlcBj zf2$xJA?6VUQJtIs?aKoFoY)P_c#kQ2arssHp{b1DUaYS48N|tUWIqH^u5I+0y#X?HLg_L9n>_eoNZuF&OLdTj)2A(9{KBseL)a*oLpPgO{M^e?k>FAK z2afLG#Y~JIiwJrYKo6worGjEk7tPx?!u*tbwx@|6!p0R_c5F^bPtHo+;mv<0rDtAf zrWw-lkjE(fx){ONrbNw5nN-7^!l^jT6Q<&#WQmeawAehHe+a1nuO_4kN^$$~=&?VL z2B|OoGmhf^o8+Dn28IwXLkG@qtlFmA3kLM29wyKB_t&Ho*V;;&V{Q})-r}Uj#>syB zC^gN^wp}V$so-d!mi85yV36nMAAI_TH&)D9Lyca`coah50f64dYU;%{B0%s$PJyLZ z!~Ct`AH6jGQNCtRTSJ&|EM-fwSkVhM#V=$isTUB1^YD_+Dzcsd2h1_Y6 zk`uNj@Z8%9KY3&m0x+g~SU#|ELIc7iHe?8$gi5EMkO5!9dg{8@SIjjDhX#gzy9XK< zdMxCLT*DR%iZqNGGSiBAIH5#Qt)vzFVu1)mz4|F-uU?C z=Usj#HC~==?!-Hbu<2@!B_wa&#ZnzMHIZwR1)B%%-$2M|WlVed3%w9#NxZoXAnO)q ze~34Jr9Z$5GJko&3BOW{lBG+QEaOdQ2Bq|Wn3z1M@k!ySerjxZftA+Fs1G1) zJkX#nSb7ZTB9}ABjHz~?zx8dr#)TK?^lL$k2l2OP*96lp7zP`OH!l3n#Q;!bdIN^G z^}>D*B^Cm(hHN76##11-z%zJj;ZaM~OgXK9I2C1Wx8@gK62j{qdLhR1IsuRj~^*sr18@`p<+t`pPDLd zKk29$q%ORcfg%Z8^B+BUM64vwc?a^f(y5meLD2OL^W%G7&i+c{$s<`ZY+h%Gc_1hC zx|Hq6Qt>Mc+KOCS6A%9nJ)I$;;xP@*oq@GKAC(mTX9YX@%m~}Kdij=hu?aOte)`Vz z@!!j#A%f(Ix<1^5@~4qsdzUjp)zGO76-Fo|l87=iKl*4?L2AlQQDa61%p0e9B24Uf z786I%wo>HnU1YmJLC1-sbHO6Esx@oL=5dAlUVgiA>o$LX_A#8&2pAtnoG{{($=`i> z?Y$3QdedEO5tbgv*(YNFr$Cq*!`YoVwO~zCQ@meCkeKY1O%z`uNR5n%%ASbO1Jm=O zL2~^C9ckMt_<#@QnnOBfTf)#kE+};}@CRO; z3@r!q&9l((dnnJlt*2WM)AWTbVv1w-+aFF02lLGVsLn&oAdck383$$52Mjd&qNGDurTmj3g~zSs2p>D$@c*2c~n$cb0mjvbDj?usM4 zoHF#a>#&Ky0{bt!!JG>?Qjp98tWzPmm25|vi!v`aVt*y2g%M6X#d}7wkRB7uN4%Yp z8}+%9g1yJfKmk^g9O>1CNp%5)D9X!;okUO{c{rB(W`6__O$e+5WYq z#dJrM6vGE)oojucN3I*;Dh6lJ;4>OJxBy=Lq44p{kPML5pKetrrvL3*{<|KGwa0Cb7Y?`#%g;uc(ygrNB znb=$~?K+x?i2PFFWNcVd5Iukw|K%1;GcoGZzyCPmlZ-vPr3hD) z@GZl<{rj)G_fb)nDdSIhU|zsEaP-ijq9x3m&kV*Rx88Z2N_`?S;B}1z(Bj3GlJ=vI zCM~vrYu3vs)D_=EnPkD%y2EMRalIxj#k);3WBd=--P0P`n>T4ZZlAK=_*RbjAd93F zPpnijPaw-O_{O0n%X)puNr+%@rKa1t7kHZr#P+rN*nMEJQa}@~(OVS*;~F{?cszcu z2qOX3Y{XL|JccUPq39|?W5{Ju@rcgEWBU~}C>KuXo$)Iso-@0hwtM&ROV4+%Wz=ri zD5+YFq$<@;C~B_K{ycFE+Fh89U0}l!g;s!Yzv1CkWj&2D0T_!?*luYx0c{-5%&oMG z^hR(7V(=Y^FA@9__AGDF0Dg??B(C$(f@%Y8~ZnCxcgvt-yQt%qkRR%U1B>;qK`0VSZXj+Rn3>&(j;I?joo&BXJ-s4tWf`AKKCEnB{< z+@~`3Q48>l3Wb>7U(5xYm6;8t)Xq6L!$B`~Zv=(09zL*cVb5o76fzEsL6<7gMey zq5R6QzvI-JNQVb_JVTLKp$w+>$V!iWAi#u*B;a2xwO(KH`mG;IzZ=e5GHuzeE#`Nh zb!5pmul&&E+>`1wtnLHN-x{?lYeHzSS++uAQZ@7Y?k)Nk&DY^?$>PO3^*O0U`^HF) z)X_9IUvo2~#amG7i~?#BBg-$EQ`#y6CiT-WWgs$6uYV z^NL>xe}8O5`!m{ik!R}-A z__f{8?B8gPwIz;`>mtaq8TQT|m7caYYftP_yU>cEXwbNjZ+ZGEeANY|s2l5O9<%eq zAegqcJDB|&7K2ucF^|fIK%7cJ@8FT28nxXq4wyT}En}?deFq%l&Du3`B4PfGJJr6} zWKBz}PeHSeC_0#9J_TY-f<3{gY`@a`4QWd88?Kke)2~Nm6ACD}0NTx%Auufv&;CsLpL;q(bN|oE;W}&ok_()BW4;4jq-9AibD@j!^K$1*nb2!M z;Fi1Kfy~*zFR@(gii5V*Fs*lF1B==hF!-AF8wnoBjPx{2l`k4C5HIK3D5~AC36zm3 zlFVp^crqDK3M(<3Htzea>(?@42&`SF(+SWd;e1LdIm=h7eC@CYue{^l`O_zU{pL$+ zSHwOgGvck6yY=bctaUpgYt(HJvpseUGxOEJdpCBG@VF@vViVcZG2k70W{((cNZH9A z3Ax5-IlJG$Z{B~M%{puR$m{NT7)8^^{XhWpXJ0f(n?SBvzoFLjzU->{jhpFxxem`m zvNNb(IL%NPIb1t!RdA_B{=>3PY{4W%GNdpvo?eh|JUkq4RIx>%9DLKz0oUA&?`4Vx z%D!!ltLu}_a4&6veMIPsRUy@7M+y>viePcbZ1$ z-oPI<{n|Ci@xP)fflxrw92cT@knuJV5_U%1JCd)oh65I!=)Hk028qCg8JcjY zZ1fmf%t8V%$L%37GKb){BjBhAGBoD!D8v@2Ad*&LKs>#J1gv56y7ijY+O|ISFdZ;C z`@GlSbrO19bXwtQS+5%P0u%AjJHAvxA(AaCB}+R65}P&ad*|5I>PHY}q-27M9?6Wg zXh0}RvS~tH6qb=>V`ACl0Gcp%8wpj{1wragMSS{BZ-yq}47;P|(7{8CCjSQ6A!*yC zMenP7w(iudQl;`Zs{`Qa>tA2^lRa$y&+N@&5atTFE*ho0u`;a_{z}sPTQ3(0Z1e6%iOkE%*2@BgL|<8{DM5PU7%zSaZ&-Qg4uM{*c_4Lmp{NlB?8UaI+{vkNwsJo z;9DWj-|%*J=6${dnns#C7_ddcbQmH#brgN|`qjckGRB>; zECGc5O16bLo=`?z&JYY{+a-B;&p=GUrNRRNkpPT;3`rM3MrB!FtXT`Gi_;>Gq245t zA?0G8;NSx0vi^5xgC&72nnM-OF92$gMeh~@jQ_;O-UV~{fsucLBO&5)26IhPNi;SC zdb__cb@e~gYCOcdE}v2N%kd>AeyPTmS6mkj@a4a^cd`=-JSTNYETB9cJ zWa7>$x!14y%{w7iKSAOaU-8g4qH8s10tP81xx0=!*)VZweJKA&4juaB#iwKOCnTKN z?-GYuN)dpG0snj+gOs|X?-VQ6{oH=|H}a!*-g)vt8e+Y4{%l;%AQF>?U}{Wx!6bRx zW?s$^FXXg=$9}{x#Rjv-*i827){7NW%eDOm3T|JUQ0>C(k)O&l2V8>787$qodDFU; zORLwcgZUb$0p{#}gPgpTDpk62wHos5Nm}yG`c0anMiM8iOfOa02*^G)|N3i_Nn!An z?q8`03UrpV-t(nYKw_Z+f{@132!LKs>TVn3D8()U6Rlm;n zw-T;@;L#zs3@4i?3lLyl+CID^7>wm|8=Fw0-TPzNqVXz(+sSdw zP8L6&$Z_XHf=%r(P-=OQ*wu>{2=@&SK4uynOVEpufI&z+!ELW#Tr4G&v5DfsrqmhQfF7RazjLh+%$L5z5^!N0>r*3=MTpA8kFldoj zuFR=tx1&yC(2z*m>UGVDJ&UlW;gadg4jnpNym&EvpiC;f;j(G%c1;TzUz6pRP$1W+ z8#wYN^U@j14;?;IqPTk~h0>JmX<@>o>J>p4_U`z_rrn#wS_B$Yfwc&OMN_@5cL{*u z5x;oWDpTu{#mOmP5hR}V%eUYGbtpDlj*_?lYVb`?Rk0=W`c}uE9D@v`AyrI;Q(K8ta*6gdo5Y7xa zozokqreZDvn6YqEA{T*ri3%qSOt=HUpkTtKUI>GWkjFfl5WNv+kHL+1O(-U!9joUTaw0v{wjW$qvnp|dxsc;m zT0t1fC7KySoFykamzr=bBJ_Tv2$*%$al(r2Xat*K@TzriCi)d49zt%Z-2h0 z^nc?@mMU((IDY(&W9%Hiq>?pbkWJ#TmWn%aJS6cFzgPt{p;K*t>UbtJBC_HE@=nr;Njuh5yIicL3T^R$C{u^qvq%AWa|% zN$5==g%Uc_y8?oWi26V*SWp3ZfKQ(aiuzj~MNp|y1f@wr5k!$9y@VE|6A~Z+67sKa z*36zY-=1&oxycRUd-o9L?%ChkYp=b(Z|0se_nbMAle>{NINzK%H!LRkjbed#(!9!Q ztJ~c@uRi~rsXOduqmgmxFMn*FZrytf9$NFefdE_K?5gcrc*+j|z`y3WO)O{Z-) zW^A~Tggrc=4K*ifA+afe1 z`{34pID1&EfobU3W2J80Q)drk^d~C3TY-xlV_4)HP2M^pUxM9QoliaXFejZQbJC4G zntk)vQeo8BLWU$hW5g&{&frys;d%r~u4>_QL@|}K!DW|hoHZ&+hg~hikCMu^29A&h zSQ-OQp~7>p;;)x2Qdv}sc)%v_y4m^Py);@|(Zb$rrwvzHX+`D4ugwUfqqK*tIsl)H zz@IL;ixdcXWA6OFT>JMeb_%!Qh#tQ`_j-I5Mx6*5HgXm7kWk-jk0E@GCH`hUxivxv z7?!=)gU?XeVdIAp^+AVPd0_+0Dyt7+r5gLjYp>7v-EG_LH^pbfMVyRwy$ zPBb_aaUO41AAC_(0u|~m?G!)CDPDkAgR+YwJQs257i2epFf@+wSDv4PE58iWu7Z=h zORyfu)~EPFA4>=S>{%}uGhxwOU>ANpeywWdVXSRcIh_q)i-KX*kwJ&lUNC)W+TcH)$U_`H$0{TJa&blHLmil3c#ZgcrcJQx?_*hv;1=SG#zi55uz65 zx4kt6hLA_M^pl5vk8B>z+3>U)|1u~&8f6*Zn*Zj^KU`+J({|e@MDdX&wjndN*==v5 z%0H;WTp2>2=wE%o`4Er?GG_7OyZ(IB$)67Q*Rcokm`$Vafb#FaxcLIEm>2*wC1=;J z6V}f!a>IZ7jn_W$Xt*_($+M`}UU}&c7yhb{uz31j`!lIW!c+2Fe1pN-Gon{tdGVW{ zI^<`koM3RgWIFV7U(p$i_;UeBn&@_zj-o%Sty#XF3_SK&ym-vGiQI$#hAS?5DSNw3 zsIiFU=&Xky`1!X_cnUX0$ugpQ)Q0U}W?cKkBjE!wc$6K;7}!0_w%tAl1ZH~_`04Vo zqZ_Wc`1&g@0))uz_t?+ixsfrii8q)Ok3U{={`1ey4tV^djAAJKq@NzOwGUxb+LEn= zvKGb)b6zP@W)J3NX9{0lruHNw2+x-AAf;~UCx(7{6rLLZ;qpO+s=70HoCiDYf};Fs0s=|)@W?P!jFyoJYwh(mwGo^>W(*ER;9sWAHrU0aQxT%`Np}o)%)CCH zbg~uc`O~&34F-q|;={Mt7FFDM<;CH2q)G~V*2DMz{M2vSeVY8)Ag@$&WEIkpj}|Qm zDO_hN$YO`x3wZY+S(c=nh3KVcxQ0VC-*)|75g|t5-*o%%*U?rU+GoV7T849itdqpx zjVD8f4+KEP0CUzU7u@%^M>H=0*IszjXLtDOIj3EC*-!t-p*QEgp&0V4xKjALmlED} z>x1_c_X<@!HE@2tJdE7X0-fVrsX%Cc7 zJ5!VosBK6X^y{eF#-kx^+S+HZsW!$&D?u^rN#O}VpU!5)zSG1oe>nI0nU~-8+MIcs zR4W1@!uuXE1?gMJQ-M~q&jRG636%j_%R0} zQEP<)@vw~Eu4e{I_J9mNlp%LzKUiswDWdSRf=TGtD8=iQgustCYh(}7d%F^wfx>U8 z|E-T1*IbEPV4$V!@>V+UX*k=$$*zgam_2~Wo2)m0H z&-nc%J0EbczEj}3%P;z`V~^CgW<)E7B)xj~?a{Lr?rKRfGq1XAk3)}88MLABBgY(f z{pAqPn|teZ zSN-_Jui9U;iTU`KPZ+oUhK4u7Y+cp-0pYJV{pp<3PupqlgQo9&ATEU2+(L3+l ze(fKOYK`%e%!YB`QOExI`m5=|eg3|D$e!Q&<%Qu!3B|l1{E~Bi`twsyz;?6C))suh56eB5KbO~>gd|1_kYS{qsRIcnGM9{p2} znv7STfAy>ToO0mj_Z+w38hESotOuX|+n?^e@Qf??J`Tc=7g4K>TzTh1wq`h!;+^&& z1cm?(MTAd0^vvJ>bk7B+2SSAbj1P+Mlo9&BZbh7A#+FFynE7F72w`A1*+pXf6w;3@CdDC?B`%0_$Y9v{5^;#=;TnLnYjM%{74-4Hf;*ZgYi&WCOL zgU|mW3cvB=V#cmlxyHe1jbOEX^AnBO8JFD3 ziN-55MA@0`xoo0u{{3J7KJ#9DsR|}~^Bp&`^aqdNGDt0qyFTR&@4h7xd`e zEvUmKg6zD((eM_8>>%ylquas9?cKe5wUoS~>c8}pPxGB|tslRuju_IdS9iQ0 zMEn2Dtn&6@krLfC8S0E|VxR}5aZPgD5SuTdf=K&MAHK=;;4kgnt@^d!7FY7=SeGPTkEAKWs+ zhs7yRMwwn?Rgm#yLFKyk88wPFFl8(C=^y!>L`#{sXSzVwkj_Wk&0CTz6HI}6{r z>b!IC764(g%(H3_{dQ^ zA9(2H=l%k;K;li^pV@oI?e^Gr;s%?HSaa;`XP$ic?|1(Gf^*-T`TS#e)+k7hV2XgeC&aL9`HDt0aZa2^($AUY^-Fx7#cc6Roi@)JR<31n%%z(iwGfT9^%awID-TdCa zgir5h)#TgFrtj1P--4Mnv+Q%WKWOOCeUA9-x!*sPt$6U>yFRhQ=KFo>m~rdiYfvWk z?$_s)7hig2*290BdE+hD{+{~BZm{9Fb@SiKK>#AN2l;bP_)*R;pI+ogAHdltHZXal zge@O3nhVb0hAeRT&oXIsKxpzNn&!4u-UW?vc`1f09-t_w`?2C1c3I=WP@VMCPkWb9G;u0ppi&7rT6F1(J z$MWqrUV|Zq5L4-f4sbYZvwj!LgKhwEu;crE5sr0;8^qviMsT&x4tCs~r$Scjn9lm+ zx6-2Qpv&?i%J5kKJ0EoDWoQ4K%DMjWjY1RG-*_a>9B60vh|C_O#Wp-- zzI53YgW+j=2+ois#0h_sY1995le0M5 zD?1&50*%`DlmCH6K>~(wHu(7VA4H?pUT=Ln8_a$JXM@lv-0>8cwKv*$&57%0=`%7$ zW+vs(fo;H0XEqx4Dv}Ft_ETs=SynZzjH`mz9mIwwK$#}s?KzEds%^2>PkjQ-?BJQa z%ckez@9otR5KAiDxFLKJIC~)byV;kHif~I30%RwcQ+MC&cR&B*WB+&}B7g0b*MIz# zb0R13@Tl{UFYk+w7r`I$X5W4_dG}2tLW;!aCeF;>R={155RYdR+K@v*h}(8RY0*Qx z2C}?}QamuoOFj6$2J1HPco=^`WbNP#qCqFsT6e**DKycgh63g9Dyz_TKWUb8dL#o>`1FoT3S24YdEsWmDy5Cz+IUwbuw=g8f+{Nt{h@9jUN-(?KKDsNz12KMk^A&$#QVj9< z#cocR4Zt3^@tRY1-XwR1W4zbkKrSRaOF@>Hfi};lvRYT;gKCj>5|AU%)Z$H2uf8&G;Gn*wg&&-V(({HMzfj0nY(qiShv|XaqKw&+S|FB?5*R?8 zYj3pHmOF1uRq(UAryqY7w&G!r9zRk&03eF%3VzcQN+@_V5pC<9@>^a0_>D3~gjT7t z!KNSEAO*YOmq=*lyUV@Xby5BX4Kj1R(N3`~$-6Vwx&C_6X#tub=vrovFaa5Pe#|8j z;jILJyZHtTQrOlX+Ean?>XR|t_t7qAzUP@y;2+7s{^PLn^T+sg%QrU_zP5jY*~pvGVg3K{ z_x7zlVg3K=m-xphTmRhkXOF1osaywq@;~qb-fpSGMGmm8bJ}FEMQ%$)M=c9S4Ff;E z$|O5)1PCVm)T58Uzv$_sg?ppLruo3g10I}??)vjgyMHP!72$Mb{j_pmET5f@*vSL4 zMnz|XNWaA*>B4uy^F{x#p1@#H_0_!3t3(-`n|Gx$?EFxF=qg2!xQ<`hSMizRI6$ie%4Zg+nr3)hBz`i=9x@ckp0 z?&BvO^rbycVbIJW002M$Nkl)$yuX9ROT1CRHEzt0G zM3{$0D$3R;48-D3@_R4O~R{ z>!0pv(6HvZWAHcsC}WGeII=j={)r_tgo(a8Of(;n2E1>j7^w%?Xe(74xYl6mXkphP z;bh-n%#j-$q`;BiW-9Xg-E}q{H)W3x&b;)`I@GKaFB)+;|LxklzH`+HtBqXQ9|9~e zDzmnGR-lOjeE_V#M_7XW zmW;yR54-uF)7mK||Dy+^(8`8xa+F)N9uL}#rH5O3@I(y{9aTL2&D#`Sv!pP8zV<4- z+Z>}v0sL0G?nyoNn*qx1Gnx%&RZQ5VK%={EyCFW9^Bc0;d;` zzwMuL)DzDk2g1i>Y9C+&FBc*J zVSckm1<%53Gd{{&x14nIjie%c%*L5kk{Q+TowpTgKEM6#b+@7{-fYV&P( zvWy_1ZQ?iGYP$nJ^*Qr1OW3t_+oj9b&pc<$xbh6`hB zazfwAdC9!+>~i2?L$Ys>&Gi$VHQ6rDWyF;k2|TSFzlbbe8G_i3cP7Cbwa!Yce}tKy za{c%0rTq!qEj;iMmH&73O*>_aTzld}p$dz9*yvFh;#_cMu^Yq(^{5*o;ensQZ#uhl z+57OLY|=A)s2~dgFg!NfdWVCKI+k{28;HRa9)VdN$}eu}%ECC~I)*>BWv8Rn#(Y5C zKu3CZKHyOML<~l8nH!DTaleBRMR0_N-{ke~)0d4ZQ-rRBN9YVyaW_NcTC*R4zvvso zIhw&M<(K4jRtirxMx-z{N&_KbwlH5oTDT$-E{fm`g#A#^;wig@4;KJNJh$+NTl9|~ z!7SJUob$k{dk_<73#xTD9lOVo_(rps8AuXf$VOq1f4wOazj)@6mH|%(i@aUe|KRvN zKedyE7zqv18jr!&ojl(91(Zn^O` z;d7}iK2~0RFpv^0G5b#Y2S#!cKrHl>~h$)d3dp4NNyM!#)e&J>)Ep# zXW7=3)<}%?lS9#}fD<0m3?3|ohV>BXAtxu1>FB|hd3*KkWg#I8u4H8o0L+isz!3el z{Sjov#GBXl`^>IzS+AimwGG|KD8H!DT*G^4d9c&n=xbzqAze zud-2^orcY5mzV_$7NA`WnY_b?@ZzMHrjiC8@5C@Z-Hk1JitU@P&r<>TzRxp$cm4D| z_QTW7lBi1;Jg48~z(dcv{x2JBF||+MFof=hcWSr~J7K6>j~+k2`sOJ+S3d9$f3JT1 zC-1OJq^eK{ck zZ2O_TR~{BVE=xjd8kqYJ9E_*z>ukK)J|Fw^2|xSQMRz=O%(uQbY{W=1L^_SO?|^}9 z9lj-GrS9F$KWWn~esuW_(>}BpdSy;CMvonT{EyH2{zZRWv0FEr6cY~vUoZ00-_OJ= z1l-)jL{c+uV6pkOJOA*~8;txr1K6>^I;aKj~CrlZSWlzk#cR1W%@INY%kO+~Yixni%dkYqcwz@1<@1iZDcUB~N zTcY={>MBdv6?KCs(R+>F7t#CT`rP05{_gYq{=9$Pf8Wn@&di)SGv~bLyk;h^+EQh9 zEkwWGhWK(ctBq0#s7hqvQaMvW=je$V`2PM^wOs)53L~7^V^pZ(W;ya0rT;Q?>n1u1 zHB=4CpNIYiIh8A{2F0Mtu3KvdZZBk3KXM2byhekMMO&C}#?-uA;uC_>Mlkk*p`S?d@(BHeWuTwP3ek(@BE95H zVC{^HNnbSY%VryV)05Zg*1ROTx7OegJkcuhv14UcB*xo;O`z2xN+yxBo8p%is?rd~MhZ!>Dt_s2j_ zxQB@03{d{QLIV90JMpA2@qLTyVyD1FEyk9A_2)26g7b)Fa^U8K>2C>;#R<;mGeW<$ zg44s$#o{=bQ%Xm2TaYTFTvRYRnW*$T%$qsY6=Unf=i?}1v|rW=bU4db>(nRj~tR|SIZ1I zEiv{Rsp_Cut6G!$vdsB{jxu#en~Wou9P@Y_MiE7D()p+`aA;h<%nPff>apEHr{rPYU2x(r_A|ZSCEgS?&iHLC$v|Fc6PmQK5(_d4jO@)y@9pIJ( zpb&HgA_z@V#peY6C%n%Ryjd+yWu<~ro;;#z458m!P<#Fw)UwI^2#=@>$)+L-8#Xg?lmPIFvZ3ghlC*XOY$BhQVS3fNq z-uHgcvayie2;e^JdVhdpzZSW)&}Evw_go{cGTvU*$fjUhJuY6PIK6rO&KdD?olUmF zxUPwJuI>%&D^WSndM6^_B9!&7Pf*^-feiGAlW{|=fj!>IsZ=`p%73*rm}j6TsP|^Z za5v*}x^(Y!dzKot+A8NNLe1L&vTKHQ@KW;_o!y2drEUgnOafPvD1A?i9FO_i6k>mq zJ?-WYz>v6MzIti+U)PM7FKy8{6UPKfoqNM?i&g-{N$2T4bIR#_Osg8Kn>8UWtPi86 zVKNNj5)NlgcrF6UIht+5I^!E2Q8!$dg&J*JX?wUtz=q|D>buWw-=Ah2_fhH=o8SzU{8h*PG|Nn_pHNyfDn zDTlLSVOG^82L6bi3~g=d%#a=UJU3=`v->iqg`3qywE2$gK@Y7@b7uIxb6USkN%CLP z(R?k>E#$uSd*=g4wfbnh*3u{^{B3T4MW(V<^sg&jbc(4rK$N&;(JspW zh;F&ns=JK&Y3e#Vl~0OE0Mdpug=m5XGqe6u_2SV$KkQ;Z%+!00UG-v`-eI`GDUhQx z-wCtxcq-qkGeTVB13p|`Ekpu8s%>NBdzG1D+8V>Bg9`$AcSbVX5yHn8nIC)ZxQ{kD z%`c@W{w_5y?Rfb%X4HYjrd{)?;B8WMlc26A|CroS8QF41itkM#^QI^Jb9ng`z z^6>rH2pIq`tM-|4VDtGyVWHai`+@@I)#&W3W1s_&?{mt>Wl`hjgew!3?`1D`Jx2Re zuFr5GyIexjCe}_~3qrWy1n}PzhEcE4pJO@Rd^dNUqL+{@23ye<4iEka$r6^PR_;%Z z+;}gcn!PqXuxVnuW1=>0;O31VpKX7lSIzQ12{dsY z$YHE`lbSs?@xpVIJ?uN7&225*rrI!i!l%_2bZtQ`X_;=p!=a>%_~B|RyA3{NK^Z#x za>0uQ-rdWLU6aTRuU1(Rqcxfu*T9j7H@+FDfDU0+WBf?zjq|@4UTNw;g26a%j+>x| zETY-XbK`y*0C5MiD%UjRU%F;@D_(-)O!bPqJF*u`S1dnG%b2R7SwBh5Au&@-W_w6N zVsd&4h@|gMyiwT7y3YC|vU%s}S0b6c+Tq-uQ+sGp+9-PBzkYsau_9{*0Ws!aCMG`* zv8wfMyO2p&fr0K)vZY$fyYTuw_DV(*w*xEePrCVoC`5-Dbm8UfVOJJr%J0!%t5C2= zq(V8r-6-Nz${e7&MYk36IHs?@L&|y7LG@^JBGX;TBF~Ml+oWPE^&yMMoM)5cNm^?J z2Bx4z0GYnWC#Uu`tQ(Rv^`0w`+H3P57w&la_c`y92`~}daG&xjnq7GPt85s`~UV{&TCXlSlv|P3C>y zGV)C7w>x?)zc6%GReUv`9mp`nEH1SEOUb{`KcV-`%_;OfidQY}ygp zRQr+Ip@|zhG&FuA)lNjMOAiB-Ow?%A>8I)txW~>AlhC$j{gHru-MuOG`)PY;#ebL6 zxcm3MWXNhp1&G~FpR>mS- zIpnGGtrT|uv~K9E-mlcQ&jx}e0u$+s%mk~E$Y5weQjj<3oYNupeY{y3kSlP46ydHd z*=?7K|BR;1XG`HXU>bkDTxB5?CH9pmGrBu|3AIq2MIx}c53cJJj4~Qy|M#sI%4J*` zT%D(eXIFh@suZ`wmJy9qA7ku$%mO{;{QATTfR!h%VXt}a1}5gx$Q#|s-)Qr!wMw>{v28^7#gn4YTn>TziO=|2%OGg!G={5FjwYQWF(U*`E)rR1C|scWO&Qt>AQ$A^Vm zol0&s7p?XCEFVv-9dFFK?dQo~5ufP$taLvYp=}tB+@Jkj)< znjPRO;~(+I`-$~8ZS3@I<73@59`CDiFBfp3#2m#`O2}k!A4cn1Rf?tF5_{-jUt{@x7ZQRSc-liokJfMfRtSaUjS#hQ=D4FfJt{z-hF z;6Q}M(lr>svuD-5hNQM^W9Vt?uRS)7Mc1(VIJ|rmRmEsPd-Ii_ylx6X3o_)J7gy+VOt*H5H8QxUUmj_8uGY6BRngpC+Cy_OvoPSruzZu~LFzZei zjHaG?!1m9D=u`uDyG4npwZ^0A)>>*8vE!{AY>*I+@DqA(1u+2hJ$?SucR@0V%M;?ZxDofz@ z+lNb*sX>y3e_}7L3A?3b{(8)O_2gvS>gd!s+Y5W#Cyhb)ermcv6#@2T4+kgx*JGOf z-!$KODeY$lSy24F`CzNa5bi9Fi*%ZNh}AtHKE@s#*n4)<@f9;pq%wHvuvj?TVmX!^ zV<`QFrCSb9{Jn1R1ed1<9;#Bu!sbJV8t>2Q&iZIma!#{U?mg>$-ha6in!uVma`6yu z<9*n7?q-?=H)ZryFWw>W0eowR*8)k3HM9Mtk&DQFUQj6m|{C zlfYL$jA_!}UDbGKJHj)?3@h%5N+z zlOE6%kDQYATO74ACq+gzhwM6yMY6H%> zWk6#@ziwZ2V%?P1V~|3ce@Nv1ta;On_*>y`Zpz-CGf3{XQVM)uK zQf0R@R4ia$00jX%zF)8y3Nruoy*q7-TnCFZ@~7VQ%Y#ZXR@hmNBcyP!pWk%18) zXnF^ziqTC3iA8HaKXx8Ju*{?$kWh(EF^WOP!p z+ukq9dVD$D9RIwb`3KpkNc@n%$`AXOh1>@C}whY+zWbAA;}CWr|KU%-E0yp z>k2H7mlQ3F|FG3b`&lFK>35M|p&I9i7ol)`2jh7ieGJqD9l%9M|BbGiHv40QrCZ%Z>BbtNmwH4ZuY|1yfq$O)6>}#wB_}T^ zlKO;qeh@O~AuMWPzSG^Iw`+GoUtMqlp#Nfy2Ur6HDNwJ-F#8~y!#2Hi;_Q@#`C&eH zZwFfDooj(=R2D;sh?5I`Zjx!dZ7XsW@ih*01$!c1Pv?Xv=g-vTt+1ovg{T&zP zO)8H-1D)x(o_%+SKF+qr@*6^TtC(mycFouLW%reb&FBMz`K3u21f;Xl4a}`U-iJ4j zVv3oyG`B!OI*9-1IQ0ra&-p*%pJq1<72kF(?i8qGD&fyqR7##a9Nyk*;S5L%^OD`m z2#{P)!Ztou|FSc-Fil@^zi9kH7Pqpno_6yOaOi()n8xSO)ER0ClJm!Q}8UbIxv?^Ll?i&ud}e-5oh_$QNnEz;xelUpLl{G~+232YWYjM^Ft140W_ zsH3(vI*4qAAg5-8Ila^_>m@4Sk-##jt8|{l_~<>loqwv?iVFmJ(v&Bt3vs9Pv7e|o ze@`Vu;cJ$ze#V`&sFLzRAuacj#If$0+Y9 z%?rg!u86nH-gAE-!j2xO{Cb@i#`J5wPYO|5^?S>uTdr{AfarP;J^tRcqVw?5L+?l| z*b2_fw*5Iy{nOX${$*VWP4rMscjJbUZu}6y7eH4&J8t(gFZ&m|J>ke>hPZM`bXm~l~ci+LHwc!w)N>* zwG@{L&%{G=?L}z;EkX}uPK^lNS+ykz@V$~AnHoH9;&{qBmuXmlt^-t}jvO|4%ln|H zWBNP~mPLIM0}LlSZl-^bCq+CRJ>>Pvg7D2{qhz$%qd6!)EQ9(>xzmvNqODme?AttE zC^6@%w*FUHz{tCk!B}dBFMahj1C>><4f8;MlMT?Uj>O)|=~hBO#}BkehT!dGa7rAH z^~68cLrYVCu4qI3^TL6ee@?s^5#5mZE;|0l@NnLDdybVy>JTaM>E}$ppkA?2>9Qyw zD&#b9?<@F*yK(=+t^KKb9-(_`=jR9zvmaPD z5U>vpvjNfoA9-}`&c8DRI!)xS!A(uM{7+})qxk6hLmS~8GL4lFu=>R#*7pF%melaG zguJ6`LkBSpM@fw?m4{dkfdp{heL5OGxv|iEd|xj&67q~v7JJGzH}iDPelI&bU+wdf z{Y(MDopN2=pCfoj)C*a(OL08=&2&Tfr3KA zg@abkFp8Oslof|0Q#w!ToKYEkO(`q0yB{(7e6fjWcgT6t@h*kw4A1>IzAxv-IYJz- zmK*}8d((N8f9G~hy%7`3NSk?8a=-W)+P)=zWEt&zU#o8HA0g#mCJ|G$MIRGR2ZBSt z-5p^)SqeUHs60XRbX+1fh)OnJ?YL5eY6?NuR3tK(c`L>D&b>K;?;d^VoyfQEXnemnc3Q|KEzd+rWUO*tc2RM!M!&T+VQmiq>)tkYx;)O?f7FI0*R?b&NCZkm05fr@ zF%DgYzV1E{K#JX?85yVtO0AwM9>-yhE$>u4d0zf{cZgyxcF%-RQfd%LP*Xv>$;D^& zdC<&zhb*;;&8jPtdi{D~($szPAnIR}Us(zMpqSx5p8_Mo6T zHfpnXbQKYtAi`|kW9374?w$~<-9mVKoZ%l|*DVekPHMgtPNuPM2z?U4bj#jTq6Px@ zuKlQtTLkTnaq5SLF+oy4`P@ z&h0V2A|2mio0f|i7K$v&QMA(02mw7?0Zvh-pA~zARgpZv#2xIK>5ZFvB&ZF3@-}Pb_;@X_w#<{J(xCC&!(WGZzH%IUkE(4pC;tg=%!;oojh>HM!0*$nS z$~$$fQ`s7q$v4t`6o(;9ghVb7vbB!dzE zknM;=)mg;6X986f7+|x}BqNNiKblu}M_f9o4>hG*B6R~}S6n!@FiT$e`U-6(H~#Sh zh0vvdCX(3k+`dO=J0E^5KMHcb^?Vz$qoR&>8Y0_eTpwlU*xv7wmZ|5NAYKSCw z+Yt40IsEzKj)3r;-rnm*RP4NA#ZSU{r~BqlGDZFCDjERIUnyl|*H=wl@iIgc^y3d5 zKB>8_JoM`O_2;febR~t0?q3X`Th?KQ-_eEV&_y-n3To#hKqdt6`1axP;=(T;$w(nm zpXk0<`iVZ8p*DkhYRddi;T#d6?h~B9i{mHg*0cfBXxYNw%T28JiQKsByuRh$*_CbV ze5B&Tb-NCVXIipPe|S!uiABzKA;99wslL%nBoUtW=aO&*^e1|a#CCkS7OiGUG$>2X zm5K}LX+kOsusj5>DbZ%P2r$3;X5$mn2YfnAAk--Hx>irPH?~;lLzrrWxRxi0QE9aZ z3oYHA3HCI9#RuDNF!z8k5XvSq{*O)?R7LM2qzrnf1?QJz8v4YIJ=3bWI2p{mkYW#~SK>OR1RjMA+| z|9>a^Ks&I(JCBPUu&O%sAJLC5ZCVt|NaX24fgNjz-UI>QS1Y;jH|-Ux!Xk%qiiCrE zB#X6i25p&7lZbF}>|mR2^`HhX)6JE6Ky7yy?i9V80^XOIc(Bv{Dr}Fe!BpaM<6ryf zH<1bZSLO8MWE{1|mY=1CKc*6~Fqe%vEsA%)^Yy&iI5{%`v`!{ZtoL_1vM(@|CAVY- z(++b);@SPn=O^DPXGI1}F8`Qe9(XfTA^mrjMD9POyjtX<$%W)9w#9W$azX9L{0kr_ z3pxABm{wEYVV%AvL5Cs3L;%f;)wqkzA@J`0**JeVnTC(iT~}lVpsyW#%!eAtzC0Hd zlJ0#T98JfHW_PS)Fu4=XjR@8nkTsyTbc4U&ZNqnvow^0}$oRsosK(b)Six`XtUBR+ zOf@Xzt$)Ai?__ZLkm6R&s_To;$1e8B$Q^zhnBKggHCi?!K0##se|W0+r0}4NMrc$E z$M$t00y*gLKO7weuI?Moyq-6=)BJ?_@}6}+m`Kn(klu#xa+`L8H57pZlhL5vEc@Br z>}Z3ScnrzJf?PB+hFed(@4p<+OVql_TSX5&H-D${x)AhS9@AM2a=e!=^Q|&LL2u?c zeN1$5!iBc5i~*H2FW#_pk?bLP66j(Rj;wTMC^yzMv*ZG)!TTMeFk>oxCN{q@mY)a zx;x3n*sm?5w)p^G3Z{DfaMf$4{uXT}SDO60=H z7hfUZzpy6Z=za^1t%@9sD`LD{x1Omm^<9yS$PH@$T)}*nSNEoSI?8x{W}F|KljDOI z6G(V|0?3&(kxW;)>Mb^mHjqEsf=25w5F~-cpsDg{Y8m2v^w2w-#6Sb(rZ{gFQ} znW^PyngM^|JRGABj+9;nMXpKtk?jdp#>b@+(aVy-*97b@Et#z4@HpKWGqgJW&XcyH z&ta=vEnM%uJ*9*DOmn6R(0p=uRWeq1Hf@9kheo^Oc+!Hji|#k4grfLys^v5zyP%me zw)UI@7J0XvCL(XTNkv7~uQ9oyAh$I1Rn-$gX4rA5W2He+4A1+NJyOy|Bxby=sDDyn zK%d*=>(lHNt2Vwa%k0qNX{J!EuT@X=Zv|m*4UJ^Zn!Nweo=MP?Mb$Cc3U?%|kX6$6 z>Eg;ZgRV1wRtKN~mj)aO5-PCVB@$=yk6ydTeB z#8c?gPXd2ce71`EE{~LX1v_|FVisHoRj%ZC>a|Kj#V?L@7e|WlXG@=ks|#>`s}jDZ z%G0L5VDyZN>P%wFwEY1_h4C>IFr-HJ83jrGi4>%PV7+HTeIhakfgLIK6G-EPL^(7WhC-TS1LMzZdHsE zc(?q_3PMrhT5L^P(IHjL0m-|~Li1#x3_8cDF-Zpgb3wCx!*{3II4~)rA-DgV?lHIW|xZ84}--KVHA%$Qq_|9Vh{&n_SiZ-k{F6cBm!~ z?}3y&Tg}I&Hx)j4BFrKg7(_lDWoi_pA8%gvsUv{cpTe>Z(s&UyB!B|Ffet*kK2s)7 zdcVU=4SVK9>hqIY#~T`{1-Z|4y@_rQ$2Bk53+PlG`_;P&$dSQ!BV2{y&(gY=@XAKj z74CYMo0l&EWXMHrKnT**-aXwB61;)wxBRR<4iMyE-9%1pO&AJlvi|7S!D&k;6=!qE z&VV7I&rq=GbX`9~Q7C%d)!jPkabg{Al27@WC7=46BSUUUMo6F3X8?*j#QiKiy07ZK zthUO3qy;h$1iOs9;m*P|F=Y{Q$^@$(kly)=L%SBI+-t_+C3^}zT zc#f`CNnwA~J%hC^v(CSnUznxMm7~yq@#L@kAN`%brSq5vFMEB9d%?pOLQ64w9G~&H zTt}af!aqWdn!@*g^5^K5n1M9=ELBMc^$17lRHz^-_lwpb)=M5Wltk^`m~pWf;#Y&8 z13wA|etlng3zer^6AewAC*Y=IXdlUZtY33)vUldm81u_v)GdurqZg64G?vC>4Wpn5 zj0A0*5YC{`eV-%}&sSr&;Z1@V;dkX#AHisCXQ+wQ8xxSB`G;s5`>YY~cX!FoKq7~t zH23J|l)sz9EEUhm2J@cofAHa~vOmd|OO@JDypJ1sV42z zU(M;{TOZ*gkHpbb)7ff>#B$maX)%NFTV5v}8!fm0zdY4<0-GMxmjIW^CEf z>H}bRLN7-J)HG}2FGvV#l9@MOwFA7?^ngRO}ZK*Fd4K4r;ULr4O}<4$h_;D~kI ze57Xnsi@7XqJb@+@FNGQ@vS403`9DuCIZP_kPPelCn=6g@&ihilnXpr_PChTzLC@v zv%8EgVLP_(pwOs3SOdEj{_w^^?JE=yImi*CBjQEO^L`C$bF>Dbm^SJG@RftcMy!6 zS!<3DQ#RO?t+qxb1NxTu`>yuA%V*(G1!l=Nko)nGr;4i+%|q&oV>!6$lOS&t$X-_+ zrCb{BUD>1e=~i4pFFOV%O4hqASlVy$30O${3vryHh)R$>Lb^=uqD!DRU1;i z04bX8MqFEGfd7h|#sIbFlo@LjP%WheplWYbw?LrD;RU!)F~2h(`BHNqP=cQpon~*m z&_mbt3%5XGkcbOAH|^3762dsKP4E?+O24*rfP&SsZMUp+ ztvGM_g0PMR0yd3&?^vF4b?Ot4)1=;a+x%uA?Vx!2WNl0jr7SqC6FqH^b<%117D*EW zX@nxm5z)iG9@@8)MMDR-YbX&_P(N5K(zH6R`ta=dYDj-1)~A8?&$Xbk<87a^2(ZXB z-IfQgxMLV)Xv`|r?WgVNcL*)hE$|q+of#R4=^Q~BS_tWgK{8I{t_)BeNK;E;1P55X z-L`BxI|qT&g}HhHu+7r`rV8G01(#h?WAf%Oajh2txApHVwRb6VHK+?r^&G-@9&wvk z2Cy`Sl@Ag5z!+}z7>&yzKo{u{yI(&8&&$r_1rb+2H+7B&h3A-$S4Yy#J@GFXZUSvX4-* zPg32;zwLK6#NrZzQ$hWY-95io6Yh>Y)<#*?sC5fj^?~UN%S9=hxI=x2TiZ!L-pH_+T9cr`yI{?R>P*-G*Pm-3z zJHumpsh!Ec!%^8U#@>ffuh-IT(T zPf0uqHrcMq_e$0O*NoWKCJx`xfyV90 za&Sf_n&FYtMKls=qE6p4SLk`j z$tW_k#orxwz}Ye5J0^$v_oO58~|Hs!9Cj!2bZTdhB$r#M*Hk{e8 zN*LkU>&Bt!IJSI+14~M!Fx&Y>uN{yiVSTjI?EK&p3-Rgl$3^(`c1r{EJ!BWcaMfI) zuRzg>GaJ!e2-pc-&+V)pLO^WG)|8nj5kRP477px4SJ2ys)I_iowO?iz1LQFl_6j?x z>)QF#$j-|#gs(r3{O}1)-zp_X;9;@A$O!<&1meLOrG)H;p4#rI(Hgg%5-#5sU1+%Y zGkE@w^rXmb=6{>>|K0}wuZ92j=@(Xwxc-173~SO#;@$TZ(8U)Wtba!|*M zBdc(_4Pwf)6eX~EGI>|+eW`|6WR}dZM!vPviL$Ca?bbNEu z?Bj7m%1!e&x6_Vp=~|DWU{0JGez(Rq<{OL zn)gfW^qTqqF~jBq+koa?3OJsD)|D}a{n6ax>xu%EYMb}x5^p&eMHb&RVB9`6y7b2g zc7Kej_6k#6Yiy4J1c&i;)F08%oyVXgPQOWHi!aI%8AiqDIY8cLLNw3hr908NHEzI+ zW^b1qJ1t_>EQ2`iodI2&-c(V`H+sd@-8Z?`c2i#WT#~9F(*o93eK%$m*&@4pjvjjWVd5N8`3bh^@j1(f%Om`=gZ$ z!I1jo1HDH*c`}qs`d3?>#2Uns-h4XR58c{#FYFM-73XI=qK?oTyOCh;l3r)NxwoV_s! zhg-gbzMSb+X8M`3?-)P6f`Rzza5cwq_g#XET}!>aQ{+q2xHJK)_Z1{cx_g}UDmcs6q`#EFu4e#K#8SMRf<;Q1;XtP;tJJE__vWdXGQtWn?s_ndkb=kA5$+()j6Y zW1Cra7;hA;+mdp6@dfWy8HMJt*Y+kI%EDwsmtWt4?c-K!_6BNnY|;m(>px#!=&=xW zw<}x1vhYBV1>+126PwVKBM5ybb^(O3LkV|iSP0qam+_|V<5r>N`A4%TK5<~ZY&t!aisQZisz3HA4k z)jQb~=Hnh=47oZ=sS3$n=;T^>#|7BsAwA|G2Tk7M_VEgLdN0em9Up|+X|Xz7S;%0- z$EUtz7vJnp7gv;mjq4Xu>`cRLGVUgs{--NdY#R&2`+TS-dFg7a|J!~iRo<}_fh`Eiz870_Nq3u z@nXrjMMb!%ri65)L>(C8jn99+z<$}gi0q#hCV#okEJxwJBb^pbTj%X^!>;A3;W@tQ zJ9sDv@+eI|J|kO?(`qi5@*(1p{HSyV;1XADtho6>uD>0QVnhYJDMcOtuzBG@H-DUN zSa&MRvl*cm$yuwoEe`!J1s7EnM5@{1G6YB+bcv~B=yYUj)Y!|0)cFJ8Lbl4w!SRSJ zjXW5Us!MQ9n(VYa1pL#~_quf}(l-a1Ob&L0_1R6-Flu^b9XoKkohxK8*8OTkBbi6$6btf?W> zw4+sFAIw)(dm`cHoICURx2=}LVJ+j@)!jY|A?X2%^~`wV`aSpZPQ-3L5`Fwc=`$nB zB3o*8Wn&Cf2Z&7i^J{H=;I+>&{*OmGOBjlH=b?8$9=DCZ%qO7gFUwuvK1X=h@>`>XnZvOIz}FO$XhkD3M|hxiz=nO ziP|;%ZS5$;a+j-NHd#<4tMlV(BX2+Jj-&4Az1xu|&oB`wvFHd#%1W-c8E(A#Dal+& zeI?%-*sKA{EH#VShkRU?1AS;HRyX7-$UO-53`SVdGQ>Nix#T1Fu01kGyETS#v0F4AGKUlrC*Of_RXozRk%t-&v z)@{a)y(9kHRx2OX6O5*vVIonoP?CsqOVd1WMrZQeR7RMz3~#ZmL{wZX>@s!0nT))C z|8u={VXaXb?Md4Qz8tLZYx&&{-Jg#tKJzNHL29>~3xg#6K0t!Vq?^>?On`e-G#f|a zgCsrgrPN7)vy;cU4x)_*yjvV>-tv4lm@P+6F6wJ&1jtkVIq-?BWN!uCgZwZW-=7dT z|M$<%0Zl*Ed$&@wRip)%QzR%^ZJaeRTL$Z}*M6L_Y&Q#Wlz(eGb~p?g8`g-K#-&3A!aN9Gp> zhNlYPIi#xd*M-;wkw|lU@~i7{ezeii3Lh%V0Xp5Dxxy%i+kD&-w{YE5qzruiSMoy< z+Iqu^LyNw9tL?sQx{5AFFGYOt;~p6#THWQfg)u~XPZY9zI;jw`dzY;>cHtBH;{|U6 z@l@#%nv$bsFouG8%P-is&D*C=Fl|>fg`t31!>dl-H{NxtsaZ%{x4Az?sPZl(nnKVt z+U3OfM%H+0de0*%Ufv@UeX&eUW7mLlYIho2)ynK+&%_WpWbK`L_`I?nK*^j9zrY@- zlg*oFJ;kkzLr0&GCY9D0+`qJ@%sr$O5z9~IERe%Jpj42oV^cGImOrY zEl;e-#|V@VeeSF`kRS`x>Q^li$sHiC(6QZn=PA_O9!rzFjarjXO}ejI`Q|Jf(eMX?{Ga*U23+^$Ux<= ze(X9S22k3g6GHphPNNG285wU>aovsf^ZKja(?gz$6H~dPVn-bn$Zm&g^;zV|tm_sG zU&L+6gp;&ZTup1q7XE#ZGX4^4gJ~SQhnBzd?J{b*^nH&zs_;(%AVsV|`ZgJ9(L1@D z$r!4nA*`>aSOY_z6-n z*(=gTQ3nrL3e&VO@cFFP2}cYQ@?ntIom+yo;hJfJI)Zn-(4@Z4~R z_9a^6m3%eqj%Vc**vAc!g^GnhtI1^pRq_@^UeV4NQdg8I9Sx`FlA$?Uo@bmD^T37R zA*n@eWQB236Q?84;2;n>gh_6q&F|&RQ6-Z>NUePa6+fE>_qFLxSx$--ierGji6^{1 z%s9hc_@i-1+waiim~9-g*De1d`7)OTKHa65A0c|{W@kUqi)*b=umn`Bo2RjVdwet@ zT%_>q$aVlt;^g%m=r>E@a&78&{ouv1T(PTJs79mX#Qh8+WgfR|Aj||TlQ__E;)V?ZD>)$Og<9notWh7{g)DC0S!rn^~(TE zmE_mXyp?kLTp#PtP}VyTceAwqs2#gbA=z}=-8U?IvQxB+JrNTo3G`M()C6nVWbK`Inw;9}{^!HBngXcro>L2>ITl5Fg!| zY-)O!kn#Gb*X8DGEYokZv8ArQ^d4FOg96?;ld*A&=lg(SwSvLxBEFLB zB0PKc_|*sW@0FeOl=0UdFNcx#dXx;Q6>TN%w^1myDLG%8k~u=ho;>NEiuzFD2&S9M3rX7r%|hGauGFg+O({{VN%m8HKPa0M6_n34 zhnrEYSJU1(U9yK)`(3pAFx!fDFI(>YDAF1p&jQn^fuH13aFT%G)3q~4 ztAIj^LVR=b>F}Xc+I}osovRp&hj2)QxBM%kaJwxMuh`D_CRdkVsf|qTa8y|CN9y{_ zJ}PC+??o}T5j?EzR)FKyi%(Zz4MZheA;u=ONE1Wuj<$gVTv^4?5-+9MFZt_4_0@fn zCSu@X{IQm$76!-K;RQhrhYaQ2u&b|VkhN&^nTW~W2360CuLRio+gFjrg+~k@$g6c( zaHlm?F`8HbPL(==F~dowNH`|;=}TdTFF#9tNyZO~Vigt%k`5n8)Q=mM^Z*i<3$_FZw2cK@Ui-P;}kj}alJTO?H>Ip|KrN7N?> z{6rrd4-<+ZX;L>#crbCD!0jEWXG~7FdMHw=c}16D!*v0FmOch8gCgcRj{rKvl8O1j zXpsh_neWeIrqxDj$)i-wI>zO+oEeKOHsz#Zwdm7)_q()e0UsclBQo=mEZ}NCLnWa| zVONnRj)VM~a|2SlP>vxZpRzJ@dE}0+ffYqsEj`Ds*3GxlY3gmQ%dE<9Ao3ShC6rMhD}dE4g@m%I~_CJ?74CehfM zcepC2x%A_MYv_%n;Dll7h)kVD@f0Vkg6wEN=ux~QAi}pMxk|+uul#VdOk4}{3%SvGd8hWrvn( zfXk*3yFJt7Ap0k2(txF;asv~l=#FaI2!EBPDED;!XX9@p^OovWV_lXSfHS^YY4^Fy z<2@Q=G&fKaCGC;MJ1~5(4V(T0Gr%Da ze6#8+*aq_UI7?*juYYWvKe{wF*Z+81Cgg|3*i^1a`sYzfHfX01(WAjcP(YsEQJ<1u<@!%{|3Ju30N+)4+#8@O!S3`;hc z{_#?;g{PFJ!S@LHe^K|IVNES<+wi?rEa(OiQBgqY5(EUKBs4`35C|m%3=))dg9T7Qxr!u6ob>3s8C1^bkG&{x8_ z+wxak{Qk)9D6|&|X4`+Z)H=xRp__A`qMS%m1;hmoTrK!=g zx66xeiwqspbT~`k0MH4@pMRTfV}+s?6`edX_$MPi(7466d>yYy`HG}nF%P`)T{Nq= zI&|1No{_s3M$|i;QF;AHc;$PQhtI`WukJnO-Wu89^DCt-;Nwd*T*dG6b99fES zT~WpjrN>qi4$bFn)$>mIs^(>gT1N~CaF_tN3#|Y$dG9+jSKo98=YLzf+C61_aaqle zlY&X@etynO87}mKuvf+xKQHs{fggWU(a;*7gt>pG>1<2I+PA!*89MU{f8_Yj(7cHo zwT#cCLsVy8m#F%sv8T_rf9!!SW(9uhxaT_g4*NLjqu%r0=fN7tm^btl|eD|8ne4tE?%3PyVe8 zFIPXiyN}*``0z{Cj(;EtKkxc_Q_f+-0B-){Qg7qkGoVlDpxfgV;ida0HXtlBt(PaZ zheg!+-+r&wbdUBtjQshrr1oCjgYV<$h(K+ru`~&M7Io);Wn7Q^iOMhFu6@OR z$-nYbpe!*_ztqzrft=w6keb%T8moPL5qFznTaq0Pna;wm8`dJ+ir=Nb&!Beo?-{)Q zEh90oU{Hv?)%=r3(feiEzj*gd?>4y3{aESRUi$8E=aR4b?2=DU;ukJ?XTQQ%y}3sd z&*#^!=?r+K&Ws%UNJZ%d{e4~1yszQ5Z(jb^JR?!$?4Kvp&3#Ll5c)`ta$lC9rIyR& zvyXpvU)mXdah5FKU95k7xOV!h?~0BFy;BBn{FdvxWgUy92Efv(sKh3;{FTsP`xj{? zzvoUpzNm|pcX<$DZlI&^&_PD-+h;xl>ZFQj)-BOwVyEaQ}EnCjDZ|nzy9=u%a(sZ*JNXR$W zasa1Hbv@E`zMkAA+LsmdQ7Z>Ww~axe3mD}54=vL{*DYeEG>CIsK$8x@eN&-A*-$I3 z<+(yek?(TMlULT|56m#(h{MQFl1;@+iN3Qthi2lAbCz+=OV6ZzXNjY2fg}`aZ3}3$ ziENFu%71v+h`H00RT|!FGbA5F)Z4$yon;X-zq2t+u+n=@)&siPlm}g=d@YwPhOO1w zMtiiI6>5Z9)ji00(Vr0Bee`Q9SHtYc0@?WN!B+OK)cv{rtcMnLB7OZJ>+jN9FYAOZ z;-+@CoX2}5-fo9vRcR_^?eVvFU5sf4nJ2iksHY1pAJ>pdbKLg7qvjA~3(rv(THXKX ze{mJ81Zck&uaJe7->yIN0;CVmRbB;oH@Un%@bAf23QvL27tt3rq=J!O8>Q=3V?e@+ zUjf!r?=jFQFtCP7iV?+D^3Kw=(zxfM7Uwe;%|y!irkNjzTQ?a7+|1UjE2}26@IMV5 z_8-DConhr2`RezVN{9X^hZ`u?>G@aNTcqy6lrBtXsgF^fTPUUjx0=W&W9%-i0%y5q zal((kY+A}gy?Bs&rTWRP209-Sv#BmsuCvE=lUU~WkHkem8pQ(e(d7;Qa~t_j5`0e< zeg<3m^-2(3HDaRuAFo03hkA>B?W>9Wt9nXSiq{;%)|lY@Bg+@pX8|r>>4#-?wDUH~ zG5q^}!Z=3RrML$!E1(=ao$o*)AEa5)qnt* z_$%SNUb#ZPS%+`mx0?Wcd{{X6j~*k_`D3B4U?01q_2Szd>AHDG6>~q0RPW-go50#Q zfNCVJ=l`8k8kDp7%r>UWEMrvhJZ{}Z#a!9JYe{J#?<-$%R=)9J_`ub$H8Y`A=AC?g z#v}Vc@4gB7aI0ZGzZj6kpPzx>Xc}@dtDfvWcPS{gmJiQfrWOiCZ3+G`bK1~{18nqA zPIOrU{tIJC-kNGP#s9f`Zcx0&=WplL&uafKVZ?u8QUeAn3rpW;t%rfH^aH^JOszSnRTDmPRt$BEBdN(VuvRR4nAY38{Qs?|N(eE03tmFbI7u!{m+CRN4)Q{R4^`1RwN zi1S_3S~;#0QCB4eKH2~4x#aot?oZv`-=BJSoXgDO+fngjvhzT3y4N<0IJOhI@^!!R z$NVUIxqQHup*f#k<+a#f*r@i-g($HWi8IKKD4Z^s_UxOa8NJh_-ddA|a-+;&n~W6R z>5DfALa{pBlaxAbFFSdcHNSLI8PqOnq&yGSy(5U z$h&&JC@rN?5h@WU`z4wZJwR3T_;MD(`{+NGiu)!L)?>pZu(2-rc!JKlYe%}*Y8UH+ zn)-g(*=d{S&~FC?i?L7~P7DiSsUMaoQ^K-edzW4X>*S%hsH;9>ELc=go)MRWhk%-4 zb3HF*3PenCpnZrvC|vYVX}e8zmxO!o1qmDr6)cPmoSH+! zu_ek0NL?cUwss-w$rI$X@K*=M| z0qROaznBv!4IXGJpE;jd3ugB{@7C(e`<5KoVu8|Yo-vJ5P2e?A5*?GsaE^wB5TGHd ze2)xA0$+utnv{F?xmV>kvOv=_!@_Gi3pjeOiZ<6Y;ba0L6)xEQ+&oHPF&X!Goj(<< zS1|77E(46vE3N>qluZ*Q&~1rv`2=R1JlH}MI@<{Ac>k3~OqsU0L4@6aa`azN8Gk%A z2a`d15sE#OtZi79SH#WtuC?>Y6O;8@QT3sEd&ZYD9X8Fc}%+S$sIaWsDV zAIk)nMu>~ws|KHkzp(IyQN#wKsV>T1lk(t^p>Z?Yg=25>x~n_iZZEB3!y(Au;YHB4 zvDSkTd8(%-GgMk>MdtUvxDzO%L8!zrf|FinxT=H470Ttc&X?;fEN67j@dTP=Ivjj` zoz$~yK+s<#zV`NfnQp}zfv^+wLBrgD@^W|1SFKnaC%^9@LojKPOn0Ae!A%BUK|eu< zU2|_?+Y^=)>Sb+6?ym+ml5lvFWE>-3-FD_NIHp}-p2sMtN*ssm-%!c8OaM=-+Xm*k z)kpf-sQOLKsih(#i6-8YyQq=jt%F&|Io`P$3Ml6CuH|-Oi&HcXzvpQvK`oenr75S1 zY4)pDhr9ZX2yRrktv41?}iNddzlc!>2~=ZA(y-%d!7ltiGQpTC3qzn>|dZUab!R z;mePKbyUMNkgdQG(e$gd6~S~Pk;_r6mhuPPiWHh%f4OIOVNL^L7+5otXf(A?O~$(r zkAYY88{JzNFSe^b!+#v-yMo>+c$kgRSMnI_a#v6rF+vrNJogTQwdk*hQHtGBVU9*v zx`d`)@+dD_PSl3PLyj}yqBJwpI13it`n~1TJRvitlDnN(O|UaK#o{4chb^l|kQD-F zhH^&2(6xygJ;2B_q)7K0Z}X*cDNX%3zg4|v_>5l-UU{5SVJ<8ks*X~q>*GA7r57;meSlK zygNfDZ#z<~_B{3cpTfN}k4^XDi=9kl__~E_;+T^b`n6UaLxlSx{h346XLk_^i2Tw3 z?7T&o&7Ir~R|QXP%~6z0nrR5U5%7ts!+gbO3jSlU;ou$o#Xd*fKI9%3oYc6`Y?HC3 zlZTTJC{(p%^EsmY7oOVhT}C6szqJZRCOyThj}^sVS+iO5Gz zgwqZv3?D02kkpV2Z)S!XH6vzkA;qz+>9VBfVlK=iM6<<`+u~zfI)AD`ht5bSJdznv z9!3o(Y%*?9e;fx#!erFq85K*qp}ngYt3Xqcg$+s&3HxaB1)_G9ePTJg9yG0UMKP7E zk7er%HWw&#$~r1^g3v4wSZCYQz*rQ~UQ1WVwZ_86Q#R>?M$QsbvIX{Z+XCYe_)PX| z6f;z))sEoIj0$9ZazwZ(`8G3$@*lbjV{DdZ}B!N z!QERJ=WYo$w4w=@5e=QM(yX_g66O0KguVW3bTi6ak;li5&tcpC~bZ#N&3p9w%j77Lr2fg?uM!%H%D9xUinCDncf!|9px%|s)hBjx6yrxN1tB$Kv1?>C?TA zaxH0GlwM+e`ViqIG1eI~pLX9{4C}kCu<>cQ*WjIy!0;I>y?2@{Y~WNRMfvc2ri2DP z2olgW1jVe>E$I@-V-!fy2YJC<&4Hy`U(>h_W9zOnYDfJ^tEw(hJu-cHv0S7?PK@S0 z>aqxn2!l#^Z8z>GP|K-?P*EyOn9^(Ky%L+UdAEEkjLSnh42=yFdX`dkMwg zW4G9IT!RuW08&Ho1$DYRmnBU@v4};Oorz3Zk3o@aUn}~_9+P1psgqZ!wW3Z`2=Z9A zvC8i{!}nfsW$yy|vU4Z0i#Q$8Zyr*P`HhsDx{MUGYD?yV)osv09(hJs(u?c{63tdn ztM9$(_cm4`EEi{k%#UN#Zc;MtXD}oFYjA1hYKFityyI--N59+?);9Lnx6%<?( zPRLuYrF{F(1%5lkWjSuguqt`Sx2V>DeYI!2rWf z-2k)iM;ohRk16~6Rjjtas z5-;s z!qdR9CQjaBS^?trNB&11=>$de5~Usijh6>IdUUd#f_=u~X~{(jxu?>(Eq^AalkP$X z;s{V|5DQ!0o5A?bJk+GE+IthpYZ7mnqG&0nDx&GX#xNx<`DZ zUjcIV*1VAt@JAm_tUALlQq@E#!jBHhuM9sOBc(*6fkXo}<5yJ&bc44azMJQxAPIuY zV*3*5<%Gyxrixc&MIt96RX`Nw@1o>RjG~1;*2J(N#(r1dAJJ6TNkqOu+0=*zGUY{x{G@H8S6e+AWLXmQf*Xe++Ob5WZ!P7iIu?9}lHawJoJ@3FNW*VYpG?9yDKtNV} zlIHVCXR3hdxx5>gf2@7Zv8eh5OTU0l+GGqtU)z>T8fvk%)q_y|<;`lrQ;39r*Sd~( zf_H|T=9^QsYRT6KQqx>xcNR>$E?E~1(HnsftjgSd9v_5jcmZMccTZL>Mn5MYTJ|Lk zz9L@|Gsw!6Y4mn?U>Qim&3|Jsxav%aavHCJR+wka_H>$wboOIWOvc6v-CS~-?Fyap zY~n+bDmLh~G&|)qH<^~Y7xoLEh95sUD9aSPmijX`CUD|g0WxCHDI^E>tCAC*8qy~e zcz9d^{{E%oxX8_9U(-a+)cZ-lqErG;a9}Iu5M)_x{$WaZzWIbU=L&*@Pax?JgJm1t zFpI+TYD|)8=;kZtF39%RrzA_ClZ0j=Zl{j~*u9CyW;gq!4+G{9-fQNqp%`%VyilI0 zkj*1f<(t;o#8Hy+6J4ni8+U%wz@lTcx6wWf_ykCKI_yp7&BW|K(2|_$*Uj@|F??m^ zacP{X4h2IPNhxh)?n%i@Jb9OJ@DIm!-+u6xoOhHu2T!7eXZBO)i|iGQ0@SSTeeUQl z|E$5O`{jdA`>;@`2=$9+l)&PP=Fd5XQMaU)p_j~s8U=DE ztyA&EpQOcV(R?dSGLjSwAKbQFo9$R+>Pjw&?Isj|dUpTmcMYHeCqFNNK zMi#;t3wQIGy7w6(gESYac9EP1y{E&eU+!|?vPcCc?96gqXVrj<;p`zhs$K}W)z{-z zuM1;fTtmorvr=RqA21fZPWs56&=8HBQkIS{2<5LJ` zZR^<(pT0ZIzpSmU<~Kc4S<}&6%`; zbMW3jo*Itv?cPz}7=W%`N!L{4ny;U9f0U#lu$(T`a)0H^{cWAe53lL!ma6*dzy3aT zj{I>5m$CLccv+X|BNOw$?=x7$PnWxD@()JMQ9z})^kBnpzrEjmAXPuK$(NS-d^sM; z3H%b{-lVRa8h)lC5f|@rw6P$Ph1^kEQI+FqVP^67yGwDa7Yo$=6t8<5K`)&r{C#mS zO~^A7_r}WI`zn!P>GT9Rccg^MSFN!Wf`@Tr)*8W6f`Qd2PrN^KcK$RfBrl5Kpio%_ zi#*Ggft$;#rm7ZcX$_=k5hUoa_v33t2=`lSsfutHu$e0D*Lc=TK^Tc8vN4!iYgD4~ z==<>7`Yzs)9Qf?0hxz5lR%Uu@G^aM*p>&}L9Ock0lR7)62WkO-;0Bvq(Ury3ViO%Ss!?Epw{N+60CwzysQlT)3s`BW<PA zZm!&bi-R|Dm?D%62K-s{fbS;V<6w>ZcX-xcit2?*EyZV+!D$5Fk8hT^9v8b5xmC0h zs`J&=urR^evT|GxLGH_`(5~ZsT$hI_TGUzVg-N&1nYJyP%`t&brQELe^UEf{opL;; z4By?gGMg{JPFHz&czhWzw2g7+vg{$ZNnWpAQ+E#b=6m#G^lz(#{ zbzaVVn-jfr(!+#idc&IZa8H)0i%&$>vJyjMZ_Y|SChLqXN2|j#EiY1b{hEO7rg_@n z2AmT9=ldrsXmsx+`t8{!B1{M0|N71w*fZRmXRb0*ys%f2G<7rihXm>VWLVofWrH+} z>qluns(Muo+R~z6nS_n^V#@_|mTkf{5a1atg!j`I?Xyo{ABp6^PHqR78pM&R&&0>p za2gCj#RkeKZqRFUijvX+W=EB8Uh&t@?OL{!z6ry(KT*}lb5I&BuEx$m%eW*oD3=eR}%;_dsbz()!V8phL@#*Yz8+aKpJUE~9CR{Gt5)SNoe($R3 zEsm~}06O&%PR!Ia7jgC2k=DT;cZ(hE)v!l9<5E8_DjD$2KXrh8UHeuaE_W}J3;Q-q z+Y5hcpRc#H3d4$=uc=}SaW8u8MnR6hl0SG@_(_n%^>`XJ@7}wXosl82-pV*B5lH|2 zCnx{{WR{Kp&aDr%Sr&Vxyna=ZWPrO zdAda(+mTBz$mH-G*=_rV+tmC=jKWOj_DxLd_3LMEo8)r{YP}Qljjw(TEi>n=z9hoG zw0v@Q<4!=g2Lq%^>j4v>Qy=}Oy(C+fvRe$W4YiD$Xf`-Ql=O|H5U?1c(dqxR`2cJ4 z`))aKHdUGcmA4N2x=+_}Al(}bXaOrg)a+w@1y5%k4)j}#3P^?KZ0Hf`)V3z6fSGvA zr6b?)oj1O4=no&hL-m=xm|sdYYV@!{Ax|jEFzA9;CbiGxCx!epNff{A^1=KRM@=1D ze{(ZAUEs-1e2T2X%2TJkxccI?oue;=)3{^ExSS8cKc{Rf3qem7be}&?d@LysU$l)XFd8{+700x;;#b5vp9-zAxy~nm7EEen{sn2HOOQ6Z+$`#A z(OXb;!j+El^Zd>=0oCE=s@haaySPp0V&HcxXEr>=pR&}Y`+&epe~2y2|9fNduam3a zxdZ=__bz_Kyx+&H`fh@JpuIma=|)ob<3#&NNvk13Pq*xmiiX~nH+cD)y0zqAv5X7m zf=EhWhqCD3C)YlFyA@;?Eanjpu4-+R#u3eW^NZQ%jaVwp5smFtg{@D2aES zB)4>BUM$Kj;0GAL{G;X>F=+{S9G9E?2_4r@oN!*+2ubI+66LUTOZfVrc_0U^Cg=mM z`T_nTfl!xZc9oK&<#hDUyL5mzQhq)|4ihLFo`ra{Y{ElAZx2Pzv-7-)HWu$yd@94H z?XwG4U*6qUqdf9h2ZCKM+{xuZPb7UQl*B zNI}Cg%_G_t-Y-22`zl%9v?;Dl?R9^_-H7zc_x}7OsqKfR4R(odwCh?a+|6IGNS*KI5w9?(8-%iPMKMaDa1Lm6Od)$f(XN02Xb77Jl^6q zt~D6-m2=9nUH0-8ozDz2CQU8rUkfxmvg7%Gq&qAO+5JJTDs_+|ATxWB$&(M<{| zF;Il|RRSFLgE$?{81gfcv~Z9EssMT0$>5_49|Si$nhR4`(ca`6uH|eGGlmV1(tJ(X zSripU&n?_{__sxwV8KF}o|cRC2N@~a@N#Zy~p zTg0`z!2WW7JZsPyfI1}j%J%AxkAAJE68a2Uy5h7T zR3_9bkBt!1uBJ4}er}g7+8QquA3xgN`G(JVg10Jqc7iF?gE zKUuMXdvy3UVQ6rb$K|PLQUe&HxwM-~Y1{qz=x1_Szq^Q^=iGY+@Aaf2IrrvG<<+Sd zR!eYC=c4t*c{os^`8a3uZlEwOB@)Ld;9GFx&IqCrO`OZly07;nkl(9$CwwJPcDE_% zbzm}qJ0_Gu08=d5MdQoEepZ;`jiw@L?nc~Up5=%pYb(F@4N%`4kX!!$RV8Li^R*Xn z&W3RCtMCEOs*W)CqojQ%KQYMivqZ^%Q+{5;f5X?km{Pl5eeJ`JHoL7t>yNFgqo=F( z*Ix3h|9PQQVfFU%69?jbEfT|)pQ;?>U7^uy-yVpCwUlaDWFD&$(HcnACsyiZlv_xw zBqOS3tsz`#Zmt2&G-9m-sN}4J0_iE&Ks_j(8+jQ-RgMF9MsLUhNKj?b=C>DC347bk z;47tXzFN##S0x87CY#LnX0#L;X9%vy+Q?BQp@J>j)5)i;l7cHZEuD;1E(6{Vq@x3MafC2yH7ec8sP>R~vSQ1Qtna+#ZJK)YiCj zJt&xvNvKSemB@h1^?=D4R3TOMg& zL#Lvg2pTbZ*Asf^#~#{bPOj8%l?6-zPERX3%*~61B|n8XTcDJ;sJGl(3UYXha;6`B zf82>Sk=gC_G2-yH1DpeW()v-;t`9PgJ)|?Is@6%J5on#S>OM|E!WW0H%|C7A95cte zu{<5=cz3j3yZ7d8g(6SPLgqTeR<{d>tP5g|Q>$~9@Mgk0{XymjGKGi(q zISK>sS0x%Xvs>XXtZLtTBG*6g_!f=O!XA{zHLPsng)SH@bkE9I1W&H!21$eInd9Kf%Xs4s{=CM_gdN&&dc z5-d?~-9nEy0j8mAlghi8+1{CRFnnNVh&&j%W6@b#1TcSU9(}#SP^BqSgKU8UG)Q4y zUd@|6Q=%4>i`6mj2qNL3n>*%Pz}!ii8+Vhe124tEw!l z5h|)j5g*3u{z8*@k7HY;Zu&eUnbSf-QBVR0DX3Wh#WQS~p@?l`1|5Pb?zJnX=U!EV zq;iJE8ff)0R@h!$$!(hsfiYNia``6VfE|BYiN$oECm4BjbK~jeuesC7l=^hOuE`N@ zfUk&k?J&a9vbnb8s={X|R^E1rK|P9;Xr{E+pfrWl9Nn3#E#lbCy+u(AK;Fq!IJ`1G zEv$x;(U8=liUEkY_Xrqx|2FSSPoT-d+1CuEUNrZ&HopI52F zbnceCS|>3;ol!@ukn(6=8s5_xwZbywXx>Z=;$=D4ghM;ve$|ljUIR(MjaI1DZZ@m# z(#@F_yBJA}3ID=o89ZjhXwQIqCb=@b&My$G(mA3!KbPEc(1PhE_Is*eR`JJ_v%IM%~dZzX|KBvrm1K?hv2}@py!|Vi39RaG9f63LYfen=y zx%p8Ergn|ykuqN0XZQpdF;*b4 z3@959pW}6Bzl8q%f9j-&@(I9RYrNk0wLa)>ss09t8RnDGO&fULX}{K?NF0sgA6#AV zAaq92&}n*(7|pXm`nClOBPHwr?6q)C=!#1;ieHaDgRJTBM_VEblDA1zLy~EI~|y5ek==^NY44FCJ16u%XO|$ z!reV0lPTkHt3H+wM&fnM9KnR6C}l|mTNK~Go@Ed*&{;6ns3=-mq%)bUm|g>*D`p|T zghD;3=uWnF=wW$5nRUNJEn2teQ?O-RLjif=9m2}pP*X_|{c;{qW?%&=&gnS~HV(w* zt+}eI{KIrohVv$UW%S_=rqz=zqj4Th?<)ln+d%#jP>!h$z}6xWuR(EqSZrNgF+NEd zcPLZ=vFcFskeQjU%2Q`}yQ&32*twteaC9-^8@aYoP!1tQz&8A42(exu%oxo zd&RlMNOUn6>uhnORxbqS{<2dH@SU9dE{MJ@V|5^kz*x~Coz^0e6z`&m7xT)LmA=W| zOmaaAx@Uu65;Wk=6$Ya$>DM|aC>lztw_E|dyMsqr1kd)vqw4S)0hq8P4j~e7v0FyN z(-mOki~ixl0xhfmn{ddD&uD^~E#WGG?eW+RZ;&8!4 zQ)xbkrKJ@0CWrs2S2tn;gcnL7)y?k(9K9+3Ibz<85)(rvb$!}3wZK4>?~ z|B(`tS&bW96>6jdgp9%|m9d{Zi1z)PogS$OfO+z9c@K+{-{r@#22;z3x!!37+ zU9p7(PM*AQdrhYm$mK@kr&TF{Ms0%vN}w@Rcl#Z8X0BwW@a)+o0@8+lfbiCLzRH9p zEQ#0im7aBPX6tKC_pau*()mi;jMme{?KwmvNjOc49GGs7Pia)s3@S?`yUE}q5xUqg zNo`FB1q`1f>I+%PHQ3Uu2k3+3IM4-xg|fC5GXmipZ{XzI&CnI}h6q9rI~9S1=lS7C zZm^YZu2-w$3yIy%3zPo*Fsv&wHHc>SWtw;O*QS9MkQUP%=^%)aI?p;k4o|+J_`iD) zo)Z0EUC1uu*`wLg&2dH`7%o_Lv80FPw&;@{rkHE8G!KNYAceyF5)O(qpTyGx?a2_& zr6IN<7ez3`w}oi3Hwm&alyI=@vYQHvWhB#WIPmyo=cv>Sc(12Xa5Io~Qza!6qJEfc z+b5Ic%s0f#QOW!O5^|Ev-E=T7>! zV!=8CZIwH+2NhqaZ?#8Cza9j9`8v9+tW-VcCOP{n4nCT8MR6njJYTXC8+DL+OY)HW z_XUP*uMZq@2Py$3v2N{KhwSO(O;jv7IlUq^gHTl2MnLKI8y&s+QmYp^k#tdYd5&XXATHoNn0sX?fItdSX@E%qfq4s+_Rxrx5SKQ1leV;%o z$cK29$KQ|x#h=RDmF&c4*YfPi#~XQ=_aL@&pb+94hgUr@E~JTMIK51cdR{W4^w6h8 zrH^jnq`*qtkZ<|Fxnm~zfm-(lUlswVgQa``pzl+)-42TY7UjzYPM#$dZ}tAFNN+e7 zU3+H!IZ+!L0zuk9K%Ot=o5EqABxP5F39$59-*`(aLtKAI$^da9y-?YLzOfC1~?nVJQ{(iXu{lW2$l%a5k ztJEv}4?jys3w2)7JB<&$JU-72mKE+{-h2481;`D66h{@zHk|eBa|&Z@u^QnsKG66E zYZr<5nkqiC3_|Z6?b%JEM!E)^nRHsMv#-^01kTSm>onq@sNE-c~127Gz zBLS0g4ay?nA7^bN2h0I@9|F3vxl%LDZ9S55SxHeM!-Z!1pw({Nm5)aBK!yDnt_~mT zKI4LS;sf*hz?+vVryspcPs0GpYVr7**2#CUN^%FtrE|?Dg-O6bsT`jtAjtRyh zyB~`%BLtwpD?e0FHurCH%huDa^X2FK`6XLgtEbfsGcZ9AK$V$>n<+^g%ZsgvjJ==s zKbbo#blpFV=4E9U^B7a~jjCA2Og+Qjc|7Xr-4UNFE_!MM*2M!g^JSA-$29= zc0D_EYLUu(CJPr&WBeg^WV78F*SG&x8U1*o#7GF%tUG)nt;0ejC+i!6<1IeZc`!%2 zN!;n(8=Qw_>5MS9pT1P_M~R>0x;--C(Ils;#_}9puJQ*me~2SxkfAYYuK6SMdeQm++6&+TI(EhYOnxL1_mD5Wwec@=xW09> zc>9I7IVChHl3)KFkNmD5IVU8)e}vJ?oW&C4Dr4V-k3KVX@X39j1m_SWy8XKma=WSR zJA)SC+4y^IgMzAKg@qbFFbS#mwc%c*GYIife{8;;&$G7vgirGZe*C7whMymk>Ul}t zGnXHQhQ2r{PJWSaMZg;JCU&)u*ikHz5FysR#NWOt91FlUL~7n2sB#=xdn-?HA-s6 z)h#t%0?}nHuRY_Hp!$?I&)58Y>(6r}cCvngRbiZ~k7=Gq&}IMWD3vjfb{Ub2d4mw0 zlc7U`d#+O!4mOS)vxJXR8hbPrL9%E0-M&qC&AF=|p%8&y{oMD{w^VAS>ZO)%(W80a z87p?yubaBu%5H@uZTcQeUzdA#6j2feBQ_#FC3E2ye~}CtIgv6HKsadK#2ql$u;6TB z!zu2PJuG;KJo3v_{C3>E5UQ?3@>=fwF7HI*#0fjOW;Mg`=6gAed-xgiyc_LlLl@=! z^b>kfld);5=HTf9nGJ~u)7HDWDdtTo%OhSd*RhwRq_a0D#C`(Q?ok|VlgWo#($24T z#>zIh@}Yc$-?mC9+;sceb_S*Xs7>6FCN9cu@)fBtkj8=NPF&(cLDSd9E0}ML7PB>Z z48_M!D|Lv~=hIP1#={(CS>JmlvF>3v^$CzP>7oc{K`o`QN10?m*As~t_A?9?LCS0) zbO9g?jL-!%=ooIxTuq0?E0R;d;j|w9^DP-^@^?fspzp_n@+#l1c0=_1ik5F#gnj#C zAlcbG0W>ordwf!VtOb=daJL9)$(Q%>z+!?eKAcrCeb4U(opO8^T0(xMxrKZuRGlPW zK>oXUl0FZk-@8XePz{8>6+E2Po*N5q{x}3Lj6EUUZryw&6s-ARef5JpBgf8Osk{Bj zuaD}dJ1M%C3X}Ct9c-qTtI{oLp>I02EU!p7{HoDJ;H^~Yj=1$_iBCCF zMNY!)$>bM#j_20V8#6*|4~B2HIYV*dFt3fv>} zDW-u9{GG6}|3J$ugaEtQOz;rp^QvXdtO{_)S&WDI< z176jMRIYZ6(kQ=a##(i?Lt-Ci0fx3cr)>ETelu}F;^V92m^$FV(ltDzb0K{&_(;)Z zW?6G4WaUxaY`45IrQS`+4CF2neaPg82h{D4%7NLzU8;I|9H^a%xtV+xK7}SW(sRdV4H!Mzh8eY8idc-KndiJS1AE* z7@NM`Y?MOSf=#!m63LW#HMfk{4VN2Vk5prqO{*7aceinW5651(`qbv2<1d+$-$NJ} zn$$X}LE9q|$9f?`bJSbsMe8Xt#T}&G$e^IzEH27MMRjTQ-2uz+?GIPI&)V^U1MCO< z4K>#^m^0k(n>bvd6=ek?6wZim@hOi``26@2yrmYCMS|Ys(aL*#@rdBlmg*mV{A+sZ z)yWs~rzJ8RooLt1i^M~mb!D|h<}P=7pWu}<_O@Vy(Em>l&BvZ)0nIs=$g>l9E#oMX zxsTTYpb(AIx*>bH0zBDP7ETAk9W#{c)FHl{OW3Bgyk;*QZDWh_8=#gLe2BuZo0&SZ0qu5=k`l0Eu#s#G;_28>bb_f%zMsbSHZR-y8lo{jY6VA-nX z86R>kgT25WzW!Qde$@7NN|KG|GCG(kc8wl4d^8_hH@EGJZGok)UyS~{Jv{)Gh{dZW z08iR9$~`Hk%wATwT}mO=keDx)Gf3@Rxja9F5EneV*utK%*inPs4*Xq-sS!xBkBZP2 zd=HULJ;4C5x|N|umtPMPG{1-QKO#&3YlSO zJu#O_3RPV4WvC-p?%%C-m~N8N22Pv3d*>3Q9IbZI{N0+6lzXR4*V9$VeD~a)bNzOO z-!E9%oR~U}pz4x)b&oGm^CNbil_ee+|JE&e?Ey*aNVtT!r#IN>Qj<`wy`0z}>*lg9 zm?X~E4PS46d#tal#3kjIsXk9Lq9V*j`;3=`?eS8J<@n}gK_s-%XgV6Zm1r@EaODoO zlG!?AN&g0^l%2XLW(zr4o(gGV@_+d73t6ofUOsbeHPGWrBx%AImYD)tIGtLdgqZc! zh8q5RZBK;=?~l|Fx!qBIv@7~o_yn=^>w{kdChwO%`F#GXM+@Ox`}GJ$imrh+_qrQ# z#koG1xtUlI#P#|Uyh7*#k6w(qgU!b`Q`Bs8L04@O{6=s`;WzVZ$JB1Q4T`%;y)>-r zUjn^6{FxE&_3~=Tn=%RM+jYtHSU*lC`|-AOjdnp+ys&a*)w3i4RMa(Lw2`Ub+R|d@(W~YhFRK`ZK1-VOdkvSD6!Pwzj5gV*iCia%FQqb4U8@WpUw{A5>mE|G zvG@Y3ceeTIpO`@a6qFDUgSs}rO$AQX>&gaQRf`8#KNG4w$Nnz!xMraR8u{L|`J@hY zd_!Y=<1*&7Qc4G==TTMg-Dl)Jrqz4jy8gis@g=CwVu-y}vGUE^sn+ z&;F7tvkYxR4ZbPO^RoPLsUfFP^=T_Hx245T7z6HQ&aW!QzhpgR`)+lO zT1$yP^+*p^A#@VCG^%4`AN@sFI{TRXQuT#GPCnI3dDvaPX9LV`vu_(EKSlA=l(<<5 zF_EL(=wHXde_Fu8Q!UGLR_BpnH{I0Lub(nrO{t~<`aC} zW$h=-e6HpGDSQ(Cs9emEfGP3l=zt``od?S&%!bzjubnf!=g#UeXDtL)+${3$`{a3I zPq@5|Q)-wi@?yxxX*+4uuGLFEkeiJ${L%77HAAJ&I=~+H^n{#`_tsRrJ_xox|L7=~ z{K~p7@u-q9F8Aw=+?5jpEC1;41>KGf^!GfT^Zh=nB(jDBJ-OecaYV`@L(8W9qtIXS zH*t^qe0^>;!;Ri0)g8=AJc<%e)8xQ46gW7e;7Rj{_|M2>( z=J#{g{7%Ba$cg@|YRr1+n^LFV9@~-@&vnxfIQQ$u!l;PaP3STxucFLcIj@jvQLLx2 zX!eN}{QxeNb3K3G3VJTZ=ENGmRZ_Rd=PieZMn*KFTC6fi7F#qd6=-hZIlDy!rJGT z@8%7huNusPp=r^q+C7ov<`3~3^-s3)tqdHe?m23ND|&i5x4OQO-SozNlU>h3io$Q2 zaxY&%+t4}b#iOz%KoGp1xkf`gj0gwmSRVeaO7mmtu*SJZ>(< zQS`|pUd?IICfG>QlKSjP(bT8l{3K2itmme-aNnDQ?|vkyD;Gbfn&%m2~a0qGhWYjGh-=FPq+8)vG7%!?oz#>jcgOtX+DtY|v zm2@dD*_U5LTW$HQdMbZUspWxIEtBfc@_fIsg%|t~p>e!@a>>&u&a--gY$Z|yY8R)~_9P(BA#loMPu>=vTSa^cD zDW?Gjv<51+7B;xnX7J^hA;_@gs{EDYaw;p6(?FEswypH+ojd8zdts)26E;V#5K`@( zI*M6-$60dHGcdl53o3#9+d}F`bFMZ24a(F#OJ5NnzDWf(G#tqnQElbRdFpYJXBou@ z&WZg@tM8etobAi2qGXqyh}dPLwH}v_r-Y-a4$QH~t&{h=P5eb7P~hsf3Rt^#pdn_6GYMdI5&bhVxjqo>`^g#}q{Nj+%{YKV?^6k$bQ# z{p6)F-Ri8?mOhYnjmV;;Dg7cV4zttV;VgpW>^B zg;HjT%qYahIoYGuZKiH9l_mmo2mra*RDC`2P@UC*aoacxc}C$ z4y2%{7-VWxMldcJ=+P!_F_@X%e zM4THisUER2Yn2mVlkqX-=PEtxh$$~v|5z`vvXP1e z7yTO5$Vm(?8qHl2c2S;61Dtou!f;^L67rbxgUCQ`7qHvp7dVLLD&rcDUG~t>E#~1<_n=aRSiC@{y z@UwkAEhJfDF*aNi$?3)v6?@FZ!kXNcJ^IvZaH6SaFI`7YE^Gk}9r$ zXk3oh1^oLRO{5E>)7rp#SC;m8b@uxJg$@Ocp4k{a8}#d=nHuF^UKQlu{;(r)Igr9k zT0ue(4%dHmIT3H@`uwY4f1mu@@NK5wDr<5ByXM}YD~#Avbq&=R9dQHFm2K8F`AoM( zX|hR(<6iu&!mQbrDDBWQ#Q;~iN^+N1Q87uY+xZ2R8(u^7U-n*DZG_H;l8=}4YZx^> zdP^JZxdJZJzbs9BGsWbURZ-!{X^7^MkOh#JTI)+71$M~&Jm{v(%Sda~UyccZWXlSk z5bTCoGOx`M*$CcEG*FP%VV!v9ES8rvlp%`8QJ^1@MFUlfj+zQ=!07t>PsV4FFcJlv0s=6uj(Q`zty)dqwZNFMPX4KW7qYE%2T)@9ZHHdKYW}ztBRcnl9tw`8ce~? zD$(`+#MqBM=N)hBv0`=H@XZ{`*zD6{@M*AhW5GN-*U#r?1hwusk6X2*r+ymoKOOWI zg>i&4O(WajZu?28gHt-8ARvX8v4H*$UQpj`tOEPUorVZ#7li8&kHL@oV5QqVd7Fai zpkj{-l92?@-mJkhsI|2`#%+MlVH^HWF*n@UAO_MoGA_;rdE`BdnMa#xdY^$#D2I8yDjnHiiL_6^;AGoDumk>WiM+cD(+98gOxh6d zZ`x>?|JIj+CUTY-nFVGug)5a>?fG&V^$^vd8;UXxfuOT;p~}qER%kuH3omy1W(}S+ z>_frXfy75atEU>9(hzb>SgU1bLr)aJ{%cJVtqKVvWhVNa4Mnv%2!9Vz^>Qnd8cWY? z9`#y+ZgFDqd25;h9S0={k`ow;p=u$h%p4mFoA6 ze*3OnMn(@6gbY|F)!d7B3H;%c$La1yoi_KSAx0FKj)Ied{we-4OrJ!AOv(q5cX$EI zl5TV55TS$r)OwSam!}qc6N%}qa`XwQU}}a1Z`}>}P0FsYfXImQP+N>z84TLfFbJQd zOSMn%2++iCQ@Yc!=+Dh+sG?6n1Dib3eM{}co6j@sTfc80Md)=(nBDx9Rjs)sOC;_- zfgBjugi$b_KTi6xh<2pQ{gz8g)d(VK>o`;ObYkt(AG=ao?~yFg8dU*3kUFD<$2eYq zmF`R)P5FmAo&BsHgLGiovQ_nmVPjeAXipx@;y>lwde+;>PQw%Wv{T;zBuj#~GlYXk zOUc}$#EvQkHXP%%&){4+leG5c9p=quAiMxq_i^X!%4wzd+ae=l&cVTUcaKU>x7g9@ zR`fs>8i?ux&}mKX^z8HQGEIjn^aSx(ECs`T@VKuUZe2XfzoM3bQ+}eSc&_7UeezB@ zJa=+;OFCCqM^EfMrf=OZ3b_5)_Ite4x15dJM63I#@I-T8trVB2z1oE0lc~|uKks^_ zdg*nP>Tc|T*m?xY&o&#yMqa3P6&y%~-B*&K$IvCTJQ4e5s@PSW4};m=92g6eDlclI z!gVrrXoDEI6Mdk*NI5Zm=Hp~fHKrFnd={?EY*42?r`7Ix+M?e&@7H|(j@4>hCEu~K z)>BETi`m)g%`!V1R2VzEcH~A!FT`-5#eAIz2kPyRUqV6WR?Kc>FI6FHMcm#%d}^0f zhXJCV0U~v`Dl-a}rq>NG1*Gy?p$=@y!)pDy5)b7gUYZlW5~gh?2RWj0Dcy~53p7jW zeQV8z*t}l>GicH?SA|}rS%*RLB05Vk{61QHg4)JI0znEstH$uK#3XCsBhX1V!I_+( z-HgCB9YGea9rgzC1RW1Jv>elbNaq2aC!yuGJhq>ck_GrQq*p*C=_N)`YtTaJg z9ECVV*)}h_DL$0L{H(9|n)!^|n=nZA7d1)=qbEcV{hNYsofauWWBzx!u(YJ%N#+&< zC|T}^c;Z;e!Io5&xRCg&XGGhN!jWj->4yBq-J;*9wixfmC!(45Zz=xX{uFy8WTUl- zuQEFQxmd7N=XTI}PhamN8*za*#AMrEYf$v4zEop7MMC0o!NbnB&int!gmqy2&eHK&HDxYkoUi`@pUjYtxhXNMNG@beeyyq49TK$pz4%v08rD$zo{0l zI$YklrrC7CSq3GU;>v%S9IvIlBJytZ2QI$7Pt-8Xsk!awwk{0v6C*wL435*5M-GkF zkgy*c(<2ggdkofT_!=fh9K98b_LVp9P8J?RQVq7Xo@vweS2o*C_ScgGSafZFiX9>9 zkKOf@@_65VT$edu2}tg4$)^O2=^4ELJ60}J(W5(;lDZ_ChcR22v7!MjV5ob|R_PXe zko6so87ciZ*n#;T{AT>M1sZ;2Sx%_yg$O1G8K7bUH=1PgCkfetjnt%iCYRlzSdzwdY#X@Z-z1LQWn0IZe8WsZOI3eAE0&6 zNc78la_v_XTRM(t{&atdB$8jrF*D2Dyh9Jg;wk4mo%Wb*O^jA`X5ZzO$S^Y~jA zn~KR?X7ezVH?85)RMa5!va$A1Pm_9VppH4&PlY(a!vULj8yA!SD-BlrhC4G*(Q3%6 z-H%VB77@2QIxB<0Ea2lKfnM-&Ns28rLd)hT-o9$B0)nyU@LtM%XKt#;6seX1QaHWx zNKx!@6V3H#V@F%&+RL?Up0^pgYwr|I5SvPxYl}H3t>s1F*wO#^W z2ZGthF~QE!C^_)i6wi>CW~d;wO@`=}1}6okL!8c6LUe&yJt|-_#iDT%nY1yYFCU4) zri5t_|Fey10+@`!!G=?$hsGv@8sjWh2Uj8l~UiS~O1e6a%5}`+yG>*ukXS&TdfaOd&KEsIMB#DRnGw-XwexZU#TQ?-< zbX4B?nGG^W*A6O?-K~saOY04YV(Q6=q+t)mPOo}4_jHp8!I}Es0-^u^A7tgp{fIUg zwNWZ@W|V&kSN?Q!VqYRS^u7nX~+m?XA$n%4S!;@+rF(!k)gQ=yHqz{-hF;V2Io0%6!ajBR3r>0aL zDQ(L8Yh^tVuzW`GgA#T2wF;2gM+I`VQbtgo$yZl@9j<8(+5fn7T?W`WqkV`$lLJ(!6zW{6cMk&u3%bz)G|Rz5og3=!2lEb4c+IoqN>a?=wmLD5TT5(E?4(+{aop<9_mH7m0{q$ zK^U#wzNJ@>(eH$B=$m2daNFj^#WODBHo)qb6D>|>$M8inSvNftqdq#dZZKRQTkY6~ z+}qQ}ug&iOpu&(l8ReVa$(6kfGk51`eE4 ziAam`q1Y`wymq_>wK>Lzc(rj>4(|Kzta#wwVIrX(CT|_tD_nJ93_And7FWP zGs=?2l=?jtN5U!UESRMKWhpJY2hn2gaC1qsv5QdgL5|A41~@miZ#Fqa}9U>9MsQ)l7s)knn?-d1sx*Zo${4nHS|P!O^FG?G!NXr zE=+${Dz(s(*$_>DHIqY(t>KM)B?l$o({gr6W8Rfw2DRQ^>gUMp+1)7ZCujQt+$V9W zm;dDn@GfVA8}&Yg%vwrxq`z1BvOStTko|j8<#V2LCSQkcWv$UZxnXJ7|FZz10RP^i z%)F4@ju}CnV$)v!jGSu5jPk~|CK$WbQ2G_R3na<)YV{U!q1Bo0%ICa*u`KoWay_&; zS=MWBY8SY<{4;P%(S6ITy~Jr)DvjN{<^xU_<6fqQ%w#NTvG8&zQqyA?m%ev@wso0I z9(fTe70O`h_s7c*4N_C=Z}SbuaKa|1oL4Aj0f9w08p>E3f&S6g_$ zB2P<^P}Qn1w}0{JQNw{ira)>S0cmO2wuUMNW2*gQQ?|-yEuL}O!@v~v5|J8S=N>eM zL)n0FdUG&?Lld;)ac{IDsV}W^so61{W^+tsf)nQ|k?szAn=*UOmzI_gkgP7CN5Zm9 zZ5?6TJgwU?;F+=5YQl$^*ljee#VMlmIxwXwD_h!rV0`|K>S?vgh|a(^kOxF+yHH@^ z&pg`#$x?*6ox+Od&p|5gyvmU}2w{@)-DE^OtL_?2DX#R2p`O=?Mj3E~6GAq^;l-O1 zJmBLPxs>w1cg^+7nE1>I@#yeWAdWNG&jM|$E`R4`PM9RCQR8h=BHpb1)xK)stneHWZL*r7Vu`%>&xX4Q$uSXKj@ZE6))=Cr``VP=jgvfTjnl|bR^3@T0TSx0x%Zjvvk~{@eww+B2}9}^ z)djdfN-~__roYjSl6Q!A5*U?#(;AakQyYhaSc03&c3+hoaX>{C>>vP82)p0v-jsx*~NKPg}N zJ5R43hAAW?U_8#m8#eH1j7+torR|^>pu>H{c@n2>N=j|?+=j1F1D?8TkMpVNJ*6Vv2Mw$1Pr^)KEWF#PFaSlYDypG zc?e8pTC2;=n@a#&=d%hC#ekVOGF|0NOnmB}ECi!s;4PP9DNpLpY%I^coy1f$RfocT z4Yl_h6I}OeTzfX@%g4`_3y85&ITEuWmJ|>^>nlLMLw!I^Y$IG$E&NSZHN|4!*>K2h zf5k95h!*~H#nbhWoPCFq*$U*HP`56Z zgIb)zJP(}om{Q>7R;r_~ztxaRQE%bgO5AOpzP5oseV z>9T)sMe|5zPA~l5qi9QZy$inIt-ar?&3FfdnqX#kv~LGMw~gSdk{H{EhuVjW&g*-# zoRcYC>!tmUrF~9oYk{_IwH4&UJ+kv~9(st`;Lgos+WS@7qn{GXfqf?Tr@VRoi?=*=QBSh_kf+tkmSM>(aN!?eqLyF0g=Bb`s}u8OI2 zhzdE~TajSlVmV%2`ELW&-`*viWyA^H*W=B5yfX3*qTM_f@@*D=`aWR_EyKRVp-wD^ zaTtu_#JC&6U|AVuzlFW(W2mQ7?1)GpOj3C$!!|&VcJ+SxQkx;hj?}su<2i`H(^r*n z^WOE`0arECnzXhW-3f!V6ZBt32J(8sc_MAknc={FUISU$6+@{0!HUdV~ufW%|f4lR@0yN zDwLv}FcLL0c+)-Kest2$#6aQOAjc>(dw zPvUnZ?4K@0M*I+!CKnaW{*}$>N%gx);`ODDnaR2KdvSQZs^ znQrXQ6`32PIY{C?lfGkk6UDq3xZQ~-s=_3 zTtmpM*JMwTV-fBs+ZB#L2Lt4y363ZS#Z!rrD^rNO&fXO zKxAOl#f5VL_@)&+VRNH+tC{GZw)CirV$5ds-x+!!lRP1F@$G)S^zYr-haT-u8uE7p ztmy}D>K8H0bs&RWP-$t$73^#b#u4V6G9e$~L+#t|k*HbBeO%52U9{mN54R&B|9Gkq zv^RCM0p~wxt8>1PH7c$UuDai#Hnpvh!HXAJz!<&UN9BuE9sE7<97Dd^POiET=FH|tr8;g; z)NDC4jdcODo|S15sp%0?6-o5Hvy&pj`GNGMwg*Px__@_U?0@O?i8W4yOX4TV`cd7) zA87=@g5~x1OVYJ_6334vHENF$7XnefC2g!2o9Rci$B$-ALpr!dxA(FNz;)thppo0s z5H!o--m}WYzzv3CzS^nchs@yJ%=0xxd1ZtoDphhe^WIO_>Ow$Kdi1sS<}Gmf0w{Uq zA*r8vKWM^w%y?D=0$Wnr*A4ir;a#_CZzrm=R5y#&9`*eAqzw6mx3f_`u{BO!vsdAN zbYL6-9lDDfxhA~(trs}8=deuta7;{GE692=ynH*%{IK77zfT6fmMPIQCi5i#hoYS| zu6QSbT(5)Ig@UD*+tbOOiq`uRuFQ21sD0=pym%~m{8&O#@*m!IxO8-AkBd1AXw36S z-mgodb;gQUW=DcAZ?#s0d~WAE?~H)>!K;+Ps{x%?0oCQEZs>8YyYX>k7>gtd)|OXM zh2Gn&afZQ`Y=Wk3?l23A$GY3l4oUpTRRq3S*ggt0sHEKRe|8Y?%DEzd$@I3@)EMWS z^vVdjX>=ZQbx|8~*N}g~ad)IxT;sSRVrnp4>ceOyRylZpWaulF~(?v;%y`iieRMCs^dj!XiXfJ*T>1r zg)WNzz>3z47UJj&8kS|@pcG_P6PD!Rgv6og{ibQv2k_R8VEB<}$Qn!V+D!1_47v?G z*cM%oGf|6a-!jsJ>2*x8?F$3C+nV;Qk1@R-E;sf{;i;aGKEC{}tIp^jV$c24fZ%<83H^Q@OxgMkGH>$SfXl|}DvJ+ytq(rD~ zdqkd1Fr(YH>4Wa#&2JR*&JC);$Kfoj(9H~^xSvhJ6vARv$xLyT?CDP3@GzSb_NuB>shh z8t{(_I|i^#@L@=X4$#ZCh3l4^L%Rh$wHlRVDhbtLFtGbQl8(`tRn{PE(+3PHX4H9Miq8`5nlxC9IY^n9xzIK3zifUSn9IT6)2LL2IIgL>Kij zDI!L8Gy@-Ky6UgA*T0DG+_H)j1oW!s zps!-;cT=s`@(3OyPl6?E2wP`X@nU=$V(c+c%;#L-OYLEv8-%`D@a3Uv2SiDTqv|2T z^g3e7F!)>#ce#AhZL$XXd?$T7*5sTXh!1bxo)cxo4)8txN?WzO;<{%Oyk)Z&Con8r zTBwisr#q^y1;5>Mh_Cm@+Y5Nr=K{G|d^lHsIRB?5>O2^N#x<5(Lro;lOupHoPi^X* zb_7tRfrF*D8|wE(-WMk6|A>4BWElUyk!{10u&%G(i$W+%!>*|2Nl%}B;mZu_{+_FS zNQQ)VPuyxqGwu9=R}H&y4gF zfokX+#SXNG!ewn*5_hqrbZGaguy1#n^KY3wjjfnv@ot(Mk{1y4BF5VI8&3NqM{zY? zF#{GPYlIV9-PNo#QtR!=HW^~olpXxwjIBp>^25cy$VAJ~Spm4^Ty5Ez06lhD8e+u; zl3f6c-$-6)OJ4l@)S?9ov?vs0kfmjdgq|Ju$+RevMZ;+??HM#9;lRHXm27N&RnWzT zyg&1PPPah0Q*=*4s0(mE?Q*AQf|gy+ORix*LEUnlMeP$TrRJYAE+R&6h#M~6$ZT#F zI@=$x@yl1vt3xeIpsSb>4Yx``vl(IA2I`mz9+}X`lZ)a1h==|o2}Z0pZnRf!_pP~l z+mTw(T((gouXEKOP{~(ID?j)yUw59o?qJp3JS5x?xXBDbWhMqJtW)oMeUT84o5h%w zZWFs1bilnulZ#e}B_U;LA^&;dv5Z(V5Qp%2$$kJUw-G5+Bc2i;LppCDK{iT}Uc#C8 zmo5*}flmA9KWT&Tzl<L7|P~#)Z@?NdYZMs)~`z5A>mr!}js9{e)I>`QR zE8kn7jqFLt-AqU|xHforCU7|KGiUPEO`Bx_O5>!n`%s$dB;mV=VroD{tW#ij5%Fa(*%-giIKOkQI zcutSOC=hs7WNh|ETJl`FEpLnj|Di$hrr~`p=>-wW3m|Es z&p4u|502kP&FtG8Zk=BISilPMJI`3gAuGkTh|RKVN3&ZkP}@ZR%m|9d$f(`?chDFY z7k2ycbP*XiBjT^hjn4tc-C25}FtcqA5Tl2=MeBFU@Dm)Zrg~Hl*?!NZ;G(p?w{Q_P zPAvOCw)Nc1cn!bIv}Hh??vrw`h)sbH|8-1Bz_@>CjSIH@9Q8B?)fAB#+{tUJuvUjSX*!kHG(9iT-*7UqSvqNS1xza>q4f_MWr`gnI z)-~nWD8mm~1kLlV@ zJ4U$`r}1%ZQqiVtsjc+vHhK|l<(i!5LA!5t*`9)|xKc2x*)TWj@`6t`L)(Ff?OPc5 zR&kZR7FDJ}t5KUnh4vJ>^RHOm{V@T*;?>dWAQ#q)R+H=~*Q)(|?)!kr8nv1@I5j`= z;OTi(E1k366)KfKGOMO&FAiR%f9<0?^}@LXA}q*~JYMyoH0elw!GFIx*tF7Buhk{_ zU4eRfCe!j{+2Yy+@jAT`O6@+7MfOsjxZbJ#-a%8}ik zyvMYed~eM~YY}bBi7~0E;S4Z5pC?5 zY)(eJsQG=S(w=r#!_(nYo5XHY3ox?8-uPMTR;k%w{sMopriKpBzhJck)1_dE;_Po{t#qOtMGKJC9FXYG1r?aA*_ z&D7~XUMl~V?2#s;XO{Z$`7IC00{gO{xw&}{L)iV}o91bk>dG0RBay9}b|hrlClFla z5`5L(=Vf~JzK?GS1sB=MvhdEYlRW8)lw3unu^tI~w=AjVtNs?WJ=dOVDX>@Mg&WL4 z4;x!7n^D@F{Be2?5Q-kiFABrXlY0{?7ps$K&nlqZjbQ0|%*d^gXq*@k&W-f~1H zyG0sk+qJmvRStWs<^PtxBx8$9y!)GIr2=05=x8nJc1rwoUS&b>J5t88ZgJxm?lmDz zSfGU+m#7()zS?}Mq<~T9q24|^W@P;GyE6vYjlL; zV@c^7IlL@Wn&5EZbC$E`H4coZeNi>&4EWW+mn1HSl-h`Eil3X`Cn*YRHt!h+Le5<7 zRgc+#8 zn(k_!y%U3ATl7j-Ow(>fT(A5uncUE%>VHWhpgk|2s;~2()n)RU z#&zS;a9CK*JUeM=oA6o(1a|u5tDhHiW#G*RzA*IOw7EijGW>GcI@@JG);fJeJ=BtM zLxYlC##(^5vp=~z`371dDhdcT%n$Vbc4%GoYqcJ7*3k_% zCOE%xS4<1^p^N1ZhCqUgf7~AZH;f$LV%afY2{6MV;1uQiE}N%2p#OqX#fMnhXW_Xy z^`7|9O806UNIS$2LX{tINY$L4;}Jc#!Ps0pu(;E}d-ks^aT|$ucq|ujVLnCwLT$Zi z%JE!SF2g7pj4lTc+A7FB@?-WBjafl(J?o}D;;p?w-gR{Lclb1rrl+Uz(9d5NjWuJqaz&Gdg1djQg^Pp8%%5bqiC*YKmlZ0nU7`0a;tA z;yhW+ipys{d1m#1aqC?qM^^+cCF6p=88<>}TEllFU@0{iH*3Kh-d4)G`r*aTj<)Kx zu)1~DDHAcs)UQ&c%y_cli>sP2S?6AeXCKAML8;gXDSZNqJ>FvOG>HpieI(B9Ia9}u za_$mm(C)wdkPE5&?OeInT_sE`{a{P>JV|3?*Ed;h>sXzP*XxT-lZp0s0QXJfTb@agu_6Y8807?6H%WeLXYCMo=9CthRS9;}Ur)ay(KSf(mHbtm zW4%6a%JzM$loz3AMt$v4IEWo_xSb6RSbUnXAqbQ*s6ubtlD|sAh-bXV%{-J_u#6H5 zVeui6xb2oS&UKpk6BRvQeEZvNK{|I9*-lM&6#Ma;-PQ9d&V zj^omhybfgB1L7_SuAJ!Rx4FYe!|J(*I|8n7dsy7;p4l9D+`mPbEf+uAIEL$gE5-Y$ z+UoF!H%5g-RVMDQZ#lzw9qaeT^0yXw)I7FM6|K20FMm0|_ULAqno# zXU+b!(^#2u92kTEpX~^xmcK%1?1gT0t1+c=J{Fa{{wf(I){Q=Ax+ko&$&Y{O(443F zS+K>!_!ua&Z(e@6mbD0QYShC;_1Ff_AC9B>n&qtZ=;JFFOlT^V8^tQ~sC-#Oy6L9E zU%An1TW*Pq>`;ekhkSfKb0R*1C_I!s3~ZLB#!;onj^@lCKVwz4Q}!CuNu9mjQfW`t zn+^^hMiek_?;`Mx308%&N^e^9JY~LGFg06(N`qM4vlP|$)}JCrHIpnVZMY9k7aj>2 z`S#)ovhZ*mwi>7YQiG}`3{az0)9qG~{xcAF@4tag8PeNp@;PHhz zq%A7=t7JycEwqJWwOh9*f|4|m$ustt^g56vm!JD$tTJlL6&d!LX8ERL`9^j5Ts8O7 zi{$dacL8W7`xrrCtNuFv0v;f2q3N4^I397W24OJhn$-|0^OfO#bDpDrwWcCnh$riL5C~OfEHA;ahdgSxVXl;iKJDG)q75Gk|&8i8O3JpK3 z+i4>U(VBQEG!PSOrf0myi(?ANhlN``)K+#^eIP1=uem8QcjC%Cw+ReyjTuU3ku?qv z#I$VaR5EflTYYNwUoNkPmIo%fwkY0U@(r&AixzR(qep>5&4C2;DzL3=zbiw7qereh z7Ey2=A$%{hpE+$C-wCKczTW!@VA*xUMwn8Ul4y`>qwvnuGe5H@d4i%1tlf=hh-c={r}}H< zpp=wK+A&va6!Z<&;%P%dI}b`5@GIGX_jjiqFscL}s9{a0olZAHy=GQd|E{*ml)4x( ze6Yshr`eDLF*c)|LPIA=VBWw!Xe)Z-v(vQAUjg2Xc zu1pUBoEc~|g(+!dvqa6RLmt9w!?sst-xmEc6#HsWuHb*uW#OZu~ zpcw9hHP=46Gorw;U%$;thr#(%2fzdA%;Jx0?sOq;owGI3Oy`X$%MGM%Mht>;2sQ zg%W`3{~|*@h^Ch`w+YpUw|Wrt1%%cfWxcTu`pRsBuHO{Cb3x)1#d5m}!6f*>6+aS{ zQy=C`ZyerkbTd7Rytk}5J>P?GM*UhuuME||mztO6M}scZ^WkD}yqP50jn*)mfzkHIsKBU003{#xCJrSjdpxsq(1 zm$oUDt8Gw&Nm%M9(qJ39Q2Bf;*XGJ5X^aF!FGMuIWf>`& zC`$ZGFKZSsuv#-_BS9b{lJi=qXFT(>`)C0d-FN-Uu2L=@8X^3&Q@gt+wzWgGUF9k! z{+kpLW2h{cc(CO&V``Hw8K?=mHq;b5_n!YZ%^L*$+6TEdJzJhOKv?zjIrDG}iGva` zp0Z@EcBxMXTN=L(yahN(cb&BShm4l~c*Xh%w~ktg(rMRg&&dtNc;xzBeH}^pQXDtw z(TlLfp0QJZf1#r+a{MT)D`lSJDs^0ehbNxlWv1LhZGPdV&*4wxB8277uq(g;_|tpU z)hZ`;exXDDp$L%1i-xz*=^sp5iI9DAXUN4ZeO6VIy;vL;=gD31XN6#&=u=wsOSfJ4 z>TxhWiDU4-c6O}&8XAxD8{M)0&HOUL?mzxyN6_Ct^s2_vT7$)2$Aafd#xdvsOV;qj z+#SA<@NFm1h7_?4H!F*Ha`8G3Q$`_m&1dX&<)&OZ-*G8T*8@ZompLrHT+Uj-nUFYn zd>pq(Mz|y(iBy`nXmT(KRbwDuqOH{ANh+VW^I4G%y|{YF$SysgU2J(_^KbyZ!*oNY zZO#9?pI$~AP0>AJbGv8zs6m6~v|#=FCD*e45WXoSC4tB)h&o9Iq%0UsX7LQ#GIKsg z6n??fm43qxKSyXdH~q*W`9=#Fg2pI=lGhI8Lha;5GEUn>aWvX=jJWPuZ6DFjGBe4^ z#@8S3);!#TA3T7c$`Z6B_~zfnx$u@eeoBd>zI@#$#4Pl&nJ(^xs7Sn&oz3-<~|3*6{97WzL!F zB!Ou>`%vr<=4fey{65h9tHC~S%N3y#)XK>dM4awsCyw9ZQw7?dbTJip4YD7poqg4S z5gnJERc+A!SrOu5S|v&p7NX8b&5Brt+)AD0Jdt{<_Oq?biRfsdnZ(R7CoKyHW=DHv z`LoqxL`KFrwP~nuxPjXgm2Qd$(t0CcDZHN4m-N(SyD6&tRb=k#FNFut4|y;5`$;oNeBvo3^g0Ih^#eLu-Nzvt0U zeJ%V8r?fTq(TL*AqDEvPoiR+rv;CFt%cDO}YQ*Ths|jo2ba~rt&C|8%eR!Z=BH`W4 z52kCP8-ux&jby=SQh>7QpZ%XVoNEVvHuz4qMgC|;BhZh`dfR|KhcN*{UC{`*6S z75tv_Po8DVvNB>aB0ajNnjq6s891J=*;nw&iKj^=Yt1E9ny2AE*TlD92{<}?$#Jg5 zS@#I2w1U3-o;UerPIy|-AxhWHb4Gf04H0sMQSuZmMH@H*Y$u1uW!MRXjUGPX+tjs< z>`zy(ORm{9KY+H`d`rG+Gt2R)#8vw6zv^ykbfB!~&6}_D1n9-efy(q38{YF-a{jEdfwu3q8lVvvCRqwnW zB_76)RJ^nG;jb)SQ3LR(wy@l3F$bi7kZ|ztUHxNZx8I-*C=XN5@$lCzP-%4@o>pSpKiob^jKv|t_5DVq2|wFwosXj+9N*| zyu9?49e6HI?7AAlQ98D(=#4*fVo<2(dbq1idx6n!t~}+^n%lGl}3>u?euGkOuwpJ~k|%1c60$Dd8I>K*#lC?6lkC2!PR;afLT1|93*35uy`bs&!S0&JQc(Fgj<#(R^*{B1BI=>#dDzj+cIL}CxDt0X_&mneZXof=h04<_J zO{h859dS;tdEf6Wv~kR_IqWTH(qEd#@7BeAPn>8TcbOeJNVxbOk6G`L(K;DM6We4s zwgz-Z^64BXaH5{*gH=|}^p{_G2Z=pw@zN8bN=-qhVVj zxE-Ph$BBjl6xl>q+`4ez1S$Jm%&p=md3uvEiS#HqPycaJ;-RuLA~I#A`6~TP_OS7< zGS=Rco+PfNi+^`>0r~K)UqZ_3 z(v`IDePvH5li&jDx?RRU-DcrTJhI{I`fS94RK-7rR%gj@h*~~d>|;|TjE=%qYN31& zZ^b=U_pLum+?`Jt>#-5hp|-P=lChi(v+z2RAj1)`WP@xAH^cTMQ>>l_aWs0LV?{v+z%0v1XVN)loId}faKt0US$|dA^*fT2#M(+P>T7UfP?lv@vr~g&_W8bkT5HsrIj@{t;ORYeH_VV{Wl5=U9 ztvYv{KNQ#^HqMg830AIgR=h-kIQXVlN6gww8`W=es*%$vHxZS+ zhp&JhQoBQw_g6urUnyl6nYI#>NZ(~9lRj22F;m(BJ-#RonBaZ;TzG|xCf_W9u>W__ z*-M}VKxua1Z`_HdJm3Z-*TX0*C+8a|N7WHV^f)qcY>`*0l=-ooCmoESw3G6lkk)pA z(ro&T?9tjw(=$!5R!00flm@awjUV+g=o#v-=4j^Po5GdagWx!M+Ur2xlz|a^bq{UU zqv=JoDnPOnc-vEoxdk-r~2zkm4g_2lhA*F_Xy zSY~)pc2!(yUr0N^nrU3+YXx6N2^2rxhtu1)B4ZAil7oUfzP5w>P+8DAc}3H{m|A*U zlL<}nnNL`{O<02Fso&UNHKk4-jUyT^UV~AdG9B5=#dF$)$%vmP({|#4gd&dEi}el? zT|7FjLjZ76oexTAJ9GkF%<#9+Nd8v3?Yh`P@?uiEoTI_S!U0}Omb*-6J!=KeI&0;+ zspm4HZWscA1v?8s+_Mc`(osW*&u!K*!sq=7T|o3LuN(WyVL{r`Uxk_IP#yFp{m{!N zlz7HQf_|$9f$GMFt5SxL&;79k(VO2r%_K5_H>4jyGKZFke)>w>XFJ1BG4g|i=ZTuP zv&nN8p)%WIYmO-i2x&cv&b6JF}a$YK}D@;W`P0^La8(+zDsr1?~`{MuBIFRWB8*ERyj6 zBGeJLjD*3ZEo2GxAXm5a))SUh-+QnJHeMBHZ-ho_t1+y8Vg-&GiJvX8Z^K@63B;-Ty9n67)U?=@Bfnna2&QBpHDmOrMU7l2<^50XFYE9 zfv}{T+c-rnr`yW|@QY#m*0U6+ZdX%{Y7;BJ?sbVm_-`-a5vyaN8l>r6Wr%?63@>ii zM8eK`FObYrR2t3a;nc4DB-9nEC{N@xRhP83EBO3A%DTyuWnq&~YBC{KwR5eo6_oNZ zGwmhwz);Xz4z?4_#9CT!Qn}di>z!Or-#6A~gb~d2Jj{#27hfFQQ-pYMd@^FV^V5Iu z_;ikYQ5qsIClqz6W4*nJut@>p z=hsW^m^RjBsjqXpg4foUqeZ~zYk>%FQl4C~by(wX;EiN!Q&2LSId`D<~{f`d9MBY5Y|G9vgLCki(*_~>(4 zUyE?*7rLR1qL}XHCr7=Hl>URsmq6Skc{b(|B!tUa-akS@ta~NV@}ydWHQr66$Mv?Q zl5P}KU@y(-=Z*oHcuTpGb%zUo(yHFh!|D_Q=f=#G*q4d;pSyhJ$UYiGvO38>vxx51 zpqg-_n;?P{+6y>d@9{&mi43sj6Dk+SWE~#>PR7AaXWO`JomC*cn zLHVqiEk5Apk6C650V`mj3AM+3L^N0&&0kbO^MuLFlm2OkcQB5|r_% z=Ju@m_7@*SnH)!MR~i3QLFXOIQ{!K+?X1sJU^`@pY~yB5 zv8<{!!gDs*ND<*0m)ZDoOs1al1Bu)d+o&Uq@=Bve5Q`!^f#MOfMttb1AyeuO4lc{xrrtg2n3^p1vUV&l^m0 zuaRmhk#E9Oa^y|L&ck|Gv%Kj7&~nORmEet8qZrHj&1AASdw7wB&$Th8o^bzxbx`1# z*=gl)9I4NQ;`4)+UB=O!U!J7O@y=A_b?xu?#1@Ki!-;V`o;DP{^H6lJ2GBLce6|*; zPH;|h4!Kvr*8KcDo*vbq+umFyxYt&AY&Q?z6EM42t%U2D5_UTKfEIehnSqw45vMBg z#jiZ#YN@@)Z1$a*ZMoY9vv*QBt&F3=EEgy#n(HWrZ+;aMMsJkO^x3}gakmL%mvd?x zl^n%;3Up-xK^~RvbE%8dzkYcrd)nrudov6#?2cS&7M@JcZcbmM)FKegBHK{0JOgch zN;aRI*-NRupuT#VwikJR>tdS-vwk~O6yy>acy4cc0eMmVqy9G9$632{zCsz|z53xp zCw5NL$`+Yv95Y-z2Q~FD(tL`fL8A1GG3)rTm`!Z-IRzjuaZXXYwmf8VUoiyCXKkqb zKTMrvSd?GX?o~i3Y3Yy_fuXx3l@>(0yPFx1Qo2LRff%DFm%HV9Ye?& z-*evUI$!4FT>E+Uvv#a?-@gT7eAVteVVdrEivDxTlZlrp9sc?Yi8OE+{2|1zg!S#t zXNksm&+N#xtLfTf_~&iF#yxeA6{L}8N>cpXCqtbS+kNdc-G+Nbci17&+x$v&BSK=byXj%@GG(ImJjK)O_mS~RBOlV)SJk4KZ4LUDy-jSxREz~0 zr%LJb(xw#4-I~+FF-~C?H>6l$4!V>Z=|t=FSa)kyfC>%mwiZ$~#}@R#`Y;;Vr_u4U zsexXpP=ag7?NJJ$gQbsExANjzztj#PIe$Su5tWU!9f@Ybh}O;xXFO#wfW+M>YwH#{ z5f812M^BP}+ckZqJj^8eH%!X0;MBcrF0gL?al9iBUfyB$j~!kD-s~mB;Nt%1AmS7v ztCQ_o1zPyjBXn+6U)!;UXQ*qv+z(*4lVX>O72h$AcGIdyfXkHU^yuDlbn)X^Q>GFt zIz?FuDJ;eAUL@9P3cM-IF3hN0{g-}gAEMk&f4Z@ni8^70ZR7yiy}6@R)bjKxj3 zMBhxn!PF*OI`jGVn|4cM@_{lL{w2D%gqc|iVdTAuS1C!eFsf10!DE(tGLt(&6&eWsLabnaRCqLxmTqVtvTTTkYQ zCZU2B67jrvim4~WXL)UnkE&IKv7SW(l)j{qK8xCKg0WBW%Hc`WL}VJ68#$}F#sAZ( zNtU)GgnsC06!CaV&Pl(qZnT;$#jh@ci2PLoD_?ncE82e2s{+=31Gv0?-#IOy9R1Yh zxik^9AoB@ZbUHxUw7?TCoJ8v>>4suDb(ip)2jC+E4X4YC5bt39W9+XqP!x5pxAoW{An{zOg+&3z@XlA!=6!u1`@_a=a~191NH!J4&`*g?qt&s!O?_jh`s<2IDQ& z5i2+{$fuI=&r+enT)1D|0HXCCT*6TyOOG!XH0b=iB~;BWVyr;^!5cY!VoDvl|MONO|$v0*_lD|un-AnuQT+R!8r+ZuzKkDa~`LLP` zw2$F8TjACa4<^Wu7HT}nZ5%WWFW8J#cfD@@rm-REotBr8`?ovvnmA$DGx9K8I$#?| z3&tnU`$SK=`0%Nd6n0``8hFo?`k-ai=`Cwywx}5e|AF`R-kDkZ8Z>UhBl+%qeeT<kY+LV?K}W6IA-%UboUa1xN4-e4KiSH^ z7Z3QJ7C@$)`TmkNji9dN3-F(08r!P`)8S5zywp49tZ1sHpdg(&C>|NBO zRnONx1Do04FIcm$H$Hvsu5jVYAI~YKCK|`;i09Kj?xGs;2a215kA2qTa)0ULOqKaE z_Z$!VWOYkUaX?cCG~_aG#}rHhW@zXsCuDtzuMeqn3$MOMS*Xpux^(Xd)9aQ{rPB)C zn1AW$ZAHID4vJg6hXmM_3B4;uyB=o%q@I90yh{L_0p7#$29FpeyYP@H$B=M?l76cG ze)&Xo$qqpl6&!K}kV;*qas1ibXq`LNmwp2XrVc#M?ljO2S@0r8axP|(ofPp!o-NTj zO$Ld`Yw%oKn2=J%v+5SP6Rh+LRIQm7-fdhH#{30YBZ47RT>q&v0nd^x`6qrE;AZ(or7*>>T`8LiTG0~NaK|4`78%WW=R*$x ze(8gSx~uX2@y9b7{U|eA_>|A00o{rDQ*ScLh;&NZg}R1EKwjub{H2S5Vxrwu~!?w_Swdc8Eq!`Bkd zw5E!g?&O*7UhY3r05zmZkP5k)!cOX?tmoYS9GPLyxUQ&tdRlm3-^-9KnoJ;Q-wEi( zZb2paP%(*YihVPSN~59j&_rsCzWf*xi(c4;R0XQg+iEo`zL6(Qou?l8GBUp05DagS zfSgRp-B0~(?$~Kyj{YkSkSt?uq$h3oiU_NHCnfqksdkohFjC2#k~jO$pEEu@ zhiD80tmw;Q%{{4sgenx+L{@@$UOju2J`c!|80H zUKq=3-}(<2Anwt$_7CdOl{KODbFw{+lEvxNDf2X;)TB0U=ykxa*~1a|vQxeHnMn=& zEi`r)ey!(tZzlVp-Os+Bq3R-M0Wuq{_y>^o-yTXo3-jb{?_6ZP&k4V@Mw2qFUCGqK z)fpO|b<>yy6>Dx@`1=%(=S~D18Nt7v{ivs#f>*x zbB=b}N10T{tOd0!oXfxT*@qyM%Rv(7K0>3a>~|^01U=5S8l&&xxAwv>n-ewlNevuW z4EFiLY2&*JLW>uZJ`*7o)vH^Trk=J(G{yVJ@O5K~2;B13EHbi?eJh^@)d$;bZWD9a zq1b`gCqW)f5Kr+?Nv_})Wf&!8Y0}QxwZfj<$aLdq`9&HD8^6$I_K!+??WL>&#kxoz zGzxjEd)n#QlL@On&pT)DG*k1xi0o%})7q6OlA-+F&gBjJVxOcOjd1!?L`QAi`B*9q z{It0FckdWLLwLuwpv*Hc;(Ihp%hNEbRBsdNCzS^N|7vWb`Eq^Gm$^K6LN5e-Vs-<%cjw=P5)9`Kv24)?1Vulmh;%x~!+N6pEgX^i(%{ zb6raiN~L5qq5uJThM;yrP)C0e?Ku{vB8R)$8WEI z4n8;OYh`1-)>{RgRX6n;30lmT0TxQ($ym*)ep(z}BNmXzxdbRS13q5<9z5e9bl?xk zVqd#uE`us~R@_y&anAy`77{6X`N7?}{KPmDV`*gC zG`j?ZIwc?>n`)uMO0YL@<<=ZExK%z58NOhV;*{iKq4{AM?3PC%L%H#rohkklI#7Xg z8i~RMUHeQv`O9k>(Bq%!1C!H9BOW|i(~TzP;!t<<0<$c(e(ZERE1lCLvEnu&JzV7? zTiA=v>mfpqm|k#X2s>VFb`^3A10WKIOD*3BB^d#NMhKxIK{Qs+;!?qhq{b?M_9A%;oHT$xqt6qS>EEJ@_gDzI{zQr~bYZ zS8&#&vQt)?D<}X2>yBk=AN-f+@S3X`bd+1mzg2?+Mi)_ZkNA1CkYcK!x}}7dRkXM( z9B|tA4SdRx`~TGumatVH);?Ed4-$QgHM3J|_7Z zHyrq*8QVa}WM;lDIMeW^TGKc6KX#HtzUR=8IN*t0mNu-Ry4bxp%{(rw0k?SF?1wsc zk#M1My0^)zb?PBeGUz*zx3JXhcsh`C*c+0gVSA|-|2YDf)SdTT`pSw8cTD(k%hM~i zl&X>GV*`+Z+4>-tv5;JUHIj8$1xP(}{UkM^+s9Z>hYGH4D33vxaj~UoEe7v>Z z5I;+;+>Wh8O-!dhsJ9QGgi@d3l(W`s7~gn`+o+zLBEpk^O=qNOwMnb}u_w9$Ol`{P zr8GY7zIKF}LIVXaOl>xUZM+USH5-u~&e~}H-%xFHg8qg{z#j==>al@Uv1D5Bz){um zT0_dSc$;hil{kRFJJ)YuZe>WA*pQ4fxksh2o&UdfJ=Y}m9 zPkz^dP!K%3kc$iuPQbp1BngV@_y98bEP1aDymsuJAZmKqBh|18S2PIb+Sz>k)Z-KY1%26k7?|djFe{7wP+H zB&>@1;oLQ4eq(~76uPVp>t38dMMvL^vB%al?`-L4QA7aHlq(-2u-+SA2ONG|7Zy~FbddBPsmuTaQkgh& zFSbCb5)?t=tef2cCbxRM$0D|98O!=!*J%COZQ*nyYWuFR@4r^8r0)7wuIbRcdY$Ev zvCzyC6^gPcQZBc$6r1-y5IVpWb#*W0p=i?Q^`|i3!7n7`mCqlBU{IKJqDaFjoV)7f zxb6$ux$QBl4p7?}6!Hr6SE{nH_Hk1?ReVWyj7z)CSv=~cLI3_yB z_qW1oF2=8|%iycjoj8amCFY6IOtd5Q3!@)**;%bsj!2-qqRQrO-kpOM`g2+^c=-&E z+uyIHn|5%pVLH$f*_Id9M zKF8VYoPNIFIvL;Dm%=}AV6qGzgG=WAH`H{*y!SLI1>V|{EU#8qM5@GEe2Tyao-*4g)dx2@?sU>4jHV%SDJ6S2*0hrt5MJ+3 ziN(ojCtJb2W201rRzW4QPys|HbvM1ztJ~k0&{#}-Y)i7?wJysxr&=#t~>&e zh_-jyQAxF|-|jwam!(jWZ-X`alFFqUp(4T`VbytWvwSaq8U2gDCoSKj52xtelRmnY zYn3$HpOx5d;MXyZr3{Yrh3BGe)ay|b`uQTgCV7*TotZ0=AIekpr%!UZiezC z8n$b<@&ZQ<&EoYPp_k5W{#ja;+rs2@WjNqX9G&)dDm(e z{~0eP$ZHLDru@%JdA(JKe=gt;)_03M_ohNFGE0)pi(2;(Z)c?(-L0+cD!TrmLLOaN z^=Pd}x|362AK&#q?I zB4T|^Nxy=JBSo8#<2S#?hX5;i7dT6a0}(-V8!aC_N=72F^Mt$nl&SI80$VEPYC3k7 z7TNpJK>z;Hj1L}-FQwZxK~`f;-)tR!F~u4$H9Z-W`DmW??d(V7Wg`$I;}C5@o#l}? z)*aKw9!@JY$|+D6{R!EHc%*I7btM|S*KCC8Pc;Ws$(rZ1kaRH&!QdzL;@`N-I^apaAeM))iyCsJ?40?niJ`_H&~guU z7MFIY)1c^0%Amu_mwrqcu+SPo+4HH&2_m=0g zMz|P9=^So1K&b+{(9M;%lJVm4;t-Chq#YpNv+fHkb3YosT$fee{H4op%4>&&Kbp8t z*}M&!yYn3DpSI3E&X>YNVOG@=@x9~p(}fvh`l|r%fdiY$KtQ@yjB6$K+@H!32MG$( z!MBih(&av9ijwEm_*|mpGTbv@$ozXPO`JjX@%0Om|>Yq|J z%N5W2{$$Qze@p0(j^qvcq)${x<`H@e|0Dc<-I>0xAZBnrp|HjgEDRu$z?-$_D)c{* zdOZIzWrLEIWr;kqLz(Yq9hUBEd1+J_i8#aB#CRfut6HbTxztJ>1DDL55@#`T28b(A zQmM=lfTBz7^?bMPy)u@c<5)W3PwO-I#mtejcuhO$okOoA&&bN)5KoW@hVXZ_9idYPA36N^&vm~et9M~vhq{}z$7eE&E&?j}bvq4D!Ex=^uSPDXkDhtR?D0Lu(4-^q83%h=kk;khf? zwQ1kpG{h-If175|pmqP_(tdwDv0S=onD;X}qJ}r{6N0u?g463a=NH~NY=ulp3dfg{ zweVD?38se=rX!N+m|k}i&71@FK3PKmg&;nF_YmPrI|lRu~O^%hIL`zAXeJy zZ!msmz9Pf2pDLbMz`&*uGLah0zdw7bh}GdPSx(0m3z&fMidcE zIwQj|6yj5E0HgO7C)ksgL7NyC_>})6h?ye)mq?oag^X=VoP^mwI4LjOx@^T+7lyP` z@y(o5{T-zW6}!RCgAO|Ldms*}6=mlIBzWJm?_ zgx5UNQlh49%xv4tW$CnrETuhu_~%Cubkpr(e+JT$J-q={C<^WzOEVGO2V9bIxb>1p zHLdxe0yiduSjF`HWu;2Sr7PHImQVeHaRr%UKvv#j;Zs?8*Rgq)Z@zGkGhKgSvV1I| z6-G1sx%=}pQn99kI(wg=D%*@vkv)_1U`dYc6>*ns9o1L_<6P^m*{OQ>Wc*~R^@}l{Ji*4xm^sP1O0rHj)~_NgvS05T5jS?1*Z3 zNHF>pgnE!!C5|zx^Z92xt%9iw;)|^26uB2wfYb8f$Y>=-l&5thvIFnY4jH{M!?5K( zd#HT0)1&G;%A*YpE&|GHTYNDj*7C^pFh;NlBH&-_Dy09l(qo;NVE0|MJ}3lHdFv-O z)efGkz+yAO4Qd=8m)ejt>6Xb^06-(K583GbHMfhpEq;25%lUq1GrvF9>)=it;o;C8 zldIfhBD;CtLG{>y5ufsH)Mlp7nDk~He~o&7yi6jGw}+10gv_8+gqW?|uG-bl;*3g)$uWjgsUwYx3s**VSqhh9Wtjzayb@%bKaV+p<& zk4hxeIii44XWJvYW{ae7_oPc6H(Q4T7b4Hbst)6;Ru%?gaxeAh9689dvzKC=>%Sgv zzoKsAiCw`rqhb}EtT%VJ`di$7o^^VRE3DNR+bcsqCN190@gKDPZ% zuz2>;)^oo=R5724Bg4-^v5Mf>?M>my`s@!zT5|<8)xMbJ%cOD-b7)a>0hqV{oqPDi zxQlU^zu`S%Un_|5$n>#ZPIR#7BE3In_rvbQvpR3ea3bQmU#z@eJZJBye7LE^+?pn& z&=XmWS&kN8wJog2jDxfjDx9V=cg-T8#yr%>YA;baQbK=TBns+kHA^2$_g_$ zGBCF74q75E7j{=|sknX1V*1u8r<4t{LIhUqq+33YQU=d`Kel}0)7F^RZt6PqyfJ*p zkw+-%-2102p5^D&EqFkt@w4d6En+Gn<-}SyiW^RjK}p6#W3qOCQz0_k&rJQbb>aQ6Fy zkNZm7SE+(x(%8q*1@=JPV}AIq;Z2+`#Yq#wYdTMo<--nHk&kwue<7fI|R*N7f&NkW>M zN6)W@xrTJHgxu9hW4wG!zp)M?Sq7Ir+3Jh5*Ryrg#>)(Q{9u=?&iqPtmUFz-y~Rt+ z9+NlsUybwVo`^cJL|8R1$rp6TP=vz9dQP^_Qx*KFvrCo`O!gIv(l0 zZ$EAbKFLvS#hc%8&IFj#UsI4i{5wI`{0y!rnZw|Kc4}tl3I^p#PbFs$Y}%iQCCv>IBk1~(iVG%ZF= zW_ql;$aJF^iVGoz1-0-vT)n4P@RUd_!nJuV?2}o4G?t6cvS%+jX$m!ge`r$Ib~O*` z)|MR?8>KH$H8^?yD%y^3QYhfAXD=(of#QX+QG{wK24`V00NII1nZkzvJP$9!+%o$} z6)ROFtzAT)K~{desu^x&#p~HhVXzxBu1+!J)_@7^(c7Zb=(3v5H0vjTmm_KsLnXK? z3A#8v{AudxTe~@TQ(49}ZY_obh&HydBbs1U97ef(KFuwR+1Uq#RG;{Y7jap@SM@^y z@E0CzRo|r`wZKjp`UCno7_aV}9WnT&&DOndUs_5jD;NH$F6k~d{!q;!-6yj6EyFSh znmw4-jayT-OWfowL=X4;{c*JuyBXgALGxQ&uf*WYlYU%M-lo_vy4jeQS@;`mM;G_q z;Z4Y65#Gsze-qfmJkF0lmr|o=zk&RT%*-Z>QH`ew{t!CLVqQjIUN)@71Zhs;^*dQ@ zd3A>5Zw<>cp3@%XBdFA}p(Su&fCVC?KXw-3Ilf9$dCD?V>NWY9;ZM|7PO%m2eyb=c zV*7h1mmqH92npNxlW!)0k;zrBK6i{QIvrZ$U!c#3fAc!V*8_TRLd6;7DiwmIwL)R$sDYE3G&4|} z^~SWG-+lR|4|DK{&sA%Y%NkC!Tqr_HEEW8518U-_Wf|nng#9GTZ+KswAu>8Nn3*Fh zq=HL@3yYjE=GA~{@10J#uPm9K4H3iH3GWuk=oj1^yn3F$WK@jC4xcA2=v#G*DBzoY zw_j;ow~`Hr#Ie%vtA{TW4Sa#ciHh5N4Z#u!{LY0KRM#3Yn`lx#`}&yU{kDb(_yc;a z$zST#v|fnEBZuD)XYsO}*Lu4V=O0T$SK&Wd*N5F%8O#_iVAc?mGoAL}y71Efyk@Q= z3!_iOx-C*2-Z7U-zLdTnAa8c#Ab-;xt^6+-JiL5u?PH4?m5;8CW-bST33Gp zag``})VA3X%sftZtY$N$O2K%2K9P6UGR6dhPZeC-&)s!Ld>*ZbTX>73sy~)DjFNg3 zA=yy{;G+`~D!q0;V}U8-@&MOv~>^S$yfB5i|{@XlQx^=0phk;g=D<5s8gBCt~5$DvKDK!_4Nh(%ZYc+l+Aa7D*y_68Rf4hboAdf{%AUAWvy;vkD!&B`Xvh z3#KJxnaHwKE8+8MmAAM#<$E|iXOE}Y3z@0jo(Pe_-no^V_i zkEK|ir%SZT*sWYWiu1cJh9`=CF&h1eJWKn%?B;kWoVk&u(WF^v?RLEcm+3-LOFZ?` zIp_OpIwF%cxVzX`FK_XG?fu6RuSN-#yR5F3!?h>lnM-{R(rJ_3)N~xNi1LCgVrt;n zuB2mFd(1&^M&z@L>$O>fa)OM@TPJgyf7|`#z?N8^nskp$!OX`ZxZQM&nT94mJ)S9w z@Xcbp84*r8%kq#=C~k@m|J$A*y(!cY14ItL)R--gDA7$RSmu=oi-eICe>w@D`lyiC z-~>E^fCzo`aNk}@03S*0$|WrZe=NW%X`>##@_XmvS-K#}AW-#nO-s}M7G+(h(4Vnl z?lewOwz5+A?x42q63}2jHbSl!PJ-%9uq3`!SB5}BUJFx_wv>PE~`z) z<5dMwvjk(j;Z3DOdr}}Gqt!Bp1P>c)vFt_ z@y-($O^c!TdwwR>w`2^l#BX0@9u#6e@`%Nnv*)^%>=TFTbLER?5bfYwcu=lGVAir3 z!BSnv7H@}t?UX2*&Y{gyf3nq-S%Xr`T0GoLi@&kRVc9*-TvsKN1A6e)w1FWL>*#c< z=Fr^WE-7^WNpv`E^hiI^<=p5&aU;XuQcR+>O&b400?pUeB8IGgeq6ko2zoC#*YR(D zhwaOPkx@Aey3T>vf!_O(#2`qyrQ#i2s-KEIsZjsBp?-}=z04!3QTx7$FA40WQpW-u z`emXdN40elP|*wxa3*i6-iAaD-zB3Dsme@cV`;Z(_aPC*!d^6=ETP{IolOf?s?H1e zyc}}h!sn5?ZW3md|8HWV_%M3o0EavWxZBLgSd9eYZ(@T7eDsg#IA$EP z2BilxLp#L`4d9&{m?1O({#tAjIlC~1fw{Pq0M8su{FKnpfHS%y!%-cp%&}K2iDH66 zqkX|}4*q6Y!m4m2jUtx)0 z1X8`t>C>UK2_%hozAa$!gAEt(8`ce7Oc1X*T;w&08ZK>U+2V~kb|&5qUb3&~=M@U5SPEd7oM?HI z<>2p;7uC*3A6=t2m5~jwpOe$LukkCsWm9@BIf(BK=WL@?UQ^rdT2Yo5cT*wljPE!2 zFod$Q2A7?<{{3}kTcQ5EdTZf1<8RGe16qx0!zdvjKqJ2Z=aC?dFY;;;0BZDM8$Ai1aj-mfefvYU`Wofe_{oFUG!|++kXWyMU=$kD%3o^C+CvuYOQ^<{maQb9* z{Le1KYw|MmkH9InpZb>66;&|7c_mebCY4&R#jjweGXMQR2h2`nM$qDE^4V{}U;fm^ z;?~%GznbR?$=$_vh5?VXE@|86-r0b2Ja5x;?QqOJ27l7Gp>x{N{|In-?|1J|heX57 z(iPaawLQi_?I$vx_Z=decs>T{`#kdDd`SduTz*#(*=sxVP8qtw6+6`@tiQ>JC- z>4UH4Gz3{8uABd=&>%p|{tU}gM5dklr1*a)weiAH2L1~}TF)U=>Lx=uxkhA+bXVkm zdE+xZIGqzqS-65l6K~^LkMmzlk}R&fD}M^iEig};YIOFd%`;E*B|^?zZi1Zp-R$9~ zfi7k5cD^GTfoF9&66(s?3wN`wW3@R80Yit4I%x|oAMHaTj!q>@IFopO?1t4Te`-(z}F21KQ&?Ca@Hx` z6BT2JPa=~ypEOsXr(+F%i=aN}yR*72?74S3M`LZ%$ih}dd$R=iUDAxd!?&JCbtX+n z`+8&eS}g@O>!p5|yVOSgci^0Lf8GsLF|ZC{ehRDI5Up`Tx7O(u`8eL!=TrupFa9wJ z-JWos%3Nu5W`TyXrmdI@F`--C#%zpVSO^zAeCwV6sE`rCUVYL}{ zKqpqr7PzzI?1p(-+DKUCKak2WrVuQ->#PKgy-big83WHrhHQtAqXi>kcDL#aYY}J- z`+u+f$HlB}>RA2-+`a8ml`GA(@yX`RMDN)DH;{B|o*2txrMX@=nDFu?JEX_CpFGf6 zluRUHJ{nu`{dAS>Q&&G>k9aR)Fal8Rx6x-Em#TbSq%~ z$C(Q&AU>w?ZtQK5NGWf713~GOVcfhXhmbqwAoMPN7(7y;3?GLr(3n+8-8Q*BZYIT> zFv}FSzcR)J!T1&^#cAF+4@`UUs(ruH!5fv5r9iOM#@G7B#EH?{ z;`@sx_1g*@l<{f>j(Ik(C*sTj-Qe*Pfkvaphsc5#TG@WydgVzh`JnvrEDTPCWVCRY z-sgJNg;oTVW9(HK1;2jhg0eriBi%d>mW)Ns9~&iQCBt|$hT$r@6yeNd<^w;r(Nda* z1L3648H`%D^8QYR(x!@<{>GAD&RUvM%s%r+-mdgEGS-Uxn%j5|!m10e6Vlr|8oBC^ z6JD;ra8iO+lmaV3i{_&}S!SuTu1cmwF0v=F!tElx4oWA3t}VfKqV32%_Q^(x#=85` zke!9%4-N*em*KWaYo&Afkz6tIzH(s;fS_ZP+~Ib3psQ__a&sGVPYM{}U)NPjXBGA- z*AKhj{tLi2d#VX}>Jij$Fy)%VXT6sIzjhsGX!mi1ZHO)dU9wFa<}cAJ9sqe#VL*_3 z(bV=|ab`QyhCXD>>am$foL~Mc>-eJs^DD`E3K5z>d)b5HVO!A(gpo+Q+qa^(nl#Y8 zHf$}6)0208ON&Gw+RqMkk?bs%XS^8;@IDi(^|Qcqp<9!yyd;q`@P%6&(1E33ZbUSg zD~W#(>$3PWt*c4C4}5FoI%#D=w7qb=k;UC15OobzbWKR2gJ3VZj|#tAEUJO7z$9O5 zteB-k=XyB38SPXK?_jlF$fOy!$;)#xCsT%o`iJQ}u&^NO*kls?4}A7#I|C9eBf5Nt zl4J1NW}Uon3vs{%b7ReDDXUi9wy;lD=!Vqa04YPs4bhO{c8ecP!!H`>VeNiVnZWIGj%7&wj1y+_MOI|_?`-@s@$90}2}$pjv9TSNDUbOO zq4zD$mK&wNH9MD-{?z1}0v&Lcvob>6*?!;fKnc;oW2Ez1>b%?XJ+B9UU)#ff>0)kc zi_K?qQWSd`9XuahwOsoB-9d~HJO;--@z!nS(#?oO`0gLQ{NnlOae#?Xwo!)7k61Qx zMO(Vjm*pgU3AdsT!2{7e*k?Uv5M=On;a|R2)#4o1bPrc^p=aR;;=k=}o1>?N#BCow ztm%7*Vv_&uY8e*Sp`jWZmnJ_Der>I*&@QCOXH`vLKU`g=l0uiOLP3c9Fd$=%fSU#r znJawdL*p~_;o_B4b3ch3BKq7()H*9Kf^2-a%3YUkoAlI!E1+%Uu9(+ zpg)EQQFFvifsD{E$&f03qOWU&!4J2g4@W>epxhd{>=!iEx{pixznU>$D0(<_zcd8@ zNjK%Mi{Z^wj?f)Y2&a#X((-wUm>((7xpsBD3x!k+y$D1RI|Lm+Lokgsx_2=ZV5zbZ z)B2(f_kpJ%<`)>=EATsI>JgW1)C^3O{OBnw+9AxDF1= ztoMNLHz>f%%MdY=iH?}VCidJ*;isXH(CzU%ntBWqU_HZ=x!dEu52k{V_kD_OK*adZ z{q$S>>4r{#`tZgvP$k!xmVFKFpnhGOq)aaNmLxgkrzYP8DCndj@Mr85a9>Pr#+2|y z_5#|Mpu_gp7pB+!rG62|mksUEQtZ~ht|G_@OCSXh9d>Pb{RKVH@(@~Ztls@whWqsi zzgmcLb~u8RHyb&7y*>so`Xg}tUZl&YGvUX(m)bYFJA8R=*ooYwHIqQ@0G7lC1S!bh zrvy>OgS;elT6)!r9HGmrvM+-u;$)+UAJ*g!d+6z{s{K@+mM6wLPk}DA!F~I`fY-L7 zlQ-qF^ce`$HDU}JTYs^SoIcuRsB-(CK`*%vCc*FYU9=7`QBm4fT)uE;w&yAahB_Sh zBI=~lFz`>epDXxo4JAO$ zy?+)0HVv!ANEZ^DDWyUheEMflN6LZwp!1}8;B6d8mSx{p9eBRoVC-XDTG$|S!Rt1< zeWd098$6UYDK&AL_5I&Ts{y`5Cdo)Jn4O+GkJV!IOim5S?>5viMHVwjn^9fNO?Z&h zscyT0_v|iE$sy|f5=+REkzwNafyKiONA%Q97}SbYC=Y-Hy5qR&TLx@#TR6@5BTvLI ze0pon#|e=~026bv5pJ!edkigeF!ZAGZwAZFX%SuxX98S1+Ynz!l3McbiQM1*t%rYk zsE2<%qVhiL;?Vob2ftSHv+Ozk<9x07+VaYQT2LDUsfD9cs3JEtePQ@M8A-VITdG)G zSbQKbk-NQS(tXrqbnIOVr|sQ8V_CYY#NrcjA|x@HkFGbLkMUj!oERWqEr;Tqpp<1Mc(w|^HQ4WDMUjb$`y_UM;L5{(v_ z&$gba@+4?1qD#|C&Mo}Am)jDg^h!ujAoGD^{kSTX~_piY+_mx!2k6w;M>G>0}O5< znng&6VBO*Qd7D_qiOH&`UDjCA6FQip>&)iO%?8Vy>Utf@L>`K?Z;IT*%+z~>-@Z}O z!u5&`iKFzE(vm{a^0HK8f1R*oMJ_{r?lvszSEbR7*lxN8Yc;*~i|t#&cnOGs!*(a!>8mfvHQ)Vy+%uA1Zq> z;Ke>Dm*UKKSBtnF4 zOhPm3L*DkI3J7g$jp47vm|!zw8})_e$an-S^%I7``hQo7oufMj5=OFsE7;2Y7^qFC znD#XvYUJ?xACkR9tHVlsgS%~=o8R)LDAqPTphy`~{?a5->l>&)1k>=LVB;kZ?#(}J z=dbU~oyu>#qR-ww>@v-WWm5+k^J5VCVcsA{700MXc!u*&KV<6H`}s1E$r@uVPBHO9Y}|BIcN5%QSC zoIdL9WQF=`#zJ))i-m3CwoK%7StFm$yjhA;kb+8=W_OILU)CFXD%rU_gQU76;cz}y z+SNga*q;RQ=$?CpZZL(cdRI7G7Dkx@DWg%zuclcgkr?sBSZP#GlMNEmNGNz~cb7We zymofFBk*em#(M~l7Z?5c-EgG7^Tqsou~VOuVPWxtgUE(IbLeYqqoS~e)S$7!^jva` zqe82Ze5_)Z{|k=N!&j1%II9UeG4SCr2|z3*ZjrsL{xp0@=I(__P~pOzQ#7Tb77$d; ztf86G>*6hW)~(ygjt?Uc3})i{>N&+fks*?}5qXz(M^&b1e=5QG;ggm)MV@8XXaRR> zH#X&QOP(o>^;^S&Lq|TNrOJ#I-^{etGwD089j-kd`JwsF_FK?H5+8S~*VX+g>LX*? ziOc=m!;Qs5eCP>Z?L(*B^)sWLf;?d`!%Jo!YxxP!Z{PFp|Je(rlM6@TUxUbvW&OG|dgR5iWp| zL`SuaQlu``4UpR3*b*b@#Ik)1-QWq!yRy0nX?lR&No17 zox&l)k+i2V_@sKwu+0C%+Iz({wRP{p+X5meNSCG*DbkApLRGq(D!q3SsnR=PgCH$L zkeU#PNbf=jy@=8Yp$9`%dI=y>qRfy``K^nrIoBL(t}#b@#`7ldzNl!D zS)fz5qUba6>%G}U++|wyULGZZMuDI_0d3ld&08e;4cBq9M$!F5u7x6v#qV-~Z#*@b zoR-|Al~PT=5^}4M8YIEQHKnnuIZ29jeR)nn>#H^9=>J*!Md0!S0_S(@m;O18LwX-4 zaypV@@P_9*L;T~>==g#O7;2|p54{-F#z#WKr zpTLgrc+yn(m&dpxs#*)cO2J08@yN+Yj{EXgwJ z4dS)@0;6*m4n5W6m;S7j{BD)sC+g>4YECk1cXSenm(jIendZyyq zNFkFoYSd}J2V|WIi8{Ka?R=qnM~xIsDS_~m10uPH7YPG`H&eoX?M9GbW17;FYigsz(l|I z2^LQtLvOuyx160LS)jD-AO0foG-$%+c0&?_t997Y?I1_t4yn-1&+y-lQE*4J7{uhxH`)Jn>_zko$6H=?^W1kT zu1#Xh5<1YghWoV5XsyK20?~otn zSU9t3&8-WulsjEoKOO1Jg6G69Q^PQ9jg1@;Gp!0PhKRS9;(+Tiks$J)^R?Qkwo7ZB zU#4mqf&!(;9cX9q4rY>XE=HNYY*}U&{QpA(R0dnCT}O8a;;+d*yr-H zffU3%l|j1~whNhnALyU$Rgh#5lv_C=764}3<7e*fTNeQ^jXdn8Zd69bK3*pwkj++62| z-|6PC@?YQslklS*X@V z+IrzKbmJk-J#!j&{!T_F=0K759U`3iE+_cBgSKZc( znO!j)(=5hH%1%c*%P_n$H2_5_cy27gdNukjdNtzbzE?#i)FH}sJ5EBb4)$Eu0;?A< zatL?s%6{^f`Zd(fj`jV^F?8%7*Ac3cJ3oeY-paUL&OcsQ)Jmr(^$I)u1qfJzCg`8fh>I$AO7iQo0TYK9dZw7UO^J$2*cZIFV?*_10b+HAY(lL4!wtje_ah*?@x;UpqHCH=rA@7~4B*2rIEN zd3sBktq7PaAod~R<9QSVZ3JoMuoV;` z|AmAKo6S3`nkY(cR6LMrL6&ijQ+nzzlAQ8oDO0HtZ5QcC!Jx2)_Mz}4`C<4hkzeWy z9<{TGK$8*^HZ7}G`9O}KxS*@5kcI12b1Zr__`PwKic~zYy}frO8EMgEHax$w)aC%3%ei2R?Qqxh7Z5`xN*)mi;>2z~bkoN=}VX6?`uQUR#` zES&4cod!&WPw^s;$9LmxT82E2>6Zf!@$j8jn}-Okb^K|)il5vr_1=Zm#0IIOp!QM& z#2xqVr8btFB{zbWx03u#tHe)j7IEHN^^jQ-xW1)+zoE6GH$Vt17+GaeJkLEz6iF4V;V;qJ@_$y_d0?0y&>}kjvuYFxYyBes8?W#x%5?WmiP?70 zCUi<`d(Vx(a~3VN*(9jbp#`3zOk0HezcDJ|X;3_}Dj}H%t=sO_jo#wL(ON?F9ClCP z!>=&CCfP+sy3UgPlIWJ03ELTtTJ50bVTj`#>;cD9rBljczM>TQI1ScmP({=17vU1z4-cb^7fd-hf!G4JhEXT`v&R1Wt@~kk z8V&!)Cz`XK+oj&g{2?=^4!-rD)%H^U6F&^r5&7gnL2dG&DrMyMaNd= zH8^6-iKV;@70;|U+~mR+v>PPiJ$?QE`WZ2y5+~muBM}fyC*mqJfg z9(^G$U>7z0Kr!m)ZPbt3Z7Bf$Px0cFKlrb@QL<)XDn&!5Kj;&Eod~4HA%x1wF1LHV z<7*%3T>sv)daE?@P2qah@1TXByiYpNE_!0AHiv6_+3Ovoh(vOb*OE;Q$g&lmO1NlP zm66;j!@nqw?&q~~3b>+o<{bC!JvuDVSE0k;~rIsrga8_UYNFUwL`#_YY+-T$U{ZJXua zg}!O~e)rW&mmNMH$u5mYL2BCS_59;SGXs@6<+0p|E(r&3)C#P^lfwR~1)qVW6x#(ZKVx>+oW8LhuxkA-2a8AA zw%(DiXjuJYJzc;s6SRA0(h~yzp2<``tUMrnlh=0uuOs(6p&nK=JhrepTI4Hd%992J z%4ydOzYWrhTgw(3>lz5}Z=7pT(SP^kGSjz^w4SUZvNg5a z#b)9kYQ@C}F0`Y~)EoKBKMM!A)~Vb6_qaK!(b#7Fr-G!~EWKnqIzYs*yX`?4iNeKZ z%I4HB#oDaxnV2%v_}^&)zxgoJ*b6>{+4bL5`NxYW)n!rUCBEzNpWUN)`3I#JgUu zxi_vIx)r)l*UogPbprpC8IbIs9yY>b9hlZZPJwUMu5XG|gxHIhmu2>@ejsXe{q_{X zlY~AmA|-C6X7h@2bB^_@w838VuDLLh4@2UC5lfhrORMn(e+|#GC-yzPKh)Iey!~3; zD$9XFd?KUXV1}p`?oYTo~mz#xs zWmgSy<)6Tsy5o2$s}>PEvq*fFGD-OjH(1$)na#MR0oz$O=jwypY~=iMw6^`zduMmC zIg!uj@bK^q`7dkkGGK0-8PZX7HazV$ygAihz02{7?sCol5dTVVc9{8Rd-xTM zR1Y%r40qVBmVZ`+M0l$ZNRs zk=M3P$6&4lDR2IN%G25G7c49R;wKlnEmN|yCR5#R^s)xma7ywiX95nIABFl^E_#4n zmu(6fz&jw1_mFg$bt86Mtcv}D zGF&j$1jpX?&+`tx9{a1@fJ3%(%j{C{p3;`E&W{9z_sVni(^yjrpq#x}^h8B!4|A$$ z#Qa{Qtm2ZvM7;LcEe7G2&BcM=exurhqFO4Ih7`8{eqlgn!ocwB=A)KYy;#E0iX>aY9XvO^e%c}D{y1WOU0K80dc6>ZNtW(Ue0$AVX6oE#$C2|EB?LCgS{fEHYk5U{10~ zmVt7v6&U7iBap}F=>N>g?k5fGIp!tpGd#aCxWM2Vx|@Rr6%trTeS~JMkW7=qWSbED zG-Q7_XWDEuyaDTJ)WubTVjyP`byRR;kdDi8rEeypayXo4lHG7fbh73<@!h;(mxN*p z3hs}yl-x|bs8on~PTeYtJ6l;7X3XZ@ZHmQ65dNc0e;rnixR|}P552vP`Z*F`F z_muIP6Ie+%(%WC<>K(4tzvQmWw5k900Y+2-aXn~dFMDSCxMEMHs5p;Hv#KV6x+9NY zUR@Wj;cyfCM$fq_%_k5paMG886kpjBbJ@SA5UTl91o zIbfM;-glCi6M$Sb?+4?T2N(JsGA|I2&WB(iYS#tX4wc>ZREl)FV8!{=IvqQa!KTAa zUDg3g=i_YW_}-z8m3xhRt?Z#X5f!Va52KAr7M0P+-`^t+6wJ}!ynui=9g;2PLmy$8 zz$JGvsJSk)(P5m zj06FHe>q>PwMy&Dve}rZMdP{B;`d{{%0@aY`Yd3v;UJpbaI~*(=i*y5QvX~c4?I91 zcA^$*zEmy~<77{XIk^w#!ePJca38@L2g(bP2d5*a--)LeE9{a(2NO;3$wn2A(`z>~ zP8e-EI)6ZDF_d*3sSqW_oaY=M4zP0f0CBPB(0Kb^vl@Dcu z*dfZSOQz1;f6{dHj%buhKcn;)l_}r3MD__&LU_=bp$d){1y3kmeF# zn6(6jLaVVG!K-~6Py?2dWof{}FM_>r)koXYb$ILpxAwCP8+Gb{xmd#W$EE_v;Uvx$ zgzB{Q$&jq9s6>F1W?$yfJHOq$Qj&rm)>s0kp6##YB{$mMeQ_ zET#Q6q=N=5xY={v2{Pa^6kM&pP;YINrs6)BIHXFhii;9LN(Bz~nXLFRn;kY$Ho=XgX zx}pvuVs&7z@N3JA4@C)6R0hO`oc2ty{2|#kcEmKv&T#t#XyCE)Qv5v4gA{L@DtarV zeiD=WWp6CsVULh{PbK7Er19NKi1{oU`WGfMfAuz`w$uoNx#PlHVI=_V`tv`ilsN9w z|6C+V-#A^}PQr(JbhF4h(sU!mKGrU`C8qhn;@tV!(aXqT%*x8{iWIKeH^?`8SPm=6 zhi>3gLpnM9g6LH{id;G?Ow-jyavSsPe!Z|X=}k63zxl{~3iFfZF)p5kU@RGrhPmF^ zz>SQgl($F0IJyBWG0KL0odk^L-Tg{Xa5G)?sOF8csZ>>R0+oH*UAZ?RhjW{cTHJLt z5MrgQx8Ls(gS>EOX*PWvQQaLJ$ZcKmsMj57y^3FxU&{=x<~7`Y36SK-v8xztN=3*l zHz!^Y3rSBfe_SzDE_;mlLej|uReb&klJ^~4b&cz7E&@vzWIX|91b-sk9cLqn!e9-f z<_Ea)0-h0?`ncWM7pH2>Xye+XwwME`B16UwbH5q(GUJ1l1}* zDXGtvIB|G4nqKJ8&o|?Ukjs&yjJ$jIfTGo&=v zrmm2Ti=T=&FDV9fpeqfrbUoO2ujVMU`@zh8A=;FT0+y_6EdcRJ0l?vzyf4pkbIXd3T2b8y6E&~GNbNJw5 zb~Gvu!Xaaz_f48@FC*q(vdo?rS_r80RS(MnsC`T&q~ofg*h1*PNY84;qz`!a8B^bX}6leOlk$Opl@qNyw#MqLkxBXR+2)3VsL7vkEnmSV7^t1_Mc*EOIZZV^&?!&|yX*_q96dy;)mK`uNS*Na-;S zi~X<-*+<=KRtGgz$O^HKPqFmbp7d(?kA1MQ>%?q4*@gXEnTSo@VGv>^yva+4?q#mH zvZVeWyJiD%7h!z{imq5}f!6usZ)i@0{^r<=eI2EZ{Ey({v#h@C-`i*`pI zT&Ol+YxG7y=_ku&wfkr@&{`$!d99x5klf*ja6l!o3P`lwcIN|Z#+`MSS#PWJvp+s% zHzV==WGW>X#j&69FG4oDwbpV}b#bb_$)j)2>jvZD{^}Ujw`DrYrfbr_s{yS>aR-@B zurZ(Aq1w*TUyCP`32%&oTQ7bNHD{*w8M0w~^7xI{HsaSqyffv98>cew0-(V|%Ua>_ z`hqLA8o_+dBy;jqqp671XDh`h2Sp^!_iKbE>_r-cIAIYQ=&4caCm@lr8OP&$;oJT@ zlsp&;U2#fXhok!{NkdzR1Yr;M1h+mGFxFZ9+)U)yf!8$X#yFzWK6^kV=z13hLwqsVuSc$dl@V;iTqxmcflDREMv^p5E z#1$UpUKZJ%6}{wtR2++0$+7Q-V6hi)^&M_|KX~_w#_vN_4x>IhX*iB>viIkVYHI^M zUA)}9w*R*5o`^_atB_a??b`yD|5`KuGHaOK$it&F_6t9h5Q1GbMpmF@KIGKXcXX<8 zKUiP2yOxl^ju3D z@L9EGaj0_gD^;cTIpe~ea>dCJGi+G6l((H@!ngc-eKTm!gRmk1DckLL`;Z8dra@f+ zM?j#s<}m!agrkmb@8Q$St0p@*f@tEWjXX$ETnCL04AE~Ex_57$3j0*p*L)Nn0)O~MdOCf1=-{zPT@d_Ht9 zn%+-xp+?`OZ)HMx9p2tQFppMRPnH0%Odk@2`Ys12oof(QOUO8`2trb|+kuC!X|24I zuX%=(nTryfow;?=5icj5Cbx30OFITh%s~T(5ZPa{*rX2B3RZ1zVa?# zp8Zj68UN2K1M5&%_!7&p{AtUMS(aGKK9FG?`_iGRHS^1p^@Ya*vy(o=_4}%AwPsuu z>8X_?f`R+&hJ9B9K03M}!dS49X!rS188JIt&@T^2*(0+M?6l!OvwO2ZE$X%!`@}z~ z_Tx9GlHTb-x;9_zI@j6c>z9$TWJnjH_K@P=;+L8xdA0HFJi8O#hJuTT75!?fqY7@2ixlYcoPEyt+g{crpDKZdd`T{xaU8#EBD8txB-iM_eThj6-~$^%2Pp^PwWg&_CSb~jSo$6y)z-&THX!SvzOr@{fUQ77vK=q^DC zJ+fgEI31qm)(T%{?iGXPH@~LGfpA!ve&c0h zrrD`VdRe1RY5w>vi~unM4(%R(%N}$dw<6h}6}~#Tb3$PBb=rd}RX`tzZY10koCP>MJKfOjtu*?wVirsg5p`*k@)YwMfRaV>Pm6(&KVY43?8VKqVS zl-(S^%z)Z!DyK%M;ZBQWi$0Q8s(PiY2k0fPW-nl|qz%P4;j>_fx`3kPog@?1ZPhl? zOrV?m%W4}@gWc})3Db_p+$C=&Zk$2~@~IWmzBHlioa)Ngx`T&Rb)%Y=5pjqsl~Y3G zu8>}9`hJZL6IaWISWR9C1@ni1yHzhwjB2>H8Sg^gzA-Yj7wa7p@b8yF7pp1^GRA6TR9*iA!fBg`xpm)-y%5jS0;R1cb*c0VngpzW09}xa*kuFs!i2 zI4Yq5WJ{{$b&$v`?8ud$ov$0gauI@bgU*j>yd{}cM>~CGy{Lz%WtCHuin;gn40sogEXj?GIdP)EVtrGI zK*pmoRX(6GAI{P@{@=o*8KcqDRSWi>n`~WMFaL%XoY+uy%)EL3nX|ha*#kY5d`@pp z*t(}fS*5Tw2ET_LSM|Rwes5|k(}#;iX34TwbmXF}QZkJRGxI}A#NvrYmtNI*iYUgi#b%>)^Fk|$ z`JmD2JaRE(QFYs)vh7Fm&e;IA`ux@E?e2u#KE;fMwY|qhjp6pe_elpzh;^WrE;rye zK$#G!%eCykD-47VI1_i-snCR)G0F+-q02MLpV!w@+2SX{ z(|z=`G=oaes!uDu)LQ1Y+ZEg{%9+UNzr3AvDQAN9T-+>VgEz5M0 zI-f%Fru7H^9@dOHfUgge=49FHRA`k=J6MR+#Y;b=hp^pH&_War>*4 zn_QqKcrz~e0WXl#hXixATLu`;uheX8GXQz)EH;T(-+#g=svF z<$;muzJdhqAQ=nw+xnVi0+_=rFN>|u)qCJu>F$4Ea#PXUH>L>*ML?_Fwh& zA19Gf_D09kg8NOjg&`sZpZfk4R|@_Pvd2xaV5F^mzPlyjUs%^|Z#UxI++_j7`I6HC zXK+BoawSG7!$O!ygViXK}ermPiJW9P`x|U=H8XUQu`&{P5{IjL`US(+nrq z%zOl271(GfhVMf^G6M)%Ca~P1huo8e)J}N(87LRRcXaLq`>Z24-J1hH2pl_$X6BRY ztEe<(p_H;L>DOj-OXoOay_+GlTIs0xY|PAK^lP2VaNN)Icl0i~l8ZhLPdU*mYRs=| zc~NdFGzlVFpt}RIiZQhm#6%?Pcnz}RgF3DE18O~~UxV=(?!mZm6VE)0RoPb-Ll(o> zISD_eM1u*;;U~fd`1qHL=kMTwN>@;v^YM1hfT-$Z>7HevLqoB~RmxX{Gd=}6aibkr z{2Nke2i#t_vkzMPA$qwy45{-C)Y9wh zfD;XktWGU!1cetAgG?Oyk?bVXsclu)%v-@p;otEh_Fk4z-`A!>^}|E-y%9YzSCYLC z2SL2cW|0B!Z%IY?XbNZn{!Y3Uu2)g8q-Kk<5N_Oo(!%D2lPL(?N;Qr$dA(@2oTqkm|mUNhHG8O zL>`8HP%q%(?Sfn9(w^t4T;gqVW?KhkosP@DudQ^bwiuumN;3n@GvF*-i$2M!3g-g{ zPLM#YT;wohLokTtb4m2_T;p2#!xCzed#4wuqHMacE-zom46OVB*)>;n;~BK+uXG{A zvg5}~_F4V3Ogza;k@AwQqoOPBr%z?Lc369e+_8%^Tobh&Cw>b->UFah!|&H#uByx@ z^=U_44DTo?D`@!l7o;er3}m}O2ON;V%LsP@1bH=3Nbd^;gYpywQ}p?OFhYfC29Oj| z7`&L9*E)VWC$}=$@-Y#~KEYP;wE~p}K!1V6VjKAs?+tJy8-Rm7s|VePUP?!PI&pvq zUtjC(cW{k<{-v+f7FK>!Z;j$~1k=)TNA-`9r%0>7e7#7<%M})0Wde?9#i59Ssrvw~#x%a@WSNed9+xAF1Kqy~dc_^NP`F`Y%$1UZ@fvWj4ZNNHjf zl?y`HP*InPZ#_9Wm+ywK+V#CnHY9H514j1zc*pN~D&5OiqQdf+@wEUs#V3PTp){E; zg0cP1))lOd64RFOh-FW6Y2K({+KJaj7#Pk_dO9&}$U;RW$_h2B7eW8EXw)ImP2Jj9 zy`ePFG~*)sl630W`#^k$l85@uMBr!Krys5v83@zB1YVs+9-pw2?vFl`ifD4{`IlQ@NHFUgusLyAUnwERtVgq`2Q(aGY zkzKBm3q!jq>Wr@=s!+Rc4hAO^>{eZ5asQme1`99;SG{_cbj9d+AE#rl!A>K-o-(WTz7W@>w_e8eR`aI%#OhQ3zfn+J8@O6=+YgJ zN0#>(Zt*|msnyo=df{y0;=2(hLqRn3{*V;?O^BLxgz_B+x?5{^YGeR0CTDm_3DCrcK1%r@_yhNoCSpUCP1?gCp=R+(i9vttBY*0n zZ~yHDATBSqQ1+0TBj){xqT4%rg?=s-NTfU-UoHuzpaU^DcRW>8f8?d4_%Ve?L{{`L zGg|RU=EtOcoUJjFNH-Od64oyA@`aur%tqZe-zeRgk2cjWgs<(%5XAU1n*Zb|Dipb<%fl;-${!t!DeWfQ?&li{7q6$=jIhc^bVK%3V4+#fI1P zz2W$lN8s0{H=EU~i}*-eF>qjWdc-1WRCP_dgR?3^j5w2p=( z@68aC^)`~*-6W<;Q33tfynz}QbezJt_0X6E&W)hj3Bg%LJ&=qe%t^F_KH_EU zguH*|(OL>=Y^C;=crIIcKkVkn_?tYJSke1WTO${^SL`nw{Pu!+Snn%`S?}tB!)~0SS7?#NKzO`ZzQu!GnB z-(z@leHsuMWdlt{2AsglGoZU}Xg|AR`a1)EY;~ytgNEnsP&%Xawr2_GpB1oj?5)%Q z`j^%|>GvG0@3pbTg2*+S7U>1N=@^2x*nVx-MymoEO_>$-&{;l`1!dPk98wkAO8)PY z1uCrYV{e|nnx8k+-zPPp4kzaPecW*O{<@OZt@fo1T{XY8H z+3m+g{g3zy8AciR+0R25j$RwZ8>RXD;Mp)X$~57!$zj?ud}C-=kq^FPUnDr2E_ zsUJERxp`wp@1%sw`Lsp;zPkQMSxu!`S^M#ScgbHiZ+1TDFc-irOzLg+agx)K)!Z2oJGlRR5?XcQh`L+ny&XLb-WN2$3+ex3Ov4ejqt*PIP-Rz4XjEkE9aXSg9UD) zeJ0MvTCWYBDPH4QSX*Y~-ZHdCaCMt>ja9Eb8=~x~Nr`7cTJy%pocUxloJ}U=Nn6-C zbG4{vc(=FUdpHT5Q#!|)@eGeDd|WF$HqK8Fn}IXEUWX#rx0oOA1=8JlH===MW&Xg4 zlVN;UV3Qf=(RCa7OjlW!JA3(*wtC%R#%)t@JI7U_PtDiUU--Hi3)Tcf%3CSv3*b&G z_oZwPQjX>;z(CcTI1{2^dT9NPmi|Dv~kSq`E^igU?1{Nvvwgj zMFg1yO{xi}wG5fhZ@^1xt86~xffGVo#{l1`b##kM2ti5Z`!8u8iqS?%*bX3^6i$e| zqfHfw>4EOQ?92$KDPIRck`31bus(J_hX*i#nDq?c!tr*>=I+9H2{6Bg-@b)~fe`+X zx$kex58~prz!&V9*y-wM-119MKa*UoyjWG5y|@~~M_P2QQV)NL01Y9UUiL<$|1>r` zPG$5#FyxKU-|87t{XRO*@iIxRgFe-X9;A+S1edH>8NZFiGOp#j5gD@8A^m5&pUb%< zNYRCwM*KcLf+I&ij+i|K?YO;A5n735%vLR_Xv2om1#5cw_0V+{_UMC47ZzijKq3`lIJ-it#hkmRZ zhfy*ke)_6LX6W6SuWnvLKv>b1pGWB>Jrz^#Qv(y@7MtBDnJPEEj}!5^t;D67SV63i zwhB_}CJzi;ALq**J0tYy8|_Mpd$9MC`H3|plVUO20$`6U(9H6(h zz;T$*SOusTO_Fim@7F0iED(_*(Nsau^VU?vG&Ea11pJ)K-1rDAPqt3R{3J1CN<#(v z&x6$E$7=sjCoY>iJISELNh91z@C zlSDwCVmomiNd?X7`aJCO{2THKEtb}zQH#uWL-UtC^=%bGyY={9i60B^_WgVh&fGIE z4))!(s9-wbh0}<@<`pvaHFmwArT+7mZ|{*N-Ku-Wb3+ z4$MPm9{0>tx&khnf7yl8mcD`nzPvDK_+FmfVp;P|bK=9tPg0992uM23ojM5C>i&e) zcIeGyo5yIulg}ZMBjmEeP!L0J?*~aa}Ku1G`^c#f`=diQ#kFsZgSYmgBd+^A6IipD*57x_j zK02Vx)jKP~v|B5J5CtO>ejDEm_oEn+<#O_PiZi%+Z8GW@#hb+RSbPy9$HfZCVdReX z>Z;W{{?f4)0}&Fh_-6A;6~(!%aT6j-xjhjAE6}EY%J2Oa*DB%?xS+sIE4>&d^;Y&F z{eglvvOI3sP}ZvV0h4SNmP?U?b?A3phQdV5^c%&C~(9l+XYu}v;x;PT9cL)wM_>yeGgEWTF{W9@!vJyg!x z=T_;$7`FOiG89HF8aJ#Y2}%UZV5Oy6t7W|5CyS0N{fo1;)0w%#J}Lv4BcE z8UeXSP7YB1Oyv}d`uwzZmXOQd_kq#ogE&Q$xIqS0JCb(gq`L^+T;QWEp}lWFC2j_E zxhQRxVLF?5$~rhQ39$(a+V<^7Q3K3o77I(9O( zc{Y!6_3Ob!x0Vz6^#m52$q|?4QOr`vwxK|~F2Mcdi867qnS;MNk=DX>H&x_Pv$2$s z4y>NR;o@Wfzov9oYV{9TH=}Zq zoYU9MR%^jZa2zPcMD?Qk-Wo63%(yn!@jHMc)89AL&FLcptpEql0*Da3M|WNi;%vsp zAbiu~uG97c$bt%0dx!rf$4d$eL_g3}MN%G;DvdTr-ZWj8OrD;U^n&Q-cT%B=+H-@L z4iKUsPJai%LW!`7_!XnXPp9Rr0vu5ufML$wzYTjSp;R87M%x0YdRkZkCgGCG7U? z>>nmiyIxm~SvtR&k3mi@q^)MVmEx+WQH&Yx^i#}o1NGiNrK{J?2tQ5~OzJj0nJcEJ zf!mE1+Eeed8}(W6G~+Y-W#|xPh3CNunIOJ5^0!scIy%QgaO5~bPEOeIa7T7AW%!J~ zh>{;%0z*UsS>?FF;}RFGy&m{R&8ZbbgfH#%+3?hzvq$!Jq}MYifo9^?gAiWrLaGw0 z>mYaY8!`?i{`u3Y`s5V#J7l|0U=5&ydeIY@7(4B6K0dHnmfU5i_TC?==4lGLjqXig zaN7402s5L4+Vc3Sn_r1GJmKWIG3On(p|9gA$|qMS6Z)RWbIndo*WTvD z_WS6>`KnJzk1|%lYkpLJ2lQe(;5wkd9*^oOu1SeF)%Sc(5`GS2(Q>JR<&=8PR0@Mh zoVG{JDRyBRpnmd~5la{t+(@<5UQe0it{bE^;hrQ46vbUrgq?ML(xG?b7|5?%`cU|l zxA~))IX9ZgDBo01uSBnEbS30+Tk+cbl44Y;-~#=(aD25`QTs*(mEr}HVuXe%FR+1H zUT_V?EWHJ-+7!E4gm4O6VZJX^kj5oouLVCC z4LM;TP--OVr^f`F#*3y?V*06Xm)_$@Qt?ep4|0#iQSYM|6N}-an=$tKcH<0b108Ob zZ5j8)?o{22+%5ga5CHY2=xcAV@XxV$g5JKevdF2h9@<&?!k1ofvkV5Gl1u#5vlAw1 z+AlKDcs=Zek&p0a#$eIr1Dw3OTQ3#aj^U$LN5*9ZZZObRaNQDAra@-vatS@0HWUd0 zd%`T(yu>;$e3ue=T;^IY-9Ycs+><98Dg$)*QX$IN@z;*+nZ>Ysom`(Z)P*r0x?}W^ zI9yZ>+>!(tiYCRq&y|ujK93UzV3l_fYC1!W<2t|joTvK8O6k(%-$Jg z6duZ(_JzxxW&3SS(NB4m!p9^gh}xg`Z@*(V6s+ZKM;A^!XWx5cY|I$%x`1yHuriLd zOpj41Or+ZZ>1kLmEJ|3eyywe?vurA)KI#|pw_X)}@ZO2rg7i@(NJ;l;!ICHCSo!(; znzMill0%|UDy5u;vY+!O_qE%w2P|&ZR^KMQmYSD}eCKRJ846z{@;zeb1Cf{ZGNV8^ zL7!gGSmiXV>k0!%!&G`_x*;DfUCI?rHy`M5kRqxPn+COpevChb85z^&%RtF?Ij2|@ z=T4>c(&5BZQCEMy@^>Ya&4O8b=^p!unf0StmGuAjqVB9h$TIyA^(SrhxS7AcIB=>)&mrU* zoJo7MSPwRUvd)#bGqwWN(cfz^>s_1MbCGv+4Ly)}@H|DhGygu{O!pQZQ2z zxj!CgP&=iSp*TPguV>UwTYhb8CwH#}PNE?5=Bu*PBFnVA{5Mo zaJ(c)6X8!PC#h$(h&olgcKNAX)SGiruFZg(A$<_LQLzejROc}?rVyV*ilhgyb|>?~ zOz6Vr#S-<0Z`vhs_~=V4ZaWfXYZ_lNR59Mi%8^;{VeelQn-82L9-Wl$l?}k z@zV4ldia${+Tu9hNle@?)PQk-_C|0!e|WWC(?@4?EJ`W2S~MAP#5*B~+d|jO%V}OQ zxVjZcR^wih$v>b^UY}{U86;3dWcVG&TM=d<0KY`^(yc3^mA?!15tXiP&l9VbqO@iy z)Y}&cIsmrh4^I6cSSwbMW?)NMe@(nV+x z_XIV~K|wxL8L9Gzz$Kf(@k_`+Y8pi5YwaGfwUKI8Q;p$;*Z$paCa41%uGD`-Sym(K zFUO{yiH(1(9`QFVjneIF$F~Gi)2Gb6)G+&p09nzo0N5UhT~`G;P3i&P5y-MGOxqV| zSTTrXRY68#pzhmDwQSFH)bCxFh@&p}yX)CEx9_5rE?5|G^(&^^#>)9NIXvR-TF2xb z!)*Y0EVcUA|3%b!hqL*s((Xa6GO}G`kscG8;@hT) zA<80Ca^?*Kp^ONWE#M3<6wks{c-H>F+i7%zvmGlWpzBP)DG_xG-LfmB(dOjT!!hJl zn)po?Y4+D8VdKB;jRhu?@rPutkC*+!8fRjY zPK6X1S=%aHQdCvHyN?R_7_Xig0vXh#GXEKCh+k-BvU{gJ%s5zSxw)|8 zP0s?42#0X61l8?fJh=IPr$O(t+nr1z6cU8xIh9lAD@3yjhz?DehO~?`?(VxY&#XZ{)!S$?Zy}DBSUlhh|Ou5rw`DjW&dNyM|Wwk-b!5aY=O$1 zeo~I90`u;__VSkk0MNWkc?X{gJY1gWqh(3{?Q|uCgTH3}AyFV8m0ioa-4-UDZnObN zFn_1FBRs*nG(Q@h$d#}ke%)^6t>BS)N72T8k|orPR&(K09*i-gK;om2JpSp`Y6<>K$HlH_~`(1j0WfT(Qa2 zX9bEfZ6rTg%4v8B@RK0lhSgpskj*72ed?gwcj@8FX(h_|??X}utwJBQYp(P4?2+Qi z?Vm}gtv~oXp5chMnj+MxPveim2u161XoBt+6P(z`ElFYR*J*fchS%#!DA^qgy_GB! zy2m|g3!mNZUu6&JQ=uv*B6)DUmSUwx$K5H>O=}oTWUUYBmo9;H9_MXC|A~L#ML2L# zQ4X)T$IO*GM$$WIs3kdI^*n4E`b6a%86L@d+@~%Vo>u?3bS?Kb&9$MHKT;71t|>M7 z_c58@iwApu^@BPg$F3rofF)1duLnC%+2O96Pc2gfN0LexGA{A1&9*|iRvV>VfkU0w zho?htBcLgeGH0?6YlNiXgmcCZgL~B~c+)@tbjqi!ham)Wz3+GqWGX`19~X2BT*K_U z8-FN^%Ti8k=bPC5Am84i3m{1Nc0XZawSkhQ~8U88L%3&il?Z?Dd&eq^6sR? zVsOcul}dslo48h+BJSivV%R7|SAL9k`d?iC2|whFIrM1=^*$hI^vMZucnHT(6xX{z z(NHXmD06&ZkIs?AVhpol_<%y~YE&c`C()tW;@hn9HJJml*EMe_v*0iMHag!Hwrpnd zB@73PbTw3C4l?HLEBkpO{G$p(B@+oT9mgwa^(cg!{nUkMbD>5n-}ngg#B2`ZN?NSZ z5N>&-B_`Zh#*T#EsNUr$FWi5JNTg$NfT*WsQPKRsL$8*)1X)`!DZpbap=Y-4KXjl=e#&qfKiX(0Q7PQI2-mCV#Iyb zuczc-q!$f~Ynr*Zsh`5@2owPb0KvIv(t?ZuBHLqJEq}@?*#rWgvGKB@hi_`L(RZIh zx8l(D`tEw84_Wb6fuu*F^HmfKU_2F;neiVYa^;Igg)L06VwlA8ZSbX0n?*x!QML6B z5VU87@6j2X$k1(eyI=pEf9mlgHV(j+!Xr1|P>{zPly|#l!)|K!a*Y%4;;OQ@nJlqP zmb55FsDw36r_B3*9uw*x1L@vUW&(P7Cg1;EripmT-h--8ABs^i?m0-c@{!~`NEJH( z6<;Kh$8p@fc4YlLq^6-9^P;!nSvx0pUq#G%@QDwCL)Hcy=#;5d;wjM9ux_}Jw=rCh zozQwwd`3H;&u6Vw;2VxphU&~uAJ%N4%cwum7 zns=B>AoSD~@_v_L;4i#PW+MhQ18s0yGEz4vOIyLn?>JsU0v}~JUj0C`-sm59>fG^} zxb1BNE8EunQ!6BXTuUC_K499v97llUWndsM%K2~&HJ#-*uP0ySV+6~$vW{)niNwGF6E_nf7Vqz&aaOOS?cC~nE{XRdC~VV| znAVafgaqU_@kd(Qg#H|e*+iN_B|Bb(RAc4cKX+^FrQf_&p)O zT#jCRg`R#0e*;P2=-YYr(9k;SOEr<72xaSia@9eGcim%(KUi_m@EiQN)e7L)8?oS^ z^7{Q^pW>xu8Fv*Iu@Wv3YGyAKeK|=ZENizzUi4b4SdbO?ML~FFf*0DwVDz)B@6kCu ztT8izdIz=BivXKde`+$*)(KQ}=(Zv=hn{dG9{Lpjm zrgG2c2Rtd+72Eawqce{iZ z$)0iv5h~tkA&QXie=zGFA)h1MLbCMktBJuQYO`KUzP6J6N)nm+`-Ws}-YlTydv?dg zCaGXPSHh^yvrsP52U$(b1lCEg^rklpypGThYU~zEoS9NNKNZ;Lk_TaRqdGrJc@Sl; zC6W{rq>e#Oykt*EvgV$>qJo-5eb&L2>5I6uUT$g!?&vnqPi3;SR#>*0dnAg98+&zk7u6^OM-gW?p#eV1 z<9~%9&8ymM(Ppy z4vi~gVB)Ftu=O_;uGPnkiUe=~zI}w32`L5wVCgW35&)Za3hVToG2=ECEcJUn)x|T& z%ZhjTxp_H0KNeQxf{FaJ))#!W$Qw-k^S6XFcaFX?1#QesNetT6_W3Ss+URVUWT;xq zpHnf5c|wn`(72n-uI%aiQG9BOV%DLbvLw z#D~E{_i0-zZ0o)gbNw0_g3sZsd%J8vW8BS4;l1X|TbqXp3g2qz6e_5NDrm{gigVzI zOpZ_grgIaa-SyT9wyV@ncN=4@bcXUG42q8Z&U_@#m`(D*0MlfM5#KIeE{0QaAVDN1 z!FQd^6GtCu2b4RBX&Cj?05A7wUl175pd&s@|D-d^%`enl>7fbSVKz10qI@!Uz6iS+ z!XU44W_PzU=I0Y_jYsPf+(Ty4SL(k4*APr%OWwN>5O{eo-A&SuC}Ks_jL3N) z<5by9JJOC|l3sTAb(A4;BD`(`osHKU^Mj*Ug@@BDy-curHrCE?~SUrfCxQDHsK+?$0 z88cFtj?2*vm0$|g=?%G#S+^n-2~YoIo|hA@Bl{})m5p_6bSI~;LAJkEKIh!!;PQf{ zJiqz4`Bt;%Yv4LfmN^>04Q@oqSWoMz{Ao|ScFNKK3gbKEKzD&s>Jg$-;UpvceFm*+ z0)7W_WLcA}*dA4ywxoqTI8}PHYZJ!UDY!YSCwenPss7T2)>GosZF!MW>%-yn&tLUo zb$8VXdb_6jhB!S_7WRJ5r8BnZ1lM5An-l4C&@b#XUlz7Zi^u;On&+uD5rPyjqI|C$ zE_$Td6Q{nO{Nmt$W0}WMD<>@_7HdQ|PV3me4KLnL3oKTm+#4(z>R-A_=o2#-K(w6A z&J5s$xg1K&UFf-@m)dDhC zQG6b9*3ql-zX^NR=Ntb-zwX5nS0W{jBdMx-1!`8-;gx%Oh;?CNlbe2n$O`1 z_FI>lSZ@dKsxR7y7z|pJv`8q&L1kJN)Gp1(>gI^A(NFgauUzD+fZEtnQstuG-vJ)@ zt;Fie8kxC9-eo4X&4;z;_vVhbEN(2MTW|UZ_HVy!&i*r`r)j&PZdQ@6I2l7= z`7*F~C;MeV!v*NJ*=m_bWmEgU zr^0cVtV)>3Ksm*%nxeGt^}B!S|IHb&iBl0D`QYg)vcjticun|s66(^-rR{;tGPW8L zw-h3XZ-tu$N0}$*%Qs?4HOvGMQIP%@y*4h$Bv22l=f3sQr9YobQnHZd{3@b42QI12fNL^AJL# zds$TSCDyTK$G>xWoVshORZr{9UmiD7mec_Cf=3w4UzPe!JNGsGl2rN3nm5P{6vjTr zpAZxtz4X>!&?+{~NKkZFXmW8FwqGRbCi#9}Ln*vgp3e=J0p8l%Z_<}~LodQpC0vGCDn=Azg>4=wGY!#LisMA?^F0naWkH}vQRm8@Fz(=d9plgt@zgOKQ z;CL2&yW3A6gcfwv!s6p(Tk0~e5O_OO$O3Xs5q?Y7ZK#WKA zb`ozz)bYTI(?l(pS{(M)_Qj=ZVZ)is>L9W@TdUKArG-NB166CxiRKl z?KU>g+PEQOQj|^OGEv=it9}Wjt0Wz8rWWY1=?bx@Ku-*rs>@dlA165$=he@Owt)xd zzWSYd5V$&2pdLc>=I^K_t+^Jb=JOgj}!o)REY=z5a1X~_M4!A{I-%Uv&A zW3u!1w29lbAd z&Yv>_ABIE5`V@wWyg7II7pFgU7BI`zIEL0_{k7}Fd@$KpDL+IghU9M3)v!NBaY$A> zVP>y?x+P;7CR|1JhcihKOOCLM2?s1vYgD9- z?p4ZMPA-{1jhfqh3mUvfD$0@du$EhkJc!1AwzD?Kcl}0h7O0mnAj+V)C}~ zxf%p4BAkJn*v+ugs|u;zOy~H(K{;>drGKcyaDAw+`IUIynEcl-M>F58;mxA^8{*Ag zLHE&%g4t5`x1Py$N~~ZblZoftaIEbj$-7W(TTx{}`w;G9I^k!=(rW;RBDD7L?G%&T zwQID~)pX$90s7AVT$m|L%s84JbUANOlbQ3<(_>=u&1>V6ojNoBO_Svs(78(>Z|`vi zD>7goW5QZrIxM2s1YN@Q0q-uR?zW&M)(b5+MHpu-C-LRLV+isJG3!}{F|k0xY?CU< z)_iBp!PM6O5+Dyo_Ll+4ePf^Zm~jvLETyDnc$4=p?q!PKg$qcLTU{X=ramGILw z65^xc-b)XpQlk`#&mb`Mvz-W`FkxjV!$E`RWb9`F1=0~e(F*j#R{tl(uHGwTTagj0 za>%pVIs$q8Zv41I$8R$fOo)5%XVvFjYkT;dpjqe1`vohV{zrCw zd9-YDe1r4GnUHs#+4udc;-JH%hy3;eAIoL{JxOO)b#42Hz|5A*xJ_qg`9sh$R9l=+ zTTKx{bL~a@zEyH%nXYHQ<7G(o5#_08s>`nk}n(ZRB0Lf0;3zxpcjzv1-|455)Sd39d?wjx3%g zH)kGp{}514)dMy=z)|ttE)=x^sJI({sCesIcM=8jY#kWxh1q#N7vK%cOi+)|h`)F} zEg7rTM&#|>LL=;0@c6JYg_mI!DGRu)W{0lZ2mw-UaV1mB8KnNa)<*{;?Gl)9u$Dza zRQTaLT!!||)wL>MC7<8=h4rnakXH90u=4Ifjb(C^TVRAyF97h|TQFS1y{vsUFjAtB z4FChEEQX<~6#qFYtu5!t$jh}d+6t2s+t7(R_Tv0T8bvkYTF6h|9kI~O2 zF9j95FY)a7l5gaqkt~%NcJ?Hd*-%7yyYx|X|E{i1df&zPl%uwPm#FsQN!{PszwQ$> z+#0FEj6kcw(Y2mkfrd|m4A^Q6n;2dT<-Y9-@2>^C|5t>q|7&z*kJPaMZ|fobjx2>2 z!vFYaI(rLi*QEk{(A~%UvYYu6)6F}gCTzZMBg~t2x?ZENN97Owk1_v5Hp&SML>9c3 z(|4(5Km+r|KeuJXMV72~INns0mvDGZ6G6JamwodA@4a;UquZT3HYH+Wr9VOB;ZSQJ z)h(Xk^KMv2+GV&H&f4a8SAkAfE`UG+{r`P>GX^1p^(-k0vlocDw8|@|ewOi_0Lm7n zkMMa1_+m==)6yZ=r6~WS7Y1_k{J@9?RKF_J|0n^a?XNd_GF4@QZBQ0rulC&8K*Emu z;T{2=jva&`5Ct8m@oKYQ$F$jJslZ_S%2aIm?3{I}7PQ<1M#r7Oz1O}W;yg;az}ERa zH;&_}20*dbR$~c3L{%6P$VN^Rs2P^W902$O-PLC(_BpAMAsyqP9j!eDk^R);@Nt zH?-dK54P@T1um`Cy2^4WY7Epk9`ON~~ z3Mdp@x=ESfig+rLX*IzPve&nXg3@0JJvu(i*p$4ZzdE6h(g>2jOippOc)R_CMZ~3C zVynR3bwc-jiy&ejAptl|Pu@lF*kv_g{Rax%Th-7Fn(+&II@OyY`=8Hbmv+;m)n-~N4p(Lxue@YRp>~mq z+gELGnVTnK9tdui9?SI5~X5$&Fu;S2JaG zG;SOz3w)G|7_+f|A9L?GN>Afs&&9z&t36n;eg0=Y6OG^6F?lB^X%y#GH>JLPx`J{e z@=nJl^hZPf8Ag}CZ4ePL!lS;o%sS^5fe&jME!@wlM|a5iFIGpqNxFQ`v&B6Clk^Rs(xVM?-NcK^D}xRPTCG0mfH7`*W1&W*Q zG2w|daeg{qzwEUx{xBx9{Qb+0xe_JA?^fodwF!I|kYKjqq^ZdamMythdxF%pCUC*9 z*X{_jXifwVF*~xqy<B|8@m>bLMy|n}w-Z{t8+a ziY{6Cr+`xIh!WjBVXa@3Zt_Q=%1ezZ20TsNh6M3t(o{rP9GNphtsx`ti%u(A%-HXZ$sE^*|Do?ZS{MU58w{ks!oG8l%sMukiGe?E_E zY5wSllx2#JOJpyXi7h$a+TwdcrLvEu<$APBM&?8Ex+OdPwZ(uve`w%l>mAPjkb#4p zJJb8G=lV2UnMP~FzMRYQ@JP=w4t(m!*>ttdy*F*blJ{pSSwbq}b|2CNN{2wqzWU?h z1j0MjQUmcfN!VS7-N0Q)UN;g6uIzhSj z4S)BNMIIbJ;-*czhZ+W5$~f8Ii+{F7s*gmO}OC za5bC24>6U{*01DTv`;|U{CW{z*L*8Yn%xQlDhzARx(&pUh?RwuE0>Hp@vBZB675?| zCKcLBs3)>Zd_D8J&p?dMjA90x^J=E|RRDtYL^&GUdTV`deg}5BSi0LNy~A=!$yyP? zD@?$+T_htu5J5mRIJ*m_WlDb?DDAxkWBohWo;vk(t}bu^M!B@_bG}(rD|<~2H^xY8 zSeZI>;Ows_&FVC}l)25zX!++18?m`u2bxgjamLuSuM)$b@5+ZwYM#Bg2sDqOIsEg` z?>5F!MhS}#BCm>nI&m}7v^FwBjq(<8dpGjd)mzRD+m;m0l#PwE7lcEgMNqw=>gjrF zXEpbRT7;kY41T8EtnbKGH%CYfhk!!Oo=OiII|!-A(H^DLAOK;DSyG?Bz+)ce(!~{Y zCAJf9N;pKj3BiEzgA(vQ2?L{~flc=e7=9nvTT3HyWJu~tyM+ZI09s)Bm+NU-Pj!>@ zx`mPNgJ%YK3+hz{eR9}p0B(bHcsk~gcqe~MahJs8>}SELvM#>8h5fm5sPLKL$*X-K!b0F-|^hp| zX!>4FvA)*HCrM4zz}5tP_Z>@|m``(4#WO24v!f)oF?zOt>9KiKs?Vz0f!SaeX`<+q z*Jjw&tuW{2!{uw!D7nW+>4T|dhB@sQN#~3C7u#2-kdPKlzp>n8ZfNaJ+5XoCYQX~G zK(xg!|0A&==c7Opj?92Q2)^ou)Cs;smhUdfAvbPfMF?$Oe z8!QtJY}&Iow$M zk4!6ezQ0}HRatBZR3e(DO;G=_0TNwzOgb`~St>^OqpV-+e}RYCyNOs$|E7Mm?)>EC zSFu;?*vXvx(vL?-5Y^UL^-w`acU56`)zyEWqxI56BMqSonFT(8@eI6g(AE@X*w#B_ zB<$$W&7#W=hiG%h8TGt&+>WYuY$;%K!!@mAYmCnZ^cXBH0JQFDQFDL4(AfZrL1E3d zoWSx?m>(jMUYcL!%Bqg-sz$hdLa^{TqWo=K<8OCU)g_)do37S7hiqxh*38W|v0g-Q z{E8y*x4WMsn({F3kUHPJD*f-;6b1TCiX@SWjLl=I{k$OKm1w$@N-4|JQu91EW@4kI z($msXKU<#X*wj(|)0z`im)@5HsCp(>Yx~P;8Ua70XCqE_t*sH4t+NrEJ46m*yu&2 zcOlT$?AkTKBL7e~g6*W@wVH()g`xsnkiu55vHes^o{nO+6CQZ#OK)~IakpDN<;eLW zYmSnOU|wc8OW6OB7sl-qJXf+6vwy9kdk>>FlHc_Oj0#%gl`=sj-D zX^S10)sez4yzkUGiN6}9BOqUD9vD2MU|M~dy~+7LY~&+j$V3|n>VznP zNNT^+e2hp-d(^%4^mEH|-rK++HL_-*jNXQ#XI5H0IvylrXHVPugukvM!RYI1V(=f{ zkzf17GN;n}Ov;uO932i>QXdW<<<5#5FdT}3+Z@{-T@`P`=x=OBY^1n4b=`#kQ!+o&Abg43xDhXf%k3AXdSfMa z`9>Zch0#~fX=`%(zga8yBylaMuIr4Kz}m6EH$U=S*;uezpq8tpsomT%OZ#$^P^%Wf zfiHjiVs=!hdsLb?4u#5*N@L=ah8t<{u#`XTFR%KCtZpTl6Qa$c=}lfHX=heEJ-z^d z0E|T=iv?BrW^R8>yNb1IVswW!;wA%70IAaQPp!}&kG=NVKfAPQ`ie78U`Gn9oJQ4U z-`vdLU4HxGw`=k{D5ej)ntNsoV&LOS`M?hOTO-mB+{=qDn>oIMs?Z=L0C3eCZvUmH zf7f~j6ysROVCin+G5^BArQuk;_gn_%iuxe{!cX<1pP%_1?G9{^2^Jx<#`>HZuefG* zD&x&VkvIMQO-Mn0w#v=ZgTT7|g%*E%cfr1H?oA&d;Z>ODtdp@#XS|cO<<97D>KR$1mXGI5!h&C)e@q?fE|BW^E`0{O)-m zn}zGRJ-IuFh#>9pg!wY{01GO8xF%uO_jTySvTQ=ON&A&(X`c|nO-Qz4u}*q1{x!uzX3vOoqp%FT^QO^yt# zyT6+((&{x*2Assw0sLn0C(Bkh4TEub>I}-X=e4S3CNA^Q8onCKm{0GW|GfGH6-0B#FAd#}NvgBx+d>;i zmjJh!gZJ%iW8e&U;ANJQ41u)daR^S9C~`|x7Hta%Sssp%kC2q+P@N~6ynW8nsC`%R z{%|Ajf$eFPu|I*tNA?Nx`h-n$&Z7@$_uA%*V?_|+tw;UqO4re{6Xqp<*V8u8cU#)d zK6@QzCR_%FPL1dk7!rp?X+iE%HLpuf+m|0`2OQp=HX+RkB@XrI=gYlG(WN5|KHCE* zN{u$PRVFZwO=C2qgBX|vv%>5RdeUk6V~`$q29f98IGvbsi?+ZeTO?-wjMjtFY$QHo zrlOI2J-T@0o73#cee+bE+0!Depxq{*L39WeTH_h@ko3Wqm-Xw=n~`}J@H!jzM&RzO z9SeY{)T!QKN2XiPntTqU4_h&}51Pc~=t6DbmBwy+f0v!7-)}DKSIBx!LC|{_XLV%q z4Xf&K);iLc=Z*YmDqt>5wV(=G9b z*L9r{?>;yAo$kyuZ;Be2`)<5tnQ~cbJWYaLuWt@#h|*+@BOyBAQsUqro=$^1GQknT8Q}^5utuPLdRh6+I=@Q2XQcL`q9#WXG{`TrURj@m zAQv_*Q2QYD;%H;gvoj0-bMSMusqup0jC%=s%s;vksWEiiSnazLZFRWOt2bVk4=80m zX4cEiC<#lYYnw)BnJ(>mti9ydGG(=x-uye70hw=1sFmR-93H_joU^@iK7Ay3yz~mi zoX3i?R_>~N580>}(z^M|+BFi`k5`I;@=TkiZ7cxAUlvV{`Ykm8o6_oc>k{gs@a0bp zz4^v*(JyKK)vyr|u;WRY{dh?vOfLH5dy`bO5cK#h!P;TfOGUZpEjY=ZBKM^C(7b7~ z>&!QR`12XhUg-`S^F{5=>8-w^3Ml8!_u;jcFa$?9`-+GSNZri&;+Eg zKwgCN^>K&)y#(M{%tys0Z%|Wl)Fr$W)WDm8qO*rO>no~X63&#JRKqtFv}TO#CYl{) zz_^ZtjiR=Va5h}DcFism`N@SQG;lb+6)BZ_X%@7!fNy0l@%_=Q!^|}Bm*-Bsu9k)8 z0AyYVbURzZj~RU~lKDc|(Zau{ccJY|&_zP${9qf%flq{NVd-u)R{@l{D zMEl(vxwcM1dwZ|kBEWt8-Xn+E`@31n(zd1i#DkX3gRLp;KNfcWdwA?3w&f=)n%@$Yl=r zXjuO4S8ty0r1BGF>uM$c#UWJC;(V#FJ@$ve{2M5sV!(yLaFS8-4VCR{&v>BM^mVqbhG^@9e1X`*c9u(zRcMF zim@{*G3I{4aGg{BKk3s6n<2~T08peZpSznFC)Q-eA>n2=t!d(cwg2yQn99@h80BdC z|MEr7x3$6Pc>FG2n0#H2in}VtC(l@1CR<2et|yW2=FaT24gu-H5Se1GaIuEMQ!XJMf~5tuP&hxcuBtwMe4u zUHkEa)bmi2ki(e4K-9-iCCGTVoo3dQPM>#jrY-V!!{YM0H(0*Qy<9{1;k;wzQmr$S ztNym$O&GoT#drCO?`*iyaQzA=xsQ1KMd*99*}h+@XO!3G?rh6(*8XIU%;{vml(fC_ z^?&K0ZI>Bs3~K`=(kxP&&)cq+_NN(5f7#trt~}B&H%>0ygV)#jnU&p>ySqM47$5AS z3_AFpqf))b4%hkj@pvmr&@I)bN2?+8@2AG4uY(vIQ?dLX1B(K63>W3EPF}<4rWtyZ zxl3{RgInln+U;3<5Gd&E4Ph)rXOYs8>zbE7!Bg%Ea`jklmr+7OL5D;4AzkN) zXLrviR?olMs0gTj|LFgiEmY7)4e$*eLvCY}(;Gl%U)rf;fHLY1p9*Uf0_U7ZeBV8a zDmh}0i;P{_5FyU`{?Z&X*~?Tz5+EdS5}yN=3cWp<(tjXQd9%5!)pX*|hL4 zbYuAHK%K6#Z7VjP_;s0_r}lB@REvJ5&-gazW-RRwPrwl=xOjthUu=*~+M8b%>qF6& z205E@`bHI z-TR;gO|6Pk`nJ`vw%fMkD3oJ*>*ZRgyFhk|2pg!sPSaZy8Ib_WLSrh1nV|k_Cbw>} zrX26AS3~JDpW4g)vfA|b<{nP>RsP=cgRL8%c_SX&lwX2q@yGrE*ljX2D!(ImsPOLN z=h1iINUlGTT&^oLrw)WPEsUTOT%ZM^FhguQ{wbH1V4l^Q2TQJsj?Krn_XuchU48&u zwhA+_SCxJUUxfQvh19+;5({`US=qF#=P0{t8swg zdfSP*4A)+?8#>Fo75~j&U%=&q&gOu}|1t;Wsw}%gTgm>nXma~s1X^3&;6U;#m>dg1 zhui&6+0;K*BnOqrrR~sIp4_dd=nS-CTUf^W?;qth0RQ=PA*rjstrw+W>1-pbz{%WJp3daibA z>jS?z2$#R^QCayG+TRGLy8yeAIh6)+ z#_}>Yp2x%D6ESk<4I{Rt!%nn_4crX)W@d6U&f++I<~?WSiE`mNdj_KnbJ)~Z8s6LW zgkJKQ6&u)N)ztzuqs$(uuDQ_1-hzgaKb;aKpSTqdTk2#G7J<_9Q&rp?_Tt_QW9)Dr zyb=os4E3k0L}=ZF^7js7XUx1=;)%9yk)XC4iDlUhUhO$w^d>B<;Z5tIF!Yj)b09x7 zJj_KxWe)$Kk28MKN9@=COz9pHikstef}Sxf`b_Gm7SI~`xK^0dX?kR-O!>{^Eqn>2Y-g<5riU#kc5TxwO-oXFrBu0(IS!GULy=Y>kBVRXR#E8EtCS zPP`<2FM8cpc>umssXx9MRj15~@Zr21PoC$8fu?zWc$s#!i-7yrWaDL4(_y!|t3H~j zbkbYLHn?n-X5gh_T>_K%NYgqBtZM&sxeeR9i*2wz_^M%EcJ9BY>lIa5kjpNKSKNx} z)L+V613AXYy%)Jug;v|^)>b&__TPA|KleX1<-zz*SGHjik6m?cn`CzCTi9-%3;JmK z&L9I&W)_!|zt=Owkze~I+ed+Xu`@9{WvlOFLg*L?N^nAiQx}cGpl^9SWFH)iWb)p} zvNxEleAU|%T{Seh!!Yd!-9W^z4-bb~jEBcn81ikJ0^!2%oY&t39D5&ME0dndoINl6 z_jIpAGow=SeDG1$HS_m>GgU%Z!hllM<$$g*pxJFPWhKaht1~KS<~t=4^G3xmhOUmJ zbc<)sb%VX`&gZ~ZfA?H!@9%NCB`hypcai{YFW}p+E@Eu- zKdINO=k@TzfDHA*{h;%v#V7y!0i97oNT6bX_|2#NUqoo#|4pMd$KU0nsWzp7m)n;5f|Ii`~9H-dAH{xbnxK;2*nsGG@0f zzTCID%=iWhR5y<|mSCEX5gC7{=sr8t7dPWun&=-H({e`Sbt8CBULxX$W4k`R=_*Z{ zFICP1{ouJo7PE`ZdxF}R%H96HNr>qGQo5%uR1AIz0RvF@#zM4JNBcovY=S}zX#Lqj zDtD2d)U*G>y6Hf}g7t_00X^0Uz^rtRCCrO`A2|udr_=Z*pZIgnaiu-h9}L2ydv{?f z2%|jxcSXcWu8Uyfgs3FvVxHd%<4ns3KXUmF1JqU;8%6X}YCkvsE3VC7>NZ|tP<{sj zn5^NgI$IE zuKE?*xyVvniTwQ+2DIW?V8ReO&sSu1n;4IFr8Dr3fFF!%K4UpJ&SYhYLfY%p=&tzI z%&hFNQDOCy({jMJ?+MJX0p2CedZA0%BzrXL*m}g^4m;i%LOmJn9+9bMAuESJp0%gwZg&dvgN_f*cfJn8SuD)jpvVa%e% zdLL7m_;G|zN5)lOR~5Krfwx|RB)6Ko)0J}P>KWn`1KwTU2)V7|k%`prr0bdD3vAx1 z7KEXTQeNW@+S}gcG`fHH$JOI43079amt(ZvLWdIW64m)vJ-&`AJesl&ZqP>ILQ~JK z7P)u1pUcd-l9n&lv>qi_ezqX|oSDD1v{E9c9RbYpoW^&uY(YD^!#Tgxc$LP$o4}yW z$~#u}?(xxLR^WZz75c67ZKIsR2Y z782G#8gClrMEv9MeW4w@KapQ4w5K$EsgCj-OBtpA?4Yip(xY6o=C+(`#;|R(*Y}&R zcR+}g(ZKqLr81#Y+QVPt8LIaO?$6cDJr)@-zXS2{-BIfPR6=Y)usQxrceL?da5K&|Fps$roF4nhCp!vEgwfXN& zOyB=h)H1Yl-gH_8wAI-=E$7{Y5KkALelxB1m{E#nCD=-w4%n9|W;bm-=3m%vdOX8e z-?jY6rEYgc7FbqyxAk~gT+r#GZY<~BXJN58;&u62a=+Vaq-9FKsJA0`EjHx+nAr#) zWuZd%fhRvp!BfB3Pj;lzKc3U{+Qk?e+i!j3lAojpt6u$KSb9@3QusRB+pvS|_xyK5 z@kQ6GQ+lX9nDKwo6uV|K=4#wCn)F%zGc^Xz^Hm-H3k{2=E62^z^BW1KHMHZCX^)!Z z_v{*P4q`@v2uGwY9Ufwc@IiXNzZ6WWSr|I(~{kL-V>N?X3Q z@QEk!qI60}&tWGuTrGtOJ#WNWeFc^7$udUuyqy#tk>j6Nz+=DIc-GFplW%y^J|xgVHqGr`Eg701Od{G~Hl)Ec;n}3&`D5$e zf=))5ioW}F$p1X4C2pv*m>u2%i8`caNE~z%9_09DpXbu!Q8pGgFDcuSi5)AVr^}2` znxP@^(8ql#5Yf}M7!eU+>v4JrbQOPy)G^YQ#)zC!=A!wNv7HD*#5;41`wmCvZFe^* z(geLUuDdTM1a7U)IUgPZvpb8pHV|*KKKKrgi3f=fR0%&E7y>J>+4%)tx@RuCJ93)2 z5)p_$xU#tDq!-FnJ0>GNC=_prWcqgB=!&F0P>B3KJqk8uY#-dDD~0LqRq~XVC|LFT zmei5eS;okpI@#Rxf;Gr2D|F(OJ;%4JlKX*BHFk;pBTgXStj)oR(NrJeN5z)hvuX1M zezu-m`tMqgmLFT`iuNyqdxf(POAK0~xI8Xa!}AO%7?E3KSx-|bnD-2WHVlHoTxtT0 zJNr}!UB9x~R$i6Abv5Nk{H&(|<-1B?Gt1`Qdo`cZdR@Q-3sZD8&icVeDoq~5r~x#w z`fM}ye*nKgK);pW7>!g!TI`F^)HH5X3WVXt> zwa@sg$-Q8i-%cHxvto5(Y&5lg*-pFS&O>VcE#7G`=f5w!_c^{9daZ?@-{$q}kXJnP zh=n`ssOQ>Rqg=FX`{nx_xZ)8t~GPq8$g} z#J}VI583H}RgzinbJP>ED+JBbokqiY7cXxnd*5{T=-9aHqfeBtB1-mpk7EW_H1xaN zeb=ov%}OZtQQx=>SD*z&aXnZsp)+S(jF-E;3cZk8pMPUBMnib7+t%W(xd?<-^u3RH za{AwR{@2-T^3klWX+uB0x%*L1IQo@;%u7t;>fSA0HCn#vFqzfT_8d;*TKnh%m#?ng zxn?x%ahJo7&boEIAuu;blB~J7nU~HroTe#dw+${x>R&e`{>>J0&ChWUlH#YG2g7fE z^PA~~iRaH>``XtSCCY7leXaY+Pkxf?VXhmwRspPqu5fx)p7WgN_zKDw8x}Ik{Z?l1 zAmb*S9?uaOwe8J9;KGb)#U}s?Jd+)Gx>Dc^-tz`n=~o&u=y{kXE0+?Q=x0T6*LoAE`GF&@kHzgQ_ODaI@*| zJZ?gboO5NG;r=O<0|K38wzHH~r^D(P9p|$}Cf9ikdK#Z@EbrW21SRTAFyQBv|2N>nMgKixITZ-aJhhH zT2S;fT&DK)fVa#Lel}U4Wuf8n>H%-8prL1kc`@isL=Q>5(C3#8m(FHkTE32>s(7#h z%X%|ECh4s)Vd66r^g3s!OY%ssH3uvf+?7MRBtKS8u8smMkr@hz<8sDgh8sDg9;~bcA%Htd9_7aaGiih!yYAxDi zd?Vw8G`d!XbdHaS<8y7EHzG&G}lc-$0 z0CL8E|+q!cZjjy<9~MV$2{epjZJsVqQ%?KTeOIj)U2>N zPihnZ8r4td8MwtaDLwWNKEhW;5VVY<4mJAQ;t?CW_}l$EYB}h&ADggIqt>gQZ8Og} zepJtz8I+KA8P902)$$MaOT^tgpD#GfAIxvEWXG}V5C~U1Of{d(oO z$6?1`fBsiVAAJ*=bQa8CywfhrR_wNXm%W$ny8F(D9LX@5A}M_FAj-PiZfVB+@Il1w zcOZ)NDE2@0$>00&cQ>`Kt1R4Z`@N2QJj!-y%DaeOEk3H!?6g5=Tr+tsmNtha%eLR+ zkYiQz@^L$4bH$-Y^Rd_scdl*X@_PK;P1}fNP^Jh1_nI~LY^5@FPSx6w_FVplv^G8G8lu*9U9NxUj6D=)pQebUnAp6z;X;LrQzZd2C z<~P4t$7jfSG|0ydr7xmrGuaZ*nHT_`FfpAOL$(2Mf@Ulv?{A`q$5iB_l#&oxO2KS` zr-hH5M-*La;R`Rjr;zy6Bir4AlpAe5ckPxJs7%U|3Ts-0y0xAEsQoQfM*TnL2MD_-#mG1Tpf zM>3f3V%U*hWO|CVnL9%;8ZIAITDfv%jqKFWYUz}vesR}pik&c&9+*eN zWgIwOzNw&1*JIu~dAQD|I|CC9m*=!J+)fxuy*Pv5hKoG8D*C9V)I*+ai_2~fh^$jN zAqQW4JDeP$_0I;)Y>QAtCnW~dbiR)_AvdQq2Z0aSxL=7Uz@PZUC+PAoN_^`>iV!D%< zH;y|5IpT6$3wvcNJqf*nNIlO@cW31 z%#7A8FWhcN=0oeYag9s&{F233l|urNhxcgR-pa%-!Y^D(*Ta&&%ACL5c1t#IXKPUw zS^4!3;w@ajcPxx_sWk z(H)xW2Z1zb49EV@x6>Wg7G#M^YRsUtZNgH@%X$elJ^^iG+sqG^!NRHUlR@&kmRBDp z^^8R3wGlMbGvoLecfi#qvsZEf$s;i?M=^0N!5b3*pk>|%2H3MZv|o6X!f}#&NRK{9 z@&}NdV^uu4*sStbh4i`bQjbmC%37Y4K#0jvX2mvVupiUX9;PR8=2Lq0wPEvghSpQh zPScdi$^0T=B2s4P=fx8?!834`?u<_U&cf1l=t1`kdTB$TVCQpG&wlUA1bJ zFwTT_uF^i7!I4n?Q~*$TdawqVvZ(u6{0J4aKrBA3O4;8{c>t4z}L>CI!9G@Wh9UjD@Pi-Jv6QPB-l^tI6LHdQ^nw?>fF2MI zBYd1lL0NlI+H6Yjn$XD7r(6IE45KaR%Rq@xdv$8Q@P#j^N25qdfl%ssJ~iafZO~>l z6!<%^QVu++A)oJV>wn8)Rkhj8PV(SGSHSDT*p;~-W`B4pl1#v~HgCSJfQ!1MC9JUijkbSgYAY8Ql zj?y}3qZce*ylhePo>5C!WHgTs4t@T{hNjR-0Bp$1H^0n_Aj`^WqmJOQTj zH>ao*(s*Yp4el&SZzM{|MkY+5jgcwq2U?B!$DPh--WC)_;WA2*fS6IHzB32<#HFW7 znQ%n>MHB$S)NIn*s|5~L%7JxaB#Wm{7F+FQLfq3TeC8$%YhbERgap*?g(ks{D?+ak zrd;S&V{B~QvsMH(TdmK0Bqt`VTo2HlAq|a(em&8G*WxI(z{R1djz0Qm@QKkTkE7^D zl%6{~3BT|Qzwnl~yakyj*4!B8Rb0`Yg(nJkj`V2oJO+*CRpj|7vVXrqGBiU1zejFD zC2O6<$y6n~Ab<<29j#BW$y2B~WtF3OP!8H#Yi>Ked)sPyWdehXZv~j`dh9qfTjygSvtK^&xw`=SPLJ+OOxwPwnCZYHgzg(?GzN310r`U?#loveW%M7DmA`Q zBpeMjzHtjV?aon*@r`WKJb0?id_^`%K+jSr@7vAk_nKPK^my&WdRO)_80jkNL>5YU zv8Mgdf0V7LuGqr22Fyy6ZHck6d|Gu@?tqnir_m!PhI-y*XcW&m- zxsx|;#P1n2cx^=Qrh(*@a`k1T7n5ah#Z$au-I*kVI5^02zF^oYJ<;4qBu}T@{CbDK zU+!T#F_V?@s`HVCV|nM{yiZT8PJCYu6L@MDfoqq(h(-=;s5`g9U0#I}29soBogq3I z(#!G>{^=c-3&G5VUYm7#))ywbck1gX(&X9<6NEnU2`-(cIv>IM^jN)LCSa1JGu0v) zbEZBWC&aREVo+M2oT*QKVe+S~o=SeLoeMpfsmC2W8h*w4A!pEOWPd_i55{ou73187 zEhs2&STH`H{tM>j;AL12r<|vl8cbJ%MPg3g^-ut|?k0y{9z81P%9Sg(E;(!rkM$(( zyrk2~?;>|v?`7rWmeYA*)0cmro)UTIPQa+(0(4_>qr=daGb zq_QcM$h?C3>%ac%{tg497W+Q&#)}~;FV_!lGjwM-Hvk2B-=<%W1<*MmC|7i+pMJXB z=^|muus*8k^-A9Lpv}U)Q(0hU+??}0%%D3%6-uHkk~3slr5|E+03UftKeRU3l#_Fu zXM{8Zsy3aeidIK44?-Px)v8ri)(zLD%z;+Rz+<&q5~I~}otntZLG2O^gTvKuks0;p zWEl7-UzM>~%Oo%j7ePyB!{v2Hri1~2*W`vfq-p+Ubm>zgxo>TE0XP&E{RPq&Fe|OM zkbzKYkuuvrpWf~O(n|+K(t~6`+Ws*$QlwGVuP$sn|Gm)c1Ajm7%VyM|_=0uu7N|+PhC*#F=i~a)oo{TA|te>q= z=TK|JNyH>q0zR!eAKa5co5fmS)7m%rCtGu@tlE-E5ag?%8B3ZX3d+<7KVw^Rl-gIC z8X#t^g~@#S^ky?*dO7)Y&y)mC7xTy^9TTR`L8!RO7lwC9m{AoC!}qvMM@W1Im5kL& zF$m9VDIEE^)~9oX=bSHj$xC={$$T9828uA5l}RLE;%N_yH_>D!8Ov;Rov_HnI(-ZN z*aYS_Z`Vd>^$mA`dqsJUn@4Qp`q^&PIn)|)D(w+}X60;DE_Un?*RT~!YaOwT`wsnv ze{H?#qO-pF#?c6=HjwMKK^uu6Z{ilQhMFOJ^H-RWrgeuQ_1a&;l$;~aVGh!C@;J!> zS6em4XXb6HC3o#LEYX0tt(E~_E4KlLtD-iX%*43KmiaXDFnX!i_PAAbI83Ge+vZdI z4r;hAOOd}kGkG-bn8-+3Qpxs}J6?4jsi_&n55`~Y)-ZVVy6ENAY0+lHj9cp+k+JR! zSkvHn27u6`!HX?CMdv!1!63{oB&2h&BLd3Yw5*cmrZvh$>*SPNuPLB?SFKuQcavMN z$#|Oz!AN?v)eWSA=!v7inOmPmJT)@$!W0RxnfjVXn#5$3K0UO>HGFy@D~)sqPhW!; zMg@A2hFL8waBHB2t2-7>tR}2WCylPSU_i)BYJ6Iio;11c^UC5#Ub%9mz-%(#mK}|R z7UkzY_c;bfu+YxwDC8=FCm-m6aaYLfvz~NAO7sXJdIjA0Lsv-wP7fiC&$tihk zJvgs5*xI)xKhgA6bdeqn*T`(Rz^td$o^r}5Xt)ODB@v$z&~PV7Z!{+Z zgOk6r;R4|GY#Q!G7hQzRI76oS+-tAVhvJeUWi?zT=SNAy)r6o4V05%+@z|1vo1A^J z%S!{YEU>0a^i{aqVjmSZt~`annmn}^Rt{>ZXF#=INl0G@k_!jin%|ghB)Dsg@r}nH zfBbBt)U}!nl#B6=(lEY}5Vop*I%~t2`%0KqRz)kCkh5sn%SH3(3+F-m;$--)Tah%` z#-xE+{1ejHb|#M+n_F9&oEWv#*Oug1*G4Dd1>c)u+RQ0Q_il=&g?tS{VzV)rV z4vCd_Rd{t#Fb;Iw<8k}S)tx@|Dt(c5C+9R}>2&N=@&={#T&P=~gFpr9^+O&UGDQ08 zU;jE!D=Co*^|YrwZChV;&mrf+g;cY8l{pv-!gKY-;_1~3Yr)xfb)A6LM z<7M|ks4HT~-vCoiWa>qjnw0CZcp^6(>1Hdv6SF>}5fuB@0T z#6E_C&5mXUfRZB=;~S-9$d6&^E+p&pEdYrDCLXOPd(}MB$SbXVxq+Dw!d^5XwIW^l zO>luQTT$j?HYvSXuocM*#tD@Kc#j;zyUHYv540J|4LV(?O15@zMfl?g78 zHf2N@STGr!o5&nA2qC505}VSPQ$i|DXbjd`nO5XJKo&=gU;sfd7;dF$74 ztmZ;oE?kI++0XP2KKS4|R@OttN|_PPny;76Yrdkzw_V}k=7Di@pZe6Nc!T|0-}+X5 zEIfTJYg`S37+X`$t4L<$HsGI7W2y(?vbKj2n!o(6F=_6M#B_Rbg4200 z8B!kP6e>7b7-+@=Va5^Y-i*=28jI#M?MN@*bQg&<#-NlYk;p8`pT3gQ3fMfU7t+2m{IMZ_X#Zr1$%geoTH(}mU;3yM{2O3B@v z`Z7ALbSX;D+gQbf?%Wl+s_7|%CmE1)lOm`T{Lnaz@Jl2*!}jUUFpNQ_S6+FgvSKbp z=WIa3IQ(@W2;jB&Q=hV75(KtV$Ye;_FobkWFW!i46GAt-eoMzrSquw0ZBr_r__$;E zcj9r!Oo_i2U@Fy&f!uIGs^O*!6`gf$lgRlD=LsiF`bN6nfT|m=dpF=T%={LL=rmk< z)@3c;H&Me?V%wsVxX8pAP8K*Y@iWU82ZWz;?@T7ItgP(4D5R+1xTDA*9TMc731Gdh13^a0X!u(;_ zqL~z%b0#-`GKd^gr!|q5anV0(Y9ZMiqK2hLFl({_pnTNs%ies#9=2NO%cM__W9DX_ zLNErwa*?6>G787O%?bH*y7b}u^qFTT;jEOiV#{i=tUSHxI;@#xGSYdP!KPL-IE3bJ z*)H6`Xu7D*HNzaN_pOe7CkvNdb{YK$EUQ+n;z^j^p5t35z8+(udL2J~RXLfkwlZKr z({S(=aE1owLgzT-6xpNy?5>}QLR-0w1P{1_zs_WCL32M+y?hdp2~hXlci#zY(-d|Z z%!CkIZqj}!c>1p4foB7*bC*@sX4Y-=HN%S@zC zt7Lt1M99f&`mRV!xiK$`+zS_4Ej?ab`_fjJ;+|0za^b^%%+SdwV(?-}cQsrzn0Q4> z0Fc6PmCOr=eAyd4NIt6K_qV~2KMN+nT9gtO+|04WbucD1Tu8)8!&L}1T)fO<1_97Z z5JqchQeSJ7gV%C2oAg$)X=V35@%XndnnWjrU}WiY=4 zxwAH`$iM#m3Sjq6ya};7qLFhDTt?Vf#E$?~%4Vhsb!Ib0!*@CF7B%Y{wBw8PpxfF`0>siqlqYIuL?PthMBMO)zrP z)fdI8-xr6`-?Welm1NL!Y7WTT=BR>Yjb8e~~LWRrH zq4XwJ=2g^PAu{7_nU??n7Z`L8A+S9E`Ol}(O0L&^^-v?FQ9V#DC1z!Arms^)Sz}~P zz4CctH9R6Q@=D)~(($rMCUl?3ncp@_KnFMls{7%jcdKPaH(D(R?rgPZopl!NvQEzp z7ZVq?j5lUyXc4$*xSmMArb>+(t~B%usyl;|3gZ!^tQ#&tlVJ{Wy(7$7oBH^5Kx9~y z4~Tup+3HD#3wJs)@_;y7lC6l>-3x?Nj!$EAY`xsv|?KXtv24zZLUgU(lAk$LHLZlv}nXbIR1$mZoeBkcekW{FcL< z|JR^ zot|{xu~=Vtu_>=Y&o$!bKmYkcIjha#UCFGRO7P~5_gwqUFRojCbz@ktVCjlo_I&gc z_3cbSIXe+(?RA&maQ2rrZdgBW@sfRye$oyHG|$A)-+A5TSDyKWP3zV!STJw9z4kxw zNn?Z2#4zukO*jAWJ0R~|ebw64S8ZH>=aS_+F5Pqg-4A)xZigK=Z{flLto_q>GIU{; zPrA$~+qVZ-Gqy1;L9+rQ883X1 z!Px8DH?Eg8*AvEN5I3{JD>spc(MCct&!NUlm^@occ0QT8$!$fG#WbK65;h@oF`K_Y z`uAJ>uG=(5H_UbPP7Nn1c=$kjr&UV2O__=qnYjSCtIsK)1@OoPdb*{V?OGESs`YRn zb+dGP;^|s3qJBaMyv#5GHY*cHZ@Q}Wc|k^JaOBV1*J-GnlE?9>bkf^eQxO*{U>0sP zYCviZ0a1GDGrB}W}K8<4^Ca<{qFok~lYzs$icGf_F)UXI4 z`Qa4S&P|>fFqWo4%a-^c2HHiOGJc)Yo`v`Y>rtSwWLc z8segaO^b7@1)frZQJ#8Dn#~$!C9U>P{^U=Xu9?0!l(bsDy~pQyl!J4l;W8tJyjasI z?tDZpqyy2)IZS|LLCKnwBqtsX7peR2zrP!G%UlwRRU`%eW9RD z@%Kd$3048jX+e5UlT0zl2)7+~+|efcBDA}Gto@|5aAA|&lpb=1R(WMgy}`y$#?%q* zh=8x$4O6U)Xp@9&MZi>Ek)+o&8sY2HYVt@^CMM$+GMicos$vt2 zLs&+mh|IT7m7@R6-~3I6xl#62t5$iCaOUhw@E`r?NB*)2JmS96-?XEzw7 z$C^tJlJTYl0o;Cgx{^H_!V9DRYjOa9uaYr|?shuKDMZ4>mSvIGkWdNh_%dqj~ zn=d_o%>`dmhs2uio`2vIet!N!Dy{M2AIsf0U3bap@44g3A2%EA0>fQvZ@=sMYi_yl z>?{BCqlY~A*Ou@15a%Pzc+O?H$OoAC6W>J5x|6M0XZ?({-oKC9W4mFGhBz7w(wk4% zYdn*I-W0R7%JFADc@;wXGK&YjDyeaZp)fjywVGyAxn?~GoIEa13mdD+ipSV_cU<#h zn!)C~n{Jo+ix%$k&?9z!m_BkjV9@@$`YZpj_J(VDKE8C<-BvyMIm7lp5B(k2UA5_+ zjSH79-}ktuEZ^rKCDyPhkNNZOylM5VKmOib*IaSe^;g|>>U< zX&B+fCNk?pO6JhScr^M|nfaJIF@1W^NMXd*x8_oHyi0m+yV9d|Rsg0&)-X)*Xrk7%*0Iu>vKXYvSkl{Cy9mrWeN$x4=4t={KmbWZK~y#ykCa$= zFqxc^g-I_*M4&S5!WBDLakOMKnT*r0J~>gcX&xM9HLU=-i42S-P1-b%@vt$4B17G^ z-_dg=;1CM!O?#Q`O>cS=<23|$&wJiOcLqETt?!k{OG;#cjY&>@O5(&5PqZ~dTgHma zV{GnBai`~weh}sdr7Keyt~6G~qZ)^h07fuJF<^qmUh*wm*i60T)QH&$AY`mW3X|6K z?X&9ILe~3!?TqV!O_|&@jx=07-v&&?zJ9oZifdwK0HxL9f=R8m!03A=MO_=?(g}c` zZV~2c;Gzn5T1*2Q6g|!>xfC1$h`AOeSINl22ASguqFj+vAwh#=n7eNY-MiNa?(mJ98unV z+sz5%^&ajmuRZHC`@b;YOuanpw8T^=tm*k%1~ zH(&mxk3I4QuO)<|I7zukFQsH7%{h&L7dxnL&+?WBJsSV(8#kJ2Zc*$$K`07HT>3I5 z;QLIo&9DzIeOkZ&&+lA+>rKRMclYLN&;H`U&wBBmN1iCmApd(dUHQe2D>u$n2R-g* z7i_=1Hn05hzt0=I&@^w!&MUGz1K$PnHedaX&tCb3kKWTxkH<+AFLunlTh`om#SgCf z%BS`|`bn#v@k0HYGOLtvDXOusHk=&?q$a_B{m4PTYo^9%9L5*_c;Moc&IZVlv_4Z27_W zWji@5ULn_`M@Ihhg5iW=7j!3?DqlRe@?H>TRV`e4ky2*^o+eefPlK#6bK4zLmX3+E zJUKd(9;$`cIiRVV1S6?=_@MzB?04QJ;T>N4YLY}H&wx;MrYI{>NoK^;x#ylMFFGdZ z&M7BCH$9uQL{51(lL<~P476K%cd&Pb%)5sHROA21D902*$l4FgMR1WQfIfE_4h zD-Eu#HUUnn^wOjZjN4H(jHZ@eQ96;B5+h_D5M`}>Yp={LWpQd6~ z7i~#nWfPNf^3>63OZ2@edRkMqm9_-4OdR48BUwmAr#DstCS9?4Q!v)jc(c$XzLpcL z=$S)uO0Hwo7%?O=F8bc~_hcz&lmnW%3F>09rNha-I%91Uhfisv6dir zVI)iM45jQ`8_A>|lu@1XOjL)=ochUJ#A5LEs#U8P`ieeHj*|JLoM6dxOlW=tzG=ln z(6_whEh3TCrJ&=R8rJ3>d+f36uDhDb7#vg1{GAv=!JHt2z$+gCg>0)K#1u#seR9emwF+UwscguwS&;IPs^4`sQ`({MRFMjchzwwQ4 zNM;%fcW3ZZ`ot$bAxByTcZjvGGU1*S0Cx=lxcnf8`ZX~pfFK@ziCna(CzMh!Rri^8_LP%s`@@CB=&1e*mp$}YvBB@jp zioh=ai@*2_131vs4cFFMIPJ94q@>bmwF%&T=}&(0lhUZ+GJ2wOVK|V3(r_7Vkvx4e z1K1daAuDfrso}E8xIzDT`g~`r@2p9+GpIVPvklRCK%6bf{R&U#g`aq9eB-^79zC+O z4aM@O8Tvwd^dkTbZtO#D3%#4r)*`>3w4$wEuA(jFyO&c|-Hb=VP9C=<+^26#fVXK| z+UnKYS$33kZR?o^5uf|q=Q0?(K+;vgt;m^Yo|*crYQFi33HP_Z{q2dFo6Kd(G%Z|r zJ97&$E3fK)Z_JxJxauZu46t$ix+~83)R<~~qj2Y@R0}mM6Ib7t|DR?|qD>2yE?afV zsmHwPPapUHz3Qts%~wShEL>Qy4Y<1gxLg*xP$M18CNEsFG1x&p8s6O2 z!i?iGiX>f@X%d?iScF9tWt5F;@3`U{pY8;B7u(v*npiE`7vKB$_uuy8^4ChQN6|I1 z@lHCq28(IMFjQ}yzb5ab<9E$(w$mhK9`QH==yqiSezqoNCgxpfya>efZ5-47{LlYf zUy^2$I1Yo{O?hZTP0r;jy%^|kzx{URJm%#%u726%n<-oa^Wg2ZuYGNLH9y}uw40=u zz08Zzwvs7@NjQ1FQs#_mlsP^FWhQ1ZzMm-n6Td?ycG3ILmY8ir4Ddcrj( zcds$qLgU1?1^EMxlkNjrK0h&Uc` zC;eGICXfuXF4o{YEZEAtijs5BqVzqcky}yd%w#L_&y8x(ok2xpUBE@Uof!)$Yo9D@ zolxRsO&;TCj4Q8Pi`fhawWe;ke8q|}LgGnZ$+aF%rUmtfUFm<-h%zBR{pnB7*l>lc z8?N+@NT50&lpK+*2P~O}%gv8{QPyQoYsz8}Ui$P!79d@$lqmu7O@_oso@~t=ge*d% zOwP2{vNWkr-;`^NG?_zIbb7dP`4Vfv=V;VigZ8N#xd4MwOnX_Z{(lbE^`$(dHkX-%G#(<}AJGO_OC>C@X0 zL7YOWElI;zS-;sKd`1^u`iz1sm_wvB8Pd0IONmT|Y)f?D0!tntr4d8=N|V-FmWFjE z&?%>HGG`7orjv?UxET0`i7sQ#4qTVh`^>SO7i;M?=7Qr(U-}Z?8>-n9i#N%6V1he8 zf96hNWQ~xpF({NTeD`${;hy=Jzv$6$i45jajOV@9%aOGK7TOR*!DOpsl#bpI*u{c< z&2OP)YvLP>z!a6^kI)Ck>(2fH57M=P3ov>p_?Nztn{gTAdhpkO=j*p$^J4&*04&;Z z#bGb`-+LbU#HBkk5N7_O9e3U5=#!58)jvGoY#7GS6x!CRB}oqT0gMuT-*t0S*MTK)CU-M#h}N0C)~Ut=qVbyl_q^W1vr1$X@L zLT}=`+ar!X;w7&??hWrb{`K!U;-zoc{m94D8u_V3 zR3B$h2>TX7ogvSGG&u_08{BF2)AXQMUN5l9ZPbmZ}2WZf6y}zIJtge~ZgOP^oDB0oz(jn4? zOv6os8g6P*CPZ$|R^CGQ*@nB<@UHo-9+B~lo^@Q~8z+Zgd?U{{CS#_t`eu@cBn)os zbJ5^NS@+y?&!R2$dCfuJ*>e2rm;>Q|Ll@eT``72zmOA&sj5VpY1WW^E`|#P8=%3wo zZK+P6@Aqr{+zPHh40hw<{^1XQIL(#c=G)%(HhR5z_QL#L^emucc8nvAI3lyka9q|f z`j68DN^T6ePG$5mO5SQ@L^m@@^Abw>O4d&!c@k6fXrzfdqUoR9_MHv5N*TXU;ry6E z26BPvK=i1V?6|^weDUP+Fa3x4t(U9xtc%MT*UWX_7)>95%K<0-{Puh7Coh+R*+%z& z;?M2<=qLB2)0EEt`Y@>%7|o0|;5^gaGGjFOdaF0LxW;MrprG2Pl??7xSx9XFaBEdQpu9VG=?4bJNS@i zz4Y*x{_ow6dcyL3S0+-j;}$0Arzxgtte=N=ktH6);g{tbS(6;o|B!`uXh+<)+cAEckfAwKh&&%ELlui%rGC^^VeQbs;{mZ)Nq3wg~g zG|&9$+>tLloD>Mhxqy^7T@sfQC{|1??|Rp}wD0B?Yf3N0GMG)v4-{bzEyTs-nFMEH z7KY)_v+%mty^fb~H3Uht{1E8onEZ<9go?_C%^&{ohiO#2a|)qc9`&e4O_+~E7=AeV ziIFqioyDa>U)gy<*l$sZhu2Iz(|KpYu9Hsd1Z9x}JX&zUR9GkQY$FpL)}tioRbr8* z)++;LFKgmuP2XgYwGUtTI*^2VT{F8*0b^CeJ?yZw_*3O59>rP)x zfa#f|;mS*n_O&1+81onjh9&iwY_0V&wU8#dk;vMV$?MchRz$4@lbp!Zn+iesB!j%_ zwY*ZF8Z($&$mEwPbL(SEW{qzY5;6{DA3?+jbrB9^fTxYmGd}e~LHYenqoCX`O=xkb(a*dZJy)w_# zXWY~$zx0VrxzkHZuaix_%13hY+XX?^^kvS@VyrBW_n}Ip@I3b61jf@IkCm0)aXa}H zT}a7drY%We&1DvCNt=o1KA*A5YD+NCmXbkuF`EiO`ADB}<(un_h(1Y>&0xKJ&2PPs zTqz2`rM3hC`ebVIoO90M)%?NnzL^uB!i42d`mc=i76^9&|JsNAyKjW?&uPOK`Qu|%fhNr2A zdp0$i_8$T~=sAospXeao{P~M_+-c9lPiPz(7gmi$(S_++H<`B{%?j4mqb+t%*+FT) zn7<7%h$YVqG{|GvN@WrZp2K%tq8<(BfFS;Y#QF>&ujG_Qn7S_`eJ#s{wRl>gsfYI8 zD2t(fpow}I5Y)9d4AJbbc9CMl%&z1g`-*>yciv?Po45K~Us-p{^}~7|jly78B@c9J z63Kq+4=-51<~r+_?!47e2;dxQY7cM?cDhZjcDWp^OVu^m=-XQv5ti7VXMBP!7hbqXlWTF>g&`+QROjL(zr6r+ zYT!8#Hw#JwjRj;O>;0%$saIU|+!b?2%5XeswC25Xc93K^5e_}{P_F-(nN^z!Mpj(o z*;GWWrI(z@)JtX!OnI6?COH=C+LW4%p8DjMK6A?u={2PcRU)&zS^t?EGBCXCD~4hX zhDQ`1`p}2CeDayS=1&q?71v0v7I@WEn~ZPb$H=T1U=)rHlmd_5zLIy^X(wNOan0bl z3+jY%OeNoiaC(?{VU|s6Qnr_^(kK%lGtbo9HwC+hut+SC$jK&0X>$Ev-#GUxXYp?^ zxg=i7M^r+loSH2fo$=Ezdb$3I{k~mmoY;&xsNphmEN>5Xg0)vVO;XZo{b?(XO`6h) zDKN6(PH>lrZdx~7y7BrNZp}l(O{L!SPR+d_&satGLJs%a5IMe)_nSJ$H_|6XUmKS) zlp@>hoGVs?8&xM92ifFijIoqzlMHU;z{;W4gB$5?E1sWhi~8&Q#@1 z@Z9a7nbNm_`C7x8GxYrg3_^HS4h+bgV?(mFHy)QIADEx*a*GB|+>WS7eN;~G5W4N8`bmySQaE87zVWR(nTX!bi@QA`baG@`XP*}IhCWWGsnaqf@ zXMNevbrBPzaRHmMv_q^v3f;wk$EWS_Ad~@8#op#!3?|tuk>BxJ=GoE2G-mV45}LCn(J&6XPk2$V9tZAQ!jjK z#F-^e*%1qu+^q2hC&vpKuAfh0l?!{0qb+@yWb`^i zS+$-qB9{e>Z={=vP6W9yR)__;W=6xwQu@qO;M5mZvsusJMtUnW%m*bCHM8*am{!Cc z7n+Q&!HxEXE#-`h$xAY%EI=~Y1*P;gUYOJiS?l}SXU*Kl=6n!@N#E2YvQIAZfhyys zEQV?LYYZ9&Uh&X?Gp@d2goY<+opVGfC0}@fWN|xp#iAr7Zx#h~ozRwemzZa6XpFsg zJlk*h|JzAROYKpkC~BA3vo)($?M>B)iWS5jMeV&;P_@MNi8(Z$Hc@61j;+609yA_>166X&PqS%P^2&1icc` zs!^k~ladso@J#-Oc>mRwhXMOul}s~M)J0Rh_HT`0rgQJWSqCv zW6rb01O<8PTzXgQxwOBtQ;+Z>7deaaUD(kg$e)<5tz#40$e=|`!1&o)$|Gpl0MiGv z2u!cgy*wnRB@Nme#8sf_6F2KM4-1LeXZ{1iVV!&!yd)XzmYlD69#mZ*4SjMCF&jQY zsGL8&HUpSX6I6Xh_fqzeyZR~|X{N*s*G6#Z5xpm<^?b=F_i69KI?cbkv@dvQN%i;T zfnSA2zy<yE#Bj`6pb$u-q@D(~m{w!qs}5Wp8)PFAmFeY5GqHbPAV{T|1$c!X zu)suH{w=Pt8Vib7vlukjiOJFgh)a$>S?c@NVMp~?^GM+DIyt9l ziqw4(M=A(mm)n?Kqu`geTOA(?+UBuE=bL@zrzNWso5w9`=ph5?7y}$yu=i2vOjW3H z?d^JoW@Q!$ZR%G$7scb;&-YL-X{wMa%FjB6zi2s4;DgAewFDZ`m5&G+DsXOxo_m%c{UvP!R8DB%s(c#;JIAJi851jsZqab&2ziYkX)t5VMi>215h zOBQgazI6A~*Nwm&n_Zpx=DJ7w#cQtwc|eOHnqe$_+Drxh-+v&*4sk7RcLt02-sz{#WQ}lH@BodN!r_IuT+9_f>U`{)z1ECywPtSOxsgWDh>y8Unr?r1h*g zIdLa>V69g0BO#gI=?aFF+>2X?(Z)40#=ekSj@18>&vBl&Fk^72?qsge_wuN58o}X0 z2_gNH?5Hz;haE0`a|wQp`!6(vc^K(zuD?#^ul8uwy9E6RtBw`AZBE`=CRg2j;mIJM4pv((wlQsTy8=b*CB%J#qdD&Z*_W!eGl6#P zPbA`MlW$D#dbFfUOyHbLf`OaS=xyWxE|b(lRE83r)*o2cusIBg5)RR*N=|z3;(cle zDv;Covq6)_2Gav^dMN+kf9c|_Tu+G8Ny$_O7F86 zto9ato>AG>ISc%k#U#*tpE6OV?e%QCpZ62%FWF{70Y)i`F<@`f?QdJ5@c{fZwrI`b z6;67Zl>rCNhrW$Y@>PP{h~t4*mjeRd`L$*)!l6_K)QbZrzB!eE!KIaZzdsfW6l-nr z-2Gd*b!hesnQe|!Gh}u7=qql+v4dm0y1n9<9B2FPrfgNA!G|!M=$Rd;Ez-APRSkR; zPm4h7_J35n6bH5^=hgR!FOc3ltYz&Rn@Dw2{ltCQL1R#IWawr~S=(f|#r9mz0!^=N z{Kb)IJ~5$|6_4WNRS=@qR%K!qT8a?JRJvGr?K=`GZQ9wwgb3M>e(*z$!EFlqSZn;T zD;uSF7y8RPyqYU33zsqU(-gT8Jy7U{AVXc<%r~clktv_NI^H2Xmr(_y;G;zvwz-o- zb`zgI;OugvktM(5-D+=x`O!_Uq^*;Z1v&9L+VzcKfvQfp_92P zO>4#Ak;M}2yOa5dMCdI;9tdVdm%|M5v&jI5%?Z1EO}CdSw~B()b_ii2AmmgylUHaa z2?57Wnek{>LVLx1$J^GMSaEb{Rlq(n%Q1d)lhMSpKV@abVkKB=UiQjJIhMLKqiCdr z1ZnJXa_Yz#Rd0^OBT(Z`*dc;hpHQ@(R)4Axq2b-iZ!1j64(+Vl=;H8$%|H1 z`-eFZ)zmsFM4Zt|eTyUWudKPec^^!7Kz>cp@%T1Qv?Whj&m@hQ`rH znk#u0e1HF5?63`H7ur7)@?n%slC}akU#jYrITX@%z)~rMGG$1FewzNO)0& zD-3J&@ATai1_OR;>)aK{E^{>z;MG6~*$BvPKe-=fr#-smQ1;k1t%}+&;C|;)!=lxn zo)+Q!BGMm&%&(jjj!Oj^HQw{uYZ8APoI!}$JPvZB-(k(|TKC+h;t@S80Qw!|A5h>T z1^T`hnBQiH3rPqrm^dx%XmmSiEKS0QY|`~(((Wm@W1zr5{DKN@Z~9tr1URm-f}B}2#sM!}Vz}=C93$Vq1v>Hxa?>aj zdHb@sCIlr5iK*#6iKu2t7uSnPF4EwumxfIYg!^UU+Cpn)GOg7An~jF?7os0lD(WaH zX_bND3NqqXr;%^e>I-{byN=|F4Z$>$X4dgXnPW}`Ut3}1r!qsWU>}+MvdwZK587WL z-hB=o9p2gV6hvR$W?qAottQAn(_@@I4|LW6nf9H3c6m_+*p9_uUEWqjBQMfTGF9urG7G8Wp|er{UvcdLN>>BQzKcL27HPSf)Z?_OE3A=O=s1 z9dfXA#q?n&G+^ObkGlObC6&%;RTv_?wa&$;5efp0PN z|3<7^_P|&F8iP`jGPu^zAA8H+~wM{vPA6)Os9+=*i;zx9GwbeWG7tN)u=_ahRZC4#JEJ z(b|xn*Bzca&p>L&DwG;As{ZVso}gKqW(o+4)hWM{8iE8Gh7AW0GtmZDoQ_|#)NL^H z+{utl2syaOvQN$Ti$Ic93CqlYwA%5kWNGP!QIg@QLOIUy{3;TJSST|r2fZKWX4SLB ztxvt)8WR`AP(7=e@w9%kODSbF8Sy2(NAEl^At8g!Ah|5g0em>@^2h%9ZoQPepO288{mf88loZti-s_AhHc znz`g&qrm}&J6%5ADEC_}?Mm&P#e##tif|N=Q0oa}LU?MXk)HVJIEN4i@Dh@Yu&=Qp zLfB-~9ENgdtpU4P(va!+Ax+coqWIZ|7cYi4IIw^)}&{NL8kx0SMKtr!_! zh~IzgZ)_h60ls-!nDTTw^Q)FJ-l{gLJ8Muc4Bkim^zzz@F&Oie0!N}bND7&~hAG$z z@0J&!EmiO=1n#U<@VpRZb;qTatTm~<8DO}HpFQ$5*^4H%c~$FFm0xb7*8mMB*X&ci z&>r9|geXGQ98J;dC2pHttXi5@l*B&yE>&OP$=}94LYB4^N;|x)7j_`?McNEWznVW2 znY7>;J&eG8y{7aTW7@!RKzp@oc>eig^|bQbDJ-HJTbINgqAj~^IV3>9;*Uxi8?>T) zwGeRsA@nr|4hL$=%LJ;QnzA=!Mj-eMK^Jz}7eF*=uB0EXU5k0eimg&s!s4y1$z3Z3 zhV_4p!?_~(+Ga|r^1Oy~#{s$0eyJiy99-UZ0wzX?Fg2z^s+yVCa=FlFpZ}<;Wsi=D z|8nM26#xB5@%qy1fHRP|OQe3iLi1{=1*=Ord}Y(e9HP*jR5 zK)Ycf4&V!}sl8MzVyH-5zX!I=f5{CSy;$qI0;&EKpxvmt$&pOPYq0S7VJnw< zR%Q3_D&4(N)4){5?hY~s8i_Bxa*Ao#B*c*xf7m;yz^wVpa;!(F*Oc3=R>Q9R)^TMJ z)r#Azw-g%mgfYqPV3UFA?1Uzo(0DH!w(co2f&z%S^zlft7A3H676HQuud6?kg63sQSO-DKFW79%>zpAVFdNRU)(W9>LRR`bi{_N_k|W>N{vQClTZ z4gW0WS2%o~q&BlTpJ)0fS$Eg|PN?av>>+_=k+G2cu;sRRdB&`Vu!21=& zW|zqUqtFlIu`4BBV?Y#s0KG9>xTW!UJ*=(q)g;yZV?mQ#q9h>=SZSJmn+T}wt&qm$ zjG13`PtuG7djt0U!^`9?N-!>YnR5(1>A0h>q>EgR{3?5J*8}v$<}O>WKVK1x+AU3y zIr$l)lBV2ru^$8DlSJ+C`P!qAE%ZmncPj%%Z}v^^PtkJhPUMj^3Qq^_=h6xdA(}M% zDvaO7l$8WCoKJ1}U-<=uvPcZ}XXZCUMxW2KLPX$S4*oUBzHr=f)sd(B?AKila| zTMSTAj%ozP4XKG2>}H!?c-kf{=^b}h9i!R8Q9{H7u9f#W;gf0Nqy6`?)2}LAG9S*~ z2w&iPe^J<=%EEe~@s?;la`8O^01Qt^9c&$lQ2S~$oLNLK*L&DI)_V$~S+dS+tOBAD zgf^dDEV4b(8!0XonwTke>5XCSni(1quh(1RM-%M87#+6?G7hmL4j_`o`uLWjCf0g0 zNKed4@9Fj=dw{t#0+(dbfSuHSI-s(K;G!HL-~9Liale^!Y5yZD>51@Pqp-~?FQ}xf z%SZLi`OJ$cNi-K|WUiP>;6!6Z)+Wh-8zMagX!wy1%~{Xv==<}Ssp2?A_p-LgWIaNK z{{NYVHuH0|&MPUC?&NB#k1^|3>SC^`?Xg+sRxrz_bNv}JJy!Ll8n@neY{opZ=T?-` zvyvudruT4J>EE<}ldP#kh5W9&Utb8~9%my-7wpCB!^QPB(l`$oG;%FsFyxh^&I&$y z%u_C=Bmz9Jp-EbhXy}8a$CdN^T9voGBlhoM!9$(wSEAvz`0D3M9lp82?l|=@V_r$) zXm7T-J;|)CEkgWtE@8Dgiqgc<_}7%xEw58P0o{nHD&+%I+}9M@RSXnax%}J z_;!R$j9GPZu~o&!0z13ELFA^BqH1W*Y`QcaPQ=~JuN{&z@rL)={B_&PRomX*fyT$n zyfm8U(X1f5*X*u?xU&k-HGt6TCvA)aR74=`wR5liE}rDKKf7Z7w$+MwoX+d<)B!f} zW@c@40#_Vuq@BCLX*;LNV3<@4;7YD*-$WQ7IK*h|V8hP{$E*c-Oh5R2JINv6X-#cpDH z!YmV!o{gBlOmB7zQlnVhpTEmDZak~>SEZzesfIW$pP%ygv;k4sy!Msi=;RY#=`b&8 z!p+5UbKwNY^knW`Hn#2SWM~YYlbOdXeVVEF!lp|3`}DU{g&)}32n+|fE$@52a@@3N zsw6l#I@|Xv_Z9LSZnb{td%9V#!uO+Vnng77ve=>o^8Ccm^KjCCqZ@lyigGgY>F;Iq zUpSUJGp)zo$Am6Ie%FDj+FGjx9;5qOp2Od+SN|k2Aug@heO-|)`PB1MCFq@o=nKbe|xsX)BRa2-@w{7-UBXl-xubwG?V=-3LAoRoJ~ zkuXsp-PY6RxD#cG|*QXg5}LXdvd z1*7}Vl%`k-cfT^vqz@};3c$p3dp=OgjU>?`kWD|-td3o$6|UX~vrtEB@Ea0%2q($R z)o|CDfF>F?$))qz%`P(mZFau@z2e{_1xO~Zo<7TF9cJz7YfnRznUd2=*dXp1yM&dC$g zxZ9=?m3!?f?n|K2eyH!w+}+n@MFY{Zv!@!ukrKuUZyQmXe0Ob({(DZ29h?1v53XLo zn#SpuLn}r1()o`CLN@VVezSpn5GaocuRD#Oz4tPc#6mWQnb|Jk{QM1@$yg8i^K=otL+auXws9EEW*fcDC|wpubA&E_yL; zIqSFYla{8VuWwSUEG=YHmxu!PVyLnKR*pu!%`dAq%y+2);6=HtMARSuN1J8wD&_bW zT%ei!<3}YgK&m5r(U=Ggu?nH!AU5gAlA}raa(^OvSUNkE~rd}^0P zHksiT)Jy=J^!7MfKb~KO|1C)fywSie=tSds~9t4 z<_68@za*wY7l|8U9xQFuE+*{sUR_Bh;k!MUOUnytJ*{w(+9}BwcRFAkW6h42Oga2) zocT#cJ99DBx4l4bI-WsNP1P_Apr_;GB+LwQ`lFcTn%X#P()b&>$Y=MxH!)j(N(Yv6 zx%eF9)N;|Xg#>k2RC~5;_(-nthvHDrj~nW|iLH-fC0xg?$hzHI4t&m4mOZC4a+@~R zTxXY^M_Oa@6KJDL%-VMnaNPo>5;A)OZXD|zDn?VphUBqzMNixU7NFJ2svjClp2sUV ze8~6P$Q<8%WR#_sbA2LW?siu6++&hvHHW*16B3t_Q0Kf2H*f!KhHrMUF)6a-s=>~% z5wsz(=R3wUsFWem(EJMWB&|ePS$it3>cH>OuB-2-gW?CjGwmxSAq37*BX+d{uu8J{ zkDczd%b5k7xVDzss?QTt)ENX;Yf&$n41L7hy0!MB9q&zpF79)FdN+GA+kPfA#ED$L z1p5`~7e^&1FX%Uf;OKLC0C~+jw6$-SNuwvguOj)=Cy3GR8(c9m4M;$Y7w?as_i8}L zalpMZu98WKAgFm9qWq+i(9n{+PSHj>2t0Az#oh~LrXLcdn{C3Ps;Z2Gkyf8;WARM}jwG!A}}2*{^;mW~S*0m~nt z!_fwxt;!OTTtE0`8;54$aj2U21O?Xj{IQfNPl9CD>!b++_7xfu4mGH3D=NI@5?-;uC7T1(;HpoLILIpYQ=8wq)$TSt?I{hXLxN?xOSTSI^b za3(kbC#C~%lJnVZZWKGsijnxO=Hc|G(c)kC<0wiV<7LSNzO?(4p{F9aqbYNmkdoJ| z(YODf`+`STBC9;W*wjfbhNd(2FcQ!sXpFS8T|1z@buBJWR^E+qz(%(2XlZ;FR{m1J z?eb1c%C!bkAK3^y0KLp_J5ZhHy|;=l4%(^V3Ol*I7TR@eX`<1FyUkTsU7kE1N^Kq> zb!@{d$egd^3orWi$FtdUFv=-dx$SJ z35dGwssGd`y{eU)V9RlxYL8~v`}{=^92u(Fd0*!f3o{_{P4I)kLsz}&vV)8L1JkRe zyIcPkw|O5I@8GlWoKqZ?!SqzOtqwG+<{C1%@7{1j?Ab$haMM_PP=nnQF@d1=DuzdQ zsCT6qX~s>g>N60ldM77GRCGvQkriLp>PQx=!Ygis7njRp)ZEYKI~Y6kcY;`SE`J8P zLw|_#tS-nbPKXaN;TrsGzoy0Vj8rR>kncMY4K_U_NwQ?-Edm@FJ~iN^wYSkY5UkpU zhYFF2DmOauc>>oF&V9mDXk_V<!(FE3E-OBCpCO3qOd-nCosCWlZNe$Xkx!@IUziCHNd%zQPz@{ z^C!c9n|`T=EH*Vx{4Yt4vu2tkK2VC5AY4fC6y({E>Eb=~5?9bI$*@{27F%Ltv`Z)+ zPIa(#wwF*b6_e;zvX8+P_)yNm%H5!8(1<)TD(nM&=WB<5huh{T=N&UCGpf&p~M}Zrs(l zqCz!bu_zj>m8;tSym+q=_JvcomXzPXGezQ!Z;~%rkA7Zk;ie{siEGIU8#g_deoMDX z@e7x2c6v4~=HUAmZGS?vYwX8QzS0Adc5FQj-Gwyl zwJw-R?5z_{R=OEn=wB~5&NaSY(Va*EdAO|1Wq-+z5xM?5v*5S{-jecI)ZI0Yx@c%< z>WpT+YgqF9eX4KjK6E=tDGDA~7y}vy`-_NKb)?gAZ!(BkzP>Qs5 z>mRdB<3L>~|B9hls*ObD>o4sMs5!7rmHTQ+Z}`9Ob%G8@WK;)FUz$}nl{c&&M z=MmWfz9Tl+(q^h1+iU6APcs2S-u?sSjXmhrt%9--1F{o%r_av_d<EHjdnaf^@PLhz$2H*&YU<~fB)uY*m* z`7;u_}#pt*mW?iKNJ7YhJmO8QJO|ZfuvsC-E z_a!nlhG2NMrKniO?-WH3%pyHKRh!#9S2hV(u#<*a2E$YmUJ`jY6j=xp^*TM#ai00^ z+nTJyK`xsxO!>tqCT-(VbSd{~0r|}*I|4@`00KzmVjtnl(&NK_N^hLu%Oajy`}4i@ zO}u&aKrhvHW0~q?C75hS)MLh&#O3y-5SqLxVrhKNPM?lN%{;9nq~#pE`p0)ok(94p zYr`Du3a`G*jDT+d<5kB@2Q>IA?+xY44g;Z{d0QRVmJAs0uWYY+VuuZfwvHx|3z47}1N%Zf{~b>Q3uiij zD*dDeaf61nek|8j2Ku}{XtZymy4-3Q8j~b~ydYQ>CpYuULVb)fIob1lmeZ9|F0JOBvE=|#f4KDE&#zM%uW~~=x5q}6lroxh|^~=H|Ohwo~B}3T9W-y z6_SmX4hX(qb`e(Qz~aAq`#>HfoZ=6cC6}P;;DlB7dIcGdDv1hP*o?L;Xe2g$&HzvGf#fAy-y&+t4nJ6s^5Zwqj?A3 zE5T$3{h5@rGmd1@K!;9NDPnYgPL^VD%M`aPfu7x4>UFH-AxI+FPQ}?5&n@k_dLvGl zXtxj6Uz4T3Pi4f#*ARNDRC6*b>z!Od$ySqT9Prxusa8_kOjU!mt{X$gE{*RpJFkyc@1nPzzH?!y{K6o3=SeEl0|0#TiqZrD}mfM9Fx4I^(qRv-ubMPJ+M?@Wu1JGz7yxbUgi@Kqi~{x!&jU0t6gs@;8j z`r7e^^Xoi~M*G+2>LkVCf`h>X53)$>KMl@3;QEybx0ip}W^g@`*S_*+m)MRZ-MWes zxEl$NR9DS^km57Pkc)e=TWh`lJsTN{#OXn94K~(uHpud~4N+2LvB=8JR643Ld8F_p zaJ-*x9y1PnzMRDAA2;?_sy&X?AeHdiP@&g2(CV20LSRnv7P^#Hk^a6r&*y7 zT2$?q7zub&a40TesoLc4%9I&Jq&}w+bHz8 zGUhs9^m;Xfx2@wlOllitBtUDE8R&FboX0Oy9YJ5apjI1l&ZEY2Ych(R%P)^^Y_n7T9nv8WV&`=1kM(l4v>R-aT@&Wo zk2&;Zg~MrV_4or^2W;G^v}@}acgPiU$R$Q5J|qeNScB4 zwx^p*5C0bnpijxlMkCNNuLyNye@vG5z%Nr-U!IP=qraL6=C%WK@25yF41zSWPKsV+ zr9miwpFa-o?@5^LavqB{PpQ`FybTkdDwaj>LJfDGxqOU2(f$#%mzaq=V>z87fWR?O z%=gdFzqxjmGBq9I)w(F_<`aB`hTzq)4m1u@BtPh4V+#Yz!A#EtazBjY1yAVzEi0>- z`n)RqY-c2qAr&H|6rsI+kshpx8~E2 zspsCnzw>;#`(9@^y<&_?=^K7(A%aP|dydi+C5uC6=Q=svc?%<`$qMpDl_MSiZ-a~f zQd9A4<)XK6xbqW~p@0uR1zi%ly$Ai>Q;#OkpBY6kT*K$1nIw8KMN&LxJx%#C;O-{~ zD{ZyHGqY^^We=xr=V?mx4qJKvb_fAoMFS;Kyx_WQ!BdH?WjO@)F3PN!qHgU?Cu0g+}9g~AtAzc#+i1;dQxj@fo<@5nsuJgwF#%z<^rYb6> zbjD-7F{Er>OC0o z(t@g=j!!HCe&spSpIC2~pn6s~dfGRG;6ydmCGGOI+T_YnbwtOr?39AZb=W$#!c2$r zj5t4as6)tYZLC$kpMT(+`1m?l=L3>0;n>Eep5Z_sr((a2_c^aW?VAL%u@-m5W9`v6l`b1cy)$AtPhZ%Kw(r(`w9Ms!F>9u2$qbfVj=kv|3f3Zner!5bUwnLMq=teVF{=Zd4m8%bq zRe*=`(2t&1*?tQ1U86}-mg$=dAI6~V*V^12UsgyT_RkW zsWmb_9s_KYfbav{Z_0&yOi2&Ca4W6eSDszDc%mU`iQRz=ixmXuogSJg#Wql3F+SHE zqC)5qPsb#=0xIKv!?tJBKND9rt+Dbh&8l!u57e-Xm|=bid$Y}7REwBC^v&$qCh%h_o%m|C{300VhaejL3S`Ha zks^`WBTgTMd=c?AnihSCPN(~xtW8{!XBA8TLKJ!s{J(!C)dt+Go&MLXasO)EMo=JN ziF|m3uBI3$W<;S(``|hQymL!X%sYCZY0HV5okpe$0{oIztsUNjj%+-I6N39tgH{4I zST||bEM09(k{o^CS!YHb*IFfWH}CM-Aw$0}r&CT?&Vw6_C^QL!;1sBUKDmKQT&!Cu z=@m-DF@5Q6kq2>?kmGQT=6w9VUj27YZi&}>>aig0z;Mf*Pl2NJiE_}RMuCjzQq|nP zVk@{^q`yna?{xL($rMY}uvS-Fy0ynzbng+zhxtns+u6J%vDD*BG+6X!U-6)!LQx7? zUVuA$E_838Y)bzua*}s>Yiw>S^}m5j`qCMwljQzqK*#T%96Dt$gz2MAf0C;Z&~Z0q ze`eWle{R6zJ;$)$t%dsWQ9JOdn!5>n&~PYGQO(x|*4pJmcQOaVD~x)T@cmv3vXGW=~*%s*#3=TgdDWwXM5KVB@0-pmTp z+(y*A-S|v(?siFE<9YD?{(<)Y%#4ijCP3s7S&H+ktM&H*f!4#LLoYr&#*%7<6iMNF z+;k!{qtRphJ}JpsJ*9us#jUg~2*T|45|;juD7h4D@n%vjhPg=IqCGrmK4A z8y%xxpPFV7$kSZ1 zeKGcjx~_BFL!|t78GJV}hB!g<{^ip&DI%~w5AA5xK<)A$f+?movl&74@BA=o_Wp1& zog#mmewa5sJkno4g;={)GmybRB^KC}NIb>Std?r+*gfJ$g(hVd?6PErL&J_{Uj7jomG#y>C7x zEn_(Eji!^7nl|{-M1>Aya zy0-CO)kJ|$#ZfWrP6l33bcEx|?=vnL_e9+(RiYc91K+~$IhdpfQ4H25bWZ23R^#Z< zn*}#rRcke8J#(?626g#Y`;E!jZRnz`%JxgU-b;$a8|+F)PPN+P8NSSUV7qns-_D1` zi?|yYWPWxd&AjrVa@+Sk_^9+pYpRrOHj_u=J(>uQjw@4%DZn@Hr&Byi1)NPTO;2j9 zLd4oyR<~7a1lnlWlc34GPP66#ZJX~O&181qWS75k&#_L27ZAax^*?iB?cjr%Ju#nf z=J*wnVVQBB0l|63lWcYwI7+v))9Vi^us{3_bSQ(kA5-6~mPWpANVAR)< z2zhHV{sI!N-38uB8fAPVkUOXIK^jH}kMtEs@J4(Z5pQEu{8LqBf&#{t}8HV-C;G%TMcFZK~ zi$~wOcK_w@L@-6VknEk}Ky6TLVd8PMP7A3U?!VTDD3AvCw_PwHIoHH^1^VIcrzrnx z|6PmARsyB3X7d>K0NEf$^{~5910X^xKNg)@|Pv_H$C`42Squ~o3-KV3P4 zZ}nr(|3S7IMVG6xeU+Qo)x^Nv~BO-o3bB=+C>C|8jQ zHbT32D}PEuYE+nY;OYmWdAQ}B{ooS*OTyCoN9^gT1e-KXDo3wgiP1k=O{FE@*wBF8 z9#3tv5F>$exXcxxg1+SA-yEO)3sA=y2-|kxXL$Obg0aZ5XZ{l<2CwxY9Dv|LK7@?z(2zXcZ0~2ZD(wt6m>(;127S$m<++ z;;LC*=1={eXEm`6D8Ax>*DHg2S*ph;*5Vg#wlc$xV*DMIA4Z}0$c}6Y6<_E9=PuT4 z6BkoH&d8of^Zz4tG^~}2f-Is=JU>%pYg-J(@kyOBXJCv%AIOYP_r^8&-g*3xtFoTz z(hfXWJW@0oe^G%P-%fz!C{$R7=Ua!9sf(_P1e}P#(mKWS`@5v}!Nh@QKX$#$T@b8c zbS@wA?^3FKduR7aM%eXC+{P7Tu2+eb|7*ll7uz5Y;0tYBBMvzKZ8Y@C^QbrKfRUti za*eC1)x`>VI&j)OlD!F>vsK9n?2YJ`{?dB-JEH#u<7XU>o!$d_PzlYhPqeJ4<{0>R zy3>IQc%q%*>t1_D_xM6*_eejxLb8}zd1KG9NA;FzkH1va|FR&+@)#Y%K<#R>w`;Tm z#G3JB>?sKC0;#3SfL5a=Cl-qMy`&t zs&tze=sSOZZ@U#im1U{89fTzfP{ZP*Tk&YbzL{K|o#th=QLG1Du^cD+e);L#|2;O~ zVOp;DxMo}xt~Bl_zr<{oll0M1T_q|blZ`H4uN-;)|4g+J=kKn-SxOKAj|BgZrB9Bg z&@oZEv8GulA@%IN8nf@dJM8LC6rk}Lf8o7f7AT^I3oiE7AjQ~j-yK)H#9UPt1h+o3 zsLKDCtCiu4(#a1({U^t2^gwhioTcpQF_p=Wbk$w*_iQ5~ypL0;5oNNOfdRY5E8;l( z&wr3ck+pe91SQ|uJcuvgM7C}V_rK2zV;H<3(f&c=zS6WK^R$4~ z22wo224H-#&9-iPk!-+e{njW?!$RkbySaQu>^NWd8}A(WqSvy__k!I@B+3x{DB1_VmLw56RxMRhciWZj7$G5^nZkb zEd@c7GW;gJH|jDsZU=T7YdC&~ur7qOA1BGvky=xbu4}^KH?08T4P*2deQJby8)(VC zS{ZZT-q6yvt-!aI^s8O%chvo*y0g@!gJT+l6KrEFWNyu3()2o8oefel zngEZaL!k(?MRlCbncLD?=siHhZJNJWAT>@lF$Dq=XKvBOY}0gD^7KOWK9s7sMb#A3 zWTuV{54gY7=6hyzb|Dc(BEr58Bb-AIY@UH$ikg1DNU+HM!GBI!E82_QcJUAY6)Boc z%|2=9WT-#{qXt&Myy1eU+@;^F3(SMO{8)`n#$C(bJd!EP6qB z=Vp1Y9^Ag<^_wV+!L1WaEPb7pd%T^0;1U08g5*w(@jCgd>%Ki4TiL<@IS6L(UGb6J z9aob2XF>ep%V=fu#*DG~VgJF!UPv15W)v%Nv#24tr^a`K&i0@+@M}C=(C^selXhOV z*3T@*<~sSJ;Q{DIC9kvvu6BWQnlS1AU1{MKGIUHDox8F$i5^KY8Oq8fx|}&oHyVC+tj#w*Iq&=2@+iU< zjFI;s_a7?VioC@9{!=IK4e!zXW!a?FIS;<|?9V~|U@Np$tky37Sin)b^u;b^H5z~v zJ^N$HweiD>@hDtr%B4?N-}LmKQ@~XYwiTEF*@((=YB#icy8NHQ&EIbi$+z{Gy-%u~ z8a=VDIFi=QK|nw+n=Oa@R;Xy9buGR}r5J z8@4t9{LlvQN^Sg{xBffI6Tyvg@WfM>I><5_vUO}&tp6?2e$#e`HJ8Q?i?p6p$$%;0 zYxsU)jEioQRJ@HCDR#frwA-sI^W#md>0!k|_4*K%-)=b}C5Wawp!R46o8xgrbt?SbbewMCsSVxODMB55|`Z`1Ut^bjxwy6HpeI|#R~+) zQdwPpX+E5IiPEuIQIu~f|L66ppCE$TqYBg6KCH!VSyuRfq+e@0JCiA>M*MxS?*6tz zGc%H#Lcl!*tS?qX3magX^wncc5?_Z8f(3BzowJCI>rK9-s2`v49RIwR0n6HG0Kody zGu0W36Bbp2(7nVu?ez8kMi<=Y#96vOqIwKaw>~SBotfatKy#9uEU9#0f9R_i6%0Tk zjhzZ-V4bg_m?C#fLmDH1YheH}IJ-U&8v3R+= z@-u}&wQBki1{n+=z1XBXNR&g$ArzpbNx5jBz2zbzvAHPU-q#Y96{nzv2b|;vfkF`R z(81?B9?BZx=z^lbmifpc)~EbS>O(ZWQ&gjk&Evk4@`AePp0NB_@xd6k+mKY5a*jGU z(#SM^x|DLG-^yd0h7NaNvVM3zADkg>jbdTE7!YYcdR}a>VdSKscNpu@io*5&8vVvp zRz5q%?NR+5<|+xyX+$ktpjO+E*JdH^yhS8{rb46tvk$=~^=V}8{7QMSXl>s-PmO~2 zWQYcLcQ}7`j&hUgR)cl643P=;0i7~7o;i1Q2LSB2=8p~5Omv4~HaK&ohfVw)4dK<< zj~YhIdHoKtX5P2P0avb0moj%JqYMp0gA`n~c1O3`NE&Du_U0^V^cG8(Q1aV5*g={G*7y|dd5&~W<(@QJ|Q*>LdPBjfin z?|(*pNzP0K)RyEeb&1576p!^voZgUA?XZ9FP5H#JTWo+`3AiJ5`+OpPB3q+~@gu_* zmyxAfa`x@OF`aSGv6e65ZXeCRkMk89ZQ*MDu(&N`2UkG6g3`CVMd#ikh7CWkIeeli`&I?0J0;)w5zGQ>l)+y>br~wO&pCrjpdV*Ls03($Gf(Izh~a&>tBbJ> zy>?f)#z=fXrsl9whZ1YvY_F$4>85B#Idg6!EM2YVddh=hdb=P^DA!m|Ge$jJdk(sI z-1w8oN36)GZ~d3brNxZy-q4Xa!)RtcN?x#;Guv)8$!`Kk@-a|+cs?QpH~2_D3%DN> zkBe>k^9?n9vG-#}TbzMe9p^XH%(-WIaiIjq@8Gy#Y0&@0*n9u8{r&&L?^pXmt1VSj zsnMFXON_+cs%GsK+6HM+8hfj%+G$ZcXluqMR?Q9sA-0HFV$T{eVqMR^zn|;#!*#oE zx9j=?cs|cL&y#Z=_w|^#Q}~u)4Q#+)(j}1X9G#X!8OvkK+OJ;n#0eD-UPTCHCQT~P zK|Aa;iu%*ovs9|K#hSKSmbVTeuYskS5;*I`aN4`j+P9qw1z36!5s`tLnEv~Rco9fqIE^S-ecYhlhmKgym?i7s$cMxp~5=On0pvsmb16ys>B&B zUw)f5al5qXV>cM1wem4IU7ud7!BL#jZ7ugqk7~X}tTRc61KaJueB3_l&%IRGdixch zuC(tUV^+?(*P}Hbq9BWtuSJy%dG%edW-7$Nz-A@|tr^eWw!P%GRr02e z{4o(DXuE8raP)(|TsPkNs=EC{hVs}q&cf9%&R;UGUtuz?2fFO(H@N@prdadhE6#Jx zBag3$QPzt(@`~Bl??LiUN;}x#eCI%mHGLOW>p{Pnz3fAvPw#hj$F|eOFo6fbRf}AN znNvlpHMl*`(!)%@zU1XEU5mEd)HpaI3c~_Rj4reWhRx_=9B&X{wfD^t__*GIll!{M zw(O^)nqpd_L{3GJ4Ma{%X|)q&5Ri|)f?US2j#oRq;VK+v<(k8laK3g*0qVa1ppM}! zYM)%Z%GS3*Ht2hy)MA_cd9Zi#pf0dok-q4@uK}j$l>4Z*kZbQPjMH{yHq?h4J5R-V zF~D;$@ycR5`}JA|4j#1gQ@Ax?b%XY^-1h0L!rT|7XL+=lzulGh9C@_agAOl__ONE= zy1dNd;hX-{Ua#(nTKZ8pr^$ft5^Oy-cmIw5kxq2|7#hm=_1d#FXmHE-VZrMGhi(SL zYWFH)zvwJG1@wDu&ARbq%;R@%f#tk^u8resRBOj5>drVkknkPjoi8=?@_)Oh~N ze$#vSTaAzVyY;g6$JC2D$K3*_P2zKPm3ymWVY?6?(!y}^jz*k`Ksw%osdR00h+g|s z(3%aTzGchQr}tQggQrw7vAm}Z;K=Uc5`~&?Qr(vNTKnqac?r;~9O zzDKg2XXinz$seSk;#^+ne?NvZBT~|mu4pfprlWs}Ap@^8J#-Qut9Exc7@|+#_-a{l zhb$5GVvEl<4CW5y`?-VV+RnipF9;|L4)owXaK*)-E#`i?Rj3+gU+u_e933NJ+JOvT1UH zohZUbU<-+=-&a@iUcpLuTWX@r(4EBqLU|0l*yx>pgU@NR{}vzQRoH@09=<#e8#<$( z>F?!!K?ou(E-Rn*CS4rF5&ap9n|@bsN4eiEc-e$4Z#o^x{Xm)Q@4d|-*039xbJ3aq zL214f-@5(Dvox2>(<3PSQR|`; znoRN1l%6NS^0>?$?d5orKq_Y>f1LA$eQd+Glqt%zxze6Jk|E{`P)y-K$p+@u|s`l290$F09%W4 zE<1fgqoATJ$t}HC_PZ?Qeoc>2L#n`|*QLXX`>_J#QRXn2dw*C$=2&X(%|quXk7ENy z%R{_1yB1T$#RT$v8 z!|Q3_CQNFRr|oKF&|1T_`VpW=+BJzc>(=Ag<)FVu3ETaaV)Drhb;y(;tvoVeD(0IC zszim!ds6R*R3BOe^uIT#4+Qd5CUIgrn?BTDBIJ-h4Nu9VLN(o&IG9e-u)H+vkLB)F zwi9-_z`5GySM;Di$Fb!NYn^x7$SvziL1Pw52OXojvn>vr+j$h-2bY&|Tb1Es3!Ljs zJUcgQQ~Q(VHErh}*WTwWpQ#-<=h>3tk z>z9*PUsgRjG!bjrH-;AdR8RVu%PIW_)^9#07x7ZF9C_WVkM4|em0?%(xw3WJJ$(r* z&OT5=A1783v?!dJxVR-Fb2;?Snmr+^va=PrvpL1SV&;sZ`D&u!cDlA<-JsP~Sb$l` zLnf@!cI&;%*m&d)9Eo(K1y3(vk+5#4D3U4vd|Jz2ocb=@zBu3DgU(s$a~bk*KUf*F zDg+@ta?f5Bg^ckQ67ppQMiw4)iR-sejh3BfGb?dkZ>4e+PZ|DzKq4hDim0BAuEB39 zz;oJr(CWNR@KJEij3rQVE#nVRH{lf{hWrpDF}B=RSEI^ca;K`jc+GfTzYdwNr5|Bt z%zeE}U<-UE0B3(06r-o#i3Vzx)0vh1Z;rt~-a)$v$rm5w9Qv-Z5fiS!)&?Mf48E_p z7fgnOe7s(*=~+B-;@(O$h<{popW3ladl0OscoLIwHl+YOq2k+lBq$*Oz9qcl1}6_ zaP-L&CK&%9;tX3f4BeD?c=~acURY;Tm;YNW)~8rITL-G8flS~w`4JxtHgM!d5`_UC zDP&Hl+b-GcyO1Pnraeqg|7>&&;FmIap6|8<^L&MtS*OF8hV_{Y&o#Ww7ySUhEezv zMn9rbgEwBJrITB^_e${VV%qMi%4LhE`cSU+#Tm0wQ%7d7d_%r)OhXJYn=ak;i4$!e z8|+|nj_>Omj@6!8Q(&3Q0cMD(SkigC-)U_gFYlXT5OR?$o@<&vE=<<9$PaOVPgW+;-+6bVY3%G_8uEgA`#Snem5)p4ziMyM`MG1?1CO!MZK_Y z_M6H5aBg4NYf#XzL?aurhGS9KOVDn;$itLX62#;$_H#Ll8hq)xSZ5m+c3oKPl|u0n z+D&WG+8m7+N`ZD~XLoA?=X_fWajz9lD+3|5Q*Q2{*T-m<_3l>((8iat2^CEXa!^&6 zKYs>)ss9a`SF|*ieY5swAB%5aTJwtL#|4PNSBhD8b2E?R5Wi*z??GU}vw0~X(|l`$ zSF%yr3tnQ>m|zXvyyorT4=VQiH|Nz(yhL_q0=LhGmCuIr)sf9F2|-!tFYZ0;AxCpg zc0j7A(yWFRH11dKgFa&$Y8>lI-<$KVcsIEhYYl@x@-la3#@H<&1_hYTZUIS7lO$SG z13Cum0;s@~)LWfm)oGdr#3Yy$k4KoHXnZ%&--MtHtBBHco5r3kv=c7}|8r>}2R-G5 z!x6A?qrOt0V@eck7X%25ewaG}QH zSWmE1H4AyT46{&Qb$(~2?r6Yc;5`OIFM}9fPT_^8?h_B7K`^f!R;E$Of)J!^V8XUY zNxW56O;A@hff8upL@3;)n4*=3d^;LoXCDXLA0>lJ2T7) z4AadfkJD^bO}a8@l&^t|0_*UzSxQY??-do+{sgWQ?8MLU(y76RjaYUM_qmj_S$GZ0 zzxisN*|PX-h{jY)y*?F)C_EYk=2wR84qs)q)ur0&{&Fe3#jiPV?eJ*CP`Pxi!^?n5 z&^}~>uUBqMar^ii9_5}D65YdrshA$xQ%j1DLO)fiH-)W!658cx>ah_z9O(i;E=9UP z37IwvN84FhRt1T$;C%GxJKMUCYS4eZDs|H#yInko<1liG^UwV4S*d5a`~dtJUiLZm zMf1hkHUO_)8F9}D+Rx=u1mdanpWbKqHYDCGRH9-Ny;AQ4J#Rg1sVMdeiM$$N`c(P@ za+3SOy~u`RtpJJX>J$lbV9;8UJBK-W-cDOcAQJdQSTPj{Co7x?Oh|8aeL~P|}oL)ClK{4iksyJ;sok|?8 z7VT(WznFJaKKb}-i)@&4F>$do#^_X$FXMU?*&n>w4#EfjAVu~MJNwDVHS_UXAg=80ak${cL1?qR z)$xeAum}Y!6dJnpts-=8*s>w6K8baJK4ZZj&I79`RLzO5JG{S{Z>Qy4pKiLxvWJxt z%i`MEFH}AQIy{vskA(*(kET90P&#&)c=)KjD!(Xqm}|c!k)p~t==D6R{IYc|FH4bq z(x_pIpN0LC<6LD5PjHh4%eMwVU>L+lm$m=JL?K4`&gqEZ*@*n?u|c8fQA&Mt{%XF7 z^4bFOvDw3-Z~vx(J_Fngj$i3{(8}IFDpQ!H(l+*v_0)`Rp8j6c`ba%p`j{crL0>8J zN?6dm)%pe#N)FHn&?;Ww0bhkFEIEBcA> z!>#QOl3cIT)SmKIm=)IFFd^4}>_mq=boWWiwb4E+x6M4{GNEP58!$aU!SwBYzI9&z zX8aOOM}E^kQ!ofP6hE_Kh;E zi>x^`dz?!w6vDB$l9>rr8Uk^_7md09=S6TfVf)IQ7SV4HOSL)3?l}i(7bhfmAAQ^n_w&f4oQsBz^`nldwUsY1$fV=c z?&LE}n{qcR2Uwf4fnri|X}ZIzOj+Tiiay}5P^m)v%Zp=@=taYkXgE-TEFi)>7O!)bw(_D zgQ5?g7bmDU19>*vMNg2TTNYy!bCPAiQG*gRT}KJ~eCiQ>eFnyNlysPwB!;TZR{ZU# zfprhu+sl5^QU40X)#cgRX_-pfV~zihCZntx9Sm>iBauiOZ3%$R>8qmTc-z-MsDxZc zYFsox}uha~L=EYTRTX>AAB>XR5wwv0 zl_EO&llgRB{QRrI`q7_AWmosz?ZYHv@admlq4NSEZx`%Nf3mb1wUu)-oUxTaZYBH# za(!q2b|pQRi9z(mjj5i#Z*C1APJ+%)9$$C;6E@G2NI%G#xF>C=*(x3FAc`4=tq*_# zC3O2X43<8PJEM~t61+4OyrVv@rcH7V`{A!6*5iaL2~4%wOAU0G{a?t1n{Zbo0~Rua zKtb;eGb4QZ)AWP}?n7e8%at<`)p;+@gFUvZNnqneW+Mgfn1XxLPNK)aQ!3^EKc4d7 zkL9g&=W=pY`yg2=K2SGn$66B045oEn@>~6=w+7b>cFKR)bY-O*!$R@0fjcmz)8Z*G zKT)5l&5unSOJ z#lQE%ww*xE>be{WwukY!3QE^Zw|vz7ZPq@G+kd&F460U&WRCpI87*K`n0J#?=> z^!2d5M=f@%2?ap!E(GXUkaPZaNAo-f7n2E$=w6|SKi``jany!@933FyHj)Nd^HNR>{y?&t_*H6}}!T(L3jmysd~^&E!v9oEO=q!jhY1<>g^M zqt?>1Moy<1@B&qQ_x^Y-Itkq5kiyZOCyBsL$@G2*Jtckn`iyQ_m9|!wD%fU$wck&x z@hi5`o8t|CRzK>HWAukXkM|@)RN#5UY%8TQ{Lr0$nQ$@herDIMdxs*| z4d?_STYP30_1kt)@257_=8l`OiVNaOl=PqUiZ7JQ97nb`utr{(eTCBnmVScud$s(U zpF^dNS3)~^kW;S;IYPbGwHV4}eU(Ea2IRAA53~90K$muVd`wxw8?c2|_|H;p=@YT$&R{6d7 zUR7WuVdN=G*7?II7soFUc-{#63xukG$<(qEX8G&R7T_=9`|7Fp_}M@-YQ3}Au^FJd zS<`|oijkVwnYHgjeO1EouU!;MsO7JEZGPJD;#aJUK_K6929WfpGTH;>91=-sGikNe z^_|iClxh~2=tH@KMw3XC-PE2I2X@K>kZbdwnRxcxv04|?vdo-j{6g0Z zQ^gDZ&vTtd*Dm)GlcmRIju-`q3Fo@nVF>PW>7PPA>t?L|qGxgNLeOfMS)3=+##ZUE z1HSZKz8*w3lB7uFcI{- zVzc+7OlJPXoNzP*=yX`IwD|6aVzN4(?05>NF1$YOSAO<;KZkg$y9Wbo?1N_KeG~x5 z%XxMaopKa?0x3M44~5fbl}?EjOog<}O!oi4dmGYGaF3#-28OZ3*;ur?`~Gk&8sJdA zv>s~|xuWEj81F|qcbX7u+gHz9b=f{w0o7c+7%Tq#M8hc8)Dx#bB~qFzj(@L%SS zI*tGa{{R04d|Kp)S+sa37M@Z*5W?lwvSn0~jzRzBtUMZ?kwX5!vQvFnh{x&s`qR^?@uEkaypKD@f zf7{N~<+C=Nw8o0puQ$I31XgA53zipBl}S4jWUE^iLGB2G@>zoRRzU+0k#L(VX4%#7YZ zZk6|~(1IB#?YUl`ook4=%M^PPAIJsE07s9J7zJ`d;y?f*SZ}3SJ91yrPB#*uQc*hQ z>D{8cK(T@$+l|FQ>)`{;<{TDKlKjUjnzv@d|6U^fgnoUAlM4tk15^eiBETZGmy zYk;(+h)ZP@eWUTZ0GZkTA~!R{Z&XA0#XTw$}*H#ryEO4VvR}kE= zr4=miKvHVjE?BpH$oB{M54_rvPPX>k$LH~sp%XeapN!DOkb!WqD#R~+Sr^^{zDU@% zV{}s!0~W;>z>gPOFp6|uaF{BYP{?U%CGur0+*u8yCFuzpqVp$5e8+^#(nmiezwm#X zTjTQC%kR4UL^+1k&7b6$Ie7?Y{aAkyWfHNK zUrRq7KoXMd5Oy+?%#xmg@kh$Fh#)d`g1MIlt~QOlX4)_A>Ta-bL;;JGGvbhpDb;Uf za6Xt9)bq(R0OksBT!s}e2*F>*l+L?)mKUi^)lMB~WpH2~yRcsJo>-5!S1chUP!^{C zvDdLy+)Erw+4;MdUN^y2xQb=r3I|F~ZPR5bmnM&{uXGK@yr;GbEl(+QIudK^uZmo0 z%V6cCfw!(QZBoJy8W#$$Vo&i&?0-H8yQJ>aNnNhB`xrjqQV%8D&#!CNm?FNu9UJb+q3bWms=ww2=acbz!GO;;&a>TRM3wZk5Y3Xm7i=)v>~q|GCpz-rJJqO~+$lkV9h5Axpx3rwGi2 zfA$IjyCA>RY(+c|tX(ovh9N(3p!=WPEX)D^CT9uG0$NQWeWinVxgiG$hJBoP;={;8U7!0uCw8hTVlP3;)EK6`n%<}R+MyJ)w7FVF*^#f0%@cWS$}Ju~N#Asp2W zytLyPd48<-B%#GPJt`fYM2iZu8^cED`1BUGYA?T<*K#8eSz|k{%RxK;E7#$WyG*&# za&9bU*w>(5=+neFyyFREn*@&IU@x1RPEs!eO1i)Mrjn}n4W6;|keBYX&Sb2)!x@6?hlF>j_J}P+rbnKy_{ALZH{4d^ z*=S>s45ja(uuVh+_{F}k4MnI%wy@LNLus|J-P1PNc%bIbRwtVZxM#*kM210_Hy3t4Y7oGTJ#z;`OlI>0ebi_DtXhxlDVEQW^&~ z0}H%~<-K!y8S3;_ng-%7`jXSJ99f$Z@gN)ntH5*V+3jArr5zQEZasZZ%VfMsPS29( z#zIp1G=^nV(x84Lvt?hk-6V%HVi>$;@^uu=nld^DP!7C`aXbef4Qw4zd%w1p@{}NP zXq_pE(Js9+4X4zzoScL&dy?!%WwMPMi9L$EJ39t|l&E{?PI7$5Ze@U6%i!lUj^ zy*9xb(6|n`sPUyRPI_~v!_H*iTJMi^R_o)GrE6dQ*}UzxtLJ;ZrDYT2Qa>mW`r0L8 z*iad2RrWHCN4m`MikDBN5j*@Y(uXl2XI{6j>P6?>GZDkLOjjB;Y-o}f6&{#@U6`95 zhaKzJ`9SfvktNM(wBsfBU&vxEdFQ@`NyO7Tb&IP}^+SYcaEWYf_v>9Ff*erqQFj-u)lv6tVRO80_q{oNZcDcHy(o7wfD zLt9Ty?H(N;5sy))l;Mo3yUBFeeCGZEK~~>eT6JV zFN*k^yOe{wThh2if&ke|;h1>vs_w8#GE>{fA7VPfyvHd*YKGsgatv29{HWi~s8`WO7)I}j)-ZGu4d4w)_eGQUw%E4%_3j57x z2I^Ye`tP>*?1~@+5|2qqyv4jDl`Jg!{g^#;bRE_bv$M}o9 z)clbuhL=7iFnaNyk_|Zc(>*(T(G*b7pT+4Z`3HpG81-O%UwHT2#vS-ok{vTMH7vFg zwewWxZMeY|^Q%ApvVR>_!mqBe<}sL}PT8Tf<`r6H^_jFCHKyth=$V;jRn1xN&y?hU zI;l<}+D(}_`4VJEw=CIP)=z#ZpQ^VWv%6|X6pLmhBzE#@7Z)d*{n_ zVaD7P9!Yt3iNkX0T|vTCGaD;=*XmZnM_RN~US8f*33nvj%L3G$Qqi5i@3D{v2X#Mt zwd2inmU^*kAAmJZS^ke;CPJ`s;Hm#>GY;Kex#Xxnt4|vzc%LJNR%RQ(N-18(I-uJO z0kk#Ku|&eo8zrU1UGcPDJi+p~1g=za$0T&`wXmuDrg0?rEc4e^Z*>f!qNmoWS{}A$ zQfQ$`M+PdB|FYMd%8G^jE=kH;3NFzOD2fTV^s%z}y~ry&cA|=ZI8i_0;%hgT6+6I? z)s&Eem0{56cq|yU=FaIxg-noZAEs{t;e-ZHgFAsvj;xpy1g6je(KQrx0l%=G0rhUPoH&2PttZd*rJN9EmY8} z5%FBN-Ha=pJ8JyP{4s3uytYn`eP*ZSk`O?CwYHfpqaXfc{>p zIj%mPbtT)p{qkE0?JfH#W)Bhp&+Vp^5Lc*h$JTKPFJDpKyuVK)ts{0x)ki`}&4v2; z5>qWF_HD0BM&Fjk<-F0?|5h_VVg!aP^KXR`+pB9POn4%Vkr-#WSz0eOUUIslJ&D59 zG81!mP6o+tY65f(=?`3gz(;))vu+EYG(gc(a$tqLN1Jwc~FNi5G&O1Ebo56B=Y{Rw- z-&hF^eAwX}Zh(X++II;kAM32|j|6b^!Rw?0aX+2--+6hiEy*f?eN++ak_2;5$V82NsTXuM9vCW^ ziqUILu1_zU30=#5dgJ3EZYjRpsoOcbQL6{4@vyLa$!!P3=g_#UniakHwY%)bCj%1~ zgTd3)0J|`c*G!Yw!5$)`y?X4fpcXSTKb{xO|Co(ViRZ(d(5x)u_d2$z0GSE+>|u&E z^X>A_DvWV{=p6%a%9_aaP&bG?Yv787b)ZRA`+pTpi`QM5LBC5bje=&fR;>(o$*79f zAiUvS(!!64rR^wbqC^v~WRZyVT>R%9YAO4t^IDJlsQ;CD#O88y7o8r$MsH^6Hw=BJ z7GuKW?8=1sL8!HXmO5RISBrOfj-8j4!%d4mqyEz}<2`AGLCvXSee05Uk@iW`4G68t zs^I2)BtxT3?(i}5bsqtHxxY?3bb;l+=f6K++4#xK9KoA7>T%@Jg6bMfH5IE&JS6R$%VcvvnscnVKjGelq1}7Qn z3+?ZB`}P*I#i}@2B?n2OpYmfy(1g?Xlw(y3^X%nw*L!^>8ollz_NSl^(Fs_LwAnr?XG@_#MB<6WQ82WxslTm84E^b z%L2@nqYrYt$2K;U8EeRhuP_~&#EM%z)}3Q>BxKLNmcj>hN3wNjZiu>T$gpa+weNn= zt>rt>N&T%0B5ZTwJPe&1&lS8AVf5%<)_C4yK=iQOXEuV+K3GJg^(@F6AjzTZA7-;X z8@VGTuwCQlEz+}-?9gIh?%mK+>y9Wh)9fMHzrXVtq*vQJiE_SOB#X;6)!NP$CZ}b zbbDBqY!*Gw_7>C4Wh!PZ$n^C)p8@k1q&lsT%{w7e6@}xT3FS<=+r+mMjg>bO(Jz~e zWmYi`f-4^P<2^QICI%OR?QQhd zjQ7}6iC9p!w5WH6)>pEZC2nh>Fe4!&vQyMs$c$6UovOeiU+58%Tt;dj4088M+P$39 zuAzv~l>r^1QD?dl(l}OL{SOv)B>jpNN%9R(n^etNjaPIo_vcZ;2Sl`Fi0`gjQbSJ* zUv|dQoo#*LlY^snB0(ZK+CeW;+Ts#OW|%e(AorluJK9RS2^&qXuzkLIQuhOvkBco@ z)^xz>!csxo*73dEAO1j-ZPtyffL6~xlTb3_Z{^HN z?A(LbZEA~l04HOLL)NPAdRhG~;(HjEF3N{X*N&y+o3qm!478S(U3fjS_&VKu&#ph( zxFx$Fa{^WgHSKqP;xYh>d+CL%lo7Pg3 zPj&eFL`eTVCn1+7d$0HQYcyTfGD&USQmikD#2=H_Utci4w%Co9=T}bGI+Bf`9WgX0 zPB-Ky7ydT?j67rPZjDYeR7fs7(xEbcl2iqmIXDGUVtgE?NG7wKYu*JuWh%kDXu;~) zwxV{hc1>J;4eQaF#TjL4=((mh^g!n5=Cd30(j<)IT1GVXe%^-QTmR><+M??PGP$p+WS5~naFo-9d-ltEX;gkq^d@HM|{L)TlY#6 zcgB)Ayv(f7%s-xPHa?MvP7<_j0&IMfwdJZvCDUaFv%0=7YU1UI{14my$(-I8x~b)Z z?q}c=YscPtDVJggbe~Erwc~A&zS=A2GiKuySZnGjZ+4Y8Th2VQy4Jw(+2b*2>42xB zZ7iF&2(d^bw-8wf=Z2HbZl?;n zR8G&S@D}pYLqM#EXnbBr;Dac>qqHy#ipt9s0G$WaYd6w}BlZ0CrEG`ak)}^KkPf$2 zJIp^%A}k!Ln@FYl2Hb!S2kq`u9M6NI0Wsb;koM{IPfP5x1A-Jhn{keXcBL-xG<7OP z()8iDy8q_&wbyYiRo9jsw`=4F92bXone;RchAzXyDWga^UiZZBcBV_fWujXgsWNTq zMnzJKshs&Prx+n<6|nN{aS;35gN^tgA70GsgzoM5@*~V6uDQ^bXiIRHU?@z!hfW=$qEaag@*_AUdtY#8R`Lsm%24wq+5 zC42P$7}jFAVy?hqsUZL8M}3cE*GNOPbL>ZG(Nyf{6G8VtcGBp|!NB8op1vLHAkZI# zh0V(`Wo6Yp^10`N7-<6{^Nqjlk|Z%dGKq&_?cr$q*36q)+y?(iqAe|5+H;f=nc8l+ z%j3*aFZj87kWGwqCxwsh0}g8p>TEQ;JJpm-R$P1~DSO#4LH?grpdf+5e8-xUQi^gf zzx-1lNi8i)U&<)4U$)z{5WAQO545WNp9^RU=;4WZR(ZQ@TVM8;%(BUlTYrGLg^@%~ ze^c__yEF=665$j*%V`}w{;Yo)h%f@u_KDg3KYXNp;NK=!9!ftL{d1QNgREsPbP&Fm z#I1;8k2Z=#J2vHP(JZy%FBaBkR^~!_y6CP9I5^%*-E_5}KkLR^bs-g6Nw! z^%xI3@PtR`81zxIjtCk+HU@8T)uc+*75P1Uy7}49Bsu9Lt;n-6aMOM@03FOD(JGGS ze39F+sL@G*tNg>zqOo={RuK_?47(ab#@K>hT;}b3*Qut7M012=iN!CJo8fSgJFS$3 z%4eGT*VuO4n@b*1@_1`*WMFT5;fV1ttxr)sQNIV92)-fa@Y+ZR6SCtXdwGm5MwV~s zio<<~<_bb%WwiL+BA@&fIr#^CMtt5P2}#CywrvTkd@ zBJ}8OU6hiE1V0}MJdl_KK*rs9bR|)k7%*0sZf7I=%849atR2Kj@TyyW&0grysa0gi z3q#frQUP%wq2SVSFB?474giWCO-V2p^%gG}VXemGbR%{e3j2#Ueu`k#q$%tltTY`D zq}_OrOSzL~R>FGFDQ5BK8`t>v@~}LHHoUYGJFCU}F_jK9lOdN;qrRT-}o!3$!Cl@2y z7$cc9s73E>L6`Q)>}Tc)FYk^BB3Wm4Qbr9rSX*~u-o^O$`(X?0g4$|9C_!hkm{LcQa!wZ> z^R>O`Kzwg4eXNw;Vr`L6m(cmfhZw6@EK%@t-#Q@s>i{VaFId>yDuK_mMcgyEs#Cc< zs|Rn9Rzq==L|k&dm4i2oC@|kvKE7>wN{@HXlMY{epr`w8u~*JKC4;Kk`QxwR_j(LN z<5_$QEOrsSw=HXEgVw^@OpW=NqY5SpP{g5^ZwvCO%e_I;}RxmgzDLDQnTwAC?9;29} zSZn$jaP)zk>`B86yBq>To{1#3?MykXqF>_%mK`E4ASY(;JHutiy#QQL+}x;YROM0# z?iIsw29`1n(c4tU05;@O<^n>7GW5%GWg%!Q(m8H86e%a4s4k5~r?Yb}Fv5HNGk^NN zGS}AzYx&e%N&wtuWu>=RilEcuBcJPTSkhh}kFAo@5tg1Gf%mHIvI^u_2|dIM!J+FS z-d*q%uuJs+V?1tO;PCO4zh-W>tNP0yV+y|aqUd-O3wBC3SM#`pFf8e&ux9_-eFe=g zSe+~&56b>t7A6Iw5hNT>UNT9A4EVH=*81p|!wz@v+5bD&nBF}R(}#v82s=F#8!Gww z7AC1{66VHzoiZZLmOOL-^q=<4s(2DaMHN#3?W+rX1)G#h@vzMX`l4(R`Q@vHYa4l} zlg(`WBt(Itt&hdB;jQrl0X4lBfV3~scnc9{Vb7r%<4r~MZCQsZxIV7@C$tossGPs= z5)hi-985H>OZoA?*k5T@*pj#=rnD`kX}qM3YGT>`2I(~LM`>)~-zk3f%gfa2$&O%-2!V?4&~V=mG)!_RlkJFhD*;+mcJ101C23sr zC9~19=LTrV%|Zugk-NZXl|x>=s31d7X!=Xk=ABfo1YG!YkJ8+-F-f<{2EWtt>}W;9 zL+3IWm2*ap(rO(==xN!G!_Z~GXi`32f-Q{92FlBVC8jC%8SxRF2#wx)b$7*njm{F( z;vF;*cPr;-eTZcIO#!tj4x+3gCZ(4==5f&Z!A2N}eAKhpN~A12TtfUj=8c^07MLli zMW&Bz982MgOQjR_0-n}Inh+(DStkuC?l+~mJS zrcXs-o-02xG>XQCQExL%_F%*bKvwJJmgBob@{K)lOv;ewq@fZEU%WmQ26|}UV_E9B zuwx$eHio8YIae;>$4~FDF?L_c2SsWG8KUZIC?%Yh;suRm%QsOS4!%1%+Fw1^IKIhV z-J|==r@zBL4kafUT_KAX%SRa}I~Qj_S=dSwwaNrv`UHze^Wmodr)B}fOECE7kbwnc zGPGC}{!}+DY=>~)=IyWi$Vtd$`8_j!!f5Gab3>`qFoIJZ8CdJ}RqHDt4aMwti$FSG zrLC*8Tg(Hb=orOoQ3SC&EPz3-wy_+hMcjMD3@zxnLfB+Dos0eIQX=R zHP(NXwPqL$Jur6H2xfTqz5>dPgz8dx>>}1&=Y6_J-`-cefdRQ_yJ&SKNM~)zmu=x2 z9&45!b%a=$;aTs&NRcupoQ1noT50;S_nkwg$8C@l1uZD@P}JRD*VF2D@`K4M2S^%Z zghY$u##ThXSYkbCBbGn5BInVI#dihot@v6#FDGF$T>^WC5g^E#-K!hgW!+!iLI(vf ziV^G;CASQNd1el@R8gn<-K8+fc88+VycU>V>lR#`(p&X$6)YVO%9Qg;wi+{@mB5TV zv!5RkFyO%U9v~x4a}65x`?Wl>1ADXiy;o3Rmq4XL5{eHC6mj<>X;(dOcphXmjxxHl zTps)&w+TKO0g)LE&my{dl=*(PQb*?*IQ2;wM8pBt#rRJ;+hZx<6CjYj-T`(a zL6ScQ!CL-?$R2@dCuRY<>SAAoJT~th2S#3jWg=g?9iVT#mb>u5xnW*BB~xfua~&GvpX|)y|Osc zM08tQ=%Il$_lQI`9a@WiErx$-jEXMS@rUkQ>?r4A_< z{|#*dtj>Ua5AHm&HzOliy!f!LkafT%Tu@xMu27Ij+u4W>M<(EI$h$V|xWLN@TNy5J zhbns!O4{#)XQv-#D0HbK9@O@>iSBF+LYNrNUIAcAfZRL<*jnW^ggHZh8p%UOI!Qgt zuC6d?rZb&XZX7?O^<3J~9@||X67Y0(AuR`-u<#JqfH0fA_MkuKs9m7p;}1kt3K-@w zm1@=L#0KQX%`Y5!1@Pp1ROFHaInp-;SM;0Q*32+#LT%SXsZv-TMN-G#><99n7f?p< zNbR9%iEbzu=am94@TlJ?l0L;I(^DF?_rE-i)=U9 z6pppJ4Bzz-6)|WheaYyX{o{I#l99k28;+WZGXgU z?&XF6vn7nP?YCuRGWr)7`UC_p|7&0v!Y3QNk+q{S8)RJp>?HK)*vIn&5jxTNtcNq(doRdtn^NA@oUMr%hC!LKL%Zqsi4v!Zu-_$TK>#68F{K=F3D6JWDn;npXoG zV@n$W+Hd;i_|BFTZxwF^HYpd59hc2g%4!d%x2HT0rzNR@Nb$Rr$5<*Z#^!RoyPvc5ifUuoL6LO*VEoKyb^8{7XD7zjBXH+mTm=U7>-Gf?i?b*=ni2t zjE>*DNJ?&a@y{PKFF4m?w4t`(#lClAp=pi9RhB=JS7YTZ~fro&Kk(dww zg=M%`?6ym-^TLWaBc_*ibSvh^ltY!joG@wEhtt>2LJNsoRV6W0IPj+Gwb?gU(|iiSm}!G3|Fx6K7ZHRtte7UI>$emRJ{DtmJWdoYq16uX0^`H?d&3aVB1a%Vx@SEBGu*tA)xc6K3OB<#poBzJ(DM!&~$c;$UXY^aj znnAvD4ox03b}UYmcIYvhE{I{6_83-j)@tH_uo0~uhs*0vzH{l847DjE~p{SuG2D%hqIz0%(|r?)o?4ugY#NPBxwwNl$g9Q7D?#{|Bb}NV4QhmQAvQQ6%$_&MgOv`sxF5bXfFr=(Cu+0+QtWK=C z(!C8!Dk*yJlybEPLNcGFPY6774hTjh-c)v52!fd!w$yfGV1qIM@L5gtD8jSLqbS!p z+{!fv*pdQ9Hpdaq+hZDrqC$*WstU5}dK*7a4l-k8YHdwW_AIvB*NbJ@+|b$^U(y8) zK>h9TBi3~X3}_gR0@X`9RbfqwFUvOE-X{;kCF`XT0g!ND_#zCkAx=3HiKypWk@HpQ z@RS=QhPB&GWIV5LUO)3EouA1vkprivH1FoLy9%jDYaSw75Hq_Mp=s_YdglaInL#F- zc%!_$NeA1N-`K9Wmh+yuKTgKO^YdTULFJJde8rDI(GfI zD$(#7wMI$snU z5-a;eGW(O^6c7eqwO>b8NnKvK1?5A^_vp~$qkP&QZ0ysXluy-$V4_eO?Plc+p3$IP zUZAfsuo665Da_i^j7m9S6j?aa;HbY8h{U*WlSYJ=Pn%g&Qzm|x87JuYi2(udnOFM% z?XY*iz_w=ktTRI}wqe8QkkeB@W?c17Q!^{qM3kfRw!<9pX_hO%U6zKiuBEy8bB0@b zY4O(U0b-QY@gJJKz{@o@ze}uwBj(fd-)i3#LG#kEVLzbl8Mm`{cW_pY=fKnk`BZ0} zTS^%J&f6psvDP+G6V+h)C$1G3QXm|&IN_QM?Af);0e5+Tq7ST&Cx}rq2PyL@IxHG; z2wHr$`3k-^ZI6gR)SlMV&5z+Ic?NHm9)}U$D9P#s5m4(_%ld_w-Z;hHoNf(Uv0cU( zV`;daOY+WZ2G^XP$^}=Ij7>(%tWPW}+UW;79x!>tCK%xh|v zo;4_uWl&Cc{iVxZnudO@2iqcq$1y!{Un|?2g{n~1cB(&CFHr;t$qa73%yQ5}V6>8k8 z-JA=XkXDwR&&Fvn`Phd~#^rdMHtg&p1H^0vi02oC&@23=f$J>OqTBiAZ>V1kxU?2h zI!D_)Mm{8=02=$o8Vo~UhorNyaK%-7e%ANaMP$?tleWYK=o48soqmQC-eNN2%f^^z z^vDB#lmp38TcE+7mWh*lougcb4;^^|39f~f9y$&lm_YY7JbG|X8JmCiQPlxd8!u(M zG-mkQ^gGX;sBO5FuNLR!Gh1S?y)t;*`q^db+(W0kh?>PJrRfs{cpIn0jfHy&NO{Q8 zUL!`E7=0GE6m07m8EaHUj@wYpK!RUT|9b=4{X|rCGeec%zb1Gg1b+SnS&9SPZ5|&v z&R`5yM(gT`nMeUMG=7F-*{tc*z*K}R{z zxA-MM*zd8;W5J6RI(NSpcAkYFtf@luVQvjZy`lxKJ##Gwe<3Ji3-ZiuqrfS_DbegX zI7dB-n1K{MRhL<&@B3 zlr1#NIseZAYFX^ytkM1}h@a`v1nEUDspHa%nTmwIi13(Y;J%8}XrJUtLp2=?TA?E6#yn zm_lz@%ndv_1h;Nrg{;@0ZqGM{S;N$fQHljPX7ZKe_K6A0rnYjnq?hwxg%PG}-X01# zZbJCjX!ke@q4%2NrpxDis{vAJm&8S;wdz(Z{`SgUZ+`Hz?rBU7bwO*^42~R;61F1q z<&`jc+ySjk54kEsD3L>?#pq3}<;1U@Y0Ev)oKZZ9K95>LdI)Rg_m)$ssTr&oFH%bOk7_!1|XWUt90)OvM@M@T-q zCg|DF$+G+NzXRv19_Ony^BZo=Wa~wF>jh3DmSqU%=~;89bp@+m{t|zC79ZHf#d_e% zcIry9i$2IMQ&K9nQ7p~gbuI}sjvF70LLOA8OW{&ML zy@!k)pY{vW%m!?y-a5G@-Qj}^O|rosnlPq&7g+OKZ0jYm^>U{5dM4g)l}i#iHSFX& zGc%8cc{Xpz?RigsDjpVXIm+47_&YOyB9ZUpGnPf^QHPw1EqSLU=Mm|1l(lg6MZbwN zZ`7UK%wawp_i??h^?Jt{k~OnGPsY*fzQ*GZszk1aCU+^mhH^GEwFg#Y5bt@uV z7UAbBPe&iyX(!yPT@(%kpESko%->bN%-;o(+oEy*Eu7~p7^;-K#`5#-pPm3(4*|?- z?13Yvek0hbK>s$$=FPW4zEgwDO8kLd%s!p~EH8gN6Qv~S7}S6!O)pO_B}-}#eG(Svh#^pywM$WvTGxlY zL_iDlkP;@w`*Ga_hjPZsyySmRMn14_9!$Rh!UL#&k@f|z#Zf4cWIzc zL2+LkZKQHGn_$J|Q!^f*RwcLA@rz{3TeC92hv7a`qIjjPfX1zuOJ3&y+*{}0fE^Jh`K@e9P&RV2=o8DzV+Y= z#hpt0^@_p`*j-91;N~drmfQT0TURHY<}&a0DDb9@P(J9igG6Z6{mDvQ0HzLS3~KwK zO&U{yXu%q{ic8FJ)b8T|Sp9ch*Mjq(5ppr_>>c_}TW_y#m)LIZ8=2L$U@J4`7D>WA z98@^^@~)@yTzy77Q$t62L`)A8B4nQ;KctdyX)A+(*{F)9I}PVMvhH&0b=|G9p#5QG zL=|7&{-eD8A3SoKxcA2SASpjz$I%<`<%XloZbj~5y6SowC-D+^zx4BaU3Ndf2fo$8 zmhHh746+47AX)zszo#UjgUZ-jz&M+u_D}GPpli&c0||dk9O*^(ud^5`abV~%KTA{* z7=X1no_y`$M^s^I>zcfi5SqT=sM2k($?d58j4E=zVZ9*6Qn}Y_n=6w;#(}&*$CQdd zr6lh>BTv=l8M)N*wh_MhXX)dx6CCq6oA-t&qurW3-*V18NaLI6?ZByR5q&hm!sYkX zDBJaDoWSqefm6Zy=ts@q{ze;Jw{idi(w7wZMcFpS8p5iM?R?fNDXoe|ULpLyhvvYLY>L z>xC)XYPlw?_JKqc?0|LB$7@O4tg9&K|CBKLU<&5EUbz-+TOUr_zU*HJlaP(}h5gTx z+1O#1miCa4yMrRjSR+8@a@^unTBu0K77LCT@(vh=te>VLFxlmy??{CvWSaKFh5Y`# zVgr?ToyA{I#(#hDK6npDs~(i6C20K`aDOK^g5o1?6!5L%KWm%+;{6;p3jENf_LGeHsRb>fnduMSw6Y_dX<2gTGaQ}Xsoqo z5m0oo+PWplivmYhA2l|^8++9S{X#^EKO>mV zTbX#vDh+*mm3N!OCFF-Tf8Hid?D8?lFq416VpkdgY~K+uieWoXWb5Dlv(P-f zcn?4Kf3bqX=Yc&K{spj`-3saF>f_FR0rk#co3(rVnSWC&bYE3_DDy9o<@NIW`q*Qw z4SG+uox)B*nf@X2`nB_u%##d^2)E zbm|!85UlU?wxp=JCDu^zv9RcS#rM??Q6CLNy%Tw<&suNH>|&K|#E7-L9Q#F|s?pYY zvcGqT@tVtH$!l(n8kS%{#*NS7!Ei*(nw{!l3MX+De64HKb`A8WXV7Ar4-;l9u*wI6 zk!a@aSA^djb4d*Pg=&O5pZ_FXnaF5v1r*L~HEhM|!qmgoDLZsIbZnIHehT9zGm|BM++yv=C*7SsT?HrLdN1?{U+PHk! zsU?NV-oHG9!56nR!UX1*vBKCPrU3oxa}Ar)z#Jbi?~!`S)%ubld}bBaahdAbwZA)? zC1iGGQa|`pIM#MYl#t~k_gCusgKjL8@Za zTDu45>+?JFCP_$Oj@rb&6+k7q|eogu`^{mL>_PtIMB46w<6Wsb4YOZdKh7 zANI)mKzcVmu_o_M}Y~dPXFfKbV8UyuXcAiwCe*PY< zo@>I2?**J9fuAzDsDU-|ij~x*T-0VaWik=g`i5;i5AQ>m`Te{WF6ZeOF|+9_s}E#p z?T_0OZ;rKYlUv89yUzk|_in!*P2mCs0o}Gc{yQEFzr<2Pmu&R-2|B5;t^Po`!;Dpx z=3~J#QBqa`q~z~%1-Gi-hA>tcFzX`}ciMTj#lCQ8Ps)EWjY zDG5rcMHFwGUs;6VTu}l+2xpJ(B>T{l^JZ|SI)9LPBHXPZqMXnBL`@M7evq1hMp>MSpVfwV#t<<6J;K4x!j;{FU%`Yf&r5Cm&(C@r-Cv) z`;WSJgr$yd??ixuCAR#BAk9CLKQedcoG*Mul z$$zKjH-mxwg6iq@y;|W&CAhRA$JZwFQx9{(q`4-=#j;o&!EJMKYVESE>Sn9T?K|CA z!tXQC0Zt0GvPIUI&H2y`9A~HhG;ghxj9kFRm zeyIhmYGp2%Fc+r8iK3L_2g*{JZ7SGMd~#8OVV3iw-Ye?KL%wSra$G_Mf5hCjyy5?0 zP?U^3GdwMzTlsq}Hh_k*XhEo)h78=6@eWMM_YSQ@Lio0*zac1cFQ8)Lyi4oqb+0JI z8z-zEEfhhctNiim>x)G&>rpyzM3zdQud91VGEbF#GU$fb?s{_1t5>Ez(ay4=km@w9 zlz)O>SbK@iK;zVSgSyqEMCs?c=rS?AJdcieT5#PutsS;&BBt}XaiINoUH-Ah-R(@i z_P+md4kfnkSikd5e2ZlET^h{m^%odA{O3dMGO8^U#u0&{jtl2U&-pf zpf-a6A6_Dy=yxoG`jS=F>nX<4uPzJ}V+{jLzk_LePo#dKe&@Y<4+RV^&M}x05>l9< zC8A?9)pst|Cov^3g=n`a1ToU>G47YWp2FP~=z7QAo;Wn~%z9U?V*Jkh`?b@#YhY(3 z_@+VvLdjHw4V}#5wa)}sN$aJLZ~9b7BXR(>2&v1MsRL^k{oOLkHP??h^LYn8Gv%>$ zwysRWn|969482n<<5<)9f(qI4p8!JFRu9eO*wZF2Z-#}7Gnuph>`ZjcwuOi595uUB zM=!esF=zgs_s7jN^AE9E0t$PObLG*27s8oS&*(O;I^C7`U9!{m<{SPze5u)?u$d(P ze84z=MhLnDp(71&h$QkXS`@^+@2_m^ucHMUOV)`>!i4aZ09^36EodKtzvn~s=WsSf zUYaYWmL;YV;gI@{RlRRb`GyUV%U2u^^X&+JVoPe;EbVh<-xM_A3N@txl`*AwR|bgCzrgC!o1>8NQ;Ly4|FY|n!y2QU#RTx7GlZhT7t_op#*GH;pC zt7Fn~$`;e$0o%Fhq4G$)YMvVZgpA37ZF;!%_A6Ve>bpj$gU_ND=04j3`8f0=-^ycX zwn0T%U+mL!lNCM+!C^B)gWDgAZc&-(UO1!c5ul77!Gmhe+5nazioBdC`Dl`Q7cRj7 z0E?gzo8p;Wn`c7M?e4-g^6*6K#O=ek{zA^yv*^}0QY6jjD^_h{50Dz|ga>Vl!+Yvq z#`A}%gRAYxt4+V%h7DZULVthU_iJ?fwSU6gHm=n*ffr%w@Tb@OrVYf}6&SESY2QTm zp=#V>dK{0WlsH3bRVXZv*cB%%R@GU8rpr|l%-D6_}pDJ9UtvXH}qpwG?mplY>@cOXpmbV@~3c2zxp$Ac&BP=wc(=< zf4Q%5Bp>SARUB&Z5DAa7;fez;{6pKRXl3{MQ4t`KhK_q$;$)bx-1|&zdgzD7&(k}0 z%KS=FhF);`?|OD&36CAjUeT6+(AN!pyV$mhmm+NwNFYqrUlt%Q-w;I|;UFkwrPxU1 zS2p@-cHiT7zhEBQtPgG0CZ96uPU)V1i%SU2CinN36r0+qEuk7}Ck ziZAn1)t*hWT&DJ7`+8f{;I-?iP0sZO+$A20Mpcy<{k7vFV?j zE$5Wc!qHpITq;gH&Q-vHsgr5uaj$Ig6c-eK9q zzVF%Mtb@Jy*h4kmT-XzT{=d6C4`(Mpu!NB-_BN@t2%qfkEwMXysKlbPEdqNz{Rrn# z8#>8hM*zmOl5}7K&Pb^>G2;o5Uq|&~NFgEY$D!h#%qo6J_cXf~E>_xX-+2^(uUA|) zLF%HdSA(tDy~$SA*6DAQ1PErzf&s34B!kVbTm-q`QCPP}`Pto91C%b4J;s@9)*R<6 ztf%#vPnY#VEA-aYv&*a9>eTIQ>Mhdl2TtOPNQa>QQNx7m!8_tk;jC*usRb1)#7M^? z;E9QH$98178;oH$X2QMV;NK)t|5C~{XNg=b)N_tfCy+iNqa!=~jHsKN4`|eTuDEQWTB_~AW;FE@+-*X(}O(ph97Ff%KGeq zHw#@SFIT_SakD*A&uVC0l%NLxxfQamgOCo1w_Ik)lN#R$mb+Bv6ud33O0%-v%E1qq z#RgFbryjq%N8(tYb6;Qoa2+*aqD$4X3}^`{wmn#Wb>SJX;aQ({KYo-?Xu&4$e8_fU z*DTib(y=OJJG%L`KmHW~y6vM68KVh)E z&Yx{1oT}3~Ft4c_Q)qq`Zf?PU07&iCQM3VX8i+YoHUIiu^2U@*N%Tpox0vYkyuGo+ z)&D8)asBi|*2SMADtKd$Jt$-Ximk3YG6(z&*|X{Vihz2`2bWZ>65Uu_uH9{tiu_6t zNp8-Kl3Iy*e3p|M-5vq~*O+G7;B;IT8c zjR%u+OkEf2?nwK_+}$-$G%mr;`$Dc#g3AGK!4AN4xvTh_fAJv;xaVen8!Tk;!r*?p zqpwaU&Sy>7%RViXT{1{RXFaIhFammWzHyXu|MJe7?>yFj4WUn^1L|_)AM6fpJyvh+ zk`)uIE@qm<8nXJo}}`%-ULHc#$4Ppjd?EG|E$s)AQQJoatJ!S;Dfe@gs02vlpJ>IeFK%9 z48U2I(8BlzX`-v4ukXLvlSq}f;sR?$(r1eXd7R=FU6a!;dt z3gwAv5#-8lIqZPfc8V?QC52B5o|o(A2B5sxEhyeXZzC8&Tmfl};8N5Oz50;x_-YY& z37DEbylSuke{ZPwg-RD3^}IGgy0ssYHc6(_ZROb)?fEUZ?NOyA*nlJQ6TA~{<9csU za1FL;U@L-{v1$C83)cZJHt)0C{s9^}vQ#1JB{oFo9f78|6_mE_73DRkq1}rd2GoVP zaeK{_1W#?VQ}{u%vSEWg>hF-4Z3Cr>M^PdJFgtg&ZwOie>4x^eELt)qNOLDqbw$lU z{Br36tW2*V2qL{#U02Say6q-6;hEEJ8(ENDsAAZD@<9VOJsa>;+M?+2JZLcDd0h!8 zrZmN=y?N*2#PU*r$z*=#7cU(Ta`t14_+Ylmc;~j%@X7>QNwDAnrg`!3vDx{}P}KvI zM&I_O;^VGr{(P9&`Caob^7R-^&pKBML#)Yh4pYcFIb#Y)d6?^GSqRY2lFDp(-ukVG zLJv&D^a0acU*cq8>;#=HtzA+7fOU z<-QJo^>^WBXW<#2hHZsGOm+D=%JOblfXU!})bZtgYWCBKY|7{0DdS&e$_)dN)JPOl z6~Y}!{dqF?@0&u%nc5829m$0tmVsDAPC!=@s&w; zO)|BjM_va}9cYKfkJjiXW;-3ZuGxh%ab??scCv?`;~x$H7X~K8TJ*Duw^lW?AFZ0E ze%bi^|7!uvcq~zjq}1QdMnp#|jQZkz{ubn3t@3OPsLC6qn;V~hOG>HMXZmM`y=1+V z*YZQD-?Tcn92GlXJMQv(OSy};X~xRb^W};tN&nE}u|0wMeskP`{*kDLeD(b8ySNR} zYMqf}ot-&HSF)P;PVa!951P=urhj7+Kk+E}mu#xR9wa-yP|R*3peFSf z&EGTYW=4@Aj(2vn6YBW%k=gh{LV#q`izP2(=C-Zo$9!q$a8qLkPsK7$X19GEvr^p5 zch4&0_(RjDWbixHY9uS#p$W}kQ>(Wd*}O{ztfZaUODmY&IyRoS2BgRDe0~vKEgd)4 z-*T-B8nV(zeT|>jY_12Zb#Bfg?7bcEl<*#A8~FU6*|wd2X*W}jLn+X^&JulT9Av&z z#@i-vX>t9e%F%}l37E+>s4eBWa8g(OaI81u8#^QR{UU=Ko{e`m#UWfB4X!XOLExgO zwDJ0i5a{Zv|5smIm2Kw!Z^0HD3Nz@~h6C88S+xl0B0@G=46J{U<4nkLB%aI0w>3z_ zF`oZ)d0_2sk_315M9#pML#u#MYnS*C;)XolXlQ?Tjlc&%hX&)VJHt~wzcvoKa`8L6 zGRWGwF1Pmcs{C|i!$_m&HIz*jzO+%$?<0?P)KF&a$zN=&P8nO4^1jLa z#Qs?D5g`&K|ESUStBi==srbC7Gv1RWr|Yvh{yn3GT5L{Q|1>`bNuO&a5TiyF_us4& z;!lb^$G@L7qNhW?t6XgTU3OBEH#=@s@pg~|4@Eu^Gfj^JY&`Jv%ExJxT;ZuT5-Y~>ofr*md}kt zL{W3z!<~}m=9?k%pyPD{)PIdC7cwoqkwTY!)YKF&FEG#PoHv<~;qD;4#^*O!w&!ac z*dk+||3!8`lnVurP)5-K9{S%1t)r%%qWiM3QIhqT&{|H4o{oMej+$ugnwJO`%|gJ+ zBP}n*TxqheaBBSx8Fi`Ak=h%DW27^9C8lcj!GMxO|*6kagmjlefC+7MQJKH)HQeXXPrJNctJ@9 zTI-6D5<>%n-@P7I&AHRXI3I!`ZK z@^5xPbAS4Z1U3uVo`vCi$*g8s<*I%^W&sXKZW^Q?-lgGvh;CiLfb!Hrz1bzklShT}Qw=lo77lJsDEfQe1;XSKm%c>mv zfYj9dlR(^y&I;?SlsxLXmtS8;otHAo{4gid_=m_|f=iL2!7$4;UuZu$Oh=1IKl44N zBEzNjN-W)+vA*}=5#!CL)EfFBQ5S`Zhc~k$*CsJ3G|OMsE1muAQ%R3lz8@Y&jW~m^ zU_EndhVRXW_QytKy;=P8%L+9H+BOJHS-D>c>0&5T9Jf9gYr}>Yy*F~`MA^%YCY9T- zIHi83qYNbGeIVuZq_}#qKny^u*d`nu^yCM%T9?=)rHz?_$%qs65fa^z8G z(YU#vIb*_T^TU(TO`>}-y%wm|Z3ZIzcCba~Hb8lwqEO!%*O8IML_u4(ag5{kH#_wPvpp_MQ{`P%{GV$)AyM zyb>W1XV(E)^Y8DtIV@p{pIK?Gs=@U``Q@}#_+`AtfS6+~eDc+4fzHji(q@|;@lVK9 z5GFXhO^v01npW{-gYUW^cljSH4D&9?=FS(+nvY{=O@v$arqhvUZ{V*i;2Z6KF5{0p zPd%yiGhOU*4D$N3#vExsKD2t$<&q`a(!Iu`N@RYlWZoqi!fHkY6!jzQ7sN=R6mGBF zx-BS+1qA}p$oZ74E{g0HU!2`&giOrr%}HIaE=wPFyMnnGuR*ybB1o$2LDWS*PD(7i z1c#PtIv;j6Tg9nZgWV#XA31*r-IdVQLHNM^B8brA(-+W|P5bp|ootV}B z_u69UYROjqvVX+ughJtn?z1*!z~2rW?2*RA^-T93jD5W;_xZPxJb%-~c3mE6#T3y% z7$?>&gUx*Y4C&heOyGL~3wmY3pXdBXLTZ500VcicR94cGp;L^3K?7PR2T~gMh zRTD;~7=5D{d7>R&*+rqm=sztGACyr@C6AM zeS|M%_4GGZg{Q_4NHnud_*@)ns2@cax_Lf()jeNAqXWXrr4F+BJa#FNy$FA&dp1Sz zeCok)`!i;iT484|G~sv*0Tp542m{TAjt3|_QJ>?nmo72STxO7zlmo@iD#xu&n)KOLpWM{J&l~l zAf|BJZEQKFkoUNf#s_BY{MRQ()pSn=sV5QmTgBzY=Q-Zf%1;MHwTsG@bb0p^eC!ey zr!Rs}_tiq~S#JNrED9|Vr7|i@y`=wjvKnjokV2dlR94&f{Et_s=JK&QEuUuN`w1KPkEDF%s!1xCbO0 zcWOE_UxDKpv7Sws?Z1$-OLeM{f}b?`xk7#_{?Txy5`8QafyD)J=v+>~e4JS8MPPKr zsbqGj^U9g?^W-Hphct2vvjd;20`l~2aahS;c;{J!iQPC@=D*90|2+Ho@c&^08r+DC zxmmVvgZD0MQyj$YF2hCsnpdjhu(`eR>2=d7W$%iQ^uldm6$m43mY?rN+C}t;bt#C1 z9vrC|L!sn1kdG#n*QZQV4xY(%_Fo)l=eL3%fQM?=XJGq5^G06)IfLCS$A>>!VMz&p zH3HX-?;#hnCnMN|;-6U4Wvlb;;v~vb&wiWjHwkI4KUHWI4u*8QDlJ*Vg^$RWfK*R% zI4xdtNJ&nwj7dvuoX{Wm_~Hsxv=gkR)HJ4xASwrBI5Vhvh{(nArZ0rxYM0Fu(blDC{pmKYn*SAGTsH{a_{+ua>Z-5b*PqUNG&j=TDaP|=fa#2g zd}#WuV7WBXBQ@iRduw=}<322*q)fIOuQ8X823#aOwl|>dG7oc(a|AkOfn~DU=RqgJ zaj6FGIl^O{rx2HL=s6WtsWTkPC@0h_aC^feX@X)4saQ!D=mKsH|k&yQMQAn9uwOaM9|~kmejT<0%7?rJ91qzwjYr zI7ebk-wolgJrB0K1r_%{c(S8+FMg$HwB2We0A{Ij)&oeyesoD_+BOX@;glI zWoqGLOiBCFCeIyA>}aa1km`;u1Q3>N=I)fX z7-R4VRgU{_NbU2;vRK@I{k#0SOk;3ne!<3noV)G)x-Jd(MX&DIwI=x;h3+V0Z9kge zytzge?eU(yJZ6aD?)~6}qYo44loauYtCPL2Xn~NS0};uGu>kw6pG&{<`M`zKiyk9L zwiqfM9XvBTi3%!NqO`pIWd>sG8*&xya4a(J-0KyQfqsQl!H!*sk^@CpB(f}n$6nHyB)t#!E^bRGRR`XOwTZi|Wgmp0-2O z=#i7h0IOV(&aj)=C9EZVKT!$cE`ADu{%XtPW`sKa>9 z|0+Klt(YJ)q@-UPXu)IMPP)49OqM2YrhEP^^*2rPw9OyW?^H#mpM++9mDrWHlqD=v z|4Eti#X(vRCq}rV;hetYJ~Pd!dbSuwQPD@{|7N}`T+2J*vx0shvkQInT7>#3ADu!H z+DZQDNx3?MpIiKhC|5FmgDGUWDGR_{U0^VK9(xR}D%NR=hbNUP_$hSlUN!j_(2roY z!k4v1e|u=`)>X##kcc0wKIN@k<2;3W-9q%Az*T%4yTJ_A%uLR;g?gV!NFr+5OjO(V zZ0qYsSa|kLG?%HZ)7mqDtzm1#?oQ49A-jIfw%kgUD=^@>m4<^*feTEs)Hl*i#0zG- z^ejzW;qVS4uy#e1*_5Bu%l?dOP^{iNZ)V2PTSjg&JzN7F$Oh8*T%v3w2BgI`Nv(ug67ZEhT#kb(Gjjj|>&09Ir z4ZVFv>dM`$sj?%ap5LfBB5q&cdiwKV(kc3hZJi@N;B%^C=iC9^+tWY=&tvN}E|QVgR;Zc6x2Rden8Og-3V zHL$RZQIofw2k>3Ij{ELw2d=+oPj?fWSW=Qx7@N{Z4Wvu8e__dJE!q1f{A&Wd(p4yp z8)S1vdD0oY6fO}8GC#@%t!WmvMI`-xxE4D-FpTeArPB(w4&qJ_$b03ls#%@vr7J~0 zzGa&6Y7vBtRH?1rX2kX201Kz5-ByAt7kLu?VUEeRhCT(LKM;z0$3po zHNG0LroB45mo?R~*ZS;FkB9hXm(e?RBz*(aKg@`v-Sk^1 zR{8Vx9!*&@SuI=`n@dxh474}W&0bxkiVAZ#02h`(5&X|)U=zBuzP3=MVIrVkU-j>N z-WP2y&O^y03o4GUbh&Gs3#MtTTZKlmm%IQ_j>`wOFB#gc_Y`|}jY{qxsySCzB|q5S zbE~T5(nF*@s2;36Z7qP-MARm`JwVdUI;t`R;f;BldiU#n%Lp~CG>$NV)9Cg+vc0pF z$$lb-!?G5+Jrno(OXt}#i^U638uQ>Ben2Mw>QU8sr19)B?H#)0w9|4j*Sbl8PJ2!*sp{KlaMDAn|V>4KDdmwl;C3d^y33UG$WCrMhzORcOaZoA(zA*TfD6{a_sQPYp>50P=M>B(E z&L@iY|7;k=-(lRA`l~8p+Ab+>_bK@Q1mN(_!H)8Q;~p!dCO2yki|RoV(S;s>dYc2D zrApp8xG2NBON^-UTRPLiSr7KgPKr6cv7?VjxDAAZ>hB*m& zN4W#3ZPZ|7Ms4Tw(t5RFP|j2{&3vqyw)W6RzvH!$a@B|Awm*1K{4vqt2%5~Cg z_V+yog9tdaVr>k!=KbiXMEwB> zq-^_-&=01_dzlO~kp)ANaV+S2f_tDYXugd&|Dv(EY;6l3%gL#6>#P$^!}TN52>n`_ z26F!&4=3$V`=1=wygsbXyg8&a?28+K)+K9-Hh#|MPOU`z{-Z+<*#|Xk|EUUivrx3~ z<+JvXQ`J?f?w)o{AF-aMdgzaJ7Go;oF4&NcHf`65%(6ipsX%v=422s9q zK4I!l*#ek-DVATR6}dH})8?W@a4G?6R$}k$n;M;kedsd?6gS=^FVn}6KLFf#IM%+( znfRFl4mMwr0KvLyX&m-mm)3c1P5IUxrh=J3%U&7qGd?qb)Ifbkt!b!ZFKYYGQgJU`%YHb{US7qb<)4z!vS=)E zaA!FH@jIxHjO}u=*TM}tJpHg`2>p;y!S--Gr|r+wc+kj2^lT@SVSvQqMvv_K6a&rm z5nQeMvg-nl(Z-`5YP1P(7UroWvYe@&V_H{}#|7_AoI=DYh?-rvyG`-1MA|t)hyBY& z)>C4-=w1NdkSUE6xL;iA!=W+zG6}0H-91;OA-z^5n#qGjFF}8FSWDqp%1EbN|D$ zwB~JVOhme>mmRQP;j-l4S$k_|B=@vK=_|s`*J+P>y zL;~-pLVy%Fs>UX!5Qvfjzj4psMlpK*+EO@5LTM1f{2)8ka)V~sDb{I{O90QxS@p7n zm#r=xVDPbvZ_Y>V279C5X6o4O!J&_|wjcHNC0d&VE83jiI|%(*JtX+$H6Ar62I1Ew zPUw_GY;AH(R3)Tb^>qjtb!R$PjWv{gW&8_L=+hYUa~P~f+P_a(y{MF4?3U!IHLaC_ zEGBXK0SUuXkn5+ZdVSx_rIz3#KK0F-zrN0j3w+#T_jrobbbSg!7{KbUcK2>}D{i^z zTCZFWo3h2|;%(0%6T&#ux${~(=Vox@(i}gR7ka}z2MM;BdXVO}&-Jj?O({5!m9Ugp z2(JS-SNu<`<$1FIHtEDqph%*gEmBjE7V+1;=R&)1NU~J+)BP~>oIA=yGA5nT9V=fZ zSH$hu72iY%EdpLm^ql0SHn)d5EWFBO?8#8?ZR~EW2cX&oKTtU zS9V>bzRc9bN~)rg*Lm~BdP)>u3)Jmf>yD%IV5)RYkQawOY+E@ZB^8D+>DHnM%F-*< z579R3cjZ>1Cq-L4UzxPM!Vp5(VvP!(B$m~Rm5+x{(u)m>M~k-h(fgZz&UhsR&(f#v z7@EL#mQw2hvT*3ayw1P$%rGoV4`toErR;l#x6M52qf*q0blx=S()6w$C+U zQ###gdyOE{RRt!tv05)6jVxU&JRlnaRo!OR71LhFD3k*x28cwb`uO-(E;0^L^i~2b zXT2Sby>YO*ie~5tC-&>Tc$L!tjQ{95Y@<8X7_`v0CZGNjL9ID8z_^Yk#3IqF-TR7XLh*(6^ zdW20^ChT!j5{c`rN;_OBy8K6V*7lNnku;6jXGA>>Fo(EHxEBPRfj8qO#2IkgOa$){ z(#ZLfRQt}{OW3NW2aibIJ%It(g5O50KaMTf=l$=vNpdaoRXU;%{Pv@5+<&Fv1}P+Wv@G5@l14oVxG9w!(j4m z#K$GyO+9hBcTsX~_9n>`6%4TZhBZam<=z|@AIVY&@T=+sRGE5zSBp0#EInl`0JRbd z`pN``nvlu0jUi61rys2en0ujDhEty;y!_8^U3)79ln!2@k%4u=%-3SrYW@vL9UOlA znr%$g@#Kqbrp4mGpBS^0l<&HjuiP`2I9V%iVbO<9{@X&atvDi($5J@mGmlCwNO(%3 zOp2&0)3dZT7iQHD?3;DpzjibP|JP6^yGuEc#^$oYmt$qvAl_>Ap$7#m8apV#H_vd= z7bb>varDRl=NG4*LG`q%7A^+8cLe-*VLFl`mRK@L`xtiPBmHS=*+JHTroo+#!5{RJ z-_(f8xpM&UDdkw>)oLd4Xi~gtGMydEI@`L#Ax+qBtkFILp%9RdUgXbGA78e3*F zuw-kWW0sXdG3$WIsCe1ot_*CCe(bMLnRc*SBkI>f31kclf*csly_V3ashp-{m7m|H zHSl^sg+pVJcPvj%84xcR_fR5v5ofEHRGZmF!`XlJWmh z_nz@=zW@LCr`49)v}#kOb}6Y^#iz7YwQ8maS}JMn9mMFMs8v;a)ZF~ zS9T{1Z_RFM5G-TdTugN6`4RZujsJ>Q1-E+m1PK=M;()L!Zk<^Pgu=J8Tpj{xG#* z8JPuk&;euI59>B+iH2R@4X-VI6tqUrV8Iz$QHaUx%_JESKz!q$XMeG##XvR7wR{Lu zUFj0F9t&2ZV2@gCel*_6GhLg>b@#wTyjT!TyNsM4Kn)Fx;^yLmcGpZ#_g2%zm2lq^ zt_C}nX<4uC^nDD;@7~z&%_)lKt`+E#qZlsH&wg(=_3;VF1Wt>o21{q@l}Gn~?EcHz zY+mJ?#fk3p!;yOCYDpiG_~^_R7vYXm#?nBjT@02_mA?9!Z!+Y ze5N(O##?rsR`#FT?O9uNmK_KsjE6F$xT**=8d%@N;*WB*n6hm^fGrU zdlSq`zg%?T> zBD5%h{g~a0$5Ga#9+fsv)x(}=Ka0E`_goaeUbZ!(nTRz#9QMFTqcim+M`Mhzijy6amd*{d@8W9s1kWJL}pic4T2C>DA` zi^d1n^kt1le^ef)Z2|$b2=)MA46b%JF7ma~pQp5w!iNNJkk{+71{f;{CZoO<)vcEY z{np~#43{#-7j1HvD&OV3qZjG9vsN)TX4PTp6OhOvF32Y56!(bZ_RM#|k1v!3ayry{ zmBbph-|xg?xmcAi_^DuS2ylgRcqYejEYdX9J7F=iFW)d32RJYR12;AjiQ8!-l?r8` zyFdc7)U#9*;|OClq5ZCrzT+nuw%?B1=)V2UX;FEdi|f-fxuR%>>7a*0HbM&BN=Kly zu<5`hiw>gt!?!$^-!(Q+m{H_d1QhqP%odm8Df&=m^ZINWWw@e2M&+)_qAY%q8jE_S z8*!!W);djrP#=2ik&UWtc@XG=06cn8z7i|vrG5Q`VZn*x$fZS)>`A})U|GGTuIzgM z;>QAX^Qo?znY3}+tz~>WK}UX{GsmW>%nU|udu|QcP6`{-G$4qDa7@y zst3KVaKwD&|3L7R>2#a|DiuB6p=BT3UXTw9?C%O!1DcFFtB>zb3Tg7%d!cx`gD>N5?1S;kPB0H}GE?{}Ug-zxFl{7PVNa2u~OQHCfTs zldd_)HUbt*`{0U2N@eAv>OT$~Q zcsf~4{BM3tN)l?9M^wd5iTwCNc)j1AYIadBDR89Jt{ot0sB*2^CP;QiIYYOGo_rVq z&pe}wp#AU>$uBxwCfRxkFfu9%dkj>GXH<=91XL}I35^_Jqxxn)$TY1&oLO093IR`4 z>3d}VDS^GL?e6Y!0;jKt-FEr0YNB&bFy-p+uU0)#Vs{ztmbR-fJ~*aDKp8W$vP;fC zdT_j;V`Sd2=4~+F#D9>{W4sA^dsh;bIp=6SkdTn*V3RC+ec|J*L(hVL`sl9A1yQ^0 za+Iq?4`a2UONh@q6bo-2Ox@JP$vl8Pex@wXaaI6bJH6kbymdz)CpffIeL4}XW=j(c zgE!}JYkyd}fn*;ze~8Rj{CNfhw(R_+_w*LuuirLSfn==NQR}(Kc}r!v9n5c+W0&aR z!NYH4kuzEwbXx3*Rw=jF<21g3alPb=yysgf26Lo&!I!0edu z(K0r=R&~BS`hOe?X~+xvACw#3B6qTB-u>8{SXXD+#{G7Aux#$r!16^|9R)q}=Mmg3 zoWEmlK5&*r_9{6>GdV;e+v-7bVSXgAzo~kj8_xbAHrDwcuKLr8*@vEeqs0&2V4WtD z`6(H+Sk7ferumW{lUZsBs9CAcw+{(3?_Y>TqT>M>j=_R|zMHMj(vIw6p^y*JFQ(V4 z%#C)siBnKSmEXop+f3wvUhHc)*IkkstHzdT&mqfi>%4TINy%+ zDK2oYRrW8KMFk-PT)#M`(Zl7RZD#?z4Jmjofc{)Rk-i?=J}a9o;ozy*N=s|QI;t$3 zW%deN74VOU*0iUxBw2x;$0*t4=36bA2b+l3EZq4va}1}hn7f`n5$(zV{4h^FI^alW*1^mG4S;H|)26!Fw-NzXEl)SU=YE@tJG*T;6bf>O=n= zN8qL>)P1H;Y2P5~dqG+|2ZMOqxM-%NyWf3(_qX@2zWn+pDC2=e#Rh62@n%D;FP@^^ zP!Y%*jbqeVXq7v@kvj13U0Q5*Nl(?_a&qLNqAQdGR(=hQyZz(qg*a&T*JPyDe;#oK zpgsHy*1Y3W#-wen2&)bdpDqGHv%Xf9@}nyN@c)DC4c^3zg@QQQz%? zRJ7}udY70aA|+0rG@ci3c?t*{6xdbyYb5))4tPE_ABpio?OXbHj7?fbrZX|lJqFsb zHeLN`B98j@D=&#}47?XE2?aL182l(8%pv6js%~97ygyh$+^#A~3-0&Q_>o}?K@|1y zG}UAadu&uZpEPjk@ORZE8aB6oXsxNTOGwOV@7dd$-puVv>Yz(`?2WuA#m>q=E#6gn z!2HJGV2+sRCSbt_Xx8sR^XmF&Kz^EYHN0{fb3rpW z&GZH8h}0Q$OhDv!Q~^y3{zn_Sp? z&lv|<1)D_MY8@5^zPCT09e(iGzZ5%9JI+L2J$k_g<8dth_7NcYhWj(Jtv4c1WmzG` zo8hBkupQ_IMv4{=w40l3o>pb^NM*r~+S2V^_=ky6C-420M}HFUTJSVL8D!j05yl%0 z4P$Kl?DQ=t?-gnOmf{Dz6V2s?Rchd-tHa%9AM>jaHK~qA@ldMM-7#g}z_e$8btR#K zMoeJe52KCi0DMpOT*kVe`C34qa+48@<@YAXa!Qb^y2p}D9iFT~;<1-VO@;)!WHiSM z8UoPP{Vq+7)qTh0m4EAH1H5}Ed2)SR=}NfAVtSiWr&>Jm zWwy+67e*%7S)6;6fNE}zX)J1ED{V0Scu>*z{B{1e^huI<3j#wq9n zHknNm{x)Z_F?xtOUPcT5)`N}n8QXw#*yr41XiQ7lq35y(FN$U(tpXL&3}?V<`rLL5R>#`$WDTuX0l!?YgcQZq|Gg4yS=?jDpHJ0Eu3P%^}%qEji&LkUk7-gZR8 z1sBxvejiwpxH?E<%lyzsr2Y<{;RZu<++{MzmY{?-a>0Iqxuz}J44tfvZ9vwQXf!F| zNbCqqG~?HlRz>hNE+Y?amC9e}P#{UIQHZM3U~*ST%z6Bh0(ot3FVu&n`UE z-q9;`I#;(E*wLyzqg9)Y2`=4bobz+tp_P}EW*M@sI5`dADw zJ4}vu@_PXllJklw0UJG;(>vmeieN@vZV7cv zu(Pa%g03>=k=q|&7ENyYkH!2-4ZVs+5#Rz|!ae40vOM)?H;OlvkG}1Q6zQcOc?EH> z*k(7gzk0c=Y3!xdkkz%FTtAOblD*anb32kihbms5j>}<|Zj?B?Tp26fKE~Emc#u1j zT5G$wUGYLmSv9L>DJ_Oi4bWAFP}2fFw2Y{;1OIcYG(X3b$DtA9CoDTOqs=Dw)L}u3 zTVc~vR$TU)gHJg-L8za0vi&Wql&XUOjK3ecljGR3n^NQzG5^QNa~y3^jIsLoK>_nD zFp}v)?U+~s$ZJ#R?lpT|;rRT?B$Q>Zir%^;_t0j1fNWv}E1QH`BSOugfaG-($@?(& z<~5RcHlJg(ez#&ukOdd!8PPeBPd~!3*7zN@CvNDD&_o8(m*&4zOd0fdnW%0`?xU$R zmtHPBjI2zHEFPuYyHe&7F9_BF-PcJUXBUvXAnNM3N?6DP6h|Eb`X_y4B~gp_1c zAiuo5K9$56-e{>?u54;VHO#iTgf852Om{rlRnix56bu2@8g=kW1lktLtn42?_6@R8 zb66am@`{IZm-}@JYDL(l%5<{}1U=ur%Oa?VcRGEC>(R`LVCEL+dqtWtu-ho}7P@6O zU%DNqUwYy?_r-)NTwY=XYCWkdga!&5k{!Fpfs8^^$hN;7RLTu}ci)7_dYuiFO6-bb z{_|!UvaMCouj;Hm3)3W5eP=nV|4kRGX2%=Ob4L|V(#9iP-0r!mwR!}qwRi*zCR?&H zz^=Hosga3(D$z7tv?Md4HFI0I6zCkgLw6Wf0$ z^+2P< zFIrIS*sc(;H z1e_+3DA{a69yn_Tn==+qkHmVzeGNscA$37A;Hrz1=^Up5-o-M%2>ZQUeMG}c(%u*b zVb-s*{xdyN11QG`1omT69Ui3zUpcd_sz;SxHx~;8jSisHp zVu+B+E-eCc{=+aAlbs)H)>_8vma_HP2Phf(^w-!1OA<2Pn&403Tp0BegrTQOGgj17 z^J3LU>r(#J8grvo&2#t5lK5hu{o>jSs>dNm*Lnlgg)YMStVWqpKJR#-+0}nQvJdL+ z;v2cJgKnLBvWuDHUY!}dlhbXN6oxVYh>@VOywRE@W#{M&3*uV)Y|yjLy%O*tc0qD3 zzC4x(b1XEX)Oy*yidU$B<^vEz(iojtNZfNn0HjJ4F(QC28^=3{A@$+!DrQ!ZQ`Mys zbK;mL*w6PKgd3SZUA)*fma^`*RGBftKq3c z{b>ON?N$q)#w1mY)!*ORA_YNgDaV~t(^(0}PX*gYwKgd?Pa_cgV_G>x;@!P!tXCu{lJi$}%QU zngEB+t65w&jzSw%QPuEc*#!FFf6~Xvd7j>%7|C=gT(SLETcGA?SRx3_M>!lt;;Ol_ z1_bb=m&#$IbTs?_U4KLmZCWjG^Qi|zSPxs*z(7*Q;?5|`p|~UT)Swv<*G7&S`Ha<= znc!64xWeg~PYW!93W}mY$qcPMfF{*ybUc?sBt>!#5Fun?05W-wVG&Ko{9Hb4mv0ro zKa><;yq2X*9=BpM~ zKuCY&fby$<6MHiS{015-p^lcx8n=wfmbLBdQta%_se!B=U+>5sIC-u_t8_zf1f`Oi z-hpoNTdQ)5kmOfDmEHohvUMs!gFCz)Ev-7gu=DC(0NVQJOYk{#lGc+`ZKEIGkv|(- zpo9&m%LTV;afu{dUCRk>*W+&5)B%RBZP|q2%f}n{-Gpz&$H{1Vw`(;X3ZJd)3fH$Q zXVWm2tmvAZl&(Ea7o00z4XLgb@O6?c`?Z+AZhc0wt&Kw;4atgUv?QxN0@t-D0QO=D zD4Snuv>SLP!&4KFAIS^4wzPU^>F-*jj*{VjtF=~`he1A0VmLOR~yAg44(+D7G>hyhJXYIVl+hKJ7;#rLbc*ej{GPSgFNHW!(*$%GgF zA`x;XDtxW3Z*O9ljyWS1_2+C)>D%89{U3%m33S2S#BpH@iM^4%S0A1PxhdJ5Zl$sr zVkEythUgT$^&yf6<&=40fRy-A^h$gcxMMIHcI;(ycdx3L-biEpOR3QUowr}6qF&ZyGwy*}W(23w1Jd=^h?*z(|QlMg7Arz7m((MIFf6QtiZY*z!jf2r zCfEK&KBoDK&%v%k7!DZcwji~t{6`t9Z}~%4mn3#OPgkp8<+E>$8BaQ(lkczZCFPvT zZfzbM*6QB7gB+LBy4Bu4P_>vcJgX(jjHY`bh(NdA`shYm@Ez9GR(yxVHmL&Il6Ov5 z7#Hz&;DVM7HwkGQEeg&V83}r`aqHsSsT*t+jBIv49w4(tMpsU*Sh-|)s2>VOYd0)! zzcc%xSYot|1~)9W#m67mj!!=@VSbdL%O5V=^R zPa?ziBZDvAoE&$H1dCPYPktSHL~|?bqK#mJTqoDDV;n$5lCVnDvrZEfXi;UQZtV`C$B zA=I-BV9Cpab&7X%gPsJ$*=S{)qCKm!S3sM5NR4g(zU$q1{PLkKWFu$5c@xVH96W-I zvc+5L9?TCJX`GqBt*O}B~yX z=_3v|N(o`3sZ*8I5k=xO=>(MoEQBo5Hkm6>)oRu1?3?~elsjf;;d9Si>Z`~0#Cz6h zFc`;dlL*(-P-xKg6U(q>UtkaoWl-S*FVs>ehVC6zvJ#M(o?<{SKDy`b`f4C;wT>8? z_LE-80JM#el@N|F3oH*_0PmiCROmxg>ZJ9jf0rBweOk?M!RPrX`R^D8e2%gRmfxtl zL;oA+2;m{^@6erW+A8nc8EF;6t40{#@{gg2*q=xqAPmeHr`bs?ic|4?%Q7-DR9Rj;o`Hl+aoEAag8 zXA=sft#V!?P;L0cnE|AA+UOb*5T94yXjCC|QJY3L;p(MW)Y&5YkTo?5J=o=E_!s`) z<{B?ev}2MtZhg@g0VO2&Wv%e*-b1&J9xkG_mp*Ffl+k3b$<6ty#iwHE3W60)1pEjH zJ_|SV;q89b8_tFgpvzE2MSA-8#X_?iUV9|0M=9k^!YUK;A3VIZ$Bb0=?;rFF`r?Ro z3kCYbIAl6{c#2|29LukObs6n>^o8J!w=_HuzXR7(IE&E3u|u7f0Q*)4(Lo~_K$Eie6|6%R1Ob3 zpqFo)pAyFSUg}DRs%F;3a=s1QDwZJZ8$q}S@ z3O37IvKDq)`h1+^km)3=3Bhkh(1ww<49H;zZa|s9r3PH%9V}f}qF2ai7-c4wlc3jd zu7W;U7=OhKn#wZXo33HzN$Pndws%6Kc{2K10Dfmx(mvg)z6 z^h-&)$;V3)x1DdAcKoE=iIFIYWLJ8VPx&OL`TLqbI&?TFumhcXmFXX*B4-&tc2X*z zz-heai-w(i`6?q`5iC=ywVn!}a)9z$RcUSl@SGXir)@6xwrSvZ2I1g;~oZ6?;9YH;YX1RN9 z;aV&i-7~q3#}7AZs(JNce5%=5tS?<~elKL}8u^bV2I$I~i^e}rf7RFYRCq4;@Wthh zKC=Hd){1`ir`Zo$=d4;Lz^WV7(A0 zX`?2k-4%ZN`dG%b?B6=q&g;k}UK}SrbNdT7>35C$d2s&l?>kMsK;vt0)x60gfU|8F zyU2IZe9K2myd$%6S6CSz=9crFHuSJle{Oqm? z=~;T7%6;@+9!8)KJDCy@*x#qC2_-`Mq?u3m-~syOi~DUJXJ7b3dpdLd$B*Pg@n!_? z!F+Xzx5{Wrx%*x_+90OaI%C40ar(5gcV@A-BwMO>-L}&N(`)6LkTK#h>0JEF4mh$d zgr}`JtjcI2!-Jt(ly4QULxgASGl2h}rb$5^zRIw&<4>+ni%0{*LJIQYtFlWbot4|{ zq>IJN>_-gB_0z2mAKL7-*y}|ZMavEuR9m;ig%tjI%s6$qkz>rG7Vq78Ua6pmj~M4YpzVkA(d(4#DDZVs-LDC_~e;*Xxy; zyLHRmd}@Kvi*a4KP8v%i)IdFL`e&)XPNIclkNX=3lu>+orRDHc$0S(hH~N51r)st@<|QJq~}j zQ_#zmRQp4D#L4qMCy-xHw=fhBNNrV~Hl%@NHGU>)2{XDL75?nWz(FdQ3G`_#U|~Cs zA2@$1Iq!{JtwiGLSjckidN>SP^sAyLN@d|Tq3sh-Ou(ovTgJ%}?=ZozEA$p|? zum9e-Ud>j`(vQ8((%I{&Rv8N%d@ogfdehIBS%?eIlJH!_J<+!Dp^jVrcb5p#ZODoM zUR6)ys^ve9sWIWNIp`3+uFqzAmRy>%nRFoJ7jDKO(81g}2jAH(y$m zmNbDj7T!`A^S>xMRe-z`$fKYCK|vo9SgEYb2j5jy~9B~fSQDt|$ zUx)vf-3@H+JEJy9@WxM+|Fusnh4LpHuD!tZkHU z^dk;CpP0~wyOYZ}^a6|Yz2k{n8Gh$!nPtc<4t?(Gh~)A`$vGtCu_y3V$eeAnOyp1= zC=4^d_E3UEuiu@xi*lLZKd~zN!`q+opPpv?L$!@pEW}MV4y;%S@Ei3z z#5W2mJ1F5$v6-)xme$Lk#eamV_MFTyrfz4bnt8eb@K)7YRDVr@ICSU|<5H9*WIkP7 z_k@*3Fuav-=EP_SnB>SEw|UD02#$KfNbew_FV)7QGQDmwj&~#BZFpHajI!AUNDUS* zRc6`L;srr6P->68l=r>&!Dd8MbwzTKrhv2N=*e$Fnfd1isRC!r5xTPFT(8`X2I+>s znLVrhxlom|*Aw}wcOXH2KM}j=gVDZctC%~lVD-o#rI4Ic#~jX|h+Udp-xtviR`6-vUGbl`|&GfAqD*v6Hao57WG#?zdGd zeGqZKpeJVT-c%p=pnhYXnLaX&O+PNw<@e6weZt=9^{!3AV{LH2Q-d}M4%4X* z@uKdtPuj8ZVFy>3#FeziDuE&Q&l?ZVA|vY+RF3=>R~wodOiX4@%HT9-3^;?b87FZ|#o!V7YT7J`R_oV<>>osF5c@~9Q=rDgRYkfN0t zHBuXAFQhA|lht^=8*1hL3AGj;9xFdxGrjtoWVMK1tmMcQz7*n*RuKWvGGB$}Pks|@vAB78({^7ajpswzIyhw**xBFLjrN@}B}e#-R@XeG#hr5lI0!mI~5 z=xX)sUAjNpC?B?&Pl3^+l^bUEAYrGCU(>y81elvbeO4_Ui=SAl3b9*tlzz?@Up@z} zWi&YV5Uea$t?>8SI+~Wf;}>u*Y?c?8TuNddkEzQ_MYhRgy&whMaPJ#4vOc-DY<@06 z!a1EHxt*<$b-lvv0}WX0h<7mrz$^s4TpKDnANS)&vKniqoI{&qzVLu)M5AGPeQZ;BN(|K3uWJbO2$JcMLxE(dfZlQ1sVdaM5n>|UY zjUtVuX}d_xuuVvYxo!GSDxq`6_S8mu?JRU@3e%mzZGR<5LM4}L{~@H#wqo>fDn3+k zLcLTq5b>eep3zfY6|rg6H|y)6J2;Lfs+ps zgK`jKVXeK0gp=;>xlI^LjeO8fQgy$o{601GSf`U*=L3T*EsF{@`@C#bjte7)G;bsA zgAPt=LLH8}1|*JKl|ftKE~ej6QDJ>y`}Pgb68+`s*FF}WE$z(^i+=>mO$E%w-LD|a zdpUFBG)#fJ=^>}?ASpwDM0yD|Q&&|PXam%P9btpX?~=ywvfpOXwc|*nqt;#|WE$x92sYrpeDoYe;PmC);E zx1-hT6?bjZ8IvpLl3CN|f8V@vI!?0RTRGc^*?ihtnzJ2}0Nd1(*f#SBJ_$Q&Po?L? z1P=BeFFs6w9fk#UZzqJ1Exm-cp{FY&VVg%5J}}`E<+J$|!m;WsVM7NIG`1bKzbzcR z6c#u_jq&+)1rfMa8v5R>MT=T(A463j``c`|S-2c(A@sqh2 z76QHm&O6^P2?B0e0W;>!)hY~VK#KpgTyH^!^O_pA0{tDmv?SlEyRHXzUc+4E_^k3f z-NnA3bmz6oYr2oy5h^X|aT2?7wr%re=CbxhYW14b87gq98gXakakx_BGeDiE zkx89a!krbO+KxpkIr+R_LqXov3ALS@Nj<|?r^%Lu8%HE(?V=gG9Nq5X4k<+bj9UED zB->FUL-9VBuF5tL`irY=-nE@$`t+W+pfU0A5M^Z0+&mF|lrsW^EEj^y!shC<<49va z$9}#Kpo5+cP*!$=6b{$$VJCxyC+#vhyT`jv;t|dI$HUQA-ol}~%8PhjIIQ1e3C~MP z+|U1@eAXEpK-Os%Sv8Z0N7&043-psKh{IAES$5~R?a=jezy6dx_Ver?7FW2>M&*+2 z$@m%7O#jT07Vw zeYP=mo^qPRZ#>OSi1Zc{5}(xtVcX`FOxcW~^p~nL| zg2GGo2Su2QBdp{Ee#rd&m2!idTWi*RA4hZY)m@B=3u2s?)a8u)^7S)yoD@P*1V(5s z6`FgXo`GTl<}|EA5YgO)WBH-RInt`T7C=1#Ds#mnd;j7^HSMn1@$@8=-4Xa6Tj5x_ z=hAJPg(4~V&5DVTY!qf7)VuJiV)sLWcY8LaZhgNT0}ZHQb)E7XTHkE`WPAGLOA+g> zurgDFn<^vduu}vx2t4c`R@AORVykOg zarYr~dSfD>>&Kd9tqOV?JM+vqmPdiQ=HWyY2UEL#`Wy>H=WW)%kO}nXJ(B2MZOwCuN{;*MZox4`1Y2o5# z^Zo(%<`rd0mDN9mBHtofE)iLxj#)WsV#uvw^T?XhiB8Hv7x%E=sVXpM={+4UKhOVM zzb~bDKsIczgu%4Wp8p;}H$71-4Rd`OdCbV&IiiB8@Qsp$M8Hz@0v@SZ)*h)5^#nGh z)PmL-wSBqZ8S_axPFvBX1x-i&st*^W=~R88P3WPR zsV6i@o1d-=j9E>Q>MwWm&sr45Td65M1Ombw#;xcBhf*3J=@i2wQll+eAj@9vo;j ztpGd@7s`_)=fuB(mcA10YTeF0fhycuGir@C?Q&a>83= zX}Ecn+$B|fIaiUZOJdrxBF#Jxy&a(fHxQRF6Ih|velj);DlTA3mdWBR>aZFamKf}4 z#E2z3MxYU*O~SgtALH%%uT~Rr--V_%J!Pl$r__9Xas5(5W&7%>0zu0OA+5HQgUP98 zXlDq#(StNkw4J&*w{bHvL6cOek7zd4tCk-dP!}y^b9c)Igruik1vl}=9$3Z9MEQE( zV}fuN7kJ-`aeflu#~4>*E4}ao`_AsXpq}?N-@;S|10(TW7^tfd+|l(;&b-3Y>z*oP zynIpy^lQxNbFAP+L76PQHm`6yq6*Q>cV~hte0f!`OCqvoop~54&M}(v;BnZc5(~3? z5$fJgXd}JB#jWU?WEt_1Wpc0xRwAjp`>O7thTObal3mG)_5vg1Qi+$P(r)w*FN&hx z!n0X2DT>m`0soqZgY1s*$|-;P*PDCJZDx3HrF{Ru_waL!x+V-0b{wG_jEnioW&eo- zVTMY@V44@yx7TCUPiNdVVFo)ld44sY?HPw{iVA;O#16#Bg>3DX6gFe!n`(8~l8OXnXPGV4|)c^P)kbiQ1izL!b9gsbYF%;YJzvFp=U?BXX`-C zHTjdLjP)4E^y%60cz?6lW#IIN?WX@DO@a1*Pa40r{e_MIgy7Ytq|GTm{F0EXte9Pl zTphV&-~M>5-x1zO+0UkWz|mKlsV|pWbb>bn_nCx!6o-wnk?lDlM(A=Uh7M`u48h(yXE%4t)|bGB|Km$%@3CsTyflHWh)g z+vYArmNHj}H=Q+Df}gJkOPlGc987+ecws&2@VP(avmU5vI+@kS6i!WDGPNzfTo}*3 zl^zKZF%qmtvun}V%ZuM}!}ZAP#|E>Rf0po*m@>`={+_N!lA)E?y~w5Edme$62c zj$x-E?8LdG=FQbo_z~SE$((!2%*KY)OrZBZK;=+p?T@3&uMA}h7m^#7_RfyU82DjU z+Q-P6X38z;uT5*b1Y{%XSmw0*+3(_{A!riiRLEO{xv!lgh2Z%0v6rFuumstzDV3wQ zZBD;cYp?WdzXTFS{k46q`M_^CUXOieGv7Ekjp78_8gBO7RJP@2thW!=l+-bJh;vzz zKQEs?%wg0t+y&+Ki%9AgYsseF_i!$s&l4-)Yz}J6L3*_-mE7AZMI@rpq}y7?eO87h z1Y?uUVktrJMg==i)XdhlSa0%RryN`=)%T~hoJ;a`bfru8LcDvF1aq5kgN5RN+Kl~R zO_fNPXVZ>S)$q8Q=f?2@p|XoR^4RHgoi0+?o2&{%G^4|pJ{G@U5}KDaejx635CjEc z+uyJgcO3rP{JI8y(Pog=lxytN?(NTriQVX$gVoRdsDK(%QPc#R*V0fbY};1hef|6^ z|KL;0eqnMX^lh+LaDRjR>Y=SqJYXtSZD$uJvq;TK<~A!9*M^4XeAdb8S1#}qTwE&7 z@BKkb=pI5OGKpZBr@4samn9g~c0*P^Ur4DHbl?jqq2GSWUXyk{oV0xa_}6S9-+zg? zOWoNtOE@G8Cs5hG1+3mL{udRZi>Nn_%qfd>`>)z21@d6cU7j;u>igZ|qJ{xCM9hvr_wcB1? zdhKn=aIoXFZnhc}diZe;)oTzB7`fS8PDtt$q$vs;_l6eyf znAom(;DvK)huaM)IMdVJ_*UOoNcK1tjPBh@K&StK&6y>JV7qqm0)odNK z=y#9Poax19o~g|;^F1!jfY4^W>l`b4nFR0E2#NPlVvhXeuI^C(FvG^o0FzwE;Eo^r zp$gCanr;1i1&D#)R^J*NZk_SZJV>B~ScOzY@D2di`Krhhc6kPVlcWb^BKVZ9ZN^0zMuOf61(3-JnZuCy_{Og$dV<#RDq0PJS3 z99M@L3qj3=$Z-NA^``K`>lNXvoQ1576KYuwh9R@b;)Nym%%9p$F)0R`ZUx@hT)eWk=c;V)-dzGCEqb}@JrINd7BcpxGe zV%jW`p#FqF{oBdV1YZ2Fy9vFaXiJ!~y?@w}4j)(fpW7i!X-JqM7iE?3#YO$*S>8?} zzO$kK`#mveXTBFhGE17;9O3=XmbS^2|97fW@Co&#asT0aPK}VW5cpvnkYvbQQ%B5L zYR_b9F`f)L$__i6F7i10i8q3)d<_>yN$7;} zDsWQ#L0(gbxMBgc&!KHJDq#*f?N71^Gkku0H9?|j2RIj*MuW4@{`YPT)>vC`Yb9;-?zIYv8nHGNB#^2d_Tfn<*K&US z6AZJ1CCWh!Uh52e?oFwG2aggcxeN^tR^`lXs^^NVb*A3XZW2&^SO9shJu+{Wu2NW} z2laji$n*qWC#dE;5ttn|7W?Q^5!BY2b5!Ggux@|#Acb&{9H~L^h+%(xbu(8?of{Oo zd8K&)%Qh{!Xsg`3GI+LNRntt33*y1aRNM+H)bswza9^`GX!FYBg{n|~=+Re9lIQYQ znH$j^#@Y7@%+2RIuyv~>ENm>#$KTR3#jH%}M#vUkC4EG;^^gC`Y><*E(!Qa1MM-E$?rn7;YXJYM#9% zaBa_AOtsrG#JOiVzu^F{;IBKmY~~A}eoPO!Mh)Ic(A-|M7g7ArF5%yMh5!COai6Kv z+Rr;AgwnrnJ|5@I*lF%OYQ7M4k4kUH^WKKOntby6D2h7o*)$og-jemwGJgL1UXrO* z)xM^MNV51ySFTYqJa{x`l%wC6yEsuN+?n{hQ7bUtSgGKvqH0;UNx9tSlcXh6!-#5D zrP6!DVgxVgPkNy7v`+?F@{lQ{_MHXiovn$3->Owa`n!#p?Dqbf8%t+hona^NTbquZ zmf;6sb4P}!B$w0E(c%@147qtX34UCT%c)<|zhQgY>!J0%@woVOD}zmb`huQk^6gdw zgn7@E=5Vps=RR7!{wRf=r6qq|?$;ny6OPLNqB~edG;Gndga{uw*sGF0M(4_F<#Amm zxOCTh!?LOmisV@vr48$!#xa(40}Y+jD#at6+ltp}OD_+M9GIJ}GtQecwiW-hA`(eY zK*1$}B_GWURk7xF>h_oa)jsu2?B36g?iTf0!Am|pNphZlsw>d^x!=0??v0b4hp2h> z+$NQsk~`^eXY=QVP0rFoiAt*GqDD@gwuVs^*uo5F;Ppb&>qZx`oZR+6BGIhR%p%LV z$E$$WL{A=1ax97s?M8fRov69*ggrEmFwZa-jZ`Dpno`YS)M!^&-MyYrs%7mouZJm5`_rYVv++`26#{ zG&}Ys`zMueSySl}PIJ$Ej=D?|ji!7kx^~-F_%B_$^FKPvfAyBzcfZ3~vc}Pj7)v6)gFAWJ1Ej!|`(=G`Kt5I^d6GEq;XH#c; z`UudB#C!JU*KO*MN#_Bc(^>b8C_<-3DaY;$XdMg4)#}Lh3n;84F zpdgvwX&Pr733Y2y{dy5of&PtgnOPA_Tbx?2bkw{!XA0Tuy+?3oyQ!z+E@9d)$N^_I z9wF2-=t$L2Kf6WA=0RzQGr#pz2a9e@t?}!#5lt_%l+416`kueDqDX*S>E(3bpG zVAT9NTrLOIH(X|X5GpNrbEjhN0f>yH+Kb6dg%i}6rVUHnm;b(>E7KrO0A@Lu-W9#hz1s8rYvx1IO6vOTRE4w?)0{dc_l z-=BT_QB=Twn25W$n|7*}y3W~;-^pRS++TmOOs%hE$8N-5E2QPZKIT5(bmfC>6TIh> zuLPF5w0u*ua1t1-NSp8OnU{KkJwqsiCWjB&5EUAMMV2GUO`i1u+JyHY6&tBfLpdd! zipH5{PS>X`pKBXW=jnuy;{p;Y3A)mf(G|VD;nNmEE1o?Z^)MWbneFvpY(?kzva0WX zBH@fp>$u;(ffkcnHjw>ueZ}QYQP}=Fj9lO#igIl1k>E~D4I?p`LNcs-g(fv6rK-Q$ ztx^OQa~;IIJ37yU#OI&S9I{J>4sSl>`&Jy<%M`Z9cKXYQRX~gD#}#PUspHuqTyrr( zE6?0WT-zu-+A^&pStYTgpa5rXW++It0ICszjPpMV?^!63di^hfx5ukvz;$s?etD7m z&~8p@-W~?}7l~!6rT11jq2c7|a;(LAdXxDTxEWbSGkv}YEL1{+Hr6&lS0qDj5P`TH zT2rk)u~)IZmrJ=XmsNhX>$S1P<@Cb7VnucjncKM%l!07L>Z1nTaf&F{mj^&FSeqi9 z;H$F0Fx6<29YZ$Iw=aqAm!fjP{||fb9?j<3^$V}3UAoX#DVC*0>s-;IH0o?MrOuQj zR4XhkvKs0faR^;bbteM*Zxei5~y6-;Ee)jwBZ@=&J z?lHbG_WtAhM#eBQgzK93b?2UQUUU9_zsoilrgMx7B1pT9F+(4jh^^3k9e~NuC7J|{ zg7=%JrP$6t45ub^^-ja2lV3^Zz0cnKa+MgRI9jx@nI&xfOkGE(5R+qJKnVW3;pP}p zi6*ys=Fv{7gNi;CmxPjJAE7TJ46N@;J&WZHu|u3aT+cUTIo61T15R@MO>OLDDUC+^ z&ss6RP@PvY=fBTgCVszs!?1HX$Z%8sLH&4^!`E}hleKpj*EhCHriN*!&@18vQ!mrx zHoKEFYn{UDyH;{EUSF1SKQRz*L8h6AAOc^c{*;aV^5yvbeslg+QH!!q2J z0%zi;9(>nZML&q{+uS=C8)O_daL4t*&A&VBA5Qzwsv;rAoS z{SgnMBHbpQbS>}oZo+L@%T8Q8+M;##8D2&OX%BJ{?klYH-5|!f!|zpJ&!Of&$Mal= zgl*@aC_yw^XPfZb&A9iz+LS;aRv%g07qn)dRW!6FDo@07&+^M?N7h$E#AbuYHV(#y z=JyHpn@c}o*i0>!LfqhEWJ-mFUWCHjR^(B(_DrmJtHbDK;w;yD=j)p_&1)j`{R#3@ zb}D%9VxH>qP7meQrr#-($gjg|2OJBan`0r{TMajoGJp!d2G@S%?pXUyqv!_=+fX6@ zOMywm>XtD^?xR6X$f!Wjvu3GDO}URIFGD7qQhYp+^3(T@mcFRz7(qs-+6ta+tc({X za3!)uZ%>4J+C}qhzkqSDuSQ;#8)^z(R8nQ0SsPg-J9`2T!S<8e){oGy%I`6Z%$YvA z+)&Ts%fbDKk8p(AOI*kJc5{@3(Lu>;*=htqUxyO9L)M`4cp0hGsyMwj5vY*q6b^Vf zy;;j_A0j7bQt%x&kINGcZ0$3Ag;!jO3cWftki>P)s>C9G5Z0S$Rp04~@pM6#y!Qdc z&x&6LX&%u?+k(l`LI*ss27yc7l7hl>q0miZ4+CLPCMktP-$u zTKf8QwdNle$r%@x>@n?xzV!8h+DWObXdp)t5!@We6Uq^~L%qctovW9=@gLm1_vgxv zlX|!HOy^gI&KcvsCzHSM5Aop3@3X$;8g1#4Fh5u426%Fu_my5*F@}hXZ zaMCx=vWXuK7zHkgxr`cM+3e3DK9gH7WRt$aCp7t;o7$u!oipX%E$6YxJwMPywr|Gyzm{5+1|t|x8$qo z8eNwO47y4JzHfHcSq7zj&^Iz`xhfC%{;mMKUkio4bv5}(7r_;43q5AW`%XsCV}$=x zkJu%uy4|Rlb`+M2W|#_;EDVnqk{pEO8zIRq22F$L1owh;#zOGHJ>gIn#O>9L%)tp8yZ*iv zrP0OqrxXx;il-+z#4^aDdpX69q`p1EEz^UEL+s|(L)AsVvu$|y$*6NApAknauWTes z5FBkS$d`oF_lU9SitDR>1n=CFUtFFt=d80a30-m?s7g|X>WaU^Av-t)B-KiJzrjrj zRn8V(aF+2Y^=T=@i=v#!9|K50D1nz>bSM=K0a>@pyjZ!dC+J925;kA19iPaVM#p*cNAFp^?m%-{H+ynxb z!%jZsiSaT0>rsk@@qchP|6MLuarEq66=EZ_31jlHWP264AK>7DmuJ=OMl(#*6Aad= zq0IwlC9(bbkmh0&u<-*-RQ&~ImAil}%_E>A-mGo<4d|sN>`3u!#}49_+UU!d5Zl_v zkZ>B!w<;;@&|J72yK-6r%GbT1Eb{FW^k(p}N!o_dBr{i$=Nqu#TxAGK*)dpR4By{c znm2%4un{1lGepm$b54L$G8rD4vzD<@q}&U%nRB=@T!-)UWOP;>9naJK=shwGGg$YQ z1JRsa22jUCet&LG6u^XmqVr|72#n zaj(tpT!f8^FKHDoy@kz;qj%W~1cNMsL9#!Lr;Lrp*X`-ISD@;2a+sEvWC~sxE30h& z&O-T!Ls(Jz3j15TQzo^@J6+6SCErnBJ4&Z9)NXf$|7kSdloRow#$hxqtXqwGJ5_L5 z4cxw>Ccg_vl2sAYu$#dN9Q82TkZewKwCtv<{}t>eT2>xk62~#HNY1OXPRE;@=|2|9HoWou^04t+syS z7)F_netTlKRgiRLFItRuaddZo>-KLeavk3|SP=5oT>iy9lPp)!s+*}OY6z(LvZ~1e ztH5^-i8z+ry#ss+M_C@_wg33}-;%ZcBvj92_4WC=T}Hlm_N{{OD0t3B>5$GnQbKE$ zYCN_g>K!!L+I+g*SgA5=R+y44Y|RcrIY$ZZzq7k{{l1O7-b$u$uvKf9Xw8*MWfJ|O z+uF-B^^%IXOC#&~LeA20r#Ps{h;w#oH8!V>%rjJV6df;Js45m62tQ*s0<*LOWlATc zlTWMm@h(4+kUIu-wUkRJAHQnBIisNF)N)#W?%haeqASeU>yWv!9Z)IzY4S|`*Fn#N zM+CCWljrY?BO0Y9>*pIz<=I8ZscFH7%s&e+Truu^O~3H%QJ_b6>R5QRAY?)Qm~!_| z;#m;4iPMAmVxgLS-`+G=rRLcw%@2w{Nf@8B@EMPQ2Mia7-ZX-?Ko#a6Ndj}}3%NG? zohkNvh!jro)#6uvwl1g}b@H2-1meeWmpw#UKmz!FsFaonZ&(00n8S}|dFP&ItpN6dhd z2^~C|Kt3&p?s{h-p(F62?;*rZYM4JZ-yD?KtX#DsF_{p9{c>B)NDie-kNqm)M|onq zKbZS8HC@_=5MFvjC+&5P=RMA|IeDFA;!7ieAa6^1A5oK#M{#Wm&FFV2bpJDmhvLHP z&klQ+jCYL(z}mDxKGGl4ozI*(Ca$7->%#snS8nyH{=?+HaAA3?Ovqn&JM6FV(rxz* z9L}G_S)Gr+-Z!{~*MzhIH^Kgp>M`xLZkVH1e4vU|;-8KQ=SyF^Qd(49{%V(M^L<%- z`aR;2{MRkPU~Hh#>u^hfF!+wznP)@GB%hMT75A?D;hM!1S8ilhVMX1fWo+df5Vq?w ziztlkxg&D-VxPr~j_tlE8H}B^Z`KSE@Qes?U*(;Om`#iMZHvBIYx=dQ*9{wYP9p4E zpI+d$vL;FmyK`*6)Ca?~vEn>1mhh1-zT90p(OvvbTwU8%Q|vIOMYtx(`{8^=#M}+% z<%O;c>0@_SJ-+WU{qdu@L7P*%3U8kfS3*qGy_~MG81bk7n3sq;GrYc^&JQ^`U>x!Q zI(&^K^?cu}Va8!B&5dbY0BKFG9vj>-fw3k6Cw5G}W`AF;5=8_kI*5(zmcRW&qZ6~Y zf2vGEmow)+6YyJbV%yi=?{hN;Hz)Z~<636k+IZ|X&3U7S-X7b(W^yw6WF2`?%HU29@mSaM*25z__nNqVP`)v+98muG zyZ*G52kz@{xpHyOcO>RHf09s*W@QD(d^e9sB-#fp>JTeiZKmeV`K^?^-eD#+(L6Tk zPso^D-mM&L%wu2~@!zG(vTAr!k%d|5Q^cFSWSa#~qVW8Yo;1JoT(x?b#%8ra8YvgG zD1{^L=B_B{luW6BPq{?*U({|sjHtK09rkKR`z_D39D3HBj`7YE0T=lbZ>~7i8dYe| zh&i|P-@t3;n1UBQw*xVsv*g|V_%4?}*J-#btY*`&{>z2WcYpRMe!4xkWL9nLLBBX6 zaQoaQ!{XjIo9697z25t|l-!0bi?CnZOnOd9Gx_s(CXMejzHjOLX6^ZQq3QN~Np~Z> z`n&y^4@JPU)4R2!tnmHo(ftM2d}(^rG<0-3f=TZ@i}uJ?-Spr)LuRm8eA*sk?5E18=9Bk$2G>N z_yz{$_npubqvdvt+uyEjmq>YMGj#XVje0lVu&Q=~d+Aci^R3nZtpVxqQ}B!X{xaG8 zi*?`TFe6!N4mUh+F3jBGKe5|Ih^D}n6GdnI?7NpP84;>0jk?ySdlH(he_hsbv=csz zJzv<`J4sJEtXC~jG{jAn>Pnp6Ua4zxc0oapp9u`h@f<0T1tn7cK<8<{ce)ewTmBjB zV~b9O9P%jj$>~(IZfxi;=Q>LqO$zKABgYvp-4}_5%?rFMmY1OP$0liURwQxMt7u{0 zed~}tk2;<2`15e1mL)dH;tBiNE>rjUU(=Yh2#9LQYHcwkadF$tmY&sTm6*<y8kq?Qc}UH@AIOd#=T=Pa=$R3xXq^dOufW4r%RvmHeNV=T-3;>Fh74uQ{gs`)@1)VtEAp4 ziP=mEPvs_0tFY|f$o3c`r4Hh+2f$<0L$`-LkN5NIg@aKjqY7TH@r|^bv+Nw~5aM#PZUf}DvV?*GVyfb1Fg|JnQ##4%SN%vN!${rtwc8CsaeT9A*h6b#t zl_lY?pRx3bRsd;np4oG(4rTRIvmZoXMPjm}uBAZ}7_=k44+aaJ2*S=uYNp>GqnOn% z=6Vsl*pF-ScZ#kd#jP=`&fm8C)V<2b8C{GaQ%v$;%i1Mjd~j8$n}ls;jLEuM8DG77 z>h9%7OI?jm+Mx?`TF$z<`R2`tlh=MHPb6ymQuIf``*!lxU-8%PpXEq?_YD5jzJA|& zzr#0QiM00M2s>3xX&BI-L<9$3)GvVrbwMvS{_oi|b%&Ryo4uEqO~@h}p#twL zPoSlnV^Q8>;;E1*)2Xm*ewq=vGRS0KPUNQis*#q<99fz_a{GchBt0%ogU9|zn>p?Y z(uxQSyh|1;$&vM?Or0g^susrikfUDIeJWH;tRa#9jF21@VJ&G=S9LDb704%D{I#H2 zr);%sdDeJNHCg>@?+nU^M(Q+t!b2ZM^Z0cqDB+nk%WGg3R6Wdkti1$OxTu zrgWafvs;`2Y=_xq9r=#l?T!`~cP;+)qhHVt0cvKi49u6F=NA?dBQ~yv53Ne9DSfo9 zXu1gwkI{6=LAGhd`{9x!J36lqIrKr0Q&f1$SA^CB(cEsuFLw-?{|pL8s0vd z5I4LeiFNP3&evj}(1Z?s!VE@9wnNpL)5{b>YlhElY9j`aZkq@Lk zZQbzJx@dfN_2($inIq`nFkDDQt}h$q6*h9~n0mDTEzM)1NbIPcen|SHTOvYIr>j@I7J>S!nZmlQMJdc)g$i%gm3(=!PbcBqm!X zpKy(ui?yPA_vnKy*#yJE3$L2vbWK}Ss_sEvu8L!?+!*;JZFEvl8H3HIsMjRUWoKioJVyp77w;xUjhEVBS zggXPa5OAbeE*|J^GrXvePw;;fUogu^XYy$cFlI2oryv=BM-%h8Y;GfGJ4U#>E=`AW z*}^opZZG^(Cll^NT0KI4D;_bxB{($mdz4zmreTN`{ZDjU0%~1B15A8&7}*Xx2VQMZ z$rfH3<&@c9aziauj&&J{fj_w~HMw#Z)xYgY;L%b@r=*M+4fM`RbuNb1WjiJ9mZZ8$ zr|A=u*GjB16uT&OypU>yju7{Yxf{yFT#EgWave1pWv3!J+(2~AkE`jczODZs_6+~6 z241mu^lYQS<>ipA<&-<8{%XYJ+BS$6jj5}<366Ej6PGJtG6O(&+L%w_>~_2+>_-K) z=iZ76=u-5yKQFWvauQg&h>s;jf%1T%5#RhWGRNrA$6~I`z2cZ_+5^q)U26X$`tXA- ztbh2bHMHh}D>F`!7ZQ+jnDgl^*}e+76x%*)E<0VJN7pgAOWX4$4k>MR?es8a_IDss z75sWCHU$_e`hN@s?8u|WooOQ*rVaqMMiE;eitIjuE%>U>*_HIbPVbf z{0t=z>1@89Tz@9A<{~p7eRtn^XTYS&Y6-Vlg5>ONCTh5^MZo#T1BhwcgyTzse0aGU zqLtV@7yr^l}&ua;KzCPn?37m-OimjeL*p!W%0pVF?;IyZYPf^@js|zwP z0KoSD8@!thT7vNI8^8}*1Hd99GUwr-$Ifh| zXdP3E^941@P_K^;Tlut|qY@3zwi>f03Q?}EP^>oP2>A86_TNp=;qC)sJq34Yj@j;9 z*vVzbr?(7i0W?Ymj_KayRq6k1XpQ>+eM47(d$b-9 zqxDZ##XqZ%RnxA*n;pd_O*EJ717_|9NL?1>n)X#vr z3TLd37qm7Q6;@_hiTD;kYG#z0hWMi;xdW3l6>_xj^0q5AAty~e97HTh!$e%so(1Z2 z7M3NmOJ;ggVBEU54;Mbb_|Ss>@&%A&MXgINF(wTy^iy2g5rep$Z^wzy7=_?#o0Fc9 z{+uG;o-8mI#`t8Ch*2Mlk#hM8E4Z4hgUjsbf_4j6kRm1YY@{6E$^#NeV@YO?c6N;v z;#x=tLZ+~i^RNOhLzLvnDD;(mmD3DzOQ*DXBKXH=T(7xg5CCgoYcG3(+zdv`u{#L2 z!eHbfF53$N(s2Ney?F%&PccG+5<&Xhw$HASkJ`y1-x{VLG4&wcL}d@KzKMD>(3*Gs zZ0ydfIRVXjTeqsrW<00M=-X&kP^NE}Ow@$d-{5aBYamnXBh>u%1ei|&s=6n@7r=PW z@`T~R)U3Avy)oNCb}p)04HJ~mVPeBV1!J9)Lz1sDnxg8PgTN#OBo=+>)%kzv@_u=J zsCs=gr7@m2NW*wHKRPf5E>uCbe?BXl@30rPXkex{Un7clozpUVm`0W>&$Gr85F-ZN^&rx+~|hS=EJzIiY7W@_)oss0$UXuBIDf{DapKzSl*U5OfDVI(cc)+0T&_DI%S8)Vw0X(RF%26wZa{Gn37cMl z7DR8K8D^SdpBL~|3D$030U8`Odve1Ls)+^S=RFONG0Q09w_DZ|*u^5>Hlu`RdOHyx zODd2)F5m{YBzGXR8c^qjGcwza`Y@Qt#aATlHNoZf@eGsBE;dHWqO`q>N}Ke0I1%xK z!eCrOv==s3z1b8d7v>6m)0;Xgly4{NhQbCUW#qFX zN|%?`hXU-pr^QCI-^fNk#^o<-0K^g;+@-x;$zpqgrc*<7d~o#sfBG`|N%wit{e6>P zg7#uu6pV0<%e=+_XBlL55NrOj1xeWgwKzuZc11ap#h2QF4&?B`=oFoU9EckhsFqnw zQ?~kM9-;PGe{S;1P!@TYz^~qPEmTKSUJr4pUF)cj()M2d9!j+^1GSkGSGTw`CCnz9 zX#;6W+}y_A@g!k3(q&Lj*+h(6x9uA4j*dWUx}1K7&wyRE z+gst!Z{}`hFnO|mcjbaORN2DAIV}=Rw1+RFTPXFV)2oHPvj5Zq&@e0QEpW=sz?#r* zP#keUc6AB86O(dTV1_~SBOE6tqY!)lX0x3IsFVcMoxkmLMztbVM5;|6)F8PT?-M(2 z3V()M4Y7DIcUXv&Lf>r%ij9Q{kz3wZ+J(0&eOedY*jyO5HVD_XPjN7-qR0GN;DT@; z>6A<^B`He6px@u@@(o!A@to}c^2YselL+G6o_uhq2`j3ZK9NHT-7*pP9aBIU#jjB~ z^cl({*$QmFRqn>4fcv*y4SU5_ccNSR)Qt4e)NH3q;0n4`0+@Ww&>~rHk3ywb6AcTc z_{R7c9U5?w1j4gt3KcoZI`u$ zh_}ipmkD6~C4~fcc7VeN8?Yo+XH>bTX#6&=(Awtj!=`H7sh=@(D#~!kdP^sT3x`F_ zRmatt+$cd!^sOYUqcFxHPdG4qi4->(nl0m>gXMZf=314$#TzucW>089zD(_INHoBp zZ!S(C6pV!n9!pbSh89A;pCy0kc^74`1fJWH&|g5MlK;ph`>U7EGvHKRd*JZ+im(Bl zSA5$N*GR{7J#hPHX8`s@!r&4N{tJ@>!+vU>j&gkKz~`$chz<0tXMz6 z$u7DAeB=CJ$ze41vEV!aC9tQ2Wr%9$za_vIxBL1Yg_KV3f~-zfgeJRKVNcq&)!lR& z)Q7L4OxdbU;O(9|294D7|daKgsxU1NBSr+wQ*AH!E$V=C*)|a^fZk zPKz?~UIt0}w4rYlirUQ8Za(~>uKTQL#ltP#yv)BFO`YV%5F8RzgEdLzdCmC_-PM|F zOp>VD)g%CLgp-^@qevPJd zr@lwtU3uk>mi&CGmG=y+owlb(6m*z$x4m&b#gK@<<1vk~lXrS5BI}yX8A7H!pMHdz ziz**XZP!4QP5=|R2xo*?SA`^G&dN-ep$5n#65%p<#>HwVyySAdHSg0OkbcSR4mRbX z%TADq7_iED)i;ly^2mRap8l`;IpcqO9*Y3}3S;+ZBx}yrYjyAVOw=p9qz22OR+|qX z-teKA3{k3(C*a4Kcue9|MtTh?vp~&MQ>-9aquI4L3)@fku*;fNTauUcf(f zF~$;d>=U7l9kv8;i1k>-kMY%HwO!M!SaoLM?)`n9*SmjA;Ed%-67_LO{~sLxM=)Ic zFMaC7a*?QGx%ypVx#-=U9+W5q>@# zXn@x`pz1P2c`*7Ms8PG7D07A1X}q(?s~&`RkEE)B{h!pJP?hsmASoW5q(KaT+p|-b zJ8WBCVp%Ko{~S}NQ2i<~B3b7o_lr6UlM+Jg3zd~`rnqt8G2Vd|@;^3p8#^s`fFy@0 z2}KR-266km2C`uq%?LY$$rzPDkQ~dwrQYUaCnDO(*K=aAb2J}_%TtpzQKT;Qg9~=~ z_>2i<)&PS7)i~ zNcOJs->p4#r~)12_qlW@(0`AfmS_9HiYm57mD$lwz32o{axmP5pYujG@jl06X>eKU zRW}{AvBe4*7ylk3@mK_Btq9AMc3#sAnEX^+q|bY)dv7tZCWaklzrE zvT5xicBb?f2uF;?%bYq-^R_;hxuJ6F82;gHdq<>0;t`pNE=8mDO{10tzIc3}W79oL zKkZGT=d<^qXM%}af65d`npmFWe7iHYaiy^{f7ZbnAW4ql#qV6-CkHuj6++=g1{@&RJfc}ataUyc1_ieFfDgW!#jYe>0!x6k}}jKAHr z8~ni65-hR1rchqXd8bnKi=3+=VRnI%85?q}@N|~3LspcfnB)*SlsEJ-ULyE=-{63p zo*oTNe z@4M-CAXU>-y#~Uc(@pP|ycRQ8IF9DXy{!9fz`|DG*@M!P7cXpFokHHPK7e!IT#zRz zoI_5W#{M=EcF(Ac6nt!V?bQl)d`2J|3efX}4Ju44Al1>Sr(x%xai9B)BdU7lPrO)- zPY!YCceq^H9&CNXwcwrP{%q|Dez6-MLKTD&6U2Up+E$*we1t?Y8{OY{g}m=7R(^r^ z$iMh{(9sjoa2t?h++$(K$+A@Y#6gKu_u>M6x4LP5yF9DyRN`xi6f|EgA))3EKU}AD7}03q~xlbT7j^&cBdk~KD1kjueCpN1_Scy z9(2u*%2T^~ZthEPV0G-DiT1m=^EPtvIQcr*RpVZv+)0bVBN6%DfmOdZDf3;6eL|-! zJ^FiW_i^5WSufcT{EF5RTIHB8w?O_EMjMRsRLvc#pE?8eoJVkpQ>ccSaakYv@n)3l_yx;T3=13pA` zdI29%oCy_vYLpe*uak)VZT`5a*@c29eR57p7L`}kC4#+3qTwu&KdluhPn&eEBCn9_ zOG_Y%Z~cn1Vp&x5NS^%fR%%yRw%*5orDQ7Nc`?PmdrYVJD?bdsQMi_!IL>(4lxA^iid9NtIJq3Xi-gMjsNq8a^Shf-QO>$M6B%l znkzm|1bn_i!8sjm^r~C>5lZGSx4k~*DM=bxp{}G!`+y~bl~VI~G3&?Q9QBd|HTT&n z#e*fDp;Y`7pjN&#wrLU7+*CR{cq!X+fmQ9?OD!&W}#pICl%|@h= z8-WumLWfK(sQBHzeznoxiUZzX=y4BdydGcs0=%}^-sp&I9Pc`V(eCV3ypdk)^PtTo zg8@86F*Co&zXo~M+eUPn6S9-ra?Ib+1Pimv&z{kMysS4YntDqGj)OghtL>qDHGeln zRj&}PxR3%R)(k8vE?j)ug=w6Sb}-PXTJS&ng=_NYK}bJy&vLsU_7A||u2)UjDv&uP zIrG0x1_i@jAq>pVR8?fDP4Sd12ku{t$_t0vf@ryw>F6*-;RoyBz%7H!8ez3&xWFZP z?g%dKdw!t2?%g1NKz)AwXvKoh`NOB}i{`NwC&+nIE3OvQEWwD!`fulHPqjK5EsYxs zW9H0sRg4a1r5k+KPRnJTJS-)~+j_U6DOFR^bs?P~4L*FFUFqJ285w5i$0ZT?1uGwl zvt2F*C%7{%zW|>c&Ta7vn|M2Q@rl;o7U2x_oGtCVUF+XwiXjpvG=Hped-}!EYiJ!` z(PpdeKZR0$GWN(&2o|PhUVo`uaC2JnB*98fGfLJ&YQVrg;Wkwu8mS_F`bFlTf6`lA$oi+?CDlL6XPHDq~VK2Ei zK`eHeDeV(o=sZY2W!<&653~_5+A;8Z= z^U!OBk1oz7L9jn(3%S}T`5b)IRCe;|vV1w^tXQ>^o3w6qsc4mUrnF_fv8DhzC-5Dv z+sOj!^Hc=a6B?ZY^{g&AEto?>UJiwt}H-L>F>hnGn<{AZ=qlx5i{ju6y^#0VD zP_KN{-|kr&^}%cCL8-~|1cp1X*B}-D-i*6brDu-Kye#n943>RLn)epRSbL|QLWH55 z0;NC+CZ`VJjumCt0jS1iajzJ3k>vTxj5vZ3;yq7M0Pb~BCpkO(eX=MS76@Q=xQ{|h zb!nF`2_jvS?6G>Pub&FY3A@gm7UQDC9D+fDg;L0rnX*(|QyDEu%{>3-&D}2HtU;Z- zVUN3hk5&mBpdJ@1NyA6Z%+ltJl0T7_zE)7hy8K2 zKSvwcF27{ZH34Q&Zg_`z6H8e0HLQ?iSQS&yQ?A><-TNs3mM6ub{NMSmM%693FTM#P zmMHcbFv_pUa$)1uvk<$_lG!M)o=)WK6cjnJgfjbLBcb~ zUVY>;P)tMQWl~B>`1a^jLB6|-Msvx#m!ZIJlb`_bH;FfMt7t+iT^TMKee3uR*wyat zaJv|Xfsu%futi3w*L7r_wtk;uH{B-%wwv1S%I&(`i~}gw{W%Aah}!aTq=bh7y-P#C znKBz81LV$Y5=&Efh@QDfP`^owhOdh_4<}pURyqsLp*+2gQ1P;3=`^S(xKM!uk|-oa zvHv%t)91EGk{|2|dY4!zjxF}>g9GJ<&e&z3-YG!bszSq+*#J0;GsMqP3<8ygX<7i~ zXiT)ck(%!+fPJ2OpAV(ms&J-k0Vbk5x68FQuo}R~+S&0IuLDe=6uqgA0!GqXwbhi) zS98KhpA}ipnEssLl8FqnFZ1N5Botd)NW3yEX3CNvyso^oM88@KO#)whJ*2TGuv^m-l z1cO|PfYEyrgEPEXrUi^m;sXT{Z|#Au3|_VihPAul>DM z;6r@#r>bFar+83D4$iepPK4a1x`bNLE>sr<44=H}DkDExs$!o|Lj!i*$l0cZvV0Hy zY6UcUt)!?EDxIxgH)H6`H1^T&8m z?ohHKm!wy6U4JB(AGMClUXon0x?t`x7v2q9Qiu*7no)+j+X76|rF2_f-()R8)650< z&M{HJAC-xOFvRU(Do-n9iarP;OAPINSj#qm%gtqKN-5A^Gp;7pu-iCN@)=MMFgnkI zVP>Q%c}-n)t+aF3c$-%fkf?Lccf0O{7%5ol_Cv*D15I48CrII@Q_}y&OwvFrP|o)* zi_#gr9W9aMGRl;5!`RvOua{&D*iJ2B-6{Q52AW(WNwM>k2|hp4(nBRng+tTG{^}Mg z-^vh@CnF3SPkH)~N{aeIm5Ip!DtZandW<9*Ool`Ov_-!@c@2~03ei^FRlxQn>AY^> zQJan_E>fFb{xEYkT75XXMiX7{1HxpP=r684RMS*ZQc&5vq7sA(CbFfTsgq&72(bRb zLkD{%{enI`D6cknz%K_XytsGY^lq(j|2`I)yR*J?Mx>%1V_dU@&nug?k*&a4e_NSo z&3-qNgF?{OGAh_@-1Ps5f$~or@i-u6gD?$(W${Ihjs>-EIGG~PwAJT`G0)^*Fw|NC~_ZVG+I`i$ba4yVwgypQkmEiNfx^YLbsSNI;km0nBQM7=3 zGdzSi`TSD>Pc1JMI{6QqQ#FQDm^CL5*Pz+ZrdfzjBDA*KaR5;|j_^61-45U6Xd^jg zs$>^9)D^vRy%q+YOnVywk6Ke>J44w)nxLow$plfC?&yba{iA?FF33LXa($~gp+K!J zP<4ri0vIZG_Cn;tI7#~{T?+)JFb;!@wN-H$G3_K6GYSaqa651AqVJTGU6!T@2fo-% z*1KV+-`Y^RfN&9oxPdpVM3{?e1#uL>MX$(71x1p+H09T%(LN-C!TQPs4KHWCoKG_7 zkwoE9?|Vm0g&)0zcVCl4j{o4|{pT-_l%oq@8;+j%dh~E`rN{w&yz(;x{-vuU)6dnr z_w4nFGt{{sRcYOs!&xRcqeLc@$e>@b=--1DQ1R&K^FW)jMQIFhByST|6|x0=LCFrX zpwelzAq27~4gDY(08R95uJydYqce!fjNy|bl-}v*Q9jVc1l!I!Jh1xRd`PjCio+!f z?dWO<)u7=~(7@p;h#FBk98}vYNvNt~*Y;KahIBrxBRFAS8P>r{ng!8R4tdE3YrkAc$m0{V$V^!bw4lg$*5ph%fnNc8`eVG-Z9 z2Ble_WB>D>+5MwfyMRC5a@c>kvyj=#xeE(t8_ms44U4*Gm})myjV0wNGRFp$?WG31 zHWz&7bRIma6<46SLe1P7=bR}r6&DhBat7VH9VT}R3?bgA<4E2AF>1t>|9{kfzXB~X|68T|DHr6| z&6>!$L8cj>E3Ilz$jjP`mc8p>uGoy+z9hmoY~Yu4D!l8m|-_f2m= zy9DYgGvEnYUNv#D14kp0umzpI+PF905?^k19R;oiH@2pi%^~U%8SaNa12DeQ#orf7DnGDdMTj>wcJyJc%`G|(BgG3 z)gfcfx|_qZbph;UPmZjsMLNc!bG`zC>{T6ECsoKz`HIcaiO{4^rS>ye5TjFRow^!W zo!^|3qNUG(OAoPNbf*uAs72SXo%ygsbuP9M{Ksvo7GS1H=Po-`U?4z!-Peg6oOQzC zv#o(s_U82(o%TX0G-y~KQaVcjL}jjI4D|SsgR69Gn$#+2ezv7UIAbB2>zs|NU=X|V zlSdR}Y1r3bqf1-J;X}dHBS18W4I`jcma`mcNwgJ-K2x13>vY!S8F?|zYgeG>t6r0_ zrf!BR=+4?VB9vn(&Rn=|flC!2z@#zi?K8Ji9sRgtuWDW^e!ywkN_Is$j!6<6q5Ll5@R?Hj zy9Kh&Bu8Bq=-mRXSgT#Pu5qA4{(vUtY2Obkm5KU?*-g>GkE?VLSM%i17NK#+!)jXmxwo>w%k+Vz>yH!eWn5PT0XSM=B9}Y;gA`+HeVzHpW zV_oA|ZOZd8HGt>R>)0El06z{Y_0iq#$%nGZ0Z|>yrH^leEL0XKlyN3cn&SMK2SWlW zs2hL;<0_5GM%A&T?W%(O|6(n=7UbHF*-qID_&kN=wDJMWEkj7Gm0WL=0Bzp8-*uJi zG=3Hb*IjkhIf*tR*EP43D}YWu!x$|2r3JR9M+RYB6tj^^*?N3xy98t($ODw~_wh^X zcjmuJcnM?df?-g=U|0mfK}GV*C!MLmWXo74y4bo2X3InMO-gWhXu#YUx}1H^WBpG6 zV#37;c<|kThkmXuP2Di05kJYi%1=i0;BCzN5z>m#tzRX~gp6s{}J z+j@ja2~4daW=T`83lo$X9al_zeTp(>V7Yqyb##Vda5b59pdpWXYDQv&&(Lv8w=Hn& z()m6p-6#9)-|II0ukT&%9Aw5YQbrfY4Hb5ys|!+=GfHWLy7u-Cb{=ZWo0}`EW@U)M zwdA*B-+3}i<>bdIIDn#jyG0d>m=3Z-1ELt&fTGzqBeOgL`oTmuOV6ykwX94VyhSRNoCo81eAT4XF5+d!ei4AC%PqMD45 znpXwa@m>JQfCPZg(70AG_uI}FTZPHqfi+Rj;Ql)TNbBs5m{Xg~UuKvAI}yq(r}Ga) z{U6MfVN2wCXgbZRP^&xu`FC$ahyI^$ zVs0HgmIpAF!|u&YMag5o$5)XON;fcWr3?6lNIL-FyhMk}!+eQxGpltSS3CwbHFCIj z6ydL&eq}3pRX*0vLREJ!Sbbb>+5zOu@JyNwp^8(Nwg$koBnfH>EbhoGa-#;Co>7oF zNL35ImaSPI{Y$zHISqLJZkTsTPz@d}RRXoCXL=1>1cW8OEa6#uh=%g%S zj0tmol@1%RcGb(yY`@-NMjg~;sCqUYUUtbzLM_oXfI8j01{;&00ShFFZaZVh#K8%B zut6lmhem4iv}1kUkPS8A)|0O9NEqb~z;yxjZw#f(|0Ja@r!r7QKD+8s3XE~b$Dt`k zYP14$j5~}#@XDh55rkzyVAE?Wzh1s%Z^~%RM}@8@bUE8)y~SfcFB*OUWR>iyHq^p3 zfPfwv8og|-&oa0?DXwb&-Uh2s#+3!q-|~!qiCm$*{2OcKg-7R(O_57hQxgS$W=XV@6DYwY|@Y{?FEYKK+ih|Qx%7Sj1G^nv*bwNfSNCk z8c|}1#9ab2sj~9f2&&K~3_!p%`xeBpB$s3qY%LSzJV;Y#eK#L!>zfS$9{V*#&P|5u zc&UmGqoI&ac2KtGq16Ubg?9E^jfQZ8O{zq~{9FgHTS9h-YYX{m&(iQq!$LY&(*c}} zcp*N}!Q6!LAq&jui8oA2MxmUg+dfNh%mF-Nut}CK(Ka%O`ioT35A)Tz)FB+K_fbe* zEVg2n);0OuMzet30226ISEnsDu#8(xux-NIZc|Byn5HAJTXw7gvr?5|XcosW)T^7Z zKK=uM}={okjeGy@?e*K4uNKvQHC%jzH=y$1_1l4B;=I@^n zXmE>7))|B&fq)leV9_$_v!2(us^;dASHPy$R()-~ku_9ReH1X2KSfsLMIvR*cVE187g z)(59n>Fu>sW*J6Oh*F1P$%M?e_-$JSd+@wQPR0a0m`CUM!57_QoD`_CS**ZF0IyrE zle{c5_k4cvqqi z{7FiLxi3a{#faJGcRdzhN<0|5{g40?Nobr`3QB1P2MhPXw`?smf{tcfuXD01e_fRP zK*q}Fuh|lM80>3XpE z%;k1#)r!K?YVs0ZZM`!4Q$CVDKccE zGc{F^vn6=MtSGq>D39*a-yMOiH3T_J=QU@A$*s(uDF(pqu4*;RWtv!g zeBL*f7h~6@)$L(HS8>9VrKB_mq0bTe=HR)u51Xe68f6p`p0x-~US>(ML(a!fbS-ey z*+0%&=+Y!~P|;JPrL!oATleNW*V~a0h3}?OGk-g65p-t}H&_a1pLupx^nm=cw$qhD z2jcHMw^BKA@_x{}@xKlp`sH=(Ou~zHeWaGIbV5S#@A@+Y=ab0&itVnx(flDfXYF_O zn*oZE0b>T6!#BVqTsW7@O`J$c1F0IsiuzakkR@7wuB>5G; z@_a(%$&he&*BkRTl>mFhCIF}G8_?MueDA-}^_Ed>by3%7g(3w?f#L;HWd+(n+M#lM-F>=m2d#}tj=UQ`vCjCJM4Vpl^ zKuj&!4cE=A&`Yc9SvB39iq$UrTKuqhMiPiPk&n66777$}AB3qDQG?z>TXXL^q@T>> z*CSD@E5u*{@`i#Ee9z5CX!AhgWY-mmHEv$GGeAZ6>P zNucS#;*xz;Dj9^645(oO$i{0LK@zUD+y-`^0s}C$bub@^;J5u4A`&_mwx*5R0S$PD>&t$l`r%q2ccY#noT*Di#vZvNG{n_xQ=G zvG;ygY;if0p9A?_-OdR=iISsN4!4N=)H}WmT9l1!v!+SXU=NUQ4nJ#{?Bsd{U43nS z{V5Tzd@B8Pw4hK)cE?KY1b8!W)U%F(>5su>I3-Yxwdx*kN@Hdg+>baxUKjsU8m0RG z@A6c}@OG(7B(DtiSb&wI(&`x-bpPZot7|(!>JGH^{ikx+X~j2e#fVdh{9$X{rENk( z2tzn}SXYJoyz)~^!&UlPepW^0r(@l~MGw8r^YZ#GzCZ79xgDj7NOQ^~jZLhn@(-t~ zRw0jPO!6ZcwFq(xc}n!9#ul3U!4n=|;kLx9#J)6&&MDKh0||oV&E`G#6DLIMwq1jl znd|ZB#%<1ly$KgeEzmIlVc7W_7FyMy*}?Z02w`<@NIrN5{eHBa#>Z>*3O%-03t^$^b4!#7f`JoM8#*e3HRM%@)3A z?Ocl!8H=N736$ z#P=@Mcff&3Ha+%8BPZ$E!6}2%OScH5`t4#5wa=qNL=7ORZ@bwcfS0Z*nchJvl`U+0 zq9aibiv4A{cdmN_Lf&3EY*CG)CJAif`-AMocTp^y;jiRw5q;^}O!7;@tbIAU(fxaZ z-J$Yjeq#mN@N&e;NG9z&O(m`7F-4Em+pe+NlW{GybR=L|NK&f!MR0h#?!A-tMCRG5 zw)?ROx#k~kFZjaIJl;M+Bd9w9O8v`4(E8ttKr5EM!0~0BX~NhbXz7o#7Hzc<{={$l z#?j+!g;>!Ztkk^K{i%QEw&NNhZ)_4vqr^DKMd^3M#IrU~t_rl5{Gn{Czphi5G3-+N z9E(a+cS|y8f})()6QH>W&N&E^s!PS{rE=4=366UT(+$|LwMdOR{r~441U9i0V9FkA zT_VwlX1MDnSNaITg-P6BbPK&tm6@q3! z*W7?SLD8RWOxD7f^Z}J6>HC#$ACN?e$Cv`#(oPmt17_?R55)}9G=d3S0DIYScS@Gg z`3VCG8JHMI-L}geT7B|JhrZF}gp%8n-K#{!OrGLYZQc$>X`34!qcvRZ1V}SHB45PM z(lhUIhZmHn1`j(V;sw^AFO*mZe*s_3mepLG2&j}l;eM^uj1tlc--%hZWOSen;#0%k z+~Tw&vE~a%wk|TMLW3{P)lDdfEkVzIPbXz}ujlJUw_CNMrgxa_r{3Mp#omF;GS=PC z*p?-#+Guw85SAVSa^WbUz*;-PA&@n6^&#St)^k~a`IHP2jcd!6xoWxDEKPc$A!`7H z5W*BeQcS}jWhRJ8ImwoBSY2fg@P!pnjFKJOd(Z=dmastR7sPNbI7y8hFx}C({`)CU znIhVnQ@VY02CQPdO0DEGOJ!puF8q3Z4%tSrBnjIsG-zcZqJ>d;t;|fx>8xN)pQ0{Z zj$rsZi@j{zSq4?KStAV__|4g}{qG(9e;>RjZ@P0BFjJ`+$bG72d#+ZCKSbJV%7w0I z!VVNnsyX-@bf{Lk=Xk`XGsexYx=OzdYG0}@&^*Nf6mfexumbSXoBw=%sN?|~1fy`l z!ym{UK9a~6*gu{fG2&h#PjhgfDB>i6P+6OxiMEJ`M-{3R;hr=*NRB!LzzRcK@SkpN zq?Y}lp(QjjUqHcxxX{Z6WWi<8RLi`DX9IZT&%c<~7Q*wRzqfEi4{(8;sSV68CPfMbR=^Dl)iHMPsZ}18Mo?Fw&B^c_zCTCjEDO za{#0jlYFhF=tZ1OC#rX60-Bofu*TGq`bPVg37dxT;hyEYe5!_uo?eT+Ns%?AG(Vm2 zpVk*rJrvM_uFuw17^%0F>JMNT_gtwT7%Eli_f>2wP;N>6%k#z4+w340=l0iL=c6cU z=&F*#Nen+TZ>jp6GrH4?4@JT&g+RlvyVBOmoV;M`=-+)Sp$O`Rxp z0LmYJz;s{i=krMaCeC-C%j6}J&g3)j8sg>5-4%4`<<7M3+=!N~se zxz8dQCkkVxs;7tUk1V?N7HeqVCX_lHFnxFSxx^Y@-kj(G2eN}=0!bWiGAWlh96IPK zU^GkpDu%PGUYoEsXut<38lM$2blbd2SlfiIE-v2XMhOG@q)3?jW;A$=t5R6)EjF+0 zYDXr3GO3{FK>S?#Il!rq6n3A!vhY$g*(X~gWUl~12&&cLy`ofUQIS@S<;*M;SO;3D zv|g%RT+{+dOGz!(S}&FAG~FC8-R?~Yqb1P9Tk8s)PWBIycd?&iYQ>rk8mJ_UJXFSL zEWl5DEUt46vxpRk>^oq}ze?cwwD6VTTm!MV!oJ^>3qW#$5wank&wf1suHwK+i-o;v zNw|f~_Lt{*Yjn|OBWhE~6^F#kY;UBkcNhD}d!LzR70`MeQH7)no1^1f(kyx|G^0|> zQk?d#<^|T034dMf{rvEei`t)hlV#X+t9m#S95K5~b;TTE5(m9DAz+-W#7RIEH}}%? zW&nm?n_S~!1iHjqCNk@JPLXmc_XS0Fb{{YsI3Zj+r9qd1tO|$GKf*G z1mpk2QGBv{e+vZ3u1n%xc(_jDVW?es>d%_BUJiak0oGm1iH4df4M;~G8}zY*?%^1l zl1%Jeqj|=ELThC4xW?-H^y^iJs7Y?2ZQ3H9_$**l9Y2aQVd28R@(i%cjTnY@4iF-P zv|(gd!qG+&3z6zxOk^37KIC>@50s8A9@`=w5!n7M;)W#ZkC+s^JlT5A#QTB~1Y*b3 z2+ARM%@>d?Qc@xd0TA;#AI%icwLYFSt$9D969GybV53lA z6Oqi&tkSzu1j||zXI@IKtA8S1)|J+Cf$W6~VjN@}Y7a6rQX?o^R}&O7?;c`9^V~{W zYkw%JgXgQ6Th^Z`hwK~=lYcyQA~Gn2g|4*y2Q4@sZAb5&Y!^Y_@yh-o%_(8j1GkdY-W?7VFcJ6s z;z|6_c4RjtD`=2Cy0BM)Y_r3xRGqigsbZXIE&h1Ll%@=>$ zmlA;mmF`b$w9}pevM<@NyUWQ|xQUGRI`hR*(JJH9e!?t01H&3DX(Qu#H|a)asq-7d zF@L6Z@Q8yuE8&3dmJYC-3uA}$^{$VW?P&wU2R{#YmNO-7!^hP&8?-kr9{5euBlqJ1 zCCQ)Y&D(8cXKk8MKsPF#9K7mdRppxv<1}CvirB1fH6}V>YTJn^X<>qKH(vp`7?B|G z8W+{=i{zf@L%rw1oi*QPMXJI5JpSe#boD}-ba!s|$CV9NCdis92%pG7n?x80&qNo7 zw}XlBZCL`MsCL~ycB3l(Y?NI&EQH$|v}5a&$PGN~d1(c(0KADZxxw@;*M>0wuI?c3 z0F3K|wF`;a)dYv&xibW1rr#a?Jz*@Z{j|MVqG4)fx-HJfT1spN?Y{STen=Q6@C;V0 z2G#iU#uV{8JU+=a74UqfS!F7l%6*<}mTvInJ(@6)IEmY~nZRW-|IPe+t>|GCk?332 z!2cda_1$9FFZkd6MfW}w+ynI9@4Cz{cNzLRIICtKT*212!&VKZmSbUYF7woc*EoX! zp|Hf&boI)?t{lB5oeMpeoVY@n7%Si%!*|(r)=u}`n6~? zWvo{*KvVsFdkay9MKC2q6}V@`bOG+0D{wmIW0suq-+}PUQdb3p}4#T4YuI7yHu2 z$aG0i53P&N?C#+;pKCW9C&AzA43MF??rGA8a?Uod)%!84N80DMN%mA;e*=#s-(6g! z6E`uDyJeV7q}(lh3f1D4bm`j4;lKnXbh)<-*_yfYqOMIk73yT{qX#sM-GIR)947KP zh!^wHn{7mjV+;JE4?pqHFB|iUYi^bvO9v>Cdk{5beS99|p6=TbN)?bh?b%#WUN^st znOAYE_U^hE?ut7_xddc*4GpiRNi*}*&f(D9oM%(rSxkmq+1)Eyrs@q@Y^kUKYPr3O zZRNZJFsp7)h~3axjRB-Q?5xKv6vR@eai8bg`_1cO?YD)$*9|Wx1XQk{c;bZkxl7tq6}9%svKfQ`uh8gG85vbuk%eVwx8Ux z{X_66?+4QsQ{idW=JDoOW=*5fasOzcir}-`A+!#EQD0^o!{_cjWyb3hOgxHY1i!D1 z$UN-%wH8a_*vo#m#zTonZT67Z+2+P3H_Uvf>D+A3ahnj^GnUJ;@+=~@BxhQIgq)2VgyIK|_a z0BE|AIL+lIGD)U)cjdgLZbsU+aJ=hia!9|+fSI-l3MT6?e}AC$Fz(XmEr`2)y#i-g zPO&}8s>xHe%~F7r+&g>~rN&kdF^vtm0(OXd!bbRUL}G2FWAGxbk7@q;tN8|`I)CkP zKbLhII^C*Z>!uo_os>ca{B%*rK+7?SBwza1&*L z(Ql^?Q;UHC)rg?;@>k);DG^@OPTjEDr)T<^-8oA*{DZR2N=aB*DKqKX7gU^g--A~Z zywLr-kx;}xm8*Bb zYLABPXf2C`lV*v}VP}*TI2>n~X$7MQ{NbO$xHKMUvZcV3(&gxqz3An~bz=+BJa^!3 z3ypMW*c5cpB#`6?!aArUGW)yJpph%Yu=)%aWaJ^ShP5t>2uy~+v62IV^1uevZSwM9qnvhoD z>+ChK#{B#sh*a}_-0ugIj=oJmV8QR@Iy%CSfbfN0QuTMUQk8=pi-Ck>$MDj>%;)xD zH~zxsT;enoGHrG;C#c?KA`_t)jd5*M;lLCAortdXq(5ihQAAHu~m z%T6K-1;lp(5yKF6FodIKYhCgyGh2odN=s!q!2L_zsD_{Jp?E_qdE-*n#QM zJY_~`Z}^CYv5d3$w{`d5CroyhK*_sNS;MOF7eWd6%bN?v@7{d=S?S^!limWN5Y29_ zoA<}_8^*zLPo|Y5b4x;qfemCTskBpS(4Ve6Dp?>inuaBc@hzJpOl964Ugu;PpE^)J z9aUVWKWw2^(G0tc(P%T%vCZ-6TdmZR6Ap35oF@II-KFhObQ1?iGJD1fie&3at+J7Y zpz{+2?eTnZt5=Ub%Nax<+ZsXz;V0aZYQ6SQ2A6g*L5r9L}Rzn2yEazExHa5XO6+w zM$OdudXFWM=IP|VY%~BpR8D)KycOhUI||PAFa*`)c zp-w!HQAZf>L&YAnEoUjsz+YcQOrM@ZQjQ1S%S(Bmn8`g)k6q)$cQ1UWe2gfceT^Ir zZto00ef5U-&akXKJk-#KZA9 z&7-;d+FoDc>6Giw#_l`ubUpWTKcM2K`mDO4Ss(d}4;suI%%GXGWBo8+|0yO}AgK6| z0VdFWh$vv7WoX%|#f}s}#GW0}xEup-^9??Lz zlNe&&wIl^Tue;gCtY{KF$=efURBHZ9>0Z-Kf z0d((vM3GQL2wqX^zHcAGDnZnwma3G|$C?k?Kp9wtyx$^@ zqj@=dD!GN7M@m-9AWi|Q+VqQOZFGD0)l=si{UC{daUge?CA?#FEv)Z28*bl>?Q+R( zLMQw5!6PwM->+FkND&zA?BH zPp+0Rem-aBYNaMmc;5voT||v#B*=(%7`ylLrZ6BZM7qXKi-k;Jx*s5CD=k?$qtTZ6 zl^ohT2tX`OEKw=0OHrt$f|uqkN=&_$LJRB?wJEGZV`ncZq8OvZkF%j}MOp;lYo`K_ z@rmEcT|(FFf6%HnEVx7%VtDfU>7^x^AXW`N&vE z8f6?0;D^M8E6ej=Ki{skZ3GhGPp?zOqcYd8=Rs^bF@Yq)%b7{mmfDnVY_8a07z+s4 z)XX!=WE%hTl+RMNgooebg>z;!7JBusW5Jf!YS@Fd2p~(fF`@M4z`@D%)IAgHSlH8# z^FEe5SX-j`92M7I`}TO(B!xUv%9V`aW?b^GM@n8thr>UUx^z}sQn=Wg1&f>9)xv5$}?nO?0BButQYAp>8R>TRchSB)HHNYxD z9A)2inxiP`WQ`V`n{u6T+y{@|Pg_h~^y0Y=D&vgU{e0~VWBEiXpnXZ4wfFUVZMYPx zUDs3?Pw2v7>bDUM<}oJV!NDN`p8l5^3{%>fMXAv+>~sbBZN|MO&wG1BO#IO^-3{YG z#Jdp)poQ0&u?)ih)K7tC{ZD+rG)DKP^Wpjsk43Yy$&@j>%6^7^cnDat=vB9}-V9sy zU}G^urVtmR`z6g*TUzkj>JW`+*X=WcG6{6mk`Mgh8!C39Wyh>^aAID%>faNe-#cEn z`Og}G#miNq9pCEw8ZMzRdOhP$5<>8JYn}(ipk_~hCd~`v%Y@b_74^WC8@P}ti=Syy z?rmbCj5;*Yh|m~J^p;6J?x|F5UO`oJMC$GREk)Srd~J>n!)<{kP{+%?LkiuVCm#^? zj`IdW%OrVqe9();l&^&Ysmo_5KByc*6z%P7SBRBzvN>e&GHWuhT1i3Y8r3`Ty-4Mg zAXzS+85Ui_Eh%i?8Z;;yoo^oqMTt^LD4B3b5nIK{U-2u0j-VaQ6w!j0wZOV9Y`mdn zUMOYA3%C19x>}P8HrM;zlQpU*U4jj=wVl#^CeuE6-w!!#v8k`XNL%CflcDX==u{qm zy{0;?J~vOY_YdZim%CxHz(jkQ%L)ypfH=%rdJ&`#LBhW&F1J&+5ApYmS|4_tVx==cVBY z25P8|Qik!3h_I;I95+7l%1)xyqSzC`?U%`Yq7rhiY#pQYtH6Unv4Pit_YWAUWLMv9 zTCQi|2vJ@pMDVD*OJioUD&rXx^oh8{449+VmR;%niP~zs!>E4y#r$z$%)7UV2~Ovj ztd;4tHD@@9>r}4`CR0Vb-OPnhaj=KwH=JdaB^cx18cl@Ool48QGb;kqcH_uv_7SXn z1#YUgtLjY4cC-?n9MxkU+j7ErbrsnU1m8HLzgA}tP91h`XED%Whg@m^N0w>_!fd3g z57RY%@N3qzwC2&@Ps@hT_-w2K?cP&`Y&VAt9DeGL7YTdyJFNn4W(m@0afNenr{@6> z=>XP?S9jR*(_&@q8o_eADT|9CaWt1>G<4;a7Z+ipf@`15Tz}G*cQPAA*8jGet+wh5 zZSI+HLYSPjN0a~EuT39_L7#QoDMVka*=Ep%MCw`IQWQG#yZ*rwE1O6=e;T8QiYigh znLv8Ayz*4XWzB23qObBzc~B2Z+0$jl@TtPz&ls5r-~=J{8ZX{1HjP4KV>!_rMg@g0 zre9$`ntFO|4-)WTD)_45x7T_Umf?H4e=c!Q_&rtY-aArQk@Zbdzel zjyRc?@;6CXM})w%j+fABM~nNIZq2dp+Gyj=u)rFA6$ABB@nKnu?F$hph&cq&%6?f> z-Ob1H;6NPR(r3v;IH3gku<;%>k7Xe*viQ>E!WZn?Z@6@k#Ubw0a5t|T{%3kNx6S8O zD744y17$6!K{Y1JPb5n(zcMWH)b1#FbYaV9OMc_4;>;U;<+QC2Jn>iysJe&*h$aR? z?n$sj)nK%R@p;#hki?!-jGgIIXanqaUEwTH?d&4x;X(qzy1qbqPHn~-Ci6S_JV5Ln zjSTfvQN5F05kd{ft19GQwum|s8xm_ibwo<>Bp8b zI+qeU1y7dWuEaH;@ma_DB7INR(pWI(?h>EPkxe?_c%xf6CK*RKp=I2!uLF$w1}pL2 zi3n(0(x^+a9X6i4a!K-$L+}2e>tAA4HxV_u)9vC|*ip z`iU5LrJGu1D4id<_gW+T;Kif4-75QUM2wt=8LxB6Q&YAdI}a{gwUitQjApX}OoxoN zP!>Q%J9U+;32RzCZguVh2^`_wmO`F;Y{qfDy%s#+LGRvh0>@7l27bob6~S14`SD-^ zk4GHF{n5?xrm>cV(LL1sk6;G%$%CCGTwjZS{$dx?$RC3BS}PUB&r2jQBs~ zaULRu;1FR7)x#;=B`0z>?_d{z)_ej2U(gXwRFabf2nPoTp?9{$!$pA5(oAY;o-q?_ zz#xm0w&)_#w5g`&aoKv!1y^mQCd!|8HeP#r*WHM|nA3$=E zi&mNVH4(L=1UlnN^6|JZKmFn5wrvch3-Yb)VWv!j!f*2;z@mk7;9KrmpWB!AN|X?m z*-`Edf9%$?02aR6Wd}xqhrR5mi^IPA$|p74s-486_rm3cnb2!gJ+6~Gf(g?=4^Nn-*ik+fM!kZf3Ggz!QLvdc;bRm!F_n_OMH zbsDIY_j~aqoG5M@);@LAWa zS+>{9I?0Y~1Q9zQ(B*Hl1vRV#MLEny{h?-K>C>fg`7#UEdkUX4B}J}(s;c6EyBc6= zF7VHdQ8inP{yRKXhQd&7j|^Gue`I>)*9Qv@U4iJ$Ai!?Cvh(p$;;+fNHMe{Oy_F`P zm!m4LVFjpWRbYjzAW3{5N&{!8o`+ktMDgwUH%@U|H)Dud&CQNaiTl~cAckpipM=p! zV{mhH=8qPgq{Qw_II737+}3&MEF8H7CuWb2_btGOy9(h8muEV7;XoMikbk&IDc3d@ zDfdbRZ&jPn$6V(dtitZsr^6X%g{TX&wIv0ugkurAM!zc>D1PtA(!*l7tWSeyowe$< z%d61`Kjfm|i@W1Y*PP@Q&|{b~;A#IN6A5E~z3CQhk65sBiZmVg&MXHq7 z(S-P`cONkkz*-4ac2~;RH#EhmRgRcLEXpm%%>OnyKYg*;1Xc7@M`L#peoO+Kq=79v z0P8H2_0(@~+n$eC_j-f_5SR#H@mVAW7_Qr^9zOS>4CadRX+N&N*R4u9BqnqJ)Mmwu zy<4#*xW=*|yF~uTe|X)dLke2>8gm=W1d$8C*>JiEUT%=0CeVCY zN^_aSSGG%Y9;7?`Zu4c>Ngyorq!mUY)-H-lB=jX4`(SdvZRBp`%1L>0qEDI2dxtqa z$K#iWNTQcK(N%zeHR2!RE-A>6jw61d4)ge!w{1EkkJx?4mwoH(CQ`vl8p4vo-`uWNu#NQ(S43L(n9j~MGYO8>$9qIa z*3v-Y1&4kearo)0z+10!JDeFi$M>B$V8Yj<5n+2_6S%Q!*uehFlFvSJ#aH$^;~zAz z3?Bh2l$xBVbZ_3t}7 zSr0!-P0lRANo6(1aWgWD7qK@uxo}d(DS^DmHfYGOTxcF%`w;`qo2_td?yUzJQ5yF zG?Lj)B8h*nBEPizVcZ-|Y+bN;g~WUq{%*TEEX|3mU%=BBK|1CtvIMX$$ZB#tS*C1= zH;(=wz}gIJeN151S!__IIWWNc`2{PRrQk>%5x~GWx39yk=pD_j^r}8;YJ6b;9Op-l%IW(WP zlW_y??hu@QR9QPbUpU#&I(S(7N`B^!nL^%lo|(D+p2|^R#Y^JXBs?5E~T$*VKA4%O73I zTo9|IU{%q%S-RyOolfHjkXTxp$D5%h4#n1K(KYQj%{n^Xd8Xa8r8wcIiCHIIo$UqD zpFEDqQ2;7BoMfCqJ-8H-Qbcbn6iajxf`)H(H=QqFLKCU@97+29Bw|jtxHtmt3cjf) za-uVxGbR4j&L85wOCQl)xijRJoJbS;vHe_pW8Vf|PRSdAd+{Hvfpy-j4!a&kee7u? z3%m9k=yoxtkw7{5ou}2ia}&l2$zCq~jZ9;Ls8FU(UI32ouq@vwt~KE7&Vb zwkZa zt4K)g)NNsVd|&z9bzw3@;=Pw?RobxIpB?#bSbrR@mz$2ITt`?pryTeyl9=pBij)dx zTxI&!Jt*XSVEwoIu`@DFg5n2Ud1o;MLB;r56(rCJN<4@h`zdm6KP6$us0%IMVR8Me z=49X^Gs2yCN0WNnvuA>{UwFNoX|>JMg4iEVay;Np4VDv$0#_c8uPXLpLcRd#jgoP! zy-mnH3oxviR+}zaEvnVsv?7e}KT=c|rjMiI+WNS>Wz65HP!sCyF;a$t!NP5`gS+3S zrL$XHDS_StfKJw#i)bk->G$!rN~~D>aTg^8O8ZM3sRq|1;riFLSBzq;iwo6%Y}=ll z*`>w52!|!6n(f^0*9hFq8_A5Eop`#r3!@xKR!Z3#4&);A?gZ$*tHJqD30hzu6p?=+ z7$H+>Jn??&;dq=EIrK>cQ0)X=;XCdG8`)+&EOys~(PzKyW>=puJJcH)8lN67n25`F znX{b^9ysVJ^Sa{v#yiw?u$5JbKM*>VZc87kz{0m9e7aXsHEOu>F}-Qkeu$}AUnqYF zHrJ$pwg~ioAo(-FVcH$Y>o%Rv8>F<68U5B(hy}S&xIOKMlVN!{&Rf@a!!qcExc!~| zO*^-g$Z!q#3-$E&{RdO7|A`CJ_aW*9G5kC&5(r^_9X>gs*9?=f_t-(HV0>5oixH17 znznVVt3EHf^TJ}D`WjX}O_t~1=$7lh9oaKzIvu_1iIe9Kz=ETn z`(4W{XwfFY3dfkhQ0~d)iPro?xL}!LYN)1^G51bd|GEmjhjo$yQ`4QnsBwX3K|x{S zIIpZtT)3p%#GWPk^G9k1$eO|_3T8KjEOzU`L6Le{+U+wb2*Mvm_w@3&7LO0d&orsH z=NODE`&(>4jWh^D`e#2kFf0+2EDQ{*+oAK$)d~kQRYi#N(fb&bfQ_xJxX7X+G$oFO zz4wXOL}-#m60eEamsI8>30F6EQIVK+l}CR6I#J7jVxO_Q^xU*H*Qaiv-Ysk=VuqFy zOPBIxx-^B7f4@r_W|ir?H4M{{=kPE@)zprEns`Q?voRfKb#WJ`@p1gcO*v%wv?1*$ zsG(IjrFRc4mgnOzI6lfe0(!<2glf4qKWtk^<22HT}U>NwfU2gVK;q(SdOcjeQmY@*r!V|vjx}mg> zv@5m)2}V;<47ry?ULV56rCI}?o~_ED%av3^AqH0;+c8R;Z0<-ruJ)JIb*uZr{7Z3~ zJl)DpR{{4hj#=3fvq-U`{*_Yv7mDvr*JUhjD1>61?sX@p!eFu>Q2zukH4AG3=5uQ# zzQ>@SnweX1rC;CCP%~OD`y1g8)RjZH!39~2jZ=Rjn#(^ur@5>>_X)nlkbGuQ6Tko6 z`)8k{evLazsB8_m6N0t2>C(Tfxvk$oDji5EX>l79FeSWsE@a8Fgg95741K22b~eLx zb5MP6^!W!pex*mZvHsUxL|($-aZbfp^nIquR~jm~QGw6X7HJ7tnREi=Ag-;u4YY(w9!?(I%NMm92c28S_sKb$qQbuUTt%ZyCPxH)6%%yzKwU+#K>a_4VEES!o(S2Emd1!WrsxIc0S z<22+@*3>WF=s3=DI zft=HOwSxmBEHH3#>LteY0#>awY4$?tG~dvjd_T8_HT9aj-=@A9jEdbA`Gibv_Qa+c z@AkSs{Xj*K{_%<+=7^uK%30C=1;KrMpwvql$!GOG?Pe^i;goXeBo?2Ye!{4K^nQoa zlrj>B9ADudN>6sNno0Ak;-HLvZ%UoDi##zt=6t|uEj5<+rr2A(=70``{!lPOp@lpV zN8rp4)m-HP`6TX`5RB=@VWq&o)Od*-9$UR{_bTA+79_6vZ?$6c#**yTylycm=ML^r zuF~k`m9odnm092prLp>t~RtA@PA~kkRt;sVRQ2*1pmHF;$i5CAY z?;I$dGa#~%=$^j50iJCl7JbE-LNdm&XzAqAoPU1zox;hdy?)Gc^V$s*6lkihuRhoJ z`Z3WUJJ2A0vp3#Idk{#ao8D}2zIfmuetVW5Y4&)%6*gz%SYnqyOn~GRrKGde?h=Fx zT@Y}{)X^WpoSQatKGd`un6SaYu$IjT;*ocZow^ROfl9ORmeL3_fdq>9c%qPsOOJ2?vU?yceV$Dqsm z4^q2B23=;&dtIY!!jtR~j2Fj7qhB3%q!V2~ToXY;RUWj@-HZ6%3R&<)eYm6kcUg?^ zVLa(8aO^$(t)Dm|r6?g%a7m?vF=t9WSuOI3L8M4he>6;St?A^)N;++_1&8Txmhwnf zOWkG3JC`6sFC7Xa$$q1!Q+YmL z!j?jifGtLy8i?6Qim!=eWa=+ZoKSi#Qa;l*w#1C_R%y-^7WSLB68ab@5Cpn`QSZl7 zpUY@6&OZ$qFJq@Z@|t!;I<1a$@`NNFcLriL+%ye-Q>?4jw<}CIa(oc~{$n5Gz98Uo z^0(WIZxP0Nm|xm87(6l}x2jz;tmzhDJzWMmUdIW?R<$#wg@r4V`OleNaY$^xn+n|x zOhz6CA4XL7RVPkMBx_5_^-UAD3}yp?>izo!nAv1G_k=OwrQ>2MABX)wDr&et2^oTR zzJ-fwH81b*T(3eu!@ZR^RE8)Ri1H)EE9d>uOo}UDl6yVmRKRA=Mc~pnJ#&7hb#Qgc zI4a>{>u^f8$G$RK#qigQk|<0#1ow#G+U95cvdu6C@XyRl9z_!9xZDkH{8w4i7G{>+ z`VZx0rpy~0^P!iEu??IFMtR2v5|}|HS%(V%A5c5Yb-nLRr2gC!HXHmRM&dyu+1CN0 zOX05ZXRit16viI*3Dy`dxtn6Sx5ca{G{&^+S{aB`0V*HNamAdy&TA28%W zYbmK`j-~`w=Q-?1Y2dwaeob|V`}2*X?2ierx#`S|L}dueIWWQyZYWmFw)yk=5w>J& z_U#QTk=Nbx&C2b%->1gczexBZc{;rkZ!g)CuzCvqd`C*ADPB%Gge$?e(SxzLNJnjg z_+)N{{$=fx6#&cO#6 z+q_RW6K-<0#l>}UWM)~v zijbtOL$3#*x7hAEptBB5!R}%}U0UQ8)~vm82{DRms#jqP+F@)dDY+dl8~aqw-E4OG zHt1Y~g~SxxKdXMVD+w9ga;Y2^pOWd@kS24&CGGzYZrldp@*7|w<=?7Atj*9Zwt@9*w z`PMfTYVwEg2b|N!KCk?t2e=>e^YTyc>q)HI)fTPQjTd2Ezi&p>83-0=%GPFG zKL#JyjE*~eWRcZTacyO7X=v797;X#04fXVkx=lQWB3 zIcYd7wS{+tx?8i_?_7~jw0az->k*zG3(7oShdXU$Wl{8UqS~Hmy)(9DenzUidm*0L4!ReONGkR%u zTmC10QYBw^vZ4t$J}1c&>n&Z59nf2L%^n>M1z5WRefDgcV(@dDE`I5kjXVz0Rchk) zp1jo%jvDLwSC@LPp8fst^dp*q%4`X+!ELwMsf=myNzKI6M0F+a*Recz^_c3)5=Nfq z0k-N)4X{qtR=3URrjzNEasKWia50zb8kjsaPkV9X7~b2}j=Nz~8zBoi=p#S8U3U>O zzn}rJf3%%3rXM6WyrisLR9qtNVc@gf2}t|2)YszBd^pVBRx{J_=G~UxP+6CHUY3GYftIc+6XsS07leflbxFNa9@Bef zl_;dpVg9?EIJvC+jxopQ>$%5@@J?OYVYs%KnE1<;yaPen2IWN2a!Yn+EIk9PBGx1c zD{04DGw_1CkQMLfcDp~Cy*SLJ?_ z$`ej^HXI~IhMuj=5^_RX(CaF?#ycB_?lNS@E9)^pV_$K zBcfDiX)EYjTCqkhg)f=sOUV^TJui6cbSfW1`5?*qEfWV_eQATEJkPzceMi2M8)?Mb zH7=_;c7xFMIjG*)&al|*&y%Jx0WbGkQ!Vkg_OU;)1pNq0P!>zQJXU=SoKd#eQ{{}M z^1p~$H=Q4BcTaeyKBVk0%n4jyU3i>C1+soj!+fJ8V|FrKI&;#cb-b|0P&I;*gttp0 zpK?{VWZSl$6=rVz-Qfq&AxZE)X{K0BHkI`+vx$YwYLmO6f2**bV(-C&(s9tAqc^z6)Vi4RjS@fozgNDCR0ds9%03EJ27x zF6nyXeW~w*EiR1q6ebq@yQqDcO!Ak0Xc1X{f_Cbx$lHDoW_-LHvG3%vudQBgqp&Ax zEYFJKyv9YO;){*wT=ka=n4liKi)`5nS$-94cbw)%T_r6##)AI(a6Dz3+-Rz;lNT8F zcTUuvX>*uJS}*WmxC3TEjJ|ygVOaAr=p4Q^Z5Zlh`m*)alEF&Vxj*Sy?s@W@xDkMc z>aM4$FidJ{tZjeZb2wXSLOjlr>M%-`s5|)8Iz^X{-t~A!oj}y|VIwzDZ*<$xM$CIy z$9NFgp2Mrhe_Y!sdjkzv66+-c_~x6CzEm{xoetGvAxH71U z4CJQqy5y~NDXT5*o^A6T<)V(L@8ou*qW^C5em18c$Y%OJphAk!uJVr9y|}{z&AJ^Lbqqv$?^k$`XIKAzpSz48Z$adjk$NZ>%oCrH}l1)n)tA;xZ$ zsNqK+7>o{i?lW)ajW9w9e{E6Z*a!a3-+<8hivX9mm{8EFtK7=|<_UKFcz(RCBMZ?Z z4A{OU9;E+tFmY;@J$g5j1Ivd3O-ah^i@}Dvvyqz5Jsl$8r&vAdKKk##@0S6>k%}dOA zH(9wh(Wy`Bv~Op)7xPV%2d;6if41(J8geLcTQ=*f$ru2mXV1^~m)mb=IN&p9obk@s zv5YL7dBz!huk_jHo`d9^&wrk78!vQlUpW1=(;j-{5iTqGv8=nP`)^(@6az1kkjnfH z?qUpBa7$9{ibpoZrag__tFF9~7`Qxb{1zrzse=X%{KnN+bG345Klt#&+z1Xmf?BG>KG_Q)f=XJ^eDTuS0sAUWEkm^otxHM#r3H)UwhKtNKgM(4z=aXQT`R_+r{6=HIk za@pbkXtt00rnN`PP;?MPd301cL#>gK)kIeGju<`ATwv}Fs%F_cdOD4{Xr0EBM-Rro z^S$rsnacCeKD(7gj+E*>DkScAm6c=PeOE6O`;Hg^=ysO1N+f!;?sf4VS2Ln9W%XFM zjz)GxPxmo0%8L;V{+u!7yuNJ*Vp7q*^`WO9xOUn4sc(M3RJSNn=JC~)mX#keX8$pl zpV*;WJI@@IQ}#k|<(At3aL6WsJ)KAQ3iDt7^7NBW_UqsO8{hmU*o8dSec=Voh@%(H zYuBIs+~?5C5J8S~=AiURl05){O-h;Yyw+EH5MFD@=R z8$9e~#=H7dv~O_1yAMD5(T_NF=mOE_q1^)^4D?h1TzB1dXPuUL1Y4xGVsB z=bd-1zWQorD=lao!t<}_e|dL{rD1~)Sm zPDaXZxZwth@r*Ohu%jd}P%@kwci(;YU;p}7p6{V)PMmqJy$NiR`IQ+x`PR3-Md9o0 zl`qX>i!jrw#>x_M$P-AuPRi&Tr*z?=D{I)MwAC2{_uW`mb>Gal z-&wOTTI18Dd%?-h7YlvUM3GasYE}-w*Qy#tV#)s#vaZT+fBW0hrcI+?`rYq-m&>hy zBNY@S%oWN9N0@gbFon!pg)ne|@4ffl^nBm}#wLI-VFb+6^l~-QjOrwLu55%5J zhHH=&4Z(OrAB;529N~qayO_%O?Qi47r#4?;w2ol{Jo=EVZRj_IS&UNAP?FSjma2fP z%7j{ivZ5m5(4V5#iZWr`$7?tzz1$oDT85MOjmUCy@WR;K#aSuW{KVv%-yK+Utm4=j z>5soE{qEz*O)C>0yp`zGiv==L=iZd~%~{F%Et&hTOPp|JvU9)0z0THA|9j-p?hMXxFLem@l~tZ_$+Glo#u(D^mkTCWjt|BK_nqQd?Kk%u0U# zrbPb{JLiZWm)d@xkt6%}>&K`r^$#yjPc9g>A9-Z+hckSfAEi)NyY|HwR@`xC?Ya$x zX{r9YeEPI5XP((^&pnc@D%|7K9o#4-o!1(4ie)nYz9( ziB07U|L`+|@zkh;XAKLew^4~gRPa+cb0A3Djw8~~Tm#ufdAbI&~} zHy2OdiIOr>tOy-Dbs98yFh<-yes#+&RP|X{lpTQ~lV9I1VvP8x(W9S!>M6B$6sJIF zX!m~giYr**41IP)1N&t1h$D~u`q#hCj{r$LUcL29JK4nLeb4nP`U+#0|;BRTea zmOMx%`N}CBtNsH9_)wK!PzqK1o^5(xH2kW+PW#Q|HDg`DMi-_VLx}=?ZTZXmJf9ij z=4gN)xL3c^uQ-80ovCjzN5eg>XI{pkV-7f(c^Tar<^#$jcrUmhbHB3JOUat|mtA%l zbCdZv7oP$^$xWFT-g(%d;hw5(+wQ*Wt~1U&lR|@pt>FX!9c2e)B!2jgXL5j=kGG5( zUTIW(HrdK@KvsQj5%L^+j5W-YL#yiAEwfj@GI8aDi?+N`S-*h_usB)PrF_`nPDc&s zd{q1LzU@l8loYp_u>8JP7u>k9Zjn0~RwYV{+P5v~vPeQ!Uf0~4kR(BNa}fpdt< ziKbg#v!2!=Nl>vNGbkgc3eU$$sxq2z*}o+p!hFP2v!RL4m#W;vBCeKw5f96NW= zs6lPIy|Hput@~zV)1w1)>fI>1>31yLvTppE#Y?xW$y-cR-$wD-M6e#isD!lo0y0@( zMhqVfVO)bE_1`G442OMw!XOurw*m;LOFT~7P1~Xy~KKNkXvSAL9&eA+Q^`SRGpmpL8Z45Re zR>lj2#~pVZYutK6D_F4CTV^S3R6Z3Y8z&6aCNN~rUc_dY4ry??U4F1=`SX2tAO$6m>S`*F%9R?baLe8N2@*zI7q zR0oqa%$bZ_I7f2rendVxSW?1>`3o<-^z8G`b8b`XGXH{)gCKM3ie{4Y;;>sY0>we2 zMlnajXTe5pW%%_IPnkNE+Zi&GKIGH}#^3^t>37GC<4&sS8^W-+Y@y5(D%Jt*(y0@V z3W%TkAZO_=-MjN*2oF5Cv9_wHIP0^Yi% zsCxMdn((dIuKRzAiXeFk4)}JWz#ekdZoZP%!4)mut_hNhkEZQ^6%Qf+ElSf#*ox| z?!J3lZW=hpa8Z%&KWfyZi4!;HMiW$mo(h3REOn!cJeM7+_#R!m(k14VJf}CCeoz#H zqD?7~cnI-u=%eD+z_9p%qecmX9;B6pmkCK!q*NC@PC7{pLZau8{PNh6+I5qhC>5IPnCr)NC@7uc}@g^=~>a| zi+f!8x9JHtMp*%Z}}s zo0lPGI058sUWp)C-q~{^_UPG@!57}0A&D=x-hA`T73JkL{^`vP7&M5_jop9WeasOf z7ZE)BFf3xq`a%|Ao?m}Rs+T4BFgOfl3zqFp&dw0ErQ10fo|T*DytDY$57s}oxt5=w z)2O6R#r`9Eo;$en=+13+^|T=QVVOabNi(ojlu7Yf=JotrTQzUyhG%*%+wNK_ffhOGU9WTa(#Wfi{_hl9xp2 z4lP))fPMl4U3g_^fX7EX9<;*&jD66prK3pCCyZ?I;>Apwr8{ktcr-*GhAWdZNt;bA zEc+A4hn<%T2pRiey2rKGUdxGl(M12H(>NqeIKIaVYisO#Myq_r~2{qotCZTLnZ!yF}F!Y~k;?tk!!DC)VIOKO^8qOhy z9FjMq#PWo4SyAy3y@nj#(~YJV-fWQ1EuOlergG^P9tBhDHBlWGrrNnwikllGAYd*oQizNO?VniBUo}nP14QJ}!@W zzGQXz<(Jz*mORW(9u7Sux%**>CDSuYrYGmWpBT7b^5{zwGya*_vN1DnLbCsX$-xI^ z7EDOYcrD3(=9>vt3ElB6vv6W^&O1pu`sJ-spSlu)%+eW&rBgGl+9&ooCDFs(1v0PS zncTW5Q&O5Z@}gukma*f05_J2yqtg{<9GoT~El4hwQeMJ}mO)v$Y85TN-FMrK7e*K? zpq`=8%!oiU9dU+m!Z1G0&%v+)H8zGiTRa1+gmSxCymTq|R=o-0li~zwpDB|k^LT38 z)~)5GrEkCU&Py-7#EHQmCJYE~zWw&JXkHs(b8F@y1YbO1n1ew{=1?eWapau*JO}ej zfdczr)+`2Mc)bN?Hgd!m2RCJ&tbFl?HE#J%d*S~niK=)+l*gpAI zdX`2lr^mmaz`mSS)$KlWU@YRT}P2I@6bJZ|B{A1(gSoNy;*?6#U)G^0U#Z8|&6$+0R zp{sL~Nm?&V=-_FpO9d1C+*I#evm4j0Z5Ymo^m3%VttG6iRQZZSTD_ICfyp@9E92Tr zjcgB{(qxS!!Jw5aew5c}>2V#qcksCg zV}=2xnpHLz$U0Y~mSgZAC!;*mW+)26grQOXUqKu(L3&zfx6zIuCYb)VLU^-8%} z4;yReZLMFit#(sQx-wn2by4lum9s9~Yr_}z?|EVGjw5*hP+U}cME|Qhmk)S%$^R_g zHjb~|XcmUNXWV092HMIR1~`-u?TJ}o8=Vtfwxdn@CNUm{VDl@Jv17;Hdh4xpR_O-Q zIMbjb-+ff7_Keuk`e7uDL%i7}kStpujl2V(Kgke-0`q&N`aObVYwMY@QYw2f_}^W6 zl(ezv;h<-}J?%nly@>>|fwY1SEr1pn^Y(cu#(Vdva?M_I$jPa$4Upoomz?!t%UMVt zKQPblWnr`@l^=zWIJ8Kr9=tQ~#FKVjW9H$WJhGP|Ll3+#MsL$vS=r;|S4rM=lSoc= z9!k@>;(a2T?Bt7gJ+8Ut8dil=&rRlHwuExw_X4n|9N4h3hB}lUjFOs+hg2F*ERaD% zinczZX%>bdV?dmfbl_PQ!8`Sw(pVFmqTF8RRv9nm)1<}j69yh1{=Q<6p20}KJdvzV zF#p~0@L?5xUSaG$uVj3ZeCg0erimbk}7>%obWT$c)(3@F&jTIOR*3!ctJ$kf{R<@p#B!FZTuvpmQoX(Muvm!5H3Dm2a_9%6&>z9jpcqURDoNx4x6(ho$# z*eY>~&FiU!1F67RDdudGvZrnCik>RM;%`%H7BQfZ7_d)*@0$!i zF7@8SnI+RQQ(sCPcv^D6{+ZStlhqs2PyD=SuVbUTIKjFRQf=0+_1KU)rOtzM}NsMB! zr^I-*q(6_Uii();M{!XD^EL+qb9C&P8^tJFeKZ*dFlQ0*6!@`JZXl`q#*H1TkB^-E zsZZgBkTA5i43Tij+;{)|%a$)^V1jucp4{f8p=4;g*VeLR2In)cldN3L0ILgzi;n4$ zxJGUhtxC(+_&7zWHXnHKL6H6W_2YiQ32eP%D=%Kj0vrn#En+SP8ZHNJ?kpPe^fS+J zqT&KPZk0hUB=QK14Lx8E#b)7Lu!jXijjwC6QNul?dS0yir1n;7=>eKK& z_Y4ORl}COHmd|9)DdbEA;<;!j)!^!Cug~L`-YXGSZmcOv@tiq1q-X2nM|3VTvolC_QAU-&dlbz% z;re6^_X5kKIJx6{r7eH_HdJIDkrQ$*oUP`b&@UK2D}WV zZ_e*1q{<&!%(LW;neVytPEKG)p1Wyc$T>M@7VkJn?K0vSkO54eHf`C`rBsse(mXlX{@@t6RnetzcJXWhFqY*0~+*VXl5&4FW&ef-f!Sx`fIe%2=+fBf6u`3@5w zWC)LZiC?t-SL@q$z3 z4?jn8+fJ=Hgv7>-9EE~ zr5|+ZWtaW_H@{&iD_)9c;c86S;D-Uf$OvqzvV4w=s|FD;2Y}g2Zb-{%lq$V2+V9Nl z47fr!*5!xFoL9>?PJMsHUE@~&t)4HuWon936+=3VKDhVS26sOKayBN_Y~H?n$-4Ka zt$S=u<&4eg6}xvhSwhuNvr*UEzo12>98h9VoMt_4g3coA;A0mgApOB5A zIDS8ARA}6Td8k5Xm6ws2f5F2d=27PjXD}Cw?;vP6BQHYwjSO~JC9WCH;M}rG$@V9= zsd;+BV|ar2X(Mvu)Kn1!?){2fw#baF@-lz_{{64K(mkx(xoyFM1(X0?6dwlnau#2} zqgy07+9))p<}%8_bLO4%(o8EdMHtNN#ho=lPk}JipLPkYZQ7CarYr!NPw{DKLt822 zVZuY)sZ{0K0#qQ3n=pZv839xlbflRUp+}1rDOo(B{zElBh7UywKufx|^xi%TnVnRf7+)+5J zZ$ihIQvwD)C&{=#SYpkg%EQ3+z3+XGNg31t5Hheh+6-spfw`bKW0^R^2owXRd8r?I zNX5ncAbR-xvO34`$R%bqqp!=NBO`w47)Lk;Y0U)dyIiOs@CR2s4{^Qfe9nziQ0ALIOgP@qglfuv35~<@=HZaW+wU!OOE+g{iJ78>lP=byp$Y%Orqy* zna^CGxaaEBrlpyuZc2=~hHwjWHcDMNKeK2O-(*eo*()*X43aXh{3W$|R;FYB-xFc!Vx6DX3Vvw!+H+5?ET$ha0eJuY_G1pM z#e^~gRlR%l;xQwiN9D-`w{n&cqEXJ6i@;1UY16hX@UD=?R#B#e)qG_W>UtEoQScR zm@|7eL;ssLZ)SS=$VxZ_WM z_`{0j%jwCGJ4WhR2cBn@Kf3<<(1hnScl`CQxBc#S%-p7>&&n74il83OWS)Ea=?Ct;cgSwLskN3D&%blHLzP5Rj@%EfWjCS`ttKFp z&l;XWMmLn~3L8zJvMMed%0E7R2U4I~vYv;gygQd}$LlaVy5;VGFziI=?)jOuzrwH#!4@4fd@4`jix(NM``SV8l|lTKn1-+jMM=64ogE269nh zzu;WlLL^J35;w`Qg%^L7chxu5>%N8H`6yy~jmqKD`J_)6DhU;5IEFTKQbwi|xV z?2Q{`l}9KSmj5G{vsbmx7!C-+yRpqXj{#uy!u!-b22oi~CQ8g&{ruZY{xomX%PQlt z;@10hzi_{vm+sQe-G=~gs9Z92#S_z3Ke%lBBpNnU1-xQkpRDJ@CbT5LlqO0m%G-Bq zGiX@Xljg4d=fCFvWJBFDrg$va@_OZh)irfn_U&^n(>;ocN_g?4b$RFH{PgVq$KHE@ zS6O6_|L?u&g_Z4GiPQBzo@RkC5jNV?b@ei zkO~p3tMMc+1Y2jr6c&_*Y2LBN9?Q6vkv*d^2$`I-r2>C%&NjyiAglMZmWDVyGL73ORqOiRqK zBo`zD0Wb?evLhxdE6byaNDz8p4T9kbM&yW|^;CBpY6uf)suMp1^+&iQ2`wLC5jh?h zdS!v0pszy7tBsdf5hbvP9xn38YBMu5#!sfDwgTIu81}(pD`QKrC70Gz&dyo?+|S>3 zZQ1VV)GkDYQ>v0yA_jz#U=I5UHBfSDh8#zPD1xvEpk(24z%)%m`C!JPT?W8YKvsj{ z;%G=q%JA&@zM zNDuUm}+#EJxXg;E$54%bst z45tb7JUURk%fDuJ;En73x6BA6rufgg!};>LK>jvo<6NifQU1ZFI}6`;*3Pb-^Q!Og zvwf}7oVdEfz%*$9NmsNocrH=?;a^T(j+4^ScjN?yL%@OWUgk=}h>Tv&fYE{IL@t7j z*uH{nM4WtsuT4gHj>3Q0a12}oj0lYNX=yPEa84)X7A$|CucWk;{(@N?2W=*`Y6W0A z5}%KCY^oHz;WCZ>p>oDPzTn3nkp?}@m*U-I%})Tk1mLk}pN-K3Yx}gf-n#DUt1%e} z0Z$1~0>CRzjFrqn=_aT%LUUG`r`9i=1#yaG!x7@Pir=ReF$3v5ajjz7=;?;fJ+qO-6{q>nMW<2-8 z3x^LG!URC_#RB!?k3IIv%P$*xE6}i`Ka1NBJoJ#XLeS6p{`e@%`-(oEQsrk6rTG1bvSV#nXlw}~*u+pd&At&g2&MhR z;Pb+lyP8KwlvUOA@7kQ=qrGJS3%|U#d&4edHg;O1#A%? zY389;u+y?7h-A0SgZ+y~NnS~mr_F_6F!HqNg~q10(-lh2abwhHOXKiQ4*8PZxJfBE9T^-0mop^8F3(t$?p4?+Ao4Z!si$tc^;RyL z5IL|Dt3nhBAY677$OeN=Be3wwQ&?>LuQyVeFq zc_;RgxP|X;8Q-pdhw<%O4E}umpSSJ%p`v>K?y@ysZn-_m->hrv;qlQ))V=3(gea$#Nn{#j`lT)Sa0sqc^4jEE(}B z;s7bd1sT0B%QBGh4Nx@&&%_%V7DYYw1?DZ(B)h2P*99Xf+EI3=MOJD$l3FJ`3{8 zctVjj*Fk|POcGVE5G^qVsWi<41`J>*!!#0rCj%Xsv7ir>eDf^b<42Q~m34s1z-8x! z*S>xGGUdZ|V@nX>*bYd-!y@cB^?*lhS{9E$TaLgsfyOq7)Yx!;H6d|78{)GfAL6@SFD_f@_c9wQZ|N! zo$Ae*pd&_%zx z1AUu-#*>i~#uO~bve2rr9z&br4+NlAOL37Gp_leeUj-?-P^8^99(V>1M4w-z@pU;o zP_Wat>IYxW+Q8Q@J0m9g2A>&NH{Z8%rthauo#v@d>yCjFuMBKl=qs&YPfFmZ3;Z1# zHfMO#+PfpL@O``+oi2wt{mn5?&W|7X^LMf69LVVBOY832vxSp2*jM3C?c_glte5?O z->D%qs;vCbFJMr{hECn)I}tZsE;pflmLCH{0Iz~){)S;|M|O6vLk{6gjV4krp$e;POS#}CdhC@T_08{Pe!-l0@ zF{OFmL;W#cZ<@+oyQ>Qes)~x)KjY^Tm)5B<9n%j)bABUi=-8ep5LO*n<6__bNAJH+ ztH@%>$m5Qq<5oGC6{dgo8Hcc7U}BpFLo$ks(@B;tUQFkOWdUcjG42i{`qLj~&ARfk z%kUnptggnthM$RX5!+JmA;NR@j@xgimH5;3*Y`Z+5ahG9jcXD|4n{T{*REOf>@&~Im_8kQ7F$=wb1t%bMioQHZ@&JTOB2zAg+8yZh-Uk@ z0kq_y4vLz#A*XEd<|0lv2-U=(P;CHtPfLj%+O6IRVyKi|SpLz%-LY&o2}y$zfjNa0 z&wY`-d1s|R%AXt`IqtBO$O!oieb~MlGaRte9iAI>V%{uk)4^E3^Yga-pZu81W-R%5 zMNbR_%eIz1{^>Ss8MRfl!wzkc)u!(2V@k>16L&_?R9S|)1y@P8IIzPsSUym)8FoNc z%g$-R3U3z|s;=C!Z)J7~rRu6}1m_jkJpX0(y6o~ucYDswXPU3T{1nVy+$B~`J(yGL zkGn+%*Yn@vGh&IU2i7Y+H?w2`B!TM6M%2>!%bfJ`ToHm1ww@G}Hf0<9?f9lJ*vUph zEf6_4gLdAu`~)I|3)oGLSGsk!IsD_n2L)%yTw)Xw3^XinaCQUc3_i(+9(<6Sswi9% zW~5%voZ+T0Xutr>8HhKc+@LT3()3mGY_cF)@eBo_lsB6Nd!<=&wkWG=YEC-&WNGlP zx%S$tuDr5f?_MFGd8Cu#3?LS^8?Jdg@;`!cUuEuR8?HIa|7@2wBe?Da z*N*0~X`?&+^WPh;+PZ%MKfBvR*x9OwL0eH+^P;cyc*0(kOaN>gtlcqTX3{HW2q_un ze#~IAva+xvFiYg9KQU)Wtc>kggyj4`nh8k+B%2Nx#sxq7P}0n_#U@IM5<)R`f5F$i z)Q5;7Q4JbN?PSi(m5`+P!V51b9)J-%<_xMeTsGbVhsTl@Lk-SWWDKFim`-qvXAFl} zx^|KyO7x*jaLvLBsNaTgHI^Mov9VYiuevcxW(AnZp&d&q6e%7V7&Ryd0>d4yS;63f zdO|5rMpWGL$m9xvf>FtuL62k)0l8xw&ta_F5=;}X!etWxlM9+-DjW5diJ!zu*63^~78VjQJZXz;69`;dRq@r1Rd@aP@wSqD zv;6DI)WQ+KR!m?UZ)b#VK*Z`3N+A}>inrwm(oRLmJOCq0eiXhwt}L}B*s!qhGm$P| zH0CF~v{Vk?2n)jY*J`ILodFnvs;#^r6}Dr=h(Ig=b3e=(5^K{7Bc4_OR4tDjo3aU* zPmf(QxCogA&+oq{BbZ!mO;oG^M1@P^d!Y=O+VK@%dF7SZcjVmM#^i_uc30sKMb>a_ zfQ2f@@&%?1>X@_x6e)Xjv4N7bnlnTJswhHi`L^>$ct&w#>wdI%Q&=l%TnK*SDnEp2b#5ZI`wzfFR0$Q*q^n; zcgV3$)?la0k%g2GEi$8R7*7C8}9(A5WqVBiM)7I9OY2zhbdGhlyOC*+gBQt-_f95;# zC}te&nE1zzRQGA6{uwP@C?33CR<1q|ZwvMYpnBLG1qXexq{S1LJuE|-_Fx zw<2G)65>lPzE~_2E_d(&B_Fl)^`Pjpm>y7oEIiO7*pvd>G6iGj#ikFZv@zit+Hxv) z@7eqLXFFeg-Pkf@LgEf@MSetFOskP2JKk_ZZ2NXxT_Z@d_iXF-{BOT5{9$foetw{I zf1uJF1rpghrS+&2+h1|HE#KC})|n$SFjP@97-XL~Y7|k-Prm!+o7}y7IlzJ$1-Bar zJ<`|~N?EbYI(FQ+Yp=f^-wKjrjdbjpXKq-(p3j*@W0ay8*R5T9)AiS5_+Y#~`O-_z z96J`*58$cKK8rKR-|o9_)5eYOy!|%gx`~q};n0EYD#ZLl*}Q$dL4H7;pp)6Q%|vv2?9!qV3(WbT>zm`SSW3J zff3R?IXztT5jKnxbnpIDb0hm5g}X$)IKRD zlBr1jtRYrXi=rqZ0-f5#LVzb$YI4-JT)dD1Q4!9JWqH?9K2de$BbxZDYlaO-dgYHj z(~^vnSF_ltR?Q;z7Mrak2z~vB+=t)U9I#3?(&sSPq*!VyYR??ndhC$4Zra8ht*Wvn zy!AyzX5-VKcm>{q9M`)lYD6EVih4#ytGchC9-OtKV zK7qigC!Uz;T3Vqo711R&ylx4KkQK|8u^US~?1Iubp1EyXE?3^Zbt~gt5(p+2`azLK zFYL-QH=EYwKQxyXm1Hz2uv1TAhcPUa3rVcWV8S{-}|er=FfQAtf%=SeA`O zMuII?22si>^5s=1Ayyh-0*c*x-+fF!$DDQ=QB2u{9@uQOOi2-!G}y(Kp~B%+SP~M9 zov4km=~X3_V8`0hY)bEUsLcN%U~}#d`>PAs<$*x~V?<(9n`5%BZ60HeLfKogY4MJK zeV6kfP8+miEo0ILrCl&A^UpC+W=hY`Nq*Q)QdE__B=@!IK-I8}YudNyr+P;DquI2= z4X>2}4zTc-RPQaU++I<`?$%(7Sy{cms+P@)rt%-GU>9+NGWqSP!Pq94Iokv*FVKU6 zp`T;sjLE}}r8{JI1G8u~XE1PKkfeqHeom=O48N#DHm?V^tBxm1UYj65IOdpX)27+V z<4r(zWMJvOkMCXVT!^|7L8)W#$A1wRLH3!4ROE{URX3sU8@HAZFSOCfH^* zQh*RJoMovT+X@C?TVatV7#shKY8*HLqdI;mhFQWQDFTxm77gmN91z=}K=zk?wsrZ_ zE5BP)xC@W07-?u5B>lgEWGYSx^W{<#sOpVP82G)YoS4@!WN-$O?Hzz2fSC@Zq`VZA z1h~Kx`Th1J^DP~ z_GQkVO@T$9`i{NWm)(wdGH$25M# z9Hd1+m-Vk}#}9NPrPf{MFhI2kLGeC!<9Nfe`VX0y~->J0mzF#_)A zo^lK{@$s1*JJNb^gB?~CPVUIe%w(qtqf07qLL;sjk`|%>q;UYCrqRN%4nX#$){z{| z3)@U+c|ZB|Q(6NlHz}Dh?!~|&>OcCdDJ?0OGkg2Pk5ZPfBa>Ps9XGPe-~MV| zY2r|^s|;@#8U%KK_&O+}@dgb~cVlit_uNA%uD6VYhkH=M`CG0T>dK9_>fVGQ%JaIhDykZ z0RR5CVvlR2aic8KwZCl#FIE}Sp{GZtOPoOz79Yzvhs-!KpypuZFDzPYtYN zw4afCVLl05eRL<`Qx_3x;<=P&9eQ(>QU2I%( zA~7z+MkYqr*~>g;0#e*pQhV3CTc%Fw-ZGxc^#WBjfhf0sPs9}>Og9jfRe`t&=c41% zu0E-qoD~X^3NT?rn`=gOZ0TNoA{nWwHk$=5JE`5pBg_QB4i(k4gL^i+?d*SCHdIy`B`H;G8I+IG^UIc1)W*g5$?VN%WeFagD@y}s9+7)%B*4h&OmKP8F@;5lUEh{!V52ym6Yf!vFLf`S!c;CS6DRG25?L> z3x^g8;(mSm;;P0uI5cjCm&Sk~)rNNgES4aMelDdUP%eoRjPi;=k?Cg5mIguOf_F zg;`g@1Xd*=v}1+Oj;$xjeQ7LK@q;@~UXcCh?#eRPKEahGd4>CP;-iwH{R!ojg?Xi0 zO_^gp^F{PdIrH%LSBNb`G(j0R#Mac5ugHBTB`z}|D!D~m2M!I*E!z0+jchiV!$l_$ zM@B>rO8Gb1T<|>uh77HPrIxVR)ropL!DkXy$C?$AAiA;$S zJr$#rf)}xNEZs8$GK?BbNRS{=4Cv{V^;Lx^MgcntBms+zoR%k|c#aHR?Zaf4EhlQu zKpCtI9Dar+7_*MdUW5xdwq%9F^%r^@Z)4DeC2opZ2aXa;!D3rsivdlK@X}JFC#M6j ziXua8l}S->mn1Vc(NOTBp-59WJPHDktVN%7)>&9pFkmr4Qpv)uPw@QKY+vGj2Q*Z? z2$5H42*7rtf|RG?ZFU=Oi~u}?W)z@IpqErMAqtS1DJ5e|Q~0CJOP^Ww{ro+fFv8&m z$G}4q5mNbMKqR*jkkF@y_rm%bIv*BI$RUwBG=qmga%>o#Nf)~qQS33{C*Xox978B8jET62bK?)Yb}{_1iq7;yH4!1CA`=yT1!Jee zs|Mu-6HbGY97rUS6I(-I+F=zUyei60mnhGxok}NjqRxbhWWTDRI0uL{g~EU<|h$#>^2d42@7I3yFmMRpUik-Th7vNJ*Ifagu?E|tT0A@#a z_7hJ&i8+HxOj`K4F;KH*@|wjVK_{F*o1E5G7D!EQY#Lzpk4Rda_JPL)L)G*S9q6PO z%Rc-3^O!TTGBRkWvobUJ5U|j3V(8Rop238{2ZhOlZ7heOwCcb( zJE~2a)YHebzwT;NcFYsX%ZipS-}BD9h2MTxomc2?_x54PFbjxg%IkLM{3>pTD=A9q zw*IO!lOHa@%`YtM+PQPf=FPDb(B{w#rn+Z}Gs&kl&fT-8jl1;*X?^nB3~pUlK|a7BEPdLXzsgo?|#w67t3`6nkcO|8{fbA;Ri6n^}+k^ z;~^5CkkEh7py4A%P-zLo(s~fY&*;)AQ&_^mB?k)#-uUdOXXTa(fKQD|7JzPoHiyFC z02JhKESLD!_3zhD8eXNB)M*1baSYR#9G;@FjRAH}?cdaf;f;#)9os9ZYukhm=I1i& zPH7&|FDn7R2BJ(&a;sB#61$5kX06H}GpKcH%cwqC@f>X$P9qXH;j(-CxRVDa5AD$^ zF*ewG#6zLKvAtVqW?RHX-FJTG$pc%rN@(hwpyLO$`pY>P z13Q_`bapO8qoy%-tmIlSd{Dv;QtD=8lW?D;XDQ%id8)|S|0>gzw zcWX9=LK=LLFD(X6S!~*>HyIAiGJ>fGx!v}Gk(et=WSXaF=KgrRK_tRWlmwze0a}d( zTkkbQ#%JrMV3HGn>L|n_Mmmm=$YA;+mM1bFdH7*mXNiDco>jHA*Ij=-BQdx{-#hQR zYyN@-IjnVs6fWl`luthC+^G`=P)S;z&=W)xfDMIfUs?U6v1Q=!!S6%oOG2)vbEH28 zTgDk#6U3GQJ5x5!^r7#&`IcL7{Xhu0v!w~FPgn?&mcjr+AOJ7{n2wAUZYl|lBrLo# zpT)*%Pep~8SYd$+n5P1Y7bH)?9CzGtLdh6UqHM~>U|}(nlmBMupEiC&;lJ~$wpTl4 z+%STnVa1*g660H^M0RQ!m)<#f*vfrxqYwEms@RcVy0v+{cROf+Jz{K5tF8Qb*9XmG zIv$#O#@75r|K4<6e#O@68Z(HGjZEs)Y}j$#ZjFy=#W#nsV_U(J=^Jj^Q@IkZgP2)L zmO_+rY)mY@AYBeM$wRY=e3*L5W4JJCDIOdgmhth&A4BgcSbgQkqIEvylvAj{EcWur;7bLP0qYGn z+`#-NTyhf_#7eBR15EhwVrY<_x(!|yOd2F;0*P>p7vsi_!!1iQ^2a~^5vw`9qx4u! zuv0@YWUxa78mYQGITck7d})l4&j&9IIVJ;`bZK+gip5T@`|i7sPm-}dW+NQ9xM4@j z0ZI`l3Gt0ICjVFADFM1?q#>37ucQdf=P4MTS~%gtg(e(-a4~_1NTb<>4>=E=;g2Y- zedAXDDcAdwI{Lc~sa-QCV(s_NlFyy~C;5-NH1N|Wf#Tf2hM)Wca-8Hw+tR?&8P3l2 zX0|!(LSM^tCwGT4Yg#RUD9UAVv!ki%n5vd7$-A4`?!Wp7Hx_77YNOy3-`RBj*(n}a+iuUbGZPTV(*RGuD z*hI~-^${3t*^I+wEA+fDf)vRTYwIbo;Svryh5>BeS6}ab_4U$Cn-GpYLK|m4wv6`e z+KxS|?FAS3V~q<{RYBqIS6c&HS=yQB zpL>o-%AvHpywYt^IX`3mym@jQ22nI4I6mM2yKTn~w!d-jK!=VU@txtE3tD*gfG`k; zoxuiYThL068$X_5k6p-C;&mWHuK^rgcEk~!ivgCKP1fN-Q&V%=>8G=e{QK{|d;PW7 zz|&LkWL1W>5CHRgSuT5EB$(gM8YOPqy0yHd1iz1V>FJyR8ltNAm*zbsHT3|^8TB}S z>7_$*%$1`%;68oEVQohD52oI9?UJ1nO#8hVB-2_HRE4B zb=i#ED*D3V?T~i*VQof*$|uBR{z~fPHa4m*#qR0xo!g}od-2G$^Ny)oGY2EZVyt|U*eEM4J0AWn#=CtlC++D;t5QaP=@%;-*YES zpleWiytHU!c5+>`WZCDVt9XGeX5+?T&JZ%qsY%_HwxMv{%c~w+oyU2oK}p0&vm5D; zJ}zsFv1P;sKfT55jJyB-FQ!lDz$vJ2x%E~#5qmtq>6JuJZw>&H#w-(2Z z?)E@Jj5!39lSen@e>-c-ujsQ9OyxclBlw1(uy$>T%)4^ zz``Pw@&rScvCJ_(Q-L;tNTYs?966Fh`(-K%B^3yJ2j>*9Lz1e`rUnN3GVzw=5X(S{ z`9HBD1TKQWn!3F543E_)1%yjpRRRey`%)x$s)SjF5Z^F2LXdzE3#C4_{;*2+&O7hu zWB`58A6o|FOPPf5!Lz`{bcLNP;)H?_J{5c)PB`HN<}d zr-bnb;Wki~!N30cYcNE?C^`TTNFAdIW-nk*EUz{tu)dm9U1=T~gb0~ZDqs?GgQTJ` zs*+ixDMkRE@G``=B?xb6b>+OBTdtY&#;)?>Qr96_`fK+oosrucg#?KNJKbH* z3%E*A+mS$CuthSI;m{VdUAM84=_W5V$oix826*byT}s3^rtqH2T! zk9>kk5aB}4Q}6(S=P9t@?O0oa4I^CDu<>;;ZGKk2z|isj`S1AFeCr%K(%1ha-{3L6 z{d)r|vIAc|>p$dZC#G58$n%}KZ~Ak$IP>3ePQK2giBXl$v$neS^T+6^YMQt64;dS% zt#a1Ssa-LXuB*2HY37!b>MCw;4eZ$zk(A-4xHY!X%o%6 zErA-yQWZTQOW&9baBq|eXI-vCQW{H3@OJ9JG|?v>e>_)v;u*!k2kc%!j4i?D5?(2P zTQ{NMsNIkX!M{Q;#|GOnGcz;)c;%H}fAb9nGOXk*W1@ylpmJ=az#J2f1%yKAm73c3 z&_kgVyiFD{;Q8?U~KrIkSlb_f1(MdGV3zkI>OiL_T-hQdLf zUw!|5t?NM}(+|6BgQ{BnGFPau6NEHn0K<+F0?ijURt^|A5QEDnAAKbHBd2)(ez}%a z>nOwC%gr^S1Gl7qD{YSznt@?xbx?Brv zTQ>w#Pb-%mYl0y4Y3qiB#E^+Z@F5B2Z+QJ7n~;T^L+jsBr@gZ04ujff%$uS&qfnwblD_yx%T^{{~hxlKUC>i5f9 z5q=!S)**Dc`T2N<(csG%#v~dF@NO(#_rl6Y*5ohaD{<9BTT_sXw{UmnybNp^ZQ_lA zjC8SO+<)JFUwr;K0`Q}G&%NWE0Vhoy7*Wg>;@w;$QP4;%PYG*QuV&zv?_Oh{(Ww*j z6gAVMoE}^rX6_8)J_O1rFd?Uo>+=(Kj5ZAts5~$b`Oa#vRfA>Q)Ku z(XhG(4~~!KZ-0v8Kl!qJP86sAvi&3@VMtPZ){@UG8)KYtD4U+EUs~sz!Sn<{7 zeS7Nqp~jZyJQ5pv6`y1)re5h;_}>MR3x>jo7&2r?6M``Ah%+!H8D6|LBeOCrbOi1R z;&W!t1_(q+)E zLNKVQ;Z>0b4v>}-19QBBz5>W6PgPinc>n-F07*naRHBluMALB36kot593*OlL~A;ov(xWLAFf%QK+^WXK2xTJv_Tla3S z{qALd$u8EsBF?^}mMdLX&+@IGhcjz!d~^RvSDMn*Qg#;*E111DSZ z?-=*+Ade^-VNOt2X^tpDAH87Uj+CBCY*b&k{w{`RHwrS4~8q8-fn5_Si##E zP4plFrEAyidf~hF$zd`pdK8&DvKeL z05RT1zZ5tZgLMeJUGTeMos&_4s6nm)c5L6yXkl+&9%HA?3>(~IH4uqI`;9l`a&6_dbV9tOySYoaeO;OUa@IHk$?dH4Hm6MH7dcZ!Q_ zHL(48tM-0SQC(hHTeiJq>8x!JpVQ)BZpAdW7S)<=(Hw7Kwhx&;5+emaTtvVZ5$8`l zs>2@#wZEuEY&r_iE6v$5^~ap&bIaF>p0&Q(7Ex`E>Tu1#4i`rGVtb~XHh24DoA>`v zUcJxAv;yn|$teIaqQ8r-8}&k9=gSg6We%1q0AHhu5`d=wumCWP$4^1F9@$+fa}vxP zEMZZB#0^`MV2u|RF_Z|>qnlFNc*%=KDIheWlF+DnbeS9v!7F${l4X_!2ucHBmjka7 zkPpCN20law*b;-lL}qv1tUhMJWbb z28IdY0x5WYDTW|;XD|R|WTT7{i?xC~Xjm&|IAt?ZL^!gdIg4-hFCtt}0+1tuDyi%s zi6sb`oR)%7gOzGh2aifv3hue*9?nUTqXD3#l8A1?CDM3;)F4P#cJQGXrEd&C2vMSy z!05?94RfqcAc<8n*-=H_1p_bi?{EC^Uu);i*}bKP!<#u&%>5v$jbNzw6a0=Om8LZb z#xewU72`GJmjM|3&}<1p=FuMl8!zskOY?as=kd#2)_zahMXM5Zg~> z6ac#bUorJZX!J{0GMiUnTW61+82+exV$O$NcEG8^-cJ@$k7x7+mNo$!?hAhTfWUJa5z^uePcA?^22!r zW8(!5=e2dSQ?&6-^AcN;y>t7cj}|XoW(Hy`4_cjCWI}wqi!Vw!`IIR27t{FkM!V55D=R)Rk@G$<%HXBXjL}0$T5Qzfz*@y?>R=9}-U_4SPK9^? zTc*HvX*9+gPO0U9e4ES0+ezCr;VFj4+PpS=Amw1ow|NB)&r3LcV>BYcrng}`*2W8L z$BKNx$kR^I#y8Dt$2Kj&j)jX?_Jh3m;)`;CrGCFGopYlQV|dD0g@ZJeoZ*3Sy1_x- z45T+~+Efs_&=5TI94gVXcW*g3!?Y#vGfs9hWBNq_TqFa72@@u;k-^qVU?mGOI1baL zyYIONKE()N^J>3L6q?wgfC;b%0a`FuhoCq#bYE4tQ1VJUU@p>fjTG<-0omt@=;@z+ zDz6YDjgCaw4dA{)xnlWpcJX(ssX--zA$FWi`!+ng{Gqi4KVvg?bqF@QykN^XF7x!$ zJ56XCAM|a+7x=9=-{k5U)z)ajy?YfEV$rzuw%c^q2~ngXngDA?T#bT5J3;bfpC%jV zSju7@j-QN02=;7Jj1Wg51c)I}R8(eWCK%CN0Fex=fKA|mdG!R<5Ihoiig1C2*v9;_ zIoy-LL3!NHPjcK65VY?<}ICD4^N0muBxsqD%d{SJw zTNH*WCA#y-&UbZdeSBP$*`U#>_0aTIeOK@K{HL97Y~44vvSz4QHEewh$gsj7N8GS0vPG_ zS!%G&8N@a%5JLtYJt_o%uAPN4Mi`7UbWVs$Sy@?}tj0WzHEpHGhjZk}k(e_S^E&~A z)AVpD32KWd1^6K_J>njMG9dX%1rRd!mB^Gy>A^F0p&4SXMUGIuLEUV6fg3D;EEmin znI>_M12a`pMVE$yT)u7~GM3S7O_=}WH6Hq1?fOx7j0CGWRgW9qiNF>mKZ7t1z zW!?PmcdVvnG%fb`#r{Wx%W0I2{gf!o??P>4VSx`*^>6S8)v{`WtU*Sl4+lghTr?Ln z4ra}ve!w|`&x-9W_0hljjpl`OH@R_^R*2-Vx>EU>3Gj=HEL(Gd`4Auz4Pob|)oZS~ z#wPnkB`9{hLNMNO@F!a|rH;Wx$2M!$EPN)lCE+XP8xeuTGGP#7#4jeH1hhPARKjcK zP3~L_or*6s_Rpue(DH|u&0pvMhAP44BMJvHA;8SUuOM;|tA(biA6&q9`c1V<{_PZG z2bO=~>vKYF+it#d|LVMSnScMTnh)-a7=Mq`{Ya-M+rMLlZ{FL1!Dslo9v;jiTSzL3 zxe0}aERc{Gang0Blij(_*}B|URpw;$cLtcFB?Hr+@aJ!-P3i3GGRSF`#09PgSO5P> z`@dcSj3xtt?GHRyJbz(zNpP^FP8KYSrCoe+o3qY}YM188NNkr*sTWVIF5O?cVrBUF zO^8{lGUNC9U!O+~Ush3J;jG!SF+k8t@acJM8PNCW*%Oxy{2F%UZhd(*AA($_P>!3`c0$$&wFdiUzZp%w4F`z}X;(y2*b zZ>xcG2dVSnv#^2xTGt*uW_|lDx5gSr@$TLA8|QmBXTG*(TC=`hBp+H7zA(#9fXlX5IG76OFd%Seqjj_X+jA*uMhm~Zwvw?!zR`y7K%{b zO{@z`y5^O@5W;a7@UTyf5tQII@7wU4v1P14QNg%RbA&Pjp5rpcjLAF)TZX7j6t3=^ zHHHHPr)C(FlW@7}8H>hy?}7Qt9e3y+WILy@i<+8HJ@trn30P!eG|2Wn4qC1#FK0SX zP*g;t+Pp;zRAL#J=9H<5ElNvC_0%K|I#AD?VJp#CN~&CA1mHzZW0c?{BVw~!jTlh2 z?T74#G3&sxzVPe#=vJ*0yETt%(<$Zf((218`YrqB@2kw6zx&CG+I@r4Cdx^lm_O+E zI6K2Q?)d#FF`c_6j~?9q53Sh6GeiI&Ygid{mLl|EuFke6~I-5r4^ZnC=!5S1Y2HdwQYemY{nz?MIrQ z+Hy44ka=kM7@0FLOJ`FUqZam0k`~_}V-K54A*lCC_Jl9`gw@MMz_InpQ|zWKgZwWn82YY)bbda8^eUBWovGiLpW@wD6lQ9 zX*tky3dW8nrsjR~b=5v=;l~}ojDy z1r-d6$8dVl5l3i(jJ&bD^NRk&rnYT6?AT-R*I=*8{rB8+_kH&r(yJE=hz=an&hC;4 z6m{#HV#xUV&2}aTqE}O=N z!}YOao0enq+LX2gp|LUOqr}fV{WQ~8)sqet_lon+Kc8+Fq*6jpknUB9*(tx9%Me5F zG!Xp5`>B1qc4`XY!;pCM4E5I{%|#e)?P0Z`Akd@@W2o_wF#xe(gjfN66^tgqt4BVR z5{y2yPVQh;zh9yxb;<10Pdz1x8nLJm8rcTZb+ZhNC<0H|gHZ-+3Ukt=NnC!!z!Cy9 z*}88Nei`erWw_$tbO{;Gh4z?qnvj+dJk?v19yn+aE*J=(@%iUw_#H|HnH-bMWlSKj zXk=z)hGI-E2(EDR0MDx(3zBsj=0zp@_p?pr-M8NQ_S6bQPN6}AP zOBP_Ks0x&>%l{%iqSdG#_mfLD8LEdF`%&exXjd3T>V8Ppsp$5xc)(QSGspLJ}fTjHZz zx~9W=xuJX`xmOBLM{#rtVEQuk?7tu6Xa>B*`FZ45r;c)R6*YZ`BXNMOD%wuiv;ieCw13+#?I9#aCA8V}MYu)hbs z8hjrxXE4LDX?`<|y9cS;>~@*kcwqYrrI zB4TXKdOl_1k-)16@!&nur06_I4~vKrq(`PQQj@5xI>VGaUQ5%+kn7Q-W=c>XSWFrz2n)U9JamVeE&71Lo zH2i6>a5v$cnxev;Z@$fe(qx>;fw5?Wf;pt$b_`6b4?M!?j!!!ERAkak)5>v! z$q%z;t@!!pac7^+goMRa+~ruLS-WNpM}bbCG2^W_-kAFAvwi#Z!*@dqM}Y>j01^UH zXQv6nr$q}F{_%<{#-Drc$)}t`ySIDSu6N&l8<%p-F^#bwbo}#QX8+@pZTN4^y?+q* zdK`55|4o6}`4tyFzIyMz%4H87=1wim(&&FvfnS@}i(loe+E#LSw`SvxYP+wTwS;BMuf<#iK;)i5+zM4Zm8o5VT47=AW%3&LJD<-CImub zSu%x>#(VF-kCJ$QV`C-O z5Hu9P3yVZ@`}o{hv%sWhWsN-Hgngx@_yy^KsciwTw8E2lU zGHnSWN5nRiU>)v2XzHlzL-ouVq6bPqqdFqRO0Qd*6)P*k*9Y3*w`ay0tyAMMB1soN0@8zFb~glvlZ_ed3^@nOC<;Om}=$ zekU%<9~+6|g+HRYrXsI&)6$%2YxAb(R&1=UEhQli{=``$BXP)K?Jw@t_SC3IbLc(6 z*0Z{%YGdxKg*nfyEu3EEUh>M9g+)s!!=L2B&WI8qlORz59=yD|to0<@7+@AQS(&H1 zvOSjB6Tu>+=Uh6-SS;04G+jxWS6c_j1fxg=RDuACr?bb<;9CFR^*Dw=$s8L$b~_bS zK;(E{aWv3VDjG}C#tScWAfo;4^ zZ%b2s@Ihe7oohjb(w4v^HY+R3&N8tCp_He1PY5h0Xpp{{4;dqM-Nyn#7zG&)yBHr{ z9(1ap~Rx0s;59WqX&Z~9?UPksu#LVDRS&|ZA^G7 zLMhMiScBmt#jL*0`uUuo;orL0*ZVl9)1ca}NBURHtX=ehzvofD?n9iz#sFN{E^%;-#aqn zP*Rc9!*#HA3x~roc4-J4F4LwF2{y*&vgL>4!+C9t4T}!p(nO|BX=57G+?Hd*!fWgK zYo*!rww%UN5gPj$PAM3VWNul@|Mt7>k32$4SI6$oog!kQ6A$Ut>96-kwrpYZhL=WU zt5(UJD_m5N``*Va-`2?yh|xC(x0ty=@Z<96hI0f*VEnXjAx#0M2hKrfImb>>Vb06o z;6Q8&>`eLW(@$@^=EXvN0N= z504d$#EKjb&A%7tUH{_7jJDAej!A{=%BR=9KYuSvye3_@GGP}aq_>X#+XbD@9%9_1 zB-oM#)0m#&>C*GsLb!jF50HFtsC!%I*6%Ke_bp8w$89gQ8{=3JMtOFP|gl7LLjqduqnml(X4;L=eECS$0&RBu8*c*Zkg2@A0LOhgA0N65$ zSwUJcjcrgYu}pQaMCkE{N)NjzCN-W+Awk*^SRmt-p!CdDki#O0__itj*8*a6z(PdU zJSF21Wq%|Rg+7Ah>1tQclmHDs9MgV#Zm%+hUSAI)pjxR+FbX#asQwMc+DLvK&L!pg zSgy-$aH2@d)xUp#e1&W-U^ED6;ZY{o>Fg@ozbJ3(&eCG{a9@MutD``%Q_z7>Iaq=4 zJEm6fF(<_&^k|tD<&TPWF9~wf60su9llbp_rH+v2%ZqjRIuWhj33(GiKVeC%1X?X30s(F)?9_!xCmal=9J`=X3fmq(g!nCR~OEm zm-F}&Rk?ZIrkqpEd-h7d{<^qMnFo?$LicVdr;Mr0-ibhuDA?I(>TCRF6UNZE!j-yB z#F0lGh3i&hDN>g(cHlsY_ZA&J$9x{aK|BvU@c3hovHzd8nku2ZxUgOR$3JpKtJFY@ z1-Mtd_u+@w4)Cp@S>wbz>LT>`bfvXzi*baLMA z!b~dNFN4~-Vsq)j^~L8LmD()Mn56$NgpU^PDJ?bgKs!{_1ooHJTyk8yj!CgwcULUi zRJ?gl)z4c?PP>2kzyH#wf2Za)@&5&Q$)C^I-^R_GrQRD}J{dP%p)nYjK7=!z26nTCf%t|S zZ^V|t#oF(O>dqi?U+{ zZ)3C^L=I6BM4BzfhH=`bpB8h5;O!GJ1;Z#gD=Uk=l*))?8|GD&_uy?xqEs2`C~NP$ z%9IrV{UNr0M-j!@wk%3>N|s6nfKu}-uNsR@goRwHxUCY_km19J%j&UMOvsd|4I>!t z^ zAC}3JCku_6?_l1+2vQP)SJ=bxg5k-;2EQ0Q(p5Mgj+~DRFD77>ATpKA#!I4t5i-Fj zivo>t1yVaXy+;O?e(Bq}Ht@!c{%hX!we93P<&U@}>fS3C1*AGhYEWco7t_+w&g*Us^;oo&=O?h+?Gczt!# zdfKtJo_3C;w(xy2kbUCmXa0QcHE==v`Sckhjy)E;15L5;GQ)rN#h1RH^#kn8)frSx zzU1O|X=!zmv)9&4yUDkEy}xXqX)lIO@SSsCtvmkX_=JDG_PVzEnxte_OiV=TDcFx>+^TwTh7N48sj;rrJ|6(Sd-le6+$Oze?eR^Mg#T8gp zpxKj~yL9QYmtT9GT+0!CBZnU|a>R%(ojWyHPei8uD+mU$N*@m3m$CDO7fQEncbiu? z-u25MhY5%FNxkU&=7$Zoxol~+d;@br{?H-p!!F;vr93;ww9W)G3*omngI&fay@@307!)xoN zO;t5D_;qvh2$ljmYg+=pOJ-KqlT)Yu`Gy<#oH^sOtfZuMYAP39jXeH%equ`A){P|C z_VVy!kNx?^8`)Bhj|Q`a=1ED1^z6xb#@wZXaFNeXj9A&9&EZ{UK%xMi&fSQd_qkHhef6YRXDW!3-NVjK)V);%S$INVc`L zVJ$$AN=a2?^<-Kr>5({*z_XD)?DE5r=8-|%evxdP0n1hU|pjQ zwrOUJp}jK`e^^_*J+I=faT$rR&a_#(ON=W=T{!^T>DsU>upVW2E`;7L2jL3m70L$b z3rlM5etXlN!txtW%NWw7h0r7=Mm&324@prl0wuqC-Krrd)Oz26%WtyUo7QpPQ2S$1DIE@U}o;FyKj=c^p@^FclJloh8{1FS&bD z!78*Rv0y{v#vyb}=X1woj!TIP-neAT3~K12L@SbD*U3${+ya2DAxJ>FuBupd&0Qyf zz&-cg+uq&bBGN=LU{Oy1o~X)|4YXh=R$fuT9+MuuddceY38O~s+Oy~6Q%_}IJf;~u zKH!x#X{rD43da8YB{|QN)NPC^)z3Hg5>I;9*L6Pef|ro)olH_NMwDYPYf zUd`FJxL4}whqfKxDmDZ63T`}!qzf-!Qc%6!j23Dm0_M6#X@g8Jm=M_}C8kT)loRpD zNJ;1%;Wt^6tn%c0DXGfK&R@1B@8k9PU*=bA;h<0^u*R3?*9W@v*yDAu8LPc6IIN?0z~IuB@Gtz$8m-VgJQfA z7%Z}A$)Xm^{&)uQ+wp4u3eZR@^klSLocf~|nl_YrU7^uJdI(rEWr4>aInlsW8y7Z6 z8uCbx5rLW^ZF^jP$kAtY9>}@B2~jcUbs5qkCgCsh->oPsrDgR9JSeX;#3M$GV5g5v zO`Dd26*J&)tU0hVP_nYG0V_D=BvN7y5>bWsv7^GAK$e{@k~F zgPCC-eUZOqx^M5+!0c(pB$<@zKYG$ZQ!*rov*mCuSh92($BcFB+7(OOSKoctA-zM- zPzOgfzVGi3;PPT{=4r=b??y(@&|GuQVTws&D+$b*!9I z^J#!QPBfQK{sY|sULnuD^wP;Eoz$;S@3`1FY7rj_GjMlTv#H6!@7eu{F)Ltkm~`QV zw5eom@v>#FzWLS>habiQ02Kf)Dh_BEcH~gH8&>6XwWdIq$8sJN>yv7}(WSEH4fh4;;0u z7=YNW=O!U2sY(0`u^8CJi)8K<;v=PE^J;?Z9}z}F*KjcX6hUNEy$4%UThuLl)WbnR zL8M7ns(^rW2u-DT=^d0_1Pl;*Kv6_`?v}HDIH}q^#3!Y-qP0Iv-TSw?tFLDfreaHNq_26Y*z3EZO7T(AJz2tE$NaAU1vWIi z9HqDif9ZLP4`9PW9M`7$ptRJ26{!hyWXt4IdIVs(Ka0PhfgOxcO^7#if zGum_|6_rbMu}bbnkwR`h?m|%0kLKAv`P`%kUuOv^>E|30fcBy%f1dDFo$t`pns;Ap z2q-ss@y{F0?(JYcoVFn8d0QVEYb@BY%QHTey1ug{-pdzA_TKLmV@W2-JEPbHt z*<^#5%6l#6yh*XRepB^jj9np0gInW06Ll@+(+7NH|HKgZK1nh7PemjGDjQoT_+uN3 zUubjcy}utDsq8+gpTpC4H)WrthV1pvKt}a14Fr)_ulZu5CCBw1{b&3uP|P#P9kRyb zD|;)`%kZXmbKx8H63Y{5`xV5g!jiQyXN)b`n^#F{^u<;*cgd}HNjR=(V}5ExUOa|8 zCATqr>YnQ!n989BA6(rU?SLKx?zhaV{b&&}cB|kt9kX9>#cb&5Dv1*~X{XqJ(sBK6 zz)guBTfLwC{P(tppN8}|xBb#vPI6fW_o7yRTHZVV=>2Qqp3r>F>B@7bE@FE7iRrf= zEC#~t-ws)jVrlwYMzrrl`d_~uxSIT?pQj~~nX=Ra%LRyS);2LI#c8X*RMYwISC#Vd zKfX~(QL5Z*_j}qc9hFjoh2Gumt*v7p?B%~3o>8V1#LspZM-F}w~%)s;HLaI~3?Q>{vA^Il1Tk=kQRb?X)S*1}T z67lfgyWcpq*hTLTPl}MxGP!?rX(1tb1N`W|WBiMO^y5xlYAcJmJJXF+ewe6AWY}=b zLy`N>pT8b@{p9VoVI>g*GXh<19RMWj$Ua@d@O{|?xLMFX=$r}NeR@!T89zqqaVUEo z9xI&8$TW{7U`0Fk0uGCVcCJ< z%M=KUBjZw5{7=9=I1M``zd_y|3GRdZZp|o{xXekvwQQLKSfy0F3+77(02C!3z|X4S zv6Av9*|<)y)}lDuTC=eL>~JpAt}%1n`HGfVK%o0(Rj zxVot+L38zntX6e1VNc`jayxIS&hz*-A>x3fu@%t`{M3L$d2+`5un{NVG8@An)3wR9 z3^|d_?d6VC)vn?Sy9)$>p2Sk?Ecjo@Q3jt?p}KoaHRe8QFoFZ|x$~V<+u*C8| z)#cPX%3B#DdfQLRZgwNI;-1viYJx>`HO2Wjx2#Hik=V>KFlLy`lK{EG%%e}i$nxt8K*Aa5sgMuR$%h&H)vO zog9&oVIUL1I}UrevKy5poE|(VxzdDF2qpf+?yH2L#C2IED>vHwYL~ntgw4G_)X->7 zdTP|#62$l>fl7t^z?9nTy9l2w3jr)WL~OPNo}K#LCk1iLlk*pmH98A7N;xSneYd8= ztTtZ`>p`uA0uH|qvG1C^X2F$R(07kC%Zs)CYaXVqL6}b{CVzwYZ3fMGbTbmw2Yii&qDbVL6|u3bK(XyAsTJIdM3_t6!9xzj{el8FOEGSOP{W zq=x5=Ip~1ZP-gR1k3U!Qxj_?^GEsu`s%G?}1|Qtdc&|6*a2F#l^v(NUWE3aK-xK%? zJI(RNV%xR&83mVc^<{vi_&q$m&C7~Y`Rfk?E&QJ(X?9yvKiVMC;%+|w`LnoHIAm$w znmjQ;_q_Im2X}?C1n-wX!sQ8nbkScYcjRvZxGFn|=PhD5CwFnEhJ~onZLxiOJU9EX z7jsbdeX{EJ!5KkLO4Ki3b@|`IaD+02&~x9xYjob}O*5LONhpZO6v?R`-u-~<+aaTP9>}Q#xPZQ*JBpt>pkPRTOBkfDqAS4O=WHl zj83=fF1Nli-AvB3HB3o1E)*2`2R3`G2=(@~G_NgB;GYP$wku5{YVw$Y?cuxXuY%4$ zhNiM}$-QTHrWXqR%>BH-(J4Y&(_Q{k;Z-RT(h{6sB-Ww`4T;c29_TBRvQr;hu~g=J ze@cnlOZagv_*{A*P;V?tJle*yZ>p;~I3JR)t3beJOjGDHs|RhReib=$AKqcmrdEHm z&ZBg=fpaWYW?;D~4CiS9JVL2acQYEP#oM`1X7#Cu#cRGdQ&{OBm80`YgC(ZHQn(#k zmb=fFFx5@t)7BC==REzVyNjQl`iv(YO?x%KUTb0Uj;1(>R~0k9`>VLmXqG`ODF?Vg zT8$L}>34m8U_|N4P|78;`J`j|_ZfD^;k!@if8hQ(G7}QfM=m69(r+di(?Y(_QGCPU zinktuDcX;Ilb;3gvK6Hy-Hj{V(MJ_%QAXUQK-Ky>Oq+QLb!e9lHBE_Qx;??U=WR5QNR{tOs!aLWh=BS4IS&i%vSi-Vrl ztb^5%L6#kGyVOTl#6T?IG+f7m*911lN3_a*j zW$_;X4ubJP$WR=TWlP)^JUQQ(yQ)4*Cwf*Ex*fwnPA52<9pfuI#U)L|jM%`|+j{2S z_AclZ#mlaLs&U>eNwJS$`Dop6@`YT`^w#F18-N*xx5Wt;l|ce<=Ql?={hay&_zo*O(sJU;+qpaBX0NS?PAIfRTx*;4tb;?BZLIsL%2$y0;u7`0>#0!!ilNL*kEgYop;H%zCYM+aZy6y7IEaR9QcIHH_d#fpw($zo35h{-RzU_`F*)o3!X=Np1#7Q2q^)M_zxC= z%9t*fenG(F+0~IH9yJBmQiXL5pz$$}XZ{~+$V+(=#^VTkaG*<#g=+*HJEa%u8eMWj zSM%3Dc)_;KA#N!&wFwA2oZ-5Y*O&Xa92nOxAN^MkkLj zVW2|m28T)Wt3UeL??jo+`^aVR)O%dcBzGUFYWK6}3FyxbTkv5JnLKul@=yvKIBoKt zmqdTrV>o3-M}a_KyZn6MD!B=Ym-U*7l|=spfF-XN1{G?_1yJ-|^tpkRzJ^nCJUcm` z-ZmC|2L8zyNp1a_7hK=>2Z3;v?hRLWxDu=U+1~cYLxb~exY7%fNQJ^CxAg^ejB&aVH2Ua z)%RV)F>iN9of(|(EjGNL(HE! zeqiBm{*SyA_R@vTv73sMg)*~r0|%;S%(l-f)8?o=jPkQZ!XQes+^kgV1YI>V~T-x)366baV;C2IJtAvk}~u4-Kcn1 zlehno(bZ+r@K0vN;qpD@iNF8qf%HiOQ!;p?rR>Hk1VmX@?g^;!1frVPSpG%gqH<>uIw8V%?K8jmXZv40sLYw^E4} zy01^hv&Kn#Z#xSp+B`dxgDdI?L&CJ|M+qYTA^hwh`@VrUA0|T4&6(7lcpxn(Be%zL+9Hb`w~KI1gU#SDwz*g&6TO26jeC?`mldFIAxJ z*zl%Fjj%ZUB)=6u#RDOdKgD;;CQTW|ccGEIyc^KT!+vKellY`@pcPub7MFjn1UEuvB-sYiNqOeg#`*`SE0MNT zOa#_Dj!j2GqRY6N)Fj6B`caOhj>RTzw#X9dF{_ql@p2_#vv$r?(>~^bL)Kz`QvY)L$VRTQ_(Ebh|@fV#kTsHXZ3#hclSy%>SS=DAls+-lp0QZJ)ui zAkRxB&9%dD#Z}4In8!@dErl$)F5lFc?S3;2z1gGQu?pyS@D9CNNggHzE~u<&GGPq65H6eK4x%mjG&~P%0Jg-J6 z)VTEMczO>0ppt=r&u5)-%lT6|Ti(!2ika}35ztE>P`Mj|ogV?i)Djvem{BlQ8wm&s zYS`E*5@i`GkhGh%XUjp~abEXeUSuE?w)SUP z_o>F6M?}47uK_GI+)`J2@cG}0s zn{KT_7o)V3Rs6;XlfRgv4fZAA?r0k_p$b$j;fSh0@wBRKPx`DPhbB#~3wZE|e=r_} z+oL?3pJvw=`>BF6H=Ge0iV)k+_w9>w*iDGg8&I!SZ66 z&O%-U5t?hfLAqh#0n4`~h-g+C-UsTD0t)?dq}&e&362kI-#GFX1AyO|}9>rb90r zEt#Wh?6h?9jH(@q3-#-N+=LZ-THiI0CW#ZTWRC*PM$&QkG(KT@zfL*I?W}^np@iLl z=kU8%J3$kE%bH<0;^kIdFc%mz>$xn62(0q$oKH1Vz%Nd~w&&`sp^jtKW=*h@5+N5o z#V=20Y$S~u6iI%cSx#G#)Kt)ChcMOPsd(0DIL6n@H)_&Eb0){(+e;rPz83N4?TI+_ zJg?UE7rm`yk8 z3+enyvsdqRi$$8p!Q4&~5keEyPks=tx9@0yciy$=gxO(?1$yG#DNagnrF8gXi=ZjS}1kS%SoYC18|m`;*m6ZD`akbyIP5T1KZ*t)uYnp;sjJ zJr<{eekzt%Y1W;QMJ!_27b$6?)4Ll?59@e_y{Ma8-d6C4WJ*cw(bkHNODQUR#XK6n zKCh(r9J?}0kBD3SaM5%gfXlIJgdV;4VCXcz7^DEk4joKb-ZYWIYxa<;_YRY;yxe1* z*+>P}P{7$UA8e8fkJIFTXqRAgMVYJW_cu(m2BzhKDO%u&sq7dE|Ec5hK1jZ)_u55( z5!DkLl6X%L%5BY*r*z80mP}(E_qXIU@HX1p%!(UeuV)oYF90QmT^4CH{w5IR=CGh`BXVgAjJ$e(IvX!>4ro*yVTkrP zfTDxCm%eV?*?;@82|N~Cg|EE|@g9^5h?7+f9QBWXt#|Gza|s zJE9ejz}!YjL9lcGnl5+=aq;r<%Asc=1f8;TO=t?m&Olage%jXLxg7^yAbjzwx8>W= z-dIvLpZsCezoUS+U~mp#lhC@3;}4J) z!$X;+vyJK*n{@+|Ym5a_;^yE=R{{M(!8LGW=7H2IuFg~K@S3(xs?2{lX>N*ZLMkNA zw`dp57;wy$9>XR@Uu69sj+~ffl3TOk>~Yu-bOTT$Yr{8M8i6}&x_?f_#?3jCf33@*^uo_+^f$oyVNQe!@xb9{X300Cc{_RqT z$a9uSDq~H!hj~%1Wb-SNvq8eKs2B~`zMa?K7+{ggc_^@RkCMMO^X7;j($XZ@m9&~e znI%q4+M*N)bnX?UL@S(V(q2|FYy29_lxeKm_}e`&ktKnHXa#ROS$0EFG+&>;kDPlqUKge1R>Q;fzQ8jg5tb+rGE}-R@zC=i!G4 z)=K`fUEShg&eE^Mi-|tAOIbM5`xK8mUknY+byk6zmHO-ll}gU7OC+oZv1b?rt7tW{ zE(f}qN6+E#G706?g`GU&`S}chqor^W(N!DKcn@Pj$Z=jTcuSqec>#y!R99e8ZIYdGTyxk z;8tUb%zrKU2o=nA2-snEaZ^@w69{J@*tm(YumCEKxae)Fm)HmkgUy8>V5zC)3Vi_X z*M6WGE3n|SU71`M$ZMSvysHSvP^LecrV!c5^NtI78uA7+8>Rk>7lv6$8>MbpL=SUU zrlW`6M6skKC~S?K;S|&y_(TPLJ&19l1{u-aH<+mJAMT z)3pLw9zByJfMq`77loEDP=FjxKAGop({W4_kbtAzaVpQw^+I?6Aqwc2sV*90ChtP# zQ(dy_2RqFLF!hhv5WQ+lujc2A4=v%gIVmJ)@{DuJvJgh?H!fmMB7^j`hjsu^?Rjz% zjZHkS@|~(lfsw=w%Q*cS&RwXmmA}p}`?vw0c-JbniI&G#;X=W0dj8vOs83D`lgABmKrS+LU`nG7Ms1$(3=;_6^fqu zz&qmz$~lf4prx1aTdI=c>EE((uWhgMJBb<4E0e2~K60LR%4)eQ^ISnB^5a5%$nfTV zXQv8t$U?zwb~qwE$Xb=h)^&3Sz-)($lljqsgyHvugWyiLZ2uWunX~+R6z#W)&vl*W z=0?(}#R)-IRQmje^=kWC1Qy{t`E6YJ!{qTXvmdIy=GsA@GPSp4)J%NA{jIn`g&Q#=4ZC1I zZgVds052TNSo~%a2W;?2a^&^*d3uQ`N`8xG>Y=Cz;}^P3aQ&g(ss5|_x=!4Z^dPfb z5L|0YN8zmh#&;YFVodx>A^X}dwUD4{T|EX0 zDrg%eBwgcXpMF3`KgVd6lj8bwPr@2p`lsfuqX6va_gEF_xVAyCJKud4gw--j$YpI^ zkapX-I_u@$fhFeMm*LH)y4U*<@9DmKlq=YyVX6(DxRrpwopM_nS&Wj2G@)1p*l(Cm zihdOb7_BtkC{Rt;e8ZuYJ+_JSR0d%;+dvO=Axl?Q{~tQAJTuU~BMNS#%xL zY0P&6fIPcqSo)$~mwIS(Hfgdg7_!pd+qF_l! zss{NvDUIAio9@kDM%gyqz_nh`}q=F7*(K0=l?yc3938{K1 zmlxus2akxJ`f9yOyRWEdl;WsPsGTArt&qW|Fh4lJg-I@%kQ>3*NVc{n`@7Ffi7(c~ z=*uhj8Jk|)>bk=cA#jEK3v`{mhTb#E&dhu$?ss*!sEt_rK?)NsO02ES`moxGuskyy^8pbFPohWyCy4 z81U!)^q4Nzif^i0HjzzZnN3KqYe>*`eck3q?>tZ9W1>a*Oh}|i8)+QUI zmhQC_uo5H8G0+NIJ@*?>T;g61CW9e21L>bydU5iXcYeOsC+-=KGQ^HnRnyJ$% zH`24L%L5xVY_gUj&1Jqa1N*3w1SqcZUcK?W!@f&+`@`las>QBL|GT+)a1HNn-zQX6 zt`e4_BpbHKJ=w@t*=E9!Ae@>!;WLvEx|{YPG2D&6?Mb3VZi10ahI5UoX48_<8#VPs z+*c2-*(V$MuOa+5$GS}Q}0l`&pE?*O>|Kc-~;cn z950Qx#L+J+X%7{A@43Km-&^1Rx|%bd&Y9IDs{EblFNbj$XssUn2*JKddi=IV$-jo#PxFl81y3RP(n3se<9#- z`_4pE-Q3{FH4cRF%NEykU0PF?u^p8rO7*Ly+~9yD(&blcn__11xu(`(=G zd#9T>?SIJ+1MwCFrXPaxC*LJ6Z1z;r83XpKYyu21EYP9;8h^&i8n=IDRY&P2<8ggA zj5JNp+B}lWw1{|W-RsdE`MRnG*%m34mI7}D!dj_Ch}WV}O6kO|uJmSpTi4p7(5`*o z+-G*vn|}ZbIyL{UBQFwoA?J59S|-!NHvEHl z);DO_yUw8+w`AwH?~|8xz98-vKDh8Dy3QKc0g-)G3tfn`h8U*Fb_F1|VU(esQ>OcS z@0k1MOj}x=EH8f4-E9BzB*2>dIS%%Nh>0_+sS3Ad5LnSJEU7UzcsD4TJB}d$o`VYP zKn26-ulI;mhyM12fmIHOEHQOr$>A_}pcYZT)rxNebx9ORPO(*h7y>nrs4o8#k|YB$)>aA1ni$tpLPRlP6dk zi{utiA^aH@k`efI+PFO;r^@us91TXXBJKaT%wmO=)Boj`CDFOCa@TAT zLo3ZpJL7`cIln|!>z;K=hg+CDPts@pG}QPusY=ZCn`kWR zT@sfin@Cd6n@0gDiaB|zvF&GEU@jGfx3$*TmH_ezn+}&Z(e))2cEi^ZMm)>#JKSJ7 zmNC89xb=xoO~Qpzm@;Bto8YTrN_TTsz65xDwoLHN4{-H42uZ#tiaK22&a~AAvxC)X4XQ@v>`cW;`9qZLnz_tm z;b)KG>BevkOsQ!kH~*xz<`dM#&u=_D?3=g#yR3)Yrf{tq%iV3%)A_J)nBGX)L1{Mi zS*Io$YZ7U9r~B&8Gc|wSd5=X1=iCf*>-$#ps_#}?tyQ3jrxWm7Nat0(93+L@Y3y-P z$6))uf;YAzLJI+c9@q-GRdGQcG^A9?(Cu6^g`Q{k`^ma5c^pb(JN}Dk&?cs0Ow%{y z4%#+@ElkUos&tzw21qisvS)xl_HUYAov(2DdYKGZ;jLFb6fW)V>DHn6%9PHao2XRncEHt(v-;<6U58kkyN!m5CnEtVc!-6~a4mM+-( zRJz{}zp5+S6EFb+>&kbYy4=i4?FC>JZ{n38|L``67K~$eNvU@!pFk z4uC4O9V9`-5Fx~iqq zWGaNhRy#p6&FcgA$tykCYsdZDWtk?^w1Z;FWDhlUWor&bxGKe_`9tz~@6b85di+2s|9v02_C zb5W^$#ggve)Pz!k1z#d~-fU_6)6Ob*9BOiX$&+nT!5~8Ko2E_ zv0aoqasvJwkNMEejb5CpmKcCeBZFow;Nou;;!T+O2X>^v)3iXeA%3Ua=@)Sm8#&lv zkqij6Y(F=;30NMZ{P=Y4V>QC^B?4X78sfVc#?>vza?o8eRYMYY>+os8xE$|Jgfv|M zj{jB|Gp(k{FQX`T-oiE}u8&|BHqCz#LJMcH6S=_S!*s$(`oU^_hY8PwX-~&R3P+js zQ6-OnO*imj-ina(mB%K~1Nbc4GdN@$OWGb!#6)cy@W`_b3D0zJ-B8vopG@l_aHmWr zu)S|6Ck4AQBPIz%n<0c%#@c7I*~IG8V2O!>&zi+W!1?p|1OFGiw-w39*}HwTSC>h> zSEX0?u}%+2*xl%s^i>-?;CgB6#gGhRQ4jG`gS5Nr6`f&y>iHU+D-99s2bOCT&DzY| zrB7T8!}47CQtn>SKzL#NAsQynCM`&Pov9S4V~Q;i+AI><%ftnx z22!|Jnkq`QBQHFnt0neZ#?MYeYJuZeH-06_N4&<;MpN*1X_r}M_l@9G2cUaEQ5Z-T zFxoBpg5VV0a#y&pKzSqaQH=)8yej7AvX<>JR%OmPZf}oyQRdTH(z=E#qbHr_lYWsV z))U){SoRXrkx9PqaN#H!SHE9-Cj8&~h!(Dz!qK?a8VcvJw{7V3!B(;14O~j~L_5mP zgYi6C05P{7CLe}rlCt?GtaI&q7ksYG7TVv&h@?pRA-`SOxvgWe$>T^ir1iHx46gaH z9xQC`8fZlKZ5+0Gk7pv7aK}3?6P6zGgi)nn{=ggSu~GIB9VF{Fned3vbrY!@{FbK< zE@_l|tv_GHm?YVzH^_ecsEvh$OGNoNKmG>{g26P8udOT4?!QoYlk;Z(QA?VRV2LIP z%2lLODSDktGwo6qngzRI=6;hpvjR#q9JAtQG?$tT5$oKnhmmr}-Q`*Jo{$3|zK0O! zana|EYqTlq=Q8&aE%zb@41B9;6~Qud>zW!sl+T3_T{(!Sr+WtuiQh$`pc{VOovp`> zBw#rUEPZW}llF!ay%==R^&-}KU!*nA0qFqoWcAfRB1}LpyR3yqw(~;g7xt%+rFlPi zFR_mgiswZPl_7t)BWAQ*b^k>)vZ}huX;)!z z6-P)8?hic0a+y=t=B}JCdFvw1ii8ry#N4)PAEak_=mzQoJ$O;k)zQmbj%5FK2EJ|n zSfD`HDdUkdUeh`^r)W8))!*szMe)YyQnzuRRMeV)){Z`~0PoMmK4Jn{cOH`-C=m9V zcvz2j13mEi;-{Xq_wb|4`0RM*z~yc?n`@NCwh-cS6NO*{FS17L%j?ZMu@s0RTRm)g zu79-!gG2mOH|_~75=sX5o@wd!_bg=&XIVd+kn@L{=`aQ;srR@```C{b&i(|E!~hzSOunCM5V6W^!f(K{&G>=Mbj07MKPoP=o@{QN zIZ~+|*z#8v1z&X%>EXEqdq%GvDT>&}%`T$qZYL5LXUb2pFnOm=2ozWL_7 zdOzvvd9+itPC>EB4>}>AZ$EvD23UpNFLp)&Kin9MOXV@s9Y+rkb~<_BcMUxt%)rzp z9|Bav`3Fw#m>c(;5Nj_8HdiXLMME=TX>MXk%HC;rWBK6P z6FW2(s`+8q>x*)G(Bp*h*%J3@;Dgg{@wY7OePfbO+PJoo{26N=s}s3&%mWD;O89zb zo0vCM&~MpA{co%ct~S_i+Uc^D;DvePlGWsiyX6QWFiJDs$~TklO>_HS8h0xl)oN!4 zjh)#eum9wJ6lKF+ql15)zMN7@Q z_L0ufcVxMccrjDEXk{;6t@vTVhxaTzZa}B23Sw{?e);}GE}ZmXBRQjiFUqPjf2yfL zQB`+0#+$yosbMd~N}yI4JhAOTDp9(Z!tFNZBT0fycug|(fHp&Ug^anxky%{StNPEB zCGCz1%=@Zqtb^Ty1FtlBwedAEK^{h}7S2PCQ*TLHIlBo{Gx!gjxB z@+4g0THTTjg&SC$gLHrOZpQsTKCC-xoFX;UshLJr(M7XBEwD(De0R%43~h}f;b{|{ zsvW|$uL%3zzPEJM)B5-?l(@r67(c4M#|=h-@YR-0sB%B+Hjj-oX}qqq7vuqcAK-Cg zH3&O$F%X2hY%SgZ@weaUuTz`OWuoW+9?S3~GRx<;-XJrGd8gkjyLV-1UdN;mbybnA z0!qrxoEc;xY}(TEt&QgjcL$w)&*P%>Tmm-wA_YHrScTu`_nK``^$y;UCW$v!U9P2y zHIo79GD>>+m<;Y7jI(gZ<;ZsgK^TgB^Hva;-koR$S=^2~ul{6!?tujixYQ?D!pTeO z`u6SZVOQrvh#OcucX6xlYiPfPcM=phxZ>{3PbUmHx>hX|Edk~gJKZ>`a#9a81b|Ow zKs6kb_6Y5eQEY{0N)X zjRhFL4{IIwD$F^mea$iM)t^%STJr1oG+ga>faDimwYH*Xv{gZCfGP8H;$E2uuZXlN4Cu-TEIvoF0E(+37i@pMV>_8XcTO%~+5XO69atJH#Ha4^ zB95^kjN)LDeFTamc;t>1r{L2&nI2KjMbb3O3h#QUBuy9P7n!e~x3=Z_F1Zk|0FQVl z^B6l3A^3gs+4kMS=;kQSgaMzG1e4bpTc%=Gln&!L(wm;A$J4)u(|fMBBzvyLY)^O# zdVVpkj~g_sL#SO?!RCiXNO*-@_Q;_S-2zr4;J_kK2p>PN)Y5%HXpr{MAG|ZMYvt>? zvC72Mb^}}ZV5WO$H@>^-6 z7r(ENxU*^CM&~O2w-x9>i`D{K5O7UQ4h1jw(vMr5q_PR1pUSP*^Jrh5R9%D+^=4q$ zrhX)za1wfXR8z(V-N?>^dIS66J*C-#`7dRb>XdBi%}LwP#c#lra*Y8fMJAq@!zI5R zU{A-!T1geiHmj$08LQ4A1Ri{2(1ytptJS1FvSIIe65YFIVH2qtCwYMdgIDhxRs2Kb$n6@%j?^w3i=mV^pe?vWB zps5}&6tUA#CXs98nq<`=Q|C;yOVAX{qfukB?mBA%e!8h}CR>l0e#e_3>6tfFc`E@n znDV_fz^`iSA)^B~aFA77j!U{9D7Kx{%SZPdBCJ}Gpi%ZLJ@r2A9dE1fAJmbmJliJT zwrQz$H9rO|*{fO0)JyG7z0Ig3-)=GgHs6$qE21h8K^RwUHI{o)B<>F9^^YXPN!qQQ-eO%1`v><9GnGgH~zPW8q|fA9fh*HXCTJZLS25 zw-g?S(Rcl|It-l0SYLEDV}eE-8@D6hh^#QrGnXhfD5nX?9Byip>dY)S+mP z9A#eq)adp^rFRL6`p=ULn48!k%<)eWQ_^CC|2Xw?zJM#{E0*}ko9|l`+*4^TOq3$4 zw#%}$`lv>EOE1i;sHsI)o^o8OK0)2ergbFgM|r&bphW(73HY_=zo%uoPF$AukhiAo z*jTr!sH|u@%OPE}NV>0L>C=yob0GRr2lD3KZiXAXqKv1orM3eJg2$}iO6HhTS82eY zXOFnkUt4wp5<5XDUM7U?X zpWAXDtM16;zm13?9f%IKOH=*78_*F2e?HT2)lvO}slgnp@~vgak6D%Dx4Dl+P1;Cd zNREk*gO1MFb-5x0n4)H{LRq~oTG;iODD624ga?5=Wt`RH${Bf?;~Le#1Gpb z^m9nI=_PASyIgx4jc)IPhaI6?-{(;u^F;@USX?SU+&SW)hL~h7U@}$2tL!HoZE4&W zYZ^;Si!sa(IIJ6v{JQ7A3$3CRBrg?;ON$it3(F5`lyZe5^mKH0SjnDZ2gO_5Va#5b4Sts}B|? zDwjV=<&S<*hP*>2Bk`*jXm=Cjm$P%mo_l4bR=0wQfWd-;6Dz6MR_Yge7+zd!^~hG< zt(miDz=#DFVI6rq@mj_;UP!gfAyH0z4VZW~{n?ngnfWi*Ln{(9lXq}=`a!nP>(NY6 zP!eOCu;j%zxBZQ1o9n`?8w?D3xrMxQdZjk#5a)C&hfx79s)udjQdjBGa%+rBblPHfW)A#DSF zxs2Y{2)`?mu{@R2=2J%bM-sjOLbPsy#5=$IgP+uhJ60#BqdVjNNQH^j5BkToAtQz2 zY~4o(4-8ba8E1rRp?2G|!uSKgK>}ic84$ES>e7E_DS0r~e@@i*3jeh%1RZgK8v4Pr zv?_7dB6z$SO?Q2?na)GFq? z44l0oVit6)rt^yq*OBr*%iaA9VGaYKsdNHJ|I_Q1L6%p*>C5MP0Gv~Xrd8p=&v!EU zWnv0`Es)jFTs5A-c(1kh08w9LTrxsgpfI)OLje$y~MN6}uzF9NYBw1+7<$ln|~=1n4Tl%`;$c zY+~<$&y#lJZ~j_uEJN|=@|8kQyW#v_X;g8t58{#0MHGmA z^ICN=MgN34^Bl4fWSG7;6~ay&s4a9g{EVej%&ixerO?RyC4 zGpy9x7@zFbiQl{~nve&E`CAXHxXtE^rf98Ont{Yyn$&D?;17z5*(Fq1|d8onR{b!DWOgsHkD)U1F$#KqctaWaoVk&e!cyXr# zWYB0POZL3vxlDnumm#wKPcBb)zz9fTr4ercIG56g<3qQYFWa@m$D`#VmnoeXGdB&8 z!rsyvIa!;p;*6>v1099qMqR^m6xZPsR})6|#xbbO7B5$-GDnZNDN(8c!=brCU0fOU z`mDW{PRyU`+M$(mId8bJGwFvKU*@FV+v4~~ot!%9sj{Q8JqaRjT?lrs13&gu;gC9l zc!Z`^fO(+0MhetQ&1?$O>xvUG^Acm#rDQg;^}pWBo_^SSF0nN5n)^{_Rg9_59z(csg5V zFD+yYWNs?|BenhHc~d8QlPxG-g=Bmw$M-SZb?~|LsDr@3oJW!Fj!3jk05B zFc6fAJIUs5v+&UpdBpb7(DS6|BF04DU=;_xKB@5(vTfCG09LL-Q~H1B*u&!-+V@+R zu?E+-N-s4A(!1%Q;0N>bL7`j5E(GH`Z+$eT_Ax?(y#re5yu51m+9qCC_E;&GR}q=n z60b5^c=HBPO5~s(me4O37$q0@SE>lihCow`utD>xGy^^~bnu zxMWK^Or@ad0|=Q!GRESEVJaZcp(o-!t{m*p)dLhwRwWv-=zDCgV;4VYhYD{`_p96W zOU}CeUjbcPlSky0a#vp3(qZ&(d-`0r4GOXrDIgTy!<&=dI*{ue$I;l~Ik`xqg-7rf zFT-dudldz$irTm0a1mDT3f(qoS+5_P>*XrNED%k$Reu5^-V!{Y#k5@q+b4?Kbpix~ zZVZ}L?}45UW=>~kQWb=$M!pkG$Cu&fF?*2PgN;gvl@hAr3ij4L?C-G&+z31Czw1#X z9AtiRON{bn)GjfT{6e<-ypt=LY);F;ePy;Ce9V}8={@XkHD_OCQ+eaGdM~ugPqE>t zfuT^VF??x^S2z2-*ea^T6}kTMKb5G^BDRwaXTkLL4Rl@~-`;<>-ZzDJGpBTwDy!bb zw!{otqP-9PKa5M*YG^e{z-CwEK#Bnf=wS4tv zjLYpekK>rBt;Dnzsk5AT6HyT!MT+cmTJepYRrSWqptQEVuT7WbTRHC_!5exIK;nd6 zCur21nTr+wGoHxG1pxqOwd7%&lRco_WS#QlDvz9^Y9_Fkb@h1XRQ#EA^jd!dJ4C_9 zBGw?nY|6>LQ6ZZdz z)AF$ZA?6bqD|fMCFvxiM@?mEb+Rgfu5j=iOrUl3 zfgU@&_;irf01{%_eOu><^Im07Cj6OmS4{?N?p&V=7cTE6sELc>;wMN0H|!X}fnpce z#nj{B*E#Y|G*`?e7K7FOdg~38!N`2?+he6#Gdh^gj4NZ@(bS6~rH=9<2sOg&79XqAAsppBV+-HC61p@6t zZ^8dB9kdgYgbKlP#0T&nHL}jdgXX>%oRZ# zb29eH+b~F8y$u_VffhX>i)_9!=Yv0jMi#6}bZRGV2cem(e%v;vg--rsjx874oZ-gp#_&hDVzheb!nrzcB+B3L{aN{_UL{aPPZXW7vEut!1oaJw zZ^?0xPxZdf0&ql>(}`y)LuUW1Z5j?r-%L?b4esxF2~O$GM&|r5oXE`@FCpFD&e*I6 z9A_~7@3Wa%2Bu8q(rJMlhV{fS5^@jN)i#oF;Af6#zgP_dW&Ei$s%nlZXnB~6eU5rD zsm)KnpS4gezT<{Gi?5m7=apX+#{6Na1C`4zmV~#eH zX~Pn<-tiL0sSuHDz|y9`fmtZnIQP!H1*CF^BGt}TDo~bz?>+ucd0!gUWSXtpZMCIE zStwvoAb-(|aaliZ7&))lei9oKrOgghJCj(6C z>GuVv@&9l&cZ+4xLn=GvEIBlDB7Wp-SE+sH{18rd%4PmGmFQ+vCD$F$uO5zr!TH!t z{StFvyduQ?fyy;DgQC|$wpZKCL1&tqwxdHBmQB1QY}h%XHF_2BFQ~g#se0CxnA1zt zSRpKz4K^Faj#_O~9^*L`ILTt-zw~-)%Qz*L01GiS8oEs(E{dGk49Ac3{wdxC+{4F_}jVMHj6s5~44R0?je-Rd#(PbY{Zb$e8f ztjQDn`-j%$`#2XkHOAjD`QP1xLa?l7Q4k&_O!17fKf%2+||w zTd-4=rr|_KQ=(q(#yl^rhp8>#gW&o2+2%p^V)w`A1nX@u=?0@f!BDBGP9eiQK6--b zM(Q4Jpyo_Gl7#~&aq8~{Qizqyn|s%tYa&3aiBs`-|4|}o`u2l~3)2Dv)K-9W;uJLh3Z|Iu@Nv5bI-r_O))qwDAQ!%g7u9S2j^ z(NFNlAtAgQqmR$893WeD8NCuaA0ffAyNlh4uib^h&s|VQuOaG+XWI3w9)3+pYm4P` z`Hx_^rir5!Y`I?=(68y|ry?vB)2)EdG&J{EVmnhm1xc`O0Fz+B*jD%yck&blYtSjY zW#F1($0g zg>ce)q|_6kvc> z#Vv&b9wVB-dBuUa!>-GCI}BD&K}#^Mf>Y$z^%c13KzIc>31q8*NAM_~#N^mQ;Sk&>a-MPTvo`tVfx zI$W(D_VZGj0K3tz0b*JEhgo>`k!EdMpWI%j9xAwN7_R z!FuL4l~J&aHff?HV8!~FoQbb6(N`%WKgB2lY&I$@h@!E5-zGM7{pqDN1Or*bP=H&2 z^>a5?(5IKky$&qYb-SaOjETJZhcfB!(&5h3xS&jX*LJrICEV6@@?LaRKd6voPpM(o z-9d-0pZ<86(YxO15cT2stY|B*PbbiqvXM%jgqaZGu>hr$s086QE0xIy+M4=jDx*zw zQB+C#fcmVJq!bGP``d@pqeTFT4+6_1xd~w+?df*v8d{P+U6d~z0AW$3iD1)a$g8}d zFv}^SOobMn@!_q9i9Jky7T1c+QiAm~!%ao_Ubn*)CTI+tn2shYV^s%>6AAS-)|H*) zH~Yy#R$h7vpY~e3(%qb4-VOyIvT3Jk5fdDKbvMhxPZmyu|BP6qZReRAZd9nPhgniU zGb*g~8p6qPd9Qx>dizWdSPQn6(Q%k~f-hj7!^M{%;9m3?_`#EFT#3F}cp z__T_EF$-Ag@)dKnb*N~<&rtjb5%V}icmT`x6c3~?b_MEMD|)@Zq>9m?X8*3h3Z@CaEzaQVd3fRLIHUL~ee6CPiGGO>rbZiPXP z!|2X^xoiq1>>Q>cmTTXh2ACy=-{~dzp{0?e%AA3ua8Bxj?oA*<2d>?>A@{;1i4Zq0d` z!#I=is-GgiKHolgR6o&zqBPBcZo8`TdmPjUOH8ccCz?4|tWHB1mKO#Ca?t5-_Dcmh zCPMM}a4OP}w84i4`l@B)m($p5K&Op1l6v}8;QS5DV_V}OUTB;8DpUV$6XM@k@qb$2 zZ$Bb_U+LF~=y8!z^>nhS*O)KN=&kau|CoH<5qXv|g3Qi(uq*21;c#iUKhHK6DbzD; zDEu@h(@uU3EI$SB^6q1S`A%IZvp_a|i(M=%NjYq-KGehtnnaA25WLb6=C ze09C#`k*7-J>h@tse@Q2x+g@lR(?IImv+5%eyWo}>lBLssbP5o>gt_L&5SGY(GcSN zeJ0$M4-4-q)r0r{v~AIs(Kq&3BD|`8S{=n!VF1rbu2S|0=KB)dh@EW{6Dd|i*C>Vlz0;0l%5;2e$y8Th zs9OSp@3tbNB|0#eEKwK5{b6x!>XkN&snY^YX{drbz)c?WB{D)UL>-aVk}pPXGND>k zbi&jsEOk6E1*(Tg8>+I70dB+7iLUe(+eW7Y;nO-sJ;^zKIuN3&socf; z3IXA3!wn_&VhhRjuz=Bx#r5AVOhC1aenv!sTi`~`C8E!MgE+gL`MzAloT(QX&502C zN!B0sAcxE858a4p*x|aFaut z8wyjDcHIi~5GlN0GypIH!At9*M=NBr@nh^%M5l6ZP7L9jS6$&3E%BqB6%5gEgq#8g z2=Rcy0Ji7E;$lf6&2EY~@!StM%M%`qysWmmz;Z}jPV33A=ZGCp2BZeUMHb*_8DiYX zF{fV1x!p!&n8zYbO_UjKfD0dIa_5u%+$ne}$6pqJ2T+Wz@*$RJ*@4&-CV+9ZFqZKy z)Afp@LA)?)o=#kXZC z1Jj5Ca3@AD1*)72*sCd0I?lOVhK2QpYC#xgMe`p})ba`1>vgNit_sW!?w-wR{@z095 zNi|lT?AN|07>~29AxCwx9JZ^6`zGhC{ zHDyD1UDla^r2VSKf0tfaRFJ{Hl)?u7VYcxIb53$}Rb4WAnd7Tlj{|F+%ynZ}24FH0 z9@Y~6^e6SpHWp`{WzZPQ$?0U|r33_k*uaB96%vN@^{_lTfJTJT(&^CPn+AzD3{XeE zyJWBAEH7wUsEekqqpqlzOL}3g!n(xpn2kPc3HKrpQ~iea;u4tb9_(}u#St(pIXH4zp?)4alPi!Riw;DxN5Sgc z_jXHEN%)+NE?47sbwePrW!@jiFbgRcUC(lXQKwHroKyHo_vN%iS!q+ASXl;Wox#-} ztmZ;U4LBf_1mx1TH}g`%XT}i>%UC3t;;DNED1AaC)xk5a1PF+3P?j$Ulc5}*dLW&T zuDzrsb|UpirJHp7dg-|p!hrlQ;Np;WSgxfj7owkdu}G;TpUi|?nb7r9K=k^qQX_!* zKo6$H0vf|m2=~(k;iw|#^v(w-Q;@)U5DJ$EP2aW?>6PD^AY4oYPdKLl&fH78Dkw2J zy^$0kNp@EKlt*RwQKAv4P<|dLyek!i@9CtYplu_yUQ1xEQB|0KE8farWC!Yu; zReW!Q=MU;+atBPm2g>v~DxdP0aO7MuS!k_@gHb&xP7m#aL|;2cy7dy-qjsE+$LZIt z`h6{EvUC|0eH2?mp^quhzLau<0kTGCf1E3XMt&Nlgi*ZHzB6|JMa%!`gVMO9K@8C; zB`V|2>@jUw>m;Eptcp*w2XymgQ|WXk(;&Uv{M2sWt4Kh< z8C{I7ZWXD}ibQA%VM2V3qPULDBA}NP2jJGKMRvr-yH3N{hFyl}KjvnzZ>GI}I`JHG z{01YhHw|Gmt$qmbHYuvuC8{}5ucijMP3!jjJ~4H0cssQYQV#*<<$0k}r#6I0Z^A|h zSq>0lcck4EI4xQa{!S7ZzyfefD%0UMc*@c*yr-Sg!j%>GC zkH2xf|E+p|2&MgSmtuVP6Mh!5*xbN{a6tz<0oX`pAY*)#^XvxG@-0frrSRb8jn@om zvQk?98d_0vYDIP2O{#C*7lcc_b^i1rl8<7&0xE?7A{eg?xJJZob{E&x4e%)aYhEW%f!A%=ks#)?re2GB0qPpl_t9kd#k( zHnFixVn6)$*p86QL(2!|d(LD zxLOgEEM=5=E4DLPH1&C7Rlyh?A2UIn>wjw`U~^`yFPO}Nhib=xs-*hRPtg`^^-pg- zzV|0dLa48%FR&61h0gu(tZ+8HzKh-AzvOVLEq1%}+auEInB1I|;?V!7UU5Gt9u|(u zg>k~q)}~zVINr%k(>gvcv?B5W1BuyGVheMYk0C>O>0^RX)uJA3mV{D3Hc$dYOk@Ry z=n`9kSa81JIjkqIL4ZD6J`tqO7-_{dsGYHhiQ;zfxs~=nLQ^+~D3MA{{K)CZN2N&A zNszGynit2nXuqB|Vg&^NzFP(*cBNBu#YQbVX@9zXC^HYlxl*C~ce$Pkd&xw4;UnGf zs%*O9jEdL^e#q6h!UP0C9yK!7JV{i#XLKmoT}QKB)4#JHQ2k}J^PHUsReGM zUxEI+I@wvxJ%|HAfMX%zRWq*=RWn$cNVA{x^qi+`I1^wQ zqjR(uu&Bdho{TFEv;*|}a@GKC%}JMN$Kx5O@9uZ^jCCl0UjQHJ>*ud4IVo=a6rk!> zhnqr8b?bE@+@E>+?x$JD3U{VENvQ+~5?~wfVkZ@W1+JdDYhv2>2EMMGUt$J$fZ#NM zvo_T{(8sBBAEvIDAFVL;Yu}rzureyQ((QJMIc9v$tB$ zc;rwJuzJj0ogAQ0=}CvfU^15X1X%;{9V~Zt8Qs%vKcEzdsmn-azWD`x{9Z3?L@IR; z#xjN|iaHGU?@R`+vZQ^`2I#I|N1=3*T(Dfy7U6!kgDd+Q*^-D=P3=zN!t;9}Kv242 zn$*MBpFAGi0SkJwKYZF0V_C%L7zMa6Y?xnsPliu*pM%&U0KyRD6(%9L0kV9-WoNuD zh^;b!7=2@dR6(hm?FLL?A^TGAIiYpD9kJOjq*oi4s^ANFhTMKnS-3N;#upR}Y37bvmQLg@=oALf2V*7Q%tNmkz@OmJsn|%LbS{pK^XTzUzr(Q&^@GfuD5o z_ER)UU4KwddRGnYNqdw4N#@J1Kh}4mwKt~s%G0=!sB>3{giqFjDI>vQ2^!6 z{BF+kl_m2x`dxTQ67cvSXaovrx&*hzIH(&9(5&LWQOy|V+X495;r~V-`>f|?iW@tP zI;f(ac-<1IHzD6Vug63Qo#GJS|HZTeoMcUOTd$}Q@X?0ZYQ;-bTBT#q7^bI7#rbyUAD>J zkwiI+FLx$1bd|>XM`gjX(Yo0Hhh!xfbgQ!^%3-4jh%ieaLr5QZN`fKHCtTF&#E4c@ zi}+;A!cjXQJtB%IQKu_4rKfd>WYgC^uL-mOH6pqvQoyg|plaiY(adg50C26bD3?;~ z#MT(+uA6w>Y^uC9HEu8&AhN{;uQeSK-2ie5AF%u2qYdgYQ_71;$wCnQ`dc+`J5JpV@Hzk9)dj(E{4hwYE2c#g(N_upvo zJaOrOy|nxiO^T&!dfdlAV9*1b3Dy72l2pAg>!n}F_SBj1cYnFJ$$n+^3DumNA~|gT zERBhktuy5_;THlY8`o?m2{*D}frs#D<&s^>ETBeA!1Xhw2No)n5jU^KgYeKPfwm~h zU@iXK%7S})B?+H7A=6}ps=ygT=>?(NU$>5u!rc;DAS~wSV6D%Z&Y@>$rGU^R7%(ZD zeV!(*(3Bpfv>^DX3$D*z3x7UQs=|JwmRJo$C|H^pc_aM|qeNKvr4%gDeiYAyj^FUEh}lka)A*r#Ul`lMBz z?!MsmU^76~M7Oefgt=9hkLy*w_SBy^4uZ)8VURNe#aIZh?39nx! ziPk)z90U$02t6}ow4$<@bAdvV9NsMj(OzwHRImHv@x@DpV|f7>%6a7NXqG=N3#6LJ zgo->%MgYpMRFZBHs#LZMuWHIFm3)@JZ0E{nIv>&3JkWjfNFp|*oaf8-6qu(`B{L{m zI6upDJtLDKEDr(AZJ?P`n9f~T8mTvJ8Aw-FP2|IEO}rdP(JW^)N;gDZ3st;F%PFye zol157a9KE&S7R0nv|dw%KO^sDxf>#~-VdaajS<@Om<&T6OnkuCYr?843Dhm5VIfc= zlL4YT)#uN+^J=+iV}7e$#U|GDVC<8^2L;MaOn9b^YpQA@VPZP%>BO`OB*Uf<2#fG3 zE`TRs2Bz)P#)i1w=0}*a_ELJzdN;o&<+z>nOcB75^r!T;9$DoSP@( zfB?Zo%qNv@GHvmHq#uE7gNeaES4U)pyqhvJx=Un&YRoT3&$g#Qgjyy*F>m{x$j^_m z%AR$<2J%=53Y9zr&-fN%0DhI3Kc~{rI8O&2vwZ zUxZ58hRNzM_iN<7M>C8NnEv#8j#2wE3FDehkU)nhQWC_hRR3y1(Efe&*BYI%h7P zg$AAlZl!Px%I(tass1&IM7b)Hg^Z%2qmt2A%5q=OJ}__eHkY*<`0vIH|5YmB7)JNu zp1?AVA4!ufP5DaN0lR@% zJPWH7;_D>+M3Q6 zm@PAAag7W&J(f_i_QI#d)ArR!V0qTryQQxm95i_sp+}qZk+th9QyD(&J$T=C%d%_U zG(YZ`;$hys7iSlX8+nj<)0W{g7--0VxosV$`59+rv|a0<;G#vN+faPT!p{D!?Q8W4 zJ56tF_hUN@61OdE^Q+Z)M~bMWqpdeK+0IRWOvtqzesGbS=DTn2zt#WtG*ej#m3CuB zh3B>8mFM%P+cnW1c;wCi>73(9zXSvEXr3iU z`sO+F?vt}>_Z{zc`s)#MXv)?)$OK3DFK{hvCgF~B-Y465D3F~rx^?uyL0g1c*01k% zRCvDGXD6r${H|%+{Yu%WTLC&(sds82lnP)L1IJE@gv| zlvZ&|CglA{qFXeco2lta(%3bTnKiKIy`Gj^a?o8-exLRL#P93`CE>uBj!WfXzn}Wq z23^IOzkRBS_5OSyJoT#61q}fVI%7=#OOSzB9Vu}MdfSG1@IK|9U--;43Ibf}yK+#t zSLzTX=2%q0F8ywMJCuzZ(xB)!$j_zjug*Wj|B}PISNdB_vX+<8$793ON2MuwdC&td z-QI1eA+>YwFPwZg&HCD$pc`?|El{i-b3 zpdD44r7c*w;zY3&>G^6T{wPvoDAMV@bhUx_nawhAXLOEcw%u!} zP7Y8uY#G@N1oF_ktNKHX?KFj7dF-3zi2z5$OwLd~BfOv2i;T(&nJJbJO(YN0qRIog zS}4FWHzJ8L{7aEHpuu&`=vyh_s?zwGiYOxB+1^NSh)*Oo{B= zrQt*hBGLht>0Av2&eW_6amI(tS%oWK!qOy!P8Eg88GUkCS!hHEfH&~diA?h}x^jZ& zd)Ns?x>Cu=r6zZk3b$&RZ4dR0BtH4P$&?a*x(S z6Dz+0;T1hJg`S6gW6n_EGi08fEtG1Vu>3{wN15QCD{~GT1pKb8M^`^^$c<9as~56! z)glbJpXblkUSriY@rt!r7G3^D^cW!6o+zdMDla`0;OMyOXFSLo#7etwJL6-DjGpXc z#(ygNxV>Zall%muABSsv)hzMpO8_#PbAPycgDZm8 zOO&sotT%^-Mm~=O+%(h`%zb&ba<`U(AwJ$9yX2gxX+@;nD$9khrlC!?mye7q0na_1 zuh9RCLxuhxhx%pzk&UWwWXz`5uvjguVj2siJv6ximr?)C?*3Nn=ui^-l$q1PfOyk`QCF%xs-puSf z%bQ`J<;|TQZfp%IbQ+I!ZEQ~Q_b!S#rhkQl`6j%Y5Dpq$=%r{DsMLF`7IM%Sklk!G zajmwt>0?vxlJw3_T-(W)oS_$n*MIT+;otmhx%{%l8u!{T%Ko;yV`Zt0>Uu94y=*bD~;8SpUaN;u-MyP2CRL5 z$gD1(aC~&^eRzgy4b|JU(8JolEOUT3T$7XE=J|>@>#Ns%B-2iO+FDBfTu}Kf zRC?`NRyePBtMG!z5#`HVxmM%uW7s69R;zCs>p9prx}KYlz{OT+_(U!~7-@m0t?-Qn z3LLrJQZXK;Bc0&(I^=jq>#+ARCRbjcIMFT1l;E=;I?tPSjqqTQ&Z; zJ)g>7|M+jz3>nho?lVkV7mX?4rp2K0ryJjAq0rPLyG^D_#JE#?zh2Y|TI|_+xeyxow(q|^ z-9DeV{K0g)*ZTg}(#JNLf}yALU%Fl4_?>4@tJXbIwjvfa>j_)m0%+?SJ^Ke-3*w-6 zHy1h>r!kF2ij&i{@@Tl|P_8m^=T>iF+c@Z`gvg-aYtNC8{mis{@M=s7czKV})P_=S z{?mHIOU{vjeJ^iqmOQG-DUWF#?P=O!Z;ySrSs0oX+Q>anj#zRnT3XFkCK5$H><{18 z-EOjf3q|zwJ6u@Du}|olQl6jZaK{%Uq<7=1R9p23fX$?;S#PtO{{oGh@*JOVl!-*C z-3Y>)AZJ9Z4ZEZL? z{(o^=29SUy4^^?*CVuS+H6#bG<>vWc#ym4{QUq;y4Oe}*wj3aeSPY23IFZkmQzEb< z_}M3Bs{M-*9E2|S=P?B2eO%-FYF7@nz!}mUt@X^s_lhnNF&oh^lrQRAQ@rU)Vq(Sx z3X^SZy>#}r?jpOeSStHgu$f`*Gr0}aZZdf&aF#)-o|(^I^`abl9&Uj`e;X#GxRNKI zUTf9$)+rAn`{d!gJzx0@uVl+?DK6^F)AHGdwMIRmheq}~DTV^D<&R!A=0=kXhn+F^ zH7cB#^UQGf;*9AV&oGLIrgn!aN^M>}ea>?f(-_yp8S0>SIO{C`?8Hqu|5dA?6?dmi zwY5yqg~SxCTCLCfDfm`RTmvu{==rr)_FCqNdi{c)eR`8&noVb#kXi=PilIY{3BmIZ z)vA6)X?v1VM%Zf$jqA-f8uF?FVts+#@KapCfGhHIWP8&SYLMR0^ReS?URPD^!k^Da z2GH*`adL;fm;N!K$jP;D5%;c*M*n-v;rYYPogcrg?{Gg4Wg*{>eCsHy1rBBuH~Xn; z(1!5^pYA!gn5KDto3xHap?}zc6|2dQ;3~W9a-#X03N4(_L<~DrZ=$>E$G?UVc-*HRye|&8YZT z2=~FJ<~c3GtuoPb61s*CLK6^-kBq)a3}w_z1YEu^5R*?3l@aDQo#duK~DKvAfv?#v(IDV5FCiY5zR;S_Extp{1j02 zTUDp=l(97i(~^Vp*}cvQY|>oXycw|8!odbBkY9%qZ|Ga*&zuJ!#^%6jy%E=K_D9<1 z82$*i_WbfK`mG8*U;0fg{LEjB?h-ay&mYS6sL=0<>)#U5);^87o{e3;f+W3t5h}`F z?`uu@hVj_hw2U}5z`XT2h_hXCk~3Ewtq{u$;&7D<6TS?d#$0>}2ZwJ|Wn_CA!oM`> zUlob|Ed&HUem!LHId1)$YSDJE?dPpq_1pQOt!?@7!xBHc^}Kw)_S~Rjs12{h8jihV z^k(ab&07!bGQpVTFwSoNy0G26s+oVbeX?o6tli}lAJ-3!UZ&q4?i&@=@0<}9iC44) z0ZSc=d~$%*%LOCC2DxRaOdzKH#vN?iiJfPS%?)zrAHl0dUm)Xf*RjTwxRCczAkGoN z(nzF=K(pc(sX*f=M$7agtP%g#o!Fr-mrdV)+_0Kk-t@S9Vf|Z@UUBQ06Bn>Mw|=V0 zwQE{*C%9JKYzX0~j>GfUe%JlB9y#k`aF@^dOh#%r1?)A$s3 z;^k`Q%qj1m7i|q|2di%ijjaci-c150+4!sf5o5Hmy@Tu96+N&13HY)V5-n)RH5(N>dL68mxkRn|{!~%(hj`UswN|BCK0U=25L3)$W zktRtfCNN39+|OF?_hZ&fCX<<2bLO0DpS|~$XfsoN7Dj$XDk>@#g9rD_si^3SD9Z)| z4P{U3N)Cduq7F3I*P*H&6WpPExaVeL;BI0b*NoCP zsM%h1iN0dE)aS1LG36G+70xH_%r}JTXl~rmSxLJ1@|rQGGPHa=VJL<@5dg z8_!)N7yl}N+9V0c`fk|v#^CSYzf&Y3Z`Q<9Zn)$W8<)9Df=eG)A%E-gXuzxM7JuJv zh(;P+W-PwZ$PM!veLk(Vno>AzI;Pvk0RJ;C9R7?mhwP%aQQlf4e2|Ef!;X5#%FSwj zbZDeA_;bhir-rU&`dXRQ-ButRjh7v4TD3ml0#mbDaPs0ZUWs*C_m?_NL-ZWX-j&~O z^cg~OBk_uBJa|V^S4L?Tfbl0ia`^BpbPQSSPKDnmlN1|i>^FTkJ$YQB-5>RFGM9nR-D09Mq_lLd=B6BFvYK zuXn>0_?tel{h7&xFAm(uVn0*ef~rYKOkKqAhs`PWAtn&fw_1vFdu2WOJLv2wAuH@) z|858Gh6%i1g_d}2Evq8~5BzGO){KT%TruBUM@dURAI+^hCH}nCW@}|r_K|nq?y}mK zT$%~SN?}P}g88f#%&97tcIcH>H)^h_$Ae!E_*$bfJ5w%}78EBIOc?CUe$cPzDE3Xb zD2nDDDIWbyG$+dgc!vNCmkC*P*Ls(DqJO@+&|v?`BN=_PySmSdcW@l235q}{AQLU^ z>qia%Yx!{hqRX%?91{)h0&Ea8UI$!vO-42G`P;Qg88e7dmD>!4p-h0Czk% z|J)pA`v<}u3A%f_8fnBt%pC@`_}HxW$5@?-{+;R!2P#B3VcQ-ZtKixS(#eZH zh{4>wY>dWv^GfqBWS1L`B$>ob)$4(o(%WB45apbCfRy*X90d;=Ui`GWjAGkI%PeCD zZ}0P3xyhebommvsp=;ps`SD5~_`@^X1vjrIO;Rn?c5meEz+f%9k)UVjex|w4tKA?S zr?295yeH>7jrcQtcfGzhk~ntZD*=#t%L#TZBeW;@KQ2#Foo~YVN8lxSP= zDJQ+!kGUd+?!T1Tpnk`6EnFcsax%u)3eQ(yFeQ-FG#7*ssGhaArZz`+76}+mSUHe5lI(f@ zM+SecP8Xvy&L*+53Y9&w!jfuCu*UD@S;J8LG%}F1Q+oa7G|~}QM4K&`0x*S`YHTLt zeZ6{dK+)%f5#+jQQ)Pw>>}AavM!bO(B6>YK8NYaP;|0@T>A*oibq2|26Db`LFu~6ZDDqSVC4pc`&4(ev;fC;&Y~KpF1_dh2m#}yB8q~55}tSGil!8Xx7P3EhM&cq0_~FP};ZAVUypQ zprb>p^9j>$%F^pjYTB9kMb5}xR3?1;jPvAao3^(98R8;Jk>8_!G7WB7K&jJ5U#^E& zW|ML`XZxF-kJ4g+seRhZjJtxtBig<383)P#jQ@2?^(rsFi&?McB|(t zBPRR3clO>KEQaUa6AhOG6<1mp`2c=wL3-96u&Vn08}DGUgAcO9v$(XkZ!D#PU|O*7B~?xO`bXb0i_v-nCZv*RsV{YtT4nN5&J^MB zB)|-pK@xj1iCEJwIH|(K*2UL%)^Iqu zjCTkm-t%#lE~^8b-rNr^&ht8{g_<-lQ(nTDYZ0_8wS2ZJEjOJ2zSEoEHbbVKj#95f z?wh=vFTvJt^^f0@Tz(R3|J7Gh@bu=dhn1|9u*BvWlLj#ewkz<4D;H9Sl!0x2C)HfJ zgq6-EWutO^7NjTnYS+unTuQ^~Af;?#a{%v0^j zDK+~K{{dy<`89lRxq81v@3zttsL?MW7tz~6GtPVvb6NOZ*=uN!6{+bEAChB(zhm-5 zh(k8$y`ckl%yXGn;&itnV`2b7rppBG%BPpWRrg#Q^MhdK>V)K%tluvOLwD@lYwu>{HH6+JBY*~NIHxL0e ztNI*tW>fj%dUmHp!>C+1N5yjos>0*a@AP@uw^+%6Fdd*LTJTZvWiZ~J5Un%>B`~4a zvq;xX(tek4Sa|#D$;^p>cZwJr{h+9;KvG7W(K>fd zqo`^Zfm4A--7jLI4b11iwMfk8mU<;v7WESB(7of*qseXF`vMo+n0t<}fg{K`&iM_+ zCnO)3Ehfbw*+Jk|23ctfdd#G-1lhpY0zPMKb?RH_Xwcl1)5VYnseOfBIPPijGzQoM zK~jB)F)$D-kpAMSEI5FdjKsG1h6s?3LICtJx!YD$sCpYP^2d8J_s?Gsx6_bz zTX*92*<`4y8cyq7-)Z_}HPXc=U4j)U`ISGlA~HXc9QTnPCUX+dAKduS2aAxKibY`z z6y)c74)0i)(2QW^HM{kX9sD;3S`Yvu&4kz=7(VN*+UbVEu{hWf=H*>GO{pDriXtD| z?__P*t^^Z`<#K`$>R^$YBh~&I_mT4qF7;!dWJzL5%8FTVPvTnMWD@O~b z-&ov7w*Wd-!w^RMS%`xAx5-n6%gW!C_O0WtoZ6p0=RHw1Y;%V`Az4K9{&X zfsIo2))$Svt=8cj8S4VYYu|gfB;Fim!@oz9e|aL?Bu@duA7$lZpX^I6_SXw`Ps@Ci z+$x9kBcHZ7nu}{1-klun-L*K2?WU0favPc2Pv7yFvr2=j-aL}oi?)<_bB%QD6iMKI z$1sofSe@McP}jnwkR>AE2S+VoZAXjC;}sp6XfgjzcUics7VAp$Zw^CxUTVG;My3QW zvWuHAX+xG@5-LVFc5%F9Ax6Ud(>I}NQv#M3UDtPYNNbXn5tw5^eDX+$C!6I?=(+Xt z;NV!q^3C*UAQ1KjQLK0I#2;&MU>N)6;qx>(e8@*AkP6(M)FBb^S?}ofg5>d8!-xx6 zf$#kZKAZ)7Z;L1OQU=DcZZrLs`)GP{9^&J9aC!A5u;q1ee++sPx?=s4Los}I1O0S> z1KIj<$`2|8pF6BYn`dP4R2_TF`D?Tfz9Kka*BMqZdvNne;%B{3U?~@B8j1L`nhBql z-H{pk93nHf0gWRx}57?KUf;WEE4^6heCpvmG`XxOH&Z6E46N9AZzZ)`Rx!#3K4r`aLNIrPQf^J=S z_K7Sk3C5Fkhrl=d1O4F9wPPD&c{G)>iH%U6AOE<+O2q*Lsg*UaYyBjSI zjz9GtwQj=b-er>{NM=Ep5XR5O`PEZEiOGnM;ebJwp_KY!GMwO*!QWA;8Zs<(ymNY( z3retnKk;yc>*%^NcIfW`V&U20nPCp6HFc);SY1;Q@uBN!SsRkPz@vaxOv7=YED1(@ zJBukTkydgE9NI1c2Aif3A|7K2t|nd+1dUPO(GDKKhVn=!(o34UVUfi2)yh6ZOQ>hn zW6w7#`_1@2>j@6wHTPX|LPzuhK=3}m9(X~t#eDe?N6=vW()+Vdd;KUA&I?S9PaQBZ zT8bYe+(uOR^7ja$eq@d&oH!i#%pLZiX0Y%%|5=c4&;rl8{nWVk5tb!fx8D#fnU{;( zQ01SSQqmlpF8_n?hNqX;{l0H0{^(c|DQ5JdNn2YEAivZEaSkr_N5&j#k;9Sm$SF+_ zT*6p3GyY$%BxF_cpCODfUxnm5EO;q>DyvMTaPF!4_=5|NjH35%<2J4Z=-Rgjy>9l) zc<|OPNk(dJ-!E3@+Bb|T2*JTuQ1)*MKg2_@FtTe#HRQ(y#*7q3T>4ae;im8JJ`(63WBrV`vY(7 zJoa$SK0guD3=_3zKZS0XZf_Aeg(o=lVe}(2-=9^fGSRz|r4Hp2Ev{Q)<;j#r%2T*H zJ{A;3fe0YP!ps+QkAW)#hdfk2&)r;qw_Q8KX_(FD=??H$O9Vu2C zI2^{8T@)AC-69&9qpXPb!Q)N0ILYl%;ojRwLRM<7xe`um zHlK0KO4jrlds*C-^0>+Vmvw5gs*eUyaZ6IKs@~FFC8@1*aFWrKl(S=T+oF1CFMJn$ zsm0Q+i;}8QIZ{CdQhqnVz3V>#d%7nBMpw4hA$t-Jg@(`6fo(ov(CLw)Fc0xCLUS}7 zU6;SoxJ#SfFL)GJ~@~rHCfY?(K0qw$5!8<_4GZBTl1;PaPZ$?QjSb zR?7p-VY0r8!*?JV$bb>TJOs93^u-F`g?*6>3cNK7nNwuJHWt-cSKdMKz?#->uDDf{ z+-uxGTH8^Y|04f>>+6a&gUHT*xYp30;FDm0p;nP(MH1y`J0P|tlBq(5&*9Q!8t~{+ z21svK`L`c55Eel++^HFrx$1BrgD%JTpzEgjZ1Oa+smF6pxfO2zOfz3$M9uji{1=ai}VSH;sSqtNP^3PP38vt zZ5buYFh+*0<(T#{9<1SgS&fu71+ZJna8Hw)!s>VYYUdmC6h3!+&|r@*a7;SD2aPXO z?vp5aaWRpXzGeq9;3~N|lUf`xhAII9f#Y#T^iT=u;Oq-txcHyqc5gzEUEpH!B(BkS z2+>q|jBv47PRu9C6E5de^ip43EuWqkwh8t!tS>e6IMX%nSG+FV#fE1hrQGcZef0gZ zf4!(`Sx76(c{n**mcVx!Yn6(dzB`LDAtvasp367Qv7P&tTU?laMHzRlBbCnl#-M82 z-P4)yexNNA-B>^*cGuMq@D4{euljm+IAayC=W zGjerO>{7PZ1M6%vzBKT8(Wb;j#amQo-^53HIlId|^;HEPhl*Vs0oO~F<4~rK@|mqw zD}Orjzg+4IZupa4#F5;gkY-N@U*7fY&vE2VC|Q z`{eH8w`8P^1a$xtItOf@e*Xa$d`Ll4;fk@cm(uZ$upQH;sB!!yHuA-d(sOo+M*}`~ zcW?XUKiBAQh57BGil|LsZIZ5*8M?cy|z zfbl-PaJ67gcp*6)(xgeYDDH+$BTR@(8y=q>>G>!)t+9^O_8-EgG(QTxOMO7B+V!9- zN0jR14Vyg}*RdiEhQQE*U;}4aA|EM`(5y6wULwC2x!w5pH+q^-fuTI_S=1`Hzqkhy zmd1E3g2-EPt~2YRzeJDMN6k<)a$Iy&S@Q$9GR@1&!BV z8MX}166~?6Qnrqlc@sbVr51i@AOpLkT@kRtiM1lRQhgsp9G}u8J!-v$ zn3a1W7vsD1iULy*JT1_Xq_o=mV1Uu7ZM%H7VUS$@pkSy&23|CAZ3|e;Hy}mNnFbBg z7ggB663cZzMj@t3cB`pRgg3Jm=i2OU9x5iD)>uYlJUu-IdHTri9WC;G0d=&vT}(VKAe zd2lttJqP>b<18U-9d+ViL8`(B!kTvB8$M_CiD1iDp<@Z#O=Y`aH8P_T7My{+#^^og zzk~V!Dr24PhG`slTabULSh6t1z#`f|Ct~{yoGVDE1MfnzWoGCO+3H*cRSSK92=xQ*G z5h2416cM6r*`)zqQZ9(NFQv+Z}+UxtVUts@}hi?DCmGfzvA(y9Db=-@SCXxs6M<> zxH=~0dT?7flS#&<@3(aO<`3U+M2cESNR}ppRCvuo{mD0{u7uJ};$^>5dAJ)SDFR*# z$dj4j32FtEK&mmb@y2_E2|>Kj|9k+A&s|l-YYK%(FLXi-&xEZ7;&z}{NyhDZ`*!5? z9}eOtPNbg+H`6%omN#{dN@L|M43n2fJfGF+KJtPd`y*pvg@|+P5pNQiig;+AF9CaQ z-fq#)M5wQ=tE-{Kyk+B|;M4v?YqmarLL||!!LAKKdO`HLUFQrfru-0LUK;C<_Fd7~ z8M-smA8E0`Yo1pE8?(3eQ|DRfPf z(a(Qp+IYFzV-eLG!m~}65IsY1!;pacyJf{gUq4&jy)*Y_$`RM5*gEiN$^?*Nwk9y? z80vC8l_ium+0mmNk~l*G0=x_wJ|I_Wriar$H)lW_e=Xpl_J?Tc!p7doOX%*YIpubh zlNt`66r>7jm$$BoV=g}qV3DDDEA95iG3U$z@LAKCV!8er?W(cZr<*4U+$uG9;Ya}mo( z**O-NRQ?B93N6>OGHZFNS=%IN%nb>X(ibbEU9eiG&61_(yOp^hH=qu*kWf?K3ahmx zLWCtbk9J9Thf30co{ayc3k-|dLZkb!w|rcsLgFi`tHILkYv>ADTc7?~sFdBG?J4V3 zuibkFC8ZMQUZr~YPZq!2PdC$6GGH+`))DJ7Zi8DpVgBk&)yMET#av1Oz4#}3u@>qh z=1#+%aLCAl;ioI~_U(JsLs9Dc8fW6%_1OMz(t#`pV48B%iR*&*lr26@_8s`7 z3Y;=YhQjzxD^6Wuf)pfE%z5>E;&TLfDKdW0eJcF~D|xSFvQA~|(K546Np0wjGt`L> zTUA~YbE*0=bx*{EQvakw%}jvKv*`&he16=N>vj8U)ACHov#7^UXF+y7ij3R$)dDKI3hl_t)M!u>oj<*(T_a9^lB&-uaWj9^BN9~%aDil0jO?MM-N(oGW}H$lSqCo19a8<~hY5TO zh%M5o!rac=LD9e!;6+c*Y=eB$kn6ni;b=Oe1N)On*%V;;ALsP#D~Jjn&)*zBP1(@& z5mRdYrBY@-N2J@K04-kM@SEr^4fZN6tRG4niBxtAn%uj&J%Y!$+I!^Fa%BDGt6xTCGL*;=goLI?IW#^Z2ERwN@%@z-OMJm$y#{xbobPjfE`*#f3 zUwV*)ElW26V2!LJX6SBf`4q7;CEvJmM<<)>=F9;LK2;>5C<4`LovVo>_yT@4ezGsLh%;Z!WOE!Rp-X?V!u;@u8s;H#gxu^YEVKx3=^G z!(831ehh>A5yMF7XZ{b>;IP&|=pm(T$=%`)-0uT1T|3fA{q^)6^hU8ImElNB(&MIQ zjDh(#`g#3xLW4psg2WR3`V9H+IOH3RVeQt`DcN%FMdX8N#ao)L1dVh+9anh4Gpd_A zxR1*Fys*|cx-ZDUBOe=_>qV`64J?NYkU$9t-W1r*R%7ZwQr;r&HmUS2zD#)KUyzIRD7~v z_bz^C*M!kaeV;cyf;eVboLV~dWiKJxD-ll7lEXc{R3{NQPz3o02;2>StV)U84@77X z*%goS-N;({%3rc|WKFOH&13!Fs!2H9xYo9?Ey%{?S6C#Zg!-alQRJAVsrqa5Mw7>b znDw_rjrQ7-gepA!kfEJ)V}-u62N zm`*>;li~K_FIbD-GQJSb-|2vHOY=&~TUoFWlD|P3j!NV&OD-2Uob0uTbf@ZCXoDXOrolnx^c zn%Dh@pqb}8Y=pPu7)>l1TY;y>tD5?>+G|U%34<6Zs;!?SKR*PP*s$U;cj!Ql1*Lh- z9jX)GXI4`NmoE~Nf3#T2FuHH%9qK!Y{cx2VHIM!d-FHrjAXXFD^bEf%Q)q%e{Yzj6 z;{wW6=vdD!VAu1+diSs@u0qJvRS`cN2BD3{lsp}eZHH%Gkmc5=exoD4(lfFPra~L{q^oB zuD9KIV~Rv8f2tE-R@c)10E+^9IQ?L*#-oI=xEp;P4O2QPKswZKO|nXeLh1BK zdTc%c_t*Rrc-vUS?D9wD`~yPk_XapT7ue6d1lszSf_$dFnQ+DaxYX5qK>9vo>G`Vu z)y?e!EiLKk`-Xojz~abrC6xYlDf^uDsOML_;0xkl@y$qN8Nz~l2z^t8d)hA?5?>8Y zmvQ$34H{V$F`IV(!9iR!K1&?S15EiY|8YKJgvix!QXu!t%vb;03?ju8p$UfW!7*7j-fg|0=a~Y`cjpjQ{)KeYhr)2?#7yS^*|igQn85J86yp&NE}YT zn!KMUq^?cZC7L!4_Ohl%p zAfmEwcgOYC!2k#+h58;%;Pj6phr@fP>B$t80q<7`e(K2O-v)ntb}8doz7K9l`DOxz zu(0QRx;PxQ7`X?uwKWI-_yM_f;q81z!r$i`htth*_?IV-{U%!7l`_lt`N&f7rk+9W zD~RB!l$SPFlH=T&;D0VIwAGf;Yw(|Jyz>)C2Na8T3b69U zaC#zDKC@-z;6`e5M}f-nPRmN^znUBLfdL-XU~0P)|7a7Jl-GF_f?_)|-oHfxs5(RH zoqcWQvB)#U9xH0zL>7%=x6f=(8&^CkfMW2j_I|_et_6jHFl-&{h8+wzBg-FOZ=2Wg z@$`ds!@MctxWL=@7n+sHFUi#ED*H8C8M)Oj-Jk~{Bgo;*!Z-?0pr~=&DUW?Yh{2D` zCj@UisvGXd>ZMvVtv?D?L+;^O$SrYm-Xt;xv5kGpqMd_H`gTv`lPiAU7-1W8O(&q@ zgx{o3kM>DGw`9}l@)OP_hnuwwk*;eu$}ZT+)2O1Nhd6U1AXF}8m(c4hAsoH)o&Gyz zQ@eD}jgBQd^b8X(p?w7KJb68ku=jy3dng0z=6$L&}6Tn9?4yTyNk1QGpF!r=tp6ib z(d6Jh&S2#`5&*9KlP^5^>ihp*;Gdgk zUT9r|k04?CFC}4wEtsrM zj*b1YCe9S5MB>`|BuxdkPgL6k&y_9^0^eta`;W052x<><$AT7awloiiioL@Y+)LM_ z(*E}L!!5!6{jJvCr0C=J2*_&t3T+{m>s+DoF&I4}xJ-{=^fViy)mu%8Wc5}$eYqNV z<+)eYU73Lk+S`H@n_pWgtZMb}stfS>CDUKQ3)zf3`0H0#Pe)wt?MjmQ{)vEn?~mO! z6w-QqC8_<6wyt%X3oYnbQ+wW;ncx@e?7>KrN3fQ))0lN_udrt;_1oR?!r_sPUBy3y zv>bjHt_T3J9uE(%&aojkE|&fCZ}{+LM|vTVl!s;Guj@n-McJz0o2~+2*R3Hzt8p3cX|Zc`~i&5q5&LY@~fnfk2Xc-G)8tD-^JNP4l!2y_j<~J!r4Et0~P8ELM1;tw6hS|I*+`=v9 z^Ss5`TC_9rh~h-yy^x|zcE2TbRAVvxKc;&J%uIeg2?mkspZ8PPZKste)mL?)sr|@* zq||Np3*5ZiW|Ds=gla~6Y%j1H2tyoeWP74ZAkkc6@j!ytE7RI> zTd&?=em-NXVtLRdMdP*&&k($F#)2L}eWrTtpgt#j|3va3E5TV_{oPTjVG*Q9k6%_! zp>DsoFrcw>rGZA}9TPk_VdC9B`wrh`48(OFvmSAcRXPyR{ZZ6=LWj-EWe-OtzGD|u z*g*G4ZWMK{u87`X$z8vANsCRFC4l3Gb;W?&mA^XlY7cG=3kmXzO)7cnz@F;Xacf71 zD;Cy{$oG#+{7$yQi-TP+u>)(R7$XupHz`>7aqGbMH29l=_fKh|gtEE~^dG2r4T&2O z(lgf-JcewRkh*3Yda8T5E5Xo-BB`W=lPGr58}zV23Z6d;t_H862!s6hCP4%QUYhW0 zotwPs@o(H2G!p})jyCn^xBb~Lfb)Z^;ImjyUaiYKW|s+w^%{H`DQ!+1+@yS#uqJFg zi_#<)=^lk4*AQb~fCV9fRQkLdChSj<)1liLqgdxT3QbNSjE>H?0F^B$AlyErOzs0C z&YRgSNWMS*4qEgH@sW-&h)6Qh!>8Fo>ZKW&aw6%%F9$lEduoYSS=?%*rBK$ZFdBAcZ;$hf;9E6>xHvMi) zbHJ5({zzfn_`>bk5W}@s*yFN0$;GR+_(V4N0 zb4F~$W3}_jC_PPcCD->S6{;~H16R*ij!6X|pL7gZGr;uKt~9#K+TbwhtFfi8Vq#g& zv*eXzVd>s~r!QHLF#Tf&9g+WH15WQ@6~vw;p4ma9Y|GVty_ILSR`#yv3+L#<5pT2_ z)frtRpoBnyfFUi|#fWuYnw_vGOm&rtS`r^s+o}JIJ{FKGyQai8=2{$4+fOx7%!Az5 z@QViuGJvy}Q$1SYq%q2&##^3Mru`RH{qUzZM0GnxJ7>~FWu`M3)_QVHg3~RZ;4IXEd_FRjBTd|w| z#Z<=aNn54F>j}lcGkTf^YDk#u-)oU$pIYz1>mHe(MTb4IKBHCs>F=3By2H#nvT10$ zu20%ND=W!T-$NUK+XHYn&gQY-m>3~=Z;EHK7#v6v{a{xQ0(;}5%A`g+T1iXXBx@qY z)7K7Xd{CIZ*^6*goL1Omlq+e|a zj#e`pP`EjFcO$tn8IJ0A{Vwof=FFT`D2H|fa^C&?YSSg~%$SCbt<`gqMye&OXv!^j z5Y>3BpMg%}?-3Pv@1M_ANTIOa?7rr2<%V60Wiwol7IDPbTCWu5Ttd4^EFM}jI>K6C zu5D8R3gG6If;*R6!U{waAyzSuW^`FK`3(TZ@5b9!Zr)xLDBPDWRN2WRnc zJMzcUx$esoNusx?5!eX6;T#S<^L(F3Usw#rD1wD6_Db7XPkus=6+chhIZ?%1ndU&g z(|vgvop2xxUqzk3$B@C_jL%fpOR@fJt;e5}>K^LtfNxz*=T)~dozs<7>w2z|7CMU8 z7u-+`5f?a>-J$50puYccSN)VoQ#v(M(qJZBlKkvHSbFupM2<@!p?Z*EIwZFWH!7rt zMu3~gr*+(2QdW8zqlnGKZRJ{>!}++J8gCjZ>0--Z=eXFsIZns?U-d<|8y5 z;B&!-Tzh>cJR|kg50xhgU(01A@@@>-2CApaABib=QLXBV1hoAcYXQ4HS_l<{@4jzFNO+EAW!rsMY)-PFHH$b&(C)8XYeRX3^V{fdA(O z6OVHe?-9@5DmT(hf=YPC_}D$kfC3mOPFJ?G7tEzb__vKbs_!`jY6cHKY6cSI^$uIh z>-xYdL4*Tn>xiiz1-~##eh0=OdE{io3yK^QOAaDFZxjan*Sz&?Zzhha5qkH5SAvB3 zFJOgh((GwtXBuMZfFoS&wE%3#qaTVzzDJ)}+;2KnjQzB1I9H`l;aFXk$ zzisI$MauVPKXY>Q{K{D`@RT3yCLL}jI@ZEOjC2uF!dL;AV8uOm@E-%gg9@1b54a7CCQ|dGY!ok{^TVMzF6jJ@%JB zy=&+H6T!w1(4{lCs>Vl`uRht!UbE4{i>@q_a)jAAS3k4#UzhZ`=jRF?Kt|L6VJ5Dg z_|#`ch!zPh_3y3GhsMqtuA2jP0Ro*dphJfH3-PD}6)|ehnJl1}t^H};=&~kk?MA~p ze+we|du=c`=MD}S+rdJK_W#(U*tytu=dMbHOtU%*t-k&uk2qDV8O_8$guT_kt!ru( z4U+5N(-2&3`c=};0)ri+U|AT&9$p9kw_dt#;rxa$h5-?5t#^rw$IwBxAW{YVCRrTL zFZvoo8?9t^R{sZj=0j1xNNuDH7!P=fR!8#Ajiq0xRi9#tZ_Ja@Ax09*DWO!Rgz^a~ zF9Gp{M6ow2};tr zJ{`sH%7jsCmliciWy?B9N}>KGN26J$Id+pDFRtFQahb`>6FpgdY}4HFJrSQN^B}f< z@4edlHEP~Zj-n71@ep%%Sf+*Yh>j`EE$Hq*=!KpZh3o8*=HmmofplsYI7?`pXm6>f ztJ~IMMD+GxbOO67Y|^&iOiZ|CGG9l^C;a|B2ra+%xRp$U2Ew&4On_8Xk4fuePS4)( z*y*9C(*3vK3iTXmQ6PsQxGz3~T-ERjV!8WT%K13g8uIso} zm{!c)*MvdNqsoCw4DBAbY!V!71hW?0h_p)u`j48 zMF)Ms70wEKB4a8owP^THcd%>+$`oXTieAd-q{<1#($!Ks&`4*x46`bcCCh zQet?~Gr(drZwnp%nqO%TJpm6X&?LW{ia{8_v{1ZDcZ&HhQY!5cNe_HZ_4zO;Qy?kBUk?mqW{1DoQVpQmLrG!j+0__ zUj=6zY6IJs$;-v~<}R@4$UX71b#6ibYVeisnN@Dj4@8Yn7aq~+&F5^3HPeeDBui3~ z=knSaBvuz18so#r?jwkcFg>(iJA*0HOrZBW#h8^Wt|v`PPuKikQ@M4TjQ28LND2Ff z@UO>6XrNPmPRf6`xMiu+pua5FHpgswNxf28q?MIM2h*aH`qwf;Z*B(bnG{@fw9G?l z8*p+Y4anMD`0{z-0b3p8wJ-&t=Mv5Y1OE?Bw>xc~hr`3%n3P449PfuV<@mW^CEBcn zn;~NFJ@3pv4q39+rOiU`E8OG?5~o8jNvkYuc8YNjv;-y4)uq z^&^;$a5-Vve8{p+=(*;{(~maz<8EQ-*+CamLm1*Cfj83d<>{-T2h_eiy=8PEZw7D5 z)P05M;Qt*cEm^(me0g=X8P5q898RmV&6M?eeAc|td~S8dNs|9NWZEsMd&Z0Z1Hsrb z?Gk0BJQ@Vv6MTp_wJVB=@U{(j75ku|SYx=U-`Q6pYHxc-~|`Lxk%t|2;l3$Pop z=PHqj^A||(vm;LNZW2QMjIFF6!o6iZA5%fWET8w`#CUl8Ij7gX77V z6~U+p#9HJ+7p^Bk(^W>kF82C4226Pe{Cy~`OD!}~@j{0(`AxTD#DO-lrf*xByo-z@ z1dkxQ;gq>$#S5RHg0~~c%xc~6cuDXYREL9Rb);Nh#tAD%ib!Jp7A?$l^T~sCK~|1g z!L{P3!7NqUEScWhLbns7$7UzpFN+Znga4evIsSiGB~oKpmB zOE(z!QV7A-Q~%i7spzQ!Qt`4dovDA4_nKtIo!0@>A$-rv!rjV4HxcVc1GH+~=PclM z3%oZ_H0wzm^=ht4XlkoQ2cH?9b@oR5$dxrC=PXQP5kz6 zv-B*u4>g5cg>xd$c*`EvB*UxVH^IM9%SiQ`keJk~zhrkyWrZq>uERpU(Tj{8Pm=CF z?1o^~vB(Jso$V|<%%!n(6}5?;0_Ec^of#384AwYWI~)oR-9_<}U&x48R$+CC49O%G z@-Mgwrfi7iym662d!N)(3LQW{cr)2>Q&yK zl=tu=rAqfv>{`}y&tmXqvct$NVz&8=d1H{4_*yjc-qA z2I$^}Wkeego96qq=eKiS19UpK;8%MrFHR)M|AW4BH``5duu2WSjoI1#vy7$to9c~E zT42w1qg&@W47t<+WB6<)AkT?(^3j3OgjHT#U(^y94q!@jt)5MuI@jMlM=klRY~C{e z3>Y-u%YZV#i|NY~x9{1>=tmM_i>5wg!c-~x>m&Mj7C9gptP#EAD9&|}=>VgJKXcVl z1C9k^ZwTNPT>}KdGJxF-aDuLTV#{C+gDqm=woXEe0v+n} zSsx?!$KqNF#Ii6abVG>GqJnwE=&{mm6NmxD6Y0QLU8V4kir^XEQ9aUY@{puWFWc~~ zJBc6vO7Zs;fGJ}znRMUHW8N{x1o0;U-h**Q%dBNo9KC}cA{8o;EC^ri$z&sVwF_S z4fkdV|IL1~4i!Az%ly^9?Tq9L6Ra1+;{}ftX%GHKP>?7q4-zUJ>%2x$HU5LifuFgb z#9Ss@g*5PxRcbiW`J`2QaUNmG!o5#3i>U?K50M%#jHd2g?{O9ZflQwber*c7^}bfj zv$5u5s$^rgS%SFiU!&6tlVJ1P&u7PcYXFwFs6%4hO8gZ`oKQPA1M7A3D?f385%(U! zdW}}`I- zdo3j4?i{VWlzK)H)qZ4+Lv@~15Jo!YPGHF4rE9YnV2_#BCRG`~!p?!Q7!c>>Zp*M1 zpce(hGbWFQi#J!{JfcMbruIbieaYl}EG_#U-K$g1%JDd|gjg%pmFB|-e$o0@!@iQj zcl%JGbu9$yTS54k&ODGs)~N49&{_;E#)%CY>z;xBChz02fV=5ocU zI5i?Q7`Q0Lr*RTV&y3GHLY9Gl^~wVf{4cV^ztDpD|7vd*#eyX8UpKQ{0|q?Y z$HNpU@xuBO*ghq5lb6pe@*VfsN;rSC9Gk7WeE_p^9*Rkrr044A+#0=L;|TdR1W{*Y zMoYdzJQn_WSQf3%0-FDDza7>%`mGUi?T($;`@I7=&K@RuFuNw=-rKd3;z7lrO%%zy zhRCqB%Vh>4sxF4TCY+8awp0x$4%!+;8NV`<%X@X$eq+j0G-~cLt^2@G*ZWvr(!+ZE zkAXWj3P;9tB~zG|qHFVnGg{=2lT*4w(P!9V8aGe&xo4n;CG6d#73Mz zKg=XmP!;-Cy3KrucMWcMmu_GNu!poHmmR z~8d z#$ahMaSa z#eTA`y8}sOH~_z*2r7Gc3ep;(kh>J-{y(U7xPkZs4I(4Y{e59s0QoGqd2XikJ`o$BVdyWxu4>M=r%MVtG>wa%O0cos+Aw!f=};XLVT=XIMM?2C z+g~ZCCb&D2 zTSOw5_KGm>o`L<8u#m9vbz>9n%YsKSz%!rjpU1gBQh*j^>X1!IerO7)F#4>S{HMYKah4PP71@skZ=RXL1!tIUI|!6OFQo~%aNiC!33w$)he&iegF zg(dEJv3nnK4}@9&pc6(R+$jSeV)gt_Zx1;excRo{|IzgA@l5^k|J{+K5^|j=%C(YQ zGnY^#ibyVtluD6HA+uF-zvZ6mDkRs1klU8ruXD*QmtmOuZZ5l=?R@w7{(j#-w>|dQ z9_MlP+Vl0iy*KM1#^27|d$LMANZfw886w>VWhUE)_tSk$_HUc{ zQQhj=tti-5kLJD+lQg+ckrqgKcq>l4_Yf$6EJt-(#qpNrb5#+eu)0vPO5F_hvj&^% zG22qucTL+ccPsfmTVs9(+Xj`%%=}tMb_L7VpxlUdambuP%&!66_4M1Tp^veo5U?5~ zD}Wo5GeHw7wbSj#d`BI2fvLi)i)Ne|T3+uOq2)h<{~>1td%vFqxyD_B(J*6ofV=%J zpi)2)5J|^Hsgb+L;shjjL+1ElJ^|ljkxC+0W=?mWwsX@ggB|-7*qdu>{Pja0-1c+w znY^(BusZqJ2{CsJ2r>{yJ%vo1OdlJcfdY=sF9!Gy-qdhEp|tW?kAtbr%l}}lZ%I%! zyp;GY#`i!-xzMNS?^lkOe9749gO-@9#|EdSg@s-|EOThD$f-!&uixm3JMOv|?Zfz{ z-8-dZ?aTeWen*X#f?u}m6wN=AN;;{3_xM89p||Uw)8OVWTKi9Ju=htHqMUDBF)>}! zj@Dol+|YiaDb`AK5jmeG0^w7Tnxh|kf`l8tGI)}P@0QEr%?`iHNW5|9X;S4igy{Lv zMf($kN2I_n*HiZ;ppLH4yVDNtyY2k8gWS1l$@R7e@kh8DydT9Xal$HN!SnOCzS^h^ zd<{bG|1@Q1R9n2Sede(sG+OX^7j(`e=f-OO{pW#&%i`c=e)uH~pfl=$TXGdm&XDR2 z-d}5lzMS(2PjEUD2C@EZ*{{IaxLPxt+^1LyV@%2pwn88M#VJSnR6Mt1Xp-M$tP$%u z-m3}UPrjWRLdlpRhOTGR1XVM(F8+lCrYK#>{FIiW<;2_LNVihsdOkwFayMh>><%jg z%7!4f17L`aIL&GJ*0N>o)awla%_im_D7#&av9?=3*7V=Z=Z=EQTpu=fHq_nK;z$U= zv-jmB&OvEqr>-3V#hg!Z{H`x^CdsHV02F)!1h;qSzeAS`&5LE8!N_thojR){|K$W@ z3++4xA%5t29)7?5Nq7+u6os#Lms}{1)|qRxeinH-<;=s7pT_;;r>?~bOvJbf->ANT zk7QvEpIQ9fpb>dSpyb>F+TrEq@ZJMX3mShf(FrN|)!yB%^;n;rM&ZM+&Xi!I>&>2T z+DI5IvVx+~1e=RvI{MA`jffiXww?Ho*J`eJ29YNUL&6dKj2G9-L|(q@b2Qj9&=Q;f zY8~AuYwhMU<$b>pcu55jF@>mH^X#AF> zvG3G{efW0h_|1SB=>9UQ8uWM7o=0)rr2PhVeLSBMQHP2&F-i{v&5RDQ&q`V-uE$ymL?E93W={_A?BR9WL84%Y976ZF$Tq0O>b+>1NXO zqDLup2*vOitzW!U?1BQl;FS<`-ncy){O-G=@S-d0njL#`L5*YK;lZ|e9|IX&{bRN# z&G7mj>>AMsByuyEV2`OWUv-(9b|caA+!flpRQfRV#BvFx05XP|u7#diWFw-mY=(Em`H zGJpfOteE)<@F_Uw-(N1<5@N~?!9+|0ET{{meGI!jzvhiuq|ZbJ<7v_fLF2|)0A%(C zPurk>pHJ0>+VtdPA~OguPG+qWy?@sQ=-+jwVtOcYo5;-&=;lKRps7z=K`v4fiWs}d zMrt)WXtF?a+INl8loOmA)J~A*%o4AzsgwMct`c`NmSA)kT~K&S5DcQ@+QliL2c?aS zin57}3J0H;Y9y!~(II`KYsR}ATvWaA%jM&}ix&H|P}h_;oslw16(>aR@eFGGHM|_# zcXsFLUZV2rd-B@dQ9qqtsEjmR1Qg$xt3~>1mz?mdG!F>3ZHYFb_OJu{Kkq(RnZOmdC%MjH7g1E zgjsOyoF<<)$ZEB}Oi2t^FR1#`of*?CsCkX)qJ68#I-Tn)Gk<~hJaDYZuk~!jeT^4% zeV#;Uw*$;^C~>Ymp+~4&jhuV2l9wFN$x=NUM3jpW<85rk_uOf7y(FwJyLM*9F%ZXY zg+ga;1A#aA9`v6Tth|FhZ+?ms1oP-pJD7+fD!1BW&r5$(CS^QJOS;~!whzZjUz??k zM$J}{hskB*Lk_&3- ze@jXldrh1t=H=>p=Q!KTMfZyl&uClQrAX&=Lxg8)#D<74{{XH4V&K_FFpNSVcR!E>>|Q<)Z??8_cCup$29Wp82-fflw&wq zh-Daz{I6E#(4k&(WPZFrLj#lml z7&RDF%~_T3qDR13XM0JR1<3L;6bA@qjEu2!kknz(#!X8ZorljB?(2i#1d_P{Zda)x z+HGoXE5pf1G=Ukh}Ueqe5R`>6zT;%L205@drMs9#HOjkv8|fnztq5 z?p<@KpwmWl+qbUuR?pnuJv!LkbAhk<+@l|bIHTUsTY#A{0B-(k4=WW2L>s3=Yj0Xhzj;h`Y*#krk#wKC2DGLbcz2>d{ygtw za}-`us4>#RSEWhz*>x`CXjg-{JrN;&G1sDSLx*b>t|4y8?1kGo{zAM3{5YqF z&kBPhqs*}^&m}gKe-;Ok=&VQL!&|m%HA&KeW=|RA@zs2Z-jxhHuAYr@#OW{Go zJA!9GIXnj*2Fgs3&}>}#X^YA1AW6U6-7)Axad6`Up<{Put3SZ`)rdf?bVI1-NOmwP z_$uRKg3d0&!$(c@5p-aO6&{0Um>@wOG2ZlriQ2sP-KPp^8=p{%h(Me zVVPy5K+@pDM%zKsnN2^OI*R$r?CR{4AJBH&oh&k+7t3bVn9TA;C@Yb@_n+r_C zc-KLCa;;QT3+uW%RHwy~BY3_r8;O5OX~<>&5p@n8E{lbp=RYGH=|o0j$U}wD6@qsI zJKokoeo3YW|H91M!=)uTn*s}uyv+q9blj7k|9lEl^Br^0V8OZ;#r;c)oFP;FWeR%m z;bZ=s?|>T5{SNJ!)Tg(!mP*8o&^k1O?6uyKh0YIo(BR0nqg5w@_I@$W6*%A_ioK?w z4O)t-E1eSXA+^r9+Fyxna#!Y^J1QfZT&gqt@k_N)qQkGLm$yByFu`ACQD2?oWiGrl z%zSz$pqbR-V;9wP(B3QY736!ewk6ug6{r(YyIW)Y;iJ33zD>zUzjZ0q@~rF8YjC|k zgOsON?`<&>2ZZ2fE(-X>whIUCN5wyw{zF%5kl*y8|sq^7eMhLsjSv3%L6koQqDp?4Q%V zlRZ$=Ck1o(c)5-<&;fFu!E3w-<553w^o8WPq#L)vBBUJoYj|?rmL>9g_9aWs8!(DL)<9)j==|OIBz&+=V?+C*=I>Rr@UNf*5M0=M1n*U& zzwv!3P&HjXGGSj2_Rw8{gwSV`L;S^JkNPgI*RJ61gWXmWq;w=M0RzN&SL;sQEK_<} z^!16TQjO0jxtc2AWR(jU;Y{C*5O zfr$j1P9F$itAT%8q+gZ?U0F(Z%iS47A$?dOgE&x8NooZ97oYzlSxR=izp*1@J%Syc zEl6VQ;#9FzAVwNn9enR4d7d4P8N|?nRLWd{(`d|)nnJ#Lo_-XjKo7m5}Ei+z7^vtpOA zB`Ph*(cNYc@}Jl|*1=^{)l+IHw9r$aI828gY^;aM@D$td^liKS7(vNLLx1wUJ)>Z( zX$tq0je_Iv*5n%dr-8iTecLoSXLb3oRG3&)yN3V@pfg|&wf0QEc zJvx?|G-cG4J|h1wrPQC-iH`a0Dg}+T=C)snWdVmj1CC_AeS7X2L>V0u*%t{24BWW2 z_C-l$){V>;Ee9TkQhUaeAd~UeMKqcBxNLXU7fK7);@c{?tpvTC9S=B&ew?oZqQyCk zFqe`{ij_{ZwcvCwr36W&ib5SJPbaxEonaLU*^WT6C?v(B=RSJybb(GB0r>sOFHq_O zABUV34@xM<~rbCLDq`LTuGY{Lfj`%mFo;(;0I8tAzz?b+*+s< zARq%Q0|xGC`?0?Nz-u~F`?v0T`o`T_EEDDXh(Bc8kOG!^2bygE$&@tu8$AXCj%vi6 ztS)G;ZNMSa8j9av%>3LsIO1K(>9ZYgwf^08#H%qZOaCL+3Kqcz zf;Y;{Hr8ITnpQK{)R@BoE2!@u9M^a6dSmERC3MbO;V7nxW&sisF8HO`^7jChKhXRx zwD!CJNsF|Um$kutmYQi{^I&fW_!M#axTEXar<{a&#P4;@jqjs@o#Q6Pmwx#8783De zjT_u4AZ&%!(ug{Ak>ParM^*4fpXNhsDSDvvO5Z1(GULiOhJ6f0UG7;QX(^jlwF3Fy z+0M^4tRU`;&?m^C=uS-TNaJ13NHb?rlFNobtXTzL8Fcjo*yek-DnV_-mySCUA;)PF zqNs%;8r?jSdTTUz+}&Ii{gQlKzHjpoGo3t$?wLihi`gNPlT`$+>4h6&K3kG`er=ai z#GT6Z+q<8s2QkMcR(&z^3j-=67rY6_$!>4$vQ8)1eCFTGr#<{;yDB%D@ud;{PAj>W zuh>tQujG)^_QBG+4paB|iQ*$g%B*!pSjgkoEwC1LUi7&Kc{)erHGzxcpM+>kC6Z z2z>8u?(1xcW0#(2o(Hp$+uy5LS?F`} zd*m6Dm}tmP!;q{lJXwX$!;_h;TDhVB%YB6FvKmBjYp?3Jg!t(8K*y#0{p=$wCq8h{ z>U0(d&9!#lXF70Q7&QKdns+LFBbcvX8`gL=lao^RIs0F_)RHW0^F&4CH$w>v#MBmZ z^N8xyxy$RjKR41*yzedr8#kpG3HrK3tyDP0?+u%;Uw`77c&^XEBbUL&;K4aTR)3Ao zufQiaPh)rPGeN@&Jd87)ZX&tuSr9x~(cq*4Q+D5o%`W(^OHpY2T8m=@S!CE@FS zix>6ZDuiU65)2;LA9(IO&VN&n4*2;AtV!&!=B|SJ99Kd8b#~fw&mC5WsLOkibxjZh zSWfLMI{p6k{xRIZQY~orkv+`s>dwI_JJy>;N7ZsbX>mrIL63ksnPf4bkXDAMsB&Ls z=4kK0(!gI`U64~)`R$Z#JLKkMMT!|Bm+m9jlF^Kj_T>aQT`G3BkQYmp2Lz5vHgNs_ zx-{?8W}*iW-46H1P+?%G&QVZ|>q4OARKtwK?H#I3&E7V+ayBT_TvLMEJS???oYS1+ zGj%&g#Z1x8?`E=*TVJc>kF&C`|6Dhe=hu9EU=IWLP`N*|PFdD3N?XAoU0(03_Y}t(<0@$ZAAi}LWz4&3jJA5Gz?Qe_uj$7W5K&#xoLXYPgUSzh3y z42QRTeCeCO^OT$(a^l_WxPrbem@`y0^_N1o722b)kSN^F@mPL|7 zCl-i-5QcH>v%Rrv*ZmM&?pME3sTkR8c8X|S{Q83sM1{PPt8D%|yjFR#>@4^i&J{Tw z=BggJN!eW^Dq3}UTRJ1J7gyNA8HH@l_~C$xTR=J|9~ttXaMwE>If~WiwP8ky{LZ+) zXsM$*i40h|)g#EwDc?kQ6 zy(c&QYG8RD{}mYnHm3YsK~mJ$cegfn$%%3u34T0A{>Sq^`;Mjm_*=%x2us`?ffH-Vqw=h zLKUDpi3!Ms4%vHz4moLa+biyAo%7I#!_yD2Bn(xt67o&mjh^}#=Y)g3dVzaqlCOB?D}&!uWDn@#^<5QFH*FUx{ z$UB^S@6l@SWmEBq@3ChPQenjuTu7veyVRbqu9`+LMHTq7N8c7?GwAQLIxpQ;5`zoB zw|su0B}tHWz8;3*!8f@4I+C9~xb=H2QzjdYwYd3Pmdv+FH@(Rf(XszlWE1H|lR&9; z??Ql>`#g^?P4eSaJz_-rkf$G9Jo$!n;+|KFd__YcK_}YNZUGDBQudpthZf&tiE^Fi zU8qS1|B)t2mc7<>(MxOkGN={02>cW)0_m*cb#<=|=au(zR_YT57; zI+ML$RCqv=s<+(em(KLl{72%?W!s~+#?2HJ`#Q}#LPSS%+P{ou!W zgxk6J&oE{w?)Qn?qYOIA^@6|WmVX%E4<`bS=PyEyz?bQ-$6ou<&=5T1u*;4|Gu<9( zRrt+>kXXG~>R&kzX;&a+JeHf2gO(EUI7VyX^M71M3D}vzdJ@2>>4zN&*d3MhC!1@8 z(CBW#Qp}vujw2E8>HUi_6aZVgFgNfuEs<7~fc#tMfj0}7<^CKBEaTnc1)@bAa+=6e zq(1-)qg010m_YOB-WzCw(eJ|YNfp8Kja4|#78Xbemew|AC!W-f>TBD%+9ttmv-xke zjm;}W>H{9y*MXr9t~~+~iQ_hS@X;c3PK|G8o?4B1S=BlF1Alc*3vufpr7RoPRf4{r zPs)?K$s0Hs=98pxkgD#S_->_Du(E+T4!R$?q@IDlHvhX$((u`fUbR^z8^s4k#e0%3 z$=)8#(mL_u&Yo7@JlZ{{~}o?_x<{dM0^l)K^5&tJdwSqH2%ka zA7%G--EM0@_{aSf_eS=j98X667HYqfa=$L@5w{U)(yPsjTBZ2=lY5`t^Qs`88=DRW zOD*sJU8)hr_#c}^6a0h>{(POs_n+g;MKos7i`qoVcCb4dU*6T;8)%F|b|>E2-ti_A z+Jjgz1^!<5l*sZ@@1xK4L?aKKRDQ+}mLW^De*btLO5TZvJpE|+q~MvT=7VUVHeTqZ zwWcdS{xae}Pe;G2Ey4E3ori2{ z0^E;_NFBibMIIe{`aKeU8Ct~88No!MhQJ}KP#qH}M(Bh7uT}3IEDj%+HC}4|_G|aV~p4c7jzV|U*|K(0FLs*8#Ad_L#Tl*-pJ!^qZanCq;Nkv`f|NZ=B%6e|V6byFZ>k6*a4| z@H44-1rq(2xK!(}J3lI639qGTkrC7$4f<^fnzw2I;*$Q89cuLzL_8z?ogW=XkN8h~ z=2H7niu0=q7;ej1ge}n1S-yJ{_zIP>FK;{_zEcWKL2VqEU6?dwi_}*?{jy|v)x>Fi zALXJ*{3XM>5=*PAHd=mZsZ@{1)FV!_Cyeq>cw@<4dk?x*Wuu|jFCacys~1EW_^hjJ z$T=UlAK7~(<$6}7ve>DWxskN(}H#4Ior7EWwPf^^evL>kJp!*svX7H!!!i8f!$u{T>`>2uXT7Rf7L1NgALOsI%=OHfPpL|inEZQ0AL8?7vH2u$u=eY z%Yl!Ci6YncLm&EaLKY=LNT%{~RjIzFp(7Ku*tNY{#SzC4dYI)^StaA1 zu4EMJr-GrFL?J|+hMqs?dNYCV#;s2^XTy~WSMd`-FpJWZ<)y1=?Vs0;3dXZ^7AucRfPA0XOXdj4>{}~8OWYBi`_@6*JCAj<0(y#v9W*# z;lf@memeGB>`8@r-?9%cG9j;dU;Xy?l`V0!RIiu4c_CXH5^PQ5<9XhCP5uIS>~yVm zoZjDs>D2a4#ZxoUPs&|eH!EwLYVB@!O7MOXIetTQaCuthSOk9Yh|*)fBsutnKfCfG zEp6KRe4T}b78#9I@2vIljzoo#CV749>hOIsN6CjU0ma7<`{tD-Gl8GKnPr2^GuIE+ z!Jiw|-v6D1m#HHN6`P7gYmCY3`8=;n1;T%wP{qXAA!bW&!pTm|r&KwIF00XF_45%^ zXY2$ec;xstcCl2HdCe2&AYbGiG}ShZW1fyoOZO;?x!^T-kG zPt}WS`Or1y%6%Tya-WmngZ-I}ggkQ4r=qaxd^6~9k0-kXcx_2{)walWzIcK+Wp?rt zAMzUtqhr{vfq>b*{p?=^{KyMhUipi4a+na6suRF3UjDvS9aZzN9L(kRJFS*Kh z-q?RA=7FALnflgqpQkw+J0|9}_{juRAb59d%6Dy}!9()RZ*31$aQ{RBsEJ2(cynfy zW|-x>GYHm(-UhYT3el|qU8%_03GONDaAA$|t071%blqmVr`NAs1&?2e3#SG3Kp#Qm za<)rB1!h+z zaoxZ%7*T}s)y<@695FMk ziwW3T-Z&$>7i#uqIW}19%syYY7&c}_iiGHMRIOt&CCVE)hfM#lP$4_}`*m5#DBzb> z;FjFc8vMf!QXrT*hUXe0HWOO}>+usrl;7(KQd#XZ#SuyF@D3&hCrIn=e5T<-mWFeI zp9=alOJ?omZ2;?=#SHCVjB3+7bB!w=%8Q$C2iChFO|94S zTn4y)F@DkkA2Lw4tGkQXeTC@a^23xIewW?)S`7as^wqUo4{By=p+0vBL@bA1$kx>W zU68as(Iy&hhFM4oA}-oz`y|K-Y7w)o;D8)zhc5w2>l7jkGQD|zY zqA7SICjFqhHkw54A~$b~C` z%w*IrJPx@cxFFE%s;Iw=f-f^)}b}+th36{Gxry<#~dA+U+5mV7b zQ1H@yBWWvBx4RMB0*4X-&!!wO3Q{#k)wbBw1ziG$Es^y4n3>>_)Kk;jn$jjp(z}s7 z+-+MhWW7UwVLf=7_Li|*Suc6A87>?OQ}SKUhSS~-@z zBc$Aw>&I2(=!Uz6d7}T#X*_bHT64y~=b|+0Dm;=0FYP}wF{d`YxWx@H%t1>w3V-4= zke2}B6(qNuU`Z`}1*7^FT5|j;}#;@^w&$-vdpXhihIeyI0MJMxAyg; z4l27@);CTu=hn^@g=zEQ--ccl70hM_1jM_4z4ZQoj9NS4v(uuCsXQSlKhNOW_C6uW zC%y0aOk@i_oP#9G8I%W~+Nbk{YNAOcn`jF06_)y5xS^2bUwlY;C_QPmT>h7`%w)Gf zPp6s2BYk3>)^M8@utPTf zN^OyWt*$+|AXR^KuELge!K@`B;X~zuD(979vFM8Aqq$3#n%8kfIRKCgzm&}EBY$M3A%U<0S7O~ zDYNr$4T-Wn3Dl#`TMDYd1XS)7)#3e>f{Q$NKN3?BuUU5xk$tkRE5ul&stNLY_;bag zp#b>7+IMk~=kL63BdQaj!nOuq`oE%Da24ruE{ITn<2T*Eo}A+434ecEl`AVyOHKooyIrl>~6TQd{gX7yE8? z?!hHHA)D+_JTQVMuVe2Ma4X|#sbD%j#N?FCYfhEi6nh>Aq?GEvOv82|9_9P?B$%Zb zL^YbhL>2y9f%N z{mkW&(W9gA_}Xku*WjUr6ay!L5=4BoeIlMhsM8>gldv5`@VEqxTk$07D!6 zzmY6ma1F(tN7M4^j-I_+lJt)BkzzfQM(u+U)O9D=YuiFq01%p=)N5Oy03$8RQi-KfyOp?)H$ zlZ|63a4!Wec+^8n{+xAMac>I4hd991W>&>;W_!ABuH+!2#o>&qAm&OTauLh^i*DK~ z_AEC+o>LV3L&^>r4MC8zFiUH>G;I;>mz<`h_AJh#8hZ!L-c(Y>GR31c5%0r3>^ALY zz_uAEniP!DCdeH-#YOe?*?5pm0yqg^#7zCi;~}@>b8;f|$$%(w*dDP`rN5?%!=hL> ztb8iuKBQiK4|MaZQA(n>8KUhjA#-Q#;^WQz580iOEjsQ;T|^^${J55@7{)HW;vpKj zru+4nP+Nm83ZNul$F4fp+#BbTBJe>dCj<6o+a%poa0H@(Y==rQ@50Cvcs8Kc!~^|x zhR<;czw7KXqK>&@E-cpd;VG7^l*{Hp2MdMb?vCLH7CZn(7>q$cf&{BatidP#065* zQk)>rh(4m{@ES(7!mjZ{X-{4Ll+THqHrpJWTFaAJ)Tfak*p)vdb{!0ngUzx@O?Bf& zRt1|t28^MZ%Sga%de0DI)um7#H13VXSi=G!=c2QH*Hv&gYAJt7NBQ7YgT(IK1v502B? zfj~0QLtc`}rE~<&dLdgijy@J8wMZ zmRjlMG^Zn@_xEYVz7-E#dlO|PTSv1~#OY%ErVi=v36D&BA~{<$k`fCqb%l4N?Z?_%?LY>D)(&LaC`g{z+{vS`L<;~pWn@-U)Ou53q$tQ=21P2U=ELa-t3U$uo0W2pViv0D|- z7~a4&?jN;3+h16b=|EK@hh=ilo~R1rdT%CfIo7Ot&>d#g=xA=biT|l8w&^6)X9ioo zFD>N7ERKQZOfPWHk!VDE0KYQZM22}Ow@vPdFd+p)2oR`W^OAjkm)2`i6N3+#@OCD5 zz^FOUn?D94Ua7qCAWEgfZZjDup91oD~ATn&3L>U2;&lX#8U|5hZ5$4d%^z8re}7>|Fx0c;2SvWI4H|I zWOojq^+paPLfPnSza57^ zxFyjKYK)?D49DqC(JI&k7tc3v+8zHTOrNBlMZRX}ss@AVU_X@k4i(guS~|#k=6b9_ zWg;*MPT-Jh3E=uhNBmu@QTmP<8hc^nXZ@Z2G}+8%uXi6H@vxonm^Yxv*2oxaw~@q1 z6y?xyFvNECLgu*njSsvp(w#%N22t)b1ouOI*p`|v{;4UMm-^c(m2NKdw~^}Cu`c9d zB4SJsW^65fW&wSK`&4ut!X;=r4z@E5z1g7JuzJ6{{`SK)CGCd|K(Qxg&IHO3P-F6O zn6}*|Iy#NB#aYDUi#E~=UC8^P&0{NFl~=fo)|j1jZe2hKCuB)MKkDS# zp(&jd!XIb|PQ@%ibt%BhyKZwSt3>3kBHMAtv*J0&E@l^3+K2US+>#2IYR=NHwA^7$ zptjEVKh=kFx3LNmI|Bx+;i>ye{ZAqYMKteJ-S5K*><&ErCw5!V1st^)TS})3ZmEI4 z#MZ+dy>hy5lQz$i5O{A+vl4cp;S&Ah1J8#%y1G-IdDaU{O+L3Xt&eL?JRVi~)(6(D zQ%g`_2-Igpquk%Xb_{0$u3vI5sZIM^&>z()1TBEN%Y=oU{TV+U5gBoBDLwKg(}xSSLW;z_jirvJ`tl&EMV5!pC!eJZ;OH{;13twLBi zdTxa5I5aI8gOq7wD`0#Zcye^f+E$5Z7eG%W04)yC2;9tZ`^Y!-M zLB@^e3{(fd0?Q^83iw&>lze^tUW%ZHF2N&px(9jJe3e@%KI+eXs5ZPruTe*A`oV=O zl4fDOR#Ep7`L^;UQJt_gAoJW-dLPn{{pDVmjEm3+Uu&z*54-y( z44J>k_?Y`N-Z3iafwtet-!_k0JI-IS8qX~1T&jAj;B>I?K%H87T1%LvOy2O{PsU1u zE^R9Nl@kM>zF)hU&!gu1hN06hP7&Lrcw`>ec=XyNJ7zc|xKSK7zuzd&r8A9xrY_SX z^qq&0^`AE1ohOMrhIBDF%fK;hUl0GKTl8#&ftV}s?$(!r#};wAa_8EolaA^(p7QPx zLiGoYg)Kj)t}L6G2i``I9FDarycov1nj@A)aeEf!`EAOQS7#Ll zSjku@oavQ(Nb?)9JK;&^cm&XeeYOL6+euRUdrfmiNJw2LnOgGJ{ii6%p&N81d@b4g z>NXKdscYL04~&i1q?}dTiU}Wa?m|+NsV#zGif|sje+rJeV6?+(y2Ql?cR$?0K81E+ z(+t5glMPmR1as?gt7-kVK}2KF*KpL1;Qmy^Fk(o*sj#K!B^CYsXQA-ckT?7|FQ(e6 zb>PGP`afv874VnajY-6|Ax@#(1mRR zn1@mFm3BL|2dejb!2H|Fz289n+7vk_DTYnvrp;D)YfHf--MM6s@;C@^<3nrhsd*E` z96MUVdz#&+0B#mbJipvKuY&}~5CY8vWmnd!s1CP7JH zGeNn2t@-l)1W|?LVfw%@ZNpD>N%})&%f=4K#rp3a$)u^5N&yN8%qYXPjOghB)~Im5 zXn0mY@NgFq_Ua)Ozo4+rV^Dwh^7NDJ5K3wTW@RA)0=wVlaR8CQ2-t~zp(%Osi2Yal z%v+6Q`mO|NcIl|N=hCi{7pBbC`;*-WJPtKU3XaUGxrzAnBpy;lLdmNg4w%AWlRE&GkRFTXT zUC2GzOYf^HgalkFXP1`fWxd#R{H%TU6Rb2!!Rty|=y~i6-<=ig!b(5AZWES6QGcAq znG}O^L)!~za4!v3Dg7>)oN>`%e?ZLN`-lDp?gqBokMdtCb-d6cG1Epd`0(bj&f}BN zj>pfs)Dg<7qEKy%&bX;8h0LP&_ik+;=-rO|)OJxqg@|-P$tK2j6Rp3*9_)|E5Ay^+2{=i|9nI)}vXKs?WpZug1dqLVhAADxaayagz08R(F$Xbz=8FlOcJU9l!$j!OrZxx~Y-ZJQcGCYWbTvlCW}y24qbdZP5V)yWPQ_}TJ^d-XduW8TeBfFXIeyma^{u;l@UAq=C` zhQo$Ck*i}Saq{{fUBLpxLtC>@OAbG4jLiIr`-WKKnB--ek1M>C0@>I#;u!L&}a0QTfu7~f{)0m6qN_YjXFv{4x9E^ zypO~$GiEdJZDfur3;KH*19N#9 zM_d4o-D0Drhlc^BK)}X74QxGK@p!A~L58NdK9JggE15?&fbG>TO8LO(k-6*QcY){T zB!>m;Vs4uS3L#H1w%oe)H5oY=eK)7Q7T2_pL8KqO-HI#UD-nMxUdVJp86v#cezXT( zp#+uFoQ+|_rCK{Ff4YS`kbu(7hyxxYNh0Z{)TBG$CVlK}nDT}-ukd!_!D zqDz8zsX_Aw!3ye3fe{sAwd*tHWm2JEDggovtRO@{)xo3E7hWbv6;4D~cT;PnRwGCIH4j1sXQ8m-`j zQeEl5af<5#o-tet(N00{IS4&lsD~$Rwk3qIFVjfEnyKP~kGm`5piZ^tHbcF-u*hGC z+f45tj$|;Jq5J767agJHU$^7R7-P2|j3dRMN+7pFl{;>5+o$lxJy8e8u8|?P)JpJeQ?A613j2YIq z$@>XeHn6LXZC%w|?|AqWRlwHLdcFwd<>tq+8+89uk*^U+$VHy=V3peUPgYm+e&AlS z1X(v&f4!E)Wy3nrMp^!eF`ps(2S7}b~}%n?>A`~M-RPUQCRg$dr+Pxf>#Ek6{e zM8oye@rpPdS85=-{ZF{gMLX5vb`^cK^2ffD&MpDfb`S2qBnX(U|76>-`m;1V*HnV? zVePM7w{$I3Lssjm`mw!qYZ_j!?7Pv^uU?tI*0LQ*XjXqQFku9K0e{(ejWiWH@q{HI{FY#OEBw2Rp)4(mdrW+ zvw^4}>QX_@<%xs3NmROh`E;xhQJ|uDFA5v^tqW;1G4@+|J(+Z}^gg=}mJ6x**Nbi4 z72<3wdD2gaXdXULZI{kPOQ8YVX-&3jr>)dM1S5i|$N8&%{^&>&%5eW74m#vDyOXa2 zL)r#CIlUVXnxT&r7r+&h^l4wxJXQwWR7auSrXI0W>ELyaD-ogS@hX>RAVrnu3-3j{ z$Uxazj5xD=@P1CY;pG@9z1E+vfFH9p!Gxa&tYg?$J**%p(5I4&-B?t1?Lw5ouzqsE zdx*pQ7q{*{9hLUU5yIAzi)r% zH4P18VXMbk2si&R_1UkUB8U|p;US4W_!9<4-XL1R`b-zH78(~&K&$N#i`W^W}<=Lg5Y%)b$GjPHVpw2{moQ%ciN9@z~GDk4S-^)cy zu;fUmqNjmVp{jxsMTAHXb|?DpZ9$Ascr6n z957pVa)15yFk(+}mz7x(QmJ0c?U1ThUOI$v_)=O3OBHF2Sd%ASt zJ;;tR>-j5|;TLunzA)t3zIP~~YH|U(Wiz+iBlt&P`SHSn@XQ_TEp^9j$1eON;-|R4 z9@dzQ4Ps;ZxCgxBixvA)x;fQ|V~#a1Lux-)-thRCDad{e)M|osk2V}?4ttJ2>DAdQ zgl{$1my6f2+{HaN;%k|kirbNfL%z_<^4?eeho-L#YwB_TRuqsBL=Z$uKtd3amYk9b z;-p1{(IG9R#3t~iyHR30dr5HD)Ap_%G|fR9Elb*b4}rdq-(*~12L;rNy@V))QL4wz&2u>XPvrm;fODi z?WD)xklbz);)fpgS_aGWRJc@Tc&&SI$Q*2)BE2|HzTBVr0`sfc9OB3joOT*tQnlFG zp#Rt$+kVhPjCn@_zYLUh*^jVQ1tW61>0=Q~Hw@X15B2ToOP?B44~`$e5{3x5Cf z)SC^wXSK#^JH@O zw%t6(QCocPw~u7Q4}NXiMLtwG{g0dGdJ0BJq*7-uU@59}offLEHrT#s=ZSHRpv3;0 z^8bzck>NFjd7#jrpny0Egt&}}CJSJZE^9}laCczY&BjT=Y42Kty~%u)jYAIIG+Mrb zd7k=%ExL3U5Oi(^@`HRZF2gH^oDSU=v)`1&afMfhYZwcNx843kL1y-rK7G&kn<2Y* z7;B1%MP|V4^=wrEHkgf1`k4#moUH@M3I$g18K`Jil3Ns8F;+u_>lDwlDl4oR3T3^0 zXGhTCd;ij$U|bj(7|ANh+H_xr@pSerCJkFQ{sGY!BR9W6g(N)a{yNC+j~EYKg?vL& zTn%MPn{~`~(>`zcI=;^#b0Rz1+E5h_+TRQ)HnSUNceCzuoB7e{3BPI*EUSMjOY)l_ z{MU1Jfq?(uS%lfQl0t4&6efZ`)?buJ82e3@k{^n|qit)Wdi*rl{p6C1e+$`t5vN)7 zr~|zW%qOmu?}{U7VP<$oHi+Grcfh&QthSmXj)=%PWpJ1!WnijbQ<9f;Z=OOr?X2Lp zWY9K;bdTTnyxA!KMTn1}{+rV-V`g-GW`K49&|GGlW`?~&2hShAh@~8&?vAb+=T%|r{=i>!rJWw#_+0tFm+zLzT<6-|NGjOT^Qz`r z(?k85cZWQ2S)p>jev$ajWRW$8Q~~H=XUjFJal&8Gi1uD0JL0OgQZJ|>T z^I-bD(kx%Tr*q<8`hk3No7fgwx0>{K1#-plVU{mw=Q59av_03<3N+8PX!o3-f1*l| z*JbyS8LBw92E4-R`voM&c<0N#M3a6*z=O;6tP5_=hO;j6Pd;eUy_?Fn-LtQ>n$QS6 zU+f@YKtJsgAw=R%R45U;vp+C5h7EtxEg%_~bsyqI0!LxwdyVv$#_ktNN+EYMTi2D- zzv71+X%@d1Z0eWGY_zo?iALm3GTQv)@mt14SPvu3M}%S87W)yF_n+z+AC?A4Ne0ii zP6IPHtR~ro%VwT}@;yQGMYD>s!p#M)Hgpi~BkNQz zyE&Hn-pk|H#P&*hVc(TeID@5E1v=69>O+V%i*6${(RgnB#VSJP4xZ(^Qae}bch@m& zI)2cBVo32@_*g+!a$xzK{iZV%knaqpZZ}pq>fuHnR|;QWR*^ovQK7%R-CsbKPl#rh z*iqtR4YW9x%Pdd#U^hkaxJH%z)R832%9?OU-@*T^8RI{uetEqB+EkNIh$;ezavyNh z{tGsBuFX^GM=!BTy&Qi1XZ3EIi0dQ70!JCbW54mBT`uSakb)*NVz~Qa>37w^Xg@oY z^c?YC7Sm>I9Vh)Qx+gX}FyC2(nKT`rQEgCS!q)RgPu(Ebb))}q*6LpkNTUozn&k;1BpEvyZ`QT6(!`>W0 z!<$kP%b0RV6Jfm9Ysv0Y5)4cj8O3UTC=ONQu3nU59ib3MzE)j{;9=rujM8p;T= zvBF!vMFoO6A4HqZr5BCZ>fM!?crIO?YYL)SY> z;-drs#=rZF&J||D0%4TOVE0$f+bC|Ly(ki2KDmqw|6bkRfIGMfp3eqR0DN);OIt zijBzdb$$3ypV?DWn4o|^m#SdG#pR2C_BQuQjOL!F?kQUICWmK@0oX!kU=5d4%%A@s z$)_)^_zC02bpyMFAGZDkA)Vd&8_I_oroaOo%F4Pwye8%!dY7u5hfmGEmk zv?}<3E4ncyPA=wKnxnEFO`{3jYQx7+7v!z#WHn+j-n+&qa1>PR~ zKjiM~4U&~8l70QMcSl%&^UuBvy4ycTu~lS&2Tpg69%e3=a*FL{H8RqryA{gcJNxmb zdCv|o)Gl}9dHG-SiC_+N>wV;b%cv?F=*v&hT(ws|!a`ziF{K&==_Kq3r4kh3&PI9T zMUzrafeWgbHa_E+yFvN1rywNsK6U`ixn__Yz@c{cJZN1++k+Fd8Sr7`)g6^moAO(b z8=mPPCg4L63eiT3+Xu(%p~&0c6{i7Xwk)kmWJghczut7mglOOd@q)MnxMJn|50v=H zPubw*G~hIbHd_KpqEBoRd=cdTmP}Ft=d%DFfYfvJNL5a+fvW;zfB#=yvYewKi z8b9@A-X;Q?!Ocs^G9Bs%k~(F7IAZ)ZT{n!q)Q*d~L@M2hWf-lJd=qx>Ovu5cLFeaWVyUar>h}`yo}9 z%!~DygZh(bW)9CqXoQ^(Z-E~To7`!-Fcga-U{4nyJ})NEIuvdp#+-BQi-mX{tEI(M zqshEigrV+s!K_{FrWNd61D>6Fjwus;X&551n%jTCkfX|0)M`z1Szk}RqVjeb^K&DI&$fH?entQm1o{U2*bAri9<2K zpS{tuw6S`OQo1ack0XHyC19hH#FPf<8~W^LNihp;Dhp#2NpOVmx>eFVJPOMIKg1Jc7U`Gqh` z0b3^*!13;(-!+m6gBVX3q^!GdjP~33r$e0BDP2;C6#UglE)DwG?+sNW;nH2G<#v?- ztYrwGLMgaebEVm~BQ(Xa*@mt4+G{C-&e0E|9)aYxZvlQABQNa#bj_mn!;PHCpPl&( z3wh~X-tFA_`NBHk=g{aNrjyiQ>4Sb=)ma=N5gn5*;^F3^Rj-^in|bhk!STc&ZKC1t z@UD|O`_OMIZUwm`PRY&RYz?=*v1CGXzlFR)#u|J>Ws< zI2gmPN9H^cDBBLIP^R)LGmQRSlk+TP0#s45&K%3k|2b2y)&VP&%-&qS&V;% z42&E?FqydBf)6J6FK=Cr8AWGSp8Z}6sna6Ip41`F*Gvnm;3XaxN-_F)S-ZupjJk~Y zQUO~B;j)h&^58t7>kHA==(P>IE0le+%4$S8g6Av^cX7OnwPFSnv8n5M@&g6gee%C~ z!8<%4Ey>{QJZRy<0*v#%@2?G7DS;k7BKqJ6T1?{fRC~re1&5t6-RVY9_OX z&fuRB9=oF!CI1>zvBb!a5YUC?yKW5N~u2vk*Tq8SutQZDK zK}N@?kZ2hC3miRo5}hF1FD``bQ`1e-_K7clZ^#_p)FT{JLvzculd?B}!=r~6P zAr}4%7_!y0K5ySWc>s#Lx&C0E7Zlh?HtsVpXB|eHuk?ve1U0!U32YVrVn7!EKKT0L zY@r7>0G{jzA0wxniP2PFFW2PF`eDX+(@dri<1h$DiO})OkS4+6yKMr;kmQx!csbzW z_Q1efqWwNcaOLOy-i}utq34D>QdkOn+;3Fx(gbwp(y|V z#`qF^wWcV)!G_nXCk!YoV8J+U>oLJA!GwR7pNe0j_#geoHe0Aag!v;eNN?XiyHfiQ zZrEQuOS+mv+~?h^cx=n{%$>kr&+eC%_impfU!HX_e^$R11iX&1u5?>*8D8V??+2ty z%YOSDSN-$@fBK*P2z+hvMhseYugt|k_uQC#4z^r!=y!a*nAG<7PTS?-Z%wtss!q?R zpe5~ZS&b}m&n@NgS7_w5mRRJgG_3g7q1QgvzPqC^rF%h?{o#dVb48)ftBB0}R|45x zNay8fY~@fI*)$@-X)A;$_=ue0JQf7EC3kA|S9@+H*m&5AWf684w=#)OO50vcRd1 zwz_#@M|zRHdQR&4IIojo$t9~0JI}Uv+D9>8ut_KZ!t-V?43!`@>29Zy6@k9T!57;q zYXF$$jYn!F?UtuSvFoiv?h{ZUDPKYzvLTP=?MKe7pxIQnL1!9Q3-8Qw1nC&Pz&Iw* zITjKNyRg>kXV~H8`XCqQPUq#OjNcJRHl2XR3p>%bO`oUL-%=@5w#oe^Y_RNWp9EoC zgxc&d{ksoC{FU7)8jnaSo>4&QSVsuqkd{+N7Buq%3-|#>=6w4-7H=lz5>?=#?AKYJ`U~$6jT|gQ4Lph9Q+Wc=J z2>lns^G<_jYsmXbb4t*{Aq?0EFSLhnmzB*qcKvA%)O|hMQA@ef%UT1i&wdMxQ{!j` z!*Lg>ffrmS2Im%?w;I`@z+0$h&;N*`K+iNyh{U;WI@|7Lrw5f7^y^VL%g1E_6s^$A zbxQT(*~vqY84E7OwQs$Cv1qD2ddX3|LDLzn#qD$DETPbTSqUG(G7Gu)j9U42R)u3= z@&w|jde_6%hl&j%Jj!C*l-!KbgA&ozFp$s*Q&niL3jV>#g#f`j__NBlE2KDq$MI*B z`4RK#`0Pkae0R07W_XVWm+XDDd~~vJcSJ|CvTBt#>rt@ZIokdiYffsgS=~{M6(IbX z8r0X|TZ4>b!J17qBy}sQJWvS}ts|jK_{GRroU0=J?4us?BRB=bp0@r3&@`D3QbAi@V- zPS~A*8dv~fs@yML#bIdsO&@q>U%MN&BKE}!0=B$-_Z14K@VcQwrfUgPelhY%U$2g#LjfCv#GYA zYIZ#fo41+7j|-P+&)LkzTm&RzE2wnyAPQV=K`|fm{5*k_hc)b1uAMwSI~W%kBLb@M~4TI`F3PX!8-uIxTwRu~02ecr>+PEnh{`m+H(x%4*{Mf>UQ1 zCYytxXB(y^N2<(#%_%>_GTUg=dp>_M`+Y+)kerUMRQCcIW}#{Tgq)tA%xxOHdszRr zm_SBt7sW_%oe#_6w^t5(A7Uwgi~UohyU>N`<6ryMKg$jY=T+S?!kc1xT;*h+yhe|0 z;4y*hG&I!L7tajq4nH?B;dcx7>`#@ta!o2{vYSTNZ)@segwUOVU^&c$#z$kiG54E>6X&$~KlDkx)^)Vra4vu7Bp-9RNuj#nrt5r1fV_SMd{kv1lUrYa;rBRig~TF?FiJg}wscYz zT02~$N-;t{c5I7`fEbB6fqe5ClieF{WM3f{##C{#HaYq=8E>%>ZR6{_*kZZ8?B=C? zdQ_hB6God_&Ha+oQFI_AlQe=Dbh9GY3jDoFkz=1=1gM!`x*SA0+F33~+9`!lj~p&3 z7t|sKozFK7wz`PFt_ur-ovVx^_mu?#SN%fH&`F$8_`4i`3c*wZxZXIZX|FI70kW*A z2xzG}+fBAQTbfWHjl@EsgaeQ5Q(#VXr;j(r&SOH?1TI>Ex!axiw?hp1n^ zX#2`q`Ycs8WM3N2_S^mh`C|FrO9rE82_+5ZV`w(-L*0(ByIyJ zCTb2N;bQpH6j*J_3pQ@8wU{r^xVN3AENH>zl1S%W7SL`y$uF=7=v-8)(h`pT{TUYO z&fGpLZs)xsAvho2zqCgm2y>MxIVcFlz+$Z-wvM_u+aYMS%J3MeKEiKpO}n-%)F>d+-GKZm*IOiy>#Z6(oYFij>@Ni%Z(?Y9(ULo2qle0xFyG?M3^-e280H0v2>)uBa zhU1gM2k={RM;s<}sngtgO2v1Eba|Zaf+-%zh|^Tf7h_j3lxFOlAWaW3Bib7 z9dx}|s2k(}-l*%kTOR7&34YOTk^y`8)l$$P2)Z7UlYKvN4L9Z7N6T=z9-a_rBdb#n zU*luMrjP$=YXs~-kq1gQ)-N(?8H5CXOk+b>qT8w2-(GokE2y}&o7sUreX;_b3ST}; z_x!w5Zw(4Bxh5}i6LIal`m@a{N-d02gC^cPr(f5&QhR=T_4bUL{W%29hY`H)_z9$( zcScL{fMWfhkP_qs$=a##b|2g0VU`KC#AR>~fkwcR2}P(H0i_82Jk3|mFF!dGByKmV zNGImEoW5If_?(A4!x9Pb=a)DYgU>F*nu92c0RK4KR<1i~`uhdqf}onqa=xbC$o?j;5S$NKrn?gQRJUi?6qa)3 zmXm8JF5o4R|88Dz_qv`MZoG ziVeTRWllu}ltl!Sv86b=p##lcr^RQnfNWzHsmg1u5*QEnoOik%abAm51^G$#(yr8? z+)S6`W90QQ5YHT&?jk)z)5M{-xQ`YzRR6g9dFQ&$D7LA?VH?t?-)*B<*%b$gpIIjBW!6;S%x{BwSpeCSo7|1U}Q2 z&+0;KwyWKy!GG6$-C!3HkVFV(5yAVbpB#2orZHp`_9L+4y#?=BQ*d5*nuRF#+V+Ie zmV2e7wqSJ6!ofOS1xqw*7|-H`OM`T2;cQ}mauT-!=2lHP$Q^ckxGuGvtcG$=barQ} zx@s=b!%SKecnJ3_opW`9{1i*9cd^-GP^ck(L6%SyED9^k_zIkVJuZiiGlNnvhP0vM zl9E49$|VW4rnesrNhE}$Ev{z_{fQ3@2lcJ5SUO20obU)vq<6jx>_GxF@C1-JLFEgo zHMsp9@^LRLU27rK&x7?U9TiH-|Jip%xd%Y;t1R8pHpxkC+hc_Ms4U`eTfbFlL>Vde2*;yU~&r#K&s z6mAb_VS4f(&)xYOHv1x^ZDukByA_a?-g}h3SN-8a4ckv8-O(Q5Z3RQVgW50cI&yI?;7rFVAlr=VnR@0FQ-(7w(>`5b5v=C{Ly)3~lX;?S!F18-*p&K58b z6nF{Gxg&fIi7fm$Mpdn*B4AixI}e)&Upfc7mG*IYQ!c9nLD5QBVI&Le>@8dl)dC0F z51*Um_Ts4sQOW`DDw%)Jp790~Z{~B5 z<&o()WyP%Pl(zD;oL2Q{5BPcGXQ*v`Fl^}cl`cT6{Zp)5z4sJ!MHlg)>=Y7O1D3$$ z(#bb<(B4CLjn9k3VpJ z^ll9zed0m|3LUe-vq^>);GcRD6`vl~HD_^Z-GnUIZrTJvPQ(s^+Iww{Hv?9|D zIUlM}UMr3zQ8^b368-yX#?YhsWpmU9T8*j1<8H&@YHyBPI>nMQ2VdmU?cZE55raN4 z&k4+Eg;{6G{j}Pg{UrMa`(S1w@$vLC|8hV!WcrYyLqHSo#zz5(e%ZS7U=$yLow~y7 zdck1k*f+5+W*W;$T@va%pm%~SS=UQ$JucVGi{w!_Z2a~U#_>%ztsH9ucbs^5C}AI6 zUUTsk2%Mp18Q2WxSoL`%*uYc73i}$k=HE0z@f$_zt2oc$+z>^~uO6<~Qn1zG3B4$U zdHby+Tz~N071mvsj{+^X2|0wb<1I^tcq6Uk|DZw=lCEcPhmmjz9k+O3YM=bRAmPhS zix|`dL6GbXq2bF;yn7`+(*^JVO%LH0DX>AzTN{AdIhsS@YEK@vjdjj|D_o~2qd}79 z{2Yss7e5=moz`%0?^Sog=lIldoON;^qRmT2lhG-Q^{hTqPqah#q0}Ep0$SVyn@v>v zpB=u>ndgO?TdKa$W+-Ot>;5o%^SN|h8r9S@NtbDOZS!o@08Lw{F?-~bKegX7{1hzj0inNh z8bWT4`@RVIIV-%OY)qx|TLu4cvF8q(ss@hY18|ni@hViZMbvOR$l1;FZ`fg13YjU- zT7lo;mMIlcmt)g|b)UMOY|nL^^^djIrC4mKQZ)mnaAnE(<_Gkbo9llB^ENI9Okoem zAWa+Lh;^y0vnll~>`wtg{M!Xq#8*2$qWC~T^?x8m-^yE-jyA};%jUol0L>LwwkSg+ zyt?p|<(!+4Dk&L%PH)gt!y-wVRo#Ngm1YR~s~wrK40MK`wi!dt%UAqIsq)6OtU&QI zR0t<{V|dv^&fGPK_%N9w6=8zm%Q=N&FN|PL2$Hu5%=GWhsTfODyF7=~Y9wJdQ*?Dj zfDrjpG(r%|maL=Tw(SvdA9TaveY(r5J;l*G%pLCXx6Hkv5wN7~CtqGnh_nS?gb-6M z!Yy?C!9i{NGqA8KK#Lb|s1|aeBur>I%vN)5MdK4+U5LSV028-_YJ2PSq#!*Y2>DFyAKNc0WR{ zit~flsFl>>d%UrWEGP&q(xPu<$GRn^5IM?jC$3-8k*q%X^jh%hzwpZ;gs)xDfb^}% zB2>wDyQteKI4M25rkWx{tZJ<{bn~MT{%%5+_CHLLpUeU0{+qvYv(vRiR|`a8H~nl4 z@^*l|$aTGf*X#MkPgou}gO4E2;P^@8f57i@y1-x^K>> zC5D{|nl}WnuyuAfpR2>X8R0I4zlPjCzwojSv$6H;-)H~Cb(X$Ap^>h@VE=ZM zm*QP^YNWwwIJc|hnL+N>NObeusL(1*fXSjaU9Lv3h8HX45Q5da*Ht2Wy9$6a9Axdz zsc`QJztz}t#s6?YrFTIm5%@krXem*6a^hte!ORzl5ytutr$Y#LHxMNGUm}s&Jeyys zDUNPu(wnD734se;>O^rbALlUEu{{k~OcrK;Ry5~|(L>QVlZqm-exgvH}{ z*iZOk62?>WT-cIg?T$s5znd<9^&>`0>E5B}ZQ~737fP~-t<>T|(HDgexL&?BRe4g@ z`S>Xrn@VgKwb8))xSHLzdQwCQ9o8g3SFX2rD!(=3!>LCNYr?%=#A!t=()nrN-OMNk zqO_;u&I5T<2S%)~@Sk`%f=qJU{|i3)yJM zyd^xs={M5ktsu09UczXWuU4rj05;!2bYEPA97+sV5E1n-9*66B7-|O4 zxZ2jiQ-Yp6+Y>umD7ZEtdc2&E3H3*ip}7|~S64^&cE%LKvTERs1l9fuzIUQ~tVDkp zclS9RM+KptDBT-$YSyHR4*v6>aoRTe9?jqWV3w+Z2H@xBt=JoDUGE&!Re@rFimzhd zR=QY6RfF*_(euR;&aiGf{l_do1A^9FN8EhCsZWhJk`0hQ_;-M?xyFmeZV4Y7VAZaM zy1Dv^N{=YGI)jk1hVR%RAidrJQwrdx&*_a=?bB9G(kJ8%GGlU8Ii&@w$hx>%E1kzQ zvf^Pl+i772ZOaJF-r#S?9KWiTqNE;H&dq8jw&UoHd_y8rwnPhMd=wnEVmmJuql6Vw znYa{;xStffKPqBKAFlJhI@~}v`@n_z>pP8&?L?AV;)824Z^ikBd2$V5eYalwQa~7t9AAvXrODZ+BoG5^UDoK33e%F??t!SkPyXB8j`6ukeAwCqaO9hw3l||p+r+xMouD7E#}%>r<~T0V32$?2tQRb zxf(q%UPF0OI_F@?Q&(h=L7@I_jaZJoQ=$lkpTzSjr za!z$8``J+m9rv@N)d1JuH{u_Q+jUC6UBvnSxS_KD3wY0x`tfA|ibJYHrrVUk^i%cq zZ$YTnsOR>ZR+T>fBf3aB9y=4W+@(KOU(tA)k6${S*L&wgxkA3bHAn+?z5jV9+yTbE z^@Eng;bhQu+Uw*jMf@7ydOZjKAk&Ep59|wE)&o0I3QUS(w<7iKYid%yOvfH8&9I5P zsw-M3zzcm|Wx%%rWZpD&xGZ4&qW@f~s#`9$L9y1=~0^80!Ne38k(3vFp677MQ&yC z?L%n9+Zgq$deRR9SM_jq+e?G_nBcOleXP*=cF>2_K$xCNb<%Z6PM}x6BgzY?cY^mL z@(p@I&aRyQ@VxAC0NDPG!ChZ4ZcKR=6TaBZz4@r}l9MdQ=3<@r@Bj3Dgu}T{0LRLs zj_SD@Lavs=y!tX_M;^9d`iORb@s-K~-;H_$lbE*!hqfwaK^}yOs?Y0PjEUmEr(+9E zaKaqzqdRX35_W6)hTzpOWJmM_z7)QG+lzxh$z*?lgyZqY9TR6l`*CpRm&a}LZN+;W z5mvkV2vqo?{Q&JrR>SAE_9Ns5>CNcA%ADXQD4mpW(ono$y7_H>S*`gS<2#7Q%-2sH zu~TlR!asO4ue~V&q3>B_RsjSOw^uW=(K377e9h4)x~px;QQ;dTaV?5^-SAL&p9b_9 zAeCq;rTq$t{n7R+B=oE9AVP;XMh)7O(_kWKzQsKA2{sqzBMdlh zL!RC*clO!OF350kFsIn#t&cal=LGG=568G~3J)OYdt>H22PrP(`na_gwedk-ON3q) zr-DM%n{<>-uoQbOr@wKrtvfB7+_#I2d}Hq=MMJ9XBV^Epg8P|Jefh^qfi7L%(t`2Y z(R5~9U*2BNR9G5Gza1o@zN{=H1!VVmn~(5SQdLST?-W_edr4JQi_P!naR_?HEre?^ zz)Q&dcC<77GmjN>*jf?J@CAzTxCAqJkSzDkFA00(^&H`H$v|QCVX*=J?5h?ttb=_I zTC%#W){Xr=gqVHoI9{dCavn1j{@82-`IKJj>{~y0pkXb{$_(wdiR?9fOC5_MJ=y*8 zE~DZn89h0j?dhMpo1=^8^Fdx;M}!i+4m)E^z7IO$x_{oA|5$O@Z1{4|Uv^Y}^LW>6 zdcE!*E}pc$Nb(Yp>_ByC*Beh zRw@SNHNQU^sTOC>N_eUFSW)F@?VWV1pEPa;(~oq=7dt5HOFY=-?=X*ambdM@t5Wrc zZoHN9IP=+vt8izV|11pU5>OV5Xc#2FxS0*@@u3e0p+2FZ%>D7UO)<05^2gk77RVD! zrk4};L;##VjJny?9476#d|SzojOkrBdiS2_!Mkad3)|(ME#7Xl=&!=KjfPy&ql|EW zVKQ`*FU_4&S<_s-9{M@YX9M9J@+;5)UWG_qY+J>lSmVdaM59OC1kut5<7Im#q-%oB?EatVxNOI7t>9umS)Y-xS?d5d9-vk-# z%q=33ng}OJf-Yc1Zr9A($P!W?rRUaF{A#@Z@7f};^gL{6LVu^eV`jWVVQKY(#AqLM z8exkfl7aDvD!3xv1S@o@Vm&`XP1G7JpVNZ;W^D?Qpml^OjrF~l##bGyVZV23 zh)S)htzJ>pd+J8)3%1o@n)}pY&QeRRqw`L!)6)U4HGd!cc?5^7Ibx~Tv3cCox|xon zY8xO?@-Np~wZ_TA#(dui5j__C_gyFNb(`@8UzR(n@I6~qd*A&Br2M6)z5|x4%c#<` zA`&3hIvipR{5>dZI)ohm7n<&>zv9^jut_a_V>s45M_R*^2>GAGz_~;B^PcpxppDHOjxHUwf=%0`dB{A`V>(5 z-$gxoX5IO+Tza5%4I~r5HIEHreBfFfmNxgyXG^+R^3jFx_mZ^`C73n7+x-i`A1)sU zxAuL05}|be{R|Gg&8@&asZW2LrcRoax%cIB^U8)^u|X$w^X;?i4Tv`r$(Zp~^0^z5 zJBC0U-vGZuzbz->SIR%=`re5Ep;p#c$NFDn9ne4vQrgYe!sc6?`r@w;>bTB}>BIE> zd~Y{>%U?qX9X6wcQMPJ!dw-KWU1o}iiKzL*tV zyheEuaMc!X2=v1G3O^8vE1@~a8>eG9)#jhxX*Xva4@(sHs zGN}|<7AarRpzR*Xg=99gA5r|3?!0~+UFHx^0xfNS9yg@y`rhUzL-aHaXoLAHBn+Ip z%wQ^Qvi~e9;SuhnpDOC0_H=1+lx_U8@s<+wQ-R2VLFkQqN$eB-J~$ThfRCs2NC(Qr zi#6#(``njfWW}v7vZy{0Tl2bi`^ypfju;l>ojA!e7`Kw0%ZmVJTM@#GYXz~Oqg`MM z6E;Ufr48j%*Pe)`RT6ncju-*wAtmwjhz59W3RD9>EzFz!spH#|ibO;d>_`LJ0S$mp z3D?vN3Y(Rl=Tg|gccj?U&FF$RJgzKbXC(Od6C`8vMnG`$q4qP$7Qub;$@=i~4V8%x zuls2^NDv?6ZVK;##0_WpRHNp7cI%x$q{Q@&#@96ih+pTUVO`*>@%J_0eU8x;hEqGj zd*ZzJIA$*6 z+g@=udDaHvkrk)f-dF(Hhxr0{^Xjl??e-$&Un3yJg6?Y$h;h?RcXtv-hm2ChSihF9 zPa)t>fkE}pOv*aJ7PL#UM}?=NfYfcBEElyL1T5E)mRpz*!G64Pb5_)Y7{ozh zbwyE1Rf{WsLSB0QUI9IAV|q75irJHv1UjFEFo`B6OI6cnKbikE(;xm)%fKxAx zhTP1O_8j6rcMKb2rvH3AQi=m%M~yGybv(y08Q>4XcinsgVACg8GqCTRkz@o?&|~Ko z(qd;)0yCjAIzX$G^N%k6QfQ#Tj0O^Z(R!&v1q1p|p&vSp9_F0z&?^C0^b#1~--U{rvdw_192Jo&6`m`bP}<_>qmrTP9U96CRY_U5zLRcuU9O3NPBdgczq#DF&cr< zi*K~!4kP>MG?^_JncH=&IpmYngTI)R?mt8hWH!wV{pR$dFFcv@Mt*c; zKTzvVZu1Vw+9%yuO>xLXYG@mbdS7$HD(HuqKJ}J-d)B`Pl-4cjr$vmKFQz1J_bj#h zmp>*|2>d|;r=o0E3pGInZitc?W;Eo*ia)~AkgtG!vka}wLd;v%ZTbdMOk>3`QC1ST zZG+Q{>XJYP>}e*8C|wgNQbEy@1;l;}#IKFFWY4A;9v_TFpbBE)dU3qacCegVaT~23 zDZ%e35F2nF204afqX}&CR_*lUH;fKCY?7q>Iw3l9Wd+kqGc zM+&ZMFdRj{eUFe-2wL32en@`pXj!Je2;5^}Yvh>%r zTZszdm%>XP!hBKIZ6eJqLBw=xcYgZXk!&!)n~ z$G}kki%?xgH$%G$5%%yR?9t9(pVDAR@JD8yCbcuqhq2Wr1k(Hta5{q+W)dObtEZJ5 zJ@=lDK>SRo4VmlJhNf^EU8jgouOWD0#)u%;-L3hrUl$RNzqr?och+p?LayWPdA4CE zGa-*|>6d(VLs|?DMd&KiayFLSmr-x@HCQA%*{6o31b=BtH$4*6FUQTou8(-wQOTG4 zf`qcq)#vyS4s<0Gk;lzqiu-b@(jW<)&|QVoaGAL;FvSADhv7EryFUi??;qXa3 z`pN+5Z6Hth$}xE{()8^Dy?li!hkI;*M@mm+ts=r4vBmm`(>MY^=U0S1+(+2!mqJbZ z>F=D*tRY)$z`d`wENoRJbdQ@L$DQ;h5)~CGr^xS3VB)Q><50grNb(sf$Btsa zX6#9Tzd8=|6V~l8T`s?G7rA7Mo&?K{NM7qY$wzG(ltD+CT9a{k5(LJ)KWyKcd#px6wi`ELd_M^nh<9efrg>c2hM8+u0 zTzKt1o7ptA&)3T*R`C9(xRd)=lWi%Ci{9-(GP=Kd5}=hFsxH^HT=hI;VuVh} zo=APYA5nyuYMVhYLDNg`M#3jy9*zG1VsK9V95@4BhjE1Am|PO$?xkRQz#zrQ`RqyTk#)nfhd%5z z(xZ)!&2y*BXGDr0$hWcE)0>2$l65i%w!Qpct~!KoS{(tT|CWvISH4gqNl`yj<*1Rg zH|wL1pL3)yVRbS<^dSq~h|!xjK4sP1l=SZ` zqq_QE(|GxFIQqXUWoY|HiSuXf>rRM8I-UIIhA~dAj8lgO?F7*Z_BpGeoDyXej0?UU z-02*Pgtr#;cL@_^%XDm^ z;T%_r277k30#3e zSy-R!rqz4Rvtr1G?=zh_vODdFlQwOHB>l`C#o>9rb2zpJeb-mw+WX|~FjThyZCr^P zH5k^xGZMosN)|wOgZa3tC$)$rV0(OfVz}1=2{}e~AYu&90L<;iT zU{yipnQl5$Q2~YslCq2V-pPvs!3JRK;#c~ouTn_=d#{Jr2$N|&R?~2=zVqP@^17Kd{n-F(#+rCs&LRG@O`lB&o03*e#cAs-2f1~ zJIaoj7jsW^`h)2iWILx{)AD6r^xDs#IGbg+%gBYWmXV$%!E}5)(@4%o zm~y!5NKX0*vN1A|&y&)=Gw#B6f|wxJ!tj?k4J57|mjWC~rTF4~KnrN0L)fNG>t#o{ z#ACVZp_9iG7Kzu<+%ykrqlv*r6pO7VQws+^6%)=Ios z1k=TcqQrTN5avEOPJUet!SM}k&m$U{$R==nh_M1AF1x_KyXjO+ON68F2lkAMo|}A# z@BcvQmxsWtCWFH~A#u2eFlkR)cES;FKzZki8mQS3z-KX~X%7bxxB~P4RP_7%e13Z~ z9qYE&?!#8$<~!U=qskfK0f((eAsV&@6po#TVR6Cn&8I(M51uTPkiox3a1reve$L*G z43%aYYEiv#-^&*tFQ-)^4*XXBCJ7RS;+q8sOfE6MXO0OsqKg$KO@OiUoPc4l7v{Ek zPQ!rD)&cM}Tz-?k!j8a6KxNi5 zF@lxeRO`z7e@?j(taG}B_z58dS6*$-W%ZxBaC`jUcO22%ZzwA9_ijonIvyBu4bXqJ z7#gyVl!|K5Dh_En_&=JiJ1Xh_dsnVOGc!{&Ej3f2G&M(Inp>kZHMhA-fz;f4 z=Oj~e=3Z%udn@K18ScGN1O(ncKHqbGfAI$g&Vl=W-TU0UGp zQ&;Duk(05me}@vpvu%c#X+ZFg`_VZ!J$igo>tc04w~|&!wto0ZBA0&|#`-SV&2PTD zy?6bZ;+>m;6PI9VSC!8sW6v4}?D@~JTY2_@?(4<8r|nbsHBudB>i%-auQJ9f^4cfn zg5M66EpO@;M?0C(y|J3Y^-0*PY+`xyI{c8tZ}$(|-F18R-q$5T4tD6JB3!i`_x}vN z_}vlvAx5%(vBYQOzTwqcng|4;Jyobuevtc;-8vD3PV0euEc$YuZuL_H5C;qPy&ffg zL7SM97$Hor^o%IX=v^CM0rC56tor)cPKRhHjOV1&{%pU1dR#c6=|fA5py2ui zZfY_f4_{ z8)|tztNZx7WWIkTg`7|#-R!cE<2N7x_FNZLkhBh-dh+49u>H5v>jOi?o?i>>7-kPita4{Aaj$D51H%}X|k?YXpV*1rZskUyb#e;J|6JULHXmu3s6PN`C}qb zNn_4h0;V+%WxPbkrZ2RzH;TmW7gK=KOq^>24#8L3u;(->^~;g=Ek3zy8cx57pV<_y z9UQSpgX^|8Y@jwblI7Se^bc6rlz%!E$|%&dZB`@nHtIB(zpeN zjvqD6PxyJmaefN_UI!H=cDvqp>PVDUMtc&gX(nnj`e%ys$~04?`pHlEa$xMKM44`m z-;32ZX_Gg^Xl>Wjy2bMb48`j?fbgRWLfoh zHjsSS3V|PZd9bz~bVBZf4*e#&IBUI76~brxuBM&%@A7^9Jm}tn)Jppiru(N1`U}Sv z#KiIan#4Opsyfw4Q0`}>nU|BL;XQ^>>(x}U#WGB(sLPDZ?RNu`4`jbToLxmPP)dx- zFT*ZAxeU<3sCyKj>GuO&d=Agzk)7qla@nbG<-c*Rnb98Xa*|5qIHk=C z^%F*qU~swf#fQ=p<2(>Q1wA`*CT?qj_y_33!Gdz|7B{HZhLT@A4Dk+`eQKlO(q2)w zR~hy%vns;0U8*EeC~DXx)4IPwVAB42|CH_f+bmDCI6giYY9Q^p>y$RcvHUIw9NXrM zyLR90;GwyjKXg7ezlbocuq>f}zo=Iy@>hdxJ8P6v0{8MGws$wH!09bU0z&aLt@fFJ z@cG-JkKN*~5?Igwbg?=VTKhm8lXkj7kdd@z9MVrypw|n(y=7v%AdK8)(T+&r5=sKS z^VasCiGHrsd;?c64`)Aoza;C&)=FnXlK|L*Abn8Bm9%Ne0yA0$WHO z7k-0br2=H{I0KcYS+b_{)u|Q)q1K=FA{YBbZqf;&FLn@BsS(JXf_XNp!!;}B)8(Yq z?XXv!FUfiE?}$o}*PjFIuxpv*O~{+aG!$iZ=9-#Qj;o}Z*h@a%59(OD$kLIXBx*{$W=hR$LdC1gwIL!z!6Im{OmHj>y5logkfE2&5V6wszavlkKr^2DU zSXpp~c+8Yu?N6>B1~$Q$Aq|xJ#F){b6)Dqr(4)fB?MMmN#w9Q!dnu(!(3jjR<}CSX zg9&cJvL*+}J<$GMUIddNPN4z!dn;(5-`r_P$m--}zRErQ%C7fwJ167?Gcz=DI8_iT&-Dl#-9Y_;B&4Ed|@h0C3n^RpCgvBSASpkw7DB)FlJu~ya{4L5I*Mysqw{R z6U$men|H+B{ax-GmHEmL6%^Ulsek=qavN1U^FwZxCQYqXOaJm++BvawU-}m8BmIgr zHpMwU@M=&qY|FC&2vlXwtcw0L3{0C82+x7Sw^*NKIMP^i*Krk^F`NJ%fs z;p6g%1Ffsbi>U{O@yH#^zni>xC+NlBDaZ@*1!x?odT$y-lYhSFrU3se)dvEbaG|Uha*dV$$vjz z%>Cv3DXA&?VjzLhy8Df~yZ65<4?1<)c;Zy_G9wlT^*i3i=p{V9rlj83q4*(_L+v)# zQS~?PB5{q6?uate+b;a!m#R*+I~6nv`@6*R7$<5&;?5BW$;_$t{3bXdm(_Kr7`@Uv zywXFU;RXY6?A-zu%Ds|&CkWw!!J@&X@Y!XBepypi|4?T+a?dl}-FVwG5Y2Q;HCFma zL$2{o&~&;__hdeKqdJrQ`I%b>vxGkg7DNjaQMeePcW97^a)ew(9ICm_5Q`R+){?uL z4-BZ@R7*%4(1?&gypw2bUC&#FT}>e!`)SpLzP~~Wj(Kn$a?Rk2eJk4WF26W&p&6x$ zjkxY9zBuwj;FtNfSvRat!bteiNcs5KWe&mPBsc znQD%Qa>pG|$qIu>=UxopWnOGuj}y-0fs7)VI5tULMZn(}SLYdr=0!*0z=roz+|`e0 zf;VVKM4^??;1B=uX|&k(g>vEiITjw3sa>>p=*@^g_1}e;4opY4sdFonr{*+~{I(u> zkA2ExKQ8LpmC2*oMY?9v4UT7Xkes)^mwt{BAZ~y6#Y&f6k zasnIT`4KpH4i)#3lxlYmnGfvBdytx9KN{v>uh1i|gF@djbMN$OPM1YF-FH|97WJZrg61=F1sA09R(RWUF{lCPQ48Z7bjc;l( zyq8u0*Wx&dlkWrz3;#Hb^>WkvAke#UaQ3Cyvk^7{5kLeey7$L0&15Qrl=&`00oZ*= z`&PkYZH!%|Bwv9_50_>{Y7!)m7wq|z#BEU<1??kb61IBLemnO^`9!tJs%YMKtr)@T z@5dWxk2>tfj{TgJ+r-BMfkTgG?NUwcA@Pc4hkO4#y!|m-uhepTp>LQcTyfapw=*Wj z)&Q}(nR0LvD4upZh!gxj zQP>iz9gPC0hb3HduiES34w{qdy|I*K?1Qw-P&fm^!~YA>r~HEw8CMx9rWZcNf}#_J zB;`$0yt9vgLz|_SrMIi4fJ06xKg^tPJ-O=yV>kIl0VYr zRMScmg6RNeXrPWZXtr)mw*Jy{kVED^tS6RRUHf<4h~_@R>taWadhr0*j6z*_K+u^( zdic|(HD%FDpQLCbIau@`!p|Q5uHN%KX>$X*t-kR**TcJIREvyFuNTTZci+_E8CR2+ z(QV>^b6)?)u#jo~iHpgyprXbc=XAw(cu?A`Ll-?@b*aN%RIHP=h5yOS?L^PTdrLWs zI^3`B$ca5PviY`s6~fh@_Us;he&jyvRnFH`+ps5}ukR7Rav;_MZ?cYporU+W)<;U5 zR{qLOdLYG=$Le4H*Rk>ZW~|Mb1j1Wuv(=ZW9E#^2h~HUR!;3A8j2g)8TfVIjS-fX5 zpKf^mEBPk7n0j58((lWdR|l)}6>;(brL$pJ(zR_G@fax4b0m;;y}Z4J^6z`=})Ql^q6J(`WGFS-C4k1@tJ8#osXiM z-q8dw_2zBiur*DhRR6R4h3PVd@aO(RzFb_-+8@yslIQ(6LswMV#5b#28w7CmdfJF> zQ)3W`jmU|%Yxx?WoHX!57c+=YP1M)J*FrFrcgx>)I1|<{%u6e6`;-1s&MEr z*GO_N)CSTEZAH#=<4qG`k%b3|m$g~lB{)@#sTJMLJLr+}{w$uJ&*rll} z)4<2GcKVciq=*l#F%Ivbg5HB+#niV*DI zTTbuD^*Lv6RuR z*~2|MV1?j;_MNFSZ~Gy9cZNA$^?Ox4qwz(Lzip%bCo}dT&c`cg6Q#>`2pE()_qyXe z!byFqne2l*u&&2iT~=AoLNc)lHUP@20}lf`il9xqKz0}5PsRY&Ws4?gOb#XMwgh-Z8DH=tE!POG*i>iCjIZ9ggZq`me!jHKs2( zqZUN&k5ETEIp@ehp7i;>5C{t+OXV*i0s~@=(AnVPVEDmoCq)`ds9(@BG5JKtBg$ z;jq_l@uj8VIYmX3Q>(_o>VlxxzW5uQP3L?3!}hf;ndE;eBITPp#s1yy2^*ESxqMHy z>5Z+1U43BkgHm{<=}m`%5vk)Zc6al zvMk#SZ{A3}e!xsG68DP0`Yb8vI|KbC{w?|{Lf_8)CN@-3FjP;sav*Ix!Icw|a{JL^ zYFu5LW?mBH{8z>KjKd}e&cn!Doiqwd(K(12z7S( zP_|u=078tpDn^zN%4cb|A@YcKcXkqGV8I~yM?Q)MQ;@mywt=Wd#H1@k?ljisR_)gi zZD0h`j@G7xOgyAY&@Njk)3hBIawyu#V|HEg#m%#{>KnDo?q>C=IV0@%1P2vOykiO(1=k+^vXsnI_Qh>W;L%YvtY|Ed*+iR zL%|%vY@=-DmPb94kxf!!y8k?BqG9KlvSNTEg5e;(_bTM2h&1*pl5G@+1SYmA*$kEfVAJ_lcvA z9o!aw(j#3>-*mRRK7-V{6w|z-Y4;rKfNKPv^C}IA@tXDZ=_xq%#F@?t^`h`ty49VU zc_q{-%?tT+Sv=pBokaAxGWskP=Ms+JD}}*!aGTY!LB+W!NOTwN|2-JZw)tRl?cv*= z=LQ9B&@PN>LKvjF9{aH;?pn+gy*Xz2VCinq?Phf+iNn3w82=R;EOh{k^O^U){Mf4B zp%;TUccv0g_>spx0Gu=?rA-OcfefHe2eAj~uE^c{KiLq)=|AauuN;=QT3!V(2by#K z6RCE?N_EX-g@fEBBMT^%vOY^aJ8RT766Kt`z+L5gq;~&bJ&=94np>AyAugJ$i}PA+ zkbx-^Key5`{eS?wm7}_a4>myT*thNR0m7;)=-n=QYO>#8(`FNZuR}vVv+%_*o%W#< zyK)TvLY9q4WtY@sii^%MPy0~kl!G(|k)donHB0gavt*yOjafK(r(6e>fK)I>cSX-V zcsSJmsnwFsdj=6P7<~Q<{|PfDqxlD_a3mcm7Q?4tPrmj!-^OOwo|QKC*zH`mc@L-> z@;ml{_>9Pe#tPOx7QkjRJbZRon`FEzbF#VTzbk;m1{M3hIlmjEXS}l-w z3maE-jX%4~rC;?f8=|wzV;_6h@ru39KQm-?0E4rV*q#G4d)533z?To zHscHD>r_wbmJO8T6F>He|idvKJG~ zlHoqHF`j)(gQs}%K<(j5!E2hs#yrb(U+iMf8ThvsO)c@uOyDHb%MJ75ueWCnjIWvz z-+VP*kjPFJ0N;doQBT*zK62~cgm~9=p(8JZZKgg+CpURALzIu6I9|%Ri5#9PhCK`< z_xgu}Ac!e$U>X1uH=yWnS{@_>#~&pC&{=3Aa;G{5i$b(3f6cK6c%Okt&+d(?PdGw) z{YDX+g!`ji&>69=VHCyjnsRmBIQ4ANcEJ*w2T2;Gsqp@hC%4i2SejMhG1)_UpASXz zahYi?C39S29tcmeRmjWvQTlcqStJMl$S8|*vq57z_q|7&70PfuQhuN`ePGee=3wUDe73sa}pN*oOF5q zWv^(zx9IchEy!zelD3bgRIBLQ&b9t+xW_u+iSb13k`b7ui5W-OP#Pnn@>A?2yqN;u zIxMwQ*@90xBmMSff&E+`>gnvRvLEpCp530`ShT`ELnxN`18ea;3_xs_HK7Te_r46s z8NWK%=PMhvvnO}~EB)eoiXI{VqCxcVBeqF_%)UDNJybsxaxmSrtKtFj>oU%4B)?Fw zSW-&VB<%h4SOZgjVo)l60JWF%%EzVS$z<>f8co0;cF?C-fTOPpwb_kDP&s}dsza2I zT~n9QcC=Nq8%<#3F4beWd9zr;rC~F|7TBVm_F~}dtIA13OLPjae2#4jo*M#?StOCR zqmuT=%ba8@`At*9OUiy7F1+-pM!HZ>H=6c-uCbg{7=D=EFo-q%ODh z_DlPXu}xN^DpZ>JA=K>O05$e6^?+lJ6wZP5W`N&hY>%4 z0%X|Z@-8eCfycpqozVEC{^u}!4_{fozLB45OEi^8UZ6NRWhwfASY}Gf8V}!0zN-EM zIL&g1EdKC8Z2Ex7njmsdH$f^Vbk1##HXph~Gs&p07J|~L>=RBq@7aOfyTyS0I;H*d zA3N`z%=u7ijUnW8M?qjp6SDCh8z_>3Y$LStB8i)c(*Hd0e-*KcEj3fm{0PUBKUO(x zAWcWDQbf_$>>q8}2oe3R4P7?GQs7`onu==zk=9QkN1CXgr;(`Nr!12&FVWZ` za!l-{En@fxr$RL=7tk3LfJ9JQ@&Vi_42NHkf@|m+yaT*u0F|Yn@`-AW3k{``PL58< zLU5H7opJy78_V$(jS52erR#IFVZ*e$_DR7!43qp)6~Djyeq#?Vv-u6nXJKx?TQ1pe z;T|bz)xTb%a6(>%EVVO%jpLr&SP8n|^zls&ZC^`&yX9TkP*iPP%m(rMT?5|mcKYTY zTvd9WwH&;A^@k0T8$6~wBh0WF7u_(VTRg{pSMSxw{t@1;Z9an?nxfdjt*i8*Z(o)y z*xjj0MAM0%S8RRpPH*BMt^HEM^R8#2S6m3aU1Skb3-)qSRYLXGGnc3}(&ld$P%rNO z0cVLg%iBH4U7T*vc99v$Ufm&V_J(v0+f87l=i+YM-6DM|{ib0O#>Dr=C4-eh%zj9y zet8Ccgh6B*L<$dfpkGhS8c!WLw|0LmO#WdzQ?u7d?Ow0$MbFgl|FPt(5FUOr?~0>s z3@l7?TH+W4bLx{OH5LwbX19SUywt~ba43ZvELBwv|6%FyPD+gY2BmZ?e!9&jvVVsf z33UD70-|fJMe#=CkW|gQjIT)AI*k))%9O-EoD%ZWO5{uo<>c4O9xkyl5}mqe=bu%_KT`S>+BO;ef zKwEy}oE55@E!~i6z3v#TYikggpqA8V)!SG4pws-})~(d=Z{r^a%a?m5n3~Rz6SFJ` zOR|_XPAPi@8=m|x3vS*8*?xNTa!JN_rhFDe4uUaxVNS}X^SFim*|z$_;z#$DE-;_* z!(W|xvfT48knr<-a~?$-aWCz`V#5tg+vUKBe}twidT36V?b$eUV(GrALi~lSv9|ON z=OCfuY-$HYP3OY1uFjQ_E%)^Ep<1n6lEQDFZb-m+o1d^4tnO+%ZAhQ3m6kVTntI}c zy%G;0xiO3kqxki1lrCrv?T3Oa&Ma3PM*yYJ18KPbqGOxu^kIb%VT zAm7YXr8q|zis{aK9rCUPWCLkv}*o0W8DV!wL5j%Vu+KJpi@xSJwrX2Y2AW#1b`sAu{ zTDM|O4PY_IvGJ6ELH%~PlS0ImVq}r8RxxmY(^brdFtz;$axTU#eDh6nOZf__zIDo ztHqJD|0tK{pMu{bvp_Hpi;(bu{ubz|JX{+PAgdqis4-Drkad78Qp1neYO^hTJ4##k zjub@MHgfNbl;Ky}idhy#y9ZG7f12|lm1OBtdvfCV--Q>1v65+y&hlA~dyIhukii8& ziXs&NYL|dVq!adG)hNzpnKHP`Gz3LS+s}p8N-KHnuWi})fyIB$)H}ckvm~b}w7Ci! z2=rOh&;f5sV)b`XI4byCh3Z7cI$%c%w(SnEY^RFq7 z!;#m|uHD-i#UGfl3bg@tB|&nESUaN}@z|Ajb+UP($3Mfou<4qhzh~aZV-lvyckM5#XdH9RZn0FUNSn~s038|NSG z1TDIDSrD&1!o&5f=Q*u%O=VCYTdz+YA9Onpi<5lM3XiFsL^??y*eA8rKbLiGy)eA@U8N%=W!d+u`)w(!a@n;W->59nGV6-X`f>jY@v> zsXY-Mw_!x$ywMCBs%kr-lYGkD`_#<*#U|8f1SX{9KuEl&Om9Hgyf+I~(RO0_qRo?=q--%`!PL0kJ73NpWc7O zukgw9+(6zm$ML}Pp837C*is>z02F?lk+x}Zq!RIs4Dw$&mN4c5I#;=XX8_MBv6#3g zjouNsW~cj7q@bxCjaWa;)~Yw7v@fC~ff%ZXBUNvyUzWNk`M>(1tCp#CSK4G16ZUA2 z_HfajAc<@n>cF{A$CHzzq*R-1YJX2kt*m2~q^|{S;c+dqKo(Pf#;-OnH`7hUrvoDTdJCRdC`E6pR=&M%Y~$IR#vsgMt)GUiuPWj6PV3+M znr@A%ud)<9)QWC%$1Yr#@J+B&*WgMt{BW+@t%J}keQC&UpK*SFh}(kduZ8xs0=gD=0ZA>E6=XN; zprCA>4*X}+T0Lg0zBo-*0xJnG$-zz~SSgoM!ZQ4w{v zoyhe>TkHf@-@}tKH|LfZ{3!3P@dh2(W{^C$ZZe57k&n#Vj)}B0sImS5XF@m&y@)x^ zJc`X1tB>8hfBN?gBnx;4tJ}H%Q`!BTWlS3I4(oza1C1o4?>)&Gb^*9KLMR-klJG{# z*D&ZeE6BuoDtPJ}#gi;Dn@YK4xJfR7&p9|!McFPXQQy%LKqy7Rs~*#dEJ6BJkWWN> z5Q{-b&rksqrE6n&)bZFJp(_-oaKaG@q$o!Mi{WzMG7={0oE##W`hiv;jy5Q0)Y1TX zx%CihLsmghe@3DE$sf`nvBb4?TCj0L)DNbB>{b3P*s8LmrfCK7`H#pe&n@0Q%cg{W z=G#ucUA}Ywdt%p(zPOanu&G72d~m`z{m~~oh-COT6KYN11>w5OPeVGn{Uld+ews$x zLPAuy&^C!D`xh!^`^)PplDw}XM!wH2i7}eYllIl+WgiHTJcG2@E!fPtR+{hyGuo|< zBhsb&TFh-zwY|)P-5`JEM^_1|!CwBh!}Cp*Z4SZp+jIMJsXpNzUgZ_51`8p3s&~n7 zm(Oq+y;5lwYB9L!ve>h^{R^w+{X{dS199P4*Fpi;OKToTKAps8_wQ7@s&2 z8q)Xh%k4hcbN=w)pt#a~Q3Jx(j-z7_goG0)^$T5B^9Cuq1;hpya=1<>uqOngni8FU z>2sYc(mpc&cn{QoUR<305$)s@7JEYtH7~MZd&=&AzMqA4^Wj0gd(V8Lki`M_si*X% zVDG85zs+M`w+%e>E;c`8%w9m|Y^l9BQ{8tatV3VSqi&IdwymP@D*jQWB4h-JHNt6sGd30WaD4=ndUWDY{*p*&9Ye*npvc6?a-7X(ZX}Gpe*LJalr7tLUf*t0I1A~?o$Rs zHOt0WbYi3t1g6IVk1K_Cb08<@CbGRm9d(EqDv8h@lAwx(xRKY~Vox+wB98go0xROS zpg;XKtDp%8V*U78&$>mIRJtiD1abf^Y0%RXF4P1ar9!S$#NO|@3FID~T?>qnJu+RW zP5^&{+rgjag6Ab3%^2$hW)&^}jDeBX4tN%Mfx2Czvjv{K=JhC3l)w(3zXWDC5~jPg ziK_hN9FJGc2iB~e zsBrX7aei-`%6-?GxcOPKwkR-S_Y5GU2A}`4p8@v<%*Z|muv3+t^^$cos0p`(J`Ui( zHAP1;7{BuB^Z`?$DhK9*)&S4wQ4PVB1au+=&?yJTu@$or1bGDhlrVmRm1({^Gxd=D zX-++1PnOzsIbFaiIudxe10&ZTbI`JcDKJ4|D4+TVY9RAK)|IgQrje)K2(I_F_uHdQ zfV+R)%=@ohZuD8O6dgFfEMU)b&ttJ|F!F}nP0f`B_hp4=`92rfH1v&z5cY)MXdIz$m%*-M=s=~l{+eb_daDb)uVyf@UknUL6b;?$3YyQoT$-+8_nWsA zOcN9gnYK6nU3x5zmo@?2jQ_=grJUbvRazgpgzAvYGt6DGbLORTxv2OTppFb3U#1Su zqeSnIM))6l!Dl;Lpt3VB&UG(>@;OW$buOgD1!$W|Wq>Z8L%HVbgXB}cI26W0n^`IF zr!C`EW1|Q)BpQ@=^!~z5O_-+q(_cEEX8Yuum%7*VjYvt9OPdqXf$ood)PN9lC$>Om zVm1}TicfsT3A9O2`fpZ-XaUdQ@AS8R{%Pi7>taC0J#e_M6o4eu;iF%;;-s5k)zIuG z3!tX+YQWSQBRIW*;ouA&wlgwoEb{6AYY(xHYzR-EG;?c$_k$H^ePSkM>%l2(!y^ek zG9L*Dj-s76&;;7rKzx2AFybY-RIWB8OTDvHj*HtNuE|m#4$@G68j6MvN!l@k$4jzE zZ_5y7)}B&3e*ur|OqmRALw2+Wf-0PB-qsFMdd$PsUk|-dxRa$k!*WaT zs!@*apHJJOhE}`9{HI&Do<9gHYy3b;*?RP#$-^TgC=rSp6W|kYdGdg6HSnZ&j(la* zM#1+^E=91rt;m&)m0@y@ORKNN%(wC?7Sq-c{Kn>Gpil~>`9oIz!%iQ6)yw?Z!654V zJw|T8C;^*!CY$ipi#lOSm9!>Ln-k4#P+G4Ab8m5k#r`-ayJbKqzE9u=Z~SdBSx5y( zyvzThsVQMvjILGVe!7nwK~5A9s@);_D3kMgg&Xzq8mNEG3Dc|SXzWoM@ih0HQ?BmL zmi#1RGg$|T_q^pWfiwp_b%1F9+16xegZ+8gUXMDzO%@}sFD0@an-)u*o~@a-+b@An z608RW(A(A9cIK#FgdTK4a~eu75}@2fU1U3TC9zZO_iB z#_e`NF%DSg)3<)_4`5MSw@$|@xB(S2luWZEmDZ}qX#P5=i9(N(-Dv$fB?z8ZcGoe?oy7he_6xq%+cI!ZWX~AWoa+FQgy4=ijgYf22aj|MQH><$w zCsr*3mnqK2dCzu2s7c9L{96u;#Y3#K#s}&L+B^De&(E|vC8GUgR+*7w9-(&Y?bN5#mwm-PkX^rOE zf1O)uOgi-q&wnjkHulbvbfUtlx*Xba6*KgL2E1gEwjo)^FH+K+;hT+*?Mx@oj$63~ zv2gd4$EGZ{S}*&>Q=fQc9E&Rnl&v%jSw-Fp-I8Xq)wGSVnD};^<)uNQ)m1U8Im7i} z|5$z=%Hf4npF6Qtf*kkDbRD=OYWUHii#|^dR`03_!BRH5LJb0!51b{+Hm$i0=^dXU zCayLrN6lm68$EB<_G?aDBR{+qf29S?pd*(WV|vb7-apSK82XayM~J-thTDUq<&zhL zO}dF;-D#;PA^A<36?EVuJJ9kwvJ-Y#@DQ_o6I6eVkOjHQ(jNjLm`n9xLSnQVj+{v* z;xYau7kc)&QAZL~$rVfcLcgON;RxhH)8k9<7HRljhILZI+LMrVYi*Frqct9s83XFC zJ4=ohDg-@*6a)LGgE_$#+Q@V(I^_C2f=3S6o25?LnOWe_3+@{Y?$7}BzfqGQo1Z=> zD}leryZdX{6Sir`v}$s*BC@qd-_ zzl>XALmnhM^B~uV9|p@a!?!a5N!;21=6`{GHGwWQwUGJH5M$$LwGC-duBCe;uu>Iz zi#_353Jov8Po~dMEuc1bxkFAm1WO+NJb4=btj*hule&yLm zX?CtuJE6c$2Hrh_sjWKpc4@dm7J{ zxrlswIz+0GUdo%ZSm^d#Mt{d#B3Ob>(agUu8?idDY1cadz?UZs@YKe#P55J}B zF4={hk*D~bj;K5ep=WP+yFuuliM{p=$znnSPFRuuPAE66U#^ZqdnU%j4k2ln_e~Jm zhHTk|{XqTn_(6T)v^e9)8UUwO51>xTY{@iYt)`jQCnKFE&;{tV@;dfUomYO-u}Hr9 z^-5_(rNP|;)6!|UCC_duQO&6IpeLKYh~7HrTvFhQp(LKQ;rjXW-#UlV2folJSgcz# zmGuX&{QPxnPx(dX`vq%!p-Q4wGP37<-sJX^hICvIi|u=L+i>vqnb-e@Q$qlWzNe{4 zf2&$s4--VoAeXPZFNh7p!j8{i#cp(nnRl-ZD&ScyZZtg~n#nXv2wKVFD307q|GmGJ zmKy*o%tH3f4I<}V_kFYmygy?N&pIC$DFl9NlyJIT+lo3lc4h{d7p__9mMF4)`VOdAeX4RRyZoNvE*- z&(ynVBlknW@_d9RuG4m-*tzgUS2~wNSUoBW&r!JwOQdj`kW#^oMg((_q#VtFl|}S# zxZJlQV7iV5XxFHA2e7VG8ZBzRK9zD%{~tkCA~zf@`&DY`zf+1^v!|l}V>ED8)T~J# zYf8p=;VkFS%{n}Yba?6C)BjXeXO^J>Eq{1+&5Ks6TLD;n+$nfYe zi=i1>nO}hOL)f+;s#zuZuTh}=^oyZR&4KsHhG`tPb;-||9N^mDmpG{GM+?xb*C6tG zA{)QDyHZZ^9|w>r;o_6efyiPJI{Ny2V<(?@@2Q z${JX8wP~&=4H7O zJ_e(|&fn+FbEY`PUu(WuZ)ad8-P2N9?-g}}dqZU{gcQdIrjObB6URwmom%;NLW%gSa7l$~k;e}L0)+ql6$E#-zAz=f z%i2?(Yz;PRRZeUUCTiKPbk%@xPI*{MW{1Jg=by^-hvTQuJu^#IhSbZ z97F8|t+UOeVv0Y@9~rcdi{=bi49oc+5*|dLSX=M-@`5UU^>#^^(+ObzO87g6ihh55 zhb}q~U#f4J7Qu0$-M8{P8~PhFmfJvjL-b2%PfhN zlJyOfpvI=t*d9=`fCzR@9M5ur+NlzJ_veTkFWFrHP z9HThaL0|3?ykqDfr1FJOw0vGXNnI$oymx(sv-!&p_@9~BcBfieck|y>SJNuY_Zd~1 zs#EX|(95#f!^ZH!o^F5Y5*j&OvmrY#>8y&u$@8pTq!tNsyBR&5oA9TNd4*H2X$BQ}y1uOe7%}JD zk;}fX1E!V*37%%Qy96zbmI#M~g2QjK?E%)3nzy&Stk2d}H8SZDY@hZ8Kz13~xjWJO zTCbmO${uN%EJ;61OwbuF;C4A?1f0oti&TFIlsN0S{Q&W1$s`J&laS%2>n7Cv_)vFL zgqI8Us<^VRKwuKu+h>#yx4xhnIqd-o<#5kc*l@*Ltpr~r@2 zu31u4M{Zt&_oY1e#ESn_(1~ez=pUQ0jx=??CAPGkZj;fM4KH$$@|-Vh%wKuZ>BO9q zX?;jNFXk|1w~#n7Mf(g0WN5n3^sA6|AuZBZfqD~E{8<#Sfvc}<9zJ&}voH6ieU`j@ zjWVjctYAZo;YlUli|D8naW>FTGsCpm9MMoU^VpI7;*Hl+c1a^o!lh0n6GsOc;fH<4 zbVH~U49+>tFpN?AyKU81k548SYdi0j&u(2}$r#gD8TcpFtlE&QZXw40)jB3u)@7dC z6-^rR|6cw!7;-ql@Zv4>;bmqOLaasdR|4PM>9b;O4LZ`o&K+t8_mRfOcKb7j53YYa z&qecGxgqjcbhYg*YjbFHc?O6PsV3 zEsA8Lioiv%7o6rf;CIxKMc)i&B%~^3FKFikX0jls)}kN|W}mmbgZ_D+#=N};=J%e# zJXL=Evv4tl+KE&HCB%QSuGtHU~XxWpjdD?3DoWX!-T*eSw<85GuzC zvASJsE5}StpJmtI6)Pie9%3|bFm^4)wd7+$g^T}z!1kx3Jkt@!y$gi~vaJ4~mtHE@ z-pcSIRNc)Z&3OVQ_6IK|{Q4$KC@CDCZYi%|qb~oDdmRvO`)Pn2c^t-^;Q8_lFS6cfja)2nvR`nn{pr+XD-9qd)x^a7fLG*c6{|2$QgXrRv zAfR|ff2R|f5mMLLcbxt%63_^n28SY(pJZ45=tNpWB7q?abgOFVNiOo=G%!ZE7OAbN zt}t;54f=g6uLFZ3YynV2RX4cxeXYsdtL`#_npk_C6&47l;;%r6u2 zNvkPPC$^R>&t%4t#`e@r%BDKW^Fp_6I?Pv(%}PhijZS#s`cZDgHNta5O~8EKm(r~P z`F#n;1N#lNz_^gaD>yHSA8&e5Cg4`B2O#(Sfi$GfK>t(FjJj8cU=SA1QcBM>#V*{5 zvXDxOi-84nqOFYWzxxH4eJ00_AeSrj^X-^bG*HDX8{I}FLcc3=FyTQYmw2PL76s>W zv$fMUdV$v3z68Yveg$8&e8hkbdA?*#sFcxhM| zm%HzCsGu|B)8+KQcIu^51~81OX-8;igfpP;iU-F%#0Wc121Tq`r>Y0_GYxlN8K>L+ z_B&$7p9@gujG6ZRrT+V~M}D^EI!x;}Wf46!MmJ+OeY}Y7C(M%b$xaVu$B<=>Ek!_S z%O)iBWaHCDp=J+_hcg?UuYIp_G`$XR1I~>MfpIGLP)&JAGb!$$lX@QAmY$(JRlD^dx)o6gB?#Bu9nX_+0?* zV%W8kBH&0>#>pjoR&051yZY+hkC)S?Auq^>5ym!rgFWX9qi)P!!y6F#R;rSM%NdiW zzi<1Q#7CEQ`C@U?4ZbrmAQ^QVU#Nm7H40BkoF}@Yaht?MNANp69#eT_- zI?riNdU>Z$-7ezmYt^?){hNg&&*E~g+xiO|@GQ0p+3Y;IlK4=pVjAysI zt=xE953R%nwY4K2$*V*wO~-al&lBsuTX4#r+q#W^Q^ZHO8A%ONd~ks^-JwkUWrfvE ze=+1ykx6Aca!O1&ylU4g1|HE_KPR{vsY=)HtksdG#vZba!7ZW(5yh3-sGorN*%uM? zG;H{A`ocJZ7Fjxql$9jaoWkcIq<$Q*G|(A>T=4RC`GbiO7ip!jY*boss*%Z*B=B>? zlmmt8NKZ&VIc?WE4N^V`DcSRKN?)yC(Zi94vlY6B4jczK9I0x+FWPm;uC8WM+L0`h zYE9$reg|Od{*R{f4yW?}|9FFhP)4#k4KpDtBIi)qAv=*18KEd6#BoW%Q=<395^P`=gtxZK4O239GiXdBE=IlI{lFa zjDHjeU-tFdc{FLKM!ZXUr=VksYJ90h|6ldd_D_XU#1yG>N6J4uX+JmYmk~LT>7xZO zB?T2=5DyTRF9opqeI!n4B8mxXq<22$hVCAIRFkKA#}Q2Y9?fZdmP6 zr8-xxHISrIP5Sy3$k+i~-pIg)u{pyA5gNcrfXZYz0THUZ z2H_g^OSU`4pI?0W1l>Pc=pp~G{<771oB8IebMxz_9VMTiyg%ljAAOEGFrECYO`+0EeTKySOMwa!!%H z8NR;8o7F8fcR_X6{Czogph?DB;Pb^(oRTe{8(kZO)`XF2KiM$%Z>YhKIQd@HuOP+8 z*nj&fgE#lelG_3Kj6Ia}q&%k1rw=*6sdCg=Mv|JTNsksbRr_x*O7E8e&7e#MCqAE~ z$(_W2XaDJxHC_$&?9%K8BiKl|m%boo3O`O*P8g{Si6+-i9mFm0Z7A424s7h89M@S+)2^ zD={(wOE2Ejo1PsD2*y4a3RC@lig$NWqpjTu!x&_%pdEd~T7 z#~Bvc7QJ<;#(ANc?+rd;@lqL;8dKm||3i!ipqxEvQvNjp|bAw#2(KfqT7+I6zI- z7;ca`;Unp)MgO+4^HdL7KD|Rdfg5+jttHZpxrqjshbK_tASE1nE^hw&bB7@tXJMLi(xu2EyJhhJi8@npsxEdOp?jmRxrWY_{m(`k z-D9cwo`7r3icBBVDnLH+=1&8Sm1?t>tp;va`G%Q)~7#r%pff z=m)E8u<*73*IZ^$Damc$oR4N%$5n6mCefZ+^v~Y&x7WZaF0Mx%_(OalThXmM?}5_;(WIC6gi7eVcf}v{Z&~nR##Oy zTQa5m{>G3!jnDpK9nq2dBfkeC#wEG3WZP-FS~53An~&0;=T85YO>cNgLPIbnx2)sh z#E&O&Uj&bKbjZS;N1IkZ?z=HEVDHF!vaSc8@jq!KDW?6M8PNjVoai3;ZAtP!{AEZw zZ;y&;^UBz8@(X!BA=kV{Z>FJtIYH@clY zO&pGk3DQe$%$i5Ne3LX!ROl9f6`o~b$IaZ3#RY<%0h8Cc^1aJhI~y78u4||bS(3?l zi);x7)?b~oe0=^x-muE26Z%d=OnowlD7m%S3s9OaDOv465I0Ll+?^vl;u(&NJQ zt!4QG-D*?OS`uJN2)u1fIB#I9VqM`o@HzK7SYVo^i*)uc1U(#~yCp+14EA36l5Sr8 z4D0Y2CH88o-j$Irs_zrhC{FI%}O-Y4ix5eh5N7p&Z z@BE_vY5L8B1vqxr%df=N*!O&m+Gq!^{tA&Z^tcOagysj8r0tONZBCXr95152|KHJ# z;DGK^GHp)>37+TcyMBB-7i5>cew9{r;`&L?EmLi0k95C)3xeGu__zGOj&F!7;2van zGi~xzFY%w6QNhP{>6UF-qO>Xkjsf$Z6do!Q_XK>We}CfwijN6UPo_lW>1eH}N7-Jr z`rw{SVB{x%xkMEgT5ZRFY`p!|)?yC9IAvP$BB(;x?E9YB@<0pVD;!uOs%-=4s+~!Rr~;O61SH` zFkkbl=D}_(>|Y)#G?>$sIJi32JSy&3piC$^8@p(0kD4BQFw&GAO`n(aQWmnFVsd+~ z3$P0(&R#HGyZedzXZErcHO z3S&SG#f34r);cg2u2^@Yz!<3jnPrrn_m4-)JHz&1VbNvQP~%GEkLVs)$jwY1@if#C zvb*Zv3)Nav6_|YQu;}+tYe%_-bdSu#I3mjLl6gx2;u`Dc%!}>e8(sTE7lXai_x$wP zOB%~FpH1|}Szi@4xF?`OV?wPxHuw7^j$YH8mtw?q3!Cqg+8cfcD=pkDxIJog-Bc#= z7cd)_ofsRzN(bq-i0JiPJn{9?)wt=`i|1FiKAa9%lg)-=k5X!GY}g9m_HJFo*<&XY z_om2|??(w7&mOSN^r7to4Z2GnU?aEQe{@LMs_2t3KTv(Ax!xAEX9VAx%ji(nt+Ftz zS(VmpHswpNgp_o!HmC~Z;u}?3OMehqmsrvfFxaV6`Dmw{v~BK;MLR?AE@I1nLwWhUx1mLvcEa_;7j7#IM4)l|1slm>#H92AxpdRqF*1l6 zCb++4;kaCLHwmsAzBvWRM@RQPzhBk{*U%Cc)QPCkeaC_9_eWnC)5k4_#pzQE1=-%| z&-_>#BTaiB62)`m)cWHpXLc!cMQo)!yuaQ|3Xr;7Z_U>CyO|Lmy=*IMX7T6PaXHPv zz(=Asc@o&$b&3cE66Vx`lRBo?1RTYGt6A@Oh~4?tFx63Ff*hJ9WA$zVNkNSPJK0jrxTe__oi4(C9k7*u1w#WJ` z_)61`SZ=Dn@?<=t{<#Dleh%~XPbaXnPbb)n2Jc@NUuiChpRHf}6aK>Gj_;^c*m=iH zv64CSjs+4!h~4~ss@S9M_=gDTo$}l=VLe1(oH%_;QGU|b(Rf7;&X4nWBjJn>Tu?R5 zMp8X7y5yqrQA$pGN{5mr_H#WGD^SOHi3#tlM>74X@6XN(o`2%Bm3KUXRjgbAV@5jP zEAsm0WX;(-*J?*@P%{lW4gABag7_=IQ*s!1hj5ptvVhsq`9=_Yez|Ll@-KUKApyzO z3!FsxUezJsOR>dl0;Vo_>z3iXs;k`Jp7$BN_Ou@vSq8&dlAw@vf42N>#uZs;ZP|#Fv0Uazu4}d1J;Yr@sheQUg z-@-^pl4SZRQnJZP@iwjokg0J*z3kC|Zc-2BQLxf|=!Ie{;ULc-k11YGIz2}pk;6s@ zy>L6KbwtwYSp-t`YoM_P)*uCTk0-C+2H7fqkwpk*_y2xHD8lVk1b^!(q>L%!X)}%} zNK@NE18qh%kTxQVZw6WWQ&MJjdL3^EGanzL^n1@xXI5@Q*)5M3uyuk??u|4()M29m zaP6q36Yu?Z*Rr3MB9*Ruftq<@Mrf zhmEg82B|Pxx84rpAbV z%nZy%@7uNGH+f)~INucFAmQ}Emlz8)D5Z7*Kz!!;e| zY`;-H&E-IvB<)Y*_A|Hr`*G6a*OFJKgYWKjie-J-ah#zG9ZUJv@#Ax8*zVVr#>FwW zBZ(JAL3ZgSKi-xc`6d_lgBV1E`nmVl^mPj{NIuq@hMGh*%TZ0B{@Zg?;}vnSeMY z3-X^z!K;cG%ZFb)ZUi=tQTO_Bqs6gpkLdkA;%NOd?KMqDEd6iD#;s`Kfvnw4+{W@@ z0G?>g`?5aMNa{iCi>>+^sa8FFf|1cn(in+S;#!?@MSA1KJ$@QdgVMC6R|1?DNc|=b z&ivbg3Q~X#3KohT1pXoNF`;lIQ?CSKR#35Dq4MPV{EKezS(f4u)mz{uK|?+0y?jJw z%I^3V6ppxzfAf!se$SfymU|5w7k{mQj<#Yz9ghf@w}> z$@dQ<03Ep<_<@`^Z!CQE=i2Q`mW&^Yv@L}Zl20#*5{}@W`%XCA75QTtesj}t_`@A% zDy_>iLE9nN^2q7DYkvJH8xr>ezJ+aBH*#@J)7nb)QA8~cIO0eP8F}TTfqor{>wnYX z-MF8)nlqf`-j1B29^LA%$&8S+?-wis#9#Cvu}6Ek@W1~$VG(~3z*69csTiO&Ep(#K zf=ejd;_^Y9tn?Yy?YbjOUPn08L5i?!-0v$H%k4cLS4B9i?FZJB-8yJt(!1~%JyVf4 zFqCX0Qf198`F4j@Y08fKPJmbCflwsju$RGA-_AYejeg?i3nWw1OQqO_m3|SwjSic< zd$Oa?1nv+L!dK}}ex0HBl1QAbDu^ zBWl0GZBLRZK8jUI|LU3hKamw>H?g`jwSqKO-V*y}QWCneW+8-VGE?zb^=* z6ATmc_RLh4tBNwOe>nD*i_Rgt*X8(9>A!O=<E=OZio4xAednxd{dBbKf>S&kyH#N}Vvv=bP1`pK)#K#0!!T~8tABSIwmV#sP zgC)}ww@TZG41C@UZ*ZOh!m`=jectgeT3jOCwOgp}6Jv>1TR6EV$BMeu`eJ*&z?z(4 z{>LKMN=Ly9uueX_U@&FKE*!dZ!+SHZnF;T=)MLGupNK1URoC*G*R_2N-g9oTY4ou> z&6<8udDU*!b|$TsGiC?%s?{+fe=_K6K&yAjR5U+jK=Jklj{#;w>K31+*h4Y7u*PkQ zrIhcO#J1ismMh2oocq_!&f@)sjAzKg%6Z)kP=3|Na&N*d)GqsVq9~~C`;KRB3cnk2 znoZ+PJn6JTGjhdODxkA(MB9{64}j+$IJ$3s-IYKwWX8M7OpI6N*ytZcrQj7vmiJoY ze$r`I1Ea3L+g2+()c7M`AD>IDxgjg8;ae1-5HgxycCzk~tEQFwv-mpFr9eudZcF)e5+0QTGO?1Y} zcufkt=opzvl`eSRYqJRO6Bp}m7+m$DOpd)f#voxcj}|QgRKKA8hWOGFvZbge5%kdvokGT8H;-1IAGbN@{0+#*U-sJmshR7?d<;0#!p3Ef`y>S7>b-ceX58-6_ZF+Z4Y-15u`S@Fy(5X0&WmC}f66TCE^sZke;(h%AJd36c>HX|lx|HOkE&q3*ES5&4cbf{_zZ&D?a64o|x3}QbbFIyPNj_ezn$=)m0=(v+T zGnGhp8-9MkVht^@yJ{YgS&#}!PN0{g#tgSzDdOZl@eYA6|7TXa#uhEO(+j+Duh7*C%JIEh zz_~N_T)PDre?dxAH_r=1-BZe{Dh6%7#M@CzkjI1XD4KjX+)Xt8CLL&ZS;FG2A!Rt@ z^ad^v$NhOm>HafCeL@IAUhj%+JtkHlD<@k0fY)8G_`%C8T?P8LP$AMQygsacSIWb( z;5Vwz^_bnGK7Z}L$)uMRKXI)ohXr@;7Fvlu{h7BqtJ+)d$|x3XT;)c|>wH-f8`0d} zXw1N1dQNtO0k*e~Jwr2^&jsS{8!=pz#O}FWvU_r%H79b;!eUWnPwM;Pd4&U4OP{AY z72j1&Tfr6=Ua5mPVe% zPdo8Bs!LNBfXp?JuuuYie*Ne9SqZ790qpK0S89WZ4$!CY@KQhU{$s>TYO2ET!Hda@ zUr>Rc!k?H`mH(9n*Qqji6)?xvi)^Vw%bR?Jf?{UG;X(@!=aW69t!RIZj{X zDb5$<7;hQ2(On8ptowfFuHY+YC4%P1+1#zx+Y^tZPHNPMzGuGYQ@k_cId2+yQJ$D+ zIhpvg6lDaxrSIfr<_GOj8gD8O==1V97iLD9*bC*v!4u6efvie;u#Xr=PLal1siB?O#@M$7| z7tmcNS(gx2a_;!3J6f6wH<&MEj4j3wY*zgHAmOt3IusCS$L*E<%fs|c?1dVO_wHWr z94f?}bf_`t*5mEzxB2*MQ07r9FO)Xp<2*6h^0l7CtOWLxPZj{#oIS*c?Oo3r4oX<@B@!Cl-^R^@_glEu=z#gnV0KU z%BOESujF!?Pp#ICoSoisMW4N&Oj@(&z>jW{;xN3)R?7&`rHhV#16QOevDH?-BXRos z?C&k*J+YlusVcsc>*lYGnQl3URoOrl&=n5Z#duMOg+e%-i(Umnpb+sjTQy5^Giv&_ zJopz3g(eH_=~_zvjb1rnYskvTwuv((uz;LQywS%V%0q|^oCZ8*+V0({#mWJbtbM56 zyAAL|2|~^cx_kLDx|g8nJ#@93{q2w2Gc1%dz?~=po>L=D0OX@*2MVE(ayYOb4pd1M zKrEmpNvhjoXz?)t0V)%v3QkKfOHdmgfrelK!35HNY`KsufHdLw{J{?FP%{xLnjBMP zr4vs-3mR>j%{4c$EFv4^^%H9n zXb(UpSnjmzHQ)u}wJA3t_xi5nUTUMhfVOe%mE3nncxP+(3yWWa#=XFVgbOcgDzx- z+Bt*@a3Jwct<0tVZ_{4xu(W@{MWMock+Y|t7$rZmeo^T*`94m*NN~5LpzZ2@$wASP zM~XoX)dC2s%820mwm3D=q+Qj8X1Z@Yog#*eP2Q@w-MEZOZ|K5t1vn8`Tr>ZO^6O?| zQjpod%NC?!GER+Cx2HVXI@WkBRV~jkT`8h{e!Cd^sbh6sJh_xjUj>yI2ZpNLb+2V zJ{v=k{(3<-j-|))D}bN9jm=omUGa4$4sJxaBjrJPgzmYF9>e|1Igr0ZnvF-E#0yWQ z!|0R-%P;<4L$LLPOXdI#cm9LEtHJ)F-~A3)2+GKz$sSdAWU zB-T|Gw0#Tsr+mZr&BF|%q@%XnYY$D@SzG8L?7gh)-l)w6(*k#MJZoEIxV}_fX0O}z z5^v0hr*vmua*1hbrhDBte3$1wpMYrOt1Cki?0)7{-^E9m2d5ahl3#cwbsHZOryrb4 z|Du!48KSGAB$&Dt8!Ig&NlbJP)~Ww-a&^v(^Tba+<+nxg{9sJy>FqR>qcWB%)i~^RY zV{=#8CQBvg6Ds9HLzet%-TsDRI(S1_Cs&4>1m$4&X3&RxQ*TU=>;7PS*Bk<>WH2<%p82aavoU*DOC*oqx|B0-RHox)Qvi>$@~FAV8tBn3TgT?Nf1HRq_Bb&VA(m@=B*?HKc@cseblgr(x0om z&z^M7h1i#j%{R!FF|K%e8~La`G#2SXY6PxgOs^f;4NQ55LhRnlhi=+o4tE^aM>iw^ z)+>KRxRt4iO2oY+6V1%d0A7Yx*$ftl355Ag6v}q-QA&b{V^k9Gn8Pa`YNGX_bQ`&@ z9lLz8I}0F6i9hU&H276QGrE`*1zu(GbO2LRZRIFf@=8tP(*6i;pM5t!+!bRKao1Ub!0Tl(y-NDIzY2^Kku06L{&%%}`aNrF$_x8`nP z_f-t~7P3Q1mTR<6{oF)P!$Y}eC~6m_s-V2`6i&leqmIFB=R4&<@|P{azqbSWwp?;{ z48Y>Nk-5tjh*znZ5?NKJ6U8ZVo=^P^D7I|vxaNcpyzL|n=vY(jkkXu#J8Y`-J2-R| zdm{^mfD&#AW%A%|S>F7g?$c1}!7a~k)4t#|a?a=sj46z_|0C=#xrCDjmluLU62AsQ zuPLm0^ky#Y$0{zO4sQx+w8m6BOCtTB*i$rs9pw%r0+jh%2U`V~uC|D9)QV}|>v0Q= zBqaf?)UZtTi0W2s2bzyhv;ckbWMUG;J8Pq_8~!baBhOy+ydSIr9<7zvl>%&SA|bBA zoyR~SZG$w`!m`c6Pq#E0tXHf)c2Ua^v&lHC;Uw5VOtYM16YKMc*orVVNG; zz^rpEJ{h?5Q~K;TmJ`}TlU}oTLvMa(GS`S@0j=IP6&Z9N&%Ev|WZvSsD!X%eEM6`( zbu$uc|9k;WYBYk||4C0XF$#O<%$S|X{bV>@uyHrd$Z6J`qSN!^gQbd&Yd(eH^h>n?UfbGG<_^p?2hRQYkI{3B38==Vl7@9(Z@?4}N7d)Fc3`B7 zCRg*aL(qPFWfL8quWsJ+$zKyPE6+q}PB_LIzh8^{J9^Hc9O35q8E!n%Jk}~mQf6f@ z;Jzm-G?~ZN1V{fypm-!E#13angX7h|X~b%z2NZeI{ZHHqy8T|qW1`2E8~WEM;V3o**k zN@+`KR;p?DeFp@x!`{y zS3th>F`ap|vDgqj#|@=_A&jl3p!FX@{R1$#zuJK%;gQ#7f^#Y=>rzD@V$m?ptXnBS zjJdM~H)uJe4Gl+pG(R!oxFcd=q_lH}%3BkHWJE9EJq*qF-ziFoO9!HV8WA~l%Dl$O z!!7e>7UAo4$h*)xT#>;X9K{M^Xbtu6w-20>$?GVD{^T0nw7-XLOH6+v{!cHC%@j>V_l6W)lXt8f^{`S#fNwJgwqpl9N^M{j9w+8$kR_zKet2v z3f;iO#4T0>FJ?=w390uz8$~5l=kVv{^=#Hz{}=r!4=RTn!?0gZ#h~tkXHo%}>utQo z>jzD|iRJZdDC?P=fZjeYNnRlR*Vfb(`Z;|8ecJ7=_K=0jP=VA!Nt@>;Ox{U3f=xFI z^^c9BdgHsb*1_Xcm&L)Z(Up}oI)b|HYU3kc+Z1#XoY3a_&$%(!7tk#<f*v+r1uh;MG zMte^=`m>9=yoC=Kg)?W=6qvVFf0HEBiPlTa3dWtNa8y{$k$H6s;*7L1Qr$1hxpQ`V z(Etg@bS9c>3Rj2f#WqUQqvqX%;X*tdwH)N%vS&ym3h+aio3 zB?vdP~R}xWd0GjY5AGfe;`}4{a;-26=UI} z0?IYgHn2-YEQx55tfyG06S%!R&L&bxs&?i7LRx2(lsf-Q+ihz=77N@e3Ie$vA@-ov z#-&c5#|{_qjoUx&sF8$%ZlCKjTy;e6i=Fn{c4j2bY(ipd3+6wlIP2o`OZ|r?Y z#o{h+cgFGL@_>TF8H+o#>RnsgHw0c8tCth|$+5KSKC1~MrV!J4W(x9v9UD5XT9fPa(vO!~1aa`* zvN5bWPblT_3&tR7zt`EcKiOgP7B4Gi+kYR|&w2Wr^X2Nn6-DZff#JW|Pxb7HW_pFI zC&6kkpt~c=6x{40bX(DBh3U#kaVf7EaLRw#7XxSesRajb>At}U=yenFLwojErglLi zJHhxt-?>l5=MPSzPwb1OKP9w?`X6`L>_$x(tetcg{Z*_mpO2Bh8pVEUu?x8bgX&BQ zPDJ219~JQisHm76IDIZh{Kz&+A#G4sn4CNaCFyL>?!uYL-*~zSd5g77>4zHKkPdY$ zrz}4X837efDhxUw=M6{->Qk8N-$5F^JOi!!CjS=&AVVeZAW#VhK4t5i$x%73(L~z* z17rdk*pFat$Dut{J?RYHB)o351vH)dF91g>99F)ZJNiz-JGhhNff0pK*zUM z*!R<^FJSvpcJp`Unl{k#$zIYvpo*!UIuxb8psgWd$BFtB1E1I5el?KXCJ#3TfTMwV zST=9mZaBg}pIWJ=RfcpV)nUNUJ^b_69eTH45Me&Uo5prvt2+k-p8E6Cps`FS2VvcT zUzj`;rJy_TIg_w06$`lTv1v~9XGceO9-?Mt(_Ev`iBa1>{`0SP`Ls9t)2%{Vac#2m)lDqm~ialBmCe{Og#`o=7#>T+vc=eAIhC06lI ziT|m~PK|jRt=u9_nuU_9Q{Rqa%9-%3B`mc}HzPv5(;CV}4umwF7aK;Iy&XvMK8YRE z?k2pl+*Nk`q@M?N85VQba~=JOqjf@O1A+c$j5(z*dv-d;tS=@S_30LrFfSJ;%&B(a z%6H0tqbBC62N58V_17O9Q+j0*K&Zsu_av-bj^%kQmF9CgD(7! z1mt6Apy@>Pea(evlRz9FlY@R{mgp}}UyrgLq~D3E65GCnr|tNH90Q%5d+g$t>ct!A zew7&2-Z#;~@uqK9&sFI{9k}pIeb4ZN2#Uz3KzFgv5$~iTyg<4yA3U;6Af^i?VORJx zN!;&A^{}TEWB`0VoTb(kyNBo63%XXiF)NI_YeF}6#5Dha?be#BBW07Z zGW!%sh+J~@lL2l5G8FBc$32T;y=g^k`7djO<}WC*S@74w#qOdp2CS z!5gXi8U}C5_FiWF*@oTC8eY@X+{!{jwO2!cW9hSl@huKD*<%v_i!=k(#5t|#Juegh z=3$S5d#@d$+udYUv^MZ@Yx#efpV74HF1U#`vqjrE+zX8ge=V)Od^0CP!Hweny;@ag zHE2rrRg8_%gW`{{(=#mWVU-dqt|RPPV}f|VS)a!%25hTzlbY(ACtDxX6@K&2Nm_F= z3VO1DCpdyKdJ6Sn*!fdU&EcZ=`m$;zFxO=UyIP|Qawp*f?Y+rpuu&&5~|Ir}+Gj*4!du4>G>A2uC*2u>PaT zI{$n@SBf6x{u+6*auD7KNCjg2t5FKUv9qLA; zeL|e?zSjZ{%=f@#X|2E;*gwReWfns*TS?zm3OrXmQlNDe-;hRTD6R_kB!y{1#*M2o zV2-hh4$Oy8T!*bXr(``}E`K8a(d0@bC~)v^+4179_QI(iXq+ZT(e|#C#^tZgE>jT% znYkp2g4&rHbVO?~Hf7pveC5A*jvJi)nc9N=i5uwe@521Uh%0d0n9max5SM~2Pa^}* zpH~Gy?6gC>88w#YdbWE6_vighHLIAfSjzfy7KNL{Z#PdalTYwnT)Vn?o$0Dr0cwC_ z{g;ohDBJ=nSrW_vj5e0Z|i>>W#93rg$aL`VavRnPu zK{T++h9sq)m+yrv{7fMUp-sJawu9)|ezpQFH7~8fPsH9xfKiu(x5aKB}^6N2gi)dd}BI z<(o>`{+fCoTzOQ=lk{s!(f+d1p7!K7c3dzvyKp**k6Aa0)oY)&Q)beRGXpyt_z>Es zoV~;h`=F5hSJvuJzFn0c6F_Az-8+BxDb!$g@Ef;D{Bpq3UvA$s_t||ayudts>|HeN zLIZ9^r0uHG%8lm-#*9}|GdvzBee>dk^39j(?V!hNS5k(5++YY82^O6%()8)-+cn7n zU#uFW_8F~dHC%Jd2IQ(Mb!z@Lo#(a>0k}4&QSmBgs2$g?ha$yrVzneUUc|;)BVI&Ro{?DQ>wmV6QBkiA5}IQo zK9#pP`D<#os=!I>XerV2shsT_4W3&vx@EBAriRD(ztfJ=FqJNahdA|uqFs1?8O0y} zlz=Z8AuAAx_eG^=7@Vj@(Pew4Z?-*nvgO{7CJ0tivzhek?_PFzLMJY3eUPg#+avR+ zhF@G-{nKhy3u?R-tM@0%7X`dociY!yIeT+wVo@u%fBu8VH#}-4T|9QKG*n@ zWwA&_#V$MA{N)|Witk=?L1RQHH}cg=>RHsZG8G54Lp9z$(2>VVal`UDj-UW8_n^u1 z8^!epv&}Tt4wrnPvg8f^0ygI}+{s5{qUso{n;&L*-+By2;5TgV|IJ`a_quBNc?xUqw`D7rWk&`%~y>~cd!g&fHJ`h zW}r+2oIUz}V48ftWQDFBD=G_H({dg=d=gffLG~EwK+K!ejF;O5vC0RK*DUVf>W)dWYg@c2VllCd?noXL>xu%kD0wwg4GSp}EJG7C?! zT~h)hr0kgcoekjN>D>JV!vmhe5wcF{lhKR_i*MUEJOoi9-95v9_S=HrT#Fpk{W~6(Z0THu8No)STLa%36DDO47LjBzQ zHedj^dBD<1U?--+v41p^E41*m6u%B(e~ z{HBh~nOqg95rs?i!910$s0qL{OY;NU%eja@Q?7it_3c}lSCcoi&x^~jpJiP%Zv&)& z)*;cbg-wPTSKaJmx*Uw|=eu@Xi`@pY{%x-Kitgc|r+ExCo2Ct!#abf0mJsrjcc_RH>vPJocdsF5y!hAWzXU->4}QOYT4g! zOf`8)_;1sJr#Sd6yP}E?Q%1KF3L{yV1#NB@)4P1i@9BEtR(sU_-?X2!<>8gq86HPh zuMenA7`u^8DtVjgA~GfN=cj6NAojX_N$`~a<-9fD&Quqs6|E(X)ROmCtG#ZAR9)?A zxLIv5dkQ&qCK!<&D@3y5YI=JnrbBc^sY#Aiil2&d!%*0ukGLaL)A|UGR*z8h!&@*p z0)eTl)pFlEY>;=pCN0d(1rw1$wn+^u-v3%>eAQJJ)1yzG=ufMK4#B_Mjq|1@pf3J! zv!-DQx=nY7_j8}knk7|XPx|r7w*1$lg+_ZrUO;@HF=k^hw_tePI}$l8dTjA@PH^E~ z`W)j0QH=wphySh#YlJa1V#0cXDTNCOhM;V~W#-DRuby<(j7o_@Q>U83>X7Yddts1| zlo|ilV=VG>L@T};JJ|XD_Uj#nY!Zjcf(RzTtv4|oz0S3(@89!yjDbD`SpjSfLq7iB zR6}1om(AlifCIG5^YPr%e;wQ{biR+A0-oZoz*{oftnMSO1)L}{LL7>x%@n{)=0pMaZTf8qU z)uBG7k2n$VPdhXPdC5QT!37=7-|OKmHcT}I$bfCtnDvJLNSsIC2{CCJZ0j@+$^EK1 z^jfh?_!IO0GVs6h0kg*aBNQWyotcK7+Tq8h23crJJgFvSE=H@SX8lQ} z#cZ`YP+(x7;Lg2EaCV}86)!Dje?M9yldd8>?+WeLJ>>KNL1A)?Dd$umuvB5o0d?GF zf4(|2JTa9rdX~r076di%)5J=_f{ZtK8P7If9d3;VWLVp?cY^51tXt zXu1Qqk*x8|wsUZji;b$7zr}pqrQqf&E&FJ@Fj@B|_xLg0Y*DT{56E4;+_FGh5Hn6&U&@9l%1F z_=27|?&yyo)PR#3^YU?}5iK6%O+c zal}`^gsQvG8hZD8mXL894M9m$&ru`Z)fQIjCVzw?asG~>Q7#TmSyv+XaTm*Y)0`d_I@3eOuvmzD22zCk0Osx}cyt(NH>}RNg zLWEhID^}dUBHPG1%i(WGt?~WVktR~@B(uSG*xx+hWo#2^MK(0!Cyx$xQ=n+4AaZ=N z8>tw0kyB=5I(D+eR?=MF*;(sRme%{zoKt!U?( zLFmJQ0L@^nI^AF__Q^gE+0XIRezZc86g}^cW%Og{UY30wxFbdDvMHXO`c@Zrfa|~u zmPsUUZu|kZ7@i4G@2c%eaq%DIonX_3s=TPJF$rJ_T;l6V8J#N7ub=eZW8;16^oygS z3M+bR(D(EmrK!zM%D#$Q8dFrm1W)s6?KSwW>7)c!}6^hB~{6AojdeHQ2D=1k4e z`>UGBb;Ia4i<)kY`_r_Qi^>uM9WpibQKv4bw{s~3y zeTP3N6zV}BeRmgM)cid->IXphWJr39J)$}yDLi%Oe(ii$M(g z2jg>ZFa3qcf3f6g$ADQWC_xjo9E02GK->AF6kolUlA@zpLYZz$415=9A2b8no?i5L zxxTo9Iy{O7s0&!~un4ek4vCd51zu7<^&Qja1~0Z_t~5$#0M&k`!#2(B{g=_)s4#RF zbpo44e>d~*ROe{f)3YoK0sCrPZm09l#?>i>=cSYhIA}kqqD!BiKdzE$Nq76+r3Jg_ zes;5(06)ise+R6InaMunQ|&}u8{+to)QPFU8N6!z?!V*gI96#H0}O5Oc7#m@^TZCH0Fli z)c*B2FpG!mF}J|y;7p#9WTNxX%~gE!!0qk@Mc4!-kdt&mTbt~R@3xnk=BtMzv08pz zh0b(>JhYT>*~fJrwrtO(oMN7IpcLmyewvU%kazgvyX7LBP2hpz<$I<4iicr|o23Am3E@ng2z8{_)rSF?r{pQcEz#dlpv?gjf-^GK{9x2@X^pS`E zS1f&hl7}Nnn_-=m&UD%P*tW8Flu9CsysDW;*MqHa{8vw~QLyrBV~Kt*{2>K_h(@(d3`jwfZ6Xn?EvfBam@UFB=zM5 z1%|_-u~omphoRYXfM$wo-g3}ar^ceAyr+#3 z5?0vnf7H^k-`_Cd$;c{}q0z=qvJ)U#{rVBbFo@XU3LqKV?b{9-j+SC4YnUF*d4oZ6 z;H62<$g`rG#~-ZSAWBhNFW+r3`#%7nKwrQ6HgFNGKChL1U7v10NnJd{xE0m-Cp*_= z$?Bhto>s&ze7R~1oBXW^?Yr9PptN4E8NZu^MeM>)l{BkOeb)7d_)+cK$QOS2%=*CT zq3^$~3h2}MZ`uE*K4nP5{x{mU?Br6~x9cDOYUa`GeDPmhO(}Q!FIxp!tM$gS3(i4? zhqC`Z>!zp3RKK}ie{J?}xDfph z7U@3^%rjT^QLFh?6_f2JWd3lZ{|ZSdEpPDa`XTPW<)>6@8Gon#xzB%)CHO{|6XG z^9_%A{I@-x{fq0bzm8AXf8-+{dGt3t`ks5fc>BF~@U%S5edf$PpTFyFioXB8`|r5z z&I=cBXZgrn2Ls?!`j?-&Z2#oRN8Nbi|N6iDy?_3H{nJl>_Oq01?8UtsAO3J2Bzgb) z-+$)Jne3pAIWv=i3n$rUx%Zwg-t)!JeCkupc+?ACF|h)(P`|3B9LOXo!u>FG2wj;Vh`#G}?n$B(}!W3mjW?Y(p`DS~Z}n*Hl|DT}tD zKjnd_be(Cm8r{=V*5cp&0dspayU$&mqHUdYbhmSK7vid?PA;|~Qc!kuC=bK1##Y>& z4lmphxciIu{?{-0vAaI=SseAe&Cai=_R!LFFWD+4>UT& zLMZgY1$IOD0#h+gsiM+i~>%Ve#KDfBaqi*DkzuCyr6e_=|sy{y7(Q zXa9{pj#N&_zKY}T`3vXnz2_`%ZpYJoyMnTLQ}C**ukl@RP1XjZ%aYSqWBlDa@%b;@ z&3pK%tOYVWAf~uYSkc;PYkv5ZjKBA>$Ub|Ik49S(HKlktWIoRN`qZ`8Tm=*3Jc#;ZKK@>F z&2{Jb@Clz)IXH3hUcG)w(_b-0r-+%O59`~oNzUu1#pS|}0u%bHK$8W!P@Als5urz5Z zRyrCHHP~Y;iJD@*XpE+6KJ_yuCec`8FUc?V4i*FyLBt43RY5xJE^T}7d+Yys-gjp9 zZUGBwV*bp++%u;<=dI_QIdkSrurq+wD9z1H4UG-KU??1nv@|z$h68OatxzrKIXr%b zPNm!O4^$ca@<>Yz>GCuoVy7~ESE8WSJFGNur)mEE zqrF8q|ACj1L(P%a=`*Ik`R1Z=W5=ovP>E39hkTTNV4xb!12Dms>x0WjvLJs?`HPPU z%ftj4@bPOQBNB%+Niv0_Oqw)EW=2u|C?oi0ey&4?evzj5Qh~sI2K|BxlNiAl`cXJm zzlsmTIEej_FoQtTepF->GZn%glQJ^EYoiI~+6{ADW!a}>=Tx0G(~?~Qdg6Dp{}}r` z5WYzTYXH8IY2nA(Kj%hz`4{63pkElaMZfBobqF+tOgn{~4obgd?*m`ta}fFWfiJeA zHf$2X>Xp-2{eFu7Y5_U-uuqi_Qw%1JP!IYQADLq1BX}%k5Y%;T%IDMQM+vo!9`;Fv z!b?owK^{upv`>th57xd3IX&nn{x{Hn;lsE?fTQ)04`m+H{-}J^FrbUx_KEz5{kISQ zQj_Y{K2Th}+lQ~tf6>F?b}<@64}h({ac>F@qB)JYTzI)$cB0X?mwxtd@LyCO!TaQp z2%8*P#1^tSy@zAd~$U$c`O99QI#`tIziBzs7$zZ{G6fTmOuw+^`IADkHln zZu$Q2rkjpFW+EE#ue$%my9q!2$(4Ad?{GWWam&Lsi2c-4PyL^t{uB=L+2!9y`|xS~ zm+JSie?I>D{P}olCX*!cv<=b#InwI-eg1#^j_p`-j(Q+} zyfmRLna;D|J&Wg`f8LGpw{(bl;yGsgVA9jGe_^}$4eWpMq$DXh)$Mc^<`-~Ul*0xa zW{l4?Pe@8kM4Q;q z&``F2e`Q5gZEbZR!W}@|9yi-i_zC7s-0lT$&VS~ar}3T()Z%hGG3Akvl#&!5k2h$5 zmzST5^(KixOHNWVhQ8PbSmU2Q`>($A+hcq#AOIp3$Wa*Mng-G}zWS>Dmz4b5`(Jt` zod|M>?Y~skl#z-?&R?Sc3SNvVx~{wA4-@;E{g>K+N~B3asH^=vgz>YBD71MQ#{gSx zjspaPJ=mF8A!=8Nu=V8fD2_4^BtaY%!EEM-ts4Xk^~sMsH6CzU(!!`cE_iB5XXZ8kYBb+3Icpt;9#&O zS(wCZAVus40Bom3KC$?*_9Jx=gj}*L52au6iDp#aMZaaALYbO_QcIE%DeO~r!US{i zPcCqeieJ=)0*r{kC-FEC6J!5E86J^{$H5k2^eYnk6!`;<2WjEKFitWiLhaIn0n8XU#Y&lqG7@yJa=ANW+i;42Q!BET=xe<=ZBLW!gd z#5fGv$#)n1YM&|}mA}D&{KU#f&t?7yOV=z+#1YUt*_i=TefuA2`V z3hZV-YF3B+*AN^<*-bv5;=d@yZilXolez{q@s;^+TXj{#_AQM&Hnx?Qb=KE*G*<`P z+8p8bs5>6M;Es>?`4i()QscAJz4-%uIfcm?nJ$kni0N0CJK~LVI$cqhk58)kVmca7 zaXr$H|6TNdVgGH}Q(C`kTl3b<&6WFGYs*8e&C$+wn=N2-`f*nfmzo@xn30&1mN2Bq zm)$QhJqs1FqsOP#;`|P;-{y4LU2bDxr3h6_`e6Mp7JdFB-{Xo$lkDK8pz<&rlRbD; zjn>rWz?bRqoFnSX%t{_S0?Aq9k8^t6E{JK+D15dw_t@#4#3Z$UC=p^%CH8KAddmkf zhx=bzI~$gmqr3fU`+uCvJ?n9^8UuFM`5k+U917q4y zfZN`aA3GLPxVcnN#~6sxhLQu!KH7&v?|*R>cE-r5RjOmqC)g?Q3R@c;E3lC@r(+g<1@OZVQr3GTddo2S74S-DW z=++w-_xfw|VbVoK!;Uy&R9;Sw)9u-_drwJeY4NUI9i8prDDGl9JYG*I#4#bz%FD~& ze)rwF$|?+0@xdKLe8qX8d`!aQeR(_lA4I! zOIII9}gcj9Mz!I;qi0q_>-Py-sGx}Q-hEa zhvJ9F&+H9hHQK%*#A~@pVeEnB!_F8R5138mmvaB>qFTi28DR6rXW(d^_=kf{G-D9d z+e3!rL7z0;jFw#bt7VW=%aR>}SwjzkNPeJ83kZCcWE>JCKkCE|Ju;4xdXdKN19_Y- z&&@aAvcJ3nQ=hC9u)4P>?rgaVgPolCJswKo77{KQx#Yt0e{t5JD{emKpvGkjKA-{0I3Y0_U zM=H)e>=P|GB1ONf$&?TCOJT!)^nzyr>5>mFNr4GYmB30Q-JlFVnu$=Be5e%`IvBp- z@}!3)pKkc9%8+6#z7!FD86wk?Q3#_vjX ze;g18f_>{(+2>i*!0bkp%1I)q%!9T+GSk7Lhy52)=xslc%;clTQ9y_NH%2~(?!RsO zch~RS9%*avr{^RT7W$G?B2J&v>thQ*?=|fIOYz^fn(93ZU)uezH}=eZ-R*ZeQ=$%c zBDjHC^{GGFFlj~MYn;8{6f9oq>S&3yg+l>H{D8ddF=Mid#-xrOo7TTyXCUHBOtQx% zIpPys{#0$20EOd=`EOlmNyU=ab}V|X`rS1yzuleew7V18)3Kvt?}28_0LSa3?D&G= zww6dIJNc~RC*_PBn>pr)^r1r?ZhzE+`3|2aCB^3UW7Y$=ydX+i5g|_~)b>Z@%f8qi z4*2s@Jbt&+=VvBuPs|WHUVO|T(9xDY_9%EqsH5GVo{44ZzQSSH?=Kn-IQ{Xl_*xF< zF8rAGr?-FgXn&}EbNnMvgh8Y>_9_Lf!fOA`mW$E7@r8X7_+Q-r;?558p@>wqlmzcI zbjYTfU|fuR4&8q(`5T&66BSF9=Sb8Q78+qFVb~kZP)zZl;~$h#C(_Xp?K8%H`t&cY znRf;v0M?d)T)zVQ4DSY?R{>8B!bni>y7 z{vb-9=V)wi`g_|ZKQ(aCsxW#2YsCjJ zjfFoPKBu#yvb?Ok%)VqPM3{%Zc4j@idu6h@?7Ef}s~x2<1<`Q@yG$#K3+G|B6JvbHR68+giXN z@)r~o-gp1~R{KV+un-OQi6{>l6C22m$kOG!b*|rkw+iN$>H{> zR{Pf5e>>V+FSy{lYzm?_f4m>-iI5)?+26hBBFrT@X-`IjAoX&<2eLsG@bwNQDJkjc zr=M2(ksIHv`Pr3MK|7o})O@gImut{~fy0Ik9m@Yh3kwR6W>}fzvg81DTy)_@ZLO`s zC8E3JCbzFFB+GPARTQgco_g|0h3X+6jjXG$z4X#c31uFSKmK@HS{koJ6B=!(fRGX^ zlzkJ&&z)P=kR#Af7O1g9_g{Q^){q}h)qM zHZ0^oM}l_Npb(ic5fJ8QlYo?(lrU`Aur+H}&0n~1!=_J;9zSm8jG5!dkCR@5pCUvp z$f+-RqH44exBl@@*Z%w$^Im=R_ST)T^uQM`saZ?E z$R-3JvP(V!$rvP2_o_%k23_)zf~bLqfur^#_?8(eJ}xPLSu6o$J&2FOJhN3o7(pcA ztbiJp8hBFriOjsrMj;6Xb0RGEsgQ_az@p#8ClDc+Y8?hhtbfH2gS^OZ$zMwu2+%cX zHvJ2cI8?uiiOdKf5&@-OssJ_t zh^FQy4LrgT7yrflH`v}@^71pQZu}XZM|-j|BEe|#;AF4Ip5Sx({myt#)aiA&(P%jF z+!*BXDJ4E2(9#j?=!k1+b#;avfk32g>*mJon>Ib$*x6vqpM2zy$um<&j0z>Cy8MYY zf08pL)18pc?a1Nz(lWAXkoL#we@j=qi%v#G2lX zLn<5&d{*P7xpRMV#TAgqm(u^PUAuNyaWSxH{}?x-9;5$-TxIYi>xDrJFWKkv`0ww3 z?Y8*%1c<`ra^lv#rKK5yM32 z(M;~%Q(9F~RbEk!sTBE^0`ErP6m6Z&3zsj5OLXVuWjDA|T3cP&!;+J-J*{@UE7O>s zoib|N&}f?{B{9wI_SV(Z6&IIOl~?X1CO((TRy5XSlnZVAMYlYuf8iJ7E8JndH1|a- z31z=?{&yx%nXEBj$}rJrbk(ZW3l}azJZ|6-?~9B3;DZnF8C|I``pPS>Tyez}DhkCn z`bQ)S2+nW5xp=|ruOk)5vBw-MzcvW`r59e%NW^Gnh!@&1?*c)Tq}#DkYXiqc}++%$Uipr=Fze+Q6Mk9H7`S<+oZZ`AQ?|a|l-5Dui<8yfYeDMBPR(4R}bTCS_ zYrcYwlgQbKhkfvndjl0Nw8!Z&o9$m~KBNk4(P_sXjR1<#)J3l>A_q-DjY@HtNh**Z zCP^s(5vD5) z;OFlye|Oox-d(=#Q0;Nx#7wl1RO> zXBwz^CEbE>tv)4X`jEfM2c*$g};N+R^gp|rvZ`PHRR9}8VVc8;jTT%w-2qr-AzB}G7!5-q6eE|8k z1iv#e&YqR(0h?}DI2f`wwMMIIlG@{Kb=!8W-Lci}jGy_J+rmw?Hcy-@5OBt)aiBo6 zp@~p-WRwBQFdE(TkGrCgKt})mf#w$6rpkuoA|LoNh;TZq*R2XxMQv_RI2i1A`bC{( zJ3_53Znq~A?u^gKB~}dnVgI#c8{2-;e31_-vgD5lxh)_A|NrTKaeomChA?xnXZP-1yLRT}I#F(C-SddX=<;v zB?j^fbCctfa9@>_kdlx+!sZEjoQ`+idAF&(GXJR5n(~?zYqn#WuCpQ7(HQhN;)e_w zHvXvb=bn4E)8Xc~TA)W76@D3rwSQ4Q6&>qe2#etx`N8;~|MKTalP4h?9Yg$!Wq4s6 zQiKs_pLO=#f4f@~IN9-IPSW9s#g}s8a#7p7mtUr-Llh4CdFP+UtRPkXBgh1~KoNX> zg4G#y(rI{zpWtlaqbFehfE2M~hR`99eZ^uBt3S>;_q@|jKkeRo?kViok15PR*7(n= z1OY@F%@S*qoD3kutnxS$zXuvW&*QQe$!^k*g)o<1dZ|X@{CWMCyv5vqfo#NB^do>n z_&4P9U*Nx8lO62XMTw9&)oQV-%+i*rMG6H$5eAm@sA^BT^+c<&_?v2l?nEG4cT~7$*twWDf)&odK|gC2lgcgiDo!WTRBH*<#3)hS1Jy_UED` zRwJD|@F_u4lJZL;@Mqy8U27KxMgHW#IFz1P`7=S1k!0x?HOqlGl_k%XTOyKN2$v`- z5&}^yB>5#MR|csltF+2HQ$jCuY8-(0Mx4mUvLEIL?KC|elurykDF!|)LWs~zs2%hJ z!59gV0V!t^s`e>(Vt?o|kR<mtn@PrxznnYDRR!kz($e* zWZ@fVT{W6jD*;3UE@4@;yZs2Xz#oGDTKKG`yL@y!#Cq`z^K<&I+JCJ4$%k|&2msmA zFYATfP{-t*+-iXQiZnr%Az4z0KOC|6B2+1@X$-2QnUe=l)pA4=Nb4d0!~RQQANJp` z(tn>Bmj=5}Oe{EK@oV9hChQgmn;d%5$u56F$$QIJ-gxbb+isi^=gU0S=E)z1oifzl z#d>MZG+@{*f@1$x``p8STlSUAn{u2h8Bgd^QsHT-u&yh+?1rD_$Yy+Fxwqs{dfw;WxWi^o3p4I^Di=+0$TCJo!{EPObE!u za|ev@5yTD&$dr%}O^EkkYrp2!=$7JW!OUr)NC@7IMJYB1KE@$E6WbGM5W2UYNUeEm zi9G>w#oin!5Q0!p6IxS98BbW9o;aI3h&4D>YnHoRuKveQE?M*hzks+Z z0si8Ge~0^D$l`13e>o7u2#kBk%Qs*6z>K=ap{n7Ty$b(}5kN&yl_-w)iOd-jVd>!KC6t{-_drP>^^EN@`Vl^eh>v15*n< zxr-E!K%+n?Fnp#Krjv@(ZwCXlq+5ZcSlHPywyP;lMls z?m{@8=lg@OFndW!>B}#_0?lDil#!W%*I|l=4~1TF?|>NmVL=F%1K>$7-gZGY+>$i6 zG}qPDAq_JiHEsJt^-XoVcP3{frY0q4k4ow1Oex6gpO%uIpFUvqh7Uqf?A-*y+Ve*b z%F0XIS`}BZr>>=@qpcLPIrO}sp~&ky?X5`H26^Ksk2XpCaFG92>r_a9q7k_ z7_oZjnA#OB(}^U&ft?rx&x;o?-uUsxBgP(q6375R&CJ_8>Du@>Sz=+Of4k>zYCG?| z^G-u!T}nzCG6TfIXR7#bFc@68@O54QGh#dYoO76Dw`1YVEr0k!ds`biKL`$@d+V*Y zRg(&5y#2P71bUFZbm>yqQVL$!l;bn@zvM^$5U}t?KVX%7 zGZlx305vkK2dFj_O0|Ljm7B_+)UqfoLGdZI|FHXCPIf5K!kwsviUGr7(@ZAyIFv(3 zL97e$lmX+BpI>T~tdI^BC~^^Tka@%~Q9z{zi4eIUO(x6W@O|PV9*>^hc;n5zyZ6A0 z@e~TR<0c6M60B$yQvo7-IBzK;10Y_gVDP&o0(38&6JW#YtRFn9y~m_5Dr7uOaQ4Zq}j-w$vsU=%NvJ@V*7KmF;IToX%aNPbkR4f}@=0j#hM&Ae& zQ+H+nRB`Tr4yq7p+!98`D1RV)k|&4W8g4qp#kBe^>qWW{hGJ zveHlg`*We@M(_x=0S&;Rw$}XdM_#aG&2uLmwd9&#&w2IJloG}#z!p`pe6 zyOVPtpa*j!VJxQL{So~IH*ScG;{6f)ga=NWMytf+18(npDR5C%%>|chu0^f18W9 z&BezNhsVd~q~>Kgb2HevgBKX%k)jdskBT_D87# zk%#@4Lcjs}K#U?x^pB8bSNlwvLNk!7+44*KW|ZUnoZ22u8My$&$p-No&4V))!yg`B zdjAXZWT`tUr?E*xM^ws)2-Yi?{LFzKV>5K{TGI*`K$=&V-= z=MTf80VSwQzh?W%c(eZ{-oJJHjM`~+rWSh8k3=MqMA30<`|or+BcU+vinnjy7Vr0S z?!jhDPe~~(?1!6nJpb=MpdappeIDP=T{|%>3qeJ3FxRD0XCSAO~A@0N>?sB_t+X^|PzyzVs44GRdhdeqJFm?zmC@g2>=enV1p| zw1+G9)K*p0S9R8mIwCVMDQ?xrE9&>PojC2doPo*h&CaqWyDMz-IDF#=PP8?;>uXzf zmzK5;Yu2pR_Tl{ReCOPA&Qbp|=!?N;SHpu8O`3G# z?|=UXwSVOUU$waEs;e-~K{LVoL9MO7{`GI>&6{V*2Tj3s*IkG5YDM@-OTT;dn$>DG z=bwN6IY!l}Qu44WkrgaTsWI*U_19j5kth!Ik406WAJ6Pxn?Ij2(W86&0D#{8uYnKr z?)bbCQ>TI`4xSBIE9sl>zx*T=gpIS&OoLUEPXr(Z%7wz=sD_s!1ZA9KC^<0G!TB^M z8p#x&NQ;vL99}61C5eNmqH_yhy9A0@>5&-_&SGG>Y&iP&e?G<}nFQ0(4qi_JQM_}C z;IR-M--PeD{nlACXV7lUMw=C}ltDk&Q$%fvNeM+sLq`rDdhWSrwzRgaTetq*<^Ni@ zZte5WzuKKA%OFZ}L>pdSPS6`bGztat=Zz{eVp%Pze< zKEW>oBeP{^t->>Yc(4&i1cr!OicPvCsU(nrhx{-%4+yOr{YWMqOffPt#6wDGkY^_2 zx2n2o-n_Yh0GZRKO+Dd+6T0KGm}VDZL620Yh|)3Pd5M6dku~UN9$E_r&=$$yh?Hup zZxC$c#WC=uVJD~{pjHE4MLo5?a`%!vg^ea_@Lt?A_TlNDmiVnp^ z1VCryaz+dT_!omOLFTsFpIH6U@tMeU{)#j}KkyAmO^d-tHmQTTm3JMPsn#&yK*HuN zTeff8vU$@Mv=3jW|B5C=02Gkevrulzhk_C|nb;+NdrDI?i%TkC{t zZ~E~5TX(*>XjEoid}6xI+k!^mV0mU4ECocxCR$s0}cmv{-+gC!2Cn6f@daNX2O*lym1o7 z*pgx<1_J?v(2XelN;_gPw~RRuP>;<|YMUYsw<}}dus};4XPxo5%$xNG@rh^*`Os+0{+AEFm{oE? zbnr$9oqa3}M8vNmrUrN*rxnW-Du<>h5Ikvhif@z<%b+*VUAy{sOTWAmWvybg3I!a| zqvC%S{}q!%?etf*AJFg_`(GIjqY~)^K_CD?ENB1%jhSO~{NJ-@_qw&~reMkvRZtYB zBv>WuRQVvD0sIsB0E%!bDx0mXtp#ruP-q0`!e?e_JW#!e6z(ajbh}oNcf)6$TC3r1 zV4XCu9)VVRNX3EtSLheQ4@ST07oa?-K>Uz&xmGbqkP%GQ>Y3^D`!BZA>}YH23}Ay# zo7d;HVPheDG&>O7uwg?>GrzjRCKhd()7ig&zu`s0u=)TLKy#H9l~@A;3E)+gl9H05 z;lt}28d{p0V8v_@?EK^qibk-jsSQ0y_|7};!1q(K@{*Ht28}&t#3|o8t*x`AsF_xFmxU>Kiu{LHbJVPGm) z3i+iMU%KVyn+FaWZ22!`uBtF~Ncq_9XP$NDz4zP;(ZEJuo;UB}i!KsDQF*4-vMBz& z{PH~hi0zzr?s;OWC}rHAQr*-lei>wOnv-!YK1(~4fW%53u@DxJrUv%UsgP*B}W!=vV(@Wnyo@mrAu!vcK@+3t9>v zczXDehcCbEGN#(d^(7ZyL>c1>XF=ASh+zQ#5YMD;{;T~9sbOJqCN?IS0Vd)@91tXr z0y6KT4@ys-H2H+%j|a;DTJZXUYp%Iw;i841 zP{iwXopQ>oOD?$x`{LaH;6rGz#XrPa0f|*;WQ?%oWfk|_ci(S+`y1Ir*Ff$eA5uXI z7>z%0V95tjYAi^^!6db?82QV9)ZX6K{>GaNUwLKzhaax5sH~{3YrxvJtgP&uob2O{ zJMP+=&AP%`U@I@}BE+Oge@|T?I`G$UD={JZZl9Fg)vs{e*kQV%- zLkZWGeg%P<#Wn;8*-ZH`0f!7zs4TSoCh>@l!IxN5zfy$sAcCdkY~k~=J3ca!E<}ob zvWy&RpQh^zZHDep0w4m^9iLa-@niM7X3bjMGpqE$V{viuwr$%+jvQeK4<-A;7s(m& z0bQ~9AUPJFQZ#f+24dxdS8B1R(c4Rxx;&m$AG|NZG$hj-zlZ!ONFh<>qjexk6a3g)Vkyoi!UJmHvc7YLVs`n>fOKe)-MGD8A+cE zAX7dnHc9$y{(I6TS3sh;M_=;zgA}bP|5$tqQ`ACZ;D26Q^pgrx(n#vl{>68YsS96$ zlzsuURjps;PEVTfv)^L(eN;yq#F@1LANCE+b9`5H`k5a;@TX0WJykU4Y+JC^=H~d) z!jGXuh!y!LKJvVK)mh~$mpL5n@gw87t{(~L9r4&(dP8U4S0|?s z5CLdBG$2tJ65zoRWq?$IQ)sYVS8aW#%ie;8ACYK~9_b7_@QO!_d{|nq!HkLkzX&<( zbEgjpG&HypeWQoO@c{+lp$!h26(4I}Ard?r=6nb5m*_35mh%&+jnf*K01@jKeVFuf zKmiyK8?m*vhW!cg;b0rUf$WS+w7WndsbO^_j(<`)i3QX9R($~E;j_v8TU%sDS^#k?<}OG4Zu##it;U&6JPgkT{W# zRkD6i@tOTs5lA+r)lL2ikElNBSNoxWS)UL{`{Bq5PwzHt_!w*HRllGG`zAjA#FJB} zO@(ZfXiGk72~wA8Oa$1Me&@dYGQLp6Ol(J3DPBm+&&%7maU=N^`7=@F&l?I=54i&V zr}n>^zfb$`Q~ng096j(!fY8rUUswN&@g8>`_4=~2b37hzTU904!A_Z&)QC@v$9xC) z;`DPgij@?lC8fNw*-k(GbkZ1(V9!yH+d1NhQGSoFs;auRrMax4GK5bD2SOR?X=vV| z){rfd-!EW5RmqfQui^fAYnY7|qWDxqmM#HRQS|AKyv0U5bOBJ$bBj6P!HCmRsPgX-Jw zxZ{EQA29VxyEFV3d07Nkj6CP;v+w!aJunzp&yu&^!s|c&I6PFw6%zTwU+yXpedV?J zQVl;iefHUB_qHD)L8DPZ5tKN=Lt!~$+E2ox0O??1Oh2$nUP(OD?2hkz=K_qepLymP z6~Zrmc`bJBg+BZAUsfymq;VG@p}$A_4}zppl`S5ZHY{1PREV%mnK}haLaw^%XRWO* zKwA9fn=LJ^33!du0OOA-AE~W7esB6o4-Z3KN+ChC-!Sxx2;)FjA&g>NvVTj6z7vKwu&#kG#3^e#4dss+u!`x7vR8B$>rg&A%q7>;n!kTsa_N`Z6 zdv#NN6A^6@d=%>CSLe@r^%cL@7Y<^Cs9W-qJBk7w2~@!8VhoNPFFg0`uYUOpewGFJ zA|FvaFDNmwT{drW^p=n0(VR$>_M(@3HgDd1!37s=+O&x|cBgIZ?h%73=1(Q_%U_eG2%!33*l1_w4rN%1#QXBIa0kVjLXXw{@ zD^zMRszz~jcR(Q{Y}LRw;0^t%sFWmW@C8^-LKK}vWoAmU3twW$7$CdT&mu}ct%r{_ zW#rz4ts${M<|K_`joNmrD}pFV5PZ(+mS@7obUK!jbbPFRoO3d=R27h0f*8Fis%Va^~=?MLMUK$R7Y|$T!uk;%-mq+;;7F~AH z4@`2%A08;B{Pv(W002M$NklvhOTPwdmLACxt;J{`y4&Zc z>esTL1NrZ}tJbmtJzy}VMq?~dNHKDl8MP%8|5bDAjsJQ5_mk%yi$;P)GfxS2c0w97 zIV^q09#POd`x8vmvlR+8H@jU)?&Rh`V_mSZ!4scu$p-=2nEHao8nPvW8y@}J{&yGS z-hRSJ52iE}7F%y%CoA0bV+sKl<-o&kfS?=122-2|;kzo>p93zz4h`>R0|<1WTMP4b ze-uUpevp$R4J3AAg?6XVZg=_;BW^D(Rm7@i0?=QK_FwJ)>5Npz01uksaU;C!y+MQQ zJWiUp)SXi*1gDe{Bm!@bkfv)kKpi#-L8MZ`g&(9J>g2|yc`_5j|8vExXk(0Cqvm95x)EctQPS7q78k{WvP*avNLYy48Wkpc|i8h(_{f zf4_N8X{e*ku~}J}ixw`#myGC^kV!O(M`drm z@#ZPBWzoRW`w_{rlwb4el-%pq%GW)a=@bLQ)kVptg6HS6&^vy zK}CQPbgWarG6BVh0JZ-Tjfl4sf>cq?z!w5U*1V8XOw~~wX*IM!RapI_nG=b9kDrPE zb@jiU9i5%+oe)lGX>n3g3f^!BW1#=k(`T<*`2lug4g`by_wFq#FK=mUb#aoHQ@lHN z?W(9KA3l6Il!DDq5`6yEAFSHFcMoDHvA1S|mYxxFP@tI{0> zH{N)|icO1?`+FOfm#uwsN5<&`3JP;=uJHI1hEJY(3~nf%dw9{_EwyfslV86IM@Ahv zWaQL5Z%R{5U=QVM9G`LgEcz0gntj^r#^xq$+6sUtAAj=yZoDZwCtLMPNQUH3^sDj5 z9)Em(e*XUbWk_giZ+~Oa8<@zTD!cpd2mgAnx)L)=#F#vJQdU-$+K-wN?TSUZ{1-tn zGo27+DKjVSTk&C0YL2u?E;nKuuaVQ6@d=DhRHkF|d- z&IZWDSN2#n2@p_{fBqptUYB)f9HJm)1!5;81r-N7M)dah?tldALn+AIqLX90|rc(Fy^Qu#~(lOn11~V_&|u%i{X$3VM0oyF4iJS z=ZlW^=!YMD^qudV-`da!%B5%IWFeM~APcseMr))m&9F+A!Y_(*5Xi+H4tq&y>At;t z2MioY8Idhng3Fu;N}RHTdO2s*O+J#JMTjkjmS7Me)MvLn{@7zz|LkY2ZQKK&2jf-5 zU<7r5OGdY~v?>Zo6!{Pmget*GoRns6940={lupfpu#^J^f{eqZI1h!$ORZdbXk1Db zB}$T!-HH9 z<|ml=v&x|-KC&S%#legc##zK*VDLxS5m_OG196~z;xQA)jv4*&hK&e=*pp5^X~2N~ z0z)9xFN!M8!SG|~M{(fe3{0ViC#e|$9ESWcrzlr!(t6@Uq}}O9Ddg#GpIQ(&hOlA+ z2cjSGrhy0rhU8>=Qs|{cO7%)0kfjz=>?ZvO!Y7?%m@+WPBXLHY)NVk<$cJEhSOE(k zDT-s#V?X+siARhXvvK1mEMkw&n0c~jtq=Jyy*vG%1-}RVxU~EWVOmd5{T_^d6zD;} ziH~UV6pPPGQ$0Q8V@4bH!z#tU-hF=!?tSs1=#*n82)|@lVH964RVPX)sz4I(C4e~Q zH_A!?Vdngb{TGCyg=8zOl(1G_nDSA=BudRms07VIb{G99)pz?K?el@<--TcQ;d{|w zi+se6a>4;$L$8J>qYm^oLKJIt*b+vK?Oa=fy-O)C^L|1EsO2Op7*m2NSX*a9?T6Q2 z>n=#0Fx&$*0TH(mxc?%bz>fw|75XYfu@Q~6GZ+Q3sA9_FB8DWx#lNC@sq=%3bL7G3NWK`?`LfeIP*KMsd$zU+-!hR!md%Q1< zDGWFvm4RAX5{cs^7G;l z{TKDn7GvZ?OsO3wkO2eW!86EW5HFi z;|pC~^2g5M6-W@))wzep? z&+O>vOo;d6F5T_s*AnbbJ0=|}YpStB$ByD1jcrxgh3TV@9DMl=-@<04ZGp~)rU2f* z&(G_(YfI_w?fcpS?Vfng<`40Lj4dN0wcpv9!IsG8<>gLyG}2}(+g}^#2;sZqsriZC ze!+X5`jg8R*WMW1zO!t^Q3a!qEGjA(osyi0X7-~cJNND^&qx@Mb##AMTx7tgtd#zK z?95WWr*_-IeZvPuZQrNDDL7G?sW0mHK>o!vcn32$e*B{!{qA?a!(~Tj2fnm&>#ct> z%p01*A5Dsy0X_~SVK1Ju&OYnDd+tSr@U@p;ei{2*(1!Ie^*eXoJW!&s=bd*R3KFHe ze=#p&m2&k}P7Nw`B+&#wm^|`Fvw;-L)0rKK-TZgx(4nWD@$Hv&#{ioRJIf9pJXkW4 zW*i#UUttgxm=z<$n!DO(B(Ml`vtoo_n)?!dNsi5a2A2QGzN)NFIGQ_eF4lTTErQ9) z4EhPu3m?hI%vwx*mXPW+4q@S#(N_CrWMQ1-l>kG658{LXQYjh!L;cr9u?vkLmoAF# zp3gwgeISD7XEC4*A_FBuRq$&!9E0$ZH&#GWr6CVeVg0~>mjhojNhz%ltvd6vI3~#P zDJ1hi9?FmaK#e~0%=5U12ZkdWl7+d%!_y&rTN-P>&^7%0$}2Cr2>=`k%XqUTWe?=8O z>4XhnFuTDu3n+wEiVU*T;t!n`kHCaro$RnXgQ4JGZolomd+);}E;2}6ICJ3>4hWN# zs}GY9;SQ^T<=Ym(Wuk~%nL_iJ-jKMWA#b29-iMC?4a#B0;1`qbsFIq~Ze5l0QlV zo>~*qNgU+^;y$1MY8Bd8UU0#82M!nrsenZBUp`JIol?=jfzlsP;wmcE+>3rB@(=My z)x)6Q@?X)fV4Cy`^`Kw<%fgRgpZ2eqmI{$dp5=giJZ?Ak62nT%;*w&#b9BWOKcR@E zZmElTKcoMW=a@qYg$R88{(JIe|D*R4_}G@xk246e>_@Cv z(Azr%mULoh2%=?PV;2{kKA`F+cu0w)^={fZ#$Yx^m_sEO)Pg01qA zXd_nr*OQvlS+&#Va@j&bNCy@TU>>-`=d?U8+318Ko%?r%L!H`a83{4OD?Z9W0hat1 zT{z2~pI$VW8+>wc1orXfc2?-CTnORgDr^GiN8xR|Be+j-IJqVgZL7=P?zDTt0c^sJ zCyXG-)e*IKwuc%UPy-)?fFgffG(QJJa0+qINi$I~fVsbETq>qV;D5@9>KEk2v_JM| za6&)|LlO*7gB5h%NK%*t#p4s~?!TPiNh!H{fr_%iNN zVPG9TDCtZ{aQ4r2`Mt;_`>HykH4R~GhEj0!(Wry5cDK!&1fEp>;1&dE`^kX#FB$=N z9+q*s!k8&xTSxh09!L0=XYvS#T?4N^zjVM{=yEHe3K_^*x+mG*=BFSN^+o<`06Wih44LcBEX6PIFsVE>EwFXcr|8%|ANz*CeA z(hO+Z>{hsOFT7evN4k2sRZitks2KkR1Yi!a|0R8LXrE1bSw$#2GxNA(k6F25B}TFo z0V<0`D=Vv)E?tVn9ajH{6c7*cV*OVeO6@C=0SDgEc=w%mxJ4r1A!W$jZXY}5h<=6r z@EjP9N>N}x38R06chh&3FczWtQT~b(sm}boe1E*ZqN0qIMWWc06gL%M9@49L0s(<*o&Ju54xrluM~ z427^CWuR#E;Isj*k>dyD4a~&LCM|6(`x=Tr-u&UmtG1`4C1F`ad`C{@))wqF+t%4` z?{thCd!#qc+t5~3*HE>mrn;%T8Lf0?PDY_6-Xj`}<>Fw?8E*`RK``kDGZ^VtRaQL$GXnU9LB$+1nm*?sE-K!kbEJ z|BxXV`YhvTj3KE<@=w#d{PN53fsziqRAsY0^w7h<{mpeC2F3t&sQ;>pkdsb@1e@*L zv(LTnzWZRM2rpW+2)mZx6(O-Bp+)@b)z@C-5T7RJ#M%c1WQB&t+1HW+CPcG1s>o8J7>*0 z8IvCU^0K~v(FGTO_xz4fsJ*>&!E5uMc>3wv?zj`L$z^0@U2y*S-#!1_g2H^11chkQ zK?iB2@nhsDHyDwof0-k z>=Xa*vDX4!I-&651gNzshIRC!79u&)tLrv37e# zRwky;r%s)!`7kA;fgL+`Jo3mRzBue=NBM(A;fwmAOA#2M9{8F-+K^!Ck2x*cRKK`n z6eT7g)S%MeO}{K>W@8bNu$DFONeyYB+%-YvBicX`B@DMT=GqV;F&ty_(6T+`qe-kw z5H*kDqt6br&t-K&J6Qlee!A07S`8ebUqHIqC$R-zvl;lxJ+Y1AsFBS|H~E9EXmV2W z^|CFqRSUTnajF>l!sjDKq^EhZL&RQNO$HgP5%l7WXSSe#R2-CkgL@!=RH0e26OUIwLwQ5LL{j-9!PGCf zL{6i9Q{KrnkG|xuIRp@}$;rvrU4I<{-RSRTKg2({|FR;|(9?cczbSttD^Y#gkK%(n z9HhI8eqk;a-=x2*wgcFYDId{?**!uEp&LKZSzBln=vdk1_UB zzJ5*1t_>;K`232CpNZv%Auw5hP6<|rJ+HXR0gdR+{h_92?3w3`1{({foi%Xsq?7?e zu&ae9Au;Gm#J3V1{sa_uI6a|2d-=L`B}*6We&WHX+uq@D7ViN7s0A>7&O z@cHcgSfa+0NVN@1K49ndImg)ZLT-RlQ`f-5YLm__pohpdb{n=VrMJGaZdB zA&1wMn3WS~X#~F8A8&KJjQ3F}HrCR;{m0iSg6@>0?Bh>Ds>)yJ*NG_TCAxR*2Y9?V z@|;U{Jb8Ddvpvw-jBHtBCWM+ADG;}ZrT*jn7euOks*b)v{;T$H(N8Uk{UfHk|8oCr zB;h~)Q8n~#zFr}A%OpKSzX+;N6BrkL;rH}kEl2^fBEd&l5vYNMB)~CMZmx*S7#@f; zs^S%+#pVehLB@h3v;Fyk{jZ`BA3dXxsrU}(^yxFOcUUKG!dV^rtY9#(bm`lZPCPL_ z0iP2>oU)?yr)XQZZrxf{Q$1pM5uT-H4^5eM;nduLDJ>m!(SWnEwzj3QVfT)*E$d20 zj2$@U=;8lbxM9WoP2ax2l~t5bQCi*7(ANLRf(esGZ1`x?`u8>t$SOK=(9neO3BgF~ zC)+k`|7b5Vq-G`@J@H7l&ogSw(19ZgSO4q7SMGQ#90;Fu`plM=7O%&1^4Z56KWntd z=iF0V{^aBDx74*`i|dHb=JPm*4j4Z5loN(#kD=|7wb=1Lv}7pA(2-%^xKhQ7I%Rl( z2hW#Wa>>IFKa8}d=B5W9dhj>D{jE`4O$YEaMeF{>cI@%T=H%p7l$WzeQ`53#%T77v z6j}!Bq?xQ+w{8!<)G35bpFSN`i2WSce@T=*iuJ<* zO#j7ZrA!jI(vp(y@&P60{jX6iYGh4HzZ5ip7)GoJjN^Zs4c}YAPPYm@=iA>_Hj|Q* zrca-SZ|I^}dq>+MyxnuwS<)1Bqu=lzpa4o});x+&Z(5-hVs(gyT-0IaA-NBASo`T2a5p;YmwR`96>DRaBOv^?C2Tl}nbqg)L_S z+*+CQ2zAWGnaZKG5oX=uS_WBmd_f9} zUv~4wIs5S8MR;nBg?2gzNdlEhq%kN5xj+gQ33q<<;l~?3*^rx^hgQ3=UxDU<{8$eU z6=s+H2tAx%GUhm=(dOosjh}2p%Z0B)r>CXiD_p6msYn$%H76eT;R+yfg9Tuvid7i+ zQWu_&h8i0iFmTpKi}CS^SQE%3k{fNPuSZjbGHGdPLO%t8P6c0n+PQrfUSu6Neyqzh zIg5T4mV-E~5iByXsi|?phEK3J3Yy)Kqec$sKR_xWJ%)@~kcDUrtkzy#QzfY)e?Hm3 zKPI?c9&9U%oKk>n0-t4L{HwmMj%%Ipxw4E5X0zKm1ECe~y}NJke(+l~tZ4M;(fFv4 zO2(*|fapJ(gZO|VmmR?}?ckTB;2tFutl{TicqJ`2Nd-QxaZB%XbFUehLnQr(Jp}&+ zs)5bsNopasiBD<;pZaAw56BSlNU5XN5D_8cAT){iw{KnsCRmnSs^S`#Ww`DmKuzxDNXc(eulw6s*@1BGZb-h1y~`}c{~iblZKl9G~0 zsDj7%ub2Skg@S7T7?^(a(MPblyu4hfyI;S4LJJ9?$*ExJ?_K^yb*heyR9>`;)gq3g(+b6s5s1Q9uXxc4* z^(MsGu?bdco;S0fJt4{NP4K`Ov9wy+L$HiDR0<}|z`w~i-=P91S^oBl|2a9*;`0s| z9jz$yM1vS7Iq|I%O;%2YRYUkTTj@J*HSO9Sbvecl@#58Q=$QIoU&Nl876b9)ZFXEc zY-=`k;$;~}B;YR`dfb&i$Ba+Z>+`4Q*b}pz__nLZ$L-1O(OkQI#ATENJ7T+S+v_iX z^nX8%G}qPCCDqh%sfaHjk=k>6qv3$v<9E2@geZyul0-R@7vsPCK3WeVhM#dJlSKa7 zzr$$6aH5-De$j60NFP0R3w9{5yIQubhlRxF6x7yK!K&O|Y(c{Q7x=`L3VPU&)jlAH z5L45u%V~;zQHKI7OayuU4i=@fAf+-X# zKy`?b79JS9VkVNzoFoBfME50s;mjDHFq`!F@=E}vpNM_pTluIF9@hBTs-EO^Z~wo5 z|E6VTVrd3mP6>xXU=y2PfH1e)SzA-P@`IJhi7AN*Nw_BmGWL04&&tn;hL9n@e|CCd zva4T*KiRpzwshJ0jSV%8$;k=vz63`w!5#7869jlAX4ks1gJI?ZNyZIko$n=KkZ+ zwl;jzC6YBTHMO5-%%I^Xo>kP+*tUIr#o7-(Zf$AtCAre`6Y>Y=wA8jIro@Hq?Q>sy z{?r+rQ!y`Wwx23aoK@hKepP|Aes-W7#>+_D(EBylUiT`61cto`s;4E z;eWBaDl#Jv=x1>N^A|DEr=EKH!3Q72);m~2rYVR{OiDWawA1do>#p?lG}gr`y6IQc zj;c|GZTt2eH{N*Tx^-*uKARFF(0uAPVf>L-UU}uk7hen^XcAl#0bt@JbjUBI(}j;6 zs%Oobz4HB)n&xsjUSF_a=8PG*1pV)=cyH>|X;L`4chBD3+}!)`yZ7-Yp1`NS@V#!t zgQhsY9}UDq4?To!vwPD|qmA0zJ3DUp-y5O#;^Lj81&93L4tesVDfira&%l8Lgl0A- zK(*+9^wCH05g*2x2Z#qI>cw=}>eZ_WbO8TaxpL){$&SFJws z$RmMk(T~E1;J->Z&k_<%f{gC?NW&9kiIj{o^G(L5Oi>w@?cbLM#{YWzop;=YufS3r z!V&~{?X}n7GcI@?pH!OtC>b^7zjxo>U;p~oFU_5+6Z0q{LSuO|OfVGw@fBC#)iWuJ zYui8m@ppM$MXO>As}NXd4KtpCBayxQ7mI|kGnV4-+rK|6BmFObx&5xYp*6k_Kw%Kd zcFi@{-tvcAux6FgArT@dm6qwhPd)wA0}o(j*~bBKH^od!N~S{cX~#;0>=86 zVgU_e?(M%wjg=3S4ul@^N6}dSB9z=YtN+Rq4wM4EJTNf`r~Ii{{*ptaq9)6KHBEse zQoinA(aBN7e-u^2Ge?)0ZrY}=AMVx&nw`zhWt0L^Ci#3wvE;{=B* zF#eKDU8w~QY#71f$kGj8t_^&I%a$xl8kpqs*tp3DRKR<0bdQjaQ`bUP)Q3BR(S+3O zsW;yo4s>98!IP5fDH!HVNW*(Bz!%}*%<*L}`7mn0w5T)b$Ppf40W%$(;GoRpc?Dje!8i@G9(4`jJ^L{qWA~*NO#R*O z@F>IKN${p-I@0poDLJg$?N(XQutlmuEwCI#r=2xyHV1i~mHXzOcSh^JkCU?p$MqZ8 z*;t8>Cm|UR zRPs){sBYKxPGmXy-bIoXKi(gA(#(@E3}d?>#DQi!_rh-C#pL-01Vh zWng32%rv~2QeIY$J6X&VH8eJ$uBPTj{DUIEDfUdkR5bcxzd!ECVdGLWQY!GZ*S0`& zWrw43Prx5dNl7i=R#&lS&#-a%$(ix>HRVn9?L!9)I)29Z_>{OT`$%k5iPan>yGk;$ zGbWypHF5UP{pI`1Hr2J&1hKrY)zi4UW?PLtZo{@vur1<_dPj{OJ!8hqK|{+k`o}xd z14(HKnTZ*jJ}lm|t1K95?+A3}^~=pn9+B5SH`pHDx@~hxw!fjK$?b4?oxXkh_K9TZ z5tP578w3y7|AI?VKk`7-w2PrbhMaZQoEKljJ`^_WX#3b>kNxB)KT-7&T)?E3uJ-fn zbI!(O2g<@PUVZhI`|rng0Z3JU$3`BnO8zK>IcLtd0Y#$$y66`{P^JNj7sD#K!t%4;&!A! z=s`^+Rw^W&jZF<)z9Akve8dQB4#G;vq3w)s&$;r-s~ED`7A#!Y-rj*Zqwe%e&hGw; zis)Y`Cil?*)_EuqkkS62{U_?D*}o-!6)p)ACcZTP1rdB6N`VNtfgdX$WnUG9jD}7% zdtdbHGy!-dRU4?(1bj}mM#>5yceawMas${EE7{1^L+U~A0f?=06y zRD%RyVW{`VKmK9$nl+0SEz3h=k z9zONdQxPNe8T3=<0*~SBU3dNMh8u6{Xg79OjTHihB5T*KText+MHgR;cuf)tMLrCH z9YG?khW)g*w4Qp}=_^;R5JGJ(xBKa5o}D>krW9vxK1h@UY?PIiUv}B0OO`IBxq(J< z2HI@+5c-o(KDlq-etfFhc=?FZV=>CuX8&a4h6^sdaNCwG5(`uA5-6q>7QL|u6I-|4 ze%p_K^kbHgOcwo^Qi05&9g#l;hdf9XCgeiogH&n{cO=pwK>o;pwG1dGG}^HZ^||Mq zkCiy|6D@*Y$T%9k`_4O8uUdtDCA4n%FSdim>-ka(vw#<5lk6yJ)GhHs4yhoN!YZLk zzd zjyqSaS`Ee#OaC(LUxjRY=GkYU5qxP+8CR30{j*jwE;KNa|LN&cr^I6Lr?g3CEg(>| z!_l8Ed|_AfP>0Mz?rG254#H^vam7QR<M{IR!%!%2df2~!6IpR$?1!>Oq?IJ{ zr@&+B*K$Uj&`BxL#guYYkl;%V-RO@62N1brCeDDHb@z}@Z~BEE^2NU%^0C=2+|c>N zK=*t9`?JY^x;JOo2r)iXflrL_39vgU`MD`4+WfZsi6uLn(9i)bf*=%v+|t2B=Af;I;o(bAR~y6 zO0+`tIZt-MIm_2$ak>U8gOkZz3(aSAU5rW z-E&m{aseNLk2t*^M?!13CRFv&YP4fRrk+%c-AS=QS6j2)5lPA~qZjcgsA%Sj`7h|~f&WeMU)X0^i~Z40>@OGFeleoMhOJe+iPR(bq5WI! z4`t8)IEe*fc8!W;mx=^KH%z0J4EZCw%HNU?R1$0dG57>#y(p_-)S21-(C4t4L$ykL z45oEgPTiIbc@N~jX8WlI(1lO1FW&!xZ-oLa2$160r=9-%3oq2v)<{Rn4S2xF>eXvX zOLt?LGbBKU$R61wCp>{+iB`^Sb`1G6BbkNuI-OW+dCV~rRe(qm39%gU35Wk|M{Ibs z!4P7Y3TVX<5lR$KwEtN7TkX$*{1>@#D3yXvs-zAMrL-#JS7KxF5dpHl3jgiw#B+)O zR(d+!PAmv>yB+-Ew;fv-#Sb1l2wQbvw1(kY2zyZD&G{g=aKi9-=gwW7+uCz;a~dkz z6LQ=GNA_>6%iXhQcm9C9;S&cn*EBnAp3$R*4Ie$Qv8i$13kyD4zH?B&A=t|!r(a6J zkz;c6a*pUXaZho-9A{a!t*oY^s;;pS{9<3fnort55r$%3UvOVl>GI9579C#{3jBZE zy$66D#g+a2-kUc$Ni!OavPPo_l2AZoV?ap47!Vm7j2B}gg422pPB`FQFJODUWE-$a z#sq^6h-k7%1{;Yalye@%k!CbGz3+GK?e2Lq8p(iRzukXJ)33X_>ejjE-m2=ZuC8iX zy>8Q6&o7;M!s$Qy+SR{*@V1=%WJ@EX<9KjxSh8ekao^ms0ezEgS!DwU@7P}3-mtxT zV{PCFFl+1@{n%KfTYy+rpzN*yKFs1kmbKuD{5|iV zT{<9ziM=K8voOF^_~gs48erY>3$d~kGBF(C=lsK#Uta7;F_(rUNmUzc!a2thL5*Mz zq5Q}cCs1vriQb7}K}yEm7Tq6K(V0ON%|z>$jmVwyt)AOcK1fJw&q%SAt5@>`M>12> zZ##iOPSDUXK(|MX0kgKn*l_t2E7kQDBCN_(*&9><^R00PllqpIm!E$6nWHyH@|6kg@f;+`}Qm6TN1tcR?66O&pz|iQ&0J~a1nmm@8hP=pk`4TSs+zOcgx6; zYgVmUzH%9J;}OhSS*Mf3L< z3s0^kdqGtQMv0wt;tAWfZzoVD zipBV5r=h++IElsP%z2S^nlF~j^ zn>TaSgvB9Y_9Ks+dfI866UR6Ts1*6~F2OKq!GWc9UHy>HYM52^BrTU&xQgOfSDg z322NRJM~zWSW{XiSv2rlwruf=8%cZ__sHx=PCeyRuBMk?R`TP^=T~2S1)QqWQnBHc z!$yu6xn?ah5HB+X@y(kzGkkI0+_%ObFy2i}WX}l6>Ggvb7?&Rc17C<=V5PaD8O!by z-ODecjUOXTFXvs9b-63wPW!qoyOrNP_g zX`1?*+Xqf!P*uF8p??0ocP+jDmW+ZtK9QJm$#?i#gb6#0d^w@t<#gjm;jzS~d;Zvw zRa9IcZ>19gWfROLMc0Q~aHcCw)HRCj0sHOWR#!)jOh0|HsBdO|uQ&^(YwIq5Py7a6 z_&fR6iLAztwDUb@#?S6GE8AlYnVIq2Y=~DwsV5&9m6djrtRL&kxvW%AN35l;`lP$> zp??}YS-tvY4jz+LP#ROgM7|~4Lq0i+QLEw*PRyYE#$tJ8{pHcg%mhor62-Jj#~<9E zm>=`3ku|8jqcPcTivgRfs7prjbzej2&1#Qj5BkCx>z@8oTXSP(NjZGri;t)c+~{S?I3 z;}9_SRerfDq}r4skS{&tyDmS%Ph|77PXAH7P(BAHOck{!5eVzaLpr^F=YHYb8*aK$ zchoqY;6YE@?YG~4b+{-7>5;#;`{%`r7r*`XJ9hi+z-Zs&UpVJ%%T)MbTE}VpOAi9N z)(?9tzuo*_V7j*No&0!~d+-yrUt&pr${IOdKcvftUC@3)<(GLnSbdi7zH6&@WwJtd zNpIOU)m~Is#1l$_dm)w}C=*ER6baVrnc>U7xMR=suCuHD;X z#f_u)8^7PUVGX;Rc5bQPy|ekrd*`iLw!SUZaQLwk4nO*!=7#2~E!&o_Ub(BPv2APK zn+x7pK7UnCqOdT#uurd2NgI^QFxZ^j>{zm6$L=k8rP;Mx8(CR)*VdYq%hnY2DcHGd z$MwJe+1$71<`(1>6cyy<78K|A>Qh{vXpQ&oQ@m?qqh6vlB>R>2-+%J358M8RuN`GD z4B>X0GD-@*7WZv-cZ&2}IcJM`j;= z^wAu!J^b)&d6@X+8Pi)fm*#5fzsGKUk#N+G4_gr+AUcM}b3HPY`x_{uA2q}u3 ziuEZivtN1M9_5#|&qNBO9F#O5B(z@t*qP-EHUjr7Uwf0^WBq9?A-?nZk=&?f zXp8ZM9_2fv4_kh%aaZ*d)(+NFKr3O?VCK#nJ4|9Ki)r6b+Vqb7@EaHY_!}4g@aq>U zMo^P-1+UV^^Pw=?v0EZtJ}JKXW5#1{JXGoNM!l_lQ`P3bzV_PB ze{t2p2OlzgSmni+T=Iu|?^(2X5#6Gc3bf#pIpoV)bX| zT6vg?<`%|}v$LlZDJteAA1R$ zd8*>+DV_J;jPqZ=ZY>W^JkJwKAp2kZ;un#9l_nOu>#naaP_OZtvk16!t&zpXR2fQ%*h=BnpnxQJwv#2mds4=1Cj_IKnUc zXcOmKx2dOt16KKd=bd+IYid@nUUmNYUnatFE=MWBmk zSjCf1KDlGZj@7GIGXVr&weW$H#EZGuPp`XelhASj-iHS0neuR4aH(6iD&$7@}A zIW-^&$UsKv=AT{h2Nps8F8b!pm-x_e{=pw3x`UaNO(^gm4!1U;VQJ(*({Hk`6eYI({s=nk;eEZT%H&<<@ux;A3 zX~v8hhF6u%b=O?$0*P!nD`3=OS@#A^9e5x8#JZN8Zf(nL5MM2N`;??AsSLGjRf&{FDZIGSt7rm-87+?9Wk4 z$$n!^J09S3nFEWqFlyf;|FP7{)C|sVu#qq%Zr|k>sN(0b3qDA=KzMQLu4L8?;b6M9FG5boW29jj=|IBAByA&@~NEpaB_BXvZFA+_^~*9y?=B3Br>QUfBD_r61#PN>de2y9#|4@Z|-PnPgNAB zhA?T8KDH(E0InO^iNfivT7Wg4A1w~L7;%`|`Tyt3Z^RE6qWTT_s&d4I9@XT1WM~^6 zMXlp-Bl~v4&sBq|Xp#JV^WUTYD>>;YWU8=Xz);AXLHT8DZ@=<#d_3ZZo9TDodw2f( z^Px)m?4^E_v{;Ohu)F|6o?%1F4DRSgkJ&GhZ{-L%vV(Lq{vk#GF@XvB7|U-u|J}-$ z`GPVusFtzFF?LC2^@Fhx6g>g7tsDPRL8=h9OZzBw_FjIol9{|o9X@i@Set!8Jd-^w$sG^KebuhI76#{n|UO`1BFecC2dKw7#w>QD0tJ zl;1mR+m;=x=T_BjYTUYU`|1@d-+K3*x88aCjW=F@?XR!AIOm1e-}qa0Tj9hZhtHgL zW}m#itxc^>ZSDDevzpp>u31sl($Q4cRKI@3#&xUKyvuOw#eaL|=@+iQ;`WChc{HQA z{>$G#{=ma0O`CF-^J4|Ai~qZQtwIyO&JWbVzf-JZZN_pc#)rmU>ri)Gd+l`uBhxo; zu$9oQXZI&(o^d)dJ#hA;vm^UH^6(?NUyQ|aa&k^S`DB-G(G+Mo%KrmELH&={8(u=K zY+_#}bX%w0-;xj^QrWhBl;5>$*LZRwmi6_y;mEvu{b_8NEK`*uLJ;qY8#|t(%5QD$ zZpLevaa|eS$mhhBQPZiZBg74878NKiF8_Bm6oM}TQxx0>>ru)C6Q-pvo z4nNNJH{xf{@`Zi%39g|D>HiqYuQeX>CpnjxwL`<9GrcL&6)|GuC>CVXL&Fi=;G;nL z$=CelHW9gO&LkV^j965nO0J8NPx5s6GM5RJsT5nW5X*1P?p=&T((O3|33S1hv?4NZ zSW@ke&VKyi2mf^7{>2x!ZrN;u zM!5Gvux>5YMrphmGU6LIZM@^QJMOsscAC(fyu9j?98KurYX`KIb z`QqFt#&`-hG9@@3r(gdP%_Sj1c;kJ~J@*W)sNngKkR?kQ45m0r!`zk)4L|#jfA?xD zFDLKyS6{0bRzUvLd26KO9H{*~IeU*wnd#DMPNff)lnug9hb;P-1kT+-H3^l{>i$Dt_#(_4g9Ba4&CToGyTpvk6{+I>Nr{X6$0@VR|5W*v-Be}7 zpLqPSB}*5RJKvMZV1NXHG8dIIrcWPMQR)0d_!U_$EnOHQd*>Z@oP7>m{@VKVDZB5! z`ztDjqB#64S-Mn`+`35Z;Pao>H{blbi_^i07*naRFSL2EQll0 zPdnqx@4h#m@4}Eq%m*KM;Je@d0m)I7%aX$bqHUhU`LS6QRlfmS&*Wpz2I|4lni#e~vzt^Wz?q5Qghy{-&i*)BmO1YafSXVw1fmdJ>d`e z527i(9|rORLt!zqon;{cB7S7(0Dqc&JztS)($e@bSC-LbSxJ)71kmItf8)pf2gw|C ztN#<(+2_8ut7TX9^2Z)tc=JzUZOsD?|Mc(~GfNLRIFomAjJe3vM9U^S;6IYTC;sM^ z9ehEVNsof&VofEq_N;x4rC)|o@BxdgR=0hdhJ`W>DeJVxTH`dJwn}=s|L@uUDqm@| zcl$>ChzD&epL;x(ljvWXBn`&tNU+)rAG2`T8ccksBcr}GRkbbEyrpL5-M8uM|3osg zcc1LR`)AlB9m*44-G92u2MiXmi=FhFL@V=3I`0?R{b__JE@=mx~F}6#-CQc zj4DU{KU98=3E7bY_s^dE-wCt05&v%g|H1Ox*q(Y|Y5eXbv6|+1yiKk^<{Kc`OrrMiOa|PP*KVN>k)&H`l?gpJ#egY)cSE=%A=9W_dzDN9C zKM1}-bonL^#3ro={(JO)yMHFVu*jb&TJ(4z*Xw6``OiG#j2myfv4cKH@=}T9KDy(N z4?HmbfCKi;5AXbV#U(AUhyl&aNE~zQR9Y(%ntW^Z4KmjS!Q?w1R-UEI$5?*T+uz8( zY4T<7^!iN%3%MwD|G{*hMETvkeOtzk9a+g_c5V){fc7r#rRNLgfNN}EIRzF9)#oev zuSx6LTibYvxO~O3Ne50GHhgGfTkWvPeFs+b>eH)e)x7Fgo}O2`y*{fbdE|+c#~n1h ztal#zhj_4;?URbjREJwy-_7ZQIuE zJGRpA(N;OS|FoG0jUGQ@-Lk4*{^t!j`8mn#tkEOYw`CN{#}I3`3Kk%~WncdEUrehF z;9S-*-*o;tJ?iMA51Mk|+_&bDf`Q?@WA-FU&89q%q%v&b@xwoLWWRp>c!Ni-y!v`; z-n_|E4qCH%^*isrV+rENedai3f45Sm?7aRbyFKlzD%v>mSPR{CD5gRl9k`1g3Dnj}nFD*OUplvO<*ZpWWKu#~*#Pxm7P*;bBWv)#<07 zVaFsFJFqKMc7Q^=cGbN2!V7%h{GsYML!$bWhbbm%{UMxY7g>y?OZ8a!v|Ea{>Ak5+z@UJoKmK4AY;Lt-GAU9G=Wo_eN_ zpmSvN=f`OB6>6=%NCUf`R4}fB4WCc$%9%Jd>q#ZF{OUPx--M+T1~zx(ZPZ@=yK z+?*WNWssgK;wYp)LmlgAOEot)My6E*#vn%LM`a0_VlXUe!hw8=2%qzvA490vE0?cS zB-@4!t8o0-SnqahO*O9cD4%Q$6|p2t_TVQTW4?e(F1kb^&S)&o=OjFHKKkgR#9*dq zUS{X!#t!JNp=DW;v z8>ES|5;*dVq}VMt-^@HyVoY2HMYmsB`OmMu`jnGT@zv)4aPJ?ky!uM#C$z6(lSXQU z;0R*0Hsku7|8)Nswb+T*vASl(guHN;j*j2lbp6>HDe4#GKlEtO)iQhbYzC%QSMRK? z*`o>?MbI#eEs4$vb-1XRL(;j>DF|M*;4dZ2ab0Rwthc{I7=kvl`5o58_ zPdlCd9de2IC-BTO&${~Rt4&mD>9XbN{L53)?8~wHr^b+Ro1&IarWY=;9Bk?3mrJTL z1KXxoP7?b$Z`Coy-TM}Klahtl(hul>9PTQRDN&2^A4JDWT42mzkO*%MFn|q z9PvZVbHx=`g!A}zyzmTuN|GfN0 z{D7ZohZNL2O3Po!Pk^8DYy1fTSH1jl2xCwOf4ZQ}ebi5twfIr{qBJ3Xm*p5p64+O) zD;*Ld`LZwchV`dq;IVWl>4pYYR38X>P=1lqrTsGeUG1ybBnK6dS$v$)2OWu|gutvR z5b;C%l3sotzeNe_XKNqjrvU>(lGUqZ+8F@MApRskpI}^|OQfqx}drNc2 z?w#!owHeGaszF%6I`PhrSmvKW6-qVJq7*9Zl96KJK}H^#6gpShzkmB*m`r<6U%rt+ zgEqdYp&i!RWqN0U@p#}$Enm{ow!8MY8?NJf8@d@Xi%T;4RwfE<_nnrU!~Z~e0=LmPqyxEWC>f; zAN-jwY+t#krJ*62R}^b$&F|Bnd()`=il0!vVH8LL1z5I_P6BP(8C9;!|4{jrZGe%1 z9SuV_e*SlqUq&E3u`+i1!uVD^8nmJh2cTE6omW`HlhxVzEVU#1LS^LtqSfjpSIxZ8 zKiJ_((wH{>r8gVtk!rR`_n(hYevQbmezV$^TE^t7tdb}4Wq0MfuktJWAJhG_+?+oc zGBv66czmVG28aAp#V2;)fm8Yq7(iP?6oUs>vE@sb&42H`1JvJPo+?b8<;z)n?wRM9 z%E9F8&KUIu3>G>6$2|*Ev#Dn%K8KL%V>$!O+0ecL5GaX$jIEW zv4JXT-TIB0g{iUug~h!`uX=ye{l9;tp{>5Ty)o7sdv$aqTRL)@vI<$=D#5J$#YM&3 z1u;N?!6Pj7pqH9$DHgojKKISTjyQO8)s|gVbwwqG%m9+x)Us(!RaTtWLwS?NPad`Z zh?2fV+%l|OyXx85Z{(LIMobvAdF{@1o7Q~T_SZ^=2xBB1%)w7oe!u&@@11@+<27he z6W8Bx1213Qe2Q!g^9!r+$sbl@(u&{w=GUAdVfLesO__2q)9sNQYg1pJaoQOiNq%TD zPJ{ZnTl;G+K@K*ZQ*i%4j-)jJJ={Obx-x9f_YZGqxz88e{gTPdY15~Bw*@lO?|%X( z@#=v~5(pc6ZGZpt(4XaRDoT8I(8gn$PVAIBA3`L5jWfKNB%+*SzZy#Tq}$(rP<~bSJ1FhFeEpN_Vlr46K)3 zPJ&o9KNBrZCy|UgN5ExmMs#o}feJysN_}|MoWSlYu(-H)lEHup6kGC5TytZyZYeri zIn{bzYOP7lCR+{c-;dGVhaGn4kAD1vRcqGHdGY1<-hJoiKmYl^{@cHeAG<%RVKFvr z!lX$=lXJ+iXd-cLb!70ae#VUH%u>OMtqT?|eEH><{{A->ruf^Y4I9Z`P+cDm@k#?o zu1pvqVpM|(wZ?VGwbZznekzB1@h>9*hQDArkYDDR;>^W+gLAV zAN&(V`Lc;ROJF-e)26^#vo2tA3Of~p%33($!2;2CVZ(+^tnTUj%$qycBeVaP^Uh;0 zX(m6yf5HhTSR5&{eEBlTH0?a$oAch(nelkjO*dsw0~=14?{V0Z*7*@5Y$1Mf(HUEU zS;?BuKIijKJn@7nl%lzK?D5BrKKd9E>z}7FE<^q$l*C9l^UO0D(_z>oK9Ya*QAe}t ztu1kq+OToGbHdf?>go+X>2xf0$iaskeDJ|;EIR(u{qw{VPh{Q%mdwDGd|*OvJ?%<2 zemqmhEJXuEfud><&Hq&)Di3oaC)r>U#)lP52ojWV4>X1uoPux@7_ZvAsj!GynhEH_ zzi}V3DaA!p$WLcDp&w)UHMvrd1N#A!+=Fv|mA%nmjv>TcGL2RL5X*c)&dCQUUzzn4 z<4jR>#o`VfG9(bpA{3i&*U?-?GJ-d4+(bl|&s~L5gz_y(ym#+j2=W@18M7n)U4A$F zDo0Poo@93G$}jLE|BvKb{i*mK0jtfLHOt}|oDl!uL4y^^h~aaj!qKCHlUQu-JS%R7 zpULelT6g@P{p<;jpQ{Y}a``5k;D8?yZ1tNcm?=%2?(!r5QGAo_Xg*4w{BID zNEVk+I)prA(Cqt<+h2x|rQ$WDjd5%3OGt;UlAgU7M-h*em1gX$+5PwHuRG}LmoPs? zTkX!4Eo(C~voo^tY2n>w-T4tfZTx~No&UY~zo@TzcWyj8R@Pg(b!148{@=Kt70jQ|DN$Hwl>G#WBa@A z?%!q<)4@!SrdG=&s@YApf&9#zoLFY8wIRM^!8`Q6S57-_%`*?j+gsb~YiaO{1`LYi za{xbF{!D!*;*9nWrv z|8{=7sv*|Fe0;{gO1=8YRO6QA#0TXzWMt}Ncq^oxEIWQTA5gh~o@`!@ytOg@t+-k3 zZP(SS_%W1U%SrVo`PqLf+R9v6>wR0;w3#o+PcVghi{SwO$JPF-Sco3o-&*w~x8r|$ z{mJTthaGnK`t=V8tn8~aEDpk_!TasEe^!>t;GX4c%A3gMdL;9~?zVYr`9^TooejlJrn=TXrF|w(o-}>Nw9=B&HEY)J0S%vBZ>ri< zv$M9NITdfsbLv;I7so;Q(xas|Gn4Tg`ajcfj2ty;#WLN=@MX%>V~!=ctQ6)E z$u}?HPX3IWZ>F<@^fB)L|Kn#r<9(;g&dtp}@uU;n4|Sc)!i~t?+FxA}K$FTo^dHMk zUgJsS>*29dSiRDt|NP+c%T0*xZ$I^^BN$-f5*)wF_xm3sK@WEz^m~WMKco8D@v~yj zv(G+Dw7uOwcj8w9vBneu?YG>_R4`rm-^2fVtpB6(o2Id4ZiWk)3i#+zemDHifbj;t zk5qnnVAs6^s+mPTj`C|0l=|bk6w?McHV()||HP$L9bvd27{eCbpo9*lRCd*V1B)Oy zgdC7Sa^qJ<(#t)WUe6~%zJ`Qcv$FJ-nA)1_$ii!F6o_@5u;5%c)w`-OY&yXWbrpWU*hih*G2k0hQDLXKdO8elq$HtJA)98LYNde(dOq^J7NJTEFY z$sx?Lmexq)811w-{=;LXA_CKhM)7ymGDIS@i(TbXB;Sv99?G}w63+F{jW}*R-~20r z$I;&Yii?Y#bKo#zs3nwQOEf24dWN#E-%)xpKkX`N+Gi?c6VX@v%oAw?qkAh{qp$;;5rO zjSy2$2$Yd2L|R!$myFv)oMAp>md-zM07B=>h>;`ay*ZDxn>JR78neaHWx@9m1bp@c ztuoW2e-eJsl!N%#)pKSTj!)65tc_<;!$v%9!h`+s0+HsLD281mZ=R(?|0h2}&BgeL z=77MRIJd~8MSf|i-h!gDbz@7YdPTx@+)qJK@s5MVl@c?5(CPqn3LU76jzgBq{#+>&3vWZ^N^F->L` zOqg|2W>IgZ((tPuA;2k8I{k-04;LY!{QO>J4Li29x8kZ~f>)upzZdtrOm&wG>z}=| zwqeVX_g?+^)yMqgr%kmr&0E%|+FLXG4oepHN}!$?>Y@FWT_73}5-F0>@e8mc5llsU z@$<)Z%mcCki#`p!>f8l1TnseuN*TBr~mic^vKS52eZQlP>>AXnrM=HOPW)k`3-LejY$~q|fTqYMmvHDFsJO8E# z_|t76Hwoccv_Q1vN`mF3V{Js()BnW?8$|Gn=SyERH$Q3Rck0w*c#f*6*@;AX9???q zS6_XNS+4X_&9-#;NPGQ_*SBw@NwUr4=M@w%$F|{2uOB4L&H+pZA{dOJ@D*(!VK4Ye z%q9wb3(J>6L;sYA1a=gA5TU)5ukcusjlADn$=Q1qNLaG!qG)Tii>;mWI|un&f4nUxo!PQ2Mrk5zgO>~ zj+RtI0}H$~GXq&|b={JU3+FDL+uqW`px{q`>ZmdMk6piRbyIC~OIt@%eFKw-3>Z_^ z|A5K)#o6=!y7twY_e=W~Y{>n?#f)VkmO?sv4BAy0N~?ax2Qv%PF!IcQHzqfhercjS>rmY4Tq z9z#wRE?D@+>#zU)&A$@`FelDDv9M70lLk!bc;d;Yv^;_l!iuva zhL1p&3X_R*Tm~-w#^2u9S-lf&t9NXFan2lOeswuf{Qx%ie-Af}jaM;b#Rsy|>Ib4l z{%`RJ_(1g!V%R!MUs%2r`ccZSz9glvM))NM{{M;c%gZ298fqmrCDF6cnOV))LTNNn zhP)oNB*p&-RNn8gqFtT_sTEc*0P0qgBv$x%5Y); z=!72s`05p_4rSIBv8znDCC5mRGkD;j?|t`smtOkq-F3UEqh~+-2(v0wZQatVu#i{c zUpnu677*!O$`l3~EyCJhD!8heWcZ2nd`z17v;SzUsi|cGA%+IsapxU1J9jeh%-c?x z&Dc>zh6E`9KPX<9b}^C@ z$oDAbpMm^Na|E-o84N)oiOW|G${fME=P)`x@$xG>xFuq#+it&&mp>7iLS>2=B=b+f z1+Vz=A3v8a(!!qBbkks`W?!zDE|>%xn@)jsViP7#%*oENxrAenJv#fPIWHeG^_WP0 zV|~L9fAqsh{(+O3?Fl@R(rMqQrVjlRiy409?f3)!30C$3z#k{VtUsOWSD}8@bS5e? zEQ0exfWjn)1aWn-1*u|(9(ouNb*jzOmcw`+fqWJ-@+g+db|yS3zd_?IXeP8da@^`rzWlrg1zC7`t_HO;k~);;=_^Cn*O_5KG;Y}vY@y}3T8&yYl6X*|D}CCUO|hQZ}?*p2^m z{4!XYeWh!5DpO0nT62iY;6!S?K@)s{yLeSoqP^{i@Bg^HwFytk$Slk#>X(sMLY?RQ zNTQobl_J-hDUD%9>xe zaMiunry8kO8hL&jIO(9a>T10l%jCwXv;4|W&3_{K=JJS>NUoW8Bi8)W-8HBmxN1>M z+3x-$xw`69pia8{uKXyLHWf9`rL+7BglYA^s{Asp=c_Nq=kg6PpNN?`f$1ZSP@0=L z)SyT~^oX3+!or|8&~=Rb)4a{<3Dcb$IkcB0l~_iR>H*=#UK&5*^Pecc>HOBvz- z#vWQ(v!P|f-|M&3t>4mAl~NZ z0kcm&^HiULKEn@eu1i%fTCg-TBU!t#ap}BeZOv`vm8D0Yb3o4+d_plO7t_n{n$@en@|Caf{k})w^4Xt1 zd+OAwM05OM4gr3XE%}Zg%X8AL{6_rz`Jq2ccOhXi{%emv{xdpu_PWZy>^tAN)+XBL z`RLCNJ;WF8=Irp+rT(`fCt%1?ttgkbEv@|aC_m&!(GAc4RQXj267SkfC4T?O@@tca znRQI6c9hlVSSpGam0R;Ci>j*2ln?YLQeX*qRBWsgaH&T(=Tnl*TA}>FmMZ!Zpp2xI zj+BahuU;keU=zv^SlNhN!kM;_wbnj2P?cgenJf{?r`IE!{)t{CCB1ro{|Ddu-uJ$< zYSrp_Z_Ru2?{j~3&99h*f&uUc9(drV=q>!zr&#MmebCA_fRUAxU6x%|*0+qW7_PeN zN~VFFJ8vG#{Lmx5O{;rYM(ViYBOqBrJ%y=QMa3}Hm;mxu?k0?@h6;4@6<1RsrnGX@ zW}1im__bk@Mp{yJtY5cI%>u_g%9n(&Bp8jzcNzu3N|!*`W|^hsEw$zO`jr@A;iC+e z!?5!RlRJRga>Wuv7lKZD$-soBi1#Gech%}uI?*$BYG}m}g@^d11%7OiF{+HGjTn|Z z9e;o+#)JipO-;sh?Cw9@ee}^si5IIN^|Pn{m?Y&N;I-%`KixmIMyaA`5DlZhgaD^n zw@yJ~Bv#7c>K^6mx+W{Nc*>`xt;Op>ClEQ}oeP0a%5C*1B5B$zTzdgP7k-T&4#Ziw zS+H}ZK*i^m===}J;gHP<43dTAiv%jj_>b~cLS>|=pJ4gbXm!uUGO{u*V;OFt5q@pH zw~`*JV&Ot6m$fD&j7chQR8<8S34tP=@)@*s(N{0I;o9p+L#&x6&t$b`x;0tc{_VHk z`rh}yzkK;}qbWZ5z{yA9LWV$GhWE+kN zf5i4h`5OO1W$3(AJ~U-Oytt)u=6O>txTM2Ay2d9ml9|O7Lr=Qp4=eui#GBXr=-IEG zKlA=a5(T}P;Z2yG_OSd~2>1K}e&kzSCkl*D;w!(+i|41Z8gk2qW#$)VmiF&_U(w0G zqA9Ly<(BQC{0=(*-1XmnbXQ$oc~*`d8QWBHh&{6q18Bao`PRL^O$bmIp4pC=}*Uw zU@*1cby_9pwYH`;;Z<*}u_2LN+^g4+fy{Qq>{nU6OJlvtcuQe^-zgvNBKd@xe4$df z@gpWJ#=gt1(gdlDds10CtF~k^Vij-P|WV|A~HZQYKy2*fI1uhhs z058AB7ddLCSz}Rq>8_Y2+ftE+EvT7$Zhw^^ICH2eX9*6Xfz84~#_h zzohkee{1qpcGJqQYoi>rxq`;6PE6Cl%Hc(Cxq6f@3GzMrwe(UAIND5fppVB1<@?o6 z{A3@{ObjzLB5WiiPYCjCt7Ie!;Tnl&8#6~_6(n05I<_p>T{b9l@Pxi4{jypbVhi6` zvvNUIc6qE|KyG1K?x?cz{9-2PO*FMOHng!qEyLw9+LO@DcDzbEz=3(W`Aj2{mz|kEAiHH(%Tq5rUA<}h z*4-P*DvM*gb0$xk_~`?uXU223Z`)RS_>cpR8dyHOcTP?=BOI43UNwDxD^qCd9Lt(P z`!24G!}R*cITatGQ~WrwgcCVCC+C~r`quZq|9!3)X94R@5edu>6<4O`oAfizxaHPc z{g}b6f)Yy=6y&pLKy?4Cu(a}vlfUxHD-vLCh!waGlq?hYx4-|bv=4KI$XbNq{F5e4 z3h@(1HXE!G!n~NZ9h;h(ETMpm89nA#*Zg;S{l-J+OE0}- zPK%RB)QaWHr3)OyV@#S*szqw$c;wxyS8rD2b$<99;n`=NaozbSX4zK$zz07tBPR|i)qyJ?Qs8MsNp!(ur$>-g z)rv{H*D_%?A@eFKLFS&2J#vCARbF0#i^?AQ&rnk!wP92VPNm*|d%ndsvxWJPof06y zoJf=@eY|pbCI6WbXZ?l^>(*|-Ul%M``1`x=hE0Z@^H6l`)MHuTr>soV9V->|am#_D zCldSZzds)|G8re1_1yE%6Jo*@#|s9thG%AEFkNuuBBlWn()m%fDCE&5#`LFv;5Ybz zpNe6_JQO`vtX{?3%>e6?TnP6nU;c{PPLRt_%f{_1D;WQ@Qo0>xvmMJg6%`ip6%S{` zVTD><5hJQI6vh9P+VAmW{mT0(d8h`p1{9eYVyuKJt%Ou~-8hOWQ54Cb(hrgv0J9iJ zBuI@NH;$!LK6lpVls_wU{{Hu@ZE^L_f6g&t_vSx-)kLskLz2PvGYhKO6}%qj_M6Wq z2?G)zwK^wc$AU5zh~IgbNkEQm%&jea#$kC`Ir53*267$IF7kinE`5ZHP5c}F;9Qsd zszN2xt`aBg8BtE1{^QIlQHaL1)2TFc`#KmhmCY0!?(L8qWDh~I3!c=^Jc81PJ#wF*6lGhpTuIPxx&)H{v{`zaLwJ6Md z-uNTtrTy`bp~7?1K9$NZ$iMgAdy|=+&nglR0^`DS596n(9AyjP6Vsjidkn$SpGnrv zuqi(>jc^jKNRB_y&q)dR&&|y($SK8!gpT3L=#w^)m~ykTm?d6r6_sV;ZGWmw$a07sXL1+z$3teu;p_?myx$s3$t* zBa4&0mEUyvZvP(fNB-$o1C=9Ygu&R52@&b%cA)U?{4BWs#`ajtkfWv_H0x^}EzMT{ zGdw}v2E2)581TsBj&Issv+%y(zVhQA9r4qv@w9Q16;KO_Gk z!m~3cA%E)iKTYuM6j;X45J`7{ybK&>}`@^nEwF( z;y>L(;=xmS!E#@yCN9X)o&(AkxknV zTYixm@uM7spOAoDKMTSWoDzbY!~>yQ{ya~5ub7S%rk z2M(M%b?S>R%<1T86>UUj;{W#cd++(f*T3;i=SPp{6bl0Hy_Y3L92f*M^WABuoz9yc z`A^8dtDdCNkV+95xCe10CuB!S(&g()=>Nv2;)Yu{eme075r`|VTV&w)3!IKvI;XR#}61aVNh$V zZT-s4yLYwq>DPPw$(4y@2VWU#J*!lzwthEj^ChYH=pbRWE38Wu!mLb*SVop^qvK2| z#8Ts}P0cKAo`_}TC3AAAj#+-Zg9XU>v}N1w=J?iBZZeOh1FGw*w{6+JdF9qI!?IXa z8)dhxs7qut%>LsO`6cwo; zxQ%HYaot%zm~(h?k9^^U7c$M_u3fuyMO4|J6(jS9!2ex7W4g=x_T92&3ldDYc1)kg z%8VSyWVXqpBu(`jogLGsPxlLU?#o24`tUSX)~B?JVJ3#(+F!|$+t{ok zkIa6AV?iV^+)LAVoS8xIyXW4*Vl4*~No#FsV_wK8z|*=w_Sar_9sdK=aSb!yym_yY}K&iB`*tV@(UwiFU##4jJ%dc=6COKZ{|DF7F zEx(d48-xr=PmddD>n0;8l5e=+m5pQ{qI{V~l5bpwYzEZIf1bVjW`WUth=r!%S6Xcb zH86UY`i*M>Wc8=>?+r`X*%2t{Ki1QsSg4^2GYFNArj=Mz|Ah^R;819Sn=>vM7jSGD zhQtYsB__6RPApskbuOXW6vaGJ-1bW1uz@8a8_$k^XQ?Jd4LR$)O1n;Ee(pe3Y8?ilrRh)2dQ?^cjAZ1 zpQv0|YlEPjJF4%y@4oZS`xnxf!wCW4LNOfbSmyC8*dt%u5TJ|FMH%V>|2fxn9h1;f zT6iRXNM+@M_ufNbVL`!@PdupvfuKMH;w+_3eRG941|grB>D1D!T)EQCVSb_9oFA^c z|M;nN5m4b2MC|}dK#b9Azx=OXvN|k5@XxX`efspV#bOnQO_3~LBOpi!-R>-0LJ?sI zqPghQnssYD{+Kai0uiw|U&mQ2?bfYZ4XND|CtSwxZ~-?k%!hckLGm4Wr1)20vu;wjat=J+gWS-LoSkZE zZfmTE-n@Q;M2JB**)C#__E(&N&5D2&P$J7tm^mfv|6KV^6prcWJtGnOb4MfI#9JQK zKinkoOqEJ8k-R)m$|aIMrbjhEJ8hZZzfxexzZ^@}kxf2G`30@Tm8vFQ zvN)2b0$N{f$i7z5N@t$;_k%$i-7Fu{N3|<_TnMRZ@f94JM<^lXM`*6PX&Q z*50DkZSlku^{9rPCo;3>`^ewfuzlW=)mv8W1W!&uMqb}sUIK63w0XlSK9*nKYe3#1 zr;ZpnqI}TMez_UB@4vlp$>QZp-q=(*cE~ATICfCZ7@*6>_G4BKO&9#S`9!$v==!3u zZrn&@^?(;rXfZNalkVb+F6QMWo?BVNKfI`&bXp7oksIzn3O6ii6vIAW(9>$FwJ8Ra3 z7k~AuNhU;4E>c_uXK&GoXEH3h5(7pdCWp9e=~0x+Lk~S<^_;HoeqC89LHG8>L`clZ zC!c)lt+#r#haY;F2@N|5F^&AxaCPBFNy6f0<(B};I^S98{?p|L-~yVY9U-!e5b4AY z>_;#aK_n0ml1m7uAGZ8{`Vf6-u1r3@@*8w^I7El+8Q8IT33*~YtVENcyB{e6@j`E# z3@F7RhYRKnrXFWf+msiB;c+CNmfi#h@?p>W=m!;?ic z__`q{JDY5{HPZ63Y=|&VF)0Cl&1Go!J}u2n+qQ2-hi9h?AT+DK6>i0(?*RNNp#*7^EnAMIDOk@0-9oFXNK3b+r*FX2N{FZoCAKy z%~FXU@^mwD==}KJFiQtNm^|q~#zE;M9;Z5BXo+*KR48N*M3#^Q0sjWTl3On3j>;|Y zN=f2WnOYX43&5Bh_hE$%A6h|Y2L~*SwRYW_p+hU&zJYv8FF_%Gx338kT&l4C{RUQT z-pKK)l`Ge+T{mdRU^4sQ{v#V zzx?GXKq9)ZFn{2H0cH`&lN<*D$;Ov-v4>J}2S3MtonYiEvf5o&SKMwDQ}9ANL=GgUaep;scLeEWd6) z)MP^v|7zB*;YD-hXHQ}Yar}(tLC;ED76EMR!|jqpCkWy_A{SpYpN??b(@pbI~x{1aQCt& z9-46Od4r}LTEF}KM1Dzb>3~G9KB>H7@^pSg1T-1z$aG-BX!^eUhuh-GjNt=$$;jO| zFEWFBdzCKYcEq+;Gpr+H%0(Be%S8^ENEDW^u1&=Mx>p|GwSIA{52yh(8R-636@%qT zkltCN64EVqLeNr!2e>q>VnB1%3Vl7n94pD(g2FyAyS0%4&Hp3*#dAj+59+DiIjOc* zBUnm!y`%CAI$owFGWwnHnRQRy-`>_*FkmPK&g?U=amz+7X7?(=7kc#1h@T$)U)d-| z_wp6gZ~sTiZ+!IkQm;Ou5lbDsHx_X$E+HpV^UDWI+)`!mv`s6=6ly$n+0>&lD5^*0 zx3(p|rp|7fbqA&mVmac;Tl|>q`DsYS_V3kFoSTWEaYQmOhmjy+b8M$b*8LkhJ?F_~ zxg=?34ig^K(tu>!XZx#EJ=|Y)@gHg9@zH)Rzpm4vg){j&nd%AnD{7>j}s05kapM?rkiaO^QtUwrX}HhuUeVIby%%10l6 zlsO|U|5!sq<1^1bs}iW{L;MqI%0W|@`L1XGS7!P%4Gh1)a2jag_?>@0w!#+hBl#q@ zO&(y!h6I^i?thexUGFA2NbD(f5{PmB&3{C#Qhk!;myE535=L>SqCp{m1Tn*H3}Z2h zdgu2WSbF%ela88kNHRBL{f4z0S8S&CntI|P4Bc2?wYgz)`-)9#Qu(R=gZm7Km9#Nb zj^4zK=H1P;>O5rpT7svbw$>yMBNQ=RMLutZMOFqMt}q9`LKyAW*3=enj3*kCO}pF5 zdi6`RY-H$OA|t!KuA?Zk&$x;S@!^yIUij)67o6C;B)7h!KAG9NW6kdQ%NK3hustg$ zOlbUbo|T!|(n>W;oyz!=9?I|6zxlOm zufMjrkxxQY@8@J?xqYn+2M&QIp4aJTp03Ug#ZWOREGn8YV@8klr`OMTEuBhjZH(0; zkrz%5_0O>C$Al~EY#&1r`cJloZR-A6X0u%~BU#?BeBi)A1NsjbR#|b@S)XGv1ivWx ziY1(0ercF8lNqcUD<%vvxso3{tDzLz)R5>*WgU>mL!U?F)z|dnW>}oSd4jwMGKnT3 z#&-#~-uhcbi^pGn_2q8$8`(tn&kl`fd-H#lN4Bo=2}rMh{_mDw?g#?jOn>YIi6LAv!w#G4nt$*Jx^j5WInlnBHjU#YB?vGzz3senwH z8Bqal)1w}ZA$>0$t*vbp6~jyeHN9w?Kc zxK5A+TKTo~#)jqok;^XvJqzjJExu8tGv$hOBg*QBbpN-043!qr8>{RT)8ur1;3^4< zW5RflDRFINZk*Nwd_IS6xNZ1r|U9OpE})v`(|HDj?;k zTmpQGr_I96?F2(gIhGGsemxB`-A9$lhbh0HitqzKZszEO$rN&|+IKI%%2R6VCBGR6 zNk)y-QT@rKbonMu5u6|0^69YCzI&G6fEK%h7lc@B$Kv_1?2eKN2ez}ksC{Muxd`IM z6q;Y8i!=ZLKmbWZK~&`rs{T4sEgfb1?axi@qCedMis3~??U{KQxxJWSG@buF@f(vd z)_G~z`F>`8uTNk7U)e+Vi{&v*h8O|Bp5<485-Pun%pt5&RV0wo%kPZ)9)0@wqZcH4 zO&rY^C|*Cq6)Dv^fnN$3ezkYZt13-WOsG9_aJ+R;d{b3MRdwrIxBhy`>_1Mr=+ff8 z?R9n4nKfmJ;sMz#m6Dxn03E1U_4*;b{La7pl8ipRv$GR=T2-?^Gb$s=)C}^7CpK?u zZ)>ROGj4)?E0Rj&7iJar;RS`vY0_&pEu#IMI5wL#*0ffUrc&a8QO&;Qrq)Q2toB43 zlQLxR0gL&fQpP)~Yuh%iZ|XO0yxc`yAsIQvy(!oMT)BZtfiW4B-{$(JL~*QS+$4tX z%gE3KzdS9|)~L2fZC1!(=yoqUCRb$86`%XN#ar-tfC78@Ljep}ekRMbO z%2ApAKUsd`Ib|Ic7sWRJI>n3|5XKtA`gjSi)HKF)6e%>(N)HH6XBuq}V`gIFMCEs9 zQ*3Rme3JeV@h{d%Wsa(LQV)m(QiC5--khD2jiYl4)r!bvT@~3@KUi77E%hQw=B9mg zgPTVviP9pKh^ia-zpwUJxtM*^>mLFQpsMTi_R-sSL6hg`CN%(v&VV2)xc*W31s?8d zKm|2jzJSP`5M`+Le*Y|droLoJjP6q<)K2^m${at1_n4_izxMj;?G5^TQgo_Qs$=fo z=gvIor1F0KVUqRBHZm`sSd}Q`a|RG$X-WdJHrKT1oRcW1RTR@a z%^cU*7Js395^~D48p=iX!u-@3bOO^qk?K{RJ9+BJi3g7!II!==^*djA>$SpOc|76w z>Q`LSr&rzXx~6zr@BH44d2Qnl8QC6d&&kcsP3AN-H?U$$b3|(l?d_GhS-J6syr%7`A*1?^pD=#%gzWtMf|i!Hs_ONX zBPv!eVNJ#l*Zwjc9y0(2b_ggE|HhEWhQ@y7{aKX;qSRW6xxl{8e^|bP@_W>gN42)H zczIZUWl%Fv8vlNdH7LI<@!QncjHpOHSG|16^s*xo%q6vHd*=^UpGT=jG|e7lE387$>4nQ@274Yvc~3 zqq(W6zP|a=OD|(&-M~Qu&pGFumtT3Ah+q*At6n4gI1ZhQS6+4H%P+so%pG&zntSd! z=Xf&&I=q8h9cm?m%9D~xHDxuK%;G}~IvVc0>khrUQyYf!$XV%A<;dCsBBzE88>UR< zblNzK_))Ocql%#8fS<_K1VF7li97i}{IJ7Mnt75ru}Hb}(o6Zmt*KdGbL}brop;}P zY0exIFZo39jojF}$?xO`Nq(!LSdL9cvo4&KkqI9PV?CPLk3I^1Qt_nz0r7wJl1qdF z`5mzbANTI9fgOlfvX9CD(Z{rJxh{b}~>M_lZ%VU_2ff4=i0NQv0_=YN??MCSjcmtDGJ zd$q+hk#=Fhf(6WZ3wnj8Qvd!R|4u^3ALiA?|1G(8MhK~7KgI0opC(L<=s?kX@Sl($ z#z~!v)8ZoAwhvH#9k+z*KQD1* zaWabJhxo;h0Qd#%0T2}(L){ZU#F8{?KtP|4p8=P~MZBiA_V;(+1-TMwa<$_)4S%Qq zbHPzaux+6ZoTzg2>}-#;Aj%%)*K**p|62LvtgC$;l4Mv!L`C>#U2uV>$W=(}jyvvn z^s(8(mBtTrrPmKV;`b!3bC6tFNe+9$KUDcu!tVG3Ig;k^y2^*2e_&ryrPXiUHJU|n|F#IZJ~S}cf~KzY~~aEay{ao>2CmepzRkyTP*SxocFEt^V@&y>$CSVA+0NL0@ z9YC;3hkCi086hK;Ib=Y_i*va=w& zov&=NGx?v&&FaWv#l>VQCnF=9>GCpJ3O1F)QnegpCJ4{Y?8r`bH_)OIyNT03tKpf70a#Mb$*R@BF`3eutbFZ!OVlS}0>!hm_X4#TUi^c=aE<>s{9&D@_iLrm1lhw!)foV`0%XySSH`(o3B_-?AX%z z*Qr$ia(&sFd=)3!IsaUW%9oD!C||m6X}ZbxaIU2Lzsn~Dn?274Ajp=O{HOB(<%CSz zKfq4{;t7>7bh&>JA8cvuS0J%FeoyR(Kp-w$rt&xW>G)aSebh+CvAQsiA7{Aq+`02G z38Oq-dFkaA9xUlYGWW2e~k;V2@S{h?(~CZ zOd4=#R!e?nLl%QL5)6z;cC^;C?poO}w6fo2mwmISZ$VpQ?6En2zVn%1Z;UN3A681g z){dRqR=vMs(}t?Lrut-FqOrPp`RkkCd~Esrzbz_1D5KBVj=#))`r#)ZdT-s!+Z)%^ ztZI*MFFW(}nGj`Mb>6>yTMG%<@?H`xlz&|C;9F2g=J?9H&|L(RsCEsG=fYx^`Dbyu& zMttV@X}|yd?>4NjVl2j!k3V_QMHkaeIC#iFCbp!DnvoIoMk^{Z`EJw}-LU+5pFi>U zci;V+TW;nf3kG`dJpbaHm#Q{xL^3^~)|(-yB&2J@(4wzPAY6|Q){^LuPTZL%A75Nl zEPdJBpNtCEB@8F7Hr`FKm*S!~2(;3G($K8U`M>f%uYBssr)=JA1@a*D`d?rF>ido)IJYrjduJttJW` zIqJYOV=;Bbm%sd#TW`6=635PDX4f;$_`&5r95Z?p9Yzw#?P+uKiWMu~di!m<|23mB zfhOPeI_-=zrXD->g%{>Xp;Y|amtOkNpC7*LJD2Uh-~K}@DypluuU)tP>A(Eto_p_^ za^MtZBr(*wY0yFBW4a~pJY`y}#~**3rEu4-T}N&W%}vZ7{=y3{OrAVhIfKAF3JJEU z`}dJYXAc=V^w_CWb8>U(CR?$5*>7*X^@ba7bmPE3$IK9EIU4>%;)*M-xc~k?GKUjF z=FOeQxXl}Hyy@^G4rgVbwQJYTd2tRsAhor%4)TmM&!A6=n23_eI7+3~u33wG`i0!S zsDuT<25&VrHLF*z#tLpCBG_U;@DFtDoLk3_0(KzbWQWO|ANY#%TCQm6F)jv9ouU@s< z(JDuaz>8l&PsLcAX!V-aRB=}CP)#L<4XYsEK#ssjnn4#BHZQ*%bmEuvkRRE)v;G`4 zY7`TW+;a1+ikwQFe%k40opt69esDR9Mll!-v22V5U9oJ%yYIcTe!YsD;}7JE#=w7w zsX+DsXK$w-q9nvL3?35Uw^Zi;&}(`*ev2UfUA}1`jUS0h zUK;Th@^3bktiU41zp%OrCY@B0Z#mkrVK6h1&@LMO@J*fYTU#R}h6Zm+7s!cHNtW!G z$EOOhMD2=&@xu?Ne36fR`$$r2Lzs^8k#ixYqyfYFAn{OR_8%*>W#nz@}qrLJbfgkQaJ-NJx2UBq1*$?WN~I zs?!^JkU$6_kkCR2E%auJZQOg6C98L7_y7COx%XXpK^J{7HfBmec@^#?$+>I-43A%Akiae? zx(}5<*MGp+^PTTz8p@-eeeaT3<)rb6+L|QKf6ANW>BJR5pHqe;q?i}c7DVekf6R&y79C<*~kiXsd z>CjFUlvGqzf`$^{GF^hFS&V(mkjS)jkSx3BTh68uuw_dV1IdzfQ5hl`{n-uo-y1JZ zR*gzpFM_!n%ug{Q8L6Qt+5o~@PvtTFnma0fzZAYY9hq!~kE8t@zU)_-Ui=fP;9X$L(vWy#B;1Vk@uwB`+b4r;8HF)cE7y-2B{Q zeSHI|q7r6_rz@)6rn&r6LN3009B0IhGzl*%C`BEcgbDrAlKnHqZ>+E;Q+IA+-NHB% zICx!=yi`+S^`pI(?XZCl)I&iru8?pFqWE3319oH@k}8!|H=>?eMfVsQDe2B;7mUf& zlqm*v#f$`w@uvgr3so*r5w|}c-q_L3pGLb{o8@jmtpD%tTt#p#GFRlGmewCuGg`Cdx79HUh;d}hjvat62 zlBG-g?I{DvuPQH27we>xPTshA^Fxn36tv)yjm0qw-tmrixJ7}C_{pJ2<|nE1FY1*} zDtQ{tI~$yqFk$_#Bs$=fn=A~R5Fp~;O(`71)OlsnrpS^Jg7`9r(s{9=noiJ|4@o4V zN*`>0^YE3&1Tn!}CKqlHU&p(MUvfzy3MeS(W95u?d|ZX@Y;5XiFKpS?)z?FBFjg`; zU0XKsu>B9Ls;t=A)O`D`4|KM5wY7C;6aABB)iP~)M`Od*#_c!*T0Xe=$*NJMd(NMl z$Rx{)in=;m>sz;!6;<$BsPQ#p>&8tu_<#fIckG}~zHZ%yzutd)Rc&eAym6&v>1S6j zVJ(~$EsJ_b_Z4SZi?nOylWVuGs!vy?OU4x!RuymCwzZ%)Jz?yGzLMVR1tn!w$(?IE zRz9|A>YlZ+&cv1_?MoV;*zR0U~>l!;pWould|dZGWLmCX>a2`=cnn zBKy8_@oOf_??0On@f@j(x%29uT>mkt8N9sDD6^ zhkJGp1>|@CR}7eV4o6B3O*aJBC4JeJ8(&k;2`9~*Ic?Ug87G~5Vt;?{laDXD=ia*?dGyhbeeC0H ztt~89d(5%N@@y4vT^T)kv`HuvdtuxNv19|o)u)_#3MD-D=wr+c;W-yx*WxeZpb3?z z;)b@|GSTi*+3GCE3IoPQxVZ`yqJ%(xFG~q^ieVb{?d&AS2H^DCRqZONoYNdsN65yA z!$&{d@5U+gG`&z+w9XW_yh9kBm?diKZg zAAIORo-R`hAw-qv@*l82OVI@Uu;UxfI&_jsJrWKIp&$4+0E?sG}h4LI1HM1-C?T zeA^9V;7iP?pNB!+xpd~sU;1zOS7tT|=HIp#j~{dNF}K}*n_uCR($z22j*{8Ujc}nQ zD8>w?Gn})ODojcEFQ@z92j73?l~?xa`6Di5o10to=%Z~Ui6kNY?Ad$#*MD97wzs|A z4eIju_Vmo2Gkb`ArH+e~U9)cOteLVYP%h)uS6zM9*=N}qFC?(AlCW!HNBBx1#SG0* zRZyU+dSG1t+$ER%o*}m87Ei`xDE>23adO>c2ICr=8h6wy+`z0JUCqQI7n_XY$A}Ua z!4~LYN2;IMfwlbl$7(ar!&Ok=IyZg?O2Y7tF;c5{wz$ZztC|3UjQkq`Mt#3!}Ouhb}#(|&#R2fx5!P~@{8e2;1zV@qVn6ut%g*B-o} zZ)bMXlMfy6p-=R+w|WCe3JI)8;rC7PlS&mn`^RfC{e63#a!SRX2OsjCtM0q-%#9D; z_?I)!IOVt3@V3o@F_TzFP1=w-+yBfDzOm>#Urvv&NL7?9_{=vZA8~AA?38%ASo}!8 zNT7iKJo$qJBsIo!4HjEU|QA>jyb{tPIotx2kyN|Oo!p9l7mtcXAfs&GOnJ%UWQZ}TG{z+A*hW zz5f;_kq-2Bk*c(I5*Y&f8vbDYzpM70o{S!ZiQ>zOlkO7C{=8+;u5Cl`Bm2H~;#a{v z;nZyXA7d?BSQCI7VXef1liI6~Rn=q*Ns?t8EyhDG`XPl`L{a?0-eX&EJrTl#D$v1F ziYzjq&<$!%2DiwZIATmkNhxpK9Kru1uHfJLg@7qk%|8%9br~6*)2M!k>K|PKHm9Oi z?GbE){Btgs|1beAkYheMgxW78U=Jn2246L-Ku3~4EKI3MB8hD$6d@nqR14}iX&FkZ zA`G7s=x@Hvx5j=16$S$M`Ti4+AA9Vv{LZw6^?uA4$WIfqdDG@cAAR`AMT^vP(U7kq zh@TY~9J*k}j2R{#{76@}Tz-%uluO!p`6eD!&F5n%jI{V7frR|NWl2pkY9&VgkxO3~y;>tw-=#JS7 zCazktdHJvIYTeQG{&O!9`D|zu_8(x=?JEloZGTDV1P*V%3`IbL^{4fpgY)1!k9Hj4 zn|*`(3wImvNA@4fDE`Cvis~O-842IwcS!$%gu9Yo=e50T@hkR%&HPgyGz5Qm{Bqgz zMDZ2zFIGVaAQ;@@M)2W!M~(B8Pvj@@kkTpICI*p@-_c0@pZ{#7Llw$|=4S8;)<>aWTiwNai+^ z&iW^U1!6nuV<;?~f1pUqqTEkC{S=RAM);&>xPTi!-ZH(dew*%ZgH7#dgfFSY4R@8Z z7CrscF-ITm_fU4$BiIT@6TFCoAwQA)QioC<+>%82$hOzsdtLj7Kk}|$#7Ab{2Y%uk zPTID0D;hZeaANAXk^xRvXE(zTXfBoG@ljQkKl$;Ge)F5(NN@8`ClP$4y6EGd_{GnD zUQsbhH6LR5hp}rl5Fgo--gqL94*AlpN}&#pQjkT0Q;N$kDW#!E*J*vo$~uxTx@3OA zrzU`esj!LoA%~^R^Pe1iu|??-xLp!%VPWd$zx+jIMMWsT%I@pw?ds}L*_5P=j8fW> zt#)RD`Z1UIFL|9Gb0Cw450aS7A}uF=IjJZbiGT4SRiv`Y^Sf&8vWdN*EFF^0dbcu7 zfX_GlFl1xVY7klrz~m8VX8!#BK6}X}gZNfdNUYs8Std>|U%vcZ?|L_H9aBk$Pcm)N z%A!agL@4!w&U!vcCA|ar`}s|HX#7fQVUR)0>A>VSs)Q_;MjPQMg=wvQD>ge}LM{DE z)}wNMI87SjaozfL&po$v`?l>AClHk(IRxOFG2^?r@vB5i9(?EekItI_AwW<+b57~a zRJJdMg%U~(z8?iuq46KpZ$VL>*cu2+EJbqT*YT}pGz%IRDkvv@?VL(Rp55ac$-)Q0 zIFT#qpOyWUAN}GRKm6ra7P1D4PUP0l0WX2jNRp^K`HjuXS8jUp5en6fm?~-4!0zPC zR2>Zs^$*=!R8n5Zgu&u6V2}FQwR?Z;+f3TO@g4usvvtM5_6<8%EM0lk4{v+-Nx!Zw ze(uWevi{t(BaS@h)<-9w^yXCEY>jl}@Gr4t>k;DDP0cmaaftk0mn$c18c!e)yw!Uj zaQ>>MvHQQiv!P|h#@^*?33q;y)tw}5w8fYTqkAi6xk(IKHHuf{QhV`a%k-)B^@7lhBH^Mq2x&E`_50?X3!9A&8qe@{2LP9e!O08yVdqHn= zTVZ7-FF|D7F~=0HE4XqgV%fYcAL$WIZfT-gGAuCP>A zESbnw?lqsMOBjw)JH&#o$wFW}JcqZxJl>orjSaLHWcqsGKn!XeMNThH5HDHCE%>-o zj$C`oHNaL(<;`M=;)=@VjqCBybQSNNRr@F#=I}pad|3h{rb9p2KH|~*BY&__JmuQ< z&lJBUdt^$F-~|ve2QzL^f`Kjh75l7LiaGQn2dn-OMs69DbUn9p#5Xeh1=U>VKVRJY z$qF|1Ay(DDsz}DBq#9Di6>=q=++F-)6!Q<&&oZNKyj7ZUtLSFmIJer3U3V{FL;g7t z)ekSwzQWXIzeQW0g_=iz_^E6vVH`hC{^9r@zyC=4OIadL5}_3T24|>8pV) z@$VJ4F*L;gy(o$1?AdS8?C0_irMrq>Px-?8&t?XB0=(qKml-8+lM(rGt0CHpj9+zL z;on*zM(Besk|cMRu)CO^j^54dw?Falv(G)X^1(kXzTyjaJp9L%o1SiZ_~sQo4Vkvh zJfYvUZF>{Xux#B{zq4g$cdWaiy`jFozQ4U-`{vfQ8@7$DtZHc6@%ZWoA7A^xx^2rh z?O6N3GY>7_xPtdyZLM$Gw0YZ(ZB0$}?ak{tr&rH8|Ml%IJMc%PZ6UeVMJR8@BD(-q^8a)3$Zj{qnCX*Dfueo+uwzGQN6d!(&Aq%a~tU zx!@QkP-_z{<^a-2vAN6k*Hc>>89a(FvWN_255DoAVAn9M>GcFUZ7z-Du!XK_jwnf9 zj`-abz7?qY**zhWKhOVNLgMTniC>Cv|3sKves&+flF;}eyYpkcAEI8e=%n?}`Q~u< z@v96DhaxiU;E7!S|L2Y0BnpWo)TG&h5pzLvEAvOwpAV;By#K@{B_ zc_hds*~G8#?XK2=jZC}jrcw7w3_HG>3Zwg-F?qp z#p#lovE%ohx6fPN@|HKA@d(PeavP&_ieU;QeVtu1DXo|Pam zVT&1N1^i@l{0YY~Ic6Sa(DNEdoMnb$~|TAAY7zn|klP z_x$9iKfC*`yOu6nvTEgOEg@r2WMlB<@FNaC<>XV&IOBAK4e(Qi=>X6d`_03*G7D3M z=HHU1FjeIIBaYj*Zeranx7>XB<(IQ;$Fil%nwpy;aw!+li_>YIl{o6Cqf8@dJLo?g zvC!#^nYZ3{+wXsW^=~f!4g9lGrrJUcX{if4ZtS?@PdNUqZ+$Bq2${cd<3=5P&LmU5 zBQN5j;~#y@(LeviFW&X8clj~z&;jJmzx~bTSxF)A!?BnkfR)!??KyYPJr6wS;7dOD z*$Lw(aK`-;S;c>j|G)b^?>Y9k;~4pR=;4QXPKUD1XxZ|L^7-@jJ?rdqK5*fML3+s* z{vmLJbgY}U0?Bf$x~U?*d&!;B13e5H!`?Nm-dvchCh;2MHZ8UzLv zOKHdOQ#=rFiSy)lkro`XVDZwWU;N@1Zocs*dxf|wVR^H$Y15{am6f{e06M;xD1FRn zou9C2(^+j?HveIl1bn*uMTJG!DYCEh2-O!?mJ$~sFAHDi`9Df2wV25krukveL>P*n zE=qd>AW{W|t}?mEcHh1C{m-|)#V+_*;DYzQr?jMu2aj37i>-e9_7%%lZr-vf$R3Y< z@>8FB^MCxut)NIYi??hMM5Q!1Jd@{ci!ucJSvZW zN^$=4;#c7A*=C5yzNA;FIrb$LN%`;b72uOD2#}HMXP4iym@dK$wh{5UHi)0Y4~bp= zD1H_D@^r~>o7HK?RdB2?opk?nE#+fwr5tg14=M{jd9J+nE5HvEFZjm7KOQpu{?B~) zjBEd9gW0@VJj+^9L1&* z0VkqD_O&y|{p`B7Z5!`->zU2#7BzR4mzN}_)y8#Ina>El#~@zj#Ih%3)-*E*Z(## zx;)0y3=9w!0Kz+2f?HYAu_%as9q!LEV}ctgF6zlFR$&mYf&A?|*O4-v@_vPeG-OcP zS~c(DML(vH6LT9r@@uFhn{93fvajxtL+u`mhfGriyyQjgm&Pvw`%;X#iu-@8ww~d|8(D&!A#_Z{Fb6Z6v@|I%#LN~k zQmZu+c~t~PH2$4tn0}OL9jVwrx0avn!D;X$te}4}e|xQVvhh8uvXe&XmNDXISMf{f za4VAC(wIpRrThHc`oS)HA|>PxWD>Bm`}oSWua~3Dq5r!-D3F4ieBgjKd=p?-@(UwJ zYWGh+W4b6nU4HcOij6GVILz@MPLS7bK%oo1`Lm~i#DWib2rnPsIsN`Z#!ij&An(9Q64Ak@be;&TT zB{-WHm69>0b;~!cTd|4SGhI?tkV=+MOqW&`$H(=iD>EB1Teog+WP#?g(WMn*`WEbS zP-#&~b92-FQ>JGUy>xPxtXRg34{nj_7dN!Fb#(0LN@WYHYsMUQ(7}&A{OsAspSNiF zlb8MOtM#p0i&I72{R7F;c>9`;jn8bIST&i_Q-$%q?*6XE{%m*Gyf;kg>}r4P-eL$I8zwf)kR|d*_GK4X z12*JAd=~)Z5%DF&F#8HD02sXgk`d&!`SMG2nR8_MBYf~Y4WC}3=`zUA3yELTLKO#S z@{K`319>C=A1=R&%kh6#hI4E$q<&M4@nYl4XmtEt@gpkSs6+V?OES$M|9SF9{zJ-K z`Tx=Q1Xiaq&+B@A{rlb&<12XOcD#{}UfS2|hFKS*X* z0(?3z7XE@#nh@W>J#CPKH((um-0|zy^VT>0un0qviB^Oj2UW4!R7l8PebrTOc*6;5 zTnT={o4H7#AdxUs2uueo!l&zNCWCSf3v$-oHCpnV;L>t@y9 z5~*c$&u09~OLOAO@(a|GDaJ$X0{$a@^6&%d4B0QqbB6iP_U$`Xu3W*Zuu4lx%14z= zuA9QEtz>s?gOMr=$|12vTrUwW+)|e`MJi> zRYPL~OmQ#rr|YkO;~U?IVvg_pkd&2)&pPMqhI(7Iw{BauY-v+dGqc+&t15W}uWr&LZkWYM$d5DNmVe&(bt9vr<2yf)^t|y~ zH0Ka1O-6OQm+56&Bg*e28|FV6zcMN^ng53s>Im^=;SnieOcH69)~~`HGTK8 zM}%jolt5nYS>(?V2}>`({K>C>U?5p={C9pbVc!KTs+sL)5q)mZQ}s)i-2Tz`F~U|m z|BwSeaY=gi{z;}33q|JR^Digu-??K0C#omUtUK}4DQ|d7dQ45WpeVsjKCi3Tb^Q=f zX9BrtMvPwrkj#<%zT-MY{O88+wnYy={NeYuZCsHYGm04yGwXOME58yDjEqR|$O*AR zZY@Pa_lmoE_L;ay_nZ+5wjGVxE%iOg;>zQ{@)hKd7ZxXKCXbr3M>btf(NX+fF{Lz7 zRkGh+jLf2gwbI#Sgr7EB+t=K3qnd}Pbr7rv>dY5T#S|H0NL z@8A2}|J=IhG2YBlcg!ikx_|CIek1?PjW1M?RIhIQQYM7t`pT*aTZq6V}fH zu+9B5l0UxWl33&a4)9`2M!2DwEIgg0qFC5c@1sq``*s1METD^I|NoJ)q`6-2s1J=E|CoxuL4D-u!(OrVx=(ie2t3oYps!} zo?dHPD+$$9I8aXJ!4skHyQ53JrpWE>GlTVMWt17E1E*qWRf6hU7-|}#Iaam()V^MLc zt)pwp^7{IfJ7dXs<=EoV@x_H(E9y5jB?|hJr3D4W@#2cYoojf+rKh)ZpkhqPxgR)T z`n(CNmu|lAhQ;kI9g7#O^w^UR=nq-Ls1DWf)4BcU$iBOZ--u*`lam0#{%flL{7T2K zQMF*)3 zArC*Y@5>m!_dNPI@k1xz$V2uWUVjd6zdT4oRpmdCe@5|Zb&%6*+R%O*yAWe%#kk*+ z$@9<(IDV;7wG>!I$rR|Cg=BdU(K!{MfsBan?VD#(0w9%7C(L*TuR@B7OD=r>`#*E> zfAOO^JuG}ezpJ=~F}O0Z7DMK|^WVx0f@6<48Zg26eA($7 z{*}n^{X&HAMF%H$9lzto)r=imBT<9+GTlS`LH-bg-@+-qG<##ya9|;AmQam$ReR4 zx%ei-3yNQ$27Sg04&`@^ltAMzA|coR@l3lDmYDpF=%0iBV*-;hBrL4iITc>Ym)}@d zZ5`JC(isjTeRJTVlM%j`Wbi>2;q!;53zFKNsM7C~j*tAiir=~KdKc~3qyKZkJr{p) zAd{SZ#@XfLCw1-EzUsC=_qEi=lj$i3AAP_lK3_0#Mgi|+Vati%Q*L_nH< z>EB%6TT{!T8@y1(3$*y<*%nQ-P(3Ig;n!jNg&0}5>+NGSyuW7tetO7(`i+9gtKLWu zznhgs=diM?E2Hy2hpvO}vK%d!f`yjexuLgnEjXlGZZq`GpTR~Mx`gcqj0 zYY_U5#FS7jZY;L+&)3BJvr`Y|=NT@@=Yq($WLSUI?|ujzcNhhQh2s{Sy#47XVodl- zr?Ue+smjq(@%ifKsD8lTR7Ti7My40VE|5v~0^47(27M+}$oO|Axju6IM(uCp|A>^w z&!GRvgG5$H*ly!@_9qAK`BOo>Eu)tWGntXs1ydTBB_*Ytg&z{^CVuGZhj_Pe?b*~>?Qd+QN{h~#y7fl*lci`-ODoU#xJ9hTO+L;V8 zW7gz})5kU@wt~aDY+YNsZ(Vao)3jR8uj2-sHJeeH)wWAMESu@7O+2(pFM7 zsYES~NO!D*Q70FhR1e9kM9(m*uW-FAJmofXDrTO? zekYp1eC_NO$aRT`*4}e_E#mAzEsI#!gBj)y$X@qa) zkt%_xCYX{ULpY-+7vI}%5?dB&s!G=QVc$0xR%YWL`Bj#Ql&kO#8{Gg-* z{$ea)8$o$?KBF_6Hf^$!BId^5x0#=J|IGCt&dcI{mt*)ZI(~I;t>+(17|D|xUmWVp z(D>cs-S3)y&bhapb9novRV%N%+U7VgXCO9e^6bO@`_js(v*J|~3W`dQ9N}Dk;30wE zwBK;;J@@_R8z21krH6m|^Qr3b46cJO<^NCPw`%&F)9!t|yR+@~(_h!J;pt`THPkbA zdc2^hK({rz*~G>eoPC9#zV$>4aoW4rtmNXA{r7y~BPU$;Q!SR&-QK@_eWJLM*9k=N z8!xrA=ABiecx=Qv&xTK7%-e+dPS%CsWwfQ0nI6^|#XSp>>5@c2QEulC zDcnEJ15|si{PAUp(tc*CWP7?f$qycyhNwLmlV@!Pc!$Eqfz}P{$vgR^Gqyi`dv8Zq zrn`rm_fg{}=C)t%pEAAbLYE&96~!g|3IuttN&!8XG0NqCo%T0Xm6&nCz}o+b_oGuG zDyj)%^%&YPO~K)*6up61wx&{*ePd76$9fCmOe0_jM6CtGVzO<#k3XAH%W574*+fM; zTbu5Pr^@y7mKhOV#b-|Y$f3?)c<2@X7{@$A7LYaln_cJM&QhfMC&+1zZb@W4*-*mg z;M++{@B-qOS|+fvf(p}pzkF=6WV@5QOrd7u)`VFDFAiK#WsB=<-Q@ zN&?<4neGTT*#0tY@W>;NT)lRU4?9^OL)L;OvN1Q|*kg|^)=EQy8L|Bh@dLtLQ{!JO z!+eY1@}7&sJo+lgZw)i8Z6k3rikd_82u-DJx4C z6t=Xs^8gtSCU>;(18YZE;=#-O}09&T@R1DLZLS_4NJ5R!F2BADVpAk6t6!t2_uc{1M2o@q&wIuzQy2n_a&dCx_a8Fnl@Ss2ZhGD5Z^gR#9ld@m*tG{jdy{N6<4 zM{%)OQBl#gSO4ye|2VU~y;CJ2s>)`MdXz?$Lp5|gJ-wg(%q5RM{`l9v_O(e9Cvur2 zsy`^h=7U`Ghie~sdF9m~7xMWx$hc;m zm@omxFnr=9Dh1NKh(IKR=7U3;Wy$X-CUB5TT1XO4JEHudHZqC>Ad&6JH9V`z2=Qf5 z+w5cxKjcAvf?ZA(GP<|65m;b@|^sne}L|yyCa659tqEaHuI5`i~4T#JA{p7|88GSj6dY?p8C$6@kf{i5NmCW6_p^&Yb1Wt#pS2n_9P5l|GL@j zTQ)tjqO@*8cI;@x3&w(IPcg3gn+evLE%rCjWqG~0l?9tuFqam@1@#AeW31tWlt|I%+yYN%<-+!^=kUpmS zV2|?nQwzVc#98IeYJT^v;5^Q5~B%P`?#bkNang zU&wNXq~Ocm{!VyveB)KIo-M=$&!DLHL}$qw1GO)#R7y?g?YYJ8W7}eV41Lg?DkK%1 ziYL9XR;j&$A2uovSd9%k8`g6NU2dq12Ir7`N;C7>X}- z$*Vsud{y7VwvRIUx$QpXmL%1oG*S7`3uFd`B#l`2HdwRA$ee8bF_wRQA06+jqL_t)iUVYOQx8L_?X2+M56f+UN zFV1gR^P#mNU;f>(o{fe;@>`u1)X&I+Fj4Q|Hq@8cW_|NM< zYi38Fj~WWKJMv6B8>lKHxFIb83Mh%-`Ef^iRhar8CIAOs$(uAezdQp%&=1Ggu&*f9 zaJxYny5Y0ZqqRX00tl3?Sh3>If4T9lzumEZ!$y`b)#EMvq!3Iw zGZ|3O1~#l5k--UboPhb)z3%m|Kl*5vuh(4LpnjJoQUk#(oAD!xU0C;6i7(oMDG!cs ztUL44$MOrqfW<%I=uCwS^YfU-ssyAG0=EdS+H4f{HcTYGc<_c6+18PgG`10+e>dP@~$x6Rc;1Vh%Rt4so{YAvFQ@lhCHtdkc?CSXHmM@ks1$?yCGrGOeR zGXy`9e>nfS@;h{ZPk90p|1OCYfsQXaenF5_q5Ogsu$2~`cRgiaXE5|19g@K*G_#S_ z6S-88{~vbv;hqW)U%7JSzVr7z?bOqbJnBf6v+3>a?da&(uwla={_y)HOP;gr0=?)H zpI{D%9Xho5cTr73E662ecbNQAG?9R7q!B;=NcqRDpPbJf%s;q;g*py!ki{DIlSF$o*Qa8T8ua`Q6){u0b*nanlw`|!E zFDTpleV?LyU@1<8T_?1Tf4=xo@|VkfLlDvIO=eE+-sHImcQvg1!nth+oO@5!=G z2U7v)0$39i=gvu;4vkKkHRxo=p54J}wK#%!S73-u{nIOvW30f)(xw1qC#AcpN|`Dm zmr?5Hi(iVsC|>!Pmm zXLceu(3BU2aYzDENqrAAS6>XrVw3779eUUyH{X0KbEbhYGC56%7o30Y+q|`P`IS2C zpX9}_*@!V8{oG1H{)-i*uql_sb3#N-c*_JD61nXpJvmK6&tj5`KUZx~ekt-=ZGUlO z=FS;u486)NxNC({Xy6~lAta2UZ2gA#=9L3yUsU|Wo!zVnJNux-%>80(pUG}mo$YEB zWyTIjZZTNPw0CrE+E71s%%+KB##L8VcV@c!OMCV@X8N8@Q(L=Q)~(vw-xAx|+FjV6 znm(^)?9_^Z!rq6TdGhgPkM_npifU6#U23iGSoQ3hqT<5J$)m>27&EY@Fw-9|uPG@g zj%|9H6-oOU5iYJS96Pycul=Wu8#{XQx}9J8$d4yX95Z!3FQSqPG6P!Ku&FHT8$mq0 z|Ex>uu%o~s3l2W^nB!2qVMoKt)vMO5UcGVSCUt*-%f@cJ>88Kk^*7!-Ib+5Q*B1Ng zf^Bl4(0_c7OrGH7ieKSVqA93SL;R@zd8Ol5Mt04OV?-dnUY-84^Bm9)5!wR#ih18c z{>@ykc>LOsf^PXD{$GasUla-SvAYecSg3AfZV|1^@+txj^~FK>PnU3HyOX5%Aty1n zB{Sb134=^=ZX|pX*lFQ_Pb!tnx5^hB=j|pao=T;D@cr-YGjH$yY#(n0ABgj6c5QAL znwSB8j)D)XYj-y6c>nv~KW*C7i$8U7cTcCwPv=T#sKPAXSbWZ$ITwHGlYhPSuS=IK zdE)UWcohN{C>r@yP4h5;3>s{bg+l?zBLUzk$DjZFXGtnE-3W-o;rt*Q=j9whase9t6w!`C4?na2pu4~NPR z1xjT2N^kh~H3GhH)msSspB=LLKg$~UKPA|q1;|MF62Zh#*&+XteRU>ReiP8~^{LF3 zfP%HBQ+fO-*+}ta=am=79nL>jIX@!)%M!l`I~;#k{5wA`jN6y&dGhO!b28GH_2 zL$(Mu+Qotaew$Y|hB@q3*B?*FCOUii0f&97xftujZvO!9kYadNZ>~W}S2za~{GTGsrGkPK94b;J=hY87 zW%a|qVEfB>#>6w@{74zo^|%Q zEF-F9Ir7&|s6FoZV^qk9{d411gc?3a`TQH(Mm%y!7Uw^&eb;%FYxc_FCx8v{ou7Y# z_LsVmL8E~IIW$_u6TC%^H>vR)z0b*)zPc7qm&L|UE+{U^PMcTIumy+Ac6DR`&OkB0 zxV@RTfFx|Pnm&z2?&@iSyLvjBJDZ<;_Ng20xb=<)?%ml~pYAPUoVly4t*|yV`-n-0 zykYN2)2b%zUsYI=*tV(Z-uoVI?Ao@WesfbpS5bHQX$#(R^6TDIGAiBO)!VeZqi;1b zQG3VxJ2P!tdpnz0G&nnUX60dT+-skMr*B%mZ91_-8(Hffnp!<475E ze>1Tp24P+u6k*t2v0B-NuiOF%5;`1WND6BpfAZs>obskq8+SDH@f@gH5^@;6-W9+T zitG~t8lz#46*qqH!-c>4^)F{kXEB@^`|p21-Q>F2vu5+2$ll(b6)RTqCe){%dTRCR zHEY+d>FpuTHS@NBWl)8yX&&b2fJp$Vd*}ch$LH-Q3l}b=@-lUu7n4P4`4cy`hZNcg zIhQ*iUCj-*D@O%TReO(Cf?Ie=B7D{W} z2AT5gOJI}E&;m(>16puWyIj>dQ`zeW4z=wfeo44{{sU125$JCcm}%j&1iNQH)Of|5;%uy>yK|tZSoH!$X|r2YtMF6{6k_be zK;yFIi4%^tyU}=QA$Oya6DYH9UpAA(SC|K;o{3@DfQLvgD@|^1;BjybIdL)5+o59Q zS!%FN82ivkmw;43|Dj{qrkXvgwQQ+XN4lNk{@C!DIl>ZNr8TTh%5|NjXOu|$i?khI zaOT^>_xN3L({+GNKU5RJROKQs8`POhWLS0ekC@{f=S_@7#bXz|q2Z|q!Q$B^7DX?r zlqo5R9FhM=^6!@a|Lgb_xB2yh#}^m=@%W9E&dpRF9dEcU+lwWb`~pqATC$X9@>$G- zSPT7gV@pA64;`zx5~^&ONc}ivlS3&=nNXC7?K>vR`?+d&z?H7B2V@{Kp$*q>qIjr3<)2QM$vPk>H^Qxoz-n!Nbc z6$>S7C{cpCAj`sONk$<9W}W|#UvX>~hWaPv8(q$)EQWIbEbT9v!asi)ooDu#*U_6{ zlo-0iBic$ieZ|YGGR->%{``x9s_~4bQp+YvOS7r8x~#sTKerti6$;qz>&&D(%lg|h zZS6f1XI0izl{~X}$>LS(Mpsu%o;Ysw#LDtfrE=(^Y}?MR9h=$*`g?g*cec5xu6oj2 zPB`b}qfhQdSX4jeKi%S>ItKE<3Hvkl9LVL?L$7`if~f(fG*gF#l@t$iPzt7{qKK& z&D-DpHh10|Z@lsDyYD*Y=wp;Eo9*cA=<4i*HX5hVqpNX90p#GT7?Nrk?4H)vHi2hj z#l^*?rMxIj5F~DDYSNuyCRRPB+MF=k-`9Wd-~YaO<0fi~IkRWaox9hlQRRcKW&(QN z2p?1S_YbUEt$$5Bo4UGqq2$=fywPyV6kdQJDUi}iw~UG2zP^^GW|np=FDv72v8$`| z?z`{Vx@GIkSu>A1>L{;rk?r9}9_F|9@#7{Odf0->%Bp}YnOllV8PnJCd7IBOPd~GK z{GzZs`rz|5-mV*yA!3Z1Xgb#Rf4*%PuvOwG!xVw}faR0yLq+ zlpz-{rA%~is8e?^ew5$qLRfD;Cxb~Kn&4N`(;G5HZn+~lWa<=&wziS z4C4nGJelKL`p}n^K_Gej>#S|MB*Us0zJ}r>^E)|!9 z&lxV|@b8?cNbuzYuAK8sUbkI|dQW4GS=*A*+4)z|On?ChYgR#!K9DigORPo{-9;_$;m`?~PP z|G(o`hLtDD^!gkUzg%IU!(2lCMoRaG$p0;~UC_nAf80ePAJByt7QZCR#Sac&Qv9k& zZHVO_ZF&Z0(08zY=DK8$UO*ol1`<&;N(Vmr6!J ztABF&7ZJ}FU*rc80cy? z+Hj0mXJ1zDtRCbi(bG#R9$)C!ZvkQzccG8MFZ=~WG^{pntsgb1PU9VkWTuZhVP^Yl z7KSk0+={^*{$ttAPdt_y9V;#2mYF*xZom-6Y@&3Tu3qk$vn7?Id%L^fjwU*lF6D6< z!D%zF9Q%6vyXx}GSVbgNI;yX;!@3SuEm_Y6o(s}N%n$8vig#|=NJ}&E*f%vTy0@>h zi?^i0aOqgyL?-|DFi{yf@*B_)`R%~dn?&|uq#+x45>ZkP+7~POX<_ScH~8`v5?CtR z_dUnYi(h1u;KB~@ReXp~7CZ1FwJQ*P2tSfR7j_-LvDu%>Zo4s&=*kQf>Cs~7W`2fZ ziY^z95M$mC+sfuxXSeSA^?q7Ck%VM&T-^$1c?k=Jk~T4=v~PU6E0)xXG7|fF+h6cS zh0!KHf%JYLyK2qtnm(kOSTp5bzV=r>VWm@X+P;RdRUA?b+INtD6ExsfI=|HTwKAl; z{sAIQMI}PWhV1LVv8WVQR`ae1qi=BD)nSZ_Quk64#1D-RcO+R4&1D;oY(j!0Z zuQtlS7IGXyRY&m8)3Q;)dF{JN_OI4|mILrZsZX?l0Bw4@$L^5KAH)w^cE`qeNpWo4 z)Yz0cu?drt^;)tVGKCM`lpjQM6hZ#0m=I1yu#rwWSB|dqB1)v2o+ZSWn5! zNmC0G$*Q6;+3kI6mTuTo->{>;y{oA+)*Y*;DBWky-iJ*;v9zdo+4?2te(H>#zFr0b zdON##TCA(9HC`ln@YPgFYE*ea>A0e%tsQqhyQrA;JLcDPwr6hYu6clIs* z`=;1MlAA5hf6$PV_;FtU*~4Ek$aN{AKiibvVl#HmIp^GR%Pm)3b(LTE#6_QY{P8Ck z9K(SQI`BY#lNN>9ED6p!`)rD^vgBNLTEyV#Kl$m;KJoF7tJcZJ&pr2?E3UZ0@gIEf zK?WDtC*$VLo5ze9^Q~`v>$~6mUQ^RfV_5RwpXB}Wm%p4UEFA8i?QQKp|K%@lz4ewy z9(uT|yVD%qJX`Ta_V|e>zTtD9``jT57TB>VnazIj3!lIA(#t6HSHJqz5l0?*%E>45 z?mv&y1GoxHg5d5TW`K)-u!(;t+qjacu~gZKmYkZ|K%@C zgi*YRY86#lT6)w`M=o6WLk3(di$3z?7s~V1KT3ks!`k1VDDf<0-}3O4lYg&3|5PAG6t{7kB zeHNE6k&_a@ z)Fp|g{lJvA8)PxP99vRZfy41hASmgozzEW-dDOsp9{#T6k4lX2(FLK{xFoT`w3_TBAtTtCO;qG^D7X@>__;fzU-zQHY3ZqA|YodPMrADAOAELpOkJF z&*J<@LQ&!8^hqg!YWmg%eIaHJ|F#=OUua*YRH|Te5bPe|yZpoPmBHi(!F(9UGl9&1 zB7Q7Y0Iq%cm_8%^bMZsSLHUslevn7}IH=)=xn%u-1_QT}loI((9+mYn#IKZ;$w1>H z@_#>J8hJq@_>X5)8VQP2#3hM2a5UFSf*-`|qj|5S+0OVG^dA*#3CU_6JH)<%R$Mq6 z2n9F<{5yWdnYyq!^@F&7#o|}C8;WlhmtEM9Ci0&szTEuSmrJCv??I2Yt3mw}?8|!w z?Q2=&8@@TnOOD@~z2*;ew5_@Q_Gw3-km>I(DB#sCls32-zC`^$kKgjTslAP{;;Mlp zZ(%3Kb=L`ftbIj%&H;U3OHfp3XS`R$mTvS+E&Pd8`;+Jx}x3m|JFK2a9%t0N(f+G6)gKmVT z21uoX!dUUV8EbC85!J`;y>H`F_a%$d?VC0rS#iyH3G0#$t)JyTUb&bCwl9MZw=XB% zKSl4qQu|vpF+TNu*{wgyb~2I$ohkh~C|zY=V>{m2pIy?z5@)n4ah4+`CV0Tfn(=tA zyhi;rD&Ve6cHip$>XLNe8zRz3!}iJrrd)u3QT;3|<+XomwirsIT>5WbX~=AbCSg3g zol#{%o}h0aivM`@TM!EezU96)S|p9?Jorxg<5%+>RLx(mkPzy`r9UrCD91& zZ%y@>-~H|?W<7i7wlI|{VWxrIYls?A4_iXnMfE$>6IT;&Yk z|HADrRKq_bJfco2OKb737SlJb@Q(|z^($hV*TyTyXF6K4&)l1>u8r{|OM9cjg4A$D z1~~U@efwu;({R4enS`dRC_v#qbIy|1jg zsJJ3kR?hpvV^imj8C6}%BaIJfZ)*fc96nXg21DKbZTNOGtBhQqaHp z!skDK%{A9(y_i`1nP;C}w{9&zSi;z;Z+g?WFTKpTiv8->mowNA`HxVI9B_!RtFO9B z?r!&(r@!fR5hk*;EZIgf1$ZpBb=y`RYJT8>2aHllqcn7F1~QjkcG-guJ$&!I_effu zR?w1>L*{aP`ct1IEvN0Oc2y8Ll{kAVzpdSR>kk$#{J;lzTdETA38r=vv7I}2zWI!| ztXj?DQ!0ypciwsDb=Tc+_dR#reA7*wV*`p+Bj5jl_dod1Ln<|_-^lRHGtZuX{`t=> zUFzPV)RJ2owYPWNa_g=4&7XhirQd$v`z{c<(6@{GFEBl;skN*46+9athBk8l#Kn81 z8dH_7*NE9o5kq%K!WPJbp)~M85^Rdk^sg`f z_2GvfX3erGX7ZC1aDDP;`@{JGP1g|4Hm4}!$E5Qs5+Xny+Grl|L6BC(M%{(SroO6B3bYU!^e>RaVZOlFmtgyJq|hm*1}i2}v0OIX?2_$v+$) zt(7JRUucHQX_apW_BF)+Vd{nRBhge-Xajph{#8&^@C)-Fuw1-c`6*&J{&4wUOYu8| zf0rMDL*~sGuIY>9C-NW9H{3sSO0nWzJU(6^`ToC*UsXtIC?dE0r-@&?Cs$?cP8OPd zckLG`6}|sJ{3c7wdYTiR>zBuRyR!W~@|xFh{1#N}o}1we)sNP0@r!_NSLv|J>KgIq zK8|HRR82w)lO_9^^sHKu`iCvS7)FL+2wk-G8W=VpFP9neNED_t*<;-ux7Zz^5C=MT z4rr>pI3|}lR`{(0jRJ73Wz`z)j7v*tt@{v++XY=SalmiuW^QNC#Df;7B%o8NOi2ZB zB#|Wg9)izdrlT`eHIS-I_AxVC@~bPQejm$3Xp}S7(XcgMSh~-LzOeOgf9UVwy{AQV z_iFY#ylcm1X_QPpum1q*1yeTL@cNmn5=Bd1ah#YY~_W~W?`>96F53WM2mBuoy#D>U;Q_4w%Si!a|H+h8(H;?P_m z0W-^kIMpzl{ipACNOfm2ot8CsA%C8IHBLotQ&otD!x``70TYO=YeLPF-%d&;{V74d z{{XC9Y`eB^a3`y>NG-+mc$G~?a{l#c7{l>Zq+JUOA*~Q1|6xB!;3-3(5&0phnB<>+ z#0rWOKR^tVUwB?LpFCJmUN(Ak6)&S5J$iIWSsB`d07?`Vo6k=mzm0CGypVq~ivmp@ zh)S`Y5Xhxe4ih3gJpw-R3pvn-Z*37jg7?gV3Q-65o*zHRK8zoe&GZZLO$ZYv#7B}` z`6UAP3=$Xz~X+6O=B^<^KrDgM}Pnf8|0y60ls*T*{Ag>Pl>tsO6% zYO}qyW6P$UxBm6s^=miPl}?;mG41iSkN@GB>sGg~8h2Rw_z&-O#M|dgo;|L1PIYFW zzpbTx(PN7rTk^olRjc~D`$nZI7wmoHmp=B@qmMbdY*cB@kPLK6=y2#%*0%cUmKOCYX^gO(Y%q4f10=4ECR?2rP+x zkv{rHE! z_{A?sQBqtox_Y$ZKlpP;I6PVe;fD_6pGepVjw2 z^uhOo7C7)o{5YmQ`OCpaG51@upIx#9!`l~$0tbIOdseAuVH zwe2>3X`U54p23UX4St}$;XC+ne5z<6aDh)rsU1Gm+b{u5Y`f&iEP-93H^j(^dP>0| zm#mM&H-irX>;gbxEfKRh*zci^wCHH1D zn*17oR-po{oiyp1-(PdwvBxnQ0qn#1BqST5%B@g^&I=^z%!WIlF<=7|&->802>Bf)y4~KdF z?+lslBiL6=!>}ikrq0cf_;P+C|FlGw&2kyCK*()hZk&I|FJ9!dh*I0!lQ=s_7ldQ7 z$p2p>@!Pfh(VpiX>sb2ms~NxQ6tc>k?&LIP@EVWbfkxttUmaDHTxWhnAA6 zEJ@%$U`FjDAY96m-_@gBHqYpLhC&8q;=fu_>wmto?JvP@LPq@SR-ewVc#tSAqw4i(qZR=asJh5@0 zBQbgO^tYb)?hD`a(b9p6AO89W-(L954XZact#58^Y#ksJnUGgZEt+Ay!ZB0JX6#kN z`?sIDXWczlE#A7SFlON(XTriQrH+#zx>yi=lQ>P(XP7sDn*oS=bd+6A?xgF zpK|yn8rj$N=9U&e`^k$wv31KfhEBF@-Fm`tCpbQ-SmfR$RQ$4|%)9`MXCqU|V~;;> z;ldv*U%s5dh4t&!vd~6LODp$-6KiW7zpuCNyWjtQ6u-I;HFY;{+RPr)jh|b-{Gy94 zf(StGzyE$NmX()1_}~LK{pBXKF!{6K8^Pj-``V9v^kcm}-PDR`GGqFTn{K(KvAJpO z+BMBhP1j%l=h|8xt2A50vS0YZ7g}0cynb{40bg!p6o`y)b9|{bwEh8#B*^cZ*_O;h z+AlZ#s}sLSV|O@i90?^l-7Uks(^G+HTd1=i2^UeA z#i~bq#hT~h40JO-K${{n7=PH7IM2m*L{q|(2+cgk;rZmqD`>kE>Kn)=31t*U*PeXh z@w3l4`^Fn@>g#94D_|rE!cwB-GjAm~5^Cvh8!90c*5VJr7j)U7tBt3V8D2zp zBA!a7jyUY_-~R5m9EF=Z(NOv2ExrZ1t7x4FE=2OHXtb~k5kF3mr1vZ)q32af1F}m3 z46Ftd#=oCOV+S#QNaMB)_}>k_*bse#o}d!R9pbxCk^DpPQ7=dHLHQ9%Fp|yib5l_k zp=cs;O-i%x{=-p-uN;ypH)lS+sc!gK(+*^AatPG;*P(5@=f~3oIx0UWMwTByE`a9# z?@3jhY~ifqpMiZNR)_IFh_7NoAq~XvLw=+Te~xeQ<@|8U%F`tW_hhy&{Y`$(DYFp6 zEl++ESqRSms};WiKau~tbgrgU4*1vUTz-&K`Z^(@eT^IAfhKmS|2!XlP)s0AKv8%U z(55tO0nbKkAcLBR=Z~*k`5h_n3RsuV*g5kkw`1u4ZrA6>cl$=32AuQrvc<0zV6~x( z3P1mj-+<=etE*$6uXCWcn}@R#C8epch1^DGd%9vRTlzcO#UJ;0#l-`ioyk$v1A2!S zKl1$F$8Y1}r;8`XiVAs$Di#Q5#pP+vW5w7m(lz`|^RX4~+2uBM>5t-oU27`>TIWl&Tik=Elvwv8OW%WnA- z4^GTJ{y3{ctd6A*&9gBK%vcmI`}L)SOhGywPnTBBoLB$&ecX`83)6ISGFe_V64V3i z1hMwd62J1a9Pv&5SD^h(X2+ir-+Jvp?-mNA8>p$=+HfX)Jz5`2GFaBjVgh=#F*P-B zr%)Y5%VRdgj7%BYZ(OFjgkQnsNP*Bp<(HQYZGUOGEWIq`KI!$dK?J=TuUa`TQAnll z0AFAHgrpB(YpCbNefKF}tMJpT1lp zlh8HhLezdmX*_MnkB)NrK_lT9n((D1$V$d}z8%xb;*6#Qsy7~PmPBboeB})A4a5=} zzNO-lLE2X+KRJXSmgV;sPIV4GaGoT@7lv=-HH23xeszDrh%B{+sG(yD0~(Jl(*IG( z){ge>_MXn>ZrO($4gX;;6i(V%gi3C0I@II@sobATGoYKSt()B5*`C;5SW{fv)Yx?7 zfd@~XF|Mt(;fW`fuidbn2UU98`*{?xJTvOZ6AnFK(h+;k*z5NDZu#?Huide|v8$;o z)|cuX=fehWOZ3inzB9l#=`?8J7s$`TLCNH=fA#D0&OcWfLV3*4e>8e9XrhEQbrCtn=pWvvX%tDX+|l>=DX;`WdHBtebSykw@A{Z04C~ zo)$%R|BD(XoG9Z(7RvylJn>9d&@01j~h46wJ9npI{D<2SlQ(G+8Sqjct7E^j`$Ir zB;n6C5M(3H$TM{*H+~r;Fsr(K#qsmi51|=^7!={9>B#V-3o17#ER-P!-&Kp`cS>j% zfDyL7qfKlH?M23~mW>LF(GC!7Au79e5$7gcy*#_@DVWk@Sxg!`rB+8j-|_Ki^+mB{`ZvWuc5HuSGDs2T!%ll;mnuDtKQ`_4Z5oVKr+Uj`rz5Xf(mL!~yA7;G+TaA~;7TY&F}RxU8&-{rS7p_F!je`@s2@Flls3iv^BiD8>(jPMQF zC8Wra@yW}UQ3f%~CRgSqmDWMiNI@CBUU-X&sC{ulZuR*HCy_=o0zXy@Iib^gsPL+txc9ltl9 ze{%DZzatw->1kOu< z>Ehp%P`MHRVH*2Z3NvU>{~JP3|NP6ezwuFX2dZB8|JXYZIJvI+-cO(HOmH~XJiul(kOVR2E&zxz zH7`b$*es2V?b+I!&1W`j*qqO0R;)hiyskM%Oq;W_XV=5qH~sdywY&E08|p4H(~&7> zo7ZR1f7vD1{qp*6{qR3FZP+|ISjZ(BJKDRrTi)HhCo#re!n{qJ>FZ3s4W~)$ zxv#%_*8pnbk(NwO(~&j`;6=Kc4GC8BS^I|Kkc(&zlaCJstb|xKw59_Db{i!VDM(Be zmC^FV>?b*I-W<7%B2t?+ZZ_JKhz&S8J3II6*~|IHO&fW#<3%qzT@jQg!1C?4-Tu&n z4;o~7)$u1Bd)#r3Z`47(+n&;ce${8MV(Eu2$Zatl?H%WwefE!j_#>jbd%FAj`gxMe z67V5;0cfb_kvc2S!>lk)k^hvFPd@(mRn<-d6S>yJ5x zokz>fEzQ^iPz2ALJO8rF{t6la$!0U7quOd`pr4LOMe*;WAO46}fzQu<{_`_tbV?LZ zLjNV?V;}q9uYBeCDk+h?`s!~o-w{a6wFN$N$IsLh4r$Yk1l7m&74c&U=)n)8|7Ehn z8!~~wS$G8V0)mphR3<6;kneJMnO=U7(b_g*?GI38Qyw++mZu7z0*N)H7_=3nYZVjX z8%9u`Rm{-kGXXw^a@N^rZ{51}Q=j@IFQ(nOYbVx29CB(DqmERNv^=s1a_it+GaYnr zoRG@GWuB@afi@4)+S+!}MHjv8ZEu~{*(m_YM_NAu3FGoRDFWflW;DD;CIGptUivl! zTkRrD(WD^Vp~di>wkmvyrz~ES#3qD)x|Z<9=6L=O4c}!8^c&)tY~%4&wBcLl=i*9! zlgU4oMDj=$9`^CL@%R$jd7wm{S}e;en@E06v1f?`PJYt8snE#?kz;JQu0yJuP_NA94HlR z=XRyi+^0z411TkQ7L?EUC!bqHZG02Z5>Fz(iVpA%xe8yBIKCx=TP}@ZmFIE>B1@h) zKhlEPu8JphAi!kuJHApHKPFayF96F9h?fqq{fTpnC8j3-LWs-nLYjq4`7auTiNJq9 z_|8jBk$eQxpfUYQv!=g_eLi>mciZ*X_pSRqKa|aR#j9=gygjT9A-cefP&8fO3?qnnXTKuwC-tqo7Fk)Zckf&^$BVp@Lei>wZtX`Fc(n}YoA9$$w zXO~|3@~?k|uD7(mTW=uEXH%^+lWCvt80Wt`e)YXnQ)^>AU_3~wKAOX1WcIpT7{v}p zS{EO~Xj(F7>exzA8<2bZ1FIAIUk2=h8++G;F8lTFjCe%OAaH z{k1=&rZbN}vHRYe*wC*}4~i%2yJksO@%EWq)`nR#1J*eCBmR|(a^xh4XVuV0pq8-x zao-8>l}dQzQb-(MSt(b475$$Op@!gmN+MkgC6GVFvt)`? z5&=4-BoHQF1QVy*VkN5LpCHnW!&fTFR>OaZG8vT1;QT&EC95pu9Eb0;RpFB~&Ofpd z!aozgHvfl)@3IB_*!`^C^J=jh1&SYLz$G^JpG-DR&o4P?PFrJB&))u%&t06ZTh5}u zNABCU{lPuEHuV$=?At32$>UU-c*;G}UpTO_x4tXCZ&S~T=Pg{&F^hFlzgu%l_wa$; z{d-!M)OB~))pyhtTkBKxJav~@zj?z)|Mf!;-@j>iZ=t!qv1|I=G3`UmlZpLw6s3|D ziDdGbbT-+)XLO`@cx03*-~{b4MPqif)9JCSegSrBjd=-1bq;O(O!Wh!79>lQEgQbqYcJTfew<*Z!J<+2yQ{`H^wm7I6tXwkhi_i}Eus`x zPF!-y8%##W5rCp}DE*~xz4R?_ev4m&?>`BLOPV zuj84~34h6wrFY(W7bom~GCVY#ZO(#&^4zXi8gu3Q-={1IemG}3o-8u)_g#5qE|((> zmA4rXtB-1>V!HQrZ`r!lib$Noaq>yzmxQF|FZAC(`IGZlc)}*8glylwV`y-wq0yH& zj+5W>M)FH&N#S37ihmP0#8(-pTnQ~k=T-jY2riKlDX~CJwkN=Uul?HR@f`iIF6jb% z3VA&K%VdYZ>^Hj*$m(bPgjT=+D}ZQP&~eyHYowj+?=1jpQ&&yY)8`M&qP;~nq#=YRg^?|kRG zTefbIt%(|X8>X4{qzV}wLuxZYNW!7EiQiUS(a8i4t*~C`C1;;=`FpS6)yF)NBRSBj zPtscParr@2tiaX&QaMTyKiX`+#5mIU@itGTS{gAP@eJ&IwXN^T-OmrBcnE zpBTQ9n`j2bIS`f^6!4Ld=#XT@gC6OGy2s@=NhG;L@x*N?zXbB|xcnpw@NHD0Y(Tp! zszQN&BYrG-FfKVbKc-)oLehXGxIiuyVo-tnic;D@Kwa52<*FSC?bD+jG~(Z~1^6li zEH6uElAq5weqf(gM#PUQO-Lx8C?ZeLoGR4$mrwz|(khAUU#hX}619Tw_8-dc`X!qs z7veT3BaT1LK5O~!@cvEdmX@2|_3Gc2Mw7$k zx#zud#miq_-_|iw-%?6u>smWg4GpPmo;SzpF&Wm=A`X+O6Z+pZ*L?MkE8m&w$T!rN zmmXQK`+Ujn-J;qDJixu+?lr^YKK;E{QWx-54Gl*NNJ@@J_-lOOvBSZpyeYt`|3s_kTOtPu(*%XrdQR+_s&!_$14N)|)b5 zV|ED=?Pic|3~YuiSuB?qwImj_u%)ejufh`4`yE36Yq!&4JLp49zc6n)Ihg;t-K!+! zeH{I-ZAl;$vjVE78!`5Iv0vO|E3ylZR%z;Ap6LG94bbXLRa9V5eGUKA_XSc&66e&@ z;4iY$kUPb4VKwn~!88%_LwJN!)&G)FzucHIl5CE?6hO-{9Y9DG=qrvAu|xVo+zhKUIybt>-(DJaAzD)~#E%ZQIVFy&^S_ zms-D`H?5f9%BOTxRsEw5?I*hM$OVE$;s8nR&l4ylvG#{-FiVNbkF@8SnpZ@fxzy0lR?(YOI(_XLwxYU6;LU z`HE$AdeffSpw@{%x+*zQeo3op3cMyiRw=S7H>sQ-Ra!Nu=0{2ei$+Gq!x+s5002M$ zNkl|rw(;;?3`{5+Q$4-^Vr8E7;`Eh*GNAeq#@CkdS z_-|bPD*i={QA8S!$iL*j#*6b$loSOE5&s0hN~mAcf~5%guja=nj_|Dylin-(9gi>7 z#`GKEkCQ(%cq!%4@KuE=|CP3-Xt#eThkxmnU>%Bmy7_NmsKBkZyc%oc?mU<8-@LAF z<`J00NPpj5U;6yX@4O<{*fcggq#HWylbdKvlP*i7Iv#%;|VH7DQN!Umn%K*5Zi$rm;afM)XZd!QZ+v$8XpckC@ zcjKZK>wfWTbhPw;T(sjC*Nhbh_CI(ZYmvL2_x#;=-bCHg+1x?e2O~j)KV*%@KQ8eh z>DTNYN+;UC1tP1&jqu~`&qVw`djFeTa%JL%A0{*XJjBAI_E|%7gCvP{4sHtsnlFX@ zyFtmn{I(5HLM#v#gZ_n+2tVv+#Id)}2kC$H?1tKK;cL7|DV)aSuW3I@?j{)1uLAX_ zYMO)T+fo8;_ypMaoV>{IOL0{*CK)Ffi@e(>K&V zlF8+lAGd(niButd|81L&JZg4xYtQKLSazVUaV+1GZCd-#=E6W>xVJE{eVEl8`9yx+ zq9fATbfGjl&^O3)Ff{SVp>_(Xc$SQpj7v7;()Jv@+Ln-|l-wLD{uTKzsWxtSgmkjna$#3w%D z5ro<9A&Mnx-aIWlG2F0!A${n-M;vj4xh-JH2OnIwa^(usoyr4K^*Ln!1pz4aUjiee z`bTRTEF^&9OWqvkN6BOD!_(G(HEQ8NBSFKcJ5?vjs%eTR0YRn=PE^zIoM5q8zkD^{ zM2)Pw%>tYx843xsK*R7&1Edm<#0W(RmBtV8Yk_=_SFr|%#Sq20uJvH2WwP8P*a zf1{)o0e+x8{Kd)~tK*p}X(Cn>QYB+O%n8bi{6_c`PkIZ|gIZ=_bqukuATBv&$4 z@HHFdRU_VHwbFFP|ImI6SZavmkMv8LP<{%qB0vf?DEW;aP6B+%2}vQo5he+3CMAxK zYnkjqGg(ycumn4oCQWEN5k6@pMIeUwQS|^YnoNF^E@CafpSYr(Jn ztk+NHa^HO6DMy|EvXkHae!Tj(A9(AMm%p~>p*7ck;IfN;b~}Sj%$CFnz4rutV7&Ho zg6#K@`rl30{XA8dY;G89Zpd`3$TM%kgTA}=m-lw(OZmj${+@@w`_+fPo4BP^)LjS; zz?0JIN4N*$5f#R{Y_ZREu8>SGnw4sA({^1vPQZ&-`O>JIu}w;kc(5zKe5{nZd)*+7 z`Ah%p|5)!^=GpFC`z$Saz~khDaxxRYyzGM4{{BOMxp`Z5Mw?<(tb3^8tC7;- ziH`Qn)*XrMd&bH;H{bQSf4)Pz-E;q20Ee8~p+gHXbEUC#M;n{vi5lSS298017Rm5* zOmAjvJDOFBy#Zn_o3^4!OixPv(C~O-}-QAPNnKLnKB-@`d?DX%y&;- zl*sh)_JE)N*%`@n>HdHFOgX{c=P;DDMwByakW_J6ZvFn{or1MWYRciUG*X^>Chx>e z*0*=HEIf+W#vD$`dan3I?-)F(3WGPg)$FJCFi7i8XN1KhZsMimBXn<101G0VR6v^ zVsUPCHXcy_2%*EHBYpjS>~DjP?U!s)>xF(=uIr2Q_0l~l!N9#Ak$xG9p%gE}3w4o5 z#&gr`GpRJ*pQ?O}SQ8uwCbg1PJB0~zbYygBaA1^WPd2_Uxqh*r`5emU(PI#bIP5d0<_}REnE^%orhxj3g ztW?aZa>ju1ugZ#Y1?%=%uBB*Q%f`7dJify5NtOMBc5wd<2vn&CZQ>yn0m9eW(`Ee3 z{!9`}qfRXeLw9x=XhCh*aH?7Zh?b5SExoLU8cSaDAHOSacF&y; z7WbCAH~01T6f(Txs6N%uR?l-Qh4L_6fEK=u^3!itaWwh1_yAL}t#dSn<{uj&@oPCs z6Qx?7Ck^sE3>_Sc^!R5s?y8i=RZB~3S+D|Y(A2K#J|ehO0$nKOE?B%40H zQ|K{#oU2UAqJv{o zkzD|~Kt;c2_I2+a9vbOa&&Lu~Rh37P8(W(4b#*hQbvCoLS~mE#%2cba3(+Ec743Bk zNSuFCn*sy<#_i`&^c&$@#tHlwY8>C?=kpNlC&D+R3HVl9;g9EMJU)pc z`>(BEbmsUI?K9vf!jI|K`Edy+)33)$0P8dPl1T(MhV>hq#I^DtxBt5Qs<0oC{e<`` z#-pB8|D_0HEVBQ}+J}k!h;Vrm^3f)MGtNGz{CCQKVrE8t8k<;@6C;D=-u~>sNO^R)P%KiAd?A&}HZ>AZZ=5{A3ZtU3`z%$vtKlEohu#p~Z#>c8|4KRTl z(kF(J<|d9)D#bn|EcJjnO}i)wqH!-FOkS}&)_BCU^4L&ucr=;KOVLnWV8Mz_f#?-H zmV-wT};(qmheB6pNENa5tFdN9Mn|;&Ls&H~FPEy;p@YR3SGSpedAJ zp>DqQmJ2R?jjwq3r5_Q#2FpGtKry<907tpCz`?4jiij`UmqsCiNAkAzj&FS9>;BMI zynnQ9>0uy{UlSd|my1jsp9pKb|H|n($YG*TS+3&vCKPcZtaktFMlH@P85CFfW@&Vh9vjnUr7}XvfpSnlWv#Nkpl?(b* z<;Iw#)btF`;jpJihK-e&!y&!-g7|lPl(j_sNFl^a*%kkF?Sy&(K+3A}U-;5jd{YJ!1QReNVhIkI$XH~}OLA&##yES4`0 zpW=2U6XV83NnjB#f>_oV;v-eiyh;M1^yRY>5x(dajIwjs`{Wq5G1AL)u;p%hJCW!h zpxz{Od}b~UO}6HaDzPWvdtFqpRUn+ACriK&bEGxp=NTJ{cKJQ6%TJ}iBt$)!VOp$O z7%n4xON>C4M(Ok~&W{{4#5V?9d~Zor6aI@4C)uj@sQ6(j**5v({G(ovtCa zs4D!Q$d5|0hBB_-fWy!}0hFr9uQGKwzLyTB1P=A9gjMouSR(&Ym5G$e1@c!ICe-Xl zMN3c>WXd!+vV(7Rjq!ij{_FA!#)|o)`meR7G5ZN*8qbeQ7}*bzk$;WTuQBHQMC_@S z&a+ulT)%FgM33{YDgQP5fHddFT?QH>{-3k{%e}j9bkphI{QU>(=hdC^{(mSA^wDf= zxZ&rW$E+G19Ke=lop{>ZQ(ye$1uHMU|5n*us;G6bw$+r}|B(H6sCRc^EXnRv42YT4 zhkqXBpv9qsOJ(J_Y#CqQlxSp z=$4ZwSL`Pb+D8H&-pU<%qFfj}`OjaAL+M%b_OANb3lqT>YZPGpOU%DczWhULKl1Ut zi(6*1Z9D_RG0H8b?Qs}QEtIl}G{sZlBs;n3+0azFxxUy^Uq*h~I5o_4h|D3_Hl)JW z9aMa%7O##g>|9>-8vVe%YTj%PxAF?3m1dsqy~1H#aS8DHnKVf;L(A zr5y0jsE5X+SQvf71GiZtV!bhqg&J>$vQS7`iH7K``c`9NvXWixFm<*5T7VRj%Khid zPC4pN-`YNV`sUyL`uQLGziYqw`S!&}H_uyeVAF%S<`x4~(dwtd-b{k1AeE&5R8m(u zo}vA3vVLx2_A8SczQC{&itpZ=5b;i!^}bN3yFL`|l}B^QWP3JwWD7rb%eD`z|3z0& zCPJ;dw-Ur2S4Dp9RJTcibh0W1Em0^6jr5CZNufCnGd9<+wk|OJs{b|7rFq?~`e4-U zGVoubfL8m6`=~fNBv~_7&adt=H`K~l+muUw5`iNNRq8mw8HN zcgc(16cT00AG`gpN)mb)c=C0Qq6XQ za#WUvr+i5%RzFGctnPo=Ey)@ts?6Fio4TUyfSR!czpe604tBD{_?i32XLC#?)F*SP zT!wp;bTWJM-rFAh-Hr@zyY3%(@Xl@RGn;p89_g6XI#?L$nm_ISKdk@FwRbi)<&Qmn z5wCRF-`mqzm;3$i@80=7sE`4mVB^*66T^69n=B+*)5)y-e86VnTii>{48x- zb4y0E4oo5(UlHfy=bLZ2`MT?_lLyjoWwS3i=NyWFF`2lHpX>7Zi!Xltr$7BEc~+v# zvi*Ph$A9|KkAB$S*9QpVy#95s$4_*BCwzr)sF%h6HNdumAhSR-Q~80~^zgnFJ`fb< zR4#{7j{Wo}KjDS4E`LK~{_EfP<}1%Xp9sqv-2ZOdyhZPSb#SwLB+<7W0t?U8CYb!h z(%llFKf$T6z<&uTmlrQuBzv)jig(ZuHktqO<{GhNRa&xW36gp3-~j-eRR-URlFJA8 z7f6Uu(fBRGuhuX5LNcMm#8hP{LOfe7)ihZRp8og0W*atFs><-krybIZHIGuPw1J!0 zJvKFqC8L_u)s$k{g4{YcgR!=s+Un~HvS!fPrx+`Xh%(a zGlq)%B&^AQm3||9m)|eR>kLGf!j(i7{Sr`<|46??NA_ueMzSLsTIDq4KaO7|f5gvZ z@*|_C-Tn`be-QhL>=S+fdB*wgM?U-SAN$-F|Ng3fkMs*aPo)1! z=V@jiPQT)RmdA3xeEaKixqLd4zU9LoI`b1>KxYhf?!D$q5B~61uU~V!nFmb`H!ZG^ zarP+#F#B-}sIq_LAK10Nxsm%W-U15-ICz40-^!}{A_k~x9r-Uq;Zv)XldSsCFNzGE z^dloD^o=+jOBJ-T!%NdCLn94ylVfYw5B2qp6?^($dezrh)y#es`58yl&s~~oY>i7u zsbIzZ_s=$OFOMC#YwZZ*<1vQL11vD`mYE)_;+tU8Vev6;?}Wg9h4R<_#X2cDAW(F&f7FO9Hyipb$V=B#}& zHI9@>7GlMTLa>AJkH;XU#FgY4pIBl(q)B@!#uS=L?!42z`jQ01?&((EwDhoi$=>J_7tDea(RsG`X6Y_NQUvBeezj~};8uv*9 z#nL_9TEBz9*sE^-j7dzB-^%oJ88;KmfRz*T8wRH7`3xOZ{2bhW<N>)*ZRa<^JokUU^ ziKP!ZF3&Sge$X&nGapYTEhxQerdr1R*O9z-Rqc;661D%H1`x-0H2d`WgQQZN{}L-2 z{18iOp}4BTkM#R2=zpOUqI?e2o&nL1op8Z4$Lv`%yZ7}R*uTI3KyPnf@16sDyZ7(i zzI(@xZQIw}w{CQBbosI+Jw5xo_w;w~=sU1|aO*lAzVF-o(5~Bmz2?TB+`Vbb_L1Vy ztOcFveEQyd*KXdl^S9Ui;lF>mcH^3zy?cjvhb+t0d6Q{g+f%awP%~~YE^(Lu*ZZa< z2Oh}tkNl%)M>8X%qjhz4EzOM$4b4o6(7`Z~P8Q}jczz|zQV{KWh|vhgEhJE8=o&T0 zZ*HH`v>h@!dA(XI6GNefDkJ>B2R|UIFr#_?3tqt9iyo`&Vf*%$x4gyLU4?z)>Te?b zH@|T;=YG8YFD~XJvd=JBW$+I>bbJ*dQFL(P_-g+-C||KU<#7A_-uJ%emxO=hBOiU$ zD_;px90UaXE5eVmiz_?qTjg*N8*ml2KOStK4y*|m_*t}Qkqd-4H{E4~4?|4`wIsqI(8{WTuXy>kaN_up{F*YVW)vijVu(GNG1@lz!~P%1Zu zFV*PKNHaP-nW8+>88Z1v<0@39P=2Kf?Z@-RgMuX#y04z{?#r~Di;^O!b{|FY@KV<@56W`?r zC*nV>t7K3XFT-UD`48j^t0~T7{uRkD8AAJ4)r|j%_$p({e?v~E{MYTr?e5V1_hVOm z;qR~d;>SM!#i;*sCdF%Lb3*3aTnT&*w|{~|qjg^luH63JzqGxR((!CV`BmTiuihP7 zfA;n_5A5BQ=5Z4q7D*?!T=&!4|M_#T`|Yjxu*&h|A#QN379coW04^M#6K{WtgM;Pa zRC_bapEVq1A^;42f3tcO%SF9U40!!a#psJsF1<0MU2{g8HQ++B+HR^K5Kl1RJe{Gv zNOi%~M>2WOLxqt-vNSYw-j}~tE{$aJ4f*MF>SixaH?{M^6y!OG|JJonJL&Jf#E*h^ zt{Fu%W`U~kRVzI-WB+J>`03%! zowa8Fj^**%$t7xgVIoc%Wa zZ(o`vImRBj0aA6J-TiOl@b_Z#fc1n(NEZ`Bx8Hr+f&Siw3l^1%rBY#RWVkfcGrVK% zfsOa$3QlhyXn{WT>tYscW&M79qJ{pB`HK2 zp_D`Ke?@xG&Ik98as5i*a^e&J^iMb5aHDW_mH6NXY~n+unUO18Ho~@wr=5PfRUo;0 z=dK_9@Q2s_@>)eE5-&Xcg^Ly~R#5;*wRYGUx)3TdnB<_Q(Af>>!*o!&iMM#Oh35eh07+ySCDK zhfCgGkwJ04x&Hc_{`)3R>)>Q|`~U8DzO!xHHVK+YoN&S_l}&Xl&K0d|_kf@<8NF9f z_Y{-(m&B(b@pOWa{Hl~49t?`P)icR|y_rDb@bDVw<`VBQTOfwI2DV@c(IoK-4JkA_ z+}{*P;t*1a!VAw7W;Fs8D|}1lCZHjk;~S2}SPRaj@)|zU`ME_0g&iV3g~joe%kWJU zeMIuBL~e-L~@^N)!Mm12jLXLMMMVu=qF;dWdlzp`2WI6sK2at)sk z;)7VzF9-&z;jfZ~{8!-%*-Ng)PlR73e*hTTk6*_5nM{5aXNU7Y5kG)Zb*bGr`*hmW zv~h&`wR4M!$q#rHzA{)J97^tVa8==_ru-A}U1rButf2Mb`mMqzd?No!9kc(yFkO&i&Ck0!MA>P@?T$G=<7S< zUp~iLWhNNrzwoR}ZoP%klE$K_S`8GpF_O#6GW7@TgM})5qW?bllc(vGYCA1AyeaN5v`Iip#;IBli9?U?W1?D z87K~y+Lx|)`4_%K@@#!uWf#r>3<~(W!86$jIiCPnrg|4 zjL5(Mi*+2-kCwpKRiIz?!4N6@=cZ6RRpb}7)EIV=mJHCB>g^w^pEb8s9MR$n>XON) zGL7-}E#7__o8`?n-!N88FFxmN>nIs;iWhIS7)S7rQNM*Eq;>fKBbe~jWWkeGpJ+g8 z6F4$W%_wO$N%qOuQzHttJn!4Sb+j+p+%Y;tRSxb_eHjPSTc)=lw`3pYD zQR!b)lW?4B=$42e;j3un3XdxL7oA~L9N#m0adCbke8UOwBLGe&!w;a02?`B`6dfDw2ocDRvvbwT=w6Tr}Su83H6V2zWK8csb4ugFQ~(@FW^c{ z>kvPDViC3YC&IE>8i_Cb$bS4XuHT9N8{k_F`EdL6E0c1Z{VQC&1oDFqpaRT3Ew8*a zl3x)fA5jVp@%`K~ViPLVGF9Y{+h-h~TU`AyO`!IQDG%05;2BJP=tPClG$&IH^>x{N zE|tw5=ou&sj-Xfq%SEiUR2V513Zp#dQX-W1Fqc!KBTRgZ^z9v5ck?zj=HTC!wR@3? zoi&)$Ko9&=b7V8&Rxr=UXsUvjy8@9#AWkTNW)y+=W2#)!9r9V4UHV88)2J3kMu&%n z5t&Usa_LNCeZHZgv8ka!w1nh1|8agGrHUUJj~Z=bPE`iL3`XAp|LfPUKl{aJy#M_l zurP&R^xBKgJo8M)Col+Cv!Ub7Z+Q!8{QNbqVJjS8F_gUIFD{ArSC;trSu$JMiVz55 zRmsoF4y6!>nw%$r4942G*0wgye<&`odGjVE5JVvVfBn~gopAi|_uO-@!5Y5BJ3e_R z(95)7AxH9aAz1{+)=#KTrvW_8UNZ59OWq)f6rX(0-@HfrmBj25>F&Arp1=LuD~XmR zClVk2=tmsiKds~dUnzuR6|@X-`&WARUrQ4tGs;z>0t5cz_|6ZpL2)Lj^AqP^(Uao~ zS_n^(|MDo8+py|{tbw?)%ka@Zs)0l(0Ht#RuhwIcUX;1%nA-FJqUi|`(M#fhU)OcO zwTysV0#ZdSG(--sG~6+wEv3@AV9MoS0VG06evcKPqzh7*Y$mYDFO9kUw0dL+@yQ~( z14Fa)h9WFW83uU(dBu$L10=q{WyFtg7!sKbl%}Xj@IA`J2r9!lhoukrffxCjfFuR< zy6e$qDOn(qpN6bDSODoMf(svujN>ORKdGz~$#3Iv$2XBYB1k|;=suFe=vNUg{6u^S z;Q*moklH{L9*h#8-&U1sRV6!z2*@#*aQ3d>D<`C!ls7*pG`5;gj3# zhZ8SLc$660r{vVoM)Lcg%cA*iv;PPmZpr0|K=5RwiSRjBxa6&kPqe#_Qn(lusRI7T z`In_3rI{GO!uKMO<8b|#+%CULcm7TN251ZzewF+xTPe&$B}#BL<-evw@$xM2-}>bz zF$mxWd1TFPGI1=yjKK`VG%oMaf7O~lx1Z2@|A_u8tLC4LCEPhBy2f=+jQ{4EnwT{x z_Vnc0ErUDrQnKUdV<18-Py$6Zkihs-srH505j2>XI=S^tjG-aR(3r4ZJUDFm9Y)YzW4Du_S4|E}607rjTByEV#8W2k|hRd6G z6!-P;aDS@MGjPg#{%Yp&t5{2$ZJnO$T##t*%rMg-rCV-3_3c;s_V!JMgvYV({HvyU zGyePCue-n0(b`&AIzP|6kIVzkfkA%Njr4kDugNI`L4IB_kS=N9PFqf$#D|7K@QAuZ z$kyG3?%qOOF4;Q0t2{CSQkjR|8=Kg1_2B*Qn*aD@B3)>jHK)|uuXfJJm&v-^MmMsl zE><(UYa<;37j?tJL69`1*$xS&n(&_hPTjzh<+}V}_s;zEmflSdrt@_jOONf}a6dcy zS>KeZZ=Uv?>wj}yrTK3?a3H(CRL+}OiaegU675DHPg9u+M#jpE8cOr>ye@+6SJV>) zMhn-6(*MfZWs8~%z_Nf3>``{+N~S=C$+}cDnl|NW@k`y)1ghSzMqE|o6xDp{8M zFfjouvOZ?U=N~bxefphu{+=gCXfoN-Wh}{aaOqqoYcmq5L;--Y(UBszZK<3#S|TGY zrC#yO)CN0|vVeqwXj$!6j3CNmk_92K3^Q*gT=Z5XmYM0@y}g9~{*K?Dbka$m{oH3?aOw-%+S@j6+{hLumtTJQfddD~MD`0V zyzsOaywD)H6du9Gq(uXM2>z=~k+65yqj!NKg}B6OmzrBiL0OU&0B+%L%^XnkBJ9C990sQdu+H;A>FEKcUy=2boB55PafniqOgn%F!2ij_HKq%bSnOpm5km>n*87)zuUhxPZnxi#QB%}DL*y3={b=;m6BQ?zsHa(rXvV@A%G-;w*tqRBpw86=C-8@*ATSek`@;t;j#+ zzivMh?ceoVlb>%X(NunHw0Gd>SO4jvbI;{@kR(f)r_W2bPs=uT*aQu= z5hI6qe(zXoTWWq+x-OF{5U1xC>_shFuPfFF*aAt-_)dt1Cm%Kt zt@a>|u+)|D$6DYj@rBt8#I)Wx=xDcl@St*200^JD@M4-ARQH&{OQc*2Sp; zDqL09@f7**g3Ips-R^okfeiv+o>xzBU#1&BepII8W*u;RTn|h;f$)M>7-TZ4iL#4nV36)g`deI;U9ohIK;L7yO zj=rVh(j`lrLs^3<6etYj%rnm#=G@&%ftEZ6Vy<;w>K2c=i`Su7U0T`d-h^XAbmaU7oiM)s*1 zssKt*A+TU*`iShuwGMQv^ zW~g8tMqyK>WqpWYEgUwl9jwjdH3PzvG!(6w zXT6a{lQH?a{JCub3IGAt0xK1N$^JyMBHA9K_0u5)0se8}g%{R0Hd-Rx!I|Y* zocS-Zc|v{3&pk@H?ZV1Xm=4thQ#Jc^`7gNOf`9zSf5eq2Z^MQSt4}?J7k3V`WU)|) zd_+0LCmAdFR<7H<;;k@2WK_cr^Jn*`Xt-roN!if7n;oU=hKPT5Xir6Z;j}VocqBgGprH5D6p&wGC5Rb9sa3S#x7JP50zBL4*^Dq@P%m-RN)({(-BDEdLpf* zug0ec50^K{hmHRbpG#{}W#U0iEd%Y8^PnsHFF5aN?~96zM?s+k=)KPG=rNBi>PD)@9Jv1mk71-;%?_9@CO z={WfV0}Kt*_$TGz`>#2c^@0v-!}^r2VJRe>LutY5U>w2kpZX;=dhB zmYsIh)xZDb-}bEAWZMqV7Hg70cfmZvq1$Kfh-sOjp_HmiEjjn1lP-H_qVvc^Ca-lB zhwQ&co%_bMKlwj<@7p>ww|?869KZbKvWdp}B(ok3?B-vWs?XtkXo&|^j9+ynjBDxN z)d=IT#ZrRDN(KhUdV324Bgvr=x~Fonzcy4$ptv7A$(*y}b{A=_|i`?^}DvGI#H9Zm26XHKeDvrSdF~=IylF71Y`a z)@?%-3sWd5nHe4_3=T=op`mi$V0olCoJp1n2m0!|W-LAD<+D#aqoJ*}T-TIqnU-p5 z%QUp4@{QbKx?P2Br|dK8e~W#6xj78!(}m$-^;sOy&Ie_BgUKlytPo=1Xf3TDK53E2M^hJtO+xRKEFLt8=gI;m6qUcAu)MQP7Y?R>hzEV63SwQ?Ap- zgz!`8UyOAnw{R!W3E@2If9KAd_x5+bLwIC%QYl2^pBe{#VF`&skcs_oG`?a1RS+8> zD~G|mz9hM7z`>gTx4rGHk(`bn6(oTrIv)~H$p~SpWdeSc{NxaNcsPCqHjMUMFMjbE zXPoh3$5%F^*~$%=4r&nusegZB``-&+bKz@VbD`h?wo(u*ID#|(svh4GN^(h4<6D#QV{mMN8V zDyL^$*z$;c)*cO07p>d#B~*HCEVJ770$JNEhQ3G)NJDCTZL*eUU+AHjfh-l6q@gL~ zlXr7zZzX>ni%8a(Qub~~pkJ|H&A(}*;=cj&Rvtx?+Y+egpaDoER<2n2rnkK19q;%{ zbpoa&)Pb(de>?(6GQG82oFQ&+FP=%UxY>)r1L%UT1;Lb%sd5`^bg zU#>|d01gRZCt5?|rf&wUI`glbc+Y#@)7IL0#TD;mH(SuXnj$AUlBx5~ec2bk_{AAB zW>}aP_pG@8)f~9`UAF+^QzShe{|zQPoa=z`$bx-hk`ndKIJJ#Y@8T^H!dMm?+c854 z$=;Ns50}yGn^W$PJ%B90kjr(0a>?WPSeV&{Vq~6XkdBY?(Y?i*DSLh^#c*5<`4Xi> z_?)N}3-MikBvck_N%=M;(u~KqL}UcZG5wSDgh;aBa52>`8-#bS6?}48Mly`YC)7}? z@Cmm5ZydfvH2jJD6T^W-M3E-Wj|&>g8SoRyPcDlJ;Aq&QBu#Bl$h6Qh5E1{|H}@Za+#PkScyWM7b;SOTP!fCkye&Pa2aekU!$z5tDS6@%l+- zLanN@3p*@X&5Yrt^v3l+{`Q0a`Sl&Y{KZgfdGCQ1CSi~^pXUZ%zZ=@S z$+7l&4J2$N^+dlJ_fBgHPFVGvnM80Yg{$3)2>^_06!hW9@um&A;ziLK} zUj5{1_|~N;1x!iN{02YPvwbr+83)$i+rRCRTx;i^+kVB|M`4h+Mh*kWL-3?hnPwH1 z=Q+{;HaF&)8>fpY(^c?grfzJA3Dm^7e@^5l>K~{2Uo)Yp@w3_=IZ|jdk^MX!`d37ROLbZi_N8Xg)LvLR@=q=nW>2Mho7$UCNA;ziww?5K9DWVV7D^0|CX`ExlIl&NaalFidf2PJ#|`=JkgXw~tn{^p9yH*Q#ukS=CpeZz6b zAN!^^z4_wTU;Oi*|4e-r9SL=_?T10iQ*ndL+H-k6e&(&ZN@upqjwmJ+geP*@{P6Gy zLe}Limqmz@a`_FHTyoYqXYmBhtvB7Wb^CUNGzrSh%`Nb7(VzXP?5Rv`Mro#Jc zmmmDe?(aOKI5GZ>y=wi&kwv2t@y`!2%Z2klS^rCpYJ5N~BWQ7a&aqR+_sIl$X#ykt zV#GoE@%LX?CTS}2Phvl04(7RqXjE#b5zet$oxs;Oas+2yQ}yp!2gp&k>|`4^BuCcW zF4;6S$;RoLUNkE1unfOy?)Zl7`g;#`-QUf6z`3yy0aWCOMQ%WNZb5Wld@0u0-S z$K${Gr6(F{Dw5nTgO6vr|L$6I-~RjV=zH*ip*`FC_iP{BziV{&&SHOeVNk@9-=*tV9Fa+8>)?fvF`usGn)VsB zKaatGx7>8&?w|dlXZIt0+a4O)`pC$EAzq}bp?a2;E}3lS?1qlE`qs|6<{8sgE^A-9 za@xWL2&N^Y`G#~ID_EE(s@Jcf@*8#fW+&t9$5rd{yMFKe^1t2n(RZ_Ogl+n?9)rmh z#!#v$sd)Z{$E(`A1AQ$k_+O; zifplwEEPxD?^kO_tZgJyh6I^xR-2@9rkN6rO?Ah`DL0Rr^yN7U<)*nrTulSBk&`+P zra8#SdJ`(b0up{M`0roegiHPr{de2eZA+IeVe-gzct4Qm#lnRP(5D$N@#+dq_CS7B!um(c8?>LQBgyP=VgiZ08zpNvetSCH|JE&K3Jd|u zJjUWu$CT-nQ1oH>FSCGwpe$QiuAuO>a@lnM`+@+-S`&#`di!H4845=4$H zSp5(22~-554W%^R6~LLN<-!7WrF_8Z%XCIPU;F2;#X37W+L<2PzGFv6d;5kB8!cX` z2yAU_$>-~jTrhv<&RyHLZ9nN@OAfe4-XHYckX%j z-g7sLwGhgN)aAYV_8r)NfHz>DeDaC&=g+_Xx*PESGtYYQ_rLdjni=?MXsn+;y>nVe z2Qzjc(gt1eo-15l@I^&jeq+UXqY8-N=M(u+BtHkz?j4MO3N$OWf`D>ObE53V+C+Q_ z6W9W-|&SH+2^C- zNBm4?pOfKRosC%&T2&V5s=Ll7)i2UXCjIvheH#NY`@j-iIvdO=K>yRIT z2W7j0T#k@HLD&)Ek89t2{H7!-Oi(IO=_Q*kEs!|XWKw)0r}|&`IY|FF)&IuqC$MQk z%*4&#aXT3vQHSThZvW%$vj+atr~hqiYAh5AJaNK22V(n_hbB~2^%S!j%+|^;@kau9 zegwB;z9Tu7D`!etpFxn;>2c4=i;8$Z3mbDNFrCb4M^ioRqSi%wdXSh&2_{iA6_VDH zG`;U!8)XYdOFHzdi@x~JEV1Pf|GIZzF&*=>v~Su%iFNsuLms_ToA=e~#C}~@es7c1 z>P>QgH(C2&txeFLN%yolkFWpzpC9@=pXs1}EVwx9th20P6E?EfXy2Z=@iUK#a7@;I zim`*X509t)hhES-+S*?5{1;eNEB0?qc<5ixqVcna1pF`|Q-xQQ>&?L9y8q440?9Az zz+a>>KX4+AfhfnARa-7g4azi6l8IXb_QAJbssTXIW8zF25fCMZis2KaQA-qVWoek8 zKLE^cg{VeRKQVD`_*{Pf2jAz>v$M{8@qwP6Z98@xfBbPY-nWr+|dF9KEbYCv$nOZ_2Gvf>gyX|o0%W~ z^e0S}O0WzvrxqM>#GE;EI;VG@y86`L+;HQqx7|u3v0&j5_us#!f1sb)Stf!cWhx0^ zE|adStH1m`mt%=GYEbdJOrG0*SP{$3x0@LAa$B8^(WRpTtTA!*^l)QzBUu=?S6X<}%KY z<5v<#Y&*U!vNAaX{*`^a{BeF<&KSN_GS2=TpFrXBzv5q#vtk&Mh!9a>@=xYpkJ*1K zRX!+wwfrIf@P_;c_g`^E8p98Yv+7VO!RpW=wVLgZaSpynaKiQJ=f=N>NBHhv5&!Zi zB{Ta8+aI#W+c$#~zRRW){)fd?@$+2sUk$bNze0%kH2*SUMNdev&dnYmTp9kj{_E0k zV=BXXZuzg>X~Vzo8-gx)2)mrjTASl~k%B^9$#8DQ0ZSsxuxZ;R!`HHDX4B zg&q8A%Umk69%`vI5zGTGYM^Mw{P3V;ttYp-@aqRh7G(1tf9VP=FFL`ZES6o?CtAYv0~Zw zZ95ssJp9PR+ji{SwQF~wSfq`>VI?6^=l+>pU9?sM1APPiS|z+^-@g4l`-cVx z134veD6)9AN~vo6Gg9?yl1_pn#t-^%{_S26?h#G*;T!+~iaZEE5#OK!O@Z*J{0j?3 zyMCQ^$2b1fW(2HP7B{@nF=`-$Nz*F=5{E8?ey{Y3ayx{C51 zBF0Z3K!7hqNKIC}-O7e%0G5jj|qZE(D{w28|&Q=XmD)5OyeJ}$FUMyiZY1jaTaf8?DoyNRxS5nb(>}zc$BivC5 zLl3>MEJN!#nE%M~Xkuj}4oy#J=3*>SEZQm!J(0%_9x@ZX2FGSQ_)z!FkHvg5xTOd_ zwSV`@KzAkcxZZ@oVsHql`yn>2VIDYijG+ONtoN`q~lLZ5zAc}#=;HQ z#QXRmf=4!MNrFtHqa&aF?C0M1{`Y%#P9@&;u6Lh%?zutx?Bh@4*$~XXR8x-F_U-k)U;8D+Y+H7rLnUzAs@Vt3O&7Z5 zb{3dikfXTED2OPUU-Cd1FBDUQnO!p$9JS!I7oJWd(91(CeZ8eZ zv2|LzR9{YH>$1Q7@9Tf_o9pv+xnfZdvoUkZ^KD1YpTBbDiX}^ysuos}nEc{UnXL+{ z0~N;8bSdR|oJ279Gmho zO^FxAr6fYFByjzPC|>%c`c=QH@&p&nv#747-zxbj0(|5@i2bIj_iXXR2L@Pi&lj@hSC@N9Dxrza~D} zIh5Y`p3X~`b8sc*5LlO?2U|2;(M*Hpp1Hl$6a;-GuRh^zP~7ipq@s?B;UtO4T~_~P zxf0BE&Oe#}7ykBw2zchVyRFnVvHy}!f zbPqlvDoA%uL_oU3VggD^cWsn33`q$cB_%B>G3gHJknY|XvBB7W`#$G9f9;%|vvclk z_kF#u>w49ipDQK~lNmrm!K|tAjmr)nlvi>y+0;?l(9jUHx}tc}=Dc+2!u0s)#RkF5 z7!wVJD6xUU6u05)x8D*~oD@CCPj> zs8IW;3WIp_)b3`hWaO1ZT-w5LjBtG4dC~gR=xdKW4@-`282o} zl4$sIV9H=f(4|t1olEBT9BpY?nU!|0tK3(=mP(rL^uO(!SQDdtJAQ~pteS4*NAq;x zejg5ucv3sL_q1UiRRr?eO`s2#)bbPz6ZEkl=@w9E1?kaF*K#$66 z;bZ19dTCXR`SofFsNR2pYD_-XPx$Kc>S*z=^0eaD#~)=}7utN5W?OOYb7U$A)h%x@ z?SGKZ?Dksq4q#R1pQa{{z+AvTfh0}@(uHR@y%8tkM>Qp$C`VeA*@J>@R&B7I8kUKWJ7=D;XNGZ?T}8p5&G(lAHJ6 zP3>!?kXu%xJD5?#3s(oA2d>>$U-(sX0*Y~=1O+sLtNPaX$RxqCF98hxFR`sv?;?u- z>?Yu^?<6FY(k|i@fow7r3P3a5M+I@_tm|XKvq*uDy#%ck^iPaHpWuD^-SOd!O%VMG z)C1v1V4NN?mT8xu#MyG)nhRsJCDas z)o2!)jLM2}TYE8P-`23^)ewDiFBbW#gjg~WcxI?f_rNJTocUlRZ-RV*g)3MnYe(>W zD{T4h#GD6fcgFBXJGC0-xT$d6EQ2_`Y&pJGk*_*`$HuQ!-`p_nAs;Vgq*&J)RLSK4w#Gf$)U@NjktP|XeHmbhZQbT7z}IcFWenfG8oh-5aR3v2`7pmX zuij*!nECr7-1q0_wPreBQ9pze=`3$3dDpHemPIdsW-ejGh?F&qjaZJ(2%E_ixEF7o zHcZlyF3hGrNcv7cvuiy4pGg$+%f^O=S|~>r85tRGe$cAQVw&)hjN3wEukKrg$1L%p z0sA_aSLZc9J=DB5u>mMe4b7YDpoK}hnPgi3m(QVh1DTX32_*l90yP z5a(MjtgeEd(oQ#gsKN|dn14gJ+3IN4H|Xj}(Ox9D3Z+l#m_JoP_LyXQRo?jep8jjM zU0reA(XP07ri)Yy=~YUM9UP~a!}@`F%~%NoT)-2_4q~6|LZhOT$Ek~g;tq5g@+xyL z%n;1nA8LK|=oVkqYuDQiy=2LD?D~?fHOg-CCw${MjTLW3N?_iR)-VmZ5y`%y0DRNg zYX0JL z;GDrs2)7O|w%gM*{LuMF&Gbv+8;j7U7dz~;0?&S7|4jXir5tBkuhgB#3H|V?4o5nW zJe~LrC!7m8W`C8o;S#W#+g!p!CYR-Bxr;u^0=+_qLlj!Fm$u8YvA{@KDE+x+b0i-PAu9n@!P4UW%=nC z5g8`XN|Qf)gsJx-C-Jb&zu;fH@eI-^c*-HN-<$n9%4R7RaTCfvcbjqVHMYF_DS4HH zPu_DDeUFrtlhgCc|CV1=x+ji56fx>iX2JjXKpdsn(o(jsM&Qta-ubepmJ_Cp(y|SZ zAo)_XJr>PA108&3do@RJTb=29FJW`)RIBZ?+Dy^5AITtYK^PsYLqwS{7D4Sg6dCcmu_i-o7gl5NwyVY~Mn+`Za7uD)}dUBLI zR_AnivVVA(*dkj^AIy{dq{adt&@cRlLOG` zd-JX3bFGugmBK}+;juea%qF9f5x75kSgm$^pgeTrpWs_zVA#yr&vI*FXM}B>hl1|& z2=0EU*-Jp`pc~rt{sye8{0LqQ9(5KvgLcy$?>cV4zdY;{@)g6_{%`MI)Su$iNOvFn zqU*&?LcyP;o6PNWwSv2ZIi>_F>j0@J3)iRdD8{l18Q;(^K9aGODho+fesd$X4ZjaW zUJ}}zM~!$yg!-!+w_A-wTyHBxhySK7YIG!t&d1#3w%VZ#AvuHB=K^Tl+WrF zK2F*KE!HTNr%K{4weNjjm0Hk>+6;&H{$Tp7Q9}_r~Bm#)~M28}K`Cw8nvsyAJ=mp?ghRs}RjNsRbA7hnYJH~EC zQ2S~g){Z<#5!ddf@M6?<28WYwA)V;e^o{;g-w}a6}Frf zZ+)0LD3T36LS<5WpDwHhZJi|2xvbQJ0XGyg(h1Xg!=FwUisPm?9#KoVhTQo)K(hXK zMW59`P5K+4x#q?_$5V{utthFGas@j+_a=YAdUJI#)UdWTVeCUB00Et@BVjaH+Z#PEDk#LI?MOBM#lD(?FUYc8`+<5=sr`0tuZj;itE1j_?QxU&CQDWOXBbPu zy~*rbMRnC>+sn=C#C~(+oo63l5`hmUR648Y?JY}5{cg(6dB?@^w`+=`pC1W7DT?ry z!Y%vK+2B=gh!(7G2(I0(Rm60&z`&dLG(;I}4M1j`maYi&>~dG@7(#H$D#z1QOy68R z!PB(U0SNQ>@4GZsY=5TG7nA$ml{+1FF;QIm znwkWsxv^M79;oS)Zb5XRi(Jpqow681I|*?XyWAp+ey7TNZV^-2G{Wb+JYp+NV#O9y z;RJsUYBHnK-!ldZ+V}~S)}FDh7>d6 z>Rmcc(wPMHZqC(8KTzwuCkdvAClIoTc{M$LK(9X0PHMm2ewwd4X2cTXz*$53k~C+H zXx=ZnC6q>He)k3-7v?g_E*I=4^Zv$-iJq5&o>GJVAt{6Ee~(MdR^C<0jGtgTPJ?Eo z+a-Sq{h7}|*4Ha9n^kuZxZ9^@W>+KU=ST7(*WP>1A(*_ODwt}6%lnO&bmCvORJbU=W-tIOI0h4gf@<~|4^1Uxk1@QuXe zk&hZLNrwz55kxn->FE`c_t{+I>hWpg()e#t+IR8{*zF9k!9H}R?+3z$0r>1wF57hu8$iHTCJ%drv@G!<+hyt)& z?V~YdAOcp3bMP_T7M;Gp7VWwHO)G%MQ{2}T-$)l^68?Z0gqaedPq+|}9`ra^iBA%L7 zXV4vNbn?n02RS+^tcV8>ARj5>7q9!f1QhWa{q`ajDks-jt1RonRK?FsSxVL^9w zEwDtC8BV2Zy<8^&?_ZBPc5)({n>YJ`hETGk4g2N*x5#+k5ZVY>^`CZHSFAgkYsmY} z)~h{oXO@h7NnE$E`+3+E#S>m(|2ucX+m^#UBKy(NuVYk6-s{!M>2H|c^HW0*Gg?`l zNpio~ZKquk$meb&9jSutt0j&@1`k2;4-wER7i@o+Jklf!?ga>l(dt5{wVZ7182rZR ztsMQ)doBQg^GCGtmJKnnY(N*nEhxp!+_ggQ{Btjw^Vl`)GK~NJ%$PV#R%p38`tkj+ zJSqz4=|pX)KsiJDY^|j-BYTW*#>=CXR==z7%ON-CoMP{Z>W}^w+JnJhBcuD@8)`#> z)V;j80MyP74!=?MemszWwP?@bW{(me4xPrM9GJi!;*fbZmLu!N5oih>oC|Jj`GIVo z;WY2KnJxe5Oa|~AX!QFjQ@LVh-a!rBEwhCGa?X@cD)K+(R0Ln`xmmad|Cq}YnMwD# z2tw&-E3W!4?XKY8A2Z^AxnBt7oW-M}0W$ALvTmu)T3=kq>U2Wo{xjfzVTTBf`s;^2 zf0YKX?LyJz1XyGiIJpwZX&P+wBmPnXf4iXhENEe$9exbU^z$u~MixN^D`fONIU-tK zGOQcnKD$|k)py$~@=PvAjWgK1sU?=X{UJ^2qr;Zp<%{hX?t+e~1zVZ7pjB0L==B!Y z9^Ir#X8KeR3B_o zUJ);$a6gBOb?TEf z*#tzzguPA+Wp575VN{XIVT_XrJx0OtK6c1QN#M@gi2Pa=fG)pxw`ouHKmpCv{gQv5 z{ge>qqO;R)={DzHn-k+;psC+!!9nba7y~|b&eJ`emGnpH|ocfjUT0r1e zZ}Xk3lVcB3@lDPzqt6}-0?N)e`_YAQC-j1S1GJAw(w*)_HnR+Npn|* zvyuL1hFI}J&7ak?I~j7{`CqqJJM|t4CBx(zl>qWi=ZN)gZEYPHmw%*If`WpS@n=^v zrz%^LckOe7MyiIwPQf)hS(tGyGu%P}w(2mm`Ymd4Rt{XzC<8tk>`#+nS}PvBsXjUo zufZXH-Md_z!VvtE`Y?=)Kx%Scxnw_Ou`W#q0O474a)8?w3xRKorf zGFfF8^M7{lwwLW`#9}>F&eQ3i;Y>=JCoc#xLT!^o$Vz>$^6La|Z^>tJRjPl`;<#;U z2F+lft;fu^rQ}q9P2ULM{&LOl#s4c%COua7*uLL&$EHrl#9Tw(y##JLRYT5J4J0)| z&ghUI#++TJMw0_)Kclraa>RVrzY&)}QeZK-y}LQ#(PS(z{?fLZFwA5eBBjE2 z$G_KJ2C|H%-|l})sOk<_H2PeeCD9`*P!GyKB~8VoeS^4g?5=#R9TfVAt?ezOD5{1*RdyWOBT6+IWopu?=YX!@$l;U$ zg*qPq?cMlxZg(T5YYP`${h(^=^H67YX={*^BljysC^Wl<`7ZmN^6t0Y1M7-;pX_Ui zV~?&E?mLKbME`-7Jp34v(}B~ApbHtgit{F9|*2dKr?x2m?RX%n;z#1?yGpSZtRI7wsUv?F0+*}?4J+}r3ZM@rZ zUGE|EG-3VG>m>QhufnZ!w`-49hY!sG)jFRRTk>sg8=0%yf-WCE8VmF`!h`Q5DShlY zvsgKitY5mzs@sO|xaso-w&3$rke0Xq?xJKWRun#*hvGtRV91u#-xp6>FMcaRfyl2L zlPSGARvkzn+;pY-ysV$41HZG=wbAy%W4IU8yjR25uP0|;mnb|rmBTyaFB%v146@Qc zA>=X>5$V8IS-5UjExvi>RL*gS5$F`o(YW!$cEF$?Z5q1#OSAh?vv!!xbr9KS>15rF z)+MKKzL-uUx@!xIQ;thH0$%d!rretU)TcKBD#f=0p zogep)UOJ#-m;PIDz{T4eLx`ZD7)rDutTgebH7@XW`^nwdPIW|xe^Ebl-;MkAshyh? zWNH(ob|Csd{T{Py^jf;WkAA2>|M3cZANv*87ZJe-%t#38P{QQMpr-KqVAOU z`6ZOInLj|;DgcvFoA5*nncyE#*k9y=1|Su>(Ig98L=!wff1%ubaTk(`XCfis@tx{K z;5vD?MV04y$~zz@BS&qoE}O{0h2&SnAz-*70rQbGMbVXP^ICpfL|RYvV6S@OEeAq- zu&PfNl1lv`>~z*ft!quqqCUCImY15r{R3Jh>S!4}GSU23okIJNUBzMWp2!W+6Fn{! zrzE-SQo?qxv9a&iqJruSRji*C-iKDs2=D4DOp>!%%M0U_aXFZ7uknI+Eeii|tF)?h zU#C*XsP`vui*I+{eUvM%Ao}>G>nEXd48q1p zJkCyl4A^bKOx0e$&riw6@!_BL+g+DzG_9}pG()dtnL*-rZfRDR);3XvJ8ptt#cNHh zPLQK#F}jIk9Qflo7{PJjMhUcnoUW3q7)m1O>JzX!WJ8*pC2H8DjbF-OYGA_dCtONN zDZ~VA&usB~0jnO)C-n<6943$h=i-((|H!r~=l@%rg+%UwZ46gDY!4k#K7byv7L`!3 z-vt?PTh_)%+R?Rv$LL{v_yvqCp^#okn&{+271v|yc9ZKBJ3jK@U4nI90SNNmz5RZJ zgU-n9fv;^H$uFY*S9Zyaa0+H}ogOY9a^&Y&PCUzluA}tQsmU=ix%X`!7V7^IRkU^X z_bVGTxd#+20DLzjua9?|qZBwrN{io#twa#>IT%}7#+>ve$TOX_`W_$rZCB($p;of5 zkUXxgB4ESVDQWYTh^VX6Dd#^O^NlE#o0co9m1$$E^wozQ;EN>D5M;_`e70Ay_70U2cHICBnOdv-CGAzJBH5id)SkYWi^0}k(5W=$ADT01nOb12J< zKj_fp0iWZ#+BUAczQtR;GM@#|9qcVNcm<5&hT?=-M2o<(sSH>bQvPv} zSl-1vyd7yfGHk36YB=)1(-n~PJfDw1v*nGWlez;v2^)7ymXJ);8d@8emi$)9N0nb< z7EQ7O7uf$20%E@~=+qSYFB(1ZhYFG1(-HO`b0J#5?R1<|5_@9nU}Fk_LL(jnVwcG^fijfyHUTbeFB7BllD+hzi1NS3a6+`?#1inqU;qKlBV{ACb_Wv znalwQ0E~jAfWE^%;W&VZP!AH0PC+VZ70Zd7h&l|Feofea@n%4<9cpglkpd*Flol<{ z;X&`N~?Q7-9RMMP>C%uGKG0js(zdtAAi*=YM4jvrQ+4`!F}o)-8S$ zr0*V@R(tWxJ2)eTbFEfsfY9EhAXDD~0b<|+6l+N(;pLEwl&5RB8dx|7VoCf<->{j6 zZ6HhYhU_p7F|c#`#^%h{|l|1DY(uOwZqEtnWF44`XIX zeW>!7f>qX{e>J|~Uud_79<-;UjnY*k$WTv|TjT*10dQW~>Y93YbdB0?zta}*@^6}) z+0*Ku!`*aGA2Pg|b-T$sICd5=V3YVgE(w2ieo3{odTpz&W4+#V+7US3c+C!n_%00p z6jUjNVYYIwx%FyaxEdJ^qLC#$AxD1`RX2zHGAkgq7LG^l79h)z70m7}Vm*gN^!4kv z=0u5h<0FMRN5)^E&;AWD!4^p-n}kV7T*jF1zak)i0K;*NdYnQ1bNTGZGhLQxKI+Oi$9I87QN~eAt4s#i$Tp-u!fltA*4V5#pijcPG(&zc4kE@#+__P@k zV0U)h$guvsN*&y6z`Y1_Eg^)PBm&N_1cP}}ZH2C|mbRufBH^Oq8Sj53{WH2jKB496Se%g9 z?Ee!+{})*1%LM!eDF_9W0BJv_TA84s8#)#5Zi%hsy5crkJ?oX^P*;~X;9kI*PS5{V z1Aofr0D**`iSrlcC(+oB=v1a^$|&gX`wtG|J$&->3R`6VKJKm_VV$iR7E1aId~xa= zA+L8=p;BPGT@YMlunI;e2QR8#731?ERLdzca6{GZR{C8)9kjyfd;yUOSxQBa2Iwf`77Az4p)G%b=zRGezF*qrX#5Xo&V zZJGrMQc+SNbgi+Il-Hw!gF+%AE1)+^b@0p4rj;-vaP05WB18(6Kzr-SF{1&yI5xRc z(bui*uo$%2WSt}*GH*8bz0Ps8@s;Gt{5ubCu+{{`A8Hmqw|lxWXLuhfE35W7V+Z{& z0JSrc^CmfSu0>}zmQkeKb^hJr&(cO$P|eQ|5IqwYJ(K@rt&k^nN`oi_1n@eI72@1` z*z1DBQ37W#)ltC%4nfI3)Mko%@?)WMXl8uOo)5n=7iS z^uH2=6Lvbu0Af+dPv!{dl@K-luO=WX3Q$ZNuK!IN5On01m`DS{9Lya!J7ew`$F33B zktb6nbch~81JAX;(xn~{5^~FI@ivT1RZVh2aU1!>pL!_W0$kSENNbLo>+y zE7vu^i1lwm1S#6$^=oIL4?iav6?mT?sPb+8(s`A+dVAt0k$kKrcd}UPY{jwZeY(;< z7mOKx9*qee8*t+ZC~9+-_5N(`6mFpsN1?aa13~Y7dd$oHLF*0vxpnw_j^+Wk?n!l} zZNEI>8t?Gw^|^~VQy{?3Ocqx-n^ji-Bs(jJ$|DvfAK-;!y6Ze0mxQ~<*! zClEorhQibbv_K64+Qi^QDW$hNvTpOuj*|4PKBt#^51<8QlUKGo)z=1R4-~h|&V@2x z{*e#ARWtt*str_mM)pee)xN=!hXZcrOFU0liVP{4%jf!$e&XlUdhg@SNRZ>%@V%95 ziM#dvN!KM?qKv9YHJ?O5OU5(XMjc4$(S}LiLzL_wH0$u|_RB7jFy_9J0eatPG*L~4 zlL&dDn?srJ9zAgi*XlF>dDe8(A{$XZ0gOE}O`4ax4Z%i~_|wpH-iHuJBK{&ur+`Qx z*t&N;wXqR%(@lmW+d$4{yut4~%Nf!gy<_8vyDnsKDa1aDlF>kieNWaJ1WolzX;w1z z2-Oe`(UwW$*glWJ1?wg5!R7$lRqysdwy024buVyCt#1tG_u}Q(HJ+@+lIRVWvl4M*R%2ix&@Q4_y(Z(gB%L?v|L)u{EH!^@Zo zTuwfN-ONtTVR>aaQ`S=xQ_V5B@Uq*7cbK=&{57K(vg@^@I=hzPy(>BQ^=P3E;hc>d z=d05tiwI4h^Mi$7^(IeP<{NIDd`}k4-ddqI6uqLZxKSe!@dhdlMMXvEUFb?%i0$b2 zV;{E;jQ*>DzXOG2%Cau1s=Hl7eaiCh7(Z{fyd%p<-(i6&ZPlpWA*)<%cHNFkH%m{K zzHxG&k@cAT+$wwWlXKEC*}@nu`5pGS4LuA4uHQU5shICTH}p!60=!Qx8uM)u=U|I* z8Yd$v`=_6nY`M(QWn!)7el%RHPKWID;vc9DtaBAUa=Z|CXEVwK`C^(S^-GEn>hcYi zZT`q#zhJ~a-?jSmc9b^z4{oRv$kZ9njs_pWuzj)}U!q`J00%Du2UFUar8a!6DAnLO z1^CA9!1QgL8nswkxfH7!QJ(_0H;DQnI)C*sjsQS?2VXm;sNy{@Xh_wmkYSOK+v_X% z?n$fY@4f$+cniAZ_$4^%rDXNLsm#bybpA+uk{5s1iGe@x@?aZd00pc8l~Mcluz-6G z=zL*c9#TnI;K6}&RV})y(VEpl+&_OUipQPosYfrQ(@y@jNB6w+nX7tN zF8GtRqnZ98+rv4x`5Nzqy70E(GOba=Bw$Uu8@|I0+YS7*+?GYRM$WL>hRk7lAhzk% zXJT+%TKeqgO2~D2GR@R^X5dEj`g+3*jdr~{XW`^xo!`(`NZI`p>m@W%SW!Z2pQ&>? zN3w~Pbq~cF5xTUvVPc4bL#5S2yCMGc;Io0X%ljkSt?EFpIF2N5hEze{G&H)TQAPz( zreQ?#_n+j`AB!^pW*JqD97P^NsABc!dyUZs^u+3DRxk&UZ(IcO19E`Dkqkn2SI@=~3zU#L zS}M6u zl2owDII|;zPngYw?eWpKbsm1|Y#qd>8 zh@YdChwG_T$hgvNMRmx9&{C5tRF>aK(Afgs?lk{InO6DdoRE+e8b^n_z0jlbvpC)} zQK`B9*4BORJ9)aNM@z;qr^3I1ocUT4t2>;I^-R*iKNIcv8XXDY^dhqY`wT3-H7BHv zIJ9{P{l9>@cLkqeDfEdf21!Lhdc#JxYVqy|qG3}D`6x9q#YO+*tuH?6e z$^4P0b|!tYQsU$A^qfWqDlGtB@CEUb#%6!ErM$2tF{L2=pa2wKfuRZ>hX%)dv+X~m zYgg+k1;UGV*`=7v1k}D%FZS?UL?^019IMwx#lF`L$o=3Cnib;YNK%;1f3S*}fp_w2 z_vISOUH3e{85CX*I_6a+!qsvs#&1<$IBXv+FsVLiG~|jt-ao}4(Fqx1Dt{l!lx`wD zj=HrPxY{SC`iDW?nyk}l*-h-Qc3j0|;!6{aatVl$NaF%Mq#$cC&cC2st_*$=LAex4 z`8x5AU!SuT-*_W->I8G(8&4GRud#qnC^s2` zrN_-xBN*(6JQNiUx$S*e-FX4NMLimW``eMRXa-Y->j=HspC|2H)#;2&M9b(n%}z)p zsfAZYNGXzpu0Hrfwm$RS;sctD#zoEf4cGh%^I3TM06XHz{f(dEoW{zoXa(^OmN2sl z>3h$eO}w0yRwn07=YF-dEUj4`%Cx~7MDFJrx}Oh5n^jb0wFes+F3#_6oqT+!?HQh1 zu#r34=Kt!7K`E)e+~YLqfLkwdRQbk-U8(|7vaylJTiw_QhAdaj`JImLvj{!bIX_xG zc4iIUo?!(EAEkj-TKiI+@!UN1&C z1?bdmymfggMQe<{!iHL!rwGSwx{c-TnpgRwN(-U&CbMDEVXO8r`+1`WgS1a8^ zy#DK-KbxhSZ7&bItRc{rLWRVQ1go8eUzl22wbl=l{Lj{#++5+ezQM8gev(Q>u7zVz znIiAXyWem0P(<@6M6HsoamJ}?c~FEkcM2D0hCM>afB7}0#K`JQkX~BFyNv!8e$%w( z^-|*TS7qlND#%;JqH}d3oA`hAsYTnng;)3r7l%JhCmxhkyQ8q^70kjM?^pl7>b4r2 zVGRc&FVmP<)r1fR%ZAmS2mgUD&CP`^YeGN1_%9#P*B0RQ^XJdsYdKH7^>PT`m>hQO zkj8O9&&G{m4Yx5HDM#kYkHar^-e~gNt+w{!D0qXlCqC67g5B?uiRFdottA0gtXfya zp=SXohgJgaqnqnLpC4VvXYN#M>1d>7iZ^9(at9sk865oT_3I(qzWX@wgHkeJuyxMoEbghmX&sPnej)*9uy=m_qWta) z_<(|z(9Sy{dGck|1d>SS2&j#aRB&TI4Dos59%#GN0#?z-*Du?h*+%)`nlr=k@S@}% z>;)lU-X%&vLShfF`0`y3LryGBB3;p##=a>y7+1XKhMjPA^^QQoy670qI%4|h@+}iY zMe?~--U^TgreZ$&m!Vzyo0xEk5y=$!@sQSvOKk=G#*YxZr;T+bW?EO$uBOCk@hvvF zt7ZU=#QGh9&bITpP*2e3X}v(wkl-P|K3SpxwixELO;97ZC1i6IVS=%S9K93WQzH}c zu&jc>5Ll|`AE-3wYAe2~_SgUQQ14@^rFX|hW1I6nmjWOJoWJK@)~1Q{j`QjLg`824 z{6_bdDBC3x)07TZ0lzI0pH93*<`rQJSld~VfyX1Mxed~`kfX~li_u7;$JcX`_^$d<=j@i`^_;Tk>bU1lMI z(RxJbv;a5h13iBk4ZdtfZ4l+3Z_HR6)$A9d?l%)$cg{SW{ABy@?NF=yzWHs&9M96h zKCa^pcq@iZu2L1*+DMG;E9eTJZJT{zrDNi3lAJTg;4HG2odcTWaHGlIXjpRond94w z@{LG; zIoi6xZ)v?BQoZb&t$c$=!f^Lj*WUZ+6Q+4w#v9ixaoc+r`IS#egH|zPeue1fSHC{Q zf{~BNMcU6Nhef_6%ue)-SeUTF9Dek_{raP}1onL)A^MkColSGZ6a%ynT3o!7C29XI zU_1SQ!Hh2`L8!MRW5JVXhTri%J_W#vmm1!uP;EtN+Ab*O{LSe!I4zoUe+K>&hMr^Z zevaPzvrl*E*0Qarf+IGzvg-J-&%k#V)OKE{mHAFEsJrxi(<~j55Y>K)wI14ZUV#9d zW@e=Sc~9w~U(=9M13YE_xvP1Z{8RWQ6>zpY4g?%qlkmro5;hSOSS4ww5!pjhRf^+muLQ1^?pDYm3QJT;gEyxAJk7aC@*Mg# zf8&KuAANeV7JlOeM!8O#=0OXeW-syl_^phWD{sVO9C4j}9 zYO*{ooH|++r-3y3xCuqtoTrjH#)1J(w18_tl`KTfcl6dz!1f3aIPUWZ_ z?5uk${Aoh@?+0p-R7Y*xJ9|`5^UnKXT`RUpPPe0|wq)3PGf6AszAnyD0!#1HcqHP<(nu3Y8L3K|=V>JNETT6c6 zfDLI|0&->0fv8Cpl{-K8f*!nDseZDOJK*T}gxm&GsVA<~So`Cl!{A`zHZ!UJGgW=p z0zL=V1X-#6Lq5Zaht4reQ+gMEg;k8O2RJwuUJ-My?JE}5~;O42wFbTL<;YV>sKyY(Ou zEX_N$?hw)`K#*W>qUadV47z%&IB) z&`w3!YXy7vJ7E9x$g!L5$Ffb|xvZBlB3~9#AHjJwRkCgj!>%c+z^`E-;xK8|jo3%u z#U|ZFOahprH_y=q{m>p@J|Y@DCj?L@{?;JTNMhQDpr#Ej{(P`W${A$pGEnQ_k_vLL zyXNdD+LuRHyaO`9;+eju&!=LMz~9d=H90qp~JE>kDfAa!UpBx~ z+9gEl2`G<`4^femfc`I$n`FBi0ZVdoD!P%)hEun_1c!xRp1U1_%9CWt=UGyB!_Y{~ zPqw^5CiyaKWLy6Cj(Ri!!h!DNq`^dUVK{aSadzAA_aB_X+E;FUXyUvIG15lDVJBCF zlDKBYMkaib@{*+u#?Bl4hDe^_Eyh58-R!3CL;>62+pxR-E+n zn(-aCASCP?M9~sviN*HrfDWtL(S$dBaWwWqoaZShMGLBR?2HxG!MGcJZ9V7aGDqMy z9aHoe&dd*+j)+^8}8fGS!F# z@Tmd@Hh{>NBRPt!pm;I7@k{hITQEtee|Xtzfiq99M@U5ish`sr(tZ#M6SnD}?p_J@ z-DOe@B?1LL$({c^WUUxD87rCN)GN~Mr6>v~xqTjhKpeX*PiOz3V^1x1z)lQ_sSuN~ z#UY4-d?RNC^Nsp|&T5MUissHJ%uR2M4d_Z>ekx|B)p_WDI%#Y1UlHz{0JtGyL1)^} z;v%s>0~lZplVK@)x;fqj!k*>ICNg<*mhqq#$6L%eST)^)YlrppoL@E?_H^REafDF` zkFZr~b;Bo7$T%AiS1wdtR5Fd|<8L@54Pz~?bjNM;Hu|DjBA_3dclug^K*;DddRV@^w zNYOVV|1p(pRmT>{48qqL`BLC=cU3Ic6Yqkv1`R9e2{9`#zVI{W;$|Pm!8SnB1zn!^ zI_qAb(8sNSbz*iUlDQsmTy$Ux+YVgP8S$k09m zRou|8tXoA91W{%K{J6d#2M_4KWV{~laqo2>KuxNw@ z2)Mf;Ea-@$7=u$&jqI*}jbfZ1qgWkfOyDuL{t))y4X4M`7uU|rWfCHvDSmtuDhlYE zA;BAR^jfI4qGx*$;zRTF5w)4l^a&HH$xt%+wP9vVK!AEnzdn})uC6dc1acdOzZqSO z^ZR{uG8^>lm2*L@^DE>Ndv0+j^dIX8TJ~wkGb1h4lCdARfmfKW1=e6k!lA_za9!G= znbxQ9C>!~dyy63+k0Qs=+3*;LBF<}5^zrGW*JZ(eNrEyG@j%ljF8!(h{zAI575@Fy z^#XJpf6)zx1oY^tezR@&MyItT9hpJ`NP!;TV#2=EEEXG9o87iyjBYz5Anp8N*d2I9) zbai=+DOzOTO}wD_WjC%LYfuf3BHp+o90!;IY_E~(>D~(}CyQ{9*S-_1_F*EKk4QB| z>3Yviu+i-bJpqEv7=M`J{Y3AL4JGxC{I8q!k-`u1z`s0}$GfNBfUQ)(v*(H7J=xOE zr)?)%^oPr6rfX9y3}98;mW zY0rAep&L3`>n~gxOipIa?o!FCWnG?|0I7zT5fgV?yddXg#D512i&kXZr!t1q0Evi= zDjZ>e@@ZOp>2)$jD953E=|cqNOpqd)zR~I5A5|*vl0-JLF_8UZV&k*;<8Cza+P){+ zGzV*@bFz2<3l}Z*-PH_~n_lg=@53I45{L!izZ0-U`vctmJpv zMeM_?B>X4bq{#xNsT-r51J8bwatQb^3i85@?udFdj$WJJJsvxifuoV^hjjj83UDv_ zm=X$RTH0_g70BN*MwA!jJ;2nj8^oB;XhXcYu>Y?> zM-z|b0t--%tn+$D_Cd;I-Oh3??rsEB?rcC@Gijx+QR4eljVH1^lcjCav?Ijrn0AGU z%z1-MMeMc^`-c)@MkF4PIQ-7aEwV3#ifCP@RQPuVJ^ry_i*_?t=Zxc+&m4riAg?LE z7Kyt#l?9{im};B*Aq=IDXltIv=xw;wO>mZGA-ASV8EW0Rk;tjW(h!?P#vVc(+jLLrWYJ@L9LU6^h$vWCzcG4UdG3YHJi;3ERgBm( zAFtb^{zIC*SGUCh=?gL z4M*Ox7V9|mcYeJ|j_!-Lk-%^+qvYE8j_IE72XvM#t?;*iFFB(h>c!kY)t^^yr!`KA-B-0#+zrtNmI&uV? zbnNGHQWl^JxB!<9SVqToP&fpt?uT|KVJvt7f&^SP(WC1Lx|9hYTV6cxsp z6G>T>C&BCOiiRgHtc+o)f{X&(cdC&v9z08r49E zgay!&xrw&nBMySFs{RWg9Z%w-rE5g%8 zMIE9AyoI7Yk`_XZnOplT9p0osVhqJ8dpD~%;$L<5^BBs&d&(lGfN`h0Al^K+oa``7 z(58ONIa$ZE++H?d>;*w*`nw}HS-r}Yx@roE|I4!rb98ompG`mO=YNyTpmF3!c!Uc79{>VD z{k}SX!Id4)zkqSgq?#cZKZdVyfJQ=hcCYI%Fq55s!HLBy-70>-9{BO`6UG_~nO$;O zbbW$yiyt0{e;yDevwu;G4FQQBalj}I{p;fs_#b2aj38l@6I}y>4FT+f#Wa)sQi2`R z_fXP`S;b~({QYlU_x*2P|GgWoi&gF6vrZ9P3n7yZgPhtKWWObwh28nEDzcNRRg*z*Hhpg+UM`|%9gMVOJ8ERoa@W1+%uM`y((F;NF;6o2p@7ouWi;Gl) zW>+LT#J^;Oj(>pxenyWOO+f+#xY=Qiy#7&q#S$ZsCPw0u$rp)FV*n*6-CJ3S-*921 z#ex%0JYhip{wP2H^PkVdBU`v7o_+3_y85~l{fQmHZ!i8$f3#(DY#{?Qh1raHL~hGh zzxvhUqGB=>OZ@kL|9ADi>L~vzcmB2fQeH?o@#FES6jf$tD8%sr^Y_bN{>tg6k3$0x z(Ady`r?FzllCHsbzWd!#qel}&o9WMen~l?xD^N4jjdA%=@)tc2B;<#OVSGmY>%@$E z#XsB7>I~x#g~R-!j9iQZ+d9TK{w3+~(Hsf#iRZ8BpAw&;j$|PumxRdHRQ+kaZ4v&X z@q@~Y{IF33U#IGYM?4^I`}u3*4<&5T)~W%fEy+Oe8quLDLUoOK*(L z{c-HoH-ic~PMh(6!2C7-+tc6t3n*+vb+qX)^l!%a$>uK@qbJCiUX^{0#=8DkoRKnz`ms}a=Vb}KH|A;40s82BAAuEcP!180)75>C8+{zW;& z8X6ItTt5Q}8-n}g@nh8qk%0i1;UJXef zl)7<&^~A)+UT7)gEq-{pj#5_9R`6r%2b0HM;A}_{DaOL^iU39II@F)@t2W4U(WUjL zEW-)n-{xOjj%)H12oM;bq18A5KkSRWv3D8H%SHAt=1hMZ|EU%=fJdkl>P>$fgbM39 zT%lGf7BrdD$Ip3XM*o6&Gn1HT&D%TzRViiAj@@w?L8GkDdo zpz+ycAZGXm&JL-8+fayFDnlt6g<9^liDEnEzP<4i4G5x*?Obaes!AoqR}punoAIw> zwINJ${XjX}((t{8IFR0{{8&5F-{WW0pHK(u`WXrX4t1Dz&f^ImP%f|DxDE zVU(q+me+`Ge;|u|Fokk)%zG_iG`J*zC@YvT)|NhCI119mG!xQqq&UQ96WRe`nIt8& zV>u4Ep=E0VkFH>LAuDrY_DX#yoWfBFH?`2k|Eoq1`lg z3X6$pAbL4;%%Qj3ax=BXX{P)os_CW$2c&F^;STf>5o?6#|)+9E zR_K5&iaN|+pfd!q(_CSe0|b~pf1~*5gqm{=5wa8NuX5!t!;%V0#tzIY5U<<$Gdv%|Tsr zY{UN6->r{*c}e1b)&+Z-C<_7Vto&^Vnl>e>v0s(g{$Dw-JrG+|5!670-GvNg)E_#5 zW*?uV{I56vLKU-1rZz+G<8{ZJgHoy4%QqVIp50d;%kN;09fBpEGGCo_TNAs7G z=v6~OBnAHN@)s2FqiUdoGZfA+N@|G1cxWGg;4sP|O|uY0+@ynOa^tbaPsSL>b`XzO zjc@q1#F&27(fOM*;HnQ2&%KD=_@BTi(wY{pIATBSpPiu?J#s=~k!C&JmjNh;b?T`K z9`S|*_ShiquK@X(X&7Y!L?ASEA_O)rw*3?65m z?Be2rwzQpJpbXvNmNViTy|Q0!?1-HRT~UBuc!Eu0cqL}a`dO6{qJ9))H&n;eS5qq% z9LdSfzvR^OW8{ODDE}6Rb-^w-TfMLOHd?Y7q$^>wFe22<^n9a)4>E$GOd-`F39AXN#$ zy_0N>!ltURHj!v!fnEH!#Ya-Z>>U2gY=GawBu#9V?2v=52a!`;zBZ?A^G3=+NPLdBu3jLsM&f$Ck>MA6r$syYcdWJ!|}B zhd=)7r~mh^Crx2yk5ZlLBg;c0!2$)8Haw6!=}`_Y+*BQ&=saWzBLy4l58@J3lbinS zCcEZGyqJy`nLW$d9|6@cgIZYnDVCKwz4aFQ^(Da^FUOow@uSJ#zHT-@Qd5 zyd_T9Sn$w)Ka`Kpwj@<>^>~4RqJ!Z04>~jAvkPk!T1sSWnex{JZ7;hFGO53B8+sub zpA-_Z8)_r|_4+$Dq>So*!zZ4SG`5FeJVHR@%dya#ZkFePuJ`+(8NvB!dw{ZUc7{EE zJ`!bWZ}_5wRlo-v_16aVTrbdv*un>vUU91ec*Z|!co&4GsvA>+v}gBH8s&zJ1GLu1dDxhn_`xgnM`buZ3pa_6 zDt3U4LhKTCLcozjVj+&XZ!^DGwQK6}fn*K&N)g*r?C&*yqx|FeBk?17aTwv>3gI~% zFn$0T@~>mOPIF!yl;m= zfDqCZ>4Qx`z$La_KVz_g0LTdTt9X>=&XRa)nyFm-G%RUY#5&+bDP;kg$u`l0-FD5u76xxK{)Y7zPOI3VRPELL4 z+iTYEudiOcX7!w<&nM9}poQ^+yV2N;A2clORHF98$Bim?%$GL&;FJM^ z7Y>s+(PtCe#t(|EmfZzGWmVPOxz8KhqsELLHDVM6u*HH4FTBufg5>OHo~f^|$AL5_ zdXyi^xAm{-4}9@~>NflUM*6WFf1m^VVTTOE&j6!iSiNKCjz=GP6vwoJ(=P=X9l9G0 z9Sk4!QG8ChI%fm6;b!sUjve!81vv`wPmy-@m#&hGO{R%XrKR|d7{jO4LPNAzyi(HP zQ!VFY!WlDW9kHo2xe67fIPH*DtcdCl)F?g$rN8gY@DUio)Y}CJU}vE^krz;lGKisl z3cl$Q(LY1{prJ6qrtwel6#p9jCzQXU(lj@9WBfd@>5neZdCK@eecI+POSFMoW2wFP zBu&tswq@Yo_%a0GQJApWxhBc{WdZ&$(&jMJP|;rgsw;tn{s#u|A*J3TW z>mht013f6p%^A}LA6ZMZ;w`JZAeYO^M69KGZ@l@(8)EtEoP1bN2<c8duWq@=z|7OfDMr1g@5J5r4u{|<=rpFK4D83+%;QtfzH- ze}La{{!&#J@*IDt5XV?QaRcl!V>O>j4!y|6EIWa|fNfI{UjTyL6ljHYgs)W>7-C^>cS*5`gL1{uZ! z8q#cYJG+KHC_w>w0yc!Oqf84U0}rs^r#ujy!bkrW|7d{%N(zsaHmaZv0r8I*XoW*r z0Xy(fM5jOHF>$n=e^G{;DyBbiLwx8=HiVBdqDeu%vM%f;2lq`vkD$)p1EXQ`9s8#8 zZSbM2ErWsN=)ouM$%Bz7=F!fqaQ>BX#7E~+m-v>yAwRMc_G2T9R8cCE6>_2z`a;Oz z-mTx{^=$r#Avf1Q>-y4@rTW7>;U1XP;>H3QeMllkQ>9_ z3-}h4GMETHq-_YWy^{!Ggl0Pq3?3gpGmHO>Y)e;)xW0kiV?d(s@YaU9))POUIP^r; zsYBN*1Y7f3I+qunbLttrdiELIeP~%h*`lRyyz%-Q&&^x7Y5lG}n`^oibiV$oE5A1B zAC|3s_2F6f)b88Ad()oUUHf)b?Zo4Catm_?pV+gwXYTw*R=)J)ssY1#TzuV`qmLN* z;+)sL{@#3cAM(Te=hlWFm6(d5hVGRw08Hgy%`@XkB$y!XNTLJiJ3=d5mByP*oF>HejRjY5XX zQOn_nukJL|-<_Hx+1dh^K-gGp{4~@z;@GBA{LGFi5Y^w~0}(u<3I{d^IYW#K+Z(## zbN1}nd`1aX4KA8+k)uok{rdJh>BJMnRsvrwnLBqLA+?42voDXI;$PFBh1^0WbY}Wf zQ&kaEz&C&UTfU2*{ru;6To#&RBN5iKfw2`fqs@wl&yPz=T^zF+jd2Qxuj(p2euChp zZ{6fD)nb25&26{ciuQ!z)+Rn3|KS{@g@B~OZY*y)Y`a4R~ z#%Hg93ABd~u@rvz!CR<~^-zR=i_a)O9~zT4ZSjLzuRp3_ zYzjZt-1KK#ZXtrsYB%8b&1222iRRYUnr*GS=Ev4P)Vg;iKE_QX;2Hjr<*nUI`0?vF z->-tIi57ldH<4&+T(+m_pWceEtmgU*_dLc11A8S(N`jpHSfUV5Eaufh%lg>%r-N7O z5{+ye(cj}+S$p~$KT-X8Ed9O3QdTe#eDntW{odnK4I_g`*s}R67D6_865%I3K0W`G zq%5Qz@8K&?f#>7XPFV`+w(d*!FQiO?P@~NFkjjY9)_))Qo5Bx?b$&?YpH=3h2>&S2 zmKy)wL8!&?$M&y7PaB^-epG)uj_Qx*f4%&r?y{ZZvlipB2IHK9m>0_oQi>R(s21rG zGc$A*4HKLd_L&MrCYsNfAN$VzL0)Tc@o6^x^J^Ep<^?A9cm*Tc)RjM`Ug}uEuTBjg z61q@fJC4QpClero<&>Xaq^(tkGF)zjy7Luhm!;O?AKkzE;3e6;jIfbA+RIQnU65NIH{w4oxYB#yGaEAmrarwpO zz)-&sLlr2N4o-i=M~U#Q9zujrTeP9ZJT@yN>hX0(4V}P_ovp9G-ZE=u^MemH+;MyT zZMQc3=MV5(56v^4XkE6Xb?cVoLr~SyI!ecPVMXP!Meu1{s6P*REESV4F7j37@hR4! z{wV^o*0naq()nSR=p|&v$B)Y8_DuS-BQfgrPS@XcEc}f88$Uk3lKdMw_G~6{lCLVF z|Dm)Imt4QGU4)+l;u}9~g-!jq3rl~lV!K@@2MRvy0xfXJ9F3Fi~!16&2jZHywY}*@^v4TXuVLgu> zJFKR%dH$>=!ww%%*1ZV$W{!O1_(6D~Q{3Q{U9waBI7#ZHG9*oYbcD(m@xzG2p$BNPQKLeO@EcK3j70@kW`DCPGO?MR|)Izhbowf7k8fr z=NyR9-USGRO;H{YIas@HJqL7x2K=d10ih&0dKoYV+mI;;qjlJ?l;e(yu@L`d)U>Hn zaROLqI$|Ws8=GPu;YYx!cBLrH_BEN|ba#1Ncz7C%vZ8YnQ|M`*7qu#4oE$M^i0?WQf_(Bg`=(YOr$ z1wyKSBlv72D_|#V;AVUG;^N=yZ*+zFGk=NCk4iZAk;e!=PaI33e?xp3lE6)0cFK5Q zh0!H^v@j=NT$Iqy;*il^*e{9LkX>3m9a2?FgzhKJ=nuDOasri-<5?A(( zUDZFYN6_hWIm4bzyaOlegCIcxM)Dl|%vbeOi9eo~IJsNUC5&gPrnk$W436nSxe>2+ zn&h8FN;&qb`0@PHm``;6Raex69tru+=C8#k9di)puf-2C-scwsCR2WS|Ki|pH-9;B zz&`z%gQBZ^)Y$l@ISoXFQLSWRI}AIHAJ$?^$uHGnPDi?jl^WhCSGY z?fPB@I0k=Qm++##6$6)BK)PWfu&ns_~GxhL_GIdFP+`3V-E3q+Mq7L0~{=)o-QJvEnFTc?U zlTC;pMuJoWF0lE`3v6yK;mrl;Z|ev4$U29QQ^>)5HiQ`2V()0e`IqAZZx4|l^~>o+ z?NI`mWS^!#X#oV4ATT#PGlf=Q12ur-vGfOZ=nD8OK(#2vhces;JXk0QF@`g>Q*%fH zg`8sJ_+a>;hV!4Qj31Io8XW(ig*3u~kRSC-<$M@${-JuH0Sv%Od_0GQ16KOOYU|qU zWL&UT`jaTyLC3K`OtOmGN6z0?T)}PL(D=d&@!7Ll)~-)f?g{GfEplEA!RVj`2Q965 zstCSk-o1O?pn*B(otHoUtlW{qgW^*3N2|Dr0I(O14=fU8QW5++VvV=#{M+98mUJl$ zn7>A@qu5@|zeXx~M=&T1NH!eC7-h*vUO#iKN#|LR>DVtoamEgz7($f-)(cBaOSDo$ z`WTTQO&JO}@Ouik#YX`lxH)`%$&L6-c~z$4=O*x`SuSo#G&eQbZ!i&n`N5YVhsPF7 zZ{D>n$Sr6Yaa2yHa;o4GoC2|URZDS8*WKIqH`Ud@_4=w`J@SX==e)Ri{f?&G#+JgS zjcY2_ytZZiTU$nSI{X`F|MSs(k6X9vy%Wzny4Q%VTeojX#9LdNVPKV(K?DC3raf4KpuNO|lH-<~c>@5d7N z+;cbimmdHtF%oHPS}&6(UBUypI`QOFeAEbs68sqLj2R|YR8)A;ME)QvPo+O(On;5U zrm#8B4W6hMj*Fkr3iPZk)CXO0ehd9;_|(sBiH#fAD->9SUrS{aP{`?z!VuX;OA9sl zmyJ-B5~$&xv01ZbHa0d|x4w|(VQR6v zYi;$nOg4U~w<#rj(xnZ&;Faox`E~Pm_>mm+4f5~)#XY9KWGR*s|0p+nyhQzL@SdM2 zK8{o3XV0EJ|9s=Wuun8dBnpf0=m0tflh1Y6UDw!zH?KAh9H;PO`0B{<4~b7Ue|`J_ zSpSpxMV;E=pF~4C!jGMzK&?<~i=`w#7C&BpLeqnqJQSbSyCXLZKV&Ht|F(a+_SPG& z{r-*DNAabaDU#~n4)RwfIDGU^<=W3*RAI}=4{6Dye>y%SJ^xYtaoiq$r=hXtTrNPl zHbX!7vPuxtH@DR6X<0Y3wPImTOS8EqcFoXO5ym!roCE7Oti!Nr z*!Y8&l=x?y%d-ZR68|XFmO1`WW%y{NhNggS>jRy^$&<$UEJve+Vn5=%$6hP9-X%K!uvSf+(o5`qAK{wTR0Ns9Dl@u>Cdi-iz@of z_<@k!>xUJnI6kE4PaS;xw4*;83z-LL_{8Ss z^asAhr|EC_I6k2M9@*Acp#6LA9GdL3fZ&^6UT#j6Ll$efsv@UQvPL&6_qaTej@@uD zwsTn-H6i2JgMU(P!H1X7Tli^lG=RNS)`_Dw;$Mp&G(olTZ#RIAgEaiuiWMsbNRwd{ z``1VZND)aT48JwbyVvU2b&eB^jwrLuv=z;eTjtU0OT{IqIERqO!IPD;R|MlPgW!J8qXaM}< zjz4b7lqp9XaYQ|CY{z16zx~cne*BaF_>cb}Xj=XfmkGxzY*GJyjQj;!lpm;#r&XcE zBECKSVH^#VsoZlDONvi4hGMQ%`lA5-SbhQjfcd4y4*$=TztkUUnU>_)iKw?k;g?m@ea>8QVu;GR$neZ9C^kAq2zxAe7o#%(OY5B`mghY4Nub7YU zpRT_dYWe{)J%5!2z~#8Yh#+>m`HSu$XXaBls=~o%_c67A3A8+WLRn^O~BAS}J!|PQRya)uO!O zPDNvm?>6cBSf}#V#=50ddEaeG+%YI`LjJL}v6n01_8?vm;6WewAIp!$8xs4L=Dd7* z;>^Mrep4L(4#xQx{KHj#e1q4r`I{d9Ocy1Lg({OZo4*>q{*w9Y#|I8v96t3DSzq~) zBs%^u!cCNeEZVv8Sz{+hA(SHih3Ei0etYv*I6j9Xmtq4uj8}lQ_-Tl@Y}{A%$ITn| zG&UX8tJf*L`jr=#MB@iN+W7XTte?I9IEeCN^GbXA8$UFYE(1;r!t#iF<#aQ=}%x@+p8fjPN)tyn}4}BR{RiOPvxWs4L%mc<2L^qHRJ~y zW-bWW{KZZT4!S~r{A2WTsZpB+A(qWcci4VD0a z#PBaSPtOfiKhR%h;DoTrPg0|2GDW@OacFG1z}_T2PtxaKPMgl<)&^a(6MQO!Rr(5O zXqCmG@x!}C5QU23RtzWt0LF4{(%;tq0+%T4!`6~$Uj0t$mOU#7}n3@PCg!*NIi}pp}(Q<$6r6=W=Dvl zfaJ{qA7bDH>Fv5AH%61i5>+8r6HV+zLbbOU^$vf7$efOdzv@A$5U*~W{ifA)n~hE>*_bH-S*TI&zyJm zxFJJFEZnp3gY9n*Ik(%;Gfr6i`!y|1t6Q>IPZyKOIia(3jfG1pyq{rwN!w_*GG z*7_jG59+F#O3L#)m6Q|}7Y-fWXY2Ys^M1XuxTvJ4tb_)&r$00hKU7E#gV8#6oKi(2 zibj!nsmp5@#}BA-kD7)g+tPMnLg3rq`qmYfU2ZJjbKkw)yLY#!Ks*rGprL`f!Nc?} znlSOfhaN;ZK%RQ?$>WYaZpM>O3Y8L*Cr_q&fC%wD|AYwjmkH1T1vWVd9esuVwV8!t zHfD=|^r?mxQPd5mhQDa>q6PEkvx`KcAin@#&(Zv(xS|;Jp}gS}R0g5US(z4}(vHQ^#=^u zl7pO=38C4Z?u_JD$NHn2(23+3={EQuH-CGKO>A5j;Qb!{XdtFnh8oFj%?%oZ*2d~!?K81H z#{@l(3ck=gck#BWm!|)E@2p=1&5anHFcURxe0Sdq)5<<~RnLpQ+)~tOU0w6PZf^a4 zzx)YBM?W1b*%#Y~$9CXh+W1OK0qKdygZdS*myZoTgP%DE>EaE8w2hXj{v}oNhv%OR zq~$MU#EW(j|F+?$lD~LZXfBrw zVJ`40MAzyZi!f-CEbREZqofK;BYdv{Y+-=pod68h(_gT{ieC9y5`ZxPJfFQiqSZ33@?6b!{NJkTuj|ih@ZvahBiKa$Vv-! zQl+4rjSyob^WF-%r6SlU36pfE1~6FDpawIbWDtbBCR7Pfga$To{l(CtEPJLoKoh2M z$TB;JoP98a%O>h7e#}HngY3m|$r}jl3;NS+Ja2#>5J3q};C$-NPT3hdl`=Yqr6|)* zSeRqcgomMjDX#{Cp!A2#-^8w6O|xe<{Pd^6uH7x@0l%1xd-L%Y+~WukwfXru{rbfE z4a_a?+FDeSXl@MlRVOxWN^ITPT3?GV3FqSX6ZM91%l`eX&&_LIw!HAFs|vn)ZP2S1 zV}d%;K-dHQc_6>|^(UxsYE#hIKj`uoYD>)w`h#ZXFzkZ%GKTj4?XRD41Hl4QK_=VG zc?xlbEXM+jW5_nn1kSZ|B0lz{+j2orJnBS5BDMxR_nz*hgOfO{;l^7P8b5H%C|1V1 z>;k7DzGP9$YJk|O?%uHkcjwqPH@CF3w5hqV84=7kDgl_loeo)pGo}|eMk9ObRNQIk zz`jQxKk~U}UwL%*Q;7J%Lk1jr;iwIJH|*b6w{PeEmZn%)PWN$#pL^xSSKal%U0=WD z+RBalF=BKn?N(5dlUqYn@c^4z?>wYKIDPlmzO;i4#YR82N!73j6=qI}bQJiYwiB-??|PG#cfI zP(TEcvss&*4EAD-ZMUnt#JkuY~uh33l#0 zKQ?C$_7Mrj3;J4}`cQHHOc(h7yW`hIxO}9i^1ft{_8~5nd{IOL z6T|FjR3|xhpvaO16Nby|EApjSthamAp!Ri9uP}aN{0^l|BhdA>Z_ZB~R|a@s{DL#Z z?~qMF71g2Giw#G*`;YVBpXBk)`DdSO-u_s+tm|aO=_brvQ~c=MVDam6%JJL2YI8XX zSKH)02|~^#?VGZbqhI`zLCHy31dvC1=^bgm6D4BH+6QFE7F2Cz@=1NA@mo0}74b9|J}N6+k*M?3p{rcg!!&+uHmX{n0YUCb#_5SeSO`K`k@WO8YWDf*f>n@TC$b% zFEZgP43WU{%x&%Mv~M-FH4NgdTaQ?&ib>%iLx+H_y_K#otgWrbCT*>46s)QOb&$OIbmEEJ0PsrCR4UuSOT@3O-u-x6TRT~&>^u2{6aGa|j zXp8+YjlpvN!P7Fc|1;RL9q&|dNsy^!D5Ltu;Na7v#&JQVUV8UUIu3~6XxrN0$=g&R zSBb`}s^$id0o>Kuc5@R~-G!=UfL4>K7+zg- z(5R8G+h^M0W5$eRrC@JNY&5ItOk31=>Hb&vT@(9!+5Q(@#pGTPP>28kKmbWZK~&3* zpLqy^%W)eW;2+3_Ng*^pDYNh3@e6k4r(oImL_+$GpM@jg-~DHxe5&l{pH|AfJyR+a zc?3Y0qs9Vn8wjMS=wXX>+2N$+P< z9Z9vDV0@?-ol zQL!M!kxIt=^ae8_km2?-n}e06LTLJuO^p!xpn=LVbEj;wy|pS*T&l)-j1%DpSWE)) z2@EAjB)X6X{1g;sRVyD;{Q@l0iVoYzz=8MPtS10|N>LX@Dd2UHm%c-?_^*A5BAD6b zQ2t3VRc(_R;#SGz7G{YSE83@gGQPF!l5fR?T16z**w6|8CoY_aXvK=|TYu8?(_5oL z5B#wtM+MTeVOT#bd-^H4)6NVJKRi2jY>_FC%1V9?myS5%VUXs{Ha*H6QQOt>$_tUTdf6icn zU&@*VQRbf=zXRnbxZ-apg+M7{{Ig^5Vi!PN+$e1yOnqsVuy3Jbs*)lJZpXnyy+r@3 zTFf8H<%@8+zPW%zn*?4%Z92|n#ikN^6(P~t z&{-u=u`c=fAtOF}Hz{kfEs1awQc{pC*Ov$MW^m9ZKh7T4rV_{}k&`rD`sZ5^GMXDv z*HAMgS6P$GRcD@ke3j;oZQ?`F_@lv;F(@4MG#9q6+qrYb_`_dyz>$X?w9oKKE7z>} z)z9zWv3*BdOHXB2)j8*%e9;+~eEsUn-tm#Qckbw(JZW-fR8?nBM`bpfsje6@A(!tg zK7G&Xifm9-onyihsdV?4l|4Lw7Mwl(lU~*ku&2!){^CoN#(`?dG9Nqov$H~3#=pB= zoIz(Sqfx5tMlX3j{i#oV-~%7zH1EYd_`odq;jBpnMt}LQKJhi9M~~jNZ9Cj7UAp9n zC!U=9)Kf^NeLnY`a~O%Lpf<@*Hrdybgx|=Ng6Y-|zZx@Z?n!52icg9lz;+TX68tm- zS6y}GoH=v+GMYjC)<=`;4*Z~Am0 zvYJ#@uyhhtpPu{luBKh2H0^GBWcDMcpL&Mor!DNsV7Zjf_l#9)Ll@ur@Drzzy-zTx zyOAWtqB_FQeCAW{|G)=G<8tDG2Of}Q8C0cRe#*ZP_ww%q3O=shYu~-Z@1>W1e$&Q{ zWX2>ThL8BhH^1TXbGhs_*IawnnP*u~A3XWwlRv)k2Bt+~`;v*x_@$s7B=`mY?~Gq9 z?yX-_bAhg}6m}ZR+XwuoAOC1-9&cFmksOJ#n1{bHzGRqz^8e}a>-b&Q1V2j2+_8O~ zpHxHhhnM09PT0tAf9C&oXcBs9Hh%iXuh%hHe3^Td$FE=`zhC8EkF{@GgGpnEY%`I0Js#>qT=GqT_=tEasd1anQ#&m;=7y%r=yi#4-)3o#2FSY&U z&Qb6Fc>5uTe|Kx~(_=@RQG3ke!BcdD3~;J~sFDf%Ql5v;+JdcHgJq+m14O%R&i_FC zg+lSx|NO~MfA-Tvace?A8h>lYgSWn}{($`tIOp}}T=2#>RoC#EOrQAWa{3$J_{Ohp z`!(ZNIFdJE4{BxCj{Hla)PK>Q%jrCyl1imR4nE|j8*dsmyy1nVFMRIvm$tU-A->e5 zF#cBmYvXLSdua$ntjcuckw?Ajo$osOn4^&|iK=seeFwDv^2L{o-<7LYUHXO37a6%m z0SlRb`qQ6&&wDP~tN#p;Pt)+5JAQNBHP@GTe>;;IHFESFzqx(Fgb64uMhvHHA%aAN zcn~9sFOT1*T}|Ko<~NosT`Iz?>*qbeH=O^5#7yGD+aHRvQPBQ+xiiT?ylmn`~A(7KcB0r@=RM=fn*5V%TYnq%nrpmIpWouYUEJ;1)s=aaVw z&poV%1Oyu)L{~iAiSj++@O?Am_A`Cu+-yc#Z~x1a{0sOU_hn1|x^bP23t~D2Qx{MG z<^lHNrjFJv>)RgRxbcXwV=kOJvL5SXZ(vO#t&`|eVYIK*~Gqs_^0QWh<|AO zOmSXSrhx&gYBRC2Zz^Aa;`S}&UuI1+#QtfB^8dZq4raXaL5H{Cd1X`w{9xk^GHz5I!3%q5tMp@ z%<&U~k>g9*N-2RHP8cOJ5Gc&Am0;IhcG5C{NZAbv8YhDdD{2`u!y*M@5mHB^SrF;^ zoGgsQDj3{L2uH~CYK53YH3+6J%;@?>r^z7wm0N(|!M;tVsx#8eM9RRz7&iWivr~@J zz_8lbky!ZD)PM!LiPyu5@?mD!^=3sf{FaP_8OfF+t}>`M;vpoL50*xG*KQEmFK+{Qn9CKmh@ zRHd9Z5u$}%?d#97Y1iV}6t%ujcLl z9!_Q=O+Qo&OuvRuElkkpMV{(t$dK%`DHUg*Q@ZqX`8)0?+<9kV+42mNuDN_M%tn9u zLuW^8%?CeRaq1}+J;Dz(k1z4#*GEbuIo72Qe%0Hj|0Sa-DNl%1wZiEUC0u^u(pKk7 z$_LS)@jHloqzk_wIDS>`8*t1Y%JG|-{s)hreI8K`ExM(%l+@*~+|vwZLCYyfA%?B% zl6N^xK1Qd3*cwkFI&<;gy9v%WmL&s)NkP*uAzr3X3~e zZ#(VGW4f9;f4S&4`FzjN$(7x0!^Z8KYj5p(YQem_e|k>PoSQOj-^{SEtI%Chh^n%= zTy2i!=C&_tD+GnA8t5XdSt!*!habpN`zcQ~HJJ*e#Y9u#H$KHdtZ)(u9Ev`r_U9)d zPW*WEx>yfy1koakJ&mp?t4iPTj(2_St6$r+Ws}t-!pJ5$;1a#|p`5ZW^RaI{|BXMs z;YRj3_rCYNS8x3(Pv$~itmtcB$pEwJL5%jTKheGm=G*01u&7Z*kn-z`BTm``kbH?W zA#wXU{xxe?zvn&geQeI-=58we)_;5J$)}#8J&+(m4X8>XAN=gF*=&vow~Y*TnsoCR z?_)geuHW9N(f|*FAN=6^uRZ%5Kv9ZAtE{79TIX`Wn_YnP-rzU-8k`1iNH{qnC|wrR^2nMpJ%FCVG@m`_nU_8)Z~ z#y_o?%T)<8jB3!Rx_|u6p8d$pH{WFPCFGmm{3bU!)A&90)KlJk;hXNb<4&0-ioWoL z&%gfM*H7GM5@5v>pqzk`Jo3FCt5=oaBD%43GDP%0Sq$_)Gd>L|8Wd8YI9>3Ds z=_DSk{dE5~R5A@Pu`di0U#74j6->4ie>zX(+EcK%eI{S(|K5JO|B%(?J3k)3iA3t{ zjV}avWy(*E1hn%H^C>^hzvYNCNuYZ;J64*^dy8M!s2~57O7VL|W{p1br3B8 zV?TU!$?ti@kGrxLz56}i`OddbKl9A{?z>O*iFV>tdkW1U>RR^PhEKes?#$O0F1qBa z*>RVR8gW|f33H<-xoc#LcNBBFqlZp~=4eG@WVnZcCTweOTky=h1q&D4a?353edV%KPdNopP9GV0ZkoqBsF}HP@AH$;vbaJf zvle)o-^#$M+RR_WKz(5l?rGUmEal_&$3S2VWl2Wo>VIV|BLn_J?w{?v|Ni@CJ#as( zL|yik%SZC?1%S-HmP>i14N~NbcJ#N!Zx1U>gGrgiKWidjdH2t9`LX|)eD+nF^)Jv{ z%+s}FY8B>41gL)hc=`CHKgIL4X>9;Ozv{bJz3NrRjU6|hG#v|_+_&7mq`LpV_~MJV z+;lS;&3}{`FP2CljQih&5z_vF`0eQk3M|`16?CisD}~wWXxO+g%Ue`HeEIkd+FPQo z4tSHHbQNO0A*7-_EOrMwR|k_1)qy0&_T?mpG5@&jwsqm6hk+0;eF@)1Xaa>K^zf78N*oZL`>WA>&3csVY zvAuoq&K=L~*uG+S6SE%kH*a3uwEM%|T_=p6IBv*LAu-780P|)0U#Y^MQC9As1N+}{ z`wqnK)v9)JkG*kMf4BpJ$gX5NM;0oK{nKhz{b~Fv#gLn*xPK(c+c%`7{0pIHHu3Dg z(*761nMSn!5pVU;HL@%+#bV;0!m9u>$C46g$#uLUnp_;(6hU`)^u-?rkIoMpcpOb) zr1HpI>4-7mkco;0(%>5Z$HTaM!_!O~s32Ld@_1;b{8-SakcuJB+J9t<0^4vYLdo$% zwq%L}-BH)8Bblt02#jZRG=8%Yobj(;nMr=OQfkP02t$G#-(V>pUT@=nDg?izN)2i4 z_>m6~m5^#0&FhQJ>`N{+l@Mo|$$yA>GbH)hQ9V2wW8)vwN&^5KM{|t4I#Q9nbmJbM{?t&mMLp(w!eC0hE%at$h3(hLyRBaT6;p`DpQ+ zbBn+Jb4YQ={h?&8hUOhxXLlMI!-Rs*r5eG`wMVMDZi7%<_tiuh2C z|5xG{Q2(EgpR>sb7^At9VJ3DH_g7P_4KF0!E;VIG?CVZ%-liftt>|D+{>_fMXCg{F zR#>?_pMNSZn*(t9++Rws>yU5K;>=ksXDAqi65XNsZP49PSUhLLs3{G_p2EX-K0j*Q zkUdS^BgZzr|FR3_&0DZzUF+h<*LAe=BSX<4ubwjb$PvYS>HfRs){Us#xMA1$iK8mJ zD_Mx9X>0qorMsi8nW@vJXGUc@yW5LhB_8jO3R<4C)Dd;&OBo)s<9TLAnD{A(S?Y#X zlfbnhNXn1!D|~r)PpyIU7Fx7S{!HcgSmDqc5TcwEzRI$5UgI^Qoj>{&^(^)^S#_iT)TeF z(j~k?vHZD(3p+Y74X2g+zyl8W{`bCTusKqfKrIZxAD;aP@^xq&EMK)UE)fND=g#E< z-nFLh8`e1dh{F$e|Jl>hdjGw%tmcSSS5G_bbY!}48O8S#|2*&fH-7&IKNQXU%=;gH zf8F)htBOM;`8t=^(be&R^aYkMIQH0M z-us^SkgXj5Rp0sUpZ|DodwV+oc#Qp$OD?(R5BEq`62FAFcPra>H9h?B!_L2k2NaNE z%1ixxAA0B^qeqSA@Fn6`f>okdK6{jgUTRUtBfiS~-}oncIsY!7f&=-NU`dZR+y9&5 z3;e)v+x;`*_iw_Fmn0ZmfpYw$^yA0L@|@-JQ~Rdz>;CD!jgHhJlFe&++h^n7oXkv6 z7Qf`|$B+C^Mp64Oko?78efT#%b#X#&B&*R@InUf%#O0Ih{DVJj|Eb44A7x`~E+DBE zsr~vFvbq1K{9JMEO@b8#U%upGtEGSY>HGi)B#yxxqnJ@D+W=q7R%th_;72D^6e{KzFA`S`~_+1%9p^!%r+$=1Sf5`|#oa!puO(J}k} z^-n!I{K9vA{cZ35)YxGs*S_YN;EC?2ji)P_>8OlCE#$%Ph6T@#4o-A-5rBPDY^i-k zzlNcbukooE)?iWUkXwUoeR76N6sLdl#w{QHkB?q<#bs~4;LT<#6?6Q2i80qu6hfpu z;!LvPCnHMqldpcp;1vv_73W&j2gcL@)qGmNVVKI~bAtQ0(X9JsZP~o#rkidWK76=k z<|Ez=oQxv>aU+2RY2akjY5jQyK2!f;a=;SB4_H(*$!s!Jn$7kHk<&{Q{-t&<@4D5ZNI*4&AK&?KlUHx zlz9MzGMti-e>i@l_T9X{WPHXx#CySE!w4qw43z8pQt=yg^WIYrNboK$V#G2R6KJDr z$dFeaf^F+E-JPy-uMObG6jy^r!3Td_Ds+cc)#5@|I?RyaKMbBwmjRShXs)}ViC zUs=c6|22D>f4y|+W1C;x+|kA@paUBkK5^91Cr;c~Oc`HPMEh`f7@j+Izviya1zWfL za{1DwyLRmE>iqVCc^@v7-Y{i)V|8`PKLxRo%kMXS!DU3nc!-34oCk>-58~A_M%Bh{)=S zT8)fmGUdYN>9GVAl5H$VsokU^AG@0Ml9>T=wRVJ;T^%i0S{hm-ccGYZ8_paVOdVUS z9$w-6E77WP1$yPH8nBBv+Bd;KgoejVKagPYD>#y@35LXfQv7PW*xSqER(a$_feu1#O>qnlP%@8q1g3Qdh{Rj!ggzzX-%@ltJ=^UxGMoo8=)t z2l|i%J=kI}O*PM_%Bj&bRZf#nR-o=|D*WL4`9~foX@S!$i*8g6Z_Iu5vdXhw8;%-D zPSbIa_7j2TKVsaNH~C!M?Kh+1v!4r&I=b}r%S+4Gs$T9#KK5ugdW;#JJ@6nEjElsa zkU|o5as2;W{3h)K(8}8%VU|7Z&kmA(v!E0)lgn#ke$wn{idmdnN~!(%ufG0G99Rgw zeH#S7`8M`V;JYp{ep5$=&B_Dh+KO*wm1pXNJ0%Gu>$Fc`i4kQZO)-FU^G><13ajT% zE3^#e4Wd@%M2(smNlfMIKrU|}Y#%wQ)a1vQjBpjCz2P)}*2*wb+wx*}$JU+IL%H`< zG4r@d)b#wOwq?(*$n;c&nYQyj`062bHM#oi&aFESJ@Vj8b>ZH7{(8bO$1ESZX8g3_ zfBg01yH~dD+Spc69S$8cq_fn{3@NKZ=(bd`RN%fBlN*RJeW~52HDOGF3h~_5<4rJB z;X_0iEPZkkjZ{nruAu8@S!rVQ&7U%e|CF^vYc)+eMTpXxsF9E$6DN_*k5Z!O;)~z+ zt#5s+xw%<~Cf{nbS`+yyXso~HHF}d}`*wbw10B?&u=D!Yzn&+YV-%>tym^A3ANoDx zj58$JWg}m9p$7T=53l|q|AkYv1*1ld+G4-KGhqBtcS`bA$q*o9&%GfMo$pD z_UzaG@)y4x+AuU_LiMohb1{GRS!cyn^QAreq--X<`f95Gp{J00_{fp`psMaLg|+?X+Awr<_Z5wA5p_Sj=5op_S-&sp7A@M4Bn zW`;T?DwQwO!pKE!JJ6}Ph)f|(q`06brGkq;aPfD({q5aY&G?ahIj=1GBQN)p&pb=a zBQgf;b*tX!>K|PF!yo*Rv%ZZ)_ifv>Pt8$u`ByGqwR)8VA*v$$(T{$FROeP6p*o^* zL~4!%l4VFu3fp%f9@j zFQKTNlYHj~aCK~7BMdi9+WvS6=RXkzm_F?T;wY{;DT7HwruMZOCBx+_ zbi7ujB^zp4d~s~_3wb&I)c;fbas8&zD~n(GnBjyl*-j$M{YMxaBpjK1=SNv(t+;&_ zP?0EwwWm>+h70Ca!tVr`jNbU|$B!f?n^=)2r_{-ly=s@Qs>wt1V>}}tX+o6XC%=gk z(ti9)6xjr>7r)ATyr_h!X7ZKC%9}KOP^oIu=HI>YCBlSA212N)uYEb2;{WD#w>W8E zzT|yQE+t1_w?xa z9ox2T>0!|kEN7h-zcc;iFMWB^gh|I8dz||}mYpzu90hIIZPuj4!`vb75EMFXGFlv! z^i~KfdT6mgo4_*WG^a9g!i4Hddm{iaZaU7Df|27!jU72!H#5!ac_qKKy<_vHO+Bn^ zBAfbM&0z83MPL2u9wvZQmwrBVGzdAvp5%f@fmsqu#L@+%y0BuhJYdM+d6C6}g+ zQNCS*we^qROjmnp>xyvw!b~=&$w9FW9?OYPe%OP{yIfURQ(sY8X^v%qYwZGN+0fcN zYweobmM`L#`+g0>E|`A6+YdSz_Y)kUQc%NB+WxR+$C*>6?b|Tyu9eGfTmAw!UDoYx z+B&+G$&PaSIsiUNl$o@}_&x4)5T~1e$~R)w(*7&8uNRT10r5+knNQX5hpsM_$BL4A z`^pzeL86#kSbj=D^Ho#Y+fPHQM8{7~9askcu$Yn@&e;Ad=zqyCc1@<2W>@t}nr^&v z66a+GnmmS-2vWntXxi)x5*cS_Q+VYK(eo>!TKzyb1@vdv&Ec_=quNH^rc!_CfcZ54 z*JP8Duqemz_*w0b=7w|vA^xRqk=RrGa#Gm%Z>7Yo2Hq;Jo)ZSZOF`raKk4jQS53O& z7tu(g!IN>l9tt4S{;6ql`3$8gtsE+Z%06fWkZfg%`lN1&S3YEFTF^RiJ-Vd_q?ADu z%1D4krXAE|iDN)PoE=eC)Q~7tQcP5<27`-KV@ju#Ak^|tBS2uxh)UGMENFr5rk5!% zX~@6xcfqoi!H^-*+ujs0OQ1BWR3*~Z?l1_FG%zPbK2i+1RY!WlKV{=Ot+*t~733Ha zXt)t&qID3@_1FZNrpf4tq^evCa8PGOq^vAJs^JF|NnKZdEC2M|sFV42nz2m9gt1jW zzOmxaLs`GdIJe>^f)50h^7)WwHhE+NJ#rNhy%=+yrfIMI+SQnrF=_yTOrQ)wL{t+dd5L$-+ z|IP8Up3ZVNay7tBiusZ02l<#mje=ShIVF1*c2ZEAzR**TS=oVUt`o|wvi;@jj7%TP z7tv-(3jpouVJ+C|BVX&tPzN=$Q3YPoLzmUlU0KB%Y4y2G4a3d9-n~rg=jV&77w-sb zx`&RckAlM5HCxU+{}^23Pq#gG!s!PedC1JMV@E7qv3%~M%eQZ6FLalxM^@Q=rhIo# zS5>LHM4YIypuWDktFw^dcZq~Wt*noq81{xNZIi1YqDbYG;v94WEEfd7io^;yKSoXb zA^ghf3_*sPVpB{cTVqwu4_%?fjpr%n$5O(fLmRk_f91EojYMm5C45l%)XVhD8_z%g zh8u508*eRWq~H7|euu?7$e8~WzX4I00-68R7z6c6;fE+bTC&{FIakgN7lp8uJ~ZRw-mwlqt8xm+vE5N;-0S6g&w4t1di$J z@-0EVk#$DY!rq|q>i|Vo3RTsZ>TY|h)vZGvb_YTVl1G)4;ZNer3g~Q-%d)xz)nZhM z+9bZbYEqYRv1#WEU--n=J6nJ7Zx*iB95MBL`)#+~a?5|_3wbA$XXP&Y%2)22 zHOu{J;R`#CM545|pxK}F^*7rgo7AOE-~qCOjarCwl}HS7D|{a$-}8x_G1k2Bmq>w(7} zfBe<|@-LL5EYipoVrYiZK2(LDOZ}4?J)8TT#=m~|R_drnR{p;w zemyI>Wp_SQ6Hm`wd+oLS(0f&tW`)RtU-XM#4D{tMe<{yI6?GYY3u=nFL}z9MIz_}#UU(j-ZwPd8WP!Xd+hOr?{f0)oo-UgEcT2NPCiByOVm zg%`-dW1RtOz_srQI$HEw&D?2laVFDk6twLMpS#cFQl?ftdYs62EnKraPtIfLOx)=l zde|ps;U(n<)~~sH<#K{?|6z@9I^duSXB_BQjhEQI9={?x>0f3HAMv`W(|M4pI+r_r z(&Ph1j8diL?X!1n*8ftUZKBAlcO_DO%Kg)`Dua1%IsYm?AbyoEk z!_Sr3w_HB^<_R{{vN~xMb@mu|||CGy*{Zsb0lI{q!30hlJhT`H}2o0q9 zr}&905!<>PhI|hoPB2)~{DN&eo1(8@8!cTI)UsfxLJNAh!o-upu|tD{C-W)`ZKa+M zlSHh;r#fhMES3}=33;@pe(7i+1Y3=^T4Um$SD=_fpq0*W9qWyl8CwG=#TPT70!e-*r;BiC;Zqf+nK=>?QcSksARpkRmOX{1=k@&IcctAdy{8)! z*#i!&{>Hbm#~zOq3YiL4oR7|q(vl^Gr=H@SmuTJEVE68@vN{+yCcEGMxiinooN|iV zLqp{9X@bHj)2e=bNAQ976&Eck6?#IJ>waco{>B?B-}P>vNhz0aT&XC~Rdrwd9Coy~ zFJHNG;WN+hWc|+VJMg_>jSbWHn||!E$4;9%ZD>P-*6vQOf4jT7dFosboAXvusdVL) zSDkatIX*-;{rk&L`T5`Rv%C)r;J4oM(UJ7phN*S^B&Yn&`U&+~-b~A2ax(Kg+{--G z*qt)4}iDM>Lk89bqW;^R@Y+ltoa=)QLp>*pH{?s_We$tFl9o?PV zHa9&pf7#mQTZ-LUyrx*4t1g86^j}cmb{z?NjFXq$@{B%uNUC7f4xW0bELC@OwHJCA z!d4JJRg8EEMMZ_KlmKZesBu)x`YOaRX`)UpNe*kV%my5bH~+Me90e$gK2RZ3*+FkcU%l-}5iH;D#G-Lb9C?Sp1A< zH^}3Wsoui)Cxs0#s9IpXVzWL%*sNhR0n#Z>Vz-jTjj6@DI`C&1ta!z776`5wGiL1g z3HrD1)G2Sh@WP{xvZdy&c05n5unSCzV1NSiuc`4v%_3M_-S#qZQ zv+o;nlpltT67~^7d&QMN0mlk6ytG12(#XYTDAGTv0&56O<;n0bre&wufYFhwdkMvWT%)hjOd+9lilGl^fGOuFWpYtB65 zOk^ONm1}=^_0?bh+Sj}i9*0r7D)qmZf2k`gQ8NN{mO*@?T`xZ_#H*H+zWjK)Hwf&L zIgy~sh;eTpoU$aF;U{AfU#i>6YCz%gg-aVn{CZD3^p&spk}ZTO&K+m?=ftn!0R}$7 zZt6d-GAW#h)gaV$RqRTOKR+AiN#eJhA97IK`GG>o?;F2f*7;X3GdxM;rv+2_soiqg?8D6uMV$TDEM}vl@=mHL8?7VPrhQ z7O&5){rRn5ef0@f?lW2rO5>Lsb3(7P zc$KshBE=4FFIcAYxN+k?@ySo_GkNmmmtWr5!*Av)|C)7cANtEfr=NbR`@iK48iqEo zBl+0F^s238y=qD3i2u4!H~B+`Xa!4hLQQHTrKpwCCLhxqhCU3@X&q%+ytpr+CJ5p%YA!K8RReu(k!W|=U% zg5`o$4o)QKQ}v)RxaqZ9c9uGuGr7?z|EQ)sg?y=HYi9Y=x{<^p0F*>!w-}zIB>^F! zqBNv|r)NFg@XIy23&kflZ+v3QwjKQDPE>sRfd`#BX^QMEi%A3(cxEa}@f$Ezeb9(e zQ-?KHbDv6fPzm2C;`Sw}-}qK%$f^slSUyK=B;)_I9)8H@-#LD!QQ`)fT5nDMq#N?V z#-VG$PTbA|RHt6q__-fHF4F~iEv}`iEAvk&p44ktIMDBj;-wz@I!$?8RB%O#OJl^y z!)3NlOC_}^vuRs!#Seqc+oSqQhL{LKXR6b!c%}nhy}zz+?12J)%>XPzZiMzBPt@H1H* z{OOL;&g~)qa$j?LFk_}vQpNg|eWkAWp=!-g0iW8JFD=cwhg+l}O9hS@pE>`nnVM>~ zU|!3LW#Qd-l&Y&}|Ah=e6?XN&8)G%~tUw&@JGC@@dZwxx!D12WkgsL~VEpR^D?L?x zGCmg=4{fW-Kq5d!QvxiRf5MgHcXQLe3BUNatmKY0qKJ(jcrT4shkVu!3kuT2vdFUR zHs|8jLm>@Xq!^pBdZ$#YG*S@RyDI*s!IC85i?-|LXo6>(;2fVd1AAgWK|IJ zuS&6zK4~L2ZSMNg7lS>mYC7{##UV4Q-u>Rpt4;s_C{#BY0npx&f8w$HjW?F&FD!Pp z(MGZExcwAu6s%bpEnQL?I<)$fQ^+fqGWoJHTduNV-^taV`$G43zMY>pFQ^PkJ9hVc z|9jaJPYU+iucEpd-He~uT~)*YyN-NH{UBmVCf*0X>E@dkJ-?{4gGD5%#Pf~-(+=iDl(nmyUz(CeE;HTt>i8s`P4>lp5+T)-Y&t7j zyyEdQ!%}M%Q&u-HH(nBgm8IebPd4>k?iQC`DP`QSI8YrWq&b?&{G}r{)B#%TZBeP|4jo zt(n7HS`<_p%7jLd23KkTAm7!~!&NJDjTIG4eB^2?7|3y24Usf3(6Msz38P2mQKU(v zWl}t-R-ILncFKE7Cx>*mma?04u1GbD;XY(SysU^z_!%>HEH~UF)Ty&+6!z1WLUmHB zReaj1r~34PW%m*~mGUoy=3d~htjyUj`I#10K&RC4OO|v@c3e20DgHxd9;~Zp3kIN_iyO;~`&Dw62l2Kwjw$n$xC=P<{JNU&Jmavw85+ssOHG}c* z_?hm}(sZ_^mXst)+gCY1Bs>0I|93ozd`iK*&ppL5DPV20(8S=wabsjPj_3rDuQo#5 z#{N^8%XM{jn*Rs{o7bzV%lLQsI+euNgAY95nLU3hf6(|n`Q($*yS|=3jo;O)S9=Pd z2t>X&nnr1CcRQtzLHUt7W)AV6xL9mo<1qD4olQF5lpo5NBJL`&f2RECl*v#1XP|wZ z2Z0}qe_;}^#D6#>A>m)8{wv~_8od3a8mmAbGGc z6JIYGzmga88{?04W>BXwna-x|n~seCKJrOrdCLc%RL{Y!%cZLr;)F*Z6wI8K$z`XV zP&aYXsOH7IDJHP~VFS9-u)3jledc##M~rW;2zoMlJfnw&MhWGbaM9{T-@fWYJGQ^b zJf@VUB=xHQlFFScR%~9k=JF9kudE$9q+(ocX-s!%Lq1$q8MSA_QFP689YUY@PoMb3 zGffSPJoaQ++1DG%UKf$)^NZv6+;b1>EAi@x$qN@On1A}|r#U34V^Rk2Bbm^qcGKkd zv9EEM+LsLU#DZ(aRQ_I``q&qToIY*(Men(YWi0ICLG9La&p+SZ)>c6_WB&P~%c_KC4go%?DJ$K>0>YtYXTKsb&5H zj&lE(vex*hQ}f{Hy^?hKF3a10BgP4gOixkwO;s0?EmV05T+#{|q>2`*XTq*W5IBmO!xvwSo|zZ$ zL>+hiPIygFUCmUvTSuUTR%YZwGcIh>g{_Pq2{H=IzBUPEnJDZ0IDWk>21S~dkPv)9 z;?YHuln520R-Ml)@r)Ko%?h5L)`1k|7TZbtZ$`0HL=njjm|XHSY5=;+s3jsQ5A7Y5^t{Pc%Km5?cH{5Uo3m58w)C))<>WS;) zdf~~Z=FVTR;Lf}6{NyJ-`Gz;V0gPfwc_hdJP{6$$-wFScAmes*mX^Mdzx6+5QKIfO zCsnZ&V_luhYDSQxvhgjVbOv#*v`CluRK5!0y$hesKk{&C&1x2A&5jtA{q!e-nmR#} zIF;Z_@;U4)UtWqOm1@-tkPm*H_07vC{nr`q{E$hGx|h!@Z=anmFq%@rudPh}p#7tC zqMM2<;(&)HxZJ{(Al6eQ1e8iK7mF{>!K{cUma^R4u`5Y)QJaJSj1*HuShO1wqXpR} zU+KoD$>oZ|v#TwMtF`KdPpZUVKtZ#`-E#byi4Qhr0f13k%Ws=i?B3DJnpluhQ&YpO zr|ursW#>|nS&DAGz?-k=Zs#6DPxqd@X7VJM`{fmkVzW6O9pHD*s7Xp8HAf{G-cV+v zLe)d_LxZWx!Vt+_ZQW?cODsGSF`{vV^&r5M+Gr$wMVqTk!<{%moTjtl&Z?7+wy{H) zkXH_^lPVr>W&FEXmsrk^%ke@Kmj6meYl9Lo<@R;?i8QN=*if}9V&6(cd%1wwno|Si z<5*-%PN-J!lM-v?kz-aBAhT~OU-C35qwipv&;L0gQ6@j-M@W22?3?<( zW1~np|CU!pTs|1ckP6YxKNP<)lvMtIef(O(Dy3*K%nIfBRj)c>@sBQP`yg1x52x@? zOy&qJ7GVNw`KL09W0P;8svvX!)H$|T(l<_?24i>2Oav^D{mj# zI8r|YAnG&-QKSk4i#X;B+qP_tqHDTZy12>2)AbdT>oaFnhv!!WdZ$B~NKM)Y_EP`H zGdbryq5G%nq#|-;g%JB``^;o6%S%}|CJHug+T_BKpSEvl`{Vp*#BPD$!KmjQ%Y15`r9|d~yT~}Vk zj}sRp#N@cN_O?#mDs63R#iT$w?6AX_6LI?{Oo%1hRIe&Atw>4jYbTVN_Ag3ty$1Hb z`JQ6iE+QI#QYPf(s-kMf#D3IYzEaux^}kGZ=t>DEW4#5<3rCIPgHihg*=q66H69Mp z(G+&J$QH&k^364PhH!PbW&ypMNMeG8OVSAka$8?`b)Vm|WpY8|xKg$%^-uD36!IHd znpf`Hy}PGt!jSrl4m*-xU2!rEqTo0`<^Hckajw8i`AJQm##ie9WcB{#zo-8dHmh51 zYZOTTzVRy+{=xC{%f;`@_P>mQeUn&4uTQ&q+wnz;3P_Y^cYmmxj*GD>`HUTdV4?Zk>{dwWJ=Vi_~J2?5&@Kvu4 z_nT1~S}z4VJA+3bFMa>}rNv9}0txX-+2MsSU|WrP)4UBx;z(-N5O5@bn8 zRR;CHc8c=gw;JRAmwlHnl7vg+Lpn^-{ZSg1zqj~}R<9}Db$7m!#fE~4e3U)=tg1I$ zkg2H!SRB7dd-93SE3U}jdW#t3K6R<0$QwF17a%b@o=w^Pr)6f&#An#3+Ba!u{g?cQ zN0{II=G+-)=PD|r%4{(gmVW=cV8e?^UcOb%`7e)O?)ct%>#bk-;-!DO_s{s3a=F50 z1LRXk39=jSL*-X4U-^eW{?Sr;!N_FBylSw0;bbS-L>p6OqlV`neYE?IySsk*>-;Z& z*)!+yLHl0?sQHZHm4VnS0Y&@wzZK&qMC&)?@4mg`*T3qx@%qA?IZ+Ewo@!IE*xLom zQ{rX-fi^yLh-r4tdfolvH|c*(h$MS+=u)vINwIvR*(g*t8OwQaIH~_rV>v%$j2GWF zUDYS^pBz?9S%uzWOG??<_$xk;xj4m4ncWt*BD5e1?WDLwYqRi@d?Ey@h0^4kZ>yUZ zDP~1k)#S$+3=$I@UYGQycdpD;RaWu#3NIn^{s{$mnu(0OAG>Vl@?51>xX&_$R*bqL z9t{Y2rY@VQ)I0`UGt6QdEL-B2zIjIvXRky@-PXX+#D2(ot3~=+CJx}F!diXBMmoFK=9xO2`>9e`WLHFEO|qXeQoR~`*JDA3q%)ReDRy! z^d_>w|8vhj_s2i};et0_;1xJOkIa5(_9Kr7XDO_%sbU4$pZxSE##vHb>Is9{*Yq+J z%HF?!)Y({RL`rd~(Y&>Shp2hyRI9|2#{03n!!lGS#Z3S4o3M#DC`;tC*Rq|<5jj+* z$aSUO-x|*HlJ()9?Lodn!$IV<>^yVr{Lq=q>m84L)=;ddRy1M3W+sw@1=&ITojl;*)F+ZXW`Nl)9eN+EW zPGLL=5=Y7`8U~E7#6@goeUs(Cr~d^M;Ml-lw1>3irM<{?3N&WS7`BRdhP}qGv@u#t zm)M9VMruLM`tXngqP9{vahNLfx{tE$sj^h3Q%eMl42WqxWHyHXIig@VWpaCCOd*NKAH*|cf% zoH=tAEO>Uuj;-_>#~y#|*S>lM-~nJth#xUP3VWg~wr_&glN2KrBS+;v^r6yq*OYeb zDm*toJL~@Fw3CBemF(z-ih`!*rr8hAe(dqbn1x~Oty51s{fskCH$#&pHlu}`xd#=p zpvm#a7Z%Jbvb1wC(|z-eHB+Vp!x{vS&G-?LM7}H=$8Rh_r{pc7lszc{6>*|uXzSZP z>!i7>%jf(+`Bsb*+8jSSY5ygWu{FJ3Em1)R6J?0v@(#-?koqbYyN`jkL*?a?g{h!f}grte*YCHdbJPr2~9$Q5R&(6N`pt~dMa4)m6hu@?O#=q-ahJR3gj3n8YDii*twDTW3`QH5BguBY<08n&9up zkI`UMQoY5Ok>R?UJfw*0Sbg@ARPm}&oufPWhxkVOn9{v*@$@&k(|OhtSBlOz);`>MMCjbB)h`-m(&Po(^h zieGgEan}^LZ_-9uNTosIcgFN!;R=2QN{_7h^g3-R2 z!Kyj48hDn{>nYq`?3mG$J+iV^;IzF={94bTT*?bRYkvTM_n4{(XEy1M$b zeUN>1(7*px?8LEW?XQB)T|o3%#j4z@_x|1DRCzt}_%?iG9T2};wrpagMJLNS<<0~( z)iua6XVZ7T@q?mN|IB!*)F3OP&ZPZ|>LH8WYEkvlEbe!cZx!8J|Eo4wa6lYaBtx;U z56RZIVd(2W!sYG1)|RuOj>h`TDm3rrXSv_S>LQjRu3I z?%MbzHCavVTXgyvryq9sVI6H9G=aKzgBOS&V8!wW+Lu!1jfOLU$2aMF>wklt8-j?J zO}NFb%P6^pCOfngi#$}UzEuCqax6jH?x?ecVU?VMn$eY}c+}*eZfLOf8O>jChVfGS zuBdfKYzQBpki6#EVEbyRCySz2K04%#aMYBbhhI8Kf>h|UnTGLEE|*wcr*?I9%-g!P zD__`e*sy~~jv6F>B~tkN`A^4*xu7K`^9xnZr#jJ*h_5Z+mgmp5}ZZ8d_65W7zOB z_SyG@36snNd)o)GI3RwJ&BV~+o!kDp>BUuhT6T2sj$JsWzV_geW6zl~W%{rYQas-N z*8W#8MVtO>Xsq^xA$G8Sb&zi_aJ668fc~25;CfOcKDkr)-1DU$-^yDaL3O2kjbyGl z#*7G>+e)3CcCk>(Oxw3uJ%sy+VSS@sFpzw^V26qL5`c2tl#Go=&9Qt-(QQs6o)6BD z$u|u+VWAlh0*?BBRLbD^S420#_z_`bq6Gt6=~@z{DY)OljKG zaO#v26P^!0to3|nJrIr<8BU%ohvXl*^jx4dM8eLNL>+sEM9B!pK3tfPDD^xW-JB>? zUAj2v4KL&<8>uEAG;04{J_n>qJtpU6aM#n5TejwZdTZg{`&f05b}7f|1e3;BU;KgWdFO?bCa3bU|7z-Xc15dJ6*jET zo^-MxnhpXloh4A}9C>ubDW{j_&n*O1yb%*T@z?Okqcao6#l9iO60x!`Ybg}+ix(}r z{<`Z|u3TZd1Pu)hXPtG{XFvPd{rBI$ng=nAacuXgPknOf(ig7((e-VuZEt`3+b2vM zPnGba*&i`2X(Caq@NM=LqikjTa8y%U_3n!bzr8Ek-BQusm4E)(%ZvDqr>3W;r?GL^u*SwS&pbVj4%1at8-CDP3`P|vzPj>-rNQ!*Mc(h4GpDe4 zaX5YgGkk`>y#IBD%mQ&xiC94XE&VU>jSp$X2JwvFR9rG*{yhU07Addr`d4kPI7FOn z=1G!5P2yJ~i6JVqOcrvM(aE>2k@G~b++BXbKiS&=l7eBBPO@!;o-^q3tqNJv6Vc<7eCxC>48{7Ud2cD?8It=y~5t zemjMQ^m%xW_L!%ZSeQd^;m`~fXoqN#2xT^!G8V-T^ph;QLqy$KnkpWFVF?gg1T-Uu zJs$#C&2N%9#V?b@nyRSj4hADt8YU<8twf11e%yp4!4L9HW92shk}nD+JC^S(xqM}| zWS5Ufj#K%Rp%w$9e0!5GbY=3r<`h50O-CiC@-2_bhy%WD&!BLqo<=DYw74mkZw{aW zsr+*M5~PBXG$=n-yjT9-_!kL=A%-Lb252z(mZp!F_RqcWFZEM?5TdG$P0?odO)F02 ziysx^znAU9KgnzOmu3oh{y5v-{8OT^oZ)x>Dd)#(`EQC}fT&-PFZJO^1|`Yb2X;7&<9GVRXwgcZ z`lA~$UdY72RTJW^Ev=vW?5E#w-uZ96@Zavd^Y$JZYK3`NIc#KmW!RMox3p~Q>FTVm z8Nx#fZTXfRtvkn8XSx_-4XxIj-^y%2xVtXZ{>31_1hERd71d7 zSy3=Z3#6LSlKARpUv%8LbLYK(zLy0`h$94}%k*heWk~x^<)*t(q@KVT9myYJ`G(#L zP^s7~w=W46qjriaIW1RrW(SGi-`#T$m)Obx0ReZqMvfXYT1VnYqJ_p(EBBxHlu*U| zD0>-x)ypBp)rU$yigPHU6d{(zr@oblzUA^Ys<81Ba_nJtve8P5AHk6jN2OXMB=(i*4%va&7b}3XI#+NzxMT}UAr0C)9E)1 z8FDdifa`^q9u2jn3C#@D(EkQI)>5JBqUe$FZzVv0LJVc6alw6*9#e6k6q z$zL-!+Obith3j(#RaF<9_^zO3E2|x#qrh^COCQEx4Z}m8?0^R8E~#xj-K+OBcjgOY zhSW`I;M!U`vWY6?i1CBdFpwZzZaK@?+rNt+7306z7aF5r;f~GMEL`}^_N{v0m+2OR zT#DM4?QVKt{f29vd*+=7&OB@1X$K4+>72VtzVGs>dQD6718Z0P?1d%mT>Ba}^o_sV7bU)Q zzI4~FyVk6H)u=JwIq}p67OH1Hz|}SU&JVRPH0~_sXRTlT`&Fx+*|ybLS1n<zIt04KJ3Ktt`wuC^L3n zz{Z;yk(}XEPY1ua4H~(%jErF9i0FXv;fxW{(8}mn^Mh^O5|OP4rX3LGvlZ33QmuXr zN{P}*n=Y+^q?$dpEdULiZ#)N{`-RFJ;9tg#kagQtj6~>BcLW53s{>MH8o|P zeJ)tFE^67M+4zwo!vppU4mmhtb$+vm7%|;ZM0=(q0;B=gF9jZT@6u#cO|7DzJp328 z~<7D855+m9=X5 z)LCWFu^#2HBuOH3j7Hul@|9ql+8=RiehBn6YYNXVD)B~%F6c7Z6JC{>I@JU!H)V^z z{uNJl1f^~s--nCLe*0Emdu`^h!!y$xJV``z!N ziIXZ=+(!iUx&F}6KJ(elTeb*x z5Kh}~`bF=)h?x#G$`(haJWQ+9%!6ir@4Hvg3fAygkqeR8<5g)akCFvh^_c%qtS>)u z)4ICKH=JJz?kTQcmuYP)JUA=IPc%?6_Dxj=6;_bqk}vfwS*K)<7(cr%KQnQk;G|Pa z^XC;;twihMoY|TE56B*Ta5;X{IJK0{cohTX55%9iLGSq1j7~;*3}_@=vSaxIoXF=y zIeuws_(^n^vQO>%&`Kr!9JRV(V8b0tBNvw^0~*QnlTe%{hv93EKgtDLwd`_ErmJP@RXK=|WD7s^`WqIGOEixU)> zR3U;j>1gc&rD>wkOu$n7?DXS@nB=@wCx-iOB1tW)o60^(Ar+b8NBx!!)J3IcSp5lVU;r{WFzM#cm@ZFe3`%e`)-ha)O6g+qCQWsf&6YKc`~<(cFK`4~ceK z_#G#E22jb03t6Ohvs!xbyJY_y$d4IA07Sk1w0ZmXzws+hjafWPeJ-$-=0#0z*hSY@nPY#}45$ZZcZ40o8S}x2eOJDN4 z-~RTfqmMfL@T2E1c$#x~W7V+X+%|3QF3x<-IbXWr!EHM?=6TLz#K?(9y=o60Tp6^p zn3=af)S&RS8QV#CH4$wqrYuu$aWlTPI;ZWWXEGiiTJ2$?lTH@FZ1IdGJ+Bqc;xdHiw< zLc=L$^R7E?zk{CH=xL93Z3?!o4LX|PNMkF8Uz27=!zTo* z=VB}&HANWKWipjf?I^8yYW$-M%Cz?6ceZ!1fbf{Ay0Q9=4FD$Xzc`6Epp(vLieEa4 z3^^oU1N+~NEiHdqxBB{L=R0W)xvHbbjGfXjv@V-#$rqOH+OcNOo?SiNKU=zFb4S+) z4>@$wkOr1?bN@HVl8@|Xwr%;%iskpMST@4xUh$**JV^1MmCP zbQGgSyLK$zv7J`u@r@haxnsuyl1rtiw0?KRp0=nmEYN^Xm>3Ql z#k(V6XQpD_%HrtyXvDCvp;iH;2F2`~h?VVP`KsRD3gnofC8GsBHvt463;>7!FvM9H z(nyc-JrI?hBq=M0U=xS7QNmMsxqalw%(!tuYinug;^^oT!l4bAM)D7jnq`>wgyX^km*}R(Qn0 z@(T42(pTQYAv^5;c2;KobK%e-(Ob_8rcM-oI|mS22+gA!1|gxaP6?zMLonwztPeN6 z7-kBYQZ{?o;n9RiK`tjC(+EsYyFClXcVUGJ3*?lM1{QQ^Uq)D0TYE^9QgX~0m zzkY4k)z_AOduOSmi-)Z8SA3)DOJ9tpP9+Z-sbYaJi_#^G9g{uoUn(}-+08ZP?p+bf zcIj2bDVhSEUt)KAM(Pd`2T(MRQqK`?RBqhkR{uE>mOo?xu?k*@+y zrqyW%$};5P$ZFIL52Tt4@_T?K=72%U(lZg$d|6mx27_O4%CbZ<@9iiR8+rpKvDtvkNL;H$VH3;@;_x^HrY@l9> zA22&z23WFOH~6s~JD>1xTA?WaKMk_V&| zKT!JFmztHws>EL%VTwP2rsSA4+jIF;++V)v69)@kdM$ZegxLHbVo_vlse{e{s-_Q@q^(v zMf$=vMI|7SPl7nHPpSGpEq?tn%WNtN2gPsZGaDGcu59AZefSY8qPjmnk}UD%^1YxX znQT#OOd^4tseOUu{$s@@M4M7^Mb%{)Vkfg?p%^57NuIH9uzE|-%)*L@lmzV*eN;h0 z#7~*<=x1GRU2A(Aj|yfQhqdrKEIh;!2E+G1WavQ$cV~lc9u{ShQ^w%+!P>cx=DWMA z>g!ltn{~6nf?SOmHN`G%=*qd?%@p&?f?!s$=fh`d<@flA^r*?gEA0|Ni&Sy8nJ~qBe?L z{`If^{O3RaPoA&l6$M>HWM&?8&?T2#;uPln83#4;Bp7`LPy2@7G?AvM|Fdtl zZ(#q6a(gzj)*>hQrwcExY^rL4YP`;U$|x|E9glKKs5p`Q?yhLZ8tlPsAk7EqAB)rs z35Jc;g;G^5uf&)9SJ=>?t3BAWH5$o%#=s;cX9{FH(Nlgo&)0;LGgJ}PTVP8DXwrAAH0JTW1mJY`Z!;J~!u)$G2_V)X}zJ$F>7UjvQKPzXq+M zar|y*Yk6?}y4jmv+}+iE#OTogGaDL1qoxT&-YrLHdrZa=Esl@E6}%MNyt8zZ5*xpSihe=VJ~e=v1i zP^0+)S(TNLa2RF{R_Kt3O$6FF6c)=PLOu&@zKgvQCmm!{G#BDv2pxTy1g~?<6DBwn~5!5|JBq) z!-nx%T&8&s^G~JL_VAatN85J0V?Yd}I8T`|V0IKm42U2gL6Fd7geG)1oim-P zySgjqx^?e)|8MPcZ`}&rz&Orxeveb#_ny1Mx7YgC+TrZ8&pzMyk-u?gDvvD87^KNR8&|>^G~{v z`|s;LcqmMD`$0T7Zgk#N9||T<0m#PB@*wkPbx=BZ`f*7o_p^0mn@9E`+}U2 zBUPAWkiAS0FE+e7T(r=?>o?tcuz=+~`@(~Vyy2BfV?zp3lIg&K{rB8;H}WDlcht{5 z>&&aJx+*U(*SRhUOeLeWKfZ88p{cI9WY<369paTa6n3-jXm2kmDIGOx6fdkaej-}P zD}D;{{X%wousL@VI$M4{v4Gg3(Z&xDO|;P~vm2aKA*+4!cr~s!TrZVlIe|nsLckDx z_B+FmqD*T8OBXM7#AF7aWZ;{Ug3FlI5{Y!!jY2d0m~13IR{v|5L&&d{9cf<0zRI!R z_8qB8Gh|c9l)>RrS<2`+F2eXDpR8yK;7t6a{($B1`F43tH|mupT@|9ge)vv#a;g3+ zKZjZtJ%PgND^*sF+jsa z+ogq1y_C&xlv{XgN2+?IWs6iqrsOqs!WT8u@TsP}EJ{zsP?(;BK>$P|r6f$-Pm)Gz zqJ$bmwa!+rO-mV0mByu$-lm`9+cFw&))r*=x127AKqCOvIuQZL+KI3_n)1Qd!FX*7_^qP)GttM$qs6@;iPAQ%Y)DWPAep_%Q;-yzvv=EQx+v zTo(R7{~?}()&P8t0sNbUvXP5@tNucT^M6J?YDW5w|1tGfZ66J{miJkAV=AcPaLCW3 zBIb5AMRwF^wux7 zDGhpJA@88=VU9e%cgKSdzw)ym#`E)eSaRf;QM4^P^+sphTa>Er;ijV=#FThF$V?fB}onM^Pl^?D%o@d6kv^CtYgipU;XMY;2a9lI_>10ci#E?yMC|Mn_{&Q0r}l1Q@A)5*sg#1 z%%?wn-00D&&L$$lQ<7Q=KGDp0=Zx>D_h$h=LMTq;D3McKIdo`p#@YwvQSl@rp-@GE zOB7{Qb~^=4;$bF9su6@bR{cxo5LN`B6cTu-{+Ae660kiy$YA)nxw)VJ!WS;T{BkUV zQ8sPb_^D5QYQu*0!e-zSc%S;rr`bXs(I|q~^a$Pffk~C()FO^hNx|`-?SB)UUTcG% z)umLq593Cj|1I+JN@;S%LZo9v@eutlHw=&L*3D8=jZMt-?0RuopljwhFIAD};o7=Z zN_pMwUSn-gF~&>fg{_Ce4Ue+JfhI{Pg*)RD7kHDFN$Un~&uZk2vXYHQhja4%{8F() z4Y8dX6Uk&3Yiz@i*YD+XA4_9)#5D#Z8>L0pPjTi1boS+^Dd${31cIzDCGu;EimsY9 zhcxF*oIJXi9c^TYjD7QB!JMj^Z=G`5wNE|z%E5zc>W}pFrcybQEz|!p(eR56>vz;Q zmgVLx8#U^yC!XRKD1f<&qD55kZx*Lxi9-fgw3l)VT3*-7@ZDn`b-xo>X%E;lnq-x`rpX z&z>;h!fCVSR%s^1amW`ZBa2IJI_2%RZmj